▶ ⚡ Lazy mode the whole guide as 5 copy-paste prompts — click to expand

Don't want to read? Fair. The only thing you do by hand is (1) install Hermes (here's the full getting-started guide) and (2) create a long-lived token in HA (Profile → Security → Long-Lived Access Tokens → name it hermes-agent). Then paste these prompts into Hermes, in order, and it does the rest:

1

"Add this to ~/.hermes/.env: HASS_URL=<your HA URL> and HASS_TOKEN=<the token I just made>. Then verify you can reach HA: pull /api/config and tell me the version and location name."

2

"Install the hab CLI (balloob/home-assistant-build-cli) as a prebuilt binary into ~/.local/bin, log it in with the same URL and token, and confirm 'hab automation list' works."

3

"Audit my Home Assistant: pull /api/states, classify every unavailable/unknown entity into harmless helpers vs integration-level dead vs genuinely broken, map automations to rooms, and report the top 10 fixes. Change nothing."

4

"Create a home-assistant skill. Record: the HA URL and token location, how to reach the host over SSH, which integrations own which domains, my entity naming conventions, and every pitfall you hit during the audit. Add these rules: backup or snapshot before any write, preview every change with --plan before applying, never guess entity IDs — verify against live state first, reload instead of restart, disable instead of delete. Update the skill whenever you learn something new."

5

"Fix the top item from your audit, following those rules. Show me the plan first."

That's it — you're at the same place as the full guide. Everything after this is just more prompts; the full prompt library below covers new hardware, dashboards, log triage and more. Come back and read the safety sections later when the agent earns your trust.

What "autonomous" means here: I message Hermes on Telegram — "the average temperature sensor looks wrong" — and it pulls the live state, finds a renamed entity breaking a template, shows me the one-line fix, applies it, reloads, and confirms the value recovered. No YAML editor. No SSH. No me.

Everything below is how that's wired: the agent host, the control channel into HA, the skills that teach it my house, the discipline that keeps it from breaking anything, and a copy-paste prompt library at the end.

0.The architecture at a glance

Piece	Role	Where it runs
Hermes agent	The AI. Talks to me on Telegram, reasons, runs commands.	My main Linux box (any always-on machine works)
hab CLI	The write-path into HA. Balloob's Go CLI built for LLM-driven HA management.	On the Hermes host (~/.local/bin/hab)
HA REST/WebSocket API	Read state, call services, reload, deploy dashboards.	Home Assistant itself (:8123)
Long-lived token	The one credential that unlocks all of it.	Created in HA, stored in ~/.hermes/.env
Skills	Mine. Markdown files that teach Hermes my house, my pitfalls, my conventions.	~/.hermes/skills/
SSH (optional)	Config-file surgery HA's API can't do (.storage edits, log forensics).	Key-only SSH to the HA host

1.Prerequisites

• A working Home Assistant instance (OS, Container, or Supervised — all fine).
• Hermes agent installed and reachable — mine fronts Telegram so I can drive it from my phone (see Getting started with Hermes if you haven't set it up yet). Any agent with a shell tool works in principle; this guide documents Hermes because that's what I run.
• Network reachability from the Hermes host to HA on port 8123 (LAN or via your reverse proxy).
• 20 minutes.

2.Create the HA long-lived access token

In Home Assistant: click your profile (bottom-left) → Security → Long-Lived Access Tokens → Create Token. Name it something obvious like hermes-agent. Copy it — HA shows it once.

3.Wire the token into Hermes

On the Hermes host, drop the token and URL into the agent's environment file:

# ~/.hermes/.env
HASS_URL=https://ha.example.com        # or http://192.168.x.x:8123
HASS_TOKEN=eyJhbGciOi...               # the token from step 2

Setting HASS_TOKEN auto-activates Hermes' built-in Home Assistant toolset (ha_list_entities, ha_get_state, ha_list_services, ha_call_service) after a gateway restart. But honestly? Those four tools are the least important part of this setup. The real power comes next.

4.Install hab — the write path

HA's REST API can read everything and call services, but it can't create automations, disable entities, manage scripts, or preview changes. hab (by Paulus Schoutsen, HA's founder) closes that gap — it's a CLI explicitly built so an LLM can manage HA safely.

# single prebuilt binary, no Go toolchain needed
curl -sL https://github.com/balloob/home-assistant-build-cli/releases/latest/download/hab-linux-amd64 \
  -o ~/.local/bin/hab && chmod +x ~/.local/bin/hab

# auth (same token)
hab auth login --url https://ha.example.com --access-token eyJhbGciOi...

Why hab matters for an agent:

Capability	REST API	hab CLI
Read states / call services	✓	✓
Create/update automations & scripts	✗	✓
Disable entities (registry writes)	✗ (404s)	✓
Preview a change before applying (--plan)	✗	✓
Trigger a config backup before writes	✗	✓ (hab backup create)
JSON output an LLM can parse reliably	~	✓

The --plan flag is the single most important feature: the agent shows a diff of what it's about to do, then applies it. That's the difference between "AI changed something in my house" and "AI showed me the change, I said yes."

5.Give it SSH for the 10% the API can't do

Some operations have no API at all: editing .storage JSON (e.g. repointing a generic_thermostat's heater), forensic log analysis, or fixing a YAML file that broke the config check. For those, Hermes has a dedicated key-only SSH account on the HA host. Guardrails that make this safe:

• Read-only by default. The agent reads configuration.yaml, .storage, and logs freely — but writes go through me unless I've pre-approved the specific operation.
• Snapshot before touching .storage: cp -a /config/.storage /config/.storage.bak-$(date +%s). One bad write to .storage can soft-brick an instance; with a snapshot it's a 30-second rollback.
• Never recreate the container blind. A docker run with the wrong /config bind mount makes HA look "factory reset" while your data sits safe on another path. The agent's first command in any recovery is docker inspect to find the real mount, cross-checked by entity counts.

6.The discipline that makes it safe

Autonomy without rails is how you get a house that reboots itself at dinner time. These are the rules my agent operates under — they're encoded in its skill files so they survive across sessions:

1. Backup → plan → apply. Before any write: hab backup create (or a .storage snapshot for file edits), then --plan preview shown to me, then apply. Skipping the preview is what causes "WTF dude" moments.
2. Never guess entity IDs. Every automation/card/script reference is verified against a live /api/states pull first. A plausible-but-wrong ID (input_boolean.nattlage when the real one is switch.nattlage) either silently drops the automation or makes Spook fire ghosts. Batch-verify the whole list, print MISSING vs OK, fix, then deploy.
3. Reload, don't restart. automation.reload / template reload over WebSocket is enough for most fixes. The house stays up.
4. Disable, don't delete. Dead entities and duplicate automations get disabled — reversible. Deletes only when proven orphaned.
5. Check config_entries before removing a "dead" device. If it shares a live integration entry, removing it kills the whole integration. (Near-miss with my MQTT entry.)
6. Show, don't tell, for anything visual. Dashboard changes get a headless-browser screenshot sent to me on Telegram before and after — reasoning about layout from DOM geometry instead of pixels is how dashboards end up ugly.
7. Scope discipline. Do only what was asked. No unrequested sudo, no "while I'm in here" refactors.

7.Skills — teaching the agent your house

An agent fresh out of the box knows Home Assistant in general. It does not know that my HA runs in Swedish, that night mode is switch.nattlage not input_boolean.nattlage, that Magic Areas owns the lights so manual light automations are superseded, or that the Dreame vacuum exposes room cleaning as dreame_vacuum.vacuum_clean_segment instead of the core service.

That knowledge lives in skills — markdown files under ~/.hermes/skills/ that load automatically when the task matches. Mine has a home-assistant skill that grew organically: every time the agent hit a pitfall on my instance, we wrote it down. Structure that works:

~/.hermes/skills/smart-home/home-assistant/
├── SKILL.md                          # the audit method, the safety rules, the pitfalls
├── references/
│   ├── hab-cli.md                  # full command map + JSON gotchas + safe-write flow
│   ├── ha_break_fix.md             # recovery playbooks (wrong mount, notify renames...)
│   ├── ha_vacuum_automation.md     # the Dreame room-cleaning pattern that works here
│   ├── ha_lovelace_dashboards.md   # how to build+deploy+screenshot dashboards
└── scripts/
    └── ha_reaudit.py               # before/after delta audit, proves a fix landed

The content isn't generic documentation — it's decisions: "disable, don't delete", "this instance needs plural triggers:/actions: keys in hand-written YAML", "notify services are notify.mobile_app_<device_slug> — confirm the live name before bulk-renaming". Every line in that file is a scar from something that once went wrong.

8.How work actually flows (a real session)

Yesterday, verbatim, compressed:

What the agent did, unprompted beyond that:

1. SSH'd to the HA host, read the report: 8 missing actions, 49 missing entities, mapped to their source files.
2. Cross-checked every candidate against the live API instead of trusting the report — found my phone had been re-added as roberths_s22_* (the dashboard still referenced the old S10 names), the Dreame vacuum's room-clean service had been renamed by an integration update, and one template sensor in sensors.yaml referenced an entity renamed months ago.
3. Verified the impact live: the "average temperature" sensor was silently stuck at a cached 12.2°C while the real average was 23.8°C — because the template failed and HA kept serving the stale value. Cooling automations were making decisions on garbage.
4. Reported back with a triage: one-line fix (do now?), dashboard reference renames (batch, show before save), dead-device cards (needs my call).

Total my input: one message, then "yes". That's the loop you're building toward.

9.The prompt library

These are the prompts I actually run, in the order you'd use them after setup. The first three bootstrap the agent's knowledge of your house; the rest are standing operations.

Bootstrap (run once)

Daily operations

10.Getting data in: MQTT & BLE (supporting act)

The agent configures HA — but it's also the fastest way to get new data sources into HA. Two patterns from my setup:

• MQTT as the universal ingest: a Mosquitto broker on the Docker host, HA's MQTT integration pointed at it. Anything that can publish to a topic becomes an HA entity via MQTT discovery.
• BLE presence without ESP32s: a Raspberry Pi 3B running a tiny Python (bleak) scanner that publishes selected BLE MACs to MQTT; HA mqtt_room sensors turn those into device_tracker entities. My phone and earbuds track room-presence this way — no beacons to buy. The agent wrote, deployed, and systemd-hardened the whole thing from one prompt.

Why mention it in an agent guide? Because the value compounds: the agent that knows your automations also knows your sensors. Ask it "why didn't the kitchen clean trigger?" and it can trace person detection → BLE tracker → MQTT → automation condition in one pass.

11.Optional: a watchdog inside HA

The agent audits on demand; Watchman (HACS) audits continuously. It scans every config file and dashboard for references to entities and services that don't exist and writes a report. The pairing is natural: Watchman detects, Hermes triages and fixes. My actual prompt from yesterday is in section 8 — point the agent at the report path and let it verify each finding against live state before touching anything (Watchman flags stale dashboard references too, and those aren't always worth churn).

12.What this costs & honest limits

• Cost: hab, the APIs, and skills are free. The agent's LLM usage is the only meter — routine HA ops are cheap on a mid-tier model; reserve the expensive models for multi-file forensics.
• It will still surprise you. Rails reduce blast radius; they don't make it zero. The --plan preview is non-negotiable.
• Some things stay manual: pairing new hardware, anything requiring the HA app, and initial integration auth flows (OAuth redirects). The agent handles everything after the pairing dance.
• The skill file is the moat. Two people can run this same guide; the one whose agent has six months of house-specific scars in its skills gets a categorically better operator. Start the file today.

Reference links

• Getting started with Hermes — install, provider, gateway, first week
• Optimizing Hermes — model routing, memory hygiene, skill curation
• Hermes Agent docs — install, toolsets, skills system
• hab — home-assistant-build-cli — the write-path CLI
• HA REST API · HA WebSocket API
• Watchman — missing-entity watchdog (HACS)
• HA MQTT integration · mqtt_room presence
• bleak — cross-platform BLE library used by the Pi scanner

Runs in production on my homelab: 1,700+ entities, Magic Areas, Dreame vacuum fleet automation, BLE presence, per-room climate — all agent-managed. Questions or your own war stories? Find me via the main site.