[ci] Put the per-board skill templates under their package directory and tidy the drivers

.agents/skills/hardware-test/SKILL.md

@@ -115,7 +115,7 @@ Per-topic scripts, each described in the matching `reference/recipes/<component>
 (host driver, Zephyr driver probe), `rtsp_server/` (protocol client, raw rate reader, measurement set,
 API image grab), `cse7766/` (API observer with ping), `zephyr_coredump/` (reader flow, decode with the
 Zephyr tools), `libretiny_prefs/` (rtc preference harness header), `libretiny_crash/` (BK72xx crash probe header and
-capture decoder), `api_outgoing/` (wake cycle and Wi-Fi drop timing readers), `ld2450/` (zone number driver, loopback harness driver with number and binary sensor states and TX frame summarizer), `api/` (raw plaintext native API client for user defined actions), `dfrobot_sen0395/` (API driver and case runner), `homeassistant_light/` (direct bulb driver, held-report case driver), `usb_audio/` (URL player), `speaker_media_player/` (API player driver, pipeline task case runner), `web_server/` (SSE client, request mix, SSE and HTTPD case runners), `mqtt/` (broker config with TLS, client-certificate and WebSocket listeners, test CA with server and client certificates, transport case runner, flash-and-drive runner, CLI TLS check without a board), `udp/` (datagram recorder, `udp.write` payload checker and case driver), `entity_icons/` (icon and device class readback over the API, MQTT discovery, web_server and dump_config, with a comparer).
+capture decoder), `api_outgoing/` (wake cycle and Wi-Fi drop timing readers), `ld2450/` (zone number driver, loopback harness driver with number and binary sensor states and TX frame summarizer), `api/` (raw plaintext native API client for user defined actions, store_yaml recovery client over Noise or plaintext), `http_request/` (update manifest server, update state driver), `modem/` (button and sensor driver over the plaintext API, also used for the BLE MIDI boards), `dfrobot_sen0395/` (API driver and case runner), `homeassistant_light/` (direct bulb driver, held-report case driver), `usb_audio/` (URL player), `speaker_media_player/` (API player driver, pipeline task case runner), `web_server/` (SSE client, request mix, SSE and HTTPD case runners), `mqtt/` (broker config with TLS, client-certificate and WebSocket listeners, test CA with server and client certificates, transport case runner, flash-and-drive runner, CLI TLS check without a board), `udp/` (datagram recorder, `udp.write` payload checker and case driver), `entity_icons/` (icon and device class readback over the API, MQTT discovery, web_server and dump_config, with a comparer).
 
 ## Templates in `$SKILL/templates`
 
@@ -132,7 +132,10 @@ capture decoder), `api_outgoing/` (wake cycle and Wi-Fi drop timing readers), `l
   `globals-storage-dut.yaml`, `usb-audio-waveshare-s3-eth-dut.yaml`, `usb-audio-sendspin-s3-eth-dut.yaml`,
   `cast-pro-1-sendspin-control.yaml`, `aux-2-preprod-stock-dut.yaml`, `aux-2-preprod-usb-audio-dut.yaml`,
   `dfrobot-sen0395-dut.yaml`, `aux-2-preprod-speaker-player-dut.yaml`, `web-server-sse-dut.yaml`,
-  `mqtt-transport-dut.yaml`, `mqtt-tls-dut.yaml`, `udp-write-dut.yaml`, `entity-icon-dut.yaml`.
+  `mqtt-transport-dut.yaml`, `mqtt-tls-dut.yaml`, `udp-write-dut.yaml`, `entity-icon-dut.yaml`,
+  and the directories `store_yaml/` (package with secrets one directory up, per-board files),
+  `http_request_update/` (manifest URL setter shapes, per-board files), `modem/` (modem only and modem plus
+  Wi-Fi boot checks) and `ble_midi/` (fake BLE MIDI instrument and the client).
 
 ## When something looks wrong
 

.agents/skills/hardware-test/reference/recipes/README.md

@@ -4,14 +4,17 @@ Each flow names the pull requests it was used on, so their result tables show wh
 
 One file per component, named after it:
 
-- [`api.md`](api.md): noise handshake and OTA encryption on the ESP8266, outgoing connection wake timing on deep sleep and after a Wi-Fi drop, user defined action listing and execution with a raw plaintext client
+- [`api.md`](api.md): noise handshake and OTA encryption on the ESP8266, outgoing connection wake timing on deep sleep and after a Wi-Fi drop, user defined action listing and execution with a raw plaintext client, store_yaml recovery over Noise or plaintext, user defined action listing and execution with a raw plaintext client
+- [`ble_midi.md`](ble_midi.md): BLE MIDI client against a fake ESP32 instrument, every packet shape and the send actions
 - [`captive_portal.md`](captive_portal.md): fallback AP and /wifisave hammer
 - [`cse7766.md`](cse7766.md): Sonoff S31 parser check over the API
 - [`espnow.md`](espnow.md): ESP8266 broadcast receive
 - [`hdmi_cec.md`](hdmi_cec.md): two-board wire
+- [`http_request.md`](http_request.md): update manifest URL changed at run time, every setter shape, a swap under a running check
 - [`ld2450.md`](ld2450.md): polygon zones, zone number round trip through the radar's readback, ESP8266 loopback harness without a radar
 - [`ld2412.md`](ld2412.md): output pin level through the basic configuration command, held back writes
 - [`libretiny.md`](libretiny.md): rtc preference emulation, BK72xx crash report on the next boot
+- [`modem.md`](modem.md): PPPoS modem component boot check with no modem attached, loop blocking and disable timing
 - [`mqtt.md`](mqtt.md): broker bench, QoS, reconnect, crash log over MQTT, WebSocket transports with a local and a public broker, TLS trust sources (own CA, client certificate, certificate bundle, no verification) with the CLI check
 - [`nrf52.md`](nrf52.md): dongle board support
 - [`online_image.md`](online_image.md): stalled and malformed downloads

.agents/skills/hardware-test/reference/recipes/api.md

@@ -40,3 +40,19 @@ length delimited strings and `int[]` as repeated sint32 (field 7); `ALLOW_MISSIN
 defaults) and `templates/api-actions-strings-dut.yaml` (required strings) log what arrived. String arguments are
 `StringRef`s that are not null terminated: copy them with `snprintf("%.*s", size, c_str())` into a buffer before
 logging, and do not pass `%.*s` to `logger.log`, whose validator counts it as one pattern.
+
+## store_yaml recovery over the native API (#16445)
+
+`scripts/api/syrecover.py HOST KEY OUTDIR [ORIGDIR]` (KEY `-` for plaintext) builds the client library's
+`APINoiseFrameHelper` around a stub `APIConnection` subclass (the compiled helper type checks the connection, and
+`ConnectionParams` needs every field), sends Hello, Authentication (never answered without a password) and
+`DeviceCapabilitiesRequest`, then `GetYamlRequest` (161), collects `GetYamlResponse` (162) frames by field number
+(1 data, 2 done, 3 total_size, 4 encoding) until done, requests again, decompresses with `compression.zstd` and
+unpacks the envelope with the tree's `unpack_envelope` (run with `PYTHONPATH` on that tree). Configs in
+`templates/store_yaml/`: entry files in `boards/`, a package one directory above them with `!secret` and an inline
+OTA password,
+so the redaction and the `..` paths are exercised. Repetitive YAML compresses into one chunk even at 22 KB; a 6 KB
+include of random base64 lines with `include_secrets: true` gives four chunks (1390 byte packets). Rough edge at
+0bd3738430: `include_secrets: true` fails the build when a package in another directory uses `!secret` and has no
+`secrets.yaml` beside it; a copy next to the package works around it.
+Results: https://github.com/esphome/esphome/pull/16445#issuecomment-6335435397

.agents/skills/hardware-test/reference/recipes/ble_midi.md

@@ -0,0 +1,19 @@
+# ble_midi
+
+## BLE MIDI client against a fake instrument (#18508)
+
+No MIDI hardware needed: `templates/ble_midi/midiperi.yaml` turns a second ESP32 (an S3 with ethernet, so the
+API stays reachable while BLE is busy) into a BLE MIDI peripheral with `esp32_ble_server`: the MIDI service
+advertised, the MIDI characteristic with read, write, write without response and notify, an `on_write` that logs
+the bytes, buttons that notify hand built packets (channel messages, two messages in one packet, running status,
+SysEx in one packet, split over two, oversize, cut short, clock, aftertouch, orphan data bytes, a 50 packet burst)
+and a restart button for the disconnect test. `midicli.yaml` is the component test config on a classic devkit with
+`wifi:`, `api:`, the peripheral's BLE address in `PERI_MAC` (the base MAC plus 2, printed by `esp32_ble` in
+`dump_config`), `on_sysex` and `on_message` lambdas that print the bytes, debug sensors and out of range send
+buttons. Drive both over the plaintext API with `scripts/modem/mdrive.py HOST LOGFILE press:NAME wait:S` 3 s
+apart and compare the peripheral's `TX` and `RX` lines with the client's decoded lines. Gotcha:
+`ble_server.characteristic.set_value` latches the value of its first play and re-applies it before every notify,
+so the peripheral script sets the value from a lambda (`id(midi_io).set_value(std::vector<uint8_t>(pkt));
+id(midi_io).notify();`). At VERBOSE the client logs four lines per message, so a burst shows an
+`esp32_ble took a long time` of a couple of seconds that is the serial port, not the parser.
+Results: https://github.com/esphome/esphome/pull/18508#issuecomment-6348272717

.agents/skills/hardware-test/reference/recipes/http_request.md

@@ -0,0 +1,17 @@
+# http_request
+
+## Update manifest URL changed at run time (#21373)
+
+`scripts/http_request/manifest_server.py PORT [DELAY]` serves `stable.json`, `beta.json` and `slow.json` (answered
+after DELAY s) with both chip families and logs every request. `templates/http_request_update/upd-common.yaml`
+(secret `manifest_base` is the server URL) has the update entity on the stable manifest, buttons for every
+`set_source_url` shape (`std::string` local, literal, temporary, `.c_str()` of a local, the current URL back in),
+`Check Then Swap` (ESP32 task path: check the slow manifest and replace the URL in the same lambda), `Arm Swap In
+Check` plus a `logger: on_message` lambda that replaces the URL from inside the inline check (ESP8266; the log line
+from inside `on_message` is swallowed, judge by the next check's result), and a `Heap` button. `upd-base-common.yaml`
+keeps only the shapes that compile on a tree with the bare pointer setter; `boards/upd8266.yaml` and `boards/upd32.yaml` are the
+per-board files, one directory below the package their `../` include points at. Drive with `scripts/http_request/updrive.py HOST KEY LOGFILE press:NAME wait:S`, which prints every
+update state (latest version, summary). The ESP8266 Arduino client gets `Error reading manifest: -1` on roughly a
+quarter of the fetches from this Python server on every build, so count checks against the base before blaming a
+PR. `http_request` on the ESP8266 needs `verify_ssl: false` even for plain http URLs.
+Results: https://github.com/esphome/esphome/pull/21373#issuecomment-6337544933

.agents/skills/hardware-test/reference/recipes/modem.md

@@ -0,0 +1,18 @@
+# modem
+
+## PPPoS modem component with no modem attached (#6721)
+
+A boot check on a classic ESP32 devkit with the modem UART pins left open. `templates/modem/modem-common.yaml` is
+the package (`api:`, debug `loop_time` and `free` sensors, uptime, buttons for `modem.enable`, `modem.disable` and
+`modem.send_at`, a 20 s alive log); `modemo.yaml` is the modem only build with `reboot_timeout: 2min` and the
+component test config's `on_boot` block at `priority: -100` (earlier, `modem.send_at` runs before the modem's
+`setup()` and logs `Modem DCE not ready`); `modemw.yaml` adds `wifi:` with `reboot_timeout: 0s` so the API stays
+reachable. Expect `Modem not responding at 115200 baud` once per pass with each pass blocking the loop about 1 s,
+the reboot at the configured timeout (`SW_CPU_RESET`), and a `modem.disable` with no modem answering to block
+the loop about 37 s (`Failed to leave data mode`, then the radio off timeout) without tripping the 30 s watchdog
+floor because the component feeds it in between. Drive the Wi-Fi build with
+`scripts/modem/mdrive.py HOST LOGFILE list press:NAME wait:S` (plaintext API; prints sensor and text sensor
+states, saves the device log). Before 8cc8df4a1f a modem only config with `api:` failed validation because a
+config dependent `AUTO_LOAD` added `network` after the dependency check; an explicit `network:` block was the
+workaround. Results: https://github.com/esphome/esphome/pull/6721#issuecomment-6347395159 and
+https://github.com/esphome/esphome/pull/6721#issuecomment-6348520416

.agents/skills/hardware-test/scripts/api/syrecover.py

@@ -0,0 +1,190 @@
+"""Pull the store_yaml blob off a device over the native API (plaintext or Noise) and unpack it.
+
+usage: syrecover.py HOST KEY OUTDIR [ORIGDIR]   (KEY of - means a plaintext API)
+Prints the capability flag, chunk count, advertised and received sizes, encoding, the envelope's files, and
+when ORIGDIR is given compares each recovered file with ORIGDIR/<path>.
+Run with PYTHONPATH pointing at the esphome tree that has store_yaml (for unpack_envelope).
+"""
+
+import asyncio
+from pathlib import Path
+import sys
+import time
+
+from aioesphomeapi import api_pb2
+from aioesphomeapi._frame_helper.noise import APINoiseFrameHelper
+from aioesphomeapi._frame_helper.plain_text import APIPlaintextFrameHelper
+from aioesphomeapi.connection import APIConnection, ConnectionParams
+from aioesphomeapi.zeroconf import ZeroconfManager
+from compression import zstd
+
+from esphome.components.store_yaml import unpack_envelope
+
+HOST, KEY, OUT = sys.argv[1], sys.argv[2], Path(sys.argv[3])
+ORIG = Path(sys.argv[4]) if len(sys.argv) > 4 else None
+HELLO, HELLO_RESP, CONNECT, CONNECT_RESP = 1, 2, 3, 4
+CAPS_REQ, CAPS_RESP = 149, 150
+GET_YAML_REQ, GET_YAML_RESP = 161, 162
+
+
+def decode_varint(buf: bytes, pos: int) -> tuple[int, int]:
+    value = shift = 0
+    while True:
+        b = buf[pos]
+        pos += 1
+        value |= (b & 0x7F) << shift
+        if not b & 0x80:
+            return value, pos
+        shift += 7
+
+
+def fields(payload: bytes) -> dict[int, list]:
+    out: dict[int, list] = {}
+    pos = 0
+    while pos < len(payload):
+        key, pos = decode_varint(payload, pos)
+        fnum, wt = key >> 3, key & 7
+        if wt == 0:
+            v, pos = decode_varint(payload, pos)
+        elif wt == 2:
+            ln, pos = decode_varint(payload, pos)
+            v = payload[pos : pos + ln]
+            pos += ln
+        elif wt == 5:
+            v = payload[pos : pos + 4]
+            pos += 4
+        else:
+            raise ValueError(f"wire type {wt}")
+        out.setdefault(fnum, []).append(v)
+    return out
+
+
+async def main() -> None:
+    loop = asyncio.get_running_loop()
+    queue: asyncio.Queue[tuple[int, bytes]] = asyncio.Queue()
+    errors: list[Exception] = []
+
+    class Shim(APIConnection):
+        """Only the two callbacks the frame helper uses; the real connection logic is bypassed."""
+
+        def process_packet(self, msg_type_proto, data) -> None:
+            queue.put_nowait((msg_type_proto, bytes(data)))
+
+        def report_fatal_error(self, err: Exception) -> None:
+            errors.append(err)
+
+    params = ConnectionParams(
+        addresses=[HOST],
+        port=6053,
+        password=None,
+        client_info="syrecover",
+        keepalive=15,
+        zeroconf_manager=ZeroconfManager(),
+        noise_psk=None if KEY == "-" else KEY,
+        expected_name=None,
+        expected_mac=None,
+        timezone=None,
+        provide_time=True,
+    )
+    shim = Shim(params, None, False, HOST)
+    if KEY == "-":
+        helper = APIPlaintextFrameHelper(
+            connection=shim, client_info="syrecover", log_name=HOST
+        )
+    else:
+        helper = APINoiseFrameHelper(
+            connection=shim,
+            noise_psk=KEY,
+            expected_name=None,
+            expected_mac=None,
+            client_info="syrecover",
+            log_name=HOST,
+        )
+    await loop.create_connection(lambda: helper, HOST, 6053)
+    await asyncio.wait_for(helper.ready_future, 10)
+    print(f"transport {'plaintext' if KEY == '-' else 'noise'}: handshake done")
+
+    async def expect(msg_type: int, timeout: float = 10) -> bytes:
+        while True:
+            t, d = await asyncio.wait_for(queue.get(), timeout)
+            if t == msg_type:
+                return d
+
+    hello = api_pb2.HelloRequest(
+        client_info="syrecover", api_version_major=1, api_version_minor=10
+    )
+    helper.write_packets([(HELLO, hello.SerializeToString())], False)
+    await expect(HELLO_RESP)
+    # Sent like the client library does; a device without a password never answers it
+    helper.write_packets(
+        [(CONNECT, api_pb2.AuthenticationRequest().SerializeToString())], False
+    )
+    helper.write_packets(
+        [(CAPS_REQ, api_pb2.DeviceCapabilitiesRequest().SerializeToString())], False
+    )
+    caps = fields(await expect(CAPS_RESP))
+    supported = None
+    if 7 in caps:
+        supported = fields(caps[7][0]).get(1) == [1]
+    print(f"capabilities: store_yaml field present={7 in caps} supported={supported}")
+
+    t0 = time.monotonic()
+    helper.write_packets([(GET_YAML_REQ, b"")], False)
+    chunks: list[bytes] = []
+    total = encoding = None
+    done = False
+    while not done:
+        f = fields(await expect(GET_YAML_RESP, 20))
+        if 1 in f:
+            chunks.append(f[1][0])
+        done = f.get(2) == [1]
+        if 3 in f:
+            total = f[3][0]
+        if 4 in f:
+            encoding = f[4][0].decode()
+    dt = time.monotonic() - t0
+    compressed = b"".join(chunks)
+    print(f"chunks {len(chunks)} sizes {[len(c) for c in chunks]}")
+    print(
+        f"received {len(compressed)} bytes, advertised {total}, encoding {encoding!r}, {dt:.2f} s"
+    )
+    # a second request right after: the device must answer again, not stay stuck
+    helper.write_packets([(GET_YAML_REQ, b"")], False)
+    again = 0
+    done = False
+    while not done:
+        f = fields(await expect(GET_YAML_RESP, 20))
+        again += len(f.get(1, [b""])[0])
+        done = f.get(2) == [1]
+    print(f"second request: {again} bytes")
+    helper.close()
+    if errors:
+        print("transport errors:", errors)
+    envelope = zstd.decompress(compressed)
+    files = unpack_envelope(envelope)
+    print(
+        f"envelope {len(envelope)} bytes, {len(files)} files, ratio {len(compressed) / len(envelope):.2f}"
+    )
+    OUT.mkdir(parents=True, exist_ok=True)
+    for rel, content in files.items():
+        target = OUT / rel.replace("../", "up_/")
+        if not target.resolve().is_relative_to(OUT.resolve()):
+            print("skipping a path outside the output directory:", rel)
+            continue
+        target.parent.mkdir(parents=True, exist_ok=True)
+        target.write_bytes(content)
+        note = ""
+        if ORIG is not None:
+            orig = (ORIG / rel).resolve()
+            if orig.exists():
+                note = (
+                    "IDENTICAL to original"
+                    if orig.read_bytes() == content
+                    else f"DIFFERS from original ({orig.stat().st_size} bytes)"
+                )
+            else:
+                note = "no original"
+        print(f"  {rel}: {len(content)} bytes {note}")
+
+
+asyncio.run(main())

.agents/skills/hardware-test/scripts/http_request/manifest_server.py

@@ -0,0 +1,58 @@
+"""Serve update manifests: stable.json, beta.json and slow.json (answered after a delay); logs every request."""
+
+from http.server import BaseHTTPRequestHandler, ThreadingHTTPServer
+import json
+import sys
+import time
+
+PORT = int(sys.argv[1])
+DELAY = float(sys.argv[2]) if len(sys.argv) > 2 else 4.0
+
+
+def manifest(name: str, version: str) -> bytes:
+    builds = [
+        {
+            "chipFamily": fam,
+            "ota": {
+                "path": f"/{name}.bin",
+                "md5": "d41d8cd98f00b204e9800998ecf8427e",
+                "summary": f"{name} channel",
+            },
+        }
+        for fam in ("ESP8266", "ESP32")
+    ]
+    return json.dumps(
+        {"name": f"Bench {name}", "version": version, "builds": builds}
+    ).encode()
+
+
+DOCS = {
+    "/stable.json": manifest("stable", "1.0.0"),
+    "/beta.json": manifest("beta", "2.0.0-beta1"),
+    "/slow.json": manifest("slow", "3.0.0-slow"),
+}
+
+
+class Handler(BaseHTTPRequestHandler):
+    def do_GET(self) -> None:  # noqa: N802
+        body = DOCS.get(self.path)
+        print(
+            f"{time.strftime('%H:%M:%S')} {self.client_address[0]} GET {self.path} -> {404 if body is None else 200}",
+            flush=True,
+        )
+        if body is None:
+            self.send_error(404)
+            return
+        if self.path == "/slow.json":
+            time.sleep(DELAY)
+        self.send_response(200)
+        self.send_header("Content-Type", "application/json")
+        self.send_header("Content-Length", str(len(body)))
+        self.end_headers()
+        self.wfile.write(body)
+
+    def log_message(self, *args) -> None:  # quiet default log
+        return
+
+
+ThreadingHTTPServer(("0.0.0.0", PORT), Handler).serve_forever()

.agents/skills/hardware-test/scripts/http_request/updrive.py

@@ -0,0 +1,129 @@
+"""Drive the http_request update harness over the API: press buttons, print every update state and save the log.
+usage: updrive.py HOST KEY(- for plaintext) LOGFILE OP [OP ...]
+  OP: list | press:NAME | set:NAME=VALUE | sel:NAME=OPTION | wait:SECONDS
+"""
+
+import asyncio
+import pathlib
+import sys
+import time
+
+from aioesphomeapi import (
+    APIClient,
+    BinarySensorState,
+    ButtonInfo,
+    LogLevel,
+    NumberInfo,
+    NumberState,
+    SelectInfo,
+    SelectState,
+    SensorInfo,
+    SensorState,
+    TextSensorInfo,
+    TextSensorState,
+    UpdateState,
+)
+
+HOST, KEY, LOGFILE = sys.argv[1], sys.argv[2], sys.argv[3]
+OPS = sys.argv[4:]
+T0 = time.time()
+
+
+def stamp() -> str:
+    return f"{time.time() - T0:7.3f}"
+
+
+async def main() -> None:
+    with pathlib.Path(LOGFILE).open("w", encoding="utf-8") as log:
+        c = APIClient(HOST, 6053, None, noise_psk=None if KEY == "-" else KEY)
+        await c.connect(login=True)
+        info = await c.device_info()
+        print(
+            f"device {info.name} esphome {info.esphome_version} compiled {info.compilation_time}"
+        )
+        ents, _ = await c.list_entities_services()
+        by_key = {e.key: e for e in ents}
+        buttons = {e.name: e for e in ents if isinstance(e, ButtonInfo)}
+        selects = {e.name: e for e in ents if isinstance(e, SelectInfo)}
+
+        def on_state(s) -> None:
+            e = by_key.get(s.key)
+            if e is None:
+                return
+            if isinstance(s, TextSensorState):
+                print(
+                    f"{stamp()} TXT {e.name} = {s.state!r}{' (missing)' if s.missing_state else ''}",
+                    flush=True,
+                )
+            elif isinstance(s, UpdateState):
+                print(
+                    f"{stamp()} UPD {e.name} latest={s.latest_version!r} title={s.title!r} summary={s.release_summary!r} in_progress={s.in_progress}{' (missing)' if s.missing_state else ''}",
+                    flush=True,
+                )
+            elif isinstance(s, BinarySensorState):
+                print(
+                    f"{stamp()} BIN {e.name} = {s.state}{' (missing)' if s.missing_state else ''}",
+                    flush=True,
+                )
+            elif isinstance(s, NumberState):
+                print(
+                    f"{stamp()} NUM {e.name} = {s.state:.0f}{' (missing)' if s.missing_state else ''}",
+                    flush=True,
+                )
+            elif isinstance(s, SelectState):
+                print(
+                    f"{stamp()} SEL {e.name} = {s.state!r}{' (missing)' if s.missing_state else ''}",
+                    flush=True,
+                )
+            elif isinstance(s, SensorState) and isinstance(e, SensorInfo):
+                print(f"{stamp()} SEN {e.name} = {s.state:.0f}", flush=True)
+
+        def on_log(msg) -> None:
+            log.write(f"{stamp()} {msg.message.decode(errors='replace')}\n")
+
+        c.subscribe_states(on_state)
+        c.subscribe_logs(on_log, log_level=LogLevel.LOG_LEVEL_VERBOSE)
+        await asyncio.sleep(1.5)
+        for op in OPS:
+            if op == "list":
+                for n in sorted(buttons):
+                    print("button:", n)
+                for n in sorted(selects):
+                    print("select:", n, selects[n].options)
+                for e in ents:
+                    if isinstance(e, TextSensorInfo):
+                        print("text_sensor:", e.name)
+                continue
+            kind, _, rest = op.partition(":")
+            if kind == "wait":
+                await asyncio.sleep(float(rest))
+                continue
+            if kind == "press":
+                if rest not in buttons:
+                    sys.exit(f"no button named {rest!r}; have {sorted(buttons)}")
+                print(f"{stamp()} >>> press {rest}", flush=True)
+                log.write(f"{stamp()} >>> press {rest}\n")
+                c.button_command(buttons[rest].key)
+            elif kind == "set":
+                name, _, value = rest.partition("=")
+                numbers = {e.name: e for e in ents if isinstance(e, NumberInfo)}
+                if name not in numbers:
+                    sys.exit(f"no number named {name!r}")
+                print(f"{stamp()} >>> set {name} = {value}", flush=True)
+                log.write(f"{stamp()} >>> set {name} = {value}\n")
+                c.number_command(numbers[name].key, float(value))
+            elif kind == "sel":
+                name, _, value = rest.partition("=")
+                if name not in selects:
+                    sys.exit(f"no select named {name!r}; have {sorted(selects)}")
+                print(f"{stamp()} >>> select {name} = {value}", flush=True)
+                log.write(f"{stamp()} >>> select {name} = {value}\n")
+                c.select_command(selects[name].key, value)
+            else:
+                sys.exit(f"unknown op {op!r}")
+            await asyncio.sleep(2.0)
+        await asyncio.sleep(1)
+        await c.disconnect()
+
+
+asyncio.run(main())

.agents/skills/hardware-test/scripts/modem/mdrive.py

@@ -0,0 +1,86 @@
+"""Drive the modem test build over the API: press buttons, print sensor states, save the device log.
+usage: mdrive.py HOST LOGFILE OP [OP ...]
+  OP: list | press:NAME | wait:SECONDS
+"""
+
+import asyncio
+import pathlib
+import sys
+import time
+
+from aioesphomeapi import (
+    APIClient,
+    ButtonInfo,
+    LogLevel,
+    SensorInfo,
+    SensorState,
+    TextSensorInfo,
+    TextSensorState,
+)
+
+HOST, LOGFILE = sys.argv[1], sys.argv[2]
+OPS = sys.argv[3:]
+T0 = time.time()
+
+
+def stamp() -> str:
+    return f"{time.time() - T0:7.3f}"
+
+
+async def main() -> None:
+    with pathlib.Path(LOGFILE).open("w", encoding="utf-8") as log:
+        c = APIClient(HOST, 6053, None)
+        await c.connect(login=True)
+        info = await c.device_info()
+        print(
+            f"device {info.name} esphome {info.esphome_version} compiled {info.compilation_time}"
+        )
+        ents, _ = await c.list_entities_services()
+        by_key = {e.key: e for e in ents}
+        buttons = {e.name: e for e in ents if isinstance(e, ButtonInfo)}
+
+        def on_state(s) -> None:
+            e = by_key.get(s.key)
+            if e is None:
+                return
+            if isinstance(s, TextSensorState):
+                print(
+                    f"{stamp()} TXT {e.name} = {s.state!r}{' (missing)' if s.missing_state else ''}",
+                    flush=True,
+                )
+            elif isinstance(s, SensorState) and isinstance(e, SensorInfo):
+                print(f"{stamp()} SEN {e.name} = {s.state:.1f}", flush=True)
+
+        def on_log(msg) -> None:
+            log.write(f"{stamp()} {msg.message.decode(errors='replace')}\n")
+            log.flush()
+
+        c.subscribe_states(on_state)
+        c.subscribe_logs(on_log, log_level=LogLevel.LOG_LEVEL_DEBUG)
+        await asyncio.sleep(1.5)
+        for op in OPS:
+            if op == "list":
+                for n in sorted(buttons):
+                    print("button:", n)
+                for e in ents:
+                    if isinstance(e, (TextSensorInfo, SensorInfo)):
+                        print("entity:", type(e).__name__, e.name)
+                continue
+            kind, _, rest = op.partition(":")
+            if kind == "wait":
+                await asyncio.sleep(float(rest))
+                continue
+            if kind == "press":
+                if rest not in buttons:
+                    sys.exit(f"no button named {rest!r}; have {sorted(buttons)}")
+                print(f"{stamp()} >>> press {rest}", flush=True)
+                log.write(f"{stamp()} >>> press {rest}\n")
+                c.button_command(buttons[rest].key)
+            else:
+                sys.exit(f"unknown op {op!r}")
+            await asyncio.sleep(2.0)
+        await asyncio.sleep(1)
+        await c.disconnect()
+
+
+asyncio.run(main())

.agents/skills/hardware-test/templates/ble_midi/midicli.yaml

@@ -0,0 +1,149 @@
+substitutions:
+  peri_mac: "${PERI_MAC}"
+
+esphome:
+  name: midicli
+
+esp32:
+  board: esp32dev
+  framework:
+    type: esp-idf
+
+logger:
+  level: VERBOSE
+  logs:
+    esp32_ble_tracker: DEBUG
+    ble_client: DEBUG
+    esp32_ble: DEBUG
+    api: DEBUG
+    scheduler: DEBUG
+    sensor: DEBUG
+    wifi: DEBUG
+
+wifi:
+  ssid: !secret wifi_ssid
+  password: !secret wifi_password
+  power_save_mode: none
+
+api:
+  reboot_timeout: 0s
+
+esp32_ble_tracker:
+
+ble_client:
+  - mac_address: ${peri_mac}
+    id: midi_client
+
+ble_midi:
+  id: test_midi
+  ble_client_id: midi_client
+  max_sysex_size: 64
+  pair: false
+  on_connect:
+    - logger.log: "MIDI device connected"
+    - ble_midi.send_program_change:
+        id: test_midi
+        program: 0
+  on_disconnect:
+    - logger.log: "MIDI device disconnected"
+  on_note_on:
+    - logger.log:
+        format: "Note on %u velocity %u channel %u"
+        args: [note, velocity, channel]
+  on_note_off:
+    - logger.log:
+        format: "Note off %u channel %u"
+        args: [note, channel]
+  on_control_change:
+    - logger.log:
+        format: "CC %u = %u on channel %u"
+        args: [control_number, value, channel]
+  on_program_change:
+    - logger.log:
+        format: "Program %u on channel %u"
+        args: [program, channel]
+  on_pitch_bend:
+    - logger.log:
+        format: "Pitch bend %d on channel %u"
+        args: [value, channel]
+  on_sysex:
+    - lambda: |-
+        ESP_LOGI("midi", "SysEx of %u bytes: %s", (unsigned) data.size(), format_hex_pretty(data.data(), data.size()).c_str());
+  on_message:
+    - lambda: |-
+        ESP_LOGI("midi", "Raw message of %u bytes: %s", (unsigned) data.size(), format_hex_pretty(data.data(), data.size()).c_str());
+
+sensor:
+  - platform: debug
+    free:
+      name: Heap Free
+    loop_time:
+      name: Loop Time
+
+debug:
+  update_interval: 30s
+
+button:
+  - platform: template
+    name: "MIDI send tests"
+    on_press:
+      - ble_midi.send_note_on:
+          id: test_midi
+          note: 60
+      - ble_midi.send_note_on:
+          id: test_midi
+          note: !lambda "return 60;"
+          velocity: !lambda "return 100;"
+          channel: !lambda "return 1;"
+      - ble_midi.send_note_off:
+          id: test_midi
+          note: 60
+          velocity: 0
+          channel: 1
+      - ble_midi.send_control_change:
+          id: test_midi
+          control_number: 7
+          value: 64
+      - ble_midi.send_pitch_bend:
+          id: test_midi
+          value: -8192
+      - ble_midi.send_sysex:
+          id: test_midi
+          payload: [0x7E, 0x7F, 0x06, 0x01]
+      - ble_midi.send_raw:
+          id: test_midi
+          data: !lambda "return std::vector<uint8_t>{0x90, 0x3C, 0x40};"
+      - if:
+          condition:
+            ble_midi.connected:
+              id: test_midi
+          then:
+            - logger.log: "Still connected"
+          else:
+            - logger.log: "Not connected"
+  - platform: template
+    name: "MIDI send out of range"
+    on_press:
+      - ble_midi.send_note_on:
+          id: test_midi
+          note: !lambda "return 200;"
+      - ble_midi.send_control_change:
+          id: test_midi
+          control_number: 1
+          value: 1
+          channel: !lambda "return 16;"
+      - ble_midi.send_pitch_bend:
+          id: test_midi
+          value: 8191
+      - ble_midi.send_sysex:
+          id: test_midi
+          payload: [0xF0, 0x7E, 0x7F, 0x09, 0x01, 0xF7]
+      - ble_midi.send_raw:
+          id: test_midi
+          data: !lambda "return std::vector<uint8_t>(140, 0x00);"
+  - platform: template
+    name: "MIDI send long sysex"
+    on_press:
+      - ble_midi.send_sysex:
+          id: test_midi
+          payload: !lambda "return std::vector<uint8_t>(100, 0x11);"

.agents/skills/hardware-test/templates/ble_midi/midiperi.yaml

@@ -0,0 +1,177 @@
+esphome:
+  name: midiperi
+
+esp32:
+  board: esp32-s3-devkitc-1
+  variant: esp32s3
+  flash_size: 16MB
+  framework:
+    type: esp-idf
+
+logger:
+  hardware_uart: USB_SERIAL_JTAG
+  level: DEBUG
+
+api:
+  reboot_timeout: 0s
+
+ota:
+  - platform: esphome
+
+ethernet:
+  type: W5500
+  clk_pin: GPIO13
+  mosi_pin: GPIO11
+  miso_pin: GPIO12
+  cs_pin: GPIO14
+  interrupt_pin: GPIO10
+  reset_pin: GPIO9
+
+esp32_ble_server:
+  id: ble_server
+  manufacturer: FakeMIDI
+  on_connect:
+    - lambda: ESP_LOGI("peri", "central connected %d", id);
+  on_disconnect:
+    - lambda: ESP_LOGI("peri", "central disconnected %d", id);
+  services:
+    - uuid: 03B80E5A-EDE8-4B33-A751-6CE34EC4C700
+      advertise: true
+      characteristics:
+        - id: midi_io
+          uuid: 7772E5DB-3868-4112-A1A9-F2669D106BF3
+          read: true
+          write: true
+          write_no_response: true
+          notify: true
+          on_write:
+            - lambda: |-
+                ESP_LOGI("peri", "RX %u bytes: %s", (unsigned) x.size(), format_hex_pretty(x.data(), x.size()).c_str());
+
+script:
+  - id: emit
+    parameters:
+      pkt: std::vector<uint8_t>
+    then:
+      - lambda: |-
+          ESP_LOGI("peri", "TX %u bytes: %s", (unsigned) pkt.size(), format_hex_pretty(pkt.data(), pkt.size()).c_str());
+          id(midi_io).set_value(std::vector<uint8_t>(pkt));
+          id(midi_io).notify();
+
+button:
+  - platform: restart
+    name: P Restart
+  - platform: template
+    name: P Note On
+    on_press:
+      - script.execute:
+          id: emit
+          pkt: !lambda 'return {0x80, 0x80, 0x90, 0x3C, 0x64};'
+  - platform: template
+    name: P Note Off
+    on_press:
+      - script.execute:
+          id: emit
+          pkt: !lambda 'return {0x80, 0x80, 0x80, 0x3C, 0x00};'
+  - platform: template
+    name: P Control Change
+    on_press:
+      - script.execute:
+          id: emit
+          pkt: !lambda 'return {0x80, 0x80, 0xB0, 0x07, 0x40};'
+  - platform: template
+    name: P Program Change
+    on_press:
+      - script.execute:
+          id: emit
+          pkt: !lambda 'return {0x80, 0x80, 0xC5, 0x05};'
+  - platform: template
+    name: P Pitch Bend
+    on_press:
+      - script.execute:
+          id: emit
+          pkt: !lambda 'return {0x80, 0x80, 0xE0, 0x7F, 0x7F};'
+  - platform: template
+    name: P Two Notes One Packet
+    on_press:
+      - script.execute:
+          id: emit
+          pkt: !lambda 'return {0x80, 0x80, 0x90, 0x3C, 0x64, 0x81, 0x91, 0x3E, 0x50};'
+  - platform: template
+    name: P Running Status
+    on_press:
+      - script.execute:
+          id: emit
+          pkt: !lambda 'return {0x80, 0x80, 0x90, 0x40, 0x64, 0x41, 0x64};'
+  - platform: template
+    name: P Sysex One Packet
+    on_press:
+      - script.execute:
+          id: emit
+          pkt: !lambda 'return {0x80, 0x80, 0xF0, 0x7E, 0x7F, 0x06, 0x01, 0x80, 0xF7};'
+  - platform: template
+    name: P Sysex Split
+    on_press:
+      - script.execute:
+          id: emit
+          pkt: !lambda 'return {0x80, 0x80, 0xF0, 0x41, 0x10, 0x42};'
+      - delay: 200ms
+      - script.execute:
+          id: emit
+          pkt: !lambda 'return {0x80, 0x12, 0x00, 0x40, 0x80, 0xF7};'
+  - platform: template
+    name: P Sysex Oversize
+    on_press:
+      - script.execute:
+          id: emit
+          pkt: !lambda 'std::vector<uint8_t> v{0x80, 0x80, 0xF0}; for (int i = 0; i < 17; i++) v.push_back(i); return v;'
+      - delay: 200ms
+      - script.execute:
+          id: emit
+          pkt: !lambda 'std::vector<uint8_t> v{0x80}; for (int i = 0; i < 19; i++) v.push_back(0x20 + i); return v;'
+      - delay: 200ms
+      - script.execute:
+          id: emit
+          pkt: !lambda 'std::vector<uint8_t> v{0x80}; for (int i = 0; i < 19; i++) v.push_back(0x40 + i); return v;'
+      - delay: 200ms
+      - script.execute:
+          id: emit
+          pkt: !lambda 'std::vector<uint8_t> v{0x80}; for (int i = 0; i < 19; i++) v.push_back(0x60 + i); return v;'
+      - delay: 200ms
+      - script.execute:
+          id: emit
+          pkt: !lambda 'return {0x80, 0x80, 0xF7};'
+  - platform: template
+    name: P Sysex Cut Short
+    on_press:
+      - script.execute:
+          id: emit
+          pkt: !lambda 'return {0x80, 0x80, 0xF0, 0x41, 0x10, 0x80, 0x90, 0x3C, 0x64};'
+  - platform: template
+    name: P Clock
+    on_press:
+      - script.execute:
+          id: emit
+          pkt: !lambda 'return {0x80, 0x80, 0xF8};'
+  - platform: template
+    name: P Aftertouch
+    on_press:
+      - script.execute:
+          id: emit
+          pkt: !lambda 'return {0x80, 0x80, 0xD3, 0x45};'
+  - platform: template
+    name: P Orphan Data Byte
+    on_press:
+      - script.execute:
+          id: emit
+          pkt: !lambda 'return {0x80, 0x80, 0x3C, 0x64};'
+  - platform: template
+    name: P Burst
+    on_press:
+      - repeat:
+          count: 50
+          then:
+            - script.execute:
+                id: emit
+                pkt: !lambda 'return {0x80, 0x80, 0x90, 0x30, 0x40, 0x80, 0x80, 0x30, 0x00};'
+            - delay: 20ms

.agents/skills/hardware-test/templates/http_request_update/boards/upd32.yaml

@@ -0,0 +1,10 @@
+esphome:
+  name: upd32
+
+esp32:
+  board: esp32dev
+  framework:
+    type: esp-idf
+
+packages:
+  common: !include ../upd-common.yaml

.agents/skills/hardware-test/templates/http_request_update/boards/upd8266.yaml

@@ -0,0 +1,8 @@
+esphome:
+  name: upd8266
+
+esp8266:
+  board: d1_mini
+
+packages:
+  common: !include ../upd-common.yaml

.agents/skills/hardware-test/templates/http_request_update/upd-base-common.yaml

@@ -0,0 +1,52 @@
+substitutions:
+  base: !secret manifest_base
+
+wifi:
+  ssid: !secret wifi_ssid
+  password: !secret wifi_password
+  power_save_mode: none
+
+logger:
+  level: DEBUG
+api:
+  reboot_timeout: 0s
+
+http_request:
+  verify_ssl: false
+  timeout: 10s
+
+ota:
+  - platform: http_request
+    id: ota_http
+
+update:
+  - platform: http_request
+    id: upd
+    name: Firmware
+    source: ${base}/stable.json
+    update_interval: 1h
+
+button:
+  - platform: template
+    name: Check
+    on_press:
+      - lambda: id(upd).check();
+  - platform: template
+    name: URL Slow Cstr
+    on_press:
+      - lambda: |-
+          std::string url = "${base}/slow.json";
+          id(upd).set_source_url(url.c_str());
+  - platform: template
+    name: URL Stable Literal
+    on_press:
+      - lambda: id(upd).set_source_url("${base}/stable.json");
+  - platform: template
+    name: Heap
+    on_press:
+      - lambda: |-
+          #ifdef USE_ESP8266
+          ESP_LOGI("harness", "heap free %u", ESP.getFreeHeap());
+          #else
+          ESP_LOGI("harness", "heap free %u", (unsigned) heap_caps_get_free_size(MALLOC_CAP_INTERNAL));
+          #endif

.agents/skills/hardware-test/templates/http_request_update/upd-common.yaml

@@ -0,0 +1,95 @@
+substitutions:
+  base: !secret manifest_base
+
+wifi:
+  ssid: !secret wifi_ssid
+  password: !secret wifi_password
+  power_save_mode: none
+
+logger:
+  level: DEBUG
+  on_message:
+    - level: DEBUG
+      then:
+        - lambda: |-
+            // Inline check path: replace the URL from inside the check (ESP8266 runs the check on the main loop)
+            if (id(swap_in_check) && strstr(message, "Content-Length") != nullptr) {
+              id(swap_in_check) = false;
+              std::string url = "${base}/beta.json";
+              id(upd).set_source_url(url);
+              ESP_LOGI("harness", "URL replaced from inside the check");
+            }
+
+api:
+  reboot_timeout: 0s
+
+http_request:
+  verify_ssl: false
+  timeout: 10s
+
+ota:
+  - platform: http_request
+    id: ota_http
+
+update:
+  - platform: http_request
+    id: upd
+    name: Firmware
+    source: ${base}/stable.json
+    update_interval: 1h
+
+globals:
+  - id: swap_in_check
+    type: bool
+    initial_value: "false"
+
+button:
+  - platform: template
+    name: Check
+    on_press:
+      - lambda: id(upd).check();
+  - platform: template
+    name: URL Beta String
+    on_press:
+      - lambda: |-
+          std::string url = "${base}/beta.json";
+          id(upd).set_source_url(url);
+  - platform: template
+    name: URL Slow Cstr
+    on_press:
+      - lambda: |-
+          std::string url = "${base}/slow.json";
+          id(upd).set_source_url(url.c_str());
+  - platform: template
+    name: URL Stable Literal
+    on_press:
+      - lambda: id(upd).set_source_url("${base}/stable.json");
+  - platform: template
+    name: URL Temporary
+    on_press:
+      - lambda: id(upd).set_source_url(std::string("${base}/beta.json"));
+  - platform: template
+    name: Same URL Back
+    on_press:
+      - lambda: id(upd).set_source_url(id(upd).get_source_url());
+  - platform: template
+    name: Check Then Swap
+    on_press:
+      - lambda: |-
+          // ESP32: the check task reads slow.json while the URL is replaced under it
+          id(upd).check();
+          std::string url = "${base}/beta.json";
+          id(upd).set_source_url(url);
+  - platform: template
+    name: Arm Swap In Check
+    on_press:
+      - lambda: id(swap_in_check) = true;
+  - platform: template
+    name: Heap
+    on_press:
+      - lambda: |-
+          #ifdef USE_ESP8266
+          ESP_LOGI("harness", "heap free %u", ESP.getFreeHeap());
+          #else
+          ESP_LOGI("harness", "heap free %u", (unsigned) heap_caps_get_free_size(MALLOC_CAP_INTERNAL));
+          #endif

.agents/skills/hardware-test/templates/modem/modem-common.yaml

@@ -0,0 +1,65 @@
+esp32:
+  board: esp32dev
+  framework:
+    type: esp-idf
+  watchdog_timeout: 30s
+
+logger:
+  level: DEBUG
+
+api:
+  reboot_timeout: 0s
+
+modem:
+  id: atmodem
+  rx_pin: GPIO26
+  tx_pin: GPIO27
+  baud_rate: 115200
+  model: SIM7600
+  apn: internet
+  pin_code: "0000"
+  init_at:
+    - ATI
+  urc:
+    name: Modem URC
+  reboot_timeout: ${modem_reboot_timeout}
+  on_connect:
+    logger.log: "modem connected"
+  on_disconnect:
+    logger.log: "modem disconnected"
+  on_sync:
+    logger.log: "modem synced"
+
+sensor:
+  - platform: uptime
+    name: Uptime
+    update_interval: 30s
+  - platform: debug
+    free:
+      name: Heap Free
+    loop_time:
+      name: Loop Time
+
+debug:
+  update_interval: 30s
+
+button:
+  - platform: template
+    name: Modem Enable
+    on_press:
+      - modem.enable
+  - platform: template
+    name: Modem Disable
+    on_press:
+      - modem.disable
+  - platform: template
+    name: Modem Send AT
+    on_press:
+      - modem.send_at: "AT+CGSN"
+
+interval:
+  - interval: 20s
+    then:
+      - lambda: |-
+          ESP_LOGI("alive", "uptime %u s enabled=%d connected=%d", (unsigned) (millis() / 1000),
+                   id(atmodem).is_enabled(), id(atmodem).is_connected());

.agents/skills/hardware-test/templates/modem/modemo.yaml

@@ -0,0 +1,26 @@
+substitutions:
+  modem_reboot_timeout: 2min
+
+esphome:
+  name: modemo
+  on_boot:
+    priority: -100
+    then:
+      - modem.disable
+      - modem.enable
+      - if:
+          condition:
+            modem.enabled:
+          then:
+            modem.send_at: "AT+CGSN"
+          else:
+            modem.enable:
+      - if:
+          condition:
+            not:
+              modem.connected:
+          then:
+            logger.log: modem not connected at boot
+
+packages:
+  common: !include modem-common.yaml

.agents/skills/hardware-test/templates/modem/modemw.yaml

@@ -0,0 +1,12 @@
+substitutions:
+  modem_reboot_timeout: 0s
+
+esphome:
+  name: modemw
+
+wifi:
+  ssid: !secret wifi_ssid
+  password: !secret wifi_password
+
+packages:
+  common: !include modem-common.yaml

.agents/skills/hardware-test/templates/store_yaml/boards/sy8266.yaml

@@ -0,0 +1,15 @@
+esphome:
+  name: sy8266
+
+esp8266:
+  board: d1_mini
+
+api:
+  encryption:
+    key: !secret api_key
+  reboot_timeout: 0s
+
+packages:
+  common: !include ../sy-common.yaml
+
+store_yaml:

.agents/skills/hardware-test/templates/store_yaml/boards/sy8266s.yaml

@@ -0,0 +1,16 @@
+esphome:
+  name: sy8266s
+
+esp8266:
+  board: d1_mini
+
+api:
+  encryption:
+    key: !secret api_key
+  reboot_timeout: 0s
+
+packages:
+  common: !include ../sy-common.yaml
+
+store_yaml:
+  include_secrets: true

.agents/skills/hardware-test/templates/store_yaml/sy-common.yaml

@@ -0,0 +1,24 @@
+# shared package for the store_yaml hardware test
+substitutions:
+  greeting: "hello from the package"
+
+wifi:
+  ssid: !secret wifi_ssid
+  password: !secret wifi_password
+  power_save_mode: none
+
+logger:
+  level: DEBUG
+
+ota:
+  - platform: esphome
+    password: recoverme123
+
+sensor:
+  - platform: template
+    name: Package Sensor
+    lambda: return 42.0;
+    update_interval: 60s
+
+packages:
+  extra: !include sy-extra.yaml

.agents/skills/hardware-test/templates/store_yaml/sy-extra.yaml

@@ -0,0 +1,6 @@
+# second level include
+text_sensor:
+  - platform: template
+    name: Greeting
+    lambda: return std::string("${greeting}");
+    update_interval: 60s