[light] Add a StringRef overload for LightCall::set_effect (#21358)

Co-authored-by: J. Nick Koston <nick@home-assistant.io>
Co-authored-by: J. Nick Koston <nick@koston.org>

.agents/skills/hardware-test/SKILL.md

@@ -115,7 +115,7 @@ Per-topic scripts, each described in the matching `reference/recipes/<component>
 (host driver, Zephyr driver probe), `rtsp_server/` (protocol client, raw rate reader, measurement set,
 API image grab), `cse7766/` (API observer with ping), `zephyr_coredump/` (reader flow, decode with the
 Zephyr tools), `libretiny_prefs/` (rtc preference harness header), `libretiny_crash/` (BK72xx crash probe header and
-capture decoder), `api_outgoing/` (wake cycle and Wi-Fi drop timing readers), `ld2450/` (zone number driver, loopback harness driver with number and binary sensor states and TX frame summarizer), `api/` (raw plaintext native API client for user defined actions), `dfrobot_sen0395/` (API driver and case runner), `homeassistant_light/` (direct bulb driver, held-report case driver), `usb_audio/` (URL player), `speaker_media_player/` (API player driver, pipeline task case runner), `web_server/` (SSE client, request mix, SSE and HTTPD case runners), `mqtt/` (broker config with TLS, client-certificate and WebSocket listeners, test CA with server and client certificates, transport case runner, flash-and-drive runner, CLI TLS check without a board), `udp/` (datagram recorder, `udp.write` payload checker and case driver), `entity_icons/` (icon and device class readback over the API, MQTT discovery, web_server and dump_config, with a comparer).
+capture decoder), `api_outgoing/` (wake cycle and Wi-Fi drop timing readers), `ld2450/` (zone number driver), `dfrobot_sen0395/` (API driver and case runner), `homeassistant_light/` (direct bulb driver, held-report case driver), `usb_audio/` (URL player), `speaker_media_player/` (API player driver, pipeline task case runner), `web_server/` (SSE client, request mix, SSE and HTTPD case runners), `mqtt/` (broker config with TLS, client-certificate and WebSocket listeners, test CA with server and client certificates, transport case runner, flash-and-drive runner, CLI TLS check without a board), `udp/` (datagram recorder, `udp.write` payload checker and case driver), `entity_icons/` (icon and device class readback over the API, MQTT discovery, web_server and dump_config, with a comparer).
 
 ## Templates in `$SKILL/templates`
 

.agents/skills/hardware-test/reference/boards.md

@@ -7,9 +7,6 @@ Ports are not listed on purpose: they follow the USB hub port and change on ever
 
 - Opening the port usually resets the board through DTR, but not always; `serial_capture.py --reset`
   pulses RTS to capture from boot. 74880 baud shows the ROM boot messages (`rst cause`, `boot mode`).
-- UART0 can be looped back onto itself (`USC0(0) |= (1 << UCLBE)` in an `on_boot` lambda) so a UART
-  component is testable without its device: frames written with `uart.write` arrive on RX. Log over the
-  API with `logger: baud_rate: 0`, since the serial port carries the component's bytes.
 - `esphome logs` on some CH340s emits thousands of duplicated garbage lines right after the RTS reset;
   a plain pyserial capture at 115200 is reliable.
 - A failed esptool write can wedge the adapter (`termios` error 22, or an open that hangs) until replug.

.agents/skills/hardware-test/reference/recipes/README.md

@@ -4,13 +4,12 @@ Each flow names the pull requests it was used on, so their result tables show wh
 
 One file per component, named after it:
 
-- [`api.md`](api.md): noise handshake and OTA encryption on the ESP8266, outgoing connection wake timing on deep sleep and after a Wi-Fi drop, user defined action listing and execution with a raw plaintext client
+- [`api.md`](api.md): noise handshake and OTA encryption on the ESP8266, outgoing connection wake timing on deep sleep and after a Wi-Fi drop
 - [`captive_portal.md`](captive_portal.md): fallback AP and /wifisave hammer
 - [`cse7766.md`](cse7766.md): Sonoff S31 parser check over the API
 - [`espnow.md`](espnow.md): ESP8266 broadcast receive
 - [`hdmi_cec.md`](hdmi_cec.md): two-board wire
-- [`ld2450.md`](ld2450.md): polygon zones, zone number round trip through the radar's readback, ESP8266 loopback harness without a radar
-- [`ld2412.md`](ld2412.md): output pin level through the basic configuration command, held back writes
+- [`ld2450.md`](ld2450.md): polygon zones, zone number round trip through the radar's readback
 - [`libretiny.md`](libretiny.md): rtc preference emulation, BK72xx crash report on the next boot
 - [`mqtt.md`](mqtt.md): broker bench, QoS, reconnect, crash log over MQTT, WebSocket transports with a local and a public broker, TLS trust sources (own CA, client certificate, certificate bundle, no verification) with the CLI check
 - [`nrf52.md`](nrf52.md): dongle board support
@@ -21,7 +20,7 @@ One file per component, named after it:
 - [`scheduler.md`](scheduler.md): loop wake A/B
 - [`udp.md`](udp.md): datagrams, multicast, sockets, `udp.write` payload shapes
 - [`usb_cdc_acm.md`](usb_cdc_acm.md): nRF52840 dongle host driver
-- [`wifi.md`](wifi.md): reboot_timeout with a fallback AP, DHCP lease cache, Easy Connect (DPP), WISP bridge with an ethernet upstream
+- [`wifi.md`](wifi.md): reboot_timeout with a fallback AP, DHCP lease cache, Easy Connect (DPP)
 - [`zephyr.md`](zephyr.md): preferences on the nRF52840 dongle
 - [`zephyr_coredump.md`](zephyr_coredump.md): crash and decode on nRF52 boards
 - [`zwave_proxy.md`](zwave_proxy.md): frame checksums on a live proxy

.agents/skills/hardware-test/reference/recipes/api.md

@@ -28,15 +28,3 @@ connected first, and the loop-time maxima. For the loop cost of polling, repeat
 so after a drop every dial is closed and HA's incoming reconnect wins 11 to 33 s later; the measurable effect is the
 first dial landing in the loop the network flag flips.
 Results: https://github.com/esphome/esphome/pull/21085#issuecomment-6254742791
-
-## User defined action listing and execution without a client library (#18882)
-
-`scripts/api/rawactions.py HOST list` speaks the plaintext native API over a raw socket and decodes every
-`ListEntitiesServicesResponse` by field number (argument fields 1 name, 2 type, 3 description, 4 example, 5 optional,
-6 default_value), so new wire fields show up without waiting for the client library. `rawactions.py HOST exec ACTION
-k=v ...` sends `ExecuteServiceRequest` with `sint32` ints (field 5, not the legacy int32 field 2), fixed32 floats,
-length delimited strings and `int[]` as repeated sint32 (field 7); `ALLOW_MISSING=1` leaves arguments out on purpose
-(the device then drops the call). `templates/api-actions-dut.yaml` (D1 mini, three actions, optional variables with
-defaults) and `templates/api-actions-strings-dut.yaml` (required strings) log what arrived. String arguments are
-`StringRef`s that are not null terminated: copy them with `snprintf("%.*s", size, c_str())` into a buffer before
-logging, and do not pass `%.*s` to `logger.log`, whose validator counts it as one pattern.

.agents/skills/hardware-test/reference/recipes/ld2412.md

@@ -1,13 +0,0 @@
-# ld2412
-
-## Output pin level and the basic configuration command (#21353)
-
-`templates/ld2412-select-only-dut.yaml` (ESP32-S3, the module on GPIO43 and GPIO44 at 115200, only the output pin
-level select, the module's OUT pin on GPIO38 as a binary sensor plus the presence sensor) with `uart: debug:
-direction: BOTH`. Drive the select with `scripts/ld2450/lddrive.py HOST KEY LOGFILE sel:NAME=OPTION` and read the TX
-dump for command `02` carrying the level byte, then the OUT pin binary sensor against presence: with level high the
-pin follows presence, with level low it is inverted. A module that never answers the boot query (no `<<<` lines)
-only shows the held back path: `Command 02 held back until the module reports its settings`, a query `12`, and after
-2 s `Command 02 not sent, the module did not report its settings` with the select republished to the last known
-level. With the full set of numbers configured a select change also needs every number to have a state, so wait
-for the boot query answer before judging.

.agents/skills/hardware-test/reference/recipes/ld2450.md

@@ -13,30 +13,3 @@ selects over the API and prints every LD2450 state update. After each set the hu
 the radar and republishes all twelve coordinates from the answer, so set each coordinate of each zone to a distinct
 value and check the readbacks; no target is needed. Readbacks can arrive late over Wi-Fi, so match them by content.
 Results: https://github.com/esphome/esphome/pull/21146#issuecomment-6278026922
-
-## ESP8266 loopback harness, no radar needed (#21349)
-
-`templates/ld2450-loopback.yaml` (package; substitutions `name` and `baud`) runs the hub on UART0 (GPIO1 and GPIO3)
-with `logger: baud_rate: 0` and sets the UART0 loopback bit in `on_boot` (`USC0(0) |= (1 << UCLBE)`), so everything
-the hub or a `uart.write` action sends comes back on RX. Template buttons inject periodic frames (all four direction
-strings, targets 2 and 3), query zone acks (zone type 0, 1 and 2 for the select readback) and call the name based
-setters with a valid and a bogus name. `uart: debug: direction: TX` delimited on the command footer dumps every
-command the hub sends. Drive it with `scripts/ld2450/lddrive.py HOST KEY LOGFILE press:NAME sel:NAME=OPTION wait:S`
-(logs over the API at VERBOSE, 2 s between steps because the API log stream drops lines while `send_command_`
-blocks), then `txsum.py LOGFILE` lists the TX frames per step; diff the summaries of the two builds. The hub reads its
-own commands back and logs `Invalid status` for each, which is harmless. The boot readback of the baud rate select
-comes from the UART setting, so a second build at `baud: "9600"` checks the flash table lookup.
-Results: https://github.com/esphome/esphome/pull/21349#issuecomment-6332320026
-
-## Frame splitter collisions (#21357)
-
-The loopback template also carries the zone 1 numbers and seven collision buttons: a periodic frame with the
-command footer inside target 1, zone acks with `55 CC` and with `04 03 02 01` inside the coordinates, a periodic
-frame cut after 12 bytes followed by a zone ack, a frame missing its last byte followed by a complete one, a bogus
-255 byte length then a frame, and garbage then a frame. Judge by the state updates plus the `Handling Periodic` and
-`Handling Ack` lines. Through a real radar the set zone ack is sometimes corrupted by the radar itself (its tail
-replaced by the periodic frame it was sending, for example `04 03 03 03 00 ... 55 CC`), so count readbacks over at
-least 24 selects per build and read the raw bytes with `uart: debug: direction: RX` (`!extend` the package's uart
-id) before blaming the parser; the API log stream drops lines under that dump volume, so some misses stay
-unclassified.
-Results: https://github.com/esphome/esphome/pull/21357#issuecomment-6333244886

.agents/skills/hardware-test/reference/recipes/wifi.md

@@ -23,17 +23,3 @@ Only an Android phone configures: Settings, network details, Add device, Share W
 QR scanner makes the phone an enrollee). Render the URI with the `qrcode` module and open the PNG.
 `esp32: framework: log_level: DEBUG` plus `sdkconfig_options: CONFIG_WPA_DEBUG_PRINT: y` prints the
 supplicant state machine.
-
-## WISP bridge with an ethernet upstream (#7147)
-
-`templates/wisp-ethernet-dut.yaml` (ESP32-S3 with a W5500, `wifi: ap:` with `ap_mode: WISP` and no station) and
-`templates/wisp-client.yaml` (D1 mini joining the bridge, `http_request` to a web server on the wired LAN and to
-`http://example.com/` every 15 s, `time: sntp`). Secrets: `wisp_psk` and `lan_http` (serve a text file on the host
-with `python -m http.server 8787 --bind 0.0.0.0` from a scratch directory). Judge by the client's `GET status 200`
-lines, the web server log showing the device's ethernet address as the source (NAT), the client's DHCP info (DNS1 is
-the LAN resolver) and `SNTP synced`. The device logs `Starting AP` and `DHCP DNS server: <resolver>` on its USB
-console; flashing over the air reboots it for repeat boots. Before the AP address and DHCP server moved ahead of the
-AP start (the fix in that PR), the first AP start failed on every boot with `esp_netif_dhcps_option failed: 20484`
-(DHCP server already started by the AP start event) and `Failed to enable NAT`, and the bridge only came up on the
-retry 90 s later; a client that cannot find the SSID for the first minute on an older build is that. With the fix
-the bridge is up 6 s after boot. Any change to the ESP-IDF AP start should also rerun the captive portal hammer.

.agents/skills/hardware-test/scripts/api/rawactions.py

@@ -1,211 +0,0 @@
-"""Plaintext native API client that lists user actions with their raw argument fields and executes one.
-
-usage: rawactions.py HOST list
-       rawactions.py HOST exec ACTION key=value [...]   (every argument must be given; types from the listing)
-Fields of ListEntitiesServicesArgument: 1 name, 2 type, 3 description, 4 example, 5 optional, 6 default_value.
-"""
-
-import os
-import socket
-import struct
-import sys
-
-HOST = sys.argv[1]
-CMD = sys.argv[2]
-ARGS = sys.argv[3:]
-TYPE_NAMES = {
-    0: "bool",
-    1: "int",
-    2: "float",
-    3: "string",
-    4: "bool[]",
-    5: "int[]",
-    6: "float[]",
-    7: "string[]",
-}
-
-
-def varint(v: int) -> bytes:
-    out = bytearray()
-    while True:
-        b = v & 0x7F
-        v >>= 7
-        if v:
-            out.append(b | 0x80)
-        else:
-            out.append(b)
-            return bytes(out)
-
-
-def svarint(v: int) -> bytes:
-    return varint((v << 1) ^ (v >> 63))
-
-
-def decode_varint(buf: bytes, pos: int) -> tuple[int, int]:
-    value = shift = 0
-    while True:
-        b = buf[pos]
-        pos += 1
-        value |= (b & 0x7F) << shift
-        if not b & 0x80:
-            return value, pos
-        shift += 7
-
-
-def fields(payload: bytes) -> list[tuple[int, int, object]]:
-    out = []
-    pos = 0
-    while pos < len(payload):
-        key, pos = decode_varint(payload, pos)
-        fnum, wt = key >> 3, key & 7
-        if wt == 0:
-            v, pos = decode_varint(payload, pos)
-        elif wt == 2:
-            ln, pos = decode_varint(payload, pos)
-            v = payload[pos : pos + ln]
-            pos += ln
-        elif wt == 5:
-            v = payload[pos : pos + 4]  # fixed32, caller unpacks
-            pos += 4
-        else:
-            raise ValueError(f"wire type {wt}")
-        out.append((fnum, wt, v))
-    return out
-
-
-def frame(mtype: int, payload: bytes) -> bytes:
-    return b"\x00" + varint(len(payload)) + varint(mtype) + payload
-
-
-def field_bytes(fnum: int, data: bytes) -> bytes:
-    return varint((fnum << 3) | 2) + varint(len(data)) + data
-
-
-def field_varint(fnum: int, v: int) -> bytes:
-    return varint((fnum << 3) | 0) + varint(v)
-
-
-def recv_frames(
-    sock: socket.socket, until: set[int], limit: float = 5.0
-) -> list[tuple[int, bytes]]:
-    sock.settimeout(limit)
-    buf = bytearray()
-    frames = []
-    while True:
-        try:
-            chunk = sock.recv(4096)
-        except TimeoutError:
-            return frames
-        if not chunk:
-            return frames
-        buf += chunk
-        while True:
-            if len(buf) < 3:
-                break
-            if buf[0] != 0:
-                raise ValueError("not plaintext")
-            try:
-                ln, p = decode_varint(buf, 1)
-                mt, p = decode_varint(buf, p)
-            except IndexError:
-                break
-            if len(buf) < p + ln:
-                break
-            frames.append((mt, bytes(buf[p : p + ln])))
-            del buf[: p + ln]
-            if mt in until:
-                return frames
-
-
-def main() -> None:
-    sock = socket.create_connection((HOST, 6053), timeout=5)
-    sock.sendall(
-        frame(
-            1, field_bytes(1, b"rawactions") + field_varint(2, 1) + field_varint(3, 10)
-        )
-    )  # HelloRequest
-    recv_frames(sock, {2})
-    sock.sendall(frame(3, b""))  # ConnectRequest, no password
-    recv_frames(sock, {4})
-    sock.sendall(frame(11, b""))  # ListEntitiesRequest
-    frames = recv_frames(sock, {19})
-    services = {}
-    for mt, payload in frames:
-        if mt != 41:
-            continue
-        name = key = None
-        args = []
-        for fnum, _, v in fields(payload):
-            if fnum == 1:
-                name = v.decode()
-            elif fnum == 2:
-                key = struct.unpack("<I", v)[0]
-            elif fnum == 3:
-                a = {
-                    "name": None,
-                    "type": 0,
-                    "description": None,
-                    "example": None,
-                    "optional": None,
-                    "default": None,
-                }
-                for af, _, av in fields(v):
-                    if af == 1:
-                        a["name"] = av.decode()
-                    elif af == 2:
-                        a["type"] = av
-                    elif af == 3:
-                        a["description"] = av.decode()
-                    elif af == 4:
-                        a["example"] = av.decode()
-                    elif af == 5:
-                        a["optional"] = bool(av)
-                    elif af == 6:
-                        a["default"] = av.decode()
-                    else:
-                        a[f"field{af}"] = av
-                args.append(a)
-        services[name] = (key, args)
-    if CMD == "list":
-        for name, (key, args) in services.items():
-            print(f"action {name} key={key}")
-            for a in args:
-                extra = {
-                    k: v
-                    for k, v in a.items()
-                    if k not in ("name", "type") and v is not None
-                }
-                print(f"  {a['name']}: {TYPE_NAMES.get(a['type'], a['type'])} {extra}")
-        return
-    name = ARGS[0]
-    key, args = services[name]
-    payload = varint((1 << 3) | 5) + struct.pack("<I", key)
-    given = dict(kv.split("=", 1) for kv in ARGS[1:])
-    for a in args:
-        if a["name"] not in given:
-            if os.environ.get("ALLOW_MISSING"):
-                continue  # leave the argument out of the request on purpose
-            sys.exit(f"missing {a['name']}")
-        raw = given[a["name"]]
-        t = a["type"]
-        if t == 0:
-            av = field_varint(1, 1 if raw.lower() == "true" else 0)
-        elif t == 1:
-            av = varint((5 << 3) | 0) + svarint(int(raw))  # sint32 int_
-        elif t == 2:
-            av = varint((3 << 3) | 5) + struct.pack("<f", float(raw))
-        elif t == 3:
-            av = field_bytes(4, raw.encode())
-        elif t == 5:
-            av = b"".join(
-                varint((7 << 3) | 0) + svarint(int(x)) for x in raw.split(",") if x
-            )  # repeated sint32 int_array
-        else:
-            sys.exit(f"type {t} not supported here")
-        payload += field_bytes(2, av)
-    sock.sendall(frame(42, payload))  # ExecuteServiceRequest
-    print(f"sent {name} with {given}")
-    sock.close()
-
-
-main()

.agents/skills/hardware-test/scripts/ld2450/lddrive.py

@@ -1,123 +0,0 @@
-"""Drive an LD24xx harness over the API: press buttons, set selects and numbers, print state updates, save the log.
-
-usage: lddrive.py HOST KEY LOGFILE OP [OP ...]   (KEY of - means a plaintext API)
-  OP: list | press:NAME | sel:NAME=OPTION | set:NAME=VALUE | wait:SECONDS
-Steps are 2 s apart: the API log stream drops lines while the hub blocks in send_command_.
-"""
-
-import asyncio
-from pathlib import Path
-import sys
-import time
-
-from aioesphomeapi import (
-    APIClient,
-    BinarySensorState,
-    ButtonInfo,
-    LogLevel,
-    NumberInfo,
-    NumberState,
-    SelectInfo,
-    SelectState,
-    SensorInfo,
-    SensorState,
-    TextSensorInfo,
-    TextSensorState,
-)
-
-HOST, KEY, LOGFILE = sys.argv[1], sys.argv[2], sys.argv[3]
-OPS = sys.argv[4:]
-T0 = time.time()
-STEP_GAP_S = 2.0
-
-
-def stamp() -> str:
-    return f"{time.time() - T0:7.3f}"
-
-
-async def run(c: APIClient, log) -> None:
-    info = await c.device_info()
-    print(
-        f"device {info.name} esphome {info.esphome_version} compiled {info.compilation_time}"
-    )
-    ents, _ = await c.list_entities_services()
-    by_key = {e.key: e for e in ents}
-    buttons = {e.name: e for e in ents if isinstance(e, ButtonInfo)}
-    selects = {e.name: e for e in ents if isinstance(e, SelectInfo)}
-    numbers = {e.name: e for e in ents if isinstance(e, NumberInfo)}
-
-    def on_state(s) -> None:
-        e = by_key.get(s.key)
-        if e is None:
-            return
-        missing = " (missing)" if getattr(s, "missing_state", False) else ""
-        if isinstance(s, TextSensorState):
-            print(f"{stamp()} TXT {e.name} = {s.state!r}{missing}", flush=True)
-        elif isinstance(s, BinarySensorState):
-            print(f"{stamp()} BIN {e.name} = {s.state}{missing}", flush=True)
-        elif isinstance(s, NumberState):
-            print(f"{stamp()} NUM {e.name} = {s.state:.0f}{missing}", flush=True)
-        elif isinstance(s, SelectState):
-            print(f"{stamp()} SEL {e.name} = {s.state!r}{missing}", flush=True)
-        elif (
-            isinstance(s, SensorState)
-            and isinstance(e, SensorInfo)
-            and "Count" in e.name
-        ):
-            print(f"{stamp()} CNT {e.name} = {s.state:.0f}", flush=True)
-
-    def on_log(msg) -> None:
-        log.write(f"{stamp()} {msg.message.decode(errors='replace')}\n")
-
-    c.subscribe_states(on_state)
-    c.subscribe_logs(on_log, log_level=LogLevel.LOG_LEVEL_VERBOSE)
-    await asyncio.sleep(1.5)
-    for op in OPS:
-        if op == "list":
-            for n in sorted(buttons):
-                print("button:", n)
-            for n in sorted(selects):
-                print("select:", n, selects[n].options)
-            for e in ents:
-                if isinstance(e, TextSensorInfo):
-                    print("text_sensor:", e.name)
-            continue
-        kind, _, rest = op.partition(":")
-        if kind == "wait":
-            await asyncio.sleep(float(rest))
-            continue
-        if kind == "press":
-            if rest not in buttons:
-                sys.exit(f"no button named {rest!r}; have {sorted(buttons)}")
-            print(f"{stamp()} >>> press {rest}", flush=True)
-            log.write(f"{stamp()} >>> press {rest}\n")
-            c.button_command(buttons[rest].key)
-        elif kind == "set":
-            name, _, value = rest.partition("=")
-            if name not in numbers:
-                sys.exit(f"no number named {name!r}; have {sorted(numbers)}")
-            print(f"{stamp()} >>> set {name} = {value}", flush=True)
-            log.write(f"{stamp()} >>> set {name} = {value}\n")
-            c.number_command(numbers[name].key, float(value))
-        elif kind == "sel":
-            name, _, value = rest.partition("=")
-            if name not in selects:
-                sys.exit(f"no select named {name!r}; have {sorted(selects)}")
-            print(f"{stamp()} >>> select {name} = {value}", flush=True)
-            log.write(f"{stamp()} >>> select {name} = {value}\n")
-            c.select_command(selects[name].key, value)
-        else:
-            sys.exit(f"unknown op {op!r}")
-        await asyncio.sleep(STEP_GAP_S)
-    await asyncio.sleep(1)
-
-
-async def main() -> None:
-    c = APIClient(HOST, 6053, None, noise_psk=None if KEY == "-" else KEY)
-    await c.connect(login=True)
-    with Path(LOGFILE).open("w", encoding="utf-8") as log:
-        await run(c, log)
-    await c.disconnect()
-
-
-asyncio.run(main())

.agents/skills/hardware-test/scripts/ld2450/txsum.py

@@ -1,33 +0,0 @@
-"""Summarize an lddrive.py log: for each step, the LD2450 TX frames, commands and publishes.
-
-usage: txsum.py LOGFILE   (diff the output of two builds)
-"""
-
-from pathlib import Path
-import re
-import sys
-
-ANSI = re.compile(r"\x1b\[[0-9;]*m")
-out: list[str] = []
-with Path(sys.argv[1]).open(encoding="utf-8", errors="replace") as log:
-    for raw in log:
-        line = ANSI.sub("", raw.rstrip("\n"))
-        m = re.match(r"\s*[\d.]+ (.*)", line)
-        if not m:
-            continue
-        text = m.group(1)
-        if text.startswith(">>>"):
-            out.append(f"\n# {text[4:]}")
-        elif "[uart_debug" in text:
-            out.append(f"  TX {text.split('>>> ')[1]}")
-        elif "[ld2450" in text and (
-            "Sending COMMAND" in text
-            or "zone type" in text.lower()
-            or "Query zone" in text
-        ):
-            out.append(f"  LOG {text.split(']: ', 1)[1]}")
-        elif "[select:" in text and ">>" in text:
-            out.append(f"  SEL {text.split(']: ', 1)[1]}")
-        elif "[text_sensor:098]" in text:
-            out.append(f"  TXT {text.split(']: ', 1)[1]}")
-print("\n".join(out))

.agents/skills/hardware-test/templates/api-actions-dut.yaml

@@ -1,65 +0,0 @@
-esphome:
-  name: ${name}
-
-esp8266:
-  board: d1_mini
-
-wifi:
-  ssid: !secret wifi_ssid
-  password: !secret wifi_password
-  power_save_mode: none
-
-logger:
-  level: DEBUG
-
-api:
-  reboot_timeout: 0s
-  actions:
-    - action: play_buzzer
-      description: Play a song or build one from parts
-      variables:
-        song_str:
-          type: string
-          description: Full RTTTL string
-          example: "scale:d=4,o=5,b=100:c,d,e"
-          required: false
-        notes:
-          type: string
-          required: false
-        octave:
-          type: int
-          default: 5
-        bpm:
-          type: int
-          default: 100
-        tempo:
-          type: float
-          default: 1.5
-        loud:
-          type: bool
-          default: true
-        label:
-          type: string
-          default: ""
-      then:
-        - lambda: |-
-            char s[40], n[40], l[40];
-            snprintf(s, sizeof(s), "%.*s", (int) song_str.size(), song_str.c_str());
-            snprintf(n, sizeof(n), "%.*s", (int) notes.size(), notes.c_str());
-            snprintf(l, sizeof(l), "%.*s", (int) label.size(), label.c_str());
-            ESP_LOGD("main", "play_buzzer song=%s(%d) notes=%s(%d) octave=%d bpm=%d tempo=%.1f loud=%d label=%s(%d)", s,
-                     (int) song_str.size(), n, (int) notes.size(), octave, bpm, tempo, loud, l, (int) label.size());
-    - action: plain_action
-      variables:
-        value: int
-      then:
-        - logger.log:
-            format: "plain_action value=%d"
-            args: ["value"]
-    - action: list_action
-      variables:
-        values: int[]
-      then:
-        - logger.log:
-            format: "list_action n=%d"
-            args: ["values.size()"]

.agents/skills/hardware-test/templates/api-actions-strings-dut.yaml

@@ -1,28 +0,0 @@
-esphome:
-  name: ${name}
-
-esp8266:
-  board: d1_mini
-
-wifi:
-  ssid: !secret wifi_ssid
-  password: !secret wifi_password
-  power_save_mode: none
-
-logger:
-  level: DEBUG
-
-api:
-  reboot_timeout: 0s
-  actions:
-    - action: str_action
-      variables:
-        song_str: string
-        notes: string
-        octave: int
-      then:
-        - lambda: |-
-            char s[40], n[40];
-            snprintf(s, sizeof(s), "%.*s", (int) song_str.size(), song_str.c_str());
-            snprintf(n, sizeof(n), "%.*s", (int) notes.size(), notes.c_str());
-            ESP_LOGD("main", "str_action song=%s(%d) notes=%s(%d) octave=%d", s, (int) song_str.size(), n, (int) notes.size(), octave);

.agents/skills/hardware-test/templates/ld2412-select-only-dut.yaml

@@ -1,57 +0,0 @@
-substitutions:
-  name: ld2412sel
-
-esphome:
-  name: ${name}
-
-esp32:
-  variant: esp32s3
-  board: esp32-s3-devkitc-1
-  flash_size: 8MB
-  framework:
-    type: esp-idf
-
-logger:
-  level: VERBOSE
-
-api:
-  reboot_timeout: 0s
-
-ota:
-  - platform: esphome
-
-wifi:
-  ssid: !secret wifi_ssid
-  password: !secret wifi_password
-  power_save_mode: none
-
-uart:
-  - id: uart_ld2412
-    tx_pin: GPIO43
-    rx_pin: GPIO44
-    baud_rate: 115200
-    debug:
-      direction: BOTH
-      dummy_receiver: false
-      after:
-        timeout: 10ms
-        bytes: 120
-
-ld2412:
-  id: ld2412_radar
-  uart_id: uart_ld2412
-
-select:
-  - platform: ld2412
-    ld2412_id: ld2412_radar
-    out_pin_level:
-      name: "LD2412 Hardware output pin level"
-
-binary_sensor:
-  - platform: gpio
-    name: "LD2412 Out Pin"
-    pin: GPIO38
-  - platform: ld2412
-    ld2412_id: ld2412_radar
-    has_target:
-      name: "LD2412 Presence"

.agents/skills/hardware-test/templates/ld2450-loopback.yaml

@@ -1,258 +0,0 @@
-esphome:
-  name: ${name}
-  on_boot:
-    priority: -100
-    then:
-      - lambda: |-
-          // UART0 TX looped back into its own RX so injected frames reach the hub
-          USC0(0) |= (1 << UCLBE);
-          ESP_LOGI("harness", "UART0 loopback on");
-
-esp8266:
-  board: d1_mini
-
-wifi:
-  ssid: !secret wifi_ssid
-  password: !secret wifi_password
-
-api:
-  reboot_timeout: 0s
-
-logger:
-  baud_rate: 0
-  level: VERBOSE
-
-uart:
-  id: radar_uart
-  tx_pin: GPIO1
-  rx_pin: GPIO3
-  baud_rate: ${baud}
-  debug:
-    direction: TX
-    dummy_receiver: false
-    after:
-      delimiter: [0x04, 0x03, 0x02, 0x01]
-
-ld2450:
-  id: radar
-  uart_id: radar_uart
-
-select:
-  - platform: ld2450
-    baud_rate:
-      name: Baud Rate
-    zone_type:
-      name: Zone Type
-
-text_sensor:
-  - platform: ld2450
-    version:
-      name: Radar Version
-    target_1:
-      direction:
-        name: T1 Direction
-    target_2:
-      direction:
-        name: T2 Direction
-    target_3:
-      direction:
-        name: T3 Direction
-
-sensor:
-  - platform: ld2450
-    target_count:
-      name: Target Count
-
-button:
-  # periodic frames: header, 3 x (x, y, speed, resolution), footer
-  - platform: template
-    name: Frame Approach
-    on_press:
-      - uart.write:
-          id: radar_uart
-          data: [0xAA, 0xFF, 0x03, 0x00,
-                 0x00, 0x00, 0xE8, 0x83, 0x14, 0x00, 0x64, 0x00,
-                 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00,
-                 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00,
-                 0x55, 0xCC]
-  - platform: template
-    name: Frame Away
-    on_press:
-      - uart.write:
-          id: radar_uart
-          data: [0xAA, 0xFF, 0x03, 0x00,
-                 0x00, 0x00, 0xE8, 0x83, 0x14, 0x80, 0x64, 0x00,
-                 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00,
-                 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00,
-                 0x55, 0xCC]
-  - platform: template
-    name: Frame Stationary
-    on_press:
-      - uart.write:
-          id: radar_uart
-          data: [0xAA, 0xFF, 0x03, 0x00,
-                 0x00, 0x00, 0xE8, 0x83, 0x00, 0x00, 0x64, 0x00,
-                 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00,
-                 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00,
-                 0x55, 0xCC]
-  - platform: template
-    name: Frame None
-    on_press:
-      - uart.write:
-          id: radar_uart
-          data: [0xAA, 0xFF, 0x03, 0x00,
-                 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00,
-                 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00,
-                 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00,
-                 0x55, 0xCC]
-  - platform: template
-    name: Frame T2 T3
-    on_press:
-      - uart.write:
-          id: radar_uart
-          data: [0xAA, 0xFF, 0x03, 0x00,
-                 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00,
-                 0xF4, 0x81, 0x20, 0x83, 0x1E, 0x00, 0x64, 0x00,
-                 0x2C, 0x01, 0xB0, 0x84, 0x0A, 0x80, 0x64, 0x00,
-                 0x55, 0xCC]
-  # query zone acks: header, length, command C1, status 01, 00 00, zone type, 00, 24 coordinate bytes, footer
-  - platform: template
-    name: Zone Ack Disabled
-    on_press:
-      - uart.write:
-          id: radar_uart
-          data: [0xFD, 0xFC, 0xFB, 0xFA, 0x1E, 0x00, 0xC1, 0x01, 0x00, 0x00, 0x00, 0x00,
-                 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00,
-                 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00,
-                 0x04, 0x03, 0x02, 0x01]
-  - platform: template
-    name: Zone Ack Detection
-    on_press:
-      - uart.write:
-          id: radar_uart
-          data: [0xFD, 0xFC, 0xFB, 0xFA, 0x1E, 0x00, 0xC1, 0x01, 0x00, 0x00, 0x01, 0x00,
-                 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00,
-                 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00,
-                 0x04, 0x03, 0x02, 0x01]
-  - platform: template
-    name: Zone Ack Filter
-    on_press:
-      - uart.write:
-          id: radar_uart
-          data: [0xFD, 0xFC, 0xFB, 0xFA, 0x1E, 0x00, 0xC1, 0x01, 0x00, 0x00, 0x02, 0x00,
-                 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00,
-                 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00,
-                 0x04, 0x03, 0x02, 0x01]
-  # name based setters, the path lambdas use
-  - platform: template
-    name: Set Zone Filter By Name
-    on_press:
-      - lambda: id(radar).set_zone_type("Filter");
-  - platform: template
-    name: Set Zone Bogus By Name
-    on_press:
-      - lambda: id(radar).set_zone_type("Bogus");
-  - platform: template
-    name: Set Baud 57600 By Name
-    on_press:
-      - lambda: id(radar).set_baud_rate("57600");
-  - platform: template
-    name: Set Baud Bogus By Name
-    on_press:
-      - lambda: id(radar).set_baud_rate("Bogus");
-  # target 1 at x 772, y 258: bytes 04 03 02 01 are the command footer
-  - platform: template
-    name: Frame CmdFooter In Payload
-    on_press:
-      - uart.write:
-          id: radar_uart
-          data: [0xAA, 0xFF, 0x03, 0x00,
-                 0x04, 0x03, 0x02, 0x01, 0x00, 0x00, 0x64, 0x00,
-                 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00,
-                 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00,
-                 0x55, 0xCC]
-  # zone ack, type Detection, zone 1 x1 = 0xCC55: bytes 55 CC are the periodic footer
-  - platform: template
-    name: Zone Ack DataFooter In Payload
-    on_press:
-      - uart.write:
-          id: radar_uart
-          data: [0xFD, 0xFC, 0xFB, 0xFA, 0x1E, 0x00, 0xC1, 0x01, 0x00, 0x00, 0x01, 0x00,
-                 0x55, 0xCC, 0xC8, 0x00, 0xDC, 0x05, 0xB8, 0x0B, 0x00, 0x00, 0x00, 0x00,
-                 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00,
-                 0x04, 0x03, 0x02, 0x01]
-  # zone ack, type Filter, zone 1 x1 = 0x0304 and y1 = 0x0102: bytes 04 03 02 01 are the command footer
-  - platform: template
-    name: Zone Ack CmdFooter In Payload
-    on_press:
-      - uart.write:
-          id: radar_uart
-          data: [0xFD, 0xFC, 0xFB, 0xFA, 0x1E, 0x00, 0xC1, 0x01, 0x00, 0x00, 0x02, 0x00,
-                 0x04, 0x03, 0x02, 0x01, 0xDC, 0x05, 0xB8, 0x0B, 0x00, 0x00, 0x00, 0x00,
-                 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00,
-                 0x04, 0x03, 0x02, 0x01]
-  # a periodic frame cut after 12 bytes, then a complete zone ack (type Disabled, zeros)
-  - platform: template
-    name: Cut Frame Then Zone Ack
-    on_press:
-      - uart.write:
-          id: radar_uart
-          data: [0xAA, 0xFF, 0x03, 0x00, 0x00, 0x00, 0xE8, 0x83, 0x14, 0x00, 0x64, 0x00,
-                 0xFD, 0xFC, 0xFB, 0xFA, 0x1E, 0x00, 0xC1, 0x01, 0x00, 0x00, 0x00, 0x00,
-                 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00,
-                 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00,
-                 0x04, 0x03, 0x02, 0x01]
-  # a periodic frame missing its last byte, then a complete one (target 1 moving away)
-  - platform: template
-    name: Cut Frame Then Frame
-    on_press:
-      - uart.write:
-          id: radar_uart
-          data: [0xAA, 0xFF, 0x03, 0x00,
-                 0x00, 0x00, 0xE8, 0x83, 0x14, 0x00, 0x64, 0x00,
-                 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00,
-                 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00,
-                 0x55,
-                 0xAA, 0xFF, 0x03, 0x00,
-                 0x00, 0x00, 0xE8, 0x83, 0x14, 0x80, 0x64, 0x00,
-                 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00,
-                 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00,
-                 0x55, 0xCC]
-  # a command header claiming a 255 byte payload, then a complete periodic frame (target 1 stationary)
-  - platform: template
-    name: Bogus Length Then Frame
-    on_press:
-      - uart.write:
-          id: radar_uart
-          data: [0xFD, 0xFC, 0xFB, 0xFA, 0xFF, 0x00,
-                 0xAA, 0xFF, 0x03, 0x00,
-                 0x00, 0x00, 0xE8, 0x83, 0x00, 0x00, 0x64, 0x00,
-                 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00,
-                 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00,
-                 0x55, 0xCC]
-  # five garbage bytes, then a complete periodic frame (target 1 approaching)
-  - platform: template
-    name: Garbage Then Frame
-    on_press:
-      - uart.write:
-          id: radar_uart
-          data: [0x01, 0x02, 0x03, 0x42, 0x99,
-                 0xAA, 0xFF, 0x03, 0x00,
-                 0x00, 0x00, 0xE8, 0x83, 0x14, 0x00, 0x64, 0x00,
-                 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00,
-                 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00,
-                 0x55, 0xCC]
-
-number:
-  - platform: ld2450
-    presence_timeout:
-      name: Presence Timeout
-    zone_1:
-      x1:
-        name: Z1 X1
-      y1:
-        name: Z1 Y1
-      x2:
-        name: Z1 X2
-      y2:
-        name: Z1 Y2

.agents/skills/hardware-test/templates/wisp-client.yaml

@@ -1,57 +0,0 @@
-substitutions:
-  name: wispcli
-  ap_ssid: wisp-bridge
-
-esphome:
-  name: ${name}
-
-esp8266:
-  board: d1_mini
-
-logger:
-  level: DEBUG
-
-wifi:
-  ssid: ${ap_ssid}
-  password: !secret wisp_psk
-  power_save_mode: none
-  reboot_timeout: 0s
-
-http_request:
-  esp8266_disable_ssl_support: true
-  timeout: 8s
-
-time:
-  - platform: sntp
-    id: sntp_time
-    on_time_sync:
-      - logger.log:
-          format: "SNTP synced"
-          level: INFO
-
-interval:
-  - interval: 15s
-    then:
-      - http_request.get:
-          url: !secret lan_http
-          capture_response: true
-          on_response:
-            - logger.log:
-                format: "LAN GET status %d body %s"
-                args: ["response->status_code", "body.c_str()"]
-                level: INFO
-          on_error:
-            - logger.log:
-                format: "LAN GET failed"
-                level: WARN
-      - http_request.get:
-          url: http://example.com/
-          on_response:
-            - logger.log:
-                format: "WAN GET status %d"
-                args: ["response->status_code"]
-                level: INFO
-          on_error:
-            - logger.log:
-                format: "WAN GET failed"
-                level: WARN

.agents/skills/hardware-test/templates/wisp-ethernet-dut.yaml

@@ -1,40 +0,0 @@
-substitutions:
-  name: wispeth
-  ap_ssid: wisp-bridge
-
-esphome:
-  name: ${name}
-
-esp32:
-  board: esp32-s3-devkitc-1
-  variant: esp32s3
-  flash_size: 16MB
-  framework:
-    type: esp-idf
-
-logger:
-  hardware_uart: USB_SERIAL_JTAG
-  level: VERBOSE
-
-api:
-  encryption:
-    key: !secret api_key
-  reboot_timeout: 0s
-
-ota:
-  - platform: esphome
-
-ethernet:
-  type: W5500
-  clk_pin: GPIO13
-  mosi_pin: GPIO11
-  miso_pin: GPIO12
-  cs_pin: GPIO14
-  interrupt_pin: GPIO10
-  reset_pin: GPIO9
-
-wifi:
-  ap:
-    ssid: ${ap_ssid}
-    password: !secret wisp_psk
-    ap_mode: WISP

CODEOWNERS

@@ -700,6 +700,7 @@ esphome/components/veml7700/* @latonita
 esphome/components/version/* @esphome/core
 esphome/components/vevor_dtx/* @Bofr5555
 esphome/components/vl53l1x/* @will-tm
+esphome/components/vnc/* @MindFreeze
 esphome/components/voice_assistant/* @jesserockz @kahrendt
 esphome/components/wake_on_lan/* @clydebarrow @willwill2will54
 esphome/components/watchdog/* @oarcher

esphome/components/api/__init__.py

@@ -1,5 +1,7 @@
+from collections.abc import Callable
 from ipaddress import IPv4Address, IPv6Address
 import logging
+import math
 import re
 from typing import Any
 
@@ -27,6 +29,7 @@ from esphome.const import (
     CONF_CAPTURE_RESPONSE,
     CONF_DATA,
     CONF_DATA_TEMPLATE,
+    CONF_DEFAULT,
     CONF_DELAY,
     CONF_ENCRYPTION,
     CONF_EVENT,
@@ -136,6 +139,7 @@ SERVICE_ARG_FALLBACK_TYPES: dict[str, MockObj] = {
 CONF_BATCH_DELAY = "batch_delay"
 CONF_CUSTOM_SERVICES = "custom_services"
 CONF_EXAMPLE = "example"
+CONF_REQUIRED = "required"
 CONF_HOMEASSISTANT_SERVICES = "homeassistant_services"
 CONF_HOMEASSISTANT_STATES = "homeassistant_states"
 CONF_LISTEN_BACKLOG = "listen_backlog"
@@ -247,12 +251,66 @@ def _validate_supports_response(value: Any) -> str:
     return cv.enum(SUPPORTS_RESPONSE_OPTIONS, lower=True)(value)
 
 
-VARIABLE_SCHEMA = cv.Schema(
-    {
-        cv.Required(CONF_TYPE): cv.one_of(*SERVICE_ARG_NATIVE_TYPES, lower=True),
-        cv.Optional(CONF_DESCRIPTION): cv.string_strict,
-        cv.Optional(CONF_EXAMPLE): cv.string_strict,
-    }
+def _not_boolean(value: Any) -> Any:
+    # bool is an int subclass, and "true" cannot be parsed as a number by the client
+    if isinstance(value, bool):
+        raise cv.Invalid("Expected a number, got a boolean")
+    return value
+
+
+def _finite(value: float) -> float:
+    if not math.isfinite(value):
+        raise cv.Invalid("Expected a finite number")
+    return value
+
+
+def _no_nul(value: str) -> str:
+    # The default is a C string in flash, so a NUL would cut it short
+    if "\0" in value:
+        raise cv.Invalid("A default cannot contain a NUL character")
+    return value
+
+
+# The wire carries the default as text; the client parses it into ExecuteServiceArgument's
+# types, so the values are kept within what int32 and float hold
+FLOAT32_MAX = 3.4028235e38
+_DEFAULT_VALUE_VALIDATORS: dict[str, Callable[[Any], Any]] = {
+    "bool": cv.boolean,
+    "int": cv.All(_not_boolean, cv.int_range(min=-(2**31), max=2**31 - 1)),
+    "float": cv.All(
+        _not_boolean,
+        cv.float_,
+        _finite,
+        cv.float_range(min=-FLOAT32_MAX, max=FLOAT32_MAX),
+    ),
+    "string": cv.All(cv.string_strict, _no_nul),
+}
+assert _DEFAULT_VALUE_VALIDATORS.keys() == _SERVICE_ARG_SCALAR_TYPES.keys()
+
+
+def _validate_variable_optionality(value: ConfigType) -> ConfigType:
+    """Cross-validate required/default against the variable type."""
+    var_type = value[CONF_TYPE]
+    if _is_optional_var(value) and var_type not in _DEFAULT_VALUE_VALIDATORS:
+        raise cv.Invalid("Array variables cannot be optional")
+    if CONF_DEFAULT in value:
+        if value.get(CONF_REQUIRED) is True:
+            raise cv.Invalid("A variable with a default cannot set required: true")
+        value[CONF_DEFAULT] = _DEFAULT_VALUE_VALIDATORS[var_type](value[CONF_DEFAULT])
+    return value
+
+
+VARIABLE_SCHEMA = cv.All(
+    cv.Schema(
+        {
+            cv.Required(CONF_TYPE): cv.one_of(*SERVICE_ARG_NATIVE_TYPES, lower=True),
+            cv.Optional(CONF_DESCRIPTION): cv.string_strict,
+            cv.Optional(CONF_EXAMPLE): cv.string_strict,
+            cv.Optional(CONF_REQUIRED): cv.boolean,
+            cv.Optional(CONF_DEFAULT): cv.valid,
+        }
+    ),
+    _validate_variable_optionality,
 )
 
 # Accepts the plain `name: type` shorthand or the full mapping form
@@ -464,7 +522,28 @@ def _has_action_metadata(actions: list[ConfigType]) -> bool:
     )
 
 
-def _action_strings(conf: ConfigType, has_metadata: bool) -> list[str | None]:
+def _is_optional_var(var_: ConfigType) -> bool:
+    return CONF_DEFAULT in var_ or var_.get(CONF_REQUIRED) is False
+
+
+def _has_optional_args(actions: list[ConfigType]) -> bool:
+    return any(
+        _is_optional_var(var_)
+        for conf in actions
+        for var_ in conf[CONF_VARIABLES].values()
+    )
+
+
+def _default_to_wire(value: Any) -> str:
+    """Serialize a validated default for the wire; clients coerce by arg type."""
+    if isinstance(value, bool):
+        return "true" if value else "false"
+    return str(value)
+
+
+def _action_strings(
+    conf: ConfigType, has_metadata: bool, has_optional: bool
+) -> list[str | None]:
     """Strings of one action in the table order UserServiceStatic (user_services.h) expects."""
     # An empty description or example is treated as unset
     strings: list[str | None] = [conf[CONF_ACTION]]
@@ -477,6 +556,15 @@ def _action_strings(conf: ConfigType, has_metadata: bool) -> list[str | None]:
                 var_.get(CONF_DESCRIPTION) or None,
                 var_.get(CONF_EXAMPLE) or None,
             ]
+        if has_optional:
+            # The default slot doubles as the optional flag: nullptr for a required variable,
+            # "" for an optional one without a default (an empty string default is the same)
+            if not _is_optional_var(var_):
+                strings.append(None)
+            elif (default := var_.get(CONF_DEFAULT)) is None:
+                strings.append("")
+            else:
+                strings.append(_default_to_wire(default))
     return strings
 
 
@@ -485,9 +573,7 @@ def _validate_wizard(config: ConfigType) -> ConfigType:
     return config
 
 
-FINAL_VALIDATE_SCHEMA = cv.All(
-    _validate_wizard,
-)
+FINAL_VALIDATE_SCHEMA = _validate_wizard
 
 
 def _add_action_strings(index: int, strings: list[str | None]) -> MockObj:
@@ -546,6 +632,9 @@ async def to_code(config: ConfigType) -> None:
         has_metadata = _has_action_metadata(actions)
         if has_metadata:
             cg.add_define("USE_API_USER_DEFINED_ACTION_METADATA")
+        has_optional = _has_optional_args(actions)
+        if has_optional:
+            cg.add_define("USE_API_USER_DEFINED_ACTION_OPTIONAL_ARGS")
         # Collect all triggers first, then register all at once with initializer_list
         triggers: list[cg.MockObj] = []
         for index, conf in enumerate(actions):
@@ -589,7 +678,7 @@ async def to_code(config: ConfigType) -> None:
                     native = SERVICE_ARG_NATIVE_TYPES[var_type]
                 service_template_args.append(native)
                 func_args.append((native, name))
-            strings = _action_strings(conf, has_metadata)
+            strings = _action_strings(conf, has_metadata, has_optional)
             table = _add_action_strings(index, strings)
             # Template args: supports_response mode, then user service arg types
             templ = cg.TemplateArguments(supports_response, *service_template_args)

esphome/components/api/api.proto

@@ -608,6 +608,7 @@ message ListEntitiesCoverResponse {
   bool supports_stop = 12;
   uint32 device_id = 13 [(field_ifdef) = "USE_DEVICES"];
   bool supports_stop_tilt = 14;
+  bool supports_toggle = 15;
 }
 
 // Deprecated in API version 1.1
@@ -670,6 +671,7 @@ message CoverCommandRequest {
   bool stop = 8;
   uint32 device_id = 9 [(field_ifdef) = "USE_DEVICES"];
   bool stop_tilt = 10;
+  bool toggle = 11;
 }
 
 // ==================== FAN ====================
@@ -1172,6 +1174,10 @@ message ListEntitiesServicesArgument {
   ServiceArgType type = 2;
   string description = 3 [(field_ifdef) = "USE_API_USER_DEFINED_ACTION_METADATA", (progmem) = true];
   string example = 4 [(field_ifdef) = "USE_API_USER_DEFINED_ACTION_METADATA", (progmem) = true];
+  // Advisory: ExecuteServiceRequest still requires every argument
+  bool optional = 5 [(field_ifdef) = "USE_API_USER_DEFINED_ACTION_OPTIONAL_ARGS"];
+  // Always text: bool "true"/"false", numbers parseable as int/float; empty means no default
+  string default_value = 6 [(field_ifdef) = "USE_API_USER_DEFINED_ACTION_OPTIONAL_ARGS", (progmem) = true];
 }
 message ListEntitiesServicesResponse {
   option (id) = 41;
@@ -1783,6 +1789,8 @@ message MediaPlayerSupportedFormat {
   uint32 num_channels = 3;
   MediaPlayerFormatPurpose purpose = 4;
   uint32 sample_bytes = 5;
+  // Constant bitrate in kbps for lossy formats; 0 means no preference
+  uint32 bitrate = 6;
 }
 message ListEntitiesMediaPlayerResponse {
   option (id) = 63;

esphome/components/api/api_connection.cpp

@@ -555,6 +555,7 @@ uint16_t APIConnection::try_send_cover_info(EntityBase *entity, APIConnection *c
   msg.supports_tilt = traits.get_supports_tilt();
   msg.supports_stop = traits.get_supports_stop();
   msg.supports_stop_tilt = traits.get_supports_stop_tilt();
+  msg.supports_toggle = traits.get_supports_toggle();
   return fill_and_encode_entity_info_with_device_class(cover, msg, msg.device_class, conn, remaining_size);
 }
 void APIConnection::on_cover_command_request(const CoverCommandRequest &msg) {
@@ -567,6 +568,8 @@ void APIConnection::on_cover_command_request(const CoverCommandRequest &msg) {
     call.set_command_stop();
   if (msg.stop_tilt)
     call.set_command_stop_tilt();
+  if (msg.toggle)
+    call.set_command_toggle();
   call.perform();
 }
 #endif
@@ -1128,6 +1131,7 @@ uint16_t APIConnection::try_send_media_player_info(EntityBase *entity, APIConnec
     media_format.num_channels = supported_format.num_channels;
     media_format.purpose = static_cast<enums::MediaPlayerFormatPurpose>(supported_format.purpose);
     media_format.sample_bytes = supported_format.sample_bytes;
+    media_format.bitrate = supported_format.bitrate;
   }
   return fill_and_encode_entity_info(media_player, msg, conn, remaining_size);
 }

esphome/components/api/api_pb2.cpp

@@ -484,6 +484,7 @@ uint8_t *ListEntitiesCoverResponse::encode_msg(const void *self, uint8_t *__rest
   pos = ProtoEncode::encode_uint32(pos PROTO_ENCODE_DEBUG_ARG, 13, msg.device_id);
 #endif
   pos = ProtoEncode::encode_bool(pos PROTO_ENCODE_DEBUG_ARG, 14, msg.supports_stop_tilt);
+  pos = ProtoEncode::encode_bool(pos PROTO_ENCODE_DEBUG_ARG, 15, msg.supports_toggle);
   return pos;
 }
 uint32_t ListEntitiesCoverResponse::calc_size_msg(const void *self) {
@@ -506,6 +507,7 @@ uint32_t ListEntitiesCoverResponse::calc_size_msg(const void *self) {
   size += ProtoSize::calc_uint32(1, msg.device_id);
 #endif
   size += ProtoSize::calc_bool(1, msg.supports_stop_tilt);
+  size += ProtoSize::calc_bool(1, msg.supports_toggle);
   return size;
 }
 uint8_t *CoverStateResponse::encode_msg(const void *self, uint8_t *__restrict__ pos PROTO_ENCODE_DEBUG_PARAM) {
@@ -565,6 +567,9 @@ void CoverCommandRequest::decode_field(void *self, uint32_t tag, const uint8_t *
     case proto_tag(10, WIRE_TYPE_VARINT):
       msg.stop_tilt = value.as_bool();
       break;
+    case proto_tag(11, WIRE_TYPE_VARINT):
+      msg.toggle = value.as_bool();
+      break;
     default:
       break;
   }
@@ -1364,6 +1369,12 @@ uint8_t *ListEntitiesServicesArgument::encode_msg(const void *self,
 #endif
 #ifdef USE_API_USER_DEFINED_ACTION_METADATA
   pos = ProtoEncode::encode_progmem_string(pos PROTO_ENCODE_DEBUG_ARG, 4, msg.example);
+#endif
+#ifdef USE_API_USER_DEFINED_ACTION_OPTIONAL_ARGS
+  pos = ProtoEncode::encode_bool(pos PROTO_ENCODE_DEBUG_ARG, 5, msg.optional);
+#endif
+#ifdef USE_API_USER_DEFINED_ACTION_OPTIONAL_ARGS
+  pos = ProtoEncode::encode_progmem_string(pos PROTO_ENCODE_DEBUG_ARG, 6, msg.default_value);
 #endif
   return pos;
 }
@@ -1377,6 +1388,12 @@ uint32_t ListEntitiesServicesArgument::calc_size_msg(const void *self) {
 #endif
 #ifdef USE_API_USER_DEFINED_ACTION_METADATA
   size += ProtoSize::calc_length(1, msg.example.size());
+#endif
+#ifdef USE_API_USER_DEFINED_ACTION_OPTIONAL_ARGS
+  size += ProtoSize::calc_bool(1, msg.optional);
+#endif
+#ifdef USE_API_USER_DEFINED_ACTION_OPTIONAL_ARGS
+  size += ProtoSize::calc_length(1, msg.default_value.size());
 #endif
   return size;
 }
@@ -2368,6 +2385,7 @@ uint8_t *MediaPlayerSupportedFormat::encode_msg(const void *self, uint8_t *__res
   pos = ProtoEncode::encode_uint32(pos PROTO_ENCODE_DEBUG_ARG, 3, msg.num_channels);
   pos = ProtoEncode::encode_uint32(pos PROTO_ENCODE_DEBUG_ARG, 4, static_cast<uint32_t>(msg.purpose));
   pos = ProtoEncode::encode_uint32(pos PROTO_ENCODE_DEBUG_ARG, 5, msg.sample_bytes);
+  pos = ProtoEncode::encode_uint32(pos PROTO_ENCODE_DEBUG_ARG, 6, msg.bitrate);
   return pos;
 }
 uint32_t MediaPlayerSupportedFormat::calc_size_msg(const void *self) {
@@ -2378,6 +2396,7 @@ uint32_t MediaPlayerSupportedFormat::calc_size_msg(const void *self) {
   size += ProtoSize::calc_uint32(1, msg.num_channels);
   size += msg.purpose ? 2 : 0;
   size += ProtoSize::calc_uint32(1, msg.sample_bytes);
+  size += ProtoSize::calc_uint32(1, msg.bitrate);
   return size;
 }
 uint8_t *ListEntitiesMediaPlayerResponse::encode_msg(const void *self,

esphome/components/api/api_pb2.h

@@ -911,7 +911,7 @@ class BinarySensorStateResponse final : public StateResponseProtoMessage {
 class ListEntitiesCoverResponse final : public InfoResponseProtoMessage {
  public:
   static constexpr uint16_t MESSAGE_TYPE = 13;
-  static constexpr uint8_t ESTIMATED_SIZE = 59;
+  static constexpr uint8_t ESTIMATED_SIZE = 61;
 #ifdef HAS_PROTO_MESSAGE_DUMP
   const LogString *message_name() const override { return LOG_STR("list_entities_cover_response"); }
 #endif
@@ -921,6 +921,7 @@ class ListEntitiesCoverResponse final : public InfoResponseProtoMessage {
   ProgmemStringRef device_class{};
   bool supports_stop{false};
   bool supports_stop_tilt{false};
+  bool supports_toggle{false};
   static uint8_t *encode_msg(const void *self, uint8_t *pos PROTO_ENCODE_DEBUG_PARAM);
   uint8_t *encode(ProtoWriteBuffer &buffer PROTO_ENCODE_DEBUG_PARAM) const {
     return encode_msg(this, buffer.get_pos() PROTO_ENCODE_DEBUG_ARG);
@@ -958,7 +959,7 @@ class CoverStateResponse final : public StateResponseProtoMessage {
 class CoverCommandRequest final : public CommandProtoMessage {
  public:
   static constexpr uint16_t MESSAGE_TYPE = 30;
-  static constexpr uint8_t ESTIMATED_SIZE = 27;
+  static constexpr uint8_t ESTIMATED_SIZE = 29;
 #ifdef HAS_PROTO_MESSAGE_DUMP
   const LogString *message_name() const override { return LOG_STR("cover_command_request"); }
 #endif
@@ -968,6 +969,7 @@ class CoverCommandRequest final : public CommandProtoMessage {
   float tilt{0.0f};
   bool stop{false};
   bool stop_tilt{false};
+  bool toggle{false};
   void decode(const uint8_t *buffer, size_t length) {
     ProtoDecodableMessage::decode_fields(this, buffer, length, &decode_field);
   }
@@ -1591,6 +1593,12 @@ class ListEntitiesServicesArgument final : public ProtoMessage {
 #endif
 #ifdef USE_API_USER_DEFINED_ACTION_METADATA
   ProgmemStringRef example{};
+#endif
+#ifdef USE_API_USER_DEFINED_ACTION_OPTIONAL_ARGS
+  bool optional{false};
+#endif
+#ifdef USE_API_USER_DEFINED_ACTION_OPTIONAL_ARGS
+  ProgmemStringRef default_value{};
 #endif
   static uint8_t *encode_msg(const void *self, uint8_t *pos PROTO_ENCODE_DEBUG_PARAM);
   uint8_t *encode(ProtoWriteBuffer &buffer PROTO_ENCODE_DEBUG_PARAM) const {
@@ -2261,6 +2269,7 @@ class MediaPlayerSupportedFormat final : public ProtoMessage {
   uint32_t num_channels{0};
   enums::MediaPlayerFormatPurpose purpose{};
   uint32_t sample_bytes{0};
+  uint32_t bitrate{0};
   static uint8_t *encode_msg(const void *self, uint8_t *pos PROTO_ENCODE_DEBUG_PARAM);
   uint8_t *encode(ProtoWriteBuffer &buffer PROTO_ENCODE_DEBUG_PARAM) const {
     return encode_msg(this, buffer.get_pos() PROTO_ENCODE_DEBUG_ARG);

esphome/components/api/api_pb2_dump.cpp

@@ -1246,6 +1246,7 @@ const char *ListEntitiesCoverResponse::dump_to(DumpBuffer &out) const {
   dump_field(out, ESPHOME_PSTR("device_id"), this->device_id);
 #endif
   dump_field(out, ESPHOME_PSTR("supports_stop_tilt"), this->supports_stop_tilt);
+  dump_field(out, ESPHOME_PSTR("supports_toggle"), this->supports_toggle);
   return out.c_str();
 }
 const char *CoverStateResponse::dump_to(DumpBuffer &out) const {
@@ -1271,6 +1272,7 @@ const char *CoverCommandRequest::dump_to(DumpBuffer &out) const {
   dump_field(out, ESPHOME_PSTR("device_id"), this->device_id);
 #endif
   dump_field(out, ESPHOME_PSTR("stop_tilt"), this->stop_tilt);
+  dump_field(out, ESPHOME_PSTR("toggle"), this->toggle);
   return out.c_str();
 }
 #endif
@@ -1641,6 +1643,12 @@ const char *ListEntitiesServicesArgument::dump_to(DumpBuffer &out) const {
 #endif
 #ifdef USE_API_USER_DEFINED_ACTION_METADATA
   dump_progmem_string_field(out, ESPHOME_PSTR("example"), this->example);
+#endif
+#ifdef USE_API_USER_DEFINED_ACTION_OPTIONAL_ARGS
+  dump_field(out, ESPHOME_PSTR("optional"), this->optional);
+#endif
+#ifdef USE_API_USER_DEFINED_ACTION_OPTIONAL_ARGS
+  dump_progmem_string_field(out, ESPHOME_PSTR("default_value"), this->default_value);
 #endif
   return out.c_str();
 }
@@ -2097,6 +2105,7 @@ const char *MediaPlayerSupportedFormat::dump_to(DumpBuffer &out) const {
   dump_field(out, ESPHOME_PSTR("num_channels"), this->num_channels);
   dump_field(out, ESPHOME_PSTR("purpose"), static_cast<enums::MediaPlayerFormatPurpose>(this->purpose));
   dump_field(out, ESPHOME_PSTR("sample_bytes"), this->sample_bytes);
+  dump_field(out, ESPHOME_PSTR("bitrate"), this->bitrate);
   return out.c_str();
 }
 const char *ListEntitiesMediaPlayerResponse::dump_to(DumpBuffer &out) const {

esphome/components/api/user_services.cpp

@@ -31,6 +31,13 @@ ListEntitiesServicesResponse UserServiceStatic::encode_list_service_response_(
 #ifdef USE_API_USER_DEFINED_ACTION_METADATA
     arg.description = this->str_(base + 1);
     arg.example = this->str_(base + 2);
+#endif
+#ifdef USE_API_USER_DEFINED_ACTION_OPTIONAL_ARGS
+    // The default slot doubles as the optional flag: nullptr is required, "" optional without a default
+    const char *d = progmem_read_ptr(&this->strings_[base + USER_ACTION_ARG_BASE_STRINGS]);
+    arg.optional = d != nullptr;
+    if (d != nullptr)
+      arg.default_value = ProgmemStringRef(d, ESPHOME_strlen_P(d));
 #endif
   }
   return msg;

esphome/components/api/user_services.h

@@ -40,13 +40,20 @@ template<typename T> enums::ServiceArgType to_service_arg_type();
 // All strings live in one PROGMEM pointer table emitted by codegen (see _action_strings in
 // __init__.py), so each service costs a single pointer of RAM. Layout: the action name, then
 // each argument name; with USE_API_USER_DEFINED_ACTION_METADATA the action description follows
-// the name and every argument is (name, description, example). Unset metadata is nullptr.
+// the name and every argument is (name, description, example); with
+// USE_API_USER_DEFINED_ACTION_OPTIONAL_ARGS every argument ends with its default: nullptr for a
+// required argument, "" for an optional one without a default. Unset is nullptr.
 #ifdef USE_API_USER_DEFINED_ACTION_METADATA
 static constexpr size_t USER_ACTION_HEADER_STRINGS = 2;
-static constexpr size_t USER_ACTION_ARG_STRINGS = 3;
+static constexpr size_t USER_ACTION_ARG_BASE_STRINGS = 3;
 #else
 static constexpr size_t USER_ACTION_HEADER_STRINGS = 1;
-static constexpr size_t USER_ACTION_ARG_STRINGS = 1;
+static constexpr size_t USER_ACTION_ARG_BASE_STRINGS = 1;
+#endif
+#ifdef USE_API_USER_DEFINED_ACTION_OPTIONAL_ARGS
+static constexpr size_t USER_ACTION_ARG_STRINGS = USER_ACTION_ARG_BASE_STRINGS + 1;
+#else
+static constexpr size_t USER_ACTION_ARG_STRINGS = USER_ACTION_ARG_BASE_STRINGS;
 #endif
 class UserServiceStatic : public UserServiceDescriptor {
  public:

esphome/components/ethernet/__init__.py

@@ -46,6 +46,7 @@ from esphome.const import (
     CONF_TYPE,
     CONF_USE_ADDRESS,
     CONF_VALUE,
+    CONF_WIFI,
     KEY_CORE,
     KEY_FRAMEWORK_VERSION,
     Platform,
@@ -562,6 +563,25 @@ CONFIG_SCHEMA = cv.All(
 )
 
 
+def _final_validate_wifi(config: ConfigType) -> None:
+    """A wifi station competes with ethernet for the primary interface, so both must be in network: priority:."""
+    full_config = fv.full_config.get()
+    if (wifi_config := full_config.get(CONF_WIFI)) is None:
+        return
+
+    from esphome.components.wifi import wifi_stays_ap_only
+
+    if wifi_stays_ap_only(wifi_config, full_config):
+        return  # an access point does not claim the primary interface
+
+    priority_ifaces = get_priority_interfaces_from_full_config(full_config)
+    if missing := [i for i in (DOMAIN, CONF_WIFI) if i not in priority_ifaces]:
+        raise cv.Invalid(
+            "When ethernet and a wifi station are used together, 'network: priority:' "
+            f"must list both interfaces; missing: {', '.join(missing)}"
+        )
+
+
 def _final_validate_spi(config: ConfigType) -> None:
     if not CORE.is_esp32:
         return  # SPI interface validation is ESP32-only
@@ -846,22 +866,7 @@ def _final_validate_rmii_pins(config: ConfigType) -> None:
 
 def _final_validate(config: ConfigType) -> None:
     """Final validation for Ethernet component."""
-    # Allow ethernet + wifi coexistence only when both are declared in network: priority:.
-    if "wifi" in fv.full_config.get():
-        priority_ifaces = get_priority_interfaces_from_full_config(fv.full_config.get())
-        missing = [i for i in ("ethernet", "wifi") if i not in priority_ifaces]
-        if missing and priority_ifaces:
-            # A priority list exists but is incomplete: point at what to add.
-            raise cv.Invalid(
-                "When ethernet and wifi are used together, 'network: priority:' must "
-                f"list both interfaces; missing: {', '.join(missing)}"
-            )
-        if missing:
-            raise cv.Invalid(
-                "Component ethernet cannot be used together with component wifi "
-                "unless both are listed under 'network: priority:'"
-            )
-
+    _final_validate_wifi(config)
     _final_validate_spi(config)
     _final_validate_rmii_pins(config)
 

esphome/components/light/light_call.h

@@ -1,6 +1,7 @@
 #pragma once
 
 #include "light_color_values.h"
+#include "esphome/core/string_ref.h"
 
 namespace esphome {
 
@@ -133,6 +134,8 @@ class LightCall {
   LightCall &set_effect(const std::string &effect) { return this->set_effect(effect.data(), effect.size()); }
   /// Set the effect of the light by its name (const char * overload to resolve ambiguity).
   LightCall &set_effect(const char *effect) { return this->set_effect(effect, strlen(effect)); }
+  /// Set the effect of the light by its name (StringRef overload to resolve ambiguity, zero-copy).
+  LightCall &set_effect(StringRef effect) { return this->set_effect(effect.c_str(), effect.size()); }
   /// Set the effect of the light by its name and length (zero-copy from API).
   LightCall &set_effect(const char *effect, size_t len);
   /// Set the effect of the light by its internal index number (only for internal use).

esphome/components/media_player/__init__.py

@@ -3,6 +3,7 @@ from collections.abc import Callable
 from esphome import automation
 import esphome.codegen as cg
 from esphome.components import audio
+from esphome.components.const import CONF_BITRATE
 import esphome.config_validation as cv
 from esphome.const import (
     CONF_ENTITY_CATEGORY,
@@ -55,6 +56,30 @@ FORMAT_MAPPING = {
 }
 
 
+# MP3 constant bitrates in kbps; the encoder rounds anything else to the nearest one
+MP3_BITRATES = (
+    8,
+    16,
+    24,
+    32,
+    40,
+    48,
+    56,
+    64,
+    80,
+    96,
+    112,
+    128,
+    144,  # MPEG-2 only, at 16, 22.05 and 24 kHz
+    160,
+    192,
+    224,
+    256,
+    320,
+)
+validate_mp3_bitrate = cv.one_of(*MP3_BITRATES, int=True)
+
+
 def build_supported_format_struct(
     format_config: ConfigType, purpose: MockObj
 ) -> cg.StructInitializer:
@@ -75,6 +100,10 @@ def build_supported_format_struct(
     if format_config[CONF_FORMAT] != "MP3":
         args.append(("sample_bytes", 2))
 
+    # Omitted means no preference, and the client keeps encoding as it always has.
+    if bitrate := format_config.get(CONF_BITRATE):
+        args.append(("bitrate", bitrate))
+
     return cg.StructInitializer(*args)
 
 
@@ -91,6 +120,14 @@ def validate_preferred_format(
         inherit_property_from(CONF_NUM_CHANNELS, audio_device_key)(config)
         inherit_property_from(CONF_SAMPLE_RATE, audio_device_key)(config)
 
+        # A constant bitrate is only meaningful for the one lossy format whose
+        # encoder Home Assistant can be told to fix
+        if CONF_BITRATE in config and config.get(CONF_FORMAT) != "MP3":
+            raise cv.Invalid(
+                f"{CONF_BITRATE} is only supported for the MP3 format",
+                path=[CONF_BITRATE],
+            )
+
         # Opus only supports 48 kHz
         if config.get(CONF_FORMAT) == "OPUS" and config.get(CONF_SAMPLE_RATE) != 48000:
             raise cv.Invalid("Opus only supports a sample rate of 48000 Hz")

esphome/components/media_player/media_player.h

@@ -77,6 +77,7 @@ struct MediaPlayerSupportedFormat {
   uint32_t num_channels;
   MediaPlayerFormatPurpose purpose;
   uint32_t sample_bytes;
+  uint32_t bitrate;
 };
 
 // Base features always reported for all media players

esphome/components/mipi_spi/models/lilygo.py

@@ -52,6 +52,7 @@ ST7796.extend(
     height=480,
     offset_width=49,
     draw_rounding=1,
+    data_rate="40MHz",
     cs_pin=39,
     reset_pin=47,
     dc_pin=9,

esphome/components/speaker/media_player/__init__.py

@@ -13,6 +13,7 @@ from esphome.components import (
 )
 from esphome.components.const import (
     CONF_ANNOUNCEMENT_VOLUME,
+    CONF_BITRATE,
     CONF_VOLUME_INCREMENT,
     CONF_VOLUME_INITIAL,
     CONF_VOLUME_MAX,
@@ -107,6 +108,7 @@ PIPELINE_SCHEMA = cv.Schema(
         cv.Optional(CONF_FORMAT, default="FLAC"): cv.enum(audio.AUDIO_FILE_TYPE_ENUM),
         cv.Optional(CONF_SAMPLE_RATE): cv.int_range(min=1),
         cv.Optional(CONF_NUM_CHANNELS): cv.int_range(1, 2),
+        cv.Optional(CONF_BITRATE): media_player.validate_mp3_bitrate,
     }
 )
 

esphome/components/speaker_source/media_player.py

@@ -3,6 +3,7 @@ import esphome.codegen as cg
 from esphome.components import audio, media_player, media_source, speaker
 from esphome.components.const import (
     CONF_ANNOUNCEMENT_VOLUME,
+    CONF_BITRATE,
     CONF_SOURCES,
     CONF_VOLUME_INCREMENT,
     CONF_VOLUME_INITIAL,
@@ -78,6 +79,7 @@ PIPELINE_SCHEMA = cv.Schema(
         cv.Optional(CONF_FORMAT, default="FLAC"): cv.enum(audio.AUDIO_FILE_TYPE_ENUM),
         cv.Optional(CONF_SAMPLE_RATE): cv.int_range(min=1),
         cv.Optional(CONF_NUM_CHANNELS): cv.int_range(1, 2),
+        cv.Optional(CONF_BITRATE): media_player.validate_mp3_bitrate,
     }
 )
 

esphome/components/vnc/__init__.py

@@ -0,0 +1,138 @@
+"""VNC (RFB) client that shows a remote screen on a display."""
+
+from esphome import automation
+import esphome.codegen as cg
+from esphome.components import display, network, socket
+from esphome.components.const import (
+    BYTE_ORDER_LITTLE,
+    CONF_HOST,
+    CONF_RECONNECT_INTERVAL,
+)
+import esphome.config_validation as cv
+from esphome.const import (
+    CONF_AUTO_CLEAR_ENABLED,
+    CONF_DISPLAY_ID,
+    CONF_ID,
+    CONF_ON_CONNECT,
+    CONF_ON_DISCONNECT,
+    CONF_PLATFORM,
+    CONF_PORT,
+)
+import esphome.final_validate as fv
+from esphome.types import ConfigType
+
+CODEOWNERS = ["@MindFreeze"]
+DEPENDENCIES = ["display", "network"]
+AUTO_LOAD = ["socket"]
+DOMAIN = "vnc"
+MULTI_CONF = True
+
+CONF_MAX_UPDATE_DELAY = "max_update_delay"
+CONF_UPDATE_DELAY = "update_delay"
+
+vnc_ns = cg.esphome_ns.namespace("vnc")
+VncClient = vnc_ns.class_("VncClient", cg.Component)
+
+# Drivers that cannot take another component's pixels yet.
+UNSUPPORTED_DISPLAYS = dict.fromkeys(
+    ("mipi_dsi", "mipi_rgb", "mipi_spi"), "it is not supported by vnc yet"
+) | {
+    "waveshare_epaper": "it draws every colour except pure black as ink; use epaper_spi"
+}
+
+
+def _register_writer(config: ConfigType) -> ConfigType:
+    display.register_external_writer(config[CONF_DISPLAY_ID], DOMAIN)
+    return config
+
+
+CONFIG_SCHEMA = cv.All(
+    cv.Schema(
+        {
+            cv.GenerateID(): cv.declare_id(VncClient),
+            cv.GenerateID(CONF_DISPLAY_ID): cv.use_id(display.Display),
+            cv.Required(CONF_HOST): socket.ipv4_host,
+            cv.Optional(CONF_PORT, default=5900): cv.port,
+            cv.Optional(CONF_RECONNECT_INTERVAL, default="5s"): cv.All(
+                cv.positive_time_period_milliseconds,
+                cv.Range(min=cv.TimePeriod(seconds=1)),
+            ),
+            cv.Optional(
+                CONF_UPDATE_DELAY, default="200ms"
+            ): cv.positive_time_period_milliseconds,
+            cv.Optional(
+                CONF_MAX_UPDATE_DELAY, default="1s"
+            ): cv.positive_time_period_milliseconds,
+            cv.Optional(CONF_ON_CONNECT): automation.validate_automation({}),
+            cv.Optional(CONF_ON_DISCONNECT): automation.validate_automation({}),
+        }
+    ).extend(cv.COMPONENT_SCHEMA),
+    _register_writer,
+    socket.consume_sockets(1, DOMAIN),
+    network.require_ipv4(DOMAIN),
+)
+
+
+def _final_validate(config: ConfigType) -> None:
+    full_config = fv.full_config.get()
+    disp_id = config[CONF_DISPLAY_ID]
+    disp_conf = fv.get_declaring_config(disp_id)
+
+    if (platform := disp_conf.get(CONF_PLATFORM)) in UNSUPPORTED_DISPLAYS:
+        raise cv.Invalid(
+            f"Display '{disp_id}' uses the {platform} platform: "
+            f"{UNSUPPORTED_DISPLAYS[platform]}",
+            path=[CONF_DISPLAY_ID],
+        )
+    if (
+        display.get_display_metadata(disp_id).has_writer
+        or disp_conf.get(CONF_AUTO_CLEAR_ENABLED) is True
+    ):
+        raise cv.Invalid(
+            f"vnc draws the whole of display '{disp_id}'; remove lambda, pages, "
+            "auto_clear_enabled and show_test_card from it",
+            path=[CONF_DISPLAY_ID],
+        )
+    if len(writers := display.get_external_writers(disp_id)) > 1:
+        raise cv.Invalid(
+            f"Display '{disp_id}' is drawn by more than one component: {', '.join(writers)}",
+            path=[CONF_DISPLAY_ID],
+        )
+    lvgl_configs = full_config.get("lvgl") or []
+    if isinstance(lvgl_configs, dict):
+        lvgl_configs = [lvgl_configs]
+    for lvgl_config in lvgl_configs:
+        if any(d.id == disp_id.id for d in lvgl_config.get("displays", [])):
+            raise cv.Invalid(
+                f"Display '{disp_id}' is also used by LVGL", path=[CONF_DISPLAY_ID]
+            )
+
+
+FINAL_VALIDATE_SCHEMA = _final_validate
+
+_CALLBACK_AUTOMATIONS = (
+    automation.CallbackAutomation(CONF_ON_CONNECT, "add_on_connect_callback"),
+    automation.CallbackAutomation(CONF_ON_DISCONNECT, "add_on_disconnect_callback"),
+)
+
+
+async def to_code(config: ConfigType) -> None:
+    disp = await cg.get_variable(config[CONF_DISPLAY_ID])
+    # Ask the server for the display's own byte order, so block-copying drivers skip the per-pixel path
+    meta = display.get_display_metadata(config[CONF_DISPLAY_ID])
+    var = cg.new_Pvariable(config[CONF_ID], disp, meta.byte_order != BYTE_ORDER_LITTLE)
+    await cg.register_component(var, config)
+    socket.require_tcp_client_link()
+    cg.add(var.set_host(config[CONF_HOST]))
+    cg.add(var.set_port(config[CONF_PORT]))
+    cg.add(var.set_reconnect_interval(config[CONF_RECONNECT_INTERVAL]))
+    cg.add(var.set_update_delay(config[CONF_UPDATE_DELAY]))
+    cg.add(var.set_max_update_delay(config[CONF_MAX_UPDATE_DELAY]))
+    await automation.build_callback_automations(var, config, _CALLBACK_AUTOMATIONS)
+
+
+VNC_ID_SCHEMA = automation.maybe_simple_id(
+    {cv.GenerateID(): cv.use_id(VncClient)},
+)
+
+automation.register_apply_condition("vnc.is_connected", VNC_ID_SCHEMA, "is_connected()")

esphome/components/vnc/vnc_client.cpp

@@ -0,0 +1,702 @@
+#include "vnc_client.h"
+
+#include "esphome/components/network/util.h"
+#include "esphome/core/application.h"
+#include "esphome/core/hal.h"
+#include "esphome/core/log.h"
+
+#include <algorithm>
+#include <cinttypes>
+#include <cstring>
+
+namespace esphome::vnc {
+
+ESPHOME_LOG_TAG(TAG, "vnc");
+
+// Below the 50 ms blocking warning, and the same slice it8951 uses for its transfers.
+static constexpr uint32_t LOOP_BUDGET_MS = 20;
+static constexpr uint32_t HANDSHAKE_TIMEOUT_MS = 15000;
+static constexpr uint32_t STABLE_SESSION_MS = 60000;
+static constexpr uint32_t MAX_BACKOFF_MS = 60000;
+static constexpr uint32_t CONFIG_RETRY_MS = 300000;
+static constexpr uint32_t HEARTBEAT_MS = 60000;
+static constexpr uint32_t MIN_REQUEST_SPACING_MS = 50;
+// The longest the first refresh waits for a display with a loop to park once after boot.
+static constexpr uint32_t DISPLAY_START_MS = 5000;
+// Longer than any driver's own busy timeout: a panel that stays busy this long is stuck
+static constexpr uint32_t DISPLAY_BUSY_TIMEOUT_MS = 120000;
+static constexpr uint16_t MAX_CURSOR_SIDE = 1024;
+static constexpr uint32_t MAX_TEXT_LEN = 63;
+
+static constexpr uint8_t SECURITY_NONE = 1;
+static constexpr uint8_t SECURITY_VNC_AUTH = 2;
+
+static constexpr int32_t ENCODING_RAW = 0;
+static constexpr int32_t ENCODING_RICH_CURSOR = -239;
+
+static constexpr uint8_t MSG_FRAMEBUFFER_UPDATE = 0;
+static constexpr uint8_t MSG_SET_COLOUR_MAP = 1;
+static constexpr uint8_t MSG_BELL = 2;
+static constexpr uint8_t MSG_SERVER_CUT_TEXT = 3;
+
+static inline uint16_t read_be16(const uint8_t *p) { return encode_uint16(p[0], p[1]); }
+static inline uint32_t read_be32(const uint8_t *p) { return encode_uint32(p[0], p[1], p[2], p[3]); }
+static inline void write_be16(uint8_t *p, uint16_t v) {
+  p[0] = v >> 8;
+  p[1] = v;
+}
+static inline void write_be32(uint8_t *p, uint32_t v) {
+  p[0] = v >> 24;
+  p[1] = v >> 16;
+  p[2] = v >> 8;
+  p[3] = v;
+}
+
+static int parse_three_digits(const uint8_t *p) {
+  int value = 0;
+  for (int i = 0; i < 3; i++) {
+    if (p[i] < '0' || p[i] > '9') {
+      return -1;
+    }
+    value = value * 10 + (p[i] - '0');
+  }
+  return value;
+}
+
+void VncClient::setup() {
+  this->setup_ms_ = App.get_loop_component_start_time();
+  this->link_.begin(TAG);
+}
+
+void VncClient::dump_config() {
+  ESP_LOGCONFIG(TAG,
+                "VNC:\n"
+                "  Server: %s:%u\n"
+                "  Reconnect Interval: %" PRIu32 "ms\n"
+                "  Update Delay: %" PRIu32 "ms\n"
+                "  Max Update Delay: %" PRIu32 "ms",
+                this->link_.host(), this->link_.port(), this->link_.reconnect_interval(), this->update_delay_ms_,
+                this->max_update_delay_ms_);
+}
+
+void VncClient::on_shutdown() { this->link_.close(); }
+
+void VncClient::loop() {
+  const uint32_t now = App.get_loop_component_start_time();
+  this->pass_start_ms_ = millis();
+  if (this->display_->is_failed()) {
+    this->link_.close();  // a connect still in flight would otherwise keep its socket for good
+    if (this->state_ != VncState::VNC_STATE_IDLE) {
+      this->session_end_(now, VncFailure::VNC_FAILURE_TRANSIENT, LOG_STR("Display failed"));
+    }
+    this->status_set_error(LOG_STR("Display failed"));
+    this->disable_loop();
+    return;
+  }
+  // A display that parks its loop when idle is busy while that loop runs. Displays that never
+  // park (no loop, or one that always runs) update synchronously, so they are never busy.
+  if (this->display_->is_idle()) {
+    this->display_parks_ = true;
+    this->display_busy_ = false;
+  } else {
+    this->display_busy_ = this->display_parks_;
+  }
+  // A stuck panel, or another owner of the display's loop, would otherwise freeze the screen silently
+  if (!this->display_busy_) {
+    this->busy_since_ms_ = now;
+    if (this->busy_warned_) {
+      this->busy_warned_ = false;
+      // The connection path owns the warning while no session runs
+      if (this->state_ == VncState::VNC_STATE_RUNNING) {
+        this->status_clear_warning();
+      }
+    }
+  } else if (!this->busy_warned_ && now - this->busy_since_ms_ >= DISPLAY_BUSY_TIMEOUT_MS) {
+    this->busy_warned_ = true;
+    this->status_set_warning(LOG_STR("Display busy too long"));
+  }
+
+  if (!this->link_.connected()) {
+    if (this->state_ != VncState::VNC_STATE_IDLE) {
+      this->session_lost_(now);
+    }
+    this->connect_step_(now);
+    if (!this->link_.connected()) {
+      return;
+    }
+    this->start_handshake_(now);
+  }
+  if (this->state_ == VncState::VNC_STATE_HANDSHAKE && now - this->session_start_ms_ >= HANDSHAKE_TIMEOUT_MS) {
+    this->fail_(this->lost_failure_(), LOG_STR("Handshake timed out"));
+    return;
+  }
+
+  const bool blocked = this->in_update_ && this->display_busy_;
+  // A display still starting up (it8951) can go from its start-up straight into a refresh without parking.
+  const bool display_starting =
+      !this->display_parks_ && this->display_->has_overridden_loop() && now - this->setup_ms_ < DISPLAY_START_MS;
+  // Wait until the screen stops changing, so an animation becomes one refresh showing where it ended.
+  if (this->refresh_pending_ && !this->in_update_ && !this->display_busy_ && !display_starting &&
+      (now - this->last_change_ms_ >= this->update_delay_ms_ ||
+       now - this->first_change_ms_ >= this->max_update_delay_ms_)) {
+    // Synchronous displays flush inside update(), so this pass does nothing else.
+    this->refresh_pending_ = false;
+    this->display_->update();
+    this->display_busy_ = this->display_parks_ && !this->display_->is_idle();
+  } else if (!blocked && (this->rx_pending_ || this->link_.ready())) {
+    this->service_rx_(now);
+  }
+  if (this->state_ == VncState::VNC_STATE_RUNNING) {
+    // Mid-frame there is nothing to send, so a vanished server would never fail a write.
+    if (this->in_update_ && !blocked && !this->rx_pending_ && now - this->last_rx_ms_ >= HEARTBEAT_MS) {
+      this->fail_(VncFailure::VNC_FAILURE_TRANSIENT, LOG_STR("Server stopped sending"));
+      return;
+    }
+    this->pace_(now);
+  }
+  this->link_.flush_tx();
+  if (this->rx_pending_ && !(this->in_update_ && this->display_busy_)) {
+    this->high_freq_.start();
+  } else {
+    this->high_freq_.stop();
+  }
+}
+
+void VncClient::connect_step_(uint32_t now) {
+  const bool up = network::is_connected();
+  if (up && !this->network_was_up_) {
+    this->backoff_ms_ = 0;
+    this->hold_ms_ = 0;
+  }
+  this->network_was_up_ = up;
+  if (!up) {
+    return;
+  }
+  if (!this->link_.connecting() && (now - this->hold_start_ms_ < this->hold_ms_ || this->link_.in_backoff())) {
+    return;
+  }
+  this->link_.poll();
+  // DNS failures, refused connects and connect timeouts all leave no socket and start the link's backoff.
+  if (!this->link_.connected() && !this->link_.connecting() && this->link_.in_backoff()) {
+    this->session_end_(now, VncFailure::VNC_FAILURE_TRANSIENT, LOG_STR("Cannot reach the server"));
+  }
+}
+
+void VncClient::reset_session_() {
+  this->rx_pos_ = 0;
+  this->rx_len_ = 0;
+  this->rx_pending_ = false;
+  this->in_update_ = false;
+  this->request_outstanding_ = false;
+  this->refresh_pending_ = false;
+  this->frame_drawn_ = false;
+  this->step_ = VncStep::VNC_STEP_VERSION;
+  this->reason_failure_ = VncFailure::VNC_FAILURE_TRANSIENT;
+  this->high_freq_.stop();
+}
+
+void VncClient::start_handshake_(uint32_t now) {
+  this->reset_session_();
+  this->state_ = VncState::VNC_STATE_HANDSHAKE;
+  this->session_start_ms_ = now;
+  this->last_rx_ms_ = now;
+}
+
+VncFailure VncClient::lost_failure_() const {
+  if (this->state_ == VncState::VNC_STATE_HANDSHAKE) {
+    if (this->step_ == VncStep::VNC_STEP_REASON_LENGTH || this->step_ == VncStep::VNC_STEP_REASON) {
+      return this->reason_failure_;
+    }
+  }
+  return VncFailure::VNC_FAILURE_TRANSIENT;
+}
+
+void VncClient::session_lost_(uint32_t now) {
+  this->session_end_(now, this->lost_failure_(), LOG_STR("Connection lost"));
+}
+
+VncParse VncClient::fail_(VncFailure failure, const LogString *reason) {
+  this->link_.close();
+  this->link_.note_attempt();
+  this->session_end_(App.get_loop_component_start_time(), failure, reason);
+  return VncParse::VNC_PARSE_STOP;
+}
+
+void VncClient::session_end_(uint32_t now, VncFailure failure, const LogString *reason) {
+  const bool was_running = this->state_ == VncState::VNC_STATE_RUNNING;
+  this->reset_session_();
+  this->state_ = VncState::VNC_STATE_IDLE;
+  this->hold_start_ms_ = now;
+  if (was_running) {
+    const uint32_t lasted = now - this->session_start_ms_;
+    ESP_LOGW(TAG, "%s after %" PRIu32 "s", LOG_STR_ARG(reason), lasted / 1000);
+    this->disconnect_callback_.call();
+    this->status_set_warning();
+    if (lasted >= STABLE_SESSION_MS) {
+      this->backoff_ms_ = 0;
+      this->hold_ms_ = 0;
+      return;
+    }
+    failure = VncFailure::VNC_FAILURE_TRANSIENT;
+  }
+  switch (failure) {
+    case VncFailure::VNC_FAILURE_CONFIG:
+      this->hold_ms_ = CONFIG_RETRY_MS;
+      ESP_LOGE(TAG, "%s; retrying in %" PRIu32 "s", LOG_STR_ARG(reason), this->hold_ms_ / 1000);
+      this->status_set_error(reason);
+      return;
+    default: {
+      const uint32_t base = this->link_.reconnect_interval();
+      this->backoff_ms_ =
+          this->backoff_ms_ == 0 ? base : std::min(this->backoff_ms_ * 2, std::max(MAX_BACKOFF_MS, base));
+      this->hold_ms_ = this->backoff_ms_;
+      if (!was_running) {
+        ESP_LOGW(TAG, "%s; retrying in %" PRIu32 "s", LOG_STR_ARG(reason), this->hold_ms_ / 1000);
+      }
+      this->status_set_warning();
+      return;
+    }
+  }
+}
+
+bool VncClient::budget_exhausted_() const { return millis() - this->pass_start_ms_ >= LOOP_BUDGET_MS; }
+
+void VncClient::service_rx_(uint32_t now) {
+  for (;;) {
+    VncParse result = this->parse_();
+    if (result == VncParse::VNC_PARSE_STOP) {
+      return;
+    }
+    if (result == VncParse::VNC_PARSE_PAUSE) {
+      this->rx_pending_ = true;
+      return;
+    }
+    // Less than one item is left, so the buffer always has room after compacting.
+    if (this->rx_pos_ != 0) {
+      this->rx_len_ -= this->rx_pos_;
+      std::memmove(this->rx_, this->rx_ + this->rx_pos_, this->rx_len_);
+      this->rx_pos_ = 0;
+    }
+    if (this->budget_exhausted_()) {
+      this->rx_pending_ = true;
+      return;
+    }
+    ssize_t count = this->link_.read(this->rx_ + this->rx_len_, RX_BUFFER_SIZE - this->rx_len_);
+    if (count < 0) {
+      this->session_lost_(now);
+      return;
+    }
+    if (count == 0) {
+      this->rx_pending_ = false;
+      return;
+    }
+    this->rx_len_ += static_cast<uint16_t>(count);
+    this->rx_total_ += count;
+    this->last_rx_ms_ = now;
+  }
+}
+
+void VncClient::skip_(uint32_t count, VncStep next) {
+  this->skip_left_ = count;
+  this->after_skip_ = next;
+  this->step_ = VncStep::VNC_STEP_SKIP;
+}
+
+bool VncClient::send_client_init_() {
+  const uint8_t shared = 1;
+  return this->send_(&shared, 1);
+}
+
+VncParse VncClient::parse_() {
+  for (;;) {
+    const uint8_t *p = this->rx_ + this->rx_pos_;
+    const size_t avail = this->rx_len_ - this->rx_pos_;
+    switch (this->step_) {
+      case VncStep::VNC_STEP_VERSION: {
+        if (avail < 12) {
+          return VncParse::VNC_PARSE_MORE;
+        }
+        const int major = parse_three_digits(p + 4);
+        const int minor = parse_three_digits(p + 8);
+        if (std::memcmp(p, "RFB ", 4) != 0 || p[7] != '.' || p[11] != '\n' || major < 0 || minor < 0) {
+          return this->fail_(VncFailure::VNC_FAILURE_CONFIG, LOG_STR("Not a VNC server"));
+        }
+        if (major < 3) {
+          return this->fail_(VncFailure::VNC_FAILURE_CONFIG, LOG_STR("Unsupported RFB version"));
+        }
+        this->minor_ = (major > 3 || minor >= 8) ? 8 : (minor == 7 ? 7 : 3);
+        this->rx_pos_ += 12;
+        uint8_t reply[12] = {'R', 'F', 'B', ' ', '0', '0', '3', '.', '0', '0', '8', '\n'};
+        reply[10] = '0' + this->minor_;
+        if (!this->send_(reply, sizeof(reply))) {
+          return this->fail_send_();
+        }
+        this->step_ = this->minor_ == 3 ? VncStep::VNC_STEP_SECURITY_TYPE_33 : VncStep::VNC_STEP_SECURITY_COUNT;
+        break;
+      }
+      case VncStep::VNC_STEP_SECURITY_COUNT: {
+        if (avail < 1) {
+          return VncParse::VNC_PARSE_MORE;
+        }
+        this->sec_count_ = p[0];
+        this->rx_pos_ += 1;
+        if (this->sec_count_ == 0) {
+          // A refusal before authentication, such as a lockout caused by another client.
+          this->step_ = VncStep::VNC_STEP_REASON_LENGTH;
+        } else {
+          this->step_ = VncStep::VNC_STEP_SECURITY_LIST;
+        }
+        break;
+      }
+      case VncStep::VNC_STEP_SECURITY_LIST: {
+        if (avail < this->sec_count_) {
+          return VncParse::VNC_PARSE_MORE;
+        }
+        uint8_t chosen = 0;
+        bool wants_password = false;
+        for (uint8_t i = 0; i < this->sec_count_; i++) {
+          if (p[i] == SECURITY_NONE) {
+            chosen = p[i];
+            break;
+          }
+          wants_password |= p[i] == SECURITY_VNC_AUTH;
+        }
+        this->rx_pos_ += this->sec_count_;
+        if (chosen == 0) {
+          return this->fail_(VncFailure::VNC_FAILURE_CONFIG, wants_password ? LOG_STR("The server requires a password")
+                                                                            : LOG_STR("No supported security type"));
+        }
+        if (!this->send_(&chosen, 1)) {
+          return this->fail_send_();
+        }
+        if (this->minor_ == 8) {
+          this->step_ = VncStep::VNC_STEP_SECURITY_RESULT;
+        } else {
+          if (!this->send_client_init_()) {
+            return this->fail_send_();
+          }
+          this->step_ = VncStep::VNC_STEP_SERVER_INIT;
+        }
+        break;
+      }
+      case VncStep::VNC_STEP_SECURITY_TYPE_33: {
+        if (avail < 4) {
+          return VncParse::VNC_PARSE_MORE;
+        }
+        const uint32_t type = read_be32(p);
+        this->rx_pos_ += 4;
+        if (type == 0) {
+          this->step_ = VncStep::VNC_STEP_REASON_LENGTH;
+        } else if (type == SECURITY_NONE) {
+          if (!this->send_client_init_()) {
+            return this->fail_send_();
+          }
+          this->step_ = VncStep::VNC_STEP_SERVER_INIT;
+        } else {
+          return this->fail_(VncFailure::VNC_FAILURE_CONFIG, type == SECURITY_VNC_AUTH
+                                                                 ? LOG_STR("The server requires a password")
+                                                                 : LOG_STR("No supported security type"));
+        }
+        break;
+      }
+      case VncStep::VNC_STEP_SECURITY_RESULT: {
+        if (avail < 4) {
+          return VncParse::VNC_PARSE_MORE;
+        }
+        const uint32_t result = read_be32(p);
+        this->rx_pos_ += 4;
+        if (result == 0) {
+          if (!this->send_client_init_()) {
+            return this->fail_send_();
+          }
+          this->step_ = VncStep::VNC_STEP_SERVER_INIT;
+          break;
+        }
+        this->reason_failure_ = VncFailure::VNC_FAILURE_CONFIG;
+        this->step_ = VncStep::VNC_STEP_REASON_LENGTH;
+        break;
+      }
+      case VncStep::VNC_STEP_REASON_LENGTH: {
+        if (avail < 4) {
+          return VncParse::VNC_PARSE_MORE;
+        }
+        this->text_len_ = static_cast<uint8_t>(std::min(read_be32(p), MAX_TEXT_LEN));
+        this->rx_pos_ += 4;
+        this->step_ = VncStep::VNC_STEP_REASON;
+        break;
+      }
+      case VncStep::VNC_STEP_REASON: {
+        if (avail < this->text_len_) {
+          return VncParse::VNC_PARSE_MORE;
+        }
+        ESP_LOGE(TAG, "Server: %.*s", static_cast<int>(this->text_len_), reinterpret_cast<const char *>(p));
+        return this->fail_(this->reason_failure_, LOG_STR("Connection refused"));
+      }
+      case VncStep::VNC_STEP_SERVER_INIT: {
+        if (avail < 24) {
+          return VncParse::VNC_PARSE_MORE;
+        }
+        this->fb_w_ = read_be16(p);
+        this->fb_h_ = read_be16(p + 2);
+        // Reused for the length of the name that follows.
+        this->skip_left_ = read_be32(p + 20);
+        this->rx_pos_ += 24;
+        if (this->fb_w_ == 0 || this->fb_h_ == 0) {
+          return this->fail_(VncFailure::VNC_FAILURE_TRANSIENT, LOG_STR("The server screen is empty"));
+        }
+        this->text_len_ = static_cast<uint8_t>(std::min(this->skip_left_, MAX_TEXT_LEN));
+        this->step_ = VncStep::VNC_STEP_SERVER_NAME;
+        break;
+      }
+      case VncStep::VNC_STEP_SERVER_NAME: {
+        if (avail < this->text_len_) {
+          return VncParse::VNC_PARSE_MORE;
+        }
+        ESP_LOGI(TAG, "Server '%.*s', %ux%u, RFB 3.%u", static_cast<int>(this->text_len_),
+                 reinterpret_cast<const char *>(p), this->fb_w_, this->fb_h_, this->minor_);
+        this->rx_pos_ += this->text_len_;
+        this->skip_(this->skip_left_ - this->text_len_, VncStep::VNC_STEP_SESSION_START);
+        break;
+      }
+      case VncStep::VNC_STEP_SESSION_START:
+        if (!this->start_session_(App.get_loop_component_start_time())) {
+          return this->fail_send_();
+        }
+        this->step_ = VncStep::VNC_STEP_MESSAGE;
+        break;
+      case VncStep::VNC_STEP_MESSAGE: {
+        if (avail < 1) {
+          return VncParse::VNC_PARSE_MORE;
+        }
+        switch (p[0]) {
+          case MSG_FRAMEBUFFER_UPDATE:
+            if (avail < 4) {
+              return VncParse::VNC_PARSE_MORE;
+            }
+            this->rects_left_ = read_be16(p + 2);
+            this->rx_pos_ += 4;
+            this->in_update_ = true;
+            this->update_start_ms_ = millis();
+            this->update_start_bytes_ = this->rx_total_ - (this->rx_len_ - this->rx_pos_);
+            this->rect_done_();
+            break;
+          case MSG_SET_COLOUR_MAP:
+            if (avail < 6) {
+              return VncParse::VNC_PARSE_MORE;
+            }
+            this->rx_pos_ += 6;
+            this->skip_(6u * read_be16(p + 4), VncStep::VNC_STEP_MESSAGE);
+            break;
+          case MSG_BELL:
+            this->rx_pos_ += 1;
+            break;
+          case MSG_SERVER_CUT_TEXT:
+            if (avail < 8) {
+              return VncParse::VNC_PARSE_MORE;
+            }
+            this->rx_pos_ += 8;
+            this->skip_(read_be32(p + 4), VncStep::VNC_STEP_MESSAGE);
+            break;
+          default:
+            ESP_LOGW(TAG, "Unknown message type %u", p[0]);
+            return this->fail_(VncFailure::VNC_FAILURE_TRANSIENT, LOG_STR("Protocol error"));
+        }
+        break;
+      }
+      case VncStep::VNC_STEP_RECT: {
+        if (this->display_busy_) {
+          return VncParse::VNC_PARSE_PAUSE;
+        }
+        if (avail < 12) {
+          return VncParse::VNC_PARSE_MORE;
+        }
+        this->rect_x_ = read_be16(p);
+        this->rect_y_ = read_be16(p + 2);
+        this->rect_w_ = read_be16(p + 4);
+        this->rect_h_ = read_be16(p + 6);
+        const auto encoding = static_cast<int32_t>(read_be32(p + 8));
+        this->rx_pos_ += 12;
+        this->rects_left_--;
+        if (encoding == ENCODING_RAW) {
+          if (static_cast<uint32_t>(this->rect_x_) + this->rect_w_ > this->fb_w_ ||
+              static_cast<uint32_t>(this->rect_y_) + this->rect_h_ > this->fb_h_) {
+            ESP_LOGW(TAG, "Rectangle %ux%u at %u,%u is outside the %ux%u screen", this->rect_w_, this->rect_h_,
+                     this->rect_x_, this->rect_y_, this->fb_w_, this->fb_h_);
+            return this->fail_(VncFailure::VNC_FAILURE_TRANSIENT, LOG_STR("Protocol error"));
+          }
+          if (this->rect_w_ == 0 || this->rect_h_ == 0) {
+            this->rect_done_();
+          } else {
+            this->pos_x_ = 0;
+            this->pos_y_ = 0;
+            this->step_ = VncStep::VNC_STEP_RAW;
+          }
+        } else if (encoding == ENCODING_RICH_CURSOR) {
+          if (this->rect_w_ > MAX_CURSOR_SIDE || this->rect_h_ > MAX_CURSOR_SIDE) {
+            return this->fail_(VncFailure::VNC_FAILURE_TRANSIENT, LOG_STR("Protocol error"));
+          }
+          const uint32_t size = static_cast<uint32_t>(this->rect_w_) * this->rect_h_ * this->bpp_ +
+                                ((this->rect_w_ + 7u) / 8u) * this->rect_h_;
+          this->skip_(size, VncStep::VNC_STEP_RECT_DONE);
+        } else {
+          ESP_LOGW(TAG, "Unsupported encoding %" PRId32, encoding);
+          return this->fail_(VncFailure::VNC_FAILURE_TRANSIENT, LOG_STR("Protocol error"));
+        }
+        break;
+      }
+      case VncStep::VNC_STEP_RECT_DONE:
+        this->rect_done_();
+        break;
+      case VncStep::VNC_STEP_RAW: {
+        if (this->display_busy_) {
+          return VncParse::VNC_PARSE_PAUSE;
+        }
+        const size_t pixels = avail / this->bpp_;
+        if (pixels == 0) {
+          return VncParse::VNC_PARSE_MORE;
+        }
+        if (this->budget_exhausted_()) {
+          return VncParse::VNC_PARSE_PAUSE;
+        }
+        const auto count = static_cast<uint16_t>(std::min<size_t>(pixels, this->rect_w_ - this->pos_x_));
+        this->draw_row_(this->rect_x_ + this->pos_x_, this->rect_y_ + this->pos_y_, count, p);
+        this->rx_pos_ += count * this->bpp_;
+        this->pos_x_ += count;
+        if (this->pos_x_ == this->rect_w_) {
+          this->pos_x_ = 0;
+          if (++this->pos_y_ == this->rect_h_) {
+            this->rect_done_();
+          }
+        }
+        break;
+      }
+      case VncStep::VNC_STEP_SKIP: {
+        const auto count = static_cast<uint32_t>(std::min<size_t>(avail, this->skip_left_));
+        this->rx_pos_ += count;
+        this->skip_left_ -= count;
+        if (this->skip_left_ != 0) {
+          return VncParse::VNC_PARSE_MORE;
+        }
+        this->step_ = this->after_skip_;
+        break;
+      }
+    }
+  }
+}
+
+void VncClient::rect_done_() {
+  if (this->rects_left_ == 0) {
+    this->frame_done_();
+  } else {
+    this->step_ = VncStep::VNC_STEP_RECT;
+  }
+}
+
+void VncClient::frame_done_() {
+  this->in_update_ = false;
+  this->request_outstanding_ = false;
+  this->step_ = VncStep::VNC_STEP_MESSAGE;
+  if (this->frame_drawn_) {
+    const uint32_t now = App.get_loop_component_start_time();
+    this->frame_drawn_ = false;
+    if (!this->refresh_pending_) {
+      this->first_change_ms_ = now;
+    }
+    this->last_change_ms_ = now;
+    this->refresh_pending_ = true;
+    ESP_LOGD(TAG, "Update: %" PRIu32 " bytes in %" PRIu32 "ms",
+             this->rx_total_ - (this->rx_len_ - this->rx_pos_) - this->update_start_bytes_,
+             millis() - this->update_start_ms_);
+  }
+}
+
+void VncClient::draw_row_(uint16_t x, uint16_t y, uint16_t count, const uint8_t *pixels) {
+  if (y >= this->vis_h_ || x >= this->vis_w_) {
+    return;
+  }
+  const int visible = std::min<int>(count, this->vis_w_ - x);
+  this->display_->draw_pixels_at(x, y, visible, 1, pixels, this->bitness_, this->big_endian_);
+  this->frame_drawn_ = true;
+}
+
+bool VncClient::start_session_(uint32_t now) {
+  const bool rgb565 = this->display_->get_display_type() != display::DISPLAY_TYPE_BINARY;
+  this->bpp_ = rgb565 ? 2 : 1;
+  this->bitness_ = rgb565 ? display::COLOR_BITNESS_565 : display::COLOR_BITNESS_332;
+
+  const int disp_w = this->display_->get_width();
+  const int disp_h = this->display_->get_height();
+  this->vis_w_ = std::min<int>(this->fb_w_, disp_w);
+  this->vis_h_ = std::min<int>(this->fb_h_, disp_h);
+  if (this->fb_w_ != disp_w || this->fb_h_ != disp_h) {
+    ESP_LOGW(TAG, "Server screen is %ux%u but the display is %dx%d; showing the top-left %ux%u", this->fb_w_,
+             this->fb_h_, disp_w, disp_h, this->vis_w_, this->vis_h_);
+  }
+
+  // SetPixelFormat: true colour, in the byte order the display takes
+  uint8_t pixel_format[20] = {};
+  pixel_format[4] = rgb565 ? 16 : 8;
+  pixel_format[5] = rgb565 ? 16 : 8;
+  pixel_format[6] = this->big_endian_;
+  pixel_format[7] = 1;
+  write_be16(pixel_format + 8, rgb565 ? 31 : 7);
+  write_be16(pixel_format + 10, rgb565 ? 63 : 7);
+  write_be16(pixel_format + 12, rgb565 ? 31 : 3);
+  pixel_format[14] = rgb565 ? 11 : 5;
+  pixel_format[15] = rgb565 ? 5 : 2;
+  if (!this->send_(pixel_format, sizeof(pixel_format))) {
+    return false;
+  }
+
+  // SetEncodings: TightVNC uses the first real encoding it supports for every rectangle.
+  uint8_t encodings[12] = {2, 0, 0, 2};
+  write_be32(encodings + 4, ENCODING_RAW);
+  write_be32(encodings + 8, static_cast<uint32_t>(ENCODING_RICH_CURSOR));
+  if (!this->send_(encodings, sizeof(encodings)) || !this->send_update_request_(false, now)) {
+    return false;
+  }
+  this->state_ = VncState::VNC_STATE_RUNNING;
+  this->session_start_ms_ = now;
+  if (!this->busy_warned_) {
+    this->status_clear_warning();  // a stuck display keeps its warning through a reconnect
+  }
+  this->status_clear_error();
+  this->connect_callback_.call();
+  return true;
+}
+
+void VncClient::pace_(uint32_t now) {
+  if (this->display_busy_ || this->in_update_ || now - this->last_request_ms_ < MIN_REQUEST_SPACING_MS) {
+    return;
+  }
+  // Repeating a parked request costs nothing on the server and detects a dead link.
+  if (!this->request_outstanding_ ||
+      (now - this->last_rx_ms_ >= HEARTBEAT_MS && now - this->last_request_ms_ >= HEARTBEAT_MS)) {
+    this->send_update_request_(true, now);
+  }
+}
+
+bool VncClient::send_(const uint8_t *data, size_t len) {
+  if (this->link_.tx_free() < len) {
+    this->link_.flush_tx();
+    if (this->link_.tx_free() < len) {
+      return false;
+    }
+  }
+  this->link_.queue(data, len);
+  return true;
+}
+
+bool VncClient::send_update_request_(bool incremental, uint32_t now) {
+  uint8_t msg[10] = {3, static_cast<uint8_t>(incremental ? 1 : 0)};
+  write_be16(msg + 6, this->vis_w_);
+  write_be16(msg + 8, this->vis_h_);
+  if (!this->send_(msg, sizeof(msg))) {
+    return false;
+  }
+  this->request_outstanding_ = true;
+  this->last_request_ms_ = now;
+  return true;
+}
+
+}  // namespace esphome::vnc

esphome/components/vnc/vnc_client.h

@@ -0,0 +1,162 @@
+#pragma once
+
+#include "esphome/components/display/display.h"
+#include "esphome/components/socket/tcp_client_link.h"
+#include "esphome/core/component.h"
+#include "esphome/core/helpers.h"
+
+#include <cstddef>
+#include <cstdint>
+
+namespace esphome::vnc {
+
+enum class VncState : uint8_t {
+  VNC_STATE_IDLE,
+  VNC_STATE_HANDSHAKE,
+  VNC_STATE_RUNNING,
+};
+
+enum class VncStep : uint8_t {
+  VNC_STEP_VERSION,
+  VNC_STEP_SECURITY_COUNT,
+  VNC_STEP_SECURITY_LIST,
+  VNC_STEP_SECURITY_TYPE_33,
+  VNC_STEP_SECURITY_RESULT,
+  VNC_STEP_REASON_LENGTH,
+  VNC_STEP_REASON,
+  VNC_STEP_SERVER_INIT,
+  VNC_STEP_SERVER_NAME,
+  VNC_STEP_SESSION_START,
+  VNC_STEP_MESSAGE,
+  VNC_STEP_RECT,
+  VNC_STEP_RECT_DONE,
+  VNC_STEP_RAW,
+  VNC_STEP_SKIP,
+};
+
+enum class VncParse : uint8_t {
+  VNC_PARSE_MORE,   // the next step needs bytes that have not arrived
+  VNC_PARSE_PAUSE,  // out of time, or the display is busy; resume without new data
+  VNC_PARSE_STOP,   // the session ended
+};
+
+enum class VncFailure : uint8_t {
+  VNC_FAILURE_TRANSIENT,
+  VNC_FAILURE_CONFIG,
+};
+
+/// RFB (VNC) client that draws the remote framebuffer into a display.
+class VncClient : public Component {
+ public:
+  explicit VncClient(display::Display *display, bool big_endian = false) : display_(display), big_endian_(big_endian) {}
+
+  void set_host(const char *host) { this->link_.set_host(host); }
+  void set_port(uint16_t port) { this->link_.set_port(port); }
+  void set_reconnect_interval(uint32_t ms) { this->link_.set_reconnect_interval(ms); }
+  void set_update_delay(uint32_t ms) { this->update_delay_ms_ = ms; }
+  void set_max_update_delay(uint32_t ms) { this->max_update_delay_ms_ = ms; }
+
+  void setup() override;
+  void loop() override;
+  void dump_config() override;
+  void on_shutdown() override;
+  float get_setup_priority() const override { return setup_priority::AFTER_WIFI; }
+
+  bool is_connected() const { return this->state_ == VncState::VNC_STATE_RUNNING; }
+
+  template<typename F> void add_on_connect_callback(F &&callback) {
+    this->connect_callback_.add(std::forward<F>(callback));
+  }
+  template<typename F> void add_on_disconnect_callback(F &&callback) {
+    this->disconnect_callback_.add(std::forward<F>(callback));
+  }
+
+ protected:
+  static constexpr size_t RX_BUFFER_SIZE = 512;
+
+  void connect_step_(uint32_t now);
+  void start_handshake_(uint32_t now);
+  void service_rx_(uint32_t now);
+  VncParse parse_();
+  void pace_(uint32_t now);
+  bool start_session_(uint32_t now);
+  bool send_client_init_();
+  void frame_done_();
+  void rect_done_();
+  void draw_row_(uint16_t x, uint16_t y, uint16_t count, const uint8_t *pixels);
+  void skip_(uint32_t count, VncStep next);
+  bool budget_exhausted_() const;
+  bool send_(const uint8_t *data, size_t len);
+  bool send_update_request_(bool incremental, uint32_t now);
+  VncParse fail_(VncFailure failure, const LogString *reason);
+  VncParse fail_send_() { return this->fail_(VncFailure::VNC_FAILURE_TRANSIENT, LOG_STR("Send failed")); }
+  VncFailure lost_failure_() const;
+  void session_lost_(uint32_t now);
+  void session_end_(uint32_t now, VncFailure failure, const LogString *reason);
+  void reset_session_();
+
+  display::Display *display_;
+  socket::TcpClientLink link_;
+  LazyCallbackManager<void()> connect_callback_;
+  LazyCallbackManager<void()> disconnect_callback_;
+  HighFrequencyLoopRequester high_freq_;
+
+  uint32_t update_delay_ms_{200};
+  uint32_t max_update_delay_ms_{1000};
+  uint32_t backoff_ms_{0};
+  uint32_t hold_ms_{0};
+  uint32_t hold_start_ms_{0};
+  uint32_t setup_ms_{0};
+  uint32_t busy_since_ms_{0};
+  uint32_t session_start_ms_{0};
+  uint32_t update_start_ms_{0};
+  // The first and the latest frame drawn since the display last refreshed.
+  uint32_t first_change_ms_{0};
+  uint32_t last_change_ms_{0};
+  uint32_t rx_total_{0};
+  uint32_t update_start_bytes_{0};
+  uint32_t last_request_ms_{0};
+  uint32_t last_rx_ms_{0};
+  uint32_t pass_start_ms_{0};
+  uint32_t skip_left_{0};
+
+  uint16_t rx_pos_{0};
+  uint16_t rx_len_{0};
+  uint16_t fb_w_{0};
+  uint16_t fb_h_{0};
+  uint16_t vis_w_{0};
+  uint16_t vis_h_{0};
+  uint16_t rects_left_{0};
+  uint16_t rect_x_{0};
+  uint16_t rect_y_{0};
+  uint16_t rect_w_{0};
+  uint16_t rect_h_{0};
+  // Raw: column and row inside the rect.
+  uint16_t pos_x_{0};
+  uint16_t pos_y_{0};
+
+  uint8_t bpp_{2};
+  uint8_t minor_{8};
+  uint8_t sec_count_{0};
+  uint8_t text_len_{0};  // clamped to MAX_TEXT_LEN
+  display::ColorBitness bitness_{display::COLOR_BITNESS_565};
+  VncState state_{VncState::VNC_STATE_IDLE};
+  VncStep step_{VncStep::VNC_STEP_VERSION};
+  VncStep after_skip_{VncStep::VNC_STEP_MESSAGE};
+  VncFailure reason_failure_{VncFailure::VNC_FAILURE_TRANSIENT};
+
+  bool rx_pending_{false};
+  bool in_update_{false};
+  bool request_outstanding_{false};
+  bool refresh_pending_{false};
+  bool frame_drawn_{false};
+  bool display_parks_{false};
+  bool display_busy_{false};
+  bool busy_warned_{false};
+  bool network_was_up_{false};
+  bool big_endian_;  // the pixel order the display takes
+
+  uint8_t rx_[RX_BUFFER_SIZE];
+};
+
+}  // namespace esphome::vnc

esphome/components/wifi/__init__.py

@@ -39,6 +39,7 @@ from esphome.const import (
     CONF_ENABLE_BTM,
     CONF_ENABLE_ON_BOOT,
     CONF_ENABLE_RRM,
+    CONF_ETHERNET,
     CONF_FAST_CONNECT,
     CONF_GATEWAY,
     CONF_HIDDEN,
@@ -431,8 +432,26 @@ def _apply_min_auth_mode_default(config):
     return config
 
 
+def wifi_has_sta(config: ConfigType) -> bool:
+    """Whether the validated config has at least one station network."""
+    return bool(config.get(CONF_NETWORKS))
+
+
+# Components that save station credentials at run time
+_PROVISIONERS = ("captive_portal", "improv_ble", "improv_serial")
+
+
+def wifi_stays_ap_only(config: ConfigType, full_config: ConfigType) -> bool:
+    """Whether no station can appear at run time: no networks, no DPP, no provisioner."""
+    return not (
+        wifi_has_sta(config)
+        or CONF_DPP in config
+        or any(c in full_config for c in _PROVISIONERS)
+    )
+
+
 def final_validate(config):
-    has_sta = bool(config.get(CONF_NETWORKS, True))
+    has_sta = wifi_has_sta(config)
     has_ap = CONF_AP in config
     has_dpp = CONF_DPP in config
     full_config = fv.full_config.get()
@@ -447,9 +466,11 @@ def final_validate(config):
 
     ap_is_wisp = has_ap and config[CONF_AP][CONF_AP_MODE] == WIFI_AP_MODE_WISP
     if ap_is_wisp:
-        if not has_sta:
+        # The clients are routed through the station, or through ethernet
+        if not has_sta and CONF_ETHERNET not in full_config:
             raise cv.Invalid(
-                "WISP mode requires at least one STA network (wifi.ssid or wifi.networks)"
+                "WISP mode needs a station network (wifi.ssid or wifi.networks), "
+                "or ethernet as the upstream interface"
             )
         if has_captive_portal:
             raise cv.Invalid(
@@ -458,14 +479,27 @@ def final_validate(config):
         # Every client packet passes through the WiFi driver, none through the device's own sockets
         require_high_performance_wifi()
         if CONF_DNS_SERVERS not in config[CONF_AP]:
-            # The access point passes the station's resolvers on, which a static address has to name
-            for network in [config, *config.get(CONF_NETWORKS, [])]:
+            # The access point passes the upstream's resolvers on, which a static address has to name
+            upstreams = [
+                ([], config),
+                *(
+                    ([CONF_NETWORKS, i], network)
+                    for i, network in enumerate(config.get(CONF_NETWORKS, []))
+                ),
+                (None, full_config.get(CONF_ETHERNET, {})),
+            ]
+            for path, network in upstreams:
                 static = network.get(CONF_MANUAL_IP)
-                if static and static[CONF_DNS1].is_unspecified:
+                if not static or not static[CONF_DNS1].is_unspecified:
+                    continue
+                if path is None:
                     raise cv.Invalid(
-                        "manual_ip needs dns1 in WISP mode unless ap.dns_servers is set",
-                        path=[CONF_MANUAL_IP, CONF_DNS1],
+                        "ethernet manual_ip needs dns1 in WISP mode unless ap.dns_servers is set"
                     )
+                raise cv.Invalid(
+                    "manual_ip needs dns1 in WISP mode unless ap.dns_servers is set",
+                    path=[*path, CONF_MANUAL_IP, CONF_DNS1],
+                )
 
     if has_ap and not has_captive_portal and not has_web_server and not ap_is_wisp:
         _LOGGER.warning(
@@ -565,7 +599,7 @@ def _report_provisioning_credentials(config):
     """
     from esphome.components import provisioning
 
-    if config.get(CONF_NETWORKS):
+    if wifi_has_sta(config):
         provisioning.report_hardcoded_credentials("wifi")
     elif CONF_AP in config:
         # An access point with no station credentials: the AP shuts down when the

esphome/components/wifi/wifi_component.cpp

@@ -783,12 +783,9 @@ void WiFiComponent::start() {
     // Without fast_connect: go straight to scanning (or hidden mode if all networks are hidden)
     this->start_initial_connection_();
 #endif
-#ifdef USE_WIFI_AP
-  } else if (this->has_ap()) {
+#if defined(USE_WIFI_AP) && !defined(USE_WIFI_AP_WISP)
+  } else if (this->has_ap()) {  // a bridge waits in loop() for the upstream's resolver
     this->setup_ap_config_();
-    if (!std::isnan(this->output_power_) && !this->wifi_apply_output_power_(this->output_power_)) {
-      ESP_LOGV(TAG, "Setting Output Power Option failed");
-    }
 #ifdef USE_CAPTIVE_PORTAL
     // Where the radio scans alongside the AP, the portal can list networks.
     if (captive_portal::global_captive_portal != nullptr && !WIFI_AP_EXCLUSIVE) {
@@ -797,7 +794,7 @@ void WiFiComponent::start() {
     }
 #endif
     this->start_ap_portal_();
-#endif  // USE_WIFI_AP
+#endif  // USE_WIFI_AP && !USE_WIFI_AP_WISP
   }
 #ifdef USE_WIFI_DPP
   // Without networks there is nothing to try first, so listen for a configurator right away
@@ -937,15 +934,7 @@ void WiFiComponent::loop() {
         return;
     }
 
-#ifdef USE_WIFI_AP_WISP
-    // The bridge comes up once it has a resolver to offer; a start that failed is tried again after ap_timeout
-    if (!this->ap_setup_ && this->ap_dns_known_() &&
-        (this->ap_changed_ == 0 || now - this->ap_changed_ > this->ap_timeout_)) {
-      ESP_LOGI(TAG, "Starting AP");
-      this->ap_changed_ = now;
-      this->setup_ap_config_();
-    }
-#elif defined(USE_WIFI_AP)
+#if defined(USE_WIFI_AP) && !defined(USE_WIFI_AP_WISP)
     // Once the provisioning window has closed, don't bring up the fallback AP (or
     // the captive portal on it) - the device must stay unprovisionable until it is
     // power-cycled.
@@ -975,7 +964,7 @@ void WiFiComponent::loop() {
         this->start_ap_portal_();
       }
     }
-#endif  // USE_WIFI_AP
+#endif  // USE_WIFI_AP && !USE_WIFI_AP_WISP
 
 #ifdef USE_IMPROV_BLE
     if (improv_ble::global_improv_component != nullptr && !improv_ble::global_improv_component->is_active() &&
@@ -1013,6 +1002,16 @@ void WiFiComponent::loop() {
     }
   }
 
+#ifdef USE_WIFI_AP_WISP
+  // The bridge comes up once it has a resolver to offer; a start that failed is tried again after ap_timeout
+  if (!this->ap_setup_ && this->state_ != WIFI_COMPONENT_STATE_DISABLED && this->ap_dns_known_() &&
+      (this->ap_changed_ == 0 || now - this->ap_changed_ > this->ap_timeout_)) {
+    ESP_LOGI(TAG, "Starting AP");
+    this->ap_changed_ = now;
+    this->setup_ap_config_();
+  }
+#endif
+
 #ifdef USE_WIFI_DPP
   // Without networks no cooldown starts the listen, so a failed or abandoned enrollment is listened for again
   // here, at most once per timeout
@@ -1098,6 +1097,11 @@ void WiFiComponent::setup_ap_config_() {
   // Reset before the AP exists: afterwards only the platform's event path writes the counter
   this->ap_clients_ = 0;
   this->ap_setup_ = this->wifi_start_ap_(this->ap_);
+  // The station path applied it in start(); without a station the radio only exists from here
+  if (this->ap_setup_ && !this->has_sta() && !std::isnan(this->output_power_) &&
+      !this->wifi_apply_output_power_(this->output_power_)) {
+    ESP_LOGV(TAG, "Setting Output Power Option failed");
+  }
 #if defined(USE_WIFI_AP_EXCLUSIVE) || defined(USE_WIFI_AP_WISP)
   // A failed start must not leave the radio in AP mode: on an exclusive radio it would refuse
   // the STA attempts until the next try, and a bridge would take clients it cannot serve.

esphome/components/wifi/wifi_component_esp_idf.cpp

@@ -127,6 +127,10 @@ void event_handler(void *arg, esp_event_base_t event_base, int32_t event_id, voi
 #if USE_NETWORK_IPV6
   } else if (event_base == IP_EVENT && event_id == IP_EVENT_GOT_IP6) {
     memcpy(&event.data.ip_got_ip6, event_data, sizeof(ip_event_got_ip6_t));
+#endif
+#if defined(USE_WIFI_AP_WISP) && !defined(USE_WIFI_AP_WISP_STATIC_DNS)
+  } else if (event_base == IP_EVENT && event_id == IP_EVENT_ETH_GOT_IP) {  // NOLINT(bugprone-branch-clone)
+    // no data: the bridge rereads the resolvers an ethernet upstream just received
 #endif
   } else if (event_base == IP_EVENT && event_id == IP_EVENT_STA_LOST_IP) {  // NOLINT(bugprone-branch-clone)
     // no data
@@ -1123,6 +1127,10 @@ void WiFiComponent::wifi_process_event_(IDFWiFiEvent *data) {
     this->notify_disconnect_state_listeners_();
 #endif
 
+#if defined(USE_WIFI_AP_WISP) && !defined(USE_WIFI_AP_WISP_STATIC_DNS)
+  } else if (data->event_base == IP_EVENT && data->event_id == IP_EVENT_ETH_GOT_IP) {
+    this->apply_ap_dns_();
+#endif
   } else if (data->event_base == IP_EVENT && data->event_id == IP_EVENT_STA_GOT_IP) {
     const auto &it = data->data.ip_got_ip;
     ESP_LOGV(TAG, "static_ip=" IPSTR " gateway=" IPSTR, IP2STR(&it.ip_info.ip), IP2STR(&it.ip_info.gw));
@@ -1397,10 +1405,6 @@ bool WiFiComponent::wifi_scan_start_(bool passive) {
 bool WiFiComponent::wifi_ap_ip_config_(const optional<ManualIP> &manual_ip) {
   esp_err_t err;
 
-  // enable AP
-  if (!this->wifi_mode_({}, true))
-    return false;
-
   // Check if the AP interface is initialized before using it
   if (s_ap_netif == nullptr) {
     ESP_LOGW(TAG, "AP interface not initialized");
@@ -1478,8 +1482,8 @@ bool WiFiComponent::wifi_ap_ip_config_(const optional<ManualIP> &manual_ip) {
 #endif
 
   err = esp_netif_dhcps_start(s_ap_netif);
-
-  if (err != ESP_OK) {
+  // Before the access point is up, its start event brings the server up with these settings
+  if (err != ESP_OK && err != ESP_ERR_ESP_NETIF_IF_NOT_READY) {
     ESP_LOGE(TAG, "esp_netif_dhcps_start failed: %d", err);
     return false;
   }
@@ -1488,6 +1492,17 @@ bool WiFiComponent::wifi_ap_ip_config_(const optional<ManualIP> &manual_ip) {
 }
 
 bool WiFiComponent::wifi_start_ap_(const WiFiAP &ap) {
+  // The address and DHCP settings go in before the access point starts: its start event starts the DHCP
+  // server on the event task, and settings written after that race it.
+#ifdef USE_WIFI_MANUAL_IP
+  if (!this->wifi_ap_ip_config_(ap.get_manual_ip())) {
+#else
+  if (!this->wifi_ap_ip_config_({})) {
+#endif
+    ESP_LOGE(TAG, "wifi_ap_ip_config_ failed:");
+    return false;
+  }
+
   // enable AP
   if (!this->wifi_mode_({}, true))
     return false;
@@ -1524,19 +1539,6 @@ bool WiFiComponent::wifi_start_ap_(const WiFiAP &ap) {
     ESP_LOGE(TAG, "esp_wifi_set_config failed: %d", err);
     return false;
   }
-
-#ifdef USE_WIFI_MANUAL_IP
-  if (!this->wifi_ap_ip_config_(ap.get_manual_ip())) {
-    ESP_LOGE(TAG, "wifi_ap_ip_config_ failed:");
-    return false;
-  }
-#else
-  if (!this->wifi_ap_ip_config_({})) {
-    ESP_LOGE(TAG, "wifi_ap_ip_config_ failed:");
-    return false;
-  }
-#endif
-
   return true;
 }
 
@@ -1681,7 +1683,7 @@ bool WiFiComponent::write_ap_dns_() {
 #ifndef USE_WIFI_AP_WISP_STATIC_DNS
 void WiFiComponent::apply_ap_dns_() {
   if (!this->ap_setup_)
-    return;  // loop() brings the bridge up once the station has a resolver, and its setup writes them
+    return;  // loop() brings the bridge up once the upstream has a resolver, and its setup writes them
   // The DHCP server holds what it offers, so there is nothing to cache; a failed update is noticed again with the
   // next lease. A resolver the station lost stays offered: set_dns_info rejects an empty address.
   bool changed = false;

esphome/core/__init__.py

@@ -658,6 +658,8 @@ class EsphomeCore:
         self.main_statements: list[Statement] = []
         # A list of statements to insert in the global block (includes and global variables)
         self.global_statements: list[Statement] = []
+        # Generated code emitted after the user includes, so it can use what they declare
+        self.global_statements_after_includes: list[Statement] = []
         # A map of platformio libraries to add to the project (shortname: (name, version, repository))
         self.platformio_libraries: dict[str, Library] = {}
         # A set of build flags to set in the platformio project
@@ -728,6 +730,7 @@ class EsphomeCore:
         self.variables = {}
         self.main_statements = []
         self.global_statements = []
+        self.global_statements_after_includes = []
         self.platformio_libraries = {}
         self.build_flags = set()
         self.cmake_args = {}
@@ -1045,8 +1048,8 @@ class EsphomeCore:
         except RuntimeError as e:
             raise EsphomeError(str(e)) from e
 
-    def add(self, expression, prepend=False) -> "Statement":
-        """Add an expression or statement to the main setup() block."""
+    @staticmethod
+    def _as_statement(expression) -> "Statement":
         from esphome.cpp_generator import Expression, Statement, statement
 
         if isinstance(expression, Expression):
@@ -1055,7 +1058,11 @@ class EsphomeCore:
             raise ValueError(
                 f"Add '{expression}' must be expression or statement, not {type(expression)}"
             )
+        return expression
 
+    def add(self, expression, prepend=False) -> "Statement":
+        """Add an expression or statement to the main setup() block."""
+        expression = self._as_statement(expression)
         if prepend:
             self.main_statements.insert(0, expression)
         else:
@@ -1064,14 +1071,7 @@ class EsphomeCore:
         return expression
 
     def add_global(self, expression, prepend=False) -> "Statement":
-        from esphome.cpp_generator import Expression, Statement, statement
-
-        if isinstance(expression, Expression):
-            expression = statement(expression)
-        if not isinstance(expression, Statement):
-            raise ValueError(
-                f"Add '{expression}' must be expression or statement, not {type(expression)}"
-            )
+        expression = self._as_statement(expression)
         if prepend:
             self.global_statements.insert(0, expression)
         else:
@@ -1079,6 +1079,13 @@ class EsphomeCore:
         _LOGGER.debug("Adding global: %s", expression)
         return expression
 
+    def add_global_after_includes(self, expression) -> "Statement":
+        """Add a global after the user includes, for generated code that may use them."""
+        expression = self._as_statement(expression)
+        self.global_statements_after_includes.append(expression)
+        _LOGGER.debug("Adding global after includes: %s", expression)
+        return expression
+
     def add_library(self, library: Library):
         if not isinstance(library, Library):
             raise TypeError(
@@ -1258,7 +1265,7 @@ class EsphomeCore:
         from esphome.cpp_generator import statement
 
         global_code = []
-        for exp in self.global_statements:
+        for exp in (*self.global_statements, *self.global_statements_after_includes):
             text = str(statement(exp))
             text = text.rstrip()
             global_code.append(text)

esphome/core/config.py

@@ -590,7 +590,8 @@ async def add_arduino_global_workaround():
 
 @coroutine_with_priority(CoroPriority.FINAL)
 async def add_includes(includes: list[str], is_c_header: bool = False) -> None:
-    # Add includes at the very end, so that the included files can access global variables
+    # After the id declarations, so the headers can use id(x); the generated function bodies
+    # and flash tables come later still, so their lambdas can use the headers.
     for include in includes:
         path = CORE.relative_config_path(include)
         if path.is_dir():

esphome/core/defines.h

@@ -351,6 +351,7 @@
 #define USE_API_USER_DEFINED_ACTIONS
 #define USE_API_CUSTOM_SERVICES
 #define USE_API_USER_DEFINED_ACTION_METADATA
+#define USE_API_USER_DEFINED_ACTION_OPTIONAL_ARGS
 #define USE_API_USER_DEFINED_ACTION_RESPONSES
 #define USE_API_USER_DEFINED_ACTION_RESPONSES_JSON
 #define API_MAX_SEND_QUEUE 8

esphome/cpp_generator.py

@@ -536,8 +536,10 @@ def shared_progmem_array(
     used |= {str(i) for i in CORE.variables}
     used |= set(RESERVED_IDS) | CORE.loaded_integrations
     id_ = ID(ensure_unique_string(name, used), is_declaration=True, type=type_)
-    # Global, so any scope can use it; anything a lambda references is already declared.
-    CORE.add_global(ProgmemAssignmentExpression(type_, id_, rhs, constexpr))
+    # After the user includes, so a lambda in the table can use what their headers declare.
+    CORE.add_global_after_includes(
+        ProgmemAssignmentExpression(type_, id_, rhs, constexpr)
+    )
     array = MockObj(id_, ".")
     CORE.register_variable(id_, array)
     if share:
@@ -789,15 +791,16 @@ def static_function(
     """Emit ``static <return_type> <name>(parameters) { body }`` at global scope and return an
     expression naming it, for use as a template argument or a function pointer.
 
-    Every id the body names must already be declared, which holds when the statements were
-    rendered through ``get_variable`` or ``process_lambda``.
+    The declaration goes with the other globals, so a template argument can name the function
+    at once; the body goes after the user includes, where every id and everything those
+    headers declare is in scope.
     """
-    params = ParameterListExpression(*parameters)
-    add_global(
-        RawStatement(
-            f"static {safe_exp(return_type)} {name}({params}) {{\n"
-            f"{indent(chr(10).join(body))}\n}}"
-        )
+    signature = (
+        f"static {safe_exp(return_type)} {name}({ParameterListExpression(*parameters)})"
+    )
+    add_global(RawStatement(f"{signature};"))
+    add_global_after_includes(
+        RawStatement(f"{signature} {{\n{indent(chr(10).join(body))}\n}}")
     )
     return RawExpression(name)
 
@@ -816,6 +819,13 @@ def add_global(expression: SafeExpType | Statement, prepend: bool = False):
     CORE.add_global(expression, prepend)
 
 
+def add_global_after_includes(expression: SafeExpType | Statement):
+    """Add a global after the user's ``includes:`` headers, for generated code whose
+    lambdas may use what those headers declare.
+    """
+    CORE.add_global_after_includes(expression)
+
+
 def add_library(name: str, version: str | None, repository: str | None = None):
     """Add a library to the codegen library storage.
 

esphome/espidf/component.py

@@ -151,9 +151,10 @@ def generate_cmakelists_txt(component: IDFComponent) -> str:
             str((read_path / Path(d)).resolve()) for d in build_include_dirs
         ]
 
-    # Split build_flags list into private and public lists
+    # Split build_flags list into private and public lists. Warnings and the optimisation level only concern the
+    # library's own sources, so a library built with -O2 does not change how its users are compiled.
     private_build_flags, public_build_flags = split_list_by_condition(
-        build_flags, lambda a: a if a.startswith("-W") else None
+        build_flags, lambda a: a if a.startswith(("-W", "-O")) else None
     )
 
     # Generate the component

requirements.txt

@@ -12,7 +12,7 @@ pyserial==3.5
 platformio==6.2.0
 esptool==5.6.0
 click==8.5.0
-aioesphomeapi==46.19.0
+aioesphomeapi==46.21.0
 aiohappyeyeballs==2.7.1  # Happy Eyeballs for requests downloads; already pulled in by aioesphomeapi
 zeroconf==0.151.5
 puremagic==2.2.0

tests/component_tests/api/test_action_metadata.py

@@ -112,7 +112,7 @@ def test_empty_metadata_is_unset() -> None:
         "description": "",
         "variables": {"b": {"type": "int", "description": "", "example": "ex"}},
     }
-    strings = _action_strings(conf, has_metadata=True)
+    strings = _action_strings(conf, has_metadata=True, has_optional=False)
     assert strings == ["a", None, "b", None, "ex"]
 
 

tests/component_tests/api/test_action_optional_args.py

@@ -0,0 +1,158 @@
+"""Tests for optional user-defined action variables (required/default)."""
+
+from collections.abc import Callable
+from pathlib import Path
+import re
+
+import pytest
+
+from esphome.components.api import (
+    _action_strings,
+    _has_optional_args,
+    validate_variable,
+)
+from esphome.config_validation import Invalid
+from esphome.core import CORE
+
+CONFIG = "tests/component_tests/api/test_action_optional_args.yaml"
+CONFIG_ESP8266 = "tests/component_tests/api/test_action_optional_args_esp8266.yaml"
+CONFIG_SHORTHAND = "tests/component_tests/api/test_action_metadata_shorthand.yaml"
+
+
+def test_defaults_are_emitted_in_the_string_table(
+    generate_main: Callable[[str | Path], str],
+) -> None:
+    """Each argument gains a trailing default slot: "" marks optional without a default."""
+    main_cpp = generate_main(CONFIG)
+
+    assert (
+        "static constexpr const char * api_action0_strings[] PROGMEM = {"
+        '"play_buzzer", "song_str", "", "notes", "", "octave", "5", '
+        '"bpm", "100", "loud", "true"};' in main_cpp
+    )
+    # An all-required action still carries the default slots (as nullptr)
+    assert (
+        "static constexpr const char * api_action1_strings[] PROGMEM = {"
+        '"plain_action", "value", nullptr};' in main_cpp
+    )
+    assert "USE_API_USER_DEFINED_ACTION_OPTIONAL_ARGS" in {d.name for d in CORE.defines}
+
+
+def test_esp8266_defaults_are_progmem_arrays(
+    generate_main: Callable[[str | Path], str],
+) -> None:
+    """On ESP8266 each default is its own PROGMEM array, referenced from the table."""
+    main_cpp = generate_main(CONFIG_ESP8266)
+
+    def array_for(text: str) -> str:
+        match = re.search(
+            rf'static constexpr char (progmem_str\w*)\[\] PROGMEM = "{text}";', main_cpp
+        )
+        assert match, text
+        return match.group(1)
+
+    assert (
+        "static constexpr const char * api_action0_strings[] PROGMEM = {"
+        f"{array_for('play_buzzer')}, {array_for('song_str')}, {array_for('')}, "
+        f"{array_for('notes')}, {array_for('')}, {array_for('octave')}, {array_for('5')}, "
+        f"{array_for('bpm')}, {array_for('100')}, {array_for('loud')}, "
+        f"{array_for('true')}}};" in main_cpp
+    )
+
+
+def test_required_variables_emit_no_define(
+    generate_main: Callable[[str | Path], str],
+) -> None:
+    generate_main(CONFIG_SHORTHAND)
+
+    assert "USE_API_USER_DEFINED_ACTION_OPTIONAL_ARGS" not in {
+        d.name for d in CORE.defines
+    }
+
+
+def test_action_strings_orders_default_after_metadata() -> None:
+    conf = {
+        "action": "a",
+        "description": "d",
+        "variables": {
+            "b": {"type": "int", "example": "1", "default": 5},
+            "c": {"type": "bool", "default": False},
+        },
+    }
+    assert _action_strings(conf, has_metadata=True, has_optional=True) == [
+        "a",
+        "d",
+        "b",
+        None,
+        "1",
+        "5",
+        "c",
+        None,
+        None,
+        "false",
+    ]
+
+
+@pytest.mark.parametrize(
+    ("var", "expected"),
+    [
+        ({"type": "int", "required": False}, ""),
+        ({"type": "string", "default": ""}, ""),
+        ({"type": "float", "default": 5}, "5.0"),
+    ],
+)
+def test_default_slot_marks_optional_and_keeps_the_type(
+    var: dict, expected: str
+) -> None:
+    actions = [{"action": "a", "variables": {"b": validate_variable(var)}}]
+    assert _has_optional_args(actions)
+    assert _action_strings(actions[0], has_metadata=False, has_optional=True) == [
+        "a",
+        "b",
+        expected,
+    ]
+
+
+@pytest.mark.parametrize(
+    ("var", "expected"),
+    [
+        ({"type": "int", "default": "5"}, 5),
+        ({"type": "int", "default": -(2**31)}, -(2**31)),
+        ({"type": "int", "default": 2**31 - 1}, 2**31 - 1),
+        ({"type": "bool", "default": "true"}, True),
+        ({"type": "float", "default": "1.5"}, 1.5),
+        ({"type": "float", "default": 3.4028235e38}, 3.4028235e38),
+        ({"type": "string", "default": "0123"}, "0123"),
+    ],
+)
+def test_default_is_validated_against_type(var: dict, expected: object) -> None:
+    assert validate_variable(var)["default"] == expected
+
+
+@pytest.mark.parametrize(
+    "var",
+    [
+        {"type": "int[]", "required": False},
+        {"type": "string[]", "default": "x"},
+        {"type": "int", "default": 5, "required": True},
+        {"type": "int", "default": "not_a_number"},
+        # ExecuteServiceArgument.int_ is a signed 32-bit value
+        {"type": "int", "default": 2**31},
+        {"type": "int", "default": -(2**31) - 1},
+        # A boolean is an int in Python but "true" is not a number on the wire
+        {"type": "int", "default": True},
+        {"type": "float", "default": False},
+        # ExecuteServiceArgument.float_ is a 32-bit float
+        {"type": "float", "default": 1e40},
+        {"type": "float", "default": "nan"},
+        {"type": "float", "default": "inf"},
+        # A string default is what the user typed, and a C string in flash
+        {"type": "string", "default": 123},
+        {"type": "string", "default": "a\0b"},
+        {"type": "string", "default": "\0a"},
+        {"type": "float", "default": -1e40},
+    ],
+)
+def test_variable_rejects_invalid_optionality(var: dict) -> None:
+    with pytest.raises(Invalid):
+        validate_variable(var)

tests/component_tests/api/test_action_optional_args.yaml

@@ -0,0 +1,14 @@
+esphome:
+  name: test
+
+esp32:
+  board: esp32dev
+
+wifi:
+  ssid: MySSID
+  password: password1
+
+logger:
+
+packages:
+  api: !include test_action_optional_args_common.yaml

tests/component_tests/api/test_action_optional_args_common.yaml

@@ -0,0 +1,26 @@
+api:
+  actions:
+    - action: play_buzzer
+      variables:
+        song_str:
+          type: string
+          required: false
+        notes:
+          type: string
+          required: false
+        octave:
+          type: int
+          default: 5
+        bpm:
+          type: int
+          default: 100
+        loud:
+          type: bool
+          default: true
+      then:
+        - logger.log: Action Called
+    - action: plain_action
+      variables:
+        value: int
+      then:
+        - logger.log: Action Called

tests/component_tests/api/test_action_optional_args_esp8266.yaml

@@ -0,0 +1,14 @@
+esphome:
+  name: test
+
+esp8266:
+  board: d1_mini
+
+wifi:
+  ssid: MySSID
+  password: password1
+
+logger:
+
+packages:
+  api: !include test_action_optional_args_common.yaml

tests/component_tests/ethernet/test_ethernet.py

@@ -11,9 +11,13 @@ from esphome.components.esp32 import (
     VARIANT_ESP32,
     VARIANT_ESP32S3,
 )
-from esphome.components.ethernet import CONF_CLOCK_SPEED, CONFIG_SCHEMA, _final_validate
+from esphome.components.ethernet import (
+    CONF_CLOCK_SPEED,
+    CONFIG_SCHEMA,
+    _final_validate_wifi,
+)
 from esphome.components.network import _validate_priority_list
-from esphome.const import CONF_PRIORITY, PlatformFramework
+from esphome.const import CONF_AP, CONF_NETWORKS, CONF_PRIORITY, PlatformFramework
 from esphome.core import CORE
 import esphome.final_validate as fv
 
@@ -28,24 +32,37 @@ _CH390_CONFIG = {
 }
 
 
-def test_rejects_wifi_and_ethernet_without_priority() -> None:
-    """Wi-Fi + ethernet without a network: priority: list must be rejected."""
-    fv.full_config.set({"wifi": {}, "ethernet": {}})
-    with pytest.raises(Invalid, match="cannot be used together with component wifi"):
-        _final_validate({})
+_WIFI_STA_CONFIG = {CONF_NETWORKS: [{"ssid": "MySSID", "password": "password1"}]}
+_WIFI_AP_CONFIG = {CONF_AP: {"ssid": "espap", "password": "mypassword"}}
 
 
-def test_rejects_wifi_and_ethernet_with_incomplete_priority() -> None:
-    """A priority list missing an interface is rejected and names what's missing."""
-    fv.full_config.set(
-        {
-            "wifi": {},
-            "ethernet": {},
-            "network": {CONF_PRIORITY: _validate_priority_list(["ethernet"])},
-        }
-    )
-    with pytest.raises(Invalid, match=r"must.*list both interfaces; missing: wifi"):
-        _final_validate({})
+@pytest.mark.parametrize(
+    ("wifi", "priority", "extra", "match"),
+    [
+        (_WIFI_STA_CONFIG, None, {}, "missing: ethernet, wifi"),
+        (_WIFI_STA_CONFIG, ["ethernet"], {}, "missing: wifi"),
+        (_WIFI_STA_CONFIG, ["wifi", "ethernet"], {}, None),
+        (_WIFI_AP_CONFIG, None, {}, None),
+        # A station can still appear at run time: DPP or a provisioner saves credentials
+        ({"dpp": {}}, None, {}, "missing: ethernet, wifi"),
+        ({"dpp": {}, **_WIFI_AP_CONFIG}, None, {}, "missing: ethernet, wifi"),
+        (_WIFI_AP_CONFIG, None, {"captive_portal": {}}, "missing: ethernet, wifi"),
+        (_WIFI_AP_CONFIG, None, {"improv_serial": {}}, "missing: ethernet, wifi"),
+    ],
+)
+def test_wifi_station_needs_a_full_priority_list_but_an_access_point_does_not(
+    wifi: dict, priority: list[str] | None, extra: dict, match: str | None
+) -> None:
+    """A station, configured or provisioned later, competes with ethernet; an access point does not."""
+    full = {"wifi": wifi, "ethernet": {}, **extra}
+    if priority:
+        full["network"] = {CONF_PRIORITY: _validate_priority_list(priority)}
+    fv.full_config.set(full)
+    if match is None:
+        _final_validate_wifi({})
+    else:
+        with pytest.raises(Invalid, match=match):
+            _final_validate_wifi({})
 
 
 @pytest.mark.parametrize("clock_speed", ["26.67MHz", "72MHz"])

tests/component_tests/speaker_source/test_media_player.py

@@ -1,14 +1,22 @@
-"""Tests for the speaker_source media player announcement volume validation."""
+"""Tests for the speaker_source media player pipeline validation."""
+
+from typing import Any
 
 import pytest
 
 from esphome import config_validation as cv
-from esphome.components.const import CONF_ANNOUNCEMENT_VOLUME
+from esphome.components import audio
+from esphome.components.const import CONF_ANNOUNCEMENT_VOLUME, CONF_BITRATE
 from esphome.components.speaker_source.media_player import (
     CONF_ANNOUNCEMENT_PIPELINE,
     CONF_MEDIA_PIPELINE,
+    PIPELINE_SCHEMA,
     _validate_announcement_volume,
+    _validate_pipeline,
 )
+from esphome.const import CONF_FORMAT, CONF_NUM_CHANNELS, CONF_SAMPLE_RATE
+
+PIPELINE = {"speaker": "spk", "sources": ["src"]}
 
 
 def test_announcement_volume_with_both_pipelines() -> None:
@@ -29,3 +37,49 @@ def test_no_announcement_volume_with_one_pipeline() -> None:
 def test_announcement_volume_needs_both_pipelines(pipeline: str) -> None:
     with pytest.raises(cv.Invalid, match="requires both"):
         _validate_announcement_volume({CONF_ANNOUNCEMENT_VOLUME: 0.6, pipeline: {}})
+
+
+@pytest.mark.parametrize("bitrate", [8, 48, 144, 320])
+def test_pipeline_accepts_mp3_bitrates(bitrate: int) -> None:
+    config = PIPELINE_SCHEMA({**PIPELINE, CONF_FORMAT: "MP3", CONF_BITRATE: bitrate})
+    assert config[CONF_BITRATE] == bitrate
+
+
+def test_pipeline_without_bitrate() -> None:
+    assert CONF_BITRATE not in PIPELINE_SCHEMA(PIPELINE)
+
+
+@pytest.mark.parametrize("bitrate", [0, -48, 50, 48000])
+def test_pipeline_rejects_bitrates_the_encoder_cannot_use(bitrate: int) -> None:
+    with pytest.raises(cv.Invalid, match="Unknown value"):
+        PIPELINE_SCHEMA({**PIPELINE, CONF_FORMAT: "MP3", CONF_BITRATE: bitrate})
+
+
+def _final_validate(monkeypatch: pytest.MonkeyPatch, **overrides: Any) -> dict:
+    """Run the pipeline's final validator with the speaker checks isolated."""
+    monkeypatch.setattr(
+        audio, "final_validate_audio_schema", lambda *args, **kwargs: lambda c: c
+    )
+    # Both audio properties are set, so nothing is inherited from the speaker.
+    config = PIPELINE_SCHEMA(
+        {**PIPELINE, CONF_SAMPLE_RATE: 48000, CONF_NUM_CHANNELS: 1, **overrides}
+    )
+    return _validate_pipeline(config)
+
+
+def test_final_validation_keeps_mp3_bitrate(monkeypatch: pytest.MonkeyPatch) -> None:
+    config = _final_validate(monkeypatch, **{CONF_FORMAT: "MP3", CONF_BITRATE: 48})
+    assert config[CONF_BITRATE] == 48
+
+
+def test_final_validation_without_bitrate(monkeypatch: pytest.MonkeyPatch) -> None:
+    assert CONF_BITRATE not in _final_validate(monkeypatch, **{CONF_FORMAT: "FLAC"})
+
+
+@pytest.mark.parametrize("format_", ["FLAC", "NONE", "WAV"])
+def test_final_validation_rejects_bitrate_for_other_formats(
+    format_: str, monkeypatch: pytest.MonkeyPatch
+) -> None:
+    with pytest.raises(cv.Invalid, match="only supported for the MP3 format") as exc:
+        _final_validate(monkeypatch, **{CONF_FORMAT: format_, CONF_BITRATE: 48})
+    assert exc.value.path == [CONF_BITRATE]

tests/component_tests/vnc/test_init.py

@@ -0,0 +1,406 @@
+"""Code generation and validation tests for the vnc component."""
+
+from collections.abc import Callable
+from pathlib import Path
+import re
+
+import pytest
+
+from esphome.config import read_config
+from esphome.core import CORE
+
+TEST_CARD = "show_test_card()"
+UPDATE_NEVER = "set_update_interval(4294967295UL)"
+
+BASE = """
+esphome:
+  name: vnc-test
+
+esp32:
+  board: esp32dev
+
+logger:
+
+wifi:
+  ssid: MySSID
+  password: password1
+
+spi:
+  clk_pin: GPIO14
+  mosi_pin: GPIO12
+  miso_pin: GPIO13
+"""
+
+IT8951 = """
+display:
+  - platform: it8951
+    id: epd
+    model: m5stack-m5paper
+"""
+
+VNC = """
+vnc:
+  - host: 192.168.1.20
+"""
+
+EPAPER_SPI = """
+  - platform: epaper_spi
+    id: other
+    model: ssd1677
+    dimensions:
+      width: 200
+      height: 200
+    cs_pin: GPIO5
+    dc_pin: GPIO21
+    busy_pin: GPIO22
+    reset_pin: GPIO25
+"""
+
+
+S3_BASE = """
+esphome:
+  name: vnc-test
+
+esp32:
+  board: esp32-s3-devkitc-1
+  framework:
+    type: esp-idf
+
+psram:
+  mode: octal
+  speed: 80MHz
+
+wifi:
+  ssid: MySSID
+  password: password1
+"""
+
+RPI_DPI_RGB = """
+display:
+  - platform: rpi_dpi_rgb
+    id: rgb
+    dimensions:
+      width: 800
+      height: 480
+    de_pin: 40
+    hsync_pin: 39
+    vsync_pin: 41
+    pclk_pin: 42
+    data_pins:
+      red: [45, 48, 47, 21, 14]
+      green: [5, 6, 7, 15, 16, 4]
+      blue: [8, 3, 46, 9, 1]
+"""
+
+
+OPENTHREAD_BASE = """
+esphome:
+  name: vnc-test
+
+esp32:
+  variant: esp32c6
+  framework:
+    type: esp-idf
+
+openthread:
+  tlv: 0e080000000000010000000300001035060004001fffe00208e227ac6a7f24052f0708fdb753eb517cb4d3051062b2442a928d9ea3b947a1618fc4085a030f4f70656e5468726561642d393837330102987304105330d857354330133c05e1fd7ae81a910c0402a0f7f8
+
+spi:
+  clk_pin: GPIO6
+  mosi_pin: GPIO7
+"""
+
+C6_EPAPER = """
+display:
+  - platform: epaper_spi
+    id: epd
+    model: ssd1677
+    dimensions:
+      width: 200
+      height: 200
+    cs_pin: GPIO5
+    dc_pin: GPIO4
+    busy_pin: GPIO3
+    reset_pin: GPIO2
+"""
+
+
+@pytest.fixture(autouse=True)
+def _clear_toolchain() -> None:
+    # A host build earlier in the same session leaves the host toolchain selected.
+    CORE.toolchain = None
+
+
+@pytest.fixture
+def write_yaml(tmp_path: Path) -> Callable[..., Path]:
+    def writer(*parts: str, base: str = BASE) -> Path:
+        path = tmp_path / "vnc.yaml"
+        path.write_text(base + "".join(parts), encoding="utf-8")
+        return path
+
+    return writer
+
+
+@pytest.fixture
+def config_error(
+    write_yaml: Callable[..., Path], capsys: pytest.CaptureFixture[str]
+) -> Callable[..., str]:
+    """Validate a config that must fail and return the printed errors."""
+
+    def validate(*parts: str, base: str = BASE) -> str:
+        CORE.config_path = write_yaml(*parts, base=base)
+        assert read_config({}) is None
+        return capsys.readouterr().out
+
+    return validate
+
+
+def test_it8951_owned_by_vnc(
+    generate_main: Callable[[str | Path], str], write_yaml: Callable[..., Path]
+) -> None:
+    main_cpp = generate_main(write_yaml(IT8951, VNC))
+
+    assert TEST_CARD not in main_cpp
+    assert f"epd->{UPDATE_NEVER}" in main_cpp
+    assert re.search(r"new\(\w+\) vnc::VncClient\(epd, \w+\);", main_cpp)
+    assert 'set_host("192.168.1.20");' in main_cpp
+    assert "set_port(5900);" in main_cpp
+    assert "set_reconnect_interval(5000);" in main_cpp
+    assert "set_update_delay(200);" in main_cpp
+    assert "set_max_update_delay(1000);" in main_cpp
+
+
+def test_epaper_spi_owned_by_vnc(
+    generate_main: Callable[[str | Path], str], write_yaml: Callable[..., Path]
+) -> None:
+    main_cpp = generate_main(
+        write_yaml("display:", EPAPER_SPI, "\nvnc:\n  - host: 192.168.1.20\n")
+    )
+
+    assert TEST_CARD not in main_cpp
+    assert f"other->{UPDATE_NEVER}" in main_cpp
+    assert re.search(r"new\(\w+\) vnc::VncClient\(other, \w+\);", main_cpp)
+
+
+def test_display_with_generated_id(
+    generate_main: Callable[[str | Path], str], write_yaml: Callable[..., Path]
+) -> None:
+    main_cpp = generate_main(
+        write_yaml("display:\n  - platform: it8951\n    model: m5stack-m5paper\n", VNC)
+    )
+
+    assert TEST_CARD not in main_cpp
+    assert UPDATE_NEVER in main_cpp
+    assert "vnc::VncClient(" in main_cpp
+
+
+def test_only_the_vnc_display_loses_test_card(
+    generate_main: Callable[[str | Path], str], write_yaml: Callable[..., Path]
+) -> None:
+    main_cpp = generate_main(
+        write_yaml(
+            IT8951, EPAPER_SPI, "\nvnc:\n  - host: 192.168.1.20\n    display_id: epd\n"
+        )
+    )
+
+    assert f"other->{TEST_CARD}" in main_cpp
+    assert f"epd->{TEST_CARD}" not in main_cpp
+    assert f"epd->{UPDATE_NEVER}" in main_cpp
+
+
+def test_options(
+    generate_main: Callable[[str | Path], str], write_yaml: Callable[..., Path]
+) -> None:
+    main_cpp = generate_main(
+        write_yaml(
+            IT8951,
+            """
+vnc:
+  - host: vnc.local
+    port: 5901
+    reconnect_interval: 2s
+    update_delay: 0ms
+    max_update_delay: 3s
+    on_connect:
+      - logger.log: up
+    on_disconnect:
+      - logger.log: down
+""",
+        )
+    )
+
+    assert 'set_host("vnc.local");' in main_cpp
+    assert "set_port(5901);" in main_cpp
+    assert "set_reconnect_interval(2000);" in main_cpp
+    assert "set_update_delay(0);" in main_cpp
+    assert "set_max_update_delay(3000);" in main_cpp
+    assert "add_on_connect_callback(" in main_cpp
+    assert "add_on_disconnect_callback(" in main_cpp
+
+
+def test_two_clients_on_two_displays(
+    generate_main: Callable[[str | Path], str], write_yaml: Callable[..., Path]
+) -> None:
+    main_cpp = generate_main(
+        write_yaml(
+            IT8951,
+            EPAPER_SPI,
+            """
+vnc:
+  - host: 192.168.1.20
+    display_id: epd
+  - host: 192.168.1.21
+    display_id: other
+""",
+        )
+    )
+
+    assert re.search(r"vnc::VncClient\(epd, \w+\);", main_cpp)
+    assert re.search(r"vnc::VncClient\(other, \w+\);", main_cpp)
+    assert TEST_CARD not in main_cpp
+
+
+@pytest.mark.parametrize(
+    ("display_options", "error"),
+    [
+        pytest.param(
+            "    lambda: |-\n      it.fill(Color::BLACK);\n",
+            "vnc draws the whole of display 'epd'",
+            id="lambda",
+        ),
+        pytest.param(
+            "    pages:\n      - id: page1\n        lambda: |-\n"
+            "          it.fill(Color::BLACK);\n",
+            "vnc draws the whole of display 'epd'",
+            id="pages",
+        ),
+        pytest.param(
+            "    show_test_card: true\n",
+            "vnc draws the whole of display 'epd'",
+            id="show_test_card",
+        ),
+        pytest.param(
+            "    auto_clear_enabled: true\n",
+            "vnc draws the whole of display 'epd'",
+            id="auto_clear_enabled",
+        ),
+    ],
+)
+def test_display_with_own_writer_rejected(
+    display_options: str, error: str, config_error: Callable[..., str]
+) -> None:
+    assert error in config_error(IT8951, display_options, VNC)
+
+
+def test_lvgl_on_same_display_rejected(config_error: Callable[..., str]) -> None:
+    out = config_error(IT8951, VNC, "\nlvgl:\n  displays: [epd]\n")
+
+    assert "Display 'epd' is also used by LVGL" in out
+
+
+def test_lvgl_on_other_display_accepted(
+    write_yaml: Callable[..., Path],
+) -> None:
+    CORE.config_path = write_yaml(
+        IT8951,
+        EPAPER_SPI,
+        "\nvnc:\n  - host: 192.168.1.20\n    display_id: epd\n",
+        "\nlvgl:\n  displays: [other]\n",
+    )
+
+    assert read_config({}) is not None
+
+
+def test_two_clients_on_one_display_rejected(
+    config_error: Callable[..., str],
+) -> None:
+    out = config_error(
+        IT8951, "\nvnc:\n  - host: 192.168.1.20\n  - host: 192.168.1.21\n"
+    )
+
+    assert "Display 'epd' is drawn by more than one component: vnc, vnc" in out
+
+
+def test_mipi_spi_rejected(config_error: Callable[..., str]) -> None:
+    out = config_error(
+        """
+display:
+  - platform: mipi_spi
+    id: epd
+    model: st7735
+    cs_pin: GPIO15
+    dc_pin: GPIO27
+    reset_pin: GPIO26
+""",
+        VNC,
+    )
+
+    assert "Display 'epd' uses the mipi_spi platform" in out
+
+
+def test_rotated_rpi_dpi_rgb_accepted(write_yaml: Callable[..., Path]) -> None:
+    CORE.config_path = write_yaml(RPI_DPI_RGB, "    rotation: 90\n", VNC, base=S3_BASE)
+
+    assert read_config({}) is not None
+
+
+def test_ipv4_kept_on_with_openthread(
+    generate_main: Callable[[str | Path], str], write_yaml: Callable[..., Path]
+) -> None:
+    generate_main(write_yaml(C6_EPAPER, VNC, base=OPENTHREAD_BASE))
+
+    define = next(d for d in CORE.defines if d.name == "USE_NETWORK_IPV4")
+    assert str(define.value) == "true"
+
+
+def test_ipv4_disabled_rejected(config_error: Callable[..., str]) -> None:
+    out = config_error(
+        "network:\n  enable_ipv4: false\n", C6_EPAPER, VNC, base=OPENTHREAD_BASE
+    )
+
+    assert "IPv4 is required by vnc" in out
+
+
+def test_waveshare_epaper_rejected(config_error: Callable[..., str]) -> None:
+    out = config_error(
+        """
+display:
+  - platform: waveshare_epaper
+    id: epd
+    model: 1.54in
+    cs_pin: GPIO15
+    dc_pin: GPIO27
+    busy_pin: GPIO26
+    reset_pin: GPIO25
+""",
+        VNC,
+    )
+
+    assert "Display 'epd' uses the waveshare_epaper platform" in out
+    assert "use epaper_spi" in out
+
+
+@pytest.mark.parametrize(
+    ("vnc_options", "error"),
+    [
+        pytest.param(
+            "    reconnect_interval: 500ms\n",
+            "value must be at least 1s",
+            id="reconnect",
+        ),
+        pytest.param("    port: 0\n", "value must be at least 1", id="port"),
+    ],
+)
+def test_option_rejected(
+    vnc_options: str, error: str, config_error: Callable[..., str]
+) -> None:
+    out = config_error(IT8951, VNC, vnc_options)
+
+    assert error in out
+
+
+@pytest.mark.parametrize("host", ["::1", "fe80::1", "2001:db8::10"])
+def test_ipv6_host_rejected(host: str, config_error: Callable[..., str]) -> None:
+    out = config_error(IT8951, f'\nvnc:\n  - host: "{host}"\n')
+
+    assert "IPv6 addresses are not supported" in out

tests/components/api/common-base.yaml

@@ -78,6 +78,7 @@ api:
           type: string
           description: Name to greet
           example: World
+          default: World
       then:
         - logger.log:
             format: Hello World %s!

tests/components/ethernet/test-wisp.esp32-idf.yaml

@@ -0,0 +1,15 @@
+ethernet:
+  type: LAN8720
+  mdc_pin: 23
+  mdio_pin: 32
+  clk:
+    pin: 0
+    mode: CLK_EXT_IN
+  phy_addr: 0
+  power_pin: 33
+
+wifi:
+  ap:
+    ssid: WISP AP
+    password: wisp12345678
+    ap_mode: wisp

tests/components/light/common.yaml

@@ -145,6 +145,10 @@ esphome:
       - light.turn_on:
           id: test_monochromatic_light
           effect: !lambda 'return "Strobe";'
+      # Inlined lambda returning a StringRef, the shape an api action string variable takes (issue #21327)
+      - light.turn_on:
+          id: test_monochromatic_light
+          effect: !lambda 'return light::EFFECT_NONE_REF;'
       # Test effect lambda with non-empty args (repeat passes uint32_t iteration)
       - repeat:
           count: 3

tests/components/speaker_source/common.yaml

@@ -35,7 +35,8 @@ media_player:
     announcement_volume: 60%
     announcement_pipeline:
       speaker: announcement_mixer_speaker_id
-      format: FLAC
+      format: MP3
+      bitrate: 48
       num_channels: 1
       sources:
         - announcement_audio_file_source

tests/components/template/common-base.yaml

@@ -409,6 +409,10 @@ cover:
           id: template_cover_actions
           position: 0.0
           tilt: 0.0
+    toggle_action:
+      - cover.template.publish:
+          id: template_cover_actions
+          current_operation: IDLE
     stop_action:
       - cover.template.publish:
           id: template_cover_actions

tests/components/vnc/__init__.py

@@ -0,0 +1,15 @@
+# This file's presence makes pytest treat this directory as a package named
+# "vnc"; required for cpp unit testing.
+from esphome.components import socket as socket_component
+from esphome.types import ConfigType
+from tests.testing_helpers import ComponentManifestOverride
+
+
+def override_manifest(manifest: ComponentManifestOverride) -> None:
+    # MULTI_CONF components get no config in a unit test build, so to_code would never run.
+    manifest.multi_conf = False
+
+    async def to_code_testing(config: ConfigType) -> None:
+        socket_component.require_tcp_client_link()
+
+    manifest.to_code = to_code_testing

tests/components/vnc/common.yaml

@@ -0,0 +1,27 @@
+vnc:
+  - id: vnc_client
+    display_id: test_display_screen
+    host: 192.0.2.10
+    port: 5901
+    reconnect_interval: 10s
+    update_delay: 300ms
+    max_update_delay: 2s
+    on_connect:
+      - logger.log: VNC connected
+    on_disconnect:
+      - logger.log: VNC disconnected
+
+interval:
+  - interval: 60s
+    then:
+      - if:
+          condition:
+            vnc.is_connected: vnc_client
+          then:
+            - logger.log: VNC is connected
+      - if:
+          condition:
+            vnc.is_connected:
+              id: vnc_client
+          then:
+            - logger.log: VNC is still connected

tests/components/vnc/test.bk72xx-ard.yaml

@@ -0,0 +1,7 @@
+packages:
+  test_display: !include ../../test_build_components/common/test_display/test_display.yaml
+  vnc: !include common.yaml
+
+wifi:
+  ssid: MySSID
+  password: password1

tests/components/vnc/test.esp32-idf.yaml

@@ -0,0 +1,7 @@
+packages:
+  test_display: !include ../../test_build_components/common/test_display/test_display.yaml
+  vnc: !include common.yaml
+
+wifi:
+  ssid: MySSID
+  password: password1

tests/components/vnc/test.esp8266-ard.yaml

@@ -0,0 +1,7 @@
+packages:
+  test_display: !include ../../test_build_components/common/test_display/test_display.yaml
+  vnc: !include common.yaml
+
+wifi:
+  ssid: MySSID
+  password: password1

tests/components/vnc/test.host.yaml

@@ -0,0 +1,3 @@
+packages:
+  test_display: !include ../../test_build_components/common/test_display/test_display.yaml
+  vnc: !include common.yaml

tests/components/vnc/test.rp2040-ard.yaml

@@ -0,0 +1,7 @@
+packages:
+  test_display: !include ../../test_build_components/common/test_display/test_display.yaml
+  vnc: !include common.yaml
+
+wifi:
+  ssid: MySSID
+  password: password1

tests/components/vnc/test_backoff_host.cpp

@@ -0,0 +1,164 @@
+#include "vnc_test_helpers.h"
+
+#ifdef USE_HOST
+
+namespace esphome::vnc::testing {
+
+/// Runs one pass at time t, then lets an asynchronous connect finish before time moves on.
+static void settle_at(Session &s, uint32_t t) {
+  set_now(t);
+  s.loop();
+  for (int i = 0; i < 10000 && s.client.link().connecting(); i++) {
+    usleep(100);
+    s.loop();
+  }
+}
+
+/// Times at which refused connect attempts ended, starting at the current time.
+static std::vector<uint32_t> refused_attempts(Session &s, size_t count, uint32_t step = 100) {
+  std::vector<uint32_t> times;
+  uint32_t last_start = s.client.hold_start_ms();
+  const uint32_t start = now_ms();
+  for (uint32_t t = start; times.size() < count && t - start < 3600000; t += step) {
+    settle_at(s, t);
+    if (s.client.hold_start_ms() != last_start) {
+      last_start = s.client.hold_start_ms();
+      times.push_back(t);
+    }
+  }
+  return times;
+}
+
+static std::vector<uint32_t> gaps(const std::vector<uint32_t> &times) {
+  std::vector<uint32_t> out;
+  for (size_t i = 1; i < times.size(); i++) {
+    out.push_back(times[i] - times[i - 1]);
+  }
+  return out;
+}
+
+TEST(VncBackoff, RefusedConnectsBackOff) {
+  FakeDisplay d;
+  Session s(&d);
+  auto times = refused_attempts(s, 7);
+  ASSERT_EQ(times.size(), 7u);
+  EXPECT_EQ(times[0], 1000u);
+  EXPECT_EQ(gaps(times), std::vector<uint32_t>({5000, 10000, 20000, 40000, 60000, 60000}));
+  EXPECT_EQ(s.client.state(), VncState::VNC_STATE_IDLE);
+  EXPECT_TRUE(s.client.status_has_warning());
+  EXPECT_FALSE(s.client.status_has_error());
+}
+
+TEST(VncBackoff, ShortReconnectInterval) {
+  FakeDisplay d;
+  Session s(&d);
+  s.client.set_reconnect_interval(1000);
+  auto times = refused_attempts(s, 9);
+  EXPECT_EQ(gaps(times), std::vector<uint32_t>({1000, 2000, 4000, 8000, 16000, 32000, 60000, 60000}));
+}
+
+TEST(VncBackoff, ReconnectIntervalAboveTheCap) {
+  FakeDisplay d;
+  Session s(&d);
+  s.client.set_reconnect_interval(90000);
+  auto times = refused_attempts(s, 4, 1000);
+  EXPECT_EQ(gaps(times), std::vector<uint32_t>({90000, 90000, 90000}));
+}
+
+TEST(VncBackoff, StableSessionResetsBackoff) {
+  FakeDisplay d;
+  Session s(&d);
+  auto times = refused_attempts(s, 3);
+  ASSERT_EQ(s.client.backoff_ms(), 20000u);
+
+  set_now(times.back() + 1000);
+  s.start(64, 48);
+  s.run_for(60000, 1000);
+  ASSERT_TRUE(s.client.is_connected());
+  int disconnects = 0;
+  s.client.add_on_disconnect_callback([&disconnects]() { disconnects++; });
+  s.close_peer();
+  s.loop();
+  EXPECT_EQ(disconnects, 1);
+  EXPECT_EQ(s.client.state(), VncState::VNC_STATE_IDLE);
+  EXPECT_EQ(s.client.backoff_ms(), 0u);
+  EXPECT_EQ(s.client.hold_ms(), 0u);
+
+  const uint32_t dropped_at = now_ms();
+  auto after = refused_attempts(s, 2);
+  ASSERT_EQ(after.size(), 2u);
+  // The link's own interval spaces the first retry; the backoff then starts over.
+  EXPECT_EQ(after[0] - dropped_at, 5000u);
+  EXPECT_EQ(after[1] - after[0], 5000u);
+}
+
+TEST(VncBackoff, ShortSessionKeepsEscalating) {
+  FakeDisplay d;
+  Session s(&d);
+  auto times = refused_attempts(s, 3);
+  ASSERT_EQ(s.client.backoff_ms(), 20000u);
+
+  set_now(times.back() + 1000);
+  s.start(64, 48);
+  s.run_for(10000, 1000);
+  s.close_peer();
+  s.loop();
+  EXPECT_EQ(s.client.state(), VncState::VNC_STATE_IDLE);
+  EXPECT_EQ(s.client.hold_ms(), 40000u);
+}
+
+TEST(VncBackoff, PeerCloseWhileRunningRetries) {
+  FakeDisplay d;
+  Session s(&d);
+  int disconnects = 0;
+  s.client.add_on_disconnect_callback([&disconnects]() { disconnects++; });
+  s.start(64, 48);
+  s.close_peer();
+  s.loop();
+  EXPECT_FALSE(s.client.is_connected());
+  EXPECT_EQ(disconnects, 1);
+  EXPECT_EQ(s.client.hold_ms(), 5000u);
+  EXPECT_TRUE(s.client.status_has_warning());
+  const uint32_t dropped_at = now_ms();
+  auto after = refused_attempts(s, 1);
+  ASSERT_EQ(after.size(), 1u);
+  EXPECT_EQ(after[0] - dropped_at, 5000u);
+}
+
+TEST(VncBackoff, NextSessionStartsClean) {
+  FakeDisplay d;
+  Session s(&d);
+  s.start(64, 48);
+  set_now(now_ms() + 100);
+  // Leave a frame half parsed and a refresh pending, then lose the link.
+  s.deliver(Rfb().fbu(1).raw_rect(0, 0, 2, 1, {1, 2}));
+  s.deliver(Rfb().fbu(1).rect(0, 0, 4, 4, 0).u8(0x02));
+  ASSERT_TRUE(s.client.in_update());
+  s.close_peer();
+  s.loop();
+  EXPECT_FALSE(s.client.in_update());
+  EXPECT_FALSE(s.client.refresh_pending());
+
+  s.attach();
+  EXPECT_EQ(s.client.step(), VncStep::VNC_STEP_VERSION);
+  EXPECT_EQ(s.client.rx_buffered(), 0);
+  s.deliver(handshake_38(64, 48));
+  ASSERT_TRUE(s.client.is_connected());
+  s.take();
+  set_now(now_ms() + 100);
+  s.deliver(Rfb().fbu(1).raw_rect(3, 3, 1, 1, {0x7777}));
+  EXPECT_EQ(d.pixel(3, 3), expect_px(0x7777, 2));
+  EXPECT_EQ(hex(s.take()), hex(fbur(true, 64, 48)));
+}
+
+TEST(VncBackoff, ShutdownClosesTheLink) {
+  FakeDisplay d;
+  Session s(&d);
+  s.start(64, 48);
+  s.client.on_shutdown();
+  EXPECT_FALSE(s.client.link().connected());
+}
+
+}  // namespace esphome::vnc::testing
+
+#endif  // USE_HOST

tests/components/vnc/test_decode_host.cpp

@@ -0,0 +1,586 @@
+#include "vnc_test_helpers.h"
+
+#include <random>
+
+#ifdef USE_HOST
+
+namespace esphome::vnc::testing {
+
+static constexpr int32_t RAW = 0;
+
+static std::vector<uint16_t> pattern(size_t count, uint16_t seed) {
+  std::vector<uint16_t> out(count);
+  for (size_t i = 0; i < count; i++) {
+    out[i] = static_cast<uint16_t>(seed + i * 0x0841u);
+  }
+  return out;
+}
+
+/// Checks that a w x h block at x,y shows the given wire pixels, row-major.
+static void expect_block(const FakeDisplay &d, int x, int y, int w, int h, const std::vector<uint16_t> &px,
+                         uint8_t bpp) {
+  for (int row = 0; row < h; row++) {
+    for (int col = 0; col < w; col++) {
+      ASSERT_EQ(d.pixel(x + col, y + row), expect_px(px[row * w + col], bpp)) << "at " << x + col << "," << y + row;
+    }
+  }
+}
+
+static void expect_solid(const FakeDisplay &d, int x, int y, int w, int h, uint16_t px, uint8_t bpp) {
+  expect_block(d, x, y, w, h, std::vector<uint16_t>(static_cast<size_t>(w) * h, px), bpp);
+}
+
+class VncDecode : public ::testing::Test {
+ protected:
+  void start(FakeDisplay &display, uint16_t fb_w, uint16_t fb_h) {
+    this->session_ = std::make_unique<Session>(&display);
+    this->session_->start(fb_w, fb_h);
+    // Past the 50 ms request spacing, so the next request goes out as soon as a frame ends.
+    set_now(now_ms() + 100);
+  }
+  Session &s() { return *this->session_; }
+  std::unique_ptr<Session> session_;
+};
+
+TEST_F(VncDecode, Raw16bpp) {
+  FakeDisplay d(64, 48);
+  this->start(d, 64, 48);
+  auto px = pattern(12, 0x1234);
+  s().deliver(Rfb().fbu(1).raw_rect(3, 2, 4, 3, px));
+  expect_block(d, 3, 2, 4, 3, px, 2);
+  EXPECT_EQ(d.drawn_count(), 12u);
+  EXPECT_EQ(hex(s().take()), hex(fbur(true, 64, 48)));
+  EXPECT_EQ(d.updates(), 0);
+  s().run_for(300);
+  EXPECT_EQ(d.updates(), 1);
+}
+
+TEST_F(VncDecode, Raw8bpp) {
+  FakeDisplay d(64, 48, display::DISPLAY_TYPE_BINARY);
+  this->start(d, 64, 48);
+  std::vector<uint16_t> px = {0x00, 0xFF, 0xE0, 0x1C, 0x03, 0x92, 0x49, 0x24, 0xB6, 0x6D};
+  s().deliver(Rfb().fbu(1).raw_rect(10, 20, 5, 2, px, 1));
+  expect_block(d, 10, 20, 5, 2, px, 1);
+  EXPECT_EQ(d.drawn_count(), 10u);
+  EXPECT_TRUE(s().client.is_connected());
+}
+
+TEST_F(VncDecode, RawSeveralRectsInOneUpdate) {
+  FakeDisplay d(64, 48);
+  this->start(d, 64, 48);
+  auto a = pattern(6, 0x0101);
+  auto b = pattern(4, 0xF00F);
+  s().deliver(Rfb().fbu(2).raw_rect(0, 0, 3, 2, a).raw_rect(60, 46, 4, 1, b));
+  expect_block(d, 0, 0, 3, 2, a, 2);
+  expect_block(d, 60, 46, 4, 1, b, 2);
+  EXPECT_EQ(hex(s().take()), hex(fbur(true, 64, 48)));
+}
+
+TEST_F(VncDecode, EmptyUpdateRequestsAgainWithoutRefresh) {
+  FakeDisplay d(64, 48);
+  this->start(d, 64, 48);
+  s().deliver(Rfb().fbu(0));
+  EXPECT_EQ(hex(s().take()), hex(fbur(true, 64, 48)));
+  s().run_for(2000);
+  EXPECT_EQ(d.updates(), 0);
+}
+
+TEST_F(VncDecode, RichCursorIsSkippedAndNeverRefreshes) {
+  FakeDisplay d(64, 48);
+  this->start(d, 64, 48);
+  s().deliver(Rfb().fbu(1).cursor(3, 4, 5, 4));
+  EXPECT_EQ(d.draw_calls(), 0);
+  EXPECT_EQ(hex(s().take()), hex(fbur(true, 64, 48)));
+  s().run_for(3000);
+  EXPECT_EQ(d.updates(), 0);
+  EXPECT_TRUE(s().client.is_connected());
+}
+
+class VncCursorSkip : public ::testing::TestWithParam<display::DisplayType> {};
+
+TEST_P(VncCursorSkip, PayloadLengthMatchesThePixelFormat) {
+  FakeDisplay d(64, 48, GetParam());
+  Session s(&d);
+  s.start(64, 48);
+  const uint8_t bpp = s.client.bpp();
+  // 9 wide needs a 2-byte mask row; the raw rect after it only lands right if the skip was exact.
+  s.deliver(Rfb().fbu(3).cursor(0, 0, 9, 3, bpp).raw_rect(7, 7, 2, 1, {0x00E0, 0x001C}, bpp).cursor(1, 1, 0, 0, bpp));
+  EXPECT_EQ(d.pixel(7, 7), expect_px(0x00E0, bpp));
+  EXPECT_EQ(d.pixel(8, 7), expect_px(0x001C, bpp));
+  EXPECT_EQ(d.drawn_count(), 2u);
+  EXPECT_TRUE(s.client.is_connected());
+  EXPECT_FALSE(s.client.in_update());
+}
+
+INSTANTIATE_TEST_SUITE_P(Formats, VncCursorSkip,
+                         ::testing::Values(display::DISPLAY_TYPE_COLOR, display::DISPLAY_TYPE_BINARY));
+
+TEST_F(VncDecode, ZeroSizeCursorEndsTheFrame) {
+  FakeDisplay d(64, 48);
+  this->start(d, 64, 48);
+  s().deliver(Rfb().fbu(1).cursor(0, 0, 0, 0));
+  EXPECT_FALSE(s().client.in_update());
+  EXPECT_EQ(hex(s().take()), hex(fbur(true, 64, 48)));
+}
+
+TEST_F(VncDecode, OversizedCursorReconnects) {
+  FakeDisplay d(64, 48);
+  this->start(d, 64, 48);
+  s().deliver(Rfb().fbu(1).rect(0, 0, 1025, 1, -239));
+  EXPECT_EQ(s().client.state(), VncState::VNC_STATE_IDLE);
+}
+
+TEST_F(VncDecode, LargeCutTextIsSkipped) {
+  FakeDisplay d(64, 48);
+  this->start(d, 64, 48);
+  s().deliver(Rfb().cut_text(200 * 1024).fbu(1).raw_rect(1, 1, 1, 1, {0x4321}));
+  EXPECT_TRUE(s().client.is_connected());
+  EXPECT_EQ(d.pixel(1, 1), expect_px(0x4321, 2));
+  EXPECT_EQ(d.drawn_count(), 1u);
+}
+
+TEST_F(VncDecode, BellAndColourMapAreSkipped) {
+  FakeDisplay d(64, 48);
+  this->start(d, 64, 48);
+  s().deliver(Rfb().bell().colour_map(5, 3).bell().colour_map(0, 0).fbu(1).raw_rect(2, 2, 1, 1, {0x0F0F}));
+  EXPECT_TRUE(s().client.is_connected());
+  EXPECT_EQ(d.pixel(2, 2), expect_px(0x0F0F, 2));
+  EXPECT_EQ(d.drawn_count(), 1u);
+}
+
+TEST(VncDecodeClip, ServerLargerThanDisplay) {
+  FakeDisplay d(64, 48);
+  Session s(&d);
+  s.attach();
+  s.deliver(handshake_38(100, 80));
+  ASSERT_TRUE(s.client.is_connected());
+  Bytes tx = s.take();
+  EXPECT_EQ(hex(Bytes(tx.end() - 10, tx.end())), hex(fbur(false, 64, 48)));
+  set_now(now_ms() + 100);
+
+  auto raw = pattern(90 * 70, 0x0021);
+  Rfb f;
+  f.fbu(3).raw_rect(5, 5, 90, 70, raw);
+  // A raw rect straddling both edges, and one entirely off screen.
+  f.raw_rect(56, 40, 20, 20, std::vector<uint16_t>(20 * 20, 0x5000));
+  f.raw_rect(70, 50, 16, 16, pattern(256, 1));
+  s.deliver(f);
+  EXPECT_EQ(d.out_of_bounds(), 0);
+  for (int y = 5; y < 40; y++) {
+    for (int x = 5; x < 56; x++) {
+      ASSERT_EQ(d.pixel(x, y), expect_px(raw[(y - 5) * 90 + (x - 5)], 2)) << x << "," << y;
+    }
+  }
+  expect_solid(d, 56, 40, 8, 8, 0x5000, 2);
+  EXPECT_EQ(d.pixel(4, 4), UNSET);
+  EXPECT_TRUE(s.client.is_connected());
+  EXPECT_EQ(hex(s.take()), hex(fbur(true, 64, 48)));
+}
+
+TEST(VncDecodeClip, ServerSmallerThanDisplay) {
+  FakeDisplay d(64, 48);
+  Session s(&d);
+  s.start(20, 10);
+  EXPECT_EQ(s.client.vis_w(), 20);
+  EXPECT_EQ(s.client.vis_h(), 10);
+  s.deliver(Rfb().fbu(1).raw_rect(19, 9, 1, 1, {0x1111}));
+  EXPECT_EQ(d.pixel(19, 9), expect_px(0x1111, 2));
+  s.deliver(Rfb().fbu(1).raw_rect(20, 0, 1, 1, {0x1111}));
+  EXPECT_EQ(s.client.state(), VncState::VNC_STATE_IDLE);
+}
+
+TEST_F(VncDecode, UnknownMessageTypeReconnects) {
+  FakeDisplay d(64, 48);
+  this->start(d, 64, 48);
+  int disconnects = 0;
+  s().client.add_on_disconnect_callback([&disconnects]() { disconnects++; });
+  s().deliver(Rfb().u8(7));
+  EXPECT_EQ(s().client.state(), VncState::VNC_STATE_IDLE);
+  EXPECT_FALSE(s().client.link().connected());
+  EXPECT_EQ(s().client.hold_ms(), 5000u);
+  EXPECT_EQ(disconnects, 1);
+}
+
+TEST_F(VncDecode, UnsupportedEncodingReconnects) {
+  FakeDisplay d(64, 48);
+  this->start(d, 64, 48);
+  // CopyRect, which the client never asked for.
+  s().deliver(Rfb().fbu(1).rect(0, 0, 4, 4, 1).u16(0).u16(0));
+  EXPECT_EQ(s().client.state(), VncState::VNC_STATE_IDLE);
+}
+
+struct OutsideCase {
+  uint16_t x, y, w, h;
+  int32_t encoding;
+};
+
+class VncRectOutside : public ::testing::TestWithParam<OutsideCase> {};
+
+TEST_P(VncRectOutside, Reconnects) {
+  FakeDisplay d(64, 48);
+  Session s(&d);
+  s.start(64, 48);
+  const auto &c = GetParam();
+  s.deliver(Rfb().fbu(1).rect(c.x, c.y, c.w, c.h, c.encoding).fill(64, 0));
+  EXPECT_EQ(s.client.state(), VncState::VNC_STATE_IDLE);
+  EXPECT_EQ(d.draw_calls(), 0);
+}
+
+INSTANTIATE_TEST_SUITE_P(Rects, VncRectOutside,
+                         ::testing::Values(OutsideCase{60, 0, 5, 1, RAW}, OutsideCase{0, 47, 1, 2, RAW},
+                                           OutsideCase{65535, 0, 2, 1, RAW}, OutsideCase{0, 65535, 1, 2, RAW},
+                                           OutsideCase{64, 0, 1, 1, RAW}, OutsideCase{0, 0, 65, 1, RAW}));
+
+// --- A frame whose last step needs no bytes must still finish when its last byte arrives.
+
+class VncZeroByteEnd : public VncDecode {
+ protected:
+  void deliver_bytewise(const Bytes &stream, size_t frame_end) {
+    for (size_t i = 0; i < stream.size(); i++) {
+      s().deliver(Bytes{stream[i]});
+      Bytes tx = s().take();
+      if (i + 1 == frame_end) {
+        EXPECT_EQ(hex(tx), hex(fbur(true, 64, 48))) << "no request after the final byte";
+      } else {
+        EXPECT_EQ(tx.size(), 0u) << "unexpected bytes after byte " << i;
+      }
+    }
+    EXPECT_FALSE(s().client.in_update());
+    EXPECT_EQ(s().client.step(), VncStep::VNC_STEP_MESSAGE);
+    EXPECT_EQ(s().client.rx_buffered(), 0);
+  }
+};
+
+TEST_F(VncZeroByteEnd, ZeroSizeCursor) {
+  FakeDisplay d(64, 48);
+  this->start(d, 64, 48);
+  Rfb f;
+  f.fbu(2).raw_rect(0, 0, 2, 1, {0x7001, 0x7002}).cursor(4, 4, 0, 0);
+  this->deliver_bytewise(f.bytes, f.bytes.size());
+  s().run_for(300);
+  EXPECT_EQ(d.updates(), 1);
+}
+
+TEST_F(VncZeroByteEnd, ZeroSizeRawRect) {
+  FakeDisplay d(64, 48);
+  this->start(d, 64, 48);
+  Rfb f;
+  f.fbu(2).raw_rect(0, 0, 1, 1, {0x7001}).rect(64, 48, 0, 0, RAW);
+  this->deliver_bytewise(f.bytes, f.bytes.size());
+  s().run_for(300);
+  EXPECT_EQ(d.updates(), 1);
+}
+
+TEST_F(VncZeroByteEnd, EmptyCutTextAfterFrame) {
+  FakeDisplay d(64, 48);
+  this->start(d, 64, 48);
+  Rfb f;
+  f.fbu(1).raw_rect(0, 0, 1, 1, {0x7001});
+  const size_t frame_end = f.bytes.size();
+  f.cut_text(0).colour_map(0, 0);
+  this->deliver_bytewise(f.bytes, frame_end);
+  s().run_for(300);
+  EXPECT_EQ(d.updates(), 1);
+  s().deliver(Rfb().fbu(1).raw_rect(1, 0, 1, 1, {0x7002}));
+  EXPECT_EQ(d.pixel(1, 0), expect_px(0x7002, 2));
+}
+
+// --- The 20 ms budget: with a slow display, one row per pass.
+
+TEST_F(VncDecode, BudgetPausesBetweenRectsAndFinishesOnTheLastOne) {
+  FakeDisplay d(64, 48);
+  this->start(d, 64, 48);
+  d.set_draw_delay_us(25000);
+  Rfb f;
+  f.fbu(3)
+      .raw_rect(0, 0, 16, 1, std::vector<uint16_t>(16, 0x0A0A))
+      .raw_rect(16, 0, 16, 1, std::vector<uint16_t>(16, 0x0B0B))
+      .raw_rect(32, 0, 16, 1, std::vector<uint16_t>(16, 0x0C0C));
+  s().send(f);
+  s().loop();
+  EXPECT_EQ(d.draw_calls(), 1);
+  EXPECT_TRUE(s().client.in_update());
+  EXPECT_TRUE(s().client.rx_pending());
+  EXPECT_TRUE(HighFrequencyLoopRequester::is_high_frequency());
+  s().loop();
+  EXPECT_EQ(d.draw_calls(), 2);
+  s().loop();
+  EXPECT_EQ(d.draw_calls(), 3);
+  EXPECT_FALSE(s().client.in_update());
+  EXPECT_EQ(hex(s().take()), hex(fbur(true, 64, 48)));
+  d.set_draw_delay_us(0);
+  s().loop(2);
+  EXPECT_FALSE(s().client.rx_pending());
+  EXPECT_FALSE(HighFrequencyLoopRequester::is_high_frequency());
+  expect_solid(d, 32, 0, 16, 1, 0x0C0C, 2);
+}
+
+TEST_F(VncDecode, BudgetPausesInsideARawRect) {
+  FakeDisplay d(64, 48);
+  this->start(d, 64, 48);
+  d.set_draw_delay_us(25000);
+  auto px = pattern(4 * 3, 0x2000);
+  s().send(Rfb().fbu(1).raw_rect(0, 0, 4, 3, px));
+  s().loop();
+  EXPECT_EQ(d.draw_calls(), 1);
+  EXPECT_TRUE(s().client.in_update());
+  s().loop(2);
+  EXPECT_EQ(d.draw_calls(), 3);
+  EXPECT_FALSE(s().client.in_update());
+  d.set_draw_delay_us(0);
+  expect_block(d, 0, 0, 4, 3, px, 2);
+}
+
+// --- Split-feed property: however the bytes are cut, the result is the same and matches a model.
+
+struct Model {
+  int fb_w{0};
+  int fb_h{0};
+  uint8_t bpp{2};
+  std::vector<int32_t> px;
+  void set(int x, int y, uint16_t v) { this->px[static_cast<size_t>(y) * this->fb_w + x] = v; }
+};
+
+struct Generated {
+  int disp_w{0};
+  int disp_h{0};
+  display::DisplayType type{display::DISPLAY_TYPE_COLOR};
+  uint16_t vis_w{0};
+  uint16_t vis_h{0};
+  Bytes handshake;
+  Bytes handshake_tx;
+  std::vector<Bytes> frames;
+  Model model;
+};
+
+static Generated generate(uint32_t seed) {
+  std::mt19937 rng(seed);
+  auto rnd = [&rng](int lo, int hi) { return std::uniform_int_distribution<int>(lo, hi)(rng); };
+  Generated g;
+  g.disp_w = rnd(4, 70);
+  g.disp_h = rnd(4, 50);
+  const int type = rnd(0, 2);
+  g.type = type == 0 ? display::DISPLAY_TYPE_COLOR
+                     : (type == 1 ? display::DISPLAY_TYPE_GRAYSCALE : display::DISPLAY_TYPE_BINARY);
+  const uint8_t bpp = g.type == display::DISPLAY_TYPE_BINARY ? 1 : 2;
+  const int fb_w = rnd(1, 90);
+  const int fb_h = rnd(1, 64);
+  g.vis_w = std::min(fb_w, g.disp_w);
+  g.vis_h = std::min(fb_h, g.disp_h);
+  g.model.fb_w = fb_w;
+  g.model.fb_h = fb_h;
+  g.model.bpp = bpp;
+  g.model.px.assign(static_cast<size_t>(fb_w) * fb_h, -1);
+  auto pixel = [&]() { return static_cast<uint16_t>(bpp == 2 ? rnd(0, 0xFFFF) : rnd(0, 0xFF)); };
+
+  Rfb hs;
+  switch (rnd(0, 2)) {
+    case 0:
+      hs.version("RFB 003.003\n").security_33(1);
+      g.handshake_tx = cat({version_reply('3'), {1}});
+      break;
+    case 1:
+      hs.version("RFB 003.007\n").security_list({1});
+      g.handshake_tx = cat({version_reply('7'), {1}, {1}});
+      break;
+    default:
+      hs.version().security_list({16, 1}).security_result(0);
+      g.handshake_tx = cat({version_reply('8'), {1}, {1}});
+      break;
+  }
+  hs.server_init(fb_w, fb_h, std::string(rnd(0, 80), 'n'));
+  g.handshake = hs.bytes;
+  g.handshake_tx = cat({g.handshake_tx, bpp == 2 ? set_pixel_format_565() : set_pixel_format_332(), set_encodings(),
+                        fbur(false, g.vis_w, g.vis_h)});
+
+  const int frames = rnd(3, 8);
+  for (int frame = 0; frame < frames; frame++) {
+    Rfb f;
+    for (int extra = rnd(0, 2); extra > 0; extra--) {
+      switch (rnd(0, 2)) {
+        case 0:
+          f.bell();
+          break;
+        case 1:
+          f.cut_text(rnd(0, 700));
+          break;
+        default:
+          f.colour_map(rnd(0, 255), rnd(0, 4));
+          break;
+      }
+    }
+    const int rects = rnd(0, 5);
+    f.fbu(rects);
+    for (int r = 0; r < rects; r++) {
+      const int kind = rnd(0, 9);
+      if (kind == 8) {
+        f.cursor(rnd(0, 100), rnd(0, 100), rnd(0, 9), rnd(0, 9), bpp);
+        continue;
+      }
+      if (kind == 9) {
+        const int x = rnd(0, fb_w);
+        const int y = rnd(0, fb_h);
+        f.rect(x, y, rnd(0, 1) ? 0 : rnd(0, fb_w - x), 0, RAW);
+        continue;
+      }
+      const int x = rnd(0, fb_w - 1);
+      const int y = rnd(0, fb_h - 1);
+      const int w = rnd(1, std::min(fb_w - x, 48));
+      const int h = rnd(1, std::min(fb_h - y, 40));
+      f.rect(x, y, w, h, RAW);
+      for (int yy = 0; yy < h; yy++) {
+        for (int xx = 0; xx < w; xx++) {
+          uint16_t p = pixel();
+          f.px(p, bpp);
+          g.model.set(x + xx, y + yy, p);
+        }
+      }
+    }
+    g.frames.push_back(f.bytes);
+  }
+  return g;
+}
+
+static void expect_matches_model(const FakeDisplay &d, const Generated &g, uint32_t seed) {
+  for (int y = 0; y < g.disp_h; y++) {
+    for (int x = 0; x < g.disp_w; x++) {
+      uint32_t expected = UNSET;
+      if (x < g.vis_w && y < g.vis_h) {
+        const int32_t v = g.model.px[static_cast<size_t>(y) * g.model.fb_w + x];
+        if (v >= 0) {
+          expected = expect_px(static_cast<uint16_t>(v), g.model.bpp);
+        }
+      }
+      ASSERT_EQ(d.pixel(x, y), expected) << "seed " << seed << " at " << x << "," << y;
+    }
+  }
+}
+
+enum class Cut { WHOLE, BYTES, RANDOM };
+
+static std::vector<Bytes> cut(const Bytes &stream, Cut how, std::mt19937 &rng) {
+  std::vector<Bytes> chunks;
+  size_t pos = 0;
+  while (pos < stream.size()) {
+    size_t len = stream.size() - pos;
+    if (how == Cut::BYTES) {
+      len = 1;
+    } else if (how == Cut::RANDOM) {
+      len = std::min<size_t>(len, std::uniform_int_distribution<size_t>(1, 700)(rng));
+    }
+    chunks.emplace_back(stream.begin() + pos, stream.begin() + pos + len);
+    pos += len;
+  }
+  return chunks;
+}
+
+struct FeedResult {
+  std::vector<uint32_t> pixels;
+  Bytes tx;
+};
+
+static FeedResult run_parser(const Generated &g, Cut how, uint32_t seed) {
+  FakeDisplay d(g.disp_w, g.disp_h, g.type);
+  Session s(&d);
+  s.attach();
+  Bytes stream = g.handshake;
+  for (const auto &frame : g.frames) {
+    stream.insert(stream.end(), frame.begin(), frame.end());
+  }
+  std::mt19937 rng(seed * 7919u + static_cast<uint32_t>(how));
+  for (const auto &chunk : cut(stream, how, rng)) {
+    EXPECT_EQ(s.client.feed(chunk), VncParse::VNC_PARSE_MORE) << "seed " << seed;
+  }
+  EXPECT_EQ(s.client.state(), VncState::VNC_STATE_RUNNING) << "seed " << seed;
+  EXPECT_EQ(s.client.step(), VncStep::VNC_STEP_MESSAGE) << "seed " << seed;
+  EXPECT_EQ(s.client.rx_buffered(), 0) << "seed " << seed;
+  EXPECT_FALSE(s.client.in_update()) << "seed " << seed;
+  EXPECT_EQ(d.out_of_bounds(), 0);
+  expect_matches_model(d, g, seed);
+  return {d.pixels(), s.take()};
+}
+
+TEST(VncSplitFeed, ParserGivesTheSameResultHoweverTheBytesAreCut) {
+  for (uint32_t seed = 1; seed <= 50; seed++) {
+    const Generated g = generate(seed);
+    const FeedResult whole = run_parser(g, Cut::WHOLE, seed);
+    EXPECT_EQ(hex(whole.tx), hex(g.handshake_tx)) << "seed " << seed;
+    const FeedResult bytes = run_parser(g, Cut::BYTES, seed);
+    const FeedResult random = run_parser(g, Cut::RANDOM, seed);
+    EXPECT_EQ(whole.pixels, bytes.pixels) << "seed " << seed;
+    EXPECT_EQ(whole.pixels, random.pixels) << "seed " << seed;
+    EXPECT_EQ(hex(whole.tx), hex(bytes.tx)) << "seed " << seed;
+    EXPECT_EQ(hex(whole.tx), hex(random.tx)) << "seed " << seed;
+    if (::testing::Test::HasFailure()) {
+      break;
+    }
+  }
+}
+
+/// End to end over the socket: the server sends each frame only after the client asked for it.
+static FeedResult run_socket(const Generated &g, Cut how, uint32_t seed) {
+  FakeDisplay d(g.disp_w, g.disp_h, g.type);
+  Session s(&d);
+  s.attach();
+  s.deliver(g.handshake);
+  EXPECT_EQ(s.client.state(), VncState::VNC_STATE_RUNNING) << "seed " << seed;
+  EXPECT_EQ(hex(s.take()), hex(g.handshake_tx)) << "seed " << seed;
+  std::mt19937 rng(seed * 104729u + static_cast<uint32_t>(how));
+  Bytes tx;
+  auto collect = [&]() {
+    Bytes more = s.take();
+    tx.insert(tx.end(), more.begin(), more.end());
+  };
+  auto wait_for_requests = [&](int count) {
+    for (int i = 0; i < 100 && count_fburs(tx, true) < count; i++) {
+      s.tick(10);
+      collect();
+    }
+    EXPECT_EQ(count_fburs(tx, true), count) << "seed " << seed;
+  };
+  for (size_t k = 0; k < g.frames.size(); k++) {
+    if (k > 0) {
+      wait_for_requests(static_cast<int>(k));
+    }
+    for (const auto &chunk : cut(g.frames[k], how, rng)) {
+      s.send(chunk);
+      for (int i = 0; i < 1000 && (s.server_bytes_waiting() || s.unread() != 0 || s.client.rx_pending()); i++) {
+        s.tick(1);
+      }
+      collect();
+    }
+  }
+  wait_for_requests(static_cast<int>(g.frames.size()));
+  s.run_for(2000);
+  collect();
+  EXPECT_EQ(s.client.state(), VncState::VNC_STATE_RUNNING) << "seed " << seed;
+  EXPECT_EQ(d.out_of_bounds(), 0);
+  expect_matches_model(d, g, seed);
+  if (d.drawn_count() != 0) {
+    EXPECT_GE(d.updates(), 1) << "seed " << seed;
+  }
+  Bytes expected_tx;
+  for (size_t i = 0; i < g.frames.size(); i++) {
+    expected_tx = cat({expected_tx, fbur(true, g.vis_w, g.vis_h)});
+  }
+  EXPECT_EQ(hex(tx), hex(expected_tx)) << "seed " << seed;
+  return {d.pixels(), tx};
+}
+
+TEST(VncSplitFeed, SocketGivesTheSameResultHoweverTheBytesAreCut) {
+  for (uint32_t seed = 101; seed <= 130; seed++) {
+    const Generated g = generate(seed);
+    const FeedResult whole = run_socket(g, Cut::WHOLE, seed);
+    const FeedResult bytes = run_socket(g, Cut::BYTES, seed);
+    const FeedResult random = run_socket(g, Cut::RANDOM, seed);
+    EXPECT_EQ(whole.pixels, bytes.pixels) << "seed " << seed;
+    EXPECT_EQ(whole.pixels, random.pixels) << "seed " << seed;
+    if (::testing::Test::HasFailure()) {
+      break;
+    }
+  }
+}
+
+}  // namespace esphome::vnc::testing
+
+#endif  // USE_HOST

tests/components/vnc/test_handshake_host.cpp

@@ -0,0 +1,342 @@
+#include "vnc_test_helpers.h"
+
+#ifdef USE_HOST
+
+namespace esphome::vnc::testing {
+
+static Bytes running_tail(const Bytes &pixel_format, uint16_t w, uint16_t h) {
+  return cat({pixel_format, set_encodings(), fbur(false, w, h)});
+}
+
+TEST(VncHandshake, Rfb38NoneExactBytes) {
+  FakeDisplay display(64, 48);
+  Session s(&display);
+  int connects = 0;
+  s.client.add_on_connect_callback([&connects]() { connects++; });
+  s.attach();
+  EXPECT_EQ(s.client.state(), VncState::VNC_STATE_HANDSHAKE);
+
+  s.deliver(Rfb().version());
+  EXPECT_EQ(hex(s.take()), hex(version_reply('8')));
+  s.deliver(Rfb().security_list({1}));
+  EXPECT_EQ(hex(s.take()), hex({1}));
+  s.deliver(Rfb().security_result(0));
+  EXPECT_EQ(hex(s.take()), hex({1}));  // ClientInit, shared
+  EXPECT_FALSE(s.client.is_connected());
+  s.deliver(Rfb().server_init(80, 40));
+  // The first request covers only what the display can show.
+  EXPECT_EQ(hex(s.take()), hex(running_tail(set_pixel_format_565(), 64, 40)));
+  EXPECT_TRUE(s.client.is_connected());
+  EXPECT_EQ(connects, 1);
+}
+
+TEST(VncHandshake, Rfb38NonePipelined) {
+  FakeDisplay display(64, 48);
+  Session s(&display);
+  s.attach();
+  s.deliver(handshake_38(32, 100));
+  EXPECT_EQ(hex(s.take()), hex(cat({version_reply('8'), {1}, {1}, running_tail(set_pixel_format_565(), 32, 48)})));
+  EXPECT_TRUE(s.client.is_connected());
+}
+
+struct PixelFormatCase {
+  display::DisplayType type;
+  bool rgb565;
+};
+
+class VncPixelFormatSelection : public ::testing::TestWithParam<PixelFormatCase> {};
+
+TEST_P(VncPixelFormatSelection, SetPixelFormatMatchesTheDisplay) {
+  const auto &param = GetParam();
+  FakeDisplay display(64, 48, param.type);
+  Session s(&display);
+  s.attach();
+  s.deliver(handshake_38(64, 48));
+  Bytes tx = s.take();
+  const Bytes expected = running_tail(param.rgb565 ? set_pixel_format_565() : set_pixel_format_332(), 64, 48);
+  ASSERT_GE(tx.size(), expected.size());
+  EXPECT_EQ(hex(Bytes(tx.end() - expected.size(), tx.end())), hex(expected));
+  EXPECT_EQ(s.client.bpp(), param.rgb565 ? 2 : 1);
+}
+
+INSTANTIATE_TEST_SUITE_P(Formats, VncPixelFormatSelection,
+                         ::testing::Values(PixelFormatCase{display::DISPLAY_TYPE_COLOR, true},
+                                           PixelFormatCase{display::DISPLAY_TYPE_GRAYSCALE, true},
+                                           PixelFormatCase{display::DISPLAY_TYPE_BINARY, false}));
+
+TEST(VncHandshake, Rfb37NoneHasNoSecurityResult) {
+  FakeDisplay display(64, 48);
+  Session s(&display);
+  s.attach();
+  s.deliver(Rfb().version("RFB 003.007\n"));
+  EXPECT_EQ(hex(s.take()), hex(version_reply('7')));
+  s.deliver(Rfb().security_list({1}));
+  // Type and ClientInit go out together: 3.7 sends no SecurityResult for None.
+  EXPECT_EQ(hex(s.take()), hex({1, 1}));
+  s.deliver(Rfb().server_init(64, 48));
+  EXPECT_EQ(hex(s.take()), hex(running_tail(set_pixel_format_565(), 64, 48)));
+  EXPECT_TRUE(s.client.is_connected());
+}
+
+TEST(VncHandshake, Rfb33None) {
+  FakeDisplay display(64, 48);
+  Session s(&display);
+  s.attach();
+  s.deliver(Rfb().version("RFB 003.003\n"));
+  EXPECT_EQ(hex(s.take()), hex(version_reply('3')));
+  s.deliver(Rfb().security_33(1));
+  EXPECT_EQ(hex(s.take()), hex({1}));
+  s.deliver(Rfb().server_init(64, 48));
+  EXPECT_EQ(hex(s.take()), hex(running_tail(set_pixel_format_565(), 64, 48)));
+  EXPECT_TRUE(s.client.is_connected());
+}
+
+TEST(VncHandshake, Rfb33RefusalIsTransient) {
+  FakeDisplay display(64, 48);
+  Session s(&display);
+  s.attach();
+  s.deliver(Rfb().version("RFB 003.003\n").security_33(0).reason("Too many security failures"));
+  EXPECT_EQ(s.client.state(), VncState::VNC_STATE_IDLE);
+  EXPECT_EQ(s.client.hold_ms(), 5000u);
+  EXPECT_FALSE(s.client.status_has_error());
+}
+
+TEST(VncHandshake, Rfb33PasswordWithoutKeyIsConfig) {
+  FakeDisplay display(64, 48);
+  Session s(&display);
+  s.attach();
+  s.deliver(Rfb().version("RFB 003.003\n"));
+  EXPECT_EQ(hex(s.take()), hex(version_reply('3')));
+  s.deliver(Rfb().security_33(2).challenge());
+  EXPECT_EQ(s.take().size(), 0u);
+  EXPECT_EQ(s.client.state(), VncState::VNC_STATE_IDLE);
+  EXPECT_EQ(s.client.hold_ms(), 300000u);
+  EXPECT_TRUE(s.client.status_has_error());
+}
+
+struct VersionCase {
+  const char *server;
+  char reply_minor;
+};
+
+class VncVersionMapping : public ::testing::TestWithParam<VersionCase> {};
+
+TEST_P(VncVersionMapping, RepliesWithTheMappedVersion) {
+  FakeDisplay display(64, 48);
+  Session s(&display);
+  s.attach();
+  s.deliver(text(GetParam().server));
+  EXPECT_EQ(hex(s.take()), hex(version_reply(GetParam().reply_minor)));
+  EXPECT_EQ(s.client.state(), VncState::VNC_STATE_HANDSHAKE);
+  EXPECT_EQ(s.client.step(),
+            GetParam().reply_minor == '3' ? VncStep::VNC_STEP_SECURITY_TYPE_33 : VncStep::VNC_STEP_SECURITY_COUNT);
+}
+
+INSTANTIATE_TEST_SUITE_P(Versions, VncVersionMapping,
+                         ::testing::Values(VersionCase{"RFB 003.008\n", '8'}, VersionCase{"RFB 003.889\n", '8'},
+                                           VersionCase{"RFB 004.001\n", '8'}, VersionCase{"RFB 005.000\n", '8'},
+                                           VersionCase{"RFB 003.007\n", '7'}, VersionCase{"RFB 003.005\n", '3'},
+                                           VersionCase{"RFB 003.003\n", '3'}));
+
+class VncBadVersion : public ::testing::TestWithParam<const char *> {};
+
+TEST_P(VncBadVersion, IsAConfigFailureHeldFor300s) {
+  FakeDisplay display(64, 48);
+  Listener listener;
+  Session s(&display);
+  s.client.set_port(listener.port());
+  s.attach();
+  s.deliver(text(GetParam()));
+  EXPECT_EQ(s.take().size(), 0u);
+  EXPECT_EQ(s.client.state(), VncState::VNC_STATE_IDLE);
+  EXPECT_EQ(s.client.hold_ms(), 300000u);
+  EXPECT_TRUE(s.client.status_has_error());
+  const uint32_t failed_at = now_ms();
+  // No attempt while the hold runs.
+  while (now_ms() - failed_at < 299000) {
+    s.tick(1000);
+  }
+  EXPECT_EQ(listener.accepted(), 0);
+  EXPECT_FALSE(s.client.link().connecting());
+  set_now(failed_at + 300000);
+  EXPECT_TRUE(s.wait_until([&]() { return listener.accepted() == 1; }));
+}
+
+INSTANTIATE_TEST_SUITE_P(Versions, VncBadVersion,
+                         ::testing::Values("HTTP/1.1 400", "RFB 002.000\n", "RFB 003,008\n", "RFB 0x3.008\n",
+                                           "SSH-2.0-Open"));
+
+TEST(VncHandshake, PasswordRequiredWithoutKeyIsConfig) {
+  FakeDisplay display(64, 48);
+  Session s(&display);
+  s.attach();
+  s.deliver(Rfb().version());
+  EXPECT_EQ(hex(s.take()), hex(version_reply('8')));
+  s.deliver(Rfb().security_list({2, 16}));
+  // Never pick 16, never answer type 2 without a key.
+  EXPECT_EQ(s.take().size(), 0u);
+  EXPECT_EQ(s.client.state(), VncState::VNC_STATE_IDLE);
+  EXPECT_EQ(s.client.hold_ms(), 300000u);
+  EXPECT_TRUE(s.client.status_has_error());
+  EXPECT_FALSE(s.client.link().connected());
+}
+
+TEST(VncHandshake, OnlyUnsupportedTypesIsConfig) {
+  FakeDisplay display(64, 48);
+  Session s(&display);
+  s.attach();
+  s.deliver(Rfb().version());
+  EXPECT_EQ(hex(s.take()), hex(version_reply('8')));
+  s.deliver(Rfb().security_list({16}));
+  EXPECT_EQ(s.take().size(), 0u);
+  EXPECT_EQ(s.client.state(), VncState::VNC_STATE_IDLE);
+  EXPECT_EQ(s.client.hold_ms(), 300000u);
+  EXPECT_TRUE(s.client.status_has_error());
+}
+
+TEST(VncHandshake, EmptyListWithReasonIsTransient) {
+  FakeDisplay display(64, 48);
+  Session s(&display);
+  s.attach();
+  s.deliver(Rfb().version());
+  EXPECT_EQ(hex(s.take()), hex(version_reply('8')));
+  s.deliver(Rfb().security_list({}).reason("Too many authentication failures"));
+  EXPECT_EQ(s.take().size(), 0u);
+  EXPECT_EQ(s.client.state(), VncState::VNC_STATE_IDLE);
+  EXPECT_EQ(s.client.hold_ms(), 5000u);
+  EXPECT_EQ(s.client.backoff_ms(), 5000u);
+  EXPECT_TRUE(s.client.status_has_warning());
+  EXPECT_FALSE(s.client.status_has_error());
+}
+
+TEST(VncHandshake, EmptyReasonEndsAtOnce) {
+  FakeDisplay display(64, 48);
+  Session s(&display);
+  s.attach();
+  s.deliver(Rfb().version());
+  s.deliver(Rfb().security_list({}).u32(0));
+  EXPECT_EQ(s.client.state(), VncState::VNC_STATE_IDLE);
+  EXPECT_EQ(s.client.hold_ms(), 5000u);
+}
+
+TEST(VncHandshake, LongReasonIsCutWithoutWaitingForAllOfIt) {
+  FakeDisplay display(64, 48);
+  Session s(&display);
+  s.attach();
+  s.deliver(Rfb().version());
+  // Announces 100000 bytes but sends only the first 63 the client logs.
+  s.deliver(Rfb().security_list({}).u32(100000).fill(63, 'r'));
+  EXPECT_EQ(s.client.state(), VncState::VNC_STATE_IDLE);
+  EXPECT_EQ(s.client.hold_ms(), 5000u);
+}
+
+TEST(VncHandshake, FullSecurityListWithoutAUsableType) {
+  FakeDisplay display(64, 48);
+  Session s(&display);
+  s.attach();
+  s.deliver(Rfb().version());
+  Rfb list;
+  list.u8(255);
+  for (int i = 0; i < 255; i++) {
+    list.u8(static_cast<uint8_t>(i == 1 || i == 2 ? 16 : i));
+  }
+  s.deliver(list);
+  EXPECT_EQ(s.client.state(), VncState::VNC_STATE_IDLE);
+  EXPECT_EQ(s.client.hold_ms(), 300000u);
+}
+
+TEST(VncHandshake, DumpConfig) {
+  FakeDisplay display(64, 48);
+  Session s(&display);
+  s.client.dump_config();
+}
+
+TEST(VncHandshake, HandshakeTimeout) {
+  FakeDisplay display(64, 48);
+  Session s(&display);
+  s.attach();
+  const uint32_t start = now_ms();
+  s.deliver(Rfb().version());
+  s.at(start + 14999);
+  EXPECT_EQ(s.client.state(), VncState::VNC_STATE_HANDSHAKE);
+  EXPECT_TRUE(s.client.link().connected());
+  s.at(start + 15000);
+  EXPECT_EQ(s.client.state(), VncState::VNC_STATE_IDLE);
+  EXPECT_EQ(s.client.hold_ms(), 5000u);
+  EXPECT_FALSE(s.client.link().connected());
+  EXPECT_FALSE(s.client.status_has_error());
+}
+
+TEST(VncHandshake, NoneSecurityFailureIsConfig) {
+  FakeDisplay display(64, 48);
+  Session s(&display);
+  s.attach();
+  s.deliver(Rfb().version().security_list({1, 16}));
+  EXPECT_EQ(hex(s.take()), hex(cat({version_reply('8'), {1}})));
+  s.deliver(Rfb().security_result(1).reason("Server is busy"));
+  EXPECT_EQ(s.client.state(), VncState::VNC_STATE_IDLE);
+  EXPECT_EQ(s.client.hold_ms(), 300000u);
+}
+
+TEST(VncHandshake, EmptyServerName) {
+  FakeDisplay display(64, 48);
+  Session s(&display);
+  s.attach();
+  s.deliver(Rfb().version().security_list({1}).security_result(0).server_init(64, 48, ""));
+  EXPECT_TRUE(s.client.is_connected());
+  EXPECT_EQ(hex(s.take()), hex(cat({version_reply('8'), {1}, {1}, running_tail(set_pixel_format_565(), 64, 48)})));
+}
+
+TEST(VncHandshake, LongServerNameIsSkipped) {
+  FakeDisplay display(64, 48);
+  Session s(&display);
+  s.attach();
+  s.deliver(Rfb().version().security_list({1}).security_result(0).server_init(64, 48, std::string(700, 'n')));
+  ASSERT_TRUE(s.client.is_connected());
+  s.take();
+  s.deliver(Rfb().fbu(1).raw_rect(0, 0, 1, 1, {0xF800}));
+  EXPECT_EQ(display.pixel(0, 0), expect_px(0xF800, 2));
+  EXPECT_TRUE(s.client.is_connected());
+}
+
+TEST(VncHandshake, RunningOnlyAfterTheWholeServerName) {
+  FakeDisplay display(64, 48);
+  Session s(&display);
+  s.attach();
+  Bytes init = Rfb().version().security_list({1}).security_result(0).server_init(64, 48, std::string(700, 'n')).bytes;
+  s.deliver(Bytes(init.begin(), init.end() - 1));
+  EXPECT_EQ(s.client.state(), VncState::VNC_STATE_HANDSHAKE);
+  s.deliver(Bytes(init.end() - 1, init.end()));
+  EXPECT_TRUE(s.client.is_connected());
+}
+
+TEST(VncHandshake, EmptyServerScreenIsTransient) {
+  FakeDisplay display(64, 48);
+  Session s(&display);
+  s.attach();
+  s.deliver(Rfb().version().security_list({1}).security_result(0).server_init(0, 48));
+  EXPECT_EQ(s.client.state(), VncState::VNC_STATE_IDLE);
+  EXPECT_EQ(s.client.hold_ms(), 5000u);
+}
+
+TEST(VncHandshake, DisconnectCallbackOnlyAfterRunning) {
+  FakeDisplay display(64, 48);
+  Session s(&display);
+  int disconnects = 0;
+  s.client.add_on_disconnect_callback([&disconnects]() { disconnects++; });
+  s.attach();
+  s.deliver(Rfb().version());
+  s.close_peer();
+  s.loop(2);
+  EXPECT_EQ(disconnects, 0);
+
+  s.start(64, 48);
+  s.close_peer();
+  s.loop(2);
+  EXPECT_EQ(disconnects, 1);
+  EXPECT_FALSE(s.client.is_connected());
+}
+
+}  // namespace esphome::vnc::testing
+
+#endif  // USE_HOST

tests/components/vnc/test_pacing_host.cpp

@@ -0,0 +1,385 @@
+#include "vnc_test_helpers.h"
+
+#ifdef USE_HOST
+
+namespace esphome::vnc::testing {
+
+static Bytes frame(uint16_t x = 0, uint16_t pixel = 0x1234) {
+  return Rfb().fbu(1).raw_rect(x, 0, 2, 2, {pixel, pixel, pixel, pixel}).bytes;
+}
+
+static const Bytes INCREMENTAL = fbur(true, 64, 48);
+
+// The session starts at t=1000, when the first (full) request goes out.
+class VncPacing : public ::testing::Test {
+ protected:
+  void SetUp() override {
+    this->s_ = std::make_unique<Session>(&this->d_);
+    this->s_->start(64, 48);
+  }
+  Session &s() { return *this->s_; }
+  FakeDisplay d_;
+  std::unique_ptr<Session> s_;
+};
+
+TEST_F(VncPacing, UpdateOnlyAfterTheScreenStopsChanging) {
+  set_now(1100);
+  s().deliver(frame());
+  EXPECT_EQ(d_.updates(), 0);
+  s().at(1299);
+  EXPECT_EQ(d_.updates(), 0);
+  s().at(1300);
+  EXPECT_EQ(d_.updates(), 1);
+  s().run_for(5000);
+  EXPECT_EQ(d_.updates(), 1);
+}
+
+TEST_F(VncPacing, UpdateHasItsOwnPass) {
+  set_now(1100);
+  s().deliver(frame(0));
+  s().send(frame(10, 0x4321));
+  ASSERT_GT(s().unread(), 0);
+  s().at(1300);
+  EXPECT_EQ(d_.updates(), 1);
+  // Nothing was read in the pass that refreshed the display.
+  EXPECT_GT(s().unread(), 0);
+  EXPECT_EQ(d_.pixel(10, 0), UNSET);
+  s().loop();
+  EXPECT_EQ(s().unread(), 0);
+  EXPECT_EQ(d_.pixel(10, 0), expect_px(0x4321, 2));
+}
+
+TEST_F(VncPacing, FastFramesPostponeTheUpdateUntilTheMaximumDelay) {
+  // A frame every 100 ms: never quiet for 200 ms, so only the 1 s limit forces a refresh.
+  std::vector<uint32_t> update_times;
+  int last = 0;
+  for (uint32_t t = 1100; t <= 2500; t += 100) {
+    s().send(frame(t % 60));
+    s().at(t);
+    if (d_.updates() != last) {
+      update_times.push_back(t);
+      last = d_.updates();
+      s().loop();
+    }
+    EXPECT_EQ(s().unread(), 0);
+  }
+  EXPECT_EQ(update_times, std::vector<uint32_t>({2100}));
+  s().at(2699);
+  EXPECT_EQ(d_.updates(), 1);
+  s().at(2700);
+  EXPECT_EQ(d_.updates(), 2);
+}
+
+TEST_F(VncPacing, NoUpdateInTheMiddleOfAFrameEvenPastTheMaximumDelay) {
+  set_now(1100);
+  s().deliver(frame());
+  Bytes second = Rfb().fbu(1).raw_rect(0, 10, 4, 1, {1, 2, 3, 4}).bytes;
+  set_now(1200);
+  s().deliver(Bytes(second.begin(), second.end() - 3));
+  ASSERT_TRUE(s().client.in_update());
+  s().run_for(1400);
+  EXPECT_EQ(d_.updates(), 0);
+  s().deliver(Bytes(second.end() - 3, second.end()));
+  EXPECT_FALSE(s().client.in_update());
+  EXPECT_EQ(d_.updates(), 0);
+  s().tick(10);
+  EXPECT_EQ(d_.updates(), 1);
+}
+
+TEST(VncPacingDelays, CustomDelays) {
+  FakeDisplay d;
+  Session s(&d);
+  s.client.set_update_delay(50);
+  s.client.set_max_update_delay(300);
+  s.start(64, 48);
+  set_now(1100);
+  s.deliver(frame());
+  s.at(1149);
+  EXPECT_EQ(d.updates(), 0);
+  s.at(1150);
+  EXPECT_EQ(d.updates(), 1);
+  // Frames every 40 ms: the 300 ms limit applies.
+  int updates = 1;
+  uint32_t first_forced = 0;
+  for (uint32_t t = 1200; t < 1700; t += 40) {
+    s.send(frame(t % 60));
+    s.at(t);
+    if (d.updates() != updates && first_forced == 0) {
+      first_forced = t;
+    }
+    updates = d.updates();
+    s.loop();
+  }
+  EXPECT_EQ(first_forced, 1520u);
+}
+
+TEST(VncPacingDelays, ZeroDelayRefreshesOnTheNextPass) {
+  FakeDisplay d;
+  Session s(&d);
+  s.client.set_update_delay(0);
+  s.start(64, 48);
+  set_now(1100);
+  s.deliver(frame());
+  s.loop();
+  EXPECT_EQ(d.updates(), 1);
+}
+
+TEST_F(VncPacing, NextRequestRightAfterEachFrameWithA50msFloor) {
+  set_now(1010);
+  s().deliver(frame());
+  EXPECT_EQ(s().take().size(), 0u);
+  s().at(1049);
+  EXPECT_EQ(s().take().size(), 0u);
+  s().at(1050);
+  EXPECT_EQ(hex(s().take()), hex(INCREMENTAL));
+  set_now(1500);
+  s().deliver(frame());
+  EXPECT_EQ(hex(s().take()), hex(INCREMENTAL));
+  // One request stays outstanding; nothing more until the next frame or the heartbeat.
+  s().run_for(50000, 100);
+  EXPECT_EQ(s().take().size(), 0u);
+}
+
+TEST_F(VncPacing, CursorOnlyFrameNeverRefreshes) {
+  set_now(1100);
+  s().deliver(Rfb().fbu(1).cursor(1, 1, 4, 4));
+  EXPECT_EQ(hex(s().take()), hex(INCREMENTAL));
+  s().run_for(5000);
+  EXPECT_EQ(d_.updates(), 0);
+}
+
+TEST_F(VncPacing, HeartbeatAfterAMinuteOfSilence) {
+  set_now(1100);
+  s().deliver(frame());
+  EXPECT_EQ(hex(s().take()), hex(INCREMENTAL));
+  while (now_ms() + 1000 < 61099) {
+    s().tick(1000);
+  }
+  EXPECT_EQ(s().take().size(), 0u);
+  s().at(61099);
+  EXPECT_EQ(s().take().size(), 0u);
+  s().at(61100);
+  EXPECT_EQ(hex(s().take()), hex(INCREMENTAL));
+  s().at(121099);
+  EXPECT_EQ(s().take().size(), 0u);
+  s().at(121100);
+  EXPECT_EQ(hex(s().take()), hex(INCREMENTAL));
+}
+
+TEST_F(VncPacing, DataResetsTheHeartbeat) {
+  set_now(1100);
+  s().deliver(frame());
+  s().take();
+  set_now(40000);
+  s().deliver(Rfb().bell());
+  s().at(61100);
+  EXPECT_EQ(s().take().size(), 0u);
+  s().at(100000);
+  EXPECT_EQ(hex(s().take()), hex(INCREMENTAL));
+}
+
+TEST_F(VncPacing, ServerSilentMidFrameEndsTheSession) {
+  set_now(1100);
+  s().deliver(frame());
+  s().take();
+  Bytes partial = frame();
+  partial.pop_back();
+  s().deliver(partial);
+  ASSERT_TRUE(s().client.in_update());
+  s().at(1100 + 59999);
+  EXPECT_EQ(s().client.state(), VncState::VNC_STATE_RUNNING);
+  s().at(1100 + 60000);
+  EXPECT_EQ(s().client.state(), VncState::VNC_STATE_IDLE);
+  EXPECT_FALSE(s().client.link().connected());
+}
+
+// --- A display that parks its loop when idle (it8951, epaper_spi).
+
+class VncPacingParked : public ::testing::Test {
+ protected:
+  void SetUp() override {
+    this->s_ = std::make_unique<Session>(&this->d_);
+    this->s_->start(64, 48);
+  }
+  Session &s() { return *this->s_; }
+  FakeParkingDisplay d_;
+  std::unique_ptr<Session> s_;
+};
+
+TEST_F(VncPacingParked, BusyDisplayHoldsDrawsAndRequests) {
+  set_now(1100);
+  s().deliver(frame());
+  EXPECT_EQ(hex(s().take()), hex(INCREMENTAL));
+  s().at(1300);
+  ASSERT_EQ(d_.updates(), 1);
+  ASSERT_TRUE(d_.busy());
+
+  // The server answers the outstanding request while the panel refreshes.
+  s().send(frame(20, 0x5555));
+  s().at(1310);
+  EXPECT_TRUE(s().client.display_busy());
+  EXPECT_TRUE(s().client.in_update());
+  EXPECT_FALSE(HighFrequencyLoopRequester::is_high_frequency());
+  s().run_for(3000);
+  EXPECT_EQ(d_.draws_while_busy(), 0);
+  EXPECT_EQ(d_.pixel(20, 0), UNSET);
+  EXPECT_EQ(s().take().size(), 0u);
+
+  d_.finish();
+  s().tick(10);
+  EXPECT_EQ(d_.pixel(20, 0), expect_px(0x5555, 2));
+  EXPECT_EQ(hex(s().take()), hex(INCREMENTAL));
+  s().run_for(300);
+  EXPECT_EQ(d_.updates(), 2);
+}
+
+TEST(VncPacingParkedDelay, NoRequestInTheRefreshPassWithZeroDelay) {
+  FakeParkingDisplay d;
+  Session s(&d);
+  s.client.set_update_delay(0);
+  s.start(64, 48);
+  // The frame ends 20 ms after the request, inside the 50 ms floor, so the next request waits.
+  set_now(1020);
+  s.deliver(frame());
+  EXPECT_EQ(s.take().size(), 0u);
+  s.at(1060);
+  ASSERT_EQ(d.updates(), 1);
+  ASSERT_TRUE(d.busy());
+  EXPECT_EQ(s.take().size(), 0u) << "request sent while the panel refreshes";
+  d.finish();
+  s.tick(10);
+  EXPECT_EQ(hex(s.take()), hex(INCREMENTAL));
+}
+
+TEST_F(VncPacingParked, FailedDisplayEndsTheSessionAndStops) {
+  set_now(1100);
+  s().deliver(frame());
+  s().take();
+  s().at(1300);
+  ASSERT_EQ(d_.updates(), 1);
+  d_.mark_failed();
+  s().tick(10);
+  EXPECT_EQ(s().client.state(), VncState::VNC_STATE_IDLE);
+  EXPECT_FALSE(s().client.link().connected());
+  EXPECT_TRUE(s().client.status_has_error());
+  EXPECT_TRUE(s().client.is_idle());
+}
+
+TEST_F(VncPacingParked, NoRequestWhileBusyBetweenFrames) {
+  // Inside the 50 ms floor, so the next request is still due when the panel turns busy.
+  set_now(1010);
+  s().deliver(Rfb().fbu(0));
+  EXPECT_EQ(s().take().size(), 0u);
+  d_.external_update();
+  s().run_for(2000);
+  EXPECT_EQ(s().take().size(), 0u);
+  d_.finish();
+  s().tick(10);
+  EXPECT_EQ(hex(s().take()), hex(INCREMENTAL));
+}
+
+TEST_F(VncPacingParked, ExternalUpdateMidFrameStopsReading) {
+  Bytes f = Rfb().fbu(2).raw_rect(0, 0, 2, 1, {1, 2}).raw_rect(0, 5, 2, 1, {3, 4}).bytes;
+  set_now(1100);
+  s().deliver(Bytes(f.begin(), f.begin() + 4 + 12 + 4 + 6));
+  ASSERT_TRUE(s().client.in_update());
+  d_.external_update();
+  s().send(Bytes(f.begin() + 4 + 12 + 4 + 6, f.end()));
+  s().run_for(1000);
+  EXPECT_GT(s().unread(), 0);
+  EXPECT_EQ(d_.draws_while_busy(), 0);
+  EXPECT_FALSE(HighFrequencyLoopRequester::is_high_frequency());
+  d_.finish();
+  s().tick(10);
+  EXPECT_EQ(s().unread(), 0);
+  EXPECT_EQ(d_.pixel(1, 5), expect_px(4, 2));
+  EXPECT_FALSE(s().client.in_update());
+}
+
+// --- Displays that never report busy must never stall the client.
+
+template<typename D> static void runs_frames_without_deadlock() {
+  D d;
+  Session s(&d);
+  s.start(64, 48);
+  for (int i = 0; i < 5; i++) {
+    set_now(now_ms() + 300);
+    s.deliver(frame(i * 4, 0x1000 + i));
+    EXPECT_EQ(hex(s.take()), hex(INCREMENTAL)) << "frame " << i;
+    s.run_for(300);
+    EXPECT_EQ(d.updates(), i + 1) << "frame " << i;
+    EXPECT_EQ(d.pixel(i * 4, 0), expect_px(0x1000 + i, 2));
+  }
+}
+
+TEST(VncPacingDisplays, PlainDisplay) { runs_frames_without_deadlock<FakeDisplay>(); }
+TEST(VncPacingStart, FirstRefreshWaitsForAStartingDisplayToPark) {
+  FakeStartingDisplay d;
+  Session s(&d);
+  s.start(64, 48);
+  set_now(1100);
+  s.deliver(frame());
+  s.at(1300);
+  EXPECT_EQ(d.updates(), 0);
+  d.finish_start();
+  s.tick(10);
+  ASSERT_EQ(d.updates(), 1);
+  // Known to park now, so nothing is drawn until this refresh is over.
+  s.deliver(frame(10));
+  EXPECT_EQ(d.draws_while_busy(), 0);
+  d.finish();
+  s.tick(10);
+  EXPECT_EQ(d.pixel(10, 0), expect_px(0x1234, 2));
+}
+
+TEST(VncPacingStart, LoopingDisplayRefreshesAfterTheStartWindow) {
+  FakeLoopingDisplay d;
+  Session s(&d);
+  s.start(64, 48);
+  set_now(1100);
+  s.deliver(frame());
+  s.at(1000 + 4999);
+  EXPECT_EQ(d.updates(), 0);
+  s.at(1000 + 5000);
+  EXPECT_EQ(d.updates(), 1);
+}
+
+TEST(VncPacingDisplays, FreeRunningDisplay) { runs_frames_without_deadlock<FakeFreeRunDisplay>(); }
+TEST(VncPacingDisplays, SynchronousParkedDisplay) { runs_frames_without_deadlock<FakeSyncParkedDisplay>(); }
+
+TEST_F(VncPacingParked, FailedDisplayClosesAConnectStillInFlight) {
+  s().client.forget_session();
+  ASSERT_TRUE(s().client.link().connected());
+  d_.mark_failed();
+  s().tick(10);
+  EXPECT_FALSE(s().client.link().connected());
+  EXPECT_TRUE(s().client.status_has_error());
+}
+
+TEST_F(VncPacingParked, DisplayBusyTooLongSetsAWarningUntilItParks) {
+  d_.external_update();
+  s().run_for(120000, 1000);
+  EXPECT_TRUE(s().client.status_has_warning());
+  // A new session does not hide the stuck display
+  s().start(64, 48);
+  EXPECT_TRUE(s().client.status_has_warning());
+  d_.finish();
+  s().tick(10);
+  EXPECT_FALSE(s().client.status_has_warning());
+}
+
+TEST_F(VncPacingParked, DisplayParkingKeepsAConnectionWarning) {
+  d_.external_update();
+  s().run_for(120000, 1000);
+  s().close_peer();
+  s().tick(10);
+  ASSERT_EQ(s().client.state(), VncState::VNC_STATE_IDLE);
+  EXPECT_TRUE(s().client.status_has_warning());
+  d_.finish();
+  s().tick(10);
+  EXPECT_TRUE(s().client.status_has_warning());
+}
+
+}  // namespace esphome::vnc::testing
+
+#endif  // USE_HOST

tests/components/vnc/validate-implicit-id.host.yaml

@@ -0,0 +1,13 @@
+packages:
+  test_display: !include ../../test_build_components/common/test_display/test_display.yaml
+
+vnc:
+  - host: 192.0.2.10
+
+interval:
+  - interval: 60s
+    then:
+      - if:
+          condition: vnc.is_connected
+          then:
+            - logger.log: VNC is connected

tests/components/vnc/validate-it8951.esp32-idf.yaml

@@ -0,0 +1,18 @@
+packages:
+  spi: !include ../../test_build_components/common/spi/esp32-idf.yaml
+
+wifi:
+  ssid: MySSID
+  password: password1
+
+display:
+  - platform: it8951
+    id: m5paper_display
+    model: m5stack-m5paper
+    cs_pin: GPIO22
+    reset_pin: GPIO21
+    busy_pin: GPIO27
+    sleep_when_done: false
+
+vnc:
+  - host: vnc.local

tests/components/vnc/vnc_test_helpers.h

@@ -0,0 +1,581 @@
+#pragma once
+
+#include <gtest/gtest.h>
+
+#include <algorithm>
+#include <cerrno>
+#include <csignal>
+#include <cstdint>
+#include <cstdio>
+#include <cstring>
+#include <fcntl.h>
+#include <functional>
+#include <initializer_list>
+#include <memory>
+#include <netinet/in.h>
+#include <string>
+#include <sys/ioctl.h>
+#include <sys/socket.h>
+#include <unistd.h>
+#include <vector>
+
+#include "esphome/components/display/display.h"
+#include "esphome/components/socket/socket.h"
+#include "esphome/components/vnc/vnc_client.h"
+#include "esphome/core/application.h"
+#include "esphome/core/hal.h"
+#include "esphome/core/helpers.h"
+
+#ifdef USE_HOST
+
+namespace esphome::vnc::testing {
+
+using Bytes = std::vector<uint8_t>;
+
+inline void set_now(uint32_t ms) { LoopBlockingGuard guard(nullptr, nullptr, ms); }
+inline uint32_t now_ms() { return App.get_loop_component_start_time(); }
+
+inline std::string hex(const Bytes &bytes) {
+  std::string out;
+  char buf[4];
+  for (size_t i = 0; i < bytes.size(); i++) {
+    snprintf(buf, sizeof(buf), i == 0 ? "%02X" : " %02X", bytes[i]);
+    out += buf;
+  }
+  return out;
+}
+
+inline Bytes cat(std::initializer_list<Bytes> parts) {
+  Bytes out;
+  for (const auto &part : parts) {
+    out.insert(out.end(), part.begin(), part.end());
+  }
+  return out;
+}
+
+inline Bytes text(const std::string &s) { return Bytes(s.begin(), s.end()); }
+
+// ---------------------------------------------------------------------------
+// Displays
+
+inline constexpr uint32_t UNSET = 0xFF000000;
+
+inline uint32_t pack(Color c) { return (static_cast<uint32_t>(c.r) << 16) | (static_cast<uint32_t>(c.g) << 8) | c.b; }
+
+/// The colour a wire pixel must show as, decoded the way the display API decodes it.
+inline uint32_t expect_px(uint16_t wire, uint8_t bpp) {
+  return pack(display::ColorUtil::to_color(wire, bpp == 2 ? display::COLOR_BITNESS_565 : display::COLOR_BITNESS_332));
+}
+
+class FakeDisplay : public display::Display {
+ public:
+  FakeDisplay(int width = 64, int height = 48, display::DisplayType type = display::DISPLAY_TYPE_COLOR)
+      : width_(width), height_(height), type_(type), pixels_(static_cast<size_t>(width) * height, UNSET) {}
+
+  using display::Display::draw_pixels_at;
+
+  void update() override { this->updates_++; }
+  display::DisplayType get_display_type() override { return this->type_; }
+  int get_width_internal() override { return this->width_; }
+  int get_height_internal() override { return this->height_; }
+
+  void draw_pixel_at(int x, int y, Color color) override {
+    if (x < 0 || y < 0 || x >= this->width_ || y >= this->height_) {
+      this->out_of_bounds_++;
+      ADD_FAILURE() << "pixel drawn outside the display at " << x << "," << y;
+      return;
+    }
+    this->pixels_[static_cast<size_t>(y) * this->width_ + x] = pack(color);
+  }
+
+  void draw_pixels_internal(int x_start, int y_start, int w, int h, const uint8_t *ptr, display::ColorBitness bitness,
+                            bool big_endian, int x_offset, int y_offset, int x_pad) override {
+    this->draw_calls_++;
+    if (w <= 0 || h <= 0 || x_start < 0 || y_start < 0 || x_start + w > this->width_ || y_start + h > this->height_) {
+      this->out_of_bounds_++;
+      ADD_FAILURE() << "block " << w << "x" << h << " at " << x_start << "," << y_start << " does not fit "
+                    << this->width_ << "x" << this->height_;
+    }
+    if (this->busy()) {
+      this->draws_while_busy_++;
+      ADD_FAILURE() << "drawn while the display is busy";
+    }
+    if (this->draw_delay_us_ != 0) {
+      usleep(this->draw_delay_us_);
+    }
+    display::Display::draw_pixels_internal(x_start, y_start, w, h, ptr, bitness, big_endian, x_offset, y_offset, x_pad);
+  }
+
+  virtual bool busy() const { return false; }
+
+  uint32_t pixel(int x, int y) const { return this->pixels_[static_cast<size_t>(y) * this->width_ + x]; }
+  const std::vector<uint32_t> &pixels() const { return this->pixels_; }
+  size_t drawn_count() const {
+    return static_cast<size_t>(
+        std::count_if(this->pixels_.begin(), this->pixels_.end(), [](uint32_t p) { return p != UNSET; }));
+  }
+  int updates() const { return this->updates_; }
+  int draw_calls() const { return this->draw_calls_; }
+  int out_of_bounds() const { return this->out_of_bounds_; }
+  int draws_while_busy() const { return this->draws_while_busy_; }
+  void set_draw_delay_us(uint32_t us) { this->draw_delay_us_ = us; }
+
+ protected:
+  int width_;
+  int height_;
+  display::DisplayType type_;
+  std::vector<uint32_t> pixels_;
+  int updates_{0};
+  int draw_calls_{0};
+  int out_of_bounds_{0};
+  int draws_while_busy_{0};
+  uint32_t draw_delay_us_{0};
+};
+
+/// Behaves like it8951 and epaper_spi: parked (loop disabled) while idle,
+/// update() starts a refresh that runs in its loop until finish().
+class FakeParkingDisplay : public FakeDisplay {
+ public:
+  FakeParkingDisplay(int width = 64, int height = 48, display::DisplayType type = display::DISPLAY_TYPE_GRAYSCALE)
+      : FakeDisplay(width, height, type) {
+    this->disable_loop();
+  }
+  void update() override {
+    this->updates_++;
+    this->enable_loop();
+  }
+  /// The refresh is over and the driver parks its loop again.
+  void finish() { this->disable_loop(); }
+  /// Something other than the VNC client started a refresh.
+  void external_update() { this->enable_loop(); }
+  bool busy() const override { return !this->is_idle(); }
+};
+
+/// Starts with its loop running and parks once its start-up is done, like it8951 at boot.
+class FakeStartingDisplay : public FakeParkingDisplay {
+ public:
+  FakeStartingDisplay() {
+    this->component_state_ |= COMPONENT_HAS_LOOP;
+    this->enable_loop();
+  }
+  void finish_start() {
+    this->started_ = true;
+    this->finish();
+  }
+  bool busy() const override { return this->started_ && FakeParkingDisplay::busy(); }
+
+ protected:
+  bool started_{false};
+};
+
+/// A display whose loop always runs (never reports idle).
+class FakeFreeRunDisplay : public FakeDisplay {
+ public:
+  FakeFreeRunDisplay(int width = 64, int height = 48) : FakeDisplay(width, height) {
+    this->disable_loop();
+    this->enable_loop();
+  }
+};
+
+/// A display with its own loop that never parks.
+class FakeLoopingDisplay : public FakeFreeRunDisplay {
+ public:
+  FakeLoopingDisplay() { this->component_state_ |= COMPONENT_HAS_LOOP; }
+};
+
+/// Parked from setup and refreshes synchronously inside update(), like headless SDL.
+class FakeSyncParkedDisplay : public FakeDisplay {
+ public:
+  FakeSyncParkedDisplay(int width = 64, int height = 48) : FakeDisplay(width, height) { this->disable_loop(); }
+};
+
+// ---------------------------------------------------------------------------
+// Client under test
+
+class TestVncClient : public VncClient {
+ public:
+  using VncClient::VncClient;
+  ~TestVncClient() {
+    this->high_freq_.stop();
+    this->link_.close();
+  }
+
+  /// The state a connect still in flight has: a socket, but no session yet.
+  void forget_session() { this->state_ = VncState::VNC_STATE_IDLE; }
+
+  /// Hand the client a connected stream, as connect_step_() does after a successful connect.
+  void adopt(int fd) {
+    // connect_step_() has seen the network up before any connect succeeds.
+    this->network_was_up_ = true;
+    this->link_.adopt(std::make_unique<socket::Socket>(fd));
+    this->start_handshake_(now_ms());
+  }
+
+  /// Parse bytes straight from memory, the way service_rx_() would, without a socket read.
+  VncParse feed(const uint8_t *data, size_t len) {
+    VncParse result = VncParse::VNC_PARSE_MORE;
+    size_t offset = 0;
+    do {
+      const size_t room = RX_BUFFER_SIZE - this->rx_len_;
+      const size_t count = std::min(room, len - offset);
+      std::memcpy(this->rx_ + this->rx_len_, data + offset, count);
+      this->rx_len_ += static_cast<uint16_t>(count);
+      this->rx_total_ += count;
+      offset += count;
+      for (int guard = 0;; guard++) {
+        this->pass_start_ms_ = millis();
+        result = this->parse_();
+        if (result != VncParse::VNC_PARSE_PAUSE || this->display_busy_ || guard > 100000) {
+          break;
+        }
+      }
+      if (result != VncParse::VNC_PARSE_MORE) {
+        return result;
+      }
+      if (this->rx_pos_ != 0) {
+        this->rx_len_ -= this->rx_pos_;
+        std::memmove(this->rx_, this->rx_ + this->rx_pos_, this->rx_len_);
+        this->rx_pos_ = 0;
+      } else if (count == 0) {
+        ADD_FAILURE() << "parser made no progress with a full buffer";
+        return result;
+      }
+    } while (offset < len);
+    return result;
+  }
+  VncParse feed(const Bytes &bytes) { return this->feed(bytes.data(), bytes.size()); }
+
+  socket::TcpClientLink &link() { return this->link_; }
+  VncState state() const { return this->state_; }
+  VncStep step() const { return this->step_; }
+  uint32_t hold_ms() const { return this->hold_ms_; }
+  uint32_t hold_start_ms() const { return this->hold_start_ms_; }
+  uint32_t backoff_ms() const { return this->backoff_ms_; }
+  bool rx_pending() const { return this->rx_pending_; }
+  bool in_update() const { return this->in_update_; }
+  bool refresh_pending() const { return this->refresh_pending_; }
+  bool display_busy() const { return this->display_busy_; }
+  uint16_t rx_buffered() const { return this->rx_len_ - this->rx_pos_; }
+  uint16_t vis_w() const { return this->vis_w_; }
+  uint16_t vis_h() const { return this->vis_h_; }
+  uint8_t bpp() const { return this->bpp_; }
+};
+
+// ---------------------------------------------------------------------------
+// Server-side byte builder (RFB 3.x, server to client)
+
+class Rfb {
+ public:
+  Bytes bytes;
+
+  Rfb &u8(uint8_t v) {
+    this->bytes.push_back(v);
+    return *this;
+  }
+  Rfb &u16(uint16_t v) { return this->u8(v >> 8).u8(v & 0xFF); }
+  Rfb &u32(uint32_t v) { return this->u16(v >> 16).u16(v & 0xFFFF); }
+  Rfb &s32(int32_t v) { return this->u32(static_cast<uint32_t>(v)); }
+  Rfb &raw(const Bytes &b) {
+    this->bytes.insert(this->bytes.end(), b.begin(), b.end());
+    return *this;
+  }
+  Rfb &str(const std::string &s) { return this->raw(text(s)); }
+  Rfb &fill(size_t count, uint8_t value) {
+    this->bytes.insert(this->bytes.end(), count, value);
+    return *this;
+  }
+  /// One pixel in the client's format: little-endian RGB565 (bpp 2) or RGB332 (bpp 1).
+  Rfb &px(uint16_t v, uint8_t bpp = 2) {
+    this->u8(v & 0xFF);
+    if (bpp == 2) {
+      this->u8(v >> 8);
+    }
+    return *this;
+  }
+
+  Rfb &version(const std::string &v = "RFB 003.008\n") { return this->str(v); }
+  Rfb &security_list(std::initializer_list<uint8_t> types) {
+    this->u8(static_cast<uint8_t>(types.size()));
+    for (uint8_t t : types) {
+      this->u8(t);
+    }
+    return *this;
+  }
+  Rfb &security_33(uint32_t type) { return this->u32(type); }
+  Rfb &security_result(uint32_t result) { return this->u32(result); }
+  Rfb &reason(const std::string &s) { return this->u32(s.size()).str(s); }
+  Rfb &challenge() {
+    for (uint8_t i = 0; i < 16; i++) {
+      this->u8(i);
+    }
+    return *this;
+  }
+  Rfb &server_init(uint16_t w, uint16_t h, const std::string &name = "esphome-test") {
+    // The server's own pixel format; the client replaces it with SetPixelFormat.
+    this->u16(w).u16(h).u8(32).u8(24).u8(0).u8(1).u16(255).u16(255).u16(255).u8(16).u8(8).u8(0).fill(3, 0);
+    return this->u32(name.size()).str(name);
+  }
+
+  Rfb &fbu(uint16_t rects) { return this->u8(0).u8(0).u16(rects); }
+  Rfb &rect(uint16_t x, uint16_t y, uint16_t w, uint16_t h, int32_t encoding) {
+    return this->u16(x).u16(y).u16(w).u16(h).s32(encoding);
+  }
+  Rfb &raw_rect(uint16_t x, uint16_t y, uint16_t w, uint16_t h, const std::vector<uint16_t> &pixels, uint8_t bpp = 2) {
+    this->rect(x, y, w, h, 0);
+    for (uint16_t p : pixels) {
+      this->px(p, bpp);
+    }
+    return *this;
+  }
+  Rfb &cursor(uint16_t x, uint16_t y, uint16_t w, uint16_t h, uint8_t bpp = 2) {
+    this->rect(x, y, w, h, -239);
+    for (uint32_t i = 0; i < static_cast<uint32_t>(w) * h; i++) {
+      this->px(0xA5A5, bpp);
+    }
+    return this->fill(((w + 7u) / 8u) * h, 0xFF);
+  }
+  Rfb &bell() { return this->u8(2); }
+  Rfb &cut_text(uint32_t len, uint8_t value = 'x') { return this->u8(3).fill(3, 0).u32(len).fill(len, value); }
+  Rfb &colour_map(uint16_t first, uint16_t count) {
+    this->u8(1).u8(0).u16(first).u16(count);
+    for (uint16_t i = 0; i < count; i++) {
+      this->u16(i).u16(i).u16(i);
+    }
+    return *this;
+  }
+};
+
+/// A full RFB 3.8 None handshake up to and including ServerInit.
+inline Bytes handshake_38(uint16_t w, uint16_t h) {
+  return Rfb().version().security_list({1}).security_result(0).server_init(w, h).bytes;
+}
+
+// ---------------------------------------------------------------------------
+// Client-side messages the tests expect
+
+inline Bytes fbur(bool incremental, uint16_t w, uint16_t h) {
+  return Rfb().u8(3).u8(incremental ? 1 : 0).u16(0).u16(0).u16(w).u16(h).bytes;
+}
+inline Bytes set_pixel_format_565() { return {0, 0, 0, 0, 16, 16, 0, 1, 0, 31, 0, 63, 0, 31, 11, 5, 0, 0, 0, 0}; }
+inline Bytes set_pixel_format_332() { return {0, 0, 0, 0, 8, 8, 0, 1, 0, 7, 0, 7, 0, 3, 5, 2, 0, 0, 0, 0}; }
+inline Bytes set_encodings() { return {2, 0, 0, 2, 0, 0, 0, 0, 0xFF, 0xFF, 0xFF, 0x11}; }
+inline Bytes version_reply(char minor) { return text(std::string("RFB 003.00") + minor + "\n"); }
+
+/// Splits a stream of client messages that are all FramebufferUpdateRequests.
+inline int count_fburs(const Bytes &tx, bool incremental) {
+  int count = 0;
+  for (size_t i = 0; i + 10 <= tx.size(); i += 10) {
+    if (tx[i] == 3 && (tx[i + 1] != 0) == incremental) {
+      count++;
+    }
+  }
+  return count;
+}
+
+// ---------------------------------------------------------------------------
+// Ports
+
+/// A loopback port that was free a moment ago, so a connect to it is refused.
+inline uint16_t refused_port() {
+  int probe = ::socket(AF_INET, SOCK_STREAM, 0);
+  struct sockaddr_in addr{};
+  addr.sin_family = AF_INET;
+  addr.sin_addr.s_addr = htonl(INADDR_LOOPBACK);
+  socklen_t len = sizeof(addr);
+  EXPECT_EQ(::bind(probe, reinterpret_cast<struct sockaddr *>(&addr), len), 0);
+  EXPECT_EQ(::getsockname(probe, reinterpret_cast<struct sockaddr *>(&addr), &len), 0);
+  ::close(probe);
+  return ntohs(addr.sin_port);
+}
+
+/// A loopback listener that counts the connects that reach it.
+class Listener {
+ public:
+  Listener() {
+    this->fd_ = ::socket(AF_INET, SOCK_STREAM, 0);
+    struct sockaddr_in addr{};
+    addr.sin_family = AF_INET;
+    addr.sin_addr.s_addr = htonl(INADDR_LOOPBACK);
+    socklen_t len = sizeof(addr);
+    EXPECT_EQ(::bind(this->fd_, reinterpret_cast<struct sockaddr *>(&addr), len), 0);
+    EXPECT_EQ(::listen(this->fd_, 16), 0);
+    EXPECT_EQ(::getsockname(this->fd_, reinterpret_cast<struct sockaddr *>(&addr), &len), 0);
+    fcntl(this->fd_, F_SETFL, fcntl(this->fd_, F_GETFL, 0) | O_NONBLOCK);
+    this->port_ = ntohs(addr.sin_port);
+  }
+  ~Listener() {
+    for (int fd : this->accepted_) {
+      if (fd >= 0) {
+        ::close(fd);
+      }
+    }
+    ::close(this->fd_);
+  }
+  uint16_t port() const { return this->port_; }
+  /// Connections accepted so far, including any waiting now.
+  int accepted() {
+    for (;;) {
+      int fd = ::accept(this->fd_, nullptr, nullptr);
+      if (fd >= 0) {
+        this->accepted_.push_back(fd);
+      } else if (errno == ECONNABORTED) {
+        // The client gave up on it already; it still was an attempt.
+        this->accepted_.push_back(-1);
+      } else if (errno != EINTR) {
+        break;
+      }
+    }
+    return static_cast<int>(this->accepted_.size());
+  }
+
+ protected:
+  int fd_{-1};
+  uint16_t port_{0};
+  std::vector<int> accepted_;
+};
+
+// ---------------------------------------------------------------------------
+// Test harness: the test plays the server on the far end of a socketpair.
+
+class Session {
+ public:
+  explicit Session(display::Display *display, uint32_t start_ms = 1000) : client(display) {
+    signal(SIGPIPE, SIG_IGN);
+    set_now(start_ms);
+    this->client.set_host("127.0.0.1");
+    this->client.set_port(refused_port());
+    this->client.setup();
+  }
+  ~Session() { this->close_peer(); }
+
+  /// A new TCP session: the client gets one end, the test keeps the other.
+  void attach() {
+    this->close_peer();
+    int fds[2];
+    ASSERT_EQ(socketpair(AF_UNIX, SOCK_STREAM, 0, fds), 0);
+    fcntl(fds[1], F_SETFL, fcntl(fds[1], F_GETFL, 0) | O_NONBLOCK);
+    this->peer_fd = fds[1];
+    this->client_fd = fds[0];
+    this->out_.clear();
+    this->out_pos_ = 0;
+    this->client.adopt(fds[0]);
+  }
+  void close_peer() {
+    if (this->peer_fd >= 0) {
+      ::close(this->peer_fd);
+      this->peer_fd = -1;
+    }
+  }
+
+  /// Queue server bytes and write as many as the socket takes now.
+  void send(const Bytes &bytes) {
+    this->out_.insert(this->out_.end(), bytes.begin(), bytes.end());
+    this->pump();
+  }
+  void send(const Rfb &rfb) { this->send(rfb.bytes); }
+  void pump() {
+    while (this->out_pos_ < this->out_.size() && this->peer_fd >= 0) {
+      ssize_t n = ::write(this->peer_fd, this->out_.data() + this->out_pos_, this->out_.size() - this->out_pos_);
+      if (n <= 0) {
+        break;
+      }
+      this->out_pos_ += n;
+    }
+    if (this->out_pos_ == this->out_.size()) {
+      this->out_.clear();
+      this->out_pos_ = 0;
+    }
+  }
+  bool server_bytes_waiting() const { return this->out_pos_ < this->out_.size(); }
+  /// Bytes the client has not read from its socket yet.
+  int unread() {
+    int count = 0;
+    if (this->client.link().connected()) {
+      ioctl(this->client_fd, FIONREAD, &count);
+    }
+    return count;
+  }
+
+  /// Everything the client sent since the last call.
+  Bytes take() {
+    this->client.link().flush_tx();
+    Bytes out;
+    if (this->peer_fd < 0) {
+      return out;
+    }
+    uint8_t buf[4096];
+    for (;;) {
+      ssize_t n = ::read(this->peer_fd, buf, sizeof(buf));
+      if (n <= 0) {
+        break;
+      }
+      out.insert(out.end(), buf, buf + n);
+    }
+    return out;
+  }
+
+  void loop(int count = 1) {
+    for (int i = 0; i < count; i++) {
+      this->pump();
+      this->client.loop();
+    }
+  }
+  /// Move the virtual clock to an absolute time and run one pass.
+  void at(uint32_t ms) {
+    set_now(ms);
+    this->loop();
+  }
+  /// Advance the virtual clock and run one pass.
+  void tick(uint32_t ms) { this->at(now_ms() + ms); }
+  /// Advance in steps, running one pass per step.
+  void run_for(uint32_t ms, uint32_t step = 10) {
+    const uint32_t end = now_ms() + ms;
+    while (static_cast<int32_t>(end - now_ms()) > 0) {
+      this->tick(std::min(step, end - now_ms()));
+    }
+  }
+  bool loop_until(const std::function<bool()> &done, int max_loops = 1000) {
+    for (int i = 0; i < max_loops && !done(); i++) {
+      this->loop();
+    }
+    return done();
+  }
+  /// Runs passes at the current virtual time until done, sleeping briefly so the kernel can finish a connect.
+  bool wait_until(const std::function<bool()> &done, int max_ms = 2000) {
+    for (int i = 0; i < max_ms * 5 && !done(); i++) {
+      this->loop();
+      if (!done()) {
+        usleep(200);
+      }
+    }
+    return done();
+  }
+  /// Send and keep running passes at the current time until the client has read and parsed it all.
+  void deliver(const Bytes &bytes, int max_loops = 100000) {
+    this->send(bytes);
+    this->loop_until(
+        [this]() { return !this->server_bytes_waiting() && this->unread() == 0 && !this->client.rx_pending(); },
+        max_loops);
+  }
+  void deliver(const Rfb &rfb) { this->deliver(rfb.bytes); }
+
+  /// Full RFB 3.8 None handshake up to RUNNING; the client's handshake bytes are discarded.
+  void start(uint16_t fb_w, uint16_t fb_h) {
+    this->attach();
+    this->deliver(handshake_38(fb_w, fb_h));
+    ASSERT_EQ(this->client.state(), VncState::VNC_STATE_RUNNING);
+    this->take();
+  }
+
+  TestVncClient client;
+  int peer_fd{-1};
+  int client_fd{-1};
+
+ protected:
+  Bytes out_;
+  size_t out_pos_{0};
+};
+
+}  // namespace esphome::vnc::testing
+
+#endif  // USE_HOST

tests/integration/conftest.py

@@ -236,6 +236,10 @@ async def yaml_config(request: pytest.FixtureRequest, unused_tcp_port: int) -> s
         external_components_path = str(FIXTURES_DIR / "external_components")
         content = content.replace("EXTERNAL_COMPONENT_PATH", external_components_path)
 
+    # Replace the user includes path placeholder if present
+    if "INCLUDES_PATH" in content:
+        content = content.replace("INCLUDES_PATH", str(FIXTURES_DIR / "includes"))
+
     if shared_name is not None:
         # _compile verifies the marked test compiles this content unmodified
         request.node._shared_yaml_content = content

tests/integration/fixtures/api_action_metadata.yaml

@@ -22,5 +22,18 @@ api:
         value: int
       then:
         - logger.log: "Plain action called"
+    - action: optional_args_action
+      variables:
+        song_str:
+          type: string
+          required: false
+        octave:
+          type: int
+          default: 5
+        loud:
+          type: bool
+          default: true
+      then:
+        - logger.log: "Optional action called"
 
 logger:

tests/integration/fixtures/api_cover_toggle.yaml

@@ -0,0 +1,26 @@
+esphome:
+  name: api-cover-toggle-test
+host:
+api:
+logger:
+  level: DEBUG
+
+cover:
+  - platform: template
+    name: "Toggle Cover"
+    id: toggle_cover
+    toggle_action:
+      - cover.template.publish:
+          id: toggle_cover
+          position: 0.5
+    stop_action:
+      - cover.template.publish:
+          id: toggle_cover
+          position: 0.25
+  - platform: template
+    name: "Plain Cover"
+    id: plain_cover
+    stop_action:
+      - cover.template.publish:
+          id: plain_cover
+          position: 0.25

tests/integration/fixtures/includes/lambda_order.h

@@ -0,0 +1,6 @@
+#pragma once
+
+static constexpr int HEADER_CONSTANT = 42;
+
+// Names the global by its id, so the header must come after the id declarations.
+inline int header_reads_global() { return header_value->value() * 2; }

tests/integration/fixtures/includes_lambda_order.yaml

@@ -0,0 +1,27 @@
+esphome:
+  name: includes-lambda-order
+  includes:
+    - INCLUDES_PATH/lambda_order.h
+  on_boot:
+    # An apply action value is rendered at file scope.
+    - globals.set:
+        id: header_value
+        value: !lambda return HEADER_CONSTANT;
+
+host:
+api:
+logger:
+
+globals:
+  - id: header_value
+    type: int
+    initial_value: "0"
+
+sensor:
+  - platform: template
+    name: Header Value
+    update_interval: 100ms
+    lambda: return header_reads_global();
+    filters:
+      # A lambda in a flash table.
+      - filter_out: !lambda return HEADER_CONSTANT - 1;

tests/integration/fixtures/vnc_client.yaml

@@ -0,0 +1,40 @@
+esphome:
+  name: vnc-client-test
+
+host:
+
+api:
+  actions:
+    - action: take_snapshot
+      variables:
+        filename: string
+      then:
+        - snapshot.take:
+            id: disp
+            filename: !lambda return filename;
+
+logger:
+  level: DEBUG
+
+display:
+  - platform: snapshot
+    id: disp
+    update_interval: never
+    dimensions:
+      width: 64
+      height: 48
+
+vnc:
+  id: remote
+  display_id: disp
+  host: 127.0.0.1
+  # Rewritten by the test to the fake server's port
+  port: 15900
+  reconnect_interval: 1s
+  # Longer than the time the next request may take, so the two can be told apart
+  update_delay: 300ms
+  max_update_delay: 500ms
+  on_connect:
+    - logger.log: "VNC_TEST_CONNECTED"
+  on_disconnect:
+    - logger.log: "VNC_TEST_DISCONNECTED"

tests/integration/raw_api_client.py

@@ -1,7 +1,7 @@
 """Minimal plaintext native-api client over a raw socket.
 
 Reads only when told to, so tests control when the TCP pipe backs up toward
-the device; payloads are skipped and only message types are counted.
+the device; message types are counted and each frame is kept for inspection.
 """
 
 from __future__ import annotations
@@ -9,6 +9,7 @@ from __future__ import annotations
 import asyncio
 from collections import Counter
 import socket
+import struct
 from typing import Self
 
 from aioesphomeapi import api_pb2
@@ -49,6 +50,36 @@ def decode_varint(buf: bytearray, pos: int) -> tuple[int, int] | None:
     return None
 
 
+WIRE_VARINT, WIRE_LENGTH, WIRE_FIXED32 = 0, 2, 5
+
+
+def tag(field: int, wire_type: int) -> bytes:
+    return encode_varint((field << 3) | wire_type)
+
+
+def decode_fields(payload: bytes) -> dict[int, list[int | bytes]]:
+    """Split a protobuf payload into field number -> raw values; a test reads fields the client lacks."""
+    fields: dict[int, list[int | bytes]] = {}
+    buf = bytearray(payload)
+    pos = 0
+    while pos < len(buf):
+        key, pos = decode_varint(buf, pos)
+        field, wire_type = key >> 3, key & 0x07
+        if wire_type == WIRE_VARINT:
+            value, pos = decode_varint(buf, pos)
+        elif wire_type == WIRE_LENGTH:
+            length, pos = decode_varint(buf, pos)
+            value = bytes(buf[pos : pos + length])
+            pos += length
+        elif wire_type == WIRE_FIXED32:
+            value = struct.unpack_from("<I", buf, pos)[0]
+            pos += 4
+        else:
+            raise AssertionError(f"unexpected wire type {wire_type} for field {field}")
+        fields.setdefault(field, []).append(value)
+    return fields
+
+
 def encode_frame(msg_type: int, payload: bytes) -> bytes:
     """Encode one plaintext api frame: 0x00, payload length, message type."""
     return b"\x00" + encode_varint(len(payload)) + encode_varint(msg_type) + payload
@@ -59,15 +90,17 @@ class FrameParser:
 
     def __init__(self) -> None:
         self._buf = bytearray()
+        self.frames: list[tuple[int, bytes]] = []
 
     def feed(self, data: bytes) -> list[int]:
         self._buf.extend(data)
         types: list[int] = []
-        while (msg_type := self._try_parse()) is not None:
-            types.append(msg_type)
+        while (frame := self._try_parse()) is not None:
+            self.frames.append(frame)
+            types.append(frame[0])
         return types
 
-    def _try_parse(self) -> int | None:
+    def _try_parse(self) -> tuple[int, bytes] | None:
         buf = self._buf
         if not buf:
             return None
@@ -80,8 +113,9 @@ class FrameParser:
         msg_type, pos = type_decoded
         if len(buf) - pos < size:
             return None
+        payload = bytes(buf[pos : pos + size])
         del buf[: pos + size]
-        return msg_type
+        return msg_type, payload
 
 
 class RawApiClient:
@@ -145,5 +179,9 @@ class RawApiClient:
 
         await asyncio.wait_for(_read_loop(), timeout)
 
+    def frames(self, msg_type: int) -> list[bytes]:
+        """Payloads of every frame of msg_type received so far."""
+        return [payload for kind, payload in self._parser.frames if kind == msg_type]
+
     def close(self) -> None:
         self._sock.close()

tests/integration/rfb_fake_server.py

@@ -0,0 +1,629 @@
+"""A deterministic fake RFB (VNC) server for the vnc integration tests.
+
+It behaves like the TightVNC 1.3.10 Xvnc that the Home Assistant VNC add-on runs: the same
+handshake for RFB 3.3, 3.7 and 3.8, the same security lists and failure messages, and a
+connection that closes on any client message type it does not know.
+
+The framebuffer is an RGB888 image the test draws into. Updates are translated into the pixel
+format the client asked for and encoded as Raw or a RichCursor shape. Every client
+message is recorded, and the test decides when to answer: ``wait_for("fbur")`` then
+``send_update(...)``.
+"""
+
+from __future__ import annotations
+
+import asyncio
+from collections.abc import Callable, Iterable, Sequence
+import contextlib
+from dataclasses import asdict, dataclass
+import logging
+import random
+import re
+import struct
+from typing import Any, Literal, Self
+
+_LOGGER = logging.getLogger(__name__)
+
+SECURITY_NONE = 1
+SECURITY_TIGHT = 16
+
+ENCODING_RAW = 0
+ENCODING_RICH_CURSOR = -239
+
+#: ``chunk`` value that writes in random 1-7 byte pieces.
+HOSTILE_CHUNK = -1
+
+#: TightVNC closes the connection on a longer ClientCutText.
+MAX_CLIENT_CUT_TEXT = 1 << 20
+
+
+RGB = tuple[int, int, int]
+Encoding = Literal["raw", "rich-cursor"]
+Rect = tuple[int, int, int, int, Encoding]
+Message = dict[str, Any]
+
+_PIXEL_FORMAT = struct.Struct(">BBBBHHHBBB3x")
+_VERSION_RE = re.compile(rb"RFB (\d{3})\.(\d{3})\n")
+
+
+@dataclass(frozen=True)
+class PixelFormat:
+    """An RFB PIXEL_FORMAT. Field names match the keys of a set_pixel_format message."""
+
+    bpp: int
+    depth: int
+    big_endian: bool
+    true_colour: bool
+    red_max: int
+    green_max: int
+    blue_max: int
+    red_shift: int
+    green_shift: int
+    blue_shift: int
+
+    @classmethod
+    def unpack(cls, data: bytes) -> PixelFormat:
+        """Decode the 16 byte wire form."""
+        bpp, depth, big_endian, true_colour, *rest = _PIXEL_FORMAT.unpack(data)
+        return cls(bpp, depth, bool(big_endian), bool(true_colour), *rest)
+
+    def pack(self) -> bytes:
+        """Encode the 16 byte wire form."""
+        return _PIXEL_FORMAT.pack(*asdict(self).values())
+
+    @property
+    def bytes_per_pixel(self) -> int:
+        """Bytes per pixel on the wire."""
+        return self.bpp // 8
+
+    def value_of(self, rgb: RGB) -> int:
+        """The pixel value a real server sends for an RGB888 colour."""
+        r, g, b = rgb
+        value = (
+            (((r * self.red_max + 127) // 255) << self.red_shift)
+            | (((g * self.green_max + 127) // 255) << self.green_shift)
+            | (((b * self.blue_max + 127) // 255) << self.blue_shift)
+        )
+        return value & ((1 << self.bpp) - 1)
+
+    def to_bytes(self, value: int) -> bytes:
+        """A pixel value in wire byte order."""
+        return value.to_bytes(
+            self.bytes_per_pixel, "big" if self.big_endian else "little"
+        )
+
+
+#: TightVNC's native format at ``-depth 16`` on a little-endian host.
+RGB565_LE = PixelFormat(16, 16, False, True, 31, 63, 31, 11, 5, 0)
+RGB565_BE = PixelFormat(16, 16, True, True, 31, 63, 31, 11, 5, 0)
+RGB332 = PixelFormat(8, 8, False, True, 7, 7, 3, 5, 2, 0)
+
+
+def _string(text: str) -> bytes:
+    data = text.encode()
+    return struct.pack(">I", len(data)) + data
+
+
+class _ProtocolError(Exception):
+    """The client broke the protocol; TightVNC would close the connection."""
+
+
+@dataclass(eq=False)
+class _Session:
+    reader: asyncio.StreamReader
+    writer: asyncio.StreamWriter
+    pixel_format: PixelFormat
+    task: asyncio.Task[None] | None
+    minor: int = 8
+
+
+class FakeRfbServer:
+    """An RFB server with a scripted framebuffer, serving one client at a time.
+
+    A new connection closes the previous one, as the add-on's ``-nevershared`` does.
+    """
+
+    def __init__(
+        self,
+        width: int,
+        height: int,
+        *,
+        version: bytes = b"RFB 003.008\n",
+        security_types: list[int] | None = None,
+        name: str = "fake rfb",
+        native_format: PixelFormat = RGB565_LE,
+        refuse_with_reason: str | None = None,
+        handshake_chunk: int | None = None,
+        seed: int = 0,
+    ) -> None:
+        """Set up the server; nothing listens until ``start()``.
+
+        Args:
+            width: Framebuffer width.
+            height: Framebuffer height.
+            version: The 12 byte ProtocolVersion the server sends.
+            security_types: The 3.7+ list. Defaults to TightVNC's ``[1, 16]``.
+            name: Desktop name in ServerInit.
+            native_format: Pixel format in ServerInit, used until SetPixelFormat.
+            refuse_with_reason: Refuse every connection before authentication with this
+                reason, as TightVNC does after too many failures.
+            handshake_chunk: ``chunk`` used for the server's handshake writes.
+            seed: Seed for the hostile chunk sizes.
+        """
+        self.width = width
+        self.height = height
+        self.version = version
+        self.security_types = (
+            list(security_types)
+            if security_types is not None
+            else [SECURITY_NONE, SECURITY_TIGHT]
+        )
+        self.name = name
+        self.native_format = native_format
+        self.refuse_with_reason = refuse_with_reason
+        self.handshake_chunk = handshake_chunk
+        self.image = bytearray(width * height * 3)
+        #: Every parsed client message, from every connection, in arrival order.
+        self.messages: list[Message] = []
+        self.protocol_errors: list[str] = []
+        self.client_version: bytes | None = None
+        self.shared_flag: int | None = None
+        self.encodings: list[int] = []
+        self._rng = random.Random(seed)
+        self._server: asyncio.Server | None = None
+        self._session: _Session | None = None
+        self._open: set[_Session] = set()
+        self._pending: list[Message] = []
+        self._changed = asyncio.Event()
+        self._write_lock = asyncio.Lock()
+        self._port = 0
+        self._connections = 0
+        self._sessions = 0
+
+    async def __aenter__(self) -> Self:
+        """Start on a free loopback port if not started yet."""
+        if self._server is None:
+            await self.start()
+        return self
+
+    async def __aexit__(self, *exc_info: object) -> None:
+        """Stop the server."""
+        await self.stop()
+
+    @property
+    def port(self) -> int:
+        """The bound port, 0 before ``start()``."""
+        return self._port
+
+    @property
+    def connections(self) -> int:
+        """TCP connections accepted so far."""
+        return self._connections
+
+    @property
+    def sessions(self) -> int:
+        """Connections that got as far as ServerInit."""
+        return self._sessions
+
+    @property
+    def client_connected(self) -> bool:
+        """True while a client connection is open."""
+        return self._session is not None
+
+    @property
+    def pixel_format(self) -> PixelFormat:
+        """The current client's pixel format, or the native one with no client."""
+        return self._session.pixel_format if self._session else self.native_format
+
+    async def start(self, host: str = "127.0.0.1", port: int = 0) -> int:
+        """Listen and return the bound port."""
+        self._server = await asyncio.start_server(self._handle, host, port)
+        self._port = self._server.sockets[0].getsockname()[1]
+        return self._port
+
+    async def stop(self) -> None:
+        """Close the listener and every connection, and wait for their handlers."""
+        if (server := self._server) is None:
+            return
+        self._server = None
+        server.close()
+        for session in list(self._open):
+            session.writer.transport.abort()
+        if tasks := [s.task for s in self._open if s.task is not None]:
+            _, pending = await asyncio.wait(tasks, timeout=5.0)
+            for task in pending:
+                task.cancel()
+            if pending:
+                await asyncio.wait(pending)
+        await server.wait_closed()
+
+    def set_pixel(self, x: int, y: int, rgb: RGB) -> None:
+        """Set one framebuffer pixel."""
+        self.fill_rect(x, y, 1, 1, rgb)
+
+    def get_pixel(self, x: int, y: int) -> RGB:
+        """Read one framebuffer pixel."""
+        self._check_rect(x, y, 1, 1)
+        offset = (y * self.width + x) * 3
+        r, g, b = self.image[offset : offset + 3]
+        return r, g, b
+
+    def fill_rect(self, x: int, y: int, w: int, h: int, rgb: RGB) -> None:
+        """Fill a framebuffer rectangle with one colour."""
+        self._check_rect(x, y, w, h)
+        row = bytes(rgb) * w
+        for line in range(y, y + h):
+            offset = (line * self.width + x) * 3
+            self.image[offset : offset + w * 3] = row
+
+    async def wait_until(
+        self,
+        predicate: Callable[[], bool],
+        timeout: float = 5.0,
+        what: str = "condition",
+    ) -> None:
+        """Wait until ``predicate()`` is true; it is checked after every server event.
+
+        Raises:
+            AssertionError: If it is still false after ``timeout``.
+        """
+        if not await self._wait(lambda: True if predicate() else None, timeout):
+            raise AssertionError(f"{what} not met within {timeout}s; {self._summary()}")
+
+    async def wait_for(
+        self,
+        type_name: str,
+        timeout: float = 5.0,
+        predicate: Callable[[Message], bool] | None = None,
+    ) -> Message:
+        """Return and consume the oldest unconsumed message of this type.
+
+        Messages of other types stay queued, so the order of calls does not matter. Only the
+        current connection's messages are queued: a new connection drops the old ones.
+
+        Raises:
+            AssertionError: If no matching message arrives within ``timeout``.
+        """
+
+        def take() -> Message | None:
+            for index, message in enumerate(self._pending):
+                if message["type"] == type_name and (
+                    predicate is None or predicate(message)
+                ):
+                    return self._pending.pop(index)
+            return None
+
+        if (message := await self._wait(take, timeout)) is None:
+            raise AssertionError(
+                f"no {type_name} message within {timeout}s; {self._summary()}"
+            )
+        return message
+
+    def pending(self) -> list[Message]:
+        """Messages not consumed by ``wait_for()`` yet."""
+        return list(self._pending)
+
+    def discard_pending(self) -> None:
+        """Drop every unconsumed message."""
+        self._pending.clear()
+
+    def build_update(
+        self,
+        rects: Sequence[Rect],
+        *,
+        pixel_format: PixelFormat | None = None,
+    ) -> bytes:
+        """Encode one FramebufferUpdate from the image.
+
+        Each rect is ``(x, y, w, h, encoding)``. For ``rich-cursor`` x and y are the hotspot
+        and the shape is a fixed pattern.
+
+        Args:
+            rects: The rectangles, in order.
+            pixel_format: Format to encode in; defaults to the current client's.
+        """
+        fmt = pixel_format or self.pixel_format
+        out = bytearray(struct.pack(">BxH", 0, len(rects)))
+        for x, y, w, h, encoding in rects:
+            if encoding == "rich-cursor":
+                out += struct.pack(">HHHHi", x, y, w, h, ENCODING_RICH_CURSOR)
+                out += _cursor_shape(fmt, w, h)
+                continue
+            self._check_rect(x, y, w, h)
+            values = self._values(fmt, x, y, w, h)
+            if encoding == "raw":
+                out += struct.pack(">HHHHi", x, y, w, h, ENCODING_RAW)
+                out += _pixels(fmt, (v for row in values for v in row))
+            else:
+                raise ValueError(f"unknown encoding {encoding!r}")
+        return bytes(out)
+
+    async def send_update(
+        self,
+        rects: Sequence[Rect],
+        chunk: int | None = None,
+        *,
+        delay: float = 0.0,
+    ) -> None:
+        """Send one FramebufferUpdate built by ``build_update()``.
+
+        Args:
+            rects: The rectangles, in order.
+            chunk: Split the write into pieces of at most this many bytes; ``HOSTILE_CHUNK``
+                uses seeded random 1-7 byte pieces.
+            delay: Sleep between pieces.
+        """
+        session = self._require_session()
+        data = self.build_update(rects, pixel_format=session.pixel_format)
+        await self._write(session, data, chunk, delay)
+
+    async def send_cut_text(
+        self, text: bytes, chunk: int | None = None, *, delay: float = 0.0
+    ) -> None:
+        """Send ServerCutText."""
+        await self.send_raw(
+            struct.pack(">B3xI", 3, len(text)) + text, chunk, delay=delay
+        )
+
+    async def send_bell(self) -> None:
+        """Send Bell."""
+        await self.send_raw(b"\x02")
+
+    async def send_colour_map(
+        self,
+        first: int = 0,
+        colours: Sequence[RGB] = (),
+        chunk: int | None = None,
+    ) -> None:
+        """Send SetColourMapEntries; colours are 16 bit per channel."""
+        data = struct.pack(">BxHH", 1, first, len(colours))
+        data += b"".join(struct.pack(">HHH", *colour) for colour in colours)
+        await self.send_raw(data, chunk)
+
+    async def send_raw(
+        self, data: bytes, chunk: int | None = None, *, delay: float = 0.0
+    ) -> None:
+        """Send arbitrary bytes to the current client."""
+        await self._write(self._require_session(), data, chunk, delay)
+
+    async def close_client(self, *, abort: bool = False) -> None:
+        """Close the current client connection and wait for its handler to finish."""
+        if (session := self._session) is None:
+            return
+        if abort:
+            session.writer.transport.abort()
+        else:
+            session.writer.close()
+        if session.task is not None:
+            await asyncio.wait([session.task], timeout=5.0)
+
+    def _check_rect(self, x: int, y: int, w: int, h: int) -> None:
+        if min(x, y, w, h) < 0 or x + w > self.width or y + h > self.height:
+            raise ValueError(
+                f"rect {w}x{h}+{x}+{y} is outside the {self.width}x{self.height} framebuffer"
+            )
+
+    def _values(
+        self, fmt: PixelFormat, x: int, y: int, w: int, h: int
+    ) -> list[list[int]]:
+        cache: dict[bytes, int] = {}
+        rows = []
+        for line in range(y, y + h):
+            offset = (line * self.width + x) * 3
+            data = bytes(self.image[offset : offset + w * 3])
+            row = []
+            for i in range(0, len(data), 3):
+                rgb = data[i : i + 3]
+                if (value := cache.get(rgb)) is None:
+                    value = cache[rgb] = fmt.value_of((rgb[0], rgb[1], rgb[2]))
+                row.append(value)
+            rows.append(row)
+        return rows
+
+    def _require_session(self) -> _Session:
+        if self._session is None:
+            raise RuntimeError("no RFB client is connected")
+        return self._session
+
+    def _summary(self) -> str:
+        return (
+            f"connections={self._connections}, sessions={self._sessions}, "
+            f"pending={[m['type'] for m in self._pending]}, "
+            f"protocol_errors={self.protocol_errors}"
+        )
+
+    def _bump(self) -> None:
+        self._changed.set()
+        self._changed = asyncio.Event()
+
+    async def _wait[T](self, check: Callable[[], T | None], timeout: float) -> T | None:
+        loop = asyncio.get_running_loop()
+        deadline = loop.time() + timeout
+        while (result := check()) is None:
+            if (remaining := deadline - loop.time()) <= 0:
+                return None
+            with contextlib.suppress(TimeoutError):
+                await asyncio.wait_for(self._changed.wait(), remaining)
+        return result
+
+    async def _write(
+        self, session: _Session, data: bytes, chunk: int | None, delay: float = 0.0
+    ) -> None:
+        if chunk is not None and chunk != HOSTILE_CHUNK and chunk < 1:
+            raise ValueError(f"bad chunk size {chunk}")
+        writer = session.writer
+        async with self._write_lock:
+            if chunk is None:
+                writer.write(data)
+                await writer.drain()
+                return
+            pos = 0
+            while pos < len(data):
+                size = self._rng.randint(1, 7) if chunk == HOSTILE_CHUNK else chunk
+                writer.write(data[pos : pos + size])
+                await writer.drain()
+                pos += size
+                await asyncio.sleep(delay)
+
+    async def _reply(self, session: _Session, data: bytes) -> None:
+        await self._write(session, data, self.handshake_chunk)
+
+    def _record(self, session: _Session, message: Message) -> None:
+        self.messages.append(message)
+        if session is self._session:
+            self._pending.append(message)
+        self._bump()
+
+    async def _handle(
+        self, reader: asyncio.StreamReader, writer: asyncio.StreamWriter
+    ) -> None:
+        session = _Session(reader, writer, self.native_format, asyncio.current_task())
+        self._connections += 1
+        if (previous := self._session) is not None:
+            previous.writer.close()
+        self._session = session
+        self._open.add(session)
+        self._pending.clear()
+        self._bump()
+        try:
+            await self._serve(session)
+        except (asyncio.IncompleteReadError, ConnectionError):
+            pass
+        except _ProtocolError as err:
+            self.protocol_errors.append(str(err))
+        except Exception as err:  # noqa: BLE001 - reported through protocol_errors
+            _LOGGER.exception("Fake RFB server failed")
+            self.protocol_errors.append(f"internal error: {err!r}")
+        finally:
+            if self._session is session:
+                self._session = None
+            writer.close()
+            with contextlib.suppress(OSError):
+                await writer.wait_closed()
+            self._open.discard(session)
+            self._bump()
+
+    async def _serve(self, session: _Session) -> None:
+        await self._reply(session, self.version)
+        session.minor = self._negotiate_version(await session.reader.readexactly(12))
+        if not await self._negotiate_security(session):
+            return
+        self.shared_flag = (await session.reader.readexactly(1))[0]
+        server_init = (
+            struct.pack(">HH", self.width, self.height)
+            + self.native_format.pack()
+            + _string(self.name)
+        )
+        await self._reply(session, server_init)
+        self._sessions += 1
+        self._bump()
+        while True:
+            await self._read_message(session)
+
+    def _negotiate_version(self, data: bytes) -> int:
+        self.client_version = data
+        if (match := _VERSION_RE.fullmatch(data)) is None:
+            raise _ProtocolError(f"bad client version {data!r}")
+        if int(match[1]) != 3:
+            raise _ProtocolError(f"unsupported client version {data!r}")
+        minor = int(match[2])
+        if minor > 8:
+            return 8
+        return minor if minor in (7, 8) else 3
+
+    async def _negotiate_security(self, session: _Session) -> bool:
+        reason = self.refuse_with_reason
+        if reason is None and not self.security_types:
+            reason = ""
+        if session.minor == 3:
+            if reason is not None:
+                await self._reply(session, struct.pack(">I", 0) + _string(reason))
+                return False
+            chosen = next(
+                (t for t in self.security_types if t == SECURITY_NONE),
+                self.security_types[0],
+            )
+            await self._reply(session, struct.pack(">I", chosen))
+        else:
+            if reason is not None:
+                await self._reply(session, b"\0" + _string(reason))
+                return False
+            offer = bytes([len(self.security_types), *self.security_types])
+            await self._reply(session, offer)
+            chosen = (await session.reader.readexactly(1))[0]
+            if chosen not in self.security_types:
+                raise _ProtocolError(
+                    f"client chose security type {chosen}, offered {self.security_types}"
+                )
+        if chosen == SECURITY_NONE:
+            if session.minor >= 8:
+                await self._reply(session, struct.pack(">I", 0))
+            return True
+        raise _ProtocolError(f"security type {chosen} is not emulated")
+
+    async def _read_message(self, session: _Session) -> None:
+        read = session.reader.readexactly
+        match kind := (await read(1))[0]:
+            case 0:
+                fmt = PixelFormat.unpack((await read(19))[3:])
+                self._record(session, {"type": "set_pixel_format", **asdict(fmt)})
+                if fmt.bpp not in (8, 16, 32):
+                    raise _ProtocolError(f"unsupported bits per pixel {fmt.bpp}")
+                if not fmt.true_colour:
+                    raise _ProtocolError("colour map formats are not emulated")
+                session.pixel_format = fmt
+            case 2:
+                (count,) = struct.unpack(">xH", await read(3))
+                self.encodings = list(
+                    struct.unpack(f">{count}i", await read(4 * count))
+                )
+                self._record(
+                    session, {"type": "set_encodings", "encodings": self.encodings}
+                )
+            case 3:
+                incremental, x, y, w, h = struct.unpack(">BHHHH", await read(9))
+                self._record(
+                    session,
+                    {
+                        "type": "fbur",
+                        "incremental": bool(incremental),
+                        "x": x,
+                        "y": y,
+                        "w": w,
+                        "h": h,
+                    },
+                )
+            case 4:
+                down, keysym = struct.unpack(">B2xI", await read(7))
+                self._record(
+                    session, {"type": "key", "down": bool(down), "keysym": keysym}
+                )
+            case 5:
+                mask, x, y = struct.unpack(">BHH", await read(5))
+                self._record(session, {"type": "pointer", "mask": mask, "x": x, "y": y})
+            case 6:
+                (length,) = struct.unpack(">3xI", await read(7))
+                if length > MAX_CLIENT_CUT_TEXT:
+                    raise _ProtocolError(f"client cut text of {length} bytes")
+                self._record(session, {"type": "cut_text", "text": await read(length)})
+            case _:
+                raise _ProtocolError(f"unknown client message type {kind}")
+
+
+def _pixels(fmt: PixelFormat, values: Iterable[int]) -> bytes:
+    cache: dict[int, bytes] = {}
+    out = bytearray()
+    for value in values:
+        if (data := cache.get(value)) is None:
+            data = cache[value] = fmt.to_bytes(value)
+        out += data
+    return bytes(out)
+
+
+def _cursor_shape(fmt: PixelFormat, w: int, h: int) -> bytes:
+    light = fmt.to_bytes(fmt.value_of((255, 255, 255)))
+    dark = fmt.to_bytes(fmt.value_of((0, 0, 0)))
+    pixels = b"".join(
+        light if (x + y) % 2 else dark for y in range(h) for x in range(w)
+    )
+    return pixels + b"\x5a" * (((w + 7) // 8) * h)

tests/integration/test_api_action_metadata.py

@@ -38,7 +38,7 @@ async def test_api_action_metadata(
         _, services = await client.list_entities_services()
 
         by_name = {service.name: service for service in services}
-        assert set(by_name) == {"play_buzzer", "plain_action"}
+        assert set(by_name) == {"play_buzzer", "plain_action", "optional_args_action"}
         # Keys are hashed at codegen time and must match what the client expects
         for name, service in by_name.items():
             assert service.key == fnv1_hash(name), name
@@ -57,6 +57,17 @@ async def test_api_action_metadata(
         assert plain.description == ""
         assert plain.args[0].description == ""
 
+        # Optional flags and defaults, as (optional, default) per argument
+        def flags(service) -> dict[str, tuple[bool, str]]:
+            return {arg.name: (arg.optional, arg.default_value) for arg in service.args}
+
+        assert flags(plain) == {"value": (False, "")}
+        assert flags(by_name["optional_args_action"]) == {
+            "song_str": (True, ""),
+            "octave": (True, "5"),
+            "loud": (True, "true"),
+        }
+
         await client.execute_service(
             buzzer, {"song_str": "two_short:d=4,o=5,b=100:16e6,16e6", "volume": 3}
         )

tests/integration/test_api_cover_toggle.py

@@ -0,0 +1,108 @@
+"""The native API advertises cover toggle support and runs the toggle action on request."""
+
+from __future__ import annotations
+
+import asyncio
+from collections.abc import Callable
+import struct
+
+from aioesphomeapi import CoverInfo, CoverState, EntityState, api_pb2
+import pytest
+
+from .log_utils import LineWaiter
+from .raw_api_client import (
+    MESSAGE_TYPE_OF,
+    WIRE_FIXED32,
+    WIRE_VARINT,
+    RawApiClient,
+    decode_fields,
+    tag,
+)
+from .state_utils import InitialStateHelper, StateWaiter, require_entity
+from .types import APIClientConnectedFactory, RunCompiledFunction
+
+COVER_COMMAND = MESSAGE_TYPE_OF[api_pb2.CoverCommandRequest]
+LIST_COVER = MESSAGE_TYPE_OF[api_pb2.ListEntitiesCoverResponse]
+LIST_DONE = MESSAGE_TYPE_OF[api_pb2.ListEntitiesDoneResponse]
+# ListEntitiesCoverResponse fields; the installed client library does not know supports_toggle yet
+FIELD_ENTITY_KEY, FIELD_SUPPORTS_TOGGLE = 2, 15
+# CoverCommandRequest fields
+FIELD_KEY, FIELD_STOP, FIELD_TOGGLE = 1, 8, 11
+
+
+@pytest.mark.asyncio
+async def test_api_cover_toggle(
+    yaml_config: str,
+    run_compiled: RunCompiledFunction,
+    api_client_connected: APIClientConnectedFactory,
+    unused_tcp_port: int,
+) -> None:
+    lines = LineWaiter()
+    async with (
+        run_compiled(yaml_config, line_callback=lines.callback),
+        api_client_connected() as client,
+        RawApiClient(unused_tcp_port) as raw,
+    ):
+        entities, _ = await client.list_entities_services()
+        toggle_cover = require_entity(entities, "toggle_cover", CoverInfo, exact=True)
+        plain_cover = require_entity(entities, "plain_cover", CoverInfo, exact=True)
+
+        positions: dict[int, list[float]] = {toggle_cover.key: [], plain_cover.key: []}
+        waiter = StateWaiter()
+
+        def on_state(state: EntityState) -> None:
+            if isinstance(state, CoverState) and state.key in positions:
+                positions[state.key].append(state.position)
+            waiter.on_state(state)
+
+        def position_is(key: int, value: float) -> Callable[[EntityState], bool]:
+            return lambda s: (
+                isinstance(s, CoverState)
+                and s.key == key
+                and s.position == pytest.approx(value, abs=0.01)
+            )
+
+        initial = InitialStateHelper(entities)
+        client.subscribe_states(initial.on_state_wrapper(on_state))
+        await initial.wait_for_initial_states()
+
+        # Capability discovery: only the cover with a toggle_action advertises toggle
+        await raw.connect()
+        await raw.send_message(api_pb2.ListEntitiesRequest())
+        await raw.read_until_frame(LIST_DONE)
+        supports_toggle = {}
+        for payload in raw.frames(LIST_COVER):
+            fields = decode_fields(payload)
+            key = fields[FIELD_ENTITY_KEY][0]
+            supports_toggle[key] = bool(fields.get(FIELD_SUPPORTS_TOGGLE, [0])[0])
+        assert supports_toggle == {toggle_cover.key: True, plain_cover.key: False}
+
+        def key_field(key: int) -> bytes:
+            return tag(FIELD_KEY, WIRE_FIXED32) + struct.pack("<I", key)
+
+        toggle = tag(FIELD_TOGGLE, WIRE_VARINT) + b"\x01"
+        stop = tag(FIELD_STOP, WIRE_VARINT) + b"\x01"
+
+        # A toggle request runs the toggle action, which publishes 0.5
+        toggled = waiter.expect(
+            position_is(toggle_cover.key, 0.5), label="toggle action"
+        )
+        await raw.send_raw(COVER_COMMAND, key_field(toggle_cover.key) + toggle)
+        await toggled
+
+        # A cover without toggle_action drops the request with a warning; its state does not change
+        await raw.send_raw(COVER_COMMAND, key_field(plain_cover.key) + toggle)
+        await lines.wait_for("Plain Cover", "toggle unsupported", timeout=2.0)
+        await asyncio.sleep(0.5)
+        assert all(p == pytest.approx(1.0) for p in positions[plain_cover.key])
+
+        # Stop and toggle in one request: stop wins and the toggle action does not run
+        before = len(positions[toggle_cover.key])
+        stopped = waiter.expect(
+            position_is(toggle_cover.key, 0.25), label="stop action"
+        )
+        await raw.send_raw(COVER_COMMAND, key_field(toggle_cover.key) + stop + toggle)
+        await stopped
+        await asyncio.sleep(0.5)
+        after = positions[toggle_cover.key][before:]
+        assert all(p == pytest.approx(0.25, abs=0.01) for p in after), after

tests/integration/test_api_decode_wire_types.py

@@ -16,16 +16,19 @@ from aioesphomeapi import (
 )
 import pytest
 
-from .raw_api_client import MESSAGE_TYPE_OF, RawApiClient, encode_varint
+from .raw_api_client import (
+    MESSAGE_TYPE_OF,
+    WIRE_FIXED32,
+    WIRE_LENGTH,
+    WIRE_VARINT,
+    RawApiClient,
+    encode_varint,
+    tag,
+)
 from .state_utils import InitialStateHelper, StateWaiter, require_entity
 from .types import APIClientConnectedFactory, RunCompiledFunction
 
 SWITCH_COMMAND = MESSAGE_TYPE_OF[api_pb2.SwitchCommandRequest]
-WIRE_VARINT, WIRE_LENGTH, WIRE_FIXED32 = 0, 2, 5
-
-
-def tag(field: int, wire_type: int) -> bytes:
-    return encode_varint((field << 3) | wire_type)
 
 
 @pytest.mark.asyncio

tests/integration/test_includes_lambda_order.py

@@ -0,0 +1,31 @@
+"""Lambdas rendered at file scope can use the user's includes, which can name ids."""
+
+from __future__ import annotations
+
+from aioesphomeapi import SensorInfo, SensorState
+import pytest
+
+from .state_utils import require_entity, wait_for_state
+from .types import APIClientConnectedFactory, RunCompiledFunction
+
+
+@pytest.mark.asyncio
+async def test_includes_lambda_order(
+    yaml_config: str,
+    run_compiled: RunCompiledFunction,
+    api_client_connected: APIClientConnectedFactory,
+) -> None:
+    """The apply action and the sensor filter use the header, the header reads the global."""
+    async with run_compiled(yaml_config), api_client_connected() as client:
+        entities, _ = await client.list_entities_services()
+        sensor = require_entity(entities, "header_value", SensorInfo)
+        state = await wait_for_state(
+            client,
+            lambda s: (
+                isinstance(s, SensorState)
+                and s.key == sensor.key
+                and not s.missing_state
+            ),
+        )
+        # globals.set stored HEADER_CONSTANT and the header read it back doubled.
+        assert state.state == 84.0

tests/integration/test_vnc_client.py

@@ -0,0 +1,466 @@
+"""Integration test for the vnc component against a fake RFB server.
+
+One compiled device runs every step in order against one server: the first frame, request
+pacing, a reconnect after the server closes the link, a stream cut into tiny pieces, and an
+unknown message type that makes the client drop the link.
+
+The client asks for the next frame only once the previous one is fully drawn, so snapshots are
+taken after that request arrives. The snapshot display draws as soon as pixels arrive, so the
+picture does not depend on when the client refreshes the display.
+"""
+
+from __future__ import annotations
+
+import asyncio
+from collections.abc import Callable
+import contextlib
+from dataclasses import asdict
+from itertools import pairwise
+from pathlib import Path
+import random
+import re
+import struct
+
+from aioesphomeapi import APIClient, UserService
+import pytest
+
+from .bmp_utils import Bmp, wait_for_bmp
+from .rfb_fake_server import (
+    HOSTILE_CHUNK,
+    RGB,
+    RGB565_BE,
+    RGB565_LE,
+    FakeRfbServer,
+    Message,
+    PixelFormat,
+    Rect,
+)
+from .types import APIClientConnectedFactory, RunCompiledFunction
+
+DISPLAY_WIDTH = 64
+DISPLAY_HEIGHT = 48
+SERVER_WIDTH = 80
+SERVER_HEIGHT = 60
+
+FIXTURE_PORT = "port: 15900"
+CONNECTED = "VNC_TEST_CONNECTED"
+DISCONNECTED = "VNC_TEST_DISCONNECTED"
+FRAME_LOG = re.compile(r"Update: (\d+) bytes in \d+ ?ms")
+LONG_OPERATION = "vnc took a long time"
+
+# Well under the fixture's 300 ms update_delay: the next request must not wait for the refresh.
+QUICK_REQUEST_S = 0.25
+# Requests are at least 50 ms apart; leave room for jitter in when they arrive.
+REQUEST_SPACING_S = 0.035
+# The fixture's reconnect_interval.
+RECONNECT_S = 1.0
+# Long enough for a frame that drew something to be logged.
+LOG_SETTLE_S = 0.5
+
+# Raw rects that cross the right and bottom edges of the display.
+FULL_FRAME: list[Rect] = [
+    (0, 0, SERVER_WIDTH, 13, "raw"),
+    (0, 13, 37, SERVER_HEIGHT - 13, "raw"),
+    (37, 13, SERVER_WIDTH - 37, SERVER_HEIGHT - 13, "raw"),
+]
+
+LogMatch = str | re.Pattern[str] | Callable[[str], bool]
+
+
+class LogWatch:
+    """Device log lines with the time each one arrived."""
+
+    def __init__(self) -> None:
+        self.lines: list[tuple[float, str]] = []
+        self._changed = asyncio.Event()
+
+    def callback(self, line: str) -> None:
+        self.lines.append((asyncio.get_running_loop().time(), line))
+        self._changed.set()
+        self._changed = asyncio.Event()
+
+    def times(self, match: LogMatch) -> list[float]:
+        """When each matching line came."""
+        if isinstance(match, str):
+            text = match
+
+            def test(line: str) -> bool:
+                return text in line
+
+        elif isinstance(match, re.Pattern):
+            pattern = match
+
+            def test(line: str) -> bool:
+                return pattern.search(line) is not None
+
+        else:
+            test = match
+        return [when for when, line in self.lines if test(line)]
+
+    def count(self, match: LogMatch) -> int:
+        return len(self.times(match))
+
+    def frame_sizes(self) -> list[int]:
+        """The byte counts of the 'Update:' lines, in order."""
+        return [
+            int(found[1]) for _, line in self.lines if (found := FRAME_LOG.search(line))
+        ]
+
+    async def wait_count(
+        self, match: LogMatch, count: int, timeout: float = 10.0
+    ) -> float:
+        """Wait until ``match`` was logged ``count`` times and return when the last one came."""
+        loop = asyncio.get_running_loop()
+        deadline = loop.time() + timeout
+        while len(times := self.times(match)) < count:
+            if (remaining := deadline - loop.time()) <= 0:
+                raise AssertionError(
+                    f"{match!r} logged {len(times)} times within {timeout}s, "
+                    f"expected {count}"
+                )
+            with contextlib.suppress(TimeoutError):
+                await asyncio.wait_for(self._changed.wait(), remaining)
+        return times[count - 1]
+
+
+def paint(server: FakeRfbServer, seed: int) -> None:
+    """Draw solid bands, boxes and noise.
+
+    Noise also runs along the right and bottom edges of the display, so clipping errors show.
+    """
+    rng = random.Random(seed)
+
+    def colour() -> RGB:
+        return rng.randrange(256), rng.randrange(256), rng.randrange(256)
+
+    width, height = server.width, server.height
+    for x in range(0, width, 20):
+        server.fill_rect(x, 0, min(20, width - x), height, colour())
+    accent = colour()
+    for _ in range(10):
+        server.fill_rect(
+            rng.randrange(width - 8),
+            rng.randrange(height - 8),
+            rng.randint(1, 8),
+            rng.randint(1, 8),
+            accent,
+        )
+    for _ in range(10):
+        server.fill_rect(
+            rng.randrange(width - 4),
+            rng.randrange(height - 4),
+            rng.randint(1, 4),
+            rng.randint(1, 4),
+            colour(),
+        )
+    noise = [
+        (x, y)
+        for x in range(DISPLAY_WIDTH - 6, min(width, DISPLAY_WIDTH + 6))
+        for y in range(height)
+    ]
+    noise += [
+        (x, y)
+        for y in range(DISPLAY_HEIGHT - 5, min(height, DISPLAY_HEIGHT + 5))
+        for x in range(width)
+    ]
+    noise += [(x, y) for x in range(8, 24) for y in range(16, 32)]
+    for x, y in noise:
+        server.set_pixel(x, y, colour())
+
+
+def shown_colour(fmt: PixelFormat, rgb: RGB) -> RGB:
+    """The colour a snapshot holds for a server pixel, after the trip through RGB565."""
+    value = fmt.value_of(rgb)
+    return (
+        ((value >> fmt.red_shift) & fmt.red_max) * 255 // fmt.red_max,
+        ((value >> fmt.green_shift) & fmt.green_max) * 255 // fmt.green_max,
+        ((value >> fmt.blue_shift) & fmt.blue_max) * 255 // fmt.blue_max,
+    )
+
+
+def assert_shows(bmp: Bmp, server: FakeRfbServer) -> None:
+    """Assert the snapshot is the top-left of the server screen, pixel for pixel."""
+    assert (bmp.width, bmp.height, bmp.bits) == (DISPLAY_WIDTH, DISPLAY_HEIGHT, 24)
+    mismatches = []
+    for y in range(DISPLAY_HEIGHT):
+        row = (DISPLAY_HEIGHT - 1 - y) * DISPLAY_WIDTH * 3
+        for x in range(DISPLAY_WIDTH):
+            b, g, r = bmp.pixels[row + x * 3 : row + x * 3 + 3]
+            expected = shown_colour(RGB565_LE, server.get_pixel(x, y))
+            if (r, g, b) != expected:
+                mismatches.append(((x, y), (r, g, b), expected))
+    assert not mismatches, (
+        f"{len(mismatches)} pixels differ; (x, y), shown, expected: {mismatches[:10]}"
+    )
+
+
+def full_request(incremental: bool) -> Message:
+    return {
+        "type": "fbur",
+        "incremental": incremental,
+        "x": 0,
+        "y": 0,
+        "w": DISPLAY_WIDTH,
+        "h": DISPLAY_HEIGHT,
+    }
+
+
+def cut_text(text: bytes) -> bytes:
+    return struct.pack(">B3xI", 3, len(text)) + text
+
+
+class Device:
+    """The API actions of the fixture."""
+
+    def __init__(
+        self, client: APIClient, services: list[UserService], snapshot_dir: Path
+    ) -> None:
+        self.client = client
+        self.actions = {service.name: service for service in services}
+        self.snapshot_dir = snapshot_dir
+
+    async def run(self, action: str, **data: int | str) -> None:
+        await self.client.execute_service(self.actions[action], data)
+
+    async def snapshot(self, name: str) -> Bmp:
+        await self.run("take_snapshot", filename=name)
+        return await wait_for_bmp(self.snapshot_dir / f"{name}.bmp")
+
+
+async def next_request(
+    server: FakeRfbServer, timeout: float = 5.0
+) -> tuple[Message, float]:
+    """Wait for the next FramebufferUpdateRequest and return it and when it came."""
+    request = await server.wait_for("fbur", timeout=timeout)
+    return request, asyncio.get_running_loop().time()
+
+
+async def exchange(
+    server: FakeRfbServer,
+    data: bytes,
+    *,
+    log: LogWatch | None = None,
+    frame: bytes | None = None,
+    chunk: int | None = None,
+    timeout: float = 5.0,
+) -> tuple[float, float]:
+    """Send bytes that end a frame and wait for the incremental request that follows.
+
+    With ``log``, also wait for the frame's 'Update:' line and check its byte count; ``frame``
+    is the FramebufferUpdate inside ``data`` when other messages surround it.
+
+    Returns how long after the write the request came, and when it came.
+    """
+    logged = log.count(FRAME_LOG) if log is not None else 0
+    await server.send_raw(data, chunk)
+    sent = asyncio.get_running_loop().time()
+    request, requested = await next_request(server, timeout)
+    assert request == full_request(incremental=True)
+    if log is not None:
+        await expect_logged(log, data if frame is None else frame, logged)
+    return requested - sent, requested
+
+
+async def expect_logged(log: LogWatch, frame: bytes, before: int) -> None:
+    """Wait for the 'Update:' line after the first ``before`` and check its byte count."""
+    await log.wait_count(FRAME_LOG, before + 1)
+    sizes = log.frame_sizes()
+    assert len(sizes) == before + 1, f"expected {before + 1} logged frames: {sizes}"
+    # The four byte message header may or may not be counted.
+    assert sizes[-1] in (len(frame) - 4, len(frame)), (
+        f"logged {sizes[-1]} bytes for a {len(frame)} byte update"
+    )
+
+
+@pytest.mark.asyncio
+async def test_vnc_client(
+    yaml_config: str,
+    run_compiled: RunCompiledFunction,
+    api_client_connected: APIClientConnectedFactory,
+    tmp_path: Path,
+    monkeypatch: pytest.MonkeyPatch,
+) -> None:
+    """The client renders, paces and recovers from drops and bad streams."""
+    snapshot_dir = tmp_path / "snapshots"
+    monkeypatch.setenv("ESPHOME_SNAPSHOT_DIR", str(snapshot_dir))
+    loop = asyncio.get_running_loop()
+    log = LogWatch()
+    server = FakeRfbServer(SERVER_WIDTH, SERVER_HEIGHT, name="vnc client test")
+    paint(server, seed=1)
+
+    async with server:
+        assert FIXTURE_PORT in yaml_config
+        yaml_config = yaml_config.replace(FIXTURE_PORT, f"port: {server.port}")
+        async with (
+            run_compiled(yaml_config, line_callback=log.callback),
+            api_client_connected() as client,
+        ):
+            _, services = await client.list_entities_services()
+            device = Device(client, services, snapshot_dir)
+
+            # Render: the session setup is exact and the first frame is pixel-exact.
+            pixel_format = await server.wait_for("set_pixel_format", timeout=15.0)
+            # The snapshot display declares no byte order, so the client asks for big-endian pixels
+            assert pixel_format == {"type": "set_pixel_format", **asdict(RGB565_BE)}
+            encodings = await server.wait_for("set_encodings")
+            assert encodings["encodings"] == [0, -239]
+            request, _ = await next_request(server)
+            assert request == full_request(incremental=False)
+            assert [m["type"] for m in server.messages[:3]] == [
+                "set_pixel_format",
+                "set_encodings",
+                "fbur",
+            ]
+            assert server.client_version == b"RFB 003.008\n"
+            assert server.shared_flag == 1
+            await log.wait_count(CONNECTED, 1)
+            assert log.count("Server 'vnc client test', 80x60, RFB 3.8") == 1
+            assert log.count("showing the top-left 64x48") == 1
+
+            elapsed, _ = await exchange(
+                server, server.build_update(FULL_FRAME), log=log
+            )
+            assert elapsed < QUICK_REQUEST_S, f"next request after {elapsed:.3f}s"
+            assert_shows(await device.snapshot("render"), server)
+
+            # Pacing: one request stays parked while nothing changes.
+            await asyncio.sleep(1.0)
+            assert not [m for m in server.pending() if m["type"] == "fbur"]
+
+            # The next request goes out as soon as a frame is drawn, before the refresh.
+            server.fill_rect(30, 10, 10, 10, (250, 10, 10))
+            server.fill_rect(58, 40, 22, 20, (10, 250, 10))
+            frame = server.build_update(
+                [(30, 10, 10, 10, "raw"), (58, 40, 22, 20, "raw")]
+            )
+            elapsed, _ = await exchange(server, frame, log=log)
+            assert elapsed < QUICK_REQUEST_S, f"next request after {elapsed:.3f}s"
+            assert_shows(await device.snapshot("pacing"), server)
+
+            # A frame that only changes what the display cannot show draws nothing.
+            logged = log.count(FRAME_LOG)
+            server.fill_rect(
+                DISPLAY_WIDTH, 0, SERVER_WIDTH - DISPLAY_WIDTH, SERVER_HEIGHT, (1, 2, 3)
+            )
+            server.fill_rect(
+                0,
+                DISPLAY_HEIGHT,
+                SERVER_WIDTH,
+                SERVER_HEIGHT - DISPLAY_HEIGHT,
+                (200, 100, 50),
+            )
+            hidden = server.build_update(
+                [
+                    (
+                        DISPLAY_WIDTH,
+                        0,
+                        SERVER_WIDTH - DISPLAY_WIDTH,
+                        SERVER_HEIGHT,
+                        "raw",
+                    ),
+                    (
+                        0,
+                        DISPLAY_HEIGHT,
+                        SERVER_WIDTH,
+                        SERVER_HEIGHT - DISPLAY_HEIGHT,
+                        "raw",
+                    ),
+                ]
+            )
+            elapsed, requested = await exchange(server, hidden)
+            assert elapsed < QUICK_REQUEST_S, f"next request after {elapsed:.3f}s"
+
+            # Empty and cursor-only updates draw nothing either, and requests stay 50 ms
+            # apart.
+            requested_at = [requested]
+            for update in (
+                server.build_update([]),
+                *(
+                    server.build_update([(2 + i, 3, 10, 16, "rich-cursor")])
+                    for i in range(3)
+                ),
+            ):
+                elapsed, requested = await exchange(server, update)
+                assert elapsed < QUICK_REQUEST_S, f"next request after {elapsed:.3f}s"
+                requested_at.append(requested)
+            gaps = [later - earlier for earlier, later in pairwise(requested_at)]
+            assert min(gaps) >= REQUEST_SPACING_S, f"requests {gaps} apart"
+            await asyncio.sleep(LOG_SETTLE_S)
+            assert log.count(FRAME_LOG) == logged, (
+                "a frame that drew nothing was logged"
+            )
+            assert_shows(await device.snapshot("hidden"), server)
+
+            # Reconnect: the server closes the link; the client comes back after the backoff
+            # and sets the session up again.
+            server.discard_pending()
+            closed = loop.time()
+            await server.close_client()
+            await log.wait_count(DISCONNECTED, 1)
+            await server.wait_until(
+                lambda: server.connections == 2, timeout=10.0, what="second connection"
+            )
+            reconnected = loop.time()
+            assert reconnected - closed >= RECONNECT_S - 0.05, (
+                f"reconnected after {reconnected - closed:.3f}s"
+            )
+            await log.wait_count(CONNECTED, 2)
+            assert await server.wait_for("set_pixel_format") == pixel_format
+            assert (await server.wait_for("set_encodings"))["encodings"] == [0, -239]
+            request, _ = await next_request(server)
+            assert request == full_request(incremental=False)
+            assert server.sessions == 2
+
+            # Hostile stream: 1-7 byte pieces, a long cut text, bells, a colour map and
+            # cursor rects around a frame.
+            paint(server, seed=2)
+            frame = server.build_update(
+                [
+                    (3, 4, 12, 18, "rich-cursor"),
+                    (0, 0, DISPLAY_WIDTH, 20, "raw"),
+                    (DISPLAY_WIDTH, 0, SERVER_WIDTH - DISPLAY_WIDTH, 60, "raw"),
+                    (0, 0, 0, 0, "rich-cursor"),
+                    (0, 20, 30, SERVER_HEIGHT - 20, "raw"),
+                    (30, 20, SERVER_WIDTH - 30, SERVER_HEIGHT - 20, "raw"),
+                    (1, 1, 7, 9, "rich-cursor"),
+                ]
+            )
+            colour_map = struct.pack(">BxHH", 1, 0, 2) + bytes(12)
+            long_cut = bytes(
+                random.Random(3).randrange(32, 127) for _ in range(200_000)
+            )
+            payload = (
+                cut_text(long_cut)
+                + b"\x02"
+                + colour_map
+                + frame
+                + b"\x02"
+                + cut_text(b"")
+            )
+            await exchange(
+                server, payload, log=log, frame=frame, chunk=HOSTILE_CHUNK, timeout=30.0
+            )
+            assert_shows(await device.snapshot("hostile"), server)
+
+            # An unknown message type drops the link. This session was too short to count as
+            # stable, so the backoff has doubled.
+            server.discard_pending()
+            sent = loop.time()
+            await server.send_raw(b"\x09")
+            await log.wait_count(DISCONNECTED, 2)
+            assert log.count("Unknown message type 9") == 1
+            await server.wait_until(
+                lambda: server.connections == 3, timeout=15.0, what="third connection"
+            )
+            assert loop.time() - sent >= 2 * RECONNECT_S - 0.1, (
+                f"reconnected after {loop.time() - sent:.3f}s"
+            )
+            await log.wait_count(CONNECTED, 3)
+            request, _ = await next_request(server)
+            assert request == full_request(incremental=False)
+            paint(server, seed=6)
+            await exchange(server, server.build_update(FULL_FRAME), log=log)
+            assert_shows(await device.snapshot("third"), server)
+
+            assert server.protocol_errors == []
+            assert log.count(LONG_OPERATION) == 0

tests/unit_tests/components/test_wifi.py

@@ -327,10 +327,14 @@ def test_wisp_requires_idf_5_1(wisp_core: None) -> None:
 
 
 @pytest.mark.usefixtures("wisp_core")
-def test_wisp_requires_sta_network() -> None:
-    """WISP has nothing to share without an upstream network."""
-    with pytest.raises(cv.Invalid, match="at least one STA network"):
-        _final_validate({CONF_NETWORKS: [], CONF_AP: _wisp_ap()})
+def test_wisp_requires_sta_network_or_ethernet() -> None:
+    """WISP has nothing to share without an upstream network; ethernet is one too."""
+    ap_only = {CONF_NETWORKS: [], CONF_AP: _wisp_ap()}
+    with pytest.raises(cv.Invalid, match="ethernet as the upstream"):
+        _final_validate(ap_only)
+    _final_validate(ap_only, {"ethernet": {}})
+    with pytest.raises(cv.Invalid, match="ethernet manual_ip needs dns1"):
+        _final_validate(ap_only, {"ethernet": {CONF_MANUAL_IP: _manual_ip("0.0.0.0")}})
 
 
 @pytest.mark.usefixtures("wisp_core")

tests/unit_tests/core/test_config.py

@@ -26,7 +26,7 @@ from esphome.const import (
     KEY_TARGET_PLATFORM,
     Toolchain,
 )
-from esphome.core import CORE, KEY_CONTROLLER_REGISTRY_CONTROLLERS, config
+from esphome.core import CORE, ID, KEY_CONTROLLER_REGISTRY_CONTROLLERS, config
 from esphome.core.config import (
     CONF_RELATION,
     CONF_SUSPEND_LOOP,
@@ -1114,6 +1114,40 @@ async def test_add_includes_with_single_file(
     assert any('#include "my_header.h"' in inc for inc in includes_added)
 
 
+@pytest.mark.asyncio
+async def test_add_includes_sit_between_declarations_and_generated_code(
+    tmp_path: Path, mock_copy_file_if_changed: Mock
+) -> None:
+    """A header can name an id, and generated functions and tables can use the header."""
+    CORE.config_path = tmp_path / "config.yaml"
+    CORE.build_path = tmp_path / "build"
+    CORE.build_path.mkdir(parents=True, exist_ok=True)
+    header = tmp_path / "my_header.h"
+    header.write_text("static constexpr int MY_CONSTANT = 42;")
+    CORE.config = {}
+
+    value = cg.new_Pvariable(ID("my_value", is_declaration=True, type=cg.int32))
+    cg.static_function(
+        "esphome__my_action__fn", cg.void, [], [f"*{value} = MY_CONSTANT;"]
+    )
+    cg.shared_progmem_array("my_table", cg.int32, [cg.RawExpression("MY_CONSTANT")])
+    await config.add_includes([str(header)])
+    # A job that runs after the includes still lands before the generated code.
+    cg.add_global(cg.RawStatement("static constexpr int FINAL_JOB = 1;"))
+
+    section = CORE.cpp_global_section
+    expected_order = [
+        "*const my_value",
+        "static void esphome__my_action__fn();",
+        '#include "my_header.h"',
+        "static constexpr int FINAL_JOB = 1;",
+        "static void esphome__my_action__fn() {",
+        "my_table[] PROGMEM",
+    ]
+    positions = [section.index(text) for text in expected_order]
+    assert positions == sorted(positions), section
+
+
 @pytest.mark.asyncio
 @pytest.mark.skipif(os.name == "nt", reason="Unix-specific test")
 async def test_add_includes_with_directory_unix(

tests/unit_tests/test_automation.py

@@ -785,8 +785,9 @@ class MockCodegen(NamedTuple):
     new_pvariable: MagicMock
     register_parented: AsyncMock
     add_global: MagicMock
+    add_global_after_includes: MagicMock
     add: MagicMock
-    calls: MagicMock  # new_pvariable and add_global attached, to check their order
+    calls: MagicMock  # the global emitters and new_pvariable attached, to check their order
 
 
 @pytest.fixture
@@ -848,6 +849,9 @@ def mock_cg() -> Generator[MockCodegen]:
             "esphome.codegen.register_parented", new_callable=AsyncMock
         ) as register_parented,
         patch("esphome.cpp_generator.add_global") as add_global,
+        patch(
+            "esphome.cpp_generator.add_global_after_includes"
+        ) as add_global_after_includes,
         patch("esphome.codegen.add") as add,
     ):
         get_variable.return_value = PARENT_OBJ
@@ -855,8 +859,15 @@ def mock_cg() -> Generator[MockCodegen]:
         calls = MagicMock()
         calls.attach_mock(new_pvariable, "new_pvariable")
         calls.attach_mock(add_global, "add_global")
+        calls.attach_mock(add_global_after_includes, "add_global_after_includes")
         yield MockCodegen(
-            get_variable, new_pvariable, register_parented, add_global, add, calls
+            get_variable,
+            new_pvariable,
+            register_parented,
+            add_global,
+            add_global_after_includes,
+            add,
+            calls,
         )
 
 
@@ -984,7 +995,7 @@ async def _run_apply_condition(
 
 
 def _apply_definition(mock_cg: MockCodegen) -> str:
-    return str(mock_cg.add_global.call_args.args[0])
+    return str(mock_cg.add_global_after_includes.call_args.args[0])
 
 
 @pytest.mark.asyncio
@@ -998,8 +1009,16 @@ async def test_register_apply_action_entry(
     action_id, template_arg = mock_cg.new_pvariable.call_args.args
     assert action_id == ID("obj_1")
     assert str(template_arg) == "<esphome__obj_1__fn, int32_t>"
-    # The definition must precede the storage line that names the function.
-    assert [c[0] for c in mock_cg.calls.mock_calls] == ["add_global", "new_pvariable"]
+    # The declaration must precede the storage line that names the function; the body
+    # goes after the user includes.
+    assert [c[0] for c in mock_cg.calls.mock_calls] == [
+        "add_global",
+        "add_global_after_includes",
+        "new_pvariable",
+    ]
+    assert str(mock_cg.add_global.call_args.args[0]) == (
+        "static void esphome__obj_1__fn(const std::remove_cvref_t<int32_t> & x);"
+    )
     assert _apply_definition(mock_cg).startswith(
         "static void esphome__obj_1__fn(const std::remove_cvref_t<int32_t> & x) {"
     )

tests/unit_tests/test_cpp_generator.py

@@ -866,7 +866,8 @@ class TestSharedProgmemArray:
         c = cg.shared_progmem_array("table", ct.uint8, [4])
         assert a is b
         assert str(a) != str(c)
-        assert sum("PROGMEM" in str(st) for st in CORE.global_statements) == 2
+        late = CORE.global_statements_after_includes
+        assert sum("PROGMEM" in str(st) for st in late) == 2
         assert not any("PROGMEM" in str(st) for st in CORE.main_statements)
 
     def test_share_false_always_emits_a_new_array(self) -> None:
@@ -874,7 +875,8 @@ class TestSharedProgmemArray:
         a = cg.shared_progmem_array("table", ct.uint8, [1], share=False)
         b = cg.shared_progmem_array("table", ct.uint8, [1], share=False)
         assert str(a) != str(b)
-        assert sum("PROGMEM" in str(st) for st in CORE.global_statements) == 2
+        late = CORE.global_statements_after_includes
+        assert sum("PROGMEM" in str(st) for st in late) == 2
 
     def test_is_static_pointer_only_for_placement_new(self) -> None:
         CORE.config = {}
@@ -910,7 +912,9 @@ class TestSharedProgmemArray:
         CORE.config = {}
         cg.shared_progmem_array("table", ct.uint8, [1], constexpr=False)
         (statement,) = (
-            str(st) for st in CORE.global_statements if "PROGMEM" in str(st)
+            str(st)
+            for st in CORE.global_statements_after_includes
+            if "PROGMEM" in str(st)
         )
         assert statement.startswith(
             "ESPHOME_FLASH_CONSTINIT static uint8_t const table[] PROGMEM = {1}"
@@ -948,7 +952,7 @@ class TestProgmemString:
         assert a is b
         assert str(a) == "progmem_str"
         assert str(c) == "progmem_str_2"
-        assert [str(st) for st in CORE.global_statements] == [
+        assert [str(st) for st in CORE.global_statements_after_includes] == [
             'static constexpr char progmem_str[] PROGMEM = "level";',
             'static constexpr char progmem_str_2[] PROGMEM = "say \\042hi\\042";',
         ]
@@ -957,6 +961,7 @@ class TestProgmemString:
         self._set_platform(PLATFORM_ESP32)
         assert str(cg.shared_progmem_string('say "hi"')) == '"say \\042hi\\042"'
         assert not CORE.global_statements
+        assert not CORE.global_statements_after_includes
 
 
 def test_extern_progmem_array_is_an_externally_linked_global() -> None:
@@ -967,6 +972,22 @@ def test_extern_progmem_array_is_an_externally_linked_global() -> None:
     ]
 
 
+def test_static_function_declares_early_and_defines_after_includes() -> None:
+    fn = cg.static_function("esphome__act__fn", ct.void, [(ct.int32, "x")], ["return;"])
+    assert str(fn) == "esphome__act__fn"
+    assert [str(st) for st in CORE.global_statements] == [
+        "static void esphome__act__fn(int32_t x);"
+    ]
+    assert [str(st) for st in CORE.global_statements_after_includes] == [
+        "static void esphome__act__fn(int32_t x) {\n  return;\n}"
+    ]
+
+
+def test_add_global_after_includes_rejects_non_statements() -> None:
+    with pytest.raises(ValueError, match="must be expression or statement"):
+        CORE.add_global_after_includes("not a statement")
+
+
 class TestEnumOptions:
     def test_class_enum_with_prefix(self) -> None:
         mode = cg.MockObj("esphome::foo", "::").enum("Mode", is_class=True)

tests/unit_tests/test_espidf_component.py

@@ -254,6 +254,20 @@ target_link_libraries(${COMPONENT_LIB} INTERFACE
     )
 
 
+def test_generate_cmakelists_txt_optimisation_flags_are_private(
+    tmp_component: IDFComponent,
+) -> None:
+    # A library built with -O2 must not change how the components that use it are compiled
+    src_dir = tmp_component.path / "src"
+    src_dir.mkdir()
+    (src_dir / "main.c").write_text("int main() {}")
+    tmp_component.data = {"build": {"flags": ["-O2", "-DTEST"]}}
+
+    content = generate_cmakelists_txt(tmp_component)
+    assert 'target_compile_options(${COMPONENT_LIB} PUBLIC\n  "-DTEST"\n)' in content
+    assert 'target_compile_options(${COMPONENT_LIB} PRIVATE\n  "-O2"\n)' in content
+
+
 def test_generate_cmakelists_txt_uses_forward_slashes_on_windows(
     tmp_component, monkeypatch: pytest.MonkeyPatch
 ) -> None: