Compare commits

..
Author SHA1 Message Date
kbx81 70c6021986 Merge branch 'central-netif' into multi-interface-poc
# Conflicts:
#	esphome/components/wifi/wifi_component.cpp
2026-05-23 20:19:32 -05:00
Keith Burzinski 1485675928 Merge branch 'dev' into central-netif 2026-05-23 20:09:33 -05:00
Keith BurzinskiandJ. Nick Koston 3ed1356bb6 type
Co-authored-by: J. Nick Koston <nick+github@koston.org>
2026-05-23 20:08:08 -05:00
J. Nick Koston d6bc4fea1c Merge branch 'dev' into central-netif 2026-05-23 13:34:04 -05:00
kbx81 f818bddac8 Merge remote-tracking branch 'upstream/dev' into multi-interface-poc 2026-05-21 21:39:48 -05:00
kbx81 1f0af903ea feat(ethernet): Unit B + B+ — enable_on_boot lifecycle with lazy-init
Brings ethernet to feature parity with wifi's per-interface lifecycle, and
applies the same lazy-init pattern as wifi's Unit B+ so enable_on_boot:false
genuinely reclaims DMA-capable internal SRAM.

Unit B — lifecycle API surface (mirrors WiFiComponent):
- New `enable_on_boot: true` (default) YAML option on ethernet.
- New set_enable_on_boot(), enable(), disable(), is_disabled(), is_enabled()
  methods on EthernetComponent.
- ESP32 path: enable() calls esp_eth_start(), disable() calls esp_eth_stop().
- RP2040 path: stub methods that log a warning — arduino-pico's
  LwipIntfDev doesn't expose a clean start/stop hook; schema parity only.

Unit B+ — lazy-init refactor (mirrors WiFiComponent::wifi_lazy_init_()):
- New ethernet_lazy_init_() method (idempotent, guarded by
  ethernet_initialized_ flag) holds the entire heavy init body that used to
  live in setup(): SPI bus init, netif creation, MAC/PHY allocation, eth
  driver install, netif attach, event handler registration.
- setup() becomes thin: 300ms power-stabilization delay, then if
  enable_on_boot_=true call lazy_init + esp_eth_start, else mark
  disabled_=true and return.
- enable() calls ethernet_lazy_init_() first, then esp_eth_start() — so a
  runtime enable after enable_on_boot:false works end-to-end.

Safe-default getter guards — external callers (sendspin, ethernet_info,
mdns, etc.) may invoke MAC/IP/duplex queries before/regardless of whether
ethernet is enabled. Without guards these call into esp_eth_ioctl(null, ...)
and esp_netif_get_*(null, ...), producing error spam + erroneous
mark_failed() calls during dump_config():
- get_eth_mac_address_raw() falls back to esp_read_mac(ESP_MAC_ETH) — the
  hardware MAC, same value the driver would have returned.
- get_duplex_mode() returns ETH_DUPLEX_HALF.
- get_link_speed() returns ETH_SPEED_10M.
- get_ip_addresses() returns empty (zero) addresses.
- dump_connect_params_() early-returns with "(uninitialized)" log line.

For a user's reboot-to-toggle workflow with both interfaces declared but
only one active per boot: the inactive interface costs zero DMA-capable
memory. WiFi-side reclaims ~15-30KB DMA-capable, ethernet-side reclaims
~3-8KB (W5500 SPI driver is gentler than wifi).

Field-tested on ESP32-S3 + W5500. Verifies clean dump_config() output and
no false "ethernet was marked as failed" state when ethernet is dormant.
2026-05-21 21:38:45 -05:00
kbx81 ed289390df feat(wifi): Unit B+ — defer esp_wifi_init() to lazy-init
WiFi already had enable_on_boot + enable()/disable()/is_disabled() lifecycle,
but enable_on_boot:false didn't actually save any memory. wifi_pre_setup_()
called esp_wifi_init() and esp_netif_create_default_wifi_sta() unconditionally
during setup(), which allocates ~15-30KB of DMA-capable internal SRAM (RX/TX
buffers, driver state, PHY init). The flag only skipped esp_wifi_start() in
the followup branch — the driver was already resident, just not associated.

This commit splits wifi_pre_setup_() into two parts:

- wifi_pre_setup_() (light, kept in setup() always): MAC setup, event group
  creation, WIFI_EVENT/IP_EVENT handler registration. No DMA allocation.

- wifi_lazy_init_() (heavy, NEW): esp_netif_create_default_wifi_sta()/_ap(),
  esp_wifi_init(), esp_wifi_set_storage(). The DMA-allocating calls.
  Guarded by wifi_initialized_ flag for idempotency.

setup() calls wifi_lazy_init_() only when enable_on_boot_=true. The else
branch sets WIFI_COMPONENT_STATE_DISABLED without any heavy init — the
dormant interface costs zero DMA-capable memory.

enable() calls wifi_lazy_init_() before start(), so a runtime enable after
boot-time disable does the heavy init on demand. Idempotent — subsequent
enable/disable cycles don't re-allocate.

disable() is unchanged — it stops wifi but doesn't deinit. A future
"release_on_disable" variant could call esp_wifi_deinit() to actually free
the memory at runtime, but that requires coordinating with consumers
holding wifi-bound sockets and is out of scope here.

ESP-IDF only. Other platforms (Arduino on ESP32, ESP8266) keep the existing
behavior — their wifi_pre_setup_() lives in different per-platform files.

Field-tested on ESP32-S3 with W5500 SPI ethernet + audio + bluetooth_proxy.
Before Unit B+: ~14KB free internal during peak load, crash on W5500 SPI
DMA buffer allocation. After Unit B+: ~32KB free internal, Min Free 78KB
in some test configurations — sufficient headroom for the other DMA
consumers (I2S audio, BT controller) to operate.
2026-05-21 21:32:38 -05:00
kbx81 8f3010ac64 feat(network): Unit A — explicit default-route management
Builds on PR #14012's NetworkComponent + PR #14255's priority list to make
the user's stated interface priority actually drive runtime default-route
selection. Without this, ESP-IDF's auto-selection picks the default netif by
each netif's hardcoded `route_prio` field (WiFi STA = 100, Ethernet = 50,
WiFi AP = 10) — which inverts the user's intent on same-subnet
multi-homing configurations where wifi+ethernet share a broadcast domain.

Changes:

- NetworkComponent gains an IP_EVENT handler registered in setup() that
  re-arbitrates the default netif on every interface up/down. The handler
  walks the priority list in order, picks the highest-priority netif that
  is up, and calls esp_netif_set_default_netif() on it. ESP-IDF then sets
  its internal "manual override" flag so subsequent auto-selection events
  don't undo our choice.
- New StaticVector<NetworkPriorityEntry, 4> stores the priority list with
  zero heap allocation. The interface-name string pointer is a YAML literal
  with static storage duration.
- The timeout_ms field is parsed and stored but not yet consumed by Unit A;
  it's wired up for Unit D (runtime timeout fallback).
- New getters get_active_interface() / get_active_netif() expose the
  currently-active interface for Unit C consumers.
- Python codegen iterates CORE.data[KEY_NETWORK_PRIORITY] and emits
  add_priority_entry() calls per YAML order.

Field-tested on ESP32-S3 with W5500 SPI ethernet + WiFi STA on the same
subnet. The log line "[network] Default interface: <name>" confirms the
arbitration logic fires correctly on IP_EVENT_*_GOT_IP.

Standalone — no schema changes, single-interface configs unaffected.
2026-05-21 21:29:47 -05:00
kbx81 9d9af645ac Merge remote-tracking branch 'upstream/dev' into multi-interface-poc 2026-05-20 23:52:07 -05:00
Rapsssito 9bfae9e782 Remove redundant esp_netif_init 2026-05-20 09:07:57 +02:00
kbx81 eb64707d94 fix(network): lower NETWORK_PRIORITY_BASE below NetworkComponent's own priority
PR #14255 sets NETWORK_PRIORITY_BASE = 300.0, but PR #14012's
NetworkComponent uses setup_priority::AFTER_BLUETOOTH = 300.0f. When the
highest-priority interface (first in the priority list) tied with
NetworkComponent at 300, the runtime tie-break was determined by
registration order — and NetworkComponent registers AFTER ethernet (its
codegen runs at CoroPriority.NETWORK_SERVICES = 55, below COMMUNICATION
= 60 used by ethernet/wifi).

Result: ethernet's setup() ran before NetworkComponent::setup(),
esp_netif_init() had not yet been called, esp_netif_new() returned
NULL, and EthernetComponent::setup() dereferenced NULL in
esp_netif_attach() — LoadProhibited crash at boot.

Drop the base to 250.0 (matches the historical setup_priority::WIFI /
::ETHERNET default, so a single-entry priority list behaves identically
to a no-priority-block config) and shrink the step to 5.0 to keep all
interfaces in the same priority band, above BEFORE_CONNECTION (220.0)
and below AFTER_BLUETOOTH (300.0).
2026-05-19 23:16:01 -05:00
kbx81 7814e99b6f fix(network): enable USE_SETUP_PRIORITY_OVERRIDE when priority is configured
PR #14255 generates calls to Component::set_setup_priority(float) from
ethernet/wifi to_code(), but that method's body in core/component.cpp is
gated by #ifdef USE_SETUP_PRIORITY_OVERRIDE. Without the define the
declaration exists but no implementation is linked, producing:

  undefined reference to `esphome::Component::set_setup_priority(float)`

The existing convention in cpp_helpers.register_component() is to add
the define whenever CONF_SETUP_PRIORITY appears in a component's YAML.
Mirror that here: when the user declares `network: priority:`, the
priority-driven setup_priority overrides will be emitted, so the define
must be on.
2026-05-19 23:10:32 -05:00
kbx81 8ad6813d44 Merge PR #14255: network priority
Resolves conflicts with PR #14012 (centralized netif init):
- wifi_component_esp_idf.cpp: dropped pr-14255's ESP_ERR_INVALID_STATE
  tolerance hunk (made moot by #14012 removing the call entirely).
- ethernet/__init__.py: kept dev's refactored _to_code_esp32 structure;
  added pr-14255's priority lookup and conditional CONFIG_ESP_WIFI_ENABLED
  gating; preserved dev's top-level import shape.
- network/__init__.py: merged CONF_ID import (#14012) with CONF_PRIORITY
  + CONF_TIMEOUT imports (pr-14255).

Also fixed two latent bugs in pr-14255 where `"wifi" in net_priority`
compared a string against a list of dicts (always False). Replaced with
set comprehension over the normalized interface names.
2026-05-19 22:45:14 -05:00
kbx81 1aa0a489f6 Merge PR #14012: centralize ESP32 network init 2026-05-19 22:37:40 -05:00
Keith Burzinski be3ccd29f6 Merge branch 'dev' into central-netif 2026-05-19 18:03:15 -05:00
Rodrigo MartínandKeith Burzinski 73b8491936 Update esphome/components/network/network_component.h
Co-authored-by: Keith Burzinski <kbx81x@gmail.com>
2026-05-19 17:50:02 +02:00
Rapsssito 1332ebe729 Exclude from non ESP32 devices 2026-05-19 09:20:08 +02:00
kbx81 028a54422e preen 2026-05-18 18:31:31 -05:00
kbx81 de53e7a6b1 preen 2026-05-18 18:24:54 -05:00
kbx81 0d1d00b654 Merge branch 'dev' into central-netif 2026-05-18 18:19:46 -05:00
pre-commit-ci-lite[bot] 578196ab85 [pre-commit.ci lite] apply automatic fixes 2026-02-28 22:24:08 +00:00
Roy Walker d9b712ee5f Fix timeout to use ESPhome built-in function. 2026-02-28 16:22:28 -06:00
Roy Walker 1a61cd622e Add support for timeouts before next network connection is turned up and add support for openthread and modem. 2026-02-28 13:41:55 -06:00
pre-commit-ci-lite[bot] 26bdf58daf [pre-commit.ci lite] apply automatic fixes 2026-02-26 17:36:14 +00:00
Roy Walker c915a2b8f5 Remove duplicate logger and fix import. 2026-02-26 11:34:18 -06:00
Roy Walker 7fdb95c2ef Merge branch 'rwalker777-network-priority' of https://github.com/rwalker777/esphome into rwalker777-network-priority 2026-02-26 11:31:10 -06:00
Roy Walker e44365abca Remove duplicate CONF_OUTPUT_POWER from import. 2026-02-26 11:30:20 -06:00
rwalker777 532641d523 Merge branch 'esphome:dev' into rwalker777-network-priority 2026-02-26 11:30:07 -06:00
rwalker777 bc36892e7d Merge branch 'dev' into rwalker777-network-priority 2026-02-24 13:45:27 -06:00
Roy Walker 3a02c2f8af Fix validation on priority import. 2026-02-24 11:16:21 -06:00
pre-commit-ci-lite[bot] 4a1f9af319 [pre-commit.ci lite] apply automatic fixes 2026-02-24 17:08:56 +00:00
rwalker777 9e29bdfdad Merge branch 'esphome:dev' into rwalker777-network-priority 2026-02-24 11:03:34 -06:00
Roy Walker 20c975103b Fix Wifi not connecting with Ethernet config but disconnected. 2026-02-22 20:30:34 -06:00
Roy Walker 549b9f85ae Fix wifi so it doesn't double register. 2026-02-22 19:20:13 -06:00
Roy Walker 0fe2310db4 Fix wifi and ethernet coexisting. 2026-02-22 18:42:35 -06:00
Roy Walker 5af3e5caef Fix stab at network priority support. 2026-02-22 18:22:47 -06:00
Rapsssito 47854ff9de Add missing imports 2026-02-16 13:31:19 +01:00
Rapsssito 8a1ddfb1cc Typo 2026-02-16 13:21:13 +01:00
Rapsssito cde89212fc Typo 2026-02-16 13:17:00 +01:00
Rapsssito 0a518c1e4c Switch to a network component 2026-02-16 13:13:23 +01:00
Rapsssito 8c7d2d984e Just store if it is initialized 2026-02-16 12:47:14 +01:00
Rapsssito 8390a98614 [ethernet, network, openthread, wifi] centralize esp32 netif intialization 2026-02-16 12:36:22 +01:00
10325 changed files with 136614 additions and 617416 deletions

No files matched your search

-241
View File
@@ -1,241 +0,0 @@
---
name: automations
description: Write ESPHome automation building blocks - triggers (on_* keys), actions and conditions. Use when adding or changing an on_* trigger, an action such as my_component.do_something, a condition, a callback registration method (add_on_*_callback), or code that calls build_callback_automation, build_automation, register_*_action or register_*_condition.
---
# Automations: triggers, actions and conditions
Automations have three building blocks: **Triggers** (fire when something happens), **Actions** (do
something), and **Conditions** (check if something is true). The helpers named here are documented in
their docstrings in `esphome/automation.py`.
## Triggers: callback method (preferred)
Use `build_callback_automation()` for simple triggers. This eliminates the need for a C++ Trigger class by
using a lightweight pointer-sized forwarder struct registered directly as a callback. No `CONF_TRIGGER_ID`
in the schema.
**Python:**
```python
from esphome import automation
CONFIG_SCHEMA = cv.Schema(
{
cv.GenerateID(): cv.declare_id(MyComponent),
cv.Optional(CONF_ON_STATE): automation.validate_automation({}),
}
).extend(cv.COMPONENT_SCHEMA)
async def to_code(config):
var = cg.new_Pvariable(config[CONF_ID])
await cg.register_component(var, config)
for conf in config.get(CONF_ON_STATE, []):
await automation.build_callback_automation(
var, "add_on_state_callback", [(bool, "x")], conf
)
```
`build_callback_automation` arguments: `parent`, `callback_method` (C++ method name), `args` (template args
as `[(type, name)]` tuples), `config`, and optional `forwarder` (defaults to `TriggerForwarder<Ts...>`).
For boolean filtering (e.g. `on_press`/`on_release`), use built-in forwarders with `args=[]`:
```python
for conf_key, forwarder in (
(CONF_ON_PRESS, automation.TriggerOnTrueForwarder),
(CONF_ON_RELEASE, automation.TriggerOnFalseForwarder),
):
for conf in config.get(conf_key, []):
await automation.build_callback_automation(
var, "add_on_state_callback", [], conf, forwarder=forwarder
)
```
When the callback's parameters differ from the automation's arguments, or the trigger should only fire for
some values, pass `params`, `forward` and `when`; the helper then generates a capture-less lambda (stored
inline, nothing allocated). Name the parent with `automation.parent_ref(var)` and build `forward` from
`MockObj` calls, not f-strings of C++:
```python
parent = automation.parent_ref(var)
index = cg.RawExpression("index")
await automation.build_callback_automation(
var,
"add_on_state_callback",
[(cg.StringRef, "x"), (cg.size_t, "i")],
conf,
params=[(cg.size_t, "index")],
forward=[cg.StringRef(parent.option_at(index)), index],
)
```
`build_parent_callback_automation`, `build_trigger_callback`, `build_callback_automations` and
`build_trigger_automations` cover the other shapes; see their docstrings in `esphome/automation.py`.
When every action in a callback automation is inline (see Actions below), no `Automation` or action objects
are generated: the callback calls one generated function, at the same RAM cost as a hand-written lambda. A
custom forwarder blocks this unless its filter is also given as `params`, `forward` and `when`, so always pass
them together:
```python
automation.CallbackAutomation(
CONF_ON_IDLE,
"add_on_state_callback",
forwarder=StateEnterForwarder.template(State.IDLE),
params=[(State, "state")],
forward=[],
when=f"state == {State.IDLE}",
)
```
**C++: no trigger class needed.** The callback registration method must be templatized to accept both
`std::function` and lightweight forwarder structs (which avoid heap allocation):
```cpp
class MyComponent : public Component {
public:
// Must be a template -- accepts both std::function and pointer-sized forwarder structs
template<typename F> void add_on_state_callback(F &&callback) {
this->state_callback_.add(std::forward<F>(callback));
}
protected:
// Use CallbackManager when callbacks are always registered (e.g. core components)
CallbackManager<void(bool)> state_callback_;
// Use LazyCallbackManager when callbacks are often not registered -- saves 8 bytes
// (nullptr vs empty std::vector) per instance when no callbacks are added
// LazyCallbackManager<void(bool)> state_callback_;
};
```
Never type a registration method's parameter as `std::function`; that forces a heap allocation for every
forwarder struct:
```cpp
// Bad
void add_on_state_callback(std::function<void(bool)> &&callback) {
this->state_callback_.add(std::move(callback));
}
```
| Type | Idle overhead (32-bit) | When to use |
|------|----------------------|-------------|
| `CallbackManager<void(Ts...)>` | 12 bytes (empty `std::vector`) | Callbacks are always or almost always registered |
| `LazyCallbackManager<void(Ts...)>` | 4 bytes (`nullptr`) | Callbacks are often not registered (common case) |
| `AutomationList<N, Ts...>` + `build_automation_lists` | 4 bytes per automation + count, none when unused | A hub fires its own `on_*` automations with no filter (4 B per automation vs 8 B per `Callback`) |
| `automation.new_automation(args, conf)` + the parent's own fixed table | the table entry (pointer plus its own data) | Like `AutomationList`, but each automation needs a stored value of its own, e.g. a threshold the parent compares against |
Both live in `esphome/core/helpers.h` and accept `std::function`, lambdas and forwarder structs through
their templatized `add()` method. `LazyCallbackManager` is a drop-in replacement for `CallbackManager` that defers allocation until the first
callback is added. Prefer it for entity-level callbacks where most instances have no subscribers.
## Triggers: Trigger class method
Use `build_automation()` with a `Trigger<Ts...>` subclass only when the forwarder needs **mutable state
beyond a single `Automation*` pointer** (e.g. edge detection tracking previous state, timing logic).
**Python:**
```python
TurnOnTrigger = my_ns.class_("TurnOnTrigger", automation.Trigger.template())
CONFIG_SCHEMA = cv.Schema(
{
cv.Optional(CONF_ON_TURN_ON): automation.validate_automation(
{cv.GenerateID(CONF_TRIGGER_ID): cv.declare_id(TurnOnTrigger)}
),
}
)
async def to_code(config):
for conf in config.get(CONF_ON_TURN_ON, []):
trigger = cg.new_Pvariable(conf[CONF_TRIGGER_ID], var)
await automation.build_automation(trigger, [], conf)
```
**C++:**
```cpp
class TurnOnTrigger : public Trigger<> {
public:
explicit TurnOnTrigger(MyComponent *parent) : last_on_{false} {
parent->add_on_state_callback([this](bool state) {
if (state && !this->last_on_)
this->trigger();
this->last_on_ = state;
});
}
protected:
bool last_on_;
};
```
## Actions
```cpp
template<typename... Ts> class MyAction : public Action<Ts...> {
public:
explicit MyAction(MyComponent *parent) : parent_(parent) {}
void play(const Ts &...) override { this->parent_->do_something(); }
protected:
MyComponent *parent_;
};
```
Register it without writing a builder:
```python
automation.register_simple_action(
"my_component.do_something", MyAction, schema, synchronous=True
)
```
The constructor receives the object named by `config[CONF_ID]`. Use `register_bare_action` for a
no-argument constructor, `register_parented_action` for a class deriving from `Parented<T>`, and the
`@automation.register_action(...)` decorator only when the builder must also set fields.
Use `synchronous=True` for actions that run to completion inside `play()` without deferring. Use
`synchronous=False` if the action may suspend/defer execution (e.g. `delay`, `wait_until`, `script.wait`)
or store trigger arguments for later use.
**Actions that only forward templatable values to their parent need no C++ class.** Register them with
`register_apply_action`; do not write a `TEMPLATABLE_VALUE` class or a builder for this shape.
```python
automation.register_apply_action(
"my_component.set_gains",
schema,
automation.ApplyField(CONF_KP, "set_kp", cg.float_),
automation.ApplyField(CONF_KI, "set_ki", cg.float_),
)
```
The `ApplyField`, `ApplyCall` and `register_apply_action` docstrings in `esphome/automation.py` cover the
rest; `cover.control` and `cover.template.publish` are in-tree examples. An action whose input is a byte
payload passes `automation.ApplyBytes(CONF_DATA, "send({})")` in the same call; a constant payload stays in
flash and a lambda's vector is forwarded as data and size. `TEMPLATABLE_VALUE` with `cg.templatable` stays
for actions whose `play()` has real logic beyond forwarding values.
Adjacent `register_apply_action` and `lambda:` actions in a list merge into one generated function and one
object, which saves 12 to 16 bytes of RAM per merged action. An action with its own C++ class ends the merged
run, so prefer `register_apply_action` wherever the action allows it. Because of this, `build_action_list` can
return fewer objects than the config has entries. A builder that wraps `apply_action_builder` may do other work
first, but must return its result unchanged and not use it. A builder that must generate its own statements, and
is registered with `automation.ApplyAction`, can call `apply_statements_action` and still merge; `script.execute`
is an example. `if`, `choose`, `while` and `repeat` whose bodies are all inline become C++ control flow in the
same function, and `register_apply_condition` and `lambda:` conditions become plain expressions there.
## Conditions
```cpp
template<typename... Ts> class MyCondition : public Condition<Ts...> {
public:
explicit MyCondition(MyComponent *parent) : parent_(parent) {}
bool check(const Ts &...) override { return this->parent_->is_active(); }
protected:
MyComponent *parent_;
};
```
Register with `automation.register_simple_condition("my_component.is_active", MyCondition, schema)`;
`register_bare_condition`, `register_parented_condition` and the decorator follow the action rules.
**Conditions that only test their parent need no C++ class either.** Register them with
`register_apply_condition`; the expression is applied to the parent, and an `ApplyCall` compares against
config values.
```python
automation.register_apply_condition("my_component.is_active", schema, "is_active()")
automation.register_apply_condition(
"my_component.state_is",
schema,
automation.ApplyCall("state == {}", ((CONF_STATE, cg.bool_),)),
)
```
`cover.is_open`, `rtttl.is_playing` and `component.is_idle` are in-tree examples.
-68
View File
@@ -1,68 +0,0 @@
---
name: breaking-changes
description: Decide whether an ESPHome change breaks public API and handle it - what counts as public C++ and Python API, the deprecation window, ESPDEPRECATED for C++, cv.rename_key and deprecation warnings for config keys, and the migration notes a PR needs. Use when removing, renaming or changing the signature of a config option, a public C++ member or a core Python helper.
---
# Public API and breaking changes
Full policy: https://developers.esphome.io/contributing/code/#public-api-and-breaking-changes
## What is public
- **C++ in components:** only features documented at [esphome.io](https://esphome.io) are public API.
Undocumented `public` members are internal.
- **C++ core and base classes** (`esphome/core/`, `Component`, `Sensor`, etc.): all `public` members are
public API.
- **Components with global accessors** (`global_api_server`, etc.): all `public` members are public API
(except config setters).
- **Python:** all documented configuration options are public API. Python code in `esphome/core/` that
existing core components use is stable API. Other Python code is internal unless explicitly documented
for external component use.
- Changing the signature of a lambda that code generation injects is not a breaking change.
## Policy
- Aim for a **6-month deprecation window** when possible.
- Clean breaks are allowed for signature changes, deep refactorings and resource constraints.
- The migration path goes in the PR description (it generates the release notes).
- A blog post is required for core/base class changes or significant architectural changes.
Checklist:
- [ ] Clear justification (RAM/flash savings, architectural improvement)
- [ ] Explored non-breaking alternatives
- [ ] Added deprecation warnings if possible (`ESPDEPRECATED` for C++)
- [ ] Documented migration path in the PR description with before/after examples
- [ ] Updated all internal usage and esphome.io
- [ ] Tested backward compatibility during the deprecation period
## Deprecating C++
```cpp
// Remove before 2026.6.0
ESPDEPRECATED("Use new_method() instead. Removed in 2026.6.0", "2025.12.0")
void old_method() { this->new_method(); }
```
## Deprecating config keys
For a renamed key, use the shared `cv.rename_key` validator with `removed_in` (and `component` for
context); it warns and migrates automatically:
```python
CONFIG_SCHEMA = cv.All(
cv.rename_key(
CONF_OLD_KEY, CONF_NEW_KEY, removed_in="2026.6.0", component="my_component"
),
cv.Schema({...}),
)
```
For other deprecations, warn manually during validation:
```python
# Remove before 2026.6.0
if CONF_OLD_KEY in config:
_LOGGER.warning(
f"'{CONF_OLD_KEY}' deprecated, use '{CONF_NEW_KEY}'. Removed in 2026.6.0"
)
config[CONF_NEW_KEY] = config.pop(CONF_OLD_KEY) # Auto-migrate
```
Cover the old syntax with a `validate.*.yaml` test (see the `component-tests` skill), and keep the language
schema correct (see the `config-validation` skill).
-181
View File
@@ -1,181 +0,0 @@
---
name: code-review
description: Review guidance for ESPHome pull requests. Use this when reviewing a pull request that changes ESPHome Python, C++, or component code, to check it against the project's coding conventions, embedded-systems memory rules, testing requirements, and breaking-change policy.
---
# Reviewing ESPHome pull requests
ESPHome parses YAML into C++ firmware for memory-constrained microcontrollers
(ESP32, ESP8266, RP2040, LibreTiny). Review changes with that in mind: RAM and
flash are scarce, and code runs unattended for months.
`AGENTS.md` in the repository root is the contributor guide, and the skills it
lists (`component-codegen`, `config-validation`, `automations`,
`embedded-memory`, `component-tests`, `breaking-changes`) hold the detail. They
are the authority when they disagree with this summary. The developer
documentation at https://developers.esphome.io explains the component lifecycle
and the reasoning behind these rules. This skill lists the concrete things worth
flagging in a review; read the matching skill for the detail behind any item.
Only raise findings that the diff actually introduces or changes. Do not ask for
drive-by cleanup of pre-existing code the PR did not touch.
## Scope: shared code does not ride a component PR
Check the file list first. A PR titled for one component (`[ld2450] ...`) must
not also change code other components build on: `esphome/core/`, the entity
platform components (`esphome/components/sensor/`, `esphome/components/text_sensor/`
including its `filter.*`, `binary_sensor/`, `number/`, ...), the buses
(`esphome/components/{i2c,spi,uart}/`), `network/`, `socket/`, the shared
constants (`esphome/components/const/`, `esphome/const.py`),
`esphome/automation.py`, `esphome/cpp_generator.py` or `esphome/config_validation.py`.
Flag it and ask for the shared change to move into its own PR, which lands first
and is reviewed by that code's owners; the component PR then builds on it. This
holds even when the shared change is small or only the component uses it today.
The exception is a shared change that must update several components in the same
PR so none of them breaks in between (a `[core]` or `[components]` PR that adds a
helper and moves its callers at once).
Not shared code: the component's own platform files (`esphome/components/<c>/sensor.py`,
`text_sensor/__init__.py`, ...), its new `USE_*` lines in `esphome/core/defines.h`,
and its `CODEOWNERS` entry.
## Memory and embedded constraints (highest value)
Heap allocation after `setup()` is treated as a reliability bug, not a
performance nit, because it fragments a small shared heap. Flag:
- New heap allocation on a hot path or after setup that could be avoided.
- `std::vector` where the size is known at compile time (use `std::array`, or
`StaticVector<T, N>` when a `push_back` API is needed) or fixed at runtime
init (use `FixedVector<T>`).
- Listener / child-entity registration lists stored as `std::vector`; these have
a compile-time-known count and should use `cg.slot_list()` plus
`SLOT_LIST` (or `cg.slot_counter()` plus `StaticVector`).
- `std::vector<uint8_t>` for a byte buffer that never grows: prefer
`std::unique_ptr<uint8_t[]>` or `std::array`.
- `std::map` / `std::set` / `std::unordered_map` for small datasets (1-16
elements): a `std::vector` of a small struct with linear search is lighter.
- `std::deque` anywhere: it allocates 512-byte blocks and should be avoided.
- `std::string` storing a value set once from config: prefer `StringRef` (the
literal already lives in flash).
- `std::string` / `std::to_string` / string-returning helpers on hot paths where
a buffer or view API exists.
- A `set_timeout` / `set_interval` / `defer` lambda whose captures exceed 8 bytes
on 32-bit or are not trivially copyable (`this` plus two 4-byte values, `this`
plus a member function pointer, a `std::string`, a `std::function`):
`std::function` then heap-allocates each time it is scheduled. Flag it on
retries, re-armed timeouts and hot-path `defer`; a `set_interval` set up once
in `setup()` is a single allocation. `[this]` plus one 4-byte value fits.
## C++ conventions
- Include what you use: a file referencing a symbol must include the header that
declares it, even if it currently arrives transitively. New or changed uses of
a symbol need the matching include.
- Prefix all member access with `this->`.
- Naming: `lower_snake_case` for functions/methods/variables, `UpperCamelCase`
for classes/structs/enums, `UPPER_SNAKE_CASE` for namespace-scope constants,
trailing underscore on protected/private fields.
- `enum class` values must be prefixed with the enum name in `UPPER_SNAKE_CASE`
(e.g. `UARTFlushResult::UART_FLUSH_RESULT_SUCCESS`). Bare names like `SUCCESS`,
`FAIL`, or `OK` collide with SDK macros on some platforms and break the build.
- Prefer `const`/`enum` over `#define`; `#define` is only for conditional
compilation and code-generation sizes.
- Never call `millis()` in a `loop()` body; use
`App.get_loop_component_start_time()`. A rate-limit gate below ~16 ms (the loop
period) does nothing.
- Pick the timing primitive by cadence: gated `loop()` under 250 ms,
`set_interval` at 500 ms and above.
- Do not override a base method to return the value it already returns (e.g.
`get_setup_priority()` returning `setup_priority::DATA`).
- Wrap string literals passed as printf `%s` args in `LOG_STR_LITERAL()`.
- Required, invariant dependencies should be constructor parameters, not setters.
- Callback registration methods must be templated (`template<typename F>`), not
typed as `std::function`, so lightweight forwarders avoid a heap allocation.
- Two-space indent, `using` over `typedef`, wrap at 120 columns.
## Use the existing helper
New components often hand-write code a helper already covers. Ask for the
helper; `AGENTS.md` section 4 and the `component-codegen` skill have the detail.
| Seen in the PR | Ask for |
| --- | --- |
| `if (this->x_sensor_ != nullptr) this->x_sensor_->publish_state(v);` | `PUBLISH_STATE(this->x_sensor_, v);` (`esphome/core/entity_base.h`) |
| set warning / `return` / clear warning around a bus result | `if (!this->status_comm_result(ok)) return;` |
| hand-written `set_x_sensor()` plus `x_sensor_` member | `SUB_SENSOR(x)`, `SUB_BINARY_SENSOR`, `SUB_TEXT_SENSOR`, `SUB_SWITCH`, ... |
| `if CONF_X in config: s = await sensor.new_sensor(config[CONF_X])` plus setter | `sensors = sensor.sub_sensors(config)` then `await sensors(CONF_X, hub.set_x_sensor)` |
| hand-written `{"a": MyEnum.MY_ENUM_A, ...}` dict | `MyEnum.options("a", "b", prefix="MY_ENUM_")` |
| loops on `available()` / `read_byte()` | UART `read_available(on_byte)`, `discard_available()`, `LineReader<N>`, `FrameReader<N, HEADER...>` |
| hand-rolled CRC, checksum or bit loop | `crc8`, `crc16`, `xor_bytes`, `sum16_bytes`, `encode_uint16`, ... (`esphome/core/helpers.h`) |
| integer split into buffer bytes by hand (`buf[0] = v >> 8; buf[1] = v;`) | `put_be16`/`24`/`32`, `put_le16`, `put_le32` (`esphome/core/helpers.h`) |
| bare `mark_failed()`, or `ESP_LOGE` then `mark_failed()` | `this->mark_failed(LOG_STR("reason"))` or `LOG_STR(ESP_LOG_MSG_COMM_FAIL)` |
| button/switch/number/select C++ class that only calls the hub | `ApplyButton`, `ApplySwitch`, `ApplyNumber`, `ApplySelect` with `apply=` |
| action or condition class that only forwards to the parent | `register_apply_action` / `register_apply_condition` |
| enum-to-string `switch` returning literals | `PROGMEM_STRING_TABLE` with `get_log_str` / `get_progmem_str` |
| `if (x := config.get(CONF_X)) is not None: cg.add(var.set_x(x))` | `cg.add_optional(var.set_x, config.get(CONF_X))`, when the setter takes the value unchanged |
## Python conventions
- Type-annotate every new function signature (params and return), new dataclass
fields, and new module-level variables. Import `ConfigType` from
`esphome.types`.
- Use the walrus operator to avoid a double lookup, e.g.
`if (blah := config.get(CONF_BLAH)) is not None:`.
- Reuse existing validators from `config_validation.py` (`cv.rename_key`,
`cv.has_exactly_one_key`, etc.) via `cv.All(...)` instead of hand-rolling.
- A new or changed validator function that wraps, chooses or builds a schema
must be decorated with `@schema_extractor(...)` and handle `SCHEMA_EXTRACT`,
or `script/build_language_schema.py` silently drops the keys behind it (see
the `config-validation` skill).
- `esphome/const.py` is frozen: no new `CONF_` constants there. Define them in
the component's own `.py`, or in `esphome/components/const/__init__.py` when
shared. The same constant defined in three or more component files fails CI.
- State that must persist during code generation goes in `CORE.data` namespaced
under the component `DOMAIN` (a `@dataclass`), not module-level mutable globals.
- Prefer callback-based triggers via `build_callback_automation()`; only use a
`Trigger<Ts...>` subclass when the forwarder needs mutable state.
## Testing and coverage
- New and changed lines and branches need test coverage, including defensive
early-returns, error paths, and no-op guards. A mocked-out function is not
covered; exercise the real call path too.
- Component YAML tests live in `tests/components/<component>/`. Never define
buses (uart, i2c, spi, modbus) directly in a test file: pull them from
`tests/test_build_components/common/` through dict-style `packages:` so CI can
group builds. List-style packages or top-level merge keys block grouping.
- Config-only checks use the `validate.*.yaml` prefix; compiled checks use
`test.*.yaml`.
## Naming user-facing functions
Documented lambda methods, actions and config keys are public API, so a bad name
is hard to fix once released. Flag:
- A boolean where `true` means off (`display_off(true)`): name the positive
state, e.g. `set_power(bool on)`.
- A name that differs from a sibling component's name for the same operation.
Before a release, rename outright; after, follow the breaking-change policy
below.
## Breaking changes and public API
- Base classes under `esphome/core/` and documented config options are public
API. Undocumented `public` members of a component are internal.
- A breaking change needs justification, a migration path in the PR description,
and a deprecation window where feasible (`ESPDEPRECATED` in C++,
`cv.rename_key(..., removed_in=...)` in Python). Changing a codegen-injected
lambda signature is not a breaking change.
## Process and PR hygiene
- PR titles start with a `[tag]` prefix: the component name (e.g. `[uart] ...`)
or `[core]` for shared code.
- Prose in docs, comments, and commit messages should be plain English. Keep
inline comments short and only where the code is not self-explanatory; do not
restate what the code says.
- Verify the PR fills out `.github/PULL_REQUEST_TEMPLATE.md` and adds
`CODEOWNERS` entries for a new component.
-276
View File
@@ -1,276 +0,0 @@
---
name: component-codegen
description: Write the Python side of an ESPHome component - __init__.py and platform files (sensor.py, switch.py, ...), DOMAIN and other metadata, to_code, hub child entities with sub_sensors and friends, pins and referenced components, hub-forwarding entities (ApplyButton, ApplySwitch), uart/i2c/spi constructor helpers, and CORE.data state. Use when creating a component or changing its code generation.
---
# Component code generation
Read https://developers.esphome.io/architecture/components/ for the component lifecycle and how a
component is registered. Schema rules and schema extraction are in the `config-validation` skill;
automations (`on_*`, actions, conditions) in the `automations` skill.
## Files
```
components/[component_name]/
├── __init__.py # Component configuration schema and code generation
├── [component].h # C++ header file (if needed)
├── [component].cpp # C++ implementation (if needed)
└── [platform]/ # Platform-specific implementations
├── __init__.py # Platform-specific configuration
├── [platform].h # Platform C++ header
└── [platform].cpp # Platform C++ implementation
```
## Metadata
- `DOMAIN`: required in every component's `__init__.py`, set to the component's own name as a plain string
(e.g. `DOMAIN = "my_component"`). Other code refers to the component through it, e.g.
`from esphome.components.my_component import DOMAIN`. Keep it in alphabetical order with the other
metadata constants. CI (`lint_component_domain`) fails without it.
- `DEPENDENCIES`: list of required components.
- `AUTO_LOAD`: components to automatically load. Not a way to put entities under the component's own key;
see "Entities of a hub" below.
- `CONFLICTS_WITH`: incompatible components.
- `CODEOWNERS`: GitHub usernames responsible for maintenance.
- `MULTI_CONF`: whether multiple instances are allowed.
## Basic pattern
```python
import esphome.codegen as cg
import esphome.config_validation as cv
from esphome.const import CONF_KEY, CONF_ID
DOMAIN = "my_component"
CONF_PARAM = "param" # A constant that does not yet exist in esphome/const.py
my_component_ns = cg.esphome_ns.namespace("my_component")
MyComponent = my_component_ns.class_("MyComponent", cg.Component)
CONFIG_SCHEMA = cv.Schema(
{
cv.GenerateID(): cv.declare_id(MyComponent),
cv.Required(CONF_KEY): cv.string,
cv.Optional(CONF_PARAM, default=42): cv.int_,
}
).extend(cv.COMPONENT_SCHEMA)
async def to_code(config):
var = cg.new_Pvariable(config[CONF_ID])
await cg.register_component(var, config)
cg.add(var.set_key(config[CONF_KEY]))
cg.add(var.set_param(config[CONF_PARAM]))
```
```cpp
namespace esphome::my_component {
class MyComponent : public Component {
public:
void setup() override;
void loop() override;
void dump_config() override;
void set_key(const char *key) { this->key_ = StringRef(key); }
void set_param(int param) { this->param_ = param; }
protected:
StringRef key_;
int param_{0};
};
} // namespace esphome::my_component
```
When the setter takes the value unchanged, use `cg.add_optional(var.set_blah, config.get(CONF_BLAH))`; it
is not awaited, and it skips only `None`, so `0` and `False` are still set. Required, invariant
dependencies are constructor arguments to `cg.new_Pvariable()`, not setters.
## Entity platforms
**Sensor:**
```python
from esphome.components import sensor
CONFIG_SCHEMA = sensor.sensor_schema(MySensor).extend(
cv.polling_component_schema("60s")
)
async def to_code(config):
var = await sensor.new_sensor(config)
await cg.register_component(var, config)
```
**Binary sensor:**
```python
from esphome.components import binary_sensor
CONFIG_SCHEMA = binary_sensor.binary_sensor_schema().extend({...})
async def to_code(config):
var = await binary_sensor.new_binary_sensor(config)
```
**Switch:**
```python
from esphome.components import switch
CONFIG_SCHEMA = switch.switch_schema().extend({...})
async def to_code(config):
var = await switch.new_switch(config)
```
## Entities of a hub
A new component's entities go in platform files (`sensor.py`, `binary_sensor.py`, ...) that take the hub's
id, never under the hub's own key with their platforms added to `AUTO_LOAD` to make that work. Auto loading
a platform because the C++ uses its type is fine. See
https://developers.esphome.io/architecture/components/#exposing-multiple-entities.
```yaml
# Bad - entities under the hub's key
my_hub:
connected:
name: Connected
# Good - the hub, then one platform entry that holds its entities
my_hub:
id: hub1
binary_sensor:
- platform: my_hub
my_hub_id: hub1 # optional with a single hub
connected:
name: Connected
```
**Optional child entities of a hub:** bind the config once with `sensor.sub_sensors(config)` (or
`sub_binary_sensors`, `sub_text_sensors`, `sub_buttons`, `sub_switches`, `sub_numbers`, `sub_selects` in
their domains), adding `parent=hub` for entities that derive from `Parented<T>`, then make one call per
key, even when there is only one. A call creates the entity only when its key is configured, passes it to
the setter and returns it (or `None`); extra arguments such as `min_value` or `options` go on the call.
Always name the setter explicitly on the object that owns it, never with `getattr` and an f-string, and
keep that variable short (`var` for the component itself, `hub` for one fetched with `cg.get_variable`) so
the calls fit on one line. Loops whose setter also takes an index, such as `set_gate_threshold(x, n)`, stay
as they are.
```python
# my_hub/sensor.py
async def to_code(config):
hub = await cg.get_variable(config[CONF_MY_HUB_ID])
sensors = sensor.sub_sensors(config)
await sensors(CONF_TEMPERATURE, hub.set_temperature_sensor)
await sensors(CONF_HUMIDITY, hub.set_humidity_sensor)
# my_hub/button.py
async def to_code(config):
hub = await cg.get_variable(config[CONF_MY_HUB_ID])
buttons = button.sub_buttons(config, parent=hub)
await buttons(CONF_RESTART, hub.set_restart_button)
```
**Hub forwarding entities need no C++ class:** a button, switch, number or select that only calls its hub
uses `button.ApplyButton` (`switch.ApplySwitch`, `number.ApplyNumber`, `select.ApplySelect`) in its schema
and `apply=` on the `sub_*` call. The body is a call on the hub with `entity` and `state`, `value` or
`index`/`option` in scope; on a switch, number or select `publish="before"` or `"after"` publishes the
state, and `publish="on_success"` publishes only when the hub call returns true (a write the device can
refuse). `publish="momentary"` is only for legacy action switches (restart, factory reset): it publishes off
and runs the body only when turned on. Never use it for a new entity; an action is a button.
```python
switches = switch.sub_switches(config, parent=hub)
await switches(CONF_BLUETOOTH, None, apply="set_bluetooth(state)", publish="after")
```
Restore an owned or standalone switch in `setup()` with `sw->restore()`, or `restore(default)` to still
drive a state when the restore mode is disabled. A hub that applies its switches' restored states itself
reads `sw->get_initial_write_state()`, which already applies `inverted`.
**A hub that forwards data to its children:** when what each child does with the data is known at build time
(decode a register field, match a key or tag, publish a value), declare a `cg.HubFunction` once in the hub's
`__init__.py` and have each platform add one item per entity, built from codegen expressions. Code generation writes
one function per hub. Pass `define=` and put the hub's `HUB_PUBLISHER(<name>, args...)` (in a public section; it
ends on `public:`) and its call under that `#ifdef`; call the pointer only when it is not null. The children stay
plain entities, with no listener class or list. A child that lost a `Component` base keeps its dump lines through
a second hub function (`"dump"`, a `const char *` tag parameter, no define of its own) that the hub's
`dump_config()` calls inside the value function's `#ifdef`; every child adds to both.
**Entities that each watch a register field:** when every field (register, shift, mask, match) is known at
build time, use the rule above (see `bq25186`). A binary sensor tests its field value against an on-values
bitmap (`match`); a one-bit field reads the bit (simplified from bq25186's `_state_expression`):
```python
# hub __init__.py
STATUS_PUBLISHER = cg.HubFunction(
"status", # esphome__<hub>__status_fn, passed to HUB_PUBLISHER(status, ...)'s setter
cg.void,
[(BQ25186StatusRegisters.const_ref(), "status")],
define="USE_BQ25186_STATUS_PUBLISHER",
)
STATUS = STATUS_PUBLISHER.param("status")
# platform to_code, per configured entity
value = (STATUS[reg] >> shift) & mask
if mask != 0x01 or match != 0b10: # 0b10: a one-bit field that is on when set
value = (cg.MockObj(f"{match:#06b}") >> value) & 1
STATUS_PUBLISHER.add(hub, parent_ref(var).publish_state(value))
```
Store a field description per entity (a struct in a `SlotList` sized by `cg.slot_list`) only when the
field is chosen at run time, as in `sen6x`, where the status bit depends on the variant read in `setup()`.
Never a template per bit or a virtual listener base (a second vptr per entity).
## Other helpers
- **Pins and referenced components:** set them with `await cg.add_pin(var.set_cs_pin, config[CONF_CS_PIN])`
and `await cg.add_variable(var.set_bus, config[CONF_BUS_ID])`; both skip a `None` value, so pass
`config.get(KEY)` for an optional key, and return what they created. Use `cg.register_parented` for
`set_parent`.
- `uart.new_uart_component(config, *ctor_args)` (and `i2c.new_i2c_component`, `spi.new_spi_component`,
`modbus.new_modbus_client_component`, `ble_client.new_ble_node_component`) constructs, registers and
attaches in one call.
- A fixed buffer sized from the config is a file-scope static (BSS, not heap):
`buf = cg.static_buffer(config[CONF_ID], cg.uint8, size)`, named after its owner so `esphome analyze-memory`
counts it there; pass `buf` to the constructor.
A buffer that may live in PSRAM takes `psram=True`: PSRAM builds get a zeroed `psram_buffer<T>(size)` allocation
instead, so pass `buf` once and mark the component failed in `setup()` if it is nullptr. `align=4` emits
`alignas(4)`, so a word-aligned byte buffer stays `uint8`. `size` may be an expression such as `MyClass.static_("buffer_size")(n)`, which renders
`ns::MyClass::buffer_size(n)`.
- `display.add_writer(var, config)` sets the lambda.
- `MyEnum.options("on", prefix="MY_ENUM_")` builds an enum dict.
- `button.restart_button_schema(cls)` (and `factory_reset_`, `query_`) are button presets.
- `fv.get_declaring_config(id)` finds a hub's config in final validation.
## State during code generation
Use `CORE.data` for component state that needs to persist during configuration generation, namespaced
under the component's `DOMAIN` and held in a `@dataclass`. Avoid module-level mutable globals: they
persist between compilation runs if the host process (e.g. device-builder) doesn't fork/exec, whereas
`CORE.data` is cleared between runs.
```python
from dataclasses import dataclass, field
from esphome.core import CORE
DOMAIN = "my_component"
@dataclass
class MyComponentData:
feature_enabled: bool = False
item_count: int = 0
items: list[str] = field(default_factory=list)
def _get_data() -> MyComponentData:
if DOMAIN not in CORE.data:
CORE.data[DOMAIN] = MyComponentData()
return CORE.data[DOMAIN]
def request_feature() -> None:
_get_data().feature_enabled = True
```
Don't use flat keys such as `CORE.data["my_component_feature"] = True`. Some components use `TypedDict`
instead of a dataclass; both are acceptable.
-101
View File
@@ -1,101 +0,0 @@
---
name: component-tests
description: Write and run tests for ESPHome components - YAML build tests (test.*.yaml, validate.*.yaml, common.yaml) under tests/components, shared bus packages and CI test grouping, C++ gtest unit tests, and host integration tests including the uart_mock fixture. Use when adding or changing files under tests/, or when a code change needs test coverage.
---
# Component tests
## C++ unit tests
gtests live in `tests/components/<component>/*.cpp` and run with `script/cpp_unit_test.py <component>`; see
`tests/components/README.md`. Set the loop clock in a test with `LoopBlockingGuard{nullptr, nullptr,
now_ms}`; the runner resets it before every test.
## Integration tests
`tests/integration/` runs real firmware on the host. Serial components prove timeouts and recovery with the
`uart_mock` fixture (`tests/integration/test_uart_mock_*.py`). Its `responses` match on content every time
they see it; to answer a repeated request differently, use an `on_tx` lambda with a static counter and
`inject_to_rx_buffer` (see `uart_mock_ld2412_requery_after_giveup.yaml`).
## YAML build tests
```
tests/
├── test_build_components/
│ └── common/ # Shared bus packages (uart, i2c, spi, etc.)
│ ├── uart/ # UART at default baud rate
│ ├── uart_115200/ # UART at 115200 baud
│ ├── i2c/ # I2C bus
│ └── spi/ # SPI bus
└── components/[component]/
├── common.yaml # Component-only config (no bus definitions)
├── test.esp32-idf.yaml # config + compile
├── test.esp8266-ard.yaml # config + compile
├── test-variant.esp32-idf.yaml # variant test, config + compile
├── validate.esp32-idf.yaml # config-only (never compiled)
└── validate-legacy.esp32-idf.yaml # config-only variant
```
Run them with `script/test_build_components`. Use `-c <component>` to test specific components and
`-t <target>` for specific platforms.
- **Keep build-test variants few:** one `test.<platform>.yaml` per platform the code actually differs on.
A variant per board for platform-identical code only adds CI time.
- **Config-only test files (`validate.*.yaml`):** use this prefix when a YAML file only needs to exercise
schema/validation paths and does not need to be compiled. CI runs `validate.*.yaml` files with
`esphome config` only and skips them during compile. The grammar mirrors `test.*.yaml`:
- `validate.<platform>.yaml`: base config-only test
- `validate-<variant>.<platform>.yaml`: config-only variant
Use this for deprecated-syntax migration tests, schema edge cases, or platform-specific validation
branches where building firmware adds no signal. A component may have any mix of `test.*.yaml` and
`validate.*.yaml` files. Validate files never participate in bus grouping; each one runs as its own
`esphome config` invocation.
When a PR's only edits to a component are `validate.*.yaml` files (no source changes, no `test.*.yaml`
changes, and the component isn't pulled in as a dependency of another changed component), CI skips the
compile stage for that component and only runs config validation. This is decided in
`script/determine-jobs.py` via `_component_change_is_validate_only` and surfaced as the
`validate_only_components` output that the `test-build-components-split` job consumes.
## Grouping with packages
Components that use shared bus packages can be grouped together in CI to reduce build count. **Never define
buses (uart, i2c, spi, modbus) directly in test YAML files**; always use packages from
`test_build_components/common/`.
All includes in test files must go through dict-style `packages:` so that batch grouping works; the
grouping scripts only understand dict-style packages. Never use list-style packages
(`packages: [- !include ...]`) or top-level merge keys (`<<: !include common.yaml`). Bus packages are keyed
by the bus name; the component's `common.yaml` is keyed by the component name (e.g.
`cst328: !include common.yaml`):
```yaml
# test.esp32-idf.yaml - everything included via named packages
packages:
uart: !include ../../test_build_components/common/uart_115200/esp32-idf.yaml
my_component: !include common.yaml
```
```yaml
# common.yaml - component config only, NO bus definitions
my_component:
id: my_instance
sensor:
- platform: my_component
name: My Sensor
```
Components that define buses directly are flagged as "NEEDS MIGRATION" and cannot be grouped, increasing
CI build time. `!extend` and `!remove` in a `test.*.yaml` also make the component ungroupable; add a
standalone entity with its own id instead.
A test that needs a shared component in a particular state (`esp32_ble` security settings, an option that
PSRAM rejects) cannot be grouped: the merged block wins and the whole batch fails. Add the component to
`ISOLATED_COMPONENTS` in `script/analyze_component_buses.py`, or move the option to a `test-<variant>` file.
To verify that all components can be tested together without ID conflicts or configuration issues:
```bash
./script/test_component_grouping.py -e config --all
```
This tests all components in a single build to catch conflicts that might not appear when testing
components individually. Use `-e config` for fast configuration validation, or `-e compile` for full
compilation testing.
-124
View File
@@ -1,124 +0,0 @@
---
name: config-validation
description: Write or change ESPHome configuration validation - CONFIG_SCHEMA, cv.Schema, cv.typed_schema, custom validator functions, config keys and CONF_ constants, renamed or deprecated keys, and the language schema that script/build_language_schema.py extracts from them. Use whenever a change touches a component's schema or a validator in config_validation.py, automation.py or a component's .py files.
---
# Configuration validation
ESPHome validates YAML with `voluptuous` schemas built from `esphome/config_validation.py` (`cv`). The
same schemas are also walked by `script/build_language_schema.py`, which produces the JSON language schema
that editors and the dashboard use for completion and checking. A validation change is only finished when
both still work.
## Writing schemas
- **Type hints:** validators are typed like any other function: `def validate_x(config: ConfigType) ->
ConfigType:`, with `ConfigType` imported from `esphome.types`.
- **One schema per model:** when a component's options depend on a `model` key, use
`cv.typed_schema({...}, key=CONF_MODEL, default_type=...)` with each model's own keys and defaults, not
one schema plus a validator that rejects keys and fills defaults afterwards.
- **Reuse existing validators:** before writing a custom validator, check for an existing one in
`config_validation.py` and compose it in `cv.All(...)` rather than duplicating logic across components.
For example, rename a config key with `cv.rename_key(CONF_OLD, CONF_NEW, removed_in="2026.6.0")`, and
reject mutually-exclusive keys with `cv.has_at_most_one_key(...)` / `cv.has_exactly_one_key(...)`. See
how `api` composes `cv.has_exactly_one_key` + `cv.rename_key`.
- **Common validators:** `cv.int_`, `cv.float_`, `cv.string`, `cv.boolean`, `cv.int_range(min=0,
max=100)`, `cv.positive_int`, `cv.percentage`.
- **Complex validation:** `cv.All(cv.string, cv.Length(min=1, max=50))`, `cv.Any(cv.int_, cv.string)`.
- **Platform-specific:** `cv.only_on(["esp32", "esp8266"])`, `esp32.only_on_variant(...)`,
`cv.only_on_esp32`, `cv.only_on_esp8266`, `cv.only_on_rp2040`.
- **Framework-specific:** `cv.only_with_framework(...)`, `cv.only_with_arduino`. On the ESP32, Arduino is
built as a component of ESP-IDF, so an ESP-IDF API, sdkconfig option or version floor is there on both
frameworks: gate on `esp32.require_idf_version(cv.Version(5, 5, 3), "psram_dma")`, which checks the
platform and the ESP-IDF under either framework, as `spi`, `usb_host` and `wifi` do. Reserve
`cv.only_with_framework("esp-idf")` for a build that has been tried on Arduino and fails there, and write
the actual failure next to it (`sendspin`: arduino-esp32 below ESP-IDF 6.0 bundles a second libsodium,
and from 6.0 sendspin-cpp's enumerators collide with `Arduino.h` macros such as `DISPLAY`).
- **Schema extensions:**
```python
CONFIG_SCHEMA = cv.Schema({ ... })
.extend(cv.COMPONENT_SCHEMA)
.extend(uart.UART_DEVICE_SCHEMA)
.extend(i2c.i2c_device_schema(0x48))
.extend(spi.spi_device_schema(cs_pin_required=True))
```
- **Constants:** `esphome/const.py` is frozen; do not add new `CONF_` constants there. Define a
component-local constant in the component's own `.py`; for a constant shared by multiple components, add
it to `esphome/components/const/__init__.py`. CI (`lint_constants_usage`) fails if the same constant is
defined in three or more component files. Constants used in core files (those not under
`esphome/components`) may be added to `esphome/const.py` but will require adjustment to the CI check.
- **Renamed or removed keys:** see the `breaking-changes` skill for `cv.rename_key` and manual
deprecation warnings.
- **Config-only tests:** schema edge cases and deprecated-syntax migrations go in `validate.*.yaml` test
files (see the `component-tests` skill).
## Schema extraction
`script/build_language_schema.py` sets `esphome.schema_extractors.EnableSchemaExtraction = True` before it
imports any component, then walks every `CONFIG_SCHEMA` (and platform schema) recursively. It understands
`cv.Schema`, `cv.All`, `cv.Any`, dicts of `cv.Required` / `cv.Optional` keys, `.extend()`, `cv.ensure_list`,
`cv.typed_schema`, registries and the known `cv` validators. **A plain function it cannot see into**: a
custom validator that builds or picks a schema at validation time hides every key behind it, and the
dumper silently emits nothing for them. CI only runs the script with `--check`, which does not compare the
output, so a lost schema is not caught.
When a validator function wraps, chooses or builds a schema (rather than checking a single value), make it
visible to the dumper:
1. Decorate it with `@schema_extractor("<type>")` from `esphome.schema_extractors`.
2. At the top of the function, return the data for that type when it is called with `SCHEMA_EXTRACT`
instead of a config value. Compare with `is` (`value is SCHEMA_EXTRACT`); older code uses `==`.
```python
from esphome.schema_extractors import SCHEMA_EXTRACT, schema_extractor
@schema_extractor("schema")
def _outgoing_connection_schema(config: ConfigType | None) -> ConfigType:
if config is SCHEMA_EXTRACT:
return _OUTGOING_CONNECTION_SCHEMA
if config is None:
config = {}
return _OUTGOING_CONNECTION_SCHEMA(config)
```
The type names the dumper accepts, and what `SCHEMA_EXTRACT` must return for each, are in the
`hidden_schemas` branch of `convert()` in `script/build_language_schema.py`. The common ones:
| Type | Return for `SCHEMA_EXTRACT` | Example |
|------|-----------------------------|---------|
| `"schema"` | the schema (or a representative one) to walk | `api` `_outgoing_connection_schema`, `restore_state_schema` in `esphome/core/entity_helpers.py`, `model_schema_extractor` in `esphome/components/mipi/__init__.py` |
| `"enum"` | the dict of options | `sensor_entity_category` in `esphome/components/sensor/__init__.py` |
| `"one_of"` | the list of options | `cv.one_of` |
| `"maybe"` | `(validator, key)` for a scalar shorthand | `cv.maybe_simple_value`, `maybe_conf` in `esphome/automation.py` |
| `"templatable"` | the inner validator | `cv.templatable` |
| `"automation"` | the automation schema | `automation.validate_automation` |
Points to watch:
- The dumper matches the function object by `repr`, so the decorated function must be the exact object
placed in the schema. In a factory that returns a new closure per call, decorate the inner function (as
`restore_state_schema` and `cv.maybe_simple_value` do), not the factory.
- The decorator only records anything when extraction is enabled, and that flag is set before imports, so
the decorator must run at import time (module level or inside a factory called at module level).
- A schema chosen per config value (for example by `model`) has no single answer: return a representative
schema, as `model_schema_extractor` does.
- Plain value checks (`cv.All(cv.string, my_check)`) need nothing: the dumper still sees the surrounding
types and keys.
- `cv.Schema`, `cv.typed_schema`, `cv.ensure_list` and the `cv` helpers are already handled; prefer
composing them over a custom function, which also avoids the extractor work.
### Checking the output
Build the language schema into a scratch directory and look at the component's JSON (it takes about ten
seconds):
```bash
python3 script/build_language_schema.py --output-path /tmp/schema
python3 -c "import json; print(json.dumps(json.load(open('/tmp/schema/my_component.json')), indent=1))" | less
```
Every key a user can write should appear under `config_vars` with a sensible `type`; compare against the
same output from `dev` when changing an existing schema. If the script prints `Unknown extracted schema
type` or raises, the decorator type or the value returned for `SCHEMA_EXTRACT` is wrong. Unit tests for
the dumper live in `tests/script/test_build_language_schema.py`.
-156
View File
@@ -1,156 +0,0 @@
---
name: display-test
description: Test an ESPHome display driver on a real panel with a camera. Use when a task asks to verify a display (LCD, e-paper, OLED, RLCD) on hardware, check orientation/rotation, partial updates, colours, contrast, or that text and shapes render correctly. Covers finding the USB port, flashing a test pattern, capturing the panel with any camera, and reading the photo.
allowed-tools: Read, Write, Edit, Bash, Glob, Grep
---
# Display testing with a camera
Point a camera at the panel to get visual evidence that a driver renders correctly. Any camera
and capture tool works; this skill covers the common ones. `$SKILL` below is this directory
(`.agents/skills/display-test`).
## Ask before using a camera
A camera can show people and rooms, not only the panel. Before the first capture, ask the user
for permission and for which camera to use. List the cameras the host has (the commands below
show how), name the one you intend to use, and wait for the user to confirm it. Do not pick a
camera yourself, and do not fall back to another camera if the chosen one fails; ask again.
The permission covers this session and the camera the user named. Ask again for a different
camera or in a new session. If the user declines, stop and report that the display was not
checked visually.
## Capturing a frame
Grab a still image, skipping the first few frames so auto-exposure and white balance settle.
Read the JPEG with the Read tool; the image is shown visually.
### macOS
The bundled Swift tool uses AVFoundation and avoids the frame-grabbing problems some USB
cameras have with other tools:
```bash
mkdir -p /tmp/camsnap /tmp/display-test
swiftc -O "$SKILL/scripts/camsnap.swift" -o /tmp/camsnap/camsnap
/tmp/camsnap/camsnap <camera-name-substring> /tmp/display-test/after.jpg 30
```
The camera argument is a case-insensitive substring of the name AVFoundation reports. If it does
not match, the tool lists the cameras it found, so a run with a name that matches nothing lists
the cameras without capturing. Use a substring that matches only the camera the user approved.
It creates the output directory if needed and reports encoding or write errors on stderr. Other
options include `ffmpeg` with `avfoundation` input, `imagesnap`, or Photo Booth's still capture.
### Linux
Use `ffmpeg` with the v4l2 input, `fswebcam`, or `v4l2-ctl`. With `ffmpeg`, `-ss 2` after the
input discards the first two seconds of frames while exposure settles:
```bash
mkdir -p /tmp/display-test
ffmpeg -f v4l2 -i /dev/video0 -ss 2 -frames:v 1 -y /tmp/display-test/after.jpg
# or
fswebcam -d /dev/video0 --skip 10 /tmp/display-test/after.jpg
```
List devices with `v4l2-ctl --list-devices`.
### Windows
Use `ffmpeg` with the DirectShow input (`-ss 2` skips the first frames, as on Linux), or any
camera application that can save a still image:
```powershell
mkdir C:\temp -Force | Out-Null
ffmpeg -f dshow -list_devices true -i dummy # find the device name
ffmpeg -f dshow -i video="<device name>" -ss 2 -frames:v 1 -y C:\temp\after.jpg
```
### Camera caveats
- The board may sit at an angle. Do not trust absolute pixel positions or aspect ratios from the
photo; judge orientation, ordering and approximate spacing, using the corner markers as
references.
- Reflections or a glossy cover can brighten one side. Capture again if a region is unclear.
- A moving pattern is caught at an arbitrary phase. Take several captures, or use a static marker
per position.
## Board and port
Find the port; names change on every replug:
```bash
python3 "$SKILL/../hardware-test/scripts/ports.py"
```
Native-USB ESP32-S3/C3/C6/H2 boards show up as `USB JTAG/serial debug unit` and need
`logger: hardware_uart: USB_SERIAL_JTAG`.
## Test pattern
Write a scratch config outside the checkout. Use one `name:` per board and include:
- a white background, a black border and a filled square at the centre, drawn in the first
update (full-window check and scale);
- a 60x20 bar near each corner, one bar per later update with `auto_clear_enabled: false`, so
each bar is written by its own partial window. The bars are wider than tall, which shows a
90 or 270 degree rotation; the corners exercise the window alignment;
- a 20x20 square under the left end of the top-left bar, drawn in the same update, which marks
the top-left corner and shows a 180 degree rotation or a mirrored image.
Use `Color::BLACK` and `Color::WHITE` in lambdas. Do not use the old `COLOR_OFF`/`COLOR_ON`
constants; black must mean black.
A ready template is in `$SKILL/templates/display-pattern.yaml`. Substitute the display platform,
pins and (for a driver with fonts) a font. The marker pattern needs no font.
## Build, flash, capture
Run from the worktree under test so `python -m esphome` imports the checkout, not an installed
copy:
```bash
cd <worktree>
python3 -c "import esphome; print(esphome.__file__)" # must print the worktree path
PYTHONUNBUFFERED=1 python3 -m esphome compile /tmp/display-test/display-pattern.yaml
PYTHONUNBUFFERED=1 python3 -m esphome upload /tmp/display-test/display-pattern.yaml \
--device /dev/cu.usbmodemXXXX
sleep 6 # boot plus a few update intervals
<capture command> /tmp/display-test/after.jpg
```
The device path differs by OS and host; use the path printed by `ports.py`. If the USB console
wedges during flashing (esptool `No serial data received`), replug the board and retry. A board
powered only by a battery may need the USB cable for flashing; the panel keeps its image when
power is removed, which is useful for checking that an image is bistable.
## What to check
1. **Orientation**: the extra square sits under the bar in the expected top-left corner. If it is
in another corner, the image is rotated or mirrored. Test `rotation:` values with the same
pattern. The logical width and height swap at 90 and
270 degrees, so the 60x20 corner bars should then appear taller than wide on the physical
panel.
2. **Coverage**: the border touches all four edges, with no dead rows or columns at the edges.
3. **Partial updates**: with `auto_clear_enabled: false`, draw one marker per update. Each should
appear exactly at its requested position with no smearing, offset or mirrored placement.
Include all four corners.
4. **Text and contrast**: sharp glyphs, correct polarity (black text on white, not inverted), no
ghosting from the previous frame.
5. **Speed**: if the driver logs transfer sizes, check that a full refresh and a partial window
take the expected times.
Record the build commit, board, config, and the captured image path when reporting results.
## When a partial update looks wrong
Take a marker-only capture before assuming the mapping is wrong: the camera's perspective makes
absolute positions deceptive. If markers land in mirrored positions, the controller's address
order (`MADCTL` MX/MY/MV) or the RAM block mapping is inverted for the window; verify the
full-frame path separately, then fix the window computation.
## Add findings
Add new capture tools, panel quirks, or recovery steps here so the next session starts ahead.
@@ -1,71 +0,0 @@
// camsnap <camera name substring> <output.jpg> [frames to skip]
// macOS only: grabs one frame from an AVFoundation camera after letting exposure settle.
// See SKILL.md for capture tools on other systems.
import AVFoundation
import AppKit
import CoreImage
let args = CommandLine.arguments
guard args.count >= 3 else {
FileHandle.standardError.write("usage: camsnap <camera name> <out.jpg> [skip]\n".data(using: .utf8)!)
exit(2)
}
let wanted = args[1].lowercased()
let outPath = args[2]
let skip = args.count > 3 ? Int(args[3]) ?? 30 : 30
let discovery = AVCaptureDevice.DiscoverySession(
deviceTypes: [.external, .builtInWideAngleCamera], mediaType: .video, position: .unspecified)
guard let device = discovery.devices.first(where: { $0.localizedName.lowercased().contains(wanted) }) else {
let names = discovery.devices.map { $0.localizedName }.joined(separator: ", ")
FileHandle.standardError.write("no camera matching '\(wanted)'; found: \(names)\n".data(using: .utf8)!)
exit(1)
}
final class Grabber: NSObject, AVCaptureVideoDataOutputSampleBufferDelegate {
var seen = 0
let skip: Int
let out: String
init(skip: Int, out: String) { self.skip = skip; self.out = out }
func captureOutput(_ output: AVCaptureOutput, didOutput sampleBuffer: CMSampleBuffer,
from connection: AVCaptureConnection) {
seen += 1
guard seen > skip, let pixels = CMSampleBufferGetImageBuffer(sampleBuffer) else { return }
let image = CIImage(cvPixelBuffer: pixels)
let rep = NSBitmapImageRep(ciImage: image)
guard let jpeg = rep.representation(using: .jpeg, properties: [.compressionFactor: 0.9]) else {
FileHandle.standardError.write("failed to encode \(out) as JPEG\n".data(using: .utf8)!)
exit(1)
}
let url = URL(fileURLWithPath: out)
try? FileManager.default.createDirectory(
at: url.deletingLastPathComponent(), withIntermediateDirectories: true)
do {
try jpeg.write(to: url)
} catch {
FileHandle.standardError.write("failed to write \(out): \(error)\n".data(using: .utf8)!)
exit(1)
}
print("saved \(out) after \(seen) frames")
exit(0)
}
}
let session = AVCaptureSession()
session.sessionPreset = .high
guard let input = try? AVCaptureDeviceInput(device: device), session.canAddInput(input) else {
FileHandle.standardError.write("cannot open \(device.localizedName)\n".data(using: .utf8)!)
exit(1)
}
session.addInput(input)
let output = AVCaptureVideoDataOutput()
output.videoSettings = [kCVPixelBufferPixelFormatTypeKey as String: kCVPixelFormatType_32BGRA]
let grabber = Grabber(skip: skip, out: outPath)
output.setSampleBufferDelegate(grabber, queue: DispatchQueue(label: "camsnap"))
session.addOutput(output)
session.startRunning()
DispatchQueue.main.asyncAfter(deadline: .now() + 15) {
FileHandle.standardError.write("timed out waiting for frames\n".data(using: .utf8)!)
exit(1)
}
RunLoop.main.run()
@@ -1,43 +0,0 @@
# Display test pattern: corner markers, centre reference and partial-update markers.
# Replace the display platform and pins with the driver under test.
esphome:
name: display-pattern
esp32:
board: esp32-s3-devkitc-1
framework:
type: esp-idf
logger:
hardware_uart: USB_SERIAL_JTAG
level: DEBUG
spi:
clk_pin: GPIO11
mosi_pin: GPIO12
display:
- platform: st7305
id: test_display
cs_pin: GPIO40
dc_pin: GPIO5
reset_pin: GPIO41
update_interval: 1s
auto_clear_enabled: false
lambda: |-
static int frame = 0;
if (frame == 0) {
it.fill(Color::WHITE);
it.rectangle(0, 0, it.get_width(), it.get_height(), Color::BLACK);
it.filled_rectangle(it.get_width() / 2 - 10, it.get_height() / 2 - 10, 20, 20, Color::BLACK);
} else if (frame == 1) {
it.filled_rectangle(20, 20, 60, 20, Color::BLACK);
it.filled_rectangle(20, 50, 20, 20, Color::BLACK);
} else if (frame == 2) {
it.filled_rectangle(20, it.get_height() - 40, 60, 20, Color::BLACK);
} else if (frame == 3) {
it.filled_rectangle(it.get_width() - 80, 20, 60, 20, Color::BLACK);
} else if (frame == 4) {
it.filled_rectangle(it.get_width() - 80, it.get_height() - 40, 60, 20, Color::BLACK);
}
frame++;
-175
View File
@@ -1,175 +0,0 @@
---
name: embedded-memory
description: Choose ESPHome C++ containers and storage that avoid heap use and fragmentation on microcontrollers - std::array, StaticVector with cg.slot_list/SLOT_LIST, FixedVector, std::unique_ptr byte buffers, StringRef, RAMAllocator, and what to avoid (std::vector growth, std::map, std::deque, std::string, new (std::nothrow)). Use when adding or changing C++ members, containers, buffers, strings or registration lists, or when reducing RAM or flash use.
---
# Memory on embedded targets
ESP devices run for months with small heaps shared between Wi-Fi, BLE, LWIP, and application code. Over
time, repeated allocations of different sizes fragment the heap. Failures happen when the largest
contiguous block shrinks, even if total free heap is still large. We have seen field crashes caused by
this.
**Heap allocation after `setup()` should be avoided unless absolutely unavoidable.** Every
allocation/deallocation cycle contributes to fragmentation. ESPHome treats runtime heap allocation as a
long-term reliability bug, not a performance issue. Helpers that hide allocation (`std::string`,
`std::to_string`, string-returning helpers) are being deprecated and replaced with buffer and view based
APIs.
## Container guidelines
1. **Compile-time-known sizes:** use `std::array` instead of `std::vector` when size is known at compile
time.
```cpp
// Bad - generates STL realloc code
std::vector<int> values;
// Good - no dynamic allocation
std::array<int, MAX_VALUES> values;
```
Use `cg.add_define("MAX_VALUES", count)` to set the size from Python configuration.
**For byte buffers:** avoid `std::vector<uint8_t>` unless the buffer needs to grow. Use
`std::unique_ptr<uint8_t[]>` instead. It does **not** provide bounds checking or iterator support like
`std::vector<uint8_t>`; use it only when you do not need these features.
```cpp
// Bad - STL overhead for simple byte buffer
std::vector<uint8_t> buffer;
buffer.resize(256);
// Good - minimal overhead, single allocation
std::unique_ptr<uint8_t[]> buffer = std::make_unique<uint8_t[]>(256);
// Or if size is constant:
std::array<uint8_t, 256> buffer;
```
When the size comes from the config, declare it with `cg.static_buffer()` in `to_code` and pass the pointer
to the constructor, so it lands in BSS (see the `component-codegen` skill).
2. **Compile-time-known fixed sizes with vector-like API:** use `StaticVector` from
`esphome/core/helpers.h` for compile-time fixed size with a `push_back()` interface (no dynamic
allocation).
```cpp
// Bad - generates STL realloc code (_M_realloc_insert)
std::vector<ServiceRecord> services;
services.reserve(5); // Still includes reallocation machinery
// Good - compile-time fixed size, no dynamic allocation
StaticVector<ServiceRecord, MAX_SERVICES> services;
services.push_back(record1);
```
Use `cg.add_define("MAX_SERVICES", count)` to set the size from Python configuration.
**Listener and child-entity registration lists are the most common case, and the most commonly
missed.** When the hub only forwards data to its children and each child's handling is known at build
time, keep no list at all: use `cg.HubFunction` (see the component-codegen skill). When the hub must
store the children (it iterates them for other reasons, or picks fields at run time), a `register_*()`
method called once per child at code generation time has a count that is known at compile time, so it
should never be a `std::vector`. Declare the list with `SLOT_LIST` and register each child through the
call that `cg.slot_list()` returns, passing the hub's `DOMAIN`.
```python
# hub component's __init__.py
register_listener = cg.slot_list(
"MY_COMPONENT_LISTENER_COUNT", "register_listener", DOMAIN
)
# a platform of the hub
async def to_code(config: ConfigType) -> None:
...
hub = await cg.get_variable(config[CONF_MY_COMPONENT_ID])
cg.add(register_listener(hub, var))
```
```cpp
class MyComponent : public Component {
public:
SLOT_LIST(MyComponentListener, listeners_, register_listener, MY_COMPONENT_LISTENER_COUNT)
```
Do not put `#ifdef` around the macro or the loops over the list. Put `#if MY_COMPONENT_LISTENER_COUNT`
only around code that needs a child's own component (its header, its entity type).
If registering does more than append, write the method under `SLOT_LIST_REQUIRES(count)` and store into a
`[[no_unique_address]] SlotList<Entry, count>` member.
If the C++ side adds itself to the list, or the define gates more than the list, use `cg.slot_counter()`
keyed by hub (`_request_slot(str(hub))`) and put the storage behind `#ifdef`.
Request slots from `to_code`, not from a job that runs after `CoroPriority.FINAL`; a late request
raises rather than silently undercounting.
3. **Runtime-known sizes:** use `FixedVector` from `esphome/core/helpers.h` when the size is only known at
runtime initialization.
```cpp
// Bad - generates STL realloc code (_M_realloc_insert)
std::vector<TxtRecord> txt_records;
txt_records.reserve(5); // Still includes reallocation machinery
// Good - runtime size, single allocation, no reallocation machinery
FixedVector<TxtRecord> txt_records;
txt_records.init(record_count); // Initialize with exact size at runtime
```
This eliminates `_M_realloc_insert` and `_M_default_append` template instantiations (200-500 bytes per
instance), needs no upper bound, and is compatible with protobuf code generation when using
`[(fixed_vector) = true]`.
4. **Small datasets (1-16 elements):** use `std::vector` or `std::array` with simple structs instead of
`std::map`/`std::set`/`std::unordered_map`.
```cpp
// Bad - 2KB+ overhead for red-black tree/hash table
std::map<std::string, int> small_lookup;
std::unordered_map<int, std::string> tiny_map;
// Good - simple struct with linear search (std::vector is fine)
struct LookupEntry {
const char *key;
int value;
};
std::vector<LookupEntry> small_lookup = {
{"key1", 10},
{"key2", 20},
{"key3", 30},
};
// Or std::array if size is compile-time constant:
// std::array<LookupEntry, 3> small_lookup = {{ ... }};
```
Linear search on small datasets is often faster than hashing or tree overhead, but this depends on
lookup frequency; in hot paths the O(1) vs O(n) difference may still matter. It is the heavy containers
(`map`, `set`, `unordered_map`) that should be avoided for small datasets unless profiling shows
otherwise.
5. **Strings set once from configuration:** use `StringRef` (`esphome/core/string_ref.h`) rather than
`std::string`. Code generation passes a string literal that lives in flash for the life of the program,
so storing a `std::string` copies it onto the heap for nothing. `StringRef` is a non-owning pointer plus
length; it does not copy, and it must only ever refer to storage that outlives it (a string literal, or
a buffer owned elsewhere).
```cpp
// Bad - heap copy of a literal that is already in flash
void set_keys(std::string keys) { this->keys_ = std::move(keys); }
std::string keys_;
```
```cpp
// Good - no allocation
void set_keys(const char *keys) { this->keys_ = StringRef(keys); }
StringRef keys_;
```
6. **Avoid `std::deque`:** it allocates in 512-byte blocks regardless of element size, guaranteeing at
least 512 bytes of RAM usage immediately. This is a major source of crashes on memory-constrained
devices.
7. **Never use `new (std::nothrow)`:** on ESP-IDF exceptions are disabled, so a failed nothrow allocation
aborts instead of returning `nullptr`. Use `RAMAllocator` from `esphome/core/helpers.h`; CI rejects
`std::nothrow`.
8. **Detection:** look for these patterns in compiler output:
- Large code sections with STL symbols (vector, map, set)
- `alloc`, `realloc`, `dealloc` in symbol names
- `_M_realloc_insert`, `_M_default_append` (vector reallocation)
- Red-black tree code (`rb_tree`, `_Rb_tree`)
- Hash table infrastructure (`unordered_map`, `hash`)
Callback storage (`CallbackManager` vs `LazyCallbackManager`) and templated registration methods are
covered in the `automations` skill.
## Where to spend effort
Prioritize container optimization for core components (API, network, logger), widely used components
(mdns, wifi, ble), and components causing flash size complaints. Avoiding heap allocation after `setup()`
is always required regardless of component type; the priority list is about effort spent on container
migration (e.g. `std::vector` to `StaticVector`).
-197
View File
@@ -1,197 +0,0 @@
---
name: hardware-test
description: Test an ESPHome change on real boards over USB. Use this whenever a task asks to test, verify, reproduce, bench, A/B or soak a pull request, a branch or a crash on hardware, or mentions flashing, serial logs, esphome upload, a captive portal or fallback AP test, or results to post on a PR. Covers finding the serial ports, building from the right checkout, flashing each chip family (including the LibreTiny kits that stock tools cannot flash), bounded log capture, proven test flows and how to report results.
allowed-tools: Read, Write, Edit, Bash, Glob, Grep
---
# Hardware testing
The goal of a hardware test is evidence a reviewer can trust: which build ran on which board, what was
driven, what the logs show, and the same flow on a `dev` build when a quirk could be pre-existing.
`$SKILL` below is this directory (`.agents/skills/hardware-test`).
## Ground rules
1. **Serial ports cannot be shared.** One bounded capture per port, stopped before any flash on that
port, never an open-ended `esphome logs`. Stop a capture with `pkill -f "[s]erial_capture.py <port> "`,
naming the port so a capture on another board survives; the bracket keeps the pattern from matching
the shell that runs it. When more than one agent session shares the bench, run a copy of the script
under a session-specific name (`cap<topic>.py`) so another session's `pkill` cannot take your capture
down. Never run `esphome logs` in a `while true` loop: the loop respawns it and holds the port.
2. **Credentials stay local.** Put a `secrets.yaml` next to the scratch configs and reference it with
`!secret`. A copy of a device's own config keeps using the secrets file next to that config: two
checkouts can hold different networks, and the wrong one leaves the device unreachable after the OTA. Never paste a config with credentials into a PR, never commit scratch configs or logs.
3. **Build from the tree under test.** `python -m esphome` puts the current directory first on
`sys.path`, so run it with the checkout or worktree as the working directory and pass the yaml by
its absolute path. A venv `esphome` entry point may import an installed snapshot instead of the
checkout. Confirm with `python -c 'import esphome; print(esphome.__file__)'`, and for an A/B grep
the build's `src/` for the change before trusting any number. Tool calls sent together share one
shell, so a `cd` in one can land another's compile in the wrong tree: pass `PYTHONPATH=<worktree>` on
every compile and upload command and print `esphome.__file__` into the compile log.
4. **PR build and baseline build get different `name:` values.** Builds share
`.esphome/build/<name>`, so the same name means `esphome upload` can flash the wrong binary. Saved
Wi-Fi credentials are keyed by the config hash when the config has `networks:`, so a test that must
start unprovisioned gets a fresh name every run. A config with `ap:` only and no `networks:` uses one
fixed key shared by every such config on the board, so a new name does not help there: wipe once at
boot with an `on_boot` lambda that resets the preferences and saves a marker key
(`templates/dut-ap-only-wipe-once-bk72xx.yaml`).
5. **Flash and RAM deltas are measured against the PR's merge base** (`git merge-base origin/dev HEAD`
checked out in a throwaway worktree), with the baseline commit named in the table. When a delta
looks odd, diff symbol sizes (`nm -S -C --size-sort` from the matching toolchain) before quoting it.
When a change moves storage between the heap and static RAM, the static number alone misleads: also log
the free heap at the end of setup (`on_boot` at priority -100) over three boots of each build and report
the total in use (static RAM plus heap used).
6. **Prove a quirk is pre-existing before blaming the PR:** run the identical flow on a `dev` build in
the same session and say so in the results.
7. **Pipe safely.** Set `PYTHONUNBUFFERED=1` on any `esphome run` or `esphome logs` whose output is
piped, or lines land minutes late. Bound every long command with `timeout` (coreutils) or
`perl -e 'alarm shift; exec @ARGV' <seconds> <command>` on macOS without it. Save an exit code with `rc=$?`
right after the command: in `echo "$(date +%T) rc=$?"` the command substitution runs first and `$?` reads 0.
## Workflow
1. **Find the boards:** `python $SKILL/scripts/ports.py` (any Python with pyserial, for example the
ESPHome venv). Port names follow the USB hub port and change on every replug, so never reuse a name
from an earlier session. CH340 adapters (D1 mini, BW15 kit, BK7238, CB3S) all look the same; tell
them apart with a 10 s capture using `serial_capture.py --reset` (ESP8266 prints its ROM banner at
74880 baud, LibreTiny boards print `[LT]` and ESPHome lines, BK7238 prints its bootloader version).
`ports.py --wait 2e8a` blocks until a Pico re-enumerates.
2. **Make a scratch directory** outside the checkout, with `secrets.yaml` and the templates you need.
One yaml per board per variant, file name equal to `name:`.
3. **Compile** in the right tree:
`(cd <worktree> && PYTHONUNBUFFERED=1 timeout 1500 python -m esphome compile <abs yaml> > <name>-compile.log 2>&1)`.
Keep the RAM and flash lines at the end. Two ESP-IDF or two LibreTiny builds at the same time race
over tool caches; run them one at a time.
4. **Flash** per chip family (table below). Stop the capture on that port first.
5. **Capture** with `serial_capture.py PORT SECONDS OUT` in the background, then poll the file with
`grep` in an `until ... sleep 2` loop with a cap. USB CDC boards (Pico, nRF52, native USB ESP32)
drop the port on reboot; the script reopens it until its deadline.
6. **Drive** the test: a second board running a client config, `curl`, an aioesphomeapi script that
presses a template button and subscribes to logs, MQTT publishes, or `interval:` and `on_boot:`
automations in the config itself.
7. **Analyze** with `grep -a -c` counts over the capture and keep an `-events.txt` extract of the lines
that matter. Decode crashes with the platform's `addr2line` (see `reference/boards.md`).
8. **Report** on the PR. Every result comment starts with this two-line GitHub alert, then a blank line:
```
> [!NOTE]
> Automated testing results
```
Then one short sentence saying what was tested (builds and SHAs, board, what was driven), a results table
(build SHA, board and framework, what was driven, counts, result, plus the `dev` control row), short bullets
for the findings, and the setup and exact flow in a collapsed `<details>` block. Answer any question asked on
the PR directly in the comment. No credentials, SSIDs, IP addresses, port names or other bench details.
## Flashing by chip family
| Board | Flash | Log |
|---|---|---|
| ESP8266 D1 mini (CH340) | `python -m esphome upload <yaml> --device <port>` | `serial_capture.py` at 115200; `esphome logs` on some CH340s emits garbage after the RTS reset. A failed esptool write can wedge the adapter until the board is replugged. `esptool erase_flash` before a test that needs an unprovisioned device: the preference sector survives a reflash |
| ESP32 devkit (CP2102 or CH340) | `esphome upload --device <port>`, ESP-IDF or Arduino | `serial_capture.py`. Default Wi-Fi light sleep delays API driven actions by seconds; judge timing from device logs. `power_save_mode: none` for OTA overlap tests |
| ESP32-S3, C3, C6, H2 native USB | `esphome upload --device /dev/cu.usbmodem*` | `logger: hardware_uart: USB_SERIAL_JTAG`. The USB console can wedge when the app takes over the console (0 bytes, esptool `No serial data received`); replug, or watch through `esphome logs --device <name>.local` |
| Pico W / Pico 2 W | `esphome upload --device <port>` (1200 baud touch); first flash ever via BOOTSEL and `--device /Volumes/RPI-RP2` or `RP2350` | port re-enumerates after upload: `ports.py --wait 2e8a` then capture |
| BW15 kit (RTL8720CF, rtl87xx) | `~/.platformio/penv/.libretiny/bin/python $SKILL/scripts/flash_rtl87xx_kit.py flash write -d <port> <build>/.pioenvs/<name>/*.uf2`. Plain `esphome upload` and plain ltchiptool time out because opening the port asserts DTR/RTS and holds the chip in reset | `serial_capture.py --released`, never `esphome logs` |
| BW12 and other AmebaZ modules (RTL8710BX, rtl87xx) on a plain USB-UART | `~/.platformio/penv/.libretiny/bin/python -m ltchiptool flash write -B 115200 -b 115200 -t 120 -d <port> <uf2>`, started first, then strap by hand: CEN to GND, TX2 to GND, release CEN, release TX2. Without `-B 115200` the system data read at 1.5 Mbaud fails on CH340s (`Data read error at 0x9000`) and nothing is written. A 1 MB image takes about 80 s. A development board with DTR and RTS wired to CEN and the download pin needs no strap | `serial_capture.py`; a reset pulse does not reach CEN on a plain adapter, so a capture starts mid-run |
| BK7238 / CB3S (bk72xx, RTS on CEN) | `~/.platformio/penv/.libretiny/bin/python $SKILL/scripts/flash_bk72xx.py flash write -d <port> <uf2>` (about 75 s) | logs are on UART2 by default; `logger: hardware_uart: UART1` puts them on the flashing adapter. `serial_capture.py --reset` gives a chip-enable reset; a plain open may or may not reset it, check for the bootloader banner |
| LN882H | ltchiptool forces 1 Mbaud for writes; weak adapters need `LN882H_YM_BAUDRATE` patched to 115200 in the penv's `ln882htool.py`; short BOOT to GND while the tool connects | OTA after the first flash |
| nRF52840 Dongle (PCA10059) | Nordic `nrfutil nrf5sdk-tools dfu usb-serial` on the bootloader port, or `dfu: true` in the config so the 1200 baud touch pin-resets into the bootloader; adafruit-nrfutil is refused | app has two CDC ports, ESPHome log on the second |
| Adafruit-style nRF52 (UF2 bootloader) | `UF2BOOT` volume or adafruit-nrfutil; `dfu: true` works through GPREGRET | single CDC port |
| Any reachable device | `esphome upload <yaml> --device <ip or name.local>`; `--file <bin>` for a prebuilt image | `esphome logs --device <ip>` bounded with `timeout` |
ESP-IDF OTA image: `.esphome/build/<name>/build/firmware.ota.bin`. LibreTiny OTA port is 8892.
## Scripts in `$SKILL/scripts`
- `ports.py`: list ports with VID:PID, product, serial and a board guess; `--wait VID[:PID]`,
`--product NAME`, `--serial SN` (one specific board, whatever path it comes back under) and
`--serial-of PORT` (the serial number behind a path).
- `serial_capture.py PORT SECONDS OUT [--baud] [--released] [--reset] [--append] [--ms]`: bounded timestamped
capture (`--ms` adds milliseconds for timing work), ANSI stripped, `LOG_SECRET` text written as `<secret>`, reopens a vanished port, exits at the
deadline. OUT is truncated unless `--append` is given, so counts over it cover one run.
- `flash_rtl87xx_kit.py`: ltchiptool wrapper for the BW15 kit whose CH340 drives CEN (RTS) and the boot
strap (DTR). `flash read` needs the family `realtek-ambz2`; `flash write` does not.
- `flash_bk72xx.py`: ltchiptool wrapper for BK72xx boards whose RTS drives CEN. It misses the link now
and then; retry up to three times.
- `api_button.py HOST KEY list|press NAME [...]`: list a device's template buttons or press them over the API; the
recipes drive their test configs with it.
- `apiconn.py`: `connect(host, tries)` (API connect and login with retries, `API_KEY` from the env) and `log_text()`
(an API log line without ANSI codes), imported by the newer API drivers.
- `wifisave_hammer.sh KIND DUT_PORT DUT_YAML DUT_TREE CLIENT_PORT CLIENT_YAML CLIENT_TREE [SETTLE]`:
the fallback AP plus captive portal hammer end to end (compile, flash, capture, counts) with the
templates below. KIND is `esp`, `pico`, `bw15` or `bk`. It ends with `RESULT PASS` (exit 0),
`RESULT FAIL` (exit 3: a crash, a reboot after the AP came up, or the device never joined the network),
`RESULT INVALID`
(exit 2: a precondition was missed, the stimulus was insufficient, or a capture did not last the run)
or exits 1 when a build, flash or port step failed.
Per-topic scripts, each described in the matching `reference/recipes/<component>.md`: `online_image/`
(server with prefixed misbehaviour cases, PNG, JPEG, BMP and QOI generators, API drivers, first pass timer), `zwave_proxy/` (checksum verifier, device info), `usb_cdc_acm/`
(host driver, Zephyr driver probe), `rtsp_server/` (protocol client, raw rate reader, measurement set,
API image grab), `cse7766/` (API observer with ping), `zephyr_coredump/` (reader flow, decode with the
Zephyr tools), `libretiny_prefs/` (rtc preference harness header), `libretiny_crash/` (BK72xx crash probe header,
capture decoder, RTL8720C ROM dump versus report matcher), `api_outgoing/` (wake cycle and Wi-Fi drop timing readers), `ld2450/` (zone number driver, loopback harness driver with number and binary sensor states and TX frame summarizer, frame stream harness with presence hold and link loss drivers, BT MAC driver), `api/` (raw plaintext native API client for user defined actions, store_yaml recovery client over Noise or plaintext), `http_request/` (update manifest server, update state driver, request test server, action case driver, concurrency, reboot, update check and OTA drivers, `ota_auth/` OTA request cases with basic auth), `modem/` (button and sensor driver over the plaintext API, also used for the BLE MIDI boards), `dfrobot_sen0395/` (API driver and case runner), `homeassistant_light/` (direct bulb driver, held-report case driver), `usb_audio/` (URL player), `speaker_media_player/` (API player driver, pipeline task case runner), `web_server/` (SSE client, request mix, SSE and HTTPD case runners), `mqtt/` (`ha_discovery/`, `disable/`, `retained_burst/` and `reconnect/` runners; broker config with TLS, client-certificate and WebSocket listeners, test CA with server and client certificates, transport case runner, flash-and-drive runner, CLI TLS check without a board, MAC suffix runner with per-variant configs for two boards, `shutdown/` shutdown message runner and matcher), `udp/` (datagram recorder, `udp.write` payload checker and case driver), `entity_icons/` (icon and device class readback over the API, MQTT discovery, web_server and dump_config, with a comparer), `uart_sim/` (API driver and flash, capture and drive runner for the simulated UART sensors), `remote_base/` (loopback runner and API log decoder per sent frame), `xiaomi_ble/` (frame and header generator with the expected counts, injecting, advertising and receiving configs, counter and runner), `ble_terminate/` (proxy and peer configs, connect and disconnect driver, peer disconnect matcher, runner), `i2c_slave/` (D1 mini slave and BK7238 master configs), `wpa3_ap/` (PMF AP with join and leave logging, BK7238 and CB3S DUTs, runner and summary), `wisp_ap/` (WISP upstream with AP buttons, WISP and ALWAYS_ON DUTs, client, runner and per phase summary), `wifi_restart/` (WPA3 mixed WISP upstream, static IP DUTs with syslog, runner and summary), `ota_timing/` (OTA upload timer with deflate, raw and TCP modes, API build check, A/B runner and median summary, `inflate_bench/` on-device inflate benchmark with stream generator, configs and syslog runner), `gpio_hold/` (ISR pin hold self test with ESP32 and S3 configs), `ble_event_inject/` (GAP and GATTC event injector header and proxy package), `light_sleep_wake/` (LEDC wave and edge timer for light sleep wakeup on a C3, external wave generator and per-call edge detail probe for peripheral power-down on the P4), `manual_ip/` (station and AP manual IP configs, DNS probe and runners), `web_handlers/` (late handler, web_server, captive_portal and prometheus variants, web OTA runner), `bk72xx_flash/` (flash readback with retries, per partition diff of two dumps), `ble_server_security/` (secure GATT server and pairing client configs, driver, summary and runner), `prometheus/` (datetime metrics configs and driver), `zephyr_ble_server/` (nRF52 server, looping ESP32 client and UF2 flash and count runner), `ble_adv_triggers/` (advertiser with six raw frames, ESP32 and BK72xx receivers with eleven triggers, runner and checker), `sun/` (longitude sweep package, runner and checker), `sensor_filters/` (every filter shape with a fixed input sequence, per-board packages and runner), `led_driver_sniff/` (SPI slave capture header with selftest, config generator, window comparer and runner), `hub_input_sim/` (keypad, wiegand and rdm6300 simulation on a D1 mini, API log runner), `value_range/` (on_value_range sequence with a restart, runner), `ezo_fake_bus/` (fake EZO I2C bus and step config, runner), `twai_self/` (TWAI self-reception header, on_frame config and counter runner), `ota_automations/` (OTA automation package for both OTA platforms, four case runner, repeated web upload runner and per case summary), `output_button/` (pad edge timer header, button package and runner), `homeassistant_button/` (raw plaintext API client for action calls and wizard inputs, package and runner), `uart_button/` (UART0 loopback package, API driver with TX and RX counts, runner), `interval/` (plain and delayed interval package, timing summary and runner), `smp_udp_ota/` (SMP reset fault injector, OTA runner, per mode result check, MCUboot swap limit, running image check and slot blanking UF2, Thread device config), `ld2420_sim/` (fake LD2420 answering the hub's commands on a UART0 loopback, API state recorder, two-build comparer and runner), `nrf52_tcp_probe/` (TCP and UDP socket probe config, build and log check, API check over Thread), `rs485_frame/` (simulated DLE/STX/ETX bus device, frame and gate cases, runner), `hx711/` (fake HXLine truncated
## Templates in `$SKILL/templates`
- `dut-fallback-ap.yaml` (package): a device that never finds its network, falls back to its AP and
serves the captive portal. `example-dut.yaml` is the per-run file with the platform block.
- `client-wifisave.yaml` (package): joins that AP, posts N bogus `/wifisave` requests, then the real
credentials from `!secret`. `example-client.yaml` is the per-run file. `bogus_saves: "1"` gives a
plain provisioning check.
- Per-topic device configs, each described in `reference/recipes/<component>.md`: `online-image-dut.yaml`,
`usb-cdc-acm-dongle-dut.yaml`, `nrf52-preferences-dut.yaml`, `rtsp-waveshare-s3-eth-dut.yaml`,
`waveshare-s3-eth-camdiag.yaml`, `s31-cse7766-dut.yaml`, `nrf52-coredump-crash-dut.yaml`,
`bk72xx-rtc-prefs-dut.yaml`, `bk72xx-crash-probe-dut.yaml`, `dut-ap-only-wipe-once-bk72xx.yaml`,
`homeassistant-light-dut.yaml`, `deep-sleep-outgoing-dut.yaml`, `outgoing-wifi-drop-dut.yaml`,
`globals-storage-dut.yaml`, `usb-audio-waveshare-s3-eth-dut.yaml`, `usb-audio-sendspin-s3-eth-dut.yaml`,
`cast-pro-1-sendspin-control.yaml`, `aux-2-preprod-stock-dut.yaml`, `aux-2-preprod-usb-audio-dut.yaml`,
`dfrobot-sen0395-dut.yaml`, `aux-2-preprod-speaker-player-dut.yaml`, `web-server-sse-dut.yaml`,
`mqtt-transport-dut.yaml`, `mqtt-tls-dut.yaml`, `udp-write-dut.yaml`, `entity-icon-dut.yaml`,
`online-image-s3-eth-dut.yaml`,
and the directories `store_yaml/` (package with secrets one directory up, per-board files),
`http_request_update/` (manifest URL setter shapes, per-board files), `modem/` (modem only and modem plus
Wi-Fi boot checks), `ble_midi/` (fake BLE MIDI instrument and the client), `http_request_action/` (request
action package with per-board files and an OTA flash variant), `led_wire/` (DUT config and header that reads the
LED strip pads back with RMT RX and an SPI slave), `uart_sim/` (simulator header, common package, per-build parts and run files) and `remote_base/` (common
package, transmit script, per-board templates and the ESP8266 software sender), `i2c_fake/` (fake I2C device
headers with their test configs, see `fake_i2c.md`).
- `microphone_http_stream/`: base config, fake counter and USB microphone variants and the multi-reader client,
described in `reference/recipes/microphone_http_stream.md`.
## When the PR keeps moving
Record every head in the stack (`gh pr view N --json headRefOid`), build, and recheck before flashing; if anything
moved, wait five minutes and check again so the test runs on a settled head. When a head moved only by merging dev,
compare the PR's own diff (`git diff $(git merge-base OLD origin/dev) OLD` against the same for NEW); if it is
identical and the dev commits do not touch the code under test, earlier results still describe it, but rebuild so the
comment quotes the exact commits. Fetch `origin dev` before computing a merge base.
## When something looks wrong
- A capture stopped early or a line is missing: check the timestamps before trusting it.
- A script that pipes into `tee` never ends although its work is done: a background command written as
`(cd dir && server & echo $!)` backgrounds the whole `cd && server` list, so a shell stays behind holding the
pipe open and `$!` is that shell, not the server. Write `(cd dir && exec server > out 2>&1 < /dev/null) &`.
- The client never joins the AP: another board may still run an AP with the same SSID, or the device
booted with saved credentials because the name was reused. One AP device at a time on the bench.
- Builds come out byte-identical between branches: the wrong tree was imported (rule 3). The reverse
too: two binaries that differ only in the build timestamp and in `__LINE__` values of log calls below
the changed code are the same build; map the offsets with `objdump -h` and `addr2line` first.
- A boot the host cannot capture (a power cut, an OTA-only board): keep the boot report in a flash
preference and repeat it in a periodic alive line, as `scripts/libretiny_prefs/ltprefs.hpp` does.
- Multicast from the host never arrives on some access points: test it device to device. Broadcast
is lossy under Wi-Fi power save, unicast is not. Throttle UDP floods to a fixed packet rate.
- The merge base comes from a freshly fetched `origin/dev`; a stale local `dev` puts dependency bumps into the
PR diff and builds the control from the wrong commit.
- A `.local` name can stop resolving in the middle of a run; keep the address from the boot log and drive by it.
- Never point `framework: source:` at a local `file://` copy of LibreTiny to try a patch: PlatformIO installs it over
the shared `~/.platformio/platforms/libretiny`, so every later build on the machine uses the patched copy. Patch
through the ESPHome side (a linker `-Wl,-wrap,<symbol>` or a define) instead, and if it happened, restore the
directory from a clean copy of the same release.
- `wc -l` pads its count on macOS, so `tail -n +$n0` needs `$((n0+0))` or an arithmetic expansion.
- `api:` reboots a board after 15 min without a client: `reboot_timeout: 0s` on `api:` and `mqtt:` for
soak tests.
`reference/boards.md` has per-chip facts and crash decoding; `reference/recipes/` has the proven
flows. Add new findings there.
@@ -1,416 +0,0 @@
# Board facts
Ports are not listed on purpose: they follow the USB hub port and change on every replug. Run
`scripts/ports.py`.
## ESP8266 (D1 mini, NodeMCU, CH340 or CP2102)
- Opening the port usually resets the board through DTR, but not always; `serial_capture.py --reset`
pulses RTS to capture from boot. 74880 baud shows the ROM boot messages (`rst cause`, `boot mode`).
- UART0 can be looped back onto itself (`USC0(0) |= (1 << UCLBE)` in an `on_boot` lambda) so a UART
component is testable without its device: frames written with `uart.write` arrive on RX. Log over the
API with `logger: baud_rate: 0`, since the serial port carries the component's bytes.
- `esphome logs` on some CH340s emits thousands of duplicated garbage lines right after the RTS reset;
a plain pyserial capture at 115200 is reliable.
- A failed esptool write can wedge the adapter (`termios` error 22, or an open that hangs) until replug.
- The preference sector survives a reflash: a provisioned API key or saved Wi-Fi credentials stay
active. `esptool erase_flash` gives a clean device.
- `Panic core_esp8266_main.cpp:191 __yield` with `ctx: sys` means a yield from lwIP context (a web
handler, an lwIP callback). Decode stacks with
`xtensa-lx106-elf-addr2line -e <build>/.pioenvs/<name>/firmware.elf -f -C -i <addr...>` from the
toolchain under `~/.platformio/packages/toolchain-xtensa/bin/` (PlatformIO builds) or the ESPHome
cache directory (native toolchain builds).
- Constant data lives in RAM unless marked `PROGMEM`; with 80 KB of RAM a few hundred bytes are worth a
row in the PR.
- AsyncMqttClient drops inbound topics over 128 characters; topics of 255 or more wrap a counter and
stall the link until the broker drops TCP.
## ESP32 classic devkit
- Takes ESP-IDF and Arduino. Default Wi-Fi light sleep delays API driven actions by up to 6 s; judge
timings from device logs, not from the host. `power_save_mode: none` for OTA overlap tests (an IDF web
upload of 739 KB took 107 s with power save and 45 s without).
- Receives only a few percent of IPv6 multicast on some access points with power save on; a raw lwIP
socket loses the same, so that is the AP or driver, not a PR.
- Decode with `xtensa-esp32-elf-addr2line` from the IDF tools directory against
`<build>/.pioenvs/<name>/firmware.elf` (PlatformIO) or `<build>/build/<name>.elf` (IDF native).
- Clean reboot between runs with the capture stopped: `esptool --after hard_reset chip_id`.
- Trigger a forced crash only after the app is marked valid (45 s after boot is plenty), never on an
image still pending OTA rollback.
- Signed OTA testing: build with `esp32.framework.advanced.signed_ota_verification.verification_keys`
under a different name, sign with `python -m espsecure sign_data --version 2 --keyfile
tests/components/esp32/dummy_signing_key.pem`, upload with `esphome upload --file`. The verifier
refuses unsigned images afterwards, so restore with a signed plain build.
- An ESP32 whose bootloader has no app rollback is protected only by `safe_mode` after a bad OTA. Before sending a
change to such a device that nobody can reach by hand, boot the same code on a devkit first and keep a fallback
image of the merge base built.
## ESP32-S3, C3, C6, H2 with native USB
- `/dev/cu.usbmodem*`, esptool works, `logger: hardware_uart: USB_SERIAL_JTAG`.
- The USB console can die at the moment the app takes over the console during a reboot (log stops after
`Disabling RNG early entropy source`, then 0 bytes, esptool reports `No serial data received`). Only a
USB replug fixes it. Fallback: `esphome logs --device <name>.local` (API log stream, which drops lines
while Wi-Fi is off).
- The ESP32-P4 before silicon revision 3 is bricked by rev3-only firmware (boot loop at bootloader entry). The
Waveshare ESP32-P4-WIFI6 rev1.3 board is such pre-rev3 silicon.
- S3, C3 and C6 all enumerate as `303a:1001`; confirm the chip with `esptool --port <port> chip_id` before flashing.
- ESP32-C6 test configs: set `wifi: power_save_mode: none`. With the default light sleep, API commands and state
updates arrived seconds late and a 1.2 s step check failed at random on base and PR alike (esphome#22339).
- An S3 on Arduino with `logger: hardware_uart: USB_CDC` enumerates as TinyUSB CDC (`303a:0009`). If that build
crash loops (a FastLED plus NeoPixelBus build aborts in `check_rmt_legacy_driver_conflict`), esptool cannot catch
the port: hold BOOT, tap RST, keep BOOT held about 2 s, then flash with `--before no-reset`. A plain RTS hard reset
leaves it in the loader on USB JTAG; `--after watchdog-reset` boots the app. Keep Arduino LED tests on the classic
ESP32.
- A board left in the ROM bootloader after an upload (BOOT held, or the USB JTAG reset landing in download mode
again) starts its app without a replug through `esptool --port <port> --after watchdog-reset chip-id`; flash a
minimal Wi-Fi plus API image first when recovering from a crash-looping one.
## RP2040 / RP2350 (Pico W, Pico 2 W)
- The CDC port re-enumerates after every reboot and after `esphome upload` (1200 baud touch). Re-detect
with `ports.py --wait 2e8a` before capturing; `serial_capture.py` reopens on its own if the port
vanishes mid-capture.
- First flash ever: BOOTSEL, then `esphome upload --device /Volumes/RPI-RP2` (or `RP2350`).
- arduino-pico creates the lwIP netif in `WiFi.begin()`, so any IGMP join during component setup fails;
joins must wait for `network::is_connected()`.
- Parallel Pico builds race on package extraction after a PlatformIO package wipe; one at a time.
- An RTS pulse from pyserial does not reset the board; reboot with `App.safe_reboot()` or the touch.
- LEAmDNS re-enters `update()` from the receive handler on the Pico too (measured), so an ESP8266-only guard does
not cover it.
- An image with only `ota: platform: web_server`, built from a tree without the fix for uploads of unknown size
(before 2026-10-19), cannot be updated again over the network; see `recipes/ota.md` before flashing one.
- RP2350 `panic()` goes through `_exit()` and `__breakpoint()` to a HardFault, and ESPHome's rp2 crash handler
reboots within about a second (USB back 1 to 2 s after the panic). A panic during component setup therefore
boot-loops before USB CDC ever enumerates, which looks like a dead board: no port and no BOOTSEL drive. Recover
with BOOTSEL held on replug, then flash to the `RP2350` volume. To see the crash, trigger it after boot.
- A Pico 2 W can sample any of its own GPIOs from a spare PIO block (PIO2) at 25 MHz while another peripheral
drives the pin, since PIO reads the pad without taking it. That decodes WS2812-style pulses with no wires or
LEDs. Turn a light on at full brightness before probing, or color correction writes zeros. The sample FIFO
overruns now and then; count only full captures.
- A Pico 2 W also makes a byte-exact fake I2C sensor: a raw RP2350 I2C0 interrupt handler (GP4/GP5) that answers
each RD_REQ with one byte from a 256-byte register file, so any read length works (used for BME280/BMP280 in
esphome#22236). The ESP32-S3 I2C slave v2 driver is not usable for register emulation: answering from a task or
straight from the interrupt both gave shifted multi-byte reads.
## BW15 kit (RTL8720CF, LibreTiny rtl87xx)
- The kit's CH340 wires RTS to CEN and DTR to PA00 (boot strap). A plain pyserial open asserts both and
holds the chip in reset, which is why `esphome upload`, plain `ltchiptool` and `esphome logs` all fail
("Timeout while linking", or silence). `scripts/flash_rtl87xx_kit.py` sets DTR high and RTS released
on every open, pulses RTS once so the ROM answers `ping`, and after the command drops DTR and pulses
RTS to boot the app. Run it with the PlatformIO LibreTiny penv Python.
- `flash write -d <port> <uf2>` takes no family argument; `flash read -d <port> -s 0 -l <len>
realtek-ambz2 <out>` does.
- Capture with `serial_capture.py --released`. The ESPHome log is on the flashing UART already.
- The macOS CH340 driver only pushes DTR/RTS to the wire once a session has driven them, so the first
plain open after a replug streams logs and the second is silent; always open with the lines released.
- Driver quirks seen on `dev`: `WiFi.begin` can refuse the real network on the first try after bogus
credentials (it connects after two cooldowns); right after connecting while the AP is still up the
driver can bus-fault in `rtw_is_adapter_up` (`R0 = 0xdeadbeef`) and reboot; the AP restarts on every
STA retry, so stations cannot stay joined; `WiFi.begin` fails synchronously for an absent network and
lands in a cooldown at once; connecting blocks the loop 2.5 to 4.1 s.
- Decode faults with `arm-none-eabi-addr2line -e <build>/.pioenvs/<name>/raw_firmware.elf -f -C -i <PC>`
from `~/.platformio/packages/toolchain-gccarmnoneeabi/bin/` (`raw_firmware.elf`, not `firmware.elf`).
Fault dumps print PC, LR and R0 to R3.
- The AmebaZ2 SDK sets `LWIP_TCPIP_CORE_LOCKING 0`.
- The `rtw_is_adapter_up` bus fault after `Disabling AP` happens on LibreTiny 1.13.0 and 1.14.0 alike
(2 of 4 and 3 of 3 runs in #21531); run at least four rounds per build before blaming a PR for it.
- On a fault the ROM handler (ram_vector_table entries 3 to 7) prints a dump of about 4 KB in 0.4 s and
then spins; only the watchdog resets the chip (10 s after `arch_init`, 1 s once the ESPHome crash
handler re-arms it). `.ram.bss.noinit*` (`.system_restore_data`) survives that reset. The core has no
FPU. For a crash that fires once, guard an `interval` lambda with
`!libretiny::crash_handler_has_data()`. For a handler mode (MSP) fault, point vector 16+11 (SDIOD,
unused) at a trapping function and set bit 11 in NVIC ISER and ISPR (0xE000E100, 0xE000E200).
- OTA slots (esphome#22012): a serial `ltchiptool flash write` fills both OTA1 and OTA2, and the slot flag decides
which one boots (`OTA1 USE` or `OTA2 USE` in the ROM banner). Each OTA writes the other slot, so a test that needs
the OTA1 to OTA2 direction runs two OTAs in a row. On dev before #22111 a deflate OTA into OTA2 corrupted the
image, and the next restart hung at `System_Init1` until a serial reflash.
- If the chip still waits in the ROM after a serial flash, pulse CEN to GND once (TX2 floating) to boot the app.
`expected ACK` failures at random blocks mid-transfer can come from one USB hub port; another port fixed them.
- Its Wi-Fi is slow and drops (esphome#21742): a 300 KB deflate OTA takes 3 to 10 minutes, so give the uploader 30
minutes, not 10.
- Wrong-MD5 OTA test without changing firmware: monkeypatch `espota2.hashlib.md5` so only the image checksum (data
over 64 KB) returns a wrong hexdigest. Give each build its own `name:` and read `device_info().name` after a
restart to see which image booted.
## BK7238, WB3S, CB3S (BK7231N) (LibreTiny bk72xx, CH340, RTS on CEN)
- Stock ltchiptool closes and reopens the port after its reset, and pyserial raises both lines on open,
holding CEN low through the BootROM window. `scripts/flash_bk72xx.py` opens with both released and
pulses the reset right before the link checks. A write takes about 75 s. Only a CEN or power reset
reaches the downloader; no software reboot does.
- ESPHome logs on UART2 by default; `logger: hardware_uart: UART1` puts them on the flashing adapter.
- Opening the port may reset the chip (bootloader banner on open) or may not, depending on the adapter and
what pyserial does with DTR and RTS; `serial_capture.py --reset` pulses RTS on CEN and always does, and
`lt_get_reboot_reason()` reports that as Power-On. Check for the banner at the start of a capture before
assuming either.
- `flash_bk72xx.py` misses the link now and then (`Timed out attempting to link with chip`); retry up to 3
times before suspecting the board.
- Crashes: `__builtin_trap()` and a call into unmapped memory both end as an undefined-instruction trap
(`bk_misc_get_start_type()` 6, `lt_get_reboot_reason()` Crash); reads, unaligned loads and stores to
unmapped addresses never fault on the ARM968. Without Wi-Fi the SDK trap handler spins with interrupts off
for 10 s and the hardware watchdog resets the chip. With Wi-Fi connected a dev build hangs forever after a
crash, and never resets from a plain `portDISABLE_INTERRUPTS(); while (1) {}` either: the boot-time
watchdog is dead once Wi-Fi is up, for any lockup. The crash handler fix (#21012) re-arms the watchdog
from its wrapper, after which the reboot comes within a second (BK7238 and BK7231N).
- The SDK prints nothing on UART1 under `sdk_silent: all`; the bootloader then prints the banked
`cpsr/spsr/r13/r14` of SVC, IRQ, FIQ and SYS plus `ST:` (the retention register) before `go os_addr`.
A CEN or power reset wipes ITCM (reads `0xaaaaaaaa`), so only a watchdog reset keeps a record there. What
survives that reboot on the BK7238: ITCM (`0x3F0000..0x3F4000`) and the top 96 KB of SRAM
(`0x428000..0x440000`, the heap); the bootloader zeroes SRAM up to about `0x420000` and rewrites
`0x400024..0x4000FF` with its own register dump every boot, so neither a `.noinit` after `.bss` nor the
SDK's saved register banks are usable. Code is compiled Thumb, the SDK assembly is ARM; in an exception
handler only r0 to r3 are shared by all modes. BK7231N ITCM is `0x3FEE00..0x400000` (4.5 KB); a `.noinit`
placed there after `.itcm.code` works for a crash record.
- Pick the board that matches the module's flash layout. A NiceMCU XH-WB3S (BK7238) needs `board: xh-wb3s` (plain
Beken layout, RF calibration at 0x1E0000). Built as `generic-bk7238-tuya` (calibration at 0x1E3000, download
partition over 0x1E0000) the firmware boots and may even join a known network, but scans come back empty and its
AP is invisible to clients, with no warning in the log. If scans are empty, check the layout first.
- The BK7238 bootloader prints `App verify failed! Need to recovery factory firmware.` on every boot with any
build; it is harmless. Even with the right layout an ESP32 client did not see the BK7238 fallback AP on
LibreTiny 1.13.0 or 1.14.0, so the wifisave hammer cannot run on it.
- Moving a BK7231N device (CB3S) that sits on its fallback AP to another network: join the host's Wi-Fi to
`<name>-fallback`, OTA to 192.168.4.1, then rejoin the host's own network. Moving it with `/wifisave` instead can
leave it in the captive portal's BLIND_RETRY state (no scan, `Beacon Timeout` every 46 s) until it reboots, and a
log-only serial adapter cannot reset it. Rejoining may change the host's address; update a syslog target that
points at the old one.
- Callbacks run on the `async_tcp` task and log lines can lose a byte. Wi-Fi connect takes tens of ms;
radio enable 160 to 278 ms, disable about 200 ms.
- OTA install: the BK7238 bootloader installs any complete download whose own CRC passes at the next restart, even
when ESPHome refused the update; an incomplete download fails `rt_ota_part_fw_verify` (`Verify firmware CRC32
... !=`) and the old image stays.
- Power save: on the BK7238, `power_save_mode: light` with LibreTiny 1.14.0 gives ping times of 3 to 6 s because
the BDK sets a listen interval of 30 beacons when `CFG_LOW_VOLTAGE_PS` is 1 (esphome#19491).
## BW12 and other AmebaZ boards (RTL8710BX, LibreTiny rtl87xx)
- On a bare module wired to a plain USB-UART (the bench's BW12), download mode is a hand strap: CEN to
GND, TX2 to GND, release CEN, release TX2. Such an adapter does not drive CEN, so neither ltchiptool nor
`serial_capture.py --reset` can reset the chip. A development board that wires DTR and RTS to CEN and the
download pin enters download mode from the tools, like the BW15 kit; the notes below on baud rates still apply.
- ltchiptool links at 1.5 Mbaud and reads the system data at 0x9000 at that rate before it writes at
115200. A CH340 drops bytes there (`Data read error at 0x9000`, `Timed out waiting for Quiet-Handshake`)
and the run gives up without writing. `-B 115200 -b 115200` moves the link to 115200 first and works.
Start the command with `-t 120` so it is already polling while the board is strapped.
- After a failed run the ROM may sit at another baud rate and no longer link; strap again.
- The device builder frontend's AmebaZ flasher follows ltchiptool (link and system read at 1.5 Mbaud),
so it can hit the same read failure on the same adapters.
- Power the module from the adapter's 5 V pin into its own regulator; the adapter's 3.3 V pin cannot
carry the Wi-Fi transmit peaks.
- Capture with `serial_capture.py --released`. With DTR and RTS asserted on the open port, a software
reboot (`App.safe_reboot()`, the restart button) lands in the ROM downloader (`UARTIMG_Download 2`)
and stays there until CEN is pulsed. After a flash the chip also waits in the ROM until CEN is pulsed.
- A full flash takes 50 to 220 s at 115200. While the board is on Wi-Fi, OTA (port 8892) saves the strap,
but the board's Wi-Fi latency (one to two seconds a ping) can make the upload time out.
- `sdk_silent: none` under `framework:` keeps the Wi-Fi driver's join log (`start auth to`, `auth success,
start assoc`, `association success`, `sta recv deauth reason code(N)`); LibreTiny mutes it during the
connect otherwise. `wifi_get_last_error()` after a failed `WiFi.begin()` gives the SDK's reason
(1 no network, 2 connect failed, 3 handshake failed, 6 auth failed); LibreTiny only logs `ret=-1`.
- Some modules ship with no RF calibration in efuse, only the chip ID and MAC. Dump the logical map with
the ROM's `EFUSE_LogicalMap_Read(buf)` (512 bytes): TX power indexes are at 0x10 to 0x1A, crystal trim
at 0xC9, MAC at 0x11A; 0xFF means the driver uses its default (trim 0x20). Linking with
`-Wl,-wrap,EFUSE_LogicalMap_Read_ACUT` lets a wrapper rewrite blank bytes before the driver parses
the map on each radio start. See `recipes/libretiny.md` for the per-boot trial harness used to A/B them.
- Scan results report WPA2-AES access points as `WPA PSK`: LibreTiny truncates `rtw_security_t` to
`uint8_t` before mapping it. Cosmetic. Open networks come out as `WIFI_AUTH_INVALID` (255) on both
AmebaZ and AmebaZ2, so `auth != WIFI_AUTH_OPEN` marks them encrypted.
- After a BSSID plus channel join, LibreTiny's `set_pscan_channel(ch, false)` leaves the driver with no
channel to scan, so the next SSID only join fails at once (no `start auth`), and the first scan after a BSSID
join comes back empty (no results in about 62 ms on the BW15). A `wifi_set_pscan_chan` list of channels 1 to 13
returns 0 but leaves only the last channel enabled, and re-applying the channel plan does not help either
(AmebaZ2 reports plan 0xFF; AmebaZ's `rtw_set_chplan_cmd` rejects plans above 0x58 except 0x7F). What works
(esphome#21749, BW15): set the one-channel pscan list yourself and pass channel 0, so the join's own scan
rebuilds the full list; setting a placeholder SSID `-` before narrowing (esphome#21754) kept joins as fast as
dev with the first scan full on every boot. Not yet confirmed on AmebaZ.
- A 4-way handshake that never finishes after association is reported as `RTW_WRONG_PASSWORD` on AmebaZ
even with the right password; AmebaZ2 reports a real wrong password as a 4-way handshake timeout.
- The driver also posts a SCAN_DONE (number 0) inside every join, on AmebaZ2 too.
## BW16 kit (RTL8720DN, AmebaD)
- The kit's own USB-C (CH340) is on LOG_UART and flashes fine. Its two-transistor circuit resets while
RTS alone is asserted and straps BURN while DTR alone is; both asserted does nothing. ROM entry is
esptool's classic sequence (`dtr:false,rts:true` 100 ms, `dtr:true,rts:false` 400 ms, both false); the
first pulse after a replug can be lost, retry up to three times.
- Any reopen resets the chip, 460800 is flaky, 115200 is solid: one open per session, in-place baud
changes only. The ROM and the loader NAK continuously while idle, so ACK scans must drain first.
- The AmebaD core has no mDNS responder (`mdns: disabled: true`).
## LN882H
- ltchiptool forces 1 Mbaud for writes; adapters that cannot hold it fail with "Timeout while linking"
or die mid-write. Patch `LN882H_YM_BAUDRATE` in the penv's `ltchiptool/soc/ln882h/util/ln882htool.py`
to 115200 (lost on reprovision). Weak 3.3 V from the adapter: feed the module's 5 V pad instead. Short
BOOT to GND while the tool connects. OTA afterwards.
- `USE_WIFI_AP_EXCLUSIVE` (one radio mode at a time) exists only on LN882x; compile-check changes there.
- The ESPHome OTA port on LN882x is 8820, not LibreTiny's 8892 (BK72xx, RTL87xx); helper scripts that hardcode
8892 get `Connection refused`.
- WL2H-U modules with the Tuya flash layout: app at 0x007000 (0x113000 long), OTA at 0x11A000 (0xA8000 long). A
full OTA upload over a slow link takes 2 to 14 minutes.
- UF2 block order differs per family: LN882H puts the device image first and the flasher image after it, BK72xx
puts the flasher image first and the device image last. The LN882H bootloader installs a complete, valid OTA
partition even when the upgrade state is still `UPG_STATE_DOWNLOAD_ING` (esphome#22111).
- Without serial access to recover, never send a corrupted image; wrong-MD5 and cut-upload tests with valid
images are safe.
## nRF52840
- Nordic PCA10059 dongle: the factory Open DFU bootloader enumerates as `1915:521F` on its own port
name; the app as `2FE3:0100` with two CDC ports, ESPHome log on the second. Only Nordic
`nrfutil nrf5sdk-tools` works (`pkg generate --hw-version 52 --sd-req 0x00 --application-version 1
--application zephyr.hex app.zip`, then `dfu usb-serial -pkg app.zip -p <port>`); ESPHome downloads
it on the first build. GPREGRET does not enter the bootloader; a pin reset through P0.19 does, which
`dfu: true` wires up on this board. A pulsing red LED means bootloader; the side RESET button
re-enters it.
- Makerdiary MDK USB dongle: UF2 bootloader without SoftDevice, app at 0x1000, `UF2BOOT` volume,
adafruit-nrfutil, GPREGRET 0x57 works. The running app enumerates as `1915:CAFE` or `239A:0029`. RGB LED
active low (red P0.23, green P0.22, blue P0.24), button P0.18.
- Both boards reflash without a button once they run a `dfu: true` build: `esphome upload --device <any app CDC
port>` on the PCA10059, a 1200 bps open of the log port plus a copy of `zephyr.uf2` to the volume on the ItsyBitsy.
- Adafruit ItsyBitsy nRF52840 Express (UF2 bootloader 0.9.0 with S140 6.1.1, bootloader port `239a:0051` plus an
`ITSY840BOOT` volume): the board DTS already has `cdc_acm_uart0`, so the logger port is the first CDC
port and a `usb_cdc_acm` interface 1 the second. A `dfu: true` build enters the bootloader on the
1200 bps touch (an app without it needs a double tap on RESET); `adafruit-nrfutil` over the bootloader
serial port can fail, copying `.pioenvs/<name>/zephyr/zephyr.uf2` to the volume works (the copy reports
an I/O error as the board reboots, which is normal). Reboot a running `dfu: true` app with a 2001 bps touch.
- The PCA10059 board DTS already defines a `cdc_acm_uart` node, so every ESPHome CDC port is one USB port
later than its number (log on `cdc_acm_uart0` is the second port). A third CDC instance needs
`CONFIG_USB_MAX_NUM_TRANSFERS` above 4 or it never moves data (see `recipes/usb_cdc_acm.md`).
- USB CDC boot output: the Zephyr legacy stack's `cdc_acm` TX ring is 1 KB and `uart_poll_out` drops when it
is full, so anything printed in one burst beyond about 1 KB (a coredump, a long table) is lost even with
the port open, and the logger warns `logger took a long time`. Print one line per `loop()` instead.
- Zephyr settings survive a reflash: key a `restore_value` global by a `run` substitution when a test needs
a never-saved value.
- Thread: nRF52 `openthread:` accepts only `tlv:`; the individual keys (network_key, channel, ...) are rejected at
validation. To list nearby networks with channel and PAN ID, call
`otThreadDiscover(InstanceLock::acquire().get_instance(), 0, 0xFFFF, false, false, cb, nullptr)` from an
`interval:` lambda in an included header; results arrive on the OpenThread thread and log fine.
- A wrong network key on a real network's channel and PAN ID makes the device form its own partition as leader on
that PAN within about 30 s; take it off Thread at once. Never try guessed datasets against a network in use.
## Waveshare ESP32-S3-ETH with the OV5640 camera module
- Native USB console (`303a:1001`), ESP32-S3R8, 16 MB flash, 8 MB octal PSRAM, W5500 on SPI (MOSI 11,
MISO 12, SCLK 13, CS 14, INT 10, RST 9), TF card on 4..7, WS2812 on 21.
- Camera header (vendor schematic): XCLK 3, SIOD 48, SIOC 47, VSYNC 1 (a 0 ohm link to GPIO1, alternative
GPIO16), HREF 2, PCLK 39, D0..D7 = 41, 45, 46, 42, 40, 38, 15, 18. PWDN is tied to GND and RESET is an RC to
3.3 V on the board, so no reset or power-down GPIO exists.
- It can run from PoE and USB-C at once, so the USB-C can move from the host to a USB device for `usb_uart` or USB
host tests while the board stays up; flash and read logs over Ethernet while it is unplugged. In USB host mode
the USB-C is the host port after the first flash: log over the API with `logger: hardware_uart: UART0`.
- A crash-looping image can wedge its USB console until a BOOT replug. Never force allocation failures on it by
hogging RAM: Wi-Fi then aborts in `operator new` or the loop task trips the task watchdog.
- GPIO8 switches the camera 3.3 V rail through a P-channel MOSFET: LOW = camera powered, gate pulled up so
the camera is off by default. `power_down_pin: GPIO8` only pulls it low 10 ms before the probe, which is
not enough; use `switch: gpio GPIO8 restore_mode: ALWAYS_OFF setup_priority: 800` plus `on_boot:
priority: 700` with `delay(1200)` and `esp32_camera: setup_priority: 200`
(`templates/rtsp-waveshare-s3-eth-dut.yaml`). `esphome config` rejects a config that uses GPIO8 for both the switch
and `power_down_pin` (pin reuse), so it is one or the other.
- The sensor at SCCB 0x3C answers only with XCLK running, so the ESPHome i2c scan at boot (before XCLK)
reporting no devices on 48/47 is expected. If the SCCB ID register (0x3100 = 0x78) reads fine but every
0x30xx register reads 0x00 (`Detected camera not supported`, 0x106), reseat the ribbon before blaming
software; `templates/waveshare-s3-eth-camdiag.yaml` runs that probe with LEDC XCLK and raw SCCB reads.
## ESP Thread Border Router board v1.2 (S3 host + H2 RCP)
- S3 host on USB JTAG serial, H2 RCP on its own USB. Spinel UART1 from the host: `rx_pin: GPIO17`,
`tx_pin: GPIO18`, RCP reset `GPIO7` active low, 460800. The config needs `network: enable_ipv6: true`.
- The shipped H2 firmware fails `CheckRadioCapabilities` with IDF 5.5; build `examples/openthread/ot_rcp`
for esp32h2 with ESPHome's own IDF (`IDF_PATH` at the cached framework, its penv and tools,
`IDF_PYTHON_CHECK_CONSTRAINTS=no`) and flash with esptool at 0x0, 0x8000 and 0x10000.
- A crash-looping S3 pulses the H2 reset every boot, so the H2 cannot be flashed and the S3 will not
enter its bootloader over USB. Hold S3 BOOT, tap S3 RST, flash the H2, release, then replug both.
- `dns-sd -L <name> _meshcop._udp local.` shows the `omr` prefix; it disappears when routing is dead.
Keep one capture per chip; never reset the H2 over USB mid-test unless that is the test (the host
logs `Unexpected RCP reset: RESET_OTHER` and recovers).
## Building against a patched LibreTiny
- Clone LibreTiny at the tag ESPHome pins, commit the patch, and set `framework: version: <tag version>` plus
`source: file://<clone>` under `bk72xx:` or `rtl87xx:`. PlatformIO installs it as
`~/.platformio/platforms/libretiny` (the stock copy is `libretiny@src-<hash>` and is not touched).
- A later commit in the clone is not picked up by the next build; patch the installed copy or remove it. Remove
`~/.platformio/platforms/libretiny` when done.
- BDK headers such as `power_save_pub.h` have no `extern "C"`; wrap them when including them from the Arduino
core's C++ files.
## LibreTiny crash and abort() facts (esphome#21596, #22288)
- `abort()` on dev: the BK7238 hangs for good (the main loop stops and the watchdog never fires once Wi-Fi is up;
recover with an RTS pulse on CEN, `serial_capture.py --reset`). The BW15 (RTL8720CF) restarts with no crash
record. The LN882H reboots only after about 18 s through the watchdog (`WDT Reset`), with no record: the SDK's
`abort()` and `_exit()` (`components/libc/newlib_stub.c`) and its FreeRTOS `configASSERT` are spin loops. With
#22288 all three reboot in 1 to 2 s and log `Reason: Abort`, with the PC at the call site and no `LR:` line.
- `assert()`, `std::terminate` and a failed `new` all reach `abort()` (nano libsupc++). `raise(SIGABRT)` returns,
because `_kill` is a nosys stub.
- `__builtin_trap()` faults on every board. A store to `nullptr` or 0xDEADBEE0 does not fault on the BK7238. On
the BW15 a `nullptr` store does not fault, and loads and stores at 0xDEADBEE0 give only imprecise bus faults
(CFSR 0x400, no BFARVALID), so testing the fault address path needs another Cortex-M chip.
- BK7238 safe mode: each quick crash reboot counts toward it (`Last reset too quick; invoke in N restarts`), so
back-to-back crash tests can drop the board into safe mode.
- LN882H: the previous crash is replayed to every new API log client until the next crash; that is expected, not a
double record. Once, the first `__builtin_trap()` after a fresh OTA took about 12 s to reboot; retries took 2 to
3 s and it did not reproduce.
- `~/.platformio/packages/toolchain-gccarmnoneeabi` can vanish between a build and the decode (it comes back with
the next LibreTiny build). Until then the `arm-none-eabi-addr2line` and objdump of
`toolchain-rp2040-earlephilhower` read any ARM ELF.
- RTL8720C: a noreturn `bl abort` can be the last instruction of a function, so the raw return address points at
padding addr2line cannot place. Decode `(addr & ~1) - 2` to get the call line; #22288 records it that way.
- Abort reports (#21596): early heads put LR 0x00000000 first in the address list; later heads drop a zero LR, so
an abort report starts with the backtrace.
## Waveshare ESP32-C6-GEEK
- Native USB JTAG serial (`303a:1001`), 16 MB flash, 1.14" ST7789 LCD and TF card; the vendor docs give no GPIO
numbers. Pins that read a fixed level against the internal pulls: 1, 2, 7, 20, 23 high, 6 low. Free for pad
tests: 0, 3, 4, 5, 18, 19, 21, 22 (see `recipes/gpio_hold.md`).
## Waveshare ESP32-P4-Module-DEV-KIT (USB host)
- ESP32-P4 rev v1.3, 16 MB flash, console on a CH343 (`1a86:55d3`). Other boards use the same chip for their
console, so check with `esptool chip_id`.
- USB host for `usb_host`: set the Host/Device jumper to HOST and use the bottom USB-A port nearest the Ethernet
jack (the high-speed OTG controller); the other USB-A ports and the USB-C marked "USB" do not reach it. The ports
are unlabelled. The port powers the device.
- ESP-IDF programs the FIFO split only when a device connects: read `GRXFSIZ`, `GNPTXFSIZ` and `HPTXFSIZ` at
`DR_REG_USB2_BASE` + 0x24, 0x28 and 0x100 after enumeration. GHWCFG3 (0x4C, high 16 bits) reads 896 lines. The
default split is RX 512, NPTX 256, PTX 128.
- Not every P4 board has a USB host port; check before planning a `usb_host` test on one.
## Gelidus Research PS1 v2.3 (LD2420 presence sensor)
- ESP32-D0WD-V3 rev 3.1 on a CH340 (`1a86:7523`); `esphome upload --device <port>` works directly.
- LD2420 UART: TX GPIO17, RX GPIO16, 115200 (radar firmware v1.6.1). Status LED GPIO23, night light PWM on GPIO25.
- The vendor package is at github.com/GelidusResearch/grps (`packages/grps-ld2420.yaml`); copy it locally for tests.
- The radar needs a person in its field. Energy frames are `F4 F3 F2 F1 23 00 <presence> <range cm LE16> ...`; a
UART `debug:` block on RX shows them.
## ESP32-C3 Super Mini (native USB, 4 MB XMC flash)
- Shows up as `303a:1001`; confirm with `esptool --port <port> chip_id`. Board `esp32-c3-devkitm-1`,
`logger: hardware_uart: USB_SERIAL_JTAG`. Avoid GPIO2, 8 and 9 (strapping; 8 is the LED, 9 is BOOT).
- A 6-pin BME680 breakout (CS, ADDR/MISO, SCL, SDA, GND, VCC) on SDA GPIO6 and SCL GPIO7 with CS and ADDR left
open answers at 0x77 through the breakout's pull-ups, and the `bme680` platform reads fine.
- BME688 parallel mode never reads in `bme68x_bsec2` on dev: the read timer is re-armed by each 30 ms run before
its 41.6 ms delay ends. Entering parallel mode at all needs gas estimates subscribed at `BSEC_SAMPLE_RATE_SCAN`
and no IAQ outputs.
## mDNS responder tests (#22267)
- When the host's multicast does not reach the device, query its port 5353 directly as legacy unicast (a
python-zeroconf packet sent from a normal UDP socket) to dump records or hammer the responder.
- On a crowded Wi-Fi channel, back-to-back query loss swung from 0 to 14% between builds for no code reason. Judge
responder timing on an Ethernet board.
## nRF54LM20 DK (PCA10184, on-board J-Link)
- The power switch must be ON or nothing enumerates. The ESPHome log (uart20) is on the second J-Link VCOM port;
the first is unused.
- Flash with `esphome upload <yaml> --device JLINK` (sdk-nrf toolchain); the LM20 needs no west `--device` override.
The J-Link re-enumerates after the flash, so start the capture after the port comes back.
- Reset without halting: JLinkExe `-device NRF54LM20A_M33 -if SWD -autoconnect 1` with `w4 0xE000ED0C 0x05FA0004`
(plain `r` and `g` halt and resume, no reboot).
- LED0 P1.22, LED1 P1.25; BUTTON0 P1.26, BUTTON1 P1.09, BUTTON2 P1.08, BUTTON3 P0.05 (active low, pull-up). GPIO
P1 registers at 0x500D8200 (IN +0x00C, PIN_CNF +0x080), P0 at 0x5010A200.
- Binary sensor state changes are not logged at DEBUG; log the states or count `on_press` to see button presses.
@@ -1,72 +0,0 @@
# Proven test flows
Each flow names the pull requests it was used on, so their result tables show what to post.
One file per component, named after it:
- [`api.md`](api.md): noise handshake and OTA encryption on the ESP8266, outgoing connection wake timing on deep sleep and after a Wi-Fi drop, user defined action listing and execution with a raw plaintext client, store_yaml recovery over Noise or plaintext
- [`ble_midi.md`](ble_midi.md): BLE MIDI client against a fake ESP32 instrument, every packet shape and the send actions
- [`captive_portal.md`](captive_portal.md): fallback AP and /wifisave hammer
- [`cse7766.md`](cse7766.md): Sonoff S31 parser check over the API
- [`espnow.md`](espnow.md): ESP8266 broadcast receive
- [`hdmi_cec.md`](hdmi_cec.md): two-board wire
- [`http_request.md`](http_request.md): update manifest URL changed at run time, every setter shape, a swap under a running check, non-blocking request actions with loop gap, queue, cancel and reboot checks, OTA request cases with basic auth and MD5 sources
- [`ld2420.md`](ld2420.md): hub setup and sensor states with a simulated module on a UART0 loopback (energy and simple mode)
- [`ld2450.md`](ld2450.md): polygon zones, zone number round trip through the radar's readback, ESP8266 loopback harness without a radar, presence hold and link loss on a frame stream, BT MAC readback
- [`ld2412.md`](ld2412.md): output pin level through the basic configuration command, held back writes
- [`libretiny.md`](libretiny.md): rtc preference emulation, BK72xx and RTL8720C crash report on the next boot with a ROM dump matcher, BK7238 BLE bond store safety with a full flash backup and per partition diff, BK7238 OTA versus the board layout and bootloader
- [`modem.md`](modem.md): PPPoS modem component boot check with no modem attached, loop blocking and disable timing
- [`mqtt.md`](mqtt.md): broker bench, QoS, reconnect, crash log over MQTT, WebSocket transports with a local and a public broker, TLS trust sources (own CA, client certificate, certificate bundle, no verification) with the CLI check, MAC suffix topics and will messages on two boards, shutdown message delivery on restart, disable and deep sleep
- [`nrf52.md`](nrf52.md): dongle board support, zephyr_ble_server connection slot under reordered callbacks, SMP over UDP OTA over Thread with reset faults, MCUboot swap limit and stuck slot recovery, TCP stack probe
- [`online_image.md`](online_image.md): stalled and malformed downloads, PNG, JPEG, BMP and QOI decode time per loop pass with large images and small buffers
- [`openthread.md`](openthread.md): border router on the S3 host plus H2 RCP
- [`ota.md`](ota.md): upload without a configuration and the refusal checks, overlap and lock, timeouts, deflate, upload timing A/B per build and board with deflate and raw, on-device inflate benchmark, OTA automations on two platforms
- [`rtsp_server.md`](rtsp_server.md): camera streaming on the Waveshare ESP32-S3-ETH
- [`scheduler.md`](scheduler.md): loop wake A/B, interval startup delay and control from lambdas
- [`udp.md`](udp.md): datagrams, multicast, sockets, `udp.write` payload shapes
- [`usb_cdc_acm.md`](usb_cdc_acm.md): nRF52840 dongle host driver
- [`wifi.md`](wifi.md): reboot_timeout with a fallback AP, DHCP lease cache, Easy Connect (DPP), WISP bridge with an ethernet upstream, reboots against a WPA2/WPA3 PMF access point, WISP and ALWAYS_ON AP switching, AP restarts with WPA3 mixed and LIGHT power save
- [`zephyr.md`](zephyr.md): preferences on the nRF52840 dongle
- [`zephyr_coredump.md`](zephyr_coredump.md): crash and decode on nRF52 boards
- [`zwave_proxy.md`](zwave_proxy.md): frame checksums on a live proxy
- [`globals.md`](globals.md): restoring globals with storage rtc and flash across every reset kind
- [`homeassistant.md`](homeassistant.md): light platform mirrored from a local Home Assistant, held reports versus later calls
- [`usb_audio.md`](usb_audio.md): USB audio speaker and microphone, the usb_host FIFO split, and Music Assistant playback through sendspin
- [`dfrobot_sen0395.md`](dfrobot_sen0395.md): SEN0395 mmWave command flow, module quirks and overlong lines through its error reply
- [`speaker.md`](speaker.md): media player pipeline tasks on the Connect AUX-2 with the `speaker` platform player
- [`web_server.md`](web_server.md): state SSE stress and the HTTPD task stack on web_server_idf, entity icon and device class readback
- [`uart_sim.md`](uart_sim.md): UART sensors simulated on a classic ESP32 with loopback and cross-wired UARTs
- [`gpio_expander.md`](gpio_expander.md): interrupt driven expander inputs, loop idle checks, testing a device out of reach
- [`led_strip.md`](led_strip.md): forcing a failed strip allocation, RMT symbols per chip, loop cost per strip
- [`remote_base.md`](remote_base.md): remote_receiver protocol decoding through a one-pin loopback on ESP32 and ESP8266, no IR hardware
- [`xiaomi_ble.md`](xiaomi_ble.md): Xiaomi, ATC and PVVX parsing by injecting scan results into the ESP32 tracker or advertising them from a second ESP32, real AES-CCM frames, duplicate filter model
- [`ble_terminate.md`](ble_terminate.md): proxy disconnect against a peer with peripheral latency
- [`i2c_slave.md`](i2c_slave.md): fake I2C slave on a D1 mini, ESP32 hardware master as the control
- [`fake_i2c.md`](fake_i2c.md): fake I2C devices with no chip: in-firmware buses for many drivers, a Pico 2 W BME280/BMP280 target, a DS2482-800 simulator, NACK injection for vl53l0x
- [`gpio_hold.md`](gpio_hold.md): GPIO hold through the ISR pin API, pad read back with no wiring, per chip pin notes
- [`ble_event_inject.md`](ble_event_inject.md): made up GAP and GATTC events fed into the ESP32 BLE handlers from a button
- [`light_sleep_gpio_wake.md`](light_sleep_gpio_wake.md): GPIO wakeup from auto light sleep with an LEDC wave that keeps running in sleep, and with an external wave and peripheral power-down on the ESP32-P4
- [`manual_ip_and_web_handlers.md`](manual_ip_and_web_handlers.md): station, AP and Ethernet manual IP, web handlers added late and web OTA with web_server, captive_portal or prometheus
- [`ble_server_security.md`](ble_server_security.md): esp32_ble_server pairing, encrypted and MITM characteristics, subscription before and after pairing, bond reuse per ESP-IDF version
- [`prometheus.md`](prometheus.md): date, time and datetime metrics in UTC and another time zone
- [`ble_adv_triggers.md`](ble_adv_triggers.md): BLE service and manufacturer data advertise triggers over the air on the ESP32 and BK72xx trackers, with a second ESP32 advertising known frames
- [`sun.md`](sun.md): on_sunrise and on_sunset by sweeping the observer's longitude, one full sweep in about 17 minutes
- [`sensor_filters.md`](sensor_filters.md): every sensor, binary_sensor and text_sensor filter shape fed a fixed input sequence, outputs compared bit for bit between builds
- [`led_driver_sniff.md`](led_driver_sniff.md): bit-banged LED drivers captured on the ESP32's own pads with SPI slaves through the GPIO matrix, no chips
- [`hub_input_sim.md`](hub_input_sim.md): matrix_keypad, wiegand, rdm6300 and key_collector driven from a D1 mini's own pins and UART loopback
- [`value_range.md`](value_range.md): sensor and number on_value_range with a fixed sequence and a restart
- [`ezo_fake_bus.md`](ezo_fake_bus.md): the ezo component against a fake I2C bus that answers like an EZO circuit
- [`twai_self.md`](twai_self.md): esp32_can on_frame with the controller hearing its own transmit pad, no transceiver
- [`buttons.md`](buttons.md): output button pad timing, homeassistant button through a raw API client with the wizard input, uart button on the UART0 loopback
- [`rs485_frame.md`](rs485_frame.md): DLE/STX/ETX bus on an ESP32-C3 with a cross-wired simulated bus device (frame cases, gate timing)
- [`hx711.md`](hx711.md): fake HX711 driven from the other core, every published value matched to the served conversion
- [`automation_trace.md`](automation_trace.md): automation codegen A/B, a marker trace of chains, conditions, loops and script modes over the API
- [`bluetooth_proxy.md`](bluetooth_proxy.md): held GATT link after a half-open API session, frozen Home Assistant stand-in
- [`crash_report.md`](crash_report.md): crash buttons, fetching the previous boot's crash report over the API, per-chip abort facts
- [`https_idf.md`](https_idf.md): web_server_idf HTTPS test component, listener switch, stalled TLS clients, bad key
- [`mdns.md`](mdns.md): legacy unicast mDNS dump, hammer and latency straight to the device
- [`sim_buses.md`](sim_buses.md): PCF8574/HD44780, MAX7219, SPI spy, packet_interface and SN74HC165 simulators, status LED probe, fake Home Assistant, cg.static_buffer probes
- [`sigma_delta.md`](sigma_delta.md): pulse density of the sigma_delta speaker pin read inside the ESP32-P4, rtttl note onset and release
- [`microphone_http_stream.md`](microphone_http_stream.md): raw PCM over HTTP with a counter fake mic or a USB mic, multi-reader client
- [`i2s_audio.md`](i2s_audio.md): I2S microphone fed by an I2S TX channel on the same S3 through three jumpers
- [`voice_assistant.md`](voice_assistant.md): late TTS start on the Voice PE with a slow announcement URL
@@ -1,58 +0,0 @@
# api
## Noise handshake and OTA encryption on the ESP8266 (#19062, #19233)
Host side timing is useless on a weak link; use an on-device bench with the minimum of five runs over
two boots. Provision a key with `noise_encryption_set_key` through aioesphomeapi and upload with
`espota2.run_ota(host, 8266, None, bin, noise_psk=key)` when the key is not in the yaml.
## Outgoing connection wake timing on a deep sleep build (#21077)
`templates/deep-sleep-outgoing-dut.yaml`: classic ESP32 ESP-IDF, `wifi: fast_connect`, `api: encryption` with a bare
`outgoing_connection:`, `deep_sleep` 15 s run and 15 s sleep, a `homeassistant` text sensor, and `millis()` logged
from `wifi: on_connect`, `api: on_client_connected` and the sensor's first `on_value`. Home Assistant's dial-in
listener is on 6054; reuse a device name it already has. Flash by USB, `serial_capture.py PORT 360 OUT --reset` for
12 cycles, then `scripts/api_outgoing/cycles.py OUT LABEL` prints per cycle the wifi, client and state millis with
the client-wifi and state-client gaps. Wi-Fi takes 4 to 8 s after a wake, so compare the client-wifi gap, and run
the PR twice around one base run because the spread is a few hundred ms.
Results: https://github.com/esphome/esphome/pull/21077#issuecomment-6254098014
## Outgoing connection on a mains build after a Wi-Fi drop (#21085)
`templates/outgoing-wifi-drop-dut.yaml`: classic ESP32, `outgoing_connection: delay: 2s`, a "Wifi cycle" button doing
`wifi.disable`, 4 s, `wifi.enable`, a `debug` loop-time sensor, millis from the wifi and api triggers. Flash by USB,
`serial_capture.py PORT SECONDS OUT --ms`, press the button N times 40 s apart with `api_button.py DEV_IP KEY press
"Wifi cycle"`, then `wificycles.py CAPTURE LABEL` prints per drop the Wi-Fi-to-dial gap, dials and dials closed, who
connected first, and the loop-time maxima. For the loop cost of polling, repeat with an unreachable
`host: 192.0.2.1`. Home Assistant refuses a dial-in while its own session is READY (`Refusing dial-in adoption`),
so after a drop every dial is closed and HA's incoming reconnect wins 11 to 33 s later; the measurable effect is the
first dial landing in the loop the network flag flips.
Results: https://github.com/esphome/esphome/pull/21085#issuecomment-6254742791
## User defined action listing and execution without a client library (#18882)
`scripts/api/rawactions.py HOST list` speaks the plaintext native API over a raw socket and decodes every
`ListEntitiesServicesResponse` by field number (argument fields 1 name, 2 type, 3 description, 4 example, 5 optional,
6 default_value), so new wire fields show up without waiting for the client library. `rawactions.py HOST exec ACTION
k=v ...` sends `ExecuteServiceRequest` with `sint32` ints (field 5, not the legacy int32 field 2), fixed32 floats,
length delimited strings and `int[]` as repeated sint32 (field 7); `ALLOW_MISSING=1` leaves arguments out on purpose
(the device then drops the call). `templates/api-actions-dut.yaml` (D1 mini, three actions, optional variables with
defaults) and `templates/api-actions-strings-dut.yaml` (required strings) log what arrived. String arguments are
`StringRef`s that are not null terminated: copy them with `snprintf("%.*s", size, c_str())` into a buffer before
logging, and do not pass `%.*s` to `logger.log`, whose validator counts it as one pattern.
## store_yaml recovery over the native API (#16445)
`scripts/api/syrecover.py HOST KEY OUTDIR [ORIGDIR]` (KEY `-` for plaintext) builds the client library's
`APINoiseFrameHelper` around a stub `APIConnection` subclass (the compiled helper type checks the connection, and
`ConnectionParams` needs every field), sends Hello, Authentication (never answered without a password) and
`DeviceCapabilitiesRequest`, then `GetYamlRequest` (161), collects `GetYamlResponse` (162) frames by field number
(1 data, 2 done, 3 total_size, 4 encoding) until done, requests again, decompresses with `compression.zstd` (Python 3.14, or the `zstandard` package) and
unpacks the envelope with the tree's `unpack_envelope` (run with `PYTHONPATH` on that tree). Configs in
`templates/store_yaml/`: entry files in `boards/`, a package one directory above them with `!secret` and an inline
OTA password,
so the redaction and the `..` paths are exercised. Repetitive YAML compresses into one chunk even at 22 KB; a 6 KB
include of random base64 lines with `include_secrets: true` gives four chunks (1390 byte packets). Rough edge at
0bd3738430: `include_secrets: true` fails the build when a package in another directory uses `!secret` and has no
`secrets.yaml` beside it; a copy next to the package works around it.
Results: https://github.com/esphome/esphome/pull/16445#issuecomment-6335435397
@@ -1,24 +0,0 @@
# Automation code generation A/B (#21978 to #21980)
A change to how actions, conditions or scripts are generated must not change what runs or in which order.
`scripts/automation_trace/automation-trace-common.yaml` (a package) logs a marker `M <case> <values>` at every step.
It covers:
- merged action chains, delays inside chains, nested `if`/`else` with and/or/not/xor, `choose` with `else`;
- `while` with and without a delay, nested and dynamic `repeat`, `wait_until` and `for:`;
- `single`, `restart`, `queued` and `parallel` scripts with parameters, a self `script.stop` mid-chain;
- `on_value_range`, alarm panel triggers, and parallel scripts with `max_runs` 2, 3 and 4 that execute themselves
from an `if` branch.
`run.sh LABEL` writes the per-run file (board, Wi-Fi from `secrets.yaml`, the package), builds and flashes over
`PORT`, takes the device address from the serial log and runs `drive.py HOST OUT`. The driver records the boot run,
a "Run All" press and API-driven triggers (Chain Button twice, Num 70 and 20, Sw on and off, Sel c and a). The
runner keeps the part from `DRIVER press Run All` on in `out-at<LABEL>.trace`; diff dev against each PR head. The
boot run is partly missed while the API connects, so it is left out of the diff.
- Pass: the Run All pass reaches `M done`, no `SHOULD_NOT_RUN` or `NOT_REACHED` marker appears, and the trace is the
same on both builds.
- The interleaving of the `queued` and `parallel` scripts with later steps is timing dependent, but it was identical
across builds and runs.
- To check a real device config as well, OTA the PR build and toggle one light and one switch from an API client,
returning each to its original state, and compare states and logs with the base build. Get the owner's approval
before flashing a device that is in use.
@@ -1,28 +0,0 @@
# BLE service and manufacturer data advertise triggers (#21822, #21823, #21827)
`scripts/ble_adv_triggers` checks `on_ble_service_data_advertise` and `on_ble_manufacturer_data_advertise` over the
air on the ESP32 and BK72xx trackers.
- Advertiser: `advtrig.yaml` on any second ESP32 with BLE. Every 250 ms it switches to the next of six raw
non-connectable advertisements (`adv_frames.hpp`, random static addresses `C4:C1:38:00:00:21` to `26`):
| Address | Contents |
|---|---|
| `...:21` | service data 181C `AA01` and manufacturer data 02E5 `BB01` |
| `...:22` | service data 32-bit 12345678 `AA02` |
| `...:23` | service data 128-bit 12345678-9ABC-DEF0-1122-334455667788 (wire order) `AA03` |
| `...:24` | manufacturer data 181C `BB04` (the service UUID of `...:21`, other data kind) |
| `...:25` | two service data 181C entries, `AA05` then `AA06` (only the first may fire) |
| `...:26` | the 128-bit UUID in printed byte order, `AA07` (must not match) |
- Receivers: `rx-esp32.yaml` (classic ESP32, passive scan) and `rx-bk72xx.yaml` include the same eleven triggers
(`triggers.yaml`). `rx.hpp` counts each trigger's hits by payload (`AAnn`, `BBnn`, anything else as `other`) and
logs the table every 20 s, plus all adverts and those from the advertiser.
- `run.sh esp32|bk72xx LABEL` builds, flashes, captures 150 s and runs `check.py`, which prints PASS or FAIL. Run it
once per build (dev merge base and each PR head). About 550 hits per trigger on ESP32 and 170 on a BK7238 in that
time.
- `mac_address: 00:00:00:00:00:00` meant no filter on the ESP32 tracker before #21823 and filters on that address
after it (BK72xx always filtered). Pass `ZERO_MAC=any` for an ESP32 build from before #21823.
- BK7238: with no `wifi:` in the config the tracker logs `Scan started` but receives nothing at all, on dev too. The
BK config therefore joins Wi-Fi.
- Results: https://github.com/esphome/esphome/pull/21827#issuecomment-6489703755
@@ -1,30 +0,0 @@
# Injecting BLE GAP and GATTC events (#21698)
Some BLE paths (repeated connection parameter update failures, a disconnect in the middle of them) are hard to get
from a real peer on demand. `scripts/ble_event_inject` feeds made up events straight into the ESP32 BLE handlers.
- `ESP32BLE::gap_event_handler` and `gattc_event_handler` are protected statics and the class is `final`.
`cp_inject.hpp` reaches them through explicit template instantiation, where access checks do not apply. Its
`cpinject::run()` sends six labelled steps (`STEP1` to `STEP6`): four peers failing five times each, a success
followed by more failures, and disconnects between failures.
- `cp-inject.yaml` is a package for a copy of a proxy config: it adds the include, a VERBOSE logger with the busy
tags back at DEBUG, and an `Inject conn param events` button. Press it with `scripts/api_button.py` while a
log capture (`esphome logs ... --device <ip>` under `timeout`) runs, then count the warning and verbose lines per
step on the PR and on dev.
- The handlers push into a queue and event pool that take a single writer, normally the Bluetooth task. Injecting
from the main task while real events arrive can corrupt them, so the button stops the scan, waits for it to end,
injects and starts it again. Run it with no active proxy connections (no Home Assistant BLE client holding one).
- Fake disconnects use `gattc_if` 0xFE: every client and proxy connection filters on its own `gattc_if_`, so the
queued event reaches only the shared handler. Do not use 0xFF (`ESP_GATT_IF_NONE`), which every client accepts.
- The fake peers (`C0:FF:EE:00:00:0n`) are never connected, so a live proxy keeps serving its real connections;
put it back on its own config afterwards.
## ThermoPro advertisements as scan results (#22047, #22054)
`scripts/ble_event_inject/tpinject.hpp` holds the advertisement records from the `thermopro_ble` unit tests (TP357,
TP357S, TP393, TP960R, TP970R, TP972S, TP25, plus invalid and 13 byte probe frames) and `tpinject::inject_all()`
feeds each one to `global_esp32_ble_tracker->gap_scan_event_handler()` with the device name and the manufacturer
data, the way `scripts/xiaomi_ble` does. Frame N comes from `A4:C1:38:00:10:0N` (up to `:10`), so give each
`thermopro_ble` sensor that MAC, include the header and call `tpinject::inject_all();` from an `interval:` or a
template button. Compare the published states (or the sensor DEBUG lines) on the PR and on dev; no ThermoPro
device is needed.
@@ -1,19 +0,0 @@
# ble_midi
## BLE MIDI client against a fake instrument (#18508)
No MIDI hardware needed: `templates/ble_midi/midiperi.yaml` turns a second ESP32 (an S3 with ethernet, so the
API stays reachable while BLE is busy) into a BLE MIDI peripheral with `esp32_ble_server`: the MIDI service
advertised, the MIDI characteristic with read, write, write without response and notify, an `on_write` that logs
the bytes, buttons that notify hand built packets (channel messages, two messages in one packet, running status,
SysEx in one packet, split over two, oversize, cut short, clock, aftertouch, orphan data bytes, a 50 packet burst)
and a restart button for the disconnect test. `midicli.yaml` is the component test config on a classic devkit with
`wifi:`, `api:`, the peripheral's BLE address in the `peri_mac` substitution, given on the command line with
`-s peri_mac <mac>` (the base MAC plus 2, printed by `esp32_ble` in `dump_config`), `on_sysex` and `on_message` lambdas that print the bytes, debug sensors and out of range send
buttons. Drive both over the plaintext API with `scripts/modem/mdrive.py HOST LOGFILE press:NAME wait:S` 3 s
apart and compare the peripheral's `TX` and `RX` lines with the client's decoded lines. Gotcha:
`ble_server.characteristic.set_value` latches the value of its first play and re-applies it before every notify,
so the peripheral script sets the value from a lambda (`id(midi_io).set_value(std::vector<uint8_t>(pkt));
id(midi_io).notify();`). At VERBOSE the client logs four lines per message, so a burst shows an
`esp32_ble took a long time` of a couple of seconds that is the serial port, not the parser.
Results: https://github.com/esphome/esphome/pull/18508#issuecomment-6348272717
@@ -1,67 +0,0 @@
# esp32_ble_server pairing and encrypted characteristics (#17562)
`scripts/ble_server_security`: a classic ESP32 devkit runs the secure GATT server (`server.yaml`) and a second ESP32
(an S3 here) runs `ble_client` as the central (`client.yaml`). The server has a `read_encrypt` battery level
(`99.9`), a plain characteristic, a `write_encrypt` characteristic that logs what it receives, a `read_encrypt_mitm`
characteristic and a `read_encrypt` characteristic that notifies every 2 s. `run.sh TAG IO AUTH` builds both with the
pairing mode as substitutions, pairs once, reconnects twice, and `summary.py` prints PASS or FAIL. The server needs
`esp32_ble_server` security from #17562 or later: point `SERVER_TREE` at that checkout (the client builds from
`TREE`, dev is fine). `summary.py` needs each refused read to show an `Error reading char` reply, so a run where the
client never connected fails instead of passing.
- `ble_client` never starts pairing on its own (a phone does, after an `insufficient authentication` reply); the
client's `Pair` button calls `BLEClientBase::pair()`. Read errors show as `Error reading char at handle N, status=5`
(not authenticated: no bond) or `status=15` (not encrypted: bond known, link not encrypted yet).
- The `ble_client` text sensor subscribes once, at connect, before any pairing, so a protected characteristic refuses
it. The `Subscribe` button writes the CCCD again after pairing; `ble_client` drops its service list after discovery,
so it writes the characteristic handle + 1 (where ESP-IDF's GATT server puts the CCCD).
- Numeric comparison (`display_yes_no` with `sc_mitm_bond`) tells a fresh pairing from a bond reuse: the
`numeric comparison` line only appears on a fresh pairing. Just Works cannot show it.
- ESP-IDF 5.5.6 drops the bond on the second disconnect on both roles (`Deleting unpaired device`), so that reconnect
pairs from scratch; 5.5.5 keeps it (`-s idf 5.5.5`).
- The server keeps its bonds in NVS. One holding an authenticated bond for the client refuses a Just Works re-pairing
silently; erase the devkit's flash before switching from nc to jw.
- `summary.py` failed both modes at 58636914ab: `read_encrypt_mitm` was readable with no pairing, because
`BLEService::create_characteristic()` took an 8 bit `esp_gatt_char_prop_t` and dropped the MITM bits (1 << 8 and
1 << 9). Check the generated `create_characteristic` call and the property type when a flag above bit 7 is added.
## ESP32 central against the secure server (#17562, #22081)
`scripts/ble_server_security/esp32central-*` swaps the roles: an ESP32-S3 runs the secure server
(`esp32central-srv.yaml`: plain, encrypted, MITM read and write, and an encrypted notify characteristic, numeric
comparison accepted automatically) and a classic ESP32 devkit runs `esp32central-cli.yaml` (`ble_client` with
`display_yes_no` and `sc_mitm_bond`, text sensors that log every value, a MITM write and a link cycle every 40 s).
`bonds.hpp` logs the bond list Bluedroid loaded from NVS on both boards.
- The server never starts pairing (a peripheral only answers a security request), and `ble_client` does not start
it on an insufficient authentication reply (status 5) the way Android does. The client's `on_connect` calls
`esp_ble_set_encryption(remote_bda, ESP_BLE_SEC_ENCRYPT_MITM)` instead.
- `esp32central-flow.sh TAG` erases both boards, flashes them, captures a fresh pairing with three link cycles
(phase A), then resets both together so each loads its bond (phase B), and prints per phase the connects, numeric
comparisons (a fresh pairing), MITM reads, server writes and `Deleting unpaired device` lines. It passes when
phase B reads the MITM characteristic with no numeric comparison and no `Deleting unpaired device` line.
- On ESP-IDF 5.5.6 every other reconnect pairs again (`Deleting unpaired device` on the disconnect after a bonded
reconnect), the regression #22081 describes.
## Bluedroid hex parsing bench (#22081)
`scripts/bluedroid_hex_bench/`: `hexbench.hpp` times Bluedroid's `sscanf` parsing (`string_to_bdaddr` and the
`btc_config_get_bin` loop) against the parsers in #22081's `esp32_ble/bluedroid_hex.h` on the same inputs, valid and
invalid, and checks that the outputs match. `run.sh LABEL` (env `TREE`, `PORT`, `SC`, `PY`, `BOARD`) builds
`bench.yaml`, flashes, captures 25 s and fails on any mismatch or a missing `DONE`. Classic ESP32: about 7 to 9 times
faster for addresses, 6 to 7 times for 16 and 28 byte keys.
## ble_client on_notify with a real server (#21821)
`scripts/ble_notify/`: `server.yaml` (ESP32-S3 GATT server; a numeric and a text characteristic notify every 2 s,
with uint16 frame and empty notification rounds, set through `id(c)->set_value(...)` and `notify()` in a lambda) and
`client.yaml` (classic ESP32 client with two `on_notify` triggers per sensor, one of them delayed, and a link cycle
every 60 s). `run.sh LABEL` (env `TREE`, `SERVER_PORT`, `CLIENT_PORT`, `SERVER_BLE_MAC`, `SC`, `PY`) flashes both,
captures the client 150 s and counts `on_notify` against `on_value` per sensor.
- `run.sh` also requires `All clients established, services released`: before #21821 it never appears with a text
sensor `on_notify`. The delayed txt `on_notify` may miss only the last notify of the capture.
- The server takes one connection. Park any other client that would connect to it with
`esptool --after no-reset chip_id` (the board stays in the bootloader).
- An ESP32-C6 devkit whose native USB port refuses esptool can be flashed and logged through its UART bridge port
with `logger: hardware_uart: UART0`.
@@ -1,27 +0,0 @@
# BLE terminate with peripheral latency (#20232)
`scripts/ble_terminate`: an ESP32-S3 `bluetooth_proxy` (active) over Ethernet as the central (`btproxy.yaml`) and a
classic devkit `esp32_ble_server` peer (`btpeer.yaml`). The peer asks for 50 ms / latency 10 / 6 s with
`esp_ble_gap_update_conn_params` 4 s after connect; earlier requests are overwritten by the proxy's own "medium
conn params" after service discovery. Read the proxy's `Conn params updated` line (VERBOSE, API logs) to confirm;
the peer side never logs the event. The proxy only connects to a device its scanner has seen, so `btdrive.py`
subscribes to raw advertisements first.
Measure the peer's own disconnect (`serial_capture.py --ms`) after the client's disconnect: about 0.2 to 0.8 s when
the terminate arrived, about 6 s (the supervision timeout) when it was lost. The client side disconnect time is the
API reply and says nothing. On dev the S3 lost 6 of 12; the PR lost 0 of 16.
`run.sh LABEL ROUNDS` writes `central_bda.hpp` from `PROXY_BT_MAC`, builds and flashes both boards, runs the rounds
and `an.py`, which prints PASS only when every round completed, every peer disconnect came within 3 s and the last
parameter update the proxy logged before each disconnect had latency=10 (a round without it did not test peripheral
latency and counts as inconclusive).
Controller build check: ESP-IDF 5.5.6 ships `libbtdm_app.a` byte for byte the same as 5.5.5 for esp32c3 and esp32s3
(build 51d9dfd). Without hardware:
```bash
strings -a <idf>/components/bt/controller/lib_esp32c3_family/<chip>/libbtdm_app.a | grep -E '^[0-9a-f]{7}$'
```
On a board: a button lambda logging `btdm_controller_get_compile_version()` (declared `extern "C"`) over the API,
since the ESP-IDF init line only reaches the UART.
@@ -1,28 +0,0 @@
# bluetooth_proxy
## Held GATT link after a half-open API session (#21256, #22077)
When Home Assistant stops answering without closing its socket, the proxy must not keep the BLE link it opened for
that session. `scripts/bluetooth_proxy_half_open/` reproduces it with:
- `proxy.yaml`: the proxy under test on a classic ESP32 devkit (ESP-IDF, VERBOSE logger with the busy tags at DEBUG,
encrypted API). Build it from the tree under test with a `secrets.yaml` (`wifi_ssid`, `wifi_password`, `api_key`).
- `peripheral.yaml`: a second ESP32 (an S3 here) as a peripheral that only advertises while nobody is connected
(`esp32_ble_server` takes one client by default), with manufacturer data `FF FF 21 56` so the client can find it.
- `ha.py`: the Home Assistant stand-in (aioesphomeapi, `client_info` "Home Assistant test"). Roles: `owner` connects
to the peripheral through the proxy and idles, `observe` counts the peripheral's adverts and the slot report and can
ask for a connect at the end, `swap` connects at once and retries on the first advert. Every role stays subscribed
to advertisements: with no subscriber the proxy's loop disconnects every link at once, which hides the bug (an
early run failed with `disconnect while backend idle` for that reason).
`cases.sh PROXY_HOST LABEL` (env `PROXY_PORT`, `PERI_PORT`, `SC`, `PY`, `API_KEY`) captures both boards and runs
case C (old client killed, new client after 4 s), case H (old client frozen with `SIGSTOP`, so the socket stays open
with no FIN; new client after 4 s, watched 300 s, then it asks to connect and releases) and case N (frozen and nobody
reconnects for 200 s). `cases.sh PROXY_HOST LABEL swap` instead repeats H three times with the new client
connecting at once (`ha.py ... swap ADDR`) and retrying on the first advert. Both end with PASS or FAIL: the cases
fail when an observer exits non-zero, never sees the peripheral advertise (the held link) or the proxy resets, and
swap fails when a round never connects. An owner that never got the link also fails (a rig problem, not a result).
- The proxy drops a frozen session as unresponsive after about 144 s.
- Before #22077, case H holds the link for good: 0 adverts, and the address stays in the slot report. With #22077 the
link is released in the same millisecond as the `replaces` warning. In case 4 the first request logs `Connection
request ignored, state: DISCONNECTING` (connected=false at 0.2 s) and the retry connects at 0.5 to 0.7 s.
@@ -1,46 +0,0 @@
# buttons
Button platforms converted to `button::ApplyButton` change no behaviour on the wire, so each test compares what a
press does on dev and on the PR: the pad, the action call or the bytes sent. All three run on a D1 mini with no
wiring and no other device.
## output button pad timing (#21907)
`scripts/output_button`: two `gpio` outputs on GPIO5 and GPIO4 and three output buttons (500 ms and 1000 ms on
GPIO5, 200 ms on GPIO4). `edges.hpp` attaches a CHANGE interrupt to each output pad and stamps every edge with
`micros()`; on ESP8266 a pad reads back its own output level. A script runs four cases three times: one press, a second
press 250 ms in (the reset restarts, high 750 ms), both outputs at once, and the 1000 ms button then the 500 ms
button on the same output 300 ms later (low at 800 ms). `run.sh LABEL` flashes, captures 45 s and prints the edge
lines; diff dev against the PR. Each button keeps its own reset timer, so the 1000 ms button's reset fires later on a
pad that is already low and shows no edge.
Results: https://github.com/esphome/esphome/pull/21907#issuecomment-6516597178
## homeassistant button through a raw API client (#21908)
`scripts/homeassistant_button/ha-button-common.yaml` has three `homeassistant` buttons: `button.doorbell`,
`input_button.chime`, and one with no `entity_id` that is an `api: wizard:` input. An `on_boot` press runs before
any client connects. `habdrive.py HOST` is a plaintext API client that subscribes to Home Assistant actions, presses
each button and prints every action call (service and data). It sets the wizard input with `WizardInputSetRequest`
(message 158, key = FNV-1 hash of the input's id), first to an `input_button`, then to a `button`, then to an invalid
value the device must reject. `run.sh LABEL` flashes, captures and drives it. Expect `button.press` or
`input_button.press` per entity, `No entity ID set yet` and no call before the wizard input is set, and `No clients
connected` from the boot press.
- `homeassistant` entities are internal by default, so the client sees no buttons unless they set `internal: false`.
- Wait for each reply type instead of a fixed time: the hello reply can take more than a second, and a fixed wait
shifts every later reply by one step.
- Results: https://github.com/esphome/esphome/pull/21908#issuecomment-6516693220
## uart button on the UART0 loopback (#21906)
`scripts/uart_button/uart-button-common.yaml` puts four uart buttons on UART0 of a D1 mini: a 10 byte string, five
raw bytes with `0x00` and `0xFF`, the same string again (shares one generated function on the PR), and 64 bytes. It
sets the UART0 loopback bit in `on_boot`, so every press comes back on RX, and `uart: debug: direction: BOTH` logs
both sides. `run.sh LABEL HOST` flashes by USB and runs `ubdrive.py`, which presses each button 3 times over the API
and saves the log. Every `>>>` line must equal the configured payload and its `<<<` line.
- Set the loopback bit at `on_boot` priority -100. At 800 the UART setup runs afterwards and clears it.
- Nothing else reads the bus, so the debugger needs `dummy_receiver: true` to log RX.
- UART0 is the bus, so `logger: baud_rate: 0` and logs go over the API. The device address comes from DHCP or mDNS.
- An RX line sometimes lands after the next press marker when the log stream lags. That is not a byte difference.
- Results: https://github.com/esphome/esphome/pull/21906#issuecomment-6516762337
@@ -1,21 +0,0 @@
# captive_portal
## Captive portal `/wifisave` hammer (#20802, #20784)
Reproduces a `/wifisave` arriving during the STA cooldown, and proves a fix on every board.
Device under test: `templates/example-dut.yaml` (bogus network, `ap:` with `ap_timeout: 15s`,
`captive_portal:`). Client: `templates/example-client.yaml` on an ESP32 or a D1 mini; it joins the
device's AP, posts 15 bogus saves 3 s apart, then the real credentials. With the portal up the cooldown
is 30 s, so nearly every save lands inside one. Driver: `scripts/wifisave_hammer.sh`. Settle 40 s on
ESP boards, 90 s on the BW15 (its first attempt on the real network fails, see boards.md).
Counts that matter: `Requested WiFi Settings Change` (saves received), `Exiting cooldown early`, crash
markers, boots (`Log initialized`), and the client's `bogus save status 200` and `real save status 200`.
A healthy run: every save answered 200, one cooldown exit per save (two on the BW15, whose retry enters
the adapter restart), no panic, and the real credentials connect the device within about 10 s.
Fault injection for AP start failures: wrap `this->ap_setup_ = this->wifi_start_ap_(this->ap_)` in
`setup_ap_config_()` with an `HWTEST_AP_FAIL_COUNT` build flag
(`esphome: platformio_options: build_flags: -DHWTEST_AP_FAIL_COUNT=2`), raise the portal's "Captive
portal started" from LOGV to LOGI, and after provisioning curl the device's STA address: connection
refused proves the portal's server is gone.
@@ -1,26 +0,0 @@
# Crash report over the API (#21596, #22288)
#21596 lets an API client fetch the crash recorded on the previous boot; #22288 records `abort()` as a crash on
LibreTiny. `scripts/crash_report/crash-common.yaml` (a package: Wi-Fi from `secrets.yaml`, plaintext API, OTA,
DEBUG logger) adds three buttons: `crash abort`, `crash null store` and `crash div zero`. The per-run file supplies
the board and the name. Flash it, then run the drivers (they import `scripts/apiconn.py`, so run them in place or
put `scripts/` on `PYTHONPATH`):
- `drive.py HOST BUTTON_OBJECT_ID` (for example `crash_abort`) reads the capability, presses the button, reconnects
after the reboot and fetches the report twice (reason, PC, cause, fault address, build time, per-core backtrace),
subscribes to logs with `dump_config` to see what is replayed, and checks the capability from a second client.
It reads an earlier report first, so a stale one cannot pass. It exits 1 when the device still answers after the
press, does not offer a report after the crash, or the report has no reason or a PC of 0.
- `peek.py HOST` prints only the capability and how many replayed `CRASH DETECTED` lines a new log client gets. To
only press a button, use `scripts/api_button.py HOST - press "crash abort"`.
- Decode the PC and the backtrace with the build's ELF (`addr2line -e firmware.elf`). On RTL8720C a noreturn
`bl abort` can be the last instruction of a function, so the return address points at padding: decode
`(addr & ~1) - 2` to get the call line.
- `__builtin_trap()` faults on every board. A store to `nullptr` does not fault on every chip (BK7238 and RTL8720C
ignore it), so the `crash null store` button proves nothing there.
- With #22288, `abort()` reboots BK7238, RTL8720C and LN882H boards in 1 to 2 s and logs `Reason: Abort` with the PC
at the call site. Before it, the BK7238 hangs until reset, the RTL8720C restarts with no record and the LN882H
waits about 18 s for its watchdog.
- Quick crash reboots count toward BK72xx safe mode, so space out back-to-back crash tests on those boards.
- A device that keeps the previous crash replays it to every new log client until the next crash; that is one
record, not two.
@@ -1,11 +0,0 @@
# cse7766
## cse7766 on a Sonoff S31 (#20948)
`templates/s31-cse7766-dut.yaml`: `logger: baud_rate: 0` because the CSE7766 owns the UART, VERBOSE logger
with everything but cse7766 quieted, 2 s throttles, and a `globals` frame counter fed by the voltage sensor's
`on_raw_value` and logged every 10 s (about 96 frames per 10 s). OTA only, every reflash reboots the relay.
`scripts/cse7766/run_obs.sh IP SECONDS LABEL [OUTDIR] [NAME_FILTER]` subscribes to states and logs over the
API with a concurrent ping and prints the frame counter, API gaps and warnings. API state gaps of 10 to 20 s
can appear on a weak Wi-Fi path while the device counter stays flat, so judge a UART parser by the
device-side counter, not by the API cadence. With nothing plugged in, current and power read 0.
@@ -1,21 +0,0 @@
# dfrobot_sen0395
## DFRobot SEN0395 mmWave on an ESP32 DevKit V1 (#20906, #20979)
Wiring: V to VIN (5 V, the module wants 3.6 to 5 V), G to GND, sensor TX to GPIO16 (RX2), sensor RX to GPIO17
(TX2), IO2 to GPIO18 (presence output, a gpio binary sensor), I1 unconnected. `templates/dfrobot-sen0395-dut.yaml`
has the hub, both detection binary sensors, the four switches, settings buttons and junk-line buttons.
`scripts/dfrobot_sen0395/ctl.py HOST KEY press:NAME sw:NAME=on|off wait:S` drives it and prints binary sensor and
switch updates; `cases.sh DEV_IP KEY CAPTURE LABEL` runs the whole flow against a running serial capture. Flash by
USB, `serial_capture.py PORT 540 CAPTURE --reset`, then the cases; compare the `Sending command` and `Message:` lines
and the counts of `Done`, `Error`, prompts and frames between builds. The module echoes every command, so the echoed
lines are the bytes the component sent and can be compared between builds as a set.
Module facts: it answers `Done` or `Error` and then the `leapMMW:/>` prompt without a line end; a junk line gets
`<line> is not recognized as a CLI command` (36 chars longer than the line, so 220 and 240 byte junk gives 256 and
276 char replies); input lines of 256 bytes or more make the module assert (`..\User_code\main.c line:232`) and
restart, after which the sensor is stopped; a factory reset restarts the module about 7 s later; `sensorStop` on a
stopped sensor answers `sensor stopped already` then `Error`; a settings press takes 5 to 6 s (stop, change, save,
start). Free heap on the ESP32 is Wi-Fi noise (192 to 198 kB) and says nothing about the component.
Results: https://github.com/esphome/esphome/pull/20979#issuecomment-6279129146 and
https://github.com/esphome/esphome/pull/20906#issuecomment-6279129604
@@ -1,7 +0,0 @@
# espnow
## ESP-NOW on the ESP8266 (#14407)
The NONOS SDK only delivers broadcast ESP-NOW frames while the station is associated or a soft-AP is
up; a hidden, open, client-less soft-AP (60 s beacon, DHCP stopped) makes broadcasts arrive. In AP-only
mode the device transmits from the soft-AP MAC.
@@ -1,37 +0,0 @@
# EZO circuits without hardware: a fake I2C bus (#21840)
`scripts/ezo_fake_bus`: the real `ezo` component runs on a classic ESP32, but an `on_boot` priority 900 lambda calls
`id(ph).set_i2c_bus(&fakeezo::bus)`. `fake_ezo.hpp` is an `i2c::I2CBus` subclass that answers like an EZO pH circuit
and logs every write and read with the ms since the command: `R`, `i`, `Slope,?`, `Cal,?`, `T,?`, `L,?`, `Name,?`,
and test replies `pend` (254 twice), `bad` (2), `nodata` (255), `nak` (read NACK) and `full31` (32 bytes with no
terminator), and `sNN` (an NN-character reply, terminated when the read has room). `ezo.yaml` runs the steps:
command bursts larger than the queue, custom commands, calibration, LED, sleep and address changes, and the error
replies.
- The same trick works for any I2C device whose driver only needs write and read: swap the bus pointer, no wiring.
A real `i2c:` block is still needed so the config validates.
- `run.sh LABEL` builds, flashes, captures 90 s and writes `out-ezo<LABEL>.events`. Compare dev and the PR: the
replies, callback texts, read values and the wait before each read (912 ms for reads and calibration, 304 ms for
the rest) must match; differences in queue limits are the PR's intended change.
- Call plain `millis()` in a fake: Arduino builds define a `millis()` macro (`esphome::millis()`), so
`esphome::millis()` breaks there, and on ESP-IDF the plain name resolves through `using namespace esphome`.
- Found with it: dev read past its 32-byte buffer on an unterminated reply (a custom callback got 56 bytes), and
two queued address changes both went to the first new address.
- Results: https://github.com/esphome/esphome/pull/21840#issuecomment-6493565259
## Trigger argument lifetime (StringRef or an owning copy, #22204)
Give each text trigger two automations (`- then:` list items): a synchronous one that logs `x`, `x.size()` and
`strlen(x.c_str())` and hands `x` to a parallel `script` (`text: !lambda return x;`, the script delays 2.5 s), and
one with `delay: 3s` or `wait_until` that logs `x` afterwards. Fire bursts so newer replies reuse the reply buffer
during the wait, then diff the synchronous texts, in order, against the delayed and script texts. Check the generated
`main.cpp` for `StringRef` or `TriggerForwarder<std::string>`. The fake's `sNN` replies give texts of any length.
## A real EZO circuit
An EZO-HUM ships in UART mode (blinking green, streams `33.91\r` every second at 9600 baud). To switch it to I2C
without bridging PGND, flash a one-shot software UART build (TX to the EZO RX, RX from the EZO TX) that writes
`I2C,111\r`; the LED turns blue and the circuit answers at 0x6F. Replies: `i` gives `HUM,1.01`, `Status`
`?STATUS,S,3.31`, `O,?` `?O,HUM,T,DEW`, and `R` with all outputs `34.82,22.03,Dew,5.85`; `Name,x` returns no text.
EZO-HUM rejects `T,?`, `Slope,?` and `Cal,?` (syntax error), so `on_t`, `on_slope` and `on_calibration` need another
circuit or the fake bus. Log every callback with an `EV` prefix and count them per step.
@@ -1,153 +0,0 @@
# Fake I2C devices without a chip
## Fake devices on an in-firmware I2C bus
When there is no chip and no wiring, subclass `i2c::I2CBus` in an include, implement `write_readv()` only, answer
the registers the driver reads and check each write. Point the component at it from an `on_boot` lambda that runs
before the component's `setup()`: `id(x)->set_i2c_bus(&fake::bus)` at priority 950, which is ahead of every I2C
component (hubs set up at IO, 900, sensors at DATA, 600). The configs here that use 700 to 1000 work for their
components; use 950 for a new one. Keep a real `i2c:` block in the config so validation passes. Bulk
loops in a button lambda must call `App.feed_wdt()` and keep their per-iteration logs below INFO: 2000 DEBUG lines at
115200 baud trip the task watchdog, which is a fault of the harness and not of the PR.
`templates/i2c_fake/` holds the fakes; each `*-common.yaml` or `*-test.yaml` is a classic ESP32 or ESP32-S3 config
that includes its `.hpp` (add `esphome: name:` for the `-common` ones):
- `ccsfake.hpp`, `ccs-common.yaml`: CCS811 at 0x5A (#21927). Answers setup and update reads and checks every
ENV_DATA write against the humidity and temperature the case set; "CCS Cases" runs edge values, "CCS Sweep" 2000
random ones and logs `SWEEP checked N bad M bad_reads R bad_writes W`. A read without a register, of a register
the fake does not model or of the wrong length gets a NACK and counts in `bad_reads`; a write other than the
driver's SW_RESET, APP_START, MEAS_MODE and BASELINE writes (each with its own length) gets a NACK and counts in
`bad_writes`. The CASES and SWEEP lines end in PASS only when something was checked and `bad`, `bad_reads` and
`bad_writes` stayed 0, and in FAIL otherwise.
- `eefake.hpp`, `ee-common.yaml`: HTE501 (0x40) and TEE501 (0x48) with CRC words; "Next" steps through raw
temperature words and logs the value a signed hundredths decode should give (#21930). A command other than
identification and measure, and a read longer than the reply, gets a NACK and counts as `bad`. The DONE line
200 ms later, after both reads, shows the running `bad` total, the measure replies each driver read and the
published values; it ends in PASS only while `bad` is 0, both drivers read a reply this case and both published
the expected temperature (and HTE501 the fixed humidity).
- `scdfake.hpp`, `scd-common.yaml`: SCD4x at 0x62 and 0x63 with Sensirion CRC; logs every `0xE000` ambient pressure
write, and the CASE and SHOW lines count writes with a bad CRC apart from the good ones. A command the scd4x driver
does not send, one sent with the wrong length (2 bytes bare, 5 bytes with one CRC word), or any other command with a
bad CRC, gets a NACK and counts as `bad writes`. A read after a command the fake does not answer, or one longer than the command's
reply, gets a NACK and counts as `unknown reads`. The second instance has a fixed `ambient_pressure_compensation`
to cover the setup path (#21935). Each CASE line ends in PASS only when bad CRC, bad writes and unknown reads did
not move, the `0xE000` writes match `expect_e000()` (none, or one with the expected value) and, for a source case,
CO2 was published. `RANGE_CHECKED` in `scdfake.hpp` picks the rule: false truncates as dev does, true skips values
outside 700 to 1200 hPa and rounds as #21935 did. The SHOW line ends in PASS when nothing bad was counted, neither
instance failed and 0x63 got exactly one write of its fixed value.
- `sxfake.hpp`, `sx-common.yaml`: two SX1509 register files with reset values and the `7D 12/34` reset, recording
every write. Only a byte that lands on RegReset (0x7D) drives the reset, so a burst through it keeps the bytes for
0x7E and 0x7F. A read without a register, or a transfer past 0x7F, gets a NACK and `dump_log()` warns with the
count. Compare per address: hub setup order can legitimately move between hubs (#21897). Outputs of an Apply
platform are distinct template types, so a lambda list of them needs
`std::initializer_list<output::FloatOutput *>{...}`. Each case ends with an END line, PASS only while no write was
dropped and no transfer was bad; the writes themselves are judged by the diff.
- `msfake.hpp`: eight MS8607 pressure/temperature fakes at 0x76 and up and humidity fakes at 0x40 and up, with faults
per device (reset NACKs, bad PROM CRC4, zero D1 every Nth read, humidity CRC8 errors, out of range D2, full scale
humidity). It counts every command, so skipped conversions show in `report()`, and an ADC read with no conversion
pending answers 0 like the real chip and counts as `adc_no_conv`, and a humidity read with no measure pending gets
a NACK like the real chip in no-hold mode and counts as `read_no_measure` (a reset aborts either); each injected
fault counts when it is delivered (`inj_reset_nack` with `reset_nack_left`, `inj_zero_d1`, `inj_bad_crc`), so a
run that never reached a fault path shows 0 there; a PROM read, conversion or humidity measure
before that device's first reset counts as `no_reset`; a humidity read of any shape but a bare
3 byte read, and a pressure/temperature read that is not a 2 byte read of a PROM word (even addresses 0xA0 to 0xAC) or
a 3 byte ADC read, gets a NACK and counts as `bad_reads`; a bare read answers the last PROM or ADC pointer and never
repeats a reset or conversion; a command write the fake does not know, or one longer than a
byte, gets a NACK and counts as `bad_cmds`. Point both the component and its `humidity_i2c_id` device at the bus. That
driver logs no values, so give each sensor an INFO `on_value` log (#16487).
- `htfake.hpp`, `ht16k33-char-test.yaml`: HT16K33 chips at 0x70 to 0x77 (0x74 and 0x76 NACK) keeping the 16 byte
display RAM and a command trace, rendered back as text for the Adafruit 0.56" 7-segment backpack ("12:34",
decimal points, "!" for stray bytes); a read gets a NACK and counts as malformed. The test runs 64 checks (setup order, font, colon and dots, chained chips,
printf/strftime/clock, dimming, blink, standby, NACK) and ends with `ALL DONE pass=N fail=M` and PASS or FAIL (#8520). It found a
crash (null glyph table) when one display had `add_characters` and another did not.
- `ssfake.hpp`, `seesaw-test.yaml`: Adafruit seesaw at 0x36 (ATtiny817), 0x49 (SAMD09) and 0x4A (NACK): status,
GPIO bulk with the pull latch, ADC, touch, encoder and NeoPixel with chunk offsets, and the write to read gap per
module. A read it does not model, a write without a register and a write to a module or register it does not model
(or with the wrong data length) get a NACK and count in `bad`. A 100 ms interval step machine drives it and ends with `ALL DONE` and PASS or FAIL (#13538).
- `bldcfake.hpp` (three M5Unit-BLDC units at 0x65 to 0x67 with simple motor physics, one that never spins down),
`gtfake.hpp` (GT911 at 0x5D: switches, max values, touch points and key byte) and `sdpfake.hpp` (SDP3x/SDP8xx at
0x21 to 0x26: product IDs, an unknown product, a failing read every third time, a failing ID read, stop refused
while idle) have no config yet; wire them the same way. Each fake sets up its own faults in its constructor, so
there is no init step to call. Each NACKs a read it does not model or one of the wrong
length, and a write or command it does not know, and counts them as `bad_reads` and `bad_writes` (`bad_cmds` for
the SDP) in `report()`; check those are 0. The SDP report also counts each injected ID read NACK at 0x24 as
`id_fail`, so 0 there means the driver never reached that fault.
Check the PR head again just before posting results: a head that moves during a run means a retest.
## Pico 2 W as an I2C target (#22236)
`scripts/pico_i2c_target/`: `picofake.hpp` is a raw RP2350 I2C0 interrupt handler on GP4 (SDA) and GP5 (SCL) at 0x76 that
answers one byte per read request from a 256 byte register file, so any read length is byte exact. `picofake.yaml`
plays a BME280 (`chip_id` 0x60) or BMP280 (0x58) with the datasheet calibration; every 15 s it switches between the
datasheet example readings and the 0x80000 "no measurement" values, and logs `STAT` counters every 5 s. `bad`
counts reads and writes of registers the drivers do not use (soft reset only with 0xB6, read pointer wrapping past
0xFF). The DUT is
an ESP32-C3 (`dut-common.yaml`, GPIO6/GPIO7) with two sensors on the one fake (`dut-bme280.yaml`,
`dut-bmp280.yaml`), the second with oversampling NONE. `run.sh bme280|bmp280 LABEL` flashes the DUT, starts the
fake capture and waits for a `STAT` line as the baseline, then resets and captures the DUT and prints the distinct
values per sensor. The fake capture runs on until a `STAT` line arrives after the DUT capture ends, so that line
covers the whole run. It prints PASS when every sensor gave a numeric state, that `STAT` line came, and the fake's
`abrt` (read replies the controller aborted) and `bad` counts did not move from the baseline; diff the output of dev
and the PR.
The ESP32-S3 I2C slave v2 driver cannot emulate a register device: answering from a task and straight from the ISR
both gave shifted multi-byte reads. Use the Pico.
## DS2482-800 simulator (ds248x)
`scripts/ds248x_sim/`: `dssim.yaml` with `sim.hpp` turns a Pico 2 W into a DS2482-800 at 0x18 (GP4 SDA, GP5 SCL)
with four DS18B20s: channel 0 at 21.5 °C, channel 1 at 33.0 °C, channel 2 at -5.25 and 40.0 °C. The DUT is an
ESP32-C3 (`dspd.yaml`, GPIO4/GPIO5 plus GND); the internal pull-ups are enough at 50 kHz. Faults are one character
on the Pico's USB console: `c` NACKs the read pointer write after the next channel select, `z` NACKs the 9 reads
after the next 0xBE, `b` and `k` stick busy with a failed reset or config write, `x` and `X` make it dead and alive.
`SC=<serial_capture.py> python3 drive.py TAG SIM_PORT DUT_PORT [c,cd,z,b,k,dead]` (default `c,z,b,k,dead`; `cd`
runs the `c` fault densely, 28 times about 3 s apart) resets the DUT per phase and prints
readings, wrong values, log counts and the simulator's counters (`wrong_ch` counts ROM matches on the wrong
channel). The DUT is captured with `--ms` so the log can be split at the last fault: the recovery window starts
`SETTLE` (3) seconds after it. A phase fails when any of the four sensors got no correct reading in the recovery
window, the DUT logged a CRC, held low, busy timeout or no response line in it, a temperature is wrong, a phase other
than `dead` logged `recovery failed`, `DS248x hub failed` or a component marked failed, a ds248x warning is still
set at the end of the phase, the DUT capture
failed (stderr in `TAG-<phase>.err`), a fault moved its injection counter (counted when the DUT meets the
fault, not when it is armed) by other than the number of faults sent (`arm_lost` counts arms that landed while one
of the same fault was still pending and merged into it), a NACK fault never moved `nacked`, a write delivered data while a NACK was armed
(`nack_miss`, also counted when the write is followed by a repeated start into a read; the NACK never reached the wire and stays pending), the DUT sent an address-only write (`empty_wr`), a write was cut off by a repeated start into another write and
never processed (`dropped`),
a fault is still pending (`pend=1`) at the end of the
phase, the simulator counted a config write with a wrong complement (`cfg_bad`) or an unknown, short or overlong
command (`bad_cmd`), a counter is missing from the simulator's CNT line, the simulator aborted a read reply
(`tx_abrt`, not gated in the `dead` phase, which switches the controller off), a phase other than `b` and `k` (which
reset the chip under the driver) counted `wrong_ch` or a 1-Wire command without the active pull-up bit (`ops_no_apu`),
the simulator never logged `DEAD` for the `dead`
phase, or the DUT log of the `dead` phase has no sign of the outage (`did not respond`, a ds248x warning,
`DS248x hub failed` or a component marked failed). It ends with PASS or `FAIL:` and the failed phases. The whole
phase CRC, held low, busy timeout and no response counts are printed for information: the faults cause them.
- Sensors polled at the same interval go through the channels in blocks, so a stale channel cache never shows.
Poll them at different intervals, as `dspd.yaml` does (1, 1.3, 1.7 and 2.3 s).
- The NACKs use the RP2350 `slv_data_nack_only` register, set at STOP.
- A NACKed write and an address-only write both end with no data, so the simulator cannot tell them apart.
The ds248x driver never sends one, `dspd.yaml` sets `scan: false` and the simulator only sees STOPs after its
own address, so `empty_wr` (address-only writes outside a NACK window) stays 0; a nonzero count means `nacked`
may include writes that never carried a NACKed byte.
## I2C fault injection in software (vl53l0x)
`scripts/i2c_fault/`: `faultbus.h` wraps the real `I2CBus` and returns NACK for chosen transfers to one address. An
`on_boot` priority 800 lambda swaps the sensor's bus (`id(d)->set_i2c_bus(&fb)`) before its setup, and an interval
arms the faults. `vlpf.yaml` runs a VL53L0X on an ESP32-S3 (GPIO8/GPIO9) through quiet, fault A (the next
measurement-in-progress read fails), quiet, fault B (the next measurement start fails) and quiet phases over 150 s,
with a model ID probe every 10 s to tell a driver stall from a dead chip. `run.sh LABEL` builds, flashes, captures
and runs `an.py`, which counts readings per `PHASE` line. Each `PHASE` line clears a fault that was armed and has
not fired yet and logs the fired totals so far, so `an.py` counts the fires of each phase. It fails when the phases are not quiet, faultA, quiet,
faultB, quiet (a short capture or a restart), the first quiet phase has no readings, a later quiet phase reads less
than half as often as the first, a later quiet phase reads NaN (`nan` or `-nan`) or logs the driver's "update called before prior
reading" warning when the first did not (a target out of range reads NaN in every phase), a quiet phase has a failed
model ID probe, one that did not return 0xEE or no successful probe at all,
anything crashed or rebooted, or a fault did not fire in its own phase or fired in another one.
Prefer this to glitching the wire. Holding SDA low by rerouting the pin (`esp_rom_gpio_connect_out_signal` to
`SIG_GPIO_OUT_IDX` and back to `I2CEXT0_SDA_OUT_IDX`) wedges the ESP-IDF controller, which then needs
`i2c_master_bus_reset()` after each glitch, and glitches timed onto writes left the VL53L0X answering 0x00 until a
power cycle.
@@ -1,15 +0,0 @@
# globals
## Restoring globals with storage: rtc and flash (#13954)
`templates/globals-storage-dut.yaml` is the shared package: five restoring globals (rtc int, flash int, default int,
rtc `std::string`, rtc `uint8_t[2]`), bumped and logged with the reset reason on every boot, saved and synced after
2 s, odd boots `App.safe_reboot()`. Platform files add the board (`packages: dut: !include globals-storage-dut.yaml`), the `debug` reset-reason text sensor and, on the
ESP32, `deep_sleep: run_duration: 12s, sleep_duration: 8s` (even boots sleep). No network needed: flash by USB,
`serial_capture.py PORT 110 OUT --reset` gives the reset pulse plus a run of soft restarts and sleeps; the ESP32's
EN pulse is a `POWERON_RESET` that empties RTC memory, the D1 mini's RST pulse (`rst cause:2`) keeps it, so the
ESP8266 needs a real power cut by hand; arm long `--released` captures on both ports first (the capture
reopens the port when it comes back, the very first boot lines are still lost, read the next boot's restored
values). Unchanged-build check: the config with every `storage:` removed, built on the PR and on the dev merge
commit under the PR's own commits (not the merge base, which is older dev), `cmp -l` plus an `nm -S` symbol diff;
only the template names differ. Results: https://github.com/esphome/esphome/pull/13954#issuecomment-6254794712
@@ -1,13 +0,0 @@
# GPIO expanders
## Interrupt driven base class (#21469, #21468)
Any device with a PCF8574 input expander whose INT line goes to an ESP32 GPIO works; the inputs should change
during the test (end switches on a gate or cover are ideal). Before flashing, check with `nm` that
`gpio_expander::interrupt_pin_isr` lands in IRAM (0x4008xxxx on a classic ESP32). Then confirm `dump_config` lists
the interrupt pin and the input states match reality, and that the `pcf8574` entry is absent from the runtime stats
while idle (its loop is disabled). Drive the inputs (for a cover, `cover_command(position=1.0)` and then `0.0` over
the API, waiting for each end state): every input edge should log, `pcf8574` should show a small count in that
minute's stats and vanish in the next. For a device that cannot be reached physically, boot the same code on a
bench board first and keep a merge base image built as the fallback; an ESP32 bootloader without rollback support
leaves only `safe_mode` to recover a bad image.
@@ -1,17 +0,0 @@
# ESP32 GPIO hold through the ISR pin API (#21576)
`scripts/gpio_hold/hold_test.hpp` builds `ESP32InternalGPIOPin`s by hand, calls `to_isr()` and reads each pad back
with `gpio_get_level`: 20 ISR writes, a raw `gpio_set_level` control (a held pad must not change, an unheld one must)
and ISR `pin_mode` pulls (input must drop to `NONE` between pull changes, because the ISR path only applies pulls when
`FLAG_INPUT` toggles). It needs no wiring: unconnected pins read the pulls. Each pin runs held and unheld, then the
round logs `ROUND n PASS failures=0` or `FAIL`.
- `USE_GPIO_HOLD` comes from any pin with `hold_state: true`, so the configs add a `gpio` output on a spare pin.
- `gpio-hold-esp32.yaml` (classic devkit) runs three rounds 5 s after boot and has a `rerun` button;
`gpio-hold-esp32s3.yaml` (Waveshare ESP32-S3-ETH, where most pins go to the W5500, TF card, camera, WS2812 or
USB) runs a round every 15 s.
- C3: ESP-IDF 5.5 has `SOC_RTCIO_PIN_COUNT` 0, so `rtc_gpio_is_valid_gpio()` is false on every pin and only the
digital hold path runs.
- C6 (Waveshare ESP32-C6-GEEK, native USB): LP pads GPIO0 to 7 are RTC IO with hold. GPIO1, 2, 7, 20, 23 read high
and GPIO6 low whatever the pull (LCD, TF card or resistors); clean pins are 0, 3, 4, 5, 18, 19, 21, 22. Dev
already holds LP pads through `gpio_hal_hold_en`, so the PR and dev look the same there. H2 is untested.
@@ -1,9 +0,0 @@
# hdmi_cec
## HDMI-CEC two-board wire (#9189)
Two boards on one wire: promiscuous mode, a 0x8F request answered with 0x90 0x00, a `destination:`
filter on the reply handler, template buttons for each target plus lambdas calling
`send(source, dest, span)` as another initiator. Healthy numbers: device-side turnaround 15 to 22 ms,
round trip 144 to 156 ms. Press buttons over the API; judge timings from device logs (ESP32 light sleep
lags the press).
@@ -1,47 +0,0 @@
# homeassistant
## homeassistant light platform against a local Home Assistant (#11354)
A local Home Assistant dev install (`hass -c config`, https://127.0.0.1:8123 with a self-signed cert); its ESPHome
entries and entity ids are in `config/.storage/`. Two ESPHome RGBCW bulbs with a plaintext API are the lights, so an
"HA side" change is made directly on a bulb with `scripts/homeassistant_light/bulb.py BULB_IP - on|off|state|restart`.
Add the test devices to HA by hand with the shared `api_key` and "Allow the device to perform Home Assistant actions".
`templates/homeassistant-light-dut.yaml` is a package: two mirrors, buttons for each change, a "Log state" button, a
30 s alive line, an `on_boot` `light.turn_on` for the change-before-first-write path, and VERBOSE logging with the
noisy tags at DEBUG (a tag cannot exceed the global level). A platform file sets the board and includes it:
```yaml
esphome:
name: ha-light-dut
esp32:
board: esp32dev
framework:
type: esp-idf
packages:
dut: !include homeassistant-light-dut.yaml
```
Flash by USB, start a bounded `esphome logs` capture per device, then press buttons with
`scripts/api_button.py DEV_IP KEY press "Button"` and read the bulb with `bulb.py`. Cases: each
button against the bulb and the device log; a direct bulb change that both mirrors follow; restart the bulb, send a
change, expect the 5 s `did not report the change back` and the fallback (with an off and with an on last report);
OTA reboot with a serial capture running to see the boot-time change (the API log client reconnects too late).
Lessons: the light core clamps mireds to the range HA reported; a flash sends nothing; a change sent while HA is
connected but not yet subscribed is dropped by the api server (`client has not subscribed to actions (yet)`); after a
fallback to an off report the mirror keeps the unsent brightness; a mirror that never saw its light on since boot has
no levels to restore. Results: https://github.com/esphome/esphome/pull/11354#issuecomment-6253630336 and
https://github.com/esphome/esphome/pull/11354#issuecomment-6254161409
## Held report versus a later call (#21144)
A report that arrives during a transition is held and applied when the transition ends, unless a call came after
it. `scripts/homeassistant_light/heldcase.py DEV_IP KEY BULB_IP same|many [BULB_KEY]` (a key of `-` is plaintext) keeps one API connection to the mirror
device and one to the bulb so the steps land inside a 3 s fade: press "A 50 fade 3s", 0.7 s later set the bulb to
90% directly (the report is held), 1.4 s in press the same fade again (or "A many calls", 16 calls in one loop
pass). Pass: the mirror and the bulb end at the second call's target. Only `same` reproduces the bug: a build
without the fix logs `Applying the report that arrived during the transition` and the bulb stays at 90%. `many`
ends at 100% on both builds because the last call's target differs from the held report, so it is a regression
check only, not a reproduction. Separate button presses are too slow (about 1.5 s each) to land inside the fade.
The mirrors are not API entities, so read their state from the "Log state" button's line. Results: https://github.com/esphome/esphome/pull/21144#issuecomment-6278257695
@@ -1,48 +0,0 @@
# http_request
## Update manifest URL changed at run time (#21373)
`scripts/http_request/manifest_server.py PORT [DELAY]` serves `stable.json`, `beta.json` and `slow.json` (answered
after DELAY s) with both chip families and logs every request. `templates/http_request_update/upd-common.yaml`
(secret `manifest_base` is the server URL) has the update entity on the stable manifest, buttons for every
`set_source_url` shape (`std::string` local, literal, temporary, `.c_str()` of a local, the current URL back in),
`Check Then Swap` (ESP32 task path: check the slow manifest and replace the URL in the same lambda), `Arm Swap In
Check` plus a `logger: on_message` lambda that replaces the URL from inside the inline check (ESP8266; the log line
from inside `on_message` is swallowed, judge by the next check's result), and a `Heap` button. `upd-base-common.yaml`
keeps only the shapes that compile on a tree with the bare pointer setter (for the base build, copy a board file
and point its `common` include at `../upd-base-common.yaml` with a different `name:`); `boards/upd8266.yaml` and `boards/upd32.yaml` are the
per-board files, one directory below the package their `../` include points at. Drive with `scripts/http_request/updrive.py HOST KEY LOGFILE press:NAME wait:S`, which prints every
update state (latest version, summary). The ESP8266 Arduino client gets `Error reading manifest: -1` on roughly a
quarter of the fetches from this Python server on every build, so count checks against the base before blaming a
PR. `http_request` on the ESP8266 needs `verify_ssl: false` even for plain http URLs.
Results: https://github.com/esphome/esphome/pull/21373#issuecomment-6337544933
## Non-blocking request actions (#21486)
`scripts/http_request/hsrv.py PORT` serves `/fast`, `/slow?d=S`, `/big` and `/chunked` (`n` bytes at `rate` B/s,
deterministic bytes `(i*7+3)&0xFF`), `/post` (echoes length and byte sum), `/redirect` and `/hang`.
`templates/http_request_action/hr-common.yaml` (substitutions `name`, `bufsize`; per-board files `hr32.yaml` and
`hr66.yaml`, `-s name hr32base` on the command line for the base build; `hrota32.yaml` adds the `otaflash` action
for `otadrive.py`) has API actions `req_get`, `req_post`, `burst` (N runs of a `mode: parallel` script on one action),
`cancel` (`mode: single` script run twice, then `script.stop`) and `reboot_during`, and logs
`RESULT n status len sum took maxgap`, where maxgap is the longest main loop gap during the request (an
`interval: 1ms` lambda plus the time since its last pass, so a fully blocking request counts too). Drive with
`scripts/http_request/hrdrive.py IP http://HOST:PORT LABEL [https]` (`ONLY=case,case` and `ROUNDS=n` env; the
`cross` case runs two actions on one component). `burstonly.py IP URL N` checks concurrency against the server's
completion times, `reboottest.py IP URL` times `App.safe_reboot()` with requests in flight, `updcheck.py IP LABEL N`
runs N update checks over the API (for real device configs that use `update: platform: http_request`) and
`otadrive.py` triggers `ota.http_request.flash` (the API variable is `sum`; `md5` is a reserved variable name).
`execute_service` must be awaited on current aioesphomeapi or nothing is sent. Keep the ESP32 `bufsize` at 32 KB or
less: the action copies the body into a `std::string` while the buffer is still held, and 80 KB aborts with out of
memory on dev and on the PR alike. `http_request` refuses BK72xx and RTL87xx at validation.
## OTA request cases with basic auth (#21556)
`scripts/http_request/ota_auth/`: `ho-common.yaml` (API actions for `ota.http_request.flash` with `md5_url`, `md5`
and basic auth, `flash()` with no URL, an invalid URL, the `md5_computed()`/`md5_expected()` getters), `fwsrv.py`
(serves a www dir, `/auth/` needs `otauser:otapass`), `hodrive.py` (11 cases, reconnects after each good flash) and
`run.sh BUILD BOARD` (compiles from `TREE`, serial flash, serves that exact image; `TREE`, `P32` or `P66`, `SC` and `HOST_IP` in
the environment). On the D1 mini the authenticated MD5 read fails
with `Error reading MD5: -1` on dev too, and stored credentials carry over to the next flash on both builds.
Results: https://github.com/esphome/esphome/pull/21556#issuecomment-6381067561
@@ -1,26 +0,0 @@
# HTTPS on web_server_idf (#22185)
#22185 adds an HTTPS developer API to `web_server_idf` with no YAML option, so `scripts/https_idf/` brings a test
component: `ext/httpstest` (`httpstest.hpp`) serves `/hw` (a status line with mode, request count and free heap),
`/hw/big` (16 KB) and `/hw/switch/{http,https,https8443,bad}`. A switch runs on the main loop through `defer()`, or
3 s later with a `/later` suffix (a stalled TLS client holds the server task, so its switch is armed first), and
calls `set_ssl_credentials(cert, key, port)`: plain HTTP on 80, HTTPS on 443 or 8443, or a key that does not match
the certificate. Its schema uses `web_server_idf.consume_ssl_sockets`. The run-time switch is the follow-up API; on a
tree that only has #22185, drop the switch paths and set the credentials once before `init()`.
Copy the directory into a scratch directory with a `secrets.yaml`, then `run.sh LABEL` (env `TREE`, `PORT`, `SC`,
`PY`): it runs `make_certs.sh` (a self-signed P-256 certificate, its key and an unrelated key in
`ext/httpstest/certs.h`; test material, never commit it), builds `https-dut.yaml` (ESP32-C3, logs on USB), flashes,
takes the address from the serial log and runs:
- `t1.sh HOST`: one GET with the TLS time, 20 sequential, 6 parallel three times and the 16 KB body. The fixture has
no event source, so the 501 that #22185 sends for an event stream on HTTPS is not covered here.
- `stall.py HOST`: arms a switch to HTTP and to 8443, opens two idle TCP clients (no ClientHello) on 443 before it
runs, then switches back; every switch must be accepted and work, and the idle clients must be closed.
- `t2.sh HOST`: HTTP and back, an armed switch to 8443 with a client open that sends a handshake record header and
never the rest, 60 requests that must all get a 200 for a heap check (compare the `/hw` heap before and after),
then the bad key. The server must log the error and stay off, without falling back to plain HTTP; reset the board
afterwards.
The runner fails on any unexpected status, a missing key error for the bad key, a reset or a task watchdog. `https-soak.yaml` adds a switch between 443
and 8443 every 20 s for a long run with clients attached. Each HTTPS connection takes about 26 KB of internal RAM
(no PSRAM on the C3), so watch the `ALIVE heap=... min=...` lines.
@@ -1,21 +0,0 @@
# matrix_keypad, wiegand, rdm6300 and key_collector without hardware (#21787, #21788, #21790)
All on a D1 mini with no wiring: `scripts/hub_input_sim/hub.yaml` and `sim.hpp`.
- Keypad: one row (GPIO5) and columns GPIO4, 14 and 12 with pull-ups. `sim::press(pin)` switches the column to
`OUTPUT_OPEN_DRAIN` driven low, so the single row reads that key pressed on every scan; `sim::release` goes back
to `INPUT_PULLUP`.
- Wiegand: D0 on GPIO13, D1 on GPIO0. Call `sim::wiegand_arm()` once, then `sim::wiegand(value, bits)` sends the
pulses (80 us low, 1.5 ms apart, most significant bit first). `pinMode(OUTPUT_OPEN_DRAIN)` clears the pull-up and,
with the output latch low, makes a falling edge that adds a bit: write `GPOS` high before the mode switch and set
`GPF |= 1 << GPFPU` after. The interrupt type survives `pinMode`.
- rdm6300 reads UART0 with its loopback bit set (`USC0(0) |= 1 << UCLBE`), so the logger runs with `baud_rate: 0`
and the log goes over the API. Frames are 0x02, 12 ASCII hex characters (version, 4 id bytes, xor) and 0x03. The
uid binary sensor pulses on and off per read.
- key_collector takes both the keypad and wiegand as `source_id`.
- `run.sh LABEL` flashes over serial, reads the API log until `DONE` and writes `out-hubsim<LABEL>.ev`; diff the
`[ev` lines of dev and the PR.
- Wiegand frames with valid parity: 26 bit 0x3579bdf (11259375), 34 bit 0x3bd5b7dde (3735928559), 37 bit
0x1b4b4b4b4a (24253932965). 0x3579bdd fails parity, so only `on_raw` fires for it.
- rc522 and sx1509 need their chips; compile them only.
- Results: https://github.com/esphome/esphome/pull/21790#issuecomment-6491675405
@@ -1,28 +0,0 @@
# hx711 without a chip: fake HX711 on the same ESP32 (#8899)
`scripts/hx711`: classic ESP32, no wiring. SCK is GPIO32, DOUT GPIO33 (configured as an input with pull-up, so DOUT
reads busy until the fake runs). `hxsim.hpp` starts from `on_boot` priority -100 (after the component has set up its
pins), switches both pins to `GPIO_MODE_INPUT_OUTPUT` (it reads the clock and drives the DOUT level the component
reads back) and runs a task pinned to core 0 (the ESPHome loop runs on core 1; core 0's idle task is removed from
the task watchdog).
- Ready means DOUT low; the task waits for the clock with interrupts off in 2 ms windows. Each rising edge puts the
next bit out and the 25th sets DOUT high; extra pulses pick the next conversion (25 A/128, 26 B/32, 27 A/64), and
each conversion takes 100 ms. Clock high for more than 60 us is power down (back to A/128), clock low powers up.
- Values encode the conversion: A/128 is 128000+n, A/64 64000+n, B/32 -(32000+n), so a published value shows which
conversion it came from. `check.py CAP...` matches every `Got value` against the served values and exits non-zero
on an unmatched value, no values or a crash.
- `mode`: 0 normal, 1 DOUT stuck high, 2 DOUT left low after the read.
- The waiting and shifting code must run from IRAM. `IRAM_ATTR` on inline class members either gets inlined into
flash code or fails to link ("dangerous relocation: l32r: literal placed after use"); use static free functions
with `IRAM_ATTR __attribute__((noinline))` and check with `xtensa-esp32-elf-nm` that they sit at 0x4008xxxx. Run
from flash, the first transfer after boot lost one clock pulse (the value shifted right by one bit). Log from the
main loop (an interval drains an event ring), never inside the critical section.
- Runs: `hxbase` (65 s), `hxpr1` channel B plus set_gain and power actions (115 s), `hxpr2`
power_down_after_reading (45 s). `hx-common.yaml` starts each run's `run_all` script and logs the `DONE` line.
`build.sh NAME TREE LABEL` writes `NAME<LABEL>.yaml` with its own name (so dev and the PR builds of the same config
do not share a build directory) and builds it; then `flashcap.sh DIR NAME<LABEL> TREE SECONDS` (`PORT`, `SC` in
the environment), which fails without a `DONE` line and runs `check.py`.
- The configs have no Wi-Fi on purpose: the fake keeps core 0 busy with interrupts off in 2 ms windows, which would
starve the Wi-Fi task. Keep the harness without Wi-Fi, API or OTA.
- Result on #8899: no missed pulses over 141 transfers, and every published value matched.
@@ -1,16 +0,0 @@
# Fake I2C slave (#21586)
`scripts/i2c_slave`: a D1 mini running an Arduino `Wire` slave at 0x40 (`i2cslave.yaml` with `i2cslave.hpp`): a fake
HDC1080 plus a 256 byte register file, and reads of 0x7E and 0x7F stretch the clock for 0.5 and 5 ms. It needs
`libraries: [Wire]` and 160 MHz. With jumper wires only and the internal pull ups it works to about 50 kHz; it cannot
keep up with a true 100 kHz hardware master.
The master (`i2cbk.yaml`, a BK7238 on P26/P24, needs the software I2C bus from that PR) writes a counter, reads it
back with a repeated start, probes an absent address and reads both stretch registers every 2 s, logging `RW OK`
(the read back matches, the absent address NACKs and both stretched reads succeed) or `RW BAD` with running totals.
Control: the same blocks on a classic ESP32 devkit (SDA GPIO21, SCL GPIO22) with hardware I2C at 25 and 50 kHz got
every repeated start read right, so failures on a bit banged master at a similar real clock point at the master.
On the NiceMCU WB3S V1 board (BK7238), P26 and P24 are on the right-hand header.
Fake devices that need no real chip (in-firmware buses, the Pico 2 W target, the DS2482-800 simulator, NACK
injection) are in [fake_i2c.md](fake_i2c.md).
@@ -1,14 +0,0 @@
# i2s_audio
## I2S microphone loopback on one S3 (#22130)
With no I2S microphone at hand, `scripts/i2s_loopback/` makes one. `dcgen.hpp` runs an ESP-IDF std TX channel on
I2S1 as master (GPIO4 BCLK, 5 WS, 6 DOUT, 16-bit stereo) in its own task, switching patterns (DC step, opposite DC
per channel, DC plus a 1 kHz tone, full and half scale 50 Hz squares) through `dcgen::set_mode()`. Jumper 4 to 15,
5 to 16 and 6 to 17 to an `i2s_audio` microphone on I2S0 with `i2s_mode: secondary`. `dcgen::on_data` (registered
with `add_data_callback`) keeps per-segment and last 500 ms stats plus a 1 ms trace of channel 0, and `report()`
logs `SEG` and `SETTLE` lines from an interval. `run.sh LABEL [SAMPLE_RATE] [CHANNEL]` flashes and captures 60 s,
and `sum.py` tabulates. The segment min and max include the tail of the previous pattern (capture latency), so judge
with the tail values. The runner fails on a missing segment, a full-scale square under rms 8000 (no signal through the
jumpers), on either channel for a stereo run, or a crash after the first segment.
Results: https://github.com/esphome/esphome/pull/22130#issuecomment-6615554179
@@ -1,13 +0,0 @@
# ld2412
## Output pin level and the basic configuration command (#21353)
`templates/ld2412-select-only-dut.yaml` (ESP32-S3, the module on GPIO43 and GPIO44 at 115200, only the output pin
level select, the module's OUT pin on GPIO38 as a binary sensor plus the presence sensor) with `uart: debug:
direction: BOTH`. Drive the select with `scripts/ld2450/lddrive.py HOST KEY LOGFILE sel:NAME=OPTION` and read the TX
dump for command `02` carrying the level byte, then the OUT pin binary sensor against presence: with level high the
pin follows presence, with level low it is inverted. A module that never answers the boot query (no `<<<` lines)
only shows the held back path: `Command 02 held back until the module reports its settings`, a query `12`, and after
2 s `Command 02 not sent, the module did not report its settings` with the select republished to the last known
level. With the full set of numbers configured a select change also needs every number to have a state, so wait
for the boot query answer before judging.
@@ -1,21 +0,0 @@
# ld2420 without a module (#21914)
`scripts/ld2420_sim/` runs the hub on a D1 mini UART0 with the loopback bit set and no radar. A `uart: debug` TX hook
(delimiter `04 03 02 01`, the command footer) hands each command frame the hub sends to `ldsim::on_tx()`, which writes
a success ack right behind it: the hub reads its own echo and then the ack. The acks carry the version (the `fw`
substitution), the min and max gate and timeout registers and the gate parameters, so `setup()` completes (`Sim stats`
logs `cmds 21 acks 21`). Template buttons then inject energy frames (`Energy steps`) or simple mode `ON`, `OFF` and
`Range N` lines (`Simple steps`). Firmware v1.5.3 or older keeps the hub in Simple Mode.
- `run.sh LABEL energy|simple HOST` builds, flashes and records two `lddrive.py` runs over the API; `compare.py` takes
the same run of two builds and prints PASS when the state sequences after the steps button match, every setup
command was answered, a Distance and a Presence state arrived and no ERROR line was logged.
- The loopback bit is set at `on_boot` priority 900: after the UART is set up and before the hub's blocking `setup()`.
- `fire_trigger_()` hands the debug lambda the debugger's own buffer, which writing the ack changes; `on_tx()` copies
it first, and ignores its own acks (command high byte 0x01) when the nested call sees them.
- The hub also reads its own echoed command frames. A read register command carries `00 00` where an ack has its error
word, so the echo counts as that command's ack and every real ack lands one command late. The Firmware text sensor
published during `setup()` therefore shows `v0.0.0` in this harness; the config dump and the Simple Mode check see
the simulated version. A real module does not echo, so this harness cannot check that text sensor value.
- Energy mode and simple mode publish at most once a second; a burst of 5 frames 100 ms apart publishes only the first.
- Results: https://github.com/esphome/esphome/pull/21914#issuecomment-6517294344
@@ -1,68 +0,0 @@
# ld2450
## LD2450 polygon zones (#20154)
To make a target "leave" without anyone moving, set the radar's Zone-1 numbers over the target and
switch the zone type to `Filter` (the count drops to 0 within 0.5 s); set it back to `Disabled` after.
Writing those numbers blocks the ld2450 loop about 50 ms.
## Zone numbers round trip (#21146)
`scripts/ld2450/zonedrive.py HOST KEY(- for plaintext) set:NAME=VALUE sel:NAME=OPTION wait:S list` sets numbers and
selects over the API and prints every LD2450 state update. After each set the hub sends the set zone command, queries
the radar and republishes all twelve coordinates from the answer, so set each coordinate of each zone to a distinct
value and check the readbacks; no target is needed. Readbacks can arrive late over Wi-Fi, so match them by content.
Results: https://github.com/esphome/esphome/pull/21146#issuecomment-6278026922
## ESP8266 loopback harness, no radar needed (#21349)
`templates/ld2450-loopback.yaml` (package; substitutions `name` and `baud`) runs the hub on UART0 (GPIO1 and GPIO3)
with `logger: baud_rate: 0` and sets the UART0 loopback bit in `on_boot` (`USC0(0) |= (1 << UCLBE)`), so everything
the hub or a `uart.write` action sends comes back on RX. Template buttons inject periodic frames (all four direction
strings, targets 2 and 3), query zone acks (zone type 0, 1 and 2 for the select readback) and call the name based
setters with a valid and a bogus name. `uart: debug: direction: TX` delimited on the command footer dumps every
command the hub sends. Drive it with `scripts/ld2450/lddrive.py HOST KEY LOGFILE press:NAME sel:NAME=OPTION wait:S`
(logs over the API at VERBOSE, 2 s between steps because the API log stream drops lines while `send_command_`
blocks), then `txsum.py LOGFILE` lists the TX frames per step; diff the summaries of the two builds. The hub reads its
own commands back and logs `Invalid status` for each, which is harmless. The boot readback of the baud rate select
comes from the UART setting, so a second build at `baud: "9600"` checks the flash table lookup.
Results: https://github.com/esphome/esphome/pull/21349#issuecomment-6332320026
## Frame splitter collisions (#21357)
The loopback template also carries the zone 1 numbers and seven collision buttons: a periodic frame with the
command footer inside target 1, zone acks with `55 CC` and with `04 03 02 01` inside the coordinates, a periodic
frame cut after 12 bytes followed by a zone ack, a frame missing its last byte followed by a complete one, a bogus
255 byte length then a frame, and garbage then a frame. Judge by the state updates plus the `Handling Periodic` and
`Handling Ack` lines. Through a real radar the set zone ack is sometimes corrupted by the radar itself (its tail
replaced by the periodic frame it was sending, for example `04 03 03 03 00 ... 55 CC`), so count readbacks over at
least 24 selects per build and read the raw bytes with `uart: debug: direction: RX` (`!extend` the package's uart
id) before blaming the parser; the API log stream drops lines under that dump volume, so some misses stay
unclassified.
Results: https://github.com/esphome/esphome/pull/21357#issuecomment-6333244886
## Frame stream harness: presence hold and link loss (#21733, #21734)
`scripts/ld2450/stream-harness.yaml` is a loopback harness that feeds the hub one periodic frame every 100 ms from a
mode global set by API buttons: no target, target 1 approaching, silent, or 12 garbage bytes per tick (what a radar
at another baud rate looks like). `has_target` has `filters: []` so the hub's own hold is what gets timed, and every
change logs `presence raw ON|OFF at <millis>`. Add `stream-connected.yaml` as a second package on builds whose ld2450
has the link watchdog (`connected` binary sensor).
- `holddrive.py HOST OUT`: for `presence_timeout` 5 and 0, three seconds of target frames, then empty frames; prints
the ms from the press to the off line, then sets 0 and 20 and reads the number back after a restart. On dev 0
still held 5 s and snapped back to 5; with #21733 it clears on the next empty frame (under 200 ms) and stays 0.
- `linkdrive.py HOST OUT`: steps through target frames, 10 s of silence, empty frames, 10 s of garbage, target
frames, then the hub's radar restart followed by 3.5 s and by 15 s of silence; prints every state change and link
log line with its time. Dev freezes every entity through silence and garbage. Run against a real radar too: a
radar restart and a Bluetooth toggle must not report a loss (the Apollo R PRO-1 ETH did not).
- A freshly booted ESP8266 delays API commands sent in its first seconds and then runs them back to back; the
drivers wait 8 s after connecting before timed presses.
## BT MAC sensors (#21731)
`scripts/ld2450/macdrive.py HOST OUT` reads every LD24xx BT MAC, Bluetooth and firmware entity, turns each Bluetooth
switch off, then on, and prints the values after each step. The MAC sensors are often `disabled_by_default` in
vendor configs but still publish over the API. On the Apollo R PRO-1 ETH both radars (LD2450 fw 2.04, LD2412 fw
1.26) have the multicast bit set in the first MAC byte, which dev showed as `unknown`. A radar MAC identifies a
device: quote only its first byte in PR text.
@@ -1,29 +0,0 @@
# Bit-banged LED drivers without the chips (#21841 to #21847)
The GPIO LED drivers (bp5758d, bp1658cj, sm2235, sm2335, tlc5971, tlc5947, my9231, sm16716) never read the chip
back, so a classic ESP32 devkit can capture its own pads with no wiring. `scripts/led_driver_sniff`.
- `sniff.hpp`: SPI2 and SPI3 run as DMA slaves fed from the driver's clock and data pads through the GPIO matrix
(`esp_rom_gpio_connect_in_signal`, pad input enabled with `gpio_input_enable`). Mode 1 samples on the rising edge
of its clock input (measured, not what the mode table suggests): SPI2 takes the clock straight (rising-edge bits,
`R` lines) and SPI3 inverted (falling-edge bits, `F` lines; my9231 clocks a bit on both edges). CS is GPIO2,
driven by the test. PCNT counts clock and data edges and latch rises per window.
- ESP32 slave DMA drops a partial last word, so `close()` pads the clock with whole pulses to a 32-bit boundary plus
32, and the bit count comes from PCNT. `selftest()` sends a known 57-bit pattern with the clock idling low and
high; `selftest.py` checks it, and `run.sh` runs it on every capture.
- bp5758d, bp1658cj and sm10bit_base (sm2235, sm2335) release the data line for the ack with `pin_mode(INPUT)`, and
`pin_mode(OUTPUT)` disables the pad input. The `sniff` local component (`ext/`) links with
`-Wl,--wrap=gpio_set_direction` and the header re-enables input on the sniffed pins. The ack bit reads the
floating line, the same on every build.
- `gen.py` writes `led.yaml`: all eight drivers on separate pins (bp5758d 25/26, bp1658cj 23/4, sm2235 18/19, sm2335
21/22, tlc5971 27/32, tlc5947 33/13 latch 14, my9231 16/17, sm16716 5/12, selftest 15/0) and a script per driver:
one window per step (open, set levels, 200 ms of main loop, close), a repeated step that must send nothing, two
direct `loop()` calls timed with `esp_timer`, and an idle window.
- `run.sh LABEL` builds, flashes, captures until `DONE` and checks the selftest. `compare.py BASE PR DRIVER` diffs
every window of one driver between two runs and decodes it (9-bit groups for the BP and SM frames, 12-bit words
for tlc5947, 16-bit for tlc5971, the two-edge interleave for my9231, 50 zero bits plus 25-bit groups for
sm16716). Loop timing varies about 3 % between builds even for drivers a PR does not touch.
- my9231 toggles DCKI with `!digital_read()`. ESP32 output pins have the pad input disabled, so without the
sniffer's input enable the read is always 0 and the clock never toggles (on dev too).
- Out-of-range channels: run `esphome config` with channel = last and last + 1 on dev and the PR.
- Results: https://github.com/esphome/esphome/pull/21844#issuecomment-6491228132
@@ -1,63 +0,0 @@
# Addressable LED strips (#21447, #21445)
- To exercise a failed strip allocation, use a classic ESP32 without PSRAM with `num_leds: 65535` and
`channel_colors: GRBW` (262 kB, bigger than any internal block). `num_leds` is uint16 in the drivers, larger
values wrap. On dev such a strip boot-loops the board.
- Never force allocation failures by hogging RAM instead: Wi-Fi then aborts in `operator new` or the loop task
trips the task watchdog.
- On the ESP32-S3 give each RMT strip `rmt_symbols: 48`, or the first channel takes all 192 symbols.
- A board with a few WS2812 on a pin already defined as a light can be extended with `id: !extend <light id>` to
add effects; restore its own config afterwards.
- Loop cost of a strip (#19337): give each strip an `addressable_lambda` effect with `update_interval: 0ms` that
changes every pixel, and add `runtime_stats: log_interval: 30s`; compare the main loop and light times per pass,
leaving out the first period after boot. The strips need no LEDs attached. A classic ESP32 has 512 RMT symbols in
all, so three strips need `rmt_symbols: 128` each.
## RMT channel and memory budget (#22172)
`scripts/rmt_budget/`: each line of `cases.txt` is a mix of strips, transmitters and receivers with their
`rmt_symbols` (`s:96`, `t:def`, `r:48`, ...) on an ESP32-S3 or ESP32-C6, and `gen_case.py CASE` turns it into a
config whose strips have an `output_id` and which logs `STATUS so0=<is_failed> tx1=...` every 3 s. `run.sh CASE...`
builds each case under one name per board (so builds stay incremental), flashes, captures 15 s and prints the STATUS
line and the RMT driver errors; it exits 1 when a case printed no STATUS. `verdict.sh CASE...` runs only
`--only-generate` and prints the config errors, the RMT memory warnings and the components registered, to compare
with what the board then does. No LEDs are needed: allocation failures show at boot.
- On the S3 USB console the ESP-IDF `no free tx channels` lines are printed before the port reopens; the C6 on
UART0 shows them.
- S3 TX channels can use RX memory (384 symbols at most) but only start on blocks 0 to 3, so two 192 symbol strips
do not fit.
- 17 of 17 predictions of the PR matched the driver. Results:
https://github.com/esphome/esphome/pull/22172#issuecomment-6615083139
## Capture the wire, not the buffer (#21577)
`templates/led_wire/wire.hpp` reads what the strips send, with no wiring:
- RMT RX channels on the `esp32_rmt_led_strip` pads at 10 MHz, decoded with a high-time threshold per chipset
(ws2812 0.4/0.8 µs: threshold 7 ticks; sk6812 0.3/0.6 µs: threshold 4);
- an SPI2 DMA slave fed from the `spi_led_strip` pads, with `spi: interface: spi3` for the strip.
`led-wire-dut.yaml` (classic ESP32) drives a GRB ws2812 strip, a GRBW sk6812 strip, an SPI strip, an RGB light and a
partition through steps S0 to S8 (on, `addressable_set`, lambda writes, partition, effect, power limit, off), logging
the decoded `RMT`, `SPI` and buffer `GET` lines and `DONE` per step, then runs `stress_step()` over 300 shows and
logs bad frames per strip. Diff the step lines of dev and the PR.
- A comparison of the buffers alone missed a PR bug in which the start frame was not sent.
- Give the captured strip `rmt_symbols` that hold a whole frame (192) and capture one RX channel at a time: with 64
symbols the TX refill interrupt stretches bits on dev and the PR alike while the capture runs. The substitutions
at the top of the config pick the strip.
## rp2040_pio_led_strip: a self-sampling PIO probe (Pico 2 W)
`scripts/pio_probe/pioscope.hpp` samples any GPIO from PIO2 at 25 MHz while another peripheral drives it (PIO reads
the pad without taking the pin) and decodes WS2812-style pulses into bit timings and bytes; no wires or LEDs.
`pa.yaml` probes five strips on PIO0 and PIO1 (WS2812 and two custom timings), `pc.yaml` one strip after claiming
DMA channels 0 to 11. `run.sh pa|pc LABEL` builds, flashes, captures 40 s and prints the `RESULT` lines; a correct
frame of two LEDs set to `Color(0xA5, 0x3C, 0x0F)` reads `3ca50f3ca50f`.
- Turn the light on at full brightness before probing, or color correction writes zeros.
- The probe FIFO overruns once in a while: count only full captures. `run.sh` fails unless every strip gave at least
one correct frame.
- An RP2350 `panic()` escalates to a HardFault and the rp2 crash handler reboots within about a second, so a panic
during component setup boot-loops before USB CDC enumerates and the board looks dead (no port, no BOOTSEL drive).
Hold BOOTSEL while plugging it in and flash the UF2 drive. To see such a fault, trigger it after boot.
@@ -1,162 +0,0 @@
# libretiny
## LibreTiny rtc preference emulation (#21007)
The PR keeps `storage: rtc` preferences in FlashDB and deletes them on the first load or save after a boot
whose `lt_get_reboot_reason()` is not software, watchdog, crash or sleep. Only the families listed in
`_libretiny_rtc_supported()` accept the config keys; on others force the backend in with
`platformio_options: build_flags: ["-DUSE_LIBRETINY_RTC_PREFERENCES"]` after confirming the rejection.
- Harness: `scripts/libretiny_prefs/ltprefs.hpp` plus `templates/bk72xx-rtc-prefs-dut.yaml`. `ltprefs_boot()`
loads one rtc and one flash preference, a second pair that shares one key as rtc and as flash, and a third
rtc entry (so a hard reset drops three entries in one pass), logs the reason and both results, saves `100 + step`, syncs; after
a templated delay `ltprefs_exit()` bumps the flash-backed step and runs the step's action (`lt_reboot()`,
`__builtin_trap()`, `lt_reboot_wdt()` or idle). The last 8 boot reports are kept in a flash preference and
repeated in the 10 s alive line, so a boot missed by the capture (a power cut, an OTA-only board) is still
readable. Bump `run:` for fresh keys.
- A store to an unmapped address does not fault on the ARM968, the board carries on: use `__builtin_trap()`
for the crash row. The SDK crash handler reboots about 10 s later. Works on BK7238 and BK7231N.
- Hard resets: `serial_capture.py ... --reset` pulses RTS on CEN (reported as Power-On). A real power cut
needs a hand or a smart plug; the history in the alive line covers the boot the host cannot see. On the
Beken SDK a chip-enable reset and a power cut both report Power-On, there is no separate reset-pin source.
- Step 0 and step 8 wait 30 s before acting so the host can open the port (and pulse a reset) after a flash
without losing the first boot; the flash tool's own reset starts the sequence immediately otherwise.
- OTA-only board: keep the device's wifi, api and ota blocks verbatim, keep safe_mode, read logs with
`esphome logs <yaml> --device <name>.local` (the API attaches late, hence the history), and restore the
device's own config afterwards.
- Plain-build comparison: `raw_firmware.bin` differs by the timestamp and by `__LINE__` in log calls of
functions below the added code; map offsets through `objdump -h` load addresses and `addr2line` before
calling a byte difference a code change.
## BK72xx crash report on the next boot (#21012)
Harness: `scripts/libretiny_crash/bkprobe.hpp` plus `templates/bk72xx-crash-probe-dut.yaml`. A flash
preference holds the step; `on_boot` (priority -100) logs the reset reason, the SDK start type, two canaries
and hex dumps of the low SRAM, a set of marker words and the record area after `.bss`, waits 30 s on step 0
(so a capture can be opened after the flash) or 5 s, saves `step + 1`, syncs, then acts: 0 `__builtin_trap()`,
1 call into unmapped memory, 2 unmapped and unaligned reads and a store (no fault on the ARM968), 3
`lt_reboot()`. A `globals: restore_value` step counter does not work here: it is only saved from the global's
loop, which a crash 50 ms later never reaches; save an `ESPPreferenceObject` and `global_preferences->sync()`
first. Bump `run:` for fresh keys. Keep the `wifi:` block: the bug the fix addresses only shows with Wi-Fi up.
Flow: `flash_bk72xx.py` (retry up to three times), one `serial_capture.py ... --reset` capture of 150 to 230 s,
then `grep -a "libretiny.crash\|probe"`. Decode with
`arm-none-eabi-addr2line -pfiaC -e .pioenvs/<name>/raw_firmware.elf <addr...>` (the ELF is `raw_firmware.elf`
on LibreTiny), or run `scripts/libretiny_crash/decode_capture.py <yaml> <capture>` from the config directory
to feed the capture through the CLI's `LogLineProcessor`.
Expected: `*** CRASH DETECTED ON PREVIOUS BOOT ***`, `Reason: Undefined instruction`, PC on the `udf`,
LR = PC + 1, BT0..BT7 walking the lambda, `StatelessLambdaAction`, `Action::play_complex`, `DelayAction`,
`Scheduler::call`, `Application::loop`. The wild call reports where the core ended up (it executes zeros as
Thumb NOPs until it wraps into real code), not the called address. Without the fix a dev build with Wi-Fi
connected never comes back from the crash; see the BK72xx section in `boards.md`.
OTA-only board: copy the device's own config, add the probe include and the step machine (step 0 crashes
45 s after boot, later steps idle), keep its wifi, api, ota and safe_mode blocks and use the secrets file
next to that config. A LibreTiny OTA is a 1.5 MB UF2 and takes minutes on a slow link, so do not wrap the
upload in a short timeout. Read the replayed report with `esphome logs ... --device <name>.local` (an API
client gets it on subscribe) or on the board's serial log port if it has one.
## RTL8720C crash report on the next boot (#21648)
`templates/rtl8720c-crash-buttons-dut.yaml` has one API button per case: `trap` (`__builtin_trap()`, usage fault),
`bus fault` (store to 0x70000000, imprecise, CFSR 0x400), `wild call` (call to 0xFFFFFFF1, MemManage fault with PC
0xFFFFFFF0 and LR in the caller), `isr trap` (a trap from a spare interrupt pended by software, hard fault), `hang`
(interrupts off, the watchdog resets it and nothing must be reported) and `restart`. Every boot logs
`boot reason N, crash record yes|no|n/a`; `n/a` means the handler is not built, as on a dev control build.
Flow per build: flash with `flash_rtl87xx_kit.py`, start one `serial_capture.py PORT 600 OUT --released --ms`, then
`scripts/libretiny_crash/rtl_drive.py HOST OUTDIR TREE YAML trap restart "bus fault" "wild call" "isr trap" hang restart`.
The `restart` after the first crash checks the report shows only once. Time each reset from the serial capture
(the ROM fault dump to the next `ROM Version` line), not from the API port: the board is back before an API client
notices it went away. Compare the reported PC with the `PC =` line of the ROM dump, and check the decoded lines in the
`caseN-*.txt` files. Pass the UF2 by name (`firmware.uf2`): the build directory holds a second identically sized copy.
Regression rerun of the same flow (#21728): `scripts/libretiny_crash/rtl_rom_vs_report.py CAPTURE` lists, for each ROM
fault dump in a `--ms` capture, the ROM's CFSR and PC, the ms to the next `ROM Version` banner and the Reason, CFSR, PC
and SP the next boot reported, so two builds compare line by line. The ROM lines start with a bare CR; split the
capture on newlines only. The imprecise bus fault's PC moves between builds (it is wherever the core was when the
fault was raised); compare it with that run's ROM dump, not with another build. SP stays the same per fault (8 word
frame, no FPU). On the BK7238 time the reboot from the button press to the next `BK7238_1.0.14` banner (0.6 to 0.7 s).
## BK7238 BLE and flash sector safety (#21735)
The Beken BLE bond store reads its partition at boot and erases that sector when the CRC does not match; before
LibreTiny 1.14 the partition was unmapped and pointed at sector 0, erasing the bootloader (#18646). To prove a change
like this safe on a board that cannot be recovered without a dump:
1. Take a full backup first: `scripts/bk72xx_flash/readback.sh full-before` (ltchiptool family `beken-7238`, about
190 s for 2 MB). Keep it until the board is back on a known build.
2. Serial flash a build without the feature, then deliver the feature build by OTA (the path users take). Use
`bk72xx: framework: sdk_silent: none` with `logger: hardware_uart: UART1`, so the SDK's own lines (bootloader
banner, `CRC fail, remove bonded info`) show up in the capture.
3. Soak with Wi-Fi up, OTA again from the new build, pulse CEN a few times (`serial_capture.py --reset`), and check
each boot comes back through the bootloader.
4. Read the whole chip again and run `scripts/bk72xx_flash/partdiff.py full-before.bin full-after.bin
<libretiny>/boards/_base/beken-72xx.json`: bootloader, calibration and userdata must be unchanged; app, download
and one KVS sector change normally.
Check what already sits in the target partition before the first boot: on an XH-WB3S that shipped with Tuya firmware,
0x1E1000 (the `ble_bonding` partition of the generic Beken layout) held the Tuya `diff2ya` record (magic `ba dc cd
ab`), which the bond store erased on its first boot. Nothing depended on it there, but say so in the results.
## BK7238 OTA and the board layout (issue #18222)
A BK7238 has one of two bootloaders, and the board in the YAML must match it: Beken OEM (`BK7238_1.0.14` in the
boot banner, `generic-bk7238`, OTA as RBL to `download` at 0x132000) or Tuya T1 2.0.0 (`generic-bk7238-tuya`,
`t1-*`, `download` at 0x12C000 plus a `diff2ya` record at 0x1E1000). `ltchiptool uf2 info` on `firmware.uf2` shows
the layout an image targets (`FAL_PTABLE`, `OTA_PART_INFO`). Measured on the OEM bootloader (2026-10-27):
| Running | Uploaded | Result |
|---|---|---|
| Tuya layout | Tuya layout | upload succeeds, bootloader prints `App verify failed! Need to recovery factory firmware.`, old build stays |
| Tuya layout | OEM layout | applied |
| OEM layout | Tuya layout | `End error: 9` (`UPDATE_ERROR_ACTIVATE`: no `RBL` at the running download offset), old build stays |
| OEM layout | OEM layout | applied |
Run it with a full backup first (`scripts/bk72xx_flash/readback.sh`), `sdk_silent: none` so the bootloader's lines
show, and each result checked against `build_info.json` over the API. A Tuya-layout build on an OEM-layout module
reads the RF calibration from 0x1E3000: copy the 0x1E0000 sector there for the test and erase it afterwards. The
Tuya OTA writes up to about 0x1EF000 and its own settings at 0x1E4000, so write those sectors back from the backup
and check with `partdiff.py`.
## RTL8710B Wi-Fi join diagnosis (issue #21742)
For a board that joins rarely: build `dev` with `sdk_silent: none`, log `wifi_get_last_error()` and
`rtw_join_status` after each failed `WiFi.begin()`, and dump the efuse map once (see the AmebaZ section in
`boards.md`). The driver log tells the stages apart: `start auth to` with nothing after it is the AP not answering,
`auth success, start assoc` then `association success` then error 3 is a failed 4-way handshake, and
`sta recv deauth reason code(6)` is the AP dropping the session mid-handshake. Tally per BSSID: if the share of
auth frames answered falls with distance while receive stays strong, the board's transmit path is weak.
A/B a value one boot at a time so time of day and AP state hit both sides equally. Keep the trial counter in a
`restore_value` global, set the value in an `on_boot` lambda at priority 600 (before Wi-Fi starts), and reboot
after a fixed window:
```yaml
esphome:
on_boot:
priority: 600
then:
- lambda: |-
// value under test, read by the code being measured
wifi::diag_value = (id(trial_idx) % 2) ? 0x3A : 0;
ESP_LOGW("trial", "TRIAL %d start value=0x%02X", id(trial_idx), wifi::diag_value);
- delay: 75s
- lambda: |-
bool ok = wifi::global_wifi_component->is_connected();
ESP_LOGW("trial", "TRIAL %d value=0x%02X result=%s", id(trial_idx), wifi::diag_value,
ok ? "JOINED" : "FAILED");
id(trial_idx)++;
App.safe_reboot();
```
Turn off `api:` and `wifi:` `reboot_timeout`, and leave out `ap:` unless the AP is the variable under test.
Do not cycle trials with `wifi.disable` and `wifi.enable`: doing that while connected reset the RTL8710B on the next
scan. Results from 2026-10-27, on one BW12 with blank efuse calibration: crystal trim `0x20`, `0x30` and `0x38`
all joined 20 of 20 trials, and boosted TX power indexes changed nothing against an AP that answered 0 of 43
auth frames.
Known LibreTiny behaviour seen here: after a join with a BSSID and channel, LibreTiny's partial-scan reset leaves
the next SSID-only join with nothing to scan (`RTW_NONE_NETWORK` in about 18 ms). That is a firmware bug, not
the board; passing channel 0 avoids it.
@@ -1,36 +0,0 @@
# GPIO wakeup from auto light sleep (#21074)
`scripts/light_sleep_wake` needs no wiring. LEDC on `LEDC_USE_RC_FAST_CLK` with
`sleep_mode = LEDC_SLEEP_MODE_KEEP_ALIVE` keeps toggling GPIO10 at 2 Hz through light sleep, and an
`ESP32InternalGPIOPin` interrupt (`ANY_EDGE`) on the same pad timestamps every edge in an IRAM handler.
`wakeprobe::report()` logs every 30 s: edges seen and expected, the shortest and longest gap, how many edges came
more than 20 ms off the half period, and the SLEEP share from `esp_pm_dump_locks()` (needs `profiling: true`).
- Native ESP-IDF builds need `include_builtin_idf_components: [esp_driver_ledc]`.
- After `ledc_channel_config` call `gpio_input_enable(pin)`, never `gpio_set_direction(INPUT_OUTPUT)`: that routes the
pad back to a plain GPIO output, the wave stops and the pad reads a constant 1.
- The C3 shares one LEDC clock across timers, so a board whose own config drives LEDs through `ledc` needs a test
config without them. Logging over the API works with `logger: baud_rate: 0`.
- Read the reports over the API with `esphome logs ... --device <ip>` under `timeout`. Run PR, dev, then the PR
again. On a C3 bulb dev caught every edge but handled 8 of 845 more than 20 ms late
(up to 45 ms), the PR none; both slept 57 to 58% of the time.
## Peripheral power-down needs an external signal (ESP32-P4)
LEDC in `LEDC_SLEEP_MODE_KEEP_ALIVE` keeps the TOP power domain on, so the self-generated wave above never tests
`power_down_peripherals`. With that option the PR's ext1 path also hands an LP pin to the LP IO block, and an LEDC
wave on the same pad no longer reaches the input (the pad reads 0).
- `wake-stim.yaml` puts a 2 Hz square wave on GPIO25 of a classic ESP32 (LEDC on REF_TICK). Wire it to GPIO4 of the
board under test; boards on the same USB host share ground.
- `wake_detail.hpp` with `wake-detail-esp32p4.yaml` (substitution `power_down`): GPIO4 is the any-edge interrupt pin
and GPIO22, jumpered to it, is read as the true level. The first 40 s hold a `NO_LIGHT_SLEEP` lock, then sleep is
allowed. Each report counts interrupt calls, real edges (the GPIO22 level changed) and calls with no change, per
phase, and two windows (from 9.5 s and from 69.5 s) log every call with its gap and both levels. A light-sleep
exit callback counts wakes when `CONFIG_PM_LIGHT_SLEEP_CALLBACKS` is set.
- Waveshare ESP32-P4-WIFI6 (pre-rev3 silicon, CH343 console): `engineering_sample: true`, `flash_size: 32MB` and
`logger: hardware_uart: UART0`, or the app logs nothing on that port. No Wi-Fi needed. GPIO4 is the left header,
12th pin from the USB end; GPIO22 the right header, 9th.
- Result on #21074 at 2ef417bd81 with peripherals powered down: the PR caught all 644 real edges (dev 66) but added
4642 calls while the line was high, one per sleep cycle (about 16 ms), with the level read correctly. Peripherals
on: both builds caught every edge. https://github.com/esphome/esphome/pull/21074#issuecomment-6503939850
@@ -1,31 +0,0 @@
# Manual IP and web handler flows (#21708, #21709, #21711, #21712)
## Station manual IP (`scripts/manual_ip/run-sta.sh`)
Set `static_ip` to the board's own DHCP lease so nothing collides, and DNS2 to a server DHCP does not hand out
(9.9.9.9), so the dump and `dns_getserver()` prove the manual values are in use. A missing SSID with its own
`manual_ip` comes first, so the real network is a later table row and inherits the top level `manual_ip`. The probe
logs both DNS servers and resolves a name every 20 s; `ipaddr_ntoa` uses one static buffer, so it uses
`ipaddr_ntoa_r` to log two addresses in one line. PASS needs the address, both DNS servers, a lookup and a ping.
## AP manual IP (`scripts/manual_ip/run-ap.sh`)
The DUT has only a missing SSID, `ap_timeout: 10s` and `captive_portal:`, so its AP comes up on 10.42.7.1; the other
board joins as a client and fetches `http://10.42.7.1/` every 10 s. A D1 mini AP drops some client requests on dev
too, so compare failure counts with a dev run and do not expect zero.
## Ethernet manual IP
Same idea on an Ethernet board: its lease as `manual_ip` plus an outside DNS2 and the DNS probe from
`scripts/manual_ip/dnsprobe.hpp`; restart it over the API and check again.
## Web handlers (`scripts/web_handlers/run.sh`)
`webbase.yaml` adds a GET /probe handler from `on_boot` at priority -100 (the shape of an external component adding
a handler late) and takes a variant: `ws` (web_server), `cp` (captive_portal only, started on the station network by
a button calling `global_captive_portal->start()`) or `prom` (captive_portal and prometheus). The runner fetches
/probe and /, then uploads the same image through web OTA (`curl -F update=@<bin> http://ip/update`; ESP-IDF
`build/<name>.bin`, not `firmware.ota.bin`; ESP8266 `.pioenvs/<name>/firmware.bin`) and checks over the API that the
board is back on that build. On dev too, prometheus GET / gets no answer on ESP32 and 404 on ESP8266.
A background `(cd dir && cmd) &` does not change the parent shell's directory: give capture scripts absolute paths.
@@ -1,18 +0,0 @@
# mDNS responder (#22267)
`scripts/mdns/mdns-common.yaml` (a package) adds a `_hwtest._tcp` service with a short and a 62 character TXT
value and logs free, minimum and largest free heap every 10 s; `mdns-wifi.yaml` adds Wi-Fi from `secrets.yaml` and
switches it off and on every 120 s so the responder restarts. The host's multicast often does not reach the boards
(a host with several network interfaces, or a mesh Wi-Fi network), so the scripts send legacy unicast queries (source port not 5353)
straight to the device's port 5353 with python-zeroconf packets:
- `mq.py IP HOSTNAME dump` asks for the `_esphomelib` and `_hwtest` PTR records, the A record and the SRV and TXT
records of both services, prints the answers and exits 1 when one is missing. Diff the dump between builds.
- `mq.py IP HOSTNAME hammer N [TIMEOUT]` sends N questions back to back (0.5 s timeout each) and counts misses per
question. It exits 1 when nothing was answered, or when more than `MAX_MISS_PCT` percent (env) were missed.
- `mq.py IP HOSTNAME lat N` sends A queries 300 ms apart and prints the median, p90, p99 and maximum latency and how
many answers came after 1.5 s or never.
Judge responder timing on an Ethernet board: on a crowded Wi-Fi network back-to-back query loss swung between 0 and
14% between builds for no code reason. Run the hammer across a Wi-Fi off and on cycle too, and compare the heap
lines before and after many cycles.
@@ -1,26 +0,0 @@
# microphone_http_stream (#21593)
`microphone_http_stream` serves raw microphone PCM at `/audio/microphones/<stream_id>` (and a listing at
`/audio/microphones`) on the web server. `templates/microphone_http_stream/`:
- `mh-base.yaml`: Waveshare ESP32-S3-ETH on PoE (logs over the API on Ethernet, the native USB port is the USB host
port), encrypted API, `web_server` on 80 with a dummy sensor and switch, mic start and stop buttons and a line
every 5 s with the mic state and internal and PSRAM heap. Any ESP32-S3 with PSRAM works for `mh-fake.yaml`.
- `mh-fake.yaml`: the stream fed by `ext/fakemic`, a test microphone that publishes 32 ms chunks of a 16-bit counter
(one count per sample) from its own task, so a reader can check every sample. Its "Mic stall" button stops the
chunks for 6 s.
- `mh-usb.yaml`: the stream fed by a USB microphone on `usb_audio` (48 kHz, 16-bit stereo, channel 0 streamed).
Copy the directory next to a `secrets.yaml` with `api_key`, flash, then run
`mhclient.py HOST STREAM_ID [--clients N] [--seconds S] [--pattern] [--stall K] [--stagger S] [--expect-reject R]`.
Each reader prints its HTTP status, format headers, sample count and rate, the longest gap between reads and, with
`--pattern`, every break in the counter. `--stall K` makes reader K stop reading after 2 s while keeping its socket
open, then checks that the device closed it. `--expect-reject R` marks the last R readers as past `max_clients`;
they must get a non-200 status or a close or reset before any samples, and a timeout or a served reader fails.
The script exits 1 when the listing is not a 200 naming the stream, the stalled reader was not dropped, a reader past
the limit was not refused, or any other reader got no 200, no samples, an early EOF, a read error or a broken counter.
Cases worth running on each head: one reader with `--pattern` (rate must match the sample rate, 0 breaks), the
`max_clients` limit plus one with `--expect-reject 1` (note what the extra one gets; the others must be unaffected), a stalled reader among
three (the device should drop the stalled one only), the "Mic stall" button during a read (a gap of about 6 s, then
the counter continues), mic start and stop while readers are attached, and the heap line after many connect and
disconnect cycles. On the USB microphone, check the rate and `rms` instead of the pattern with `--rate 16000 --min-rms 1`, and add `--max-gap-ms` to fail on long gaps; these set the exit code.
@@ -1,18 +0,0 @@
# modem
## PPPoS modem component with no modem attached (#6721)
A boot check on a classic ESP32 devkit with the modem UART pins left open. `templates/modem/modem-common.yaml` is
the package (`api:`, debug `loop_time` and `free` sensors, uptime, buttons for `modem.enable`, `modem.disable` and
`modem.send_at`, a 20 s alive log); `modemo.yaml` is the modem only build with `reboot_timeout: 2min` and the
component test config's `on_boot` block at `priority: -100` (earlier, `modem.send_at` runs before the modem's
`setup()` and logs `Modem DCE not ready`); `modemw.yaml` adds `wifi:` with `reboot_timeout: 0s` so the API stays
reachable. Expect `Modem not responding at 115200 baud` once per pass with each pass blocking the loop about 1 s,
the reboot at the configured timeout (`SW_CPU_RESET`), and a `modem.disable` with no modem answering to block
the loop about 37 s (`Failed to leave data mode`, then the radio off timeout) without tripping the 30 s watchdog
floor because the component feeds it in between. Drive the Wi-Fi build with
`scripts/modem/mdrive.py HOST LOGFILE list press:NAME wait:S` (plaintext API; prints sensor and text sensor
states, saves the device log). Before 8cc8df4a1f a modem only config with `api:` failed validation because a
config dependent `AUTO_LOAD` added `network` after the dependency check; an explicit `network:` block was the
workaround. Results: https://github.com/esphome/esphome/pull/6721#issuecomment-6347395159 and
https://github.com/esphome/esphome/pull/6721#issuecomment-6348520416
@@ -1,141 +0,0 @@
# mqtt
## MQTT (#20433, #17935, #20762, #20767, #20768)
A local mosquitto with `allow_anonymous true`, `log_type all` and `log_dest file`: the broker log gives
each subscription's QoS, the device address (`New client connected from <ip> as <name>-<mac>`) and exact
subscribe counts. Kill and restart mosquitto to exercise `on_disconnect` (reason 0) and resubscription.
Keep `reboot_timeout: 0s` on `api:` and `mqtt:`. The ESP-IDF outbox swings free heap about 2 kB right
after bulk subscribe and unsubscribe; read the heap seconds later over several rounds before calling a
leak. Spawned subscriptions need about 2.5 s to reach the broker before the next publish.
`esphome logs --device MQTT` only uses the MQTT transport when the config has no `api:`. Driver pattern: a
paho script publishes and polls the serial capture file.
## WebSocket transports on ESP-IDF (#16056)
One build per transport from `templates/mqtt-transport-dut.yaml`, with `scripts/mqtt/mosquitto-ws.conf` as the
local broker (mosquitto 2.x from brew has `websockets` support; `log_type all` prints `New connection from <ip>:<port>
on port <listener>`, which proves which listener each client used) and `scripts/mqtt/make_test_ca.sh BROKER_IP` for
the TLS listener's certificate, both run in the scratch directory so keys, logs and outputs stay out of the checkout. `scripts/mqtt/cases.sh NAME KEY BROKER_HOST SUB_PORT CAPTURE [DEV_HOST] [restart]` subscribes
to `NAME/#`, sends a switch command and an `on_message` topic, presses the 3000 byte publish button over the API,
restarts the local broker when the last argument is `restart` (the PID in `mosquitto.pid`, written when the broker
was started with `mosquitto -c mosquitto-ws.conf & echo $! > mosquitto.pid` in the working directory next to the
certificates), checks every listener in the broker config answers again, and prints the broker's message counts with the device log. `mosquitto_sub` cannot speak
WebSocket, so it subscribes on the plain listener of the same broker. Cases: `tcp` on 1883, `ws` on 9001, `wss` on
9002 with `certificate_authority` (the certificate names an address, so `skip_cert_cn_check: true`), `wss` to
test.mosquitto.org on 8081 with no CA for the certificate bundle, and two negatives that must never connect: a
different CA (`mbedtls_ssl_handshake returned -0x2700`) and the self-signed server through the bundle
(`esp-x509-crt-bundle: Failed to verify certificate`, `-0x3000`). `CONFIG_MQTT_TRANSPORT_WEBSOCKET` is on by
default in the generated sdkconfig. The TLS session costs about 29 kB of internal heap while connected; the public
broker carries other devices' `esphome/discover` traffic, which is noise. Results:
https://github.com/esphome/esphome/pull/16056#issuecomment-6301034019
## TLS trust sources on ESP-IDF (#14318)
One build per trust source from `templates/mqtt-tls-dut.yaml`: broker only (port must resolve to 1883),
`certificate_authority` with no port (must resolve to 8883), `verify_ssl: false` with no CA, the CA plus
`client_certificate` and `client_certificate_key` on the 8884 listener that requires one, `transport: wss` with the CA,
`tls: true` to test.mosquitto.org on 8886 for the certificate bundle, and `tls: true` to the local self-signed broker as
the negative. `scripts/mqtt/mosquitto-ws.conf` now has the 8883 and 8884 listeners and `scripts/mqtt/make_test_ca.sh`
also writes `client.crt`/`client.key`. Before flashing, check each build's `src/esphome/core/defines.h` for
`USE_MQTT_TLS`, `USE_MQTT_CERT_BUNDLE` and `USE_MQTT_TLS_INSECURE` and its `sdkconfig.<name>` for
`CONFIG_MBEDTLS_CERTIFICATE_BUNDLE` and `CONFIG_ESP_TLS_INSECURE`: only the bundle builds may carry the bundle and only
the unverified build the insecure flags. `scripts/mqtt/flash_and_drive.sh TREE YAML PORT KEY BROKER_HOST
local|public|neg` runs one build end to end (flash, bounded capture, `cases.sh` with a restart for the local broker,
counts and the broker's per-listener connection lines) and prints the config dump's `TLS:` line (`CA certificate`,
`certificate bundle` or `no verification`). The CLI side (`esphome/mqtt.py`) is covered without a board by
`scripts/mqtt/cli_tls_check.sh TREE [neg] YAML...`, which runs `esphome clean-mqtt` under a timeout: status 124
after `Connected to MQTT broker!` is a pass, and with `neg` each config must instead fail with
`CERTIFICATE_VERIFY_FAILED` and never connect. Python's default SSL
context rejects a CA that lacks a `keyUsage` extension (`CA cert does not include key usage extension`) while mbedTLS
on the device accepts it, which is why the test CA carries `basicConstraints` and `keyUsage`. A broker kill shows as one
`Last esp-tls error: 0x8008` on every build, TLS or not. Results:
https://github.com/esphome/esphome/pull/14318#issuecomment-6311863827
## MAC suffix topics (#9315)
`scripts/mqtt/mac_suffix/run.sh X BUILD` compiles (from `TREE`) and flashes variant X (a defaults, b custom `topic_prefix`, c explicit topics,
d no suffix) of BUILD (`pr` or `base`) on an ESP32 and a D1 mini, records the broker with
`mosquitto_sub -F '@H:@M:@S r=%r %t %p'`, presses Restart through the `cmd_t` found in the discovery payloads, then
holds both chips in the bootloader for 40 s (`esptool --after no-reset chip-id`) so the broker sends the will; a
plain reset only gives "session taken over" and no will. The two boards' configs live in separate folders because
variants a to c share a `name` (the MAC suffix tells them apart), and the same name in one folder shares a build
directory. A D1 mini CH340 can garble the serial capture after an RTS reset, so read its topics from the broker.
`esphome logs --device MQTT` on a current tree needs paho-mqtt 2.x; an older venv with 1.6.1 fails, so use a venv
made from the tree's requirements.txt. A Restart press never delivered the shutdown message on any build tested.
## Shutdown message delivery (#21566)
`scripts/mqtt/shutdown/`: `sd-common.yaml` sets birth/will/shutdown payloads `hello`/`dead`/`bye` so the shutdown
message and the will cannot be confused, and takes `restart`, `disable` and `sleep` on `<name>/cmd`.
`sdrun.sh V N` (`TREE`, `P32` and `P66` in the environment) compiles and flashes the ESP32 and D1 mini, restarts each N times, disables MQTT once and deep sleeps the ESP32 three
times (never the D1 mini: without the GPIO16 to RST wire it never wakes); `match.py OUT` counts a `bye` within 5 s of
each command. Dev: ESP32 0/10 restarts, D1 mini 1 to 3/10. Put the broker address (`broker_ip`) in the secrets.yaml next to
the included package, and quote `"${name}/status"` inside flow mappings.
Results: https://github.com/esphome/esphome/pull/21566#issuecomment-6381448061
Retest at c9cf3c2813 (shutdown message sent at QoS 1, teardown waits for the broker's ack): 10/10 restarts on both
boards and 3/3 deep sleeps, against 1/10, 3/10 and 0/3 on base. An earlier 100 ms hold gave 3/10, 6/10 and 2/3, so
judge a delivery fix by counts over 10 or more restarts, never one or two presses.
Results: https://github.com/esphome/esphome/pull/21566#issuecomment-6382076613
## MQTT discovery against a throwaway Home Assistant (#21956)
Run a Home Assistant dev install with its own config directory (`hass -c <scratch>/ha`) holding `http:
server_port: 8129`, `api:`, `config:`, `onboarding:` and MQTT debug logging, plus a local mosquitto on 1883. In
`scripts/mqtt/ha_discovery/`, `onboard.py DIR` creates a generated test user (the credentials stay in DIR) and a
token, `flow.py TOKENFILE` adds the MQTT entry (the `protocol` and `other_settings: {set_client_cert: false,
set_ca_cert: "off", transport: tcp}` fields are required), and `hadrive.py TOKENFILE NAME` reads the device's time,
date and datetime entities, calls `set_value` on each and reads again; it exits 1 on a missing entity, a failed
call or a state that does not read back the value that was set. The three share the REST calls in `ha_rest.py`. `dt-common.yaml` is the device side (classic ESP32, three
template datetime entities, `${broker}`).
- The HA dev branch keeps a YAML port change as a pending HTTP config and reverts it after 5 minutes. Before
starting HA again, set `data.stable.server_port` to 8129 and `data.pending` to null in `.storage/http`.
- ESPHome MQTT unique ids are name based: clear the previous build's retained discovery (`mosquitto_pub -r -n` on
each config topic) and expect the entity ids of the first build.
## mqtt.disable and broker drops on ESP-IDF (#22230)
`scripts/mqtt/disable/disable-common.yaml` disables MQTT for 30 s every 60 s (every third cycle starts with fast
toggles, a disable while connecting), publishes a tick every 2 s and logs `EV CONNECT`, `EV DISCONNECT` and
`EV MSG`. `run.sh LABEL outage|drop SECONDS` (env `TREE`, `PORT`, `SC`, `PY`, `BROKER`, `BROKER_PORT`) starts a
private mosquitto so it can stop and start it, flashes and captures:
- `outage` publishes to `md<LABEL>/in` every second and stops the broker inside a disabled window; `analyze.py`
prints per window the device connects, inbound messages, broker connects, ticks that reached the broker and the
time from enable to connect, and fails the run on a device connect or an inbound message inside a window.
- `drop` stops the broker 6 s after every enable for 15 s. This is the case that found a watchdog reset: calling
`esp_mqtt_client_stop()` while the esp-mqtt task sleeps in WAIT_RECONNECT blocks the caller for up to
reconnect_timeout/2 (5 s), 6 of 6 task watchdog resets on an S3 and a C6.
Both modes also fail when the device never connects or the broker was never stopped, and `outage` when no inbound
message arrived.
Use the host's current address for `BROKER`: a stale one gives `esp-tls select() timeout`. Before #22230 the client
reconnects on its own during disable (14 of 16 windows). Results:
https://github.com/esphome/esphome/pull/22230#issuecomment-6614854671
## Retained message burst at connect (#22200)
`scripts/mqtt/retained_burst/`: `burst-topics.yaml` (a package) subscribes 25 `on_message` topics plus 15 switch
command topics, `burst-common.yaml` logs the counts 15 s after the connect, and `seed.sh BROKER_PORT` leaves all 40 retained on a local mosquitto (`seed.sh BROKER_PORT clear` removes them).
`run.sh LABEL BOOTS` (env `TREE`, `PORT`, `SC`, `PY`, `BROKER`, `BROKER_PORT`, default board an ESP32-C3) resets the
board BOOTS times and prints `RESULT got=x/25 switches_on=y/15` and any `Dropped` lines per boot; it passes only with
40 of 40 on every boot. Before #22200 the C3 dropped exactly 49 events (80 events minus 31 slots): the main loop
drains nothing during the burst.
## Forced reconnect mid-burst (#22248)
`scripts/mqtt/reconnect/`: `kick-common.yaml` includes the same `burst-topics.yaml` and counts per connection with
a `mode: restart` script on `on_connect`, logging `RESULT` 6 s after each connect. It and `run.sh` use files from
`../retained_burst/`, so copy the whole `mqtt/` directory to the scratch directory. `kick.py BROKER_LOG BROKER_PORT
PREFIX CYCLES` tails the broker log and takes over the device's client id 0 to 400 ms after every other connect
(mosquitto drops it with "session taken over"), then leaves the next connection alone for 8 s and ends it. It
starts by kicking the connection that is already up and skips its own connections (from 127.0.0.1 with the same id).
`run.sh LABEL CYCLES` (env as above plus `BROKER_LOG`) seeds, flashes, waits for the first connect and runs the
kicker. The device counts its first connection as CONN 1, so the kicked ones are the even CONN numbers and the
left-alone ones the odd numbers from 3 up; kick.py prints each left-alone connection with its time and CONN number,
and `judge.py` pairs those with the RESULT lines by time and passes when every one has `got=25/25`. A reset after
the first connect fails the run. Retained switch commands fire `on_turn_on` only on the first connection, so
`switches_on` is not judged.
@@ -1,103 +0,0 @@
# nrf52
## nRF52840 dongle boards (#19009, #20814)
Test config: the two LEDs as inverted GPIO outputs, the button as an inverted pull-up binary sensor, a
10 s alive interval. To retest the nrfutil download path, move the cached `sdk-nrf/nrfutil` aside first.
## ItsyBitsy over UF2 when serial DFU fails (#20481)
Serial DFU to the ItsyBitsy nRF52840 fails about half the time ("Target is not in DFU mode"). Reliable path: a
1200 baud touch on the app port, wait for the `ITSY840BOOT` volume, convert `zephyr/zephyr.hex` with the SDK's
`zephyr/scripts/build/uf2conv.py -f 0xADA52840 -c` and copy the .uf2 to the volume (the copy ends in an I/O error
when the board reboots, which is normal). macOS BLE scans give UUIDs, not MAC addresses.
## zephyr_ble_server connection slot under reordered callbacks (#21723)
`scripts/zephyr_ble_server`: the ItsyBitsy runs `server.yaml` (one connection slot) and an ESP32 runs `client.yaml`,
which connects, drops the link 300 ms later and reconnects as soon as the server advertises again. `run.sh NAME 90`
flashes the UF2, counts the server's `Connected`/`Disconnected`/`Advertising` lines and the client's connects,
and ends with PASS (more than one server connection) or FAIL; an empty capture or a silent client fails the setup.
- The connect and disconnect callbacks each `defer()` to the main loop and the bug needs the disconnect handled
first, which cannot be forced over the air. Force it with a test-only patch on both trees: replace the connect
callback's `defer(...)` with `set_timeout(5000, ...)`. A leaked reference then shows as one connection and nothing
after it (the dump still lists the closed link); a fixed build keeps cycling (17 in 90 s on #21723).
- `dfu: true` makes the next flash a 1200 baud touch; an app built without it needs a double tap on RESET. The
Zephyr app enumerates as `2fe3:0100`, the bootloader as `239a:0051`, so find the board by its USB serial number.
- macOS is a poor central here: it keeps old bonds with a board (`Peer removed pairing information` on connect)
and clearing them is a system setting. Use an ESP32 `ble_client`; the server's random static address comes from
its own log.
## SMP over UDP OTA over Thread, with reset faults (#17245, #21910)
Setup: a Thread border router (see openthread.md) whose OMR prefix the laptop routes to, and an ItsyBitsy nRF52840
(the Adafruit bootloader carries the MCUboot layout; the nRF52840 Dongle's Open DFU bootloader is rejected for
`zephyr_mcumgr` OTA). `scripts/smp_udp_ota/smp-udp-ota-dut.yaml` joins the network with `force_dataset: true` and
logs its build label every 10 s. The first image goes in over UF2, every later one with
`ota.sh MODE YAML VER ADDR` to the device's OMR address. A full image (about 380 KB) takes about 130 s.
- An ESP Thread Border Router board running ESPHome `openthread: border_router: rcp:` works as the border router
(pins in `boards.md`). Give both it and the device a fixed dataset with `force_dataset: true`; each keeps an old
stored dataset otherwise. The host learns the route to the OMR /64 from the border router's RA; ping the
device's OMR address before the first upload.
- Install the smpclient version pinned in requirements. An older one fails with `unexpected keyword argument
'timeout_s'` and a missing `get_max_cbor_and_data_size`, which looks like a PR bug. Check the version with the
Python of the venv that runs ESPHome, not a system pip.
- The build holds two `zephyr.uf2` files. Copy `.pioenvs/<name>/zephyr/zephyr.uf2` (the app, from 0x26000 up,
about 827 KB). `mcuboot/zephyr/zephyr.uf2` (about 70 KB) lands on the app slot and the board stops enumerating;
double tap RESET and copy the right one.
- To log the device's addresses, format `otIp6GetUnicastAddresses()` by hand: `network::IPAddress::str_to()`
does not link on nRF52 (no `inet_ntop`).
- A new image refuses the next upload (`No free slot. Testing mode but not confirmed yet.`) until it confirms itself.
Wait about 4 minutes between uploads.
- An image that is already in the secondary slot is only marked and reset (a few seconds). Change `ver` for every
upload that has to transfer, and expect a host-only change to rebuild byte-identical firmware.
- `inject.py` faults the reset on the host: `lostreply` drops the reply of a reset the device got, `lostreq` drops the
first request, `lostreq2` drops every reset. The device resets after a short delay, so even after a lost reply the
client usually still finds the old image running with the new one pending and resends the reset to the old image,
which is safe. `lostreq2` must end in an error with the old image still running. `ota.sh` ends with `check.sh`,
which prints PASS or FAIL for the mode, and `inject.py` fails the run when no reset was faulted.
- Without `libc_nano` the config overflows the 396,800 B slot by about 4 KB with Zephyr's TCP stack linked (about
11 KB of it, unused by SMP over UDP). With TCP left out (#21910) it fits with the full libc. A config with `api:`
keeps TCP.
- **The real OTA size limit is lower than the link limit.** The nRF Connect SDK partition manager overrides the device
tree, so read `partitions.yml` in the build directory, not `zephyr.dts`. On the ItsyBitsy layout `mcuboot_primary` is
0x61000 (97 sectors of 4 KB, 0x2f000 to 0x90000) and the app partition 0x60e00 (396,800 B); `mcuboot_secondary`
is 0x90000 to 0xf1000 and `settings_storage` 0xf2000 to 0xf4000. MCUboot swap using move keeps one sector for
its trailer and one for the move, so it only swaps in an image up to (97 - 1 - 1) × 4 KB = 389,120 B
(`app_max_sectors()` and `app_max_size()` in bootutil `swap_move.c`). An image between 389,120 and 396,800 B links,
uploads and the CLI reports `Successfully uploaded program`, but MCUboot never swaps it in: it stays pending in the
secondary slot, and every later upload and the SMP image erase fail with `NO_FREE_SLOT`. OTA stays stuck until
the slot is cleared over USB. Measured: 388,787 B swapped in, 392,155 B did not. `ota.sh` checks `app_update.bin`
against this limit with `mcuboot_slot.py limit` before it uploads, and afterwards asks the device which image runs
(`mcuboot_slot.py active`, by the image's SHA-256), since the CLI's exit code alone is not proof. That check passes once the
new image runs and says whether it has confirmed itself; add `--expect confirmed --wait 300` to prove it stays.
- With #17245 at 03d45d1aaa the build itself fails above the limit (`The application image is N bytes, but MCUboot
can only swap in L bytes on this board`), and after a confirmed reset the client waits for the device and logs
`runs the new image` or fails with `rebooted but did not swap in the new image`. After a dropped reset reply it still
ends with `The reboot of ... is not confirmed` without that wait, so `ota.sh`'s own image check is what proves those
runs.
- Clearing a stuck pending image: the SMP image erase is refused while the image is pending, and resets do not help.
Over USB, touch the app's CDC port at 1200 baud to enter the UF2 bootloader, then copy the 16-block UF2 from
`mcuboot_slot.py blank-uf2 BUILD OUT`, which writes 0xFF over the secondary slot's header page (`mcuboot_secondary`
in `partitions.yml`, 0x90000 on the ItsyBitsy). The bootloader resets on its own once the last block is written
(`cp` may report an I/O error then, which is normal), MCUboot finds no image header and clears the slot, and the
image states show slot 0 only.
- Over USB, the Adafruit UF2 bootloader only accepts blocks inside its app area (its `CURRENT.UF2` maps 0x1000 to
0xE9000). A UF2 with a block outside that area is never completed, and the bootloader then ignores every
later file, even a good app UF2, until a physical reset (double tap RESET or replug). Keep hand-made UF2 writes inside the app area.
- Results: https://github.com/esphome/esphome/pull/17245#issuecomment-6516152182,
https://github.com/esphome/esphome/pull/17245#issuecomment-6518292041 (swap limit),
https://github.com/esphome/esphome/pull/17245#issuecomment-6519010038 (retest)
## TCP stack only when a component uses TCP (#21910)
`scripts/nrf52_tcp_probe/tcp-probe.yaml` joins the test Thread network with no TCP user and opens a TCP and a UDP
socket twice from an interval. Build it per tree with `-s name NAME`, flash over UF2, capture the USB log, then
`check.sh BUILD CAPTURE`: it reads `CONFIG_NET_TCP` and the packet and buffer counts from
`.pioenvs/<name>/zephyr/zephyr/.config` and wants `tcp null udp ok` twice plus the `TCP stack not built` error exactly
once on a build without TCP, or `tcp ok udp ok` twice and no error with it. Add `api:` to see TCP and the larger pools
come back, and connect with `apicheck.py ADDR` over Thread. Measured on the ItsyBitsy: 12.9 KB less flash and 17.1 KB
less RAM without TCP (Zephyr's default pools, 16/16 instead of 24/48), and SMP over UDP OTA still works with them.
Results: https://github.com/esphome/esphome/pull/21910#issuecomment-6518287139
@@ -1,64 +0,0 @@
# online_image
## online_image stalled downloads (#20594)
One classic ESP32, both frameworks (the ESP32 Arduino build also uses `http_request_idf.cpp`). The host
serves the images: `scripts/online_image/make_images.py <www>` writes 16x16 BMP/PNG/JPEG/QOI plus an 8 bit
palette BMP with a 1078 byte header; `scripts/online_image/oi_server.py <www> 8765 <log>` answers one
misbehaviour per path name (`ok`, `truncated`, `large_header`, `empty`, `stall` holds the socket 40 s,
`closed_early`, `slow` pauses 2.5 s, `chunked`, `chunked_cut`, `noclen`, `huge` claims 20 MB) with an ETag
and a 304 on If-None-Match. `templates/online-image-dut.yaml` has two AUTO-format images (default and
256 byte buffer), API actions `fetch`/`fetch_small` (set_url with `update: false`, then component.update)
and `refetch`/`refetch_small` (component.update only, keeps the cache headers), and a restart button.
`scripts/online_image/drive.py <device ip> <api key> <capture> http://<host ip>:8765 pr|base` runs the
case list over aioesphomeapi and judges each from the serial capture. ESP-IDF facts: a stalled server
ends with `Error downloading image: -28679` (EAGAIN) after the transport timeout, an early close with
`-1`, both before and after the PR; a body without Content-Length and without chunking reads as complete
(Size: 0) and finishes. A build without the fix hangs on the truncated, oversized header, empty, chunked PNG
and cut chunked cases and every later update logs `Image already being updated.`, so a baseline run needs
a reboot per group, the driver's 45 s cap per case, and an outer `timeout` that covers the whole list.
A case expected as `any` passes on `finished` or `error`, never on no outcome.
## PNG decode time limit (#21364, #21391)
The same server also serves any plain file in its www directory by name (404 for a missing file or a path
outside it). `scripts/online_image/make_big_pngs.py <www>` (after `make_images.py`) writes a 320x240 gradient (618 B, also as `ok.png` so the misbehaviour cases carry a
real multi-row PNG), a flat 640x480 PNG (1.6 KB for 307200 pixels), 231 KB of noise and a JPEG. The device is
`templates/online-image-dut.yaml` with `type: BINARY` on the main image (a 640x480 RGB565 does not fit a classic
ESP32 without PSRAM) and `debug` loop time and heap sensors at 2 s; the 256 byte buffer image stays RGB565, so do
not point big images at it (out of memory on every build). `scripts/online_image/pngdrive.py HOST KEY LOGFILE
BASEURL LABEL` (default case set `png`; `CASES=short` for the six case list) calls the actions, waits for the
finished or error line and reports the case time, the highest loop time and the lowest free heap. The per pass `online_image.image took a long time` values come from
the serial capture and only print on a new maximum, so read the first fetch of each image. The pass that decodes a
whole buffer is the decoder cost (before the change 510 ms gradient and 1980 ms flat, after it 77 and 140 ms); for
the noisy PNG the longest pass is the blocking 64 KB read on both builds (0.7 to 3.7 s). Alternate PR, base, PR
for a timing claim.
For a replacement PR, compare against the parent PR head, not dev. `CASES=noise` fetches the noisy PNG three times
(run it right after a boot, alternating builds); `CASES=end` runs flat and gradient as plain, `noclen_`, `chunked_`,
`half_` and `stall_` through the prefixed server cases below.
## JPEG decoded as it downloads (#21342)
`oi_server.py` serves prefixed cases on any file: `half_<file>` (first half, matching Content-Length),
`noclen_<file>` (close delimited), `chunked_<file>` (1 KB chunks), `stall_<file>` (a third, then the socket held
40 s). `scripts/online_image/make_jpegs.py <www>` writes noise, gradient, 12 KB EXIF and progressive JPEGs. Device:
`templates/online-image-dut.yaml` with `type: BINARY` on both entities and `buffer_size: 4096` on the second;
`CASES=jpeg scripts/online_image/pngdrive.py HOST KEY LOGFILE BASEURL LABEL` runs the JPEG list; every case set
prints case time, highest loop time and lowest free heap. Before the change a JPEG needs a contiguous block of its file size (233 KB never
allocates on a classic ESP32, and after a few failures 69 KB does not either until a reboot); after it the 64 KB
buffer stays 64 KB and 4 KB is enough for a baseline JPEG. The first pass still reads up to `buffer_size` in one
blocking read, so the 64 KB entity blocks 1.7 to 4 s and the 4 KB one 272 ms at most; judge the decode budget on
the small buffer entity. Already on dev: a JPEG without Content-Length (plain or chunked) ends with `Could not open
image for decoding: 4` then `Invalid type`.
## BMP and QOI decode time limit (#21392)
Use a wired board (`templates/online-image-s3-eth-dut.yaml`: Waveshare ESP32-S3-ETH, W5500, PSRAM, `type: RGB565`)
rather than a classic ESP32 on Wi-Fi: with a weak link the reads block up to the 4.5 s timeout and the per case loop
numbers are noise. `scripts/online_image/make_bmp_qoi.py <www>` writes 1 bit BMP 640x480, 8 and 24 bit BMP 320x240,
QOI 640x480 one colour (all runs) and checkerboard, QOI 320x240 gradient and noise. `pngdrive.py` with
`CASES=bmpqoi` runs each as plain, `noclen_`, `chunked_`, `half_` plus stalls. The `loop_time` sensor sits at 75
to 160 ms even for a 16x16 image, so for decode passes use `scripts/online_image/firstpass.py HOST KEY BASEURL
LABEL IMAGE...`: it reboots the device before each fetch and reports that fetch's `took a long time` maximum. A
`half_` file with matching Content-Length reports `download finished` with "not caching it" on dev too.
@@ -1,5 +0,0 @@
# openthread
## Thread border router (#18104)
See boards.md for the ESP Thread BR board. Check that the H2 answers Spinel before blaming a PR.
@@ -1,86 +0,0 @@
# ota
## OTA overlap and lock (#20588), OTA timeouts (#19041), deflate (#19037)
All overlap orders (a web upload during a native OTA and the reverse) must be refused; the lock release
after an interrupted web upload waits about 14 s (ESP8266) or 7 s (ESP-IDF) for disconnect detection. A
refused web client on `web_server_idf` never gets the busy reply; the connection drops at reboot.
`pkill -f "a\|b"` does not alternate; use two pkills.
## OTA upload timing A/B (#21639, #21641, #21642)
`scripts/ota_timing/run.sh N PHASE...` times native OTA uploads per build (phase) and per board. It builds every
board from `TREE_<PHASE>`, does one warm-up upload per board (it installs the phase's build and is left out of the
results), checks over the API that each board reports the compile time from that build's `build_info.json`
(`apiinfo.py`), then runs N rounds of every mode on every board. Boards are interleaved per round so they share the
same air; put a nearby ESP32 on the same Wi-Fi in `BOARDS` as a network control. `summ.py` prints medians: never
judge from single runs, slow Wi-Fi gave 33 to 235 s for the same upload in one night.
- `otatime.py` drives `espota2.run_ota()` from the tree on `PYTHONPATH` and prints one `RESULT` line with the
`Update took` split, the compressed size, whether UDP was used and the time at each tenth of the upload.
An upload that leaves no `RESULT` line (board gone, script crash, timeout) is recorded as `rc: -1`, and `summ.py`
fails when a board and mode has fewer than N rows or an upload time cannot be read.
- Boards with an `api: encryption: key` also encrypt OTA: set `API_KEY` for `otatime.py` and `apiinfo.py`. LibreTiny
LN882H listens for OTA on 8820, not 8892; BK72xx and RTL87xx use 8892.
- Raw mode leaves `CLIENT_FEATURE_SUPPORTS_DEFLATE` out of the client's offer. Never switch deflate off after the
offer (`if deflate and 0:`): once the device accepted deflate it expects a deflate stream.
- `-tcp` modes make `espota2._loss_reason` return None, so the client asks to stay on TCP. Clearing
`CLIENT_FEATURE_SUPPORTS_UDP` instead desyncs the handshake (`receiving UDP token response: timed out`).
- LibreTiny trimmed UF2 (#21642): the CLI sends what `esphome.components.libretiny.uf2.device_ota_uf2()` returns for
`firmware.uf2`; that module only exists with #21642, so on older trees the full UF2 goes out. Compare the sizes
in the `Compressed to` lines, and rely on the API compile time check to show that the trimmed image booted.
`receiving chunk result: Unknown error from ESP` is error 0xFF sent by the device, not a dropped connection. The
device sends it for a decode or flash write error, and also when no data arrives for 105 s, so on a weak link it
usually means a stall. A failure about 105 s after the last progress, or the same error on a raw upload, points to
the link; to be sure, capture the device log (`Inflate err`, `Inflate overrun`, `No data received`).
Results: https://github.com/esphome/esphome/issues/21639#issuecomment-6409628227 (dev and LibreTiny 1.14.0 on a
CB3S), https://github.com/esphome/esphome/pull/21642#issuecomment-6412972562 (trimmed UF2 on CB3S, BK7238 and BW15),
https://github.com/esphome/esphome/pull/21641#issuecomment-6412976859 (new decoder on the same three).
## On-device inflate benchmark (#21639, #21641)
`scripts/ota_timing/inflate_bench`: `gen_streams.py NAME=IMAGE...` cuts a 128 KB slice from each image and writes
`bench_streams.hpp` with raw deflate streams (level 9, 4 KB and 32 KB windows). `bench.hpp` inflates every stream with
each ring size at least the encoder window, three times, and logs the best time plus input and output rates, then
`BENCHDONE`. `bench.sh CONFIG DEVICE` builds and uploads `bench-esp32s3.yaml` or `bench-bk72xx.yaml`, collects one pass
over syslog and fails when a case did not reach the end of its stream.
- LibreTiny defines `micros` as a macro, so the header calls `(esphome::micros)()`.
- Keep the streams near 130 KB on a BK7231N: an image with about 330 KB of extra incompressible data failed its
OTA with "Writing OTA data to flash memory failed". Use one image and `--windows 12` there.
- `bench.hpp` targets the decoder interface with `dest_limit`. #21641 drops it (the output runs into the window up
to `dest`); after that lands, remove the `dest_limit` line.
Results: https://github.com/esphome/esphome/issues/21639#issuecomment-6409786224
## Upload without a configuration (#21738)
`esphome upload --file IMAGE --device IP` with no YAML picks the OTA port from the image header and takes the password
from `--ota-password` or `ESPHOME_OTA_PASSWORD` and the key from `ESPHOME_OTA_KEY`. Test each chip family with its own
current image (`build/firmware.ota.bin` for ESP-IDF, `.pioenvs/<name>/firmware.bin` for ESP8266, `firmware.uf2` for
LibreTiny) and check the build over the API; switch one board between two builds to show a new image lands.
Point every case that should be refused before connecting at an unroutable address (192.0.2.1), never at a board: a
refusal prints no `Connecting to` line, and an image that slips through cannot reach a device. An ESP32-S3
`firmware.factory.bin` slipped through on 2026-10-27 (its bootloader header at offset 0 looks like an app), the device
reported `OTA successful` and its bootloader rejected the slot at boot; send the board its proper app image afterwards.
A factory image has the partition table magic `AA 50` at 0x8000 and the app at 0x10000.
## OTA automations on two OTA platforms (#21903)
`scripts/ota_automations/ota-automations-common.yaml` (a package) puts `esphome` and `web_server` OTA in one config,
each with all six automations (`on_begin`, `on_progress`, `on_end`, `on_abort`, `on_error`, `on_state_change`)
logging a platform tag and the argument. `run.sh esp32|esp8266 LABEL` builds, flashes by serial, and runs four cases
under one capture: a native upload, a native upload whose uploader is SIGKILLed after the first progress fire, a
web_server upload of a random 64 KB file and a web_server upload of the real image. `summ.py OUT_DIR` prints each
case's fires in order with progress collapsed to a count; diff dev against the PR.
- `web_server: ota: false` removes `/update` (404), so leave it out when the web platform is under test.
- A killed native uploader ends in `state 4` and `error 255`, not `abort`. `on_abort` only comes from a web upload
cut on the network. On ESP8266 those happen by themselves (1 in 4 uploads on dev), so count them on both builds with
`webrep.sh IP IMAGE COUNT` before blaming the PR.
- A random file through web_server gives `error 128` on the ESP32 and `error 132` on the ESP8266.
- A SIGTERM to the uploader lets it finish the transfer; use SIGKILL for the native error path.
- Results: https://github.com/esphome/esphome/pull/21903#issuecomment-6516529409
@@ -1,10 +0,0 @@
# prometheus
## Date, time and datetime metrics (#9941)
`scripts/prometheus/prom-datetime.yaml` is a D1 mini with `web_server`, `prometheus` and three optimistic template
datetime entities; `prom-datetime-tz.yaml` adds an sntp time component with `HST10` (the zone applies without a sync).
`datetime_metrics.py HOST` prints the rows of `/metrics`, sets the entities over the API to 2040 values (past 2038 on
purpose) and prints them again with their UTC reading. Dates are local midnight in ms, times ms since midnight,
datetimes ms since the epoch, so a datetime set as an instant reads the same in both zones. It exits 1 when a time
or datetime row does not hold the value it set (dates depend on the zone and are only printed).
@@ -1,30 +0,0 @@
# remote_base / remote_receiver
## Loopback on one pin, no IR hardware (#21560 to #21563)
`templates/remote_base/`: `rb-common.yaml` (API action `tx_all`, a dumper registered from `on_boot` through
`register_dumper()` standing in for an external component), `rb-txscript.yaml` (14 `remote_transmitter.transmit_*`
frames with `tx <proto>` log markers), `rb32.tmpl` and `rb66.tmpl` (`@V@` is the build label).
`scripts/remote_base/run.sh LABEL B N` (`TREE`, `P32` or `P66`, `SC` in the environment) writes the run file from the
template, builds and flashes it from `TREE` (`esphome run --no-logs`), finds the IP and runs `rbdrive.py IP N`, which segments the API log by the
markers and prints what each sent frame decoded as.
- ESP32: `remote_transmitter` and two `remote_receiver`s on the same GPIO (`allow_other_uses`), carrier 100%, no
inversion. RMT reads its own output, every frame decodes. Second receiver `dump: [nec, raw]` shows raw runs only
where no primary matched.
- ESP8266: the transmitter sends with interrupts off, so a receiver on the same pin sees nothing. `softtx.hpp`
bit-bangs frames built with each protocol's `encode()` with interrupts on, and an `interval` sets the pin back to
OUTPUT after the receiver's setup. Bit-banged NEC decodes as NEC about 2 of 3 times (timing), on every build.
- Dish never decodes in loopback on dev either. Pronto matches every frame, so raw on `dump: all` never prints.
- API log lines can drop under bursts; rerun with more rounds before calling a shortfall.
- `register_dumper()` returns nothing, so the config logs `ext dumper register called`, not whether it stuck. On dev
the list has a fixed size and a dumper beyond it is dropped with `No dumper slot` (ERROR); judge by that line and by
`ext dumper saw` counts in the `rbdrive.py` table, which only appear when the dumper is really registered.
`rbdrive.py` exits 1 when the external dumper saw no frame, so a build that rejects the registration fails the
drive; set `EXPECT_REGISTER=1` on `run.sh` to also fail the run on any `No dumper slot` line.
Results: https://github.com/esphome/esphome/pull/21563#issuecomment-6381304387
Retest of the moved stack with 5 rounds per build: https://github.com/esphome/esphome/pull/21563#issuecomment-6382075726.
A D1 mini CH340 sometimes fails the 460800 baud upload; esphome retries at 115200 by itself, but a runner that reads
only the last line sees "Failed to upload" first, so rerun before blaming the build.
@@ -1,22 +0,0 @@
# rs485_frame DLE/STX/ETX bus without a bus (#16758)
`scripts/rs485_frame`: an ESP32-C3 (logger on USB_SERIAL_JTAG) with the hub's UART on tx GPIO3 rx GPIO4 and a
simulated bus device on tx GPIO4 rx GPIO3, cross-wired through the GPIO matrix (`allow_other_uses: true` on all
four pins), 9600 baud, no wires. The simulator is polled from an `interval` and logs with the `sim` tag; the test
script logs with `hwt`.
- `simrs.hpp` builds sum16 big endian, header-inclusive frames (`10 00` stuffing, `10 10` with
`simrs::double_mode`) and logs each frame the hub sends (`RXF`) with start and end poll times, the silence since
the previous frame and the time since the simulator's own last byte. `rs-base.yaml` (board, cross-wired UARTs,
poll interval, the hub's CRC and a log line per received frame) and `buttons.yaml` are shared packages.
- Case files: `common.yaml` runs good / bad CRC / garbage / split at 25 and 150 ms / overlong / max length / two
frames in one write / a cut frame followed by a new DLE STX / a bad escape, then button presses that the hub must
hold until a `01 01` gate frame arrives (`replace_latest` keeps only the last press). `rsi2.yaml` (idle_gap gate),
`rsx.yaml` (fixed_delay gate), `rsd.yaml` (sniffer_only, double escape).
- `run.sh CASES LABEL [SECONDS]` (`CASES` is `common`, `rsi2`, `rsx` or `rsd`; `TREE`, `PORT`, `SC` in the
environment) writes a run file with its own name, builds and flashes with `esphome run --no-logs`, captures and
writes `out-rs<CASES><LABEL>.log` with time stamps and line numbers stripped. It fails without a `DONE` line or on
a crash. Diff the logs of dev and the PR; for the gate timing cases read the `RXF` lines (silence before each frame
the hub sent).
- On ESP32 at 9600 baud the default `rx_full_threshold` is 8, so a component does not see bus bytes for about 9 ms.
Queue a test frame at least 20 ms after the bus starts talking, or it collides on any idle-gap design.
@@ -1,18 +0,0 @@
# rtsp_server
## RTSP camera streaming on the Waveshare ESP32-S3-ETH (#17990)
`templates/rtsp-waveshare-s3-eth-dut.yaml` (OV5640 pins, GPIO8 camera power switch with the 1.2 s boot delay,
`rtsp_server`, logger VERBOSE for `rtsp_server`); swap the `wifi:` block for the W5500 `ethernet:` block when
the cable is in, and add `network: enable_high_performance: true` for the big TCP buffers. Flash over native
USB, capture the log with `serial_capture.py`, take the IP from the last `IP Address` line that is not
0.0.0.0 (or `dns-sd -Q <name>.local`). `scripts/rtsp_server/rtsp_client.py IP [cases]` covers play, udp,
pipeline, badreq, replay, getparam, teardown, third, stall, timeout; `rawrate2.py IP quiet|chatty|rtcp|bigbuf`
measures the raw receive rate; `measure.sh IP LABEL LOGFILE [OUTDIR]` runs the fixed set (ping, ffmpeg 12 s
twice, two ffmpeg clients, raw reader) and appends to `results.txt`; `cam_api.py IP` pulls one image over the
API; `ffmpeg -rtsp_transport tcp -i rtsp://IP:554/ -frames:v 30 -f image2 f%03d.jpg` plus Pillow checks
decoding. The 5760 byte default TCP send buffer makes the stream stall over Wi-Fi (about 1 fps, dozens of
stall drops per run); `enable_high_performance` gives several fps on Wi-Fi and removes the stalls on Ethernet.
Frame counts vary a lot between runs (auto exposure changes the JPEG size), so always measure twice. Home
Assistant's stream pipeline does not accept MJPEG over RTSP; its ffmpeg camera opens one RTSP session per
snapshot, so `max_connections` needs headroom, and go2rtc transcoding to H.264 is the way to a live view.
@@ -1,23 +0,0 @@
# scheduler
## Scheduler and loop wake A/B (#20356)
An external component whose template button arms an SDK `os_timer` (50 ms, one shot) on the ESP8266;
the callback runs in SYS context, records `millis()` and calls `set_timeout` or `defer`; the timeout
logs the elapsed time and loop count delta. Drive with an aioesphomeapi script (`subscribe_logs` plus
`button_command`). Build the same yaml twice with different names, one per tree.
## interval startup_delay and control from lambdas (#21799)
`scripts/interval/interval-common.yaml` has plain 1 s and 2.5 s intervals, a 1 s interval with `startup_delay: 5s`, a
3 s one with `startup_delay: 500ms`, and a 15 s control interval (`startup_delay: 0s`). The control interval
changes the plain 1 s interval to 500 ms with `id(i_plain).set_update_interval(500)` and `start_poller()`, then
stops it and the 5 s delayed one with `stop_poller()`. Every run logs `millis()`. `run.sh LABEL` flashes a D1 mini,
captures 40 s from boot and runs `ivsum.py`, which prints the run count, first run after boot and gaps per interval.
- Check that each delayed interval's first run comes at or after its delay, that the gaps equal the interval, that
the 500 ms change takes effect, and that nothing runs after a stop.
- The first run of an interval without a delay falls at a different time on every boot (the scheduler spreads first
runs), so do not compare those offsets between builds.
- `main.cpp` shows the class each interval gets (`IntervalTrigger` or `DelayedIntervalTrigger` on #21799).
- Results: https://github.com/esphome/esphome/pull/21799#issuecomment-6516798398
@@ -1,20 +0,0 @@
# Sensor, binary_sensor and text_sensor filter A/B (#21831, #21813, #21830)
`scripts/sensor_filters` needs only a board. `sensors.yaml` holds 29 template sensors with every offset, multiply,
clamp, NTC, calibrate_linear, delta and lambda filter shape, plus a sensor with id `x` (a value lambda that reads an
id named `x`). `sf.yaml` adds binary and text sensor lambda filters and an interval that feeds the 50 inputs of
`sf.hpp` (NaN, ±inf, -0.0, values on and across the bounds) 200 ms apart, logging one line per input with every
filter output as float bits (`-` means dropped), then `DONE`.
- `run.sh esp8266|bk72xx LABEL` writes the per-run file, builds, flashes, captures and writes `out-sf<kind><LABEL>.out`
(100 lines). Compare a PR build with dev using `cmp`: identical files mean identical filter output, including
which values were dropped. ESP8266 matters because its float math is in software.
- The template sensor ids are `flt1`, `flt2` ...: `s8` and `s16` clash with ESP8266 SDK typedefs.
- Grep the build's `main.cpp` for the new code (for example `clamp_filter_value`, `delta_passes`, generated filter
functions) to prove the build came from the PR tree.
- `BASELINE` sets the delta baseline: a lambda works everywhere, a plain number crashed before #21811.
- Between boards the only difference is the sign of the NaN that `to_ntc_temperature` returns for 0 Ω (libm), on dev
too, so compare builds on the same board.
- The ESP8266 esptool stub can hit serial noise on a CH340 and the BK7238 link sometimes times out once; the runner
retries.
- Results: https://github.com/esphome/esphome/pull/21831#issuecomment-6490999545
@@ -1,36 +0,0 @@
# sigma_delta speaker and rtttl sample taps
## Pulse density of the speaker pin, measured inside the chip (ESP32-P4, #18702)
Nothing is wired to the pin. `scripts/sigma_delta/sdprobe.hpp` calls `gpio_input_enable()` on the SDM output pin,
and a core 0 task reads it with `gpio_get_level()` 2000 times per window, with pseudo-random gaps of 0 to 255 cycles
(xorshift) that decorrelate the samples from the 1 MHz modulator. One window is about 2 ms and the density noise is
about 0.5% sd. The expected duty for an int8 density d is (d + 128) / 256, so 16-bit PCM at +50% reads 75%, 0 reads
50%, and full scale reads 99.6% or 0%.
The same header has:
- a core 0 feeder task (DC, sine, stereo L/R, a burst of N frames, any 8/16/24/32 bit mono or stereo format) that
calls `speaker->play(..., 20 ms)`;
- a played-frame counter on `add_audio_output_callback()`: frames per second over 4 s matches the rounding of the
10 MHz timer exactly;
- a CPU probe: a priority 10 task pinned to the loop core spins for 200 ms, and the iterations while playing against
idle give the interrupt share (44.1 kHz cost 20% of core 1).
`gen.py tpl.yaml > NAME.yaml` builds the whole sequence (the device name comes from `esphome -s name NAME`): DC levels,
volume, mute, formats, 2 Hz and 50 Hz sines with DUMP lines, sample rates, underrun, stop mid-play, finish, rtttl and a
play/stop hammer. `gen2.py` (both share the step helpers in `steps.py`) builds the inverted pin run (set the speaker
pin to `{number: GPIO21, inverted: true}` in a copy of `tpl.yaml`) plus the CPU cost per rate. `run.sh LABEL
[gen|gen2]` generates, builds, flashes, captures with `--ms` until `ALLDONE` and prints the `RES` lines; for `gen` it
then runs `an.py`, which fits the sines and prints the level edges of the underrun, stop and finish dumps. GPIO21 is
free on the Waveshare ESP32-P4 Module DEV KIT. Long operation warnings in the log come from the probe's own lambdas.
## rtttl click measurement (#22075)
`scripts/rtttl_edge_tap/edgetap.hpp`: `EdgeTap`, a pass-through `speaker::Speaker` that forwards to the real speaker
and reads the int16 samples rtttl writes. `rtttl.set_speaker()` it from a lambda, and call `sync()` from an
`interval: 10ms`, because `is_running()` and `is_stopped()` read the base class state. Use tunes with a rest between
every note, so a note is a run of samples between runs of 8 or more zeros. It logs each note's onset and release
level (the largest of the first or last 4 samples as a share of the note's peak): dev about 95%, the 32-sample fade
at most 12.5%. `rtttl-edge-tap-dut.yaml` is the AUX-2 config with a USB speaker as the mixer output (it needs the
AUX-2 package next to it, see [usb_audio.md](usb_audio.md)); `run.sh LABEL` plays eight tunes and prints the note
reports. Any speaker works the same way: wrap the one rtttl writes to.
@@ -1,75 +0,0 @@
# Simulated chips on the bus, fake Home Assistant, static buffers
`scripts/sim_buses/` and `scripts/static_buffer/`. Each header is added with `esphome: includes:` and wired in an
`on_boot` lambda, so no chip is needed. The wrappers take the real bus or SPI delegate and pass through every
address they do not simulate.
- `lcdsim.hpp`: `LcdSimBus`, an `i2c::I2CBus` wrapper that plays PCF8574 backpacks (P0 RS, P2 EN, P4 to P7 D4 to D7)
driving HD44780 controllers. It decodes the EN-strobed nibbles and rebuilds the rows written after each 0x80 and
0xC0 address command. `add()` up to three addresses, hand it to each `lcd_pcf8574` with `set_i2c_bus()` at
`on_boot` priority 800, and call `report()` from an interval. `lcd_an.py SERIAL` checks the rows against the
patterns its docstring lists (0x27 20x4, 0x26 16x2, 0x25 64x4), and fails on a display whose frame count does not
grow between its first and last report, on a segment longer than its rows or on
data past the 128 byte segment buffer, which the report counts as `overflow`, on a report from an address it does
not expect, or on any read (`reads`; the driver never reads, so the simulator NACKs them). A 64x4 display hangs the
code before #22331 in
setup.
- HT16K33: use `templates/i2c_fake/htfake.hpp` with its test runner (see [fake_i2c.md](fake_i2c.md)).
- `mxsim.hpp`: `MxSniff`, an `spi::SPIDelegate` in front of the real one that keeps what each MAX7219 in a chain
latches. Swap it in at `on_boot` priority 399 through `DelegatePeek::of(id(x))`:
`auto &d = DelegatePeek::of(id(x)); d = new MxSniff(d, chips);`. `mx_an.py SERIAL CHAIN...`
summarizes the reports, CHAIN being each name passed to `report()`, and fails on a named chain with no report line,
on malformed transactions (byte count not two per chip, or a partial byte from `write()` or
`write_cmd_addr_data()`, which are recorded and forwarded too), on a named chain with fewer than two report lines,
or a chain whose transaction count stays 0 or stops growing, and prints PASS otherwise.
- `spysim.hpp`: the same delegate trick (both headers carry `DelegatePeek`, so one config can include both), logging length and FNV-1a hash of every transaction over 1000 bytes (whole
frames, used for `pixoo`). `write()` and `write_cmd_addr_data()` are hashed and forwarded like `write_array()`;
a partial byte or a bus width other than 1 is hashed as whole bytes, so the SPY line carries `odd`, the count of
such transactions, and a dev vs PR hash match only holds when `odd` is 0 on both.
- `pisim.hpp`: drives two `packet_interface` stacks on cross-wired UARTs through `top()`, with payloads whose bytes
are a function of their index so every byte is checked, and the index and length each side sent (last 256) so a
truncated or split packet, or one whose slot a later index reused, counts as `rx_bad`. Send indices 0, 1, 2 and so
on from each side: a skipped index counts as `rx_lost` and an old or repeated one as `rx_bad`. `report(tag, true)` once both links are idle ends each
line in PASS only when that side received every packet the peer sent intact and the peer had no send refused;
`missing` is the rest, packets still in flight or lost at the tail. `packet_transport` drops packets carrying its own device
name, so a one-board test must use the layers directly.
- `hcsim.hpp`: a software SN74HC165 chain on ESP-IDF (data GPIO6, clock GPIO7, load GPIO10). A falling load edge
latches the pattern and each rising clock edge drives the next bit onto the data pad, which is in
`GPIO_MODE_INPUT_OUTPUT` so the component reads it from the same pin. Call `hcsim::report(label, loads0, clocks0)`
after each window with `hcsim::loads` and `hcsim::clocks` from its start; it logs `INVALID` instead of a result
when the GPIO or interrupt setup failed or the window saw no load or no clock edge, so a dead simulator cannot pass
as a wrong but steady pattern.
- `ledprobe.hpp`: samples a status LED (or any output) pin every 2 ms from an `esp_timer`, off the loop task, and
prints blink count, on time, period and the longest gap per window. It measures `status_led` while the loop is
blocked (#16407). Pass `ledprobe::n` from the window start as the mark; a window with more than 400 edges logs
`INVALID` and no result line instead of passing for a stalled LED. So does every window when the pin or timer
setup failed or the timer has not sampled for 100 ms, so a dead probe cannot read as a quiet LED.
- `fakeha.py HOST OUT_PREFIX`: an aioesphomeapi client standing in for Home Assistant for `homeassistant` select
mirrors. It answers state subscriptions from a table, pushes option and state changes (full, overfull, quoted,
unparseable), checks each step through a "Dump" button, captures the action calls, churns 600 option updates
while watching the heap, and checks the entity options after a reconnect. A step fails when either DUMP line is
missing, and a rejected update also needs the device's rejection log line, so an update that never arrived cannot
pass. Exits 1 on a failed step and prints PASS otherwise.
- `wizard_drive.py HOST`: finishes the device wizard through aioesphomeapi 46.28 (`device_wizard`,
`wizard_input_set`), answers the subscriptions the inputs create and checks the values reached the entities.
## Static buffers (`cg.static_buffer`)
`scripts/static_buffer/` holds two local external components and `static-buffer-dut.yaml`, which loads them:
- `sbtest` allocates three buffers with `cg.static_buffer` (37 `uint8_t`, 5 `uint32_t`, and `const char *` sized by
a C++ expression) and checks at setup that they are zeroed and aligned, then fills each completely; at
`dump_config()` it checks no other instance's fill reached them. Two instances prove the buffers are distinct.
Both `SBTEST` lines end in `PASS` or `FAIL`; the `dump_config()` one passes only when the setup checks passed too.
- `sbpsram` asks for `psram=True` and uses the pointer codegen hands it: the static buffer without PSRAM, or a
zeroed `psram_buffer<T>()` with a `psram:` block, which logs `external=1` (nullptr if the allocation failed).
It also gets a plain 8 word `uint32_t` static buffer, which must be non-null, aligned and zeroed at start and read
back its fill (`plain=1`). Its `dump_config()` line ends in `PASS` only when both readbacks match, both buffers
were zeroed at start and, with PSRAM, the PSRAM one is in external RAM.
## Gotchas
- Some "classic" devkits with a CP2102N carry an ESP32-PICO-D4. GPIO16 and GPIO17 are its flash pins, and an
ESP-IDF build that drives them hits the interrupt watchdog. Use `board: pico32` and GPIO9/GPIO10.
- An `addressable_light` display writes nothing until its light is turned on.
- Before calling a rebased PR's change trivial, diff the PR's own diff (head against its parent) old against new: a
move that looks like a merge only can carry real commits from below.
@@ -1,62 +0,0 @@
# speaker
## Media player pipeline tasks (#21203)
The `speaker` media player platform (`speaker/media_player/`) creates `ann_read`, `ann_decode`, `med_read` and
`med_decode` FreeRTOS tasks on every play and deletes them after a stop. The AUX-2 and sendspin packages use the
`speaker_source` player instead, which has its own pipeline and never runs that code, so a test of
`speaker/media_player/*` has to swap the player: `templates/aux-2-preprod-speaker-player-dut.yaml` removes the
package's `external_media_player` with `id: !remove` and adds a `speaker` platform player on the package's resampler
speakers, plus a "Task names" button whose lambda calls `xTaskGetHandle()` on the four names and logs internal free
heap. Serve FLAC files from the host (`ffmpeg -f lavfi -i "sine=frequency=440:duration=90" -ac 2 -ar 48000 -c:a flac
media90.flac`, a 3 s mono one for announcements, `python3 -m http.server 8765 --directory www`); `audio_http`
refuses WAV and MP3. `scripts/speaker_media_player/cases.sh HOST KEY MEDIA_URL ANN_URL CAPTURE LABEL` runs three
cycles through `drive.py` (`play:URL | ann:URL | stop | press:NAME | wait:S` on one API connection) with the probe
pressed idle, during media, during the announcement, after it and after the stop. Pass: idle none, media `med_*`
only, announcement all four, after stop none, every cycle. API commands land a second or two late over Wi-Fi, so read
the probe against the `State changed to` lines rather than the driver's timestamps. Internal free heap after a stop
moved by 2.5 kB between two boots of the same build, so a heap delta needs more than one boot per build. Results:
https://github.com/esphome/esphome/pull/21203#issuecomment-6299581943
## Streamed TTS with a pause (#21090, #21091)
Home Assistant streams TTS while it is still being generated, so the player must start on the first part instead of
waiting for the whole file. `scripts/speaker_tts_pause/`:
- `mk.py INTRO_AUDIO CONT_AUDIO www/s1.flac` builds one 48 kHz mono FLAC stream (the intro padded to a 4096-sample
frame boundary, then the continuation) and writes the byte offset of the split to `s1.flac.cut`. Make the intro
with any TTS, e.g. `say -o intro.aiff "..."` on macOS; build a short one (`s1`) and a long one (`s2`).
- `srv.py PORT www` serves the file chunked: the header and intro at once, then it holds the connection open for
`PAUSE` seconds (8), then the rest.
- `tts-common.yaml` swaps the AUX-2's `speaker_source` player for the `speaker` platform player and counts played
frames with `add_audio_output_callback` on both resampler speakers into the counters in `frames.h` (one `[frames`
log line per change, read over serial with `--ms`). Its `on_boot` must be a list entry: a dict `on_boot` replaces
the package's list and the AUX-2 never starts Wi-Fi or Ethernet.
- `run.sh LABEL=TREE...` flashes each build and drives two short-intro and two long-intro announcements and one
media play through `scripts/speaker_media_player/drive.py` (set `DRIVE` when running from a scratch copy); `an.py`
prints first audio, the server's send times and how much of the intro played before the rest arrived, per request.
A request fails when it never played the whole intro or stopped short of 95 % of the stream.
Before the two PRs the player waited for the rest of the stream (8.2 s) for both a 22.5 KB and a 50 KB intro; with
both it starts at about 1 s.
## Voice PE wake word audio (#21885)
`scripts/voice_wake_word_audio/va.py HOST KEY OUTDIR accept,reject,timeout` stands in for Home Assistant
(aioesphomeapi 46.27.0 or later, `wake_word_audio=True`), answers each run by mode and saves both channels as WAV;
`locate.py OUTDIR` finds the played phrase in them by envelope cross-correlation against `wake16k.wav`.
`run.sh HOST TAG [ROUNDS]` runs reject and timeout rounds while playing the phrase every 9 s, starting once `va.py`
is READY. `va.py` fails a START before READY or during a run, and `run.sh`, `stale.py` and `mww.py` fail at once
when the player fails or hangs for 30 s.
- Give the test build its own API key so Home Assistant cannot hold the voice assistant subscription.
- Make the phrase with any TTS (`say -o wake.aiff "Okay Nabu"`, then `ffmpeg -i wake.aiff -ac 1 -ar 16000
wake16k.wav`) and play it from the host near the device (`PLAY`, default `afplay`). On macOS the terminal needs
audio permission first; before that `afplay` hangs.
- The PE mute switch only mutes samples. To stop microWakeWord, add `mww-buttons.yaml` (template buttons calling
`micro_wake_word.stop` and `start`); `mww.py` checks that no run starts while it is stopped.
- Stale clip check: `stale.py` says a spoken marker word (`marker.aiff`), stops microWakeWord for 3 s, starts it and
says the wake word; `stalean.py` measures loud audio before the phrase in each clip and exits 1 above
`MAX_LOUD_MS` (default 100). The loud level has no default: calibrate `LOUD_LEVEL` on each setup from the phrase
and pre-phrase peaks it prints (a run without it prints them and fails), somewhere between room noise and the
marker's peak. A trial whose phrase peak is under twice `LOUD_LEVEL` fails as too quiet to judge. A steady 1 kHz
beep never reaches the PE stream (the XMOS processing removes it), so the marker has to be speech.
@@ -1,19 +0,0 @@
# sun on_sunrise and on_sunset without waiting for the sun (#21803)
`scripts/sun/sun-common.yaml` (a package) puts the observer on the equator and, once SNTP has synced, searches the
longitudes for one where the sun stands near -5° and rises as the observer moves east. It holds that longitude for
one 60 s elevation poll first: the poll before saw the sun at longitude 0, and the jump could read as a crossing.
Crossings until then are logged as `crossing before the sweep ignored` and not counted. An interval then moves the
longitude by 0.125° every 5 s, which sweeps the elevation from -5° to +6° and back at about 1.5° a minute, reversing
above +6° and below -5°. Five triggers (sunrise at the default, 2° and 4°, sunset at the default and -2°) fire from
the real 60 s elevation poll; each logs the elevation it fired at and counts in `fires`, logged every 30 s.
- The sweep stops after one full pass and logs `sweep done`. `run.sh esp32|esp8266 LABEL` builds, flashes and
captures up to 20 minutes (a pass takes about 17), then `check.py` wants `sweep done` and the counter at 5 with no
trigger firing twice. Run dev and the PR on the same board; two boards can run in parallel.
- At 1.5° a minute a crossing fires up to about 1.7° past its elevation, one poll late at most.
- The seconds past the minute of the fire lines show the poll phase: separate pollers fire at different seconds,
one shared poll fires every trigger in the same second.
- `id(sun_id)` is a pointer in lambdas (`->`). Judge by the `fires` counter: the classic devkit's serial log can
garble single lines.
- Results: https://github.com/esphome/esphome/pull/21803#issuecomment-6490173072
@@ -1,24 +0,0 @@
# CAN (esp32_can) without a transceiver (#21792, #21795)
`scripts/twai_self`: `esp32_can` with `mode: NO_ACK`, TX on GPIO25 and RX on GPIO26. After boot `canself.hpp` enables
the input on GPIO25 and routes it to the controller's receive signal (`esp_rom_gpio_connect_in_signal(25,
TWAI_RX_IDX, false)`), so the controller hears its own transmit pad. Frames go out with `twai_transmit_v2` and
`TWAI_MSG_FLAG_SELF` (the driver handle is reached through explicit template instantiation), so every frame runs
through the normal receive path and `on_frame`.
- `can.yaml` has nine `on_frame` entries (standard and extended ids, masks, remote-only and data-only filters, two
entries with the same filter, an id with bits outside its mask that must never fire) and sends 20 rounds of 7
frames. `run.sh LABEL` writes `out-can<LABEL>.counts` (fires per trigger, id, flags and data); diff dev and the PR.
- Set `rx_queue_len: 32`: with the default 5, a burst of 7 frames loses the last two most of the time, on every
build.
- ESPHome's own `canbus.send` does not set the self flag, so its frames are not received in this setup.
- Results: https://github.com/esphome/esphome/pull/21795#issuecomment-6491777381
## ESP32-C3 and extra frame shapes (#21950)
The C3 works the same way: `variant: esp32c3`, TX GPIO5 and RX GPIO6, `TX_PAD = GPIO_NUM_5` in `canself.hpp`, and
the logger on USB_SERIAL_JTAG. `canx.hpp` adds a classic frame with DLC 12 (`TWAI_MSG_FLAG_DLC_NON_COMP`), the
`send_data` overloads (standard, extended, remote, empty, a 10-byte payload that must be clamped, a vector) and a
`twai_get_status_info_v2` dump before and after (`canx::tx_status(id(can0), "before")`). The `get()` that reads the
private handle is a friend found only by argument-dependent lookup: call `get(canself::Handle{})`, never
`canself::get(...)`.
@@ -1,74 +0,0 @@
# Simulated UART sensors on a classic ESP32 (#21449, #21446)
No sensor needed. `templates/uart_sim/sim.hpp` (included with `esphome: includes:`) has frame builders for wts01,
ds1603l, sm300d2 and ufm01, `sim::loopback(id(uart))` (ESP-IDF `uart_set_loop_back` on the component's own UART),
`sim::send()` that logs the bytes, and `sim::probe`, a SEN0465 responder polled from a 10 ms interval. The run files
`fra.yaml`, `frb.yaml` and `frc.yaml` set the name and include `fr-common.yaml` plus their `fr*-parts.yaml` as
packages; copy them with a different name (`frabase` and so on) for the base build.
- Push-style sensors: put the component on a UART, switch it to loopback in `on_boot`, and inject frames from
template buttons (`fra-parts.yaml`, `frb-parts.yaml`). ufm01 enters its active stream by itself when a 1 s
interval streams frames during its 2 s startup wait.
- Request/response sensors: cross-wire two UARTs through the GPIO matrix (UART1 tx 25 rx 26, UART2 tx 26 rx 25,
`allow_other_uses: true` on all four pins); the second UART runs the simulated device (`frc-parts.yaml`).
A select sets the probe's misbehaviour (tail, FF tail, silent, split, wrong gas).
- Polled readers: run `update()` from a script and log `micros()` around it to measure blocking
(sm300d2 on dev blocks about 104 ms on an empty buffer).
- The logger stays on UART0, so three builds cover five components. Drive with `scripts/uart_sim/frdrive.py HOST
LOG press:NAME sel:NAME=OPT num:NAME=V wait:S @FILE` (`@FILE` reads one op per line), or
`scripts/uart_sim/run.sh NAME TREE SECONDS OPS...` (build and flash from TREE with `esphome run --no-logs`,
capture, drive; `PORT` and `SC` in the environment, `ERASE=1` erases the flash first, `DRIVER` picks another
API driver).
## Particulate sensors: gcja5, pm1006, sds011 (#21939)
`templates/uart_sim/pmsim.hpp` adds frame builders `sim::gcja5`, `sim::pm1006` and `sim::sds011`, each with a bad
checksum or bad end flag. `fp-parts.yaml` puts gcja5 on UART1 (9600 8E1, with a template binary sensor that shows
`status_has_warning()`) and pm1006 on UART2; `fs-parts.yaml` has sds011 alone. Write the run files the same way as
`fra.yaml` (`fp.yaml`, `fs.yaml` with their own name).
- Every button injects one case with its own PM value, so each published state names its case: good, two frames,
bad checksum, end or tail, garbage before a frame, a cut-off frame followed by a good one, a repeated first header
byte, and splits of 300 and 700 ms (either side of the 500 ms gap reset).
- `run.sh NAME TREE 150 @fp-ops.txt` runs the op list from the file; `fs-ops.txt`, `fd-ops.txt` and `fj-ops.txt`
are the other lists.
- The loopback also feeds each component its own writes (pm1006 request, sds011 setup and sleep/work commands),
which must not disturb the next frame.
- The API coalesces two states published in the same loop pass, so "two frames" shows only the second value.
- Found with it: on dev gcja5 published bad and misaligned frames as readings (268435456).
## 4-byte distance sensors: a01nyub, a02yyuw, jsn_sr04t (#21948)
`sim::ds1603l(mm, bad_ck)` builds the same `FF H L sum` frame; `fd-parts.yaml` (a01nyub on UART1, a02yyuw on UART2) and `fj-parts.yaml`
(jsn_sr04t, polled every 15 s), with the same case list as above plus a reading below the minimum distance. With
loopback the jsn_sr04t request byte `55` can land inside a split frame and break it; check the `Request read out`
timestamps before calling a difference between builds.
## LD6002B TinyFrame parser (#19836, #19837)
`templates/uart_sim/ld6002b/`: `ld6sim.hpp` is a fake module on a UART cross-wired to the component's UART (UART1
tx 25 rx 26, UART2 tx 26 rx 25), set up by the shared `ld6-base.yaml` package. It ACKs every command with an empty
frame of the same type and logs the bytes it received (`MOD RX`), so the bytes on the wire can be diffed between
builds. Target reports carry a distinct X per
case, and `T1 X` shows which frames got through.
- `ld6-parser-cases.yaml` (used by `ld6-common.yaml`, which turns the query answers off): good, byte by byte (one
byte per poll, so the component's loop runs in between), garbage before a frame, bad and lost header or data
checksums, an oversized payload, a burst of 20. The script runs by itself from `on_boot`; `ld6p.yaml` is the run file
(copy it with another name for the base build). `run.sh NAME TREE` builds, flashes over serial and captures,
and fails without a `CASE done` line.
- Garbage that contains `01` right before a frame loses that frame on dev and on the PR (a false header, no rescan).
- Full component A/B: `ld6sim.hpp` also answers every query with a fixed report (delay, sensitivity, trigger,
installation, Z range, low power, low power sleep, both area reports, version); `ld6x-common.yaml` has the full
entity set and runs the parser cases too (`ld6x.yaml` is its run file); `ld6drive.py HOST` presses every button,
sets every number and select and toggles every switch over the API. Run it with `ERASE=1 DRIVER=ld6drive.py
scripts/uart_sim/run.sh NAME TREE 100`: switch and select states and the area id are stored preferences, and a previous build's run changes the
next build's boot state.
- `m4-common.yaml` and `m5-common.yaml` run the same fake against `ld6004`: target and area reports, work status and
radar on/off reports, malformed reports, and the point cloud and target display switches across a reboot
(`m5` reboots once and checks the restored switch states). `m4.yaml` and `m5.yaml` are the run files, and like
`ld6p.yaml` each can be copied with another name for the base build. Run m5 with `ERASE=1 run.sh NAME TREE 75`: the boot
counter and switch states are stored preferences that survive a flash, and the second boot's `CASE done` comes
about 49 s after reset. Only the second boot logs `CASE done`, so a skipped reboot fails the run.
See `rs485_frame.md` for the same cross-wired UART trick on an ESP32-C3.
@@ -1,26 +0,0 @@
# udp
## UDP, multicast, sockets (#20556 to #20559, #20585, #20590)
A listener on port 9 needs no privileges on macOS. Test multicast device to device; host to device
multicast never arrives on some access points, and `239.255.x.x` groups are unreliable even to a stock
listener. Broadcast to boards is lossy under power save, unicast is not. Flood at a fixed packet rate
(the host hits `ENOBUFS` unthrottled); 1500 packets per second survived on every board. A forced Wi-Fi
reconnect on the Pico keeps IGMP membership. Set a long loop interval from `wifi.on_connect`
(`App.set_loop_interval(20000)`), not `on_boot`, or Wi-Fi connect crawls. The `logger.log` action
rejects `%.*s`; use a lambda.
## udp.write payload shapes (#21263, ApplyBytes)
`templates/udp-write-dut.yaml` puts every payload shape behind a template button (constant 4 bytes twice, 3, 9, 0,
128, 129 and 600 bytes, a 300 byte lambda, a simple value, a string) plus an `on_receive` that replies with the
received bytes reversed. `scripts/udp/write_cases.sh DEVICE_IP KEY` records the device's datagrams with
`udp_listen.py` on the test host, presses the buttons two seconds apart, sends one unicast datagram for the echo and
compares everything byte for byte with `check.py`; a missing datagram shifts every later one, so read the first
mismatch. Before flashing, check the generated `main.cpp`: on ESP8266 a constant payload is copied out of its
`PROGMEM` table into `uint8_t esphome__bytes[N]` up to the stack limit and into `std::make_unique<uint8_t[]>` beyond,
identical payloads share one table, a lambda is sent from its vector. An empty constant payload is generated as
`send_packet(nullptr, 0)`, and the raw lwIP UDP `sendto` used on ESP8266 rejected a null buffer with `EINVAL`
(`sendto() error 22` in the log, no datagram) until it accepted `nullptr` with a zero length; the old
`TemplatableBytes::visit` had passed a zero length stack buffer instead. Compare sizes against the stacked PR's own
head when the PR is stacked, not against dev. Results: https://github.com/esphome/esphome/pull/21263
@@ -1,52 +0,0 @@
# usb_audio
## USB audio on the Waveshare ESP32-S3-ETH (#21081, #11728)
`templates/usb-audio-waveshare-s3-eth-dut.yaml`: ESP-IDF, PoE, logs over the API on Ethernet (the native USB port is
the host port, so there is no console), `usb_audio` hub, a 48 kHz 16-bit microphone, a 48 kHz 24-bit speaker (many
speakerphones offer only 24-bit; a wrong format logs the device's alternates), buttons for capture start and stop,
a non-blocking beep fed from an 8 ms interval with `play(buf, len, 0)` (a blocking `play()` on the main loop trips
the task watchdog), and a status interval with mic bytes, speaker state and heap. Drive the buttons with
`scripts/api_button.py IP KEY press "Beep"`. Expect 192 kB/s microphone bytes and the FIFO split
RX 80 / NPTX 16 / PTX 104 in the build's `defines.h`; with the balanced split the 294-byte OUT pipe is refused
(`EP MPS (294) exceeds supported limit (128)`). A device whose OUT endpoint is larger than 600 bytes cannot be
claimed on an S3 at all.
## Music Assistant through sendspin
`templates/usb-audio-sendspin-s3-eth-dut.yaml` overlays the template above with the Apollo CAST-1 wiring: resampler
speakers into a mixer whose inputs never time out, `speaker_source` media player with the sendspin source, and
`sendspin: unpaired_access: true`. A `media_player: platform: speaker` is not a media-source consumer and gives
"Player over its buffer", no audio. Music Assistant finds the device by mDNS within 15 s of boot.
Read the result from the device, not only by ear: sendspin's sync task logs `Lost sync (N us off)` (DEBUG, tag
`SsSync`) for every correction over 5 ms, a positive N inserts that much silence, and each one is an audible click.
For the chunk lead histogram, edit the managed library in the build directory,
`.esphome/build/<name>/managed_components/sendspin__sendspin-cpp/src/sync_task.cpp` (an `SS_LOGD` before the
late-chunk skip in `decode_chunk()` printing the chunk's lead against `new_audio_client_playtime`), update only that
file's `size` and sha256 `hash` in the component's `CHECKSUMS.json`, and compile again; deleting `.component_hash` or
recomputing it makes the manager refuse or refetch. Check the edit is in the image with `strings firmware.ota.bin`.
A lead below -5 ms makes the library skip the whole chunk; an I2S speaker keeps it within 0.1 ms of zero, which is
the figure a USB speaker has to match.
Gotchas: wait two minutes after a reboot before the measured stream (safe mode's boot counter write stalls the cores
for about 140 ms once, 60 to 75 s after boot); with the serial console on, a DEBUG log line per report saturates
UART0 and trips the task watchdog, so use `logger: baud_rate: 0` and the API log; two overlapping `esphome logs`
captures double every count; and every audio test must include a pause and resume cycle, since a stream end that
leaves less than one transfer parked in the driver was what hung the speaker after #11728 merged.
## Controls
Apollo CAST PRO-1 (`templates/cast-pro-1-sendspin-control.yaml`): `Core.yaml` next to it is
`Integrations/ESPHome/Core.yaml` from ApolloAutomation/CAST_PRO-1, which also defines the `version` substitution.
Keep its framework log level at ERROR (DEBUG floods driver output and trips the watchdog); the package reboots itself
into Ethernet-only once it sees a link and appends the MAC suffix to the name. `wifi: enable_on_boot: false` keeps
sendspin off Wi-Fi.
Nabu Casa Connect AUX-2 (a pre-production unit): build from `packages/aux-2-pre-prod.yaml` at NabuCasa/aux-2 commit
283f2e64 (dropped in bde41e01), not the public `aux-2.yaml`, whose W5500, codec and I2S pins differ (the public one
reads W5500 chip ID 0x00 and fails the codec's I2C writes). Point its local `external_components` source at
`github://NabuCasa/aux-2@main`. Templates `aux-2-preprod-stock-dut.yaml` (I2S codec) and
`aux-2-preprod-usb-audio-dut.yaml` (the USB speaker as the mixer output via `!extend mixing_speaker`). The console is
the CH343 (1a86:55d3); first flash over it, OTA after. Forcing the Wi-Fi flag off with `restore_value: no` boot-loops
when Ethernet fails, so let the package switch itself.
@@ -1,20 +0,0 @@
# usb_cdc_acm
## usb_cdc_acm on the nRF52840 dongle (#11634)
`templates/usb-cdc-acm-dongle-dut.yaml`: logger on USB_CDC port 0, `usb_cdc_acm` interface 0 disabled,
interface 1 echoing from a 20 ms lambda, 0x01-prefixed commands for burst, stream, flush, peek, line coding
and counters. Build and flash with `dfu: true` through the app port; the dongle then shows three CDC ports:
the board DTS node `cdc_acm_uart` (unused), `cdc_acm_uart0` (log) and `cdc_acm_uart1` (data), in that order.
Capture the log port with `serial_capture.py`, drive the data port with `scripts/usb_cdc_acm/drive.py
DATA_PORT` (11 cases, prints PASS/FAIL and a summary) and grep the log for `Opened`, `Closed`, `buffer full`,
`flush result`, `line coding`. Boot lines are held in the driver's 1 KB ring until the host opens the port, so
a capture started later still shows the first 1 KB of the boot and then a truncated line.
With three CDC instances the Zephyr legacy stack's `CONFIG_USB_MAX_NUM_TRANSFERS=4` starves the last instance
(each instance holds an OUT read and an IN ZLP), so interface 1 shows DTR changes but moves no data.
`scripts/usb_cdc_acm/cdc_probe.hpp` (include it, call `cdc_probe()` from an interval) reads the Zephyr driver
directly to tell a dead endpoint from a component bug. Raising the transfer pool to 8, or disabling the
board's `cdc_acm_uart` node, fixes it. Design limit of the driver: `fifo_fill` holds `tx_ready` until the
write callback and `write_array` drops when the ring is full, so a 64 KiB full-duplex echo loses one 64 byte
tail on every buffer size.
@@ -1,15 +0,0 @@
# sensor and number on_value_range A/B (#21814, #21817, #21818)
`scripts/value_range/value-range-common.yaml` (a package) has one template sensor with seven triggers (constant
bounds, one-sided, a lambda bound on `x`, a bound from another sensor's state that stays NaN until step 22, both
bounds NaN, and an identical twin) and one template number with three triggers. It feeds them 30 values 200 ms apart
(on the bounds, just past them, repeats, NaN, ±inf, a moving bound), then syncs preferences and restarts, then
sends 50, 95 and 50 to check the restored in-range flag. Every fire logs its step and `x`.
- `run.sh bk72xx|esp8266|esp32 LABEL` writes the per-build file with a fresh `tag`, builds, flashes, captures 90 s
(through the restart) and writes `out-rng<kind><LABEL>.seq`; diff dev against each PR head.
- The sequence primes every trigger first (bound 0, value -500, bound NaN): the in-range flags are stored in flash,
so without priming a fresh build starts from whatever the previous run left, and the first fires differ.
- The phase global resets when `tag` changes, so each reflash runs the whole sequence.
- Log line numbers differ between trees; the runner strips them before writing the `.seq` file.
- Results: https://github.com/esphome/esphome/pull/21818#issuecomment-6491846503
@@ -1,15 +0,0 @@
# voice_assistant
## Voice PE late TTS start (#19408)
Only the subscribed client (Home Assistant) may send `VoiceAssistantAnnounceRequest`, so both builds carry
`scripts/voice_announce/slow-announce.yaml`, a template button that calls `id(va).on_announce(msg)` with
`start_conversation = true` and a URL on the host. `drive.py HOST KEY DELAY OUT` serves a FLAC (WAV as `audio/wav`
is rejected by micro_decoder) only after DELAY seconds, presses the button and timestamps the device logs. It
holds the voice assistant subscription throughout, since without a subscriber the device drops the announcement
state at once. It imports `va_common.py` from `scripts/voice_wake_word_audio/`, so keep the two directories side
by side.
`run.sh DEVICE LABEL=CONFIG=TREE...` builds, OTAs and runs both builds twice. The player gives up after 6 header
attempts of 5 s.
The wake word audio flow is in [`speaker.md`](speaker.md).
@@ -1,61 +0,0 @@
# web_server
## State SSE on web_server_idf (#21109) and the HTTPD task stack (#21038)
`templates/web-server-sse-dut.yaml` (ESP32 classic, ESP-IDF, `web_server` version 3) publishes eight sensors every
200 ms (small state events on the 1024 byte stack buffer) and three text sensors whose JSON carries the string twice
(`value` and `state`): 600 chars for about 1.3 kB (the 2048 byte tail), 1800 chars for about 3.7 kB (tail grown to
4096) and 2800 chars for about 5.7 kB, over the 5120 byte JSON room, which must log `State event does not fit`
and never put a truncated frame on the wire. An `interval` lambda logs `uxTaskGetStackHighWaterMark()` of the loop
task (`nullptr`) and of the HTTPD task (`xTaskGetHandle("httpd")`) every 5 s; the ESPHome loop task is `loopTask`
on ESP32. Keep `power_save_mode: none`, and drive the device by address: mDNS names dropped out mid-run once.
`scripts/web_server/sse.py HOST SECONDS [--clients N] [--stall] [--churn CYCLES]` opens raw TCP connections to
`/events`, decodes the chunked body and parses every `state` event as JSON; `--stall` adds a connection with a 2 kB
receive buffer that never reads (the device must log `Closing stuck stream` on ESP-IDF and `Closing stuck
EventSource connection` elsewhere, after the 20 s stall timeout), `--churn` opens and resets a connection CYCLES
times. `sse_cases.sh HOST KEY LABEL CAPTURE` runs the set: 1 client 60 s, 4 clients 60 s, a reader plus a stalled
client 45 s, a reader plus 10 abrupt closes, then two clients with the "Burst" button (50 sensor and 50 text
publishes in one loop pass). Pass: 0 invalid JSON on every client, the oversize sensor dropped with its warning on
every publish, the stalled session closed after about 20 s, no reset. The loop task watermark is a minimum since
boot, so compare the value after the first client, after four clients and after the burst between the builds (the
SSE split in #21109 left 1280 B more on the ESP32 classic).
For the HTTPD task, `web_server: auth: type: digest` (the default `auth` is Basic) and
`scripts/web_server/httpd_cases.sh HOST USER PASS LABEL CAPTURE`: `httpmix.sh` sends a `curl --anyauth` request
mix per round (page, `/sensor/<Name>`, `/text_sensor/<Name>`, `/switch/<Name>`, a `POST` with `Content-Length: 0`
to `/switch/<Name>/toggle`, an unknown path, which the IDF server closes without a response, an unauthenticated
request answered 401, and a 2 s `/events` stream), 20 rounds alone and 12 rounds next to two `ssecurl.sh` SSE
clients. Entity URLs use the entity name, URL encoded, not an object id. The HTTPD watermark follows the deepest
request of the boot; compare after the mix and after the SSE round, with two boots when the numbers are close.
Results: https://github.com/esphome/esphome/pull/21109#issuecomment-6300582328 and
https://github.com/esphome/esphome/pull/21038#issuecomment-6301310182
## Entity icon and device class readback (#21111)
Covers every path that reads an entity's icon or device class: the native API listing, MQTT discovery, the web_server
`/events` listing and REST endpoints, and `dump_config`. `templates/entity-icon-dut.yaml` (ESP8266 board with wifi,
encrypted api, ota, web_server and mqtt against a local broker) carries a sensor with a 63 character icon, one with no
icon or device class, an internal sensor, and binary_sensor, switch, button, text_sensor, number, select, event, cover
and text entities with icons and device classes. The per-run file supplies `name` and `broker_ip` through
substitutions. Flash the PR and base builds under different names, then from the scratch directory:
```sh
scripts/entity_icons/readback.sh DEVICE_IP KEY NAME BROKER_HOST cap-NAME.txt # once per build
scripts/entity_icons/compare.sh ic-base ic-pr # api, web, mqttn, dump all identical
```
`readback.sh` takes about 25 s once the device is on the network: 12 s of `mosquitto_sub` on `homeassistant/#` and
`NAME/#`, the API listing (`api_list.py`, one JSON line per entity with icon, device class, unit, options and event
types) and `web_dump.py` (6 s on `/events`, keeping only the listing rows that carry `domain`, then a GET of each
listed id with `?detail=all` and one retry, because the ESP8266 async server occasionally leaves a request unanswered
within 5 s). The web ids are `domain/Name`, not object ids, and `/event/<name>` has no GET handler (404 on every
build). `mqtt_norm.py` drops the `dev` block (compile time) and the `NAME/debug` topic (heap probe), so a clean run
differs in nothing. `dump_config` prints icons and device classes only for some domains; button, event and
binary_sensor icons never appear there.
Found with this flow on 2026-10-21: since #19577 every ESP8266 `web_server` build that lists a cover, climate, lock,
valve, update, alarm_control_panel or water_heater entity reboots with `Exception (3) LoadStoreErr` in
`json::linked(const char *)` (the ROM `strlen` on a flash address), fixed by #21316. The cover in the template keeps
that path covered; before #21316 is merged, apply its one line change to both trees or the web_server comparison
cannot run.
@@ -1,95 +0,0 @@
# wifi
## `reboot_timeout` with a fallback AP (#9958)
Only one board runs the AP at a time, or stations join the wrong one. AP probe: bogus network,
`ap: {ssid, password, ap_timeout: 5s}`, `reboot_timeout: 60s`, `logger: level: VERBOSE` (ESP32 and
ESP8266 log `AP client connected MAC=` at verbose), `on_boot: logger.log: BOOT`. Station probes:
`reboot_timeout: 0s`, `interval: 150s` running `wifi.disable`, `delay: 20s`, `wifi.enable` (60 s cycles
are too short to cover the due moment). Pass pattern: "Can't connect; rebooting" exactly 60 s after boot
with no station joined, no reboot for minutes while one is, then a reboot within a second of the last
disconnect. The BW15 restarts its AP on every STA retry and cannot be tested this way.
## DHCP lease cache (#17767)
Cycle 20 s awake, `deep_sleep.enter 30s`, wake, 20 s, `App.safe_reboot()`, logging `millis()` at the
`wifi.connected` condition. An RTS hardware reset must fall back to DHCP; a software reboot and a deep
sleep wake must apply the cache. `esphome run --no-logs` silently skipped the upload on one board; use
`upload` then `logs`.
## Wi-Fi Easy Connect (DPP) enrollee (#17934)
Only an Android phone configures: Settings, network details, Add device, Share Wi-Fi (the Wi-Fi list's
QR scanner makes the phone an enrollee). Render the URI with the `qrcode` module and open the PNG.
`esp32: framework: log_level: DEBUG` plus `sdkconfig_options: CONFIG_WPA_DEBUG_PRINT: y` prints the
supplicant state machine.
## WISP bridge with an ethernet upstream (#7147)
`templates/wisp-ethernet-dut.yaml` (ESP32-S3 with a W5500, `wifi: ap:` with `ap_mode: WISP` and no station) and
`templates/wisp-client.yaml` (D1 mini joining the bridge, `http_request` to a web server on the wired LAN and to
`http://example.com/` every 15 s, `time: sntp`). Secrets: `wisp_psk` and `lan_http` (serve a text file on the host
with `python -m http.server 8787 --bind 0.0.0.0` from a scratch directory). Judge by the client's `GET status 200`
lines, the web server log showing the device's ethernet address as the source (NAT), the client's DHCP info (DNS1 is
the LAN resolver) and `SNTP synced`. The device logs `Starting AP` and `DHCP DNS server: <resolver>` on its USB
console; flashing over the air reboots it for repeat boots. Before the AP address and DHCP server moved ahead of the
AP start (the fix in that PR), the first AP start failed on every boot with `esp_netif_dhcps_option failed: 20484`
(DHCP server already started by the AP start event) and `Failed to enable NAT`, and the bridge only came up on the
retry 90 s later; a client that cannot find the SSID for the first minute on an older build is that. With the fix
the bridge is up 6 s after boot. Any change to the ESP-IDF AP start should also rerun the captive portal hammer.
## Reboot against a WPA2/WPA3 PMF access point (#21590)
- AP: `scripts/wpa3_ap/wdap.yaml` on a classic devkit. ESPHome `wifi: ap:` only, then an on_boot lambda switches it
to `WIFI_AUTH_WPA2_WPA3_PSK`, PMF capable, and registers a WIFI_EVENT handler that logs `STA JOIN` and
`STA LEAVE ... reason N` (the include header pulls in `esp_wifi.h`). It disables Wi-Fi after 420 s.
- AP leave reason 3 means the station sent a deauth; reason 2 means the AP found a stale association when the
station came back; several reason 2 before one join are rejected reconnects.
- DUTs (`wdbk.yaml` BK7238, `wdcb.yaml` CB3S, both on `wd-common.yaml`) list the AP at priority 10 and the main
network at 0, and reboot 30 s after joining the AP and 150 s after joining the main network. So they move to the
AP while it is up and fall back to the main network (where they can be flashed over the air) once its window
ends. `run.sh BUILD` flashes the AP, does OTA for the CB3S and a serial flash for the BK7238, then captures one AP
window; `an.py` prints each station's join and leave sequence and fails on a rejected reconnect, an AP that did
not switch to WPA2/WPA3 (`AP CONFIG FAILED`), a DUT missing from the AP events (`DUT_MACS`) or a BK7238 that never
joined the AP.
- Gotchas: the LibreTiny build directory has two `.uf2` files, so flash `firmware.uf2`. The BK7238 serial flash
sometimes fails to link while the chip is in a reboot loop; retry it 3 or 4 times. OTA fails if the DUT reboots
mid-upload; retry.
- Finding: BDK 3.0.78 rejects about 2 reconnects per reboot when the station left without a deauth; BDK 3.0.33
does not.
## WISP and ALWAYS_ON AP rig (#20077)
`scripts/wisp_ap`: the upstream is an ESP32-S3 with Ethernet running a WISP bridge (`wisp-upstream.yaml`) with its AP on
10.40.0.1/24 and web buttons `ap off` and `ap on`. Its subnet must not be 192.168.4.0/24, the DUT AP's default, or
nothing routes. The DUT is a classic devkit (`dut-wisp.yaml` or `dut-always-on.yaml`); the client is a D1 mini on
the DUT's AP (`wispcli.yaml`) logging its DNS server, a name lookup, a GET by name, the AP itself and a LAN file by
IP. `run.sh DUT START SECONDS T:STATE...` switches the upstream AP at the given times and `sum.py` groups the client
results by phase. Capture the D1 mini without `--reset` (a CH340 floods garbage after an RTS reset). Clients keep
the captive portal's DNS answer for its TTL.
## AP restart with WPA3 mixed and LIGHT power save (#21531)
`scripts/wifi_restart`: the WISP upstream (`wisp-upstream-wpa3.yaml`) switched to WPA2/WPA3 mixed by a script that
runs again after every `ap on`, DUTs (`rebk.yaml`, `recb.yaml` on `re-common.yaml`) with a static IP on the bridge's
subnet, a fallback AP, `power_save_mode: LIGHT` and syslog to the host through the NAT. `run.sh BUILD` switches the
AP off and on three times; `an.py` fails on a reboot, a crash or a DUT that is not back on syslog within 120 s of an
AP start. After 19 minutes the DUTs run `wifi.configure` to the main network; that keeps the static IP, so the next
OTA needs the host to have an address on the bridge's subnet as well (an interface alias). The CB3S log is on UART1
(TX1), so its configs need `logger: hardware_uart: UART1`. Runners must `pkill` only their own ports; a global pkill
stops the other runs' captures.
## Channel and network switching with a bench AP (#22142)
`scripts/wifi_channel/`: `bench-ap.yaml` makes an ESP32-S3 an AP-only bench network on a fixed channel (11 by
default; pick one your normal network does not use), and `dut-common.yaml` gives the device under test that AP at
priority 10 plus your normal network, logs every `wifi_info` ssid and channel update (with the channel the component
reports and whether it is connected), and cycles `wifi.disable` and `wifi.enable` every 90 s. `run.sh LABEL` (env
`TREE`, `PORT`, `AP_PORT`, `SC`, `PY`, and `BENCH_CHANNEL` when the AP is not on 11) flashes the device and captures 420 s while it takes the AP down with
`esptool --port <AP port> --after no-reset chip_id` (the board stays in the bootloader) and back with
`--after hard-reset`. The channel sensor must follow every move between the two networks: the run passes when the
connected channel starts and ends on the bench channel with another channel in between.
- Dropping the AP under a connected device gives a `Beacon Timeout` link loss.
- The 90 s interval fires once right after boot.
- A `Disconnected ... Association Leave` right after `Connecting to` is the aborted attempt, not a missed connect.
@@ -1,38 +0,0 @@
# Xiaomi BLE
## Xiaomi, ATC and PVVX parsing without the sensors (#21483)
No Xiaomi devices are needed, and one ESP32 cannot hear its own advertising, so the frames are injected into the
tracker as scan results: everything after the radio runs as normal.
- `scripts/xiaomi_ble/gen.py` writes `frames.hpp` and `expected.json`: plain MiBeacon (0x50), encrypted v5 23 byte
with MAC and 19 byte without, real AES-CCM (`cryptography` AESCCM, tag 4, AAD 0x11, nonce = MAC reversed + pid +
cnt + 3 byte ext counter), ATC (13 byte, big endian) and PVVX (15 byte, little endian) 0x181A frames, a corrupted
tag every other cycle and a counter 0 boot frame. It models the old shared duplicate counter (`base`) and the per
device one (`pr`) and prints the expected publish counts.
- `xbinject.yaml` (ESP-IDF devkit) includes `frames.hpp`; an `interval:` builds a `BLEScanResult` and calls
`global_esp32_ble_tracker->gap_scan_event_handler(r)`. `count.py` counts the `Got Xiaomi` blocks (sensor
publishes are not logged at DEBUG) and compares them with `expected.json`.
- On ESP32 the tracker's own triggers sit in `listeners_`, which runs before `neutral_listeners_` (the
ble_device_base devices), so an `on_ble_service_data_advertise` never sees what a Xiaomi listener did to the
shared data. Check that directly: `xb_rewrite_probe(id(xa))` builds an `ESPBTDevice` with `parse_scan_rst()`,
calls the component's `parse_device()` and logs the service data before and after (`[probe` lines).
- `run.sh inject pr|base` builds, flashes, captures and prints PASS when the counts match that build's model and,
on the PR build, the probe's service data is the same before and after `parse_device()`.
Base dropped a second device's frame with the same counter, the frame after a bad tag and a first frame with
counter 0, and rewrote the shared data in place; the PR fixed all of them.
## Over the air with a second ESP32
- Advertiser: an ESP32-S3 with Ethernet (OTA over Ethernet), `xbadv.yaml` with `adv_frames.hpp` from
`gen.py --ota`. Per frame: stop advertising, `esp_ble_gap_set_rand_addr`, `config_adv_data_raw`, start
non-connectable advertising with `own_addr_type` random, 20 ms interval, 250 ms dwell. Random static addresses
need the top two bits set, so these use C4:C1:38 instead of A4. `esp32_ble:` alone brings up the stack. The
web_server button starts a run: `curl -X POST -d "" http://<advertiser>/button/start/press` (no body gives 411).
Its `failed sends` sensor counts only GAP calls rejected when queued; a frame the controller drops later shows
up as missing publishes in `count.py`, the same as radio loss.
- Receiver: `xbrecv.yaml` on the devkit, passive scan with window = interval = 100 ms. `run.sh ota pr|base` (with
`ADV` set) flashes it, resets it with a capture and presses start 18 s later, so the counter 0 boot frame is the
first one it hears.
- Each frame arrives about 4 times; the counts matched the injection run exactly apart from one radio loss, so the
over the air comparison allows two missing publishes per tag.
@@ -1,10 +0,0 @@
# zephyr
## Zephyr preferences on the nRF52840 dongle (#20810)
`templates/nrf52-preferences-dut.yaml`: logger on USB_CDC at VERBOSE, `flash_write_interval: 5s`, one
`globals` with `restore_value` incremented every 15 s. Give each build its own global id so each first boot
sees a never-saved key (settings survive a reflash). Flow per build: flash through the log port, 40 s capture
of the first boot, then start a second capture and reboot with a 2001 baud touch on the dongle's other CDC
port (the dfu rate callback is global, and the capture holds the log port); `serial_capture.py` reopens the
log port after the USB re-enumeration and gets the buffered boot lines (`load setting`, `Loaded N settings`).
@@ -1,17 +0,0 @@
# zephyr_coredump
## zephyr_coredump crash and decode on nRF52 boards (#13122)
`templates/nrf52-coredump-crash-dut.yaml` (dongle; change `board:` to `adafruit_itsybitsy_nrf52840` for the
ItsyBitsy): `zephyr_coredump:`, `dfu: true`, a `restore_value` global keyed by the `run` substitution
(settings survive a reflash, so bump `run` for every new crash), and an `on_boot` sequence that sets the
flag, yields 3 s so the globals loop saves it (a blocking lambda never gets there and the board crash-loops),
syncs preferences, then writes to 0xDEADBEEF. Flash, keep every port closed for 80 s, then
`scripts/zephyr_coredump/flow.sh LABEL PORT ELF [OUTDIR]` does the three opens (3 s cut, 100 s full print,
20 s reopen) and calls `decode.sh`, which keeps the first complete `#CD:BEGIN#..END#` block, runs Zephyr's
`coredump_serial_log_parser.py` and `coredump_gdbserver.py` from the sdk-nrf ESPHome installed and prints
`bt` and `info threads` with its `arm-zephyr-eabi-gdb`. The ELF is `.pioenvs/<name>/zephyr/zephyr/zephyr.elf`.
Expect about 9 KB dumps, 140 to 150 `#CD:` lines, a 673 ms loop warning for the erase, and `Has stored
dump: 0` on the reopen. Check `compiled on` in every capture: another session flashing the same board
mid-test looks exactly like a failed test otherwise. Boards without a network cannot exercise the API and
MQTT delivery paths of the component; say so in the results.
@@ -1,23 +0,0 @@
# zwave_proxy
## zwave_proxy frame checksums on a live proxy (#20922)
A Z-Wave proxy (ESP32-S3 with the module on `usb_uart`) is a live device: flash it over OTA and read logs
over the API, there is no serial console because the native USB is the host port for the module, so the
boot-time `GET_NETWORK_IDS` exchange is never captured. Home Assistant's Z-Wave JS holds the single API
subscription (it polls `0x3B` every 30 s and resubscribes after every reboot), so do not subscribe from a
script, and keep the device `name:` so it stays on Z-Wave JS; give each build its own scratch directory
instead.
1. `logger: level: VERY_VERBOSE` with `logs: zwave_proxy: VERY_VERBOSE` and DEBUG for the rest: the
`Sending:`, `Received frame:` and `CHECKSUM Received ... Calculated` lines are `ESP_LOGVV`, which needs
the compile-time level.
2. `md5` the two `firmware.ota.bin` files to be sure the A and B builds differ, then `esphome upload
--device <name>.local` and a `timeout 330 esphome logs --device <name>.local` capture started right after
the OTA to catch the resubscribe traffic.
3. `scripts/zwave_proxy/verify_checksums.py <captures>` recomputes the XOR (init 0xFF, LENGTH..payload) over
every logged frame and counts bad checksums and ACK/NAK sent and received; `scripts/zwave_proxy/devinfo.py
<name>.local` prints the firmware version, compile time and the Home ID from the API (a Home ID proves the
device-built `GET_NETWORK_IDS` frame was accepted at boot).
4. Pre-existing: one `zwave_proxy took a long time for an operation` warning on the first frame after the
log client connects, caused by VERY_VERBOSE logging; it appears on dev too.
@@ -1,23 +0,0 @@
# Run artifacts the runners write next to their scripts; none of these patterns match a tracked file.
out-*/
out-*.*
.esphome/
secrets.yaml
__pycache__/
ble_adv_triggers/rxbk72xx*.yaml
ble_adv_triggers/rxesp32*.yaml
ezo_fake_bus/ezo?*.yaml
homeassistant_button/habtn*.yaml
hub_input_sim/hubsim*.yaml
interval/ivl*.yaml
led_driver_sniff/led*.yaml
ota_automations/otaauto*.yaml
output_button/outbtn*.yaml
remote_base/rb*.yaml
sensor_filters/sf?*.yaml
sun/sunesp*.yaml
twai_self/can?*.yaml
uart_button/uartbtn*.yaml
value_range/rng*.yaml
ld2420_sim/ldenergy*.yaml
ld2420_sim/ldsimple*.yaml
@@ -1,211 +0,0 @@
"""Plaintext native API client that lists user actions with their raw argument fields and executes one.
usage: rawactions.py HOST list
rawactions.py HOST exec ACTION key=value [...] (every argument must be given; types from the listing)
Fields of ListEntitiesServicesArgument: 1 name, 2 type, 3 description, 4 example, 5 optional, 6 default_value.
"""
import os
import socket
import struct
import sys
HOST = sys.argv[1]
CMD = sys.argv[2]
ARGS = sys.argv[3:]
TYPE_NAMES = {
0: "bool",
1: "int",
2: "float",
3: "string",
4: "bool[]",
5: "int[]",
6: "float[]",
7: "string[]",
}
def varint(v: int) -> bytes:
out = bytearray()
while True:
b = v & 0x7F
v >>= 7
if v:
out.append(b | 0x80)
else:
out.append(b)
return bytes(out)
def svarint(v: int) -> bytes:
return varint((v << 1) ^ (v >> 63))
def decode_varint(buf: bytes, pos: int) -> tuple[int, int]:
value = shift = 0
while True:
b = buf[pos]
pos += 1
value |= (b & 0x7F) << shift
if not b & 0x80:
return value, pos
shift += 7
def fields(payload: bytes) -> list[tuple[int, int, object]]:
out = []
pos = 0
while pos < len(payload):
key, pos = decode_varint(payload, pos)
fnum, wt = key >> 3, key & 7
if wt == 0:
v, pos = decode_varint(payload, pos)
elif wt == 2:
ln, pos = decode_varint(payload, pos)
v = payload[pos : pos + ln]
pos += ln
elif wt == 5:
v = payload[pos : pos + 4] # fixed32, caller unpacks
pos += 4
else:
raise ValueError(f"wire type {wt}")
out.append((fnum, wt, v))
return out
def frame(mtype: int, payload: bytes) -> bytes:
return b"\x00" + varint(len(payload)) + varint(mtype) + payload
def field_bytes(fnum: int, data: bytes) -> bytes:
return varint((fnum << 3) | 2) + varint(len(data)) + data
def field_varint(fnum: int, v: int) -> bytes:
return varint((fnum << 3) | 0) + varint(v)
def recv_frames(
sock: socket.socket, until: set[int], limit: float = 5.0
) -> list[tuple[int, bytes]]:
sock.settimeout(limit)
buf = bytearray()
frames = []
while True:
try:
chunk = sock.recv(4096)
except TimeoutError:
return frames
if not chunk:
return frames
buf += chunk
while True:
if len(buf) < 3:
break
if buf[0] != 0:
raise ValueError("not plaintext")
try:
ln, p = decode_varint(buf, 1)
mt, p = decode_varint(buf, p)
except IndexError:
break
if len(buf) < p + ln:
break
frames.append((mt, bytes(buf[p : p + ln])))
del buf[: p + ln]
if mt in until:
return frames
def main() -> None:
sock = socket.create_connection((HOST, 6053), timeout=5)
sock.sendall(
frame(
1, field_bytes(1, b"rawactions") + field_varint(2, 1) + field_varint(3, 10)
)
) # HelloRequest
recv_frames(sock, {2})
sock.sendall(frame(3, b"")) # ConnectRequest, no password
recv_frames(sock, {4})
sock.sendall(frame(11, b"")) # ListEntitiesRequest
frames = recv_frames(sock, {19})
services = {}
for mt, payload in frames:
if mt != 41:
continue
name = key = None
args = []
for fnum, _, v in fields(payload):
if fnum == 1:
name = v.decode()
elif fnum == 2:
key = struct.unpack("<I", v)[0]
elif fnum == 3:
a = {
"name": None,
"type": 0,
"description": None,
"example": None,
"optional": None,
"default": None,
}
for af, _, av in fields(v):
if af == 1:
a["name"] = av.decode()
elif af == 2:
a["type"] = av
elif af == 3:
a["description"] = av.decode()
elif af == 4:
a["example"] = av.decode()
elif af == 5:
a["optional"] = bool(av)
elif af == 6:
a["default"] = av.decode()
else:
a[f"field{af}"] = av
args.append(a)
services[name] = (key, args)
if CMD == "list":
for name, (key, args) in services.items():
print(f"action {name} key={key}")
for a in args:
extra = {
k: v
for k, v in a.items()
if k not in ("name", "type") and v is not None
}
print(f" {a['name']}: {TYPE_NAMES.get(a['type'], a['type'])} {extra}")
return
name = ARGS[0]
key, args = services[name]
payload = varint((1 << 3) | 5) + struct.pack("<I", key)
given = dict(kv.split("=", 1) for kv in ARGS[1:])
for a in args:
if a["name"] not in given:
if os.environ.get("ALLOW_MISSING"):
continue # leave the argument out of the request on purpose
sys.exit(f"missing {a['name']}")
raw = given[a["name"]]
t = a["type"]
if t == 0:
av = field_varint(1, 1 if raw.lower() == "true" else 0)
elif t == 1:
av = varint((5 << 3) | 0) + svarint(int(raw)) # sint32 int_
elif t == 2:
av = varint((3 << 3) | 5) + struct.pack("<f", float(raw))
elif t == 3:
av = field_bytes(4, raw.encode())
elif t == 5:
av = b"".join(
varint((7 << 3) | 0) + svarint(int(x)) for x in raw.split(",") if x
) # repeated sint32 int_array
else:
sys.exit(f"type {t} not supported here")
payload += field_bytes(2, av)
sock.sendall(frame(42, payload)) # ExecuteServiceRequest
print(f"sent {name} with {given}")
sock.close()
main()
@@ -1,221 +0,0 @@
"""Pull the store_yaml blob off a device over the native API (plaintext or Noise) and unpack it.
usage: syrecover.py HOST KEY OUTDIR [ORIGDIR] (KEY of - means a plaintext API)
Prints the capability flag, chunk count, advertised and received sizes, encoding, the envelope's files, and
when ORIGDIR is given compares each recovered file with ORIGDIR/<path>.
Run with PYTHONPATH pointing at the esphome tree that has store_yaml (for unpack_envelope). Needs Python 3.14
(compression.zstd) or the zstandard package.
"""
import asyncio
from pathlib import Path
import sys
import time
from aioesphomeapi import api_pb2
from aioesphomeapi._frame_helper.noise import APINoiseFrameHelper
from aioesphomeapi._frame_helper.plain_text import APIPlaintextFrameHelper
from aioesphomeapi.connection import APIConnection, ConnectionParams
from aioesphomeapi.zeroconf import ZeroconfManager
from esphome.components.store_yaml import unpack_envelope
try:
from compression import zstd
except ImportError:
try:
import zstandard as zstd
except ImportError:
sys.exit("needs Python 3.14 (compression.zstd) or the zstandard package")
HOST, KEY, OUT = sys.argv[1], sys.argv[2], Path(sys.argv[3])
ORIG = Path(sys.argv[4]) if len(sys.argv) > 4 else None
HELLO, HELLO_RESP, CONNECT, CONNECT_RESP = 1, 2, 3, 4
DISCONNECT_REQ = 5
FATAL = -1 # queued by report_fatal_error so a waiting expect() stops at once
CAPS_REQ, CAPS_RESP = 149, 150
GET_YAML_REQ, GET_YAML_RESP = 161, 162
def decode_varint(buf: bytes, pos: int) -> tuple[int, int]:
value = shift = 0
while True:
b = buf[pos]
pos += 1
value |= (b & 0x7F) << shift
if not b & 0x80:
return value, pos
shift += 7
def fields(payload: bytes) -> dict[int, list]:
out: dict[int, list] = {}
pos = 0
while pos < len(payload):
key, pos = decode_varint(payload, pos)
fnum, wt = key >> 3, key & 7
if wt == 0:
v, pos = decode_varint(payload, pos)
elif wt == 2:
ln, pos = decode_varint(payload, pos)
v = payload[pos : pos + ln]
pos += ln
elif wt == 5:
v = payload[pos : pos + 4]
pos += 4
else:
raise ValueError(f"wire type {wt}")
out.setdefault(fnum, []).append(v)
return out
async def main() -> None:
loop = asyncio.get_running_loop()
queue: asyncio.Queue[tuple[int, bytes]] = asyncio.Queue()
errors: list[Exception] = []
class Shim(APIConnection):
"""Only the two callbacks the frame helper uses; the real connection logic is bypassed."""
def process_packet(self, msg_type_proto, data) -> None:
queue.put_nowait((msg_type_proto, bytes(data)))
def report_fatal_error(self, err: Exception) -> None:
errors.append(err)
queue.put_nowait((FATAL, b""))
params = ConnectionParams(
addresses=[HOST],
port=6053,
password=None,
client_info="syrecover",
keepalive=15,
zeroconf_manager=ZeroconfManager(),
noise_psk=None if KEY == "-" else KEY,
expected_name=None,
expected_mac=None,
timezone=None,
provide_time=True,
)
shim = Shim(params, None, False, HOST)
if KEY == "-":
helper = APIPlaintextFrameHelper(
connection=shim, client_info="syrecover", log_name=HOST
)
else:
helper = APINoiseFrameHelper(
connection=shim,
noise_psk=KEY,
expected_name=None,
expected_mac=None,
client_info="syrecover",
log_name=HOST,
)
await loop.create_connection(lambda: helper, HOST, 6053)
try:
await asyncio.wait_for(helper.ready_future, 10)
except Exception as exc: # noqa: BLE001
sys.exit(f"handshake failed: {exc!r} {errors}")
print(f"transport {'plaintext' if KEY == '-' else 'noise'}: handshake done")
async def expect(msg_type: int, timeout: float = 10) -> bytes:
while True:
t, d = await asyncio.wait_for(queue.get(), timeout)
if t == msg_type:
return d
if t == FATAL:
sys.exit(
f"transport error while waiting for message {msg_type}: {errors}"
)
if t == DISCONNECT_REQ:
sys.exit(
f"device closed the connection while waiting for message {msg_type}"
)
print(f"skipping message type {t} while waiting for {msg_type}")
hello = api_pb2.HelloRequest(
client_info="syrecover", api_version_major=1, api_version_minor=10
)
helper.write_packets([(HELLO, hello.SerializeToString())], False)
await expect(HELLO_RESP)
# Sent like the client library does; a device without a password never answers it
helper.write_packets(
[(CONNECT, api_pb2.AuthenticationRequest().SerializeToString())], False
)
helper.write_packets(
[(CAPS_REQ, api_pb2.DeviceCapabilitiesRequest().SerializeToString())], False
)
caps = fields(await expect(CAPS_RESP))
supported = None
if 7 in caps:
supported = fields(caps[7][0]).get(1) == [1]
print(f"capabilities: store_yaml field present={7 in caps} supported={supported}")
t0 = time.monotonic()
helper.write_packets([(GET_YAML_REQ, b"")], False)
chunks: list[bytes] = []
total = encoding = None
done = False
while not done:
f = fields(await expect(GET_YAML_RESP, 20))
if 1 in f:
chunks.append(f[1][0])
done = f.get(2) == [1]
if 3 in f:
total = f[3][0]
if 4 in f:
encoding = f[4][0].decode()
dt = time.monotonic() - t0
compressed = b"".join(chunks)
print(f"chunks {len(chunks)} sizes {[len(c) for c in chunks]}")
print(
f"received {len(compressed)} bytes, advertised {total}, encoding {encoding!r}, {dt:.2f} s"
)
# a second request right after: the device must answer again, not stay stuck
helper.write_packets([(GET_YAML_REQ, b"")], False)
again = 0
done = False
while not done:
f = fields(await expect(GET_YAML_RESP, 20))
again += len(f.get(1, [b""])[0])
done = f.get(2) == [1]
print(f"second request: {again} bytes")
helper.close()
if errors:
sys.exit(f"transport errors: {errors}")
if encoding != "zstd":
sys.exit(f"unexpected encoding {encoding!r}, expected 'zstd'")
if total is not None and len(compressed) != total:
sys.exit(f"received {len(compressed)} bytes but the device advertised {total}")
if again != len(compressed):
sys.exit(
f"the second request returned {again} bytes, the first {len(compressed)}"
)
envelope = zstd.decompress(compressed)
files = unpack_envelope(envelope)
print(
f"envelope {len(envelope)} bytes, {len(files)} files, ratio {len(compressed) / len(envelope):.2f}"
)
OUT.mkdir(parents=True, exist_ok=True)
for rel, content in files.items():
target = OUT / rel.replace("../", "up_/")
if not target.resolve().is_relative_to(OUT.resolve()):
print("skipping a path outside the output directory:", rel)
continue
target.parent.mkdir(parents=True, exist_ok=True)
target.write_bytes(content)
note = ""
if ORIG is not None:
orig = (ORIG / rel).resolve()
if orig.exists():
note = (
"IDENTICAL to original"
if orig.read_bytes() == content
else f"DIFFERS from original ({orig.stat().st_size} bytes)"
)
else:
note = "no original"
print(f" {rel}: {len(content)} bytes {note}")
asyncio.run(main())
@@ -1,33 +0,0 @@
"""Press a template button on the test device over the API, or list its buttons.
usage: api_button.py HOST KEY list | press "Button name" [...] (KEY of - means a plaintext API)
"""
import asyncio
import sys
from aioesphomeapi import APIClient, ButtonInfo
HOST, KEY = sys.argv[1], sys.argv[2]
CMD, ARGS = sys.argv[3], sys.argv[4:]
async def main() -> None:
c = APIClient(HOST, 6053, None, noise_psk=None if KEY == "-" else KEY)
await c.connect(login=True)
ents, _ = await c.list_entities_services()
buttons = {e.name: e for e in ents if isinstance(e, ButtonInfo)}
if CMD == "list":
print("\n".join(sorted(buttons)))
else:
for name in ARGS:
if name not in buttons:
sys.exit(f"no button named {name!r}; have {sorted(buttons)}")
c.button_command(buttons[name].key)
print("pressed", name)
await asyncio.sleep(0.3)
await asyncio.sleep(0.5)
await c.disconnect()
asyncio.run(main())
@@ -1,73 +0,0 @@
"""Per wake cycle timings from a serial capture of the deep sleep timing config.
usage: cycles.py CAPTURE [LABEL]
A cycle starts at the boot banner; the numbers are the device's millis() from the timing lines, and the
capture clock gap from the first line of the cycle to the "Dialing" line.
"""
import pathlib
import re
import statistics
import sys
cap = (
pathlib.Path(sys.argv[1]).read_text(encoding="utf-8", errors="replace").splitlines()
)
label = sys.argv[2] if len(sys.argv) > 2 else ""
TS = re.compile(r"^\[(\d\d):(\d\d):(\d\d)(?:\.(\d+))?\]")
def secs(line):
m = TS.match(line)
if not m:
return None
h, mi, s, f = m.groups()
return int(h) * 3600 + int(mi) * 60 + int(s) + (int(f) / 10 ** len(f) if f else 0)
cycles = []
cur = None
for line in cap:
if "rst:0x" in line or (cur is None and "Log initialized" in line):
cur = {"start": secs(line), "reset": line.split("]", 1)[-1].strip()[:60]}
cycles.append(cur)
continue
if cur is None:
continue
if "Dialing" in line:
cur.setdefault("dial_cap", secs(line))
m = re.search(r"wifi connected at (\d+) ms", line)
if m:
cur.setdefault("wifi", int(m.group(1)))
m = re.search(r"client connected at (\d+) ms \((.*?)\)", line)
if m and "client" not in cur:
cur["client"] = int(m.group(1))
cur["who"] = m.group(2)
m = re.search(r"first HA state '(.*?)' at (\d+) ms", line)
if m and "state" not in cur:
cur["state"] = int(m.group(2))
print(f"{label} cycles: {len(cycles)}")
print(
f"{'n':>2} {'wifi ms':>8} {'dial +ms':>9} {'client ms':>10} {'state ms':>9} {'client-wifi':>11} {'state-client':>12} client reset"
)
for i, c in enumerate(cycles, 1):
dial = ""
if c.get("dial_cap") is not None and c.get("start") is not None:
dial = f"{(c['dial_cap'] - c['start']) * 1000:.0f}"
cw = c["client"] - c["wifi"] if "client" in c and "wifi" in c else ""
sc = c["state"] - c["client"] if "state" in c and "client" in c else ""
print(
f"{i:>2} {c.get('wifi', ''):>8} {dial:>9} {c.get('client', ''):>10} {c.get('state', ''):>9} {cw:>11} {sc:>12} {c.get('who', '')} {c.get('reset', '')[:28]}"
)
for c in cycles:
if "client" in c and "wifi" in c:
c["client-wifi"] = c["client"] - c["wifi"]
if "state" in c and "client" in c:
c["state-client"] = c["state"] - c["client"]
for key in ("wifi", "client", "state", "client-wifi", "state-client"):
vals = [c[key] for c in cycles if key in c]
if vals:
print(
f"{key}: n={len(vals)} min={min(vals)} median={statistics.median(vals):.0f} max={max(vals)}"
)
@@ -1,107 +0,0 @@
"""Per Wi-Fi drop timings from a serial capture of the outgoing connection timing config.
usage: wificycles.py CAPTURE [LABEL]
A cycle starts at "wifi cycle disabling"; the numbers are the device's millis() from the timing lines.
"dial" says whether the client that connected first came from the device's own dial ("Dialing" seen after the
reconnect and before the hello) or from an incoming connection ("Accept" seen first).
"""
import pathlib
import re
import statistics
import sys
cap = (
pathlib.Path(sys.argv[1]).read_text(encoding="utf-8", errors="replace").splitlines()
)
label = sys.argv[2] if len(sys.argv) > 2 else ""
TS = re.compile(r"^\[(\d\d):(\d\d):(\d\d)(?:\.(\d+))?\]")
def secs(line):
m = TS.match(line)
if not m:
return None
h, mi, se, f = m.groups()
return int(h) * 3600 + int(mi) * 60 + int(se) + (int(f) / 10 ** len(f) if f else 0)
cycles = []
cur = None
loop_max = []
for line in cap:
m = re.search(r"loop time max (\d+) ms", line)
if m:
loop_max.append(int(m.group(1)))
if "wifi cycle disabling" in line:
cur = {"order": []}
cycles.append(cur)
continue
if cur is None:
continue
m = re.search(r"wifi connected at (\d+) ms", line)
if m and "wifi" not in cur:
cur["wifi"] = int(m.group(1))
cur["wifi_cap"] = secs(line)
if "Dialing" in line:
cur["order"].append("dial")
cur["dials"] = cur.get("dials", 0) + 1
if "dial_cap" not in cur and "wifi_cap" in cur:
cur["dial_cap"] = secs(line)
elif "dial_cap" not in cur and "dial_before_wifi_cap" not in cur:
cur["dial_before_wifi_cap"] = secs(line)
if (
re.search(r"client disconnected at \d+ ms \(\d+\.\d+\.\d+\.\d+\)", line)
and "wifi" in cur
and "client" not in cur
):
cur["dial_closed"] = cur.get("dial_closed", 0) + 1
if "Accept " in line and "wifi" in cur:
cur["order"].append("accept")
m = re.search(r"client connected at (\d+) ms \((.*?)\)", line)
if m and "wifi" in cur and "client" not in cur:
cur["client"] = int(m.group(1))
cur["who"] = m.group(2).split(" ")[0]
cur["via"] = cur["order"][0] if cur["order"] else "?"
m = re.search(r"Connect (failed|timeout)", line)
if m:
cur.setdefault("connect_errors", 0)
cur["connect_errors"] += 1
print(f"{label} cycles: {len(cycles)}")
print(
f"{'n':>2} {'wifi ms':>8} {'dial-wifi cap ms':>16} {'client ms':>10} {'client-wifi':>11} via dials closed client"
)
for i, c in enumerate(cycles, 1):
cw = c["client"] - c["wifi"] if "client" in c and "wifi" in c else ""
c["client-wifi"] = cw
dw = ""
if "wifi_cap" in c and c.get("wifi_cap") is not None:
# the first dial of the cycle, whether it was logged before or after the Wi-Fi line
if c.get("dial_before_wifi_cap") is not None:
dw = f"{(c['dial_before_wifi_cap'] - c['wifi_cap']) * 1000:.0f}"
elif "dial_cap" in c and c["dial_cap"] is not None:
dw = f"{(c['dial_cap'] - c['wifi_cap']) * 1000:.0f}"
c["dial-wifi"] = dw
print(
f"{i:>2} {c.get('wifi', ''):>8} {dw:>16} {c.get('client', ''):>10} {cw:>11} {c.get('via', ''):7s} {c.get('dials', 0):>5} {c.get('dial_closed', 0):>6} {c.get('who', '')}"
)
for via in ("dial", "accept"):
vals = [
c["client-wifi"]
for c in cycles
if c.get("via") == via and c["client-wifi"] != ""
]
if vals:
print(
f"client-wifi via {via}: n={len(vals)} min={min(vals)} median={statistics.median(vals):.0f} max={max(vals)}"
)
dws = [float(c["dial-wifi"]) for c in cycles if c.get("dial-wifi") not in ("", None)]
if dws:
print(
f"dial after wifi connected (capture clock): n={len(dws)} min={min(dws):.0f} median={statistics.median(dws):.0f} max={max(dws):.0f}"
)
if loop_max:
print(
f"loop time max per 5 s: n={len(loop_max)} min={min(loop_max)} median={statistics.median(loop_max):.0f} max={max(loop_max)}"
)
@@ -1,60 +0,0 @@
"""Shared helpers for the API drivers: connect with retries, and the text of an API log line.
Import it with this directory on sys.path (the drivers add their parent directory, runners set PYTHONPATH).
"""
import asyncio
import contextlib
import os
import re
import sys
from aioesphomeapi import (
APIClient,
EncryptionPlaintextAPIError,
InvalidAuthAPIError,
InvalidEncryptionKeyAPIError,
RequiresEncryptionAPIError,
)
ANSI = re.compile(r"\x1b\[[0-9;]*[A-Za-z]")
# a wrong key or a plaintext/Noise mismatch does not go away with a retry
FATAL = (
EncryptionPlaintextAPIError,
InvalidAuthAPIError,
InvalidEncryptionKeyAPIError,
RequiresEncryptionAPIError,
)
async def connect(host: str, tries: int = 30, timeout: float = 10) -> APIClient:
"""Connect and log in, retrying once a second; exits after TRIES failures with the last error.
API_KEY in the env selects Noise; unset, empty or - is plaintext.
"""
key = os.environ.get("API_KEY")
last: BaseException | None = None
for _ in range(tries):
c = APIClient(
host, 6053, None, noise_psk=None if key in (None, "", "-") else key
)
try:
await asyncio.wait_for(c.connect(login=True), timeout)
except FATAL as e:
sys.exit(f"no API connection to {host}: {type(e).__name__}: {e}")
except Exception as e: # noqa: BLE001
last = e
# release the half-open socket so retries do not use up the device's API slots
with contextlib.suppress(Exception):
await c.disconnect(force=True)
await asyncio.sleep(1)
else:
return c
sys.exit(
f"no API connection to {host} after {tries} tries: {type(last).__name__}: {last}"
)
def log_text(msg) -> str:
"""The message of a subscribe_logs callback as text, without ANSI colour codes."""
return ANSI.sub("", msg.message.decode(errors="replace"))
@@ -1,348 +0,0 @@
# Automation code generation trace: every step logs a marker "M <case> <values>". A package: the per-run file from
# run.sh supplies the board, Wi-Fi and name. drive.py records the markers over the API.
logger:
level: DEBUG
api:
reboot_timeout: 0s
ota:
- platform: esphome
globals:
- {id: g_a, type: int, initial_value: "0"}
- {id: g_b, type: int, initial_value: "0"}
- {id: g_tick, type: int, initial_value: "0"}
- {id: g_ticking, type: bool, initial_value: "false"}
- {id: g_par, type: int, initial_value: "0"}
- {id: g_par3, type: int, initial_value: "0"}
- {id: g_par4, type: int, initial_value: "0"}
interval:
- interval: 100ms
then:
- if:
condition:
lambda: return id(g_ticking);
then:
- lambda: id(g_tick)++;
number:
- platform: template
id: num
name: Num
optimistic: true
min_value: 0
max_value: 100
step: 1
on_value:
- logger.log: {format: "M num_on_value %.0f", args: [x]}
- if:
condition:
lambda: return x > 50;
then:
- logger.log: {format: "M num_high %.0f", args: [x]}
else:
- logger.log: {format: "M num_low %.0f", args: [x]}
switch:
- platform: template
id: sw
name: Sw
optimistic: true
on_turn_on:
- logger.log: "M sw_on"
- globals.set: {id: g_b, value: !lambda "return id(g_b) + 1;"}
- logger.log: {format: "M sw_on_b %d", args: [id(g_b)]}
on_turn_off:
- logger.log: "M sw_off"
select:
- platform: template
id: sel
name: Sel
optimistic: true
options: [a, b, c]
initial_option: a
on_value:
- logger.log: {format: "M sel_on_value %s idx %d", args: [x.c_str(), (int) i]}
sensor:
- platform: template
id: sens
name: Sens
update_interval: never
on_value_range:
- above: 50
then:
- logger.log: {format: "M sens_above %.0f", args: [x]}
- below: 10
then:
- logger.log: {format: "M sens_below %.0f", args: [x]}
alarm_control_panel:
- platform: template
id: acp
name: Acp
codes: ["1234"]
requires_code_to_arm: false
arming_away_time: 0s
on_armed_away:
- logger.log: "M acp_armed_away"
on_disarmed:
- logger.log: "M acp_disarmed"
on_state:
- logger.log: "M acp_state"
button:
- platform: template
name: Run All
on_press:
- logger.log: "M press_run_all"
- script.execute: master
- platform: template
name: Chain Button
on_press:
- logger.log: "M cb_1"
- lambda: id(g_a) = 7;
- globals.set: {id: g_b, value: "3"}
- logger.log: {format: "M cb_2 %d %d", args: [id(g_a), id(g_b)]}
- switch.toggle: sw
- logger.log: "M cb_3"
script:
- id: s_single
mode: single
parameters: {x: int}
then:
- logger.log: {format: "M single_start %d", args: [x]}
- delay: 150ms
- logger.log: {format: "M single_end %d", args: [x]}
- id: s_restart
mode: restart
parameters: {x: int, s: string}
then:
- logger.log: {format: "M restart_start %d %s", args: [x, s.c_str()]}
- delay: 150ms
- logger.log: {format: "M restart_end %d %s", args: [x, s.c_str()]}
- id: s_queued
mode: queued
max_runs: 5
parameters: {x: int}
then:
- logger.log: {format: "M queued_start %d", args: [x]}
- delay: 60ms
- logger.log: {format: "M queued_end %d", args: [x]}
- id: s_parallel
mode: parallel
parameters: {x: int}
then:
- logger.log: {format: "M parallel_start %d", args: [x]}
- delay: !lambda "return 50 + x * 30;"
- logger.log: {format: "M parallel_end %d", args: [x]}
- id: s_selfstop
mode: single
then:
- logger.log: "M selfstop_1"
- lambda: id(g_a) = 100;
- script.stop: s_selfstop
- logger.log: "M selfstop_SHOULD_NOT_RUN"
- lambda: id(g_a) = 200;
- id: s_par_self
mode: parallel
max_runs: 2
then:
- lambda: id(g_par)++;
- logger.log: {format: "M parself_run %d running=%d", args: [id(g_par), (int) id(s_par_self).is_running()]}
- if:
condition:
lambda: return id(g_par) < 4;
then:
- script.execute: s_par_self
- delay: 200ms
- id: s_par3
mode: parallel
max_runs: 3
then:
- lambda: id(g_par3)++;
- logger.log: {format: "M par3_run %d", args: [id(g_par3)]}
- if:
condition:
lambda: return id(g_par3) < 8;
then:
- script.execute: s_par3
- delay: 200ms
- id: s_par4
mode: parallel
max_runs: 4
then:
- lambda: id(g_par4)++;
- logger.log: {format: "M par4_run %d", args: [id(g_par4)]}
- if:
condition:
lambda: return id(g_par4) < 8;
then:
- script.execute: s_par4
- delay: 200ms
- id: master
mode: single
then:
- logger.log: "M c1_chain_start"
- lambda: id(g_a) = 1;
- globals.set: {id: g_b, value: "2"}
- logger.log: {format: "M c1_vals %d %d", args: [id(g_a), id(g_b)]}
- switch.turn_on: sw
- number.set: {id: num, value: 42}
- select.set: {id: sel, option: b}
- lambda: |-
int local = id(g_a) + id(g_b);
if (local > 0) { ESP_LOGI("main", "M c1_lambda_local %d", local); return; }
ESP_LOGI("main", "M c1_lambda_NOT_REACHED");
- logger.log: "M c1_after_return"
- logger.log: "M c2_delay_chain_start"
- delay: 50ms
- logger.log: "M c2_after_delay"
- lambda: id(g_a) = 5;
- delay: 0ms
- logger.log: {format: "M c2_end %d", args: [id(g_a)]}
- logger.log: "M c3_if_start"
- repeat:
count: 4
then:
- if:
condition:
and:
- lambda: return iteration % 2 == 0;
- not:
lambda: return iteration == 2;
then:
- logger.log: {format: "M c3_even_not2 %d", args: [iteration]}
else:
- if:
condition:
or:
- lambda: return iteration == 1;
- xor:
- lambda: return iteration == 2;
- lambda: return false;
then:
- logger.log: {format: "M c3_or_xor %d", args: [iteration]}
else:
- logger.log: {format: "M c3_else %d", args: [iteration]}
- logger.log: "M c4_choose_start"
- repeat:
count: 3
then:
- choose:
- if:
lambda: return iteration == 0;
then:
- logger.log: {format: "M c4_choice0 %d", args: [iteration]}
- if:
lambda: return iteration == 1;
then:
- logger.log: {format: "M c4_choice1 %d", args: [iteration]}
- else:
- logger.log: {format: "M c4_default %d", args: [iteration]}
- logger.log: "M c5_while_start"
- globals.set: {id: g_a, value: "0"}
- while:
condition:
lambda: return id(g_a) < 4;
then:
- logger.log: {format: "M c5_while %d", args: [id(g_a)]}
- lambda: id(g_a)++;
- logger.log: {format: "M c5_while_done %d", args: [id(g_a)]}
- globals.set: {id: g_a, value: "0"}
- while:
condition:
lambda: return id(g_a) < 3;
then:
- logger.log: {format: "M c5_while_delay %d", args: [id(g_a)]}
- delay: 20ms
- lambda: id(g_a)++;
- logger.log: {format: "M c5_while_delay_done %d", args: [id(g_a)]}
- logger.log: "M c6_repeat_start"
- repeat:
count: 2
then:
- lambda: id(g_b) = iteration;
- repeat:
count: 3
then:
- logger.log: {format: "M c6_nested outer=%d inner=%d", args: [id(g_b), iteration]}
- repeat:
count: !lambda "return id(g_b) + 2;"
then:
- logger.log: {format: "M c6_dyn %d", args: [iteration]}
- delay: 10ms
- logger.log: "M c7_wait_until_start"
- lambda: |-
id(g_tick) = 0;
id(g_ticking) = true;
- wait_until:
condition:
lambda: return id(g_tick) >= 3;
timeout: 2s
- logger.log: {format: "M c7_waited %d", args: [(int) (id(g_tick) >= 3)]}
- wait_until:
condition:
for:
time: 300ms
condition:
lambda: return id(g_tick) >= 3;
timeout: 3s
- lambda: id(g_ticking) = false;
- logger.log: "M c7_for_done"
- logger.log: "M c8_scripts_start"
- script.execute: {id: s_single, x: 1}
- script.execute: {id: s_single, x: 2}
- script.wait: s_single
- script.execute: {id: s_restart, x: 1, s: one}
- delay: 40ms
- script.execute: {id: s_restart, x: 2, s: two}
- script.wait: s_restart
- script.execute: {id: s_queued, x: 1}
- script.execute: {id: s_queued, x: 2}
- script.execute: {id: s_queued, x: 3}
- script.wait: s_queued
- script.execute: {id: s_parallel, x: 2}
- script.execute: {id: s_parallel, x: 1}
- script.wait: s_parallel
- logger.log: "M c9_selfstop_start"
- script.execute: s_selfstop
- script.wait: s_selfstop
- logger.log: {format: "M c9_selfstop_g_a %d", args: [id(g_a)]}
- logger.log: "M c10_sensor_start"
- sensor.template.publish: {id: sens, state: 60}
- sensor.template.publish: {id: sens, state: 30}
- sensor.template.publish: {id: sens, state: 5}
- logger.log: "M c11_acp_start"
- alarm_control_panel.arm_away: acp
- delay: 100ms
- alarm_control_panel.disarm: {id: acp, code: "1234"}
- delay: 100ms
- logger.log: "M c12_parself_start"
- globals.set: {id: g_par, value: "0"}
- script.execute: s_par_self
- delay: 800ms
- logger.log: {format: "M c12_parself_total %d", args: [id(g_par)]}
- logger.log: "M c13_parmax_start"
- globals.set: {id: g_par3, value: "0"}
- script.execute: s_par3
- delay: 800ms
- logger.log: {format: "M c13_par3_total %d", args: [id(g_par3)]}
- globals.set: {id: g_par4, value: "0"}
- script.execute: s_par4
- delay: 800ms
- logger.log: {format: "M c13_par4_total %d", args: [id(g_par4)]}
- switch.turn_off: sw
- logger.log: "M done"
esphome:
on_boot:
priority: -100
then:
- logger.log: "M boot"
- delay: 8s
- script.execute: master
@@ -1,72 +0,0 @@
"""Record every 'M ...' marker from the device log while driving the automation trace config.
usage: drive.py HOST OUT env: API_KEY (unset for a plaintext API)
Order: wait for the boot run's 'M done', press Run All and wait for 'M done' again, press Chain Button twice, set Num
70 then 20, toggle Sw on and off, select c then a. Writes one marker per line to OUT (timestamps dropped). Exits 1
when the Run All pass never reached 'M done'.
"""
import asyncio
import pathlib
import re
import sys
from aioesphomeapi import LogLevel
sys.path.insert(0, str(pathlib.Path(__file__).resolve().parents[1]))
from apiconn import connect, log_text # noqa: E402
HOST, OUT = sys.argv[1], sys.argv[2]
marks: list[str] = []
done = asyncio.Event()
async def main() -> bool:
cli = await connect(HOST)
def on_log(m) -> None:
if mm := re.search(r"\]: (M .*)$", log_text(m)):
marks.append(mm.group(1).strip())
if mm.group(1).strip() == "M done":
done.set()
cli.subscribe_logs(on_log, log_level=LogLevel.LOG_LEVEL_DEBUG)
infos, _ = await cli.list_entities_services()
by = {i.name: i for i in infos}
try:
await asyncio.wait_for(done.wait(), 40)
except TimeoutError:
marks.append("DRIVER boot run not seen")
marks.append("DRIVER press Run All")
done.clear()
cli.button_command(by["Run All"].key)
ok = True
try:
await asyncio.wait_for(done.wait(), 40)
except TimeoutError:
marks.append("DRIVER run all timed out")
ok = False
steps = (
("cb1", lambda: cli.button_command(by["Chain Button"].key)),
("cb2", lambda: cli.button_command(by["Chain Button"].key)),
("num70", lambda: cli.number_command(by["Num"].key, 70.0)),
("num20", lambda: cli.number_command(by["Num"].key, 20.0)),
("swon", lambda: cli.switch_command(by["Sw"].key, True)),
("swoff", lambda: cli.switch_command(by["Sw"].key, False)),
("selc", lambda: cli.select_command(by["Sel"].key, "c")),
("sela", lambda: cli.select_command(by["Sel"].key, "a")),
)
for label, send in steps:
marks.append(f"DRIVER {label}")
send()
await asyncio.sleep(0.8)
await asyncio.sleep(1)
await cli.disconnect()
return ok
OK = asyncio.run(main())
pathlib.Path(OUT).write_text("\n".join(marks) + "\n", encoding="utf-8")
print(f"{len(marks)} lines")
sys.exit(0 if OK else 1)
@@ -1,40 +0,0 @@
#!/bin/bash
# run.sh LABEL: build the automation trace config as at<LABEL> from TREE, flash it over PORT, read the device address
# from the serial log, record the marker trace with drive.py and keep the part from "DRIVER press Run All" on in
# out-at<LABEL>.trace. Diff the .trace files of dev and the PR; the boot run is partly missed while the API connects.
# Run it from a scratch copy of this directory with a secrets.yaml (wifi_ssid, wifi_password).
# env: TREE (esphome checkout), PORT (serial port), SC (path to serial_capture.py), PY (checkout python),
# BOARD (esp32 board, default esp32dev), VARIANT (default esp32), API_KEY (only if you add api encryption)
R=$(cd "$(dirname "$0")" && pwd); L=${1:?label}; T=${TREE:?set TREE}; P=${PORT:?set PORT}
PY=${PY:-$T/venv/bin/python}; SC=${SC:?set SC}; NAME=at$L; OUT=$R/out-$NAME; rm -rf "$OUT"; mkdir -p "$OUT"
cat > "$R/$NAME.yaml" <<YAML
esphome:
name: $NAME
esp32:
board: ${BOARD:-esp32dev}
variant: ${VARIANT:-esp32}
framework:
type: esp-idf
wifi:
ssid: !secret wifi_ssid
password: !secret wifi_password
packages:
trace: !include automation-trace-common.yaml
YAML
pkill -f "[s]erial_capture.py $P"
(cd "$T" && $PY -m esphome run --no-logs "$R/$NAME.yaml" --device "$P" > "$OUT/upload.log" 2>&1) || { echo "build or upload failed"; exit 1; }
grep -a -E '^(RAM|Flash):' "$OUT/upload.log"
$PY "$SC" "$P" 120 "$OUT/serial.txt" --reset > /dev/null 2>&1 &
CAP=$!; trap 'kill $CAP 2>/dev/null' EXIT
H=; for _ in $(seq 60); do
H=$(grep -a -o -E 'IP Address: [0-9.]+' "$OUT/serial.txt" 2>/dev/null | tail -1 | awk '{print $3}'); [ -n "$H" ] && break; sleep 1
done
[ -n "$H" ] || { echo "FAIL: no IP address in the serial log"; exit 1; }
PYTHONPATH=$(dirname "$SC") $PY "$R/drive.py" "$H" "$OUT/markers.txt"; RC=$?
sed -n '/^DRIVER press Run All/,$p' "$OUT/markers.txt" > "$R/out-$NAME.trace"
CR=$(grep -a -c -E 'Guru|abort\(\)|Backtrace|task_wdt' "$OUT/serial.txt")
echo "$NAME: $(wc -l < "$R/out-$NAME.trace") trace lines, crashes $CR"
[ "$CR" = 0 ] || { echo "FAIL: a crash or watchdog in the serial log"; exit 1; }
[ "$RC" = 0 ] || { echo "FAIL: Run All never reached M done"; exit 1; }
grep -q -E "SHOULD_NOT_RUN|NOT_REACHED" "$R/out-$NAME.trace" && { echo "FAIL: a step that must be skipped ran"; exit 1; }
echo PASS
@@ -1,64 +0,0 @@
"""Compare two full flash dumps of a BK72xx board per partition of its LibreTiny board layout.
usage: partdiff.py BEFORE.bin AFTER.bin LAYOUT.json
LAYOUT.json is a board base file from the LibreTiny platform (for example boards/_base/beken-72xx.json); its
"flash" map ("name": "0xSTART+0xLENGTH") names the partitions. Prints the 4 KB sectors that changed in each one,
plus any changed sector outside the map (a WARN, to check by hand). Exits 1 when the bootloader, calibration or
userdata partition changed.
"""
import json
import pathlib
import sys
before, after = (pathlib.Path(p).read_bytes() for p in sys.argv[1:3])
if len(before) != len(after):
sys.exit(
f"dump sizes differ: {len(before)} and {len(after)} bytes; compare two whole chip reads"
)
parts = []
for name, span in json.loads(pathlib.Path(sys.argv[3]).read_text(encoding="utf-8"))[
"flash"
].items():
start, length = (int(x, 16) for x in span.split("+"))
parts.append((start, start + length, name))
parts.sort()
PROTECTED = ("bootloader", "calibration", "userdata")
if missing := [n for n in PROTECTED if n not in {name for _, _, name in parts}]:
sys.exit(
f"the layout has no {', '.join(missing)} partition; use the Beken board base file"
)
if (end := max(e for _, e, _ in parts)) > len(before):
sys.exit(
f"the layout reaches {end:#x} but the dumps are {len(before):#x} bytes; read the whole chip"
)
SECTOR = 0x1000
covered = set()
protected = []
for start, end, name in parts:
changed = [
a
for a in range(start, end, SECTOR)
if before[a : a + SECTOR] != after[a : a + SECTOR]
]
covered.update(range(start, end, SECTOR))
if changed and name in PROTECTED:
protected.append(name)
listed = " ".join(hex(a) for a in changed[:4]) if 0 < len(changed) <= 4 else ""
print(
f"{name:12} {len(changed):4} of {(end - start) // SECTOR:4} sectors changed {listed}"
)
other = [
a
for a in range(0, len(before), SECTOR)
if a not in covered and before[a : a + SECTOR] != after[a : a + SECTOR]
]
print(
f"{'unmapped':12} {len(other):4} sectors changed {' '.join(hex(a) for a in other[:8])}"
)
if other:
print(
"WARN: sectors outside the layout changed; check what they held in the backup"
)
if protected:
sys.exit(f"FAIL: {', '.join(protected)} changed")
@@ -1,19 +0,0 @@
#!/bin/bash
# usage: readback.sh TAG [START LENGTH] read flash from a BK72xx over its adapter into TAG.bin (whole chip by default)
# env: PORT (adapter), FAMILY (ltchiptool family, default beken-7238), FLASH_BK (path to flash_bk72xx.py),
# LT (LibreTiny penv python, default ~/.platformio/penv/.libretiny/bin/python), SIZE (whole chip bytes, default
# 2097152). A whole 2 MB chip takes about 190 s. The read must come back at the expected size.
T=${1:?tag}; P=${PORT:?set PORT}; FB=${FLASH_BK:?set FLASH_BK}; LT=${LT:-$HOME/.platformio/penv/.libretiny/bin/python}
ARGS=(); [ -n "$2" ] && ARGS=(-s "$2" -l "${3:?length}")
WANT=$(( ${3:-${SIZE:-2097152}} ))
: > "$T.log"
ok=
for i in 1 2 3; do
echo "### try $i" >> "$T.log"
rm -f "$T.bin"
timeout 900 "$LT" "$FB" flash read -d "$P" "${ARGS[@]}" "${FAMILY:-beken-7238}" "$T.bin" >> "$T.log" 2>&1 &&
[ "$(wc -c < "$T.bin" | tr -d ' ')" = "$WANT" ] && ok=1 && break
sleep 3
done
[ -n "$ok" ] || { echo "FAIL: read $T (expected $WANT bytes, see $T.log)"; exit 1; }
shasum -a 256 "$T.bin"
@@ -1,61 +0,0 @@
#pragma once
// Rotating non-connectable advertisements with known service and manufacturer data (random static addresses).
#include <esp_gap_ble_api.h>
#include <cstring>
#include "esphome/core/log.h"
namespace af {
struct Frame {
uint8_t last; // address C4:C1:38:00:00:<last>
uint8_t len;
uint8_t adv[31];
};
// AD structures after the flags. 0x16 = service data 16-bit, 0x20 = 32-bit, 0x21 = 128-bit, 0xFF = manufacturer.
static const Frame FRAMES[] = {
// 21: service 181C AA01 + manufacturer 02E5 BB01
{0x21, 12, {5, 0x16, 0x1C, 0x18, 0xAA, 0x01, 5, 0xFF, 0xE5, 0x02, 0xBB, 0x01}},
// 22: service 32-bit 12345678 AA02
{0x22, 8, {7, 0x20, 0x78, 0x56, 0x34, 0x12, 0xAA, 0x02}},
// 23: service 128-bit 12345678-9ABC-DEF0-1122-334455667788 (wire order) AA03
{0x23, 20, {19, 0x21, 0x88, 0x77, 0x66, 0x55, 0x44, 0x33, 0x22, 0x11,
0xF0, 0xDE, 0xBC, 0x9A, 0x78, 0x56, 0x34, 0x12, 0xAA, 0x03}},
// 24: manufacturer 181C BB04 (same id as the service UUID of 21, other data kind)
{0x24, 6, {5, 0xFF, 0x1C, 0x18, 0xBB, 0x04}},
// 25: service 181C AA05 then a second 181C entry AA06 (only the first may fire)
{0x25, 12, {5, 0x16, 0x1C, 0x18, 0xAA, 0x05, 5, 0x16, 0x1C, 0x18, 0xAA, 0x06}},
// 26: the 128-bit UUID in printed byte order AA07 (must not match)
{0x26, 20, {19, 0x21, 0x12, 0x34, 0x56, 0x78, 0x9A, 0xBC, 0xDE, 0xF0,
0x11, 0x22, 0x33, 0x44, 0x55, 0x66, 0x77, 0x88, 0xAA, 0x07}},
};
static constexpr int COUNT = sizeof(FRAMES) / sizeof(FRAMES[0]);
static uint32_t failed = 0;
static uint32_t sent = 0;
inline void send(int i) {
const Frame &f = FRAMES[i];
uint8_t adv[31];
uint8_t n = 0;
adv[n++] = 2;
adv[n++] = 0x01;
adv[n++] = 0x06;
memcpy(adv + n, f.adv, f.len);
n += f.len;
esp_ble_gap_stop_advertising();
uint8_t addr[6] = {0xC4, 0xC1, 0x38, 0x00, 0x00, f.last};
esp_err_t e1 = esp_ble_gap_set_rand_addr(addr);
esp_err_t e2 = esp_ble_gap_config_adv_data_raw(adv, n);
esp_ble_adv_params_t p{};
p.adv_int_min = 0x20; // 20 ms
p.adv_int_max = 0x20;
p.adv_type = ADV_TYPE_NONCONN_IND;
p.own_addr_type = BLE_ADDR_TYPE_RANDOM;
p.channel_map = ADV_CHNL_ALL;
p.adv_filter_policy = ADV_FILTER_ALLOW_SCAN_ANY_CON_ANY;
esp_err_t e3 = esp_ble_gap_start_advertising(&p);
sent++;
if (e1 != ESP_OK || e2 != ESP_OK || e3 != ESP_OK) {
failed++;
ESP_LOGW("af", "frame %d failed: %d %d %d", i, e1, e2, e3);
}
}
} // namespace af
@@ -1,41 +0,0 @@
# Advertiser for the service and manufacturer data trigger test: any ESP32 with BLE. Every 250 ms it switches to
# the next of six raw non-connectable advertisements in adv_frames.hpp (addresses C4:C1:38:00:00:21 to 26).
esphome:
name: advtrig
includes:
- adv_frames.hpp
esp32:
board: esp32-s3-devkitc-1
variant: esp32s3
framework:
type: esp-idf
logger:
wifi:
ssid: !secret wifi_ssid
password: !secret wifi_password
api:
reboot_timeout: 0s
ota:
- platform: esphome
esp32_ble:
globals:
- id: idx
type: int
initial_value: "0"
interval:
- interval: 250ms
then:
- lambda: |-
af::send(id(idx));
id(idx) = (id(idx) + 1) % af::COUNT;
- interval: 30s
then:
- lambda: ESP_LOGI("af", "sent %u, failed %u", (unsigned) af::sent, (unsigned) af::failed);
@@ -1,72 +0,0 @@
"""check.py CAPTURE [--zero-mac any|filter]: judge the last count report of a receiver capture.
Expected per trigger (see triggers.yaml): T1 AA01+AA05, T2 AA01, T3 nothing, T4 AA02, T5 AA03, T6 BB01, T7 BB04,
T8 the same count as T2, T9 (mac 00:00:00:00:00:00) AA01+AA05 with --zero-mac any or nothing with filter,
T10 AA03, T11 nothing. "other" on a trigger without a MAC filter is another device in range and is only reported.
Prints PASS or FAIL with the reason and exits 0 or 1.
"""
import argparse
import pathlib
import re
import sys
NO_MAC = {1, 7, 10}
def main() -> int:
parser = argparse.ArgumentParser(
description=__doc__, formatter_class=argparse.RawDescriptionHelpFormatter
)
parser.add_argument("capture")
parser.add_argument("--zero-mac", choices=["any", "filter"], default="filter")
args = parser.parse_args()
zero = args.zero_mac
text = pathlib.Path(args.capture).read_text(encoding="utf-8", errors="replace")
rows: dict[int, dict[str, int]] = {}
for m in re.finditer(r"count T(\d+):([^\n]*)", text):
counts = dict(re.findall(r"(AA\d\d|BB\d\d|other)=(\d+)", m.group(2)))
rows[int(m.group(1))] = {k: int(v) for k, v in counts.items()}
if len(rows) != 11:
print(
f"FAIL: found {len(rows)} trigger rows, want 11 (no report in the capture?)"
)
return 1
want = {
1: {"AA01", "AA05"},
2: {"AA01"},
3: set(),
4: {"AA02"},
5: {"AA03"},
6: {"BB01"},
7: {"BB04"},
8: {"AA01"},
9: {"AA01", "AA05"} if zero == "any" else set(),
10: {"AA03"},
11: set(),
}
# With --zero-mac any, the all-zero MAC means no filter, so T9 also hears other devices
no_mac = NO_MAC | {9} if zero == "any" else NO_MAC
bad = []
for t, keys in want.items():
got = {k for k in rows[t] if k != "other"}
if got != keys:
bad.append(f"T{t} fired on {sorted(got)}, want {sorted(keys)}")
if rows[t].get("other") and t not in no_mac:
bad.append(
f"T{t} has other={rows[t]['other']} although it filters on a MAC"
)
if keys and min(rows[t].get(k, 0) for k in keys) < 10:
bad.append(
f"T{t} has fewer than 10 hits on one of {sorted(keys)}: {rows[t]}"
)
if rows[8] != rows[2]:
bad.append(f"T8 {rows[8]} differs from T2 {rows[2]} (same filter)")
for t in sorted(rows):
print(f"T{t}: {rows[t]}")
print("FAIL: " + "; ".join(bad) if bad else "PASS")
return 1 if bad else 0
if __name__ == "__main__":
sys.exit(main())
@@ -1,21 +0,0 @@
#!/bin/bash
# run.sh esp32|bk72xx LABEL: build rx-<kind>.yaml as rx<kind><LABEL> from TREE, flash it, capture 150 s, run check.py.
# The advertiser (advtrig.yaml) must already be running on a second ESP32 in range.
# env: TREE (esphome checkout to build from), PORT (receiver serial port), SC (path to serial_capture.py),
# PY (checkout python), BOARD (bk72xx board, default xh-wb3s), BKPY + BKFLASH (python and flash_bk72xx.py for
# bk72xx), ZERO_MAC (any or filter, what mac_address 00:00:00:00:00:00 should do on this build; default filter)
R=$(cd "$(dirname "$0")" && pwd); K=${1:?esp32 or bk72xx}; L=${2:?label}; T=${TREE:?set TREE}; P=${PORT:?set PORT}
PY=${PY:-$T/venv/bin/python}; SC=${SC:?set SC}; NAME=rx$K$L; OUT=$R/out-$NAME; rm -rf "$OUT"; mkdir -p "$OUT"
sed -e "s/\${name}/$NAME/" -e "s/\${board}/${BOARD:-xh-wb3s}/" "$R/rx-$K.yaml" > "$R/$NAME.yaml"
pkill -f "[s]erial_capture.py $P"
if [ "$K" = esp32 ]; then
(cd "$T" && $PY -m esphome run --no-logs "$R/$NAME.yaml" --device "$P" > "$OUT/upload.log" 2>&1) || { echo "build or upload failed"; exit 1; }
$PY "$SC" "$P" 150 "$OUT/serial.txt" --reset > /dev/null 2>&1
else
(cd "$T" && $PY -m esphome compile "$R/$NAME.yaml" > "$OUT/compile.log" 2>&1) || { echo "build failed"; exit 1; }
${BKPY:?set BKPY} "${BKFLASH:?set BKFLASH}" flash write -d "$P" "$R/.esphome/build/$NAME/.pioenvs/$NAME/firmware.uf2" > "$OUT/flash.log" 2>&1 || { echo "flash failed"; exit 1; }
$PY "$SC" "$P" 150 "$OUT/serial.txt" > /dev/null 2>&1
fi
[ -s "$OUT/serial.txt" ] || { echo "serial capture got nothing"; exit 1; }
grep -a "adverts" "$OUT/serial.txt" | tail -1
$PY "$R/check.py" "$OUT/serial.txt" --zero-mac "${ZERO_MAC:-filter}"
@@ -1,39 +0,0 @@
# Receiver on a BK72xx board (bk72xx_ble_tracker). substitutions: name, board
# Wi-Fi must be configured: without it the BK7238 tracker logs "Scan started" and never receives anything.
esphome:
name: ${name}
includes:
- rx.hpp
bk72xx:
board: ${board}
logger:
hardware_uart: UART1
level: DEBUG
wifi:
ssid: !secret wifi_ssid
password: !secret wifi_password
bk72xx_ble:
enable_on_boot: true
bk72xx_ble_tracker:
scan_parameters:
interval: 100ms
window: 30ms
continuous: true
<<: !include triggers.yaml
on_ble_advertise:
- then:
- lambda: |-
rx::all++;
if ((x.address_uint64() >> 8) == 0xC4C1380000ULL) rx::ours++;
interval:
- interval: 20s
then:
- lambda: |-
rx::report("count");
ESP_LOGI("rx", "adverts %u, from the advertiser %u", (unsigned) rx::all, (unsigned) rx::ours);
@@ -1,32 +0,0 @@
# Receiver on a classic ESP32 (esp32_ble_tracker). substitutions: name
esphome:
name: ${name}
includes:
- rx.hpp
esp32:
board: esp32dev
framework:
type: esp-idf
logger:
level: DEBUG
esp32_ble_tracker:
scan_parameters:
interval: 100ms
window: 100ms
active: false
<<: !include triggers.yaml
on_ble_advertise:
- then:
- lambda: |-
rx::all++;
if ((x.address_uint64() >> 8) == 0xC4C1380000ULL) rx::ours++;
interval:
- interval: 20s
then:
- lambda: |-
rx::report("count");
ESP_LOGI("rx", "adverts %u, from the advertiser %u", (unsigned) rx::all, (unsigned) rx::ours);
@@ -1,37 +0,0 @@
#pragma once
// Per trigger counts of the received payloads: AAnn (service data) and BBnn (manufacturer data) from the advertiser,
// anything else counted as "other".
#include <cstdint>
#include <cstdio>
#include <vector>
#include "esphome/core/log.h"
namespace rx {
static constexpr int TRIGGERS = 12;
static uint32_t cnt[TRIGGERS][16];
static uint32_t other[TRIGGERS];
static uint32_t all, ours;
inline void hit(int t, const std::vector<uint8_t> &x) {
if (x.size() == 2 && (x[0] == 0xAA || x[0] == 0xBB) && x[1] >= 1 && x[1] <= 7) {
cnt[t][(x[0] == 0xBB ? 8 : 0) + x[1]]++;
} else {
other[t]++;
}
}
inline void report(const char *label) {
for (int t = 1; t < TRIGGERS; t++) {
char buf[160];
int n = snprintf(buf, sizeof(buf), "T%d:", t);
for (int k = 0; k < 16; k++) {
if (cnt[t][k])
n += snprintf(buf + n, sizeof(buf) - n, " %s%02u=%u", k >= 8 ? "BB" : "AA", (unsigned) (k & 7),
(unsigned) cnt[t][k]);
}
if (other[t])
n += snprintf(buf + n, sizeof(buf) - n, " other=%u", (unsigned) other[t]);
ESP_LOGI("rx", "%s %s", label, buf);
}
}
} // namespace rx
@@ -1,44 +0,0 @@
# Shared trigger block, included under the tracker key.
on_ble_service_data_advertise:
- service_uuid: 181C
then:
- lambda: rx::hit(1, x);
- mac_address: C4:C1:38:00:00:21
service_uuid: 181C
then:
- lambda: rx::hit(2, x);
- mac_address: C4:C1:38:00:00:22
service_uuid: 181C
then:
- lambda: rx::hit(3, x);
- mac_address: C4:C1:38:00:00:22
service_uuid: "12345678"
then:
- lambda: rx::hit(4, x);
- mac_address: C4:C1:38:00:00:23
service_uuid: 12345678-9ABC-DEF0-1122-334455667788
then:
- lambda: rx::hit(5, x);
- mac_address: C4:C1:38:00:00:21
service_uuid: 181C
then:
- lambda: rx::hit(8, x);
- mac_address: 00:00:00:00:00:00
service_uuid: 181C
then:
- lambda: rx::hit(9, x);
- service_uuid: 12345678-9ABC-DEF0-1122-334455667788
then:
- lambda: rx::hit(10, x);
on_ble_manufacturer_data_advertise:
- mac_address: C4:C1:38:00:00:21
manufacturer_id: 02E5
then:
- lambda: rx::hit(6, x);
- manufacturer_id: 181C
then:
- lambda: rx::hit(7, x);
- mac_address: C4:C1:38:00:00:22
manufacturer_id: 02E5
then:
- lambda: rx::hit(11, x);
@@ -1,39 +0,0 @@
# Add to a copy of a bluetooth_proxy config: packages: {cp_inject: !include cp-inject.yaml}
esphome:
includes:
- cp_inject.hpp
logger:
level: VERBOSE
logs:
esp32_ble_tracker: DEBUG
bluetooth_proxy: DEBUG
bluetooth_connection: DEBUG
api: DEBUG
api.service: DEBUG
api.connection: DEBUG
api.socket: DEBUG
scheduler: DEBUG
sensor: DEBUG
component: DEBUG
button:
- platform: template
name: Inject conn param events
# The handlers push into a queue that takes one writer only, the Bluetooth task. Stop the scan (and keep the
# proxy free of real connections) so nothing else writes while the button does.
on_press:
- esp32_ble_tracker.stop_scan:
- wait_until:
condition:
lambda: return !esp32_ble_tracker::global_esp32_ble_tracker->scan_running();
timeout: 5s
- delay: 1s
- lambda: |-
if (esp32_ble_tracker::global_esp32_ble_tracker->scan_running()) {
ESP_LOGE("cpinject", "scan still running after 5 s, nothing injected");
return;
}
cpinject::run();
- esp32_ble_tracker.start_scan:
continuous: true
Loaded 100 of 10325 files, more files were not shown because too many files have changed in this diff. Show more