Co-authored-by: pre-commit-ci-lite[bot] <117423508+pre-commit-ci-lite[bot]@users.noreply.github.com>
Co-authored-by: J. Nick Koston <nick@home-assistant.io>
The neutral engine reports both the user disconnect trigger and the
connect action's abort hook from one callback list, so an on_disconnect
automation that calls ble_client.connect cancelled the attempt it had
just started and never ran the rest of its chain. The legacy engine
avoided this by keying the two on different events (CLOSE_EVT for the
trigger, DISCONNECT_EVT for the abort); the hook now skips the abort
while a fresh attempt is in flight.
connected() no longer reports a link with an accepted teardown in
flight, so a connect action started in that window stops taking the
already-connected fast path, and a second disconnect() in that window
is ignored rather than settling the link twice.
The write/notify completion contract claimed every completion arrives
from the event loop, which the rp2 backend contradicts for the two
local-only operations; it now names them and requires consumers to arm
their pending state before issuing an op.
Co-authored-by: pre-commit-ci-lite[bot] <117423508+pre-commit-ci-lite[bot]@users.noreply.github.com>
Co-authored-by: J. Nick Koston <nick@home-assistant.io>
Co-authored-by: J. Nick Koston <nick@koston.org>
Co-authored-by: pre-commit-ci-lite[bot] <117423508+pre-commit-ci-lite[bot]@users.noreply.github.com>
Co-authored-by: J. Nick Koston <nick@koston.org>
Co-authored-by: J. Nick Koston <nick@home-assistant.io>
Co-authored-by: pre-commit-ci-lite[bot] <117423508+pre-commit-ci-lite[bot]@users.noreply.github.com>
Co-authored-by: J. Nick Koston <nick@home-assistant.io>
Co-authored-by: pre-commit-ci-lite[bot] <117423508+pre-commit-ci-lite[bot]@users.noreply.github.com>
Co-authored-by: J. Nick Koston <nick@home-assistant.io>
Co-authored-by: pre-commit-ci-lite[bot] <117423508+pre-commit-ci-lite[bot]@users.noreply.github.com>
Co-authored-by: J. Nick Koston <nick@home-assistant.io>
Co-authored-by: J. Nick Koston <nick@koston.org>
Co-authored-by: pre-commit-ci-lite[bot] <117423508+pre-commit-ci-lite[bot]@users.noreply.github.com>
Co-authored-by: J. Nick Koston <nick@home-assistant.io>
Co-authored-by: pre-commit-ci-lite[bot] <117423508+pre-commit-ci-lite[bot]@users.noreply.github.com>
Co-authored-by: J. Nick Koston <nick@koston.org>
Co-authored-by: J. Nick Koston <nick@home-assistant.io>
Co-authored-by: pre-commit-ci-lite[bot] <117423508+pre-commit-ci-lite[bot]@users.noreply.github.com>
Co-authored-by: J. Nick Koston <nick@koston.org>
Co-authored-by: J. Nick Koston <nick@home-assistant.io>
Co-authored-by: pre-commit-ci-lite[bot] <117423508+pre-commit-ci-lite[bot]@users.noreply.github.com>
Co-authored-by: J. Nick Koston <nick@home-assistant.io>
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
Co-authored-by: pre-commit-ci-lite[bot] <117423508+pre-commit-ci-lite[bot]@users.noreply.github.com>
Co-authored-by: pre-commit-ci-lite[bot] <117423508+pre-commit-ci-lite[bot]@users.noreply.github.com>
Co-authored-by: J. Nick Koston <nick@koston.org>
Co-authored-by: J. Nick Koston <nick@home-assistant.io>
Co-authored-by: pre-commit-ci-lite[bot] <117423508+pre-commit-ci-lite[bot]@users.noreply.github.com>
Co-authored-by: J. Nick Koston <nick@home-assistant.io>
Co-authored-by: J. Nick Koston <nick@koston.org>
Co-authored-by: pre-commit-ci-lite[bot] <117423508+pre-commit-ci-lite[bot]@users.noreply.github.com>
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
Co-authored-by: J. Nick Koston <nick@home-assistant.io>
Co-authored-by: J. Nick Koston <nick@koston.org>
Co-authored-by: pre-commit-ci-lite[bot] <117423508+pre-commit-ci-lite[bot]@users.noreply.github.com>
Co-authored-by: J. Nick Koston <nick@koston.org>
Co-authored-by: J. Nick Koston <nick@home-assistant.io>
Co-authored-by: pre-commit-ci-lite[bot] <117423508+pre-commit-ci-lite[bot]@users.noreply.github.com>
Co-authored-by: J. Nick Koston <nick@koston.org>
Co-authored-by: J. Nick Koston <nick@home-assistant.io>
Co-authored-by: Claude Sonnet 5 <noreply@anthropic.com>
Co-authored-by: pre-commit-ci-lite[bot] <117423508+pre-commit-ci-lite[bot]@users.noreply.github.com>
Co-authored-by: J. Nick Koston <nick@home-assistant.io>
Co-authored-by: J. Nick Koston <nick@koston.org>
Co-authored-by: pre-commit-ci-lite[bot] <117423508+pre-commit-ci-lite[bot]@users.noreply.github.com>
Co-authored-by: J. Nick Koston <nick@koston.org>
Co-authored-by: J. Nick Koston <nick@home-assistant.io>
Co-authored-by: pre-commit-ci-lite[bot] <117423508+pre-commit-ci-lite[bot]@users.noreply.github.com>
Co-authored-by: J. Nick Koston <nick@home-assistant.io>
Co-authored-by: pre-commit-ci-lite[bot] <117423508+pre-commit-ci-lite[bot]@users.noreply.github.com>
Co-authored-by: J. Nick Koston <nick@home-assistant.io>
Co-authored-by: J. Nick Koston <nick@koston.org>
Co-authored-by: pre-commit-ci-lite[bot] <117423508+pre-commit-ci-lite[bot]@users.noreply.github.com>
Co-authored-by: J. Nick Koston <nick@home-assistant.io>
Co-authored-by: pre-commit-ci-lite[bot] <117423508+pre-commit-ci-lite[bot]@users.noreply.github.com>
Co-authored-by: J. Nick Koston <nick@home-assistant.io>
Co-authored-by: J. Nick Koston <nick@koston.org>
Co-authored-by: pre-commit-ci-lite[bot] <117423508+pre-commit-ci-lite[bot]@users.noreply.github.com>
Co-authored-by: J. Nick Koston <nick@home-assistant.io>
Co-authored-by: pre-commit-ci-lite[bot] <117423508+pre-commit-ci-lite[bot]@users.noreply.github.com>
Co-authored-by: J. Nick Koston <nick@home-assistant.io>
Co-authored-by: Claude <noreply@anthropic.com>
Co-authored-by: pre-commit-ci-lite[bot] <117423508+pre-commit-ci-lite[bot]@users.noreply.github.com>
Co-authored-by: J. Nick Koston <nick@home-assistant.io>
Co-authored-by: pre-commit-ci-lite[bot] <117423508+pre-commit-ci-lite[bot]@users.noreply.github.com>
Co-authored-by: J. Nick Koston <nick@koston.org>
Co-authored-by: J. Nick Koston <nick@home-assistant.io>
Co-authored-by: pre-commit-ci-lite[bot] <117423508+pre-commit-ci-lite[bot]@users.noreply.github.com>
Co-authored-by: J. Nick Koston <nick@koston.org>
Co-authored-by: J. Nick Koston <nick@home-assistant.io>
Co-authored-by: pre-commit-ci-lite[bot] <117423508+pre-commit-ci-lite[bot]@users.noreply.github.com>
Co-authored-by: J. Nick Koston <nick@koston.org>
Co-authored-by: J. Nick Koston <nick@home-assistant.io>
Co-authored-by: pre-commit-ci-lite[bot] <117423508+pre-commit-ci-lite[bot]@users.noreply.github.com>
Co-authored-by: J. Nick Koston <nick@home-assistant.io>
Co-authored-by: pre-commit-ci-lite[bot] <117423508+pre-commit-ci-lite[bot]@users.noreply.github.com>
Co-authored-by: J. Nick Koston <nick@koston.org>
Co-authored-by: J. Nick Koston <nick@home-assistant.io>
Co-authored-by: pre-commit-ci-lite[bot] <117423508+pre-commit-ci-lite[bot]@users.noreply.github.com>
Co-authored-by: J. Nick Koston <nick@koston.org>
Co-authored-by: J. Nick Koston <nick@home-assistant.io>
Co-authored-by: pre-commit-ci-lite[bot] <117423508+pre-commit-ci-lite[bot]@users.noreply.github.com>
Co-authored-by: J. Nick Koston <nick@koston.org>
Co-authored-by: J. Nick Koston <nick@home-assistant.io>
Co-authored-by: Cursor <cursoragent@cursor.com>
Co-authored-by: pre-commit-ci-lite[bot] <117423508+pre-commit-ci-lite[bot]@users.noreply.github.com>
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
Co-authored-by: J. Nick Koston <nick@home-assistant.io>
Co-authored-by: J. Nick Koston <nick@koston.org>
Co-authored-by: pre-commit-ci-lite[bot] <117423508+pre-commit-ci-lite[bot]@users.noreply.github.com>
Co-authored-by: J. Nick Koston <nick@home-assistant.io>
Co-authored-by: pre-commit-ci-lite[bot] <117423508+pre-commit-ci-lite[bot]@users.noreply.github.com>
Co-authored-by: J. Nick Koston <nick@home-assistant.io>
Signed-off-by: Petr Helebrant <petr@helebrant.cz>
Co-authored-by: pre-commit-ci-lite[bot] <117423508+pre-commit-ci-lite[bot]@users.noreply.github.com>
Co-authored-by: J. Nick Koston <nick@koston.org>
Co-authored-by: J. Nick Koston <nick@home-assistant.io>
Co-authored-by: pre-commit-ci-lite[bot] <117423508+pre-commit-ci-lite[bot]@users.noreply.github.com>
Co-authored-by: J. Nick Koston <nick@koston.org>
Co-authored-by: J. Nick Koston <nick@home-assistant.io>
Co-authored-by: pre-commit-ci-lite[bot] <117423508+pre-commit-ci-lite[bot]@users.noreply.github.com>
Co-authored-by: J. Nick Koston <nick@home-assistant.io>
Co-authored-by: J. Nick Koston <nick@koston.org>
A table-build failure that keeps the link for legacy nodes now charges the
backoff, so an unwalkable database stops retrying at advertisement cadence.
The write action re-resolves from the current table only. The neutral engine
drops a discovery completion that raced a teardown. Slot charging and the
cap-check skip read one mapping, so a platform cannot be skipped uncharged.
The registry entry now carries its materializer gate, so rp2 builds stop
receiving a Bluedroid define. Also: parent before the capacity check in
register_gatt_node (neutral parity) and BluedroidServiceTable is
non-copyable (owning raw pointer).
USE_BLUEDROID_GATT_SERVICE_TABLE compiles the Bluedroid materializer and
nothing else; the old name read like a cross-backend consumer capability
(rp2 always materializes and ignores it).
Each backend arm now has a codegen-emitted USE_BLE_GATT_BACKEND_<NAME>
define, making the alias ladder order-independent and letting the proxy
capability constants describe the backend instead of the platform (a second
esp32 backend carries its own facts). The legacy raw-gattc engine is
selected by USE_BLE_CLIENT_LEGACY_ENGINE instead of platform identity, so
esp32's future move to the neutral engine is one line per language. The
address-scoped maintenance calls key on the stack define they wrap.
Connect-timing ownership (the backend integrates with its platform's
scan/connect arbitration), the asynchronous-completion rule for accepted
ops, and the reserved error ranges with the wire-frozen values - all learned
the hard way by the first out-of-tree backend.
has_legacy_nodes_() lives next to the invariant it derives from; both
bailout outcomes reach one promotion loop; the carve assert relaxes to the
descending chain the comment claims; the notify op string names the IDF
call like its siblings.
The static_assert enforces what the carve comment claims; the node header
now states that aborting inside on_connected() can leave on_disconnect
unpaired; the passive-proxy test comment names the current gate; the
pending-regs comment lands on the right member (lost to an aborted batch).
A failed search still poisons the whole discovery, but an allocation or walk
failure is table-specific: on a mixed client the legacy nodes keep the link
and only the gatt nodes skip that connection.
The write action's own tag left Automation::TAG with zero readers;
automation.cpp existed only to define it. Also notes the unpaired legacy
disconnect trigger on the failed-discovery path and re-anchors the
gatt_nodes_ comment.
One ConnectBackoff shared by both engines replaces the forked hold-off
fields and constants (the duration is derived from the failure count). The
pending-registration scan lives once, sized by its own bound instead of the
node counter. The failure-suppressing SEARCH_CMPL arm moves to the call site
so the dispatcher stays void, notify-state logging is one helper on the
neutral wording, and the op-check helper takes the backend's spelling.
Failed notify enables, reads, writes and unsubscribes now leave the same
breadcrumbs as the neutral engine; the shared-completion guarantee moves to
the node contract header.
set_enabled(true) resets the hold-off (neutral-engine parity), and every
failing build() return now resets the counts so a failed build can never
present a non-empty view.
A discovery the bridge classifies as failed now suppresses the legacy
SEARCH_CMPL fan-out, so user automations never observe the doomed link as a
connection (neutral-engine parity, including the mid-fan-out node abort).
Also documents that same-handle subscriptions from mixed node kinds are
unsupported during the migration window.
A peer that reliably fails discovery (walk error, allocation failure) would
otherwise reconnect on its next sighting forever, holding a controller slot
each cycle - the neutral engine already backs off on this path.
Neutral fan-out runs before the legacy nodes so on_connect triggers observe
resolved gatt nodes (the neutral engine's order). Pending notify
registrations move to a fixed array (no post-setup heap). Gatt nodes also
join nodes_, collapsing the twin state loops. One define now means 'the
neutral node surface is compiled in' on both engines. The service count
moves into BluedroidServiceTable, free() resets it, bridge ops log failures,
the write action carries its own tag, and the automation headers drop
includes that left with the write actions.
Platforms whose BLE stack owns a connection budget (esp32, rp2) report an
overcommit exactly once through that stack; the neutral cap check covers
future backend platforms without one. The write action logs unmatched
completions while a chain is parked, and the schema-dump gap is documented
as deliberate.
The raw-gattc and neutral twins collapse into ble_write_action.h, registered
through register_gatt_node on both engines. On esp32 this is the in-tree
exerciser of the node bridge; builds without a ble_write action compile no
bridge code.
BLEClientFeatures (gatt_node / raw_gattc / security) with a per-platform
provider set, capability-named validation errors, NODE_BLE_CLIENT_SCHEMA +
register_gatt_node for migrated components, and the security actions gated on
the feature instead of the platform. esp32 auto-loads bluetooth_connection
for the shared materializer (compiles empty without a neutral node).
Nodes registered through register_gatt_node() get the neutral surface:
service table at SEARCH_CMPL (honoring the search status the base ignores),
translated completion/notify fan-outs, pairing results, and teardown via
on_disconnect_complete. Bridge-initiated notify registrations are intercepted
before the base's automatic CCCD write - the CCCD is the node's job on the
neutral contract. Compiled only under USE_BLE_CLIENT_GATT_NODES; builds
without a neutral node are unchanged.
BluedroidServiceTable carries the two-pass build verbatim in its own TU; the
backend embeds one instance. ble_client's esp32 engine becomes the second
consumer next.
The neutral callbacks (plus a new on_pairing_result hook) live unconditionally
in ble_client_node.h; the raw esp32 surface stays beneath them for unmigrated
nodes. Adds find_descriptor beside find_cccd and pair()/unpair() on the
neutral client. Drops the node 'client' pointer and address, which nothing
reads.
Every teardown path routes through release_services(), so the
materialized table cannot outlive its link - said at free_service_table_
so the next reviewer need not re-derive it.
- The rejection test registers the hub and matches the extra-keys error,
so the missing-tracker error can no longer satisfy it vacuously
- register_ble_node logs and drops loudly at capacity: a push_back past
a StaticVector's bound is a silent no-op, and an undersized slot count
must show at boot, not as an unresolvable node
- Every walker failure names the failing call and status; the descriptor
cap gets its own message so a >64-descriptor characteristic is
diagnosable instead of a permanent silent backoff loop
The schema split's accept side was tested; the reject side was not - a
key leaking from the legacy arm into the neutral schema would have
passed silently.
- The empty-table bailout ran after state_ went CONNECTED, so its
teardown's terminal report read as a completed connection and fired
on_disconnect with no preceding on_connect - the exact trigger
contract violation the sibling error branch avoids. The check now
runs first and the teardown resolves through connect_failed; a
node-less client still reaches CONNECTED through the single
assignment after the guard
- The esp32 arm claims its slot through consume_gatt_slot like every
other claimant (behavior-identical: it forwards to the same esp32
validator), so the ledger's one-spelling contract holds
- A zero-service table on a client with nodes is treated as a failed
discovery (warn, release, charge the backoff, disconnect): a real GATT
peer always exposes at least GAP/GATT, so an empty table means the
materialization failed and the connection must retry instead of
sitting inert behind a successful on_connect
- The fan-out abort guard also tests cancel_requested_, covering the
normal async teardown a node starts from on_connected (the state-only
check caught just the synchronous refusal)
- nodes_ becomes a StaticVector sized by ESPHOME_BLE_CLIENT_MAX_NODES:
the client requests a baseline slot, each neutral write action (which
registers itself in its constructor) requests one, and the define
rides defines.h for analysis - no realloc machinery on the rp2 target
- The Bluedroid empty-table return logs its quiet cases; the
service_table=False stub notes the misconfiguration it implies
A gate-refused send to a peer that is not api_connection_ (the
subscribe-time path during a handover) must not arm a retry that
loop() would aim at a different connection.
Overkill for its window: the address latch, drain, reuse guard and
session hygiene bought coverage for a case the client's own timeouts
and the retried connections-free state already heal. The notification
goes out when it can; the connections-free retry (the hardware-verified
fix) stays.
- Both retry latches clear on subscribe/unsubscribe: a new subscriber
must not receive a connected=false for an address from the previous
session (it resyncs through its own subscribe requests)
- The app-register failure path states its BLEClientBase parity so the
no-retry behavior reads as deliberate
The stack-down settle skipped release_services() locally, but its
report routes into the wrapper's reset which calls it anyway - the
warn survived. Gating the clean on an active stack inside
release_services() covers every path, and the stack-down branch goes
back to the plain call.
- The stack-down settle resets the stream latches directly instead of
calling release_services(): the dying stack invalidates its own cache,
and the newly checked cache_clean would warn on every OTA or
ble.disable with a live connection
- static_asserts pin the exactly-sized state bitfields so a future
enumerator truncates loudly at compile time
The connections-free retry left its sibling asymmetric: on the same
full buffer the slot could report free while the device still read as
connected, with nothing resending the connected=false. A one-deep latch
(address + error) drains at the same 100 ms cadence and re-latches on a
repeat failure; a re-reservation of the address clears it, since the
client re-requesting the connect has already acted on the disconnect
and a late resend would shadow the new connection. A lost
connected=true stays client-timeout territory.
An unknown or misspelled platform name yielded an empty set - the file
would be filtered out of every build (link failure at the end of a long
compile) and the subset guard passed vacuously.
The one IDF call in the backend that skipped check_and_log_error_. A
failed clean leaves a stale database the next connection could serve as
authoritative, with nothing in the log to point at it. (The call fails
only at dispatch - stack down - not for an empty cache, so this cannot
warn-spam routine teardowns.)
The collapse of the CONNECTING branch dropped a real race fix: HA
re-requesting a connect while a scheduled teardown was still pending
used to cancel the teardown and let the in-flight open complete;
without it the request was ignored and HA paid a teardown plus a fresh
connect during exactly the reconnect churn this path sees most.
cancel_gatt_disconnect() joins the contract: true only for a scheduled
teardown that has not started closing (Bluedroid clears the latched
want_disconnect_); rp2 and the stubs return false since their teardowns
start inside gatt_disconnect(). The wrapper's cancel_teardown() returns
its state to CONNECTING and the proxy handler carries the dev branch
verbatim.
- Both esp_ble_gap_set_prefer_conn_params calls route through
check_and_log_error_ like the class this PR replaces did; a rejected
preference leaves the link on the controller default interval, which
is exactly the WiFi-coex failure the shared constants exist to avoid,
so it must be visible in logs
- The connections-free retry moves below the 100 ms gate so a full TCP
buffer is retried at the loop cadence instead of every iteration
- The late-OPEN_EVT reclaim was the one unchecked IDF call in the
backend; a failed close there leaks a live link nothing tracks
- The mid-stream services_released_ park now leaves a trace like its
API-lost sibling, so a stuck GetServices is explicable from logs
Round 2 found no code issues - the round-1 mechanisms verify clean on
disk (every IDLE transition through the one door, the teardown-guard
trio minimal-complete per event, the error latch airtight, the reset
lists disjoint by construction). What remained was comment drift:
disconnect_pending() spelling, the OPEN_EVT and cached-MTU comments,
the contract header re-wrap, the two claim-a-slot docstrings
disambiguated, and ragged wraps from earlier text excisions.
- set_idle_() is the single door back to IDLE (bootstrap, open-fail and
the DISCOVERED park went through bare set_state), so the per-attempt
reset list holds only per-attempt latches
- CFG_MTU joins OPEN_EVT and SEARCH_CMPL in suppressing reports while a
teardown owns the link - one spelling of the guard across all three
events, and the wrapper's race arm becomes defense instead of the only
cover
- The connections-free retry drain compiles on every proxy build (the
advertisement-only arm sends the message too; the latch already did)
- latch_pending_error_ makes first-cause-wins the mechanism at all three
latch sites; the dead freed-slot refused branch and its retired
vocabulary go; the initiate_connection/start_connect_ pair collapses
- UNSET_CONN_ID hoisted next to the field it initializes; count-status
shadow renamed; stale busy-error rationale replaced with the real one
(a repeat call would re-arm the teardown timer)
- Fixture states the batch-grouping caveat like its rp2 sibling; the
get_service_table stub carries a greppable direct-consumer warning
Log lines ride the same API connection; a debug line at the exact
moment the TCP buffer is full adds traffic when it can least afford it
(the api layer's own buffer-full log is verbose for the same reason).
Field bug on a Pico W: the slot freed device-side but the API client
kept free=0 with the address still allocated. send_message drops the
response when the TCP buffer is full (a boot storm makes that likely
right when HA reconnects), and nothing ever resent it - the client's
slot state is verbatim the last response received, so it stayed stale
until reboot. The cached response is current by construction, so a
pending bit retried from loop() is an idempotent resync; the
subscribe-time send heals through the same bit.
- The esp32 arm's ESPHOME_BLE_GATT_CLIENT_COUNT shadowed the unanchored
regex; the pin now searches the USE_RP2 platform block
- The contract states what the wrapper relies on: nonzero from
gatt_disconnect means nothing to tear down, an accepted teardown
always reaches a terminal report
A client with only connect/disconnect automations paid the build/free
cycle on every reconnect for nothing - post-setup heap churn on the
Bluedroid direct-consumer path. release_services() stays unconditional.
notify_characteristic() had no completion path to the node that asked;
the hook lands now while the surface has no external users, with the
client fanning out after its breadcrumb.
The contract test's own #define only covers its translation unit;
ble_gatt_client.cpp needs the define from the build to link the
find_characteristic/find_cccd definitions the lookup tests call.
The Bluedroid on-demand materializer, the neutral lookup helpers' host
tests, and the USE_BLE_GATT_SERVICE_TABLE define move here, next to
their first consumer (the neutral engine's table resolution and the
service_table codegen flag). One behavior fix rides along: hitting
MAX_DESCRIPTORS_PER_CHARACTERISTIC now fails the walk like every other
inconsistency instead of truncating the table silently.
discover_services() already returned the contract's not-connected value;
the read/write/notify/pair ops fell through to an arbitrary stack error
instead. Dead for the proxy (the wrapper gates on connected()), live for
a direct consumer racing an op against a teardown.
Nothing in this PR emits USE_BLE_GATT_SERVICE_TABLE, so the Bluedroid
materializer, the neutral find_service/find_characteristic/find_cccd
helpers, their host tests, and the defines.h entry were scaffolding with
no caller here. They move to the PR that introduces their first
consumer. The contract op stays: get_service_table() keeps its {} stub
(the proxy streams in place).
- on_notify_state and on_pairing_result are handled on the neutral
client: a failed registration or pairing logs instead of vanishing
into the default no-op on a frozen surface
- A node tearing the link down during the connect fan-out leaves a
warning, since on_disconnect then fires with no preceding on_connect
- An INVALID_OFFSET/NOT_FOUND before the count from the same cache is a
contradiction, not an end-of-range: the walker fails the build and the
streamer aborts with the real cause instead of silently truncating
(the walker's descriptor loop keeps its terminator - the 64 cap is its
only bound)
- unconditional_disconnect_'s unset-conn-id path drives the slot to a
terminal reported state instead of leaning on the scheduled-teardown
timer
- find_characteristic/find_cccd log a corrupt range before returning the
not-found sentinel (moved out of line - no ESP_LOG in headers)
- connect()/disconnect() log a rejected user action like the legacy
engine instead of silently ignoring it
- The service_table docstring states the flag is Bluedroid-only forward
scaffolding and that rp2's proxy hub must keep its materializer
regardless (it streams through get_service_table())
- Review-round comments tightened to repo style
- handle_search_cmpl_ takes the event status: a failed discovery no
longer reads as a clean zero-service success (the empty cached
database satisfies the count calls), and the table invalidation plus
param step-down still run on the failure path
- A late successful OPEN_EVT on a slot the teardown net already gave up
is closed instead of leaking a live controller link with no conn id
- The counting pass logs its failure like the two paths below it
- static_assert pins the wrapper's compile-time streamer detection; a
signature drift would silently fall back to a table proxy builds
compile without
- find_characteristic/find_cccd range math widened to 32-bit (correct by
type; the wrapped-end case was already loop-safe)
- The fan-out early return releases the borrowed table itself instead of
relying on the backend's own teardown release
- A registered non-esp32 backend platform without a slot cap now fails
validation loudly instead of failing open (the test pin remains)
- new_gatt_backend's docstring says the service_table flag is forward
scaffolding for the first esp32 direct consumer, not load-bearing on
any current build
Both backends report the no-response completion (Bluedroid at
WRITE_CHAR_EVT, rp2 synthesized or via the can-send path); the neutral
write action already relies on it.
The measured layout showed 56+48 = 104 B/slot vs dev's 96. Two changes
that only pay together (the wrapper is 8-aligned, so nothing under a
full 8 helps):
- The tail packs into 2 bytes of bitfields; remote_addr_type_ becomes a
start_connect_ parameter (written and read on adjacent lines only)
- The wrapper's duplicate 10 s teardown timer is gone: the backend owns
the whole safety window. Its timer now also arms for a teardown
scheduled during CONNECTING (the one case the wrapper covered alone),
a late OPEN_EVT on a given-up slot closes the link instead of
resurrecting it, and the wrapper's transient-refusal branch collapses
because both backends return nonzero only when already idle
48 + 48 = 96 B/slot, the split at dev parity.
The measured layout showed 56+48 = 104 B/slot vs dev's 96. Two changes
that only pay together (the wrapper is 8-aligned, so nothing under a
full 8 helps):
- The tail packs into 2 bytes of bitfields; remote_addr_type_ becomes a
start_connect_ parameter (written and read on adjacent lines only)
- The wrapper's duplicate 10 s teardown timer is gone: the backend owns
the whole safety window. Its timer now also arms for a teardown
scheduled during CONNECTING (the one case the wrapper covered alone),
a late OPEN_EVT on a given-up slot closes the link instead of
resurrecting it, and the wrapper's transient-refusal branch collapses
because both backends return nonzero only when already idle
48 + 48 = 96 B/slot, the split at dev parity.
- SEARCH_CMPL honors the event's own status: a failed discovery leaves
an empty cached database that the count calls read as a clean zero,
which would have become an authoritative empty service list
- The loop stays enabled while a link exists and settles only at IDLE -
the stack-down recovery needs a tick to run in ESTABLISHED, which the
settle-on-established optimization was silently blocking
- The pre-started search uses the OPEN_EVT conn id (replaced-class
parity)
- read_descriptor and notify_characteristic passthroughs join the frozen
surface so the first migrated node can subscribe without extending the
client; the comment now says which ops have callers today
- The choke-point test also validates through the public
BLE_CLIENT_SCHEMA, so removing the cv.All wiring fails a test
- DOMAIN no longer sits between the BTstack comment and the constant it
explains
- A refused MTU request no longer wedges the connection: OPEN_EVT
reports with the default MTU so the consumer proceeds (previously no
CFG_MTU_EVT meant no connected report and a slot that never freed)
- Disabling the BLE stack settles a live link with a connected=false
report before forgetting it, so the consumer frees its slot instead of
holding a phantom connection
- A refused security response answers the pairing request with the
failure instead of hanging it
- The service-table walk mismatch logs its reason before discarding
- The shared streamer's two bounds-check aborts use abort_service_stream
with an ATT Unlikely Error cause (new shared GATT_ERR_UNLIKELY)
- A refused MTU request no longer wedges the connection: OPEN_EVT
reports with the default MTU so the consumer proceeds (previously no
CFG_MTU_EVT meant no connected report and a slot that never freed)
- Disabling the BLE stack settles a live link with a connected=false
report before forgetting it, so the consumer frees its slot instead of
holding a phantom connection
- A refused security response answers the pairing request with the
failure instead of hanging it
- The service-table walk mismatch logs its reason before discarding
- The shared streamer's two bounds-check aborts use abort_service_stream
with an ATT Unlikely Error cause (new shared GATT_ERR_UNLIKELY)
The three coordinated flags become one SearchState (NONE / PRESTARTED /
PRESTART_DONE / CLAIMED / REPORT_PENDING): illegal combinations are
unrepresentable, delivery and the two reset sites collapse to single
assignments, and the requested-meets-done subtlety becomes a named
state. A second discover_services() while one is in flight now returns 0
instead of issuing a duplicate search (one completion is already owed).
Same RAM: the enum sits in a 4-bit bitfield in the same flags byte.
- The connected fan-out stops if a node tears the link down mid-loop
(latent until a backend settles a teardown synchronously)
- A synchronously refused disconnect logs its own warning - a backend/
client state divergence is no longer the most quietly logged outcome
- An action-initiated connect before any sighting warns that the public
address type is assumed (an opaque repeated failure becomes
self-diagnosing); the connect() doc no longer mentions a config key
that does not exist
- The explicit-connections ledger test asserts the exact charged count;
the host-test override comment names the current gate macro
The latches were cleared only in set_idle_(), but the stack-down branch
in loop() reaches IDLE through set_state() without it, so after a
ble.disable/enable cycle the next connection's discover_services() would
complete immediately from the previous connection's latched result and
HA would cache an authoritative empty service list. tracker_connect_()
now resets the latch alongside the two per-attempt flags it already
clears, and delivery consumes the whole latch so a re-discovery on a
live link issues a real search instead of re-reporting the first
result.
- Explicit `connections:` entries now charge the slot ledger like the
generated ones (dev let them evade the controller budget); pinned by a
test
- set_enabled(true) clears the connect backoff - a re-enable is an
explicit try-again
- The neutral engine's source filter uses the shared
frameworks_for_platforms helper; ledger tests use the conftest fixture
- abort_service_stream() on the wrapper (which owns the latch discipline)
replaces five identical field-poke sequences in the streamer
- The pre-start reuses check_and_log_error_ like the serialized path
- discover_services() sets the requested flag where it sticks instead of
set-then-rollback; deliver_pending_search_() returns void (no caller
reads it); loop()'s IDLE arm folds into the settle arm
- SEARCH_CMPL during a teardown skips the param/count work whose result
is never delivered
- frameworks_for_platforms() in config_helpers derives framework sets
from platform lists; the hub.cpp filter entry is now a named map pinned
by a test against the proxy's platform list
The split serialized service discovery behind the MTU exchange: the
backend reported connected only at CFG_MTU_EVT and the wrapper started
discovery on that report, costing one ATT round trip per uncached
connection. The backend now pre-starts the search at OPEN_EVT (it knows
the connection type) and completes the consumer's discover_services()
from it: SEARCH_CMPL latches silently until requested, the flush after
the connected report delivers in the same event drain, and a refused
pre-start falls back to the serialized path. Net RAM cost is zero (the
flags pack into one bitfield byte plus a status byte, replacing the two
existing bools).
Also from review: SEARCH_CMPL no longer clobbers a teardown in progress
with ESTABLISHED, the five streamer abort sites latch pending_error_ so
HA sees the real Bluedroid status instead of a generic HCI reason, and a
completed re-discovery frees a materialized service table before the
counts its offsets derive from move.
The hub wrapper serves the proxy's API surface, but its gate keyed on the
backend define alone while the backend guarded its own proxy pieces on
USE_BLUETOOTH_PROXY. Narrow BLUETOOTH_CONNECTION_HAS_GATT to require both
so a future backend-only consumer build agrees with the backend's guards.
No change to any build this PR can produce (only the proxy emits
USE_BLE_GATT_CLIENT here).
- Skip the hub-platform GATT slot cap in testing mode (mirrors
esp32_ble.validate_connection_slots) so grouped builds validate
- Derive the neutral engine's source filter from the backend registry so
a new platform cannot validate and then fail at link
- Log the real code on refused service discovery and cancelled connects;
read completions get the same breadcrumb writes have
- Comment accuracy: the dumper arm is the legacy shape, not a superset
- Ledger tests: real-validator end-to-end path and the testing-mode skip
- Fold the one-line event-handler forwarders, the state accessor renames,
and the report helper into direct calls; drop the dead forward
declaration and the tracker_client() accessor (the backend is the
tracker client)
- Hoist DEFAULT_ATT_MTU next to the other shared GATT constants and use
it in all three engines
- Emit BLUETOOTH_PROXY_MAX_CONNECTIONS once, in _connections_to_code
- Drop the unused esp_gatt_common_api.h include and DOMAIN constant
- Comment accuracy: stale shim/esp32-parity wording, registry roles,
request_gatt_client() caller
- Drop unused includes (ble_client_state.h, <utility>) and the unused
notify/indicate property constants (no CCCD path yet)
- Note the frozen node-facing surface on the uncalled backend forwarders
- Keep the legacy-only 'name' key out of the neutral schema
- Move the slot-ledger test to tests/component_tests/bluetooth_connection
- Comment accuracy fixes (choke-point wording, wrapped lines, backoff note)
description:Test an ESPHome change on real boards over USB. Use this whenever a task asks to test, verify, reproduce, bench, A/B or soak a pull request, a branch or a crash on hardware, or mentions flashing, serial logs, esphome upload, a captive portal or fallback AP test, or results to post on a PR. Covers finding the serial ports, building from the right checkout, flashing each chip family (including the LibreTiny kits that stock tools cannot flash), bounded log capture, proven test flows and how to report results.
allowed-tools:Read, Write, Edit, Bash, Glob, Grep
---
# Hardware testing
The goal of a hardware test is evidence a reviewer can trust: which build ran on which board, what was
driven, what the logs show, and the same flow on a `dev` build when a quirk could be pre-existing.
`$SKILL` below is this directory (`.agents/skills/hardware-test`).
## Ground rules
1.**Serial ports cannot be shared.** One bounded capture per port, stopped before any flash on that
port, never an open-ended `esphome logs`. Stop a capture with `pkill -f "[s]erial_capture.py <port> "`,
naming the port so a capture on another board survives; the bracket keeps the pattern from matching
the shell that runs it. When more than one agent session shares the bench, run a copy of the script
under a session-specific name (`cap<topic>.py`) so another session's `pkill` cannot take your capture
down. Never run `esphome logs` in a `while true` loop: the loop respawns it and holds the port.
2.**Credentials stay local.** Put a `secrets.yaml` next to the scratch configs and reference it with
`!secret`. A copy of a device's own config keeps using the secrets file next to that config: two
checkouts can hold different networks, and the wrong one leaves the device unreachable after the OTA. Never paste a config with credentials into a PR, never commit scratch configs or logs.
3.**Build from the tree under test.**`python -m esphome` puts the current directory first on
`sys.path`, so run it with the checkout or worktree as the working directory and pass the yaml by
its absolute path. A venv `esphome` entry point may import an installed snapshot instead of the
checkout. Confirm with `python -c 'import esphome; print(esphome.__file__)'`, and for an A/B grep
the build's `src/` for the change before trusting any number. Tool calls sent together share one
shell, so a `cd` in one can land another's compile in the wrong tree: pass `PYTHONPATH=<worktree>` on
every compile and upload command and print `esphome.__file__` into the compile log.
4.**PR build and baseline build get different `name:` values.** Builds share
`.esphome/build/<name>`, so the same name means `esphome upload` can flash the wrong binary. Saved
Wi-Fi credentials are keyed by the config hash when the config has `networks:`, so a test that must
start unprovisioned gets a fresh name every run. A config with `ap:` only and no `networks:` uses one
fixed key shared by every such config on the board, so a new name does not help there: wipe once at
boot with an `on_boot` lambda that resets the preferences and saves a marker key
(`templates/dut-ap-only-wipe-once-bk72xx.yaml`).
5.**Flash and RAM deltas are measured against the PR's merge base** (`git merge-base origin/dev HEAD`
checked out in a throwaway worktree), with the baseline commit named in the table. When a delta
looks odd, diff symbol sizes (`nm -S -C --size-sort` from the matching toolchain) before quoting it.
6.**Prove a quirk is pre-existing before blaming the PR:** run the identical flow on a `dev` build in
the same session and say so in the results.
7.**Pipe safely.** Set `PYTHONUNBUFFERED=1` on any `esphome run` or `esphome logs` whose output is
piped, or lines land minutes late. Bound every long command with `timeout` (coreutils) or
`perl -e 'alarm shift; exec @ARGV' <seconds> <command>` on macOS without it.
## Workflow
1.**Find the boards:**`python $SKILL/scripts/ports.py` (any Python with pyserial, for example the
ESPHome venv). Port names follow the USB hub port and change on every replug, so never reuse a name
from an earlier session. CH340 adapters (D1 mini, BW15 kit, BK7238, CB3S) all look the same; tell
them apart with a 10 s capture using `serial_capture.py --reset` (ESP8266 prints its ROM banner at
74880 baud, LibreTiny boards print `[LT]` and ESPHome lines, BK7238 prints its bootloader version).
`ports.py --wait 2e8a` blocks until a Pico re-enumerates.
2.**Make a scratch directory** outside the checkout, with `secrets.yaml` and the templates you need.
One yaml per board per variant, file name equal to `name:`.
Keep the RAM and flash lines at the end. Two ESP-IDF or two LibreTiny builds at the same time race
over tool caches; run them one at a time.
4.**Flash** per chip family (table below). Stop the capture on that port first.
5.**Capture** with `serial_capture.py PORT SECONDS OUT` in the background, then poll the file with
`grep` in an `until ... sleep 2` loop with a cap. USB CDC boards (Pico, nRF52, native USB ESP32)
drop the port on reboot; the script reopens it until its deadline.
6.**Drive** the test: a second board running a client config, `curl`, an aioesphomeapi script that
presses a template button and subscribes to logs, MQTT publishes, or `interval:` and `on_boot:`
automations in the config itself.
7.**Analyze** with `grep -a -c` counts over the capture and keep an `-events.txt` extract of the lines
that matter. Decode crashes with the platform's `addr2line` (see `reference/boards.md`).
8.**Report** on the PR as a table: build SHA, board and framework, what was driven, counts, result,
plus the `dev` control row and the exact flow in a `<details>` block. No credentials, no port names,
no IP addresses.
## Flashing by chip family
| Board | Flash | Log |
|---|---|---|
| ESP8266 D1 mini (CH340) | `python -m esphome upload <yaml> --device <port>` | `serial_capture.py` at 115200; `esphome logs` on some CH340s emits garbage after the RTS reset. A failed esptool write can wedge the adapter until the board is replugged. `esptool erase_flash` before a test that needs an unprovisioned device: the preference sector survives a reflash |
| ESP32 devkit (CP2102 or CH340) | `esphome upload --device <port>`, ESP-IDF or Arduino | `serial_capture.py`. Default Wi-Fi light sleep delays API driven actions by seconds; judge timing from device logs. `power_save_mode: none` for OTA overlap tests |
| ESP32-S3, C3, C6, H2 native USB | `esphome upload --device /dev/cu.usbmodem*` | `logger: hardware_uart: USB_SERIAL_JTAG`. The USB console can wedge when the app takes over the console (0 bytes, esptool `No serial data received`); replug, or watch through `esphome logs --device <name>.local` |
| Pico W / Pico 2 W | `esphome upload --device <port>` (1200 baud touch); first flash ever via BOOTSEL and `--device /Volumes/RPI-RP2` or `RP2350` | port re-enumerates after upload: `ports.py --wait 2e8a` then capture |
| BW15 kit (RTL8720CF, rtl87xx) | `~/.platformio/penv/.libretiny/bin/python $SKILL/scripts/flash_rtl87xx_kit.py flash write -d <port> <build>/.pioenvs/<name>/*.uf2`. Plain `esphome upload` and plain ltchiptool time out because opening the port asserts DTR/RTS and holds the chip in reset | `serial_capture.py --released`, never `esphome logs` |
| BK7238 / CB3S (bk72xx, RTS on CEN) | `~/.platformio/penv/.libretiny/bin/python $SKILL/scripts/flash_bk72xx.py flash write -d <port> <uf2>` (about 75 s) | logs are on UART2 by default; `logger: hardware_uart: UART1` puts them on the flashing adapter. `serial_capture.py --reset` gives a chip-enable reset; a plain open may or may not reset it, check for the bootloader banner |
| LN882H | ltchiptool forces 1 Mbaud for writes; weak adapters need `LN882H_YM_BAUDRATE` patched to 115200 in the penv's `ln882htool.py`; short BOOT to GND while the tool connects | OTA after the first flash |
| nRF52840 Dongle (PCA10059) | Nordic `nrfutil nrf5sdk-tools dfu usb-serial` on the bootloader port, or `dfu: true` in the config so the 1200 baud touch pin-resets into the bootloader; adafruit-nrfutil is refused | app has two CDC ports, ESPHome log on the second |
| Adafruit-style nRF52 (UF2 bootloader) | `UF2BOOT` volume or adafruit-nrfutil; `dfu: true` works through GPREGRET | single CDC port |
| Any reachable device | `esphome upload <yaml> --device <ip or name.local>`; `--file <bin>` for a prebuilt image | `esphome logs --device <ip>` bounded with `timeout` |
ESP-IDF OTA image: `.esphome/build/<name>/build/firmware.ota.bin`. LibreTiny OTA port is 8892.
## Scripts in `$SKILL/scripts`
-`ports.py`: list ports with VID:PID, product, serial and a board guess; `--wait VID[:PID]`,
`--product NAME`, `--serial SN` (one specific board, whatever path it comes back under) and
`--serial-of PORT` (the serial number behind a path).
-`serial_capture.py PORT SECONDS OUT [--baud] [--released] [--reset] [--append] [--ms]`: bounded timestamped
capture (`--ms` adds milliseconds for timing work), ANSI stripped, `LOG_SECRET` text written as `<secret>`, reopens a vanished port, exits at the
deadline. OUT is truncated unless `--append` is given, so counts over it cover one run.
-`flash_rtl87xx_kit.py`: ltchiptool wrapper for the BW15 kit whose CH340 drives CEN (RTS) and the boot
strap (DTR). `flash read` needs the family `realtek-ambz2`; `flash write` does not.
-`flash_bk72xx.py`: ltchiptool wrapper for BK72xx boards whose RTS drives CEN. It misses the link now
and then; retry up to three times.
-`api_button.py HOST KEY list|press NAME [...]`: list a device's template buttons or press them over the API; the
capture decoder), `api_outgoing/` (wake cycle and Wi-Fi drop timing readers), `ld2450/` (zone number driver), `dfrobot_sen0395/` (API driver and case runner), `homeassistant_light/` (direct bulb driver, held-report case driver), `usb_audio/` (URL player), `speaker_media_player/` (API player driver, pipeline task case runner), `web_server/` (SSE client, request mix, SSE and HTTPD case runners), `mqtt/` (broker config with WebSocket listeners, test CA, transport case runner).
## Templates in `$SKILL/templates`
-`dut-fallback-ap.yaml` (package): a device that never finds its network, falls back to its AP and
serves the captive portal. `example-dut.yaml` is the per-run file with the platform block.
-`client-wifisave.yaml` (package): joins that AP, posts N bogus `/wifisave` requests, then the real
credentials from `!secret`. `example-client.yaml` is the per-run file. `bogus_saves: "1"` gives a
plain provisioning check.
- Per-topic device configs, each described in `reference/recipes/<component>.md`: `online-image-dut.yaml`,
## Restoring globals with storage: rtc and flash (#13954)
`templates/globals-storage-dut.yaml` is the shared package: five restoring globals (rtc int, flash int, default int,
rtc `std::string`, rtc `uint8_t[2]`), bumped and logged with the reset reason on every boot, saved and synced after
2 s, odd boots `App.safe_reboot()`. Platform files add the board (`packages: dut: !include globals-storage-dut.yaml`), the `debug` reset-reason text sensor and, on the
ESP32, `deep_sleep: run_duration: 12s, sleep_duration: 8s` (even boots sleep). No network needed: flash by USB,
`serial_capture.py PORT 110 OUT --reset` gives the reset pulse plus a run of soft restarts and sleeps; the ESP32's
EN pulse is a `POWERON_RESET` that empties RTC memory, the D1 mini's RST pulse (`rst cause:2`) keeps it, so the
ESP8266 needs a real power cut by hand; arm long `--released` captures on both ports first (the capture
reopens the port when it comes back, the very first boot lines are still lost, read the next boot's restored
values). Unchanged-build check: the config with every `storage:` removed, built on the PR and on the dev merge
commit under the PR's own commits (not the merge base, which is older dev), `cmp -l` plus an `nm -S` symbol diff;
only the template names differ. Results: https://github.com/esphome/esphome/pull/13954#issuecomment-6254794712
A report that arrives during a transition is held and applied when the transition ends, unless a call came after
it. `scripts/homeassistant_light/heldcase.py DEV_IP KEY BULB_IP same|many [BULB_KEY]` (a key of `-` is plaintext) keeps one API connection to the mirror
device and one to the bulb so the steps land inside a 3 s fade: press "A 50 fade 3s", 0.7 s later set the bulb to
90% directly (the report is held), 1.4 s in press the same fade again (or "A many calls", 16 calls in one loop
pass). Pass: the mirror and the bulb end at the second call's target. Only `same` reproduces the bug: a build
without the fix logs `Applying the report that arrived during the transition` and the bulb stays at 90%. `many`
ends at 100% on both builds because the last call's target differs from the held report, so it is a regression
check only, not a reproduction. Separate button presses are too slow (about 1.5 s each) to land inside the fade.
The mirrors are not API entities, so read their state from the "Log state" button's line. Results: https://github.com/esphome/esphome/pull/21144#issuecomment-6278257695
A local mosquitto with `allow_anonymous true`, `log_type all` and `log_dest file`: the broker log gives
each subscription's QoS, the device address (`New client connected from <ip> as <name>-<mac>`) and exact
subscribe counts. Kill and restart mosquitto to exercise `on_disconnect` (reason 0) and resubscription.
Keep `reboot_timeout: 0s` on `api:` and `mqtt:`. The ESP-IDF outbox swings free heap about 2 kB right
after bulk subscribe and unsubscribe; read the heap seconds later over several rounds before calling a
leak. Spawned subscriptions need about 2.5 s to reach the broker before the next publish.
`esphome logs --device MQTT` only uses the MQTT transport when the config has no `api:`. Driver pattern: a
paho script publishes and polls the serial capture file.
## WebSocket transports on ESP-IDF (#16056)
One build per transport from `templates/mqtt-transport-dut.yaml`, with `scripts/mqtt/mosquitto-ws.conf` as the
local broker (mosquitto 2.x from brew has `websockets` support; `log_type all` prints `New connection from <ip>:<port>
on port <listener>`, which proves which listener each client used) and `scripts/mqtt/make_test_ca.sh BROKER_IP` for
the TLS listener's certificate, both run in the scratch directory so keys, logs and outputs stay out of the checkout. `scripts/mqtt/cases.sh NAME KEY BROKER_HOST SUB_PORT CAPTURE [DEV_HOST] [restart]` subscribes
to `NAME/#`, sends a switch command and an `on_message` topic, presses the 3000 byte publish button over the API,
restarts the local broker when the last argument is `restart` (the PID in `mosquitto.pid`, written when the broker
was started with `mosquitto -c mosquitto-ws.conf & echo $! > mosquitto.pid` in the working directory next to the
certificates), checks its three ports answer again, and prints the broker's message counts with the device log. `mosquitto_sub` cannot speak
WebSocket, so it subscribes on the plain listener of the same broker. Cases: `tcp` on 1883, `ws` on 9001, `wss` on
9002 with `certificate_authority` (the certificate names an address, so `skip_cert_cn_check: true`), `wss` to
test.mosquitto.org on 8081 with no CA for the certificate bundle, and two negatives that must never connect: a
different CA (`mbedtls_ssl_handshake returned -0x2700`) and the self-signed server through the bundle
(`esp-x509-crt-bundle: Failed to verify certificate`, `-0x3000`). `CONFIG_MQTT_TRANSPORT_WEBSOCKET` is on by
default in the generated sdkconfig. The TLS session costs about 29 kB of internal heap while connected; the public
broker carries other devices' `esphome/discover` traffic, which is noise. Results:
show(){ tail -n +$(($1+1))"$LOG"| sed 's/\x1b\[[0-9;]*m//g'| grep -a 'dfrobot\|Detected\|\[W\]\|\[E\]\|sen\]'| grep -a -v 'JYBSS\|took a long time'| cut -c1-200;}
# cases.sh NAME KEY BROKER_HOST SUB_PORT CAPTURE [DEV_HOST] [restart] : subscribe to NAME/# on the broker, send a
# switch command and an on_message topic, press the 3000 byte publish button over the API (KEY, - for plaintext),
# optionally restart the local broker (its PID from mosquitto.pid in the working directory, written when it was started
# with `mosquitto -c mosquitto-ws.conf & echo $! > mosquitto.pid`), and show the device log. Outputs (sub-NAME.txt, mosquitto.out) go to the working directory, which should be the scratch directory.
[ -n "$KEY"]&&[ -r "$LOG"]||{echo"usage: cases.sh NAME KEY BROKER_HOST SUB_PORT CAPTURE [DEV_HOST] [restart] (CAPTURE must be a readable file)";exit 2;}
show(){ tail -n +$(($1+1))"$LOG"| sed 's/\x1b\[[0-9;]*m//g'| grep -a 'mqtt\|\[probe:\|cmd payload\|Relay\|\[W\]\|\[E\]\|Guru\|abort\|Backtrace\|ESPHome version'| grep -a -v 'took a long time\|Publish.*Counter\|sensor/counter'| cut -c1-170;}
for port in 18839001 9002;do nc -z -w 3"$B""$port"||{echo"broker port $port not listening after the restart";exit 1;};done
tail -n +$((n1+1))"$LOG"| grep -a -q "MQTT_EVENT_CONNECTED\|\[mqtt:[0-9]*\]: Connected"||{echo"the device did not reconnect after the broker restart";exit 1;}
else
echo"=== $N: no broker restart (pass 'restart' as the 7th argument for a local broker)"
fi
wait$sub_pid;sub_rc=$?
["$sub_rc"=124]||{echo"subscriber ended with status $sub_rc: $(cat "$WORK/sub-$N.err")";exit 1;}
# Copy next to the certificates in the scratch directory and start it there with
# `mosquitto -c mosquitto-ws.conf & echo $! > mosquitto.pid` (cases.sh restarts that PID): plain MQTT, MQTT over WebSocket, and MQTT over WebSocket with TLS (server.crt and
# server.key from make_test_ca.sh in the same directory). log_type all gives the listener port per connection.
f"{frames} frames in 10 s ({frames/10:.1f} fps), {packets} RTP packets, {gaps} seq gaps, frame size min {min(sizes)ifsizeselse0} max {max(sizes)ifsizeselse0}; sdp has m=video {'m=video'insdp.decode()}",
f"two clients: {fa} and {fb} frames in 4 s each, gaps {ga}/{gb}; {third}",
)
elifcase=="stall":
a=Client()
setup_play(a)
b=Client()
setup_play(b)
# stop reading on a for 4 s while b keeps playing
a.s.settimeout(None)
fb1,_,gb1,_=b.read_frames(4)
a.s.settimeout(1.0)
fa,_,ga,_=a.read_frames(4)
fb2,_,gb2,_=b.read_frames(3)
a.send("TEARDOWN")
a.read_response()
a.close()
b.send("TEARDOWN")
b.read_response()
b.close()
say(
case,
fb1>10andfb2>10andfa>5,
f"b kept {fb1} then {fb2} frames while a stalled 4 s; a recovered {fa} frames after resuming (gaps {ga}); b gaps {gb1}/{gb2}; device log must show stalled/dropping frame",
)
elifcase=="timeout":
c=Client(timeout=70)
setup_play(c)
c.s.settimeout(
2
)# a server that stops sending but never closes must not hang the run
t0=time.monotonic()
dead=False
try:
whiletime.monotonic()-t0<68:
try:
d=c.s.recv(65536)
exceptTimeoutError:
continue
ifnotd:
dead=True
break
exceptConnectionError:
dead=True
elapsed=time.monotonic()-t0
say(
case,
deadand50<=elapsed<=68,
f"connection closed by server after idle: {dead} at {elapsed:.0f} s (expect the 60 s session timeout when the client sends nothing)",
[ -r "$LOG"]||{echo"usage: cases.sh HOST KEY MEDIA_URL ANN_URL CAPTURE LABEL (CAPTURE must be a readable file)";exit 2;}
show(){ tail -n +$(($1+1))"$LOG"| sed 's/\x1b\[[0-9;]*m//g'| grep -a '\[tasks:\|speaker_media_player\|speaker_mixer\|\[W\]\|\[E\]\|Guru\|abort\|Backtrace'| grep -a -v 'took a long time'| cut -c1-180;}
[ -r "$LOG"]||{echo"usage: httpd_cases.sh HOST USER PASS LABEL CAPTURE (CAPTURE must be a readable file)";exit 2;}
probes(){ tail -n +$(($1+1))"$LOG"| sed 's/\x1b\[[0-9;]*m//g'| grep -a -o "\[[0-9:]*\]\[I\]\[probe:[0-9]*\]: .*httpd min free stack [0-9]*\|\[[0-9:]*\]\[[WE]\]\[[a-z_.]*:[0-9]*\].*\|Guru.*"| grep -a -v "took a long time\|does not fit\|\]\[httpd\]: /"| sed 's/\[I\]\[probe:[0-9]*\]: loopTask min free stack \([0-9]*\), httpd min free stack \([0-9]*\)/ loop \1 httpd \2/'| uniq -f1 | cut -c1-140;}
if["$c"=000]&&["$page"=200]&&{["$rc"=52]||["$rc"=56];};thenok=$((ok+1));elsebad=$((bad+1));echo"unknown path: got $c, curl status $rc, with the page at $page";fi
[ -r "$LOG"]||{echo"usage: sse_cases.sh HOST KEY LABEL CAPTURE (CAPTURE must be a readable file)";exit 2;}
show(){ tail -n +$(($1+1))"$LOG"| sed 's/\x1b\[[0-9;]*m//g'| grep -a '\[probe:\|\[W\]\|\[E\]\|web_server\|Guru\|abort\|Backtrace\|ESPHome version'| grep -a -v 'took a long time'| cut -c1-200;}
REBOOTS=$BANNERS;["$LOSSES" -gt "$REBOOTS"]&&REBOOTS=$LOSSES# one reboot can show as both, so take the larger
else
REBOOTS=$BANNERS;["$LOSSES" -gt 0]&&INVALID="${INVALID:+$INVALID; }the DUT port dropped $LOSSES time(s) mid-run (adapter or hub, not the chip)"
fi
STA=$(LC_ALL=C grep -a 'IP Address:'"$DLOG"| grep -v -c '192\.168\.4\.');REAL=$(LC_ALL=C grep -a -o 'real save status [-0-9]*'"$CLOG"| tail -1 | awk '{print $4}')
SAVES=$(count 'Requested WiFi Settings Change'"$DLOG");EXITS=$(count 'cooldown early'"$DLOG");ANSWERED=$(count 'bogus save status 200'"$CLOG")
echo"[hammer] $DNAME: saves requested $SAVES, cooldown exits $EXITS, crash markers $CRASHES, after the AP came up: boot banners $BANNERS, port losses $LOSSES, reboots $REBOOTS; STA addresses $STA"
FAILED=$(count 'bogus save status -1'"$CLOG")
echo"[hammer] $CNAME: bogus saves answered $ANSWERED, failed $FAILED, real save status ${REAL:-none}"
# Only whole messages, matched where the message starts, so a credential line or an odd SSID never gets in
LC_ALL=C grep -a -E "^\[[0-9:]+\](\[[A-Z]\]\[[a-z_]+:[0-9]+\]: *(Exiting cooldown early|Starting fallback AP|Captive portal|Requested WiFi Settings Change|Retry phase|Connected\$|IP Address: )|\[capture\] port lost|($CRASH_RE))""$DLOG"|LC_ALL=C tr -cd '\11\12\15\40-\176'| sed -E 's/(IP Address: )[^ ]+/\1<redacted>/'| cut -c1-110 | head -80 > "$DDIR/$DNAME-events.txt"
echo"[hammer] events: $DDIR/$DNAME-events.txt"
stop_tails
[ -z "$INVALID"]||{echo"[hammer] RESULT INVALID: $INVALID";exit 2;}
# Provisioning is proven by the device joining the network; the client's answer to the real save can time
# out because the device leaves the AP before replying, so its status is information only
# Run at least 60 s after the board rebooted from its crash, with the port closed. Proves the stored
# coredump waits for a reader and survives a reader that goes away:
# 1) open the port for 3 s and close it (a reader that leaves mid-print)
# 2) open it for 100 s: the whole dump must print again, then be erased
# 3) open it again for 20 s: dump_config must report no stored dump
# then decode.sh parses the first complete dump and prints the GDB backtrace.
# Needs a python with pyserial (PYTHON, default python3).
D=${0:A:h}
L=$1;P=$2;ELF=$3;S=${4:-.}
PY=${PYTHON:-python3}
CAP=$D/../serial_capture.py
capture(){# SECONDS OUTFILE: stop the flow when the capture fails (busy or missing port)
if ! $PY$CAP$P$1$2 > $2.err 2>&1;then
echo"capture of $2 failed:"; cat $2.err;exit1
fi
}
capture 3$S/$L-brief.txt
echo"$L brief 3 s open: $(wc -l < $S/$L-brief.txt) lines, CD lines $(grep -a -c '#CD:'$S/$L-brief.txt), END $(grep -a -c '#CD:END#'$S/$L-brief.txt)"
sleep 5
capture 100$S/$L-full.txt
echo"$L full 100 s open: $(wc -l < $S/$L-full.txt) lines, CD lines $(grep -a -c '#CD:'$S/$L-full.txt), BEGIN $(grep -a -c '#CD:BEGIN#'$S/$L-full.txt), END $(grep -a -c '#CD:END#'$S/$L-full.txt)"
grep -a "crashtest\|zephyr_coredump\|Reset reason\|Has stored\| Size:\| Error:\|took a long\|compiled on"$S/$L-full.txt | grep -a -v "#CD:[0-9A-F]"| head -16
sleep 3
capture 20$S/$L-after.txt
echo"$L reopen 20 s: CD lines $(grep -a -c '#CD:'$S/$L-after.txt); $(grep -a 'Has stored\| Size:'$S/$L-after.txt | sed -E 's/^\[[^]]*\]//'| tr '\n'' ')"
# MQTT transport test (#16056, needs that PR's tree for the transport option): one build per transport, name and mqtt block per file (see reference/recipes/mqtt.md).
ESP_LOGI("test", "counters rx=%u tx_attempted=%u (write_array drops silently when the ring is full)", (unsigned) id(rx_total), (unsigned) id(tx_attempted));
} else {
ESP_LOGW("test", "unknown cmd 0x%02X", cmd);
}
continue;
}
id(cdc1).write_array(buf, n);
id(tx_attempted) += n;
}
- interval:100ms
then:
- lambda:|-
if (id(stream_left) > 0) {
char line[101];
int len = snprintf(line, sizeof(line), "STREAM %03d ", 100 - id(stream_left));
# W503: Line break occurred before a binary operator
# E203: Whitespace before ':'
# D202 No blank lines allowed after function docstring
# TODO fix flake8
# D100 Missing docstring in public module
# D101 Missing docstring in public class
# D102 Missing docstring in public method
# D103 Missing docstring in public function
# D104 Missing docstring in public package
# D105 Missing docstring in magic method
# D107 Missing docstring in __init__
# D200 One-line docstring should fit on one line with quotes
# D205 1 blank line required between summary line and description
# D209 Multi-line docstring closing quotes should be on a separate line
# D400 First line should end with a period
# D401 First line should be in imperative mood
ignore =
E501,
W503,
E203,
D202,
D100,
D101,
D102,
D103,
D104,
D105,
D107,
D200,
D205,
D209,
D400,
D401,
exclude = api_pb2.py
Loaded 100 of 5202 files, more files were not shown because too many files have changed in this diff.
Show more
Reference in new issue
Block a user
Blocking a user prevents them from interacting with repositories, such as opening or commenting on pull requests or issues. Learn more about blocking a user.