Co-authored-by: pre-commit-ci-lite[bot] <117423508+pre-commit-ci-lite[bot]@users.noreply.github.com>
Co-authored-by: J. Nick Koston <nick@home-assistant.io>
Co-authored-by: J. Nick Koston <nick@koston.org>
Co-authored-by: pre-commit-ci-lite[bot] <117423508+pre-commit-ci-lite[bot]@users.noreply.github.com>
Co-authored-by: J. Nick Koston <nick@home-assistant.io>
Co-authored-by: Claude <noreply@anthropic.com>
Co-authored-by: pre-commit-ci-lite[bot] <117423508+pre-commit-ci-lite[bot]@users.noreply.github.com>
Co-authored-by: J. Nick Koston <nick@koston.org>
Co-authored-by: J. Nick Koston <nick@home-assistant.io>
Squash of #22111 for the release branch, with the conflicts resolved:
release has no UpdateLock in the LibreTiny OTA backend, so abort() has no
lock check and a successful end() clears wrote_; release negotiates inline
in perform_ota(), so the RTL8710B check is made there.
Co-authored-by: pre-commit-ci-lite[bot] <117423508+pre-commit-ci-lite[bot]@users.noreply.github.com>
Co-authored-by: J. Nick Koston <nick@koston.org>
Co-authored-by: J. Nick Koston <nick@home-assistant.io>
Co-authored-by: pre-commit-ci-lite[bot] <117423508+pre-commit-ci-lite[bot]@users.noreply.github.com>
Co-authored-by: J. Nick Koston <nick@home-assistant.io>
Co-authored-by: pre-commit-ci-lite[bot] <117423508+pre-commit-ci-lite[bot]@users.noreply.github.com>
Co-authored-by: J. Nick Koston <nick@home-assistant.io>
Co-authored-by: J. Nick Koston <nick@koston.org>
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
Co-authored-by: pre-commit-ci-lite[bot] <117423508+pre-commit-ci-lite[bot]@users.noreply.github.com>
Co-authored-by: J. Nick Koston <nick@home-assistant.io>
Co-authored-by: J. Nick Koston <nick@koston.org>
Co-authored-by: pre-commit-ci-lite[bot] <117423508+pre-commit-ci-lite[bot]@users.noreply.github.com>
Co-authored-by: J. Nick Koston <nick@home-assistant.io>
Co-authored-by: J. Nick Koston <nick@koston.org>
Co-authored-by: pre-commit-ci-lite[bot] <117423508+pre-commit-ci-lite[bot]@users.noreply.github.com>
Co-authored-by: J. Nick Koston <nick@koston.org>
Co-authored-by: J. Nick Koston <nick@home-assistant.io>
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
Co-authored-by: pre-commit-ci-lite[bot] <117423508+pre-commit-ci-lite[bot]@users.noreply.github.com>
Co-authored-by: pre-commit-ci-lite[bot] <117423508+pre-commit-ci-lite[bot]@users.noreply.github.com>
Co-authored-by: J. Nick Koston <nick@koston.org>
Co-authored-by: J. Nick Koston <nick@home-assistant.io>
Co-authored-by: pre-commit-ci-lite[bot] <117423508+pre-commit-ci-lite[bot]@users.noreply.github.com>
Co-authored-by: J. Nick Koston <nick@home-assistant.io>
The preamble flag stays set when no sync word follows, so a packet from
another network held every send back until max_busy ran out. A link
without LoRa now needs a sync_value.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Without one the radio cannot tell that a packet is arriving, so listen
before talk does nothing.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
- Without LoRa, require the SX127x radio to have a sync word or preamble
detection, as listen before talk cannot see a packet arriving otherwise
- Warn once when a send finds the radio failed
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
- Warn when a send goes ahead over a busy period at its limit, as that
means a stuck busy indication or a jammed channel
- Log an empty packet at verbose level, and say why it counts as sent
- Name the radio id if code generation cannot find its configuration
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
- Reject a radio with rx_start false when the link or a peer has a layer
that needs packets back from the peer, as the radio sleeps after sending
- Log a send over a long busy period at debug level, so a lost reception
can be traced
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
- Make both links one RadioLink template in packet_interface, with shared
code generation in radio_link.py, so a fix applies to both radios
- While sending is held back, watch the radio from loop() and end the
busy period once it is free, so a later reception starts a new period
instead of being sent over
- Reject an SX127x link without LoRa unless the radio is in packet mode
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The radio links have no buffer of their own, so code generation passes
the configured max_packet_size to the constructor.
Co-Authored-By: Claude Code <noreply@anthropic.com>
- Hold a send back for at most the airtime of the largest packet, worked
out from the radio settings and at least 2 s, instead of a fixed 2 s
that a LoRa packet at a high spreading factor outlasts
- End a busy period when a send finds the radio free, so a busy flag that
never clears is released however rarely sends are tried
- Detect an FSK reception on the SX127x from its preamble and sync word
flags
- Test the radio links' final validation
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Each packet is one radio packet, with the RSSI and SNR of received
packets in the metadata. A radio cannot hear while it sends, so a link
refuses to send while its radio is receiving a packet, for up to two
seconds, and the layers above send again shortly.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
- Work out each layer's packet size during validation, as the C++ does,
and reject a fragmentation max_packet_size whose fragments do not all
fit in the reliability queue at once, or need more than 128 fragments
- Fit the test peer's largest packet in its queue
- Reuse interface_layers() in final_validate_interface()
- Keep the earlier tests in place in test_packet_interface.py
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
A broadcast from a device with its own port went to that port, whose
layers drop it, so a broadcast port could not reach reliable peers. It now
goes to the broadcast port when there is one.
Fragmentation allocates its buffers with RAMAllocator and fails setup when
they cannot be allocated. A fragment from another source no longer drops
the packet being joined.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
AddressingPort derives max_packet_size from the link in its constructor,
and the fragmentation layer passes its configured size to the base
constructor.
Co-Authored-By: Claude Code <noreply@anthropic.com>
- Reject an addressing destination or peer that is this device's own
address, whose packets would be dropped as echoes
- Show 0 fragments, not SIZE_MAX, for a link with no room
- Test the check that applications use a peer's id
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
As in the other layers, addressing ports and fragmentation get no room
when max_packet_size is smaller than their header, and fragmentation
fails its setup instead of dividing by zero.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The addressing layer lets several devices share a channel such as an
RS485 bus or a radio, with 16-bit addresses. A link can carry several
peers, each with its own stack of layers and its own id. Layers that work
between two devices only are rejected with broadcast addressing. The
fragmentation layer splits packets larger than the layers below allow,
and joins them again.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
- Set a warning once the interface has accepted no packets for a minute,
and log refused dictionary packets as data packets are
- Clear remote entities that a provider's dictionary does not name also
when the provider goes back to a dictionary already seen
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
- Check a data packet before asking for its dictionary, so a damaged
packet does not make the provider resend everything
- When a provider's complete dictionary does not name a remote entity,
warn and clear that entity's state, as a typo in provider or remote_id,
or an entity the provider no longer sends, was otherwise silent
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
A dictionary packet now carries its part index and a last-part flag. A dictionary counts as received, and is not
asked for again, only once its last part follows every other part in order. Until then, data under it sends a
request even when some remote entities are already known.
A refused request no longer starts the request interval. A short or malformed packet, or one with another format
version, is logged as a warning at most once a minute.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
With a session and a reliability layer, the update interval defaults to
never, and unchanged states are not sent again, so a receiver's timeout
cleared them on a working link. The session layer already clears the
states when the other device cannot be reached.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Each data record now starts with a 16-bit tag holding the entity's number
and type, and the form of its payload, so that more entity types can be
added and a receiver can skip records of types it does not know. A binary
sensor's state is in the tag; a sensor has a 4 byte state, or none.
Data and dictionary packets are checked in full before anything in them
is used, so a damaged or forged packet changes no state and teaches nothing.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
A dictionary that names none of the remote entities was asked for again
every 5 seconds while any remote entity was unresolved, and each request
made its provider resend everything. A provider's new dictionary was not
asked for once every remote entity was resolved.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The mock link takes max_packet_size in its constructor, so a test that
needs a different size builds a local link.
Co-Authored-By: Claude Code <noreply@anthropic.com>
- Compare received ids by length and bytes, so an id with an embedded NUL
cannot read past the configured string
- Fail setup when max_packet_size cannot hold the device name and the
longest entity id, instead of skipping the id at every resend
- Reject two entities of one type sent with the same id
- Avoid an unused variable without remote entities
- Use matching ids in the test configuration
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Sends sensor and binary sensor states to other devices over a packet
interface. A dictionary sent on connect, on request and at an interval
names the entities, so data packets refer to them by number. The update
interval and the handling of a lost peer follow the layers the interface
has, and remote states can time out.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
A single SHA-256 of the key text let a captured packet check about 2^30
guesses for every one that passes, so a weak key text fell quickly to
an offline search. Derive the key with PBKDF2-HMAC-SHA256 at 600,000
iterations instead; this runs at build time, so the device pays nothing.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The rolling code accepts a packet only once the peer has sent back this
device's boot number, so a device that only receives never gets a packet.
The session's keep-alives make sure packets go both ways.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Both layers derive max_packet_size from the layer below in their
constructors instead of assigning it in the body.
Co-Authored-By: Claude Code <noreply@anthropic.com>
- Fail setup, instead of underflowing, when max_packet_size leaves the
encryption or session layer no room
- Warn when the boot number cannot be saved
- Warn once when the peer's packets carry an older boot number, which
happens when its flash is erased
- Test the counter wrapping to a new boot number
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The session layer sends keep-alives and reports whether the peer can be
reached, including after it restarts or a reliability layer below loses
data. The encryption layer encrypts and checks each packet with XXTEA,
with an optional rolling code against replays. Components that a layer
needs, such as binary_sensor, are loaded only when it is configured.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Such a type would be missing from the link schema, and selecting it would
fail with an unrelated error.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
While the peer is away the layers above keep sending, so the message
repeated about once a second until the link came up again.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
A frame larger than max_packet_size, usually from a peer with a larger
setting, is warned about once. A link reset that the peer does not answer
is warned about after max_retries tries, and a full queue is logged at
debug level while the link is down.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Every module in layers/ is imported automatically, so two modules
registering the same name silently replaced one with the other, in an
order decided by pkgutil. Registration now raises, so the mistake is
found when the configuration is loaded.
Co-Authored-By: Claude Code <noreply@anthropic.com>
A peer that has not heard the UA repeats its SABM, and cannot have sent a
numbered frame meanwhile. Treating that as a restart threw away queued
frames and reported the session lost for a single dropped UA. A SABM is
now a restart only if the peer has sent a numbered frame since the link
came up.
Co-Authored-By: Claude Code <noreply@anthropic.com>
max_packet_size is derived from the layer below in the constructor
instead of being assigned in the body.
Co-Authored-By: Claude Code <noreply@anthropic.com>
- Keep App alive across the HDLC tests, as the test main requires
- Run a layer's loop in the tests only while it has asked for it, so
the loop's disable and request bookkeeping is tested, and wait for the
retry timer in the tests of refused sends
- Report no send capacity while the layer has no send queue
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
- Reset the link after max_retries even when it is 255, where the retry
counter used to wrap
- Require a reliability timeout of at least 50 ms
- Drop received frames larger than the layer's max_packet_size
- Fail setup, instead of aborting, when the send queue cannot be
allocated or max_packet_size leaves no room
- Test the ack delay timer and a refused REJ
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Layers stack on a link and are configured on its packet_interface entry.
Each layer type registers itself with its kind, so a new type needs only
a new module. The first is a reliability layer in the style of HDLC
asynchronous balanced mode, with go-back-N retransmission.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The link derives max_packet_size from the span it is given, so a packet
larger than the copy buffer cannot be copied into it.
Co-Authored-By: Claude Code <noreply@anthropic.com>
The copy buffer is sized by code generation to max_packet_size, but
set_max_packet_size() can raise the limit afterwards, and copying a split
packet would then overflow it. The link now takes the buffer as a span
and refuses a split packet larger than it.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
A read cuts a datagram longer than the buffer to fit, so a 600 byte
datagram arrived as a valid looking 508 byte packet. The buffer is now one
byte longer, and a read that fills it is dropped. The split packet test
also checks the bytes joined in the copy buffer.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Code generation declares the UDP link's buffer for packets split across
buffers as a static array of max_packet_size bytes, so it is not
allocated at run time.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
- Enforce max_packet_size on send and receive
- Copy split packets into a buffer allocated in setup, not on the stack
- Add unit tests for the link's send and receive paths
- Explain why the broadcast socket is always enabled
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Each packet is one UDP datagram, using the settings of the udp component.
The udp component now reports whether a packet was sent, and an ENOMEM
refusal from lwIP is logged at verbose level as the packet can be sent
again.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Co-authored-by: J. Nick Koston <nick@home-assistant.io>
Co-authored-by: pre-commit-ci-lite[bot] <117423508+pre-commit-ci-lite[bot]@users.noreply.github.com>
Co-authored-by: pre-commit-ci-lite[bot] <117423508+pre-commit-ci-lite[bot]@users.noreply.github.com>
Co-authored-by: J. Nick Koston <nick@koston.org>
Co-authored-by: J. Nick Koston <nick@home-assistant.io>
Co-authored-by: pre-commit-ci-lite[bot] <117423508+pre-commit-ci-lite[bot]@users.noreply.github.com>
Co-authored-by: J. Nick Koston <nick@home-assistant.io>
Co-authored-by: J. Nick Koston <nick@koston.org>
Co-authored-by: J. Nick Koston <nick@home-assistant.io>
Co-authored-by: pre-commit-ci-lite[bot] <117423508+pre-commit-ci-lite[bot]@users.noreply.github.com>
Co-authored-by: pre-commit-ci-lite[bot] <117423508+pre-commit-ci-lite[bot]@users.noreply.github.com>
Co-authored-by: J. Nick Koston <nick@home-assistant.io>
Co-authored-by: pre-commit-ci-lite[bot] <117423508+pre-commit-ci-lite[bot]@users.noreply.github.com>
Co-authored-by: J. Nick Koston <nick@home-assistant.io>
Co-authored-by: J. Nick Koston <nick@koston.org>
Co-authored-by: pre-commit-ci-lite[bot] <117423508+pre-commit-ci-lite[bot]@users.noreply.github.com>
Co-authored-by: J. Nick Koston <nick@koston.org>
Co-authored-by: J. Nick Koston <nick@home-assistant.io>
Co-authored-by: pre-commit-ci-lite[bot] <117423508+pre-commit-ci-lite[bot]@users.noreply.github.com>
Co-authored-by: J. Nick Koston <nick@home-assistant.io>
Co-authored-by: pre-commit-ci-lite[bot] <117423508+pre-commit-ci-lite[bot]@users.noreply.github.com>
Co-authored-by: J. Nick Koston <nick@home-assistant.io>
Co-authored-by: pre-commit-ci-lite[bot] <117423508+pre-commit-ci-lite[bot]@users.noreply.github.com>
Co-authored-by: J. Nick Koston <nick@home-assistant.io>
Co-authored-by: J. Nick Koston <nick@koston.org>
Co-authored-by: pre-commit-ci-lite[bot] <117423508+pre-commit-ci-lite[bot]@users.noreply.github.com>
Co-authored-by: J. Nick Koston <nick@home-assistant.io>
Co-authored-by: pre-commit-ci-lite[bot] <117423508+pre-commit-ci-lite[bot]@users.noreply.github.com>
Co-authored-by: J. Nick Koston <nick@koston.org>
Co-authored-by: J. Nick Koston <nick@home-assistant.io>
Co-authored-by: pre-commit-ci-lite[bot] <117423508+pre-commit-ci-lite[bot]@users.noreply.github.com>
Co-authored-by: J. Nick Koston <nick@koston.org>
Co-authored-by: J. Nick Koston <nick@home-assistant.io>
Co-authored-by: pre-commit-ci-lite[bot] <117423508+pre-commit-ci-lite[bot]@users.noreply.github.com>
Co-authored-by: J. Nick Koston <nick@home-assistant.io>
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
Co-authored-by: pre-commit-ci-lite[bot] <117423508+pre-commit-ci-lite[bot]@users.noreply.github.com>
Co-authored-by: pre-commit-ci-lite[bot] <117423508+pre-commit-ci-lite[bot]@users.noreply.github.com>
Co-authored-by: J. Nick Koston <nick@koston.org>
Co-authored-by: J. Nick Koston <nick@home-assistant.io>
Co-authored-by: pre-commit-ci-lite[bot] <117423508+pre-commit-ci-lite[bot]@users.noreply.github.com>
Co-authored-by: J. Nick Koston <nick@home-assistant.io>
Co-authored-by: J. Nick Koston <nick@koston.org>
Co-authored-by: pre-commit-ci-lite[bot] <117423508+pre-commit-ci-lite[bot]@users.noreply.github.com>
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
Co-authored-by: J. Nick Koston <nick@home-assistant.io>
Co-authored-by: J. Nick Koston <nick@koston.org>
Co-authored-by: pre-commit-ci-lite[bot] <117423508+pre-commit-ci-lite[bot]@users.noreply.github.com>
Co-authored-by: J. Nick Koston <nick@koston.org>
Co-authored-by: J. Nick Koston <nick@home-assistant.io>
Co-authored-by: pre-commit-ci-lite[bot] <117423508+pre-commit-ci-lite[bot]@users.noreply.github.com>
Co-authored-by: J. Nick Koston <nick@koston.org>
Co-authored-by: J. Nick Koston <nick@home-assistant.io>
Co-authored-by: Claude Sonnet 5 <noreply@anthropic.com>
Co-authored-by: pre-commit-ci-lite[bot] <117423508+pre-commit-ci-lite[bot]@users.noreply.github.com>
Co-authored-by: J. Nick Koston <nick@home-assistant.io>
Co-authored-by: J. Nick Koston <nick@koston.org>
Co-authored-by: pre-commit-ci-lite[bot] <117423508+pre-commit-ci-lite[bot]@users.noreply.github.com>
Co-authored-by: J. Nick Koston <nick@koston.org>
Co-authored-by: J. Nick Koston <nick@home-assistant.io>
Co-authored-by: pre-commit-ci-lite[bot] <117423508+pre-commit-ci-lite[bot]@users.noreply.github.com>
Co-authored-by: J. Nick Koston <nick@home-assistant.io>
Co-authored-by: pre-commit-ci-lite[bot] <117423508+pre-commit-ci-lite[bot]@users.noreply.github.com>
Co-authored-by: J. Nick Koston <nick@home-assistant.io>
Co-authored-by: J. Nick Koston <nick@koston.org>
Co-authored-by: pre-commit-ci-lite[bot] <117423508+pre-commit-ci-lite[bot]@users.noreply.github.com>
Co-authored-by: J. Nick Koston <nick@home-assistant.io>
Co-authored-by: pre-commit-ci-lite[bot] <117423508+pre-commit-ci-lite[bot]@users.noreply.github.com>
Co-authored-by: J. Nick Koston <nick@home-assistant.io>
Co-authored-by: J. Nick Koston <nick@koston.org>
Co-authored-by: pre-commit-ci-lite[bot] <117423508+pre-commit-ci-lite[bot]@users.noreply.github.com>
Co-authored-by: J. Nick Koston <nick@home-assistant.io>
Co-authored-by: pre-commit-ci-lite[bot] <117423508+pre-commit-ci-lite[bot]@users.noreply.github.com>
Co-authored-by: J. Nick Koston <nick@home-assistant.io>
Co-authored-by: Claude <noreply@anthropic.com>
Co-authored-by: pre-commit-ci-lite[bot] <117423508+pre-commit-ci-lite[bot]@users.noreply.github.com>
Co-authored-by: J. Nick Koston <nick@home-assistant.io>
Co-authored-by: pre-commit-ci-lite[bot] <117423508+pre-commit-ci-lite[bot]@users.noreply.github.com>
Co-authored-by: J. Nick Koston <nick@koston.org>
Co-authored-by: J. Nick Koston <nick@home-assistant.io>
Co-authored-by: pre-commit-ci-lite[bot] <117423508+pre-commit-ci-lite[bot]@users.noreply.github.com>
Co-authored-by: J. Nick Koston <nick@koston.org>
Co-authored-by: J. Nick Koston <nick@home-assistant.io>
Co-authored-by: pre-commit-ci-lite[bot] <117423508+pre-commit-ci-lite[bot]@users.noreply.github.com>
Co-authored-by: J. Nick Koston <nick@koston.org>
Co-authored-by: J. Nick Koston <nick@home-assistant.io>
Co-authored-by: pre-commit-ci-lite[bot] <117423508+pre-commit-ci-lite[bot]@users.noreply.github.com>
Co-authored-by: J. Nick Koston <nick@home-assistant.io>
Co-authored-by: pre-commit-ci-lite[bot] <117423508+pre-commit-ci-lite[bot]@users.noreply.github.com>
Co-authored-by: J. Nick Koston <nick@koston.org>
Co-authored-by: J. Nick Koston <nick@home-assistant.io>
Co-authored-by: pre-commit-ci-lite[bot] <117423508+pre-commit-ci-lite[bot]@users.noreply.github.com>
Co-authored-by: J. Nick Koston <nick@koston.org>
Co-authored-by: J. Nick Koston <nick@home-assistant.io>
Co-authored-by: pre-commit-ci-lite[bot] <117423508+pre-commit-ci-lite[bot]@users.noreply.github.com>
Co-authored-by: J. Nick Koston <nick@koston.org>
Co-authored-by: J. Nick Koston <nick@home-assistant.io>
Co-authored-by: Cursor <cursoragent@cursor.com>
Co-authored-by: pre-commit-ci-lite[bot] <117423508+pre-commit-ci-lite[bot]@users.noreply.github.com>
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
Co-authored-by: J. Nick Koston <nick@home-assistant.io>
Co-authored-by: J. Nick Koston <nick@koston.org>
Co-authored-by: pre-commit-ci-lite[bot] <117423508+pre-commit-ci-lite[bot]@users.noreply.github.com>
Co-authored-by: J. Nick Koston <nick@home-assistant.io>
Co-authored-by: pre-commit-ci-lite[bot] <117423508+pre-commit-ci-lite[bot]@users.noreply.github.com>
Co-authored-by: J. Nick Koston <nick@home-assistant.io>
Signed-off-by: Petr Helebrant <petr@helebrant.cz>
Co-authored-by: pre-commit-ci-lite[bot] <117423508+pre-commit-ci-lite[bot]@users.noreply.github.com>
Co-authored-by: J. Nick Koston <nick@koston.org>
Co-authored-by: J. Nick Koston <nick@home-assistant.io>
Co-authored-by: pre-commit-ci-lite[bot] <117423508+pre-commit-ci-lite[bot]@users.noreply.github.com>
Co-authored-by: J. Nick Koston <nick@koston.org>
Co-authored-by: J. Nick Koston <nick@home-assistant.io>
Co-authored-by: pre-commit-ci-lite[bot] <117423508+pre-commit-ci-lite[bot]@users.noreply.github.com>
Co-authored-by: J. Nick Koston <nick@home-assistant.io>
Co-authored-by: J. Nick Koston <nick@koston.org>
Co-authored-by: pre-commit-ci-lite[bot] <117423508+pre-commit-ci-lite[bot]@users.noreply.github.com>
Co-authored-by: J. Nick Koston <nick@home-assistant.io>
Co-authored-by: pre-commit-ci-lite[bot] <117423508+pre-commit-ci-lite[bot]@users.noreply.github.com>
Co-authored-by: J. Nick Koston <nick@home-assistant.io>
Co-authored-by: pre-commit-ci-lite[bot] <117423508+pre-commit-ci-lite[bot]@users.noreply.github.com>
Co-authored-by: J. Nick Koston <nick@koston.org>
Co-authored-by: J. Nick Koston <nick@home-assistant.io>
Co-authored-by: pre-commit-ci-lite[bot] <117423508+pre-commit-ci-lite[bot]@users.noreply.github.com>
Co-authored-by: J. Nick Koston <nick@home-assistant.io>
Co-authored-by: pre-commit-ci-lite[bot] <117423508+pre-commit-ci-lite[bot]@users.noreply.github.com>
Co-authored-by: J. Nick Koston <nick@home-assistant.io>
Co-authored-by: J. Nick Koston <nick@koston.org>
Co-authored-by: pre-commit-ci-lite[bot] <117423508+pre-commit-ci-lite[bot]@users.noreply.github.com>
Co-authored-by: J. Nick Koston <nick@koston.org>
Co-authored-by: J. Nick Koston <nick@home-assistant.io>
Co-authored-by: pre-commit-ci-lite[bot] <117423508+pre-commit-ci-lite[bot]@users.noreply.github.com>
Co-authored-by: J. Nick Koston <nick@home-assistant.io>
Co-authored-by: Claude Opus 5 <noreply@anthropic.com>
Co-authored-by: pre-commit-ci-lite[bot] <117423508+pre-commit-ci-lite[bot]@users.noreply.github.com>
Co-authored-by: J. Nick Koston <nick@koston.org>
Co-authored-by: J. Nick Koston <nick@home-assistant.io>
Co-authored-by: Cursor <cursoragent@cursor.com>
Co-authored-by: pre-commit-ci-lite[bot] <117423508+pre-commit-ci-lite[bot]@users.noreply.github.com>
Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Co-authored-by: J. Nick Koston <nick@home-assistant.io>
Co-authored-by: Claude Sonnet 4.6 <noreply@anthropic.com>
Co-authored-by: pre-commit-ci-lite[bot] <117423508+pre-commit-ci-lite[bot]@users.noreply.github.com>
Co-authored-by: J. Nick Koston <nick@home-assistant.io>
Co-authored-by: Claude Opus 5 <noreply@anthropic.com>
Co-authored-by: pre-commit-ci-lite[bot] <117423508+pre-commit-ci-lite[bot]@users.noreply.github.com>
description:Write ESPHome automation building blocks - triggers (on_* keys), actions and conditions. Use when adding or changing an on_* trigger, an action such as my_component.do_something, a condition, a callback registration method (add_on_*_callback), or code that calls build_callback_automation, build_automation, register_*_action or register_*_condition.
---
# Automations: triggers, actions and conditions
Automations have three building blocks: **Triggers** (fire when something happens), **Actions** (do
something), and **Conditions** (check if something is true). The helpers named here are documented in
their docstrings in `esphome/automation.py`.
## Triggers: callback method (preferred)
Use `build_callback_automation()` for simple triggers. This eliminates the need for a C++ Trigger class by
using a lightweight pointer-sized forwarder struct registered directly as a callback. No `CONF_TRIGGER_ID`
| `CallbackManager<void(Ts...)>` | 12 bytes (empty `std::vector`) | Callbacks are always or almost always registered |
| `LazyCallbackManager<void(Ts...)>` | 4 bytes (`nullptr`) | Callbacks are often not registered (common case) |
| `AutomationList<N, Ts...>` + `build_automation_lists` | 4 bytes per automation + count, none when unused | A hub fires its own `on_*` automations with no filter (4 B per automation vs 8 B per `Callback`) |
| `automation.new_automation(args, conf)` + the parent's own fixed table | the table entry (pointer plus its own data) | Like `AutomationList`, but each automation needs a stored value of its own, e.g. a threshold the parent compares against |
Both live in `esphome/core/helpers.h` and accept `std::function`, lambdas and forwarder structs through
their templatized `add()` method. `LazyCallbackManager` is a drop-in replacement for `CallbackManager` that defers allocation until the first
callback is added. Prefer it for entity-level callbacks where most instances have no subscribers.
## Triggers: Trigger class method
Use `build_automation()` with a `Trigger<Ts...>` subclass only when the forwarder needs **mutable state
beyond a single `Automation*` pointer** (e.g. edge detection tracking previous state, timing logic).
description:Decide whether an ESPHome change breaks public API and handle it - what counts as public C++ and Python API, the deprecation window, ESPDEPRECATED for C++, cv.rename_key and deprecation warnings for config keys, and the migration notes a PR needs. Use when removing, renaming or changing the signature of a config option, a public C++ member or a core Python helper.
---
# Public API and breaking changes
Full policy: https://developers.esphome.io/contributing/code/#public-api-and-breaking-changes
## What is public
- **C++ in components:** only features documented at [esphome.io](https://esphome.io) are public API.
Undocumented `public` members are internal.
- **C++ core and base classes** (`esphome/core/`, `Component`, `Sensor`, etc.): all `public` members are
public API.
- **Components with global accessors** (`global_api_server`, etc.): all `public` members are public API
(except config setters).
- **Python:** all documented configuration options are public API. Python code in `esphome/core/` that
existing core components use is stable API. Other Python code is internal unless explicitly documented
for external component use.
- Changing the signature of a lambda that code generation injects is not a breaking change.
## Policy
- Aim for a **6-month deprecation window** when possible.
- Clean breaks are allowed for signature changes, deep refactorings and resource constraints.
- The migration path goes in the PR description (it generates the release notes).
- A blog post is required for core/base class changes or significant architectural changes.
description:Review guidance for ESPHome pull requests. Use this when reviewing a pull request that changes ESPHome Python, C++, or component code, to check it against the project's coding conventions, embedded-systems memory rules, testing requirements, and breaking-change policy.
---
# Reviewing ESPHome pull requests
ESPHome parses YAML into C++ firmware for memory-constrained microcontrollers
(ESP32, ESP8266, RP2040, LibreTiny). Review changes with that in mind: RAM and
flash are scarce, and code runs unattended for months.
`AGENTS.md` in the repository root is the contributor guide, and the skills it
| set warning / `return` / clear warning around a bus result | `if (!this->status_comm_result(ok)) return;` |
| hand-written `set_x_sensor()` plus `x_sensor_` member | `SUB_SENSOR(x)`, `SUB_BINARY_SENSOR`, `SUB_TEXT_SENSOR`, `SUB_SWITCH`, ... |
| `if CONF_X in config: s = await sensor.new_sensor(config[CONF_X])` plus setter | `sensors = sensor.sub_sensors(config)` then `await sensors(CONF_X, hub.set_x_sensor)` |
| `if (x := config.get(CONF_X)) is not None: cg.add(var.set_x(x))` | `cg.add_optional(var.set_x, config.get(CONF_X))`, when the setter takes the value unchanged |
## Python conventions
- Type-annotate every new function signature (params and return), new dataclass
fields, and new module-level variables. Import `ConfigType` from
`esphome.types`.
- Use the walrus operator to avoid a double lookup, e.g.
`if (blah := config.get(CONF_BLAH)) is not None:`.
- Reuse existing validators from `config_validation.py` (`cv.rename_key`,
`cv.has_exactly_one_key`, etc.) via `cv.All(...)` instead of hand-rolling.
- A new or changed validator function that wraps, chooses or builds a schema
must be decorated with `@schema_extractor(...)` and handle `SCHEMA_EXTRACT`,
or `script/build_language_schema.py` silently drops the keys behind it (see
the `config-validation` skill).
-`esphome/const.py` is frozen: no new `CONF_` constants there. Define them in
the component's own `.py`, or in `esphome/components/const/__init__.py` when
shared. The same constant defined in three or more component files fails CI.
- State that must persist during code generation goes in `CORE.data` namespaced
under the component `DOMAIN` (a `@dataclass`), not module-level mutable globals.
- Prefer callback-based triggers via `build_callback_automation()`; only use a
`Trigger<Ts...>` subclass when the forwarder needs mutable state.
## Testing and coverage
- New and changed lines and branches need test coverage, including defensive
early-returns, error paths, and no-op guards. A mocked-out function is not
covered; exercise the real call path too.
- Component YAML tests live in `tests/components/<component>/`. Never define
buses (uart, i2c, spi, modbus) directly in a test file: pull them from
`tests/test_build_components/common/` through dict-style `packages:` so CI can
group builds. List-style packages or top-level merge keys block grouping.
- Config-only checks use the `validate.*.yaml` prefix; compiled checks use
`test.*.yaml`.
## Naming user-facing functions
Documented lambda methods, actions and config keys are public API, so a bad name
is hard to fix once released. Flag:
- A boolean where `true` means off (`display_off(true)`): name the positive
state, e.g. `set_power(bool on)`.
- A name that differs from a sibling component's name for the same operation.
Before a release, rename outright; after, follow the breaking-change policy
below.
## Breaking changes and public API
- Base classes under `esphome/core/` and documented config options are public
API. Undocumented `public` members of a component are internal.
- A breaking change needs justification, a migration path in the PR description,
and a deprecation window where feasible (`ESPDEPRECATED` in C++,
`cv.rename_key(..., removed_in=...)` in Python). Changing a codegen-injected
lambda signature is not a breaking change.
## Process and PR hygiene
- PR titles start with a `[tag]` prefix: the component name (e.g. `[uart] ...`)
or `[core]` for shared code.
- Prose in docs, comments, and commit messages should be plain English. Keep
inline comments short and only where the code is not self-explanatory; do not
restate what the code says.
- Verify the PR fills out `.github/PULL_REQUEST_TEMPLATE.md` and adds
description:Write the Python side of an ESPHome component - __init__.py and platform files (sensor.py, switch.py, ...), DOMAIN and other metadata, to_code, hub child entities with sub_sensors and friends, pins and referenced components, hub-forwarding entities (ApplyButton, ApplySwitch), uart/i2c/spi constructor helpers, and CORE.data state. Use when creating a component or changing its code generation.
---
# Component code generation
Read https://developers.esphome.io/architecture/components/ for the component lifecycle and how a
component is registered. Schema rules and schema extraction are in the `config-validation` skill;
automations (`on_*`, actions, conditions) in the `automations` skill.
## Files
```
components/[component_name]/
├── __init__.py # Component configuration schema and code generation
├── [component].h # C++ header file (if needed)
├── [component].cpp # C++ implementation (if needed)
description:Write and run tests for ESPHome components - YAML build tests (test.*.yaml, validate.*.yaml, common.yaml) under tests/components, shared bus packages and CI test grouping, C++ gtest unit tests, and host integration tests including the uart_mock fixture. Use when adding or changing files under tests/, or when a code change needs test coverage.
---
# Component tests
## C++ unit tests
gtests live in `tests/components/<component>/*.cpp` and run with `script/cpp_unit_test.py <component>`; see
`tests/components/README.md`. Set the loop clock in a test with `LoopBlockingGuard{nullptr, nullptr,
now_ms}`; the runner resets it before every test.
## Integration tests
`tests/integration/` runs real firmware on the host. Serial components prove timeouts and recovery with the
`uart_mock` fixture (`tests/integration/test_uart_mock_*.py`). Its `responses` match on content every time
they see it; to answer a repeated request differently, use an `on_tx` lambda with a static counter and
`inject_to_rx_buffer` (see `uart_mock_ld2412_requery_after_giveup.yaml`).
description:Write or change ESPHome configuration validation - CONFIG_SCHEMA, cv.Schema, cv.typed_schema, custom validator functions, config keys and CONF_ constants, renamed or deprecated keys, and the language schema that script/build_language_schema.py extracts from them. Use whenever a change touches a component's schema or a validator in config_validation.py, automation.py or a component's .py files.
---
# Configuration validation
ESPHome validates YAML with `voluptuous` schemas built from `esphome/config_validation.py` (`cv`). The
same schemas are also walked by `script/build_language_schema.py`, which produces the JSON language schema
that editors and the dashboard use for completion and checking. A validation change is only finished when
both still work.
## Writing schemas
- **Type hints:** validators are typed like any other function: `def validate_x(config: ConfigType) ->
ConfigType:`, with `ConfigType` imported from `esphome.types`.
- **One schema per model:** when a component's options depend on a `model` key, use
`cv.typed_schema({...}, key=CONF_MODEL, default_type=...)` with each model's own keys and defaults, not
one schema plus a validator that rejects keys and fills defaults afterwards.
- **Reuse existing validators:** before writing a custom validator, check for an existing one in
`config_validation.py` and compose it in `cv.All(...)` rather than duplicating logic across components.
For example, rename a config key with `cv.rename_key(CONF_OLD, CONF_NEW, removed_in="2026.6.0")`, and
reject mutually-exclusive keys with `cv.has_at_most_one_key(...)` / `cv.has_exactly_one_key(...)`. See
how `api` composes `cv.has_exactly_one_key` + `cv.rename_key`.
The type names the dumper accepts, and what `SCHEMA_EXTRACT` must return for each, are in the
`hidden_schemas` branch of `convert()` in `script/build_language_schema.py`. The common ones:
| Type | Return for `SCHEMA_EXTRACT` | Example |
|------|-----------------------------|---------|
| `"schema"` | the schema (or a representative one) to walk | `api` `_outgoing_connection_schema`, `restore_state_schema` in `esphome/core/entity_helpers.py`, `model_schema_extractor` in `esphome/components/mipi/__init__.py` |
| `"enum"` | the dict of options | `sensor_entity_category` in `esphome/components/sensor/__init__.py` |
| `"one_of"` | the list of options | `cv.one_of` |
| `"maybe"` | `(validator, key)` for a scalar shorthand | `cv.maybe_simple_value`, `maybe_conf` in `esphome/automation.py` |
| `"templatable"` | the inner validator | `cv.templatable` |
| `"automation"` | the automation schema | `automation.validate_automation` |
Points to watch:
- The dumper matches the function object by `repr`, so the decorated function must be the exact object
placed in the schema. In a factory that returns a new closure per call, decorate the inner function (as
`restore_state_schema` and `cv.maybe_simple_value` do), not the factory.
- The decorator only records anything when extraction is enabled, and that flag is set before imports, so
the decorator must run at import time (module level or inside a factory called at module level).
- A schema chosen per config value (for example by `model`) has no single answer: return a representative
schema, as `model_schema_extractor` does.
- Plain value checks (`cv.All(cv.string, my_check)`) need nothing: the dumper still sees the surrounding
types and keys.
- `cv.Schema`, `cv.typed_schema`, `cv.ensure_list` and the `cv` helpers are already handled; prefer
composing them over a custom function, which also avoids the extractor work.
### Checking the output
Build the language schema into a scratch directory and look at the component's JSON (it takes about ten
description:Test an ESPHome display driver on a real panel with a camera. Use when a task asks to verify a display (LCD, e-paper, OLED, RLCD) on hardware, check orientation/rotation, partial updates, colours, contrast, or that text and shapes render correctly. Covers finding the USB port, flashing a test pattern, capturing the panel with any camera, and reading the photo.
allowed-tools:Read, Write, Edit, Bash, Glob, Grep
---
# Display testing with a camera
Point a camera at the panel to get visual evidence that a driver renders correctly. Any camera
and capture tool works; this skill covers the common ones. `$SKILL` below is this directory
(`.agents/skills/display-test`).
## Ask before using a camera
A camera can show people and rooms, not only the panel. Before the first capture, ask the user
for permission and for which camera to use. List the cameras the host has (the commands below
show how), name the one you intend to use, and wait for the user to confirm it. Do not pick a
camera yourself, and do not fall back to another camera if the chosen one fails; ask again.
The permission covers this session and the camera the user named. Ask again for a different
camera or in a new session. If the user declines, stop and report that the display was not
checked visually.
## Capturing a frame
Grab a still image, skipping the first few frames so auto-exposure and white balance settle.
Read the JPEG with the Read tool; the image is shown visually.
### macOS
The bundled Swift tool uses AVFoundation and avoids the frame-grabbing problems some USB
description:Choose ESPHome C++ containers and storage that avoid heap use and fragmentation on microcontrollers - std::array, StaticVector with cg.slot_list/SLOT_LIST, FixedVector, std::unique_ptr byte buffers, StringRef, RAMAllocator, and what to avoid (std::vector growth, std::map, std::deque, std::string, new (std::nothrow)). Use when adding or changing C++ members, containers, buffers, strings or registration lists, or when reducing RAM or flash use.
---
# Memory on embedded targets
ESP devices run for months with small heaps shared between Wi-Fi, BLE, LWIP, and application code. Over
time, repeated allocations of different sizes fragment the heap. Failures happen when the largest
contiguous block shrinks, even if total free heap is still large. We have seen field crashes caused by
this.
**Heap allocation after `setup()` should be avoided unless absolutely unavoidable.** Every
allocation/deallocation cycle contributes to fragmentation. ESPHome treats runtime heap allocation as a
long-term reliability bug, not a performance issue. Helpers that hide allocation (`std::string`,
`std::to_string`, string-returning helpers) are being deprecated and replaced with buffer and view based
APIs.
## Container guidelines
1.**Compile-time-known sizes:** use `std::array` instead of `std::vector` when size is known at compile
time.
```cpp
// Bad - generates STL realloc code
std::vector<int> values;
// Good - no dynamic allocation
std::array<int, MAX_VALUES> values;
```
Use `cg.add_define("MAX_VALUES", count)` to set the size from Python configuration.
**For byte buffers:** avoid `std::vector<uint8_t>` unless the buffer needs to grow. Use
`std::unique_ptr<uint8_t[]>` instead. It does **not** provide bounds checking or iterator support like
`std::vector<uint8_t>`; use it only when you do not need these features.
description:Test an ESPHome change on real boards over USB. Use this whenever a task asks to test, verify, reproduce, bench, A/B or soak a pull request, a branch or a crash on hardware, or mentions flashing, serial logs, esphome upload, a captive portal or fallback AP test, or results to post on a PR. Covers finding the serial ports, building from the right checkout, flashing each chip family (including the LibreTiny kits that stock tools cannot flash), bounded log capture, proven test flows and how to report results.
allowed-tools:Read, Write, Edit, Bash, Glob, Grep
---
# Hardware testing
The goal of a hardware test is evidence a reviewer can trust: which build ran on which board, what was
driven, what the logs show, and the same flow on a `dev` build when a quirk could be pre-existing.
`$SKILL` below is this directory (`.agents/skills/hardware-test`).
## Ground rules
1.**Serial ports cannot be shared.** One bounded capture per port, stopped before any flash on that
port, never an open-ended `esphome logs`. Stop a capture with `pkill -f "[s]erial_capture.py <port> "`,
naming the port so a capture on another board survives; the bracket keeps the pattern from matching
the shell that runs it. When more than one agent session shares the bench, run a copy of the script
under a session-specific name (`cap<topic>.py`) so another session's `pkill` cannot take your capture
down. Never run `esphome logs` in a `while true` loop: the loop respawns it and holds the port.
2.**Credentials stay local.** Put a `secrets.yaml` next to the scratch configs and reference it with
`!secret`. A copy of a device's own config keeps using the secrets file next to that config: two
checkouts can hold different networks, and the wrong one leaves the device unreachable after the OTA. Never paste a config with credentials into a PR, never commit scratch configs or logs.
3.**Build from the tree under test.**`python -m esphome` puts the current directory first on
`sys.path`, so run it with the checkout or worktree as the working directory and pass the yaml by
its absolute path. A venv `esphome` entry point may import an installed snapshot instead of the
checkout. Confirm with `python -c 'import esphome; print(esphome.__file__)'`, and for an A/B grep
the build's `src/` for the change before trusting any number. Tool calls sent together share one
shell, so a `cd` in one can land another's compile in the wrong tree: pass `PYTHONPATH=<worktree>` on
every compile and upload command and print `esphome.__file__` into the compile log.
4.**PR build and baseline build get different `name:` values.** Builds share
`.esphome/build/<name>`, so the same name means `esphome upload` can flash the wrong binary. Saved
Wi-Fi credentials are keyed by the config hash when the config has `networks:`, so a test that must
start unprovisioned gets a fresh name every run. A config with `ap:` only and no `networks:` uses one
fixed key shared by every such config on the board, so a new name does not help there: wipe once at
boot with an `on_boot` lambda that resets the preferences and saves a marker key
(`templates/dut-ap-only-wipe-once-bk72xx.yaml`).
5.**Flash and RAM deltas are measured against the PR's merge base** (`git merge-base origin/dev HEAD`
checked out in a throwaway worktree), with the baseline commit named in the table. When a delta
looks odd, diff symbol sizes (`nm -S -C --size-sort` from the matching toolchain) before quoting it.
When a change moves storage between the heap and static RAM, the static number alone misleads: also log
the free heap at the end of setup (`on_boot` at priority -100) over three boots of each build and report
the total in use (static RAM plus heap used).
6.**Prove a quirk is pre-existing before blaming the PR:** run the identical flow on a `dev` build in
the same session and say so in the results.
7.**Pipe safely.** Set `PYTHONUNBUFFERED=1` on any `esphome run` or `esphome logs` whose output is
piped, or lines land minutes late. Bound every long command with `timeout` (coreutils) or
`perl -e 'alarm shift; exec @ARGV' <seconds> <command>` on macOS without it. Save an exit code with `rc=$?`
right after the command: in `echo "$(date +%T) rc=$?"` the command substitution runs first and `$?` reads 0.
## Workflow
1.**Find the boards:**`python $SKILL/scripts/ports.py` (any Python with pyserial, for example the
ESPHome venv). Port names follow the USB hub port and change on every replug, so never reuse a name
from an earlier session. CH340 adapters (D1 mini, BW15 kit, BK7238, CB3S) all look the same; tell
them apart with a 10 s capture using `serial_capture.py --reset` (ESP8266 prints its ROM banner at
74880 baud, LibreTiny boards print `[LT]` and ESPHome lines, BK7238 prints its bootloader version).
`ports.py --wait 2e8a` blocks until a Pico re-enumerates.
2.**Make a scratch directory** outside the checkout, with `secrets.yaml` and the templates you need.
One yaml per board per variant, file name equal to `name:`.
Keep the RAM and flash lines at the end. Two ESP-IDF or two LibreTiny builds at the same time race
over tool caches; run them one at a time.
4.**Flash** per chip family (table below). Stop the capture on that port first.
5.**Capture** with `serial_capture.py PORT SECONDS OUT` in the background, then poll the file with
`grep` in an `until ... sleep 2` loop with a cap. USB CDC boards (Pico, nRF52, native USB ESP32)
drop the port on reboot; the script reopens it until its deadline.
6.**Drive** the test: a second board running a client config, `curl`, an aioesphomeapi script that
presses a template button and subscribes to logs, MQTT publishes, or `interval:` and `on_boot:`
automations in the config itself.
7.**Analyze** with `grep -a -c` counts over the capture and keep an `-events.txt` extract of the lines
that matter. Decode crashes with the platform's `addr2line` (see `reference/boards.md`).
8.**Report** on the PR. Every result comment starts with this two-line GitHub alert, then a blank line:
```
> [!NOTE]
> Automated testing results
```
Then one short sentence saying what was tested (builds and SHAs, board, what was driven), a results table
(build SHA, board and framework, what was driven, counts, result, plus the `dev` control row), short bullets
for the findings, and the setup and exact flow in a collapsed `<details>` block. Answer any question asked on
the PR directly in the comment. No credentials, SSIDs, IP addresses, port names or other bench details.
## Flashing by chip family
| Board | Flash | Log |
|---|---|---|
| ESP8266 D1 mini (CH340) | `python -m esphome upload <yaml> --device <port>` | `serial_capture.py` at 115200; `esphome logs` on some CH340s emits garbage after the RTS reset. A failed esptool write can wedge the adapter until the board is replugged. `esptool erase_flash` before a test that needs an unprovisioned device: the preference sector survives a reflash |
| ESP32 devkit (CP2102 or CH340) | `esphome upload --device <port>`, ESP-IDF or Arduino | `serial_capture.py`. Default Wi-Fi light sleep delays API driven actions by seconds; judge timing from device logs. `power_save_mode: none` for OTA overlap tests |
| ESP32-S3, C3, C6, H2 native USB | `esphome upload --device /dev/cu.usbmodem*` | `logger: hardware_uart: USB_SERIAL_JTAG`. The USB console can wedge when the app takes over the console (0 bytes, esptool `No serial data received`); replug, or watch through `esphome logs --device <name>.local` |
| Pico W / Pico 2 W | `esphome upload --device <port>` (1200 baud touch); first flash ever via BOOTSEL and `--device /Volumes/RPI-RP2` or `RP2350` | port re-enumerates after upload: `ports.py --wait 2e8a` then capture |
| BW15 kit (RTL8720CF, rtl87xx) | `~/.platformio/penv/.libretiny/bin/python $SKILL/scripts/flash_rtl87xx_kit.py flash write -d <port> <build>/.pioenvs/<name>/*.uf2`. Plain `esphome upload` and plain ltchiptool time out because opening the port asserts DTR/RTS and holds the chip in reset | `serial_capture.py --released`, never `esphome logs` |
| BW12 and other AmebaZ modules (RTL8710BX, rtl87xx) on a plain USB-UART | `~/.platformio/penv/.libretiny/bin/python -m ltchiptool flash write -B 115200 -b 115200 -t 120 -d <port> <uf2>`, started first, then strap by hand: CEN to GND, TX2 to GND, release CEN, release TX2. Without `-B 115200` the system data read at 1.5 Mbaud fails on CH340s (`Data read error at 0x9000`) and nothing is written. A 1 MB image takes about 80 s. A development board with DTR and RTS wired to CEN and the download pin needs no strap | `serial_capture.py`; a reset pulse does not reach CEN on a plain adapter, so a capture starts mid-run |
| BK7238 / CB3S (bk72xx, RTS on CEN) | `~/.platformio/penv/.libretiny/bin/python $SKILL/scripts/flash_bk72xx.py flash write -d <port> <uf2>` (about 75 s) | logs are on UART2 by default; `logger: hardware_uart: UART1` puts them on the flashing adapter. `serial_capture.py --reset` gives a chip-enable reset; a plain open may or may not reset it, check for the bootloader banner |
| LN882H | ltchiptool forces 1 Mbaud for writes; weak adapters need `LN882H_YM_BAUDRATE` patched to 115200 in the penv's `ln882htool.py`; short BOOT to GND while the tool connects | OTA after the first flash |
| nRF52840 Dongle (PCA10059) | Nordic `nrfutil nrf5sdk-tools dfu usb-serial` on the bootloader port, or `dfu: true` in the config so the 1200 baud touch pin-resets into the bootloader; adafruit-nrfutil is refused | app has two CDC ports, ESPHome log on the second |
| Adafruit-style nRF52 (UF2 bootloader) | `UF2BOOT` volume or adafruit-nrfutil; `dfu: true` works through GPREGRET | single CDC port |
| Any reachable device | `esphome upload <yaml> --device <ip or name.local>`; `--file <bin>` for a prebuilt image | `esphome logs --device <ip>` bounded with `timeout` |
ESP-IDF OTA image: `.esphome/build/<name>/build/firmware.ota.bin`. LibreTiny OTA port is 8892.
## Scripts in `$SKILL/scripts`
- `ports.py`: list ports with VID:PID, product, serial and a board guess; `--wait VID[:PID]`,
`--product NAME`, `--serial SN` (one specific board, whatever path it comes back under) and
`--serial-of PORT` (the serial number behind a path).
- `serial_capture.py PORT SECONDS OUT [--baud] [--released] [--reset] [--append] [--ms]`: bounded timestamped
capture (`--ms` adds milliseconds for timing work), ANSI stripped, `LOG_SECRET` text written as `<secret>`, reopens a vanished port, exits at the
deadline. OUT is truncated unless `--append` is given, so counts over it cover one run.
- `flash_rtl87xx_kit.py`: ltchiptool wrapper for the BW15 kit whose CH340 drives CEN (RTS) and the boot
strap (DTR). `flash read` needs the family `realtek-ambz2`; `flash write` does not.
- `flash_bk72xx.py`: ltchiptool wrapper for BK72xx boards whose RTS drives CEN. It misses the link now
and then; retry up to three times.
- `api_button.py HOST KEY list|press NAME [...]`: list a device's template buttons or press them over the API; the
recipes drive their test configs with it.
- `apiconn.py`: `connect(host, tries)` (API connect and login with retries, `API_KEY` from the env) and `log_text()`
(an API log line without ANSI codes), imported by the newer API drivers.
capture decoder, RTL8720C ROM dump versus report matcher), `api_outgoing/` (wake cycle and Wi-Fi drop timing readers), `ld2450/` (zone number driver, loopback harness driver with number and binary sensor states and TX frame summarizer, frame stream harness with presence hold and link loss drivers, BT MAC driver), `api/` (raw plaintext native API client for user defined actions, store_yaml recovery client over Noise or plaintext), `http_request/` (update manifest server, update state driver, request test server, action case driver, concurrency, reboot, update check and OTA drivers, `ota_auth/` OTA request cases with basic auth), `modem/` (button and sensor driver over the plaintext API, also used for the BLE MIDI boards), `dfrobot_sen0395/` (API driver and case runner), `homeassistant_light/` (direct bulb driver, held-report case driver), `usb_audio/` (URL player), `speaker_media_player/` (API player driver, pipeline task case runner), `web_server/` (SSE client, request mix, SSE and HTTPD case runners), `mqtt/` (`ha_discovery/`, `disable/`, `retained_burst/` and `reconnect/` runners; broker config with TLS, client-certificate and WebSocket listeners, test CA with server and client certificates, transport case runner, flash-and-drive runner, CLI TLS check without a board, MAC suffix runner with per-variant configs for two boards, `shutdown/` shutdown message runner and matcher), `udp/` (datagram recorder, `udp.write` payload checker and case driver), `entity_icons/` (icon and device class readback over the API, MQTT discovery, web_server and dump_config, with a comparer), `uart_sim/` (API driver and flash, capture and drive runner for the simulated UART sensors), `remote_base/` (loopback runner and API log decoder per sent frame), `xiaomi_ble/` (frame and header generator with the expected counts, injecting, advertising and receiving configs, counter and runner), `ble_terminate/` (proxy and peer configs, connect and disconnect driver, peer disconnect matcher, runner), `i2c_slave/` (D1 mini slave and BK7238 master configs), `wpa3_ap/` (PMF AP with join and leave logging, BK7238 and CB3S DUTs, runner and summary), `wisp_ap/` (WISP upstream with AP buttons, WISP and ALWAYS_ON DUTs, client, runner and per phase summary), `wifi_restart/` (WPA3 mixed WISP upstream, static IP DUTs with syslog, runner and summary), `ota_timing/` (OTA upload timer with deflate, raw and TCP modes, API build check, A/B runner and median summary, `inflate_bench/` on-device inflate benchmark with stream generator, configs and syslog runner), `gpio_hold/` (ISR pin hold self test with ESP32 and S3 configs), `ble_event_inject/` (GAP and GATTC event injector header and proxy package), `light_sleep_wake/` (LEDC wave and edge timer for light sleep wakeup on a C3, external wave generator and per-call edge detail probe for peripheral power-down on the P4), `manual_ip/` (station and AP manual IP configs, DNS probe and runners), `web_handlers/` (late handler, web_server, captive_portal and prometheus variants, web OTA runner), `bk72xx_flash/` (flash readback with retries, per partition diff of two dumps), `ble_server_security/` (secure GATT server and pairing client configs, driver, summary and runner), `prometheus/` (datetime metrics configs and driver), `zephyr_ble_server/` (nRF52 server, looping ESP32 client and UF2 flash and count runner), `ble_adv_triggers/` (advertiser with six raw frames, ESP32 and BK72xx receivers with eleven triggers, runner and checker), `sun/` (longitude sweep package, runner and checker), `sensor_filters/` (every filter shape with a fixed input sequence, per-board packages and runner), `led_driver_sniff/` (SPI slave capture header with selftest, config generator, window comparer and runner), `hub_input_sim/` (keypad, wiegand and rdm6300 simulation on a D1 mini, API log runner), `value_range/` (on_value_range sequence with a restart, runner), `ezo_fake_bus/` (fake EZO I2C bus and step config, runner), `twai_self/` (TWAI self-reception header, on_frame config and counter runner), `ota_automations/` (OTA automation package for both OTA platforms, four case runner, repeated web upload runner and per case summary), `output_button/` (pad edge timer header, button package and runner), `homeassistant_button/` (raw plaintext API client for action calls and wizard inputs, package and runner), `uart_button/` (UART0 loopback package, API driver with TX and RX counts, runner), `interval/` (plain and delayed interval package, timing summary and runner), `smp_udp_ota/` (SMP reset fault injector, OTA runner, per mode result check, MCUboot swap limit, running image check and slot blanking UF2, Thread device config), `ld2420_sim/` (fake LD2420 answering the hub's commands on a UART0 loopback, API state recorder, two-build comparer and runner), `nrf52_tcp_probe/` (TCP and UDP socket probe config, build and log check, API check over Thread), `rs485_frame/` (simulated DLE/STX/ETX bus device, frame and gate cases, runner), `hx711/` (fake HXLine truncated
## Templates in `$SKILL/templates`
- `dut-fallback-ap.yaml` (package): a device that never finds its network, falls back to its AP and
serves the captive portal. `example-dut.yaml` is the per-run file with the platform block.
- `client-wifisave.yaml` (package): joins that AP, posts N bogus `/wifisave` requests, then the real
credentials from `!secret`. `example-client.yaml` is the per-run file. `bogus_saves: "1"` gives a
plain provisioning check.
- Per-topic device configs, each described in `reference/recipes/<component>.md`: `online-image-dut.yaml`,
and the directories `store_yaml/` (package with secrets one directory up, per-board files),
`http_request_update/` (manifest URL setter shapes, per-board files), `modem/` (modem only and modem plus
Wi-Fi boot checks), `ble_midi/` (fake BLE MIDI instrument and the client), `http_request_action/` (request
action package with per-board files and an OTA flash variant), `led_wire/` (DUT config and header that reads the
LED strip pads back with RMT RX and an SPI slave), `uart_sim/` (simulator header, common package, per-build parts and run files) and `remote_base/` (common
package, transmit script, per-board templates and the ESP8266 software sender), `i2c_fake/` (fake I2C device
headers with their test configs, see `fake_i2c.md`).
- `microphone_http_stream/`: base config, fake counter and USB microphone variants and the multi-reader client,
described in `reference/recipes/microphone_http_stream.md`.
## When the PR keeps moving
Record every head in the stack (`gh pr view N --json headRefOid`), build, and recheck before flashing; if anything
moved, wait five minutes and check again so the test runs on a settled head. When a head moved only by merging dev,
compare the PR's own diff (`git diff $(git merge-base OLD origin/dev) OLD` against the same for NEW); if it is
identical and the dev commits do not touch the code under test, earlier results still describe it, but rebuild so the
comment quotes the exact commits. Fetch `origin dev` before computing a merge base.
## When something looks wrong
- A capture stopped early or a line is missing: check the timestamps before trusting it.
- A script that pipes into `tee` never ends although its work is done: a background command written as
`(cd dir && server & echo $!)` backgrounds the whole `cd && server` list, so a shell stays behind holding the
pipe open and `$!` is that shell, not the server. Write `(cd dir && exec server > out 2>&1 < /dev/null) &`.
- The client never joins the AP: another board may still run an AP with the same SSID, or the device
booted with saved credentials because the name was reused. One AP device at a time on the bench.
- Builds come out byte-identical between branches: the wrong tree was imported (rule 3). The reverse
too: two binaries that differ only in the build timestamp and in `__LINE__` values of log calls below
the changed code are the same build; map the offsets with `objdump -h` and `addr2line` first.
- A boot the host cannot capture (a power cut, an OTA-only board): keep the boot report in a flash
preference and repeat it in a periodic alive line, as `scripts/libretiny_prefs/ltprefs.hpp` does.
- Multicast from the host never arrives on some access points: test it device to device. Broadcast
is lossy under Wi-Fi power save, unicast is not. Throttle UDP floods to a fixed packet rate.
- The merge base comes from a freshly fetched `origin/dev`; a stale local `dev` puts dependency bumps into the
PR diff and builds the control from the wrong commit.
- A `.local` name can stop resolving in the middle of a run; keep the address from the boot log and drive by it.
- Never point `framework: source:` at a local `file://` copy of LibreTiny to try a patch: PlatformIO installs it over
the shared `~/.platformio/platforms/libretiny`, so every later build on the machine uses the patched copy. Patch
through the ESPHome side (a linker `-Wl,-wrap,<symbol>` or a define) instead, and if it happened, restore the
directory from a clean copy of the same release.
- `wc -l` pads its count on macOS, so `tail -n +$n0` needs `$((n0+0))` or an arithmetic expansion.
- `api:` reboots a board after 15 min without a client: `reboot_timeout: 0s` on `api:` and `mqtt:` for
soak tests.
`reference/boards.md` has per-chip facts and crash decoding; `reference/recipes/` has the proven
Each flow names the pull requests it was used on, so their result tables show what to post.
One file per component, named after it:
- [`api.md`](api.md): noise handshake and OTA encryption on the ESP8266, outgoing connection wake timing on deep sleep and after a Wi-Fi drop, user defined action listing and execution with a raw plaintext client, store_yaml recovery over Noise or plaintext
- [`ble_midi.md`](ble_midi.md): BLE MIDI client against a fake ESP32 instrument, every packet shape and the send actions
- [`captive_portal.md`](captive_portal.md): fallback AP and /wifisave hammer
- [`cse7766.md`](cse7766.md): Sonoff S31 parser check over the API
- [`http_request.md`](http_request.md): update manifest URL changed at run time, every setter shape, a swap under a running check, non-blocking request actions with loop gap, queue, cancel and reboot checks, OTA request cases with basic auth and MD5 sources
- [`ld2420.md`](ld2420.md): hub setup and sensor states with a simulated module on a UART0 loopback (energy and simple mode)
- [`ld2450.md`](ld2450.md): polygon zones, zone number round trip through the radar's readback, ESP8266 loopback harness without a radar, presence hold and link loss on a frame stream, BT MAC readback
- [`ld2412.md`](ld2412.md): output pin level through the basic configuration command, held back writes
- [`libretiny.md`](libretiny.md): rtc preference emulation, BK72xx and RTL8720C crash report on the next boot with a ROM dump matcher, BK7238 BLE bond store safety with a full flash backup and per partition diff, BK7238 OTA versus the board layout and bootloader
- [`modem.md`](modem.md): PPPoS modem component boot check with no modem attached, loop blocking and disable timing
- [`mqtt.md`](mqtt.md): broker bench, QoS, reconnect, crash log over MQTT, WebSocket transports with a local and a public broker, TLS trust sources (own CA, client certificate, certificate bundle, no verification) with the CLI check, MAC suffix topics and will messages on two boards, shutdown message delivery on restart, disable and deep sleep
- [`nrf52.md`](nrf52.md): dongle board support, zephyr_ble_server connection slot under reordered callbacks, SMP over UDP OTA over Thread with reset faults, MCUboot swap limit and stuck slot recovery, TCP stack probe
- [`online_image.md`](online_image.md): stalled and malformed downloads, PNG, JPEG, BMP and QOI decode time per loop pass with large images and small buffers
- [`openthread.md`](openthread.md): border router on the S3 host plus H2 RCP
- [`ota.md`](ota.md): upload without a configuration and the refusal checks, overlap and lock, timeouts, deflate, upload timing A/B per build and board with deflate and raw, on-device inflate benchmark, OTA automations on two platforms
- [`rtsp_server.md`](rtsp_server.md): camera streaming on the Waveshare ESP32-S3-ETH
- [`scheduler.md`](scheduler.md): loop wake A/B, interval startup delay and control from lambdas
- [`wifi.md`](wifi.md): reboot_timeout with a fallback AP, DHCP lease cache, Easy Connect (DPP), WISP bridge with an ethernet upstream, reboots against a WPA2/WPA3 PMF access point, WISP and ALWAYS_ON AP switching, AP restarts with WPA3 mixed and LIGHT power save
- [`zephyr.md`](zephyr.md): preferences on the nRF52840 dongle
- [`zephyr_coredump.md`](zephyr_coredump.md): crash and decode on nRF52 boards
- [`zwave_proxy.md`](zwave_proxy.md): frame checksums on a live proxy
- [`globals.md`](globals.md): restoring globals with storage rtc and flash across every reset kind
- [`homeassistant.md`](homeassistant.md): light platform mirrored from a local Home Assistant, held reports versus later calls
- [`usb_audio.md`](usb_audio.md): USB audio speaker and microphone, the usb_host FIFO split, and Music Assistant playback through sendspin
- [`dfrobot_sen0395.md`](dfrobot_sen0395.md): SEN0395 mmWave command flow, module quirks and overlong lines through its error reply
- [`speaker.md`](speaker.md): media player pipeline tasks on the Connect AUX-2 with the `speaker` platform player
- [`web_server.md`](web_server.md): state SSE stress and the HTTPD task stack on web_server_idf, entity icon and device class readback
- [`uart_sim.md`](uart_sim.md): UART sensors simulated on a classic ESP32 with loopback and cross-wired UARTs
- [`gpio_expander.md`](gpio_expander.md): interrupt driven expander inputs, loop idle checks, testing a device out of reach
- [`led_strip.md`](led_strip.md): forcing a failed strip allocation, RMT symbols per chip, loop cost per strip
- [`remote_base.md`](remote_base.md): remote_receiver protocol decoding through a one-pin loopback on ESP32 and ESP8266, no IR hardware
- [`xiaomi_ble.md`](xiaomi_ble.md): Xiaomi, ATC and PVVX parsing by injecting scan results into the ESP32 tracker or advertising them from a second ESP32, real AES-CCM frames, duplicate filter model
- [`ble_terminate.md`](ble_terminate.md): proxy disconnect against a peer with peripheral latency
- [`i2c_slave.md`](i2c_slave.md): fake I2C slave on a D1 mini, ESP32 hardware master as the control
- [`fake_i2c.md`](fake_i2c.md): fake I2C devices with no chip: in-firmware buses for many drivers, a Pico 2 W BME280/BMP280 target, a DS2482-800 simulator, NACK injection for vl53l0x
- [`gpio_hold.md`](gpio_hold.md): GPIO hold through the ISR pin API, pad read back with no wiring, per chip pin notes
- [`ble_event_inject.md`](ble_event_inject.md): made up GAP and GATTC events fed into the ESP32 BLE handlers from a button
- [`light_sleep_gpio_wake.md`](light_sleep_gpio_wake.md): GPIO wakeup from auto light sleep with an LEDC wave that keeps running in sleep, and with an external wave and peripheral power-down on the ESP32-P4
- [`manual_ip_and_web_handlers.md`](manual_ip_and_web_handlers.md): station, AP and Ethernet manual IP, web handlers added late and web OTA with web_server, captive_portal or prometheus
- [`ble_server_security.md`](ble_server_security.md): esp32_ble_server pairing, encrypted and MITM characteristics, subscription before and after pairing, bond reuse per ESP-IDF version
- [`prometheus.md`](prometheus.md): date, time and datetime metrics in UTC and another time zone
- [`ble_adv_triggers.md`](ble_adv_triggers.md): BLE service and manufacturer data advertise triggers over the air on the ESP32 and BK72xx trackers, with a second ESP32 advertising known frames
- [`sun.md`](sun.md): on_sunrise and on_sunset by sweeping the observer's longitude, one full sweep in about 17 minutes
- [`sensor_filters.md`](sensor_filters.md): every sensor, binary_sensor and text_sensor filter shape fed a fixed input sequence, outputs compared bit for bit between builds
- [`led_driver_sniff.md`](led_driver_sniff.md): bit-banged LED drivers captured on the ESP32's own pads with SPI slaves through the GPIO matrix, no chips
- [`hub_input_sim.md`](hub_input_sim.md): matrix_keypad, wiegand, rdm6300 and key_collector driven from a D1 mini's own pins and UART loopback
- [`value_range.md`](value_range.md): sensor and number on_value_range with a fixed sequence and a restart
- [`ezo_fake_bus.md`](ezo_fake_bus.md): the ezo component against a fake I2C bus that answers like an EZO circuit
- [`twai_self.md`](twai_self.md): esp32_can on_frame with the controller hearing its own transmit pad, no transceiver
- [`buttons.md`](buttons.md): output button pad timing, homeassistant button through a raw API client with the wizard input, uart button on the UART0 loopback
- [`rs485_frame.md`](rs485_frame.md): DLE/STX/ETX bus on an ESP32-C3 with a cross-wired simulated bus device (frame cases, gate timing)
- [`hx711.md`](hx711.md): fake HX711 driven from the other core, every published value matched to the served conversion
- [`automation_trace.md`](automation_trace.md): automation codegen A/B, a marker trace of chains, conditions, loops and script modes over the API
- [`bluetooth_proxy.md`](bluetooth_proxy.md): held GATT link after a half-open API session, frozen Home Assistant stand-in
- [`crash_report.md`](crash_report.md): crash buttons, fetching the previous boot's crash report over the API, per-chip abort facts
- [`https_idf.md`](https_idf.md): web_server_idf HTTPS test component, listener switch, stalled TLS clients, bad key
- [`mdns.md`](mdns.md): legacy unicast mDNS dump, hammer and latency straight to the device
- [`sim_buses.md`](sim_buses.md): PCF8574/HD44780, MAX7219, SPI spy, packet_interface and SN74HC165 simulators, status LED probe, fake Home Assistant, cg.static_buffer probes
- [`sigma_delta.md`](sigma_delta.md): pulse density of the sigma_delta speaker pin read inside the ESP32-P4, rtttl note onset and release
- [`microphone_http_stream.md`](microphone_http_stream.md): raw PCM over HTTP with a counter fake mic or a USB mic, multi-reader client
- [`i2s_audio.md`](i2s_audio.md): I2S microphone fed by an I2S TX channel on the same S3 through three jumpers
- [`voice_assistant.md`](voice_assistant.md): late TTS start on the Voice PE with a slow announcement URL
## User defined action listing and execution without a client library (#18882)
`scripts/api/rawactions.py HOST list` speaks the plaintext native API over a raw socket and decodes every
`ListEntitiesServicesResponse` by field number (argument fields 1 name, 2 type, 3 description, 4 example, 5 optional,
6 default_value), so new wire fields show up without waiting for the client library. `rawactions.py HOST exec ACTION
k=v ...` sends `ExecuteServiceRequest` with `sint32` ints (field 5, not the legacy int32 field 2), fixed32 floats,
length delimited strings and `int[]` as repeated sint32 (field 7); `ALLOW_MISSING=1` leaves arguments out on purpose
(the device then drops the call). `templates/api-actions-dut.yaml` (D1 mini, three actions, optional variables with
defaults) and `templates/api-actions-strings-dut.yaml` (required strings) log what arrived. String arguments are
`StringRef`s that are not null terminated: copy them with `snprintf("%.*s", size, c_str())` into a buffer before
logging, and do not pass `%.*s` to `logger.log`, whose validator counts it as one pattern.
## store_yaml recovery over the native API (#16445)
`scripts/api/syrecover.py HOST KEY OUTDIR [ORIGDIR]` (KEY `-` for plaintext) builds the client library's
`APINoiseFrameHelper` around a stub `APIConnection` subclass (the compiled helper type checks the connection, and
`ConnectionParams` needs every field), sends Hello, Authentication (never answered without a password) and
`DeviceCapabilitiesRequest`, then `GetYamlRequest` (161), collects `GetYamlResponse` (162) frames by field number
(1 data, 2 done, 3 total_size, 4 encoding) until done, requests again, decompresses with `compression.zstd` (Python 3.14, or the `zstandard` package) and
unpacks the envelope with the tree's `unpack_envelope` (run with `PYTHONPATH` on that tree). Configs in
`templates/store_yaml/`: entry files in `boards/`, a package one directory above them with `!secret` and an inline
OTA password,
so the redaction and the `..` paths are exercised. Repetitive YAML compresses into one chunk even at 22 KB; a 6 KB
include of random base64 lines with `include_secrets: true` gives four chunks (1390 byte packets). Rough edge at
0bd3738430: `include_secrets: true` fails the build when a package in another directory uses `!secret` and has no
`secrets.yaml` beside it; a copy next to the package works around it.
## BLE MIDI client against a fake instrument (#18508)
No MIDI hardware needed: `templates/ble_midi/midiperi.yaml` turns a second ESP32 (an S3 with ethernet, so the
API stays reachable while BLE is busy) into a BLE MIDI peripheral with `esp32_ble_server`: the MIDI service
advertised, the MIDI characteristic with read, write, write without response and notify, an `on_write` that logs
the bytes, buttons that notify hand built packets (channel messages, two messages in one packet, running status,
SysEx in one packet, split over two, oversize, cut short, clock, aftertouch, orphan data bytes, a 50 packet burst)
and a restart button for the disconnect test. `midicli.yaml` is the component test config on a classic devkit with
`wifi:`, `api:`, the peripheral's BLE address in the `peri_mac` substitution, given on the command line with
`-s peri_mac <mac>` (the base MAC plus 2, printed by `esp32_ble` in `dump_config`), `on_sysex` and `on_message` lambdas that print the bytes, debug sensors and out of range send
buttons. Drive both over the plaintext API with `scripts/modem/mdrive.py HOST LOGFILE press:NAME wait:S` 3 s
apart and compare the peripheral's `TX` and `RX` lines with the client's decoded lines. Gotcha:
`ble_server.characteristic.set_value` latches the value of its first play and re-applies it before every notify,
so the peripheral script sets the value from a lambda (`id(midi_io).set_value(std::vector<uint8_t>(pkt));
id(midi_io).notify();`). At VERBOSE the client logs four lines per message, so a burst shows an
`esp32_ble took a long time` of a couple of seconds that is the serial port, not the parser.
When there is no chip and no wiring, subclass `i2c::I2CBus` in an include, implement `write_readv()` only, answer
the registers the driver reads and check each write. Point the component at it from an `on_boot` lambda that runs
before the component's `setup()`: `id(x)->set_i2c_bus(&fake::bus)` at priority 950, which is ahead of every I2C
component (hubs set up at IO, 900, sensors at DATA, 600). The configs here that use 700 to 1000 work for their
components; use 950 for a new one. Keep a real `i2c:` block in the config so validation passes. Bulk
loops in a button lambda must call `App.feed_wdt()` and keep their per-iteration logs below INFO: 2000 DEBUG lines at
115200 baud trip the task watchdog, which is a fault of the harness and not of the PR.
`templates/i2c_fake/` holds the fakes; each `*-common.yaml` or `*-test.yaml` is a classic ESP32 or ESP32-S3 config
that includes its `.hpp` (add `esphome: name:` for the `-common` ones):
-`ccsfake.hpp`, `ccs-common.yaml`: CCS811 at 0x5A (#21927). Answers setup and update reads and checks every
ENV_DATA write against the humidity and temperature the case set; "CCS Cases" runs edge values, "CCS Sweep" 2000
random ones and logs `SWEEP checked N bad M bad_reads R bad_writes W`. A read without a register, of a register
the fake does not model or of the wrong length gets a NACK and counts in `bad_reads`; a write other than the
driver's SW_RESET, APP_START, MEAS_MODE and BASELINE writes (each with its own length) gets a NACK and counts in
`bad_writes`. The CASES and SWEEP lines end in PASS only when something was checked and `bad`, `bad_reads` and
`bad_writes` stayed 0, and in FAIL otherwise.
-`eefake.hpp`, `ee-common.yaml`: HTE501 (0x40) and TEE501 (0x48) with CRC words; "Next" steps through raw
temperature words and logs the value a signed hundredths decode should give (#21930). A command other than
identification and measure, and a read longer than the reply, gets a NACK and counts as `bad`. The DONE line
200 ms later, after both reads, shows the running `bad` total, the measure replies each driver read and the
published values; it ends in PASS only while `bad` is 0, both drivers read a reply this case and both published
the expected temperature (and HTE501 the fixed humidity).
-`scdfake.hpp`, `scd-common.yaml`: SCD4x at 0x62 and 0x63 with Sensirion CRC; logs every `0xE000` ambient pressure
write, and the CASE and SHOW lines count writes with a bad CRC apart from the good ones. A command the scd4x driver
does not send, one sent with the wrong length (2 bytes bare, 5 bytes with one CRC word), or any other command with a
bad CRC, gets a NACK and counts as `bad writes`. A read after a command the fake does not answer, or one longer than the command's
reply, gets a NACK and counts as `unknown reads`. The second instance has a fixed `ambient_pressure_compensation`
to cover the setup path (#21935). Each CASE line ends in PASS only when bad CRC, bad writes and unknown reads did
not move, the `0xE000` writes match `expect_e000()` (none, or one with the expected value) and, for a source case,
CO2 was published. `RANGE_CHECKED` in `scdfake.hpp` picks the rule: false truncates as dev does, true skips values
outside 700 to 1200 hPa and rounds as #21935 did. The SHOW line ends in PASS when nothing bad was counted, neither
instance failed and 0x63 got exactly one write of its fixed value.
-`sxfake.hpp`, `sx-common.yaml`: two SX1509 register files with reset values and the `7D 12/34` reset, recording
every write. Only a byte that lands on RegReset (0x7D) drives the reset, so a burst through it keeps the bytes for
0x7E and 0x7F. A read without a register, or a transfer past 0x7F, gets a NACK and `dump_log()` warns with the
count. Compare per address: hub setup order can legitimately move between hubs (#21897). Outputs of an Apply
platform are distinct template types, so a lambda list of them needs
`std::initializer_list<output::FloatOutput *>{...}`. Each case ends with an END line, PASS only while no write was
dropped and no transfer was bad; the writes themselves are judged by the diff.
-`msfake.hpp`: eight MS8607 pressure/temperature fakes at 0x76 and up and humidity fakes at 0x40 and up, with faults
per device (reset NACKs, bad PROM CRC4, zero D1 every Nth read, humidity CRC8 errors, out of range D2, full scale
humidity). It counts every command, so skipped conversions show in `report()`, and an ADC read with no conversion
pending answers 0 like the real chip and counts as `adc_no_conv`, and a humidity read with no measure pending gets
a NACK like the real chip in no-hold mode and counts as `read_no_measure` (a reset aborts either); each injected
fault counts when it is delivered (`inj_reset_nack` with `reset_nack_left`, `inj_zero_d1`, `inj_bad_crc`), so a
run that never reached a fault path shows 0 there; a PROM read, conversion or humidity measure
before that device's first reset counts as `no_reset`; a humidity read of any shape but a bare
3 byte read, and a pressure/temperature read that is not a 2 byte read of a PROM word (even addresses 0xA0 to 0xAC) or
a 3 byte ADC read, gets a NACK and counts as `bad_reads`; a bare read answers the last PROM or ADC pointer and never
repeats a reset or conversion; a command write the fake does not know, or one longer than a
byte, gets a NACK and counts as `bad_cmds`. Point both the component and its `humidity_i2c_id` device at the bus. That
driver logs no values, so give each sensor an INFO `on_value` log (#16487).
-`htfake.hpp`, `ht16k33-char-test.yaml`: HT16K33 chips at 0x70 to 0x77 (0x74 and 0x76 NACK) keeping the 16 byte
display RAM and a command trace, rendered back as text for the Adafruit 0.56" 7-segment backpack ("12:34",
decimal points, "!" for stray bytes); a read gets a NACK and counts as malformed. The test runs 64 checks (setup order, font, colon and dots, chained chips,
printf/strftime/clock, dimming, blink, standby, NACK) and ends with `ALL DONE pass=N fail=M` and PASS or FAIL (#8520). It found a
crash (null glyph table) when one display had `add_characters` and another did not.
-`ssfake.hpp`, `seesaw-test.yaml`: Adafruit seesaw at 0x36 (ATtiny817), 0x49 (SAMD09) and 0x4A (NACK): status,
GPIO bulk with the pull latch, ADC, touch, encoder and NeoPixel with chunk offsets, and the write to read gap per
module. A read it does not model, a write without a register and a write to a module or register it does not model
(or with the wrong data length) get a NACK and count in `bad`. A 100 ms interval step machine drives it and ends with `ALL DONE` and PASS or FAIL (#13538).
-`bldcfake.hpp` (three M5Unit-BLDC units at 0x65 to 0x67 with simple motor physics, one that never spins down),
`gtfake.hpp` (GT911 at 0x5D: switches, max values, touch points and key byte) and `sdpfake.hpp` (SDP3x/SDP8xx at
0x21 to 0x26: product IDs, an unknown product, a failing read every third time, a failing ID read, stop refused
while idle) have no config yet; wire them the same way. Each fake sets up its own faults in its constructor, so
there is no init step to call. Each NACKs a read it does not model or one of the wrong
length, and a write or command it does not know, and counts them as `bad_reads` and `bad_writes` (`bad_cmds` for
the SDP) in `report()`; check those are 0. The SDP report also counts each injected ID read NACK at 0x24 as
`id_fail`, so 0 there means the driver never reached that fault.
Check the PR head again just before posting results: a head that moves during a run means a retest.
## Pico 2 W as an I2C target (#22236)
`scripts/pico_i2c_target/`: `picofake.hpp` is a raw RP2350 I2C0 interrupt handler on GP4 (SDA) and GP5 (SCL) at 0x76 that
answers one byte per read request from a 256 byte register file, so any read length is byte exact. `picofake.yaml`
plays a BME280 (`chip_id` 0x60) or BMP280 (0x58) with the datasheet calibration; every 15 s it switches between the
datasheet example readings and the 0x80000 "no measurement" values, and logs `STAT` counters every 5 s. `bad`
counts reads and writes of registers the drivers do not use (soft reset only with 0xB6, read pointer wrapping past
0xFF). The DUT is
an ESP32-C3 (`dut-common.yaml`, GPIO6/GPIO7) with two sensors on the one fake (`dut-bme280.yaml`,
`dut-bmp280.yaml`), the second with oversampling NONE. `run.sh bme280|bmp280 LABEL` flashes the DUT, starts the
fake capture and waits for a `STAT` line as the baseline, then resets and captures the DUT and prints the distinct
values per sensor. The fake capture runs on until a `STAT` line arrives after the DUT capture ends, so that line
covers the whole run. It prints PASS when every sensor gave a numeric state, that `STAT` line came, and the fake's
`abrt` (read replies the controller aborted) and `bad` counts did not move from the baseline; diff the output of dev
and the PR.
The ESP32-S3 I2C slave v2 driver cannot emulate a register device: answering from a task and straight from the ISR
both gave shifted multi-byte reads. Use the Pico.
## DS2482-800 simulator (ds248x)
`scripts/ds248x_sim/`: `dssim.yaml` with `sim.hpp` turns a Pico 2 W into a DS2482-800 at 0x18 (GP4 SDA, GP5 SCL)
with four DS18B20s: channel 0 at 21.5 °C, channel 1 at 33.0 °C, channel 2 at -5.25 and 40.0 °C. The DUT is an
ESP32-C3 (`dspd.yaml`, GPIO4/GPIO5 plus GND); the internal pull-ups are enough at 50 kHz. Faults are one character
on the Pico's USB console: `c` NACKs the read pointer write after the next channel select, `z` NACKs the 9 reads
after the next 0xBE, `b` and `k` stick busy with a failed reset or config write, `x` and `X` make it dead and alive.
`SC=<serial_capture.py> python3 drive.py TAG SIM_PORT DUT_PORT [c,cd,z,b,k,dead]` (default `c,z,b,k,dead`; `cd`
runs the `c` fault densely, 28 times about 3 s apart) resets the DUT per phase and prints
readings, wrong values, log counts and the simulator's counters (`wrong_ch` counts ROM matches on the wrong
channel). The DUT is captured with `--ms` so the log can be split at the last fault: the recovery window starts
`SETTLE` (3) seconds after it. A phase fails when any of the four sensors got no correct reading in the recovery
window, the DUT logged a CRC, held low, busy timeout or no response line in it, a temperature is wrong, a phase other
than `dead` logged `recovery failed`, `DS248x hub failed` or a component marked failed, a ds248x warning is still
set at the end of the phase, the DUT capture
failed (stderr in `TAG-<phase>.err`), a fault moved its injection counter (counted when the DUT meets the
fault, not when it is armed) by other than the number of faults sent (`arm_lost` counts arms that landed while one
of the same fault was still pending and merged into it), a NACK fault never moved `nacked`, a write delivered data while a NACK was armed
(`nack_miss`, also counted when the write is followed by a repeated start into a read; the NACK never reached the wire and stays pending), the DUT sent an address-only write (`empty_wr`), a write was cut off by a repeated start into another write and
never processed (`dropped`),
a fault is still pending (`pend=1`) at the end of the
phase, the simulator counted a config write with a wrong complement (`cfg_bad`) or an unknown, short or overlong
command (`bad_cmd`), a counter is missing from the simulator's CNT line, the simulator aborted a read reply
(`tx_abrt`, not gated in the `dead` phase, which switches the controller off), a phase other than `b` and `k` (which
reset the chip under the driver) counted `wrong_ch` or a 1-Wire command without the active pull-up bit (`ops_no_apu`),
the simulator never logged `DEAD` for the `dead`
phase, or the DUT log of the `dead` phase has no sign of the outage (`did not respond`, a ds248x warning,
`DS248x hub failed` or a component marked failed). It ends with PASS or `FAIL:` and the failed phases. The whole
phase CRC, held low, busy timeout and no response counts are printed for information: the faults cause them.
- Sensors polled at the same interval go through the channels in blocks, so a stale channel cache never shows.
Poll them at different intervals, as `dspd.yaml` does (1, 1.3, 1.7 and 2.3 s).
- The NACKs use the RP2350 `slv_data_nack_only` register, set at STOP.
- A NACKed write and an address-only write both end with no data, so the simulator cannot tell them apart.
The ds248x driver never sends one, `dspd.yaml` sets `scan: false` and the simulator only sees STOPs after its
own address, so `empty_wr` (address-only writes outside a NACK window) stays 0; a nonzero count means `nacked`
may include writes that never carried a NACKed byte.
## I2C fault injection in software (vl53l0x)
`scripts/i2c_fault/`: `faultbus.h` wraps the real `I2CBus` and returns NACK for chosen transfers to one address. An
`on_boot` priority 800 lambda swaps the sensor's bus (`id(d)->set_i2c_bus(&fb)`) before its setup, and an interval
arms the faults. `vlpf.yaml` runs a VL53L0X on an ESP32-S3 (GPIO8/GPIO9) through quiet, fault A (the next
measurement-in-progress read fails), quiet, fault B (the next measurement start fails) and quiet phases over 150 s,
with a model ID probe every 10 s to tell a driver stall from a dead chip. `run.sh LABEL` builds, flashes, captures
and runs `an.py`, which counts readings per `PHASE` line. Each `PHASE` line clears a fault that was armed and has
not fired yet and logs the fired totals so far, so `an.py` counts the fires of each phase. It fails when the phases are not quiet, faultA, quiet,
faultB, quiet (a short capture or a restart), the first quiet phase has no readings, a later quiet phase reads less
than half as often as the first, a later quiet phase reads NaN (`nan` or `-nan`) or logs the driver's "update called before prior
reading" warning when the first did not (a target out of range reads NaN in every phase), a quiet phase has a failed
model ID probe, one that did not return 0xEE or no successful probe at all,
anything crashed or rebooted, or a fault did not fire in its own phase or fired in another one.
Prefer this to glitching the wire. Holding SDA low by rerouting the pin (`esp_rom_gpio_connect_out_signal` to
`SIG_GPIO_OUT_IDX` and back to `I2CEXT0_SDA_OUT_IDX`) wedges the ESP-IDF controller, which then needs
`i2c_master_bus_reset()` after each glitch, and glitches timed onto writes left the VL53L0X answering 0x00 until a
## Restoring globals with storage: rtc and flash (#13954)
`templates/globals-storage-dut.yaml` is the shared package: five restoring globals (rtc int, flash int, default int,
rtc `std::string`, rtc `uint8_t[2]`), bumped and logged with the reset reason on every boot, saved and synced after
2 s, odd boots `App.safe_reboot()`. Platform files add the board (`packages: dut: !include globals-storage-dut.yaml`), the `debug` reset-reason text sensor and, on the
ESP32, `deep_sleep: run_duration: 12s, sleep_duration: 8s` (even boots sleep). No network needed: flash by USB,
`serial_capture.py PORT 110 OUT --reset` gives the reset pulse plus a run of soft restarts and sleeps; the ESP32's
EN pulse is a `POWERON_RESET` that empties RTC memory, the D1 mini's RST pulse (`rst cause:2`) keeps it, so the
ESP8266 needs a real power cut by hand; arm long `--released` captures on both ports first (the capture
reopens the port when it comes back, the very first boot lines are still lost, read the next boot's restored
values). Unchanged-build check: the config with every `storage:` removed, built on the PR and on the dev merge
commit under the PR's own commits (not the merge base, which is older dev), `cmp -l` plus an `nm -S` symbol diff;
only the template names differ. Results: https://github.com/esphome/esphome/pull/13954#issuecomment-6254794712
A report that arrives during a transition is held and applied when the transition ends, unless a call came after
it. `scripts/homeassistant_light/heldcase.py DEV_IP KEY BULB_IP same|many [BULB_KEY]` (a key of `-` is plaintext) keeps one API connection to the mirror
device and one to the bulb so the steps land inside a 3 s fade: press "A 50 fade 3s", 0.7 s later set the bulb to
90% directly (the report is held), 1.4 s in press the same fade again (or "A many calls", 16 calls in one loop
pass). Pass: the mirror and the bulb end at the second call's target. Only `same` reproduces the bug: a build
without the fix logs `Applying the report that arrived during the transition` and the bulb stays at 90%. `many`
ends at 100% on both builds because the last call's target differs from the held report, so it is a regression
check only, not a reproduction. Separate button presses are too slow (about 1.5 s each) to land inside the fade.
The mirrors are not API entities, so read their state from the "Log state" button's line. Results: https://github.com/esphome/esphome/pull/21144#issuecomment-6278257695
## Update manifest URL changed at run time (#21373)
`scripts/http_request/manifest_server.py PORT [DELAY]` serves `stable.json`, `beta.json` and `slow.json` (answered
after DELAY s) with both chip families and logs every request. `templates/http_request_update/upd-common.yaml`
(secret `manifest_base` is the server URL) has the update entity on the stable manifest, buttons for every
`set_source_url` shape (`std::string` local, literal, temporary, `.c_str()` of a local, the current URL back in),
`Check Then Swap` (ESP32 task path: check the slow manifest and replace the URL in the same lambda), `Arm Swap In
Check` plus a `logger: on_message` lambda that replaces the URL from inside the inline check (ESP8266; the log line
from inside `on_message` is swallowed, judge by the next check's result), and a `Heap` button. `upd-base-common.yaml`
keeps only the shapes that compile on a tree with the bare pointer setter (for the base build, copy a board file
and point its `common` include at `../upd-base-common.yaml` with a different `name:`); `boards/upd8266.yaml` and `boards/upd32.yaml` are the
per-board files, one directory below the package their `../` include points at. Drive with `scripts/http_request/updrive.py HOST KEY LOGFILE press:NAME wait:S`, which prints every
update state (latest version, summary). The ESP8266 Arduino client gets `Error reading manifest: -1` on roughly a
quarter of the fetches from this Python server on every build, so count checks against the base before blaming a
PR. `http_request` on the ESP8266 needs `verify_ssl: false` even for plain http URLs.
#22185 adds an HTTPS developer API to `web_server_idf` with no YAML option, so `scripts/https_idf/` brings a test
component: `ext/httpstest` (`httpstest.hpp`) serves `/hw` (a status line with mode, request count and free heap),
`/hw/big` (16 KB) and `/hw/switch/{http,https,https8443,bad}`. A switch runs on the main loop through `defer()`, or
3 s later with a `/later` suffix (a stalled TLS client holds the server task, so its switch is armed first), and
calls `set_ssl_credentials(cert, key, port)`: plain HTTP on 80, HTTPS on 443 or 8443, or a key that does not match
the certificate. Its schema uses `web_server_idf.consume_ssl_sockets`. The run-time switch is the follow-up API; on a
tree that only has #22185, drop the switch paths and set the credentials once before `init()`.
Copy the directory into a scratch directory with a `secrets.yaml`, then `run.sh LABEL` (env `TREE`, `PORT`, `SC`,
`PY`): it runs `make_certs.sh` (a self-signed P-256 certificate, its key and an unrelated key in
`ext/httpstest/certs.h`; test material, never commit it), builds `https-dut.yaml` (ESP32-C3, logs on USB), flashes,
takes the address from the serial log and runs:
-`t1.sh HOST`: one GET with the TLS time, 20 sequential, 6 parallel three times and the 16 KB body. The fixture has
no event source, so the 501 that #22185 sends for an event stream on HTTPS is not covered here.
-`stall.py HOST`: arms a switch to HTTP and to 8443, opens two idle TCP clients (no ClientHello) on 443 before it
runs, then switches back; every switch must be accepted and work, and the idle clients must be closed.
-`t2.sh HOST`: HTTP and back, an armed switch to 8443 with a client open that sends a handshake record header and
never the rest, 60 requests that must all get a 200 for a heap check (compare the `/hw` heap before and after),
then the bad key. The server must log the error and stay off, without falling back to plain HTTP; reset the board
afterwards.
The runner fails on any unexpected status, a missing key error for the bad key, a reset or a task watchdog. `https-soak.yaml` adds a switch between 443
and 8443 every 20 s for a long run with clients attached. Each HTTPS connection takes about 26 KB of internal RAM
(no PSRAM on the C3), so watch the `ALIVE heap=... min=...` lines.
Each reader prints its HTTP status, format headers, sample count and rate, the longest gap between reads and, with
`--pattern`, every break in the counter. `--stall K` makes reader K stop reading after 2 s while keeping its socket
open, then checks that the device closed it. `--expect-reject R` marks the last R readers as past `max_clients`;
they must get a non-200 status or a close or reset before any samples, and a timeout or a served reader fails.
The script exits 1 when the listing is not a 200 naming the stream, the stalled reader was not dropped, a reader past
the limit was not refused, or any other reader got no 200, no samples, an early EOF, a read error or a broken counter.
Cases worth running on each head: one reader with `--pattern` (rate must match the sample rate, 0 breaks), the
`max_clients` limit plus one with `--expect-reject 1` (note what the extra one gets; the others must be unaffected), a stalled reader among
three (the device should drop the stalled one only), the "Mic stall" button during a read (a gap of about 6 s, then
the counter continues), mic start and stop while readers are attached, and the heap line after many connect and
disconnect cycles. On the USB microphone, check the rate and `rms` instead of the pattern with `--rate 16000 --min-rms 1`, and add `--max-gap-ms` to fail on long gaps; these set the exit code.
A local mosquitto with `allow_anonymous true`, `log_type all` and `log_dest file`: the broker log gives
each subscription's QoS, the device address (`New client connected from <ip> as <name>-<mac>`) and exact
subscribe counts. Kill and restart mosquitto to exercise `on_disconnect` (reason 0) and resubscription.
Keep `reboot_timeout: 0s` on `api:` and `mqtt:`. The ESP-IDF outbox swings free heap about 2 kB right
after bulk subscribe and unsubscribe; read the heap seconds later over several rounds before calling a
leak. Spawned subscriptions need about 2.5 s to reach the broker before the next publish.
`esphome logs --device MQTT` only uses the MQTT transport when the config has no `api:`. Driver pattern: a
paho script publishes and polls the serial capture file.
## WebSocket transports on ESP-IDF (#16056)
One build per transport from `templates/mqtt-transport-dut.yaml`, with `scripts/mqtt/mosquitto-ws.conf` as the
local broker (mosquitto 2.x from brew has `websockets` support; `log_type all` prints `New connection from <ip>:<port>
on port <listener>`, which proves which listener each client used) and `scripts/mqtt/make_test_ca.sh BROKER_IP` for
the TLS listener's certificate, both run in the scratch directory so keys, logs and outputs stay out of the checkout. `scripts/mqtt/cases.sh NAME KEY BROKER_HOST SUB_PORT CAPTURE [DEV_HOST] [restart]` subscribes
to `NAME/#`, sends a switch command and an `on_message` topic, presses the 3000 byte publish button over the API,
restarts the local broker when the last argument is `restart` (the PID in `mosquitto.pid`, written when the broker
was started with `mosquitto -c mosquitto-ws.conf & echo $! > mosquitto.pid` in the working directory next to the
certificates), checks every listener in the broker config answers again, and prints the broker's message counts with the device log. `mosquitto_sub` cannot speak
WebSocket, so it subscribes on the plain listener of the same broker. Cases: `tcp` on 1883, `ws` on 9001, `wss` on
9002 with `certificate_authority` (the certificate names an address, so `skip_cert_cn_check: true`), `wss` to
test.mosquitto.org on 8081 with no CA for the certificate bundle, and two negatives that must never connect: a
different CA (`mbedtls_ssl_handshake returned -0x2700`) and the self-signed server through the bundle
(`esp-x509-crt-bundle: Failed to verify certificate`, `-0x3000`). `CONFIG_MQTT_TRANSPORT_WEBSOCKET` is on by
default in the generated sdkconfig. The TLS session costs about 29 kB of internal heap while connected; the public
broker carries other devices' `esphome/discover` traffic, which is noise. Results:
`scripts/mqtt/mac_suffix/run.sh X BUILD` compiles (from `TREE`) and flashes variant X (a defaults, b custom `topic_prefix`, c explicit topics,
d no suffix) of BUILD (`pr` or `base`) on an ESP32 and a D1 mini, records the broker with
`mosquitto_sub -F '@H:@M:@S r=%r %t %p'`, presses Restart through the `cmd_t` found in the discovery payloads, then
holds both chips in the bootloader for 40 s (`esptool --after no-reset chip-id`) so the broker sends the will; a
plain reset only gives "session taken over" and no will. The two boards' configs live in separate folders because
variants a to c share a `name` (the MAC suffix tells them apart), and the same name in one folder shares a build
directory. A D1 mini CH340 can garble the serial capture after an RTS reset, so read its topics from the broker.
`esphome logs --device MQTT` on a current tree needs paho-mqtt 2.x; an older venv with 1.6.1 fails, so use a venv
made from the tree's requirements.txt. A Restart press never delivered the shutdown message on any build tested.
## Shutdown message delivery (#21566)
`scripts/mqtt/shutdown/`: `sd-common.yaml` sets birth/will/shutdown payloads `hello`/`dead`/`bye` so the shutdown
message and the will cannot be confused, and takes `restart`, `disable` and `sleep` on `<name>/cmd`.
`sdrun.sh V N` (`TREE`, `P32` and `P66` in the environment) compiles and flashes the ESP32 and D1 mini, restarts each N times, disables MQTT once and deep sleeps the ESP32 three
times (never the D1 mini: without the GPIO16 to RST wire it never wakes); `match.py OUT` counts a `bye` within 5 s of
each command. Dev: ESP32 0/10 restarts, D1 mini 1 to 3/10. Put the broker address (`broker_ip`) in the secrets.yaml next to
the included package, and quote `"${name}/status"` inside flow mappings.
## MQTT discovery against a throwaway Home Assistant (#21956)
Run a Home Assistant dev install with its own config directory (`hass -c <scratch>/ha`) holding `http:
server_port: 8129`, `api:`, `config:`, `onboarding:` and MQTT debug logging, plus a local mosquitto on 1883. In
`scripts/mqtt/ha_discovery/`, `onboard.py DIR` creates a generated test user (the credentials stay in DIR) and a
token, `flow.py TOKENFILE` adds the MQTT entry (the `protocol` and `other_settings: {set_client_cert: false,
set_ca_cert: "off", transport: tcp}` fields are required), and `hadrive.py TOKENFILE NAME` reads the device's time,
date and datetime entities, calls `set_value` on each and reads again; it exits 1 on a missing entity, a failed
call or a state that does not read back the value that was set. The three share the REST calls in `ha_rest.py`. `dt-common.yaml` is the device side (classic ESP32, three
template datetime entities, `${broker}`).
- The HA dev branch keeps a YAML port change as a pending HTTP config and reverts it after 5 minutes. Before
starting HA again, set `data.stable.server_port` to 8129 and `data.pending` to null in `.storage/http`.
- ESPHome MQTT unique ids are name based: clear the previous build's retained discovery (`mosquitto_pub -r -n` on
each config topic) and expect the entity ids of the first build.
## mqtt.disable and broker drops on ESP-IDF (#22230)
`scripts/mqtt/disable/disable-common.yaml` disables MQTT for 30 s every 60 s (every third cycle starts with fast
toggles, a disable while connecting), publishes a tick every 2 s and logs `EV CONNECT`, `EV DISCONNECT` and
`scripts/mqtt/retained_burst/`: `burst-topics.yaml` (a package) subscribes 25 `on_message` topics plus 15 switch
command topics, `burst-common.yaml` logs the counts 15 s after the connect, and `seed.sh BROKER_PORT` leaves all 40 retained on a local mosquitto (`seed.sh BROKER_PORT clear` removes them).
`run.sh LABEL BOOTS` (env `TREE`, `PORT`, `SC`, `PY`, `BROKER`, `BROKER_PORT`, default board an ESP32-C3) resets the
board BOOTS times and prints `RESULT got=x/25 switches_on=y/15` and any `Dropped` lines per boot; it passes only with
40 of 40 on every boot. Before #22200 the C3 dropped exactly 49 events (80 events minus 31 slots): the main loop
drains nothing during the burst.
## Forced reconnect mid-burst (#22248)
`scripts/mqtt/reconnect/`: `kick-common.yaml` includes the same `burst-topics.yaml` and counts per connection with
a `mode: restart` script on `on_connect`, logging `RESULT` 6 s after each connect. It and `run.sh` use files from
`../retained_burst/`, so copy the whole `mqtt/` directory to the scratch directory. `kick.py BROKER_LOG BROKER_PORT
PREFIX CYCLES` tails the broker log and takes over the device's client id 0 to 400 ms after every other connect
(mosquitto drops it with "session taken over"), then leaves the next connection alone for 8 s and ends it. It
starts by kicking the connection that is already up and skips its own connections (from 127.0.0.1 with the same id).
`run.sh LABEL CYCLES` (env as above plus `BROKER_LOG`) seeds, flashes, waits for the first connect and runs the
kicker. The device counts its first connection as CONN 1, so the kicked ones are the even CONN numbers and the
left-alone ones the odd numbers from 3 up; kick.py prints each left-alone connection with its time and CONN number,
and `judge.py` pairs those with the RESULT lines by time and passes when every one has `got=25/25`. A reset after
the first connect fails the run. Retained switch commands fire `on_turn_on` only on the first connection, so
case list over aioesphomeapi and judges each from the serial capture. ESP-IDF facts: a stalled server
ends with `Error downloading image: -28679` (EAGAIN) after the transport timeout, an early close with
`-1`, both before and after the PR; a body without Content-Length and without chunking reads as complete
(Size: 0) and finishes. A build without the fix hangs on the truncated, oversized header, empty, chunked PNG
and cut chunked cases and every later update logs `Image already being updated.`, so a baseline run needs
a reboot per group, the driver's 45 s cap per case, and an outer `timeout` that covers the whole list.
A case expected as `any` passes on `finished` or `error`, never on no outcome.
## PNG decode time limit (#21364, #21391)
The same server also serves any plain file in its www directory by name (404 for a missing file or a path
outside it). `scripts/online_image/make_big_pngs.py <www>` (after `make_images.py`) writes a 320x240 gradient (618 B, also as `ok.png` so the misbehaviour cases carry a
real multi-row PNG), a flat 640x480 PNG (1.6 KB for 307200 pixels), 231 KB of noise and a JPEG. The device is
`templates/online-image-dut.yaml` with `type: BINARY` on the main image (a 640x480 RGB565 does not fit a classic
ESP32 without PSRAM) and `debug` loop time and heap sensors at 2 s; the 256 byte buffer image stays RGB565, so do
not point big images at it (out of memory on every build). `scripts/online_image/pngdrive.py HOST KEY LOGFILE
BASEURL LABEL` (default case set `png`; `CASES=short` for the six case list) calls the actions, waits for the
finished or error line and reports the case time, the highest loop time and the lowest free heap. The per pass `online_image.image took a long time` values come from
the serial capture and only print on a new maximum, so read the first fetch of each image. The pass that decodes a
whole buffer is the decoder cost (before the change 510 ms gradient and 1980 ms flat, after it 77 and 140 ms); for
the noisy PNG the longest pass is the blocking 64 KB read on both builds (0.7 to 3.7 s). Alternate PR, base, PR
for a timing claim.
For a replacement PR, compare against the parent PR head, not dev. `CASES=noise` fetches the noisy PNG three times
(run it right after a boot, alternating builds); `CASES=end` runs flat and gradient as plain, `noclen_`, `chunked_`,
`half_` and `stall_` through the prefixed server cases below.
## JPEG decoded as it downloads (#21342)
`oi_server.py` serves prefixed cases on any file: `half_<file>` (first half, matching Content-Length),
`noclen_<file>` (close delimited), `chunked_<file>` (1 KB chunks), `stall_<file>` (a third, then the socket held
# Simulated chips on the bus, fake Home Assistant, static buffers
`scripts/sim_buses/` and `scripts/static_buffer/`. Each header is added with `esphome: includes:` and wired in an
`on_boot` lambda, so no chip is needed. The wrappers take the real bus or SPI delegate and pass through every
address they do not simulate.
-`lcdsim.hpp`: `LcdSimBus`, an `i2c::I2CBus` wrapper that plays PCF8574 backpacks (P0 RS, P2 EN, P4 to P7 D4 to D7)
driving HD44780 controllers. It decodes the EN-strobed nibbles and rebuilds the rows written after each 0x80 and
0xC0 address command. `add()` up to three addresses, hand it to each `lcd_pcf8574` with `set_i2c_bus()` at
`on_boot` priority 800, and call `report()` from an interval. `lcd_an.py SERIAL` checks the rows against the
patterns its docstring lists (0x27 20x4, 0x26 16x2, 0x25 64x4), and fails on a display whose frame count does not
grow between its first and last report, on a segment longer than its rows or on
data past the 128 byte segment buffer, which the report counts as `overflow`, on a report from an address it does
not expect, or on any read (`reads`; the driver never reads, so the simulator NACKs them). A 64x4 display hangs the
code before #22331 in
setup.
- HT16K33: use `templates/i2c_fake/htfake.hpp` with its test runner (see [fake_i2c.md](fake_i2c.md)).
-`mxsim.hpp`: `MxSniff`, an `spi::SPIDelegate` in front of the real one that keeps what each MAX7219 in a chain
latches. Swap it in at `on_boot` priority 399 through `DelegatePeek::of(id(x))`:
`auto &d = DelegatePeek::of(id(x)); d = new MxSniff(d, chips);`. `mx_an.py SERIAL CHAIN...`
summarizes the reports, CHAIN being each name passed to `report()`, and fails on a named chain with no report line,
on malformed transactions (byte count not two per chip, or a partial byte from `write()` or
`write_cmd_addr_data()`, which are recorded and forwarded too), on a named chain with fewer than two report lines,
or a chain whose transaction count stays 0 or stops growing, and prints PASS otherwise.
-`spysim.hpp`: the same delegate trick (both headers carry `DelegatePeek`, so one config can include both), logging length and FNV-1a hash of every transaction over 1000 bytes (whole
frames, used for `pixoo`). `write()` and `write_cmd_addr_data()` are hashed and forwarded like `write_array()`;
a partial byte or a bus width other than 1 is hashed as whole bytes, so the SPY line carries `odd`, the count of
such transactions, and a dev vs PR hash match only holds when `odd` is 0 on both.
-`pisim.hpp`: drives two `packet_interface` stacks on cross-wired UARTs through `top()`, with payloads whose bytes
are a function of their index so every byte is checked, and the index and length each side sent (last 256) so a
truncated or split packet, or one whose slot a later index reused, counts as `rx_bad`. Send indices 0, 1, 2 and so
on from each side: a skipped index counts as `rx_lost` and an old or repeated one as `rx_bad`. `report(tag, true)` once both links are idle ends each
line in PASS only when that side received every packet the peer sent intact and the peer had no send refused;
`missing` is the rest, packets still in flight or lost at the tail. `packet_transport` drops packets carrying its own device
name, so a one-board test must use the layers directly.
-`hcsim.hpp`: a software SN74HC165 chain on ESP-IDF (data GPIO6, clock GPIO7, load GPIO10). A falling load edge
latches the pattern and each rising clock edge drives the next bit onto the data pad, which is in
`GPIO_MODE_INPUT_OUTPUT` so the component reads it from the same pin. Call `hcsim::report(label, loads0, clocks0)`
after each window with `hcsim::loads` and `hcsim::clocks` from its start; it logs `INVALID` instead of a result
when the GPIO or interrupt setup failed or the window saw no load or no clock edge, so a dead simulator cannot pass
as a wrong but steady pattern.
-`ledprobe.hpp`: samples a status LED (or any output) pin every 2 ms from an `esp_timer`, off the loop task, and
prints blink count, on time, period and the longest gap per window. It measures `status_led` while the loop is
blocked (#16407). Pass `ledprobe::n` from the window start as the mark; a window with more than 400 edges logs
`INVALID` and no result line instead of passing for a stalled LED. So does every window when the pin or timer
setup failed or the timer has not sampled for 100 ms, so a dead probe cannot read as a quiet LED.
-`fakeha.py HOST OUT_PREFIX`: an aioesphomeapi client standing in for Home Assistant for `homeassistant` select
mirrors. It answers state subscriptions from a table, pushes option and state changes (full, overfull, quoted,
unparseable), checks each step through a "Dump" button, captures the action calls, churns 600 option updates
while watching the heap, and checks the entity options after a reconnect. A step fails when either DUMP line is
missing, and a rejected update also needs the device's rejection log line, so an update that never arrived cannot
pass. Exits 1 on a failed step and prints PASS otherwise.
-`wizard_drive.py HOST`: finishes the device wizard through aioesphomeapi 46.28 (`device_wizard`,
`wizard_input_set`), answers the subscriptions the inputs create and checks the values reached the entities.
## Static buffers (`cg.static_buffer`)
`scripts/static_buffer/` holds two local external components and `static-buffer-dut.yaml`, which loads them:
-`sbtest` allocates three buffers with `cg.static_buffer` (37 `uint8_t`, 5 `uint32_t`, and `const char *` sized by
a C++ expression) and checks at setup that they are zeroed and aligned, then fills each completely; at
`dump_config()` it checks no other instance's fill reached them. Two instances prove the buffers are distinct.
Both `SBTEST` lines end in `PASS` or `FAIL`; the `dump_config()` one passes only when the setup checks passed too.
-`sbpsram` asks for `psram=True` and uses the pointer codegen hands it: the static buffer without PSRAM, or a
zeroed `psram_buffer<T>()` with a `psram:` block, which logs `external=1` (nullptr if the allocation failed).
It also gets a plain 8 word `uint32_t` static buffer, which must be non-null, aligned and zeroed at start and read
back its fill (`plain=1`). Its `dump_config()` line ends in `PASS` only when both readbacks match, both buffers
were zeroed at start and, with PSRAM, the PSRAM one is in external RAM.
## Gotchas
- Some "classic" devkits with a CP2102N carry an ESP32-PICO-D4. GPIO16 and GPIO17 are its flash pins, and an
ESP-IDF build that drives them hits the interrupt watchdog. Use `board: pico32` and GPIO9/GPIO10.
- An `addressable_light` display writes nothing until its light is turned on.
- Before calling a rebased PR's change trivial, diff the PR's own diff (head against its parent) old against new: a
move that looks like a merge only can carry real commits from below.
are too short to cover the due moment). Pass pattern: "Can't connect; rebooting" exactly 60 s after boot
with no station joined, no reboot for minutes while one is, then a reboot within a second of the last
disconnect. The BW15 restarts its AP on every STA retry and cannot be tested this way.
## DHCP lease cache (#17767)
Cycle 20 s awake, `deep_sleep.enter 30s`, wake, 20 s, `App.safe_reboot()`, logging `millis()` at the
`wifi.connected` condition. An RTS hardware reset must fall back to DHCP; a software reboot and a deep
sleep wake must apply the cache. `esphome run --no-logs` silently skipped the upload on one board; use
`upload` then `logs`.
## Wi-Fi Easy Connect (DPP) enrollee (#17934)
Only an Android phone configures: Settings, network details, Add device, Share Wi-Fi (the Wi-Fi list's
QR scanner makes the phone an enrollee). Render the URI with the `qrcode` module and open the PNG.
`esp32: framework: log_level: DEBUG` plus `sdkconfig_options: CONFIG_WPA_DEBUG_PRINT: y` prints the
supplicant state machine.
## WISP bridge with an ethernet upstream (#7147)
`templates/wisp-ethernet-dut.yaml` (ESP32-S3 with a W5500, `wifi: ap:` with `ap_mode: WISP` and no station) and
`templates/wisp-client.yaml` (D1 mini joining the bridge, `http_request` to a web server on the wired LAN and to
`http://example.com/` every 15 s, `time: sntp`). Secrets: `wisp_psk` and `lan_http` (serve a text file on the host
with `python -m http.server 8787 --bind 0.0.0.0` from a scratch directory). Judge by the client's `GET status 200`
lines, the web server log showing the device's ethernet address as the source (NAT), the client's DHCP info (DNS1 is
the LAN resolver) and `SNTP synced`. The device logs `Starting AP` and `DHCP DNS server: <resolver>` on its USB
console; flashing over the air reboots it for repeat boots. Before the AP address and DHCP server moved ahead of the
AP start (the fix in that PR), the first AP start failed on every boot with `esp_netif_dhcps_option failed: 20484`
(DHCP server already started by the AP start event) and `Failed to enable NAT`, and the bridge only came up on the
retry 90 s later; a client that cannot find the SSID for the first minute on an older build is that. With the fix
the bridge is up 6 s after boot. Any change to the ESP-IDF AP start should also rerun the captive portal hammer.
## Reboot against a WPA2/WPA3 PMF access point (#21590)
- AP: `scripts/wpa3_ap/wdap.yaml` on a classic devkit. ESPHome `wifi: ap:` only, then an on_boot lambda switches it
to `WIFI_AUTH_WPA2_WPA3_PSK`, PMF capable, and registers a WIFI_EVENT handler that logs `STA JOIN` and
`STA LEAVE ... reason N` (the include header pulls in `esp_wifi.h`). It disables Wi-Fi after 420 s.
- AP leave reason 3 means the station sent a deauth; reason 2 means the AP found a stale association when the
station came back; several reason 2 before one join are rejected reconnects.
- DUTs (`wdbk.yaml` BK7238, `wdcb.yaml` CB3S, both on `wd-common.yaml`) list the AP at priority 10 and the main
network at 0, and reboot 30 s after joining the AP and 150 s after joining the main network. So they move to the
AP while it is up and fall back to the main network (where they can be flashed over the air) once its window
ends. `run.sh BUILD` flashes the AP, does OTA for the CB3S and a serial flash for the BK7238, then captures one AP
window; `an.py` prints each station's join and leave sequence and fails on a rejected reconnect, an AP that did
not switch to WPA2/WPA3 (`AP CONFIG FAILED`), a DUT missing from the AP events (`DUT_MACS`) or a BK7238 that never
joined the AP.
- Gotchas: the LibreTiny build directory has two `.uf2` files, so flash `firmware.uf2`. The BK7238 serial flash
sometimes fails to link while the chip is in a reboot loop; retry it 3 or 4 times. OTA fails if the DUT reboots
mid-upload; retry.
- Finding: BDK 3.0.78 rejects about 2 reconnects per reboot when the station left without a deauth; BDK 3.0.33
does not.
## WISP and ALWAYS_ON AP rig (#20077)
`scripts/wisp_ap`: the upstream is an ESP32-S3 with Ethernet running a WISP bridge (`wisp-upstream.yaml`) with its AP on
10.40.0.1/24 and web buttons `ap off` and `ap on`. Its subnet must not be 192.168.4.0/24, the DUT AP's default, or
nothing routes. The DUT is a classic devkit (`dut-wisp.yaml` or `dut-always-on.yaml`); the client is a D1 mini on
the DUT's AP (`wispcli.yaml`) logging its DNS server, a name lookup, a GET by name, the AP itself and a LAN file by
IP. `run.sh DUT START SECONDS T:STATE...` switches the upstream AP at the given times and `sum.py` groups the client
results by phase. Capture the D1 mini without `--reset` (a CH340 floods garbage after an RTS reset). Clients keep
the captive portal's DNS answer for its TTL.
## AP restart with WPA3 mixed and LIGHT power save (#21531)
`scripts/wifi_restart`: the WISP upstream (`wisp-upstream-wpa3.yaml`) switched to WPA2/WPA3 mixed by a script that
runs again after every `ap on`, DUTs (`rebk.yaml`, `recb.yaml` on `re-common.yaml`) with a static IP on the bridge's
subnet, a fallback AP, `power_save_mode: LIGHT` and syslog to the host through the NAT. `run.sh BUILD` switches the
AP off and on three times; `an.py` fails on a reboot, a crash or a DUT that is not back on syslog within 120 s of an
AP start. After 19 minutes the DUTs run `wifi.configure` to the main network; that keeps the static IP, so the next
OTA needs the host to have an address on the bridge's subnet as well (an interface alias). The CB3S log is on UART1
(TX1), so its configs need `logger: hardware_uart: UART1`. Runners must `pkill` only their own ports; a global pkill
stops the other runs' captures.
## Channel and network switching with a bench AP (#22142)
`scripts/wifi_channel/`: `bench-ap.yaml` makes an ESP32-S3 an AP-only bench network on a fixed channel (11 by
default; pick one your normal network does not use), and `dut-common.yaml` gives the device under test that AP at
priority 10 plus your normal network, logs every `wifi_info` ssid and channel update (with the channel the component
reports and whether it is connected), and cycles `wifi.disable` and `wifi.enable` every 90 s. `run.sh LABEL` (env
`TREE`, `PORT`, `AP_PORT`, `SC`, `PY`, and `BENCH_CHANNEL` when the AP is not on 11) flashes the device and captures 420 s while it takes the AP down with
`esptool --port <AP port> --after no-reset chip_id` (the board stays in the bootloader) and back with
`--after hard-reset`. The channel sensor must follow every move between the two networks: the run passes when the
connected channel starts and ends on the bench channel with another channel in between.
- Dropping the AP under a connected device gives a `Beacon Timeout` link loss.
- The 90 s interval fires once right after boot.
- A `Disconnected ... Association Leave` right after `Connecting to` is the aborted attempt, not a missed connect.
if (esp32_ble_tracker::global_esp32_ble_tracker->scan_running()) {
ESP_LOGE("cpinject", "scan still running after 5 s, nothing injected");
return;
}
cpinject::run();
- esp32_ble_tracker.start_scan:
continuous:true
Loaded 100 of 8687 files, more files were not shown because too many files have changed in this diff.
Show more
Reference in new issue
Block a user
Blocking a user prevents them from interacting with repositories, such as opening or commenting on pull requests or issues. Learn more about blocking a user.