Compare commits

..
Author SHA1 Message Date
clydebarrow c691a71cc5 Merge branch 'pi05' into pi06 2026-10-25 18:06:18 +11:00
clydebarrow 07164fc8b1 Merge remote-tracking branch 'upstream/dev' into pi05
# Conflicts:
#	esphome/components/packet_interface/__init__.py
#	esphome/core/defines.h
2026-10-25 18:06:16 +11:00
clydebarrowandClaude Opus 5.5 923893f6df [packet_interface] Name a peer in the error for a link used with peers
The link may have only one peer, so the error no longer says several.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-23 13:39:40 +11:00
clydebarrowandClaude Opus 5.5 72f767cb4f [packet_interface] Log packets and fragments too short for their header
Every other drop in these layers is logged at verbose level.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-23 13:39:40 +11:00
clydebarrowandClaude Opus 5.5 b007bbb01c [packet_interface] Check at validation that fragments fit the reliability queue
- Work out each layer's packet size during validation, as the C++ does,
  and reject a fragmentation max_packet_size whose fragments do not all
  fit in the reliability queue at once, or need more than 128 fragments
- Fit the test peer's largest packet in its queue
- Reuse interface_layers() in final_validate_interface()
- Keep the earlier tests in place in test_packet_interface.py

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-23 13:39:39 +11:00
clydebarrowandClaude Opus 5.5 7ac9ba6daf [packet_interface] Use the shared max_packet_size constant in the fragmentation layer
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-23 13:39:39 +11:00
clydebarrowandClaude Opus 5.5 8b752525f5 [entity_transport] Find an addressing peer's layers for the timeout check
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-23 13:39:39 +11:00
clydebarrowandClaude Opus 5.5 07e1195ffa [packet_interface] Route broadcasts to the broadcast port, and keep a join
A broadcast from a device with its own port went to that port, whose
layers drop it, so a broadcast port could not reach reliable peers. It now
goes to the broadcast port when there is one.

Fragmentation allocates its buffers with RAMAllocator and fails setup when
they cannot be allocated. A fragment from another source no longer drops
the packet being joined.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-23 13:39:39 +11:00
clydebarrowandClaude Code 5ef072d03d [packet_interface] Take the addressing and fragmentation sizes at construction
AddressingPort derives max_packet_size from the link in its constructor,
and the fragmentation layer passes its configured size to the base
constructor.

Co-Authored-By: Claude Code <noreply@anthropic.com>
2026-10-23 13:39:39 +11:00
clydebarrowandClaude Opus 5.5 4d61784ba6 [packet_interface] Keep App alive across tests
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-23 13:39:39 +11:00
clydebarrowandClaude Opus 5.5 be4ea37f21 [packet_interface] Fix clang-tidy findings in tests
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-23 13:39:39 +11:00
clydebarrowandClaude Opus 5.5 9915cf6827 [packet_interface] Fixes from code review
- Reject an addressing destination or peer that is this device's own
  address, whose packets would be dropped as echoes
- Show 0 fragments, not SIZE_MAX, for a link with no room
- Test the check that applications use a peer's id

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-23 13:39:39 +11:00
clydebarrowandClaude Opus 5.5 beeeee07e8 [packet_interface] Use a local to_vector() in tests
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-23 13:39:21 +11:00
clydebarrowandClaude Opus 5.5 e9c386f57e [packet_interface] Leave no room, instead of underflowing, on a small link
As in the other layers, addressing ports and fragmentation get no room
when max_packet_size is smaller than their header, and fragmentation
fails its setup instead of dividing by zero.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-23 13:39:21 +11:00
clydebarrowandClaude Opus 5.5 01fbfeb16a [packet_interface] Add addressing and fragmentation layers
The addressing layer lets several devices share a channel such as an
RS485 bus or a radio, with 16-bit addresses. A link can carry several
peers, each with its own stack of layers and its own id. Layers that work
between two devices only are rejected with broadcast addressing. The
fragmentation layer splits packets larger than the layers below allow,
and joins them again.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-23 13:39:21 +11:00
pre-commit-ci-lite[bot] 369354a116 apply automatic formatting fixes 2026-10-23 13:39:03 +11:00
clydebarrowandClaude Opus 5.5 105d1bb088 [entity_transport] Warn when packets are refused for long, and check every complete dictionary
- Set a warning once the interface has accepted no packets for a minute,
  and log refused dictionary packets as data packets are
- Clear remote entities that a provider's dictionary does not name also
  when the provider goes back to a dictionary already seen

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-23 13:39:03 +11:00
clydebarrowandClaude Opus 5.5 1488af2b32 [entity_transport] Fixes from code review
- Check a data packet before asking for its dictionary, so a damaged
  packet does not make the provider resend everything
- When a provider's complete dictionary does not name a remote entity,
  warn and clear that entity's state, as a typo in provider or remote_id,
  or an entity the provider no longer sends, was otherwise silent

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-23 13:39:03 +11:00
clydebarrowandClaude Opus 5.5 74b8de8064 [entity_transport] Ask again for a dictionary with a lost part, and warn on dropped packets
A dictionary packet now carries its part index and a last-part flag. A dictionary counts as received, and is not
asked for again, only once its last part follows every other part in order. Until then, data under it sends a
request even when some remote entities are already known.

A refused request no longer starts the request interval. A short or malformed packet, or one with another format
version, is logged as a warning at most once a minute.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-23 13:39:03 +11:00
clydebarrowandClaude Opus 5.5 5393d192b4 [entity_transport] Reject a timeout on a packet interface with a session layer
With a session and a reliability layer, the update interval defaults to
never, and unchanged states are not sent again, so a receiver's timeout
cleared them on a working link. The session layer already clears the
states when the other device cannot be reached.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-23 13:39:03 +11:00
clydebarrowandClaude Opus 5.5 670934b3d8 [entity_transport] Tag each data record, and check whole packets before use
Each data record now starts with a 16-bit tag holding the entity's number
and type, and the form of its payload, so that more entity types can be
added and a receiver can skip records of types it does not know. A binary
sensor's state is in the tag; a sensor has a 4 byte state, or none.

Data and dictionary packets are checked in full before anything in them
is used, so a damaged or forged packet changes no state and teaches nothing.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-23 13:39:03 +11:00
clydebarrowandClaude Opus 5.5 f76dce46aa [entity_transport] Use the shared provider and remote id constants
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-23 13:39:03 +11:00
clydebarrowandClaude Opus 5.5 d44bccba3f [entity_transport] Ask only for dictionaries not yet received
A dictionary that names none of the remote entities was asked for again
every 5 seconds while any remote entity was unresolved, and each request
made its provider resend everything. A provider's new dictionary was not
asked for once every remote entity was resolved.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-23 13:39:03 +11:00
clydebarrowandClaude Opus 5.5 5b35282546 [entity_transport] Fail when the packet buffer cannot be allocated
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-23 13:39:03 +11:00
clydebarrowandClaude Code aa1d7fa26b [entity_transport] Set the test link's size at construction
The mock link takes max_packet_size in its constructor, so a test that
needs a different size builds a local link.

Co-Authored-By: Claude Code <noreply@anthropic.com>
2026-10-23 13:39:03 +11:00
clydebarrowandClaude Opus 5.5 3862504cdd [packet_interface] Keep App alive across tests
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-23 13:39:03 +11:00
clydebarrowandClaude Opus 5.5 ee3efdf15e [entity_transport] Brace multi-line if bodies
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-23 13:39:03 +11:00
clydebarrowandClaude Opus 5.5 db46ed33a2 [packet_interface] Fix clang-tidy findings in tests
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-23 13:39:03 +11:00
clydebarrowandClaude Opus 5.5 e09257a992 [packet_interface] Use a local to_vector() in tests
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-23 13:39:03 +11:00
clydebarrowandClaude Opus 5.5 1fa3dd37fe [entity_transport] Fixes from code review
- Compare received ids by length and bytes, so an id with an embedded NUL
  cannot read past the configured string
- Fail setup when max_packet_size cannot hold the device name and the
  longest entity id, instead of skipping the id at every resend
- Reject two entities of one type sent with the same id
- Avoid an unused variable without remote entities
- Use matching ids in the test configuration

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-23 13:39:03 +11:00
clydebarrowandClaude Opus 5.5 a7ca6a9608 [entity_transport] Add entity transport component
Sends sensor and binary sensor states to other devices over a packet
interface. A dictionary sent on connect, on request and at an interval
names the entities, so data packets refer to them by number. The update
interval and the handling of a lost peer follow the layers the interface
has, and remote states can time out.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-23 13:39:03 +11:00
clydeandClaude Opus 5.5 e046551825 [packet_interface] Stretch the XXTEA key text with PBKDF2
A single SHA-256 of the key text let a captured packet check about 2^30
guesses for every one that passes, so a weak key text fell quickly to
an offline search. Derive the key with PBKDF2-HMAC-SHA256 at 600,000
iterations instead; this runs at build time, so the device pays nothing.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-23 13:39:00 +11:00
pre-commit-ci-lite[bot] 18b5b7a24a apply automatic formatting fixes 2026-10-23 12:28:10 +10:00
clydebarrowandClaude Opus 5.5 179d20ba77 [packet_interface] Limit the session keep_alive to 1 h and timeout to 3 h
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-23 12:28:10 +10:00
clydebarrowandClaude Opus 5.5 1bc4779d96 [packet_interface] Fail the XXTEA layer when its buffers cannot be allocated
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-23 12:28:10 +10:00
clydebarrowandClaude Opus 5.5 366713ba05 [packet_interface] Check the layers in the link schema
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-23 12:28:10 +10:00
clydebarrowandClaude Opus 5.5 47b8aae1b5 [packet_interface] Require a session layer with the rolling code
The rolling code accepts a packet only once the peer has sent back this
device's boot number, so a device that only receives never gets a packet.
The session's keep-alives make sure packets go both ways.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-23 12:28:10 +10:00
pre-commit-ci-lite[bot] a9c155e4b4 apply automatic formatting fixes 2026-10-23 12:28:10 +10:00
clydebarrowandClaude Code 708e117a20 [packet_interface] Take the session and encryption sizes at construction
Both layers derive max_packet_size from the layer below in their
constructors instead of assigning it in the body.

Co-Authored-By: Claude Code <noreply@anthropic.com>
2026-10-23 12:28:10 +10:00
clydebarrowandClaude Opus 5.5 c6d1c63884 [packet_interface] Keep App alive across tests
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-23 12:28:10 +10:00
clydebarrowandClaude Opus 5.5 faef5a98bf [packet_interface] Fix clang-tidy findings in tests
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-23 12:28:10 +10:00
clydebarrowandClaude Opus 5.5 7c86318c1a [packet_interface] Use a local to_vector() in tests
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-23 12:28:10 +10:00
clydebarrowandClaude Opus 5.5 58ef6ef3d9 [packet_interface] Fixes from code review
- Fail setup, instead of underflowing, when max_packet_size leaves the
  encryption or session layer no room
- Warn when the boot number cannot be saved
- Warn once when the peer's packets carry an older boot number, which
  happens when its flash is erased
- Test the counter wrapping to a new boot number

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-23 12:28:10 +10:00
clydebarrowandClaude Opus 5.5 a6d2ae15a1 [packet_interface] Add session and XXTEA encryption layers
The session layer sends keep-alives and reports whether the peer can be
reached, including after it restarts or a reliability layer below loses
data. The encryption layer encrypts and checks each packet with XXTEA,
with an optional rolling code against replays. Components that a layer
needs, such as binary_sensor, are loaded only when it is configured.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-23 12:28:10 +10:00
clydebarrowandClaude Opus 5.5 63cfdf2b4d [packet_interface] Reject a layer type registered after the schemas are built
Such a type would be missing from the link schema, and selecting it would
fail with an unrelated error.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-23 12:28:10 +10:00
clydebarrowandClaude Opus 5.5 6411a8062f [packet_interface] Log a full queue during a link reset only once
While the peer is away the layers above keep sending, so the message
repeated about once a second until the link came up again.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-23 12:28:10 +10:00
clydebarrowandClaude Opus 5.5 c18ac5bac9 [packet_interface] Count the link reset tries outside the condition
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-23 12:28:10 +10:00
clydebarrowandClaude Opus 5.5 78396dd142 [packet_interface] Warn when the HDLC link cannot come up
A frame larger than max_packet_size, usually from a peer with a larger
setting, is warned about once. A link reset that the peer does not answer
is warned about after max_retries tries, and a full queue is logged at
debug level while the link is down.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-23 12:28:10 +10:00
clydebarrowandClaude Code 557e3c3e9e [packet_interface] Reject a duplicate layer type name
Every module in layers/ is imported automatically, so two modules
registering the same name silently replaced one with the other, in an
order decided by pkgutil. Registration now raises, so the mistake is
found when the configuration is loaded.

Co-Authored-By: Claude Code <noreply@anthropic.com>
2026-10-23 12:28:10 +10:00
clydebarrowandClaude Code 8fd05e93b4 [packet_interface] Answer a repeated SABM without resetting the link
A peer that has not heard the UA repeats its SABM, and cannot have sent a
numbered frame meanwhile. Treating that as a restart threw away queued
frames and reported the session lost for a single dropped UA. A SABM is
now a restart only if the peer has sent a numbered frame since the link
came up.

Co-Authored-By: Claude Code <noreply@anthropic.com>
2026-10-23 12:28:10 +10:00
clydebarrowandClaude Code f81c6847f3 [packet_interface] Take the reliability layer's size at construction
max_packet_size is derived from the layer below in the constructor
instead of being assigned in the body.

Co-Authored-By: Claude Code <noreply@anthropic.com>
2026-10-23 12:28:10 +10:00
clydebarrowandClaude Opus 5.5 d555d64300 [packet_interface] Fixes from code review
- Keep App alive across the HDLC tests, as the test main requires
- Run a layer's loop in the tests only while it has asked for it, so
  the loop's disable and request bookkeeping is tested, and wait for the
  retry timer in the tests of refused sends
- Report no send capacity while the layer has no send queue

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-23 12:28:10 +10:00
clydebarrowandClaude Opus 5.5 267e3d5d70 [packet_interface] Fix clang-tidy findings in tests
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-23 12:28:10 +10:00
clydebarrowandClaude Opus 5.5 2da760ea4f [packet_interface] Use a local to_vector() in tests
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-23 12:28:10 +10:00
clydebarrowandClaude Opus 5.5 0b34f811a4 [packet_interface] Fixes from code review
- Reset the link after max_retries even when it is 255, where the retry
  counter used to wrap
- Require a reliability timeout of at least 50 ms
- Drop received frames larger than the layer's max_packet_size
- Fail setup, instead of aborting, when the send queue cannot be
  allocated or max_packet_size leaves no room
- Test the ack delay timer and a refused REJ

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-23 12:28:10 +10:00
clydebarrowandClaude Opus 5.5 28833fe0f3 [packet_interface] Add layer registry and HDLC reliability layer
Layers stack on a link and are configured on its packet_interface entry.
Each layer type registers itself with its kind, so a new type needs only
a new module. The first is a reliability layer in the style of HDLC
asynchronous balanced mode, with go-back-N retransmission.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-23 12:28:10 +10:00
clydebarrowandClaude Code 929aac8fac [udp] Take the packet size from the copy buffer
The link derives max_packet_size from the span it is given, so a packet
larger than the copy buffer cannot be copied into it.

Co-Authored-By: Claude Code <noreply@anthropic.com>
2026-10-23 13:28:08 +11:00
clydebarrowandClaude Opus 5.5 11cc5bb008 [udp] Limit a split packet to the copy buffer
The copy buffer is sized by code generation to max_packet_size, but
set_max_packet_size() can raise the limit afterwards, and copying a split
packet would then overflow it. The link now takes the buffer as a span
and refuses a split packet larger than it.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-23 13:28:08 +11:00
clydebarrowandClaude Opus 5.5 2d6be6c27a [udp] Drop datagrams longer than the receive buffer
A read cuts a datagram longer than the buffer to fit, so a 600 byte
datagram arrived as a valid looking 508 byte packet. The buffer is now one
byte longer, and a read that fills it is dropped. The split packet test
also checks the bytes joined in the copy buffer.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-23 13:28:08 +11:00
clydebarrowandClaude Opus 5.5 bfa6e68039 [packet_interface] Fix clang-tidy findings in tests
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-23 13:28:08 +11:00
clydebarrowandClaude Opus 5.5 c39566f312 [udp] Keep the copy buffer in static memory
Code generation declares the UDP link's buffer for packets split across
buffers as a static array of max_packet_size bytes, so it is not
allocated at run time.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-23 13:28:08 +11:00
clydebarrowandClaude Opus 5.5 5197140f2c [packet_interface] Use a local to_vector() in tests
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-23 13:28:08 +11:00
clydebarrowandClaude Opus 5.5 ee0c6a412e [udp] Fixes from code review
- Enforce max_packet_size on send and receive
- Copy split packets into a buffer allocated in setup, not on the stack
- Add unit tests for the link's send and receive paths
- Explain why the broadcast socket is always enabled

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-23 13:28:08 +11:00
clydebarrowandClaude Opus 5.5 373c6398b2 [udp] Add UDP packet interface link
Each packet is one UDP datagram, using the settings of the udp component.
The udp component now reports whether a packet was sent, and an ENOMEM
refusal from lwIP is logged at verbose level as the packet can be sent
again.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-23 13:28:08 +11:00
29 changed files with 3245 additions and 33 deletions

No files matched your search

+1
View File
@@ -196,6 +196,7 @@ esphome/components/ens160_base/* @latonita @vincentscode
esphome/components/ens160_i2c/* @latonita
esphome/components/ens160_spi/* @latonita
esphome/components/ens210/* @itn3rd77
esphome/components/entity_transport/* @clydebarrow
esphome/components/epaper_spi/* @esphome/core
esphome/components/es7210/* @kahrendt
esphome/components/es7243/* @Chorty
@@ -0,0 +1,175 @@
"""Send sensor and binary sensor states to other devices over a packet interface."""
from collections.abc import Callable
from typing import Any
import esphome.codegen as cg
from esphome.components import packet_interface
from esphome.components.binary_sensor import BinarySensor
from esphome.components.const import CONF_PROVIDER, CONF_REMOTE_ID
from esphome.components.packet_interface import (
CONF_PACKET_INTERFACE,
final_validate_interface,
interface_layers,
)
from esphome.components.packet_interface.layers import CONF_SESSION
from esphome.components.sensor import Sensor
import esphome.config_validation as cv
from esphome.const import (
CONF_BINARY_SENSORS,
CONF_ID,
CONF_SENSORS,
CONF_TIMEOUT,
CONF_UPDATE_INTERVAL,
)
from esphome.cpp_generator import MockObj, MockObjClass
from esphome.types import ConfigType
CODEOWNERS = ["@clydebarrow"]
DEPENDENCIES = ["packet_interface"]
DOMAIN = "entity_transport"
MULTI_CONF = True
CONF_BROADCAST_ID = "broadcast_id"
CONF_ENTITY_TRANSPORT_ID = "entity_transport_id"
entity_transport_ns = cg.esphome_ns.namespace("entity_transport")
EntityTransport = entity_transport_ns.class_("EntityTransport", cg.Component)
_request_remote_sensor_slot = cg.slot_counter("ENTITY_TRANSPORT_REMOTE_SENSOR_COUNT")
_request_remote_binary_sensor_slot = cg.slot_counter(
"ENTITY_TRANSPORT_REMOTE_BINARY_SENSOR_COUNT"
)
def _advertised_id(conf: ConfigType) -> str:
return conf.get(CONF_BROADCAST_ID, conf[CONF_ID].id)
def _unique_advertised_ids(entities: list[ConfigType]) -> list[ConfigType]:
"""A receiver tells entities of one type apart by their advertised ids, so each must be different."""
seen: set[str] = set()
for index, conf in enumerate(entities):
if (advertised := _advertised_id(conf)) in seen:
raise cv.Invalid(
f"The id '{advertised}' is sent for two entities", path=[index]
)
seen.add(advertised)
return entities
def _local_entity(cls: MockObjClass) -> Callable[[Any], Any]:
return cv.maybe_simple_value(
cv.Schema(
{
cv.Required(CONF_ID): cv.use_id(cls),
cv.Optional(CONF_BROADCAST_ID): cv.All(
cv.validate_id_name, cv.Length(max=255)
),
}
),
key=CONF_ID,
)
CONFIG_SCHEMA = cv.Schema(
{
cv.GenerateID(): cv.declare_id(EntityTransport),
cv.Required(CONF_PACKET_INTERFACE): cv.use_id(packet_interface.PacketInterface),
# Defaults to a periodic resend unless the interface has session and reliability layers
cv.Optional(CONF_UPDATE_INTERVAL): cv.All(
cv.update_interval, cv.Range(min=cv.TimePeriod(seconds=1))
),
# Remote entities lose their state when none arrives for this long
cv.Optional(CONF_TIMEOUT): cv.All(
cv.positive_time_period_milliseconds,
cv.Range(min=cv.TimePeriod(seconds=1)),
),
# Entities are numbered with one byte
cv.Optional(CONF_SENSORS): cv.All(
cv.ensure_list(_local_entity(Sensor)),
cv.Length(max=255),
_unique_advertised_ids,
),
cv.Optional(CONF_BINARY_SENSORS): cv.All(
cv.ensure_list(_local_entity(BinarySensor)),
cv.Length(max=255),
_unique_advertised_ids,
),
}
).extend(cv.COMPONENT_SCHEMA)
def _final_validate(config: ConfigType) -> ConfigType:
"""With a session layer, which clears the states when the peer is lost, nothing would refresh an unchanged state."""
layers = interface_layers(config[CONF_PACKET_INTERFACE])
if CONF_TIMEOUT in config and layers is not None and CONF_SESSION in layers:
raise cv.Invalid(
f"Not allowed with a '{CONF_SESSION}' layer, which already clears the states when the other device "
"cannot be reached; unchanged states may never be sent again, so they would time out on a working link",
path=[CONF_TIMEOUT],
)
return config
FINAL_VALIDATE_SCHEMA = cv.All(
final_validate_interface(CONF_PACKET_INTERFACE), _final_validate
)
def _validate_remote_id(config: ConfigType) -> ConfigType:
if CONF_REMOTE_ID not in config and not config[CONF_ID].is_manual:
raise cv.Invalid(f"'{CONF_REMOTE_ID}' is required unless '{CONF_ID}' is given")
return config
def remote_entity_schema(base_schema: cv.Schema) -> cv.Schema:
"""Schema for an entity whose state comes from another device."""
return cv.All(
base_schema.extend(
{
cv.GenerateID(CONF_ENTITY_TRANSPORT_ID): cv.use_id(EntityTransport),
cv.Required(CONF_PROVIDER): cv.valid_name,
cv.Optional(CONF_REMOTE_ID): cv.All(
cv.string_strict, cv.Length(min=1, max=255)
),
}
),
_validate_remote_id,
)
def _remote_id(config: ConfigType) -> str:
return config.get(CONF_REMOTE_ID) or config[CONF_ID].id
async def register_remote_sensor(config: ConfigType, var: MockObj) -> None:
hub = await cg.get_variable(config[CONF_ENTITY_TRANSPORT_ID])
_request_remote_sensor_slot(str(hub))
cg.add(hub.add_remote_sensor(config[CONF_PROVIDER], _remote_id(config), var))
async def register_remote_binary_sensor(config: ConfigType, var: MockObj) -> None:
hub = await cg.get_variable(config[CONF_ENTITY_TRANSPORT_ID])
_request_remote_binary_sensor_slot(str(hub))
cg.add(hub.add_remote_binary_sensor(config[CONF_PROVIDER], _remote_id(config), var))
async def to_code(config: ConfigType) -> None:
interface = await cg.get_variable(config[CONF_PACKET_INTERFACE])
var = cg.new_Pvariable(config[CONF_ID], interface)
await cg.register_component(var, config)
if (interval := config.get(CONF_UPDATE_INTERVAL)) is not None:
cg.add(var.set_update_interval(interval))
if (timeout := config.get(CONF_TIMEOUT)) is not None:
cg.add(var.set_remote_timeout(timeout))
sensors = config.get(CONF_SENSORS, [])
binary_sensors = config.get(CONF_BINARY_SENSORS, [])
if count := len(sensors) + len(binary_sensors):
cg.add(var.set_entity_count(count))
for conf in sensors:
sensor = await cg.get_variable(conf[CONF_ID])
cg.add(var.add_sensor(_advertised_id(conf), sensor))
for conf in binary_sensors:
sensor = await cg.get_variable(conf[CONF_ID])
cg.add(var.add_binary_sensor(_advertised_id(conf), sensor))
@@ -0,0 +1,11 @@
from esphome.components import binary_sensor
from esphome.types import ConfigType
from . import register_remote_binary_sensor, remote_entity_schema
CONFIG_SCHEMA = remote_entity_schema(binary_sensor.binary_sensor_schema())
async def to_code(config: ConfigType) -> None:
var = await binary_sensor.new_binary_sensor(config)
await register_remote_binary_sensor(config, var)
@@ -0,0 +1,683 @@
#include "entity_transport.h"
#include <algorithm>
#include <cmath>
#include <cstring>
#include "esphome/core/application.h"
#include "esphome/core/log.h"
namespace esphome::entity_transport {
using packet_interface::PacketBuffer;
using packet_interface::PacketMetaData;
ESPHOME_LOG_TAG(TAG, "entity_transport");
// The packet type byte holds the format version in its high nibble
static constexpr uint8_t FORMAT_VERSION = 1;
static constexpr uint8_t PACKET_DICTIONARY = 0x11;
static constexpr uint8_t PACKET_DATA = 0x12;
static constexpr uint8_t PACKET_REQUEST = 0x13;
// Entity types, in dictionary sections and data records; 0 is not used
static constexpr uint8_t TYPE_SENSOR = 1;
static constexpr uint8_t TYPE_BINARY_SENSOR = 2;
static constexpr uint8_t MAX_TYPE = 31;
// Packet type and dictionary id
static constexpr size_t HEADER_SIZE = 5;
// A dictionary packet's part byte follows the header: the part's index in bits 0-6, and on the last part, bit 7
static constexpr uint8_t PART_INDEX_MASK = 0x7F;
static constexpr uint8_t PART_LAST = 0x80;
static constexpr uint8_t PART_NONE = 0xFF; // no dictionary is being received in order
// A data record starts with a 16-bit tag: the entity's number in bits 0-7, its type in bits 8-12, the payload form in
// bits 13-14, and with PAYLOAD_NONE, the state in bit 15. Any record can be skipped without knowing its type.
static constexpr size_t TAG_SIZE = 2;
static constexpr uint8_t TAG_TYPE_SHIFT = 8;
static constexpr uint8_t TAG_PAYLOAD_SHIFT = 13;
static constexpr uint16_t TAG_VALUE_BIT = 0x8000;
static constexpr uint8_t PAYLOAD_NONE = 0; // the state is in the tag's value bit
static constexpr uint8_t PAYLOAD_WORD = 1; // 4 bytes follow
static constexpr uint8_t PAYLOAD_LENGTH = 2; // a length byte follows, then that many bytes
static constexpr uint8_t PAYLOAD_NO_STATE = 3; // the entity has no state
static constexpr size_t WORD_SIZE = 4;
static constexpr size_t MAX_RECORD_SIZE = TAG_SIZE + WORD_SIZE;
// A dictionary section starts with the entity type, first number and count
static constexpr size_t SECTION_HEADER_SIZE = 3;
static constexpr size_t MAX_STRING_LENGTH = 255;
static constexpr uint32_t DEFAULT_UPDATE_INTERVAL_MS = 15000;
static constexpr uint32_t REQUEST_INTERVAL_MS = 5000;
static constexpr uint32_t RETRY_DELAY_MS = 1000;
static constexpr uint32_t RETRY_TIMER_ID = 0;
static constexpr uint32_t TIMEOUT_CHECK_INTERVAL_MS = 1000;
static constexpr uint32_t WARNING_INTERVAL_MS = 60000;
// How long the interface may refuse every packet before it is reported
static constexpr uint32_t REFUSED_WARNING_MS = 60000;
static void put_uint32(uint8_t *buf, uint32_t value) {
for (size_t i = 0; i != 4; i++)
buf[i] = value >> (i * 8);
}
// True if the NUL terminated str is the same as the len bytes at data
static bool matches(const char *str, const char *data, size_t len) {
return strlen(str) == len && memcmp(str, data, len) == 0;
}
struct Record {
uint8_t number;
uint8_t type;
uint8_t payload;
bool value;
const uint8_t *data; // what follows the tag
};
static Record parse_record(std::span<const uint8_t> packet, size_t pos) {
uint16_t tag = encode_uint16(packet[pos + 1], packet[pos]);
return {static_cast<uint8_t>(tag), static_cast<uint8_t>((tag >> TAG_TYPE_SHIFT) & MAX_TYPE),
static_cast<uint8_t>((tag >> TAG_PAYLOAD_SHIFT) & 3), (tag & TAG_VALUE_BIT) != 0,
packet.data() + pos + TAG_SIZE};
}
// The size of the data record at pos, or 0 if it does not fit in the packet
static size_t record_size(std::span<const uint8_t> packet, size_t pos) {
if (pos + TAG_SIZE > packet.size())
return 0;
size_t size = TAG_SIZE;
uint8_t payload = parse_record(packet, pos).payload;
if (payload == PAYLOAD_WORD) {
size += WORD_SIZE;
} else if (payload == PAYLOAD_LENGTH) {
if (pos + TAG_SIZE == packet.size())
return 0;
size += 1 + packet[pos + TAG_SIZE];
}
return pos + size <= packet.size() ? size : 0;
}
// Checks every record of a data packet, so that a damaged or forged packet changes no state at all
static bool data_well_formed(std::span<const uint8_t> packet) {
for (size_t pos = HEADER_SIZE; pos != packet.size();) {
size_t size = record_size(packet, pos);
if (size == 0)
return false;
Record record = parse_record(packet, pos);
if (record.type == 0 || (record.value && record.payload != PAYLOAD_NONE))
return false;
// Types this device knows must use their own payload form; others are skipped
if (record.type == TYPE_SENSOR && record.payload != PAYLOAD_WORD && record.payload != PAYLOAD_NO_STATE)
return false;
if (record.type == TYPE_BINARY_SENSOR && record.payload != PAYLOAD_NONE && record.payload != PAYLOAD_NO_STATE)
return false;
pos += size;
}
return true;
}
// Checks the device name and every section of a dictionary packet before anything is learnt from it
static bool dictionary_well_formed(std::span<const uint8_t> packet) {
size_t pos = HEADER_SIZE + 1;
if (pos >= packet.size())
return false;
pos += 1 + packet[pos];
if (pos > packet.size())
return false;
while (pos != packet.size()) {
if (pos + SECTION_HEADER_SIZE > packet.size())
return false;
uint8_t type = packet[pos];
size_t first = packet[pos + 1];
size_t count = packet[pos + 2];
pos += SECTION_HEADER_SIZE;
// Numbers are one byte, and a data record can name types up to MAX_TYPE only
if (type == 0 || type > MAX_TYPE || count == 0 || first + count > 256)
return false;
for (size_t i = 0; i != count; i++) {
if (pos == packet.size())
return false;
pos += 1 + packet[pos];
if (pos > packet.size())
return false;
}
}
return true;
}
void EntityTransport::setup() {
this->interface_ = this->interface_->top();
this->max_packet_size_ = this->interface_->get_max_packet_size();
const auto &name = App.get_name();
size_t longest_id = 0;
for (const auto &entity : this->entities_)
longest_id = std::max(longest_id, strlen(entity.id));
// A dictionary packet must hold the device name and any one id, which also leaves room for a data record
if (name.size() > MAX_STRING_LENGTH || longest_id > MAX_STRING_LENGTH ||
HEADER_SIZE + 2 + name.size() + SECTION_HEADER_SIZE + 1 + longest_id > this->max_packet_size_) {
this->mark_failed(LOG_STR("max_packet_size too small for the device name and entity ids"));
return;
}
RAMAllocator<uint8_t> allocator;
this->buffer_ = allocator.allocate(this->max_packet_size_);
if (this->buffer_ == nullptr) {
this->mark_failed(LOG_STR("Not enough memory for the packet buffer"));
return;
}
// The dictionary id changes when the device name or any entity changes
uint32_t hash = fnv1a_hash(name.c_str());
for (const auto &entity : this->entities_) {
hash = fnv1a_hash_extend(hash, entity.is_sensor ? TYPE_SENSOR : TYPE_BINARY_SENSOR);
hash = fnv1a_hash_extend(hash, entity.id);
}
this->dictionary_id_ = hash;
// [&entity] is safe: FixedVector never reallocates, and the parent pointer keeps the capture to one pointer
for (auto &entity : this->entities_) {
#ifdef USE_SENSOR
if (entity.is_sensor) {
static_cast<sensor::Sensor *>(entity.entity)->add_on_state_callback([&entity](float) {
entity.parent->mark_updated_(entity);
});
}
#endif
#ifdef USE_BINARY_SENSOR
if (!entity.is_sensor) {
// The full state callback also fires for the first state
static_cast<binary_sensor::BinarySensor *>(entity.entity)
->add_full_state_callback(
[&entity](optional<bool>, optional<bool>) { entity.parent->mark_updated_(entity); });
}
#endif
}
this->interface_->add_packet_interface_listener(
[this](const PacketBuffer &data, const PacketMetaData &) { this->receive_(data); });
bool session = this->interface_->has_capability(packet_interface::CAPABILITY_SESSION);
if (session)
this->interface_->add_on_session_state_callback([this](bool connected) { this->on_session_(connected); });
// Without a session and reliable delivery, a lost dictionary or state is only replaced by a periodic resend
if (this->update_interval_ms_ == 0) {
this->update_interval_ms_ = session && this->interface_->has_capability(packet_interface::CAPABILITY_RELIABILITY)
? SCHEDULER_DONT_RUN
: DEFAULT_UPDATE_INTERVAL_MS;
}
if (this->update_interval_ms_ != SCHEDULER_DONT_RUN)
this->set_interval(this->update_interval_ms_, [this] { this->send_all_(); });
if (this->remote_timeout_ms_ != 0)
this->set_interval(TIMEOUT_CHECK_INTERVAL_MS, [this] { this->clear_remote_states_(true); });
this->send_all_();
}
void EntityTransport::dump_config() {
ESP_LOGCONFIG(TAG,
"Entity Transport:\n"
" Packet Interface: %s\n"
" Max Packet Size: %zu\n"
" Encryption: %s\n"
" Reliability: %s\n"
" Session: %s\n"
" Dictionary ID: %08" PRIX32,
this->interface_->get_name(), this->max_packet_size_,
YESNO(this->interface_->has_capability(packet_interface::CAPABILITY_ENCRYPTION)),
YESNO(this->interface_->has_capability(packet_interface::CAPABILITY_RELIABILITY)),
YESNO(this->interface_->has_capability(packet_interface::CAPABILITY_SESSION)), this->dictionary_id_);
if (this->update_interval_ms_ == SCHEDULER_DONT_RUN) {
ESP_LOGCONFIG(TAG, " Update Interval: never");
} else {
ESP_LOGCONFIG(TAG, " Update Interval: %" PRIu32 "ms", this->update_interval_ms_);
}
if (this->remote_timeout_ms_ != 0) {
ESP_LOGCONFIG(TAG, " Remote Timeout: %" PRIu32 "ms", this->remote_timeout_ms_);
}
for (const auto &entity : this->entities_) {
ESP_LOGCONFIG(TAG, " %s %u: %s", entity.is_sensor ? LOG_STR_LITERAL("Sensor") : LOG_STR_LITERAL("Binary Sensor"),
entity.number, entity.id);
}
#ifdef ENTITY_TRANSPORT_REMOTE_SENSOR_COUNT
for (const auto &remote : this->remote_sensors_) {
ESP_LOGCONFIG(TAG, " Remote Sensor: %s from %s", remote.remote_id, remote.provider);
}
#endif
#ifdef ENTITY_TRANSPORT_REMOTE_BINARY_SENSOR_COUNT
for (const auto &remote : this->remote_binary_sensors_) {
ESP_LOGCONFIG(TAG, " Remote Binary Sensor: %s from %s", remote.remote_id, remote.provider);
}
#endif
}
void EntityTransport::send_all_() {
this->dictionary_pending_ = true;
this->dictionary_next_ = 0;
this->dictionary_part_ = 0;
for (auto &entity : this->entities_)
entity.updated = true;
this->enable_loop();
}
void EntityTransport::mark_updated_(LocalEntity &entity) {
entity.updated = true;
this->enable_loop();
}
void EntityTransport::loop() {
this->disable_loop();
// States are only sent once the dictionary that names them has been accepted
if ((this->dictionary_pending_ && !this->send_dictionary_()) || !this->send_updated_()) {
this->sending_refused_();
this->set_timeout(RETRY_TIMER_ID, RETRY_DELAY_MS, [this] { this->enable_loop(); });
return;
}
this->refusing_ = false;
if (this->status_has_warning()) {
ESP_LOGI(TAG, "Packets are accepted again");
this->status_clear_warning();
}
}
// A link that is down or full refuses packets for a while; one that refuses them for long sends nothing at all
void EntityTransport::sending_refused_() {
uint32_t now = App.get_loop_component_start_time();
if (!this->refusing_) {
this->refusing_ = true;
this->refused_since_ms_ = now;
return;
}
if (!this->status_has_warning() && now - this->refused_since_ms_ >= REFUSED_WARNING_MS) {
ESP_LOGW(TAG, "The packet interface has accepted no packets for %" PRIu32 "s, so no states are sent",
(now - this->refused_since_ms_) / 1000);
this->status_set_warning();
}
}
size_t EntityTransport::write_header_(uint8_t type) const {
this->buffer_[0] = type;
put_uint32(this->buffer_ + 1, this->dictionary_id_);
return HEADER_SIZE;
}
bool EntityTransport::send_dictionary_() {
uint8_t *buf = this->buffer_;
const auto &name = App.get_name();
size_t header = this->write_header_(PACKET_DICTIONARY);
size_t part_offset = header++;
buf[header++] = name.size();
memcpy(buf + header, name.c_str(), name.size());
header += name.size();
size_t length = header;
size_t count_offset = 0;
uint8_t section_type = 0; // none
// A dictionary in several packets resumes after the last packet accepted, so a refused one does not resend the rest
for (size_t i = this->dictionary_next_; i != this->entities_.size(); i++) {
const auto &entity = this->entities_[i];
uint8_t type = entity.is_sensor ? TYPE_SENSOR : TYPE_BINARY_SENSOR;
size_t id_length = strlen(entity.id);
bool new_section = type != section_type;
if (length + (new_section ? SECTION_HEADER_SIZE : 0) + 1 + id_length > this->max_packet_size_) {
buf[part_offset] = this->dictionary_part_ & PART_INDEX_MASK;
if (!this->interface_->send_to_interface(PacketBuffer(buf, length), {})) {
ESP_LOGV(TAG, "Dictionary packet of %zu bytes not accepted, will retry", length);
return false;
}
this->dictionary_next_ = i;
this->dictionary_part_++;
length = header;
new_section = true;
}
if (new_section) {
buf[length++] = type;
buf[length++] = entity.number;
count_offset = length;
buf[length++] = 0;
section_type = type;
}
buf[count_offset]++;
buf[length++] = id_length;
memcpy(buf + length, entity.id, id_length);
length += id_length;
}
if (length != header) {
buf[part_offset] = (this->dictionary_part_ & PART_INDEX_MASK) | PART_LAST;
if (!this->interface_->send_to_interface(PacketBuffer(buf, length), {})) {
ESP_LOGV(TAG, "Dictionary packet of %zu bytes not accepted, will retry", length);
return false;
}
}
this->dictionary_pending_ = false;
this->dictionary_next_ = 0;
this->dictionary_part_ = 0;
return true;
}
// Writes the entity's data record, and returns its size
size_t EntityTransport::write_record_(const LocalEntity &entity, uint8_t *record) const {
uint8_t type = entity.is_sensor ? TYPE_SENSOR : TYPE_BINARY_SENSOR;
uint8_t payload = PAYLOAD_NO_STATE;
bool value = false;
size_t size = TAG_SIZE;
#ifdef USE_SENSOR
if (entity.is_sensor) {
float state = static_cast<sensor::Sensor *>(entity.entity)->get_state();
if (!std::isnan(state)) {
payload = PAYLOAD_WORD;
uint32_t bits;
memcpy(&bits, &state, sizeof(bits));
put_uint32(record + TAG_SIZE, bits);
size += WORD_SIZE;
}
}
#endif
#ifdef USE_BINARY_SENSOR
if (!entity.is_sensor) {
auto *sensor = static_cast<binary_sensor::BinarySensor *>(entity.entity);
if (sensor->has_state()) {
payload = PAYLOAD_NONE;
value = sensor->state;
}
}
#endif
uint16_t tag =
entity.number | (type << TAG_TYPE_SHIFT) | (payload << TAG_PAYLOAD_SHIFT) | (value ? TAG_VALUE_BIT : 0);
record[0] = tag;
record[1] = tag >> 8;
return size;
}
// Sends every updated entity, in as few packets as fit; returns false if a packet was refused
bool EntityTransport::send_updated_() {
size_t length = this->write_header_(PACKET_DATA);
size_t first = 0;
for (size_t i = 0; i != this->entities_.size(); i++) {
auto &entity = this->entities_[i];
if (!entity.updated)
continue;
uint8_t record[MAX_RECORD_SIZE];
size_t size = this->write_record_(entity, record);
if (length + size > this->max_packet_size_) {
if (!this->flush_(length, first, i))
return false;
length = HEADER_SIZE;
first = i;
}
memcpy(this->buffer_ + length, record, size);
length += size;
}
return this->flush_(length, first, this->entities_.size());
}
// Sends the packet holding the updated entities from first up to last, and clears their flags if it was accepted
bool EntityTransport::flush_(size_t length, size_t first, size_t last) {
if (length == HEADER_SIZE)
return true;
if (!this->interface_->send_to_interface(PacketBuffer(this->buffer_, length), {})) {
ESP_LOGV(TAG, "Packet of %zu bytes not accepted, will retry", length);
return false;
}
for (size_t i = first; i != last; i++)
this->entities_[i].updated = false;
return true;
}
void EntityTransport::receive_(const PacketBuffer &data) {
// Packet interface layers deliver packets in one piece
if (!data.is_contiguous()) {
ESP_LOGW(TAG, "Packet split across buffers is not supported");
return;
}
auto packet = data.as_span();
if (packet.size() < HEADER_SIZE) {
if (this->warning_due_()) {
ESP_LOGW(TAG, "Packet of %zu bytes is too short", packet.size());
}
return;
}
if (packet[0] >> 4 != FORMAT_VERSION) {
if (this->warning_due_()) {
ESP_LOGW(TAG, "Packet format version %u not supported; the sender may run another ESPHome version",
packet[0] >> 4);
}
return;
}
uint32_t dictionary_id = encode_uint32(packet[4], packet[3], packet[2], packet[1]);
if (dictionary_id == this->dictionary_id_) {
// Either a request for our dictionary, or our own packet heard on a broadcast channel
if (packet[0] == PACKET_REQUEST && packet.size() == HEADER_SIZE) {
ESP_LOGD(TAG, "Dictionary requested");
this->send_all_();
}
return;
}
if (packet[0] == PACKET_DICTIONARY) {
this->receive_dictionary_(packet, dictionary_id);
} else if (packet[0] == PACKET_DATA) {
this->receive_data_(packet, dictionary_id);
} else if (packet[0] != PACKET_REQUEST) {
ESP_LOGV(TAG, "Unknown packet type 0x%02X", packet[0]);
}
}
void EntityTransport::receive_dictionary_(std::span<const uint8_t> packet, uint32_t dictionary_id) {
if (!dictionary_well_formed(packet)) {
if (this->warning_due_()) {
ESP_LOGW(TAG, "Dictionary packet of %zu bytes is malformed", packet.size());
}
return;
}
size_t pos = HEADER_SIZE;
uint8_t part = packet[pos++];
uint8_t index = part & PART_INDEX_MASK;
// The dictionary is complete, and need not be asked for again, once its last part follows all the others in order
bool in_order = index == 0 || (this->rx_dictionary_id_ == dictionary_id && index == this->rx_next_part_);
this->rx_dictionary_id_ = dictionary_id;
this->rx_next_part_ = in_order ? (index + 1) & PART_INDEX_MASK : PART_NONE;
bool completed = in_order && (part & PART_LAST) != 0;
bool first_time = completed && !this->dictionary_seen_(dictionary_id);
if (first_time) {
this->seen_dictionaries_[this->seen_next_] = dictionary_id;
this->seen_next_ = (this->seen_next_ + 1) % this->seen_dictionaries_.size();
}
size_t provider_length = packet[pos++];
const auto *provider = reinterpret_cast<const char *>(packet.data() + pos);
pos += provider_length;
while (pos != packet.size()) {
uint8_t type = packet[pos];
uint8_t number = packet[pos + 1];
uint8_t count = packet[pos + 2];
pos += SECTION_HEADER_SIZE;
for (uint8_t i = 0; i != count; i++) {
size_t id_length = packet[pos++];
this->learn_(provider, provider_length, type, number + i, reinterpret_cast<const char *>(packet.data() + pos),
id_length, dictionary_id);
pos += id_length;
}
}
// A provider may go back to a dictionary already seen, so every complete one is checked
if (completed)
this->forget_unnamed_(provider, provider_length, dictionary_id, first_time);
}
void EntityTransport::forget_unnamed_(const char *provider, size_t provider_length, uint32_t dictionary_id,
bool first_time) {
// A remote entity that its provider's whole dictionary does not name would otherwise keep no or a stale state
// silently
#ifdef ENTITY_TRANSPORT_REMOTE_SENSOR_COUNT
for (auto &remote : this->remote_sensors_) {
if (!matches(remote.provider, provider, provider_length) || (remote.known && remote.dictionary_id == dictionary_id))
continue;
if (remote.known || first_time) {
ESP_LOGW(TAG, "'%s' does not send '%s'", remote.provider, remote.remote_id);
}
remote.known = false;
if (remote.has_state) {
remote.has_state = false;
remote.entity->publish_state(NAN);
}
}
#endif
#ifdef ENTITY_TRANSPORT_REMOTE_BINARY_SENSOR_COUNT
for (auto &remote : this->remote_binary_sensors_) {
if (!matches(remote.provider, provider, provider_length) || (remote.known && remote.dictionary_id == dictionary_id))
continue;
if (remote.known || first_time) {
ESP_LOGW(TAG, "'%s' does not send '%s'", remote.provider, remote.remote_id);
}
remote.known = false;
if (remote.has_state) {
remote.has_state = false;
remote.entity->invalidate_state();
}
}
#endif
}
void EntityTransport::learn_(const char *provider, size_t provider_length, uint8_t type, uint8_t number, const char *id,
size_t id_length, uint32_t dictionary_id) {
#ifdef ENTITY_TRANSPORT_REMOTE_SENSOR_COUNT
if (type == TYPE_SENSOR) {
for (auto &remote : this->remote_sensors_) {
if (matches(remote.provider, provider, provider_length) && matches(remote.remote_id, id, id_length)) {
remote.dictionary_id = dictionary_id;
remote.number = number;
remote.known = true;
}
}
}
#endif
#ifdef ENTITY_TRANSPORT_REMOTE_BINARY_SENSOR_COUNT
if (type == TYPE_BINARY_SENSOR) {
for (auto &remote : this->remote_binary_sensors_) {
if (matches(remote.provider, provider, provider_length) && matches(remote.remote_id, id, id_length)) {
remote.dictionary_id = dictionary_id;
remote.number = number;
remote.known = true;
}
}
}
#endif
}
bool EntityTransport::dictionary_seen_(uint32_t dictionary_id) const {
return std::find(this->seen_dictionaries_.begin(), this->seen_dictionaries_.end(), dictionary_id) !=
this->seen_dictionaries_.end();
}
void EntityTransport::receive_data_(std::span<const uint8_t> packet, uint32_t dictionary_id) {
bool known = false;
bool has_remotes = false;
#ifdef ENTITY_TRANSPORT_REMOTE_SENSOR_COUNT
for (const auto &remote : this->remote_sensors_) {
known |= remote.known && remote.dictionary_id == dictionary_id;
has_remotes = true;
}
#endif
#ifdef ENTITY_TRANSPORT_REMOTE_BINARY_SENSOR_COUNT
for (const auto &remote : this->remote_binary_sensors_) {
known |= remote.known && remote.dictionary_id == dictionary_id;
has_remotes = true;
}
#endif
if (!has_remotes)
return;
// Checked first, so a damaged packet does not make the provider resend everything
if (!data_well_formed(packet)) {
if (this->warning_due_()) {
ESP_LOGW(TAG, "Data packet of %zu bytes is malformed", packet.size());
}
return;
}
// A dictionary not yet received in full may name our entities, or be a provider's new one; one received in full
// named all of them it could, so asking again would only make that provider resend everything
if (!this->dictionary_seen_(dictionary_id))
this->request_dictionary_(dictionary_id);
if (!known)
return;
[[maybe_unused]] uint32_t now = App.get_loop_component_start_time();
for (size_t pos = HEADER_SIZE; pos != packet.size(); pos += record_size(packet, pos)) {
[[maybe_unused]] Record record = parse_record(packet, pos);
#ifdef ENTITY_TRANSPORT_REMOTE_SENSOR_COUNT
if (record.type == TYPE_SENSOR) {
float value = NAN;
if (record.payload == PAYLOAD_WORD) {
uint32_t bits = encode_uint32(record.data[3], record.data[2], record.data[1], record.data[0]);
memcpy(&value, &bits, sizeof(value));
}
for (auto &remote : this->remote_sensors_) {
if (remote.known && remote.dictionary_id == dictionary_id && remote.number == record.number) {
remote.last_state_ms = now;
remote.has_state = true;
remote.entity->publish_state(value);
}
}
}
#endif
#ifdef ENTITY_TRANSPORT_REMOTE_BINARY_SENSOR_COUNT
if (record.type == TYPE_BINARY_SENSOR) {
for (auto &remote : this->remote_binary_sensors_) {
if (!remote.known || remote.dictionary_id != dictionary_id || remote.number != record.number)
continue;
remote.last_state_ms = now;
remote.has_state = true;
if (record.payload == PAYLOAD_NO_STATE) {
remote.entity->invalidate_state();
} else {
remote.entity->publish_state(record.value);
}
}
}
#endif
}
}
void EntityTransport::request_dictionary_(uint32_t dictionary_id) {
uint32_t now = App.get_loop_component_start_time();
if (this->request_sent_ && now - this->last_request_ms_ < REQUEST_INTERVAL_MS)
return;
uint8_t packet[HEADER_SIZE] = {PACKET_REQUEST};
put_uint32(packet + 1, dictionary_id);
// A refused request is tried again with the next data packet under that dictionary id
if (!this->interface_->send_to_interface(PacketBuffer(packet, sizeof(packet)), {})) {
ESP_LOGV(TAG, "Request for dictionary %08" PRIX32 " not accepted", dictionary_id);
return;
}
this->request_sent_ = true;
this->last_request_ms_ = now;
ESP_LOGD(TAG, "Requested dictionary %08" PRIX32, dictionary_id);
}
// Dropped packets point to a device with other firmware or a fault, and may arrive steadily, so warn only now and then
bool EntityTransport::warning_due_() {
uint32_t now = App.get_loop_component_start_time();
if (this->warned_ && now - this->last_warning_ms_ < WARNING_INTERVAL_MS)
return false;
this->warned_ = true;
this->last_warning_ms_ = now;
return true;
}
// A newly connected or restarted peer needs the dictionary and every state; while it is lost, its entities have no
// state
void EntityTransport::on_session_(bool connected) {
if (connected) {
this->send_all_();
return;
}
this->clear_remote_states_(false);
}
// Clears the state of every remote entity, or with timed_out_only, of those whose state is older than the timeout
void EntityTransport::clear_remote_states_(bool timed_out_only) {
[[maybe_unused]] uint32_t now = App.get_loop_component_start_time();
#ifdef ENTITY_TRANSPORT_REMOTE_SENSOR_COUNT
for (auto &remote : this->remote_sensors_) {
if (timed_out_only && (!remote.has_state || now - remote.last_state_ms <= this->remote_timeout_ms_))
continue;
remote.has_state = false;
remote.entity->publish_state(NAN);
}
#endif
#ifdef ENTITY_TRANSPORT_REMOTE_BINARY_SENSOR_COUNT
for (auto &remote : this->remote_binary_sensors_) {
if (timed_out_only && (!remote.has_state || now - remote.last_state_ms <= this->remote_timeout_ms_))
continue;
remote.has_state = false;
remote.entity->invalidate_state();
}
#endif
}
} // namespace esphome::entity_transport
@@ -0,0 +1,171 @@
#pragma once
#include <array>
#include <span>
#include "esphome/components/packet_interface/packet_interface.h"
#include "esphome/core/component.h"
#include "esphome/core/defines.h"
#include "esphome/core/entity_base.h"
#include "esphome/core/helpers.h"
#ifdef USE_SENSOR
#include "esphome/components/sensor/sensor.h"
#endif
#ifdef USE_BINARY_SENSOR
#include "esphome/components/binary_sensor/binary_sensor.h"
#endif
namespace esphome::entity_transport {
/**
* Sends the states of local sensors and binary sensors over a packet interface, and publishes the states received
* from other devices to remote entities.
*
* Each device numbers its sensors and its binary sensors from 0, and sends a dictionary that gives the id for each
* number. Data packets then refer to an entity by its number. Every packet carries the sender's dictionary id, a hash
* of its device name and dictionary, so that a receiver knows which device sent it and which dictionary applies.
* All values are little endian.
*
* Dictionary packet:
* - type: 1 byte, PACKET_DICTIONARY;
* - dictionary id: 4 bytes;
* - part: 1 byte, the packet's index in the dictionary in bits 0-6, counting from 0 and wrapping, and bit 7 set on
* the last packet;
* - sender's device name: length (1 byte), then the name;
* - one or more sections, each: entity type (1 byte), first number (1 byte), count (1 byte), then for each entity
* its id: length (1 byte), then the id.
*
* Data packet:
* - type: 1 byte, PACKET_DATA;
* - dictionary id: 4 bytes;
* - records, to the end, each a 16-bit tag, then its payload. The tag holds the entity's number in bits 0-7, its type
* in bits 8-12, and the payload form in bits 13-14: none, with the state in bit 15 (binary sensors); 4 bytes (a
* sensor's float); a length byte, then that many bytes; or no state, with no payload. The form gives the record's
* size, so a receiver skips records of types it does not know.
*
* Every packet is checked in full before anything in it is used, so a damaged or forged packet changes no state.
*
* Dictionary request packet, sent by a device that receives data with a dictionary id it has not received in full,
* with every part in order:
* - type: 1 byte, PACKET_REQUEST;
* - the dictionary id wanted: 4 bytes.
*
* Changed states are sent from loop(). The dictionary and all states are sent at boot, when a session layer reports
* the peer connected, when the dictionary is requested, and every update interval. Unless the interface has both a
* session and a reliability layer, the update interval defaults to DEFAULT_UPDATE_INTERVAL_MS. When the interface
* refuses a packet, sending is tried again after a delay, and a warning is set once it has refused every packet for a
* minute. With a remote timeout, a remote entity loses its state when
* none arrives for it within the timeout.
*/
class EntityTransport : public Component {
public:
explicit EntityTransport(packet_interface::PacketInterface *interface) : interface_(interface) {}
void setup() override;
void loop() override;
void dump_config() override;
/// 0 chooses the default for the interface's layers; SCHEDULER_DONT_RUN never resends.
void set_update_interval(uint32_t interval_ms) { this->update_interval_ms_ = interval_ms; }
/// Remote entities lose their state when none arrives for this long; 0 keeps the last state.
void set_remote_timeout(uint32_t timeout_ms) { this->remote_timeout_ms_ = timeout_ms; }
void set_entity_count(size_t count) { this->entities_.init(count); }
// Sensors must be added before binary sensors, so that data packets hold sensor records first
#ifdef USE_SENSOR
void add_sensor(const char *id, sensor::Sensor *sensor) {
this->entities_.push_back({sensor, id, this, this->sensor_count_++, true, true});
}
#endif
#ifdef USE_BINARY_SENSOR
void add_binary_sensor(const char *id, binary_sensor::BinarySensor *sensor) {
this->entities_.push_back({sensor, id, this, this->binary_sensor_count_++, false, true});
}
#endif
#ifdef ENTITY_TRANSPORT_REMOTE_SENSOR_COUNT
void add_remote_sensor(const char *provider, const char *remote_id, sensor::Sensor *sensor) {
this->remote_sensors_.push_back({provider, remote_id, sensor});
}
#endif
#ifdef ENTITY_TRANSPORT_REMOTE_BINARY_SENSOR_COUNT
void add_remote_binary_sensor(const char *provider, const char *remote_id, binary_sensor::BinarySensor *sensor) {
this->remote_binary_sensors_.push_back({provider, remote_id, sensor});
}
#endif
protected:
struct LocalEntity {
EntityBase *entity; // a sensor::Sensor or a binary_sensor::BinarySensor, as is_sensor says
const char *id;
EntityTransport *parent;
uint8_t number;
bool is_sensor;
bool updated;
};
template<typename T> struct RemoteEntity {
const char *provider;
const char *remote_id;
T *entity;
uint32_t dictionary_id{0}; // learnt from the provider's dictionary
uint32_t last_state_ms{0};
uint8_t number{0};
bool known{false};
bool has_state{false}; // a state arrived, and has not yet timed out
};
void send_all_();
void mark_updated_(LocalEntity &entity);
bool send_dictionary_();
bool send_updated_();
size_t write_record_(const LocalEntity &entity, uint8_t *record) const;
bool flush_(size_t length, size_t first, size_t last);
size_t write_header_(uint8_t type) const;
void receive_(const packet_interface::PacketBuffer &data);
void receive_dictionary_(std::span<const uint8_t> packet, uint32_t dictionary_id);
void receive_data_(std::span<const uint8_t> packet, uint32_t dictionary_id);
void learn_(const char *provider, size_t provider_length, uint8_t type, uint8_t number, const char *id,
size_t id_length, uint32_t dictionary_id);
void request_dictionary_(uint32_t dictionary_id);
bool dictionary_seen_(uint32_t dictionary_id) const;
/// Clears each remote entity of \p provider that its complete dictionary does not name, warning if the entity was
/// known or the dictionary is \p first_time seen, so a resent dictionary does not repeat the warning.
void forget_unnamed_(const char *provider, size_t provider_length, uint32_t dictionary_id, bool first_time);
bool warning_due_();
void sending_refused_();
void on_session_(bool connected);
void clear_remote_states_(bool timed_out_only);
packet_interface::PacketInterface *interface_;
FixedVector<LocalEntity> entities_;
#ifdef ENTITY_TRANSPORT_REMOTE_SENSOR_COUNT
StaticVector<RemoteEntity<sensor::Sensor>, ENTITY_TRANSPORT_REMOTE_SENSOR_COUNT> remote_sensors_;
#endif
#ifdef ENTITY_TRANSPORT_REMOTE_BINARY_SENSOR_COUNT
StaticVector<RemoteEntity<binary_sensor::BinarySensor>, ENTITY_TRANSPORT_REMOTE_BINARY_SENSOR_COUNT>
remote_binary_sensors_;
#endif
// Packets are built here, allocated in setup()
uint8_t *buffer_{nullptr};
size_t max_packet_size_{0};
uint32_t update_interval_ms_{0};
uint32_t remote_timeout_ms_{0};
uint32_t dictionary_id_{0};
uint32_t last_request_ms_{0};
uint32_t last_warning_ms_{0};
uint32_t refused_since_ms_{0}; // when the interface began refusing every packet
uint32_t rx_dictionary_id_{0}; // the dictionary whose parts are arriving
// The last dictionary ids received in full, so a provider whose dictionary names none of our entities is not asked
// again
std::array<uint32_t, 8> seen_dictionaries_{};
uint8_t seen_next_{0};
uint8_t rx_next_part_{0}; // the index of its next part, or PART_NONE if one was missed
uint8_t dictionary_part_{0}; // the index of the next dictionary packet to send
uint8_t sensor_count_{0};
uint8_t binary_sensor_count_{0};
size_t dictionary_next_{0}; // the first entity of the next dictionary packet to send
bool dictionary_pending_{true};
bool request_sent_{false};
bool warned_{false};
bool refusing_{false};
};
} // namespace esphome::entity_transport
@@ -0,0 +1,11 @@
from esphome.components import sensor
from esphome.types import ConfigType
from . import register_remote_sensor, remote_entity_schema
CONFIG_SCHEMA = remote_entity_schema(sensor.sensor_schema())
async def to_code(config: ConfigType) -> None:
var = await sensor.new_sensor(config)
await register_remote_sensor(config, var)
@@ -1,3 +1,4 @@
from collections.abc import Callable
import importlib
import pkgutil
from typing import Any
@@ -6,14 +7,24 @@ import esphome.codegen as cg
from esphome.components.const import CONF_MAX_PACKET_SIZE
import esphome.config_validation as cv
from esphome.const import CONF_ID
from esphome.core import ID
from esphome.cpp_generator import MockObj, MockObjClass
import esphome.final_validate as fv
from esphome.types import ConfigType
from . import layers
from .layers import (
ADDRESSING,
BROADCAST_ADDRESS,
CONF_ADDRESSING,
CONF_DESTINATION,
CONF_PEERS,
STACK_ORDER,
PacketInterface,
build_stack,
check_packet_sizes,
check_session,
check_single_peer,
packet_interface_ns,
stack_auto_load,
stack_schema,
@@ -29,12 +40,17 @@ CONF_PACKET_INTERFACE = "packet_interface"
for module_info in pkgutil.iter_modules(layers.__path__):
importlib.import_module(f".layers.{module_info.name}", package=__package__)
LINK_LAYER_SCHEMAS = stack_schema()
LINK_LAYER_SCHEMAS = {
cv.Optional(CONF_ADDRESSING): ADDRESSING.schema(),
**stack_schema(),
}
__all__ = [
"CONF_MAX_PACKET_SIZE",
"CONF_PACKET_INTERFACE",
"interface_layers",
"PacketInterface",
"final_validate_interface",
"new_packet_interface",
"packet_interface_ns",
"packet_interface_schema",
@@ -42,7 +58,7 @@ __all__ = [
def AUTO_LOAD(config: list[ConfigType] | None = None) -> list[str]:
"""Load only what the configured layers use.
"""Load only what the configured layers use, on the link and in each addressing peer.
Without a config, as tooling that lists dependencies calls it, everything a layer could load.
"""
@@ -51,14 +67,71 @@ def AUTO_LOAD(config: list[ConfigType] | None = None) -> list[str]:
loads: set[str] = set()
for interface in config:
loads |= stack_auto_load(interface)
for peer in interface.get(CONF_ADDRESSING, {}).get(CONF_PEERS, []):
loads |= stack_auto_load(peer)
return sorted(loads)
def _validate_layers(config: ConfigType) -> ConfigType:
"""With peers, the layers go in each peer; otherwise check them against the one destination."""
check_session(config)
size = config[CONF_MAX_PACKET_SIZE]
if (addressing := config.get(CONF_ADDRESSING)) is None:
check_packet_sizes(config, size)
return config
size = ADDRESSING.get_type(addressing).packet_size(addressing, size)
if CONF_PEERS in addressing:
for kind in STACK_ORDER:
if kind.key in config:
raise cv.Invalid(
f"With '{CONF_PEERS}', '{kind.key}' goes in each peer",
path=[kind.key],
)
for index, peer in enumerate(addressing[CONF_PEERS]):
try:
check_packet_sizes(peer, size)
except cv.Invalid as err:
err.prepend([CONF_ADDRESSING, CONF_PEERS, index])
raise
return config
addressing.setdefault(CONF_DESTINATION, BROADCAST_ADDRESS)
check_single_peer(config, addressing[CONF_DESTINATION])
check_packet_sizes(config, size)
return config
def interface_layers(interface_id: ID) -> ConfigType | None:
"""During final validation, the configuration holding the layers of the packet interface with this id.
The id is a link's, or an addressing peer's.
"""
for interface in fv.full_config.get().get(DOMAIN, []):
if interface[CONF_ID].id == interface_id.id:
return interface
for peer in interface.get(CONF_ADDRESSING, {}).get(CONF_PEERS, []):
if peer[CONF_ID].id == interface_id.id:
return peer
return None
def final_validate_interface(key: str) -> Callable[[ConfigType], ConfigType]:
"""Final validation for an application: a link with peers must be used through one of the peers."""
def validator(config: ConfigType) -> ConfigType:
interface_id: ID = config[key]
interface = interface_layers(interface_id)
if interface is not None and (
peers := interface.get(CONF_ADDRESSING, {}).get(CONF_PEERS)
):
raise cv.Invalid(
f"'{interface_id.id}' has peers, so use the id of one of them, such as '{peers[0][CONF_ID].id}'",
path=[key],
)
return config
return validator
def packet_interface_schema(
class_: MockObjClass,
default_max_packet_size: int,
@@ -86,10 +159,17 @@ async def new_packet_interface(config: ConfigType, *args: Any) -> MockObj:
The link's constructor takes its maximum packet size, usually as a std::span over a buffer that code generation
sized from the configuration, so the two cannot differ.
Returns the link layer; applications reach the top layer with PacketInterface::top().
Returns the link layer; applications reach the top layer with PacketInterface::top(). With several addressing
peers, applications use the peers' ids instead.
"""
var = cg.new_Pvariable(config[CONF_ID], *args)
cg.add(var.set_name(config[CONF_ID].id))
await cg.register_component(var, config)
await build_stack(config, var)
if (addressing := config.get(CONF_ADDRESSING)) is None:
await build_stack(config, var)
return var
for port, layers_config in await ADDRESSING.get_type(addressing).build(
addressing, var
):
await build_stack(config if layers_config is None else layers_config, port)
return var
@@ -0,0 +1,97 @@
#include "basic_addressing.h"
#ifdef USE_PACKET_INTERFACE_BASIC_ADDRESSING
#include "esphome/core/log.h"
namespace esphome::packet_interface {
ESPHOME_LOG_TAG(TAG, "packet_interface.addressing");
void BasicAddressing::setup() {
this->link_->add_packet_interface_listener(
[this](const PacketBuffer &data, const PacketMetaData &meta_data) { this->receive_(data, meta_data); });
}
void BasicAddressing::dump_config() {
ESP_LOGCONFIG(TAG,
"Basic Addressing on '%s':\n"
" Address: 0x%04X",
this->link_->get_name(), this->address_);
}
bool BasicAddressing::send(const PacketBuffer &data, const PacketMetaData &meta_data, uint16_t destination) {
if (meta_data.destination_address != 0)
destination = meta_data.destination_address;
uint8_t header[HEADER_SIZE] = {static_cast<uint8_t>(destination), static_cast<uint8_t>(destination >> 8),
static_cast<uint8_t>(this->address_), static_cast<uint8_t>(this->address_ >> 8)};
PacketBuffer packet(header, HEADER_SIZE, &data, nullptr, 0);
return this->link_->send_to_interface(packet, meta_data);
}
void BasicAddressing::receive_(const PacketBuffer &data, const PacketMetaData &meta_data) {
// Lower layers deliver packets in one piece; the payload is passed up without copying
if (!data.is_contiguous()) {
ESP_LOGW(TAG, "%s: Packet split across buffers is not supported", this->link_->get_name());
return;
}
auto packet = data.as_span();
if (packet.size() < HEADER_SIZE) {
ESP_LOGV(TAG, "%s: Packet of %zu bytes is shorter than the address header, dropped", this->link_->get_name(),
packet.size());
return;
}
uint16_t destination = encode_uint16(packet[1], packet[0]);
uint16_t source = encode_uint16(packet[3], packet[2]);
if (source == this->address_ || (destination != this->address_ && destination != BROADCAST_ADDRESS))
return;
AddressingPort *source_port = nullptr;
AddressingPort *broadcast_port = nullptr;
for (auto *port : this->ports_) {
if (port->destination_ == source) {
source_port = port;
} else if (port->destination_ == BROADCAST_ADDRESS) {
broadcast_port = port;
}
}
// A broadcast goes to the broadcast port, even from a device with a port of its own, as that port's layers expect
// only its point-to-point packets; anything else goes to the source's port, or to the broadcast port if it has none
AddressingPort *target = source_port;
if (broadcast_port != nullptr && (destination == BROADCAST_ADDRESS || source_port == nullptr))
target = broadcast_port;
if (target == nullptr) {
ESP_LOGV(TAG, "%s: Packet from 0x%04X dropped", this->link_->get_name(), source);
return;
}
PacketMetaData received = meta_data;
received.source_address = source;
received.destination_address = destination;
target->on_receive_from_interface_(PacketBuffer(packet.data() + HEADER_SIZE, packet.size() - HEADER_SIZE), received);
}
AddressingPort::AddressingPort(BasicAddressing *addressing, uint16_t destination, bool only_port)
: PacketInterface(
remaining_packet_size(addressing->get_link()->get_max_packet_size(), BasicAddressing::HEADER_SIZE)),
addressing_(addressing),
destination_(destination) {
if (only_port) {
this->stack_on_(addressing->get_link(), CAPABILITY_ADDRESSING);
} else {
this->inherit_from_(addressing->get_link(), CAPABILITY_ADDRESSING);
}
// A link with no room leaves the layers above none either, so they fail their setup
addressing->add_port(this);
}
void AddressingPort::dump_config() {
ESP_LOGCONFIG(TAG,
"Addressing Port '%s':\n"
" Max Packet Size: %zu",
this->name_, this->max_packet_size_);
if (this->destination_ == BasicAddressing::BROADCAST_ADDRESS) {
ESP_LOGCONFIG(TAG, " Destination: broadcast");
} else {
ESP_LOGCONFIG(TAG, " Destination: 0x%04X", this->destination_);
}
}
} // namespace esphome::packet_interface
#endif
@@ -0,0 +1,69 @@
#pragma once
#include "esphome/core/defines.h"
#ifdef USE_PACKET_INTERFACE_BASIC_ADDRESSING
#include "esphome/core/helpers.h"
#include "packet_interface.h"
namespace esphome::packet_interface {
class AddressingPort;
/**
* An addressing layer for channels shared by several devices, such as an RS485 bus or a LoRa radio.
*
* Each packet starts with a 16-bit destination address and a 16-bit source address, little endian.
* BROADCAST_ADDRESS sends to every device. A packet is dropped unless it is addressed to this device or broadcast, and
* a packet from this device's own address, such as an echo on a half-duplex bus, is dropped too.
*
* Above this layer are one or more ports, each the bottom of its own stack of layers. A port with a destination
* address is a point-to-point link to that device: it sends to it and receives only its packets, so the reliability
* and session layers and the rolling code, which work between two devices only, can be used above it. A port with
* the broadcast destination sends to every device, and receives every broadcast, as well as packets from devices
* that have no port of their own. Without a broadcast port, a device's broadcasts go to its own port.
*/
class BasicAddressing : public Component {
public:
static constexpr uint16_t BROADCAST_ADDRESS = 0xFFFF;
/// Destination and source addresses.
static constexpr size_t HEADER_SIZE = 4;
BasicAddressing(PacketInterface *link, uint16_t address) : link_(link), address_(address) {}
void setup() override;
void dump_config() override;
PacketInterface *get_link() const { return this->link_; }
void add_port(AddressingPort *port) { this->ports_.push_back(port); }
/// Sends \p data to \p destination, or to the destination in \p meta_data if set.
bool send(const PacketBuffer &data, const PacketMetaData &meta_data, uint16_t destination);
protected:
void receive_(const PacketBuffer &data, const PacketMetaData &meta_data);
PacketInterface *link_;
const uint16_t address_;
StaticVector<AddressingPort *, PACKET_INTERFACE_ADDRESSING_PORT_COUNT> ports_;
};
/// The bottom of the stack of layers for one destination, or for broadcast, on an addressed channel.
class AddressingPort : public PacketInterface {
public:
/// With \p only_port, the port is the top of the link, so top() on the link reaches this port's stack.
AddressingPort(BasicAddressing *addressing, uint16_t destination, bool only_port);
void dump_config() override;
bool send_to_interface(const PacketBuffer &data, const PacketMetaData &meta_data) override {
return this->addressing_->send(data, meta_data, this->destination_);
}
uint16_t get_destination() const { return this->destination_; }
protected:
friend class BasicAddressing;
BasicAddressing *addressing_;
const uint16_t destination_;
};
} // namespace esphome::packet_interface
#endif
@@ -0,0 +1,160 @@
#include "basic_fragmentation.h"
#ifdef USE_PACKET_INTERFACE_BASIC_FRAGMENTATION
#include <algorithm>
#include <cstring>
#include "esphome/core/helpers.h"
#include "esphome/core/log.h"
namespace esphome::packet_interface {
ESPHOME_LOG_TAG(TAG, "packet_interface.fragmentation");
static constexpr uint8_t LAST_FRAGMENT = 0x80;
static constexpr uint8_t INDEX_MASK = 0x7F;
BasicFragmentation::BasicFragmentation(PacketInterface *lower, size_t max_packet_size)
: PacketInterface(max_packet_size), lower_(lower) {
this->stack_on_(lower, CAPABILITY_FRAGMENTATION);
}
size_t BasicFragmentation::fragments_for_(size_t size) const {
size_t fragment_size = this->fragment_size_();
if (fragment_size == 0)
return SIZE_MAX;
return std::max<size_t>(1, (size + fragment_size - 1) / fragment_size);
}
void BasicFragmentation::setup() {
if (this->fragment_size_() == 0) {
this->mark_failed(LOG_STR("max_packet_size too small for the configured layers"));
return;
}
if (this->fragments_for_(this->max_packet_size_) > MAX_FRAGMENTS) {
this->mark_failed(LOG_STR("max_packet_size needs more than 128 fragments"));
return;
}
// Checked now, while the layer below holds nothing; a failed layer below reports its own failure instead
this->queue_too_short_ =
!this->lower_->is_failed() && this->fragments_for_(this->max_packet_size_) > this->lower_->get_send_capacity();
RAMAllocator<uint8_t> allocator;
this->tx_buffer_ = allocator.allocate(this->fragment_size_() + this->max_packet_size_);
if (this->tx_buffer_ == nullptr) {
this->mark_failed(LOG_STR("Not enough memory for the packet buffers"));
return;
}
this->rx_buffer_ = this->tx_buffer_ + this->fragment_size_();
// Numbering from a random start makes it unlikely that a packet joins fragments from before a restart
this->tx_number_ = random_uint32();
this->lower_->add_packet_interface_listener(
[this](const PacketBuffer &data, const PacketMetaData &meta_data) { this->receive_(data, meta_data); });
}
void BasicFragmentation::dump_config() {
// A link with no room for fragments fails setup, which core reports below
size_t fragments = this->fragment_size_() == 0 ? 0 : this->fragments_for_(this->max_packet_size_);
ESP_LOGCONFIG(TAG,
"Basic Fragmentation on '%s':\n"
" Max Packet Size: %zu\n"
" Fragment Size: %zu\n"
" Fragments in the Largest Packet: %zu",
this->name_, this->max_packet_size_, this->fragment_size_(), fragments);
if (this->queue_too_short_) {
ESP_LOGW(TAG, " The reliability queue is too short for the largest packets, which cannot be sent");
}
}
bool BasicFragmentation::send_to_interface(const PacketBuffer &data, const PacketMetaData &meta_data) {
if (this->tx_buffer_ == nullptr)
return false;
size_t size = data.size();
if (size > this->max_packet_size_) {
ESP_LOGW(TAG, "%s: Packet of %zu bytes is larger than %zu", this->name_, size, this->max_packet_size_);
return false;
}
size_t fragments = this->fragments_for_(size);
if (fragments > this->lower_->get_send_capacity()) {
ESP_LOGV(TAG, "%s: No room for %zu fragments", this->name_, fragments);
return false;
}
uint8_t header[HEADER_SIZE] = {this->tx_number_++, 0};
if (fragments == 1) {
header[1] = LAST_FRAGMENT;
return this->lower_->send_to_interface(PacketBuffer(header, HEADER_SIZE, &data, nullptr, 0), meta_data);
}
size_t fragment_size = this->fragment_size_();
for (size_t index = 0; index != fragments; index++) {
size_t length = data.copy_to(this->tx_buffer_, fragment_size, index * fragment_size);
header[1] = index | (index + 1 == fragments ? LAST_FRAGMENT : 0);
PacketBuffer body(this->tx_buffer_, length);
if (!this->lower_->send_to_interface(PacketBuffer(header, HEADER_SIZE, &body, nullptr, 0), meta_data)) {
// The peer drops the part already sent when the next packet starts
ESP_LOGV(TAG, "%s: Fragment %zu of %zu not accepted", this->name_, index, fragments);
return false;
}
}
return true;
}
void BasicFragmentation::receive_(const PacketBuffer &data, const PacketMetaData &meta_data) {
// Lower layers deliver packets in one piece; a packet in one fragment is passed up without copying
if (!data.is_contiguous()) {
ESP_LOGW(TAG, "%s: Packet split across buffers is not supported", this->name_);
return;
}
auto fragment = data.as_span();
if (fragment.size() < HEADER_SIZE) {
ESP_LOGV(TAG, "%s: Fragment of %zu bytes is shorter than its header, dropped", this->name_, fragment.size());
return;
}
uint8_t number = fragment[0];
uint8_t index = fragment[1] & INDEX_MASK;
bool last = (fragment[1] & LAST_FRAGMENT) != 0;
const uint8_t *payload = fragment.data() + HEADER_SIZE;
size_t length = fragment.size() - HEADER_SIZE;
if (index == 0 && last) {
// A packet in one fragment leaves a packet being joined from another device intact
if (this->rx_joining_ && meta_data.source_address == this->rx_source_) {
ESP_LOGV(TAG, "%s: Incomplete packet %u dropped", this->name_, this->rx_number_);
this->rx_joining_ = false;
}
if (length > this->max_packet_size_) {
ESP_LOGV(TAG, "%s: Packet %u is larger than %zu, dropped", this->name_, number, this->max_packet_size_);
return;
}
this->on_receive_from_interface_(PacketBuffer(payload, length), meta_data);
return;
}
if (index == 0) {
if (this->rx_joining_) {
ESP_LOGV(TAG, "%s: Incomplete packet %u dropped", this->name_, this->rx_number_);
}
this->rx_joining_ = true;
this->rx_number_ = number;
this->rx_source_ = meta_data.source_address;
this->rx_next_index_ = 0;
this->rx_length_ = 0;
} else if (this->rx_joining_ && meta_data.source_address != this->rx_source_) {
// Only this fragment is lost, so of two devices sending at once, the one that started last still gets through
ESP_LOGV(TAG, "%s: Fragment %u from 0x%04X dropped", this->name_, index, meta_data.source_address);
return;
} else if (!this->rx_joining_ || number != this->rx_number_ || index != this->rx_next_index_) {
ESP_LOGV(TAG, "%s: Fragment %u of packet %u dropped", this->name_, index, number);
this->rx_joining_ = false;
return;
}
if (this->rx_length_ + length > this->max_packet_size_) {
ESP_LOGV(TAG, "%s: Packet %u is larger than %zu, dropped", this->name_, number, this->max_packet_size_);
this->rx_joining_ = false;
return;
}
memcpy(this->rx_buffer_ + this->rx_length_, payload, length);
this->rx_length_ += length;
this->rx_next_index_++;
if (last) {
this->rx_joining_ = false;
this->on_receive_from_interface_(PacketBuffer(this->rx_buffer_, this->rx_length_), meta_data);
}
}
} // namespace esphome::packet_interface
#endif
@@ -0,0 +1,60 @@
#pragma once
#include "esphome/core/defines.h"
#ifdef USE_PACKET_INTERFACE_BASIC_FRAGMENTATION
#include "packet_interface.h"
namespace esphome::packet_interface {
/**
* A fragmentation layer: packets up to this layer's max_packet_size are split into fragments that fit the layer
* below, and joined again at the other end.
*
* Each fragment starts with a 2 byte header: an 8-bit packet number, then the fragment's index in bits 0-6, with bit
* 7 set on the last fragment. A packet that fits in one fragment still has the header, and is sent and passed up
* without being copied.
*
* A packet is started only if the layer below can take all of its fragments now (get_send_capacity()), so a full
* reliability queue does not leave the peer with part of a packet. The receiver joins one packet at a time, and drops
* it when a fragment is missing or arrives with another packet number, or when the same source, or another one,
* starts a new packet of several fragments. Other fragments from another source address are dropped, leaving the
* packet being joined intact. Above a reliability layer, fragments arrive in order and none are missing.
*/
class BasicFragmentation : public PacketInterface {
public:
/// Packet number, then index and last flag.
static constexpr size_t HEADER_SIZE = 2;
/// The index has 7 bits.
static constexpr size_t MAX_FRAGMENTS = 128;
BasicFragmentation(PacketInterface *lower, size_t max_packet_size);
void setup() override;
void dump_config() override;
bool send_to_interface(const PacketBuffer &data, const PacketMetaData &meta_data) override;
protected:
void receive_(const PacketBuffer &data, const PacketMetaData &meta_data);
size_t fragment_size_() const {
size_t lower_size = this->lower_->get_max_packet_size();
return lower_size > HEADER_SIZE ? lower_size - HEADER_SIZE : 0;
}
size_t fragments_for_(size_t size) const;
PacketInterface *lower_;
// One fragment being sent, and the packet being joined; one block allocated in setup(), and tx_buffer_ stays null
// if that fails
uint8_t *tx_buffer_{nullptr};
uint8_t *rx_buffer_{nullptr};
size_t rx_length_{0};
uint16_t rx_source_{0};
uint8_t tx_number_{0};
uint8_t rx_number_{0};
uint8_t rx_next_index_{0};
bool rx_joining_{false};
bool queue_too_short_{false};
};
} // namespace esphome::packet_interface
#endif
@@ -2,7 +2,7 @@
Each module in this package implements one layer type and registers it with its kind, for example:
@RELIABILITY.register("hdlc", schema=HDLC_SCHEMA)
@RELIABILITY.register("hdlc", schema=HDLC_SCHEMA, single_peer=lambda config: True)
async def build_hdlc(config: ConfigType, lower: MockObj) -> MockObj:
...
@@ -16,17 +16,23 @@ from typing import Any
import esphome.codegen as cg
import esphome.config_validation as cv
from esphome.const import CONF_ENCRYPTION, CONF_KEY, CONF_TYPE
from esphome.const import CONF_ENCRYPTION, CONF_FRAGMENTATION, CONF_KEY, CONF_TYPE
from esphome.cpp_generator import MockObj
from esphome.types import ConfigType
packet_interface_ns = cg.esphome_ns.namespace("packet_interface")
PacketInterface = packet_interface_ns.class_("PacketInterface", cg.Component)
CONF_ADDRESSING = "addressing"
CONF_DESTINATION = "destination"
CONF_PEERS = "peers"
CONF_RELIABILITY = "reliability"
CONF_SESSION = "session"
# Builds a layer above lower and returns it
BROADCAST_ADDRESS = 0xFFFF
# Builds a layer above lower and returns it. An addressing layer instead returns a list of (port, config) pairs, one
# for each stack it carries, where config holds the layers for that stack, or is None for the layers on the link.
LayerBuilder = Callable[[ConfigType, MockObj], Awaitable[Any]]
@@ -38,16 +44,37 @@ def _nothing(config: ConfigType) -> list[str]:
return []
def _same_size(config: ConfigType, size: int) -> int:
return size
def _no_limit(config: ConfigType) -> int | None:
return None
def _fits(config: ConfigType, size: int, capacity: int | None) -> None:
"""Accept any layer below."""
@dataclass(frozen=True)
class LayerType:
"""One implementation of a kind of layer."""
schema: Any
# Returns the schema; called once every layer type is registered, as some schemas contain other layers
schema_factory: Callable[[], Any]
build: LayerBuilder
# True if this configuration keeps state for one peer, so cannot be used with broadcast addressing
single_peer: Callable[[ConfigType], bool]
# True if this configuration works only when the peer sends packets back, which a session layer makes sure of
needs_session: Callable[[ConfigType], bool]
# The components this configuration needs loaded
auto_load: Callable[[ConfigType], list[str]]
# The largest packet this configuration takes, from the largest that the layer below takes, as the C++ works it out
packet_size: Callable[[ConfigType, int], int]
# How many packets this configuration takes at once, or None for no limit
send_capacity: Callable[[ConfigType], int | None]
# Raises cv.Invalid if this configuration cannot work above a layer with this packet size and send capacity
check_lower: Callable[[ConfigType, int, int | None], None]
class LayerKind:
@@ -68,11 +95,19 @@ class LayerKind:
self,
name: str,
*,
schema: Any,
schema: Any = None,
schema_factory: Callable[[], Any] | None = None,
single_peer: Callable[[ConfigType], bool] = _never,
needs_session: Callable[[ConfigType], bool] = _never,
auto_load: Callable[[ConfigType], list[str]] = _nothing,
packet_size: Callable[[ConfigType, int], int] = _same_size,
send_capacity: Callable[[ConfigType], int | None] = _no_limit,
check_lower: Callable[[ConfigType, int, int | None], None] = _fits,
) -> Callable[[LayerBuilder], LayerBuilder]:
"""Register a type with its schema."""
"""Register a type with its schema, or with a function returning a schema that contains other layers."""
def fixed_schema() -> Any:
return schema
def decorator(build: LayerBuilder) -> LayerBuilder:
if self.frozen:
@@ -84,7 +119,16 @@ class LayerKind:
raise ValueError(
f"{self.key} layer type '{name}' is already registered"
)
self.types[name] = LayerType(schema, build, needs_session, auto_load)
self.types[name] = LayerType(
schema_factory or fixed_schema,
build,
single_peer,
needs_session,
auto_load,
packet_size,
send_capacity,
check_lower,
)
return build
return decorator
@@ -103,7 +147,7 @@ class LayerKind:
return cv.All(
self._expand,
cv.typed_schema(
{name: layer.schema for name, layer in self.types.items()},
{name: layer.schema_factory() for name, layer in self.types.items()},
default_type=self.default_type,
),
)
@@ -112,11 +156,15 @@ class LayerKind:
return self.types[config[CONF_TYPE]]
# An addressing type's configuration has either a destination or a list of peers, each with an id, a destination
# and its own layers.
ADDRESSING = LayerKind(CONF_ADDRESSING, "basic")
ENCRYPTION = LayerKind(CONF_ENCRYPTION, "xxtea", shorthand=CONF_KEY)
RELIABILITY = LayerKind(CONF_RELIABILITY, "hdlc")
FRAGMENTATION = LayerKind(CONF_FRAGMENTATION, "basic")
SESSION = LayerKind(CONF_SESSION, "basic")
# The kinds stacked above the link, from the bottom up
STACK_ORDER = (ENCRYPTION, RELIABILITY, SESSION)
# The kinds stacked above the link, or above each port of an addressing layer, from the bottom up
STACK_ORDER = (ENCRYPTION, RELIABILITY, FRAGMENTATION, SESSION)
def stack_schema() -> dict:
@@ -139,6 +187,41 @@ def check_session(config: ConfigType) -> None:
)
def check_single_peer(config: ConfigType, destination: int) -> None:
"""Layers that keep state for one peer need a destination other than broadcast."""
if destination != BROADCAST_ADDRESS:
return
for kind in STACK_ORDER:
if (layer := config.get(kind.key)) is not None and kind.get_type(
layer
).single_peer(layer):
raise cv.Invalid(
f"This '{kind.key}' works between two devices only, so it needs a '{CONF_DESTINATION}' other than "
"broadcast",
path=[kind.key],
)
def remaining_packet_size(size: int, used: int) -> int:
return max(size - used, 0)
def check_packet_sizes(config: ConfigType, size: int) -> None:
"""Check each layer in config against the layer below it; size is the largest packet that the stack sits on takes."""
capacity = None
for kind in STACK_ORDER:
if (layer := config.get(kind.key)) is None:
continue
layer_type = kind.get_type(layer)
try:
layer_type.check_lower(layer, size, capacity)
except cv.Invalid as err:
err.prepend([kind.key])
raise
size = layer_type.packet_size(layer, size)
capacity = layer_type.send_capacity(layer)
def stack_auto_load(config: ConfigType) -> set[str]:
"""The components that the layers configured in config need loaded."""
loads: set[str] = set()
@@ -0,0 +1,132 @@
"""Addressing layer with 16-bit addresses, carrying one stack of layers or one for each of several peers."""
from typing import Any
import esphome.codegen as cg
import esphome.config_validation as cv
from esphome.const import CONF_ADDRESS, CONF_ID
from esphome.core import ID
from esphome.cpp_generator import MockObj
from esphome.types import ConfigType
from . import (
ADDRESSING,
BROADCAST_ADDRESS,
CONF_DESTINATION,
CONF_PEERS,
PacketInterface,
check_session,
check_single_peer,
packet_interface_ns,
remaining_packet_size,
stack_schema,
)
CONF_PORT_ID = "port_id"
# The destination and source addresses
HEADER_SIZE = 4
BasicAddressing = packet_interface_ns.class_("BasicAddressing", cg.Component)
AddressingPort = packet_interface_ns.class_("AddressingPort", PacketInterface)
_request_port_slot = cg.slot_counter("PACKET_INTERFACE_ADDRESSING_PORT_COUNT")
def _destination(value: Any) -> int:
"""A device address, or broadcast."""
if isinstance(value, str) and value.lower() == "broadcast":
return BROADCAST_ADDRESS
return cv.hex_int_range(1, BROADCAST_ADDRESS)(value)
def _validate_peer(config: ConfigType) -> ConfigType:
check_session(config)
check_single_peer(config, config[CONF_DESTINATION])
return config
def _unique_destinations(peers: list[ConfigType]) -> list[ConfigType]:
destinations = [peer[CONF_DESTINATION] for peer in peers]
if len(set(destinations)) != len(destinations):
raise cv.Invalid("Each peer must have a different destination")
return peers
def _not_own_address(config: ConfigType) -> ConfigType:
"""Packets from this device's own address are dropped as echoes, so it cannot be a destination."""
address = config[CONF_ADDRESS]
if config.get(CONF_DESTINATION) == address:
raise cv.Invalid(
"Must not be this device's own address", path=[CONF_DESTINATION]
)
for index, peer in enumerate(config.get(CONF_PEERS, [])):
if peer[CONF_DESTINATION] == address:
raise cv.Invalid(
"Must not be this device's own address",
path=[CONF_PEERS, index, CONF_DESTINATION],
)
return config
# A function, as each peer holds the other layers, whose schemas exist only once every layer type is registered
def _basic_addressing_schema() -> cv.Schema:
peer_schema = cv.All(
cv.Schema(
{
cv.Required(CONF_ID): cv.declare_id(AddressingPort),
cv.Required(CONF_DESTINATION): _destination,
**stack_schema(),
}
),
_validate_peer,
)
return cv.All(
cv.Schema(
{
cv.GenerateID(): cv.declare_id(BasicAddressing),
cv.GenerateID(CONF_PORT_ID): cv.declare_id(AddressingPort),
cv.Required(CONF_ADDRESS): cv.hex_int_range(1, BROADCAST_ADDRESS - 1),
cv.Optional(CONF_DESTINATION): _destination,
cv.Optional(CONF_PEERS): cv.All(
cv.ensure_list(peer_schema), cv.Length(min=1), _unique_destinations
),
}
).extend(cv.COMPONENT_SCHEMA),
cv.has_at_most_one_key(CONF_DESTINATION, CONF_PEERS),
_not_own_address,
)
async def _new_port(
port_id: ID, addressing: MockObj, destination: int, only_port: bool
) -> MockObj:
_request_port_slot(str(addressing))
port = cg.new_Pvariable(port_id, addressing, destination, only_port)
await cg.register_component(port, {})
return port
@ADDRESSING.register(
"basic",
schema_factory=_basic_addressing_schema,
packet_size=lambda config, size: remaining_packet_size(size, HEADER_SIZE),
)
async def build_basic_addressing(
config: ConfigType, link: MockObj
) -> list[tuple[MockObj, ConfigType | None]]:
cg.add_define("USE_PACKET_INTERFACE_BASIC_ADDRESSING")
hub = cg.new_Pvariable(config[CONF_ID], link, config[CONF_ADDRESS])
await cg.register_component(hub, config)
if (peers := config.get(CONF_PEERS)) is None:
port = await _new_port(
config[CONF_PORT_ID], hub, config[CONF_DESTINATION], True
)
return [(port, None)]
stacks = []
# Each peer has its own stack, named after the peer for logs and saved preferences
for peer in peers:
port = await _new_port(peer[CONF_ID], hub, peer[CONF_DESTINATION], False)
cg.add(port.set_name(peer[CONF_ID].id))
stacks.append((port, peer))
return stacks
@@ -0,0 +1,56 @@
"""Fragmentation layer that splits packets larger than the layers below allow, and joins them again."""
import math
import esphome.codegen as cg
from esphome.components.const import CONF_MAX_PACKET_SIZE
import esphome.config_validation as cv
from esphome.const import CONF_ID
from esphome.cpp_generator import MockObj
from esphome.types import ConfigType
from . import FRAGMENTATION, PacketInterface, packet_interface_ns
# The packet number, and the fragment's index with the flag on the last fragment
HEADER_SIZE = 2
MAX_FRAGMENTS = 128
BasicFragmentation = packet_interface_ns.class_("BasicFragmentation", PacketInterface)
BASIC_FRAGMENTATION_SCHEMA = cv.Schema(
{
cv.GenerateID(): cv.declare_id(BasicFragmentation),
# The packet being joined is held in a buffer of this size
cv.Optional(CONF_MAX_PACKET_SIZE, default=1024): cv.int_range(16, 16384),
}
).extend(cv.COMPONENT_SCHEMA)
def _check_lower(config: ConfigType, size: int, capacity: int | None) -> None:
"""The largest packet must fit in the fragments that the layer below can take at once."""
fragment_size = size - HEADER_SIZE
if fragment_size <= 0:
raise cv.Invalid("The layers below leave no room for fragments")
fragments = max(1, math.ceil(config[CONF_MAX_PACKET_SIZE] / fragment_size))
limit = MAX_FRAGMENTS if capacity is None else min(MAX_FRAGMENTS, capacity)
if fragments > limit:
raise cv.Invalid(
f"A packet of {config[CONF_MAX_PACKET_SIZE]} bytes needs {fragments} fragments of {fragment_size} bytes, "
f"but at most {limit} can be sent at once",
path=[CONF_MAX_PACKET_SIZE],
)
# Fragments from different devices are told apart by the addressing layer's source address, so of two devices sending
# fragmented packets at once, one still gets through; without addressing, the receiver can drop both
@FRAGMENTATION.register(
"basic",
schema=BASIC_FRAGMENTATION_SCHEMA,
packet_size=lambda config, size: config[CONF_MAX_PACKET_SIZE],
check_lower=_check_lower,
)
async def build_basic_fragmentation(config: ConfigType, lower: MockObj) -> MockObj:
cg.add_define("USE_PACKET_INTERFACE_BASIC_FRAGMENTATION")
var = cg.new_Pvariable(config[CONF_ID], lower, config[CONF_MAX_PACKET_SIZE])
await cg.register_component(var, config)
return var
@@ -57,6 +57,7 @@ BASIC_SESSION_SCHEMA = cv.All(
@SESSION.register(
"basic",
schema=BASIC_SESSION_SCHEMA,
single_peer=lambda config: True,
auto_load=lambda config: ["binary_sensor"] if CONF_BINARY_SENSOR in config else [],
)
async def build_basic_session(config: ConfigType, lower: MockObj) -> MockObj:
@@ -6,12 +6,15 @@ from esphome.const import CONF_ID, CONF_TIMEOUT
from esphome.cpp_generator import MockObj
from esphome.types import ConfigType
from . import RELIABILITY, PacketInterface, packet_interface_ns
from . import RELIABILITY, PacketInterface, packet_interface_ns, remaining_packet_size
CONF_ACK_DELAY = "ack_delay"
CONF_MAX_QUEUE_LENGTH = "max_queue_length"
CONF_MAX_RETRIES = "max_retries"
# The control byte and the CRC-16
FRAME_OVERHEAD = 3
HdlcReliability = packet_interface_ns.class_("HdlcReliability", PacketInterface)
@@ -47,7 +50,13 @@ HDLC_SCHEMA = cv.All(
)
@RELIABILITY.register("hdlc", schema=HDLC_SCHEMA)
@RELIABILITY.register(
"hdlc",
schema=HDLC_SCHEMA,
single_peer=lambda config: True,
packet_size=lambda config, size: remaining_packet_size(size, FRAME_OVERHEAD),
send_capacity=lambda config: config[CONF_MAX_QUEUE_LENGTH],
)
async def build_hdlc(config: ConfigType, lower: MockObj) -> MockObj:
cg.add_define("USE_PACKET_INTERFACE_HDLC")
var = cg.new_Pvariable(config[CONF_ID], lower)
@@ -8,7 +8,7 @@ from esphome.const import CONF_ID, CONF_KEY
from esphome.cpp_generator import MockObj
from esphome.types import ConfigType
from . import ENCRYPTION, PacketInterface, packet_interface_ns
from . import ENCRYPTION, PacketInterface, packet_interface_ns, remaining_packet_size
CONF_ROLLING_CODE_ENABLE = "rolling_code_enable"
@@ -18,6 +18,10 @@ CONF_ROLLING_CODE_ENABLE = "rolling_code_enable"
KEY_SALT = b"esphome packet_interface xxtea"
KEY_ITERATIONS = 600_000
# The check word, and with the rolling code the boot numbers and counter
CHECK_SIZE = 4
ROLLING_SIZE = 12
XxteaEncryption = packet_interface_ns.class_("XxteaEncryption", PacketInterface)
XXTEA_SCHEMA = cv.Schema(
@@ -35,12 +39,19 @@ def derive_key(text: str) -> list[int]:
return [int.from_bytes(digest[i : i + 4], "little") for i in range(0, 32, 4)]
# The rolling code keeps the last counter of one peer, and accepts a packet only once the peer has sent back this
# device's boot number
@ENCRYPTION.register(
"xxtea",
schema=XXTEA_SCHEMA,
# The rolling code accepts a packet only once the peer has sent back this device's boot number
single_peer=lambda config: config[CONF_ROLLING_CODE_ENABLE],
needs_session=lambda config: config[CONF_ROLLING_CODE_ENABLE],
auto_load=lambda config: ["xxtea"],
# XXTEA works on whole 32-bit words
packet_size=lambda config, size: remaining_packet_size(
size & ~3,
CHECK_SIZE + (ROLLING_SIZE if config[CONF_ROLLING_CODE_ENABLE] else 0),
),
)
async def build_xxtea(config: ConfigType, lower: MockObj) -> MockObj:
cg.add_define("USE_PACKET_INTERFACE_XXTEA")
@@ -7,7 +7,12 @@ from esphome.components.packet_transport import (
import esphome.config_validation as cv
from esphome.types import ConfigType
from .. import CONF_PACKET_INTERFACE, PacketInterface, packet_interface_ns
from .. import (
CONF_PACKET_INTERFACE,
PacketInterface,
final_validate_interface,
packet_interface_ns,
)
CODEOWNERS = ["@clydebarrow"]
DEPENDENCIES = ["packet_interface"]
@@ -22,6 +27,8 @@ CONFIG_SCHEMA = transport_schema(PacketInterfaceTransport).extend(
}
)
FINAL_VALIDATE_SCHEMA = final_validate_interface(CONF_PACKET_INTERFACE)
async def to_code(config: ConfigType) -> None:
interface = await cg.get_variable(config[CONF_PACKET_INTERFACE])
+5
View File
@@ -76,6 +76,8 @@
#define USE_DISPLAY
#define DISPLAY_PAGE_CHANGE_CALLBACK_COUNT 1
#define DUCO_DEVICE_COUNT 4
#define ENTITY_TRANSPORT_REMOTE_BINARY_SENSOR_COUNT 1
#define ENTITY_TRANSPORT_REMOTE_SENSOR_COUNT 1
#define USE_EMONTX_VERSION
#define USE_EMONTX_VERSION_TEXT_SENSOR
#define USE_ENTITY_DEVICE_CLASS
@@ -274,6 +276,9 @@
#define USE_OTA_STATE_LISTENER
#define USE_OUTPUT
#define USE_OUTPUT_FLOAT_POWER_SCALING
#define PACKET_INTERFACE_ADDRESSING_PORT_COUNT 2
#define USE_PACKET_INTERFACE_BASIC_ADDRESSING
#define USE_PACKET_INTERFACE_BASIC_FRAGMENTATION
#define USE_PACKET_INTERFACE_BASIC_SESSION
#define USE_PACKET_INTERFACE_HDLC
#define USE_PACKET_INTERFACE_XXTEA
@@ -0,0 +1,19 @@
import esphome.codegen as cg
from esphome.core import CORE
from esphome.types import ConfigType
from tests.testing_helpers import ComponentManifestOverride
def override_manifest(manifest: ComponentManifestOverride) -> None:
async def to_code_testing(config: ConfigType) -> None:
cg.add_define("ENTITY_TRANSPORT_REMOTE_BINARY_SENSOR_COUNT", 2)
cg.add_define("ENTITY_TRANSPORT_REMOTE_SENSOR_COUNT", 2)
cg.add_define("USE_PACKET_INTERFACE_BASIC_SESSION")
# No entity platforms are configured, so count the test's own entities to emit USE_SENSOR and friends
CORE.platform_counts["sensor"] += 2
CORE.platform_counts["binary_sensor"] += 2
manifest.to_code = to_code_testing
# A multi_conf component gets an empty list in the test build, so to_code would not run
manifest.multi_conf = False
manifest.dependencies = manifest.dependencies + ["sensor", "binary_sensor"]
@@ -0,0 +1,33 @@
packet_interface:
- platform: uart
id: entity_transport_link
reliability:
sensor:
- platform: template
id: entity_transport_sensor
lambda: return 1.0;
- platform: entity_transport
provider: remote-device
remote_id: entity_transport_sensor
name: Remote sensor
binary_sensor:
- platform: template
id: entity_transport_binary_sensor
lambda: return true;
- platform: entity_transport
id: remote_binary_sensor
provider: remote-device
remote_id: door
name: Remote binary sensor
entity_transport:
- packet_interface: entity_transport_link
update_interval: 60s
timeout: 3min
sensors:
- entity_transport_sensor
binary_sensors:
- id: entity_transport_binary_sensor
broadcast_id: door
@@ -0,0 +1,552 @@
#include <gtest/gtest.h>
#include <cmath>
#include <cstdint>
#include <cstring>
#include <initializer_list>
#include <memory>
#include <vector>
#include "esphome/components/entity_transport/entity_transport.h"
#include "esphome/core/application.h"
namespace esphome::entity_transport::testing {
// Copies a packet into a vector, for comparing in tests
static std::vector<uint8_t> to_vector(const esphome::packet_interface::PacketBuffer &data) {
std::vector<uint8_t> result(data.size());
data.copy_to(result.data(), result.size());
return result;
}
using packet_interface::PacketBuffer;
using packet_interface::PacketMetaData;
using Packet = std::vector<uint8_t>;
static constexpr const char *DEVICE_NAME = "this-device";
static constexpr uint8_t PACKET_DICTIONARY = 0x11;
static constexpr uint8_t PACKET_DATA = 0x12;
static constexpr uint8_t PACKET_REQUEST = 0x13;
// A packet interface that records what is sent, can refuse to send, and can act as a session layer
class MockInterface : public packet_interface::PacketInterface {
public:
explicit MockInterface(size_t max_packet_size = 64) : packet_interface::PacketInterface(max_packet_size) {}
bool send_to_interface(const PacketBuffer &data, const PacketMetaData &) override {
if (!this->accept || this->budget == 0)
return false;
this->budget--;
this->sent.push_back(to_vector(data));
return true;
}
void inject(const Packet &packet) { this->on_receive_from_interface_(PacketBuffer(packet), {}); }
void set_capabilities(uint8_t capabilities) { this->capabilities_ = capabilities; }
void report_session(bool connected) { this->session_callback_.call(connected); }
// Deliver everything sent so far to another interface
void deliver_to(MockInterface &other) {
auto packets = std::move(this->sent);
this->sent.clear();
for (auto &packet : packets)
other.inject(packet);
}
std::vector<Packet> sent;
bool accept{true};
size_t budget{SIZE_MAX}; // packets accepted before refusing, like a full queue
};
class TestTransport : public EntityTransport {
public:
using EntityTransport::EntityTransport;
uint32_t get_dictionary_id() const { return this->dictionary_id_; }
uint32_t get_update_interval() const { return this->update_interval_ms_; }
using EntityTransport::clear_remote_states_;
};
static Packet header(uint8_t packet_type, uint32_t id) {
return {packet_type, static_cast<uint8_t>(id), static_cast<uint8_t>(id >> 8), static_cast<uint8_t>(id >> 16),
static_cast<uint8_t>(id >> 24)};
}
// The start of a dictionary packet from this device's name: the header, the part byte, then the name
static Packet dictionary_start(uint32_t id, uint8_t part = 0x80) {
Packet packet = header(PACKET_DICTIONARY, id);
packet.push_back(part);
packet.push_back(strlen(DEVICE_NAME));
packet.insert(packet.end(), DEVICE_NAME, DEVICE_NAME + strlen(DEVICE_NAME));
return packet;
}
class EntityTransportTest : public ::testing::Test {
protected:
void SetUp() override {
App.~Application();
new (&App) Application();
static char name[] = "this-device";
static char friendly_name[] = "";
App.pre_setup(name, sizeof(name) - 1, friendly_name, 0);
}
void TearDown() override {
App.~Application();
new (&App) Application(); // later tests and the test listener use App
}
// The sender, with a sensor and a binary sensor
std::unique_ptr<TestTransport> make_sender_(MockInterface *link = nullptr) {
auto transport = std::make_unique<TestTransport>(link != nullptr ? link : &this->link_a_);
transport->set_entity_count(2);
transport->add_sensor("temp", &this->temp_);
transport->add_binary_sensor("door", &this->door_);
transport->setup();
return transport;
}
// The receiver, showing the sender's entities; both run on this device, so they share its name
std::unique_ptr<TestTransport> make_receiver_(const char *provider = DEVICE_NAME, uint32_t timeout_ms = 0) {
auto transport = std::make_unique<TestTransport>(&this->link_b_);
transport->set_remote_timeout(timeout_ms);
transport->add_remote_sensor(provider, "temp", &this->remote_temp_);
transport->add_remote_binary_sensor(provider, "door", &this->remote_door_);
transport->setup();
return transport;
}
MockInterface link_a_, link_b_;
sensor::Sensor temp_, remote_temp_;
binary_sensor::BinarySensor door_, remote_door_;
};
TEST_F(EntityTransportTest, SendsDictionaryThenStates) {
this->temp_.publish_state(21.5f);
this->door_.publish_state(true);
auto sender = this->make_sender_();
sender->loop();
uint32_t id = sender->get_dictionary_id();
Packet dictionary = dictionary_start(id);
dictionary.insert(dictionary.end(), {1, 0, 1, 4, 't', 'e', 'm', 'p', 2, 0, 1, 4, 'd', 'o', 'o', 'r'});
Packet data = header(PACKET_DATA, id);
// Tags: sensor 0 with a 4 byte state, then binary sensor 0 with its state in the top bit
data.insert(data.end(), {0x00, 0x21, 0x00, 0x00, 0xAC, 0x41, 0x00, 0x82});
ASSERT_EQ(this->link_a_.sent.size(), 2);
EXPECT_EQ(this->link_a_.sent[0], dictionary);
EXPECT_EQ(this->link_a_.sent[1], data);
}
TEST_F(EntityTransportTest, SendsOnlyChangedStates) {
auto sender = this->make_sender_();
sender->loop();
this->link_a_.sent.clear();
this->temp_.publish_state(3.0f);
sender->loop();
Packet data = header(PACKET_DATA, sender->get_dictionary_id());
data.insert(data.end(), {0x00, 0x21, 0x00, 0x00, 0x40, 0x40});
ASSERT_EQ(this->link_a_.sent.size(), 1);
EXPECT_EQ(this->link_a_.sent[0], data);
}
TEST_F(EntityTransportTest, StatesReachRemoteEntities) {
this->temp_.publish_state(12.25f);
this->door_.publish_state(true);
auto sender = this->make_sender_();
auto receiver = this->make_receiver_();
sender->loop();
this->link_a_.deliver_to(this->link_b_);
EXPECT_EQ(this->remote_temp_.get_state(), 12.25f);
EXPECT_TRUE(this->remote_door_.state);
this->door_.invalidate_state();
this->temp_.publish_state(NAN);
sender->loop();
ASSERT_EQ(this->link_a_.sent.size(), 1);
// Both records have no state, and so no payload
EXPECT_EQ(Packet(this->link_a_.sent[0].begin() + 5, this->link_a_.sent[0].end()), (Packet{0x00, 0x61, 0x00, 0x62}));
this->link_a_.deliver_to(this->link_b_);
EXPECT_FALSE(this->remote_door_.has_state());
EXPECT_TRUE(std::isnan(this->remote_temp_.get_state()));
}
TEST_F(EntityTransportTest, SplitsAcrossPackets) {
MockInterface link(5 + 2 + strlen(DEVICE_NAME) + 3 + 5);
this->temp_.publish_state(1.0f);
this->door_.publish_state(true);
auto sender = this->make_sender_(&link);
auto receiver = this->make_receiver_();
sender->loop();
EXPECT_EQ(link.sent.size(), 3); // one dictionary packet for each entity, then one data packet
link.deliver_to(this->link_b_);
EXPECT_EQ(this->remote_temp_.get_state(), 1.0f);
EXPECT_TRUE(this->remote_door_.state);
}
TEST_F(EntityTransportTest, RefusedPacketSentAgain) {
this->link_a_.accept = false;
auto sender = this->make_sender_();
sender->loop();
EXPECT_TRUE(this->link_a_.sent.empty());
this->link_a_.accept = true;
sender->loop();
EXPECT_EQ(this->link_a_.sent.size(), 2);
}
TEST_F(EntityTransportTest, LongRefusalWarns) {
this->link_a_.accept = false;
auto sender = this->make_sender_();
{
LoopBlockingGuard guard{nullptr, nullptr, 1000};
}
sender->loop();
{
LoopBlockingGuard guard{nullptr, nullptr, 60999};
}
sender->loop();
EXPECT_FALSE(sender->status_has_warning());
{
LoopBlockingGuard guard{nullptr, nullptr, 61000};
}
sender->loop();
EXPECT_TRUE(sender->status_has_warning());
this->link_a_.accept = true;
sender->loop();
EXPECT_FALSE(sender->status_has_warning());
EXPECT_EQ(this->link_a_.sent.size(), 2);
}
TEST_F(EntityTransportTest, UnknownDictionaryRequested) {
this->temp_.publish_state(5.0f);
auto sender = this->make_sender_();
auto receiver = this->make_receiver_();
sender->loop();
this->link_a_.sent.erase(this->link_a_.sent.begin()); // the dictionary is lost
this->link_a_.deliver_to(this->link_b_);
EXPECT_FALSE(this->remote_temp_.has_state());
ASSERT_EQ(this->link_b_.sent.size(), 1);
EXPECT_EQ(this->link_b_.sent[0], header(PACKET_REQUEST, sender->get_dictionary_id()));
this->link_b_.deliver_to(this->link_a_);
sender->loop();
this->link_a_.deliver_to(this->link_b_);
EXPECT_EQ(this->remote_temp_.get_state(), 5.0f);
}
TEST_F(EntityTransportTest, ChangedDictionaryRequested) {
this->temp_.publish_state(5.0f);
auto sender = this->make_sender_();
auto receiver = this->make_receiver_();
sender->loop();
this->link_a_.deliver_to(this->link_b_);
EXPECT_EQ(this->remote_temp_.get_state(), 5.0f);
// The provider restarts with other entities, and its new dictionary is lost
sensor::Sensor temp;
MockInterface link;
TestTransport changed(&link);
changed.set_entity_count(1);
changed.add_sensor("temp", &temp);
changed.setup();
temp.publish_state(6.0f);
changed.loop();
link.sent.erase(link.sent.begin());
link.deliver_to(this->link_b_);
ASSERT_EQ(this->link_b_.sent.size(), 1);
EXPECT_EQ(this->link_b_.sent[0], header(PACKET_REQUEST, changed.get_dictionary_id()));
this->link_b_.deliver_to(link);
changed.loop();
link.deliver_to(this->link_b_);
EXPECT_EQ(this->remote_temp_.get_state(), 6.0f);
}
TEST_F(EntityTransportTest, MalformedDataNotRequested) {
auto receiver = this->make_receiver_();
// A data packet under an unknown dictionary, cut off inside its first record
Packet data = header(PACKET_DATA, 0x12345678);
data.insert(data.end(), {0x00, 0x21, 0x00});
this->link_b_.inject(data);
EXPECT_TRUE(this->link_b_.sent.empty());
}
TEST_F(EntityTransportTest, EntityLeftOutOfNewDictionaryCleared) {
this->temp_.publish_state(5.0f);
this->door_.publish_state(true);
auto sender = this->make_sender_();
auto receiver = this->make_receiver_();
sender->loop();
this->link_a_.deliver_to(this->link_b_);
EXPECT_TRUE(this->remote_door_.state);
// The provider restarts without the door
sensor::Sensor temp;
MockInterface link;
TestTransport changed(&link);
changed.set_entity_count(1);
changed.add_sensor("temp", &temp);
changed.setup();
temp.publish_state(6.0f);
changed.loop();
link.deliver_to(this->link_b_);
EXPECT_EQ(this->remote_temp_.get_state(), 6.0f);
EXPECT_FALSE(this->remote_door_.has_state());
}
TEST_F(EntityTransportTest, EntityLeftOutOfSeenDictionaryCleared) {
// The provider names only the sensor, then both entities, then goes back to naming only the sensor
sensor::Sensor temp;
MockInterface link;
auto make_temp_only = [&] {
auto transport = std::make_unique<TestTransport>(&link);
transport->set_entity_count(1);
transport->add_sensor("temp", &temp);
transport->setup();
return transport;
};
temp.publish_state(4.0f);
auto receiver = this->make_receiver_();
make_temp_only()->loop();
link.deliver_to(this->link_b_);
this->door_.publish_state(true);
this->make_sender_()->loop();
this->link_a_.deliver_to(this->link_b_);
EXPECT_TRUE(this->remote_door_.state);
make_temp_only()->loop();
link.deliver_to(this->link_b_);
EXPECT_EQ(this->remote_temp_.get_state(), 4.0f);
EXPECT_FALSE(this->remote_door_.has_state());
}
TEST_F(EntityTransportTest, LostDictionaryPartRequested) {
// One entity in each dictionary packet
MockInterface link(5 + 2 + strlen(DEVICE_NAME) + 3 + 5);
this->temp_.publish_state(2.0f);
this->door_.publish_state(true);
auto sender = this->make_sender_(&link);
auto receiver = this->make_receiver_();
sender->loop();
ASSERT_EQ(link.sent.size(), 3);
EXPECT_EQ(link.sent[0][5], 0x00);
EXPECT_EQ(link.sent[1][5], 0x81);
link.sent.erase(link.sent.begin() + 1); // the part naming the binary sensor is lost
link.deliver_to(this->link_b_);
EXPECT_EQ(this->remote_temp_.get_state(), 2.0f);
EXPECT_FALSE(this->remote_door_.has_state());
// The sensor is known, but the dictionary is not complete, so it is asked for
ASSERT_EQ(this->link_b_.sent.size(), 1);
EXPECT_EQ(this->link_b_.sent[0], header(PACKET_REQUEST, sender->get_dictionary_id()));
this->link_b_.deliver_to(link);
sender->loop();
link.deliver_to(this->link_b_);
EXPECT_TRUE(this->remote_door_.state);
// Now complete, it is not asked for again
{
LoopBlockingGuard guard{nullptr, nullptr, 10000};
}
this->door_.publish_state(false);
sender->loop();
link.deliver_to(this->link_b_);
EXPECT_FALSE(this->remote_door_.state);
EXPECT_TRUE(this->link_b_.sent.empty());
}
TEST_F(EntityTransportTest, RefusedRequestSentAgain) {
auto receiver = this->make_receiver_();
Packet data = header(PACKET_DATA, 0x12345678);
this->link_b_.accept = false;
this->link_b_.inject(data);
this->link_b_.accept = true;
this->link_b_.inject(data);
ASSERT_EQ(this->link_b_.sent.size(), 1);
EXPECT_EQ(this->link_b_.sent[0], header(PACKET_REQUEST, 0x12345678));
}
TEST_F(EntityTransportTest, RequestsLimited) {
auto receiver = this->make_receiver_();
Packet data = header(PACKET_DATA, 0x12345678);
data.insert(data.end(), {0x00, 0x61}); // sensor 0 with no state
this->link_b_.inject(data);
this->link_b_.inject(data);
EXPECT_EQ(this->link_b_.sent.size(), 1);
}
TEST_F(EntityTransportTest, OtherProviderAndOwnPacketsIgnored) {
this->temp_.publish_state(5.0f);
auto sender = this->make_sender_();
auto receiver = this->make_receiver_("other-device");
sender->loop();
auto packets = this->link_a_.sent;
this->link_a_.deliver_to(this->link_b_);
EXPECT_FALSE(this->remote_temp_.has_state());
// The dictionary named none of the receiver's entities, so it is not asked for again
EXPECT_TRUE(this->link_b_.sent.empty());
for (auto &packet : packets)
this->link_a_.inject(packet);
this->link_a_.inject(header(PACKET_REQUEST, 0x12345678));
sender->loop();
EXPECT_TRUE(this->link_a_.sent.empty());
}
TEST_F(EntityTransportTest, MalformedPacketsIgnored) {
auto sender = this->make_sender_();
auto receiver = this->make_receiver_();
sender->loop();
Packet dictionary = this->link_a_.sent[0];
Packet data = this->link_a_.sent[1];
this->link_b_.inject(Packet(dictionary.begin(), dictionary.end() - 1));
this->link_b_.inject(dictionary);
this->link_b_.inject(Packet(data.begin(), data.end() - 1));
this->link_b_.inject({PACKET_DATA, 1, 2});
this->link_b_.inject({0x21, 0, 0, 0, 0});
EXPECT_FALSE(this->remote_temp_.has_state());
}
// Data packets from the sender's dictionary, holding the given records
class EntityTransportRecordTest : public EntityTransportTest {
protected:
void SetUp() override {
EntityTransportTest::SetUp();
this->sender_ = this->make_sender_();
this->receiver_ = this->make_receiver_();
this->sender_->loop();
this->link_a_.sent.resize(1); // the dictionary only
this->link_a_.deliver_to(this->link_b_);
}
void inject_data_(std::initializer_list<uint8_t> records) {
Packet data = header(PACKET_DATA, this->sender_->get_dictionary_id());
data.insert(data.end(), records);
this->link_b_.inject(data);
}
std::unique_ptr<TestTransport> sender_, receiver_;
};
TEST_F(EntityTransportRecordTest, WholePacketCheckedBeforeUse) {
// A good record for each entity, then a record cut short
this->inject_data_({0x00, 0x21, 0x00, 0x00, 0x10, 0x41, 0x00, 0x82, 0x00, 0x21, 0x00});
// A good record, then a sensor record without its state, then a binary sensor record with a payload
this->inject_data_({0x00, 0x21, 0x00, 0x00, 0x10, 0x41, 0x00, 0x01});
this->inject_data_({0x00, 0x82, 0x00, 0x22, 0x00, 0x00, 0x00, 0x00});
// The value bit is only for records without a payload, and type 0 is not used
this->inject_data_({0x00, 0xA1, 0x00, 0x00, 0x10, 0x41});
this->inject_data_({0x00, 0x82, 0x00, 0x00});
// A length byte past the end
this->inject_data_({0x00, 0x82, 0x00, 0x45, 0x03, 0x01});
EXPECT_FALSE(this->remote_temp_.has_state());
EXPECT_FALSE(this->remote_door_.has_state());
this->inject_data_({0x00, 0x21, 0x00, 0x00, 0x10, 0x41, 0x00, 0x82});
EXPECT_EQ(this->remote_temp_.get_state(), 9.0f);
EXPECT_TRUE(this->remote_door_.state);
}
TEST_F(EntityTransportRecordTest, UnknownTypesSkipped) {
// Type 5 with 3 bytes after a length byte, type 6 with a 4 byte payload, type 7 without a state, then the sensor
this->inject_data_({0x00, 0x45, 0x03, 0x01, 0x02, 0x03, 0x01, 0x26, 0x00, 0x00,
0x00, 0x00, 0x02, 0x67, 0x00, 0x21, 0x00, 0x00, 0x10, 0x41});
EXPECT_EQ(this->remote_temp_.get_state(), 9.0f);
}
TEST_F(EntityTransportTest, PartlyValidDictionaryNotLearnt) {
auto receiver = this->make_receiver_();
static constexpr uint32_t OTHER_ID = 0x0BADF00D;
Packet dictionary = dictionary_start(OTHER_ID);
// A good section naming the sensor, then sections cut short, of type 0, and of more than 255 numbers
dictionary.insert(dictionary.end(), {1, 0, 1, 4, 't', 'e', 'm', 'p'});
for (Packet bad : {Packet{2, 0, 2, 4, 'd', 'o', 'o', 'r'}, Packet{0, 0, 1, 1, 'x'}, Packet{2, 255, 2, 1, 'a', 1, 'b'},
Packet{2, 0, 1}}) {
Packet packet = dictionary;
packet.insert(packet.end(), bad.begin(), bad.end());
this->link_b_.inject(packet);
}
Packet data = header(PACKET_DATA, OTHER_ID);
data.insert(data.end(), {0x00, 0x21, 0x00, 0x00, 0x80, 0x3F});
this->link_b_.inject(data);
EXPECT_FALSE(this->remote_temp_.has_state());
this->link_b_.inject(dictionary);
this->link_b_.inject(data);
EXPECT_EQ(this->remote_temp_.get_state(), 1.0f);
}
TEST_F(EntityTransportTest, UpdateIntervalFollowsCapabilities) {
EXPECT_EQ(this->make_sender_()->get_update_interval(), 15000);
this->link_a_.set_capabilities(packet_interface::CAPABILITY_SESSION);
EXPECT_EQ(this->make_sender_()->get_update_interval(), 15000);
this->link_a_.set_capabilities(packet_interface::CAPABILITY_SESSION | packet_interface::CAPABILITY_RELIABILITY);
EXPECT_EQ(this->make_sender_()->get_update_interval(), SCHEDULER_DONT_RUN);
auto transport = std::make_unique<TestTransport>(&this->link_a_);
transport->set_update_interval(5000);
transport->setup();
EXPECT_EQ(transport->get_update_interval(), 5000);
}
TEST_F(EntityTransportTest, SessionConnectSendsAllAndLossClearsRemote) {
this->link_a_.set_capabilities(packet_interface::CAPABILITY_SESSION);
this->link_b_.set_capabilities(packet_interface::CAPABILITY_SESSION);
this->temp_.publish_state(7.0f);
auto sender = this->make_sender_();
auto receiver = this->make_receiver_();
sender->loop();
this->link_a_.deliver_to(this->link_b_);
this->link_a_.report_session(true);
sender->loop();
EXPECT_EQ(this->link_a_.sent.size(), 2);
this->link_b_.report_session(false);
EXPECT_TRUE(std::isnan(this->remote_temp_.get_state()));
EXPECT_FALSE(this->remote_door_.has_state());
}
TEST_F(EntityTransportTest, RefusedDictionaryPartResumes) {
// One entity in each dictionary packet
MockInterface link(5 + 2 + strlen(DEVICE_NAME) + 3 + 5);
this->temp_.publish_state(4.0f);
this->door_.publish_state(true);
auto sender = this->make_sender_(&link);
auto receiver = this->make_receiver_();
link.budget = 1;
sender->loop();
ASSERT_EQ(link.sent.size(), 1);
link.budget = SIZE_MAX;
sender->loop();
int dictionaries = 0;
for (auto &packet : link.sent)
dictionaries += packet[0] == PACKET_DICTIONARY;
EXPECT_EQ(dictionaries, 2); // the first part was not sent again
link.deliver_to(this->link_b_);
EXPECT_EQ(this->remote_temp_.get_state(), 4.0f);
EXPECT_TRUE(this->remote_door_.state);
}
TEST_F(EntityTransportTest, RemoteStatesTimeOut) {
this->temp_.publish_state(6.0f);
this->door_.publish_state(true);
auto sender = this->make_sender_();
auto receiver = this->make_receiver_(DEVICE_NAME, 5000);
{
LoopBlockingGuard guard{nullptr, nullptr, 1000};
}
sender->loop();
this->link_a_.deliver_to(this->link_b_);
{
LoopBlockingGuard guard{nullptr, nullptr, 6000};
}
receiver->clear_remote_states_(true);
EXPECT_EQ(this->remote_temp_.get_state(), 6.0f);
EXPECT_TRUE(this->remote_door_.has_state());
{
LoopBlockingGuard guard{nullptr, nullptr, 6001};
}
receiver->clear_remote_states_(true);
EXPECT_TRUE(std::isnan(this->remote_temp_.get_state()));
EXPECT_FALSE(this->remote_door_.has_state());
}
TEST_F(EntityTransportTest, IdMatchingOnlyUpToNulNotLearnt) {
auto receiver = this->make_receiver_();
static constexpr uint32_t OTHER_ID = 0x0BADF00D;
Packet dictionary = dictionary_start(OTHER_ID);
// The id "temp" followed by a NUL and more bytes, nine in all
dictionary.insert(dictionary.end(), {1, 0, 1, 9, 't', 'e', 'm', 'p', 0, 'a', 'b', 'c', 'd'});
this->link_b_.inject(dictionary);
Packet data = header(PACKET_DATA, OTHER_ID);
data.insert(data.end(), {0x00, 0x21, 0x00, 0x00, 0x80, 0x3F});
this->link_b_.inject(data);
EXPECT_FALSE(this->remote_temp_.has_state());
}
TEST_F(EntityTransportTest, PacketTooSmallForIdsFailsSetup) {
// One byte short of a dictionary packet holding the device name and a four character id
MockInterface small_link(5 + 2 + strlen(DEVICE_NAME) + 3 + 1 + 4 - 1);
auto sender = this->make_sender_(&small_link);
EXPECT_TRUE(sender->is_failed());
EXPECT_FALSE(this->make_sender_()->is_failed());
}
} // namespace esphome::entity_transport::testing
@@ -0,0 +1,3 @@
packages:
uart: !include ../../test_build_components/common/uart/esp32-idf.yaml
entity_transport: !include common.yaml
@@ -4,6 +4,9 @@ from tests.testing_helpers import ComponentManifestOverride
def override_manifest(manifest: ComponentManifestOverride) -> None:
async def to_code_testing(config: object) -> None:
cg.add_define("PACKET_INTERFACE_ADDRESSING_PORT_COUNT", 4)
cg.add_define("USE_PACKET_INTERFACE_BASIC_ADDRESSING")
cg.add_define("USE_PACKET_INTERFACE_BASIC_FRAGMENTATION")
cg.add_define("USE_PACKET_INTERFACE_BASIC_SESSION")
cg.add_define("USE_PACKET_INTERFACE_HDLC")
cg.add_define("USE_PACKET_INTERFACE_XXTEA")
@@ -0,0 +1,226 @@
#include <gtest/gtest.h>
#include <cstdint>
#include <initializer_list>
#include <memory>
#include <vector>
#include "esphome/components/packet_interface/basic_addressing.h"
#include "esphome/components/packet_interface/hdlc_reliability.h"
#include "esphome/core/application.h"
namespace esphome::packet_interface::addressing_testing {
// Copies a packet into a vector, for comparing in tests
static std::vector<uint8_t> to_vector(const esphome::packet_interface::PacketBuffer &data) {
std::vector<uint8_t> result(data.size());
data.copy_to(result.data(), result.size());
return result;
}
using Packet = std::vector<uint8_t>;
static constexpr uint16_t BROADCAST = BasicAddressing::BROADCAST_ADDRESS;
// A link layer that records what is sent, so a test can deliver it to other devices
class MockLink : public PacketInterface {
public:
explicit MockLink(size_t max_packet_size = 64) : PacketInterface(max_packet_size) {}
bool send_to_interface(const PacketBuffer &data, const PacketMetaData &) override {
this->sent.push_back(to_vector(data));
return true;
}
void inject(const Packet &packet) { this->on_receive_from_interface_(PacketBuffer(packet), {}); }
std::vector<Packet> sent;
};
// What one port receives
struct Port {
Port(BasicAddressing *addressing, uint16_t destination, bool only_port) {
this->port = std::make_unique<AddressingPort>(addressing, destination, only_port);
this->port->add_packet_interface_listener([this](const PacketBuffer &data, const PacketMetaData &meta_data) {
this->received.push_back(to_vector(data));
this->sources.push_back(meta_data.source_address);
});
}
void send(const Packet &data, uint16_t destination = 0) {
PacketMetaData meta_data;
meta_data.destination_address = destination;
this->port->send_to_interface(PacketBuffer(data), meta_data);
}
std::unique_ptr<AddressingPort> port;
std::vector<Packet> received;
std::vector<uint16_t> sources;
};
// A device on the bus with one port per destination given
struct Device {
Device(uint16_t address, std::initializer_list<uint16_t> destinations) : addressing(&this->link, address) {
this->link.set_name("bus");
for (uint16_t destination : destinations)
this->ports.push_back(std::make_unique<Port>(&this->addressing, destination, destinations.size() == 1));
this->addressing.setup();
}
explicit Device(uint16_t address, uint16_t destination = BROADCAST) : Device(address, {destination}) {}
Port &port(size_t index = 0) { return *this->ports[index]; }
// Every device on the bus hears every packet, including its own
void deliver(std::initializer_list<Device *> bus) {
for (auto &packet : this->link.sent) {
for (auto *device : bus)
device->link.inject(packet);
}
this->link.sent.clear();
}
MockLink link;
BasicAddressing addressing;
std::vector<std::unique_ptr<Port>> ports;
};
TEST(BasicAddressingTest, HeaderAndCapability) {
Device a(0x1234, 0x5678);
EXPECT_EQ(a.port().port->get_max_packet_size(), 64 - BasicAddressing::HEADER_SIZE);
EXPECT_TRUE(a.port().port->has_capability(CAPABILITY_ADDRESSING));
EXPECT_EQ(a.link.top(), a.port().port.get());
a.port().send({0xAA});
a.port().send({0xBB}, 0x0042);
EXPECT_EQ(a.link.sent, (std::vector<Packet>{{0x78, 0x56, 0x34, 0x12, 0xAA}, {0x42, 0x00, 0x34, 0x12, 0xBB}}));
}
TEST(BasicAddressingTest, LinkTooSmallLeavesNoRoom) {
MockLink link(BasicAddressing::HEADER_SIZE);
BasicAddressing addressing(&link, 0x0001);
AddressingPort port(&addressing, 0x0002, true);
EXPECT_EQ(port.get_max_packet_size(), 0);
}
TEST(BasicAddressingTest, BroadcastReachesEveryoneButSender) {
Device a(1), b(2), c(3);
a.port().send({0x10});
a.deliver({&a, &b, &c});
EXPECT_TRUE(a.port().received.empty());
EXPECT_EQ(b.port().received, std::vector<Packet>{{0x10}});
EXPECT_EQ(c.port().received, std::vector<Packet>{{0x10}});
EXPECT_EQ(b.port().sources, std::vector<uint16_t>{1});
}
TEST(BasicAddressingTest, UnicastReachesOnlyDestination) {
Device a(1, 2), b(2), c(3);
a.port().send({0x20});
a.deliver({&b, &c});
EXPECT_EQ(b.port().received, std::vector<Packet>{{0x20}});
EXPECT_TRUE(c.port().received.empty());
}
TEST(BasicAddressingTest, PointToPointAcceptsOnlyPeer) {
Device a(1, 2), b(2, 1), c(3);
c.port().send({0x30}); // broadcast from a third device
c.port().send({0x31}, 2); // addressed to b, from the wrong device
a.port().send({0x32}); // from the peer
c.deliver({&b});
a.deliver({&b});
EXPECT_EQ(b.port().received, std::vector<Packet>{{0x32}});
EXPECT_EQ(b.port().sources, std::vector<uint16_t>{1});
}
TEST(BasicAddressingTest, PeersHaveSeparateStacks) {
Device hub(1, {2, 3, BROADCAST}), b(2, 1), c(3, 1), d(4);
EXPECT_EQ(hub.link.top(), &hub.link); // with several ports, applications use the ports
EXPECT_EQ(hub.port(0).port->top(), hub.port(0).port.get());
b.port().send({0x02});
c.port().send({0x03});
d.port().send({0x04});
b.deliver({&hub});
c.deliver({&hub});
d.deliver({&hub});
EXPECT_EQ(hub.port(0).received, std::vector<Packet>{{0x02}});
EXPECT_EQ(hub.port(1).received, std::vector<Packet>{{0x03}});
EXPECT_EQ(hub.port(2).received, std::vector<Packet>{{0x04}}); // only devices without a port of their own
hub.port(1).send({0x13});
hub.deliver({&b, &c, &d});
EXPECT_EQ(c.port().received, std::vector<Packet>{{0x13}});
EXPECT_TRUE(b.port().received.empty());
EXPECT_TRUE(d.port().received.empty());
}
TEST(BasicAddressingTest, BroadcastFromPeerReachesBroadcastPort) {
Device hub(1, {2, BROADCAST}), b(2, {1, BROADCAST});
hub.port(1).send({0x10}); // a state every device needs
hub.port(0).send({0x11}); // point to point
hub.deliver({&b});
EXPECT_EQ(b.port(0).received, std::vector<Packet>{{0x11}});
EXPECT_EQ(b.port(1).received, std::vector<Packet>{{0x10}});
EXPECT_EQ(b.port(1).sources, std::vector<uint16_t>{1});
}
TEST(BasicAddressingTest, WithoutBroadcastPortPeerBroadcastReachesItsPort) {
Device hub(1, {2, 3}), b(2, BROADCAST);
b.port().send({0x20});
b.deliver({&hub});
EXPECT_EQ(hub.port(0).received, std::vector<Packet>{{0x20}});
EXPECT_TRUE(hub.port(1).received.empty());
}
TEST(BasicAddressingTest, WithoutBroadcastPortUnknownDevicesDropped) {
Device hub(1, {2, 3}), d(4);
d.port().send({0x04});
d.deliver({&hub});
EXPECT_TRUE(hub.port(0).received.empty());
EXPECT_TRUE(hub.port(1).received.empty());
}
TEST(BasicAddressingTest, ShortPacketDropped) {
Device a(1);
a.link.inject({0xFF, 0xFF, 0x02});
EXPECT_TRUE(a.port().received.empty());
}
// Each test replaces App with a fresh one; the reliability layer's timers need its scheduler
class AddressingReliabilityTest : public ::testing::Test {
protected:
void SetUp() override {
App.~Application();
new (&App) Application();
static char name[] = "addressing-tests";
static char friendly_name[] = "";
App.pre_setup(name, sizeof(name) - 1, friendly_name, 0);
}
void TearDown() override {
App.~Application();
new (&App) Application(); // later tests and the test listener use App
}
};
// A reliability layer on each port, recording what it delivers
struct ReliablePort {
explicit ReliablePort(AddressingPort *port) : hdlc(port) {
this->hdlc.add_packet_interface_listener(
[this](const PacketBuffer &data, const PacketMetaData &) { this->received.push_back(to_vector(data)); });
this->hdlc.setup();
}
void send(const Packet &data) { this->hdlc.send_to_interface(PacketBuffer(data), {}); }
HdlcReliability hdlc;
std::vector<Packet> received;
};
TEST_F(AddressingReliabilityTest, SeveralReliablePeersOnOneBus) {
Device hub(1, {2, 3}), b(2, 1), c(3, 1);
ReliablePort hub_b(hub.port(0).port.get()), hub_c(hub.port(1).port.get());
ReliablePort b_hub(b.port().port.get()), c_hub(c.port().port.get());
hub_b.send({0x12});
hub_c.send({0x13});
b_hub.send({0x21});
c_hub.send({0x31});
for (int i = 0; i != 20; i++) {
for (auto *port : {&hub_b, &hub_c, &b_hub, &c_hub})
port->hdlc.loop();
for (auto *device : {&hub, &b, &c})
device->deliver({&hub, &b, &c});
}
EXPECT_EQ(b_hub.received, std::vector<Packet>{{0x12}});
EXPECT_EQ(c_hub.received, std::vector<Packet>{{0x13}});
EXPECT_EQ(hub_b.received, std::vector<Packet>{{0x21}});
EXPECT_EQ(hub_c.received, std::vector<Packet>{{0x31}});
}
} // namespace esphome::packet_interface::addressing_testing
@@ -0,0 +1,229 @@
#include <gtest/gtest.h>
#include <cstdint>
#include <memory>
#include <vector>
#include "esphome/components/packet_interface/basic_fragmentation.h"
#include "esphome/components/packet_interface/hdlc_reliability.h"
#include "esphome/core/application.h"
namespace esphome::packet_interface::fragmentation_testing {
// Copies a packet into a vector, for comparing in tests
static std::vector<uint8_t> to_vector(const esphome::packet_interface::PacketBuffer &data) {
std::vector<uint8_t> result(data.size());
data.copy_to(result.data(), result.size());
return result;
}
using Packet = std::vector<uint8_t>;
// A link layer that records what is sent, with a limited send capacity
class MockLink : public PacketInterface {
public:
explicit MockLink(size_t max_packet_size = 12) : PacketInterface(max_packet_size) {}
bool send_to_interface(const PacketBuffer &data, const PacketMetaData &) override {
this->sent.push_back(to_vector(data));
return true;
}
size_t get_send_capacity() const override { return this->capacity; }
void inject(const Packet &packet, uint16_t source = 0) {
PacketMetaData meta_data;
meta_data.source_address = source;
this->on_receive_from_interface_(PacketBuffer(packet), meta_data);
}
std::vector<Packet> sent;
size_t capacity{SIZE_MAX};
};
struct Peer {
explicit Peer(PacketInterface *lower) : fragmentation(lower, 100) {
this->fragmentation.add_packet_interface_listener(
[this](const PacketBuffer &data, const PacketMetaData &) { this->received.push_back(to_vector(data)); });
this->fragmentation.setup();
}
bool send(const Packet &data) { return this->fragmentation.send_to_interface(PacketBuffer(data), {}); }
BasicFragmentation fragmentation;
std::vector<Packet> received;
};
static Packet make_packet(size_t size) {
Packet packet(size);
for (size_t i = 0; i != size; i++)
packet[i] = i * 7 + 1;
return packet;
}
class TestHdlc : public HdlcReliability {
public:
using HdlcReliability::HdlcReliability;
using HdlcReliability::on_timeout_;
};
// Each test replaces App with a fresh one; the reliability layer's timers need its scheduler
class BasicFragmentationTest : public ::testing::Test {
protected:
void SetUp() override {
App.~Application();
new (&App) Application();
static char name[] = "fragmentation-tests";
static char friendly_name[] = "";
App.pre_setup(name, sizeof(name) - 1, friendly_name, 0);
}
void TearDown() override {
App.~Application();
new (&App) Application(); // later tests and the test listener use App
}
// Deliver what a has sent to b
static void deliver(MockLink &from, MockLink &to) {
for (auto &packet : from.sent)
to.inject(packet);
from.sent.clear();
}
MockLink link_a_, link_b_;
};
TEST_F(BasicFragmentationTest, SizesAndCapability) {
Peer a(&this->link_a_);
EXPECT_EQ(a.fragmentation.get_max_packet_size(), 100);
EXPECT_TRUE(a.fragmentation.has_capability(CAPABILITY_FRAGMENTATION));
EXPECT_EQ(this->link_a_.top(), &a.fragmentation);
}
TEST_F(BasicFragmentationTest, SmallPacketIsOneFragment) {
Peer a(&this->link_a_), b(&this->link_b_);
a.send({1, 2, 3});
ASSERT_EQ(this->link_a_.sent.size(), 1);
EXPECT_EQ(this->link_a_.sent[0].size(), 3 + BasicFragmentation::HEADER_SIZE);
EXPECT_EQ(this->link_a_.sent[0][1], 0x80); // index 0, last
deliver(this->link_a_, this->link_b_);
EXPECT_EQ(b.received, (std::vector<Packet>{{1, 2, 3}}));
}
TEST_F(BasicFragmentationTest, LargePacketSplitAndJoined) {
Peer a(&this->link_a_), b(&this->link_b_);
for (size_t size : {10, 11, 50, 100}) {
Packet packet = make_packet(size);
ASSERT_TRUE(a.send(packet));
EXPECT_EQ(this->link_a_.sent.size(), (size + 9) / 10); // 10 bytes of payload in each 12 byte fragment
EXPECT_EQ(this->link_a_.sent.back()[1] & 0x80, 0x80);
deliver(this->link_a_, this->link_b_);
EXPECT_EQ(b.received.back(), packet);
}
EXPECT_EQ(b.received.size(), 4);
}
TEST_F(BasicFragmentationTest, CompositePacketSplitAcrossFragments) {
Peer a(&this->link_a_), b(&this->link_b_);
Packet pre = make_packet(7), mid(13, 0xAA), post(9, 0x55);
PacketBuffer body(mid.data(), mid.size());
ASSERT_TRUE(
a.fragmentation.send_to_interface(PacketBuffer(pre.data(), pre.size(), &body, post.data(), post.size()), {}));
deliver(this->link_a_, this->link_b_);
Packet expected = pre;
expected.insert(expected.end(), mid.begin(), mid.end());
expected.insert(expected.end(), post.begin(), post.end());
EXPECT_EQ(b.received, std::vector<Packet>{expected});
}
TEST_F(BasicFragmentationTest, MissingFragmentDropsOnlyThatPacket) {
Peer a(&this->link_a_), b(&this->link_b_);
a.send(make_packet(30));
this->link_a_.sent.erase(this->link_a_.sent.begin() + 1);
deliver(this->link_a_, this->link_b_);
EXPECT_TRUE(b.received.empty());
a.send(make_packet(25));
deliver(this->link_a_, this->link_b_);
EXPECT_EQ(b.received, std::vector<Packet>{make_packet(25)});
}
TEST_F(BasicFragmentationTest, NotStartedWithoutRoomForAllFragments) {
Peer a(&this->link_a_);
this->link_a_.capacity = 2;
EXPECT_FALSE(a.send(make_packet(30))); // needs 3
EXPECT_TRUE(this->link_a_.sent.empty());
EXPECT_TRUE(a.send(make_packet(20)));
EXPECT_FALSE(a.send(make_packet(101)));
}
TEST_F(BasicFragmentationTest, InterleavedSourcesNotMixed) {
Peer a(&this->link_a_), c(&this->link_a_), b(&this->link_b_);
a.send(make_packet(20));
auto from_a = std::move(this->link_a_.sent);
this->link_a_.sent.clear();
c.send(make_packet(20));
auto from_c = std::move(this->link_a_.sent);
// Fragments from devices 1 and 2 arrive interleaved, perhaps with the same packet number
this->link_b_.inject(from_a[0], 1);
this->link_b_.inject(from_c[0], 2);
this->link_b_.inject(from_a[1], 1);
this->link_b_.inject(from_c[1], 2);
// Nothing is mixed from the two, and the packet started last gets through
EXPECT_EQ(b.received, std::vector<Packet>{make_packet(20)});
}
TEST_F(BasicFragmentationTest, OneFragmentPacketFromAnotherSourceKeepsJoin) {
Peer a(&this->link_a_), c(&this->link_a_), b(&this->link_b_);
a.send(make_packet(20));
auto from_a = std::move(this->link_a_.sent);
this->link_a_.sent.clear();
c.send(make_packet(5));
auto from_c = std::move(this->link_a_.sent);
this->link_b_.inject(from_a[0], 1);
this->link_b_.inject(from_c[0], 2);
this->link_b_.inject(from_a[1], 1);
EXPECT_EQ(b.received, (std::vector<Packet>{make_packet(5), make_packet(20)}));
}
TEST_F(BasicFragmentationTest, OverReliabilityWithLosses) {
// Fragmentation above HDLC on each side, with every fourth frame from a dropped on the way; not every third, as
// each resend is a burst of three, so the same fragment would be lost every time
TestHdlc hdlc_a(&this->link_a_), hdlc_b(&this->link_b_);
hdlc_a.setup();
hdlc_b.setup();
// HDLC adds 3 bytes to the 12 byte link, leaving 7 bytes of payload in each fragment
Peer a(&hdlc_a), b(&hdlc_b);
Packet packet = make_packet(20); // 3 fragments, within the queue of 4
ASSERT_TRUE(a.send(packet));
int count = 0;
for (int i = 0; i != 30; i++) {
hdlc_a.loop();
hdlc_b.loop();
if (i % 5 == 4)
hdlc_a.on_timeout_();
for (auto &frame : this->link_a_.sent) {
if (++count % 4 != 0)
this->link_b_.inject(frame);
}
this->link_a_.sent.clear();
deliver(this->link_b_, this->link_a_);
}
EXPECT_EQ(b.received, std::vector<Packet>{packet});
}
TEST_F(BasicFragmentationTest, OversizedSingleFragmentDropped) {
// A link packet holds more than this layer's max_packet_size
BasicFragmentation fragmentation(&this->link_a_, 4);
std::vector<Packet> received;
fragmentation.add_packet_interface_listener(
[&received](const PacketBuffer &data, const PacketMetaData &) { received.push_back(to_vector(data)); });
fragmentation.setup();
this->link_a_.inject({0x01, 0x80, 1, 2, 3, 4, 5});
EXPECT_TRUE(received.empty());
this->link_a_.inject({0x02, 0x80, 1, 2, 3, 4});
EXPECT_EQ(received, std::vector<Packet>{Packet({1, 2, 3, 4})});
}
TEST_F(BasicFragmentationTest, LinkTooSmallFailsSetup) {
MockLink link(BasicFragmentation::HEADER_SIZE);
BasicFragmentation fragmentation(&link, 100);
fragmentation.setup();
EXPECT_TRUE(fragmentation.is_failed());
Packet packet{1};
EXPECT_FALSE(fragmentation.send_to_interface(PacketBuffer(packet), {}));
}
} // namespace esphome::packet_interface::fragmentation_testing
+31 -14
View File
@@ -9,24 +9,41 @@ packet_interface:
- platform: uart
id: packet_interface_uart
max_packet_size: 256
reliability:
timeout: 300ms
ack_delay: 50ms
max_queue_length: 3
max_retries: 4
encryption:
key: Johnny Cash
rolling_code_enable: true
session:
keep_alive: 5s
timeout: 12s
binary_sensor:
name: UART session
addressing:
address: 0x0001
peers:
- id: packet_interface_peer_a
destination: 0x0002
encryption:
key: Johnny Cash
rolling_code_enable: true
reliability:
timeout: 300ms
max_queue_length: 3
max_retries: 4
session:
keep_alive: 5s
timeout: 12s
binary_sensor:
name: Peer A session
- id: packet_interface_peer_b
destination: 0x0003
reliability:
max_queue_length: 5
fragmentation:
max_packet_size: 1024
session:
- id: packet_interface_bus
destination: broadcast
- platform: udp
id: packet_interface_network
udp_id: packet_interface_udp
addressing:
address: 0x0001
destination: 0x0002
encryption: Johnny Cash
reliability:
fragmentation:
session:
sensor:
@@ -36,7 +53,7 @@ sensor:
packet_transport:
- platform: packet_interface
packet_interface: packet_interface_uart
packet_interface: packet_interface_peer_a
sensors:
- packet_interface_sensor
- platform: packet_interface
+116
View File
@@ -0,0 +1,116 @@
"""Unit tests for the entity_transport component."""
import pytest
from esphome import config_validation as cv
from esphome.components.const import CONF_PROVIDER, CONF_REMOTE_ID
from esphome.components.entity_transport import (
CONFIG_SCHEMA,
FINAL_VALIDATE_SCHEMA,
remote_entity_schema,
)
from esphome.components.sensor import sensor_schema
from esphome.const import (
CONF_ID,
CONF_NAME,
CONF_TIMEOUT,
CONF_UPDATE_INTERVAL,
SCHEDULER_DONT_RUN,
)
from esphome.core import ID
import esphome.final_validate as fv
def test_remote_id_required_without_manual_id(setup_core) -> None:
"""A remote entity needs remote_id unless its own id is given."""
schema = remote_entity_schema(sensor_schema())
with pytest.raises(cv.Invalid, match="remote_id"):
schema({CONF_NAME: "Temp", CONF_PROVIDER: "remote-device"})
assert schema(
{CONF_NAME: "Temp 2", CONF_PROVIDER: "remote-device", CONF_REMOTE_ID: "temp"}
)
config = schema(
{CONF_PROVIDER: "remote-device", CONF_ID: ID("temp", is_manual=True)}
)
assert config[CONF_ID].id == "temp"
def test_update_interval_never(setup_core) -> None:
"""update_interval accepts never, and is left out when not given."""
base = {"packet_interface": "link"}
assert CONF_UPDATE_INTERVAL not in CONFIG_SCHEMA(base)
config = CONFIG_SCHEMA({**base, CONF_UPDATE_INTERVAL: "never"})
assert config[CONF_UPDATE_INTERVAL].total_milliseconds == SCHEDULER_DONT_RUN
with pytest.raises(cv.Invalid):
CONFIG_SCHEMA({**base, CONF_UPDATE_INTERVAL: "500ms"})
def test_timeout(setup_core) -> None:
"""The timeout is optional, and at least a second."""
base = {"packet_interface": "link"}
assert CONF_TIMEOUT not in CONFIG_SCHEMA(base)
assert (
CONFIG_SCHEMA({**base, CONF_TIMEOUT: "1min"})[CONF_TIMEOUT].total_seconds == 60
)
with pytest.raises(cv.Invalid):
CONFIG_SCHEMA({**base, CONF_TIMEOUT: "500ms"})
@pytest.mark.parametrize(
("layers", "allowed"),
[({}, True), ({"reliability": None}, True), ({"session": {}}, False)],
)
def test_timeout_not_with_session(setup_core, layers: dict, allowed: bool) -> None:
"""With a session layer, unchanged states are never sent again, so a timeout would clear them on a working link."""
config = CONFIG_SCHEMA({"packet_interface": "link", CONF_TIMEOUT: "1min"})
token = fv.full_config.set(
{"packet_interface": [{CONF_ID: ID("link", is_manual=True), **layers}]}
)
try:
if allowed:
FINAL_VALIDATE_SCHEMA(config)
else:
with pytest.raises(cv.Invalid, match="session") as error:
FINAL_VALIDATE_SCHEMA(config)
assert error.value.path == [CONF_TIMEOUT]
# Without a timeout, any layers are fine
FINAL_VALIDATE_SCHEMA(CONFIG_SCHEMA({"packet_interface": "link"}))
finally:
fv.full_config.reset(token)
def test_timeout_not_with_session_on_peer(setup_core) -> None:
"""The layers of an addressing peer are found by the peer's id."""
peers = [
{CONF_ID: ID("plain", is_manual=True)},
{CONF_ID: ID("with_session", is_manual=True), "session": {}},
]
token = fv.full_config.set(
{
"packet_interface": [
{CONF_ID: ID("bus", is_manual=True), "addressing": {"peers": peers}}
]
}
)
try:
FINAL_VALIDATE_SCHEMA(
CONFIG_SCHEMA({"packet_interface": "plain", CONF_TIMEOUT: "1min"})
)
with pytest.raises(cv.Invalid, match="session"):
FINAL_VALIDATE_SCHEMA(
CONFIG_SCHEMA(
{"packet_interface": "with_session", CONF_TIMEOUT: "1min"}
)
)
finally:
fv.full_config.reset(token)
def test_advertised_ids_unique(setup_core) -> None:
"""Two entities of one type cannot be sent with the same id."""
base = {"packet_interface": "link"}
assert CONFIG_SCHEMA({**base, "sensors": ["a", {"id": "b", "broadcast_id": "c"}]})
with pytest.raises(cv.Invalid, match="'a' is sent for two entities"):
CONFIG_SCHEMA({**base, "sensors": ["a", {"id": "b", "broadcast_id": "a"}]})
# Sensors and binary sensors are numbered separately, so may share an id
assert CONFIG_SCHEMA({**base, "sensors": ["a"], "binary_sensors": ["a"]})
+192
View File
@@ -9,6 +9,8 @@ from esphome import config_validation as cv
from esphome.components import packet_interface
from esphome.components.packet_interface import CONF_MAX_PACKET_SIZE
from esphome.components.packet_interface.layers import (
CONF_ADDRESSING,
CONF_FRAGMENTATION,
CONF_RELIABILITY,
CONF_SESSION,
RELIABILITY,
@@ -23,6 +25,8 @@ from esphome.components.packet_interface.layers.hdlc_reliability import (
from esphome.components.uart import packet_interface as uart_packet_interface
from esphome.components.udp import packet_interface as udp_packet_interface
from esphome.const import CONF_ENCRYPTION
from esphome.core import ID
import esphome.final_validate as fv
@pytest.mark.parametrize(
@@ -94,6 +98,7 @@ def test_layer_types_registered() -> None:
assert {kind.key: set(kind.types) for kind in STACK_ORDER} == {
CONF_ENCRYPTION: {"xxtea"},
CONF_RELIABILITY: {"hdlc"},
CONF_FRAGMENTATION: {"basic"},
CONF_SESSION: {"basic"},
}
@@ -178,6 +183,149 @@ def test_rolling_code_requires_session(setup_core) -> None:
uart_packet_interface.CONFIG_SCHEMA({CONF_ENCRYPTION: "secret"})
def test_addressing_defaults_to_broadcast(setup_core) -> None:
"""Without a destination, packets are broadcast; broadcast can also be named."""
for addressing in ({"address": 1}, {"address": 1, "destination": "broadcast"}):
config = uart_packet_interface.CONFIG_SCHEMA({CONF_ADDRESSING: addressing})
assert config[CONF_ADDRESSING]["destination"] == 0xFFFF
with pytest.raises(cv.Invalid):
uart_packet_interface.CONFIG_SCHEMA({CONF_ADDRESSING: {"address": 0xFFFF}})
@pytest.mark.parametrize(
"layers",
[
{CONF_RELIABILITY: None},
{CONF_SESSION: None},
{
CONF_ENCRYPTION: {"key": "secret", "rolling_code_enable": True},
CONF_SESSION: None,
},
],
)
def test_two_device_layers_need_destination(setup_core, layers: dict) -> None:
"""Layers that work between two devices only cannot be used with broadcast addressing."""
with pytest.raises(cv.Invalid, match="destination"):
uart_packet_interface.CONFIG_SCHEMA({CONF_ADDRESSING: {"address": 1}, **layers})
uart_packet_interface.CONFIG_SCHEMA(
{CONF_ADDRESSING: {"address": 1, "destination": 2}, **layers}
)
def _peer(peer_id: str | ID, destination: Any, **layers: Any) -> dict:
return {"id": peer_id, "destination": destination, **layers}
def test_addressing_peers(setup_core) -> None:
"""Each peer has its own destination and layers."""
config = uart_packet_interface.CONFIG_SCHEMA(
{
CONF_ADDRESSING: {
"address": 1,
"peers": [
_peer("peer_a", 2, reliability=None, session=None),
_peer("peer_b", 3, encryption="secret"),
_peer("everyone", "broadcast"),
],
}
}
)
peers = config[CONF_ADDRESSING]["peers"]
assert [peer["destination"] for peer in peers] == [2, 3, 0xFFFF]
assert CONF_RELIABILITY in peers[0]
@pytest.mark.parametrize(
("addressing", "layers", "match"),
[
({"peers": [_peer("a", 2), _peer("b", 2)]}, {}, "different destination"),
({"peers": [_peer("a", "broadcast", reliability=None)]}, {}, "broadcast"),
({"peers": [_peer("a", 2)]}, {CONF_RELIABILITY: None}, "goes in each peer"),
({"destination": 2, "peers": [_peer("a", 3)]}, {}, "destination"),
],
)
def test_addressing_peers_invalid(
setup_core, addressing: dict, layers: dict, match: str
) -> None:
"""Peers need different destinations, two-device layers need a destination, and layers go in the peers."""
with pytest.raises(cv.Invalid, match=match):
uart_packet_interface.CONFIG_SCHEMA(
{CONF_ADDRESSING: {"address": 1, **addressing}, **layers}
)
def test_fragmentation_defaults(setup_core) -> None:
"""An empty fragmentation key joins packets of up to 1024 bytes."""
fragmentation = uart_packet_interface.CONFIG_SCHEMA({CONF_FRAGMENTATION: None})[
CONF_FRAGMENTATION
]
assert fragmentation["type"] == "basic"
assert fragmentation["max_packet_size"] == 1024
with pytest.raises(cv.Invalid):
uart_packet_interface.CONFIG_SCHEMA(
{CONF_FRAGMENTATION: {"max_packet_size": 8}}
)
@pytest.mark.parametrize(
("layers", "valid"),
[
# 508 - 3 leaves fragments of 503 bytes: 1024 bytes take 3, 2048 bytes take 5
({CONF_RELIABILITY: None, CONF_FRAGMENTATION: None}, True),
(
{CONF_RELIABILITY: None, CONF_FRAGMENTATION: {"max_packet_size": 2048}},
False,
),
(
{
CONF_RELIABILITY: {CONF_MAX_QUEUE_LENGTH: 5},
CONF_FRAGMENTATION: {"max_packet_size": 2048},
},
True,
),
({CONF_FRAGMENTATION: {"max_packet_size": 16384}}, True),
],
)
def test_fragments_fit_reliability_queue(setup_core, layers: dict, valid: bool) -> None:
"""Every fragment of the largest packet must fit in the reliability queue at once."""
if valid:
uart_packet_interface.CONFIG_SCHEMA(layers)
return
with pytest.raises(cv.Invalid, match="5 fragments") as err:
uart_packet_interface.CONFIG_SCHEMA(layers)
assert err.value.path == [CONF_FRAGMENTATION, CONF_MAX_PACKET_SIZE]
@pytest.mark.parametrize(
("link", "match"),
[
# 256 - 4 for addressing - 16 for the rolling code - 3 for reliability leaves fragments of 231 bytes
({}, "5 fragments of 231 bytes"),
# 24 - 4 - 16 - 3 leaves 1 byte, too few for the fragment header
({CONF_MAX_PACKET_SIZE: 24}, "no room"),
],
)
def test_fragments_checked_in_peers(setup_core, link: dict, match: str) -> None:
"""Fragments are checked against the layers of each peer, below the addressing layer."""
peer = _peer(
"a",
2,
encryption={"key": "secret", "rolling_code_enable": True},
reliability=None,
fragmentation=None,
session=None,
)
with pytest.raises(cv.Invalid, match=match) as err:
uart_packet_interface.CONFIG_SCHEMA(
{
CONF_MAX_PACKET_SIZE: 256,
**link,
CONF_ADDRESSING: {"address": 1, "peers": [peer]},
}
)
assert err.value.path[:4] == [CONF_ADDRESSING, "peers", 0, CONF_FRAGMENTATION]
@pytest.mark.parametrize(
("config", "loads"),
[
@@ -185,6 +333,15 @@ def test_rolling_code_requires_session(setup_core) -> None:
({CONF_RELIABILITY: None, CONF_SESSION: None}, []),
({CONF_SESSION: {"binary_sensor": {"name": "Link"}}}, ["binary_sensor"]),
({CONF_ENCRYPTION: "secret"}, ["xxtea"]),
(
{
CONF_ADDRESSING: {
"address": 1,
"peers": [_peer("a", 2, encryption="secret"), _peer("b", 3)],
}
},
["xxtea"],
),
],
)
def test_auto_load_follows_layers(
@@ -214,3 +371,38 @@ def test_xxtea_key_stretched() -> None:
assert key != xxtea_encryption.derive_key("Johnny Cash ")
sha = hashlib.sha256(b"Johnny Cash").digest()
assert key[0] != int.from_bytes(sha[:4], "little")
@pytest.mark.parametrize(
"addressing",
[
{"address": 1, "destination": 1},
{"address": 1, "peers": [_peer("a", 2), _peer("b", 1)]},
],
)
def test_addressing_own_address_rejected(setup_core, addressing: dict) -> None:
"""A device cannot send to its own address, as its own packets are dropped as echoes."""
with pytest.raises(cv.Invalid, match="own address"):
uart_packet_interface.CONFIG_SCHEMA({CONF_ADDRESSING: addressing})
def test_application_needs_peer_id_with_peers(setup_core) -> None:
"""An application must use one peer's id, not the id of a link with peers."""
validator = packet_interface.final_validate_interface("packet_interface")
token = fv.full_config.set(
{
"packet_interface": [
{"id": ID("bus"), CONF_ADDRESSING: {"peers": [_peer(ID("a"), 2)]}},
{"id": ID("single"), CONF_ADDRESSING: {"destination": 2}},
]
}
)
try:
with pytest.raises(
cv.Invalid, match="has peers, so use the id of one of them, such as 'a'"
):
validator({"packet_interface": ID("bus")})
assert validator({"packet_interface": ID("single")})
assert validator({"packet_interface": ID("a")})
finally:
fv.full_config.reset(token)