Compare commits

..
Author SHA1 Message Date
J. Nick Koston 5ac2fa8811 [api] Print the dropped action's service with its length
The field can now start as a null StringRef, so the log passes the size and an empty
literal instead of a pointer that may be null.
2026-09-08 04:42:58 +02:00
J. Nick Koston e715696bd7 [core] Bound StringRef's JSON conversion by the view length 2026-09-08 04:15:24 +02:00
J. Nick Koston 82d399400e [core] Convert a null StringRef to an empty JSON string and pin null against null 2026-09-08 03:44:24 +02:00
J. Nick Koston a7bf937001 [core] Keep StringRef::str() inline
The std::string range constructor reads nothing for a zero length, so the guard only
pushed str() out of line; the memcmp guards stay.
2026-09-08 03:22:05 +02:00
J. Nick Koston 63b5331e47 [core] Let StringRef carry a null pointer at zero length
The generated api messages start their encode only string fields that way. starts_with,
compare and str() no longer hand a null pointer to memcmp or the std::string range
constructor when there is nothing to compare or copy, the class comment states the
contract, and gtest cases pin every member on a null, empty view.
2026-09-08 03:14:44 +02:00
J. Nick Koston 4c72948575 [api] Say why the pointer buffer base keeps its constructor 2026-09-08 00:24:14 +02:00
J. Nick Koston 77d850cbe2 [api] Drop the unused array_size on the pointer buffer base 2026-09-08 00:07:54 +02:00
J. Nick Koston a97e4ebc1c [api] Pass needs_decode at both bytes buffer sites and guard the second unconditional copy path 2026-09-07 23:46:15 +02:00
J. Nick Koston d09f642eb2 [api] Name the null default invariant once and thread needs_decode through the pointer buffer base
The dead size parameter goes, the string type reads the inherited flag, the forced short
string path asserts against it, and the generated declarations say why the pointer may be
null.
2026-09-07 23:23:19 +02:00
J. Nick Koston 144cd419ad [api] Test which string fields get the null default 2026-09-07 23:11:47 +02:00
J. Nick Koston 7910c372cd [api] Default response only string fields to a null StringRef
A StringRef field that is only ever encoded is skipped when empty before its pointer is
read, and the dump helper checks empty() first, so pointing it at "" buys nothing while
costing one store per field in every message constructor. Fields that are decoded or force
encoded keep the empty string default.
2026-09-07 23:04:33 +02:00
J. Nick Koston c2a4981e25 [api] Order the wrong wire type assertions on the raw socket
The barrier was a command on the aioesphomeapi connection, which nothing orders against the
raw frames under test; a well formed frame on the raw socket itself now closes each block and
the assertion checks the exact states seen since the marker.
2026-09-07 16:22:42 +02:00
J. Nick Koston 097d487261 [api] Cover truncated bodies in the decode integration test and pin the 64 bit field rejection
Three malformed frames (a tag with a dangling continuation bit, a length prefix past the
payload, a two byte fixed32) must stop the decode loop without taking the connection down;
a generator test records that a double field is rejected before it could reach the loop.
2026-09-07 15:58:28 +02:00
J. Nick Koston 89082c7b2d [api] Cover repeated and sub message fields in the decode case tests 2026-09-07 15:57:14 +02:00
J. Nick Koston fb3befe377 [api] Second cleanup pass over the decode generator
Repeated fields encode their elements through one encode_element() hook instead of two
isinstance ladders, the fixed32 precomputed tag path owns its own guard, the generated
switches drop the dead default case, StringRef takes the byte pointer directly, the
three hand written tag expressions in proto.h go through proto_tag(), and stale comments
about the previous decode design go. The compiled functions are unchanged.
2026-09-07 15:57:14 +02:00
J. Nick Koston 4a3a787756 [api] Return the varint parse result as a struct again
The out parameter form regressed the host, where the 16 byte result already travels in
registers, by about 20 percent on the direct varint parse benchmarks. The void
decode_field and low word bool changes stay.
2026-09-07 15:57:13 +02:00
J. Nick Koston 5aec93d7e8 [api] Trim the decode path: void decode_field, bool from the low word, slow varint out parameter
decode_field() no longer returns a bool that only fed a verbose log; unknown fields are
skipped silently like every other protobuf decoder does, and each message loses the
return value materialisation. Bools read the low 32 bits of the varint, which drops the
second compare on 64 bit varint builds. The multi byte varint path writes its value
through an out parameter instead of returning a 16 byte struct, which takes the spills
out of the decode loop and count_repeated_field.
2026-09-07 15:57:13 +02:00
J. Nick Koston a79cd1550c [api] Collapse the generator's per wire type decode hooks into one
A type now sets a single decode_expr; the wire type it already declares picks the case
label. Drops the unused force_str helper, two dead decode_length overrides, a duplicate
field builder in the generator tests and a needless list copy in the state waiter. The
generated files are unchanged.
2026-09-07 15:57:13 +02:00
J. Nick Koston 6d23e4c842 [api] Key the generated decode switch on the wire tag on every target
The host no longer gets its own switch shape through USE_HOST; every build compiles the
same switch on the field's wire tag.
2026-09-07 15:57:13 +02:00
J. Nick Koston cbdcfe640c [api] Read fixed32 fields with byte loads and test the varint wire type first
ESP-IDF passes -fno-builtin-memcpy, so the four byte memcpy in the decode loop was an
out of line call on every fixed32 field; host compilers fold the byte loads back into one
load. Checking the varint wire type first keeps the common path to one taken branch on
xtensa.
2026-09-07 15:57:13 +02:00
J. Nick Koston 2bc20d8721 [api] Derive decode cases from the type's wire type
decode_case() reads wire_type instead of taking it at every call, the
expression and the store statement are two small hooks that repeated
fields override, and message fields build one body. No generated case
declares a local any more, so the braced form and its test go; the
compiler rejects a jump over a local if one ever appears. The wire
type test now proves a dropped frame with an ordering marker instead
of assuming it, and shares StateWaiter.
2026-09-07 15:57:13 +02:00
J. Nick Koston 2dbb3e055b [api] Add an integration test for decode wire type handling
Hand built frames check that decode_field() takes a field with its
declared wire type, drops the same field sent length delimited or as
fixed32, ignores a varint key, and skips an unknown field with a two
byte tag before decoding the rest. Client commands cover two byte tags
and varints, a two byte length prefix and a negative fixed32 float.
2026-09-07 15:57:13 +02:00
J. Nick Koston 7549309ad3 [api] Trim the decode dispatch comments 2026-09-07 15:57:13 +02:00
J. Nick Koston e600180417 [api] Emit one decode case per field from a single generator property
With one decode_field() switch per message, the three per wire type
content properties only differed in the attribute they read; a single
decode_content built from decode_expr() replaces them, and repeated
fields reuse the element type's expression. Case bodies with several
statements get their block from the body itself instead of a caller
flag, the fixed byte array body copies straight from the payload
instead of through a heap std::string, and the decode comments no
longer restate the switch keying explained next to the macros.
2026-09-07 15:57:13 +02:00
J. Nick Koston 0641dae9d2 [api] Scope generated decode cases that declare locals
A case body with a declaration or several statements now gets its own
block, as the per wire type overrides had, so no jump to a later case
label crosses an initialization.
2026-09-07 15:57:13 +02:00
J. Nick Koston bcf812d62b [api] Keep the decode loop register resident and inline the varint fast path
CodSpeed showed the single virtual costing 7 to 18 percent on the
decode benchmarks. The x86-64 disassembly pointed at the call, not the
switch: passing the field number and wire type alongside the tag plus
a 16 byte union payload kept five values live across the call, so the
compiler spilled this, the end pointer and half of the payload to the
stack and reloaded them for every field.

decode_field() now takes only the tag, the payload pointer (already
the loop cursor) and one scalar that holds the varint or fixed32 value
or the payload length. The generated override wraps them in a
ProtoFieldValue that never exists in memory. On the host the switch
key is the field number derived with one shift and the guard compares
the whole tag against the constant the case declares, which is the
same two instructions the old per wire type dispatch cost.

The loop also handles single byte varints inline instead of going
through the parse result struct, which drops the materialized consumed
count and its add on every tag and small value.
2026-09-07 15:57:13 +02:00
J. Nick Koston b0ce7f58f3 [api] Collapse the three protobuf decode virtuals into one
Every decodable message overrode up to three virtuals, one per wire
type, so each carried a five slot vtable and up to three functions
with their own prologue and return tails. The shared decode loop now
parses the payload for the wire type into a ProtoFieldValue and calls
a single decode_field() virtual with the tag, the field number and the
wire type; the generated override is one switch.

The switch key is chosen per target through PROTO_DECODE_KEY. Embedded
builds compile switches to compare chains (ESP-IDF passes
-fno-jump-tables), so they key on the full wire tag, one compare per
field with no separate wire type check. The host compiler builds a
jump table for the dense field number switch, so there the key is the
field number and PROTO_DECODE_GUARD rejects a mismatched wire type.
Both forms drop a field that arrives with a wire type it does not
declare, exactly as the per wire type virtuals did.

Per decodable message the vtable shrinks from 20 to 12 bytes on
xtensa and the extra decode functions fold into one; the shared loop
shrinks as well. Host instruction counts per decoded field are
unchanged apart from the guard compare, which replaces the prologue of
the separate function it used to call.
2026-09-07 15:57:12 +02:00
J. Nick Koston 455e1d6374 [api] Assert the switch frame count instead of reading for it separately 2026-09-07 15:57:12 +02:00
J. Nick Koston 3af1d50bce [api] Trim the field free message test and a duplicated generator note 2026-09-07 15:57:12 +02:00
J. Nick Koston 842f354a05 [api] Add an integration test for field free messages
Ping, device info, list entities done and disconnect all travel through
the ProtoMessage static entry points now that the no-op thunk is gone.
2026-09-07 15:57:12 +02:00
J. Nick Koston af9b59d4bd [api] Trim the type erased entry point comments 2026-09-07 15:57:12 +02:00
J. Nick Koston 55fc5a10de [api] Tighten the ProtoMessage default entry point comment 2026-09-07 15:57:12 +02:00
J. Nick Koston 79927b918b [api] Clarify which encode entry points forward on ProtoMessage
The base class defaults are independent no-ops; only generated message
classes forward encode() and calculate_size() to their statics.
2026-09-07 15:57:12 +02:00
J. Nick Koston 1b070629bc [api] Make generated encode and size entry points type erased
Every message sent through send_message or the entity paths needed a
proto_encode_msg<T> thunk (17 bytes on xtensa) and, for entity state
and info messages, a calc_size<T> thunk, because the generated encode
and calculate_size were member functions and the connection code wants
plain function pointers over const void *.

The generator now emits the bodies as static encode_msg(const void *)
and calc_size_msg(const void *) functions, so &T::encode_msg is
already a MessageEncodeFn and the thunks disappear. The member
encode() and calculate_size() remain as inline forwarders for direct
callers. ProtoMessage carries the same static defaults for messages
without fields, which also removes the separate no-op encode thunk.
2026-09-07 15:57:12 +02:00
J. Nick Koston c4e1360cdf [api] Check the encoded end against the reserved size under ESPHOME_DEBUG_API
The fixed32 store helper moves to a private section since it neither bounds checks nor
advances the cursor, its comment describes the path each target takes, the generated
file scan flags any ProtoEncode call that does not assign the cursor, and StateWaiter
timeouts can carry a label so gathered waits are told apart.
2026-09-07 15:57:10 +02:00
J. Nick Koston 7c774699d7 [api] Outline the fixed32 writers on ARM cores without unaligned access too
Cortex-M0+ and ARM9 turn the four byte unaligned store into a memcpy call with a stack
temporary at every fixed32 field, and the outlined helper itself became a memcpy call
there, so the helper now spells out the byte stores. Xtensa and host objects are byte for
byte unchanged; on the RP2040 bench config the api object loses 28 bytes and the fixed32
memcpy calls.
2026-09-07 15:24:30 +02:00
J. Nick Koston ea71a24a9b [api] Mark the last two raw varint writers nodiscard and make StateWaiter failures visible
A predicate that raises now fails its wait instead of dying inside the state callback,
and a timeout names the predicate it was waiting for.
2026-09-07 14:01:34 +02:00
J. Nick Koston 709a1e1eb6 [api] Mark the raw encode helpers nodiscard too and drop a duplicate cursor test
The generated file scan already covers every emitted call, so the parametrized copy of
the same assertion goes.
2026-09-07 13:48:53 +02:00
J. Nick Koston 822b701792 [api] Mark the cursor returning encode helpers nodiscard
A call that drops the returned cursor would silently truncate the message, so the
compiler now warns on it and a unit test scans the generated file for the same mistake.
Also corrects the outlining comment for ESP8266, where the inline write is a few byte
stores rather than one, and the RAW_ENCODE_MAP annotation.
2026-09-07 12:37:48 +02:00
J. Nick Koston d2e4d2c46a [api] Outline the fixed32 writers only where memcpy is a call
On the ESP8266 the inline write was already a single store, so the
outlined helper cost a call per fixed32 field: sensor state encode went
from 615 to 864 ns on a d1 mini. ESP32 builds pass -fno-builtin-memcpy,
where the shared copy is both smaller and faster (562 to 328 ns on an
atom), so the gate is now USE_ESP32.
2026-09-07 11:40:01 +02:00
J. Nick Koston adbbda4072 [api] Emit every encode call through one generator helper
_encode_call() owns the cursor assignment and the _force suffix, so
the convention lives in one place instead of at every emission site;
the fixed32 fast path is an arm of the generic encode_content keyed by
a per type value template. write_fixed32_le uses convert_little_endian
instead of its own byte order switch. The integration test shares a
StateWaiter from state_utils and leaves the disconnect to the fixture.
2026-09-07 11:09:07 +02:00
J. Nick Koston 252bf6ea6a [api] Add an integration test for the encode branch boundaries
Covers a zero float that is skipped on the wire, a fixed32 state, a
negative int32, list entity strings and text states whose length
prefix needs two varint bytes, a two byte field tag through the
device info area, and the field free disconnect exchange.
2026-09-07 10:58:31 +02:00
J. Nick Koston 8ec9305688 [api] Trim the encode helper comments 2026-09-07 10:48:18 +02:00
J. Nick Koston 490aca17e6 [api] Share the fixed32 emission between float and fixed32 fields
One helper next to the other precomputed tag paths decides how a
single byte tag fixed32 field is written; the float and fixed32 types
only differ in the value expression. Drop the non forced std::string
encode_string overload, which the generator never emits, and build the
generator tests from one block of field type constants.
2026-09-07 10:33:21 +02:00
J. Nick Koston b77e2441d4 [api] Undefine PROTO_OUTLINE_FOR_SIZE after the encode helpers
The macro only exists for the two fixed32 writers in ProtoEncode, so
drop it once the class is complete instead of leaking it into every
translation unit that includes proto.h.
2026-09-07 10:09:46 +02:00
J. Nick Koston 3b14f4dfc8 [api] Pass the encode cursor by value through the protobuf helpers
The ProtoEncode helpers took the write cursor by reference and a
bool force flag. At -Os the compiler outlines most of them, so every
call site had to keep pos in a stack slot and pass its address, plus
a constant for the flag. The helpers now take the cursor by value and
return the advanced cursor, so consecutive calls chain through the
return register; forced fields call a _force overload instead of
passing a flag.

The fixed32 writers use __builtin_memcpy, which stays a builtin under
ESP-IDF's -fno-builtin-memcpy, and are outlined on embedded targets so
each fixed32 or float field is a short call instead of an inline
memcpy call. Non-forced float and fixed32 fields with a single-byte
tag share the same writer behind a zero check.

Generated encode bodies shrink by 18 percent on an ESP32 IDF proxy
build (2360 to 1932 bytes for 27 messages); entity messages gain the
most, for example ListEntitiesSensorResponse::encode 190 to 134 bytes
and SensorStateResponse::encode 78 to 49 bytes.
2026-09-07 09:21:54 +02:00
95 changed files with 6267 additions and 5958 deletions
+2 -11
View File
@@ -244,20 +244,11 @@ jobs:
steps: steps:
- name: Check out code from GitHub - name: Check out code from GitHub
uses: actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1 # v7.0.1 uses: actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1 # v7.0.1
- name: Read prek version from requirements_test.txt
id: prek
# requirements_test.txt is the only place the version is pinned, so a
# Dependabot bump there is picked up here without a second edit.
run: |
if ! version=$(sed -nE 's/^prek==([^[:space:]#]+).*/\1/p' requirements_test.txt) || [ -z "$version" ]; then
echo "::error::No prek== pin found in requirements_test.txt."
exit 1
fi
echo "version=$version" >> "$GITHUB_OUTPUT"
- name: Run prek - name: Run prek
uses: j178/prek-action@4e14d07f9231acabce116ccfca13b13dd9755ece # v3.0.0 uses: j178/prek-action@4e14d07f9231acabce116ccfca13b13dd9755ece # v3.0.0
with: with:
prek-version: ${{ steps.prek.outputs.version }} # Keep in sync with requirements_test.txt.
prek-version: "0.4.11"
# This job only runs on pull requests, so nothing ever populates # This job only runs on pull requests, so nothing ever populates
# the cache on dev. Every run would miss and then write a per-pull # the cache on dev. Every run would miss and then write a per-pull
# request copy, which is what the old seed-cache job existed to # request copy, which is what the old seed-cache job existed to
@@ -1,94 +0,0 @@
# Keeps pre-commit hook revs in sync with the requirements files.
#
# Dependabot only bumps the pins in requirements*.txt. Some of those tools
# are pinned again as hook revs in .pre-commit-config.yaml. This workflow
# runs script/sync_dependency_versions.py against the pull request branch
# and pushes a commit with the revs updated.
name: Sync dependency versions
on:
# pull_request_target rather than pull_request so the App secret is
# available on Dependabot pull requests (pull_request runs opened by
# Dependabot only see Dependabot secrets). The job below only touches
# branches in this repository and only ever executes the script from the
# base branch checkout, so fork code never runs with the token.
pull_request_target:
types: [opened, synchronize, reopened]
paths:
- requirements_dev.txt
- requirements_test.txt
- .pre-commit-config.yaml
- script/sync_dependency_versions.py
# The push to the pull request branch uses the App token minted below, so
# the workflow's GITHUB_TOKEN does not need any scopes.
permissions: {}
concurrency:
group: ${{ github.workflow }}-${{ github.event.pull_request.number }}
cancel-in-progress: true
jobs:
sync:
name: Sync pinned versions
runs-on: ubuntu-latest
# Same-repository branches only: a push to a fork is not possible with
# this token, and it keeps untrusted heads out of a privileged job.
if: >-
github.repository == 'esphome/esphome'
&& github.event.pull_request.head.repo.full_name == github.repository
steps:
- name: Generate a token
id: generate-token
uses: actions/create-github-app-token@bcd2ba49218906704ab6c1aa796996da409d3eb1 # v3.2.0
with:
client-id: ${{ vars.ESPHOME_GITHUB_APP_CLIENT_ID }}
private-key: ${{ secrets.ESPHOME_GITHUB_APP_PRIVATE_KEY }}
# A push made with the workflow's own GITHUB_TOKEN would not start
# CI on the new commit; a push with the App token does.
permission-contents: write # git push of the sync commit to the pull request branch
- name: Check out base branch
# Provides the script that runs below. Deliberately the base branch
# so the pull request cannot change what executes here.
uses: actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1 # v7.0.1
with:
ref: ${{ github.event.pull_request.base.sha }}
persist-credentials: false
- name: Check out pull request branch
# No allow-unsafe-pr-checkout here on purpose: checkout v7 only
# refuses heads that live in a different repository, and the job
# condition above already limits runs to same-repository branches.
# Leaving it off keeps that refusal as a backstop for fork heads.
uses: actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1 # v7.0.1
with:
ref: ${{ github.event.pull_request.head.ref }}
path: pull-request
token: ${{ steps.generate-token.outputs.token }}
- name: Set up Python
uses: actions/setup-python@5fda3b95a4ea91299a34e894583c3862153e4b97 # v7.0.0
with:
python-version: "3.12"
- name: Install yamlrocks
# The script edits YAML through yamlrocks. Take the pin from the
# base branch requirements so this workflow has no copy of its own.
run: pip install "$(grep -E '^yamlrocks==' requirements_test.txt | cut -d'#' -f1)"
- name: Sync pinned versions
run: python script/sync_dependency_versions.py --root pull-request
- name: Push changes
working-directory: pull-request
run: |
if git diff --quiet; then
echo "All pinned versions already match the requirements files."
exit 0
fi
git config user.name "esphome[bot]"
git config user.email "115708604+esphome[bot]@users.noreply.github.com"
git commit -am "Sync pinned tool versions with requirements files"
git push
+3 -2
View File
@@ -1,6 +1,7 @@
--- ---
# See https://pre-commit.com for more information # See https://pre-commit.com for more information
# See https://pre-commit.com/hooks.html for more hooks # See https://pre-commit.com/hooks.html for more hooks
ci: ci:
autoupdate_commit_msg: 'pre-commit: autoupdate' autoupdate_commit_msg: 'pre-commit: autoupdate'
autoupdate_schedule: off # Disabled until ruff versions are synced between deps and pre-commit autoupdate_schedule: off # Disabled until ruff versions are synced between deps and pre-commit
@@ -10,7 +11,7 @@ ci:
repos: repos:
- repo: https://github.com/astral-sh/ruff-pre-commit - repo: https://github.com/astral-sh/ruff-pre-commit
# Ruff version. # Ruff version.
rev: v0.16.6 rev: v0.16.3
hooks: hooks:
# Run the linter. # Run the linter.
- id: ruff - id: ruff
@@ -41,7 +42,7 @@ repos:
- id: pyupgrade - id: pyupgrade
args: [--py312-plus] args: [--py312-plus]
- repo: https://github.com/adrienverge/yamllint.git - repo: https://github.com/adrienverge/yamllint.git
rev: v1.38.0 rev: v1.37.1
hooks: hooks:
- id: yamllint - id: yamllint
exclude: ^(\.clang-format|\.clang-tidy)$ exclude: ^(\.clang-format|\.clang-tidy)$
+1 -1
View File
@@ -840,7 +840,7 @@ file does, and it is the authority when they disagree. The most useful starting
cv.rename_key( cv.rename_key(
CONF_OLD_KEY, CONF_NEW_KEY, removed_in="2026.6.0", component="my_component" CONF_OLD_KEY, CONF_NEW_KEY, removed_in="2026.6.0", component="my_component"
), ),
cv.Schema({...}), cv.Schema({ ... }),
) )
``` ```
For other deprecations, warn manually during validation: For other deprecations, warn manually during validation:
+1 -1
View File
@@ -22,7 +22,7 @@ RUN \
-r /requirements.txt -r /requirements.txt
# Install the ESPHome Device Builder dashboard. # Install the ESPHome Device Builder dashboard.
RUN uv pip install --no-cache-dir esphome-device-builder==1.14.5 RUN uv pip install --no-cache-dir esphome-device-builder==1.14.4
RUN \ RUN \
platformio settings set enable_telemetry No \ platformio settings set enable_telemetry No \
+3 -1
View File
@@ -23,7 +23,9 @@ from esphome.util import safe_print
if TYPE_CHECKING: if TYPE_CHECKING:
from collections.abc import Callable from collections.abc import Callable
from aioesphomeapi.api_pb2 import SubscribeLogsResponse # pylint: disable=no-name-in-module from aioesphomeapi.api_pb2 import (
SubscribeLogsResponse, # pylint: disable=no-name-in-module
)
_LOGGER = logging.getLogger(__name__) _LOGGER = logging.getLogger(__name__)
+56 -51
View File
@@ -13,7 +13,7 @@ void Anova::dump_config() { LOG_CLIMATE("", "Anova BLE Cooker", this); }
void Anova::setup() { void Anova::setup() {
this->codec_ = make_unique<AnovaCodec>(); this->codec_ = make_unique<AnovaCodec>();
this->poll_step_ = PollStep::IDLE; this->current_request_ = 0;
} }
void Anova::loop() { void Anova::loop() {
@@ -22,15 +22,6 @@ void Anova::loop() {
this->disable_loop(); this->disable_loop();
} }
void Anova::write_request_(AnovaPacket *pkt) {
auto status =
esp_ble_gattc_write_char(this->parent_->get_gattc_if(), this->parent_->get_conn_id(), this->char_handle_,
pkt->length, pkt->data, ESP_GATT_WRITE_TYPE_NO_RSP, ESP_GATT_AUTH_REQ_NONE);
if (status) {
ESP_LOGW(TAG, "[%s] esp_ble_gattc_write_char failed, status=%d", this->parent_->address_str(), status);
}
}
void Anova::control(const ClimateCall &call) { void Anova::control(const ClimateCall &call) {
auto mode_val = call.get_mode(); auto mode_val = call.get_mode();
if (mode_val.has_value()) { if (mode_val.has_value()) {
@@ -47,11 +38,22 @@ void Anova::control(const ClimateCall &call) {
ESP_LOGW(TAG, "Unsupported mode: %d", mode); ESP_LOGW(TAG, "Unsupported mode: %d", mode);
return; return;
} }
this->write_request_(pkt); auto status =
esp_ble_gattc_write_char(this->parent_->get_gattc_if(), this->parent_->get_conn_id(), this->char_handle_,
pkt->length, pkt->data, ESP_GATT_WRITE_TYPE_NO_RSP, ESP_GATT_AUTH_REQ_NONE);
if (status) {
ESP_LOGW(TAG, "[%s] esp_ble_gattc_write_char failed, status=%d", this->parent_->address_str(), status);
}
} }
auto target_temp = call.get_target_temperature(); auto target_temp = call.get_target_temperature();
if (target_temp.has_value()) { if (target_temp.has_value()) {
this->write_request_(this->codec_->get_set_target_temp_request(*target_temp)); auto *pkt = this->codec_->get_set_target_temp_request(*target_temp);
auto status =
esp_ble_gattc_write_char(this->parent_->get_gattc_if(), this->parent_->get_conn_id(), this->char_handle_,
pkt->length, pkt->data, ESP_GATT_WRITE_TYPE_NO_RSP, ESP_GATT_AUTH_REQ_NONE);
if (status) {
ESP_LOGW(TAG, "[%s] esp_ble_gattc_write_char failed, status=%d", this->parent_->address_str(), status);
}
} }
} }
@@ -60,7 +62,6 @@ void Anova::gattc_event_handler(esp_gattc_cb_event_t event, esp_gatt_if_t gattc_
case ESP_GATTC_DISCONNECT_EVT: { case ESP_GATTC_DISCONNECT_EVT: {
this->current_temperature = NAN; this->current_temperature = NAN;
this->target_temperature = NAN; this->target_temperature = NAN;
this->poll_step_ = PollStep::IDLE;
this->publish_state(); this->publish_state();
break; break;
} }
@@ -82,8 +83,8 @@ void Anova::gattc_event_handler(esp_gattc_cb_event_t event, esp_gatt_if_t gattc_
} }
case ESP_GATTC_REG_FOR_NOTIFY_EVT: { case ESP_GATTC_REG_FOR_NOTIFY_EVT: {
this->node_state = espbt::ClientState::ESTABLISHED; this->node_state = espbt::ClientState::ESTABLISHED;
this->poll_step_ = PollStep::IDLE; this->current_request_ = 0;
this->update(); // begin the first poll cycle immediately this->update();
break; break;
} }
case ESP_GATTC_NOTIFY_EVT: { case ESP_GATTC_NOTIFY_EVT: {
@@ -100,30 +101,33 @@ void Anova::gattc_event_handler(esp_gattc_cb_event_t event, esp_gatt_if_t gattc_
this->mode = this->codec_->running_ ? climate::CLIMATE_MODE_HEAT : climate::CLIMATE_MODE_OFF; this->mode = this->codec_->running_ ? climate::CLIMATE_MODE_HEAT : climate::CLIMATE_MODE_OFF;
} }
if (this->codec_->has_unit()) { if (this->codec_->has_unit()) {
ESP_LOGD(TAG, "Anova units is %s", (this->codec_->unit_ == 'f') ? "fahrenheit" : "celsius"); this->fahrenheit_ = (this->codec_->unit_ == 'f');
ESP_LOGD(TAG, "Anova units is %s", this->fahrenheit_ ? "fahrenheit" : "celsius");
this->current_request_++;
} }
this->publish_state(); this->publish_state();
// Advance the poll cycle to its next request based on the reply we got. if (this->current_request_ > 1) {
switch (this->poll_step_) { AnovaPacket *pkt = nullptr;
case PollStep::SET_UNIT: switch (this->current_request_++) {
this->poll_step_ = PollStep::STATUS; case 2:
this->write_request_(this->codec_->get_read_device_status_request()); pkt = this->codec_->get_read_target_temp_request();
break; break;
case PollStep::STATUS: case 3:
this->poll_step_ = PollStep::TARGET; pkt = this->codec_->get_read_current_temp_request();
this->write_request_(this->codec_->get_read_target_temp_request()); break;
break; default:
case PollStep::TARGET: this->current_request_ = 1;
this->poll_step_ = PollStep::CURRENT; break;
this->write_request_(this->codec_->get_read_current_temp_request()); }
break; if (pkt != nullptr) {
case PollStep::CURRENT: auto status =
this->poll_step_ = PollStep::IDLE; // full cycle complete esp_ble_gattc_write_char(this->parent_->get_gattc_if(), this->parent_->get_conn_id(), this->char_handle_,
break; pkt->length, pkt->data, ESP_GATT_WRITE_TYPE_NO_RSP, ESP_GATT_AUTH_REQ_NONE);
default: if (status) {
// A reply to an ad-hoc control() write, outside a managed cycle. ESP_LOGW(TAG, "[%s] esp_ble_gattc_write_char failed, status=%d", this->parent_->address_str(), status);
break; }
}
} }
break; break;
} }
@@ -132,26 +136,27 @@ void Anova::gattc_event_handler(esp_gattc_cb_event_t event, esp_gatt_if_t gattc_
} }
} }
void Anova::set_unit_of_measurement(const char *unit) { this->want_fahrenheit_ = !strncmp(unit, "f", 1); } void Anova::set_unit_of_measurement(const char *unit) { this->fahrenheit_ = !strncmp(unit, "f", 1); }
void Anova::update() { void Anova::update() {
if (this->node_state != espbt::ClientState::ESTABLISHED) if (this->node_state != espbt::ClientState::ESTABLISHED)
return; return;
if (this->poll_step_ != PollStep::IDLE) {
// The previous cycle never finished within a full polling interval -- a if (this->current_request_ < 2) {
// reply was missed or a write failed. Restart the cycle rather than stall; AnovaPacket *pkt;
// the polling interval itself acts as the timeout. A late reply from the if (this->current_request_ == 0) {
// abandoned cycle is harmless: state decoding happens on every notify pkt = this->codec_->get_set_unit_request(this->fahrenheit_ ? 'f' : 'c');
// regardless of step, and each notify sends at most one follow-up request. } else {
ESP_LOGW(TAG, "[%s] Poll cycle incomplete (step %u); restarting cycle", this->parent_->address_str(), pkt = this->codec_->get_read_device_status_request();
static_cast<uint8_t>(this->poll_step_)); }
auto status =
esp_ble_gattc_write_char(this->parent_->get_gattc_if(), this->parent_->get_conn_id(), this->char_handle_,
pkt->length, pkt->data, ESP_GATT_WRITE_TYPE_NO_RSP, ESP_GATT_AUTH_REQ_NONE);
if (status) {
ESP_LOGW(TAG, "[%s] esp_ble_gattc_write_char failed, status=%d", this->parent_->address_str(), status);
}
this->current_request_++;
} }
// Re-assert the configured unit at the start of every poll cycle, then fall
// through the status/temperature reads via the notification handler. Always
// command the configured unit (want_fahrenheit_) -- never the last value the
// device reported, or a drift to 'c' would lock itself in.
this->poll_step_ = PollStep::SET_UNIT;
this->write_request_(this->codec_->get_set_unit_request(this->want_fahrenheit_ ? 'f' : 'c'));
} }
} // namespace esphome::anova } // namespace esphome::anova
+2 -11
View File
@@ -37,20 +37,11 @@ class Anova final : public climate::Climate, public esphome::ble_client::BLEClie
void set_unit_of_measurement(const char *unit); void set_unit_of_measurement(const char *unit);
protected: protected:
// A poll cycle re-asserts the configured unit, then reads device state.
// Re-asserting every cycle prevents the cooker from silently reverting to
// its default (Celsius); previously the unit was only set once on
// connection, so a drift persisted (and corrupted the F/C interpretation of
// subsequent readings) until the BLE link was re-established.
enum class PollStep : uint8_t { SET_UNIT, STATUS, TARGET, CURRENT, IDLE };
void write_request_(AnovaPacket *pkt);
std::unique_ptr<AnovaCodec> codec_; std::unique_ptr<AnovaCodec> codec_;
void control(const climate::ClimateCall &call) override; void control(const climate::ClimateCall &call) override;
uint16_t char_handle_; uint16_t char_handle_;
bool want_fahrenheit_{true}; // configured target unit; never overwritten by device replies uint8_t current_request_;
PollStep poll_step_{PollStep::IDLE}; bool fahrenheit_;
}; };
} // namespace esphome::anova } // namespace esphome::anova
+6 -1
View File
@@ -2255,7 +2255,12 @@ bool APIConnection::send_message_(uint32_t payload_size, uint16_t message_type,
// Capacity reserved above, cannot fail // Capacity reserved above, cannot fail
(void) shared_buf.resize(write_start + payload_size); (void) shared_buf.resize(write_start + payload_size);
ProtoWriteBuffer buffer{&shared_buf, write_start}; ProtoWriteBuffer buffer{&shared_buf, write_start};
encode_fn(msg, buffer PROTO_ENCODE_DEBUG_INIT(&shared_buf)); uint8_t *end = encode_fn(msg, buffer PROTO_ENCODE_DEBUG_INIT(&shared_buf));
#ifdef ESPHOME_DEBUG_API
assert(end == shared_buf.data() + shared_buf.size());
#else
(void) end;
#endif
return this->send_buffer(ProtoWriteBuffer{&shared_buf}, message_type); return this->send_buffer(ProtoWriteBuffer{&shared_buf}, message_type);
} }
// encode_to_buffer is defined inline in api_connection.h (ESPHOME_ALWAYS_INLINE) // encode_to_buffer is defined inline in api_connection.h (ESPHOME_ALWAYS_INLINE)
+6 -24
View File
@@ -345,11 +345,7 @@ class APIConnection final : public APIServerConnectionBase {
/// Returns false as soon as the TCP buffer is full. Marked nodiscard so we /// Returns false as soon as the TCP buffer is full. Marked nodiscard so we
/// have no silent failures: every caller must handle (or log) a refusal. /// have no silent failures: every caller must handle (or log) a refusal.
template<typename T> [[nodiscard]] bool send_message(const T &msg) { template<typename T> [[nodiscard]] bool send_message(const T &msg) {
if constexpr (T::ESTIMATED_SIZE == 0) { return this->send_message_(T::calc_size_msg(&msg), T::MESSAGE_TYPE, &T::encode_msg, &msg);
return this->send_message_(0, T::MESSAGE_TYPE, &encode_msg_noop, &msg);
} else {
return this->send_message_(msg.calculate_size(), T::MESSAGE_TYPE, &proto_encode_msg<T>, &msg);
}
} }
/// Clear the shared write buffer and reserve space for the first message. /// Clear the shared write buffer and reserve space for the first message.
@@ -405,16 +401,6 @@ class APIConnection final : public APIServerConnectionBase {
void process_state_subscriptions_(); void process_state_subscriptions_();
#endif #endif
// Size thunk — converts void* back to concrete type for direct calculate_size() call
template<typename T> static uint32_t calc_size(const void *msg) {
return static_cast<const T *>(msg)->calculate_size();
}
// Shared no-op encode thunk for empty messages (ESTIMATED_SIZE == 0)
static uint8_t *encode_msg_noop(const void *, ProtoWriteBuffer &buf PROTO_ENCODE_DEBUG_PARAM) {
return buf.get_pos();
}
// Non-template buffer management for send_message // Non-template buffer management for send_message
bool send_message_(uint32_t payload_size, uint16_t message_type, MessageEncodeFn encode_fn, const void *msg); bool send_message_(uint32_t payload_size, uint16_t message_type, MessageEncodeFn encode_fn, const void *msg);
@@ -433,11 +419,7 @@ class APIConnection final : public APIServerConnectionBase {
// Hot paths (state/info) go through fill_and_encode_entity_state/info instead. // Hot paths (state/info) go through fill_and_encode_entity_state/info instead.
// batch_message_type_ is already set by dispatch_message_ before reaching here. // batch_message_type_ is already set by dispatch_message_ before reaching here.
template<typename T> static uint16_t encode_message_to_buffer(T &msg, APIConnection *conn, uint32_t remaining_size) { template<typename T> static uint16_t encode_message_to_buffer(T &msg, APIConnection *conn, uint32_t remaining_size) {
if constexpr (T::ESTIMATED_SIZE == 0) { return encode_to_buffer_slow(T::calc_size_msg(&msg), &T::encode_msg, &msg, conn, remaining_size);
return encode_to_buffer_slow(0, &encode_msg_noop, &msg, conn, remaining_size);
} else {
return encode_to_buffer_slow(msg.calculate_size(), &proto_encode_msg<T>, &msg, conn, remaining_size);
}
} }
// Non-template core — fills state fields and encodes // Non-template core — fills state fields and encodes
@@ -449,7 +431,7 @@ class APIConnection final : public APIServerConnectionBase {
template<typename T> template<typename T>
static uint16_t fill_and_encode_entity_state(EntityBase *entity, T &msg, APIConnection *conn, static uint16_t fill_and_encode_entity_state(EntityBase *entity, T &msg, APIConnection *conn,
uint32_t remaining_size) { uint32_t remaining_size) {
return fill_and_encode_entity_state(entity, msg, &calc_size<T>, &proto_encode_msg<T>, conn, remaining_size); return fill_and_encode_entity_state(entity, msg, &T::calc_size_msg, &T::encode_msg, conn, remaining_size);
} }
// Non-template core — fills info fields, allocates buffers, and encodes // Non-template core — fills info fields, allocates buffers, and encodes
@@ -461,7 +443,7 @@ class APIConnection final : public APIServerConnectionBase {
template<typename T> template<typename T>
static uint16_t fill_and_encode_entity_info(EntityBase *entity, T &msg, APIConnection *conn, static uint16_t fill_and_encode_entity_info(EntityBase *entity, T &msg, APIConnection *conn,
uint32_t remaining_size) { uint32_t remaining_size) {
return fill_and_encode_entity_info(entity, msg, &calc_size<T>, &proto_encode_msg<T>, conn, remaining_size); return fill_and_encode_entity_info(entity, msg, &T::calc_size_msg, &T::encode_msg, conn, remaining_size);
} }
// Non-template core — fills device_class, then delegates to fill_and_encode_entity_info // Non-template core — fills device_class, then delegates to fill_and_encode_entity_info
@@ -475,8 +457,8 @@ class APIConnection final : public APIServerConnectionBase {
static uint16_t fill_and_encode_entity_info_with_device_class(EntityBase *entity, T &msg, static uint16_t fill_and_encode_entity_info_with_device_class(EntityBase *entity, T &msg,
StringRef &device_class_field, APIConnection *conn, StringRef &device_class_field, APIConnection *conn,
uint32_t remaining_size) { uint32_t remaining_size) {
return fill_and_encode_entity_info_with_device_class(entity, msg, device_class_field, &calc_size<T>, return fill_and_encode_entity_info_with_device_class(entity, msg, device_class_field, &T::calc_size_msg,
&proto_encode_msg<T>, conn, remaining_size); &T::encode_msg, conn, remaining_size);
} }
#ifdef USE_VOICE_ASSISTANT #ifdef USE_VOICE_ASSISTANT
@@ -46,7 +46,13 @@ inline uint16_t ESPHOME_ALWAYS_INLINE APIConnection::encode_to_buffer(uint32_t c
return 0; return 0;
} }
ProtoWriteBuffer buffer{&shared_buf, shared_buf.size() - calculated_size}; ProtoWriteBuffer buffer{&shared_buf, shared_buf.size() - calculated_size};
encode_fn(msg, buffer PROTO_ENCODE_DEBUG_INIT(&shared_buf)); uint8_t *end = encode_fn(msg, buffer PROTO_ENCODE_DEBUG_INIT(&shared_buf));
#ifdef ESPHOME_DEBUG_API
// A body that writes fewer bytes than calculate_size() promised would ship stale buffer bytes
assert(end == shared_buf.data() + shared_buf.size());
#else
(void) end;
#endif
return total_calculated_size; return total_calculated_size;
} }
File diff suppressed because it is too large Load Diff
File diff suppressed because it is too large Load Diff
+3 -2
View File
@@ -433,8 +433,9 @@ void APIServer::send_homeassistant_action(const HomeassistantActionRequest &call
// Home Assistant subscribes to actions shortly *after* authenticating, so actions // Home Assistant subscribes to actions shortly *after* authenticating, so actions
// fired right at connection time (on_client_connected, on_time_sync, ...) can // fired right at connection time (on_client_connected, on_time_sync, ...) can
// arrive before the subscription and are lost - warn instead of failing silently. // arrive before the subscription and are lost - warn instead of failing silently.
ESP_LOGW(TAG, "Home Assistant %s '%s' dropped; %s", ESP_LOGW(TAG, "Home Assistant %s '%.*s' dropped; %s",
call.is_event ? LOG_STR_LITERAL("event") : LOG_STR_LITERAL("action"), call.service.c_str(), call.is_event ? LOG_STR_LITERAL("event") : LOG_STR_LITERAL("action"),
static_cast<int>(call.service.size()), call.service.empty() ? "" : call.service.c_str(),
this->is_connected() ? LOG_STR_LITERAL("client has not subscribed to actions (yet)") this->is_connected() ? LOG_STR_LITERAL("client has not subscribed to actions (yet)")
: LOG_STR_LITERAL("no client connected")); : LOG_STR_LITERAL("no client connected"));
} }
+59 -58
View File
@@ -214,73 +214,74 @@ void ProtoDecodableMessage::decode(const uint8_t *buffer, size_t length) {
const uint8_t *ptr = buffer; const uint8_t *ptr = buffer;
const uint8_t *end = buffer + length; const uint8_t *end = buffer + length;
while (ptr < end) { // Single-byte varints dominate, so that case advances the cursor inline.
// Parse field header - ptr < end guarantees len >= 1 auto read_varint = [&](proto_varint_value_t &value) ESPHOME_ALWAYS_INLINE {
if (ptr == end)
return false;
if (*ptr < 0x80) [[likely]] {
value = *ptr++;
return true;
}
auto res = ProtoVarInt::parse_non_empty(ptr, end - ptr); auto res = ProtoVarInt::parse_non_empty(ptr, end - ptr);
if (!res.has_value()) { if (!res.has_value())
return false;
value = res.value;
ptr += res.consumed;
return true;
};
while (ptr < end) {
proto_varint_value_t tag_value;
if (!read_varint(tag_value)) {
ESP_LOGV(TAG, "Invalid field start at offset %ld", (long) (ptr - buffer)); ESP_LOGV(TAG, "Invalid field start at offset %ld", (long) (ptr - buffer));
return; return;
} }
uint32_t tag = static_cast<uint32_t>(res.value); uint32_t tag = static_cast<uint32_t>(tag_value);
uint32_t field_type = tag & WIRE_TYPE_MASK; uint32_t field_type = tag & WIRE_TYPE_MASK;
uint32_t field_id = tag >> 3; // Length-delimited fields move this past the length prefix
ptr += res.consumed; const uint8_t *data = ptr;
proto_varint_value_t scalar;
switch (field_type) { if (field_type == WIRE_TYPE_VARINT) [[likely]] {
case WIRE_TYPE_VARINT: { // VarInt if (!read_varint(scalar)) {
res = ProtoVarInt::parse(ptr, end - ptr); ESP_LOGV(TAG, "Invalid VarInt at offset %ld", (long) (ptr - buffer));
if (!res.has_value()) {
ESP_LOGV(TAG, "Invalid VarInt at offset %ld", (long) (ptr - buffer));
return;
}
if (!this->decode_varint(field_id, res.value)) {
ESP_LOGV(TAG, "Cannot decode VarInt field %" PRIu32 " with value %" PRIu64 "!", field_id,
static_cast<uint64_t>(res.value));
}
ptr += res.consumed;
break;
}
case WIRE_TYPE_LENGTH_DELIMITED: { // Length-delimited
res = ProtoVarInt::parse(ptr, end - ptr);
if (!res.has_value()) {
ESP_LOGV(TAG, "Invalid Length Delimited at offset %ld", (long) (ptr - buffer));
return;
}
uint32_t field_length = static_cast<uint32_t>(res.value);
ptr += res.consumed;
if (field_length > static_cast<size_t>(end - ptr)) {
ESP_LOGV(TAG, "Out-of-bounds Length Delimited at offset %ld", (long) (ptr - buffer));
return;
}
if (!this->decode_length(field_id, ProtoLengthDelimited(ptr, field_length))) {
ESP_LOGV(TAG, "Cannot decode Length Delimited field %" PRIu32 "!", field_id);
}
ptr += field_length;
break;
}
case WIRE_TYPE_FIXED32: { // 32-bit
if (end - ptr < 4) {
ESP_LOGV(TAG, "Out-of-bounds Fixed32-bit at offset %ld", (long) (ptr - buffer));
return;
}
uint32_t val;
#if __BYTE_ORDER__ == __ORDER_LITTLE_ENDIAN__
// Protobuf fixed32 is little-endian — direct load on LE platforms
memcpy(&val, ptr, 4);
#else
val = encode_uint32(ptr[3], ptr[2], ptr[1], ptr[0]);
#endif
if (!this->decode_32bit(field_id, Proto32Bit(val))) {
ESP_LOGV(TAG, "Cannot decode 32-bit field %" PRIu32 " with value %" PRIu32 "!", field_id, val);
}
ptr += 4;
break;
}
default:
ESP_LOGV(TAG, "Invalid field type %" PRIu32 " at offset %ld", field_type, (long) (ptr - buffer));
return; return;
}
} else {
switch (field_type) {
case WIRE_TYPE_LENGTH_DELIMITED: {
proto_varint_value_t length_value;
if (!read_varint(length_value)) {
ESP_LOGV(TAG, "Invalid Length Delimited at offset %ld", (long) (ptr - buffer));
return;
}
uint32_t field_length = static_cast<uint32_t>(length_value);
if (field_length > static_cast<size_t>(end - ptr)) {
ESP_LOGV(TAG, "Out-of-bounds Length Delimited at offset %ld", (long) (ptr - buffer));
return;
}
data = ptr;
scalar = field_length;
ptr += field_length;
break;
}
case WIRE_TYPE_FIXED32: {
if (end - ptr < 4) {
ESP_LOGV(TAG, "Out-of-bounds Fixed32-bit at offset %ld", (long) (ptr - buffer));
return;
}
// Byte loads instead of memcpy: ESP-IDF passes -fno-builtin-memcpy, which made this a call
scalar = encode_uint32(ptr[3], ptr[2], ptr[1], ptr[0]);
ptr += 4;
break;
}
default:
ESP_LOGV(TAG, "Invalid field type %" PRIu32 " at offset %ld", field_type, (long) (ptr - buffer));
return;
}
} }
this->decode_field(tag, data, scalar);
} }
} }
+228 -166
View File
@@ -170,40 +170,43 @@ class ProtoVarInt {
class ProtoMessage; class ProtoMessage;
class ProtoSize; class ProtoSize;
class ProtoLengthDelimited { /// Case label for decode_field(): the wire tag of a field, so a field that arrives with another wire
/// type matches no case.
constexpr uint32_t proto_tag(uint32_t field_id, uint32_t wire_type) { return (field_id << 3) | wire_type; }
/// One decoded field: the payload pointer and a scalar holding the varint or fixed32 value, or the
/// length of a length-delimited field. The wire type in the tag says which applies; accessors do not check.
class ProtoFieldValue {
public: public:
explicit ProtoLengthDelimited(const uint8_t *value, size_t length) : value_(value), length_(length) {} ProtoFieldValue(const uint8_t *data, proto_varint_value_t scalar) : data_(data), scalar_(scalar) {}
std::string as_string() const { return std::string(reinterpret_cast<const char *>(this->value_), this->length_); }
// Direct access to raw data without string allocation proto_varint_value_t as_varint() const { return this->scalar_; }
const uint8_t *data() const { return this->value_; } // A bool is sent as 0 or 1, so the low word is enough and saves a second compare with 64 bit varints
size_t size() const { return this->length_; } bool as_bool() const { return static_cast<uint32_t>(this->scalar_) != 0; }
/// Decode the length-delimited data into a message instance. // Length-delimited accessors
const uint8_t *data() const { return this->data_; }
size_t size() const { return static_cast<size_t>(this->scalar_); }
std::string as_string() const { return std::string(reinterpret_cast<const char *>(this->data_), this->size()); }
/// Decode the length-delimited payload into a message instance.
/// Template preserves concrete type so decode() resolves statically. /// Template preserves concrete type so decode() resolves statically.
template<typename T> void decode_to_message(T &msg) const; template<typename T> void decode_to_message(T &msg) const { msg.decode(this->data_, this->size()); }
protected: // Fixed32 accessors
const uint8_t *const value_; uint32_t as_fixed32() const { return static_cast<uint32_t>(this->scalar_); }
const size_t length_; int32_t as_sfixed32() const { return static_cast<int32_t>(this->as_fixed32()); }
};
class Proto32Bit {
public:
explicit Proto32Bit(uint32_t value) : value_(value) {}
uint32_t as_fixed32() const { return this->value_; }
int32_t as_sfixed32() const { return static_cast<int32_t>(this->value_); }
float as_float() const { float as_float() const {
union { union {
uint32_t raw; uint32_t raw;
float value; float value;
} s{}; } s{};
s.raw = this->value_; s.raw = this->as_fixed32();
return s.value; return s.value;
} }
protected: private:
const uint32_t value_; const uint8_t *data_;
proto_varint_value_t scalar_;
}; };
// NOTE: Proto64Bit class removed - wire type 1 (64-bit fixed) not supported // NOTE: Proto64Bit class removed - wire type 1 (64-bit fixed) not supported
@@ -252,7 +255,7 @@ class ProtoWriteBuffer {
* *
* Following https://protobuf.dev/programming-guides/encoding/#structure * Following https://protobuf.dev/programming-guides/encoding/#structure
*/ */
void encode_field_raw(uint32_t field_id, uint32_t type) { this->encode_varint_raw((field_id << 3) | type); } void encode_field_raw(uint32_t field_id, uint32_t type) { this->encode_varint_raw(proto_tag(field_id, type)); }
/// Single-pass encode for repeated submessage elements. /// Single-pass encode for repeated submessage elements.
/// Thin template wrapper; all buffer work is in the non-template core. /// Thin template wrapper; all buffer work is in the non-template core.
template<typename T> void encode_sub_message(uint32_t field_id, const T &value); template<typename T> void encode_sub_message(uint32_t field_id, const T &value);
@@ -287,19 +290,31 @@ class ProtoWriteBuffer {
uint8_t *pos_; uint8_t *pos_;
}; };
// A four byte unaligned store is a memcpy call on ESP-IDF (-fno-builtin-memcpy) and on ARM cores without
// unaligned access (Cortex-M0+, ARM9), so those targets share one outlined byte store helper per fixed32
// field. Elsewhere the write inlines to a single store, or on ESP8266 to a few stores that measured
// faster than a call, so it stays inline.
#if defined(USE_ESP32) || (defined(__arm__) && !defined(__ARM_FEATURE_UNALIGNED))
#define PROTO_OUTLINE_FOR_SIZE __attribute__((noinline))
#define PROTO_FIXED32_BYTE_STORES true
#else
#define PROTO_OUTLINE_FOR_SIZE inline
#define PROTO_FIXED32_BYTE_STORES false
#endif
// Varint encoding thresholds — used by both proto_encode_* free functions and ProtoSize. // Varint encoding thresholds — used by both proto_encode_* free functions and ProtoSize.
constexpr uint32_t VARINT_MAX_1_BYTE = 1 << 7; // 128 constexpr uint32_t VARINT_MAX_1_BYTE = 1 << 7; // 128
constexpr uint32_t VARINT_MAX_2_BYTE = 1 << 14; // 16384 constexpr uint32_t VARINT_MAX_2_BYTE = 1 << 14; // 16384
/// Static encode helpers for generated encode() functions. /// Static encode helpers for the generated encode bodies. Each takes the write cursor by value and
/// Generated code hoists buffer.pos_ into a local uint8_t *__restrict__ pos, /// returns it advanced, so outlined calls at -Os chain through the return register instead of a
/// then calls these methods which take pos by reference. No struct, no overhead. /// stack slot. Helpers without a _force suffix skip fields holding the proto3 default.
/// For sub-messages, pos is synced back to buffer before the call and reloaded after.
class ProtoEncode { class ProtoEncode {
public: public:
/// Write a multi-byte varint directly through a pos pointer. /// Write a multi-byte varint directly through a pos pointer.
template<typename T> template<typename T>
static inline void encode_varint_raw_loop(uint8_t *__restrict__ &pos PROTO_ENCODE_DEBUG_PARAM, T value) { [[nodiscard]] static inline uint8_t *encode_varint_raw_loop(uint8_t *__restrict__ pos PROTO_ENCODE_DEBUG_PARAM,
T value) {
do { do {
PROTO_ENCODE_CHECK_BOUNDS(pos, 1); PROTO_ENCODE_CHECK_BOUNDS(pos, 1);
*pos++ = static_cast<uint8_t>(value | 0x80); *pos++ = static_cast<uint8_t>(value | 0x80);
@@ -307,48 +322,49 @@ class ProtoEncode {
} while (value > 0x7F); } while (value > 0x7F);
PROTO_ENCODE_CHECK_BOUNDS(pos, 1); PROTO_ENCODE_CHECK_BOUNDS(pos, 1);
*pos++ = static_cast<uint8_t>(value); *pos++ = static_cast<uint8_t>(value);
return pos;
} }
static inline void ESPHOME_ALWAYS_INLINE encode_varint_raw(uint8_t *__restrict__ &pos PROTO_ENCODE_DEBUG_PARAM, [[nodiscard]] static inline uint8_t *ESPHOME_ALWAYS_INLINE
uint32_t value) { encode_varint_raw(uint8_t *__restrict__ pos PROTO_ENCODE_DEBUG_PARAM, uint32_t value) {
if (value < VARINT_MAX_1_BYTE) [[likely]] { if (value < VARINT_MAX_1_BYTE) [[likely]] {
PROTO_ENCODE_CHECK_BOUNDS(pos, 1); PROTO_ENCODE_CHECK_BOUNDS(pos, 1);
*pos++ = static_cast<uint8_t>(value); *pos++ = static_cast<uint8_t>(value);
return; return pos;
} }
encode_varint_raw_loop(pos PROTO_ENCODE_DEBUG_ARG, value); return encode_varint_raw_loop(pos PROTO_ENCODE_DEBUG_ARG, value);
} }
/// Encode a varint that is expected to be 1-2 bytes (e.g. zigzag RSSI, small lengths). /// Encode a varint that is expected to be 1-2 bytes (e.g. zigzag RSSI, small lengths).
static inline void ESPHOME_ALWAYS_INLINE encode_varint_raw_short(uint8_t *__restrict__ &pos PROTO_ENCODE_DEBUG_PARAM, [[nodiscard]] static inline uint8_t *ESPHOME_ALWAYS_INLINE
uint32_t value) { encode_varint_raw_short(uint8_t *__restrict__ pos PROTO_ENCODE_DEBUG_PARAM, uint32_t value) {
if (value < VARINT_MAX_1_BYTE) [[likely]] { if (value < VARINT_MAX_1_BYTE) [[likely]] {
PROTO_ENCODE_CHECK_BOUNDS(pos, 1); PROTO_ENCODE_CHECK_BOUNDS(pos, 1);
*pos++ = static_cast<uint8_t>(value); *pos++ = static_cast<uint8_t>(value);
return; return pos;
} }
if (value < VARINT_MAX_2_BYTE) [[likely]] { if (value < VARINT_MAX_2_BYTE) [[likely]] {
PROTO_ENCODE_CHECK_BOUNDS(pos, 2); PROTO_ENCODE_CHECK_BOUNDS(pos, 2);
*pos++ = static_cast<uint8_t>(value | 0x80); *pos++ = static_cast<uint8_t>(value | 0x80);
*pos++ = static_cast<uint8_t>(value >> 7); *pos++ = static_cast<uint8_t>(value >> 7);
return; return pos;
} }
encode_varint_raw_loop(pos PROTO_ENCODE_DEBUG_ARG, value); return encode_varint_raw_loop(pos PROTO_ENCODE_DEBUG_ARG, value);
} }
static inline void ESPHOME_ALWAYS_INLINE encode_varint_raw_64(uint8_t *__restrict__ &pos PROTO_ENCODE_DEBUG_PARAM, [[nodiscard]] static inline uint8_t *ESPHOME_ALWAYS_INLINE
uint64_t value) { encode_varint_raw_64(uint8_t *__restrict__ pos PROTO_ENCODE_DEBUG_PARAM, uint64_t value) {
if (value < VARINT_MAX_1_BYTE) [[likely]] { if (value < VARINT_MAX_1_BYTE) [[likely]] {
PROTO_ENCODE_CHECK_BOUNDS(pos, 1); PROTO_ENCODE_CHECK_BOUNDS(pos, 1);
*pos++ = static_cast<uint8_t>(value); *pos++ = static_cast<uint8_t>(value);
return; return pos;
} }
encode_varint_raw_loop(pos PROTO_ENCODE_DEBUG_ARG, value); return encode_varint_raw_loop(pos PROTO_ENCODE_DEBUG_ARG, value);
} }
/// Encode a 48-bit MAC address (stored in a uint64) as varint. /// Encode a 48-bit MAC address (stored in a uint64) as varint.
/// Real MAC addresses occupy the full 48 bits (OUI in upper 24), so the /// Real MAC addresses occupy the full 48 bits (OUI in upper 24), so the
/// fast path -- any non-zero bit in the top 6 of 48 -- emits exactly 7 bytes /// fast path -- any non-zero bit in the top 6 of 48 -- emits exactly 7 bytes
/// with no per-byte branch. Falls back to the general loop otherwise. /// with no per-byte branch. Falls back to the general loop otherwise.
/// Caller must guarantee value fits in 48 bits (checked in debug builds). /// Caller must guarantee value fits in 48 bits (checked in debug builds).
static inline void ESPHOME_ALWAYS_INLINE encode_varint_raw_48bit(uint8_t *__restrict__ &pos PROTO_ENCODE_DEBUG_PARAM, [[nodiscard]] static inline uint8_t *ESPHOME_ALWAYS_INLINE
uint64_t value) { encode_varint_raw_48bit(uint8_t *__restrict__ pos PROTO_ENCODE_DEBUG_PARAM, uint64_t value) {
#ifdef ESPHOME_DEBUG_API #ifdef ESPHOME_DEBUG_API
assert(value < (1ULL << (MAC_ADDRESS_SIZE * 8)) && "encode_varint_raw_48bit: value exceeds 48 bits"); assert(value < (1ULL << (MAC_ADDRESS_SIZE * 8)) && "encode_varint_raw_48bit: value exceeds 48 bits");
#endif #endif
@@ -363,38 +379,39 @@ class ProtoEncode {
pos[4] = static_cast<uint8_t>((value >> 28) | 0x80); pos[4] = static_cast<uint8_t>((value >> 28) | 0x80);
pos[5] = static_cast<uint8_t>((value >> 35) | 0x80); pos[5] = static_cast<uint8_t>((value >> 35) | 0x80);
pos[6] = static_cast<uint8_t>(value >> 42); pos[6] = static_cast<uint8_t>(value >> 42);
pos += 7; return pos + 7;
return;
} }
encode_varint_raw_64(pos PROTO_ENCODE_DEBUG_ARG, value); return encode_varint_raw_64(pos PROTO_ENCODE_DEBUG_ARG, value);
} }
static inline void ESPHOME_ALWAYS_INLINE encode_field_raw(uint8_t *__restrict__ &pos PROTO_ENCODE_DEBUG_PARAM, [[nodiscard]] static inline uint8_t *ESPHOME_ALWAYS_INLINE
uint32_t field_id, uint32_t type) { encode_field_raw(uint8_t *__restrict__ pos PROTO_ENCODE_DEBUG_PARAM, uint32_t field_id, uint32_t type) {
encode_varint_raw(pos PROTO_ENCODE_DEBUG_ARG, (field_id << 3) | type); return encode_varint_raw(pos PROTO_ENCODE_DEBUG_ARG, proto_tag(field_id, type));
} }
/// Write a single precomputed tag byte. Tag must be < 128. /// Write a single precomputed tag byte. Tag must be < 128.
static inline void ESPHOME_ALWAYS_INLINE write_raw_byte(uint8_t *__restrict__ &pos PROTO_ENCODE_DEBUG_PARAM, [[nodiscard]] static inline uint8_t *ESPHOME_ALWAYS_INLINE
uint8_t b) { write_raw_byte(uint8_t *__restrict__ pos PROTO_ENCODE_DEBUG_PARAM, uint8_t b) {
PROTO_ENCODE_CHECK_BOUNDS(pos, 1); PROTO_ENCODE_CHECK_BOUNDS(pos, 1);
*pos++ = b; *pos++ = b;
return pos;
} }
/// Reserve one byte for later backpatch (e.g., sub-message length). /// Reserve one byte for later backpatch (e.g., sub-message length).
/// Advances pos past the reserved byte without writing a value. /// Advances pos past the reserved byte without writing a value.
static inline void ESPHOME_ALWAYS_INLINE reserve_byte(uint8_t *__restrict__ &pos PROTO_ENCODE_DEBUG_PARAM) { [[nodiscard]] static inline uint8_t *ESPHOME_ALWAYS_INLINE
reserve_byte(uint8_t *__restrict__ pos PROTO_ENCODE_DEBUG_PARAM) {
PROTO_ENCODE_CHECK_BOUNDS(pos, 1); PROTO_ENCODE_CHECK_BOUNDS(pos, 1);
pos++; return pos + 1;
} }
/// Write raw bytes to the buffer (no tag, no length prefix). /// Write raw bytes to the buffer (no tag, no length prefix).
static inline void ESPHOME_ALWAYS_INLINE encode_raw(uint8_t *__restrict__ &pos PROTO_ENCODE_DEBUG_PARAM, [[nodiscard]] static inline uint8_t *ESPHOME_ALWAYS_INLINE
const void *data, size_t len) { encode_raw(uint8_t *__restrict__ pos PROTO_ENCODE_DEBUG_PARAM, const void *data, size_t len) {
PROTO_ENCODE_CHECK_BOUNDS(pos, len); PROTO_ENCODE_CHECK_BOUNDS(pos, len);
std::memcpy(pos, data, len); std::memcpy(pos, data, len);
pos += len; return pos + len;
} }
/// Encode tag + 1-byte length + raw string data. For strings with max_data_length < 128. /// Encode tag + 1-byte length + raw string data. For strings with max_data_length < 128.
/// Tag must be a single-byte varint (< 128). Always encodes (no zero check). /// Tag must be a single-byte varint (< 128). Always encodes (no zero check).
static inline void encode_short_string_force(uint8_t *__restrict__ &pos PROTO_ENCODE_DEBUG_PARAM, uint8_t tag, [[nodiscard]] static inline uint8_t *encode_short_string_force(uint8_t *__restrict__ pos PROTO_ENCODE_DEBUG_PARAM,
const StringRef &ref) { uint8_t tag, const StringRef &ref) {
#ifdef ESPHOME_DEBUG_API #ifdef ESPHOME_DEBUG_API
assert(ref.size() < 128 && "encode_short_string_force: string exceeds max_data_length < 128"); assert(ref.size() < 128 && "encode_short_string_force: string exceeds max_data_length < 128");
#endif #endif
@@ -402,137 +419,191 @@ class ProtoEncode {
pos[0] = tag; pos[0] = tag;
pos[1] = static_cast<uint8_t>(ref.size()); pos[1] = static_cast<uint8_t>(ref.size());
std::memcpy(pos + 2, ref.c_str(), ref.size()); std::memcpy(pos + 2, ref.c_str(), ref.size());
pos += 2 + ref.size(); return pos + 2 + ref.size();
} }
/// Write a precomputed tag byte + 32-bit value in one operation. /// Write a precomputed tag byte + 32-bit value. Outlined on embedded: one copy beats inline stores per field.
static inline void ESPHOME_ALWAYS_INLINE write_tag_and_fixed32(uint8_t *__restrict__ &pos PROTO_ENCODE_DEBUG_PARAM, [[nodiscard]] static PROTO_OUTLINE_FOR_SIZE uint8_t *write_tag_and_fixed32(
uint8_t tag, uint32_t value) { uint8_t *__restrict__ pos PROTO_ENCODE_DEBUG_PARAM, uint8_t tag, uint32_t value) {
PROTO_ENCODE_CHECK_BOUNDS(pos, 5); PROTO_ENCODE_CHECK_BOUNDS(pos, 5);
pos[0] = tag; pos[0] = tag;
#if __BYTE_ORDER__ == __ORDER_LITTLE_ENDIAN__ write_fixed32_le(pos + 1, value);
std::memcpy(pos + 1, &value, 4); return pos + 5;
#else
pos[1] = static_cast<uint8_t>(value & 0xFF);
pos[2] = static_cast<uint8_t>((value >> 8) & 0xFF);
pos[3] = static_cast<uint8_t>((value >> 16) & 0xFF);
pos[4] = static_cast<uint8_t>((value >> 24) & 0xFF);
#endif
pos += 5;
} }
static inline void encode_string(uint8_t *__restrict__ &pos PROTO_ENCODE_DEBUG_PARAM, uint32_t field_id, [[nodiscard]] static inline uint8_t *encode_string_force(uint8_t *__restrict__ pos PROTO_ENCODE_DEBUG_PARAM,
const char *string, size_t len, bool force = false) { uint32_t field_id, const char *string, size_t len) {
if (len == 0 && !force) pos = encode_field_raw(pos PROTO_ENCODE_DEBUG_ARG, field_id, 2); // type 2: Length-delimited string
return;
encode_field_raw(pos PROTO_ENCODE_DEBUG_ARG, field_id, 2); // type 2: Length-delimited string
// NOLINTNEXTLINE(readability-inconsistent-ifelse-braces) -- false positive on [[likely]] attribute // NOLINTNEXTLINE(readability-inconsistent-ifelse-braces) -- false positive on [[likely]] attribute
if (len < VARINT_MAX_1_BYTE) [[likely]] { if (len < VARINT_MAX_1_BYTE) [[likely]] {
PROTO_ENCODE_CHECK_BOUNDS(pos, 1 + len); PROTO_ENCODE_CHECK_BOUNDS(pos, 1 + len);
*pos++ = static_cast<uint8_t>(len); *pos++ = static_cast<uint8_t>(len);
} else { } else {
encode_varint_raw_loop(pos PROTO_ENCODE_DEBUG_ARG, len); pos = encode_varint_raw_loop(pos PROTO_ENCODE_DEBUG_ARG, len);
PROTO_ENCODE_CHECK_BOUNDS(pos, len); PROTO_ENCODE_CHECK_BOUNDS(pos, len);
} }
std::memcpy(pos, string, len); std::memcpy(pos, string, len);
pos += len; return pos + len;
} }
static inline void encode_string(uint8_t *__restrict__ &pos PROTO_ENCODE_DEBUG_PARAM, uint32_t field_id, [[nodiscard]] static inline uint8_t *encode_string(uint8_t *__restrict__ pos PROTO_ENCODE_DEBUG_PARAM,
const std::string &value, bool force = false) { uint32_t field_id, const char *string, size_t len) {
encode_string(pos PROTO_ENCODE_DEBUG_ARG, field_id, value.data(), value.size(), force); if (len == 0)
return pos;
return encode_string_force(pos PROTO_ENCODE_DEBUG_ARG, field_id, string, len);
} }
static inline void encode_string(uint8_t *__restrict__ &pos PROTO_ENCODE_DEBUG_PARAM, uint32_t field_id, [[nodiscard]] static inline uint8_t *encode_string_force(uint8_t *__restrict__ pos PROTO_ENCODE_DEBUG_PARAM,
const StringRef &ref, bool force = false) { uint32_t field_id, const std::string &value) {
encode_string(pos PROTO_ENCODE_DEBUG_ARG, field_id, ref.c_str(), ref.size(), force); return encode_string_force(pos PROTO_ENCODE_DEBUG_ARG, field_id, value.data(), value.size());
} }
static inline void encode_bytes(uint8_t *__restrict__ &pos PROTO_ENCODE_DEBUG_PARAM, uint32_t field_id, [[nodiscard]] static inline uint8_t *encode_string(uint8_t *__restrict__ pos PROTO_ENCODE_DEBUG_PARAM,
const uint8_t *data, size_t len, bool force = false) { uint32_t field_id, const StringRef &ref) {
encode_string(pos PROTO_ENCODE_DEBUG_ARG, field_id, reinterpret_cast<const char *>(data), len, force); return encode_string(pos PROTO_ENCODE_DEBUG_ARG, field_id, ref.c_str(), ref.size());
} }
static inline void encode_uint32(uint8_t *__restrict__ &pos PROTO_ENCODE_DEBUG_PARAM, uint32_t field_id, [[nodiscard]] static inline uint8_t *encode_string_force(uint8_t *__restrict__ pos PROTO_ENCODE_DEBUG_PARAM,
uint32_t value, bool force = false) { uint32_t field_id, const StringRef &ref) {
if (value == 0 && !force) return encode_string_force(pos PROTO_ENCODE_DEBUG_ARG, field_id, ref.c_str(), ref.size());
return;
encode_field_raw(pos PROTO_ENCODE_DEBUG_ARG, field_id, 0);
encode_varint_raw(pos PROTO_ENCODE_DEBUG_ARG, value);
} }
static inline void encode_uint64(uint8_t *__restrict__ &pos PROTO_ENCODE_DEBUG_PARAM, uint32_t field_id, [[nodiscard]] static inline uint8_t *encode_bytes(uint8_t *__restrict__ pos PROTO_ENCODE_DEBUG_PARAM,
uint64_t value, bool force = false) { uint32_t field_id, const uint8_t *data, size_t len) {
if (value == 0 && !force) return encode_string(pos PROTO_ENCODE_DEBUG_ARG, field_id, reinterpret_cast<const char *>(data), len);
return;
encode_field_raw(pos PROTO_ENCODE_DEBUG_ARG, field_id, 0);
encode_varint_raw_64(pos PROTO_ENCODE_DEBUG_ARG, value);
} }
static inline void encode_bool(uint8_t *__restrict__ &pos PROTO_ENCODE_DEBUG_PARAM, uint32_t field_id, bool value, [[nodiscard]] static inline uint8_t *encode_bytes_force(uint8_t *__restrict__ pos PROTO_ENCODE_DEBUG_PARAM,
bool force = false) { uint32_t field_id, const uint8_t *data, size_t len) {
if (!value && !force) return encode_string_force(pos PROTO_ENCODE_DEBUG_ARG, field_id, reinterpret_cast<const char *>(data), len);
return; }
encode_field_raw(pos PROTO_ENCODE_DEBUG_ARG, field_id, 0); [[nodiscard]] static inline uint8_t *encode_uint32_force(uint8_t *__restrict__ pos PROTO_ENCODE_DEBUG_PARAM,
uint32_t field_id, uint32_t value) {
pos = encode_field_raw(pos PROTO_ENCODE_DEBUG_ARG, field_id, 0);
return encode_varint_raw(pos PROTO_ENCODE_DEBUG_ARG, value);
}
[[nodiscard]] static inline uint8_t *encode_uint32(uint8_t *__restrict__ pos PROTO_ENCODE_DEBUG_PARAM,
uint32_t field_id, uint32_t value) {
if (value == 0)
return pos;
return encode_uint32_force(pos PROTO_ENCODE_DEBUG_ARG, field_id, value);
}
[[nodiscard]] static inline uint8_t *encode_uint64_force(uint8_t *__restrict__ pos PROTO_ENCODE_DEBUG_PARAM,
uint32_t field_id, uint64_t value) {
pos = encode_field_raw(pos PROTO_ENCODE_DEBUG_ARG, field_id, 0);
return encode_varint_raw_64(pos PROTO_ENCODE_DEBUG_ARG, value);
}
[[nodiscard]] static inline uint8_t *encode_uint64(uint8_t *__restrict__ pos PROTO_ENCODE_DEBUG_PARAM,
uint32_t field_id, uint64_t value) {
if (value == 0)
return pos;
return encode_uint64_force(pos PROTO_ENCODE_DEBUG_ARG, field_id, value);
}
[[nodiscard]] static inline uint8_t *encode_bool_force(uint8_t *__restrict__ pos PROTO_ENCODE_DEBUG_PARAM,
uint32_t field_id, bool value) {
pos = encode_field_raw(pos PROTO_ENCODE_DEBUG_ARG, field_id, 0);
PROTO_ENCODE_CHECK_BOUNDS(pos, 1); PROTO_ENCODE_CHECK_BOUNDS(pos, 1);
*pos++ = value ? 0x01 : 0x00; *pos++ = value ? 0x01 : 0x00;
return pos;
} }
static inline void encode_fixed32(uint8_t *__restrict__ &pos PROTO_ENCODE_DEBUG_PARAM, uint32_t field_id, [[nodiscard]] static inline uint8_t *encode_bool(uint8_t *__restrict__ pos PROTO_ENCODE_DEBUG_PARAM,
uint32_t value, bool force = false) { uint32_t field_id, bool value) {
if (value == 0 && !force) if (!value)
return; return pos;
encode_field_raw(pos PROTO_ENCODE_DEBUG_ARG, field_id, 5); return encode_bool_force(pos PROTO_ENCODE_DEBUG_ARG, field_id, value);
}
/// Tag + fixed32 for multi-byte tags; single-byte tags use write_tag_and_fixed32.
[[nodiscard]] static PROTO_OUTLINE_FOR_SIZE uint8_t *encode_fixed32_force(
uint8_t *__restrict__ pos PROTO_ENCODE_DEBUG_PARAM, uint32_t field_id, uint32_t value) {
pos = encode_field_raw(pos PROTO_ENCODE_DEBUG_ARG, field_id, 5);
PROTO_ENCODE_CHECK_BOUNDS(pos, 4); PROTO_ENCODE_CHECK_BOUNDS(pos, 4);
#if __BYTE_ORDER__ == __ORDER_LITTLE_ENDIAN__ write_fixed32_le(pos, value);
std::memcpy(pos, &value, 4); return pos + 4;
pos += 4; }
#else [[nodiscard]] static inline uint8_t *encode_fixed32(uint8_t *__restrict__ pos PROTO_ENCODE_DEBUG_PARAM,
*pos++ = (value >> 0) & 0xFF; uint32_t field_id, uint32_t value) {
*pos++ = (value >> 8) & 0xFF; if (value == 0)
*pos++ = (value >> 16) & 0xFF; return pos;
*pos++ = (value >> 24) & 0xFF; return encode_fixed32_force(pos PROTO_ENCODE_DEBUG_ARG, field_id, value);
#endif
} }
// NOTE: Wire type 1 (64-bit fixed: double, fixed64, sfixed64) is intentionally // NOTE: Wire type 1 (64-bit fixed: double, fixed64, sfixed64) is intentionally
// not supported to reduce overhead on embedded systems. All ESPHome devices are // not supported to reduce overhead on embedded systems. All ESPHome devices are
// 32-bit microcontrollers where 64-bit operations are expensive. If 64-bit support // 32-bit microcontrollers where 64-bit operations are expensive. If 64-bit support
// is needed in the future, the necessary encoding/decoding functions must be added. // is needed in the future, the necessary encoding/decoding functions must be added.
static inline void encode_float(uint8_t *__restrict__ &pos PROTO_ENCODE_DEBUG_PARAM, uint32_t field_id, float value, [[nodiscard]] static inline uint8_t *encode_float(uint8_t *__restrict__ pos PROTO_ENCODE_DEBUG_PARAM,
bool force = false) { uint32_t field_id, float value) {
uint32_t raw = float_to_raw(value); return encode_fixed32(pos PROTO_ENCODE_DEBUG_ARG, field_id, float_to_raw(value));
if (raw == 0 && !force)
return;
encode_fixed32(pos PROTO_ENCODE_DEBUG_ARG, field_id, raw);
} }
static inline void encode_int32(uint8_t *__restrict__ &pos PROTO_ENCODE_DEBUG_PARAM, uint32_t field_id, int32_t value, [[nodiscard]] static inline uint8_t *encode_float_force(uint8_t *__restrict__ pos PROTO_ENCODE_DEBUG_PARAM,
bool force = false) { uint32_t field_id, float value) {
return encode_fixed32_force(pos PROTO_ENCODE_DEBUG_ARG, field_id, float_to_raw(value));
}
[[nodiscard]] static inline uint8_t *encode_int32_force(uint8_t *__restrict__ pos PROTO_ENCODE_DEBUG_PARAM,
uint32_t field_id, int32_t value) {
if (value < 0) { if (value < 0) {
// negative int32 is always 10 byte long // negative int32 is always 10 byte long
encode_uint64(pos PROTO_ENCODE_DEBUG_ARG, field_id, static_cast<uint64_t>(value), force); return encode_uint64_force(pos PROTO_ENCODE_DEBUG_ARG, field_id, static_cast<uint64_t>(value));
return;
} }
encode_uint32(pos PROTO_ENCODE_DEBUG_ARG, field_id, static_cast<uint32_t>(value), force); return encode_uint32_force(pos PROTO_ENCODE_DEBUG_ARG, field_id, static_cast<uint32_t>(value));
} }
static inline void encode_int64(uint8_t *__restrict__ &pos PROTO_ENCODE_DEBUG_PARAM, uint32_t field_id, int64_t value, [[nodiscard]] static inline uint8_t *encode_int32(uint8_t *__restrict__ pos PROTO_ENCODE_DEBUG_PARAM,
bool force = false) { uint32_t field_id, int32_t value) {
encode_uint64(pos PROTO_ENCODE_DEBUG_ARG, field_id, static_cast<uint64_t>(value), force); if (value == 0)
return pos;
return encode_int32_force(pos PROTO_ENCODE_DEBUG_ARG, field_id, value);
} }
static inline void encode_sint32(uint8_t *__restrict__ &pos PROTO_ENCODE_DEBUG_PARAM, uint32_t field_id, [[nodiscard]] static inline uint8_t *encode_int64(uint8_t *__restrict__ pos PROTO_ENCODE_DEBUG_PARAM,
int32_t value, bool force = false) { uint32_t field_id, int64_t value) {
encode_uint32(pos PROTO_ENCODE_DEBUG_ARG, field_id, encode_zigzag32(value), force); return encode_uint64(pos PROTO_ENCODE_DEBUG_ARG, field_id, static_cast<uint64_t>(value));
} }
static inline void encode_sint64(uint8_t *__restrict__ &pos PROTO_ENCODE_DEBUG_PARAM, uint32_t field_id, [[nodiscard]] static inline uint8_t *encode_int64_force(uint8_t *__restrict__ pos PROTO_ENCODE_DEBUG_PARAM,
int64_t value, bool force = false) { uint32_t field_id, int64_t value) {
encode_uint64(pos PROTO_ENCODE_DEBUG_ARG, field_id, encode_zigzag64(value), force); return encode_uint64_force(pos PROTO_ENCODE_DEBUG_ARG, field_id, static_cast<uint64_t>(value));
} }
/// Sub-message encoding: sync pos to buffer, delegate, get pos from return value. [[nodiscard]] static inline uint8_t *encode_sint32(uint8_t *__restrict__ pos PROTO_ENCODE_DEBUG_PARAM,
uint32_t field_id, int32_t value) {
return encode_uint32(pos PROTO_ENCODE_DEBUG_ARG, field_id, encode_zigzag32(value));
}
[[nodiscard]] static inline uint8_t *encode_sint32_force(uint8_t *__restrict__ pos PROTO_ENCODE_DEBUG_PARAM,
uint32_t field_id, int32_t value) {
return encode_uint32_force(pos PROTO_ENCODE_DEBUG_ARG, field_id, encode_zigzag32(value));
}
[[nodiscard]] static inline uint8_t *encode_sint64(uint8_t *__restrict__ pos PROTO_ENCODE_DEBUG_PARAM,
uint32_t field_id, int64_t value) {
return encode_uint64(pos PROTO_ENCODE_DEBUG_ARG, field_id, encode_zigzag64(value));
}
[[nodiscard]] static inline uint8_t *encode_sint64_force(uint8_t *__restrict__ pos PROTO_ENCODE_DEBUG_PARAM,
uint32_t field_id, int64_t value) {
return encode_uint64_force(pos PROTO_ENCODE_DEBUG_ARG, field_id, encode_zigzag64(value));
}
/// Sub-message encoding: sync pos to buffer, delegate, read the cursor back.
template<typename T> template<typename T>
static inline void encode_sub_message(uint8_t *__restrict__ &pos PROTO_ENCODE_DEBUG_PARAM, ProtoWriteBuffer &buffer, [[nodiscard]] static inline uint8_t *encode_sub_message(uint8_t *__restrict__ pos PROTO_ENCODE_DEBUG_PARAM,
uint32_t field_id, const T &value) { ProtoWriteBuffer &buffer, uint32_t field_id, const T &value) {
buffer.set_pos(pos); buffer.set_pos(pos);
buffer.encode_sub_message(field_id, value); buffer.encode_sub_message(field_id, value);
pos = buffer.get_pos(); return buffer.get_pos();
} }
template<typename T> template<typename T>
static inline void encode_optional_sub_message(uint8_t *__restrict__ &pos PROTO_ENCODE_DEBUG_PARAM, [[nodiscard]] static inline uint8_t *encode_optional_sub_message(uint8_t *__restrict__ pos PROTO_ENCODE_DEBUG_PARAM,
ProtoWriteBuffer &buffer, uint32_t field_id, const T &value) { ProtoWriteBuffer &buffer, uint32_t field_id,
const T &value) {
buffer.set_pos(pos); buffer.set_pos(pos);
buffer.encode_optional_sub_message(field_id, value); buffer.encode_optional_sub_message(field_id, value);
pos = buffer.get_pos(); return buffer.get_pos();
}
private:
/// Unaligned little endian store of four bytes: byte stores where the outlined helper lives (ESP-IDF, ARM
/// without unaligned access), otherwise a memcpy the compiler folds into one store. Callers bounds check
/// and advance the cursor themselves.
static inline void ESPHOME_ALWAYS_INLINE write_fixed32_le(uint8_t *__restrict__ pos, uint32_t value) {
if constexpr (PROTO_FIXED32_BYTE_STORES) {
// Spelled out so the outlined helper does not itself become a memcpy call
pos[0] = static_cast<uint8_t>(value);
pos[1] = static_cast<uint8_t>(value >> 8);
pos[2] = static_cast<uint8_t>(value >> 16);
pos[3] = static_cast<uint8_t>(value >> 24);
} else {
const uint32_t le = convert_little_endian(value);
__builtin_memcpy(pos, &le, 4);
}
} }
}; };
#undef PROTO_OUTLINE_FOR_SIZE
#undef PROTO_FIXED32_BYTE_STORES
#ifdef HAS_PROTO_MESSAGE_DUMP #ifdef HAS_PROTO_MESSAGE_DUMP
/** /**
@@ -624,11 +695,12 @@ class DumpBuffer {
class ProtoMessage { class ProtoMessage {
public: public:
// Non-virtual defaults for messages with no fields. // Non-virtual defaults for messages with no fields; generated classes hide all four. The
// Concrete message classes hide these with their own implementations. // static encode_msg/calc_size_msg take const void * so &T::encode_msg needs no thunk.
// All call sites use templates to preserve the concrete type, so virtual static uint8_t *encode_msg(const void *self, ProtoWriteBuffer &buffer PROTO_ENCODE_DEBUG_PARAM) {
// dispatch is not needed. This eliminates per-message vtable entries for return buffer.get_pos();
// encode/calculate_size, saving ~1.3 KB of flash across all message types. }
static uint32_t calc_size_msg(const void *self) { return 0; }
uint8_t *encode(ProtoWriteBuffer &buffer PROTO_ENCODE_DEBUG_PARAM) const { return buffer.get_pos(); } uint8_t *encode(ProtoWriteBuffer &buffer PROTO_ENCODE_DEBUG_PARAM) const { return buffer.get_pos(); }
uint32_t calculate_size() const { return 0; } uint32_t calculate_size() const { return 0; }
#ifdef HAS_PROTO_MESSAGE_DUMP #ifdef HAS_PROTO_MESSAGE_DUMP
@@ -663,10 +735,10 @@ class ProtoDecodableMessage : public ProtoMessage {
protected: protected:
~ProtoDecodableMessage() = default; ~ProtoDecodableMessage() = default;
virtual bool decode_varint(uint32_t field_id, proto_varint_value_t value) { return false; } /// Store one decoded field; \p scalar is the varint or fixed32 value, or the length of the
virtual bool decode_length(uint32_t field_id, ProtoLengthDelimited value) { return false; } /// length-delimited payload at \p data. An unknown field or wrong wire type matches no case and is skipped.
virtual bool decode_32bit(uint32_t field_id, Proto32Bit value) { return false; } /// Three register arguments keep the decode loop free of spills.
// NOTE: decode_64bit removed - wire type 1 not supported virtual void decode_field(uint32_t tag, const uint8_t *data, proto_varint_value_t scalar) {}
}; };
class ProtoSize { class ProtoSize {
@@ -792,7 +864,7 @@ class ProtoSize {
* @return The number of bytes needed to encode the field ID and wire type * @return The number of bytes needed to encode the field ID and wire type
*/ */
static constexpr uint32_t field(uint32_t field_id, uint32_t type) { static constexpr uint32_t field(uint32_t field_id, uint32_t type) {
uint32_t tag = (field_id << 3) | (type & WIRE_TYPE_MASK); uint32_t tag = proto_tag(field_id, type & WIRE_TYPE_MASK);
return varint(tag); return varint(tag);
} }
@@ -876,24 +948,14 @@ class ProtoSize {
// Implementation of methods that depend on ProtoSize being fully defined // Implementation of methods that depend on ProtoSize being fully defined
// Encode thunk — converts void* back to concrete type for direct encode() call
template<typename T> uint8_t *proto_encode_msg(const void *msg, ProtoWriteBuffer &buf PROTO_ENCODE_DEBUG_PARAM) {
return static_cast<const T *>(msg)->encode(buf PROTO_ENCODE_DEBUG_ARG);
}
// Thin template wrapper; delegates to non-template core in proto.cpp. // Thin template wrapper; delegates to non-template core in proto.cpp.
template<typename T> inline void ProtoWriteBuffer::encode_sub_message(uint32_t field_id, const T &value) { template<typename T> inline void ProtoWriteBuffer::encode_sub_message(uint32_t field_id, const T &value) {
this->encode_sub_message(field_id, &value, &proto_encode_msg<T>); this->encode_sub_message(field_id, &value, &T::encode_msg);
} }
// Thin template wrapper; delegates to non-template core. // Thin template wrapper; delegates to non-template core.
template<typename T> inline void ProtoWriteBuffer::encode_optional_sub_message(uint32_t field_id, const T &value) { template<typename T> inline void ProtoWriteBuffer::encode_optional_sub_message(uint32_t field_id, const T &value) {
this->encode_optional_sub_message(field_id, value.calculate_size(), &value, &proto_encode_msg<T>); this->encode_optional_sub_message(field_id, T::calc_size_msg(&value), &value, &T::encode_msg);
}
// Template decode_to_message - preserves concrete type so decode() resolves statically
template<typename T> void ProtoLengthDelimited::decode_to_message(T &msg) const {
msg.decode(this->value_, this->length_);
} }
template<typename T> const char *proto_enum_to_string(T value); template<typename T> const char *proto_enum_to_string(T value);
+156 -208
View File
@@ -9,10 +9,6 @@ namespace esphome::atm90e32 {
static const char *const TAG = "atm90e32"; static const char *const TAG = "atm90e32";
static const LogString *offset_calibration_name(bool power_offsets) {
return power_offsets ? LOG_STR("Power offset") : LOG_STR("Offset");
}
static uint32_t pref_hash(const char *prefix, const char *name_space) { static uint32_t pref_hash(const char *prefix, const char *name_space) {
auto hash = fnv1_hash(prefix); auto hash = fnv1_hash(prefix);
return fnv1_hash_extend(hash, name_space); return fnv1_hash_extend(hash, name_space);
@@ -207,12 +203,13 @@ void ATM90E32Component::setup() {
// Initialize flash storage for power offset calibrations // Initialize flash storage for power offset calibrations
uint32_t po_hash = pref_hash("_power_offset_calibration_", cs); uint32_t po_hash = pref_hash("_power_offset_calibration_", cs);
this->power_offset_pref_ = global_preferences->make_preference<OffsetCalibration[3]>(po_hash, true); this->power_offset_pref_ = global_preferences->make_preference<PowerOffsetCalibration[3]>(po_hash, true);
bool migrated_power_offset = false; bool migrated_power_offset = false;
if (has_distinct_legacy_namespace) { if (has_distinct_legacy_namespace) {
uint32_t legacy_po_hash = pref_hash("_power_offset_calibration_", legacy_cs); uint32_t legacy_po_hash = pref_hash("_power_offset_calibration_", legacy_cs);
auto legacy_power_offset_pref = global_preferences->make_preference<OffsetCalibration[3]>(legacy_po_hash, true); auto legacy_power_offset_pref =
OffsetCalibration power_offset_data[3]{}; global_preferences->make_preference<PowerOffsetCalibration[3]>(legacy_po_hash, true);
PowerOffsetCalibration power_offset_data[3]{};
int migration_status = int migration_status =
migrate_legacy_pref_if_needed(this->power_offset_pref_, legacy_power_offset_pref, &power_offset_data); migrate_legacy_pref_if_needed(this->power_offset_pref_, legacy_power_offset_pref, &power_offset_data);
migrated_power_offset = migration_status > 0; migrated_power_offset = migration_status > 0;
@@ -227,20 +224,20 @@ void ATM90E32Component::setup() {
global_preferences->sync(); global_preferences->sync();
} }
this->restore_offset_calibrations_(OffsetCalibrationType::OFFSET_CALIBRATION_TYPE_VOLTAGE_CURRENT); this->restore_offset_calibrations_();
this->restore_offset_calibrations_(OffsetCalibrationType::OFFSET_CALIBRATION_TYPE_POWER); this->restore_power_offset_calibrations_();
} else { } else {
ESP_LOGI(TAG, "[CALIBRATION][%s] Power & Voltage/Current offset calibration is disabled. Using config file values.", ESP_LOGI(TAG, "[CALIBRATION][%s] Power & Voltage/Current offset calibration is disabled. Using config file values.",
cs); cs);
for (uint8_t phase = 0; phase < 3; ++phase) { for (uint8_t phase = 0; phase < 3; ++phase) {
this->write16_(this->voltage_offset_registers[phase], this->write16_(this->voltage_offset_registers[phase],
static_cast<uint16_t>(this->offset_phase_[phase].first_offset)); static_cast<uint16_t>(this->offset_phase_[phase].voltage_offset_));
this->write16_(this->current_offset_registers[phase], this->write16_(this->current_offset_registers[phase],
static_cast<uint16_t>(this->offset_phase_[phase].second_offset)); static_cast<uint16_t>(this->offset_phase_[phase].current_offset_));
this->write16_(this->power_offset_registers[phase], this->write16_(this->power_offset_registers[phase],
static_cast<uint16_t>(this->power_offset_phase_[phase].first_offset)); static_cast<uint16_t>(this->power_offset_phase_[phase].active_power_offset));
this->write16_(this->reactive_power_offset_registers[phase], this->write16_(this->reactive_power_offset_registers[phase],
static_cast<uint16_t>(this->power_offset_phase_[phase].second_offset)); static_cast<uint16_t>(this->power_offset_phase_[phase].reactive_power_offset));
} }
} }
@@ -320,8 +317,8 @@ void ATM90E32Component::log_calibration_status_() {
cs); cs);
for (uint8_t phase = 0; phase < 3; ++phase) { for (uint8_t phase = 0; phase < 3; ++phase) {
ESP_LOGW(TAG, "[CALIBRATION][%s] | %c | %6d | %6d | %6d | %6d |", cs, 'A' + phase, ESP_LOGW(TAG, "[CALIBRATION][%s] | %c | %6d | %6d | %6d | %6d |", cs, 'A' + phase,
this->config_offset_phase_[phase].first_offset, this->offset_phase_[phase].first_offset, this->config_offset_phase_[phase].voltage_offset_, this->offset_phase_[phase].voltage_offset_,
this->config_offset_phase_[phase].second_offset, this->offset_phase_[phase].second_offset); this->config_offset_phase_[phase].current_offset_, this->offset_phase_[phase].current_offset_);
} }
ESP_LOGW(TAG, ESP_LOGW(TAG,
"[CALIBRATION][%s] ===============================================================================", cs); "[CALIBRATION][%s] ===============================================================================", cs);
@@ -338,8 +335,10 @@ void ATM90E32Component::log_calibration_status_() {
cs); cs);
for (uint8_t phase = 0; phase < 3; ++phase) { for (uint8_t phase = 0; phase < 3; ++phase) {
ESP_LOGW(TAG, "[CALIBRATION][%s] | %c | %6d | %6d | %6d | %6d |", cs, 'A' + phase, ESP_LOGW(TAG, "[CALIBRATION][%s] | %c | %6d | %6d | %6d | %6d |", cs, 'A' + phase,
this->config_power_offset_phase_[phase].first_offset, this->power_offset_phase_[phase].first_offset, this->config_power_offset_phase_[phase].active_power_offset,
this->config_power_offset_phase_[phase].second_offset, this->power_offset_phase_[phase].second_offset); this->power_offset_phase_[phase].active_power_offset,
this->config_power_offset_phase_[phase].reactive_power_offset,
this->power_offset_phase_[phase].reactive_power_offset);
} }
ESP_LOGW(TAG, ESP_LOGW(TAG,
"[CALIBRATION][%s] ===============================================================================", cs); "[CALIBRATION][%s] ===============================================================================", cs);
@@ -373,7 +372,7 @@ void ATM90E32Component::log_calibration_status_() {
ESP_LOGI(TAG, "[CALIBRATION][%s] --------------------------------------------------------------", cs); ESP_LOGI(TAG, "[CALIBRATION][%s] --------------------------------------------------------------", cs);
for (uint8_t phase = 0; phase < 3; phase++) { for (uint8_t phase = 0; phase < 3; phase++) {
ESP_LOGI(TAG, "[CALIBRATION][%s] | %c | %6d | %6d |", cs, 'A' + phase, ESP_LOGI(TAG, "[CALIBRATION][%s] | %c | %6d | %6d |", cs, 'A' + phase,
this->offset_phase_[phase].first_offset, this->offset_phase_[phase].second_offset); this->offset_phase_[phase].voltage_offset_, this->offset_phase_[phase].current_offset_);
} }
ESP_LOGI(TAG, "[CALIBRATION][%s] ==============================================================\\n", cs); ESP_LOGI(TAG, "[CALIBRATION][%s] ==============================================================\\n", cs);
} }
@@ -386,7 +385,8 @@ void ATM90E32Component::log_calibration_status_() {
ESP_LOGI(TAG, "[CALIBRATION][%s] ---------------------------------------------------------------------", cs); ESP_LOGI(TAG, "[CALIBRATION][%s] ---------------------------------------------------------------------", cs);
for (uint8_t phase = 0; phase < 3; phase++) { for (uint8_t phase = 0; phase < 3; phase++) {
ESP_LOGI(TAG, "[CALIBRATION][%s] | %c | %6d | %6d |", cs, 'A' + phase, ESP_LOGI(TAG, "[CALIBRATION][%s] | %c | %6d | %6d |", cs, 'A' + phase,
this->power_offset_phase_[phase].first_offset, this->power_offset_phase_[phase].second_offset); this->power_offset_phase_[phase].active_power_offset,
this->power_offset_phase_[phase].reactive_power_offset);
} }
ESP_LOGI(TAG, "[CALIBRATION][%s] =====================================================================\n", cs); ESP_LOGI(TAG, "[CALIBRATION][%s] =====================================================================\n", cs);
} }
@@ -756,68 +756,36 @@ void ATM90E32Component::save_gain_calibration_to_memory_() {
} }
} }
void ATM90E32Component::finish_offset_calibration_(const OffsetCalibration (&previous)[3], bool previous_restored, void ATM90E32Component::save_offset_calibration_to_memory_() {
bool previous_using_saved, OffsetCalibrationType type) {
const bool power_offsets = type == OffsetCalibrationType::OFFSET_CALIBRATION_TYPE_POWER;
const char *cs = this->get_calibration_id_(); const char *cs = this->get_calibration_id_();
const LogString *name = offset_calibration_name(power_offsets); bool success = this->offset_pref_.save(&this->offset_phase_);
OffsetCalibration(*offsets)[3] = power_offsets ? &this->power_offset_phase_ : &this->offset_phase_; global_preferences->sync();
ESPPreferenceObject *preference = power_offsets ? &this->power_offset_pref_ : &this->offset_pref_; if (success) {
bool *has_stored =
power_offsets ? &this->has_stored_power_offset_calibration_ : &this->has_stored_offset_calibration_;
bool *restored = power_offsets ? &this->restored_power_offset_calibration_ : &this->restored_offset_calibration_;
bool *mismatches = power_offsets ? this->power_offset_calibration_mismatch_ : this->offset_calibration_mismatch_;
const bool writes_verified = this->verify_offset_writes_(type);
bool saved = false;
bool synced = false;
if (writes_verified) {
saved = preference->save(offsets);
synced = global_preferences->sync();
}
if (writes_verified && saved && synced) {
this->using_saved_calibrations_ = true; this->using_saved_calibrations_ = true;
*has_stored = true; this->restored_offset_calibration_ = true;
*restored = true; for (bool &phase : this->offset_calibration_mismatch_)
for (uint8_t phase = 0; phase < 3; phase++) phase = false;
mismatches[phase] = false; ESP_LOGI(TAG, "[CALIBRATION][%s] Offset calibration saved to memory.", cs);
ESP_LOGI(TAG, "[CALIBRATION][%s] %s calibration saved to memory. %s calibration completed and verified.", cs, } else {
LOG_STR_ARG(name), LOG_STR_ARG(name)); this->using_saved_calibrations_ = false;
return; ESP_LOGE(TAG, "[CALIBRATION][%s] Failed to save offset calibration to memory!", cs);
} }
}
if (writes_verified) { void ATM90E32Component::save_power_offset_calibration_to_memory_() {
ESP_LOGE(TAG, "[CALIBRATION][%s] Failed to save %s calibration to memory!", cs, LOG_STR_ARG(name)); const char *cs = this->get_calibration_id_();
bool success = this->power_offset_pref_.save(&this->power_offset_phase_);
global_preferences->sync();
if (success) {
this->using_saved_calibrations_ = true;
this->restored_power_offset_calibration_ = true;
for (bool &phase : this->power_offset_calibration_mismatch_)
phase = false;
ESP_LOGI(TAG, "[CALIBRATION][%s] Power offset calibration saved to memory.", cs);
} else {
this->using_saved_calibrations_ = false;
ESP_LOGE(TAG, "[CALIBRATION][%s] Failed to save power offset calibration to memory!", cs);
} }
for (uint8_t phase = 0; phase < 3; phase++) {
this->write_offsets_to_registers_(phase, previous[phase].first_offset, previous[phase].second_offset, type);
}
const bool rollback_verified = this->verify_offset_writes_(type);
bool rollback_persisted = false;
if (writes_verified) {
OffsetCalibration rollback[3]{};
prepare_offset_rollback(previous, previous_restored, rollback);
const bool rollback_saved = preference->save(&rollback);
const bool rollback_synced = global_preferences->sync();
rollback_persisted = rollback_saved && rollback_synced;
if (!rollback_saved || !rollback_synced) {
ESP_LOGE(TAG, "[CALIBRATION][%s] Failed to persist restored %s calibration values!", cs, LOG_STR_ARG(name));
}
}
*restored = previous_restored;
if (rollback_persisted)
*has_stored = previous_restored;
this->using_saved_calibrations_ = previous_using_saved;
if (!rollback_verified) {
ESP_LOGE(TAG, "[CALIBRATION][%s] %s calibration failed; rollback readback verification failed.", cs,
LOG_STR_ARG(name));
return;
}
ESP_LOGE(TAG, "[CALIBRATION][%s] %s calibration failed; previous values restored.", cs, LOG_STR_ARG(name));
} }
void ATM90E32Component::run_offset_calibrations() { void ATM90E32Component::run_offset_calibrations() {
@@ -835,16 +803,11 @@ void ATM90E32Component::run_offset_calibrations() {
ESP_LOGI(TAG, "[CALIBRATION][%s] | Phase | offset_voltage | offset_current |", cs); ESP_LOGI(TAG, "[CALIBRATION][%s] | Phase | offset_voltage | offset_current |", cs);
ESP_LOGI(TAG, "[CALIBRATION][%s] ------------------------------------------------------------------", cs); ESP_LOGI(TAG, "[CALIBRATION][%s] ------------------------------------------------------------------", cs);
OffsetCalibration previous_offsets[3] = {this->offset_phase_[0], this->offset_phase_[1], this->offset_phase_[2]};
const bool previous_restored = this->restored_offset_calibration_;
const bool previous_using_saved = this->using_saved_calibrations_;
for (uint8_t phase = 0; phase < 3; phase++) { for (uint8_t phase = 0; phase < 3; phase++) {
int16_t voltage_offset = calibrate_offset(phase, true); int16_t voltage_offset = calibrate_offset(phase, true);
int16_t current_offset = calibrate_offset(phase, false); int16_t current_offset = calibrate_offset(phase, false);
this->write_offsets_to_registers_(phase, voltage_offset, current_offset, this->write_offsets_to_registers_(phase, voltage_offset, current_offset);
OffsetCalibrationType::OFFSET_CALIBRATION_TYPE_VOLTAGE_CURRENT);
ESP_LOGI(TAG, "[CALIBRATION][%s] | %c | %6d | %6d |", cs, 'A' + phase, voltage_offset, ESP_LOGI(TAG, "[CALIBRATION][%s] | %c | %6d | %6d |", cs, 'A' + phase, voltage_offset,
current_offset); current_offset);
@@ -852,8 +815,7 @@ void ATM90E32Component::run_offset_calibrations() {
ESP_LOGI(TAG, "[CALIBRATION][%s] ==================================================================\n", cs); ESP_LOGI(TAG, "[CALIBRATION][%s] ==================================================================\n", cs);
this->finish_offset_calibration_(previous_offsets, previous_restored, previous_using_saved, this->save_offset_calibration_to_memory_();
OffsetCalibrationType::OFFSET_CALIBRATION_TYPE_VOLTAGE_CURRENT);
} }
void ATM90E32Component::run_power_offset_calibrations() { void ATM90E32Component::run_power_offset_calibrations() {
@@ -872,25 +834,18 @@ void ATM90E32Component::run_power_offset_calibrations() {
ESP_LOGI(TAG, "[CALIBRATION][%s] | Phase | offset_active_power | offset_reactive_power |", cs); ESP_LOGI(TAG, "[CALIBRATION][%s] | Phase | offset_active_power | offset_reactive_power |", cs);
ESP_LOGI(TAG, "[CALIBRATION][%s] ---------------------------------------------------------------------", cs); ESP_LOGI(TAG, "[CALIBRATION][%s] ---------------------------------------------------------------------", cs);
OffsetCalibration previous_offsets[3] = {this->power_offset_phase_[0], this->power_offset_phase_[1],
this->power_offset_phase_[2]};
const bool previous_restored = this->restored_power_offset_calibration_;
const bool previous_using_saved = this->using_saved_calibrations_;
for (uint8_t phase = 0; phase < 3; ++phase) { for (uint8_t phase = 0; phase < 3; ++phase) {
int16_t active_offset = calibrate_power_offset(phase, false); int16_t active_offset = calibrate_power_offset(phase, false);
int16_t reactive_offset = calibrate_power_offset(phase, true); int16_t reactive_offset = calibrate_power_offset(phase, true);
this->write_offsets_to_registers_(phase, active_offset, reactive_offset, this->write_power_offsets_to_registers_(phase, active_offset, reactive_offset);
OffsetCalibrationType::OFFSET_CALIBRATION_TYPE_POWER);
ESP_LOGI(TAG, "[CALIBRATION][%s] | %c | %6d | %6d |", cs, 'A' + phase, active_offset, ESP_LOGI(TAG, "[CALIBRATION][%s] | %c | %6d | %6d |", cs, 'A' + phase, active_offset,
reactive_offset); reactive_offset);
} }
ESP_LOGI(TAG, "[CALIBRATION][%s] =====================================================================\n", cs); ESP_LOGI(TAG, "[CALIBRATION][%s] =====================================================================\n", cs);
this->finish_offset_calibration_(previous_offsets, previous_restored, previous_using_saved, this->save_power_offset_calibration_to_memory_();
OffsetCalibrationType::OFFSET_CALIBRATION_TYPE_POWER);
} }
void ATM90E32Component::write_gains_to_registers_() { void ATM90E32Component::write_gains_to_registers_() {
@@ -904,26 +859,35 @@ void ATM90E32Component::write_gains_to_registers_() {
this->write16_(ATM90E32_REGISTER_CFGREGACCEN, 0x0000); this->write16_(ATM90E32_REGISTER_CFGREGACCEN, 0x0000);
} }
void ATM90E32Component::write_offsets_to_registers_(uint8_t phase, int16_t first_offset, int16_t second_offset, void ATM90E32Component::write_offsets_to_registers_(uint8_t phase, int16_t voltage_offset, int16_t current_offset) {
OffsetCalibrationType type) { // Save to runtime
const bool power_offsets = type == OffsetCalibrationType::OFFSET_CALIBRATION_TYPE_POWER; this->offset_phase_[phase].voltage_offset_ = voltage_offset;
OffsetCalibration &offsets = power_offsets ? this->power_offset_phase_[phase] : this->offset_phase_[phase]; this->phase_[phase].voltage_offset_ = voltage_offset;
offsets.first_offset = first_offset;
offsets.second_offset = second_offset;
if (power_offsets) {
this->phase_[phase].active_power_offset_ = first_offset;
this->phase_[phase].reactive_power_offset_ = second_offset;
} else {
this->phase_[phase].voltage_offset_ = first_offset;
this->phase_[phase].current_offset_ = second_offset;
}
const uint16_t *first_registers = power_offsets ? this->power_offset_registers : this->voltage_offset_registers; // Save to flash-storable struct
const uint16_t *second_registers = this->offset_phase_[phase].current_offset_ = current_offset;
power_offsets ? this->reactive_power_offset_registers : this->current_offset_registers; this->phase_[phase].current_offset_ = current_offset;
// Write to registers
this->write16_(ATM90E32_REGISTER_CFGREGACCEN, 0x55AA); this->write16_(ATM90E32_REGISTER_CFGREGACCEN, 0x55AA);
this->write16_(first_registers[phase], static_cast<uint16_t>(first_offset)); this->write16_(voltage_offset_registers[phase], static_cast<uint16_t>(voltage_offset));
this->write16_(second_registers[phase], static_cast<uint16_t>(second_offset)); this->write16_(current_offset_registers[phase], static_cast<uint16_t>(current_offset));
this->write16_(ATM90E32_REGISTER_CFGREGACCEN, 0x0000);
}
void ATM90E32Component::write_power_offsets_to_registers_(uint8_t phase, int16_t p_offset, int16_t q_offset) {
// Save to runtime
this->phase_[phase].active_power_offset_ = p_offset;
this->phase_[phase].reactive_power_offset_ = q_offset;
// Save to flash-storable struct
this->power_offset_phase_[phase].active_power_offset = p_offset;
this->power_offset_phase_[phase].reactive_power_offset = q_offset;
// Write to registers
this->write16_(ATM90E32_REGISTER_CFGREGACCEN, 0x55AA);
this->write16_(this->power_offset_registers[phase], static_cast<uint16_t>(p_offset));
this->write16_(this->reactive_power_offset_registers[phase], static_cast<uint16_t>(q_offset));
this->write16_(ATM90E32_REGISTER_CFGREGACCEN, 0x0000); this->write16_(ATM90E32_REGISTER_CFGREGACCEN, 0x0000);
} }
@@ -983,78 +947,89 @@ void ATM90E32Component::restore_gain_calibrations_() {
ESP_LOGW(TAG, "[CALIBRATION][%s] No stored gain calibrations found. Using config file values.", cs); ESP_LOGW(TAG, "[CALIBRATION][%s] No stored gain calibrations found. Using config file values.", cs);
} }
void ATM90E32Component::restore_offset_calibrations_(OffsetCalibrationType type) { void ATM90E32Component::restore_offset_calibrations_() {
const bool power_offsets = type == OffsetCalibrationType::OFFSET_CALIBRATION_TYPE_POWER;
const char *cs = this->get_calibration_id_(); const char *cs = this->get_calibration_id_();
const LogString *name = power_offsets ? LOG_STR("power offset") : LOG_STR("offset");
OffsetCalibration(*offsets)[3] = power_offsets ? &this->power_offset_phase_ : &this->offset_phase_;
OffsetCalibration(*config_offsets)[3] =
power_offsets ? &this->config_power_offset_phase_ : &this->config_offset_phase_;
ESPPreferenceObject *preference = power_offsets ? &this->power_offset_pref_ : &this->offset_pref_;
bool *has_stored =
power_offsets ? &this->has_stored_power_offset_calibration_ : &this->has_stored_offset_calibration_;
bool *restored = power_offsets ? &this->restored_power_offset_calibration_ : &this->restored_offset_calibration_;
bool *mismatches = power_offsets ? this->power_offset_calibration_mismatch_ : this->offset_calibration_mismatch_;
const bool *has_first = power_offsets ? this->has_config_active_power_offset_ : this->has_config_voltage_offset_;
const bool *has_second = power_offsets ? this->has_config_reactive_power_offset_ : this->has_config_current_offset_;
for (uint8_t i = 0; i < 3; ++i) for (uint8_t i = 0; i < 3; ++i)
(*config_offsets)[i] = (*offsets)[i]; this->config_offset_phase_[i] = this->offset_phase_[i];
bool have_data = this->offset_pref_.load(&this->offset_phase_);
const bool have_data = preference->load(offsets);
bool all_zero = true; bool all_zero = true;
if (have_data) { if (have_data) {
for (const auto &phase : *offsets) { for (auto &phase : this->offset_phase_) {
if (phase.first_offset != 0 || phase.second_offset != 0) { if (phase.voltage_offset_ != 0 || phase.current_offset_ != 0) {
all_zero = false; all_zero = false;
break; break;
} }
} }
} }
*has_stored = have_data && !all_zero; if (have_data && !all_zero) {
*restored = false; this->restored_offset_calibration_ = true;
for (uint8_t phase = 0; phase < 3; phase++) {
auto &offset = this->offset_phase_[phase];
bool mismatch = false;
if (this->has_config_voltage_offset_[phase] &&
offset.voltage_offset_ != this->config_offset_phase_[phase].voltage_offset_)
mismatch = true;
if (this->has_config_current_offset_[phase] &&
offset.current_offset_ != this->config_offset_phase_[phase].current_offset_)
mismatch = true;
if (mismatch)
this->offset_calibration_mismatch_[phase] = true;
}
} else {
for (uint8_t phase = 0; phase < 3; phase++)
this->offset_phase_[phase] = this->config_offset_phase_[phase];
ESP_LOGW(TAG, "[CALIBRATION][%s] No stored offset calibrations found. Using default values.", cs);
}
for (uint8_t phase = 0; phase < 3; phase++) { for (uint8_t phase = 0; phase < 3; phase++) {
mismatches[phase] = false; write_offsets_to_registers_(phase, this->offset_phase_[phase].voltage_offset_,
if (*has_stored) { this->offset_phase_[phase].current_offset_);
mismatches[phase] = }
(has_first[phase] && (*offsets)[phase].first_offset != (*config_offsets)[phase].first_offset) || }
(has_second[phase] && (*offsets)[phase].second_offset != (*config_offsets)[phase].second_offset);
void ATM90E32Component::restore_power_offset_calibrations_() {
const char *cs = this->get_calibration_id_();
for (uint8_t i = 0; i < 3; ++i)
this->config_power_offset_phase_[i] = this->power_offset_phase_[i];
bool have_data = this->power_offset_pref_.load(&this->power_offset_phase_);
bool all_zero = true;
if (have_data) {
for (auto &phase : this->power_offset_phase_) {
if (phase.active_power_offset != 0 || phase.reactive_power_offset != 0) {
all_zero = false;
break;
}
} }
} }
if (!*has_stored) { if (have_data && !all_zero) {
for (uint8_t phase = 0; phase < 3; phase++) this->restored_power_offset_calibration_ = true;
(*offsets)[phase] = (*config_offsets)[phase]; for (uint8_t phase = 0; phase < 3; ++phase) {
ESP_LOGW(TAG, "[CALIBRATION][%s] No stored %s calibrations found. Using default values.", cs, LOG_STR_ARG(name)); auto &offset = this->power_offset_phase_[phase];
} bool mismatch = false;
if (this->has_config_active_power_offset_[phase] &&
for (uint8_t phase = 0; phase < 3; phase++) { offset.active_power_offset != this->config_power_offset_phase_[phase].active_power_offset)
this->write_offsets_to_registers_(phase, (*offsets)[phase].first_offset, (*offsets)[phase].second_offset, type); mismatch = true;
} if (this->has_config_reactive_power_offset_[phase] &&
const bool initial_values_verified = this->verify_offset_writes_(type); offset.reactive_power_offset != this->config_power_offset_phase_[phase].reactive_power_offset)
if (initial_values_verified) { mismatch = true;
const auto state = resolve_offset_restore_state(*has_stored, true, false); if (mismatch)
*restored = state.restored; this->power_offset_calibration_mismatch_[phase] = true;
ESP_LOGI(TAG, "[CALIBRATION][%s] %s calibration values verified.", cs, LOG_STR_ARG(name)); }
return;
}
this->using_saved_calibrations_ = false;
for (uint8_t phase = 0; phase < 3; phase++)
mismatches[phase] = false;
for (uint8_t phase = 0; phase < 3; phase++) {
(*offsets)[phase] = (*config_offsets)[phase];
this->write_offsets_to_registers_(phase, (*offsets)[phase].first_offset, (*offsets)[phase].second_offset, type);
}
const auto state = resolve_offset_restore_state(*has_stored, false, this->verify_offset_writes_(type));
*restored = state.restored;
if (state.values_verified) {
ESP_LOGE(TAG, "[CALIBRATION][%s] %s calibration restore failed verification; config values verified.", cs,
LOG_STR_ARG(name));
} else { } else {
ESP_LOGE(TAG, "[CALIBRATION][%s] %s calibration restore and config fallback both failed verification.", cs, for (uint8_t phase = 0; phase < 3; ++phase)
LOG_STR_ARG(name)); this->power_offset_phase_[phase] = this->config_power_offset_phase_[phase];
ESP_LOGW(TAG, "[CALIBRATION][%s] No stored power offsets found. Using default values.", cs);
}
for (uint8_t phase = 0; phase < 3; ++phase) {
write_power_offsets_to_registers_(phase, this->power_offset_phase_[phase].active_power_offset,
this->power_offset_phase_[phase].reactive_power_offset);
} }
} }
@@ -1109,14 +1084,14 @@ void ATM90E32Component::clear_gain_calibrations() {
void ATM90E32Component::clear_offset_calibrations() { void ATM90E32Component::clear_offset_calibrations() {
const char *cs = this->get_calibration_id_(); const char *cs = this->get_calibration_id_();
if (!this->has_stored_offset_calibration_) { if (!this->restored_offset_calibration_) {
ESP_LOGI(TAG, "[CALIBRATION][%s] No stored offset calibrations to clear. Current values:", cs); ESP_LOGI(TAG, "[CALIBRATION][%s] No stored offset calibrations to clear. Current values:", cs);
ESP_LOGI(TAG, "[CALIBRATION][%s] --------------------------------------------------------------", cs); ESP_LOGI(TAG, "[CALIBRATION][%s] --------------------------------------------------------------", cs);
ESP_LOGI(TAG, "[CALIBRATION][%s] | Phase | offset_voltage | offset_current |", cs); ESP_LOGI(TAG, "[CALIBRATION][%s] | Phase | offset_voltage | offset_current |", cs);
ESP_LOGI(TAG, "[CALIBRATION][%s] --------------------------------------------------------------", cs); ESP_LOGI(TAG, "[CALIBRATION][%s] --------------------------------------------------------------", cs);
for (uint8_t phase = 0; phase < 3; phase++) { for (uint8_t phase = 0; phase < 3; phase++) {
ESP_LOGI(TAG, "[CALIBRATION][%s] | %c | %6d | %6d |", cs, 'A' + phase, ESP_LOGI(TAG, "[CALIBRATION][%s] | %c | %6d | %6d |", cs, 'A' + phase,
this->offset_phase_[phase].first_offset, this->offset_phase_[phase].second_offset); this->offset_phase_[phase].voltage_offset_, this->offset_phase_[phase].current_offset_);
} }
ESP_LOGI(TAG, "[CALIBRATION][%s] ==============================================================\n", cs); ESP_LOGI(TAG, "[CALIBRATION][%s] ==============================================================\n", cs);
return; return;
@@ -1129,11 +1104,10 @@ void ATM90E32Component::clear_offset_calibrations() {
for (uint8_t phase = 0; phase < 3; phase++) { for (uint8_t phase = 0; phase < 3; phase++) {
int16_t voltage_offset = int16_t voltage_offset =
this->has_config_voltage_offset_[phase] ? this->config_offset_phase_[phase].first_offset : 0; this->has_config_voltage_offset_[phase] ? this->config_offset_phase_[phase].voltage_offset_ : 0;
int16_t current_offset = int16_t current_offset =
this->has_config_current_offset_[phase] ? this->config_offset_phase_[phase].second_offset : 0; this->has_config_current_offset_[phase] ? this->config_offset_phase_[phase].current_offset_ : 0;
this->write_offsets_to_registers_(phase, voltage_offset, current_offset, this->write_offsets_to_registers_(phase, voltage_offset, current_offset);
OffsetCalibrationType::OFFSET_CALIBRATION_TYPE_VOLTAGE_CURRENT);
ESP_LOGI(TAG, "[CALIBRATION][%s] | %c | %6d | %6d |", cs, 'A' + phase, voltage_offset, ESP_LOGI(TAG, "[CALIBRATION][%s] | %c | %6d | %6d |", cs, 'A' + phase, voltage_offset,
current_offset); current_offset);
} }
@@ -1143,7 +1117,6 @@ void ATM90E32Component::clear_offset_calibrations() {
this->offset_pref_.save(&zero_offsets); // Clear stored values in flash this->offset_pref_.save(&zero_offsets); // Clear stored values in flash
global_preferences->sync(); global_preferences->sync();
this->has_stored_offset_calibration_ = false;
this->restored_offset_calibration_ = false; this->restored_offset_calibration_ = false;
for (bool &phase : this->offset_calibration_mismatch_) for (bool &phase : this->offset_calibration_mismatch_)
phase = false; phase = false;
@@ -1153,14 +1126,15 @@ void ATM90E32Component::clear_offset_calibrations() {
void ATM90E32Component::clear_power_offset_calibrations() { void ATM90E32Component::clear_power_offset_calibrations() {
const char *cs = this->get_calibration_id_(); const char *cs = this->get_calibration_id_();
if (!this->has_stored_power_offset_calibration_) { if (!this->restored_power_offset_calibration_) {
ESP_LOGI(TAG, "[CALIBRATION][%s] No stored power offsets to clear. Current values:", cs); ESP_LOGI(TAG, "[CALIBRATION][%s] No stored power offsets to clear. Current values:", cs);
ESP_LOGI(TAG, "[CALIBRATION][%s] ---------------------------------------------------------------------", cs); ESP_LOGI(TAG, "[CALIBRATION][%s] ---------------------------------------------------------------------", cs);
ESP_LOGI(TAG, "[CALIBRATION][%s] | Phase | offset_active_power | offset_reactive_power |", cs); ESP_LOGI(TAG, "[CALIBRATION][%s] | Phase | offset_active_power | offset_reactive_power |", cs);
ESP_LOGI(TAG, "[CALIBRATION][%s] ---------------------------------------------------------------------", cs); ESP_LOGI(TAG, "[CALIBRATION][%s] ---------------------------------------------------------------------", cs);
for (uint8_t phase = 0; phase < 3; phase++) { for (uint8_t phase = 0; phase < 3; phase++) {
ESP_LOGI(TAG, "[CALIBRATION][%s] | %c | %6d | %6d |", cs, 'A' + phase, ESP_LOGI(TAG, "[CALIBRATION][%s] | %c | %6d | %6d |", cs, 'A' + phase,
this->power_offset_phase_[phase].first_offset, this->power_offset_phase_[phase].second_offset); this->power_offset_phase_[phase].active_power_offset,
this->power_offset_phase_[phase].reactive_power_offset);
} }
ESP_LOGI(TAG, "[CALIBRATION][%s] =====================================================================\n", cs); ESP_LOGI(TAG, "[CALIBRATION][%s] =====================================================================\n", cs);
return; return;
@@ -1173,21 +1147,20 @@ void ATM90E32Component::clear_power_offset_calibrations() {
for (uint8_t phase = 0; phase < 3; phase++) { for (uint8_t phase = 0; phase < 3; phase++) {
int16_t active_offset = int16_t active_offset =
this->has_config_active_power_offset_[phase] ? this->config_power_offset_phase_[phase].first_offset : 0; this->has_config_active_power_offset_[phase] ? this->config_power_offset_phase_[phase].active_power_offset : 0;
int16_t reactive_offset = int16_t reactive_offset = this->has_config_reactive_power_offset_[phase]
this->has_config_reactive_power_offset_[phase] ? this->config_power_offset_phase_[phase].second_offset : 0; ? this->config_power_offset_phase_[phase].reactive_power_offset
this->write_offsets_to_registers_(phase, active_offset, reactive_offset, : 0;
OffsetCalibrationType::OFFSET_CALIBRATION_TYPE_POWER); this->write_power_offsets_to_registers_(phase, active_offset, reactive_offset);
ESP_LOGI(TAG, "[CALIBRATION][%s] | %c | %6d | %6d |", cs, 'A' + phase, active_offset, ESP_LOGI(TAG, "[CALIBRATION][%s] | %c | %6d | %6d |", cs, 'A' + phase, active_offset,
reactive_offset); reactive_offset);
} }
ESP_LOGI(TAG, "[CALIBRATION][%s] =====================================================================\n", cs); ESP_LOGI(TAG, "[CALIBRATION][%s] =====================================================================\n", cs);
OffsetCalibration zero_power_offsets[3]{{0, 0}, {0, 0}, {0, 0}}; PowerOffsetCalibration zero_power_offsets[3]{{0, 0}, {0, 0}, {0, 0}};
this->power_offset_pref_.save(&zero_power_offsets); this->power_offset_pref_.save(&zero_power_offsets);
global_preferences->sync(); global_preferences->sync();
this->has_stored_power_offset_calibration_ = false;
this->restored_power_offset_calibration_ = false; this->restored_power_offset_calibration_ = false;
for (bool &phase : this->power_offset_calibration_mismatch_) for (bool &phase : this->power_offset_calibration_mismatch_)
phase = false; phase = false;
@@ -1242,31 +1215,6 @@ bool ATM90E32Component::verify_gain_writes_() {
return success; // Return true if all writes were successful, false otherwise return success; // Return true if all writes were successful, false otherwise
} }
bool ATM90E32Component::verify_offset_writes_(OffsetCalibrationType type) {
const bool power_offsets = type == OffsetCalibrationType::OFFSET_CALIBRATION_TYPE_POWER;
const char *cs = this->get_calibration_id_();
const LogString *name = offset_calibration_name(power_offsets);
const LogString *first_name = power_offsets ? LOG_STR("active") : LOG_STR("voltage");
const LogString *second_name = power_offsets ? LOG_STR("reactive") : LOG_STR("current");
const OffsetCalibration *offsets = power_offsets ? this->power_offset_phase_ : this->offset_phase_;
const uint16_t *first_registers = power_offsets ? this->power_offset_registers : this->voltage_offset_registers;
const uint16_t *second_registers =
power_offsets ? this->reactive_power_offset_registers : this->current_offset_registers;
bool success = true;
for (uint8_t phase = 0; phase < 3; phase++) {
const uint16_t first = this->read16_(first_registers[phase]);
const uint16_t second = this->read16_(second_registers[phase]);
if (!offset_register_value_matches(first, offsets[phase].first_offset) ||
!offset_register_value_matches(second, offsets[phase].second_offset)) {
ESP_LOGE(TAG, "[CALIBRATION][%s] %s readback failed for Phase %s: %s %d/%d, %s %d/%d.", cs, LOG_STR_ARG(name),
phase_labels[phase], LOG_STR_ARG(first_name), static_cast<int16_t>(first), offsets[phase].first_offset,
LOG_STR_ARG(second_name), static_cast<int16_t>(second), offsets[phase].second_offset);
success = false;
}
}
return success;
}
#ifdef USE_TEXT_SENSOR #ifdef USE_TEXT_SENSOR
void ATM90E32Component::check_phase_status() { void ATM90E32Component::check_phase_status() {
uint16_t state0 = this->read16_(ATM90E32_REGISTER_EMMSTATE0); uint16_t state0 = this->read16_(ATM90E32_REGISTER_EMMSTATE0);
+22 -49
View File
@@ -13,40 +13,6 @@
namespace esphome::atm90e32 { namespace esphome::atm90e32 {
inline bool offset_register_value_matches(uint16_t actual, int16_t expected) {
return actual == static_cast<uint16_t>(expected);
}
struct OffsetCalibration {
int16_t first_offset{0};
int16_t second_offset{0};
};
static_assert(sizeof(OffsetCalibration[3]) == 12, "Offset calibration preference layout must remain compatible");
enum class OffsetCalibrationType : uint8_t {
OFFSET_CALIBRATION_TYPE_VOLTAGE_CURRENT,
OFFSET_CALIBRATION_TYPE_POWER,
};
struct OffsetRestoreState {
bool restored;
bool values_verified;
};
inline OffsetRestoreState resolve_offset_restore_state(bool has_stored_values, bool initial_values_verified,
bool fallback_values_verified) {
if (initial_values_verified)
return {has_stored_values, true};
return {false, fallback_values_verified};
}
inline void prepare_offset_rollback(const OffsetCalibration (&previous)[3], bool had_stored_values,
OffsetCalibration (&rollback)[3]) {
for (uint8_t phase = 0; phase < 3; phase++)
rollback[phase] = had_stored_values ? previous[phase] : OffsetCalibration{};
}
class ATM90E32Component final : public PollingComponent, class ATM90E32Component final : public PollingComponent,
public spi::SPIDevice<spi::BIT_ORDER_MSB_FIRST, spi::CLOCK_POLARITY_HIGH, public spi::SPIDevice<spi::BIT_ORDER_MSB_FIRST, spi::CLOCK_POLARITY_HIGH,
spi::CLOCK_PHASE_TRAILING, spi::DATA_RATE_1MHZ> { spi::CLOCK_PHASE_TRAILING, spi::DATA_RATE_1MHZ> {
@@ -105,19 +71,19 @@ class ATM90E32Component final : public PollingComponent,
this->has_config_current_gain_[phase] = true; this->has_config_current_gain_[phase] = true;
} }
void set_voltage_offset(uint8_t phase, int16_t offset) { void set_voltage_offset(uint8_t phase, int16_t offset) {
this->offset_phase_[phase].first_offset = offset; this->offset_phase_[phase].voltage_offset_ = offset;
this->has_config_voltage_offset_[phase] = true; this->has_config_voltage_offset_[phase] = true;
} }
void set_current_offset(uint8_t phase, int16_t offset) { void set_current_offset(uint8_t phase, int16_t offset) {
this->offset_phase_[phase].second_offset = offset; this->offset_phase_[phase].current_offset_ = offset;
this->has_config_current_offset_[phase] = true; this->has_config_current_offset_[phase] = true;
} }
void set_active_power_offset(uint8_t phase, int16_t offset) { void set_active_power_offset(uint8_t phase, int16_t offset) {
this->power_offset_phase_[phase].first_offset = offset; this->power_offset_phase_[phase].active_power_offset = offset;
this->has_config_active_power_offset_[phase] = true; this->has_config_active_power_offset_[phase] = true;
} }
void set_reactive_power_offset(uint8_t phase, int16_t offset) { void set_reactive_power_offset(uint8_t phase, int16_t offset) {
this->power_offset_phase_[phase].second_offset = offset; this->power_offset_phase_[phase].reactive_power_offset = offset;
this->has_config_reactive_power_offset_[phase] = true; this->has_config_reactive_power_offset_[phase] = true;
} }
void set_freq_sensor(sensor::Sensor *freq_sensor) { freq_sensor_ = freq_sensor; } void set_freq_sensor(sensor::Sensor *freq_sensor) { freq_sensor_ = freq_sensor; }
@@ -205,16 +171,16 @@ class ATM90E32Component final : public PollingComponent,
float get_chip_temperature_(); float get_chip_temperature_();
bool get_publish_interval_flag_() { return publish_interval_flag_; }; bool get_publish_interval_flag_() { return publish_interval_flag_; };
void set_publish_interval_flag_(bool flag) { publish_interval_flag_ = flag; }; void set_publish_interval_flag_(bool flag) { publish_interval_flag_ = flag; };
void restore_offset_calibrations_(OffsetCalibrationType type); void restore_offset_calibrations_();
void restore_power_offset_calibrations_();
void restore_gain_calibrations_(); void restore_gain_calibrations_();
void save_offset_calibration_to_memory_();
void save_gain_calibration_to_memory_(); void save_gain_calibration_to_memory_();
void finish_offset_calibration_(const OffsetCalibration (&previous)[3], bool previous_restored, void save_power_offset_calibration_to_memory_();
bool previous_using_saved, OffsetCalibrationType type); void write_offsets_to_registers_(uint8_t phase, int16_t voltage_offset, int16_t current_offset);
void write_offsets_to_registers_(uint8_t phase, int16_t first_offset, int16_t second_offset, void write_power_offsets_to_registers_(uint8_t phase, int16_t p_offset, int16_t q_offset);
OffsetCalibrationType type);
void write_gains_to_registers_(); void write_gains_to_registers_();
bool verify_gain_writes_(); bool verify_gain_writes_();
bool verify_offset_writes_(OffsetCalibrationType type);
bool validate_spi_read_(uint16_t expected, const char *context = nullptr); bool validate_spi_read_(uint16_t expected, const char *context = nullptr);
void log_calibration_status_(); void log_calibration_status_();
const char *get_calibration_id_(); const char *get_calibration_id_();
@@ -253,10 +219,19 @@ class ATM90E32Component final : public PollingComponent,
uint32_t cumulative_reverse_active_energy_{0}; uint32_t cumulative_reverse_active_energy_{0};
} phase_[3]; } phase_[3];
OffsetCalibration offset_phase_[3]; struct OffsetCalibration {
int16_t voltage_offset_{0};
int16_t current_offset_{0};
} offset_phase_[3];
OffsetCalibration config_offset_phase_[3]; OffsetCalibration config_offset_phase_[3];
OffsetCalibration power_offset_phase_[3];
OffsetCalibration config_power_offset_phase_[3]; struct PowerOffsetCalibration {
int16_t active_power_offset{0};
int16_t reactive_power_offset{0};
} power_offset_phase_[3];
PowerOffsetCalibration config_power_offset_phase_[3];
struct GainCalibration { struct GainCalibration {
uint16_t voltage_gain{1}; uint16_t voltage_gain{1};
@@ -290,8 +265,6 @@ class ATM90E32Component final : public PollingComponent,
bool enable_offset_calibration_{false}; bool enable_offset_calibration_{false};
bool enable_gain_calibration_{false}; bool enable_gain_calibration_{false};
const char *instance_id_{nullptr}; const char *instance_id_{nullptr};
bool has_stored_offset_calibration_{false};
bool has_stored_power_offset_calibration_{false};
bool restored_offset_calibration_{false}; bool restored_offset_calibration_{false};
bool restored_power_offset_calibration_{false}; bool restored_power_offset_calibration_{false};
bool restored_gain_calibration_{false}; bool restored_gain_calibration_{false};
+3 -4
View File
@@ -313,10 +313,9 @@ FileDecoderState AudioDecoder::decode_mp3_() {
this->output_transfer_buffer_->increase_buffer_length( this->output_transfer_buffer_->increase_buffer_length(
this->audio_stream_info_.value().frames_to_bytes(samples_decoded)); this->audio_stream_info_.value().frames_to_bytes(samples_decoded));
} }
} else if (result == micro_mp3::MP3_STREAM_INFO_READY || result == micro_mp3::MP3_STREAM_INFO_CHANGED) { } else if (result == micro_mp3::MP3_STREAM_INFO_READY) {
// Header parsed: capture stream info and resize the output buffer to fit one full frame. // First successful header parse: capture stream info and resize the output buffer to fit one full frame.
// microMP3 always outputs 16-bit PCM. MP3_STREAM_INFO_CHANGED is handled identically: despite its // microMP3 always outputs 16-bit PCM.
// negative value it is documented as recoverable, so it must not reach the catch-all below.
this->audio_stream_info_ = this->audio_stream_info_ =
audio::AudioStreamInfo(16, this->mp3_decoder_->get_channels(), this->mp3_decoder_->get_sample_rate()); audio::AudioStreamInfo(16, this->mp3_decoder_->get_channels(), this->mp3_decoder_->get_sample_rate());
this->free_buffer_required_ = this->free_buffer_required_ =
+10 -24
View File
@@ -22,23 +22,6 @@ class Automation {
static const char *const TAG; static const char *const TAG;
}; };
// Base for nodes that never read the parent's services.
// The parent releases its services only once every node reports Established, so a node that never
// reports it keeps that memory allocated for the life of the connection.
class BLEClientServicelessNode : public BLEClientNode {
public:
// Final so that Established is always reported on SEARCH_CMPL, before the derived node sees the event.
void gattc_event_handler(esp_gattc_cb_event_t event, esp_gatt_if_t gattc_if, esp_ble_gattc_cb_param_t *param) final {
if (event == ESP_GATTC_SEARCH_CMPL_EVT)
this->node_state = espbt::ClientState::ESTABLISHED;
this->on_gattc_event(event, gattc_if, param);
}
protected:
// Derived nodes handle GATT events here rather than by overriding the handler above.
virtual void on_gattc_event(esp_gattc_cb_event_t event, esp_gatt_if_t gattc_if, esp_ble_gattc_cb_param_t *param) {}
};
// implement on_connect automation. // implement on_connect automation.
class BLEClientConnectTrigger final : public Trigger<>, public BLEClientNode { class BLEClientConnectTrigger final : public Trigger<>, public BLEClientNode {
public: public:
@@ -78,7 +61,7 @@ class BLEClientDisconnectTrigger final : public Trigger<>, public BLEClientNode
} }
}; };
class BLEClientPasskeyRequestTrigger final : public Trigger<>, public BLEClientServicelessNode { class BLEClientPasskeyRequestTrigger final : public Trigger<>, public BLEClientNode {
public: public:
explicit BLEClientPasskeyRequestTrigger(BLEClient *parent) { parent->register_ble_node(this); } explicit BLEClientPasskeyRequestTrigger(BLEClient *parent) { parent->register_ble_node(this); }
void loop() override {} void loop() override {}
@@ -88,7 +71,7 @@ class BLEClientPasskeyRequestTrigger final : public Trigger<>, public BLEClientS
} }
}; };
class BLEClientPasskeyNotificationTrigger final : public Trigger<uint32_t>, public BLEClientServicelessNode { class BLEClientPasskeyNotificationTrigger final : public Trigger<uint32_t>, public BLEClientNode {
public: public:
explicit BLEClientPasskeyNotificationTrigger(BLEClient *parent) { parent->register_ble_node(this); } explicit BLEClientPasskeyNotificationTrigger(BLEClient *parent) { parent->register_ble_node(this); }
void loop() override {} void loop() override {}
@@ -99,7 +82,7 @@ class BLEClientPasskeyNotificationTrigger final : public Trigger<uint32_t>, publ
} }
}; };
class BLEClientNumericComparisonRequestTrigger final : public Trigger<uint32_t>, public BLEClientServicelessNode { class BLEClientNumericComparisonRequestTrigger final : public Trigger<uint32_t>, public BLEClientNode {
public: public:
explicit BLEClientNumericComparisonRequestTrigger(BLEClient *parent) { parent->register_ble_node(this); } explicit BLEClientNumericComparisonRequestTrigger(BLEClient *parent) { parent->register_ble_node(this); }
void loop() override {} void loop() override {}
@@ -332,17 +315,19 @@ template<typename... Ts> class BLEClientRemoveBondAction final : public Action<T
BLEClient *parent_{nullptr}; BLEClient *parent_{nullptr};
}; };
template<typename... Ts> class BLEClientConnectAction final : public Action<Ts...>, public BLEClientServicelessNode { template<typename... Ts> class BLEClientConnectAction final : public Action<Ts...>, public BLEClientNode {
public: public:
BLEClientConnectAction(BLEClient *ble_client) { BLEClientConnectAction(BLEClient *ble_client) {
ble_client->register_ble_node(this); ble_client->register_ble_node(this);
ble_client_ = ble_client; ble_client_ = ble_client;
} }
void on_gattc_event(esp_gattc_cb_event_t event, esp_gatt_if_t gattc_if, esp_ble_gattc_cb_param_t *param) override { void gattc_event_handler(esp_gattc_cb_event_t event, esp_gatt_if_t gattc_if,
esp_ble_gattc_cb_param_t *param) override {
if (this->num_running_ == 0) if (this->num_running_ == 0)
return; return;
switch (event) { switch (event) {
case ESP_GATTC_SEARCH_CMPL_EVT: case ESP_GATTC_SEARCH_CMPL_EVT:
this->node_state = espbt::ClientState::ESTABLISHED;
this->parent()->run_later([this]() { this->play_next_tuple_(this->var_); }); this->parent()->run_later([this]() { this->play_next_tuple_(this->var_); });
break; break;
// if the connection is closed, terminate the automation chain. // if the connection is closed, terminate the automation chain.
@@ -379,13 +364,14 @@ template<typename... Ts> class BLEClientConnectAction final : public Action<Ts..
std::tuple<Ts...> var_{}; std::tuple<Ts...> var_{};
}; };
template<typename... Ts> class BLEClientDisconnectAction final : public Action<Ts...>, public BLEClientServicelessNode { template<typename... Ts> class BLEClientDisconnectAction final : public Action<Ts...>, public BLEClientNode {
public: public:
BLEClientDisconnectAction(BLEClient *ble_client) { BLEClientDisconnectAction(BLEClient *ble_client) {
ble_client->register_ble_node(this); ble_client->register_ble_node(this);
ble_client_ = ble_client; ble_client_ = ble_client;
} }
void on_gattc_event(esp_gattc_cb_event_t event, esp_gatt_if_t gattc_if, esp_ble_gattc_cb_param_t *param) override { void gattc_event_handler(esp_gattc_cb_event_t event, esp_gatt_if_t gattc_if,
esp_ble_gattc_cb_param_t *param) override {
if (this->num_running_ == 0) if (this->num_running_ == 0)
return; return;
switch (event) { switch (event) {
@@ -6,7 +6,6 @@ namespace esphome::dallas_temp {
static const char *const TAG = "dallas.temp.sensor"; static const char *const TAG = "dallas.temp.sensor";
static const uint8_t DALLAS_MODEL_DS18S20 = 0x10; static const uint8_t DALLAS_MODEL_DS18S20 = 0x10;
static const uint8_t DALLAS_MODEL_DS18B20 = 0x28;
static const uint8_t DALLAS_COMMAND_START_CONVERSION = 0x44; static const uint8_t DALLAS_COMMAND_START_CONVERSION = 0x44;
static const uint8_t DALLAS_COMMAND_READ_SCRATCH_PAD = 0xBE; static const uint8_t DALLAS_COMMAND_READ_SCRATCH_PAD = 0xBE;
static const uint8_t DALLAS_COMMAND_WRITE_SCRATCH_PAD = 0x4E; static const uint8_t DALLAS_COMMAND_WRITE_SCRATCH_PAD = 0x4E;
@@ -155,14 +154,7 @@ float DallasTemperatureSensor::get_temp_c_() {
default: default:
break; break;
} }
// undocumented test for powerup measurement of 85
// https://github.com/cpetrich/counterfeit_DS18B20#solution-to-the-85-c-problem
if ((this->address_ & 0xff) == DALLAS_MODEL_DS18B20) {
if ((temp == 85 * 16) && (this->scratch_pad_[6] == 0xc)) {
ESP_LOGD(TAG, "dropping reading caused by sensor reset");
return NAN;
}
}
return temp / 16.0f; return temp / 16.0f;
} }
+2 -6
View File
@@ -66,15 +66,11 @@ const char *DebugComponent::get_reset_reason_(std::span<char, RESET_REASON_BUFFE
unsigned reason = esp_reset_reason(); unsigned reason = esp_reset_reason();
if (reason < sizeof(RESET_REASONS) / sizeof(RESET_REASONS[0])) { if (reason < sizeof(RESET_REASONS) / sizeof(RESET_REASONS[0])) {
if (reason == ESP_RST_SW || reason == ESP_RST_WDT) { if (reason == ESP_RST_SW) {
// On some ESP32-S3 configurations (e.g. SPIRAM with fetch-instructions/rodata),
// esp_restart() intermittently produces RTCWDT_RTC_RST (ESP_RST_WDT) instead of
// ESP_RST_SW. Check the stored reboot source for both reset reasons so a software
// reboot that ends up as WDT still reports the correct source.
auto pref = global_preferences->make_preference(REBOOT_MAX_LEN, auto pref = global_preferences->make_preference(REBOOT_MAX_LEN,
fnv1_hash_extend(fnv1_hash(REBOOT_KEY), App.get_name().c_str())); fnv1_hash_extend(fnv1_hash(REBOOT_KEY), App.get_name().c_str()));
char reboot_source[REBOOT_MAX_LEN]{}; char reboot_source[REBOOT_MAX_LEN]{};
if (pref.load(&reboot_source) && reboot_source[0] != '\0') { if (pref.load(&reboot_source)) {
reboot_source[REBOOT_MAX_LEN - 1] = '\0'; reboot_source[REBOOT_MAX_LEN - 1] = '\0';
snprintf(buf, size, "Reboot request from %s", reboot_source); snprintf(buf, size, "Reboot request from %s", reboot_source);
} else { } else {
+5 -1
View File
@@ -23,7 +23,11 @@ from esphome.const import (
) )
from esphome.types import ConfigType from esphome.types import ConfigType
from . import CONF_DEBUG_ID, FILTER_SOURCE_FILES, DebugComponent # noqa: F401 pylint: disable=unused-import from . import ( # noqa: F401 pylint: disable=unused-import
CONF_DEBUG_ID,
FILTER_SOURCE_FILES,
DebugComponent,
)
DEPENDENCIES = ["debug"] DEPENDENCIES = ["debug"]
+5 -1
View File
@@ -9,7 +9,11 @@ from esphome.const import (
) )
from esphome.types import ConfigType from esphome.types import ConfigType
from . import CONF_DEBUG_ID, FILTER_SOURCE_FILES, DebugComponent # noqa: F401 pylint: disable=unused-import from . import ( # noqa: F401 pylint: disable=unused-import
CONF_DEBUG_ID,
FILTER_SOURCE_FILES,
DebugComponent,
)
DEPENDENCIES = ["debug"] DEPENDENCIES = ["debug"]
+9 -2
View File
@@ -3,11 +3,18 @@ import esphome.codegen as cg
# Re-exported for the many esp32-side users; defined in esphome.const # Re-exported for the many esp32-side users; defined in esphome.const
# and esphome.espidf so the upload/logs fast path can use them without # and esphome.espidf so the upload/logs fast path can use them without
# importing this package. # importing this package.
from esphome.const import KEY_ESP32, KEY_FLASH_SIZE, KEY_IDF_VERSION, KEY_VARIANT # noqa: F401 # pylint: disable=unused-import from esphome.const import ( # noqa: F401 # pylint: disable=unused-import
KEY_ESP32,
KEY_FLASH_SIZE,
KEY_IDF_VERSION,
KEY_VARIANT,
)
# Back compat for external components only; in-tree callers import it # Back compat for external components only; in-tree callers import it
# from esphome.espidf directly. # from esphome.espidf directly.
from esphome.espidf import variant_to_idf_target # noqa: F401 # pylint: disable=unused-import from esphome.espidf import ( # noqa: F401 # pylint: disable=unused-import
variant_to_idf_target,
)
KEY_BOARD = "board" KEY_BOARD = "board"
KEY_SDKCONFIG_OPTIONS = "sdkconfig_options" KEY_SDKCONFIG_OPTIONS = "sdkconfig_options"
@@ -91,14 +91,7 @@ void I2SAudioSpeakerBase::loop() {
this->speaker_task_handle_ = nullptr; this->speaker_task_handle_ = nullptr;
this->stop_i2s_driver_(); this->stop_i2s_driver_();
// ALL_BITS includes COMMAND_START. Take the bits from the clear itself, not from the snapshot at xEventGroupClearBits(this->event_group_, SpeakerEventGroupBits::ALL_BITS);
// the top of loop(): the audio source's task can raise a start at any point above, including
// during stop_i2s_driver_(), and nothing would ever re-issue it.
const EventBits_t bits_before_clear = xEventGroupClearBits(this->event_group_, SpeakerEventGroupBits::ALL_BITS);
if (bits_before_clear & SpeakerEventGroupBits::COMMAND_START) {
ESP_LOGD(TAG, "Start requested while stopping; keeping the request");
xEventGroupSetBits(this->event_group_, SpeakerEventGroupBits::COMMAND_START);
}
this->status_clear_error(); this->status_clear_error();
this->on_task_stopped(); this->on_task_stopped();
@@ -5,7 +5,6 @@
#include <esp_log.h> #include <esp_log.h>
#include <driver/uart.h> #include <driver/uart.h>
#include <soc/soc_caps.h>
#ifdef USE_LOGGER_UART_SELECTION_USB_SERIAL_JTAG #ifdef USE_LOGGER_UART_SELECTION_USB_SERIAL_JTAG
#include <driver/usb_serial_jtag.h> #include <driver/usb_serial_jtag.h>
@@ -77,11 +76,7 @@ void init_uart(uart_port_t uart_num, uint32_t baud_rate, int tx_buffer_size) {
uart_config.parity = UART_PARITY_DISABLE; uart_config.parity = UART_PARITY_DISABLE;
uart_config.stop_bits = UART_STOP_BITS_1; uart_config.stop_bits = UART_STOP_BITS_1;
uart_config.flow_ctrl = UART_HW_FLOWCTRL_DISABLE; uart_config.flow_ctrl = UART_HW_FLOWCTRL_DISABLE;
#if SOC_UART_SUPPORT_XTAL_CLK
uart_config.source_clk = UART_SCLK_XTAL;
#else
uart_config.source_clk = UART_SCLK_DEFAULT; uart_config.source_clk = UART_SCLK_DEFAULT;
#endif
uart_param_config(uart_num, &uart_config); uart_param_config(uart_num, &uart_config);
// The logger only writes to UART, never reads, so use the minimum RX buffer. // The logger only writes to UART, never reads, so use the minimum RX buffer.
// ESP-IDF requires rx_buffer_size > UART_HW_FIFO_LEN (128 bytes). // ESP-IDF requires rx_buffer_size > UART_HW_FIFO_LEN (128 bytes).
+1 -2
View File
@@ -15,7 +15,6 @@ from ..defines import (
from ..types import LvCompound, LvType from ..types import LvCompound, LvType
from . import Widget, WidgetType, get_widgets from . import Widget, WidgetType, get_widgets
from .buttonmatrix import CONF_BUTTONMATRIX from .buttonmatrix import CONF_BUTTONMATRIX
from .label import CONF_LABEL
from .textarea import CONF_TEXTAREA, lv_textarea_t from .textarea import CONF_TEXTAREA, lv_textarea_t
CONF_KEYBOARD = "keyboard" CONF_KEYBOARD = "keyboard"
@@ -50,7 +49,7 @@ class KeyboardType(WidgetType):
) )
def get_uses(self): def get_uses(self):
return CONF_KEYBOARD, CONF_TEXTAREA, CONF_BUTTONMATRIX, CONF_LABEL return CONF_KEYBOARD, CONF_TEXTAREA, CONF_BUTTONMATRIX
async def to_code(self, w: Widget, config: dict): async def to_code(self, w: Widget, config: dict):
add_lv_use("KEY_LISTENER") add_lv_use("KEY_LISTENER")
+1 -2
View File
@@ -10,7 +10,6 @@ from ..types import lv_obj_t
from . import Widget, WidgetType from . import Widget, WidgetType
from .canvas import CONF_CANVAS from .canvas import CONF_CANVAS
from .img import CONF_IMAGE from .img import CONF_IMAGE
from .label import CONF_LABEL
CONF_QRCODE = "qrcode" CONF_QRCODE = "qrcode"
CONF_DARK_COLOR = "dark_color" CONF_DARK_COLOR = "dark_color"
@@ -42,7 +41,7 @@ class QrCodeType(WidgetType):
) )
def get_uses(self): def get_uses(self):
return CONF_CANVAS, CONF_IMAGE, CONF_LABEL return CONF_CANVAS, CONF_IMAGE
async def to_code(self, w: Widget, config): async def to_code(self, w: Widget, config):
await w.set_property( await w.set_property(
+1 -2
View File
@@ -28,7 +28,6 @@ from ..types import LV_EVENT, LvType, ObjUpdateAction, lv_obj_t, lv_obj_t_ptr
from . import Widget, WidgetType, add_widgets, get_widgets, set_obj_properties from . import Widget, WidgetType, add_widgets, get_widgets, set_obj_properties
from .button import button_spec from .button import button_spec
from .buttonmatrix import CONF_BUTTONMATRIX, buttonmatrix_spec from .buttonmatrix import CONF_BUTTONMATRIX, buttonmatrix_spec
from .label import CONF_LABEL
from .obj import obj_spec from .obj import obj_spec
CONF_TABVIEW = "tabview" CONF_TABVIEW = "tabview"
@@ -75,7 +74,7 @@ class TabviewType(WidgetType):
) )
def get_uses(self): def get_uses(self):
return CONF_BUTTONMATRIX, TYPE_FLEX, CONF_BUTTON, CONF_LABEL return CONF_BUTTONMATRIX, TYPE_FLEX, CONF_BUTTON
async def to_code(self, w: Widget, config: dict): async def to_code(self, w: Widget, config: dict):
await w.set_property( await w.set_property(
-3
View File
@@ -67,9 +67,6 @@ void MQTTJSONLightComponent::send_discovery(JsonObject root, mqtt::SendDiscovery
if (traits.supports_color_mode(ColorMode::RGB_COLD_WARM_WHITE)) if (traits.supports_color_mode(ColorMode::RGB_COLD_WARM_WHITE))
color_modes.add(ESPHOME_F("rgbww")); color_modes.add(ESPHOME_F("rgbww"));
if (traits.supports_color_capability(ColorCapability::BRIGHTNESS))
root[ESPHOME_F("brightness")] = true;
if (traits.supports_color_mode(ColorMode::COLOR_TEMPERATURE) || if (traits.supports_color_mode(ColorMode::COLOR_TEMPERATURE) ||
traits.supports_color_mode(ColorMode::COLD_WARM_WHITE)) { traits.supports_color_mode(ColorMode::COLD_WARM_WHITE)) {
root[MQTT_MIN_MIREDS] = traits.get_min_mireds(); root[MQTT_MIN_MIREDS] = traits.get_min_mireds();
+6 -1
View File
@@ -14,7 +14,12 @@ from esphome.const import (
) )
from esphome.core import CORE, TimePeriod from esphome.core import CORE, TimePeriod
from . import FILTER_SOURCE_FILES, Nextion, nextion_ns, nextion_ref # noqa: F401 pylint: disable=unused-import from . import ( # noqa: F401 pylint: disable=unused-import
FILTER_SOURCE_FILES,
Nextion,
nextion_ns,
nextion_ref,
)
from .base_component import ( from .base_component import (
CONF_AUTO_WAKE_ON_TOUCH, CONF_AUTO_WAKE_ON_TOUCH,
CONF_COMMAND_SPACING, CONF_COMMAND_SPACING,
+21 -88
View File
@@ -18,16 +18,6 @@ void RFBridgeComponent::ack_() {
} }
bool RFBridgeComponent::parse_bridge_byte_(uint8_t byte) { bool RFBridgeComponent::parse_bridge_byte_(uint8_t byte) {
if (this->bucket_frame_candidate_ && byte == RF_CODE_START) {
// A queued next frame proves the trailing 0x55 really was the bucket
// frame's terminator: Portisch builds pulse entries from alternating
// signal edges, so the two level bits inside one pulse byte are always
// opposite — 0xAA (two high-level nibbles) cannot occur in pulse data.
// Finalize before this byte starts the new frame, so back-to-back
// deliveries are split even when loop() never observed a quiet gap
// between them.
this->finish_bucket_frame_();
}
size_t at = this->rx_buffer_.size(); size_t at = this->rx_buffer_.size();
this->rx_buffer_.push_back(byte); this->rx_buffer_.push_back(byte);
const uint8_t *raw = &this->rx_buffer_[0]; const uint8_t *raw = &this->rx_buffer_[0];
@@ -94,21 +84,26 @@ bool RFBridgeComponent::parse_bridge_byte_(uint8_t byte) {
break; break;
} }
case RF_CODE_RFIN_BUCKET: { case RF_CODE_RFIN_BUCKET: {
if (at == 2) { if (byte != RF_CODE_STOP) {
// The count byte: Portisch sends at most 7 buckets + sync, so 0 or return true;
// >8 cannot be a genuine capture — reject before it can occupy the
// buffer for a full frame timeout.
return byte != 0 && byte <= B1_MAX_BUCKET_COUNT;
} }
// 0x55 is legal DATA inside a B1 frame: bucket durations are sent
// with only their HIGH byte masked to 7 bits, so a duration such as uint8_t buckets = raw[2] << 1;
// 0x0155 puts a raw 0x55 low byte inside the table — the first 0x55 std::string str;
// must therefore not end the capture. The header declares the table char next_byte[3]; // 2 hex chars + null
// length (raw[2] pairs), so a 0x55 there is always data; one at or
// past the first pulse index is a terminator CANDIDATE, confirmed for (uint32_t i = 0; i <= at; i++) {
// once the UART goes quiet (finish_bucket_frame_ in loop()). buf_append_printf(next_byte, sizeof(next_byte), 0, "%02X", raw[i]);
this->bucket_frame_candidate_ = byte == RF_CODE_STOP && at >= 3 + static_cast<size_t>(raw[2]) * 2; str += next_byte;
return true; if ((i > 3) && buckets) {
buckets--;
}
if ((i < 3) || (buckets % 2) || (i == at - 1)) {
str += " ";
}
}
ESP_LOGI(TAG, "Received RFBridge Bucket: %s", str.c_str());
break;
} }
default: default:
ESP_LOGW(TAG, "Unknown action: 0x%02X", action); ESP_LOGW(TAG, "Unknown action: 0x%02X", action);
@@ -124,47 +119,6 @@ bool RFBridgeComponent::parse_bridge_byte_(uint8_t byte) {
return false; return false;
} }
void RFBridgeComponent::finish_bucket_frame_() {
if (this->rx_buffer_.size() < 4) {
// The candidate flag requires a header + non-empty bucket table, so
// this cannot happen while flag and buffer stay consistent; guard the
// raw[2] / size-1 reads against any future divergence anyway.
this->rx_buffer_.clear();
this->bucket_frame_candidate_ = false;
return;
}
const uint8_t *raw = this->rx_buffer_.data();
const size_t at = this->rx_buffer_.size() - 1;
uint8_t buckets = raw[2] << 1;
std::string str;
char next_byte[3]; // 2 hex chars + null
for (uint32_t i = 0; i <= at; i++) {
buf_append_printf(next_byte, sizeof(next_byte), 0, "%02X", raw[i]);
str += next_byte;
if ((i > 3) && buckets) {
buckets--;
}
if ((i < 3) || (buckets % 2) || (i == at - 1)) {
str += " ";
}
}
ESP_LOGI(TAG, "Received RFBridge Bucket: %s", str.c_str());
// Deliberately NOT ACKed: Portisch's B1 command handler leaves its
// last_sniffing_command at the previous mode (RF_CODE_RFIN), and its
// host-ACK handler re-arms sniffing from that stale value — so ACKing a
// bucket delivery silently reverts the radio to standard sniffing and
// ends bucket capture. Its delivery path is fire-and-forget and never
// waits for a host ACK. Stock Itead firmware never sends B1 frames, so
// suppressing this ACK cannot change stock-firmware behavior.
// https://github.com/esphome/esphome/issues/17682
this->rx_buffer_.clear();
this->bucket_frame_candidate_ = false;
}
void RFBridgeComponent::write_byte_str_(const std::string &codes) { void RFBridgeComponent::write_byte_str_(const std::string &codes) {
uint8_t code; uint8_t code;
int size = codes.length(); int size = codes.length();
@@ -176,31 +130,12 @@ void RFBridgeComponent::write_byte_str_(const std::string &codes) {
void RFBridgeComponent::loop() { void RFBridgeComponent::loop() {
const uint32_t now = App.get_loop_component_start_time(); const uint32_t now = App.get_loop_component_start_time();
size_t avail = this->available(); if (now - this->last_bridge_byte_ > 50) {
if (avail == 0 && this->bucket_frame_candidate_ && now - this->last_bridge_byte_ > BUCKET_CANDIDATE_QUIET_MS) {
// The trailing 0x55 was followed by UART quiet, so it really was the
// frame terminator and not an interior data byte.
this->finish_bucket_frame_();
this->last_bridge_byte_ = now;
}
const bool receiving_bucket = this->rx_buffer_.size() >= 2 && this->rx_buffer_[1] == RF_CODE_RFIN_BUCKET;
if (receiving_bucket) {
// Never declare an in-progress bucket frame dead while its continuation
// bytes are already queued: a stalled loop() otherwise discards a live
// frame that the UART buffer proves is still arriving.
if (avail == 0 && now - this->last_bridge_byte_ > BUCKET_FRAME_TIMEOUT_MS) {
ESP_LOGD(TAG, "Discarding incomplete RFBridge Bucket frame (%u bytes)",
static_cast<unsigned>(this->rx_buffer_.size()));
this->rx_buffer_.clear();
this->bucket_frame_candidate_ = false;
this->last_bridge_byte_ = now;
}
} else if (now - this->last_bridge_byte_ > 50) {
this->rx_buffer_.clear(); this->rx_buffer_.clear();
this->bucket_frame_candidate_ = false;
this->last_bridge_byte_ = now; this->last_bridge_byte_ = now;
} }
size_t avail = this->available();
while (avail > 0) { while (avail > 0) {
uint8_t buf[64]; uint8_t buf[64];
size_t to_read = std::min(avail, sizeof(buf)); size_t to_read = std::min(avail, sizeof(buf));
@@ -211,14 +146,12 @@ void RFBridgeComponent::loop() {
for (size_t i = 0; i < to_read; i++) { for (size_t i = 0; i < to_read; i++) {
if (this->rx_buffer_.size() > MAX_RX_BUFFER_SIZE) { if (this->rx_buffer_.size() > MAX_RX_BUFFER_SIZE) {
this->rx_buffer_.clear(); this->rx_buffer_.clear();
this->bucket_frame_candidate_ = false;
} }
if (this->parse_bridge_byte_(buf[i])) { if (this->parse_bridge_byte_(buf[i])) {
ESP_LOGVV(TAG, "Parsed: 0x%02X", buf[i]); ESP_LOGVV(TAG, "Parsed: 0x%02X", buf[i]);
this->last_bridge_byte_ = now; this->last_bridge_byte_ = now;
} else { } else {
this->rx_buffer_.clear(); this->rx_buffer_.clear();
this->bucket_frame_candidate_ = false;
} }
} }
} }
-13
View File
@@ -30,17 +30,6 @@ static const uint8_t RF_CODE_BEEP = 0xC0;
static const uint8_t RF_CODE_STOP = 0x55; static const uint8_t RF_CODE_STOP = 0x55;
static const uint8_t RF_DEBOUNCE = 200; static const uint8_t RF_DEBOUNCE = 200;
static const size_t MAX_RX_BUFFER_SIZE = 512; static const size_t MAX_RX_BUFFER_SIZE = 512;
// ~10 byte times at 19200 baud: long enough to prove the UART went quiet
// after a possible bucket-frame terminator, short enough to finish well
// before the next radio capture can be delivered.
static const uint32_t BUCKET_CANDIDATE_QUIET_MS = 5;
// Portisch drains a B1 frame's header, bucket table, and pulse data as
// separate UART writes, so an in-progress bucket frame tolerates a longer
// inter-region gap than the generic 50 ms inter-byte timeout.
static const uint32_t BUCKET_FRAME_TIMEOUT_MS = 250;
// Portisch's uart_put_RF_buckets sends at most 7 buckets plus the sync
// bucket, so a B1 count byte above 8 (or 0) is malformed for any protocol.
static const uint8_t B1_MAX_BUCKET_COUNT = 8;
struct RFBridgeData { struct RFBridgeData {
uint16_t sync; uint16_t sync;
@@ -78,12 +67,10 @@ class RFBridgeComponent final : public uart::UARTDevice, public Component {
void ack_(); void ack_();
void decode_(); void decode_();
bool parse_bridge_byte_(uint8_t byte); bool parse_bridge_byte_(uint8_t byte);
void finish_bucket_frame_();
void write_byte_str_(const std::string &codes); void write_byte_str_(const std::string &codes);
std::vector<uint8_t> rx_buffer_; std::vector<uint8_t> rx_buffer_;
uint32_t last_bridge_byte_{0}; uint32_t last_bridge_byte_{0};
bool bucket_frame_candidate_{false};
CallbackManager<void(RFBridgeData)> data_callback_; CallbackManager<void(RFBridgeData)> data_callback_;
CallbackManager<void(RFBridgeAdvancedData)> advanced_data_callback_; CallbackManager<void(RFBridgeAdvancedData)> advanced_data_callback_;
+3 -14
View File
@@ -1,13 +1,10 @@
#include "tuya.h" #include "tuya.h"
#include "esphome/components/network/util.h"
#include "esphome/core/gpio.h" #include "esphome/core/gpio.h"
#include "esphome/core/helpers.h" #include "esphome/core/helpers.h"
#include "esphome/core/log.h" #include "esphome/core/log.h"
#include "esphome/core/util.h" #include "esphome/core/util.h"
#ifdef USE_NETWORK
#include "esphome/components/network/util.h"
#endif
#ifdef USE_WIFI #ifdef USE_WIFI
#include "esphome/components/wifi/wifi_component.h" #include "esphome/components/wifi/wifi_component.h"
#endif #endif
@@ -25,14 +22,6 @@ static const int MAX_RETRIES = 5;
// Max bytes to log for datapoint values (larger values are truncated) // Max bytes to log for datapoint values (larger values are truncated)
static constexpr size_t MAX_DATAPOINT_LOG_BYTES = 16; static constexpr size_t MAX_DATAPOINT_LOG_BYTES = 16;
static bool network_is_connected() {
#ifdef USE_NETWORK
return network::is_connected();
#else
return false;
#endif
}
void Tuya::setup() { void Tuya::setup() {
this->set_interval("heartbeat", 15000, [this] { this->send_empty_command_(TuyaCommandType::HEARTBEAT); }); this->set_interval("heartbeat", 15000, [this] { this->send_empty_command_(TuyaCommandType::HEARTBEAT); });
if (this->status_pin_ != nullptr) { if (this->status_pin_ != nullptr) {
@@ -565,14 +554,14 @@ void Tuya::send_empty_command_(TuyaCommandType command) {
} }
void Tuya::set_status_pin_() { void Tuya::set_status_pin_() {
bool is_network_ready = network_is_connected() && remote_is_connected(); bool is_network_ready = network::is_connected() && remote_is_connected();
this->status_pin_->digital_write(is_network_ready); this->status_pin_->digital_write(is_network_ready);
} }
uint8_t Tuya::get_wifi_status_code_() { uint8_t Tuya::get_wifi_status_code_() {
uint8_t status = 0x02; uint8_t status = 0x02;
if (network_is_connected()) { if (network::is_connected()) {
status = 0x03; status = 0x03;
// Protocol version 3 also supports specifying when connected to "the cloud" // Protocol version 3 also supports specifying when connected to "the cloud"
+12 -4
View File
@@ -1,4 +1,5 @@
from typing import Any from collections.abc import Callable
from typing import Any, NoReturn
from esphome import automation from esphome import automation
from esphome.automation import Trigger from esphome.automation import Trigger
@@ -47,10 +48,17 @@ UDP_SCHEMA = cv.Schema(
) )
def is_relocated(option: str) -> Callable[[Any], NoReturn]:
def validator(value: Any) -> NoReturn:
raise cv.Invalid(
f"The '{option}' option should now be configured in the 'packet_transport' component"
)
return validator
RELOCATED = { RELOCATED = {
cv.Optional(x): cv.invalid( cv.Optional(x): is_relocated(x)
f"The '{x}' option should now be configured in the 'packet_transport' component"
)
for x in ( for x in (
CONF_PROVIDERS, CONF_PROVIDERS,
CONF_ENCRYPTION, CONF_ENCRYPTION,
+16 -4
View File
@@ -22,6 +22,10 @@ namespace esphome {
* pointer. When it is default constructed, it has empty string. You can freely copy or move around this struct, but * pointer. When it is default constructed, it has empty string. You can freely copy or move around this struct, but
* never free its pointer. str() function can be used to export the content as std::string. StringRef is adopted from * never free its pointer. str() function can be used to export the content as std::string. StringRef is adopted from
* <https://github.com/nghttp2/nghttp2/blob/29cbf8b83ff78faf405d1086b16adc09a8772eca/src/template.h#L376> * <https://github.com/nghttp2/nghttp2/blob/29cbf8b83ff78faf405d1086b16adc09a8772eca/src/template.h#L376>
*
* A StringRef may carry a null pointer while its length is zero (the generated api messages start their encode only
* string fields that way). Every member treats that as the empty string; only c_str() hands the null pointer on, so
* callers that print or copy through c_str() must check empty() first.
*/ */
class StringRef { class StringRef {
public: public:
@@ -78,7 +82,7 @@ class StringRef {
/// True if the view begins with the given prefix (std::string::starts_with-like) /// True if the view begins with the given prefix (std::string::starts_with-like)
bool starts_with(const StringRef &prefix) const { bool starts_with(const StringRef &prefix) const {
return len_ >= prefix.len_ && std::memcmp(base_, prefix.base_, prefix.len_) == 0; return len_ >= prefix.len_ && (prefix.len_ == 0 || std::memcmp(base_, prefix.base_, prefix.len_) == 0);
} }
bool starts_with(const char *prefix) const { return this->starts_with(StringRef(prefix)); } bool starts_with(const char *prefix) const { return this->starts_with(StringRef(prefix)); }
bool starts_with(const std::string &prefix) const { return this->starts_with(StringRef(prefix)); } bool starts_with(const std::string &prefix) const { return this->starts_with(StringRef(prefix)); }
@@ -92,14 +96,15 @@ class StringRef {
return actual; return actual;
} }
std::string str() const { return std::string(base_, len_); } std::string str() const { return std::string(base_, len_); } // fine for {nullptr, 0}: nothing is read
const uint8_t *byte() const { return reinterpret_cast<const uint8_t *>(base_); } const uint8_t *byte() const { return reinterpret_cast<const uint8_t *>(base_); }
operator std::string() const { return str(); } operator std::string() const { return str(); }
/// Compare (compatible with std::string::compare) /// Compare (compatible with std::string::compare)
int compare(const StringRef &other) const { int compare(const StringRef &other) const {
int result = std::memcmp(base_, other.base_, std::min(len_, other.len_)); size_type common = std::min(len_, other.len_);
int result = common == 0 ? 0 : std::memcmp(base_, other.base_, common);
if (result != 0) if (result != 0)
return result; return result;
if (len_ < other.len_) if (len_ < other.len_)
@@ -258,7 +263,14 @@ inline double stod(const StringRef &str, size_t *pos = nullptr) {
#ifdef USE_JSON #ifdef USE_JSON
// NOLINTNEXTLINE(readability-identifier-naming) // NOLINTNEXTLINE(readability-identifier-naming)
inline void convertToJson(const StringRef &src, JsonVariant dst) { dst.set(src.c_str()); } inline void convertToJson(const StringRef &src, JsonVariant dst) {
// Bounded by the view length; a null, empty view becomes "" rather than JSON null
if (src.empty()) {
dst.set("");
return;
}
dst.set(JsonString(src.c_str(), src.size()));
}
#endif // USE_JSON #endif // USE_JSON
} // namespace esphome } // namespace esphome
+4 -1
View File
@@ -69,7 +69,10 @@ def _make_create_connection() -> Callable[..., socket.socket]:
from aiohappyeyeballs import start_connection from aiohappyeyeballs import start_connection
from urllib3.exceptions import LocationParseError from urllib3.exceptions import LocationParseError
from urllib3.util.connection import _set_socket_options, allowed_gai_family # noqa: PLC2701 from urllib3.util.connection import ( # noqa: PLC2701
_set_socket_options,
allowed_gai_family,
)
from urllib3.util.timeout import _DEFAULT_TIMEOUT # noqa: PLC2701 from urllib3.util.timeout import _DEFAULT_TIMEOUT # noqa: PLC2701
from esphome import async_thread from esphome import async_thread
+1 -5
View File
@@ -616,15 +616,11 @@ def _make_registry_client() -> Any:
elsewhere, not by the PlatformIO registry. elsewhere, not by the PlatformIO registry.
""" """
from platformio.package.manager._registry import PackageManagerRegistryMixin from platformio.package.manager._registry import PackageManagerRegistryMixin
from platformio.registry.client import RegistryClient
class _Registry(PackageManagerRegistryMixin): class _Registry(PackageManagerRegistryMixin):
def __init__(self) -> None: def __init__(self) -> None:
self._registry_client = None
self.pkg_type = "library" self.pkg_type = "library"
self._registry_client = RegistryClient()
# The probe sleeps ~500 ms per lookup (see runner.patch_registry_private_packages);
# instance-level so the ESPHome process never patches PlatformIO's class
self._registry_client.allowed_private_packages = lambda: False
@staticmethod @staticmethod
def is_system_compatible(value: Any, custom_system: Any = None) -> bool: def is_system_compatible(value: Any, custom_system: Any = None) -> bool:
-2
View File
@@ -951,10 +951,8 @@ def main(argv: list[str]) -> int:
"""Subprocess entry point: ``prefetch <build_dir> <env_name>``.""" """Subprocess entry point: ``prefetch <build_dir> <env_name>``."""
from esphome.core import CORE from esphome.core import CORE
from esphome.log import setup_log from esphome.log import setup_log
from esphome.platformio.runner import patch_registry_private_packages
signal.signal(signal.SIGTERM, _sigterm) signal.signal(signal.SIGTERM, _sigterm)
patch_registry_private_packages()
raw_level = os.environ.get("ESPHOME_PREFETCH_LOG_LEVEL") raw_level = os.environ.get("ESPHOME_PREFETCH_LOG_LEVEL")
try: try:
level = int(raw_level) if raw_level is not None else logging.INFO level = int(raw_level) if raw_level is not None else logging.INFO
+1 -13
View File
@@ -2,8 +2,7 @@
Invoked via ``python -m esphome.platformio.runner`` instead of Invoked via ``python -m esphome.platformio.runner`` instead of
``python -m platformio`` so that the patches (incremental rebuild ``python -m platformio`` so that the patches (incremental rebuild
preservation, download retries, skipping the private-package probe) apply preservation, download retries) apply inside the subprocess. Running
inside the subprocess. Running
PlatformIO in a subprocess keeps its ``sys.path`` mutations and other PlatformIO in a subprocess keeps its ``sys.path`` mutations and other
global state from leaking into the ESPHome process. global state from leaking into the ESPHome process.
""" """
@@ -106,16 +105,6 @@ def patch_file_downloader() -> None:
FileDownloader.__init__ = patched_init FileDownloader.__init__ = patched_init
def patch_registry_private_packages() -> None:
"""Skip PlatformIO's private-package probe; it sleeps ~500 ms per lookup.
ESPHome never uses private packages, so the answer is always False.
"""
from platformio.registry.client import RegistryClient
RegistryClient.allowed_private_packages = staticmethod(lambda: False) # type: ignore[method-assign]
_IGNORE_LIB_WARNINGS = "(?:Hash|Update)" _IGNORE_LIB_WARNINGS = "(?:Hash|Update)"
# Regex patterns matched against each line of PlatformIO output. Lines that # Regex patterns matched against each line of PlatformIO output. Lines that
# match are dropped by RedirectText before they reach the parent process. # match are dropped by RedirectText before they reach the parent process.
@@ -163,7 +152,6 @@ FILTER_PLATFORMIO_LINES = [
def main() -> int: def main() -> int:
patch_structhash() patch_structhash()
patch_file_downloader() patch_file_downloader()
patch_registry_private_packages()
# Wrap stdout/stderr with RedirectText before PlatformIO runs: # Wrap stdout/stderr with RedirectText before PlatformIO runs:
# #
+2 -2
View File
@@ -1,4 +1,4 @@
# Useful stuff when working in a development environment # Useful stuff when working in a development environment
clang-format==13.0.1 # .pre-commit-config.yaml rev synced by script/sync_dependency_versions.py clang-format==13.0.1 # also change in .pre-commit-config.yaml and Dockerfile when updating
clang-tidy==22.1.8 clang-tidy==22.1.8
yamllint==1.38.0 # .pre-commit-config.yaml rev synced by script/sync_dependency_versions.py yamllint==1.38.0 # also change in .pre-commit-config.yaml when updating
+4 -5
View File
@@ -1,9 +1,8 @@
pylint==4.0.8 pylint==4.0.8
flake8==7.3.0 # .pre-commit-config.yaml rev synced by script/sync_dependency_versions.py flake8==7.3.0 # also change in .pre-commit-config.yaml when updating
ruff==0.16.6 # .pre-commit-config.yaml rev synced by script/sync_dependency_versions.py ruff==0.16.5 # also change in .pre-commit-config.yaml when updating
pyupgrade==3.21.2 # .pre-commit-config.yaml rev synced by script/sync_dependency_versions.py pyupgrade==3.21.2 # also change in .pre-commit-config.yaml when updating
prek==0.5.2 # .github/workflows/ci.yml reads this pin prek==0.5.1 # also change in .github/workflows/ci.yml when updating
yamlrocks==0.6.1 # used by script/sync_dependency_versions.py
# Unit tests # Unit tests
pytest==9.1.1 pytest==9.1.1
+237 -298
View File
@@ -28,6 +28,11 @@ class WireType(IntEnum):
END_GROUP = 4 # groups (deprecated) END_GROUP = 4 # groups (deprecated)
FIXED32 = 5 # fixed32, sfixed32, float FIXED32 = 5 # fixed32, sfixed32, float
@property
def cpp_name(self) -> str:
"""The matching constant in proto.h."""
return f"WIRE_TYPE_{self.name}"
# Generate with # Generate with
# protoc --python_out=script/api_protobuf -I esphome/components/api/ api_options.proto # protoc --python_out=script/api_protobuf -I esphome/components/api/ api_options.proto
@@ -126,9 +131,10 @@ def camel_to_snake(name: str) -> str:
return re.sub("([a-z0-9])([A-Z])", r"\1_\2", s1).lower() return re.sub("([a-z0-9])([A-Z])", r"\1_\2", s1).lower()
def force_str(force: bool) -> str: def _encode_call(func: str, *args: str, force: bool = False) -> str:
"""Convert a boolean force value to string format for C++ code.""" """Emit one ProtoEncode call; every helper takes the cursor and returns it advanced."""
return str(force).lower() suffix = "_force" if force else ""
return f"pos = ProtoEncode::{func}{suffix}({', '.join(('pos', *args))});"
class TypeInfo(ABC): class TypeInfo(ABC):
@@ -223,55 +229,39 @@ class TypeInfo(ABC):
def class_member(self) -> str: def class_member(self) -> str:
return f"{self.cpp_type} {self.field_name}{{{self.default_value}}};" return f"{self.cpp_type} {self.field_name}{{{self.default_value}}};"
@property def decode_case(self, body: str) -> str:
def decode_varint_content(self) -> str: """Emit one decode_field() case, keyed on the field's wire tag."""
content = self.decode_varint return f"case proto_tag({self.number}, {self.wire_type.cpp_name}):\n" + indent(
if content is None: f"{body}\nbreak;"
return None )
return f"case {self.number}: this->{self.field_name} = {content}; break;"
decode_varint = None # Expression that reads this field from `value`; None when the type is never decoded.
decode_expr: str | None = None
def _decode_store(self, expr: str) -> str:
return f"this->{self.field_name} = {expr};"
@property @property
def decode_length_content(self) -> str: def decode_content(self) -> str | None:
content = self.decode_length """The decode_field() case for this field, or None when it is never decoded."""
if content is None: expr = self.decode_expr
return None return None if expr is None else self.decode_case(self._decode_store(expr))
return f"case {self.number}: this->{self.field_name} = {content}; break;"
decode_length = None
@property
def decode_32bit_content(self) -> str:
content = self.decode_32bit
if content is None:
return None
return f"case {self.number}: this->{self.field_name} = {content}; break;"
decode_32bit = None
@property
def decode_64bit_content(self) -> str:
content = self.decode_64bit
if content is None:
return None
return f"case {self.number}: this->{self.field_name} = {content}; break;"
decode_64bit = None
# Mapping from encode_func to raw encode expression template. # Mapping from encode_func to raw encode expression template.
# When a forced field has a single-byte tag, the code generator emits # When a forced field has a single-byte tag, the code generator emits
# write_raw_byte(tag) + raw encode instead of the full encode_* method, # write_raw_byte(tag) + raw encode instead of the full encode_* method,
# eliminating the zero-check branch and encode_field_raw indirection. # eliminating the zero-check branch and encode_field_raw indirection.
# {value} is replaced with the actual field expression. # {value} is replaced with the actual field expression.
RAW_ENCODE_MAP: dict[str, str] = { RAW_ENCODE_MAP: dict[str, tuple[str, str]] = {
"encode_uint32": "ProtoEncode::encode_varint_raw(pos, {value});", "encode_uint32": ("encode_varint_raw", "{value}"),
"encode_uint64": "ProtoEncode::encode_varint_raw_64(pos, {value});", "encode_uint64": ("encode_varint_raw_64", "{value}"),
"encode_sint32": "ProtoEncode::encode_varint_raw_short(pos, encode_zigzag32({value}));", "encode_sint32": ("encode_varint_raw_short", "encode_zigzag32({value})"),
"encode_sint64": "ProtoEncode::encode_varint_raw_64(pos, encode_zigzag64({value}));", "encode_sint64": ("encode_varint_raw_64", "encode_zigzag64({value})"),
"encode_int64": "ProtoEncode::encode_varint_raw_64(pos, static_cast<uint64_t>({value}));", "encode_int64": ("encode_varint_raw_64", "static_cast<uint64_t>({value})"),
"encode_bool": "ProtoEncode::write_raw_byte(pos, {value} ? 0x01 : 0x00);", "encode_bool": ("write_raw_byte", "{value} ? 0x01 : 0x00"),
} }
# Fixed32 value expression for the shared tag+fixed32 writer; None for other wire types
fixed32_value_template: str | None = None
def _encode_with_precomputed_tag(self, value_expr: str) -> str | None: def _encode_with_precomputed_tag(self, value_expr: str) -> str | None:
"""Try to emit a precomputed-tag encode for a field. """Try to emit a precomputed-tag encode for a field.
@@ -288,12 +278,17 @@ class TypeInfo(ABC):
return None return None
max_val = self.max_value max_val = self.max_value
# Only use RAW_ENCODE_MAP for forced fields or fields with max_value # Only use RAW_ENCODE_MAP for forced fields or fields with max_value
raw_expr = None raw = None
if self.force or max_val is not None: if self.force or max_val is not None:
raw_expr = self.RAW_ENCODE_MAP.get(self.encode_func) raw = self.RAW_ENCODE_MAP.get(self.encode_func)
if raw_expr is None: if raw is None:
return None return None
body = f"ProtoEncode::write_raw_byte(pos, {tag});\n{raw_expr.format(value=value_expr)}" func, arg = raw
body = (
_encode_call("write_raw_byte", str(tag))
+ "\n"
+ _encode_call(func, arg.format(value=value_expr))
)
if self.force: if self.force:
return body return body
# Non-forced with max_value: inline zero-check + raw encode # Non-forced with max_value: inline zero-check + raw encode
@@ -314,23 +309,44 @@ class TypeInfo(ABC):
return None return None
# When max_len < 128, length varint is always 1 byte # When max_len < 128, length varint is always 1 byte
len_encode = ( len_encode = (
f"ProtoEncode::write_raw_byte(pos, static_cast<uint8_t>({len_expr}));" _encode_call("write_raw_byte", f"static_cast<uint8_t>({len_expr})")
if max_len is not None and max_len < 128 if max_len is not None and max_len < 128
else f"ProtoEncode::encode_varint_raw(pos, {len_expr});" else _encode_call("encode_varint_raw", len_expr)
) )
return "\n".join(
(
_encode_call("write_raw_byte", str(tag)),
len_encode,
_encode_call("encode_raw", data_expr, len_expr),
)
)
def _encode_fixed32_with_precomputed_tag(self, value: str) -> str | None:
"""Single-byte tag fixed32 write, or None for other types and multi-byte tags."""
tag = self.calculate_tag()
if self.fixed32_value_template is None or tag >= 128:
return None
value_expr = self.fixed32_value_template.format(value=value)
if self.force:
return _encode_call("write_tag_and_fixed32", str(tag), value_expr)
return ( return (
f"ProtoEncode::write_raw_byte(pos, {tag});\n" f"if (uint32_t raw = {value_expr}; raw != 0) [[likely]] {{\n"
f"{len_encode}\n" f" {_encode_call('write_tag_and_fixed32', str(tag), 'raw')}\n"
f"ProtoEncode::encode_raw(pos, {data_expr}, {len_expr});" "}"
) )
@property @property
def encode_content(self) -> str: def encode_content(self) -> str:
if result := self._encode_with_precomputed_tag(f"this->{self.field_name}"): value = f"this->{self.field_name}"
if result := self._encode_with_precomputed_tag(value):
return result return result
if self.force: if result := self._encode_fixed32_with_precomputed_tag(value):
return f"ProtoEncode::{self.encode_func}(pos, {self.number}, this->{self.field_name}, true);" return result
return f"ProtoEncode::{self.encode_func}(pos, {self.number}, this->{self.field_name});" return _encode_call(self.encode_func, str(self.number), value, force=self.force)
def encode_element(self, number: int, element: str) -> str:
"""Encode one element of a repeated field; elements are always written."""
return _encode_call(self.encode_func, str(number), element, force=True)
encode_func = None encode_func = None
@@ -550,17 +566,17 @@ def create_field_type_info(
# For messages that decode (SOURCE_CLIENT or SOURCE_BOTH), use pointer # For messages that decode (SOURCE_CLIENT or SOURCE_BOTH), use pointer
# for zero-copy access to the receive buffer # for zero-copy access to the receive buffer
if needs_decode: if needs_decode:
return PointerToBytesBufferType(field, None) return PointerToBytesBufferType(field, needs_decode)
# For SOURCE_SERVER (encode only), explicit annotation is still needed # For SOURCE_SERVER (encode only), explicit annotation is still needed
if get_field_opt(field, pb.pointer_to_buffer, False): if get_field_opt(field, pb.pointer_to_buffer, False):
return PointerToBytesBufferType(field, None) return PointerToBytesBufferType(field, needs_decode)
return BytesType(field, needs_decode, needs_encode) return BytesType(field, needs_decode, needs_encode)
# Special handling for string fields - use StringRef for zero-copy # Special handling for string fields - use StringRef for zero-copy
if field.type == 9: if field.type == 9:
return PointerToStringBufferType(field, None) return PointerToStringBufferType(field, needs_decode)
validate_field_type(field.type, field.name) validate_field_type(field.type, field.name)
if field.type == 11: if field.type == 11:
@@ -605,7 +621,6 @@ class DoubleType(FixedSizeTypeMixin, TypeInfo):
# Unsupported but defined for completeness # Unsupported but defined for completeness
cpp_type = "double" cpp_type = "double"
default_value = "0.0" default_value = "0.0"
decode_64bit = "value.as_double()"
encode_func = "encode_double" encode_func = "encode_double"
wire_type = WireType.FIXED64 # Uses wire type 1 according to protobuf spec wire_type = WireType.FIXED64 # Uses wire type 1 according to protobuf spec
@@ -631,10 +646,12 @@ class DoubleType(FixedSizeTypeMixin, TypeInfo):
class FloatType(FixedSizeTypeMixin, TypeInfo): class FloatType(FixedSizeTypeMixin, TypeInfo):
cpp_type = "float" cpp_type = "float"
default_value = "0.0f" default_value = "0.0f"
decode_32bit = "value.as_float()" decode_expr = "value.as_float()"
encode_func = "encode_float" encode_func = "encode_float"
wire_type = WireType.FIXED32 # Uses wire type 5 wire_type = WireType.FIXED32 # Uses wire type 5
fixed32_value_template = "float_to_raw({value})"
def dump(self, name: str) -> str: def dump(self, name: str) -> str:
o = f'snprintf(buffer, sizeof(buffer), "%g", {name});\n' o = f'snprintf(buffer, sizeof(buffer), "%g", {name});\n'
o += "out.append(buffer);" o += "out.append(buffer);"
@@ -658,7 +675,7 @@ class Int64Type(VarintTypeMixin, TypeInfo):
cpp_type = "int64_t" cpp_type = "int64_t"
_varint_max_bits = 64 _varint_max_bits = 64
default_value = "0" default_value = "0"
decode_varint = "static_cast<int64_t>(value)" decode_expr = "static_cast<int64_t>(value.as_varint())"
encode_func = "encode_int64" encode_func = "encode_int64"
wire_type = WireType.VARINT # Uses wire type 0 wire_type = WireType.VARINT # Uses wire type 0
@@ -679,7 +696,7 @@ class UInt64Type(VarintTypeMixin, TypeInfo):
cpp_type = "uint64_t" cpp_type = "uint64_t"
_varint_max_bits = 64 _varint_max_bits = 64
default_value = "0" default_value = "0"
decode_varint = "value" decode_expr = "value.as_varint()"
encode_func = "encode_uint64" encode_func = "encode_uint64"
wire_type = WireType.VARINT # Uses wire type 0 wire_type = WireType.VARINT # Uses wire type 0
@@ -697,11 +714,11 @@ class UInt64Type(VarintTypeMixin, TypeInfo):
return self._get_simple_size_calculation(name, force, "uint64") return self._get_simple_size_calculation(name, force, "uint64")
@property @property
def RAW_ENCODE_MAP(self) -> dict[str, str]: # noqa: N802 def RAW_ENCODE_MAP(self) -> dict[str, tuple[str, str]]: # noqa: N802
if self.mac_address: if self.mac_address:
return { return {
**TypeInfo.RAW_ENCODE_MAP, **TypeInfo.RAW_ENCODE_MAP,
"encode_uint64": "ProtoEncode::encode_varint_raw_48bit(pos, {value});", "encode_uint64": ("encode_varint_raw_48bit", "{value}"),
} }
return TypeInfo.RAW_ENCODE_MAP return TypeInfo.RAW_ENCODE_MAP
@@ -714,7 +731,7 @@ class Int32Type(VarintTypeMixin, TypeInfo):
cpp_type = "int32_t" cpp_type = "int32_t"
_varint_max_bits = 64 # int32 is sign-extended to 64 bits in protobuf _varint_max_bits = 64 # int32 is sign-extended to 64 bits in protobuf
default_value = "0" default_value = "0"
decode_varint = "static_cast<int32_t>(value)" decode_expr = "static_cast<int32_t>(value.as_varint())"
encode_func = "encode_int32" encode_func = "encode_int32"
wire_type = WireType.VARINT # Uses wire type 0 wire_type = WireType.VARINT # Uses wire type 0
@@ -734,7 +751,6 @@ class Int32Type(VarintTypeMixin, TypeInfo):
class Fixed64Type(FixedSizeTypeMixin, TypeInfo): class Fixed64Type(FixedSizeTypeMixin, TypeInfo):
cpp_type = "uint64_t" cpp_type = "uint64_t"
default_value = "0" default_value = "0"
decode_64bit = "value.as_fixed64()"
encode_func = "encode_fixed64" encode_func = "encode_fixed64"
wire_type = WireType.FIXED64 # Uses wire type 1 wire_type = WireType.FIXED64 # Uses wire type 1
@@ -760,7 +776,7 @@ class Fixed64Type(FixedSizeTypeMixin, TypeInfo):
class Fixed32Type(FixedSizeTypeMixin, TypeInfo): class Fixed32Type(FixedSizeTypeMixin, TypeInfo):
cpp_type = "uint32_t" cpp_type = "uint32_t"
default_value = "0" default_value = "0"
decode_32bit = "value.as_fixed32()" decode_expr = "value.as_fixed32()"
encode_func = "encode_fixed32" encode_func = "encode_fixed32"
wire_type = WireType.FIXED32 # Uses wire type 5 wire_type = WireType.FIXED32 # Uses wire type 5
@@ -769,15 +785,7 @@ class Fixed32Type(FixedSizeTypeMixin, TypeInfo):
o += "out.append(buffer);" o += "out.append(buffer);"
return o return o
@property fixed32_value_template = "{value}"
def encode_content(self) -> str:
tag = self.calculate_tag()
if self.force and tag < 128:
# Emit combined tag+value write: precomputed tag + direct memcpy
return f"ProtoEncode::write_tag_and_fixed32(pos, {tag}, this->{self.field_name});"
if self.force:
return f"ProtoEncode::{self.encode_func}(pos, {self.number}, this->{self.field_name}, true);"
return f"ProtoEncode::{self.encode_func}(pos, {self.number}, this->{self.field_name});"
def get_size_calculation(self, name: str, force: bool = False) -> str: def get_size_calculation(self, name: str, force: bool = False) -> str:
field_id_size = self.calculate_field_id_size() field_id_size = self.calculate_field_id_size()
@@ -797,7 +805,7 @@ class BoolType(VarintTypeMixin, TypeInfo):
_varint_max_bits = 1 _varint_max_bits = 1
cpp_type = "bool" cpp_type = "bool"
default_value = "false" default_value = "false"
decode_varint = "value != 0" decode_expr = "value.as_bool()"
encode_func = "encode_bool" encode_func = "encode_bool"
wire_type = WireType.VARINT # Uses wire type 0 wire_type = WireType.VARINT # Uses wire type 0
@@ -817,7 +825,7 @@ class StringType(TypeInfo):
default_value = "" default_value = ""
reference_type = "std::string &" reference_type = "std::string &"
const_reference_type = "const std::string &" const_reference_type = "const std::string &"
decode_length = "value.as_string()" decode_expr = "value.as_string()"
encode_func = "encode_string" encode_func = "encode_string"
wire_type = WireType.LENGTH_DELIMITED # Uses wire type 2 wire_type = WireType.LENGTH_DELIMITED # Uses wire type 2
@@ -851,9 +859,12 @@ class StringType(TypeInfo):
f"this->{self.field_name}_ref_.size()", f"this->{self.field_name}_ref_.size()",
): ):
return result return result
if self.force: return _encode_call(
return f"ProtoEncode::encode_string(pos, {self.number}, this->{self.field_name}_ref_, true);" "encode_string",
return f"ProtoEncode::encode_string(pos, {self.number}, this->{self.field_name}_ref_);" str(self.number),
f"this->{self.field_name}_ref_",
force=self.force,
)
def dump(self, name): def dump(self, name):
# If name is 'it', this is a repeated field element - always use string # If name is 'it', this is a repeated field element - always use string
@@ -929,6 +940,9 @@ class MessageType(TypeInfo):
def can_use_dump_field(cls) -> bool: def can_use_dump_field(cls) -> bool:
return False return False
def encode_element(self, number: int, element: str) -> str:
return _encode_call("encode_sub_message", "buffer", str(number), element)
@property @property
def cpp_type(self) -> str: def cpp_type(self) -> str:
return self._field.type_name[1:] return self._field.type_name[1:]
@@ -951,15 +965,9 @@ class MessageType(TypeInfo):
@property @property
def encode_content(self) -> str: def encode_content(self) -> str:
# Sub-message encoding needs buffer for backpatch/sync # Sub-message encoding needs buffer for backpatch/sync
return f"ProtoEncode::{self.encode_func}(pos, buffer, {self.number}, this->{self.field_name});" return _encode_call(
self.encode_func, "buffer", str(self.number), f"this->{self.field_name}"
@property )
def decode_length(self) -> str:
# Override to return None for message types because we can't use template-based
# decoding when the specific message type isn't known at compile time.
# Instead, we use the non-template decode_to_message() method which allows
# runtime polymorphism through virtual function calls.
return None
@property @property
def public_content(self) -> list[str]: def public_content(self) -> list[str]:
@@ -976,19 +984,14 @@ class MessageType(TypeInfo):
) )
@property @property
def decode_length_content(self) -> str: def decode_content(self) -> str:
# Custom decode that doesn't use templates body = f"value.decode_to_message(this->{self.field_name});"
if self._track_presence: if self._track_presence:
# decode_to_message() cannot report failure, so setting the flag # decode_to_message() cannot report failure, so setting the flag
# afterwards only documents intent; a status-returning decode could # afterwards only documents intent; a status-returning decode could
# gate it for real without touching callers. # gate it for real without touching callers.
return ( body += f"\nthis->has_{self.name} = true;"
f"case {self.number}:\n" return self.decode_case(body)
f" value.decode_to_message(this->{self.field_name});\n"
f" this->has_{self.name} = true;\n"
f" break;"
)
return f"case {self.number}: value.decode_to_message(this->{self.field_name}); break;"
def dump(self, name: str) -> str: def dump(self, name: str) -> str:
return f"{name}.dump_to(out);" return f"{name}.dump_to(out);"
@@ -1027,7 +1030,7 @@ class BytesType(TypeInfo):
reference_type = "std::string &" reference_type = "std::string &"
const_reference_type = "const std::string &" const_reference_type = "const std::string &"
encode_func = "encode_bytes" encode_func = "encode_bytes"
decode_length = "value.as_string()" decode_expr = "value.as_string()"
wire_type = WireType.LENGTH_DELIMITED # Uses wire type 2 wire_type = WireType.LENGTH_DELIMITED # Uses wire type 2
@property @property
@@ -1058,9 +1061,13 @@ class BytesType(TypeInfo):
f"this->{self.field_name}_ptr_", f"this->{self.field_name}_len_" f"this->{self.field_name}_ptr_", f"this->{self.field_name}_len_"
): ):
return result return result
if self.force: return _encode_call(
return f"ProtoEncode::encode_bytes(pos, {self.number}, this->{self.field_name}_ptr_, this->{self.field_name}_len_, true);" "encode_bytes",
return f"ProtoEncode::encode_bytes(pos, {self.number}, this->{self.field_name}_ptr_, this->{self.field_name}_len_);" str(self.number),
f"this->{self.field_name}_ptr_",
f"this->{self.field_name}_len_",
force=self.force,
)
def dump(self, name: str) -> str: def dump(self, name: str) -> str:
ptr_dump = f"format_hex_pretty(this->{self.field_name}_ptr_, this->{self.field_name}_len_)" ptr_dump = f"format_hex_pretty(this->{self.field_name}_ptr_, this->{self.field_name}_len_)"
@@ -1127,16 +1134,12 @@ class PointerToBufferTypeBase(TypeInfo):
def can_use_dump_field(cls) -> bool: def can_use_dump_field(cls) -> bool:
return False return False
# Only here to make needs_decode required: the null string default keys off it, so a call
# site must not fall back on the base class default
def __init__( def __init__(
self, field: descriptor.FieldDescriptorProto, size: int | None = None self, field: descriptor.FieldDescriptorProto, needs_decode: bool
) -> None: ) -> None:
super().__init__(field) super().__init__(field, needs_decode)
self.array_size = 0
@property
def decode_length(self) -> str | None:
# This is handled in decode_length_content
return None
@property @property
def wire_type(self) -> WireType: def wire_type(self) -> WireType:
@@ -1170,17 +1173,20 @@ class PointerToBytesBufferType(PointerToBufferTypeBase):
f"this->{self.field_name}", f"this->{self.field_name}_len" f"this->{self.field_name}", f"this->{self.field_name}_len"
): ):
return result return result
if self.force: return _encode_call(
return f"ProtoEncode::encode_bytes(pos, {self.number}, this->{self.field_name}, this->{self.field_name}_len, true);" "encode_bytes",
return f"ProtoEncode::encode_bytes(pos, {self.number}, this->{self.field_name}, this->{self.field_name}_len);" str(self.number),
f"this->{self.field_name}",
f"this->{self.field_name}_len",
force=self.force,
)
@property @property
def decode_length_content(self) -> str | None: def decode_content(self) -> str:
return f"""case {self.number}: {{ return self.decode_case(
this->{self.field_name} = value.data(); f"this->{self.field_name} = value.data();\n"
this->{self.field_name}_len = value.size(); f"this->{self.field_name}_len = value.size();",
break; )
}}"""
def dump(self, name: str) -> str: def dump(self, name: str) -> str:
return ( return (
@@ -1214,34 +1220,53 @@ class PointerToStringBufferType(PointerToBufferTypeBase):
def can_use_dump_field(cls) -> bool: def can_use_dump_field(cls) -> bool:
return True return True
@property
def _starts_null(self) -> bool:
"""A field that is only encoded, and skipped when empty, never has its pointer read
before it is set, so it can default to a null StringRef and the message constructs as
one zero fill. Any encode path that copies unconditionally must check this."""
return not self._needs_decode and not self.force
@property @property
def public_content(self) -> list[str]: def public_content(self) -> list[str]:
if self._starts_null:
return [
f"StringRef {self.field_name}{{nullptr, 0}}; // null until set, encode only"
]
return [f"StringRef {self.field_name}{{}};"] return [f"StringRef {self.field_name}{{}};"]
@property @property
def encode_content(self) -> str: def encode_content(self) -> str:
max_len = self.max_data_length max_len = self.max_data_length
if max_len is not None and max_len < 128 and self.force: if max_len is not None and max_len < 128 and self.force:
assert not self._starts_null, (
"unconditional copy of a field that may start null"
)
tag = self.calculate_tag() tag = self.calculate_tag()
if tag < 128: if tag < 128:
return f"ProtoEncode::encode_short_string_force(pos, {tag}, this->{self.field_name});" return _encode_call(
"encode_short_string_force", str(tag), f"this->{self.field_name}"
)
if result := self._encode_bytes_with_precomputed_tag( if result := self._encode_bytes_with_precomputed_tag(
f"this->{self.field_name}.c_str()", f"this->{self.field_name}.c_str()",
f"this->{self.field_name}.size()", f"this->{self.field_name}.size()",
): ):
assert not self._starts_null, (
"unconditional copy of a field that may start null"
)
return result return result
if self.force: return _encode_call(
return f"ProtoEncode::encode_string(pos, {self.number}, this->{self.field_name}, true);" "encode_string",
return ( str(self.number),
f"ProtoEncode::encode_string(pos, {self.number}, this->{self.field_name});" f"this->{self.field_name}",
force=self.force,
) )
@property @property
def decode_length_content(self) -> str | None: def decode_content(self) -> str:
return f"""case {self.number}: {{ return self.decode_case(
this->{self.field_name} = StringRef(reinterpret_cast<const char *>(value.data()), value.size()); f"this->{self.field_name} = StringRef(value.data(), value.size());",
break; )
}}"""
def dump(self, name: str) -> str: def dump(self, name: str) -> str:
# Not used since we use dump_field, but required by abstract base class # Not used since we use dump_field, but required by abstract base class
@@ -1310,14 +1335,13 @@ class PackedBufferTypeInfo(TypeInfo):
] ]
@property @property
def decode_length_content(self) -> str: def decode_content(self) -> str:
"""Store pointer to buffer and calculate count of packed varints.""" """Store pointer to buffer and calculate count of packed varints."""
return f"""case {self.number}: {{ return self.decode_case(
this->{self.field_name}_data_ = value.data(); f"this->{self.field_name}_data_ = value.data();\n"
this->{self.field_name}_length_ = value.size(); f"this->{self.field_name}_length_ = value.size();\n"
this->{self.field_name}_count_ = count_packed_varints(value.data(), value.size()); f"this->{self.field_name}_count_ = count_packed_varints(value.data(), value.size());",
break; )
}}"""
@property @property
def encode_content(self) -> str: def encode_content(self) -> str:
@@ -1402,17 +1426,11 @@ class FixedArrayBytesType(TypeInfo):
] ]
@property @property
def decode_length_content(self) -> str: def decode_content(self) -> str:
o = f"case {self.number}: {{\n" return self.decode_case(
o += " const std::string &data_str = value.as_string();\n" f"this->{self.field_name}_len = std::min<size_t>(value.size(), {self.array_size});\n"
o += f" this->{self.field_name}_len = data_str.size();\n" f"memcpy(this->{self.field_name}, value.data(), this->{self.field_name}_len);",
o += f" if (this->{self.field_name}_len > {self.array_size}) {{\n" )
o += f" this->{self.field_name}_len = {self.array_size};\n"
o += " }\n"
o += f" memcpy(this->{self.field_name}, data_str.data(), this->{self.field_name}_len);\n"
o += " break;\n"
o += "}"
return o
@property @property
def encode_content(self) -> str: def encode_content(self) -> str:
@@ -1421,9 +1439,13 @@ class FixedArrayBytesType(TypeInfo):
f"this->{self.field_name}", f"this->{self.field_name}_len", max_len=max_len f"this->{self.field_name}", f"this->{self.field_name}_len", max_len=max_len
): ):
return result return result
if self.force: return _encode_call(
return f"ProtoEncode::encode_bytes(pos, {self.number}, this->{self.field_name}, this->{self.field_name}_len, true);" "encode_bytes",
return f"ProtoEncode::encode_bytes(pos, {self.number}, this->{self.field_name}, this->{self.field_name}_len);" str(self.number),
f"this->{self.field_name}",
f"this->{self.field_name}_len",
force=self.force,
)
def dump(self, name: str) -> str: def dump(self, name: str) -> str:
return f"out.append(format_hex_pretty({name}, {name}_len));" return f"out.append(format_hex_pretty({name}, {name}_len));"
@@ -1471,7 +1493,7 @@ class UInt32Type(VarintTypeMixin, TypeInfo):
cpp_type = "uint32_t" cpp_type = "uint32_t"
_varint_max_bits = 32 _varint_max_bits = 32
default_value = "0" default_value = "0"
decode_varint = "value" decode_expr = "value.as_varint()"
encode_func = "encode_uint32" encode_func = "encode_uint32"
wire_type = WireType.VARINT # Uses wire type 0 wire_type = WireType.VARINT # Uses wire type 0
@@ -1494,13 +1516,21 @@ class UInt32Type(VarintTypeMixin, TypeInfo):
class EnumType(VarintTypeMixin, TypeInfo): class EnumType(VarintTypeMixin, TypeInfo):
_varint_max_bits = 32 _varint_max_bits = 32
def encode_element(self, number: int, element: str) -> str:
return _encode_call(
self.encode_func,
str(number),
f"static_cast<uint32_t>({element})",
force=True,
)
@property @property
def cpp_type(self) -> str: def cpp_type(self) -> str:
return f"enums::{self._field.type_name[1:]}" return f"enums::{self._field.type_name[1:]}"
@property @property
def decode_varint(self) -> str: def decode_expr(self) -> str:
return f"static_cast<{self.cpp_type}>(value)" return f"static_cast<{self.cpp_type}>(value.as_varint())"
default_value = "" default_value = ""
wire_type = WireType.VARINT # Uses wire type 0 wire_type = WireType.VARINT # Uses wire type 0
@@ -1520,9 +1550,9 @@ class EnumType(VarintTypeMixin, TypeInfo):
@property @property
def encode_content(self) -> str: def encode_content(self) -> str:
value_expr = f"static_cast<uint32_t>(this->{self.field_name})" value_expr = f"static_cast<uint32_t>(this->{self.field_name})"
if self.force: return _encode_call(
return f"ProtoEncode::{self.encode_func}(pos, {self.number}, {value_expr}, true);" self.encode_func, str(self.number), value_expr, force=self.force
return f"ProtoEncode::{self.encode_func}(pos, {self.number}, {value_expr});" )
def dump(self, name: str) -> str: def dump(self, name: str) -> str:
return f"out.append_p(proto_enum_to_string<{self.cpp_type}>({name}));" return f"out.append_p(proto_enum_to_string<{self.cpp_type}>({name}));"
@@ -1547,7 +1577,7 @@ class EnumType(VarintTypeMixin, TypeInfo):
class SFixed32Type(FixedSizeTypeMixin, TypeInfo): class SFixed32Type(FixedSizeTypeMixin, TypeInfo):
cpp_type = "int32_t" cpp_type = "int32_t"
default_value = "0" default_value = "0"
decode_32bit = "value.as_sfixed32()" decode_expr = "value.as_sfixed32()"
encode_func = "encode_sfixed32" encode_func = "encode_sfixed32"
wire_type = WireType.FIXED32 # Uses wire type 5 wire_type = WireType.FIXED32 # Uses wire type 5
@@ -1573,7 +1603,6 @@ class SFixed32Type(FixedSizeTypeMixin, TypeInfo):
class SFixed64Type(FixedSizeTypeMixin, TypeInfo): class SFixed64Type(FixedSizeTypeMixin, TypeInfo):
cpp_type = "int64_t" cpp_type = "int64_t"
default_value = "0" default_value = "0"
decode_64bit = "value.as_sfixed64()"
encode_func = "encode_sfixed64" encode_func = "encode_sfixed64"
wire_type = WireType.FIXED64 # Uses wire type 1 wire_type = WireType.FIXED64 # Uses wire type 1
@@ -1600,7 +1629,7 @@ class SInt32Type(VarintTypeMixin, TypeInfo):
cpp_type = "int32_t" cpp_type = "int32_t"
_varint_max_bits = 32 # zigzag encoding keeps it 32-bit _varint_max_bits = 32 # zigzag encoding keeps it 32-bit
default_value = "0" default_value = "0"
decode_varint = "decode_zigzag32(static_cast<uint32_t>(value))" decode_expr = "decode_zigzag32(static_cast<uint32_t>(value.as_varint()))"
encode_func = "encode_sint32" encode_func = "encode_sint32"
wire_type = WireType.VARINT # Uses wire type 0 wire_type = WireType.VARINT # Uses wire type 0
@@ -1621,7 +1650,7 @@ class SInt64Type(VarintTypeMixin, TypeInfo):
cpp_type = "int64_t" cpp_type = "int64_t"
_varint_max_bits = 64 _varint_max_bits = 64
default_value = "0" default_value = "0"
decode_varint = "decode_zigzag64(value)" decode_expr = "decode_zigzag64(value.as_varint())"
encode_func = "encode_sint64" encode_func = "encode_sint64"
wire_type = WireType.VARINT # Uses wire type 0 wire_type = WireType.VARINT # Uses wire type 0
@@ -1701,9 +1730,9 @@ def _generate_inline_encode_block(
lines = [] lines = []
lines.append(f"auto &sub_msg = {element};") lines.append(f"auto &sub_msg = {element};")
lines.append(f"ProtoEncode::write_raw_byte(pos, {tag});") lines.append(_encode_call("write_raw_byte", str(tag)))
lines.append("uint8_t *len_pos = pos;") lines.append("uint8_t *len_pos = pos;")
lines.append("ProtoEncode::reserve_byte(pos);") lines.append(_encode_call("reserve_byte"))
# Generate inline field encoding for each sub-message field # Generate inline field encoding for each sub-message field
for field in sub_desc.field: for field in sub_desc.field:
@@ -1774,18 +1803,11 @@ class FixedArrayRepeatedType(TypeInfo):
def _encode_element(self, element: str) -> str: def _encode_element(self, element: str) -> str:
"""Helper to generate encode statement for a single element.""" """Helper to generate encode statement for a single element."""
if isinstance(self._ti, EnumType): if isinstance(self._ti, MessageType) and _is_inline_encode(self._ti.cpp_type):
return f"ProtoEncode::{self._ti.encode_func}(pos, {self.number}, static_cast<uint32_t>({element}), true);" return _generate_inline_encode_block(
# Repeated message elements use encode_sub_message (force=true is default) self.number, self._ti.cpp_type, element
if isinstance(self._ti, MessageType): )
if _is_inline_encode(self._ti.cpp_type): return self._ti.encode_element(self.number, element)
return _generate_inline_encode_block(
self.number, self._ti.cpp_type, element
)
return f"ProtoEncode::encode_sub_message(pos, buffer, {self.number}, {element});"
return (
f"ProtoEncode::{self._ti.encode_func}(pos, {self.number}, {element}, true);"
)
@property @property
def cpp_type(self) -> str: def cpp_type(self) -> str:
@@ -2079,55 +2101,23 @@ class RepeatedTypeInfo(TypeInfo):
return self._ti.wire_type return self._ti.wire_type
@property @property
def decode_varint_content(self) -> str: def decode_expr(self) -> str | None:
# Pointer fields don't support decoding return self._ti.decode_expr
if self._use_pointer:
return None def _decode_store(self, expr: str) -> str:
content = self._ti.decode_varint return f"this->{self.field_name}.push_back({expr});"
if content is None:
return None
return (
f"case {self.number}: this->{self.field_name}.push_back({content}); break;"
)
@property @property
def decode_length_content(self) -> str: def decode_content(self) -> str | None:
# Pointer fields don't support decoding # Pointer fields don't support decoding
if self._use_pointer: if self._use_pointer:
return None return None
content = self._ti.decode_length if isinstance(self._ti, MessageType):
if content is None and isinstance(self._ti, MessageType): return self.decode_case(
# Special handling for non-template message decoding f"this->{self.field_name}.emplace_back();\n"
return f"case {self.number}: this->{self.field_name}.emplace_back(); value.decode_to_message(this->{self.field_name}.back()); break;" f"value.decode_to_message(this->{self.field_name}.back());"
if content is None: )
return None return super().decode_content
return (
f"case {self.number}: this->{self.field_name}.push_back({content}); break;"
)
@property
def decode_32bit_content(self) -> str:
# Pointer fields don't support decoding
if self._use_pointer:
return None
content = self._ti.decode_32bit
if content is None:
return None
return (
f"case {self.number}: this->{self.field_name}.push_back({content}); break;"
)
@property
def decode_64bit_content(self) -> str:
# Pointer fields don't support decoding
if self._use_pointer:
return None
content = self._ti.decode_64bit
if content is None:
return None
return (
f"case {self.number}: this->{self.field_name}.push_back({content}); break;"
)
@property @property
def _ti_is_bool(self) -> bool: def _ti_is_bool(self) -> bool:
@@ -2135,15 +2125,7 @@ class RepeatedTypeInfo(TypeInfo):
return isinstance(self._ti, BoolType) return isinstance(self._ti, BoolType)
def _encode_element_call(self, element: str) -> str: def _encode_element_call(self, element: str) -> str:
"""Helper to generate encode call for a single element.""" return self._ti.encode_element(self.number, element)
if isinstance(self._ti, EnumType):
return f"ProtoEncode::{self._ti.encode_func}(pos, {self.number}, static_cast<uint32_t>({element}), true);"
# Repeated message elements use encode_sub_message (force=true is default)
if isinstance(self._ti, MessageType):
return f"ProtoEncode::encode_sub_message(pos, buffer, {self.number}, {element});"
return (
f"ProtoEncode::{self._ti.encode_func}(pos, {self.number}, {element}, true);"
)
@property @property
def encode_content(self) -> str: def encode_content(self) -> str:
@@ -2152,7 +2134,7 @@ class RepeatedTypeInfo(TypeInfo):
# Special handling for const char* elements (when container_no_template contains "const char") # Special handling for const char* elements (when container_no_template contains "const char")
if "const char" in self._container_no_template: if "const char" in self._container_no_template:
o = f"for (const char *it : *this->{self.field_name}) {{\n" o = f"for (const char *it : *this->{self.field_name}) {{\n"
o += f" ProtoEncode::{self._ti.encode_func}(pos, {self.number}, it, strlen(it), true);\n" o += f" {_encode_call(self._ti.encode_func, str(self.number), 'it', 'strlen(it)', force=True)}\n"
else: else:
o = f"for (const auto &it : *this->{self.field_name}) {{\n" o = f"for (const auto &it : *this->{self.field_name}) {{\n"
o += f" {self._encode_element_call('it')}\n" o += f" {self._encode_element_call('it')}\n"
@@ -2538,10 +2520,7 @@ def build_message_type(
) -> tuple[str, str, str]: ) -> tuple[str, str, str]:
public_content: list[str] = [] public_content: list[str] = []
protected_content: list[str] = [] protected_content: list[str] = []
decode_varint: list[str] = [] decode: list[str] = []
decode_length: list[str] = []
decode_32bit: list[str] = []
decode_64bit: list[str] = []
encode: list[str] = [] encode: list[str] = []
dump: list[str] = [] dump: list[str] = []
size_calc: list[str] = [] size_calc: list[str] = []
@@ -2670,22 +2649,8 @@ def build_message_type(
if field.options.HasExtension(pb.field_ifdef): if field.options.HasExtension(pb.field_ifdef):
field_ifdef = field.options.Extensions[pb.field_ifdef] field_ifdef = field.options.Extensions[pb.field_ifdef]
if ti.decode_varint_content: if case := ti.decode_content:
decode_varint.extend( decode.extend(wrap_with_ifdef(case, field_ifdef))
wrap_with_ifdef(ti.decode_varint_content, field_ifdef)
)
if ti.decode_length_content:
decode_length.extend(
wrap_with_ifdef(ti.decode_length_content, field_ifdef)
)
if ti.decode_32bit_content:
decode_32bit.extend(
wrap_with_ifdef(ti.decode_32bit_content, field_ifdef)
)
if ti.decode_64bit_content:
decode_64bit.extend(
wrap_with_ifdef(ti.decode_64bit_content, field_ifdef)
)
if ti.dump_content: if ti.dump_content:
# Check for field_ifdef option for dump as well # Check for field_ifdef option for dump as well
field_ifdef = None field_ifdef = None
@@ -2695,49 +2660,15 @@ def build_message_type(
dump.extend(wrap_with_ifdef(ti.dump_content, field_ifdef)) dump.extend(wrap_with_ifdef(ti.dump_content, field_ifdef))
cpp = "" cpp = ""
if decode_varint: if decode:
o = f"bool {desc.name}::decode_varint(uint32_t field_id, proto_varint_value_t value) {{\n" o = f"void {desc.name}::decode_field(uint32_t tag, const uint8_t *data, proto_varint_value_t scalar) {{\n"
o += " switch (field_id) {\n" o += " const ProtoFieldValue value(data, scalar);\n"
o += indent("\n".join(decode_varint), " ") + "\n" o += " switch (tag) {\n"
o += " default: return false;\n" o += indent("\n".join(decode), " ") + "\n"
o += " }\n" o += " }\n"
o += " return true;\n"
o += "}\n" o += "}\n"
cpp += o cpp += o
prot = "bool decode_varint(uint32_t field_id, proto_varint_value_t value) override;" prot = "void decode_field(uint32_t tag, const uint8_t *data, proto_varint_value_t scalar) override;"
protected_content.insert(0, prot)
if decode_length:
o = f"bool {desc.name}::decode_length(uint32_t field_id, ProtoLengthDelimited value) {{\n"
o += " switch (field_id) {\n"
o += indent("\n".join(decode_length), " ") + "\n"
o += " default: return false;\n"
o += " }\n"
o += " return true;\n"
o += "}\n"
cpp += o
prot = "bool decode_length(uint32_t field_id, ProtoLengthDelimited value) override;"
protected_content.insert(0, prot)
if decode_32bit:
o = f"bool {desc.name}::decode_32bit(uint32_t field_id, Proto32Bit value) {{\n"
o += " switch (field_id) {\n"
o += indent("\n".join(decode_32bit), " ") + "\n"
o += " default: return false;\n"
o += " }\n"
o += " return true;\n"
o += "}\n"
cpp += o
prot = "bool decode_32bit(uint32_t field_id, Proto32Bit value) override;"
protected_content.insert(0, prot)
if decode_64bit:
o = f"bool {desc.name}::decode_64bit(uint32_t field_id, Proto64Bit value) {{\n"
o += " switch (field_id) {\n"
o += indent("\n".join(decode_64bit), " ") + "\n"
o += " default: return false;\n"
o += " }\n"
o += " return true;\n"
o += "}\n"
cpp += o
prot = "bool decode_64bit(uint32_t field_id, Proto64Bit value) override;"
protected_content.insert(0, prot) protected_content.insert(0, prot)
# Generate custom decode() override for messages with FixedVector fields # Generate custom decode() override for messages with FixedVector fields
@@ -2784,28 +2715,36 @@ def build_message_type(
) )
for line in encode for line in encode
] ]
o = f"{speed_attr}uint8_t *{desc.name}::encode(ProtoWriteBuffer &buffer PROTO_ENCODE_DEBUG_PARAM) const {{\n" o = f"{speed_attr}uint8_t *{desc.name}::encode_msg(const void *self, ProtoWriteBuffer &buffer PROTO_ENCODE_DEBUG_PARAM) {{\n"
o += f" const auto &msg = *static_cast<const {desc.name} *>(self);\n"
o += " uint8_t *__restrict__ pos = buffer.get_pos();\n" o += " uint8_t *__restrict__ pos = buffer.get_pos();\n"
o += indent("\n".join(encode_debug)) + "\n" o += indent("\n".join(encode_debug)).replace("this->", "msg.") + "\n"
o += " return pos;\n" o += " return pos;\n"
o += "}\n" o += "}\n"
cpp += o cpp += o
prot = ( public_content.append(
"uint8_t *encode(ProtoWriteBuffer &buffer PROTO_ENCODE_DEBUG_PARAM) const;" "static uint8_t *encode_msg(const void *self, ProtoWriteBuffer &buffer PROTO_ENCODE_DEBUG_PARAM);"
)
public_content.append(
"uint8_t *encode(ProtoWriteBuffer &buffer PROTO_ENCODE_DEBUG_PARAM) const {\n"
" return encode_msg(this, buffer PROTO_ENCODE_DEBUG_ARG);\n"
"}"
) )
public_content.append(prot)
# If no fields to encode or message doesn't need encoding, the default implementation in ProtoMessage will be used # If no fields to encode or message doesn't need encoding, the default implementation in ProtoMessage will be used
# Add calculate_size method only if this message needs encoding and has fields # Add calculate_size method only if this message needs encoding and has fields
if needs_encode and size_calc and not is_inline_only: if needs_encode and size_calc and not is_inline_only:
o = f"{speed_attr}uint32_t {desc.name}::calculate_size() const {{\n" o = f"{speed_attr}uint32_t {desc.name}::calc_size_msg(const void *self) {{\n"
o += f" const auto &msg = *static_cast<const {desc.name} *>(self);\n"
o += " uint32_t size = 0;\n" o += " uint32_t size = 0;\n"
o += indent("\n".join(size_calc)) + "\n" o += indent("\n".join(size_calc)).replace("this->", "msg.") + "\n"
o += " return size;\n" o += " return size;\n"
o += "}\n" o += "}\n"
cpp += o cpp += o
prot = "uint32_t calculate_size() const;" public_content.append("static uint32_t calc_size_msg(const void *self);")
public_content.append(prot) public_content.append(
"uint32_t calculate_size() const { return calc_size_msg(this); }"
)
# If no fields to calculate size for or message doesn't need encoding, the default implementation in ProtoMessage will be used # If no fields to calculate size for or message doesn't need encoding, the default implementation in ProtoMessage will be used
# dump_to method declaration in header # dump_to method declaration in header
+1 -25
View File
@@ -14,31 +14,7 @@ top=$(git rev-parse --show-toplevel 2>/dev/null) || exit 0
[ -x "$top/venv/bin/python" ] && exit 0 [ -x "$top/venv/bin/python" ] && exit 0
[ -x "$top/script/setup" ] || exit 0 [ -x "$top/script/setup" ] || exit 0
# Every worktree shares the hooks directory of the checkout it was created
# from, and the script/setup run below is the one from whichever branch was just
# checked out. Older branches install their own pre-commit hook without checking
# for a worktree: that moves the shared hook aside as pre-commit.legacy and
# replaces it with one tied to this worktree's virtual environment, so commits
# break in every checkout. To rule that out, the hooks directory is copied
# before script/setup runs and put back exactly as it was afterwards, including
# removing any file script/setup added.
hooks=$(git rev-parse --path-format=absolute --git-path hooks 2>/dev/null) || exit 0
snap=$(mktemp -d "$hooks/.post-checkout.XXXXXX") || exit 0
cp -p "$hooks"/* "$snap"/ 2>/dev/null
# Clear VIRTUAL_ENV so a checkout made from a shell with an environment already # Clear VIRTUAL_ENV so a checkout made from a shell with an environment already
# activated still gets its own, rather than having the active one repointed at # activated still gets its own, rather than having the active one repointed at
# this working tree. # this working tree.
env -u VIRTUAL_ENV "$top/script/setup" exec env -u VIRTUAL_ENV "$top/script/setup"
status=$?
for f in "$hooks"/*; do
[ -e "$snap/${f##*/}" ] || rm -f "$f"
done
# Files are moved rather than copied so a hook that is still running, such as
# this one, is swapped out atomically instead of being rewritten in place.
for f in "$snap"/*; do
cmp -s "$f" "$hooks/${f##*/}" 2>/dev/null || mv -f "$f" "$hooks/${f##*/}"
done
rm -rf "$snap"
exit $status
-17
View File
@@ -1104,10 +1104,6 @@ def get_components_per_integration_fixture() -> dict[str, set[str]]:
_TEST_FUNC_RE = re.compile(r"async def (test_\w+)") _TEST_FUNC_RE = re.compile(r"async def (test_\w+)")
# Any usage form (decorator, pytestmark assignment or list element); only
# test_*.py files are scanned, so the marker docs elsewhere cannot false-hit
_SHARED_YAML_USE_RE = re.compile(r"\bmark\.shared_yaml")
_SHARED_YAML_ARG_RE = re.compile(r"\(\s*[\"'](\w+)[\"']\s*\)")
@cache @cache
@@ -1127,19 +1123,6 @@ def get_fixture_to_test_files() -> dict[str, frozenset[str]]:
for func in _TEST_FUNC_RE.findall(content): for func in _TEST_FUNC_RE.findall(content):
base_name = func.replace("test_", "").partition("[")[0] base_name = func.replace("test_", "").partition("[")[0]
result.setdefault(base_name, set()).add(rel_path) result.setdefault(base_name, set()).add(rel_path)
# Shared fixtures are named by marker, not by a test function; each
# decorator must carry a string literal or its fixture would silently
# map to no tests
for use in _SHARED_YAML_USE_RE.finditer(content):
arg = _SHARED_YAML_ARG_RE.match(content, use.end())
if arg is None:
line = content.count("\n", 0, use.start()) + 1
raise ValueError(
f"{rel_path}:{line}: shared_yaml marker must take a "
"single-line string literal so CI test selection can map "
"its fixture"
)
result.setdefault(arg.group(1), set()).add(rel_path)
return {k: frozenset(v) for k, v in result.items()} return {k: frozenset(v) for k, v in result.items()}
-164
View File
@@ -1,164 +0,0 @@
#!/usr/bin/env python3
"""Keep pre-commit hook revs in sync with the requirements files.
Dependabot only bumps the ``package==version`` pins in ``requirements*.txt``.
Some of those tools are pinned a second time as hook ``rev`` values in
``.pre-commit-config.yaml``. This script treats the requirements files as
the source of truth and rewrites the revs to match, editing the config
through yamlrocks so comments and layout survive.
Run without arguments to apply the changes in place, or with ``--check`` to
only report drift (exit status 1 when anything is out of sync).
"""
from __future__ import annotations
import argparse
from dataclasses import dataclass
from pathlib import Path
import re
import sys
from typing import Any
import yamlrocks
REPO_ROOT = Path(__file__).resolve().parent.parent
PRECOMMIT_CONFIG = ".pre-commit-config.yaml"
class SyncError(Exception):
"""A pin could not be located in a requirements file or the config."""
@dataclass(frozen=True)
class SyncTarget:
"""A requirements pin and the pre-commit repo whose rev mirrors it."""
package: str
requirements_file: str
repo: str
SYNC_TARGETS: tuple[SyncTarget, ...] = (
SyncTarget(
"ruff", "requirements_test.txt", "https://github.com/astral-sh/ruff-pre-commit"
),
SyncTarget("flake8", "requirements_test.txt", "https://github.com/PyCQA/flake8"),
SyncTarget(
"pyupgrade", "requirements_test.txt", "https://github.com/asottile/pyupgrade"
),
SyncTarget(
"clang-format",
"requirements_dev.txt",
"https://github.com/pre-commit/mirrors-clang-format",
),
SyncTarget(
"yamllint",
"requirements_dev.txt",
"https://github.com/adrienverge/yamllint.git",
),
)
def read_requirement_version(requirements: str, package: str) -> str | None:
"""Return the ``==`` pin for ``package`` or None when it is not pinned."""
pattern = re.compile(
rf"^{re.escape(package)}==(?P<version>[^\s#]+)",
re.MULTILINE | re.IGNORECASE,
)
match = pattern.search(requirements)
return match.group("version") if match else None
def find_repo_entry(doc: Any, repo: str) -> Any:
"""Return the single ``- repo:`` block for ``repo`` in a pre-commit doc."""
try:
entries = [entry for entry in doc["repos"] if entry["repo"] == repo]
except KeyError as err:
raise SyncError(f"malformed pre-commit config, missing key {err}") from None
if len(entries) != 1:
raise SyncError(
f"expected exactly one block for repo {repo}, found {len(entries)}"
)
return entries[0]
def current_rev(entry: Any, repo: str) -> tuple[str, str]:
"""Split the block's rev into its tag prefix (``v`` or empty) and version."""
if "rev" not in entry:
raise SyncError(f"repo {repo} has no rev")
rev = entry["rev"]
if not isinstance(rev, str):
# A rev such as ``1.0`` parses as a number and cannot be compared or
# rewritten safely; quote it in the config instead.
raise SyncError(f"rev of repo {repo} is not a string: {rev!r}")
prefix = "v" if rev.startswith("v") else ""
return prefix, rev.removeprefix("v")
def sync(root: Path, *, write: bool) -> list[str]:
"""Bring every hook rev in line with its requirements pin.
Returns one description per rev that was (or, when ``write`` is False,
would be) changed. Raises SyncError when a pin cannot be found, which
means SYNC_TARGETS has gone stale and needs updating by hand.
"""
config_path = root / PRECOMMIT_CONFIG
doc = yamlrocks.loads(config_path.read_bytes(), option=yamlrocks.OPT_ROUND_TRIP)
requirements: dict[str, str] = {}
changes: list[str] = []
for target in SYNC_TARGETS:
if target.requirements_file not in requirements:
requirements[target.requirements_file] = (
root / target.requirements_file
).read_text()
version = read_requirement_version(
requirements[target.requirements_file], target.package
)
if version is None:
raise SyncError(
f"{target.requirements_file}: no '{target.package}==' pin found"
)
entry = find_repo_entry(doc, target.repo)
prefix, current = current_rev(entry, target.repo)
if current == version:
continue
changes.append(f"{target.package}: {current} -> {version}")
entry["rev"] = f"{prefix}{version}"
if changes and write:
config_path.write_bytes(doc.to_yaml())
return changes
def main(argv: list[str] | None = None) -> int:
parser = argparse.ArgumentParser(description=__doc__.splitlines()[0])
parser.add_argument(
"--check",
action="store_true",
help="report drift without modifying any file; exit 1 if out of sync",
)
parser.add_argument(
"--root",
type=Path,
default=REPO_ROOT,
help="repository checkout to operate on (default: this checkout)",
)
args = parser.parse_args(argv)
try:
changes = sync(args.root, write=not args.check)
except SyncError as err:
print(f"error: {err}", file=sys.stderr)
return 1
for change in changes:
print(change)
if args.check and changes:
return 1
return 0
if __name__ == "__main__": # pragma: no cover
sys.exit(main())
@@ -249,7 +249,7 @@ static APIBuffer build_infrared_rf_transmit_wire() {
std::memcpy(bytes + len, packed, packed_len); std::memcpy(bytes + len, packed, packed_len);
len += packed_len; len += packed_len;
// field 6: modulation = 1 (non-zero so it's actually emitted and exercises // field 6: modulation = 1 (non-zero so it's actually emitted and exercises
// decode_varint for this field, matching the documented layout above). // decode_field for this field, matching the documented layout above).
put_byte(0x30); put_byte(0x30);
put_varint(1); put_varint(1);
@@ -1,32 +0,0 @@
esphome:
name: test-keyboard-no-label
esp32:
board: esp32dev
framework:
type: esp-idf
spi:
- id: spi_bus
clk_pin: GPIO18
mosi_pin: GPIO23
display:
- platform: mipi_spi
spi_id: spi_bus
model: st7789v
id: tft_display
dimensions:
width: 240
height: 320
cs_pin: GPIO22
dc_pin: GPIO21
auto_clear_enabled: false
invert_colors: false
update_interval: never
lvgl:
displays: tft_display
widgets:
- keyboard:
id: keyboard_widget
@@ -1,34 +0,0 @@
esphome:
name: test-qrcode-no-label
esp32:
board: esp32dev
framework:
type: esp-idf
spi:
- id: spi_bus
clk_pin: GPIO18
mosi_pin: GPIO23
display:
- platform: mipi_spi
spi_id: spi_bus
model: st7789v
id: tft_display
dimensions:
width: 240
height: 320
cs_pin: GPIO22
dc_pin: GPIO21
auto_clear_enabled: false
invert_colors: false
update_interval: never
lvgl:
displays: tft_display
widgets:
- qrcode:
id: qr_widget
size: 100
text: "esphome.io"
@@ -1,35 +0,0 @@
esphome:
name: test-tabview-no-label
esp32:
board: esp32dev
framework:
type: esp-idf
spi:
- id: spi_bus
clk_pin: GPIO18
mosi_pin: GPIO23
display:
- platform: mipi_spi
spi_id: spi_bus
model: st7789v
id: tft_display
dimensions:
width: 240
height: 320
cs_pin: GPIO22
dc_pin: GPIO21
auto_clear_enabled: false
invert_colors: false
update_interval: never
lvgl:
displays: tft_display
widgets:
- tabview:
id: tabview_widget
tabs:
- name: "Tab 1"
id: tab_1
@@ -1,32 +0,0 @@
"""Widgets whose LVGL C implementation creates or references labels
internally (tab titles, key legends, the QR canvas fallback) must declare
the label dependency in ``get_uses()``. Otherwise a config that contains
no ``label`` widget of its own compiles LVGL without ``LV_USE_LABEL`` and
fails at C compile time with undefined ``lv_label_*`` symbols.
"""
from __future__ import annotations
from collections.abc import Callable
from pathlib import Path
import pytest
from esphome.components.lvgl import defines as df
@pytest.mark.parametrize(
"yaml_file",
[
"qrcode_no_label.yaml",
"keyboard_no_label.yaml",
"tabview_no_label.yaml",
],
)
def test_label_less_config_enables_lv_use_label(
generate_main: Callable[[str | Path], str],
component_config_path: Callable[[str], Path],
yaml_file: str,
) -> None:
generate_main(component_config_path(yaml_file))
assert "LV_USE_LABEL" in df.get_defines()
@@ -59,7 +59,7 @@ static void verify_mac(uint64_t mac, size_t expected_bytes) {
#ifdef ESPHOME_DEBUG_API #ifdef ESPHOME_DEBUG_API
uint8_t *proto_debug_end_ = api_buf.data() + api_buf.size(); uint8_t *proto_debug_end_ = api_buf.data() + api_buf.size();
#endif #endif
ProtoEncode::encode_varint_raw_48bit(pos PROTO_ENCODE_DEBUG_ARG, mac); pos = ProtoEncode::encode_varint_raw_48bit(pos PROTO_ENCODE_DEBUG_ARG, mac);
size_t new_len = pos - api_buf.data(); size_t new_len = pos - api_buf.data();
EXPECT_EQ(new_len, expected_bytes) << "mac=0x" << std::hex << mac << std::dec; EXPECT_EQ(new_len, expected_bytes) << "mac=0x" << std::hex << mac << std::dec;
-5
View File
@@ -1,5 +0,0 @@
from tests.testing_helpers import ComponentManifestOverride
def override_manifest(manifest: ComponentManifestOverride) -> None:
manifest.dependencies = manifest.dependencies + ["sensor", "spi"]
@@ -1,62 +0,0 @@
#include <gtest/gtest.h>
#include "esphome/components/atm90e32/atm90e32.h"
namespace esphome::atm90e32::testing {
TEST(ATM90E32OffsetRegisterVerification, AcceptsExactSignedReadback) {
EXPECT_TRUE(offset_register_value_matches(0x007B, 123));
EXPECT_TRUE(offset_register_value_matches(0xFF85, -123));
}
TEST(ATM90E32OffsetRegisterVerification, RejectsMismatchedReadback) {
EXPECT_FALSE(offset_register_value_matches(0x007C, 123));
EXPECT_FALSE(offset_register_value_matches(0xFF84, -123));
}
TEST(ATM90E32OffsetRestoreState, ReportsVerifiedStoredValuesAsRestored) {
const auto state = resolve_offset_restore_state(true, true, false);
EXPECT_TRUE(state.restored);
EXPECT_TRUE(state.values_verified);
}
TEST(ATM90E32OffsetRestoreState, ReportsVerifiedConfigFallbackAsNotRestored) {
const auto state = resolve_offset_restore_state(true, false, true);
EXPECT_FALSE(state.restored);
EXPECT_TRUE(state.values_verified);
}
TEST(ATM90E32OffsetRestoreState, ReportsFailedConfigFallbackAsUnverified) {
const auto state = resolve_offset_restore_state(true, false, false);
EXPECT_FALSE(state.restored);
EXPECT_FALSE(state.values_verified);
}
TEST(ATM90E32OffsetRestoreState, ReportsConfigWithoutStoredValuesAsNotRestored) {
const auto state = resolve_offset_restore_state(false, true, false);
EXPECT_FALSE(state.restored);
EXPECT_TRUE(state.values_verified);
}
TEST(ATM90E32OffsetPersistence, RollsBackStoredValuesOrZeroSentinel) {
const OffsetCalibration previous[3]{{1, -1}, {2, -2}, {3, -3}};
OffsetCalibration rollback[3]{};
prepare_offset_rollback(previous, true, rollback);
for (uint8_t phase = 0; phase < 3; phase++) {
EXPECT_EQ(rollback[phase].first_offset, previous[phase].first_offset);
EXPECT_EQ(rollback[phase].second_offset, previous[phase].second_offset);
}
prepare_offset_rollback(previous, false, rollback);
for (const auto &phase : rollback) {
EXPECT_EQ(phase.first_offset, 0);
EXPECT_EQ(phase.second_offset, 0);
}
}
} // namespace esphome::atm90e32::testing
+43
View File
@@ -59,4 +59,47 @@ TEST(StringRefStartsWith, RefOverloadComparesOnlyTheViewedLength) {
EXPECT_TRUE(ref.starts_with(prefix)); EXPECT_TRUE(ref.starts_with(prefix));
} }
// The generated api messages start their encode only string fields as a null pointer with zero
// length; every member must treat that exactly like the default constructed empty string.
TEST(StringRefNullEmpty, BehavesAsEmptyString) {
const StringRef null_empty{nullptr, 0};
const StringRef empty;
EXPECT_TRUE(null_empty.empty());
EXPECT_EQ(null_empty.size(), 0u);
EXPECT_EQ(null_empty.c_str(), nullptr);
EXPECT_TRUE(null_empty == empty);
EXPECT_TRUE(null_empty == "");
EXPECT_TRUE(null_empty == std::string());
EXPECT_EQ(null_empty.compare(empty), 0);
EXPECT_EQ(null_empty.compare(""), 0);
EXPECT_LT(null_empty.compare("a"), 0);
EXPECT_TRUE(null_empty.starts_with(""));
EXPECT_FALSE(null_empty.starts_with("a"));
EXPECT_EQ(null_empty.str(), std::string());
EXPECT_EQ(null_empty.substr(0), std::string());
EXPECT_EQ(null_empty.find('a'), std::string::npos);
EXPECT_EQ(null_empty.find("a"), std::string::npos);
char buf[4] = "xyz";
EXPECT_EQ(null_empty.copy(buf, sizeof(buf)), 0u);
EXPECT_EQ(null_empty.begin(), null_empty.end());
}
TEST(StringRefNullEmpty, ComparesAgainstText) {
const StringRef null_empty{nullptr, 0};
const StringRef text("abc", 3);
EXPECT_FALSE(null_empty == text);
EXPECT_FALSE(text == null_empty);
EXPECT_LT(null_empty.compare(text), 0);
EXPECT_GT(text.compare(null_empty), 0);
EXPECT_TRUE(text.starts_with(null_empty));
}
TEST(StringRefNullEmpty, TwoNullViewsAreEqual) {
const StringRef a{nullptr, 0};
const StringRef b{nullptr, 0};
EXPECT_TRUE(a == b);
EXPECT_EQ(a.compare(b), 0);
EXPECT_TRUE(a.starts_with(b));
}
} // namespace esphome::core::testing } // namespace esphome::core::testing
@@ -1,29 +0,0 @@
# Tuya without any network component (no wifi/ethernet/api), as used on
# serial-only or BLE-only Tuya MCU boards. Regression test for
# https://github.com/esphome/esphome/issues/18942
substitutions:
status_pin: P6
packages:
uart: !include ../../test_build_components/common/uart/bk72xx-ard.yaml
tuya:
status_pin: ${status_pin}
binary_sensor:
- platform: tuya
id: tuya_presence
sensor_datapoint: 101
sensor:
- platform: tuya
id: tuya_light_intensity
sensor_datapoint: 103
number:
- platform: tuya
id: tuya_far_detection
number_datapoint: 109
min_value: 0
max_value: 600
step: 1
-7
View File
@@ -21,13 +21,6 @@ The `yaml_config` fixture automatically loads YAML configurations based on the t
- The fixture file must exist or the test will fail with a clear error message - The fixture file must exist or the test will fail with a clear error message
- The fixture automatically injects a dynamic port number into the API configuration - The fixture automatically injects a dynamic port number into the API configuration
Tests marked `@pytest.mark.shared_yaml("name")` load `fixtures/name.yaml` instead
of the test-named file and compile it in a shared, hash-keyed build directory, so
the whole group pays one full compile and each test only a relink. The marker
argument must be a single-line string literal (CI test selection maps fixtures to
test files by scanning for it), and marked tests must hand the `yaml_config`
content to `run_compiled` unmodified.
### Key Fixtures ### Key Fixtures
- `run_compiled` - Combines write, compile, and run operations into a single context manager - `run_compiled` - Combines write, compile, and run operations into a single context manager
+81 -335
View File
@@ -4,22 +4,17 @@ from __future__ import annotations
import asyncio import asyncio
from collections.abc import AsyncGenerator, Callable, Generator from collections.abc import AsyncGenerator, Callable, Generator
from contextlib import AbstractAsyncContextManager, asynccontextmanager, suppress from contextlib import AbstractAsyncContextManager, asynccontextmanager
import fcntl import fcntl
from functools import cache
import hashlib
import logging import logging
import os import os
from pathlib import Path from pathlib import Path
import platform import platform
import re
import shutil
import signal import signal
import socket import socket
import subprocess import subprocess
import sys import sys
import tempfile import tempfile
import time
from typing import TextIO from typing import TextIO
from aioesphomeapi import APIClient, APIConnectionError, LogParser, ReconnectLogic from aioesphomeapi import APIClient, APIConnectionError, LogParser, ReconnectLogic
@@ -28,13 +23,7 @@ import pytest_asyncio
import esphome.config import esphome.config
from esphome.core import CORE from esphome.core import CORE
from esphome.helpers import ( from esphome.helpers import get_usable_cpu_count
get_usable_cpu_count,
read_file,
rmtree,
write_file,
write_file_if_changed,
)
from esphome.platformio.toolchain import get_idedata from esphome.platformio.toolchain import get_idedata
from .const import ( from .const import (
@@ -67,21 +56,6 @@ import pty # not available on Windows
pytest.register_assert_rewrite("tests.integration.entity_utils") pytest.register_assert_rewrite("tests.integration.entity_utils")
def pytest_configure(config: pytest.Config) -> None:
config.addinivalue_line(
"markers",
"shared_yaml(name): load fixtures/<name>.yaml and compile it in a shared, "
"hash-keyed incremental build directory",
)
FIXTURES_DIR = Path(__file__).parent / "fixtures"
REPO_ROOT = Path(__file__).resolve().parent.parent.parent
# CI caches parts of this path; keep in sync with ci.yml integration-tests.
INTEGRATION_TESTS_ROOT = Path.home() / ".esphome-integration-tests"
def _get_platformio_env(cache_dir: Path) -> dict[str, str]: def _get_platformio_env(cache_dir: Path) -> dict[str, str]:
"""Get environment variables for PlatformIO with shared cache.""" """Get environment variables for PlatformIO with shared cache."""
env = os.environ.copy() env = os.environ.copy()
@@ -104,7 +78,7 @@ def _get_platformio_env(cache_dir: Path) -> dict[str, str]:
) )
# Compile with THIS tree's esphome sources, not wherever the venv's editable # Compile with THIS tree's esphome sources, not wherever the venv's editable
# install points (which may be a different git worktree or checkout). # install points (which may be a different git worktree or checkout).
repo_root = str(REPO_ROOT) repo_root = str(Path(__file__).resolve().parent.parent.parent)
existing = env.get("PYTHONPATH") existing = env.get("PYTHONPATH")
env["PYTHONPATH"] = f"{repo_root}{os.pathsep}{existing}" if existing else repo_root env["PYTHONPATH"] = f"{repo_root}{os.pathsep}{existing}" if existing else repo_root
return env return env
@@ -114,7 +88,8 @@ def _get_platformio_env(cache_dir: Path) -> dict[str, str]:
def shared_platformio_cache() -> Generator[Path]: def shared_platformio_cache() -> Generator[Path]:
"""Initialize a shared PlatformIO cache for all integration tests.""" """Initialize a shared PlatformIO cache for all integration tests."""
# Use a dedicated directory for integration tests to avoid conflicts. # Use a dedicated directory for integration tests to avoid conflicts.
test_cache_dir = INTEGRATION_TESTS_ROOT # CI caches parts of this path; keep in sync with ci.yml integration-tests.
test_cache_dir = Path.home() / ".esphome-integration-tests"
cache_dir = test_cache_dir / "platformio" cache_dir = test_cache_dir / "platformio"
# Use a lock file in the home directory to ensure only one process initializes the cache # Use a lock file in the home directory to ensure only one process initializes the cache
@@ -137,9 +112,7 @@ def shared_platformio_cache() -> Generator[Path]:
init_dir = Path(tmpdir) init_dir = Path(tmpdir)
fixture_path = Path(__file__).parent / "fixtures" / "cache_init.yaml" fixture_path = Path(__file__).parent / "fixtures" / "cache_init.yaml"
config_path = init_dir / "cache_init.yaml" config_path = init_dir / "cache_init.yaml"
config_path.write_text( config_path.write_text(fixture_path.read_text())
fixture_path.read_text(encoding="utf-8"), encoding="utf-8"
)
# Run compilation to populate the cache # Run compilation to populate the cache
# We must succeed here to avoid race conditions where multiple # We must succeed here to avoid race conditions where multiple
@@ -189,6 +162,13 @@ def integration_test_dir() -> Generator[Path]:
yield Path(tmpdir) yield Path(tmpdir)
@pytest.fixture
def isolated_preferences(monkeypatch: pytest.MonkeyPatch, tmp_path: Path) -> None:
"""Host preferences persist per device name; give the test its own so a
provisioned key never leaks into another run."""
monkeypatch.setenv("ESPHOME_PREFDIR", str(tmp_path / "prefs"))
@pytest.fixture @pytest.fixture
def reserved_tcp_port() -> Generator[tuple[int, socket.socket]]: def reserved_tcp_port() -> Generator[tuple[int, socket.socket]]:
"""Reserve an unused TCP port by holding the socket open.""" """Reserve an unused TCP port by holding the socket open."""
@@ -208,29 +188,21 @@ def unused_tcp_port(reserved_tcp_port: tuple[int, socket.socket]) -> int:
return reserved_tcp_port[0] return reserved_tcp_port[0]
@pytest.fixture(autouse=True)
def isolated_preferences(monkeypatch: pytest.MonkeyPatch, tmp_path: Path) -> Path:
"""Give every test its own host prefs dir; prefs are keyed only by device
name, which tests sharing a fixture also share."""
prefdir = tmp_path / "prefs"
monkeypatch.setenv("ESPHOME_PREFDIR", str(prefdir))
return prefdir
@pytest_asyncio.fixture @pytest_asyncio.fixture
async def yaml_config(request: pytest.FixtureRequest, unused_tcp_port: int) -> str: async def yaml_config(request: pytest.FixtureRequest, unused_tcp_port: int) -> str:
"""Load YAML configuration based on test name.""" """Load YAML configuration based on test name."""
shared_name = _shared_yaml_name(request) # Get the test function name
# Base test name: test_ prefix and any parametrization stripped test_name: str = request.node.name
base_name = shared_name or request.node.name.replace("test_", "").partition("[")[0] # Extract the base test name (remove test_ prefix and any parametrization)
base_name = test_name.replace("test_", "").partition("[")[0]
# Load the fixture file # Load the fixture file
fixture_path = FIXTURES_DIR / f"{base_name}.yaml" fixture_path = Path(__file__).parent / "fixtures" / f"{base_name}.yaml"
if not fixture_path.exists(): if not fixture_path.exists():
raise FileNotFoundError(f"Fixture file not found: {fixture_path}") raise FileNotFoundError(f"Fixture file not found: {fixture_path}")
loop = asyncio.get_running_loop() loop = asyncio.get_running_loop()
content = await loop.run_in_executor(None, read_file, fixture_path) content = await loop.run_in_executor(None, fixture_path.read_text)
# Replace the port in the config if it contains api section # Replace the port in the config if it contains api section
if "api:" in content: if "api:" in content:
@@ -254,13 +226,11 @@ async def yaml_config(request: pytest.FixtureRequest, unused_tcp_port: int) -> s
# Replace external component path placeholder if present # Replace external component path placeholder if present
if "EXTERNAL_COMPONENT_PATH" in content: if "EXTERNAL_COMPONENT_PATH" in content:
external_components_path = str(FIXTURES_DIR / "external_components") external_components_path = str(
Path(__file__).parent / "fixtures" / "external_components"
)
content = content.replace("EXTERNAL_COMPONENT_PATH", external_components_path) content = content.replace("EXTERNAL_COMPONENT_PATH", external_components_path)
if shared_name is not None:
# _compile verifies the marked test compiles this content unmodified
request.node._shared_yaml_content = content
return content return content
@@ -270,218 +240,24 @@ async def write_yaml_config(
) -> AsyncGenerator[ConfigWriter]: ) -> AsyncGenerator[ConfigWriter]:
"""Write YAML configuration to a file.""" """Write YAML configuration to a file."""
# Get the test name for default filename # Get the test name for default filename
base_name = request.node.name.replace("test_", "").partition("[")[0] test_name = request.node.name
base_name = test_name.replace("test_", "").split("[")[0]
async def _write_config(content: str, filename: str | None = None) -> Path: async def _write_config(content: str, filename: str | None = None) -> Path:
if filename is None: if filename is None:
filename = f"{base_name}.yaml" filename = f"{base_name}.yaml"
config_path = integration_test_dir / filename config_path = integration_test_dir / filename
loop = asyncio.get_running_loop() loop = asyncio.get_running_loop()
await loop.run_in_executor(None, write_file, config_path, content) await loop.run_in_executor(None, config_path.write_text, content)
return config_path return config_path
yield _write_config yield _write_config
# Deliberately not CI-cached (ci.yml caches only platformio/ subpaths); stale
# dirs for a fixture are pruned when its content hash changes.
SHARED_BUILDS_ROOT = INTEGRATION_TESTS_ROOT / "builds"
# In the dir name (not just the hash) so pruning stays inside this checkout
_REPO_KEY = hashlib.sha256(str(REPO_ROOT).encode()).hexdigest()[:8]
# Give a contended shared build lock time for a full cold compile ahead of us
_SHARED_LOCK_TIMEOUT_S = 900
_SHARED_LOCK_POLL_S = 0.1
_SHARED_LOCK_REPORT_S = 30
# Reclaims dirs orphaned by fixture renames or deleted checkouts
_STALE_BUILD_MAX_AGE_S = 30 * 24 * 3600
# ELF path per shared build dir; constant once compiled, so resolve it only once
_shared_elf_paths: dict[Path, Path] = {}
# Dirs this process already swept; pruning is session-scoped work
_pruned_dirs: set[Path] = set()
def _shared_yaml_name(request: pytest.FixtureRequest) -> str | None:
"""Name passed to the shared_yaml marker, or None when unmarked."""
marker = request.node.get_closest_marker("shared_yaml")
if marker is None:
return None
# Exactly one \w+ positional arg: the name doubles as a build dir
# component, and CI test selection (script/helpers.py) parses the same shape
if (
len(marker.args) != 1
or marker.kwargs
or not re.fullmatch(r"\w+", str(marker.args[0]))
):
raise ValueError(
"shared_yaml marker requires exactly one \\w+ fixture name literal"
)
return marker.args[0]
def _shared_build_prefix(name: str) -> str:
return f"{name}-{_REPO_KEY}-"
@cache
def _shared_build_dir(name: str) -> Path:
"""Dir keyed by checkout and fixture source, before per-test injections."""
key = hashlib.sha256((FIXTURES_DIR / f"{name}.yaml").read_bytes()).hexdigest()[:16]
return SHARED_BUILDS_ROOT / (_shared_build_prefix(name) + key)
def _read_stamp(stamp: Path, shared_dir: Path) -> Path | None:
"""ELF path recorded by the last completed compile, or None."""
try:
text = stamp.read_text(encoding="utf-8").strip()
except FileNotFoundError:
return None
except OSError as err:
print(f"Cannot read {stamp}: {err}")
return None
if not text:
print(f"Ignoring empty stamp {stamp}")
return None
built = Path(text)
# Never trust a stamp pointing outside its own build dir as an unlink target
if shared_dir.resolve() in built.resolve().parents:
return built
print(f"Ignoring stamp {stamp} pointing outside {shared_dir}")
return None
def _unused_since(stale: Path, cutoff: float) -> bool:
"""Whether a build dir looks untouched since cutoff; unknown counts as used."""
# Newest of the .built stamp (rewritten by every completed compile) and the
# dir itself (freshened by a worker claiming the dir before locking)
newest: float | None = None
for probe in (stale / ".built", stale):
try:
mtime = probe.stat().st_mtime
except FileNotFoundError:
continue
except NotADirectoryError:
return True # a stray file where a dir should be; reclaimable
except OSError as err:
print(f"Cannot age-probe {stale}: {err}")
return False # unknown never authorizes deletion
newest = mtime if newest is None else max(newest, mtime)
return newest is not None and newest < cutoff
def _prune_stale_builds(name: str, keep: Path) -> None:
"""Remove outdated build dirs (blocking, run in executor): this checkout's
other dirs for the fixture, plus anything untouched for 30 days. Tolerates
other workers pruning the same dirs concurrently."""
cutoff = time.time() - _STALE_BUILD_MAX_AGE_S
prefix = _shared_build_prefix(name)
for stale in SHARED_BUILDS_ROOT.iterdir():
if stale == keep:
continue
same_fixture = stale.name.startswith(prefix)
if not same_fixture and not _unused_since(stale, cutoff):
continue
# Creating .lock bumps the dir mtime, so remember whether the re-probe
# under the lock can trust it
lock_preexisting = (stale / ".lock").exists()
try:
lock_file = (stale / ".lock").open("w")
except FileNotFoundError:
continue # pruned by another worker meanwhile
except NotADirectoryError:
print(f"Removing stray file {stale}")
stale.unlink(missing_ok=True)
continue
except OSError as err:
print(f"Cannot prune {stale}: {err}")
continue
with lock_file:
try:
fcntl.flock(lock_file.fileno(), fcntl.LOCK_EX | fcntl.LOCK_NB)
except BlockingIOError:
continue # still in use by another run
# Re-probe under the lock: a worker freshens its dir before
# locking, so a just-claimed dir no longer looks unused. A dir
# whose .lock we just created cannot be held by anyone, and our
# own open bumped its mtime, so its pre-open probe stands
if (
lock_preexisting
and not same_fixture
and not _unused_since(stale, cutoff)
):
continue
# rmtree tolerates races; a leftover partial tree only costs a
# rebuild, since the ELF is deleted before every compile
try:
rmtree(stale)
except OSError as err:
print(f"Failed to prune {stale}: {err}")
async def _run_esphome_compile(
config_path: Path, cwd: Path, env: dict[str, str]
) -> None:
"""Run `esphome compile`, retrying up to 3 times on a segfault."""
max_retries = 3
for attempt in range(max_retries):
# Compile using subprocess, inheriting stdout/stderr to show progress
proc = await asyncio.create_subprocess_exec(
sys.executable,
"-m",
"esphome",
"compile",
str(config_path),
cwd=cwd,
stdout=None, # Inherit stdout
stderr=None, # Inherit stderr
stdin=asyncio.subprocess.DEVNULL,
# Start in a new process group to isolate signal handling
start_new_session=True,
env=env,
close_fds=False,
)
await proc.wait()
if proc.returncode == 0:
break
if proc.returncode == -11 and attempt < max_retries - 1:
# Segfault (-11 = SIGSEGV), retry
print(
f"Compilation segfaulted (attempt {attempt + 1}/{max_retries}), retrying..."
)
await asyncio.sleep(1) # Brief pause before retry
continue
raise RuntimeError(
f"Failed to compile {config_path}, return code: {proc.returncode}. "
f"Run with 'pytest -s' to see compilation output."
)
def _resolve_compiled_binary(config_path: Path) -> Path:
"""Load the config to learn the compiled ELF path (blocking, run in executor)."""
CORE.reset() # Reset CORE state between test runs
CORE.config_path = config_path
config = esphome.config.read_config(
{"command": "compile", "config": str(config_path)}
)
if config is None:
raise RuntimeError(f"Failed to read config from {config_path}")
idedata = get_idedata(config)
binary_path = Path(idedata.firmware_elf_path)
if not binary_path.exists():
raise RuntimeError(f"Compiled binary not found at {binary_path}")
return binary_path
@pytest_asyncio.fixture @pytest_asyncio.fixture
async def compile_esphome( async def compile_esphome(
integration_test_dir: Path, integration_test_dir: Path,
shared_platformio_cache: Path, shared_platformio_cache: Path,
request: pytest.FixtureRequest,
) -> AsyncGenerator[CompileFunction]: ) -> AsyncGenerator[CompileFunction]:
"""Compile an ESPHome configuration and return the binary path.""" """Compile an ESPHome configuration and return the binary path."""
@@ -489,96 +265,66 @@ async def compile_esphome(
# Use the shared PlatformIO cache for faster compilation # Use the shared PlatformIO cache for faster compilation
# This avoids re-downloading dependencies for each test # This avoids re-downloading dependencies for each test
env = _get_platformio_env(shared_platformio_cache) env = _get_platformio_env(shared_platformio_cache)
# Retry compilation up to 3 times if we get a segfault
max_retries = 3
for attempt in range(max_retries):
# Compile using subprocess, inheriting stdout/stderr to show progress
proc = await asyncio.create_subprocess_exec(
sys.executable,
"-m",
"esphome",
"compile",
str(config_path),
cwd=integration_test_dir,
stdout=None, # Inherit stdout
stderr=None, # Inherit stderr
stdin=asyncio.subprocess.DEVNULL,
# Start in a new process group to isolate signal handling
start_new_session=True,
env=env,
close_fds=False,
)
await proc.wait()
if proc.returncode == 0:
# Success!
break
if proc.returncode == -11 and attempt < max_retries - 1:
# Segfault (-11 = SIGSEGV), retry
print(
f"Compilation segfaulted (attempt {attempt + 1}/{max_retries}), retrying..."
)
await asyncio.sleep(1) # Brief pause before retry
continue
# Other error or final retry
raise RuntimeError(
f"Failed to compile {config_path}, return code: {proc.returncode}. "
f"Run with 'pytest -s' to see compilation output."
)
# Load the config to get idedata (blocking call, must use executor)
loop = asyncio.get_running_loop() loop = asyncio.get_running_loop()
name = _shared_yaml_name(request) def _read_config_and_get_binary():
if name is None: CORE.reset() # Reset CORE state between test runs
await _run_esphome_compile(config_path, integration_test_dir, env) CORE.config_path = config_path
return await loop.run_in_executor( config = esphome.config.read_config(
None, _resolve_compiled_binary, config_path {"command": "compile", "config": str(config_path)}
) )
if config is None:
raise RuntimeError(f"Failed to read config from {config_path}")
# Shared fixture: build in a hash-keyed dir so tests sharing a config # Get the compiled binary path
# pay one full compile and later only a main.cpp (port) rebuild + relink idedata = get_idedata(config)
shared_dir = _shared_build_dir(name) return Path(idedata.firmware_elf_path)
shared_dir.mkdir(parents=True, exist_ok=True)
# Freshen the dir before locking so a concurrent age sweep, which binary_path = await loop.run_in_executor(None, _read_config_and_get_binary)
# re-probes under the lock, never reaps a dir a worker just claimed;
# if a peer reaped it already, the guarded lock open recreates it if not binary_path.exists():
with suppress(FileNotFoundError): raise RuntimeError(f"Compiled binary not found at {binary_path}")
os.utime(shared_dir)
if shared_dir not in _pruned_dirs: return binary_path
_pruned_dirs.add(shared_dir)
await loop.run_in_executor(None, _prune_stale_builds, name, shared_dir)
shared_config = shared_dir / f"{name}.yaml"
private_binary = integration_test_dir / f"{name}.elf"
content = await loop.run_in_executor(None, read_file, config_path)
if content != getattr(request.node, "_shared_yaml_content", None):
# The dir is keyed by the fixture source; a mutated config would be
# cached under a hash that does not describe it
raise RuntimeError(
"shared_yaml tests must compile the yaml_config content unmodified"
)
# flock serializes concurrent xdist workers; closing the fd releases it.
# Hand-rolled rather than filelock.FileLock: non-blocking retries keep
# the wait cancellable, while a blocking acquire in an executor thread
# would survive test cancellation holding the fd
try:
lock_file = (shared_dir / ".lock").open("w")
except FileNotFoundError:
# A peer run pruning divergent hashes reaped the dir between our
# mkdir and this open; recreate it and pay a full rebuild
shared_dir.mkdir(parents=True, exist_ok=True)
lock_file = (shared_dir / ".lock").open("w")
with lock_file:
start = time.monotonic()
last_report = start
while True:
try:
fcntl.flock(lock_file.fileno(), fcntl.LOCK_EX | fcntl.LOCK_NB)
break
except BlockingIOError:
now = time.monotonic()
if now - start > _SHARED_LOCK_TIMEOUT_S:
raise RuntimeError(
f"Timed out waiting for the {shared_dir} lock"
) from None
if now - last_report >= _SHARED_LOCK_REPORT_S:
last_report = now
print(
f"Waited {now - start:.0f}s for another worker's "
f"build of {shared_dir.name}"
)
await asyncio.sleep(_SHARED_LOCK_POLL_S)
# .built carries the ELF path of the last completed compile, so
# later workers skip the config re-read in _resolve_compiled_binary
stamp = shared_dir / ".built"
if (built := _shared_elf_paths.get(shared_dir)) is None:
built = await loop.run_in_executor(None, _read_stamp, stamp, shared_dir)
# Delete the ELF before compiling: whatever exists afterwards is
# this compile's output, so no staleness check is ever needed.
# With no usable stamp, sweep any leftover at the known layout
if built is not None:
built.unlink(missing_ok=True)
else:
# Layout-agnostic: ESPHOME_BUILD_PATH can move the build tree
for leftover in shared_dir.rglob("program"):
if leftover.is_file():
leftover.unlink()
await loop.run_in_executor(
None, write_file_if_changed, shared_config, content
)
await _run_esphome_compile(shared_config, shared_dir, env)
if built is None or not built.exists():
built = await loop.run_in_executor(
None, _resolve_compiled_binary, shared_config
)
_shared_elf_paths[shared_dir] = built
await loop.run_in_executor(None, write_file, stamp, str(built))
# Copy out before unlocking: another worker may relink firmware.elf
# while this test is still running its private copy
await loop.run_in_executor(None, shutil.copy2, built, private_binary)
return private_binary
yield _compile yield _compile
@@ -0,0 +1,43 @@
esphome:
name: api-decode-wire-types-test
host:
api:
logger:
level: DEBUG
switch:
- platform: template
name: "Wire Switch"
optimistic: true
output:
- platform: template
id: wire_dim
type: float
write_action:
- lambda: ""
light:
- platform: monochromatic
name: "Wire Light"
output: wire_dim
default_transition_length: 0s
effects:
- pulse:
name: Pulse
text:
- platform: template
name: "Wire Text"
optimistic: true
mode: text
min_length: 0
max_length: 255
number:
- platform: template
name: "Wire Number"
optimistic: true
min_value: -1000
max_value: 1000
step: 0.5
@@ -0,0 +1,11 @@
esphome:
name: api-empty-message-test
host:
api:
logger:
level: DEBUG
switch:
- platform: template
name: "Empty Message Switch"
optimistic: true
@@ -0,0 +1,58 @@
esphome:
name: api-encode-boundaries-test
# Top-level area fills DeviceInfoResponse.suggested_area (field 16, a two-byte tag)
area:
id: kitchen_area
name: Kitchen
on_boot:
- sensor.template.publish:
id: zero_then_value
state: 0.0
host:
api:
logger:
level: DEBUG
sensor:
- platform: template
name: "Zero Then Value"
id: zero_then_value
# Negative int32 takes the ten byte varint path
accuracy_decimals: -2
update_interval: never
text_sensor:
- platform: template
name: "Long Text"
id: long_text
update_interval: never
number:
- platform: template
name: "Negative Number"
optimistic: true
min_value: -1000
max_value: 1000
step: 0.5
initial_value: -123.5
select:
- platform: template
name: "Long Option Select"
optimistic: true
options:
- short
- "option-with-a-name-long-enough-that-its-length-prefix-needs-two-varint-bytes-when-the-list-entities-response-is-encoded-xxxxxxxxxx"
initial_option: short
button:
- platform: template
name: "Publish Values"
on_press:
- sensor.template.publish:
id: zero_then_value
state: 12.5
- text_sensor.template.publish:
id: long_text
state: !lambda return std::string(200, 'y');
@@ -0,0 +1,58 @@
esphome:
name: test-batch-window-filters
host:
api:
batch_delay: 0ms # Disable batching to receive all state updates
logger:
level: DEBUG
# Template sensor that we'll use to publish values
sensor:
- platform: template
name: "Source Sensor"
id: source_sensor
accuracy_decimals: 2
# Batch window filters (window_size == send_every) - use streaming filters
- platform: copy
source_id: source_sensor
name: "Min Sensor"
id: min_sensor
filters:
- min:
window_size: 5
send_every: 5
send_first_at: 1
- platform: copy
source_id: source_sensor
name: "Max Sensor"
id: max_sensor
filters:
- max:
window_size: 5
send_every: 5
send_first_at: 1
- platform: copy
source_id: source_sensor
name: "Moving Avg Sensor"
id: moving_avg_sensor
filters:
- sliding_window_moving_average:
window_size: 5
send_every: 5
send_first_at: 1
# Button to trigger publishing test values
button:
- platform: template
name: "Publish Values Button"
id: publish_button
on_press:
- lambda: |-
// Publish 10 values: 1.0, 2.0, ..., 10.0
for (int i = 1; i <= 10; i++) {
id(source_sensor).publish_state(float(i));
}
@@ -0,0 +1,111 @@
esphome:
name: uart-mock-modbus-cli-rw
host:
api:
logger:
level: VERBOSE
external_components:
- source:
type: local
path: EXTERNAL_COMPONENT_PATH
# Dummy uart entry to satisfy modbus's DEPENDENCIES = ["uart"]
# The actual UART bus used is the uart_mock component below
uart:
baud_rate: 115200
port: /dev/null
# Two virtual buses looped back to each other: the client's transmissions reach the server and the
# server's replies reach the client. auto_start so forwarding is active before the button fires.
uart_mock:
- id: virtual_uart_server
baud_rate: 9600
auto_start: true
debug:
on_tx:
- then:
- uart_mock.inject_rx:
id: virtual_uart_client
data: !lambda return data;
- id: virtual_uart_client
baud_rate: 9600
auto_start: true
debug:
on_tx:
- then:
- uart_mock.inject_rx:
id: virtual_uart_server
data: !lambda return data;
globals:
- id: stored_1
type: uint16_t
initial_value: "0"
modbus:
- uart_id: virtual_uart_server
id: virtual_modbus_server
role: server
- uart_id: virtual_uart_client
id: virtual_modbus_client
role: client
turnaround_time: 10ms
modbus_server:
- address: 1
modbus_id: virtual_modbus_server
registers:
# Writable + readable register: the read publishes what it returns, so the test can confirm the
# write half of the 0x17 ran before the read half (Modbus 6.17).
- address: 0x01
value_type: U_WORD
read_lambda: |-
id(srv_read_1).publish_state(id(stored_1));
return id(stored_1);
write_lambda: |-
id(stored_1) = x;
id(srv_write_1).publish_state(x);
return true;
# Read-only register, returned together with 0x01 by the 2-register read half.
- address: 0x02
value_type: U_WORD
read_lambda: return 0x00AA;
sensor:
# Server-side observations.
- platform: template
name: "srv_write_1"
id: srv_write_1
- platform: template
name: "srv_read_1"
id: srv_read_1
# Client-side read-back: the values the client's on_response received.
- platform: template
name: "client_read_0"
id: client_read_0
- platform: template
name: "client_read_1"
id: client_read_1
button:
- platform: template
name: "Start Scenario"
id: start_scenario_btn
on_press:
# FC 0x17: write reg 0x0001 = 0x1234, then read regs 0x0001..0x0002 back in the same transaction.
- modbus_client.read_write_multiple_registers:
address: 0x01
read_address: 0x0001
read_count: 2
write_address: 0x0001
values: [0x1234]
on_response:
then:
- lambda: |-
// values is the read-back block: reg 0x0001 (must be the just-written 0x1234) and reg 0x0002.
if (values.size() >= 2) {
id(client_read_0).publish_state(values[0]);
id(client_read_1).publish_state(values[1]);
}
@@ -0,0 +1,88 @@
esphome:
name: uart-mock-modbus-custom-pdu
host:
api:
logger:
level: VERBOSE
external_components:
- source:
type: local
path: EXTERNAL_COMPONENT_PATH
# Dummy uart entry to satisfy modbus's DEPENDENCIES = ["uart"]
# The actual UART bus used is the uart_mock component below
uart:
baud_rate: 115200
port: /dev/null
uart_mock:
- id: virtual_uart_server
baud_rate: 9600
auto_start: true
debug:
on_tx:
- then:
- uart_mock.inject_rx:
id: virtual_uart_controller
data: !lambda return data;
- id: virtual_uart_controller
baud_rate: 9600
auto_start: true
debug:
on_tx:
- then:
- uart_mock.inject_rx:
id: virtual_uart_server
data: !lambda return data;
modbus:
- uart_id: virtual_uart_server
id: virtual_modbus_server
role: server
- uart_id: virtual_uart_controller
id: virtual_modbus_controller
role: client
turnaround_time: 10ms
modbus_controller:
- address: 1
modbus_id: virtual_modbus_controller
id: modbus_controller_1
update_interval: 1s
modbus_server:
- address: 1
modbus_id: virtual_modbus_server
id: modbus_server_1
registers:
- address: 0x01
value_type: U_WORD
read_lambda: return 259;
sensor:
# Plain read to confirm the controller <-> server link is up.
- platform: modbus_controller
modbus_controller_id: modbus_controller_1
name: "plain_read"
address: 0x01
register_type: holding
value_type: U_WORD
# Custom PDU: read holding register 0x0001, count 1. The PDU is
# {function code, address hi, address lo, count hi, count lo}; the device
# address and CRC are added by the hub. The lambda parses the response payload
# (the register value, big-endian).
- platform: modbus_controller
modbus_controller_id: modbus_controller_1
name: "custom_read"
custom_pdu: [0x03, 0x00, 0x01, 0x00, 0x01]
lambda: |-
if (data.size() < 2) return {};
return (float) ((data[0] << 8) | data[1]);
button:
- platform: template
name: "Start Scenario"
id: start_scenario_btn
# This test does not have anything to start (mock is autostart)
@@ -0,0 +1,106 @@
esphome:
name: uart-mock-modbus-dep-buffer
host:
api:
logger:
level: VERBOSE
external_components:
- source:
type: local
path: EXTERNAL_COMPONENT_PATH
# Dummy uart entry to satisfy modbus's DEPENDENCIES = ["uart"]
# The actual UART bus used is the uart_mock component below
uart:
baud_rate: 115200
port: /dev/null
uart_mock:
- id: virtual_uart_server
baud_rate: 9600
auto_start: true
debug:
on_tx:
- then:
- uart_mock.inject_rx:
id: virtual_uart_controller
data: !lambda return data;
- id: virtual_uart_controller
baud_rate: 9600
auto_start: true
debug:
on_tx:
- then:
- uart_mock.inject_rx:
id: virtual_uart_server
data: !lambda return data;
globals:
- id: reg10
type: uint16_t
initial_value: "0"
modbus:
- uart_id: virtual_uart_server
id: virtual_modbus_server
role: server
- uart_id: virtual_uart_controller
id: virtual_modbus_controller
role: client
turnaround_time: 10ms
modbus_controller:
- address: 1
modbus_id: virtual_modbus_controller
id: modbus_controller_1
update_interval: 1s
modbus_server:
- address: 1
modbus_id: virtual_modbus_server
id: modbus_server_1
registers:
- address: 0x10
value_type: U_WORD
read_lambda: return id(reg10);
write_lambda: |-
id(reg10) = x;
return true;
# A number whose write_lambda uses the DEPRECATED buffer parameter (fills `payload` with a legacy raw
# frame as words: device address + function code + data) instead of the new item->write_* API. The write
# must still land with its legacy semantics, and the one-time deprecation warning must fire only once per
# entity no matter how many writes happen.
number:
- platform: modbus_controller
modbus_controller_id: modbus_controller_1
name: "buf_number"
id: buf_number
address: 0x10
register_type: holding
value_type: U_WORD
min_value: 0
max_value: 1000
step: 1
write_lambda: |-
// Legacy raw frame as words: [addr 0x01 | fc 0x06], register 0x0010, value.
payload.push_back(0x0106);
payload.push_back(0x0010);
payload.push_back((uint16_t) x);
return {};
# Reports the server-side register so the test can observe that the deprecated buffer write landed.
sensor:
- platform: template
name: "written_value"
id: written_value
update_interval: 0.5s
lambda: "return id(reg10);"
button:
- platform: template
name: "Start Scenario"
id: start_scenario_btn
# The test drives the writes via number_command; the mock is autostart.
@@ -0,0 +1,95 @@
esphome:
name: uart-mock-modbus-lambda-invert
host:
api:
logger:
level: VERBOSE
external_components:
- source:
type: local
path: EXTERNAL_COMPONENT_PATH
# Dummy uart entry to satisfy modbus's DEPENDENCIES = ["uart"]
# The actual UART bus used is the uart_mock component below
uart:
baud_rate: 115200
port: /dev/null
uart_mock:
- id: virtual_uart_server
baud_rate: 9600
auto_start: true
debug:
on_tx:
- then:
- uart_mock.inject_rx:
id: virtual_uart_controller
data: !lambda return data;
- id: virtual_uart_controller
baud_rate: 9600
auto_start: true
debug:
on_tx:
- then:
- uart_mock.inject_rx:
id: virtual_uart_server
data: !lambda return data;
globals:
- id: reg40
type: uint16_t
initial_value: "5"
modbus:
- uart_id: virtual_uart_server
id: virtual_modbus_server
role: server
- uart_id: virtual_uart_controller
id: virtual_modbus_controller
role: client
turnaround_time: 10ms
modbus_controller:
- address: 1
modbus_id: virtual_modbus_controller
id: modbus_controller_1
update_interval: 1s
modbus_server:
- address: 1
modbus_id: virtual_modbus_server
id: modbus_server_1
registers:
- address: 0x40
value_type: U_WORD
read_lambda: return id(reg40);
write_lambda: id(reg40) = x; return true;
# An active-low holding switch: the write_lambda inverts the wire value, but the entity must still
# report the REQUESTED state. assumed_state keeps the register unpolled, so the published state comes
# only from write_state() - turning ON writes 0x0000 yet the switch shows ON.
switch:
- platform: modbus_controller
modbus_controller_id: modbus_controller_1
name: "invert_switch"
register_type: holding
address: 0x40
assumed_state: true
write_lambda: |-
return !x;
sensor:
- platform: modbus_controller
modbus_controller_id: modbus_controller_1
name: "reg_40"
address: 0x40
register_type: holding
value_type: U_WORD
button:
- platform: template
name: "Start Scenario"
id: start_scenario_btn
# This test does not have anything to start (mock is autostart)
@@ -0,0 +1,97 @@
esphome:
name: uart-mock-modbus-lambda-write
host:
api:
logger:
level: VERBOSE
external_components:
- source:
type: local
path: EXTERNAL_COMPONENT_PATH
# Dummy uart entry to satisfy modbus's DEPENDENCIES = ["uart"]
# The actual UART bus used is the uart_mock component below
uart:
baud_rate: 115200
port: /dev/null
uart_mock:
- id: virtual_uart_server
baud_rate: 9600
auto_start: true
debug:
on_tx:
- then:
- uart_mock.inject_rx:
id: virtual_uart_controller
data: !lambda return data;
- id: virtual_uart_controller
baud_rate: 9600
auto_start: true
debug:
on_tx:
- then:
- uart_mock.inject_rx:
id: virtual_uart_server
data: !lambda return data;
globals:
- id: reg30
type: uint16_t
initial_value: "0"
modbus:
- uart_id: virtual_uart_server
id: virtual_modbus_server
role: server
- uart_id: virtual_uart_controller
id: virtual_modbus_controller
role: client
turnaround_time: 10ms
modbus_controller:
- address: 1
modbus_id: virtual_modbus_controller
id: modbus_controller_1
update_interval: 1s
modbus_server:
- address: 1
modbus_id: virtual_modbus_server
id: modbus_server_1
registers:
- address: 0x30
value_type: U_WORD
read_lambda: return id(reg30);
write_lambda: id(reg30) = x; return true;
# A COIL-type switch (assumed_state, write-only) whose write_lambda ignores its own coil type and instead
# drives a HOLDING-REGISTER write on the mock server through the entity itself: `item` IS the command, so
# item->write_single_register() sends a register write from a coil entity (cross-type). Returning nothing
# (an empty optional) tells the write path the lambda already dispatched the frame - no default coil write.
switch:
- platform: modbus_controller
modbus_controller_id: modbus_controller_1
name: "cross_switch"
register_type: coil
address: 0x00
assumed_state: true
write_lambda: |-
item->write_single_register(0x30, x ? 1234 : 0);
return {};
sensor:
- platform: modbus_controller
modbus_controller_id: modbus_controller_1
name: "reg_30"
address: 0x30
register_type: holding
value_type: U_WORD
button:
- platform: template
name: "Start Scenario"
id: start_scenario_btn
# This test does not have anything to start (mock is autostart)
@@ -1,233 +0,0 @@
esphome:
name: uart-mock-modbus-loopback
host:
api:
logger:
level: VERBOSE
external_components:
- source:
type: local
path: EXTERNAL_COMPONENT_PATH
# Dummy uart entry to satisfy modbus's DEPENDENCIES = ["uart"]
# The actual UART bus used is the uart_mock component below
uart:
baud_rate: 115200
port: /dev/null
# Shared loopback fixture (see the shared_yaml markers in the test file);
# register spaces are disjoint so each test only observes its own entities.
uart_mock:
- id: virtual_uart_server
baud_rate: 9600
auto_start: true
debug:
on_tx:
- then:
- uart_mock.inject_rx:
id: virtual_uart_controller
data: !lambda return data;
- id: virtual_uart_controller
baud_rate: 9600
auto_start: true
debug:
on_tx:
- then:
- uart_mock.inject_rx:
id: virtual_uart_server
data: !lambda return data;
globals:
- id: reg10
type: uint16_t
initial_value: "100"
- id: reg11
type: uint16_t
initial_value: "200"
- id: reg12
type: uint16_t
initial_value: "300"
- id: reg13
type: uint16_t
initial_value: "0xABCD"
- id: reg30
type: uint16_t
initial_value: "0"
- id: reg40
type: uint16_t
initial_value: "5"
- id: reg50
type: uint16_t
initial_value: "0"
modbus:
- uart_id: virtual_uart_server
id: virtual_modbus_server
role: server
- uart_id: virtual_uart_controller
id: virtual_modbus_controller
role: client
turnaround_time: 10ms
modbus_controller:
- address: 1
modbus_id: virtual_modbus_controller
id: modbus_controller_1
update_interval: 1s
modbus_server:
- address: 1
modbus_id: virtual_modbus_server
registers:
- address: 0x01
value_type: U_WORD
read_lambda: return 259;
- address: 0x10
value_type: U_WORD
read_lambda: return id(reg10);
write_lambda: id(reg10) = x; return true;
- address: 0x11
value_type: U_WORD
read_lambda: return id(reg11);
write_lambda: id(reg11) = x; return true;
- address: 0x12
value_type: U_WORD
read_lambda: return id(reg12);
write_lambda: id(reg12) = x; return true;
- address: 0x13
value_type: U_WORD
read_lambda: return id(reg13);
- address: 0x30
value_type: U_WORD
read_lambda: return id(reg30);
write_lambda: id(reg30) = x; return true;
- address: 0x40
value_type: U_WORD
read_lambda: return id(reg40);
write_lambda: id(reg40) = x; return true;
- address: 0x50
value_type: U_WORD
read_lambda: return id(reg50);
write_lambda: id(reg50) = x; return true;
# Byte-based offset: 2 bytes -> register 0x11 (the old code folded it in as a
# register count, hitting 0x12). assumed_state keeps the switch write-only.
switch:
- platform: modbus_controller
modbus_controller_id: modbus_controller_1
name: "offset_switch"
register_type: holding
address: 0x10
offset: 2
assumed_state: true
# Reading switch, byte offset 6 -> register 0x13; the pre-fix resolution (0x16)
# would draw ILLEGAL_DATA_ADDRESS and never publish.
- platform: modbus_controller
modbus_controller_id: modbus_controller_1
name: "read_offset_switch"
register_type: holding
address: 0x10
offset: 6
bitmask: 0x1
# Coil switch whose write_lambda dispatches a holding-register write via `item`;
# returning an empty optional suppresses the default coil write.
- platform: modbus_controller
modbus_controller_id: modbus_controller_1
name: "cross_switch"
register_type: coil
address: 0x00
assumed_state: true
write_lambda: |-
item->write_single_register(0x30, x ? 1234 : 0);
return {};
# Active-low: the write_lambda inverts the wire value but the entity must still
# report the requested state (assumed_state keeps the register unpolled).
- platform: modbus_controller
modbus_controller_id: modbus_controller_1
name: "invert_switch"
register_type: holding
address: 0x40
assumed_state: true
write_lambda: |-
return !x;
# Uses the deprecated buffer parameter (legacy raw frame as words); the write
# must land and the deprecation warning must fire only once per entity.
number:
- platform: modbus_controller
modbus_controller_id: modbus_controller_1
name: "buf_number"
id: buf_number
address: 0x50
register_type: holding
value_type: U_WORD
min_value: 0
max_value: 1000
step: 1
write_lambda: |-
// Legacy raw frame as words: [addr 0x01 | fc 0x06], register 0x0050, value.
payload.push_back(0x0106);
payload.push_back(0x0050);
payload.push_back((uint16_t) x);
return {};
sensor:
- platform: modbus_controller
modbus_controller_id: modbus_controller_1
name: "plain_read"
address: 0x01
register_type: holding
value_type: U_WORD
# Custom PDU: read holding register 0x0001; device address and CRC are added
# by the hub. The lambda parses the big-endian register value.
- platform: modbus_controller
modbus_controller_id: modbus_controller_1
name: "custom_read"
custom_pdu: [0x03, 0x00, 0x01, 0x00, 0x01]
lambda: |-
if (data.size() < 2) return {};
return (float) ((data[0] << 8) | data[1]);
- platform: modbus_controller
modbus_controller_id: modbus_controller_1
name: "reg_10"
address: 0x10
register_type: holding
value_type: U_WORD
- platform: modbus_controller
modbus_controller_id: modbus_controller_1
name: "reg_11"
address: 0x11
register_type: holding
value_type: U_WORD
- platform: modbus_controller
modbus_controller_id: modbus_controller_1
name: "reg_12"
address: 0x12
register_type: holding
value_type: U_WORD
- platform: modbus_controller
modbus_controller_id: modbus_controller_1
name: "reg_30"
address: 0x30
register_type: holding
value_type: U_WORD
- platform: modbus_controller
modbus_controller_id: modbus_controller_1
name: "reg_40"
address: 0x40
register_type: holding
value_type: U_WORD
# Reports the server-side register so the test can observe that the deprecated buffer write landed.
- platform: template
name: "written_value"
id: written_value
update_interval: 0.5s
lambda: "return id(reg50);"
button:
- platform: template
name: "Start Scenario"
id: start_scenario_btn
# Nothing to start (mock is autostart); tests drive entities directly
@@ -0,0 +1,138 @@
esphome:
name: uart-mock-modbus-reg-offset
host:
api:
logger:
level: VERBOSE
external_components:
- source:
type: local
path: EXTERNAL_COMPONENT_PATH
# Dummy uart entry to satisfy modbus's DEPENDENCIES = ["uart"]
# The actual UART bus used is the uart_mock component below
uart:
baud_rate: 115200
port: /dev/null
uart_mock:
- id: virtual_uart_server
baud_rate: 9600
auto_start: true
debug:
on_tx:
- then:
- uart_mock.inject_rx:
id: virtual_uart_controller
data: !lambda return data;
- id: virtual_uart_controller
baud_rate: 9600
auto_start: true
debug:
on_tx:
- then:
- uart_mock.inject_rx:
id: virtual_uart_server
data: !lambda return data;
globals:
- id: reg10
type: uint16_t
initial_value: "100"
- id: reg11
type: uint16_t
initial_value: "200"
- id: reg12
type: uint16_t
initial_value: "300"
- id: reg13
type: uint16_t
initial_value: "0xABCD"
modbus:
- uart_id: virtual_uart_server
id: virtual_modbus_server
role: server
- uart_id: virtual_uart_controller
id: virtual_modbus_controller
role: client
turnaround_time: 10ms
modbus_controller:
- address: 1
modbus_id: virtual_modbus_controller
id: modbus_controller_1
update_interval: 1s
modbus_server:
- address: 1
modbus_id: virtual_modbus_server
id: modbus_server_1
registers:
- address: 0x10
value_type: U_WORD
read_lambda: return id(reg10);
write_lambda: id(reg10) = x; return true;
- address: 0x11
value_type: U_WORD
read_lambda: return id(reg11);
write_lambda: id(reg11) = x; return true;
- address: 0x12
value_type: U_WORD
read_lambda: return id(reg12);
write_lambda: id(reg12) = x; return true;
- address: 0x13
value_type: U_WORD
read_lambda: return id(reg13);
write_lambda: id(reg13) = x; return true;
# A holding-register switch at 0x10 with a 2-BYTE offset. offset is byte-based, so the write must target
# register 0x10 + 2/2 = 0x11. The old (pre-fix) behavior folded offset into the address as a register
# count, hitting 0x12 instead. assumed_state keeps the switch write-only so it does not read any register.
switch:
- platform: modbus_controller
modbus_controller_id: modbus_controller_1
name: "offset_switch"
register_type: holding
address: 0x10
offset: 2
assumed_state: true
# A holding-register switch that READS its state. Byte offset 6 -> register 0x10 + 6/2 = 0x13. Post-fix
# the switch itself resolves to 0x13 (the even byte offset folds into the address as whole registers) and
# joins the 0x10..0x13 range, so no separate 0x13 sensor is needed. Pre-fix the whole byte offset folds
# into the address (0x16), where the server answers ILLEGAL_DATA_ADDRESS and the switch never publishes.
- platform: modbus_controller
modbus_controller_id: modbus_controller_1
name: "read_offset_switch"
register_type: holding
address: 0x10
offset: 6
bitmask: 0x1
sensor:
- platform: modbus_controller
modbus_controller_id: modbus_controller_1
name: "reg_10"
address: 0x10
register_type: holding
value_type: U_WORD
- platform: modbus_controller
modbus_controller_id: modbus_controller_1
name: "reg_11"
address: 0x11
register_type: holding
value_type: U_WORD
- platform: modbus_controller
modbus_controller_id: modbus_controller_1
name: "reg_12"
address: 0x12
register_type: holding
value_type: U_WORD
button:
- platform: template
name: "Start Scenario"
id: start_scenario_btn
# This test does not have anything to start (mock is autostart)
@@ -0,0 +1,124 @@
esphome:
name: uart-mock-modbus-server-test
host:
api:
logger:
level: VERBOSE
external_components:
- source:
type: local
path: EXTERNAL_COMPONENT_PATH
# Dummy uart entry to satisfy modbus's DEPENDENCIES = ["uart"]
# The actual UART bus used is the uart_mock component below
uart:
baud_rate: 115200
port: /dev/null
uart_mock:
- id: virtual_uart_dev
baud_rate: 9600
rx_full_threshold: 120
rx_timeout: 2
auto_start: false
debug:
injections:
- delay: 100ms
inject_rx: [0x01, 0x03, 0x00, 0x03, 0x00, 0x01, 0x74, 0x0A] # Read holding register 3 on device 1 (basic_read)
- delay: 100ms
# Read holding register 7 on device 2
# Reply from device 2
# Read holding register 5 on device 1 (read_after_peer_response)
inject_rx:
[
0x02,
0x03,
0x00,
0x07,
0x00,
0x01,
0x35,
0xF8,
0x02,
0x03,
0x02,
0x00,
0xF0,
0xFC,
0x00,
0x01,
0x03,
0x00,
0x05,
0x00,
0x01,
0x94,
0x0B,
]
- delay: 100ms
inject_rx: [0x02, 0x03, 0x00, 0x07, 0x00, 0x01, 0x35, 0xF8] # Read holding register 7 on device 2, with no response
- delay: 100ms
# Read holding register 7 on device 2, with no response
# Read holding register A on device 1 (read_after_peer_timeout)
inject_rx:
[
0x02,
0x03,
0x00,
0x07,
0x00,
0x01,
0x35,
0xF8,
0x01,
0x03,
0x00,
0x0A,
0x00,
0x01,
0xA4,
0x08,
]
modbus:
uart_id: virtual_uart_dev
role: server
modbus_server:
- address: 1
registers:
- address: 0x03
value_type: U_WORD
read_lambda: |-
id(basic_read).publish_state(1);
return 1;
- address: 0x05
value_type: U_WORD
read_lambda: |-
id(read_after_peer_response).publish_state(1);
return 1;
- address: 0x0A
value_type: U_WORD
read_lambda: |-
id(read_after_peer_timeout).publish_state(1);
return 1;
sensor:
- platform: template
name: "basic_read"
id: basic_read
- platform: template
name: "read_after_peer_response"
id: read_after_peer_response
- platform: template
name: "read_after_peer_timeout"
id: read_after_peer_timeout
button:
- platform: template
name: "Start Scenario"
id: start_scenario_btn
on_press:
- lambda: "id(virtual_uart_dev).start_scenario();"
@@ -0,0 +1,203 @@
esphome:
name: uart-mock-modbus-server-contro
host:
api:
logger:
level: VERBOSE
external_components:
- source:
type: local
path: EXTERNAL_COMPONENT_PATH
# Dummy uart entry to satisfy modbus's DEPENDENCIES = ["uart"]
# The actual UART bus used is the uart_mock component below
uart:
baud_rate: 115200
port: /dev/null
uart_mock:
- id: virtual_uart_server
baud_rate: 9600
# auto_start must be true for loopback fixtures: the modbus controller
# polls on its update_interval immediately at boot, so the uart_mock
# forwarding must already be active or early requests are lost and
# generate modbus warnings.
auto_start: true
debug:
on_tx:
- then:
- uart_mock.inject_rx:
id: virtual_uart_controller
data: !lambda return data;
- id: virtual_uart_controller
baud_rate: 9600
auto_start: true # See comment on virtual_uart_server above
debug:
on_tx:
- then:
- uart_mock.inject_rx:
id: virtual_uart_server
data: !lambda return data;
modbus:
- uart_id: virtual_uart_server
id: virtual_modbus_server
role: server
- uart_id: virtual_uart_controller
id: virtual_modbus_controller
role: client
turnaround_time: 10ms
modbus_controller:
- address: 1
modbus_id: virtual_modbus_controller
id: modbus_controller_1
update_interval: 1s
modbus_server:
- address: 1
modbus_id: virtual_modbus_server
id: modbus_server_1
registers:
- address: 0x01
value_type: U_WORD
read_lambda: return 99;
- address: 0x02
value_type: U_WORD_S
read_lambda: return 4660;
- address: 0x03
value_type: S_WORD
read_lambda: return -99;
- address: 0x04
value_type: S_WORD_S
read_lambda: return -2;
- address: 0x05
value_type: U_DWORD
read_lambda: return 16909060;
- address: 0x08
value_type: S_DWORD
read_lambda: return -16909060;
- address: 0x0B
value_type: U_DWORD_R
read_lambda: return 67305985;
- address: 0x0E
value_type: S_DWORD_R
read_lambda: return -67305985;
- address: 0x11
value_type: U_QWORD
read_lambda: return 72623859790382856;
- address: 0x16
value_type: S_QWORD
read_lambda: return -72623859790382856;
- address: 0x1B
value_type: U_QWORD_R
read_lambda: return 578437695752307201;
- address: 0x20
value_type: S_QWORD_R
read_lambda: return -578437695752307201;
- address: 0x25
value_type: FP32
read_lambda: return 3.14;
- address: 0x28
value_type: FP32_R
read_lambda: return 3.14;
sensor:
- platform: modbus_controller
modbus_controller_id: modbus_controller_1
name: "reg_u_word"
address: 0x01
register_type: holding
value_type: U_WORD
- platform: modbus_controller
modbus_controller_id: modbus_controller_1
name: "reg_u_word_s"
address: 0x02
register_type: holding
value_type: U_WORD_S
- platform: modbus_controller
modbus_controller_id: modbus_controller_1
name: "reg_u_word_s_raw"
address: 0x02
register_type: holding
value_type: U_WORD
- platform: modbus_controller
modbus_controller_id: modbus_controller_1
name: "reg_s_word"
address: 0x03
register_type: holding
value_type: S_WORD
- platform: modbus_controller
modbus_controller_id: modbus_controller_1
name: "reg_s_word_s"
address: 0x04
register_type: holding
value_type: S_WORD_S
- platform: modbus_controller
modbus_controller_id: modbus_controller_1
name: "reg_u_dword"
address: 0x05
register_type: holding
value_type: U_DWORD
- platform: modbus_controller
modbus_controller_id: modbus_controller_1
name: "reg_s_dword"
address: 0x08
register_type: holding
value_type: S_DWORD
- platform: modbus_controller
modbus_controller_id: modbus_controller_1
name: "reg_u_dword_r"
address: 0x0B
register_type: holding
value_type: U_DWORD_R
- platform: modbus_controller
modbus_controller_id: modbus_controller_1
name: "reg_s_dword_r"
address: 0x0E
register_type: holding
value_type: S_DWORD_R
- platform: modbus_controller
modbus_controller_id: modbus_controller_1
name: "reg_u_qword"
address: 0x11
register_type: holding
value_type: U_QWORD
- platform: modbus_controller
modbus_controller_id: modbus_controller_1
name: "reg_s_qword"
address: 0x16
register_type: holding
value_type: S_QWORD
- platform: modbus_controller
modbus_controller_id: modbus_controller_1
name: "reg_u_qword_r"
address: 0x1B
register_type: holding
value_type: U_QWORD_R
- platform: modbus_controller
modbus_controller_id: modbus_controller_1
name: "reg_s_qword_r"
address: 0x20
register_type: holding
value_type: S_QWORD_R
- platform: modbus_controller
modbus_controller_id: modbus_controller_1
name: "reg_fp32"
address: 0x25
register_type: holding
value_type: FP32
- platform: modbus_controller
modbus_controller_id: modbus_controller_1
name: "reg_fp32_r"
address: 0x28
register_type: holding
value_type: FP32_R
button:
- platform: template
name: "Start Scenario"
id: start_scenario_btn
# This test does not have anything to start (mock is autostart)
@@ -0,0 +1,147 @@
esphome:
name: uart-mock-modbus-srv-bits
host:
api:
logger:
level: VERBOSE
external_components:
- source:
type: local
path: EXTERNAL_COMPONENT_PATH
# Dummy uart entry to satisfy modbus's DEPENDENCIES = ["uart"]
# The actual UART bus used is the uart_mock component below
uart:
baud_rate: 115200
port: /dev/null
uart_mock:
- id: virtual_uart_server
baud_rate: 9600
# auto_start must be true for loopback fixtures: the modbus controller
# polls on its update_interval immediately at boot, so the uart_mock
# forwarding must already be active or early requests are lost and
# generate modbus warnings.
auto_start: true
debug:
on_tx:
- then:
- uart_mock.inject_rx:
id: virtual_uart_controller
data: !lambda return data;
- id: virtual_uart_controller
baud_rate: 9600
auto_start: true # See comment on virtual_uart_server above
debug:
on_tx:
- then:
- uart_mock.inject_rx:
id: virtual_uart_server
data: !lambda return data;
globals:
- id: stored_bit_2
type: bool
initial_value: "false"
- id: stored_bit_3
type: bool
initial_value: "true"
modbus:
- uart_id: virtual_uart_server
id: virtual_modbus_server
role: server
- uart_id: virtual_uart_controller
id: virtual_modbus_controller
role: client
turnaround_time: 10ms
modbus_controller:
- address: 1
modbus_id: virtual_modbus_controller
update_interval: 1s
id: modbus_controller_1
modbus_server:
- address: 1
modbus_id: virtual_modbus_server
id: modbus_server_1
bits:
- address: 0x00
read_lambda: return true;
- address: 0x01
read_lambda: return false;
- address: 0x02
read_lambda: return id(stored_bit_2);
write_lambda: id(stored_bit_2) = x; return true;
- address: 0x03
read_lambda: return id(stored_bit_3);
write_lambda: id(stored_bit_3) = x; return true;
# The same four bits are read both as coils (FC 0x01) and as discrete inputs
# (FC 0x02): the server serves both from one shared bit table, so the two
# views must always agree.
binary_sensor:
- platform: modbus_controller
modbus_controller_id: modbus_controller_1
name: "bit_coil_0"
address: 0x00
register_type: coil
- platform: modbus_controller
modbus_controller_id: modbus_controller_1
name: "bit_coil_1"
address: 0x01
register_type: coil
- platform: modbus_controller
modbus_controller_id: modbus_controller_1
name: "bit_coil_2"
address: 0x02
register_type: coil
- platform: modbus_controller
modbus_controller_id: modbus_controller_1
name: "bit_coil_3"
address: 0x03
register_type: coil
- platform: modbus_controller
modbus_controller_id: modbus_controller_1
name: "bit_di_0"
address: 0x00
register_type: discrete_input
- platform: modbus_controller
modbus_controller_id: modbus_controller_1
name: "bit_di_1"
address: 0x01
register_type: discrete_input
- platform: modbus_controller
modbus_controller_id: modbus_controller_1
name: "bit_di_2"
address: 0x02
register_type: discrete_input
- platform: modbus_controller
modbus_controller_id: modbus_controller_1
name: "bit_di_3"
address: 0x03
register_type: discrete_input
# write_bit_2 uses the single-coil write (FC 0x05); write_bit_3 opts into the
# multiple-coils write (FC 0x0F) so both server write paths are exercised.
switch:
- platform: modbus_controller
modbus_controller_id: modbus_controller_1
name: "write_bit_2"
address: 0x02
register_type: coil
- platform: modbus_controller
modbus_controller_id: modbus_controller_1
name: "write_bit_3"
address: 0x03
register_type: coil
use_write_multiple: true
button:
- platform: template
name: "Start Scenario"
id: start_scenario_btn
# This test does not have anything to start (mock is autostart)
@@ -0,0 +1,116 @@
esphome:
name: uart-mock-modbus-server-mult
host:
api:
logger:
level: VERBOSE
external_components:
- source:
type: local
path: EXTERNAL_COMPONENT_PATH
# Dummy uart entry to satisfy modbus's DEPENDENCIES = ["uart"]
# The actual UART bus used is the uart_mock component below
uart:
baud_rate: 115200
port: /dev/null
uart_mock:
- id: virtual_uart_server
baud_rate: 9600
# auto_start must be true for loopback fixtures: the modbus controller
# polls on its update_interval immediately at boot, so the uart_mock
# forwarding must already be active or early requests are lost and
# generate modbus warnings.
auto_start: true
debug:
on_tx:
- then:
- uart_mock.inject_rx:
id: virtual_uart_controller
data: !lambda return data;
- uart_mock.inject_rx:
id: virtual_uart_server_2
data: !lambda return data;
- id: virtual_uart_server_2
baud_rate: 9600
auto_start: true # See comment on virtual_uart_server above
debug:
on_tx:
- then:
- uart_mock.inject_rx:
id: virtual_uart_server
data: !lambda return data;
- uart_mock.inject_rx:
id: virtual_uart_controller
data: !lambda return data;
- id: virtual_uart_controller
baud_rate: 9600
auto_start: true # See comment on virtual_uart_server above
debug:
on_tx:
- then:
- uart_mock.inject_rx:
id: virtual_uart_server
data: !lambda return data;
- uart_mock.inject_rx:
id: virtual_uart_server_2
data: !lambda return data;
modbus:
- uart_id: virtual_uart_server
id: virtual_modbus_server
role: server
- uart_id: virtual_uart_server_2
id: virtual_modbus_server_2
role: server
- uart_id: virtual_uart_controller
id: virtual_modbus_client
role: client
turnaround_time: 10ms
modbus_controller:
- address: 1
modbus_id: virtual_modbus_client
update_interval: 1s
id: modbus_controller_1
- address: 2
modbus_id: virtual_modbus_client
update_interval: 1s
id: modbus_controller_2
modbus_server:
- address: 1
modbus_id: virtual_modbus_server
registers:
- address: 0x01
value_type: U_WORD
read_lambda: return 919;
- address: 2
modbus_id: virtual_modbus_server_2
registers:
- address: 0x01
value_type: U_WORD
read_lambda: return 929;
sensor:
- platform: modbus_controller
modbus_controller_id: modbus_controller_1
name: "reg_u_word"
address: 0x01
register_type: holding
value_type: U_WORD
- platform: modbus_controller
modbus_controller_id: modbus_controller_2
name: "reg_u_word_2"
address: 0x01
register_type: holding
value_type: U_WORD
button:
- platform: template
name: "Start Scenario"
id: start_scenario_btn
# This test does not have anything to start (mock is autostart)
@@ -1,5 +1,5 @@
esphome: esphome:
name: uart-mock-modbus-mesh name: uart-mock-modbus-srv-write
host: host:
api: api:
@@ -17,14 +17,13 @@ uart:
baud_rate: 115200 baud_rate: 115200
port: /dev/null port: /dev/null
# Shared 3-bus mesh (see the shared_yaml markers): addr 1 = typed registers
# backed by writable globals, addr 5 = the read/write 0x17 target, addr 2/3/6
# on the second server hub. auto_start everywhere: the controller polls at
# boot, so the forwarding must already be live or early requests generate warnings.
# Every test presses Start Scenario, so all merged actions fire in every test.
uart_mock: uart_mock:
- id: virtual_uart_server - id: virtual_uart_server
baud_rate: 9600 baud_rate: 9600
# auto_start must be true for loopback fixtures: the modbus controller
# polls on its update_interval immediately at boot, so the uart_mock
# forwarding must already be active or early requests are lost and
# generate modbus warnings.
auto_start: true auto_start: true
debug: debug:
on_tx: on_tx:
@@ -32,120 +31,79 @@ uart_mock:
- uart_mock.inject_rx: - uart_mock.inject_rx:
id: virtual_uart_controller id: virtual_uart_controller
data: !lambda return data; data: !lambda return data;
- uart_mock.inject_rx:
id: virtual_uart_server_2
data: !lambda return data;
- id: virtual_uart_server_2
baud_rate: 9600
auto_start: true
debug:
on_tx:
- then:
- uart_mock.inject_rx:
id: virtual_uart_server
data: !lambda return data;
- uart_mock.inject_rx:
id: virtual_uart_controller
data: !lambda return data;
- id: virtual_uart_controller - id: virtual_uart_controller
baud_rate: 9600 baud_rate: 9600
auto_start: true auto_start: true # See comment on virtual_uart_server above
debug: debug:
on_tx: on_tx:
- then: - then:
- uart_mock.inject_rx: - uart_mock.inject_rx:
id: virtual_uart_server id: virtual_uart_server
data: !lambda return data; data: !lambda return data;
- uart_mock.inject_rx:
id: virtual_uart_server_2
data: !lambda return data;
globals: globals:
- id: stored_1
type: uint16_t
initial_value: "0"
- id: stored_u_word - id: stored_u_word
type: uint16_t type: uint16_t
initial_value: "99" initial_value: "11"
- id: stored_u_word_s - id: stored_u_word_s
type: uint16_t type: uint16_t
initial_value: "4660" initial_value: "4660"
- id: stored_s_word - id: stored_s_word
type: int16_t type: int16_t
initial_value: "-99" initial_value: "-11"
- id: stored_s_word_s - id: stored_s_word_s
type: int16_t type: int16_t
initial_value: "-2" initial_value: "-2"
- id: stored_u_dword - id: stored_u_dword
type: uint32_t type: uint32_t
initial_value: "16909060" initial_value: "1001"
- id: stored_s_dword - id: stored_s_dword
type: int32_t type: int32_t
initial_value: "-16909060" initial_value: "-1001"
- id: stored_u_dword_r - id: stored_u_dword_r
type: uint32_t type: uint32_t
initial_value: "67305985" initial_value: "3003"
- id: stored_s_dword_r - id: stored_s_dword_r
type: int32_t type: int32_t
initial_value: "-67305985" initial_value: "-3003"
- id: stored_u_qword - id: stored_u_qword
type: uint64_t type: uint64_t
initial_value: "72623859790382856" initial_value: "5005"
- id: stored_s_qword - id: stored_s_qword
type: int64_t type: int64_t
initial_value: "-72623859790382856" initial_value: "-5005"
- id: stored_u_qword_r - id: stored_u_qword_r
type: uint64_t type: uint64_t
initial_value: "578437695752307201" initial_value: "7007"
- id: stored_s_qword_r - id: stored_s_qword_r
type: int64_t type: int64_t
initial_value: "-578437695752307201" initial_value: "-7007"
- id: stored_fp32 - id: stored_fp32
type: float type: float
initial_value: "3.14" initial_value: "1.5"
- id: stored_fp32_r - id: stored_fp32_r
type: float type: float
initial_value: "2.5" initial_value: "2.5"
- id: stored_bit_2
type: bool
initial_value: "false"
- id: stored_bit_3
type: bool
initial_value: "true"
modbus: modbus:
- uart_id: virtual_uart_server - uart_id: virtual_uart_server
id: virtual_modbus_server id: virtual_modbus_server
role: server role: server
- uart_id: virtual_uart_server_2
id: virtual_modbus_server_2
role: server
- uart_id: virtual_uart_controller - uart_id: virtual_uart_controller
id: virtual_modbus_client id: virtual_modbus_controller
role: client role: client
turnaround_time: 10ms turnaround_time: 10ms
modbus_controller: modbus_controller:
- address: 1 - address: 1
modbus_id: virtual_modbus_client modbus_id: virtual_modbus_controller
update_interval: 2s
id: modbus_controller_1 id: modbus_controller_1
update_interval: 1s
- address: 2
modbus_id: virtual_modbus_client
id: modbus_controller_2
update_interval: 1s
- address: 3
modbus_id: virtual_modbus_client
id: modbus_controller_3
update_interval: 1s
- address: 6
modbus_id: virtual_modbus_client
id: modbus_controller_6
update_interval: 1s
modbus_server: modbus_server:
- address: 1 - address: 1
modbus_id: virtual_modbus_server modbus_id: virtual_modbus_server
id: modbus_server_1
registers: registers:
- address: 0x01 - address: 0x01
value_type: U_WORD value_type: U_WORD
@@ -203,47 +161,6 @@ modbus_server:
value_type: FP32_R value_type: FP32_R
read_lambda: return id(stored_fp32_r); read_lambda: return id(stored_fp32_r);
write_lambda: id(stored_fp32_r) = x; return true; write_lambda: id(stored_fp32_r) = x; return true;
- address: 5
modbus_id: virtual_modbus_server
registers:
# Writable + readable register: srv_write_1 plus the client's read-back
# confirm the write half of the 0x17 ran before the read half (Modbus 6.17).
- address: 0x01
value_type: U_WORD
read_lambda: return id(stored_1);
write_lambda: |-
id(stored_1) = x;
id(srv_write_1).publish_state(x);
return true;
# Read-only register, returned together with 0x01 by the 2-register read half.
- address: 0x02
value_type: U_WORD
read_lambda: return 0x00AA;
- address: 2
modbus_id: virtual_modbus_server_2
registers:
- address: 0x01
value_type: U_WORD
read_lambda: return 919;
- address: 3
modbus_id: virtual_modbus_server_2
registers:
- address: 0x01
value_type: U_WORD
read_lambda: return 929;
- address: 6
modbus_id: virtual_modbus_server_2
bits:
- address: 0x00
read_lambda: return true;
- address: 0x01
read_lambda: return false;
- address: 0x02
read_lambda: return id(stored_bit_2);
write_lambda: id(stored_bit_2) = x; return true;
- address: 0x03
read_lambda: return id(stored_bit_3);
write_lambda: id(stored_bit_3) = x; return true;
sensor: sensor:
- platform: modbus_controller - platform: modbus_controller
@@ -258,12 +175,6 @@ sensor:
address: 0x02 address: 0x02
register_type: holding register_type: holding
value_type: U_WORD_S value_type: U_WORD_S
- platform: modbus_controller
modbus_controller_id: modbus_controller_1
name: "reg_u_word_s_raw"
address: 0x02
register_type: holding
value_type: U_WORD
- platform: modbus_controller - platform: modbus_controller
modbus_controller_id: modbus_controller_1 modbus_controller_id: modbus_controller_1
name: "reg_s_word" name: "reg_s_word"
@@ -336,31 +247,7 @@ sensor:
address: 0x28 address: 0x28
register_type: holding register_type: holding
value_type: FP32_R value_type: FP32_R
- platform: modbus_controller
modbus_controller_id: modbus_controller_2
name: "multi_reg_a"
address: 0x01
register_type: holding
value_type: U_WORD
- platform: modbus_controller
modbus_controller_id: modbus_controller_3
name: "multi_reg_b"
address: 0x01
register_type: holding
value_type: U_WORD
# client_read_write observations, server- and client-side.
- platform: template
name: "srv_write_1"
id: srv_write_1
- platform: template
name: "client_read_0"
id: client_read_0
- platform: template
name: "client_read_1"
id: client_read_1
# The number schema caps min/max at 16777215 (float32 integer precision), so
# the large dword/qword baselines cannot be written back through these numbers.
number: number:
- platform: modbus_controller - platform: modbus_controller
modbus_controller_id: modbus_controller_1 modbus_controller_id: modbus_controller_1
@@ -477,82 +364,8 @@ number:
max_value: 16777215 max_value: 16777215
step: 0.01 step: 0.01
# The four bits are read both as coils (FC 0x01) and discrete inputs (FC 0x02);
# the server serves both from one shared table, so the two views must agree.
binary_sensor:
- platform: modbus_controller
modbus_controller_id: modbus_controller_6
name: "bit_coil_0"
address: 0x00
register_type: coil
- platform: modbus_controller
modbus_controller_id: modbus_controller_6
name: "bit_coil_1"
address: 0x01
register_type: coil
- platform: modbus_controller
modbus_controller_id: modbus_controller_6
name: "bit_coil_2"
address: 0x02
register_type: coil
- platform: modbus_controller
modbus_controller_id: modbus_controller_6
name: "bit_coil_3"
address: 0x03
register_type: coil
- platform: modbus_controller
modbus_controller_id: modbus_controller_6
name: "bit_di_0"
address: 0x00
register_type: discrete_input
- platform: modbus_controller
modbus_controller_id: modbus_controller_6
name: "bit_di_1"
address: 0x01
register_type: discrete_input
- platform: modbus_controller
modbus_controller_id: modbus_controller_6
name: "bit_di_2"
address: 0x02
register_type: discrete_input
- platform: modbus_controller
modbus_controller_id: modbus_controller_6
name: "bit_di_3"
address: 0x03
register_type: discrete_input
# write_bit_2 uses the single-coil write (FC 0x05); write_bit_3 opts into the
# multiple-coils write (FC 0x0F) so both server write paths are exercised.
switch:
- platform: modbus_controller
modbus_controller_id: modbus_controller_6
name: "write_bit_2"
address: 0x02
register_type: coil
- platform: modbus_controller
modbus_controller_id: modbus_controller_6
name: "write_bit_3"
address: 0x03
register_type: coil
use_write_multiple: true
button: button:
- platform: template - platform: template
name: "Start Scenario" name: "Start Scenario"
id: start_scenario_btn id: start_scenario_btn
on_press: # This test does not have anything to start (mock is autostart)
# FC 0x17: write reg 0x0001 = 0x1234, then read regs 0x0001..0x0002 back in the same transaction.
- modbus_client.read_write_multiple_registers:
address: 5
read_address: 0x0001
read_count: 2
write_address: 0x0001
values: [0x1234]
on_response:
then:
- lambda: |-
// values is the read-back block: reg 0x0001 (must be the just-written 0x1234) and reg 0x0002.
if (values.size() >= 2) {
id(client_read_0).publish_state(values[0]);
id(client_read_1).publish_state(values[1]);
}
@@ -1,5 +1,5 @@
esphome: esphome:
name: uart-mock-modbus-srv-injected name: uart-mock-modbus-srv-rw
host: host:
api: api:
@@ -17,8 +17,6 @@ uart:
baud_rate: 115200 baud_rate: 115200
port: /dev/null port: /dev/null
# Shared server-role fixture (see the shared_yaml markers in the test file);
# the injections concatenate and each test waits only on its own sensors.
uart_mock: uart_mock:
- id: virtual_uart_dev - id: virtual_uart_dev
baud_rate: 9600 baud_rate: 9600
@@ -27,31 +25,18 @@ uart_mock:
auto_start: false auto_start: false
debug: debug:
injections: injections:
- delay: 100ms # FC 0x17 Read/Write Multiple Registers on device 1:
inject_rx: [0x01, 0x03, 0x00, 0x03, 0x00, 0x01, 0x74, 0x0A] # Read holding register 3 on device 1 (basic_read) # write reg 0x0001 = 0x1234 (qty 1), then read regs 0x0001..0x0002 (qty 2).
- delay: 100ms # Per Modbus 6.17 the write is performed before the read, so reg 0x0001 must
# Read holding register 7 on device 2, its reply, then read holding # read back the just-written 0x1234 in the same request.
# register 5 on device 1 (read_after_peer_response)
inject_rx: [0x02, 0x03, 0x00, 0x07, 0x00, 0x01, 0x35, 0xF8,
0x02, 0x03, 0x02, 0x00, 0xF0, 0xFC,
0x00, 0x01, 0x03, 0x00, 0x05, 0x00, 0x01, 0x94, 0x0B]
- delay: 100ms
inject_rx: [0x02, 0x03, 0x00, 0x07, 0x00, 0x01, 0x35, 0xF8] # Read holding register 7 on device 2, with no response
- delay: 100ms
# Read holding register 7 on device 2 with no response, then read
# holding register A on device 1 (read_after_peer_timeout)
inject_rx: [0x02, 0x03, 0x00, 0x07, 0x00, 0x01, 0x35, 0xF8,
0x01, 0x03, 0x00, 0x0A, 0x00, 0x01, 0xA4, 0x08]
# FC 0x17 on device 1: write reg 0x0001 = 0x1234 then read 0x0001..0x0002;
# per Modbus 6.17 the write runs first, so 0x0001 must read back 0x1234.
- delay: 100ms - delay: 100ms
inject_rx: inject_rx:
[0x01, 0x17, 0x00, 0x01, 0x00, 0x02, 0x00, 0x01, 0x00, 0x01, 0x02, 0x12, 0x34, 0x49, 0xD8] [0x01, 0x17, 0x00, 0x01, 0x00, 0x02, 0x00, 0x01, 0x00, 0x01, 0x02, 0x12, 0x34, 0x49, 0xD8]
# FC 0x17: write reg 0x0006 = 0x5678 (qty 1), then read reg 0x0006 (qty 1) - # FC 0x17: write reg 0x0003 = 0x5678 (qty 1), then read reg 0x0003 (qty 1) -
# a write and read targeting a different register block. # a write and read targeting a different register block.
- delay: 100ms - delay: 100ms
inject_rx: inject_rx:
[0x01, 0x17, 0x00, 0x06, 0x00, 0x01, 0x00, 0x06, 0x00, 0x01, 0x02, 0x56, 0x78, 0x8B, 0x55] [0x01, 0x17, 0x00, 0x03, 0x00, 0x01, 0x00, 0x03, 0x00, 0x01, 0x02, 0x56, 0x78, 0x9B, 0x10]
globals: globals:
- id: stored_1 - id: stored_1
@@ -85,18 +70,8 @@ modbus_server:
read_lambda: |- read_lambda: |-
id(rw_read_2).publish_state(0x00AA); id(rw_read_2).publish_state(0x00AA);
return 0x00AA; return 0x00AA;
# Second writable + readable register, targeted by the second request.
- address: 0x03 - address: 0x03
value_type: U_WORD
read_lambda: |-
id(basic_read).publish_state(1);
return 1;
- address: 0x05
value_type: U_WORD
read_lambda: |-
id(read_after_peer_response).publish_state(1);
return 1;
# Second writable + readable register, targeted by the second FC 0x17 request.
- address: 0x06
value_type: U_WORD value_type: U_WORD
read_lambda: |- read_lambda: |-
id(rw_read_3).publish_state(id(stored_3)); id(rw_read_3).publish_state(id(stored_3));
@@ -105,22 +80,8 @@ modbus_server:
id(stored_3) = x; id(stored_3) = x;
id(rw_write_3).publish_state(x); id(rw_write_3).publish_state(x);
return true; return true;
- address: 0x0A
value_type: U_WORD
read_lambda: |-
id(read_after_peer_timeout).publish_state(1);
return 1;
sensor: sensor:
- platform: template
name: "basic_read"
id: basic_read
- platform: template
name: "read_after_peer_response"
id: read_after_peer_response
- platform: template
name: "read_after_peer_timeout"
id: read_after_peer_timeout
- platform: template - platform: template
name: "rw_write_1" name: "rw_write_1"
id: rw_write_1 id: rw_write_1
+3 -11
View File
@@ -1,7 +1,7 @@
"""Helpers for manipulating the host platform's preferences file. """Helpers for manipulating the host platform's preferences file.
ESPHome's host platform stores preferences in ESPHome's host platform stores preferences in
``$ESPHOME_PREFDIR/<app_name>.prefs`` using a simple binary layout that ``~/.esphome/prefs/<app_name>.prefs`` using a simple binary layout that
mirrors ``HostPreferences::sync()``: mirrors ``HostPreferences::sync()``:
``[uint32_t key][uint8_t len][uint8_t data[len]]`` per entry. ``[uint32_t key][uint8_t len][uint8_t data[len]]`` per entry.
@@ -11,21 +11,13 @@ boot (e.g. forcing safe mode) or to clear stale state between runs.
from __future__ import annotations from __future__ import annotations
import os
from pathlib import Path from pathlib import Path
import struct import struct
def host_prefs_path(device_name: str) -> Path: def host_prefs_path(device_name: str) -> Path:
"""Return the on-disk prefs file path for a host-platform device. """Return the on-disk prefs file path for a host-platform device."""
return Path.home() / ".esphome" / "prefs" / f"{device_name}.prefs"
Requires ESPHOME_PREFDIR, which the autouse isolated_preferences fixture
sets; refusing the ~/.esphome/prefs fallback keeps tests off real user
data if the fixture is ever bypassed."""
prefdir = os.environ.get("ESPHOME_PREFDIR")
if not prefdir:
raise RuntimeError("ESPHOME_PREFDIR is not set; refusing the real prefs dir")
return Path(prefdir) / f"{device_name}.prefs"
def clear_host_prefs(device_name: str) -> None: def clear_host_prefs(device_name: str) -> None:
+5 -4
View File
@@ -125,11 +125,12 @@ class RawApiClient:
await self.read_until_frame(MESSAGE_TYPE_OF[api_pb2.HelloResponse]) await self.read_until_frame(MESSAGE_TYPE_OF[api_pb2.HelloResponse])
async def send_message(self, msg: message.Message) -> None: async def send_message(self, msg: message.Message) -> None:
await self.send_raw(MESSAGE_TYPE_OF[type(msg)], msg.SerializeToString())
async def send_raw(self, msg_type: int, payload: bytes) -> None:
"""Send a frame with a hand built payload, for shapes protobuf will not serialize."""
loop = asyncio.get_running_loop() loop = asyncio.get_running_loop()
await loop.sock_sendall( await loop.sock_sendall(self._sock, encode_frame(msg_type, payload))
self._sock,
encode_frame(MESSAGE_TYPE_OF[type(msg)], msg.SerializeToString()),
)
async def read_until_frame(self, msg_type: int, timeout: float = 10.0) -> None: async def read_until_frame(self, msg_type: int, timeout: float = 10.0) -> None:
"""Read until at least one frame of msg_type has been received.""" """Read until at least one frame of msg_type has been received."""
+40
View File
@@ -57,6 +57,46 @@ async def wait_for_state(
return await asyncio.wait_for(future, timeout=timeout) return await asyncio.wait_for(future, timeout=timeout)
class StateWaiter:
"""Route one state subscription to any number of predicate waits."""
def __init__(self) -> None:
self._waiters: list[
tuple[Callable[[EntityState], bool], asyncio.Future[EntityState]]
] = []
def on_state(self, state: EntityState) -> None:
for predicate, future in self._waiters:
if future.done():
continue
try:
matched = predicate(state)
except Exception as exc: # noqa: BLE001 the wait re-raises it, the callback must not die
future.set_exception(exc)
continue
if matched:
future.set_result(state)
async def expect(
self,
predicate: Callable[[EntityState], bool],
timeout: float = 5.0,
label: str | None = None,
) -> EntityState:
"""Wait for the next state matching ``predicate``; states seen before this call do not count."""
entry = (predicate, asyncio.get_running_loop().create_future())
self._waiters.append(entry)
try:
async with asyncio.timeout(timeout):
return await entry[1]
except TimeoutError:
raise TimeoutError(
f"no state matched {label or predicate} within {timeout}s"
) from None
finally:
self._waiters.remove(entry)
def find_entity[T: EntityInfo]( def find_entity[T: EntityInfo](
entities: list[EntityInfo], entities: list[EntityInfo],
object_id_substring: str, object_id_substring: str,
@@ -0,0 +1,142 @@
"""decode_field() must take fields that match their declared wire type, drop the ones that do
not, skip unknown fields, and handle two byte tags, varints and length prefixes."""
from __future__ import annotations
from collections.abc import Callable
import struct
from aioesphomeapi import (
EntityState,
LightState,
NumberState,
SwitchState,
TextState,
api_pb2,
)
import pytest
from .raw_api_client import MESSAGE_TYPE_OF, RawApiClient, encode_varint
from .state_utils import InitialStateHelper, StateWaiter, require_entity
from .types import APIClientConnectedFactory, RunCompiledFunction
SWITCH_COMMAND = MESSAGE_TYPE_OF[api_pb2.SwitchCommandRequest]
WIRE_VARINT, WIRE_LENGTH, WIRE_FIXED32 = 0, 2, 5
def tag(field: int, wire_type: int) -> bytes:
return encode_varint((field << 3) | wire_type)
@pytest.mark.asyncio
async def test_api_decode_wire_types(
yaml_config: str,
run_compiled: RunCompiledFunction,
api_client_connected: APIClientConnectedFactory,
unused_tcp_port: int,
) -> None:
async with (
run_compiled(yaml_config),
api_client_connected() as client,
RawApiClient(unused_tcp_port) as raw,
):
entities, _ = await client.list_entities_services()
switch = require_entity(entities, "wire_switch")
light = require_entity(entities, "wire_light")
text = require_entity(entities, "wire_text")
number = require_entity(entities, "wire_number")
key = tag(1, WIRE_FIXED32) + struct.pack("<I", switch.key)
on, off = tag(2, WIRE_VARINT) + b"\x01", tag(2, WIRE_VARINT) + b"\x00"
switch_states: list[bool] = []
waiter = StateWaiter()
def on_state(state: EntityState) -> None:
if isinstance(state, SwitchState) and state.key == switch.key:
switch_states.append(state.state)
waiter.on_state(state)
def switch_is(value: bool) -> Callable[[EntityState], bool]:
return lambda s: (
isinstance(s, SwitchState) and s.key == switch.key and s.state is value
)
def number_is(value: float) -> Callable[[EntityState], bool]:
return lambda s: (
isinstance(s, NumberState) and s.key == number.key and s.state == value
)
initial = InitialStateHelper(entities)
client.subscribe_states(initial.on_state_wrapper(on_state))
await initial.wait_for_initial_states()
await raw.connect()
# A well formed command: fixed32 key, varint state
await raw.send_raw(SWITCH_COMMAND, key + on)
await waiter.expect(switch_is(True))
await raw.send_raw(SWITCH_COMMAND, key + off)
await waiter.expect(switch_is(False))
# The same field with the wrong wire type is dropped, and a varint key never matches an
# entity; each of these would turn the switch on if the payload were read as a varint
seen = len(switch_states)
await raw.send_raw(SWITCH_COMMAND, key + tag(2, WIRE_LENGTH) + b"\x01\x01")
await raw.send_raw(
SWITCH_COMMAND, key + tag(2, WIRE_FIXED32) + b"\x01\x00\x00\x00"
)
await raw.send_raw(
SWITCH_COMMAND, tag(1, WIRE_VARINT) + encode_varint(switch.key) + on
)
# Ordered on the raw socket itself: this frame cannot be parsed before the bad ones, so
# the only switch state since the marker must be the one it produces
await raw.send_raw(SWITCH_COMMAND, key + on)
await waiter.expect(switch_is(True), label="switch on after wrong wire types")
assert switch_states[seen:] == [True]
await raw.send_raw(SWITCH_COMMAND, key + off)
await waiter.expect(switch_is(False))
# Truncated bodies stop the decode loop without taking the connection down: a tag with its
# continuation bit set and nothing after it, a length prefix past the end of the payload,
# and a fixed32 with two of its four bytes
seen = len(switch_states)
await raw.send_raw(SWITCH_COMMAND, key + b"\x80")
await raw.send_raw(SWITCH_COMMAND, key + tag(2, WIRE_LENGTH) + b"\x7f" + b"ab")
await raw.send_raw(SWITCH_COMMAND, tag(1, WIRE_FIXED32) + b"\x01\x02")
await raw.send_raw(SWITCH_COMMAND, key + on)
await waiter.expect(switch_is(True), label="switch on after truncated frames")
assert switch_states[seen:] == [True]
await raw.send_raw(SWITCH_COMMAND, key + off)
await waiter.expect(switch_is(False))
# A negative number goes through the fixed32 float path of a normal client
client.number_command(number.key, -77.5)
await waiter.expect(number_is(-77.5))
# An unknown field ahead of the known ones is skipped; field 200 needs a two byte tag
await raw.send_raw(
SWITCH_COMMAND, tag(200, WIRE_VARINT) + encode_varint(300) + key + on
)
await waiter.expect(switch_is(True))
# Two byte tags (effect fields 18 and 19) and a two byte varint (300 ms transition)
client.light_command(
light.key, state=True, brightness=0.5, transition_length=0.3, effect="Pulse"
)
await waiter.expect(
lambda s: (
isinstance(s, LightState) and s.key == light.key and s.effect == "Pulse"
)
)
client.light_command(light.key, effect="None", state=False)
await waiter.expect(
lambda s: isinstance(s, LightState) and s.key == light.key and not s.state
)
# A string whose length prefix needs two varint bytes
long_text = "w" * 200
client.text_command(text.key, long_text)
await waiter.expect(
lambda s: (
isinstance(s, TextState) and s.key == text.key and s.state == long_text
)
)
@@ -0,0 +1,37 @@
"""Messages without fields go through the shared ProtoMessage entry points on both directions."""
from __future__ import annotations
from aioesphomeapi import api_pb2
import pytest
from .raw_api_client import MESSAGE_TYPE_OF, RawApiClient
from .types import RunCompiledFunction
@pytest.mark.asyncio
async def test_api_empty_message_roundtrip(
yaml_config: str,
run_compiled: RunCompiledFunction,
unused_tcp_port: int,
) -> None:
async with run_compiled(yaml_config), RawApiClient(unused_tcp_port) as client:
await client.connect()
# Field free request and reply on the plain send path
await client.send_message(api_pb2.PingRequest())
await client.read_until_frame(MESSAGE_TYPE_OF[api_pb2.PingResponse])
# Field free request answered by a message with fields, and a list that ends with
# the field free ListEntitiesDoneResponse through the batching path
await client.send_message(api_pb2.DeviceInfoRequest())
await client.read_until_frame(MESSAGE_TYPE_OF[api_pb2.DeviceInfoResponse])
await client.send_message(api_pb2.ListEntitiesRequest())
await client.read_until_frame(MESSAGE_TYPE_OF[api_pb2.ListEntitiesDoneResponse])
assert (
client.frame_counts[MESSAGE_TYPE_OF[api_pb2.ListEntitiesSwitchResponse]]
== 1
)
await client.send_message(api_pb2.DisconnectRequest())
await client.read_until_frame(MESSAGE_TYPE_OF[api_pb2.DisconnectResponse])
@@ -0,0 +1,78 @@
"""Encode paths at their branch boundaries: zero skipped float, fixed32 state, negative int32,
length prefixes of two varint bytes and two byte field tags."""
from __future__ import annotations
import asyncio
from aioesphomeapi import (
NumberState,
SelectInfo,
SensorInfo,
SensorState,
TextSensorState,
)
import pytest
from .state_utils import InitialStateHelper, StateWaiter, require_entity
from .types import APIClientConnectedFactory, RunCompiledFunction
LONG_OPTION = (
"option-with-a-name-long-enough-that-its-length-prefix-needs-two-varint-bytes-"
"when-the-list-entities-response-is-encoded-xxxxxxxxxx"
)
@pytest.mark.asyncio
async def test_api_encode_boundaries(
yaml_config: str,
run_compiled: RunCompiledFunction,
api_client_connected: APIClientConnectedFactory,
) -> None:
async with run_compiled(yaml_config), api_client_connected() as client:
device_info, (entities, _) = await asyncio.gather(
client.device_info(), client.list_entities_services()
)
assert device_info.suggested_area == "Kitchen"
sensor = require_entity(entities, "zero_then_value", SensorInfo)
assert sensor.accuracy_decimals == -2
select = require_entity(entities, "long_option_select", SelectInfo)
assert len(LONG_OPTION) >= 128
assert select.options == ["short", LONG_OPTION]
text = require_entity(entities, "long_text")
number = require_entity(entities, "negative_number")
button = require_entity(entities, "publish_values")
initial = InitialStateHelper(entities)
waiter = StateWaiter()
client.subscribe_states(initial.on_state_wrapper(waiter.on_state))
await initial.wait_for_initial_states()
# A float of exactly zero is skipped on the wire and must still read as 0.0, not missing
first = initial.initial_states[sensor.key]
assert isinstance(first, SensorState)
assert first.state == 0.0 and not first.missing_state
first_number = initial.initial_states[number.key]
assert isinstance(first_number, NumberState)
assert first_number.state == -123.5
client.button_command(button.key)
await asyncio.gather(
waiter.expect(
lambda s: (
isinstance(s, SensorState)
and s.key == sensor.key
and s.state == 12.5
),
label="sensor 12.5",
),
waiter.expect(
lambda s: (
isinstance(s, TextSensorState)
and s.key == text.key
and s.state == "y" * 200
),
label="text 200 x y",
),
)
@@ -24,6 +24,7 @@ from .types import (
RunCompiledFunction, RunCompiledFunction,
) )
pytestmark = pytest.mark.usefixtures("isolated_preferences")
NEW_KEY = PROVISIONING_PSK NEW_KEY = PROVISIONING_PSK
@@ -41,6 +41,15 @@ async def _poll_until_exists(path: Path) -> None:
await asyncio.sleep(0.05) await asyncio.sleep(0.05)
@pytest.fixture(autouse=True)
def isolated_preferences(monkeypatch: pytest.MonkeyPatch, tmp_path) -> Path:
"""Keep host preferences per-test so this test never touches the real
~/.esphome/prefs and never races other tests over ESPHOME_PREFDIR."""
prefdir = tmp_path / "prefs"
monkeypatch.setenv("ESPHOME_PREFDIR", str(prefdir))
return prefdir / f"{DEVICE_NAME}.prefs"
@pytest.mark.asyncio @pytest.mark.asyncio
async def test_host_preferences_suspend_resume( async def test_host_preferences_suspend_resume(
yaml_config: str, yaml_config: str,
@@ -49,7 +58,7 @@ async def test_host_preferences_suspend_resume(
isolated_preferences: Path, isolated_preferences: Path,
) -> None: ) -> None:
"""Test that a running syncer flushes, a suspended one doesn't, and resume restores flushing.""" """Test that a running syncer flushes, a suspended one doesn't, and resume restores flushing."""
pref_file = isolated_preferences / f"{DEVICE_NAME}.prefs" pref_file = isolated_preferences
loop = asyncio.get_running_loop() loop = asyncio.get_running_loop()
saved_in_memory = loop.create_future() saved_in_memory = loop.create_future()
@@ -11,6 +11,14 @@ from .state_utils import InitialStateHelper, require_entity
from .types import APIClientConnectedFactory, RunCompiledFunction from .types import APIClientConnectedFactory, RunCompiledFunction
@pytest.fixture(autouse=True)
def isolated_preferences(monkeypatch: pytest.MonkeyPatch, tmp_path) -> None:
"""Keep host preferences per-test so RESTORE_AND_ON never loads a stale value left
behind by a previous run (host preferences otherwise persist to ~/.esphome/prefs,
keyed only by device name)."""
monkeypatch.setenv("ESPHOME_PREFDIR", str(tmp_path / "prefs"))
@pytest.mark.asyncio @pytest.mark.asyncio
async def test_light_initial_state( async def test_light_initial_state(
yaml_config: str, yaml_config: str,
+81 -82
View File
@@ -19,40 +19,23 @@ from __future__ import annotations
import asyncio import asyncio
from collections.abc import Callable from collections.abc import Callable
from dataclasses import dataclass
from aioesphomeapi import ButtonInfo, NumberInfo, SwitchInfo, TextSensorState from aioesphomeapi import ButtonInfo, NumberInfo, SwitchInfo, TextSensorState
import pytest import pytest
from .state_utils import SensorTracker, find_entity, require_entity, wait_for_state from .state_utils import SensorTracker, find_entity, wait_for_state
from .types import APIClientConnectedFactory, RunCompiledFunction from .types import APIClientConnectedFactory, RunCompiledFunction
def _swap16(value: int) -> int: @dataclass
"""Byte-swapped view of a 16-bit register as the raw U_WORD wire value.""" class RegisterTestCase:
return ((value & 0xFF) << 8) | (value >> 8) """Test parameters for a single modbus register write/read round-trip."""
initial_value: object
# Raw U_WORD view of reg_u_word_s's initial 0x1234 write_number_name: str
MESH_RAW_U_WORD_S = _swap16(4660) write_value: float
post_write_value: object
# Initial values of the mesh fixture's address 1 registers; the
# server_controller test reads them and the write test uses them as baseline.
MESH_INITIAL_VALUES: dict[str, object] = {
"reg_u_word": 99,
"reg_u_word_s": 4660,
"reg_s_word": -99,
"reg_s_word_s": -2,
"reg_u_dword": 16909060,
"reg_s_dword": -16909060,
"reg_u_dword_r": pytest.approx(67305985),
"reg_s_dword_r": pytest.approx(-67305985),
"reg_u_qword": pytest.approx(72623859790382856),
"reg_s_qword": pytest.approx(-72623859790382856),
"reg_u_qword_r": pytest.approx(578437695752307201),
"reg_s_qword_r": pytest.approx(-578437695752307201),
"reg_fp32": pytest.approx(3.14),
"reg_fp32_r": pytest.approx(2.5),
}
# --------------------------------------------------------------------------- # ---------------------------------------------------------------------------
@@ -190,7 +173,6 @@ async def test_uart_mock_modbus_no_threshold(
_assert_no_modbus_errors(error_log_lines, warning_log_lines) _assert_no_modbus_errors(error_log_lines, warning_log_lines)
@pytest.mark.shared_yaml("uart_mock_modbus_server_injected")
@pytest.mark.asyncio @pytest.mark.asyncio
async def test_uart_mock_modbus_server( async def test_uart_mock_modbus_server(
yaml_config: str, yaml_config: str,
@@ -221,7 +203,6 @@ async def test_uart_mock_modbus_server(
_assert_no_modbus_errors(error_log_lines, warning_log_lines) _assert_no_modbus_errors(error_log_lines, warning_log_lines)
@pytest.mark.shared_yaml("uart_mock_modbus_server_injected")
@pytest.mark.asyncio @pytest.mark.asyncio
async def test_uart_mock_modbus_server_read_write( async def test_uart_mock_modbus_server_read_write(
yaml_config: str, yaml_config: str,
@@ -250,8 +231,8 @@ async def test_uart_mock_modbus_server_read_write(
"rw_write_1": 4660, # 0x1234 written to reg 0x0001 "rw_write_1": 4660, # 0x1234 written to reg 0x0001
"rw_read_1": 4660, # reg 0x0001 reads back the just-written value "rw_read_1": 4660, # reg 0x0001 reads back the just-written value
"rw_read_2": 170, # 0x00AA read from reg 0x0002 in the same request "rw_read_2": 170, # 0x00AA read from reg 0x0002 in the same request
"rw_write_3": 22136, # 0x5678 written to reg 0x0006 "rw_write_3": 22136, # 0x5678 written to reg 0x0003
"rw_read_3": 22136, # reg 0x0006 reads back the just-written value "rw_read_3": 22136, # reg 0x0003 reads back the just-written value
} }
) )
@@ -260,8 +241,7 @@ async def test_uart_mock_modbus_server_read_write(
api_client_connected() as client, api_client_connected() as client,
): ):
await tracker.setup_and_start_scenario(client) await tracker.setup_and_start_scenario(client)
# The FC 0x17 injections fire last, behind four earlier 100ms delays await tracker.await_all(futures)
await tracker.await_all(futures, timeout=4.0)
_assert_no_modbus_errors(error_log_lines, warning_log_lines) _assert_no_modbus_errors(error_log_lines, warning_log_lines)
@@ -316,7 +296,6 @@ async def test_uart_mock_modbus_server_read_write_invalid(
) )
@pytest.mark.shared_yaml("uart_mock_modbus_mesh")
@pytest.mark.asyncio @pytest.mark.asyncio
async def test_uart_mock_modbus_server_controller( async def test_uart_mock_modbus_server_controller(
yaml_config: str, yaml_config: str,
@@ -327,7 +306,23 @@ async def test_uart_mock_modbus_server_controller(
line_callback, error_log_lines, warning_log_lines = _make_modbus_line_callback() line_callback, error_log_lines, warning_log_lines = _make_modbus_line_callback()
expected_values = MESH_INITIAL_VALUES | {"reg_u_word_s_raw": MESH_RAW_U_WORD_S} expected_values = {
"reg_u_word": 99,
"reg_u_word_s": 4660,
"reg_u_word_s_raw": 13330,
"reg_s_word": -99,
"reg_s_word_s": -2,
"reg_u_dword": 16909060,
"reg_s_dword": -16909060,
"reg_u_dword_r": pytest.approx(67305985),
"reg_s_dword_r": pytest.approx(-67305985),
"reg_u_qword": pytest.approx(72623859790382856),
"reg_s_qword": pytest.approx(-72623859790382856),
"reg_u_qword_r": pytest.approx(578437695752307201),
"reg_s_qword_r": pytest.approx(-578437695752307201),
"reg_fp32": pytest.approx(3.14),
"reg_fp32_r": pytest.approx(3.14),
}
tracker = SensorTracker(list(expected_values.keys())) tracker = SensorTracker(list(expected_values.keys()))
futures = tracker.expect_all(expected_values) futures = tracker.expect_all(expected_values)
@@ -335,12 +330,14 @@ async def test_uart_mock_modbus_server_controller(
run_compiled(yaml_config, line_callback=line_callback), run_compiled(yaml_config, line_callback=line_callback),
api_client_connected() as client, api_client_connected() as client,
): ):
# The controller polls from boot, so the first values can already be in
# the states the device sends on connect; matching them there saves
# waiting for the next poll
await tracker.setup_and_start_scenario(client, match_initial_states=True) await tracker.setup_and_start_scenario(client, match_initial_states=True)
await tracker.await_all(futures) await tracker.await_all(futures)
_assert_no_modbus_errors(error_log_lines, warning_log_lines) _assert_no_modbus_errors(error_log_lines, warning_log_lines)
@pytest.mark.shared_yaml("uart_mock_modbus_mesh")
@pytest.mark.asyncio @pytest.mark.asyncio
async def test_uart_mock_modbus_server_controller_write( async def test_uart_mock_modbus_server_controller_write(
yaml_config: str, yaml_config: str,
@@ -356,47 +353,51 @@ async def test_uart_mock_modbus_server_controller_write(
line_callback, error_log_lines, warning_log_lines = _make_modbus_line_callback() line_callback, error_log_lines, warning_log_lines = _make_modbus_line_callback()
# Per read-back sensor: the number entity to write through and the value; register_test_cases: dict[str, RegisterTestCase] = {
# floats read back within tolerance, everything else exactly "reg_u_word": RegisterTestCase(11, "write_u_word", 42, 42),
register_writes: dict[str, tuple[str, int | float]] = { "reg_u_word_s": RegisterTestCase(4660, "write_u_word_s", 17185, 17185),
"reg_u_word": ("write_u_word", 42), "reg_s_word": RegisterTestCase(-11, "write_s_word", -42, -42),
"reg_u_word_s": ("write_u_word_s", 17185), "reg_s_word_s": RegisterTestCase(-2, "write_s_word_s", -257, -257),
"reg_s_word": ("write_s_word", -42), "reg_u_dword": RegisterTestCase(1001, "write_u_dword", 2002, 2002),
"reg_s_word_s": ("write_s_word_s", -257), "reg_s_dword": RegisterTestCase(-1001, "write_s_dword", -2002, -2002),
"reg_u_dword": ("write_u_dword", 2002), "reg_u_dword_r": RegisterTestCase(3003, "write_u_dword_r", 4004, 4004),
"reg_s_dword": ("write_s_dword", -2002), "reg_s_dword_r": RegisterTestCase(-3003, "write_s_dword_r", -4004, -4004),
"reg_u_dword_r": ("write_u_dword_r", 4004), "reg_u_qword": RegisterTestCase(5005, "write_u_qword", 6006, 6006),
"reg_s_dword_r": ("write_s_dword_r", -4004), "reg_s_qword": RegisterTestCase(-5005, "write_s_qword", -6006, -6006),
"reg_u_qword": ("write_u_qword", 6006), "reg_u_qword_r": RegisterTestCase(7007, "write_u_qword_r", 8008, 8008),
"reg_s_qword": ("write_s_qword", -6006), "reg_s_qword_r": RegisterTestCase(-7007, "write_s_qword_r", -8008, -8008),
"reg_u_qword_r": ("write_u_qword_r", 8008), "reg_fp32": RegisterTestCase(
"reg_s_qword_r": ("write_s_qword_r", -8008), pytest.approx(1.5, abs=0.01),
"reg_fp32": ("write_fp32", 6.28), "write_fp32",
"reg_fp32_r": ("write_fp32_r", 9.42), 3.14,
pytest.approx(3.14, abs=0.01),
),
"reg_fp32_r": RegisterTestCase(
pytest.approx(2.5, abs=0.01),
"write_fp32_r",
6.28,
pytest.approx(6.28, abs=0.01),
),
} }
tracker = SensorTracker([*register_writes, "reg_u_word_s_raw"]) tracker = SensorTracker(list(register_test_cases.keys()))
# The raw U_WORD view of 0x02 pins the byte swap on the write path: the
# round trip through write_u_word_s applies the swap an even number of
# times, so only the raw sensor can catch a symmetrically dropped swap.
# Phase 1: expect initial baseline values # Phase 1: expect initial baseline values
initial_futures = tracker.expect_all( initial_futures = tracker.expect_all(
MESH_INITIAL_VALUES | {"reg_u_word_s_raw": MESH_RAW_U_WORD_S} {name: case.initial_value for name, case in register_test_cases.items()}
) )
# Phase 2: expect post-write values (registered now so on_state can match them) # Phase 2: expect post-write values (registered now so on_state can match them)
written_futures = tracker.expect_all( written_futures = tracker.expect_all(
{ {name: case.post_write_value for name, case in register_test_cases.items()}
name: pytest.approx(value, abs=0.01) if isinstance(value, float) else value
for name, (_, value) in register_writes.items()
}
| {"reg_u_word_s_raw": _swap16(register_writes["reg_u_word_s"][1])}
) )
async with ( async with (
run_compiled(yaml_config, line_callback=line_callback), run_compiled(yaml_config, line_callback=line_callback),
api_client_connected() as client, api_client_connected() as client,
): ):
# The controller polls from boot, so the baseline can already be in the
# states the device sends on connect; matching it there saves waiting for
# the next poll
entities = await tracker.setup_and_start_scenario( entities = await tracker.setup_and_start_scenario(
client, match_initial_states=True client, match_initial_states=True
) )
@@ -405,22 +406,19 @@ async def test_uart_mock_modbus_server_controller_write(
# connection is working before issuing writes # connection is working before issuing writes
await tracker.await_all(initial_futures, timeout=4.0) await tracker.await_all(initial_futures, timeout=4.0)
# Issue write commands for all register types; exact object_id match, # Issue write commands for all register types
# since several write_* names are prefixes of a sibling for case in register_test_cases.values():
numbers = { entity = find_entity(entities, case.write_number_name, NumberInfo)
e.object_id.lower(): e for e in entities if isinstance(e, NumberInfo) assert entity is not None, (
} f"{case.write_number_name} number entity not found"
for number_name, value in register_writes.values(): )
entity = numbers.get(number_name) client.number_command(entity.key, case.write_value)
assert entity is not None, f"{number_name} number entity not found"
client.number_command(entity.key, value)
# Wait for sensors to reflect the written values (round-trip write+read) # Wait for sensors to reflect the written values (round-trip write+read)
await tracker.await_all(written_futures, timeout=4.0) await tracker.await_all(written_futures, timeout=4.0)
_assert_no_modbus_errors(error_log_lines, warning_log_lines) _assert_no_modbus_errors(error_log_lines, warning_log_lines)
@pytest.mark.shared_yaml("uart_mock_modbus_mesh")
@pytest.mark.asyncio @pytest.mark.asyncio
async def test_uart_mock_modbus_server_controller_bits( async def test_uart_mock_modbus_server_controller_bits(
yaml_config: str, yaml_config: str,
@@ -466,6 +464,8 @@ async def test_uart_mock_modbus_server_controller_bits(
run_compiled(yaml_config, line_callback=line_callback), run_compiled(yaml_config, line_callback=line_callback),
api_client_connected() as client, api_client_connected() as client,
): ):
# The controller polls from boot and binary sensors drop repeats, so the
# baseline can arrive only in the states the device sends on connect
entities = await tracker.setup_and_start_scenario( entities = await tracker.setup_and_start_scenario(
client, match_initial_states=True client, match_initial_states=True
) )
@@ -476,7 +476,8 @@ async def test_uart_mock_modbus_server_controller_bits(
# Flip both writable bits: 0x02 false -> true, 0x03 true -> false # Flip both writable bits: 0x02 false -> true, 0x03 true -> false
for switch_name, value in (("write_bit_2", True), ("write_bit_3", False)): for switch_name, value in (("write_bit_2", True), ("write_bit_3", False)):
entity = require_entity(entities, switch_name, SwitchInfo) entity = find_entity(entities, switch_name, SwitchInfo)
assert entity is not None, f"{switch_name} switch entity not found"
client.switch_command(entity.key, value) client.switch_command(entity.key, value)
# Wait for both read views to reflect the written values # Wait for both read views to reflect the written values
@@ -484,7 +485,6 @@ async def test_uart_mock_modbus_server_controller_bits(
_assert_no_modbus_errors(error_log_lines, warning_log_lines) _assert_no_modbus_errors(error_log_lines, warning_log_lines)
@pytest.mark.shared_yaml("uart_mock_modbus_mesh")
@pytest.mark.asyncio @pytest.mark.asyncio
async def test_uart_mock_modbus_server_controller_multiple( async def test_uart_mock_modbus_server_controller_multiple(
yaml_config: str, yaml_config: str,
@@ -495,7 +495,7 @@ async def test_uart_mock_modbus_server_controller_multiple(
line_callback, error_log_lines, warning_log_lines = _make_modbus_line_callback() line_callback, error_log_lines, warning_log_lines = _make_modbus_line_callback()
expected_values = {"multi_reg_a": 919, "multi_reg_b": 929} expected_values = {"reg_u_word": 919, "reg_u_word_2": 929}
tracker = SensorTracker(list(expected_values.keys())) tracker = SensorTracker(list(expected_values.keys()))
futures = tracker.expect_all(expected_values) futures = tracker.expect_all(expected_values)
@@ -503,6 +503,9 @@ async def test_uart_mock_modbus_server_controller_multiple(
run_compiled(yaml_config, line_callback=line_callback), run_compiled(yaml_config, line_callback=line_callback),
api_client_connected() as client, api_client_connected() as client,
): ):
# The controller polls from boot, so the first values can already be in
# the states the device sends on connect; matching them there saves
# waiting for the next poll
await tracker.setup_and_start_scenario(client, match_initial_states=True) await tracker.setup_and_start_scenario(client, match_initial_states=True)
await tracker.await_all(futures) await tracker.await_all(futures)
_assert_no_modbus_errors(error_log_lines, warning_log_lines) _assert_no_modbus_errors(error_log_lines, warning_log_lines)
@@ -703,7 +706,6 @@ async def test_uart_mock_modbus_shared_address(
_assert_no_modbus_errors(error_log_lines, warning_log_lines) _assert_no_modbus_errors(error_log_lines, warning_log_lines)
@pytest.mark.shared_yaml("uart_mock_modbus_loopback")
@pytest.mark.asyncio @pytest.mark.asyncio
async def test_uart_mock_modbus_custom_pdu( async def test_uart_mock_modbus_custom_pdu(
yaml_config: str, yaml_config: str,
@@ -930,7 +932,6 @@ async def test_uart_mock_modbus_broadcast_write(
_assert_no_modbus_errors(error_log_lines, warning_log_lines) _assert_no_modbus_errors(error_log_lines, warning_log_lines)
@pytest.mark.shared_yaml("uart_mock_modbus_mesh")
@pytest.mark.asyncio @pytest.mark.asyncio
async def test_uart_mock_modbus_client_read_write( async def test_uart_mock_modbus_client_read_write(
yaml_config: str, yaml_config: str,
@@ -946,7 +947,9 @@ async def test_uart_mock_modbus_client_read_write(
""" """
line_callback, error_log_lines, warning_log_lines = _make_modbus_line_callback() line_callback, error_log_lines, warning_log_lines = _make_modbus_line_callback()
tracker = SensorTracker(["srv_write_1", "client_read_0", "client_read_1"]) tracker = SensorTracker(
["srv_write_1", "srv_read_1", "client_read_0", "client_read_1"]
)
futures = tracker.expect_all( futures = tracker.expect_all(
{ {
"srv_write_1": 4660, # server wrote 0x1234 to reg 0x0001 "srv_write_1": 4660, # server wrote 0x1234 to reg 0x0001
@@ -964,7 +967,6 @@ async def test_uart_mock_modbus_client_read_write(
_assert_no_modbus_errors(error_log_lines, warning_log_lines) _assert_no_modbus_errors(error_log_lines, warning_log_lines)
@pytest.mark.shared_yaml("uart_mock_modbus_loopback")
@pytest.mark.asyncio @pytest.mark.asyncio
async def test_uart_mock_modbus_register_offset( async def test_uart_mock_modbus_register_offset(
yaml_config: str, yaml_config: str,
@@ -1020,7 +1022,6 @@ async def test_uart_mock_modbus_register_offset(
) )
@pytest.mark.shared_yaml("uart_mock_modbus_loopback")
@pytest.mark.asyncio @pytest.mark.asyncio
async def test_uart_mock_modbus_lambda_write( async def test_uart_mock_modbus_lambda_write(
yaml_config: str, yaml_config: str,
@@ -1057,7 +1058,6 @@ async def test_uart_mock_modbus_lambda_write(
await tracker.await_change(wrote_30, "reg_30", timeout=4.0) await tracker.await_change(wrote_30, "reg_30", timeout=4.0)
@pytest.mark.shared_yaml("uart_mock_modbus_loopback")
@pytest.mark.asyncio @pytest.mark.asyncio
async def test_uart_mock_modbus_lambda_invert( async def test_uart_mock_modbus_lambda_invert(
yaml_config: str, yaml_config: str,
@@ -1113,7 +1113,6 @@ async def test_uart_mock_modbus_lambda_invert(
) )
@pytest.mark.shared_yaml("uart_mock_modbus_loopback")
@pytest.mark.asyncio @pytest.mark.asyncio
async def test_uart_mock_modbus_deprecated_write_buffer( async def test_uart_mock_modbus_deprecated_write_buffer(
yaml_config: str, yaml_config: str,
-28
View File
@@ -2122,34 +2122,6 @@ def test_get_cpp_changed_components_independent_of_cwd(
) == ["time"] ) == ["time"]
def test_fixture_map_includes_shared_yaml_markers() -> None:
"""Fixtures named only by shared_yaml markers must map to their test file."""
helpers.get_fixture_to_test_files.cache_clear()
mapping = helpers.get_fixture_to_test_files()
for fixture in (
"uart_mock_modbus_loopback",
"uart_mock_modbus_mesh",
"uart_mock_modbus_server_injected",
):
assert mapping[fixture] == frozenset(
{"tests/integration/test_uart_mock_modbus.py"}
)
def test_no_orphan_integration_fixtures() -> None:
"""Every fixture must reach CI test selection; an orphan selects nothing."""
helpers.get_fixture_to_test_files.cache_clear()
mapping = helpers.get_fixture_to_test_files()
fixtures_dir = (Path(__file__).parent.parent / "integration" / "fixtures").resolve()
fixtures = list(fixtures_dir.glob("*.yaml"))
assert fixtures, f"no fixtures found under {fixtures_dir}"
# cache_init is covered via INTEGRATION_TESTS_TRIGGER_FILES instead
orphans = [
f.stem for f in fixtures if f.stem != "cache_init" and f.stem not in mapping
]
assert not orphans, f"fixtures invisible to CI test selection: {orphans}"
def test_lpt_partition_balances_skewed_weights() -> None: def test_lpt_partition_balances_skewed_weights() -> None:
"""Heavy items spread across groups instead of clustering.""" """Heavy items spread across groups instead of clustering."""
items = [f"i{n}" for n in range(6)] items = [f"i{n}" for n in range(6)]
@@ -1,219 +0,0 @@
"""Unit tests for script/sync_dependency_versions.py."""
from pathlib import Path
import subprocess
import sys
import pytest
import yamlrocks
sys.path.insert(0, str((Path(__file__).parent / ".." / ".." / "script").resolve()))
import sync_dependency_versions as sync_mod # noqa: E402
PRECOMMIT = """\
# See https://pre-commit.com for more information
repos:
- repo: https://github.com/astral-sh/ruff-pre-commit
# Ruff version.
rev: v0.1.0
hooks:
- id: ruff
- repo: https://github.com/PyCQA/flake8
rev: 7.0.0
hooks:
- id: flake8
- repo: https://github.com/asottile/pyupgrade
rev: v3.0.0
hooks:
- id: pyupgrade
- repo: https://github.com/pre-commit/mirrors-clang-format
rev: v13.0.1
hooks:
- id: clang-format
- repo: https://github.com/adrienverge/yamllint.git
rev: v1.0.0
hooks:
- id: yamllint
- repo: local
hooks:
- id: pylint
"""
REQ_TEST = """\
pylint==4.0.8
flake8==7.1.0
ruff==0.2.0 # comment
pyupgrade==3.0.0
"""
REQ_DEV = """\
clang-format==13.0.1
yamllint==1.0.0
"""
RUFF_REPO = "https://github.com/astral-sh/ruff-pre-commit"
DUPLICATE_RUFF_BLOCK = f" - repo: {RUFF_REPO}\n rev: v0.3.0\n hooks: []\n"
EXPECTED_DRIFT = ["ruff: 0.1.0 -> 0.2.0", "flake8: 7.0.0 -> 7.1.0"]
EXPECTED_PRECOMMIT = PRECOMMIT.replace("rev: v0.1.0", "rev: v0.2.0").replace(
"rev: 7.0.0", "rev: 7.1.0"
)
@pytest.fixture
def root(tmp_path: Path) -> Path:
"""A fake checkout where ruff (v-prefixed) and flake8 (bare) have drifted."""
(tmp_path / ".pre-commit-config.yaml").write_text(PRECOMMIT)
(tmp_path / "requirements_test.txt").write_text(REQ_TEST)
(tmp_path / "requirements_dev.txt").write_text(REQ_DEV)
return tmp_path
def _load(text: str) -> object:
return yamlrocks.loads(text.encode(), option=yamlrocks.OPT_ROUND_TRIP)
@pytest.mark.parametrize(
("requirements", "expected"),
[
("prek==0.5.1 # comment\n", "0.5.1"),
("Prek==0.5.1\n", "0.5.1"),
("other==1.0\nprek==0.5.1\n", "0.5.1"),
("prek>=0.5.1\n", None),
("prek-extra==0.5.1\n", None),
("", None),
],
)
def test_read_requirement_version(requirements: str, expected: str | None) -> None:
assert sync_mod.read_requirement_version(requirements, "prek") == expected
def test_find_repo_entry() -> None:
entry = sync_mod.find_repo_entry(_load(PRECOMMIT), RUFF_REPO)
assert entry["rev"] == "v0.1.0"
@pytest.mark.parametrize(
("text", "message"),
[
("hooks: []\n", "missing key 'repos'"),
("repos:\n - rev: 1.0.0\n", "missing key 'repo'"),
(PRECOMMIT + DUPLICATE_RUFF_BLOCK, "found 2"),
("repos:\n - repo: other\n rev: 1.0.0\n", "found 0"),
],
)
def test_find_repo_entry_errors(text: str, message: str) -> None:
with pytest.raises(sync_mod.SyncError, match=message):
sync_mod.find_repo_entry(_load(text), RUFF_REPO)
@pytest.mark.parametrize(
("rev", "expected"),
[("v0.1.0", ("v", "0.1.0")), ("7.0.0", ("", "7.0.0")), ("'1.0'", ("", "1.0"))],
)
def test_current_rev(rev: str, expected: tuple[str, str]) -> None:
doc = _load(f"repos:\n - repo: {RUFF_REPO}\n rev: {rev}\n")
assert sync_mod.current_rev(doc["repos"][0], RUFF_REPO) == expected
@pytest.mark.parametrize(
("block", "message"),
[(" hooks: []\n", "has no rev"), (" rev: 1.0\n", "not a string: 1.0")],
)
def test_current_rev_errors(block: str, message: str) -> None:
doc = _load(f"repos:\n - repo: {RUFF_REPO}\n{block}")
with pytest.raises(sync_mod.SyncError, match=message):
sync_mod.current_rev(doc["repos"][0], RUFF_REPO)
def test_sync_reports_without_writing(root: Path) -> None:
assert sync_mod.sync(root, write=False) == EXPECTED_DRIFT
assert (root / ".pre-commit-config.yaml").read_text() == PRECOMMIT
def test_sync_writes_keeps_layout_and_is_idempotent(root: Path) -> None:
assert sync_mod.sync(root, write=True) == EXPECTED_DRIFT
assert (root / ".pre-commit-config.yaml").read_text() == EXPECTED_PRECOMMIT
assert sync_mod.sync(root, write=True) == []
def test_sync_does_not_touch_a_config_that_matches(root: Path) -> None:
(root / ".pre-commit-config.yaml").write_text(EXPECTED_PRECOMMIT)
before = (root / ".pre-commit-config.yaml").stat().st_mtime_ns
assert sync_mod.sync(root, write=True) == []
assert (root / ".pre-commit-config.yaml").stat().st_mtime_ns == before
def test_sync_missing_requirement_pin(root: Path) -> None:
(root / "requirements_dev.txt").write_text("")
with pytest.raises(sync_mod.SyncError, match="no 'clang-format==' pin"):
sync_mod.sync(root, write=True)
def test_sync_propagates_config_errors(root: Path) -> None:
(root / ".pre-commit-config.yaml").write_text(PRECOMMIT + DUPLICATE_RUFF_BLOCK)
with pytest.raises(sync_mod.SyncError, match="found 2"):
sync_mod.sync(root, write=True)
def test_main_check_reports_drift(
root: Path, capsys: pytest.CaptureFixture[str]
) -> None:
assert sync_mod.main(["--check", "--root", str(root)]) == 1
assert capsys.readouterr().out.splitlines() == EXPECTED_DRIFT
assert (root / ".pre-commit-config.yaml").read_text() == PRECOMMIT
def test_main_writes_then_check_is_clean(
root: Path, capsys: pytest.CaptureFixture[str]
) -> None:
assert sync_mod.main(["--root", str(root)]) == 0
assert capsys.readouterr().out.splitlines() == EXPECTED_DRIFT
assert sync_mod.main(["--check", "--root", str(root)]) == 0
assert capsys.readouterr().out == ""
def test_main_reports_sync_error(
root: Path, capsys: pytest.CaptureFixture[str]
) -> None:
(root / "requirements_dev.txt").write_text("")
assert sync_mod.main(["--root", str(root)]) == 1
assert (
"error: requirements_dev.txt: no 'clang-format==' pin"
in capsys.readouterr().err
)
def test_main_defaults_to_repo_root(monkeypatch: pytest.MonkeyPatch) -> None:
seen: dict[str, object] = {}
def fake_sync(root: Path, *, write: bool) -> list[str]:
seen["root"] = root
seen["write"] = write
return []
monkeypatch.setattr(sync_mod, "sync", fake_sync)
assert sync_mod.main([]) == 0
assert seen == {"root": sync_mod.REPO_ROOT, "write": True}
def test_repository_is_in_sync() -> None:
"""The real checkout must match; a failure here means a rev has drifted.
Also proves every SYNC_TARGETS entry still resolves in the real files.
"""
assert sync_mod.sync(sync_mod.REPO_ROOT, write=False) == []
def test_cli_entry_point(root: Path) -> None:
"""Run the script the way the workflow does, as a subprocess."""
script = Path(sync_mod.__file__)
result = subprocess.run(
[sys.executable, str(script), "--check", "--root", str(root)],
capture_output=True,
text=True,
check=False,
)
assert result.returncode == 1
assert result.stdout.splitlines() == EXPECTED_DRIFT
@@ -194,17 +194,17 @@ def test_superseded_device_info_fields_still_declared_in_header() -> None:
def test_superseded_device_info_fields_still_encoded_and_sized() -> None: def test_superseded_device_info_fields_still_encoded_and_sized() -> None:
"""Each superseded field must still be touched by DeviceInfoResponse's """Each superseded field must still be touched by DeviceInfoResponse's
generated encode() and calculate_size(), i.e. it is still put on the wire. generated encode_msg() and calc_size_msg(), i.e. it is still put on the wire.
""" """
encode_body = _extract_function_body(CPP_TEXT, "DeviceInfoResponse::encode") encode_body = _extract_function_body(CPP_TEXT, "DeviceInfoResponse::encode_msg")
size_body = _extract_function_body(CPP_TEXT, "DeviceInfoResponse::calculate_size") size_body = _extract_function_body(CPP_TEXT, "DeviceInfoResponse::calc_size_msg")
for field_name in SUPERSEDED_FIELDS: for field_name in SUPERSEDED_FIELDS:
assert f"this->{field_name}" in encode_body, ( assert f"msg.{field_name}" in encode_body, (
f"DeviceInfoResponse::encode() no longer references {field_name}. " f"DeviceInfoResponse::encode_msg() no longer references {field_name}. "
f"{DEPRECATED_FIELD_TRAP}" f"{DEPRECATED_FIELD_TRAP}"
) )
assert f"this->{field_name}" in size_body, ( assert f"msg.{field_name}" in size_body, (
f"DeviceInfoResponse::calculate_size() no longer references " f"DeviceInfoResponse::calc_size_msg() no longer references "
f"{field_name}. {DEPRECATED_FIELD_TRAP}" f"{field_name}. {DEPRECATED_FIELD_TRAP}"
) )
@@ -380,3 +380,13 @@ def test_api_version_minor_is_at_least_15() -> None:
"clients to see api_version >= 1.15 in HelloResponse before they will " "clients to see api_version >= 1.15 in HelloResponse before they will "
"ever request it." "ever request it."
) )
def test_generated_encode_calls_keep_the_cursor() -> None:
"""No generated ProtoEncode call may drop the returned cursor."""
dropped = [
line
for line in CPP_TEXT.splitlines()
if "ProtoEncode::" in line and "pos = ProtoEncode::" not in line
]
assert not dropped, dropped[:5]
@@ -15,9 +15,13 @@ import pytest
sys.path.insert(0, str(Path(__file__).parents[4] / "script" / "api_protobuf")) sys.path.insert(0, str(Path(__file__).parents[4] / "script" / "api_protobuf"))
import aioesphomeapi.api_options_pb2 as pb # noqa: E402
from api_protobuf import ( # noqa: E402 from api_protobuf import ( # noqa: E402
MAX_MESSAGE_ID, MAX_MESSAGE_ID,
SOURCE_CLIENT,
_make_ifdef_line, _make_ifdef_line,
build_message_type,
create_field_type_info,
get_varint64_ifdef, get_varint64_ifdef,
validate_message_id, validate_message_id,
) )
@@ -34,16 +38,26 @@ def _file_with_messages(
file_desc = descriptor_pb2.FileDescriptorProto(name="test.proto") file_desc = descriptor_pb2.FileDescriptorProto(name="test.proto")
for name, field_type, deprecated in messages: for name, field_type, deprecated in messages:
msg = file_desc.message_type.add(name=name) msg = file_desc.message_type.add(name=name)
field = msg.field.add(name="value", number=1, type=field_type) field = msg.field.add()
field.CopyFrom(_field(field_type))
field.options.deprecated = deprecated field.options.deprecated = deprecated
return file_desc return file_desc
UINT64 = descriptor_pb2.FieldDescriptorProto.TYPE_UINT64 UINT64 = descriptor_pb2.FieldDescriptorProto.TYPE_UINT64
MESSAGE = descriptor_pb2.FieldDescriptorProto.TYPE_MESSAGE
DOUBLE = descriptor_pb2.FieldDescriptorProto.TYPE_DOUBLE
INT64 = descriptor_pb2.FieldDescriptorProto.TYPE_INT64 INT64 = descriptor_pb2.FieldDescriptorProto.TYPE_INT64
SINT64 = descriptor_pb2.FieldDescriptorProto.TYPE_SINT64 SINT64 = descriptor_pb2.FieldDescriptorProto.TYPE_SINT64
UINT32 = descriptor_pb2.FieldDescriptorProto.TYPE_UINT32 UINT32 = descriptor_pb2.FieldDescriptorProto.TYPE_UINT32
INT32 = descriptor_pb2.FieldDescriptorProto.TYPE_INT32
SINT32 = descriptor_pb2.FieldDescriptorProto.TYPE_SINT32
FIXED64 = descriptor_pb2.FieldDescriptorProto.TYPE_FIXED64 FIXED64 = descriptor_pb2.FieldDescriptorProto.TYPE_FIXED64
FIXED32 = descriptor_pb2.FieldDescriptorProto.TYPE_FIXED32
FLOAT = descriptor_pb2.FieldDescriptorProto.TYPE_FLOAT
BOOL = descriptor_pb2.FieldDescriptorProto.TYPE_BOOL
STRING = descriptor_pb2.FieldDescriptorProto.TYPE_STRING
BYTES = descriptor_pb2.FieldDescriptorProto.TYPE_BYTES
def test_no_varint64_fields() -> None: def test_no_varint64_fields() -> None:
@@ -107,3 +121,190 @@ def test_message_id_at_maximum_is_accepted() -> None:
def test_message_id_above_maximum_is_rejected() -> None: def test_message_id_above_maximum_is_rejected() -> None:
with pytest.raises(ValueError, match="exceeds the plaintext"): with pytest.raises(ValueError, match="exceeds the plaintext"):
validate_message_id(MAX_MESSAGE_ID + 1, "TooBigMessage") validate_message_id(MAX_MESSAGE_ID + 1, "TooBigMessage")
def _field(
field_type: int, number: int = 1, *, force: bool = False, repeated: bool = False
) -> descriptor_pb2.FieldDescriptorProto:
field = descriptor_pb2.FieldDescriptorProto(
name="value", number=number, type=field_type
)
if repeated:
field.label = descriptor_pb2.FieldDescriptorProto.LABEL_REPEATED
if force:
field.options.Extensions[pb.force] = True
return field
def _encode_field(
field_type: int, number: int = 1, force: bool = False, repeated: bool = False
) -> str:
"""Return the encode statement the generator emits for one encode-only field."""
field = _field(field_type, number, force=force, repeated=repeated)
return create_field_type_info(
field, needs_decode=False, needs_encode=True
).encode_content
SCALAR_TYPES = [
BOOL,
UINT32,
INT32,
UINT64,
INT64,
SINT32,
FLOAT,
FIXED32,
STRING,
BYTES,
]
@pytest.mark.parametrize("field_type", SCALAR_TYPES)
def test_forced_fields_use_the_force_overload_or_raw_writes(field_type: int) -> None:
content = _encode_field(field_type, force=True)
assert (
"_force(" in content
or "write_raw_byte(" in content
or "write_tag_and_fixed32(" in content
), content
@pytest.mark.parametrize("field_type", [FLOAT, FIXED32])
def test_single_byte_tag_fixed32_shares_the_outlined_writer(field_type: int) -> None:
unconditional = _encode_field(field_type, force=True)
assert unconditional.count("write_tag_and_fixed32(pos, 13,") == 1, unconditional
guarded = _encode_field(field_type, force=False)
assert guarded.startswith("if ("), guarded
assert "[[likely]]" in guarded
assert "write_tag_and_fixed32(pos, 13," in guarded
@pytest.mark.parametrize("field_type", [FLOAT, FIXED32])
def test_multi_byte_tag_fixed32_falls_back_to_the_generic_helper(
field_type: int,
) -> None:
content = _encode_field(field_type, number=16)
assert "write_tag_and_fixed32" not in content, content
assert content.startswith("pos = ProtoEncode::encode_"), content
def _decode_case(field_type: int, number: int, *, repeated: bool = False) -> str:
"""Return the decode_field() case the generator emits for one decoded field."""
field = _field(field_type, number, repeated=repeated)
if field_type == MESSAGE:
field.type_name = ".Sub"
return create_field_type_info(
field, needs_decode=True, needs_encode=False
).decode_content
@pytest.mark.parametrize(
("needs_decode", "force", "member"),
[
(False, False, "StringRef value{nullptr, 0}; // null until set, encode only"),
(True, False, "StringRef value{};"),
(False, True, "StringRef value{};"),
],
)
def test_string_fields_default_to_null_only_when_never_read(
needs_decode: bool, force: bool, member: str
) -> None:
"""Only a string that is neither decoded nor force encoded may start as a null StringRef."""
ti = create_field_type_info(
_field(STRING, force=force), needs_decode=needs_decode, needs_encode=True
)
assert ti.public_content == [member]
@pytest.mark.parametrize(
("field_type", "number", "wire_type", "accessor"),
[
(UINT32, 2, "WIRE_TYPE_VARINT", "value.as_varint()"),
(BOOL, 3, "WIRE_TYPE_VARINT", "value.as_bool()"),
(STRING, 1, "WIRE_TYPE_LENGTH_DELIMITED", "value.data()"),
(FLOAT, 4, "WIRE_TYPE_FIXED32", "value.as_float()"),
(FIXED32, 5, "WIRE_TYPE_FIXED32", "value.as_fixed32()"),
],
)
def test_decode_cases_carry_field_number_and_wire_type(
field_type: int, number: int, wire_type: str, accessor: str
) -> None:
"""Each decoded field yields one case keyed on its number and declared wire type."""
case = _decode_case(field_type, number)
lines = case.splitlines()
assert lines[0] == f"case proto_tag({number}, {wire_type}):", case
assert accessor in lines[1], case
assert lines[-1].strip() == "break;", case
@pytest.mark.parametrize(
("field_type", "repeated", "wire_type", "store"),
[
(UINT32, True, "WIRE_TYPE_VARINT", "this->value.push_back(value.as_varint());"),
(
STRING,
True,
"WIRE_TYPE_LENGTH_DELIMITED",
"this->value.push_back(value.as_string());",
),
(
MESSAGE,
False,
"WIRE_TYPE_LENGTH_DELIMITED",
"value.decode_to_message(this->value);",
),
(
MESSAGE,
True,
"WIRE_TYPE_LENGTH_DELIMITED",
"value.decode_to_message(this->value.back());",
),
],
)
def test_repeated_and_message_fields_decode_through_the_same_case_shape(
field_type: int, repeated: bool, wire_type: str, store: str
) -> None:
"""Repeated and sub message fields land in the one switch with their own store."""
case = _decode_case(field_type, 7, repeated=repeated)
lines = case.splitlines()
assert lines[0] == f"case proto_tag(7, {wire_type}):", case
assert store in case, case
if field_type == MESSAGE and repeated:
assert "this->value.emplace_back();" in case, case
assert lines[-1].strip() == "break;", case
def test_a_fixed64_field_fails_at_generation_time() -> None:
"""The decode loop has no 64 bit wire type path, so such a field must never reach it silently."""
desc = descriptor_pb2.DescriptorProto(name="Wide")
desc.field.add(name="ratio", number=1, type=DOUBLE)
with pytest.raises(
ValueError, match="64-bit type 'double' .*ratio.* not supported"
):
build_message_type(desc, {}, {"Wide": SOURCE_CLIENT})
def test_message_gets_a_single_decode_field_override() -> None:
"""All wire types of a decoded message land in one decode_field() switch."""
desc = descriptor_pb2.DescriptorProto(name="Mixed")
desc.field.add(name="name", number=1, type=STRING)
desc.field.add(name="count", number=2, type=UINT32)
desc.field.add(name="level", number=3, type=FLOAT)
header, cpp, _ = build_message_type(desc, {}, {"Mixed": SOURCE_CLIENT})
decl = "void decode_field(uint32_t tag, const uint8_t *data, proto_varint_value_t scalar) override;"
assert header.count(decl) == 1
assert (
cpp.count(
"void Mixed::decode_field(uint32_t tag, const uint8_t *data, proto_varint_value_t scalar) {"
)
== 1
)
assert "switch (tag) {" in cpp
assert "const ProtoFieldValue value(data, scalar);" in cpp
for number, wire_type in (
(1, "WIRE_TYPE_LENGTH_DELIMITED"),
(2, "WIRE_TYPE_VARINT"),
(3, "WIRE_TYPE_FIXED32"),
):
assert f"case proto_tag({number}, {wire_type}):" in cpp, cpp
@@ -1,37 +0,0 @@
"""Tests for the udp component configuration schema."""
from __future__ import annotations
import pytest
from esphome.components import udp
from esphome.components.packet_transport import (
CONF_BINARY_SENSORS,
CONF_ENCRYPTION,
CONF_PING_PONG_ENABLE,
CONF_PROVIDERS,
CONF_ROLLING_CODE_ENABLE,
CONF_SENSORS,
)
import esphome.config_validation as cv
@pytest.mark.parametrize(
"option",
[
CONF_PROVIDERS,
CONF_ENCRYPTION,
CONF_PING_PONG_ENABLE,
CONF_ROLLING_CODE_ENABLE,
CONF_SENSORS,
CONF_BINARY_SENSORS,
],
)
def test_relocated_option_rejected(option: str) -> None:
"""Options that moved to packet_transport raise a pointing error."""
with pytest.raises(cv.Invalid) as exc_info:
udp.CONFIG_SCHEMA({option: True})
assert (
f"The '{option}' option should now be configured in the 'packet_transport' component"
in str(exc_info.value)
)
@@ -7,7 +7,6 @@ exercised in their own test modules)."""
import json import json
import logging import logging
from pathlib import Path from pathlib import Path
from unittest.mock import Mock
import pytest import pytest
@@ -229,24 +228,6 @@ def test_resolve_registry_version_raises_without_pkg_file(monkeypatch):
_resolve_registry_version("owner", "pkg", set()) _resolve_registry_version("owner", "pkg", set())
def test_make_registry_client_skips_private_package_probe(monkeypatch):
"""Our client answers the probe locally without patching PlatformIO's class."""
from platformio.account.client import AccountClient
from platformio.registry.client import RegistryClient
pio_probe = RegistryClient.__dict__["allowed_private_packages"]
monkeypatch.setattr(
AccountClient,
"get_account_info",
Mock(side_effect=AssertionError("account probe must not run")),
)
client = lib._make_registry_client().get_registry_client_instance()
assert client.allowed_private_packages() is False
assert RegistryClient.__dict__["allowed_private_packages"] is pio_probe
def _patch_registry_resolve(monkeypatch: pytest.MonkeyPatch) -> None: def _patch_registry_resolve(monkeypatch: pytest.MonkeyPatch) -> None:
"""Stub the registry lookup so tests never touch the network.""" """Stub the registry lookup so tests never touch the network."""
monkeypatch.setattr( monkeypatch.setattr(
@@ -1225,20 +1225,6 @@ def test_main_runs_prefetch(tmp_path: Path) -> None:
mock_prefetch.assert_called_once_with(tmp_path, "testenv") mock_prefetch.assert_called_once_with(tmp_path, "testenv")
def test_main_skips_private_package_probe_before_prefetch(tmp_path: Path) -> None:
"""The registry probe patch is applied before any package manager runs."""
order: list[str] = []
with (
patch.object(pf, "_prefetch", side_effect=lambda *_: order.append("prefetch")),
patch(
"esphome.platformio.runner.patch_registry_private_packages",
side_effect=lambda: order.append("patch"),
),
):
assert pf.main([str(tmp_path), "testenv"]) == 0
assert order == ["patch", "prefetch"]
def test_main_bad_argv_is_a_distinct_exit( def test_main_bad_argv_is_a_distinct_exit(
caplog: pytest.LogCaptureFixture, caplog: pytest.LogCaptureFixture,
) -> None: ) -> None:
@@ -6,9 +6,7 @@ from collections.abc import Callable
import io import io
import sys import sys
from types import ModuleType from types import ModuleType
from unittest.mock import Mock
from platformio.registry.client import RegistryClient
import pytest import pytest
from esphome.platformio import runner from esphome.platformio import runner
@@ -32,7 +30,6 @@ def _prepare_main(
monkeypatch.setattr(sys, "stderr", stream) monkeypatch.setattr(sys, "stderr", stream)
monkeypatch.setattr(runner, "patch_structhash", lambda: None) monkeypatch.setattr(runner, "patch_structhash", lambda: None)
monkeypatch.setattr(runner, "patch_file_downloader", lambda: None) monkeypatch.setattr(runner, "patch_file_downloader", lambda: None)
monkeypatch.setattr(runner, "patch_registry_private_packages", lambda: None)
platformio = ModuleType("platformio") platformio = ModuleType("platformio")
platformio_main = ModuleType("platformio.__main__") platformio_main = ModuleType("platformio.__main__")
@@ -94,40 +91,3 @@ def test_main_still_filters_a_drained_partial_line(
assert runner.main() == 0 assert runner.main() == 0
assert buf.getvalue() == b"" assert buf.getvalue() == b""
def test_main_applies_registry_private_packages_patch(
monkeypatch: pytest.MonkeyPatch,
) -> None:
"""The probe is patched before PlatformIO runs."""
order: list[str] = []
_prepare_main(monkeypatch, lambda: order.append("pio") or 0)
monkeypatch.setattr(
runner, "patch_registry_private_packages", lambda: order.append("patch")
)
assert runner.main() == 0
assert order == ["patch", "pio"]
# Snapshot PlatformIO's own probe at import, before any test can patch it
_PIO_PROBE = RegistryClient.__dict__["allowed_private_packages"]
def test_patch_registry_private_packages_skips_account_probe(
monkeypatch: pytest.MonkeyPatch,
) -> None:
"""Answers False without touching the account client."""
from platformio.account.client import AccountClient
monkeypatch.setattr(RegistryClient, "allowed_private_packages", _PIO_PROBE)
monkeypatch.setattr(
AccountClient,
"get_account_info",
Mock(side_effect=AssertionError("account probe must not run")),
)
runner.patch_registry_private_packages()
assert RegistryClient.allowed_private_packages() is False
assert RegistryClient().allowed_private_packages() is False