github/esphome
2
0
Fork 0
mirror of https://github.com/esphome/esphome.git synced 2026-10-08 03:54:03 +00:00
Code Issues Packages Projects Releases Wiki Activity

Merge pull request #20305 from esphome/bump-2026.10.0b1

Browse Source 
2026.10.0b1
This commit is contained in:
Jesse Hills
2026-10-08 12:14:49 +13:00
committed by GitHub
parent 0b2b16e664 5e4990badc
commit 33cf262616
2285 changed files with 93312 additions and 35343 deletions
Download Patch File Download Diff File Expand all files Collapse all files
.agents/skills/code-review/SKILL.md
+116
View File
@@ -0,0 +1,116 @@
---
name: code-review
description: Review guidance for ESPHome pull requests. Use this when reviewing a pull request that changes ESPHome Python, C++, or component code, to check it against the project's coding conventions, embedded-systems memory rules, testing requirements, and breaking-change policy.
---
# Reviewing ESPHome pull requests
ESPHome parses YAML into C++ firmware for memory-constrained microcontrollers
(ESP32, ESP8266, RP2040, LibreTiny). Review changes with that in mind: RAM and
flash are scarce, and code runs unattended for months.
`AGENTS.md` in the repository root is the full contributor guide and the
authority when it disagrees with this summary. The developer documentation at
https://developers.esphome.io explains the component lifecycle and the reasoning
behind these rules. This skill lists the concrete things worth flagging in a
review; read `AGENTS.md` for the detail behind any item.
Only raise findings that the diff actually introduces or changes. Do not ask for
drive-by cleanup of pre-existing code the PR did not touch.
## Memory and embedded constraints (highest value)
Heap allocation after `setup()` is treated as a reliability bug, not a
performance nit, because it fragments a small shared heap. Flag:
- New heap allocation on a hot path or after setup that could be avoided.
- `std::vector` where the size is known at compile time (use `std::array`, or
`StaticVector<T, N>` when a `push_back` API is needed) or fixed at runtime
init (use `FixedVector<T>`).
- Listener / child-entity registration lists stored as `std::vector`; these have
a compile-time-known count and should use `cg.slot_counter()` plus
`StaticVector`.
- `std::vector<uint8_t>` for a byte buffer that never grows: prefer
`std::unique_ptr<uint8_t[]>` or `std::array`.
- `std::map` / `std::set` / `std::unordered_map` for small datasets (1-16
elements): a `std::vector` of a small struct with linear search is lighter.
- `std::deque` anywhere: it allocates 512-byte blocks and should be avoided.
- `std::string` storing a value set once from config: prefer `StringRef` (the
literal already lives in flash).
- `std::string` / `std::to_string` / string-returning helpers on hot paths where
a buffer or view API exists.
## C++ conventions
- Include what you use: a file referencing a symbol must include the header that
declares it, even if it currently arrives transitively. New or changed uses of
a symbol need the matching include.
- Prefix all member access with `this->`.
- Naming: `lower_snake_case` for functions/methods/variables, `UpperCamelCase`
for classes/structs/enums, `UPPER_SNAKE_CASE` for namespace-scope constants,
trailing underscore on protected/private fields.
- `enum class` values must be prefixed with the enum name in `UPPER_SNAKE_CASE`
(e.g. `UARTFlushResult::UART_FLUSH_RESULT_SUCCESS`). Bare names like `SUCCESS`,
`FAIL`, or `OK` collide with SDK macros on some platforms and break the build.
- Prefer `const`/`enum` over `#define`; `#define` is only for conditional
compilation and code-generation sizes.
- Never call `millis()` in a `loop()` body; use
`App.get_loop_component_start_time()`. A rate-limit gate below ~16 ms (the loop
period) does nothing.
- Pick the timing primitive by cadence: gated `loop()` under 250 ms,
`set_interval` at 500 ms and above.
- Do not override a base method to return the value it already returns (e.g.
`get_setup_priority()` returning `setup_priority::DATA`).
- Wrap string literals passed as printf `%s` args in `LOG_STR_LITERAL()`.
- Required, invariant dependencies should be constructor parameters, not setters.
- Callback registration methods must be templated (`template<typename F>`), not
typed as `std::function`, so lightweight forwarders avoid a heap allocation.
- Two-space indent, `using` over `typedef`, wrap at 120 columns.
## Python conventions
- Type-annotate every new function signature (params and return), new dataclass
fields, and new module-level variables. Import `ConfigType` from
`esphome.types`.
- Use the walrus operator to avoid a double lookup, e.g.
`if (blah := config.get(CONF_BLAH)) is not None:`.
- Reuse existing validators from `config_validation.py` (`cv.rename_key`,
`cv.has_exactly_one_key`, etc.) via `cv.All(...)` instead of hand-rolling.
- `esphome/const.py` is frozen: no new `CONF_` constants there. Define them in
the component's own `.py`, or in `esphome/components/const/__init__.py` when
shared. The same constant defined in three or more component files fails CI.
- State that must persist during code generation goes in `CORE.data` namespaced
under the component `DOMAIN` (a `@dataclass`), not module-level mutable globals.
- Prefer callback-based triggers via `build_callback_automation()`; only use a
`Trigger<Ts...>` subclass when the forwarder needs mutable state.
## Testing and coverage
- New and changed lines and branches need test coverage, including defensive
early-returns, error paths, and no-op guards. A mocked-out function is not
covered; exercise the real call path too.
- Component YAML tests live in `tests/components/<component>/`. Never define
buses (uart, i2c, spi, modbus) directly in a test file: pull them from
`tests/test_build_components/common/` through dict-style `packages:` so CI can
group builds. List-style packages or top-level merge keys block grouping.
- Config-only checks use the `validate.*.yaml` prefix; compiled checks use
`test.*.yaml`.
## Breaking changes and public API
- Base classes under `esphome/core/` and documented config options are public
API. Undocumented `public` members of a component are internal.
- A breaking change needs justification, a migration path in the PR description,
and a deprecation window where feasible (`ESPDEPRECATED` in C++,
`cv.rename_key(..., removed_in=...)` in Python). Changing a codegen-injected
lambda signature is not a breaking change.
## Process and PR hygiene
- PR titles start with a `[tag]` prefix: the component name (e.g. `[uart] ...`)
or `[core]` for shared code.
- Prose in docs, comments, and commit messages should be plain English. Keep
inline comments short and only where the code is not self-explanatory; do not
restate what the code says.
- Verify the PR fills out `.github/PULL_REQUEST_TEMPLATE.md` and adds
`CODEOWNERS` entries for a new component.
.claude/skills/pr-workflow/SKILL.md → .agents/skills/pr-workflow/SKILL.md
View File
.claude/skills
Symlink
+1
View File
@@ -0,0 +1 @@
../.agents/skills
.devcontainer/Dockerfile
-3
View File
@@ -31,7 +31,4 @@ RUN \
platformio settings set enable_telemetry No \
&& platformio settings set check_platformio_interval 1000000
COPY script/platformio_install_deps.py platformio.ini ./
RUN ./platformio_install_deps.py platformio.ini --libraries --platforms --tools
WORKDIR /workspaces
.github/ISSUE_TEMPLATE/bug_report.yml
+23
View File
@@ -9,6 +9,29 @@ body:
If you have a feature request or enhancement, please [request them here instead][fr].
[fr]: https://github.com/orgs/esphome/discussions
- type: markdown
attributes:
value: |
## Use of AI in bug reports
AI tools are good at carrying out well-defined tasks, but they are not good at troubleshooting.
Please do NOT paste an AI-generated wall of text into the issue template - if the AI hasn't solved
your problem, its wild guesses are not likely to help.
Please DO include your own words and observations, compile/boot logs, and
especially a minimal reproducible example of your YAML configuration that demonstrates the problem.
It is however quite acceptable to use AI to translate your *own* report,
if you aren't a competent English speaker.
If you really think it will be useful to include an AI's analysis, preferably wrap it in a `<details>` block which will be collapsed by default.
If you are using AI to help solve a problem, rather than asking it to speculate about what the problem is,
it can be more useful to ask it to create a step-by-step troubleshooting procedure.
AI is also useful for generating boilerplate code, such as a minimal reproducible example of your YAML
configuration that demonstrates the problem.
Used properly, AI can be a useful tool to help you solve your problem, but don't let it get in the way.
- type: textarea
validations:
required: true
.github/actions/build-image/action.yaml
+2 -2
View File
@@ -42,7 +42,7 @@ runs:
- name: Build and push to ghcr by digest
id: build-ghcr
uses: docker/build-push-action@53b7df96c91f9c12dcc8a07bcb9ccacbed38856a # v7.3.0
uses: docker/build-push-action@c3c9e263c25d99ce0380d002d59b67737d91b0dc # v7.4.0
env:
DOCKER_BUILD_SUMMARY: false
DOCKER_BUILD_RECORD_UPLOAD: false
@@ -67,7 +67,7 @@ runs:
- name: Build and push to dockerhub by digest
id: build-dockerhub
uses: docker/build-push-action@53b7df96c91f9c12dcc8a07bcb9ccacbed38856a # v7.3.0
uses: docker/build-push-action@c3c9e263c25d99ce0380d002d59b67737d91b0dc # v7.4.0
env:
DOCKER_BUILD_SUMMARY: false
DOCKER_BUILD_RECORD_UPLOAD: false
.github/actions/cache-arduino8266/action.yml
+39
View File
@@ -0,0 +1,39 @@
name: Cache Arduino ESP8266
description: >
Resolve the pinned Arduino core and xtensa toolchain versions and cache the
native ESP8266 install (~110 MB framework + toolchain; no ccache store, the
seed job saves before any compile runs). Exports
ESPHOME_ARDUINO8266_PREFIX to the job so every later step installs into
the cached path; the Python venv must already be restored. Mirrors
cache-esp-idf: only dev-branch pushes write the shared cache, everything
else restores.
runs:
using: composite
steps:
- name: Resolve the native toolchain cache key
# Versions are pinned in code, not a hashable file; resolve them so a
# bump changes the cache key. Assignment form so errexit catches a
# resolver failure.
id: version
shell: bash
run: |
# One owner for the install prefix: exported here and referenced by
# the cache steps below via env, so the caller's install and the
# cached path cannot diverge.
echo "ESPHOME_ARDUINO8266_PREFIX=$HOME/.esphome-arduino8266" >> "$GITHUB_ENV"
. venv/bin/activate
key=$(python -c 'from esphome.components.esp8266 import RECOMMENDED_ARDUINO_FRAMEWORK_VERSION as f; from esphome.arduino8266.framework import FRAMEWORK_RELEASES, TOOLCHAIN_VERSION as t; print(f"{FRAMEWORK_RELEASES[f].tag}-{t}")')
[ -n "$key" ] || exit 1
echo "key=$key" >> "$GITHUB_OUTPUT"
- name: Cache the native toolchain (write on dev)
if: github.ref == 'refs/heads/dev'
uses: actions/cache@55cc8345863c7cc4c66a329aec7e433d2d1c52a9 # v6.1.0
with:
path: ${{ env.ESPHOME_ARDUINO8266_PREFIX }}
key: ${{ runner.os }}-esp8266-native-${{ steps.version.outputs.key }}
- name: Restore the native toolchain (off dev)
if: github.ref != 'refs/heads/dev'
uses: actions/cache/restore@55cc8345863c7cc4c66a329aec7e433d2d1c52a9 # v6.1.0
with:
path: ${{ env.ESPHOME_ARDUINO8266_PREFIX }}
key: ${{ runner.os }}-esp8266-native-${{ steps.version.outputs.key }}
.github/actions/restore-python/action.yml
+1 -1
View File
@@ -32,7 +32,7 @@ runs:
# detects the activated venv via ``VIRTUAL_ENV`` so the venv layout
# downstream jobs rely on is preserved.
if: steps.cache-venv.outputs.cache-hit != 'true'
uses: astral-sh/setup-uv@20cfd1bf945f4377ade1205e4dbc17946fc9a30d # v10.0.1
uses: astral-sh/setup-uv@c18668ad3cf93ea998bef934396af7bb5c839dc7 # v10.2.0
with:
enable-cache: true
# Pull request saves land in per-PR scopes nothing else can
.github/skills
Symlink
+1
View File
@@ -0,0 +1 @@
../.agents/skills
.github/workflows/ci-api-proto.yml
+1 -1
View File
@@ -29,7 +29,7 @@ jobs:
- name: Set up uv
# ``--system`` (below) installs into the setup-python interpreter;
# no venv is created or restored by this workflow.
uses: astral-sh/setup-uv@20cfd1bf945f4377ade1205e4dbc17946fc9a30d # v10.0.1
uses: astral-sh/setup-uv@c18668ad3cf93ea998bef934396af7bb5c839dc7 # v10.2.0
with:
enable-cache: true
# Pull-request-only workflow: a save could never be shared and
.github/workflows/ci-docker.yml
+5 -4
View File
@@ -12,15 +12,14 @@ on:
- ".github/workflows/ci-docker.yml"
- "requirements*.txt"
- "pyproject.toml"
- "platformio.ini"
- "esphome/idf_component.yml"
- "script/platformio_install_deps.py"
# Core, build pipeline, toolchain, and target-platform changes can change
# how a toolchain is set up or built, so re-run the per-toolchain compile
# smoke test when they change.
- "esphome/core/**"
- "esphome/writer.py"
- "esphome/build_gen/**"
- "esphome/build_helpers/**"
- "esphome/espidf/**"
- "esphome/platformio/**"
- "esphome/components/bk72xx/**"
@@ -67,7 +66,7 @@ jobs:
with:
python-version: "3.12"
- name: Set up Docker Buildx
uses: docker/setup-buildx-action@37fe631027851001ddb9b187196cc803df7f5f0e # v4.3.0
uses: docker/setup-buildx-action@594f3bf4285d9ea8dc53c9a0c9c4092420091003 # v4.4.0
- name: Determine tag and whether to push
id: tag
@@ -159,7 +158,7 @@ jobs:
with:
python-version: "3.12"
- name: Set up Docker Buildx
uses: docker/setup-buildx-action@37fe631027851001ddb9b187196cc803df7f5f0e # v4.3.0
uses: docker/setup-buildx-action@594f3bf4285d9ea8dc53c9a0c9c4092420091003 # v4.4.0
- name: Log in to the GitHub container registry
uses: docker/login-action@dbcb813823bdd20940b903addbd779551569679f # v4.6.0
@@ -197,6 +196,7 @@ jobs:
# the default.
id:
- esp8266-arduino
- esp8266-arduino-native
- esp32-arduino-platformio
- esp32-arduino-esp-idf
- esp32-idf-platformio
@@ -219,5 +219,6 @@ jobs:
run: |
docker run --rm \
-v "${{ github.workspace }}/docker/test_configs:/config" \
-e ESPHOME_LDGEN_STRICT=1 \
"ghcr.io/esphome/esphome-amd64:${{ needs.check-docker.outputs.tag }}" \
compile "${{ matrix.id }}.yaml"
.github/workflows/ci.yml
+132 -43
View File
@@ -49,7 +49,7 @@ jobs:
# detects the activated venv via ``VIRTUAL_ENV`` so downstream jobs
# that ``. venv/bin/activate`` see an identical layout.
if: steps.cache-venv.outputs.cache-hit != 'true'
uses: astral-sh/setup-uv@20cfd1bf945f4377ade1205e4dbc17946fc9a30d # v10.0.1
uses: astral-sh/setup-uv@c18668ad3cf93ea998bef934396af7bb5c839dc7 # v10.2.0
with:
enable-cache: true
# Pull request saves land in per-PR scopes nothing else can
@@ -102,6 +102,8 @@ jobs:
device-builder: ${{ steps.determine.outputs.device-builder }}
esp32-platformio: ${{ steps.determine.outputs.esp32-platformio }}
esp32-platformio-components: ${{ steps.determine.outputs.esp32-platformio-components }}
esp8266-native: ${{ steps.determine.outputs.esp8266-native }}
esp8266-native-components: ${{ steps.determine.outputs.esp8266-native-components }}
changed-components: ${{ steps.determine.outputs.changed-components }}
changed-components-with-tests: ${{ steps.determine.outputs.changed-components-with-tests }}
directly-changed-components-with-tests: ${{ steps.determine.outputs.directly-changed-components-with-tests }}
@@ -165,6 +167,8 @@ jobs:
echo "device-builder=$(echo "$output" | jq -r '.device_builder')" >> $GITHUB_OUTPUT
echo "esp32-platformio=$(echo "$output" | jq -r '.esp32_platformio')" >> $GITHUB_OUTPUT
echo "esp32-platformio-components=$(echo "$output" | jq -r '.esp32_platformio_components')" >> $GITHUB_OUTPUT
echo "esp8266-native=$(echo "$output" | jq -r '.esp8266_native')" >> $GITHUB_OUTPUT
echo "esp8266-native-components=$(echo "$output" | jq -r '.esp8266_native_components')" >> $GITHUB_OUTPUT
echo "changed-components=$(echo "$output" | jq -c '.changed_components')" >> $GITHUB_OUTPUT
echo "changed-components-with-tests=$(echo "$output" | jq -c '.changed_components_with_tests')" >> $GITHUB_OUTPUT
echo "directly-changed-components-with-tests=$(echo "$output" | jq -c '.directly_changed_components_with_tests')" >> $GITHUB_OUTPUT
@@ -183,6 +187,33 @@ jobs:
path: .temp/components_graph.json
key: components-graph-${{ hashFiles('esphome/components/**/*.py') }}
seed-esp8266-native-cache:
name: Seed the esp8266 native toolchain cache
runs-on: ubuntu-24.04
needs:
- common
# PR-branch cache saves are invisible to other PRs, so dev pushes seed
# the shared entry the component matrix, the memory impact jobs and
# test-esp8266-native restore. Only dev: the composite action saves
# nowhere else, so a beta/release push would download the toolchain and
# discard it.
if: github.event_name == 'push' && github.ref == 'refs/heads/dev'
timeout-minutes: 15
steps:
- name: Check out code from GitHub
uses: actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1 # v7.0.1
- name: Restore Python
uses: ./.github/actions/restore-python
with:
python-version: ${{ env.DEFAULT_PYTHON }}
cache-key: ${{ needs.common.outputs.cache-key }}
- name: Cache the native toolchain
uses: ./.github/actions/cache-arduino8266
- name: Install the native toolchain
run: |
. venv/bin/activate
python -c "from esphome.arduino8266.framework import check_and_install; from esphome.components.esp8266 import RECOMMENDED_ARDUINO_FRAMEWORK_VERSION; check_and_install(RECOMMENDED_ARDUINO_FRAMEWORK_VERSION)"
ci-custom:
name: Run script/ci-custom
runs-on: ubuntu-24.04
@@ -244,11 +275,20 @@ jobs:
steps:
- name: Check out code from GitHub
uses: actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1 # v7.0.1
- name: Read prek version from requirements_test.txt
id: prek
# requirements_test.txt is the only place the version is pinned, so a
# Dependabot bump there is picked up here without a second edit.
run: |
if ! version=$(sed -nE 's/^prek==([^[:space:]#]+).*/\1/p' requirements_test.txt) || [ -z "$version" ]; then
echo "::error::No prek== pin found in requirements_test.txt."
exit 1
fi
echo "version=$version" >> "$GITHUB_OUTPUT"
- name: Run prek
uses: j178/prek-action@4e14d07f9231acabce116ccfca13b13dd9755ece # v3.0.0
with:
# Keep in sync with requirements_test.txt.
prek-version: "0.4.11"
prek-version: ${{ steps.prek.outputs.version }}
# This job only runs on pull requests, so nothing ever populates
# the cache on dev. Every run would miss and then write a per-pull
# request copy, which is what the old seed-cache job existed to
@@ -259,7 +299,7 @@ jobs:
# Pushes any fixes the hooks made back to the pull request. This step
# must keep its default name: the GitHub App that performs the push
# locates the workflow run by that name.
- uses: pre-commit-ci/lite-action@5d6cc0eb514c891a40562a58a8e71576c5c7fb43 # v1.1.0
- uses: pre-commit-ci/lite-action@062bca0919bc9d6e66755cc05074b70c77e111fc # v1.2.0
if: always()
with:
msg: apply automatic formatting fixes
@@ -312,7 +352,7 @@ jobs:
. venv/bin/activate
pytest -vv --cov-report=xml --tb=native --durations=30 -n auto tests --ignore=tests/integration/
- name: Upload coverage to Codecov
uses: codecov/codecov-action@fb8b3582c8e4def4969c97caa2f19720cb33a72f # v7.0.0
uses: codecov/codecov-action@303a32d7a59b442fa8d48b6a1cc6825c09c847a5 # v7.1.1
- name: Save Python virtual environment cache
if: github.ref == 'refs/heads/dev'
uses: actions/cache/save@55cc8345863c7cc4c66a329aec7e433d2d1c52a9 # v6.1.0
@@ -341,7 +381,7 @@ jobs:
- name: Check out code from GitHub
uses: actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1 # v7.0.1
- name: Report empty upload to Codecov
uses: codecov/codecov-action@fb8b3582c8e4def4969c97caa2f19720cb33a72f # v7.0.0
uses: codecov/codecov-action@303a32d7a59b442fa8d48b6a1cc6825c09c847a5 # v7.1.1
with:
run_command: empty-upload
force: true
@@ -360,14 +400,9 @@ jobs:
matrix:
bucket: ${{ fromJson(needs.determine-jobs.outputs.integration-test-buckets) }}
env:
# What the cache steps persist; libdeps is excluded (keyed per xdist
# worker and env, it never crosses runs).
INTEGRATION_PIO_CACHE_PATH: |
~/.esphome-integration-tests/platformio/platforms
~/.esphome-integration-tests/platformio/packages
~/.esphome-integration-tests/platformio/appstate.json
~/.esphome-integration-tests/platformio/.cache
~/.esphome-integration-tests/platformio/.esphome.pio.stamp.json
# Registry libraries (noise-c, libsodium, ArduinoJson, lvgl) the host builds
# download, shared per xdist worker by tests/integration/conftest.py
INTEGRATION_LIBRARY_CACHE_PATH: ~/.esphome-integration-tests/pio_components
steps:
- name: Check out code from GitHub
uses: actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1 # v7.0.1
@@ -387,14 +422,13 @@ jobs:
uses: actions/setup-python@5fda3b95a4ea91299a34e894583c3862153e4b97 # v7.0.0
with:
python-version: "3.13"
- name: Restore integration PlatformIO cache
# Native platform + toolchain installed by shared_platformio_cache in
# tests/integration/conftest.py; a miss self-heals, so no restore-keys.
id: pio-cache
- name: Restore integration library cache
# A miss or a changed pin self-heals with a download, so no restore-keys
id: library-cache
uses: actions/cache/restore@55cc8345863c7cc4c66a329aec7e433d2d1c52a9 # v6.1.0
with:
path: ${{ env.INTEGRATION_PIO_CACHE_PATH }}
key: integration-pio-v1-${{ runner.os }}-py${{ steps.python.outputs.python-version }}-${{ hashFiles('requirements.txt', 'tests/integration/fixtures/cache_init.yaml', 'esphome/components/host/__init__.py') }}
path: ${{ env.INTEGRATION_LIBRARY_CACHE_PATH }}
key: integration-libraries-v1-${{ runner.os }}-${{ hashFiles('esphome/components/json/__init__.py', 'esphome/components/noise/__init__.py', 'esphome/components/lvgl/__init__.py', 'esphome/components/improv_base/__init__.py') }}
- name: Restore Python virtual environment
id: cache-venv
uses: actions/cache@55cc8345863c7cc4c66a329aec7e433d2d1c52a9 # v6.1.0
@@ -404,7 +438,7 @@ jobs:
- name: Set up uv
# Only needed on cache miss to populate the venv.
if: steps.cache-venv.outputs.cache-hit != 'true'
uses: astral-sh/setup-uv@20cfd1bf945f4377ade1205e4dbc17946fc9a30d # v10.0.1
uses: astral-sh/setup-uv@c18668ad3cf93ea998bef934396af7bb5c839dc7 # v10.2.0
with:
enable-cache: true
# Pull request saves land in per-PR scopes nothing else can
@@ -462,16 +496,16 @@ jobs:
# A full cron period of margin for the weekly refresh
retention-days: 14
- name: Print ccache statistics
# esphome stores the PlatformIO ccache under the machine-global cache
# dir (see _ccache_env() in esphome/platformio/toolchain.py).
run: CCACHE_DIR="$HOME/.cache/esphome/platformio-ccache" ccache -s
- name: Save integration PlatformIO cache
# esphome stores the host build's ccache under the machine-global
# cache dir (see get_build_env() in esphome/host/toolchain.py).
run: CCACHE_DIR="$HOME/.cache/esphome/host/ccache" ccache -s
- name: Save integration library cache
# Bucket 0 only; the others would race the same immutable key.
if: success() && (github.ref == 'refs/heads/dev' || contains(github.event.pull_request.labels.*.name, 'ci-cache-write')) && strategy.job-index == 0 && steps.pio-cache.outputs.cache-hit != 'true'
if: success() && (github.ref == 'refs/heads/dev' || contains(github.event.pull_request.labels.*.name, 'ci-cache-write')) && strategy.job-index == 0 && steps.library-cache.outputs.cache-hit != 'true'
uses: actions/cache/save@55cc8345863c7cc4c66a329aec7e433d2d1c52a9 # v6.1.0
with:
path: ${{ env.INTEGRATION_PIO_CACHE_PATH }}
key: ${{ steps.pio-cache.outputs.cache-primary-key }}
path: ${{ env.INTEGRATION_LIBRARY_CACHE_PATH }}
key: ${{ steps.library-cache.outputs.cache-primary-key }}
import-time:
name: Check import esphome.__main__ time
@@ -592,7 +626,7 @@ jobs:
apt-get install -y libc6-dbg
- name: Run CodSpeed benchmarks
uses: CodSpeedHQ/action@373d6868929f444bc08d901fd0eb0ad52a8875ea # v5.2.1
uses: CodSpeedHQ/action@c4fd08a3a159bd0cc208da1e0edf32b8c47d75e5 # v5.4.0
with:
run: |
. venv/bin/activate
@@ -1088,6 +1122,11 @@ jobs:
uses: ./.github/actions/cache-sdk-nrf
with:
restore-only: true
- name: Cache the native ESP8266 toolchain
# Only batches whose test platforms include esp8266; never saves
# here, it reuses the install the dev seed job cached.
if: matrix.batch.needs_arduino8266
uses: ./.github/actions/cache-arduino8266
- name: Validate and compile components with intelligent grouping
run: |
. venv/bin/activate
@@ -1186,9 +1225,28 @@ jobs:
if [ -n "$compile_csv" ]; then
# Run compilation with grouping and isolation
python3 script/test_build_components.py -e compile -c "$compile_csv" -f --isolate "$directly_changed_csv"
# The bootloader has no ESPHome code and these builds never
# flash; the check_idf_py batch keeps the full build so the
# native sub-build and the equivalence check stay covered.
skip_flag="--skip-bootloader"
if [[ "${{ matrix.batch.check_idf_py }}" == "true" ]]; then
skip_flag=""
fi
python3 script/test_build_components.py -e compile -c "$compile_csv" -f --isolate "$directly_changed_csv" $skip_flag
if [[ "${{ matrix.batch.check_idf_py }}" == "true" ]]; then
# The real idf.py must find nothing to configure or build in a
# tree built above; catches drift on ESP-IDF bumps.
echo "Checking the native ESP-IDF build matches idf.py"
python3 script/check_idf_py_equivalence.py
fi
else
echo "All components in this batch are validate-only -- skipping compile stage."
if [[ "${{ matrix.batch.check_idf_py }}" == "true" ]]; then
# determine-jobs and this step disagree on what compiles; fail
# rather than let the check run nowhere.
echo "::error::This batch was picked for the idf.py check but compiled nothing"
exit 1
fi
fi
- name: Print ccache statistics
@@ -1228,7 +1286,7 @@ jobs:
# compile validates config first, so a separate config pass is
# redundant for this smoke test. ESP-IDF framework via PlatformIO:
python3 script/test_build_components.py -e compile -t esp32-idf -c "$TEST_COMPONENTS" -f --toolchain platformio
python3 script/test_build_components.py -e compile -t esp32-idf -c "$TEST_COMPONENTS" -f --toolchain platformio --fail-on-no-tests
echo ""
echo "ESP-IDF-via-PlatformIO build passed! Starting Arduino smoke test..."
@@ -1237,6 +1295,40 @@ jobs:
# Arduino framework via PlatformIO (only components with an esp32-ard test are built):
python3 script/test_build_components.py -e compile -t esp32-ard -c "$TEST_COMPONENTS" -f --toolchain platformio
test-esp8266-native:
name: Test esp8266 components with the native toolchain
runs-on: ubuntu-24.04
needs:
- common
- determine-jobs
if: github.event_name == 'pull_request' && needs.determine-jobs.outputs.esp8266-native == 'true'
env:
# Computed by script/determine-jobs.py (ESP8266_NATIVE_TEST_COMPONENTS)
TEST_COMPONENTS: ${{ needs.determine-jobs.outputs.esp8266-native-components }}
steps:
- name: Check out code from GitHub
uses: actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1 # v7.0.1
- name: Restore Python
uses: ./.github/actions/restore-python
with:
python-version: ${{ env.DEFAULT_PYTHON }}
cache-key: ${{ needs.common.outputs.cache-key }}
- name: Cache the native toolchain
uses: ./.github/actions/cache-arduino8266
- name: Run native toolchain compile test
run: |
. venv/bin/activate
echo "Testing components: $TEST_COMPONENTS"
echo ""
# ESP8266 Arduino built directly (no PlatformIO); compile validates
# config first, so a separate config pass is redundant.
python3 script/test_build_components.py -e compile -t esp8266-ard -c "$TEST_COMPONENTS" -f --toolchain arduino --fail-on-no-tests
device-builder:
name: Test downstream esphome/device-builder
runs-on: ubuntu-24.04
@@ -1265,7 +1357,7 @@ jobs:
# install step (order-of-magnitude faster on cold boots,
# with its own wheel cache). actions/setup-python still
# provides the interpreter.
uses: astral-sh/setup-uv@20cfd1bf945f4377ade1205e4dbc17946fc9a30d # v10.0.1
uses: astral-sh/setup-uv@c18668ad3cf93ea998bef934396af7bb5c839dc7 # v10.2.0
with:
enable-cache: true
# Pull request saves land in per-PR scopes nothing else can
@@ -1406,12 +1498,9 @@ jobs:
python-version: ${{ env.DEFAULT_PYTHON }}
cache-key: ${{ needs.common.outputs.cache-key }}
- name: Cache platformio
if: steps.check-script.outputs.skip != 'true' && steps.check-tests.outputs.skip != 'true' && steps.cache-memory-analysis.outputs.cache-hit != 'true'
uses: actions/cache/restore@55cc8345863c7cc4c66a329aec7e433d2d1c52a9 # v6.1.0
with:
path: ~/.platformio
key: platformio-memory-${{ fromJSON(needs.determine-jobs.outputs.memory_impact).platform }}-${{ hashFiles('platformio.ini') }}
- name: Cache the native ESP8266 toolchain
if: steps.check-script.outputs.skip != 'true' && steps.check-tests.outputs.skip != 'true' && steps.cache-memory-analysis.outputs.cache-hit != 'true' && fromJSON(needs.determine-jobs.outputs.memory_impact).needs_arduino8266
uses: ./.github/actions/cache-arduino8266
- name: Build, compile, and analyze memory
if: steps.check-script.outputs.skip != 'true' && steps.check-tests.outputs.skip != 'true' && steps.cache-memory-analysis.outputs.cache-hit != 'true'
@@ -1496,11 +1585,9 @@ jobs:
with:
python-version: ${{ env.DEFAULT_PYTHON }}
cache-key: ${{ needs.common.outputs.cache-key }}
- name: Cache platformio
uses: actions/cache/restore@55cc8345863c7cc4c66a329aec7e433d2d1c52a9 # v6.1.0
with:
path: ~/.platformio
key: platformio-memory-${{ fromJSON(needs.determine-jobs.outputs.memory_impact).platform }}-${{ hashFiles('platformio.ini') }}
- name: Cache the native ESP8266 toolchain
if: fromJSON(needs.determine-jobs.outputs.memory_impact).needs_arduino8266
uses: ./.github/actions/cache-arduino8266
- name: Build, compile, and analyze memory
id: extract
run: |
@@ -1599,6 +1686,7 @@ jobs:
needs:
- common
- seed-apt-cache
- seed-esp8266-native-cache
- determine-jobs
- ci-custom
- pylint
@@ -1614,6 +1702,7 @@ jobs:
- clang-tidy-esp32-variants
- test-build-components-split
- test-esp32-platformio
- test-esp8266-native
- device-builder
- memory-impact-target-branch
- memory-impact-pr-branch
.github/workflows/codeql.yml
+2 -2
View File
@@ -56,7 +56,7 @@ jobs:
# Initializes the CodeQL tools for scanning.
- name: Initialize CodeQL
uses: github/codeql-action/init@cdf488f595d80d6e07e03d4674febd5ab45fa938 # v4.37.9
uses: github/codeql-action/init@2892aa5e19bbd11bc0cff5427e3b750a04d9e3c2 # v4.38.2
with:
languages: ${{ matrix.language }}
build-mode: ${{ matrix.build-mode }}
@@ -84,6 +84,6 @@ jobs:
exit 1
- name: Perform CodeQL Analysis
uses: github/codeql-action/analyze@cdf488f595d80d6e07e03d4674febd5ab45fa938 # v4.37.9
uses: github/codeql-action/analyze@2892aa5e19bbd11bc0cff5427e3b750a04d9e3c2 # v4.38.2
with:
category: "/language:${{matrix.language}}"
.github/workflows/external-component-bot.yml
+14 -8
View File
@@ -125,14 +125,20 @@ jobs:
}
async function getEsphomeAndComponentChanges(github, owner, repo, prNumber) {
const changedFiles = await github.rest.pulls.listFiles({
owner: owner,
repo: repo,
pull_number: prNumber,
});
const changedFiles = await github.paginate(
github.rest.pulls.listFiles,
{
owner: owner,
repo: repo,
pull_number: prNumber,
per_page: 100,
}
);
const esphomeChanges = changedFiles.data
.filter(file => file.filename !== "esphome/core/defines.h" && file.filename.startsWith('esphome/'))
// Files used only for development and CI, which do not affect use as an external component
const ignoredFiles = ["esphome/core/defines.h", "esphome/idf_component.yml"];
const esphomeChanges = changedFiles
.filter(file => !ignoredFiles.includes(file.filename) && file.filename.startsWith('esphome/'))
.map(file => {
const match = file.filename.match(/esphome\/([^/]+)/);
return match ? match[1] : null;
@@ -144,7 +150,7 @@ jobs:
}
const uniqueEsphomeChanges = [...new Set(esphomeChanges)];
const componentChanges = changedFiles.data
const componentChanges = changedFiles
.filter(file => file.filename.startsWith('esphome/components/'))
.map(file => {
const match = file.filename.match(/esphome\/components\/([^/]+)\//);
.github/workflows/lock.yml
+1 -1
View File
@@ -14,4 +14,4 @@ jobs:
permissions:
issues: write # issues.lock on closed issues
pull-requests: write # issues.lock on closed pull requests
uses: esphome/workflows/.github/workflows/lock.yml@0fdd5e311b7e744069166696072a1a9cbc5fbeb6 # 2026.8.1
uses: esphome/workflows/.github/workflows/lock.yml@cc3e76de337dc59bc1cba8da58d963cd23b873f1 # 2026.9.0
.github/workflows/release.yml
+2 -2
View File
@@ -123,7 +123,7 @@ jobs:
python-version: "3.12"
- name: Set up Docker Buildx
uses: docker/setup-buildx-action@37fe631027851001ddb9b187196cc803df7f5f0e # v4.3.0
uses: docker/setup-buildx-action@594f3bf4285d9ea8dc53c9a0c9c4092420091003 # v4.4.0
- name: Log in to docker hub
uses: docker/login-action@dbcb813823bdd20940b903addbd779551569679f # v4.6.0
@@ -202,7 +202,7 @@ jobs:
merge-multiple: true
- name: Set up Docker Buildx
uses: docker/setup-buildx-action@37fe631027851001ddb9b187196cc803df7f5f0e # v4.3.0
uses: docker/setup-buildx-action@594f3bf4285d9ea8dc53c9a0c9c4092420091003 # v4.4.0
- name: Log in to docker hub
if: matrix.registry == 'dockerhub'
.github/workflows/stale.yml
+2 -2
View File
@@ -16,7 +16,7 @@ jobs:
# No GITHUB_TOKEN permissions: the reusable workflow mints an ESPHome
# GitHub App token so the labels, comments and closures come from
# esphome[bot] instead of github-actions[bot].
uses: esphome/workflows/.github/workflows/stale.yml@a1c1485ab46ef41a84a6a9d8abd7fa4b7628fd70 # main
uses: esphome/workflows/.github/workflows/stale.yml@cc3e76de337dc59bc1cba8da58d963cd23b873f1 # main
secrets:
ESPHOME_GITHUB_APP_PRIVATE_KEY: ${{ secrets.ESPHOME_GITHUB_APP_PRIVATE_KEY }}
with:
@@ -33,7 +33,7 @@ jobs:
and will be closed if no further activity occurs within 7 days.
If you are the author of this PR, please leave a comment if you want
to keep it open. Also, please rebase your PR onto the latest dev
to keep it open. Also, please merge the latest dev branch into your
branch to ensure that it's up to date with the latest changes.
Thank you for your contribution!
.github/workflows/status-check-labels.yml
-4
View File
@@ -7,10 +7,6 @@ on:
permissions:
pull-requests: read # issues.listLabelsOnIssue to detect blocking labels (needs-docs, needs-developer-docs, merge-after-release, chained-pr)
concurrency:
group: ${{ github.workflow }}-${{ github.event.pull_request.number }}
cancel-in-progress: true
jobs:
check:
name: Check blocking labels
.github/workflows/sync-dependency-versions.yml
+94
View File
@@ -0,0 +1,94 @@
# Keeps pre-commit hook revs in sync with the requirements files.
#
# Dependabot only bumps the pins in requirements*.txt. Some of those tools
# are pinned again as hook revs in .pre-commit-config.yaml. This workflow
# runs script/sync_dependency_versions.py against the pull request branch
# and pushes a commit with the revs updated.
name: Sync dependency versions
on:
# pull_request_target rather than pull_request so the App secret is
# available on Dependabot pull requests (pull_request runs opened by
# Dependabot only see Dependabot secrets). The job below only touches
# branches in this repository and only ever executes the script from the
# base branch checkout, so fork code never runs with the token.
pull_request_target:
types: [opened, synchronize, reopened]
paths:
- requirements_dev.txt
- requirements_test.txt
- .pre-commit-config.yaml
- script/sync_dependency_versions.py
# The push to the pull request branch uses the App token minted below, so
# the workflow's GITHUB_TOKEN does not need any scopes.
permissions: {}
concurrency:
group: ${{ github.workflow }}-${{ github.event.pull_request.number }}
cancel-in-progress: true
jobs:
sync:
name: Sync pinned versions
runs-on: ubuntu-latest
# Same-repository branches only: a push to a fork is not possible with
# this token, and it keeps untrusted heads out of a privileged job.
if: >-
github.repository == 'esphome/esphome'
&& github.event.pull_request.head.repo.full_name == github.repository
steps:
- name: Generate a token
id: generate-token
uses: actions/create-github-app-token@bcd2ba49218906704ab6c1aa796996da409d3eb1 # v3.2.0
with:
client-id: ${{ vars.ESPHOME_GITHUB_APP_CLIENT_ID }}
private-key: ${{ secrets.ESPHOME_GITHUB_APP_PRIVATE_KEY }}
# A push made with the workflow's own GITHUB_TOKEN would not start
# CI on the new commit; a push with the App token does.
permission-contents: write # git push of the sync commit to the pull request branch
- name: Check out base branch
# Provides the script that runs below. Deliberately the base branch
# so the pull request cannot change what executes here.
uses: actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1 # v7.0.1
with:
ref: ${{ github.event.pull_request.base.sha }}
persist-credentials: false
- name: Check out pull request branch
# No allow-unsafe-pr-checkout here on purpose: checkout v7 only
# refuses heads that live in a different repository, and the job
# condition above already limits runs to same-repository branches.
# Leaving it off keeps that refusal as a backstop for fork heads.
uses: actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1 # v7.0.1
with:
ref: ${{ github.event.pull_request.head.ref }}
path: pull-request
token: ${{ steps.generate-token.outputs.token }}
- name: Set up Python
uses: actions/setup-python@5fda3b95a4ea91299a34e894583c3862153e4b97 # v7.0.0
with:
python-version: "3.12"
- name: Install yamlrocks
# The script edits YAML through yamlrocks. Take the pin from the
# base branch requirements so this workflow has no copy of its own.
run: pip install "$(grep -E '^yamlrocks==' requirements_test.txt | cut -d'#' -f1)"
- name: Sync pinned versions
run: python script/sync_dependency_versions.py --root pull-request
- name: Push changes
working-directory: pull-request
run: |
if git diff --quiet; then
echo "All pinned versions already match the requirements files."
exit 0
fi
git config user.name "esphome[bot]"
git config user.email "115708604+esphome[bot]@users.noreply.github.com"
git commit -am "Sync pinned tool versions with requirements files"
git push
.github/workflows/sync-device-classes.yml
+1 -1
View File
@@ -47,7 +47,7 @@ jobs:
# setup-python interpreter so subsequent ``prek`` /
# ``script/run-in-env.py`` steps find the deps without a
# ``uv run`` prefix.
uses: astral-sh/setup-uv@20cfd1bf945f4377ade1205e4dbc17946fc9a30d # v10.0.1
uses: astral-sh/setup-uv@c18668ad3cf93ea998bef934396af7bb5c839dc7 # v10.2.0
with:
enable-cache: true
# Pin uv version so the action does not have to fetch the
.pre-commit-config.yaml
+3 -4
View File
@@ -1,7 +1,6 @@
---
# See https://pre-commit.com for more information
# See https://pre-commit.com/hooks.html for more hooks
ci:
autoupdate_commit_msg: 'pre-commit: autoupdate'
autoupdate_schedule: off # Disabled until ruff versions are synced between deps and pre-commit
@@ -11,7 +10,7 @@ ci:
repos:
- repo: https://github.com/astral-sh/ruff-pre-commit
# Ruff version.
rev: v0.16.3
rev: v0.16.10
hooks:
# Run the linter.
- id: ruff
@@ -19,7 +18,7 @@ repos:
# Run the formatter.
- id: ruff-format
- repo: https://github.com/PyCQA/flake8
rev: 7.3.0
rev: 7.4.1
hooks:
- id: flake8
additional_dependencies:
@@ -42,7 +41,7 @@ repos:
- id: pyupgrade
args: [--py312-plus]
- repo: https://github.com/adrienverge/yamllint.git
rev: v1.37.1
rev: v1.38.0
hooks:
- id: yamllint
exclude: ^(\.clang-format|\.clang-tidy)$
AGENTS.md
+67 -4
View File
@@ -322,6 +322,25 @@ file does, and it is the authority when they disagree. The most useful starting
var = await switch.new_switch(config)
```
- **Optional child entities of a hub:** bind the config once with `sensor.sub_sensors(config)` (or
`sub_binary_sensors`, `sub_text_sensors`, `sub_buttons`, `sub_switches`, `sub_numbers`,
`sub_selects` in their domains), adding `parent=hub` for entities that derive from `Parented<T>`,
then make one call per key, even when there is only one. A call creates the entity only when its key
is configured, passes it to the setter and returns it (or `None`); extra arguments such as
`min_value` or `options` go on the call. Always name the setter explicitly on the object that owns
it, never with `getattr` and an f-string, and keep that variable short (`var` for the component
itself, `hub` for one fetched with `cg.get_variable`) so the calls fit on one line. Loops whose
setter also takes an index, such as `set_gate_threshold(x, n)`, stay as they are.
```python
async def to_code(config):
var = cg.new_Pvariable(config[CONF_ID])
sensors = sensor.sub_sensors(config)
await sensors(CONF_TEMPERATURE, var.set_temperature_sensor)
await sensors(CONF_HUMIDITY, var.set_humidity_sensor)
buttons = button.sub_buttons(config, parent=var)
await buttons(CONF_RESTART, var.set_restart_button)
```
* **Automations (Triggers, Actions, Conditions):**
Automations have three building blocks: **Triggers** (fire when something happens), **Actions** (do something), and **Conditions** (check if something is true).
@@ -431,7 +450,31 @@ file does, and it is the authority when they disagree. The most useful starting
MyComponent *parent_;
};
```
Register with `@automation.register_action("my_component.do_something", MyAction, schema, synchronous=True)`. Use `synchronous=True` for actions that run to completion inside `play()` without deferring. Use `synchronous=False` if the action may suspend/defer execution (e.g. `delay`, `wait_until`, `script.wait`) or store trigger arguments for later use.
Register it without writing a builder:
```python
automation.register_simple_action(
"my_component.do_something", MyAction, schema, synchronous=True
)
```
The constructor receives the object named by `config[CONF_ID]`. Use `register_bare_action` for a
no-argument constructor, `register_parented_action` for a class deriving from `Parented<T>`, and
the `@automation.register_action(...)` decorator only when the builder must also set fields.
Use `synchronous=True` for actions that run to completion inside `play()` without deferring. Use `synchronous=False` if the action may suspend/defer execution (e.g. `delay`, `wait_until`, `script.wait`) or store trigger arguments for later use.
**Actions that only forward templatable values to their parent need no C++ class.** Register them
with `register_apply_action`; do not write a `TEMPLATABLE_VALUE` class or a builder for this shape.
```python
automation.register_apply_action(
"my_component.set_gains",
schema,
automation.ApplyField(CONF_KP, "set_kp", cg.float_),
automation.ApplyField(CONF_KI, "set_ki", cg.float_),
)
```
The `ApplyField`, `ApplyCall` and `register_apply_action` docstrings in `esphome/automation.py` cover
the rest; `cover.control` and `cover.template.publish` are in-tree examples. `TEMPLATABLE_VALUE` with
`cg.templatable` stays for actions whose `play()` has real logic beyond forwarding values.
* **Conditions:**
```cpp
@@ -443,7 +486,21 @@ file does, and it is the authority when they disagree. The most useful starting
MyComponent *parent_;
};
```
Register with `@automation.register_condition("my_component.is_active", MyCondition, schema)`.
Register with `automation.register_simple_condition("my_component.is_active", MyCondition, schema)`;
`register_bare_condition`, `register_parented_condition` and the decorator follow the action rules.
**Conditions that only test their parent need no C++ class either.** Register them with
`register_apply_condition`; the expression is applied to the parent, and an `ApplyCall` compares
against config values.
```python
automation.register_apply_condition("my_component.is_active", schema, "is_active()")
automation.register_apply_condition(
"my_component.state_is",
schema,
automation.ApplyCall("state == {}", ((CONF_STATE, cg.bool_),)),
)
```
`cover.is_open`, `rtttl.is_playing` and `component.is_idle` are in-tree examples.
* **Type Hints:** Type-hint all function signatures, including test functions and config validators (e.g. `def validate_x(config: ConfigType) -> ConfigType:`, `def test_x() -> None:`). Import `ConfigType` from `esphome.types`.
@@ -553,6 +610,7 @@ file does, and it is the authority when they disagree. The most useful starting
4. **Lint:** Run `prek` to ensure code is compliant.
5. **Commit:** Commit your changes. There is no strict format for commit messages.
6. **Pull Request:** Submit a PR against the `dev` branch. The Pull Request title must start with a `[tag]` prefix. For component work, use the component name (e.g., `[display] Fix bug`, `[abc123] Add new component`); for changes to shared/core code that isn't tied to a single component, use `[core]` (e.g., `[core] Add validator`). Update documentation, examples, and add `CODEOWNERS` entries as needed. Pull requests should always be made using the `.github/PULL_REQUEST_TEMPLATE.md` template - fill out all sections completely without removing any parts of the template.
7. **Comments:** When commenting on GitHub PRs or issues, don't tag contributors, especially bots. Avoid referring to list items (e.g. from reviews) with the form #nn - this will be interpreted by GitHub as a reference to issue or PR nn. Keep comments short and exclude irrelevant details, backstories, restatement of previous comments and anything that is already obvious to the reader.
* **Documentation Contributions:**
* Documentation is hosted in the separate `esphome/esphome.io` repository.
@@ -628,6 +686,9 @@ file does, and it is the authority when they disagree. The most useful starting
_request_listener_slot()
cg.add(hub.register_listener(var))
```
When several instances each own a list declared at the same size (one per hub of a
`MULTI_CONF` component), pass the owning object as the key, `_request_listener_slot(str(hub))`;
the define is then the largest count any one key requested instead of the total.
```cpp
#ifdef MY_COMPONENT_LISTENER_COUNT
void register_listener(MyComponentListener *listener);
@@ -695,7 +756,9 @@ file does, and it is the authority when they disagree. The most useful starting
6. **Avoid `std::deque`:** It allocates in 512-byte blocks regardless of element size, guaranteeing at least 512 bytes of RAM usage immediately. This is a major source of crashes on memory-constrained devices.
7. **Detection:** Look for these patterns in compiler output:
7. **Never use `new (std::nothrow)`:** On ESP-IDF exceptions are disabled, so a failed nothrow allocation aborts instead of returning `nullptr`. Use `RAMAllocator` from `esphome/core/helpers.h`; CI rejects `std::nothrow`.
8. **Detection:** Look for these patterns in compiler output:
- Large code sections with STL symbols (vector, map, set)
- `alloc`, `realloc`, `dealloc` in symbol names
- `_M_realloc_insert`, `_M_default_append` (vector reallocation)
@@ -839,7 +902,7 @@ file does, and it is the authority when they disagree. The most useful starting
cv.rename_key(
CONF_OLD_KEY, CONF_NEW_KEY, removed_in="2026.6.0", component="my_component"
),
cv.Schema({ ... }),
cv.Schema({...}),
)
```
For other deprecations, warn manually during validation:
CODEOWNERS
+27 -1
View File
@@ -100,6 +100,7 @@ esphome/components/bmp581_i2c/* @danielkent-net @kahrendt
esphome/components/bmp581_spi/* @danielkent-net @kahrendt
esphome/components/bp1658cj/* @Cossid
esphome/components/bp5758d/* @Cossid
esphome/components/bridge/* @kbx81
esphome/components/bthome_mithermometer/* @nagyrobi
esphome/components/button/* @esphome/core
esphome/components/bytebuffer/* @clydebarrow
@@ -111,6 +112,8 @@ esphome/components/captive_portal/* @esphome/core
esphome/components/cc1101/* @gabest11 @lygris
esphome/components/ccs811/* @habbie
esphome/components/cd74hc4067/* @asoehlke
esphome/components/cdc_acm_uart/* @kbx81
esphome/components/cdc_acm_uart/bridge/* @kbx81
esphome/components/ch422g/* @clydebarrow @jesterret
esphome/components/ch423/* @dwmw2
esphome/components/chsc6x/* @kkosik20
@@ -122,6 +125,7 @@ esphome/components/combination/* @Cat-Ion @kahrendt
esphome/components/const/* @esphome/core
esphome/components/coolix/* @glmnet
esphome/components/copy/* @OttoWinter
esphome/components/counter/* @clydebarrow
esphome/components/cover/* @esphome/core
esphome/components/cs5460a/* @balrog-kun
esphome/components/cse7761/* @berfenger
@@ -179,15 +183,16 @@ esphome/components/esp32_camera_web_server/* @ayufan
esphome/components/esp32_can/* @Sympatron
esphome/components/esp32_hosted/* @swoboda1337
esphome/components/esp32_hosted/update/* @swoboda1337
esphome/components/esp32_improv/* @jesserockz
esphome/components/esp32_rmt/* @jesserockz
esphome/components/esp32_rmt_led_strip/* @jesserockz
esphome/components/esp8266/* @esphome/core
esphome/components/esp_ldo/* @clydebarrow
esphome/components/espectre/* @francescopace
esphome/components/espnow/* @jesserockz
esphome/components/espnow/packet_transport/* @EasilyBoredEngineer
esphome/components/ethernet_info/* @gtjadsonsantos
esphome/components/event/* @nohat
esphome/components/exponential_moving_average/* @clydebarrow
esphome/components/exposure_notifications/* @OttoWinter
esphome/components/ezo/* @ssieb
esphome/components/ezo_pmp/* @carlos-sarmiento
@@ -242,8 +247,11 @@ esphome/components/hmac_md5/* @dwmw2
esphome/components/hmac_sha256/* @dwmw2
esphome/components/hoermann_hcp/* @zweckj
esphome/components/homeassistant/* @esphome/core @OttoWinter
esphome/components/homeassistant/button/* @jesserockz
esphome/components/homeassistant/number/* @landonr
esphome/components/homeassistant/select/* @jesserockz
esphome/components/homeassistant/switch/* @Links2004
esphome/components/homeassistant/text/* @jesserockz
esphome/components/honeywell_hih_i2c/* @Benichou34
esphome/components/honeywellabp/* @RubyBailey
esphome/components/honeywellabp2_i2c/* @jpfaff
@@ -263,8 +271,10 @@ esphome/components/i2s_audio/* @jesserockz
esphome/components/i2s_audio/microphone/* @jesserockz
esphome/components/i2s_audio/speaker/* @jesserockz @kahrendt
esphome/components/iaqcore/* @yozik04
esphome/components/icnt86/* @danepowell
esphome/components/ili9xxx/* @clydebarrow @nielsnl68
esphome/components/improv_base/* @esphome/core
esphome/components/improv_ble/* @jesserockz
esphome/components/improv_serial/* @esphome/core
esphome/components/ina226/* @latonita @Sergio303
esphome/components/ina260/* @mreditor97
@@ -277,6 +287,7 @@ esphome/components/inkplate/* @jesserockz @JosipKuci
esphome/components/integration/* @OttoWinter
esphome/components/internal_temperature/* @Mat931
esphome/components/interval/* @esphome/core
esphome/components/ir_rf_base/* @bdraco @kbx81
esphome/components/ir_rf_proxy/* @kbx81
esphome/components/it8951/* @koosoli @limengdu @Passific
esphome/components/jsn_sr04t/* @Mafus1
@@ -418,6 +429,7 @@ esphome/components/pn7150_i2c/* @jesserockz @kbx81
esphome/components/pn7160/* @jesserockz @kbx81
esphome/components/pn7160_i2c/* @jesserockz @kbx81
esphome/components/pn7160_spi/* @jesserockz @kbx81
esphome/components/pn71xx/* @jesserockz @kbx81
esphome/components/power_supply/* @esphome/core
esphome/components/preferences/* @esphome/core
esphome/components/provisioning/* @esphome/core
@@ -425,6 +437,7 @@ esphome/components/psram/* @esphome/core
esphome/components/pulse_meter/* @cstaahl @stevebaxter @TrentHouliston
esphome/components/pvvx_mithermometer/* @pasiz
esphome/components/pylontech/* @functionpointer
esphome/components/pzem6l24/* @nuttytree
esphome/components/qmi8658/* @clydebarrow
esphome/components/qmp6988/* @andrewpc
esphome/components/qr_code/* @wjtje
@@ -453,6 +466,7 @@ esphome/components/rtl87xx/* @kuba2k2
esphome/components/rtttl/* @glmnet @ximex
esphome/components/runtime_image/* @clydebarrow @guillempages @kahrendt
esphome/components/runtime_stats/* @bdraco
esphome/components/rx8025t/* @remcom
esphome/components/rx8130/* @beormund
esphome/components/safe_mode/* @jsuanet @kbx81 @paulmonigatti
esphome/components/scd4x/* @martgras @sjtrny
@@ -474,6 +488,7 @@ esphome/components/sendspin/image/* @kahrendt
esphome/components/sendspin/media_player/* @kahrendt
esphome/components/sendspin/media_source/* @kahrendt
esphome/components/sendspin/sensor/* @kahrendt
esphome/components/sendspin/switch/* @kahrendt
esphome/components/sendspin/text_sensor/* @kahrendt
esphome/components/sensirion_common/* @martgras
esphome/components/sensor/* @esphome/core
@@ -532,6 +547,7 @@ esphome/components/st7735/* @SenexCrenshaw
esphome/components/st7789v/* @kbx81
esphome/components/st7920/* @marsjan155
esphome/components/statsd/* @Links2004
esphome/components/stcc4/* @j9brown
esphome/components/stts22h/* @B48D81EFCC
esphome/components/substitutions/* @esphome/core
esphome/components/sun/* @OttoWinter
@@ -542,11 +558,15 @@ esphome/components/sx126x/* @swoboda1337
esphome/components/sx127x/* @swoboda1337
esphome/components/sy6970/* @linkedupbits
esphome/components/syslog/* @clydebarrow
esphome/components/systa_bus/* @Mat931
esphome/components/t6615/* @tylermenezes
esphome/components/tas2780/* @remcom
esphome/components/tas58xx/* @mrtoy-me @remcom
esphome/components/tc74/* @sethgirvan
esphome/components/tca9548a/* @andreashergert1984
esphome/components/tca9555/* @mobrembski
esphome/components/tcl112/* @glmnet
esphome/components/tcp_uart/* @Bascht74
esphome/components/tee501/* @Stock-M
esphome/components/teleinfo/* @0hax
esphome/components/tem3200/* @bakerkj
@@ -555,6 +575,7 @@ esphome/components/template/datetime/* @rfdarter
esphome/components/template/event/* @nohat
esphome/components/template/fan/* @ssieb
esphome/components/text/* @mauritskorse
esphome/components/tfluna/* @candrews
esphome/components/thermopro_ble/* @sittner
esphome/components/thermostat/* @kbx81
esphome/components/time/* @esphome/core
@@ -586,11 +607,14 @@ esphome/components/uart/* @esphome/core
esphome/components/uart/button/* @ssieb
esphome/components/uart/event/* @eoasmxd
esphome/components/uart/packet_transport/* @clydebarrow
esphome/components/uart_mux/* @kbx81
esphome/components/uart_tcp/* @Bascht74
esphome/components/udp/* @clydebarrow
esphome/components/ufire_ec/* @pvizeli
esphome/components/ufire_ise/* @pvizeli
esphome/components/ufm01/* @ljungqvist
esphome/components/ultrasonic/* @ssieb @swoboda1337
esphome/components/unicode/* @esphome/core
esphome/components/update/* @jesserockz
esphome/components/uponor_smatrix/* @kroimon
esphome/components/usb_cdc_acm/* @kbx81
@@ -630,9 +654,11 @@ esphome/components/wts01/* @alepee
esphome/components/x9c/* @EtienneMD
esphome/components/xdb401/* @RT530
esphome/components/xgzp68xx/* @gcormier
esphome/components/xiaomi_body_scale/* @dckiller51
esphome/components/xiaomi_hhccjcy10/* @fariouche
esphome/components/xiaomi_lywsd02mmc/* @juanluss31
esphome/components/xiaomi_lywsd03mmc/* @ahpohl
esphome/components/xiaomi_mccgq02hl/* @ahpohl @morph027
esphome/components/xiaomi_mhoc303/* @drug123
esphome/components/xiaomi_mhoc401/* @vevsvevs
esphome/components/xiaomi_rtcgq02lm/* @jesserockz
Doxyfile
+1 -1
View File
@@ -48,7 +48,7 @@ PROJECT_NAME = ESPHome
# could be handy for archiving the generated documentation or if some version
# control system is used.
PROJECT_NUMBER = 2026.9.1
PROJECT_NUMBER = 2026.10.0b1
# Using the PROJECT_BRIEF tag one can provide an optional one line description
# for a project that appears at the top of each page and should give viewer a
docker/Dockerfile
+1 -4
View File
@@ -22,16 +22,13 @@ RUN \
-r /requirements.txt
# Install the ESPHome Device Builder dashboard.
RUN uv pip install --no-cache-dir esphome-device-builder==1.14.9
RUN uv pip install --no-cache-dir esphome-device-builder==1.21.0
RUN \
platformio settings set enable_telemetry No \
&& platformio settings set check_platformio_interval 1000000 \
&& mkdir -p /piolibs
COPY script/platformio_install_deps.py platformio.ini /
RUN /platformio_install_deps.py /platformio.ini --libraries
ARG BUILD_VERSION
LABEL \
docker/docker_entrypoint.sh
+6 -2
View File
@@ -21,10 +21,14 @@ export PLATFORMIO_PLATFORMS_DIR="${pio_cache_base}/platforms"
export PLATFORMIO_PACKAGES_DIR="${pio_cache_base}/packages"
export PLATFORMIO_CACHE_DIR="${pio_cache_base}/cache"
# Keep the native toolchain installs on the persistent cache root, not the
# container's ephemeral user cache dir (re-downloaded on every restart).
# Keep the native toolchain installs and compiler caches on the persistent
# cache root, not the container's user cache dir: it is lost on every
# restart, and not writable when the container runs as a non-root user.
export ESPHOME_ESP_IDF_PREFIX="$(dirname "${pio_cache_base}")/idf"
export ESPHOME_SDK_NRF_PREFIX="$(dirname "${pio_cache_base}")/sdk-nrf"
export ESPHOME_ARDUINO8266_PREFIX="$(dirname "${pio_cache_base}")/arduino8266"
export ESPHOME_HOST_PREFIX="$(dirname "${pio_cache_base}")/host"
export ESPHOME_PLATFORMIO_CCACHE_DIR="$(dirname "${pio_cache_base}")/platformio-ccache"
# If /build is mounted, use that as the build path
# otherwise use path in /config (so that builds aren't lost on container restart)
docker/ha-addon-rootfs/etc/s6-overlay/s6-rc.d/discovery/run
+13 -11
View File
@@ -4,7 +4,6 @@
# Home Assistant Add-on: ESPHome
# Sends discovery information to Home Assistant.
# ==============================================================================
declare config
declare port
# We only disable it when disabled explicitly
@@ -19,14 +18,17 @@ port=$(bashio::addon.ingress_port)
# Wait for the ESPHome Device Builder to become available
bashio::net.wait_for "${port}" "127.0.0.1" 300
config=$(\
bashio::var.json \
host "127.0.0.1" \
port "^${port}" \
)
# Send one discovery message; the config is a JSON string built with bashio::var.json.
send_discovery() {
local service=$1
local config=$2
if bashio::discovery "${service}" "${config}" > /dev/null; then
bashio::log.info "Successfully send ${service} discovery information to Home Assistant."
else
bashio::log.error "${service} discovery message to Home Assistant failed!"
fi
}
if bashio::discovery "esphome" "${config}" > /dev/null; then
bashio::log.info "Successfully send discovery information to Home Assistant."
else
bashio::log.error "Discovery message to Home Assistant failed!"
fi
send_discovery "esphome" "$(bashio::var.json host "127.0.0.1" port "^${port}")"
# The Device Builder MCP server, consumed by Home Assistant's mcp integration.
send_discovery "mcp" "$(bashio::var.json url "http://127.0.0.1:${port}/api/mcp")"
docker/ha-addon-rootfs/etc/s6-overlay/s6-rc.d/esphome/run
+6 -2
View File
@@ -15,10 +15,14 @@ export PLATFORMIO_PLATFORMS_DIR="${pio_cache_base}/platforms"
export PLATFORMIO_PACKAGES_DIR="${pio_cache_base}/packages"
export PLATFORMIO_CACHE_DIR="${pio_cache_base}/cache"
# Keep the native toolchain installs on the persistent /data volume, not the
# container's ephemeral user cache dir (wiped on every add-on update/restart).
# Keep the native toolchain installs and compiler caches on the persistent
# /data volume, not the container's ephemeral user cache dir (wiped on every
# add-on update/restart).
export ESPHOME_ESP_IDF_PREFIX=/data/cache/idf
export ESPHOME_SDK_NRF_PREFIX=/data/cache/sdk-nrf
export ESPHOME_ARDUINO8266_PREFIX=/data/cache/arduino8266
export ESPHOME_HOST_PREFIX=/data/cache/host
export ESPHOME_PLATFORMIO_CCACHE_DIR=/data/cache/platformio-ccache
if bashio::config.true 'leave_front_door_open'; then
export DISABLE_HA_AUTHENTICATION=true
docker/test_configs/bk72xx-arduino.yaml
+12
View File
@@ -5,3 +5,15 @@ bk72xx:
board: generic-bk7231n-qfn32-tuya
logger:
wifi:
ssid: MySSID
password: password1
ap:
# mqtt and captive_portal together pull in AsyncTCP and ESPAsyncWebServer;
# a stray ESP32 AsyncTCP copy on the library search path breaks this build
captive_portal:
mqtt:
broker: 192.168.178.84
docker/test_configs/esp8266-arduino-native.yaml
+8
View File
@@ -0,0 +1,8 @@
esphome:
name: docker-test-esp8266-native
esp8266:
board: d1_mini
toolchain: arduino
logger:
docker/test_configs/esp8266-arduino.yaml
+2
View File
@@ -3,5 +3,7 @@ esphome:
esp8266:
board: d1_mini
# The PlatformIO path stays covered whatever the default is
toolchain: platformio
logger:
esphome/__main__.py
+126 -207
View File
@@ -16,8 +16,8 @@ from typing import TYPE_CHECKING, Protocol
# cause them to be loaded before external components are processed, resulting
# in the built-in version being used instead of the external component one.
from esphome import const, platform_hooks
from esphome.build_helpers.native import analysis_backend, native_backend
from esphome.const import (
ALLOWED_NAME_CHARS,
ARGUMENT_HELP_DEVICE,
BUNDLE_EXTENSION,
CONF_API,
@@ -35,7 +35,6 @@ from esphome.const import (
CONF_LOGGER,
CONF_MDNS,
CONF_MQTT,
CONF_NAME,
CONF_NAME_ADD_MAC_SUFFIX,
CONF_OTA,
CONF_PASSWORD,
@@ -61,6 +60,7 @@ from esphome.stacktrace import LogLineProcessor
from esphome.types import ConfigType
from esphome.upload_targets import PortType, get_port_type
from esphome.util import (
ESPHOME_COMMAND,
PICOTOOL_PACKAGE,
FlashImage,
detect_rp2040_bootsel,
@@ -84,7 +84,6 @@ if TYPE_CHECKING:
_LOGGER = logging.getLogger(__name__)
ESPHOME_COMMAND = [sys.executable, "-m", "esphome"]
# Maximum buffer size for serial log reading to prevent unbounded memory growth
SERIAL_BUFFER_MAX_SIZE = 65536
@@ -149,6 +148,7 @@ class ArgsProtocol(Protocol):
file: str | None
no_logs: bool
only_generate: bool
skip_bootloader: bool
show_secrets: bool
dashboard: bool
configuration: str
@@ -817,7 +817,9 @@ def write_cpp_file() -> int:
from esphome.build_gen import espidf
espidf.write_project()
else:
elif not CORE.using_native_toolchain:
# Other native builds generate their project at compile time;
# never write a platformio.ini for them
from esphome.build_gen import platformio
platformio.write_project()
@@ -826,6 +828,14 @@ def write_cpp_file() -> int:
def compile_program(args: ArgsProtocol, config: ConfigType) -> int:
if CORE.skip_bootloader and not (CORE.is_esp32 and CORE.using_toolchain_esp_idf):
# Debug only: an orchestrator cannot see YAML toolchain overrides,
# so this is its expected no-op, and a full build is safe.
_LOGGER.debug(
"--skip-bootloader ignored: only supported on ESP32 with the "
"esp-idf toolchain"
)
CORE.skip_bootloader = False
# Keep this gate here, NOT in config validation: device-builder needs
# `esphome config` to keep succeeding with placeholders so onboarding can run.
if CONF_WIFI in config:
@@ -835,7 +845,7 @@ def compile_program(args: ArgsProtocol, config: ConfigType) -> int:
# Keep this here, NOT in codegen: config-hash and --only-generate must keep
# working on machines that cannot run the toolchain.
if CORE.is_esp8266:
if CORE.is_esp8266 and CORE.using_toolchain_platformio:
from esphome.components.esp8266 import check_rosetta
check_rosetta()
@@ -856,23 +866,20 @@ def compile_program(args: ArgsProtocol, config: ConfigType) -> int:
return rc
# Create factory.bin, ota.bin, and firmware.elf copy
toolchain.create_factory_bin()
if not toolchain.create_factory_bin():
# A build whose factory image could not be produced must not
# exit 0; downloads would serve an image from an older build.
return 1
toolchain.create_ota_bin()
toolchain.create_elf_copy()
from esphome.build_helpers.idedata import IDEDATA_BEST_EFFORT_ERRORS
from esphome.build_helpers.idedata import warn_if_idedata_missing
try:
if toolchain.get_idedata() is None:
_LOGGER.warning("No idedata was generated for this build")
except IDEDATA_BEST_EFFORT_ERRORS as err:
# The firmware already built; an idedata failure must not fail
# a successful build.
_LOGGER.warning(
"Could not generate idedata: %s (IDE, clang-tidy, and "
"memory-analysis data will be unavailable for this build)",
err,
)
_LOGGER.debug("Idedata failure detail", exc_info=True)
warn_if_idedata_missing(toolchain.get_idedata)
elif CORE.using_native_toolchain:
raise EsphomeError(
f"Toolchain '{CORE.toolchain.value}' resolved but no platform "
"backend claimed the build"
)
else:
from esphome.platformio import toolchain
@@ -975,12 +982,16 @@ def upload_using_esptool(
if file is not None:
flash_images = [FlashImage(path=file, offset="0x0")]
elif CORE.using_toolchain_esp_idf:
from esphome.espidf import toolchain
flash_images = [
FlashImage(path=toolchain.get_factory_firmware_path(), offset="0x0")
]
elif (native := native_backend()) is not None:
# Every native backend supplies its own 0x0 flash image (bootloader
# and partitions included where the target needs them)
image = native.get_factory_firmware_path()
if not image.is_file():
hint = getattr(native, "missing_image_hint", lambda: None)()
raise EsphomeError(
hint or f"{image} does not exist; compile the configuration first"
)
flash_images = [FlashImage(path=image, offset="0x0")]
else:
from esphome.platformio import toolchain
@@ -1289,10 +1300,9 @@ def _choose_ota_platform(config: ConfigType, requested: str | None) -> str:
The native API uses challenge-response auth with MD5/SHA256 hashing of a
server-issued nonce, so the password is never sent over the wire; the
``web_server`` path uses HTTP Basic auth which transmits credentials in
cleartext over the LAN. (The native path also supports gzip compression
on ESP8266, where flash space is tight; on ESP32/RP2040/LibreTiny the
backend reports ``supports_compression() == false`` and the firmware is
sent uncompressed regardless of which platform is used.) Falls back to
cleartext over the LAN. (The native path also compresses the upload:
gzip on ESP8266 and RP2040, which inflate it at reboot, and a deflate
stream on ESP32/LibreTiny, which inflate it as it arrives.) Falls back to
``web_server`` only when that is the only available platform.
"""
# Use a dict (insertion-ordered) instead of a list so error messages and
@@ -1343,8 +1353,12 @@ def _upload_via_native_api(
# fall back to a plaintext upload
noise_psk = None
plaintext_fallback = False
allow_plaintext_upload = False
if (encryption_conf := ota_conf.get(CONF_ENCRYPTION)) is not None:
noise_psk = encryption_conf.get(CONF_KEY)
allow_plaintext_upload = bool(
encryption_conf.get(espota2.CONF_ALLOW_PLAINTEXT_UPLOAD)
)
if not noise_psk:
raise EsphomeError(
"OTA encryption is configured but no key was resolved; "
@@ -1377,6 +1391,12 @@ def _upload_via_native_api(
ota_type = espota2.OTA_TYPE_UPDATE_PARTITION_TABLE
elif getattr(args, "bootloader", False):
check_partition_access("--bootloader")
if (
getattr(args, "file", None) is None
and (native := native_backend())
and (hint := getattr(native, "missing_image_hint", lambda: None)())
):
raise EsphomeError(hint)
binary = CORE.bootloader_bin
ota_type = espota2.OTA_TYPE_UPDATE_BOOTLOADER
if getattr(args, "file", None) is not None:
@@ -1395,6 +1415,7 @@ def _upload_via_native_api(
ota_type,
noise_psk,
plaintext_fallback=plaintext_fallback,
allow_plaintext_upload=allow_plaintext_upload,
)
@@ -1709,26 +1730,12 @@ def command_compile(args: ArgsProtocol, config: ConfigType) -> int | None:
if exit_code != 0:
return exit_code
if CORE.is_host:
_LOGGER.info(
"Successfully compiled program to path '%s'", _host_program_path(config)
)
_LOGGER.info("Successfully compiled program to path '%s'", CORE.firmware_bin)
else:
_LOGGER.info("Successfully compiled program.")
return 0
def _host_program_path(config: ConfigType) -> str:
"""Return the compiled host ELF path."""
if CORE.using_toolchain_esp_idf:
from esphome.espidf import toolchain
return str(toolchain.get_elf_path())
from esphome.platformio.toolchain import get_idedata
# Memoized by compile_program's own call; this is a dict lookup
return str(get_idedata(config).firmware_elf_path)
def command_upload(args: ArgsProtocol, config: ConfigType) -> int | None:
# Get devices, resolving special identifiers like OTA
devices = choose_upload_log_host(
@@ -1765,6 +1772,18 @@ def command_logs(args: ArgsProtocol, config: ConfigType) -> int | None:
def command_run(args: ArgsProtocol, config: ConfigType) -> int | None:
if (
CORE.skip_bootloader
and CORE.is_esp32
and CORE.using_toolchain_esp_idf
and any(
get_port_type(device) == PortType.SERIAL for device in (args.device or [])
)
):
# Fail before the compile: the result could never flash over serial.
# Elsewhere the flag is ignored, so serial stays fine there.
_LOGGER.error("--skip-bootloader builds cannot be flashed over serial")
return 1
exit_code = write_cpp(config)
if exit_code != 0:
return exit_code
@@ -1773,7 +1792,7 @@ def command_run(args: ArgsProtocol, config: ConfigType) -> int | None:
return exit_code
_LOGGER.info("Successfully compiled program.")
if CORE.is_host:
program_path = _host_program_path(config)
program_path = str(CORE.firmware_bin)
_LOGGER.info("Running program from path '%s'", program_path)
return run_external_process(program_path)
@@ -1965,12 +1984,12 @@ def command_update_all(args: ArgsProtocol) -> int | None:
def command_idedata(args: ArgsProtocol, config: ConfigType) -> int:
import json
if CORE.using_toolchain_esp_idf:
# Native ESP-IDF derives idedata from the build's compile_commands.json,
# so the configuration must already be compiled.
from esphome.espidf import toolchain as espidf_toolchain
native_toolchain = native_backend()
idedata = espidf_toolchain.get_idedata()
if native_toolchain is not None:
# Native toolchains derive idedata from the build's
# compile_commands.json, so the configuration must already be compiled.
idedata = native_toolchain.get_idedata()
if idedata is None:
_LOGGER.error(
"No idedata available; compile the configuration first",
@@ -2009,6 +2028,22 @@ def command_analyze_memory(args: ArgsProtocol, config: ConfigType) -> int:
from esphome.analyze_memory.cli import MemoryAnalyzerCLI
from esphome.analyze_memory.ram_strings import RamStringsAnalyzer
# Refuse an unsupported toolchain before paying for a full compile
analysis_toolchain = analysis_backend()
if analysis_toolchain is None and not CORE.using_toolchain_platformio:
_LOGGER.error(
"analyze-memory is not supported with the '%s' toolchain on %s; "
"re-run with --toolchain platformio",
CORE.toolchain.value if CORE.toolchain else "unresolved",
CORE.target_platform,
)
return 1
if (
check_supported := getattr(analysis_toolchain, "check_analysis_supported", None)
) is not None:
# Raises with the reason; before the compile, not after it
check_supported()
# Always compile to ensure fresh data (fast if no changes - just relinks)
exit_code = write_cpp(config)
if exit_code != 0:
@@ -2020,13 +2055,30 @@ def command_analyze_memory(args: ArgsProtocol, config: ConfigType) -> int:
# Get idedata for analysis
idedata = None
if CORE.using_toolchain_esp_idf:
from esphome.espidf import toolchain
if analysis_toolchain is not None:
objdump = analysis_toolchain.get_objdump_path()
readelf = analysis_toolchain.get_readelf_path()
for tool in (objdump, readelf):
if not tool.is_file():
# The analyzer would silently fall back to host
# binutils, which cannot read the target ELF
_LOGGER.error(
"%s is missing; the toolchain install may be incomplete "
"(recompile, or run 'esphome clean-all' if it persists)",
tool,
)
return 1
objdump_path = str(objdump)
readelf_path = str(readelf)
objdump_path = str(toolchain.get_objdump_path())
readelf_path = str(toolchain.get_readelf_path())
firmware_elf = toolchain.get_elf_path()
firmware_elf = analysis_toolchain.get_elf_path()
if not firmware_elf.is_file():
# The analyzer swallows tool failures, so a missing ELF would
# produce an exit-0 zeroed report
_LOGGER.error(
"%s is missing; compile the configuration first", firmware_elf
)
return 1
else:
from esphome.platformio import toolchain
@@ -2080,155 +2132,9 @@ def command_analyze_memory(args: ArgsProtocol, config: ConfigType) -> int:
def command_rename(args: ArgsProtocol, config: ConfigType) -> int | None:
from esphome import yaml_util
from esphome.cli.rename import command_rename as run
new_name = args.name
for c in new_name:
if c not in ALLOWED_NAME_CHARS:
safe_print(
color(
AnsiFore.BOLD_RED,
f"'{c}' is an invalid character for names. Valid characters are: "
f"{ALLOWED_NAME_CHARS} (lowercase, no spaces)",
)
)
return 1
# Load existing yaml file
raw_contents = CORE.config_path.read_text(encoding="utf-8")
yaml = yaml_util.load_yaml(CORE.config_path)
if CONF_ESPHOME not in yaml or CONF_NAME not in yaml[CONF_ESPHOME]:
safe_print(
color(
AnsiFore.BOLD_RED, "Complex YAML files cannot be automatically renamed."
)
)
return 1
old_name = yaml[CONF_ESPHOME][CONF_NAME]
match = re.match(r"^\$\{?([a-zA-Z0-9_]+)\}?$", old_name)
if match is None:
# Only swap the ``name:`` line that sits directly under the
# top-level ``esphome:`` block. A naked ``re.sub`` would
# also clobber any other ``name:`` line whose value happens
# to match (e.g. a sensor / output / wifi entry sharing the
# device's hostname), silently rewriting unrelated user
# configuration. The pattern anchors:
# - at the start of the line so ``friendly_name:``,
# ``device_name:`` etc. don't match the trailing ``name:``
# substring; and
# - at the end of the value (lookahead for whitespace +
# comment + EOL) so ``old_name`` doesn't match as a
# prefix of a longer value (``kitchen`` vs ``kitchen2``).
name_pattern = re.compile(
rf"^(\s*)name:\s+[\"']?{re.escape(old_name)}[\"']?(?=\s*(?:#|$))"
)
out_lines: list[str] = []
in_esphome_block = False
for line in raw_contents.splitlines(keepends=True):
if line and not line[0].isspace() and line.strip():
in_esphome_block = line.lstrip().startswith("esphome:")
out_lines.append(line)
continue
if in_esphome_block:
line = name_pattern.sub(rf'\1name: "{new_name}"', line, count=1)
out_lines.append(line)
new_raw = "".join(out_lines)
else:
old_name = yaml[CONF_SUBSTITUTIONS][match.group(1)]
if (
len(
re.findall(
rf"^\s+{match.group(1)}:\s+[\"']?{old_name}[\"']?",
raw_contents,
flags=re.MULTILINE,
)
)
> 1
):
safe_print(
color(AnsiFore.BOLD_RED, "Too many matches in YAML to safely rename")
)
return 1
new_raw = re.sub(
rf"^(\s+{match.group(1)}):\s+[\"']?{old_name}[\"']?",
f'\\1: "{new_name}"',
raw_contents,
flags=re.MULTILINE,
)
# ``new_name == old_name`` (after substitution resolution) is
# a no-op rewrite that would still queue a pointless re-flash.
# Catch it before the path-equality check below — covers the
# case where the config filename doesn't match the device name
# (e.g. ``weird-file.yaml`` whose ``esphome.name`` is
# ``kitchen``; running ``esphome rename weird-file.yaml kitchen``
# would otherwise just re-flash the same hostname).
if new_name == old_name:
safe_print(
color(
AnsiFore.BOLD_RED,
f"'{new_name}' is already the device's name.",
)
)
return 1
new_path: Path = CORE.config_dir / (new_name + ".yaml")
if new_path.resolve() == CORE.config_path.resolve():
safe_print(
color(
AnsiFore.BOLD_RED,
f"'{new_name}' is already the device's name.",
)
)
return 1
if new_path.exists():
safe_print(
color(
AnsiFore.BOLD_RED,
f"Cannot rename: {new_path} already exists. "
"Refusing to overwrite an existing configuration.",
)
)
return 1
safe_print(
f"Updating {color(AnsiFore.CYAN, str(CORE.config_path))} to {color(AnsiFore.CYAN, str(new_path))}"
)
print()
new_path.write_text(new_raw, encoding="utf-8")
rc = run_external_process(*ESPHOME_COMMAND, "config", str(new_path))
if rc != 0:
safe_print(color(AnsiFore.BOLD_RED, "Rename failed. Reverting changes."))
new_path.unlink()
return 1
cli_args = [
"run",
str(new_path),
"--no-logs",
"--device",
CORE.address,
]
if args.dashboard:
cli_args.insert(0, "--dashboard")
try:
rc = run_external_process(*ESPHOME_COMMAND, *cli_args)
except KeyboardInterrupt:
rc = 1
if rc != 0:
new_path.unlink()
return 1
if CORE.config_path != new_path:
CORE.config_path.unlink()
safe_print(color(AnsiFore.BOLD_GREEN, "SUCCESS"))
print()
return 0
return run(args, config)
PRE_CONFIG_ACTIONS = {
@@ -2263,6 +2169,15 @@ SIMPLE_CONFIG_ACTIONS = [
]
def _add_skip_bootloader_arg(parser: argparse.ArgumentParser) -> None:
parser.add_argument(
"--skip-bootloader",
help="Do not build the bootloader or the factory image; "
"the result can only be flashed over OTA.",
action="store_true",
)
def _add_states_args(parser: argparse.ArgumentParser) -> None:
"""Add mutually exclusive ``--states``/``--no-states`` flags to a parser.
@@ -2343,7 +2258,8 @@ def parse_args(argv):
metavar="{" + ",".join(t.value for t in Toolchain) + "}",
help=(
"Select toolchain for compiling. Overrides '<platform>.toolchain' in YAML. "
f"Default: {Toolchain.PLATFORMIO.value}."
"Default: the platform's native toolchain where it has one, else "
f"{Toolchain.PLATFORMIO.value}."
),
)
@@ -2403,6 +2319,7 @@ def parse_args(argv):
help="Only generate source code, do not compile.",
action="store_true",
)
_add_skip_bootloader_arg(parser_compile)
parser_upload = subparsers.add_parser(
"upload",
@@ -2466,7 +2383,7 @@ def parse_args(argv):
"-r",
action="store_true",
help="Reset the device before starting serial logs.",
default=os.getenv("ESPHOME_SERIAL_LOGGING_RESET"),
default=get_bool_env("ESPHOME_SERIAL_LOGGING_RESET"),
)
_add_states_args(parser_logs)
@@ -2499,6 +2416,7 @@ def parse_args(argv):
parser_run.add_argument(
"--no-logs", help="Disable starting logs.", action="store_true"
)
_add_skip_bootloader_arg(parser_run)
_add_states_args(parser_run)
@@ -2507,7 +2425,7 @@ def parse_args(argv):
"-r",
action="store_true",
help="Reset the device before starting serial logs.",
default=os.getenv("ESPHOME_SERIAL_LOGGING_RESET"),
default=get_bool_env("ESPHOME_SERIAL_LOGGING_RESET"),
)
parser_run.add_argument(
"--ota-platform",
@@ -2758,6 +2676,7 @@ def run_esphome(argv):
CORE.config_path = conf_path
CORE.dashboard = args.dashboard
CORE.skip_bootloader = getattr(args, "skip_bootloader", False)
if args.toolchain is not None:
# CLI toolchain wins over esp32.toolchain in YAML.
CORE.toolchain = args.toolchain
esphome/address_cache.py
+9 -2
View File
@@ -148,6 +148,13 @@ class AddressCache:
continue
hostname, ips = arg.split("=", 1)
# Normalize hostname for consistent lookups
normalized = normalize_hostname(hostname)
cache[normalized] = [ip.strip() for ip in ips.split(",")]
normalized = normalize_hostname(hostname.strip())
addresses = [ip for value in ips.split(",") if (ip := value.strip())]
if not normalized or not addresses:
_LOGGER.warning(
"Invalid cache entry: %s (hostname and at least one address are required)",
arg,
)
continue
cache[normalized] = addresses
return cache
esphome/analyze_memory/toolchain.py
+5 -1
View File
@@ -37,7 +37,7 @@ def find_elf_path(build_path: Path) -> Path | None:
"""
name = build_path.name
for candidate in (
# Native ESP-IDF: idf.py writes build/<name>.elf, which ESPHome copies
# Native ESP-IDF: the build writes build/<name>.elf, which ESPHome copies
# to build/firmware.elf (see espidf.toolchain.create_elf_copy)
build_path / "build" / "firmware.elf",
# PlatformIO
@@ -68,12 +68,16 @@ def idedata_candidates(build_path: Path) -> list[Path]:
The candidate idedata JSON paths, most specific first
"""
name = build_path.name
data_dir = build_path.parent.parent / "idedata"
# Native backends suffix the cache by toolchain (<name>.arduino.json)
suffixed = sorted(data_dir.glob(f"{name}.*.json")) if data_dir.is_dir() else []
return [
# In .pioenvs for test builds
build_path / ".pioenvs" / name / "idedata.json",
# Both toolchains cache it in the data dir, which holds this build dir:
# <data_dir>/idedata/<name>.json next to <data_dir>/build/<name>
build_path.parent.parent / "idedata" / f"{name}.json",
*suffixed,
# Regular builds, invoked from the config dir or from anywhere
Path.cwd() / ".esphome" / "idedata" / f"{name}.json",
Path.home() / ".esphome" / "idedata" / f"{name}.json",
esphome/api_client.py
+1 -3
View File
@@ -23,9 +23,7 @@ from esphome.util import safe_print
if TYPE_CHECKING:
from collections.abc import Callable
from aioesphomeapi.api_pb2 import (
SubscribeLogsResponse, # pylint: disable=no-name-in-module
)
from aioesphomeapi.api_pb2 import SubscribeLogsResponse # pylint: disable=no-name-in-module
_LOGGER = logging.getLogger(__name__)
esphome/arduino/library.py
+33 -15
View File
@@ -2,7 +2,9 @@
Bundled names build straight from the framework tree; everything else goes
through ``esphome.platformio.library``. Mirrors ``lib_ldf_mode=off``: each
library builds its own archive; all include dirs join one global path.
library builds its own archive; all include dirs join one global path. The
host build reuses it without a framework tree: nothing is bundled there and
every name resolves from the registry.
Deviations from PlatformIO: flat-layout libraries get the recursive default
source filter; ``dot_a_linkage`` is honored; bundled libraries never run a
@@ -342,13 +344,24 @@ def _check_unfulfilled_provides(
def resolve_libraries(
framework_path: Path, *, pio_platform: str, board_mcu: str, cache_key: str
framework_path: Path | None,
*,
pio_platform: str,
board_mcu: str,
cache_key: str,
framework: str | None = "arduino",
manifest_optional: bool = False,
) -> list[ArduinoLibrary]:
"""Resolve every ``cg.add_library()`` entry into an :class:`ArduinoLibrary`.
``pio_platform``/``board_mcu`` filter manifests the way PlatformIO would
for that core (e.g. ``espressif8266``/``esp8266``); ``cache_key`` keys the
shared converter's download cache.
shared converter's download cache. ``framework`` is the manifest
framework token the compatibility check warns about; None skips it.
A None ``framework_path`` means no core-bundled libraries exist (the
host build): every name resolves from the registry.
``manifest_optional`` accepts libraries without a manifest, built with
PlatformIO's default layout.
The returned list is not topologically sorted, so the caller must link
the archives inside one ``--start-group``/``--end-group`` pair (the
@@ -359,18 +372,22 @@ def resolve_libraries(
# PlatformIO's lib_ignore covers framework-bundled libraries too; the
# shared converter only filters the registry/git ones.
lib_ignore = lib_ignore_set()
# Exact directory names keep membership case-sensitive everywhere
# (an is_dir() probe would match "wire" on macOS/Windows and build
# the bundled Wire twice)
libraries_dir = framework_path / "libraries"
if not libraries_dir.is_dir():
# A registry fallback would fail later with a misleading
# package-not-found error per bundled name
raise EsphomeError(
f"{libraries_dir} is missing; the framework install may be "
"incomplete (run 'esphome clean-all')"
bundled_dir_names: frozenset[str] = frozenset()
if framework_path is not None:
# Exact directory names keep membership case-sensitive everywhere
# (an is_dir() probe would match "wire" on macOS/Windows and build
# the bundled Wire twice)
libraries_dir = framework_path / "libraries"
if not libraries_dir.is_dir():
# A registry fallback would fail later with a misleading
# package-not-found error per bundled name
raise EsphomeError(
f"{libraries_dir} is missing; the framework install may be "
"incomplete (run 'esphome clean-all')"
)
bundled_dir_names = frozenset(
p.name for p in libraries_dir.iterdir() if p.is_dir()
)
bundled_dir_names = frozenset(p.name for p in libraries_dir.iterdir() if p.is_dir())
def _provided(name: object) -> bool:
return _is_safe_library_name(name) and name in bundled_dir_names
@@ -497,12 +514,13 @@ def resolve_libraries(
backend = LibraryBackend(
platform=pio_platform,
framework="arduino",
framework=framework,
emit=_emit,
cache_key=cache_key,
# The walk must not resolve bundled names from the registry;
# _add_bundled_dependencies adds them after emit
provides=_provided,
manifest_optional=manifest_optional,
)
if external:
convert_libraries(external, backend)
esphome/arduino8266/framework.py
+107 -59
View File
@@ -3,12 +3,12 @@
Artifacts land in a machine-global cache (shared across projects, like the
ESP-IDF install in ``esphome.espidf.framework``):
<cache>/arduino8266/frameworks/<version>/ framework-arduinoespressif8266
<cache>/arduino8266/toolchains/<version>/ toolchain-xtensa (gcc 10.3)
<cache>/arduino8266/frameworks/<tag>/ the Arduino core
<cache>/arduino8266/toolchains/<tag>/ xtensa-lx106-elf gcc 10.3
Packages come from the PlatformIO registry (identical bits to the PlatformIO
backend); ``ESPHOME_ARDUINO8266_*_MIRRORS`` overrides the URLs. ninja comes
from PATH or the ninja PyPI wheel.
Both come from esphome-libs releases pinned below;
``ESPHOME_ARDUINO8266_*_MIRRORS`` overrides the URLs, with ``{VERSION}``
standing for the release tag. ninja comes from PATH or the ninja PyPI wheel.
"""
from __future__ import annotations
@@ -17,18 +17,74 @@ import os
from pathlib import Path
from typing import NamedTuple
from esphome.build_helpers.ccache import ccache_defaults_env
from esphome.build_helpers.ccache import ccache_env
from esphome.build_helpers.ninja import find_ninja
from esphome.build_helpers.tools_cache import ARDUINO8266_TOOLS_CACHE, tools_cache_path
from esphome.core import EsphomeError, Version
from esphome.framework_helpers import str_to_lst_of_str
from esphome.platformio.registry import install_package, prefetch_packages
from esphome.platformio.registry import (
Download,
PackageSpec,
Resolver,
get_systype,
install_packages,
prefetch_packages,
)
FRAMEWORK_PACKAGE = "framework-arduinoespressif8266"
TOOLCHAIN_PACKAGE = "toolchain-xtensa"
FRAMEWORK_PACKAGE = "arduino-esp8266"
_FRAMEWORK_RELEASES = "https://github.com/esphome-libs/arduino-esp8266/releases/"
class FrameworkRelease(NamedTuple):
tag: str
sha256: str
size: int
def download(self) -> Download:
archive = f"{FRAMEWORK_PACKAGE}-{self.tag}.tar.gz"
url = f"{_FRAMEWORK_RELEASES}download/{self.tag}/{archive}"
return Download(url, self.sha256, self.size)
# Arduino core version -> its build in esphome-libs/arduino-esp8266
FRAMEWORK_RELEASES: dict[Version, FrameworkRelease] = {
Version(3, 1, 2): FrameworkRelease(
"3.1.2-esphome.1",
"e80751e3123676b967143e39c61f2d8693946db4c7806f2a83dcaaf797ecd582",
37189311,
),
}
TOOLCHAIN_PACKAGE = "toolchain-xtensa-lx106-elf"
# gcc 10.3, the toolchain Arduino core 3.x builds with; the build
# generator's compile flags are tuned to it.
TOOLCHAIN_VERSION = "2.100300.220621"
TOOLCHAIN_VERSION = "10.3.0-esphome.2"
_TOOLCHAIN_RELEASES = (
"https://github.com/esphome-libs/xtensa-lx106-elf-toolchain/releases/"
)
# Registry system tag -> (sha256, size) of that host's archive
TOOLCHAIN_BUILDS: dict[str, tuple[str, int]] = {
"darwin_arm64": (
"849cede44d4d5c6ea0f14099783239f559f46327bea314281814f2652b486201",
60830321,
),
"darwin_x86_64": (
"ca69904daabf0c5983b372423e5e62f49182a793e992c052e94666852470c897",
64149487,
),
"linux_aarch64": (
"60a49a4f082bf246544bd409a9517dbbcab19bb30ac9decbee544b896aaccbd6",
67573397,
),
"linux_x86_64": (
"1fba33ca1494ec79f2776e0e37eca93282d30f8bb9992f5f4f9a655d6fff1db4",
68431336,
),
"windows_amd64": (
"af9066b0e5bf036f04f2bd9d08b89b81a7f183c57dac0abcaff71dd861cf5f3b",
67664137,
),
}
ESPHOME_ARDUINO8266_FRAMEWORK_MIRRORS = str_to_lst_of_str(
os.environ.get("ESPHOME_ARDUINO8266_FRAMEWORK_MIRRORS", "")
@@ -44,36 +100,40 @@ def get_arduino8266_tools_path() -> Path:
return tools_cache_path(*ARDUINO8266_TOOLS_CACHE)
# 3.1.1 rather than 3.1.0: the registry has no packages for 3.0.0, 3.0.1 or 3.1.0
MIN_FRAMEWORK_VERSION = Version(3, 1, 1)
def framework_package_version(ver: Version) -> str:
"""Map an Arduino core version to its registry package version (3.1.2 ->
3.30102.0; the leading 3 is the package major).
Exact registry names for 3.x cores; callers floor at MIN_FRAMEWORK_VERSION.
"""
if ver.major > 3:
def framework_release(version: Version) -> FrameworkRelease:
if (release := FRAMEWORK_RELEASES.get(version)) is None:
raise EsphomeError(
f"Arduino core {ver} is not supported yet; "
"the newest known core series is 3.x"
f"'toolchain: arduino' has no build of Arduino core {version}; "
f"available: {', '.join(str(v) for v in FRAMEWORK_RELEASES)}. "
"Use one of those or 'toolchain: platformio'"
)
if ver.major < 3:
raise EsphomeError(
f"Arduino core {ver} is not supported; ESPHome requires core 3.x"
)
return f"3.{ver.major}{ver.minor:02d}{ver.patch:02d}.0"
return release
def get_framework_path(package_version: str) -> Path:
return get_arduino8266_tools_path() / "frameworks" / package_version
def get_framework_path(tag: str) -> Path:
return get_arduino8266_tools_path() / "frameworks" / tag
def get_toolchain_path() -> Path:
return get_arduino8266_tools_path() / "toolchains" / TOOLCHAIN_VERSION
def toolchain_download() -> Download:
"""The toolchain archive for the current host."""
systype = get_systype()
if (build := TOOLCHAIN_BUILDS.get(systype)) is None:
raise EsphomeError(
f"There is no ESP8266 toolchain for this system ({systype}); "
f"supported systems are {', '.join(sorted(TOOLCHAIN_BUILDS))}. "
"Either set 'toolchain: platformio' under 'esp8266:', or point "
"ESPHOME_ARDUINO8266_TOOLCHAIN_MIRRORS at a toolchain archive"
)
sha256, size = build
archive = f"{TOOLCHAIN_PACKAGE}-{TOOLCHAIN_VERSION}-{systype}.tar.gz"
url = f"{_TOOLCHAIN_RELEASES}download/{TOOLCHAIN_VERSION}/{archive}"
return Download(url, sha256, size)
class InstalledPaths(NamedTuple):
"""Locations of the installed framework, toolchain, and ninja binary."""
@@ -84,29 +144,22 @@ class InstalledPaths(NamedTuple):
def check_and_install(framework_version: Version) -> InstalledPaths:
"""Ensure framework, toolchain, and ninja are installed; return their paths."""
if framework_version < MIN_FRAMEWORK_VERSION:
# Config validation enforces this too; keep the module honest when
# called directly.
raise EsphomeError(
f"The native toolchain requires the Arduino core "
f">= {MIN_FRAMEWORK_VERSION}, got {framework_version}"
)
release = framework_release(framework_version)
# Probe the cheap local dependency before ~110 MB of downloads
ninja_path = find_ninja()
package_version = framework_package_version(framework_version)
framework_path = get_framework_path(package_version)
framework_path = get_framework_path(release.tag)
downloads_dir = get_arduino8266_tools_path() / "downloads"
toolchain_path = get_toolchain_path()
# One spec per package: the prefetch and the installs must agree
specs = (
(
PackageSpec(
FRAMEWORK_PACKAGE,
package_version,
release.tag,
framework_path,
ESPHOME_ARDUINO8266_FRAMEWORK_MIRRORS,
("cores/esp8266", "tools/sdk", "libraries"),
),
(
PackageSpec(
TOOLCHAIN_PACKAGE,
TOOLCHAIN_VERSION,
toolchain_path,
@@ -115,10 +168,17 @@ def check_and_install(framework_version: Version) -> InstalledPaths:
("bin", "xtensa-lx106-elf"),
),
)
# Fetch both archives at once; the installs below verify and extract
prefetch_packages([spec[:4] for spec in specs], downloads_dir)
for name, version, dest, mirrors, expect in specs:
install_package(name, version, dest, mirrors, downloads_dir, expect=expect)
# Resolved only when a download is needed, so an installed toolchain
# keeps working on a host without a build; a mirror override wins
resolvers: dict[str, Resolver] = {}
if not ESPHOME_ARDUINO8266_FRAMEWORK_MIRRORS:
resolvers[FRAMEWORK_PACKAGE] = release.download
if not ESPHOME_ARDUINO8266_TOOLCHAIN_MIRRORS:
resolvers[TOOLCHAIN_PACKAGE] = toolchain_download
# Fetch both archives at once; the installs verify and extract them.
# One spec list for both, so the two phases cannot drift.
prefetch_packages(specs, downloads_dir, resolvers)
install_packages(specs, downloads_dir, resolvers)
return InstalledPaths(
framework=framework_path, toolchain=toolchain_path, ninja=ninja_path
)
@@ -143,17 +203,5 @@ def get_build_env(toolchain_path: Path, ccache: str | None) -> dict[str, str]:
*filter(None, env.get("PATH", "").split(os.pathsep)),
]
env["PATH"] = os.pathsep.join(parts)
env.update(ccache_env(ccache))
env.update(ccache_env(ccache, ARDUINO8266_TOOLS_CACHE))
return env
def ccache_env(ccache: str | None) -> dict[str, str]:
"""Return ccache settings for the build subprocess (not os.environ).
``ccache`` is the pre-resolved binary (resolve_ccache_path), or None
when disabled. Values the user already set in the environment are
respected.
"""
if ccache is None:
return {}
return ccache_defaults_env(get_arduino8266_tools_path() / "ccache")
esphome/arduino8266/toolchain.py
+212
View File
@@ -0,0 +1,212 @@
"""Native Arduino ESP8266 build driver (the PlatformIO ``run`` equivalent)."""
from __future__ import annotations
import logging
from pathlib import Path
import subprocess
from typing import TYPE_CHECKING
from esphome.build_helpers.ccache import resolve_absolute_ccache_path
from esphome.build_helpers.native import warn_ignored_platformio_options
from esphome.build_helpers.ninja import refresh_compile_commands
from esphome.const import (
CONF_COMPILE_PROCESS_LIMIT,
CONF_ESPHOME,
KEY_CORE,
KEY_FRAMEWORK_VERSION,
)
from esphome.core import CORE
from esphome.types import ConfigType
if TYPE_CHECKING:
from esphome.arduino8266.framework import InstalledPaths
_LOGGER = logging.getLogger(__name__)
# ESP8266 user RAM (matches upload.maximum_ram_size in every board manifest)
_MAX_RAM_SIZE = 81920
_RAM_SECTIONS = (".data", ".rodata", ".bss")
_FLASH_SECTIONS = (".irom0.text", ".text", ".text1", ".data", ".rodata")
def get_build_dir() -> Path:
return CORE.relative_pioenvs_path(CORE.name)
def get_elf_path() -> Path:
return get_build_dir() / "firmware.elf"
def _toolchain_tool(name: str) -> Path:
# Imported here, not at module scope: the serial upload/logs fast path
# resolves this module for its artifact paths alone, and framework
# pulls in the whole package-download stack
from esphome.arduino8266 import framework
return framework.toolchain_tool(framework.get_toolchain_path(), name)
def get_factory_firmware_path() -> Path:
"""The image to serial-flash at 0x0 (same bytes as firmware.bin: the
8266 factory copy exists for artifact-contract parity, not content)."""
return get_build_dir() / "firmware.factory.bin"
def get_addr2line_path() -> Path:
return _toolchain_tool("addr2line")
def get_objdump_path() -> Path:
return _toolchain_tool("objdump")
def get_readelf_path() -> Path:
return _toolchain_tool("readelf")
def run_compile(config: ConfigType, verbose: bool) -> int:
from esphome.arduino8266 import framework
from esphome.build_gen import arduino8266 as build_gen
from esphome.core.config import NATIVE_ARDUINO_CONSUMED_PIO_OPTIONS
warn_ignored_platformio_options(NATIVE_ARDUINO_CONSUMED_PIO_OPTIONS)
paths = framework.check_and_install(CORE.data[KEY_CORE][KEY_FRAMEWORK_VERSION])
# Resolved once: the probe is not free and three consumers need it
ccache = resolve_absolute_ccache_path()
ninja_changed = build_gen.write_project(paths, ccache)
build_dir = get_build_dir()
env = framework.get_build_env(paths.toolchain, ccache)
refresh_compile_commands(paths.ninja, build_dir, env, ninja_changed)
cmd = [str(paths.ninja)]
if verbose:
cmd.append("-v")
if jobs := config[CONF_ESPHOME].get(CONF_COMPILE_PROCESS_LIMIT):
cmd += ["-j", str(jobs)]
# Explicit targets: a generator defect that drops them fails loudly
# instead of a green no-op run leaving stale artifacts in place
targets = ["firmware.factory.bin", "firmware.ota.bin"]
cmd += targets
# cwd, not -C: drops ninja's "Entering directory" banner
_LOGGER.debug("Running: %s", " ".join(cmd))
rc = subprocess.run(
cmd, cwd=build_dir, env=env, check=False, close_fds=False
).returncode
if rc != 0:
return rc
# ninja already refused missing targets; existence covers a rule that
# ran but wrote elsewhere
build_dir_artifacts = (
get_elf_path(),
build_dir / "firmware.bin",
get_factory_firmware_path(),
build_dir / "firmware.ota.bin",
)
for artifact in build_dir_artifacts:
if not artifact.is_file():
_LOGGER.error("Build produced no %s", artifact)
return 1
if not _print_size_summary(build_dir, paths):
# Cause already warned; name the consequence for CI harnesses
_LOGGER.warning("Firmware size summary unavailable for this build")
from esphome.build_helpers.idedata import warn_if_idedata_missing
warn_if_idedata_missing(lambda: get_idedata(ccache))
return 0
def _parse_app_size(build_dir: Path, paths: InstalledPaths) -> int | None:
"""Read the app flash budget (irom0_0_seg length) from the linker script."""
from esphome.build_gen.arduino8266 import get_flash_ld_path
from esphome.components.esp8266.build_surgery import segment_length
# Warnings, not debug: without the app size the Flash summary line is
# dropped and CI's memory-impact extraction loses its flash metric.
ld_path = get_flash_ld_path(build_dir, paths)
try:
ld_text = ld_path.read_text(encoding="utf-8")
except (OSError, UnicodeDecodeError) as err:
# A corrupt script degrades the same way, never aborts the build
_LOGGER.warning("Cannot read linker script for the Flash summary: %s", err)
return None
if not (app_size := segment_length(ld_text, "irom0_0_seg")):
_LOGGER.warning("No usable irom0_0_seg in %s; skipping Flash summary", ld_path)
return None
return app_size
def _print_size_summary(build_dir: Path, paths: InstalledPaths) -> bool:
"""Print the RAM/Flash lines ``ci_memory_impact_extract.py`` parses;
False when skipped."""
from esphome.arduino8266.framework import toolchain_tool
from esphome.build_helpers.size_summary import print_size_line
try:
result = subprocess.run(
[
str(toolchain_tool(paths.toolchain, "size")),
"-A",
"-d",
str(build_dir / "firmware.elf"),
],
capture_output=True,
text=True,
check=True,
close_fds=False,
)
except (OSError, subprocess.CalledProcessError) as err:
# The summary is a bonus artifact like idedata; a truncated
# toolchain extraction must not discard an already-linked build
_LOGGER.warning("Could not summarize firmware size: %s", err)
return False
# -d prints decimal sizes; anything else trips the missing-sections guard
sections = {
parts[0]: int(parts[1])
for line in result.stdout.splitlines()
if (parts := line.split())[:1] and parts[0].startswith(".") and len(parts) >= 2
if parts[1].isdigit()
}
if missing := set(_RAM_SECTIONS + _FLASH_SECTIONS) - set(sections):
# A defaulted 0 would print a confidently wrong total for CI's metric
_LOGGER.warning(
"Size output is missing section(s) %s; skipping the size summary",
", ".join(sorted(missing)),
)
return False
# Resolve the flash budget before printing: a RAM line without its
# Flash line would skew CI's memory-impact extraction
app_size = _parse_app_size(build_dir, paths)
if not app_size:
return False
ram = sum(sections[s] for s in _RAM_SECTIONS)
flash = sum(sections[s] for s in _FLASH_SECTIONS)
print_size_line("RAM", ram, _MAX_RAM_SIZE)
print_size_line("Flash", flash, app_size)
return True
def get_idedata(ccache: str | None = None) -> dict | None:
"""Derive idedata from the build's compile_commands.json (same
contract as ``espidf.toolchain.get_idedata``)."""
from esphome.build_helpers.idedata import load_or_build_idedata
# A disabled ccache resolves to None without spawning anything, so
# re-resolving here costs nothing when the caller has no answer
launcher = ccache or resolve_absolute_ccache_path()
return load_or_build_idedata(
get_build_dir() / "compile_commands.json",
get_elf_path(),
# Suffixed so a platformio->arduino->platformio round trip on one
# config never serves the other toolchain's cache shape
CORE.relative_internal_path("idedata", f"{CORE.name}.arduino.json"),
# The compile DB's commands carry the same ccache prefix the ninja
# rules were generated with
launcher=str(launcher) if launcher else None,
)
esphome/automation.py
+437 -45
View File
@@ -1,5 +1,8 @@
from collections.abc import Callable
from dataclasses import dataclass, field
import logging
import string
from typing import Any
import esphome.codegen as cg
import esphome.config_validation as cv
@@ -18,18 +21,46 @@ from esphome.const import (
CONF_TYPE_ID,
CONF_UPDATE_INTERVAL,
)
from esphome.core import ID, Lambda
from esphome.core import CORE, ID, EsphomeError, HexInt, Lambda
from esphome.cpp_generator import (
FlashStringLiteral,
LambdaExpression,
MockObj,
MockObjClass,
TemplateArgsType,
call_lambda,
)
from esphome.schema_extractors import SCHEMA_EXTRACT, schema_extractor
from esphome.types import ConfigType
from esphome.types import ConfigType, SafeExpType
from esphome.util import Registry
def progmem_bytes(name: str, data: bytes | list[int]) -> MockObj:
"""Shared PROGMEM table for constant bytes; equal payloads share one, empty is nullptr."""
if not data:
return cg.nullptr
return cg.shared_progmem_array(
name, cg.uint8, cg.ArrayInitializer(*(HexInt(x) for x in data))
)
async def templatable_bytes(
value: Any,
args: TemplateArgsType,
set_template: MockObj,
set_static: MockObj,
table_name: str,
) -> None:
"""Set a TemplatableBytes: a lambda via set_template, constant bytes via set_static."""
if cg.is_template(value):
fn = await cg.templatable(value, args, cg.std_vector.template(cg.uint8))
cg.add(set_template(fn))
elif len(value) > 0xFFFF:
raise EsphomeError(f"Byte payload is {len(value)} bytes; the maximum is 65535")
else:
cg.add(set_static(progmem_bytes(table_name, value), len(value)))
def maybe_simple_id(*validators):
"""Allow a raw ID to be specified in place of a config block.
If the value that's being validated is a dictionary, it's passed as-is to the specified validators. Otherwise, it's
@@ -57,6 +88,7 @@ def maybe_conf(conf, *validators):
with cv.remove_prepend_path([conf]):
return validator({conf: value})
validate.inner_schema = validator
return validate
@@ -102,6 +134,101 @@ def register_condition(name: str, condition_type: MockObjClass, schema: cv.Schem
return CONDITION_REGISTRY.register(name, condition_type, schema)
async def _build_with_parent(
config: ConfigType,
automation_id: ID,
template_arg: cg.TemplateArguments,
args: TemplateArgsType,
) -> MockObj:
parent = await cg.get_variable(config[CONF_ID])
return cg.new_Pvariable(automation_id, template_arg, parent)
async def _build_without_parent(
config: ConfigType,
automation_id: ID,
template_arg: cg.TemplateArguments,
args: TemplateArgsType,
) -> MockObj:
return cg.new_Pvariable(automation_id, template_arg)
async def _build_parented(
config: ConfigType,
automation_id: ID,
template_arg: cg.TemplateArguments,
args: TemplateArgsType,
) -> MockObj:
var = cg.new_Pvariable(automation_id, template_arg)
await cg.register_parented(var, config[CONF_ID])
return var
def register_simple_action(
name: str,
action_type: MockObjClass,
schema: cv.Schema,
*,
synchronous: bool,
) -> None:
"""Register an action whose constructor takes the object named by ``config[CONF_ID]``.
Use the ``register_action`` decorator instead when the builder must also set fields.
"""
register_action(name, action_type, schema, synchronous=synchronous)(
_build_with_parent
)
def register_simple_condition(
name: str, condition_type: MockObjClass, schema: cv.Schema
) -> None:
"""Condition counterpart of ``register_simple_action``."""
register_condition(name, condition_type, schema)(_build_with_parent)
def register_bare_action(
name: str,
action_type: MockObjClass,
schema: cv.Schema,
*,
synchronous: bool,
) -> None:
"""Register an action whose constructor takes no arguments."""
register_action(name, action_type, schema, synchronous=synchronous)(
_build_without_parent
)
def register_bare_condition(
name: str, condition_type: MockObjClass, schema: cv.Schema
) -> None:
"""Condition counterpart of ``register_bare_action``."""
register_condition(name, condition_type, schema)(_build_without_parent)
def register_parented_action(
name: str,
action_type: MockObjClass,
schema: cv.Schema,
*,
synchronous: bool,
) -> None:
"""Register an action deriving from ``Parented<T>``.
The object is constructed without arguments and ``set_parent()`` receives the object
named by ``config[CONF_ID]``.
"""
register_action(name, action_type, schema, synchronous=synchronous)(_build_parented)
def register_parented_condition(
name: str, condition_type: MockObjClass, schema: cv.Schema
) -> None:
"""Condition counterpart of ``register_parented_action``."""
register_condition(name, condition_type, schema)(_build_parented)
Action = cg.esphome_ns.class_("Action")
Trigger = cg.esphome_ns.class_("Trigger")
ACTION_REGISTRY = Registry()
@@ -112,6 +239,308 @@ validate_action_list = cv.validate_registry("action", ACTION_REGISTRY)
validate_condition = cv.validate_registry_entry("condition", CONDITION_REGISTRY)
validate_condition_list = cv.validate_registry("condition", CONDITION_REGISTRY)
ApplyAction = cg.esphome_ns.class_("ApplyAction", Action)
ApplyCondition = cg.esphome_ns.class_("ApplyCondition", Condition)
def flash_string(config: ConfigType, value: str) -> str:
"""Default renderer for ``std::string`` constants; copies the literal out of flash on ESP8266."""
if CORE.is_esp8266:
return f"progmem_string({FlashStringLiteral(value)})"
return str(cg.safe_exp(value))
def literal_with_length(config: ConfigType, value: str) -> str:
"""Renderer for a ``(const char *, size_t)`` target: a plain literal plus its byte length.
The target compares or copies the bytes in place, so it needs the RAM literal rather than
the PROGMEM rendering on ESP8266, and the length saves a strlen.
"""
return f"{cg.safe_exp(value)}, {len(value.encode('utf-8'))}"
@dataclass(frozen=True)
class ApplyCall:
"""One statement from config keys, e.g. ``"set_range({}, {})"`` with ``((CONF_LOW, cg.float_), ...)``.
Each arg is ``(conf_key, type_)`` or ``(conf_key, type_, const_fn)``. A ``conf_key`` may be a
path into nested sections. A plain ``str`` ``type_`` is raw C++ type text and may use
``{parent}``. ``const_fn(config, value)`` renders a constant's argument text; a lambda or an
id bypasses it. The statement is skipped when none of its keys is set, always emitted when it
has no keys, and a partial set is a config error.
"""
target: str
args: tuple[tuple[Any, ...], ...] = ()
def __post_init__(self) -> None:
fields = [
f for _, f, _, _ in string.Formatter().parse(self.target) if f is not None
]
if any(fields):
raise ValueError(
f"apply target {self.target!r}: only bare {{}} placeholders"
)
if len(fields) != len(self.args):
raise ValueError(
f"apply target {self.target!r} has {len(fields)} "
f"placeholder(s) for {len(self.args)} config key(s)"
)
if any(len(arg) not in (2, 3) for arg in self.args):
raise ValueError(
f"apply target {self.target!r}: each arg is (conf_key, type_[, const_fn])"
)
@property
def members(self) -> list[tuple[Any, Any, Any]]:
"""Each arg as ``(conf_key, type_, const_fn or None)``."""
return [
(arg[0], arg[1], arg[2] if len(arg) == 3 else None) for arg in self.args
]
@dataclass(frozen=True)
class ApplyField:
"""One config key forwarded as ``target(value)``, or as statement ``target`` when it has ``{}``.
Double a literal brace in a template. ``conf_key`` may be a path into nested sections.
``type_`` may be a C++ type string using ``{parent}`` when the type is only known per
instance. ``const_fn(config, value)`` renders a constant's argument text when ``cg.safe_exp``
is not the right spelling (unit conversion belongs in the validator); a lambda or an id
bypasses it, so the target must also take a plain ``type_``. An absent key emits nothing.
"""
conf_key: str | tuple[str, ...]
target: str
type_: SafeExpType
const_fn: Callable[[ConfigType, Any], str] | None = None
def call(self) -> ApplyCall:
target = self.target if "{}" in self.target else f"{self.target}({{}})"
return ApplyCall(target, ((self.conf_key, self.type_, self.const_fn),))
def _config_lookup(config: ConfigType, key: str | tuple[str, ...]) -> Any:
if isinstance(key, str):
return config.get(key)
for part in key:
if (config := config.get(part)) is None:
return None
return config
def _dict_schema(schema: Any) -> Any:
"""The dict-backed cv.Schema inside cv.All and maybe_* wrappers, or None; cv.Any is not inspected."""
if isinstance(schema, dict):
return cv.Schema(schema)
if isinstance(getattr(schema, "schema", None), dict):
return schema
if isinstance(schema, cv.All):
inner = schema.validators
else:
inner = (
getattr(schema, "inner_schema", None),
) # maybe_conf / maybe_simple_value
for candidate in inner:
if candidate is not None and (found := _dict_schema(candidate)) is not None:
return found
return None
def _check_key_in_schema(
name: str, schema: Any, conf_key: str | tuple[str, ...]
) -> None:
"""Reject a key path the schema does not have; a typo would otherwise be a silent no-op.
Only dict-backed schemas, also inside cv.All and maybe_* wrappers, can be checked.
"""
for part in (conf_key,) if isinstance(conf_key, str) else conf_key:
if (schema := _dict_schema(schema)) is None:
return
markers = {
getattr(marker, "schema", marker): marker for marker in schema.schema
}
if part not in markers:
raise ValueError(f"{name}: config key {part!r} is not in the schema")
schema = schema.schema[markers[part]]
async def _apply_parent(config: ConfigType, id_key: str = CONF_ID) -> str:
# Global-scope qualified so a trigger arg named like the id cannot shadow it.
return f"::{await cg.get_variable(config[id_key])}"
def _apply_lambda_args(args: TemplateArgsType) -> TemplateArgsType:
# The generated function's parameters; a std::string arg is never copied.
return [
(cg.RawExpression(f"const std::remove_cvref_t<{cg.safe_exp(t)}> &"), arg)
for t, arg in args
]
def _apply_function(
id_: ID,
return_type: SafeExpType,
template_arg: cg.TemplateArguments,
lambda_args: TemplateArgsType,
statements: list[str],
) -> MockObj:
"""Emit the generated function and declare ``id_`` as the ``ApplyAction`` or
``ApplyCondition`` templated on it, so ``play()`` calls it directly."""
fn = cg.static_function(
f"esphome__{id_.id}__fn", return_type, lambda_args, statements
)
return cg.new_Pvariable(id_, cg.TemplateArguments(fn, *template_arg))
async def _render_values(
name: str,
target: str,
members: list[tuple[Any, Any, Any]],
values: list[Any],
config: ConfigType,
parent: str,
lambda_args: TemplateArgsType,
compare: bool = False,
) -> list[str]:
"""Render the argument text of one statement; every key must be present.
``compare``: an inlined lambda expression is parenthesized so it binds as a whole
beside an operator.
"""
if any(value is None for value in values):
keys = [key for key, _, _ in members]
raise EsphomeError(f"{name}: {target!r} needs all of {keys}")
exprs: list[str] = []
for (_, type_, const_fn), value in zip(members, values, strict=True):
if isinstance(value, Lambda):
if isinstance(type_, str):
type_ = cg.RawExpression(type_.format(parent=parent))
inner = await cg.process_lambda(value, lambda_args, return_type=type_)
expr = call_lambda(inner)
bare = compare and isinstance(expr, cg.RawExpression)
exprs.append(f"({expr})" if bare else str(expr))
elif isinstance(value, ID):
# Qualified like the parent, so a trigger arg named like the id cannot shadow it.
exprs.append(f"::{await cg.get_variable(value)}")
elif const_fn is not None:
exprs.append(const_fn(config, value))
else:
exprs.append(str(cg.safe_exp(value)))
return exprs
def _apply_values(config: ConfigType, members: list[tuple[Any, Any, Any]]) -> list[Any]:
return [_config_lookup(config, key) for key, _, _ in members]
def register_apply_action(
name: str,
schema: cv.Schema,
*fields: ApplyField | ApplyCall,
call: str | None = None,
id_key: str = CONF_ID,
) -> None:
"""Register an action that only forwards config values to its parent, with no C++ class.
Generates one static function with the parent (read from ``id_key``) and constants baked
in, lambdas called inline with the trigger args, and an ``ApplyAction`` templated on it.
A constant that is an id (``cv.use_id`` under ``cv.templatable``) is the object it names.
With ``call`` every statement targets the call object ``auto apply_call = parent->call()``,
and ``apply_call.perform()`` is appended.
"""
# An action stores the value, so a std::string constant stays in flash on ESP8266.
statements_spec = [
(
c.target,
[
(key, t, fn or (flash_string if t is cg.std_string else None))
for key, t, fn in c.members
],
)
for c in (f if isinstance(f, ApplyCall) else f.call() for f in fields)
]
_check_key_in_schema(name, schema, id_key)
for _, members in statements_spec:
for conf_key, _, _ in members:
_check_key_in_schema(name, schema, conf_key)
async def builder(
config: ConfigType,
action_id: ID,
template_arg: cg.TemplateArguments,
args: TemplateArgsType,
) -> MockObj:
parent = await _apply_parent(config, id_key)
lambda_args = _apply_lambda_args(args)
receiver = "apply_call." if call else f"{parent}->"
statements: list[str] = []
for target, members in statements_spec:
values = _apply_values(config, members)
if members and all(value is None for value in values):
continue
exprs = await _render_values(
name, target, members, values, config, parent, lambda_args
)
statements.append(f"{receiver}{target.format(*exprs)};")
if call:
statements = [
f"auto apply_call = {parent}->{call}();",
*statements,
"apply_call.perform();",
]
return _apply_function(
action_id, cg.void, template_arg, lambda_args, statements
)
register_action(name, ApplyAction, schema, synchronous=True)(builder)
def register_apply_condition(
name: str, schema: cv.Schema, check: str | ApplyCall, id_key: str = CONF_ID
) -> None:
"""Register a condition that is one expression on its parent, with no C++ class.
``check`` is applied to the parent: ``"is_playing()"`` becomes ``parent->is_playing()``; an
``ApplyCall`` such as ``ApplyCall("state == {}", ((CONF_STATE, cg.bool_),))`` compares
against config values, all of which must be present. Write ``== false`` to negate.
String constants are plain literals, so compare a ``std::string`` or ``StringRef`` member.
Generates one static predicate and an ``ApplyCondition`` templated on it.
"""
call = check if isinstance(check, ApplyCall) else ApplyCall(check)
members = call.members
_check_key_in_schema(name, schema, id_key)
for conf_key, _, _ in members:
_check_key_in_schema(name, schema, conf_key)
async def builder(
config: ConfigType,
condition_id: ID,
template_arg: cg.TemplateArguments,
args: TemplateArgsType,
) -> MockObj:
parent = await _apply_parent(config, id_key)
lambda_args = _apply_lambda_args(args)
exprs = await _render_values(
name,
call.target,
members,
_apply_values(config, members),
config,
parent,
lambda_args,
compare=True,
)
return _apply_function(
condition_id,
cg.bool_,
template_arg,
lambda_args,
[f"return {parent}->{call.target.format(*exprs)};"],
)
register_condition(name, ApplyCondition, schema)(builder)
def validate_potentially_and_condition(value):
if isinstance(value, list):
@@ -359,28 +788,15 @@ async def for_condition_to_code(
return var
@register_condition(
register_apply_condition(
"component.is_idle",
LambdaCondition,
maybe_simple_id(
{
cv.Required(CONF_ID): cv.use_id(cg.Component),
}
),
"is_idle()",
)
async def component_is_idle_condition_to_code(
config: ConfigType,
condition_id: ID,
template_arg: cg.TemplateArguments,
args: TemplateArgsType,
) -> MockObj:
comp = await cg.get_variable(config[CONF_ID])
lambda_ = await cg.process_lambda(
Lambda(f"return {comp}->is_idle();"), args, return_type=bool
)
return new_lambda_pvariable(
condition_id, lambda_, StatelessLambdaCondition, template_arg
)
@register_action(
@@ -534,44 +950,20 @@ async def lambda_action_to_code(
return new_lambda_pvariable(action_id, lambda_, StatelessLambdaAction, template_arg)
@register_action(
register_simple_action(
"component.update",
UpdateComponentAction,
maybe_simple_id(
{
cv.Required(CONF_ID): cv.use_id(cg.PollingComponent),
}
),
maybe_simple_id({cv.Required(CONF_ID): cv.use_id(cg.PollingComponent)}),
synchronous=True,
)
async def component_update_action_to_code(
config: ConfigType,
action_id: ID,
template_arg: cg.TemplateArguments,
args: TemplateArgsType,
) -> MockObj:
comp = await cg.get_variable(config[CONF_ID])
return cg.new_Pvariable(action_id, template_arg, comp)
@register_action(
register_simple_action(
"component.suspend",
SuspendComponentAction,
maybe_simple_id(
{
cv.Required(CONF_ID): cv.use_id(cg.PollingComponent),
}
),
maybe_simple_id({cv.Required(CONF_ID): cv.use_id(cg.PollingComponent)}),
synchronous=True,
)
async def component_suspend_action_to_code(
config: ConfigType,
action_id: ID,
template_arg: cg.TemplateArguments,
args: TemplateArgsType,
) -> MockObj:
comp = await cg.get_variable(config[CONF_ID])
return cg.new_Pvariable(action_id, template_arg, comp)
@register_action(
esphome/build_gen/arduino8266.py
+1216
View File
File diff suppressed because it is too large Load Diff
esphome/build_gen/espidf.py
+258 -6
View File
@@ -3,7 +3,10 @@
import json
import logging
from pathlib import Path
import re
import textwrap
from esphome.build_helpers import pch
from esphome.components.esp32 import (
get_esp32_variant,
get_excluded_builtin_components,
@@ -18,7 +21,7 @@ from esphome.framework_helpers import (
get_project_cxx_compile_flags,
get_project_link_flags,
)
from esphome.helpers import mkdir_p, write_file_if_changed
from esphome.helpers import get_bool_env, mkdir_p, write_file_if_changed
_LOGGER = logging.getLogger(__name__)
@@ -33,6 +36,83 @@ list(FILTER esphome_cxx_compile_options EXCLUDE REGEX "^-std=")
list(APPEND esphome_cxx_compile_options "-std={standard}")
idf_build_set_property(CXX_COMPILE_OPTIONS "${{esphome_cxx_compile_options}}")"""
# Drops the app archive from ldgen's inputs so app-only edits skip the
# sections.ld regeneration. Safe: no mapping fragment references it
# (run_compile re-checks each build). Filters only the top-level call;
# the prior definition stays reachable with an underscore prefix.
_LDGEN_OVERRIDE = """\
if(COMMAND __ldgen_get_lib_deps_of_target)
set_property(GLOBAL PROPERTY ESPHOME_LDGEN_ARMED 1)
function(__ldgen_get_lib_deps_of_target target out_list_var)
if(NOT COMMAND ___ldgen_get_lib_deps_of_target)
message(FATAL_ERROR "ESPHome ldgen override lost the original "
"implementation; set ESPHOME_LDGEN_FULL_DEPS=1 and rebuild.")
endif()
___ldgen_get_lib_deps_of_target(${target} ${out_list_var})
if(out_list_var STREQUAL "ldgen_libraries")
set_property(GLOBAL PROPERTY ESPHOME_LDGEN_FILTERED 1)
list(LENGTH ${out_list_var} esphome_ldgen_before)
list(REMOVE_ITEM ${out_list_var} idf::src __idf_src)
list(LENGTH ${out_list_var} esphome_ldgen_after)
if(esphome_ldgen_before EQUAL esphome_ldgen_after)
message(@SEVERITY@ "ESPHome ldgen app archive exclusion matched "
"nothing; app edits will regenerate sections.ld.")
endif()
endif()
set(${out_list_var} "${${out_list_var}}" PARENT_SCOPE)
endfunction()
else()
message(@MISSING@ "ESPHome ldgen override target not found; "
"app edits will regenerate sections.ld.")
endif()"""
# lwip sources that compile to empty objects with the option off (their own
# #if guard). (option, regex valid for both Python and CMake); a source is
# only dropped when its option is defined and off, so a renamed option
# keeps it.
LWIP_EMPTY_SOURCES: tuple[tuple[str, str], ...] = (
("CONFIG_LWIP_PPP_SUPPORT", "/netif/ppp/"),
("CONFIG_LWIP_IPV6", "/core/ipv6/"),
("CONFIG_LWIP_AUTOIP", "/core/ipv4/autoip[.]c$"),
("CONFIG_LWIP_STATS", "/core/stats[.]c$"),
)
# Drift guard only: keep every lwip source.
LWIP_FULL_SOURCES_ENV = "ESPHOME_LWIP_FULL_SOURCES"
# Drops the empty objects after project(), once the lwip target exists.
_LWIP_EMPTY_SOURCES_FILTER = f"""\
idf_build_get_property(esphome_build_components BUILD_COMPONENTS)
if(lwip IN_LIST esphome_build_components AND NOT DEFINED ENV{{{LWIP_FULL_SOURCES_ENV}}})
idf_component_get_property(esphome_lwip_lib lwip COMPONENT_LIB)
get_target_property(esphome_lwip_srcs ${{esphome_lwip_lib}} SOURCES)
@FILTERS@
set_property(TARGET ${{esphome_lwip_lib}} PROPERTY SOURCES ${{esphome_lwip_srcs}})
endif()"""
def lwip_empty_source_gate(option: str, regex: str) -> str:
return (
f" if(DEFINED {option} AND NOT {option})\n"
f' list(FILTER esphome_lwip_srcs EXCLUDE REGEX "{regex}")\n'
" endif()"
)
def _lwip_empty_sources_filter() -> str:
gates = "\n".join(lwip_empty_source_gate(*entry) for entry in LWIP_EMPTY_SOURCES)
return _LWIP_EMPTY_SOURCES_FILTER.replace("@FILTERS@", gates)
# Runs after project() so the walk has happened; catches the remaining
# silent path where the top-level out-var was renamed.
_LDGEN_OVERRIDE_CHECK = """\
get_property(esphome_ldgen_armed GLOBAL PROPERTY ESPHOME_LDGEN_ARMED)
get_property(esphome_ldgen_filtered GLOBAL PROPERTY ESPHOME_LDGEN_FILTERED)
if(esphome_ldgen_armed AND NOT esphome_ldgen_filtered)
message(@SEVERITY@ "ESPHome ldgen override never filtered the app "
"archive; app edits will regenerate sections.ld.")
endif()"""
def get_available_components() -> list[str] | None:
"""List the built-in ESP-IDF components from ``project_description.json``.
@@ -78,6 +158,85 @@ def _cmake_quote(value: str) -> str:
return f'"{escaped}"'
# CONFIG_APP_BUILD_BOOTLOADER is hidden and force-selected, so it can only be
# cleared at the CMake level (the same state IDF's RAM-app build type uses).
# The macro is IDF's __build_process_project_includes plus a few added lines;
# the flag is ignored and the bootloader builds as usual if IDF changes it.
IDF_BOOTLOADER_OVERRIDE = """\
# ESPHome bootloader skip switch; see esphome/espidf/toolchain.py.
if(ESPHOME_SKIP_BOOTLOADER)
macro(__build_process_project_includes)
idf_build_get_property(sdkconfig_cmake SDKCONFIG_CMAKE)
include(${sdkconfig_cmake})
set(CONFIG_APP_BUILD_BOOTLOADER "")
# bt's CMakeLists reads the lowercase idf_target that the (now
# skipped) bootloader project_include leaks; keep it defined, or
# its empty TARGET_SRC_NAME sends file(GLOB_RECURSE) across /.
idf_build_get_property(idf_target IDF_TARGET)
# partition_table's V1 ECDSA signing reads this key, which the
# skipped bootloader project_include also sets.
get_filename_component(SECURE_BOOT_SIGNING_KEY "${CONFIG_SECURE_BOOT_SIGNING_KEY}" ABSOLUTE BASE_DIR "${project_dir}")
idf_build_get_property(build_properties __BUILD_PROPERTIES)
foreach(build_property ${build_properties})
idf_build_get_property(val ${build_property})
set(${build_property} "${val}")
endforeach()
idf_build_get_property(build_component_targets __BUILD_COMPONENT_TARGETS)
foreach(component_target ${build_component_targets})
__component_get_property(dir ${component_target} COMPONENT_DIR)
__component_get_property(_name ${component_target} COMPONENT_NAME)
set(COMPONENT_NAME ${_name})
set(COMPONENT_DIR ${dir})
set(COMPONENT_PATH ${dir})
if(EXISTS ${COMPONENT_DIR}/project_include.cmake)
include(${COMPONENT_DIR}/project_include.cmake)
endif()
endforeach()
endmacro()
endif()
"""
# The lines the override adds to IDF's macro; idf_macro_matches() below
# strips them before comparing with the live macro.
BOOTLOADER_OVERRIDE_ADDED_LINES = (
'set(CONFIG_APP_BUILD_BOOTLOADER "")',
"idf_build_get_property(idf_target IDF_TARGET)",
(
"get_filename_component(SECURE_BOOT_SIGNING_KEY"
' "${CONFIG_SECURE_BOOT_SIGNING_KEY}" ABSOLUTE BASE_DIR "${project_dir}")'
),
)
_MACRO = re.compile(
r"macro\(__build_process_project_includes\)(.*?)endmacro\(\)", re.DOTALL
)
def _normalized_macro(text: str) -> list[str] | None:
"""The macro body as comment-free, whitespace-collapsed lines."""
if (match := _MACRO.search(text)) is None:
return None
return [
re.sub(r"\s+", " ", line)
for raw in match.group(1).splitlines()
if (line := raw.split("#", 1)[0].strip())
]
_EXPECTED_MACRO = [
line
for line in _normalized_macro(IDF_BOOTLOADER_OVERRIDE)
if line not in BOOTLOADER_OVERRIDE_ADDED_LINES
]
def idf_macro_matches(idf_path: Path) -> bool:
"""Whether IDF's macro still matches the copy the override replays."""
build_cmake = idf_path / "tools" / "cmake" / "build.cmake"
live = _normalized_macro(build_cmake.read_text(encoding="utf-8"))
return live == _EXPECTED_MACRO
def get_project_cmakelists(
minimal: bool = False, builtin_components: list[str] | None = None
) -> str:
@@ -90,9 +249,10 @@ def get_project_cmakelists(
"""
idf_target = variant_to_idf_target(get_esp32_variant())
# esp_idf_size 2.x (bundled with IDF >=6.0) made NG the default and
# removed the --ng flag; on 1.x (IDF 5.5) --ng is required to get
# --format=raw because the legacy mode doesn't support it.
# esp_idf_size 2.x (IDF >=6.0) made NG the default and removed --ng;
# 1.x (IDF 5.5) needs --ng for --format=json2. 1.x json2 also lacks
# total_size, hence the ELF fallback in espidf/size_summary.py; both
# go away together when 1.x support is dropped.
size_ng_flag = "--ng" if idf_version() < cv.Version(6, 0, 0) else ""
# Project-wide compile options: -D defines and -W warning flags (skip
@@ -122,6 +282,22 @@ def get_project_cmakelists(
else ""
)
# Stops the ~3s sections.ld regeneration on app-only edits; see
# _LDGEN_OVERRIDE. ESPHOME_LDGEN_FULL_DEPS=1 restores stock behavior;
# ESPHOME_LDGEN_STRICT=1 (CI) fails the configure when an IDF bump
# breaks the override instead of degrading to stock deps.
if get_bool_env("ESPHOME_LDGEN_FULL_DEPS"):
ldgen_override = ""
ldgen_override_check = ""
else:
strict = get_bool_env("ESPHOME_LDGEN_STRICT")
severity = "FATAL_ERROR" if strict else "WARNING"
missing = "FATAL_ERROR" if strict else "STATUS"
ldgen_override = _LDGEN_OVERRIDE.replace("@SEVERITY@", severity).replace(
"@MISSING@", missing
)
ldgen_override_check = _LDGEN_OVERRIDE_CHECK.replace("@SEVERITY@", severity)
# CMake variables registered via cg.add_cmake_arg(). Emitted before
# include(project.cmake) so values like EXCLUDE_COMPONENTS are already
# set when project.cmake seeds the component list, and on minimal
@@ -199,6 +375,9 @@ set(EXTRA_COMPONENT_DIRS ${{CMAKE_SOURCE_DIR}}/src)
include($ENV{{IDF_PATH}}/tools/cmake/project.cmake)
{IDF_BOOTLOADER_OVERRIDE}
{ldgen_override}
{cpp_standard_options}
{cxx_compile_options}
@@ -211,12 +390,22 @@ include($ENV{{IDF_PATH}}/tools/cmake/project.cmake)
project({CORE.name})
# Emit raw JSON size data for ESPHome to read post-build.
{ldgen_override_check}
{_lwip_empty_sources_filter()}
# Emit per-memory-type JSON size data for ESPHome to read post-build.
# json2 stays small; raw dumps every symbol (~2s on a large map) and
# this command runs inside the link edge, blocking everything downstream.
# The map is a BYPRODUCT so ninja knows the link writes it; IDF's size
# target depends on the map and can then be built in the same run as all.
# IDF's cmakev2 declares the map itself, so drop this line on that switch.
add_custom_command(
TARGET ${{CMAKE_PROJECT_NAME}}.elf POST_BUILD
COMMAND ${{PYTHON}} -m esp_idf_size {size_ng_flag} --format=raw
COMMAND ${{PYTHON}} -m esp_idf_size {size_ng_flag} --format=json2
-o ${{CMAKE_BINARY_DIR}}/esp_idf_size.json
${{CMAKE_PROJECT_NAME}}.map
BYPRODUCTS ${{CMAKE_BINARY_DIR}}/${{CMAKE_PROJECT_NAME}}.map
WORKING_DIRECTORY ${{CMAKE_BINARY_DIR}}
VERBATIM
)
@@ -279,9 +468,72 @@ idf_component_register(
target_link_options(${{COMPONENT_LIB}} PUBLIC
{link_opts_str}
)
{_pch_cmake_block()}"""
# Where CMake puts the .gch of the src component; ccache reads the checksum
# next to it in place of the .gch
_PCH_SUM_PATH = "build/esp-idf/src/CMakeFiles/__idf_src.dir/cmake_pch.hxx.gch.sum"
# Where the Windows gate records its choice
_PCH_CHOICE_VAR = "ESPHOME_PCH"
def _pch_cmake_block() -> str:
"""The CMake block that precompiles the core headers for the C++ sources
of the src component; empty when disabled."""
if not pch.pch_enabled():
return ""
headers = "\n".join(
f' "$<$<COMPILE_LANGUAGE:CXX>:${{CMAKE_CURRENT_SOURCE_DIR}}/{header}>"'
for header in pch.PCH_DEFAULT_HEADERS
)
block = f"""target_precompile_headers(${{COMPONENT_LIB}} PRIVATE
{headers}
)"""
if not pch.pch_needs_gcc_check():
return f"\n# ESPHome precompiled header\n{block}\n"
# Before the first configure only CMake knows the compiler version
return f"""
# ESPHome precompiled header, unless GCC bug 14940 keeps it from loading
if(CMAKE_CXX_COMPILER_ID STREQUAL "GNU" AND ({pch.PCH_WINDOWS_CMAKE_OLD_GCC}))
message(STATUS "ESPHome: GCC ${{CMAKE_CXX_COMPILER_VERSION}} cannot load a precompiled header on Windows; compiling without it")
set({_PCH_CHOICE_VAR} OFF CACHE BOOL "ESPHome precompiled header in use" FORCE)
else()
set({_PCH_CHOICE_VAR} ON CACHE BOOL "ESPHome precompiled header in use" FORCE)
{textwrap.indent(block, " ")}
endif()
"""
def _read_if_exists(path: Path) -> str:
return path.read_text(encoding="utf-8") if path.is_file() else ""
def write_pch_checksum() -> None:
"""Write the checksum ccache uses in place of the .gch: the core headers,
the framework version, the sdkconfig and the managed component versions."""
from esphome.espidf.toolchain import get_cmake_cache_value
if not pch.pch_enabled():
return
# The gate's choice, cached by configure
if pch.pch_needs_gcc_check() and get_cmake_cache_value(_PCH_CHOICE_VAR) != "ON":
return
pch.log_pch_in_use()
checksum = pch.pch_checksum(
CORE.relative_src_path(),
pch.PCH_DEFAULT_HEADERS,
(
str(idf_version()),
_read_if_exists(CORE.relative_build_path(f"sdkconfig.{CORE.name}")),
_read_if_exists(CORE.relative_build_path("dependencies.lock")),
),
)
path = CORE.relative_build_path(_PCH_SUM_PATH)
path.parent.mkdir(parents=True, exist_ok=True)
write_file_if_changed(path, checksum + "\n")
def write_project(
minimal: bool = False, builtin_components: list[str] | None = None
) -> None:
esphome/build_gen/host.py
+303
View File
@@ -0,0 +1,303 @@
"""Native ninja build generator for the host platform.
Emits ``build.ninja`` under ``.pioenvs/<name>/``: every source in the
generated ``src/`` tree plus the resolved registry libraries compiles with
the machine's compiler and links into ``program``, the name PlatformIO's
native platform produced. Build flags route the way SCons's ``ParseFlags``
did under PlatformIO: ``-D``/``-I``/``-std=``/``-W`` shapes reach the
compile lines only, ``-l``/``-L``/``-Wl,`` the link line only, everything
else both.
"""
from __future__ import annotations
from collections.abc import Iterable
import logging
import os
from pathlib import Path
import subprocess
import sys
from typing import TYPE_CHECKING
from esphome.build_helpers.ninja import escape as _e, quote_path as _q, shell_token
from esphome.build_helpers.ninja_gen import (
PATH_ARG_FLAGS,
Flag,
anchor_path_flag,
ar_rule_lines,
collect_sources,
compile_edges,
compile_rule_lines,
library_edges,
pch_edges,
pch_rule_lines,
tool_lines,
)
from esphome.build_helpers.pch import PCH_DEFAULT_HEADERS, pch_enabled
from esphome.core import CORE, EsphomeError
from esphome.framework_helpers import get_project_cxx_compile_flags
from esphome.helpers import mkdir_p, write_file_if_changed
from esphome.host.toolchain import PROGRAM_NAME, HostCompilers, find_tool, get_build_dir
from esphome.platformio.library import lex_build_flags
if TYPE_CHECKING:
from esphome.arduino.library import ArduinoLibrary
_LOGGER = logging.getLogger(__name__)
# The PlatformIO platform the host built under; registry manifests declare
# compatibility against it, as lib_compat_mode=strict checked before
PIO_PLATFORM = "native"
# Namespaces the shared library download cache (pio_components/host/)
LIBRARY_CACHE_KEY = "host"
# Flag shapes that only the compiler understands; dropped from the link line
_COMPILE_ONLY_PREFIXES = ("-D", "-U", "-I", "-std=", "-W", *PATH_ARG_FLAGS)
# Flag shapes that only the linker consumes; inert on a -c compile line
_LINK_ONLY_PREFIXES = ("-l", "-L", "-Wl,")
# Link-only flags whose argument is the next token
_LINK_ONLY_ARG_FLAGS = ("-framework", "-Xlinker", "-z")
def parse_flags(entries: Iterable[str], owner: str) -> list[Flag]:
"""Lex build flag entries into flags, each with its argument.
Entries are a set, so their order is not the user's: a flag and the
argument it takes as the next token must share one entry.
"""
flags: list[Flag] = []
for entry in entries:
it = iter(lex_build_flags(entry, owner))
for tok in it:
if tok not in PATH_ARG_FLAGS and tok not in _LINK_ONLY_ARG_FLAGS:
flags.append((tok,))
continue
arg = next(it, None)
# A path never starts with "-"; that is the next flag
if arg is None or (tok in PATH_ARG_FLAGS and arg.startswith("-")):
raise EsphomeError(
f"{owner} build flags have '{tok}' with no argument; write "
f"the flag and its argument as one entry"
)
flags.append((tok, arg))
return flags
def split_flags(flags: list[Flag]) -> tuple[list[Flag], list[Flag]]:
"""Route build flags to the compile and link lines."""
compile_flags: list[Flag] = []
link_flags: list[Flag] = []
for flag in flags:
name = flag[0]
if len(flag) > 1:
(compile_flags if name in PATH_ARG_FLAGS else link_flags).append(flag)
elif name.startswith(_LINK_ONLY_PREFIXES):
# Checked before the compile prefixes: -Wl, would match -W
link_flags.append(flag)
elif name.startswith(_COMPILE_ONLY_PREFIXES):
compile_flags.append(flag)
else:
# -g, -O, -f*, -m*, -pthread, --coverage: both lines, as SCons
compile_flags.append(flag)
link_flags.append(flag)
return compile_flags, link_flags
def _is_std(flag: Flag) -> bool:
return flag[0].startswith("-std=")
def _is_cxx_std(flag: Flag) -> bool:
return _is_std(flag) and "++" in flag[0]
def _anchored_flags(entries: Iterable[str], owner: str) -> list[Flag]:
build_path = Path(CORE.build_path)
return [
anchor_path_flag(flag, build_path)
for flag in parse_flags(sorted(entries), owner)
]
def _flag_lists() -> tuple[list[str], list[str], list[str]]:
"""The C, C++, and link flag lists (raw tokens), build_unflags applied.
``cg.set_cpp_standard`` wins over any ``-std=`` in the build flags for
C++ compiles, as PlatformIO's unflag of every other standard did; C
compiles never see a C++ standard.
"""
# The funnel warns and drops empty glued arguments (-D "") itself
compile_flags, link_flags = split_flags(
_anchored_flags(CORE.build_flags, "esphome")
)
cflags = [f for f in compile_flags if not _is_cxx_std(f)]
cxx_std = CORE.cpp_standard
cxxflags = [f for f in compile_flags if not (cxx_std and _is_std(f))]
if cxx_std:
cxxflags.insert(0, (f"-std={cxx_std}",))
cxxflags += [(tok,) for tok in get_project_cxx_compile_flags()]
# A flag is removed whole, with its argument, as PlatformIO did
unflags = set(_anchored_flags(CORE.build_unflags, "esphome build_unflags"))
# An unflag that hits nothing (a typo, or -DUSE_FOO against
# -DUSE_FOO=1) must be visible, since the user believes the flag is
# gone while it still drives the build
if unmatched := sorted(unflags - set(cflags) - set(cxxflags) - set(link_flags)):
_LOGGER.warning(
"build_unflags entries matched no build flag: %s",
", ".join(" ".join(flag) for flag in unmatched),
)
def keep(flags: list[Flag]) -> list[str]:
return [tok for flag in flags if flag not in unflags for tok in flag]
return keep(cflags), keep(cxxflags), keep(link_flags)
def _resolve_host_libraries() -> list[ArduinoLibrary]:
"""Every ``cg.add_library()`` entry, fetched from the registry.
The host has no framework, so nothing is bundled and no framework
compatibility check applies; the platform check keeps the strict
manifest gate PlatformIO's native platform enforced. Manifest-less
libraries (a bare git checkout) build with PlatformIO's default
layout, as they did under its native platform.
"""
if not CORE.platformio_libraries:
return []
from esphome.arduino.library import resolve_libraries
return resolve_libraries(
None,
pio_platform=PIO_PLATFORM,
board_mcu="host",
cache_key=LIBRARY_CACHE_KEY,
framework=None,
manifest_optional=True,
)
def _file_macro_maps(build_dir: Path) -> list[str]:
"""Flags that keep ``__FILE__`` relative to the build path.
PlatformIO compiled ``src/x.cpp`` from the build path, and tools name
things after that spelling (CodSpeed's benchmark ids). Here a source
reaches the compiler by its absolute path, or relative to the build
directory when ccache rewrites it.
"""
build_path = Path(CORE.build_path)
prefixes = (build_path, Path(os.path.relpath(build_path, build_dir)))
return [
shell_token(f"-fmacro-prefix-map={prefix}{os.sep}=", force=True)
for prefix in prefixes
]
def _compiler_version(cxx: tuple[str, ...]) -> str:
"""What the compiler says it is: its path can stay the same across an
update (the macOS shims in /usr/bin)."""
result = subprocess.run(
[*cxx, "--version"], capture_output=True, text=True, check=False
)
return result.stdout
def write_project(compilers: HostCompilers, ccache: str | None) -> bool:
"""Write the ninja build for the current configuration.
``ccache`` is the caller's already-resolved binary (None when disabled).
Returns True when ``build.ninja`` changed, so the caller can skip work
derived purely from it (the compile database) on unchanged builds.
"""
build_dir = get_build_dir()
mkdir_p(build_dir)
src_dir = CORE.relative_src_path()
if not src_dir.is_dir():
# Generated project state, not install state: clean-all would not help
raise EsphomeError(f"Generated source directory {src_dir} is missing")
cflags, cxxflags, link_flags = _flag_lists()
libraries = _resolve_host_libraries()
include_dirs = [src_dir]
for lib in libraries:
include_dirs += lib.include_dirs
includes = [f"-I{_q(d)}" for d in include_dirs]
includes += _file_macro_maps(build_dir)
# SCons's link line: $LINKFLAGS $SOURCES $_LIBDIRFLAGS $_LIBFLAGS, so
# -L and -l trail the objects while every other link token leads
lib_dirs = [Path(t[2:]) for t in link_flags if t.startswith("-L")]
libs = [t for t in link_flags if t.startswith("-l")]
linkflags = [shell_token(t) for t in link_flags if not t.startswith(("-L", "-l"))]
for lib in libraries:
lib_dirs += lib.link_dirs
libs += [f"-l{name}" for name in lib.link_libs]
linkflags += [shell_token(f) for f in lib.link_flags]
# PlatformIO's ASPPCOM passes only -D/-I user flags to assembly
asflags = [t for t in cflags if t.startswith(("-D", "-I"))]
lines = [
*tool_lines(compilers.cc, compilers.cxx, ccache),
*compile_rule_lines(),
*pch_rule_lines(),
"rule link",
" command = $cxx -o $out $linkflags @$out.rsp $archives $libdirflags $libflags",
" rspfile = $out.rsp",
" rspfile_content = $in_newline",
" description = LINK $out",
]
if any(lib.sources and lib.lib_archive for lib in libraries):
# Resolved only when an archive is built, so a system without
# binutils still links a library-free configuration
lines += ar_rule_lines(find_tool("AR", ("ar",)))
lines += [
"",
f"cflags = {' '.join([*map(shell_token, cflags), *includes])}",
f"cxxflags = {' '.join([*map(shell_token, cxxflags), *includes])}",
f"asflags = {' '.join([*map(shell_token, asflags), *includes])}",
f"linkflags = {' '.join(linkflags)}",
f"libdirflags = {' '.join(f'-L{_q(d)}' for d in lib_dirs)}",
f"libflags = {' '.join(shell_token(lib) for lib in libs)}",
"",
]
archives, direct_objs = library_edges(lines, libraries)
src_cxx_override = pch_edges(
lines,
build_dir,
src_dir,
PCH_DEFAULT_HEADERS,
# The arguments of a CXX override come before the flags
[*compilers.cxx[1:], *cxxflags],
(),
(compilers.cxx[0], _compiler_version(compilers.cxx)) if pch_enabled() else (),
compilers.cxx,
)
src_objs = compile_edges(
lines,
collect_sources(src_dir),
src_dir,
"src",
cxx_override=src_cxx_override,
)
if not src_objs:
raise EsphomeError(f"No source files found under {src_dir}")
# Archives are not topologically sorted; GNU ld needs the group to
# resolve references between them. ld64 loads archives iteratively and
# rejects the option, so macOS lists them bare.
archive_tokens = [shell_token(a) for a in archives]
if archive_tokens and sys.platform != "darwin":
archive_tokens = ["-Wl,--start-group", *archive_tokens, "-Wl,--end-group"]
lines.append(
f"build {PROGRAM_NAME}: link {' '.join(src_objs + direct_objs)} | "
f"{' '.join(_e(a) for a in archives)}"
)
lines.append(f" archives = {' '.join(archive_tokens)}")
lines.append(f"default {PROGRAM_NAME}")
lines.append("")
return write_file_if_changed(build_dir / "build.ninja", "\n".join(lines))
esphome/build_gen/platformio.py
+6
View File
@@ -1,6 +1,8 @@
from esphome.build_helpers.pch import pch_script_enabled
from esphome.const import __version__
from esphome.core import CORE
from esphome.helpers import mkdir_p, read_file, write_file_if_changed
from esphome.platformio.toolchain import copy_pch_script
from esphome.writer import find_begin_end
INI_AUTO_GENERATE_BEGIN = "; ========== AUTO GENERATED CODE BEGIN ==========="
@@ -62,6 +64,8 @@ def get_ini_content():
# Add extra script for C++ flags
CORE.add_platformio_option("extra_scripts", [f"pre:{CXX_FLAGS_FILE_NAME}"])
if pch_script_enabled():
CORE.add_platformio_option("extra_scripts", ["post:pch.py"])
# Add CMake args. A user-supplied value (str or list) is deliberately
# replaced; this option was always overwritten at FINAL priority.
@@ -106,6 +110,8 @@ def write_project():
# Write extra script for C++ specific flags
write_cxx_flags_script()
if pch_script_enabled():
copy_pch_script()
CXX_FLAGS_FILE_NAME = "cxx_flags.py"
esphome/build_helpers/ccache.py
+41 -2
View File
@@ -68,6 +68,15 @@ def resolve_ccache_path() -> str | None:
return ccache
def resolve_absolute_ccache_path() -> str | None:
"""``resolve_ccache_path`` for the ninja backends, which run their
commands from the build directory, where a relative path is lost."""
from esphome.build_helpers.ninja import absolute_tool
ccache = resolve_ccache_path()
return absolute_tool(ccache) if ccache else None
def ccache_defaults_env(cache_dir: Path) -> dict[str, str]:
"""Default ``CCACHE_*`` values for a build subprocess (not os.environ).
@@ -84,9 +93,39 @@ def ccache_defaults_env(cache_dir: Path) -> dict[str, str]:
"CORE.build_path must be set before constructing the build environment"
)
defaults = {
"CCACHE_DIR": str(cache_dir),
# ccache expands $VAR in its settings; $$ is a literal $
"CCACHE_DIR": str(cache_dir).replace("$", "$$"),
"CCACHE_NOHASHDIR": "true",
"CCACHE_DEPEND": "1",
"CCACHE_BASEDIR": str(Path(CORE.build_path).resolve()),
"CCACHE_BASEDIR": str(Path(CORE.build_path).resolve()).replace("$", "$$"),
}
return {k: v for k, v in defaults.items() if k not in os.environ}
def effective_ccache_basedir() -> str:
"""The prefix ccache strips from hashed paths: a usable user
CCACHE_BASEDIR, else the resolved build path."""
from esphome.core import CORE
raw = os.environ.get("CCACHE_BASEDIR")
if raw is not None and Path(raw).is_absolute() and len(Path(raw).parts) > 1:
return raw
return str(Path(CORE.build_path).resolve())
def ccache_env(ccache: str | None, tools_cache: tuple[str, str]) -> dict[str, str]:
"""The ccache settings for a build subprocess (not os.environ).
``ccache`` is the pre-resolved binary (resolve_ccache_path), or None when
disabled; ``tools_cache`` is the backend's tools cache spec, which holds
its ccache dir. The pch settings include ``time_macros``: a cached
object can keep an older ``__DATE__`` or ``__TIME__``.
"""
if ccache is None:
return {}
from esphome.build_helpers.pch import ccache_pch_env
from esphome.build_helpers.tools_cache import tools_cache_path
env = ccache_defaults_env(tools_cache_path(*tools_cache) / "ccache")
env.update(ccache_pch_env())
return env
esphome/build_helpers/idedata.py
+27 -1
View File
@@ -11,6 +11,7 @@ consumers (IDE integration, clang-tidy) expect:
from __future__ import annotations
from collections.abc import Callable
import json
import logging
import os
@@ -21,6 +22,8 @@ import subprocess
from esphome.core import EsphomeError
from esphome.helpers import write_file
_LOGGER = logging.getLogger(__name__)
# Everything idedata generation may raise after a successful link; idedata
# is a bonus artifact, so consumers warn instead of failing the build
IDEDATA_BEST_EFFORT_ERRORS = (
@@ -31,7 +34,21 @@ IDEDATA_BEST_EFFORT_ERRORS = (
ValueError,
)
_LOGGER = logging.getLogger(__name__)
def warn_if_idedata_missing(get_idedata: Callable[[], dict | None]) -> None:
"""Run an idedata generator, downgrading any failure to a warning:
the firmware already built."""
try:
if get_idedata() is None:
_LOGGER.warning("No idedata was generated for this build")
except IDEDATA_BEST_EFFORT_ERRORS as err:
_LOGGER.warning(
"Could not generate idedata: %s (IDE, clang-tidy, and "
"memory-analysis data will be unavailable for this build)",
err,
)
_LOGGER.debug("Idedata failure detail", exc_info=True)
# C++ translation-unit suffixes used to identify ESPHome source files.
_CXX_SUFFIXES = (".cpp", ".cc")
@@ -135,6 +152,15 @@ def _is_launcher(token: str) -> bool:
return Path(token).stem.lower() in _LAUNCHER_STEMS
def is_joined_include(tok: str) -> bool:
"""The joined ``-includefoo.h`` spelling; excludes clang's -include-pch."""
return (
tok.startswith("-include")
and tok != "-include"
and not tok.startswith("-include-")
)
def parse_entry(
entry: dict, launcher: str | None = None
) -> tuple[str, list[str], list[str], list[str]]:
esphome/build_helpers/native.py
+74
View File
@@ -0,0 +1,74 @@
"""Resolution of the native (non-PlatformIO) build backend for a config.
Kept deliberately light: the serial upload and logs fast path resolves a
backend for its artifact paths alone, so importing this must not pull in a
platform component package or the backend itself.
"""
from __future__ import annotations
from collections.abc import Collection
import importlib
import logging
from types import ModuleType
from esphome.const import Toolchain
from esphome.core import CORE, EsphomeError
_LOGGER = logging.getLogger(__name__)
# Native build backend per (target platform, toolchain)
NATIVE_TOOLCHAIN_MODULES = {
("esp32", Toolchain.ESP_IDF): "esphome.espidf.toolchain",
("esp8266", Toolchain.ARDUINO): "esphome.arduino8266.toolchain",
("host", Toolchain.HOST): "esphome.host.toolchain",
}
def native_backend() -> ModuleType | None:
"""The native build backend module for the resolved toolchain."""
if not CORE.using_native_toolchain:
return None
key = (CORE.target_platform, CORE.toolchain)
if (module_path := NATIVE_TOOLCHAIN_MODULES.get(key)) is None:
# Degrading to the PlatformIO path would build with the wrong backend
raise EsphomeError(
f"Toolchain '{CORE.toolchain.value}' has no native build backend "
f"module for platform {CORE.target_platform}"
)
return importlib.import_module(module_path)
# Binutils and the linked image for memory analysis, for toolchains that build
# without PlatformIO but have no native build backend (which supplies them)
ANALYSIS_TOOLCHAIN_MODULES = {
("nrf52", Toolchain.SDK_NRF): "esphome.components.nrf52.toolchain",
}
def analysis_backend() -> ModuleType | None:
"""The module giving objdump, readelf and the ELF of a non-PlatformIO build.
None means PlatformIO's idedata supplies them (or nothing can).
"""
if (native := native_backend()) is not None:
return native
module_path = ANALYSIS_TOOLCHAIN_MODULES.get((CORE.target_platform, CORE.toolchain))
return importlib.import_module(module_path) if module_path else None
def warn_ignored_platformio_options(consumed: Collection[str]) -> None:
"""Warn for component-added platformio options a native build drops.
User-supplied keys were already routed or warned about by
``core/config.py``; what survives into ``CORE.platformio_options`` came
from ``cg.add_platformio_option`` calls in components.
"""
for key in sorted(CORE.platformio_options or {}):
if key not in consumed:
_LOGGER.warning(
"platformio_options->%s is ignored when building with the "
"native '%s' toolchain",
key,
CORE.toolchain.value,
)
esphome/build_helpers/ninja.py
+93 -3
View File
@@ -2,14 +2,17 @@
from __future__ import annotations
import json
import logging
import os
from pathlib import Path
import re
import shutil
import subprocess
from esphome.core import EsphomeError
from esphome.framework_helpers import strip_win_long_path_prefix, tool_version_runs
from esphome.helpers import write_file_if_changed
_LOGGER = logging.getLogger(__name__)
@@ -23,11 +26,26 @@ def _ninja_runs(binary: str) -> bool:
)
# Compile rule names the generators emit; ninja's compdb tool is asked for
# exactly these, so a renamed rule fails the build instead of stranding idedata
COMPILE_RULES = ("c", "cxx", "aspp", "asm")
def absolute_tool(tool: str | Path) -> str:
"""A tool path that still resolves from the build directory.
``shutil.which`` returns a relative path for a relative PATH entry, and
ninja runs the commands from ``.pioenvs/<name>``. Symlinks are kept:
ccache's compiler links depend on the name they are called by.
"""
return strip_win_long_path_prefix(str(Path(tool).absolute()))
def find_ninja() -> Path:
"""Locate the ninja binary: a runnable PATH hit first, else the ninja
PyPI wheel."""
if binary := shutil.which("ninja"):
binary = strip_win_long_path_prefix(binary)
binary = absolute_tool(binary)
if _ninja_runs(binary):
return Path(binary)
import_error: ImportError | None = None
@@ -49,8 +67,18 @@ def find_ninja() -> Path:
def escape(value: Path | str) -> str:
"""Escape a path or token for a ninja file."""
return str(value).replace("$", "$$").replace(":", "$:").replace(" ", "$ ")
"""Escape a path or token for a ninja file.
ninja has no escape for ``|`` or a line break in a path, so those fail
here by name instead of producing a build file ninja misreads.
"""
text = str(value)
if bad := next((c for c in "|\r\n" if c in text), None):
raise EsphomeError(
f"Path {text!r} contains {bad!r}, which a ninja build file cannot "
"express; rename the file or directory"
)
return text.replace("$", "$$").replace(":", "$:").replace(" ", "$ ")
def quote_arg(tok: str) -> str:
@@ -90,3 +118,65 @@ def shell_token(tok: str, force: bool = False) -> str:
def quote_path(value: Path | str) -> str:
"""Force-quote a path for the ninja command line (shell/CreateProcess)."""
return shell_token(str(value), force=True)
def refresh_compile_commands(
ninja_path: Path, build_dir: Path, env: dict[str, str], ninja_changed: bool
) -> None:
"""Regenerate the compile DB (a pure function of build.ninja) when stale.
Freshness rides a stamp: the DB itself is written through
write_file_if_changed (its mtime feeds the idedata cache), so a
regeneration with identical content would stay "stale" forever. An
interrupted previous run may have rewritten the manifest without
regenerating the DB, hence the mtime comparison.
"""
compdb = build_dir / "compile_commands.json"
compdb_stamp = build_dir / ".compile_commands.stamp"
ninja_file = build_dir / "build.ninja"
if (
ninja_changed
or not compdb.is_file()
or not compdb_stamp.is_file()
or compdb_stamp.stat().st_mtime < ninja_file.stat().st_mtime
):
write_compile_commands(ninja_path, build_dir, env)
compdb_stamp.touch()
def write_compile_commands(
ninja_path: Path, build_dir: Path, env: dict[str, str]
) -> None:
compdb = build_dir / "compile_commands.json"
result = subprocess.run(
[str(ninja_path), "-C", str(build_dir), "-t", "compdb", *COMPILE_RULES],
env=env,
capture_output=True,
text=True,
check=False,
close_fds=False,
)
if result.returncode != 0:
# Drop any stale database so consumers (IDE integration, clang-tidy,
# the memory analyzer) can't silently read outdated data
compdb.unlink(missing_ok=True)
raise EsphomeError(f"Could not generate compile_commands.json: {result.stderr}")
try:
entries = json.loads(result.stdout)
except ValueError as err:
compdb.unlink(missing_ok=True)
raise EsphomeError(
f"ninja produced an unparsable compile database: {err} "
f"(output starts {result.stdout[:120]!r})"
) from err
if not entries:
# compdb exits 0 with [] for unknown rule names; a renamed compile
# rule must fail the build, not silently strand every consumer
compdb.unlink(missing_ok=True)
raise EsphomeError(
"ninja produced an empty compile database; the generator's rule "
"names no longer match"
)
# write_file_if_changed keeps the mtime stable on no-op builds so the
# idedata cache stays valid
write_file_if_changed(compdb, result.stdout)
esphome/build_helpers/ninja_gen.py
+279
View File
@@ -0,0 +1,279 @@
"""The parts of a ``build.ninja`` every native backend emits the same way.
Rule names match ``SOURCE_KIND_FOR_SUFFIX`` values (c, cxx, asm, aspp) and
``ninja.COMPILE_RULES``, which the compile database is asked for.
"""
from __future__ import annotations
from collections.abc import Collection, Sequence
import logging
import os
from pathlib import Path
import sys
from typing import TYPE_CHECKING
from esphome.build_helpers.idedata import is_joined_include
from esphome.build_helpers.ninja import (
escape as _e,
quote_path as _q,
shell_token as _shell_token,
)
from esphome.build_helpers.pch import (
PCH_GCH_NAME,
PCH_SUM_NAME,
log_pch_in_use,
pch_consumer_flags,
pch_identity,
pch_usable,
write_pch_headers,
)
from esphome.framework_helpers import strip_win_long_path_prefix
from esphome.helpers import write_file_if_changed
from esphome.platformio.library import SOURCE_KIND_FOR_SUFFIX
if TYPE_CHECKING:
from esphome.arduino.library import ArduinoLibrary
_LOGGER = logging.getLogger(__name__)
_BUILD_TOOL = Path(__file__).parent.parent / "build_gen" / "build_tool.py"
def collect_sources(root: Path, exclude: Collection[str] = frozenset()) -> list[Path]:
return sorted(
p
for p in root.rglob("*")
if p.suffix in SOURCE_KIND_FOR_SUFFIX and p.name not in exclude
)
def common_parent(paths: list[Path]) -> Path:
return Path(os.path.commonpath([str(p.parent) for p in paths]))
def _command(words: Sequence[Path | str]) -> str:
program, *args = words
return " ".join([_q(program), *(_shell_token(str(a)) for a in args)])
def tool_lines(
cc: Sequence[Path | str], cxx: Sequence[Path | str], ccache: str | None
) -> list[str]:
"""The file header: the compilers and the helper tools as variables.
A compiler is its program followed by any arguments it always takes.
"""
return [
"# Auto-generated by ESPHome",
"ninja_required_version = 1.5",
f"cc = {_command(cc)}",
f"cxx = {_command(cxx)}",
# The NSIS launcher starts Python with a \\?\ extended-length path
# that cmd.exe cannot spawn; same strip every other emitted binary
# path gets
f"python = {_q(strip_win_long_path_prefix(sys.executable))}",
f"buildtool = {_q(_BUILD_TOOL)}",
f"ccache = {_q(ccache) if ccache else ''}",
"",
]
def compile_rule_lines() -> list[str]:
"""The compile rules; ``$own_includes`` is empty unless an edge sets it."""
return [
"rule c",
" command = $ccache $cc -MMD -MF $out.d $own_includes $cflags $flags -c $in -o $out",
" depfile = $out.d",
" deps = gcc",
" description = CC $out",
"rule cxx",
" command = $ccache $cxx -MMD -MF $out.d $own_includes $cxxflags $flags -c $in -o $out",
" depfile = $out.d",
" deps = gcc",
" description = CXX $out",
"rule aspp",
" command = $ccache $cc -MMD -MF $out.d -x assembler-with-cpp $own_includes $asflags $flags -c $in -o $out",
" depfile = $out.d",
" deps = gcc",
" description = AS $out",
# Plain assembler, as SCons's ASCOM: no preprocessor, so no
# depfile and no $flags (defines/includes) either
"rule asm",
" command = $ccache $cc -x assembler $asflags -c $in -o $out",
" description = AS $out",
]
def pch_rule_lines() -> list[str]:
"""The precompiled header rule, for a generator that emits one."""
return [
# No $ccache: the .gch embeds build dir paths
"rule pch",
" command = $cxx -MMD -MF $out.d -x c++-header $cxxflags $flags -c $in -o $out",
" depfile = $out.d",
" deps = gcc",
" description = PCH $out",
]
def pch_edges(
lines: list[str],
build_dir: Path,
src_dir: Path,
headers: Sequence[str],
cxxflags: Sequence[str],
src_flags: Sequence[str],
identity: Sequence[str],
cxx: Sequence[Path | str],
) -> tuple[str, str] | None:
"""Emit the precompiled header for the C++ src edges.
``headers`` are folded into one prefix header, ``src_flags`` are the
flags every src edge carries, ``identity`` names what else the compile
depends on, the compiler included, and ``cxx`` is what the host rule
asks. Returns the ``cxx_override`` for ``compile_edges``, or None
without a pch.
"""
if not pch_usable(cxx):
return None
if any(
tok == "-include" or tok.startswith("--include") or is_joined_include(tok)
for tok in cxxflags
):
# $cxxflags expands first and GCC only loads a .gch for the first
# -include
_LOGGER.warning(
"A -include in the compiler flags prevents the precompiled header from "
"loading; compiling without it"
)
return None
log_pch_in_use()
source = write_pch_headers(build_dir, headers)
write_file_if_changed(
build_dir / PCH_SUM_NAME,
pch_identity([*cxxflags, *src_flags], src_dir, tuple(headers), identity) + "\n",
)
gch = _e(PCH_GCH_NAME)
# The checksum file changes with anything the .gch depends on
lines.append(f"build {gch}: pch {_e(source)} | {_e(PCH_SUM_NAME)}")
if src_flags:
lines.append(f" flags = {' '.join(src_flags)}")
lines.append(f"srccxxflags = {' '.join([*src_flags, *pch_consumer_flags()])}")
return ("$srccxxflags", gch)
def ar_rule_lines(ar: Path | str) -> list[str]:
return [
"rule ar",
f" command = $python $buildtool ar {_q(ar)} $out $out.rsp",
" rspfile = $out.rsp",
" rspfile_content = $in_newline",
" description = AR $out",
]
def compile_edges(
lines: list[str],
sources: list[Path],
root: Path,
group: str,
flags: str = "",
own_includes: str = "",
cxx_override: tuple[str, str] | None = None,
) -> list[str]:
"""Emit compile edges for ``sources``; return the object paths.
``cxx_override`` is a (flags, implicit-dep) pair applied to C++ edges
only, replacing ``flags`` (used for the precompiled header).
"""
objects = []
for src in sources:
rel = src.relative_to(root).as_posix()
obj = f"obj/{group}/{rel}.o"
escaped_obj = _e(obj)
kind = SOURCE_KIND_FOR_SUFFIX[src.suffix]
override = cxx_override if kind == "cxx" else None
implicit = f" | {override[1]}" if override else ""
lines.append(f"build {escaped_obj}: {kind} {_e(src)}{implicit}")
if own_includes:
lines.append(f" own_includes = {own_includes}")
if edge_flags := override[0] if override else flags:
lines.append(f" flags = {edge_flags}")
# Escaped once here: the returned paths only ever appear in build
# statements (archive/link inputs), which use ninja escaping
objects.append(escaped_obj)
return objects
def library_edges(
lines: list[str], libraries: list[ArduinoLibrary]
) -> tuple[list[str], list[str]]:
"""Emit every library's compile and archive edges.
Returns the archive names and the objects that link directly. A
library's own include dirs lead its compile lines, as PlatformIO searched
them first: the include path is one global list, so another library's
header of the same name would shadow them.
"""
archives: list[str] = []
direct_objs: list[str] = []
for lib in libraries:
if not lib.sources:
# Header-only libraries are legitimate; the log makes an empty
# srcFilter or broken tree traceable before link errors do
_LOGGER.debug(
"Library %s has no source files; contributing includes only",
lib.name,
)
continue
objs = compile_edges(
lines,
lib.sources,
common_parent(lib.sources),
f"lib/{lib.name}",
flags=" ".join(_shell_token(f) for f in lib.flags),
own_includes=" ".join(f"-I{_q(d)}" for d in lib.include_dirs),
)
if not lib.lib_archive:
# libArchive: false / dot_a_linkage=false: hand the objects to
# the linker directly so unreferenced-but-required symbols
# (exception handlers, weak overrides) survive
direct_objs.extend(objs)
continue
archive = f"lib{lib.name}.a"
lines.append(f"build {_e(archive)}: ar {' '.join(objs)}")
archives.append(archive)
return archives, direct_objs
# One build flag: the flag, plus its argument when that is a separate token
Flag = tuple[str, ...]
# Flags whose path operand is the next token; gcc also takes it glued on
PATH_ARG_FLAGS = ("-include", "-imacros", "-isystem", "-iquote", "-idirafter")
# Flags whose path operand is glued on
PATH_PREFIXES = ("-I", "-L", *PATH_ARG_FLAGS)
def _anchor(path: str, base: Path) -> str:
if not path or Path(path).is_absolute():
return path
return str(base / path)
def anchor_path_flag(flag: Flag, base: Path) -> Flag:
"""Anchor a flag's relative path operand at ``base``.
PlatformIO ran the compiler from the build path; ninja runs it from
``.pioenvs/<name>``, where a relative operand would point elsewhere.
"""
name, *args = flag
if args:
if name in PATH_ARG_FLAGS:
return (name, _anchor(args[0], base))
return flag
for prefix in PATH_PREFIXES:
if name.startswith(prefix):
return (prefix + _anchor(name[len(prefix) :], base),)
return flag
esphome/build_helpers/pch.py
+248
View File
@@ -0,0 +1,248 @@
"""Shared precompiled header policy for the build backends."""
from __future__ import annotations
from collections.abc import Iterable, Sequence
import hashlib
import logging
import os
from pathlib import Path
import posixpath
import re
import subprocess
import sys
from esphome.build_helpers.ccache import effective_ccache_basedir, parse_enable_env
from esphome.const import PLATFORM_NRF52
from esphome.helpers import write_file_if_changed
_LOGGER = logging.getLogger(__name__)
# The header and its sidecars live in the build directory
PCH_HEADER_NAME = "esphome_pch.h"
PCH_GCH_NAME = f"{PCH_HEADER_NAME}.gch"
# ccache hashes this instead of the .gch; also the freshness stamp
PCH_SUM_NAME = f"{PCH_GCH_NAME}.sum"
# The include list the .gch is compiled from
PCH_SOURCE_NAME = "esphome_pch_src.h"
# GCC can skip a .gch without a diagnostic and read the header of the same
# name, so that header is an error. Other tools get the include list.
PCH_GUARD_TEXT = f"""\
#if defined(__GNUC__) && !defined(__clang__) && !defined(__INTELLISENSE__)
#error "The precompiled header was not loaded"
#else
#include "{PCH_SOURCE_NAME}"
#endif
"""
# The cc1plus wrapper the PlatformIO script writes on arm64 macOS
PCH_CC1_DIR = "pch_cc1"
# What the PlatformIO script leaves in the project root, for cleanup
PCH_ARTIFACT_NAMES = (PCH_HEADER_NAME, PCH_GCH_NAME, PCH_SUM_NAME, PCH_SOURCE_NAME)
PCH_ARTIFACT_DIRS = (PCH_CC1_DIR,)
# The core headers every backend precompiles
PCH_DEFAULT_HEADERS = ("esphome/core/pch_prefix.h",)
# PlatformIO platforms that do not take the pch script
PCH_SCRIPT_EXCLUDED_PLATFORMS = frozenset(
{
PLATFORM_NRF52,
}
)
# What ccache needs to cache compiles that load a .gch
_CCACHE_PCH_SLOPPINESS = ("pch_defines", "time_macros")
# Both include forms: an angle include resolving under src/ enters the digest
_INCLUDE_RE = re.compile(rb'^\s*#\s*include\s+["<]([^">]+)[">]', re.MULTILINE)
def pch_enabled() -> bool:
"""Precompiled-header knob: default on, ``ESPHOME_PCH_ENABLE=0`` opts out."""
return parse_enable_env("ESPHOME_PCH_ENABLE") is not False
def pch_forced() -> bool:
"""``ESPHOME_PCH_ENABLE=1``: wanted even where the host rule says no."""
return parse_enable_env("ESPHOME_PCH_ENABLE") is True
# GCC bug 14940: before these releases the Windows loader maps a .gch only
# at its saved address. First fixed release per major, 16 on always fixed;
# PCH_WINDOWS_CMAKE_OLD_GCC and the pch_usable message spell the same table
PCH_WINDOWS_GCC_FIXED = {14: (14, 4), 15: (15, 3)}
PCH_WINDOWS_GCC_FIXED_DEFAULT = (16, 0)
# The same rule for CMake, which alone knows the version before configure
PCH_WINDOWS_CMAKE_OLD_GCC = (
"CMAKE_CXX_COMPILER_VERSION VERSION_LESS 14.4 OR "
"(CMAKE_CXX_COMPILER_VERSION VERSION_GREATER_EQUAL 15 AND "
"CMAKE_CXX_COMPILER_VERSION VERSION_LESS 15.3)"
)
def gcc_relocates_pch_on_windows(version: Sequence[int]) -> bool:
"""Whether a GCC of this version loads a .gch on Windows."""
if not version:
return False
fixed = PCH_WINDOWS_GCC_FIXED.get(version[0], PCH_WINDOWS_GCC_FIXED_DEFAULT)
return tuple(version[:2]) >= fixed
# GCC ends the first --version line with its version; clang names itself
_VERSION_RE = re.compile(r"\d+(?:\.\d+)+")
def gcc_version(cxx: Sequence[Path | str]) -> tuple[int, ...] | None:
"""The GCC version from ``--version``: () when it cannot be read, None
for a compiler that is not GCC."""
try:
result = subprocess.run(
[*cxx, "--version"], capture_output=True, text=True, check=False
)
except OSError as err:
_LOGGER.debug("Cannot run %s: %s", cxx[0], err)
return ()
banner = result.stdout.partition("\n")[0]
if "clang" in banner.lower():
return None
found = _VERSION_RE.findall(banner)
return tuple(int(part) for part in found[-1].split(".")) if found else ()
def pch_needs_gcc_check() -> bool:
"""Windows host with the knob unset: the compiler version decides."""
return sys.platform == "win32" and parse_enable_env("ESPHOME_PCH_ENABLE") is None
def pch_usable(cxx: Sequence[Path | str]) -> bool:
"""The knob plus the host rule; ``ESPHOME_PCH_ENABLE=1`` skips the rule."""
if not pch_enabled():
return False
if not pch_needs_gcc_check():
return True
version = gcc_version(cxx)
if version is None or gcc_relocates_pch_on_windows(version):
return True
_LOGGER.info(
"GCC %s cannot load a precompiled header on Windows (GCC bug 14940, "
"fixed in 14.4, 15.3 and 16); compiling without it "
"(set ESPHOME_PCH_ENABLE=1 to force)",
".".join(map(str, version)) or "of unknown version",
)
return False
def pch_consumer_flags() -> list[str]:
"""Flags a C++ src compile loads the pch with. The -include stays
relative: an absolute path would enter the ccache key."""
return ["-Winvalid-pch", "-Werror=invalid-pch", "-include", PCH_HEADER_NAME]
def ccache_pch_env() -> dict[str, str]:
"""What ccache needs to cache compiles that load a .gch, added to what
the user already set."""
if not pch_enabled():
return {}
sloppiness = [
item.strip()
for item in os.environ.get("CCACHE_SLOPPINESS", "").split(",")
if item.strip()
]
sloppiness += [item for item in _CCACHE_PCH_SLOPPINESS if item not in sloppiness]
env = {"CCACHE_SLOPPINESS": ",".join(sloppiness)}
if "CCACHE_PCH_EXTSUM" not in os.environ:
env["CCACHE_PCH_EXTSUM"] = "true"
return env
def pch_script_enabled() -> bool:
"""Whether this PlatformIO build takes the pch script."""
from esphome.core import CORE
return pch_enabled() and CORE.target_platform not in PCH_SCRIPT_EXCLUDED_PLATFORMS
def pch_header_text(include_headers: Iterable[str]) -> str:
"""The prefix-header source: exactly these includes, in order."""
return "".join(f'#include "{name}"\n' for name in include_headers)
def write_pch_headers(build_dir: Path, include_headers: Iterable[str]) -> Path:
"""Write the guard header and the include list; return the latter,
which is what the .gch compiles from."""
write_file_if_changed(build_dir / PCH_HEADER_NAME, PCH_GUARD_TEXT)
source = build_dir / PCH_SOURCE_NAME
write_file_if_changed(source, pch_header_text(include_headers))
return source
def _include_closure(src_dir: Path, roots: Iterable[str]) -> dict[str, bytes]:
"""Include closure of ``roots``: src-relative name -> contents.
Resolution mirrors the compiler (includer's dir, then src root). No
#ifdef evaluation: including too much is the safe direction. Headers
outside ``src_dir`` are covered by the version strings of the caller.
"""
seen: dict[str, bytes] = {}
stack: list[tuple[str, str]] = [(name, "") for name in roots]
while stack:
name, from_dir = stack.pop()
for candidate in (f"{from_dir}/{name}" if from_dir else name, name):
rel = posixpath.normpath(candidate)
if not rel.startswith("..") and (src_dir / rel).is_file():
break
else:
continue
if rel in seen:
continue
data = seen[rel] = (src_dir / rel).read_bytes()
parent = posixpath.dirname(rel)
stack.extend((inc.decode(), parent) for inc in _INCLUDE_RE.findall(data))
return seen
def pch_checksum(
src_dir: Path, include_headers: Iterable[str], extra: Iterable[str]
) -> str:
"""Digest of the prefix header's include closure plus ``extra``."""
digest = hashlib.sha256()
closure = _include_closure(src_dir, include_headers)
for name in sorted(closure):
digest.update(name.encode())
digest.update(closure[name])
digest.update(b"\0")
for item in extra:
digest.update(item.encode())
digest.update(b"\0")
return digest.hexdigest()
def pch_identity(
tokens: Iterable[str],
src_dir: Path,
include_headers: tuple[str, ...],
extra: Iterable[str],
) -> str:
"""The .sum digest: include closure, header text, ``extra`` and the
compile flags with the build path stripped, as ccache does."""
from esphome.core import CORE
flags = (
" ".join(tokens)
.replace(str(CORE.build_path), "")
.replace(effective_ccache_basedir(), "")
)
# The closure is sorted, so header order only enters via the text
return pch_checksum(
src_dir, include_headers, (pch_header_text(include_headers), *extra, flags)
)
_DISABLE_HINT = " (set ESPHOME_PCH_ENABLE=0 to disable)"
def log_pch_in_use() -> None:
_LOGGER.info("Compiling with a precompiled header%s", _DISABLE_HINT)
esphome/build_helpers/tool_runner.py
+180
View File
@@ -0,0 +1,180 @@
"""Run a native build tool (cmake, ninja) and relay its output.
Output is read from a pipe so it can be filtered here: a child that inherits
our stdout writes straight to the file descriptor, past any Python wrapper.
"""
from __future__ import annotations
import codecs
from contextlib import suppress
import logging
import os
from pathlib import Path
import re
import shutil
import subprocess
import sys
from typing import Any, TextIO
from esphome.util import ANSI_ESCAPE, RedirectText, shlex_quote
_LOGGER = logging.getLogger(__name__)
# Windows code page identifier for UTF-8, as used by ``chcp 65001``.
UTF8_CODEPAGE = 65001
# Same pattern idf.py uses to spot ninja status lines (``is_progression``).
_PROGRESS = re.compile(r"^\[\d+/\d+\]|.*\(\d+ \%\)$")
_READ_SIZE = 65536
def _get_kernel32() -> Any | None:
"""Return the Windows kernel32 module, or None on any other platform."""
if sys.platform != "win32":
return None
import ctypes
return ctypes.windll.kernel32
class Utf8Console:
"""Keep an attached Windows console on UTF-8 while a build tool runs.
esp_idf_size draws its table with Unicode box characters, and CMake
re-decodes a child's output with the console code page, which garbles the
table on any page but UTF-8. A console already on UTF-8 is left alone so
an overlapping build never records UTF-8 as the page to go back to.
"""
def __init__(self, kernel32: Any | None) -> None:
self._kernel32 = kernel32
self._codepages: tuple[int, int] | None = None
def __enter__(self) -> None:
kernel32 = self._kernel32
if kernel32 is None:
return
old_in = kernel32.GetConsoleCP()
old_out = kernel32.GetConsoleOutputCP()
# Both calls return 0 when no console is attached.
if not old_in or not old_out:
return
if old_in == UTF8_CODEPAGE and old_out == UTF8_CODEPAGE:
return
# Record first so a switch that fails part way is still undone.
self._codepages = (old_in, old_out)
kernel32.SetConsoleCP(UTF8_CODEPAGE)
kernel32.SetConsoleOutputCP(UTF8_CODEPAGE)
def __exit__(self, *exc_info: object) -> None:
if self._codepages is None:
return
old_in, old_out = self._codepages
self._codepages = None
self._kernel32.SetConsoleCP(old_in)
self._kernel32.SetConsoleOutputCP(old_out)
def _fit_terminal(text: str) -> str:
"""Elide the middle of ``text`` to fit the terminal, as idf.py does.
A width of 0 (a pipe, the dashboard) leaves the text whole.
"""
width = shutil.get_terminal_size((0, 0)).columns
if not width:
return text
if width <= 3:
return "." * width
if len(text) >= width:
keep = (width - 3) // 2
return f"{text[:keep]}...{text[len(text) - keep :]}"
return text
class ToolOutput(RedirectText):
"""RedirectText that can collapse ninja status lines into one line.
With ``progress`` each ``[n/m]`` line overwrites the previous one, the
way idf.py shows a build.
"""
def __init__(
self, out: TextIO, filter_lines: list[str] | None, progress: bool
) -> None:
super().__init__(out, filter_lines=filter_lines)
self._progress = progress
self._on_progress_line = False
def _splits_lines(self) -> bool:
return self._progress or super()._splits_lines()
def _emit_line(self, line: str) -> None:
if self._progress and _PROGRESS.match(line):
if not self._is_filtered(line):
text = _fit_terminal(line.strip("\r\n"))
self._write_color_replace(f"\r{text}\x1b[K")
self._on_progress_line = True
return
self._end_progress_line()
super()._emit_line(line)
def _end_progress_line(self) -> None:
if self._on_progress_line:
self._on_progress_line = False
self._write_color_replace(os.linesep)
def drain(self) -> None:
super().drain()
# Called from cleanup, so a broken stream must not hide the exit code.
with suppress(OSError, ValueError):
self._end_progress_line()
self._out.flush()
def run_build_tool(
cmd: list[str],
*,
cwd: Path,
env: dict[str, str],
filter_lines: list[str] | None = None,
progress: bool = False,
log_path: Path | None = None,
) -> int:
"""Run ``cmd`` and relay stdout and stderr, merged, to our stdout.
``log_path`` also gets the full, unfiltered output without color codes, as
idf.py wrote its logs (its hint patterns expect plain text). Returns the
exit code.
"""
_LOGGER.debug("Running: %s", " ".join(shlex_quote(arg) for arg in cmd))
_LOGGER.debug(" in directory: %s", cwd)
output = ToolOutput(sys.stdout, filter_lines, progress)
decoder = codecs.getincrementaldecoder("utf-8")(errors="replace")
if log_path is not None:
log_path.parent.mkdir(parents=True, exist_ok=True)
with (
Path(log_path or os.devnull).open("w", encoding="utf-8", newline="") as log,
Utf8Console(_get_kernel32()),
subprocess.Popen(
cmd,
cwd=cwd,
env=env,
stdout=subprocess.PIPE,
stderr=subprocess.STDOUT,
close_fds=False,
) as proc,
):
try:
# read1 returns as soon as anything is available, so output
# streams while the tool runs.
while chunk := proc.stdout.read1(_READ_SIZE):
text = decoder.decode(chunk)
log.write(ANSI_ESCAPE.sub("", text))
output.write(text)
if tail := decoder.decode(b"", final=True):
log.write(ANSI_ESCAPE.sub("", tail))
output.write(tail)
finally:
output.drain()
return proc.returncode
esphome/build_helpers/tools_cache.py
+13 -3
View File
@@ -16,8 +16,8 @@ def tools_cache_path(env_var: str, subdir: str) -> Path:
from esphome.helpers import get_str_env
if prefix := get_str_env(env_var, "").strip():
# resolve(): symlinked prefixes otherwise trip idf.py's
# venv-mismatch warning on every build
# resolve(): a symlinked prefix would otherwise record a second
# spelling of the same paths in the build tree
return Path(prefix).expanduser().resolve()
# appauthor=False keeps the Windows path short (no vendor segment);
# deep IDF trees run into MAX_PATH otherwise
@@ -33,4 +33,14 @@ def tools_cache_path(env_var: str, subdir: str) -> Path:
IDF_TOOLS_CACHE = ("ESPHOME_ESP_IDF_PREFIX", "idf")
SDK_NRF_TOOLS_CACHE = ("ESPHOME_SDK_NRF_PREFIX", "sdk-nrf")
ARDUINO8266_TOOLS_CACHE = ("ESPHOME_ARDUINO8266_PREFIX", "arduino8266")
TOOLS_CACHE_SPECS = (IDF_TOOLS_CACHE, SDK_NRF_TOOLS_CACHE, ARDUINO8266_TOOLS_CACHE)
# The host backend installs nothing; the entry only holds its ccache dir
HOST_TOOLS_CACHE = ("ESPHOME_HOST_PREFIX", "host")
# PlatformIO installs into its own dirs; the entry is its ccache dir itself
PLATFORMIO_CCACHE = ("ESPHOME_PLATFORMIO_CCACHE_DIR", "platformio-ccache")
TOOLS_CACHE_SPECS = (
IDF_TOOLS_CACHE,
SDK_NRF_TOOLS_CACHE,
ARDUINO8266_TOOLS_CACHE,
HOST_TOOLS_CACHE,
PLATFORMIO_CCACHE,
)
esphome/cli/__init__.py
+2
View File
@@ -0,0 +1,2 @@
"""Commands of the esphome command line, one module each, imported by
__main__ only when they run so that startup stays light."""
esphome/cli/rename.py
+164
View File
@@ -0,0 +1,164 @@
"""``esphome rename``."""
from __future__ import annotations
import argparse
from pathlib import Path
import re
from esphome import yaml_edit, yaml_util
from esphome.const import (
ALLOWED_NAME_CHARS,
CONF_ESPHOME,
CONF_NAME,
CONF_SUBSTITUTIONS,
)
from esphome.core import CORE, EsphomeError
from esphome.log import AnsiFore, color
from esphome.types import ConfigType
from esphome.util import ESPHOME_COMMAND, run_external_process, safe_print
def _revert(new_path: Path, why: str) -> int:
"""Say why the rename stopped and take the new file back; an orphan the
next attempt would trip over is reported."""
safe_print(color(AnsiFore.BOLD_RED, f"Rename failed: {why}"))
try:
new_path.unlink(missing_ok=True)
except OSError as err:
safe_print(color(AnsiFore.BOLD_RED, f"Could not remove {new_path}: {err}"))
return 1
def command_rename(args: argparse.Namespace, config: ConfigType) -> int | None:
"""Rename the device: a new file with the name line rewritten, validated
and installed, then the old file removed."""
new_name = args.name
for c in new_name:
if c not in ALLOWED_NAME_CHARS:
safe_print(
color(
AnsiFore.BOLD_RED,
f"'{c}' is an invalid character for names. Valid characters are: "
f"{ALLOWED_NAME_CHARS} (lowercase, no spaces)",
)
)
return 1
yaml = yaml_util.load_yaml(CORE.config_path)
def name_edit() -> tuple[str, yaml_edit.LineEdit]:
"""The name and the line to rewrite: the name's own line, or the
substitution's line it comes from, as a plain value in this file."""
esphome_conf = yaml.get(CONF_ESPHOME)
if not isinstance(esphome_conf, dict) or CONF_NAME not in esphome_conf:
raise EsphomeError(f"no '{CONF_ESPHOME}: {CONF_NAME}:' in the file")
old_name = str(esphome_conf[CONF_NAME])
mapping, field = esphome_conf, CONF_NAME
if match := re.match(r"^\$\{?([a-zA-Z0-9_]+)\}?$", old_name):
mapping, field = yaml.get(CONF_SUBSTITUTIONS), match.group(1)
if not isinstance(mapping, dict) or field not in mapping:
raise EsphomeError(f"the substitution '{field}' is not in the file")
old_name = str(mapping[field])
# Only read here; the rewritten text goes to a new file, so the
# source may live anywhere the config path points to
source = yaml_edit.source_of(mapping, field)
if source is None or source[0].resolve() != CORE.config_path.resolve():
raise EsphomeError(f"'{field}' was not read from {CORE.config_path}")
doc, line_no = source
text = yaml_edit.line_at(doc, line_no)
if (line_match := yaml_edit.field_line_re(field, old_name).match(text)) is None:
raise EsphomeError(f"'{field}' is not a plain value on {doc}:{line_no + 1}")
# The new value is always quoted, whatever the old line had
return old_name, yaml_edit.LineEdit(
doc, line_no, text, yaml_edit.rewrite(line_match, new_name, '"')
)
try:
old_name, edit = name_edit()
except EsphomeError as err:
safe_print(
color(
AnsiFore.BOLD_RED,
f"Complex YAML files cannot be automatically renamed: {err}",
)
)
return 1
# ``new_name == old_name`` (after substitution resolution) is
# a no-op rewrite that would still queue a pointless re-flash.
# Catch it before the path-equality check below — covers the
# case where the config filename doesn't match the device name
# (e.g. ``weird-file.yaml`` whose ``esphome.name`` is
# ``kitchen``; running ``esphome rename weird-file.yaml kitchen``
# would otherwise just re-flash the same hostname).
if new_name == old_name:
safe_print(
color(
AnsiFore.BOLD_RED,
f"'{new_name}' is already the device's name.",
)
)
return 1
new_path: Path = CORE.config_dir / (new_name + ".yaml")
if new_path.resolve() == CORE.config_path.resolve():
safe_print(
color(
AnsiFore.BOLD_RED,
f"'{new_name}' is already the device's name.",
)
)
return 1
if new_path.exists():
safe_print(
color(
AnsiFore.BOLD_RED,
f"Cannot rename: {new_path} already exists. "
"Refusing to overwrite an existing configuration.",
)
)
return 1
safe_print(
f"Updating {color(AnsiFore.CYAN, str(CORE.config_path))} to {color(AnsiFore.CYAN, str(new_path))}"
)
print()
try:
yaml_edit.write_keeping_mode(
new_path,
yaml_edit.rewritten_text(yaml_edit.read_text(CORE.config_path), [edit]),
like=CORE.config_path,
)
except EsphomeError as err:
return _revert(new_path, str(err))
if run_external_process(*ESPHOME_COMMAND, "config", str(new_path)) != 0:
return _revert(new_path, "the new configuration does not validate")
cli_args = [
"run",
str(new_path),
"--no-logs",
"--device",
CORE.address,
]
if args.dashboard:
cli_args.insert(0, "--dashboard")
try:
rc = run_external_process(*ESPHOME_COMMAND, *cli_args)
except KeyboardInterrupt:
rc = 1
if rc != 0:
return _revert(
new_path,
"the install did not finish; the device may already run the new name",
)
CORE.config_path.unlink()
safe_print(color(AnsiFore.BOLD_GREEN, "SUCCESS"))
print()
return 0
esphome/codegen.py
+3 -1
View File
@@ -31,6 +31,7 @@ from esphome.cpp_generator import ( # noqa: F401
add_global,
add_library,
add_platformio_option,
extern_progmem_array,
get_variable,
get_variable_with_full_id,
is_template,
@@ -40,8 +41,10 @@ from esphome.cpp_generator import ( # noqa: F401
progmem_array,
safe_exp,
set_cpp_standard,
shared_progmem_array,
statement,
static_const_array,
static_function,
templatable,
variable,
with_local_variable,
@@ -64,7 +67,6 @@ from esphome.cpp_types import ( # noqa: F401
Application,
Component,
ComponentPtr,
Controller,
EntityBase,
EntityCategory,
ESPTime,
esphome/component_aliases.py
+1
View File
@@ -6,5 +6,6 @@ See the component-alias section of esphome/loader.py.
# alias -> (canonical component, removal version or None)
COMPONENT_ALIASES: dict[str, tuple[str, str | None]] = {
"esp32_improv": ("improv_ble", "2027.4.0"),
"rp2040": ("rp2", "2027.7.0"),
}
esphome/components/adc/adc_sensor.h
+1 -1
View File
@@ -94,7 +94,7 @@ class ADCSensor final : public sensor::Sensor, public PollingComponent, public v
/// - SamplingMode::MIN: Use the lowest sample value
/// - SamplingMode::MAX: Use the highest sample value
/// @param sampling_mode The desired sampling mode to use for aggregating ADC samples.
void set_sampling_mode(SamplingMode sampling_mode);
void set_sampling_mode(SamplingMode sampling_mode) { this->sampling_mode_ = sampling_mode; }
/// Perform a single ADC sampling operation and return the measured value.
/// This function handles raw readings, calibration, and averaging as needed.
esphome/components/adc/adc_sensor_common.cpp
-2
View File
@@ -76,6 +76,4 @@ void ADCSensor::set_sample_count(uint8_t sample_count) {
}
}
void ADCSensor::set_sampling_mode(SamplingMode sampling_mode) { this->sampling_mode_ = sampling_mode; }
} // namespace esphome::adc
esphome/components/ags10/ags10.cpp
+14
View File
@@ -106,6 +106,20 @@ bool AGS10Component::set_zero_point_with_factory_defaults() { return this->set_z
bool AGS10Component::set_zero_point_with_current_resistance() { return this->set_zero_point_with(ZP_CURRENT); }
void AGS10Component::set_zero_point(AGS10SetZeroPointActionMode mode, uint16_t value) {
switch (mode) {
case FACTORY_DEFAULT:
this->set_zero_point_with_factory_defaults();
break;
case CURRENT_VALUE:
this->set_zero_point_with_current_resistance();
break;
case CUSTOM_VALUE:
this->set_zero_point_with(value);
break;
}
}
bool AGS10Component::set_zero_point_with(uint16_t value) {
std::array<uint8_t, 5> data{0x00, 0x0C, (uint8_t) ((value >> 8) & 0xFF), (uint8_t) (value & 0xFF), 0};
data[4] = crc8(data.data(), 4, 0xFF, 0x31, true);
esphome/components/ags10/ags10.h
+14 -36
View File
@@ -2,11 +2,19 @@
#include "esphome/components/i2c/i2c.h"
#include "esphome/components/sensor/sensor.h"
#include "esphome/core/automation.h"
#include "esphome/core/component.h"
namespace esphome::ags10 {
enum AGS10SetZeroPointActionMode {
// Zero-point reset.
FACTORY_DEFAULT,
// Zero-point calibration with current resistance.
CURRENT_VALUE,
// Zero-point calibration with custom resistance.
CUSTOM_VALUE,
};
class AGS10Component final : public PollingComponent, public i2c::I2CDevice {
public:
/**
@@ -47,6 +55,11 @@ class AGS10Component final : public PollingComponent, public i2c::I2CDevice {
*/
bool set_zero_point_with_current_resistance();
/**
* Sets zero-point by mode; the value is only used for CUSTOM_VALUE.
*/
void set_zero_point(AGS10SetZeroPointActionMode mode, uint16_t value);
/**
* Sets zero-point with the value.
*/
@@ -100,39 +113,4 @@ class AGS10Component final : public PollingComponent, public i2c::I2CDevice {
template<size_t N> optional<std::array<uint8_t, N>> read_and_check_(uint8_t a_register);
};
template<typename... Ts> class AGS10NewI2cAddressAction final : public Action<Ts...>, public Parented<AGS10Component> {
public:
TEMPLATABLE_VALUE(uint8_t, new_address)
void play(const Ts &...x) override { this->parent_->new_i2c_address(this->new_address_.value(x...)); }
};
enum AGS10SetZeroPointActionMode {
// Zero-point reset.
FACTORY_DEFAULT,
// Zero-point calibration with current resistance.
CURRENT_VALUE,
// Zero-point calibration with custom resistance.
CUSTOM_VALUE,
};
template<typename... Ts> class AGS10SetZeroPointAction final : public Action<Ts...>, public Parented<AGS10Component> {
public:
TEMPLATABLE_VALUE(uint16_t, value)
TEMPLATABLE_VALUE(AGS10SetZeroPointActionMode, mode)
void play(const Ts &...x) override {
switch (this->mode_.value(x...)) {
case FACTORY_DEFAULT:
this->parent_->set_zero_point_with_factory_defaults();
break;
case CURRENT_VALUE:
this->parent_->set_zero_point_with_current_resistance();
break;
case CUSTOM_VALUE:
this->parent_->set_zero_point_with(this->value_.value(x...));
break;
}
}
};
} // namespace esphome::ags10
esphome/components/ags10/sensor.py
+11 -51
View File
@@ -17,8 +17,6 @@ from esphome.const import (
UNIT_OHM,
UNIT_PARTS_PER_BILLION,
)
from esphome.core import ID
from esphome.cpp_generator import MockObj, TemplateArgsType
from esphome.types import ConfigType
CONF_RESISTANCE = "resistance"
@@ -28,12 +26,6 @@ DEPENDENCIES = ["i2c"]
ags10_ns = cg.esphome_ns.namespace("ags10")
AGS10Component = ags10_ns.class_("AGS10Component", cg.PollingComponent, i2c.I2CDevice)
# Actions
AGS10NewI2cAddressAction = ags10_ns.class_(
"AGS10NewI2cAddressAction", automation.Action
)
AGS10SetZeroPointAction = ags10_ns.class_("AGS10SetZeroPointAction", automation.Action)
CONFIG_SCHEMA = (
cv.Schema(
{
@@ -70,16 +62,10 @@ async def to_code(config: ConfigType) -> None:
await cg.register_component(var, config)
await i2c.register_i2c_device(var, config)
sens = await sensor.new_sensor(config[CONF_TVOC])
cg.add(var.set_tvoc(sens))
if version_config := config.get(CONF_VERSION):
sens = await sensor.new_sensor(version_config)
cg.add(var.set_version(sens))
if resistance_config := config.get(CONF_RESISTANCE):
sens = await sensor.new_sensor(resistance_config)
cg.add(var.set_resistance(sens))
sensors = sensor.sub_sensors(config)
await sensors(CONF_TVOC, var.set_tvoc)
await sensors(CONF_VERSION, var.set_version)
await sensors(CONF_RESISTANCE, var.set_resistance)
AGS10_NEW_I2C_ADDRESS_SCHEMA = cv.maybe_simple_value(
@@ -91,24 +77,11 @@ AGS10_NEW_I2C_ADDRESS_SCHEMA = cv.maybe_simple_value(
)
@automation.register_action(
automation.register_apply_action(
"ags10.new_i2c_address",
AGS10NewI2cAddressAction,
AGS10_NEW_I2C_ADDRESS_SCHEMA,
synchronous=True,
automation.ApplyField(CONF_ADDRESS, "new_i2c_address", cg.uint8),
)
async def ags10newi2caddress_to_code(
config: ConfigType,
action_id: ID,
template_arg: cg.TemplateArguments,
args: TemplateArgsType,
) -> MockObj:
var = cg.new_Pvariable(action_id, template_arg)
await cg.register_parented(var, config[CONF_ID])
address = await cg.templatable(config[CONF_ADDRESS], args, cg.uint8)
cg.add(var.set_new_address(address))
return var
AGS10SetZeroPointActionMode = ags10_ns.enum("AGS10SetZeroPointActionMode")
AGS10_SET_ZERO_POINT_ACTION_MODE = {
@@ -128,24 +101,11 @@ AGS10_SET_ZERO_POINT_SCHEMA = cv.Schema(
)
@automation.register_action(
automation.register_apply_action(
"ags10.set_zero_point",
AGS10SetZeroPointAction,
AGS10_SET_ZERO_POINT_SCHEMA,
synchronous=True,
automation.ApplyCall(
"set_zero_point({}, {})",
((CONF_MODE, AGS10SetZeroPointActionMode), (CONF_VALUE, cg.uint16)),
),
)
async def ags10setzeropoint_to_code(
config: ConfigType,
action_id: ID,
template_arg: cg.TemplateArguments,
args: TemplateArgsType,
) -> MockObj:
var = cg.new_Pvariable(action_id, template_arg)
await cg.register_parented(var, config[CONF_ID])
mode = await cg.templatable(
config.get(CONF_MODE), args, AGS10SetZeroPointActionMode
)
cg.add(var.set_mode(mode))
value = await cg.templatable(config[CONF_VALUE], args, cg.uint16)
cg.add(var.set_value(value))
return var
esphome/components/aht10/sensor.py
+3 -7
View File
@@ -57,10 +57,6 @@ async def to_code(config: ConfigType) -> None:
await i2c.register_i2c_device(var, config)
cg.add(var.set_variant(config[CONF_VARIANT]))
if temperature := config.get(CONF_TEMPERATURE):
sens = await sensor.new_sensor(temperature)
cg.add(var.set_temperature_sensor(sens))
if humidity := config.get(CONF_HUMIDITY):
sens = await sensor.new_sensor(humidity)
cg.add(var.set_humidity_sensor(sens))
sensors = sensor.sub_sensors(config)
await sensors(CONF_TEMPERATURE, var.set_temperature_sensor)
await sensors(CONF_HUMIDITY, var.set_humidity_sensor)
esphome/components/aic3204/audio_dac.py
+2 -19
View File
@@ -4,8 +4,6 @@ from esphome.components import i2c
from esphome.components.audio_dac import AudioDac
import esphome.config_validation as cv
from esphome.const import CONF_ID, CONF_MODE
from esphome.core import ID
from esphome.cpp_generator import MockObj, TemplateArgsType
from esphome.types import ConfigType
CODEOWNERS = ["@kbx81"]
@@ -14,7 +12,6 @@ DEPENDENCIES = ["i2c"]
aic3204_ns = cg.esphome_ns.namespace("aic3204")
AIC3204 = aic3204_ns.class_("AIC3204", AudioDac, cg.Component, i2c.I2CDevice)
SetAutoMuteAction = aic3204_ns.class_("SetAutoMuteAction", automation.Action)
CONFIG_SCHEMA = (
cv.Schema(
@@ -36,25 +33,11 @@ SET_AUTO_MUTE_ACTION_SCHEMA = cv.maybe_simple_value(
)
@automation.register_action(
automation.register_apply_action(
"aic3204.set_auto_mute_mode",
SetAutoMuteAction,
SET_AUTO_MUTE_ACTION_SCHEMA,
synchronous=True,
automation.ApplyField(CONF_MODE, "set_auto_mute_mode", cg.uint8),
)
async def aic3204_set_volume_to_code(
config: ConfigType,
action_id: ID,
template_arg: cg.TemplateArguments,
args: TemplateArgsType,
) -> MockObj:
paren = await cg.get_variable(config[CONF_ID])
var = cg.new_Pvariable(action_id, template_arg, paren)
template_ = await cg.templatable(config.get(CONF_MODE), args, cg.uint8)
cg.add(var.set_auto_mute_mode(template_))
return var
async def to_code(config: ConfigType) -> None:
esphome/components/aic3204/automation.h
-21
View File
@@ -1,21 +0,0 @@
#pragma once
#include "esphome/core/automation.h"
#include "esphome/core/component.h"
#include "aic3204.h"
namespace esphome::aic3204 {
template<typename... Ts> class SetAutoMuteAction final : public Action<Ts...> {
public:
explicit SetAutoMuteAction(AIC3204 *aic3204) : aic3204_(aic3204) {}
TEMPLATABLE_VALUE(uint8_t, auto_mute_mode)
void play(const Ts &...x) override { this->aic3204_->set_auto_mute_mode(this->auto_mute_mode_.value(x...)); }
protected:
AIC3204 *aic3204_;
};
} // namespace esphome::aic3204
esphome/components/airthings_wave_base/__init__.py
+6 -15
View File
@@ -85,20 +85,11 @@ async def wave_base_to_code(var: MockObj, config: ConfigType) -> None:
await ble_client.register_ble_node(var, config)
if config_humidity := config.get(CONF_HUMIDITY):
sens = await sensor.new_sensor(config_humidity)
cg.add(var.set_humidity(sens))
if config_temperature := config.get(CONF_TEMPERATURE):
sens = await sensor.new_sensor(config_temperature)
cg.add(var.set_temperature(sens))
if config_pressure := config.get(CONF_PRESSURE):
sens = await sensor.new_sensor(config_pressure)
cg.add(var.set_pressure(sens))
if config_tvoc := config.get(CONF_TVOC):
sens = await sensor.new_sensor(config_tvoc)
cg.add(var.set_tvoc(sens))
if config_battery_voltage := config.get(CONF_BATTERY_VOLTAGE):
sens = await sensor.new_sensor(config_battery_voltage)
cg.add(var.set_battery_voltage(sens))
sensors = sensor.sub_sensors(config)
await sensors(CONF_HUMIDITY, var.set_humidity)
await sensors(CONF_TEMPERATURE, var.set_temperature)
await sensors(CONF_PRESSURE, var.set_pressure)
await sensors(CONF_TVOC, var.set_tvoc)
await sensors(CONF_BATTERY_VOLTAGE, var.set_battery_voltage)
if config_battery_update_interval := config.get(CONF_BATTERY_UPDATE_INTERVAL):
cg.add(var.set_battery_update_interval(config_battery_update_interval))
esphome/components/airthings_wave_plus/sensor.py
+5 -12
View File
@@ -87,16 +87,9 @@ async def to_code(config: ConfigType) -> None:
var = cg.new_Pvariable(config[CONF_ID])
await airthings_wave_base.wave_base_to_code(var, config)
if config_radon := config.get(CONF_RADON):
sens = await sensor.new_sensor(config_radon)
cg.add(var.set_radon(sens))
if config_radon_long_term := config.get(CONF_RADON_LONG_TERM):
sens = await sensor.new_sensor(config_radon_long_term)
cg.add(var.set_radon_long_term(sens))
if config_co2 := config.get(CONF_CO2):
sens = await sensor.new_sensor(config_co2)
cg.add(var.set_co2(sens))
if config_illuminance := config.get(CONF_ILLUMINANCE):
sens = await sensor.new_sensor(config_illuminance)
cg.add(var.set_illuminance(sens))
sensors = sensor.sub_sensors(config)
await sensors(CONF_RADON, var.set_radon)
await sensors(CONF_RADON_LONG_TERM, var.set_radon_long_term)
await sensors(CONF_CO2, var.set_co2)
await sensors(CONF_ILLUMINANCE, var.set_illuminance)
cg.add(var.set_device_type(config[CONF_DEVICE_TYPE]))
esphome/components/alarm_control_panel/__init__.py
+31 -130
View File
@@ -41,18 +41,6 @@ StateAnyForwarder = alarm_control_panel_ns.class_("StateAnyForwarder")
StateEnterForwarder = alarm_control_panel_ns.class_("StateEnterForwarder")
AlarmControlPanelState = alarm_control_panel_ns.enum("AlarmControlPanelState")
ArmAwayAction = alarm_control_panel_ns.class_("ArmAwayAction", automation.Action)
ArmHomeAction = alarm_control_panel_ns.class_("ArmHomeAction", automation.Action)
ArmNightAction = alarm_control_panel_ns.class_("ArmNightAction", automation.Action)
DisarmAction = alarm_control_panel_ns.class_("DisarmAction", automation.Action)
PendingAction = alarm_control_panel_ns.class_("PendingAction", automation.Action)
TriggeredAction = alarm_control_panel_ns.class_("TriggeredAction", automation.Action)
ChimeAction = alarm_control_panel_ns.class_("ChimeAction", automation.Action)
ReadyAction = alarm_control_panel_ns.class_("ReadyAction", automation.Action)
AlarmControlPanelCondition = alarm_control_panel_ns.class_(
"AlarmControlPanelCondition", automation.Condition
)
_ALARM_CONTROL_PANEL_SCHEMA = (
cv.ENTITY_BASE_SCHEMA.extend(web_server.WEBSERVER_SORTING_SCHEMA)
@@ -196,125 +184,38 @@ async def new_alarm_control_panel(config, *args):
return var
@automation.register_action(
"alarm_control_panel.arm_away",
ArmAwayAction,
ALARM_CONTROL_PANEL_ACTION_SCHEMA,
synchronous=True,
)
async def alarm_action_arm_away_to_code(config, action_id, template_arg, args):
paren = await cg.get_variable(config[CONF_ID])
var = cg.new_Pvariable(action_id, template_arg, paren)
if code_config := config.get(CONF_CODE):
templatable_ = await cg.templatable(code_config, args, cg.std_string)
cg.add(var.set_code(templatable_))
return var
@automation.register_action(
"alarm_control_panel.arm_home",
ArmHomeAction,
ALARM_CONTROL_PANEL_ACTION_SCHEMA,
synchronous=True,
)
async def alarm_action_arm_home_to_code(config, action_id, template_arg, args):
paren = await cg.get_variable(config[CONF_ID])
var = cg.new_Pvariable(action_id, template_arg, paren)
if code_config := config.get(CONF_CODE):
templatable_ = await cg.templatable(code_config, args, cg.std_string)
cg.add(var.set_code(templatable_))
return var
@automation.register_action(
"alarm_control_panel.arm_night",
ArmNightAction,
ALARM_CONTROL_PANEL_ACTION_SCHEMA,
synchronous=True,
)
async def alarm_action_arm_night_to_code(config, action_id, template_arg, args):
paren = await cg.get_variable(config[CONF_ID])
var = cg.new_Pvariable(action_id, template_arg, paren)
if CONF_CODE in config:
templatable_ = await cg.templatable(config[CONF_CODE], args, cg.std_string)
cg.add(var.set_code(templatable_))
return var
@automation.register_action(
"alarm_control_panel.disarm",
DisarmAction,
ALARM_CONTROL_PANEL_ACTION_SCHEMA,
synchronous=True,
)
async def alarm_action_disarm_to_code(config, action_id, template_arg, args):
paren = await cg.get_variable(config[CONF_ID])
var = cg.new_Pvariable(action_id, template_arg, paren)
if code_config := config.get(CONF_CODE):
templatable_ = await cg.templatable(code_config, args, cg.std_string)
cg.add(var.set_code(templatable_))
return var
@automation.register_action(
"alarm_control_panel.pending",
PendingAction,
ALARM_CONTROL_PANEL_ACTION_SCHEMA,
synchronous=True,
)
async def alarm_action_pending_to_code(config, action_id, template_arg, args):
paren = await cg.get_variable(config[CONF_ID])
return cg.new_Pvariable(action_id, template_arg, paren)
@automation.register_action(
"alarm_control_panel.triggered",
TriggeredAction,
ALARM_CONTROL_PANEL_ACTION_SCHEMA,
synchronous=True,
)
async def alarm_action_trigger_to_code(config, action_id, template_arg, args):
paren = await cg.get_variable(config[CONF_ID])
return cg.new_Pvariable(action_id, template_arg, paren)
@automation.register_action(
"alarm_control_panel.chime",
ChimeAction,
ALARM_CONTROL_PANEL_ACTION_SCHEMA,
synchronous=True,
)
async def alarm_action_chime_to_code(config, action_id, template_arg, args):
paren = await cg.get_variable(config[CONF_ID])
return cg.new_Pvariable(action_id, template_arg, paren)
@automation.register_action(
"alarm_control_panel.ready",
ReadyAction,
ALARM_CONTROL_PANEL_ACTION_SCHEMA,
synchronous=True,
)
@automation.register_condition(
"alarm_control_panel.ready",
AlarmControlPanelCondition,
ALARM_CONTROL_PANEL_CONDITION_SCHEMA,
)
async def alarm_action_ready_to_code(config, action_id, template_arg, args):
paren = await cg.get_variable(config[CONF_ID])
return cg.new_Pvariable(action_id, template_arg, paren)
@automation.register_condition(
"alarm_control_panel.is_armed",
AlarmControlPanelCondition,
ALARM_CONTROL_PANEL_CONDITION_SCHEMA,
)
async def alarm_control_panel_is_armed_to_code(
config, condition_id, template_arg, args
# Mirrors AlarmControlPanel::arm_with_code_: arm first, set the code only when given.
for _name, _arm in (
("alarm_control_panel.arm_away", "arm_away()"),
("alarm_control_panel.arm_home", "arm_home()"),
("alarm_control_panel.arm_night", "arm_night()"),
("alarm_control_panel.disarm", "disarm()"),
):
paren = await cg.get_variable(config[CONF_ID])
return cg.new_Pvariable(condition_id, template_arg, paren)
automation.register_apply_action(
_name,
ALARM_CONTROL_PANEL_ACTION_SCHEMA,
automation.ApplyCall(_arm),
automation.ApplyField(CONF_CODE, "set_code", cg.std_string),
call="make_call",
)
for _name, _call in (
("alarm_control_panel.pending", "pending()"),
("alarm_control_panel.triggered", "triggered()"),
):
automation.register_apply_action(
_name,
ALARM_CONTROL_PANEL_ACTION_SCHEMA,
automation.ApplyCall(_call),
call="make_call",
)
for _name in ("alarm_control_panel.ready", "alarm_control_panel.is_armed"):
automation.register_apply_condition(
_name, ALARM_CONTROL_PANEL_CONDITION_SCHEMA, "is_armed_pending_or_triggered()"
)
@coroutine_with_priority(CoroPriority.CORE)
esphome/components/alarm_control_panel/alarm_control_panel.h
+9 -3
View File
@@ -130,6 +130,12 @@ class AlarmControlPanel : public EntityBase {
// is the state one of the armed states
bool is_state_armed(AlarmControlPanelState state);
/// Armed, pending (entry delay) or triggered; not ARMING (exit delay). Backs the is_armed and ready
/// conditions.
bool is_armed_pending_or_triggered() {
auto state = this->get_state();
return this->is_state_armed(state) || state == ACP_STATE_PENDING || state == ACP_STATE_TRIGGERED;
}
protected:
friend AlarmControlPanelCall;
@@ -138,11 +144,11 @@ class AlarmControlPanel : public EntityBase {
// in order to store last panel state in flash
ESPPreferenceObject pref_;
// current state
AlarmControlPanelState current_state_;
AlarmControlPanelState current_state_{ACP_STATE_DISARMED};
// the desired (or previous) state
AlarmControlPanelState desired_state_;
AlarmControlPanelState desired_state_{ACP_STATE_DISARMED};
// last time the state was updated
uint32_t last_update_;
uint32_t last_update_{0};
// the call control function
virtual void control(const AlarmControlPanelCall &call) = 0;
// state callback - passes the new state to listeners
esphome/components/alarm_control_panel/automation.h
-80
View File
@@ -27,84 +27,4 @@ static_assert(std::is_trivially_copyable_v<StateAnyForwarder>);
static_assert(sizeof(StateEnterForwarder<ACP_STATE_TRIGGERED>) <= sizeof(void *));
static_assert(std::is_trivially_copyable_v<StateEnterForwarder<ACP_STATE_TRIGGERED>>);
template<typename... Ts> class ArmAwayAction final : public Action<Ts...> {
public:
explicit ArmAwayAction(AlarmControlPanel *alarm_control_panel) : alarm_control_panel_(alarm_control_panel) {}
TEMPLATABLE_VALUE(std::string, code)
void play(const Ts &...x) override { this->alarm_control_panel_->arm_away(this->code_.optional_value(x...)); }
protected:
AlarmControlPanel *alarm_control_panel_;
};
template<typename... Ts> class ArmHomeAction final : public Action<Ts...> {
public:
explicit ArmHomeAction(AlarmControlPanel *alarm_control_panel) : alarm_control_panel_(alarm_control_panel) {}
TEMPLATABLE_VALUE(std::string, code)
void play(const Ts &...x) override { this->alarm_control_panel_->arm_home(this->code_.optional_value(x...)); }
protected:
AlarmControlPanel *alarm_control_panel_;
};
template<typename... Ts> class ArmNightAction final : public Action<Ts...> {
public:
explicit ArmNightAction(AlarmControlPanel *alarm_control_panel) : alarm_control_panel_(alarm_control_panel) {}
TEMPLATABLE_VALUE(std::string, code)
void play(const Ts &...x) override { this->alarm_control_panel_->arm_night(this->code_.optional_value(x...)); }
protected:
AlarmControlPanel *alarm_control_panel_;
};
template<typename... Ts> class DisarmAction final : public Action<Ts...> {
public:
explicit DisarmAction(AlarmControlPanel *alarm_control_panel) : alarm_control_panel_(alarm_control_panel) {}
TEMPLATABLE_VALUE(std::string, code)
void play(const Ts &...x) override { this->alarm_control_panel_->disarm(this->code_.optional_value(x...)); }
protected:
AlarmControlPanel *alarm_control_panel_;
};
template<typename... Ts> class PendingAction final : public Action<Ts...> {
public:
explicit PendingAction(AlarmControlPanel *alarm_control_panel) : alarm_control_panel_(alarm_control_panel) {}
void play(const Ts &...x) override { this->alarm_control_panel_->make_call().pending().perform(); }
protected:
AlarmControlPanel *alarm_control_panel_;
};
template<typename... Ts> class TriggeredAction final : public Action<Ts...> {
public:
explicit TriggeredAction(AlarmControlPanel *alarm_control_panel) : alarm_control_panel_(alarm_control_panel) {}
void play(const Ts &...x) override { this->alarm_control_panel_->make_call().triggered().perform(); }
protected:
AlarmControlPanel *alarm_control_panel_;
};
template<typename... Ts> class AlarmControlPanelCondition final : public Condition<Ts...> {
public:
AlarmControlPanelCondition(AlarmControlPanel *parent) : parent_(parent) {}
bool check(const Ts &...x) override {
return this->parent_->is_state_armed(this->parent_->get_state()) ||
this->parent_->get_state() == ACP_STATE_PENDING || this->parent_->get_state() == ACP_STATE_TRIGGERED;
}
protected:
AlarmControlPanel *parent_;
};
} // namespace esphome::alarm_control_panel
esphome/components/alpha3/sensor.py
+7 -23
View File
@@ -74,26 +74,10 @@ async def to_code(config: ConfigType) -> None:
await cg.register_component(var, config)
await ble_client.register_ble_node(var, config)
if flow_config := config.get(CONF_FLOW):
sens = await sensor.new_sensor(flow_config)
cg.add(var.set_flow_sensor(sens))
if head_config := config.get(CONF_HEAD):
sens = await sensor.new_sensor(head_config)
cg.add(var.set_head_sensor(sens))
if power_config := config.get(CONF_POWER):
sens = await sensor.new_sensor(power_config)
cg.add(var.set_power_sensor(sens))
if current_config := config.get(CONF_CURRENT):
sens = await sensor.new_sensor(current_config)
cg.add(var.set_current_sensor(sens))
if speed_config := config.get(CONF_SPEED):
sens = await sensor.new_sensor(speed_config)
cg.add(var.set_speed_sensor(sens))
if voltage_config := config.get(CONF_VOLTAGE):
sens = await sensor.new_sensor(voltage_config)
cg.add(var.set_voltage_sensor(sens))
sensors = sensor.sub_sensors(config)
await sensors(CONF_FLOW, var.set_flow_sensor)
await sensors(CONF_HEAD, var.set_head_sensor)
await sensors(CONF_POWER, var.set_power_sensor)
await sensors(CONF_CURRENT, var.set_current_sensor)
await sensors(CONF_SPEED, var.set_speed_sensor)
await sensors(CONF_VOLTAGE, var.set_voltage_sensor)
esphome/components/am2315c/sensor.py
+3 -7
View File
@@ -46,10 +46,6 @@ async def to_code(config: ConfigType) -> None:
await cg.register_component(var, config)
await i2c.register_i2c_device(var, config)
if temperature_config := config.get(CONF_TEMPERATURE):
sens = await sensor.new_sensor(temperature_config)
cg.add(var.set_temperature_sensor(sens))
if humidity_config := config.get(CONF_HUMIDITY):
sens = await sensor.new_sensor(humidity_config)
cg.add(var.set_humidity_sensor(sens))
sensors = sensor.sub_sensors(config)
await sensors(CONF_TEMPERATURE, var.set_temperature_sensor)
await sensors(CONF_HUMIDITY, var.set_humidity_sensor)
esphome/components/am2320/sensor.py
+3 -7
View File
@@ -48,10 +48,6 @@ async def to_code(config: ConfigType) -> None:
await cg.register_component(var, config)
await i2c.register_i2c_device(var, config)
if temperature_config := config.get(CONF_TEMPERATURE):
sens = await sensor.new_sensor(temperature_config)
cg.add(var.set_temperature_sensor(sens))
if humidity_config := config.get(CONF_HUMIDITY):
sens = await sensor.new_sensor(humidity_config)
cg.add(var.set_humidity_sensor(sens))
sensors = sensor.sub_sensors(config)
await sensors(CONF_TEMPERATURE, var.set_temperature_sensor)
await sensors(CONF_HUMIDITY, var.set_humidity_sensor)
esphome/components/am43/sensor/__init__.py
+3 -7
View File
@@ -48,10 +48,6 @@ async def to_code(config: ConfigType) -> None:
await cg.register_component(var, config)
await ble_client.register_ble_node(var, config)
if battery_level_config := config.get(CONF_BATTERY_LEVEL):
sens = await sensor.new_sensor(battery_level_config)
cg.add(var.set_battery(sens))
if illuminance_config := config.get(CONF_ILLUMINANCE):
sens = await sensor.new_sensor(illuminance_config)
cg.add(var.set_illuminance(sens))
sensors = sensor.sub_sensors(config)
await sensors(CONF_BATTERY_LEVEL, var.set_battery)
await sensors(CONF_ILLUMINANCE, var.set_illuminance)
esphome/components/animation/animation.h
-30
View File
@@ -1,8 +1,6 @@
#pragma once
#include "esphome/components/image/image.h"
#include "esphome/core/automation.h"
namespace esphome::animation {
class Animation final : public image::Image {
@@ -35,32 +33,4 @@ class Animation final : public image::Image {
int loop_current_iteration_;
};
template<typename... Ts> class AnimationNextFrameAction final : public Action<Ts...> {
public:
AnimationNextFrameAction(Animation *parent) : parent_(parent) {}
void play(const Ts &...x) override { this->parent_->next_frame(); }
protected:
Animation *parent_;
};
template<typename... Ts> class AnimationPrevFrameAction final : public Action<Ts...> {
public:
AnimationPrevFrameAction(Animation *parent) : parent_(parent) {}
void play(const Ts &...x) override { this->parent_->prev_frame(); }
protected:
Animation *parent_;
};
template<typename... Ts> class AnimationSetFrameAction final : public Action<Ts...> {
public:
AnimationSetFrameAction(Animation *parent) : parent_(parent) {}
TEMPLATABLE_VALUE(uint16_t, frame)
void play(const Ts &...x) override { this->parent_->set_frame(this->frame_.value(x...)); }
protected:
Animation *parent_;
};
} // namespace esphome::animation
esphome/components/animation/image.py
+8 -32
View File
@@ -6,8 +6,6 @@ from esphome.components.file.image import image_schema, write_image
from esphome.components.image import Image_, validate_settings
import esphome.config_validation as cv
from esphome.const import CONF_ID, CONF_REPEAT
from esphome.core import ID
from esphome.cpp_generator import MockObj, TemplateArgsType
from esphome.types import ConfigType
CODEOWNERS = ["@syndlex"]
@@ -26,17 +24,6 @@ animation_ns = cg.esphome_ns.namespace("animation")
Animation_ = animation_ns.class_("Animation", Image_)
# Actions
NextFrameAction = animation_ns.class_(
"AnimationNextFrameAction", automation.Action, cg.Parented.template(Animation_)
)
PrevFrameAction = animation_ns.class_(
"AnimationPrevFrameAction", automation.Action, cg.Parented.template(Animation_)
)
SetFrameAction = animation_ns.class_(
"AnimationSetFrameAction", automation.Action, cg.Parented.template(Animation_)
)
ANIMATION_SCHEMA = image_schema(Animation_).extend(
{
cv.Optional(CONF_LOOP): cv.All(
@@ -72,28 +59,17 @@ SET_FRAME_SCHEMA = cv.Schema(
)
@automation.register_action(
"animation.next_frame", NextFrameAction, NEXT_FRAME_SCHEMA, synchronous=True
automation.register_apply_action(
"animation.next_frame", NEXT_FRAME_SCHEMA, automation.ApplyCall("next_frame()")
)
@automation.register_action(
"animation.prev_frame", PrevFrameAction, PREV_FRAME_SCHEMA, synchronous=True
automation.register_apply_action(
"animation.prev_frame", PREV_FRAME_SCHEMA, automation.ApplyCall("prev_frame()")
)
@automation.register_action(
"animation.set_frame", SetFrameAction, SET_FRAME_SCHEMA, synchronous=True
automation.register_apply_action(
"animation.set_frame",
SET_FRAME_SCHEMA,
automation.ApplyField(CONF_FRAME, "set_frame", cg.uint16),
)
async def animation_action_to_code(
config: ConfigType,
action_id: ID,
template_arg: cg.TemplateArguments,
args: TemplateArgsType,
) -> MockObj:
paren = await cg.get_variable(config[CONF_ID])
var = cg.new_Pvariable(action_id, template_arg, paren)
if (frame := config.get(CONF_FRAME)) is not None:
template_ = await cg.templatable(frame, args, cg.uint16)
cg.add(var.set_frame(template_))
return var
async def setup_animation(config: ConfigType) -> None:
esphome/components/api/__init__.py
+242 -93
View File
@@ -1,3 +1,4 @@
from ipaddress import IPv4Address, IPv6Address
import logging
import re
from typing import Any
@@ -5,7 +6,7 @@ from typing import Any
from esphome import automation
from esphome.automation import Condition
import esphome.codegen as cg
from esphome.components.const import CONF_DESCRIPTION
from esphome.components.const import CONF_DESCRIPTION, CONF_HOST
from esphome.components.logger import request_log_listener
# ENCRYPTION_SCHEMA and validate_encryption_key are re-exported for external
@@ -13,6 +14,7 @@ from esphome.components.logger import request_log_listener
from esphome.components.noise import ( # noqa: F401
ENCRYPTION_SCHEMA,
decode_encryption_key,
enable_spare_ephemeral,
encryption_schema,
new_psk_progmem,
validate_encryption_key,
@@ -25,6 +27,8 @@ from esphome.const import (
CONF_CAPTURE_RESPONSE,
CONF_DATA,
CONF_DATA_TEMPLATE,
CONF_DELAY,
CONF_ENABLE_IPV6,
CONF_ENCRYPTION,
CONF_EVENT,
CONF_ID,
@@ -47,10 +51,14 @@ from esphome.const import (
CONF_VARIABLES,
)
from esphome.core import CORE, ID, CoroPriority, EsphomeError, coroutine_with_priority
from esphome.cpp_generator import MockObj, TemplateArgsType
from esphome.helpers import fnv1_hash
from esphome.cpp_generator import Expression, MockObj, TemplateArgsType
import esphome.final_validate as fv
from esphome.helpers import cpp_string_escape, fnv1_hash
from esphome.schema_extractors import SCHEMA_EXTRACT, schema_extractor
from esphome.types import ConfigFragmentType, ConfigType
from . import wizard
# Compat alias: downstream consumers (e.g. device-builder) referenced the
# schema by its old private name before it moved to the noise component
_encryption_schema = encryption_schema
@@ -81,10 +89,11 @@ def AUTO_LOAD(config: ConfigType) -> list[str]:
api_ns = cg.esphome_ns.namespace("api")
APIServer = api_ns.class_("APIServer", cg.Component, cg.Controller)
APIServer = api_ns.class_("APIServer", cg.Component)
HomeAssistantServiceCallAction = api_ns.class_(
"HomeAssistantServiceCallAction", automation.Action
)
HomeAssistantField = api_ns.struct("HomeAssistantField")
ActionResponse = api_ns.class_("ActionResponse")
HomeAssistantActionResponseTrigger = api_ns.class_(
"HomeAssistantActionResponseTrigger", automation.Trigger
@@ -134,8 +143,16 @@ CONF_HOMEASSISTANT_SERVICES = "homeassistant_services"
CONF_HOMEASSISTANT_STATES = "homeassistant_states"
CONF_LISTEN_BACKLOG = "listen_backlog"
CONF_MAX_SEND_QUEUE = "max_send_queue"
CONF_OUTGOING_CONNECTION = "outgoing_connection"
CONF_STATE_SUBSCRIPTION_ONLY = "state_subscription_only"
# Schema defaults that also match the C++ initializers in api_server.h; codegen
# skips the setter when the config equals them.
DEFAULT_PORT = 6053
DEFAULT_REBOOT_TIMEOUT = "15min"
DEFAULT_BATCH_DELAY = "100ms"
DEFAULT_LISTEN_BACKLOG = 4
def _register_provisioning_source(config: ConfigType) -> ConfigType:
"""Register the API as a provisioning source when encryption is enabled.
@@ -285,14 +302,77 @@ def _consume_api_sockets(config: ConfigType) -> ConfigType:
# (not max_connections, which is the upper limit rarely reached)
socket.consume_sockets(3, "api")(config)
socket.consume_sockets(1, "api", socket.SocketType.TCP_LISTEN)(config)
if CONF_OUTGOING_CONNECTION in config:
socket.consume_sockets(1, "api_outgoing_connection")(config)
return config
def _validate_outgoing_connection(config: ConfigType) -> ConfigType:
if (outgoing := config.get(CONF_OUTGOING_CONNECTION)) is None:
return config
if CONF_ENCRYPTION not in config:
raise cv.Invalid(
"outgoing_connection requires 'encryption' so the peer is verified by key",
path=[CONF_OUTGOING_CONNECTION],
)
# A device with no client reboots once reboot_timeout passes, so a delay that
# reaches it would reboot the device before it ever dials
reboot_timeout = config[CONF_REBOOT_TIMEOUT]
delay = outgoing[CONF_DELAY]
if reboot_timeout.total_milliseconds and delay >= reboot_timeout:
raise cv.Invalid(
f"delay must be shorter than reboot_timeout ({reboot_timeout}), "
"otherwise the device reboots before it dials",
path=[CONF_OUTGOING_CONNECTION, CONF_DELAY],
)
return config
def _validate_outgoing_host(value: str) -> IPv4Address | IPv6Address:
"""Only accept an address the device itself can parse.
Python accepts a scope id, which neither `inet_pton` nor lwIP's `inet6_aton`
takes, and a v4-mapped address is dialed as plain IPv4, needing no IPv6 build.
"""
address = cv.ipaddress(value)
if isinstance(address, IPv6Address):
if address.scope_id is not None:
raise cv.Invalid(
f"{value} carries a scope id, which the device cannot parse; "
"give the address without the '%' part"
)
if (mapped := address.ipv4_mapped) is not None:
return mapped
return address
_OUTGOING_CONNECTION_SCHEMA = cv.Schema(
{
cv.Optional(CONF_HOST): _validate_outgoing_host,
cv.Optional(CONF_PORT, default=6054): cv.port,
# Bounded against reboot_timeout in _validate_outgoing_connection
cv.Optional(CONF_DELAY, default="60s"): cv.positive_time_period_milliseconds,
}
)
@schema_extractor("schema")
def _outgoing_connection_schema(config: ConfigType | None) -> ConfigType:
# A bare `outgoing_connection:` block is valid; without a host the device
# dials the remembered last dial-back client
if config is SCHEMA_EXTRACT:
# Let the language-schema dumper walk host, port and delay
return _OUTGOING_CONNECTION_SCHEMA
if config is None:
config = {}
return _OUTGOING_CONNECTION_SCHEMA(config)
CONFIG_SCHEMA = cv.All(
cv.Schema(
{
cv.GenerateID(): cv.declare_id(APIServer),
cv.Optional(CONF_PORT, default=6053): cv.port,
cv.Optional(CONF_PORT, default=DEFAULT_PORT): cv.port,
# Removed in 2026.1.0 - kept to provide helpful error message
cv.Optional(CONF_PASSWORD): cv.invalid(
"The 'password' option has been removed in ESPHome 2026.1.0.\n"
@@ -305,14 +385,16 @@ CONFIG_SCHEMA = cv.All(
"Or visit https://esphome.io/components/api/#configuration-variables"
),
cv.Optional(
CONF_REBOOT_TIMEOUT, default="15min"
CONF_REBOOT_TIMEOUT, default=DEFAULT_REBOOT_TIMEOUT
): cv.positive_time_period_milliseconds,
cv.Exclusive(
CONF_SERVICES, group_of_exclusion=CONF_ACTIONS
): ACTIONS_SCHEMA,
cv.Exclusive(CONF_ACTIONS, group_of_exclusion=CONF_ACTIONS): ACTIONS_SCHEMA,
cv.Optional(CONF_ENCRYPTION): encryption_schema,
cv.Optional(CONF_BATCH_DELAY, default="100ms"): cv.All(
cv.Optional(wizard.CONF_WIZARD): wizard.WIZARD_SCHEMA,
cv.Optional(CONF_OUTGOING_CONNECTION): _outgoing_connection_schema,
cv.Optional(CONF_BATCH_DELAY, default=DEFAULT_BATCH_DELAY): cv.All(
cv.positive_time_period_milliseconds,
cv.Range(max=cv.TimePeriod(milliseconds=65535)),
),
@@ -367,6 +449,7 @@ CONFIG_SCHEMA = cv.All(
}
).extend(cv.COMPONENT_SCHEMA),
cv.rename_key(CONF_SERVICES, CONF_ACTIONS),
_validate_outgoing_connection,
_consume_api_sockets,
_register_provisioning_source,
)
@@ -423,7 +506,34 @@ def _validate_esp8266_action_strings(config: ConfigType) -> ConfigType:
return config
FINAL_VALIDATE_SCHEMA = _validate_esp8266_action_strings
def _validate_outgoing_host_ipv6(config: ConfigType) -> ConfigType:
"""An IPv6 host can never be parsed, so never dialed, without IPv6."""
if (
(outgoing := config.get(CONF_OUTGOING_CONNECTION)) is None
or (host := outgoing.get(CONF_HOST)) is None
or host.version != 6
):
return config
network_conf = fv.full_config.get().get("network") or {}
if not network_conf.get(CONF_ENABLE_IPV6):
raise cv.Invalid(
"outgoing_connection host is an IPv6 address but IPv6 is not "
"enabled; set 'network: enable_ipv6: true'",
path=[CONF_OUTGOING_CONNECTION, CONF_HOST],
)
return config
def _validate_wizard(config: ConfigType) -> ConfigType:
wizard.final_validate(config)
return config
FINAL_VALIDATE_SCHEMA = cv.All(
_validate_esp8266_action_strings,
_validate_outgoing_host_ipv6,
_validate_wizard,
)
def _add_action_strings(
@@ -456,17 +566,24 @@ async def to_code(config: ConfigType) -> None:
var = cg.new_Pvariable(config[CONF_ID])
await cg.register_component(var, config)
# Track controller registration for StaticVector sizing
CORE.register_controller()
CORE.register_controller(var)
# Request a log listener slot for API log streaming
request_log_listener()
cg.add(var.set_port(config[CONF_PORT]))
cg.add(var.set_reboot_timeout(config[CONF_REBOOT_TIMEOUT]))
cg.add(var.set_batch_delay(config[CONF_BATCH_DELAY]))
if CONF_LISTEN_BACKLOG in config:
cg.add(var.set_listen_backlog(config[CONF_LISTEN_BACKLOG]))
# Skip the setters when the config matches the C++ initializers (DEFAULT_*).
if (port := config[CONF_PORT]) != DEFAULT_PORT:
cg.add(var.set_port(port))
if (reboot_timeout := config[CONF_REBOOT_TIMEOUT]) != cv.time_period(
DEFAULT_REBOOT_TIMEOUT
):
cg.add(var.set_reboot_timeout(reboot_timeout))
if (batch_delay := config[CONF_BATCH_DELAY]) != cv.time_period(DEFAULT_BATCH_DELAY):
cg.add(var.set_batch_delay(batch_delay))
if (
listen_backlog := config.get(CONF_LISTEN_BACKLOG)
) is not None and listen_backlog != DEFAULT_LISTEN_BACKLOG:
cg.add(var.set_listen_backlog(listen_backlog))
cg.add_define("MAX_API_CONNECTIONS", config[CONF_MAX_CONNECTIONS])
cg.add_define("API_MAX_SEND_QUEUE", config[CONF_MAX_SEND_QUEUE])
@@ -571,6 +688,9 @@ async def to_code(config: ConfigType) -> None:
# Stack buffer that list-entities copies PROGMEM strings into, sized for the largest action
cg.add_define("API_USER_ACTION_STRINGS_SCRATCH_SIZE", max(scratch_size, 1))
if (wizard_config := config.get(wizard.CONF_WIZARD)) is not None:
await wizard.to_code(wizard_config)
if CONF_ON_CLIENT_CONNECTED in config:
cg.add_define("USE_API_CLIENT_CONNECTED_TRIGGER")
await automation.build_automation(
@@ -589,7 +709,7 @@ async def to_code(config: ConfigType) -> None:
if (encryption_config := config.get(CONF_ENCRYPTION, None)) is not None:
if key := encryption_config.get(CONF_KEY):
cg.add(var.set_noise_psk(new_psk_progmem(config[CONF_ID], key)))
cg.add(var.set_noise_psk(new_psk_progmem(key)))
cg.add_define("USE_API_NOISE_PSK_FROM_YAML")
else:
# No key provided, but encryption desired
@@ -602,9 +722,17 @@ async def to_code(config: ConfigType) -> None:
# and plaintext disabled. Only a factory reset can remove it.
cg.add_define("USE_API_PLAINTEXT")
cg.add_define("USE_API_NOISE")
enable_spare_ephemeral()
else:
cg.add_define("USE_API_PLAINTEXT")
if (outgoing := config.get(CONF_OUTGOING_CONNECTION)) is not None:
cg.add_define("USE_API_OUTGOING_CONNECTION")
if (host := outgoing.get(CONF_HOST)) is not None:
cg.add_define("API_OUTGOING_CONNECTION_HOST", str(host))
cg.add_define("API_OUTGOING_CONNECTION_PORT", outgoing[CONF_PORT])
cg.add_define("API_OUTGOING_CONNECTION_DELAY", outgoing[CONF_DELAY])
cg.add_define("USE_API")
cg.add_global(api_ns.using)
@@ -645,6 +773,11 @@ VARIABLES_SCHEMA = cv.Schema(
{cv.string: cv.All(_coerce_implicit_lambda, cv.templatable(cv.string_strict))}
)
# The action stores each map's entry count in a uint8_t
_FIELD_MAP_MAX = 255
DATA_FIELDS_SCHEMA = cv.All(KEY_VALUE_SCHEMA, cv.Length(max=_FIELD_MAP_MAX))
VARIABLES_FIELDS_SCHEMA = cv.All(VARIABLES_SCHEMA, cv.Length(max=_FIELD_MAP_MAX))
def _validate_response_config(config: ConfigType) -> ConfigType:
# Validate dependencies:
@@ -679,9 +812,9 @@ HOMEASSISTANT_ACTION_ACTION_SCHEMA = cv.All(
cv.Exclusive(CONF_ACTION, group_of_exclusion=CONF_ACTION): cv.templatable(
cv.string
),
cv.Optional(CONF_DATA, default={}): KEY_VALUE_SCHEMA,
cv.Optional(CONF_DATA_TEMPLATE, default={}): KEY_VALUE_SCHEMA,
cv.Optional(CONF_VARIABLES, default={}): VARIABLES_SCHEMA,
cv.Optional(CONF_DATA, default={}): DATA_FIELDS_SCHEMA,
cv.Optional(CONF_DATA_TEMPLATE, default={}): DATA_FIELDS_SCHEMA,
cv.Optional(CONF_VARIABLES, default={}): VARIABLES_FIELDS_SCHEMA,
cv.Optional(CONF_RESPONSE_TEMPLATE): cv.templatable(cv.string),
cv.Optional(CONF_CAPTURE_RESPONSE, default=False): cv.boolean,
cv.Optional(CONF_ON_SUCCESS): automation.validate_automation(single=True),
@@ -694,6 +827,62 @@ HOMEASSISTANT_ACTION_ACTION_SCHEMA = cv.All(
)
def _field_string(value: str) -> Expression:
# ESP8266 can only keep a string in flash as its own PROGMEM array
literal = cg.RawExpression(cpp_string_escape(value))
if CORE.is_esp8266:
return cg.shared_progmem_array("ha_field_str", cg.char, literal)
return literal
async def _new_service_call_action(
server_id: ID,
action_id: ID,
template_arg: cg.TemplateArguments,
args: TemplateArgsType,
is_event: bool,
service: Any,
data: dict[str, Any],
data_template: dict[str, Any],
variables: dict[str, Any],
) -> MockObj:
"""Create the action with its name and fields in one shared flash table."""
cg.add_define("USE_API_HOMEASSISTANT_SERVICES")
serv = await cg.get_variable(server_id)
field_type = HomeAssistantField.template(template_arg)
groups = [data, data_template, variables]
# A lambda may keep static state, so a table with lambdas is never shared
has_lambda = cg.is_template(service) or any(
cg.is_template(value) for group in groups for value in group.values()
)
async def field(key: str | None, value: Any, output_type: Any = None) -> Expression:
key_exp = cg.nullptr if key is None else _field_string(key)
if cg.is_template(value):
# output_type=None lets lambdas return numbers or char pointers; C++ converts them
lam = await cg.process_lambda(value, args, return_type=output_type)
return cg.RawExpression(f"{field_type}::from_lambda({key_exp}, {lam})")
return cg.ArrayInitializer(key_exp, _field_string(value), cg.nullptr)
entries = [await field(None, service, cg.std_string)]
for group in groups:
for key, value in group.items():
entries.append(await field(key, value))
table = cg.shared_progmem_array(
"ha_action_fields",
field_type,
cg.ArrayInitializer(*entries, multiline=True),
share=not has_lambda,
)
return cg.new_Pvariable(
action_id, template_arg, serv, is_event, table, *(len(g) for g in groups)
)
def _service_call_fields(config: ConfigType) -> tuple[dict[str, Any], ...]:
return config[CONF_DATA], config[CONF_DATA_TEMPLATE], config[CONF_VARIABLES]
# synchronous=False: when on_success/on_error is configured, play() stores the
# trigger args until the HomeassistantActionResponse arrives, so non-owning args
# (StringRef into the API receive buffer) must not be used.
@@ -715,36 +904,15 @@ async def homeassistant_service_to_code(
template_arg: cg.TemplateArguments,
args: TemplateArgsType,
) -> MockObj:
cg.add_define("USE_API_HOMEASSISTANT_SERVICES")
serv = await cg.get_variable(config[CONF_ID])
var = cg.new_Pvariable(action_id, template_arg, serv, False)
templ = await cg.templatable(config[CONF_ACTION], args, cg.std_string)
cg.add(var.set_service(templ))
# Initialize FixedVectors with exact sizes from config
cg.add(var.init_data(len(config[CONF_DATA])))
for key, value in config[CONF_DATA].items():
# output_type=None because lambdas can return non-string types (int,
# float, char*) that TemplatableStringValue converts via to_string.
# Static strings are manually wrapped for PROGMEM on ESP8266.
templ = await cg.templatable(value, args, None)
if isinstance(templ, str):
templ = cg.FlashStringLiteral(templ)
cg.add(var.add_data(cg.FlashStringLiteral(key), templ))
cg.add(var.init_data_template(len(config[CONF_DATA_TEMPLATE])))
for key, value in config[CONF_DATA_TEMPLATE].items():
templ = await cg.templatable(value, args, None)
if isinstance(templ, str):
templ = cg.FlashStringLiteral(templ)
cg.add(var.add_data_template(cg.FlashStringLiteral(key), templ))
cg.add(var.init_variables(len(config[CONF_VARIABLES])))
for key, value in config[CONF_VARIABLES].items():
templ = await cg.templatable(value, args, None)
if isinstance(templ, str):
templ = cg.FlashStringLiteral(templ)
cg.add(var.add_variable(cg.FlashStringLiteral(key), templ))
var = await _new_service_call_action(
config[CONF_ID],
action_id,
template_arg,
args,
False,
config[CONF_ACTION],
*_service_call_fields(config),
)
if on_error := config.get(CONF_ON_ERROR):
cg.add_define("USE_API_HOMEASSISTANT_ACTION_RESPONSES")
@@ -796,9 +964,9 @@ HOMEASSISTANT_EVENT_ACTION_SCHEMA = cv.Schema(
{
cv.GenerateID(): cv.use_id(APIServer),
cv.Required(CONF_EVENT): validate_homeassistant_event,
cv.Optional(CONF_DATA, default={}): KEY_VALUE_SCHEMA,
cv.Optional(CONF_DATA_TEMPLATE, default={}): KEY_VALUE_SCHEMA,
cv.Optional(CONF_VARIABLES, default={}): VARIABLES_SCHEMA,
cv.Optional(CONF_DATA, default={}): DATA_FIELDS_SCHEMA,
cv.Optional(CONF_DATA_TEMPLATE, default={}): DATA_FIELDS_SCHEMA,
cv.Optional(CONF_VARIABLES, default={}): VARIABLES_FIELDS_SCHEMA,
}
)
@@ -817,38 +985,15 @@ async def homeassistant_event_to_code(
template_arg: cg.TemplateArguments,
args: TemplateArgsType,
) -> MockObj:
cg.add_define("USE_API_HOMEASSISTANT_SERVICES")
serv = await cg.get_variable(config[CONF_ID])
var = cg.new_Pvariable(action_id, template_arg, serv, True)
templ = await cg.templatable(config[CONF_EVENT], args, cg.std_string)
cg.add(var.set_service(templ))
# Initialize FixedVectors with exact sizes from config
cg.add(var.init_data(len(config[CONF_DATA])))
for key, value in config[CONF_DATA].items():
# output_type=None because lambdas can return non-string types (int,
# float, char*) that TemplatableStringValue converts via to_string.
# Static strings are manually wrapped for PROGMEM on ESP8266.
templ = await cg.templatable(value, args, None)
if isinstance(templ, str):
templ = cg.FlashStringLiteral(templ)
cg.add(var.add_data(cg.FlashStringLiteral(key), templ))
cg.add(var.init_data_template(len(config[CONF_DATA_TEMPLATE])))
for key, value in config[CONF_DATA_TEMPLATE].items():
templ = await cg.templatable(value, args, None)
if isinstance(templ, str):
templ = cg.FlashStringLiteral(templ)
cg.add(var.add_data_template(cg.FlashStringLiteral(key), templ))
cg.add(var.init_variables(len(config[CONF_VARIABLES])))
for key, value in config[CONF_VARIABLES].items():
templ = await cg.templatable(value, args, None)
if isinstance(templ, str):
templ = cg.FlashStringLiteral(templ)
cg.add(var.add_variable(cg.FlashStringLiteral(key), templ))
return var
return await _new_service_call_action(
config[CONF_ID],
action_id,
template_arg,
args,
True,
config[CONF_EVENT],
*_service_call_fields(config),
)
HOMEASSISTANT_TAG_SCANNED_ACTION_SCHEMA = cv.maybe_simple_value(
@@ -872,15 +1017,17 @@ async def homeassistant_tag_scanned_to_code(
template_arg: cg.TemplateArguments,
args: TemplateArgsType,
) -> MockObj:
cg.add_define("USE_API_HOMEASSISTANT_SERVICES")
serv = await cg.get_variable(config[CONF_ID])
var = cg.new_Pvariable(action_id, template_arg, serv, True)
cg.add(var.set_service(cg.FlashStringLiteral("esphome.tag_scanned")))
# Initialize FixedVector with exact size (1 data field)
cg.add(var.init_data(1))
templ = await cg.templatable(config[CONF_TAG], args, cg.std_string)
cg.add(var.add_data(cg.FlashStringLiteral("tag_id"), templ))
return var
return await _new_service_call_action(
config[CONF_ID],
action_id,
template_arg,
args,
True,
"esphome.tag_scanned",
{"tag_id": config[CONF_TAG]},
{},
{},
)
CONF_SUCCESS = "success"
@@ -991,6 +1138,8 @@ _define_filter = filter_source_files_from_defines(
"user_services.cpp": "USE_API_USER_DEFINED_ACTIONS",
"api_frame_helper_noise.cpp": "USE_API_NOISE",
"api_frame_helper_plaintext.cpp": "USE_API_PLAINTEXT",
"api_wizard.cpp": "USE_API_WIZARD",
"api_outgoing_connection.cpp": "USE_API_OUTGOING_CONNECTION",
}
)
esphome/components/api/api.proto
+219 -7
View File
@@ -20,6 +20,8 @@ service APIConnection {
option (needs_authentication) = false;
}
rpc device_capabilities (DeviceCapabilitiesRequest) returns (DeviceCapabilitiesResponse) {}
rpc device_wizard (DeviceWizardRequest) returns (DeviceWizardResponse) {}
rpc wizard_input_set (WizardInputSetRequest) returns (void) {}
rpc list_entities (ListEntitiesRequest) returns (void) {}
rpc subscribe_states (SubscribeStatesRequest) returns (void) {}
rpc subscribe_logs (SubscribeLogsRequest) returns (void) {}
@@ -76,7 +78,9 @@ service APIConnection {
rpc serial_proxy_write(SerialProxyWriteRequest) returns (void) {}
rpc serial_proxy_set_modem_pins(SerialProxySetModemPinsRequest) returns (void) {}
rpc serial_proxy_get_modem_pins(SerialProxyGetModemPinsRequest) returns (void) {}
rpc subscribe_serial_proxy_identity(SubscribeSerialProxyIdentityRequest) returns (void) {}
rpc serial_proxy_request(SerialProxyRequest) returns (void) {}
rpc serial_proxy_set_mode(SerialProxySetModeRequest) returns (void) {}
}
@@ -112,6 +116,11 @@ message HelloRequest {
string client_info = 1;
uint32 api_version_major = 2;
uint32 api_version_minor = 3;
// Set by clients that can accept connections the device opens to them
// (see api: outgoing_connection:). The device remembers this client's
// address as the target to dial when no such client is connected.
bool outgoing_connection_target = 4 [(field_ifdef) = "USE_API_OUTGOING_CONNECTION"];
}
// Confirmation of successful connection request.
@@ -227,6 +236,7 @@ enum SerialProxyPortType {
SERIAL_PROXY_PORT_TYPE_TTL = 0;
SERIAL_PROXY_PORT_TYPE_RS232 = 1;
SERIAL_PROXY_PORT_TYPE_RS485 = 2;
SERIAL_PROXY_PORT_TYPE_USB_SERIAL = 3; // since API 1.18
}
message SerialProxyInfo {
@@ -331,6 +341,10 @@ message DeviceInfoResponse {
// all-zeros PSK, so the api encryption key can be provisioned without being
// sent in plaintext (protects against passive sniffing, not active MITM)
bool api_encryption_provisionable = 26 [(field_ifdef) = "USE_API_NOISE"];
// Device is built with the api outgoing_connection option and can open
// the TCP connection to a dial-back target itself
bool api_outgoing_connection_supported = 27 [(field_ifdef) = "USE_API_OUTGOING_CONNECTION"];
}
// ==================== DEVICE CAPABILITIES ====================
@@ -379,6 +393,11 @@ message ZWaveProxyCapabilities {
uint32 home_id = 2;
}
message WizardCapabilities {
// True when the device has a wizard configured, so DeviceWizardRequest will be answered
bool configured = 1;
}
message DeviceCapabilitiesResponse {
option (id) = 150;
option (source) = SOURCE_SERVER;
@@ -388,6 +407,90 @@ message DeviceCapabilitiesResponse {
ZWaveProxyCapabilities zwave_proxy = 3 [(field_ifdef) = "USE_ZWAVE_PROXY"];
repeated SerialProxyInfo serial_proxies = 4
[(field_ifdef) = "USE_SERIAL_PROXY", (fixed_array_size_define) = "SERIAL_PROXY_COUNT"];
WizardCapabilities wizard = 5 [(field_ifdef) = "USE_API_WIZARD"];
}
// ==================== DEVICE WIZARD ====================
// Asks the device for the onboarding wizard that Home Assistant presents when the
// device is added.
//
// The wizard is read once, when the device is added, whereas capabilities are read
// on every connect, so it is a separate message rather than part of
// DeviceCapabilitiesResponse. It is only served on an authenticated connection.
//
// A device without a wizard does not have this message at all and never answers it,
// as a device ignores messages it does not know. Clients therefore check
// DeviceCapabilitiesResponse.wizard.configured before sending it.
message DeviceWizardRequest {
option (id) = 156;
option (source) = SOURCE_CLIENT;
option (ifdef) = "USE_API_WIZARD";
// Empty
}
// The wizard, as UTF-8 JSON compressed into a single zstd frame. The device builds it
// when it is compiled and sends it unchanged, so its size varies with the wizard and
// its format can grow without new fields here. Clients decompress it, then read:
//
// {"version": 1,
// "pages": [{"title": "...", "description": "...",
// "entities": [{"key": 123, "device_id": 456, "description": "..."}],
// "inputs": [{"key": 789, "description": "...",
// "entity_filters": [{"integration": "...", "domain": ["..."],
// "device_class": ["..."], "supported_features": ["..."]}]}]}]}
//
// Values that are empty or unset are left out, and so are empty lists. Strings may be
// Home Assistant translation placeholders such as "[%key:component::domain::name%]",
// passed through verbatim. The pages are shown in the order listed. A client that does
// not know the version must not use the wizard.
//
// An entity is an entity of this device that the page shows to the user (for example to
// turn it on or off). Its key is the key ListEntitiesResponse sends for it. Its
// device_id is the device_id ListEntitiesResponse sends, and is left out when that is 0.
//
// An input is filled in by the user with the id of a Home Assistant entity. It either
// stands for one entity of the device's homeassistant platform (sensor, binary_sensor,
// text_sensor, number, switch, text, select or button) that sets no entity_id of its own,
// as Home Assistant supplies it, or it is a standalone input that only stores the entity
// id for the device's own automations. Its key is the FNV-1 hash
// of the input's ESPHome id, not an entity key. The client sends the chosen entity id to
// the device with a WizardInputSetRequest. entity_filters says which Home Assistant
// entities the user may choose, and mirrors Home Assistant's EntityFilterSelectorConfig:
// an entity matches a filter when it satisfies every field that is set, and an entity
// that matches any one of the filters is accepted.
//
// The device keeps the entity id of an input in RAM only and does not store it across
// restarts. The client therefore stores the choice itself and, after every connection
// and before it sends SubscribeHomeAssistantStatesRequest, sends a WizardInputSetRequest
// for each input. It sends one again whenever the user changes the choice.
message DeviceWizardResponse {
option (id) = 157;
option (source) = SOURCE_SERVER;
option (ifdef) = "USE_API_WIZARD";
// Not logged: the data is in flash, which ESP8266 cannot read for a dump
option (log) = false;
// The data is in flash, and the device sends it from there
bytes data = 1 [(pointer_to_buffer) = true];
}
// Sets the entity id of a wizard input (see WizardInputField.key). Clients send
// this only when DeviceCapabilitiesResponse.wizard.configured is set. There is no
// reply. The device ignores a request whose key matches no input or whose entity_id
// is empty, longer than 255 bytes or has no '.'. Otherwise it keeps the id in RAM,
// not across restarts, and when the input belongs to a homeassistant platform entity
// it subscribes to that entity's state. Clients send this for every input right
// after connecting, before SubscribeHomeAssistantStatesRequest. A request that
// arrives later still works: the device then sends the affected state subscriptions
// again.
message WizardInputSetRequest {
option (id) = 158;
option (source) = SOURCE_CLIENT;
option (ifdef) = "USE_API_WIZARD_INPUTS";
fixed32 key = 1;
string entity_id = 2 [(max_data_length) = 255];
}
message ListEntitiesRequest {
@@ -802,6 +905,9 @@ message SwitchStateResponse {
fixed32 key = 1 [(force) = true];
bool state = 2;
uint32 device_id = 3 [(field_ifdef) = "USE_DEVICES"];
// If the switch does not have a valid state yet.
// Equivalent to `!obj->has_state()` - inverse logic to make state packets smaller
bool missing_state = 4;
}
message SwitchCommandRequest {
option (id) = 33;
@@ -1243,6 +1349,9 @@ message ClimateStateResponse {
float current_humidity = 14;
float target_humidity = 15;
uint32 device_id = 16 [(field_ifdef) = "USE_DEVICES"];
// If the climate device does not have a valid state yet.
// Equivalent to `!obj->has_state()` - inverse logic to make state packets smaller
bool missing_state = 17;
}
message ClimateCommandRequest {
option (id) = 48;
@@ -1329,6 +1438,9 @@ message WaterHeaterStateResponse {
uint32 state = 6;
float target_temperature_low = 7;
float target_temperature_high = 8;
// If the water heater does not have a valid state yet.
// Equivalent to `!obj->has_state()` - inverse logic to make state packets smaller
bool missing_state = 9;
}
// Bitmask for WaterHeaterCommandRequest.has_fields
@@ -1429,7 +1541,7 @@ message ListEntitiesSelectResponse {
reserved 4; // Deprecated: was string unique_id
string icon = 5 [(field_ifdef) = "USE_ENTITY_ICON", (max_data_length) = 63];
repeated string options = 6 [(container_pointer_no_template) = "FixedVector<const char *>"];
repeated string options = 6 [(container_pointer_no_template) = "std::span<const char *const>"];
bool disabled_by_default = 7;
EntityCategory entity_category = 8;
uint32 device_id = 9 [(field_ifdef) = "USE_DEVICES"];
@@ -2673,7 +2785,7 @@ message ListEntitiesInfraredResponse {
message InfraredRFTransmitRawTimingsRequest {
option (id) = 136;
option (source) = SOURCE_CLIENT;
option (ifdef) = "USE_IR_RF || USE_RADIO_FREQUENCY";
option (ifdef) = "USE_IR_RF";
uint32 device_id = 1 [(field_ifdef) = "USE_DEVICES"];
fixed32 key = 2 [(force) = true]; // Key identifying the transmitter instance
@@ -2687,7 +2799,7 @@ message InfraredRFTransmitRawTimingsRequest {
message InfraredRFReceiveEvent {
option (id) = 137;
option (source) = SOURCE_SERVER;
option (ifdef) = "USE_IR_RF || USE_RADIO_FREQUENCY";
option (ifdef) = "USE_IR_RF";
option (no_delay) = true;
option (speed_optimized) = true;
@@ -2696,6 +2808,28 @@ message InfraredRFReceiveEvent {
repeated sint32 timings = 3 [packed = true, (container_pointer_no_template) = "std::vector<int32_t>"]; // Raw timings in microseconds (zigzag-encoded): alternating mark/space periods
}
// Sent only to the client that issued an InfraredRFTransmitRawTimingsRequest, once the
// transmitter reports that the transmission (all repeats) has finished, or immediately with
// success=false if it was refused before reaching the transmitter (unknown key, no transmitter,
// no or invalid timings) or the transmitter could not send it (not set up, hardware error).
// success=false also answers a request superseded by a newer request on the same entity, from
// any client, and a transmit that reported nothing within 30 s past its expected air time, so a
// client should not retry on it blindly. The device serializes transmits per transmitter and
// several entities may share one, so a client should keep at most one transmit outstanding per
// device, not per entity. A reply for an unknown key or a refused request is sent once and can be lost
// when the device's send buffer is full, so a client should also stop waiting on its own after the
// expected air time plus a margin. Lets clients pace requests instead of estimating durations (since API 1.18)
message InfraredRFTransmitCompleteResponse {
option (id) = 153;
option (source) = SOURCE_SERVER;
option (ifdef) = "USE_IR_RF";
option (no_delay) = true;
uint32 device_id = 1 [(field_ifdef) = "USE_DEVICES"];
fixed32 key = 2 [(force) = true]; // Key of the transmitter entity from the request
bool success = 3; // false if the request was refused, not sent, superseded, or never reported
}
// ==================== RADIO FREQUENCY ====================
// Lists available radio frequency entity instances
@@ -2726,7 +2860,8 @@ enum SerialProxyParity {
SERIAL_PROXY_PARITY_ODD = 2;
}
// Configure UART parameters for a serial proxy instance
// Configure UART parameters for a serial proxy instance. Only the subscribed client may
// configure the port; others are refused with PORT_IN_USE (since API 1.17).
message SerialProxyConfigureRequest {
option (id) = 138;
option (source) = SOURCE_CLIENT;
@@ -2752,7 +2887,8 @@ message SerialProxyDataReceived {
bytes data = 2; // Raw data received from the serial device
}
// Write data to a serial device
// Write data to a serial device. Only the subscribed client may write; writes from
// others are ignored (since API 1.17).
message SerialProxyWriteRequest {
option (id) = 140;
option (source) = SOURCE_CLIENT;
@@ -2763,7 +2899,8 @@ message SerialProxyWriteRequest {
bytes data = 2; // Raw data to write to the serial device
}
// Set modem control pin states (RTS and DTR)
// Set modem control pin states (RTS and DTR). Only the subscribed client may set them;
// others are refused with PORT_IN_USE (since API 1.17).
message SerialProxySetModemPinsRequest {
option (id) = 141;
option (source) = SOURCE_CLIENT;
@@ -2802,6 +2939,7 @@ enum SerialProxyRequestType {
// error the device answers with INVALID_ARGUMENT.
SERIAL_PROXY_REQUEST_TYPE_CONFIGURE = 3; // Acknowledges a SerialProxyConfigureRequest
SERIAL_PROXY_REQUEST_TYPE_SET_MODEM_PINS = 4; // Acknowledges a SerialProxySetModemPinsRequest
SERIAL_PROXY_REQUEST_TYPE_SET_MODE = 5; // Acknowledges a SerialProxySetModeRequest (since API 1.17)
}
enum SerialProxyStatus {
@@ -2814,7 +2952,8 @@ enum SerialProxyStatus {
SERIAL_PROXY_STATUS_INVALID_ARGUMENT = 6; // Invalid instance index or parameter value
}
// Generic request message for simple serial proxy operations
// Generic request message for simple serial proxy operations. FLUSH requires an active
// subscription; it is refused with PORT_IN_USE otherwise (since API 1.17).
message SerialProxyRequest {
option (id) = 144;
option (source) = SOURCE_CLIENT;
@@ -2838,6 +2977,79 @@ message SerialProxyRequestResponse {
string error_message = 4; // Additional detail on failure (optional)
}
// How a port treats the bytes passing through it. RAW is a plain byte pipe; PROTOCOL
// activates the port's protocol-aware tap (if one is configured), letting it observe
// traffic and inject protocol bytes such as acknowledgements. Which protocol the tap
// speaks is a property of the device configuration, discoverable from the tap
// component's own API surface. A client that is about to flash firmware selects RAW
// first, which definitively disables that injection.
enum SerialProxyMode {
SERIAL_PROXY_MODE_RAW = 0;
SERIAL_PROXY_MODE_PROTOCOL = 1;
}
// Only the subscribed client may change the mode; any other caller -- including one that
// never subscribed -- is refused with PORT_IN_USE. PROTOCOL is refused with NOT_SUPPORTED
// when the port has no protocol-aware tap configured.
message SerialProxySetModeRequest {
option (id) = 152;
option (source) = SOURCE_CLIENT;
option (ifdef) = "USE_SERIAL_PROXY";
uint32 instance = 1;
SerialProxyMode mode = 2;
}
// Subscribe to the identity of every serial proxy port. The device answers with one
// SerialProxyIdentity per port, then sends another whenever a port's identity changes,
// for the life of the connection (since API 1.18).
message SubscribeSerialProxyIdentityRequest {
option (id) = 154;
option (source) = SOURCE_CLIENT;
option (ifdef) = "USE_SERIAL_PROXY";
}
// Where a port's identity comes from
enum SerialProxyIdentitySource {
SERIAL_PROXY_IDENTITY_SOURCE_NONE = 0; // The port carries no identity
SERIAL_PROXY_IDENTITY_SOURCE_CONFIGURED = 1; // Reserved for identities stated in the device configuration; not sent yet
SERIAL_PROXY_IDENTITY_SOURCE_USB = 2; // Read from the descriptors of the USB device behind the port;
// changes when a device is attached or removed
}
enum SerialProxyIdentityFlag {
SERIAL_PROXY_IDENTITY_FLAG_NONE = 0;
SERIAL_PROXY_IDENTITY_FLAG_CONNECTED = 1; // The backend believes the device is present and usable on this port
SERIAL_PROXY_IDENTITY_FLAG_ERROR = 2; // The USB host stack refused the descriptor query; the strings and
// IDs below are empty
}
// The descriptor fields of a USB device as seen by the host stack
message UsbDeviceDescriptor {
uint32 vendor_id = 1;
uint32 product_id = 2;
uint32 bcd_device = 3;
uint32 interface_number = 4; // bInterfaceNumber the host driver binds to
}
// The identity of the device behind a port. Identifies one physical device among others of
// the same kind, so a client matches on manufacturer and product regardless of source. The
// strings are empty for source NONE, and for source USB while nothing is connected
// (since API 1.18).
message SerialProxyIdentity {
option (id) = 155;
option (source) = SOURCE_SERVER;
option (ifdef) = "USE_SERIAL_PROXY";
uint32 instance = 1;
SerialProxyIdentitySource source = 2;
uint32 flags = 3; // Bitmask of SerialProxyIdentityFlag
string manufacturer = 4;
string product = 5;
string serial_number = 6;
UsbDeviceDescriptor usb = 7; // Only sent for source USB
}
// ==================== BLUETOOTH CONNECTION PARAMS ====================
message BluetoothSetConnectionParamsRequest {
option (id) = 145;
esphome/components/api/api_connection.cpp
+120 -14
View File
@@ -198,6 +198,17 @@ APIConnection::~APIConnection() {
proxy->serial_proxy_request(this, enums::SERIAL_PROXY_REQUEST_TYPE_UNSUBSCRIBE);
}
}
#endif
// entities holding a transmit reply for this client must not answer into a freed connection
#ifdef USE_INFRARED
for (auto *infrared : App.get_infrareds()) {
infrared->on_api_connection_closed(this);
}
#endif
#ifdef USE_RADIO_FREQUENCY
for (auto *radio_frequency : App.get_radio_frequencies()) {
radio_frequency->on_api_connection_closed(this);
}
#endif
}
@@ -600,7 +611,7 @@ bool APIConnection::send_light_state(light::LightState *light) {
uint16_t APIConnection::try_send_light_state(EntityBase *entity, APIConnection *conn, uint32_t remaining_size) {
auto *light = static_cast<light::LightState *>(entity);
LightStateResponse resp;
auto values = light->remote_values;
auto values = light->get_reported_values();
auto color_mode = values.get_color_mode();
resp.state = values.is_on();
resp.color_mode = static_cast<enums::ColorMode>(color_mode);
@@ -709,6 +720,7 @@ uint16_t APIConnection::try_send_switch_state(EntityBase *entity, APIConnection
auto *a_switch = static_cast<switch_::Switch *>(entity);
SwitchStateResponse resp;
resp.state = a_switch->state;
resp.missing_state = !a_switch->has_state();
return fill_and_encode_entity_state(a_switch, resp, conn, remaining_size);
}
@@ -720,12 +732,7 @@ uint16_t APIConnection::try_send_switch_info(EntityBase *entity, APIConnection *
}
void APIConnection::on_switch_command_request(const SwitchCommandRequest &msg) {
ENTITY_COMMAND_GET(switch_::Switch, a_switch, switch)
if (msg.state) {
a_switch->turn_on();
} else {
a_switch->turn_off();
}
a_switch->control(msg.state);
}
#endif
@@ -759,6 +766,7 @@ uint16_t APIConnection::try_send_climate_state(EntityBase *entity, APIConnection
auto traits = climate->get_traits();
resp.mode = static_cast<enums::ClimateMode>(climate->mode);
resp.action = static_cast<enums::ClimateAction>(climate->action);
resp.missing_state = !climate->has_state();
if (traits.has_feature_flags(climate::CLIMATE_SUPPORTS_CURRENT_TEMPERATURE))
resp.current_temperature = climate->current_temperature;
if (traits.has_feature_flags(climate::CLIMATE_SUPPORTS_TWO_POINT_TARGET_TEMPERATURE |
@@ -993,7 +1001,9 @@ uint16_t APIConnection::try_send_select_state(EntityBase *entity, APIConnection
uint16_t APIConnection::try_send_select_info(EntityBase *entity, APIConnection *conn, uint32_t remaining_size) {
auto *select = static_cast<select::Select *>(entity);
ListEntitiesSelectResponse msg;
msg.options = &select->traits.get_options();
const auto &opts = select->traits.get_options();
const std::span<const char *const> options(opts.data(), opts.size());
msg.options = &options;
return fill_and_encode_entity_info(select, msg, conn, remaining_size);
}
void APIConnection::on_select_command_request(const SelectCommandRequest &msg) {
@@ -1452,6 +1462,7 @@ uint16_t APIConnection::try_send_water_heater_state(EntityBase *entity, APIConne
auto *wh = static_cast<water_heater::WaterHeater *>(entity);
WaterHeaterStateResponse resp;
resp.mode = static_cast<enums::WaterHeaterMode>(wh->get_mode());
resp.missing_state = !wh->has_state();
resp.current_temperature = wh->get_current_temperature();
resp.target_temperature = wh->get_target_temperature();
resp.target_temperature_low = wh->get_target_temperature_low();
@@ -1517,8 +1528,10 @@ uint16_t APIConnection::try_send_event_info(EntityBase *entity, APIConnection *c
}
#endif
#if defined(USE_IR_RF) || defined(USE_RADIO_FREQUENCY)
#ifdef USE_IR_RF
void APIConnection::on_infrared_rf_transmit_raw_timings_request(const InfraredRFTransmitRawTimingsRequest &msg) {
// Clients on API 1.18+ are told when the frame has left the transmitter; the entity owns that reply
const bool want_reply = this->client_supports_api_version(1, 18);
// Dispatch by key: infrared entities are checked first, then radio frequency entities.
// The key is unique across all entity instances on a device, so at most one lookup will succeed.
#ifdef USE_INFRARED
@@ -1528,6 +1541,7 @@ void APIConnection::on_infrared_rf_transmit_raw_timings_request(const InfraredRF
call.set_carrier_frequency(msg.carrier_frequency);
call.set_raw_timings_packed(msg.timings_data_, msg.timings_length_, msg.timings_count_);
call.set_repeat_count(msg.repeat_count);
call.set_api_connection(want_reply ? this : nullptr);
call.perform();
return;
}
@@ -1540,13 +1554,38 @@ void APIConnection::on_infrared_rf_transmit_raw_timings_request(const InfraredRF
call.set_modulation(static_cast<radio_frequency::RadioFrequencyModulation>(msg.modulation));
call.set_repeat_count(msg.repeat_count);
call.set_raw_timings_packed(msg.timings_data_, msg.timings_length_, msg.timings_count_);
call.set_api_connection(want_reply ? this : nullptr);
call.perform();
return;
}
#endif
ESP_LOGW(TAG, "IR/RF transmit for unknown key %" PRIu32, msg.key);
if (want_reply) {
// nothing will ever report for an unknown key, so answer as not started right away
#ifdef USE_DEVICES
const uint32_t device_id = msg.device_id;
#else
const uint32_t device_id = 0;
#endif
if (!this->send_infrared_rf_transmit_complete(device_id, msg.key, false)) {
API_LOG_MSG_DROPPED(TAG, "IR/RF reply");
}
}
}
bool APIConnection::send_infrared_rf_transmit_complete([[maybe_unused]] uint32_t device_id, uint32_t key,
bool success) {
InfraredRFTransmitCompleteResponse resp{};
#ifdef USE_DEVICES
resp.device_id = device_id;
#endif
resp.key = key;
resp.success = success;
return this->send_message(resp);
}
#endif
#if defined(USE_IR_RF) || defined(USE_RADIO_FREQUENCY)
#ifdef USE_IR_RF
void APIConnection::send_infrared_rf_receive_event(const InfraredRFReceiveEvent &msg) {
if (!this->send_message(msg)) {
// V: fires per decoded frame with no subscription gate, so a warning
@@ -1554,6 +1593,7 @@ void APIConnection::send_infrared_rf_receive_event(const InfraredRFReceiveEvent
ESP_LOGV(TAG, "IR/RF event dropped, TCP buffer full");
}
}
#endif
#ifdef USE_SERIAL_PROXY
@@ -1645,6 +1685,22 @@ void APIConnection::on_serial_proxy_get_modem_pins_request(const SerialProxyGetM
}
}
void APIConnection::on_subscribe_serial_proxy_identity_request() {
#ifdef USE_SERIAL_PROXY_USB_IDENTITY
// Only USB ports change identity after this snapshot
this->flags_.serial_proxy_identity_subscription = true;
#endif
for (auto *proxy : App.get_serial_proxies()) {
proxy->send_identity(this);
}
}
void APIConnection::send_serial_proxy_identity(const SerialProxyIdentity &msg) {
if (!this->send_message(msg)) {
API_LOG_MSG_DROPPED(TAG, "Serial proxy identity");
}
}
void APIConnection::on_serial_proxy_request(const SerialProxyRequest &msg) {
auto &proxies = App.get_serial_proxies();
if (msg.instance >= proxies.size()) {
@@ -1664,6 +1720,7 @@ void APIConnection::on_serial_proxy_request(const SerialProxyRequest &msg) {
break;
case enums::SERIAL_PROXY_REQUEST_TYPE_CONFIGURE:
case enums::SERIAL_PROXY_REQUEST_TYPE_SET_MODEM_PINS:
case enums::SERIAL_PROXY_REQUEST_TYPE_SET_MODE:
// Response-only discriminators; never valid in a request
ESP_LOGW(TAG, "Response-only serial proxy request type: %" PRIu32, static_cast<uint32_t>(msg.type));
status = enums::SERIAL_PROXY_STATUS_INVALID_ARGUMENT;
@@ -1676,6 +1733,19 @@ void APIConnection::on_serial_proxy_request(const SerialProxyRequest &msg) {
send_serial_proxy_ack(this, msg.instance, msg.type, status);
}
void APIConnection::on_serial_proxy_set_mode_request(const SerialProxySetModeRequest &msg) {
auto &proxies = App.get_serial_proxies();
if (msg.instance >= proxies.size()) {
ESP_LOGW(TAG, "Serial proxy instance %" PRIu32 " out of range", msg.instance);
send_serial_proxy_ack(this, msg.instance, enums::SERIAL_PROXY_REQUEST_TYPE_SET_MODE,
enums::SERIAL_PROXY_STATUS_INVALID_ARGUMENT);
return;
}
serial_proxy::SerialProxyResult result = proxies[msg.instance]->set_mode_from_client(this, msg.mode);
send_serial_proxy_ack(this, msg.instance, enums::SERIAL_PROXY_REQUEST_TYPE_SET_MODE,
serial_proxy_result_to_status(result));
}
void APIConnection::send_serial_proxy_data(const SerialProxyDataReceived &msg) {
if (!this->send_message(msg)) {
ESP_LOGV(TAG, "Serial proxy data dropped, TCP buffer full");
@@ -1802,7 +1872,7 @@ bool APIConnection::send_hello_response_(const HelloRequest &msg) {
HelloResponse resp;
resp.api_version_major = 1;
resp.api_version_minor = 16;
resp.api_version_minor = 18;
// Send only the version string - the client only logs this for debugging and doesn't use it otherwise
resp.server_info = ESPHOME_VERSION_REF;
resp.name = StringRef(App.get_name());
@@ -1825,6 +1895,19 @@ bool APIConnection::send_hello_response_(const HelloRequest &msg) {
// Auto-authenticate - password auth was removed in ESPHome 2026.1.0
this->complete_authentication_();
#ifdef USE_API_OUTGOING_CONNECTION
// With a PSK set only key-verified transports reach hello: plaintext and
// zero-PSK are rejected, and pre-activation sessions are force-closed
if (msg.outgoing_connection_target && !this->flags_.outgoing_connection_target) {
if (this->parent_->get_noise_ctx().has_psk()) {
this->flags_.outgoing_connection_target = true;
this->parent_->on_outgoing_target_client(this);
} else {
this->log_client_(ESPHOME_LOG_LEVEL_WARN, LOG_STR("Dial-back target refused; no key active"));
}
}
#endif
return this->send_message(resp);
}
@@ -1947,6 +2030,9 @@ bool APIConnection::send_device_info_response_() {
// one) so this advertisement survives the plaintext removal in 2027.2.0.
resp.api_encryption_provisionable = !this->parent_->get_noise_ctx().has_psk();
#endif
#ifdef USE_API_OUTGOING_CONNECTION
resp.api_outgoing_connection_supported = true;
#endif
#endif
#ifdef USE_DEVICES
size_t device_index = 0;
@@ -1999,6 +2085,9 @@ bool APIConnection::send_device_capabilities_response_() {
info.port_type = proxy->get_port_type();
info.configured_line_states = proxy->get_configured_modem_pins();
}
#endif
#ifdef USE_API_WIZARD
resp.wizard.configured = true;
#endif
return this->send_message(resp);
}
@@ -2213,7 +2302,13 @@ void APIConnection::on_noise_encryption_set_key_request(const NoiseEncryptionSet
}
#endif
#ifdef USE_API_HOMEASSISTANT_STATES
void APIConnection::on_subscribe_home_assistant_states_request() { state_subs_at_ = 0; }
void APIConnection::on_subscribe_home_assistant_states_request() {
#ifdef USE_API_WIZARD_LINKED_INPUTS
// Remember it, as a client that subscribed also gets the subscriptions again when a wizard input is set
this->flags_.home_assistant_states = true;
#endif
state_subs_at_ = 0;
}
#endif
bool APIConnection::try_to_clear_buffer_slow_(bool log_out_of_space) {
delay(0);
@@ -2257,8 +2352,12 @@ bool APIConnection::send_message_(uint32_t payload_size, uint16_t message_type,
#endif
// Capacity reserved above, cannot fail
(void) shared_buf.resize(write_start + payload_size);
ProtoWriteBuffer buffer{&shared_buf, write_start};
encode_fn(msg, buffer PROTO_ENCODE_DEBUG_INIT(&shared_buf));
uint8_t *end = encode_fn(msg, shared_buf.data() + write_start PROTO_ENCODE_DEBUG_INIT(&shared_buf));
#ifdef ESPHOME_DEBUG_API
proto_check_encode_end(end, shared_buf.data() + shared_buf.size());
#else
(void) end;
#endif
return this->send_buffer(ProtoWriteBuffer{&shared_buf}, message_type);
}
// encode_to_buffer is defined inline in api_connection.h (ESPHOME_ALWAYS_INLINE)
@@ -2651,6 +2750,13 @@ void APIConnection::process_state_subscriptions_() {
}
const auto &it = subs[this->state_subs_at_];
#ifdef USE_API_WIZARD_LINKED_INPUTS
// An entity id that is not set yet (a wizard input) has nothing to subscribe to; it is sent once it is set
if (it.entity_id[0] == '\0') {
this->state_subs_at_++;
return;
}
#endif
SubscribeHomeAssistantStateResponse resp;
resp.entity_id = StringRef(it.entity_id);
esphome/components/api/api_connection.h
+62 -28
View File
@@ -233,9 +233,12 @@ class APIConnection final : public APIServerConnectionBase {
void on_water_heater_command_request(const WaterHeaterCommandRequest &msg);
#endif
#if defined(USE_IR_RF) || defined(USE_RADIO_FREQUENCY)
#ifdef USE_IR_RF
void on_infrared_rf_transmit_raw_timings_request(const InfraredRFTransmitRawTimingsRequest &msg);
void send_infrared_rf_receive_event(const InfraredRFReceiveEvent &msg);
// Reply to an InfraredRFTransmitRawTimingsRequest (API 1.18+); false when the TCP buffer is
// full, the entity that owns the reply retries it then
[[nodiscard]] bool send_infrared_rf_transmit_complete(uint32_t device_id, uint32_t key, bool success);
#endif
#ifdef USE_SERIAL_PROXY
@@ -243,7 +246,11 @@ class APIConnection final : public APIServerConnectionBase {
void on_serial_proxy_write_request(const SerialProxyWriteRequest &msg);
void on_serial_proxy_set_modem_pins_request(const SerialProxySetModemPinsRequest &msg);
void on_serial_proxy_get_modem_pins_request(const SerialProxyGetModemPinsRequest &msg);
void on_subscribe_serial_proxy_identity_request();
/// Send a port identity to this client
void send_serial_proxy_identity(const SerialProxyIdentity &msg);
void on_serial_proxy_request(const SerialProxyRequest &msg);
void on_serial_proxy_set_mode_request(const SerialProxySetModeRequest &msg);
void send_serial_proxy_data(const SerialProxyDataReceived &msg);
#endif
@@ -272,6 +279,12 @@ class APIConnection final : public APIServerConnectionBase {
void on_ping_request();
void on_device_info_request();
void on_device_capabilities_request();
#ifdef USE_API_WIZARD
void on_device_wizard_request();
#endif
#ifdef USE_API_WIZARD_INPUTS
void on_wizard_input_set_request(const WizardInputSetRequest &msg);
#endif
void on_list_entities_request() { this->begin_iterator_(ActiveIterator::LIST_ENTITIES); }
void on_subscribe_states_request() {
this->flags_.state_subscription = true;
@@ -301,6 +314,10 @@ class APIConnection final : public APIServerConnectionBase {
#endif
#ifdef USE_API_HOMEASSISTANT_STATES
void on_subscribe_home_assistant_states_request();
#ifdef USE_API_WIZARD_LINKED_INPUTS
/// Tell this client about the subscriptions whose entity id is stored in the given buffer, as the buffer changed
void resend_state_subscriptions(const char *entity_id);
#endif
#endif
#ifdef USE_API_USER_DEFINED_ACTIONS
void on_execute_service_request(const ExecuteServiceRequest &msg);
@@ -316,9 +333,15 @@ class APIConnection final : public APIServerConnectionBase {
void on_noise_encryption_set_key_request(const NoiseEncryptionSetKeyRequest &msg);
#endif
// How long a new connection holds off the spare ephemeral refill
static constexpr uint32_t CONNECT_GRACE_MS = 1000;
bool is_authenticated() {
return static_cast<ConnectionState>(this->flags_.connection_state) == ConnectionState::AUTHENTICATED;
}
// An older unauthenticated connection is a stale half open client and does not count
bool is_still_connecting(uint32_t now) {
return !this->is_authenticated() && now - this->last_traffic_ < CONNECT_GRACE_MS;
}
bool is_connection_setup() {
return static_cast<ConnectionState>(this->flags_.connection_state) == ConnectionState::CONNECTED ||
this->is_authenticated();
@@ -338,18 +361,14 @@ class APIConnection final : public APIServerConnectionBase {
void on_no_setup_connection();
// Function pointer type for type-erased message encoding
using MessageEncodeFn = uint8_t *(*) (const void *, ProtoWriteBuffer &PROTO_ENCODE_DEBUG_PARAM);
using MessageEncodeFn = ProtoEncodeFn;
// Function pointer type for type-erased size calculation
using CalculateSizeFn = uint32_t (*)(const void *);
/// Returns false as soon as the TCP buffer is full. Marked nodiscard so we
/// have no silent failures: every caller must handle (or log) a refusal.
template<typename T> [[nodiscard]] bool send_message(const T &msg) {
if constexpr (T::ESTIMATED_SIZE == 0) {
return this->send_message_(0, T::MESSAGE_TYPE, &encode_msg_noop, &msg);
} else {
return this->send_message_(msg.calculate_size(), T::MESSAGE_TYPE, &proto_encode_msg<T>, &msg);
}
return this->send_message_(T::calc_size_msg(&msg), T::MESSAGE_TYPE, &T::encode_msg, &msg);
}
/// Clear the shared write buffer and reserve space for the first message.
@@ -375,6 +394,23 @@ class APIConnection final : public APIServerConnectionBase {
return this->helper_->get_peername_to(buf);
}
#ifdef USE_API_OUTGOING_CONNECTION
/// Get the peer address itself, for remembering a dial-back target
int getpeername(struct sockaddr *addr, socklen_t *addrlen) const { return this->helper_->getpeername(addr, addrlen); }
/// Outgoing connection: send our server hello immediately so the peer can
/// pick the matching key. Outgoing connections are only dialed when a PSK
/// is set, so the helper is always the noise helper. Call after start().
void mark_outgoing() {
if (this->flags_.remove) {
return; // start() failed; the connection is already being torn down
}
APIError err = static_cast<APINoiseFrameHelper *>(this->helper_.get())->send_server_hello_first();
if (err != APIError::OK) {
this->fatal_error_with_log_(LOG_STR("Server hello failed"), err);
}
}
#endif
protected:
bool try_to_clear_buffer_slow_(bool log_out_of_space);
@@ -387,6 +423,9 @@ class APIConnection final : public APIServerConnectionBase {
bool send_ping_response_();
bool send_device_info_response_();
bool send_device_capabilities_response_();
#ifdef USE_API_WIZARD
bool send_device_wizard_response_();
#endif
#ifdef USE_API_NOISE
bool send_noise_encryption_set_key_response_(const NoiseEncryptionSetKeyRequest &msg);
#endif
@@ -405,16 +444,6 @@ class APIConnection final : public APIServerConnectionBase {
void process_state_subscriptions_();
#endif
// Size thunk — converts void* back to concrete type for direct calculate_size() call
template<typename T> static uint32_t calc_size(const void *msg) {
return static_cast<const T *>(msg)->calculate_size();
}
// Shared no-op encode thunk for empty messages (ESTIMATED_SIZE == 0)
static uint8_t *encode_msg_noop(const void *, ProtoWriteBuffer &buf PROTO_ENCODE_DEBUG_PARAM) {
return buf.get_pos();
}
// Non-template buffer management for send_message
bool send_message_(uint32_t payload_size, uint16_t message_type, MessageEncodeFn encode_fn, const void *msg);
@@ -433,11 +462,7 @@ class APIConnection final : public APIServerConnectionBase {
// Hot paths (state/info) go through fill_and_encode_entity_state/info instead.
// batch_message_type_ is already set by dispatch_message_ before reaching here.
template<typename T> static uint16_t encode_message_to_buffer(T &msg, APIConnection *conn, uint32_t remaining_size) {
if constexpr (T::ESTIMATED_SIZE == 0) {
return encode_to_buffer_slow(0, &encode_msg_noop, &msg, conn, remaining_size);
} else {
return encode_to_buffer_slow(msg.calculate_size(), &proto_encode_msg<T>, &msg, conn, remaining_size);
}
return encode_to_buffer_slow(T::calc_size_msg(&msg), &T::encode_msg, &msg, conn, remaining_size);
}
// Non-template core — fills state fields and encodes
@@ -449,7 +474,7 @@ class APIConnection final : public APIServerConnectionBase {
template<typename T>
static uint16_t fill_and_encode_entity_state(EntityBase *entity, T &msg, APIConnection *conn,
uint32_t remaining_size) {
return fill_and_encode_entity_state(entity, msg, &calc_size<T>, &proto_encode_msg<T>, conn, remaining_size);
return fill_and_encode_entity_state(entity, msg, &T::calc_size_msg, &T::encode_msg, conn, remaining_size);
}
// Non-template core — fills info fields, allocates buffers, and encodes
@@ -461,7 +486,7 @@ class APIConnection final : public APIServerConnectionBase {
template<typename T>
static uint16_t fill_and_encode_entity_info(EntityBase *entity, T &msg, APIConnection *conn,
uint32_t remaining_size) {
return fill_and_encode_entity_info(entity, msg, &calc_size<T>, &proto_encode_msg<T>, conn, remaining_size);
return fill_and_encode_entity_info(entity, msg, &T::calc_size_msg, &T::encode_msg, conn, remaining_size);
}
// Non-template core — fills device_class, then delegates to fill_and_encode_entity_info
@@ -475,8 +500,8 @@ class APIConnection final : public APIServerConnectionBase {
static uint16_t fill_and_encode_entity_info_with_device_class(EntityBase *entity, T &msg,
StringRef &device_class_field, APIConnection *conn,
uint32_t remaining_size) {
return fill_and_encode_entity_info_with_device_class(entity, msg, device_class_field, &calc_size<T>,
&proto_encode_msg<T>, conn, remaining_size);
return fill_and_encode_entity_info_with_device_class(entity, msg, device_class_field, &T::calc_size_msg,
&T::encode_msg, conn, remaining_size);
}
#ifdef USE_VOICE_ASSISTANT
@@ -745,12 +770,21 @@ class APIConnection final : public APIServerConnectionBase {
uint8_t batch_first_message : 1; // For batch buffer allocation
uint8_t should_try_send_immediately : 1; // True after initial states are sent
uint8_t may_have_remaining_data : 1; // Read loop hit limit, retry without ready check
#ifdef USE_API_WIZARD_LINKED_INPUTS
uint8_t home_assistant_states : 1; // Client subscribed to Home Assistant states
#endif
#ifdef USE_API_OUTGOING_CONNECTION
uint8_t outgoing_connection_target : 1; // Client declared itself a dial-back target in its hello
#endif
#ifdef HAS_PROTO_MESSAGE_DUMP
uint8_t log_only_mode : 1;
#endif
} flags_{}; // 2 bytes total
#ifdef USE_SERIAL_PROXY_USB_IDENTITY
uint8_t serial_proxy_identity_subscription : 1;
#endif
} flags_{}; // 2 bytes; 3 with HAS_PROTO_MESSAGE_DUMP + USE_API_OUTGOING_CONNECTION + USE_SERIAL_PROXY_USB_IDENTITY
// 2-byte type immediately after flags_ (no padding between them)
// 2-byte type immediately after flags_ (one padding byte when flags_ is 3 bytes)
uint16_t batch_message_type_{0}; // Current message type during batch encoding
// 1-byte types to fill remaining space before next 4-byte boundary
// Client API versions are clamped to 255 on receive (see send_hello_response_)
esphome/components/api/api_connection_buffer.h
+8 -2
View File
@@ -45,8 +45,14 @@ inline uint16_t ESPHOME_ALWAYS_INLINE APIConnection::encode_to_buffer(uint32_t c
conn->fatal_out_of_memory_();
return 0;
}
ProtoWriteBuffer buffer{&shared_buf, shared_buf.size() - calculated_size};
encode_fn(msg, buffer PROTO_ENCODE_DEBUG_INIT(&shared_buf));
uint8_t *end =
encode_fn(msg, shared_buf.data() + shared_buf.size() - calculated_size PROTO_ENCODE_DEBUG_INIT(&shared_buf));
#ifdef ESPHOME_DEBUG_API
// A body that writes fewer bytes than calculate_size() promised would ship stale buffer bytes
proto_check_encode_end(end, shared_buf.data() + shared_buf.size());
#else
(void) end;
#endif
return total_calculated_size;
}
esphome/components/api/api_frame_helper.h
+2 -1
View File
@@ -285,7 +285,8 @@ class APIFrameHelper {
DATA = 5,
CLOSED = 6,
FAILED = 7,
EXPLICIT_REJECT = 8, // Noise only
EXPLICIT_REJECT = 8, // Noise only
CLIENT_HELLO_OUTGOING = 9, // Noise only: like CLIENT_HELLO but the server hello already went out (outgoing conn)
};
// Fast inline state check for read_packet/write_protobuf_messages hot path.
esphome/components/api/api_frame_helper_noise.cpp
+27 -20
View File
@@ -5,16 +5,13 @@
#include "esphome/components/noise/noise.h"
#include "esphome/core/application.h"
#include "esphome/core/entity_base.h"
#include "esphome/core/hal.h"
#include "esphome/core/helpers.h"
#include "esphome/core/log.h"
#include "proto.h"
#include <cstring>
#include <cinttypes>
#ifdef USE_ESP8266
#include <pgmspace.h>
#endif
namespace esphome::api {
using noise::noise_err_to_logstr;
@@ -26,11 +23,7 @@ static_assert(MAX_HANDSHAKE_SIZE == noise::MAX_HANDSHAKE_SIZE,
"api and noise component handshake size limits must match");
static const char *const TAG = "api.noise";
#ifdef USE_ESP8266
static constexpr char PROLOGUE_INIT[] PROGMEM = "NoiseAPIInit";
#else
static const char *const PROLOGUE_INIT = "NoiseAPIInit";
#endif
static constexpr size_t PROLOGUE_INIT_LEN = 12; // strlen("NoiseAPIInit")
// Maximum bytes to log in hex format (168 * 3 = 504, under TX buffer size of 512)
@@ -72,15 +65,18 @@ APIError APINoiseFrameHelper::init() {
state_ = State::FAILED;
return APIError::OUT_OF_MEMORY;
}
#ifdef USE_ESP8266
memcpy_P(dst, PROLOGUE_INIT, PROLOGUE_INIT_LEN);
#else
std::memcpy(dst, PROLOGUE_INIT, PROLOGUE_INIT_LEN);
#endif
progmem_memcpy(dst, PROLOGUE_INIT, PROLOGUE_INIT_LEN);
state_ = State::CLIENT_HELLO;
return APIError::OK;
}
#ifdef USE_API_OUTGOING_CONNECTION
APIError APINoiseFrameHelper::send_server_hello_first() {
// The peer needs our name and MAC to pick the key before its first message
this->state_ = State::CLIENT_HELLO_OUTGOING;
return this->send_server_hello_frame_();
}
#endif
#ifdef USE_API_PLAINTEXT
APIError APINoiseFrameHelper::init_from_handoff(const uint8_t *header, uint8_t header_len) {
APIError err = this->init();
@@ -253,6 +249,9 @@ APIError APINoiseFrameHelper::state_action_() {
HELPER_LOG("Bad state for method: %d", (int) this->state_);
return APIError::BAD_STATE;
case State::CLIENT_HELLO:
#ifdef USE_API_OUTGOING_CONNECTION
case State::CLIENT_HELLO_OUTGOING:
#endif
return this->state_action_client_hello_();
case State::SERVER_HELLO:
return this->state_action_server_hello_();
@@ -285,11 +284,16 @@ APIError APINoiseFrameHelper::state_action_client_hello_() {
std::memcpy(dst + 2, this->rx_buf_.data(), rx_size);
}
#ifdef USE_API_OUTGOING_CONNECTION
if (this->state_ == State::CLIENT_HELLO_OUTGOING) {
// Server hello already went out at handoff
return this->start_handshake_();
}
#endif
state_ = State::SERVER_HELLO;
return APIError::OK;
}
APIError APINoiseFrameHelper::state_action_server_hello_() {
// send server hello
APIError APINoiseFrameHelper::send_server_hello_frame_() {
const auto &name = App.get_name();
char mac[MAC_ADDRESS_BUFFER_SIZE];
get_mac_address_into_buffer(mac);
@@ -313,15 +317,18 @@ APIError APINoiseFrameHelper::state_action_server_hello_() {
// node mac, terminated by null byte
std::memcpy(msg + mac_offset, mac, MAC_ADDRESS_BUFFER_SIZE);
APIError aerr = write_frame_(msg, total_size);
return write_frame_(msg, total_size);
}
APIError APINoiseFrameHelper::state_action_server_hello_() {
APIError aerr = this->send_server_hello_frame_();
if (aerr != APIError::OK)
return aerr;
// start handshake
aerr = init_handshake_();
return this->start_handshake_();
}
APIError APINoiseFrameHelper::start_handshake_() {
APIError aerr = init_handshake_();
if (aerr != APIError::OK)
return aerr;
state_ = State::HANDSHAKE;
return APIError::OK;
}
esphome/components/api/api_frame_helper_noise.h
+8
View File
@@ -28,6 +28,12 @@ class APINoiseFrameHelper final : public APIFrameHelper {
// Seeds the already-read header bytes and pumps the handshake state machine
// until it would block.
APIError init_from_handoff(const uint8_t *header, uint8_t header_len);
#endif
#ifdef USE_API_OUTGOING_CONNECTION
// Send the server hello immediately so the peer can pick the key before
// its PSK-mixed message. Call after init(); the mode is tracked in state_
// so the helper does not grow.
APIError send_server_hello_first();
#endif
APIError loop() override;
APIError read_packet(ReadPacketBuffer *buffer) override;
@@ -39,6 +45,8 @@ class APINoiseFrameHelper final : public APIFrameHelper {
APIError state_action_();
APIError state_action_client_hello_();
APIError state_action_server_hello_();
APIError send_server_hello_frame_();
APIError start_handshake_();
APIError state_action_handshake_();
APIError state_action_handshake_read_();
APIError state_action_handshake_write_();
esphome/components/api/api_outgoing_connection.cpp
+318
View File
@@ -0,0 +1,318 @@
#include "api_outgoing_connection.h"
#if defined(USE_API) && defined(USE_API_OUTGOING_CONNECTION)
#include "api_connection.h"
#include "api_server.h"
#include "esphome/components/network/util.h"
#include "esphome/core/application.h"
#include "esphome/core/helpers.h"
#include "esphome/core/log.h"
#include <cerrno>
#include <cinttypes>
#include <cstdint>
#include <cstring>
namespace esphome::api {
static const char *const TAG = "api.outgoing";
#ifndef API_OUTGOING_CONNECTION_HOST
static constexpr uint32_t OUTGOING_TARGET_PREF_HASH = 629847102UL;
#endif
#ifndef API_OUTGOING_CONNECTION_HOST
// Read the connection's peer address into target; false when unavailable or
// of a family this build cannot dial
static bool peer_to_target(APIConnection *conn, SavedOutgoingTarget &target) {
// Zeroed because the raw lwIP getpeername() leaves sin6_scope_id untouched
struct sockaddr_storage peer = {};
socklen_t peer_len = sizeof(peer);
if (conn->getpeername((struct sockaddr *) &peer, &peer_len) != 0) {
return false;
}
const sa_family_t family = ((struct sockaddr *) &peer)->sa_family;
#if USE_NETWORK_IPV6
if (family == AF_INET6) {
const auto *addr6 = reinterpret_cast<const struct sockaddr_in6 *>(&peer);
const auto *bytes = reinterpret_cast<const uint8_t *>(&addr6->sin6_addr);
uint32_t prefix[3];
memcpy(prefix, bytes, sizeof(prefix));
// A dual-stack listener reports an IPv4 peer as ::ffff:a.b.c.d
if (prefix[0] == 0 && prefix[1] == 0 && prefix[2] == htonl(0xFFFFUL)) {
target.family = AF_INET;
memcpy(target.addr, bytes + sizeof(prefix), sizeof(struct in_addr));
return true;
}
// A link-local target is only reachable through the interface it came in
// on. Device platforms number interfaces from one; a host build can hand
// out an index too large to store, and a truncated one dials the wrong
// interface, so that target is not remembered at all.
if (addr6->sin6_scope_id > UINT8_MAX) {
return false;
}
target.family = AF_INET6;
memcpy(target.addr, bytes, sizeof(target.addr));
target.scope_id = static_cast<uint8_t>(addr6->sin6_scope_id);
return true;
}
#endif
if (family != AF_INET) {
return false;
}
const auto *addr4 = reinterpret_cast<const struct sockaddr_in *>(&peer);
target.family = AF_INET;
memcpy(target.addr, &addr4->sin_addr, sizeof(addr4->sin_addr));
return true;
}
#endif
socklen_t OutgoingConnectionManager::target_sockaddr_(struct sockaddr_storage *addr) const {
#ifdef API_OUTGOING_CONNECTION_HOST
// Validation only lets through a literal both inet_pton and inet6_aton
// accept, so this cannot fail
return socket::set_sockaddr((struct sockaddr *) addr, sizeof(*addr), API_OUTGOING_CONNECTION_HOST,
API_OUTGOING_CONNECTION_PORT);
#else
#if USE_NETWORK_IPV6
if (this->saved_.family == AF_INET6) {
auto *addr6 = reinterpret_cast<struct sockaddr_in6 *>(addr);
memset(addr6, 0, sizeof(*addr6));
addr6->sin6_family = AF_INET6;
addr6->sin6_port = htons(API_OUTGOING_CONNECTION_PORT);
memcpy(&addr6->sin6_addr, this->saved_.addr, sizeof(this->saved_.addr));
addr6->sin6_scope_id = this->saved_.scope_id;
return sizeof(*addr6);
}
#endif
if (this->saved_.family != AF_INET) {
return 0;
}
auto *addr4 = reinterpret_cast<struct sockaddr_in *>(addr);
memset(addr4, 0, sizeof(*addr4));
addr4->sin_family = AF_INET;
addr4->sin_port = htons(API_OUTGOING_CONNECTION_PORT);
memcpy(&addr4->sin_addr, this->saved_.addr, sizeof(addr4->sin_addr));
return sizeof(*addr4);
#endif
}
#ifndef API_OUTGOING_CONNECTION_HOST
void OutgoingConnectionManager::format_target_(std::span<char, socket::SOCKADDR_STR_LEN> buf) const {
struct sockaddr_storage addr;
socklen_t addr_len = this->target_sockaddr_(&addr);
if (addr_len == 0) {
buf[0] = '\0';
return;
}
// Clears buf itself if it cannot format the address
socket::format_sockaddr_to((struct sockaddr *) &addr, addr_len, buf);
}
#endif
void OutgoingConnectionManager::setup() {
#ifndef API_OUTGOING_CONNECTION_HOST
this->target_pref_ = global_preferences->make_preference<SavedOutgoingTarget>(OUTGOING_TARGET_PREF_HASH, true);
struct sockaddr_storage addr;
// dump_config() prints whichever target this leaves in place
if (this->target_pref_.load(&this->saved_) && this->target_sockaddr_(&addr) != 0) {
this->host_persisted_ = true;
} else {
// Never saved, failed its size or CRC check, or holds an unknown family
this->saved_ = {};
}
#endif
}
void OutgoingConnectionManager::loop(APIServer *server) {
if (server->has_outgoing_target_client_()) {
return; // on_target_client() already reset the dial state
}
if (this->dialed_conn_ != nullptr) {
// A live dialed session (flagged or not, e.g. a host: peer) is the
// target; a silent one dies on the handshake timeout
return;
}
const uint32_t now = App.get_loop_component_start_time();
switch (this->state_) {
case DialState::DIAL_STATE_IDLE:
// Target went away; give it the configured delay to reconnect first
this->schedule_wait_(now, IDLE_WAIT_MS);
break;
case DialState::DIAL_STATE_WAITING:
if (now - this->state_ts_ >= this->wait_) {
this->try_dial_(server, now);
}
break;
case DialState::DIAL_STATE_CONNECTING:
this->poll_connect_(server, now);
break;
}
}
void OutgoingConnectionManager::try_dial_(APIServer *server, uint32_t now) {
if (!network::is_connected()) {
// Flips within seconds of boot; recheck fast so a deep sleep wake
// window is not spent waiting
this->schedule_wait_(now, NETWORK_RETRY_MS);
return;
}
struct sockaddr_storage addr;
socklen_t addr_len = this->target_sockaddr_(&addr);
const bool at_limit = server->at_client_limit_();
// No target is the steady state until a dial-back client has ever connected
if (addr_len == 0 || at_limit || !server->noise_ctx_.has_psk()) {
// Repeats for as long as the reason holds, so keep it out of debug logs
ESP_LOGV(TAG, "Not dialing: %s",
addr_len == 0 ? LOG_STR_LITERAL("no target")
: (at_limit ? LOG_STR_LITERAL("max connections") : LOG_STR_LITERAL("no key")));
// Not a dial failure; retry without escalating the backoff
this->schedule_wait_(now, PRECONDITION_RETRY_MS);
return;
}
this->dial_socket_ = socket::socket_loop_monitored(((struct sockaddr *) &addr)->sa_family, SOCK_STREAM, IPPROTO_TCP);
if (!this->dial_socket_ || this->dial_socket_->setblocking(false) != 0) {
ESP_LOGW(TAG, "Socket %s failed: errno %d",
this->dial_socket_ ? LOG_STR_LITERAL("setblocking") : LOG_STR_LITERAL("create"), errno);
this->schedule_retry_(now);
return;
}
#ifdef API_OUTGOING_CONNECTION_HOST
ESP_LOGD(TAG, "Dialing " API_OUTGOING_CONNECTION_HOST ":%u", API_OUTGOING_CONNECTION_PORT);
#else
char host[socket::SOCKADDR_STR_LEN];
socket::format_sockaddr_to((struct sockaddr *) &addr, addr_len, host);
ESP_LOGD(TAG, "Dialing %s:%u", host, API_OUTGOING_CONNECTION_PORT);
#endif
int err = this->dial_socket_->connect((struct sockaddr *) &addr, addr_len);
if (err == 0) {
// Immediate success (possible for localhost)
this->handoff_(server, now);
return;
}
if (errno != EINPROGRESS) {
ESP_LOGW(TAG, "Connect failed: %d", errno);
this->schedule_retry_(now);
return;
}
this->state_ = DialState::DIAL_STATE_CONNECTING;
this->state_ts_ = now;
this->last_poll_ = now;
}
void OutgoingConnectionManager::poll_connect_(APIServer *server, uint32_t now) {
if (now - this->state_ts_ >= CONNECT_TIMEOUT_MS) {
ESP_LOGW(TAG, "Connect timeout");
this->schedule_retry_(now);
return;
}
if (now - this->last_poll_ < CONNECT_POLL_INTERVAL_MS) {
return;
}
this->last_poll_ = now;
int err = 0;
switch (socket::poll_connect(*this->dial_socket_, err)) {
case socket::ConnectPollResult::CONNECT_POLL_RESULT_PENDING:
break;
case socket::ConnectPollResult::CONNECT_POLL_RESULT_CONNECTED:
this->handoff_(server, now);
break;
case socket::ConnectPollResult::CONNECT_POLL_RESULT_ERROR:
ESP_LOGW(TAG, "Connect failed: %d", err);
this->schedule_retry_(now);
break;
}
}
void OutgoingConnectionManager::handoff_(APIServer *server, uint32_t now) {
this->dialed_conn_ = server->add_outgoing_client_(std::move(this->dial_socket_));
if (this->dialed_conn_ == nullptr) {
// Only preconditions (slot limit, key cleared) refuse the handoff; the
// peer is reachable, so do not escalate the backoff
this->schedule_wait_(now, PRECONDITION_RETRY_MS);
return;
}
// Connected; dialed_conn_ gates further dialing until the session settles
this->state_ = DialState::DIAL_STATE_IDLE;
}
void OutgoingConnectionManager::schedule_wait_(uint32_t now, uint32_t wait) {
this->dial_socket_.reset(); // no-op when the socket was handed off
this->state_ = DialState::DIAL_STATE_WAITING;
this->state_ts_ = now;
this->wait_ = wait;
}
void OutgoingConnectionManager::schedule_retry_(uint32_t now) {
// +/-20% jitter so a fleet of devices does not retry one server in lockstep
const uint32_t jitter_span = this->backoff_ / 5;
this->schedule_wait_(now, this->backoff_ - jitter_span + (random_uint32() % (2 * jitter_span + 1)));
this->backoff_ = std::min(this->backoff_ * 2, BACKOFF_MAX_MS);
}
void OutgoingConnectionManager::on_client_removed(APIConnection *conn, bool was_authenticated) {
if (conn != this->dialed_conn_) {
return;
}
this->dialed_conn_ = nullptr;
if (was_authenticated) {
// A working peer (e.g. a host: target that never sends the flag)
// disconnected normally; state is IDLE, so loop() applies the delay
this->backoff_ = BACKOFF_MIN_MS;
} else {
this->schedule_retry_(App.get_loop_component_start_time());
}
}
void OutgoingConnectionManager::on_target_client(APIConnection *conn) {
// The target is connected; stop any dial in flight and reset the backoff.
// A dialed connection stays tracked unless it is this one: an inbound
// target must not orphan a still-open dial.
this->dial_socket_.reset();
if (conn == this->dialed_conn_) {
this->dialed_conn_ = nullptr;
}
this->state_ = DialState::DIAL_STATE_IDLE;
this->backoff_ = BACKOFF_MIN_MS;
#ifndef API_OUTGOING_CONNECTION_HOST
SavedOutgoingTarget target{};
if (!peer_to_target(conn, target)) {
ESP_LOGW(TAG, "Not remembering this target; its address cannot be dialed");
return;
}
if (this->host_persisted_ && memcmp(&target, &this->saved_, sizeof(target)) == 0) {
return; // unchanged and already on flash; avoid flash wear
}
// Use the fresh address this boot even if the flash write fails; a failed
// write is retried on the next flagged hello via host_persisted_
this->saved_ = target;
if (!this->persist_target_()) {
ESP_LOGW(TAG, "Failed to save target");
return;
}
char host[socket::SOCKADDR_STR_LEN];
this->format_target_(host);
ESP_LOGD(TAG, "Remembered %s as the dial target", host);
#endif
}
void OutgoingConnectionManager::dump_config() const {
// The boot delay differs from delay: on deep sleep builds, so print the
// value that actually applies
ESP_LOGCONFIG(TAG,
" Outgoing connection port: %u\n"
" Outgoing connection boot delay: %" PRIu32 "ms",
API_OUTGOING_CONNECTION_PORT, BOOT_WAIT_MS);
// Both forms keep their text out of RAM on ESP8266: in the format string,
// or through LOG_STR_LITERAL
#ifdef API_OUTGOING_CONNECTION_HOST
ESP_LOGCONFIG(TAG, " Outgoing connection host: " API_OUTGOING_CONNECTION_HOST);
#else
char buf[socket::SOCKADDR_STR_LEN];
this->format_target_(buf);
ESP_LOGCONFIG(TAG, " Outgoing connection host: %s", buf[0] == '\0' ? LOG_STR_LITERAL("none remembered yet") : buf);
#endif
}
} // namespace esphome::api
#endif // USE_API && USE_API_OUTGOING_CONNECTION
esphome/components/api/api_outgoing_connection.h
+120
View File
@@ -0,0 +1,120 @@
#pragma once
#include "esphome/core/defines.h"
#if defined(USE_API) && defined(USE_API_OUTGOING_CONNECTION)
#ifndef USE_API_NOISE
#error "api outgoing_connection needs noise encryption so the peer is verified by key"
#endif
#include "esphome/components/socket/socket.h"
#include "esphome/core/preferences.h"
#include <memory>
namespace esphome::api {
class APIServer;
class APIConnection;
// Room for an IPv6 address in every build, so a remembered IPv4 target is
// still dialed after enable_ipv6 is turned on. A size that followed the build
// would also shift every preference registered after this one on ESP8266,
// where slots are positional. An IPv6 target on a build without IPv6 is
// dropped by target_sockaddr_() and relearned.
// Bytes in an IPv6 address
static constexpr size_t TARGET_ADDR_LEN = 16;
struct SavedOutgoingTarget {
// 0 when none is remembered, else AF_INET or AF_INET6
uint8_t family;
// Network order, IPv4 in the first four bytes and the rest zero
uint8_t addr[TARGET_ADDR_LEN];
// Interface a link-local IPv6 target is reachable on, 0 when it needs none.
// Free in flash: the record still rounds up to the same five words.
uint8_t scope_id;
} PACKED; // NOLINT
/// Dials out when no dial-back target client is connected. Only the TCP
/// direction flips: the device stays the Noise responder, so both sides
/// still verify by key. Targets the YAML host or the last remembered client.
class OutgoingConnectionManager {
public:
void setup();
void loop(APIServer *server);
/// A key-verified client declared itself a dial-back target; last one wins
void on_target_client(APIConnection *conn);
/// Clears the dialed-connection gate; dying unauthenticated escalates the backoff
void on_client_removed(APIConnection *conn, bool was_authenticated);
void on_shutdown() { this->dial_socket_.reset(); }
void dump_config() const;
protected:
enum class DialState : uint8_t {
DIAL_STATE_IDLE,
DIAL_STATE_WAITING,
DIAL_STATE_CONNECTING,
};
static constexpr uint32_t BACKOFF_MIN_MS = 5000;
static constexpr uint32_t BACKOFF_MAX_MS = 300000;
static constexpr uint32_t CONNECT_TIMEOUT_MS = 10000;
static constexpr uint32_t CONNECT_POLL_INTERVAL_MS = 250;
static constexpr uint32_t NETWORK_RETRY_MS = 500;
static constexpr uint32_t PRECONDITION_RETRY_MS = 5000;
// A deep sleep wake window is too short to spend on the delay, so those
// builds dial out as soon as the target is gone
#ifdef USE_DEEP_SLEEP
static constexpr uint32_t BOOT_WAIT_MS = 0;
static constexpr uint32_t IDLE_WAIT_MS = BACKOFF_MIN_MS;
#else
static constexpr uint32_t BOOT_WAIT_MS = API_OUTGOING_CONNECTION_DELAY;
static constexpr uint32_t IDLE_WAIT_MS = API_OUTGOING_CONNECTION_DELAY;
#endif
void try_dial_(APIServer *server, uint32_t now);
void poll_connect_(APIServer *server, uint32_t now);
// Hand the connected socket to the server and gate on the new connection
void handoff_(APIServer *server, uint32_t now);
// Close any half-open dial and wait a jittered backoff before retrying
void schedule_retry_(uint32_t now);
// Wait without escalating the backoff (used for unmet preconditions)
void schedule_wait_(uint32_t now, uint32_t wait);
/// Fill addr with the target and return its length, or 0 when there is none
socklen_t target_sockaddr_(struct sockaddr_storage *addr) const;
#ifndef API_OUTGOING_CONNECTION_HOST
// Write saved_ to flash, tracking success in host_persisted_
bool persist_target_() {
this->host_persisted_ = this->target_pref_.save(&this->saved_) && global_preferences->sync();
return this->host_persisted_;
}
/// Format the remembered target for a log line; empty when there is none
void format_target_(std::span<char, socket::SOCKADDR_STR_LEN> buf) const;
#endif
// Pointers first (4 bytes each on 32-bit)
std::unique_ptr<socket::Socket> dial_socket_;
// Compared only, never dereferenced
APIConnection *dialed_conn_{nullptr};
#ifndef API_OUTGOING_CONNECTION_HOST
ESPPreferenceObject target_pref_;
#endif
// 4-byte types
uint32_t backoff_{BACKOFF_MIN_MS};
uint32_t wait_{BOOT_WAIT_MS};
uint32_t state_ts_{0};
uint32_t last_poll_{0};
// Byte-aligned types last
#ifndef API_OUTGOING_CONNECTION_HOST
SavedOutgoingTarget saved_{};
// False while saved_ holds a value the flash write failed for; retried on
// the next flagged hello
bool host_persisted_{false};
#endif
DialState state_{DialState::DIAL_STATE_WAITING};
};
} // namespace esphome::api
#endif // USE_API && USE_API_OUTGOING_CONNECTION
esphome/components/api/api_pb2.cpp
+2804 -2736
View File
File diff suppressed because it is too large Load Diff
esphome/components/api/api_pb2.h
+1055 -359
View File
File diff suppressed because it is too large Load Diff
esphome/components/api/api_pb2_dump.cpp
+111 -1
View File
@@ -143,6 +143,8 @@ template<> const char *proto_enum_to_string<enums::SerialProxyPortType>(enums::S
return ESPHOME_PSTR("SERIAL_PROXY_PORT_TYPE_RS232");
case enums::SERIAL_PROXY_PORT_TYPE_RS485:
return ESPHOME_PSTR("SERIAL_PROXY_PORT_TYPE_RS485");
case enums::SERIAL_PROXY_PORT_TYPE_USB_SERIAL:
return ESPHOME_PSTR("SERIAL_PROXY_PORT_TYPE_USB_SERIAL");
default:
return ESPHOME_PSTR("UNKNOWN");
}
@@ -854,6 +856,8 @@ template<> const char *proto_enum_to_string<enums::SerialProxyRequestType>(enums
return ESPHOME_PSTR("SERIAL_PROXY_REQUEST_TYPE_CONFIGURE");
case enums::SERIAL_PROXY_REQUEST_TYPE_SET_MODEM_PINS:
return ESPHOME_PSTR("SERIAL_PROXY_REQUEST_TYPE_SET_MODEM_PINS");
case enums::SERIAL_PROXY_REQUEST_TYPE_SET_MODE:
return ESPHOME_PSTR("SERIAL_PROXY_REQUEST_TYPE_SET_MODE");
default:
return ESPHOME_PSTR("UNKNOWN");
}
@@ -878,13 +882,50 @@ template<> const char *proto_enum_to_string<enums::SerialProxyStatus>(enums::Ser
return ESPHOME_PSTR("UNKNOWN");
}
}
template<> const char *proto_enum_to_string<enums::SerialProxyMode>(enums::SerialProxyMode value) {
switch (value) {
case enums::SERIAL_PROXY_MODE_RAW:
return ESPHOME_PSTR("SERIAL_PROXY_MODE_RAW");
case enums::SERIAL_PROXY_MODE_PROTOCOL:
return ESPHOME_PSTR("SERIAL_PROXY_MODE_PROTOCOL");
default:
return ESPHOME_PSTR("UNKNOWN");
}
}
template<> const char *proto_enum_to_string<enums::SerialProxyIdentitySource>(enums::SerialProxyIdentitySource value) {
switch (value) {
case enums::SERIAL_PROXY_IDENTITY_SOURCE_NONE:
return ESPHOME_PSTR("SERIAL_PROXY_IDENTITY_SOURCE_NONE");
case enums::SERIAL_PROXY_IDENTITY_SOURCE_CONFIGURED:
return ESPHOME_PSTR("SERIAL_PROXY_IDENTITY_SOURCE_CONFIGURED");
case enums::SERIAL_PROXY_IDENTITY_SOURCE_USB:
return ESPHOME_PSTR("SERIAL_PROXY_IDENTITY_SOURCE_USB");
default:
return ESPHOME_PSTR("UNKNOWN");
}
}
#endif
template<> const char *proto_enum_to_string<enums::SerialProxyIdentityFlag>(enums::SerialProxyIdentityFlag value) {
switch (value) {
case enums::SERIAL_PROXY_IDENTITY_FLAG_NONE:
return ESPHOME_PSTR("SERIAL_PROXY_IDENTITY_FLAG_NONE");
case enums::SERIAL_PROXY_IDENTITY_FLAG_CONNECTED:
return ESPHOME_PSTR("SERIAL_PROXY_IDENTITY_FLAG_CONNECTED");
case enums::SERIAL_PROXY_IDENTITY_FLAG_ERROR:
return ESPHOME_PSTR("SERIAL_PROXY_IDENTITY_FLAG_ERROR");
default:
return ESPHOME_PSTR("UNKNOWN");
}
}
const char *HelloRequest::dump_to(DumpBuffer &out) const {
MessageDumpHelper helper(out, ESPHOME_PSTR("HelloRequest"));
dump_field(out, ESPHOME_PSTR("client_info"), this->client_info);
dump_field(out, ESPHOME_PSTR("api_version_major"), this->api_version_major);
dump_field(out, ESPHOME_PSTR("api_version_minor"), this->api_version_minor);
#ifdef USE_API_OUTGOING_CONNECTION
dump_field(out, ESPHOME_PSTR("outgoing_connection_target"), this->outgoing_connection_target);
#endif
return out.c_str();
}
const char *HelloResponse::dump_to(DumpBuffer &out) const {
@@ -1008,6 +1049,9 @@ const char *DeviceInfoResponse::dump_to(DumpBuffer &out) const {
#endif
#ifdef USE_API_NOISE
dump_field(out, ESPHOME_PSTR("api_encryption_provisionable"), this->api_encryption_provisionable);
#endif
#ifdef USE_API_OUTGOING_CONNECTION
dump_field(out, ESPHOME_PSTR("api_outgoing_connection_supported"), this->api_outgoing_connection_supported);
#endif
return out.c_str();
}
@@ -1034,6 +1078,13 @@ const char *ZWaveProxyCapabilities::dump_to(DumpBuffer &out) const {
return out.c_str();
}
#endif
#ifdef USE_API_WIZARD
const char *WizardCapabilities::dump_to(DumpBuffer &out) const {
MessageDumpHelper helper(out, ESPHOME_PSTR("WizardCapabilities"));
dump_field(out, ESPHOME_PSTR("configured"), this->configured);
return out.c_str();
}
#endif
const char *DeviceCapabilitiesResponse::dump_to(DumpBuffer &out) const {
MessageDumpHelper helper(out, ESPHOME_PSTR("DeviceCapabilitiesResponse"));
#ifdef USE_BLUETOOTH_PROXY
@@ -1057,9 +1108,29 @@ const char *DeviceCapabilitiesResponse::dump_to(DumpBuffer &out) const {
it.dump_to(out);
out.append("\n");
}
#endif
#ifdef USE_API_WIZARD
out.append(2, ' ').append_p(ESPHOME_PSTR("wizard")).append(": ");
this->wizard.dump_to(out);
out.append("\n");
#endif
return out.c_str();
}
#ifdef USE_API_WIZARD
const char *DeviceWizardResponse::dump_to(DumpBuffer &out) const {
MessageDumpHelper helper(out, ESPHOME_PSTR("DeviceWizardResponse"));
dump_bytes_field(out, ESPHOME_PSTR("data"), this->data, this->data_len);
return out.c_str();
}
#endif
#ifdef USE_API_WIZARD_INPUTS
const char *WizardInputSetRequest::dump_to(DumpBuffer &out) const {
MessageDumpHelper helper(out, ESPHOME_PSTR("WizardInputSetRequest"));
dump_field(out, ESPHOME_PSTR("key"), this->key);
dump_field(out, ESPHOME_PSTR("entity_id"), this->entity_id);
return out.c_str();
}
#endif
const char *ListEntitiesDoneResponse::dump_to(DumpBuffer &out) const {
out.append_p(ESPHOME_PSTR("ListEntitiesDoneResponse {}"));
return out.c_str();
@@ -1330,6 +1401,7 @@ const char *SwitchStateResponse::dump_to(DumpBuffer &out) const {
#ifdef USE_DEVICES
dump_field(out, ESPHOME_PSTR("device_id"), this->device_id);
#endif
dump_field(out, ESPHOME_PSTR("missing_state"), this->missing_state);
return out.c_str();
}
const char *SwitchCommandRequest::dump_to(DumpBuffer &out) const {
@@ -1672,6 +1744,7 @@ const char *ClimateStateResponse::dump_to(DumpBuffer &out) const {
#ifdef USE_DEVICES
dump_field(out, ESPHOME_PSTR("device_id"), this->device_id);
#endif
dump_field(out, ESPHOME_PSTR("missing_state"), this->missing_state);
return out.c_str();
}
const char *ClimateCommandRequest::dump_to(DumpBuffer &out) const {
@@ -1739,6 +1812,7 @@ const char *WaterHeaterStateResponse::dump_to(DumpBuffer &out) const {
dump_field(out, ESPHOME_PSTR("state"), this->state);
dump_field(out, ESPHOME_PSTR("target_temperature_low"), this->target_temperature_low);
dump_field(out, ESPHOME_PSTR("target_temperature_high"), this->target_temperature_high);
dump_field(out, ESPHOME_PSTR("missing_state"), this->missing_state);
return out.c_str();
}
const char *WaterHeaterCommandRequest::dump_to(DumpBuffer &out) const {
@@ -2699,7 +2773,7 @@ const char *ListEntitiesInfraredResponse::dump_to(DumpBuffer &out) const {
return out.c_str();
}
#endif
#if defined(USE_IR_RF) || defined(USE_RADIO_FREQUENCY)
#ifdef USE_IR_RF
const char *InfraredRFTransmitRawTimingsRequest::dump_to(DumpBuffer &out) const {
MessageDumpHelper helper(out, ESPHOME_PSTR("InfraredRFTransmitRawTimingsRequest"));
#ifdef USE_DEVICES
@@ -2728,6 +2802,15 @@ const char *InfraredRFReceiveEvent::dump_to(DumpBuffer &out) const {
}
return out.c_str();
}
const char *InfraredRFTransmitCompleteResponse::dump_to(DumpBuffer &out) const {
MessageDumpHelper helper(out, ESPHOME_PSTR("InfraredRFTransmitCompleteResponse"));
#ifdef USE_DEVICES
dump_field(out, ESPHOME_PSTR("device_id"), this->device_id);
#endif
dump_field(out, ESPHOME_PSTR("key"), this->key);
dump_field(out, ESPHOME_PSTR("success"), this->success);
return out.c_str();
}
#endif
#ifdef USE_RADIO_FREQUENCY
const char *ListEntitiesRadioFrequencyResponse::dump_to(DumpBuffer &out) const {
@@ -2805,6 +2888,33 @@ const char *SerialProxyRequestResponse::dump_to(DumpBuffer &out) const {
dump_field(out, ESPHOME_PSTR("error_message"), this->error_message);
return out.c_str();
}
const char *SerialProxySetModeRequest::dump_to(DumpBuffer &out) const {
MessageDumpHelper helper(out, ESPHOME_PSTR("SerialProxySetModeRequest"));
dump_field(out, ESPHOME_PSTR("instance"), this->instance);
dump_field(out, ESPHOME_PSTR("mode"), static_cast<enums::SerialProxyMode>(this->mode));
return out.c_str();
}
const char *UsbDeviceDescriptor::dump_to(DumpBuffer &out) const {
MessageDumpHelper helper(out, ESPHOME_PSTR("UsbDeviceDescriptor"));
dump_field(out, ESPHOME_PSTR("vendor_id"), this->vendor_id);
dump_field(out, ESPHOME_PSTR("product_id"), this->product_id);
dump_field(out, ESPHOME_PSTR("bcd_device"), this->bcd_device);
dump_field(out, ESPHOME_PSTR("interface_number"), this->interface_number);
return out.c_str();
}
const char *SerialProxyIdentity::dump_to(DumpBuffer &out) const {
MessageDumpHelper helper(out, ESPHOME_PSTR("SerialProxyIdentity"));
dump_field(out, ESPHOME_PSTR("instance"), this->instance);
dump_field(out, ESPHOME_PSTR("source"), static_cast<enums::SerialProxyIdentitySource>(this->source));
dump_field(out, ESPHOME_PSTR("flags"), this->flags);
dump_field(out, ESPHOME_PSTR("manufacturer"), this->manufacturer);
dump_field(out, ESPHOME_PSTR("product"), this->product);
dump_field(out, ESPHOME_PSTR("serial_number"), this->serial_number);
out.append(2, ' ').append_p(ESPHOME_PSTR("usb")).append(": ");
this->usb.dump_to(out);
out.append("\n");
return out.c_str();
}
#endif
#ifdef USE_BLUETOOTH_PROXY_CONNECTIONS
const char *BluetoothSetConnectionParamsRequest::dump_to(DumpBuffer &out) const {
esphome/components/api/api_pb2_includes.h
+1
View File
@@ -28,6 +28,7 @@
// Standard library includes that might be needed
#include <set>
#include <span>
#include <vector>
#include <string>
esphome/components/api/api_pb2_service.cpp
+41 -1
View File
@@ -628,7 +628,7 @@ void APIConnection::read_message_(uint32_t msg_size, uint32_t msg_type, const ui
break;
}
#endif
#if defined(USE_IR_RF) || defined(USE_RADIO_FREQUENCY)
#ifdef USE_IR_RF
case InfraredRFTransmitRawTimingsRequest::MESSAGE_TYPE: {
InfraredRFTransmitRawTimingsRequest msg;
msg.decode(msg_data, msg_size);
@@ -712,6 +712,46 @@ void APIConnection::read_message_(uint32_t msg_size, uint32_t msg_type, const ui
this->on_device_capabilities_request();
break;
}
#ifdef USE_SERIAL_PROXY
case SerialProxySetModeRequest::MESSAGE_TYPE: {
SerialProxySetModeRequest msg;
msg.decode(msg_data, msg_size);
#ifdef HAS_PROTO_MESSAGE_DUMP
this->log_receive_message_(LOG_STR("on_serial_proxy_set_mode_request"), msg);
#endif
this->on_serial_proxy_set_mode_request(msg);
break;
}
#endif
#ifdef USE_SERIAL_PROXY
case 154 /* SubscribeSerialProxyIdentityRequest is empty */: {
#ifdef HAS_PROTO_MESSAGE_DUMP
this->log_receive_message_(LOG_STR("on_subscribe_serial_proxy_identity_request"));
#endif
this->on_subscribe_serial_proxy_identity_request();
break;
}
#endif
#ifdef USE_API_WIZARD
case 156 /* DeviceWizardRequest is empty */: {
#ifdef HAS_PROTO_MESSAGE_DUMP
this->log_receive_message_(LOG_STR("on_device_wizard_request"));
#endif
this->on_device_wizard_request();
break;
}
#endif
#ifdef USE_API_WIZARD_INPUTS
case WizardInputSetRequest::MESSAGE_TYPE: {
WizardInputSetRequest msg;
msg.decode(msg_data, msg_size);
#ifdef HAS_PROTO_MESSAGE_DUMP
this->log_receive_message_(LOG_STR("on_wizard_input_set_request"), msg);
#endif
this->on_wizard_input_set_request(msg);
break;
}
#endif
default:
break;
}
esphome/components/api/api_pb2_service.h
+15 -1
View File
@@ -29,6 +29,13 @@ class APIServerConnectionBase {
void on_device_capabilities_request(){};
#ifdef USE_API_WIZARD
void on_device_wizard_request(){};
#endif
#ifdef USE_API_WIZARD_INPUTS
void on_wizard_input_set_request(const WizardInputSetRequest &value){};
#endif
void on_list_entities_request(){};
void on_subscribe_states_request(){};
@@ -213,7 +220,7 @@ class APIServerConnectionBase {
void on_z_wave_proxy_request(const ZWaveProxyRequest &value){};
#endif
#if defined(USE_IR_RF) || defined(USE_RADIO_FREQUENCY)
#ifdef USE_IR_RF
void on_infrared_rf_transmit_raw_timings_request(const InfraredRFTransmitRawTimingsRequest &value){};
#endif
@@ -235,6 +242,13 @@ class APIServerConnectionBase {
void on_serial_proxy_request(const SerialProxyRequest &value){};
#endif
#ifdef USE_SERIAL_PROXY
void on_serial_proxy_set_mode_request(const SerialProxySetModeRequest &value){};
#endif
#ifdef USE_SERIAL_PROXY
void on_subscribe_serial_proxy_identity_request(){};
#endif
#ifdef USE_BLUETOOTH_PROXY_CONNECTIONS
void on_bluetooth_set_connection_params_request(const BluetoothSetConnectionParamsRequest &value){};
#endif
esphome/components/api/api_server.cpp
+150 -56
View File
@@ -5,7 +5,6 @@
#include "api_connection.h"
#include "esphome/components/network/util.h"
#include "esphome/core/application.h"
#include "esphome/core/controller_registry.h"
#include "esphome/core/defines.h"
#include "esphome/core/hal.h"
#include "esphome/core/log.h"
@@ -57,12 +56,15 @@ APIServer::APIServer() { global_api_server = this; }
void APIServer::socket_failed_(const LogString *msg) {
ESP_LOGW(TAG, "Socket %s: errno %d", LOG_STR_ARG(msg), errno);
this->destroy_socket_();
#ifdef USE_API_OUTGOING_CONNECTION
// Dial-out needs no listener; degrade instead of stopping the component
this->status_set_error(LOG_STR("listen socket failed"));
#else
this->mark_failed();
#endif
}
void APIServer::setup() {
ControllerRegistry::register_controller(this);
#ifdef USE_API_NOISE
// Always reserve the slot: flash preferences are positional on esp8266, so
// a yaml key build must keep the layout of a runtime key build
@@ -75,43 +77,6 @@ void APIServer::setup() {
#endif
#endif
this->socket_ = socket::socket_ip_loop_monitored(SOCK_STREAM, 0).release(); // monitored for incoming connections
if (this->socket_ == nullptr) {
this->socket_failed_(LOG_STR("creation"));
return;
}
int enable = 1;
int err = this->socket_->setsockopt(SOL_SOCKET, SO_REUSEADDR, &enable, sizeof(int));
if (err != 0) {
ESP_LOGW(TAG, "Socket reuseaddr: errno %d", errno);
// we can still continue
}
err = this->socket_->setblocking(false);
if (err != 0) {
this->socket_failed_(LOG_STR("nonblocking"));
return;
}
struct sockaddr_storage server;
socklen_t sl = socket::set_sockaddr_any((struct sockaddr *) &server, sizeof(server), this->port_);
if (sl == 0) {
this->socket_failed_(LOG_STR("set sockaddr"));
return;
}
err = this->socket_->bind((struct sockaddr *) &server, sl);
if (err != 0) {
this->socket_failed_(LOG_STR("bind"));
return;
}
err = this->socket_->listen(this->listen_backlog_);
if (err != 0) {
this->socket_failed_(LOG_STR("listen"));
return;
}
#ifdef USE_LOGGER
if (logger::global_logger != nullptr) {
logger::global_logger->add_log_callback(
@@ -157,6 +122,47 @@ void APIServer::setup() {
if (this->reboot_timeout_ != 0 && !this->provisioning_pending_()) {
this->status_set_warning(LOG_STR("waiting for client connection"));
}
#ifdef USE_API_OUTGOING_CONNECTION
this->outgoing_conn_.setup();
#endif
// Listener last: on failure socket_failed_() returns early, and an
// outgoing_connection build keeps dialing out without one
this->socket_ = socket::socket_ip_loop_monitored(SOCK_STREAM, 0).release(); // monitored for incoming connections
if (this->socket_ == nullptr) {
this->socket_failed_(LOG_STR("creation"));
return;
}
int enable = 1;
int err = this->socket_->setsockopt(SOL_SOCKET, SO_REUSEADDR, &enable, sizeof(int));
if (err != 0) {
ESP_LOGW(TAG, "Socket reuseaddr: errno %d", errno);
// we can still continue
}
err = this->socket_->setblocking(false);
if (err != 0) {
this->socket_failed_(LOG_STR("nonblocking"));
return;
}
struct sockaddr_storage server;
socklen_t sl = socket::set_sockaddr_any((struct sockaddr *) &server, sizeof(server), this->port_);
if (sl == 0) {
this->socket_failed_(LOG_STR("set sockaddr"));
return;
}
err = this->socket_->bind((struct sockaddr *) &server, sl);
if (err != 0) {
this->socket_failed_(LOG_STR("bind"));
return;
}
err = this->socket_->listen(this->listen_backlog_);
if (err != 0) {
this->socket_failed_(LOG_STR("listen"));
}
}
void APIServer::loop() {
@@ -165,6 +171,19 @@ void APIServer::loop() {
this->accept_new_connections_();
}
const bool connected = network::is_connected();
#ifdef USE_NOISE_SPARE_EPHEMERAL
if (connected && !noise::has_spare_ephemeral()) {
this->refill_spare_ephemeral_();
}
#endif
#ifdef USE_API_OUTGOING_CONNECTION
if (!this->shutting_down_) {
this->outgoing_conn_.loop(this);
}
#endif
if (this->api_connection_count_ == 0) {
// Check reboot timeout - done in loop to avoid scheduler heap churn
// (cancelled scheduler items sit in heap memory until their scheduled time).
@@ -181,8 +200,7 @@ void APIServer::loop() {
}
// Process clients and remove disconnected ones in a single pass
// Check network connectivity once for all clients
if (!network::is_connected()) {
if (!connected) {
// Network is down - disconnect all clients
for (auto &client : this->active_clients()) {
client->on_fatal_error();
@@ -210,6 +228,19 @@ void APIServer::loop() {
}
}
#ifdef USE_NOISE_SPARE_EPHEMERAL
// An OTA handshake is not visible here and just pays the refill it triggered
void APIServer::refill_spare_ephemeral_() {
const uint32_t now = App.get_loop_component_start_time();
for (auto &client : this->active_clients()) {
if (client->is_still_connecting(now)) {
return;
}
}
noise::prepare_spare_ephemeral();
}
#endif
void APIServer::remove_client_(uint8_t client_index) {
auto &client = this->clients_[client_index];
@@ -225,6 +256,15 @@ void APIServer::remove_client_(uint8_t client_index) {
std::string client_peername(client->get_peername_to(peername_buf));
#endif
// Read before the swap-and-reset below destroys the connection
const bool was_authenticated = client->is_authenticated();
#ifdef USE_API_OUTGOING_CONNECTION
if (client->flags_.outgoing_connection_target) {
this->outgoing_target_count_--;
}
this->outgoing_conn_.on_client_removed(client.get(), was_authenticated);
#endif
// Close socket now (was deferred from on_fatal_error to allow getpeername)
client->helper_->close();
@@ -243,9 +283,15 @@ void APIServer::remove_client_(uint8_t client_index) {
// Last client disconnected - set warning and start tracking for reboot timeout
// (suppressed while provisioning is pending - see loop()).
// Refresh on every authenticated removal, not just the last one, so an
// unauthenticated straggler removed later (e.g. a port scan, or a dial to
// a host that accepts TCP but never speaks the API) cannot discard a
// healthy session's timestamp and trigger a spurious reboot
if (was_authenticated) {
this->last_connected_ = App.get_loop_component_start_time();
}
if (this->api_connection_count_ == 0 && this->reboot_timeout_ != 0 && !this->provisioning_pending_()) {
this->status_set_warning(LOG_STR("waiting for client connection"));
this->last_connected_ = App.get_loop_component_start_time();
}
#ifdef USE_API_CLIENT_DISCONNECTED_TRIGGER
@@ -267,7 +313,7 @@ void __attribute__((flatten)) APIServer::accept_new_connections_() {
sock->getpeername_to(peername);
// Check if we're at the connection limit
if (this->api_connection_count_ >= MAX_API_CONNECTIONS) {
if (this->at_client_limit_()) {
ESP_LOGW(TAG, "Max connections (%d), rejecting %s", MAX_API_CONNECTIONS, peername);
// Immediately close - socket destructor will handle cleanup
sock.reset();
@@ -276,18 +322,47 @@ void __attribute__((flatten)) APIServer::accept_new_connections_() {
ESP_LOGD(TAG, "Accept %s", peername);
auto *conn = new APIConnection(std::move(sock), this);
this->clients_[this->api_connection_count_++].reset(conn);
conn->start();
// First client connected - clear warning and update timestamp
if (this->api_connection_count_ == 1 && this->reboot_timeout_ != 0 && !this->provisioning_pending_()) {
this->status_clear_warning();
this->last_connected_ = App.get_loop_component_start_time();
}
this->add_client_(std::move(sock));
}
}
APIConnection *APIServer::add_client_(std::unique_ptr<socket::Socket> sock) {
auto *conn = new APIConnection(std::move(sock), this); // NOLINT(cppcoreguidelines-owning-memory)
this->clients_[this->api_connection_count_++].reset(conn);
conn->start();
// First client connected - clear warning. The reboot watchdog timestamp is
// refreshed when an authenticated client is removed (see remove_client_),
// never on bare TCP connects.
if (this->api_connection_count_ == 1 && this->reboot_timeout_ != 0 && !this->provisioning_pending_()) {
this->status_clear_warning();
}
return conn;
}
#ifdef USE_API_OUTGOING_CONNECTION
APIConnection *APIServer::add_outgoing_client_(std::unique_ptr<socket::Socket> sock) {
// Re-check at the handoff: inbound clients may have taken the last slot and
// the PSK may have been cleared since the dial started (mark_outgoing()
// needs the noise helper)
const bool at_limit = this->at_client_limit_();
if (at_limit || !this->noise_ctx_.has_psk()) {
ESP_LOGW(TAG, "Dropping outgoing connection (%s)",
at_limit ? LOG_STR_LITERAL("max connections") : LOG_STR_LITERAL("no key"));
return nullptr;
}
auto *conn = this->add_client_(std::move(sock));
// After start(): sends our server hello first so the peer can pick the key
conn->mark_outgoing();
return conn;
}
void APIServer::on_outgoing_target_client(APIConnection *conn) {
this->outgoing_target_count_++;
this->outgoing_conn_.on_target_client(conn);
}
#endif
void APIServer::dump_config() {
char addr_buf[network::USE_ADDRESS_BUFFER_SIZE];
ESP_LOGCONFIG(TAG,
@@ -304,6 +379,9 @@ void APIServer::dump_config() {
#else
ESP_LOGCONFIG(TAG, " Noise encryption: NO");
#endif
#ifdef USE_API_OUTGOING_CONNECTION
this->outgoing_conn_.dump_config();
#endif
}
void APIServer::handle_disconnect(APIConnection *conn) {}
@@ -426,7 +504,16 @@ void APIServer::on_zwave_proxy_request(const ZWaveProxyRequest &msg) {
}
#endif
#if defined(USE_IR_RF) || defined(USE_RADIO_FREQUENCY)
#ifdef USE_SERIAL_PROXY_USB_IDENTITY
void APIServer::send_serial_proxy_identity(const SerialProxyIdentity &msg) {
for (auto &c : this->active_clients()) {
if (c->flags_.serial_proxy_identity_subscription)
c->send_serial_proxy_identity(msg);
}
}
#endif
#ifdef USE_IR_RF
void APIServer::send_infrared_rf_receive_event([[maybe_unused]] uint32_t device_id, uint32_t key,
const std::vector<int32_t> *timings) {
InfraredRFReceiveEvent resp{};
@@ -439,6 +526,7 @@ void APIServer::send_infrared_rf_receive_event([[maybe_unused]] uint32_t device_
for (auto &c : this->active_clients())
c->send_infrared_rf_receive_event(resp);
}
#endif
#ifdef USE_ALARM_CONTROL_PANEL
@@ -455,8 +543,9 @@ void APIServer::send_homeassistant_action(const HomeassistantActionRequest &call
// Home Assistant subscribes to actions shortly *after* authenticating, so actions
// fired right at connection time (on_client_connected, on_time_sync, ...) can
// arrive before the subscription and are lost - warn instead of failing silently.
ESP_LOGW(TAG, "Home Assistant %s '%s' dropped; %s",
call.is_event ? LOG_STR_LITERAL("event") : LOG_STR_LITERAL("action"), call.service.c_str(),
ESP_LOGW(TAG, "Home Assistant %s '%.*s' dropped; %s",
call.is_event ? LOG_STR_LITERAL("event") : LOG_STR_LITERAL("action"),
static_cast<int>(call.service.size()), call.service.empty() ? "" : call.service.c_str(),
this->is_connected() ? LOG_STR_LITERAL("client has not subscribed to actions (yet)")
: LOG_STR_LITERAL("no client connected"));
}
@@ -599,6 +688,8 @@ bool APIServer::update_noise_psk_(const SavedNoisePsk &new_psk, const LogString
if (!c->send_message(req)) {
API_LOG_MSG_DROPPED(TAG, "Disconnect request");
}
// Force it: a session from before the key was active must not survive
c->flags_.next_close = true;
}
});
}
@@ -700,6 +791,9 @@ void APIServer::on_shutdown() {
// Close the listening socket to prevent new connections
this->destroy_socket_();
#ifdef USE_API_OUTGOING_CONNECTION
this->outgoing_conn_.on_shutdown();
#endif
// Change batch delay to 5ms for quick flushing during shutdown
this->batch_delay_ = 5;
esphome/components/api/api_server.h
+60 -32
View File
@@ -5,16 +5,17 @@
#include "api_buffer.h"
// Must precede clients_ so APIConnection is complete for default_delete (libc++).
#include "api_connection.h"
#ifdef USE_API_NOISE
#if defined(USE_API_NOISE) || defined(USE_NOISE_SPARE_EPHEMERAL)
// Only present in the build when the noise component is loaded
#include "esphome/components/noise/noise.h"
#endif
#include "api_pb2.h"
#include "api_pb2_service.h"
#include "api_outgoing_connection.h"
#include "esphome/components/socket/socket.h"
#include "esphome/core/automation.h"
#include "esphome/core/component.h"
#include "esphome/core/controller.h"
#include "esphome/core/entity_includes.h"
#include "esphome/core/log.h"
#include "esphome/core/string_ref.h"
#ifdef USE_PROVISIONING
@@ -49,8 +50,7 @@ struct SavedNoisePsk {
bool load_saved_noise_psk(noise::psk_t &out);
#endif
class APIServer final : public Component,
public Controller
class APIServer final : public Component
#ifdef USE_CAMERA
,
public camera::CameraListener
@@ -91,61 +91,65 @@ class APIServer final : public Component,
void set_noise_psk(const uint8_t *psk) { this->noise_ctx_.set_psk(psk); }
noise::NoiseContext &get_noise_ctx() { return this->noise_ctx_; }
#endif // USE_API_NOISE
#ifdef USE_API_OUTGOING_CONNECTION
// Called by APIConnection when a client declares itself a dial-back target in its hello
void on_outgoing_target_client(APIConnection *conn);
#endif
void handle_disconnect(APIConnection *conn);
#ifdef USE_BINARY_SENSOR
void on_binary_sensor_update(binary_sensor::BinarySensor *obj) override;
void on_binary_sensor_update(binary_sensor::BinarySensor *obj);
#endif
#ifdef USE_COVER
void on_cover_update(cover::Cover *obj) override;
void on_cover_update(cover::Cover *obj);
#endif
#ifdef USE_FAN
void on_fan_update(fan::Fan *obj) override;
void on_fan_update(fan::Fan *obj);
#endif
#ifdef USE_LIGHT
void on_light_update(light::LightState *obj) override;
void on_light_update(light::LightState *obj);
#endif
#ifdef USE_SENSOR
void on_sensor_update(sensor::Sensor *obj) override;
void on_sensor_update(sensor::Sensor *obj);
#endif
#ifdef USE_SWITCH
void on_switch_update(switch_::Switch *obj) override;
void on_switch_update(switch_::Switch *obj);
#endif
#ifdef USE_TEXT_SENSOR
void on_text_sensor_update(text_sensor::TextSensor *obj) override;
void on_text_sensor_update(text_sensor::TextSensor *obj);
#endif
#ifdef USE_CLIMATE
void on_climate_update(climate::Climate *obj) override;
void on_climate_update(climate::Climate *obj);
#endif
#ifdef USE_NUMBER
void on_number_update(number::Number *obj) override;
void on_number_update(number::Number *obj);
#endif
#ifdef USE_DATETIME_DATE
void on_date_update(datetime::DateEntity *obj) override;
void on_date_update(datetime::DateEntity *obj);
#endif
#ifdef USE_DATETIME_TIME
void on_time_update(datetime::TimeEntity *obj) override;
void on_time_update(datetime::TimeEntity *obj);
#endif
#ifdef USE_DATETIME_DATETIME
void on_datetime_update(datetime::DateTimeEntity *obj) override;
void on_datetime_update(datetime::DateTimeEntity *obj);
#endif
#ifdef USE_TEXT
void on_text_update(text::Text *obj) override;
void on_text_update(text::Text *obj);
#endif
#ifdef USE_SELECT
void on_select_update(select::Select *obj) override;
void on_select_update(select::Select *obj);
#endif
#ifdef USE_LOCK
void on_lock_update(lock::Lock *obj) override;
void on_lock_update(lock::Lock *obj);
#endif
#ifdef USE_VALVE
void on_valve_update(valve::Valve *obj) override;
void on_valve_update(valve::Valve *obj);
#endif
#ifdef USE_MEDIA_PLAYER
void on_media_player_update(media_player::MediaPlayer *obj) override;
void on_media_player_update(media_player::MediaPlayer *obj);
#endif
#ifdef USE_WATER_HEATER
void on_water_heater_update(water_heater::WaterHeater *obj) override;
void on_water_heater_update(water_heater::WaterHeater *obj);
#endif
#ifdef USE_API_HOMEASSISTANT_SERVICES
void send_homeassistant_action(const HomeassistantActionRequest &call);
@@ -188,18 +192,22 @@ class APIServer final : public Component,
#endif
#ifdef USE_ALARM_CONTROL_PANEL
void on_alarm_control_panel_update(alarm_control_panel::AlarmControlPanel *obj) override;
void on_alarm_control_panel_update(alarm_control_panel::AlarmControlPanel *obj);
#endif
#ifdef USE_EVENT
void on_event(event::Event *obj) override;
void on_event(event::Event *obj);
#endif
#ifdef USE_UPDATE
void on_update(update::UpdateEntity *obj) override;
void on_update(update::UpdateEntity *obj);
#endif
#ifdef USE_ZWAVE_PROXY
void on_zwave_proxy_request(const ZWaveProxyRequest &msg);
#endif
#if defined(USE_IR_RF) || defined(USE_RADIO_FREQUENCY)
#ifdef USE_SERIAL_PROXY_USB_IDENTITY
/// Tell every subscribed client that a serial proxy port's identity changed
void send_serial_proxy_identity(const SerialProxyIdentity &msg);
#endif
#ifdef USE_IR_RF
void send_infrared_rf_receive_event(uint32_t device_id, uint32_t key, const std::vector<int32_t> *timings);
#endif
@@ -268,6 +276,16 @@ class APIServer final : public Component,
protected:
// Accept incoming socket connections. Only called when socket has pending connections.
void __attribute__((noinline)) accept_new_connections_();
/// Takes the socket into a new connection and starts it; callers must have
/// checked at_client_limit_() first
APIConnection *add_client_(std::unique_ptr<socket::Socket> sock);
bool at_client_limit_() const { return this->api_connection_count_ >= MAX_API_CONNECTIONS; }
#ifdef USE_API_OUTGOING_CONNECTION
// Returns the new connection, or nullptr (socket dropped) when at the limit
APIConnection *add_outgoing_client_(std::unique_ptr<socket::Socket> sock);
bool has_outgoing_target_client_() const { return this->outgoing_target_count_ != 0; }
friend class OutgoingConnectionManager;
#endif
// Remove a disconnected client by index. Swaps with the last populated slot and resets it.
void __attribute__((noinline)) remove_client_(uint8_t client_index);
@@ -308,6 +326,8 @@ class APIServer final : public Component,
delete this->socket_;
this->socket_ = nullptr;
}
/// Log the failure, drop the listen socket, and mark the component failed
/// unless this build can still dial out
void socket_failed_(const LogString *msg);
// Pointers and pointer-like types first (4 bytes each)
socket::ListenSocket *socket_{nullptr};
@@ -319,7 +339,7 @@ class APIServer final : public Component,
#endif
// 4-byte aligned types
uint32_t reboot_timeout_{300000};
uint32_t reboot_timeout_{900000}; // Keep in sync with DEFAULT_REBOOT_TIMEOUT in __init__.py
uint32_t last_connected_{0};
// Slots [0, api_connection_count_) are populated; trailing slots are always nullptr.
@@ -356,18 +376,23 @@ class APIServer final : public Component,
#endif
// Group smaller types together
uint16_t port_{6053};
uint16_t batch_delay_{100};
// Connection limits - these defaults will be overridden by config values
// from cv.SplitDefault in __init__.py which sets platform-specific defaults.
uint8_t listen_backlog_{4};
uint16_t port_{6053}; // Keep in sync with DEFAULT_PORT in __init__.py
uint16_t batch_delay_{100}; // Keep in sync with DEFAULT_BATCH_DELAY in __init__.py
uint8_t listen_backlog_{4}; // Keep in sync with DEFAULT_LISTEN_BACKLOG in __init__.py
bool shutting_down_ = false;
uint8_t api_connection_count_{0};
#ifdef USE_API_OUTGOING_CONNECTION
// Connected clients whose hello declared them a dial-back target
uint8_t outgoing_target_count_{0};
#endif
#if defined(USE_PROVISIONING) && defined(USE_API_NOISE)
// Index assigned by the provisioning manager for reporting this transport's state.
uint8_t provisioning_source_{0};
#endif
#ifdef USE_NOISE_SPARE_EPHEMERAL
void refill_spare_ephemeral_();
#endif
#ifdef USE_API_NOISE
noise::NoiseContext noise_ctx_;
#ifndef USE_API_NOISE_PSK_FROM_YAML
@@ -375,6 +400,9 @@ class APIServer final : public Component,
#endif
ESPPreferenceObject noise_pref_;
#endif // USE_API_NOISE
#ifdef USE_API_OUTGOING_CONNECTION
OutgoingConnectionManager outgoing_conn_;
#endif
};
extern APIServer *global_api_server; // NOLINT(cppcoreguidelines-avoid-non-const-global-variables)
esphome/components/api/api_wizard.cpp
+103
View File
@@ -0,0 +1,103 @@
#include "api_wizard.h"
#ifdef USE_API_WIZARD
#include <cstring>
#include "api_connection.h"
#include "api_pb2.h"
#include "api_server.h"
#include "esphome/core/log.h"
namespace esphome::api {
static const char *const TAG = "api.wizard";
uint8_t *wizard_encode_response(const void *self, uint8_t *__restrict__ pos PROTO_ENCODE_DEBUG_PARAM) {
const auto &msg = *static_cast<const DeviceWizardResponse *>(self);
if (msg.data_len == 0)
return pos;
pos = ProtoEncode::encode_field_raw(pos PROTO_ENCODE_DEBUG_ARG, 1, 2); // type 2: Length-delimited
pos = ProtoEncode::encode_varint_raw(pos PROTO_ENCODE_DEBUG_ARG, msg.data_len);
PROTO_ENCODE_CHECK_BOUNDS(pos, msg.data_len);
progmem_memcpy(pos, msg.data, msg.data_len);
return pos + msg.data_len;
}
#ifdef USE_API_WIZARD_INPUTS
static bool wizard_entity_id_valid(const char *entity_id, size_t length) {
return length > 0 && length < WIZARD_ENTITY_ID_BUFFER_SIZE && memchr(entity_id, '.', length) != nullptr;
}
const char *wizard_set_input(const WizardInputSetRequest &msg) {
if (!wizard_entity_id_valid(msg.entity_id.c_str(), msg.entity_id.size())) {
ESP_LOGW(TAG, "Ignoring an invalid entity id for wizard input");
return nullptr;
}
for (size_t i = 0; i < API_WIZARD_INPUT_COUNT; i++) {
// The table is in flash, which ESP8266 can only read through progmem_memcpy
WizardInputEntry entry;
progmem_memcpy(&entry, &API_WIZARD_INPUTS[i], sizeof(entry));
if (entry.key != msg.key)
continue;
memcpy(entry.entity_id, msg.entity_id.c_str(), msg.entity_id.size());
entry.entity_id[msg.entity_id.size()] = '\0';
return entry.entity_id;
}
ESP_LOGW(TAG, "Ignoring an entity id for an unknown wizard input");
return nullptr;
}
#endif // USE_API_WIZARD_INPUTS
bool APIConnection::send_device_wizard_response_() {
DeviceWizardResponse resp;
resp.data = API_WIZARD_DATA;
resp.data_len = API_WIZARD_DATA_SIZE;
// Not send_message: the data is in flash, so wizard_encode_response copies it out
return this->send_message_(DeviceWizardResponse::calc_size_msg(&resp), DeviceWizardResponse::MESSAGE_TYPE,
&wizard_encode_response, &resp);
}
void APIConnection::on_device_wizard_request() {
if (!this->send_device_wizard_response_()) {
this->on_fatal_error();
}
}
#ifdef USE_API_WIZARD_INPUTS
void APIConnection::on_wizard_input_set_request(const WizardInputSetRequest &msg) {
const char *entity_id = wizard_set_input(msg);
if (entity_id == nullptr)
return;
#ifdef USE_API_WIZARD_LINKED_INPUTS
// Entities subscribed to the buffer before it held an entity id, so every client needs to learn of it now
for (auto &client : this->parent_->active_clients()) {
client->resend_state_subscriptions(entity_id);
}
#endif
}
#endif
#ifdef USE_API_WIZARD_LINKED_INPUTS
void APIConnection::resend_state_subscriptions(const char *entity_id) {
if (!this->flags_.home_assistant_states)
return;
for (const auto &it : this->parent_->get_state_subs()) {
if (it.entity_id != entity_id)
continue;
SubscribeHomeAssistantStateResponse resp;
resp.entity_id = StringRef(it.entity_id);
resp.attribute = it.attribute != nullptr ? StringRef(it.attribute) : StringRef("");
resp.once = it.once;
if (!this->send_message(resp)) {
// Could not send now: send every subscription again from the loop
this->state_subs_at_ = 0;
return;
}
}
}
#endif
} // namespace esphome::api
#endif // USE_API_WIZARD
esphome/components/api/api_wizard.h
+62
View File
@@ -0,0 +1,62 @@
#pragma once
#include "esphome/core/defines.h"
#ifdef USE_API_WIZARD
#include <cstddef>
#include <cstdint>
#include "esphome/core/hal.h"
#include "proto.h"
#include "esphome/core/string_ref.h"
namespace esphome::api {
class WizardInputSetRequest;
/// Size of the buffer holding the entity id of a wizard input. Home Assistant entity ids are at most 255 bytes.
static constexpr size_t WIZARD_ENTITY_ID_BUFFER_SIZE = 256;
/// The wizard as zstd compressed JSON (see DeviceWizardResponse), built by the generated code
/// (components/api/wizard.py) and kept in flash. API_WIZARD_DATA_SIZE bytes long.
extern const uint8_t API_WIZARD_DATA[] PROGMEM;
/// Encodes a DeviceWizardResponse like the generated encoder would. The data is in flash, which ESP8266 can only read
/// with progmem_memcpy, so the generated encoder (a plain memcpy) cannot be used. Plain memcpy elsewhere.
uint8_t *wizard_encode_response(const void *self, uint8_t *pos PROTO_ENCODE_DEBUG_PARAM);
#ifdef USE_API_WIZARD_INPUTS
/// Where the entity id of an input is kept, found by the key the client uses for it.
struct WizardInputEntry {
uint32_t key; // FNV-1 hash of the ESPHome id of the input
char *entity_id; // RAM buffer of WIZARD_ENTITY_ID_BUFFER_SIZE bytes, shared with the homeassistant entity
};
/// The inputs of the wizard, in flash. API_WIZARD_INPUT_COUNT entries long.
extern const WizardInputEntry API_WIZARD_INPUTS[] PROGMEM;
/// Apply a WizardInputSetRequest: validate it and copy the entity id into the input's buffer. Nothing is stored
/// across restarts, so the client sends the choices again after every connection.
/// Returns the buffer, or nullptr when the request was ignored.
const char *wizard_set_input(const WizardInputSetRequest &msg);
#endif
#ifdef USE_API_WIZARD_STANDALONE_INPUTS
/// An input of the wizard that is not tied to an entity of the device. The entity ID the user picks is only ever read
/// by lambdas, for example `id(input).entity_id()`. It lives in the same RAM buffer a linked input uses.
class WizardInput {
public:
explicit WizardInput(const char *entity_id) : entity_id_(entity_id) {}
/// The Home Assistant entity ID, empty until the wizard sets one.
StringRef entity_id() const { return StringRef(this->entity_id_); }
bool has_entity_id() const { return this->entity_id_[0] != '\0'; }
protected:
const char *entity_id_;
};
#endif
} // namespace esphome::api
#endif // USE_API_WIZARD
esphome/components/api/homeassistant_service.h
+98 -169
View File
@@ -4,64 +4,40 @@
#ifdef USE_API
#ifdef USE_API_HOMEASSISTANT_SERVICES
#include <functional>
#include <string>
#include <type_traits>
#include <utility>
#include <vector>
#include "api_pb2.h"
#ifdef USE_API_HOMEASSISTANT_ACTION_RESPONSES_JSON
#include "esphome/components/json/json_util.h"
#endif
#include "esphome/core/automation.h"
#include "esphome/core/helpers.h"
#include "esphome/core/progmem.h"
#include "esphome/core/string_ref.h"
namespace esphome::api {
template<typename... X> class TemplatableStringValue : public TemplatableValue<std::string, X...> {
// Verify that const char* uses the base class STATIC_STRING optimization (no heap allocation)
// rather than being wrapped in a lambda. The base class constructor for const char* is more
// specialized than the templated constructor here, so it should be selected.
static_assert(std::is_constructible_v<TemplatableValue<std::string, X...>, const char *>,
"Base class must have const char* constructor for STATIC_STRING optimization");
// Converts a lambda result to the string sent to Home Assistant
template<typename T>
requires(!std::is_pointer_v<std::remove_cvref_t<T>>) std::string field_to_string(T &&val) {
return to_string(std::forward<T>(val)); // NOLINT
}
inline std::string field_to_string(const char *val) { return val ? std::string(val) : std::string(); }
inline std::string field_to_string(std::string val) { return val; }
inline std::string field_to_string(StringRef val) { return val.str(); }
private:
// Helper to convert value to string - handles the case where value is already a string
template<typename T> static std::string value_to_string(T &&val) {
return to_string(std::forward<T>(val)); // NOLINT
/// A key and value from codegen; on ESP8266 the table and its strings are in flash.
/// The value is the constant `value`, or the result of `fn` when it is set.
template<typename... Ts> struct HomeAssistantField {
const char *key;
const char *value;
std::string (*fn)(const Ts &...);
template<typename F> static constexpr HomeAssistantField from_lambda(const char *key, F /*lambda*/) {
return {key, nullptr, &call_lambda<F>};
}
// Overloads for string types - needed because std::to_string doesn't support them
static std::string value_to_string(char *val) {
return val ? std::string(val) : std::string();
} // For lambdas returning char* (e.g., itoa)
static std::string value_to_string(const char *val) { return std::string(val); } // For lambdas returning .c_str()
static std::string value_to_string(const std::string &val) { return val; }
static std::string value_to_string(std::string &&val) { return std::move(val); }
static std::string value_to_string(const StringRef &val) { return val.str(); }
static std::string value_to_string(StringRef &&val) { return val.str(); }
public:
TemplatableStringValue() : TemplatableValue<std::string, X...>() {}
template<typename F, enable_if_t<!is_invocable<F, X...>::value, int> = 0>
TemplatableStringValue(F value) : TemplatableValue<std::string, X...>(value) {}
template<typename F, enable_if_t<is_invocable<F, X...>::value, int> = 0>
TemplatableStringValue(F f)
: TemplatableValue<std::string, X...>([f](X... x) -> std::string { return value_to_string(f(x...)); }) {}
};
template<typename... Ts> class TemplatableKeyValuePair {
public:
// Default constructor needed for FixedVector::emplace_back()
TemplatableKeyValuePair() = default;
// Keys are always string literals from YAML dictionary keys (e.g., "code", "event")
// and never templatable values or lambdas. Only the value parameter can be a lambda/template.
// Using const char* avoids std::string heap allocation - keys remain in flash.
template<typename T> TemplatableKeyValuePair(const char *key, T value) : key(key), value(value) {}
const char *key{nullptr};
TemplatableStringValue<Ts...> value;
template<typename F> static std::string call_lambda(const Ts &...x) { return field_to_string(F{}(x...)); }
};
#ifdef USE_API_HOMEASSISTANT_ACTION_RESPONSES
@@ -106,45 +82,20 @@ template<typename... Ts> using ActionResponseCallback = std::function<void(const
template<typename... Ts> class HomeAssistantServiceCallAction final : public Action<Ts...> {
public:
explicit HomeAssistantServiceCallAction(APIServer *parent, bool is_event) : parent_(parent) {
using Field = HomeAssistantField<Ts...>;
/// `fields` is a codegen table: the action or event name (no key), then the data, data_template
/// and variables entries.
HomeAssistantServiceCallAction(APIServer *parent, bool is_event, const Field *fields, uint8_t data_count,
uint8_t data_template_count, uint8_t variables_count)
: parent_(parent),
fields_(fields),
data_count_(data_count),
data_template_count_(data_template_count),
variables_count_(variables_count) {
this->flags_.is_event = is_event;
}
template<typename T> void set_service(T service) { this->service_ = service; }
// Initialize FixedVector members - called from Python codegen with compile-time known sizes.
// Must be called before any add_* methods; capacity must match the number of subsequent add_* calls.
void init_data(size_t count) { this->data_.init(count); }
void init_data_template(size_t count) { this->data_template_.init(count); }
void init_variables(size_t count) { this->variables_.init(count); }
// Keys are always string literals from the Python code generation (e.g., cg.add(var.add_data("tag_id", templ))).
// The value parameter can be a lambda/template, but keys are never templatable.
// Using const char* for keys avoids std::string heap allocation - keys remain in flash.
template<typename V> void add_data(const char *key, V &&value) {
this->add_kv_(this->data_, key, std::forward<V>(value));
}
template<typename V> void add_data_template(const char *key, V &&value) {
this->add_kv_(this->data_template_, key, std::forward<V>(value));
}
template<typename V> void add_variable(const char *key, V &&value) {
this->add_kv_(this->variables_, key, std::forward<V>(value));
}
#ifdef USE_ESP8266
// On ESP8266, ESPHOME_F() returns __FlashStringHelper* (PROGMEM pointer).
// Store as const char* — populate_service_map copies from PROGMEM at play() time.
template<typename V> void add_data(const __FlashStringHelper *key, V &&value) {
this->add_kv_(this->data_, reinterpret_cast<const char *>(key), std::forward<V>(value));
}
template<typename V> void add_data_template(const __FlashStringHelper *key, V &&value) {
this->add_kv_(this->data_template_, reinterpret_cast<const char *>(key), std::forward<V>(value));
}
template<typename V> void add_variable(const __FlashStringHelper *key, V &&value) {
this->add_kv_(this->variables_, reinterpret_cast<const char *>(key), std::forward<V>(value));
}
#endif
#ifdef USE_API_HOMEASSISTANT_ACTION_RESPONSES
template<typename T> void set_response_template(T response_template) {
this->response_template_ = response_template;
@@ -162,19 +113,61 @@ template<typename... Ts> class HomeAssistantServiceCallAction final : public Act
#endif // USE_API_HOMEASSISTANT_ACTION_RESPONSES
void play(const Ts &...x) override {
const Field *fields = this->fields_;
const size_t total = 1 + this->data_count_ + this->data_template_count_ + this->variables_count_;
// Lambda results, and on ESP8266 the RAM copies of the flash strings, must live until the send
size_t lambda_count = 0;
#ifdef USE_ESP8266
size_t flash_len = 0;
#endif
for (size_t i = 0; i < total; i++) {
lambda_count += fields[i].fn != nullptr;
#ifdef USE_ESP8266
if (fields[i].fn == nullptr)
flash_len += ESPHOME_strlen_P(fields[i].value);
if (fields[i].key != nullptr)
flash_len += ESPHOME_strlen_P(fields[i].key);
#endif
}
FixedVector<std::string> results;
results.init(lambda_count);
#ifdef USE_ESP8266
SmallBufferWithHeapFallback<128, char> flash_copy(flash_len);
char *cursor = flash_copy.get();
#endif
auto string_ref = [&](const char *str) {
#ifdef USE_ESP8266
size_t len = ESPHOME_strlen_P(str);
memcpy_P(cursor, str, len);
StringRef ref(cursor, len);
cursor += len;
return ref;
#else
return StringRef(str);
#endif
};
auto value_ref = [&](const Field &field) {
if (field.fn == nullptr)
return string_ref(field.value);
results.push_back(field.fn(x...));
return StringRef(results.back());
};
auto fill = [&](FixedVector<HomeassistantServiceMap> &dest, uint8_t count) {
dest.init(count);
for (uint8_t i = 0; i < count; i++, fields++) {
auto &kv = dest.emplace_back();
kv.key = string_ref(fields->key);
kv.value = value_ref(*fields);
}
};
HomeassistantActionRequest resp;
std::string service_value = this->service_.value(x...);
resp.service = StringRef(service_value);
resp.service = value_ref(*fields++);
resp.is_event = this->flags_.is_event;
// Local storage for lambda-evaluated strings - lives until after send
FixedVector<std::string> data_storage;
FixedVector<std::string> data_template_storage;
FixedVector<std::string> variables_storage;
this->populate_service_map(resp.data, this->data_, data_storage, x...);
this->populate_service_map(resp.data_template, this->data_template_, data_template_storage, x...);
this->populate_service_map(resp.variables, this->variables_, variables_storage, x...);
fill(resp.data, this->data_count_);
fill(resp.data_template, this->data_template_count_);
fill(resp.variables, this->variables_count_);
#ifdef USE_API_HOMEASSISTANT_ACTION_RESPONSES
#ifdef USE_API_HOMEASSISTANT_ACTION_RESPONSES_JSON
@@ -223,90 +216,26 @@ template<typename... Ts> class HomeAssistantServiceCallAction final : public Act
}
protected:
// Helper to add key-value pairs to FixedVectors
// Keys are always string literals (const char*), values can be lambdas/templates
template<typename V> void add_kv_(FixedVector<TemplatableKeyValuePair<Ts...>> &vec, const char *key, V &&value) {
auto &kv = vec.emplace_back();
kv.key = key;
kv.value = std::forward<V>(value);
}
template<typename VectorType, typename SourceType>
static void populate_service_map(VectorType &dest, SourceType &source, FixedVector<std::string> &value_storage,
Ts... x) {
dest.init(source.size());
#ifdef USE_ESP8266
// On ESP8266, all static strings from codegen are FLASH_STRING (PROGMEM),
// so is_static_string() is always false — the zero-copy STATIC_STRING fast
// path from the non-ESP8266 branch cannot trigger. We copy all keys and
// values unconditionally: keys via _P functions (may be in PROGMEM), values
// via value() which handles FLASH_STRING internally.
value_storage.init(source.size() * 2);
for (auto &it : source) {
auto &kv = dest.emplace_back();
// Key: copy from possible PROGMEM
{
size_t key_len = strlen_P(it.key);
value_storage.push_back(std::string(key_len, '\0'));
memcpy_P(value_storage.back().data(), it.key, key_len);
kv.key = StringRef(value_storage.back());
}
// Value: value() handles FLASH_STRING via _P functions internally
value_storage.push_back(it.value.value(x...));
kv.value = StringRef(value_storage.back());
}
#else
// On non-ESP8266, strings are directly readable from flash-mapped memory.
// Count non-static strings to allocate exact storage needed.
size_t lambda_count = 0;
for (const auto &it : source) {
if (!it.value.is_static_string()) {
lambda_count++;
}
}
value_storage.init(lambda_count);
for (auto &it : source) {
auto &kv = dest.emplace_back();
kv.key = StringRef(it.key);
if (it.value.is_static_string()) {
// Static string — pointer directly readable, zero allocation
kv.value = StringRef(it.value.get_static_string());
} else {
// Lambda — evaluate and store result
value_storage.push_back(it.value.value(x...));
kv.value = StringRef(value_storage.back());
}
}
#endif
}
APIServer *parent_;
TemplatableStringValue<Ts...> service_{};
FixedVector<TemplatableKeyValuePair<Ts...>> data_;
FixedVector<TemplatableKeyValuePair<Ts...>> data_template_;
FixedVector<TemplatableKeyValuePair<Ts...>> variables_;
#ifdef USE_API_HOMEASSISTANT_ACTION_RESPONSES
#ifdef USE_API_HOMEASSISTANT_ACTION_RESPONSES_JSON
TemplatableStringValue<Ts...> response_template_{""};
Trigger<JsonObjectConst, Ts...> success_trigger_with_response_;
#endif // USE_API_HOMEASSISTANT_ACTION_RESPONSES_JSON
Trigger<Ts...> success_trigger_;
Trigger<std::string, Ts...> error_trigger_;
#endif // USE_API_HOMEASSISTANT_ACTION_RESPONSES
const Field *fields_;
uint8_t data_count_;
uint8_t data_template_count_;
uint8_t variables_count_;
struct Flags {
uint8_t is_event : 1;
uint8_t wants_status : 1;
uint8_t wants_response : 1;
uint8_t has_response_template : 1;
uint8_t reserved : 5;
uint8_t reserved : 4;
} flags_{0};
#ifdef USE_API_HOMEASSISTANT_ACTION_RESPONSES
#ifdef USE_API_HOMEASSISTANT_ACTION_RESPONSES_JSON
TemplatableValue<std::string, Ts...> response_template_{};
Trigger<JsonObjectConst, Ts...> success_trigger_with_response_;
#endif // USE_API_HOMEASSISTANT_ACTION_RESPONSES_JSON
Trigger<Ts...> success_trigger_;
Trigger<std::string, Ts...> error_trigger_;
#endif // USE_API_HOMEASSISTANT_ACTION_RESPONSES
};
} // namespace esphome::api
esphome/components/api/proto.cpp
+94 -101
View File
@@ -119,7 +119,7 @@ uint32_t ProtoDecodableMessage::count_repeated_field(const uint8_t *buffer, size
}
// Single-pass encode for repeated submessage elements (non-template core).
// Writes field tag, reserves 1 byte for length varint, encodes the submessage body,
// Reserves 1 byte for length varint, encodes the submessage body,
// then backpatches the actual length. For the common case (body < 128 bytes), this is
// just a single byte write with no memmove — all current repeated submessage types
// (BLE advertisements at ~47B, GATT descriptors at ~24B, service args, etc.) take
@@ -143,51 +143,34 @@ uint32_t ProtoDecodableMessage::count_repeated_field(const uint8_t *buffer, size
//
// After writing 2-byte varint at len_pos:
// [tag][v1][v2][body ..... body]
// ^-- pos_ = element end, within buffer
void ProtoWriteBuffer::encode_sub_message(uint32_t field_id, const void *value,
uint8_t *(*encode_fn)(const void *,
ProtoWriteBuffer &PROTO_ENCODE_DEBUG_PARAM)) {
this->encode_field_raw(field_id, 2);
// Reserve 1 byte for length varint (optimistic: submessage < 128 bytes)
uint8_t *len_pos = this->pos_;
this->debug_check_bounds_(1);
this->pos_++;
uint8_t *body_start = this->pos_;
this->pos_ = encode_fn(value, *this PROTO_ENCODE_DEBUG_INIT(this->buffer_));
uint32_t body_size = static_cast<uint32_t>(this->pos_ - body_start);
if (body_size < 128) [[likely]] {
// ^-- returned cursor = element end, within buffer
uint8_t *ProtoEncode::encode_sub_message_body(uint8_t *__restrict__ pos PROTO_ENCODE_DEBUG_PARAM, const void *value,
ProtoEncodeFn encode_fn) {
// Reserve 1 byte for the length varint (optimistic: submessage < 128 bytes)
uint8_t *len_pos = pos;
PROTO_ENCODE_CHECK_BOUNDS(pos, 1);
uint8_t *body_start = pos + 1;
uint8_t *after_body = encode_fn(value, body_start PROTO_ENCODE_DEBUG_ARG);
uint32_t body_size = static_cast<uint32_t>(after_body - body_start);
if (body_size < VARINT_MAX_1_BYTE) [[likely]] {
// Common case: 1-byte varint, just backpatch
*len_pos = static_cast<uint8_t>(body_size);
return;
return after_body;
}
// Compute extra bytes needed for varint beyond the 1 already reserved
// Shift the body forward to make room for the extra length varint bytes
uint8_t extra = ProtoSize::varint(body_size) - 1;
// Shift body forward to make room for the extra varint bytes
this->debug_check_bounds_(extra);
PROTO_ENCODE_CHECK_BOUNDS(after_body, extra);
std::memmove(body_start + extra, body_start, body_size);
uint8_t *end = this->pos_ + extra;
// Write the full varint at len_pos
this->pos_ = len_pos;
this->encode_varint_raw(body_size);
this->pos_ = end;
(void) encode_varint_raw_loop(len_pos PROTO_ENCODE_DEBUG_ARG, body_size);
return after_body + extra;
}
// Non-template core for encode_optional_sub_message.
void ProtoWriteBuffer::encode_optional_sub_message(uint32_t field_id, uint32_t nested_size, const void *value,
uint8_t *(*encode_fn)(const void *,
ProtoWriteBuffer &PROTO_ENCODE_DEBUG_PARAM)) {
if (nested_size == 0)
return;
this->encode_field_raw(field_id, 2);
this->encode_varint_raw(nested_size);
#ifdef ESPHOME_DEBUG_API
uint8_t *start = this->pos_;
this->pos_ = encode_fn(value, *this PROTO_ENCODE_DEBUG_INIT(this->buffer_));
if (static_cast<uint32_t>(this->pos_ - start) != nested_size)
this->debug_check_encode_size_(field_id, nested_size, this->pos_ - start);
#else
this->pos_ = encode_fn(value, *this PROTO_ENCODE_DEBUG_INIT(this->buffer_));
#endif
uint8_t *ProtoEncode::encode_sized_sub_message_body(uint8_t *__restrict__ pos PROTO_ENCODE_DEBUG_PARAM,
uint32_t nested_size, const void *value, ProtoEncodeFn encode_fn) {
pos = encode_varint_raw(pos PROTO_ENCODE_DEBUG_ARG, nested_size);
return encode_fn(value, pos PROTO_ENCODE_DEBUG_ARG);
}
#ifdef ESPHOME_DEBUG_API
@@ -195,6 +178,20 @@ void proto_check_bounds_failed(const uint8_t *pos, size_t bytes, const uint8_t *
ESP_LOGE(TAG, "Proto encode bounds check failed in %s: need %zu bytes, %td available", caller, bytes, end - pos);
abort();
}
void proto_check_encode_end(const uint8_t *end, const uint8_t *expected) {
if (end == expected)
return;
ESP_LOGE(TAG, "Proto encode ended %td bytes off the calculated size", end - expected);
abort();
}
void proto_check_sub_message_size(uint32_t field_id, uint32_t expected, const uint8_t *len_pos, const uint8_t *end) {
ptrdiff_t actual = end - (len_pos + ProtoSize::varint(expected));
if (actual == static_cast<ptrdiff_t>(expected))
return;
ESP_LOGE(TAG, "encode_message: size mismatch for field %" PRIu32 ": calculated=%" PRIu32 " actual=%td", field_id,
expected, actual);
abort();
}
void ProtoWriteBuffer::debug_check_bounds_(size_t bytes, const char *caller) {
if (this->pos_ + bytes > this->buffer_->data() + this->buffer_->size()) {
ESP_LOGE(TAG, "ProtoWriteBuffer bounds check failed in %s: bytes=%zu offset=%td buf_size=%zu", caller, bytes,
@@ -202,85 +199,81 @@ void ProtoWriteBuffer::debug_check_bounds_(size_t bytes, const char *caller) {
abort();
}
}
void ProtoWriteBuffer::debug_check_encode_size_(uint32_t field_id, uint32_t expected, ptrdiff_t actual) {
ESP_LOGE(TAG, "encode_message: size mismatch for field %" PRIu32 ": calculated=%" PRIu32 " actual=%td", field_id,
expected, actual);
abort();
}
#endif
void ProtoDecodableMessage::decode(const uint8_t *buffer, size_t length) {
void ProtoDecodableMessage::decode_fields(void *msg, const uint8_t *buffer, size_t length, DecodeFieldFn field) {
const uint8_t *ptr = buffer;
const uint8_t *end = buffer + length;
while (ptr < end) {
// Parse field header - ptr < end guarantees len >= 1
// Single-byte varints dominate, so that case advances the cursor inline.
auto read_varint = [&](proto_varint_value_t &value) ESPHOME_ALWAYS_INLINE {
if (ptr == end)
return false;
if (*ptr < 0x80) [[likely]] {
value = *ptr++;
return true;
}
auto res = ProtoVarInt::parse_non_empty(ptr, end - ptr);
if (!res.has_value()) {
if (!res.has_value())
return false;
value = res.value;
ptr += res.consumed;
return true;
};
while (ptr < end) {
proto_varint_value_t tag_value;
if (!read_varint(tag_value)) {
ESP_LOGV(TAG, "Invalid field start at offset %ld", (long) (ptr - buffer));
return;
}
uint32_t tag = static_cast<uint32_t>(res.value);
uint32_t tag = static_cast<uint32_t>(tag_value);
uint32_t field_type = tag & WIRE_TYPE_MASK;
uint32_t field_id = tag >> 3;
ptr += res.consumed;
// Length-delimited fields move this past the length prefix
const uint8_t *data = ptr;
proto_varint_value_t scalar;
switch (field_type) {
case WIRE_TYPE_VARINT: { // VarInt
res = ProtoVarInt::parse(ptr, end - ptr);
if (!res.has_value()) {
ESP_LOGV(TAG, "Invalid VarInt at offset %ld", (long) (ptr - buffer));
return;
}
if (!this->decode_varint(field_id, res.value)) {
ESP_LOGV(TAG, "Cannot decode VarInt field %" PRIu32 " with value %" PRIu64 "!", field_id,
static_cast<uint64_t>(res.value));
}
ptr += res.consumed;
break;
}
case WIRE_TYPE_LENGTH_DELIMITED: { // Length-delimited
res = ProtoVarInt::parse(ptr, end - ptr);
if (!res.has_value()) {
ESP_LOGV(TAG, "Invalid Length Delimited at offset %ld", (long) (ptr - buffer));
return;
}
uint32_t field_length = static_cast<uint32_t>(res.value);
ptr += res.consumed;
if (field_length > static_cast<size_t>(end - ptr)) {
ESP_LOGV(TAG, "Out-of-bounds Length Delimited at offset %ld", (long) (ptr - buffer));
return;
}
if (!this->decode_length(field_id, ProtoLengthDelimited(ptr, field_length))) {
ESP_LOGV(TAG, "Cannot decode Length Delimited field %" PRIu32 "!", field_id);
}
ptr += field_length;
break;
}
case WIRE_TYPE_FIXED32: { // 32-bit
if (end - ptr < 4) {
ESP_LOGV(TAG, "Out-of-bounds Fixed32-bit at offset %ld", (long) (ptr - buffer));
return;
}
uint32_t val;
#if __BYTE_ORDER__ == __ORDER_LITTLE_ENDIAN__
// Protobuf fixed32 is little-endian — direct load on LE platforms
memcpy(&val, ptr, 4);
#else
val = encode_uint32(ptr[3], ptr[2], ptr[1], ptr[0]);
#endif
if (!this->decode_32bit(field_id, Proto32Bit(val))) {
ESP_LOGV(TAG, "Cannot decode 32-bit field %" PRIu32 " with value %" PRIu32 "!", field_id, val);
}
ptr += 4;
break;
}
default:
ESP_LOGV(TAG, "Invalid field type %" PRIu32 " at offset %ld", field_type, (long) (ptr - buffer));
if (field_type == WIRE_TYPE_VARINT) [[likely]] {
if (!read_varint(scalar)) {
ESP_LOGV(TAG, "Invalid VarInt at offset %ld", (long) (ptr - buffer));
return;
}
} else {
switch (field_type) {
case WIRE_TYPE_LENGTH_DELIMITED: {
proto_varint_value_t length_value;
if (!read_varint(length_value)) {
ESP_LOGV(TAG, "Invalid Length Delimited at offset %ld", (long) (ptr - buffer));
return;
}
uint32_t field_length = static_cast<uint32_t>(length_value);
if (field_length > static_cast<size_t>(end - ptr)) {
ESP_LOGV(TAG, "Out-of-bounds Length Delimited at offset %ld", (long) (ptr - buffer));
return;
}
data = ptr;
scalar = field_length;
ptr += field_length;
break;
}
case WIRE_TYPE_FIXED32: {
if (end - ptr < 4) {
ESP_LOGV(TAG, "Out-of-bounds Fixed32-bit at offset %ld", (long) (ptr - buffer));
return;
}
// Byte loads instead of memcpy: ESP-IDF passes -fno-builtin-memcpy, which made this a call
scalar = encode_uint32(ptr[3], ptr[2], ptr[1], ptr[0]);
ptr += 4;
break;
}
default:
ESP_LOGV(TAG, "Invalid field type %" PRIu32 " at offset %ld", field_type, (long) (ptr - buffer));
return;
}
}
field(msg, tag, data, scalar);
}
}
esphome/components/api/proto.h
+279 -214
View File
@@ -10,6 +10,7 @@
#include <cassert>
#include <cstring>
#include <type_traits>
#include <vector>
#ifdef ESPHOME_LOG_HAS_VERY_VERBOSE
@@ -55,22 +56,25 @@ inline constexpr int64_t decode_zigzag64(uint64_t value) {
return (value & 1) ? static_cast<int64_t>(~(value >> 1)) : static_cast<int64_t>(value >> 1);
}
/// Count number of varints in a packed buffer
inline uint16_t count_packed_varints(const uint8_t *data, size_t len) {
uint16_t count = 0;
while (len > 0) {
// Skip varint bytes until we find one without continuation bit
while (len > 0 && (*data & 0x80)) {
data++;
len--;
}
if (len > 0) {
data++;
len--;
count++;
/// Count varints in a packed buffer: len minus bytes with the continuation bit, summed a word at a time.
/// Word is a template parameter so tests can cover the 32-bit path on a 64-bit host.
template<typename Word = size_t> inline uint16_t count_packed_varints(const uint8_t *data, size_t len) {
constexpr size_t word_size = sizeof(Word);
constexpr Word lane_ones = ~Word{0} / 0xFF; // 0x01..01
const uint8_t *end = data + len;
size_t continuations = 0;
while (data != end) {
// Unaligned word loads fault on Xtensa
if ((reinterpret_cast<uintptr_t>(data) & (word_size - 1)) == 0 && static_cast<size_t>(end - data) >= word_size) {
Word word;
memcpy(&word, __builtin_assume_aligned(data, word_size), word_size);
continuations += (((word >> 7) & lane_ones) * lane_ones) >> (word_size * 8 - 8);
data += word_size;
} else {
continuations += *data++ >> 7;
}
}
return count;
return static_cast<uint16_t>(len - continuations);
}
/// Encode a varint directly into a pre-allocated buffer.
@@ -170,40 +174,43 @@ class ProtoVarInt {
class ProtoMessage;
class ProtoSize;
class ProtoLengthDelimited {
/// Case label for decode_field(): the wire tag of a field, so a field that arrives with another wire
/// type matches no case.
constexpr uint32_t proto_tag(uint32_t field_id, uint32_t wire_type) { return (field_id << 3) | wire_type; }
/// One decoded field: the payload pointer and a scalar holding the varint or fixed32 value, or the
/// length of a length-delimited field. The wire type in the tag says which applies; accessors do not check.
class ProtoFieldValue {
public:
explicit ProtoLengthDelimited(const uint8_t *value, size_t length) : value_(value), length_(length) {}
std::string as_string() const { return std::string(reinterpret_cast<const char *>(this->value_), this->length_); }
ProtoFieldValue(const uint8_t *data, proto_varint_value_t scalar) : data_(data), scalar_(scalar) {}
// Direct access to raw data without string allocation
const uint8_t *data() const { return this->value_; }
size_t size() const { return this->length_; }
proto_varint_value_t as_varint() const { return this->scalar_; }
// A bool is sent as 0 or 1, so the low word is enough and saves a second compare with 64 bit varints
bool as_bool() const { return static_cast<uint32_t>(this->scalar_) != 0; }
/// Decode the length-delimited data into a message instance.
// Length-delimited accessors
const uint8_t *data() const { return this->data_; }
size_t size() const { return static_cast<size_t>(this->scalar_); }
std::string as_string() const { return std::string(reinterpret_cast<const char *>(this->data_), this->size()); }
/// Decode the length-delimited payload into a message instance.
/// Template preserves concrete type so decode() resolves statically.
template<typename T> void decode_to_message(T &msg) const;
template<typename T> void decode_to_message(T &msg) const { msg.decode(this->data_, this->size()); }
protected:
const uint8_t *const value_;
const size_t length_;
};
class Proto32Bit {
public:
explicit Proto32Bit(uint32_t value) : value_(value) {}
uint32_t as_fixed32() const { return this->value_; }
int32_t as_sfixed32() const { return static_cast<int32_t>(this->value_); }
// Fixed32 accessors
uint32_t as_fixed32() const { return static_cast<uint32_t>(this->scalar_); }
int32_t as_sfixed32() const { return static_cast<int32_t>(this->as_fixed32()); }
float as_float() const {
union {
uint32_t raw;
float value;
} s{};
s.raw = this->value_;
s.raw = this->as_fixed32();
return s.value;
}
protected:
const uint32_t value_;
private:
const uint8_t *data_;
proto_varint_value_t scalar_;
};
// NOTE: Proto64Bit class removed - wire type 1 (64-bit fixed) not supported
@@ -221,6 +228,11 @@ class Proto32Bit {
proto_check_bounds_failed(pos, n, proto_debug_end_, __builtin_FUNCTION()); \
} while (0)
void proto_check_bounds_failed(const uint8_t *pos, size_t bytes, const uint8_t *end, const char *caller);
/// Aborts unless an encode body ended exactly where calculate_size() promised. A plain check rather than
/// assert(), so NDEBUG cannot switch it off.
void proto_check_encode_end(const uint8_t *end, const uint8_t *expected);
/// Aborts unless a sized sub-message (length prefix at len_pos) ended where its calculated size said.
void proto_check_sub_message_size(uint32_t field_id, uint32_t expected, const uint8_t *len_pos, const uint8_t *end);
#else
#define PROTO_ENCODE_DEBUG_PARAM
#define PROTO_ENCODE_DEBUG_ARG
@@ -252,22 +264,7 @@ class ProtoWriteBuffer {
*
* Following https://protobuf.dev/programming-guides/encoding/#structure
*/
void encode_field_raw(uint32_t field_id, uint32_t type) { this->encode_varint_raw((field_id << 3) | type); }
/// Single-pass encode for repeated submessage elements.
/// Thin template wrapper; all buffer work is in the non-template core.
template<typename T> void encode_sub_message(uint32_t field_id, const T &value);
/// Encode an optional singular submessage field — skips if empty.
/// Thin template wrapper; all buffer work is in the non-template core.
template<typename T> void encode_optional_sub_message(uint32_t field_id, const T &value);
// NOLINTBEGIN(readability-identifier-naming)
// Non-template core for encode_sub_message — backpatch approach.
void encode_sub_message(uint32_t field_id, const void *value,
uint8_t *(*encode_fn)(const void *, ProtoWriteBuffer &PROTO_ENCODE_DEBUG_PARAM));
// Non-template core for encode_optional_sub_message.
void encode_optional_sub_message(uint32_t field_id, uint32_t nested_size, const void *value,
uint8_t *(*encode_fn)(const void *, ProtoWriteBuffer &PROTO_ENCODE_DEBUG_PARAM));
// NOLINTEND(readability-identifier-naming)
void encode_field_raw(uint32_t field_id, uint32_t type) { this->encode_varint_raw(proto_tag(field_id, type)); }
APIBuffer *get_buffer() const { return buffer_; }
uint8_t *get_pos() const { return pos_; }
void set_pos(uint8_t *pos) { pos_ = pos; }
@@ -278,7 +275,6 @@ class ProtoWriteBuffer {
#ifdef ESPHOME_DEBUG_API
void debug_check_bounds_(size_t bytes, const char *caller = __builtin_FUNCTION());
void debug_check_encode_size_(uint32_t field_id, uint32_t expected, ptrdiff_t actual);
#else
void debug_check_bounds_([[maybe_unused]] size_t bytes) {}
#endif
@@ -287,19 +283,34 @@ class ProtoWriteBuffer {
uint8_t *pos_;
};
// A four byte unaligned store is a memcpy call on ESP-IDF (-fno-builtin-memcpy) and on ARM cores without
// unaligned access (Cortex-M0+, ARM9), so those targets share one outlined byte store helper per fixed32
// field. Elsewhere the write inlines to a single store, or on ESP8266 to a few stores that measured
// faster than a call, so it stays inline.
#if defined(USE_ESP32) || (defined(__arm__) && !defined(__ARM_FEATURE_UNALIGNED))
#define PROTO_OUTLINE_FOR_SIZE __attribute__((noinline))
#define PROTO_FIXED32_BYTE_STORES true
#else
#define PROTO_OUTLINE_FOR_SIZE inline
#define PROTO_FIXED32_BYTE_STORES false
#endif
// Varint encoding thresholds — used by both proto_encode_* free functions and ProtoSize.
constexpr uint32_t VARINT_MAX_1_BYTE = 1 << 7; // 128
constexpr uint32_t VARINT_MAX_2_BYTE = 1 << 14; // 16384
/// Static encode helpers for generated encode() functions.
/// Generated code hoists buffer.pos_ into a local uint8_t *__restrict__ pos,
/// then calls these methods which take pos by reference. No struct, no overhead.
/// For sub-messages, pos is synced back to buffer before the call and reloaded after.
/// Generated encode body: writes the fields at pos, returns the cursor past them.
using ProtoEncodeFn = uint8_t *(*) (const void *, uint8_t *PROTO_ENCODE_DEBUG_PARAM);
/// Static encode helpers for the generated encode bodies. Each takes the write cursor by value and
/// returns it advanced, so outlined calls at -Os chain through the return register instead of a
/// stack slot. Helpers without a _force suffix skip fields holding the proto3 default.
class ProtoEncode {
public:
/// Write a multi-byte varint directly through a pos pointer.
template<typename T>
static inline void encode_varint_raw_loop(uint8_t *__restrict__ &pos PROTO_ENCODE_DEBUG_PARAM, T value) {
[[nodiscard]] static inline uint8_t *encode_varint_raw_loop(uint8_t *__restrict__ pos PROTO_ENCODE_DEBUG_PARAM,
T value) {
do {
PROTO_ENCODE_CHECK_BOUNDS(pos, 1);
*pos++ = static_cast<uint8_t>(value | 0x80);
@@ -307,48 +318,49 @@ class ProtoEncode {
} while (value > 0x7F);
PROTO_ENCODE_CHECK_BOUNDS(pos, 1);
*pos++ = static_cast<uint8_t>(value);
return pos;
}
static inline void ESPHOME_ALWAYS_INLINE encode_varint_raw(uint8_t *__restrict__ &pos PROTO_ENCODE_DEBUG_PARAM,
uint32_t value) {
[[nodiscard]] static inline uint8_t *ESPHOME_ALWAYS_INLINE
encode_varint_raw(uint8_t *__restrict__ pos PROTO_ENCODE_DEBUG_PARAM, uint32_t value) {
if (value < VARINT_MAX_1_BYTE) [[likely]] {
PROTO_ENCODE_CHECK_BOUNDS(pos, 1);
*pos++ = static_cast<uint8_t>(value);
return;
return pos;
}
encode_varint_raw_loop(pos PROTO_ENCODE_DEBUG_ARG, value);
return encode_varint_raw_loop(pos PROTO_ENCODE_DEBUG_ARG, value);
}
/// Encode a varint that is expected to be 1-2 bytes (e.g. zigzag RSSI, small lengths).
static inline void ESPHOME_ALWAYS_INLINE encode_varint_raw_short(uint8_t *__restrict__ &pos PROTO_ENCODE_DEBUG_PARAM,
uint32_t value) {
[[nodiscard]] static inline uint8_t *ESPHOME_ALWAYS_INLINE
encode_varint_raw_short(uint8_t *__restrict__ pos PROTO_ENCODE_DEBUG_PARAM, uint32_t value) {
if (value < VARINT_MAX_1_BYTE) [[likely]] {
PROTO_ENCODE_CHECK_BOUNDS(pos, 1);
*pos++ = static_cast<uint8_t>(value);
return;
return pos;
}
if (value < VARINT_MAX_2_BYTE) [[likely]] {
PROTO_ENCODE_CHECK_BOUNDS(pos, 2);
*pos++ = static_cast<uint8_t>(value | 0x80);
*pos++ = static_cast<uint8_t>(value >> 7);
return;
return pos;
}
encode_varint_raw_loop(pos PROTO_ENCODE_DEBUG_ARG, value);
return encode_varint_raw_loop(pos PROTO_ENCODE_DEBUG_ARG, value);
}
static inline void ESPHOME_ALWAYS_INLINE encode_varint_raw_64(uint8_t *__restrict__ &pos PROTO_ENCODE_DEBUG_PARAM,
uint64_t value) {
[[nodiscard]] static inline uint8_t *ESPHOME_ALWAYS_INLINE
encode_varint_raw_64(uint8_t *__restrict__ pos PROTO_ENCODE_DEBUG_PARAM, uint64_t value) {
if (value < VARINT_MAX_1_BYTE) [[likely]] {
PROTO_ENCODE_CHECK_BOUNDS(pos, 1);
*pos++ = static_cast<uint8_t>(value);
return;
return pos;
}
encode_varint_raw_loop(pos PROTO_ENCODE_DEBUG_ARG, value);
return encode_varint_raw_loop(pos PROTO_ENCODE_DEBUG_ARG, value);
}
/// Encode a 48-bit MAC address (stored in a uint64) as varint.
/// Real MAC addresses occupy the full 48 bits (OUI in upper 24), so the
/// fast path -- any non-zero bit in the top 6 of 48 -- emits exactly 7 bytes
/// with no per-byte branch. Falls back to the general loop otherwise.
/// Caller must guarantee value fits in 48 bits (checked in debug builds).
static inline void ESPHOME_ALWAYS_INLINE encode_varint_raw_48bit(uint8_t *__restrict__ &pos PROTO_ENCODE_DEBUG_PARAM,
uint64_t value) {
[[nodiscard]] static inline uint8_t *ESPHOME_ALWAYS_INLINE
encode_varint_raw_48bit(uint8_t *__restrict__ pos PROTO_ENCODE_DEBUG_PARAM, uint64_t value) {
#ifdef ESPHOME_DEBUG_API
assert(value < (1ULL << (MAC_ADDRESS_SIZE * 8)) && "encode_varint_raw_48bit: value exceeds 48 bits");
#endif
@@ -363,38 +375,39 @@ class ProtoEncode {
pos[4] = static_cast<uint8_t>((value >> 28) | 0x80);
pos[5] = static_cast<uint8_t>((value >> 35) | 0x80);
pos[6] = static_cast<uint8_t>(value >> 42);
pos += 7;
return;
return pos + 7;
}
encode_varint_raw_64(pos PROTO_ENCODE_DEBUG_ARG, value);
return encode_varint_raw_64(pos PROTO_ENCODE_DEBUG_ARG, value);
}
static inline void ESPHOME_ALWAYS_INLINE encode_field_raw(uint8_t *__restrict__ &pos PROTO_ENCODE_DEBUG_PARAM,
uint32_t field_id, uint32_t type) {
encode_varint_raw(pos PROTO_ENCODE_DEBUG_ARG, (field_id << 3) | type);
[[nodiscard]] static inline uint8_t *ESPHOME_ALWAYS_INLINE
encode_field_raw(uint8_t *__restrict__ pos PROTO_ENCODE_DEBUG_PARAM, uint32_t field_id, uint32_t type) {
return encode_varint_raw(pos PROTO_ENCODE_DEBUG_ARG, proto_tag(field_id, type));
}
/// Write a single precomputed tag byte. Tag must be < 128.
static inline void ESPHOME_ALWAYS_INLINE write_raw_byte(uint8_t *__restrict__ &pos PROTO_ENCODE_DEBUG_PARAM,
uint8_t b) {
[[nodiscard]] static inline uint8_t *ESPHOME_ALWAYS_INLINE
write_raw_byte(uint8_t *__restrict__ pos PROTO_ENCODE_DEBUG_PARAM, uint8_t b) {
PROTO_ENCODE_CHECK_BOUNDS(pos, 1);
*pos++ = b;
return pos;
}
/// Reserve one byte for later backpatch (e.g., sub-message length).
/// Advances pos past the reserved byte without writing a value.
static inline void ESPHOME_ALWAYS_INLINE reserve_byte(uint8_t *__restrict__ &pos PROTO_ENCODE_DEBUG_PARAM) {
[[nodiscard]] static inline uint8_t *ESPHOME_ALWAYS_INLINE
reserve_byte(uint8_t *__restrict__ pos PROTO_ENCODE_DEBUG_PARAM) {
PROTO_ENCODE_CHECK_BOUNDS(pos, 1);
pos++;
return pos + 1;
}
/// Write raw bytes to the buffer (no tag, no length prefix).
static inline void ESPHOME_ALWAYS_INLINE encode_raw(uint8_t *__restrict__ &pos PROTO_ENCODE_DEBUG_PARAM,
const void *data, size_t len) {
[[nodiscard]] static inline uint8_t *ESPHOME_ALWAYS_INLINE
encode_raw(uint8_t *__restrict__ pos PROTO_ENCODE_DEBUG_PARAM, const void *data, size_t len) {
PROTO_ENCODE_CHECK_BOUNDS(pos, len);
std::memcpy(pos, data, len);
pos += len;
return pos + len;
}
/// Encode tag + 1-byte length + raw string data. For strings with max_data_length < 128.
/// Tag must be a single-byte varint (< 128). Always encodes (no zero check).
static inline void encode_short_string_force(uint8_t *__restrict__ &pos PROTO_ENCODE_DEBUG_PARAM, uint8_t tag,
const StringRef &ref) {
[[nodiscard]] static inline uint8_t *encode_short_string_force(uint8_t *__restrict__ pos PROTO_ENCODE_DEBUG_PARAM,
uint8_t tag, const StringRef &ref) {
#ifdef ESPHOME_DEBUG_API
assert(ref.size() < 128 && "encode_short_string_force: string exceeds max_data_length < 128");
#endif
@@ -402,137 +415,205 @@ class ProtoEncode {
pos[0] = tag;
pos[1] = static_cast<uint8_t>(ref.size());
std::memcpy(pos + 2, ref.c_str(), ref.size());
pos += 2 + ref.size();
return pos + 2 + ref.size();
}
/// Write a precomputed tag byte + 32-bit value in one operation.
static inline void ESPHOME_ALWAYS_INLINE write_tag_and_fixed32(uint8_t *__restrict__ &pos PROTO_ENCODE_DEBUG_PARAM,
uint8_t tag, uint32_t value) {
/// Write a precomputed tag byte + 32-bit value. Outlined on embedded: one copy beats inline stores per field.
[[nodiscard]] static PROTO_OUTLINE_FOR_SIZE uint8_t *write_tag_and_fixed32(
uint8_t *__restrict__ pos PROTO_ENCODE_DEBUG_PARAM, uint8_t tag, uint32_t value) {
PROTO_ENCODE_CHECK_BOUNDS(pos, 5);
pos[0] = tag;
#if __BYTE_ORDER__ == __ORDER_LITTLE_ENDIAN__
std::memcpy(pos + 1, &value, 4);
#else
pos[1] = static_cast<uint8_t>(value & 0xFF);
pos[2] = static_cast<uint8_t>((value >> 8) & 0xFF);
pos[3] = static_cast<uint8_t>((value >> 16) & 0xFF);
pos[4] = static_cast<uint8_t>((value >> 24) & 0xFF);
#endif
pos += 5;
write_fixed32_le(pos + 1, value);
return pos + 5;
}
static inline void encode_string(uint8_t *__restrict__ &pos PROTO_ENCODE_DEBUG_PARAM, uint32_t field_id,
const char *string, size_t len, bool force = false) {
if (len == 0 && !force)
return;
encode_field_raw(pos PROTO_ENCODE_DEBUG_ARG, field_id, 2); // type 2: Length-delimited string
[[nodiscard]] static inline uint8_t *encode_string_force(uint8_t *__restrict__ pos PROTO_ENCODE_DEBUG_PARAM,
uint32_t field_id, const char *string, size_t len) {
pos = encode_field_raw(pos PROTO_ENCODE_DEBUG_ARG, field_id, 2); // type 2: Length-delimited string
// NOLINTNEXTLINE(readability-inconsistent-ifelse-braces) -- false positive on [[likely]] attribute
if (len < VARINT_MAX_1_BYTE) [[likely]] {
PROTO_ENCODE_CHECK_BOUNDS(pos, 1 + len);
*pos++ = static_cast<uint8_t>(len);
} else {
encode_varint_raw_loop(pos PROTO_ENCODE_DEBUG_ARG, len);
pos = encode_varint_raw_loop(pos PROTO_ENCODE_DEBUG_ARG, len);
PROTO_ENCODE_CHECK_BOUNDS(pos, len);
}
std::memcpy(pos, string, len);
pos += len;
return pos + len;
}
static inline void encode_string(uint8_t *__restrict__ &pos PROTO_ENCODE_DEBUG_PARAM, uint32_t field_id,
const std::string &value, bool force = false) {
encode_string(pos PROTO_ENCODE_DEBUG_ARG, field_id, value.data(), value.size(), force);
[[nodiscard]] static inline uint8_t *encode_string(uint8_t *__restrict__ pos PROTO_ENCODE_DEBUG_PARAM,
uint32_t field_id, const char *string, size_t len) {
if (len == 0)
return pos;
return encode_string_force(pos PROTO_ENCODE_DEBUG_ARG, field_id, string, len);
}
static inline void encode_string(uint8_t *__restrict__ &pos PROTO_ENCODE_DEBUG_PARAM, uint32_t field_id,
const StringRef &ref, bool force = false) {
encode_string(pos PROTO_ENCODE_DEBUG_ARG, field_id, ref.c_str(), ref.size(), force);
[[nodiscard]] static inline uint8_t *encode_string_force(uint8_t *__restrict__ pos PROTO_ENCODE_DEBUG_PARAM,
uint32_t field_id, const std::string &value) {
return encode_string_force(pos PROTO_ENCODE_DEBUG_ARG, field_id, value.data(), value.size());
}
static inline void encode_bytes(uint8_t *__restrict__ &pos PROTO_ENCODE_DEBUG_PARAM, uint32_t field_id,
const uint8_t *data, size_t len, bool force = false) {
encode_string(pos PROTO_ENCODE_DEBUG_ARG, field_id, reinterpret_cast<const char *>(data), len, force);
[[nodiscard]] static inline uint8_t *encode_string(uint8_t *__restrict__ pos PROTO_ENCODE_DEBUG_PARAM,
uint32_t field_id, const StringRef &ref) {
return encode_string(pos PROTO_ENCODE_DEBUG_ARG, field_id, ref.c_str(), ref.size());
}
static inline void encode_uint32(uint8_t *__restrict__ &pos PROTO_ENCODE_DEBUG_PARAM, uint32_t field_id,
uint32_t value, bool force = false) {
if (value == 0 && !force)
return;
encode_field_raw(pos PROTO_ENCODE_DEBUG_ARG, field_id, 0);
encode_varint_raw(pos PROTO_ENCODE_DEBUG_ARG, value);
[[nodiscard]] static inline uint8_t *encode_string_force(uint8_t *__restrict__ pos PROTO_ENCODE_DEBUG_PARAM,
uint32_t field_id, const StringRef &ref) {
return encode_string_force(pos PROTO_ENCODE_DEBUG_ARG, field_id, ref.c_str(), ref.size());
}
static inline void encode_uint64(uint8_t *__restrict__ &pos PROTO_ENCODE_DEBUG_PARAM, uint32_t field_id,
uint64_t value, bool force = false) {
if (value == 0 && !force)
return;
encode_field_raw(pos PROTO_ENCODE_DEBUG_ARG, field_id, 0);
encode_varint_raw_64(pos PROTO_ENCODE_DEBUG_ARG, value);
[[nodiscard]] static inline uint8_t *encode_bytes(uint8_t *__restrict__ pos PROTO_ENCODE_DEBUG_PARAM,
uint32_t field_id, const uint8_t *data, size_t len) {
return encode_string(pos PROTO_ENCODE_DEBUG_ARG, field_id, reinterpret_cast<const char *>(data), len);
}
static inline void encode_bool(uint8_t *__restrict__ &pos PROTO_ENCODE_DEBUG_PARAM, uint32_t field_id, bool value,
bool force = false) {
if (!value && !force)
return;
encode_field_raw(pos PROTO_ENCODE_DEBUG_ARG, field_id, 0);
[[nodiscard]] static inline uint8_t *encode_bytes_force(uint8_t *__restrict__ pos PROTO_ENCODE_DEBUG_PARAM,
uint32_t field_id, const uint8_t *data, size_t len) {
return encode_string_force(pos PROTO_ENCODE_DEBUG_ARG, field_id, reinterpret_cast<const char *>(data), len);
}
[[nodiscard]] static inline uint8_t *encode_uint32_force(uint8_t *__restrict__ pos PROTO_ENCODE_DEBUG_PARAM,
uint32_t field_id, uint32_t value) {
pos = encode_field_raw(pos PROTO_ENCODE_DEBUG_ARG, field_id, 0);
return encode_varint_raw(pos PROTO_ENCODE_DEBUG_ARG, value);
}
[[nodiscard]] static inline uint8_t *encode_uint32(uint8_t *__restrict__ pos PROTO_ENCODE_DEBUG_PARAM,
uint32_t field_id, uint32_t value) {
if (value == 0)
return pos;
return encode_uint32_force(pos PROTO_ENCODE_DEBUG_ARG, field_id, value);
}
[[nodiscard]] static inline uint8_t *encode_uint64_force(uint8_t *__restrict__ pos PROTO_ENCODE_DEBUG_PARAM,
uint32_t field_id, uint64_t value) {
pos = encode_field_raw(pos PROTO_ENCODE_DEBUG_ARG, field_id, 0);
return encode_varint_raw_64(pos PROTO_ENCODE_DEBUG_ARG, value);
}
[[nodiscard]] static inline uint8_t *encode_uint64(uint8_t *__restrict__ pos PROTO_ENCODE_DEBUG_PARAM,
uint32_t field_id, uint64_t value) {
if (value == 0)
return pos;
return encode_uint64_force(pos PROTO_ENCODE_DEBUG_ARG, field_id, value);
}
[[nodiscard]] static inline uint8_t *encode_bool_force(uint8_t *__restrict__ pos PROTO_ENCODE_DEBUG_PARAM,
uint32_t field_id, bool value) {
pos = encode_field_raw(pos PROTO_ENCODE_DEBUG_ARG, field_id, 0);
PROTO_ENCODE_CHECK_BOUNDS(pos, 1);
*pos++ = value ? 0x01 : 0x00;
return pos;
}
static inline void encode_fixed32(uint8_t *__restrict__ &pos PROTO_ENCODE_DEBUG_PARAM, uint32_t field_id,
uint32_t value, bool force = false) {
if (value == 0 && !force)
return;
encode_field_raw(pos PROTO_ENCODE_DEBUG_ARG, field_id, 5);
[[nodiscard]] static inline uint8_t *encode_bool(uint8_t *__restrict__ pos PROTO_ENCODE_DEBUG_PARAM,
uint32_t field_id, bool value) {
if (!value)
return pos;
return encode_bool_force(pos PROTO_ENCODE_DEBUG_ARG, field_id, value);
}
/// Tag + fixed32 for multi-byte tags; single-byte tags use write_tag_and_fixed32.
[[nodiscard]] static PROTO_OUTLINE_FOR_SIZE uint8_t *encode_fixed32_force(
uint8_t *__restrict__ pos PROTO_ENCODE_DEBUG_PARAM, uint32_t field_id, uint32_t value) {
pos = encode_field_raw(pos PROTO_ENCODE_DEBUG_ARG, field_id, 5);
PROTO_ENCODE_CHECK_BOUNDS(pos, 4);
#if __BYTE_ORDER__ == __ORDER_LITTLE_ENDIAN__
std::memcpy(pos, &value, 4);
pos += 4;
#else
*pos++ = (value >> 0) & 0xFF;
*pos++ = (value >> 8) & 0xFF;
*pos++ = (value >> 16) & 0xFF;
*pos++ = (value >> 24) & 0xFF;
#endif
write_fixed32_le(pos, value);
return pos + 4;
}
[[nodiscard]] static inline uint8_t *encode_fixed32(uint8_t *__restrict__ pos PROTO_ENCODE_DEBUG_PARAM,
uint32_t field_id, uint32_t value) {
if (value == 0)
return pos;
return encode_fixed32_force(pos PROTO_ENCODE_DEBUG_ARG, field_id, value);
}
// NOTE: Wire type 1 (64-bit fixed: double, fixed64, sfixed64) is intentionally
// not supported to reduce overhead on embedded systems. All ESPHome devices are
// 32-bit microcontrollers where 64-bit operations are expensive. If 64-bit support
// is needed in the future, the necessary encoding/decoding functions must be added.
static inline void encode_float(uint8_t *__restrict__ &pos PROTO_ENCODE_DEBUG_PARAM, uint32_t field_id, float value,
bool force = false) {
uint32_t raw = float_to_raw(value);
if (raw == 0 && !force)
return;
encode_fixed32(pos PROTO_ENCODE_DEBUG_ARG, field_id, raw);
[[nodiscard]] static inline uint8_t *encode_float(uint8_t *__restrict__ pos PROTO_ENCODE_DEBUG_PARAM,
uint32_t field_id, float value) {
return encode_fixed32(pos PROTO_ENCODE_DEBUG_ARG, field_id, float_to_raw(value));
}
static inline void encode_int32(uint8_t *__restrict__ &pos PROTO_ENCODE_DEBUG_PARAM, uint32_t field_id, int32_t value,
bool force = false) {
[[nodiscard]] static inline uint8_t *encode_float_force(uint8_t *__restrict__ pos PROTO_ENCODE_DEBUG_PARAM,
uint32_t field_id, float value) {
return encode_fixed32_force(pos PROTO_ENCODE_DEBUG_ARG, field_id, float_to_raw(value));
}
[[nodiscard]] static inline uint8_t *encode_int32_force(uint8_t *__restrict__ pos PROTO_ENCODE_DEBUG_PARAM,
uint32_t field_id, int32_t value) {
if (value < 0) {
// negative int32 is always 10 byte long
encode_uint64(pos PROTO_ENCODE_DEBUG_ARG, field_id, static_cast<uint64_t>(value), force);
return;
return encode_uint64_force(pos PROTO_ENCODE_DEBUG_ARG, field_id, static_cast<uint64_t>(value));
}
encode_uint32(pos PROTO_ENCODE_DEBUG_ARG, field_id, static_cast<uint32_t>(value), force);
return encode_uint32_force(pos PROTO_ENCODE_DEBUG_ARG, field_id, static_cast<uint32_t>(value));
}
static inline void encode_int64(uint8_t *__restrict__ &pos PROTO_ENCODE_DEBUG_PARAM, uint32_t field_id, int64_t value,
bool force = false) {
encode_uint64(pos PROTO_ENCODE_DEBUG_ARG, field_id, static_cast<uint64_t>(value), force);
[[nodiscard]] static inline uint8_t *encode_int32(uint8_t *__restrict__ pos PROTO_ENCODE_DEBUG_PARAM,
uint32_t field_id, int32_t value) {
if (value == 0)
return pos;
return encode_int32_force(pos PROTO_ENCODE_DEBUG_ARG, field_id, value);
}
static inline void encode_sint32(uint8_t *__restrict__ &pos PROTO_ENCODE_DEBUG_PARAM, uint32_t field_id,
int32_t value, bool force = false) {
encode_uint32(pos PROTO_ENCODE_DEBUG_ARG, field_id, encode_zigzag32(value), force);
[[nodiscard]] static inline uint8_t *encode_int64(uint8_t *__restrict__ pos PROTO_ENCODE_DEBUG_PARAM,
uint32_t field_id, int64_t value) {
return encode_uint64(pos PROTO_ENCODE_DEBUG_ARG, field_id, static_cast<uint64_t>(value));
}
static inline void encode_sint64(uint8_t *__restrict__ &pos PROTO_ENCODE_DEBUG_PARAM, uint32_t field_id,
int64_t value, bool force = false) {
encode_uint64(pos PROTO_ENCODE_DEBUG_ARG, field_id, encode_zigzag64(value), force);
[[nodiscard]] static inline uint8_t *encode_int64_force(uint8_t *__restrict__ pos PROTO_ENCODE_DEBUG_PARAM,
uint32_t field_id, int64_t value) {
return encode_uint64_force(pos PROTO_ENCODE_DEBUG_ARG, field_id, static_cast<uint64_t>(value));
}
/// Sub-message encoding: sync pos to buffer, delegate, get pos from return value.
[[nodiscard]] static inline uint8_t *encode_sint32(uint8_t *__restrict__ pos PROTO_ENCODE_DEBUG_PARAM,
uint32_t field_id, int32_t value) {
return encode_uint32(pos PROTO_ENCODE_DEBUG_ARG, field_id, encode_zigzag32(value));
}
[[nodiscard]] static inline uint8_t *encode_sint32_force(uint8_t *__restrict__ pos PROTO_ENCODE_DEBUG_PARAM,
uint32_t field_id, int32_t value) {
return encode_uint32_force(pos PROTO_ENCODE_DEBUG_ARG, field_id, encode_zigzag32(value));
}
[[nodiscard]] static inline uint8_t *encode_sint64(uint8_t *__restrict__ pos PROTO_ENCODE_DEBUG_PARAM,
uint32_t field_id, int64_t value) {
return encode_uint64(pos PROTO_ENCODE_DEBUG_ARG, field_id, encode_zigzag64(value));
}
[[nodiscard]] static inline uint8_t *encode_sint64_force(uint8_t *__restrict__ pos PROTO_ENCODE_DEBUG_PARAM,
uint32_t field_id, int64_t value) {
return encode_uint64_force(pos PROTO_ENCODE_DEBUG_ARG, field_id, encode_zigzag64(value));
}
/// Repeated sub-message element; the constant tag is written inline.
template<typename T>
static inline void encode_sub_message(uint8_t *__restrict__ &pos PROTO_ENCODE_DEBUG_PARAM, ProtoWriteBuffer &buffer,
uint32_t field_id, const T &value) {
buffer.set_pos(pos);
buffer.encode_sub_message(field_id, value);
pos = buffer.get_pos();
[[nodiscard]] static inline uint8_t *encode_sub_message(uint8_t *__restrict__ pos PROTO_ENCODE_DEBUG_PARAM,
uint32_t field_id, const T &value) {
pos = encode_field_raw(pos PROTO_ENCODE_DEBUG_ARG, field_id, 2);
return encode_sub_message_body(pos PROTO_ENCODE_DEBUG_ARG, &value, &T::encode_msg);
}
/// Singular sub-message field, skipped when it encodes to nothing.
template<typename T>
static inline void encode_optional_sub_message(uint8_t *__restrict__ &pos PROTO_ENCODE_DEBUG_PARAM,
ProtoWriteBuffer &buffer, uint32_t field_id, const T &value) {
buffer.set_pos(pos);
buffer.encode_optional_sub_message(field_id, value);
pos = buffer.get_pos();
[[nodiscard]] static inline uint8_t *encode_optional_sub_message(uint8_t *__restrict__ pos PROTO_ENCODE_DEBUG_PARAM,
uint32_t field_id, const T &value) {
uint32_t nested_size = T::calc_size_msg(&value);
if (nested_size == 0)
return pos;
pos = encode_field_raw(pos PROTO_ENCODE_DEBUG_ARG, field_id, 2);
#ifdef ESPHOME_DEBUG_API
uint8_t *end = encode_sized_sub_message_body(pos PROTO_ENCODE_DEBUG_ARG, nested_size, &value, &T::encode_msg);
proto_check_sub_message_size(field_id, nested_size, pos, end);
return end;
#else
return encode_sized_sub_message_body(pos, nested_size, &value, &T::encode_msg);
#endif
}
/// Length and body, length backpatched after the body is written.
[[nodiscard]] static uint8_t *encode_sub_message_body(uint8_t *__restrict__ pos PROTO_ENCODE_DEBUG_PARAM,
const void *value, ProtoEncodeFn encode_fn);
/// Length and body for a precomputed size.
[[nodiscard]] static uint8_t *encode_sized_sub_message_body(uint8_t *__restrict__ pos PROTO_ENCODE_DEBUG_PARAM,
uint32_t nested_size, const void *value,
ProtoEncodeFn encode_fn);
private:
/// Unaligned little endian store of four bytes: byte stores where the outlined helper lives (ESP-IDF, ARM
/// without unaligned access), otherwise a memcpy the compiler folds into one store. Callers bounds check
/// and advance the cursor themselves.
static inline void ESPHOME_ALWAYS_INLINE write_fixed32_le(uint8_t *__restrict__ pos, uint32_t value) {
if constexpr (PROTO_FIXED32_BYTE_STORES) {
// Spelled out so the outlined helper does not itself become a memcpy call
pos[0] = static_cast<uint8_t>(value);
pos[1] = static_cast<uint8_t>(value >> 8);
pos[2] = static_cast<uint8_t>(value >> 16);
pos[3] = static_cast<uint8_t>(value >> 24);
} else {
const uint32_t le = convert_little_endian(value);
__builtin_memcpy(pos, &le, 4);
}
}
};
#undef PROTO_OUTLINE_FOR_SIZE
#undef PROTO_FIXED32_BYTE_STORES
#ifdef HAS_PROTO_MESSAGE_DUMP
/**
@@ -624,11 +705,10 @@ class DumpBuffer {
class ProtoMessage {
public:
// Non-virtual defaults for messages with no fields.
// Concrete message classes hide these with their own implementations.
// All call sites use templates to preserve the concrete type, so virtual
// dispatch is not needed. This eliminates per-message vtable entries for
// encode/calculate_size, saving ~1.3 KB of flash across all message types.
// Non-virtual defaults for messages with no fields; generated classes hide all four. The
// static encode_msg/calc_size_msg take const void * so &T::encode_msg needs no thunk.
static uint8_t *encode_msg(const void *self, uint8_t *pos PROTO_ENCODE_DEBUG_PARAM) { return pos; }
static uint32_t calc_size_msg(const void *self) { return 0; }
uint8_t *encode(ProtoWriteBuffer &buffer PROTO_ENCODE_DEBUG_PARAM) const { return buffer.get_pos(); }
uint32_t calculate_size() const { return 0; }
#ifdef HAS_PROTO_MESSAGE_DUMP
@@ -648,7 +728,13 @@ class ProtoMessage {
// Base class for messages that support decoding
class ProtoDecodableMessage : public ProtoMessage {
public:
void decode(const uint8_t *buffer, size_t length);
/// Stores one decoded field into \p msg; generated per message type. \p scalar is the varint or
/// fixed32 value, or the length of the length-delimited payload at \p data. An unknown field or
/// wrong wire type matches no case and is skipped.
using DecodeFieldFn = void (*)(void *msg, uint32_t tag, const uint8_t *data, proto_varint_value_t scalar);
/// Walk \p buffer and hand every field to \p field. The generated decode() passes the message's
/// own decode_field, so decodable messages carry no vtable.
static void decode_fields(void *msg, const uint8_t *buffer, size_t length, DecodeFieldFn field);
/**
* Count occurrences of a repeated field in a protobuf buffer.
@@ -660,14 +746,15 @@ class ProtoDecodableMessage : public ProtoMessage {
* @return Number of times the field appears in the buffer
*/
static uint32_t count_repeated_field(const uint8_t *buffer, size_t length, uint32_t target_field_id);
protected:
~ProtoDecodableMessage() = default;
virtual bool decode_varint(uint32_t field_id, proto_varint_value_t value) { return false; }
virtual bool decode_length(uint32_t field_id, ProtoLengthDelimited value) { return false; }
virtual bool decode_32bit(uint32_t field_id, Proto32Bit value) { return false; }
// NOTE: decode_64bit removed - wire type 1 not supported
// The destructor stays accessible on purpose: the generated messages are aggregates that brace
// initialise sub message members, which copies a base temporary. That trades away the compile time
// guard against deleting through this type; messages are stack locals and never owned through a base
// pointer. ProtoMessage keeps its guard for the dump builds.
};
#ifndef HAS_PROTO_MESSAGE_DUMP
// decode() passes decode_field explicitly, so nothing here may add a vtable
static_assert(!std::is_polymorphic_v<ProtoDecodableMessage>, "decodable messages carry no vtable");
#endif
class ProtoSize {
public:
@@ -792,7 +879,7 @@ class ProtoSize {
* @return The number of bytes needed to encode the field ID and wire type
*/
static constexpr uint32_t field(uint32_t field_id, uint32_t type) {
uint32_t tag = (field_id << 3) | (type & WIRE_TYPE_MASK);
uint32_t tag = proto_tag(field_id, type & WIRE_TYPE_MASK);
return varint(tag);
}
@@ -874,28 +961,6 @@ class ProtoSize {
}
};
// Implementation of methods that depend on ProtoSize being fully defined
// Encode thunk — converts void* back to concrete type for direct encode() call
template<typename T> uint8_t *proto_encode_msg(const void *msg, ProtoWriteBuffer &buf PROTO_ENCODE_DEBUG_PARAM) {
return static_cast<const T *>(msg)->encode(buf PROTO_ENCODE_DEBUG_ARG);
}
// Thin template wrapper; delegates to non-template core in proto.cpp.
template<typename T> inline void ProtoWriteBuffer::encode_sub_message(uint32_t field_id, const T &value) {
this->encode_sub_message(field_id, &value, &proto_encode_msg<T>);
}
// Thin template wrapper; delegates to non-template core.
template<typename T> inline void ProtoWriteBuffer::encode_optional_sub_message(uint32_t field_id, const T &value) {
this->encode_optional_sub_message(field_id, value.calculate_size(), &value, &proto_encode_msg<T>);
}
// Template decode_to_message - preserves concrete type so decode() resolves statically
template<typename T> void ProtoLengthDelimited::decode_to_message(T &msg) const {
msg.decode(this->value_, this->length_);
}
template<typename T> const char *proto_enum_to_string(T value);
// ProtoService removed — its methods were inlined into APIConnection.
esphome/components/api/subscribe_state.h
-1
View File
@@ -4,7 +4,6 @@
#ifdef USE_API
#include "esphome/core/component.h"
#include "esphome/core/component_iterator.h"
#include "esphome/core/controller.h"
namespace esphome::api {
class APIConnection;
esphome/components/api/wizard.py
+461
View File
@@ -0,0 +1,461 @@
"""The device wizard of the api component: schema, validation and code generation.
Home Assistant shows the wizard when the device is added. See api_wizard.h for the C++ side.
"""
from collections.abc import Callable
import importlib
import json
from types import ModuleType
from typing import Any
from esphome import automation
import esphome.codegen as cg
from esphome.components.const import CONF_DESCRIPTION
import esphome.config_validation as cv
from esphome.const import (
CONF_DEVICE_CLASS,
CONF_DEVICE_ID,
CONF_DOMAIN,
CONF_ENTITY_ID,
CONF_ID,
CONF_INTERNAL,
CONF_NAME,
CONF_PAGES,
CONF_PLATFORM,
CONF_TARGET,
)
from esphome.core import CORE, ID
import esphome.final_validate as fv
from esphome.helpers import fnv1_hash, fnv1_hash_object_id, fnv1a_32bit_hash
from esphome.types import ConfigType
API_DOMAIN = "api"
CONF_ENTITIES = "entities"
CONF_ENTITY = "entity"
CONF_INPUTS = "inputs"
CONF_INTEGRATION = "integration"
CONF_SUPPORTED_FEATURES = "supported_features"
CONF_TITLE = "title"
CONF_WIZARD = "wizard"
_API = cg.esphome_ns.namespace("api")
WizardInput = _API.class_("WizardInput")
WIZARD_ENTITY_ID_BUFFER_SIZE = (
256 # api_wizard.h; Home Assistant entity IDs are at most 255 bytes
)
# One API message must fit APIBuffer::MAX_SIZE (65535) together with the largest frame header (7 bytes,
# Noise) and footer (16 bytes, Noise MAC)
WIZARD_RESPONSE_MAX_SIZE = 65535 - 7 - 16
# Version of the JSON the wizard is sent as, see wizard_document()
WIZARD_JSON_VERSION = 1
# zstd level the JSON is compressed with. Output is deterministic for a given zstd version.
WIZARD_ZSTD_LEVEL = 19
# Wizard string limits; api.proto documents the same values
WIZARD_TITLE_MAX_LENGTH = 127
WIZARD_DESCRIPTION_MAX_LENGTH = 255
WIZARD_FILTER_MAX_LENGTH = 63
WIZARD_SUPPORTED_FEATURE_MAX_LENGTH = 127
def _wizard_text(max_length: int) -> Callable[[Any], str]:
"""A string passed to Home Assistant verbatim, so it may be a [%key:...%] translation placeholder."""
return cv.All(cv.string_strict, cv.Length(max=max_length))
def _wizard_strings(max_length: int) -> Callable[[Any], list[str]]:
"""A single string or a list of strings, always validated to a non-empty list."""
return cv.All(
cv.ensure_list(cv.All(cv.string_strict, cv.Length(min=1, max=max_length))),
cv.Length(min=1),
)
# Mirrors Home Assistant's EntityFilterSelectorConfig
WIZARD_ENTITY_FILTER_SCHEMA = cv.All(
cv.Schema(
{
cv.Optional(CONF_INTEGRATION): cv.All(
cv.string_strict, cv.Length(min=1, max=WIZARD_FILTER_MAX_LENGTH)
),
cv.Optional(CONF_DOMAIN): _wizard_strings(WIZARD_FILTER_MAX_LENGTH),
cv.Optional(CONF_DEVICE_CLASS): _wizard_strings(WIZARD_FILTER_MAX_LENGTH),
cv.Optional(CONF_SUPPORTED_FEATURES): _wizard_strings(
WIZARD_SUPPORTED_FEATURE_MAX_LENGTH
),
}
),
cv.has_at_least_one_key(
CONF_INTEGRATION, CONF_DOMAIN, CONF_DEVICE_CLASS, CONF_SUPPORTED_FEATURES
),
)
WIZARD_ENTITY_SCHEMA = cv.Schema(
{
cv.Required(CONF_ID): cv.use_id(cg.EntityBase),
cv.Optional(CONF_DESCRIPTION): _wizard_text(WIZARD_DESCRIPTION_MAX_LENGTH),
}
)
# An input is either standalone (id declares a new WizardInput) or linked to a homeassistant entity that has no
# entity_id of its own: Home Assistant sets it
WIZARD_INPUT_SCHEMA = cv.All(
cv.Schema(
{
cv.Optional(CONF_ID): cv.declare_id(WizardInput),
cv.Optional(CONF_ENTITY): cv.use_id(cg.EntityBase),
cv.Optional(CONF_DESCRIPTION): _wizard_text(WIZARD_DESCRIPTION_MAX_LENGTH),
cv.Optional(CONF_TARGET): cv.Schema(
{
cv.Required(CONF_ENTITY): cv.All(
cv.ensure_list(WIZARD_ENTITY_FILTER_SCHEMA), cv.Length(min=1)
),
}
),
}
),
cv.has_exactly_one_key(CONF_ID, CONF_ENTITY),
)
WIZARD_PAGE_SCHEMA = cv.All(
cv.Schema(
{
cv.Optional(CONF_TITLE): _wizard_text(WIZARD_TITLE_MAX_LENGTH),
cv.Optional(CONF_DESCRIPTION): _wizard_text(WIZARD_DESCRIPTION_MAX_LENGTH),
cv.Optional(CONF_ENTITIES): cv.All(
cv.ensure_list(WIZARD_ENTITY_SCHEMA), cv.Length(min=1)
),
cv.Optional(CONF_INPUTS): cv.All(
cv.ensure_list(WIZARD_INPUT_SCHEMA), cv.Length(min=1)
),
}
),
cv.has_at_least_one_key(CONF_ENTITIES, CONF_INPUTS),
)
def _wizard_inputs(wizard: ConfigType) -> list[ConfigType]:
return [conf for page in wizard[CONF_PAGES] for conf in page.get(CONF_INPUTS, [])]
def _wizard_input_id(conf: ConfigType) -> ID:
"""The ID that names an input: the one a standalone input declares, or its linked entity."""
return conf[CONF_ID] if CONF_ID in conf else conf[CONF_ENTITY]
def _validate_unique_wizard_inputs(wizard: ConfigType) -> ConfigType:
"""An input is identified on the wire by a hash of its ID, so IDs and hashes must be unique."""
seen: dict[int, str] = {}
for conf in _wizard_inputs(wizard):
input_id = _wizard_input_id(conf).id
if (key := fnv1_hash(input_id)) in seen:
if seen[key] == input_id:
raise cv.Invalid(f"Wizard input '{input_id}' is used more than once")
raise cv.Invalid(
f"Wizard inputs '{seen[key]}' and '{input_id}' have the same hash, rename one"
)
seen[key] = input_id
return wizard
WIZARD_SCHEMA = cv.All(
cv.Schema(
{
cv.Required(CONF_PAGES): cv.All(
cv.ensure_list(WIZARD_PAGE_SCHEMA), cv.Length(min=1)
),
}
),
_validate_unique_wizard_inputs,
)
# Platforms of the homeassistant component that a wizard input can stand for
WIZARD_INPUT_DOMAINS = (
"binary_sensor",
"button",
"number",
"select",
"sensor",
"switch",
"text",
"text_sensor",
)
# Platforms that act on one family of Home Assistant domains, which the input's filters must stay within
WIZARD_DOMAIN_LIMITED_PLATFORMS = ("button", "number", "select", "switch", "text")
def wizard_input_ids(api_config: ConfigType) -> set[str]:
"""The IDs of the entities that are linked inputs of the wizard in the given api config."""
if (wizard := api_config.get(CONF_WIZARD)) is None:
return set()
return {
conf[CONF_ENTITY].id for conf in _wizard_inputs(wizard) if CONF_ENTITY in conf
}
def _wizard_buffer_name(entity_id: ID) -> str:
return f"api_wizard_input_{entity_id.id}"
def wizard_input_buffer(entity_id: ID) -> str | None:
"""Name of the RAM buffer holding the Home Assistant entity ID of a wizard input.
Returns None when the entity is not an input of the wizard. The buffer is defined by the api
codegen, and the homeassistant entity of the input is given it in place of a constant.
"""
if entity_id.id not in wizard_input_ids(CORE.config.get(API_DOMAIN, {})):
return None
return _wizard_buffer_name(entity_id)
def _wizard_input_declaration(
config: fv.FinalValidateConfig, entity_id: ID
) -> tuple[str, ConfigType]:
"""The domain (like sensor) and the config an input ID is declared in."""
path = config.get_path_for_id(entity_id)[:-1]
return path[0], config.get_config_for_path(path)
def _wizard_input_filters(
conf: ConfigType, config: fv.FinalValidateConfig
) -> list[ConfigType]:
"""The entity filters of an input, with the defaults of a domain limited linked platform."""
if (filters := conf.get(CONF_TARGET, {}).get(CONF_ENTITY)) is not None:
return filters
if CONF_ENTITY in conf:
domain, _ = _wizard_input_declaration(config, conf[CONF_ENTITY])
if domains := _wizard_default_domains(domain):
return [{CONF_DOMAIN: domains}]
return []
def _wizard_defines(wizard: ConfigType) -> set[str]:
"""The defines for the parts of the wizard the configuration uses, so the rest is not compiled."""
inputs = _wizard_inputs(wizard)
defines = {"USE_API_WIZARD"}
if inputs:
defines.add("USE_API_WIZARD_INPUTS")
if any(CONF_ENTITY in conf for conf in inputs):
defines.add("USE_API_WIZARD_LINKED_INPUTS")
if any(CONF_ID in conf for conf in inputs):
defines.add("USE_API_WIZARD_STANDALONE_INPUTS")
return defines
def _wizard_default_domains(domain: str) -> list[str] | None:
"""The domains Home Assistant entities can be picked from when the input sets no target."""
if domain in WIZARD_DOMAIN_LIMITED_PLATFORMS:
platform = importlib.import_module(f"esphome.components.homeassistant.{domain}")
return list(platform.SUPPORTED_DOMAINS)
return None
def _validate_wizard_input(conf: ConfigType) -> ConfigType:
if CONF_ENTITY not in conf:
return conf
domain, declaration = _wizard_input_declaration(
fv.full_config.get(), conf[CONF_ENTITY]
)
if (
declaration.get(CONF_PLATFORM) != "homeassistant"
or domain not in WIZARD_INPUT_DOMAINS
):
raise cv.Invalid(
f"Wizard input '{conf[CONF_ENTITY].id}' must be a homeassistant "
f"{', '.join(WIZARD_INPUT_DOMAINS)} entity"
)
if CONF_ENTITY_ID in declaration:
# An entity_id in the configuration is a static entry, which is kept apart from the dynamic ones
raise cv.Invalid(
f"'{conf[CONF_ENTITY].id}' has an entity_id set in its configuration, so it cannot be a "
"wizard input. Remove entity_id to let Home Assistant set it through the wizard."
)
if domain in WIZARD_DOMAIN_LIMITED_PLATFORMS:
supported = _wizard_default_domains(domain)
for entity_filter in conf.get(CONF_TARGET, {}).get(CONF_ENTITY, []):
if not (domains := entity_filter.get(CONF_DOMAIN)):
raise cv.Invalid(
f"Every filter of a homeassistant {domain} input must set domain"
)
if unsupported := [d for d in domains if d not in supported]:
raise cv.Invalid(
f"The homeassistant {domain} does not support the domain(s) "
f"{', '.join(unsupported)}. Supported: {', '.join(supported)}"
)
return conf
def _validate_wizard_entity_exposed(value: ID) -> ID:
"""Reject entities that are internal, as they are not exposed over the API, or have no name.
The key a client knows an entity by is a hash of its name. Without a name of its own, the
device works the name out at runtime from its friendly name, which can add the MAC address,
so the key cannot be known when the wizard is built.
"""
_, declaration = _wizard_input_declaration(fv.full_config.get(), value)
if declaration.get(CONF_INTERNAL, False):
raise cv.Invalid(
f"Entity '{value.id}' is internal, so it is not exposed over the API "
"and cannot be used in the wizard"
)
if not declaration.get(CONF_NAME):
raise cv.Invalid(
f"Entity '{value.id}' has no name of its own, so its key is not known "
"when the wizard is built. Give it a name to use it in the wizard"
)
return value
_WIZARD_FINAL_VALIDATE_SCHEMA = cv.Schema(
{
cv.Optional(CONF_WIZARD): {
cv.Optional(CONF_PAGES): [
{
cv.Optional(CONF_ENTITIES): [
{cv.Optional(CONF_ID): _validate_wizard_entity_exposed}
],
cv.Optional(CONF_INPUTS): [_validate_wizard_input],
}
]
}
},
extra=cv.ALLOW_EXTRA,
)
def final_validate(config: ConfigType) -> None:
"""Final validation of the wizard in the given api config, if it has one."""
_WIZARD_FINAL_VALIDATE_SCHEMA(config)
if (wizard := config.get(CONF_WIZARD)) is not None:
size = len(wizard_blob(wizard, fv.full_config.get()))
if size > WIZARD_RESPONSE_MAX_SIZE:
raise cv.Invalid(
f"The compressed wizard is {size} bytes, {size - WIZARD_RESPONSE_MAX_SIZE} "
f"bytes over the {WIZARD_RESPONSE_MAX_SIZE} bytes one API message can hold. "
"Shorten the texts or use fewer pages, entities or filters",
path=[CONF_WIZARD],
)
WIZARD_INPUT_IS_SET_SCHEMA = cv.maybe_simple_value(
{cv.Required(CONF_ID): cv.use_id(WizardInput)}, key=CONF_ID
)
# Only for standalone inputs: a linked input is read through its homeassistant entity
automation.register_apply_condition(
"api.wizard.input_is_set", WIZARD_INPUT_IS_SET_SCHEMA, "has_entity_id()"
)
def zstd_module() -> ModuleType:
"""The zstd module: the standard library one from Python 3.14, otherwise the backport."""
try:
return importlib.import_module("compression.zstd")
except ImportError:
return importlib.import_module("backports.zstd")
def _entity_document(conf: ConfigType, config: fv.FinalValidateConfig) -> ConfigType:
"""An entity of the device that the page shows, keyed as ListEntitiesResponse keys it."""
_, declaration = _wizard_input_declaration(config, conf[CONF_ID])
document: ConfigType = {"key": fnv1_hash_object_id(declaration[CONF_NAME])}
if (device := declaration.get(CONF_DEVICE_ID)) is not None:
document["device_id"] = fnv1a_32bit_hash(device.id)
if description := conf.get(CONF_DESCRIPTION):
document[CONF_DESCRIPTION] = description
return document
def _input_document(conf: ConfigType, config: fv.FinalValidateConfig) -> ConfigType:
document: ConfigType = {"key": fnv1_hash(_wizard_input_id(conf).id)}
if description := conf.get(CONF_DESCRIPTION):
document[CONF_DESCRIPTION] = description
if filters := _wizard_input_filters(conf, config):
document["entity_filters"] = [dict(entity_filter) for entity_filter in filters]
return document
def wizard_document(wizard: ConfigType, config: fv.FinalValidateConfig) -> ConfigType:
"""The wizard as the JSON document the device sends, before it is serialised.
This is the format Home Assistant reads, and api.proto documents it for clients. Version 1:
{"version": 1,
"pages": [{"title": "...", "description": "...",
"entities": [{"key": 123, "device_id": 456, "description": "..."}],
"inputs": [{"key": 789, "description": "...",
"entity_filters": [{"integration": "...", "domain": ["..."],
"device_class": ["..."], "supported_features": ["..."]}]}]}]}
Anything empty or unset, and every empty list, is left out. Strings are passed through as
written, so they may be Home Assistant translation placeholders.
- An entity key is the key ListEntitiesResponse sends for the entity: the FNV-1 hash of the
object id made from its name (entity_helpers). device_id is the hash of the ESPHome id of
the device it belongs to (esphome/core/config.py), and is left out for the main device.
- An input key is the FNV-1 hash of the ESPHome id of the input, or of the linked entity. A linked
entity must not set entity_id, as that is a static entry kept apart from the ones the wizard sets.
- entity_filters are the filters of the input, or the default filters of a linked switch,
number, text, select or button.
"""
pages: list[ConfigType] = []
for page in wizard[CONF_PAGES]:
document: ConfigType = {}
for key in (CONF_TITLE, CONF_DESCRIPTION):
if value := page.get(key):
document[key] = value
if entities := [
_entity_document(e, config) for e in page.get(CONF_ENTITIES, [])
]:
document[CONF_ENTITIES] = entities
if inputs := [_input_document(i, config) for i in page.get(CONF_INPUTS, [])]:
document[CONF_INPUTS] = inputs
pages.append(document)
return {"version": WIZARD_JSON_VERSION, CONF_PAGES: pages}
def wizard_blob(wizard: ConfigType, config: fv.FinalValidateConfig) -> bytes:
"""The wizard document as compact, sorted UTF-8 JSON in a single zstd frame."""
text = json.dumps(
wizard_document(wizard, config),
separators=(",", ":"),
sort_keys=True,
ensure_ascii=False,
)
return zstd_module().compress(text.encode("utf-8"), level=WIZARD_ZSTD_LEVEL)
async def to_code(wizard: ConfigType) -> None:
"""Emit the compressed wizard, the table of inputs and the defines.
The API reads both tables from its own sources, so they are externally linked PROGMEM arrays.
"""
blob = wizard_blob(wizard, CORE.config)
cg.extern_progmem_array("esphome::api::API_WIZARD_DATA", cg.uint8, list(blob))
cg.add_define("API_WIZARD_DATA_SIZE", len(blob))
for define in sorted(_wizard_defines(wizard)):
cg.add_define(define)
entries: list[cg.RawExpression] = []
for conf in _wizard_inputs(wizard):
input_id = _wizard_input_id(conf)
# Every buffer starts empty, until the wizard sets it. A linked homeassistant entity uses it as its entity id.
buffer = _wizard_buffer_name(input_id)
cg.add_global(
cg.RawStatement(
f'static char {buffer}[{WIZARD_ENTITY_ID_BUFFER_SIZE}] = "";'
)
)
if CONF_ID in conf:
cg.new_Pvariable(input_id, cg.RawExpression(buffer))
entries.append(cg.RawExpression(f"{{{fnv1_hash(input_id.id)}u, {buffer}}}"))
if entries:
cg.extern_progmem_array(
"esphome::api::API_WIZARD_INPUTS",
cg.esphome_ns.namespace("api").struct("WizardInputEntry"),
entries,
)
cg.add_define("API_WIZARD_INPUT_COUNT", len(entries))
esphome/components/as3935/sensor.py
+3 -7
View File
@@ -35,10 +35,6 @@ CONFIG_SCHEMA = cv.Schema(
async def to_code(config: ConfigType) -> None:
hub = await cg.get_variable(config[CONF_AS3935_ID])
if distance_config := config.get(CONF_DISTANCE):
sens = await sensor.new_sensor(distance_config)
cg.add(hub.set_distance_sensor(sens))
if lightning_energy_config := config.get(CONF_LIGHTNING_ENERGY):
sens = await sensor.new_sensor(lightning_energy_config)
cg.add(hub.set_energy_sensor(sens))
sensors = sensor.sub_sensors(config)
await sensors(CONF_DISTANCE, hub.set_distance_sensor)
await sensors(CONF_LIGHTNING_ENERGY, hub.set_energy_sensor)

Some files were not shown because too many files have changed in this diff Show More