Compare commits

..
Author SHA1 Message Date
J. Nick Koston 3b99c325b9 [rtttl] Use a user provided default constructor for Rtttl 2026-09-12 05:35:56 -05:00
4715 changed files with 43487 additions and 151568 deletions
-116
View File
@@ -1,116 +0,0 @@
---
name: code-review
description: Review guidance for ESPHome pull requests. Use this when reviewing a pull request that changes ESPHome Python, C++, or component code, to check it against the project's coding conventions, embedded-systems memory rules, testing requirements, and breaking-change policy.
---
# Reviewing ESPHome pull requests
ESPHome parses YAML into C++ firmware for memory-constrained microcontrollers
(ESP32, ESP8266, RP2040, LibreTiny). Review changes with that in mind: RAM and
flash are scarce, and code runs unattended for months.
`AGENTS.md` in the repository root is the full contributor guide and the
authority when it disagrees with this summary. The developer documentation at
https://developers.esphome.io explains the component lifecycle and the reasoning
behind these rules. This skill lists the concrete things worth flagging in a
review; read `AGENTS.md` for the detail behind any item.
Only raise findings that the diff actually introduces or changes. Do not ask for
drive-by cleanup of pre-existing code the PR did not touch.
## Memory and embedded constraints (highest value)
Heap allocation after `setup()` is treated as a reliability bug, not a
performance nit, because it fragments a small shared heap. Flag:
- New heap allocation on a hot path or after setup that could be avoided.
- `std::vector` where the size is known at compile time (use `std::array`, or
`StaticVector<T, N>` when a `push_back` API is needed) or fixed at runtime
init (use `FixedVector<T>`).
- Listener / child-entity registration lists stored as `std::vector`; these have
a compile-time-known count and should use `cg.slot_counter()` plus
`StaticVector`.
- `std::vector<uint8_t>` for a byte buffer that never grows: prefer
`std::unique_ptr<uint8_t[]>` or `std::array`.
- `std::map` / `std::set` / `std::unordered_map` for small datasets (1-16
elements): a `std::vector` of a small struct with linear search is lighter.
- `std::deque` anywhere: it allocates 512-byte blocks and should be avoided.
- `std::string` storing a value set once from config: prefer `StringRef` (the
literal already lives in flash).
- `std::string` / `std::to_string` / string-returning helpers on hot paths where
a buffer or view API exists.
## C++ conventions
- Include what you use: a file referencing a symbol must include the header that
declares it, even if it currently arrives transitively. New or changed uses of
a symbol need the matching include.
- Prefix all member access with `this->`.
- Naming: `lower_snake_case` for functions/methods/variables, `UpperCamelCase`
for classes/structs/enums, `UPPER_SNAKE_CASE` for namespace-scope constants,
trailing underscore on protected/private fields.
- `enum class` values must be prefixed with the enum name in `UPPER_SNAKE_CASE`
(e.g. `UARTFlushResult::UART_FLUSH_RESULT_SUCCESS`). Bare names like `SUCCESS`,
`FAIL`, or `OK` collide with SDK macros on some platforms and break the build.
- Prefer `const`/`enum` over `#define`; `#define` is only for conditional
compilation and code-generation sizes.
- Never call `millis()` in a `loop()` body; use
`App.get_loop_component_start_time()`. A rate-limit gate below ~16 ms (the loop
period) does nothing.
- Pick the timing primitive by cadence: gated `loop()` under 250 ms,
`set_interval` at 500 ms and above.
- Do not override a base method to return the value it already returns (e.g.
`get_setup_priority()` returning `setup_priority::DATA`).
- Wrap string literals passed as printf `%s` args in `LOG_STR_LITERAL()`.
- Required, invariant dependencies should be constructor parameters, not setters.
- Callback registration methods must be templated (`template<typename F>`), not
typed as `std::function`, so lightweight forwarders avoid a heap allocation.
- Two-space indent, `using` over `typedef`, wrap at 120 columns.
## Python conventions
- Type-annotate every new function signature (params and return), new dataclass
fields, and new module-level variables. Import `ConfigType` from
`esphome.types`.
- Use the walrus operator to avoid a double lookup, e.g.
`if (blah := config.get(CONF_BLAH)) is not None:`.
- Reuse existing validators from `config_validation.py` (`cv.rename_key`,
`cv.has_exactly_one_key`, etc.) via `cv.All(...)` instead of hand-rolling.
- `esphome/const.py` is frozen: no new `CONF_` constants there. Define them in
the component's own `.py`, or in `esphome/components/const/__init__.py` when
shared. The same constant defined in three or more component files fails CI.
- State that must persist during code generation goes in `CORE.data` namespaced
under the component `DOMAIN` (a `@dataclass`), not module-level mutable globals.
- Prefer callback-based triggers via `build_callback_automation()`; only use a
`Trigger<Ts...>` subclass when the forwarder needs mutable state.
## Testing and coverage
- New and changed lines and branches need test coverage, including defensive
early-returns, error paths, and no-op guards. A mocked-out function is not
covered; exercise the real call path too.
- Component YAML tests live in `tests/components/<component>/`. Never define
buses (uart, i2c, spi, modbus) directly in a test file: pull them from
`tests/test_build_components/common/` through dict-style `packages:` so CI can
group builds. List-style packages or top-level merge keys block grouping.
- Config-only checks use the `validate.*.yaml` prefix; compiled checks use
`test.*.yaml`.
## Breaking changes and public API
- Base classes under `esphome/core/` and documented config options are public
API. Undocumented `public` members of a component are internal.
- A breaking change needs justification, a migration path in the PR description,
and a deprecation window where feasible (`ESPDEPRECATED` in C++,
`cv.rename_key(..., removed_in=...)` in Python). Changing a codegen-injected
lambda signature is not a breaking change.
## Process and PR hygiene
- PR titles start with a `[tag]` prefix: the component name (e.g. `[uart] ...`)
or `[core]` for shared code.
- Prose in docs, comments, and commit messages should be plain English. Keep
inline comments short and only where the code is not self-explanatory; do not
restate what the code says.
- Verify the PR fills out `.github/PULL_REQUEST_TEMPLATE.md` and adds
`CODEOWNERS` entries for a new component.
-1
View File
@@ -1 +0,0 @@
../.agents/skills
+3
View File
@@ -31,4 +31,7 @@ RUN \
platformio settings set enable_telemetry No \
&& platformio settings set check_platformio_interval 1000000
COPY script/platformio_install_deps.py platformio.ini ./
RUN ./platformio_install_deps.py platformio.ini --libraries --platforms --tools
WORKDIR /workspaces
-23
View File
@@ -9,29 +9,6 @@ body:
If you have a feature request or enhancement, please [request them here instead][fr].
[fr]: https://github.com/orgs/esphome/discussions
- type: markdown
attributes:
value: |
## Use of AI in bug reports
AI tools are good at carrying out well-defined tasks, but they are not good at troubleshooting.
Please do NOT paste an AI-generated wall of text into the issue template - if the AI hasn't solved
your problem, its wild guesses are not likely to help.
Please DO include your own words and observations, compile/boot logs, and
especially a minimal reproducible example of your YAML configuration that demonstrates the problem.
It is however quite acceptable to use AI to translate your *own* report,
if you aren't a competent English speaker.
If you really think it will be useful to include an AI's analysis, preferably wrap it in a `<details>` block which will be collapsed by default.
If you are using AI to help solve a problem, rather than asking it to speculate about what the problem is,
it can be more useful to ask it to create a step-by-step troubleshooting procedure.
AI is also useful for generating boilerplate code, such as a minimal reproducible example of your YAML
configuration that demonstrates the problem.
Used properly, AI can be a useful tool to help you solve your problem, but don't let it get in the way.
- type: textarea
validations:
required: true
+2 -2
View File
@@ -42,7 +42,7 @@ runs:
- name: Build and push to ghcr by digest
id: build-ghcr
uses: docker/build-push-action@c3c9e263c25d99ce0380d002d59b67737d91b0dc # v7.4.0
uses: docker/build-push-action@53b7df96c91f9c12dcc8a07bcb9ccacbed38856a # v7.3.0
env:
DOCKER_BUILD_SUMMARY: false
DOCKER_BUILD_RECORD_UPLOAD: false
@@ -67,7 +67,7 @@ runs:
- name: Build and push to dockerhub by digest
id: build-dockerhub
uses: docker/build-push-action@c3c9e263c25d99ce0380d002d59b67737d91b0dc # v7.4.0
uses: docker/build-push-action@53b7df96c91f9c12dcc8a07bcb9ccacbed38856a # v7.3.0
env:
DOCKER_BUILD_SUMMARY: false
DOCKER_BUILD_RECORD_UPLOAD: false
@@ -1,39 +0,0 @@
name: Cache Arduino ESP8266
description: >
Resolve the pinned Arduino core and xtensa toolchain versions and cache the
native ESP8266 install (~110 MB framework + toolchain; no ccache store, the
seed job saves before any compile runs). Exports
ESPHOME_ARDUINO8266_PREFIX to the job so every later step installs into
the cached path; the Python venv must already be restored. Mirrors
cache-esp-idf: only dev-branch pushes write the shared cache, everything
else restores.
runs:
using: composite
steps:
- name: Resolve the native toolchain cache key
# Versions are pinned in code, not a hashable file; resolve them so a
# bump changes the cache key. Assignment form so errexit catches a
# resolver failure.
id: version
shell: bash
run: |
# One owner for the install prefix: exported here and referenced by
# the cache steps below via env, so the caller's install and the
# cached path cannot diverge.
echo "ESPHOME_ARDUINO8266_PREFIX=$HOME/.esphome-arduino8266" >> "$GITHUB_ENV"
. venv/bin/activate
key=$(python -c 'from esphome.components.esp8266 import RECOMMENDED_ARDUINO_FRAMEWORK_VERSION as f; from esphome.arduino8266.framework import FRAMEWORK_RELEASES, TOOLCHAIN_VERSION as t; print(f"{FRAMEWORK_RELEASES[f].tag}-{t}")')
[ -n "$key" ] || exit 1
echo "key=$key" >> "$GITHUB_OUTPUT"
- name: Cache the native toolchain (write on dev)
if: github.ref == 'refs/heads/dev'
uses: actions/cache@55cc8345863c7cc4c66a329aec7e433d2d1c52a9 # v6.1.0
with:
path: ${{ env.ESPHOME_ARDUINO8266_PREFIX }}
key: ${{ runner.os }}-esp8266-native-${{ steps.version.outputs.key }}
- name: Restore the native toolchain (off dev)
if: github.ref != 'refs/heads/dev'
uses: actions/cache/restore@55cc8345863c7cc4c66a329aec7e433d2d1c52a9 # v6.1.0
with:
path: ${{ env.ESPHOME_ARDUINO8266_PREFIX }}
key: ${{ runner.os }}-esp8266-native-${{ steps.version.outputs.key }}
+1 -1
View File
@@ -32,7 +32,7 @@ runs:
# detects the activated venv via ``VIRTUAL_ENV`` so the venv layout
# downstream jobs rely on is preserved.
if: steps.cache-venv.outputs.cache-hit != 'true'
uses: astral-sh/setup-uv@c18668ad3cf93ea998bef934396af7bb5c839dc7 # v10.2.0
uses: astral-sh/setup-uv@20cfd1bf945f4377ade1205e4dbc17946fc9a30d # v10.0.1
with:
enable-cache: true
# Pull request saves land in per-PR scopes nothing else can
+1 -1
View File
@@ -53,7 +53,7 @@ module.exports = async ({ github, context }) => {
});
const currentLabels = currentLabelsData.map(label => label.name);
const managedLabels = currentLabels.filter(label =>
label.startsWith('component: ') || label.startsWith('platform: ') || MANAGED_LABELS.includes(label)
label.startsWith('component: ') || MANAGED_LABELS.includes(label)
);
// Check for mega-PR early - if present, skip most automatic labeling
+10 -26
View File
@@ -42,8 +42,8 @@ function generateReviewMessages(finalLabels, originalLabelCount, deprecatedInfo,
let message = `${TOO_BIG_MARKER}\n### 📦 Pull Request Size\n\n`;
message +=
`Hey @${prAuthor}, thanks for the contribution. This PR is on the ` +
`large side `;
`Hey @${prAuthor}, thanks for the contribution! Just a heads up, ` +
`this PR is on the large side `;
if (tooManyLabels && tooManyChanges) {
message +=
@@ -57,31 +57,16 @@ function generateReviewMessages(finalLabels, originalLabelCount, deprecatedInfo,
}
message += `, which makes it harder for maintainers to review.\n\n`;
let condition;
if (tooManyLabels && tooManyChanges) {
condition =
`is at most ${TOO_BIG_THRESHOLD} lines (excluding tests) and touches ` +
`at most ${MAX_LABELS} components/areas`;
} else if (tooManyLabels) {
condition = `touches at most ${MAX_LABELS} components/areas`;
} else {
condition = `is at most ${TOO_BIG_THRESHOLD} lines (excluding tests)`;
}
message +=
`**This PR will not be reviewed until it ${condition}.** We know ` +
`that is not what you want to hear after this much work, and we want ` +
`to be upfront so you do not wait for a review that is not coming: ` +
`please split the change. The label is removed automatically once ` +
`the PR ${condition}.\n\n`;
`Smaller, focused PRs tend to be reviewed much faster since they ` +
`fit into the short gaps between other maintainer work; large ones ` +
`often have to wait for a rare long uninterrupted block of time. ` +
`If you can break this up into smaller pieces that can be reviewed ` +
`independently, it will almost certainly land faster overall.\n\n`;
message +=
`There is no exception you can request. Rarely, a maintainer agrees in ` +
`\`#devs\` on [Discord](https://esphome.io/chat) to review a change that ` +
`genuinely cannot be split as one piece and adds the \`mega-pr\` label, ` +
`but that is rare enough that you should plan on splitting rather than ` +
`wait for it. Smaller, focused PRs are picked up much faster because ` +
`they fit into the short gaps between other maintainer work. For a new ` +
`component, the first PR brings the hub and a single platform (for ` +
`example \`sensor\`); each further platform is its own PR on top of it.\n\n`;
`Before putting more time in, it's also worth popping into ` +
`\`#devs\` on [Discord](https://esphome.io/chat) so we can help ` +
`you scope things and flag anything already in flight.\n\n`;
message +=
`For more details (including how to split the work up), see: ` +
`https://developers.esphome.io/contributing/submitting-your-work/` +
@@ -229,7 +214,6 @@ async function handleMaintainerAccessComment(github, context, maintainerAccess)
}
module.exports = {
generateReviewMessages,
handleReviews,
handleMaintainerAccessComment
};
@@ -1,38 +0,0 @@
const { describe, it } = require('node:test');
const assert = require('node:assert/strict');
const { generateReviewMessages } = require('../reviews');
const MAX_LABELS = 15;
const TOO_BIG_THRESHOLD = 1000;
function tooBigMessage(labelCount, additions) {
const files = [{ filename: 'esphome/components/a/a.cpp', additions, deletions: 0 }];
const messages = generateReviewMessages(['too-big'], labelCount, null, files, additions, 0, 'someone', MAX_LABELS, TOO_BIG_THRESHOLD);
assert.equal(messages.length, 1);
return messages[0];
}
describe('too-big review message', () => {
it('names the line cap when the PR is too long', () => {
const message = tooBigMessage(3, 1500);
assert.match(message, /will not be reviewed until it is at most 1000 lines \(excluding tests\)\./);
assert.match(message, /removed automatically once the PR is at most 1000 lines/);
assert.doesNotMatch(message, /components\/areas\./);
});
it('names the area cap when the PR touches too many areas but is short', () => {
const message = tooBigMessage(20, 200);
assert.match(message, /will not be reviewed until it touches at most 15 components\/areas\./);
assert.doesNotMatch(message, /until it is at most 1000 lines/);
});
it('names both caps when both are exceeded', () => {
const message = tooBigMessage(20, 1500);
assert.match(message, /is at most 1000 lines \(excluding tests\) and touches at most 15 components\/areas\./);
});
it('tells new-component authors to bring one platform first', () => {
assert.match(tooBigMessage(3, 1500), /hub and a single platform/);
assert.match(tooBigMessage(3, 1500), /mega-pr/);
});
});
-1
View File
@@ -1 +0,0 @@
../.agents/skills
+1 -1
View File
@@ -29,7 +29,7 @@ jobs:
- name: Set up uv
# ``--system`` (below) installs into the setup-python interpreter;
# no venv is created or restored by this workflow.
uses: astral-sh/setup-uv@c18668ad3cf93ea998bef934396af7bb5c839dc7 # v10.2.0
uses: astral-sh/setup-uv@20cfd1bf945f4377ade1205e4dbc17946fc9a30d # v10.0.1
with:
enable-cache: true
# Pull-request-only workflow: a save could never be shared and
+4 -5
View File
@@ -12,14 +12,15 @@ on:
- ".github/workflows/ci-docker.yml"
- "requirements*.txt"
- "pyproject.toml"
- "platformio.ini"
- "esphome/idf_component.yml"
- "script/platformio_install_deps.py"
# Core, build pipeline, toolchain, and target-platform changes can change
# how a toolchain is set up or built, so re-run the per-toolchain compile
# smoke test when they change.
- "esphome/core/**"
- "esphome/writer.py"
- "esphome/build_gen/**"
- "esphome/build_helpers/**"
- "esphome/espidf/**"
- "esphome/platformio/**"
- "esphome/components/bk72xx/**"
@@ -66,7 +67,7 @@ jobs:
with:
python-version: "3.12"
- name: Set up Docker Buildx
uses: docker/setup-buildx-action@594f3bf4285d9ea8dc53c9a0c9c4092420091003 # v4.4.0
uses: docker/setup-buildx-action@37fe631027851001ddb9b187196cc803df7f5f0e # v4.3.0
- name: Determine tag and whether to push
id: tag
@@ -158,7 +159,7 @@ jobs:
with:
python-version: "3.12"
- name: Set up Docker Buildx
uses: docker/setup-buildx-action@594f3bf4285d9ea8dc53c9a0c9c4092420091003 # v4.4.0
uses: docker/setup-buildx-action@37fe631027851001ddb9b187196cc803df7f5f0e # v4.3.0
- name: Log in to the GitHub container registry
uses: docker/login-action@dbcb813823bdd20940b903addbd779551569679f # v4.6.0
@@ -196,7 +197,6 @@ jobs:
# the default.
id:
- esp8266-arduino
- esp8266-arduino-native
- esp32-arduino-platformio
- esp32-arduino-esp-idf
- esp32-idf-platformio
@@ -219,6 +219,5 @@ jobs:
run: |
docker run --rm \
-v "${{ github.workspace }}/docker/test_configs:/config" \
-e ESPHOME_LDGEN_STRICT=1 \
"ghcr.io/esphome/esphome-amd64:${{ needs.check-docker.outputs.tag }}" \
compile "${{ matrix.id }}.yaml"
+64 -232
View File
@@ -49,7 +49,7 @@ jobs:
# detects the activated venv via ``VIRTUAL_ENV`` so downstream jobs
# that ``. venv/bin/activate`` see an identical layout.
if: steps.cache-venv.outputs.cache-hit != 'true'
uses: astral-sh/setup-uv@c18668ad3cf93ea998bef934396af7bb5c839dc7 # v10.2.0
uses: astral-sh/setup-uv@20cfd1bf945f4377ade1205e4dbc17946fc9a30d # v10.0.1
with:
enable-cache: true
# Pull request saves land in per-PR scopes nothing else can
@@ -102,8 +102,6 @@ jobs:
device-builder: ${{ steps.determine.outputs.device-builder }}
esp32-platformio: ${{ steps.determine.outputs.esp32-platformio }}
esp32-platformio-components: ${{ steps.determine.outputs.esp32-platformio-components }}
esp8266-native: ${{ steps.determine.outputs.esp8266-native }}
esp8266-native-components: ${{ steps.determine.outputs.esp8266-native-components }}
changed-components: ${{ steps.determine.outputs.changed-components }}
changed-components-with-tests: ${{ steps.determine.outputs.changed-components-with-tests }}
directly-changed-components-with-tests: ${{ steps.determine.outputs.directly-changed-components-with-tests }}
@@ -167,8 +165,6 @@ jobs:
echo "device-builder=$(echo "$output" | jq -r '.device_builder')" >> $GITHUB_OUTPUT
echo "esp32-platformio=$(echo "$output" | jq -r '.esp32_platformio')" >> $GITHUB_OUTPUT
echo "esp32-platformio-components=$(echo "$output" | jq -r '.esp32_platformio_components')" >> $GITHUB_OUTPUT
echo "esp8266-native=$(echo "$output" | jq -r '.esp8266_native')" >> $GITHUB_OUTPUT
echo "esp8266-native-components=$(echo "$output" | jq -r '.esp8266_native_components')" >> $GITHUB_OUTPUT
echo "changed-components=$(echo "$output" | jq -c '.changed_components')" >> $GITHUB_OUTPUT
echo "changed-components-with-tests=$(echo "$output" | jq -c '.changed_components_with_tests')" >> $GITHUB_OUTPUT
echo "directly-changed-components-with-tests=$(echo "$output" | jq -c '.directly_changed_components_with_tests')" >> $GITHUB_OUTPUT
@@ -187,33 +183,6 @@ jobs:
path: .temp/components_graph.json
key: components-graph-${{ hashFiles('esphome/components/**/*.py') }}
seed-esp8266-native-cache:
name: Seed the esp8266 native toolchain cache
runs-on: ubuntu-24.04
needs:
- common
# PR-branch cache saves are invisible to other PRs, so dev pushes seed
# the shared entry the component matrix, the memory impact jobs and
# test-esp8266-native restore. Only dev: the composite action saves
# nowhere else, so a beta/release push would download the toolchain and
# discard it.
if: github.event_name == 'push' && github.ref == 'refs/heads/dev'
timeout-minutes: 15
steps:
- name: Check out code from GitHub
uses: actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1 # v7.0.1
- name: Restore Python
uses: ./.github/actions/restore-python
with:
python-version: ${{ env.DEFAULT_PYTHON }}
cache-key: ${{ needs.common.outputs.cache-key }}
- name: Cache the native toolchain
uses: ./.github/actions/cache-arduino8266
- name: Install the native toolchain
run: |
. venv/bin/activate
python -c "from esphome.arduino8266.framework import check_and_install; from esphome.components.esp8266 import RECOMMENDED_ARDUINO_FRAMEWORK_VERSION; check_and_install(RECOMMENDED_ARDUINO_FRAMEWORK_VERSION)"
ci-custom:
name: Run script/ci-custom
runs-on: ubuntu-24.04
@@ -286,7 +255,7 @@ jobs:
fi
echo "version=$version" >> "$GITHUB_OUTPUT"
- name: Run prek
uses: j178/prek-action@1382ad72dea72f9d53832f2bcb65b3e87a947204 # v3.0.1
uses: j178/prek-action@4e14d07f9231acabce116ccfca13b13dd9755ece # v3.0.0
with:
prek-version: ${{ steps.prek.outputs.version }}
# This job only runs on pull requests, so nothing ever populates
@@ -299,7 +268,7 @@ jobs:
# Pushes any fixes the hooks made back to the pull request. This step
# must keep its default name: the GitHub App that performs the push
# locates the workflow run by that name.
- uses: pre-commit-ci/lite-action@062bca0919bc9d6e66755cc05074b70c77e111fc # v1.2.0
- uses: pre-commit-ci/lite-action@5d6cc0eb514c891a40562a58a8e71576c5c7fb43 # v1.1.0
if: always()
with:
msg: apply automatic formatting fixes
@@ -352,7 +321,7 @@ jobs:
. venv/bin/activate
pytest -vv --cov-report=xml --tb=native --durations=30 -n auto tests --ignore=tests/integration/
- name: Upload coverage to Codecov
uses: codecov/codecov-action@303a32d7a59b442fa8d48b6a1cc6825c09c847a5 # v7.1.1
uses: codecov/codecov-action@fb8b3582c8e4def4969c97caa2f19720cb33a72f # v7.0.0
- name: Save Python virtual environment cache
if: github.ref == 'refs/heads/dev'
uses: actions/cache/save@55cc8345863c7cc4c66a329aec7e433d2d1c52a9 # v6.1.0
@@ -381,7 +350,7 @@ jobs:
- name: Check out code from GitHub
uses: actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1 # v7.0.1
- name: Report empty upload to Codecov
uses: codecov/codecov-action@303a32d7a59b442fa8d48b6a1cc6825c09c847a5 # v7.1.1
uses: codecov/codecov-action@fb8b3582c8e4def4969c97caa2f19720cb33a72f # v7.0.0
with:
run_command: empty-upload
force: true
@@ -400,9 +369,14 @@ jobs:
matrix:
bucket: ${{ fromJson(needs.determine-jobs.outputs.integration-test-buckets) }}
env:
# Registry libraries (noise-c, libsodium, ArduinoJson, lvgl) the host builds
# download, shared per xdist worker by tests/integration/conftest.py
INTEGRATION_LIBRARY_CACHE_PATH: ~/.esphome-integration-tests/pio_components
# What the cache steps persist; libdeps is excluded (keyed per xdist
# worker and env, it never crosses runs).
INTEGRATION_PIO_CACHE_PATH: |
~/.esphome-integration-tests/platformio/platforms
~/.esphome-integration-tests/platformio/packages
~/.esphome-integration-tests/platformio/appstate.json
~/.esphome-integration-tests/platformio/.cache
~/.esphome-integration-tests/platformio/.esphome.pio.stamp.json
steps:
- name: Check out code from GitHub
uses: actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1 # v7.0.1
@@ -422,13 +396,14 @@ jobs:
uses: actions/setup-python@5fda3b95a4ea91299a34e894583c3862153e4b97 # v7.0.0
with:
python-version: "3.13"
- name: Restore integration library cache
# A miss or a changed pin self-heals with a download, so no restore-keys
id: library-cache
- name: Restore integration PlatformIO cache
# Native platform + toolchain installed by shared_platformio_cache in
# tests/integration/conftest.py; a miss self-heals, so no restore-keys.
id: pio-cache
uses: actions/cache/restore@55cc8345863c7cc4c66a329aec7e433d2d1c52a9 # v6.1.0
with:
path: ${{ env.INTEGRATION_LIBRARY_CACHE_PATH }}
key: integration-libraries-v1-${{ runner.os }}-${{ hashFiles('esphome/components/json/__init__.py', 'esphome/components/noise/__init__.py', 'esphome/components/lvgl/__init__.py', 'esphome/components/improv_base/__init__.py') }}
path: ${{ env.INTEGRATION_PIO_CACHE_PATH }}
key: integration-pio-v1-${{ runner.os }}-py${{ steps.python.outputs.python-version }}-${{ hashFiles('requirements.txt', 'tests/integration/fixtures/cache_init.yaml', 'esphome/components/host/__init__.py') }}
- name: Restore Python virtual environment
id: cache-venv
uses: actions/cache@55cc8345863c7cc4c66a329aec7e433d2d1c52a9 # v6.1.0
@@ -438,7 +413,7 @@ jobs:
- name: Set up uv
# Only needed on cache miss to populate the venv.
if: steps.cache-venv.outputs.cache-hit != 'true'
uses: astral-sh/setup-uv@c18668ad3cf93ea998bef934396af7bb5c839dc7 # v10.2.0
uses: astral-sh/setup-uv@20cfd1bf945f4377ade1205e4dbc17946fc9a30d # v10.0.1
with:
enable-cache: true
# Pull request saves land in per-PR scopes nothing else can
@@ -496,16 +471,16 @@ jobs:
# A full cron period of margin for the weekly refresh
retention-days: 14
- name: Print ccache statistics
# esphome stores the host build's ccache under the machine-global
# cache dir (see get_build_env() in esphome/host/toolchain.py).
run: CCACHE_DIR="$HOME/.cache/esphome/host/ccache" ccache -s
- name: Save integration library cache
# esphome stores the PlatformIO ccache under the machine-global cache
# dir (see _ccache_env() in esphome/platformio/toolchain.py).
run: CCACHE_DIR="$HOME/.cache/esphome/platformio-ccache" ccache -s
- name: Save integration PlatformIO cache
# Bucket 0 only; the others would race the same immutable key.
if: success() && (github.ref == 'refs/heads/dev' || contains(github.event.pull_request.labels.*.name, 'ci-cache-write')) && strategy.job-index == 0 && steps.library-cache.outputs.cache-hit != 'true'
if: success() && (github.ref == 'refs/heads/dev' || contains(github.event.pull_request.labels.*.name, 'ci-cache-write')) && strategy.job-index == 0 && steps.pio-cache.outputs.cache-hit != 'true'
uses: actions/cache/save@55cc8345863c7cc4c66a329aec7e433d2d1c52a9 # v6.1.0
with:
path: ${{ env.INTEGRATION_LIBRARY_CACHE_PATH }}
key: ${{ steps.library-cache.outputs.cache-primary-key }}
path: ${{ env.INTEGRATION_PIO_CACHE_PATH }}
key: ${{ steps.pio-cache.outputs.cache-primary-key }}
import-time:
name: Check import esphome.__main__ time
@@ -626,7 +601,7 @@ jobs:
apt-get install -y libc6-dbg
- name: Run CodSpeed benchmarks
uses: CodSpeedHQ/action@c4fd08a3a159bd0cc208da1e0edf32b8c47d75e5 # v5.4.0
uses: CodSpeedHQ/action@373d6868929f444bc08d901fd0eb0ad52a8875ea # v5.2.1
with:
run: |
. venv/bin/activate
@@ -893,20 +868,14 @@ jobs:
matrix:
include:
- id: clang-tidy
name: Run script/clang-tidy for ESP32 IDF 1/5
options: --environment esp32-idf-tidy --split-num 5 --split-at 1
name: Run script/clang-tidy for ESP32 IDF 1/3
options: --environment esp32-idf-tidy --split-num 3 --split-at 1
- id: clang-tidy
name: Run script/clang-tidy for ESP32 IDF 2/5
options: --environment esp32-idf-tidy --split-num 5 --split-at 2
name: Run script/clang-tidy for ESP32 IDF 2/3
options: --environment esp32-idf-tidy --split-num 3 --split-at 2
- id: clang-tidy
name: Run script/clang-tidy for ESP32 IDF 3/5
options: --environment esp32-idf-tidy --split-num 5 --split-at 3
- id: clang-tidy
name: Run script/clang-tidy for ESP32 IDF 4/5
options: --environment esp32-idf-tidy --split-num 5 --split-at 4
- id: clang-tidy
name: Run script/clang-tidy for ESP32 IDF 5/5
options: --environment esp32-idf-tidy --split-num 5 --split-at 5
name: Run script/clang-tidy for ESP32 IDF 3/3
options: --environment esp32-idf-tidy --split-num 3 --split-at 3
steps:
- name: Check out code from GitHub
@@ -1128,11 +1097,6 @@ jobs:
uses: ./.github/actions/cache-sdk-nrf
with:
restore-only: true
- name: Cache the native ESP8266 toolchain
# Only batches whose test platforms include esp8266; never saves
# here, it reuses the install the dev seed job cached.
if: matrix.batch.needs_arduino8266
uses: ./.github/actions/cache-arduino8266
- name: Validate and compile components with intelligent grouping
run: |
. venv/bin/activate
@@ -1231,28 +1195,9 @@ jobs:
if [ -n "$compile_csv" ]; then
# Run compilation with grouping and isolation
# The bootloader has no ESPHome code and these builds never
# flash; the check_idf_py batch keeps the full build so the
# native sub-build and the equivalence check stay covered.
skip_flag="--skip-bootloader"
if [[ "${{ matrix.batch.check_idf_py }}" == "true" ]]; then
skip_flag=""
fi
python3 script/test_build_components.py -e compile -c "$compile_csv" -f --isolate "$directly_changed_csv" $skip_flag
if [[ "${{ matrix.batch.check_idf_py }}" == "true" ]]; then
# The real idf.py must find nothing to configure or build in a
# tree built above; catches drift on ESP-IDF bumps.
echo "Checking the native ESP-IDF build matches idf.py"
python3 script/check_idf_py_equivalence.py
fi
python3 script/test_build_components.py -e compile -c "$compile_csv" -f --isolate "$directly_changed_csv"
else
echo "All components in this batch are validate-only -- skipping compile stage."
if [[ "${{ matrix.batch.check_idf_py }}" == "true" ]]; then
# determine-jobs and this step disagree on what compiles; fail
# rather than let the check run nowhere.
echo "::error::This batch was picked for the idf.py check but compiled nothing"
exit 1
fi
fi
- name: Print ccache statistics
@@ -1292,7 +1237,7 @@ jobs:
# compile validates config first, so a separate config pass is
# redundant for this smoke test. ESP-IDF framework via PlatformIO:
python3 script/test_build_components.py -e compile -t esp32-idf -c "$TEST_COMPONENTS" -f --toolchain platformio --fail-on-no-tests
python3 script/test_build_components.py -e compile -t esp32-idf -c "$TEST_COMPONENTS" -f --toolchain platformio
echo ""
echo "ESP-IDF-via-PlatformIO build passed! Starting Arduino smoke test..."
@@ -1301,125 +1246,6 @@ jobs:
# Arduino framework via PlatformIO (only components with an esp32-ard test are built):
python3 script/test_build_components.py -e compile -t esp32-ard -c "$TEST_COMPONENTS" -f --toolchain platformio
test-esp8266-native:
name: Test esp8266 components with the native toolchain
runs-on: ubuntu-24.04
needs:
- common
- determine-jobs
if: github.event_name == 'pull_request' && needs.determine-jobs.outputs.esp8266-native == 'true'
env:
# Computed by script/determine-jobs.py (ESP8266_NATIVE_TEST_COMPONENTS)
TEST_COMPONENTS: ${{ needs.determine-jobs.outputs.esp8266-native-components }}
steps:
- name: Check out code from GitHub
uses: actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1 # v7.0.1
- name: Restore Python
uses: ./.github/actions/restore-python
with:
python-version: ${{ env.DEFAULT_PYTHON }}
cache-key: ${{ needs.common.outputs.cache-key }}
- name: Cache the native toolchain
uses: ./.github/actions/cache-arduino8266
- name: Run native toolchain compile test
run: |
. venv/bin/activate
echo "Testing components: $TEST_COMPONENTS"
echo ""
# ESP8266 Arduino built directly (no PlatformIO); compile validates
# config first, so a separate config pass is redundant.
python3 script/test_build_components.py -e compile -t esp8266-ard -c "$TEST_COMPONENTS" -f --toolchain arduino --fail-on-no-tests
test-esp8266-native-windows:
name: Compile esp8266 with the native toolchain on Windows (ccache ${{ matrix.ccache }})
runs-on: windows-latest
needs:
- common
- determine-jobs
if: github.event_name == 'pull_request' && needs.determine-jobs.outputs.esp8266-native == 'true'
# A hung compiler must fail the job, not hold the runner for six hours.
timeout-minutes: 40
strategy:
fail-fast: false
matrix:
# Both ways the build runs on Windows: ESPHome Desktop bundles
# ccache and puts it on PATH, a pip install has none.
ccache: ["1", "0"]
env:
ESPHOME_CCACHE_ENABLE: ${{ matrix.ccache }}
# The test script prints characters the cp1252 console cannot encode
PYTHONUTF8: "1"
# The exact static build ESPHome Desktop ships
CCACHE_URL: https://github.com/ccache/ccache/releases/download/v4.14/ccache-4.14-windows-x86_64.zip
CCACHE_SHA256: 2568347a697e103ca1b073981c704ad76fb2507d066c38dba038dd73399d968f
steps:
- name: Check out code from GitHub
uses: actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1 # v7.0.1
- name: Restore Python
uses: ./.github/actions/restore-python
with:
python-version: ${{ env.DEFAULT_PYTHON }}
cache-key: ${{ needs.common.outputs.cache-key }}
- name: Install ccache
if: matrix.ccache == '1'
shell: pwsh
run: |
$dir = Join-Path $env:RUNNER_TEMP "ccache"
New-Item -ItemType Directory -Force $dir | Out-Null
$zip = Join-Path $dir "ccache.zip"
Invoke-WebRequest -Uri $env:CCACHE_URL -OutFile $zip
$hash = (Get-FileHash $zip -Algorithm SHA256).Hash.ToLower()
if ($hash -ne $env:CCACHE_SHA256) {
Write-Host "::error::ccache archive sha256 $hash does not match $env:CCACHE_SHA256"
exit 1
}
Expand-Archive -Path $zip -DestinationPath $dir
$bin = Get-ChildItem -Path $dir -Recurse -Filter ccache.exe | Select-Object -First 1
Add-Content -Path $env:GITHUB_PATH -Value $bin.DirectoryName
& $bin.FullName --version
- name: Compile
shell: pwsh
# One component test: web_server brings wifi, api and the async web
# server libraries, the shape of a typical ESP8266 device
run: |
. .\venv\Scripts\Activate.ps1
$arguments = "script/test_build_components.py", "-e", "compile", "-t", "esp8266-ard", "-c", "web_server", "--base-only", "--toolchain", "arduino", "--fail-on-no-tests"
$p = Start-Process -FilePath python -ArgumentList $arguments -NoNewWindow -PassThru
# Reading the handle keeps ExitCode available after the process exits
$null = $p.Handle
if (-not $p.WaitForExit(25 * 60 * 1000)) {
Write-Host "::error::The compile did not finish in 25 minutes; processes still running:"
Get-CimInstance Win32_Process |
Where-Object { $_.CommandLine -match 'xtensa|ccache|ninja|esphome' } |
Select-Object ProcessId, ParentProcessId, CreationDate, CommandLine |
Format-List | Out-String -Width 4096 | Write-Host
Get-Process |
Where-Object { $_.Name -match 'cc1|ccache|ninja|xtensa|python' } |
Select-Object Id, Name, CPU, WorkingSet64 |
Format-Table | Out-String | Write-Host
taskkill /PID $p.Id /T /F
exit 1
}
exit $p.ExitCode
- name: Show ccache statistics
# Also after a failed or killed compile: the counters say whether
# ccache was ever reached
if: always() && matrix.ccache == '1'
shell: pwsh
# The build keeps its store under the esphome tools cache
run: |
$env:CCACHE_DIR = Join-Path $env:LOCALAPPDATA "esphome\Cache\arduino8266\ccache"
ccache -s
device-builder:
name: Test downstream esphome/device-builder
runs-on: ubuntu-24.04
@@ -1448,7 +1274,7 @@ jobs:
# install step (order-of-magnitude faster on cold boots,
# with its own wheel cache). actions/setup-python still
# provides the interpreter.
uses: astral-sh/setup-uv@c18668ad3cf93ea998bef934396af7bb5c839dc7 # v10.2.0
uses: astral-sh/setup-uv@20cfd1bf945f4377ade1205e4dbc17946fc9a30d # v10.0.1
with:
enable-cache: true
# Pull request saves land in per-PR scopes nothing else can
@@ -1507,27 +1333,31 @@ jobs:
echo "::warning::ci_memory_impact_extract.py not found on target branch, skipping memory impact analysis"
fi
# The build below uses --base-only, so skip unless the target branch has
# test.<platform>.yaml for every requested component (test-*.yaml
# variants alone compile nothing, and a partial set would compare a
# smaller target build against the PR build). Checked here, not in
# determine-jobs.py, because determine-jobs runs on the PR branch and
# cannot see the target branch.
# Check if test files exist on the target branch for the requested
# components and platform. When a PR adds new test files for a platform,
# the target branch won't have them yet, so skip instead of failing.
# This check must be done here (not in determine-jobs.py) because
# determine-jobs runs on the PR branch and cannot see what the target
# branch has.
- name: Check for test files on target branch
id: check-tests
if: steps.check-script.outputs.skip != 'true'
run: |
components='${{ toJSON(fromJSON(needs.determine-jobs.outputs.memory_impact).components) }}'
platform="${{ fromJSON(needs.determine-jobs.outputs.memory_impact).platform }}"
missing=""
found=false
for component in $(echo "$components" | jq -r '.[]'); do
if [ ! -f "tests/components/${component}/test.${platform}.yaml" ]; then
missing="${missing} ${component}"
fi
# Check for test files matching the platform (test.platform.yaml or test-*.platform.yaml)
for f in tests/components/${component}/test*.${platform}.yaml; do
if [ -f "$f" ]; then
found=true
break 2
fi
done
done
if [ -n "$missing" ]; then
if [ "$found" = false ]; then
echo "skip=true" >> $GITHUB_OUTPUT
echo "::warning::No test.${platform}.yaml on target branch for${missing}, skipping memory impact analysis"
echo "::warning::No test files found on target branch for platform ${platform}, skipping memory impact analysis"
else
echo "skip=false" >> $GITHUB_OUTPUT
fi
@@ -1585,9 +1415,12 @@ jobs:
python-version: ${{ env.DEFAULT_PYTHON }}
cache-key: ${{ needs.common.outputs.cache-key }}
- name: Cache the native ESP8266 toolchain
if: steps.check-script.outputs.skip != 'true' && steps.check-tests.outputs.skip != 'true' && steps.cache-memory-analysis.outputs.cache-hit != 'true' && fromJSON(needs.determine-jobs.outputs.memory_impact).needs_arduino8266
uses: ./.github/actions/cache-arduino8266
- name: Cache platformio
if: steps.check-script.outputs.skip != 'true' && steps.check-tests.outputs.skip != 'true' && steps.cache-memory-analysis.outputs.cache-hit != 'true'
uses: actions/cache/restore@55cc8345863c7cc4c66a329aec7e433d2d1c52a9 # v6.1.0
with:
path: ~/.platformio
key: platformio-memory-${{ fromJSON(needs.determine-jobs.outputs.memory_impact).platform }}-${{ hashFiles('platformio.ini') }}
- name: Build, compile, and analyze memory
if: steps.check-script.outputs.skip != 'true' && steps.check-tests.outputs.skip != 'true' && steps.cache-memory-analysis.outputs.cache-hit != 'true'
@@ -1672,9 +1505,11 @@ jobs:
with:
python-version: ${{ env.DEFAULT_PYTHON }}
cache-key: ${{ needs.common.outputs.cache-key }}
- name: Cache the native ESP8266 toolchain
if: fromJSON(needs.determine-jobs.outputs.memory_impact).needs_arduino8266
uses: ./.github/actions/cache-arduino8266
- name: Cache platformio
uses: actions/cache/restore@55cc8345863c7cc4c66a329aec7e433d2d1c52a9 # v6.1.0
with:
path: ~/.platformio
key: platformio-memory-${{ fromJSON(needs.determine-jobs.outputs.memory_impact).platform }}-${{ hashFiles('platformio.ini') }}
- name: Build, compile, and analyze memory
id: extract
run: |
@@ -1773,7 +1608,6 @@ jobs:
needs:
- common
- seed-apt-cache
- seed-esp8266-native-cache
- determine-jobs
- ci-custom
- pylint
@@ -1789,8 +1623,6 @@ jobs:
- clang-tidy-esp32-variants
- test-build-components-split
- test-esp32-platformio
- test-esp8266-native
- test-esp8266-native-windows
- device-builder
- memory-impact-target-branch
- memory-impact-pr-branch
+2 -2
View File
@@ -56,7 +56,7 @@ jobs:
# Initializes the CodeQL tools for scanning.
- name: Initialize CodeQL
uses: github/codeql-action/init@2892aa5e19bbd11bc0cff5427e3b750a04d9e3c2 # v4.38.2
uses: github/codeql-action/init@cdf488f595d80d6e07e03d4674febd5ab45fa938 # v4.37.9
with:
languages: ${{ matrix.language }}
build-mode: ${{ matrix.build-mode }}
@@ -84,6 +84,6 @@ jobs:
exit 1
- name: Perform CodeQL Analysis
uses: github/codeql-action/analyze@2892aa5e19bbd11bc0cff5427e3b750a04d9e3c2 # v4.38.2
uses: github/codeql-action/analyze@cdf488f595d80d6e07e03d4674febd5ab45fa938 # v4.37.9
with:
category: "/language:${{matrix.language}}"
+8 -14
View File
@@ -125,20 +125,14 @@ jobs:
}
async function getEsphomeAndComponentChanges(github, owner, repo, prNumber) {
const changedFiles = await github.paginate(
github.rest.pulls.listFiles,
{
owner: owner,
repo: repo,
pull_number: prNumber,
per_page: 100,
}
);
const changedFiles = await github.rest.pulls.listFiles({
owner: owner,
repo: repo,
pull_number: prNumber,
});
// Files used only for development and CI, which do not affect use as an external component
const ignoredFiles = ["esphome/core/defines.h", "esphome/idf_component.yml"];
const esphomeChanges = changedFiles
.filter(file => !ignoredFiles.includes(file.filename) && file.filename.startsWith('esphome/'))
const esphomeChanges = changedFiles.data
.filter(file => file.filename !== "esphome/core/defines.h" && file.filename.startsWith('esphome/'))
.map(file => {
const match = file.filename.match(/esphome\/([^/]+)/);
return match ? match[1] : null;
@@ -150,7 +144,7 @@ jobs:
}
const uniqueEsphomeChanges = [...new Set(esphomeChanges)];
const componentChanges = changedFiles
const componentChanges = changedFiles.data
.filter(file => file.filename.startsWith('esphome/components/'))
.map(file => {
const match = file.filename.match(/esphome\/components\/([^/]+)\//);
+1 -1
View File
@@ -14,4 +14,4 @@ jobs:
permissions:
issues: write # issues.lock on closed issues
pull-requests: write # issues.lock on closed pull requests
uses: esphome/workflows/.github/workflows/lock.yml@cc3e76de337dc59bc1cba8da58d963cd23b873f1 # 2026.9.0
uses: esphome/workflows/.github/workflows/lock.yml@0fdd5e311b7e744069166696072a1a9cbc5fbeb6 # 2026.8.1
+2 -2
View File
@@ -123,7 +123,7 @@ jobs:
python-version: "3.12"
- name: Set up Docker Buildx
uses: docker/setup-buildx-action@594f3bf4285d9ea8dc53c9a0c9c4092420091003 # v4.4.0
uses: docker/setup-buildx-action@37fe631027851001ddb9b187196cc803df7f5f0e # v4.3.0
- name: Log in to docker hub
uses: docker/login-action@dbcb813823bdd20940b903addbd779551569679f # v4.6.0
@@ -202,7 +202,7 @@ jobs:
merge-multiple: true
- name: Set up Docker Buildx
uses: docker/setup-buildx-action@594f3bf4285d9ea8dc53c9a0c9c4092420091003 # v4.4.0
uses: docker/setup-buildx-action@37fe631027851001ddb9b187196cc803df7f5f0e # v4.3.0
- name: Log in to docker hub
if: matrix.registry == 'dockerhub'
+1 -15
View File
@@ -16,18 +16,16 @@ jobs:
# No GITHUB_TOKEN permissions: the reusable workflow mints an ESPHome
# GitHub App token so the labels, comments and closures come from
# esphome[bot] instead of github-actions[bot].
uses: esphome/workflows/.github/workflows/stale.yml@4fbce13026c393d6eb6b247e394dcd9ef1af76c2 # main
uses: esphome/workflows/.github/workflows/stale.yml@a1c1485ab46ef41a84a6a9d8abd7fa4b7628fd70 # main
secrets:
ESPHOME_GITHUB_APP_PRIVATE_KEY: ${{ secrets.ESPHOME_GITHUB_APP_PRIVATE_KEY }}
with:
# Live only on dev: a workflow_dispatch from any other branch is a dry run
dry-run: ${{ github.ref != 'refs/heads/dev' }}
days-before-stale: 90
days-before-stale-conflicting: 30
days-before-close: 7
stale-label: stale
exempt-label: not-stale
conflict-label: needs-rebase
ignored-users: esphbot,codecov-commenter
stale-pr-message: >
There hasn't been any activity on this pull request recently. This
@@ -38,18 +36,6 @@ jobs:
to keep it open. Also, please merge the latest dev branch into your
branch to ensure that it's up to date with the latest changes.
Thank you for your contribution!
conflict-pr-message: >
This pull request has merge conflicts. Please merge the latest dev
branch into your branch to resolve them.
stale-conflicting-pr-message: >
The merge conflicts on this pull request have not been resolved for
30 days, so it has been marked as stale and will be closed if they
are not resolved within 7 days.
Please merge the latest dev branch into your branch. Once the pull
request is mergeable again the stale label is removed.
Thank you for your contribution!
stale-issue-message: >
There hasn't been any activity on this issue recently. Due to the
@@ -7,6 +7,10 @@ on:
permissions:
pull-requests: read # issues.listLabelsOnIssue to detect blocking labels (needs-docs, needs-developer-docs, merge-after-release, chained-pr)
concurrency:
group: ${{ github.workflow }}-${{ github.event.pull_request.number }}
cancel-in-progress: true
jobs:
check:
name: Check blocking labels
@@ -15,8 +15,6 @@ on:
# base branch checkout, so fork code never runs with the token.
pull_request_target:
types: [opened, synchronize, reopened]
branches:
- dev
paths:
- requirements_dev.txt
- requirements_test.txt
+1 -1
View File
@@ -47,7 +47,7 @@ jobs:
# setup-python interpreter so subsequent ``prek`` /
# ``script/run-in-env.py`` steps find the deps without a
# ``uv run`` prefix.
uses: astral-sh/setup-uv@c18668ad3cf93ea998bef934396af7bb5c839dc7 # v10.2.0
uses: astral-sh/setup-uv@20cfd1bf945f4377ade1205e4dbc17946fc9a30d # v10.0.1
with:
enable-cache: true
# Pin uv version so the action does not have to fetch the
+2 -2
View File
@@ -10,7 +10,7 @@ ci:
repos:
- repo: https://github.com/astral-sh/ruff-pre-commit
# Ruff version.
rev: v0.16.10
rev: v0.16.6
hooks:
# Run the linter.
- id: ruff
@@ -18,7 +18,7 @@ repos:
# Run the formatter.
- id: ruff-format
- repo: https://github.com/PyCQA/flake8
rev: 7.4.1
rev: 7.3.0
hooks:
- id: flake8
additional_dependencies:
+3 -134
View File
@@ -67,7 +67,6 @@ file does, and it is the authority when they disagree. The most useful starting
- Function-local constants: `lower_snake_case`
- Protected/private fields: `lower_snake_case_with_trailing_underscore_`
- Favor descriptive names over abbreviations
- Device components are named brand plus model (`siemens_ira211`), without a protocol-family prefix
- Enumerator names: prefix every value of an `enum class` with the enum name converted to
`UPPER_SNAKE_CASE` (e.g. `UARTFlushResult::UART_FLUSH_RESULT_SUCCESS`). Never use bare
names like `SUCCESS`, `FAILURE`, `OK`, or `FAIL`: platform SDK headers define macros with
@@ -178,15 +177,6 @@ file does, and it is the authority when they disagree. The most useful starting
Pick the primitive by cadence: under 250 ms use a gated `loop()`; 500 ms and above use
`set_interval`. Full reasoning, including why `set_interval` costs more below 500 ms:
https://developers.esphome.io/architecture/components/advanced/#quick-rule-of-thumb
* **Repeating work:** use `set_interval`, never a `set_timeout` that re-arms itself; every re-arm
rebuilds the callback and re-queues a scheduler item, and the period drifts.
* **Scheduler ids:** name a timer only when it must be cancelled or replaced, and use a
`static constexpr uint32_t` id, never a string. Ids are per component instance and cannot clash with other
components, so number them from 0 and keep all of a component's ids together in one place.
```cpp
static constexpr uint32_t READ_TIMEOUT_ID = 0;
this->set_timeout(READ_TIMEOUT_ID, 50, [this]() { this->read_(); });
```
* **Don't override a default with the same value:** if a base class method already returns what you
want, do not override it. `Component::get_setup_priority()` returns `setup_priority::DATA`, so a
component that wants `DATA` should simply leave it alone.
@@ -194,33 +184,6 @@ file does, and it is the authority when they disagree. The most useful starting
// Bad - this is exactly what the base class already does
float get_setup_priority() const override { return setup_priority::DATA; }
```
* **Constant data lives in RAM on ESP8266** unless marked `PROGMEM`: `const` tables, class-scope
constant arrays, and in-class `std::array` initializers (a `.rodata` template plus a copy). Mark the
table `PROGMEM` and read it with `progmem_memcpy()` or `progmem_read_byte()` (`esphome/core/hal.h`);
for a table of structs copy the whole entry out and return the copy, never a pointer into flash.
On other platforms they are a plain `memcpy` and dereference, so no `#ifdef` is needed. Declare log tags with
`ESPHOME_LOG_TAG(TAG, "name")`, pass status text as `LOG_STR`, e.g.
`this->mark_failed(LOG_STR(ESP_LOG_MSG_COMM_FAIL))`, and give an action whose payload is a byte array
a `TEMPLATABLE_BYTES` field so a static code stays in flash.
* **Fixed sets of names** (status words, model names): a `PROGMEM_STRING_TABLE` (`esphome/core/progmem.h`),
logged with `get_log_str` and `LOG_STR_ARG`, copied out with `ESPHOME_strncpy_P` before `publish_state`.
A `dump_config` helper that picks a name returns `const LogString *` from `LOG_STR(...)`.
* **CRCs and bit tricks** come from `esphome/core/helpers.h` (`crc8`, `crc16`, `reverse_bits`, `encode_uint16`);
pick the helper's polynomial and bit-order arguments instead of writing a CRC loop.
* **An unset float is `NAN`**, not `optional<float>`; the optional costs 4 to 8 bytes of padding per member.
* **`PollingComponent` first runs `update()`** after a random share of the interval; call `this->update()` at the
end of `setup()` when the entities need a state at boot.
* **A multi-step command sequence** leaves the step pending when the answer is malformed, so the existing
resend and give-up logic handles it; never advance on an answer that was only logged.
* **Entities that each watch a register field:** one class taking register, shift, mask and match from a Python
tuple table, in typed `StaticVector`s sized by `cg.slot_counter`; not a template per bit, not a virtual
listener base (a second vptr per entity).
* **A global feature define** says the code is compiled in, not that this instance uses it: keep the
per-instance check (`if (this->aux_device_ == ...)`) inside the `#ifdef`.
* **Failures after `setup()`:** `mark_failed()` is for `setup()` only. A runtime read or write failure
sets `status_set_warning()` and the next success clears it; writes skip work when `is_failed()`.
* **Fixed platform facts compile to nothing:** gate on `#ifdef USE_ESP8266` and friends, not on a
runtime check of something the build already knows.
* **Logging string literals:** wrap literals passed as `%s` arguments in `LOG_STR_LITERAL()` so they
can be stored in flash rather than RAM.
```cpp
@@ -267,10 +230,6 @@ file does, and it is the authority when they disagree. The most useful starting
```
* **Component Metadata:**
- `DOMAIN`: Required in every component's `__init__.py`, set to the component's own name as a plain
string (e.g. `DOMAIN = "my_component"`). Other code refers to the component through it, e.g.
`from esphome.components.my_component import DOMAIN`. Keep it in alphabetical order with the other
metadata constants. CI (`lint_component_domain`) fails without it.
- `DEPENDENCIES`: List of required components
- `AUTO_LOAD`: Components to automatically load
- `CONFLICTS_WITH`: Incompatible components
@@ -284,7 +243,6 @@ file does, and it is the authority when they disagree. The most useful starting
import esphome.config_validation as cv
from esphome.const import CONF_KEY, CONF_ID
DOMAIN = "my_component"
CONF_PARAM = "param" # A constant that does not yet exist in esphome/const.py
my_component_ns = cg.esphome_ns.namespace("my_component")
@@ -364,25 +322,6 @@ file does, and it is the authority when they disagree. The most useful starting
var = await switch.new_switch(config)
```
- **Optional child entities of a hub:** bind the config once with `sensor.sub_sensors(config)` (or
`sub_binary_sensors`, `sub_text_sensors`, `sub_buttons`, `sub_switches`, `sub_numbers`,
`sub_selects` in their domains), adding `parent=hub` for entities that derive from `Parented<T>`,
then make one call per key, even when there is only one. A call creates the entity only when its key
is configured, passes it to the setter and returns it (or `None`); extra arguments such as
`min_value` or `options` go on the call. Always name the setter explicitly on the object that owns
it, never with `getattr` and an f-string, and keep that variable short (`var` for the component
itself, `hub` for one fetched with `cg.get_variable`) so the calls fit on one line. Loops whose
setter also takes an index, such as `set_gate_threshold(x, n)`, stay as they are.
```python
async def to_code(config):
var = cg.new_Pvariable(config[CONF_ID])
sensors = sensor.sub_sensors(config)
await sensors(CONF_TEMPERATURE, var.set_temperature_sensor)
await sensors(CONF_HUMIDITY, var.set_humidity_sensor)
buttons = button.sub_buttons(config, parent=var)
await buttons(CONF_RESTART, var.set_restart_button)
```
* **Automations (Triggers, Actions, Conditions):**
Automations have three building blocks: **Triggers** (fire when something happens), **Actions** (do something), and **Conditions** (check if something is true).
@@ -426,25 +365,6 @@ file does, and it is the authority when they disagree. The most useful starting
)
```
When the callback's parameters differ from the automation's arguments, or the trigger should only
fire for some values, pass `params`, `forward` and `when`; the helper then generates a capture-less
lambda (stored inline, nothing allocated). Name the parent with `automation.parent_ref(var)` and build
`forward` from `MockObj` calls, not f-strings of C++:
```python
parent = automation.parent_ref(var)
index = cg.RawExpression("index")
await automation.build_callback_automation(
var,
"add_on_state_callback",
[(cg.StringRef, "x"), (cg.size_t, "i")],
conf,
params=[(cg.size_t, "index")],
forward=[cg.StringRef(parent.option_at(index)), index],
)
```
`build_parent_callback_automation`, `build_trigger_callback`, `build_callback_automations` and
`build_trigger_automations` cover the other shapes; see their docstrings in `esphome/automation.py`.
**C++ -- no trigger class needed.** The callback registration method must be templatized to accept both `std::function` and lightweight forwarder structs (which avoid heap allocation):
```cpp
class MyComponent : public Component {
@@ -511,31 +431,7 @@ file does, and it is the authority when they disagree. The most useful starting
MyComponent *parent_;
};
```
Register it without writing a builder:
```python
automation.register_simple_action(
"my_component.do_something", MyAction, schema, synchronous=True
)
```
The constructor receives the object named by `config[CONF_ID]`. Use `register_bare_action` for a
no-argument constructor, `register_parented_action` for a class deriving from `Parented<T>`, and
the `@automation.register_action(...)` decorator only when the builder must also set fields.
Use `synchronous=True` for actions that run to completion inside `play()` without deferring. Use `synchronous=False` if the action may suspend/defer execution (e.g. `delay`, `wait_until`, `script.wait`) or store trigger arguments for later use.
**Actions that only forward templatable values to their parent need no C++ class.** Register them
with `register_apply_action`; do not write a `TEMPLATABLE_VALUE` class or a builder for this shape.
```python
automation.register_apply_action(
"my_component.set_gains",
schema,
automation.ApplyField(CONF_KP, "set_kp", cg.float_),
automation.ApplyField(CONF_KI, "set_ki", cg.float_),
)
```
The `ApplyField`, `ApplyCall` and `register_apply_action` docstrings in `esphome/automation.py` cover
the rest; `cover.control` and `cover.template.publish` are in-tree examples. `TEMPLATABLE_VALUE` with
`cg.templatable` stays for actions whose `play()` has real logic beyond forwarding values.
Register with `@automation.register_action("my_component.do_something", MyAction, schema, synchronous=True)`. Use `synchronous=True` for actions that run to completion inside `play()` without deferring. Use `synchronous=False` if the action may suspend/defer execution (e.g. `delay`, `wait_until`, `script.wait`) or store trigger arguments for later use.
* **Conditions:**
```cpp
@@ -547,24 +443,9 @@ file does, and it is the authority when they disagree. The most useful starting
MyComponent *parent_;
};
```
Register with `automation.register_simple_condition("my_component.is_active", MyCondition, schema)`;
`register_bare_condition`, `register_parented_condition` and the decorator follow the action rules.
**Conditions that only test their parent need no C++ class either.** Register them with
`register_apply_condition`; the expression is applied to the parent, and an `ApplyCall` compares
against config values.
```python
automation.register_apply_condition("my_component.is_active", schema, "is_active()")
automation.register_apply_condition(
"my_component.state_is",
schema,
automation.ApplyCall("state == {}", ((CONF_STATE, cg.bool_),)),
)
```
`cover.is_open`, `rtttl.is_playing` and `component.is_idle` are in-tree examples.
Register with `@automation.register_condition("my_component.is_active", MyCondition, schema)`.
* **Type Hints:** Type-hint all function signatures, including test functions and config validators (e.g. `def validate_x(config: ConfigType) -> ConfigType:`, `def test_x() -> None:`). Import `ConfigType` from `esphome.types`.
An intentionally empty body (an unused `register_dumper` builder, for example) gets a one-line docstring, not `pass`.
* **Configuration Validation:**
* **Reuse existing validators:** Before writing a custom validator, check for an existing one in `config_validation.py` and compose it in `cv.All(...)` rather than duplicating logic across components. For example, rename a config key with `cv.rename_key(CONF_OLD, CONF_NEW, removed_in="2026.6.0")`, and reject mutually-exclusive keys with `cv.has_at_most_one_key(...)` / `cv.has_exactly_one_key(...)`. See how `api` composes `cv.has_exactly_one_key` + `cv.rename_key`.
@@ -600,15 +481,6 @@ file does, and it is the authority when they disagree. The most useful starting
* **Testing:**
* **Python:** Run unit tests with `pytest`.
* **C++:** Use `clang-tidy` for static analysis.
* **C++ unit tests:** gtests in `tests/components/<component>/*.cpp`, run with
`script/cpp_unit_test.py <component>`; see `tests/components/README.md`. Set the loop clock in a test
with `LoopBlockingGuard{nullptr, nullptr, now_ms}`; the runner resets it before every test.
* **Integration tests:** `tests/integration/` runs real firmware on the host. Serial components prove
timeouts and recovery with the `uart_mock` fixture (`tests/integration/test_uart_mock_*.py`). Its
`responses` match on content every time they see it; to answer a repeated request differently, use an
`on_tx` lambda with a static counter and `inject_to_rx_buffer` (see `uart_mock_ld2412_requery_after_giveup.yaml`).
* **Keep build-test variants few:** one `test.<platform>.yaml` per platform the code actually differs
on. A variant per board for platform-identical code only adds CI time.
* **Component Tests:** YAML-based compilation tests are located in `tests/`. The structure is as follows:
```
tests/
@@ -655,7 +527,6 @@ file does, and it is the authority when they disagree. The most useful starting
name: My Sensor
```
Components that define buses directly are flagged as "NEEDS MIGRATION" and cannot be grouped, increasing CI build time.
`!extend` and `!remove` in a `test.*.yaml` also make the component ungroupable; add a standalone entity with its own id instead.
* **Testing All Components Together:** To verify that all components can be tested together without ID conflicts or configuration issues, use:
```bash
@@ -828,9 +699,7 @@ file does, and it is the authority when they disagree. The most useful starting
6. **Avoid `std::deque`:** It allocates in 512-byte blocks regardless of element size, guaranteeing at least 512 bytes of RAM usage immediately. This is a major source of crashes on memory-constrained devices.
7. **Never use `new (std::nothrow)`:** On ESP-IDF exceptions are disabled, so a failed nothrow allocation aborts instead of returning `nullptr`. Use `RAMAllocator` from `esphome/core/helpers.h`; CI rejects `std::nothrow`.
8. **Detection:** Look for these patterns in compiler output:
7. **Detection:** Look for these patterns in compiler output:
- Large code sections with STL symbols (vector, map, set)
- `alloc`, `realloc`, `dealloc` in symbol names
- `_M_realloc_insert`, `_M_default_append` (vector reallocation)
+4 -66
View File
@@ -39,12 +39,9 @@ esphome/components/am43/sensor/* @buxtronix
esphome/components/analog_threshold/* @ianchi
esphome/components/animation/* @syndlex
esphome/components/anova/* @buxtronix
esphome/components/apc1/* @ademuri
esphome/components/apds9306/* @aodrenah
esphome/components/apds9930/* @iyesin
esphome/components/api/* @esphome/core
esphome/components/aqi/* @freekode @jasstrong @ximex
esphome/components/aranet/* @patrickod
esphome/components/as5600/* @ammmze
esphome/components/as5600/sensor/* @ammmze
esphome/components/as7341/* @mrgnr
@@ -68,7 +65,6 @@ esphome/components/bedjet/climate/* @jhansche
esphome/components/bedjet/fan/* @jhansche
esphome/components/bedjet/sensor/* @javawizard @jhansche
esphome/components/beken_spi_led_strip/* @Mat931
esphome/components/bh1745/* @latonita
esphome/components/bh1750/* @OttoWinter
esphome/components/bh1900nux/* @B48D81EFCC
esphome/components/binary_sensor/* @esphome/core
@@ -104,9 +100,6 @@ esphome/components/bmp581_i2c/* @danielkent-net @kahrendt
esphome/components/bmp581_spi/* @danielkent-net @kahrendt
esphome/components/bp1658cj/* @Cossid
esphome/components/bp5758d/* @Cossid
esphome/components/bq25186/* @Rapsssito
esphome/components/bq27220/* @rggammon
esphome/components/bq76972/* @limpkin
esphome/components/bridge/* @kbx81
esphome/components/bthome_mithermometer/* @nagyrobi
esphome/components/button/* @esphome/core
@@ -132,7 +125,6 @@ esphome/components/combination/* @Cat-Ion @kahrendt
esphome/components/const/* @esphome/core
esphome/components/coolix/* @glmnet
esphome/components/copy/* @OttoWinter
esphome/components/counter/* @clydebarrow
esphome/components/cover/* @esphome/core
esphome/components/cs5460a/* @balrog-kun
esphome/components/cse7761/* @berfenger
@@ -163,10 +155,8 @@ esphome/components/ds1307/* @badbadc0ffee
esphome/components/ds1603l/* @JakeLC15
esphome/components/ds2484/* @mrk-its
esphome/components/ds248x/* @tomwellnitz
esphome/components/ds3231/* @linkedupbits
esphome/components/dsmr/* @glmnet @PolarGoose
esphome/components/duty_time/* @dudanov
esphome/components/ecocomfort2/* @gledian
esphome/components/ee895/* @Stock-M
esphome/components/ektf2232/touchscreen/* @jesserockz
esphome/components/emc2101/* @ellull
@@ -192,29 +182,24 @@ esphome/components/esp32_camera_web_server/* @ayufan
esphome/components/esp32_can/* @Sympatron
esphome/components/esp32_hosted/* @swoboda1337
esphome/components/esp32_hosted/update/* @swoboda1337
esphome/components/esp32_power_management/* @rwrozelle
esphome/components/esp32_improv/* @jesserockz
esphome/components/esp32_rmt/* @jesserockz
esphome/components/esp32_rmt_led_strip/* @jesserockz
esphome/components/esp8266/* @esphome/core
esphome/components/esp_ldo/* @clydebarrow
esphome/components/espectre/* @francescopace
esphome/components/esphome/ota/* @esphome/core
esphome/components/espnow/* @jesserockz
esphome/components/espnow/packet_transport/* @EasilyBoredEngineer
esphome/components/ethernet_info/* @gtjadsonsantos
esphome/components/event/* @nohat
esphome/components/exponential_moving_average/* @clydebarrow
esphome/components/exposure_notifications/* @OttoWinter
esphome/components/ezo/* @ssieb
esphome/components/ezo_pmp/* @carlos-sarmiento
esphome/components/ezsp_proxy_tap/* @kbx81 @puddly
esphome/components/factory_reset/* @anatoly-savchenkov
esphome/components/fastled_base/* @OttoWinter
esphome/components/feedback/* @ianchi
esphome/components/file/* @esphome/core
esphome/components/fingerprint_grow/* @alexborro @loongyh @OnFreund
esphome/components/font/* @clydebarrow @esphome/core
esphome/components/friedrich/* @rwrozelle
esphome/components/fs3000/* @kahrendt
esphome/components/ft5x06/* @clydebarrow
esphome/components/ft63x6/* @gpambrozio
@@ -260,11 +245,8 @@ esphome/components/hmac_md5/* @dwmw2
esphome/components/hmac_sha256/* @dwmw2
esphome/components/hoermann_hcp/* @zweckj
esphome/components/homeassistant/* @esphome/core @OttoWinter
esphome/components/homeassistant/button/* @jesserockz
esphome/components/homeassistant/number/* @landonr
esphome/components/homeassistant/select/* @jesserockz
esphome/components/homeassistant/switch/* @Links2004
esphome/components/homeassistant/text/* @jesserockz
esphome/components/honeywell_hih_i2c/* @Benichou34
esphome/components/honeywellabp/* @RubyBailey
esphome/components/honeywellabp2_i2c/* @jpfaff
@@ -276,7 +258,6 @@ esphome/components/http_request/ota/* @oarcher
esphome/components/http_request/update/* @jesserockz
esphome/components/htu31d/* @betterengineering
esphome/components/hub75/* @stuartparmenter
esphome/components/husb238/* @latonita
esphome/components/hydreon_rgxx/* @functionpointer
esphome/components/hyt271/* @Philippe12
esphome/components/i2c/* @esphome/core
@@ -284,12 +265,9 @@ esphome/components/i2c_device/* @gabest11
esphome/components/i2s_audio/* @jesserockz
esphome/components/i2s_audio/microphone/* @jesserockz
esphome/components/i2s_audio/speaker/* @jesserockz @kahrendt
esphome/components/i2s_clockless_led_strip/* @j9brown
esphome/components/iaqcore/* @yozik04
esphome/components/icnt86/* @danepowell
esphome/components/ili9xxx/* @clydebarrow @nielsnl68
esphome/components/improv_base/* @esphome/core
esphome/components/improv_ble/* @jesserockz
esphome/components/improv_serial/* @esphome/core
esphome/components/ina226/* @latonita @Sergio303
esphome/components/ina260/* @mreditor97
@@ -302,7 +280,6 @@ esphome/components/inkplate/* @jesserockz @JosipKuci
esphome/components/integration/* @OttoWinter
esphome/components/internal_temperature/* @Mat931
esphome/components/interval/* @esphome/core
esphome/components/ir_rf_base/* @bdraco @kbx81
esphome/components/ir_rf_proxy/* @kbx81
esphome/components/it8951/* @koosoli @limengdu @Passific
esphome/components/jsn_sr04t/* @Mafus1
@@ -314,11 +291,9 @@ esphome/components/kuntze/* @ssieb
esphome/components/lc709203f/* @ilikecake
esphome/components/lcd_menu/* @numo68
esphome/components/ld2410/* @regevbr @sebcaps
esphome/components/ld2410s/* @NovakIrs
esphome/components/ld2412/* @Rihan9
esphome/components/ld2420/* @descipher
esphome/components/ld2450/* @hareeshmu
esphome/components/ld2460/* @jesusvallejo
esphome/components/ld24xx/* @kbx81
esphome/components/ld6002b/* @hepter
esphome/components/ledc/* @OttoWinter
@@ -327,10 +302,6 @@ esphome/components/libretiny_pwm/* @kuba2k2
esphome/components/light/* @esphome/core
esphome/components/lightwaverf/* @max246
esphome/components/lilygo_t5_47/touchscreen/* @jesserockz
esphome/components/lis2dw12/* @latonita
esphome/components/lis2dw12_i2c/* @latonita
esphome/components/lis2dw12_spi/* @latonita
esphome/components/lis3dh/* @zebble
esphome/components/lm75b/* @beormund
esphome/components/ln882h_ble/* @Bl00d-B0b
esphome/components/ln882h_ble_tracker/* @Bl00d-B0b
@@ -338,7 +309,6 @@ esphome/components/ln882x/* @lamauny
esphome/components/lock/* @esphome/core
esphome/components/logger/* @esphome/core
esphome/components/logger/select/* @clydebarrow
esphome/components/lp5562/* @DennisGaida
esphome/components/lps22/* @nagisa
esphome/components/lsm6ds/* @clydebarrow
esphome/components/ltr390/* @latonita @sjtrny
@@ -349,9 +319,7 @@ esphome/components/m5stack_8angle/* @rnauber
esphome/components/mapping/* @clydebarrow
esphome/components/matrix_keypad/* @ssieb
esphome/components/max17043/* @blacknell
esphome/components/max17048/* @B48D81EFCC
esphome/components/max31865/* @DAVe3283
esphome/components/max31888/* @JiriPrchal
esphome/components/max44009/* @berfenger
esphome/components/max6956/* @looping40
esphome/components/max7219digit/* @rspaargaren
@@ -370,6 +338,7 @@ esphome/components/mcp4461/* @p1ngb4ck
esphome/components/mcp4728/* @berfenger
esphome/components/mcp47a1/* @jesserockz
esphome/components/mcp9600/* @mreditor97
esphome/components/mcp9808/* @k7hpn
esphome/components/md5/* @esphome/core
esphome/components/mdns/* @esphome/core
esphome/components/media_player/* @jesserockz
@@ -383,7 +352,6 @@ esphome/components/midea_ir/* @dudanov
esphome/components/mipi_dsi/* @clydebarrow
esphome/components/mipi_rgb/* @clydebarrow
esphome/components/mipi_spi/* @clydebarrow
esphome/components/mistral_ir/* @lerhum
esphome/components/mitsubishi/* @RubyBailey
esphome/components/mitsubishi_cn105/* @crnjan
esphome/components/mixer/speaker/* @kahrendt
@@ -422,8 +390,6 @@ esphome/components/noblex/* @AGalfra
esphome/components/noise/* @esphome/core
esphome/components/npi19/* @bakerkj
esphome/components/nrf52/* @tomaszduda23
esphome/components/ns2009/* @tchilov
esphome/components/ns2009/touchscreen/* @tchilov
esphome/components/number/* @esphome/core
esphome/components/one_wire/* @ssieb
esphome/components/online_image/* @clydebarrow @guillempages
@@ -438,7 +404,6 @@ esphome/components/pca9554/* @bdraco @clydebarrow @hwstar
esphome/components/pcf85063/* @brogon
esphome/components/pcf8563/* @KoenBreeman
esphome/components/pcm5122/* @remcom
esphome/components/pd_pioneer_ir/* @martinsah
esphome/components/pi4ioe5v6408/* @jesserockz
esphome/components/pid/* @OttoWinter
esphome/components/pipsolar/* @andreashergert1984
@@ -456,8 +421,6 @@ esphome/components/pn7150_i2c/* @jesserockz @kbx81
esphome/components/pn7160/* @jesserockz @kbx81
esphome/components/pn7160_i2c/* @jesserockz @kbx81
esphome/components/pn7160_spi/* @jesserockz @kbx81
esphome/components/pn71xx/* @jesserockz @kbx81
esphome/components/power_management/* @rwrozelle
esphome/components/power_supply/* @esphome/core
esphome/components/preferences/* @esphome/core
esphome/components/provisioning/* @esphome/core
@@ -465,7 +428,6 @@ esphome/components/psram/* @esphome/core
esphome/components/pulse_meter/* @cstaahl @stevebaxter @TrentHouliston
esphome/components/pvvx_mithermometer/* @pasiz
esphome/components/pylontech/* @functionpointer
esphome/components/pzem6l24/* @nuttytree
esphome/components/qmi8658/* @clydebarrow
esphome/components/qmp6988/* @andrewpc
esphome/components/qr_code/* @wjtje
@@ -478,8 +440,6 @@ esphome/components/rc522/* @glmnet
esphome/components/rc522_i2c/* @glmnet
esphome/components/rc522_spi/* @glmnet
esphome/components/rd03d/* @jasstrong
esphome/components/remko_ar715/* @naKruul
esphome/components/renogy_inverter_ble/* @emilioaray-dev
esphome/components/resampler/speaker/* @kahrendt
esphome/components/restart/* @esphome/core
esphome/components/rf_bridge/* @jesserockz
@@ -496,10 +456,8 @@ esphome/components/rtl87xx/* @kuba2k2
esphome/components/rtttl/* @glmnet @ximex
esphome/components/runtime_image/* @clydebarrow @guillempages @kahrendt
esphome/components/runtime_stats/* @bdraco
esphome/components/rx8025t/* @remcom
esphome/components/rx8130/* @beormund
esphome/components/safe_mode/* @jsuanet @kbx81 @paulmonigatti
esphome/components/samsung/* @jorofi
esphome/components/scd4x/* @martgras @sjtrny
esphome/components/script/* @esphome/core
esphome/components/sdl/* @bdm310 @clydebarrow
@@ -519,7 +477,6 @@ esphome/components/sendspin/image/* @kahrendt
esphome/components/sendspin/media_player/* @kahrendt
esphome/components/sendspin/media_source/* @kahrendt
esphome/components/sendspin/sensor/* @kahrendt
esphome/components/sendspin/switch/* @kahrendt
esphome/components/sendspin/text_sensor/* @kahrendt
esphome/components/sensirion_common/* @martgras
esphome/components/sensor/* @esphome/core
@@ -528,14 +485,11 @@ esphome/components/sfa30/* @ghsensdev
esphome/components/sfa40/* @NoQuarrel
esphome/components/sgp40/* @SenexCrenshaw
esphome/components/sgp4x/* @martgras @SenexCrenshaw
esphome/components/sh1122_base/* @pl4nkton
esphome/components/sh1122_spi/* @pl4nkton
esphome/components/sha256/* @esphome/core
esphome/components/shelly_dimmer/* @edge90 @rnauber
esphome/components/sht3xd/* @mrtoy-me
esphome/components/sht4x/* @sjtrny
esphome/components/shutdown/* @esphome/core @jsuanet
esphome/components/siemens_ira211/* @fonix232
esphome/components/sigma_delta_output/* @Cat-Ion
esphome/components/sim800l/* @glmnet
esphome/components/sm10bit_base/* @Cossid
@@ -552,7 +506,6 @@ esphome/components/sound_level/* @kahrendt
esphome/components/spa06_base/* @danielkent-net
esphome/components/spa06_i2c/* @danielkent-net
esphome/components/spa06_spi/* @danielkent-net
esphome/components/spd2010/* @grischard
esphome/components/speaker/* @jesserockz @kahrendt
esphome/components/speaker/media_player/* @kahrendt @synesthesiam
esphome/components/speaker_source/* @kahrendt
@@ -582,7 +535,6 @@ esphome/components/st7735/* @SenexCrenshaw
esphome/components/st7789v/* @kbx81
esphome/components/st7920/* @marsjan155
esphome/components/statsd/* @Links2004
esphome/components/stcc4/* @j9brown
esphome/components/stts22h/* @B48D81EFCC
esphome/components/substitutions/* @esphome/core
esphome/components/sun/* @OttoWinter
@@ -593,38 +545,29 @@ esphome/components/sx126x/* @swoboda1337
esphome/components/sx127x/* @swoboda1337
esphome/components/sy6970/* @linkedupbits
esphome/components/syslog/* @clydebarrow
esphome/components/systa_bus/* @Mat931
esphome/components/t6615/* @tylermenezes
esphome/components/tas2780/* @remcom
esphome/components/tas58xx/* @mrtoy-me @remcom
esphome/components/tc74/* @sethgirvan
esphome/components/tca8418/* @zebble
esphome/components/tca9548a/* @andreashergert1984
esphome/components/tca9555/* @mobrembski
esphome/components/tcl112/* @glmnet
esphome/components/tcp_uart/* @Bascht74
esphome/components/tee501/* @Stock-M
esphome/components/teleinfo/* @0hax @FredM67
esphome/components/teleinfo/* @0hax
esphome/components/tem3200/* @bakerkj
esphome/components/template/alarm_control_panel/* @grahambrown11 @hwstar
esphome/components/template/datetime/* @rfdarter
esphome/components/template/event/* @nohat
esphome/components/template/fan/* @ssieb
esphome/components/text/* @mauritskorse
esphome/components/tfluna/* @candrews
esphome/components/thermopro_ble/* @sittner
esphome/components/thermostat/* @kbx81
esphome/components/time/* @esphome/core
esphome/components/tinyusb/* @kbx81
esphome/components/tinyusb_keyboard/* @frederikbolding
esphome/components/tlc5947/* @rnauber
esphome/components/tlc5971/* @IJIJI
esphome/components/tm1621/* @Philippe12
esphome/components/tm1637/* @glmnet
esphome/components/tm1638/* @skykingjwc
esphome/components/tm1651/* @mrtoy-me
esphome/components/tmc2209/* @remcom
esphome/components/tmc22xx/* @remcom
esphome/components/tmp102/* @timsavage
esphome/components/tmp1075/* @sybrenstuvel
esphome/components/tmp117/* @Azimath
@@ -646,14 +589,11 @@ esphome/components/uart/* @esphome/core
esphome/components/uart/button/* @ssieb
esphome/components/uart/event/* @eoasmxd
esphome/components/uart/packet_transport/* @clydebarrow
esphome/components/uart_mux/* @kbx81
esphome/components/uart_tcp/* @Bascht74
esphome/components/udp/* @clydebarrow
esphome/components/ufire_ec/* @pvizeli
esphome/components/ufire_ise/* @pvizeli
esphome/components/ufm01/* @ljungqvist
esphome/components/ultrasonic/* @ssieb @swoboda1337
esphome/components/unicode/* @esphome/core
esphome/components/update/* @jesserockz
esphome/components/uponor_smatrix/* @kroimon
esphome/components/usb_cdc_acm/* @kbx81
@@ -676,7 +616,7 @@ esphome/components/web_server_idf/* @dentra
esphome/components/weikai/* @DrCoolZic
esphome/components/weikai_i2c/* @DrCoolZic
esphome/components/weikai_spi/* @DrCoolZic
esphome/components/whirlpool/* @arpena @glmnet
esphome/components/whirlpool/* @glmnet
esphome/components/whynter/* @aeonsablaze
esphome/components/wiegand/* @ssieb
esphome/components/wireguard/* @droscy @lhoracek @thomas0bernard
@@ -693,11 +633,9 @@ esphome/components/wts01/* @alepee
esphome/components/x9c/* @EtienneMD
esphome/components/xdb401/* @RT530
esphome/components/xgzp68xx/* @gcormier
esphome/components/xiaomi_body_scale/* @dckiller51
esphome/components/xiaomi_hhccjcy10/* @fariouche
esphome/components/xiaomi_lywsd02mmc/* @juanluss31
esphome/components/xiaomi_lywsd03mmc/* @ahpohl
esphome/components/xiaomi_mccgq02hl/* @ahpohl @morph027
esphome/components/xiaomi_mhoc303/* @drug123
esphome/components/xiaomi_mhoc401/* @vevsvevs
esphome/components/xiaomi_rtcgq02lm/* @jesserockz
+1 -1
View File
@@ -48,7 +48,7 @@ PROJECT_NAME = ESPHome
# could be handy for archiving the generated documentation or if some version
# control system is used.
PROJECT_NUMBER = 2026.11.0-dev
PROJECT_NUMBER = 2026.10.0-dev
# Using the PROJECT_BRIEF tag one can provide an optional one line description
# for a project that appears at the top of each page and should give viewer a
+4 -1
View File
@@ -22,13 +22,16 @@ RUN \
-r /requirements.txt
# Install the ESPHome Device Builder dashboard.
RUN uv pip install --no-cache-dir esphome-device-builder==1.23.1
RUN uv pip install --no-cache-dir esphome-device-builder==1.14.7
RUN \
platformio settings set enable_telemetry No \
&& platformio settings set check_platformio_interval 1000000 \
&& mkdir -p /piolibs
COPY script/platformio_install_deps.py platformio.ini /
RUN /platformio_install_deps.py /platformio.ini --libraries
ARG BUILD_VERSION
LABEL \
+2 -6
View File
@@ -21,14 +21,10 @@ export PLATFORMIO_PLATFORMS_DIR="${pio_cache_base}/platforms"
export PLATFORMIO_PACKAGES_DIR="${pio_cache_base}/packages"
export PLATFORMIO_CACHE_DIR="${pio_cache_base}/cache"
# Keep the native toolchain installs and compiler caches on the persistent
# cache root, not the container's user cache dir: it is lost on every
# restart, and not writable when the container runs as a non-root user.
# Keep the native toolchain installs on the persistent cache root, not the
# container's ephemeral user cache dir (re-downloaded on every restart).
export ESPHOME_ESP_IDF_PREFIX="$(dirname "${pio_cache_base}")/idf"
export ESPHOME_SDK_NRF_PREFIX="$(dirname "${pio_cache_base}")/sdk-nrf"
export ESPHOME_ARDUINO8266_PREFIX="$(dirname "${pio_cache_base}")/arduino8266"
export ESPHOME_HOST_PREFIX="$(dirname "${pio_cache_base}")/host"
export ESPHOME_PLATFORMIO_CCACHE_DIR="$(dirname "${pio_cache_base}")/platformio-ccache"
# If /build is mounted, use that as the build path
# otherwise use path in /config (so that builds aren't lost on container restart)
@@ -4,6 +4,7 @@
# Home Assistant Add-on: ESPHome
# Sends discovery information to Home Assistant.
# ==============================================================================
declare config
declare port
# We only disable it when disabled explicitly
@@ -18,17 +19,14 @@ port=$(bashio::addon.ingress_port)
# Wait for the ESPHome Device Builder to become available
bashio::net.wait_for "${port}" "127.0.0.1" 300
# Send one discovery message; the config is a JSON string built with bashio::var.json.
send_discovery() {
local service=$1
local config=$2
if bashio::discovery "${service}" "${config}" > /dev/null; then
bashio::log.info "Successfully send ${service} discovery information to Home Assistant."
else
bashio::log.error "${service} discovery message to Home Assistant failed!"
fi
}
config=$(\
bashio::var.json \
host "127.0.0.1" \
port "^${port}" \
)
send_discovery "esphome" "$(bashio::var.json host "127.0.0.1" port "^${port}")"
# The Device Builder MCP server, consumed by Home Assistant's mcp integration.
send_discovery "mcp" "$(bashio::var.json url "http://127.0.0.1:${port}/api/mcp")"
if bashio::discovery "esphome" "${config}" > /dev/null; then
bashio::log.info "Successfully send discovery information to Home Assistant."
else
bashio::log.error "Discovery message to Home Assistant failed!"
fi
@@ -15,14 +15,10 @@ export PLATFORMIO_PLATFORMS_DIR="${pio_cache_base}/platforms"
export PLATFORMIO_PACKAGES_DIR="${pio_cache_base}/packages"
export PLATFORMIO_CACHE_DIR="${pio_cache_base}/cache"
# Keep the native toolchain installs and compiler caches on the persistent
# /data volume, not the container's ephemeral user cache dir (wiped on every
# add-on update/restart).
# Keep the native toolchain installs on the persistent /data volume, not the
# container's ephemeral user cache dir (wiped on every add-on update/restart).
export ESPHOME_ESP_IDF_PREFIX=/data/cache/idf
export ESPHOME_SDK_NRF_PREFIX=/data/cache/sdk-nrf
export ESPHOME_ARDUINO8266_PREFIX=/data/cache/arduino8266
export ESPHOME_HOST_PREFIX=/data/cache/host
export ESPHOME_PLATFORMIO_CCACHE_DIR=/data/cache/platformio-ccache
if bashio::config.true 'leave_front_door_open'; then
export DISABLE_HA_AUTHENTICATION=true
-12
View File
@@ -5,15 +5,3 @@ bk72xx:
board: generic-bk7231n-qfn32-tuya
logger:
wifi:
ssid: MySSID
password: password1
ap:
# mqtt and captive_portal together pull in AsyncTCP and ESPAsyncWebServer;
# a stray ESP32 AsyncTCP copy on the library search path breaks this build
captive_portal:
mqtt:
broker: 192.168.178.84
@@ -1,8 +0,0 @@
esphome:
name: docker-test-esp8266-native
esp8266:
board: d1_mini
toolchain: arduino
logger:
-2
View File
@@ -3,7 +3,5 @@ esphome:
esp8266:
board: d1_mini
# The PlatformIO path stays covered whatever the default is
toolchain: platformio
logger:
+220 -331
View File
@@ -7,7 +7,6 @@ import importlib
import logging
import os
from pathlib import Path
import platform
import re
import sys
import time
@@ -17,8 +16,8 @@ from typing import TYPE_CHECKING, Protocol
# cause them to be loaded before external components are processed, resulting
# in the built-in version being used instead of the external component one.
from esphome import const, platform_hooks
from esphome.build_helpers.native import analysis_backend, native_backend
from esphome.const import (
ALLOWED_NAME_CHARS,
ARGUMENT_HELP_DEVICE,
BUNDLE_EXTENSION,
CONF_API,
@@ -36,6 +35,7 @@ from esphome.const import (
CONF_LOGGER,
CONF_MDNS,
CONF_MQTT,
CONF_NAME,
CONF_NAME_ADD_MAC_SUFFIX,
CONF_OTA,
CONF_PASSWORD,
@@ -48,24 +48,19 @@ from esphome.const import (
CONF_WEB_SERVER,
CONF_WIFI,
ENV_NOGITIGNORE,
KEY_CORE,
KEY_ESP32,
KEY_TARGET_PLATFORM,
KEY_VARIANT,
PLATFORM_HOST,
SECRETS_FILES,
ErrorFormat,
Toolchain,
)
from esphome.core import CORE, EsphomeError, coroutine
from esphome.enum import StrEnum
from esphome.helpers import IS_MACOS, get_bool_env, indent, is_ip_address
from esphome.helpers import get_bool_env, indent, is_ip_address
from esphome.log import AnsiFore, color, setup_log
from esphome.stacktrace import LogLineProcessor
from esphome.types import ConfigType
from esphome.upload_targets import PortType, get_port_type
from esphome.util import (
ESPHOME_COMMAND,
PICOTOOL_PACKAGE,
FlashImage,
detect_rp2040_bootsel,
@@ -73,7 +68,6 @@ from esphome.util import (
get_serial_ports,
is_picotool_usb_permission_error,
list_yaml_files,
read_secret_line,
run_external_command,
run_external_process,
safe_print,
@@ -90,6 +84,7 @@ if TYPE_CHECKING:
_LOGGER = logging.getLogger(__name__)
ESPHOME_COMMAND = [sys.executable, "-m", "esphome"]
# Maximum buffer size for serial log reading to prevent unbounded memory growth
SERIAL_BUFFER_MAX_SIZE = 65536
@@ -154,7 +149,6 @@ class ArgsProtocol(Protocol):
file: str | None
no_logs: bool
only_generate: bool
skip_bootloader: bool
show_secrets: bool
dashboard: bool
configuration: str
@@ -823,9 +817,7 @@ def write_cpp_file() -> int:
from esphome.build_gen import espidf
espidf.write_project()
elif not CORE.using_native_toolchain:
# Other native builds generate their project at compile time;
# never write a platformio.ini for them
else:
from esphome.build_gen import platformio
platformio.write_project()
@@ -834,14 +826,6 @@ def write_cpp_file() -> int:
def compile_program(args: ArgsProtocol, config: ConfigType) -> int:
if CORE.skip_bootloader and not (CORE.is_esp32 and CORE.using_toolchain_esp_idf):
# Debug only: an orchestrator cannot see YAML toolchain overrides,
# so this is its expected no-op, and a full build is safe.
_LOGGER.debug(
"--skip-bootloader ignored: only supported on ESP32 with the "
"esp-idf toolchain"
)
CORE.skip_bootloader = False
# Keep this gate here, NOT in config validation: device-builder needs
# `esphome config` to keep succeeding with placeholders so onboarding can run.
if CONF_WIFI in config:
@@ -851,7 +835,7 @@ def compile_program(args: ArgsProtocol, config: ConfigType) -> int:
# Keep this here, NOT in codegen: config-hash and --only-generate must keep
# working on machines that cannot run the toolchain.
if CORE.is_esp8266 and CORE.using_toolchain_platformio:
if CORE.is_esp8266:
from esphome.components.esp8266 import check_rosetta
check_rosetta()
@@ -872,20 +856,23 @@ def compile_program(args: ArgsProtocol, config: ConfigType) -> int:
return rc
# Create factory.bin, ota.bin, and firmware.elf copy
if not toolchain.create_factory_bin():
# A build whose factory image could not be produced must not
# exit 0; downloads would serve an image from an older build.
return 1
toolchain.create_factory_bin()
toolchain.create_ota_bin()
toolchain.create_elf_copy()
from esphome.build_helpers.idedata import warn_if_idedata_missing
from esphome.build_helpers.idedata import IDEDATA_BEST_EFFORT_ERRORS
warn_if_idedata_missing(toolchain.get_idedata)
elif CORE.using_native_toolchain:
raise EsphomeError(
f"Toolchain '{CORE.toolchain.value}' resolved but no platform "
"backend claimed the build"
)
try:
if toolchain.get_idedata() is None:
_LOGGER.warning("No idedata was generated for this build")
except IDEDATA_BEST_EFFORT_ERRORS as err:
# The firmware already built; an idedata failure must not fail
# a successful build.
_LOGGER.warning(
"Could not generate idedata: %s (IDE, clang-tidy, and "
"memory-analysis data will be unavailable for this build)",
err,
)
_LOGGER.debug("Idedata failure detail", exc_info=True)
else:
from esphome.platformio import toolchain
@@ -988,16 +975,12 @@ def upload_using_esptool(
if file is not None:
flash_images = [FlashImage(path=file, offset="0x0")]
elif (native := native_backend()) is not None:
# Every native backend supplies its own 0x0 flash image (bootloader
# and partitions included where the target needs them)
image = native.get_factory_firmware_path()
if not image.is_file():
hint = getattr(native, "missing_image_hint", lambda: None)()
raise EsphomeError(
hint or f"{image} does not exist; compile the configuration first"
)
flash_images = [FlashImage(path=image, offset="0x0")]
elif CORE.using_toolchain_esp_idf:
from esphome.espidf import toolchain
flash_images = [
FlashImage(path=toolchain.get_factory_firmware_path(), offset="0x0")
]
else:
from esphome.platformio import toolchain
@@ -1236,28 +1219,14 @@ def upload_program(
config: ConfigType, args: ArgsProtocol, devices: list[str]
) -> tuple[int, str | None]:
host = devices[0]
port_type = get_port_type(host)
ota_key = getattr(args, "ota_key", None)
if ota_key and port_type in (PortType.SERIAL, PortType.BOOTSEL):
_LOGGER.warning(
"The presented OTA key only applies to network uploads; ignored for %s",
host,
)
platform_upload = platform_hooks.get_platform_hook(
CORE.target_platform, "upload_program"
)
if platform_upload is not None and platform_upload(config, args, host):
if ota_key and port_type not in (PortType.SERIAL, PortType.BOOTSEL):
# A platform's own network path (nrf52 mcumgr) has no key handshake
_LOGGER.warning(
"The presented OTA key only applies to the %s OTA platform; "
"ignored for %s",
CONF_ESPHOME,
host,
)
return 0, host
port_type = get_port_type(host)
# MQTT and MQTTIP are also OTA paths; MQTTIP gets resolved to a real IP later by
# _resolve_network_devices(). Only SERIAL and BOOTSEL are non-OTA upload paths.
is_partition_table = getattr(args, "partition_table", False)
@@ -1300,8 +1269,6 @@ def upload_program(
network_devices = _resolve_network_devices(devices, config, args)
if chosen_platform == CONF_WEB_SERVER:
if ota_key:
raise EsphomeError(OTA_KEY_PLATFORM_ERROR)
if is_partition_table or is_bootloader:
raise EsphomeError(
f"{option_string} is only supported with the esphome OTA platform; "
@@ -1315,17 +1282,6 @@ def upload_program(
return _upload_via_native_api(config, network_devices, args)
def _ota_upload_platforms(config: ConfigType) -> dict[str, None]:
"""The network OTA platforms the config provides, one entry each, in
config order; the web_server final-validate hook merges duplicates."""
return {
ota_platform: None
for ota_item in config.get(CONF_OTA, [])
if (ota_platform := ota_item.get(CONF_PLATFORM))
in (CONF_ESPHOME, CONF_WEB_SERVER)
}
def _choose_ota_platform(config: ConfigType, requested: str | None) -> str:
"""Pick the OTA platform to use, optionally honoring ``--ota-platform``.
@@ -1333,12 +1289,22 @@ def _choose_ota_platform(config: ConfigType, requested: str | None) -> str:
The native API uses challenge-response auth with MD5/SHA256 hashing of a
server-issued nonce, so the password is never sent over the wire; the
``web_server`` path uses HTTP Basic auth which transmits credentials in
cleartext over the LAN. (The native path also compresses the upload:
gzip on ESP8266 and RP2040, which inflate it at reboot, and a deflate
stream on ESP32/LibreTiny, which inflate it as it arrives.) Falls back to
cleartext over the LAN. (The native path also supports gzip compression
on ESP8266, where flash space is tight; on ESP32/RP2040/LibreTiny the
backend reports ``supports_compression() == false`` and the firmware is
sent uncompressed regardless of which platform is used.) Falls back to
``web_server`` only when that is the only available platform.
"""
available = _ota_upload_platforms(config)
# Use a dict (insertion-ordered) instead of a list so error messages and
# membership checks see one entry per platform even if the user has
# multiple ``ota:`` items of the same platform; the web_server OTA
# platform's final-validate hook merges duplicates anyway.
available: dict[str, None] = {}
for ota_item in config.get(CONF_OTA, []):
platform = ota_item.get(CONF_PLATFORM)
if platform in (CONF_ESPHOME, CONF_WEB_SERVER):
available[platform] = None
if not available:
raise EsphomeError(
f"Cannot upload Over the Air as the {CONF_OTA} configuration is not "
@@ -1376,19 +1342,9 @@ def _upload_via_native_api(
# Fail closed: an encryption block whose key did not resolve must never
# fall back to a plaintext upload
noise_psk = None
old_noise_psk = None
plaintext_fallback = False
allow_plaintext_upload = False
if ota_key := getattr(args, "ota_key", None):
# The user presents a key the device still runs instead of the
# configured one (a key change in progress); an explicit key never
# downgrades to plaintext
noise_psk = ota_key
elif (encryption_conf := ota_conf.get(CONF_ENCRYPTION)) is not None:
if (encryption_conf := ota_conf.get(CONF_ENCRYPTION)) is not None:
noise_psk = encryption_conf.get(CONF_KEY)
allow_plaintext_upload = bool(
encryption_conf.get(espota2.CONF_ALLOW_PLAINTEXT_UPLOAD)
)
if not noise_psk:
raise EsphomeError(
"OTA encryption is configured but no key was resolved; "
@@ -1397,8 +1353,6 @@ def _upload_via_native_api(
# Ensure the key is a string, as required by the underlying OTA implementation.
# It arrives here as a SensitiveStr which aioesphomeapi rejects.
noise_psk = str(noise_psk)
if old_key := encryption_conf.get(espota2.CONF_OLD_KEY):
old_noise_psk = str(old_key)
elif api_key := static_encryption_key(config.get(CONF_API) or {}):
# Remove before 2027.3.0: the api key is tried, falling back to plaintext
noise_psk = str(api_key)
@@ -1423,12 +1377,6 @@ def _upload_via_native_api(
ota_type = espota2.OTA_TYPE_UPDATE_PARTITION_TABLE
elif getattr(args, "bootloader", False):
check_partition_access("--bootloader")
if (
getattr(args, "file", None) is None
and (native := native_backend())
and (hint := getattr(native, "missing_image_hint", lambda: None)())
):
raise EsphomeError(hint)
binary = CORE.bootloader_bin
ota_type = espota2.OTA_TYPE_UPDATE_BOOTLOADER
if getattr(args, "file", None) is not None:
@@ -1447,23 +1395,9 @@ def _upload_via_native_api(
ota_type,
noise_psk,
plaintext_fallback=plaintext_fallback,
allow_plaintext_upload=allow_plaintext_upload,
old_noise_psk=old_noise_psk,
)
def _old_key_notice(config: ConfigType) -> None:
"""After `run`, whose image was just built from this config; a plain
`upload` may send a stale image that still carries the previous key."""
from esphome import espota2
for item in config.get(CONF_OTA, []):
if item.get(CONF_PLATFORM) == CONF_ESPHOME and (
item.get(CONF_ENCRYPTION) or {}
).get(espota2.CONF_OLD_KEY):
_LOGGER.warning(espota2.OLD_KEY_REMOVE_NOTICE)
def _upload_via_web_server(
config: ConfigType, network_devices: list[str], binary: Path
) -> tuple[int, str | None]:
@@ -1775,66 +1709,27 @@ def command_compile(args: ArgsProtocol, config: ConfigType) -> int | None:
if exit_code != 0:
return exit_code
if CORE.is_host:
_LOGGER.info("Successfully compiled program to path '%s'", CORE.firmware_bin)
_LOGGER.info(
"Successfully compiled program to path '%s'", _host_program_path(config)
)
else:
_LOGGER.info("Successfully compiled program.")
return 0
ENV_OTA_KEY = "ESPHOME_OTA_KEY"
OTA_KEY_PLATFORM_ERROR = (
f"--prompt-ota-key and {ENV_OTA_KEY} only apply to the {CONF_ESPHOME} OTA "
"platform; to flash by serial instead, name the port with --device"
)
def _host_program_path(config: ConfigType) -> str:
"""Return the compiled host ELF path."""
if CORE.using_toolchain_esp_idf:
from esphome.espidf import toolchain
return str(toolchain.get_elf_path())
from esphome.platformio.toolchain import get_idedata
def _read_ota_key(args: ArgsProtocol, config: ConfigType) -> None:
"""Take the key to present to the device for this upload, from the
prompt or from ESPHOME_OTA_KEY, before any compile so the prompt is
not buried in build output."""
prompt = getattr(args, "prompt_ota_key", False)
env_key = getattr(args, "env_ota_key", None)
if not prompt and env_key is None:
return
if CORE.data.get(KEY_CORE, {}).get(KEY_TARGET_PLATFORM) == PLATFORM_HOST:
_LOGGER.warning("A host program runs locally; the presented OTA key is ignored")
return
# The HTTP path has no key handshake; decide before prompting or
# compiling. A serial target is a serial flash: the key is not even
# asked for, and --ota-platform means nothing to it. Every entry must
# be serial: a missing port is skipped and the next entry may be a host
devices = getattr(args, "device", None) or []
serial_target = bool(devices) and all(
device == "SERIAL"
or get_port_type(device) in (PortType.SERIAL, PortType.BOOTSEL)
for device in devices
)
chosen = (
_choose_ota_platform(config, getattr(args, "ota_platform", None))
if not serial_target and _ota_upload_platforms(config)
else None
)
if chosen == CONF_WEB_SERVER:
raise EsphomeError(OTA_KEY_PLATFORM_ERROR)
from esphome.components.noise import validate_encryption_key
from esphome.config_validation import Invalid
if serial_target:
_LOGGER.warning(
"The presented OTA key only applies to network uploads; ignored for %s",
", ".join(devices),
)
return
key = read_secret_line("OTA encryption key: ") if prompt else env_key
try:
validate_encryption_key(key)
except Invalid as err:
raise EsphomeError(f"Invalid OTA encryption key: {err}") from err
args.ota_key = key
# Memoized by compile_program's own call; this is a dict lookup
return str(get_idedata(config).firmware_elf_path)
def command_upload(args: ArgsProtocol, config: ConfigType) -> int | None:
_read_ota_key(args, config)
# Get devices, resolving special identifiers like OTA
devices = choose_upload_log_host(
default=args.device,
@@ -1870,19 +1765,6 @@ def command_logs(args: ArgsProtocol, config: ConfigType) -> int | None:
def command_run(args: ArgsProtocol, config: ConfigType) -> int | None:
if (
CORE.skip_bootloader
and CORE.is_esp32
and CORE.using_toolchain_esp_idf
and any(
get_port_type(device) == PortType.SERIAL for device in (args.device or [])
)
):
# Fail before the compile: the result could never flash over serial.
# Elsewhere the flag is ignored, so serial stays fine there.
_LOGGER.error("--skip-bootloader builds cannot be flashed over serial")
return 1
_read_ota_key(args, config)
exit_code = write_cpp(config)
if exit_code != 0:
return exit_code
@@ -1891,7 +1773,7 @@ def command_run(args: ArgsProtocol, config: ConfigType) -> int | None:
return exit_code
_LOGGER.info("Successfully compiled program.")
if CORE.is_host:
program_path = str(CORE.firmware_bin)
program_path = _host_program_path(config)
_LOGGER.info("Running program from path '%s'", program_path)
return run_external_process(program_path)
@@ -1908,7 +1790,6 @@ def command_run(args: ArgsProtocol, config: ConfigType) -> int | None:
exit_code, successful_device = upload_program(config, args, devices)
if exit_code == 0:
_LOGGER.info("Successfully uploaded program.")
_old_key_notice(config)
else:
_LOGGER.warning("Failed to upload to %s", devices)
return exit_code
@@ -2045,10 +1926,7 @@ def run_multiple_configs(
safe_print()
cmd = command_builder(f)
# The parent already logged the Intel macOS warning; children skip it.
rc = run_external_process(
*cmd, env={**os.environ, _INTEL_MACOS_WARNED_ENV: "1"}
)
rc = run_external_process(*cmd)
if rc == 0:
print_bar(f"[{color(AnsiFore.BOLD_GREEN, 'SUCCESS')}] {str(f)}")
@@ -2087,12 +1965,12 @@ def command_update_all(args: ArgsProtocol) -> int | None:
def command_idedata(args: ArgsProtocol, config: ConfigType) -> int:
import json
native_toolchain = native_backend()
if CORE.using_toolchain_esp_idf:
# Native ESP-IDF derives idedata from the build's compile_commands.json,
# so the configuration must already be compiled.
from esphome.espidf import toolchain as espidf_toolchain
if native_toolchain is not None:
# Native toolchains derive idedata from the build's
# compile_commands.json, so the configuration must already be compiled.
idedata = native_toolchain.get_idedata()
idedata = espidf_toolchain.get_idedata()
if idedata is None:
_LOGGER.error(
"No idedata available; compile the configuration first",
@@ -2131,22 +2009,6 @@ def command_analyze_memory(args: ArgsProtocol, config: ConfigType) -> int:
from esphome.analyze_memory.cli import MemoryAnalyzerCLI
from esphome.analyze_memory.ram_strings import RamStringsAnalyzer
# Refuse an unsupported toolchain before paying for a full compile
analysis_toolchain = analysis_backend()
if analysis_toolchain is None and not CORE.using_toolchain_platformio:
_LOGGER.error(
"analyze-memory is not supported with the '%s' toolchain on %s; "
"re-run with --toolchain platformio",
CORE.toolchain.value if CORE.toolchain else "unresolved",
CORE.target_platform,
)
return 1
if (
check_supported := getattr(analysis_toolchain, "check_analysis_supported", None)
) is not None:
# Raises with the reason; before the compile, not after it
check_supported()
# Always compile to ensure fresh data (fast if no changes - just relinks)
exit_code = write_cpp(config)
if exit_code != 0:
@@ -2158,30 +2020,13 @@ def command_analyze_memory(args: ArgsProtocol, config: ConfigType) -> int:
# Get idedata for analysis
idedata = None
if analysis_toolchain is not None:
objdump = analysis_toolchain.get_objdump_path()
readelf = analysis_toolchain.get_readelf_path()
for tool in (objdump, readelf):
if not tool.is_file():
# The analyzer would silently fall back to host
# binutils, which cannot read the target ELF
_LOGGER.error(
"%s is missing; the toolchain install may be incomplete "
"(recompile, or run 'esphome clean-all' if it persists)",
tool,
)
return 1
objdump_path = str(objdump)
readelf_path = str(readelf)
if CORE.using_toolchain_esp_idf:
from esphome.espidf import toolchain
firmware_elf = analysis_toolchain.get_elf_path()
if not firmware_elf.is_file():
# The analyzer swallows tool failures, so a missing ELF would
# produce an exit-0 zeroed report
_LOGGER.error(
"%s is missing; compile the configuration first", firmware_elf
)
return 1
objdump_path = str(toolchain.get_objdump_path())
readelf_path = str(toolchain.get_readelf_path())
firmware_elf = toolchain.get_elf_path()
else:
from esphome.platformio import toolchain
@@ -2235,9 +2080,155 @@ def command_analyze_memory(args: ArgsProtocol, config: ConfigType) -> int:
def command_rename(args: ArgsProtocol, config: ConfigType) -> int | None:
from esphome.cli.rename import command_rename as run
from esphome import yaml_util
return run(args, config)
new_name = args.name
for c in new_name:
if c not in ALLOWED_NAME_CHARS:
safe_print(
color(
AnsiFore.BOLD_RED,
f"'{c}' is an invalid character for names. Valid characters are: "
f"{ALLOWED_NAME_CHARS} (lowercase, no spaces)",
)
)
return 1
# Load existing yaml file
raw_contents = CORE.config_path.read_text(encoding="utf-8")
yaml = yaml_util.load_yaml(CORE.config_path)
if CONF_ESPHOME not in yaml or CONF_NAME not in yaml[CONF_ESPHOME]:
safe_print(
color(
AnsiFore.BOLD_RED, "Complex YAML files cannot be automatically renamed."
)
)
return 1
old_name = yaml[CONF_ESPHOME][CONF_NAME]
match = re.match(r"^\$\{?([a-zA-Z0-9_]+)\}?$", old_name)
if match is None:
# Only swap the ``name:`` line that sits directly under the
# top-level ``esphome:`` block. A naked ``re.sub`` would
# also clobber any other ``name:`` line whose value happens
# to match (e.g. a sensor / output / wifi entry sharing the
# device's hostname), silently rewriting unrelated user
# configuration. The pattern anchors:
# - at the start of the line so ``friendly_name:``,
# ``device_name:`` etc. don't match the trailing ``name:``
# substring; and
# - at the end of the value (lookahead for whitespace +
# comment + EOL) so ``old_name`` doesn't match as a
# prefix of a longer value (``kitchen`` vs ``kitchen2``).
name_pattern = re.compile(
rf"^(\s*)name:\s+[\"']?{re.escape(old_name)}[\"']?(?=\s*(?:#|$))"
)
out_lines: list[str] = []
in_esphome_block = False
for line in raw_contents.splitlines(keepends=True):
if line and not line[0].isspace() and line.strip():
in_esphome_block = line.lstrip().startswith("esphome:")
out_lines.append(line)
continue
if in_esphome_block:
line = name_pattern.sub(rf'\1name: "{new_name}"', line, count=1)
out_lines.append(line)
new_raw = "".join(out_lines)
else:
old_name = yaml[CONF_SUBSTITUTIONS][match.group(1)]
if (
len(
re.findall(
rf"^\s+{match.group(1)}:\s+[\"']?{old_name}[\"']?",
raw_contents,
flags=re.MULTILINE,
)
)
> 1
):
safe_print(
color(AnsiFore.BOLD_RED, "Too many matches in YAML to safely rename")
)
return 1
new_raw = re.sub(
rf"^(\s+{match.group(1)}):\s+[\"']?{old_name}[\"']?",
f'\\1: "{new_name}"',
raw_contents,
flags=re.MULTILINE,
)
# ``new_name == old_name`` (after substitution resolution) is
# a no-op rewrite that would still queue a pointless re-flash.
# Catch it before the path-equality check below — covers the
# case where the config filename doesn't match the device name
# (e.g. ``weird-file.yaml`` whose ``esphome.name`` is
# ``kitchen``; running ``esphome rename weird-file.yaml kitchen``
# would otherwise just re-flash the same hostname).
if new_name == old_name:
safe_print(
color(
AnsiFore.BOLD_RED,
f"'{new_name}' is already the device's name.",
)
)
return 1
new_path: Path = CORE.config_dir / (new_name + ".yaml")
if new_path.resolve() == CORE.config_path.resolve():
safe_print(
color(
AnsiFore.BOLD_RED,
f"'{new_name}' is already the device's name.",
)
)
return 1
if new_path.exists():
safe_print(
color(
AnsiFore.BOLD_RED,
f"Cannot rename: {new_path} already exists. "
"Refusing to overwrite an existing configuration.",
)
)
return 1
safe_print(
f"Updating {color(AnsiFore.CYAN, str(CORE.config_path))} to {color(AnsiFore.CYAN, str(new_path))}"
)
print()
new_path.write_text(new_raw, encoding="utf-8")
rc = run_external_process(*ESPHOME_COMMAND, "config", str(new_path))
if rc != 0:
safe_print(color(AnsiFore.BOLD_RED, "Rename failed. Reverting changes."))
new_path.unlink()
return 1
cli_args = [
"run",
str(new_path),
"--no-logs",
"--device",
CORE.address,
]
if args.dashboard:
cli_args.insert(0, "--dashboard")
try:
rc = run_external_process(*ESPHOME_COMMAND, *cli_args)
except KeyboardInterrupt:
rc = 1
if rc != 0:
new_path.unlink()
return 1
if CORE.config_path != new_path:
CORE.config_path.unlink()
safe_print(color(AnsiFore.BOLD_GREEN, "SUCCESS"))
print()
return 0
PRE_CONFIG_ACTIONS = {
@@ -2272,15 +2263,6 @@ SIMPLE_CONFIG_ACTIONS = [
]
def _add_skip_bootloader_arg(parser: argparse.ArgumentParser) -> None:
parser.add_argument(
"--skip-bootloader",
help="Do not build the bootloader or the factory image; "
"the result can only be flashed over OTA.",
action="store_true",
)
def _add_states_args(parser: argparse.ArgumentParser) -> None:
"""Add mutually exclusive ``--states``/``--no-states`` flags to a parser.
@@ -2304,25 +2286,6 @@ def _add_states_args(parser: argparse.ArgumentParser) -> None:
)
def _error_format(value: str) -> ErrorFormat:
try:
return ErrorFormat(value.lower())
except ValueError:
choices = ", ".join(ErrorFormat)
raise argparse.ArgumentTypeError(
f"invalid choice: '{value}' (choose from {choices})"
) from None
ARGUMENT_HELP_PROMPT_OTA_KEY = (
"Ask for the OTA encryption key to present to the device instead of using "
"the one in the configuration, for a device that still runs a previous key. "
"The key is read without echo, or as one line from stdin when that is not "
"a terminal; it is never taken from the command line. A calling program "
f"can hand the key over in the {ENV_OTA_KEY} environment variable instead."
)
def parse_args(argv):
options_parser = argparse.ArgumentParser(add_help=False)
options_parser.add_argument(
@@ -2380,19 +2343,7 @@ def parse_args(argv):
metavar="{" + ",".join(t.value for t in Toolchain) + "}",
help=(
"Select toolchain for compiling. Overrides '<platform>.toolchain' in YAML. "
"Default: the platform's native toolchain where it has one, else "
f"{Toolchain.PLATFORMIO.value}."
),
)
options_parser.add_argument(
"--error-format",
type=_error_format,
metavar="{" + ",".join(f.value for f in ErrorFormat) + "}",
help=(
"How configuration errors are shown. 'yaml' shows them inside the "
"failing YAML; 'line' prints one 'file:line:column: error: message' "
"line per error. Default: $ESPHOME_ERROR_FORMAT, else 'yaml'."
f"Default: {Toolchain.PLATFORMIO.value}."
),
)
@@ -2452,7 +2403,6 @@ def parse_args(argv):
help="Only generate source code, do not compile.",
action="store_true",
)
_add_skip_bootloader_arg(parser_compile)
parser_upload = subparsers.add_parser(
"upload",
@@ -2491,11 +2441,6 @@ def parse_args(argv):
help="Upload as partition table (OTA).",
action="store_true",
)
parser_upload.add_argument(
"--prompt-ota-key",
action="store_true",
help=ARGUMENT_HELP_PROMPT_OTA_KEY,
)
parser_upload.add_argument(
"--bootloader",
help="Upload as bootloader (OTA).",
@@ -2521,7 +2466,7 @@ def parse_args(argv):
"-r",
action="store_true",
help="Reset the device before starting serial logs.",
default=get_bool_env("ESPHOME_SERIAL_LOGGING_RESET"),
default=os.getenv("ESPHOME_SERIAL_LOGGING_RESET"),
)
_add_states_args(parser_logs)
@@ -2554,7 +2499,6 @@ def parse_args(argv):
parser_run.add_argument(
"--no-logs", help="Disable starting logs.", action="store_true"
)
_add_skip_bootloader_arg(parser_run)
_add_states_args(parser_run)
@@ -2563,7 +2507,7 @@ def parse_args(argv):
"-r",
action="store_true",
help="Reset the device before starting serial logs.",
default=get_bool_env("ESPHOME_SERIAL_LOGGING_RESET"),
default=os.getenv("ESPHOME_SERIAL_LOGGING_RESET"),
)
parser_run.add_argument(
"--ota-platform",
@@ -2576,11 +2520,6 @@ def parse_args(argv):
"(HTTP Basic auth) when that is the only configured platform."
),
)
parser_run.add_argument(
"--prompt-ota-key",
action="store_true",
help=ARGUMENT_HELP_PROMPT_OTA_KEY,
)
parser_clean = subparsers.add_parser(
"clean-mqtt",
@@ -2709,18 +2648,9 @@ def parse_args(argv):
args, unknown_args = parser.parse_known_args(arguments)
if unknown_args:
_LOGGER.warning("Ignored unrecognized arguments: %s", unknown_args)
else:
args = parser.parse_args(arguments)
return args
# Read here rather than as the option default, so a bad value is reported
# against the environment variable instead of --error-format
if args.error_format is None:
env_value = os.getenv("ESPHOME_ERROR_FORMAT", ErrorFormat.YAML.value)
try:
args.error_format = _error_format(env_value)
except argparse.ArgumentTypeError as err:
parser.error(f"ESPHOME_ERROR_FORMAT: {err}")
return args
return parser.parse_args(arguments)
def _warn_if_source_tree_mismatch() -> None:
@@ -2766,34 +2696,12 @@ def _warn_if_source_tree_mismatch() -> None:
)
_INTEL_MACOS_REMOVAL = "2027.6.0"
_INTEL_MACOS_WARNED_ENV = "ESPHOME_INTEL_MACOS_WARNED"
def _warn_if_intel_macos() -> None:
"""Warn that Intel (x86_64) Python on macOS loses support by _INTEL_MACOS_REMOVAL."""
if (
not IS_MACOS
or platform.machine() != "x86_64"
or _INTEL_MACOS_WARNED_ENV in os.environ
):
return
_LOGGER.warning(
"Support for Intel Macs will end in ESPHome %s or earlier. The Python "
"packages ESPHome depends on have stopped publishing Intel macOS builds, "
"so future releases will not install on this machine. On an Apple "
"Silicon Mac, switch to a native arm64 Python.",
_INTEL_MACOS_REMOVAL,
)
def run_esphome(argv):
from esphome.address_cache import AddressCache
args = parse_args(argv)
CORE.dashboard = args.dashboard
CORE.testing_mode = args.testing_mode
CORE.error_format = args.error_format
# Create address cache from command-line arguments
CORE.address_cache = AddressCache.from_cli_args(
@@ -2806,25 +2714,7 @@ def run_esphome(argv):
args.log_level = "CRITICAL"
setup_log(log_level=args.log_level)
# Taken out of the environment before any command runs, so no build
# tool inherits a key a calling program handed over
args.env_ota_key = os.environ.pop(ENV_OTA_KEY, None)
if args.command in ("upload", "run"):
# One key is one device's; several configs run in child processes
# that would never see it
if (args.env_ota_key is not None or args.prompt_ota_key) and len(
args.configuration
) > 1:
raise EsphomeError(
f"--prompt-ota-key and {ENV_OTA_KEY} take one configuration; "
f"{len(args.configuration)} were given"
)
elif args.env_ota_key is not None:
_LOGGER.warning(
"%s is set but the %s command does not use it", ENV_OTA_KEY, args.command
)
_warn_if_source_tree_mismatch()
_warn_if_intel_macos()
if args.command in PRE_CONFIG_ACTIONS:
try:
@@ -2868,7 +2758,6 @@ def run_esphome(argv):
CORE.config_path = conf_path
CORE.dashboard = args.dashboard
CORE.skip_bootloader = getattr(args, "skip_bootloader", False)
if args.toolchain is not None:
# CLI toolchain wins over esp32.toolchain in YAML.
CORE.toolchain = args.toolchain
+2 -9
View File
@@ -148,13 +148,6 @@ class AddressCache:
continue
hostname, ips = arg.split("=", 1)
# Normalize hostname for consistent lookups
normalized = normalize_hostname(hostname.strip())
addresses = [ip for value in ips.split(",") if (ip := value.strip())]
if not normalized or not addresses:
_LOGGER.warning(
"Invalid cache entry: %s (hostname and at least one address are required)",
arg,
)
continue
cache[normalized] = addresses
normalized = normalize_hostname(hostname)
cache[normalized] = [ip.strip() for ip in ips.split(",")]
return cache
+1 -5
View File
@@ -37,7 +37,7 @@ def find_elf_path(build_path: Path) -> Path | None:
"""
name = build_path.name
for candidate in (
# Native ESP-IDF: the build writes build/<name>.elf, which ESPHome copies
# Native ESP-IDF: idf.py writes build/<name>.elf, which ESPHome copies
# to build/firmware.elf (see espidf.toolchain.create_elf_copy)
build_path / "build" / "firmware.elf",
# PlatformIO
@@ -68,16 +68,12 @@ def idedata_candidates(build_path: Path) -> list[Path]:
The candidate idedata JSON paths, most specific first
"""
name = build_path.name
data_dir = build_path.parent.parent / "idedata"
# Native backends suffix the cache by toolchain (<name>.arduino.json)
suffixed = sorted(data_dir.glob(f"{name}.*.json")) if data_dir.is_dir() else []
return [
# In .pioenvs for test builds
build_path / ".pioenvs" / name / "idedata.json",
# Both toolchains cache it in the data dir, which holds this build dir:
# <data_dir>/idedata/<name>.json next to <data_dir>/build/<name>
build_path.parent.parent / "idedata" / f"{name}.json",
*suffixed,
# Regular builds, invoked from the config dir or from anywhere
Path.cwd() / ".esphome" / "idedata" / f"{name}.json",
Path.home() / ".esphome" / "idedata" / f"{name}.json",
+15 -33
View File
@@ -2,9 +2,7 @@
Bundled names build straight from the framework tree; everything else goes
through ``esphome.platformio.library``. Mirrors ``lib_ldf_mode=off``: each
library builds its own archive; all include dirs join one global path. The
host build reuses it without a framework tree: nothing is bundled there and
every name resolves from the registry.
library builds its own archive; all include dirs join one global path.
Deviations from PlatformIO: flat-layout libraries get the recursive default
source filter; ``dot_a_linkage`` is honored; bundled libraries never run a
@@ -344,24 +342,13 @@ def _check_unfulfilled_provides(
def resolve_libraries(
framework_path: Path | None,
*,
pio_platform: str,
board_mcu: str,
cache_key: str,
framework: str | None = "arduino",
manifest_optional: bool = False,
framework_path: Path, *, pio_platform: str, board_mcu: str, cache_key: str
) -> list[ArduinoLibrary]:
"""Resolve every ``cg.add_library()`` entry into an :class:`ArduinoLibrary`.
``pio_platform``/``board_mcu`` filter manifests the way PlatformIO would
for that core (e.g. ``espressif8266``/``esp8266``); ``cache_key`` keys the
shared converter's download cache. ``framework`` is the manifest
framework token the compatibility check warns about; None skips it.
A None ``framework_path`` means no core-bundled libraries exist (the
host build): every name resolves from the registry.
``manifest_optional`` accepts libraries without a manifest, built with
PlatformIO's default layout.
shared converter's download cache.
The returned list is not topologically sorted, so the caller must link
the archives inside one ``--start-group``/``--end-group`` pair (the
@@ -372,22 +359,18 @@ def resolve_libraries(
# PlatformIO's lib_ignore covers framework-bundled libraries too; the
# shared converter only filters the registry/git ones.
lib_ignore = lib_ignore_set()
bundled_dir_names: frozenset[str] = frozenset()
if framework_path is not None:
# Exact directory names keep membership case-sensitive everywhere
# (an is_dir() probe would match "wire" on macOS/Windows and build
# the bundled Wire twice)
libraries_dir = framework_path / "libraries"
if not libraries_dir.is_dir():
# A registry fallback would fail later with a misleading
# package-not-found error per bundled name
raise EsphomeError(
f"{libraries_dir} is missing; the framework install may be "
"incomplete (run 'esphome clean-all')"
)
bundled_dir_names = frozenset(
p.name for p in libraries_dir.iterdir() if p.is_dir()
# Exact directory names keep membership case-sensitive everywhere
# (an is_dir() probe would match "wire" on macOS/Windows and build
# the bundled Wire twice)
libraries_dir = framework_path / "libraries"
if not libraries_dir.is_dir():
# A registry fallback would fail later with a misleading
# package-not-found error per bundled name
raise EsphomeError(
f"{libraries_dir} is missing; the framework install may be "
"incomplete (run 'esphome clean-all')"
)
bundled_dir_names = frozenset(p.name for p in libraries_dir.iterdir() if p.is_dir())
def _provided(name: object) -> bool:
return _is_safe_library_name(name) and name in bundled_dir_names
@@ -514,13 +497,12 @@ def resolve_libraries(
backend = LibraryBackend(
platform=pio_platform,
framework=framework,
framework="arduino",
emit=_emit,
cache_key=cache_key,
# The walk must not resolve bundled names from the registry;
# _add_bundled_dependencies adds them after emit
provides=_provided,
manifest_optional=manifest_optional,
)
if external:
convert_libraries(external, backend)
+59 -107
View File
@@ -3,12 +3,12 @@
Artifacts land in a machine-global cache (shared across projects, like the
ESP-IDF install in ``esphome.espidf.framework``):
<cache>/arduino8266/frameworks/<tag>/ the Arduino core
<cache>/arduino8266/toolchains/<tag>/ xtensa-lx106-elf gcc 10.3
<cache>/arduino8266/frameworks/<version>/ framework-arduinoespressif8266
<cache>/arduino8266/toolchains/<version>/ toolchain-xtensa (gcc 10.3)
Both come from esphome-libs releases pinned below;
``ESPHOME_ARDUINO8266_*_MIRRORS`` overrides the URLs, with ``{VERSION}``
standing for the release tag. ninja comes from PATH or the ninja PyPI wheel.
Packages come from the PlatformIO registry (identical bits to the PlatformIO
backend); ``ESPHOME_ARDUINO8266_*_MIRRORS`` overrides the URLs. ninja comes
from PATH or the ninja PyPI wheel.
"""
from __future__ import annotations
@@ -17,74 +17,18 @@ import os
from pathlib import Path
from typing import NamedTuple
from esphome.build_helpers.ccache import ccache_env
from esphome.build_helpers.ccache import ccache_defaults_env
from esphome.build_helpers.ninja import find_ninja
from esphome.build_helpers.tools_cache import ARDUINO8266_TOOLS_CACHE, tools_cache_path
from esphome.core import EsphomeError, Version
from esphome.framework_helpers import str_to_lst_of_str
from esphome.platformio.registry import (
Download,
PackageSpec,
Resolver,
get_systype,
install_packages,
prefetch_packages,
)
from esphome.platformio.registry import install_package, prefetch_packages
FRAMEWORK_PACKAGE = "arduino-esp8266"
_FRAMEWORK_RELEASES = "https://github.com/esphome-libs/arduino-esp8266/releases/"
class FrameworkRelease(NamedTuple):
tag: str
sha256: str
size: int
def download(self) -> Download:
archive = f"{FRAMEWORK_PACKAGE}-{self.tag}.tar.gz"
url = f"{_FRAMEWORK_RELEASES}download/{self.tag}/{archive}"
return Download(url, self.sha256, self.size)
# Arduino core version -> its build in esphome-libs/arduino-esp8266
FRAMEWORK_RELEASES: dict[Version, FrameworkRelease] = {
Version(3, 1, 2): FrameworkRelease(
"3.1.2-esphome.3",
"3af533a85163804e164500a5b254dc23b922080e25ecb7dc37b3dd8ac84651db",
37185650,
),
}
TOOLCHAIN_PACKAGE = "toolchain-xtensa-lx106-elf"
FRAMEWORK_PACKAGE = "framework-arduinoespressif8266"
TOOLCHAIN_PACKAGE = "toolchain-xtensa"
# gcc 10.3, the toolchain Arduino core 3.x builds with; the build
# generator's compile flags are tuned to it.
TOOLCHAIN_VERSION = "10.3.0-esphome.3"
_TOOLCHAIN_RELEASES = (
"https://github.com/esphome-libs/xtensa-lx106-elf-toolchain/releases/"
)
# Registry system tag -> (sha256, size) of that host's archive
TOOLCHAIN_BUILDS: dict[str, tuple[str, int]] = {
"darwin_arm64": (
"ad0ea929238d9b527c43e0d20257acc2a5680bdb0d2b1cf8f2c99169786217e9",
60747260,
),
"darwin_x86_64": (
"997090cac80c8eb59604ec52b73fb3568e5f2d9b2971f83c3f2a5c545c9c949c",
64065752,
),
"linux_aarch64": (
"403f14d0d5755682fcd6eaea1e4c457a1d8cad30690e52fb0c804d8db266b908",
67487067,
),
"linux_x86_64": (
"64f21fa3ba736b363dc8c234dbbb39e005e574e33de08a78e9a00c55a64ae706",
68343668,
),
"windows_amd64": (
"1e62abf8df041ce5158f5770dffef955f2f981ce7f7adee88fcd5a8197705aa6",
67536405,
),
}
TOOLCHAIN_VERSION = "2.100300.220621"
ESPHOME_ARDUINO8266_FRAMEWORK_MIRRORS = str_to_lst_of_str(
os.environ.get("ESPHOME_ARDUINO8266_FRAMEWORK_MIRRORS", "")
@@ -100,40 +44,36 @@ def get_arduino8266_tools_path() -> Path:
return tools_cache_path(*ARDUINO8266_TOOLS_CACHE)
def framework_release(version: Version) -> FrameworkRelease:
if (release := FRAMEWORK_RELEASES.get(version)) is None:
# 3.1.1 rather than 3.1.0: the registry has no packages for 3.0.0, 3.0.1 or 3.1.0
MIN_FRAMEWORK_VERSION = Version(3, 1, 1)
def framework_package_version(ver: Version) -> str:
"""Map an Arduino core version to its registry package version (3.1.2 ->
3.30102.0; the leading 3 is the package major).
Exact registry names for 3.x cores; callers floor at MIN_FRAMEWORK_VERSION.
"""
if ver.major > 3:
raise EsphomeError(
f"'toolchain: arduino' has no build of Arduino core {version}; "
f"available: {', '.join(str(v) for v in FRAMEWORK_RELEASES)}. "
"Use one of those or 'toolchain: platformio'"
f"Arduino core {ver} is not supported yet; "
"the newest known core series is 3.x"
)
return release
if ver.major < 3:
raise EsphomeError(
f"Arduino core {ver} is not supported; ESPHome requires core 3.x"
)
return f"3.{ver.major}{ver.minor:02d}{ver.patch:02d}.0"
def get_framework_path(tag: str) -> Path:
return get_arduino8266_tools_path() / "frameworks" / tag
def get_framework_path(package_version: str) -> Path:
return get_arduino8266_tools_path() / "frameworks" / package_version
def get_toolchain_path() -> Path:
return get_arduino8266_tools_path() / "toolchains" / TOOLCHAIN_VERSION
def toolchain_download() -> Download:
"""The toolchain archive for the current host."""
systype = get_systype()
if (build := TOOLCHAIN_BUILDS.get(systype)) is None:
raise EsphomeError(
f"There is no ESP8266 toolchain for this system ({systype}); "
f"supported systems are {', '.join(sorted(TOOLCHAIN_BUILDS))}. "
"Either set 'toolchain: platformio' under 'esp8266:', or point "
"ESPHOME_ARDUINO8266_TOOLCHAIN_MIRRORS at a toolchain archive"
)
sha256, size = build
archive = f"{TOOLCHAIN_PACKAGE}-{TOOLCHAIN_VERSION}-{systype}.tar.gz"
url = f"{_TOOLCHAIN_RELEASES}download/{TOOLCHAIN_VERSION}/{archive}"
return Download(url, sha256, size)
class InstalledPaths(NamedTuple):
"""Locations of the installed framework, toolchain, and ninja binary."""
@@ -144,22 +84,29 @@ class InstalledPaths(NamedTuple):
def check_and_install(framework_version: Version) -> InstalledPaths:
"""Ensure framework, toolchain, and ninja are installed; return their paths."""
release = framework_release(framework_version)
if framework_version < MIN_FRAMEWORK_VERSION:
# Config validation enforces this too; keep the module honest when
# called directly.
raise EsphomeError(
f"The native toolchain requires the Arduino core "
f">= {MIN_FRAMEWORK_VERSION}, got {framework_version}"
)
# Probe the cheap local dependency before ~110 MB of downloads
ninja_path = find_ninja()
framework_path = get_framework_path(release.tag)
package_version = framework_package_version(framework_version)
framework_path = get_framework_path(package_version)
downloads_dir = get_arduino8266_tools_path() / "downloads"
toolchain_path = get_toolchain_path()
# One spec per package: the prefetch and the installs must agree
specs = (
PackageSpec(
(
FRAMEWORK_PACKAGE,
release.tag,
package_version,
framework_path,
ESPHOME_ARDUINO8266_FRAMEWORK_MIRRORS,
("cores/esp8266", "tools/sdk", "libraries"),
),
PackageSpec(
(
TOOLCHAIN_PACKAGE,
TOOLCHAIN_VERSION,
toolchain_path,
@@ -168,17 +115,10 @@ def check_and_install(framework_version: Version) -> InstalledPaths:
("bin", "xtensa-lx106-elf"),
),
)
# Resolved only when a download is needed, so an installed toolchain
# keeps working on a host without a build; a mirror override wins
resolvers: dict[str, Resolver] = {}
if not ESPHOME_ARDUINO8266_FRAMEWORK_MIRRORS:
resolvers[FRAMEWORK_PACKAGE] = release.download
if not ESPHOME_ARDUINO8266_TOOLCHAIN_MIRRORS:
resolvers[TOOLCHAIN_PACKAGE] = toolchain_download
# Fetch both archives at once; the installs verify and extract them.
# One spec list for both, so the two phases cannot drift.
prefetch_packages(specs, downloads_dir, resolvers)
install_packages(specs, downloads_dir, resolvers)
# Fetch both archives at once; the installs below verify and extract
prefetch_packages([spec[:4] for spec in specs], downloads_dir)
for name, version, dest, mirrors, expect in specs:
install_package(name, version, dest, mirrors, downloads_dir, expect=expect)
return InstalledPaths(
framework=framework_path, toolchain=toolchain_path, ninja=ninja_path
)
@@ -203,5 +143,17 @@ def get_build_env(toolchain_path: Path, ccache: str | None) -> dict[str, str]:
*filter(None, env.get("PATH", "").split(os.pathsep)),
]
env["PATH"] = os.pathsep.join(parts)
env.update(ccache_env(ccache, ARDUINO8266_TOOLS_CACHE))
env.update(ccache_env(ccache))
return env
def ccache_env(ccache: str | None) -> dict[str, str]:
"""Return ccache settings for the build subprocess (not os.environ).
``ccache`` is the pre-resolved binary (resolve_ccache_path), or None
when disabled. Values the user already set in the environment are
respected.
"""
if ccache is None:
return {}
return ccache_defaults_env(get_arduino8266_tools_path() / "ccache")
-212
View File
@@ -1,212 +0,0 @@
"""Native Arduino ESP8266 build driver (the PlatformIO ``run`` equivalent)."""
from __future__ import annotations
import logging
from pathlib import Path
import subprocess
from typing import TYPE_CHECKING
from esphome.build_helpers.ccache import resolve_absolute_ccache_path
from esphome.build_helpers.native import warn_ignored_platformio_options
from esphome.build_helpers.ninja import refresh_compile_commands
from esphome.const import (
CONF_COMPILE_PROCESS_LIMIT,
CONF_ESPHOME,
KEY_CORE,
KEY_FRAMEWORK_VERSION,
)
from esphome.core import CORE
from esphome.types import ConfigType
if TYPE_CHECKING:
from esphome.arduino8266.framework import InstalledPaths
_LOGGER = logging.getLogger(__name__)
# ESP8266 user RAM (matches upload.maximum_ram_size in every board manifest)
_MAX_RAM_SIZE = 81920
_RAM_SECTIONS = (".data", ".rodata", ".bss")
_FLASH_SECTIONS = (".irom0.text", ".text", ".text1", ".data", ".rodata")
def get_build_dir() -> Path:
return CORE.relative_pioenvs_path(CORE.name)
def get_elf_path() -> Path:
return get_build_dir() / "firmware.elf"
def _toolchain_tool(name: str) -> Path:
# Imported here, not at module scope: the serial upload/logs fast path
# resolves this module for its artifact paths alone, and framework
# pulls in the whole package-download stack
from esphome.arduino8266 import framework
return framework.toolchain_tool(framework.get_toolchain_path(), name)
def get_factory_firmware_path() -> Path:
"""The image to serial-flash at 0x0 (same bytes as firmware.bin: the
8266 factory copy exists for artifact-contract parity, not content)."""
return get_build_dir() / "firmware.factory.bin"
def get_addr2line_path() -> Path:
return _toolchain_tool("addr2line")
def get_objdump_path() -> Path:
return _toolchain_tool("objdump")
def get_readelf_path() -> Path:
return _toolchain_tool("readelf")
def run_compile(config: ConfigType, verbose: bool) -> int:
from esphome.arduino8266 import framework
from esphome.build_gen import arduino8266 as build_gen
from esphome.core.config import NATIVE_ARDUINO_CONSUMED_PIO_OPTIONS
warn_ignored_platformio_options(NATIVE_ARDUINO_CONSUMED_PIO_OPTIONS)
paths = framework.check_and_install(CORE.data[KEY_CORE][KEY_FRAMEWORK_VERSION])
# Resolved once: the probe is not free and three consumers need it
ccache = resolve_absolute_ccache_path()
ninja_changed = build_gen.write_project(paths, ccache)
build_dir = get_build_dir()
env = framework.get_build_env(paths.toolchain, ccache)
refresh_compile_commands(paths.ninja, build_dir, env, ninja_changed)
cmd = [str(paths.ninja)]
if verbose:
cmd.append("-v")
if jobs := config[CONF_ESPHOME].get(CONF_COMPILE_PROCESS_LIMIT):
cmd += ["-j", str(jobs)]
# Explicit targets: a generator defect that drops them fails loudly
# instead of a green no-op run leaving stale artifacts in place
targets = ["firmware.factory.bin", "firmware.ota.bin"]
cmd += targets
# cwd, not -C: drops ninja's "Entering directory" banner
_LOGGER.debug("Running: %s", " ".join(cmd))
rc = subprocess.run(
cmd, cwd=build_dir, env=env, check=False, close_fds=False
).returncode
if rc != 0:
return rc
# ninja already refused missing targets; existence covers a rule that
# ran but wrote elsewhere
build_dir_artifacts = (
get_elf_path(),
build_dir / "firmware.bin",
get_factory_firmware_path(),
build_dir / "firmware.ota.bin",
)
for artifact in build_dir_artifacts:
if not artifact.is_file():
_LOGGER.error("Build produced no %s", artifact)
return 1
if not _print_size_summary(build_dir, paths):
# Cause already warned; name the consequence for CI harnesses
_LOGGER.warning("Firmware size summary unavailable for this build")
from esphome.build_helpers.idedata import warn_if_idedata_missing
warn_if_idedata_missing(lambda: get_idedata(ccache))
return 0
def _parse_app_size(build_dir: Path, paths: InstalledPaths) -> int | None:
"""Read the app flash budget (irom0_0_seg length) from the linker script."""
from esphome.build_gen.arduino8266 import get_flash_ld_path
from esphome.components.esp8266.build_surgery import segment_length
# Warnings, not debug: without the app size the Flash summary line is
# dropped and CI's memory-impact extraction loses its flash metric.
ld_path = get_flash_ld_path(build_dir, paths)
try:
ld_text = ld_path.read_text(encoding="utf-8")
except (OSError, UnicodeDecodeError) as err:
# A corrupt script degrades the same way, never aborts the build
_LOGGER.warning("Cannot read linker script for the Flash summary: %s", err)
return None
if not (app_size := segment_length(ld_text, "irom0_0_seg")):
_LOGGER.warning("No usable irom0_0_seg in %s; skipping Flash summary", ld_path)
return None
return app_size
def _print_size_summary(build_dir: Path, paths: InstalledPaths) -> bool:
"""Print the RAM/Flash lines ``ci_memory_impact_extract.py`` parses;
False when skipped."""
from esphome.arduino8266.framework import toolchain_tool
from esphome.build_helpers.size_summary import print_size_line
try:
result = subprocess.run(
[
str(toolchain_tool(paths.toolchain, "size")),
"-A",
"-d",
str(build_dir / "firmware.elf"),
],
capture_output=True,
text=True,
check=True,
close_fds=False,
)
except (OSError, subprocess.CalledProcessError) as err:
# The summary is a bonus artifact like idedata; a truncated
# toolchain extraction must not discard an already-linked build
_LOGGER.warning("Could not summarize firmware size: %s", err)
return False
# -d prints decimal sizes; anything else trips the missing-sections guard
sections = {
parts[0]: int(parts[1])
for line in result.stdout.splitlines()
if (parts := line.split())[:1] and parts[0].startswith(".") and len(parts) >= 2
if parts[1].isdigit()
}
if missing := set(_RAM_SECTIONS + _FLASH_SECTIONS) - set(sections):
# A defaulted 0 would print a confidently wrong total for CI's metric
_LOGGER.warning(
"Size output is missing section(s) %s; skipping the size summary",
", ".join(sorted(missing)),
)
return False
# Resolve the flash budget before printing: a RAM line without its
# Flash line would skew CI's memory-impact extraction
app_size = _parse_app_size(build_dir, paths)
if not app_size:
return False
ram = sum(sections[s] for s in _RAM_SECTIONS)
flash = sum(sections[s] for s in _FLASH_SECTIONS)
print_size_line("RAM", ram, _MAX_RAM_SIZE)
print_size_line("Flash", flash, app_size)
return True
def get_idedata(ccache: str | None = None) -> dict | None:
"""Derive idedata from the build's compile_commands.json (same
contract as ``espidf.toolchain.get_idedata``)."""
from esphome.build_helpers.idedata import load_or_build_idedata
# A disabled ccache resolves to None without spawning anything, so
# re-resolving here costs nothing when the caller has no answer
launcher = ccache or resolve_absolute_ccache_path()
return load_or_build_idedata(
get_build_dir() / "compile_commands.json",
get_elf_path(),
# Suffixed so a platformio->arduino->platformio round trip on one
# config never serves the other toolchain's cache shape
CORE.relative_internal_path("idedata", f"{CORE.name}.arduino.json"),
# The compile DB's commands carry the same ccache prefix the ninja
# rules were generated with
launcher=str(launcher) if launcher else None,
)
+124 -701
View File
@@ -1,8 +1,5 @@
from collections.abc import Awaitable, Callable, Sequence
from dataclasses import dataclass, field
import logging
import string
from typing import Any
import esphome.codegen as cg
import esphome.config_validation as cv
@@ -21,46 +18,18 @@ from esphome.const import (
CONF_TYPE_ID,
CONF_UPDATE_INTERVAL,
)
from esphome.core import CORE, ID, EsphomeError, HexInt, Lambda
from esphome.core import ID, Lambda
from esphome.cpp_generator import (
Expression,
LambdaExpression,
MockObj,
MockObjClass,
TemplateArgsType,
call_lambda,
)
from esphome.schema_extractors import SCHEMA_EXTRACT, schema_extractor
from esphome.types import ConfigType, SafeExpType
from esphome.types import ConfigType
from esphome.util import Registry
def progmem_bytes(name: str, data: bytes | list[int]) -> MockObj:
"""Shared PROGMEM table for constant bytes; equal payloads share one, empty is nullptr."""
if not data:
return cg.nullptr
return cg.shared_progmem_array(
name, cg.uint8, cg.ArrayInitializer(*(HexInt(x) for x in data))
)
async def templatable_bytes(
value: Any,
args: TemplateArgsType,
set_template: MockObj,
set_static: MockObj,
table_name: str,
) -> None:
"""Set a TemplatableBytes: a lambda via set_template, constant bytes via set_static."""
if cg.is_template(value):
fn = await cg.templatable(value, args, cg.std_vector.template(cg.uint8))
cg.add(set_template(fn))
elif len(value) > 0xFFFF:
raise EsphomeError(f"Byte payload is {len(value)} bytes; the maximum is 65535")
else:
cg.add(set_static(progmem_bytes(table_name, value), len(value)))
def maybe_simple_id(*validators):
"""Allow a raw ID to be specified in place of a config block.
If the value that's being validated is a dictionary, it's passed as-is to the specified validators. Otherwise, it's
@@ -88,7 +57,6 @@ def maybe_conf(conf, *validators):
with cv.remove_prepend_path([conf]):
return validator({conf: value})
validate.inner_schema = validator
return validate
@@ -134,101 +102,6 @@ def register_condition(name: str, condition_type: MockObjClass, schema: cv.Schem
return CONDITION_REGISTRY.register(name, condition_type, schema)
async def _build_with_parent(
config: ConfigType,
automation_id: ID,
template_arg: cg.TemplateArguments,
args: TemplateArgsType,
) -> MockObj:
parent = await cg.get_variable(config[CONF_ID])
return cg.new_Pvariable(automation_id, template_arg, parent)
async def _build_without_parent(
config: ConfigType,
automation_id: ID,
template_arg: cg.TemplateArguments,
args: TemplateArgsType,
) -> MockObj:
return cg.new_Pvariable(automation_id, template_arg)
async def _build_parented(
config: ConfigType,
automation_id: ID,
template_arg: cg.TemplateArguments,
args: TemplateArgsType,
) -> MockObj:
var = cg.new_Pvariable(automation_id, template_arg)
await cg.register_parented(var, config[CONF_ID])
return var
def register_simple_action(
name: str,
action_type: MockObjClass,
schema: cv.Schema,
*,
synchronous: bool,
) -> None:
"""Register an action whose constructor takes the object named by ``config[CONF_ID]``.
Use the ``register_action`` decorator instead when the builder must also set fields.
"""
register_action(name, action_type, schema, synchronous=synchronous)(
_build_with_parent
)
def register_simple_condition(
name: str, condition_type: MockObjClass, schema: cv.Schema
) -> None:
"""Condition counterpart of ``register_simple_action``."""
register_condition(name, condition_type, schema)(_build_with_parent)
def register_bare_action(
name: str,
action_type: MockObjClass,
schema: cv.Schema,
*,
synchronous: bool,
) -> None:
"""Register an action whose constructor takes no arguments."""
register_action(name, action_type, schema, synchronous=synchronous)(
_build_without_parent
)
def register_bare_condition(
name: str, condition_type: MockObjClass, schema: cv.Schema
) -> None:
"""Condition counterpart of ``register_bare_action``."""
register_condition(name, condition_type, schema)(_build_without_parent)
def register_parented_action(
name: str,
action_type: MockObjClass,
schema: cv.Schema,
*,
synchronous: bool,
) -> None:
"""Register an action deriving from ``Parented<T>``.
The object is constructed without arguments and ``set_parent()`` receives the object
named by ``config[CONF_ID]``.
"""
register_action(name, action_type, schema, synchronous=synchronous)(_build_parented)
def register_parented_condition(
name: str, condition_type: MockObjClass, schema: cv.Schema
) -> None:
"""Condition counterpart of ``register_parented_action``."""
register_condition(name, condition_type, schema)(_build_parented)
Action = cg.esphome_ns.class_("Action")
Trigger = cg.esphome_ns.class_("Trigger")
ACTION_REGISTRY = Registry()
@@ -239,376 +112,6 @@ validate_action_list = cv.validate_registry("action", ACTION_REGISTRY)
validate_condition = cv.validate_registry_entry("condition", CONDITION_REGISTRY)
validate_condition_list = cv.validate_registry("condition", CONDITION_REGISTRY)
ApplyAction = cg.esphome_ns.class_("ApplyAction", Action)
ApplyCondition = cg.esphome_ns.class_("ApplyCondition", Condition)
def flash_string(config: ConfigType, value: str) -> str:
"""Default renderer for ``std::string`` constants; copies the literal out of flash on ESP8266."""
return str(cg.progmem_string(value))
def flash_compare(config: ConfigType, value: str) -> str:
"""Default renderer for ``std::string`` constants in conditions; compares in flash on ESP8266."""
if CORE.is_esp8266:
return f"::esphome::ProgmemStringRef{{PSTR({cg.safe_exp(value)}), {len(value.encode('utf-8'))}}}"
return str(cg.safe_exp(value))
def _default_std_string(
members: list[tuple[str, Any, Any]], renderer: Callable[[ConfigType, str], str]
) -> list[tuple[str, Any, Any]]:
"""Give ``cg.std_string`` members without a renderer the given default."""
return [
(key, t, fn or (renderer if t is cg.std_string else None))
for key, t, fn in members
]
def literal_with_length(config: ConfigType, value: str) -> str:
"""Renderer for a ``(const char *, size_t)`` target: a plain literal plus its byte length.
The target compares or copies the bytes in place, so it needs the RAM literal rather than
the PROGMEM rendering on ESP8266, and the length saves a strlen.
"""
return f"{cg.safe_exp(value)}, {len(value.encode('utf-8'))}"
def string_ref_literal(config: ConfigType, value: str) -> str:
"""Renderer for a ``StringRef`` comparison: a flash literal on ESP8266, else ``StringRef(literal, length)``."""
if CORE.is_esp8266:
return str(cg.FlashStringLiteral(value))
return f"StringRef({literal_with_length(config, value)})"
def _names_parent(text: str) -> bool:
return any(f == "parent" for _, f, _, _ in string.Formatter().parse(text))
@dataclass(frozen=True)
class ApplyCall:
"""One statement from config keys, e.g. ``"set_range({}, {})"`` with ``((CONF_LOW, cg.float_), ...)``.
Each arg is ``(conf_key, type_)`` or ``(conf_key, type_, const_fn)``. A ``conf_key`` may be a
path into nested sections. ``target`` and a plain ``str`` ``type_`` (raw C++ type text) may
name the parent object as ``{parent}``. ``const_fn(config, value)`` renders a constant's argument text; a lambda or an
id bypasses it. The statement is skipped when none of its keys is set, always emitted when it
has no keys, and a partial set is a config error.
"""
target: str
args: tuple[tuple[Any, ...], ...] = ()
def __post_init__(self) -> None:
fields = [
f for _, f, _, _ in string.Formatter().parse(self.target) if f is not None
]
if any(f not in ("", "parent") for f in fields):
raise ValueError(
f"apply target {self.target!r}: only {{}} and {{parent}} placeholders"
)
if (count := fields.count("")) != len(self.args):
raise ValueError(
f"apply target {self.target!r} has {count} "
f"placeholder(s) for {len(self.args)} config key(s)"
)
if any(len(arg) not in (2, 3) for arg in self.args):
raise ValueError(
f"apply target {self.target!r}: each arg is (conf_key, type_[, const_fn])"
)
@property
def names_parent(self) -> bool:
return _names_parent(self.target) or any(
isinstance(arg[1], str) and _names_parent(arg[1]) for arg in self.args
)
@property
def members(self) -> list[tuple[Any, Any, Any]]:
"""Each arg as ``(conf_key, type_, const_fn or None)``."""
return [
(arg[0], arg[1], arg[2] if len(arg) == 3 else None) for arg in self.args
]
@dataclass(frozen=True)
class ApplyField:
"""One config key forwarded as ``target(value)``, or as statement ``target`` when it has ``{}``.
Double a literal brace in a template. ``conf_key`` may be a path into nested sections.
``type_`` may be a C++ type string using ``{parent}`` when the type is only known per
instance. ``const_fn(config, value)`` renders a constant's argument text when ``cg.safe_exp``
is not the right spelling (unit conversion belongs in the validator); a lambda or an id
bypasses it, so the target must also take a plain ``type_``. An absent key emits nothing.
"""
conf_key: str | tuple[str, ...]
target: str
type_: SafeExpType
const_fn: Callable[[ConfigType, Any], str] | None = None
def call(self) -> ApplyCall:
target = self.target if "{}" in self.target else f"{self.target}({{}})"
return ApplyCall(target, ((self.conf_key, self.type_, self.const_fn),))
def _config_lookup(config: ConfigType, key: str | tuple[str, ...]) -> Any:
if isinstance(key, str):
return config.get(key)
for part in key:
if (config := config.get(part)) is None:
return None
return config
def _dict_schema(schema: Any) -> Any:
"""The dict-backed cv.Schema inside cv.All and maybe_* wrappers, or None; cv.Any is not inspected."""
if isinstance(schema, dict):
return cv.Schema(schema)
if isinstance(getattr(schema, "schema", None), dict):
return schema
if isinstance(schema, cv.All):
inner = schema.validators
else:
inner = (
getattr(schema, "inner_schema", None),
) # maybe_conf / maybe_simple_value
for candidate in inner:
if candidate is not None and (found := _dict_schema(candidate)) is not None:
return found
return None
def _check_key_in_schema(
name: str, schema: Any, conf_key: str | tuple[str, ...]
) -> None:
"""Reject a key path the schema does not have; a typo would otherwise be a silent no-op.
Only dict-backed schemas, also inside cv.All and maybe_* wrappers, can be checked.
"""
for part in (conf_key,) if isinstance(conf_key, str) else conf_key:
if (schema := _dict_schema(schema)) is None:
return
markers = {
getattr(marker, "schema", marker): marker for marker in schema.schema
}
if part not in markers:
raise ValueError(f"{name}: config key {part!r} is not in the schema")
schema = schema.schema[markers[part]]
def parent_ref(var: MockObj) -> MockObj:
"""``var`` named from global scope, so a trigger argument cannot shadow it.
Also how a generated callback names its Automation.
"""
return MockObj(f"::{var}", "->")
async def _apply_parent(config: ConfigType, id_key: str = CONF_ID) -> str:
return str(parent_ref(await cg.get_variable(config[id_key])))
def _apply_lambda_args(args: TemplateArgsType) -> TemplateArgsType:
# The generated function's parameters; a std::string arg is never copied.
return [
(cg.RawExpression(f"const std::remove_cvref_t<{cg.safe_exp(t)}> &"), arg)
for t, arg in args
]
def _apply_function(
id_: ID,
return_type: SafeExpType,
template_arg: cg.TemplateArguments,
lambda_args: TemplateArgsType,
statements: list[str],
) -> MockObj:
"""Emit the generated function and declare ``id_`` as the ``ApplyAction`` or
``ApplyCondition`` templated on it, so ``play()`` calls it directly."""
fn = cg.static_function(
f"esphome__{id_.id}__fn", return_type, lambda_args, statements
)
return cg.new_Pvariable(id_, cg.TemplateArguments(fn, *template_arg))
async def _render_values(
name: str,
target: str,
members: list[tuple[Any, Any, Any]],
values: list[Any],
config: ConfigType,
parent: str | None,
lambda_args: TemplateArgsType,
compare: bool = False,
) -> list[str]:
"""Render the argument text of one statement; every key must be present.
``compare``: an inlined lambda expression is parenthesized so it binds as a whole
beside an operator.
"""
if any(value is None for value in values):
keys = [key for key, _, _ in members]
raise EsphomeError(f"{name}: {target!r} needs all of {keys}")
exprs: list[str] = []
for (_, type_, const_fn), value in zip(members, values, strict=True):
if isinstance(value, Lambda):
if isinstance(type_, str):
type_ = cg.RawExpression(type_.format(parent=parent))
inner = await cg.process_lambda(value, lambda_args, return_type=type_)
expr = call_lambda(inner)
bare = compare and isinstance(expr, cg.RawExpression)
exprs.append(f"({expr})" if bare else str(expr))
elif isinstance(value, ID):
# Qualified like the parent, so a trigger arg named like the id cannot shadow it.
exprs.append(f"::{await cg.get_variable(value)}")
elif const_fn is not None:
exprs.append(const_fn(config, value))
else:
exprs.append(str(cg.safe_exp(value)))
return exprs
def _apply_values(config: ConfigType, members: list[tuple[Any, Any, Any]]) -> list[Any]:
return [_config_lookup(config, key) for key, _, _ in members]
def register_apply_action(
name: str,
schema: cv.Schema,
*fields: ApplyField | ApplyCall,
call: str | None = None,
id_key: str = CONF_ID,
) -> None:
"""Register an action that only forwards config values to its parent, with no C++ class.
See ``apply_action_builder``.
"""
register_action(name, ApplyAction, schema, synchronous=True)(
apply_action_builder(name, schema, *fields, call=call, id_key=id_key)
)
def apply_action_builder(
name: str,
schema: cv.Schema,
*fields: ApplyField | ApplyCall,
call: str | None = None,
id_key: str = CONF_ID,
) -> Callable[..., Awaitable[MockObj]]:
"""The builder ``register_apply_action`` registers, for a caller that wraps it.
Generates one static function with the parent (read from ``id_key``) and constants baked
in, lambdas called inline with the trigger args, and an ``ApplyAction`` templated on it.
A constant that is an id (``cv.use_id`` under ``cv.templatable``) is the object it names.
With ``call`` every statement targets the call object ``auto apply_call = parent->call()``,
and ``apply_call.perform()`` is appended.
"""
# An action stores the value, so a std::string constant stays in flash on ESP8266.
statements_spec = [
(
c.target,
_default_std_string(c.members, flash_string),
)
for c in (f if isinstance(f, ApplyCall) else f.call() for f in fields)
]
_check_key_in_schema(name, schema, id_key)
for _, members in statements_spec:
for conf_key, _, _ in members:
_check_key_in_schema(name, schema, conf_key)
async def builder(
config: ConfigType,
action_id: ID,
template_arg: cg.TemplateArguments,
args: TemplateArgsType,
) -> MockObj:
parent = await _apply_parent(config, id_key)
lambda_args = _apply_lambda_args(args)
receiver = "apply_call." if call else f"{parent}->"
statements: list[str] = []
for target, members in statements_spec:
values = _apply_values(config, members)
if not _apply_call_active(members, values):
continue
exprs = await _render_values(
name, target, members, values, config, parent, lambda_args
)
statements.append(f"{receiver}{target.format(*exprs, parent=parent)};")
if call:
statements = [
f"auto apply_call = {parent}->{call}();",
*statements,
"apply_call.perform();",
]
return _apply_function(
action_id, cg.void, template_arg, lambda_args, statements
)
return builder
def _apply_call_active(members: list[tuple[Any, Any, Any]], values: list[Any]) -> bool:
"""An ``ApplyCall`` is emitted unless it has keys and none of them is set."""
return not members or any(value is not None for value in values)
async def _render_check(
name: str,
target: str,
members: list[tuple[Any, Any, Any]],
values: list[Any],
config: ConfigType,
parent: str | None,
lambda_args: TemplateArgsType,
) -> str:
"""Render a boolean check with ``values`` compared against config."""
exprs = await _render_values(
name, target, members, values, config, parent, lambda_args, compare=True
)
return target.format(*exprs, parent=parent)
def register_apply_condition(
name: str, schema: cv.Schema, check: str | ApplyCall, id_key: str = CONF_ID
) -> None:
"""Register a condition that is one expression on its parent, with no C++ class.
``check`` is applied to the parent: ``"is_playing()"`` becomes ``parent->is_playing()``; an
``ApplyCall`` such as ``ApplyCall("state == {}", ((CONF_STATE, cg.bool_),))`` compares
against config values, all of which must be present. Write ``== false`` to negate.
A ``cg.std_string`` constant stays in flash on ESP8266 and supports only ``==`` or ``!=``
against a ``std::string`` or ``StringRef`` member; other string types are plain literals.
Generates one static predicate and an ``ApplyCondition`` templated on it.
"""
call = check if isinstance(check, ApplyCall) else ApplyCall(check)
members = _default_std_string(call.members, flash_compare)
_check_key_in_schema(name, schema, id_key)
for conf_key, _, _ in members:
_check_key_in_schema(name, schema, conf_key)
async def builder(
config: ConfigType,
condition_id: ID,
template_arg: cg.TemplateArguments,
args: TemplateArgsType,
) -> MockObj:
parent = await _apply_parent(config, id_key)
lambda_args = _apply_lambda_args(args)
values = _apply_values(config, members)
check = await _render_check(
name, call.target, members, values, config, parent, lambda_args
)
return _apply_function(
condition_id,
cg.bool_,
template_arg,
lambda_args,
[f"return {parent}->{check};"],
)
register_condition(name, ApplyCondition, schema)(builder)
def validate_potentially_and_condition(value):
if isinstance(value, list):
@@ -740,22 +243,54 @@ XorCondition = cg.esphome_ns.class_("XorCondition", Condition)
@register_condition("and", AndCondition, validate_condition_list)
@register_condition("all", AndCondition, validate_condition_list)
@register_condition("or", OrCondition, validate_condition_list)
@register_condition("any", OrCondition, validate_condition_list)
@register_condition("xor", XorCondition, validate_condition_list)
async def condition_group_to_code(
async def and_condition_to_code(
config: ConfigType,
condition_id: ID,
template_arg: cg.TemplateArguments,
args: TemplateArgsType,
) -> MockObj:
conditions = await build_condition_list(config, template_arg, args)
# A group of one is that condition for and, or and xor; skip the wrapper.
if len(conditions) == 1:
return conditions[0]
return cg.new_Pvariable(
condition_id, cg.TemplateArguments(len(conditions), *template_arg), *conditions
condition_id, cg.TemplateArguments(len(conditions), *template_arg), conditions
)
@register_condition("or", OrCondition, validate_condition_list)
async def or_condition_to_code(
config: ConfigType,
condition_id: ID,
template_arg: cg.TemplateArguments,
args: TemplateArgsType,
) -> MockObj:
conditions = await build_condition_list(config, template_arg, args)
return cg.new_Pvariable(
condition_id, cg.TemplateArguments(len(conditions), *template_arg), conditions
)
@register_condition("all", AndCondition, validate_condition_list)
async def all_condition_to_code(
config: ConfigType,
condition_id: ID,
template_arg: cg.TemplateArguments,
args: TemplateArgsType,
) -> MockObj:
conditions = await build_condition_list(config, template_arg, args)
return cg.new_Pvariable(
condition_id, cg.TemplateArguments(len(conditions), *template_arg), conditions
)
@register_condition("any", OrCondition, validate_condition_list)
async def any_condition_to_code(
config: ConfigType,
condition_id: ID,
template_arg: cg.TemplateArguments,
args: TemplateArgsType,
) -> MockObj:
conditions = await build_condition_list(config, template_arg, args)
return cg.new_Pvariable(
condition_id, cg.TemplateArguments(len(conditions), *template_arg), conditions
)
@@ -770,6 +305,19 @@ async def not_condition_to_code(
return cg.new_Pvariable(condition_id, template_arg, condition)
@register_condition("xor", XorCondition, validate_condition_list)
async def xor_condition_to_code(
config: ConfigType,
condition_id: ID,
template_arg: cg.TemplateArguments,
args: TemplateArgsType,
) -> MockObj:
conditions = await build_condition_list(config, template_arg, args)
return cg.new_Pvariable(
condition_id, cg.TemplateArguments(len(conditions), *template_arg), conditions
)
@register_condition("lambda", LambdaCondition, cv.returning_lambda)
async def lambda_condition_to_code(
config: ConfigType,
@@ -811,15 +359,28 @@ async def for_condition_to_code(
return var
register_apply_condition(
@register_condition(
"component.is_idle",
LambdaCondition,
maybe_simple_id(
{
cv.Required(CONF_ID): cv.use_id(cg.Component),
}
),
"is_idle()",
)
async def component_is_idle_condition_to_code(
config: ConfigType,
condition_id: ID,
template_arg: cg.TemplateArguments,
args: TemplateArgsType,
) -> MockObj:
comp = await cg.get_variable(config[CONF_ID])
lambda_ = await cg.process_lambda(
Lambda(f"return {comp}->is_idle();"), args, return_type=bool
)
return new_lambda_pvariable(
condition_id, lambda_, StatelessLambdaCondition, template_arg
)
@register_action(
@@ -871,13 +432,11 @@ async def if_action_to_code(
condition = await build_condition(config[cond_conf], template_arg, args)
var = cg.new_Pvariable(action_id, if_template_arg, condition)
if CONF_THEN in config:
for action in await build_action_list(config[CONF_THEN], template_arg, args):
cg.add(var.add_then_action(action))
cg.add(var.finish_then())
actions = await build_action_list(config[CONF_THEN], template_arg, args)
cg.add(var.add_then(actions))
if has_else:
for action in await build_action_list(config[CONF_ELSE], template_arg, args):
cg.add(var.add_else_action(action))
cg.add(var.finish_else())
actions = await build_action_list(config[CONF_ELSE], template_arg, args)
cg.add(var.add_else(actions))
return var
@@ -900,9 +459,8 @@ async def while_action_to_code(
) -> MockObj:
condition = await build_condition(config[CONF_CONDITION], template_arg, args)
var = cg.new_Pvariable(action_id, template_arg, condition)
for action in await build_action_list(config[CONF_THEN], template_arg, args):
cg.add(var.add_then_action(action))
cg.add(var.finish_then())
actions = await build_action_list(config[CONF_THEN], template_arg, args)
cg.add(var.add_then(actions))
return var
@@ -931,9 +489,7 @@ async def repeat_action_to_code(
cg.TemplateArguments(cg.uint32, *template_arg.args),
[(cg.uint32, "iteration"), *args],
)
for action in actions:
cg.add(var.add_then_action(action))
cg.add(var.finish_then())
cg.add(var.add_then(actions))
return var
@@ -978,20 +534,44 @@ async def lambda_action_to_code(
return new_lambda_pvariable(action_id, lambda_, StatelessLambdaAction, template_arg)
register_simple_action(
@register_action(
"component.update",
UpdateComponentAction,
maybe_simple_id({cv.Required(CONF_ID): cv.use_id(cg.PollingComponent)}),
maybe_simple_id(
{
cv.Required(CONF_ID): cv.use_id(cg.PollingComponent),
}
),
synchronous=True,
)
async def component_update_action_to_code(
config: ConfigType,
action_id: ID,
template_arg: cg.TemplateArguments,
args: TemplateArgsType,
) -> MockObj:
comp = await cg.get_variable(config[CONF_ID])
return cg.new_Pvariable(action_id, template_arg, comp)
register_simple_action(
@register_action(
"component.suspend",
SuspendComponentAction,
maybe_simple_id({cv.Required(CONF_ID): cv.use_id(cg.PollingComponent)}),
maybe_simple_id(
{
cv.Required(CONF_ID): cv.use_id(cg.PollingComponent),
}
),
synchronous=True,
)
async def component_suspend_action_to_code(
config: ConfigType,
action_id: ID,
template_arg: cg.TemplateArguments,
args: TemplateArgsType,
) -> MockObj:
comp = await cg.get_variable(config[CONF_ID])
return cg.new_Pvariable(action_id, template_arg, comp)
@register_action(
@@ -1069,141 +649,30 @@ def has_non_synchronous_actions(actions: ConfigType) -> bool:
Non-synchronous actions (delay, wait_until, script.wait, etc.) store
trigger args for later execution, making non-owning types like StringRef
unsafe.
Accepts an action list, a single action, or an automation. Only the keys of
action entries are looked up as action names; the options of an action are
only searched for nested action lists (``then:``, ``else:``, ...), so an
option that shares a name with an action, such as the ``delay:`` option of
``speaker_source.set_playlist_delay``, is not mistaken for that action.
"""
if isinstance(actions, list):
return any(has_non_synchronous_actions(item) for item in actions)
if isinstance(actions, dict):
for key, options in actions.items():
for key in actions:
if key in ACTION_REGISTRY and not ACTION_REGISTRY[key].synchronous:
return True
if _options_have_non_synchronous_actions(options):
return True
return any(
has_non_synchronous_actions(v)
for v in actions.values()
if isinstance(v, (list, dict))
)
return False
def _options_have_non_synchronous_actions(options: Any) -> bool:
"""Search an action's options for nested action lists, without treating option keys as actions."""
if isinstance(options, list):
return has_non_synchronous_actions(options)
if isinstance(options, dict):
return any(_options_have_non_synchronous_actions(v) for v in options.values())
return False
def validate_no_deferring_actions(
context: str, data: str = "the trigger data"
) -> Callable[[ConfigType], ConfigType]:
"""Return a validator that rejects deferring actions in a validated automation.
Use it for triggers whose arguments only live while the callback runs (views into a reused
buffer, a JSON document freed on return, ...): deferring actions keep the arguments for later.
:param context: Where the automation is, for the error message, e.g. ``"on_json and on_data"``.
:param data: What is only valid while the automation runs, for the error message.
"""
def validator(value: ConfigType) -> ConfigType:
if has_non_synchronous_actions(value):
raise cv.Invalid(
f"Deferring actions (delay, wait_until, script.wait, ...) are not allowed in "
f"{context}: {data} is only valid while the automation runs. Copy what you need "
"into globals first, then defer in a separate script or automation."
)
return value
return validator
def validate_synchronous_automation(
context: str,
data: str = "the trigger data",
extra_schema=None,
extra_validators=None,
single: bool = False,
) -> cv.All:
"""``validate_automation()`` that also rejects deferring actions.
See ``validate_no_deferring_actions()`` for ``context`` and ``data``; the other parameters are
passed to ``validate_automation()``.
"""
return cv.All(
validate_automation(
extra_schema=extra_schema,
extra_validators=extra_validators,
single=single,
),
validate_no_deferring_actions(context, data),
)
async def _new_automation(
args: TemplateArgsType, config: ConfigType, *ctor_args: MockObj
) -> MockObj:
"""Create the Automation for ``config`` with its actions."""
templ = cg.TemplateArguments(*(arg[0] for arg in args))
obj = cg.new_Pvariable(config[CONF_AUTOMATION_ID], templ, *ctor_args)
actions = await build_action_list(config[CONF_THEN], templ, args)
for action in actions:
cg.add(obj.add_action(action))
return obj
async def build_automation(
trigger: MockObj, args: TemplateArgsType, config: ConfigType
) -> MockObj:
return await _new_automation(args, config, trigger)
async def build_trigger_callback(
args: TemplateArgsType,
config: ConfigType,
params: TemplateArgsType,
forward: Sequence[str | Expression] | None = None,
when: str | ApplyCall | None = None,
parent: MockObj | None = None,
) -> LambdaExpression:
"""Build the Automation for ``config`` and return a stateless callback that triggers it.
``params`` are the parent callback's parameters, ``forward`` the expressions passed to
``trigger()`` (default: the parameter names; write the parent as ``parent_ref(var)``),
``when`` a filter the callback returns early on, skipped like any ``ApplyCall`` when none
of its keys is set. ``when`` may name ``parent`` as ``{parent}``, e.g.
``"{parent}->is_fully_open()"``.
"""
members: list[tuple[Any, Any, Any]] = []
if when is not None:
call = when if isinstance(when, ApplyCall) else ApplyCall(when)
members = call.members
if parent is None and call.names_parent:
raise ValueError(f"trigger filter {call.target!r} names {{parent}}")
obj = await _new_automation(args, config)
lambda_args = _apply_lambda_args(params)
statements: list[str] = []
if when is not None:
values = _apply_values(config, members)
if _apply_call_active(members, values):
check = await _render_check(
"trigger filter",
call.target,
members,
values,
config,
None if parent is None else str(parent_ref(parent)),
lambda_args,
)
statements.append(f"if (!({check}))\n return;")
if forward is None:
forward = [name for _, name in params]
statements.append(f"{parent_ref(obj)}->trigger({', '.join(map(str, forward))});")
return LambdaExpression(
["\n".join(statements)], lambda_args, capture="", return_type=cg.void
)
arg_types = [arg[0] for arg in args]
templ = cg.TemplateArguments(*arg_types)
obj = cg.new_Pvariable(config[CONF_AUTOMATION_ID], templ, trigger)
actions = await build_action_list(config[CONF_THEN], templ, args)
cg.add(obj.add_actions(actions))
return obj
async def build_callback_automation(
@@ -1212,9 +681,6 @@ async def build_callback_automation(
args: TemplateArgsType,
config: ConfigType,
forwarder: MockObj | MockObjClass | None = None,
params: TemplateArgsType | None = None,
forward: Sequence[str | Expression] | None = None,
when: str | ApplyCall | None = None,
) -> None:
"""Build an Automation and register it as a callback on the parent.
@@ -1226,9 +692,6 @@ async def build_callback_automation(
pointer-sized (single Automation* field) to fit inline in Callback::ctx_
and avoid heap allocation.
With ``params``, ``forward`` or ``when`` the callback is instead the stateless
lambda of ``build_trigger_callback``; ``forwarder`` cannot be combined with them.
:param parent: The component object (e.g., button, sensor).
:param callback_method: Name of the callback method (e.g., "add_on_press_callback").
:param args: Automation template args as list of (type, name) tuples.
@@ -1237,56 +700,22 @@ async def build_callback_automation(
TriggerForwarder<Ts...>. Pass any struct type whose aggregate init takes
a single Automation pointer (e.g., TriggerOnTrueForwarder).
"""
if params is not None or forward is not None or when is not None:
if forwarder is not None:
raise ValueError(
"forwarder cannot be combined with params, forward or when"
)
callback = await build_trigger_callback(
args, config, args if params is None else params, forward, when, parent
)
cg.add(getattr(parent, callback_method)(callback))
return
obj = await _new_automation(args, config)
arg_types = [arg[0] for arg in args]
templ = cg.TemplateArguments(*arg_types)
obj = cg.new_Pvariable(config[CONF_AUTOMATION_ID], templ)
actions = await build_action_list(config[CONF_THEN], templ, args)
cg.add(obj.add_actions(actions))
# Use template forwarder structs for deduplication. The compiler generates
# one operator() per forwarder type; different automation pointers are just
# data in the struct.
if forwarder is None:
forwarder = TriggerForwarder.template(*(arg[0] for arg in args))
forwarder = TriggerForwarder.template(templ)
# RawExpression for aggregate init — both forwarder and obj are codegen
# MockObjs (not user input), and there's no Expression type for positional
# aggregate initialization (StructInitializer uses named fields).
cg.add(getattr(parent, callback_method)(cg.RawExpression(f"{forwarder}{{{obj}}}")))
async def build_parent_callback_automation(
parent: MockObj, callback_method: str, arg: tuple[Any, str], config: ConfigType
) -> None:
"""Register an Automation that receives ``parent`` from a callback that carries nothing.
``arg`` is the automation's ``(type, name)``, e.g. ``(Fan.operator("ptr"), "x")``.
"""
await build_callback_automation(
parent, callback_method, [arg], config, params=[], forward=[parent_ref(parent)]
)
async def build_trigger_automations(
parent: MockObj | None,
config: ConfigType,
entries: tuple[tuple[str, TemplateArgsType], ...],
) -> None:
"""Instantiate each entry's Trigger class, with ``parent`` when given, and build its automations.
``entries`` are ``(conf_key, args)`` pairs; the class comes from the entry's ``CONF_TRIGGER_ID``.
"""
ctor_args = () if parent is None else (parent,)
for conf_key, args in entries:
for conf in config.get(conf_key, []):
trigger = cg.new_Pvariable(conf[CONF_TRIGGER_ID], *ctor_args)
await build_automation(trigger, args, conf)
@dataclass(frozen=True, slots=True)
class CallbackAutomation:
"""A single callback automation entry for build_callback_automations."""
@@ -1295,9 +724,6 @@ class CallbackAutomation:
callback_method: str
args: TemplateArgsType = field(default_factory=list)
forwarder: MockObj | MockObjClass | None = None
params: TemplateArgsType | None = None
forward: Sequence[str | Expression] | None = None
when: str | ApplyCall | None = None
async def build_callback_automations(
@@ -1319,7 +745,4 @@ async def build_callback_automations(
entry.args,
conf,
forwarder=entry.forwarder,
params=entry.params,
forward=entry.forward,
when=entry.when,
)
File diff suppressed because it is too large Load Diff
+17 -305
View File
@@ -3,10 +3,7 @@
import json
import logging
from pathlib import Path
import re
import textwrap
from esphome.build_helpers import pch
from esphome.components.esp32 import (
get_esp32_variant,
get_excluded_builtin_components,
@@ -21,7 +18,7 @@ from esphome.framework_helpers import (
get_project_cxx_compile_flags,
get_project_link_flags,
)
from esphome.helpers import get_bool_env, mkdir_p, write_file_if_changed
from esphome.helpers import mkdir_p, write_file_if_changed
_LOGGER = logging.getLogger(__name__)
@@ -36,83 +33,6 @@ list(FILTER esphome_cxx_compile_options EXCLUDE REGEX "^-std=")
list(APPEND esphome_cxx_compile_options "-std={standard}")
idf_build_set_property(CXX_COMPILE_OPTIONS "${{esphome_cxx_compile_options}}")"""
# Drops the app archive from ldgen's inputs so app-only edits skip the
# sections.ld regeneration. Safe: no mapping fragment references it
# (run_compile re-checks each build). Filters only the top-level call;
# the prior definition stays reachable with an underscore prefix.
_LDGEN_OVERRIDE = """\
if(COMMAND __ldgen_get_lib_deps_of_target)
set_property(GLOBAL PROPERTY ESPHOME_LDGEN_ARMED 1)
function(__ldgen_get_lib_deps_of_target target out_list_var)
if(NOT COMMAND ___ldgen_get_lib_deps_of_target)
message(FATAL_ERROR "ESPHome ldgen override lost the original "
"implementation; set ESPHOME_LDGEN_FULL_DEPS=1 and rebuild.")
endif()
___ldgen_get_lib_deps_of_target(${target} ${out_list_var})
if(out_list_var STREQUAL "ldgen_libraries")
set_property(GLOBAL PROPERTY ESPHOME_LDGEN_FILTERED 1)
list(LENGTH ${out_list_var} esphome_ldgen_before)
list(REMOVE_ITEM ${out_list_var} idf::src __idf_src)
list(LENGTH ${out_list_var} esphome_ldgen_after)
if(esphome_ldgen_before EQUAL esphome_ldgen_after)
message(@SEVERITY@ "ESPHome ldgen app archive exclusion matched "
"nothing; app edits will regenerate sections.ld.")
endif()
endif()
set(${out_list_var} "${${out_list_var}}" PARENT_SCOPE)
endfunction()
else()
message(@MISSING@ "ESPHome ldgen override target not found; "
"app edits will regenerate sections.ld.")
endif()"""
# lwip sources that compile to empty objects with the option off (their own
# #if guard). (option, regex valid for both Python and CMake); a source is
# only dropped when its option is defined and off, so a renamed option
# keeps it.
LWIP_EMPTY_SOURCES: tuple[tuple[str, str], ...] = (
("CONFIG_LWIP_PPP_SUPPORT", "/netif/ppp/"),
("CONFIG_LWIP_IPV6", "/core/ipv6/"),
("CONFIG_LWIP_AUTOIP", "/core/ipv4/autoip[.]c$"),
("CONFIG_LWIP_STATS", "/core/stats[.]c$"),
)
# Drift guard only: keep every lwip source.
LWIP_FULL_SOURCES_ENV = "ESPHOME_LWIP_FULL_SOURCES"
# Drops the empty objects after project(), once the lwip target exists.
_LWIP_EMPTY_SOURCES_FILTER = f"""\
idf_build_get_property(esphome_build_components BUILD_COMPONENTS)
if(lwip IN_LIST esphome_build_components AND NOT DEFINED ENV{{{LWIP_FULL_SOURCES_ENV}}})
idf_component_get_property(esphome_lwip_lib lwip COMPONENT_LIB)
get_target_property(esphome_lwip_srcs ${{esphome_lwip_lib}} SOURCES)
@FILTERS@
set_property(TARGET ${{esphome_lwip_lib}} PROPERTY SOURCES ${{esphome_lwip_srcs}})
endif()"""
def lwip_empty_source_gate(option: str, regex: str) -> str:
return (
f" if(DEFINED {option} AND NOT {option})\n"
f' list(FILTER esphome_lwip_srcs EXCLUDE REGEX "{regex}")\n'
" endif()"
)
def _lwip_empty_sources_filter() -> str:
gates = "\n".join(lwip_empty_source_gate(*entry) for entry in LWIP_EMPTY_SOURCES)
return _LWIP_EMPTY_SOURCES_FILTER.replace("@FILTERS@", gates)
# Runs after project() so the walk has happened; catches the remaining
# silent path where the top-level out-var was renamed.
_LDGEN_OVERRIDE_CHECK = """\
get_property(esphome_ldgen_armed GLOBAL PROPERTY ESPHOME_LDGEN_ARMED)
get_property(esphome_ldgen_filtered GLOBAL PROPERTY ESPHOME_LDGEN_FILTERED)
if(esphome_ldgen_armed AND NOT esphome_ldgen_filtered)
message(@SEVERITY@ "ESPHome ldgen override never filtered the app "
"archive; app edits will regenerate sections.ld.")
endif()"""
def get_available_components() -> list[str] | None:
"""List the built-in ESP-IDF components from ``project_description.json``.
@@ -158,114 +78,6 @@ def _cmake_quote(value: str) -> str:
return f'"{escaped}"'
# CONFIG_APP_BUILD_BOOTLOADER is hidden and force-selected, so it can only be
# cleared at the CMake level (the same state IDF's RAM-app build type uses).
# The macro is IDF's __build_process_project_includes plus a few added lines;
# the flag is ignored and the bootloader builds as usual if IDF changes it.
IDF_BOOTLOADER_OVERRIDE = """\
# ESPHome bootloader skip switch; see esphome/espidf/toolchain.py.
if(ESPHOME_SKIP_BOOTLOADER)
macro(__build_process_project_includes)
idf_build_get_property(sdkconfig_cmake SDKCONFIG_CMAKE)
include(${sdkconfig_cmake})
set(CONFIG_APP_BUILD_BOOTLOADER "")
# bt's CMakeLists reads the lowercase idf_target that the (now
# skipped) bootloader project_include leaks; keep it defined, or
# its empty TARGET_SRC_NAME sends file(GLOB_RECURSE) across /.
idf_build_get_property(idf_target IDF_TARGET)
# partition_table's V1 ECDSA signing reads this key, which the
# skipped bootloader project_include also sets.
get_filename_component(SECURE_BOOT_SIGNING_KEY "${CONFIG_SECURE_BOOT_SIGNING_KEY}" ABSOLUTE BASE_DIR "${project_dir}")
idf_build_get_property(build_properties __BUILD_PROPERTIES)
foreach(build_property ${build_properties})
idf_build_get_property(val ${build_property})
set(${build_property} "${val}")
endforeach()
idf_build_get_property(build_component_targets __BUILD_COMPONENT_TARGETS)
foreach(component_target ${build_component_targets})
__component_get_property(dir ${component_target} COMPONENT_DIR)
__component_get_property(_name ${component_target} COMPONENT_NAME)
set(COMPONENT_NAME ${_name})
set(COMPONENT_DIR ${dir})
set(COMPONENT_PATH ${dir})
if(EXISTS ${COMPONENT_DIR}/project_include.cmake)
include(${COMPONENT_DIR}/project_include.cmake)
endif()
endforeach()
endmacro()
endif()
"""
# The lines the override adds to IDF's macro; idf_macro_matches() below
# strips them before comparing with the live macro.
BOOTLOADER_OVERRIDE_ADDED_LINES = (
'set(CONFIG_APP_BUILD_BOOTLOADER "")',
"idf_build_get_property(idf_target IDF_TARGET)",
(
"get_filename_component(SECURE_BOOT_SIGNING_KEY"
' "${CONFIG_SECURE_BOOT_SIGNING_KEY}" ABSOLUTE BASE_DIR "${project_dir}")'
),
)
_MACRO = re.compile(
r"macro\(__build_process_project_includes\)(.*?)endmacro\(\)", re.DOTALL
)
def _normalized_macro(text: str) -> list[str] | None:
"""The macro body as comment-free, whitespace-collapsed lines."""
if (match := _MACRO.search(text)) is None:
return None
return [
re.sub(r"\s+", " ", line)
for raw in match.group(1).splitlines()
if (line := raw.split("#", 1)[0].strip())
]
_EXPECTED_MACRO = [
line
for line in _normalized_macro(IDF_BOOTLOADER_OVERRIDE)
if line not in BOOTLOADER_OVERRIDE_ADDED_LINES
]
def idf_macro_matches(idf_path: Path) -> bool:
"""Whether IDF's macro still matches the copy the override replays."""
build_cmake = idf_path / "tools" / "cmake" / "build.cmake"
live = _normalized_macro(build_cmake.read_text(encoding="utf-8"))
return live == _EXPECTED_MACRO
def _cmake_args_for_write(minimal: bool) -> dict[str, str]:
"""The cg.add_cmake_arg() values for one project write.
The discovery (minimal) write lets every built-in component register so
the list it finds, cached per target, serves any exclusion set; the full
write drops the excluded names from that list and passes all of them to
IDF. Names outside the framework's components directory (managed
components) stay excluded on the discovery write too, since the list
only holds built-in ones. A checkout supplied through IDF_PATH caches
nothing, so its discovery keeps every exclusion.
"""
from esphome.espidf.toolchain import _esphome_manages_idf, _get_idf_path
args = dict(CORE.cmake_args)
if (
not minimal
or not _esphome_manages_idf()
or not (excluded := args.get("EXCLUDE_COMPONENTS"))
):
return args
root = _get_idf_path() / "components"
outside = [name for name in excluded.split(";") if not (root / name).is_dir()]
if outside:
args["EXCLUDE_COMPONENTS"] = ";".join(outside)
else:
del args["EXCLUDE_COMPONENTS"]
return args
def get_project_cmakelists(
minimal: bool = False, builtin_components: list[str] | None = None
) -> str:
@@ -278,10 +90,9 @@ def get_project_cmakelists(
"""
idf_target = variant_to_idf_target(get_esp32_variant())
# esp_idf_size 2.x (IDF >=6.0) made NG the default and removed --ng;
# 1.x (IDF 5.5) needs --ng for --format=json2. 1.x json2 also lacks
# total_size, hence the ELF fallback in espidf/size_summary.py; both
# go away together when 1.x support is dropped.
# esp_idf_size 2.x (bundled with IDF >=6.0) made NG the default and
# removed the --ng flag; on 1.x (IDF 5.5) --ng is required to get
# --format=raw because the legacy mode doesn't support it.
size_ng_flag = "--ng" if idf_version() < cv.Version(6, 0, 0) else ""
# Project-wide compile options: -D defines and -W warning flags (skip
@@ -311,28 +122,13 @@ def get_project_cmakelists(
else ""
)
# Stops the ~3s sections.ld regeneration on app-only edits; see
# _LDGEN_OVERRIDE. ESPHOME_LDGEN_FULL_DEPS=1 restores stock behavior;
# ESPHOME_LDGEN_STRICT=1 (CI) fails the configure when an IDF bump
# breaks the override instead of degrading to stock deps.
if get_bool_env("ESPHOME_LDGEN_FULL_DEPS"):
ldgen_override = ""
ldgen_override_check = ""
else:
strict = get_bool_env("ESPHOME_LDGEN_STRICT")
severity = "FATAL_ERROR" if strict else "WARNING"
missing = "FATAL_ERROR" if strict else "STATUS"
ldgen_override = _LDGEN_OVERRIDE.replace("@SEVERITY@", severity).replace(
"@MISSING@", missing
)
ldgen_override_check = _LDGEN_OVERRIDE_CHECK.replace("@SEVERITY@", severity)
# CMake variables registered via cg.add_cmake_arg(). Emitted before
# include(project.cmake) so values like EXCLUDE_COMPONENTS are already
# set when project.cmake seeds the component list.
# set when project.cmake seeds the component list, and on minimal
# (discovery) writes too so excluded components never register.
cmake_args = "\n".join(
f"set({name} {_cmake_quote(value)})"
for name, value in sorted(_cmake_args_for_write(minimal).items())
for name, value in sorted(CORE.cmake_args.items())
)
# Per-project list exposed as a CMake variable so converted PIO libs
@@ -355,10 +151,10 @@ def get_project_cmakelists(
# component's REQUIRES including real IDF components). Referenced by
# src/CMakeLists and by each converted PIO lib's CMakeLists. Skipped
# on minimal writes because project_description.json may be stale.
# Excluded components are dropped here as well: the discovery list
# (cached per target) still holds them, and requiring an excluded
# component pulls it back into the build (IDF requirement expansion
# overrides EXCLUDE_COMPONENTS).
# Excluded components are dropped here as well: a stale
# project_description.json from a build without exclusions may still
# list them, and requiring an excluded component pulls it back into
# the build (IDF requirement expansion overrides EXCLUDE_COMPONENTS).
# Derived from the EXCLUDE_COMPONENTS cmake arg emitted above so the
# two can never disagree within one generated file.
builtin_components_property = (
@@ -403,9 +199,6 @@ set(EXTRA_COMPONENT_DIRS ${{CMAKE_SOURCE_DIR}}/src)
include($ENV{{IDF_PATH}}/tools/cmake/project.cmake)
{IDF_BOOTLOADER_OVERRIDE}
{ldgen_override}
{cpp_standard_options}
{cxx_compile_options}
@@ -418,31 +211,12 @@ include($ENV{{IDF_PATH}}/tools/cmake/project.cmake)
project({CORE.name})
{ldgen_override_check}
{_lwip_empty_sources_filter()}
# Emit per-memory-type JSON size data for ESPHome to read post-build.
# json2 stays small; raw dumps every symbol (~2s on a large map) and
# this command runs inside the link edge, blocking everything downstream.
# IDF's size target depends on the map, so ninja has to know the link
# writes it for size to build in the same run as all. IDF declares the
# map itself since espressif/esp-idf#19201 (and in cmakev2), and ninja
# rejects two rules for one output, so declare it here only when IDF has
# not. add_custom_command(OUTPUT) marks its outputs GENERATED.
get_source_file_property(esphome_map_declared
${{CMAKE_BINARY_DIR}}/${{CMAKE_PROJECT_NAME}}.map GENERATED)
if(esphome_map_declared)
set(esphome_map_byproducts)
else()
set(esphome_map_byproducts BYPRODUCTS ${{CMAKE_BINARY_DIR}}/${{CMAKE_PROJECT_NAME}}.map)
endif()
# Emit raw JSON size data for ESPHome to read post-build.
add_custom_command(
TARGET ${{CMAKE_PROJECT_NAME}}.elf POST_BUILD
COMMAND ${{PYTHON}} -m esp_idf_size {size_ng_flag} --format=json2
COMMAND ${{PYTHON}} -m esp_idf_size {size_ng_flag} --format=raw
-o ${{CMAKE_BINARY_DIR}}/esp_idf_size.json
${{CMAKE_PROJECT_NAME}}.map
${{esphome_map_byproducts}}
WORKING_DIRECTORY ${{CMAKE_BINARY_DIR}}
VERBATIM
)
@@ -505,72 +279,9 @@ idf_component_register(
target_link_options(${{COMPONENT_LIB}} PUBLIC
{link_opts_str}
)
{_pch_cmake_block()}"""
# Where CMake puts the .gch of the src component; ccache reads the checksum
# next to it in place of the .gch
_PCH_SUM_PATH = "build/esp-idf/src/CMakeFiles/__idf_src.dir/cmake_pch.hxx.gch.sum"
# Where the Windows gate records its choice
_PCH_CHOICE_VAR = "ESPHOME_PCH"
def _pch_cmake_block() -> str:
"""The CMake block that precompiles the core headers for the C++ sources
of the src component; empty when disabled."""
if not pch.pch_enabled():
return ""
headers = "\n".join(
f' "$<$<COMPILE_LANGUAGE:CXX>:${{CMAKE_CURRENT_SOURCE_DIR}}/{header}>"'
for header in pch.PCH_DEFAULT_HEADERS
)
block = f"""target_precompile_headers(${{COMPONENT_LIB}} PRIVATE
{headers}
)"""
if not pch.pch_needs_gcc_check():
return f"\n# ESPHome precompiled header\n{block}\n"
# Before the first configure only CMake knows the compiler version
return f"""
# ESPHome precompiled header, unless GCC bug 14940 keeps it from loading
if(CMAKE_CXX_COMPILER_ID STREQUAL "GNU" AND ({pch.PCH_WINDOWS_CMAKE_OLD_GCC}))
message(STATUS "ESPHome: GCC ${{CMAKE_CXX_COMPILER_VERSION}} cannot load a precompiled header on Windows; compiling without it")
set({_PCH_CHOICE_VAR} OFF CACHE BOOL "ESPHome precompiled header in use" FORCE)
else()
set({_PCH_CHOICE_VAR} ON CACHE BOOL "ESPHome precompiled header in use" FORCE)
{textwrap.indent(block, " ")}
endif()
"""
def _read_if_exists(path: Path) -> str:
return path.read_text(encoding="utf-8") if path.is_file() else ""
def write_pch_checksum() -> None:
"""Write the checksum ccache uses in place of the .gch: the core headers,
the framework version, the sdkconfig and the managed component versions."""
from esphome.espidf.toolchain import get_cmake_cache_value
if not pch.pch_enabled():
return
# The gate's choice, cached by configure
if pch.pch_needs_gcc_check() and get_cmake_cache_value(_PCH_CHOICE_VAR) != "ON":
return
pch.log_pch_in_use()
checksum = pch.pch_checksum(
CORE.relative_src_path(),
pch.PCH_DEFAULT_HEADERS,
(
str(idf_version()),
_read_if_exists(CORE.relative_build_path(f"sdkconfig.{CORE.name}")),
_read_if_exists(CORE.relative_build_path("dependencies.lock")),
),
)
path = CORE.relative_build_path(_PCH_SUM_PATH)
path.parent.mkdir(parents=True, exist_ok=True)
write_file_if_changed(path, checksum + "\n")
def write_project(
minimal: bool = False, builtin_components: list[str] | None = None
) -> None:
@@ -591,9 +302,10 @@ def write_project(
)
# Snapshot the exclusion set so has_outdated_files() can trigger a
# reconfigure when it changes. The ESPHOME_PROJECT_BUILTIN_COMPONENTS
# property drops excluded names, so re-including one (e.g. a config
# gains mqtt) needs a reconfigure to rebuild it from the discovery list.
# discovery reconfigure when it changes. Excluded components never
# register in project_description.json, so re-including one (e.g. a
# config gains mqtt) requires a fresh discovery pass before the
# ESPHOME_PROJECT_BUILTIN_COMPONENTS property can list it.
write_file_if_changed(
CORE.relative_build_path("exclude_components.esphomeinternal"),
";".join(get_excluded_builtin_components()),
-303
View File
@@ -1,303 +0,0 @@
"""Native ninja build generator for the host platform.
Emits ``build.ninja`` under ``.pioenvs/<name>/``: every source in the
generated ``src/`` tree plus the resolved registry libraries compiles with
the machine's compiler and links into ``program``, the name PlatformIO's
native platform produced. Build flags route the way SCons's ``ParseFlags``
did under PlatformIO: ``-D``/``-I``/``-std=``/``-W`` shapes reach the
compile lines only, ``-l``/``-L``/``-Wl,`` the link line only, everything
else both.
"""
from __future__ import annotations
from collections.abc import Iterable
import logging
import os
from pathlib import Path
import subprocess
import sys
from typing import TYPE_CHECKING
from esphome.build_helpers.ninja import escape as _e, quote_path as _q, shell_token
from esphome.build_helpers.ninja_gen import (
PATH_ARG_FLAGS,
Flag,
anchor_path_flag,
ar_rule_lines,
collect_sources,
compile_edges,
compile_rule_lines,
library_edges,
pch_edges,
pch_rule_lines,
tool_lines,
)
from esphome.build_helpers.pch import PCH_DEFAULT_HEADERS, pch_enabled
from esphome.core import CORE, EsphomeError
from esphome.framework_helpers import get_project_cxx_compile_flags
from esphome.helpers import mkdir_p, write_file_if_changed
from esphome.host.toolchain import PROGRAM_NAME, HostCompilers, find_tool, get_build_dir
from esphome.platformio.library import lex_build_flags
if TYPE_CHECKING:
from esphome.arduino.library import ArduinoLibrary
_LOGGER = logging.getLogger(__name__)
# The PlatformIO platform the host built under; registry manifests declare
# compatibility against it, as lib_compat_mode=strict checked before
PIO_PLATFORM = "native"
# Namespaces the shared library download cache (pio_components/host/)
LIBRARY_CACHE_KEY = "host"
# Flag shapes that only the compiler understands; dropped from the link line
_COMPILE_ONLY_PREFIXES = ("-D", "-U", "-I", "-std=", "-W", *PATH_ARG_FLAGS)
# Flag shapes that only the linker consumes; inert on a -c compile line
_LINK_ONLY_PREFIXES = ("-l", "-L", "-Wl,")
# Link-only flags whose argument is the next token
_LINK_ONLY_ARG_FLAGS = ("-framework", "-Xlinker", "-z")
def parse_flags(entries: Iterable[str], owner: str) -> list[Flag]:
"""Lex build flag entries into flags, each with its argument.
Entries are a set, so their order is not the user's: a flag and the
argument it takes as the next token must share one entry.
"""
flags: list[Flag] = []
for entry in entries:
it = iter(lex_build_flags(entry, owner))
for tok in it:
if tok not in PATH_ARG_FLAGS and tok not in _LINK_ONLY_ARG_FLAGS:
flags.append((tok,))
continue
arg = next(it, None)
# A path never starts with "-"; that is the next flag
if arg is None or (tok in PATH_ARG_FLAGS and arg.startswith("-")):
raise EsphomeError(
f"{owner} build flags have '{tok}' with no argument; write "
f"the flag and its argument as one entry"
)
flags.append((tok, arg))
return flags
def split_flags(flags: list[Flag]) -> tuple[list[Flag], list[Flag]]:
"""Route build flags to the compile and link lines."""
compile_flags: list[Flag] = []
link_flags: list[Flag] = []
for flag in flags:
name = flag[0]
if len(flag) > 1:
(compile_flags if name in PATH_ARG_FLAGS else link_flags).append(flag)
elif name.startswith(_LINK_ONLY_PREFIXES):
# Checked before the compile prefixes: -Wl, would match -W
link_flags.append(flag)
elif name.startswith(_COMPILE_ONLY_PREFIXES):
compile_flags.append(flag)
else:
# -g, -O, -f*, -m*, -pthread, --coverage: both lines, as SCons
compile_flags.append(flag)
link_flags.append(flag)
return compile_flags, link_flags
def _is_std(flag: Flag) -> bool:
return flag[0].startswith("-std=")
def _is_cxx_std(flag: Flag) -> bool:
return _is_std(flag) and "++" in flag[0]
def _anchored_flags(entries: Iterable[str], owner: str) -> list[Flag]:
build_path = Path(CORE.build_path)
return [
anchor_path_flag(flag, build_path)
for flag in parse_flags(sorted(entries), owner)
]
def _flag_lists() -> tuple[list[str], list[str], list[str]]:
"""The C, C++, and link flag lists (raw tokens), build_unflags applied.
``cg.set_cpp_standard`` wins over any ``-std=`` in the build flags for
C++ compiles, as PlatformIO's unflag of every other standard did; C
compiles never see a C++ standard.
"""
# The funnel warns and drops empty glued arguments (-D "") itself
compile_flags, link_flags = split_flags(
_anchored_flags(CORE.build_flags, "esphome")
)
cflags = [f for f in compile_flags if not _is_cxx_std(f)]
cxx_std = CORE.cpp_standard
cxxflags = [f for f in compile_flags if not (cxx_std and _is_std(f))]
if cxx_std:
cxxflags.insert(0, (f"-std={cxx_std}",))
cxxflags += [(tok,) for tok in get_project_cxx_compile_flags()]
# A flag is removed whole, with its argument, as PlatformIO did
unflags = set(_anchored_flags(CORE.build_unflags, "esphome build_unflags"))
# An unflag that hits nothing (a typo, or -DUSE_FOO against
# -DUSE_FOO=1) must be visible, since the user believes the flag is
# gone while it still drives the build
if unmatched := sorted(unflags - set(cflags) - set(cxxflags) - set(link_flags)):
_LOGGER.warning(
"build_unflags entries matched no build flag: %s",
", ".join(" ".join(flag) for flag in unmatched),
)
def keep(flags: list[Flag]) -> list[str]:
return [tok for flag in flags if flag not in unflags for tok in flag]
return keep(cflags), keep(cxxflags), keep(link_flags)
def _resolve_host_libraries() -> list[ArduinoLibrary]:
"""Every ``cg.add_library()`` entry, fetched from the registry.
The host has no framework, so nothing is bundled and no framework
compatibility check applies; the platform check keeps the strict
manifest gate PlatformIO's native platform enforced. Manifest-less
libraries (a bare git checkout) build with PlatformIO's default
layout, as they did under its native platform.
"""
if not CORE.platformio_libraries:
return []
from esphome.arduino.library import resolve_libraries
return resolve_libraries(
None,
pio_platform=PIO_PLATFORM,
board_mcu="host",
cache_key=LIBRARY_CACHE_KEY,
framework=None,
manifest_optional=True,
)
def _file_macro_maps(build_dir: Path) -> list[str]:
"""Flags that keep ``__FILE__`` relative to the build path.
PlatformIO compiled ``src/x.cpp`` from the build path, and tools name
things after that spelling (CodSpeed's benchmark ids). Here a source
reaches the compiler by its absolute path, or relative to the build
directory when ccache rewrites it.
"""
build_path = Path(CORE.build_path)
prefixes = (build_path, Path(os.path.relpath(build_path, build_dir)))
return [
shell_token(f"-fmacro-prefix-map={prefix}{os.sep}=", force=True)
for prefix in prefixes
]
def _compiler_version(cxx: tuple[str, ...]) -> str:
"""What the compiler says it is: its path can stay the same across an
update (the macOS shims in /usr/bin)."""
result = subprocess.run(
[*cxx, "--version"], capture_output=True, text=True, check=False
)
return result.stdout
def write_project(compilers: HostCompilers, ccache: str | None) -> bool:
"""Write the ninja build for the current configuration.
``ccache`` is the caller's already-resolved binary (None when disabled).
Returns True when ``build.ninja`` changed, so the caller can skip work
derived purely from it (the compile database) on unchanged builds.
"""
build_dir = get_build_dir()
mkdir_p(build_dir)
src_dir = CORE.relative_src_path()
if not src_dir.is_dir():
# Generated project state, not install state: clean-all would not help
raise EsphomeError(f"Generated source directory {src_dir} is missing")
cflags, cxxflags, link_flags = _flag_lists()
libraries = _resolve_host_libraries()
include_dirs = [src_dir]
for lib in libraries:
include_dirs += lib.include_dirs
includes = [f"-I{_q(d)}" for d in include_dirs]
includes += _file_macro_maps(build_dir)
# SCons's link line: $LINKFLAGS $SOURCES $_LIBDIRFLAGS $_LIBFLAGS, so
# -L and -l trail the objects while every other link token leads
lib_dirs = [Path(t[2:]) for t in link_flags if t.startswith("-L")]
libs = [t for t in link_flags if t.startswith("-l")]
linkflags = [shell_token(t) for t in link_flags if not t.startswith(("-L", "-l"))]
for lib in libraries:
lib_dirs += lib.link_dirs
libs += [f"-l{name}" for name in lib.link_libs]
linkflags += [shell_token(f) for f in lib.link_flags]
# PlatformIO's ASPPCOM passes only -D/-I user flags to assembly
asflags = [t for t in cflags if t.startswith(("-D", "-I"))]
lines = [
*tool_lines(compilers.cc, compilers.cxx, ccache),
*compile_rule_lines(ccache),
*pch_rule_lines(),
"rule link",
" command = $cxx -o $out $linkflags @$out.rsp $archives $libdirflags $libflags",
" rspfile = $out.rsp",
" rspfile_content = $in_newline",
" description = LINK $out",
]
if any(lib.sources and lib.lib_archive for lib in libraries):
# Resolved only when an archive is built, so a system without
# binutils still links a library-free configuration
lines += ar_rule_lines(find_tool("AR", ("ar",)))
lines += [
"",
f"cflags = {' '.join([*map(shell_token, cflags), *includes])}",
f"cxxflags = {' '.join([*map(shell_token, cxxflags), *includes])}",
f"asflags = {' '.join([*map(shell_token, asflags), *includes])}",
f"linkflags = {' '.join(linkflags)}",
f"libdirflags = {' '.join(f'-L{_q(d)}' for d in lib_dirs)}",
f"libflags = {' '.join(shell_token(lib) for lib in libs)}",
"",
]
archives, direct_objs = library_edges(lines, libraries)
src_cxx_override = pch_edges(
lines,
build_dir,
src_dir,
PCH_DEFAULT_HEADERS,
# The arguments of a CXX override come before the flags
[*compilers.cxx[1:], *cxxflags],
(),
(compilers.cxx[0], _compiler_version(compilers.cxx)) if pch_enabled() else (),
compilers.cxx,
)
src_objs = compile_edges(
lines,
collect_sources(src_dir),
src_dir,
"src",
cxx_override=src_cxx_override,
)
if not src_objs:
raise EsphomeError(f"No source files found under {src_dir}")
# Archives are not topologically sorted; GNU ld needs the group to
# resolve references between them. ld64 loads archives iteratively and
# rejects the option, so macOS lists them bare.
archive_tokens = [shell_token(a) for a in archives]
if archive_tokens and sys.platform != "darwin":
archive_tokens = ["-Wl,--start-group", *archive_tokens, "-Wl,--end-group"]
lines.append(
f"build {PROGRAM_NAME}: link {' '.join(src_objs + direct_objs)} | "
f"{' '.join(_e(a) for a in archives)}"
)
lines.append(f" archives = {' '.join(archive_tokens)}")
lines.append(f"default {PROGRAM_NAME}")
lines.append("")
return write_file_if_changed(build_dir / "build.ninja", "\n".join(lines))
-6
View File
@@ -1,8 +1,6 @@
from esphome.build_helpers.pch import pch_script_enabled
from esphome.const import __version__
from esphome.core import CORE
from esphome.helpers import mkdir_p, read_file, write_file_if_changed
from esphome.platformio.toolchain import copy_pch_script
from esphome.writer import find_begin_end
INI_AUTO_GENERATE_BEGIN = "; ========== AUTO GENERATED CODE BEGIN ==========="
@@ -64,8 +62,6 @@ def get_ini_content():
# Add extra script for C++ flags
CORE.add_platformio_option("extra_scripts", [f"pre:{CXX_FLAGS_FILE_NAME}"])
if pch_script_enabled():
CORE.add_platformio_option("extra_scripts", ["post:pch.py"])
# Add CMake args. A user-supplied value (str or list) is deliberately
# replaced; this option was always overwritten at FINAL priority.
@@ -110,8 +106,6 @@ def write_project():
# Write extra script for C++ specific flags
write_cxx_flags_script()
if pch_script_enabled():
copy_pch_script()
CXX_FLAGS_FILE_NAME = "cxx_flags.py"
+2 -41
View File
@@ -68,15 +68,6 @@ def resolve_ccache_path() -> str | None:
return ccache
def resolve_absolute_ccache_path() -> str | None:
"""``resolve_ccache_path`` for the ninja backends, which run their
commands from the build directory, where a relative path is lost."""
from esphome.build_helpers.ninja import absolute_tool
ccache = resolve_ccache_path()
return absolute_tool(ccache) if ccache else None
def ccache_defaults_env(cache_dir: Path) -> dict[str, str]:
"""Default ``CCACHE_*`` values for a build subprocess (not os.environ).
@@ -93,39 +84,9 @@ def ccache_defaults_env(cache_dir: Path) -> dict[str, str]:
"CORE.build_path must be set before constructing the build environment"
)
defaults = {
# ccache expands $VAR in its settings; $$ is a literal $
"CCACHE_DIR": str(cache_dir).replace("$", "$$"),
"CCACHE_DIR": str(cache_dir),
"CCACHE_NOHASHDIR": "true",
"CCACHE_DEPEND": "1",
"CCACHE_BASEDIR": str(Path(CORE.build_path).resolve()).replace("$", "$$"),
"CCACHE_BASEDIR": str(Path(CORE.build_path).resolve()),
}
return {k: v for k, v in defaults.items() if k not in os.environ}
def effective_ccache_basedir() -> str:
"""The prefix ccache strips from hashed paths: a usable user
CCACHE_BASEDIR, else the resolved build path."""
from esphome.core import CORE
raw = os.environ.get("CCACHE_BASEDIR")
if raw is not None and Path(raw).is_absolute() and len(Path(raw).parts) > 1:
return raw
return str(Path(CORE.build_path).resolve())
def ccache_env(ccache: str | None, tools_cache: tuple[str, str]) -> dict[str, str]:
"""The ccache settings for a build subprocess (not os.environ).
``ccache`` is the pre-resolved binary (resolve_ccache_path), or None when
disabled; ``tools_cache`` is the backend's tools cache spec, which holds
its ccache dir. The pch settings include ``time_macros``: a cached
object can keep an older ``__DATE__`` or ``__TIME__``.
"""
if ccache is None:
return {}
from esphome.build_helpers.pch import ccache_pch_env
from esphome.build_helpers.tools_cache import tools_cache_path
env = ccache_defaults_env(tools_cache_path(*tools_cache) / "ccache")
env.update(ccache_pch_env())
return env
+1 -27
View File
@@ -11,7 +11,6 @@ consumers (IDE integration, clang-tidy) expect:
from __future__ import annotations
from collections.abc import Callable
import json
import logging
import os
@@ -22,8 +21,6 @@ import subprocess
from esphome.core import EsphomeError
from esphome.helpers import write_file
_LOGGER = logging.getLogger(__name__)
# Everything idedata generation may raise after a successful link; idedata
# is a bonus artifact, so consumers warn instead of failing the build
IDEDATA_BEST_EFFORT_ERRORS = (
@@ -34,21 +31,7 @@ IDEDATA_BEST_EFFORT_ERRORS = (
ValueError,
)
def warn_if_idedata_missing(get_idedata: Callable[[], dict | None]) -> None:
"""Run an idedata generator, downgrading any failure to a warning:
the firmware already built."""
try:
if get_idedata() is None:
_LOGGER.warning("No idedata was generated for this build")
except IDEDATA_BEST_EFFORT_ERRORS as err:
_LOGGER.warning(
"Could not generate idedata: %s (IDE, clang-tidy, and "
"memory-analysis data will be unavailable for this build)",
err,
)
_LOGGER.debug("Idedata failure detail", exc_info=True)
_LOGGER = logging.getLogger(__name__)
# C++ translation-unit suffixes used to identify ESPHome source files.
_CXX_SUFFIXES = (".cpp", ".cc")
@@ -152,15 +135,6 @@ def _is_launcher(token: str) -> bool:
return Path(token).stem.lower() in _LAUNCHER_STEMS
def is_joined_include(tok: str) -> bool:
"""The joined ``-includefoo.h`` spelling; excludes clang's -include-pch."""
return (
tok.startswith("-include")
and tok != "-include"
and not tok.startswith("-include-")
)
def parse_entry(
entry: dict, launcher: str | None = None
) -> tuple[str, list[str], list[str], list[str]]:
-74
View File
@@ -1,74 +0,0 @@
"""Resolution of the native (non-PlatformIO) build backend for a config.
Kept deliberately light: the serial upload and logs fast path resolves a
backend for its artifact paths alone, so importing this must not pull in a
platform component package or the backend itself.
"""
from __future__ import annotations
from collections.abc import Collection
import importlib
import logging
from types import ModuleType
from esphome.const import Toolchain
from esphome.core import CORE, EsphomeError
_LOGGER = logging.getLogger(__name__)
# Native build backend per (target platform, toolchain)
NATIVE_TOOLCHAIN_MODULES = {
("esp32", Toolchain.ESP_IDF): "esphome.espidf.toolchain",
("esp8266", Toolchain.ARDUINO): "esphome.arduino8266.toolchain",
("host", Toolchain.HOST): "esphome.host.toolchain",
}
def native_backend() -> ModuleType | None:
"""The native build backend module for the resolved toolchain."""
if not CORE.using_native_toolchain:
return None
key = (CORE.target_platform, CORE.toolchain)
if (module_path := NATIVE_TOOLCHAIN_MODULES.get(key)) is None:
# Degrading to the PlatformIO path would build with the wrong backend
raise EsphomeError(
f"Toolchain '{CORE.toolchain.value}' has no native build backend "
f"module for platform {CORE.target_platform}"
)
return importlib.import_module(module_path)
# Binutils and the linked image for memory analysis, for toolchains that build
# without PlatformIO but have no native build backend (which supplies them)
ANALYSIS_TOOLCHAIN_MODULES = {
("nrf52", Toolchain.SDK_NRF): "esphome.components.nrf52.toolchain",
}
def analysis_backend() -> ModuleType | None:
"""The module giving objdump, readelf and the ELF of a non-PlatformIO build.
None means PlatformIO's idedata supplies them (or nothing can).
"""
if (native := native_backend()) is not None:
return native
module_path = ANALYSIS_TOOLCHAIN_MODULES.get((CORE.target_platform, CORE.toolchain))
return importlib.import_module(module_path) if module_path else None
def warn_ignored_platformio_options(consumed: Collection[str]) -> None:
"""Warn for component-added platformio options a native build drops.
User-supplied keys were already routed or warned about by
``core/config.py``; what survives into ``CORE.platformio_options`` came
from ``cg.add_platformio_option`` calls in components.
"""
for key in sorted(CORE.platformio_options or {}):
if key not in consumed:
_LOGGER.warning(
"platformio_options->%s is ignored when building with the "
"native '%s' toolchain",
key,
CORE.toolchain.value,
)
+3 -93
View File
@@ -2,17 +2,14 @@
from __future__ import annotations
import json
import logging
import os
from pathlib import Path
import re
import shutil
import subprocess
from esphome.core import EsphomeError
from esphome.framework_helpers import strip_win_long_path_prefix, tool_version_runs
from esphome.helpers import write_file_if_changed
_LOGGER = logging.getLogger(__name__)
@@ -26,26 +23,11 @@ def _ninja_runs(binary: str) -> bool:
)
# Compile rule names the generators emit; ninja's compdb tool is asked for
# exactly these, so a renamed rule fails the build instead of stranding idedata
COMPILE_RULES = ("c", "cxx", "aspp", "asm")
def absolute_tool(tool: str | Path) -> str:
"""A tool path that still resolves from the build directory.
``shutil.which`` returns a relative path for a relative PATH entry, and
ninja runs the commands from ``.pioenvs/<name>``. Symlinks are kept:
ccache's compiler links depend on the name they are called by.
"""
return strip_win_long_path_prefix(str(Path(tool).absolute()))
def find_ninja() -> Path:
"""Locate the ninja binary: a runnable PATH hit first, else the ninja
PyPI wheel."""
if binary := shutil.which("ninja"):
binary = absolute_tool(binary)
binary = strip_win_long_path_prefix(binary)
if _ninja_runs(binary):
return Path(binary)
import_error: ImportError | None = None
@@ -67,18 +49,8 @@ def find_ninja() -> Path:
def escape(value: Path | str) -> str:
"""Escape a path or token for a ninja file.
ninja has no escape for ``|`` or a line break in a path, so those fail
here by name instead of producing a build file ninja misreads.
"""
text = str(value)
if bad := next((c for c in "|\r\n" if c in text), None):
raise EsphomeError(
f"Path {text!r} contains {bad!r}, which a ninja build file cannot "
"express; rename the file or directory"
)
return text.replace("$", "$$").replace(":", "$:").replace(" ", "$ ")
"""Escape a path or token for a ninja file."""
return str(value).replace("$", "$$").replace(":", "$:").replace(" ", "$ ")
def quote_arg(tok: str) -> str:
@@ -118,65 +90,3 @@ def shell_token(tok: str, force: bool = False) -> str:
def quote_path(value: Path | str) -> str:
"""Force-quote a path for the ninja command line (shell/CreateProcess)."""
return shell_token(str(value), force=True)
def refresh_compile_commands(
ninja_path: Path, build_dir: Path, env: dict[str, str], ninja_changed: bool
) -> None:
"""Regenerate the compile DB (a pure function of build.ninja) when stale.
Freshness rides a stamp: the DB itself is written through
write_file_if_changed (its mtime feeds the idedata cache), so a
regeneration with identical content would stay "stale" forever. An
interrupted previous run may have rewritten the manifest without
regenerating the DB, hence the mtime comparison.
"""
compdb = build_dir / "compile_commands.json"
compdb_stamp = build_dir / ".compile_commands.stamp"
ninja_file = build_dir / "build.ninja"
if (
ninja_changed
or not compdb.is_file()
or not compdb_stamp.is_file()
or compdb_stamp.stat().st_mtime < ninja_file.stat().st_mtime
):
write_compile_commands(ninja_path, build_dir, env)
compdb_stamp.touch()
def write_compile_commands(
ninja_path: Path, build_dir: Path, env: dict[str, str]
) -> None:
compdb = build_dir / "compile_commands.json"
result = subprocess.run(
[str(ninja_path), "-C", str(build_dir), "-t", "compdb", *COMPILE_RULES],
env=env,
capture_output=True,
text=True,
check=False,
close_fds=False,
)
if result.returncode != 0:
# Drop any stale database so consumers (IDE integration, clang-tidy,
# the memory analyzer) can't silently read outdated data
compdb.unlink(missing_ok=True)
raise EsphomeError(f"Could not generate compile_commands.json: {result.stderr}")
try:
entries = json.loads(result.stdout)
except ValueError as err:
compdb.unlink(missing_ok=True)
raise EsphomeError(
f"ninja produced an unparsable compile database: {err} "
f"(output starts {result.stdout[:120]!r})"
) from err
if not entries:
# compdb exits 0 with [] for unknown rule names; a renamed compile
# rule must fail the build, not silently strand every consumer
compdb.unlink(missing_ok=True)
raise EsphomeError(
"ninja produced an empty compile database; the generator's rule "
"names no longer match"
)
# write_file_if_changed keeps the mtime stable on no-op builds so the
# idedata cache stays valid
write_file_if_changed(compdb, result.stdout)
-288
View File
@@ -1,288 +0,0 @@
"""The parts of a ``build.ninja`` every native backend emits the same way.
Rule names match ``SOURCE_KIND_FOR_SUFFIX`` values (c, cxx, asm, aspp) and
``ninja.COMPILE_RULES``, which the compile database is asked for.
"""
from __future__ import annotations
from collections.abc import Collection, Sequence
import logging
import os
from pathlib import Path
import sys
from typing import TYPE_CHECKING
from esphome.build_helpers.idedata import is_joined_include
from esphome.build_helpers.ninja import (
escape as _e,
quote_path as _q,
shell_token as _shell_token,
)
from esphome.build_helpers.pch import (
PCH_GCH_NAME,
PCH_SUM_NAME,
log_pch_in_use,
pch_consumer_flags,
pch_identity,
pch_usable,
write_pch_headers,
)
from esphome.framework_helpers import strip_win_long_path_prefix
from esphome.helpers import write_file_if_changed
from esphome.platformio.library import SOURCE_KIND_FOR_SUFFIX
if TYPE_CHECKING:
from esphome.arduino.library import ArduinoLibrary
_LOGGER = logging.getLogger(__name__)
_BUILD_TOOL = Path(__file__).parent.parent / "build_gen" / "build_tool.py"
def collect_sources(root: Path, exclude: Collection[str] = frozenset()) -> list[Path]:
return sorted(
p
for p in root.rglob("*")
if p.suffix in SOURCE_KIND_FOR_SUFFIX and p.name not in exclude
)
def common_parent(paths: list[Path]) -> Path:
return Path(os.path.commonpath([str(p.parent) for p in paths]))
def _command(words: Sequence[Path | str]) -> str:
program, *args = words
return " ".join([_q(program), *(_shell_token(str(a)) for a in args)])
def tool_lines(
cc: Sequence[Path | str], cxx: Sequence[Path | str], ccache: str | None
) -> list[str]:
"""The file header: the compilers and the helper tools as variables.
A compiler is its program followed by any arguments it always takes.
"""
return [
"# Auto-generated by ESPHome",
"ninja_required_version = 1.5",
f"cc = {_command(cc)}",
f"cxx = {_command(cxx)}",
# The NSIS launcher starts Python with a \\?\ extended-length path
# that cmd.exe cannot spawn; same strip every other emitted binary
# path gets
f"python = {_q(strip_win_long_path_prefix(sys.executable))}",
f"buildtool = {_q(_BUILD_TOOL)}",
*([f"ccache = {_q(ccache)}"] if ccache else []),
"",
]
def compile_rule_lines(ccache: str | None) -> list[str]:
"""The compile rules; ``$own_includes`` is empty unless an edge sets it.
``ccache`` is the launcher ``tool_lines`` was given; the same test here
keeps the variable and the rules in step. It is spliced in rather than
left as an empty variable: a leading space on the command line makes
CreateProcess fail on Windows.
"""
launcher = "$ccache " if ccache else ""
return [
"rule c",
f" command = {launcher}$cc -MMD -MF $out.d $own_includes $cflags $flags -c $in -o $out",
" depfile = $out.d",
" deps = gcc",
" description = CC $out",
"rule cxx",
f" command = {launcher}$cxx -MMD -MF $out.d $own_includes $cxxflags $flags -c $in -o $out",
" depfile = $out.d",
" deps = gcc",
" description = CXX $out",
"rule aspp",
f" command = {launcher}$cc -MMD -MF $out.d -x assembler-with-cpp $own_includes $asflags $flags -c $in -o $out",
" depfile = $out.d",
" deps = gcc",
" description = AS $out",
# Plain assembler, as SCons's ASCOM: no preprocessor, so no
# depfile and no $flags (defines/includes) either
"rule asm",
f" command = {launcher}$cc -x assembler $asflags -c $in -o $out",
" description = AS $out",
]
def pch_rule_lines() -> list[str]:
"""The precompiled header rule, for a generator that emits one."""
return [
# No $ccache: the .gch embeds build dir paths
"rule pch",
" command = $cxx -MMD -MF $out.d -x c++-header $cxxflags $flags -c $in -o $out",
" depfile = $out.d",
" deps = gcc",
" description = PCH $out",
]
def pch_edges(
lines: list[str],
build_dir: Path,
src_dir: Path,
headers: Sequence[str],
cxxflags: Sequence[str],
src_flags: Sequence[str],
identity: Sequence[str],
cxx: Sequence[Path | str],
) -> tuple[str, str] | None:
"""Emit the precompiled header for the C++ src edges.
``headers`` are folded into one prefix header, ``src_flags`` are the
flags every src edge carries, ``identity`` names what else the compile
depends on, the compiler included, and ``cxx`` is what the host rule
asks. Returns the ``cxx_override`` for ``compile_edges``, or None
without a pch.
"""
if not pch_usable(cxx):
return None
if any(
tok == "-include" or tok.startswith("--include") or is_joined_include(tok)
for tok in cxxflags
):
# $cxxflags expands first and GCC only loads a .gch for the first
# -include
_LOGGER.warning(
"A -include in the compiler flags prevents the precompiled header from "
"loading; compiling without it"
)
return None
log_pch_in_use()
source = write_pch_headers(build_dir, headers)
write_file_if_changed(
build_dir / PCH_SUM_NAME,
pch_identity([*cxxflags, *src_flags], src_dir, tuple(headers), identity) + "\n",
)
gch = _e(PCH_GCH_NAME)
# The checksum file changes with anything the .gch depends on
lines.append(f"build {gch}: pch {_e(source)} | {_e(PCH_SUM_NAME)}")
if src_flags:
lines.append(f" flags = {' '.join(src_flags)}")
lines.append(f"srccxxflags = {' '.join([*src_flags, *pch_consumer_flags()])}")
return ("$srccxxflags", gch)
def ar_rule_lines(ar: Path | str) -> list[str]:
return [
"rule ar",
f" command = $python $buildtool ar {_q(ar)} $out $out.rsp",
" rspfile = $out.rsp",
" rspfile_content = $in_newline",
" description = AR $out",
]
def compile_edges(
lines: list[str],
sources: list[Path],
root: Path,
group: str,
flags: str = "",
own_includes: str = "",
cxx_override: tuple[str, str] | None = None,
) -> list[str]:
"""Emit compile edges for ``sources``; return the object paths.
``cxx_override`` is a (flags, implicit-dep) pair applied to C++ edges
only, replacing ``flags`` (used for the precompiled header).
"""
objects = []
for src in sources:
rel = src.relative_to(root).as_posix()
obj = f"obj/{group}/{rel}.o"
escaped_obj = _e(obj)
kind = SOURCE_KIND_FOR_SUFFIX[src.suffix]
override = cxx_override if kind == "cxx" else None
implicit = f" | {override[1]}" if override else ""
lines.append(f"build {escaped_obj}: {kind} {_e(src)}{implicit}")
if own_includes:
lines.append(f" own_includes = {own_includes}")
if edge_flags := override[0] if override else flags:
lines.append(f" flags = {edge_flags}")
# Escaped once here: the returned paths only ever appear in build
# statements (archive/link inputs), which use ninja escaping
objects.append(escaped_obj)
return objects
def library_edges(
lines: list[str], libraries: list[ArduinoLibrary], extra_flags: str = ""
) -> tuple[list[str], list[str]]:
"""Emit every library's compile and archive edges.
``extra_flags`` (raw ninja text) precede each library's own flags.
Returns the archive names and the objects that link directly. A
library's own include dirs lead its compile lines, as PlatformIO searched
them first: the include path is one global list, so another library's
header of the same name would shadow them.
"""
archives: list[str] = []
direct_objs: list[str] = []
for lib in libraries:
if not lib.sources:
# Header-only libraries are legitimate; the log makes an empty
# srcFilter or broken tree traceable before link errors do
_LOGGER.debug(
"Library %s has no source files; contributing includes only",
lib.name,
)
continue
objs = compile_edges(
lines,
lib.sources,
common_parent(lib.sources),
f"lib/{lib.name}",
flags=" ".join(filter(None, [extra_flags, *map(_shell_token, lib.flags)])),
own_includes=" ".join(f"-I{_q(d)}" for d in lib.include_dirs),
)
if not lib.lib_archive:
# libArchive: false / dot_a_linkage=false: hand the objects to
# the linker directly so unreferenced-but-required symbols
# (exception handlers, weak overrides) survive
direct_objs.extend(objs)
continue
archive = f"lib{lib.name}.a"
lines.append(f"build {_e(archive)}: ar {' '.join(objs)}")
archives.append(archive)
return archives, direct_objs
# One build flag: the flag, plus its argument when that is a separate token
Flag = tuple[str, ...]
# Flags whose path operand is the next token; gcc also takes it glued on
PATH_ARG_FLAGS = ("-include", "-imacros", "-isystem", "-iquote", "-idirafter")
# Flags whose path operand is glued on
PATH_PREFIXES = ("-I", "-L", *PATH_ARG_FLAGS)
def _anchor(path: str, base: Path) -> str:
if not path or Path(path).is_absolute():
return path
return str(base / path)
def anchor_path_flag(flag: Flag, base: Path) -> Flag:
"""Anchor a flag's relative path operand at ``base``.
PlatformIO ran the compiler from the build path; ninja runs it from
``.pioenvs/<name>``, where a relative operand would point elsewhere.
"""
name, *args = flag
if args:
if name in PATH_ARG_FLAGS:
return (name, _anchor(args[0], base))
return flag
for prefix in PATH_PREFIXES:
if name.startswith(prefix):
return (prefix + _anchor(name[len(prefix) :], base),)
return flag
-248
View File
@@ -1,248 +0,0 @@
"""Shared precompiled header policy for the build backends."""
from __future__ import annotations
from collections.abc import Iterable, Sequence
import hashlib
import logging
import os
from pathlib import Path
import posixpath
import re
import subprocess
import sys
from esphome.build_helpers.ccache import effective_ccache_basedir, parse_enable_env
from esphome.const import PLATFORM_NRF52
from esphome.helpers import write_file_if_changed
_LOGGER = logging.getLogger(__name__)
# The header and its sidecars live in the build directory
PCH_HEADER_NAME = "esphome_pch.h"
PCH_GCH_NAME = f"{PCH_HEADER_NAME}.gch"
# ccache hashes this instead of the .gch; also the freshness stamp
PCH_SUM_NAME = f"{PCH_GCH_NAME}.sum"
# The include list the .gch is compiled from
PCH_SOURCE_NAME = "esphome_pch_src.h"
# GCC can skip a .gch without a diagnostic and read the header of the same
# name, so that header is an error. Other tools get the include list.
PCH_GUARD_TEXT = f"""\
#if defined(__GNUC__) && !defined(__clang__) && !defined(__INTELLISENSE__)
#error "The precompiled header was not loaded"
#else
#include "{PCH_SOURCE_NAME}"
#endif
"""
# The cc1plus wrapper the PlatformIO script writes on arm64 macOS
PCH_CC1_DIR = "pch_cc1"
# What the PlatformIO script leaves in the project root, for cleanup
PCH_ARTIFACT_NAMES = (PCH_HEADER_NAME, PCH_GCH_NAME, PCH_SUM_NAME, PCH_SOURCE_NAME)
PCH_ARTIFACT_DIRS = (PCH_CC1_DIR,)
# The core headers every backend precompiles
PCH_DEFAULT_HEADERS = ("esphome/core/pch_prefix.h",)
# PlatformIO platforms that do not take the pch script
PCH_SCRIPT_EXCLUDED_PLATFORMS = frozenset(
{
PLATFORM_NRF52,
}
)
# What ccache needs to cache compiles that load a .gch
_CCACHE_PCH_SLOPPINESS = ("pch_defines", "time_macros")
# Both include forms: an angle include resolving under src/ enters the digest
_INCLUDE_RE = re.compile(rb'^\s*#\s*include\s+["<]([^">]+)[">]', re.MULTILINE)
def pch_enabled() -> bool:
"""Precompiled-header knob: default on, ``ESPHOME_PCH_ENABLE=0`` opts out."""
return parse_enable_env("ESPHOME_PCH_ENABLE") is not False
def pch_forced() -> bool:
"""``ESPHOME_PCH_ENABLE=1``: wanted even where the host rule says no."""
return parse_enable_env("ESPHOME_PCH_ENABLE") is True
# GCC bug 14940: before these releases the Windows loader maps a .gch only
# at its saved address. First fixed release per major, 16 on always fixed;
# PCH_WINDOWS_CMAKE_OLD_GCC and the pch_usable message spell the same table
PCH_WINDOWS_GCC_FIXED = {14: (14, 4), 15: (15, 3)}
PCH_WINDOWS_GCC_FIXED_DEFAULT = (16, 0)
# The same rule for CMake, which alone knows the version before configure
PCH_WINDOWS_CMAKE_OLD_GCC = (
"CMAKE_CXX_COMPILER_VERSION VERSION_LESS 14.4 OR "
"(CMAKE_CXX_COMPILER_VERSION VERSION_GREATER_EQUAL 15 AND "
"CMAKE_CXX_COMPILER_VERSION VERSION_LESS 15.3)"
)
def gcc_relocates_pch_on_windows(version: Sequence[int]) -> bool:
"""Whether a GCC of this version loads a .gch on Windows."""
if not version:
return False
fixed = PCH_WINDOWS_GCC_FIXED.get(version[0], PCH_WINDOWS_GCC_FIXED_DEFAULT)
return tuple(version[:2]) >= fixed
# GCC ends the first --version line with its version; clang names itself
_VERSION_RE = re.compile(r"\d+(?:\.\d+)+")
def gcc_version(cxx: Sequence[Path | str]) -> tuple[int, ...] | None:
"""The GCC version from ``--version``: () when it cannot be read, None
for a compiler that is not GCC."""
try:
result = subprocess.run(
[*cxx, "--version"], capture_output=True, text=True, check=False
)
except OSError as err:
_LOGGER.debug("Cannot run %s: %s", cxx[0], err)
return ()
banner = result.stdout.partition("\n")[0]
if "clang" in banner.lower():
return None
found = _VERSION_RE.findall(banner)
return tuple(int(part) for part in found[-1].split(".")) if found else ()
def pch_needs_gcc_check() -> bool:
"""Windows host with the knob unset: the compiler version decides."""
return sys.platform == "win32" and parse_enable_env("ESPHOME_PCH_ENABLE") is None
def pch_usable(cxx: Sequence[Path | str]) -> bool:
"""The knob plus the host rule; ``ESPHOME_PCH_ENABLE=1`` skips the rule."""
if not pch_enabled():
return False
if not pch_needs_gcc_check():
return True
version = gcc_version(cxx)
if version is None or gcc_relocates_pch_on_windows(version):
return True
_LOGGER.info(
"GCC %s cannot load a precompiled header on Windows (GCC bug 14940, "
"fixed in 14.4, 15.3 and 16); compiling without it "
"(set ESPHOME_PCH_ENABLE=1 to force)",
".".join(map(str, version)) or "of unknown version",
)
return False
def pch_consumer_flags() -> list[str]:
"""Flags a C++ src compile loads the pch with. The -include stays
relative: an absolute path would enter the ccache key."""
return ["-Winvalid-pch", "-Werror=invalid-pch", "-include", PCH_HEADER_NAME]
def ccache_pch_env() -> dict[str, str]:
"""What ccache needs to cache compiles that load a .gch, added to what
the user already set."""
if not pch_enabled():
return {}
sloppiness = [
item.strip()
for item in os.environ.get("CCACHE_SLOPPINESS", "").split(",")
if item.strip()
]
sloppiness += [item for item in _CCACHE_PCH_SLOPPINESS if item not in sloppiness]
env = {"CCACHE_SLOPPINESS": ",".join(sloppiness)}
if "CCACHE_PCH_EXTSUM" not in os.environ:
env["CCACHE_PCH_EXTSUM"] = "true"
return env
def pch_script_enabled() -> bool:
"""Whether this PlatformIO build takes the pch script."""
from esphome.core import CORE
return pch_enabled() and CORE.target_platform not in PCH_SCRIPT_EXCLUDED_PLATFORMS
def pch_header_text(include_headers: Iterable[str]) -> str:
"""The prefix-header source: exactly these includes, in order."""
return "".join(f'#include "{name}"\n' for name in include_headers)
def write_pch_headers(build_dir: Path, include_headers: Iterable[str]) -> Path:
"""Write the guard header and the include list; return the latter,
which is what the .gch compiles from."""
write_file_if_changed(build_dir / PCH_HEADER_NAME, PCH_GUARD_TEXT)
source = build_dir / PCH_SOURCE_NAME
write_file_if_changed(source, pch_header_text(include_headers))
return source
def _include_closure(src_dir: Path, roots: Iterable[str]) -> dict[str, bytes]:
"""Include closure of ``roots``: src-relative name -> contents.
Resolution mirrors the compiler (includer's dir, then src root). No
#ifdef evaluation: including too much is the safe direction. Headers
outside ``src_dir`` are covered by the version strings of the caller.
"""
seen: dict[str, bytes] = {}
stack: list[tuple[str, str]] = [(name, "") for name in roots]
while stack:
name, from_dir = stack.pop()
for candidate in (f"{from_dir}/{name}" if from_dir else name, name):
rel = posixpath.normpath(candidate)
if not rel.startswith("..") and (src_dir / rel).is_file():
break
else:
continue
if rel in seen:
continue
data = seen[rel] = (src_dir / rel).read_bytes()
parent = posixpath.dirname(rel)
stack.extend((inc.decode(), parent) for inc in _INCLUDE_RE.findall(data))
return seen
def pch_checksum(
src_dir: Path, include_headers: Iterable[str], extra: Iterable[str]
) -> str:
"""Digest of the prefix header's include closure plus ``extra``."""
digest = hashlib.sha256()
closure = _include_closure(src_dir, include_headers)
for name in sorted(closure):
digest.update(name.encode())
digest.update(closure[name])
digest.update(b"\0")
for item in extra:
digest.update(item.encode())
digest.update(b"\0")
return digest.hexdigest()
def pch_identity(
tokens: Iterable[str],
src_dir: Path,
include_headers: tuple[str, ...],
extra: Iterable[str],
) -> str:
"""The .sum digest: include closure, header text, ``extra`` and the
compile flags with the build path stripped, as ccache does."""
from esphome.core import CORE
flags = (
" ".join(tokens)
.replace(str(CORE.build_path), "")
.replace(effective_ccache_basedir(), "")
)
# The closure is sorted, so header order only enters via the text
return pch_checksum(
src_dir, include_headers, (pch_header_text(include_headers), *extra, flags)
)
_DISABLE_HINT = " (set ESPHOME_PCH_ENABLE=0 to disable)"
def log_pch_in_use() -> None:
_LOGGER.info("Compiling with a precompiled header%s", _DISABLE_HINT)
-180
View File
@@ -1,180 +0,0 @@
"""Run a native build tool (cmake, ninja) and relay its output.
Output is read from a pipe so it can be filtered here: a child that inherits
our stdout writes straight to the file descriptor, past any Python wrapper.
"""
from __future__ import annotations
import codecs
from contextlib import suppress
import logging
import os
from pathlib import Path
import re
import shutil
import subprocess
import sys
from typing import Any, TextIO
from esphome.util import ANSI_ESCAPE, RedirectText, shlex_quote
_LOGGER = logging.getLogger(__name__)
# Windows code page identifier for UTF-8, as used by ``chcp 65001``.
UTF8_CODEPAGE = 65001
# Same pattern idf.py uses to spot ninja status lines (``is_progression``).
_PROGRESS = re.compile(r"^\[\d+/\d+\]|.*\(\d+ \%\)$")
_READ_SIZE = 65536
def _get_kernel32() -> Any | None:
"""Return the Windows kernel32 module, or None on any other platform."""
if sys.platform != "win32":
return None
import ctypes
return ctypes.windll.kernel32
class Utf8Console:
"""Keep an attached Windows console on UTF-8 while a build tool runs.
esp_idf_size draws its table with Unicode box characters, and CMake
re-decodes a child's output with the console code page, which garbles the
table on any page but UTF-8. A console already on UTF-8 is left alone so
an overlapping build never records UTF-8 as the page to go back to.
"""
def __init__(self, kernel32: Any | None) -> None:
self._kernel32 = kernel32
self._codepages: tuple[int, int] | None = None
def __enter__(self) -> None:
kernel32 = self._kernel32
if kernel32 is None:
return
old_in = kernel32.GetConsoleCP()
old_out = kernel32.GetConsoleOutputCP()
# Both calls return 0 when no console is attached.
if not old_in or not old_out:
return
if old_in == UTF8_CODEPAGE and old_out == UTF8_CODEPAGE:
return
# Record first so a switch that fails part way is still undone.
self._codepages = (old_in, old_out)
kernel32.SetConsoleCP(UTF8_CODEPAGE)
kernel32.SetConsoleOutputCP(UTF8_CODEPAGE)
def __exit__(self, *exc_info: object) -> None:
if self._codepages is None:
return
old_in, old_out = self._codepages
self._codepages = None
self._kernel32.SetConsoleCP(old_in)
self._kernel32.SetConsoleOutputCP(old_out)
def _fit_terminal(text: str) -> str:
"""Elide the middle of ``text`` to fit the terminal, as idf.py does.
A width of 0 (a pipe, the dashboard) leaves the text whole.
"""
width = shutil.get_terminal_size((0, 0)).columns
if not width:
return text
if width <= 3:
return "." * width
if len(text) >= width:
keep = (width - 3) // 2
return f"{text[:keep]}...{text[len(text) - keep :]}"
return text
class ToolOutput(RedirectText):
"""RedirectText that can collapse ninja status lines into one line.
With ``progress`` each ``[n/m]`` line overwrites the previous one, the
way idf.py shows a build.
"""
def __init__(
self, out: TextIO, filter_lines: list[str] | None, progress: bool
) -> None:
super().__init__(out, filter_lines=filter_lines)
self._progress = progress
self._on_progress_line = False
def _splits_lines(self) -> bool:
return self._progress or super()._splits_lines()
def _emit_line(self, line: str) -> None:
if self._progress and _PROGRESS.match(line):
if not self._is_filtered(line):
text = _fit_terminal(line.strip("\r\n"))
self._write_color_replace(f"\r{text}\x1b[K")
self._on_progress_line = True
return
self._end_progress_line()
super()._emit_line(line)
def _end_progress_line(self) -> None:
if self._on_progress_line:
self._on_progress_line = False
self._write_color_replace(os.linesep)
def drain(self) -> None:
super().drain()
# Called from cleanup, so a broken stream must not hide the exit code.
with suppress(OSError, ValueError):
self._end_progress_line()
self._out.flush()
def run_build_tool(
cmd: list[str],
*,
cwd: Path,
env: dict[str, str],
filter_lines: list[str] | None = None,
progress: bool = False,
log_path: Path | None = None,
) -> int:
"""Run ``cmd`` and relay stdout and stderr, merged, to our stdout.
``log_path`` also gets the full, unfiltered output without color codes, as
idf.py wrote its logs (its hint patterns expect plain text). Returns the
exit code.
"""
_LOGGER.debug("Running: %s", " ".join(shlex_quote(arg) for arg in cmd))
_LOGGER.debug(" in directory: %s", cwd)
output = ToolOutput(sys.stdout, filter_lines, progress)
decoder = codecs.getincrementaldecoder("utf-8")(errors="replace")
if log_path is not None:
log_path.parent.mkdir(parents=True, exist_ok=True)
with (
Path(log_path or os.devnull).open("w", encoding="utf-8", newline="") as log,
Utf8Console(_get_kernel32()),
subprocess.Popen(
cmd,
cwd=cwd,
env=env,
stdout=subprocess.PIPE,
stderr=subprocess.STDOUT,
close_fds=False,
) as proc,
):
try:
# read1 returns as soon as anything is available, so output
# streams while the tool runs.
while chunk := proc.stdout.read1(_READ_SIZE):
text = decoder.decode(chunk)
log.write(ANSI_ESCAPE.sub("", text))
output.write(text)
if tail := decoder.decode(b"", final=True):
log.write(ANSI_ESCAPE.sub("", tail))
output.write(tail)
finally:
output.drain()
return proc.returncode
+3 -13
View File
@@ -16,8 +16,8 @@ def tools_cache_path(env_var: str, subdir: str) -> Path:
from esphome.helpers import get_str_env
if prefix := get_str_env(env_var, "").strip():
# resolve(): a symlinked prefix would otherwise record a second
# spelling of the same paths in the build tree
# resolve(): symlinked prefixes otherwise trip idf.py's
# venv-mismatch warning on every build
return Path(prefix).expanduser().resolve()
# appauthor=False keeps the Windows path short (no vendor segment);
# deep IDF trees run into MAX_PATH otherwise
@@ -33,14 +33,4 @@ def tools_cache_path(env_var: str, subdir: str) -> Path:
IDF_TOOLS_CACHE = ("ESPHOME_ESP_IDF_PREFIX", "idf")
SDK_NRF_TOOLS_CACHE = ("ESPHOME_SDK_NRF_PREFIX", "sdk-nrf")
ARDUINO8266_TOOLS_CACHE = ("ESPHOME_ARDUINO8266_PREFIX", "arduino8266")
# The host backend installs nothing; the entry only holds its ccache dir
HOST_TOOLS_CACHE = ("ESPHOME_HOST_PREFIX", "host")
# PlatformIO installs into its own dirs; the entry is its ccache dir itself
PLATFORMIO_CCACHE = ("ESPHOME_PLATFORMIO_CCACHE_DIR", "platformio-ccache")
TOOLS_CACHE_SPECS = (
IDF_TOOLS_CACHE,
SDK_NRF_TOOLS_CACHE,
ARDUINO8266_TOOLS_CACHE,
HOST_TOOLS_CACHE,
PLATFORMIO_CCACHE,
)
TOOLS_CACHE_SPECS = (IDF_TOOLS_CACHE, SDK_NRF_TOOLS_CACHE, ARDUINO8266_TOOLS_CACHE)
+12 -20
View File
@@ -102,21 +102,25 @@ _KNOWN_FILE_EXTENSIONS = frozenset(
)
# Matches !secret references in YAML text. An optional surrounding
# quote pair around the key is allowed and ignored: YAML treats
# ``!secret 'foo'`` and ``!secret foo`` as the same key. This is
# intentionally a simple regex scan rather than a YAML parse — it may
# match inside comments or multi-line strings, which is the conservative
# direction (include more secrets rather than fewer).
_SECRET_RE = re.compile(r"""!secret\s+['"]?([^\s'"]+)""")
def _find_used_secret_keys(yaml_files: list[Path]) -> set[str]:
"""Scan YAML files for ``!secret <key>`` references."""
keys: set[str] = set()
for fpath in yaml_files:
try:
text = fpath.read_text(encoding="utf-8")
except (OSError, UnicodeDecodeError) as err:
_LOGGER.warning(
"Could not scan %s for !secret references (%s); the bundled "
"secret set may be incomplete",
fpath,
err,
)
except (OSError, UnicodeDecodeError):
continue
keys |= yaml_util.find_secret_references(text)
for match in _SECRET_RE.finditer(text):
keys.add(match.group(1))
return keys
@@ -418,18 +422,6 @@ class ConfigBundleCreator:
must ship every candidate so the remote build can pick any one.
"""
discovered = yaml_util.discover_user_yaml_files(self._config_path)
if discovered.load_errors:
_LOGGER.warning(
"Bundle may be incomplete; could not load all configuration files: %s",
"; ".join(discovered.load_errors),
)
if discovered.unresolved:
_LOGGER.warning(
"Bundle may be incomplete; %d !include path(s) use "
"substitutions and cannot be captured: %s",
len(discovered.unresolved),
", ".join(discovered.unresolved),
)
self._secrets_paths.update(discovered.secrets)
# A !secret inside a file this re-parse does not reach (for example
# a git-fetched package the builder re-fetches) still resolves
-2
View File
@@ -1,2 +0,0 @@
"""Commands of the esphome command line, one module each, imported by
__main__ only when they run so that startup stays light."""
-164
View File
@@ -1,164 +0,0 @@
"""``esphome rename``."""
from __future__ import annotations
import argparse
from pathlib import Path
import re
from esphome import yaml_edit, yaml_util
from esphome.const import (
ALLOWED_NAME_CHARS,
CONF_ESPHOME,
CONF_NAME,
CONF_SUBSTITUTIONS,
)
from esphome.core import CORE, EsphomeError
from esphome.log import AnsiFore, color
from esphome.types import ConfigType
from esphome.util import ESPHOME_COMMAND, run_external_process, safe_print
def _revert(new_path: Path, why: str) -> int:
"""Say why the rename stopped and take the new file back; an orphan the
next attempt would trip over is reported."""
safe_print(color(AnsiFore.BOLD_RED, f"Rename failed: {why}"))
try:
new_path.unlink(missing_ok=True)
except OSError as err:
safe_print(color(AnsiFore.BOLD_RED, f"Could not remove {new_path}: {err}"))
return 1
def command_rename(args: argparse.Namespace, config: ConfigType) -> int | None:
"""Rename the device: a new file with the name line rewritten, validated
and installed, then the old file removed."""
new_name = args.name
for c in new_name:
if c not in ALLOWED_NAME_CHARS:
safe_print(
color(
AnsiFore.BOLD_RED,
f"'{c}' is an invalid character for names. Valid characters are: "
f"{ALLOWED_NAME_CHARS} (lowercase, no spaces)",
)
)
return 1
yaml = yaml_util.load_yaml(CORE.config_path)
def name_edit() -> tuple[str, yaml_edit.LineEdit]:
"""The name and the line to rewrite: the name's own line, or the
substitution's line it comes from, as a plain value in this file."""
esphome_conf = yaml.get(CONF_ESPHOME)
if not isinstance(esphome_conf, dict) or CONF_NAME not in esphome_conf:
raise EsphomeError(f"no '{CONF_ESPHOME}: {CONF_NAME}:' in the file")
old_name = str(esphome_conf[CONF_NAME])
mapping, field = esphome_conf, CONF_NAME
if match := re.match(r"^\$\{?([a-zA-Z0-9_]+)\}?$", old_name):
mapping, field = yaml.get(CONF_SUBSTITUTIONS), match.group(1)
if not isinstance(mapping, dict) or field not in mapping:
raise EsphomeError(f"the substitution '{field}' is not in the file")
old_name = str(mapping[field])
# Only read here; the rewritten text goes to a new file, so the
# source may live anywhere the config path points to
source = yaml_edit.source_of(mapping, field)
if source is None or source[0].resolve() != CORE.config_path.resolve():
raise EsphomeError(f"'{field}' was not read from {CORE.config_path}")
doc, line_no = source
text = yaml_edit.line_at(doc, line_no)
if (line_match := yaml_edit.field_line_re(field, old_name).match(text)) is None:
raise EsphomeError(f"'{field}' is not a plain value on {doc}:{line_no + 1}")
# The new value is always quoted, whatever the old line had
return old_name, yaml_edit.LineEdit(
doc, line_no, text, yaml_edit.rewrite(line_match, new_name, '"')
)
try:
old_name, edit = name_edit()
except EsphomeError as err:
safe_print(
color(
AnsiFore.BOLD_RED,
f"Complex YAML files cannot be automatically renamed: {err}",
)
)
return 1
# ``new_name == old_name`` (after substitution resolution) is
# a no-op rewrite that would still queue a pointless re-flash.
# Catch it before the path-equality check below — covers the
# case where the config filename doesn't match the device name
# (e.g. ``weird-file.yaml`` whose ``esphome.name`` is
# ``kitchen``; running ``esphome rename weird-file.yaml kitchen``
# would otherwise just re-flash the same hostname).
if new_name == old_name:
safe_print(
color(
AnsiFore.BOLD_RED,
f"'{new_name}' is already the device's name.",
)
)
return 1
new_path: Path = CORE.config_dir / (new_name + ".yaml")
if new_path.resolve() == CORE.config_path.resolve():
safe_print(
color(
AnsiFore.BOLD_RED,
f"'{new_name}' is already the device's name.",
)
)
return 1
if new_path.exists():
safe_print(
color(
AnsiFore.BOLD_RED,
f"Cannot rename: {new_path} already exists. "
"Refusing to overwrite an existing configuration.",
)
)
return 1
safe_print(
f"Updating {color(AnsiFore.CYAN, str(CORE.config_path))} to {color(AnsiFore.CYAN, str(new_path))}"
)
print()
try:
yaml_edit.write_keeping_mode(
new_path,
yaml_edit.rewritten_text(yaml_edit.read_text(CORE.config_path), [edit]),
like=CORE.config_path,
)
except EsphomeError as err:
return _revert(new_path, str(err))
if run_external_process(*ESPHOME_COMMAND, "config", str(new_path)) != 0:
return _revert(new_path, "the new configuration does not validate")
cli_args = [
"run",
str(new_path),
"--no-logs",
"--device",
CORE.address,
]
if args.dashboard:
cli_args.insert(0, "--dashboard")
try:
rc = run_external_process(*ESPHOME_COMMAND, *cli_args)
except KeyboardInterrupt:
rc = 1
if rc != 0:
return _revert(
new_path,
"the install did not finish; the device may already run the new name",
)
CORE.config_path.unlink()
safe_print(color(AnsiFore.BOLD_GREEN, "SUCCESS"))
print()
return 0
+1 -6
View File
@@ -31,23 +31,17 @@ from esphome.cpp_generator import ( # noqa: F401
add_global,
add_library,
add_platformio_option,
extern_progmem_array,
get_variable,
get_variable_with_full_id,
is_static_pointer,
is_template,
new_Pvariable,
new_variable,
process_lambda,
progmem_array,
progmem_string,
safe_exp,
set_cpp_standard,
shared_progmem_array,
shared_progmem_string,
statement,
static_const_array,
static_function,
templatable,
variable,
with_local_variable,
@@ -70,6 +64,7 @@ from esphome.cpp_types import ( # noqa: F401
Application,
Component,
ComponentPtr,
Controller,
EntityBase,
EntityCategory,
ESPTime,
-1
View File
@@ -6,6 +6,5 @@ See the component-alias section of esphome/loader.py.
# alias -> (canonical component, removal version or None)
COMPONENT_ALIASES: dict[str, tuple[str, str | None]] = {
"esp32_improv": ("improv_ble", "2027.4.0"),
"rp2040": ("rp2", "2027.7.0"),
}
-1
View File
@@ -1,2 +1 @@
CODEOWNERS = ["@MrSuicideParrot"]
DOMAIN = "a01nyub"
+1 -1
View File
@@ -6,7 +6,7 @@
namespace esphome::a01nyub {
ESPHOME_LOG_TAG(TAG, "a01nyub.sensor");
static const char *const TAG = "a01nyub.sensor";
void A01nyubComponent::loop() {
uint8_t data;
-1
View File
@@ -1,2 +1 @@
CODEOWNERS = ["@TH-Braemer"]
DOMAIN = "a02yyuw"
+1 -1
View File
@@ -6,7 +6,7 @@
namespace esphome::a02yyuw {
ESPHOME_LOG_TAG(TAG, "a02yyuw.sensor");
static const char *const TAG = "a02yyuw.sensor";
void A02yyuwComponent::loop() {
uint8_t data;
-1
View File
@@ -1 +0,0 @@
DOMAIN = "a4988"
+1 -1
View File
@@ -3,7 +3,7 @@
namespace esphome::a4988 {
ESPHOME_LOG_TAG(TAG, "a4988.stepper");
static const char *const TAG = "a4988.stepper";
void A4988::setup() {
if (this->sleep_pin_ != nullptr) {
@@ -1,2 +1 @@
CODEOWNERS = ["@DAVe3283"]
DOMAIN = "absolute_humidity"
@@ -3,7 +3,7 @@
namespace esphome::absolute_humidity {
ESPHOME_LOG_TAG(TAG, "absolute_humidity.sensor");
static const char *const TAG{"absolute_humidity.sensor"};
void AbsoluteHumidityComponent::setup() {
this->temperature_sensor_->add_on_state_callback([this](float state) {
-1
View File
@@ -1 +0,0 @@
DOMAIN = "ac_dimmer"
+1 -1
View File
@@ -14,7 +14,7 @@
namespace esphome::ac_dimmer {
ESPHOME_LOG_TAG(TAG, "ac_dimmer");
static const char *const TAG = "ac_dimmer";
// Global array to store dimmer objects
static AcDimmerDataStore *all_dimmers[32]; // NOLINT(cppcoreguidelines-avoid-non-const-global-variables)
@@ -9,7 +9,7 @@
#include "esp_clk_tree.h"
#include "soc/clk_tree_defs.h"
ESPHOME_LOG_TAG(TAG, "hw_timer_esp_idf");
static const char *const TAG = "hw_timer_esp_idf";
namespace esphome::ac_dimmer {
-1
View File
@@ -9,7 +9,6 @@ from esphome.cpp_generator import MockObj
from esphome.types import ConfigType
DEPENDENCIES = ["uart"]
DOMAIN = "adalight"
adalight_ns = cg.esphome_ns.namespace("adalight")
AdalightLightEffect = adalight_ns.class_(
@@ -3,7 +3,7 @@
namespace esphome::adalight {
ESPHOME_LOG_TAG(TAG, "adalight_light_effect");
static const char *const TAG = "adalight_light_effect";
static const uint32_t ADALIGHT_ACK_INTERVAL = 1000;
static const uint32_t ADALIGHT_RECEIVE_TIMEOUT = 1000;
-1
View File
@@ -22,7 +22,6 @@ from esphome.core import CORE
from esphome.types import ConfigType
CODEOWNERS = ["@esphome/core"]
DOMAIN = "adc"
adc_ns = cg.esphome_ns.namespace("adc")
+5 -5
View File
@@ -94,7 +94,7 @@ class ADCSensor final : public sensor::Sensor, public PollingComponent, public v
/// - SamplingMode::MIN: Use the lowest sample value
/// - SamplingMode::MAX: Use the highest sample value
/// @param sampling_mode The desired sampling mode to use for aggregating ADC samples.
void set_sampling_mode(SamplingMode sampling_mode) { this->sampling_mode_ = sampling_mode; }
void set_sampling_mode(SamplingMode sampling_mode);
/// Perform a single ADC sampling operation and return the measured value.
/// This function handles raw readings, calibration, and averaging as needed.
@@ -128,10 +128,10 @@ class ADCSensor final : public sensor::Sensor, public PollingComponent, public v
#endif // USE_RP2
protected:
uint8_t sample_count_{1}; // Must match DEFAULT_SAMPLES in sensor.py
bool output_raw_{false}; // Must match CONF_RAW default in sensor.py
uint8_t sample_count_{1};
bool output_raw_{false};
InternalGPIOPin *pin_;
SamplingMode sampling_mode_{SamplingMode::AVG}; // Must match DEFAULT_SAMPLING_MODE in sensor.py
SamplingMode sampling_mode_{SamplingMode::AVG};
#ifdef USE_ESP32
float sample_autorange_();
@@ -139,7 +139,7 @@ class ADCSensor final : public sensor::Sensor, public PollingComponent, public v
bool autorange_{false};
adc_oneshot_unit_handle_t adc_handle_{nullptr};
adc_cali_handle_t calibration_handle_{nullptr};
adc_atten_t attenuation_{ADC_ATTEN_DB_0}; // Must match DEFAULT_ATTENUATION in sensor.py
adc_atten_t attenuation_{ADC_ATTEN_DB_0};
adc_channel_t channel_{};
adc_unit_t adc_unit_{};
struct SetupFlags {
+3 -1
View File
@@ -3,7 +3,7 @@
namespace esphome::adc {
ESPHOME_LOG_TAG(TAG, "adc");
static const char *const TAG = "adc";
const LogString *sampling_mode_to_str(SamplingMode mode) {
switch (mode) {
@@ -76,4 +76,6 @@ void ADCSensor::set_sample_count(uint8_t sample_count) {
}
}
void ADCSensor::set_sampling_mode(SamplingMode sampling_mode) { this->sampling_mode_ = sampling_mode; }
} // namespace esphome::adc
+1 -1
View File
@@ -6,7 +6,7 @@
namespace esphome::adc {
ESPHOME_LOG_TAG(TAG, "adc");
static const char *const TAG = "adc";
adc_oneshot_unit_handle_t ADCSensor::shared_adc_handles[2] = {nullptr, nullptr};
@@ -13,7 +13,7 @@ ADC_MODE(ADC_VCC)
namespace esphome::adc {
ESPHOME_LOG_TAG(TAG, "adc");
static const char *const TAG = "adc";
void ADCSensor::setup() {
#ifndef USE_ADC_SENSOR_VCC
@@ -5,7 +5,7 @@
namespace esphome::adc {
ESPHOME_LOG_TAG(TAG, "adc");
static const char *const TAG = "adc";
void ADCSensor::setup() {
#ifndef USE_ADC_SENSOR_VCC
+1 -1
View File
@@ -17,7 +17,7 @@
namespace esphome::adc {
ESPHOME_LOG_TAG(TAG, "adc");
static const char *const TAG = "adc";
// The on-die temperature sensor sits on the last ADC channel: input 4 on RP2040
// and RP2350A, but input 8 on RP2350B, which has eight external channels rather
+1 -1
View File
@@ -7,7 +7,7 @@
namespace esphome::adc {
ESPHOME_LOG_TAG(TAG, "adc");
static const char *const TAG = "adc";
void ADCSensor::setup() {
if (!adc_is_ready_dt(this->channel_)) {
+8 -24
View File
@@ -48,11 +48,6 @@ AUTO_LOAD = ["voltage_sampler"]
CONF_SAMPLES = "samples"
CONF_SAMPLING_MODE = "sampling_mode"
# Keep in sync with the initializers in adc_sensor.h
DEFAULT_ATTENUATION = "0db"
DEFAULT_SAMPLING_MODE = "avg"
DEFAULT_SAMPLES = 1
_attenuation = cv.enum(ATTENUATION_MODES, lower=True)
_sampling_mode = cv.enum(SAMPLING_MODES, lower=True)
@@ -70,10 +65,7 @@ def validate_config(config: ConfigType) -> ConfigType:
):
raise cv.Invalid("ESP32-S31 only supports 'attenuation: 0db'")
if (
config.get(CONF_ATTENUATION, None) == "auto"
and config.get(CONF_SAMPLES, DEFAULT_SAMPLES) > 1
):
if config.get(CONF_ATTENUATION, None) == "auto" and config.get(CONF_SAMPLES, 1) > 1:
raise cv.Invalid(
"Automatic attenuation cannot be used when multisampling is set"
)
@@ -121,16 +113,12 @@ CONFIG_SCHEMA = cv.All(
{
cv.Required(CONF_PIN): validate_adc_pin,
cv.Optional(CONF_RAW, default=False): cv.boolean,
cv.SplitDefault(CONF_ATTENUATION, esp32=DEFAULT_ATTENUATION): cv.All(
cv.SplitDefault(CONF_ATTENUATION, esp32="0db"): cv.All(
cv.only_on_esp32, _attenuation
),
cv.OnlyWith(CONF_NRF_SAADC, PLATFORM_NRF52): cv.declare_id(adc_dt_spec),
cv.Optional(CONF_SAMPLES, default=DEFAULT_SAMPLES): cv.int_range(
min=1, max=255
),
cv.Optional(
CONF_SAMPLING_MODE, default=DEFAULT_SAMPLING_MODE
): _sampling_mode,
cv.Optional(CONF_SAMPLES, default=1): cv.int_range(min=1, max=255),
cv.Optional(CONF_SAMPLING_MODE, default="avg"): _sampling_mode,
}
)
.extend(cv.polling_component_schema("60s")),
@@ -167,13 +155,9 @@ async def to_code(config: ConfigType) -> None:
pin = await cg.gpio_pin_expression(config[CONF_PIN])
cg.add(var.set_pin(pin))
# Skip the setters when the config matches the C++ initializers.
if config[CONF_RAW]:
cg.add(var.set_output_raw(True))
if (samples := config[CONF_SAMPLES]) != DEFAULT_SAMPLES:
cg.add(var.set_sample_count(samples))
if (sampling_mode := config[CONF_SAMPLING_MODE]) != DEFAULT_SAMPLING_MODE:
cg.add(var.set_sampling_mode(sampling_mode))
cg.add(var.set_output_raw(config[CONF_RAW]))
cg.add(var.set_sample_count(config[CONF_SAMPLES]))
cg.add(var.set_sampling_mode(config[CONF_SAMPLING_MODE]))
if CORE.is_esp32:
# Re-enable ESP-IDF's ADC driver (excluded by default to save compile time)
@@ -182,7 +166,7 @@ async def to_code(config: ConfigType) -> None:
if attenuation := config.get(CONF_ATTENUATION):
if attenuation == "auto":
cg.add(var.set_autorange(cg.global_ns.true))
elif attenuation != DEFAULT_ATTENUATION:
else:
cg.add(var.set_attenuation(attenuation))
variant = get_esp32_variant()
@@ -5,7 +5,6 @@ from esphome.const import CONF_ID
from esphome.types import ConfigType
DEPENDENCIES = ["spi"]
DOMAIN = "adc128s102"
MULTI_CONF = True
CODEOWNERS = ["@DeerMaximum"]
+1 -1
View File
@@ -3,7 +3,7 @@
namespace esphome::adc128s102 {
ESPHOME_LOG_TAG(TAG, "adc128s102");
static const char *const TAG = "adc128s102";
float ADC128S102::get_setup_priority() const { return setup_priority::HARDWARE; }
@@ -4,7 +4,7 @@
namespace esphome::adc128s102 {
ESPHOME_LOG_TAG(TAG, "adc128s102.sensor");
static const char *const TAG = "adc128s102.sensor";
ADC128S102Sensor::ADC128S102Sensor(uint8_t channel) : channel_(channel) {}
@@ -1 +0,0 @@
DOMAIN = "addressable_light"
@@ -3,7 +3,7 @@
namespace esphome::addressable_light {
ESPHOME_LOG_TAG(TAG, "addressable_light.display");
static const char *const TAG = "addressable_light.display";
int AddressableLightDisplay::get_width_internal() { return this->width_; }
int AddressableLightDisplay::get_height_internal() { return this->height_; }
-1
View File
@@ -1 +0,0 @@
DOMAIN = "ade7880"
+1 -1
View File
@@ -15,7 +15,7 @@
namespace esphome::ade7880 {
ESPHOME_LOG_TAG(TAG, "ade7880");
static const char *const TAG = "ade7880";
void IRAM_ATTR ADE7880Store::gpio_intr(ADE7880Store *arg) { arg->reset_done = true; }
-1
View File
@@ -1,2 +1 @@
CODEOWNERS = ["@angelnu"]
DOMAIN = "ade7953"
@@ -27,7 +27,6 @@ from esphome.cpp_generator import MockObj
from esphome.types import ConfigType
CODEOWNERS = ["@angelnu"]
DOMAIN = "ade7953_base"
CONF_CURRENT_A = "current_a"
CONF_CURRENT_B = "current_b"
@@ -5,7 +5,7 @@
namespace esphome::ade7953_base {
ESPHOME_LOG_TAG(TAG, "ade7953");
static const char *const TAG = "ade7953";
constexpr uint16_t CONFIG_DEFAULT = 0x8004u;
constexpr uint16_t CONFIG_LOCK_BIT = 0x8000u;
@@ -1,2 +1 @@
CODEOWNERS = ["@angelnu"]
DOMAIN = "ade7953_i2c"
@@ -4,7 +4,7 @@
namespace esphome::ade7953_i2c {
ESPHOME_LOG_TAG(TAG, "ade7953");
static const char *const TAG = "ade7953";
void AdE7953I2c::dump_config() {
ESP_LOGCONFIG(TAG, "ADE7953_i2c:");
@@ -1,2 +1 @@
CODEOWNERS = ["@angelnu"]
DOMAIN = "ade7953_spi"
@@ -4,7 +4,7 @@
namespace esphome::ade7953_spi {
ESPHOME_LOG_TAG(TAG, "ade7953");
static const char *const TAG = "ade7953";
// Datasheet requires at least 1.2µs after clearing CONFIG LOCK_BIT before raising CS
constexpr uint8_t CONFIG_LOCK_SETTLE_US = 2;
-1
View File
@@ -5,7 +5,6 @@ from esphome.const import CONF_ID
from esphome.types import ConfigType
DEPENDENCIES = ["i2c"]
DOMAIN = "ads1115"
MULTI_CONF = True
ads1115_ns = cg.esphome_ns.namespace("ads1115")
+1 -1
View File
@@ -4,7 +4,7 @@
namespace esphome::ads1115 {
ESPHOME_LOG_TAG(TAG, "ads1115");
static const char *const TAG = "ads1115";
static const uint8_t ADS1115_REGISTER_CONVERSION = 0x00;
static const uint8_t ADS1115_REGISTER_CONFIG = 0x01;
@@ -4,7 +4,7 @@
namespace esphome::ads1115 {
ESPHOME_LOG_TAG(TAG, "ads1115.sensor");
static const char *const TAG = "ads1115.sensor";
float ADS1115Sensor::sample() {
return this->parent_->request_measurement(this->multiplexer_, this->gain_, this->resolution_, this->samplerate_);
-1
View File
@@ -6,7 +6,6 @@ from esphome.types import ConfigType
CODEOWNERS = ["@solomondg1"]
DEPENDENCIES = ["spi"]
DOMAIN = "ads1118"
MULTI_CONF = True
CONF_ADS1118_ID = "ads1118_id"
+1 -1
View File
@@ -4,7 +4,7 @@
namespace esphome::ads1118 {
ESPHOME_LOG_TAG(TAG, "ads1118");
static const char *const TAG = "ads1118";
static const uint8_t ADS1118_DATA_RATE_860_SPS = 0b111;
void ADS1118::setup() {
@@ -4,7 +4,7 @@
namespace esphome::ads1118 {
ESPHOME_LOG_TAG(TAG, "ads1118.sensor");
static const char *const TAG = "ads1118.sensor";
void ADS1118Sensor::dump_config() {
LOG_SENSOR(" ", "ADS1118 Sensor", this);

Some files were not shown because too many files have changed in this diff Show More