A quoted include path keeps its hash in the shared-configuration scan

Locates the current key on its yaml or secrets.yaml line, writes the new
one with the editor from #20602, keeps the replaced key as old_key, checks
the device with the probe from #20601, builds, uploads with the current key
and confirms with the new one; anything short of a reached device puts the
files back.

.agents/skills/code-review/SKILL.md

@@ -0,0 +1,116 @@
+---
+name: code-review
+description: Review guidance for ESPHome pull requests. Use this when reviewing a pull request that changes ESPHome Python, C++, or component code, to check it against the project's coding conventions, embedded-systems memory rules, testing requirements, and breaking-change policy.
+---
+
+# Reviewing ESPHome pull requests
+
+ESPHome parses YAML into C++ firmware for memory-constrained microcontrollers
+(ESP32, ESP8266, RP2040, LibreTiny). Review changes with that in mind: RAM and
+flash are scarce, and code runs unattended for months.
+
+`AGENTS.md` in the repository root is the full contributor guide and the
+authority when it disagrees with this summary. The developer documentation at
+https://developers.esphome.io explains the component lifecycle and the reasoning
+behind these rules. This skill lists the concrete things worth flagging in a
+review; read `AGENTS.md` for the detail behind any item.
+
+Only raise findings that the diff actually introduces or changes. Do not ask for
+drive-by cleanup of pre-existing code the PR did not touch.
+
+## Memory and embedded constraints (highest value)
+
+Heap allocation after `setup()` is treated as a reliability bug, not a
+performance nit, because it fragments a small shared heap. Flag:
+
+- New heap allocation on a hot path or after setup that could be avoided.
+- `std::vector` where the size is known at compile time (use `std::array`, or
+  `StaticVector<T, N>` when a `push_back` API is needed) or fixed at runtime
+  init (use `FixedVector<T>`).
+- Listener / child-entity registration lists stored as `std::vector`; these have
+  a compile-time-known count and should use `cg.slot_counter()` plus
+  `StaticVector`.
+- `std::vector<uint8_t>` for a byte buffer that never grows: prefer
+  `std::unique_ptr<uint8_t[]>` or `std::array`.
+- `std::map` / `std::set` / `std::unordered_map` for small datasets (1-16
+  elements): a `std::vector` of a small struct with linear search is lighter.
+- `std::deque` anywhere: it allocates 512-byte blocks and should be avoided.
+- `std::string` storing a value set once from config: prefer `StringRef` (the
+  literal already lives in flash).
+- `std::string` / `std::to_string` / string-returning helpers on hot paths where
+  a buffer or view API exists.
+
+## C++ conventions
+
+- Include what you use: a file referencing a symbol must include the header that
+  declares it, even if it currently arrives transitively. New or changed uses of
+  a symbol need the matching include.
+- Prefix all member access with `this->`.
+- Naming: `lower_snake_case` for functions/methods/variables, `UpperCamelCase`
+  for classes/structs/enums, `UPPER_SNAKE_CASE` for namespace-scope constants,
+  trailing underscore on protected/private fields.
+- `enum class` values must be prefixed with the enum name in `UPPER_SNAKE_CASE`
+  (e.g. `UARTFlushResult::UART_FLUSH_RESULT_SUCCESS`). Bare names like `SUCCESS`,
+  `FAIL`, or `OK` collide with SDK macros on some platforms and break the build.
+- Prefer `const`/`enum` over `#define`; `#define` is only for conditional
+  compilation and code-generation sizes.
+- Never call `millis()` in a `loop()` body; use
+  `App.get_loop_component_start_time()`. A rate-limit gate below ~16 ms (the loop
+  period) does nothing.
+- Pick the timing primitive by cadence: gated `loop()` under 250 ms,
+  `set_interval` at 500 ms and above.
+- Do not override a base method to return the value it already returns (e.g.
+  `get_setup_priority()` returning `setup_priority::DATA`).
+- Wrap string literals passed as printf `%s` args in `LOG_STR_LITERAL()`.
+- Required, invariant dependencies should be constructor parameters, not setters.
+- Callback registration methods must be templated (`template<typename F>`), not
+  typed as `std::function`, so lightweight forwarders avoid a heap allocation.
+- Two-space indent, `using` over `typedef`, wrap at 120 columns.
+
+## Python conventions
+
+- Type-annotate every new function signature (params and return), new dataclass
+  fields, and new module-level variables. Import `ConfigType` from
+  `esphome.types`.
+- Use the walrus operator to avoid a double lookup, e.g.
+  `if (blah := config.get(CONF_BLAH)) is not None:`.
+- Reuse existing validators from `config_validation.py` (`cv.rename_key`,
+  `cv.has_exactly_one_key`, etc.) via `cv.All(...)` instead of hand-rolling.
+- `esphome/const.py` is frozen: no new `CONF_` constants there. Define them in
+  the component's own `.py`, or in `esphome/components/const/__init__.py` when
+  shared. The same constant defined in three or more component files fails CI.
+- State that must persist during code generation goes in `CORE.data` namespaced
+  under the component `DOMAIN` (a `@dataclass`), not module-level mutable globals.
+- Prefer callback-based triggers via `build_callback_automation()`; only use a
+  `Trigger<Ts...>` subclass when the forwarder needs mutable state.
+
+## Testing and coverage
+
+- New and changed lines and branches need test coverage, including defensive
+  early-returns, error paths, and no-op guards. A mocked-out function is not
+  covered; exercise the real call path too.
+- Component YAML tests live in `tests/components/<component>/`. Never define
+  buses (uart, i2c, spi, modbus) directly in a test file: pull them from
+  `tests/test_build_components/common/` through dict-style `packages:` so CI can
+  group builds. List-style packages or top-level merge keys block grouping.
+- Config-only checks use the `validate.*.yaml` prefix; compiled checks use
+  `test.*.yaml`.
+
+## Breaking changes and public API
+
+- Base classes under `esphome/core/` and documented config options are public
+  API. Undocumented `public` members of a component are internal.
+- A breaking change needs justification, a migration path in the PR description,
+  and a deprecation window where feasible (`ESPDEPRECATED` in C++,
+  `cv.rename_key(..., removed_in=...)` in Python). Changing a codegen-injected
+  lambda signature is not a breaking change.
+
+## Process and PR hygiene
+
+- PR titles start with a `[tag]` prefix: the component name (e.g. `[uart] ...`)
+  or `[core]` for shared code.
+- Prose in docs, comments, and commit messages should be plain English. Keep
+  inline comments short and only where the code is not self-explanatory; do not
+  restate what the code says.
+- Verify the PR fills out `.github/PULL_REQUEST_TEMPLATE.md` and adds
+  `CODEOWNERS` entries for a new component.

.claude/skills

@@ -0,0 +1 @@
+../.agents/skills

.devcontainer/Dockerfile

@@ -31,7 +31,4 @@ RUN \
     platformio settings set enable_telemetry No \
     && platformio settings set check_platformio_interval 1000000
 
-COPY script/platformio_install_deps.py platformio.ini ./
-RUN ./platformio_install_deps.py platformio.ini --libraries --platforms --tools
-
 WORKDIR /workspaces

.github/ISSUE_TEMPLATE/bug_report.yml

@@ -9,6 +9,29 @@ body:
         If you have a feature request or enhancement, please [request them here instead][fr].
 
         [fr]: https://github.com/orgs/esphome/discussions
+  - type: markdown
+    attributes:
+      value: |
+        ## Use of AI in bug reports
+
+        AI tools are good at carrying out well-defined tasks, but they are not good at troubleshooting.
+        Please do NOT paste an AI-generated wall of text into the issue template - if the AI hasn't solved
+        your problem, its wild guesses are not likely to help.
+
+        Please DO include your own words and observations, compile/boot logs, and
+        especially a minimal reproducible example of your YAML configuration that demonstrates the problem.
+
+        It is however quite acceptable to use AI to translate your *own* report,
+        if you aren't a competent English speaker.
+
+        If you really think it will be useful to include an AI's analysis, preferably wrap it in a `<details>` block which will be collapsed by default.
+
+        If you are using AI to help solve a problem, rather than asking it to speculate about what the problem is,
+        it can be more useful to ask it to create a step-by-step troubleshooting procedure.
+        AI is also useful for generating boilerplate code, such as a minimal reproducible example of your YAML
+        configuration that demonstrates the problem.
+
+        Used properly, AI can be a useful tool to help you solve your problem, but don't let it get in the way.
   - type: textarea
     validations:
       required: true

.github/actions/build-image/action.yaml

@@ -42,7 +42,7 @@ runs:
 
     - name: Build and push to ghcr by digest
       id: build-ghcr
-      uses: docker/build-push-action@53b7df96c91f9c12dcc8a07bcb9ccacbed38856a # v7.3.0
+      uses: docker/build-push-action@c3c9e263c25d99ce0380d002d59b67737d91b0dc # v7.4.0
       env:
         DOCKER_BUILD_SUMMARY: false
         DOCKER_BUILD_RECORD_UPLOAD: false
@@ -67,7 +67,7 @@ runs:
 
     - name: Build and push to dockerhub by digest
       id: build-dockerhub
-      uses: docker/build-push-action@53b7df96c91f9c12dcc8a07bcb9ccacbed38856a # v7.3.0
+      uses: docker/build-push-action@c3c9e263c25d99ce0380d002d59b67737d91b0dc # v7.4.0
       env:
         DOCKER_BUILD_SUMMARY: false
         DOCKER_BUILD_RECORD_UPLOAD: false

.github/actions/cache-arduino8266/action.yml

@@ -0,0 +1,39 @@
+name: Cache Arduino ESP8266
+description: >
+  Resolve the pinned Arduino core and xtensa toolchain versions and cache the
+  native ESP8266 install (~110 MB framework + toolchain; no ccache store, the
+  seed job saves before any compile runs). Exports
+  ESPHOME_ARDUINO8266_PREFIX to the job so every later step installs into
+  the cached path; the Python venv must already be restored. Mirrors
+  cache-esp-idf: only dev-branch pushes write the shared cache, everything
+  else restores.
+runs:
+  using: composite
+  steps:
+    - name: Resolve the native toolchain cache key
+      # Versions are pinned in code, not a hashable file; resolve them so a
+      # bump changes the cache key. Assignment form so errexit catches a
+      # resolver failure.
+      id: version
+      shell: bash
+      run: |
+        # One owner for the install prefix: exported here and referenced by
+        # the cache steps below via env, so the caller's install and the
+        # cached path cannot diverge.
+        echo "ESPHOME_ARDUINO8266_PREFIX=$HOME/.esphome-arduino8266" >> "$GITHUB_ENV"
+        . venv/bin/activate
+        key=$(python -c 'from esphome.components.esp8266 import RECOMMENDED_ARDUINO_FRAMEWORK_VERSION as f; from esphome.arduino8266.framework import FRAMEWORK_RELEASES, TOOLCHAIN_VERSION as t; print(f"{FRAMEWORK_RELEASES[f].tag}-{t}")')
+        [ -n "$key" ] || exit 1
+        echo "key=$key" >> "$GITHUB_OUTPUT"
+    - name: Cache the native toolchain (write on dev)
+      if: github.ref == 'refs/heads/dev'
+      uses: actions/cache@55cc8345863c7cc4c66a329aec7e433d2d1c52a9 # v6.1.0
+      with:
+        path: ${{ env.ESPHOME_ARDUINO8266_PREFIX }}
+        key: ${{ runner.os }}-esp8266-native-${{ steps.version.outputs.key }}
+    - name: Restore the native toolchain (off dev)
+      if: github.ref != 'refs/heads/dev'
+      uses: actions/cache/restore@55cc8345863c7cc4c66a329aec7e433d2d1c52a9 # v6.1.0
+      with:
+        path: ${{ env.ESPHOME_ARDUINO8266_PREFIX }}
+        key: ${{ runner.os }}-esp8266-native-${{ steps.version.outputs.key }}

.github/actions/restore-python/action.yml

@@ -32,7 +32,7 @@ runs:
       # detects the activated venv via ``VIRTUAL_ENV`` so the venv layout
       # downstream jobs rely on is preserved.
       if: steps.cache-venv.outputs.cache-hit != 'true'
-      uses: astral-sh/setup-uv@20cfd1bf945f4377ade1205e4dbc17946fc9a30d # v10.0.1
+      uses: astral-sh/setup-uv@c18668ad3cf93ea998bef934396af7bb5c839dc7 # v10.2.0
       with:
         enable-cache: true
         # Pull request saves land in per-PR scopes nothing else can

.github/scripts/auto-label-pr/index.js

@@ -53,7 +53,7 @@ module.exports = async ({ github, context }) => {
   });
   const currentLabels = currentLabelsData.map(label => label.name);
   const managedLabels = currentLabels.filter(label =>
-    label.startsWith('component: ') || MANAGED_LABELS.includes(label)
+    label.startsWith('component: ') || label.startsWith('platform: ') || MANAGED_LABELS.includes(label)
   );
 
   // Check for mega-PR early - if present, skip most automatic labeling

.github/scripts/auto-label-pr/reviews.js

@@ -42,8 +42,8 @@ function generateReviewMessages(finalLabels, originalLabelCount, deprecatedInfo,
     let message = `${TOO_BIG_MARKER}\n### 📦 Pull Request Size\n\n`;
 
     message +=
-      `Hey @${prAuthor}, thanks for the contribution! Just a heads up, ` +
-      `this PR is on the large side `;
+      `Hey @${prAuthor}, thanks for the contribution. This PR is on the ` +
+      `large side `;
 
     if (tooManyLabels && tooManyChanges) {
       message +=
@@ -57,16 +57,31 @@ function generateReviewMessages(finalLabels, originalLabelCount, deprecatedInfo,
     }
 
     message += `, which makes it harder for maintainers to review.\n\n`;
+    let condition;
+    if (tooManyLabels && tooManyChanges) {
+      condition =
+        `is at most ${TOO_BIG_THRESHOLD} lines (excluding tests) and touches ` +
+        `at most ${MAX_LABELS} components/areas`;
+    } else if (tooManyLabels) {
+      condition = `touches at most ${MAX_LABELS} components/areas`;
+    } else {
+      condition = `is at most ${TOO_BIG_THRESHOLD} lines (excluding tests)`;
+    }
     message +=
-      `Smaller, focused PRs tend to be reviewed much faster since they ` +
-      `fit into the short gaps between other maintainer work; large ones ` +
-      `often have to wait for a rare long uninterrupted block of time. ` +
-      `If you can break this up into smaller pieces that can be reviewed ` +
-      `independently, it will almost certainly land faster overall.\n\n`;
+      `**This PR will not be reviewed until it ${condition}.** We know ` +
+      `that is not what you want to hear after this much work, and we want ` +
+      `to be upfront so you do not wait for a review that is not coming: ` +
+      `please split the change. The label is removed automatically once ` +
+      `the PR ${condition}.\n\n`;
     message +=
-      `Before putting more time in, it's also worth popping into ` +
-      `\`#devs\` on [Discord](https://esphome.io/chat) so we can help ` +
-      `you scope things and flag anything already in flight.\n\n`;
+      `There is no exception you can request. Rarely, a maintainer agrees in ` +
+      `\`#devs\` on [Discord](https://esphome.io/chat) to review a change that ` +
+      `genuinely cannot be split as one piece and adds the \`mega-pr\` label, ` +
+      `but that is rare enough that you should plan on splitting rather than ` +
+      `wait for it. Smaller, focused PRs are picked up much faster because ` +
+      `they fit into the short gaps between other maintainer work. For a new ` +
+      `component, the first PR brings the hub and a single platform (for ` +
+      `example \`sensor\`); each further platform is its own PR on top of it.\n\n`;
     message +=
       `For more details (including how to split the work up), see: ` +
       `https://developers.esphome.io/contributing/submitting-your-work/` +
@@ -214,6 +229,7 @@ async function handleMaintainerAccessComment(github, context, maintainerAccess)
 }
 
 module.exports = {
+  generateReviewMessages,
   handleReviews,
   handleMaintainerAccessComment
 };

.github/scripts/auto-label-pr/tests/reviews.test.js

@@ -0,0 +1,38 @@
+const { describe, it } = require('node:test');
+const assert = require('node:assert/strict');
+const { generateReviewMessages } = require('../reviews');
+
+const MAX_LABELS = 15;
+const TOO_BIG_THRESHOLD = 1000;
+
+function tooBigMessage(labelCount, additions) {
+  const files = [{ filename: 'esphome/components/a/a.cpp', additions, deletions: 0 }];
+  const messages = generateReviewMessages(['too-big'], labelCount, null, files, additions, 0, 'someone', MAX_LABELS, TOO_BIG_THRESHOLD);
+  assert.equal(messages.length, 1);
+  return messages[0];
+}
+
+describe('too-big review message', () => {
+  it('names the line cap when the PR is too long', () => {
+    const message = tooBigMessage(3, 1500);
+    assert.match(message, /will not be reviewed until it is at most 1000 lines \(excluding tests\)\./);
+    assert.match(message, /removed automatically once the PR is at most 1000 lines/);
+    assert.doesNotMatch(message, /components\/areas\./);
+  });
+
+  it('names the area cap when the PR touches too many areas but is short', () => {
+    const message = tooBigMessage(20, 200);
+    assert.match(message, /will not be reviewed until it touches at most 15 components\/areas\./);
+    assert.doesNotMatch(message, /until it is at most 1000 lines/);
+  });
+
+  it('names both caps when both are exceeded', () => {
+    const message = tooBigMessage(20, 1500);
+    assert.match(message, /is at most 1000 lines \(excluding tests\) and touches at most 15 components\/areas\./);
+  });
+
+  it('tells new-component authors to bring one platform first', () => {
+    assert.match(tooBigMessage(3, 1500), /hub and a single platform/);
+    assert.match(tooBigMessage(3, 1500), /mega-pr/);
+  });
+});

.github/skills

@@ -0,0 +1 @@
+../.agents/skills

.github/workflows/ci-api-proto.yml

@@ -29,7 +29,7 @@ jobs:
       - name: Set up uv
         # ``--system`` (below) installs into the setup-python interpreter;
         # no venv is created or restored by this workflow.
-        uses: astral-sh/setup-uv@20cfd1bf945f4377ade1205e4dbc17946fc9a30d # v10.0.1
+        uses: astral-sh/setup-uv@c18668ad3cf93ea998bef934396af7bb5c839dc7 # v10.2.0
         with:
           enable-cache: true
           # Pull-request-only workflow: a save could never be shared and

.github/workflows/ci-docker.yml

@@ -12,15 +12,14 @@ on:
       - ".github/workflows/ci-docker.yml"
       - "requirements*.txt"
       - "pyproject.toml"
-      - "platformio.ini"
       - "esphome/idf_component.yml"
-      - "script/platformio_install_deps.py"
       # Core, build pipeline, toolchain, and target-platform changes can change
       # how a toolchain is set up or built, so re-run the per-toolchain compile
       # smoke test when they change.
       - "esphome/core/**"
       - "esphome/writer.py"
       - "esphome/build_gen/**"
+      - "esphome/build_helpers/**"
       - "esphome/espidf/**"
       - "esphome/platformio/**"
       - "esphome/components/bk72xx/**"
@@ -67,7 +66,7 @@ jobs:
         with:
           python-version: "3.12"
       - name: Set up Docker Buildx
-        uses: docker/setup-buildx-action@37fe631027851001ddb9b187196cc803df7f5f0e # v4.3.0
+        uses: docker/setup-buildx-action@594f3bf4285d9ea8dc53c9a0c9c4092420091003 # v4.4.0
 
       - name: Determine tag and whether to push
         id: tag
@@ -159,7 +158,7 @@ jobs:
         with:
           python-version: "3.12"
       - name: Set up Docker Buildx
-        uses: docker/setup-buildx-action@37fe631027851001ddb9b187196cc803df7f5f0e # v4.3.0
+        uses: docker/setup-buildx-action@594f3bf4285d9ea8dc53c9a0c9c4092420091003 # v4.4.0
 
       - name: Log in to the GitHub container registry
         uses: docker/login-action@dbcb813823bdd20940b903addbd779551569679f # v4.6.0
@@ -197,6 +196,7 @@ jobs:
         # the default.
         id:
           - esp8266-arduino
+          - esp8266-arduino-native
           - esp32-arduino-platformio
           - esp32-arduino-esp-idf
           - esp32-idf-platformio
@@ -219,5 +219,6 @@ jobs:
         run: |
           docker run --rm \
             -v "${{ github.workspace }}/docker/test_configs:/config" \
+            -e ESPHOME_LDGEN_STRICT=1 \
             "ghcr.io/esphome/esphome-amd64:${{ needs.check-docker.outputs.tag }}" \
             compile "${{ matrix.id }}.yaml"

.github/workflows/ci.yml

@@ -49,7 +49,7 @@ jobs:
         # detects the activated venv via ``VIRTUAL_ENV`` so downstream jobs
         # that ``. venv/bin/activate`` see an identical layout.
         if: steps.cache-venv.outputs.cache-hit != 'true'
-        uses: astral-sh/setup-uv@20cfd1bf945f4377ade1205e4dbc17946fc9a30d # v10.0.1
+        uses: astral-sh/setup-uv@c18668ad3cf93ea998bef934396af7bb5c839dc7 # v10.2.0
         with:
           enable-cache: true
           # Pull request saves land in per-PR scopes nothing else can
@@ -102,6 +102,8 @@ jobs:
       device-builder: ${{ steps.determine.outputs.device-builder }}
       esp32-platformio: ${{ steps.determine.outputs.esp32-platformio }}
       esp32-platformio-components: ${{ steps.determine.outputs.esp32-platformio-components }}
+      esp8266-native: ${{ steps.determine.outputs.esp8266-native }}
+      esp8266-native-components: ${{ steps.determine.outputs.esp8266-native-components }}
       changed-components: ${{ steps.determine.outputs.changed-components }}
       changed-components-with-tests: ${{ steps.determine.outputs.changed-components-with-tests }}
       directly-changed-components-with-tests: ${{ steps.determine.outputs.directly-changed-components-with-tests }}
@@ -165,6 +167,8 @@ jobs:
           echo "device-builder=$(echo "$output" | jq -r '.device_builder')" >> $GITHUB_OUTPUT
           echo "esp32-platformio=$(echo "$output" | jq -r '.esp32_platformio')" >> $GITHUB_OUTPUT
           echo "esp32-platformio-components=$(echo "$output" | jq -r '.esp32_platformio_components')" >> $GITHUB_OUTPUT
+          echo "esp8266-native=$(echo "$output" | jq -r '.esp8266_native')" >> $GITHUB_OUTPUT
+          echo "esp8266-native-components=$(echo "$output" | jq -r '.esp8266_native_components')" >> $GITHUB_OUTPUT
           echo "changed-components=$(echo "$output" | jq -c '.changed_components')" >> $GITHUB_OUTPUT
           echo "changed-components-with-tests=$(echo "$output" | jq -c '.changed_components_with_tests')" >> $GITHUB_OUTPUT
           echo "directly-changed-components-with-tests=$(echo "$output" | jq -c '.directly_changed_components_with_tests')" >> $GITHUB_OUTPUT
@@ -183,6 +187,33 @@ jobs:
           path: .temp/components_graph.json
           key: components-graph-${{ hashFiles('esphome/components/**/*.py') }}
 
+  seed-esp8266-native-cache:
+    name: Seed the esp8266 native toolchain cache
+    runs-on: ubuntu-24.04
+    needs:
+      - common
+    # PR-branch cache saves are invisible to other PRs, so dev pushes seed
+    # the shared entry the component matrix, the memory impact jobs and
+    # test-esp8266-native restore. Only dev: the composite action saves
+    # nowhere else, so a beta/release push would download the toolchain and
+    # discard it.
+    if: github.event_name == 'push' && github.ref == 'refs/heads/dev'
+    timeout-minutes: 15
+    steps:
+      - name: Check out code from GitHub
+        uses: actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1 # v7.0.1
+      - name: Restore Python
+        uses: ./.github/actions/restore-python
+        with:
+          python-version: ${{ env.DEFAULT_PYTHON }}
+          cache-key: ${{ needs.common.outputs.cache-key }}
+      - name: Cache the native toolchain
+        uses: ./.github/actions/cache-arduino8266
+      - name: Install the native toolchain
+        run: |
+          . venv/bin/activate
+          python -c "from esphome.arduino8266.framework import check_and_install; from esphome.components.esp8266 import RECOMMENDED_ARDUINO_FRAMEWORK_VERSION; check_and_install(RECOMMENDED_ARDUINO_FRAMEWORK_VERSION)"
+
   ci-custom:
     name: Run script/ci-custom
     runs-on: ubuntu-24.04
@@ -255,7 +286,7 @@ jobs:
           fi
           echo "version=$version" >> "$GITHUB_OUTPUT"
       - name: Run prek
-        uses: j178/prek-action@4e14d07f9231acabce116ccfca13b13dd9755ece # v3.0.0
+        uses: j178/prek-action@1382ad72dea72f9d53832f2bcb65b3e87a947204 # v3.0.1
         with:
           prek-version: ${{ steps.prek.outputs.version }}
           # This job only runs on pull requests, so nothing ever populates
@@ -268,7 +299,7 @@ jobs:
       # Pushes any fixes the hooks made back to the pull request. This step
       # must keep its default name: the GitHub App that performs the push
       # locates the workflow run by that name.
-      - uses: pre-commit-ci/lite-action@5d6cc0eb514c891a40562a58a8e71576c5c7fb43 # v1.1.0
+      - uses: pre-commit-ci/lite-action@062bca0919bc9d6e66755cc05074b70c77e111fc # v1.2.0
         if: always()
         with:
           msg: apply automatic formatting fixes
@@ -321,7 +352,7 @@ jobs:
           . venv/bin/activate
           pytest -vv --cov-report=xml --tb=native --durations=30 -n auto tests --ignore=tests/integration/
       - name: Upload coverage to Codecov
-        uses: codecov/codecov-action@fb8b3582c8e4def4969c97caa2f19720cb33a72f # v7.0.0
+        uses: codecov/codecov-action@303a32d7a59b442fa8d48b6a1cc6825c09c847a5 # v7.1.1
       - name: Save Python virtual environment cache
         if: github.ref == 'refs/heads/dev'
         uses: actions/cache/save@55cc8345863c7cc4c66a329aec7e433d2d1c52a9 # v6.1.0
@@ -350,7 +381,7 @@ jobs:
       - name: Check out code from GitHub
         uses: actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1 # v7.0.1
       - name: Report empty upload to Codecov
-        uses: codecov/codecov-action@fb8b3582c8e4def4969c97caa2f19720cb33a72f # v7.0.0
+        uses: codecov/codecov-action@303a32d7a59b442fa8d48b6a1cc6825c09c847a5 # v7.1.1
         with:
           run_command: empty-upload
           force: true
@@ -369,14 +400,9 @@ jobs:
       matrix:
         bucket: ${{ fromJson(needs.determine-jobs.outputs.integration-test-buckets) }}
     env:
-      # What the cache steps persist; libdeps is excluded (keyed per xdist
-      # worker and env, it never crosses runs).
-      INTEGRATION_PIO_CACHE_PATH: |
-        ~/.esphome-integration-tests/platformio/platforms
-        ~/.esphome-integration-tests/platformio/packages
-        ~/.esphome-integration-tests/platformio/appstate.json
-        ~/.esphome-integration-tests/platformio/.cache
-        ~/.esphome-integration-tests/platformio/.esphome.pio.stamp.json
+      # Registry libraries (noise-c, libsodium, ArduinoJson, lvgl) the host builds
+      # download, shared per xdist worker by tests/integration/conftest.py
+      INTEGRATION_LIBRARY_CACHE_PATH: ~/.esphome-integration-tests/pio_components
     steps:
       - name: Check out code from GitHub
         uses: actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1 # v7.0.1
@@ -396,14 +422,13 @@ jobs:
         uses: actions/setup-python@5fda3b95a4ea91299a34e894583c3862153e4b97 # v7.0.0
         with:
           python-version: "3.13"
-      - name: Restore integration PlatformIO cache
-        # Native platform + toolchain installed by shared_platformio_cache in
-        # tests/integration/conftest.py; a miss self-heals, so no restore-keys.
-        id: pio-cache
+      - name: Restore integration library cache
+        # A miss or a changed pin self-heals with a download, so no restore-keys
+        id: library-cache
         uses: actions/cache/restore@55cc8345863c7cc4c66a329aec7e433d2d1c52a9 # v6.1.0
         with:
-          path: ${{ env.INTEGRATION_PIO_CACHE_PATH }}
-          key: integration-pio-v1-${{ runner.os }}-py${{ steps.python.outputs.python-version }}-${{ hashFiles('requirements.txt', 'tests/integration/fixtures/cache_init.yaml', 'esphome/components/host/__init__.py') }}
+          path: ${{ env.INTEGRATION_LIBRARY_CACHE_PATH }}
+          key: integration-libraries-v1-${{ runner.os }}-${{ hashFiles('esphome/components/json/__init__.py', 'esphome/components/noise/__init__.py', 'esphome/components/lvgl/__init__.py', 'esphome/components/improv_base/__init__.py') }}
       - name: Restore Python virtual environment
         id: cache-venv
         uses: actions/cache@55cc8345863c7cc4c66a329aec7e433d2d1c52a9 # v6.1.0
@@ -413,7 +438,7 @@ jobs:
       - name: Set up uv
         # Only needed on cache miss to populate the venv.
         if: steps.cache-venv.outputs.cache-hit != 'true'
-        uses: astral-sh/setup-uv@20cfd1bf945f4377ade1205e4dbc17946fc9a30d # v10.0.1
+        uses: astral-sh/setup-uv@c18668ad3cf93ea998bef934396af7bb5c839dc7 # v10.2.0
         with:
           enable-cache: true
           # Pull request saves land in per-PR scopes nothing else can
@@ -471,16 +496,16 @@ jobs:
           # A full cron period of margin for the weekly refresh
           retention-days: 14
       - name: Print ccache statistics
-        # esphome stores the PlatformIO ccache under the machine-global cache
-        # dir (see _ccache_env() in esphome/platformio/toolchain.py).
-        run: CCACHE_DIR="$HOME/.cache/esphome/platformio-ccache" ccache -s
-      - name: Save integration PlatformIO cache
+        # esphome stores the host build's ccache under the machine-global
+        # cache dir (see get_build_env() in esphome/host/toolchain.py).
+        run: CCACHE_DIR="$HOME/.cache/esphome/host/ccache" ccache -s
+      - name: Save integration library cache
         # Bucket 0 only; the others would race the same immutable key.
-        if: success() && (github.ref == 'refs/heads/dev' || contains(github.event.pull_request.labels.*.name, 'ci-cache-write')) && strategy.job-index == 0 && steps.pio-cache.outputs.cache-hit != 'true'
+        if: success() && (github.ref == 'refs/heads/dev' || contains(github.event.pull_request.labels.*.name, 'ci-cache-write')) && strategy.job-index == 0 && steps.library-cache.outputs.cache-hit != 'true'
         uses: actions/cache/save@55cc8345863c7cc4c66a329aec7e433d2d1c52a9 # v6.1.0
         with:
-          path: ${{ env.INTEGRATION_PIO_CACHE_PATH }}
-          key: ${{ steps.pio-cache.outputs.cache-primary-key }}
+          path: ${{ env.INTEGRATION_LIBRARY_CACHE_PATH }}
+          key: ${{ steps.library-cache.outputs.cache-primary-key }}
 
   import-time:
     name: Check import esphome.__main__ time
@@ -601,7 +626,7 @@ jobs:
             apt-get install -y libc6-dbg
 
       - name: Run CodSpeed benchmarks
-        uses: CodSpeedHQ/action@373d6868929f444bc08d901fd0eb0ad52a8875ea # v5.2.1
+        uses: CodSpeedHQ/action@c4fd08a3a159bd0cc208da1e0edf32b8c47d75e5 # v5.4.0
         with:
           run: |
             . venv/bin/activate
@@ -868,14 +893,20 @@ jobs:
       matrix:
         include:
           - id: clang-tidy
-            name: Run script/clang-tidy for ESP32 IDF 1/3
-            options: --environment esp32-idf-tidy --split-num 3 --split-at 1
+            name: Run script/clang-tidy for ESP32 IDF 1/5
+            options: --environment esp32-idf-tidy --split-num 5 --split-at 1
           - id: clang-tidy
-            name: Run script/clang-tidy for ESP32 IDF 2/3
-            options: --environment esp32-idf-tidy --split-num 3 --split-at 2
+            name: Run script/clang-tidy for ESP32 IDF 2/5
+            options: --environment esp32-idf-tidy --split-num 5 --split-at 2
           - id: clang-tidy
-            name: Run script/clang-tidy for ESP32 IDF 3/3
-            options: --environment esp32-idf-tidy --split-num 3 --split-at 3
+            name: Run script/clang-tidy for ESP32 IDF 3/5
+            options: --environment esp32-idf-tidy --split-num 5 --split-at 3
+          - id: clang-tidy
+            name: Run script/clang-tidy for ESP32 IDF 4/5
+            options: --environment esp32-idf-tidy --split-num 5 --split-at 4
+          - id: clang-tidy
+            name: Run script/clang-tidy for ESP32 IDF 5/5
+            options: --environment esp32-idf-tidy --split-num 5 --split-at 5
 
     steps:
       - name: Check out code from GitHub
@@ -1097,6 +1128,11 @@ jobs:
         uses: ./.github/actions/cache-sdk-nrf
         with:
           restore-only: true
+      - name: Cache the native ESP8266 toolchain
+        # Only batches whose test platforms include esp8266; never saves
+        # here, it reuses the install the dev seed job cached.
+        if: matrix.batch.needs_arduino8266
+        uses: ./.github/actions/cache-arduino8266
       - name: Validate and compile components with intelligent grouping
         run: |
           . venv/bin/activate
@@ -1195,9 +1231,28 @@ jobs:
 
           if [ -n "$compile_csv" ]; then
             # Run compilation with grouping and isolation
-            python3 script/test_build_components.py -e compile -c "$compile_csv" -f --isolate "$directly_changed_csv"
+            # The bootloader has no ESPHome code and these builds never
+            # flash; the check_idf_py batch keeps the full build so the
+            # native sub-build and the equivalence check stay covered.
+            skip_flag="--skip-bootloader"
+            if [[ "${{ matrix.batch.check_idf_py }}" == "true" ]]; then
+              skip_flag=""
+            fi
+            python3 script/test_build_components.py -e compile -c "$compile_csv" -f --isolate "$directly_changed_csv" $skip_flag
+            if [[ "${{ matrix.batch.check_idf_py }}" == "true" ]]; then
+              # The real idf.py must find nothing to configure or build in a
+              # tree built above; catches drift on ESP-IDF bumps.
+              echo "Checking the native ESP-IDF build matches idf.py"
+              python3 script/check_idf_py_equivalence.py
+            fi
           else
             echo "All components in this batch are validate-only -- skipping compile stage."
+            if [[ "${{ matrix.batch.check_idf_py }}" == "true" ]]; then
+              # determine-jobs and this step disagree on what compiles; fail
+              # rather than let the check run nowhere.
+              echo "::error::This batch was picked for the idf.py check but compiled nothing"
+              exit 1
+            fi
           fi
 
       - name: Print ccache statistics
@@ -1237,7 +1292,7 @@ jobs:
 
           # compile validates config first, so a separate config pass is
           # redundant for this smoke test. ESP-IDF framework via PlatformIO:
-          python3 script/test_build_components.py -e compile -t esp32-idf -c "$TEST_COMPONENTS" -f --toolchain platformio
+          python3 script/test_build_components.py -e compile -t esp32-idf -c "$TEST_COMPONENTS" -f --toolchain platformio --fail-on-no-tests
 
           echo ""
           echo "ESP-IDF-via-PlatformIO build passed! Starting Arduino smoke test..."
@@ -1246,6 +1301,125 @@ jobs:
           # Arduino framework via PlatformIO (only components with an esp32-ard test are built):
           python3 script/test_build_components.py -e compile -t esp32-ard -c "$TEST_COMPONENTS" -f --toolchain platformio
 
+  test-esp8266-native:
+    name: Test esp8266 components with the native toolchain
+    runs-on: ubuntu-24.04
+    needs:
+      - common
+      - determine-jobs
+    if: github.event_name == 'pull_request' && needs.determine-jobs.outputs.esp8266-native == 'true'
+    env:
+      # Computed by script/determine-jobs.py (ESP8266_NATIVE_TEST_COMPONENTS)
+      TEST_COMPONENTS: ${{ needs.determine-jobs.outputs.esp8266-native-components }}
+    steps:
+      - name: Check out code from GitHub
+        uses: actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1 # v7.0.1
+
+      - name: Restore Python
+        uses: ./.github/actions/restore-python
+        with:
+          python-version: ${{ env.DEFAULT_PYTHON }}
+          cache-key: ${{ needs.common.outputs.cache-key }}
+
+      - name: Cache the native toolchain
+        uses: ./.github/actions/cache-arduino8266
+
+      - name: Run native toolchain compile test
+        run: |
+          . venv/bin/activate
+
+          echo "Testing components: $TEST_COMPONENTS"
+          echo ""
+
+          # ESP8266 Arduino built directly (no PlatformIO); compile validates
+          # config first, so a separate config pass is redundant.
+          python3 script/test_build_components.py -e compile -t esp8266-ard -c "$TEST_COMPONENTS" -f --toolchain arduino --fail-on-no-tests
+
+  test-esp8266-native-windows:
+    name: Compile esp8266 with the native toolchain on Windows (ccache ${{ matrix.ccache }})
+    runs-on: windows-latest
+    needs:
+      - common
+      - determine-jobs
+    if: github.event_name == 'pull_request' && needs.determine-jobs.outputs.esp8266-native == 'true'
+    # A hung compiler must fail the job, not hold the runner for six hours.
+    timeout-minutes: 40
+    strategy:
+      fail-fast: false
+      matrix:
+        # Both ways the build runs on Windows: ESPHome Desktop bundles
+        # ccache and puts it on PATH, a pip install has none.
+        ccache: ["1", "0"]
+    env:
+      ESPHOME_CCACHE_ENABLE: ${{ matrix.ccache }}
+      # The test script prints characters the cp1252 console cannot encode
+      PYTHONUTF8: "1"
+      # The exact static build ESPHome Desktop ships
+      CCACHE_URL: https://github.com/ccache/ccache/releases/download/v4.14/ccache-4.14-windows-x86_64.zip
+      CCACHE_SHA256: 2568347a697e103ca1b073981c704ad76fb2507d066c38dba038dd73399d968f
+    steps:
+      - name: Check out code from GitHub
+        uses: actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1 # v7.0.1
+
+      - name: Restore Python
+        uses: ./.github/actions/restore-python
+        with:
+          python-version: ${{ env.DEFAULT_PYTHON }}
+          cache-key: ${{ needs.common.outputs.cache-key }}
+
+      - name: Install ccache
+        if: matrix.ccache == '1'
+        shell: pwsh
+        run: |
+          $dir = Join-Path $env:RUNNER_TEMP "ccache"
+          New-Item -ItemType Directory -Force $dir | Out-Null
+          $zip = Join-Path $dir "ccache.zip"
+          Invoke-WebRequest -Uri $env:CCACHE_URL -OutFile $zip
+          $hash = (Get-FileHash $zip -Algorithm SHA256).Hash.ToLower()
+          if ($hash -ne $env:CCACHE_SHA256) {
+            Write-Host "::error::ccache archive sha256 $hash does not match $env:CCACHE_SHA256"
+            exit 1
+          }
+          Expand-Archive -Path $zip -DestinationPath $dir
+          $bin = Get-ChildItem -Path $dir -Recurse -Filter ccache.exe | Select-Object -First 1
+          Add-Content -Path $env:GITHUB_PATH -Value $bin.DirectoryName
+          & $bin.FullName --version
+
+      - name: Compile
+        shell: pwsh
+        # One component test: web_server brings wifi, api and the async web
+        # server libraries, the shape of a typical ESP8266 device
+        run: |
+          . .\venv\Scripts\Activate.ps1
+          $arguments = "script/test_build_components.py", "-e", "compile", "-t", "esp8266-ard", "-c", "web_server", "--base-only", "--toolchain", "arduino", "--fail-on-no-tests"
+          $p = Start-Process -FilePath python -ArgumentList $arguments -NoNewWindow -PassThru
+          # Reading the handle keeps ExitCode available after the process exits
+          $null = $p.Handle
+          if (-not $p.WaitForExit(25 * 60 * 1000)) {
+            Write-Host "::error::The compile did not finish in 25 minutes; processes still running:"
+            Get-CimInstance Win32_Process |
+              Where-Object { $_.CommandLine -match 'xtensa|ccache|ninja|esphome' } |
+              Select-Object ProcessId, ParentProcessId, CreationDate, CommandLine |
+              Format-List | Out-String -Width 4096 | Write-Host
+            Get-Process |
+              Where-Object { $_.Name -match 'cc1|ccache|ninja|xtensa|python' } |
+              Select-Object Id, Name, CPU, WorkingSet64 |
+              Format-Table | Out-String | Write-Host
+            taskkill /PID $p.Id /T /F
+            exit 1
+          }
+          exit $p.ExitCode
+
+      - name: Show ccache statistics
+        # Also after a failed or killed compile: the counters say whether
+        # ccache was ever reached
+        if: always() && matrix.ccache == '1'
+        shell: pwsh
+        # The build keeps its store under the esphome tools cache
+        run: |
+          $env:CCACHE_DIR = Join-Path $env:LOCALAPPDATA "esphome\Cache\arduino8266\ccache"
+          ccache -s
+
   device-builder:
     name: Test downstream esphome/device-builder
     runs-on: ubuntu-24.04
@@ -1274,7 +1448,7 @@ jobs:
         # install step (order-of-magnitude faster on cold boots,
         # with its own wheel cache). actions/setup-python still
         # provides the interpreter.
-        uses: astral-sh/setup-uv@20cfd1bf945f4377ade1205e4dbc17946fc9a30d # v10.0.1
+        uses: astral-sh/setup-uv@c18668ad3cf93ea998bef934396af7bb5c839dc7 # v10.2.0
         with:
           enable-cache: true
           # Pull request saves land in per-PR scopes nothing else can
@@ -1333,31 +1507,27 @@ jobs:
             echo "::warning::ci_memory_impact_extract.py not found on target branch, skipping memory impact analysis"
           fi
 
-      # Check if test files exist on the target branch for the requested
-      # components and platform. When a PR adds new test files for a platform,
-      # the target branch won't have them yet, so skip instead of failing.
-      # This check must be done here (not in determine-jobs.py) because
-      # determine-jobs runs on the PR branch and cannot see what the target
-      # branch has.
+      # The build below uses --base-only, so skip unless the target branch has
+      # test.<platform>.yaml for every requested component (test-*.yaml
+      # variants alone compile nothing, and a partial set would compare a
+      # smaller target build against the PR build). Checked here, not in
+      # determine-jobs.py, because determine-jobs runs on the PR branch and
+      # cannot see the target branch.
       - name: Check for test files on target branch
         id: check-tests
         if: steps.check-script.outputs.skip != 'true'
         run: |
           components='${{ toJSON(fromJSON(needs.determine-jobs.outputs.memory_impact).components) }}'
           platform="${{ fromJSON(needs.determine-jobs.outputs.memory_impact).platform }}"
-          found=false
+          missing=""
           for component in $(echo "$components" | jq -r '.[]'); do
-            # Check for test files matching the platform (test.platform.yaml or test-*.platform.yaml)
-            for f in tests/components/${component}/test*.${platform}.yaml; do
-              if [ -f "$f" ]; then
-                found=true
-                break 2
-              fi
-            done
+            if [ ! -f "tests/components/${component}/test.${platform}.yaml" ]; then
+              missing="${missing} ${component}"
+            fi
           done
-          if [ "$found" = false ]; then
+          if [ -n "$missing" ]; then
             echo "skip=true" >> $GITHUB_OUTPUT
-            echo "::warning::No test files found on target branch for platform ${platform}, skipping memory impact analysis"
+            echo "::warning::No test.${platform}.yaml on target branch for${missing}, skipping memory impact analysis"
           else
             echo "skip=false" >> $GITHUB_OUTPUT
           fi
@@ -1415,12 +1585,9 @@ jobs:
           python-version: ${{ env.DEFAULT_PYTHON }}
           cache-key: ${{ needs.common.outputs.cache-key }}
 
-      - name: Cache platformio
-        if: steps.check-script.outputs.skip != 'true' && steps.check-tests.outputs.skip != 'true' && steps.cache-memory-analysis.outputs.cache-hit != 'true'
-        uses: actions/cache/restore@55cc8345863c7cc4c66a329aec7e433d2d1c52a9 # v6.1.0
-        with:
-          path: ~/.platformio
-          key: platformio-memory-${{ fromJSON(needs.determine-jobs.outputs.memory_impact).platform }}-${{ hashFiles('platformio.ini') }}
+      - name: Cache the native ESP8266 toolchain
+        if: steps.check-script.outputs.skip != 'true' && steps.check-tests.outputs.skip != 'true' && steps.cache-memory-analysis.outputs.cache-hit != 'true' && fromJSON(needs.determine-jobs.outputs.memory_impact).needs_arduino8266
+        uses: ./.github/actions/cache-arduino8266
 
       - name: Build, compile, and analyze memory
         if: steps.check-script.outputs.skip != 'true' && steps.check-tests.outputs.skip != 'true' && steps.cache-memory-analysis.outputs.cache-hit != 'true'
@@ -1505,11 +1672,9 @@ jobs:
         with:
           python-version: ${{ env.DEFAULT_PYTHON }}
           cache-key: ${{ needs.common.outputs.cache-key }}
-      - name: Cache platformio
-        uses: actions/cache/restore@55cc8345863c7cc4c66a329aec7e433d2d1c52a9 # v6.1.0
-        with:
-          path: ~/.platformio
-          key: platformio-memory-${{ fromJSON(needs.determine-jobs.outputs.memory_impact).platform }}-${{ hashFiles('platformio.ini') }}
+      - name: Cache the native ESP8266 toolchain
+        if: fromJSON(needs.determine-jobs.outputs.memory_impact).needs_arduino8266
+        uses: ./.github/actions/cache-arduino8266
       - name: Build, compile, and analyze memory
         id: extract
         run: |
@@ -1608,6 +1773,7 @@ jobs:
     needs:
       - common
       - seed-apt-cache
+      - seed-esp8266-native-cache
       - determine-jobs
       - ci-custom
       - pylint
@@ -1623,6 +1789,8 @@ jobs:
       - clang-tidy-esp32-variants
       - test-build-components-split
       - test-esp32-platformio
+      - test-esp8266-native
+      - test-esp8266-native-windows
       - device-builder
       - memory-impact-target-branch
       - memory-impact-pr-branch

.github/workflows/codeql.yml

@@ -56,7 +56,7 @@ jobs:
 
       # Initializes the CodeQL tools for scanning.
       - name: Initialize CodeQL
-        uses: github/codeql-action/init@cdf488f595d80d6e07e03d4674febd5ab45fa938 # v4.37.9
+        uses: github/codeql-action/init@2892aa5e19bbd11bc0cff5427e3b750a04d9e3c2 # v4.38.2
         with:
           languages: ${{ matrix.language }}
           build-mode: ${{ matrix.build-mode }}
@@ -84,6 +84,6 @@ jobs:
           exit 1
 
       - name: Perform CodeQL Analysis
-        uses: github/codeql-action/analyze@cdf488f595d80d6e07e03d4674febd5ab45fa938 # v4.37.9
+        uses: github/codeql-action/analyze@2892aa5e19bbd11bc0cff5427e3b750a04d9e3c2 # v4.38.2
         with:
           category: "/language:${{matrix.language}}"

.github/workflows/external-component-bot.yml

@@ -125,14 +125,20 @@ jobs:
             }
 
             async function getEsphomeAndComponentChanges(github, owner, repo, prNumber) {
-                const changedFiles = await github.rest.pulls.listFiles({
-                    owner: owner,
-                    repo: repo,
-                    pull_number: prNumber,
-                });
+                const changedFiles = await github.paginate(
+                    github.rest.pulls.listFiles,
+                    {
+                        owner: owner,
+                        repo: repo,
+                        pull_number: prNumber,
+                        per_page: 100,
+                    }
+                );
 
-                const esphomeChanges = changedFiles.data
-                    .filter(file => file.filename !== "esphome/core/defines.h" && file.filename.startsWith('esphome/'))
+                // Files used only for development and CI, which do not affect use as an external component
+                const ignoredFiles = ["esphome/core/defines.h", "esphome/idf_component.yml"];
+                const esphomeChanges = changedFiles
+                    .filter(file => !ignoredFiles.includes(file.filename) && file.filename.startsWith('esphome/'))
                     .map(file => {
                         const match = file.filename.match(/esphome\/([^/]+)/);
                         return match ? match[1] : null;
@@ -144,7 +150,7 @@ jobs:
                 }
 
                 const uniqueEsphomeChanges = [...new Set(esphomeChanges)];
-                const componentChanges = changedFiles.data
+                const componentChanges = changedFiles
                     .filter(file => file.filename.startsWith('esphome/components/'))
                     .map(file => {
                         const match = file.filename.match(/esphome\/components\/([^/]+)\//);

.github/workflows/lock.yml

@@ -14,4 +14,4 @@ jobs:
     permissions:
       issues: write         # issues.lock on closed issues
       pull-requests: write  # issues.lock on closed pull requests
-    uses: esphome/workflows/.github/workflows/lock.yml@0fdd5e311b7e744069166696072a1a9cbc5fbeb6  # 2026.8.1
+    uses: esphome/workflows/.github/workflows/lock.yml@cc3e76de337dc59bc1cba8da58d963cd23b873f1  # 2026.9.0

.github/workflows/release.yml

@@ -123,7 +123,7 @@ jobs:
           python-version: "3.12"
 
       - name: Set up Docker Buildx
-        uses: docker/setup-buildx-action@37fe631027851001ddb9b187196cc803df7f5f0e # v4.3.0
+        uses: docker/setup-buildx-action@594f3bf4285d9ea8dc53c9a0c9c4092420091003 # v4.4.0
 
       - name: Log in to docker hub
         uses: docker/login-action@dbcb813823bdd20940b903addbd779551569679f # v4.6.0
@@ -202,7 +202,7 @@ jobs:
           merge-multiple: true
 
       - name: Set up Docker Buildx
-        uses: docker/setup-buildx-action@37fe631027851001ddb9b187196cc803df7f5f0e # v4.3.0
+        uses: docker/setup-buildx-action@594f3bf4285d9ea8dc53c9a0c9c4092420091003 # v4.4.0
 
       - name: Log in to docker hub
         if: matrix.registry == 'dockerhub'

.github/workflows/stale.yml

@@ -16,16 +16,18 @@ jobs:
     # No GITHUB_TOKEN permissions: the reusable workflow mints an ESPHome
     # GitHub App token so the labels, comments and closures come from
     # esphome[bot] instead of github-actions[bot].
-    uses: esphome/workflows/.github/workflows/stale.yml@a1c1485ab46ef41a84a6a9d8abd7fa4b7628fd70  # main
+    uses: esphome/workflows/.github/workflows/stale.yml@4fbce13026c393d6eb6b247e394dcd9ef1af76c2  # main
     secrets:
       ESPHOME_GITHUB_APP_PRIVATE_KEY: ${{ secrets.ESPHOME_GITHUB_APP_PRIVATE_KEY }}
     with:
       # Live only on dev: a workflow_dispatch from any other branch is a dry run
       dry-run: ${{ github.ref != 'refs/heads/dev' }}
       days-before-stale: 90
+      days-before-stale-conflicting: 30
       days-before-close: 7
       stale-label: stale
       exempt-label: not-stale
+      conflict-label: needs-rebase
       ignored-users: esphbot,codecov-commenter
       stale-pr-message: >
         There hasn't been any activity on this pull request recently. This
@@ -36,6 +38,18 @@ jobs:
         to keep it open. Also, please merge the latest dev branch into your
         branch to ensure that it's up to date with the latest changes.
 
+        Thank you for your contribution!
+      conflict-pr-message: >
+        This pull request has merge conflicts. Please merge the latest dev
+        branch into your branch to resolve them.
+      stale-conflicting-pr-message: >
+        The merge conflicts on this pull request have not been resolved for
+        30 days, so it has been marked as stale and will be closed if they
+        are not resolved within 7 days.
+
+        Please merge the latest dev branch into your branch. Once the pull
+        request is mergeable again the stale label is removed.
+
         Thank you for your contribution!
       stale-issue-message: >
         There hasn't been any activity on this issue recently. Due to the

.github/workflows/status-check-labels.yml

@@ -7,10 +7,6 @@ on:
 permissions:
   pull-requests: read  # issues.listLabelsOnIssue to detect blocking labels (needs-docs, needs-developer-docs, merge-after-release, chained-pr)
 
-concurrency:
-  group: ${{ github.workflow }}-${{ github.event.pull_request.number }}
-  cancel-in-progress: true
-
 jobs:
   check:
     name: Check blocking labels

.github/workflows/sync-dependency-versions.yml

@@ -15,6 +15,8 @@ on:
   # base branch checkout, so fork code never runs with the token.
   pull_request_target:
     types: [opened, synchronize, reopened]
+    branches:
+      - dev
     paths:
       - requirements_dev.txt
       - requirements_test.txt

.github/workflows/sync-device-classes.yml

@@ -47,7 +47,7 @@ jobs:
         # setup-python interpreter so subsequent ``prek`` /
         # ``script/run-in-env.py`` steps find the deps without a
         # ``uv run`` prefix.
-        uses: astral-sh/setup-uv@20cfd1bf945f4377ade1205e4dbc17946fc9a30d # v10.0.1
+        uses: astral-sh/setup-uv@c18668ad3cf93ea998bef934396af7bb5c839dc7 # v10.2.0
         with:
           enable-cache: true
           # Pin uv version so the action does not have to fetch the

.pre-commit-config.yaml

@@ -10,7 +10,7 @@ ci:
 repos:
   - repo: https://github.com/astral-sh/ruff-pre-commit
     # Ruff version.
-    rev: v0.16.6
+    rev: v0.16.10
     hooks:
       # Run the linter.
       - id: ruff
@@ -18,7 +18,7 @@ repos:
       # Run the formatter.
       - id: ruff-format
   - repo: https://github.com/PyCQA/flake8
-    rev: 7.3.0
+    rev: 7.4.1
     hooks:
       - id: flake8
         additional_dependencies:

AGENTS.md

@@ -67,6 +67,7 @@ file does, and it is the authority when they disagree. The most useful starting
         - Function-local constants: `lower_snake_case`
         - Protected/private fields: `lower_snake_case_with_trailing_underscore_`
         - Favor descriptive names over abbreviations
+        - Device components are named brand plus model (`siemens_ira211`), without a protocol-family prefix
         - Enumerator names: prefix every value of an `enum class` with the enum name converted to
           `UPPER_SNAKE_CASE` (e.g. `UARTFlushResult::UART_FLUSH_RESULT_SUCCESS`). Never use bare
           names like `SUCCESS`, `FAILURE`, `OK`, or `FAIL`: platform SDK headers define macros with
@@ -143,6 +144,9 @@ file does, and it is the authority when they disagree. The most useful starting
 
 *   **C++ Preprocessor Directives:**
     *   **Avoid `#define` for constants:** Using `#define` for constants is discouraged and should be replaced with `const` variables or enums.
+    *   **A constant with a compile-time initializer is `static constexpr`,** not `static const`, at namespace scope
+        and inside functions; `static const` is only for a value computed at run time. A function-local
+        `static constexpr` table is a static constant to clang-tidy, so it is `UPPER_SNAKE_CASE` too.
     *   **Use `#define` only for:**
         - Conditional compilation (`#ifdef`, `#ifndef`)
         - Compile-time sizes calculated during Python code generation (e.g., configuring `std::array` or `StaticVector` dimensions via `cg.add_define()`)
@@ -177,6 +181,15 @@ file does, and it is the authority when they disagree. The most useful starting
         Pick the primitive by cadence: under 250 ms use a gated `loop()`; 500 ms and above use
         `set_interval`. Full reasoning, including why `set_interval` costs more below 500 ms:
         https://developers.esphome.io/architecture/components/advanced/#quick-rule-of-thumb
+    *   **Repeating work:** use `set_interval`, never a `set_timeout` that re-arms itself; every re-arm
+        rebuilds the callback and re-queues a scheduler item, and the period drifts.
+    *   **Scheduler ids:** name a timer only when it must be cancelled or replaced, and use a
+        `static constexpr uint32_t` id, never a string. Ids are per component instance and cannot clash with other
+        components, so number them from 0 and keep all of a component's ids together in one place.
+        ```cpp
+        static constexpr uint32_t READ_TIMEOUT_ID = 0;
+        this->set_timeout(READ_TIMEOUT_ID, 50, [this]() { this->read_(); });
+        ```
     *   **Don't override a default with the same value:** if a base class method already returns what you
         want, do not override it. `Component::get_setup_priority()` returns `setup_priority::DATA`, so a
         component that wants `DATA` should simply leave it alone.
@@ -184,6 +197,36 @@ file does, and it is the authority when they disagree. The most useful starting
         // Bad - this is exactly what the base class already does
         float get_setup_priority() const override { return setup_priority::DATA; }
         ```
+    *   **Constant data lives in RAM on ESP8266** unless marked `PROGMEM`: `const` tables, class-scope
+        constant arrays, and in-class `std::array` initializers (a `.rodata` template plus a copy). Mark the
+        table `PROGMEM` and read it with `progmem_memcpy()` or `progmem_read_byte()` (`esphome/core/hal.h`);
+        for a table of structs copy the whole entry out and return the copy, never a pointer into flash.
+        On other platforms they are a plain `memcpy` and dereference, so no `#ifdef` is needed. Declare log tags with
+        `ESPHOME_LOG_TAG(TAG, "name")`, pass status text as `LOG_STR`, e.g.
+        `this->mark_failed(LOG_STR(ESP_LOG_MSG_COMM_FAIL))`, and give an action whose payload is a byte array
+        a `TEMPLATABLE_BYTES` field so a static code stays in flash.
+    *   **Fixed sets of names** (status words, model names): a `PROGMEM_STRING_TABLE` (`esphome/core/progmem.h`),
+        logged with `get_log_str` and `LOG_STR_ARG`, copied out with `ESPHOME_strncpy_P` before `publish_state`.
+        A `dump_config` helper that picks a name returns `const LogString *` from `LOG_STR(...)`.
+    *   **CRCs and bit tricks** come from `esphome/core/helpers.h` (`crc8`, `crc16`, `reverse_bits`, `encode_uint16`);
+        pick the helper's polynomial and bit-order arguments instead of writing a CRC loop.
+    *   **An unset float is `NAN`**, not `optional<float>`; the optional costs 4 to 8 bytes of padding per member.
+    *   **`PollingComponent` first runs `update()`** after a random share of the interval; call `this->update()` at the
+        end of `setup()` when the entities need a state at boot.
+    *   **A multi-step command sequence** leaves the step pending when the answer is malformed, so the existing
+        resend and give-up logic handles it; never advance on an answer that was only logged.
+    *   **Entities that each watch a register field:** one class taking register, shift, mask and match from a Python
+        tuple table, in typed `StaticVector`s sized by `cg.slot_counter`; not a template per bit, not a virtual
+        listener base (a second vptr per entity).
+    *   **A global feature define** says the code is compiled in, not that this instance uses it: keep the
+        per-instance check (`if (this->aux_device_ == ...)`) inside the `#ifdef`.
+    *   **Model-specific code** sits behind a define the Python side emits only for configs that use that model
+        (`cg.add_define("USE_SELEC_METER_EM4M")`), so the other models' builds leave it out; the runtime model check
+        stays inside the `#ifdef`.
+    *   **Failures after `setup()`:** `mark_failed()` is for `setup()` only. A runtime read or write failure
+        sets `status_set_warning()` and the next success clears it; writes skip work when `is_failed()`.
+    *   **Fixed platform facts compile to nothing:** gate on `#ifdef USE_ESP8266` and friends, not on a
+        runtime check of something the build already knows.
     *   **Logging string literals:** wrap literals passed as `%s` arguments in `LOG_STR_LITERAL()` so they
         can be stored in flash rather than RAM.
         ```cpp
@@ -230,6 +273,10 @@ file does, and it is the authority when they disagree. The most useful starting
         ```
 
     *   **Component Metadata:**
+        - `DOMAIN`: Required in every component's `__init__.py`, set to the component's own name as a plain
+          string (e.g. `DOMAIN = "my_component"`). Other code refers to the component through it, e.g.
+          `from esphome.components.my_component import DOMAIN`. Keep it in alphabetical order with the other
+          metadata constants. CI (`lint_component_domain`) fails without it.
         - `DEPENDENCIES`: List of required components
         - `AUTO_LOAD`: Components to automatically load
         - `CONFLICTS_WITH`: Incompatible components
@@ -243,6 +290,7 @@ file does, and it is the authority when they disagree. The most useful starting
         import esphome.config_validation as cv
         from esphome.const import CONF_KEY, CONF_ID
 
+        DOMAIN = "my_component"
         CONF_PARAM = "param"  # A constant that does not yet exist in esphome/const.py
 
         my_component_ns = cg.esphome_ns.namespace("my_component")
@@ -322,6 +370,25 @@ file does, and it is the authority when they disagree. The most useful starting
               var = await switch.new_switch(config)
           ```
 
+        - **Optional child entities of a hub:** bind the config once with `sensor.sub_sensors(config)` (or
+          `sub_binary_sensors`, `sub_text_sensors`, `sub_buttons`, `sub_switches`, `sub_numbers`,
+          `sub_selects` in their domains), adding `parent=hub` for entities that derive from `Parented<T>`,
+          then make one call per key, even when there is only one. A call creates the entity only when its key
+          is configured, passes it to the setter and returns it (or `None`); extra arguments such as
+          `min_value` or `options` go on the call. Always name the setter explicitly on the object that owns
+          it, never with `getattr` and an f-string, and keep that variable short (`var` for the component
+          itself, `hub` for one fetched with `cg.get_variable`) so the calls fit on one line. Loops whose
+          setter also takes an index, such as `set_gate_threshold(x, n)`, stay as they are.
+          ```python
+          async def to_code(config):
+              var = cg.new_Pvariable(config[CONF_ID])
+              sensors = sensor.sub_sensors(config)
+              await sensors(CONF_TEMPERATURE, var.set_temperature_sensor)
+              await sensors(CONF_HUMIDITY, var.set_humidity_sensor)
+              buttons = button.sub_buttons(config, parent=var)
+              await buttons(CONF_RESTART, var.set_restart_button)
+          ```
+
 *   **Automations (Triggers, Actions, Conditions):**
 
     Automations have three building blocks: **Triggers** (fire when something happens), **Actions** (do something), and **Conditions** (check if something is true).
@@ -365,6 +432,25 @@ file does, and it is the authority when they disagree. The most useful starting
                 )
         ```
 
+        When the callback's parameters differ from the automation's arguments, or the trigger should only
+        fire for some values, pass `params`, `forward` and `when`; the helper then generates a capture-less
+        lambda (stored inline, nothing allocated). Name the parent with `automation.parent_ref(var)` and build
+        `forward` from `MockObj` calls, not f-strings of C++:
+        ```python
+        parent = automation.parent_ref(var)
+        index = cg.RawExpression("index")
+        await automation.build_callback_automation(
+            var,
+            "add_on_state_callback",
+            [(cg.StringRef, "x"), (cg.size_t, "i")],
+            conf,
+            params=[(cg.size_t, "index")],
+            forward=[cg.StringRef(parent.option_at(index)), index],
+        )
+        ```
+        `build_parent_callback_automation`, `build_trigger_callback`, `build_callback_automations` and
+        `build_trigger_automations` cover the other shapes; see their docstrings in `esphome/automation.py`.
+
         **C++ -- no trigger class needed.** The callback registration method must be templatized to accept both `std::function` and lightweight forwarder structs (which avoid heap allocation):
         ```cpp
         class MyComponent : public Component {
@@ -431,7 +517,31 @@ file does, and it is the authority when they disagree. The most useful starting
           MyComponent *parent_;
         };
         ```
-        Register with `@automation.register_action("my_component.do_something", MyAction, schema, synchronous=True)`. Use `synchronous=True` for actions that run to completion inside `play()` without deferring. Use `synchronous=False` if the action may suspend/defer execution (e.g. `delay`, `wait_until`, `script.wait`) or store trigger arguments for later use.
+        Register it without writing a builder:
+        ```python
+        automation.register_simple_action(
+            "my_component.do_something", MyAction, schema, synchronous=True
+        )
+        ```
+        The constructor receives the object named by `config[CONF_ID]`. Use `register_bare_action` for a
+        no-argument constructor, `register_parented_action` for a class deriving from `Parented<T>`, and
+        the `@automation.register_action(...)` decorator only when the builder must also set fields.
+
+        Use `synchronous=True` for actions that run to completion inside `play()` without deferring. Use `synchronous=False` if the action may suspend/defer execution (e.g. `delay`, `wait_until`, `script.wait`) or store trigger arguments for later use.
+
+        **Actions that only forward templatable values to their parent need no C++ class.** Register them
+        with `register_apply_action`; do not write a `TEMPLATABLE_VALUE` class or a builder for this shape.
+        ```python
+        automation.register_apply_action(
+            "my_component.set_gains",
+            schema,
+            automation.ApplyField(CONF_KP, "set_kp", cg.float_),
+            automation.ApplyField(CONF_KI, "set_ki", cg.float_),
+        )
+        ```
+        The `ApplyField`, `ApplyCall` and `register_apply_action` docstrings in `esphome/automation.py` cover
+        the rest; `cover.control` and `cover.template.publish` are in-tree examples. `TEMPLATABLE_VALUE` with
+        `cg.templatable` stays for actions whose `play()` has real logic beyond forwarding values.
 
     *   **Conditions:**
         ```cpp
@@ -443,11 +553,29 @@ file does, and it is the authority when they disagree. The most useful starting
           MyComponent *parent_;
         };
         ```
-        Register with `@automation.register_condition("my_component.is_active", MyCondition, schema)`.
+        Register with `automation.register_simple_condition("my_component.is_active", MyCondition, schema)`;
+        `register_bare_condition`, `register_parented_condition` and the decorator follow the action rules.
+
+        **Conditions that only test their parent need no C++ class either.** Register them with
+        `register_apply_condition`; the expression is applied to the parent, and an `ApplyCall` compares
+        against config values.
+        ```python
+        automation.register_apply_condition("my_component.is_active", schema, "is_active()")
+        automation.register_apply_condition(
+            "my_component.state_is",
+            schema,
+            automation.ApplyCall("state == {}", ((CONF_STATE, cg.bool_),)),
+        )
+        ```
+        `cover.is_open`, `rtttl.is_playing` and `component.is_idle` are in-tree examples.
 
 *   **Type Hints:** Type-hint all function signatures, including test functions and config validators (e.g. `def validate_x(config: ConfigType) -> ConfigType:`, `def test_x() -> None:`). Import `ConfigType` from `esphome.types`.
+    An intentionally empty body (an unused `register_dumper` builder, for example) gets a one-line docstring, not `pass`.
 
 *   **Configuration Validation:**
+    *   **One schema per model:** when a component's options depend on a `model` key, use `cv.typed_schema({...},
+        key=CONF_MODEL, default_type=...)` with each model's own keys and defaults, not one schema plus a validator
+        that rejects keys and fills defaults afterwards.
     *   **Reuse existing validators:** Before writing a custom validator, check for an existing one in `config_validation.py` and compose it in `cv.All(...)` rather than duplicating logic across components. For example, rename a config key with `cv.rename_key(CONF_OLD, CONF_NEW, removed_in="2026.6.0")`, and reject mutually-exclusive keys with `cv.has_at_most_one_key(...)` / `cv.has_exactly_one_key(...)`. See how `api` composes `cv.has_exactly_one_key` + `cv.rename_key`.
     *   **Common Validators:** `cv.int_`, `cv.float_`, `cv.string`, `cv.boolean`, `cv.int_range(min=0, max=100)`, `cv.positive_int`, `cv.percentage`.
     *   **Complex Validation:** `cv.All(cv.string, cv.Length(min=1, max=50))`, `cv.Any(cv.int_, cv.string)`.
@@ -481,6 +609,18 @@ file does, and it is the authority when they disagree. The most useful starting
 *   **Testing:**
     *   **Python:** Run unit tests with `pytest`.
     *   **C++:** Use `clang-tidy` for static analysis.
+    *   **C++ unit tests:** gtests in `tests/components/<component>/*.cpp`, run with
+        `script/cpp_unit_test.py <component>`; see `tests/components/README.md`. Set the loop clock in a test
+        with `LoopBlockingGuard{nullptr, nullptr, now_ms}`; the runner resets it before every test.
+    *   **Integration tests:** `tests/integration/` runs real firmware on the host. Serial components prove
+        timeouts and recovery with the `uart_mock` fixture (`tests/integration/test_uart_mock_*.py`). Its
+        `responses` match on content every time they see it; to answer a repeated request differently, use an
+        `on_tx` lambda with a static counter and `inject_to_rx_buffer` (see `uart_mock_ld2412_requery_after_giveup.yaml`).
+    *   **Keep build-test variants few:** one `test.<platform>.yaml` per platform the code actually differs
+        on. A variant per board for platform-identical code only adds CI time.
+    *   **A test that needs a shared component in a particular state** (`esp32_ble` security settings, an option
+        that PSRAM rejects) cannot be grouped: the merged block wins and the whole batch fails. Add the component to
+        `ISOLATED_COMPONENTS` in `script/analyze_component_buses.py`, or move the option to a `test-<variant>` file.
     *   **Component Tests:** YAML-based compilation tests are located in `tests/`. The structure is as follows:
         ```
         tests/
@@ -527,6 +667,7 @@ file does, and it is the authority when they disagree. The most useful starting
             name: My Sensor
         ```
         Components that define buses directly are flagged as "NEEDS MIGRATION" and cannot be grouped, increasing CI build time.
+        `!extend` and `!remove` in a `test.*.yaml` also make the component ungroupable; add a standalone entity with its own id instead.
 
     *   **Testing All Components Together:** To verify that all components can be tested together without ID conflicts or configuration issues, use:
         ```bash
@@ -699,7 +840,9 @@ file does, and it is the authority when they disagree. The most useful starting
 
         6. **Avoid `std::deque`:** It allocates in 512-byte blocks regardless of element size, guaranteeing at least 512 bytes of RAM usage immediately. This is a major source of crashes on memory-constrained devices.
 
-        7. **Detection:** Look for these patterns in compiler output:
+        7. **Never use `new (std::nothrow)`:** On ESP-IDF exceptions are disabled, so a failed nothrow allocation aborts instead of returning `nullptr`. Use `RAMAllocator` from `esphome/core/helpers.h`; CI rejects `std::nothrow`.
+
+        8. **Detection:** Look for these patterns in compiler output:
            - Large code sections with STL symbols (vector, map, set)
            - `alloc`, `realloc`, `dealloc` in symbol names
            - `_M_realloc_insert`, `_M_default_append` (vector reallocation)

CODEOWNERS

@@ -39,9 +39,12 @@ esphome/components/am43/sensor/* @buxtronix
 esphome/components/analog_threshold/* @ianchi
 esphome/components/animation/* @syndlex
 esphome/components/anova/* @buxtronix
+esphome/components/apc1/* @ademuri
 esphome/components/apds9306/* @aodrenah
+esphome/components/apds9930/* @iyesin
 esphome/components/api/* @esphome/core
 esphome/components/aqi/* @freekode @jasstrong @ximex
+esphome/components/aranet/* @patrickod
 esphome/components/as5600/* @ammmze
 esphome/components/as5600/sensor/* @ammmze
 esphome/components/as7341/* @mrgnr
@@ -65,6 +68,7 @@ esphome/components/bedjet/climate/* @jhansche
 esphome/components/bedjet/fan/* @jhansche
 esphome/components/bedjet/sensor/* @javawizard @jhansche
 esphome/components/beken_spi_led_strip/* @Mat931
+esphome/components/bh1745/* @latonita
 esphome/components/bh1750/* @OttoWinter
 esphome/components/bh1900nux/* @B48D81EFCC
 esphome/components/binary_sensor/* @esphome/core
@@ -100,6 +104,10 @@ esphome/components/bmp581_i2c/* @danielkent-net @kahrendt
 esphome/components/bmp581_spi/* @danielkent-net @kahrendt
 esphome/components/bp1658cj/* @Cossid
 esphome/components/bp5758d/* @Cossid
+esphome/components/bq25186/* @Rapsssito
+esphome/components/bq27220/* @rggammon
+esphome/components/bq76972/* @limpkin
+esphome/components/bridge/* @kbx81
 esphome/components/bthome_mithermometer/* @nagyrobi
 esphome/components/button/* @esphome/core
 esphome/components/bytebuffer/* @clydebarrow
@@ -111,6 +119,8 @@ esphome/components/captive_portal/* @esphome/core
 esphome/components/cc1101/* @gabest11 @lygris
 esphome/components/ccs811/* @habbie
 esphome/components/cd74hc4067/* @asoehlke
+esphome/components/cdc_acm_uart/* @kbx81
+esphome/components/cdc_acm_uart/bridge/* @kbx81
 esphome/components/ch422g/* @clydebarrow @jesterret
 esphome/components/ch423/* @dwmw2
 esphome/components/chsc6x/* @kkosik20
@@ -122,6 +132,7 @@ esphome/components/combination/* @Cat-Ion @kahrendt
 esphome/components/const/* @esphome/core
 esphome/components/coolix/* @glmnet
 esphome/components/copy/* @OttoWinter
+esphome/components/counter/* @clydebarrow
 esphome/components/cover/* @esphome/core
 esphome/components/cs5460a/* @balrog-kun
 esphome/components/cse7761/* @berfenger
@@ -152,8 +163,10 @@ esphome/components/ds1307/* @badbadc0ffee
 esphome/components/ds1603l/* @JakeLC15
 esphome/components/ds2484/* @mrk-its
 esphome/components/ds248x/* @tomwellnitz
+esphome/components/ds3231/* @linkedupbits
 esphome/components/dsmr/* @glmnet @PolarGoose
 esphome/components/duty_time/* @dudanov
+esphome/components/ecocomfort2/* @gledian
 esphome/components/ee895/* @Stock-M
 esphome/components/ektf2232/touchscreen/* @jesserockz
 esphome/components/emc2101/* @ellull
@@ -179,24 +192,29 @@ esphome/components/esp32_camera_web_server/* @ayufan
 esphome/components/esp32_can/* @Sympatron
 esphome/components/esp32_hosted/* @swoboda1337
 esphome/components/esp32_hosted/update/* @swoboda1337
-esphome/components/esp32_improv/* @jesserockz
+esphome/components/esp32_power_management/* @rwrozelle
 esphome/components/esp32_rmt/* @jesserockz
 esphome/components/esp32_rmt_led_strip/* @jesserockz
 esphome/components/esp8266/* @esphome/core
 esphome/components/esp_ldo/* @clydebarrow
+esphome/components/espectre/* @francescopace
+esphome/components/esphome/ota/* @esphome/core
 esphome/components/espnow/* @jesserockz
 esphome/components/espnow/packet_transport/* @EasilyBoredEngineer
 esphome/components/ethernet_info/* @gtjadsonsantos
 esphome/components/event/* @nohat
+esphome/components/exponential_moving_average/* @clydebarrow
 esphome/components/exposure_notifications/* @OttoWinter
 esphome/components/ezo/* @ssieb
 esphome/components/ezo_pmp/* @carlos-sarmiento
+esphome/components/ezsp_proxy_tap/* @kbx81 @puddly
 esphome/components/factory_reset/* @anatoly-savchenkov
 esphome/components/fastled_base/* @OttoWinter
 esphome/components/feedback/* @ianchi
 esphome/components/file/* @esphome/core
 esphome/components/fingerprint_grow/* @alexborro @loongyh @OnFreund
 esphome/components/font/* @clydebarrow @esphome/core
+esphome/components/friedrich/* @rwrozelle
 esphome/components/fs3000/* @kahrendt
 esphome/components/ft5x06/* @clydebarrow
 esphome/components/ft63x6/* @gpambrozio
@@ -242,8 +260,11 @@ esphome/components/hmac_md5/* @dwmw2
 esphome/components/hmac_sha256/* @dwmw2
 esphome/components/hoermann_hcp/* @zweckj
 esphome/components/homeassistant/* @esphome/core @OttoWinter
+esphome/components/homeassistant/button/* @jesserockz
 esphome/components/homeassistant/number/* @landonr
+esphome/components/homeassistant/select/* @jesserockz
 esphome/components/homeassistant/switch/* @Links2004
+esphome/components/homeassistant/text/* @jesserockz
 esphome/components/honeywell_hih_i2c/* @Benichou34
 esphome/components/honeywellabp/* @RubyBailey
 esphome/components/honeywellabp2_i2c/* @jpfaff
@@ -255,6 +276,7 @@ esphome/components/http_request/ota/* @oarcher
 esphome/components/http_request/update/* @jesserockz
 esphome/components/htu31d/* @betterengineering
 esphome/components/hub75/* @stuartparmenter
+esphome/components/husb238/* @latonita
 esphome/components/hydreon_rgxx/* @functionpointer
 esphome/components/hyt271/* @Philippe12
 esphome/components/i2c/* @esphome/core
@@ -262,9 +284,12 @@ esphome/components/i2c_device/* @gabest11
 esphome/components/i2s_audio/* @jesserockz
 esphome/components/i2s_audio/microphone/* @jesserockz
 esphome/components/i2s_audio/speaker/* @jesserockz @kahrendt
+esphome/components/i2s_clockless_led_strip/* @j9brown
 esphome/components/iaqcore/* @yozik04
+esphome/components/icnt86/* @danepowell
 esphome/components/ili9xxx/* @clydebarrow @nielsnl68
 esphome/components/improv_base/* @esphome/core
+esphome/components/improv_ble/* @jesserockz
 esphome/components/improv_serial/* @esphome/core
 esphome/components/ina226/* @latonita @Sergio303
 esphome/components/ina260/* @mreditor97
@@ -277,6 +302,7 @@ esphome/components/inkplate/* @jesserockz @JosipKuci
 esphome/components/integration/* @OttoWinter
 esphome/components/internal_temperature/* @Mat931
 esphome/components/interval/* @esphome/core
+esphome/components/ir_rf_base/* @bdraco @kbx81
 esphome/components/ir_rf_proxy/* @kbx81
 esphome/components/it8951/* @koosoli @limengdu @Passific
 esphome/components/jsn_sr04t/* @Mafus1
@@ -288,9 +314,11 @@ esphome/components/kuntze/* @ssieb
 esphome/components/lc709203f/* @ilikecake
 esphome/components/lcd_menu/* @numo68
 esphome/components/ld2410/* @regevbr @sebcaps
+esphome/components/ld2410s/* @NovakIrs
 esphome/components/ld2412/* @Rihan9
 esphome/components/ld2420/* @descipher
 esphome/components/ld2450/* @hareeshmu
+esphome/components/ld2460/* @jesusvallejo
 esphome/components/ld24xx/* @kbx81
 esphome/components/ld6002b/* @hepter
 esphome/components/ledc/* @OttoWinter
@@ -299,6 +327,10 @@ esphome/components/libretiny_pwm/* @kuba2k2
 esphome/components/light/* @esphome/core
 esphome/components/lightwaverf/* @max246
 esphome/components/lilygo_t5_47/touchscreen/* @jesserockz
+esphome/components/lis2dw12/* @latonita
+esphome/components/lis2dw12_i2c/* @latonita
+esphome/components/lis2dw12_spi/* @latonita
+esphome/components/lis3dh/* @zebble
 esphome/components/lm75b/* @beormund
 esphome/components/ln882h_ble/* @Bl00d-B0b
 esphome/components/ln882h_ble_tracker/* @Bl00d-B0b
@@ -306,6 +338,7 @@ esphome/components/ln882x/* @lamauny
 esphome/components/lock/* @esphome/core
 esphome/components/logger/* @esphome/core
 esphome/components/logger/select/* @clydebarrow
+esphome/components/lp5562/* @DennisGaida
 esphome/components/lps22/* @nagisa
 esphome/components/lsm6ds/* @clydebarrow
 esphome/components/ltr390/* @latonita @sjtrny
@@ -316,7 +349,9 @@ esphome/components/m5stack_8angle/* @rnauber
 esphome/components/mapping/* @clydebarrow
 esphome/components/matrix_keypad/* @ssieb
 esphome/components/max17043/* @blacknell
+esphome/components/max17048/* @B48D81EFCC
 esphome/components/max31865/* @DAVe3283
+esphome/components/max31888/* @JiriPrchal
 esphome/components/max44009/* @berfenger
 esphome/components/max6956/* @looping40
 esphome/components/max7219digit/* @rspaargaren
@@ -335,7 +370,6 @@ esphome/components/mcp4461/* @p1ngb4ck
 esphome/components/mcp4728/* @berfenger
 esphome/components/mcp47a1/* @jesserockz
 esphome/components/mcp9600/* @mreditor97
-esphome/components/mcp9808/* @k7hpn
 esphome/components/md5/* @esphome/core
 esphome/components/mdns/* @esphome/core
 esphome/components/media_player/* @jesserockz
@@ -349,6 +383,7 @@ esphome/components/midea_ir/* @dudanov
 esphome/components/mipi_dsi/* @clydebarrow
 esphome/components/mipi_rgb/* @clydebarrow
 esphome/components/mipi_spi/* @clydebarrow
+esphome/components/mistral_ir/* @lerhum
 esphome/components/mitsubishi/* @RubyBailey
 esphome/components/mitsubishi_cn105/* @crnjan
 esphome/components/mixer/speaker/* @kahrendt
@@ -387,6 +422,8 @@ esphome/components/noblex/* @AGalfra
 esphome/components/noise/* @esphome/core
 esphome/components/npi19/* @bakerkj
 esphome/components/nrf52/* @tomaszduda23
+esphome/components/ns2009/* @tchilov
+esphome/components/ns2009/touchscreen/* @tchilov
 esphome/components/number/* @esphome/core
 esphome/components/one_wire/* @ssieb
 esphome/components/online_image/* @clydebarrow @guillempages
@@ -401,6 +438,7 @@ esphome/components/pca9554/* @bdraco @clydebarrow @hwstar
 esphome/components/pcf85063/* @brogon
 esphome/components/pcf8563/* @KoenBreeman
 esphome/components/pcm5122/* @remcom
+esphome/components/pd_pioneer_ir/* @martinsah
 esphome/components/pi4ioe5v6408/* @jesserockz
 esphome/components/pid/* @OttoWinter
 esphome/components/pipsolar/* @andreashergert1984
@@ -418,6 +456,8 @@ esphome/components/pn7150_i2c/* @jesserockz @kbx81
 esphome/components/pn7160/* @jesserockz @kbx81
 esphome/components/pn7160_i2c/* @jesserockz @kbx81
 esphome/components/pn7160_spi/* @jesserockz @kbx81
+esphome/components/pn71xx/* @jesserockz @kbx81
+esphome/components/power_management/* @rwrozelle
 esphome/components/power_supply/* @esphome/core
 esphome/components/preferences/* @esphome/core
 esphome/components/provisioning/* @esphome/core
@@ -425,6 +465,7 @@ esphome/components/psram/* @esphome/core
 esphome/components/pulse_meter/* @cstaahl @stevebaxter @TrentHouliston
 esphome/components/pvvx_mithermometer/* @pasiz
 esphome/components/pylontech/* @functionpointer
+esphome/components/pzem6l24/* @nuttytree
 esphome/components/qmi8658/* @clydebarrow
 esphome/components/qmp6988/* @andrewpc
 esphome/components/qr_code/* @wjtje
@@ -437,6 +478,8 @@ esphome/components/rc522/* @glmnet
 esphome/components/rc522_i2c/* @glmnet
 esphome/components/rc522_spi/* @glmnet
 esphome/components/rd03d/* @jasstrong
+esphome/components/remko_ar715/* @naKruul
+esphome/components/renogy_inverter_ble/* @emilioaray-dev
 esphome/components/resampler/speaker/* @kahrendt
 esphome/components/restart/* @esphome/core
 esphome/components/rf_bridge/* @jesserockz
@@ -453,8 +496,10 @@ esphome/components/rtl87xx/* @kuba2k2
 esphome/components/rtttl/* @glmnet @ximex
 esphome/components/runtime_image/* @clydebarrow @guillempages @kahrendt
 esphome/components/runtime_stats/* @bdraco
+esphome/components/rx8025t/* @remcom
 esphome/components/rx8130/* @beormund
 esphome/components/safe_mode/* @jsuanet @kbx81 @paulmonigatti
+esphome/components/samsung/* @jorofi
 esphome/components/scd4x/* @martgras @sjtrny
 esphome/components/script/* @esphome/core
 esphome/components/sdl/* @bdm310 @clydebarrow
@@ -474,6 +519,7 @@ esphome/components/sendspin/image/* @kahrendt
 esphome/components/sendspin/media_player/* @kahrendt
 esphome/components/sendspin/media_source/* @kahrendt
 esphome/components/sendspin/sensor/* @kahrendt
+esphome/components/sendspin/switch/* @kahrendt
 esphome/components/sendspin/text_sensor/* @kahrendt
 esphome/components/sensirion_common/* @martgras
 esphome/components/sensor/* @esphome/core
@@ -482,11 +528,14 @@ esphome/components/sfa30/* @ghsensdev
 esphome/components/sfa40/* @NoQuarrel
 esphome/components/sgp40/* @SenexCrenshaw
 esphome/components/sgp4x/* @martgras @SenexCrenshaw
+esphome/components/sh1122_base/* @pl4nkton
+esphome/components/sh1122_spi/* @pl4nkton
 esphome/components/sha256/* @esphome/core
 esphome/components/shelly_dimmer/* @edge90 @rnauber
 esphome/components/sht3xd/* @mrtoy-me
 esphome/components/sht4x/* @sjtrny
 esphome/components/shutdown/* @esphome/core @jsuanet
+esphome/components/siemens_ira211/* @fonix232
 esphome/components/sigma_delta_output/* @Cat-Ion
 esphome/components/sim800l/* @glmnet
 esphome/components/sm10bit_base/* @Cossid
@@ -503,6 +552,7 @@ esphome/components/sound_level/* @kahrendt
 esphome/components/spa06_base/* @danielkent-net
 esphome/components/spa06_i2c/* @danielkent-net
 esphome/components/spa06_spi/* @danielkent-net
+esphome/components/spd2010/* @grischard
 esphome/components/speaker/* @jesserockz @kahrendt
 esphome/components/speaker/media_player/* @kahrendt @synesthesiam
 esphome/components/speaker_source/* @kahrendt
@@ -532,6 +582,7 @@ esphome/components/st7735/* @SenexCrenshaw
 esphome/components/st7789v/* @kbx81
 esphome/components/st7920/* @marsjan155
 esphome/components/statsd/* @Links2004
+esphome/components/stcc4/* @j9brown
 esphome/components/stts22h/* @B48D81EFCC
 esphome/components/substitutions/* @esphome/core
 esphome/components/sun/* @OttoWinter
@@ -542,29 +593,38 @@ esphome/components/sx126x/* @swoboda1337
 esphome/components/sx127x/* @swoboda1337
 esphome/components/sy6970/* @linkedupbits
 esphome/components/syslog/* @clydebarrow
+esphome/components/systa_bus/* @Mat931
 esphome/components/t6615/* @tylermenezes
+esphome/components/tas2780/* @remcom
+esphome/components/tas58xx/* @mrtoy-me @remcom
 esphome/components/tc74/* @sethgirvan
+esphome/components/tca8418/* @zebble
 esphome/components/tca9548a/* @andreashergert1984
 esphome/components/tca9555/* @mobrembski
 esphome/components/tcl112/* @glmnet
+esphome/components/tcp_uart/* @Bascht74
 esphome/components/tee501/* @Stock-M
-esphome/components/teleinfo/* @0hax
+esphome/components/teleinfo/* @0hax @FredM67
 esphome/components/tem3200/* @bakerkj
 esphome/components/template/alarm_control_panel/* @grahambrown11 @hwstar
 esphome/components/template/datetime/* @rfdarter
 esphome/components/template/event/* @nohat
 esphome/components/template/fan/* @ssieb
 esphome/components/text/* @mauritskorse
+esphome/components/tfluna/* @candrews
 esphome/components/thermopro_ble/* @sittner
 esphome/components/thermostat/* @kbx81
 esphome/components/time/* @esphome/core
 esphome/components/tinyusb/* @kbx81
+esphome/components/tinyusb_keyboard/* @frederikbolding
 esphome/components/tlc5947/* @rnauber
 esphome/components/tlc5971/* @IJIJI
 esphome/components/tm1621/* @Philippe12
 esphome/components/tm1637/* @glmnet
 esphome/components/tm1638/* @skykingjwc
 esphome/components/tm1651/* @mrtoy-me
+esphome/components/tmc2209/* @remcom
+esphome/components/tmc22xx/* @remcom
 esphome/components/tmp102/* @timsavage
 esphome/components/tmp1075/* @sybrenstuvel
 esphome/components/tmp117/* @Azimath
@@ -586,11 +646,14 @@ esphome/components/uart/* @esphome/core
 esphome/components/uart/button/* @ssieb
 esphome/components/uart/event/* @eoasmxd
 esphome/components/uart/packet_transport/* @clydebarrow
+esphome/components/uart_mux/* @kbx81
+esphome/components/uart_tcp/* @Bascht74
 esphome/components/udp/* @clydebarrow
 esphome/components/ufire_ec/* @pvizeli
 esphome/components/ufire_ise/* @pvizeli
 esphome/components/ufm01/* @ljungqvist
 esphome/components/ultrasonic/* @ssieb @swoboda1337
+esphome/components/unicode/* @esphome/core
 esphome/components/update/* @jesserockz
 esphome/components/uponor_smatrix/* @kroimon
 esphome/components/usb_cdc_acm/* @kbx81
@@ -613,7 +676,7 @@ esphome/components/web_server_idf/* @dentra
 esphome/components/weikai/* @DrCoolZic
 esphome/components/weikai_i2c/* @DrCoolZic
 esphome/components/weikai_spi/* @DrCoolZic
-esphome/components/whirlpool/* @glmnet
+esphome/components/whirlpool/* @arpena @glmnet
 esphome/components/whynter/* @aeonsablaze
 esphome/components/wiegand/* @ssieb
 esphome/components/wireguard/* @droscy @lhoracek @thomas0bernard
@@ -630,9 +693,11 @@ esphome/components/wts01/* @alepee
 esphome/components/x9c/* @EtienneMD
 esphome/components/xdb401/* @RT530
 esphome/components/xgzp68xx/* @gcormier
+esphome/components/xiaomi_body_scale/* @dckiller51
 esphome/components/xiaomi_hhccjcy10/* @fariouche
 esphome/components/xiaomi_lywsd02mmc/* @juanluss31
 esphome/components/xiaomi_lywsd03mmc/* @ahpohl
+esphome/components/xiaomi_mccgq02hl/* @ahpohl @morph027
 esphome/components/xiaomi_mhoc303/* @drug123
 esphome/components/xiaomi_mhoc401/* @vevsvevs
 esphome/components/xiaomi_rtcgq02lm/* @jesserockz

Doxyfile

@@ -48,7 +48,7 @@ PROJECT_NAME           = ESPHome
 # could be handy for archiving the generated documentation or if some version
 # control system is used.
 
-PROJECT_NUMBER         = 2026.10.0-dev
+PROJECT_NUMBER         = 2026.11.0-dev
 
 # Using the PROJECT_BRIEF tag one can provide an optional one line description
 # for a project that appears at the top of each page and should give viewer a

docker/Dockerfile

@@ -22,16 +22,13 @@ RUN \
     -r /requirements.txt
 
 # Install the ESPHome Device Builder dashboard.
-RUN uv pip install --no-cache-dir esphome-device-builder==1.14.7
+RUN uv pip install --no-cache-dir esphome-device-builder==1.23.1
 
 RUN \
     platformio settings set enable_telemetry No \
     && platformio settings set check_platformio_interval 1000000 \
     && mkdir -p /piolibs
 
-COPY script/platformio_install_deps.py platformio.ini /
-RUN /platformio_install_deps.py /platformio.ini --libraries
-
 ARG BUILD_VERSION
 
 LABEL \

docker/docker_entrypoint.sh

@@ -21,10 +21,14 @@ export PLATFORMIO_PLATFORMS_DIR="${pio_cache_base}/platforms"
 export PLATFORMIO_PACKAGES_DIR="${pio_cache_base}/packages"
 export PLATFORMIO_CACHE_DIR="${pio_cache_base}/cache"
 
-# Keep the native toolchain installs on the persistent cache root, not the
-# container's ephemeral user cache dir (re-downloaded on every restart).
+# Keep the native toolchain installs and compiler caches on the persistent
+# cache root, not the container's user cache dir: it is lost on every
+# restart, and not writable when the container runs as a non-root user.
 export ESPHOME_ESP_IDF_PREFIX="$(dirname "${pio_cache_base}")/idf"
 export ESPHOME_SDK_NRF_PREFIX="$(dirname "${pio_cache_base}")/sdk-nrf"
+export ESPHOME_ARDUINO8266_PREFIX="$(dirname "${pio_cache_base}")/arduino8266"
+export ESPHOME_HOST_PREFIX="$(dirname "${pio_cache_base}")/host"
+export ESPHOME_PLATFORMIO_CCACHE_DIR="$(dirname "${pio_cache_base}")/platformio-ccache"
 
 # If /build is mounted, use that as the build path
 # otherwise use path in /config (so that builds aren't lost on container restart)

docker/ha-addon-rootfs/etc/s6-overlay/s6-rc.d/discovery/run

@@ -4,7 +4,6 @@
 # Home Assistant Add-on: ESPHome
 # Sends discovery information to Home Assistant.
 # ==============================================================================
-declare config
 declare port
 
 # We only disable it when disabled explicitly
@@ -19,14 +18,17 @@ port=$(bashio::addon.ingress_port)
 # Wait for the ESPHome Device Builder to become available
 bashio::net.wait_for "${port}" "127.0.0.1" 300
 
-config=$(\
-    bashio::var.json \
-        host "127.0.0.1" \
-        port "^${port}" \
-)
+# Send one discovery message; the config is a JSON string built with bashio::var.json.
+send_discovery() {
+    local service=$1
+    local config=$2
+    if bashio::discovery "${service}" "${config}" > /dev/null; then
+        bashio::log.info "Successfully send ${service} discovery information to Home Assistant."
+    else
+        bashio::log.error "${service} discovery message to Home Assistant failed!"
+    fi
+}
 
-if bashio::discovery "esphome" "${config}" > /dev/null; then
-    bashio::log.info "Successfully send discovery information to Home Assistant."
-else
-    bashio::log.error "Discovery message to Home Assistant failed!"
-fi
+send_discovery "esphome" "$(bashio::var.json host "127.0.0.1" port "^${port}")"
+# The Device Builder MCP server, consumed by Home Assistant's mcp integration.
+send_discovery "mcp" "$(bashio::var.json url "http://127.0.0.1:${port}/api/mcp")"

docker/ha-addon-rootfs/etc/s6-overlay/s6-rc.d/esphome/run

@@ -15,10 +15,14 @@ export PLATFORMIO_PLATFORMS_DIR="${pio_cache_base}/platforms"
 export PLATFORMIO_PACKAGES_DIR="${pio_cache_base}/packages"
 export PLATFORMIO_CACHE_DIR="${pio_cache_base}/cache"
 
-# Keep the native toolchain installs on the persistent /data volume, not the
-# container's ephemeral user cache dir (wiped on every add-on update/restart).
+# Keep the native toolchain installs and compiler caches on the persistent
+# /data volume, not the container's ephemeral user cache dir (wiped on every
+# add-on update/restart).
 export ESPHOME_ESP_IDF_PREFIX=/data/cache/idf
 export ESPHOME_SDK_NRF_PREFIX=/data/cache/sdk-nrf
+export ESPHOME_ARDUINO8266_PREFIX=/data/cache/arduino8266
+export ESPHOME_HOST_PREFIX=/data/cache/host
+export ESPHOME_PLATFORMIO_CCACHE_DIR=/data/cache/platformio-ccache
 
 if bashio::config.true 'leave_front_door_open'; then
     export DISABLE_HA_AUTHENTICATION=true

docker/test_configs/bk72xx-arduino.yaml

@@ -5,3 +5,15 @@ bk72xx:
   board: generic-bk7231n-qfn32-tuya
 
 logger:
+
+wifi:
+  ssid: MySSID
+  password: password1
+  ap:
+
+# mqtt and captive_portal together pull in AsyncTCP and ESPAsyncWebServer;
+# a stray ESP32 AsyncTCP copy on the library search path breaks this build
+captive_portal:
+
+mqtt:
+  broker: 192.168.178.84

docker/test_configs/esp8266-arduino-native.yaml

@@ -0,0 +1,8 @@
+esphome:
+  name: docker-test-esp8266-native
+
+esp8266:
+  board: d1_mini
+  toolchain: arduino
+
+logger:

docker/test_configs/esp8266-arduino.yaml

@@ -3,5 +3,7 @@ esphome:
 
 esp8266:
   board: d1_mini
+  # The PlatformIO path stays covered whatever the default is
+  toolchain: platformio
 
 logger:

esphome/__main__.py

@@ -7,6 +7,7 @@ import importlib
 import logging
 import os
 from pathlib import Path
+import platform
 import re
 import sys
 import time
@@ -16,8 +17,8 @@ from typing import TYPE_CHECKING, Protocol
 # cause them to be loaded before external components are processed, resulting
 # in the built-in version being used instead of the external component one.
 from esphome import const, platform_hooks
+from esphome.build_helpers.native import analysis_backend, native_backend
 from esphome.const import (
-    ALLOWED_NAME_CHARS,
     ARGUMENT_HELP_DEVICE,
     BUNDLE_EXTENSION,
     CONF_API,
@@ -35,7 +36,6 @@ from esphome.const import (
     CONF_LOGGER,
     CONF_MDNS,
     CONF_MQTT,
-    CONF_NAME,
     CONF_NAME_ADD_MAC_SUFFIX,
     CONF_OTA,
     CONF_PASSWORD,
@@ -48,19 +48,24 @@ from esphome.const import (
     CONF_WEB_SERVER,
     CONF_WIFI,
     ENV_NOGITIGNORE,
+    KEY_CORE,
     KEY_ESP32,
+    KEY_TARGET_PLATFORM,
     KEY_VARIANT,
+    PLATFORM_HOST,
     SECRETS_FILES,
+    ErrorFormat,
     Toolchain,
 )
 from esphome.core import CORE, EsphomeError, coroutine
 from esphome.enum import StrEnum
-from esphome.helpers import get_bool_env, indent, is_ip_address
+from esphome.helpers import IS_MACOS, get_bool_env, indent, is_ip_address
 from esphome.log import AnsiFore, color, setup_log
 from esphome.stacktrace import LogLineProcessor
 from esphome.types import ConfigType
 from esphome.upload_targets import PortType, get_port_type
 from esphome.util import (
+    ESPHOME_COMMAND,
     PICOTOOL_PACKAGE,
     FlashImage,
     detect_rp2040_bootsel,
@@ -68,6 +73,7 @@ from esphome.util import (
     get_serial_ports,
     is_picotool_usb_permission_error,
     list_yaml_files,
+    read_secret_line,
     run_external_command,
     run_external_process,
     safe_print,
@@ -84,7 +90,6 @@ if TYPE_CHECKING:
 
 _LOGGER = logging.getLogger(__name__)
 
-ESPHOME_COMMAND = [sys.executable, "-m", "esphome"]
 
 # Maximum buffer size for serial log reading to prevent unbounded memory growth
 SERIAL_BUFFER_MAX_SIZE = 65536
@@ -149,6 +154,7 @@ class ArgsProtocol(Protocol):
     file: str | None
     no_logs: bool
     only_generate: bool
+    skip_bootloader: bool
     show_secrets: bool
     dashboard: bool
     configuration: str
@@ -477,12 +483,9 @@ def has_ota() -> bool:
 
 def has_native_ota() -> bool:
     """Check if native API OTA upload is available (``platform: esphome``)."""
-    if CONF_OTA not in CORE.config:
-        return False
-    return any(
-        ota_item.get(CONF_PLATFORM) == CONF_ESPHOME
-        for ota_item in CORE.config[CONF_OTA]
-    )
+    from esphome.espota2 import esphome_ota_items
+
+    return bool(esphome_ota_items(CORE.config))
 
 
 def has_web_server_ota() -> bool:
@@ -817,7 +820,9 @@ def write_cpp_file() -> int:
         from esphome.build_gen import espidf
 
         espidf.write_project()
-    else:
+    elif not CORE.using_native_toolchain:
+        # Other native builds generate their project at compile time;
+        # never write a platformio.ini for them
         from esphome.build_gen import platformio
 
         platformio.write_project()
@@ -826,6 +831,14 @@ def write_cpp_file() -> int:
 
 
 def compile_program(args: ArgsProtocol, config: ConfigType) -> int:
+    if CORE.skip_bootloader and not (CORE.is_esp32 and CORE.using_toolchain_esp_idf):
+        # Debug only: an orchestrator cannot see YAML toolchain overrides,
+        # so this is its expected no-op, and a full build is safe.
+        _LOGGER.debug(
+            "--skip-bootloader ignored: only supported on ESP32 with the "
+            "esp-idf toolchain"
+        )
+        CORE.skip_bootloader = False
     # Keep this gate here, NOT in config validation: device-builder needs
     # `esphome config` to keep succeeding with placeholders so onboarding can run.
     if CONF_WIFI in config:
@@ -835,7 +848,7 @@ def compile_program(args: ArgsProtocol, config: ConfigType) -> int:
 
     # Keep this here, NOT in codegen: config-hash and --only-generate must keep
     # working on machines that cannot run the toolchain.
-    if CORE.is_esp8266:
+    if CORE.is_esp8266 and CORE.using_toolchain_platformio:
         from esphome.components.esp8266 import check_rosetta
 
         check_rosetta()
@@ -856,23 +869,20 @@ def compile_program(args: ArgsProtocol, config: ConfigType) -> int:
             return rc
 
         # Create factory.bin, ota.bin, and firmware.elf copy
-        toolchain.create_factory_bin()
+        if not toolchain.create_factory_bin():
+            # A build whose factory image could not be produced must not
+            # exit 0; downloads would serve an image from an older build.
+            return 1
         toolchain.create_ota_bin()
         toolchain.create_elf_copy()
-        from esphome.build_helpers.idedata import IDEDATA_BEST_EFFORT_ERRORS
+        from esphome.build_helpers.idedata import warn_if_idedata_missing
 
-        try:
-            if toolchain.get_idedata() is None:
-                _LOGGER.warning("No idedata was generated for this build")
-        except IDEDATA_BEST_EFFORT_ERRORS as err:
-            # The firmware already built; an idedata failure must not fail
-            # a successful build.
-            _LOGGER.warning(
-                "Could not generate idedata: %s (IDE, clang-tidy, and "
-                "memory-analysis data will be unavailable for this build)",
-                err,
-            )
-            _LOGGER.debug("Idedata failure detail", exc_info=True)
+        warn_if_idedata_missing(toolchain.get_idedata)
+    elif CORE.using_native_toolchain:
+        raise EsphomeError(
+            f"Toolchain '{CORE.toolchain.value}' resolved but no platform "
+            "backend claimed the build"
+        )
     else:
         from esphome.platformio import toolchain
 
@@ -975,12 +985,16 @@ def upload_using_esptool(
 
     if file is not None:
         flash_images = [FlashImage(path=file, offset="0x0")]
-    elif CORE.using_toolchain_esp_idf:
-        from esphome.espidf import toolchain
-
-        flash_images = [
-            FlashImage(path=toolchain.get_factory_firmware_path(), offset="0x0")
-        ]
+    elif (native := native_backend()) is not None:
+        # Every native backend supplies its own 0x0 flash image (bootloader
+        # and partitions included where the target needs them)
+        image = native.get_factory_firmware_path()
+        if not image.is_file():
+            hint = getattr(native, "missing_image_hint", lambda: None)()
+            raise EsphomeError(
+                hint or f"{image} does not exist; compile the configuration first"
+            )
+        flash_images = [FlashImage(path=image, offset="0x0")]
     else:
         from esphome.platformio import toolchain
 
@@ -1219,14 +1233,28 @@ def upload_program(
     config: ConfigType, args: ArgsProtocol, devices: list[str]
 ) -> tuple[int, str | None]:
     host = devices[0]
+    port_type = get_port_type(host)
+    ota_key = getattr(args, "ota_key", None)
+    if ota_key and port_type in (PortType.SERIAL, PortType.BOOTSEL):
+        _LOGGER.warning(
+            "The presented OTA key only applies to network uploads; ignored for %s",
+            host,
+        )
+
     platform_upload = platform_hooks.get_platform_hook(
         CORE.target_platform, "upload_program"
     )
     if platform_upload is not None and platform_upload(config, args, host):
+        if ota_key and port_type not in (PortType.SERIAL, PortType.BOOTSEL):
+            # A platform's own network path (nrf52 mcumgr) has no key handshake
+            _LOGGER.warning(
+                "The presented OTA key only applies to the %s OTA platform; "
+                "ignored for %s",
+                CONF_ESPHOME,
+                host,
+            )
         return 0, host
 
-    port_type = get_port_type(host)
-
     # MQTT and MQTTIP are also OTA paths; MQTTIP gets resolved to a real IP later by
     # _resolve_network_devices(). Only SERIAL and BOOTSEL are non-OTA upload paths.
     is_partition_table = getattr(args, "partition_table", False)
@@ -1269,6 +1297,8 @@ def upload_program(
     network_devices = _resolve_network_devices(devices, config, args)
 
     if chosen_platform == CONF_WEB_SERVER:
+        if ota_key:
+            raise EsphomeError(OTA_KEY_PLATFORM_ERROR)
         if is_partition_table or is_bootloader:
             raise EsphomeError(
                 f"{option_string} is only supported with the esphome OTA platform; "
@@ -1282,6 +1312,17 @@ def upload_program(
     return _upload_via_native_api(config, network_devices, args)
 
 
+def _ota_upload_platforms(config: ConfigType) -> dict[str, None]:
+    """The network OTA platforms the config provides, one entry each, in
+    config order; the web_server final-validate hook merges duplicates."""
+    return {
+        ota_platform: None
+        for ota_item in config.get(CONF_OTA, [])
+        if (ota_platform := ota_item.get(CONF_PLATFORM))
+        in (CONF_ESPHOME, CONF_WEB_SERVER)
+    }
+
+
 def _choose_ota_platform(config: ConfigType, requested: str | None) -> str:
     """Pick the OTA platform to use, optionally honoring ``--ota-platform``.
 
@@ -1289,22 +1330,12 @@ def _choose_ota_platform(config: ConfigType, requested: str | None) -> str:
     The native API uses challenge-response auth with MD5/SHA256 hashing of a
     server-issued nonce, so the password is never sent over the wire; the
     ``web_server`` path uses HTTP Basic auth which transmits credentials in
-    cleartext over the LAN. (The native path also supports gzip compression
-    on ESP8266, where flash space is tight; on ESP32/RP2040/LibreTiny the
-    backend reports ``supports_compression() == false`` and the firmware is
-    sent uncompressed regardless of which platform is used.) Falls back to
+    cleartext over the LAN. (The native path also compresses the upload:
+    gzip on ESP8266 and RP2040, which inflate it at reboot, and a deflate
+    stream on ESP32/LibreTiny, which inflate it as it arrives.) Falls back to
     ``web_server`` only when that is the only available platform.
     """
-    # Use a dict (insertion-ordered) instead of a list so error messages and
-    # membership checks see one entry per platform even if the user has
-    # multiple ``ota:`` items of the same platform; the web_server OTA
-    # platform's final-validate hook merges duplicates anyway.
-    available: dict[str, None] = {}
-    for ota_item in config.get(CONF_OTA, []):
-        platform = ota_item.get(CONF_PLATFORM)
-        if platform in (CONF_ESPHOME, CONF_WEB_SERVER):
-            available[platform] = None
-
+    available = _ota_upload_platforms(config)
     if not available:
         raise EsphomeError(
             f"Cannot upload Over the Air as the {CONF_OTA} configuration is not "
@@ -1325,14 +1356,17 @@ def _choose_ota_platform(config: ConfigType, requested: str | None) -> str:
     return CONF_WEB_SERVER
 
 
+def _esphome_ota_conf(config: ConfigType) -> ConfigType | None:
+    """The validated esphome OTA platform block, if configured."""
+    from esphome.espota2 import esphome_ota_items
+
+    return next(iter(esphome_ota_items(config)), None)
+
+
 def _upload_via_native_api(
     config: ConfigType, network_devices: list[str], args: ArgsProtocol
 ) -> tuple[int, str | None]:
-    ota_conf: ConfigType = {}
-    for ota_item in config.get(CONF_OTA, []):
-        if ota_item.get(CONF_PLATFORM) == CONF_ESPHOME:
-            ota_conf = ota_item
-            break
+    ota_conf = _esphome_ota_conf(config) or {}
 
     from esphome import espota2
     from esphome.components.noise import static_encryption_key
@@ -1342,9 +1376,19 @@ def _upload_via_native_api(
     # Fail closed: an encryption block whose key did not resolve must never
     # fall back to a plaintext upload
     noise_psk = None
+    old_noise_psk = None
     plaintext_fallback = False
-    if (encryption_conf := ota_conf.get(CONF_ENCRYPTION)) is not None:
+    allow_plaintext_upload = False
+    if ota_key := getattr(args, "ota_key", None):
+        # The user presents a key the device still runs instead of the
+        # configured one (a key change in progress); an explicit key never
+        # downgrades to plaintext
+        noise_psk = ota_key
+    elif (encryption_conf := ota_conf.get(CONF_ENCRYPTION)) is not None:
         noise_psk = encryption_conf.get(CONF_KEY)
+        allow_plaintext_upload = bool(
+            encryption_conf.get(espota2.CONF_ALLOW_PLAINTEXT_UPLOAD)
+        )
         if not noise_psk:
             raise EsphomeError(
                 "OTA encryption is configured but no key was resolved; "
@@ -1353,6 +1397,8 @@ def _upload_via_native_api(
         # Ensure the key is a string, as required by the underlying OTA implementation.
         # It arrives here as a SensitiveStr which aioesphomeapi rejects.
         noise_psk = str(noise_psk)
+        if old_key := encryption_conf.get(espota2.CONF_OLD_KEY):
+            old_noise_psk = str(old_key)
     elif api_key := static_encryption_key(config.get(CONF_API) or {}):
         # Remove before 2027.3.0: the api key is tried, falling back to plaintext
         noise_psk = str(api_key)
@@ -1377,6 +1423,12 @@ def _upload_via_native_api(
         ota_type = espota2.OTA_TYPE_UPDATE_PARTITION_TABLE
     elif getattr(args, "bootloader", False):
         check_partition_access("--bootloader")
+        if (
+            getattr(args, "file", None) is None
+            and (native := native_backend())
+            and (hint := getattr(native, "missing_image_hint", lambda: None)())
+        ):
+            raise EsphomeError(hint)
         binary = CORE.bootloader_bin
         ota_type = espota2.OTA_TYPE_UPDATE_BOOTLOADER
     if getattr(args, "file", None) is not None:
@@ -1395,19 +1447,31 @@ def _upload_via_native_api(
         ota_type,
         noise_psk,
         plaintext_fallback=plaintext_fallback,
+        allow_plaintext_upload=allow_plaintext_upload,
+        old_noise_psk=old_noise_psk,
     )
 
 
+def _old_key_notice(config: ConfigType) -> None:
+    """After `run`, whose image was just built from this config; a plain
+    `upload` may send a stale image that still carries the previous key."""
+    from esphome import espota2
+
+    for item in espota2.esphome_ota_items(config):
+        if (item.get(CONF_ENCRYPTION) or {}).get(espota2.CONF_OLD_KEY):
+            _LOGGER.warning(espota2.OLD_KEY_REMOVE_NOTICE)
+
+
 def _upload_via_web_server(
     config: ConfigType, network_devices: list[str], binary: Path
 ) -> tuple[int, str | None]:
     from esphome import web_server_ota
+    from esphome.espota2 import esphome_ota_items
     from esphome.web_server_helpers import get_web_server_connection
 
     if any(
-        ota_item.get(CONF_PLATFORM) == CONF_ESPHOME
-        and ota_item.get(CONF_ENCRYPTION) is not None
-        for ota_item in config.get(CONF_OTA, [])
+        ota_item.get(CONF_ENCRYPTION) is not None
+        for ota_item in esphome_ota_items(config)
     ):
         _LOGGER.warning(
             "This config has OTA encryption, but the web_server OTA path sends "
@@ -1709,27 +1773,66 @@ def command_compile(args: ArgsProtocol, config: ConfigType) -> int | None:
     if exit_code != 0:
         return exit_code
     if CORE.is_host:
-        _LOGGER.info(
-            "Successfully compiled program to path '%s'", _host_program_path(config)
-        )
+        _LOGGER.info("Successfully compiled program to path '%s'", CORE.firmware_bin)
     else:
         _LOGGER.info("Successfully compiled program.")
     return 0
 
 
-def _host_program_path(config: ConfigType) -> str:
-    """Return the compiled host ELF path."""
-    if CORE.using_toolchain_esp_idf:
-        from esphome.espidf import toolchain
+ENV_OTA_KEY = "ESPHOME_OTA_KEY"
+OTA_KEY_PLATFORM_ERROR = (
+    f"--prompt-ota-key and {ENV_OTA_KEY} only apply to the {CONF_ESPHOME} OTA "
+    "platform; to flash by serial instead, name the port with --device"
+)
 
-        return str(toolchain.get_elf_path())
-    from esphome.platformio.toolchain import get_idedata
 
-    # Memoized by compile_program's own call; this is a dict lookup
-    return str(get_idedata(config).firmware_elf_path)
+def _read_ota_key(args: ArgsProtocol, config: ConfigType) -> None:
+    """Take the key to present to the device for this upload, from the
+    prompt or from ESPHOME_OTA_KEY, before any compile so the prompt is
+    not buried in build output."""
+    prompt = getattr(args, "prompt_ota_key", False)
+    env_key = getattr(args, "env_ota_key", None)
+    if not prompt and env_key is None:
+        return
+    if CORE.data.get(KEY_CORE, {}).get(KEY_TARGET_PLATFORM) == PLATFORM_HOST:
+        _LOGGER.warning("A host program runs locally; the presented OTA key is ignored")
+        return
+    # The HTTP path has no key handshake; decide before prompting or
+    # compiling. A serial target is a serial flash: the key is not even
+    # asked for, and --ota-platform means nothing to it. Every entry must
+    # be serial: a missing port is skipped and the next entry may be a host
+    devices = getattr(args, "device", None) or []
+    serial_target = bool(devices) and all(
+        device == "SERIAL"
+        or get_port_type(device) in (PortType.SERIAL, PortType.BOOTSEL)
+        for device in devices
+    )
+    chosen = (
+        _choose_ota_platform(config, getattr(args, "ota_platform", None))
+        if not serial_target and _ota_upload_platforms(config)
+        else None
+    )
+    if chosen == CONF_WEB_SERVER:
+        raise EsphomeError(OTA_KEY_PLATFORM_ERROR)
+    from esphome.components.noise import validate_encryption_key
+    from esphome.config_validation import Invalid
+
+    if serial_target:
+        _LOGGER.warning(
+            "The presented OTA key only applies to network uploads; ignored for %s",
+            ", ".join(devices),
+        )
+        return
+    key = read_secret_line("OTA encryption key: ") if prompt else env_key
+    try:
+        validate_encryption_key(key)
+    except Invalid as err:
+        raise EsphomeError(f"Invalid OTA encryption key: {err}") from err
+    args.ota_key = key
 
 
 def command_upload(args: ArgsProtocol, config: ConfigType) -> int | None:
+    _read_ota_key(args, config)
     # Get devices, resolving special identifiers like OTA
     devices = choose_upload_log_host(
         default=args.device,
@@ -1765,6 +1868,19 @@ def command_logs(args: ArgsProtocol, config: ConfigType) -> int | None:
 
 
 def command_run(args: ArgsProtocol, config: ConfigType) -> int | None:
+    if (
+        CORE.skip_bootloader
+        and CORE.is_esp32
+        and CORE.using_toolchain_esp_idf
+        and any(
+            get_port_type(device) == PortType.SERIAL for device in (args.device or [])
+        )
+    ):
+        # Fail before the compile: the result could never flash over serial.
+        # Elsewhere the flag is ignored, so serial stays fine there.
+        _LOGGER.error("--skip-bootloader builds cannot be flashed over serial")
+        return 1
+    _read_ota_key(args, config)
     exit_code = write_cpp(config)
     if exit_code != 0:
         return exit_code
@@ -1773,7 +1889,7 @@ def command_run(args: ArgsProtocol, config: ConfigType) -> int | None:
         return exit_code
     _LOGGER.info("Successfully compiled program.")
     if CORE.is_host:
-        program_path = _host_program_path(config)
+        program_path = str(CORE.firmware_bin)
         _LOGGER.info("Running program from path '%s'", program_path)
         return run_external_process(program_path)
 
@@ -1790,6 +1906,7 @@ def command_run(args: ArgsProtocol, config: ConfigType) -> int | None:
     exit_code, successful_device = upload_program(config, args, devices)
     if exit_code == 0:
         _LOGGER.info("Successfully uploaded program.")
+        _old_key_notice(config)
     else:
         _LOGGER.warning("Failed to upload to %s", devices)
         return exit_code
@@ -1926,7 +2043,10 @@ def run_multiple_configs(
         safe_print()
 
         cmd = command_builder(f)
-        rc = run_external_process(*cmd)
+        # The parent already logged the Intel macOS warning; children skip it.
+        rc = run_external_process(
+            *cmd, env={**os.environ, _INTEL_MACOS_WARNED_ENV: "1"}
+        )
 
         if rc == 0:
             print_bar(f"[{color(AnsiFore.BOLD_GREEN, 'SUCCESS')}] {str(f)}")
@@ -1965,12 +2085,12 @@ def command_update_all(args: ArgsProtocol) -> int | None:
 def command_idedata(args: ArgsProtocol, config: ConfigType) -> int:
     import json
 
-    if CORE.using_toolchain_esp_idf:
-        # Native ESP-IDF derives idedata from the build's compile_commands.json,
-        # so the configuration must already be compiled.
-        from esphome.espidf import toolchain as espidf_toolchain
+    native_toolchain = native_backend()
 
-        idedata = espidf_toolchain.get_idedata()
+    if native_toolchain is not None:
+        # Native toolchains derive idedata from the build's
+        # compile_commands.json, so the configuration must already be compiled.
+        idedata = native_toolchain.get_idedata()
         if idedata is None:
             _LOGGER.error(
                 "No idedata available; compile the configuration first",
@@ -2009,6 +2129,22 @@ def command_analyze_memory(args: ArgsProtocol, config: ConfigType) -> int:
     from esphome.analyze_memory.cli import MemoryAnalyzerCLI
     from esphome.analyze_memory.ram_strings import RamStringsAnalyzer
 
+    # Refuse an unsupported toolchain before paying for a full compile
+    analysis_toolchain = analysis_backend()
+    if analysis_toolchain is None and not CORE.using_toolchain_platformio:
+        _LOGGER.error(
+            "analyze-memory is not supported with the '%s' toolchain on %s; "
+            "re-run with --toolchain platformio",
+            CORE.toolchain.value if CORE.toolchain else "unresolved",
+            CORE.target_platform,
+        )
+        return 1
+    if (
+        check_supported := getattr(analysis_toolchain, "check_analysis_supported", None)
+    ) is not None:
+        # Raises with the reason; before the compile, not after it
+        check_supported()
+
     # Always compile to ensure fresh data (fast if no changes - just relinks)
     exit_code = write_cpp(config)
     if exit_code != 0:
@@ -2020,13 +2156,30 @@ def command_analyze_memory(args: ArgsProtocol, config: ConfigType) -> int:
 
     # Get idedata for analysis
     idedata = None
-    if CORE.using_toolchain_esp_idf:
-        from esphome.espidf import toolchain
+    if analysis_toolchain is not None:
+        objdump = analysis_toolchain.get_objdump_path()
+        readelf = analysis_toolchain.get_readelf_path()
+        for tool in (objdump, readelf):
+            if not tool.is_file():
+                # The analyzer would silently fall back to host
+                # binutils, which cannot read the target ELF
+                _LOGGER.error(
+                    "%s is missing; the toolchain install may be incomplete "
+                    "(recompile, or run 'esphome clean-all' if it persists)",
+                    tool,
+                )
+                return 1
+        objdump_path = str(objdump)
+        readelf_path = str(readelf)
 
-        objdump_path = str(toolchain.get_objdump_path())
-        readelf_path = str(toolchain.get_readelf_path())
-
-        firmware_elf = toolchain.get_elf_path()
+        firmware_elf = analysis_toolchain.get_elf_path()
+        if not firmware_elf.is_file():
+            # The analyzer swallows tool failures, so a missing ELF would
+            # produce an exit-0 zeroed report
+            _LOGGER.error(
+                "%s is missing; compile the configuration first", firmware_elf
+            )
+            return 1
     else:
         from esphome.platformio import toolchain
 
@@ -2079,156 +2232,16 @@ def command_analyze_memory(args: ArgsProtocol, config: ConfigType) -> int:
     return 0
 
 
+def command_rotate_key(args: ArgsProtocol, config: ConfigType) -> int | None:
+    from esphome.cli.rotate_key import command_rotate_key as run
+
+    return run(args, config)
+
+
 def command_rename(args: ArgsProtocol, config: ConfigType) -> int | None:
-    from esphome import yaml_util
+    from esphome.cli.rename import command_rename as run
 
-    new_name = args.name
-    for c in new_name:
-        if c not in ALLOWED_NAME_CHARS:
-            safe_print(
-                color(
-                    AnsiFore.BOLD_RED,
-                    f"'{c}' is an invalid character for names. Valid characters are: "
-                    f"{ALLOWED_NAME_CHARS} (lowercase, no spaces)",
-                )
-            )
-            return 1
-    # Load existing yaml file
-    raw_contents = CORE.config_path.read_text(encoding="utf-8")
-
-    yaml = yaml_util.load_yaml(CORE.config_path)
-    if CONF_ESPHOME not in yaml or CONF_NAME not in yaml[CONF_ESPHOME]:
-        safe_print(
-            color(
-                AnsiFore.BOLD_RED, "Complex YAML files cannot be automatically renamed."
-            )
-        )
-        return 1
-    old_name = yaml[CONF_ESPHOME][CONF_NAME]
-    match = re.match(r"^\$\{?([a-zA-Z0-9_]+)\}?$", old_name)
-    if match is None:
-        # Only swap the ``name:`` line that sits directly under the
-        # top-level ``esphome:`` block. A naked ``re.sub`` would
-        # also clobber any other ``name:`` line whose value happens
-        # to match (e.g. a sensor / output / wifi entry sharing the
-        # device's hostname), silently rewriting unrelated user
-        # configuration. The pattern anchors:
-        # - at the start of the line so ``friendly_name:``,
-        #   ``device_name:`` etc. don't match the trailing ``name:``
-        #   substring; and
-        # - at the end of the value (lookahead for whitespace +
-        #   comment + EOL) so ``old_name`` doesn't match as a
-        #   prefix of a longer value (``kitchen`` vs ``kitchen2``).
-        name_pattern = re.compile(
-            rf"^(\s*)name:\s+[\"']?{re.escape(old_name)}[\"']?(?=\s*(?:#|$))"
-        )
-        out_lines: list[str] = []
-        in_esphome_block = False
-        for line in raw_contents.splitlines(keepends=True):
-            if line and not line[0].isspace() and line.strip():
-                in_esphome_block = line.lstrip().startswith("esphome:")
-                out_lines.append(line)
-                continue
-            if in_esphome_block:
-                line = name_pattern.sub(rf'\1name: "{new_name}"', line, count=1)
-            out_lines.append(line)
-        new_raw = "".join(out_lines)
-    else:
-        old_name = yaml[CONF_SUBSTITUTIONS][match.group(1)]
-        if (
-            len(
-                re.findall(
-                    rf"^\s+{match.group(1)}:\s+[\"']?{old_name}[\"']?",
-                    raw_contents,
-                    flags=re.MULTILINE,
-                )
-            )
-            > 1
-        ):
-            safe_print(
-                color(AnsiFore.BOLD_RED, "Too many matches in YAML to safely rename")
-            )
-            return 1
-
-        new_raw = re.sub(
-            rf"^(\s+{match.group(1)}):\s+[\"']?{old_name}[\"']?",
-            f'\\1: "{new_name}"',
-            raw_contents,
-            flags=re.MULTILINE,
-        )
-
-    # ``new_name == old_name`` (after substitution resolution) is
-    # a no-op rewrite that would still queue a pointless re-flash.
-    # Catch it before the path-equality check below — covers the
-    # case where the config filename doesn't match the device name
-    # (e.g. ``weird-file.yaml`` whose ``esphome.name`` is
-    # ``kitchen``; running ``esphome rename weird-file.yaml kitchen``
-    # would otherwise just re-flash the same hostname).
-    if new_name == old_name:
-        safe_print(
-            color(
-                AnsiFore.BOLD_RED,
-                f"'{new_name}' is already the device's name.",
-            )
-        )
-        return 1
-
-    new_path: Path = CORE.config_dir / (new_name + ".yaml")
-    if new_path.resolve() == CORE.config_path.resolve():
-        safe_print(
-            color(
-                AnsiFore.BOLD_RED,
-                f"'{new_name}' is already the device's name.",
-            )
-        )
-        return 1
-    if new_path.exists():
-        safe_print(
-            color(
-                AnsiFore.BOLD_RED,
-                f"Cannot rename: {new_path} already exists. "
-                "Refusing to overwrite an existing configuration.",
-            )
-        )
-        return 1
-    safe_print(
-        f"Updating {color(AnsiFore.CYAN, str(CORE.config_path))} to {color(AnsiFore.CYAN, str(new_path))}"
-    )
-    print()
-
-    new_path.write_text(new_raw, encoding="utf-8")
-
-    rc = run_external_process(*ESPHOME_COMMAND, "config", str(new_path))
-    if rc != 0:
-        safe_print(color(AnsiFore.BOLD_RED, "Rename failed. Reverting changes."))
-        new_path.unlink()
-        return 1
-
-    cli_args = [
-        "run",
-        str(new_path),
-        "--no-logs",
-        "--device",
-        CORE.address,
-    ]
-
-    if args.dashboard:
-        cli_args.insert(0, "--dashboard")
-
-    try:
-        rc = run_external_process(*ESPHOME_COMMAND, *cli_args)
-    except KeyboardInterrupt:
-        rc = 1
-    if rc != 0:
-        new_path.unlink()
-        return 1
-
-    if CORE.config_path != new_path:
-        CORE.config_path.unlink()
-
-    safe_print(color(AnsiFore.BOLD_GREEN, "SUCCESS"))
-    print()
-    return 0
+    return run(args, config)
 
 
 PRE_CONFIG_ACTIONS = {
@@ -2251,6 +2264,7 @@ POST_CONFIG_ACTIONS = {
     "clean-mqtt": command_clean_mqtt,
     "idedata": command_idedata,
     "rename": command_rename,
+    "rotate-key": command_rotate_key,
     "discover": command_discover,
     "analyze-memory": command_analyze_memory,
     "bundle": command_bundle,
@@ -2263,6 +2277,15 @@ SIMPLE_CONFIG_ACTIONS = [
 ]
 
 
+def _add_skip_bootloader_arg(parser: argparse.ArgumentParser) -> None:
+    parser.add_argument(
+        "--skip-bootloader",
+        help="Do not build the bootloader or the factory image; "
+        "the result can only be flashed over OTA.",
+        action="store_true",
+    )
+
+
 def _add_states_args(parser: argparse.ArgumentParser) -> None:
     """Add mutually exclusive ``--states``/``--no-states`` flags to a parser.
 
@@ -2286,6 +2309,25 @@ def _add_states_args(parser: argparse.ArgumentParser) -> None:
     )
 
 
+def _error_format(value: str) -> ErrorFormat:
+    try:
+        return ErrorFormat(value.lower())
+    except ValueError:
+        choices = ", ".join(ErrorFormat)
+        raise argparse.ArgumentTypeError(
+            f"invalid choice: '{value}' (choose from {choices})"
+        ) from None
+
+
+ARGUMENT_HELP_PROMPT_OTA_KEY = (
+    "Ask for the OTA encryption key to present to the device instead of using "
+    "the one in the configuration, for a device that still runs a previous key. "
+    "The key is read without echo, or as one line from stdin when that is not "
+    "a terminal; it is never taken from the command line. A calling program "
+    f"can hand the key over in the {ENV_OTA_KEY} environment variable instead."
+)
+
+
 def parse_args(argv):
     options_parser = argparse.ArgumentParser(add_help=False)
     options_parser.add_argument(
@@ -2343,7 +2385,19 @@ def parse_args(argv):
         metavar="{" + ",".join(t.value for t in Toolchain) + "}",
         help=(
             "Select toolchain for compiling. Overrides '<platform>.toolchain' in YAML. "
-            f"Default: {Toolchain.PLATFORMIO.value}."
+            "Default: the platform's native toolchain where it has one, else "
+            f"{Toolchain.PLATFORMIO.value}."
+        ),
+    )
+
+    options_parser.add_argument(
+        "--error-format",
+        type=_error_format,
+        metavar="{" + ",".join(f.value for f in ErrorFormat) + "}",
+        help=(
+            "How configuration errors are shown. 'yaml' shows them inside the "
+            "failing YAML; 'line' prints one 'file:line:column: error: message' "
+            "line per error. Default: $ESPHOME_ERROR_FORMAT, else 'yaml'."
         ),
     )
 
@@ -2403,6 +2457,7 @@ def parse_args(argv):
         help="Only generate source code, do not compile.",
         action="store_true",
     )
+    _add_skip_bootloader_arg(parser_compile)
 
     parser_upload = subparsers.add_parser(
         "upload",
@@ -2441,6 +2496,11 @@ def parse_args(argv):
         help="Upload as partition table (OTA).",
         action="store_true",
     )
+    parser_upload.add_argument(
+        "--prompt-ota-key",
+        action="store_true",
+        help=ARGUMENT_HELP_PROMPT_OTA_KEY,
+    )
     parser_upload.add_argument(
         "--bootloader",
         help="Upload as bootloader (OTA).",
@@ -2466,7 +2526,7 @@ def parse_args(argv):
         "-r",
         action="store_true",
         help="Reset the device before starting serial logs.",
-        default=os.getenv("ESPHOME_SERIAL_LOGGING_RESET"),
+        default=get_bool_env("ESPHOME_SERIAL_LOGGING_RESET"),
     )
     _add_states_args(parser_logs)
 
@@ -2499,6 +2559,7 @@ def parse_args(argv):
     parser_run.add_argument(
         "--no-logs", help="Disable starting logs.", action="store_true"
     )
+    _add_skip_bootloader_arg(parser_run)
 
     _add_states_args(parser_run)
 
@@ -2507,7 +2568,7 @@ def parse_args(argv):
         "-r",
         action="store_true",
         help="Reset the device before starting serial logs.",
-        default=os.getenv("ESPHOME_SERIAL_LOGGING_RESET"),
+        default=get_bool_env("ESPHOME_SERIAL_LOGGING_RESET"),
     )
     parser_run.add_argument(
         "--ota-platform",
@@ -2520,6 +2581,11 @@ def parse_args(argv):
             "(HTTP Basic auth) when that is the only configured platform."
         ),
     )
+    parser_run.add_argument(
+        "--prompt-ota-key",
+        action="store_true",
+        help=ARGUMENT_HELP_PROMPT_OTA_KEY,
+    )
 
     parser_clean = subparsers.add_parser(
         "clean-mqtt",
@@ -2596,6 +2662,37 @@ def parse_args(argv):
     )
     parser_rename.add_argument("name", help="The new name for the device.", type=str)
 
+    parser_rotate_key = subparsers.add_parser(
+        "rotate-key",
+        help=(
+            "Change the OTA encryption key over the air: write a new key to the "
+            "yaml, build with it, upload with the current key, confirm."
+        ),
+        parents=[mqtt_options],
+    )
+    parser_rotate_key.add_argument(
+        "configuration", help="Your YAML configuration file.", nargs=1
+    )
+    parser_rotate_key.add_argument(
+        "--device",
+        action="append",
+        help=ARGUMENT_HELP_DEVICE,
+    )
+    parser_rotate_key.add_argument(
+        "--prompt-new-key",
+        action="store_true",
+        help=(
+            "Ask for the new key instead of generating one; read without echo, "
+            "or as one line from stdin when that is not a terminal."
+        ),
+    )
+    parser_rotate_key.add_argument(
+        "-y",
+        "--yes",
+        action="store_true",
+        help="Do not ask for confirmation when the api encryption key changes too.",
+    )
+
     parser_analyze_memory = subparsers.add_parser(
         "analyze-memory",
         help="Analyze memory usage by component.",
@@ -2648,9 +2745,18 @@ def parse_args(argv):
         args, unknown_args = parser.parse_known_args(arguments)
         if unknown_args:
             _LOGGER.warning("Ignored unrecognized arguments: %s", unknown_args)
-        return args
+    else:
+        args = parser.parse_args(arguments)
 
-    return parser.parse_args(arguments)
+    # Read here rather than as the option default, so a bad value is reported
+    # against the environment variable instead of --error-format
+    if args.error_format is None:
+        env_value = os.getenv("ESPHOME_ERROR_FORMAT", ErrorFormat.YAML.value)
+        try:
+            args.error_format = _error_format(env_value)
+        except argparse.ArgumentTypeError as err:
+            parser.error(f"ESPHOME_ERROR_FORMAT: {err}")
+    return args
 
 
 def _warn_if_source_tree_mismatch() -> None:
@@ -2696,12 +2802,34 @@ def _warn_if_source_tree_mismatch() -> None:
     )
 
 
+_INTEL_MACOS_REMOVAL = "2027.6.0"
+_INTEL_MACOS_WARNED_ENV = "ESPHOME_INTEL_MACOS_WARNED"
+
+
+def _warn_if_intel_macos() -> None:
+    """Warn that Intel (x86_64) Python on macOS loses support by _INTEL_MACOS_REMOVAL."""
+    if (
+        not IS_MACOS
+        or platform.machine() != "x86_64"
+        or _INTEL_MACOS_WARNED_ENV in os.environ
+    ):
+        return
+    _LOGGER.warning(
+        "Support for Intel Macs will end in ESPHome %s or earlier. The Python "
+        "packages ESPHome depends on have stopped publishing Intel macOS builds, "
+        "so future releases will not install on this machine. On an Apple "
+        "Silicon Mac, switch to a native arm64 Python.",
+        _INTEL_MACOS_REMOVAL,
+    )
+
+
 def run_esphome(argv):
     from esphome.address_cache import AddressCache
 
     args = parse_args(argv)
     CORE.dashboard = args.dashboard
     CORE.testing_mode = args.testing_mode
+    CORE.error_format = args.error_format
 
     # Create address cache from command-line arguments
     CORE.address_cache = AddressCache.from_cli_args(
@@ -2714,7 +2842,25 @@ def run_esphome(argv):
         args.log_level = "CRITICAL"
 
     setup_log(log_level=args.log_level)
+    # Taken out of the environment before any command runs, so no build
+    # tool inherits a key a calling program handed over
+    args.env_ota_key = os.environ.pop(ENV_OTA_KEY, None)
+    if args.command in ("upload", "run"):
+        # One key is one device's; several configs run in child processes
+        # that would never see it
+        if (args.env_ota_key is not None or args.prompt_ota_key) and len(
+            args.configuration
+        ) > 1:
+            raise EsphomeError(
+                f"--prompt-ota-key and {ENV_OTA_KEY} take one configuration; "
+                f"{len(args.configuration)} were given"
+            )
+    elif args.env_ota_key is not None:
+        _LOGGER.warning(
+            "%s is set but the %s command does not use it", ENV_OTA_KEY, args.command
+        )
     _warn_if_source_tree_mismatch()
+    _warn_if_intel_macos()
 
     if args.command in PRE_CONFIG_ACTIONS:
         try:
@@ -2758,6 +2904,7 @@ def run_esphome(argv):
 
     CORE.config_path = conf_path
     CORE.dashboard = args.dashboard
+    CORE.skip_bootloader = getattr(args, "skip_bootloader", False)
     if args.toolchain is not None:
         # CLI toolchain wins over esp32.toolchain in YAML.
         CORE.toolchain = args.toolchain

esphome/address_cache.py

@@ -148,6 +148,13 @@ class AddressCache:
                 continue
             hostname, ips = arg.split("=", 1)
             # Normalize hostname for consistent lookups
-            normalized = normalize_hostname(hostname)
-            cache[normalized] = [ip.strip() for ip in ips.split(",")]
+            normalized = normalize_hostname(hostname.strip())
+            addresses = [ip for value in ips.split(",") if (ip := value.strip())]
+            if not normalized or not addresses:
+                _LOGGER.warning(
+                    "Invalid cache entry: %s (hostname and at least one address are required)",
+                    arg,
+                )
+                continue
+            cache[normalized] = addresses
         return cache

esphome/analyze_memory/toolchain.py

@@ -37,7 +37,7 @@ def find_elf_path(build_path: Path) -> Path | None:
     """
     name = build_path.name
     for candidate in (
-        # Native ESP-IDF: idf.py writes build/<name>.elf, which ESPHome copies
+        # Native ESP-IDF: the build writes build/<name>.elf, which ESPHome copies
         # to build/firmware.elf (see espidf.toolchain.create_elf_copy)
         build_path / "build" / "firmware.elf",
         # PlatformIO
@@ -68,12 +68,16 @@ def idedata_candidates(build_path: Path) -> list[Path]:
         The candidate idedata JSON paths, most specific first
     """
     name = build_path.name
+    data_dir = build_path.parent.parent / "idedata"
+    # Native backends suffix the cache by toolchain (<name>.arduino.json)
+    suffixed = sorted(data_dir.glob(f"{name}.*.json")) if data_dir.is_dir() else []
     return [
         # In .pioenvs for test builds
         build_path / ".pioenvs" / name / "idedata.json",
         # Both toolchains cache it in the data dir, which holds this build dir:
         # <data_dir>/idedata/<name>.json next to <data_dir>/build/<name>
         build_path.parent.parent / "idedata" / f"{name}.json",
+        *suffixed,
         # Regular builds, invoked from the config dir or from anywhere
         Path.cwd() / ".esphome" / "idedata" / f"{name}.json",
         Path.home() / ".esphome" / "idedata" / f"{name}.json",

esphome/arduino/library.py

@@ -2,7 +2,9 @@
 
 Bundled names build straight from the framework tree; everything else goes
 through ``esphome.platformio.library``. Mirrors ``lib_ldf_mode=off``: each
-library builds its own archive; all include dirs join one global path.
+library builds its own archive; all include dirs join one global path. The
+host build reuses it without a framework tree: nothing is bundled there and
+every name resolves from the registry.
 
 Deviations from PlatformIO: flat-layout libraries get the recursive default
 source filter; ``dot_a_linkage`` is honored; bundled libraries never run a
@@ -342,13 +344,24 @@ def _check_unfulfilled_provides(
 
 
 def resolve_libraries(
-    framework_path: Path, *, pio_platform: str, board_mcu: str, cache_key: str
+    framework_path: Path | None,
+    *,
+    pio_platform: str,
+    board_mcu: str,
+    cache_key: str,
+    framework: str | None = "arduino",
+    manifest_optional: bool = False,
 ) -> list[ArduinoLibrary]:
     """Resolve every ``cg.add_library()`` entry into an :class:`ArduinoLibrary`.
 
     ``pio_platform``/``board_mcu`` filter manifests the way PlatformIO would
     for that core (e.g. ``espressif8266``/``esp8266``); ``cache_key`` keys the
-    shared converter's download cache.
+    shared converter's download cache. ``framework`` is the manifest
+    framework token the compatibility check warns about; None skips it.
+    A None ``framework_path`` means no core-bundled libraries exist (the
+    host build): every name resolves from the registry.
+    ``manifest_optional`` accepts libraries without a manifest, built with
+    PlatformIO's default layout.
 
     The returned list is not topologically sorted, so the caller must link
     the archives inside one ``--start-group``/``--end-group`` pair (the
@@ -359,18 +372,22 @@ def resolve_libraries(
     # PlatformIO's lib_ignore covers framework-bundled libraries too; the
     # shared converter only filters the registry/git ones.
     lib_ignore = lib_ignore_set()
-    # Exact directory names keep membership case-sensitive everywhere
-    # (an is_dir() probe would match "wire" on macOS/Windows and build
-    # the bundled Wire twice)
-    libraries_dir = framework_path / "libraries"
-    if not libraries_dir.is_dir():
-        # A registry fallback would fail later with a misleading
-        # package-not-found error per bundled name
-        raise EsphomeError(
-            f"{libraries_dir} is missing; the framework install may be "
-            "incomplete (run 'esphome clean-all')"
+    bundled_dir_names: frozenset[str] = frozenset()
+    if framework_path is not None:
+        # Exact directory names keep membership case-sensitive everywhere
+        # (an is_dir() probe would match "wire" on macOS/Windows and build
+        # the bundled Wire twice)
+        libraries_dir = framework_path / "libraries"
+        if not libraries_dir.is_dir():
+            # A registry fallback would fail later with a misleading
+            # package-not-found error per bundled name
+            raise EsphomeError(
+                f"{libraries_dir} is missing; the framework install may be "
+                "incomplete (run 'esphome clean-all')"
+            )
+        bundled_dir_names = frozenset(
+            p.name for p in libraries_dir.iterdir() if p.is_dir()
         )
-    bundled_dir_names = frozenset(p.name for p in libraries_dir.iterdir() if p.is_dir())
 
     def _provided(name: object) -> bool:
         return _is_safe_library_name(name) and name in bundled_dir_names
@@ -497,12 +514,13 @@ def resolve_libraries(
 
     backend = LibraryBackend(
         platform=pio_platform,
-        framework="arduino",
+        framework=framework,
         emit=_emit,
         cache_key=cache_key,
         # The walk must not resolve bundled names from the registry;
         # _add_bundled_dependencies adds them after emit
         provides=_provided,
+        manifest_optional=manifest_optional,
     )
     if external:
         convert_libraries(external, backend)

esphome/arduino8266/framework.py

@@ -3,12 +3,12 @@
 Artifacts land in a machine-global cache (shared across projects, like the
 ESP-IDF install in ``esphome.espidf.framework``):
 
-    <cache>/arduino8266/frameworks/<version>/   framework-arduinoespressif8266
-    <cache>/arduino8266/toolchains/<version>/   toolchain-xtensa (gcc 10.3)
+    <cache>/arduino8266/frameworks/<tag>/   the Arduino core
+    <cache>/arduino8266/toolchains/<tag>/   xtensa-lx106-elf gcc 10.3
 
-Packages come from the PlatformIO registry (identical bits to the PlatformIO
-backend); ``ESPHOME_ARDUINO8266_*_MIRRORS`` overrides the URLs. ninja comes
-from PATH or the ninja PyPI wheel.
+Both come from esphome-libs releases pinned below;
+``ESPHOME_ARDUINO8266_*_MIRRORS`` overrides the URLs, with ``{VERSION}``
+standing for the release tag. ninja comes from PATH or the ninja PyPI wheel.
 """
 
 from __future__ import annotations
@@ -17,18 +17,74 @@ import os
 from pathlib import Path
 from typing import NamedTuple
 
-from esphome.build_helpers.ccache import ccache_defaults_env
+from esphome.build_helpers.ccache import ccache_env
 from esphome.build_helpers.ninja import find_ninja
 from esphome.build_helpers.tools_cache import ARDUINO8266_TOOLS_CACHE, tools_cache_path
 from esphome.core import EsphomeError, Version
 from esphome.framework_helpers import str_to_lst_of_str
-from esphome.platformio.registry import install_package, prefetch_packages
+from esphome.platformio.registry import (
+    Download,
+    PackageSpec,
+    Resolver,
+    get_systype,
+    install_packages,
+    prefetch_packages,
+)
 
-FRAMEWORK_PACKAGE = "framework-arduinoespressif8266"
-TOOLCHAIN_PACKAGE = "toolchain-xtensa"
+FRAMEWORK_PACKAGE = "arduino-esp8266"
+_FRAMEWORK_RELEASES = "https://github.com/esphome-libs/arduino-esp8266/releases/"
+
+
+class FrameworkRelease(NamedTuple):
+    tag: str
+    sha256: str
+    size: int
+
+    def download(self) -> Download:
+        archive = f"{FRAMEWORK_PACKAGE}-{self.tag}.tar.gz"
+        url = f"{_FRAMEWORK_RELEASES}download/{self.tag}/{archive}"
+        return Download(url, self.sha256, self.size)
+
+
+# Arduino core version -> its build in esphome-libs/arduino-esp8266
+FRAMEWORK_RELEASES: dict[Version, FrameworkRelease] = {
+    Version(3, 1, 2): FrameworkRelease(
+        "3.1.2-esphome.3",
+        "3af533a85163804e164500a5b254dc23b922080e25ecb7dc37b3dd8ac84651db",
+        37185650,
+    ),
+}
+
+TOOLCHAIN_PACKAGE = "toolchain-xtensa-lx106-elf"
 # gcc 10.3, the toolchain Arduino core 3.x builds with; the build
 # generator's compile flags are tuned to it.
-TOOLCHAIN_VERSION = "2.100300.220621"
+TOOLCHAIN_VERSION = "10.3.0-esphome.3"
+_TOOLCHAIN_RELEASES = (
+    "https://github.com/esphome-libs/xtensa-lx106-elf-toolchain/releases/"
+)
+# Registry system tag -> (sha256, size) of that host's archive
+TOOLCHAIN_BUILDS: dict[str, tuple[str, int]] = {
+    "darwin_arm64": (
+        "ad0ea929238d9b527c43e0d20257acc2a5680bdb0d2b1cf8f2c99169786217e9",
+        60747260,
+    ),
+    "darwin_x86_64": (
+        "997090cac80c8eb59604ec52b73fb3568e5f2d9b2971f83c3f2a5c545c9c949c",
+        64065752,
+    ),
+    "linux_aarch64": (
+        "403f14d0d5755682fcd6eaea1e4c457a1d8cad30690e52fb0c804d8db266b908",
+        67487067,
+    ),
+    "linux_x86_64": (
+        "64f21fa3ba736b363dc8c234dbbb39e005e574e33de08a78e9a00c55a64ae706",
+        68343668,
+    ),
+    "windows_amd64": (
+        "1e62abf8df041ce5158f5770dffef955f2f981ce7f7adee88fcd5a8197705aa6",
+        67536405,
+    ),
+}
 
 ESPHOME_ARDUINO8266_FRAMEWORK_MIRRORS = str_to_lst_of_str(
     os.environ.get("ESPHOME_ARDUINO8266_FRAMEWORK_MIRRORS", "")
@@ -44,36 +100,40 @@ def get_arduino8266_tools_path() -> Path:
     return tools_cache_path(*ARDUINO8266_TOOLS_CACHE)
 
 
-# 3.1.1 rather than 3.1.0: the registry has no packages for 3.0.0, 3.0.1 or 3.1.0
-MIN_FRAMEWORK_VERSION = Version(3, 1, 1)
-
-
-def framework_package_version(ver: Version) -> str:
-    """Map an Arduino core version to its registry package version (3.1.2 ->
-    3.30102.0; the leading 3 is the package major).
-
-    Exact registry names for 3.x cores; callers floor at MIN_FRAMEWORK_VERSION.
-    """
-    if ver.major > 3:
+def framework_release(version: Version) -> FrameworkRelease:
+    if (release := FRAMEWORK_RELEASES.get(version)) is None:
         raise EsphomeError(
-            f"Arduino core {ver} is not supported yet; "
-            "the newest known core series is 3.x"
+            f"'toolchain: arduino' has no build of Arduino core {version}; "
+            f"available: {', '.join(str(v) for v in FRAMEWORK_RELEASES)}. "
+            "Use one of those or 'toolchain: platformio'"
         )
-    if ver.major < 3:
-        raise EsphomeError(
-            f"Arduino core {ver} is not supported; ESPHome requires core 3.x"
-        )
-    return f"3.{ver.major}{ver.minor:02d}{ver.patch:02d}.0"
+    return release
 
 
-def get_framework_path(package_version: str) -> Path:
-    return get_arduino8266_tools_path() / "frameworks" / package_version
+def get_framework_path(tag: str) -> Path:
+    return get_arduino8266_tools_path() / "frameworks" / tag
 
 
 def get_toolchain_path() -> Path:
     return get_arduino8266_tools_path() / "toolchains" / TOOLCHAIN_VERSION
 
 
+def toolchain_download() -> Download:
+    """The toolchain archive for the current host."""
+    systype = get_systype()
+    if (build := TOOLCHAIN_BUILDS.get(systype)) is None:
+        raise EsphomeError(
+            f"There is no ESP8266 toolchain for this system ({systype}); "
+            f"supported systems are {', '.join(sorted(TOOLCHAIN_BUILDS))}. "
+            "Either set 'toolchain: platformio' under 'esp8266:', or point "
+            "ESPHOME_ARDUINO8266_TOOLCHAIN_MIRRORS at a toolchain archive"
+        )
+    sha256, size = build
+    archive = f"{TOOLCHAIN_PACKAGE}-{TOOLCHAIN_VERSION}-{systype}.tar.gz"
+    url = f"{_TOOLCHAIN_RELEASES}download/{TOOLCHAIN_VERSION}/{archive}"
+    return Download(url, sha256, size)
+
+
 class InstalledPaths(NamedTuple):
     """Locations of the installed framework, toolchain, and ninja binary."""
 
@@ -84,29 +144,22 @@ class InstalledPaths(NamedTuple):
 
 def check_and_install(framework_version: Version) -> InstalledPaths:
     """Ensure framework, toolchain, and ninja are installed; return their paths."""
-    if framework_version < MIN_FRAMEWORK_VERSION:
-        # Config validation enforces this too; keep the module honest when
-        # called directly.
-        raise EsphomeError(
-            f"The native toolchain requires the Arduino core "
-            f">= {MIN_FRAMEWORK_VERSION}, got {framework_version}"
-        )
+    release = framework_release(framework_version)
     # Probe the cheap local dependency before ~110 MB of downloads
     ninja_path = find_ninja()
-    package_version = framework_package_version(framework_version)
-    framework_path = get_framework_path(package_version)
+    framework_path = get_framework_path(release.tag)
     downloads_dir = get_arduino8266_tools_path() / "downloads"
     toolchain_path = get_toolchain_path()
     # One spec per package: the prefetch and the installs must agree
     specs = (
-        (
+        PackageSpec(
             FRAMEWORK_PACKAGE,
-            package_version,
+            release.tag,
             framework_path,
             ESPHOME_ARDUINO8266_FRAMEWORK_MIRRORS,
             ("cores/esp8266", "tools/sdk", "libraries"),
         ),
-        (
+        PackageSpec(
             TOOLCHAIN_PACKAGE,
             TOOLCHAIN_VERSION,
             toolchain_path,
@@ -115,10 +168,17 @@ def check_and_install(framework_version: Version) -> InstalledPaths:
             ("bin", "xtensa-lx106-elf"),
         ),
     )
-    # Fetch both archives at once; the installs below verify and extract
-    prefetch_packages([spec[:4] for spec in specs], downloads_dir)
-    for name, version, dest, mirrors, expect in specs:
-        install_package(name, version, dest, mirrors, downloads_dir, expect=expect)
+    # Resolved only when a download is needed, so an installed toolchain
+    # keeps working on a host without a build; a mirror override wins
+    resolvers: dict[str, Resolver] = {}
+    if not ESPHOME_ARDUINO8266_FRAMEWORK_MIRRORS:
+        resolvers[FRAMEWORK_PACKAGE] = release.download
+    if not ESPHOME_ARDUINO8266_TOOLCHAIN_MIRRORS:
+        resolvers[TOOLCHAIN_PACKAGE] = toolchain_download
+    # Fetch both archives at once; the installs verify and extract them.
+    # One spec list for both, so the two phases cannot drift.
+    prefetch_packages(specs, downloads_dir, resolvers)
+    install_packages(specs, downloads_dir, resolvers)
     return InstalledPaths(
         framework=framework_path, toolchain=toolchain_path, ninja=ninja_path
     )
@@ -143,17 +203,5 @@ def get_build_env(toolchain_path: Path, ccache: str | None) -> dict[str, str]:
         *filter(None, env.get("PATH", "").split(os.pathsep)),
     ]
     env["PATH"] = os.pathsep.join(parts)
-    env.update(ccache_env(ccache))
+    env.update(ccache_env(ccache, ARDUINO8266_TOOLS_CACHE))
     return env
-
-
-def ccache_env(ccache: str | None) -> dict[str, str]:
-    """Return ccache settings for the build subprocess (not os.environ).
-
-    ``ccache`` is the pre-resolved binary (resolve_ccache_path), or None
-    when disabled. Values the user already set in the environment are
-    respected.
-    """
-    if ccache is None:
-        return {}
-    return ccache_defaults_env(get_arduino8266_tools_path() / "ccache")

esphome/arduino8266/toolchain.py

@@ -0,0 +1,212 @@
+"""Native Arduino ESP8266 build driver (the PlatformIO ``run`` equivalent)."""
+
+from __future__ import annotations
+
+import logging
+from pathlib import Path
+import subprocess
+from typing import TYPE_CHECKING
+
+from esphome.build_helpers.ccache import resolve_absolute_ccache_path
+from esphome.build_helpers.native import warn_ignored_platformio_options
+from esphome.build_helpers.ninja import refresh_compile_commands
+from esphome.const import (
+    CONF_COMPILE_PROCESS_LIMIT,
+    CONF_ESPHOME,
+    KEY_CORE,
+    KEY_FRAMEWORK_VERSION,
+)
+from esphome.core import CORE
+from esphome.types import ConfigType
+
+if TYPE_CHECKING:
+    from esphome.arduino8266.framework import InstalledPaths
+
+_LOGGER = logging.getLogger(__name__)
+
+# ESP8266 user RAM (matches upload.maximum_ram_size in every board manifest)
+_MAX_RAM_SIZE = 81920
+
+
+_RAM_SECTIONS = (".data", ".rodata", ".bss")
+_FLASH_SECTIONS = (".irom0.text", ".text", ".text1", ".data", ".rodata")
+
+
+def get_build_dir() -> Path:
+    return CORE.relative_pioenvs_path(CORE.name)
+
+
+def get_elf_path() -> Path:
+    return get_build_dir() / "firmware.elf"
+
+
+def _toolchain_tool(name: str) -> Path:
+    # Imported here, not at module scope: the serial upload/logs fast path
+    # resolves this module for its artifact paths alone, and framework
+    # pulls in the whole package-download stack
+    from esphome.arduino8266 import framework
+
+    return framework.toolchain_tool(framework.get_toolchain_path(), name)
+
+
+def get_factory_firmware_path() -> Path:
+    """The image to serial-flash at 0x0 (same bytes as firmware.bin: the
+    8266 factory copy exists for artifact-contract parity, not content)."""
+    return get_build_dir() / "firmware.factory.bin"
+
+
+def get_addr2line_path() -> Path:
+    return _toolchain_tool("addr2line")
+
+
+def get_objdump_path() -> Path:
+    return _toolchain_tool("objdump")
+
+
+def get_readelf_path() -> Path:
+    return _toolchain_tool("readelf")
+
+
+def run_compile(config: ConfigType, verbose: bool) -> int:
+    from esphome.arduino8266 import framework
+    from esphome.build_gen import arduino8266 as build_gen
+    from esphome.core.config import NATIVE_ARDUINO_CONSUMED_PIO_OPTIONS
+
+    warn_ignored_platformio_options(NATIVE_ARDUINO_CONSUMED_PIO_OPTIONS)
+    paths = framework.check_and_install(CORE.data[KEY_CORE][KEY_FRAMEWORK_VERSION])
+    # Resolved once: the probe is not free and three consumers need it
+    ccache = resolve_absolute_ccache_path()
+    ninja_changed = build_gen.write_project(paths, ccache)
+
+    build_dir = get_build_dir()
+    env = framework.get_build_env(paths.toolchain, ccache)
+    refresh_compile_commands(paths.ninja, build_dir, env, ninja_changed)
+
+    cmd = [str(paths.ninja)]
+    if verbose:
+        cmd.append("-v")
+    if jobs := config[CONF_ESPHOME].get(CONF_COMPILE_PROCESS_LIMIT):
+        cmd += ["-j", str(jobs)]
+    # Explicit targets: a generator defect that drops them fails loudly
+    # instead of a green no-op run leaving stale artifacts in place
+    targets = ["firmware.factory.bin", "firmware.ota.bin"]
+    cmd += targets
+
+    # cwd, not -C: drops ninja's "Entering directory" banner
+    _LOGGER.debug("Running: %s", " ".join(cmd))
+    rc = subprocess.run(
+        cmd, cwd=build_dir, env=env, check=False, close_fds=False
+    ).returncode
+    if rc != 0:
+        return rc
+
+    # ninja already refused missing targets; existence covers a rule that
+    # ran but wrote elsewhere
+    build_dir_artifacts = (
+        get_elf_path(),
+        build_dir / "firmware.bin",
+        get_factory_firmware_path(),
+        build_dir / "firmware.ota.bin",
+    )
+    for artifact in build_dir_artifacts:
+        if not artifact.is_file():
+            _LOGGER.error("Build produced no %s", artifact)
+            return 1
+
+    if not _print_size_summary(build_dir, paths):
+        # Cause already warned; name the consequence for CI harnesses
+        _LOGGER.warning("Firmware size summary unavailable for this build")
+    from esphome.build_helpers.idedata import warn_if_idedata_missing
+
+    warn_if_idedata_missing(lambda: get_idedata(ccache))
+    return 0
+
+
+def _parse_app_size(build_dir: Path, paths: InstalledPaths) -> int | None:
+    """Read the app flash budget (irom0_0_seg length) from the linker script."""
+    from esphome.build_gen.arduino8266 import get_flash_ld_path
+    from esphome.components.esp8266.build_surgery import segment_length
+
+    # Warnings, not debug: without the app size the Flash summary line is
+    # dropped and CI's memory-impact extraction loses its flash metric.
+    ld_path = get_flash_ld_path(build_dir, paths)
+    try:
+        ld_text = ld_path.read_text(encoding="utf-8")
+    except (OSError, UnicodeDecodeError) as err:
+        # A corrupt script degrades the same way, never aborts the build
+        _LOGGER.warning("Cannot read linker script for the Flash summary: %s", err)
+        return None
+    if not (app_size := segment_length(ld_text, "irom0_0_seg")):
+        _LOGGER.warning("No usable irom0_0_seg in %s; skipping Flash summary", ld_path)
+        return None
+    return app_size
+
+
+def _print_size_summary(build_dir: Path, paths: InstalledPaths) -> bool:
+    """Print the RAM/Flash lines ``ci_memory_impact_extract.py`` parses;
+    False when skipped."""
+    from esphome.arduino8266.framework import toolchain_tool
+    from esphome.build_helpers.size_summary import print_size_line
+
+    try:
+        result = subprocess.run(
+            [
+                str(toolchain_tool(paths.toolchain, "size")),
+                "-A",
+                "-d",
+                str(build_dir / "firmware.elf"),
+            ],
+            capture_output=True,
+            text=True,
+            check=True,
+            close_fds=False,
+        )
+    except (OSError, subprocess.CalledProcessError) as err:
+        # The summary is a bonus artifact like idedata; a truncated
+        # toolchain extraction must not discard an already-linked build
+        _LOGGER.warning("Could not summarize firmware size: %s", err)
+        return False
+    # -d prints decimal sizes; anything else trips the missing-sections guard
+    sections = {
+        parts[0]: int(parts[1])
+        for line in result.stdout.splitlines()
+        if (parts := line.split())[:1] and parts[0].startswith(".") and len(parts) >= 2
+        if parts[1].isdigit()
+    }
+    if missing := set(_RAM_SECTIONS + _FLASH_SECTIONS) - set(sections):
+        # A defaulted 0 would print a confidently wrong total for CI's metric
+        _LOGGER.warning(
+            "Size output is missing section(s) %s; skipping the size summary",
+            ", ".join(sorted(missing)),
+        )
+        return False
+    # Resolve the flash budget before printing: a RAM line without its
+    # Flash line would skew CI's memory-impact extraction
+    app_size = _parse_app_size(build_dir, paths)
+    if not app_size:
+        return False
+    ram = sum(sections[s] for s in _RAM_SECTIONS)
+    flash = sum(sections[s] for s in _FLASH_SECTIONS)
+    print_size_line("RAM", ram, _MAX_RAM_SIZE)
+    print_size_line("Flash", flash, app_size)
+    return True
+
+
+def get_idedata(ccache: str | None = None) -> dict | None:
+    """Derive idedata from the build's compile_commands.json (same
+    contract as ``espidf.toolchain.get_idedata``)."""
+    from esphome.build_helpers.idedata import load_or_build_idedata
+
+    # A disabled ccache resolves to None without spawning anything, so
+    # re-resolving here costs nothing when the caller has no answer
+    launcher = ccache or resolve_absolute_ccache_path()
+    return load_or_build_idedata(
+        get_build_dir() / "compile_commands.json",
+        get_elf_path(),
+        # Suffixed so a platformio->arduino->platformio round trip on one
+        # config never serves the other toolchain's cache shape
+        CORE.relative_internal_path("idedata", f"{CORE.name}.arduino.json"),
+        # The compile DB's commands carry the same ccache prefix the ninja
+        # rules were generated with
+        launcher=str(launcher) if launcher else None,
+    )

esphome/automation.py

@@ -1,5 +1,8 @@
+from collections.abc import Awaitable, Callable, Sequence
 from dataclasses import dataclass, field
 import logging
+import string
+from typing import Any
 
 import esphome.codegen as cg
 import esphome.config_validation as cv
@@ -18,18 +21,46 @@ from esphome.const import (
     CONF_TYPE_ID,
     CONF_UPDATE_INTERVAL,
 )
-from esphome.core import ID, Lambda
+from esphome.core import CORE, ID, EsphomeError, HexInt, Lambda
 from esphome.cpp_generator import (
+    Expression,
     LambdaExpression,
     MockObj,
     MockObjClass,
     TemplateArgsType,
+    call_lambda,
 )
 from esphome.schema_extractors import SCHEMA_EXTRACT, schema_extractor
-from esphome.types import ConfigType
+from esphome.types import ConfigType, SafeExpType
 from esphome.util import Registry
 
 
+def progmem_bytes(name: str, data: bytes | list[int]) -> MockObj:
+    """Shared PROGMEM table for constant bytes; equal payloads share one, empty is nullptr."""
+    if not data:
+        return cg.nullptr
+    return cg.shared_progmem_array(
+        name, cg.uint8, cg.ArrayInitializer(*(HexInt(x) for x in data))
+    )
+
+
+async def templatable_bytes(
+    value: Any,
+    args: TemplateArgsType,
+    set_template: MockObj,
+    set_static: MockObj,
+    table_name: str,
+) -> None:
+    """Set a TemplatableBytes: a lambda via set_template, constant bytes via set_static."""
+    if cg.is_template(value):
+        fn = await cg.templatable(value, args, cg.std_vector.template(cg.uint8))
+        cg.add(set_template(fn))
+    elif len(value) > 0xFFFF:
+        raise EsphomeError(f"Byte payload is {len(value)} bytes; the maximum is 65535")
+    else:
+        cg.add(set_static(progmem_bytes(table_name, value), len(value)))
+
+
 def maybe_simple_id(*validators):
     """Allow a raw ID to be specified in place of a config block.
     If the value that's being validated is a dictionary, it's passed as-is to the specified validators. Otherwise, it's
@@ -57,6 +88,7 @@ def maybe_conf(conf, *validators):
         with cv.remove_prepend_path([conf]):
             return validator({conf: value})
 
+    validate.inner_schema = validator
     return validate
 
 
@@ -102,6 +134,101 @@ def register_condition(name: str, condition_type: MockObjClass, schema: cv.Schem
     return CONDITION_REGISTRY.register(name, condition_type, schema)
 
 
+async def _build_with_parent(
+    config: ConfigType,
+    automation_id: ID,
+    template_arg: cg.TemplateArguments,
+    args: TemplateArgsType,
+) -> MockObj:
+    parent = await cg.get_variable(config[CONF_ID])
+    return cg.new_Pvariable(automation_id, template_arg, parent)
+
+
+async def _build_without_parent(
+    config: ConfigType,
+    automation_id: ID,
+    template_arg: cg.TemplateArguments,
+    args: TemplateArgsType,
+) -> MockObj:
+    return cg.new_Pvariable(automation_id, template_arg)
+
+
+async def _build_parented(
+    config: ConfigType,
+    automation_id: ID,
+    template_arg: cg.TemplateArguments,
+    args: TemplateArgsType,
+) -> MockObj:
+    var = cg.new_Pvariable(automation_id, template_arg)
+    await cg.register_parented(var, config[CONF_ID])
+    return var
+
+
+def register_simple_action(
+    name: str,
+    action_type: MockObjClass,
+    schema: cv.Schema,
+    *,
+    synchronous: bool,
+) -> None:
+    """Register an action whose constructor takes the object named by ``config[CONF_ID]``.
+
+    Use the ``register_action`` decorator instead when the builder must also set fields.
+    """
+    register_action(name, action_type, schema, synchronous=synchronous)(
+        _build_with_parent
+    )
+
+
+def register_simple_condition(
+    name: str, condition_type: MockObjClass, schema: cv.Schema
+) -> None:
+    """Condition counterpart of ``register_simple_action``."""
+    register_condition(name, condition_type, schema)(_build_with_parent)
+
+
+def register_bare_action(
+    name: str,
+    action_type: MockObjClass,
+    schema: cv.Schema,
+    *,
+    synchronous: bool,
+) -> None:
+    """Register an action whose constructor takes no arguments."""
+    register_action(name, action_type, schema, synchronous=synchronous)(
+        _build_without_parent
+    )
+
+
+def register_bare_condition(
+    name: str, condition_type: MockObjClass, schema: cv.Schema
+) -> None:
+    """Condition counterpart of ``register_bare_action``."""
+    register_condition(name, condition_type, schema)(_build_without_parent)
+
+
+def register_parented_action(
+    name: str,
+    action_type: MockObjClass,
+    schema: cv.Schema,
+    *,
+    synchronous: bool,
+) -> None:
+    """Register an action deriving from ``Parented<T>``.
+
+    The object is constructed without arguments and ``set_parent()`` receives the object
+    named by ``config[CONF_ID]``.
+    """
+    register_action(name, action_type, schema, synchronous=synchronous)(_build_parented)
+
+
+def register_parented_condition(
+    name: str, condition_type: MockObjClass, schema: cv.Schema
+) -> None:
+    """Condition counterpart of ``register_parented_action``."""
+    register_condition(name, condition_type, schema)(_build_parented)
+
+
 Action = cg.esphome_ns.class_("Action")
 Trigger = cg.esphome_ns.class_("Trigger")
 ACTION_REGISTRY = Registry()
@@ -112,6 +239,376 @@ validate_action_list = cv.validate_registry("action", ACTION_REGISTRY)
 validate_condition = cv.validate_registry_entry("condition", CONDITION_REGISTRY)
 validate_condition_list = cv.validate_registry("condition", CONDITION_REGISTRY)
 
+ApplyAction = cg.esphome_ns.class_("ApplyAction", Action)
+ApplyCondition = cg.esphome_ns.class_("ApplyCondition", Condition)
+
+
+def flash_string(config: ConfigType, value: str) -> str:
+    """Default renderer for ``std::string`` constants; copies the literal out of flash on ESP8266."""
+    return str(cg.progmem_string(value))
+
+
+def flash_compare(config: ConfigType, value: str) -> str:
+    """Default renderer for ``std::string`` constants in conditions; compares in flash on ESP8266."""
+    if CORE.is_esp8266:
+        return f"::esphome::ProgmemStringRef{{PSTR({cg.safe_exp(value)}), {len(value.encode('utf-8'))}}}"
+    return str(cg.safe_exp(value))
+
+
+def _default_std_string(
+    members: list[tuple[str, Any, Any]], renderer: Callable[[ConfigType, str], str]
+) -> list[tuple[str, Any, Any]]:
+    """Give ``cg.std_string`` members without a renderer the given default."""
+    return [
+        (key, t, fn or (renderer if t is cg.std_string else None))
+        for key, t, fn in members
+    ]
+
+
+def literal_with_length(config: ConfigType, value: str) -> str:
+    """Renderer for a ``(const char *, size_t)`` target: a plain literal plus its byte length.
+
+    The target compares or copies the bytes in place, so it needs the RAM literal rather than
+    the PROGMEM rendering on ESP8266, and the length saves a strlen.
+    """
+    return f"{cg.safe_exp(value)}, {len(value.encode('utf-8'))}"
+
+
+def string_ref_literal(config: ConfigType, value: str) -> str:
+    """Renderer for a ``StringRef`` comparison: a flash literal on ESP8266, else ``StringRef(literal, length)``."""
+    if CORE.is_esp8266:
+        return str(cg.FlashStringLiteral(value))
+    return f"StringRef({literal_with_length(config, value)})"
+
+
+def _names_parent(text: str) -> bool:
+    return any(f == "parent" for _, f, _, _ in string.Formatter().parse(text))
+
+
+@dataclass(frozen=True)
+class ApplyCall:
+    """One statement from config keys, e.g. ``"set_range({}, {})"`` with ``((CONF_LOW, cg.float_), ...)``.
+
+    Each arg is ``(conf_key, type_)`` or ``(conf_key, type_, const_fn)``. A ``conf_key`` may be a
+    path into nested sections. ``target`` and a plain ``str`` ``type_`` (raw C++ type text) may
+    name the parent object as ``{parent}``. ``const_fn(config, value)`` renders a constant's argument text; a lambda or an
+    id bypasses it. The statement is skipped when none of its keys is set, always emitted when it
+    has no keys, and a partial set is a config error.
+    """
+
+    target: str
+    args: tuple[tuple[Any, ...], ...] = ()
+
+    def __post_init__(self) -> None:
+        fields = [
+            f for _, f, _, _ in string.Formatter().parse(self.target) if f is not None
+        ]
+        if any(f not in ("", "parent") for f in fields):
+            raise ValueError(
+                f"apply target {self.target!r}: only {{}} and {{parent}} placeholders"
+            )
+        if (count := fields.count("")) != len(self.args):
+            raise ValueError(
+                f"apply target {self.target!r} has {count} "
+                f"placeholder(s) for {len(self.args)} config key(s)"
+            )
+        if any(len(arg) not in (2, 3) for arg in self.args):
+            raise ValueError(
+                f"apply target {self.target!r}: each arg is (conf_key, type_[, const_fn])"
+            )
+
+    @property
+    def names_parent(self) -> bool:
+        return _names_parent(self.target) or any(
+            isinstance(arg[1], str) and _names_parent(arg[1]) for arg in self.args
+        )
+
+    @property
+    def members(self) -> list[tuple[Any, Any, Any]]:
+        """Each arg as ``(conf_key, type_, const_fn or None)``."""
+        return [
+            (arg[0], arg[1], arg[2] if len(arg) == 3 else None) for arg in self.args
+        ]
+
+
+@dataclass(frozen=True)
+class ApplyField:
+    """One config key forwarded as ``target(value)``, or as statement ``target`` when it has ``{}``.
+
+    Double a literal brace in a template. ``conf_key`` may be a path into nested sections.
+    ``type_`` may be a C++ type string using ``{parent}`` when the type is only known per
+    instance. ``const_fn(config, value)`` renders a constant's argument text when ``cg.safe_exp``
+    is not the right spelling (unit conversion belongs in the validator); a lambda or an id
+    bypasses it, so the target must also take a plain ``type_``. An absent key emits nothing.
+    """
+
+    conf_key: str | tuple[str, ...]
+    target: str
+    type_: SafeExpType
+    const_fn: Callable[[ConfigType, Any], str] | None = None
+
+    def call(self) -> ApplyCall:
+        target = self.target if "{}" in self.target else f"{self.target}({{}})"
+        return ApplyCall(target, ((self.conf_key, self.type_, self.const_fn),))
+
+
+def _config_lookup(config: ConfigType, key: str | tuple[str, ...]) -> Any:
+    if isinstance(key, str):
+        return config.get(key)
+    for part in key:
+        if (config := config.get(part)) is None:
+            return None
+    return config
+
+
+def _dict_schema(schema: Any) -> Any:
+    """The dict-backed cv.Schema inside cv.All and maybe_* wrappers, or None; cv.Any is not inspected."""
+    if isinstance(schema, dict):
+        return cv.Schema(schema)
+    if isinstance(getattr(schema, "schema", None), dict):
+        return schema
+    if isinstance(schema, cv.All):
+        inner = schema.validators
+    else:
+        inner = (
+            getattr(schema, "inner_schema", None),
+        )  # maybe_conf / maybe_simple_value
+    for candidate in inner:
+        if candidate is not None and (found := _dict_schema(candidate)) is not None:
+            return found
+    return None
+
+
+def _check_key_in_schema(
+    name: str, schema: Any, conf_key: str | tuple[str, ...]
+) -> None:
+    """Reject a key path the schema does not have; a typo would otherwise be a silent no-op.
+
+    Only dict-backed schemas, also inside cv.All and maybe_* wrappers, can be checked.
+    """
+    for part in (conf_key,) if isinstance(conf_key, str) else conf_key:
+        if (schema := _dict_schema(schema)) is None:
+            return
+        markers = {
+            getattr(marker, "schema", marker): marker for marker in schema.schema
+        }
+        if part not in markers:
+            raise ValueError(f"{name}: config key {part!r} is not in the schema")
+        schema = schema.schema[markers[part]]
+
+
+def parent_ref(var: MockObj) -> MockObj:
+    """``var`` named from global scope, so a trigger argument cannot shadow it.
+
+    Also how a generated callback names its Automation.
+    """
+    return MockObj(f"::{var}", "->")
+
+
+async def _apply_parent(config: ConfigType, id_key: str = CONF_ID) -> str:
+    return str(parent_ref(await cg.get_variable(config[id_key])))
+
+
+def _apply_lambda_args(args: TemplateArgsType) -> TemplateArgsType:
+    # The generated function's parameters; a std::string arg is never copied.
+    return [
+        (cg.RawExpression(f"const std::remove_cvref_t<{cg.safe_exp(t)}> &"), arg)
+        for t, arg in args
+    ]
+
+
+def _apply_function(
+    id_: ID,
+    return_type: SafeExpType,
+    template_arg: cg.TemplateArguments,
+    lambda_args: TemplateArgsType,
+    statements: list[str],
+) -> MockObj:
+    """Emit the generated function and declare ``id_`` as the ``ApplyAction`` or
+    ``ApplyCondition`` templated on it, so ``play()`` calls it directly."""
+    fn = cg.static_function(
+        f"esphome__{id_.id}__fn", return_type, lambda_args, statements
+    )
+    return cg.new_Pvariable(id_, cg.TemplateArguments(fn, *template_arg))
+
+
+async def _render_values(
+    name: str,
+    target: str,
+    members: list[tuple[Any, Any, Any]],
+    values: list[Any],
+    config: ConfigType,
+    parent: str | None,
+    lambda_args: TemplateArgsType,
+    compare: bool = False,
+) -> list[str]:
+    """Render the argument text of one statement; every key must be present.
+
+    ``compare``: an inlined lambda expression is parenthesized so it binds as a whole
+    beside an operator.
+    """
+    if any(value is None for value in values):
+        keys = [key for key, _, _ in members]
+        raise EsphomeError(f"{name}: {target!r} needs all of {keys}")
+    exprs: list[str] = []
+    for (_, type_, const_fn), value in zip(members, values, strict=True):
+        if isinstance(value, Lambda):
+            if isinstance(type_, str):
+                type_ = cg.RawExpression(type_.format(parent=parent))
+            inner = await cg.process_lambda(value, lambda_args, return_type=type_)
+            expr = call_lambda(inner)
+            bare = compare and isinstance(expr, cg.RawExpression)
+            exprs.append(f"({expr})" if bare else str(expr))
+        elif isinstance(value, ID):
+            # Qualified like the parent, so a trigger arg named like the id cannot shadow it.
+            exprs.append(f"::{await cg.get_variable(value)}")
+        elif const_fn is not None:
+            exprs.append(const_fn(config, value))
+        else:
+            exprs.append(str(cg.safe_exp(value)))
+    return exprs
+
+
+def _apply_values(config: ConfigType, members: list[tuple[Any, Any, Any]]) -> list[Any]:
+    return [_config_lookup(config, key) for key, _, _ in members]
+
+
+def register_apply_action(
+    name: str,
+    schema: cv.Schema,
+    *fields: ApplyField | ApplyCall,
+    call: str | None = None,
+    id_key: str = CONF_ID,
+) -> None:
+    """Register an action that only forwards config values to its parent, with no C++ class.
+
+    See ``apply_action_builder``.
+    """
+    register_action(name, ApplyAction, schema, synchronous=True)(
+        apply_action_builder(name, schema, *fields, call=call, id_key=id_key)
+    )
+
+
+def apply_action_builder(
+    name: str,
+    schema: cv.Schema,
+    *fields: ApplyField | ApplyCall,
+    call: str | None = None,
+    id_key: str = CONF_ID,
+) -> Callable[..., Awaitable[MockObj]]:
+    """The builder ``register_apply_action`` registers, for a caller that wraps it.
+
+    Generates one static function with the parent (read from ``id_key``) and constants baked
+    in, lambdas called inline with the trigger args, and an ``ApplyAction`` templated on it.
+    A constant that is an id (``cv.use_id`` under ``cv.templatable``) is the object it names.
+    With ``call`` every statement targets the call object ``auto apply_call = parent->call()``,
+    and ``apply_call.perform()`` is appended.
+    """
+    # An action stores the value, so a std::string constant stays in flash on ESP8266.
+    statements_spec = [
+        (
+            c.target,
+            _default_std_string(c.members, flash_string),
+        )
+        for c in (f if isinstance(f, ApplyCall) else f.call() for f in fields)
+    ]
+    _check_key_in_schema(name, schema, id_key)
+    for _, members in statements_spec:
+        for conf_key, _, _ in members:
+            _check_key_in_schema(name, schema, conf_key)
+
+    async def builder(
+        config: ConfigType,
+        action_id: ID,
+        template_arg: cg.TemplateArguments,
+        args: TemplateArgsType,
+    ) -> MockObj:
+        parent = await _apply_parent(config, id_key)
+        lambda_args = _apply_lambda_args(args)
+        receiver = "apply_call." if call else f"{parent}->"
+        statements: list[str] = []
+        for target, members in statements_spec:
+            values = _apply_values(config, members)
+            if not _apply_call_active(members, values):
+                continue
+            exprs = await _render_values(
+                name, target, members, values, config, parent, lambda_args
+            )
+            statements.append(f"{receiver}{target.format(*exprs, parent=parent)};")
+        if call:
+            statements = [
+                f"auto apply_call = {parent}->{call}();",
+                *statements,
+                "apply_call.perform();",
+            ]
+        return _apply_function(
+            action_id, cg.void, template_arg, lambda_args, statements
+        )
+
+    return builder
+
+
+def _apply_call_active(members: list[tuple[Any, Any, Any]], values: list[Any]) -> bool:
+    """An ``ApplyCall`` is emitted unless it has keys and none of them is set."""
+    return not members or any(value is not None for value in values)
+
+
+async def _render_check(
+    name: str,
+    target: str,
+    members: list[tuple[Any, Any, Any]],
+    values: list[Any],
+    config: ConfigType,
+    parent: str | None,
+    lambda_args: TemplateArgsType,
+) -> str:
+    """Render a boolean check with ``values`` compared against config."""
+    exprs = await _render_values(
+        name, target, members, values, config, parent, lambda_args, compare=True
+    )
+    return target.format(*exprs, parent=parent)
+
+
+def register_apply_condition(
+    name: str, schema: cv.Schema, check: str | ApplyCall, id_key: str = CONF_ID
+) -> None:
+    """Register a condition that is one expression on its parent, with no C++ class.
+
+    ``check`` is applied to the parent: ``"is_playing()"`` becomes ``parent->is_playing()``; an
+    ``ApplyCall`` such as ``ApplyCall("state == {}", ((CONF_STATE, cg.bool_),))`` compares
+    against config values, all of which must be present. Write ``== false`` to negate.
+    A ``cg.std_string`` constant stays in flash on ESP8266 and supports only ``==`` or ``!=``
+    against a ``std::string`` or ``StringRef`` member; other string types are plain literals.
+    Generates one static predicate and an ``ApplyCondition`` templated on it.
+    """
+    call = check if isinstance(check, ApplyCall) else ApplyCall(check)
+    members = _default_std_string(call.members, flash_compare)
+    _check_key_in_schema(name, schema, id_key)
+    for conf_key, _, _ in members:
+        _check_key_in_schema(name, schema, conf_key)
+
+    async def builder(
+        config: ConfigType,
+        condition_id: ID,
+        template_arg: cg.TemplateArguments,
+        args: TemplateArgsType,
+    ) -> MockObj:
+        parent = await _apply_parent(config, id_key)
+        lambda_args = _apply_lambda_args(args)
+        values = _apply_values(config, members)
+        check = await _render_check(
+            name, call.target, members, values, config, parent, lambda_args
+        )
+        return _apply_function(
+            condition_id,
+            cg.bool_,
+            template_arg,
+            lambda_args,
+            [f"return {parent}->{check};"],
+        )
+
+    register_condition(name, ApplyCondition, schema)(builder)
+
 
 def validate_potentially_and_condition(value):
     if isinstance(value, list):
@@ -243,54 +740,22 @@ XorCondition = cg.esphome_ns.class_("XorCondition", Condition)
 
 
 @register_condition("and", AndCondition, validate_condition_list)
-async def and_condition_to_code(
-    config: ConfigType,
-    condition_id: ID,
-    template_arg: cg.TemplateArguments,
-    args: TemplateArgsType,
-) -> MockObj:
-    conditions = await build_condition_list(config, template_arg, args)
-    return cg.new_Pvariable(
-        condition_id, cg.TemplateArguments(len(conditions), *template_arg), conditions
-    )
-
-
-@register_condition("or", OrCondition, validate_condition_list)
-async def or_condition_to_code(
-    config: ConfigType,
-    condition_id: ID,
-    template_arg: cg.TemplateArguments,
-    args: TemplateArgsType,
-) -> MockObj:
-    conditions = await build_condition_list(config, template_arg, args)
-    return cg.new_Pvariable(
-        condition_id, cg.TemplateArguments(len(conditions), *template_arg), conditions
-    )
-
-
 @register_condition("all", AndCondition, validate_condition_list)
-async def all_condition_to_code(
-    config: ConfigType,
-    condition_id: ID,
-    template_arg: cg.TemplateArguments,
-    args: TemplateArgsType,
-) -> MockObj:
-    conditions = await build_condition_list(config, template_arg, args)
-    return cg.new_Pvariable(
-        condition_id, cg.TemplateArguments(len(conditions), *template_arg), conditions
-    )
-
-
+@register_condition("or", OrCondition, validate_condition_list)
 @register_condition("any", OrCondition, validate_condition_list)
-async def any_condition_to_code(
+@register_condition("xor", XorCondition, validate_condition_list)
+async def condition_group_to_code(
     config: ConfigType,
     condition_id: ID,
     template_arg: cg.TemplateArguments,
     args: TemplateArgsType,
 ) -> MockObj:
     conditions = await build_condition_list(config, template_arg, args)
+    # A group of one is that condition for and, or and xor; skip the wrapper.
+    if len(conditions) == 1:
+        return conditions[0]
     return cg.new_Pvariable(
-        condition_id, cg.TemplateArguments(len(conditions), *template_arg), conditions
+        condition_id, cg.TemplateArguments(len(conditions), *template_arg), *conditions
     )
 
 
@@ -305,19 +770,6 @@ async def not_condition_to_code(
     return cg.new_Pvariable(condition_id, template_arg, condition)
 
 
-@register_condition("xor", XorCondition, validate_condition_list)
-async def xor_condition_to_code(
-    config: ConfigType,
-    condition_id: ID,
-    template_arg: cg.TemplateArguments,
-    args: TemplateArgsType,
-) -> MockObj:
-    conditions = await build_condition_list(config, template_arg, args)
-    return cg.new_Pvariable(
-        condition_id, cg.TemplateArguments(len(conditions), *template_arg), conditions
-    )
-
-
 @register_condition("lambda", LambdaCondition, cv.returning_lambda)
 async def lambda_condition_to_code(
     config: ConfigType,
@@ -359,28 +811,15 @@ async def for_condition_to_code(
     return var
 
 
-@register_condition(
+register_apply_condition(
     "component.is_idle",
-    LambdaCondition,
     maybe_simple_id(
         {
             cv.Required(CONF_ID): cv.use_id(cg.Component),
         }
     ),
+    "is_idle()",
 )
-async def component_is_idle_condition_to_code(
-    config: ConfigType,
-    condition_id: ID,
-    template_arg: cg.TemplateArguments,
-    args: TemplateArgsType,
-) -> MockObj:
-    comp = await cg.get_variable(config[CONF_ID])
-    lambda_ = await cg.process_lambda(
-        Lambda(f"return {comp}->is_idle();"), args, return_type=bool
-    )
-    return new_lambda_pvariable(
-        condition_id, lambda_, StatelessLambdaCondition, template_arg
-    )
 
 
 @register_action(
@@ -432,11 +871,13 @@ async def if_action_to_code(
     condition = await build_condition(config[cond_conf], template_arg, args)
     var = cg.new_Pvariable(action_id, if_template_arg, condition)
     if CONF_THEN in config:
-        actions = await build_action_list(config[CONF_THEN], template_arg, args)
-        cg.add(var.add_then(actions))
+        for action in await build_action_list(config[CONF_THEN], template_arg, args):
+            cg.add(var.add_then_action(action))
+        cg.add(var.finish_then())
     if has_else:
-        actions = await build_action_list(config[CONF_ELSE], template_arg, args)
-        cg.add(var.add_else(actions))
+        for action in await build_action_list(config[CONF_ELSE], template_arg, args):
+            cg.add(var.add_else_action(action))
+        cg.add(var.finish_else())
     return var
 
 
@@ -459,8 +900,9 @@ async def while_action_to_code(
 ) -> MockObj:
     condition = await build_condition(config[CONF_CONDITION], template_arg, args)
     var = cg.new_Pvariable(action_id, template_arg, condition)
-    actions = await build_action_list(config[CONF_THEN], template_arg, args)
-    cg.add(var.add_then(actions))
+    for action in await build_action_list(config[CONF_THEN], template_arg, args):
+        cg.add(var.add_then_action(action))
+    cg.add(var.finish_then())
     return var
 
 
@@ -489,7 +931,9 @@ async def repeat_action_to_code(
         cg.TemplateArguments(cg.uint32, *template_arg.args),
         [(cg.uint32, "iteration"), *args],
     )
-    cg.add(var.add_then(actions))
+    for action in actions:
+        cg.add(var.add_then_action(action))
+    cg.add(var.finish_then())
     return var
 
 
@@ -534,44 +978,20 @@ async def lambda_action_to_code(
     return new_lambda_pvariable(action_id, lambda_, StatelessLambdaAction, template_arg)
 
 
-@register_action(
+register_simple_action(
     "component.update",
     UpdateComponentAction,
-    maybe_simple_id(
-        {
-            cv.Required(CONF_ID): cv.use_id(cg.PollingComponent),
-        }
-    ),
+    maybe_simple_id({cv.Required(CONF_ID): cv.use_id(cg.PollingComponent)}),
     synchronous=True,
 )
-async def component_update_action_to_code(
-    config: ConfigType,
-    action_id: ID,
-    template_arg: cg.TemplateArguments,
-    args: TemplateArgsType,
-) -> MockObj:
-    comp = await cg.get_variable(config[CONF_ID])
-    return cg.new_Pvariable(action_id, template_arg, comp)
 
 
-@register_action(
+register_simple_action(
     "component.suspend",
     SuspendComponentAction,
-    maybe_simple_id(
-        {
-            cv.Required(CONF_ID): cv.use_id(cg.PollingComponent),
-        }
-    ),
+    maybe_simple_id({cv.Required(CONF_ID): cv.use_id(cg.PollingComponent)}),
     synchronous=True,
 )
-async def component_suspend_action_to_code(
-    config: ConfigType,
-    action_id: ID,
-    template_arg: cg.TemplateArguments,
-    args: TemplateArgsType,
-) -> MockObj:
-    comp = await cg.get_variable(config[CONF_ID])
-    return cg.new_Pvariable(action_id, template_arg, comp)
 
 
 @register_action(
@@ -649,30 +1069,141 @@ def has_non_synchronous_actions(actions: ConfigType) -> bool:
     Non-synchronous actions (delay, wait_until, script.wait, etc.) store
     trigger args for later execution, making non-owning types like StringRef
     unsafe.
+
+    Accepts an action list, a single action, or an automation. Only the keys of
+    action entries are looked up as action names; the options of an action are
+    only searched for nested action lists (``then:``, ``else:``, ...), so an
+    option that shares a name with an action, such as the ``delay:`` option of
+    ``speaker_source.set_playlist_delay``, is not mistaken for that action.
     """
     if isinstance(actions, list):
         return any(has_non_synchronous_actions(item) for item in actions)
     if isinstance(actions, dict):
-        for key in actions:
+        for key, options in actions.items():
             if key in ACTION_REGISTRY and not ACTION_REGISTRY[key].synchronous:
                 return True
-        return any(
-            has_non_synchronous_actions(v)
-            for v in actions.values()
-            if isinstance(v, (list, dict))
-        )
+            if _options_have_non_synchronous_actions(options):
+                return True
     return False
 
 
+def _options_have_non_synchronous_actions(options: Any) -> bool:
+    """Search an action's options for nested action lists, without treating option keys as actions."""
+    if isinstance(options, list):
+        return has_non_synchronous_actions(options)
+    if isinstance(options, dict):
+        return any(_options_have_non_synchronous_actions(v) for v in options.values())
+    return False
+
+
+def validate_no_deferring_actions(
+    context: str, data: str = "the trigger data"
+) -> Callable[[ConfigType], ConfigType]:
+    """Return a validator that rejects deferring actions in a validated automation.
+
+    Use it for triggers whose arguments only live while the callback runs (views into a reused
+    buffer, a JSON document freed on return, ...): deferring actions keep the arguments for later.
+
+    :param context: Where the automation is, for the error message, e.g. ``"on_json and on_data"``.
+    :param data: What is only valid while the automation runs, for the error message.
+    """
+
+    def validator(value: ConfigType) -> ConfigType:
+        if has_non_synchronous_actions(value):
+            raise cv.Invalid(
+                f"Deferring actions (delay, wait_until, script.wait, ...) are not allowed in "
+                f"{context}: {data} is only valid while the automation runs. Copy what you need "
+                "into globals first, then defer in a separate script or automation."
+            )
+        return value
+
+    return validator
+
+
+def validate_synchronous_automation(
+    context: str,
+    data: str = "the trigger data",
+    extra_schema=None,
+    extra_validators=None,
+    single: bool = False,
+) -> cv.All:
+    """``validate_automation()`` that also rejects deferring actions.
+
+    See ``validate_no_deferring_actions()`` for ``context`` and ``data``; the other parameters are
+    passed to ``validate_automation()``.
+    """
+    return cv.All(
+        validate_automation(
+            extra_schema=extra_schema,
+            extra_validators=extra_validators,
+            single=single,
+        ),
+        validate_no_deferring_actions(context, data),
+    )
+
+
+async def _new_automation(
+    args: TemplateArgsType, config: ConfigType, *ctor_args: MockObj
+) -> MockObj:
+    """Create the Automation for ``config`` with its actions."""
+    templ = cg.TemplateArguments(*(arg[0] for arg in args))
+    obj = cg.new_Pvariable(config[CONF_AUTOMATION_ID], templ, *ctor_args)
+    actions = await build_action_list(config[CONF_THEN], templ, args)
+    for action in actions:
+        cg.add(obj.add_action(action))
+    return obj
+
+
 async def build_automation(
     trigger: MockObj, args: TemplateArgsType, config: ConfigType
 ) -> MockObj:
-    arg_types = [arg[0] for arg in args]
-    templ = cg.TemplateArguments(*arg_types)
-    obj = cg.new_Pvariable(config[CONF_AUTOMATION_ID], templ, trigger)
-    actions = await build_action_list(config[CONF_THEN], templ, args)
-    cg.add(obj.add_actions(actions))
-    return obj
+    return await _new_automation(args, config, trigger)
+
+
+async def build_trigger_callback(
+    args: TemplateArgsType,
+    config: ConfigType,
+    params: TemplateArgsType,
+    forward: Sequence[str | Expression] | None = None,
+    when: str | ApplyCall | None = None,
+    parent: MockObj | None = None,
+) -> LambdaExpression:
+    """Build the Automation for ``config`` and return a stateless callback that triggers it.
+
+    ``params`` are the parent callback's parameters, ``forward`` the expressions passed to
+    ``trigger()`` (default: the parameter names; write the parent as ``parent_ref(var)``),
+    ``when`` a filter the callback returns early on, skipped like any ``ApplyCall`` when none
+    of its keys is set. ``when`` may name ``parent`` as ``{parent}``, e.g.
+    ``"{parent}->is_fully_open()"``.
+    """
+    members: list[tuple[Any, Any, Any]] = []
+    if when is not None:
+        call = when if isinstance(when, ApplyCall) else ApplyCall(when)
+        members = call.members
+        if parent is None and call.names_parent:
+            raise ValueError(f"trigger filter {call.target!r} names {{parent}}")
+    obj = await _new_automation(args, config)
+    lambda_args = _apply_lambda_args(params)
+    statements: list[str] = []
+    if when is not None:
+        values = _apply_values(config, members)
+        if _apply_call_active(members, values):
+            check = await _render_check(
+                "trigger filter",
+                call.target,
+                members,
+                values,
+                config,
+                None if parent is None else str(parent_ref(parent)),
+                lambda_args,
+            )
+            statements.append(f"if (!({check}))\n  return;")
+    if forward is None:
+        forward = [name for _, name in params]
+    statements.append(f"{parent_ref(obj)}->trigger({', '.join(map(str, forward))});")
+    return LambdaExpression(
+        ["\n".join(statements)], lambda_args, capture="", return_type=cg.void
+    )
 
 
 async def build_callback_automation(
@@ -681,6 +1212,9 @@ async def build_callback_automation(
     args: TemplateArgsType,
     config: ConfigType,
     forwarder: MockObj | MockObjClass | None = None,
+    params: TemplateArgsType | None = None,
+    forward: Sequence[str | Expression] | None = None,
+    when: str | ApplyCall | None = None,
 ) -> None:
     """Build an Automation and register it as a callback on the parent.
 
@@ -692,6 +1226,9 @@ async def build_callback_automation(
     pointer-sized (single Automation* field) to fit inline in Callback::ctx_
     and avoid heap allocation.
 
+    With ``params``, ``forward`` or ``when`` the callback is instead the stateless
+    lambda of ``build_trigger_callback``; ``forwarder`` cannot be combined with them.
+
     :param parent: The component object (e.g., button, sensor).
     :param callback_method: Name of the callback method (e.g., "add_on_press_callback").
     :param args: Automation template args as list of (type, name) tuples.
@@ -700,22 +1237,56 @@ async def build_callback_automation(
         TriggerForwarder<Ts...>. Pass any struct type whose aggregate init takes
         a single Automation pointer (e.g., TriggerOnTrueForwarder).
     """
-    arg_types = [arg[0] for arg in args]
-    templ = cg.TemplateArguments(*arg_types)
-    obj = cg.new_Pvariable(config[CONF_AUTOMATION_ID], templ)
-    actions = await build_action_list(config[CONF_THEN], templ, args)
-    cg.add(obj.add_actions(actions))
+    if params is not None or forward is not None or when is not None:
+        if forwarder is not None:
+            raise ValueError(
+                "forwarder cannot be combined with params, forward or when"
+            )
+        callback = await build_trigger_callback(
+            args, config, args if params is None else params, forward, when, parent
+        )
+        cg.add(getattr(parent, callback_method)(callback))
+        return
+    obj = await _new_automation(args, config)
     # Use template forwarder structs for deduplication. The compiler generates
     # one operator() per forwarder type; different automation pointers are just
     # data in the struct.
     if forwarder is None:
-        forwarder = TriggerForwarder.template(templ)
+        forwarder = TriggerForwarder.template(*(arg[0] for arg in args))
     # RawExpression for aggregate init — both forwarder and obj are codegen
     # MockObjs (not user input), and there's no Expression type for positional
     # aggregate initialization (StructInitializer uses named fields).
     cg.add(getattr(parent, callback_method)(cg.RawExpression(f"{forwarder}{{{obj}}}")))
 
 
+async def build_parent_callback_automation(
+    parent: MockObj, callback_method: str, arg: tuple[Any, str], config: ConfigType
+) -> None:
+    """Register an Automation that receives ``parent`` from a callback that carries nothing.
+
+    ``arg`` is the automation's ``(type, name)``, e.g. ``(Fan.operator("ptr"), "x")``.
+    """
+    await build_callback_automation(
+        parent, callback_method, [arg], config, params=[], forward=[parent_ref(parent)]
+    )
+
+
+async def build_trigger_automations(
+    parent: MockObj | None,
+    config: ConfigType,
+    entries: tuple[tuple[str, TemplateArgsType], ...],
+) -> None:
+    """Instantiate each entry's Trigger class, with ``parent`` when given, and build its automations.
+
+    ``entries`` are ``(conf_key, args)`` pairs; the class comes from the entry's ``CONF_TRIGGER_ID``.
+    """
+    ctor_args = () if parent is None else (parent,)
+    for conf_key, args in entries:
+        for conf in config.get(conf_key, []):
+            trigger = cg.new_Pvariable(conf[CONF_TRIGGER_ID], *ctor_args)
+            await build_automation(trigger, args, conf)
+
+
 @dataclass(frozen=True, slots=True)
 class CallbackAutomation:
     """A single callback automation entry for build_callback_automations."""
@@ -724,6 +1295,9 @@ class CallbackAutomation:
     callback_method: str
     args: TemplateArgsType = field(default_factory=list)
     forwarder: MockObj | MockObjClass | None = None
+    params: TemplateArgsType | None = None
+    forward: Sequence[str | Expression] | None = None
+    when: str | ApplyCall | None = None
 
 
 async def build_callback_automations(
@@ -745,4 +1319,7 @@ async def build_callback_automations(
                 entry.args,
                 conf,
                 forwarder=entry.forwarder,
+                params=entry.params,
+                forward=entry.forward,
+                when=entry.when,
             )

esphome/build_gen/espidf.py

@@ -3,7 +3,10 @@
 import json
 import logging
 from pathlib import Path
+import re
+import textwrap
 
+from esphome.build_helpers import pch
 from esphome.components.esp32 import (
     get_esp32_variant,
     get_excluded_builtin_components,
@@ -18,7 +21,7 @@ from esphome.framework_helpers import (
     get_project_cxx_compile_flags,
     get_project_link_flags,
 )
-from esphome.helpers import mkdir_p, write_file_if_changed
+from esphome.helpers import get_bool_env, mkdir_p, write_file_if_changed
 
 _LOGGER = logging.getLogger(__name__)
 
@@ -33,6 +36,83 @@ list(FILTER esphome_cxx_compile_options EXCLUDE REGEX "^-std=")
 list(APPEND esphome_cxx_compile_options "-std={standard}")
 idf_build_set_property(CXX_COMPILE_OPTIONS "${{esphome_cxx_compile_options}}")"""
 
+# Drops the app archive from ldgen's inputs so app-only edits skip the
+# sections.ld regeneration. Safe: no mapping fragment references it
+# (run_compile re-checks each build). Filters only the top-level call;
+# the prior definition stays reachable with an underscore prefix.
+_LDGEN_OVERRIDE = """\
+if(COMMAND __ldgen_get_lib_deps_of_target)
+    set_property(GLOBAL PROPERTY ESPHOME_LDGEN_ARMED 1)
+    function(__ldgen_get_lib_deps_of_target target out_list_var)
+        if(NOT COMMAND ___ldgen_get_lib_deps_of_target)
+            message(FATAL_ERROR "ESPHome ldgen override lost the original "
+                "implementation; set ESPHOME_LDGEN_FULL_DEPS=1 and rebuild.")
+        endif()
+        ___ldgen_get_lib_deps_of_target(${target} ${out_list_var})
+        if(out_list_var STREQUAL "ldgen_libraries")
+            set_property(GLOBAL PROPERTY ESPHOME_LDGEN_FILTERED 1)
+            list(LENGTH ${out_list_var} esphome_ldgen_before)
+            list(REMOVE_ITEM ${out_list_var} idf::src __idf_src)
+            list(LENGTH ${out_list_var} esphome_ldgen_after)
+            if(esphome_ldgen_before EQUAL esphome_ldgen_after)
+                message(@SEVERITY@ "ESPHome ldgen app archive exclusion matched "
+                    "nothing; app edits will regenerate sections.ld.")
+            endif()
+        endif()
+        set(${out_list_var} "${${out_list_var}}" PARENT_SCOPE)
+    endfunction()
+else()
+    message(@MISSING@ "ESPHome ldgen override target not found; "
+        "app edits will regenerate sections.ld.")
+endif()"""
+
+# lwip sources that compile to empty objects with the option off (their own
+# #if guard). (option, regex valid for both Python and CMake); a source is
+# only dropped when its option is defined and off, so a renamed option
+# keeps it.
+LWIP_EMPTY_SOURCES: tuple[tuple[str, str], ...] = (
+    ("CONFIG_LWIP_PPP_SUPPORT", "/netif/ppp/"),
+    ("CONFIG_LWIP_IPV6", "/core/ipv6/"),
+    ("CONFIG_LWIP_AUTOIP", "/core/ipv4/autoip[.]c$"),
+    ("CONFIG_LWIP_STATS", "/core/stats[.]c$"),
+)
+# Drift guard only: keep every lwip source.
+LWIP_FULL_SOURCES_ENV = "ESPHOME_LWIP_FULL_SOURCES"
+
+# Drops the empty objects after project(), once the lwip target exists.
+_LWIP_EMPTY_SOURCES_FILTER = f"""\
+idf_build_get_property(esphome_build_components BUILD_COMPONENTS)
+if(lwip IN_LIST esphome_build_components AND NOT DEFINED ENV{{{LWIP_FULL_SOURCES_ENV}}})
+    idf_component_get_property(esphome_lwip_lib lwip COMPONENT_LIB)
+    get_target_property(esphome_lwip_srcs ${{esphome_lwip_lib}} SOURCES)
+@FILTERS@
+    set_property(TARGET ${{esphome_lwip_lib}} PROPERTY SOURCES ${{esphome_lwip_srcs}})
+endif()"""
+
+
+def lwip_empty_source_gate(option: str, regex: str) -> str:
+    return (
+        f"    if(DEFINED {option} AND NOT {option})\n"
+        f'        list(FILTER esphome_lwip_srcs EXCLUDE REGEX "{regex}")\n'
+        "    endif()"
+    )
+
+
+def _lwip_empty_sources_filter() -> str:
+    gates = "\n".join(lwip_empty_source_gate(*entry) for entry in LWIP_EMPTY_SOURCES)
+    return _LWIP_EMPTY_SOURCES_FILTER.replace("@FILTERS@", gates)
+
+
+# Runs after project() so the walk has happened; catches the remaining
+# silent path where the top-level out-var was renamed.
+_LDGEN_OVERRIDE_CHECK = """\
+get_property(esphome_ldgen_armed GLOBAL PROPERTY ESPHOME_LDGEN_ARMED)
+get_property(esphome_ldgen_filtered GLOBAL PROPERTY ESPHOME_LDGEN_FILTERED)
+if(esphome_ldgen_armed AND NOT esphome_ldgen_filtered)
+    message(@SEVERITY@ "ESPHome ldgen override never filtered the app "
+        "archive; app edits will regenerate sections.ld.")
+endif()"""
+
 
 def get_available_components() -> list[str] | None:
     """List the built-in ESP-IDF components from ``project_description.json``.
@@ -78,6 +158,114 @@ def _cmake_quote(value: str) -> str:
     return f'"{escaped}"'
 
 
+# CONFIG_APP_BUILD_BOOTLOADER is hidden and force-selected, so it can only be
+# cleared at the CMake level (the same state IDF's RAM-app build type uses).
+# The macro is IDF's __build_process_project_includes plus a few added lines;
+# the flag is ignored and the bootloader builds as usual if IDF changes it.
+IDF_BOOTLOADER_OVERRIDE = """\
+# ESPHome bootloader skip switch; see esphome/espidf/toolchain.py.
+if(ESPHOME_SKIP_BOOTLOADER)
+    macro(__build_process_project_includes)
+        idf_build_get_property(sdkconfig_cmake SDKCONFIG_CMAKE)
+        include(${sdkconfig_cmake})
+        set(CONFIG_APP_BUILD_BOOTLOADER "")
+        # bt's CMakeLists reads the lowercase idf_target that the (now
+        # skipped) bootloader project_include leaks; keep it defined, or
+        # its empty TARGET_SRC_NAME sends file(GLOB_RECURSE) across /.
+        idf_build_get_property(idf_target IDF_TARGET)
+        # partition_table's V1 ECDSA signing reads this key, which the
+        # skipped bootloader project_include also sets.
+        get_filename_component(SECURE_BOOT_SIGNING_KEY "${CONFIG_SECURE_BOOT_SIGNING_KEY}" ABSOLUTE BASE_DIR "${project_dir}")
+        idf_build_get_property(build_properties __BUILD_PROPERTIES)
+        foreach(build_property ${build_properties})
+            idf_build_get_property(val ${build_property})
+            set(${build_property} "${val}")
+        endforeach()
+        idf_build_get_property(build_component_targets __BUILD_COMPONENT_TARGETS)
+        foreach(component_target ${build_component_targets})
+            __component_get_property(dir ${component_target} COMPONENT_DIR)
+            __component_get_property(_name ${component_target} COMPONENT_NAME)
+            set(COMPONENT_NAME ${_name})
+            set(COMPONENT_DIR ${dir})
+            set(COMPONENT_PATH ${dir})
+            if(EXISTS ${COMPONENT_DIR}/project_include.cmake)
+                include(${COMPONENT_DIR}/project_include.cmake)
+            endif()
+        endforeach()
+    endmacro()
+endif()
+"""
+
+# The lines the override adds to IDF's macro; idf_macro_matches() below
+# strips them before comparing with the live macro.
+BOOTLOADER_OVERRIDE_ADDED_LINES = (
+    'set(CONFIG_APP_BUILD_BOOTLOADER "")',
+    "idf_build_get_property(idf_target IDF_TARGET)",
+    (
+        "get_filename_component(SECURE_BOOT_SIGNING_KEY"
+        ' "${CONFIG_SECURE_BOOT_SIGNING_KEY}" ABSOLUTE BASE_DIR "${project_dir}")'
+    ),
+)
+
+_MACRO = re.compile(
+    r"macro\(__build_process_project_includes\)(.*?)endmacro\(\)", re.DOTALL
+)
+
+
+def _normalized_macro(text: str) -> list[str] | None:
+    """The macro body as comment-free, whitespace-collapsed lines."""
+    if (match := _MACRO.search(text)) is None:
+        return None
+    return [
+        re.sub(r"\s+", " ", line)
+        for raw in match.group(1).splitlines()
+        if (line := raw.split("#", 1)[0].strip())
+    ]
+
+
+_EXPECTED_MACRO = [
+    line
+    for line in _normalized_macro(IDF_BOOTLOADER_OVERRIDE)
+    if line not in BOOTLOADER_OVERRIDE_ADDED_LINES
+]
+
+
+def idf_macro_matches(idf_path: Path) -> bool:
+    """Whether IDF's macro still matches the copy the override replays."""
+    build_cmake = idf_path / "tools" / "cmake" / "build.cmake"
+    live = _normalized_macro(build_cmake.read_text(encoding="utf-8"))
+    return live == _EXPECTED_MACRO
+
+
+def _cmake_args_for_write(minimal: bool) -> dict[str, str]:
+    """The cg.add_cmake_arg() values for one project write.
+
+    The discovery (minimal) write lets every built-in component register so
+    the list it finds, cached per target, serves any exclusion set; the full
+    write drops the excluded names from that list and passes all of them to
+    IDF. Names outside the framework's components directory (managed
+    components) stay excluded on the discovery write too, since the list
+    only holds built-in ones. A checkout supplied through IDF_PATH caches
+    nothing, so its discovery keeps every exclusion.
+    """
+    from esphome.espidf.toolchain import _esphome_manages_idf, _get_idf_path
+
+    args = dict(CORE.cmake_args)
+    if (
+        not minimal
+        or not _esphome_manages_idf()
+        or not (excluded := args.get("EXCLUDE_COMPONENTS"))
+    ):
+        return args
+    root = _get_idf_path() / "components"
+    outside = [name for name in excluded.split(";") if not (root / name).is_dir()]
+    if outside:
+        args["EXCLUDE_COMPONENTS"] = ";".join(outside)
+    else:
+        del args["EXCLUDE_COMPONENTS"]
+    return args
+
+
 def get_project_cmakelists(
     minimal: bool = False, builtin_components: list[str] | None = None
 ) -> str:
@@ -90,9 +278,10 @@ def get_project_cmakelists(
     """
     idf_target = variant_to_idf_target(get_esp32_variant())
 
-    # esp_idf_size 2.x (bundled with IDF >=6.0) made NG the default and
-    # removed the --ng flag; on 1.x (IDF 5.5) --ng is required to get
-    # --format=raw because the legacy mode doesn't support it.
+    # esp_idf_size 2.x (IDF >=6.0) made NG the default and removed --ng;
+    # 1.x (IDF 5.5) needs --ng for --format=json2. 1.x json2 also lacks
+    # total_size, hence the ELF fallback in espidf/size_summary.py; both
+    # go away together when 1.x support is dropped.
     size_ng_flag = "--ng" if idf_version() < cv.Version(6, 0, 0) else ""
 
     # Project-wide compile options: -D defines and -W warning flags (skip
@@ -122,13 +311,28 @@ def get_project_cmakelists(
         else ""
     )
 
+    # Stops the ~3s sections.ld regeneration on app-only edits; see
+    # _LDGEN_OVERRIDE. ESPHOME_LDGEN_FULL_DEPS=1 restores stock behavior;
+    # ESPHOME_LDGEN_STRICT=1 (CI) fails the configure when an IDF bump
+    # breaks the override instead of degrading to stock deps.
+    if get_bool_env("ESPHOME_LDGEN_FULL_DEPS"):
+        ldgen_override = ""
+        ldgen_override_check = ""
+    else:
+        strict = get_bool_env("ESPHOME_LDGEN_STRICT")
+        severity = "FATAL_ERROR" if strict else "WARNING"
+        missing = "FATAL_ERROR" if strict else "STATUS"
+        ldgen_override = _LDGEN_OVERRIDE.replace("@SEVERITY@", severity).replace(
+            "@MISSING@", missing
+        )
+        ldgen_override_check = _LDGEN_OVERRIDE_CHECK.replace("@SEVERITY@", severity)
+
     # CMake variables registered via cg.add_cmake_arg(). Emitted before
     # include(project.cmake) so values like EXCLUDE_COMPONENTS are already
-    # set when project.cmake seeds the component list, and on minimal
-    # (discovery) writes too so excluded components never register.
+    # set when project.cmake seeds the component list.
     cmake_args = "\n".join(
         f"set({name} {_cmake_quote(value)})"
-        for name, value in sorted(CORE.cmake_args.items())
+        for name, value in sorted(_cmake_args_for_write(minimal).items())
     )
 
     # Per-project list exposed as a CMake variable so converted PIO libs
@@ -151,10 +355,10 @@ def get_project_cmakelists(
     # component's REQUIRES including real IDF components). Referenced by
     # src/CMakeLists and by each converted PIO lib's CMakeLists. Skipped
     # on minimal writes because project_description.json may be stale.
-    # Excluded components are dropped here as well: a stale
-    # project_description.json from a build without exclusions may still
-    # list them, and requiring an excluded component pulls it back into
-    # the build (IDF requirement expansion overrides EXCLUDE_COMPONENTS).
+    # Excluded components are dropped here as well: the discovery list
+    # (cached per target) still holds them, and requiring an excluded
+    # component pulls it back into the build (IDF requirement expansion
+    # overrides EXCLUDE_COMPONENTS).
     # Derived from the EXCLUDE_COMPONENTS cmake arg emitted above so the
     # two can never disagree within one generated file.
     builtin_components_property = (
@@ -199,6 +403,9 @@ set(EXTRA_COMPONENT_DIRS ${{CMAKE_SOURCE_DIR}}/src)
 
 include($ENV{{IDF_PATH}}/tools/cmake/project.cmake)
 
+{IDF_BOOTLOADER_OVERRIDE}
+{ldgen_override}
+
 {cpp_standard_options}
 
 {cxx_compile_options}
@@ -211,12 +418,31 @@ include($ENV{{IDF_PATH}}/tools/cmake/project.cmake)
 
 project({CORE.name})
 
-# Emit raw JSON size data for ESPHome to read post-build.
+{ldgen_override_check}
+
+{_lwip_empty_sources_filter()}
+
+# Emit per-memory-type JSON size data for ESPHome to read post-build.
+# json2 stays small; raw dumps every symbol (~2s on a large map) and
+# this command runs inside the link edge, blocking everything downstream.
+# IDF's size target depends on the map, so ninja has to know the link
+# writes it for size to build in the same run as all. IDF declares the
+# map itself since espressif/esp-idf#19201 (and in cmakev2), and ninja
+# rejects two rules for one output, so declare it here only when IDF has
+# not. add_custom_command(OUTPUT) marks its outputs GENERATED.
+get_source_file_property(esphome_map_declared
+    ${{CMAKE_BINARY_DIR}}/${{CMAKE_PROJECT_NAME}}.map GENERATED)
+if(esphome_map_declared)
+    set(esphome_map_byproducts)
+else()
+    set(esphome_map_byproducts BYPRODUCTS ${{CMAKE_BINARY_DIR}}/${{CMAKE_PROJECT_NAME}}.map)
+endif()
 add_custom_command(
     TARGET ${{CMAKE_PROJECT_NAME}}.elf POST_BUILD
-    COMMAND ${{PYTHON}} -m esp_idf_size {size_ng_flag} --format=raw
+    COMMAND ${{PYTHON}} -m esp_idf_size {size_ng_flag} --format=json2
             -o ${{CMAKE_BINARY_DIR}}/esp_idf_size.json
             ${{CMAKE_PROJECT_NAME}}.map
+    ${{esphome_map_byproducts}}
     WORKING_DIRECTORY ${{CMAKE_BINARY_DIR}}
     VERBATIM
 )
@@ -279,9 +505,72 @@ idf_component_register(
 target_link_options(${{COMPONENT_LIB}} PUBLIC
     {link_opts_str}
 )
+{_pch_cmake_block()}"""
+
+
+# Where CMake puts the .gch of the src component; ccache reads the checksum
+# next to it in place of the .gch
+_PCH_SUM_PATH = "build/esp-idf/src/CMakeFiles/__idf_src.dir/cmake_pch.hxx.gch.sum"
+# Where the Windows gate records its choice
+_PCH_CHOICE_VAR = "ESPHOME_PCH"
+
+
+def _pch_cmake_block() -> str:
+    """The CMake block that precompiles the core headers for the C++ sources
+    of the src component; empty when disabled."""
+    if not pch.pch_enabled():
+        return ""
+    headers = "\n".join(
+        f'    "$<$<COMPILE_LANGUAGE:CXX>:${{CMAKE_CURRENT_SOURCE_DIR}}/{header}>"'
+        for header in pch.PCH_DEFAULT_HEADERS
+    )
+    block = f"""target_precompile_headers(${{COMPONENT_LIB}} PRIVATE
+{headers}
+)"""
+    if not pch.pch_needs_gcc_check():
+        return f"\n# ESPHome precompiled header\n{block}\n"
+    # Before the first configure only CMake knows the compiler version
+    return f"""
+# ESPHome precompiled header, unless GCC bug 14940 keeps it from loading
+if(CMAKE_CXX_COMPILER_ID STREQUAL "GNU" AND ({pch.PCH_WINDOWS_CMAKE_OLD_GCC}))
+  message(STATUS "ESPHome: GCC ${{CMAKE_CXX_COMPILER_VERSION}} cannot load a precompiled header on Windows; compiling without it")
+  set({_PCH_CHOICE_VAR} OFF CACHE BOOL "ESPHome precompiled header in use" FORCE)
+else()
+  set({_PCH_CHOICE_VAR} ON CACHE BOOL "ESPHome precompiled header in use" FORCE)
+{textwrap.indent(block, "  ")}
+endif()
 """
 
 
+def _read_if_exists(path: Path) -> str:
+    return path.read_text(encoding="utf-8") if path.is_file() else ""
+
+
+def write_pch_checksum() -> None:
+    """Write the checksum ccache uses in place of the .gch: the core headers,
+    the framework version, the sdkconfig and the managed component versions."""
+    from esphome.espidf.toolchain import get_cmake_cache_value
+
+    if not pch.pch_enabled():
+        return
+    # The gate's choice, cached by configure
+    if pch.pch_needs_gcc_check() and get_cmake_cache_value(_PCH_CHOICE_VAR) != "ON":
+        return
+    pch.log_pch_in_use()
+    checksum = pch.pch_checksum(
+        CORE.relative_src_path(),
+        pch.PCH_DEFAULT_HEADERS,
+        (
+            str(idf_version()),
+            _read_if_exists(CORE.relative_build_path(f"sdkconfig.{CORE.name}")),
+            _read_if_exists(CORE.relative_build_path("dependencies.lock")),
+        ),
+    )
+    path = CORE.relative_build_path(_PCH_SUM_PATH)
+    path.parent.mkdir(parents=True, exist_ok=True)
+    write_file_if_changed(path, checksum + "\n")
+
+
 def write_project(
     minimal: bool = False, builtin_components: list[str] | None = None
 ) -> None:
@@ -302,10 +591,9 @@ def write_project(
     )
 
     # Snapshot the exclusion set so has_outdated_files() can trigger a
-    # discovery reconfigure when it changes. Excluded components never
-    # register in project_description.json, so re-including one (e.g. a
-    # config gains mqtt) requires a fresh discovery pass before the
-    # ESPHOME_PROJECT_BUILTIN_COMPONENTS property can list it.
+    # reconfigure when it changes. The ESPHOME_PROJECT_BUILTIN_COMPONENTS
+    # property drops excluded names, so re-including one (e.g. a config
+    # gains mqtt) needs a reconfigure to rebuild it from the discovery list.
     write_file_if_changed(
         CORE.relative_build_path("exclude_components.esphomeinternal"),
         ";".join(get_excluded_builtin_components()),

esphome/build_gen/host.py

@@ -0,0 +1,303 @@
+"""Native ninja build generator for the host platform.
+
+Emits ``build.ninja`` under ``.pioenvs/<name>/``: every source in the
+generated ``src/`` tree plus the resolved registry libraries compiles with
+the machine's compiler and links into ``program``, the name PlatformIO's
+native platform produced. Build flags route the way SCons's ``ParseFlags``
+did under PlatformIO: ``-D``/``-I``/``-std=``/``-W`` shapes reach the
+compile lines only, ``-l``/``-L``/``-Wl,`` the link line only, everything
+else both.
+"""
+
+from __future__ import annotations
+
+from collections.abc import Iterable
+import logging
+import os
+from pathlib import Path
+import subprocess
+import sys
+from typing import TYPE_CHECKING
+
+from esphome.build_helpers.ninja import escape as _e, quote_path as _q, shell_token
+from esphome.build_helpers.ninja_gen import (
+    PATH_ARG_FLAGS,
+    Flag,
+    anchor_path_flag,
+    ar_rule_lines,
+    collect_sources,
+    compile_edges,
+    compile_rule_lines,
+    library_edges,
+    pch_edges,
+    pch_rule_lines,
+    tool_lines,
+)
+from esphome.build_helpers.pch import PCH_DEFAULT_HEADERS, pch_enabled
+from esphome.core import CORE, EsphomeError
+from esphome.framework_helpers import get_project_cxx_compile_flags
+from esphome.helpers import mkdir_p, write_file_if_changed
+from esphome.host.toolchain import PROGRAM_NAME, HostCompilers, find_tool, get_build_dir
+from esphome.platformio.library import lex_build_flags
+
+if TYPE_CHECKING:
+    from esphome.arduino.library import ArduinoLibrary
+
+_LOGGER = logging.getLogger(__name__)
+
+# The PlatformIO platform the host built under; registry manifests declare
+# compatibility against it, as lib_compat_mode=strict checked before
+PIO_PLATFORM = "native"
+# Namespaces the shared library download cache (pio_components/host/)
+LIBRARY_CACHE_KEY = "host"
+
+# Flag shapes that only the compiler understands; dropped from the link line
+_COMPILE_ONLY_PREFIXES = ("-D", "-U", "-I", "-std=", "-W", *PATH_ARG_FLAGS)
+# Flag shapes that only the linker consumes; inert on a -c compile line
+_LINK_ONLY_PREFIXES = ("-l", "-L", "-Wl,")
+# Link-only flags whose argument is the next token
+_LINK_ONLY_ARG_FLAGS = ("-framework", "-Xlinker", "-z")
+
+
+def parse_flags(entries: Iterable[str], owner: str) -> list[Flag]:
+    """Lex build flag entries into flags, each with its argument.
+
+    Entries are a set, so their order is not the user's: a flag and the
+    argument it takes as the next token must share one entry.
+    """
+    flags: list[Flag] = []
+    for entry in entries:
+        it = iter(lex_build_flags(entry, owner))
+        for tok in it:
+            if tok not in PATH_ARG_FLAGS and tok not in _LINK_ONLY_ARG_FLAGS:
+                flags.append((tok,))
+                continue
+            arg = next(it, None)
+            # A path never starts with "-"; that is the next flag
+            if arg is None or (tok in PATH_ARG_FLAGS and arg.startswith("-")):
+                raise EsphomeError(
+                    f"{owner} build flags have '{tok}' with no argument; write "
+                    f"the flag and its argument as one entry"
+                )
+            flags.append((tok, arg))
+    return flags
+
+
+def split_flags(flags: list[Flag]) -> tuple[list[Flag], list[Flag]]:
+    """Route build flags to the compile and link lines."""
+    compile_flags: list[Flag] = []
+    link_flags: list[Flag] = []
+    for flag in flags:
+        name = flag[0]
+        if len(flag) > 1:
+            (compile_flags if name in PATH_ARG_FLAGS else link_flags).append(flag)
+        elif name.startswith(_LINK_ONLY_PREFIXES):
+            # Checked before the compile prefixes: -Wl, would match -W
+            link_flags.append(flag)
+        elif name.startswith(_COMPILE_ONLY_PREFIXES):
+            compile_flags.append(flag)
+        else:
+            # -g, -O, -f*, -m*, -pthread, --coverage: both lines, as SCons
+            compile_flags.append(flag)
+            link_flags.append(flag)
+    return compile_flags, link_flags
+
+
+def _is_std(flag: Flag) -> bool:
+    return flag[0].startswith("-std=")
+
+
+def _is_cxx_std(flag: Flag) -> bool:
+    return _is_std(flag) and "++" in flag[0]
+
+
+def _anchored_flags(entries: Iterable[str], owner: str) -> list[Flag]:
+    build_path = Path(CORE.build_path)
+    return [
+        anchor_path_flag(flag, build_path)
+        for flag in parse_flags(sorted(entries), owner)
+    ]
+
+
+def _flag_lists() -> tuple[list[str], list[str], list[str]]:
+    """The C, C++, and link flag lists (raw tokens), build_unflags applied.
+
+    ``cg.set_cpp_standard`` wins over any ``-std=`` in the build flags for
+    C++ compiles, as PlatformIO's unflag of every other standard did; C
+    compiles never see a C++ standard.
+    """
+    # The funnel warns and drops empty glued arguments (-D "") itself
+    compile_flags, link_flags = split_flags(
+        _anchored_flags(CORE.build_flags, "esphome")
+    )
+    cflags = [f for f in compile_flags if not _is_cxx_std(f)]
+    cxx_std = CORE.cpp_standard
+    cxxflags = [f for f in compile_flags if not (cxx_std and _is_std(f))]
+    if cxx_std:
+        cxxflags.insert(0, (f"-std={cxx_std}",))
+    cxxflags += [(tok,) for tok in get_project_cxx_compile_flags()]
+
+    # A flag is removed whole, with its argument, as PlatformIO did
+    unflags = set(_anchored_flags(CORE.build_unflags, "esphome build_unflags"))
+    # An unflag that hits nothing (a typo, or -DUSE_FOO against
+    # -DUSE_FOO=1) must be visible, since the user believes the flag is
+    # gone while it still drives the build
+    if unmatched := sorted(unflags - set(cflags) - set(cxxflags) - set(link_flags)):
+        _LOGGER.warning(
+            "build_unflags entries matched no build flag: %s",
+            ", ".join(" ".join(flag) for flag in unmatched),
+        )
+
+    def keep(flags: list[Flag]) -> list[str]:
+        return [tok for flag in flags if flag not in unflags for tok in flag]
+
+    return keep(cflags), keep(cxxflags), keep(link_flags)
+
+
+def _resolve_host_libraries() -> list[ArduinoLibrary]:
+    """Every ``cg.add_library()`` entry, fetched from the registry.
+
+    The host has no framework, so nothing is bundled and no framework
+    compatibility check applies; the platform check keeps the strict
+    manifest gate PlatformIO's native platform enforced. Manifest-less
+    libraries (a bare git checkout) build with PlatformIO's default
+    layout, as they did under its native platform.
+    """
+    if not CORE.platformio_libraries:
+        return []
+    from esphome.arduino.library import resolve_libraries
+
+    return resolve_libraries(
+        None,
+        pio_platform=PIO_PLATFORM,
+        board_mcu="host",
+        cache_key=LIBRARY_CACHE_KEY,
+        framework=None,
+        manifest_optional=True,
+    )
+
+
+def _file_macro_maps(build_dir: Path) -> list[str]:
+    """Flags that keep ``__FILE__`` relative to the build path.
+
+    PlatformIO compiled ``src/x.cpp`` from the build path, and tools name
+    things after that spelling (CodSpeed's benchmark ids). Here a source
+    reaches the compiler by its absolute path, or relative to the build
+    directory when ccache rewrites it.
+    """
+    build_path = Path(CORE.build_path)
+    prefixes = (build_path, Path(os.path.relpath(build_path, build_dir)))
+    return [
+        shell_token(f"-fmacro-prefix-map={prefix}{os.sep}=", force=True)
+        for prefix in prefixes
+    ]
+
+
+def _compiler_version(cxx: tuple[str, ...]) -> str:
+    """What the compiler says it is: its path can stay the same across an
+    update (the macOS shims in /usr/bin)."""
+    result = subprocess.run(
+        [*cxx, "--version"], capture_output=True, text=True, check=False
+    )
+    return result.stdout
+
+
+def write_project(compilers: HostCompilers, ccache: str | None) -> bool:
+    """Write the ninja build for the current configuration.
+
+    ``ccache`` is the caller's already-resolved binary (None when disabled).
+    Returns True when ``build.ninja`` changed, so the caller can skip work
+    derived purely from it (the compile database) on unchanged builds.
+    """
+    build_dir = get_build_dir()
+    mkdir_p(build_dir)
+    src_dir = CORE.relative_src_path()
+    if not src_dir.is_dir():
+        # Generated project state, not install state: clean-all would not help
+        raise EsphomeError(f"Generated source directory {src_dir} is missing")
+
+    cflags, cxxflags, link_flags = _flag_lists()
+    libraries = _resolve_host_libraries()
+
+    include_dirs = [src_dir]
+    for lib in libraries:
+        include_dirs += lib.include_dirs
+    includes = [f"-I{_q(d)}" for d in include_dirs]
+    includes += _file_macro_maps(build_dir)
+
+    # SCons's link line: $LINKFLAGS $SOURCES $_LIBDIRFLAGS $_LIBFLAGS, so
+    # -L and -l trail the objects while every other link token leads
+    lib_dirs = [Path(t[2:]) for t in link_flags if t.startswith("-L")]
+    libs = [t for t in link_flags if t.startswith("-l")]
+    linkflags = [shell_token(t) for t in link_flags if not t.startswith(("-L", "-l"))]
+    for lib in libraries:
+        lib_dirs += lib.link_dirs
+        libs += [f"-l{name}" for name in lib.link_libs]
+        linkflags += [shell_token(f) for f in lib.link_flags]
+
+    # PlatformIO's ASPPCOM passes only -D/-I user flags to assembly
+    asflags = [t for t in cflags if t.startswith(("-D", "-I"))]
+
+    lines = [
+        *tool_lines(compilers.cc, compilers.cxx, ccache),
+        *compile_rule_lines(ccache),
+        *pch_rule_lines(),
+        "rule link",
+        "  command = $cxx -o $out $linkflags @$out.rsp $archives $libdirflags $libflags",
+        "  rspfile = $out.rsp",
+        "  rspfile_content = $in_newline",
+        "  description = LINK $out",
+    ]
+    if any(lib.sources and lib.lib_archive for lib in libraries):
+        # Resolved only when an archive is built, so a system without
+        # binutils still links a library-free configuration
+        lines += ar_rule_lines(find_tool("AR", ("ar",)))
+    lines += [
+        "",
+        f"cflags = {' '.join([*map(shell_token, cflags), *includes])}",
+        f"cxxflags = {' '.join([*map(shell_token, cxxflags), *includes])}",
+        f"asflags = {' '.join([*map(shell_token, asflags), *includes])}",
+        f"linkflags = {' '.join(linkflags)}",
+        f"libdirflags = {' '.join(f'-L{_q(d)}' for d in lib_dirs)}",
+        f"libflags = {' '.join(shell_token(lib) for lib in libs)}",
+        "",
+    ]
+
+    archives, direct_objs = library_edges(lines, libraries)
+
+    src_cxx_override = pch_edges(
+        lines,
+        build_dir,
+        src_dir,
+        PCH_DEFAULT_HEADERS,
+        # The arguments of a CXX override come before the flags
+        [*compilers.cxx[1:], *cxxflags],
+        (),
+        (compilers.cxx[0], _compiler_version(compilers.cxx)) if pch_enabled() else (),
+        compilers.cxx,
+    )
+    src_objs = compile_edges(
+        lines,
+        collect_sources(src_dir),
+        src_dir,
+        "src",
+        cxx_override=src_cxx_override,
+    )
+    if not src_objs:
+        raise EsphomeError(f"No source files found under {src_dir}")
+
+    # Archives are not topologically sorted; GNU ld needs the group to
+    # resolve references between them. ld64 loads archives iteratively and
+    # rejects the option, so macOS lists them bare.
+    archive_tokens = [shell_token(a) for a in archives]
+    if archive_tokens and sys.platform != "darwin":
+        archive_tokens = ["-Wl,--start-group", *archive_tokens, "-Wl,--end-group"]
+    lines.append(
+        f"build {PROGRAM_NAME}: link {' '.join(src_objs + direct_objs)} | "
+        f"{' '.join(_e(a) for a in archives)}"
+    )
+    lines.append(f"  archives = {' '.join(archive_tokens)}")
+    lines.append(f"default {PROGRAM_NAME}")
+    lines.append("")
+
+    return write_file_if_changed(build_dir / "build.ninja", "\n".join(lines))

esphome/build_gen/platformio.py

@@ -1,6 +1,8 @@
+from esphome.build_helpers.pch import pch_script_enabled
 from esphome.const import __version__
 from esphome.core import CORE
 from esphome.helpers import mkdir_p, read_file, write_file_if_changed
+from esphome.platformio.toolchain import copy_pch_script
 from esphome.writer import find_begin_end
 
 INI_AUTO_GENERATE_BEGIN = "; ========== AUTO GENERATED CODE BEGIN ==========="
@@ -62,6 +64,8 @@ def get_ini_content():
 
     # Add extra script for C++ flags
     CORE.add_platformio_option("extra_scripts", [f"pre:{CXX_FLAGS_FILE_NAME}"])
+    if pch_script_enabled():
+        CORE.add_platformio_option("extra_scripts", ["post:pch.py"])
 
     # Add CMake args. A user-supplied value (str or list) is deliberately
     # replaced; this option was always overwritten at FINAL priority.
@@ -106,6 +110,8 @@ def write_project():
 
     # Write extra script for C++ specific flags
     write_cxx_flags_script()
+    if pch_script_enabled():
+        copy_pch_script()
 
 
 CXX_FLAGS_FILE_NAME = "cxx_flags.py"

esphome/build_helpers/ccache.py

@@ -68,6 +68,15 @@ def resolve_ccache_path() -> str | None:
     return ccache
 
 
+def resolve_absolute_ccache_path() -> str | None:
+    """``resolve_ccache_path`` for the ninja backends, which run their
+    commands from the build directory, where a relative path is lost."""
+    from esphome.build_helpers.ninja import absolute_tool
+
+    ccache = resolve_ccache_path()
+    return absolute_tool(ccache) if ccache else None
+
+
 def ccache_defaults_env(cache_dir: Path) -> dict[str, str]:
     """Default ``CCACHE_*`` values for a build subprocess (not os.environ).
 
@@ -84,9 +93,39 @@ def ccache_defaults_env(cache_dir: Path) -> dict[str, str]:
             "CORE.build_path must be set before constructing the build environment"
         )
     defaults = {
-        "CCACHE_DIR": str(cache_dir),
+        # ccache expands $VAR in its settings; $$ is a literal $
+        "CCACHE_DIR": str(cache_dir).replace("$", "$$"),
         "CCACHE_NOHASHDIR": "true",
         "CCACHE_DEPEND": "1",
-        "CCACHE_BASEDIR": str(Path(CORE.build_path).resolve()),
+        "CCACHE_BASEDIR": str(Path(CORE.build_path).resolve()).replace("$", "$$"),
     }
     return {k: v for k, v in defaults.items() if k not in os.environ}
+
+
+def effective_ccache_basedir() -> str:
+    """The prefix ccache strips from hashed paths: a usable user
+    CCACHE_BASEDIR, else the resolved build path."""
+    from esphome.core import CORE
+
+    raw = os.environ.get("CCACHE_BASEDIR")
+    if raw is not None and Path(raw).is_absolute() and len(Path(raw).parts) > 1:
+        return raw
+    return str(Path(CORE.build_path).resolve())
+
+
+def ccache_env(ccache: str | None, tools_cache: tuple[str, str]) -> dict[str, str]:
+    """The ccache settings for a build subprocess (not os.environ).
+
+    ``ccache`` is the pre-resolved binary (resolve_ccache_path), or None when
+    disabled; ``tools_cache`` is the backend's tools cache spec, which holds
+    its ccache dir. The pch settings include ``time_macros``: a cached
+    object can keep an older ``__DATE__`` or ``__TIME__``.
+    """
+    if ccache is None:
+        return {}
+    from esphome.build_helpers.pch import ccache_pch_env
+    from esphome.build_helpers.tools_cache import tools_cache_path
+
+    env = ccache_defaults_env(tools_cache_path(*tools_cache) / "ccache")
+    env.update(ccache_pch_env())
+    return env

esphome/build_helpers/idedata.py

@@ -11,6 +11,7 @@ consumers (IDE integration, clang-tidy) expect:
 
 from __future__ import annotations
 
+from collections.abc import Callable
 import json
 import logging
 import os
@@ -21,6 +22,8 @@ import subprocess
 from esphome.core import EsphomeError
 from esphome.helpers import write_file
 
+_LOGGER = logging.getLogger(__name__)
+
 # Everything idedata generation may raise after a successful link; idedata
 # is a bonus artifact, so consumers warn instead of failing the build
 IDEDATA_BEST_EFFORT_ERRORS = (
@@ -31,7 +34,21 @@ IDEDATA_BEST_EFFORT_ERRORS = (
     ValueError,
 )
 
-_LOGGER = logging.getLogger(__name__)
+
+def warn_if_idedata_missing(get_idedata: Callable[[], dict | None]) -> None:
+    """Run an idedata generator, downgrading any failure to a warning:
+    the firmware already built."""
+    try:
+        if get_idedata() is None:
+            _LOGGER.warning("No idedata was generated for this build")
+    except IDEDATA_BEST_EFFORT_ERRORS as err:
+        _LOGGER.warning(
+            "Could not generate idedata: %s (IDE, clang-tidy, and "
+            "memory-analysis data will be unavailable for this build)",
+            err,
+        )
+        _LOGGER.debug("Idedata failure detail", exc_info=True)
+
 
 # C++ translation-unit suffixes used to identify ESPHome source files.
 _CXX_SUFFIXES = (".cpp", ".cc")
@@ -135,6 +152,15 @@ def _is_launcher(token: str) -> bool:
     return Path(token).stem.lower() in _LAUNCHER_STEMS
 
 
+def is_joined_include(tok: str) -> bool:
+    """The joined ``-includefoo.h`` spelling; excludes clang's -include-pch."""
+    return (
+        tok.startswith("-include")
+        and tok != "-include"
+        and not tok.startswith("-include-")
+    )
+
+
 def parse_entry(
     entry: dict, launcher: str | None = None
 ) -> tuple[str, list[str], list[str], list[str]]:

esphome/build_helpers/native.py

@@ -0,0 +1,74 @@
+"""Resolution of the native (non-PlatformIO) build backend for a config.
+
+Kept deliberately light: the serial upload and logs fast path resolves a
+backend for its artifact paths alone, so importing this must not pull in a
+platform component package or the backend itself.
+"""
+
+from __future__ import annotations
+
+from collections.abc import Collection
+import importlib
+import logging
+from types import ModuleType
+
+from esphome.const import Toolchain
+from esphome.core import CORE, EsphomeError
+
+_LOGGER = logging.getLogger(__name__)
+
+# Native build backend per (target platform, toolchain)
+NATIVE_TOOLCHAIN_MODULES = {
+    ("esp32", Toolchain.ESP_IDF): "esphome.espidf.toolchain",
+    ("esp8266", Toolchain.ARDUINO): "esphome.arduino8266.toolchain",
+    ("host", Toolchain.HOST): "esphome.host.toolchain",
+}
+
+
+def native_backend() -> ModuleType | None:
+    """The native build backend module for the resolved toolchain."""
+    if not CORE.using_native_toolchain:
+        return None
+    key = (CORE.target_platform, CORE.toolchain)
+    if (module_path := NATIVE_TOOLCHAIN_MODULES.get(key)) is None:
+        # Degrading to the PlatformIO path would build with the wrong backend
+        raise EsphomeError(
+            f"Toolchain '{CORE.toolchain.value}' has no native build backend "
+            f"module for platform {CORE.target_platform}"
+        )
+    return importlib.import_module(module_path)
+
+
+# Binutils and the linked image for memory analysis, for toolchains that build
+# without PlatformIO but have no native build backend (which supplies them)
+ANALYSIS_TOOLCHAIN_MODULES = {
+    ("nrf52", Toolchain.SDK_NRF): "esphome.components.nrf52.toolchain",
+}
+
+
+def analysis_backend() -> ModuleType | None:
+    """The module giving objdump, readelf and the ELF of a non-PlatformIO build.
+
+    None means PlatformIO's idedata supplies them (or nothing can).
+    """
+    if (native := native_backend()) is not None:
+        return native
+    module_path = ANALYSIS_TOOLCHAIN_MODULES.get((CORE.target_platform, CORE.toolchain))
+    return importlib.import_module(module_path) if module_path else None
+
+
+def warn_ignored_platformio_options(consumed: Collection[str]) -> None:
+    """Warn for component-added platformio options a native build drops.
+
+    User-supplied keys were already routed or warned about by
+    ``core/config.py``; what survives into ``CORE.platformio_options`` came
+    from ``cg.add_platformio_option`` calls in components.
+    """
+    for key in sorted(CORE.platformio_options or {}):
+        if key not in consumed:
+            _LOGGER.warning(
+                "platformio_options->%s is ignored when building with the "
+                "native '%s' toolchain",
+                key,
+                CORE.toolchain.value,
+            )

esphome/build_helpers/ninja.py

@@ -2,14 +2,17 @@
 
 from __future__ import annotations
 
+import json
 import logging
 import os
 from pathlib import Path
 import re
 import shutil
+import subprocess
 
 from esphome.core import EsphomeError
 from esphome.framework_helpers import strip_win_long_path_prefix, tool_version_runs
+from esphome.helpers import write_file_if_changed
 
 _LOGGER = logging.getLogger(__name__)
 
@@ -23,11 +26,26 @@ def _ninja_runs(binary: str) -> bool:
     )
 
 
+# Compile rule names the generators emit; ninja's compdb tool is asked for
+# exactly these, so a renamed rule fails the build instead of stranding idedata
+COMPILE_RULES = ("c", "cxx", "aspp", "asm")
+
+
+def absolute_tool(tool: str | Path) -> str:
+    """A tool path that still resolves from the build directory.
+
+    ``shutil.which`` returns a relative path for a relative PATH entry, and
+    ninja runs the commands from ``.pioenvs/<name>``. Symlinks are kept:
+    ccache's compiler links depend on the name they are called by.
+    """
+    return strip_win_long_path_prefix(str(Path(tool).absolute()))
+
+
 def find_ninja() -> Path:
     """Locate the ninja binary: a runnable PATH hit first, else the ninja
     PyPI wheel."""
     if binary := shutil.which("ninja"):
-        binary = strip_win_long_path_prefix(binary)
+        binary = absolute_tool(binary)
         if _ninja_runs(binary):
             return Path(binary)
     import_error: ImportError | None = None
@@ -49,8 +67,18 @@ def find_ninja() -> Path:
 
 
 def escape(value: Path | str) -> str:
-    """Escape a path or token for a ninja file."""
-    return str(value).replace("$", "$$").replace(":", "$:").replace(" ", "$ ")
+    """Escape a path or token for a ninja file.
+
+    ninja has no escape for ``|`` or a line break in a path, so those fail
+    here by name instead of producing a build file ninja misreads.
+    """
+    text = str(value)
+    if bad := next((c for c in "|\r\n" if c in text), None):
+        raise EsphomeError(
+            f"Path {text!r} contains {bad!r}, which a ninja build file cannot "
+            "express; rename the file or directory"
+        )
+    return text.replace("$", "$$").replace(":", "$:").replace(" ", "$ ")
 
 
 def quote_arg(tok: str) -> str:
@@ -90,3 +118,65 @@ def shell_token(tok: str, force: bool = False) -> str:
 def quote_path(value: Path | str) -> str:
     """Force-quote a path for the ninja command line (shell/CreateProcess)."""
     return shell_token(str(value), force=True)
+
+
+def refresh_compile_commands(
+    ninja_path: Path, build_dir: Path, env: dict[str, str], ninja_changed: bool
+) -> None:
+    """Regenerate the compile DB (a pure function of build.ninja) when stale.
+
+    Freshness rides a stamp: the DB itself is written through
+    write_file_if_changed (its mtime feeds the idedata cache), so a
+    regeneration with identical content would stay "stale" forever. An
+    interrupted previous run may have rewritten the manifest without
+    regenerating the DB, hence the mtime comparison.
+    """
+    compdb = build_dir / "compile_commands.json"
+    compdb_stamp = build_dir / ".compile_commands.stamp"
+    ninja_file = build_dir / "build.ninja"
+    if (
+        ninja_changed
+        or not compdb.is_file()
+        or not compdb_stamp.is_file()
+        or compdb_stamp.stat().st_mtime < ninja_file.stat().st_mtime
+    ):
+        write_compile_commands(ninja_path, build_dir, env)
+        compdb_stamp.touch()
+
+
+def write_compile_commands(
+    ninja_path: Path, build_dir: Path, env: dict[str, str]
+) -> None:
+    compdb = build_dir / "compile_commands.json"
+    result = subprocess.run(
+        [str(ninja_path), "-C", str(build_dir), "-t", "compdb", *COMPILE_RULES],
+        env=env,
+        capture_output=True,
+        text=True,
+        check=False,
+        close_fds=False,
+    )
+    if result.returncode != 0:
+        # Drop any stale database so consumers (IDE integration, clang-tidy,
+        # the memory analyzer) can't silently read outdated data
+        compdb.unlink(missing_ok=True)
+        raise EsphomeError(f"Could not generate compile_commands.json: {result.stderr}")
+    try:
+        entries = json.loads(result.stdout)
+    except ValueError as err:
+        compdb.unlink(missing_ok=True)
+        raise EsphomeError(
+            f"ninja produced an unparsable compile database: {err} "
+            f"(output starts {result.stdout[:120]!r})"
+        ) from err
+    if not entries:
+        # compdb exits 0 with [] for unknown rule names; a renamed compile
+        # rule must fail the build, not silently strand every consumer
+        compdb.unlink(missing_ok=True)
+        raise EsphomeError(
+            "ninja produced an empty compile database; the generator's rule "
+            "names no longer match"
+        )
+    # write_file_if_changed keeps the mtime stable on no-op builds so the
+    # idedata cache stays valid
+    write_file_if_changed(compdb, result.stdout)

esphome/build_helpers/ninja_gen.py

@@ -0,0 +1,288 @@
+"""The parts of a ``build.ninja`` every native backend emits the same way.
+
+Rule names match ``SOURCE_KIND_FOR_SUFFIX`` values (c, cxx, asm, aspp) and
+``ninja.COMPILE_RULES``, which the compile database is asked for.
+"""
+
+from __future__ import annotations
+
+from collections.abc import Collection, Sequence
+import logging
+import os
+from pathlib import Path
+import sys
+from typing import TYPE_CHECKING
+
+from esphome.build_helpers.idedata import is_joined_include
+from esphome.build_helpers.ninja import (
+    escape as _e,
+    quote_path as _q,
+    shell_token as _shell_token,
+)
+from esphome.build_helpers.pch import (
+    PCH_GCH_NAME,
+    PCH_SUM_NAME,
+    log_pch_in_use,
+    pch_consumer_flags,
+    pch_identity,
+    pch_usable,
+    write_pch_headers,
+)
+from esphome.framework_helpers import strip_win_long_path_prefix
+from esphome.helpers import write_file_if_changed
+from esphome.platformio.library import SOURCE_KIND_FOR_SUFFIX
+
+if TYPE_CHECKING:
+    from esphome.arduino.library import ArduinoLibrary
+
+_LOGGER = logging.getLogger(__name__)
+
+_BUILD_TOOL = Path(__file__).parent.parent / "build_gen" / "build_tool.py"
+
+
+def collect_sources(root: Path, exclude: Collection[str] = frozenset()) -> list[Path]:
+    return sorted(
+        p
+        for p in root.rglob("*")
+        if p.suffix in SOURCE_KIND_FOR_SUFFIX and p.name not in exclude
+    )
+
+
+def common_parent(paths: list[Path]) -> Path:
+    return Path(os.path.commonpath([str(p.parent) for p in paths]))
+
+
+def _command(words: Sequence[Path | str]) -> str:
+    program, *args = words
+    return " ".join([_q(program), *(_shell_token(str(a)) for a in args)])
+
+
+def tool_lines(
+    cc: Sequence[Path | str], cxx: Sequence[Path | str], ccache: str | None
+) -> list[str]:
+    """The file header: the compilers and the helper tools as variables.
+
+    A compiler is its program followed by any arguments it always takes.
+    """
+    return [
+        "# Auto-generated by ESPHome",
+        "ninja_required_version = 1.5",
+        f"cc = {_command(cc)}",
+        f"cxx = {_command(cxx)}",
+        # The NSIS launcher starts Python with a \\?\ extended-length path
+        # that cmd.exe cannot spawn; same strip every other emitted binary
+        # path gets
+        f"python = {_q(strip_win_long_path_prefix(sys.executable))}",
+        f"buildtool = {_q(_BUILD_TOOL)}",
+        *([f"ccache = {_q(ccache)}"] if ccache else []),
+        "",
+    ]
+
+
+def compile_rule_lines(ccache: str | None) -> list[str]:
+    """The compile rules; ``$own_includes`` is empty unless an edge sets it.
+
+    ``ccache`` is the launcher ``tool_lines`` was given; the same test here
+    keeps the variable and the rules in step. It is spliced in rather than
+    left as an empty variable: a leading space on the command line makes
+    CreateProcess fail on Windows.
+    """
+    launcher = "$ccache " if ccache else ""
+    return [
+        "rule c",
+        f"  command = {launcher}$cc -MMD -MF $out.d $own_includes $cflags $flags -c $in -o $out",
+        "  depfile = $out.d",
+        "  deps = gcc",
+        "  description = CC $out",
+        "rule cxx",
+        f"  command = {launcher}$cxx -MMD -MF $out.d $own_includes $cxxflags $flags -c $in -o $out",
+        "  depfile = $out.d",
+        "  deps = gcc",
+        "  description = CXX $out",
+        "rule aspp",
+        f"  command = {launcher}$cc -MMD -MF $out.d -x assembler-with-cpp $own_includes $asflags $flags -c $in -o $out",
+        "  depfile = $out.d",
+        "  deps = gcc",
+        "  description = AS $out",
+        # Plain assembler, as SCons's ASCOM: no preprocessor, so no
+        # depfile and no $flags (defines/includes) either
+        "rule asm",
+        f"  command = {launcher}$cc -x assembler $asflags -c $in -o $out",
+        "  description = AS $out",
+    ]
+
+
+def pch_rule_lines() -> list[str]:
+    """The precompiled header rule, for a generator that emits one."""
+    return [
+        # No $ccache: the .gch embeds build dir paths
+        "rule pch",
+        "  command = $cxx -MMD -MF $out.d -x c++-header $cxxflags $flags -c $in -o $out",
+        "  depfile = $out.d",
+        "  deps = gcc",
+        "  description = PCH $out",
+    ]
+
+
+def pch_edges(
+    lines: list[str],
+    build_dir: Path,
+    src_dir: Path,
+    headers: Sequence[str],
+    cxxflags: Sequence[str],
+    src_flags: Sequence[str],
+    identity: Sequence[str],
+    cxx: Sequence[Path | str],
+) -> tuple[str, str] | None:
+    """Emit the precompiled header for the C++ src edges.
+
+    ``headers`` are folded into one prefix header, ``src_flags`` are the
+    flags every src edge carries, ``identity`` names what else the compile
+    depends on, the compiler included, and ``cxx`` is what the host rule
+    asks. Returns the ``cxx_override`` for ``compile_edges``, or None
+    without a pch.
+    """
+    if not pch_usable(cxx):
+        return None
+    if any(
+        tok == "-include" or tok.startswith("--include") or is_joined_include(tok)
+        for tok in cxxflags
+    ):
+        # $cxxflags expands first and GCC only loads a .gch for the first
+        # -include
+        _LOGGER.warning(
+            "A -include in the compiler flags prevents the precompiled header from "
+            "loading; compiling without it"
+        )
+        return None
+    log_pch_in_use()
+    source = write_pch_headers(build_dir, headers)
+    write_file_if_changed(
+        build_dir / PCH_SUM_NAME,
+        pch_identity([*cxxflags, *src_flags], src_dir, tuple(headers), identity) + "\n",
+    )
+    gch = _e(PCH_GCH_NAME)
+    # The checksum file changes with anything the .gch depends on
+    lines.append(f"build {gch}: pch {_e(source)} | {_e(PCH_SUM_NAME)}")
+    if src_flags:
+        lines.append(f"  flags = {' '.join(src_flags)}")
+    lines.append(f"srccxxflags = {' '.join([*src_flags, *pch_consumer_flags()])}")
+    return ("$srccxxflags", gch)
+
+
+def ar_rule_lines(ar: Path | str) -> list[str]:
+    return [
+        "rule ar",
+        f"  command = $python $buildtool ar {_q(ar)} $out $out.rsp",
+        "  rspfile = $out.rsp",
+        "  rspfile_content = $in_newline",
+        "  description = AR $out",
+    ]
+
+
+def compile_edges(
+    lines: list[str],
+    sources: list[Path],
+    root: Path,
+    group: str,
+    flags: str = "",
+    own_includes: str = "",
+    cxx_override: tuple[str, str] | None = None,
+) -> list[str]:
+    """Emit compile edges for ``sources``; return the object paths.
+
+    ``cxx_override`` is a (flags, implicit-dep) pair applied to C++ edges
+    only, replacing ``flags`` (used for the precompiled header).
+    """
+    objects = []
+    for src in sources:
+        rel = src.relative_to(root).as_posix()
+        obj = f"obj/{group}/{rel}.o"
+        escaped_obj = _e(obj)
+        kind = SOURCE_KIND_FOR_SUFFIX[src.suffix]
+        override = cxx_override if kind == "cxx" else None
+        implicit = f" | {override[1]}" if override else ""
+        lines.append(f"build {escaped_obj}: {kind} {_e(src)}{implicit}")
+        if own_includes:
+            lines.append(f"  own_includes = {own_includes}")
+        if edge_flags := override[0] if override else flags:
+            lines.append(f"  flags = {edge_flags}")
+        # Escaped once here: the returned paths only ever appear in build
+        # statements (archive/link inputs), which use ninja escaping
+        objects.append(escaped_obj)
+    return objects
+
+
+def library_edges(
+    lines: list[str], libraries: list[ArduinoLibrary], extra_flags: str = ""
+) -> tuple[list[str], list[str]]:
+    """Emit every library's compile and archive edges.
+
+    ``extra_flags`` (raw ninja text) precede each library's own flags.
+
+    Returns the archive names and the objects that link directly. A
+    library's own include dirs lead its compile lines, as PlatformIO searched
+    them first: the include path is one global list, so another library's
+    header of the same name would shadow them.
+    """
+    archives: list[str] = []
+    direct_objs: list[str] = []
+    for lib in libraries:
+        if not lib.sources:
+            # Header-only libraries are legitimate; the log makes an empty
+            # srcFilter or broken tree traceable before link errors do
+            _LOGGER.debug(
+                "Library %s has no source files; contributing includes only",
+                lib.name,
+            )
+            continue
+        objs = compile_edges(
+            lines,
+            lib.sources,
+            common_parent(lib.sources),
+            f"lib/{lib.name}",
+            flags=" ".join(filter(None, [extra_flags, *map(_shell_token, lib.flags)])),
+            own_includes=" ".join(f"-I{_q(d)}" for d in lib.include_dirs),
+        )
+        if not lib.lib_archive:
+            # libArchive: false / dot_a_linkage=false: hand the objects to
+            # the linker directly so unreferenced-but-required symbols
+            # (exception handlers, weak overrides) survive
+            direct_objs.extend(objs)
+            continue
+        archive = f"lib{lib.name}.a"
+        lines.append(f"build {_e(archive)}: ar {' '.join(objs)}")
+        archives.append(archive)
+    return archives, direct_objs
+
+
+# One build flag: the flag, plus its argument when that is a separate token
+Flag = tuple[str, ...]
+
+# Flags whose path operand is the next token; gcc also takes it glued on
+PATH_ARG_FLAGS = ("-include", "-imacros", "-isystem", "-iquote", "-idirafter")
+# Flags whose path operand is glued on
+PATH_PREFIXES = ("-I", "-L", *PATH_ARG_FLAGS)
+
+
+def _anchor(path: str, base: Path) -> str:
+    if not path or Path(path).is_absolute():
+        return path
+    return str(base / path)
+
+
+def anchor_path_flag(flag: Flag, base: Path) -> Flag:
+    """Anchor a flag's relative path operand at ``base``.
+
+    PlatformIO ran the compiler from the build path; ninja runs it from
+    ``.pioenvs/<name>``, where a relative operand would point elsewhere.
+    """
+    name, *args = flag
+    if args:
+        if name in PATH_ARG_FLAGS:
+            return (name, _anchor(args[0], base))
+        return flag
+    for prefix in PATH_PREFIXES:
+        if name.startswith(prefix):
+            return (prefix + _anchor(name[len(prefix) :], base),)
+    return flag

esphome/build_helpers/pch.py

@@ -0,0 +1,248 @@
+"""Shared precompiled header policy for the build backends."""
+
+from __future__ import annotations
+
+from collections.abc import Iterable, Sequence
+import hashlib
+import logging
+import os
+from pathlib import Path
+import posixpath
+import re
+import subprocess
+import sys
+
+from esphome.build_helpers.ccache import effective_ccache_basedir, parse_enable_env
+from esphome.const import PLATFORM_NRF52
+from esphome.helpers import write_file_if_changed
+
+_LOGGER = logging.getLogger(__name__)
+
+# The header and its sidecars live in the build directory
+PCH_HEADER_NAME = "esphome_pch.h"
+PCH_GCH_NAME = f"{PCH_HEADER_NAME}.gch"
+# ccache hashes this instead of the .gch; also the freshness stamp
+PCH_SUM_NAME = f"{PCH_GCH_NAME}.sum"
+# The include list the .gch is compiled from
+PCH_SOURCE_NAME = "esphome_pch_src.h"
+
+# GCC can skip a .gch without a diagnostic and read the header of the same
+# name, so that header is an error. Other tools get the include list.
+PCH_GUARD_TEXT = f"""\
+#if defined(__GNUC__) && !defined(__clang__) && !defined(__INTELLISENSE__)
+#error "The precompiled header was not loaded"
+#else
+#include "{PCH_SOURCE_NAME}"
+#endif
+"""
+
+# The cc1plus wrapper the PlatformIO script writes on arm64 macOS
+PCH_CC1_DIR = "pch_cc1"
+
+# What the PlatformIO script leaves in the project root, for cleanup
+PCH_ARTIFACT_NAMES = (PCH_HEADER_NAME, PCH_GCH_NAME, PCH_SUM_NAME, PCH_SOURCE_NAME)
+PCH_ARTIFACT_DIRS = (PCH_CC1_DIR,)
+
+# The core headers every backend precompiles
+PCH_DEFAULT_HEADERS = ("esphome/core/pch_prefix.h",)
+
+# PlatformIO platforms that do not take the pch script
+PCH_SCRIPT_EXCLUDED_PLATFORMS = frozenset(
+    {
+        PLATFORM_NRF52,
+    }
+)
+
+# What ccache needs to cache compiles that load a .gch
+_CCACHE_PCH_SLOPPINESS = ("pch_defines", "time_macros")
+
+# Both include forms: an angle include resolving under src/ enters the digest
+_INCLUDE_RE = re.compile(rb'^\s*#\s*include\s+["<]([^">]+)[">]', re.MULTILINE)
+
+
+def pch_enabled() -> bool:
+    """Precompiled-header knob: default on, ``ESPHOME_PCH_ENABLE=0`` opts out."""
+    return parse_enable_env("ESPHOME_PCH_ENABLE") is not False
+
+
+def pch_forced() -> bool:
+    """``ESPHOME_PCH_ENABLE=1``: wanted even where the host rule says no."""
+    return parse_enable_env("ESPHOME_PCH_ENABLE") is True
+
+
+# GCC bug 14940: before these releases the Windows loader maps a .gch only
+# at its saved address. First fixed release per major, 16 on always fixed;
+# PCH_WINDOWS_CMAKE_OLD_GCC and the pch_usable message spell the same table
+PCH_WINDOWS_GCC_FIXED = {14: (14, 4), 15: (15, 3)}
+PCH_WINDOWS_GCC_FIXED_DEFAULT = (16, 0)
+# The same rule for CMake, which alone knows the version before configure
+PCH_WINDOWS_CMAKE_OLD_GCC = (
+    "CMAKE_CXX_COMPILER_VERSION VERSION_LESS 14.4 OR "
+    "(CMAKE_CXX_COMPILER_VERSION VERSION_GREATER_EQUAL 15 AND "
+    "CMAKE_CXX_COMPILER_VERSION VERSION_LESS 15.3)"
+)
+
+
+def gcc_relocates_pch_on_windows(version: Sequence[int]) -> bool:
+    """Whether a GCC of this version loads a .gch on Windows."""
+    if not version:
+        return False
+    fixed = PCH_WINDOWS_GCC_FIXED.get(version[0], PCH_WINDOWS_GCC_FIXED_DEFAULT)
+    return tuple(version[:2]) >= fixed
+
+
+# GCC ends the first --version line with its version; clang names itself
+_VERSION_RE = re.compile(r"\d+(?:\.\d+)+")
+
+
+def gcc_version(cxx: Sequence[Path | str]) -> tuple[int, ...] | None:
+    """The GCC version from ``--version``: () when it cannot be read, None
+    for a compiler that is not GCC."""
+    try:
+        result = subprocess.run(
+            [*cxx, "--version"], capture_output=True, text=True, check=False
+        )
+    except OSError as err:
+        _LOGGER.debug("Cannot run %s: %s", cxx[0], err)
+        return ()
+    banner = result.stdout.partition("\n")[0]
+    if "clang" in banner.lower():
+        return None
+    found = _VERSION_RE.findall(banner)
+    return tuple(int(part) for part in found[-1].split(".")) if found else ()
+
+
+def pch_needs_gcc_check() -> bool:
+    """Windows host with the knob unset: the compiler version decides."""
+    return sys.platform == "win32" and parse_enable_env("ESPHOME_PCH_ENABLE") is None
+
+
+def pch_usable(cxx: Sequence[Path | str]) -> bool:
+    """The knob plus the host rule; ``ESPHOME_PCH_ENABLE=1`` skips the rule."""
+    if not pch_enabled():
+        return False
+    if not pch_needs_gcc_check():
+        return True
+    version = gcc_version(cxx)
+    if version is None or gcc_relocates_pch_on_windows(version):
+        return True
+    _LOGGER.info(
+        "GCC %s cannot load a precompiled header on Windows (GCC bug 14940, "
+        "fixed in 14.4, 15.3 and 16); compiling without it "
+        "(set ESPHOME_PCH_ENABLE=1 to force)",
+        ".".join(map(str, version)) or "of unknown version",
+    )
+    return False
+
+
+def pch_consumer_flags() -> list[str]:
+    """Flags a C++ src compile loads the pch with. The -include stays
+    relative: an absolute path would enter the ccache key."""
+    return ["-Winvalid-pch", "-Werror=invalid-pch", "-include", PCH_HEADER_NAME]
+
+
+def ccache_pch_env() -> dict[str, str]:
+    """What ccache needs to cache compiles that load a .gch, added to what
+    the user already set."""
+    if not pch_enabled():
+        return {}
+    sloppiness = [
+        item.strip()
+        for item in os.environ.get("CCACHE_SLOPPINESS", "").split(",")
+        if item.strip()
+    ]
+    sloppiness += [item for item in _CCACHE_PCH_SLOPPINESS if item not in sloppiness]
+    env = {"CCACHE_SLOPPINESS": ",".join(sloppiness)}
+    if "CCACHE_PCH_EXTSUM" not in os.environ:
+        env["CCACHE_PCH_EXTSUM"] = "true"
+    return env
+
+
+def pch_script_enabled() -> bool:
+    """Whether this PlatformIO build takes the pch script."""
+    from esphome.core import CORE
+
+    return pch_enabled() and CORE.target_platform not in PCH_SCRIPT_EXCLUDED_PLATFORMS
+
+
+def pch_header_text(include_headers: Iterable[str]) -> str:
+    """The prefix-header source: exactly these includes, in order."""
+    return "".join(f'#include "{name}"\n' for name in include_headers)
+
+
+def write_pch_headers(build_dir: Path, include_headers: Iterable[str]) -> Path:
+    """Write the guard header and the include list; return the latter,
+    which is what the .gch compiles from."""
+    write_file_if_changed(build_dir / PCH_HEADER_NAME, PCH_GUARD_TEXT)
+    source = build_dir / PCH_SOURCE_NAME
+    write_file_if_changed(source, pch_header_text(include_headers))
+    return source
+
+
+def _include_closure(src_dir: Path, roots: Iterable[str]) -> dict[str, bytes]:
+    """Include closure of ``roots``: src-relative name -> contents.
+
+    Resolution mirrors the compiler (includer's dir, then src root). No
+    #ifdef evaluation: including too much is the safe direction. Headers
+    outside ``src_dir`` are covered by the version strings of the caller.
+    """
+    seen: dict[str, bytes] = {}
+    stack: list[tuple[str, str]] = [(name, "") for name in roots]
+    while stack:
+        name, from_dir = stack.pop()
+        for candidate in (f"{from_dir}/{name}" if from_dir else name, name):
+            rel = posixpath.normpath(candidate)
+            if not rel.startswith("..") and (src_dir / rel).is_file():
+                break
+        else:
+            continue
+        if rel in seen:
+            continue
+        data = seen[rel] = (src_dir / rel).read_bytes()
+        parent = posixpath.dirname(rel)
+        stack.extend((inc.decode(), parent) for inc in _INCLUDE_RE.findall(data))
+    return seen
+
+
+def pch_checksum(
+    src_dir: Path, include_headers: Iterable[str], extra: Iterable[str]
+) -> str:
+    """Digest of the prefix header's include closure plus ``extra``."""
+    digest = hashlib.sha256()
+    closure = _include_closure(src_dir, include_headers)
+    for name in sorted(closure):
+        digest.update(name.encode())
+        digest.update(closure[name])
+        digest.update(b"\0")
+    for item in extra:
+        digest.update(item.encode())
+        digest.update(b"\0")
+    return digest.hexdigest()
+
+
+def pch_identity(
+    tokens: Iterable[str],
+    src_dir: Path,
+    include_headers: tuple[str, ...],
+    extra: Iterable[str],
+) -> str:
+    """The .sum digest: include closure, header text, ``extra`` and the
+    compile flags with the build path stripped, as ccache does."""
+    from esphome.core import CORE
+
+    flags = (
+        " ".join(tokens)
+        .replace(str(CORE.build_path), "")
+        .replace(effective_ccache_basedir(), "")
+    )
+    # The closure is sorted, so header order only enters via the text
+    return pch_checksum(
+        src_dir, include_headers, (pch_header_text(include_headers), *extra, flags)
+    )
+
+
+_DISABLE_HINT = " (set ESPHOME_PCH_ENABLE=0 to disable)"
+
+
+def log_pch_in_use() -> None:
+    _LOGGER.info("Compiling with a precompiled header%s", _DISABLE_HINT)

esphome/build_helpers/tool_runner.py

@@ -0,0 +1,180 @@
+"""Run a native build tool (cmake, ninja) and relay its output.
+
+Output is read from a pipe so it can be filtered here: a child that inherits
+our stdout writes straight to the file descriptor, past any Python wrapper.
+"""
+
+from __future__ import annotations
+
+import codecs
+from contextlib import suppress
+import logging
+import os
+from pathlib import Path
+import re
+import shutil
+import subprocess
+import sys
+from typing import Any, TextIO
+
+from esphome.util import ANSI_ESCAPE, RedirectText, shlex_quote
+
+_LOGGER = logging.getLogger(__name__)
+
+# Windows code page identifier for UTF-8, as used by ``chcp 65001``.
+UTF8_CODEPAGE = 65001
+
+# Same pattern idf.py uses to spot ninja status lines (``is_progression``).
+_PROGRESS = re.compile(r"^\[\d+/\d+\]|.*\(\d+ \%\)$")
+_READ_SIZE = 65536
+
+
+def _get_kernel32() -> Any | None:
+    """Return the Windows kernel32 module, or None on any other platform."""
+    if sys.platform != "win32":
+        return None
+    import ctypes
+
+    return ctypes.windll.kernel32
+
+
+class Utf8Console:
+    """Keep an attached Windows console on UTF-8 while a build tool runs.
+
+    esp_idf_size draws its table with Unicode box characters, and CMake
+    re-decodes a child's output with the console code page, which garbles the
+    table on any page but UTF-8. A console already on UTF-8 is left alone so
+    an overlapping build never records UTF-8 as the page to go back to.
+    """
+
+    def __init__(self, kernel32: Any | None) -> None:
+        self._kernel32 = kernel32
+        self._codepages: tuple[int, int] | None = None
+
+    def __enter__(self) -> None:
+        kernel32 = self._kernel32
+        if kernel32 is None:
+            return
+        old_in = kernel32.GetConsoleCP()
+        old_out = kernel32.GetConsoleOutputCP()
+        # Both calls return 0 when no console is attached.
+        if not old_in or not old_out:
+            return
+        if old_in == UTF8_CODEPAGE and old_out == UTF8_CODEPAGE:
+            return
+        # Record first so a switch that fails part way is still undone.
+        self._codepages = (old_in, old_out)
+        kernel32.SetConsoleCP(UTF8_CODEPAGE)
+        kernel32.SetConsoleOutputCP(UTF8_CODEPAGE)
+
+    def __exit__(self, *exc_info: object) -> None:
+        if self._codepages is None:
+            return
+        old_in, old_out = self._codepages
+        self._codepages = None
+        self._kernel32.SetConsoleCP(old_in)
+        self._kernel32.SetConsoleOutputCP(old_out)
+
+
+def _fit_terminal(text: str) -> str:
+    """Elide the middle of ``text`` to fit the terminal, as idf.py does.
+
+    A width of 0 (a pipe, the dashboard) leaves the text whole.
+    """
+    width = shutil.get_terminal_size((0, 0)).columns
+    if not width:
+        return text
+    if width <= 3:
+        return "." * width
+    if len(text) >= width:
+        keep = (width - 3) // 2
+        return f"{text[:keep]}...{text[len(text) - keep :]}"
+    return text
+
+
+class ToolOutput(RedirectText):
+    """RedirectText that can collapse ninja status lines into one line.
+
+    With ``progress`` each ``[n/m]`` line overwrites the previous one, the
+    way idf.py shows a build.
+    """
+
+    def __init__(
+        self, out: TextIO, filter_lines: list[str] | None, progress: bool
+    ) -> None:
+        super().__init__(out, filter_lines=filter_lines)
+        self._progress = progress
+        self._on_progress_line = False
+
+    def _splits_lines(self) -> bool:
+        return self._progress or super()._splits_lines()
+
+    def _emit_line(self, line: str) -> None:
+        if self._progress and _PROGRESS.match(line):
+            if not self._is_filtered(line):
+                text = _fit_terminal(line.strip("\r\n"))
+                self._write_color_replace(f"\r{text}\x1b[K")
+                self._on_progress_line = True
+            return
+        self._end_progress_line()
+        super()._emit_line(line)
+
+    def _end_progress_line(self) -> None:
+        if self._on_progress_line:
+            self._on_progress_line = False
+            self._write_color_replace(os.linesep)
+
+    def drain(self) -> None:
+        super().drain()
+        # Called from cleanup, so a broken stream must not hide the exit code.
+        with suppress(OSError, ValueError):
+            self._end_progress_line()
+            self._out.flush()
+
+
+def run_build_tool(
+    cmd: list[str],
+    *,
+    cwd: Path,
+    env: dict[str, str],
+    filter_lines: list[str] | None = None,
+    progress: bool = False,
+    log_path: Path | None = None,
+) -> int:
+    """Run ``cmd`` and relay stdout and stderr, merged, to our stdout.
+
+    ``log_path`` also gets the full, unfiltered output without color codes, as
+    idf.py wrote its logs (its hint patterns expect plain text). Returns the
+    exit code.
+    """
+    _LOGGER.debug("Running: %s", " ".join(shlex_quote(arg) for arg in cmd))
+    _LOGGER.debug("  in directory: %s", cwd)
+    output = ToolOutput(sys.stdout, filter_lines, progress)
+    decoder = codecs.getincrementaldecoder("utf-8")(errors="replace")
+    if log_path is not None:
+        log_path.parent.mkdir(parents=True, exist_ok=True)
+    with (
+        Path(log_path or os.devnull).open("w", encoding="utf-8", newline="") as log,
+        Utf8Console(_get_kernel32()),
+        subprocess.Popen(
+            cmd,
+            cwd=cwd,
+            env=env,
+            stdout=subprocess.PIPE,
+            stderr=subprocess.STDOUT,
+            close_fds=False,
+        ) as proc,
+    ):
+        try:
+            # read1 returns as soon as anything is available, so output
+            # streams while the tool runs.
+            while chunk := proc.stdout.read1(_READ_SIZE):
+                text = decoder.decode(chunk)
+                log.write(ANSI_ESCAPE.sub("", text))
+                output.write(text)
+            if tail := decoder.decode(b"", final=True):
+                log.write(ANSI_ESCAPE.sub("", tail))
+                output.write(tail)
+        finally:
+            output.drain()
+    return proc.returncode

esphome/build_helpers/tools_cache.py

@@ -16,8 +16,8 @@ def tools_cache_path(env_var: str, subdir: str) -> Path:
     from esphome.helpers import get_str_env
 
     if prefix := get_str_env(env_var, "").strip():
-        # resolve(): symlinked prefixes otherwise trip idf.py's
-        # venv-mismatch warning on every build
+        # resolve(): a symlinked prefix would otherwise record a second
+        # spelling of the same paths in the build tree
         return Path(prefix).expanduser().resolve()
     # appauthor=False keeps the Windows path short (no vendor segment);
     # deep IDF trees run into MAX_PATH otherwise
@@ -33,4 +33,14 @@ def tools_cache_path(env_var: str, subdir: str) -> Path:
 IDF_TOOLS_CACHE = ("ESPHOME_ESP_IDF_PREFIX", "idf")
 SDK_NRF_TOOLS_CACHE = ("ESPHOME_SDK_NRF_PREFIX", "sdk-nrf")
 ARDUINO8266_TOOLS_CACHE = ("ESPHOME_ARDUINO8266_PREFIX", "arduino8266")
-TOOLS_CACHE_SPECS = (IDF_TOOLS_CACHE, SDK_NRF_TOOLS_CACHE, ARDUINO8266_TOOLS_CACHE)
+# The host backend installs nothing; the entry only holds its ccache dir
+HOST_TOOLS_CACHE = ("ESPHOME_HOST_PREFIX", "host")
+# PlatformIO installs into its own dirs; the entry is its ccache dir itself
+PLATFORMIO_CCACHE = ("ESPHOME_PLATFORMIO_CCACHE_DIR", "platformio-ccache")
+TOOLS_CACHE_SPECS = (
+    IDF_TOOLS_CACHE,
+    SDK_NRF_TOOLS_CACHE,
+    ARDUINO8266_TOOLS_CACHE,
+    HOST_TOOLS_CACHE,
+    PLATFORMIO_CCACHE,
+)

esphome/bundle.py

@@ -102,25 +102,21 @@ _KNOWN_FILE_EXTENSIONS = frozenset(
 )
 
 
-# Matches !secret references in YAML text.  An optional surrounding
-# quote pair around the key is allowed and ignored: YAML treats
-# ``!secret 'foo'`` and ``!secret foo`` as the same key.  This is
-# intentionally a simple regex scan rather than a YAML parse — it may
-# match inside comments or multi-line strings, which is the conservative
-# direction (include more secrets rather than fewer).
-_SECRET_RE = re.compile(r"""!secret\s+['"]?([^\s'"]+)""")
-
-
 def _find_used_secret_keys(yaml_files: list[Path]) -> set[str]:
     """Scan YAML files for ``!secret <key>`` references."""
     keys: set[str] = set()
     for fpath in yaml_files:
         try:
             text = fpath.read_text(encoding="utf-8")
-        except (OSError, UnicodeDecodeError):
+        except (OSError, UnicodeDecodeError) as err:
+            _LOGGER.warning(
+                "Could not scan %s for !secret references (%s); the bundled "
+                "secret set may be incomplete",
+                fpath,
+                err,
+            )
             continue
-        for match in _SECRET_RE.finditer(text):
-            keys.add(match.group(1))
+        keys |= yaml_util.find_secret_references(text)
     return keys
 
 
@@ -422,6 +418,18 @@ class ConfigBundleCreator:
         must ship every candidate so the remote build can pick any one.
         """
         discovered = yaml_util.discover_user_yaml_files(self._config_path)
+        if discovered.load_errors:
+            _LOGGER.warning(
+                "Bundle may be incomplete; could not load all configuration files: %s",
+                "; ".join(discovered.load_errors),
+            )
+        if discovered.unresolved:
+            _LOGGER.warning(
+                "Bundle may be incomplete; %d !include path(s) use "
+                "substitutions and cannot be captured: %s",
+                len(discovered.unresolved),
+                ", ".join(discovered.unresolved),
+            )
         self._secrets_paths.update(discovered.secrets)
         # A !secret inside a file this re-parse does not reach (for example
         # a git-fetched package the builder re-fetches) still resolves

esphome/cli/__init__.py

@@ -0,0 +1,2 @@
+"""Commands of the esphome command line, one module each, imported by
+__main__ only when they run so that startup stays light."""

esphome/cli/rename.py

@@ -0,0 +1,164 @@
+"""``esphome rename``."""
+
+from __future__ import annotations
+
+import argparse
+from pathlib import Path
+import re
+
+from esphome import yaml_edit, yaml_util
+from esphome.const import (
+    ALLOWED_NAME_CHARS,
+    CONF_ESPHOME,
+    CONF_NAME,
+    CONF_SUBSTITUTIONS,
+)
+from esphome.core import CORE, EsphomeError
+from esphome.log import AnsiFore, color
+from esphome.types import ConfigType
+from esphome.util import ESPHOME_COMMAND, run_external_process, safe_print
+
+
+def _revert(new_path: Path, why: str) -> int:
+    """Say why the rename stopped and take the new file back; an orphan the
+    next attempt would trip over is reported."""
+    safe_print(color(AnsiFore.BOLD_RED, f"Rename failed: {why}"))
+    try:
+        new_path.unlink(missing_ok=True)
+    except OSError as err:
+        safe_print(color(AnsiFore.BOLD_RED, f"Could not remove {new_path}: {err}"))
+    return 1
+
+
+def command_rename(args: argparse.Namespace, config: ConfigType) -> int | None:
+    """Rename the device: a new file with the name line rewritten, validated
+    and installed, then the old file removed."""
+    new_name = args.name
+    for c in new_name:
+        if c not in ALLOWED_NAME_CHARS:
+            safe_print(
+                color(
+                    AnsiFore.BOLD_RED,
+                    f"'{c}' is an invalid character for names. Valid characters are: "
+                    f"{ALLOWED_NAME_CHARS} (lowercase, no spaces)",
+                )
+            )
+            return 1
+
+    yaml = yaml_util.load_yaml(CORE.config_path)
+
+    def name_edit() -> tuple[str, yaml_edit.LineEdit]:
+        """The name and the line to rewrite: the name's own line, or the
+        substitution's line it comes from, as a plain value in this file."""
+        esphome_conf = yaml.get(CONF_ESPHOME)
+        if not isinstance(esphome_conf, dict) or CONF_NAME not in esphome_conf:
+            raise EsphomeError(f"no '{CONF_ESPHOME}: {CONF_NAME}:' in the file")
+        old_name = str(esphome_conf[CONF_NAME])
+        mapping, field = esphome_conf, CONF_NAME
+        if match := re.match(r"^\$\{?([a-zA-Z0-9_]+)\}?$", old_name):
+            mapping, field = yaml.get(CONF_SUBSTITUTIONS), match.group(1)
+            if not isinstance(mapping, dict) or field not in mapping:
+                raise EsphomeError(f"the substitution '{field}' is not in the file")
+            old_name = str(mapping[field])
+        # Only read here; the rewritten text goes to a new file, so the
+        # source may live anywhere the config path points to
+        source = yaml_edit.source_of(mapping, field)
+        if source is None or source[0].resolve() != CORE.config_path.resolve():
+            raise EsphomeError(f"'{field}' was not read from {CORE.config_path}")
+        doc, line_no = source
+        text = yaml_edit.line_at(doc, line_no)
+        if (line_match := yaml_edit.field_line_re(field, old_name).match(text)) is None:
+            raise EsphomeError(f"'{field}' is not a plain value on {doc}:{line_no + 1}")
+        # The new value is always quoted, whatever the old line had
+        return old_name, yaml_edit.LineEdit(
+            doc, line_no, text, yaml_edit.rewrite(line_match, new_name, '"')
+        )
+
+    try:
+        old_name, edit = name_edit()
+    except EsphomeError as err:
+        safe_print(
+            color(
+                AnsiFore.BOLD_RED,
+                f"Complex YAML files cannot be automatically renamed: {err}",
+            )
+        )
+        return 1
+
+    # ``new_name == old_name`` (after substitution resolution) is
+    # a no-op rewrite that would still queue a pointless re-flash.
+    # Catch it before the path-equality check below — covers the
+    # case where the config filename doesn't match the device name
+    # (e.g. ``weird-file.yaml`` whose ``esphome.name`` is
+    # ``kitchen``; running ``esphome rename weird-file.yaml kitchen``
+    # would otherwise just re-flash the same hostname).
+    if new_name == old_name:
+        safe_print(
+            color(
+                AnsiFore.BOLD_RED,
+                f"'{new_name}' is already the device's name.",
+            )
+        )
+        return 1
+
+    new_path: Path = CORE.config_dir / (new_name + ".yaml")
+    if new_path.resolve() == CORE.config_path.resolve():
+        safe_print(
+            color(
+                AnsiFore.BOLD_RED,
+                f"'{new_name}' is already the device's name.",
+            )
+        )
+        return 1
+    if new_path.exists():
+        safe_print(
+            color(
+                AnsiFore.BOLD_RED,
+                f"Cannot rename: {new_path} already exists. "
+                "Refusing to overwrite an existing configuration.",
+            )
+        )
+        return 1
+    safe_print(
+        f"Updating {color(AnsiFore.CYAN, str(CORE.config_path))} to {color(AnsiFore.CYAN, str(new_path))}"
+    )
+    print()
+
+    try:
+        yaml_edit.write_keeping_mode(
+            new_path,
+            yaml_edit.rewritten_text(yaml_edit.read_text(CORE.config_path), [edit]),
+            like=CORE.config_path,
+        )
+    except EsphomeError as err:
+        return _revert(new_path, str(err))
+
+    if run_external_process(*ESPHOME_COMMAND, "config", str(new_path)) != 0:
+        return _revert(new_path, "the new configuration does not validate")
+
+    cli_args = [
+        "run",
+        str(new_path),
+        "--no-logs",
+        "--device",
+        CORE.address,
+    ]
+
+    if args.dashboard:
+        cli_args.insert(0, "--dashboard")
+
+    try:
+        rc = run_external_process(*ESPHOME_COMMAND, *cli_args)
+    except KeyboardInterrupt:
+        rc = 1
+    if rc != 0:
+        return _revert(
+            new_path,
+            "the install did not finish; the device may already run the new name",
+        )
+
+    CORE.config_path.unlink()
+
+    safe_print(color(AnsiFore.BOLD_GREEN, "SUCCESS"))
+    print()
+    return 0

esphome/cli/rotate_key.py

@@ -0,0 +1,285 @@
+"""``esphome rotate-key``."""
+
+from __future__ import annotations
+
+import argparse
+from pathlib import Path
+import sys
+
+from esphome import espota2
+from esphome.components.noise import (
+    generate_encryption_key,
+    static_encryption_key,
+    validate_encryption_key,
+)
+from esphome.config_validation import Invalid
+from esphome.const import CONF_API, CONF_PASSWORD, CONF_PORT
+from esphome.core import CORE, EsphomeError
+from esphome.espota2 import ProbeResult
+from esphome.log import AnsiFore, color
+from esphome.ota_key_edit import locate_key_edits, old_key_edit, with_sharers
+from esphome.types import ConfigType
+from esphome.upload_targets import PortType, get_port_type
+from esphome.util import (
+    ESPHOME_COMMAND,
+    read_secret_line,
+    run_external_process,
+    safe_input,
+    safe_print,
+)
+from esphome.yaml_edit import RestoreError, Snapshot, apply_line_edits, restore_files
+
+PRECHECK_TIMEOUT = 15.0
+# What to do when the device does not accept the key in the yaml before anything is written
+PRECHECK_ADVICE = {
+    ProbeResult.UNREACHABLE: (
+        "The device could not be reached; check that it is online, then try again."
+    ),
+    ProbeResult.REJECTED: (
+        "The device runs a different key than the yaml. Install the current "
+        "configuration first, presenting its key with --prompt-ota-key, then rotate."
+    ),
+    ProbeResult.NOT_OFFERED: (
+        "The device does not offer encryption: it runs a firmware older than "
+        "ESPHome 2026.9.0 or without a key. Install the current configuration "
+        "first, then rotate."
+    ),
+    ProbeResult.DEVICE_ERROR: (
+        "The device answered with an error (see the warning above); fix that first."
+    ),
+}
+
+
+def command_rotate_key(args: argparse.Namespace, config: ConfigType) -> int | None:
+    """Write a new key, build with it, upload with the current one, confirm."""
+    from esphome.__main__ import (
+        Purpose,
+        _esphome_ota_conf,
+        _resolve_network_devices,
+        choose_upload_log_host,
+    )
+
+    def fail(message: str) -> int:
+        safe_print(color(AnsiFore.BOLD_RED, message))
+        return 1
+
+    ota_conf = _esphome_ota_conf(config)
+    if ota_conf is None:
+        return fail("rotate-key needs the esphome OTA platform in the configuration")
+    old_key = static_encryption_key(ota_conf)
+    if old_key is None:
+        return fail(
+            "rotate-key needs 'encryption:' under the esphome OTA platform with a "
+            "key in the yaml; a key provisioned at runtime is rotated from Home "
+            "Assistant"
+        )
+    if CORE.is_host:
+        return fail("rotate-key is for devices updated over the air")
+    old_key = str(old_key)
+    remote_port = int(ota_conf[CONF_PORT])
+
+    devices = choose_upload_log_host(
+        default=args.device, check_default=None, purpose=Purpose.UPLOADING
+    )
+    if get_port_type(devices[0]) in (PortType.SERIAL, PortType.BOOTSEL):
+        return fail(
+            "rotate-key works over the air; on a serial connection change the "
+            "key in the yaml and run 'esphome run' instead"
+        )
+    network_devices = _resolve_network_devices(devices, config, args)
+
+    if args.prompt_new_key:
+        new_key = read_secret_line("New OTA encryption key: ")
+    else:
+        new_key = generate_encryption_key()
+    try:
+        validate_encryption_key(new_key)
+    except Invalid as err:
+        return fail(f"Invalid new key: {err}")
+    if new_key == old_key:
+        return fail("The new key is the same as the current one")
+
+    try:
+        edits = locate_key_edits(old_key, new_key)
+        # The previous key stays in the config as old_key so an install still
+        # reaches the device if it ends up running either key
+        edits += old_key_edit(old_key)
+        unchecked = with_sharers(edits)
+    except EsphomeError as err:
+        return fail(str(err))
+
+    warnings = []
+    # With the recommended shape the ota inherits the api key, so rotating
+    # it changes what Home Assistant connects with
+    if str(static_encryption_key(config.get(CONF_API) or {})) == old_key:
+        warnings.append(
+            "The api encryption key changes too; Home Assistant will ask for "
+            "the new key after the install."
+        )
+    shared: dict[str, set[str]] = {}
+    for edit in edits:
+        what = f"secret '{edit.secret}'" if edit.secret else edit.path.name
+        for path in edit.shared_with:
+            shared.setdefault(
+                str(path.relative_to(CORE.config_dir.resolve())), set()
+            ).add(what)
+    if unchecked:
+        warnings.append(
+            "Could not check every configuration for shared use: "
+            + "; ".join(unchecked)
+        )
+    if shared:
+        warnings.append(
+            "Other configurations share what is rewritten: "
+            + "; ".join(
+                f"{p} ({', '.join(sorted(w))})" for p, w in sorted(shared.items())
+            )
+            + ". Their devices keep the previous key until their next install; "
+            "make sure those configurations carry it as old_key."
+        )
+    if warnings:
+        for warning in warnings:
+            safe_print(color(AnsiFore.BOLD_YELLOW, warning))
+        if not args.yes:
+            if not sys.stdin.isatty():
+                return fail("Confirm with --yes when there is no terminal")
+            if safe_input("Continue? [y/N] ").strip().lower() not in ("y", "yes"):
+                return 1
+
+    safe_print("Checking that the device accepts the current key...")
+    found = espota2.probe_ota_key(
+        network_devices,
+        remote_port,
+        old_key,
+        timeout=PRECHECK_TIMEOUT,
+        retry_rejected=False,
+    )
+    if found is not ProbeResult.ACCEPTED:
+        return fail(PRECHECK_ADVICE[found])
+
+    # Both keys are shown only when a file could not be put back
+    keys = f"Previous key: {old_key}\nNew key: {new_key}"
+    try:
+        originals = apply_line_edits(edits)
+    except RestoreError as err:
+        safe_print(keys)
+        return fail(str(err))
+    except EsphomeError as err:
+        return fail(str(err))
+    except KeyboardInterrupt as err:
+        if notes := getattr(err, "__notes__", None):  # a rollback that failed too
+            safe_print(keys)
+            return fail("Interrupted; " + "; ".join(notes))
+        return fail("Interrupted before the upload")
+    # From here every exit restores or reports, so nothing sits outside the try
+    uploaded = False
+    try:
+        safe_print(
+            "Wrote the new key to "
+            + ", ".join(color(AnsiFore.CYAN, str(p)) for p in originals)
+        )
+        # Global options go before the subcommand; the compile parser is strict
+        cli_args = ["--dashboard"] if CORE.dashboard else []
+        if args.toolchain:
+            cli_args += ["--toolchain", str(args.toolchain)]
+        for key, value in args.substitution or []:
+            cli_args += ["-s", key, value]
+        cli_args += ["compile", str(CORE.config_path)]
+        if run_external_process(*ESPHOME_COMMAND, *cli_args) != 0:
+            return fail("Compiling with the new key failed")
+
+        def image_sent() -> None:
+            # The device accepted the session and may commit from here on;
+            # the edited config reaches it on either key
+            nonlocal uploaded
+            uploaded = True
+
+        rc, _ = espota2.run_ota(
+            network_devices,
+            remote_port,
+            ota_conf.get(CONF_PASSWORD),
+            CORE.firmware_bin,
+            noise_psk=old_key,
+            plaintext_fallback=False,
+            on_upload_start=image_sent,
+        )
+        if rc != 0:
+            return fail(
+                "Uploading with the current key failed. The configuration keeps "
+                "the new key with the previous one as old_key, so the next "
+                "install reaches the device whichever key it runs."
+                if uploaded
+                else "The upload did not reach the device."
+            )
+        safe_print("Waiting for the device to come back with the new key...")
+        found = espota2.probe_ota_key(network_devices, remote_port, new_key)
+        if found is not ProbeResult.ACCEPTED:
+            return fail(
+                (
+                    "The device still answers with the previous key."
+                    if found is ProbeResult.REJECTED
+                    else "The device did not come back to confirm the new key."
+                )
+                + " The configuration keeps the new key, with the previous one as "
+                "old_key, so the next install reaches the device either way."
+            )
+    except KeyboardInterrupt:
+        return fail(
+            "Interrupted; the device's key was not confirmed"
+            if uploaded
+            else "Interrupted before the upload"
+        )
+    finally:
+        # Before the upload nothing changed on the device; after it the device
+        # most likely runs the new key, and old_key covers the other case
+        if uploaded:
+            safe_print(f"New OTA encryption key: {color(AnsiFore.CYAN, new_key)}")
+        else:
+            _restore(originals, keys)
+
+    safe_print(color(AnsiFore.BOLD_GREEN, "SUCCESS"))
+    return 0
+
+
+def _restore(originals: dict[Path, Snapshot], keys: str) -> None:
+    """Put the previous key back, and drop the images a build made with the
+    new key; both keys are shown when a step fails."""
+    files = ", ".join(str(p) for p in originals)
+    try:
+        restore_files(originals)
+    except (EsphomeError, KeyboardInterrupt) as err:
+        # An interrupted restore carries its report as a note
+        why = str(err) or "; ".join(getattr(err, "__notes__", ()))
+        safe_print(color(AnsiFore.BOLD_RED, why or f"Interrupted restoring {files}"))
+        safe_print(keys)
+    else:
+        safe_print(color(AnsiFore.BOLD_YELLOW, f"Restored the previous key in {files}"))
+    build_dir = CORE.firmware_bin.parent
+    failed: list[str] = []
+    images: list[Path] = []
+    try:
+        images = [p for s in (".bin", ".uf2") for p in build_dir.glob(f"*{s}")]
+        for image in images:  # every one is tried, whatever the others did
+            try:
+                image.unlink()
+            except OSError as err:
+                failed.append(f"{image}: {err}")
+    except (OSError, KeyboardInterrupt) as err:
+        failed.append(f"{build_dir}: {str(err) or 'interrupted'}")
+    if failed:
+        safe_print(
+            color(
+                AnsiFore.BOLD_RED,
+                "Could not remove images that may hold the new key: "
+                + "; ".join(failed),
+            )
+        )
+        safe_print(keys)
+    elif images:
+        safe_print(
+            color(
+                AnsiFore.BOLD_YELLOW,
+                f"Removed the images in {build_dir}, which may hold the new key; "
+                "compile again before the next upload.",
+            )
+        )

esphome/codegen.py

@@ -31,17 +31,23 @@ from esphome.cpp_generator import (  # noqa: F401
     add_global,
     add_library,
     add_platformio_option,
+    extern_progmem_array,
     get_variable,
     get_variable_with_full_id,
+    is_static_pointer,
     is_template,
     new_Pvariable,
     new_variable,
     process_lambda,
     progmem_array,
+    progmem_string,
     safe_exp,
     set_cpp_standard,
+    shared_progmem_array,
+    shared_progmem_string,
     statement,
     static_const_array,
+    static_function,
     templatable,
     variable,
     with_local_variable,
@@ -64,7 +70,6 @@ from esphome.cpp_types import (  # noqa: F401
     Application,
     Component,
     ComponentPtr,
-    Controller,
     EntityBase,
     EntityCategory,
     ESPTime,

esphome/compiled_config.py

@@ -33,9 +33,35 @@ _CACHE_VERSION = 1
 _LAMBDA_KEY = "__esphome_lambda__"
 
 
+_CACHE_SUFFIX = ".validated.json"
+
+
 def compiled_config_path(config_filename: str) -> Path:
     """Path to the cached validated config alongside the storage sidecar."""
-    return CORE.data_dir / "storage" / f"{config_filename}.validated.json"
+    return CORE.data_dir / "storage" / f"{config_filename}{_CACHE_SUFFIX}"
+
+
+def invalidate_compiled_config() -> None:
+    """Drop every cache in the storage dir after a change to a file the
+    mtime check cannot see, such as secrets.yaml or an include; other
+    configurations may share that file."""
+    storage = compiled_config_path(CORE.config_filename).parent
+    try:
+        caches = [p for p in storage.iterdir() if p.name.endswith(_CACHE_SUFFIX)]
+    except FileNotFoundError:
+        return
+    except OSError as err:
+        raise EsphomeError(
+            f"Could not list the validated config caches in {storage}: {err}"
+        ) from err
+    for path in caches:
+        try:
+            path.unlink(missing_ok=True)
+        except OSError as err:
+            # A stale cache would present the old key on the next upload
+            raise EsphomeError(
+                f"Could not remove the validated config cache {path}: {err}"
+            ) from err
 
 
 def save_compiled_config(config: ConfigType) -> None:

esphome/component_aliases.py

@@ -6,5 +6,6 @@ See the component-alias section of esphome/loader.py.
 
 # alias -> (canonical component, removal version or None)
 COMPONENT_ALIASES: dict[str, tuple[str, str | None]] = {
+    "esp32_improv": ("improv_ble", "2027.4.0"),
     "rp2040": ("rp2", "2027.7.0"),
 }

esphome/components/a01nyub/__init__.py

@@ -1 +1,2 @@
 CODEOWNERS = ["@MrSuicideParrot"]
+DOMAIN = "a01nyub"

esphome/components/a01nyub/a01nyub.cpp

@@ -6,7 +6,7 @@
 
 namespace esphome::a01nyub {
 
-static const char *const TAG = "a01nyub.sensor";
+ESPHOME_LOG_TAG(TAG, "a01nyub.sensor");
 
 void A01nyubComponent::loop() {
   uint8_t data;

esphome/components/a02yyuw/__init__.py

@@ -1 +1,2 @@
 CODEOWNERS = ["@TH-Braemer"]
+DOMAIN = "a02yyuw"

esphome/components/a02yyuw/a02yyuw.cpp

@@ -6,7 +6,7 @@
 
 namespace esphome::a02yyuw {
 
-static const char *const TAG = "a02yyuw.sensor";
+ESPHOME_LOG_TAG(TAG, "a02yyuw.sensor");
 
 void A02yyuwComponent::loop() {
   uint8_t data;

esphome/components/a4988/__init__.py

@@ -0,0 +1 @@
+DOMAIN = "a4988"

esphome/components/a4988/a4988.cpp

@@ -3,7 +3,7 @@
 
 namespace esphome::a4988 {
 
-static const char *const TAG = "a4988.stepper";
+ESPHOME_LOG_TAG(TAG, "a4988.stepper");
 
 void A4988::setup() {
   if (this->sleep_pin_ != nullptr) {

esphome/components/absolute_humidity/__init__.py

@@ -1 +1,2 @@
 CODEOWNERS = ["@DAVe3283"]
+DOMAIN = "absolute_humidity"

esphome/components/absolute_humidity/absolute_humidity.cpp

@@ -3,7 +3,7 @@
 
 namespace esphome::absolute_humidity {
 
-static const char *const TAG{"absolute_humidity.sensor"};
+ESPHOME_LOG_TAG(TAG, "absolute_humidity.sensor");
 
 void AbsoluteHumidityComponent::setup() {
   this->temperature_sensor_->add_on_state_callback([this](float state) {

esphome/components/ac_dimmer/__init__.py

@@ -0,0 +1 @@
+DOMAIN = "ac_dimmer"

esphome/components/ac_dimmer/ac_dimmer.cpp

@@ -14,7 +14,7 @@
 
 namespace esphome::ac_dimmer {
 
-static const char *const TAG = "ac_dimmer";
+ESPHOME_LOG_TAG(TAG, "ac_dimmer");
 
 // Global array to store dimmer objects
 static AcDimmerDataStore *all_dimmers[32];  // NOLINT(cppcoreguidelines-avoid-non-const-global-variables)

esphome/components/ac_dimmer/hw_timer_esp_idf.cpp

@@ -9,7 +9,7 @@
 #include "esp_clk_tree.h"
 #include "soc/clk_tree_defs.h"
 
-static const char *const TAG = "hw_timer_esp_idf";
+ESPHOME_LOG_TAG(TAG, "hw_timer_esp_idf");
 
 namespace esphome::ac_dimmer {
 

esphome/components/adalight/__init__.py

@@ -9,6 +9,7 @@ from esphome.cpp_generator import MockObj
 from esphome.types import ConfigType
 
 DEPENDENCIES = ["uart"]
+DOMAIN = "adalight"
 
 adalight_ns = cg.esphome_ns.namespace("adalight")
 AdalightLightEffect = adalight_ns.class_(

esphome/components/adalight/adalight_light_effect.cpp

@@ -3,7 +3,7 @@
 
 namespace esphome::adalight {
 
-static const char *const TAG = "adalight_light_effect";
+ESPHOME_LOG_TAG(TAG, "adalight_light_effect");
 
 static const uint32_t ADALIGHT_ACK_INTERVAL = 1000;
 static const uint32_t ADALIGHT_RECEIVE_TIMEOUT = 1000;

esphome/components/adc/__init__.py

@@ -22,6 +22,7 @@ from esphome.core import CORE
 from esphome.types import ConfigType
 
 CODEOWNERS = ["@esphome/core"]
+DOMAIN = "adc"
 
 adc_ns = cg.esphome_ns.namespace("adc")
 

esphome/components/adc/adc_sensor.h

@@ -94,7 +94,7 @@ class ADCSensor final : public sensor::Sensor, public PollingComponent, public v
   ///   - SamplingMode::MIN: Use the lowest sample value
   ///   - SamplingMode::MAX: Use the highest sample value
   /// @param sampling_mode The desired sampling mode to use for aggregating ADC samples.
-  void set_sampling_mode(SamplingMode sampling_mode);
+  void set_sampling_mode(SamplingMode sampling_mode) { this->sampling_mode_ = sampling_mode; }
 
   /// Perform a single ADC sampling operation and return the measured value.
   /// This function handles raw readings, calibration, and averaging as needed.
@@ -128,10 +128,10 @@ class ADCSensor final : public sensor::Sensor, public PollingComponent, public v
 #endif  // USE_RP2
 
  protected:
-  uint8_t sample_count_{1};
-  bool output_raw_{false};
+  uint8_t sample_count_{1};  // Must match DEFAULT_SAMPLES in sensor.py
+  bool output_raw_{false};   // Must match CONF_RAW default in sensor.py
   InternalGPIOPin *pin_;
-  SamplingMode sampling_mode_{SamplingMode::AVG};
+  SamplingMode sampling_mode_{SamplingMode::AVG};  // Must match DEFAULT_SAMPLING_MODE in sensor.py
 
 #ifdef USE_ESP32
   float sample_autorange_();
@@ -139,7 +139,7 @@ class ADCSensor final : public sensor::Sensor, public PollingComponent, public v
   bool autorange_{false};
   adc_oneshot_unit_handle_t adc_handle_{nullptr};
   adc_cali_handle_t calibration_handle_{nullptr};
-  adc_atten_t attenuation_{ADC_ATTEN_DB_0};
+  adc_atten_t attenuation_{ADC_ATTEN_DB_0};  // Must match DEFAULT_ATTENUATION in sensor.py
   adc_channel_t channel_{};
   adc_unit_t adc_unit_{};
   struct SetupFlags {

esphome/components/adc/adc_sensor_common.cpp

@@ -3,7 +3,7 @@
 
 namespace esphome::adc {
 
-static const char *const TAG = "adc";
+ESPHOME_LOG_TAG(TAG, "adc");
 
 const LogString *sampling_mode_to_str(SamplingMode mode) {
   switch (mode) {
@@ -76,6 +76,4 @@ void ADCSensor::set_sample_count(uint8_t sample_count) {
   }
 }
 
-void ADCSensor::set_sampling_mode(SamplingMode sampling_mode) { this->sampling_mode_ = sampling_mode; }
-
 }  // namespace esphome::adc

esphome/components/adc/adc_sensor_esp32.cpp

@@ -6,7 +6,7 @@
 
 namespace esphome::adc {
 
-static const char *const TAG = "adc";
+ESPHOME_LOG_TAG(TAG, "adc");
 
 adc_oneshot_unit_handle_t ADCSensor::shared_adc_handles[2] = {nullptr, nullptr};
 

esphome/components/adc/adc_sensor_esp8266.cpp

@@ -13,7 +13,7 @@ ADC_MODE(ADC_VCC)
 
 namespace esphome::adc {
 
-static const char *const TAG = "adc";
+ESPHOME_LOG_TAG(TAG, "adc");
 
 void ADCSensor::setup() {
 #ifndef USE_ADC_SENSOR_VCC

esphome/components/adc/adc_sensor_libretiny.cpp

@@ -5,7 +5,7 @@
 
 namespace esphome::adc {
 
-static const char *const TAG = "adc";
+ESPHOME_LOG_TAG(TAG, "adc");
 
 void ADCSensor::setup() {
 #ifndef USE_ADC_SENSOR_VCC

esphome/components/adc/adc_sensor_rp2.cpp

@@ -17,7 +17,7 @@
 
 namespace esphome::adc {
 
-static const char *const TAG = "adc";
+ESPHOME_LOG_TAG(TAG, "adc");
 
 // The on-die temperature sensor sits on the last ADC channel: input 4 on RP2040
 // and RP2350A, but input 8 on RP2350B, which has eight external channels rather

esphome/components/adc/adc_sensor_zephyr.cpp

@@ -7,7 +7,7 @@
 
 namespace esphome::adc {
 
-static const char *const TAG = "adc";
+ESPHOME_LOG_TAG(TAG, "adc");
 
 void ADCSensor::setup() {
   if (!adc_is_ready_dt(this->channel_)) {

esphome/components/adc/sensor.py

@@ -48,6 +48,11 @@ AUTO_LOAD = ["voltage_sampler"]
 CONF_SAMPLES = "samples"
 CONF_SAMPLING_MODE = "sampling_mode"
 
+# Keep in sync with the initializers in adc_sensor.h
+DEFAULT_ATTENUATION = "0db"
+DEFAULT_SAMPLING_MODE = "avg"
+DEFAULT_SAMPLES = 1
+
 
 _attenuation = cv.enum(ATTENUATION_MODES, lower=True)
 _sampling_mode = cv.enum(SAMPLING_MODES, lower=True)
@@ -65,7 +70,10 @@ def validate_config(config: ConfigType) -> ConfigType:
     ):
         raise cv.Invalid("ESP32-S31 only supports 'attenuation: 0db'")
 
-    if config.get(CONF_ATTENUATION, None) == "auto" and config.get(CONF_SAMPLES, 1) > 1:
+    if (
+        config.get(CONF_ATTENUATION, None) == "auto"
+        and config.get(CONF_SAMPLES, DEFAULT_SAMPLES) > 1
+    ):
         raise cv.Invalid(
             "Automatic attenuation cannot be used when multisampling is set"
         )
@@ -113,12 +121,16 @@ CONFIG_SCHEMA = cv.All(
         {
             cv.Required(CONF_PIN): validate_adc_pin,
             cv.Optional(CONF_RAW, default=False): cv.boolean,
-            cv.SplitDefault(CONF_ATTENUATION, esp32="0db"): cv.All(
+            cv.SplitDefault(CONF_ATTENUATION, esp32=DEFAULT_ATTENUATION): cv.All(
                 cv.only_on_esp32, _attenuation
             ),
             cv.OnlyWith(CONF_NRF_SAADC, PLATFORM_NRF52): cv.declare_id(adc_dt_spec),
-            cv.Optional(CONF_SAMPLES, default=1): cv.int_range(min=1, max=255),
-            cv.Optional(CONF_SAMPLING_MODE, default="avg"): _sampling_mode,
+            cv.Optional(CONF_SAMPLES, default=DEFAULT_SAMPLES): cv.int_range(
+                min=1, max=255
+            ),
+            cv.Optional(
+                CONF_SAMPLING_MODE, default=DEFAULT_SAMPLING_MODE
+            ): _sampling_mode,
         }
     )
     .extend(cv.polling_component_schema("60s")),
@@ -155,9 +167,13 @@ async def to_code(config: ConfigType) -> None:
         pin = await cg.gpio_pin_expression(config[CONF_PIN])
         cg.add(var.set_pin(pin))
 
-    cg.add(var.set_output_raw(config[CONF_RAW]))
-    cg.add(var.set_sample_count(config[CONF_SAMPLES]))
-    cg.add(var.set_sampling_mode(config[CONF_SAMPLING_MODE]))
+    # Skip the setters when the config matches the C++ initializers.
+    if config[CONF_RAW]:
+        cg.add(var.set_output_raw(True))
+    if (samples := config[CONF_SAMPLES]) != DEFAULT_SAMPLES:
+        cg.add(var.set_sample_count(samples))
+    if (sampling_mode := config[CONF_SAMPLING_MODE]) != DEFAULT_SAMPLING_MODE:
+        cg.add(var.set_sampling_mode(sampling_mode))
 
     if CORE.is_esp32:
         # Re-enable ESP-IDF's ADC driver (excluded by default to save compile time)
@@ -166,7 +182,7 @@ async def to_code(config: ConfigType) -> None:
         if attenuation := config.get(CONF_ATTENUATION):
             if attenuation == "auto":
                 cg.add(var.set_autorange(cg.global_ns.true))
-            else:
+            elif attenuation != DEFAULT_ATTENUATION:
                 cg.add(var.set_attenuation(attenuation))
 
         variant = get_esp32_variant()

esphome/components/adc128s102/__init__.py

@@ -5,6 +5,7 @@ from esphome.const import CONF_ID
 from esphome.types import ConfigType
 
 DEPENDENCIES = ["spi"]
+DOMAIN = "adc128s102"
 MULTI_CONF = True
 CODEOWNERS = ["@DeerMaximum"]
 

esphome/components/adc128s102/adc128s102.cpp

@@ -3,7 +3,7 @@
 
 namespace esphome::adc128s102 {
 
-static const char *const TAG = "adc128s102";
+ESPHOME_LOG_TAG(TAG, "adc128s102");
 
 float ADC128S102::get_setup_priority() const { return setup_priority::HARDWARE; }
 

esphome/components/adc128s102/sensor/adc128s102_sensor.cpp

@@ -4,7 +4,7 @@
 
 namespace esphome::adc128s102 {
 
-static const char *const TAG = "adc128s102.sensor";
+ESPHOME_LOG_TAG(TAG, "adc128s102.sensor");
 
 ADC128S102Sensor::ADC128S102Sensor(uint8_t channel) : channel_(channel) {}
 

esphome/components/addressable_light/__init__.py

@@ -0,0 +1 @@
+DOMAIN = "addressable_light"

esphome/components/addressable_light/addressable_light_display.cpp

@@ -3,7 +3,7 @@
 
 namespace esphome::addressable_light {
 
-static const char *const TAG = "addressable_light.display";
+ESPHOME_LOG_TAG(TAG, "addressable_light.display");
 
 int AddressableLightDisplay::get_width_internal() { return this->width_; }
 int AddressableLightDisplay::get_height_internal() { return this->height_; }

esphome/components/ade7880/__init__.py

@@ -0,0 +1 @@
+DOMAIN = "ade7880"

esphome/components/ade7880/ade7880.cpp

@@ -15,7 +15,7 @@
 
 namespace esphome::ade7880 {
 
-static const char *const TAG = "ade7880";
+ESPHOME_LOG_TAG(TAG, "ade7880");
 
 void IRAM_ATTR ADE7880Store::gpio_intr(ADE7880Store *arg) { arg->reset_done = true; }
 

esphome/components/ade7953/__init__.py

@@ -1 +1,2 @@
 CODEOWNERS = ["@angelnu"]
+DOMAIN = "ade7953"

esphome/components/ade7953_base/__init__.py

@@ -27,6 +27,7 @@ from esphome.cpp_generator import MockObj
 from esphome.types import ConfigType
 
 CODEOWNERS = ["@angelnu"]
+DOMAIN = "ade7953_base"
 
 CONF_CURRENT_A = "current_a"
 CONF_CURRENT_B = "current_b"

esphome/components/ade7953_base/ade7953_base.cpp

@@ -5,7 +5,7 @@
 
 namespace esphome::ade7953_base {
 
-static const char *const TAG = "ade7953";
+ESPHOME_LOG_TAG(TAG, "ade7953");
 
 constexpr uint16_t CONFIG_DEFAULT = 0x8004u;
 constexpr uint16_t CONFIG_LOCK_BIT = 0x8000u;

esphome/components/ade7953_i2c/__init__.py

@@ -1 +1,2 @@
 CODEOWNERS = ["@angelnu"]
+DOMAIN = "ade7953_i2c"

esphome/components/ade7953_i2c/ade7953_i2c.cpp

@@ -4,7 +4,7 @@
 
 namespace esphome::ade7953_i2c {
 
-static const char *const TAG = "ade7953";
+ESPHOME_LOG_TAG(TAG, "ade7953");
 
 void AdE7953I2c::dump_config() {
   ESP_LOGCONFIG(TAG, "ADE7953_i2c:");

esphome/components/ade7953_spi/__init__.py

@@ -1 +1,2 @@
 CODEOWNERS = ["@angelnu"]
+DOMAIN = "ade7953_spi"

esphome/components/ade7953_spi/ade7953_spi.cpp

@@ -4,7 +4,7 @@
 
 namespace esphome::ade7953_spi {
 
-static const char *const TAG = "ade7953";
+ESPHOME_LOG_TAG(TAG, "ade7953");
 
 // Datasheet requires at least 1.2µs after clearing CONFIG LOCK_BIT before raising CS
 constexpr uint8_t CONFIG_LOCK_SETTLE_US = 2;

esphome/components/ads1115/__init__.py

@@ -5,6 +5,7 @@ from esphome.const import CONF_ID
 from esphome.types import ConfigType
 
 DEPENDENCIES = ["i2c"]
+DOMAIN = "ads1115"
 MULTI_CONF = True
 
 ads1115_ns = cg.esphome_ns.namespace("ads1115")

esphome/components/ads1115/ads1115.cpp

@@ -4,7 +4,7 @@
 
 namespace esphome::ads1115 {
 
-static const char *const TAG = "ads1115";
+ESPHOME_LOG_TAG(TAG, "ads1115");
 static const uint8_t ADS1115_REGISTER_CONVERSION = 0x00;
 static const uint8_t ADS1115_REGISTER_CONFIG = 0x01;
 

esphome/components/ads1115/sensor/ads1115_sensor.cpp

@@ -4,7 +4,7 @@
 
 namespace esphome::ads1115 {
 
-static const char *const TAG = "ads1115.sensor";
+ESPHOME_LOG_TAG(TAG, "ads1115.sensor");
 
 float ADS1115Sensor::sample() {
   return this->parent_->request_measurement(this->multiplexer_, this->gain_, this->resolution_, this->samplerate_);

esphome/components/ads1118/__init__.py

@@ -6,6 +6,7 @@ from esphome.types import ConfigType
 
 CODEOWNERS = ["@solomondg1"]
 DEPENDENCIES = ["spi"]
+DOMAIN = "ads1118"
 MULTI_CONF = True
 
 CONF_ADS1118_ID = "ads1118_id"