Compare commits

..

2 Commits

Author SHA1 Message Date
J. Nick Koston 3a4f67def8 [core] Fix delay on failed component being dropped; DRY the is_failed check
The is_failed() skip exists in two execution paths: the heap loop in call()
and should_skip_item_() (defer queue / delay:0). The previous commit only
exempted SELF_POINTER items from the heap-path check, so on multi-threaded
platforms a delay:0 continuation whose host component had failed would still
be silently dropped.

Extract a single is_item_failed_() helper (with the SELF_POINTER exemption)
and use it from both paths so they cannot drift again.

Add an integration test that schedules a delay from a component that marks
itself failed and asserts the continuation still fires (verified to fail
without the exemption).
2026-06-02 13:54:17 -05:00
J. Nick Koston 5b728f19c3 [core] Attribute "took a long time" blocking warning to its source
A blocking operation that runs inside a deferred scheduler continuation
(e.g. after a delay in a script/automation) was reported as:

    <null> took a long time for an operation (83 ms), max is 30 ms

Two problems:

* The DelayAction continuation carries no component (since #16129 dropped
  Component inheritance), so the warning had nothing to name and printed
  "<null>". Telling the user an anonymous delay action is blocking is not
  useful; naming the component that hosts the automation is.
* The threshold was hardcoded to "30 ms" but the real default is 50 ms
  (WARN_IF_BLOCKING_OVER_CS) and is adaptive per component.

DelayAction now records App.get_current_component() on the scheduler item,
so the warning names the component whose automation chain hit the delay
(falling back to "a scheduled task" when there is genuinely no current
component). This propagates across chained delays because the scheduler
restores the item's component as the current component before each callback.

For SELF_POINTER items the stored component is log-attribution only: the
key (the caller's `this`) is globally unique, so matches_item_locked_
ignores the component when matching and the is_failed() skip is bypassed.
This keeps delay cancellation (restart/parallel/stop) and always-fire
semantics unchanged.

The warning now reports the real (pre-ratchet) threshold instead of the
stale "30 ms".

Adds an integration test reproducing the deferred-block path via an
interval + delay + busy lambda and asserting the warning names a component
and reports "max is 50 ms".
2026-06-02 13:29:27 -05:00
92 changed files with 1270 additions and 3180 deletions
+1 -1
View File
@@ -1 +1 @@
0550a8ea4182dbc007660de060dd023ce22c865c8e95040a36f3d07a5b354fc6
27aaab4e0ebfc10491720345aa746fc2dffa6a3985f73ec111b12dd99078d46f
+1 -1
View File
@@ -32,7 +32,7 @@ runs:
# detects the activated venv via ``VIRTUAL_ENV`` so the venv layout
# downstream jobs rely on is preserved.
if: steps.cache-venv.outputs.cache-hit != 'true'
uses: astral-sh/setup-uv@fac544c07dec837d0ccb6301d7b5580bf5edae39 # v8.2.0
uses: astral-sh/setup-uv@08807647e7069bb48b6ef5acd8ec9567f424441b # v8.1.0
with:
enable-cache: true
# Pin uv version so the action does not have to fetch the
+1 -1
View File
@@ -24,7 +24,7 @@ jobs:
if: github.event.pull_request.state == 'open' && (github.event.action != 'labeled' || github.event.sender.type != 'Bot')
steps:
- name: Checkout
uses: actions/checkout@df4cb1c069e1874edd31b4311f1884172cec0e10 # v6.0.3
uses: actions/checkout@de0fac2e4500dabe0009e67214ff5f5447ce83dd # v6.0.2
- name: Generate a token
id: generate-token
+2 -2
View File
@@ -21,7 +21,7 @@ jobs:
runs-on: ubuntu-latest
steps:
- name: Checkout
uses: actions/checkout@df4cb1c069e1874edd31b4311f1884172cec0e10 # v6.0.3
uses: actions/checkout@de0fac2e4500dabe0009e67214ff5f5447ce83dd # v6.0.2
- name: Set up Python
uses: actions/setup-python@a309ff8b426b58ec0e2a45f0f869d46889d02405 # v6.2.0
with:
@@ -29,7 +29,7 @@ jobs:
- name: Set up uv
# ``--system`` (below) installs into the setup-python interpreter;
# no venv is created or restored by this workflow.
uses: astral-sh/setup-uv@fac544c07dec837d0ccb6301d7b5580bf5edae39 # v8.2.0
uses: astral-sh/setup-uv@08807647e7069bb48b6ef5acd8ec9567f424441b # v8.1.0
with:
enable-cache: true
# Pin uv version so the action does not have to fetch the
+1 -1
View File
@@ -21,7 +21,7 @@ jobs:
runs-on: ubuntu-latest
steps:
- name: Checkout
uses: actions/checkout@df4cb1c069e1874edd31b4311f1884172cec0e10 # v6.0.3
uses: actions/checkout@de0fac2e4500dabe0009e67214ff5f5447ce83dd # v6.0.2
- name: Set up Python
uses: actions/setup-python@a309ff8b426b58ec0e2a45f0f869d46889d02405 # v6.2.0
+1 -1
View File
@@ -42,7 +42,7 @@ jobs:
- "docker"
# - "lint"
steps:
- uses: actions/checkout@df4cb1c069e1874edd31b4311f1884172cec0e10 # v6.0.3
- uses: actions/checkout@de0fac2e4500dabe0009e67214ff5f5447ce83dd # v6.0.2
- name: Set up Python
uses: actions/setup-python@a309ff8b426b58ec0e2a45f0f869d46889d02405 # v6.2.0
with:
+1 -1
View File
@@ -20,7 +20,7 @@ jobs:
runs-on: ubuntu-latest
steps:
- name: Check out code from GitHub
uses: actions/checkout@df4cb1c069e1874edd31b4311f1884172cec0e10 # v6.0.3
uses: actions/checkout@de0fac2e4500dabe0009e67214ff5f5447ce83dd # v6.0.2
- name: Run tests
working-directory: .github/scripts/auto-label-pr
@@ -49,7 +49,7 @@ jobs:
- name: Check out code from base repository
if: steps.pr.outputs.skip != 'true'
uses: actions/checkout@df4cb1c069e1874edd31b4311f1884172cec0e10 # v6.0.3
uses: actions/checkout@de0fac2e4500dabe0009e67214ff5f5447ce83dd # v6.0.2
with:
# Always check out from the base repository (esphome/esphome), never from forks
# Use the PR's target branch to ensure we run trusted code from the main repo
+25 -28
View File
@@ -28,7 +28,7 @@ jobs:
cache-key: ${{ steps.cache-key.outputs.key }}
steps:
- name: Check out code from GitHub
uses: actions/checkout@df4cb1c069e1874edd31b4311f1884172cec0e10 # v6.0.3
uses: actions/checkout@de0fac2e4500dabe0009e67214ff5f5447ce83dd # v6.0.2
- name: Generate cache-key
id: cache-key
run: echo key="${{ hashFiles('requirements.txt', 'requirements_dev.txt', 'requirements_test.txt', '.pre-commit-config.yaml') }}" >> $GITHUB_OUTPUT
@@ -49,7 +49,7 @@ jobs:
# detects the activated venv via ``VIRTUAL_ENV`` so downstream jobs
# that ``. venv/bin/activate`` see an identical layout.
if: steps.cache-venv.outputs.cache-hit != 'true'
uses: astral-sh/setup-uv@fac544c07dec837d0ccb6301d7b5580bf5edae39 # v8.2.0
uses: astral-sh/setup-uv@08807647e7069bb48b6ef5acd8ec9567f424441b # v8.1.0
with:
enable-cache: true
# Pin uv version so the action does not have to fetch the
@@ -74,7 +74,7 @@ jobs:
if: needs.determine-jobs.outputs.python-linters == 'true'
steps:
- name: Check out code from GitHub
uses: actions/checkout@df4cb1c069e1874edd31b4311f1884172cec0e10 # v6.0.3
uses: actions/checkout@de0fac2e4500dabe0009e67214ff5f5447ce83dd # v6.0.2
- name: Restore Python
uses: ./.github/actions/restore-python
with:
@@ -97,7 +97,7 @@ jobs:
if: needs.determine-jobs.outputs.core-ci == 'true'
steps:
- name: Check out code from GitHub
uses: actions/checkout@df4cb1c069e1874edd31b4311f1884172cec0e10 # v6.0.3
uses: actions/checkout@de0fac2e4500dabe0009e67214ff5f5447ce83dd # v6.0.2
- name: Restore Python
uses: ./.github/actions/restore-python
with:
@@ -123,7 +123,7 @@ jobs:
if: needs.determine-jobs.outputs.import-time == 'true'
steps:
- name: Check out code from GitHub
uses: actions/checkout@df4cb1c069e1874edd31b4311f1884172cec0e10 # v6.0.3
uses: actions/checkout@de0fac2e4500dabe0009e67214ff5f5447ce83dd # v6.0.2
- name: Restore Python
uses: ./.github/actions/restore-python
with:
@@ -151,11 +151,11 @@ jobs:
if: needs.determine-jobs.outputs.device-builder == 'true'
steps:
- name: Check out esphome (this PR)
uses: actions/checkout@df4cb1c069e1874edd31b4311f1884172cec0e10 # v6.0.3
uses: actions/checkout@de0fac2e4500dabe0009e67214ff5f5447ce83dd # v6.0.2
with:
path: esphome
- name: Check out esphome/device-builder
uses: actions/checkout@df4cb1c069e1874edd31b4311f1884172cec0e10 # v6.0.3
uses: actions/checkout@de0fac2e4500dabe0009e67214ff5f5447ce83dd # v6.0.2
with:
repository: esphome/device-builder
ref: main
@@ -170,7 +170,7 @@ jobs:
# install step (order-of-magnitude faster on cold boots,
# with its own wheel cache). actions/setup-python still
# provides the interpreter.
uses: astral-sh/setup-uv@fac544c07dec837d0ccb6301d7b5580bf5edae39 # v8.2.0
uses: astral-sh/setup-uv@08807647e7069bb48b6ef5acd8ec9567f424441b # v8.1.0
with:
enable-cache: true
# Pin uv version so the action does not have to fetch the
@@ -189,12 +189,9 @@ jobs:
- name: Run device-builder pytest
# ``-n auto`` runs under pytest-xdist (matches device-builder's
# own CI). No ``--cov`` here -- this is purely a downstream
# smoke check against this PR's esphome code. ``tests/e2e/slow``
# is excluded: those are real multi-minute toolchain compiles
# (LibreTiny SDK clone, native ESP-IDF install) that device-builder
# runs in its own dedicated jobs, not this smoke check.
# smoke check against this PR's esphome code.
working-directory: device-builder
run: pytest -q -n auto --maxfail=5 --durations=30 --no-cov --ignore=tests/benchmarks --ignore=tests/e2e/slow
run: pytest -q -n auto --maxfail=5 --durations=30 --no-cov --ignore=tests/benchmarks
pytest:
name: Run pytest
@@ -224,7 +221,7 @@ jobs:
if: needs.determine-jobs.outputs.core-ci == 'true'
steps:
- name: Check out code from GitHub
uses: actions/checkout@df4cb1c069e1874edd31b4311f1884172cec0e10 # v6.0.3
uses: actions/checkout@de0fac2e4500dabe0009e67214ff5f5447ce83dd # v6.0.2
- name: Restore Python
id: restore-python
uses: ./.github/actions/restore-python
@@ -284,7 +281,7 @@ jobs:
benchmarks: ${{ steps.determine.outputs.benchmarks }}
steps:
- name: Check out code from GitHub
uses: actions/checkout@df4cb1c069e1874edd31b4311f1884172cec0e10 # v6.0.3
uses: actions/checkout@de0fac2e4500dabe0009e67214ff5f5447ce83dd # v6.0.2
with:
# Fetch enough history to find the merge base
fetch-depth: 2
@@ -356,7 +353,7 @@ jobs:
bucket: ${{ fromJson(needs.determine-jobs.outputs.integration-test-buckets) }}
steps:
- name: Check out code from GitHub
uses: actions/checkout@df4cb1c069e1874edd31b4311f1884172cec0e10 # v6.0.3
uses: actions/checkout@de0fac2e4500dabe0009e67214ff5f5447ce83dd # v6.0.2
- name: Set up Python 3.13
id: python
uses: actions/setup-python@a309ff8b426b58ec0e2a45f0f869d46889d02405 # v6.2.0
@@ -371,7 +368,7 @@ jobs:
- name: Set up uv
# Only needed on cache miss to populate the venv.
if: steps.cache-venv.outputs.cache-hit != 'true'
uses: astral-sh/setup-uv@fac544c07dec837d0ccb6301d7b5580bf5edae39 # v8.2.0
uses: astral-sh/setup-uv@08807647e7069bb48b6ef5acd8ec9567f424441b # v8.1.0
with:
enable-cache: true
# Pin uv version so the action does not have to fetch the
@@ -408,7 +405,7 @@ jobs:
if: github.event_name == 'pull_request' && (needs.determine-jobs.outputs.cpp-unit-tests-run-all == 'true' || needs.determine-jobs.outputs.cpp-unit-tests-components != '[]')
steps:
- name: Check out code from GitHub
uses: actions/checkout@df4cb1c069e1874edd31b4311f1884172cec0e10 # v6.0.3
uses: actions/checkout@de0fac2e4500dabe0009e67214ff5f5447ce83dd # v6.0.2
- name: Restore Python
uses: ./.github/actions/restore-python
@@ -437,7 +434,7 @@ jobs:
(github.event_name == 'pull_request' && needs.determine-jobs.outputs.benchmarks == 'true')
steps:
- name: Check out code from GitHub
uses: actions/checkout@df4cb1c069e1874edd31b4311f1884172cec0e10 # v6.0.3
uses: actions/checkout@de0fac2e4500dabe0009e67214ff5f5447ce83dd # v6.0.2
- name: Restore Python
uses: ./.github/actions/restore-python
@@ -493,7 +490,7 @@ jobs:
steps:
- name: Check out code from GitHub
uses: actions/checkout@df4cb1c069e1874edd31b4311f1884172cec0e10 # v6.0.3
uses: actions/checkout@de0fac2e4500dabe0009e67214ff5f5447ce83dd # v6.0.2
with:
# Need history for HEAD~1 to work for checking changed files
fetch-depth: 2
@@ -578,7 +575,7 @@ jobs:
GH_TOKEN: ${{ github.token }}
steps:
- name: Check out code from GitHub
uses: actions/checkout@df4cb1c069e1874edd31b4311f1884172cec0e10 # v6.0.3
uses: actions/checkout@de0fac2e4500dabe0009e67214ff5f5447ce83dd # v6.0.2
with:
# Need history for HEAD~1 to work for checking changed files
fetch-depth: 2
@@ -673,7 +670,7 @@ jobs:
steps:
- name: Check out code from GitHub
uses: actions/checkout@df4cb1c069e1874edd31b4311f1884172cec0e10 # v6.0.3
uses: actions/checkout@de0fac2e4500dabe0009e67214ff5f5447ce83dd # v6.0.2
with:
# Need history for HEAD~1 to work for checking changed files
fetch-depth: 2
@@ -767,7 +764,7 @@ jobs:
version: 1.0
- name: Check out code from GitHub
uses: actions/checkout@df4cb1c069e1874edd31b4311f1884172cec0e10 # v6.0.3
uses: actions/checkout@de0fac2e4500dabe0009e67214ff5f5447ce83dd # v6.0.2
- name: Restore Python
uses: ./.github/actions/restore-python
with:
@@ -892,7 +889,7 @@ jobs:
TEST_COMPONENTS: ${{ needs.determine-jobs.outputs.native-idf-components }}
steps:
- name: Check out code from GitHub
uses: actions/checkout@df4cb1c069e1874edd31b4311f1884172cec0e10 # v6.0.3
uses: actions/checkout@de0fac2e4500dabe0009e67214ff5f5447ce83dd # v6.0.2
- name: Restore Python
uses: ./.github/actions/restore-python
@@ -974,7 +971,7 @@ jobs:
if: github.event_name == 'pull_request' && !startsWith(github.base_ref, 'beta') && !startsWith(github.base_ref, 'release') && needs.determine-jobs.outputs.core-ci == 'true'
steps:
- name: Check out code from GitHub
uses: actions/checkout@df4cb1c069e1874edd31b4311f1884172cec0e10 # v6.0.3
uses: actions/checkout@de0fac2e4500dabe0009e67214ff5f5447ce83dd # v6.0.2
- name: Restore Python
uses: ./.github/actions/restore-python
with:
@@ -1000,7 +997,7 @@ jobs:
skip: ${{ steps.check-script.outputs.skip || steps.check-tests.outputs.skip }}
steps:
- name: Check out target branch
uses: actions/checkout@df4cb1c069e1874edd31b4311f1884172cec0e10 # v6.0.3
uses: actions/checkout@de0fac2e4500dabe0009e67214ff5f5447ce83dd # v6.0.2
with:
ref: ${{ github.base_ref }}
@@ -1182,7 +1179,7 @@ jobs:
flash_usage: ${{ steps.extract.outputs.flash_usage }}
steps:
- name: Check out PR branch
uses: actions/checkout@df4cb1c069e1874edd31b4311f1884172cec0e10 # v6.0.3
uses: actions/checkout@de0fac2e4500dabe0009e67214ff5f5447ce83dd # v6.0.2
- name: Restore Python
uses: ./.github/actions/restore-python
with:
@@ -1251,7 +1248,7 @@ jobs:
GH_TOKEN: ${{ github.token }}
steps:
- name: Check out code
uses: actions/checkout@df4cb1c069e1874edd31b4311f1884172cec0e10 # v6.0.3
uses: actions/checkout@de0fac2e4500dabe0009e67214ff5f5447ce83dd # v6.0.2
- name: Restore Python
uses: ./.github/actions/restore-python
with:
@@ -26,7 +26,7 @@ jobs:
runs-on: ubuntu-latest
steps:
- name: Checkout base branch
uses: actions/checkout@df4cb1c069e1874edd31b4311f1884172cec0e10 # v6.0.3
uses: actions/checkout@de0fac2e4500dabe0009e67214ff5f5447ce83dd # v6.0.2
with:
ref: ${{ github.event.pull_request.base.sha }}
sparse-checkout: |
@@ -29,7 +29,7 @@ jobs:
runs-on: ubuntu-latest
steps:
- name: Checkout base branch
uses: actions/checkout@df4cb1c069e1874edd31b4311f1884172cec0e10 # v6.0.3
uses: actions/checkout@de0fac2e4500dabe0009e67214ff5f5447ce83dd # v6.0.2
with:
ref: ${{ github.event.pull_request.base.sha }}
+3 -3
View File
@@ -52,11 +52,11 @@ jobs:
# your codebase is analyzed, see https://docs.github.com/en/code-security/code-scanning/creating-an-advanced-setup-for-code-scanning/codeql-code-scanning-for-compiled-languages
steps:
- name: Checkout repository
uses: actions/checkout@df4cb1c069e1874edd31b4311f1884172cec0e10 # v6.0.3
uses: actions/checkout@de0fac2e4500dabe0009e67214ff5f5447ce83dd # v6.0.2
# Initializes the CodeQL tools for scanning.
- name: Initialize CodeQL
uses: github/codeql-action/init@87557b9c84dde89fdd9b10e88954ac2f4248e463 # v4.36.1
uses: github/codeql-action/init@7211b7c8077ea37d8641b6271f6a365a22a5fbfa # v4.36.0
with:
languages: ${{ matrix.language }}
build-mode: ${{ matrix.build-mode }}
@@ -84,6 +84,6 @@ jobs:
exit 1
- name: Perform CodeQL Analysis
uses: github/codeql-action/analyze@87557b9c84dde89fdd9b10e88954ac2f4248e463 # v4.36.1
uses: github/codeql-action/analyze@7211b7c8077ea37d8641b6271f6a365a22a5fbfa # v4.36.0
with:
category: "/language:${{matrix.language}}"
+1 -1
View File
@@ -16,7 +16,7 @@ jobs:
name: Validate PR title
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@df4cb1c069e1874edd31b4311f1884172cec0e10 # v6.0.3
- uses: actions/checkout@de0fac2e4500dabe0009e67214ff5f5447ce83dd # v6.0.2
- uses: actions/github-script@3a2844b7e9c422d3c10d287c895573f7108da1b3 # v9.0.0
with:
+4 -4
View File
@@ -20,7 +20,7 @@ jobs:
branch_build: ${{ steps.tag.outputs.branch_build }}
deploy_env: ${{ steps.tag.outputs.deploy_env }}
steps:
- uses: actions/checkout@df4cb1c069e1874edd31b4311f1884172cec0e10 # v6.0.3
- uses: actions/checkout@de0fac2e4500dabe0009e67214ff5f5447ce83dd # v6.0.2
- name: Get tag
id: tag
# yamllint disable rule:line-length
@@ -60,7 +60,7 @@ jobs:
contents: read # actions/checkout to build the sdist/wheel
id-token: write # OIDC token for PyPI Trusted Publishing (pypa/gh-action-pypi-publish)
steps:
- uses: actions/checkout@df4cb1c069e1874edd31b4311f1884172cec0e10 # v6.0.3
- uses: actions/checkout@de0fac2e4500dabe0009e67214ff5f5447ce83dd # v6.0.2
- name: Set up Python
uses: actions/setup-python@a309ff8b426b58ec0e2a45f0f869d46889d02405 # v6.2.0
with:
@@ -92,7 +92,7 @@ jobs:
os: "ubuntu-24.04-arm"
steps:
- uses: actions/checkout@df4cb1c069e1874edd31b4311f1884172cec0e10 # v6.0.3
- uses: actions/checkout@de0fac2e4500dabe0009e67214ff5f5447ce83dd # v6.0.2
- name: Set up Python
uses: actions/setup-python@a309ff8b426b58ec0e2a45f0f869d46889d02405 # v6.2.0
with:
@@ -168,7 +168,7 @@ jobs:
- ghcr
- dockerhub
steps:
- uses: actions/checkout@df4cb1c069e1874edd31b4311f1884172cec0e10 # v6.0.3
- uses: actions/checkout@de0fac2e4500dabe0009e67214ff5f5447ce83dd # v6.0.2
- name: Download digests
uses: actions/download-artifact@3e5f45b2cfb9172054b4087a40e8e0b5a5461e7c # v8.0.1
+3 -3
View File
@@ -28,10 +28,10 @@ jobs:
permission-pull-requests: write # pulls.create / pulls.update to open or refresh the sync PR
- name: Checkout
uses: actions/checkout@df4cb1c069e1874edd31b4311f1884172cec0e10 # v6.0.3
uses: actions/checkout@de0fac2e4500dabe0009e67214ff5f5447ce83dd # v6.0.2
- name: Checkout Home Assistant
uses: actions/checkout@df4cb1c069e1874edd31b4311f1884172cec0e10 # v6.0.3
uses: actions/checkout@de0fac2e4500dabe0009e67214ff5f5447ce83dd # v6.0.2
with:
repository: home-assistant/core
path: lib/home-assistant
@@ -47,7 +47,7 @@ jobs:
# setup-python interpreter so subsequent ``pre-commit`` /
# ``script/run-in-env.py`` steps find the deps without a
# ``uv run`` prefix.
uses: astral-sh/setup-uv@fac544c07dec837d0ccb6301d7b5580bf5edae39 # v8.2.0
uses: astral-sh/setup-uv@08807647e7069bb48b6ef5acd8ec9567f424441b # v8.1.0
with:
enable-cache: true
# Pin uv version so the action does not have to fetch the
@@ -1,49 +0,0 @@
# Temporary workflow: proves the ESP-IDF spaced-path compile bug on Windows.
# Removed in the final commit of this PR; must never land on dev.
name: zz Windows spaced-path probe (temporary)
on:
push:
branches:
- esp32-idf-spaces-path-fix
workflow_dispatch:
permissions:
contents: read
jobs:
esp-idf-spaced-path:
name: ESP-IDF compile from a path with a space
runs-on: windows-latest
steps:
- name: Check out code from GitHub
uses: actions/checkout@df4cb1c069e1874edd31b4311f1884172cec0e10 # v6.0.3
- name: Set up Python
uses: actions/setup-python@a309ff8b426b58ec0e2a45f0f869d46889d02405 # v6.2.0
with:
python-version: "3.11"
- name: Install esphome
shell: bash
run: |
python -m pip install -U pip
pip install -e .
- name: Write a minimal ESP-IDF config into a directory with a space
shell: bash
run: |
mkdir -p "first last"
cat > "first last/probe.yaml" <<'EOF'
esphome:
name: spaces-probe
esp32:
board: esp32dev
framework:
type: esp-idf
logger:
EOF
echo "Config dir (note the space):"
pwd
ls -la "first last"
- name: Compile from the spaced path
shell: bash
run: |
esphome compile "first last/probe.yaml"
+1 -1
View File
@@ -63,7 +63,7 @@ repos:
name: Update clang-tidy hash
entry: python script/clang_tidy_hash.py --update-if-changed
language: python
files: ^(\.clang-tidy|platformio\.ini|requirements_dev\.txt|sdkconfig\.defaults|esphome/idf_component\.yml)$
files: ^(\.clang-tidy|platformio\.ini|requirements_dev\.txt)$
pass_filenames: false
additional_dependencies: []
- id: ci-custom
+2 -22
View File
@@ -695,11 +695,6 @@ def _wrap_to_code(name, comp, yaml_util):
def write_cpp(config: ConfigType) -> int:
from esphome import writer
# Refresh the storage sidecar and clean an incompatible previous build
# before regenerating any sources. This may full-wipe the build dir, so it
# has to run before write_cpp_file writes src/.
writer.update_storage_json()
if not get_bool_env(ENV_NOGITIGNORE):
writer.write_gitignore()
@@ -1428,16 +1423,7 @@ def command_wizard(args: ArgsProtocol) -> int | None:
def command_config(args: ArgsProtocol, config: ConfigType) -> int | None:
from esphome import yaml_util
if getattr(args, "no_defaults", False):
user_config = getattr(config, "user_config", None)
if user_config is None:
_LOGGER.warning(
"--no-defaults requested but the user-only config snapshot is "
"unavailable; falling back to the validated configuration."
)
else:
config = user_config
elif not CORE.verbose:
if not CORE.verbose:
config = strip_default_ids(config)
output = yaml_util.dump(config, args.show_secrets)
if not args.show_secrets:
@@ -1645,7 +1631,7 @@ def command_clean(args: ArgsProtocol, config: ConfigType) -> int | None:
from esphome import writer
try:
writer.clean_build(full=True)
writer.clean_build()
except OSError as err:
_LOGGER.error("Error deleting build files: %s", err)
return 1
@@ -2161,12 +2147,6 @@ def parse_args(argv):
parser_config.add_argument(
"--show-secrets", help="Show secrets in output.", action="store_true"
)
parser_config.add_argument(
"--no-defaults",
help="Only output the user-supplied configuration without "
"schema defaults applied.",
action="store_true",
)
parser_config_hash = subparsers.add_parser(
"config-hash", help="Calculate the hash of the configuration."
+6
View File
@@ -7,6 +7,7 @@ from esphome.components.esp32 import get_esp32_variant, idf_version
import esphome.config_validation as cv
from esphome.core import CORE
from esphome.helpers import mkdir_p, write_file_if_changed
from esphome.writer import update_storage_json
def get_available_components() -> list[str] | None:
@@ -212,6 +213,11 @@ target_link_options(${{COMPONENT_LIB}} PUBLIC
def write_project(minimal: bool = False) -> None:
"""Write ESP-IDF project files."""
# Refresh <data_dir>/storage/<name>.yaml.json so the dashboard's
# /info and /downloads endpoints can locate the build (they 404
# otherwise). This mirrors the PlatformIO build-gen path's call
# in build_gen/platformio.py:write_ini().
update_storage_json()
mkdir_p(CORE.build_path)
mkdir_p(CORE.relative_src_path())
+2 -1
View File
@@ -1,7 +1,7 @@
from esphome.const import __version__
from esphome.core import CORE
from esphome.helpers import mkdir_p, read_file, write_file_if_changed
from esphome.writer import find_begin_end
from esphome.writer import find_begin_end, update_storage_json
INI_AUTO_GENERATE_BEGIN = "; ========== AUTO GENERATED CODE BEGIN ==========="
INI_AUTO_GENERATE_END = "; =========== AUTO GENERATED CODE END ============"
@@ -58,6 +58,7 @@ def get_ini_content():
def write_ini(content):
update_storage_json()
path = CORE.relative_build_path("platformio.ini")
if path.is_file():
+1 -1
View File
@@ -2,7 +2,6 @@ import logging
from esphome import automation
import esphome.codegen as cg
from esphome.components.const import CONF_LOOP
import esphome.components.image as espImage
import esphome.config_validation as cv
from esphome.const import CONF_ID, CONF_REPEAT
@@ -15,6 +14,7 @@ DEPENDENCIES = ["display"]
MULTI_CONF = True
MULTI_CONF_NO_DEFAULT = True
CONF_LOOP = "loop"
CONF_START_FRAME = "start_frame"
CONF_END_FRAME = "end_frame"
CONF_FRAME = "frame"
-1
View File
@@ -15,7 +15,6 @@ CONF_DRAW_ROUNDING = "draw_rounding"
CONF_ENABLED = "enabled"
CONF_IGNORE_NOT_FOUND = "ignore_not_found"
CONF_LIBRETINY = "libretiny"
CONF_LOOP = "loop"
CONF_ON_PACKET = "on_packet"
CONF_ON_RECEIVE = "on_receive"
CONF_ON_STATE_CHANGE = "on_state_change"
+1 -1
View File
@@ -1,7 +1,6 @@
from esphome import automation
import esphome.codegen as cg
from esphome.components import uart
from esphome.components.const import CONF_LOOP
import esphome.config_validation as cv
from esphome.const import CONF_DEVICE, CONF_FILE, CONF_ID, CONF_VOLUME
@@ -16,6 +15,7 @@ DFPlayerIsPlayingCondition = dfplayer_ns.class_(
MULTI_CONF = True
CONF_FOLDER = "folder"
CONF_LOOP = "loop"
CONF_EQ_PRESET = "eq_preset"
CONF_ON_FINISHED_PLAYBACK = "on_finished_playback"
+13 -89
View File
@@ -3,18 +3,11 @@ from dataclasses import dataclass
from esphome import automation, core
from esphome.automation import maybe_simple_id
import esphome.codegen as cg
from esphome.components.const import (
BYTE_ORDER_BIG,
CONF_BYTE_ORDER,
CONF_DRAW_ROUNDING,
KEY_METADATA,
)
from esphome.components.const import KEY_METADATA
import esphome.config_validation as cv
from esphome.const import (
CONF_AUTO_CLEAR_ENABLED,
CONF_DIMENSIONS,
CONF_FROM,
CONF_HEIGHT,
CONF_ID,
CONF_LAMBDA,
CONF_PAGE_ID,
@@ -23,11 +16,10 @@ from esphome.const import (
CONF_TO,
CONF_TRIGGER_ID,
CONF_UPDATE_INTERVAL,
CONF_WIDTH,
SCHEDULER_DONT_RUN,
)
from esphome.core import CORE, ID, CoroPriority, coroutine_with_priority
from esphome.final_validate import full_config
from esphome.core import CORE, CoroPriority, coroutine_with_priority
from esphome.cpp_generator import MockObj
DOMAIN = "display"
IS_PLATFORM_COMPONENT = True
@@ -167,97 +159,29 @@ async def setup_display_core_(var, config):
class DisplayMetaData:
width: int = 0
height: int = 0
has_hardware_rotation: bool = False
byte_order: str = BYTE_ORDER_BIG
has_writer: bool = False
rotation: int = 0
draw_rounding: int = 0
def _get_metadata_list() -> list[tuple]:
"""Get the raw metadata list. Each entry is (id, DisplayMetaData)."""
return CORE.data.setdefault(DOMAIN, {}).setdefault(KEY_METADATA, [])
has_hardware_rotation: bool = False
def get_all_display_metadata() -> dict[str, DisplayMetaData]:
"""Get all display metadata as a dict keyed by resolved ID strings.
Must not be called before IDs have been finalised.
"""
entries = _get_metadata_list()
assert all(id_.id is not None for id_, _ in entries), (
"get_all_display_metadata called before display IDs have been resolved"
)
return {id_.id: meta for id_, meta in entries}
"""Get all display metadata."""
return CORE.data.setdefault(DOMAIN, {}).setdefault(KEY_METADATA, {})
def get_display_metadata(display_id: ID) -> DisplayMetaData:
"""Get display metadata by ID object
Must not be called before IDs have been finalised.
"""
for id_, meta in _get_metadata_list():
if id_ is display_id:
return meta
assert id_.id is not None, (
"get_display_metadata called before display IDs have been resolved"
)
if id_.id == display_id.id:
return meta
# No metadata found, display driver may not yet support it.
# Read the raw config to populate the returned data
global_config = full_config.get()
path = global_config.get_path_for_id(display_id)[:-1]
disp_config = global_config.get_config_for_path(path)
dimensions = disp_config.get(CONF_DIMENSIONS, (0, 0))
if isinstance(dimensions, dict):
dimensions = (dimensions.get(CONF_WIDTH, 0), dimensions.get(CONF_HEIGHT, 0))
elif not isinstance(dimensions, tuple) or len(dimensions) != 2:
dimensions = (0, 0)
meta = DisplayMetaData(
width=dimensions[0],
height=dimensions[1],
has_hardware_rotation=False,
byte_order=disp_config.get(CONF_BYTE_ORDER, cv.UNDEFINED),
has_writer=disp_config.get(CONF_AUTO_CLEAR_ENABLED) is True
or disp_config.get(CONF_PAGES) is not None
or disp_config.get(CONF_LAMBDA) is not None
or disp_config.get(CONF_SHOW_TEST_CARD) is True,
rotation=disp_config.get(CONF_ROTATION, 0),
draw_rounding=disp_config.get(CONF_DRAW_ROUNDING, 0),
)
_get_metadata_list().append((display_id, meta))
return meta
def get_display_metadata(display_id: str) -> DisplayMetaData | None:
"""Get display metadata by ID for use by other components."""
return get_all_display_metadata().get(display_id, DisplayMetaData())
def add_metadata(
id: ID,
id: str | MockObj,
width: int,
height: int,
has_writer: bool,
has_hardware_rotation: bool = False,
byte_order: str = BYTE_ORDER_BIG,
has_writer: bool = False,
rotation: int = 0,
draw_rounding: int = 0,
):
entries = _get_metadata_list()
assert not any(existing_id is id for existing_id, _ in entries), (
f"Duplicate display metadata for ID {id}"
)
entries.append(
(
id,
DisplayMetaData(
width=width,
height=height,
has_hardware_rotation=has_hardware_rotation,
byte_order=byte_order,
has_writer=has_writer,
rotation=rotation,
draw_rounding=draw_rounding,
),
)
get_all_display_metadata()[str(id)] = DisplayMetaData(
width, height, has_writer, has_hardware_rotation
)
+1 -1
View File
@@ -87,7 +87,7 @@ async def to_code(config):
cg.add_build_flag("-DDSMR_WATER_MBUS_ID=" + str(config[CONF_WATER_MBUS_ID]))
cg.add_build_flag("-DDSMR_THERMAL_MBUS_ID=" + str(config[CONF_THERMAL_MBUS_ID]))
cg.add_library("esphome/dsmr_parser", "1.8.0")
cg.add_library("esphome/dsmr_parser", "1.4.0")
def final_validate(config: ConfigType) -> ConfigType:
+3 -4
View File
@@ -153,9 +153,8 @@ void Dsmr::receive_encrypted_telegram_() {
bool Dsmr::parse_telegram_(const dsmr_parser::DsmrUnencryptedTelegram &telegram) {
this->stop_requesting_data_();
ESP_LOGV(TAG, "Trying to parse telegram (%zu bytes)", telegram.full_content().size());
ESP_LOGVV(TAG, "Telegram content:\n %.*s", static_cast<int>(telegram.full_content().size()),
telegram.full_content().data());
ESP_LOGV(TAG, "Trying to parse telegram (%zu bytes)", telegram.content().size());
ESP_LOGVV(TAG, "Telegram content:\n %.*s", static_cast<int>(telegram.content().size()), telegram.content().data());
MyData data;
if (const bool res = dsmr_parser::DsmrParser::parse(data, telegram); !res) {
@@ -168,7 +167,7 @@ bool Dsmr::parse_telegram_(const dsmr_parser::DsmrUnencryptedTelegram &telegram)
// Publish the telegram, after publishing the sensors so it can also trigger action based on latest values
if (this->s_telegram_ != nullptr) {
this->s_telegram_->publish_state(telegram.full_content().data(), telegram.full_content().size());
this->s_telegram_->publish_state(telegram.content().data(), telegram.content().size());
}
return true;
}
+3 -8
View File
@@ -74,8 +74,7 @@ class Dsmr : public Component, public uart::UARTDevice {
receive_timeout_(receive_timeout),
request_pin_(request_pin),
buffer_(max_telegram_length),
packet_accumulator_(buffer_, crc_check),
dlms_decryptor_(gcm_decryptor_, crc_check) {
packet_accumulator_(buffer_, crc_check) {
this->set_decryption_key_(decryption_key);
}
@@ -98,11 +97,7 @@ class Dsmr : public Component, public uart::UARTDevice {
// Remove before 2026.8.0
ESPDEPRECATED("Use 'decryption_key' configuration parameter. This method will be removed in 2026.8.0", "2026.2.0")
void set_decryption_key(const std::string &decryption_key) {
// Some YAML configs pass a string longer than 32 symbols. We only need the first 32 symbols,
// otherwise `Aes128GcmDecryptionKey::from_hex` will fail.
this->set_decryption_key_(std::string(decryption_key, 0, 32).c_str());
}
void set_decryption_key(const std::string &decryption_key) { this->set_decryption_key_(decryption_key.c_str()); }
// Sensor setters
#define DSMR_SET_SENSOR(s) \
@@ -148,7 +143,7 @@ class Dsmr : public Component, public uart::UARTDevice {
std::vector<uint8_t> buffer_;
dsmr_parser::PacketAccumulator packet_accumulator_;
Aes128GcmDecryptorImpl gcm_decryptor_;
dsmr_parser::DlmsPacketDecryptor dlms_decryptor_;
dsmr_parser::DlmsPacketDecryptor dlms_decryptor_{gcm_decryptor_};
std::array<uint8_t, 256> uart_chunk_reading_buf_;
};
} // namespace esphome::dsmr
+4 -4
View File
@@ -248,6 +248,10 @@ CONFIG_SCHEMA = cv.Schema(
device_class=DEVICE_CLASS_POWER,
state_class=STATE_CLASS_MEASUREMENT,
),
cv.Optional("electricity_switch_position"): sensor.sensor_schema(
accuracy_decimals=3,
state_class=STATE_CLASS_MEASUREMENT,
),
cv.Optional("electricity_failures"): sensor.sensor_schema(
accuracy_decimals=0,
state_class=STATE_CLASS_MEASUREMENT,
@@ -804,10 +808,6 @@ CONFIG_SCHEMA = cv.Schema(
device_class=DEVICE_CLASS_DURATION,
state_class=STATE_CLASS_MEASUREMENT,
),
cv.Optional("electricity_switch_position"): cv.invalid(
"'electricity_switch_position' has moved to the 'text_sensor' platform."
"Move it under 'text_sensor' to fix."
),
}
).extend(cv.COMPONENT_SCHEMA)
-1
View File
@@ -14,7 +14,6 @@ CONFIG_SCHEMA = cv.Schema(
cv.Optional("p1_version"): text_sensor.text_sensor_schema(),
cv.Optional("p1_version_be"): text_sensor.text_sensor_schema(),
cv.Optional("timestamp"): text_sensor.text_sensor_schema(),
cv.Optional("electricity_switch_position"): text_sensor.text_sensor_schema(),
cv.Optional("electricity_tariff"): text_sensor.text_sensor_schema(),
cv.Optional("electricity_tariff_il"): text_sensor.text_sensor_schema(),
cv.Optional("electricity_failure_log"): text_sensor.text_sensor_schema(),
+15 -16
View File
@@ -46,10 +46,10 @@ from esphome.const import (
Toolchain,
__version__,
)
from esphome.core import CORE, EsphomeError, HexInt
from esphome.core import CORE, EsphomeError, HexInt, Library
from esphome.core.config import BOARD_MAX_LENGTH
from esphome.coroutine import CoroPriority, coroutine_with_priority
from esphome.espidf.component import generate_idf_components
from esphome.espidf.component import generate_idf_component
import esphome.final_validate as fv
from esphome.helpers import copy_file_if_changed, rmtree, write_file_if_changed
from esphome.types import ConfigType
@@ -2598,6 +2598,13 @@ def _write_sdkconfig():
clean_build(clear_pio_cache=False)
def _platformio_library_to_dependency(library: Library) -> tuple[str, dict[str, str]]:
dependency: dict[str, str] = {}
name, _version, path = generate_idf_component(library)
dependency["override_path"] = str(path)
return name, dependency
def _write_idf_component_yml():
yml_path = CORE.relative_build_path("src/idf_component.yml")
dependencies: dict[str, dict] = {}
@@ -2671,21 +2678,13 @@ def _write_idf_component_yml():
)
if CORE.using_toolchain_esp_idf:
# Convert the PlatformIO libraries to ESP-IDF components as a batch so
# PlatformIO resolves the whole dependency tree at once -- deduplicating
# shared transitive deps (e.g. esphome/libsodium pulled by both noise-c
# and esp_wireguard) to a single version instead of clashing
# override_path entries.
libraries = [
library
for name, library in CORE.platformio_libraries.items()
# Try to convert PlatformIO library to ESP-IDF components
for name, library in CORE.platformio_libraries.items():
# Don't process arduino libraries
if name not in ARDUINO_DISABLED_LIBRARIES
]
for component in generate_idf_components(libraries):
dependencies[component.get_sanitized_name()] = {
"override_path": str(component.path)
}
if name in ARDUINO_DISABLED_LIBRARIES:
continue
dependency_name, dependency = _platformio_library_to_dependency(library)
dependencies[dependency_name] = dependency
if CORE.data[KEY_ESP32][KEY_COMPONENTS]:
components: dict = CORE.data[KEY_ESP32][KEY_COMPONENTS]
@@ -62,26 +62,6 @@ MANUFACTURER_NAME_CHARACTERISTIC_UUID = 0x2A29
MODEL_CHARACTERISTIC_UUID = 0x2A24
FIRMWARE_VERSION_CHARACTERISTIC_UUID = 0x2A26
# Suffix of the Bluetooth Base UUID used to expand 16/32 bit UUIDs to 128 bit.
_BASE_UUID_SUFFIX = "-0000-1000-8000-00805F9B34FB"
def uuid_is(uuid: int | str, uuid16: int) -> bool:
"""Return True if a validated UUID refers to the given 16-bit short UUID.
A service/characteristic UUID may be an ``int`` (from ``cv.hex_uint32_t``) or an
uppercase string in 16, 32 or 128 bit form (from ``bt_uuid``), so every
representation of the same UUID must be considered equivalent.
"""
if isinstance(uuid, int):
return uuid == uuid16
return uuid.upper() in (
f"{uuid16:04X}",
f"{uuid16:08X}",
f"{uuid16:08X}{_BASE_UUID_SUFFIX}",
)
# Core key to store the global configuration
KEY_NOTIFY_REQUIRED = "notify_required"
KEY_SET_VALUE = "set_value"
@@ -215,7 +195,7 @@ def create_description_cud(char_config):
return char_config
# If the config displays a description, there cannot be a descriptor with the CUD UUID
for desc in char_config[CONF_DESCRIPTORS]:
if uuid_is(desc[CONF_UUID], CUD_DESCRIPTOR_UUID):
if desc[CONF_UUID] == CUD_DESCRIPTOR_UUID:
raise cv.Invalid(
f"Characteristic {char_config[CONF_UUID]} has a description, but a CUD descriptor is already present"
)
@@ -238,7 +218,7 @@ def create_notify_cccd(char_config):
return char_config
# If the CCCD descriptor is already present, return the config
for desc in char_config[CONF_DESCRIPTORS]:
if uuid_is(desc[CONF_UUID], CCCD_DESCRIPTOR_UUID):
if desc[CONF_UUID] == CCCD_DESCRIPTOR_UUID:
# Check if the WRITE property is set
if not desc[CONF_WRITE]:
raise cv.Invalid(
@@ -264,7 +244,7 @@ def create_device_information_service(config):
# If there is already a device information service,
# there cannot be CONF_MODEL, CONF_MANUFACTURER or CONF_FIRMWARE_VERSION properties
for service in config[CONF_SERVICES]:
if uuid_is(service[CONF_UUID], DEVICE_INFORMATION_SERVICE_UUID):
if service[CONF_UUID] == DEVICE_INFORMATION_SERVICE_UUID:
if (
CONF_MODEL in config
or CONF_MANUFACTURER in config
@@ -612,7 +592,7 @@ async def to_code(config):
)
for char_conf in service_config[CONF_CHARACTERISTICS]:
await to_code_characteristic(service_var, char_conf)
if uuid_is(service_config[CONF_UUID], DEVICE_INFORMATION_SERVICE_UUID):
if service_config[CONF_UUID] == DEVICE_INFORMATION_SERVICE_UUID:
cg.add(var.set_device_information_service(service_var))
else:
cg.add(var.enqueue_start_service(service_var))
-16
View File
@@ -5,23 +5,7 @@ namespace esphome::gree {
static const char *const TAG = "gree.climate";
climate::ClimateTraits GreeClimate::traits() {
auto t = climate_ir::ClimateIR::traits();
// ClimateIR unconditionally includes HEAT_COOL in the base mode set; remove it when heat is not supported.
if (!this->supports_heat_) {
auto modes = t.get_supported_modes();
modes.erase(climate::CLIMATE_MODE_HEAT_COOL);
t.set_supported_modes(modes);
}
return t;
}
void GreeClimate::set_model(Model model) {
if (model == GREE_YAN) {
// YAN only has a vertical vane; the horizontal swing IR bytes are not defined for this model.
this->swing_modes_.erase(climate::CLIMATE_SWING_HORIZONTAL);
this->swing_modes_.erase(climate::CLIMATE_SWING_BOTH);
}
if (model == GREE_YX1FF) {
this->fan_modes_.insert(climate::CLIMATE_FAN_QUIET); // YX1FF 4 speed
this->presets_.insert(climate::CLIMATE_PRESET_NONE); // YX1FF sleep mode
-1
View File
@@ -94,7 +94,6 @@ class GreeClimate : public climate_ir::ClimateIR {
protected:
// Transmit via IR the state of this climate controller.
void transmit_state() override;
climate::ClimateTraits traits() override;
uint8_t operation_mode_();
uint8_t fan_speed_();
+19 -100
View File
@@ -1,6 +1,4 @@
import logging
import re
import sys
from esphome import pins
import esphome.codegen as cg
@@ -31,7 +29,6 @@ from esphome.config_helpers import filter_source_files_from_platform
import esphome.config_validation as cv
from esphome.const import (
CONF_ADDRESS,
CONF_DEVICE,
CONF_FREQUENCY,
CONF_I2C,
CONF_I2C_ID,
@@ -43,7 +40,6 @@ from esphome.const import (
CONF_TIMEOUT,
PLATFORM_ESP32,
PLATFORM_ESP8266,
PLATFORM_HOST,
PLATFORM_NRF52,
PLATFORM_RP2040,
PlatformFramework,
@@ -60,7 +56,6 @@ InternalI2CBus = i2c_ns.class_("InternalI2CBus", I2CBus)
ArduinoI2CBus = i2c_ns.class_("ArduinoI2CBus", InternalI2CBus, cg.Component)
IDFI2CBus = i2c_ns.class_("IDFI2CBus", InternalI2CBus, cg.Component)
ZephyrI2CBus = i2c_ns.class_("ZephyrI2CBus", I2CBus, cg.Component)
HostI2CBus = i2c_ns.class_("HostI2CBus", I2CBus, cg.Component)
I2CDevice = i2c_ns.class_("I2CDevice")
ESP32_I2C_CAPABILITIES = {
@@ -88,12 +83,6 @@ CONF_SCL_PULLUP_ENABLED = "scl_pullup_enabled"
MULTI_CONF = True
def validate_device(value):
if not re.match(r"^/(?:[^/]+/)*[^/]+$", value):
raise cv.Invalid("Device must be an absolute device path (e.g., /dev/i2c-0)")
return value
def _bus_declare_type(value):
if CORE.is_esp32:
return cv.declare_id(IDFI2CBus)(value)
@@ -101,8 +90,6 @@ def _bus_declare_type(value):
return cv.declare_id(ArduinoI2CBus)(value)
if CORE.using_zephyr:
return cv.declare_id(ZephyrI2CBus)(value)
if CORE.is_host:
return cv.declare_id(HostI2CBus)(value)
raise NotImplementedError
@@ -134,48 +121,15 @@ def validate_config(config):
return config
def validate_host_config(config):
if CORE.is_host:
# Host I2C is currently only supported on Linux
if not sys.platform.lower().startswith("linux"):
raise cv.Invalid(
"I2C is only supported on Linux for the host platform. "
f"Current platform: {sys.platform}"
)
if CONF_SDA in config or CONF_SCL in config:
raise cv.Invalid(
"'sda' and 'scl' are not supported on host platform; use 'device' instead."
)
if CONF_SDA_PULLUP_ENABLED in config or CONF_SCL_PULLUP_ENABLED in config:
raise cv.Invalid("Pull-up configuration is not supported on host platform.")
if CONF_DEVICE not in config:
raise cv.Invalid(
"'device' is required for host platform (e.g., /dev/i2c-0)."
)
return config
CONFIG_SCHEMA = cv.All(
cv.Schema(
{
cv.GenerateID(): _bus_declare_type,
cv.SplitDefault(
CONF_SDA,
esp32="SDA",
esp8266="SDA",
rp2040="SDA",
nrf52="SDA",
): pins.internal_gpio_pin_number,
cv.Optional(CONF_SDA, default="SDA"): pins.internal_gpio_pin_number,
cv.SplitDefault(CONF_SDA_PULLUP_ENABLED, esp32=True): cv.All(
cv.only_on_esp32, cv.boolean
),
cv.SplitDefault(
CONF_SCL,
esp32="SCL",
esp8266="SCL",
rp2040="SCL",
nrf52="SCL",
): pins.internal_gpio_pin_number,
cv.Optional(CONF_SCL, default="SCL"): pins.internal_gpio_pin_number,
cv.SplitDefault(CONF_SCL_PULLUP_ENABLED, esp32=True): cv.All(
cv.only_on_esp32, cv.boolean
),
@@ -185,7 +139,6 @@ CONFIG_SCHEMA = cv.All(
esp8266="50kHz",
rp2040="50kHz",
nrf52="100kHz",
host="50kHz",
): cv.All(
cv.frequency,
cv.float_range(min=0, min_included=False),
@@ -202,22 +155,10 @@ CONFIG_SCHEMA = cv.All(
),
cv.boolean,
),
cv.Optional(CONF_DEVICE): cv.All(
cv.only_on(PLATFORM_HOST), validate_device
),
}
).extend(cv.COMPONENT_SCHEMA),
cv.only_on(
[
PLATFORM_ESP32,
PLATFORM_ESP8266,
PLATFORM_RP2040,
PLATFORM_NRF52,
PLATFORM_HOST,
]
),
cv.only_on([PLATFORM_ESP32, PLATFORM_ESP8266, PLATFORM_RP2040, PLATFORM_NRF52]),
validate_config,
validate_host_config,
)
@@ -276,13 +217,7 @@ FINAL_VALIDATE_SCHEMA = _final_validate
async def to_code(config):
cg.add_global(i2c_ns.using)
cg.add_define("USE_I2C")
if CORE.is_host:
var = cg.new_Pvariable(config[CONF_ID])
await cg.register_component(var, config)
cg.add(var.set_device(config[CONF_DEVICE]))
cg.add(var.set_frequency(int(config[CONF_FREQUENCY])))
cg.add(var.set_scan(config[CONF_SCAN]))
elif CORE.using_zephyr:
if CORE.using_zephyr:
zephyr_add_prj_conf("I2C", True)
i2c = "i2c0"
if zephyr_data()[KEY_BOARD] == "xiao_ble":
@@ -309,40 +244,25 @@ async def to_code(config):
var = cg.new_Pvariable(
config[CONF_ID], MockObj(f"DEVICE_DT_GET(DT_NODELABEL({i2c}))")
)
await cg.register_component(var, config)
cg.add(var.set_sda_pin(config[CONF_SDA]))
if CONF_SDA_PULLUP_ENABLED in config:
cg.add(var.set_sda_pullup_enabled(config[CONF_SDA_PULLUP_ENABLED]))
cg.add(var.set_scl_pin(config[CONF_SCL]))
if CONF_SCL_PULLUP_ENABLED in config:
cg.add(var.set_scl_pullup_enabled(config[CONF_SCL_PULLUP_ENABLED]))
cg.add(var.set_frequency(int(config[CONF_FREQUENCY])))
cg.add(var.set_scan(config[CONF_SCAN]))
if CONF_TIMEOUT in config:
cg.add(var.set_timeout(int(config[CONF_TIMEOUT].total_microseconds)))
if CONF_LOW_POWER_MODE in config:
cg.add(var.set_lp_mode(bool(config[CONF_LOW_POWER_MODE])))
else:
var = cg.new_Pvariable(config[CONF_ID])
await cg.register_component(var, config)
await cg.register_component(var, config)
cg.add(var.set_sda_pin(config[CONF_SDA]))
if CONF_SDA_PULLUP_ENABLED in config:
cg.add(var.set_sda_pullup_enabled(config[CONF_SDA_PULLUP_ENABLED]))
cg.add(var.set_scl_pin(config[CONF_SCL]))
if CONF_SCL_PULLUP_ENABLED in config:
cg.add(var.set_scl_pullup_enabled(config[CONF_SCL_PULLUP_ENABLED]))
cg.add(var.set_sda_pin(config[CONF_SDA]))
if CONF_SDA_PULLUP_ENABLED in config:
cg.add(var.set_sda_pullup_enabled(config[CONF_SDA_PULLUP_ENABLED]))
cg.add(var.set_scl_pin(config[CONF_SCL]))
if CONF_SCL_PULLUP_ENABLED in config:
cg.add(var.set_scl_pullup_enabled(config[CONF_SCL_PULLUP_ENABLED]))
cg.add(var.set_frequency(int(config[CONF_FREQUENCY])))
cg.add(var.set_scan(config[CONF_SCAN]))
if CONF_TIMEOUT in config:
cg.add(var.set_timeout(int(config[CONF_TIMEOUT].total_microseconds)))
if CORE.using_arduino and not CORE.is_esp32:
cg.add_library("Wire", None)
if CONF_LOW_POWER_MODE in config:
cg.add(var.set_lp_mode(bool(config[CONF_LOW_POWER_MODE])))
cg.add(var.set_frequency(int(config[CONF_FREQUENCY])))
cg.add(var.set_scan(config[CONF_SCAN]))
if CONF_TIMEOUT in config:
cg.add(var.set_timeout(int(config[CONF_TIMEOUT].total_microseconds)))
if CORE.using_arduino and not CORE.is_esp32:
cg.add_library("Wire", None)
if CONF_LOW_POWER_MODE in config:
cg.add(var.set_lp_mode(bool(config[CONF_LOW_POWER_MODE])))
def i2c_device_schema(default_address):
@@ -445,6 +365,5 @@ FILTER_SOURCE_FILES = filter_source_files_from_platform(
PlatformFramework.ESP32_IDF,
},
"i2c_bus_zephyr.cpp": {PlatformFramework.NRF52_ZEPHYR},
"i2c_bus_host.cpp": {PlatformFramework.HOST_NATIVE},
}
)
-297
View File
@@ -1,297 +0,0 @@
#ifdef USE_HOST
#if defined(__linux__)
#include "i2c_bus_host.h"
#include "esphome/core/helpers.h"
#include "esphome/core/log.h"
#include <fcntl.h>
#include <linux/i2c-dev.h>
#include <linux/i2c.h>
#include <sys/ioctl.h>
#include <unistd.h>
#include <cerrno>
#include <cstdint>
#include <cstring>
namespace esphome::i2c {
static const char *const TAG = "i2c.host";
HostI2CBus::~HostI2CBus() {
if (this->file_descriptor_ != -1) {
close(this->file_descriptor_);
this->file_descriptor_ = -1;
}
}
void HostI2CBus::setup() {
ESP_LOGCONFIG(TAG, "Setting up I2C bus...");
// Open I2C device file
this->file_descriptor_ = open(this->device_.c_str(), O_RDWR);
if (this->file_descriptor_ == -1) {
int err = errno;
if (err == ENOENT) {
this->update_error_("not found");
} else if (err == EACCES) {
this->update_error_("permission denied");
} else {
this->update_error_(std::string("failed to open: ") + strerror(err));
}
this->mark_failed();
return;
}
this->initialized_ = true;
ESP_LOGCONFIG(TAG, " Device: %s", this->device_.c_str());
// Run bus scan if enabled
if (this->scan_) {
this->i2c_scan_();
}
}
void HostI2CBus::dump_config() {
ESP_LOGCONFIG(TAG, "I2C Bus:");
ESP_LOGCONFIG(TAG, " Device: %s", this->device_.c_str());
// Bus frequency cannot be set from userspace via i2c-dev; report it as informational only
ESP_LOGCONFIG(TAG, " Frequency: %u Hz (informational; not applied on host)", this->frequency_);
if (!this->first_error_.empty()) {
ESP_LOGE(TAG, " Setup Error: %s", this->first_error_.c_str());
}
if (this->scan_) {
ESP_LOGI(TAG, " Scan Results:");
for (const auto &s : this->scan_results_) {
if (s.second) {
ESP_LOGI(TAG, " 0x%02X: Found", s.first);
}
}
}
}
ErrorCode HostI2CBus::write_readv(uint8_t address, const uint8_t *write_buffer, size_t write_count,
uint8_t *read_buffer, size_t read_count) {
if (!this->initialized_) {
ESP_LOGE(TAG, "I2C bus not initialized");
return ERROR_NOT_INITIALIZED;
}
ESP_LOGVV(TAG, "I2C write_readv addr=0x%02X write=%zu read=%zu", address, write_count, read_count);
// Handle special case: probe (no write data, no read data)
// This is used for device detection during bus scanning
if (write_count == 0 && read_count == 0) {
struct i2c_msg msg;
msg.addr = address;
msg.flags = 0;
msg.len = 0;
msg.buf = nullptr;
struct i2c_rdwr_ioctl_data rdwr_data;
rdwr_data.msgs = &msg;
rdwr_data.nmsgs = 1;
int ret = ioctl(this->file_descriptor_, I2C_RDWR, &rdwr_data);
if (ret < 0) {
int err = errno;
// If I2C_RDWR not supported, try SMBus Quick command (what i2cdetect uses)
if (err == EOPNOTSUPP || err == ENOSYS) {
ESP_LOGVV(TAG, "I2C_RDWR probe failed, trying SMBus Quick for addr=0x%02X", address);
if (ioctl(this->file_descriptor_, I2C_SLAVE, address) < 0) { // NOLINT
return this->map_errno_to_error_code_(errno);
}
// Use I2C_SMBUS ioctl with Quick command
union i2c_smbus_data data;
struct i2c_smbus_ioctl_data args;
args.read_write = I2C_SMBUS_WRITE;
args.command = 0;
args.size = I2C_SMBUS_QUICK;
args.data = &data;
ret = ioctl(this->file_descriptor_, I2C_SMBUS, &args);
if (ret < 0) {
return this->map_errno_to_error_code_(errno);
}
return ERROR_OK;
}
return this->map_errno_to_error_code_(err);
}
return ERROR_OK;
}
// i2c_msg.len is a 16-bit field; reject transfers that would silently truncate
if (write_count > UINT16_MAX || read_count > UINT16_MAX) {
ESP_LOGE(TAG, "I2C transfer too large: write=%zu read=%zu (max %u)", write_count, read_count,
(unsigned) UINT16_MAX);
return ERROR_TOO_LARGE;
}
// Prepare messages for combined write-read transaction
struct i2c_msg msgs[2];
int num_msgs = 0;
// Add write message if write data present
if (write_count > 0) {
msgs[num_msgs].addr = address;
msgs[num_msgs].flags = 0; // Write
msgs[num_msgs].len = write_count;
msgs[num_msgs].buf = const_cast<uint8_t *>(write_buffer);
num_msgs++;
}
// Add read message if read data requested
if (read_count > 0) {
msgs[num_msgs].addr = address;
msgs[num_msgs].flags = I2C_M_RD; // Read
msgs[num_msgs].len = read_count;
msgs[num_msgs].buf = read_buffer;
num_msgs++;
}
// Execute I2C transaction
struct i2c_rdwr_ioctl_data rdwr_data;
rdwr_data.msgs = msgs;
rdwr_data.nmsgs = num_msgs;
int ret = ioctl(this->file_descriptor_, I2C_RDWR, &rdwr_data);
if (ret < 0) {
int err = errno;
if (err == EOPNOTSUPP || err == ENOSYS) {
ESP_LOGV(TAG, "I2C_RDWR not supported, using I2C_SLAVE fallback for addr=0x%02X", address); // NOLINT
if (ioctl(this->file_descriptor_, I2C_SLAVE, address) < 0) { // NOLINT
ESP_LOGV(TAG, "I2C_SLAVE ioctl failed: %s", strerror(errno)); // NOLINT
return this->map_errno_to_error_code_(errno);
}
// Perform write if needed
if (write_count > 0) {
ssize_t written = ::write(this->file_descriptor_, write_buffer, write_count);
if (written != (ssize_t) write_count) {
int write_err = errno;
// If write() also fails with EOPNOTSUPP, try I2C_SMBUS as last resort
if (write_err == EOPNOTSUPP || write_err == ENOSYS) {
ESP_LOGV(TAG, "I2C_SLAVE write not supported, trying I2C_SMBUS for addr=0x%02X", address); // NOLINT
// Use I2C_SMBUS_I2C_BLOCK_DATA for writes up to 32 bytes
// Standard SMBus mapping: first byte is command, remaining bytes are data
if (write_count < 1) {
ESP_LOGE(TAG, "Write size too small for I2C_SMBUS");
return ERROR_INVALID_ARGUMENT;
}
if (write_count > I2C_SMBUS_BLOCK_MAX + 1) {
ESP_LOGE(TAG, "Write size %zu exceeds I2C_SMBUS_BLOCK_MAX+1 (%d)", write_count, I2C_SMBUS_BLOCK_MAX + 1);
return ERROR_INVALID_ARGUMENT;
}
union i2c_smbus_data data;
// Standard SMBus: first byte = command, rest = data
uint8_t command = write_buffer[0];
size_t data_len = write_count - 1;
data.block[0] = data_len;
if (data_len > 0) {
memcpy(&data.block[1], write_buffer + 1, data_len);
}
struct i2c_smbus_ioctl_data args;
args.read_write = I2C_SMBUS_WRITE;
args.command = command;
args.size = I2C_SMBUS_I2C_BLOCK_DATA;
args.data = &data;
ret = ioctl(this->file_descriptor_, I2C_SMBUS, &args);
if (ret < 0) {
ESP_LOGV(TAG, "I2C_SMBUS write failed: %s", strerror(errno));
return this->map_errno_to_error_code_(errno);
}
} else {
ESP_LOGV(TAG, "I2C write failed: %s", strerror(write_err));
return this->map_errno_to_error_code_(write_err);
}
}
}
// Perform read if needed
if (read_count > 0) {
ssize_t bytes_read = ::read(this->file_descriptor_, read_buffer, read_count);
if (bytes_read != (ssize_t) read_count) {
int read_err = errno;
// If read() also fails with EOPNOTSUPP, try I2C_SMBUS as last resort
if (read_err == EOPNOTSUPP || read_err == ENOSYS) {
ESP_LOGV(TAG, "I2C_SLAVE read not supported, trying I2C_SMBUS for addr=0x%02X", address); // NOLINT
// Use I2C_SMBUS_I2C_BLOCK_DATA for reads up to 32 bytes
if (read_count > I2C_SMBUS_BLOCK_MAX) {
ESP_LOGE(TAG, "Read size %zu exceeds I2C_SMBUS_BLOCK_MAX (%d)", read_count, I2C_SMBUS_BLOCK_MAX);
return ERROR_INVALID_ARGUMENT;
}
union i2c_smbus_data data;
data.block[0] = read_count;
struct i2c_smbus_ioctl_data args;
args.read_write = I2C_SMBUS_READ;
args.command = 0; // Start register/command
args.size = I2C_SMBUS_I2C_BLOCK_DATA;
args.data = &data;
ret = ioctl(this->file_descriptor_, I2C_SMBUS, &args);
if (ret < 0) {
ESP_LOGV(TAG, "I2C_SMBUS read failed: %s", strerror(errno));
return this->map_errno_to_error_code_(errno);
}
// I2C_SMBUS_I2C_BLOCK_DATA returns the actual byte count in block[0];
// a short read means we did not receive all requested bytes
if (data.block[0] < read_count) {
ESP_LOGV(TAG, "I2C_SMBUS short read: got %u, expected %zu", data.block[0], read_count);
return ERROR_NOT_ACKNOWLEDGED;
}
// Copy data from SMBus buffer to output buffer
memcpy(read_buffer, &data.block[1], read_count);
} else {
ESP_LOGV(TAG, "I2C read failed: %s", strerror(read_err));
return this->map_errno_to_error_code_(read_err);
}
}
}
ESP_LOGVV(TAG, "I2C transaction successful (I2C_SLAVE method)"); // NOLINT
return ERROR_OK;
}
ESP_LOGV(TAG, "I2C transaction failed: %s", strerror(err));
return this->map_errno_to_error_code_(err);
}
ESP_LOGVV(TAG, "I2C transaction successful");
return ERROR_OK;
}
ErrorCode HostI2CBus::map_errno_to_error_code_(int err) {
switch (err) {
case ENXIO:
return ERROR_NOT_ACKNOWLEDGED;
case ETIMEDOUT:
return ERROR_TIMEOUT;
case EINVAL:
return ERROR_INVALID_ARGUMENT;
case ENODEV:
case ENOTTY:
return ERROR_NOT_INITIALIZED;
case EOPNOTSUPP:
case ENOSYS:
// Operation not supported - some I2C adapters don't support zero-length transactions
ESP_LOGVV(TAG, "I2C adapter does not support this operation (likely zero-length probe)");
return ERROR_NOT_ACKNOWLEDGED;
default:
ESP_LOGV(TAG, "Unmapped error code: %d (%s)", err, strerror(err));
return ERROR_UNKNOWN;
}
}
void HostI2CBus::update_error_(const std::string &error) {
if (this->first_error_.empty()) {
this->first_error_ = error;
}
ESP_LOGE(TAG, "[%s] %s", this->device_.c_str(), error.c_str());
}
} // namespace esphome::i2c
#else
#error "HostI2CBus is only supported on Linux"
#endif // defined(__linux__)
#endif // USE_HOST
-41
View File
@@ -1,41 +0,0 @@
#pragma once
#ifdef USE_HOST
#include "esphome/core/component.h"
#include "esphome/core/log.h"
#include "i2c_bus.h"
namespace esphome::i2c {
class HostI2CBus : public I2CBus, public Component {
public:
~HostI2CBus() override;
void setup() override;
void dump_config() override;
float get_setup_priority() const override { return setup_priority::BUS; }
ErrorCode write_readv(uint8_t address, const uint8_t *write_buffer, size_t write_count, uint8_t *read_buffer,
size_t read_count) override;
void set_device(const std::string &device) { this->device_ = device; }
void set_scan(bool scan) { this->scan_ = scan; }
void set_frequency(uint32_t frequency) { this->frequency_ = frequency; }
const std::string &get_device() const { return this->device_; }
protected:
void update_error_(const std::string &error);
ErrorCode map_errno_to_error_code_(int err);
std::string device_;
uint32_t frequency_{50000};
int file_descriptor_{-1};
bool initialized_{false};
std::string first_error_;
};
} // namespace esphome::i2c
#endif // USE_HOST
+1 -5
View File
@@ -461,11 +461,7 @@ async def _late_logger_init(config: ConfigType) -> None:
cg.add_define("USE_LOGGER_USB_SERIAL_JTAG")
# USB Serial JTAG code is compiled when platform supports it.
# Enable secondary USB serial JTAG console so the VFS functions are available.
if (
CORE.is_esp32
and config[CONF_HARDWARE_UART] != USB_SERIAL_JTAG
and has_serial_logging
):
if CORE.is_esp32 and config[CONF_HARDWARE_UART] != USB_SERIAL_JTAG:
require_usb_serial_jtag_secondary()
require_vfs_termios()
except cv.Invalid:
+2 -4
View File
@@ -6,7 +6,7 @@
#include <driver/uart.h>
#ifdef USE_LOGGER_UART_SELECTION_USB_SERIAL_JTAG
#ifdef USE_LOGGER_USB_SERIAL_JTAG
#include <driver/usb_serial_jtag.h>
#if ESP_IDF_VERSION < ESP_IDF_VERSION_VAL(5, 3, 0)
#include <esp_vfs_dev.h>
@@ -29,7 +29,7 @@ namespace esphome::logger {
static const char *const TAG = "logger";
#ifdef USE_LOGGER_UART_SELECTION_USB_SERIAL_JTAG
#ifdef USE_LOGGER_USB_SERIAL_JTAG
static void init_usb_serial_jtag_() {
setvbuf(stdin, NULL, _IONBF, 0); // Disable buffering on stdin
@@ -108,9 +108,7 @@ void Logger::pre_setup() {
#endif
#ifdef USE_LOGGER_USB_SERIAL_JTAG
case UART_SELECTION_USB_SERIAL_JTAG:
#ifdef USE_LOGGER_UART_SELECTION_USB_SERIAL_JTAG
init_usb_serial_jtag_();
#endif
break;
#endif
}
+53 -63
View File
@@ -7,7 +7,6 @@ import re
from esphome.automation import Trigger, build_automation, validate_automation
import esphome.codegen as cg
from esphome.components.const import (
BYTE_ORDER_BIG,
CONF_BYTE_ORDER,
CONF_COLOR_DEPTH,
CONF_DRAW_ROUNDING,
@@ -31,10 +30,12 @@ from esphome.components.image import (
from esphome.components.psram import DOMAIN as PSRAM_DOMAIN
import esphome.config_validation as cv
from esphome.const import (
CONF_AUTO_CLEAR_ENABLED,
CONF_BUFFER_SIZE,
CONF_ESPHOME,
CONF_GROUP,
CONF_ID,
CONF_LAMBDA,
CONF_LOG_LEVEL,
CONF_ON_IDLE,
CONF_PAGES,
@@ -213,73 +214,61 @@ def multi_conf_validate(configs: list[dict]):
def final_validation(config_list):
if len(config_list) != 1:
multi_conf_validate(config_list)
global_config = full_config.get()
# Resolve byte_order from display metadata before multi-config validation
for config in config_list:
metas = [get_display_metadata(disp) for disp in config[df.CONF_DISPLAYS]]
if any(m.has_writer for m in metas):
raise cv.Invalid(
"Using lambda:, pages:, auto_clear_enabled: true, or show_test_card: true in display config is not compatible with LVGL"
)
if any(m.rotation != 0 for m in metas):
raise cv.Invalid(
"use of 'rotation' in the display config is not compatible with LVGL, please set rotation in the LVGL config instead"
)
config[CONF_DRAW_ROUNDING] = max(
[m.draw_rounding for m in metas] + [config[CONF_DRAW_ROUNDING]]
)
display_byte_orders = {
m.byte_order for m in metas if m.byte_order is not cv.UNDEFINED
}
if len(display_byte_orders) > 1:
raise cv.Invalid(
"All displays configured for an LVGL instance must use the same byte_order"
)
if display_byte_orders:
display_order = next(iter(display_byte_orders))
if CONF_BYTE_ORDER in config:
if config[CONF_BYTE_ORDER] != display_order:
raise cv.Invalid(
"LVGL byte order must match the display byte order",
[CONF_BYTE_ORDER],
)
else:
config[CONF_BYTE_ORDER] = display_order
if CONF_BYTE_ORDER not in config:
config[CONF_BYTE_ORDER] = BYTE_ORDER_BIG
if (pages := config.get(CONF_PAGES)) and all(p[df.CONF_SKIP] for p in pages):
raise cv.Invalid("At least one page must not be skipped")
for display_id in config[df.CONF_DISPLAYS]:
path = global_config.get_path_for_id(display_id)[:-1]
display = global_config.get_config_for_path(path)
if CONF_LAMBDA in display or CONF_PAGES in display:
raise cv.Invalid(
"Using lambda: or pages: in display config is not compatible with LVGL"
)
# treating 0 as false is intended here.
if display.get(CONF_ROTATION):
raise cv.Invalid(
"use of 'rotation' in the display config is not compatible with LVGL, please set rotation in the LVGL config instead"
)
if display.get(CONF_AUTO_CLEAR_ENABLED) is True:
raise cv.Invalid(
"Using auto_clear_enabled: true in display config not compatible with LVGL"
)
if draw_rounding := display.get(CONF_DRAW_ROUNDING):
config[CONF_DRAW_ROUNDING] = max(
draw_rounding, config[CONF_DRAW_ROUNDING]
)
buffer_frac = config[CONF_BUFFER_SIZE]
if CORE.is_esp32 and buffer_frac > 0.5 and PSRAM_DOMAIN not in global_config:
df.LOGGER.warning("buffer_size: may need to be reduced without PSRAM")
if len(config_list) != 1:
multi_conf_validate(config_list)
for w in get_focused_widgets():
path = global_config.get_path_for_id(w)
widget_conf = global_config.get_config_for_path(path[:-1])
if df.CONF_ADJUSTABLE in widget_conf and not widget_conf[df.CONF_ADJUSTABLE]:
raise cv.Invalid(
"A non adjustable arc may not be focused",
path,
)
for w in get_refreshed_widgets():
path = global_config.get_path_for_id(w)
widget_conf = global_config.get_config_for_path(path[:-1])
if not any(isinstance(v, (Lambda, dict)) for v in widget_conf.values()):
raise cv.Invalid(
f"Widget '{w}' does not have any dynamic properties to refresh",
)
# Do per-widget type final validation for update actions
for widget_type, update_configs in df.get_updated_widgets().items():
for conf in update_configs:
for id_conf in conf.get(CONF_ID, ()):
name = id_conf[CONF_ID]
path = global_config.get_path_for_id(name)
widget_conf = global_config.get_config_for_path(path[:-1])
widget_type.final_validate(name, conf, widget_conf, path[1:])
for w in get_focused_widgets():
path = global_config.get_path_for_id(w)
widget_conf = global_config.get_config_for_path(path[:-1])
if (
df.CONF_ADJUSTABLE in widget_conf
and not widget_conf[df.CONF_ADJUSTABLE]
):
raise cv.Invalid(
"A non adjustable arc may not be focused",
path,
)
for w in get_refreshed_widgets():
path = global_config.get_path_for_id(w)
widget_conf = global_config.get_config_for_path(path[:-1])
if not any(isinstance(v, (Lambda, dict)) for v in widget_conf.values()):
raise cv.Invalid(
f"Widget '{w}' does not have any dynamic properties to refresh",
)
# Do per-widget type final validation for update actions
for widget_type, update_configs in df.get_updated_widgets().items():
for conf in update_configs:
for id_conf in conf.get(CONF_ID, ()):
name = id_conf[CONF_ID]
path = global_config.get_path_for_id(name)
widget_conf = global_config.get_config_for_path(path[:-1])
widget_type.final_validate(name, conf, widget_conf, path[1:])
async def to_code(configs):
@@ -378,7 +367,8 @@ async def to_code(configs):
# options will have CONF_ROTATION true if rotation is changed in an automation.
if CONF_ROTATION in config or df.get_options().get(CONF_ROTATION) is True:
if all(
get_display_metadata(disp).has_hardware_rotation for disp in displays
get_display_metadata(str(disp)).has_hardware_rotation
for disp in displays
):
rotation_type = RotationType.ROTATION_HARDWARE
df.LOGGER.info("LVGL will use hardware rotation via display driver")
@@ -593,7 +583,7 @@ LVGL_SCHEMA = cv.All(
cv.Optional(CONF_LOG_LEVEL, default="WARN"): cv.one_of(
*df.LV_LOG_LEVELS, upper=True
),
cv.Optional(CONF_BYTE_ORDER): cv.one_of(
cv.Optional(CONF_BYTE_ORDER, default="big_endian"): cv.one_of(
"big_endian", "little_endian", lower=True
),
cv.Optional(df.CONF_STYLE_DEFINITIONS): cv.ensure_list(
@@ -290,7 +290,6 @@ class Widget:
# Properties for linear equations
self.slope = None
self.y_int = None
self.parent = None
@staticmethod
def create(name, var, wtype: WidgetType, config: dict = None):
+8 -11
View File
@@ -430,8 +430,7 @@ class MeterType(WidgetType):
tvar, LV_PART.MAIN, await arc_style.get_var()
)
lw = Widget.create(iid, tvar, arc_indicator_type)
lw.parent = scale_var
await set_indicator_values(scale_var, lw, v)
await set_indicator_values(lw, v)
if t == CONF_TICK_STYLE:
# No object created for this
@@ -483,8 +482,7 @@ class MeterType(WidgetType):
if option in v:
props["line_" + option] = v[option]
lw = await widget_to_code(props, line_indicator_type, scale_var)
lw.parent = scale_var
await set_indicator_values(scale_var, lw, v)
await set_indicator_values(lw, v)
if t == CONF_IMAGE:
add_lv_use(CONF_IMAGE)
@@ -503,8 +501,7 @@ class MeterType(WidgetType):
}
iw = await widget_to_code(props, image_indicator_type, scale_var)
await iw.set_property(CONF_SRC, await lv_image.process(src))
iw.parent = scale_var
await set_indicator_values(scale_var, iw, v)
await set_indicator_values(iw, v)
# Hide the scale line
lv.obj_set_style_arc_opa(scale_var, LV_OPA.TRANSP, LV_PART.MAIN)
@@ -610,27 +607,27 @@ async def indicator_update_to_code(config, action_id, template_arg, args):
widget = await get_widgets(config)
async def set_value(w: Widget):
await set_indicator_values(w.parent, w, config)
await set_indicator_values(w, config)
return await action_to_code(
widget, set_value, action_id, template_arg, args, config
)
async def set_indicator_values(scale: MockObj, indicator: Widget, config):
async def set_indicator_values(indicator: Widget, config):
"""Update scale section values (replaces meter indicator values)"""
start_value = await get_start_value(config)
end_value = await get_end_value(config)
if indicator.type is arc_indicator_type:
# For scale sections, we update the range
if start_value is not None and end_value is not None:
lv.scale_set_section_range(scale, indicator.obj, start_value, end_value)
lv.scale_section_set_range(indicator.obj, start_value, end_value)
elif start_value is not None:
# If only start value, use it as both start and end (single point)
lv.scale_set_section_range(scale, indicator.obj, start_value, start_value)
lv.scale_section_set_range(indicator.obj, start_value, start_value)
elif end_value is not None:
# If only end value, assume range from 0 to end_value
lv.scale_set_section_range(scale, indicator.obj, 0, end_value)
lv.scale_section_set_range(indicator.obj, 0, end_value)
return
if start_value is None:
+1 -16
View File
@@ -37,7 +37,6 @@ from esphome.components.mipi import (
)
import esphome.config_validation as cv
from esphome.const import (
CONF_AUTO_CLEAR_ENABLED,
CONF_COLOR_ORDER,
CONF_DIMENSIONS,
CONF_DISABLED,
@@ -168,21 +167,7 @@ def _config_schema(config):
},
extra=cv.ALLOW_EXTRA,
)(config)
config = model_schema(config)(config)
model = MODELS[config[CONF_MODEL].upper()]
width, height, _offset_width, _offset_height = model.get_dimensions(config)
display.add_metadata(
config[CONF_ID],
width,
height,
has_hardware_rotation=False,
byte_order=config[CONF_BYTE_ORDER],
has_writer=requires_buffer(config)
or config.get(CONF_AUTO_CLEAR_ENABLED) is True,
rotation=config.get(CONF_ROTATION, 0),
draw_rounding=config.get(CONF_DRAW_ROUNDING, 0),
)
return config
return model_schema(config)(config)
def _final_validate(config):
+1 -16
View File
@@ -39,7 +39,6 @@ from esphome.components.rpi_dpi_rgb.display import (
)
import esphome.config_validation as cv
from esphome.const import (
CONF_AUTO_CLEAR_ENABLED,
CONF_BLUE,
CONF_COLOR_ORDER,
CONF_CS_PIN,
@@ -227,25 +226,11 @@ def _config_schema(config):
extra=cv.ALLOW_EXTRA,
)(config)
schema = model_schema(config)
config = cv.All(
return cv.All(
schema,
cv.only_on_esp32,
only_on_variant(supported=[VARIANT_ESP32S3, VARIANT_ESP32P4]),
)(config)
model = MODELS[config[CONF_MODEL].upper()]
width, height, _offset_width, _offset_height = model.get_dimensions(config)
display.add_metadata(
config[CONF_ID],
width,
height,
model.rotation_as_transform(config),
byte_order=config[CONF_BYTE_ORDER],
has_writer=requires_buffer(config)
or config.get(CONF_AUTO_CLEAR_ENABLED) is True,
rotation=config.get(CONF_ROTATION, 0),
draw_rounding=config.get(CONF_DRAW_ROUNDING, 0),
)
return config
CONFIG_SCHEMA = _config_schema
+4 -24
View File
@@ -30,7 +30,6 @@ from esphome.components.spi import TYPE_OCTAL, TYPE_QUAD, TYPE_SINGLE
import esphome.config_validation as cv
from esphome.config_validation import ALLOW_EXTRA
from esphome.const import (
CONF_AUTO_CLEAR_ENABLED,
CONF_BRIGHTNESS,
CONF_BUFFER_SIZE,
CONF_COLOR_ORDER,
@@ -48,7 +47,6 @@ from esphome.const import (
CONF_MIRROR_Y,
CONF_MODEL,
CONF_RESET_PIN,
CONF_ROTATION,
CONF_SWAP_XY,
CONF_TRANSFORM,
CONF_WIDTH,
@@ -269,28 +267,6 @@ def customise_schema(config):
if bus_mode != TYPE_QUAD and CONF_DC_PIN not in config:
raise cv.Invalid(f"DC pin is required in {bus_mode} mode")
denominator(config)
model = MODELS[config[CONF_MODEL]]
has_hardware_transform = config.get(
CONF_TRANSFORM
) != CONF_DISABLED and model.transforms == {
CONF_MIRROR_X,
CONF_MIRROR_Y,
CONF_SWAP_XY,
}
width, height, _offset_width, _offset_height = model.get_dimensions(
config, not has_hardware_transform
)
display.add_metadata(
config[CONF_ID],
width,
height,
has_hardware_transform,
byte_order=config[CONF_BYTE_ORDER],
has_writer=requires_buffer(config)
or config.get(CONF_AUTO_CLEAR_ENABLED) is True,
rotation=config.get(CONF_ROTATION, 0),
draw_rounding=config.get(CONF_DRAW_ROUNDING, 0),
)
return config
@@ -362,6 +338,7 @@ def get_instance(config):
buffer_type = cg.uint8 if color_depth == 8 else cg.uint16
frac = denominator(config)
madctl = model.get_madctl(model.get_base_transform(config), config)
has_writer = requires_buffer(config)
templateargs = [
buffer_type,
bufferpixels,
@@ -375,6 +352,9 @@ def get_instance(config):
madctl,
has_hardware_transform,
]
display.add_metadata(
config[CONF_ID], width, height, has_writer, has_hardware_transform
)
# If a buffer is required, use MipiSpiBuffer, otherwise use MipiSpi
if requires_buffer(config):
templateargs.extend(
+36 -48
View File
@@ -7,7 +7,6 @@ static const char *const TAG = "remote.rc5";
static constexpr uint32_t BIT_TIME_US = 889;
static constexpr uint8_t NBITS = 14;
static constexpr uint8_t NHALFBITS = NBITS * 2;
void RC5Protocol::encode(RemoteTransmitData *dst, const RC5Data &data) {
static bool toggle = false;
@@ -36,63 +35,52 @@ void RC5Protocol::encode(RemoteTransmitData *dst, const RC5Data &data) {
}
toggle = !toggle;
}
optional<RC5Data> RC5Protocol::decode(RemoteReceiveData src) {
// Expand the runs into half-bit levels (true = mark). Each run is exactly one
// half-bit (BIT_TIME_US) or two (2 * BIT_TIME_US); stop at anything else.
//
// halfbits[0] is reserved for the leading half-bit, which is always dropped --
// S1 is 1, so its first half sits at the idle level (at either polarity) and
// merges into the pre-frame idle. Captured half-bits start at index 1.
bool halfbits[NHALFBITS + 2];
uint8_t n = 1;
for (uint32_t i = 0; n <= NHALFBITS && src.is_valid(i); i++) {
if (src.peek_mark(BIT_TIME_US, i)) {
halfbits[n++] = true;
} else if (src.peek_space(BIT_TIME_US, i)) {
halfbits[n++] = false;
} else if (src.peek_mark(2 * BIT_TIME_US, i)) {
halfbits[n++] = true;
halfbits[n++] = true;
} else if (src.peek_space(2 * BIT_TIME_US, i)) {
halfbits[n++] = false;
halfbits[n++] = false;
} else {
break;
}
}
RC5Data out{
.address = 0,
.command = 0,
};
uint8_t field_bit;
// Expect a full frame once the leading half is restored: 27 captured halves
// (n == 28) or 26 when the final bit also ends on idle and its trailing half
// is dropped too (n == 27). A dropped edge half is the inverse of its partner
// (a Manchester bit always transitions mid-bit), so reconstruct the leading
// half (always) and the trailing half (only when it was dropped).
if (n != NHALFBITS && n != NHALFBITS - 1) {
if (src.expect_space(BIT_TIME_US) && src.expect_mark(BIT_TIME_US)) {
field_bit = 1;
} else if (src.expect_space(2 * BIT_TIME_US)) {
field_bit = 0;
} else {
return {};
}
halfbits[0] = !halfbits[1];
if (n == NHALFBITS - 1) {
halfbits[n] = !halfbits[n - 1];
if (!(((src.expect_space(BIT_TIME_US) || src.peek_space(2 * BIT_TIME_US)) ||
(src.expect_mark(BIT_TIME_US) || src.peek_mark(2 * BIT_TIME_US))) &&
(((src.expect_mark(BIT_TIME_US) || src.expect_mark(2 * BIT_TIME_US)) &&
(src.expect_space(BIT_TIME_US) || src.peek_space(2 * BIT_TIME_US))) ||
((src.expect_space(BIT_TIME_US) || src.expect_space(2 * BIT_TIME_US)) &&
(src.expect_mark(BIT_TIME_US) || src.peek_mark(2 * BIT_TIME_US)))))) {
return {};
}
const bool carrier = halfbits[1];
uint16_t bits = 0;
for (uint8_t i = 0; i < NBITS; i++) {
const bool first = halfbits[2 * i];
const bool second = halfbits[2 * i + 1];
if (first == second) {
return {}; // no midpoint transition -> not a valid Manchester bit
uint32_t out_data = 0;
for (int bit = NBITS - 4; bit >= 1; bit--) {
if ((src.expect_space(BIT_TIME_US) || src.expect_space(2 * BIT_TIME_US)) &&
(src.expect_mark(BIT_TIME_US) || src.peek_mark(2 * BIT_TIME_US))) {
out_data |= 0 << bit;
} else if ((src.expect_mark(BIT_TIME_US) || src.expect_mark(2 * BIT_TIME_US)) &&
(src.expect_space(BIT_TIME_US) || src.peek_space(2 * BIT_TIME_US))) {
out_data |= 1 << bit;
} else {
return {};
}
bits = (bits << 1) | (second == carrier ? 1 : 0);
}
if (src.expect_space(BIT_TIME_US) || src.expect_space(2 * BIT_TIME_US)) {
out_data |= 0;
} else if (src.expect_mark(BIT_TIME_US) || src.expect_mark(2 * BIT_TIME_US)) {
out_data |= 1;
}
const bool field_bit = bits & (1 << 12); // S2: the inverted 7th command bit
return RC5Data{
.address = static_cast<uint8_t>((bits >> 6) & 0x1F),
.command = static_cast<uint8_t>((bits & 0x3F) | (field_bit ? 0 : 0x40)),
};
out.command = (uint8_t) (out_data & 0x3F) + (1 - field_bit) * 64u;
out.address = (out_data >> 6) & 0x1F;
return out;
}
void RC5Protocol::dump(const RC5Data &data) {
ESP_LOGI(TAG, "Received RC5: address=0x%02X, command=0x%02X", data.address, data.command);
}
+6 -9
View File
@@ -501,21 +501,18 @@ def _generate_lwipopts_h() -> None:
in the build directory, and a pre-build script injects this directory
into the compiler include path before the framework's own include dir.
"""
from jinja2 import Environment
from jinja2 import Environment, FileSystemLoader
lwip_defines = CORE.data[KEY_RP2040].get(KEY_LWIP_OPTS)
if not lwip_defines:
return
# Read the template via pathlib and render from a string rather than using
# FileSystemLoader. jinja2's loader joins the search path with posixpath, which
# breaks on Windows extended-length paths (\\?\C:\...) where forward slashes are
# not accepted, causing a spurious TemplateNotFound (see issue #16732).
template_text = (Path(__file__).parent / "lwipopts.h.jinja").read_text(
encoding="utf-8"
template_dir = Path(__file__).parent
jinja_env = Environment(
loader=FileSystemLoader(str(template_dir)),
keep_trailing_newline=True,
)
jinja_env = Environment(keep_trailing_newline=True)
template = jinja_env.from_string(template_text)
template = jinja_env.get_template("lwipopts.h.jinja")
content = template.render(**lwip_defines)
lwip_dir = CORE.relative_build_path("lwip_override")
+4 -32
View File
@@ -20,7 +20,6 @@ Sdl = sdl_ns.class_("Sdl", display.Display, cg.Component)
sdl_window_flags = cg.global_ns.enum("SDL_WindowFlags")
CONF_CENTERED_ON_DISPLAY = "centered_on_display"
CONF_SDL_OPTIONS = "sdl_options"
CONF_SDL_ID = "sdl_id"
CONF_WINDOW_OPTIONS = "window_options"
@@ -32,8 +31,6 @@ WINDOW_OPTIONS = (
"resizable",
)
SDL_WINDOWPOS_CENTERED_MASK = 0x2FFF0000
def get_sdl_options(value):
if value != "":
@@ -50,20 +47,6 @@ def get_window_options():
return {cv.Optional(option, default=False): cv.boolean for option in WINDOW_OPTIONS}
def _validate_position(config: dict) -> dict:
if CONF_CENTERED_ON_DISPLAY in config:
if CONF_X in config or CONF_Y in config:
raise cv.Invalid(
f"Cannot specify '{CONF_CENTERED_ON_DISPLAY}' with '{CONF_X}' and '{CONF_Y}' options"
)
return config
if CONF_X in config and CONF_Y in config:
return config
if CONF_X in config or CONF_Y in config:
raise cv.Invalid(f"Must specify both '{CONF_X}' and '{CONF_Y}' options")
raise cv.Invalid("Must specify either 'x' and 'y' or 'centered_on_display'")
CONFIG_SCHEMA = cv.All(
display.FULL_DISPLAY_SCHEMA.extend(
cv.Schema(
@@ -83,13 +66,10 @@ CONFIG_SCHEMA = cv.All(
{
cv.Optional(CONF_POSITION): cv.Schema(
{
cv.Optional(CONF_X): cv.int_,
cv.Optional(CONF_Y): cv.int_,
cv.Optional(CONF_CENTERED_ON_DISPLAY): cv.int_range(
0, 128
),
cv.Required(CONF_X): cv.int_,
cv.Required(CONF_Y): cv.int_,
}
).add_extra(_validate_position),
),
**get_window_options(),
}
),
@@ -125,15 +105,7 @@ async def to_code(config):
cg.add(var.set_window_options(create_flags))
if position := window_options.get(CONF_POSITION):
if (centered := position.get(CONF_CENTERED_ON_DISPLAY)) is not None:
cg.add(
var.set_position(
SDL_WINDOWPOS_CENTERED_MASK | centered,
SDL_WINDOWPOS_CENTERED_MASK | centered,
)
)
else:
cg.add(var.set_position(position[CONF_X], position[CONF_Y]))
cg.add(var.set_position(position[CONF_X], position[CONF_Y]))
if lamb := config.get(CONF_LAMBDA):
lambda_ = await cg.process_lambda(
+3 -3
View File
@@ -28,7 +28,7 @@ class Sdl : public display::Display {
this->height_ = height;
}
void set_window_options(uint32_t window_options) { this->window_options_ = window_options; }
void set_position(int32_t pos_x, int32_t pos_y) {
void set_position(uint16_t pos_x, uint16_t pos_y) {
this->pos_x_ = pos_x;
this->pos_y_ = pos_y;
}
@@ -54,8 +54,8 @@ class Sdl : public display::Display {
int width_{};
int height_{};
uint32_t window_options_{0};
int32_t pos_x_{SDL_WINDOWPOS_UNDEFINED};
int32_t pos_y_{SDL_WINDOWPOS_UNDEFINED};
int pos_x_{SDL_WINDOWPOS_UNDEFINED};
int pos_y_{SDL_WINDOWPOS_UNDEFINED};
SDL_Renderer *renderer_{};
SDL_Window *window_{};
SDL_Texture *texture_{};
-15
View File
@@ -3,7 +3,6 @@ from __future__ import annotations
import abc
from contextlib import contextmanager
import contextvars
import copy
import functools
import heapq
import logging
@@ -169,11 +168,6 @@ class Config(OrderedDict, fv.FinalValidateConfig):
self.output_paths: list[tuple[ConfigPath, str]] = []
# A list of components ids with the config path
self.declare_ids: list[tuple[core.ID, ConfigPath]] = []
# Snapshot of the user's configuration after substitutions/packages/
# extend-remove resolution but before any schema validation defaults
# are applied. Populated by validate_config; used by `esphome config
# --no-defaults` to emit only the user-supplied keys.
self.user_config: ConfigType | None = None
self._data = {}
# Store pending validation tasks (in heap order)
self._validation_tasks: list[_ValidationStepTask] = []
@@ -1082,15 +1076,6 @@ def validate_config(
)
return result
# Snapshot the user's config before any schema validation defaults are
# applied. preload_core_config and later validation steps rewrite entries
# in-place with defaulted values; deep-copying here preserves the
# user-supplied keys for `esphome config --no-defaults`.
result.user_config = copy.deepcopy(config)
if substitutions is not None:
result.user_config[CONF_SUBSTITUTIONS] = copy.deepcopy(substitutions)
result.user_config.move_to_end(CONF_SUBSTITUTIONS, last=False)
# 2. Load partial core config
import esphome.core.config as core_config
+9 -2
View File
@@ -198,7 +198,12 @@ template<typename... Ts> class DelayAction : public Action<Ts...> {
// to avoid overhead from capturing arguments by value
if constexpr (sizeof...(Ts) == 0) {
App.scheduler.set_timer_common_(
/* component= */ nullptr, Scheduler::SchedulerItem::TIMEOUT, Scheduler::NameType::SELF_POINTER,
// The component is stored for blocking-warning log attribution only (SELF_POINTER items
// match by `this`, so it does not affect cancellation). Capturing the current component
// lets the warning name the source instead of "<null>"; it propagates across chained
// delays because the scheduler restores it as the current component before each callback.
/* component= */ App.get_current_component(), Scheduler::SchedulerItem::TIMEOUT,
Scheduler::NameType::SELF_POINTER,
/* static_name= */ reinterpret_cast<const char *>(this), /* hash_or_id= */ 0, this->delay_.value(),
[this]() { this->play_next_(); },
/* is_retry= */ false, /* skip_cancel= */ this->num_running_ > 1);
@@ -209,7 +214,9 @@ template<typename... Ts> class DelayAction : public Action<Ts...> {
// are passed as non-const lvalues to play_next_(const Ts&...) where Ts may be `T&`
auto f = [this, x...]() mutable { this->play_next_(x...); };
App.scheduler.set_timer_common_(
/* component= */ nullptr, Scheduler::SchedulerItem::TIMEOUT, Scheduler::NameType::SELF_POINTER,
// See the no-argument branch above: component is captured for log attribution only.
/* component= */ App.get_current_component(), Scheduler::SchedulerItem::TIMEOUT,
Scheduler::NameType::SELF_POINTER,
/* static_name= */ reinterpret_cast<const char *>(this), /* hash_or_id= */ 0, this->delay_.value(x...),
std::move(f),
/* is_retry= */ false, /* skip_cancel= */ this->num_running_ > 1);
+13 -11
View File
@@ -258,9 +258,11 @@ void Component::call() {
break;
}
}
bool Component::should_warn_of_blocking(uint32_t blocking_time) {
bool Component::should_warn_of_blocking(uint32_t blocking_time, uint32_t &threshold_ms_out) {
// Convert centisecond threshold to milliseconds for comparison
uint32_t threshold_ms = static_cast<uint32_t>(this->warn_if_blocking_over_) * 10U;
// Report the threshold that was exceeded (before any ratcheting below) so the warning is accurate.
threshold_ms_out = threshold_ms;
if (blocking_time > threshold_ms) {
// Set new threshold: blocking_time + increment, converted back to centiseconds
uint32_t new_threshold_ms = blocking_time + WARN_IF_BLOCKING_INCREMENT_MS;
@@ -493,17 +495,17 @@ uint64_t ComponentRuntimeStats::global_recorded_us = 0; // NOLINT(cppcoreguidel
void __attribute__((noinline, cold))
WarnIfComponentBlockingGuard::warn_blocking(Component *component, uint32_t blocking_time) {
bool should_warn;
if (component != nullptr) {
should_warn = component->should_warn_of_blocking(blocking_time);
} else {
should_warn = true; // Already checked > WARN_IF_BLOCKING_OVER_MS in caller
}
if (should_warn) {
ESP_LOGW(TAG, "%s took a long time for an operation (%" PRIu32 " ms), max is 30 ms",
component == nullptr ? LOG_STR_LITERAL("<null>") : LOG_STR_ARG(component->get_component_log_str()),
blocking_time);
// Default threshold for the null path (no component to consult); the caller already checked
// blocking_time > WARN_IF_BLOCKING_OVER_MS, so always warn in that case.
uint32_t threshold_ms = WARN_IF_BLOCKING_OVER_MS;
if (component != nullptr && !component->should_warn_of_blocking(blocking_time, threshold_ms)) {
return; // Component's (possibly ratcheted) threshold not exceeded yet
}
// A null component means the work ran in a scheduler continuation with no associated component
// (e.g. a delay inside a script/automation), so report it as a scheduled task rather than "<null>".
ESP_LOGW(TAG, "%s took a long time for an operation (%" PRIu32 " ms), max is %" PRIu32 " ms",
component == nullptr ? LOG_STR_LITERAL("a scheduled task") : LOG_STR_ARG(component->get_component_log_str()),
blocking_time, threshold_ms);
}
#ifdef USE_SETUP_PRIORITY_OVERRIDE
+1 -1
View File
@@ -326,7 +326,7 @@ class Component {
return component_source_lookup(this->component_source_index_);
}
bool should_warn_of_blocking(uint32_t blocking_time);
bool should_warn_of_blocking(uint32_t blocking_time, uint32_t &threshold_ms_out);
protected:
friend class Application;
+2 -3
View File
@@ -358,12 +358,11 @@
#define USE_LOGGER_USB_SERIAL_JTAG
#elif defined(USE_ESP32_VARIANT_ESP32C3) || defined(USE_ESP32_VARIANT_ESP32C5) || \
defined(USE_ESP32_VARIANT_ESP32C6) || defined(USE_ESP32_VARIANT_ESP32C61) || defined(USE_ESP32_VARIANT_ESP32H2) || \
defined(USE_ESP32_VARIANT_ESP32H21) || defined(USE_ESP32_VARIANT_ESP32H4) || defined(USE_ESP32_VARIANT_ESP32P4) || \
defined(USE_ESP32_VARIANT_ESP32S3) || defined(USE_ESP32_VARIANT_ESP32S31)
defined(USE_ESP32_VARIANT_ESP32H4) || defined(USE_ESP32_VARIANT_ESP32P4) || defined(USE_ESP32_VARIANT_ESP32S3) || \
defined(USE_ESP32_VARIANT_ESP32S31)
#define USE_LOGGER_USB_CDC
#define USE_LOGGER_UART_SELECTION_USB_CDC
#define USE_LOGGER_USB_SERIAL_JTAG
#define USE_LOGGER_UART_SELECTION_USB_SERIAL_JTAG
#endif
#endif
+2 -2
View File
@@ -642,8 +642,8 @@ uint32_t HOT Scheduler::call(uint32_t now) {
// Not reached timeout yet, done for this call
break;
}
// Don't run on failed components
if (item->component != nullptr && item->component->is_failed()) {
// Don't run on failed components (see is_item_failed_ for the SELF_POINTER exception).
if (this->is_item_failed_(item)) {
LockGuard guard{this->lock_};
this->recycle_item_main_loop_(this->pop_raw_locked_());
continue;
+19 -7
View File
@@ -402,7 +402,7 @@ class Scheduler {
// Fixes: https://github.com/esphome/esphome/issues/11940
if (item == nullptr)
return false;
if (item->component != component || item->type != type || (skip_removed && this->is_item_removed_locked_(item)) ||
if (item->type != type || (skip_removed && this->is_item_removed_locked_(item)) ||
(match_retry && !item->is_retry)) {
return false;
}
@@ -411,23 +411,35 @@ class Scheduler {
return false;
// STATIC_STRING: compare string content. SELF_POINTER: raw pointer equality (no strcmp).
// Other types: compare hash/ID value.
if (name_type == NameType::SELF_POINTER) {
// SELF_POINTER keys are globally unique (the caller's `this`), so the stored component is
// log-attribution only (e.g. DelayAction records the current component for blocking
// warnings) and must NOT participate in matching. Match by pointer equality alone.
return item->name_.static_name == static_name;
}
// All other name types must also match on component identity.
if (item->component != component)
return false;
if (name_type == NameType::STATIC_STRING) {
return this->names_match_static_(item->get_name(), static_name);
}
if (name_type == NameType::SELF_POINTER) {
return item->name_.static_name == static_name;
}
return item->get_name_hash_or_id() == hash_or_id;
}
// Helper to execute a scheduler item
uint32_t execute_item_(SchedulerItem *item, uint32_t now);
// Helper to check if item should be skipped
bool should_skip_item_(SchedulerItem *item) const {
return is_item_removed_(item) || (item->component != nullptr && item->component->is_failed());
// True if the item belongs to a failed component and should therefore not run.
// SELF_POINTER items (e.g. DelayAction) store the component for log attribution only, so they
// must always fire regardless of that component's failed state and are never skipped here.
bool is_item_failed_(SchedulerItem *item) const {
return item->component != nullptr && item->get_name_type() != NameType::SELF_POINTER &&
item->component->is_failed();
}
// Helper to check if item should be skipped
bool should_skip_item_(SchedulerItem *item) const { return is_item_removed_(item) || this->is_item_failed_(item); }
// Helper to recycle a SchedulerItem back to the pool.
// Takes a raw pointer — caller transfers ownership. The item is either added to the
// pool or deleted if the pool is full.
-440
View File
@@ -1,440 +0,0 @@
"""Generate clang-tidy idedata via the native ESP-IDF toolchain.
Produces idedata for clang-tidy **without an ESPHome YAML config**. Instead of
running codegen on a config, it generates a minimal ESP-IDF CMake project:
* the managed-component dependencies come from ESPHome's own
``idf_component.yml`` (arduinojson, lvgl, mdns, ...);
* the PlatformIO ``lib_deps`` (qr-code, mlx90393, ...) are converted to local
IDF components via the ESPHome PlatformIO->IDF converter;
* the ``main`` component ``REQUIRES`` every target-available builtin IDF
component, so their public include dirs land on the translation unit;
* the repo ``sdkconfig.defaults`` enables sdkconfig-gated components (bt, ...).
then runs ``idf.py reconfigure`` (configure only, no compile) and reads the
resulting ``build/compile_commands.json``. The IDF version is the esp32
component's recommended version.
``ESPHOME_IDF_COMPILE_COMMANDS`` may point at an existing build's
``compile_commands.json`` to skip generation (fast iteration).
"""
from dataclasses import dataclass
import os
from pathlib import Path
TIDY_PROJECT_NAME = "esphome_tidy"
# A do-nothing C++ app: just enough for IDF to configure a valid project. It's
# C++ (not C) so the compile command uses the C++ compiler and flags, matching
# how clang-tidy analyzes ESPHome's C++ sources.
_TIDY_MAIN_CPP = 'extern "C" void app_main() {}\n'
@dataclass(frozen=True)
class _Settings:
"""Per-environment build settings derived from the tidy env name.
The platform defines below are what a real ESPHome build adds via
cg.add_define; defines.h only *consumes* them, so without them
esphome/core/hal.h errors with "not implemented for this platform".
"""
idf_target: str # esp32, esp32s3, ...
variant: str # ESP32, ESP32S3, ...
idf_version: str # ESP-IDF version to build with
target_framework: str # "espidf" or "arduino"
platform_defines: tuple[str, ...]
# Extra idf_component.yml deps the framework needs (e.g. arduino-esp32).
framework_deps: dict[str, dict]
def _settings_for(environment: str) -> _Settings:
"""Derive build settings from a ``<target>-<framework>-tidy`` env name.
Arduino on esp32 is itself a native ESP-IDF build with the
``espressif/arduino-esp32`` component added, so both frameworks use this
path -- only the defines, IDF version, and that one component differ.
"""
from esphome.components.esp32 import (
ARDUINO_FRAMEWORK_VERSION_LOOKUP,
ARDUINO_IDF_VERSION_LOOKUP,
ESP_IDF_FRAMEWORK_VERSION_LOOKUP,
)
parts = environment.split("-")
if len(parts) != 3 or parts[2] != "tidy" or parts[1] not in ("idf", "arduino"):
raise ValueError(
f"Unsupported clang-tidy environment {environment!r}: expected "
"<target>-<framework>-tidy with framework 'idf' or 'arduino' "
"(e.g. esp32-idf-tidy, esp32s3-arduino-tidy)"
)
idf_target, framework, _ = parts
variant = idf_target.upper()
# Defines shared by both frameworks. ESPHOME_LOG_LEVEL must be set up front
# (as the PlatformIO tidy build_flags do) -- otherwise log.h's ``#ifndef``
# sets it to NONE before defines.h redefines it, a macro-redefined warning
# across nearly every source.
common_defines = (
"USE_ESP32",
f"USE_ESP32_VARIANT_{variant}",
"ESPHOME_LOG_LEVEL=ESPHOME_LOG_LEVEL_VERY_VERBOSE",
)
if framework == "arduino":
fw_version = ARDUINO_FRAMEWORK_VERSION_LOOKUP["recommended"]
return _Settings(
idf_target=idf_target,
variant=variant,
idf_version=str(ARDUINO_IDF_VERSION_LOOKUP[fw_version]),
target_framework="arduino",
platform_defines=(
*common_defines,
"USE_ARDUINO",
"USE_ESP32_FRAMEWORK_ARDUINO",
),
framework_deps=_arduino_framework_deps(str(fw_version)),
)
return _Settings(
idf_target=idf_target,
variant=variant,
idf_version=str(ESP_IDF_FRAMEWORK_VERSION_LOOKUP["recommended"]),
target_framework="espidf",
platform_defines=(
*common_defines,
"USE_ESP_IDF",
"USE_ESP32_FRAMEWORK_ESP_IDF",
),
framework_deps={},
)
def _arduino_framework_deps(version: str) -> dict[str, dict]:
"""Arduino-only managed deps merged on top of esphome/idf_component.yml.
arduino-esp32 provides Arduino.h and the arduino libraries; its version is
the recommended arduino framework version so the tidy build matches what
ESPHome ships.
"""
from esphome.components.esp32 import ARDUINO_ESP32_COMPONENT_NAME
return {ARDUINO_ESP32_COMPONENT_NAME: {"version": version}}
_TOP_CMAKELISTS = """\
# Auto-generated by ESPHome (clang-tidy idedata project)
cmake_minimum_required(VERSION 3.16)
set(IDF_TARGET {idf_target})
include($ENV{{IDF_PATH}}/tools/cmake/project.cmake)
{compile_options}
project({name})
"""
_MAIN_CMAKELISTS = """\
# Auto-generated by ESPHome (clang-tidy idedata project)
idf_component_register(
SRCS "tidy.cpp"
REQUIRES {requires}
)
"""
def _setup_core(work_dir: Path, settings: _Settings) -> None:
"""Point CORE at the tidy project + IDF version, without any YAML config."""
from esphome.components.esp32.const import KEY_ESP32, KEY_IDF_VERSION, KEY_VARIANT
import esphome.config_validation as cv
from esphome.const import KEY_CORE, KEY_TARGET_FRAMEWORK, KEY_TARGET_PLATFORM
from esphome.core import CORE
CORE.name = TIDY_PROJECT_NAME
# config_path's parent is the data dir root: the IDF install lives at
# ``<parent>/.esphome/idf`` -- keep it beside (not inside) the per-run
# project dir so clearing the project doesn't force an IDF re-download.
CORE.config_path = work_dir.parent / "tidy.yaml"
CORE.build_path = work_dir
esp32 = CORE.data.setdefault(KEY_ESP32, {})
esp32[KEY_IDF_VERSION] = cv.Version.parse(settings.idf_version)
esp32[KEY_VARIANT] = settings.variant
# The target framework drives the PlatformIO-library -> IDF-component
# converter and ESPHome's CORE.using_arduino / using_esp_idf helpers.
CORE.data.setdefault(KEY_CORE, {})[KEY_TARGET_PLATFORM] = "esp32"
CORE.data[KEY_CORE][KEY_TARGET_FRAMEWORK] = settings.target_framework
# Special IDF "components" that are tools/subprojects, not requirable by an app
# (they provide no public includes and break requirement resolution), plus our
# own ``main``.
_NON_REQUIRABLE_COMPONENTS = frozenset(
{"bootloader", "esptool_py", "partition_table", "main"}
)
def _parse_lib_deps(platformio_ini: Path, framework: str):
"""Parse the framework's ``lib_deps`` from platformio.ini into Library specs.
These are the PlatformIO libraries ESPHome components pull in via
``cg.add_library``. The set is framework-specific: the arduino envs add
libs (FastLED, NeoPixelBus, MideaUART, ...) the idf envs don't. We read the
relevant ``[common*]`` sections directly (resolving the env's ``extends``
chain) and skip the ``${...}`` cross-references and non-library entries.
"""
import configparser
from esphome.core import Library
parser = configparser.ConfigParser(interpolation=None, strict=False)
parser.read(platformio_ini)
sections = [("common", "lib_deps_base"), ("common", "lib_deps")]
if framework == "arduino":
sections += [
("common:arduino", "lib_deps"),
("common:esp32-arduino", "lib_deps"),
]
else:
sections += [
("common:idf", "lib_deps"),
("common:esp32-idf", "lib_deps"),
]
tokens: list[str] = []
for section, key in sections:
if parser.has_option(section, key):
tokens += parser.get(section, key).splitlines()
libs: list[Library] = []
seen: set[str] = set()
for token in tokens:
token = token.split(";", 1)[0].strip() # drop trailing ; comment
# Skip blanks, ${...} cross-refs, and +<...> source filters.
if not token or token.startswith(("${", "+<")) or token in seen:
continue
seen.add(token)
if "://" in token or ".git" in token:
libs.append(Library(token, None, token)) # git repository (with #ref)
elif "@" in token:
name, _, version = token.partition("@")
libs.append(Library(name, version))
# A bare name (SPI, Wire, WiFi, Networking, "ESP32 Async UDP", ...) is an
# Arduino framework built-in provided by arduino-esp32, not a convertible
# registry library (no owner/version), so skip it.
return libs
def _convert_pio_libs(
platformio_ini: Path, framework: str
) -> dict[str, dict[str, str]]:
"""Convert the PlatformIO libs to IDF components; return manifest deps.
Returns a mapping suitable for an ``idf_component.yml`` ``dependencies``
block (``{name: {"override_path": <converted component dir>}}``), reusing
ESPHome's own PlatformIO->IDF converter (registry/git resolution, no pio).
The whole library set is resolved as a single batch so a shared transitive
dependency (e.g. esphome/libsodium pulled by both noise-c and esp_wireguard)
is deduplicated to one component instead of clashing override_path entries.
"""
from esphome.espidf.component import generate_idf_components
libraries = _parse_lib_deps(platformio_ini, framework)
deps: dict[str, dict[str, str]] = {}
for component in generate_idf_components(libraries):
deps[component.get_sanitized_name()] = {"override_path": str(component.path)}
return deps
def _arduino_excluded_stubs(work_dir: Path) -> dict[str, dict]:
"""Stub the arduino-bundled IDF components ESPHome doesn't use.
arduino-esp32 declares deps (libsodium, RainMaker, modbus, ...) that ESPHome
replaces with its own library (noise-c) or doesn't use; point each at an
empty override_path component so the IDF manager doesn't resolve/download
them -- notably so ``espressif/libsodium`` doesn't clash with the converted
noise-c's ``libsodium``. Mirrors esp32's ``_write_idf_component_yml``.
Components ESPHome's own idf_component.yml provides (e.g. lan867x for
ethernet) are NOT stubbed -- those are real deps we need, and arduino-esp32
resolves to the same component rather than conflicting.
"""
import yaml
from esphome.components.esp32 import (
ARDUINO_EXCLUDED_IDF_COMPONENTS,
_idf_component_dep_name,
_idf_component_stub_name,
)
esphome_dir = Path(__file__).resolve().parent.parent
base_manifest = yaml.safe_load(
(esphome_dir / "idf_component.yml").read_text(encoding="utf-8")
)
esphome_deps = set(base_manifest.get("dependencies") or {})
stubs_dir = work_dir / "component_stubs"
stubs_dir.mkdir(parents=True, exist_ok=True)
deps: dict[str, dict] = {}
for component in sorted(ARDUINO_EXCLUDED_IDF_COMPONENTS):
if _idf_component_dep_name(component) in esphome_deps:
continue # ESPHome needs this one for real (don't stub it away)
stub_path = stubs_dir / _idf_component_stub_name(component)
stub_path.mkdir(exist_ok=True)
(stub_path / "CMakeLists.txt").write_text(
"idf_component_register()\n", encoding="utf-8"
)
deps[_idf_component_dep_name(component)] = {
"version": "*",
"override_path": str(stub_path),
}
return deps
def _write_tidy_project(
work_dir: Path,
requires: list[str],
extra_deps: dict[str, dict[str, str]],
settings: _Settings,
) -> None:
"""Generate the minimal IDF CMake project (top + main + idf_component.yml)."""
main_dir = work_dir / "main"
main_dir.mkdir(parents=True, exist_ok=True)
compile_options = "\n".join(
f'idf_build_set_property(COMPILE_OPTIONS "-D{define}" APPEND)'
for define in settings.platform_defines
)
(work_dir / "CMakeLists.txt").write_text(
_TOP_CMAKELISTS.format(
name=TIDY_PROJECT_NAME,
compile_options=compile_options,
idf_target=settings.idf_target,
),
encoding="utf-8",
)
(main_dir / "CMakeLists.txt").write_text(
_MAIN_CMAKELISTS.format(requires=" ".join(requires)), encoding="utf-8"
)
(main_dir / "tidy.cpp").write_text(_TIDY_MAIN_CPP, encoding="utf-8")
# Managed components: ESPHome's own manifest (arduinojson, lvgl, mdns, ...),
# plus the converted PlatformIO libs as local (override_path) deps. Placing
# it in main/ makes every dep a requirement of the main component, so their
# public includes land on the tidy translation unit.
import yaml
esphome_dir = Path(__file__).resolve().parent.parent # esphome/espidf -> esphome
manifest = yaml.safe_load(
(esphome_dir / "idf_component.yml").read_text(encoding="utf-8")
)
manifest.setdefault("dependencies", {}).update(extra_deps)
(main_dir / "idf_component.yml").write_text(
yaml.safe_dump(manifest, sort_keys=False), encoding="utf-8"
)
# ESPHome's static-analysis sdkconfig (repo root): enables the flags any
# component sets (e.g. CONFIG_BT_ENABLED) so sdkconfig-gated IDF components
# register and expose their includes. IDF reads ``sdkconfig.defaults`` from
# the project root.
(work_dir / "sdkconfig.defaults").write_text(
(esphome_dir.parent / "sdkconfig.defaults").read_text(encoding="utf-8"),
encoding="utf-8",
)
def _generate_compile_commands(
work_dir: Path, settings: _Settings, platformio_ini: Path
) -> Path:
"""Generate the tidy project and run ``idf.py reconfigure`` (no build).
Two-phase, like a real ESPHome build: a first configure with no builtin
requires discovers which components actually register for the target (e.g.
``esp_tee`` only registers on c5/c6/h2), then a second configure requires
that discovered set so their public includes reach the tidy TU.
"""
import logging
from esphome.build_gen.espidf import get_available_components
from esphome.espidf import toolchain
# Surface ESPHome's INFO logs (ESP-IDF framework download/extract/install,
# git-library clones) -- they go through logging, which the clang-tidy
# script otherwise leaves at WARNING so the first-run downloads look silent.
logging.basicConfig(level=logging.INFO, format="%(message)s")
_setup_core(work_dir, settings)
# Framework deps (e.g. arduino-esp32) + PlatformIO libs converted to local
# IDF components, all added to the manifest as deps.
extra_deps = dict(settings.framework_deps)
extra_deps.update(_convert_pio_libs(platformio_ini, settings.target_framework))
if settings.target_framework == "arduino":
# Stub the arduino-bundled components ESPHome doesn't use (avoids the
# libsodium clash with noise-c and ~26 unused heavy downloads).
extra_deps.update(_arduino_excluded_stubs(work_dir))
# Phase 1: discover the components available for this target.
_write_tidy_project(work_dir, [], extra_deps, settings)
if toolchain.run_reconfigure() != 0:
raise RuntimeError("idf.py reconfigure (discovery) failed")
requires = sorted(
set(get_available_components() or []) - _NON_REQUIRABLE_COMPONENTS
)
# Phase 2: require every available builtin component.
_write_tidy_project(work_dir, requires, extra_deps, settings)
if toolchain.run_reconfigure() != 0:
raise RuntimeError("idf.py reconfigure failed")
return work_dir / "build" / "compile_commands.json"
def _idedata_from_tidy_project(compile_commands: Path) -> dict:
"""Assemble idedata from the single tidy translation unit.
Unlike a real ESPHome build (many ``/src/esphome/`` TUs unioned), the tidy
project has one TU (``main/tidy.cpp``) that -- by requiring every component --
already carries the full include set, so we parse it directly.
"""
import json
from esphome.espidf.idedata import _get_toolchain_includes, _parse_entry
entries = json.loads(Path(compile_commands).read_text(encoding="utf-8"))
entry = next((e for e in entries if e["file"].endswith("tidy.cpp")), None)
if entry is None:
raise RuntimeError(f"tidy.cpp not found in {compile_commands}")
cxx_path, defines, includes, cxx_flags = _parse_entry(entry)
return {
"cxx_path": cxx_path,
"cxx_flags": cxx_flags,
"defines": defines,
"includes": {
"build": includes,
"toolchain": _get_toolchain_includes(cxx_path),
},
}
def load_idedata(environment: str, temp_folder: str, platformio_ini: Path) -> dict:
if explicit := os.environ.get("ESPHOME_IDF_COMPILE_COMMANDS"):
compile_commands = Path(explicit)
else:
# The tidy env is ``<target>-<framework>-tidy`` (e.g. esp32-idf-tidy,
# esp32s3-arduino-tidy); derive the target, variant and framework.
settings = _settings_for(environment)
# Resolve to an absolute path: ``override_path`` entries in the generated
# component manifests are interpreted by the IDF component manager relative
# to the manifest's own directory, so a relative work dir would be
# mis-resolved (doubled under ``main/``).
work_dir = (
Path(temp_folder)
/ f"idf-tidy-{settings.idf_target}-{settings.target_framework}"
).resolve()
compile_commands = _generate_compile_commands(
work_dir, settings, platformio_ini
)
if not compile_commands.is_file():
raise RuntimeError(f"compile_commands.json not found: {compile_commands}")
return _idedata_from_tidy_project(compile_commands)
+287 -279
View File
@@ -1,6 +1,4 @@
from collections import deque
from collections.abc import Callable
from dataclasses import dataclass, field
import glob
import hashlib
import itertools
@@ -10,7 +8,7 @@ import os
from pathlib import Path
import re
import tempfile
from typing import Any, TypeVar
from typing import TypeVar
from urllib.parse import urlparse, urlsplit, urlunsplit
from esphome import git, yaml_util
@@ -156,6 +154,72 @@ class IDFComponent:
self.path = self.source.download(self.get_sanitized_name(), force=force)
def _get_package_from_pio_registry(
username: str | None, pkgname: str, requirements: str
) -> tuple[str, str, str | None, str | None]:
"""
Fetch package information from PlatformIO registry.
This function queries the PlatformIO registry to find a library package
that matches the given criteria and returns its metadata including version
and download URL.
Args:
username: The owner/username of the package (can be None)
pkgname: The name of the package
requirements: Version requirements (e.g., "^1.0.0")
Returns:
tuple[str, str, str | None, str | None]:
A tuple containing (owner, name, version, download_url)
where version and download_url can be None if not found
"""
from platformio.package.manager._registry import PackageManagerRegistryMixin
from platformio.package.meta import PackageSpec
# Create a minimal PackageManagerRegistry class
class PackageManagerRegistry(PackageManagerRegistryMixin):
def __init__(self):
self._registry_client = None
self.pkg_type = "library"
@staticmethod
def is_system_compatible(value, custom_system=None):
return True
pio_registry = PackageManagerRegistry()
# Fetch package metadata from registry
package = pio_registry.fetch_registry_package(
PackageSpec(
owner=username,
name=pkgname,
)
)
owner = package["owner"]["username"]
name = package["name"]
# Find the best matching version based on requirements
version = pio_registry.pick_best_registry_version(
package.get("versions"),
PackageSpec(owner=username, name=pkgname, requirements=requirements),
)
# If no version found, return with None for version and URL
if not version:
return owner, name, None, None
# Find the compatible package file for this version
pkgfile = pio_registry.pick_compatible_pkg_file(version["files"])
# If no package file found, return with None for URL but valid version
if not pkgfile:
return owner, name, version["name"], None
return owner, name, version["name"], pkgfile["download_url"]
def _apply_extra_script(component: IDFComponent) -> None:
"""Run a PIO ``extraScript`` and fold its captured env vars into
``component.data["build"]["flags"]`` so the existing -L/-l/-D
@@ -275,6 +339,77 @@ def _collect_filtered_files(src_dir: PathType, src_filters: list[str]) -> list[s
return [r for r in selected if Path(r).is_file()]
def _convert_library_to_component(library: Library) -> IDFComponent:
"""
Convert a Library object to an IDFComponent object by resolving its metadata.
This function handles the conversion of library specifications to component
objects, resolving versions through PlatformIO registry when needed or
parsing direct repository URLs.
Args:
library: The Library object containing name, version, and/or repository information
Returns:
IDFComponent: The resolved component with name, version, and URL
Raises:
RuntimeError: If no artifact can be found for the library
"""
name = None
version = None
source = None
# Repository is provided directly
if library.repository:
# Parse repository URL: path becomes the component name, fragment
# (if any) becomes the git ref stored on GitSource. A missing
# fragment is fine -- clone_or_update leaves the depth-1 clone on
# the remote's default branch, matching PIO's lib_deps behavior
# and external_components handling.
split_result = urlsplit(library.repository)
# Sanitize name
name = str(split_result.path).strip("/")
name = name.removesuffix(".git")
# IDF Component Manager only accepts "*", a 40-char commit hash, or
# semver here. The actual git ref is preserved in GitSource.ref;
# override_path makes this field cosmetic at build time.
version = "*"
repository = urlunsplit(split_result._replace(fragment=""))
ref = split_result.fragment.strip() or None
source = GitSource(str(repository), ref)
# Version is provided - resolve using PlatformIO registry
elif library.version:
name = library.name
if "/" not in name:
owner, pkgname = None, name
else:
owner, pkgname = name.split("/", 1)
owner, pkgname, version, url = _get_package_from_pio_registry(
owner, pkgname, library.version
)
if url is None:
raise RuntimeError(
f"Can't find an pkg file from PlatformIO registry for library {library}"
)
name = _owner_pkgname_to_name(owner, pkgname)
source = URLSource(url)
if source is None:
raise RuntimeError(f"Can't find an artifact associated to library {library}")
assert name, "Missing library name"
assert version, "Missing library version"
return IDFComponent(name, version, source)
def _split_list_by_condition(
items: list[str], match_fn: Callable[[str], str | None]
) -> tuple[list[str], list[str]]:
@@ -464,8 +599,8 @@ def generate_idf_component_yml(component: IDFComponent) -> str:
if "dependencies" not in data:
data["dependencies"] = {}
# Every dependency has been resolved and downloaded before this runs,
# so .path is always set.
# Every dependency goes through _generate_idf_component →
# component.download() before this runs, so .path is always set.
data["dependencies"][dependency.get_sanitized_name()] = {
"override_path": str(dependency.path),
}
@@ -522,6 +657,81 @@ def _check_library_data(data: dict):
)
def _process_dependencies(component: IDFComponent):
"""
Process library dependencies and generate ESP-IDF components.
Args:
component: IDFComponent object being processed
Returns:
None
"""
name, version = component.name, component.version
dependencies = component.data.get("dependencies")
if not dependencies:
return
# PIO's library.json accepts both the list-of-dicts form and the
# shorthand dict form ``{"owner/Name": "version_spec"}``. Normalize
# the dict form so the loop below sees a uniform list. Iterating a
# dict gives string keys, which would silently fail the
# ``"name" in dependency`` substring check and skip every entry.
if isinstance(dependencies, dict):
normalized = []
for raw_name, spec in dependencies.items():
if "/" in raw_name:
owner, pkgname = raw_name.split("/", 1)
else:
owner, pkgname = None, raw_name
entry = {"name": pkgname, "owner": owner}
if isinstance(spec, dict):
entry.update(spec)
else:
entry["version"] = spec
normalized.append(entry)
dependencies = normalized
_LOGGER.info("Processing %s@%s component dependencies...", name, version)
for dependency in dependencies:
# Validate dependency structure
if not all(k in dependency for k in ("name", "version")):
_LOGGER.debug("Ignore invalid library: %s", dependency)
continue
try:
_check_library_data(dependency)
except InvalidIDFComponent as e:
_LOGGER.debug(
"Skip %s@%s: %s", dependency["name"], dependency["version"], str(e)
)
continue
# The version field may actually contain a URL
version = dependency["version"]
url = None
try:
result = urlparse(version)
if all([result.scheme, result.netloc]):
url, version = version, None
except (TypeError, ValueError):
pass
# Generate ESP-IDF component from PlatformIO library
component.dependencies.append(
_generate_idf_component(
Library(
_owner_pkgname_to_name(
dependency.get("owner", None), dependency.get("name")
),
version,
url,
)
)
)
def _parse_library_json(library_json_path: PathType):
"""
Load and parse a JSON file describing a library.
@@ -562,294 +772,92 @@ def _parse_library_properties(library_properties_path: PathType):
return data
def _make_registry_client() -> Any:
"""Create a minimal PlatformIO registry client with no system filtering.
``is_system_compatible`` is forced True so version selection is driven purely
by the requested version requirements -- ESP-IDF/target compatibility is
handled elsewhere, not by the PlatformIO registry.
def _generate_idf_component(library: Library, force: bool = False) -> IDFComponent:
"""
from platformio.package.manager._registry import PackageManagerRegistryMixin
Generate an ESP-IDF component from a library specification.
class _Registry(PackageManagerRegistryMixin):
def __init__(self) -> None:
self._registry_client = None
self.pkg_type = "library"
This function resolves the library, downloads it, processes metadata files,
and generates necessary ESP-IDF build files (CMakeLists.txt, idf_component.yml).
@staticmethod
def is_system_compatible(value: Any, custom_system: Any = None) -> bool:
return True
Args:
library: The library specification containing name, version, and repository URL
force: If True, forces re-download of the library even if it exists locally
return _Registry()
def _resolve_registry_version(
owner: str | None, pkgname: str, requirements: set[str]
) -> tuple[str, str, str, str]:
"""Resolve a registry package to the single highest version satisfying ALL
the given requirements; return ``(owner, name, version, download_url)``.
Intersecting every requirement (rather than resolving each consumer in
isolation) makes the result independent of processing order and guarantees
no stated constraint is violated -- e.g. ``esphome/libsodium`` requested as
both ``==1.10021.0`` and ``^1.10018.1`` resolves to ``1.10021.0``.
Returns:
IDFComponent: The generated component object with resolved metadata
"""
from platformio.package.meta import PackageSpec
_LOGGER.info("Generate IDF component for %s library ...", library)
registry = _make_registry_client()
package = registry.fetch_registry_package(PackageSpec(owner=owner, name=pkgname))
owner = package["owner"]["username"]
name = package["name"]
# Resolve component name, version and url
component = _convert_library_to_component(library)
name, version = component.name, component.version
# Chaining the per-requirement filter intersects all constraints.
versions = package.get("versions") or []
for requirement in sorted(requirements):
versions = registry.get_compatible_registry_versions(
versions, PackageSpec(owner=owner, name=name, requirements=requirement)
)
if not versions:
raise RuntimeError(
f"No version of {owner}/{name} satisfies all requirements "
f"{sorted(requirements)} requested across the library tree"
)
# Download the library
component.download(force)
best = registry.pick_best_registry_version(versions)
pkgfile = registry.pick_compatible_pkg_file(best["files"])
if not pkgfile:
raise RuntimeError(f"No package file for {owner}/{name}@{best['name']}")
return owner, name, best["name"], pkgfile["download_url"]
# Paths to component metadata and build files
library_json_path = component.path / "library.json"
library_properties_path = component.path / "library.properties"
cmakelists_txt_path = component.path / "CMakeLists.txt"
idf_component_yml_path = component.path / "idf_component.yml"
# Bundled CMakeLists.txt / idf_component.yml are ignored -- library
# authors' IDF support is frequently broken (bogus REQUIRES, hard-coded
# arduino-esp32, etc.). We always regenerate.
def _normalize_dependencies(dependencies: Any) -> list[dict]:
"""Normalize a library manifest's ``dependencies`` to a list of dicts.
PIO's library.json accepts both the list-of-dicts form and the shorthand
dict form (``{"owner/Name": "version_spec"}``); normalize the latter so
callers see a uniform list.
"""
if not dependencies:
return []
if isinstance(dependencies, dict):
normalized = []
for raw_name, spec in dependencies.items():
if "/" in raw_name:
owner, pkgname = raw_name.split("/", 1)
else:
owner, pkgname = None, raw_name
entry = {"name": pkgname, "owner": owner}
if isinstance(spec, dict):
entry.update(spec)
else:
entry["version"] = spec
normalized.append(entry)
return normalized
return [d for d in dependencies if isinstance(d, dict)]
@dataclass
class _LibNode:
"""A node in the library dependency graph being resolved as a batch."""
key: str
is_git: bool
owner: str | None = None
pkgname: str | None = None
requirements: set[str] = field(default_factory=set)
url: str | None = None
ref: str | None = None
edges: set[str] = field(default_factory=set)
def _node_key(
name: str | None, version: str | None, repository: str | None
) -> tuple[str, bool, tuple[str | None, str | None]]:
"""Return ``(key, is_git, locator)`` for a library or dependency spec.
The key is derived from the *input* spec (the registry name as written, or
the git URL path), not the resolved canonical name. So a package referenced
inconsistently -- bare ``name`` vs ``owner/name``, or git vs registry -- maps
to distinct keys and isn't deduplicated; ``generate_idf_components`` warns
about that after resolution rather than merging the nodes.
"""
if repository:
split_result = urlsplit(repository)
key = str(split_result.path).strip("/").removesuffix(".git")
ref = split_result.fragment.strip() or None
url = urlunsplit(split_result._replace(fragment=""))
return key, True, (url, ref)
if name and "/" in name:
owner, pkgname = name.split("/", 1)
if library_json_path.is_file():
component.data = _parse_library_json(library_json_path)
elif library_properties_path.is_file():
component.data = _parse_library_properties(library_properties_path)
else:
owner, pkgname = None, name
return name, False, (owner, pkgname)
raise RuntimeError(
"Invalid PIO library: missing library.json and/or library.properties"
)
# Check if the component is usable with ESP-IDF before executing any
# third-party Python from the library (``_apply_extra_script`` below).
_check_library_data(component.data)
# If the library declares a PIO ``extraScript``, run it against a
# fake SCons env so we can fold its captured LIBPATH/LIBS/etc into
# the build-flag pipeline ``generate_cmakelists_txt`` consumes
# below. Without this, libraries that wire per-MCU archive linking
# via extraScript fail to link under native ESP-IDF.
_apply_extra_script(component)
# Handle the dependencies (convert PlatformIO library to ESP-IDF component if needed)
_process_dependencies(component)
_LOGGER.debug("Generating CMakeLists.txt for %s@%s ...", name, version)
write_file_if_changed(
cmakelists_txt_path,
generate_cmakelists_txt(component),
)
_LOGGER.debug("Generating idf_component.yml for %s@%s ...", name, version)
write_file_if_changed(
idf_component_yml_path,
generate_idf_component_yml(component),
)
return component
def generate_idf_components(libraries: list[Library]) -> list[IDFComponent]:
"""Resolve and convert a batch of PlatformIO libraries to IDF components.
Resolves the whole set together rather than each library independently: it
walks the dependency graph collecting every version *requirement* per
component name, then resolves each name once to a single version satisfying
all of them. So a transitive dependency shared under
different specs (e.g. ``esphome/libsodium``, pulled by both ``noise-c`` and
``esp_wireguard``) becomes one component instead of two clashing
``override_path`` entries -- order-independently, and without ever violating
a stated constraint.
The returned list holds the top-level components (those directly requested);
transitive dependencies are converted too and wired into each component's
generated manifest.
def generate_idf_component(
library: Library, force: bool = False
) -> tuple[str, str, Path]:
"""
nodes: dict[str, _LibNode] = {}
Generate an ESP-IDF component and return its name, version, and path.
def add_spec(name: str | None, version: str | None, repository: str | None) -> str:
key, is_git, locator = _node_key(name, version, repository)
node = nodes.get(key) or _LibNode(key=key, is_git=is_git)
nodes[key] = node
if is_git:
node.is_git = True
node.url, node.ref = locator
else:
node.owner, node.pkgname = locator
if version:
node.requirements.add(version)
return key
This is a wrapper function that calls _generate_idf_component and returns
the standardized tuple format (name, version, path).
top_level = [
add_spec(library.name, library.version, library.repository)
for library in libraries
]
Args:
library: The library specification containing name, version, and repository URL
force: If True, forces re-download of the library even if it exists locally
# Collect + resolve to a fixpoint: a node is (re)resolved whenever its
# requirement set has grown since the last time, so every requirement in the
# graph is accounted for before conversion.
components: dict[str, IDFComponent] = {}
resolved_requirements: dict[str, frozenset[str]] = {}
top_level_keys = set(top_level)
worklist = deque(dict.fromkeys(top_level))
while worklist:
key = worklist.popleft()
node = nodes[key]
# A node is queued once per referring edge; skip the (uncached) registry
# lookup + download + dependency walk unless its requirement set grew
# since the last resolve. Requirements only ever grow, so this still
# converges the fixpoint and terminates dependency cycles.
requirements = frozenset(node.requirements)
if resolved_requirements.get(key) == requirements:
continue
resolved_requirements[key] = requirements
if node.is_git:
component = IDFComponent(key, "*", GitSource(node.url, node.ref))
else:
owner, name, version, url = _resolve_registry_version(
node.owner, node.pkgname, node.requirements
)
component = IDFComponent(
_owner_pkgname_to_name(owner, name), version, URLSource(url)
)
component.download()
library_json_path = component.path / "library.json"
library_properties_path = component.path / "library.properties"
if library_json_path.is_file():
component.data = _parse_library_json(library_json_path)
elif library_properties_path.is_file():
component.data = _parse_library_properties(library_properties_path)
else:
raise RuntimeError(
f"Invalid PIO library {key}: missing library.json and "
"library.properties"
)
try:
_check_library_data(component.data)
except InvalidIDFComponent as e:
# Skip an incompatible transitive dependency, but fail fast if a
# top-level library the build explicitly requested is incompatible.
if key in top_level_keys:
raise RuntimeError(
f"Requested library {key} is not compatible with ESP-IDF: {e}"
) from e
_LOGGER.debug("Skip incompatible dependency %s: %s", key, str(e))
continue
components[key] = component
# Requirements changed (we got past the short-circuit above), so
# (re)walk this component's dependencies.
node.edges = set()
for dependency in _normalize_dependencies(component.data.get("dependencies")):
if "name" not in dependency or "version" not in dependency:
continue
try:
_check_library_data(dependency)
except InvalidIDFComponent as e:
_LOGGER.debug("Skip dependency %s: %s", dependency.get("name"), str(e))
continue
# The version field may actually be a URL (git/archive dependency).
dep_version = dependency["version"]
dep_url = None
try:
parsed = urlparse(dep_version)
if all([parsed.scheme, parsed.netloc]):
dep_url, dep_version = dep_version, None
except (TypeError, ValueError):
pass
dep_key = add_spec(
_owner_pkgname_to_name(dependency.get("owner"), dependency.get("name")),
dep_version,
dep_url,
)
node.edges.add(dep_key)
worklist.append(dep_key)
# A git source wins over any registry version requested for the same
# component. That's intentional, but warn so a dropped registry pin isn't a
# silent surprise.
for node in nodes.values():
if node.is_git and node.requirements:
_LOGGER.warning(
"Library %s is requested both from a git source (%s) and as "
"registry version(s) %s; using the git source.",
node.key,
node.url,
sorted(node.requirements),
)
# Two graph nodes that resolve to the same component name (e.g. a package
# referenced both bare and as ``owner/name``) are not deduplicated and can
# produce conflicting component definitions. Warn so it's not silent.
canonical_keys: dict[str, str] = {}
for node_key, component in components.items():
canonical = component.get_sanitized_name()
if canonical_keys.setdefault(canonical, node_key) != node_key:
_LOGGER.warning(
"Library %s is referenced under multiple names (%s and %s); these "
"are not deduplicated. Reference it consistently as %s.",
canonical,
canonical_keys[canonical],
node_key,
canonical,
)
# Wire each component's dependencies to the single resolved instances, then
# regenerate build files.
for key, component in components.items():
component.dependencies = [
components[dep_key]
for dep_key in sorted(nodes[key].edges)
if dep_key in components
]
for component in components.values():
_apply_extra_script(component)
write_file_if_changed(
component.path / "CMakeLists.txt",
generate_cmakelists_txt(component),
)
write_file_if_changed(
component.path / "idf_component.yml",
generate_idf_component_yml(component),
)
return [components[key] for key in top_level if key in components]
Returns:
tuple[str, str, Path]: A tuple containing (component_name, component_version, component_path)
"""
component = _generate_idf_component(library, force)
return component.get_sanitized_name(), component.version, component.path
+2 -2
View File
@@ -6,8 +6,8 @@ section instead of static fields. The script runs under SCons during
PIO's build and mutates the active ``Environment`` (``env.Append``,
``env.Replace``, ) chiefly to set ``LIBPATH``/``LIBS`` per chip MCU.
ESPHome's PIO→IDF converter doesn't run SCons, so these scripts were
previously ignored and any library
ESPHome's PIO→IDF converter (``_generate_idf_component``) doesn't run
SCons, so these scripts were previously ignored and any library
relying on them failed to link under ``toolchain: esp-idf``. This
module provides a small shim that ``exec``s an extra-script with a
fake ``env`` object, captures the common ``env.Append(...)`` calls,
+1 -3
View File
@@ -1005,9 +1005,7 @@ def _check_esphome_idf_framework_install(
idf_tools_path = framework_path / "tools" / "idf_tools.py"
_LOGGER.info("Checking ESP-IDF %s framework ...", version)
# Logged every invocation (not just on install) so the user can verify the
# override. A changed URL needs ``esphome clean-all`` to force a re-download
# (``esphome clean`` only wipes the build dir, not the extracted framework
# under <data_dir>/idf/frameworks/<version>).
# override. A changed URL needs ``esphome clean`` to force a re-download.
if source_url:
_LOGGER.info("Using framework source override: %s", source_url)
+3 -26
View File
@@ -16,7 +16,7 @@ from esphome.const import (
KEY_TARGET_PLATFORM,
Toolchain,
)
from esphome.core import CORE, EsphomeError
from esphome.core import CORE
from esphome.helpers import write_file_if_changed
from esphome.types import CoreType
@@ -101,7 +101,6 @@ class StorageJSON:
core_platform: str | None = None,
toolchain: str | None = None,
area: str | None = None,
framework_version: str | None = None,
) -> None:
# Version of the storage JSON schema
assert storage_version is None or isinstance(storage_version, int)
@@ -142,8 +141,6 @@ class StorageJSON:
self.toolchain = toolchain
# The area of the node
self.area = area
# The framework version the build used (for esp32, the resolved ESP-IDF version)
self.framework_version = framework_version
def as_dict(self):
return {
@@ -165,7 +162,6 @@ class StorageJSON:
"core_platform": self.core_platform,
"toolchain": self.toolchain,
"area": self.area,
"framework_version": self.framework_version,
}
def to_json(self):
@@ -177,12 +173,10 @@ class StorageJSON:
@staticmethod
def from_esphome_core(esph: CoreType, old: StorageJSON | None) -> StorageJSON:
hardware = esph.target_platform.upper()
framework_version: str | None = None
if esph.is_esp32:
from esphome.components import esp32
hardware = esp32.get_esp32_variant(esph)
framework_version = str(esp32.idf_version())
return StorageJSON(
storage_version=1,
name=esph.name,
@@ -206,7 +200,6 @@ class StorageJSON:
core_platform=esph.target_platform,
toolchain=esph.toolchain.value if esph.toolchain is not None else None,
area=esph.area,
framework_version=framework_version,
)
@staticmethod
@@ -256,7 +249,6 @@ class StorageJSON:
core_platform = storage.get("core_platform")
toolchain = storage.get("toolchain")
area = storage.get("area")
framework_version = storage.get("framework_version")
return StorageJSON(
storage_version,
name,
@@ -276,7 +268,6 @@ class StorageJSON:
core_platform,
toolchain,
area,
framework_version,
)
@staticmethod
@@ -320,24 +311,10 @@ class StorageJSON:
# esp32.get_esp32_variant(). target_platform on disk is the variant
# (e.g. "ESP32S3"); core_platform is the family (e.g. "esp32").
if target_platform == const.PLATFORM_ESP32:
from esphome.components.esp32.const import KEY_ESP32, KEY_IDF_VERSION
from esphome.components.esp32.const import KEY_ESP32
from esphome.const import KEY_VARIANT
esp32_data = {KEY_VARIANT: self.target_platform}
if self.framework_version:
import esphome.config_validation as cv
try:
esp32_data[KEY_IDF_VERSION] = cv.Version.parse(
self.framework_version
)
except ValueError as err:
raise EsphomeError(
f"Could not parse the framework version "
f"{self.framework_version!r} from {storage_path()}. "
f"Please clean the build files and recompile."
) from err
CORE.data[KEY_ESP32] = esp32_data
CORE.data[KEY_ESP32] = {KEY_VARIANT: self.target_platform}
def __eq__(self, o) -> bool:
return isinstance(o, StorageJSON) and self.as_dict() == o.as_dict()
+31 -86
View File
@@ -93,12 +93,9 @@ def storage_should_clean(old: StorageJSON | None, new: StorageJSON) -> bool:
``src_version`` differs, ``build_path`` differs, the build
``toolchain`` differs (e.g. switching between the PlatformIO and
native ESP-IDF toolchains, which produce incompatible build trees),
the ``framework`` or ``framework_version`` differs (e.g. switching
arduino <-> esp-idf, or bumping the ESP-IDF version, which also
produce incompatible build trees), or a previously loaded
integration was removed in *new*. Adding integrations or changing
unrelated fields (friendly name, esphome version, etc.) does not
trigger a clean.
or a previously loaded integration was removed in *new*. Adding
integrations or changing unrelated fields (friendly name, esphome
version, etc.) does not trigger a clean.
Used by esphome-device-builder (esphome/device-builder) to gate
its remote-build artifact materialiser so a local remote local
@@ -116,10 +113,6 @@ def storage_should_clean(old: StorageJSON | None, new: StorageJSON) -> bool:
return True
if old.toolchain != new.toolchain:
return True
if old.framework != new.framework:
return True
if old.framework_version != new.framework_version:
return True
# Check if any components have been removed
return bool(old.loaded_integrations - new.loaded_integrations)
@@ -133,13 +126,6 @@ def storage_should_update_cmake_cache(old: StorageJSON, new: StorageJSON) -> boo
def update_storage_json() -> None:
"""Refresh the storage sidecar and clean an incompatible build.
Runs at the start of ``write_cpp`` -- BEFORE any source/project files are
regenerated -- so the clean below can safely ``full``-wipe the whole build
directory (a switch of toolchain/framework/version also drops the stale
project scaffolding, not just the compiled objects).
"""
path = storage_path()
old = StorageJSON.load(path)
new = StorageJSON.from_esphome_core(CORE, old)
@@ -160,7 +146,7 @@ def update_storage_json() -> None:
)
else:
_LOGGER.info("Core config or version changed, cleaning build files...")
clean_build(clear_pio_cache=False, full=True)
clean_build(clear_pio_cache=False)
elif storage_should_update_cmake_cache(old, new):
_LOGGER.info("Integrations changed, cleaning cmake cache...")
clean_cmake_cache()
@@ -497,89 +483,48 @@ def write_cpp(code_s):
def clean_cmake_cache():
# Drop the CMake cache so a component-set change forces a reconfigure.
# PlatformIO keeps it under .pioenvs/<name>/; the native ESP-IDF toolchain
# keeps it under build/ (where espidf's has_outdated_files() treats a
# missing CMakeCache.txt as stale). Only one exists for a given build.
cmake_cache_paths = (
CORE.relative_pioenvs_path(CORE.name, "CMakeCache.txt"),
CORE.relative_build_path("build", "CMakeCache.txt"),
)
for cmake_cache_path in cmake_cache_paths:
if cmake_cache_path.is_file():
_LOGGER.info("Deleting %s", cmake_cache_path)
cmake_cache_path.unlink()
pioenvs = CORE.relative_pioenvs_path()
if pioenvs.is_dir():
pioenvs_cmake_path = pioenvs / CORE.name / "CMakeCache.txt"
if pioenvs_cmake_path.is_file():
_LOGGER.info("Deleting %s", pioenvs_cmake_path)
pioenvs_cmake_path.unlink()
def clean_build(clear_pio_cache: bool = True, *, full: bool = False):
"""Remove build artifacts.
By default only the compiled outputs are removed (``.pioenvs`` /
``.piolibdeps`` / the native ESP-IDF ``build`` and ``managed_components``
dirs) while the generated ``src/`` and project files are kept. This is what
in-build callers need: they regenerate a source/sdkconfig and then force a
rebuild without discarding the sources they just wrote.
``full=True`` wipes the entire build directory instead. Used by the
``esphome clean`` command and by the pre-build clean in
``update_storage_json`` (which runs before sources are regenerated) -- in
both cases nothing is mid-regeneration, so the next compile rebuilds from
scratch. It also drops stale project scaffolding the allow-list keeps (e.g. a
leftover platformio.ini / CMakeLists.txt from the other toolchain), making a
toolchain switch reliable.
"""
def clean_build(clear_pio_cache: bool = True):
# Allow skipping cache cleaning for integration tests
if os.environ.get("ESPHOME_SKIP_CLEAN_BUILD"):
_LOGGER.warning("Skipping build cleaning (ESPHOME_SKIP_CLEAN_BUILD set)")
return
if full:
if CORE.build_path is not None:
build_path = Path(CORE.build_path)
if build_path.is_dir():
_LOGGER.info("Deleting %s", build_path)
rmtree(build_path)
else:
pioenvs = CORE.relative_pioenvs_path()
if pioenvs.is_dir():
_LOGGER.info("Deleting %s", pioenvs)
rmtree(pioenvs)
piolibdeps = CORE.relative_piolibdeps_path()
if piolibdeps.is_dir():
_LOGGER.info("Deleting %s", piolibdeps)
rmtree(piolibdeps)
dependencies_lock = CORE.relative_build_path("dependencies.lock")
if dependencies_lock.is_file():
_LOGGER.info("Deleting %s", dependencies_lock)
dependencies_lock.unlink()
# Native ESP-IDF toolchain artifacts: the IDF CMake/ninja build dir
# and the Component Manager's fetched managed components live under
# the project's build path, not under .pioenvs / .piolibdeps.
for name in ("build", "managed_components"):
idf_path = CORE.relative_build_path(name)
if idf_path.is_dir():
_LOGGER.info("Deleting %s", idf_path)
rmtree(idf_path)
# The idedata cache is derived from the build but lives under the data dir,
# not the build path, so it must be removed separately in both modes.
pioenvs = CORE.relative_pioenvs_path()
if pioenvs.is_dir():
_LOGGER.info("Deleting %s", pioenvs)
rmtree(pioenvs)
piolibdeps = CORE.relative_piolibdeps_path()
if piolibdeps.is_dir():
_LOGGER.info("Deleting %s", piolibdeps)
rmtree(piolibdeps)
dependencies_lock = CORE.relative_build_path("dependencies.lock")
if dependencies_lock.is_file():
_LOGGER.info("Deleting %s", dependencies_lock)
dependencies_lock.unlink()
idedata_cache = CORE.relative_internal_path("idedata", f"{CORE.name}.json")
if idedata_cache.is_file():
_LOGGER.info("Deleting %s", idedata_cache)
idedata_cache.unlink()
# Native ESP-IDF toolchain artifacts: the IDF CMake/ninja build dir
# and the Component Manager's fetched managed components live under
# the project's build path, not under .pioenvs / .piolibdeps.
for name in ("build", "managed_components"):
idf_path = CORE.relative_build_path(name)
if idf_path.is_dir():
_LOGGER.info("Deleting %s", idf_path)
rmtree(idf_path)
if not clear_pio_cache:
return
# The native ESP-IDF toolchain caches PlatformIO libraries converted to IDF
# components under <data_dir>/pio_components, shared across builds and keyed
# by source hash (the analog of PlatformIO's global package cache). Drop it
# on an explicit clean so a corrupt/stale converted lib is re-fetched.
pio_components = CORE.relative_internal_path("pio_components")
if pio_components.is_dir():
_LOGGER.info("Deleting %s", pio_components)
rmtree(pio_components)
# Clean PlatformIO cache to resolve CMake compiler detection issues
# This helps when toolchain paths change or get corrupted
try:
+1 -1
View File
@@ -37,7 +37,7 @@ lib_deps_base =
wjtje/qr-code-generator-library@1.7.0 ; qr_code
functionpointer/arduino-MLX90393@1.0.2 ; mlx90393
pavlodn/HaierProtocol@0.9.31 ; haier
esphome/dsmr_parser@1.8.0 ; dsmr
esphome/dsmr_parser@1.4.0 ; dsmr
https://github.com/esphome/TinyGPSPlus.git#v1.1.0 ; gps
; This is using the repository until a new release is published to PlatformIO
https://github.com/Sensirion/arduino-gas-index-algorithm.git#3.2.1 ; Sensirion Gas Index Algorithm Arduino Library
+1 -1
View File
@@ -9,7 +9,7 @@ tzlocal==5.3.1 # from time
tzdata>=2026.2 # from time
pyserial==3.5
platformio==6.1.19
esptool==5.3.0
esptool==5.2.0
click==8.3.3
esphome-dashboard==20260425.0
aioesphomeapi==45.3.1
+2 -24
View File
@@ -28,12 +28,6 @@ from helpers import (
temp_header_file,
)
# Limit the ESP-IDF tool install to esp32 for clang-tidy: the one xtensa-esp-elf
# toolchain bundles the s2/s3 compilers too, so all xtensa tidy envs still
# reconfigure while the large riscv32-esp-elf toolchain is skipped. Must be set
# before esphome.espidf.framework is imported (lazily, via load_idedata).
os.environ.setdefault("ESPHOME_IDF_DEFAULT_TARGETS", "esp32")
def clang_options(idedata):
cmd = []
@@ -58,10 +52,6 @@ def clang_options(idedata):
"-mfix-esp32-psram-cache-issue",
"-mfix-esp32-psram-cache-strategy=memw",
"-fno-tree-switch-conversion",
# GCC-only flags emitted by the native ESP-IDF toolchain build
"-freorder-blocks",
"-fno-jump-tables",
"-fno-shrink-wrap",
)
if "zephyr" in triplet:
@@ -107,20 +97,8 @@ def clang_options(idedata):
]
)
# Copy compiler flags, dropping: ones clang doesn't understand; -Werror*
# (clang-tidy enforces .clang-tidy's WarningsAsErrors, and a build -Werror
# would bypass the -clang-diagnostic-* suppressions); and -std= (the native
# ESP-IDF build defaults to gnu++2b, but ESPHome compiles with gnu++20 per
# platformio.ini -- analyzing as C++23 flags code that doesn't build under
# gnu++20). Force gnu++20 to match the real build.
cmd.extend(
flag
for flag in idedata["cxx_flags"]
if flag not in omit_flags
and not flag.startswith("-Werror")
and not flag.startswith("-std=")
)
cmd.append("-std=gnu++20")
# copy compiler flags, except those clang doesn't understand.
cmd.extend(flag for flag in idedata["cxx_flags"] if flag not in omit_flags)
# defines
cmd.extend(f"-D{define}" for define in idedata["defines"])
-6
View File
@@ -105,12 +105,6 @@ def calculate_clang_tidy_hash(repo_root: Path | None = None) -> str:
sdkconfig_content = read_file_bytes(sdkconfig_path)
hasher.update(sdkconfig_content)
# Hash esphome/idf_component.yml: its managed deps drive the ESP-IDF
# build's include set, which clang-tidy analyzes.
idf_component_path = repo_root / "esphome" / "idf_component.yml"
if idf_component_path.exists():
hasher.update(read_file_bytes(idf_component_path))
return hasher.hexdigest()
+42 -62
View File
@@ -70,7 +70,6 @@ from helpers import (
get_changed_components,
get_component_from_path,
get_component_test_files,
get_component_test_platforms,
get_components_with_dependencies,
get_cpp_changed_components,
get_fixture_to_test_files,
@@ -78,6 +77,7 @@ from helpers import (
get_target_branch,
git_ls_files,
is_validate_only_file,
parse_test_filename,
root_path,
)
from split_components_for_ci import create_intelligent_batches
@@ -169,6 +169,24 @@ MEMORY_IMPACT_FALLBACK_COMPONENT = "api" # Representative component for core ch
MEMORY_IMPACT_FALLBACK_PLATFORM = Platform.ESP32_IDF # Most representative platform
MEMORY_IMPACT_MAX_COMPONENTS = 40 # Max components before results become nonsensical
# Platform-specific components that can only be built on their respective platforms
# These components contain platform-specific code and cannot be cross-compiled
# Regular components (wifi, logger, api, etc.) are cross-platform and not listed here
PLATFORM_SPECIFIC_COMPONENTS = frozenset(
{
"esp32", # ESP32 platform implementation
"esp8266", # ESP8266 platform implementation
"rp2040", # Raspberry Pi Pico / RP2040 platform implementation
"libretiny", # LibreTiny base platform implementation
"bk72xx", # Beken BK72xx platform implementation (uses LibreTiny)
"rtl87xx", # Realtek RTL87xx platform implementation (uses LibreTiny)
"ln882x", # Winner Micro LN882x platform implementation (uses LibreTiny)
"host", # Host platform (for testing on development machine)
"nrf52", # Nordic nRF52 platform implementation (uses Zephyr)
"zephyr", # Zephyr RTOS platform implementation
}
)
# Platform preference order for memory impact analysis
# This order is used when no platform-specific hints are detected from filenames
# Priority rationale:
@@ -988,24 +1006,23 @@ def detect_memory_impact_config(
] = {} # Track which platforms each component supports
for component in sorted(changed_component_set):
# Discover the platforms this component has BASE tests for, using the
# same logic as the build runner (get_component_test_platforms wraps the
# shared get_component_test_files + parse_test_filename helpers). Base
# tests only: the memory impact CI build runs test_build_components.py
# with --base-only, which compiles base test.<platform>.yaml files but
# never variant test-<variant>.<platform>.yaml files. Counting
# variant-only platforms here would let us select a platform the build
# then has nothing to compile for, producing no memory output.
available_platforms = {
Platform(platform)
for platform in get_component_test_platforms(component)
if platform in MEMORY_IMPACT_PLATFORM_PREFERENCE
}
# Look for test files on preferred platforms
test_files = get_component_test_files(component, all_variants=True)
if not test_files:
continue
# Check if component has tests for any preferred platform
available_platforms = [
platform
for test_file in test_files
if (platform := parse_test_filename(test_file)[1]) != "all"
and platform in MEMORY_IMPACT_PLATFORM_PREFERENCE
]
if not available_platforms:
continue
component_platforms_map[component] = available_platforms
component_platforms_map[component] = set(available_platforms)
components_with_tests.append(component)
# If no components have tests, don't run memory impact
@@ -1067,56 +1084,19 @@ def detect_memory_impact_config(
)
platform = _select_platform_by_count(platform_counts)
# Keep only components that have a base test on the selected platform.
# The merged build runs test_build_components.py -t <platform> --base-only,
# so a component without a base test.<platform>.yaml compiles nothing and
# contributes no memory output. This also covers platform-specific
# components (esp32, esp8266, etc.), which only have tests on their own
# platform. When components don't share a common platform we build the
# largest subset that does, dropping the rest.
def components_supporting(target: Platform) -> list[str]:
return [
component
for component in components_with_tests
if target in component_platforms_map.get(component, set())
]
compatible_components = components_supporting(platform)
# A platform hint (or no-common-platform fallback) can pick a platform that
# no changed component actually has a base test for, leaving nothing to
# build. In that case fall back to the platform supported by the most
# components. component_platforms_map is non-empty (guarded above) and every
# value is a non-empty platform set (components with no supported platform
# are skipped at discovery), so this always yields a buildable platform with
# at least one compatible component.
if not compatible_components:
platform = _select_platform_by_count(
Counter(
p for platforms in component_platforms_map.values() for p in platforms
)
)
compatible_components = components_supporting(platform)
# Defensive backstop: unreachable given the invariant above, but guards
# against a future regression in platform selection silently passing an
# empty component list to the build.
if not compatible_components:
return {"should_run": "false"}
# Log components dropped because they lack a base test on the selected
# platform so partial-subset builds are visible in CI logs.
dropped_components = [
# Filter out platform-specific components that are incompatible with selected platform
# Platform components (esp32, esp8266, rp2040, etc.) can only build on their own platform
# Other components (wifi, logger, etc.) are cross-platform and can build anywhere
compatible_components = [
component
for component in components_with_tests
if component not in compatible_components
if component not in PLATFORM_SPECIFIC_COMPONENTS
or platform in component_platforms_map.get(component, set())
]
if dropped_components:
print(
f"Memory impact: Dropping components without a base test on "
f"{platform}: {dropped_components}",
file=sys.stderr,
)
# If no components are compatible with the selected platform, don't run
if not compatible_components:
return {"should_run": "false"}
# Debug output
print("Memory impact analysis:", file=sys.stderr)
+19 -46
View File
@@ -149,31 +149,6 @@ def get_component_test_files(
return files
def get_component_test_platforms(component: str, *, base_only: bool = True) -> set[str]:
"""Return the set of platforms a component has compilable test files for.
Uses the same discovery as ``test_build_components.py`` (``get_component_test_files``
+ ``parse_test_filename``) so callers agree with what the build runner would
actually compile. With ``base_only=True`` (the default, matching the
memory-impact build's ``--base-only``), only base ``test.<platform>.yaml``
files are considered; variant ``test-<variant>.<platform>.yaml`` files are
excluded. The ``"all"`` platform sentinel is excluded.
Args:
component: Component name (e.g. "wifi")
base_only: If True, only consider base test files (default).
Returns:
Set of platform identifiers (e.g. {"esp32-idf", "esp8266-ard"}).
"""
platforms: set[str] = set()
for test_file in get_component_test_files(component, all_variants=not base_only):
platform = parse_test_filename(test_file)[1]
if platform != "all":
platforms.add(platform)
return platforms
def is_validate_only_file(test_file: Path) -> bool:
"""Return True if the given path is a config-only validate file.
@@ -664,22 +639,26 @@ def load_idedata(environment: str) -> dict[str, Any]:
start_time = time.time()
print(f"Loading IDE data for environment '{environment}'...")
# Reuse the clang-tidy input hash as the cache key: it already covers every
# file baked into the generated idedata (platformio.ini, sdkconfig.defaults,
# esphome/idf_component.yml), so this can't drift from that file list. A
# content hash -- unlike an mtime comparison -- stays correct across git
# checkouts, which don't preserve mtimes.
from clang_tidy_hash import calculate_clang_tidy_hash
platformio_ini = Path(root_path) / "platformio.ini"
temp_idedata = Path(temp_folder) / f"idedata-{environment}.json"
temp_hash = Path(temp_folder) / f"idedata-{environment}.hash"
changed = False
if (
not platformio_ini.is_file()
or not temp_idedata.is_file()
or platformio_ini.stat().st_mtime >= temp_idedata.stat().st_mtime
):
changed = True
cache_key = calculate_clang_tidy_hash()
changed = (
not temp_idedata.is_file()
or not temp_hash.is_file()
or temp_hash.read_text().strip() != cache_key
)
if "idf" in environment:
# remove full sdkconfig when the defaults have changed so that it is regenerated
default_sdkconfig = Path(root_path) / "sdkconfig.defaults"
temp_sdkconfig = Path(temp_folder) / f"sdkconfig-{environment}"
if not temp_sdkconfig.is_file():
changed = True
elif default_sdkconfig.stat().st_mtime >= temp_sdkconfig.stat().st_mtime:
temp_sdkconfig.unlink()
changed = True
if not changed:
data = json.loads(temp_idedata.read_text())
@@ -690,12 +669,7 @@ def load_idedata(environment: str) -> dict[str, Any]:
# ensure temp directory exists before running pio, as it writes sdkconfig to it
Path(temp_folder).mkdir(exist_ok=True)
platformio_ini = Path(root_path) / "platformio.ini"
if "esp32" in environment:
from esphome.espidf.clang_tidy import load_idedata as idf_load_idedata
data = idf_load_idedata(environment, temp_folder, platformio_ini)
elif "nrf" in environment:
if "nrf" in environment:
from helpers_zephyr import load_idedata as zephyr_load_idedata
data = zephyr_load_idedata(environment, temp_folder, platformio_ini)
@@ -706,7 +680,6 @@ def load_idedata(environment: str) -> dict[str, Any]:
match = re.search(r'{\s*".*}', stdout.decode("utf-8"))
data = json.loads(match.group())
temp_idedata.write_text(json.dumps(data, indent=2) + "\n")
temp_hash.write_text(cache_key + "\n")
elapsed = time.time() - start_time
print(f"IDE data generated and cached in {elapsed:.2f} seconds")
+15 -1
View File
@@ -42,7 +42,6 @@ from script.analyze_component_buses import (
from script.helpers import (
get_component_test_files,
is_validate_only_file,
parse_test_filename,
split_conflicting_groups,
)
from script.merge_component_configs import merge_component_configs
@@ -123,6 +122,21 @@ def find_component_tests(
return dict(component_tests)
def parse_test_filename(test_file: Path) -> tuple[str, str]:
"""Parse test filename to extract test name and platform.
Args:
test_file: Path to test file
Returns:
Tuple of (test_name, platform)
"""
parts = test_file.stem.split(".")
if len(parts) == 2:
return parts[0], parts[1] # test, platform
return parts[0], "all"
def get_platform_base_files(base_dir: Path) -> dict[str, list[Path]]:
"""Get all platform base files.
+10 -5
View File
@@ -1,11 +1,15 @@
# ESP-IDF sdkconfig defaults used for development purposes only, not used during runtime. Used for static analysis
# (clang-tidy) -- by both the PlatformIO and the native ESP-IDF toolchain paths -- and when PlatformIO is run directly
# from the source directory (e.g. by IDEs). This should enable all flags that are set by any component.
# ESP-IDF sdkconfig defaults used for development purposes only, not used during runtime. Used when PlatformIO is ran
# directly from the source directory, e.g. by IDEs or for static analysis (clang-tidy). This should enable all flags
# that are set by any component.
# esp32
CONFIG_COMPILER_OPTIMIZATION_DEFAULT=n
CONFIG_COMPILER_OPTIMIZATION_SIZE=y
CONFIG_PARTITION_TABLE_CUSTOM=y
#CONFIG_PARTITION_TABLE_CUSTOM_FILENAME="partitions.csv"
CONFIG_PARTITION_TABLE_SINGLE_APP=n
CONFIG_FREERTOS_HZ=1000
CONFIG_ESP_TASK_WDT_INIT=y
CONFIG_ESP_TASK_WDT=y
CONFIG_ESP_TASK_WDT_PANIC=y
CONFIG_ESP_TASK_WDT_CHECK_IDLE_TASK_CPU0=n
CONFIG_ESP_TASK_WDT_CHECK_IDLE_TASK_CPU1=n
@@ -14,7 +18,8 @@ CONFIG_ESP_TASK_WDT_CHECK_IDLE_TASK_CPU1=n
CONFIG_BT_ENABLED=y
# esp32_camera
CONFIG_SPIRAM=y
CONFIG_RTCIO_SUPPORT_RTC_GPIO_DESC=y
CONFIG_ESP32_SPIRAM_SUPPORT=y
# zigbee
CONFIG_ZB_ENABLED=y
@@ -4,145 +4,77 @@ from unittest.mock import patch
import pytest
from esphome.components.const import BYTE_ORDER_BIG, BYTE_ORDER_LITTLE
from esphome.components.display import (
DisplayMetaData,
add_metadata,
get_all_display_metadata,
get_display_metadata,
)
from esphome.config import Config
from esphome.core import ID
from esphome.final_validate import full_config
from esphome.cpp_generator import MockObj
def test_add_metadata_basic():
"""Test adding metadata with an ID object."""
def test_add_metadata_with_string_id():
"""Test adding metadata with a plain string ID."""
with patch("esphome.components.display.CORE.data", {}):
add_metadata(ID("my_display"), 320, 240)
meta = get_display_metadata(ID("my_display"))
add_metadata("my_display", 320, 240, True)
meta = get_display_metadata("my_display")
assert meta == DisplayMetaData(
width=320,
height=240,
has_hardware_rotation=False,
byte_order=BYTE_ORDER_BIG,
width=320, height=240, has_writer=True, has_hardware_rotation=False
)
def test_add_metadata_with_all_fields():
"""Test adding metadata with all fields set."""
def test_add_metadata_with_mockobj_id():
"""Test adding metadata with a MockObj ID (converted via str())."""
with patch("esphome.components.display.CORE.data", {}):
add_metadata(
ID("my_display"),
480,
320,
has_hardware_rotation=True,
byte_order=BYTE_ORDER_LITTLE,
)
meta = get_display_metadata(ID("my_display"))
mock_id = MockObj("my_display_obj")
add_metadata(mock_id, 480, 320, False, has_hardware_rotation=True)
meta = get_display_metadata("my_display_obj")
assert meta == DisplayMetaData(
width=480,
height=320,
has_hardware_rotation=True,
byte_order=BYTE_ORDER_LITTLE,
width=480, height=320, has_writer=False, has_hardware_rotation=True
)
def test_add_metadata_hardware_rotation_default():
"""Test that has_hardware_rotation defaults to False."""
with patch("esphome.components.display.CORE.data", {}):
add_metadata(ID("disp"), 128, 64)
meta = get_display_metadata(ID("disp"))
add_metadata("disp", 128, 64, False)
meta = get_display_metadata("disp")
assert meta.has_hardware_rotation is False
assert meta.byte_order == BYTE_ORDER_BIG
def test_add_metadata_with_byte_order():
"""Test adding metadata with explicit byte_order."""
def test_get_display_metadata_missing_returns_none():
"""Test that querying a non-existent ID returns None."""
with patch("esphome.components.display.CORE.data", {}):
add_metadata(ID("disp"), 240, 320, byte_order=BYTE_ORDER_LITTLE)
meta = get_display_metadata(ID("disp"))
assert meta.byte_order == BYTE_ORDER_LITTLE
def test_get_display_metadata_missing_reads_raw_config():
"""Querying a non-existent ID falls back to raw config lookup."""
with patch("esphome.components.display.CORE.data", {}):
# Set up a minimal full_config with a display entry so the fallback
# path in get_display_metadata can find the display config.
fc = Config()
fc["display"] = [
{
"id": ID("no_such_display", True),
"auto_clear_enabled": True,
"dimensions": {"width": 320, "height": 240},
"byte_order": BYTE_ORDER_LITTLE,
"rotation": 90,
},
{
"id": ID("other_display", True),
"auto_clear_enabled": "undefined",
"dimensions": (1024, 600),
},
]
fc.declare_ids.append((ID("no_such_display", True), ["display", 0, "id"]))
fc.declare_ids.append((ID("other_display", True), ["display", 1, "id"]))
full_config.set(fc)
data = get_display_metadata(ID("no_such_display"))
assert data.width == 320
assert data.height == 240
assert data.has_hardware_rotation is False
assert data.has_writer is True
assert data.byte_order == BYTE_ORDER_LITTLE
assert data.rotation == 90
data = get_display_metadata(ID("other_display"))
assert data.width == 1024
assert data.height == 600
data = get_display_metadata("no_such_display")
assert data.width == 0
assert data.height == 0
assert data.has_writer is False
assert data.has_hardware_rotation is False
def test_add_multiple_displays():
"""Test adding metadata for multiple displays."""
with patch("esphome.components.display.CORE.data", {}):
add_metadata(ID("disp_a"), 320, 240)
add_metadata(ID("disp_b"), 128, 64, has_hardware_rotation=True)
add_metadata("disp_a", 320, 240, True)
add_metadata("disp_b", 128, 64, False, has_hardware_rotation=True)
all_meta = get_all_display_metadata()
assert len(all_meta) == 2
assert all_meta["disp_a"] == DisplayMetaData(320, 240, False)
assert all_meta["disp_b"] == DisplayMetaData(128, 64, True, BYTE_ORDER_BIG)
assert all_meta["disp_a"] == DisplayMetaData(320, 240, True, False)
assert all_meta["disp_b"] == DisplayMetaData(128, 64, False, True)
def test_add_duplicate_id_asserts():
"""Adding metadata for the same ID object twice should assert."""
def test_add_metadata_overwrites_existing():
"""Test that adding metadata for the same ID overwrites the previous entry."""
with patch("esphome.components.display.CORE.data", {}):
id_obj = ID("disp")
add_metadata(id_obj, 320, 240)
with pytest.raises(AssertionError, match="Duplicate"):
add_metadata(id_obj, 640, 480)
add_metadata("disp", 320, 240, True)
add_metadata("disp", 640, 480, False, has_hardware_rotation=True)
meta = get_display_metadata("disp")
assert meta == DisplayMetaData(640, 480, False, True)
def test_metadata_is_frozen():
"""Test that DisplayMetaData instances are immutable (frozen dataclass)."""
meta = DisplayMetaData(320, 240, False, BYTE_ORDER_BIG)
meta = DisplayMetaData(320, 240, True, False)
with pytest.raises(AttributeError):
meta.width = 640
with pytest.raises(AttributeError):
meta.byte_order = BYTE_ORDER_LITTLE
def test_get_all_metadata_asserts_on_unresolved_id():
"""get_all_display_metadata should assert if any ID has id=None."""
with patch("esphome.components.display.CORE.data", {}):
add_metadata(ID(None), 320, 240)
with pytest.raises(AssertionError, match="resolved"):
get_all_display_metadata()
def test_get_metadata_asserts_on_unresolved_id():
"""get_display_metadata should assert if any ID has id=None."""
with patch("esphome.components.display.CORE.data", {}):
add_metadata(ID(None), 320, 240)
with pytest.raises(AssertionError, match="resolved"):
get_display_metadata(ID("anything"))
@@ -1,47 +0,0 @@
"""Tests for esp32_ble_server configuration helpers."""
import pytest
from esphome.components.esp32_ble_server import (
CCCD_DESCRIPTOR_UUID,
CUD_DESCRIPTOR_UUID,
DEVICE_INFORMATION_SERVICE_UUID,
uuid_is,
)
@pytest.mark.parametrize(
"uuid",
[
DEVICE_INFORMATION_SERVICE_UUID, # int form (cv.hex_uint32_t)
"180A", # 16 bit short form (bt_uuid)
"180a", # lowercase is normalized by bt_uuid but guard anyway
"0000180A", # 32 bit form
"0000180A-0000-1000-8000-00805F9B34FB", # full 128 bit form
],
)
def test_uuid_is_matches_all_representations(uuid) -> None:
"""All representations of the same 16 bit UUID must compare equal."""
assert uuid_is(uuid, DEVICE_INFORMATION_SERVICE_UUID)
@pytest.mark.parametrize(
"uuid",
[
0x1818, # Cycling Power Service (different int)
"1818", # different 16 bit short form
"0000180B", # adjacent UUID
"0000180A-0000-1000-8000-00805F9B34FC", # wrong base UUID suffix
],
)
def test_uuid_is_rejects_other_uuids(uuid) -> None:
"""A different UUID must not be mistaken for the device information service."""
assert not uuid_is(uuid, DEVICE_INFORMATION_SERVICE_UUID)
@pytest.mark.parametrize("uuid16", [CUD_DESCRIPTOR_UUID, CCCD_DESCRIPTOR_UUID])
def test_uuid_is_matches_descriptor_short_strings(uuid16) -> None:
"""Reserved descriptor UUIDs match whether given as int or short string."""
assert uuid_is(uuid16, uuid16)
assert uuid_is(f"{uuid16:04X}", uuid16)
assert uuid_is(f"{uuid16:08X}", uuid16)
@@ -1,177 +0,0 @@
"""Tests for LVGL final_validation display metadata checks."""
from __future__ import annotations
import pytest
from esphome.components.const import BYTE_ORDER_BIG, BYTE_ORDER_LITTLE, CONF_BYTE_ORDER
from esphome.components.display import add_metadata
from esphome.components.lvgl import final_validation
from esphome.config import Config
from esphome.config_validation import Invalid
from esphome.const import KEY_CORE, KEY_TARGET_FRAMEWORK, KEY_TARGET_PLATFORM
from esphome.core import CORE, ID
from esphome.final_validate import full_config
@pytest.fixture(autouse=True)
def _setup_core():
"""Ensure CORE.data has enough context for final_validation."""
CORE.data[KEY_CORE] = {
KEY_TARGET_PLATFORM: "host",
KEY_TARGET_FRAMEWORK: "",
}
full_config.set(Config())
yield
CORE.reset()
def _register_displays(*display_ids: str) -> None:
"""Register display IDs in full_config so get_path_for_id works."""
fc = full_config.get()
display_list = [{"id": ID(d, True)} for d in display_ids]
fc["display"] = display_list
for i, disp_id in enumerate(display_ids):
fc.declare_ids.append((ID(disp_id, True), ["display", i, "id"]))
def _make_lvgl_config(
display_ids: list[str],
byte_order: str | None = None,
) -> dict:
"""Build a minimal LVGL config dict for final_validation."""
_register_displays(*display_ids)
config = {
"displays": [ID(d, True) for d in display_ids],
"log_level": "WARN",
"color_depth": 16,
"transparency_key": 0x000400,
"draw_rounding": 2,
"buffer_size": 0,
}
if byte_order is not None:
config[CONF_BYTE_ORDER] = byte_order
return config
class TestByteOrderAutoConfig:
"""Test that LVGL auto-configures byte_order from display metadata."""
def test_inherits_big_endian_from_display(self) -> None:
"""LVGL should inherit big_endian from display metadata."""
add_metadata(ID("my_disp"), 320, 240, byte_order=BYTE_ORDER_BIG)
configs = [_make_lvgl_config(["my_disp"])]
final_validation(configs)
assert configs[0][CONF_BYTE_ORDER] == BYTE_ORDER_BIG
def test_inherits_little_endian_from_display(self) -> None:
"""LVGL should inherit little_endian from display metadata."""
add_metadata(ID("my_disp"), 320, 240, byte_order=BYTE_ORDER_LITTLE)
configs = [_make_lvgl_config(["my_disp"])]
final_validation(configs)
assert configs[0][CONF_BYTE_ORDER] == BYTE_ORDER_LITTLE
def test_defaults_to_big_endian_when_no_metadata(self) -> None:
"""LVGL should default to big_endian when display has no metadata."""
configs = [_make_lvgl_config(["my_disp"])]
final_validation(configs)
assert configs[0][CONF_BYTE_ORDER] == BYTE_ORDER_BIG
class TestByteOrderExplicitMismatchError:
"""Test that LVGL rejects explicit byte_order mismatch with display."""
def test_raises_on_mismatch(self) -> None:
"""Explicit LVGL byte_order different from display should raise."""
add_metadata(ID("my_disp"), 320, 240, byte_order=BYTE_ORDER_LITTLE)
configs = [_make_lvgl_config(["my_disp"], byte_order=BYTE_ORDER_BIG)]
with pytest.raises(
Invalid, match="LVGL byte order must match the display byte order"
):
final_validation(configs)
def test_no_error_when_matching(self) -> None:
"""Explicit LVGL byte_order matching display should pass."""
add_metadata(ID("my_disp"), 320, 240, byte_order=BYTE_ORDER_BIG)
configs = [_make_lvgl_config(["my_disp"], byte_order=BYTE_ORDER_BIG)]
final_validation(configs)
class TestByteOrderMultipleDisplays:
"""Test byte_order validation with multiple displays."""
def test_consistent_displays_inherit(self) -> None:
"""All displays with same byte_order should set LVGL byte_order."""
add_metadata(ID("disp_a"), 320, 240, byte_order=BYTE_ORDER_LITTLE)
add_metadata(ID("disp_b"), 128, 64, byte_order=BYTE_ORDER_LITTLE)
configs = [_make_lvgl_config(["disp_a", "disp_b"])]
final_validation(configs)
assert configs[0][CONF_BYTE_ORDER] == BYTE_ORDER_LITTLE
def test_inconsistent_displays_raises(self) -> None:
"""Displays with different byte_order should raise an error."""
add_metadata(ID("disp_a"), 320, 240, byte_order=BYTE_ORDER_BIG)
add_metadata(ID("disp_b"), 128, 64, byte_order=BYTE_ORDER_LITTLE)
configs = [_make_lvgl_config(["disp_a", "disp_b"])]
with pytest.raises(Invalid, match="same byte_order"):
final_validation(configs)
class TestHasWriterCheck:
"""Test that LVGL rejects displays with has_writer set."""
def test_display_with_writer_raises(self) -> None:
"""Display with lambda/pages/auto_clear should be rejected."""
add_metadata(ID("my_disp"), 320, 240, has_writer=True)
configs = [_make_lvgl_config(["my_disp"])]
with pytest.raises(Invalid, match="not compatible with LVGL"):
final_validation(configs)
def test_display_without_writer_passes(self) -> None:
"""Display without writer should pass."""
add_metadata(ID("my_disp"), 320, 240, has_writer=False)
configs = [_make_lvgl_config(["my_disp"])]
final_validation(configs)
class TestRotationCheck:
"""Test that LVGL rejects displays with non-zero rotation."""
def test_display_with_rotation_raises(self) -> None:
"""Display with rotation should be rejected."""
add_metadata(ID("my_disp"), 320, 240, rotation=90)
configs = [_make_lvgl_config(["my_disp"])]
with pytest.raises(Invalid, match="rotation.*not compatible with LVGL"):
final_validation(configs)
def test_display_without_rotation_passes(self) -> None:
"""Display with rotation=0 should pass."""
add_metadata(ID("my_disp"), 320, 240, rotation=0)
configs = [_make_lvgl_config(["my_disp"])]
final_validation(configs)
class TestDrawRoundingMerge:
"""Test that display draw_rounding is merged into LVGL config."""
def test_display_draw_rounding_overrides_lower(self) -> None:
"""Display draw_rounding higher than LVGL default should win."""
add_metadata(ID("my_disp"), 320, 240, draw_rounding=8)
configs = [_make_lvgl_config(["my_disp"])]
final_validation(configs)
assert configs[0]["draw_rounding"] == 8
def test_display_draw_rounding_does_not_lower(self) -> None:
"""Display draw_rounding lower than LVGL config should not reduce it."""
add_metadata(ID("my_disp"), 320, 240, draw_rounding=1)
configs = [_make_lvgl_config(["my_disp"])]
configs[0]["draw_rounding"] = 4
final_validation(configs)
assert configs[0]["draw_rounding"] == 4
def test_zero_draw_rounding_no_change(self) -> None:
"""Display with draw_rounding=0 should not affect LVGL config."""
add_metadata(ID("my_disp"), 320, 240, draw_rounding=0)
configs = [_make_lvgl_config(["my_disp"])]
final_validation(configs)
assert configs[0]["draw_rounding"] == 2
@@ -3,15 +3,22 @@
from collections.abc import Callable
from pathlib import Path
from esphome.components.const import BYTE_ORDER_BIG
from esphome.components.display import get_all_display_metadata, get_display_metadata
from esphome.components.display import (
DisplayMetaData,
get_all_display_metadata,
get_display_metadata,
)
from esphome.components.esp32 import (
KEY_BOARD,
KEY_VARIANT,
VARIANT_ESP32,
VARIANT_ESP32S3,
)
from esphome.components.mipi_spi.display import CONFIG_SCHEMA, FINAL_VALIDATE_SCHEMA
from esphome.components.mipi_spi.display import (
CONFIG_SCHEMA,
FINAL_VALIDATE_SCHEMA,
get_instance,
)
from esphome.const import PlatformFramework
from tests.component_tests.types import SetCoreConfigCallable
@@ -31,32 +38,38 @@ def test_metadata_native_quad_default_test_card(
PlatformFramework.ESP32_IDF,
platform_data={KEY_BOARD: "esp32-s3-devkitc-1", KEY_VARIANT: VARIANT_ESP32S3},
)
config = CONFIG_SCHEMA({"model": "JC3636W518", "id": "jc3232w518"})
meta = get_display_metadata(config["id"])
config = validated_config({"model": "JC3636W518"})
get_instance(config)
meta = get_display_metadata(str(config["id"]))
assert meta is not None
assert meta.width == 360
assert meta.height == 360
# final validation auto-enables show_test_card when no drawing methods are configured
assert meta.has_writer is True
assert meta.has_hardware_rotation is True
assert meta.byte_order == BYTE_ORDER_BIG
def test_metadata_single_mode_with_dc_pin(
set_core_config: SetCoreConfigCallable,
) -> None:
"""A single-mode display with no explicit drawing gets metadata from schema validation."""
"""A single-mode display with no explicit drawing gets a test card from final validation."""
set_core_config(
PlatformFramework.ESP32_IDF,
platform_data={KEY_BOARD: "esp32dev", KEY_VARIANT: VARIANT_ESP32},
)
config = CONFIG_SCHEMA(
{"model": "ST7735", "dc_pin": 18, "id": "single_mode_with_dc_pin"}
config = validated_config(
{
"model": "ST7735",
"dc_pin": 18,
}
)
meta = get_display_metadata(config["id"])
get_instance(config)
meta = get_display_metadata(str(config["id"]))
assert meta is not None
assert meta.width == 128
assert meta.height == 160
assert meta.has_writer is True
assert meta.has_hardware_rotation is True
assert meta.byte_order == BYTE_ORDER_BIG
def test_metadata_custom_dimensions(
@@ -67,22 +80,47 @@ def test_metadata_custom_dimensions(
PlatformFramework.ESP32_IDF,
platform_data={KEY_BOARD: "esp32dev", KEY_VARIANT: VARIANT_ESP32},
)
config = CONFIG_SCHEMA(
config = validated_config(
{
"model": "custom",
"dc_pin": 18,
"dimensions": {"width": 480, "height": 320},
"init_sequence": [[0xA0, 0x01]],
"id": "custom_dimensions",
}
)
meta = get_display_metadata(config["id"])
get_instance(config)
meta = get_display_metadata(str(config["id"]))
assert meta is not None
assert meta.width == 480
assert meta.height == 320
# final validation auto-enables show_test_card
assert meta.has_writer is True
assert meta.has_hardware_rotation is True
def test_metadata_with_test_card_has_writer(
set_core_config: SetCoreConfigCallable,
) -> None:
"""When show_test_card is enabled, has_writer should be True."""
set_core_config(
PlatformFramework.ESP32_IDF,
platform_data={KEY_BOARD: "esp32dev", KEY_VARIANT: VARIANT_ESP32},
)
config = validated_config(
{
"model": "custom",
"dc_pin": 18,
"dimensions": {"width": 240, "height": 240},
"init_sequence": [[0xA0, 0x01]],
"show_test_card": True,
}
)
get_instance(config)
meta = get_display_metadata(str(config["id"]))
assert meta is not None
assert meta.has_writer is True
def test_metadata_no_swap_xy_not_full_hardware_rotation(
set_core_config: SetCoreConfigCallable,
) -> None:
@@ -92,8 +130,9 @@ def test_metadata_no_swap_xy_not_full_hardware_rotation(
platform_data={KEY_BOARD: "esp32-s3-devkitc-1", KEY_VARIANT: VARIANT_ESP32S3},
)
# JC3248W535 has swap_xy=cv.UNDEFINED -> transforms={mirror_x, mirror_y} only
config = CONFIG_SCHEMA({"model": "JC3248W535", "id": "jc3248w535"})
meta = get_display_metadata(config["id"])
config = validated_config({"model": "JC3248W535"})
get_instance(config)
meta = get_display_metadata(str(config["id"]))
assert meta is not None
assert meta.has_hardware_rotation is False
@@ -106,7 +145,7 @@ def test_metadata_multiple_displays_independent(
PlatformFramework.ESP32_IDF,
platform_data={KEY_BOARD: "esp32dev", KEY_VARIANT: VARIANT_ESP32},
)
CONFIG_SCHEMA(
config_a = validated_config(
{
"id": "disp_a",
"model": "custom",
@@ -115,7 +154,7 @@ def test_metadata_multiple_displays_independent(
"init_sequence": [[0xA0, 0x01]],
}
)
CONFIG_SCHEMA(
config_b = validated_config(
{
"id": "disp_b",
"model": "custom",
@@ -124,16 +163,13 @@ def test_metadata_multiple_displays_independent(
"init_sequence": [[0xA0, 0x01]],
}
)
get_instance(config_a)
get_instance(config_b)
all_meta = get_all_display_metadata()
assert all_meta["disp_a"].width == 320
assert all_meta["disp_a"].height == 240
assert all_meta["disp_a"].has_hardware_rotation is True
assert all_meta["disp_a"].byte_order == BYTE_ORDER_BIG
assert all_meta["disp_b"].width == 128
assert all_meta["disp_b"].height == 64
assert all_meta["disp_b"].has_hardware_rotation is True
assert all_meta["disp_b"].byte_order == BYTE_ORDER_BIG
# final validation auto-enables show_test_card for both
assert all_meta["disp_a"] == DisplayMetaData(320, 240, True, True)
assert all_meta["disp_b"] == DisplayMetaData(128, 64, True, True)
def test_metadata_via_code_generation_native(
@@ -143,13 +179,12 @@ def test_metadata_via_code_generation_native(
"""Full code generation for native.yaml should produce correct metadata."""
generate_main(component_fixture_path("native.yaml"))
all_meta = get_all_display_metadata()
# native.yaml: model JC3636W518 -> 360x360, full hardware rotation
# native.yaml: model JC3636W518 -> 360x360, no writer, full hardware rotation
assert len(all_meta) == 1
meta = next(iter(all_meta.values()))
assert meta.width == 360
assert meta.height == 360
assert meta.has_hardware_rotation is True
assert meta.byte_order == BYTE_ORDER_BIG
assert meta == DisplayMetaData(
width=360, height=360, has_writer=True, has_hardware_rotation=True
)
def test_metadata_via_code_generation_lvgl(
@@ -159,10 +194,9 @@ def test_metadata_via_code_generation_lvgl(
"""Full code generation for lvgl.yaml should produce correct metadata."""
generate_main(component_fixture_path("lvgl.yaml"))
all_meta = get_all_display_metadata()
# lvgl.yaml: model ST7735 -> 128x160, full hw rotation
# lvgl.yaml: model ST7735 -> 128x160, no writer (lvgl draws directly), full hw rotation
assert len(all_meta) == 1
meta = next(iter(all_meta.values()))
assert meta.width == 128
assert meta.height == 160
assert meta.has_hardware_rotation is True
assert meta.byte_order == BYTE_ORDER_BIG
assert meta == DisplayMetaData(
width=128, height=160, has_writer=False, has_hardware_rotation=True
)
-4
View File
@@ -1,4 +0,0 @@
packages:
i2c: !include ../../test_build_components/common/i2c/host.yaml
<<: !include common.yaml
@@ -1,3 +0,0 @@
logger:
hardware_uart: UART0
baud_rate: 0
@@ -1 +0,0 @@
<<: !include common-uart0_no_logging.yaml
+2 -2
View File
@@ -13,13 +13,13 @@ i2s_audio:
speaker:
- platform: i2s_audio
id: mixer_output_speaker_id
id: speaker_id
dac_type: external
i2s_dout_pin: ${dout_pin}
bits_per_sample: 32bit
channel: stereo
- platform: mixer
output_speaker: mixer_output_speaker_id
output_speaker: speaker_id
bits_per_sample: 32
num_channels: 2
source_speakers:
-22
View File
@@ -10,28 +10,6 @@ display:
dimensions:
width: 450
height: 600
window_options:
position:
x: 100
y: 100
- platform: sdl
id: second_display
dimensions:
width: 450
height: 600
window_options:
position:
centered_on_display: 1
- platform: sdl
id: third_display
dimensions:
width: 450
height: 600
window_options:
position:
centered_on_display: 0
binary_sensor:
- platform: sdl
@@ -0,0 +1,21 @@
esphome:
name: scheduler-blocking-warning
host:
api:
logger:
level: DEBUG
# An interval fires via the scheduler (so the current component is the interval),
# defers through a delay, then busy-blocks well over the 50 ms warn threshold inside
# the deferred continuation. The blocking warning must be attributed to the interval
# component (captured at schedule time) instead of "<null>".
interval:
- interval: 500ms
then:
- delay: 10ms
- lambda: |-
const uint32_t start = millis();
// Spin for longer than WARN_IF_BLOCKING_OVER_MS (50 ms) to trip the warning.
while (millis() - start < 80) {
}
@@ -0,0 +1,32 @@
esphome:
name: scheduler-delay-failed
host:
api:
logger:
level: DEBUG
globals:
- id: started
type: bool
restore_value: false
initial_value: "false"
# The interval fires with itself as the current component, schedules a delay (which
# captures that component for log attribution), then marks itself failed. The delay
# continuation must still fire: a failed component must not drop an already-scheduled
# delay, because the SELF_POINTER scheduler item stores the component for attribution
# only.
interval:
- interval: 100ms
id: host_interval
then:
- if:
condition:
lambda: "return !id(started);"
then:
- lambda: |-
id(started) = true;
id(host_interval)->mark_failed();
- delay: 200ms
- logger.log: "DELAY_FIRED_AFTER_FAIL"
@@ -0,0 +1,91 @@
"""Integration test for blocking-warning source attribution.
A blocking operation that runs inside a deferred scheduler continuation (e.g. after
a ``delay`` in a script/automation) used to be reported as
``<null> took a long time for an operation (NN ms), max is 30 ms`` because the
continuation carries no component. The warning should instead name the component that
was current when the delay was scheduled and report the real threshold (50 ms).
"""
from __future__ import annotations
import asyncio
import re
import pytest
from .types import APIClientConnectedFactory, RunCompiledFunction
# Matches: "<source> took a long time for an operation (NN ms), max is NN ms"
WARN_PATTERN = re.compile(
r"took a long time for an operation \((\d+) ms\), max is (\d+) ms"
)
@pytest.mark.asyncio
async def test_scheduler_blocking_warning(
yaml_config: str,
run_compiled: RunCompiledFunction,
api_client_connected: APIClientConnectedFactory,
) -> None:
"""Deferred blocking work is attributed to a real component, not "<null>"."""
loop = asyncio.get_running_loop()
warning_future: asyncio.Future[str] = loop.create_future()
def check_output(line: str) -> None:
if WARN_PATTERN.search(line) and not warning_future.done():
warning_future.set_result(line)
async with (
run_compiled(yaml_config, line_callback=check_output),
api_client_connected() as client,
):
device_info = await client.device_info()
assert device_info is not None
# The interval fires, defers via delay, then busy-blocks > 50 ms in the
# continuation, which should trip the blocking warning.
warning_line = await asyncio.wait_for(warning_future, timeout=10.0)
# The deferred block must be attributed to a real component, not "<null>".
assert "<null>" not in warning_line, (
f"Warning should name a component, got: {warning_line}"
)
# The delay was scheduled from a known component (the interval), so the warning
# must name it rather than falling back to the generic scheduled-task label.
assert "a scheduled task" not in warning_line, (
f"Warning should name the interval component, got: {warning_line}"
)
# The reported threshold must be the real default (50 ms), not the stale "30 ms".
match = WARN_PATTERN.search(warning_line)
assert match is not None
assert match.group(2) == "50", f"Expected 'max is 50 ms', got: {warning_line}"
@pytest.mark.asyncio
async def test_scheduler_delay_runs_on_failed_component(
yaml_config: str,
run_compiled: RunCompiledFunction,
api_client_connected: APIClientConnectedFactory,
) -> None:
"""A delay must still fire even when its host component is marked failed.
DelayAction records the current component on its scheduler item purely for log
attribution. That component must not gate execution: the scheduler skips items
belonging to failed components, but SELF_POINTER items (delays) are exempt. This
guards the is_item_failed_() exception on both the heap and defer-queue paths.
"""
loop = asyncio.get_running_loop()
fired: asyncio.Future[bool] = loop.create_future()
def check_output(line: str) -> None:
if "DELAY_FIRED_AFTER_FAIL" in line and not fired.done():
fired.set_result(True)
async with (
run_compiled(yaml_config, line_callback=check_output),
api_client_connected() as client,
):
assert await client.device_info() is not None
# If the failed host component wrongly dropped the delay, this times out.
await asyncio.wait_for(fired, timeout=10.0)
+17 -83
View File
@@ -1426,15 +1426,7 @@ def test_detect_memory_impact_config_core_python_only_changes(tmp_path: Path) ->
@pytest.mark.usefixtures("mock_target_branch_dev")
def test_detect_memory_impact_config_no_common_platform(tmp_path: Path) -> None:
"""Test memory impact detection when components have no common platform.
The merged build runs with --base-only on a single platform, so components
without a base test on the selected platform cannot be built and must be
dropped. We build the largest subset that shares the selected platform
rather than handing the runner components it has nothing to compile for
(which previously produced "0 passed, 0 failed" and a failed memory
extraction).
"""
"""Test memory impact detection when components have no common platform."""
# Create test directory structure
tests_dir = tmp_path / "tests" / "components"
@@ -1461,70 +1453,12 @@ def test_detect_memory_impact_config_no_common_platform(tmp_path: Path) -> None:
result = determine_jobs.detect_memory_impact_config()
# No common platform: pick the most preferred platform among those supported
# (esp8266-ard outranks esp32-idf in the preference list) and build only the
# components that have a base test on it. wifi (esp32-idf only) is dropped.
# Should pick the most frequently supported platform
assert result["should_run"] == "true"
assert result["platform"] == "esp8266-ard"
assert result["components"] == ["logger"]
assert result["use_merged_config"] == "true"
def test_detect_memory_impact_config_variant_only_platform_excluded(
tmp_path: Path,
) -> None:
"""Regression test for the const + shelly_dimmer memory-impact failure.
Reproduces https://github.com/esphome/esphome/actions/runs/26746938473
where a platform hint selected esp32-idf even though neither changed
component had a base test.esp32-idf.yaml. The merged --base-only build then
found nothing to compile ("0 passed, 0 failed") and memory extraction
failed. Also covers a component whose only esp32-idf test is a *variant*
(test-*.esp32-idf.yaml): --base-only never compiles variants, so it must
not count toward platform availability.
"""
tests_dir = tmp_path / "tests" / "components"
# const: base test only on esp32-s3-idf
const_dir = tests_dir / "const"
const_dir.mkdir(parents=True)
(const_dir / "test.esp32-s3-idf.yaml").write_text("test: const")
# shelly_dimmer: base test only on esp8266-ard
shelly_dir = tests_dir / "shelly_dimmer"
shelly_dir.mkdir(parents=True)
(shelly_dir / "test.esp8266-ard.yaml").write_text("test: shelly_dimmer")
# mdns: only a VARIANT test on esp32-idf (no base test.esp32-idf.yaml).
# --base-only would never build it, so it must be excluded entirely.
mdns_dir = tests_dir / "mdns"
mdns_dir.mkdir(parents=True)
(mdns_dir / "test-min.esp32-idf.yaml").write_text("test: mdns")
with (
patch.object(determine_jobs, "root_path", str(tmp_path)),
patch.object(helpers, "root_path", str(tmp_path)),
patch.object(determine_jobs, "changed_files") as mock_changed_files,
):
# The "_esp32" filename yields an esp32-idf platform hint, reproducing
# the original bug where the hint picked a platform no component could
# build as a base test.
mock_changed_files.return_value = [
"esphome/components/const/const.cpp",
"esphome/components/shelly_dimmer/shelly_dimmer_esp32.cpp",
"esphome/components/mdns/mdns.cpp",
]
result = determine_jobs.detect_memory_impact_config()
# The esp32-idf hint is unbuildable (no base test), so we fall back to the
# platform supported by the most components, broken by preference order:
# esp8266-ard (shelly_dimmer) outranks esp32-s3-idf (const). Only the
# component with a base test on the selected platform is returned; the
# variant-only mdns is excluded entirely.
assert result["should_run"] == "true"
assert result["platform"] == "esp8266-ard"
assert result["components"] == ["shelly_dimmer"]
assert set(result["components"]) == {"wifi", "logger"}
# When no common platform, picks most commonly supported
# esp8266-ard is preferred over esp32-idf in the preference list
assert result["platform"] in ["esp32-idf", "esp8266-ard"]
assert result["use_merged_config"] == "true"
@@ -1611,16 +1545,12 @@ def test_detect_memory_impact_config_includes_base_bus_components(
@pytest.mark.usefixtures("mock_target_branch_dev")
def test_detect_memory_impact_config_variant_only_components_skipped(
tmp_path: Path,
) -> None:
"""Components with only variant tests are skipped for memory impact.
def test_detect_memory_impact_config_with_variant_tests(tmp_path: Path) -> None:
"""Test memory impact detection for components with only variant test files.
Components like improv_serial and ethernet only have variant test files
(test-*.yaml), no base test.<platform>.yaml. The memory-impact build runs
test_build_components.py with --base-only, which never compiles variants, so
these components have nothing buildable and must not be selected. Selecting
them previously produced "0 passed, 0 failed" and a failed memory extraction.
This verifies that memory impact analysis works correctly for components like
improv_serial, ethernet, mdns, etc. which only have variant test files
(test-*.yaml) instead of base test files (test.*.yaml).
"""
# Create test directory structure
tests_dir = tmp_path / "tests" / "components"
@@ -1651,8 +1581,12 @@ def test_detect_memory_impact_config_variant_only_components_skipped(
result = determine_jobs.detect_memory_impact_config()
# Neither component has a base test, so nothing is buildable under --base-only
assert result["should_run"] == "false"
# Should detect both components even though they only have variant tests
assert result["should_run"] == "true"
assert set(result["components"]) == {"improv_serial", "ethernet"}
# Both components support esp32-idf
assert result["platform"] == "esp32-idf"
assert result["use_merged_config"] == "true"
# Tests for clang-tidy split mode logic
@@ -1,20 +0,0 @@
esphome:
name: componenttestesp32h2idf
friendly_name: $component_name
esp32:
board: esp32-h2-devkitm-1
framework:
type: esp-idf
# Use custom partition table with larger app partition (3MB)
# Default IDF partitions only allow 1.75MB which is too small for grouped tests
partitions: ../partitions_testing.csv
logger:
level: VERY_VERBOSE
packages:
component_under_test: !include
file: $component_test_file
vars:
component_test_file: $component_test_file
@@ -1,7 +0,0 @@
# Common I2C configuration for host platform tests
i2c:
- id: i2c_bus
device: /dev/i2c-0
frequency: 100kHz
scan: true
+29 -3
View File
@@ -12,6 +12,13 @@ from esphome.build_gen import platformio
from esphome.core import CORE
@pytest.fixture
def mock_update_storage_json() -> Generator[MagicMock]:
"""Mock update_storage_json for all tests."""
with patch("esphome.build_gen.platformio.update_storage_json") as mock:
yield mock
@pytest.fixture
def mock_write_file_if_changed() -> Generator[MagicMock]:
"""Mock write_file_if_changed for tests."""
@@ -19,7 +26,9 @@ def mock_write_file_if_changed() -> Generator[MagicMock]:
yield mock
def test_write_ini_creates_new_file(tmp_path: Path) -> None:
def test_write_ini_creates_new_file(
tmp_path: Path, mock_update_storage_json: MagicMock
) -> None:
"""Test write_ini creates a new platformio.ini file."""
CORE.build_path = str(tmp_path)
@@ -41,7 +50,9 @@ framework = arduino
assert platformio.INI_AUTO_GENERATE_END in file_content
def test_write_ini_updates_existing_file(tmp_path: Path) -> None:
def test_write_ini_updates_existing_file(
tmp_path: Path, mock_update_storage_json: MagicMock
) -> None:
"""Test write_ini updates existing platformio.ini file."""
CORE.build_path = str(tmp_path)
@@ -86,7 +97,9 @@ framework = arduino
assert "platform = old" not in file_content
def test_write_ini_preserves_custom_sections(tmp_path: Path) -> None:
def test_write_ini_preserves_custom_sections(
tmp_path: Path, mock_update_storage_json: MagicMock
) -> None:
"""Test write_ini preserves custom sections outside auto-generate markers."""
CORE.build_path = str(tmp_path)
@@ -135,6 +148,7 @@ monitor_speed = 115200
def test_write_ini_no_change_when_content_same(
tmp_path: Path,
mock_update_storage_json: MagicMock,
mock_write_file_if_changed: MagicMock,
) -> None:
"""Test write_ini doesn't rewrite file when content is unchanged."""
@@ -160,3 +174,15 @@ def test_write_ini_no_change_when_content_same(
call_args = mock_write_file_if_changed.call_args[0]
assert call_args[0] == ini_file
assert content in call_args[1]
def test_write_ini_calls_update_storage_json(
tmp_path: Path, mock_update_storage_json: MagicMock
) -> None:
"""Test write_ini calls update_storage_json."""
CORE.build_path = str(tmp_path)
content = "[env:test]\nplatform = esp32"
platformio.write_ini(content)
mock_update_storage_json.assert_called_once()
+221 -405
View File
@@ -21,15 +21,13 @@ from esphome.espidf.component import (
URLSource,
_check_library_data,
_collect_filtered_files,
_node_key,
_normalize_dependencies,
_convert_library_to_component,
_parse_library_json,
_parse_library_properties,
_resolve_registry_version,
_process_dependencies,
_split_list_by_condition,
generate_cmakelists_txt,
generate_idf_component_yml,
generate_idf_components,
)
@@ -164,6 +162,43 @@ def test_generate_cmakelists_txt_references_project_managed_components_variable(
assert "${ESPHOME_PROJECT_MANAGED_COMPONENTS}" in content
def test_generate_idf_component_overwrites_bundled_files(
tmp_path: Path,
monkeypatch: pytest.MonkeyPatch,
esp32_idf_core: None,
) -> None:
# A library that ships its own CMakeLists.txt + idf_component.yml must
# have both replaced by ESPHome's generated content. Library authors'
# bundled IDF metadata is frequently broken (bogus REQUIRES, hard-coded
# frameworks), so we always regenerate from library.json.
from esphome.espidf.component import _generate_idf_component
(tmp_path / "src").mkdir()
(tmp_path / "src" / "main.cpp").write_text("// dummy\n")
(tmp_path / "library.json").write_text(json.dumps({"name": "tripwire-lib"}))
(tmp_path / "CMakeLists.txt").write_text("# TRIPWIRE_BUNDLED_CMAKELISTS\n")
(tmp_path / "idf_component.yml").write_text("# TRIPWIRE_BUNDLED_MANIFEST\n")
fake_component = IDFComponent(
"owner/tripwire-lib", "1.0.0", source=URLSource("http://dummy")
)
fake_component.path = tmp_path
monkeypatch.setattr(
esphome.espidf.component,
"_convert_library_to_component",
lambda _lib: fake_component,
)
monkeypatch.setattr(fake_component, "download", lambda force=False: None)
_generate_idf_component(Library("owner/tripwire-lib", "1.0.0", None))
cml = (tmp_path / "CMakeLists.txt").read_text()
manifest = (tmp_path / "idf_component.yml").read_text()
assert "TRIPWIRE_BUNDLED_CMAKELISTS" not in cml
assert "TRIPWIRE_BUNDLED_MANIFEST" not in manifest
assert "idf_component_register" in cml
def test_generate_idf_component_yml_basic(tmp_component):
tmp_component.data = {"description": "test", "repository": {"url": "http://aaa"}}
result = generate_idf_component_yml(tmp_component)
@@ -384,58 +419,200 @@ empty=
assert "empty" not in result
def test_node_key_git_with_ref():
key, is_git, locator = _node_key(
"name", None, "https://github.com/foo/bar.git#v1.2.3"
def test_convert_library_with_repository():
lib = Library("name", None, "https://github.com/foo/bar.git#v1.2.3")
result = _convert_library_to_component(lib)
assert result.name == "foo/bar"
assert result.version == "*"
assert isinstance(result.source, GitSource)
assert result.source.ref == "v1.2.3"
def test_convert_library_with_branch_ref():
lib = Library("name", None, "https://github.com/foo/bar.git#some-branch")
result = _convert_library_to_component(lib)
assert result.name == "foo/bar"
assert result.version == "*"
assert isinstance(result.source, GitSource)
assert result.source.ref == "some-branch"
def test_convert_library_missing_ref_uses_default_branch():
"""A bare URL with no #ref clones the remote's default branch.
Matches PIO's lib_deps behavior and external_components handling --
git.clone_or_update with ref=None leaves the depth-1 clone on
whatever branch the remote HEAD points at.
"""
lib = Library("name", None, "https://github.com/foo/bar.git")
result = _convert_library_to_component(lib)
assert result.name == "foo/bar"
assert result.version == "*"
assert isinstance(result.source, GitSource)
assert result.source.ref is None
def test_convert_library_registry(monkeypatch):
lib = Library("foo/bar", "^1.0.0", None)
monkeypatch.setattr(
esphome.espidf.component,
"_get_package_from_pio_registry",
lambda o, n, r: ("foo", "bar", "1.2.3", "http://example.com/pkg.zip"),
)
assert key == "foo/bar"
assert is_git is True
assert locator == ("https://github.com/foo/bar.git", "v1.2.3")
result = _convert_library_to_component(lib)
assert result.name == "foo/bar"
assert result.version == "1.2.3"
assert isinstance(result.source, URLSource)
def test_node_key_git_branch_ref():
key, is_git, locator = _node_key(
"name", None, "https://github.com/foo/bar.git#some-branch"
def test_process_dependencies_adds_valid_dependency(tmp_component, monkeypatch):
tmp_component.data = {
"dependencies": [
{
"name": "foo",
"version": "1.0",
}
]
}
monkeypatch.setattr(
esphome.espidf.component,
"_generate_idf_component",
lambda lib: esphome.espidf.component.IDFComponent(
lib.name, lib.version, source=URLSource("http://dummy.com")
),
)
assert (key, is_git, locator[1]) == ("foo/bar", True, "some-branch")
monkeypatch.setattr(esphome.espidf.component, "_check_library_data", lambda x: None)
_process_dependencies(tmp_component)
assert len(tmp_component.dependencies) == 1
def test_node_key_git_no_ref():
_key, is_git, locator = _node_key("name", None, "https://github.com/foo/bar.git")
assert is_git is True
assert locator == ("https://github.com/foo/bar.git", None)
def test_process_dependencies_skips_invalid(tmp_component):
tmp_component.data = {
"dependencies": [
{"name": "foo", "version": "1.0", "platforms": ["arduino"]},
{"invalid": "entry"},
]
}
_process_dependencies(tmp_component)
assert tmp_component.dependencies == []
def test_node_key_registry_owner_name():
key, is_git, locator = _node_key("foo/bar", "^1.0.0", None)
assert (key, is_git, locator) == ("foo/bar", False, ("foo", "bar"))
def test_process_dependencies_dict_form(tmp_component, monkeypatch):
"""PIO library.json shorthand ``{"owner/Name": "version"}`` is honored.
Iterating a dict gives string keys, which would silently fail the
``"name" in dependency`` substring check. Normalize to list-of-dicts
first so the dict form (used by e.g. tesla-ble for its nanopb dep)
is treated the same as the verbose list form.
"""
captured: list[Library] = []
def test_node_key_registry_bare_name():
key, is_git, locator = _node_key("bar", "1.0", None)
assert (key, is_git, locator) == ("bar", False, (None, "bar"))
def fake_generate(library):
captured.append(library)
return IDFComponent(
library.name, library.version, source=URLSource("http://dummy.com")
)
def test_normalize_dependencies_none():
assert _normalize_dependencies(None) == []
def test_normalize_dependencies_list_form():
deps = [{"name": "foo", "version": "1.0"}]
assert _normalize_dependencies(deps) == [{"name": "foo", "version": "1.0"}]
def test_normalize_dependencies_dict_form():
out = _normalize_dependencies({"nanopb/Nanopb": "^0.4.91", "BareName": "1.2.3"})
assert {"name": "Nanopb", "owner": "nanopb", "version": "^0.4.91"} in out
assert {"name": "BareName", "owner": None, "version": "1.2.3"} in out
def test_normalize_dependencies_dict_form_nested_spec():
out = _normalize_dependencies(
{"nanopb/Nanopb": {"version": "^0.4.91", "platforms": "espidf"}}
tmp_component.data = {
"dependencies": {
"nanopb/Nanopb": "^0.4.91",
"BareName": "1.2.3",
}
}
monkeypatch.setattr(
esphome.espidf.component, "_generate_idf_component", fake_generate
)
assert out == [
monkeypatch.setattr(esphome.espidf.component, "_check_library_data", lambda x: None)
_process_dependencies(tmp_component)
assert len(tmp_component.dependencies) == 2
names = sorted(lib.name for lib in captured)
versions = sorted(lib.version for lib in captured)
assert names == ["BareName", "nanopb/Nanopb"]
assert versions == ["1.2.3", "^0.4.91"]
def test_process_dependencies_dict_form_with_url_value(tmp_component, monkeypatch):
"""A dict-value that's a URL gets routed to ``repository`` like the list form."""
captured: list[Library] = []
def fake_generate(library):
captured.append(library)
return IDFComponent(library.name, "*", source=URLSource("http://dummy.com"))
tmp_component.data = {
"dependencies": {
"foo/Bar": "https://github.com/foo/bar.git#main",
}
}
monkeypatch.setattr(
esphome.espidf.component, "_generate_idf_component", fake_generate
)
monkeypatch.setattr(esphome.espidf.component, "_check_library_data", lambda x: None)
_process_dependencies(tmp_component)
assert len(captured) == 1
assert captured[0].name == "foo/Bar"
assert captured[0].version is None
assert captured[0].repository == "https://github.com/foo/bar.git#main"
def test_process_dependencies_dict_form_with_nested_spec(tmp_component, monkeypatch):
"""A dict-value that's itself a dict is merged into the entry.
PIO's library.json allows ``{"owner/Name": {"version": "...", ...}}``
for entries that need fields beyond just a version (platforms,
frameworks, etc.). The extra fields flow into _check_library_data
via the entry merge.
"""
captured: list[Library] = []
checked: list[dict] = []
def fake_generate(library):
captured.append(library)
return IDFComponent(
library.name, library.version, source=URLSource("http://dummy.com")
)
tmp_component.data = {
"dependencies": {
"nanopb/Nanopb": {"version": "^0.4.91", "platforms": "espidf"},
}
}
monkeypatch.setattr(
esphome.espidf.component, "_generate_idf_component", fake_generate
)
monkeypatch.setattr(
esphome.espidf.component,
"_check_library_data",
checked.append,
)
_process_dependencies(tmp_component)
assert len(captured) == 1
assert captured[0].name == "nanopb/Nanopb"
assert captured[0].version == "^0.4.91"
# Extra spec fields reach _check_library_data so platform/framework
# gating still applies.
assert checked == [
{
"name": "Nanopb",
"owner": "nanopb",
@@ -443,364 +620,3 @@ def test_normalize_dependencies_dict_form_nested_spec():
"platforms": "espidf",
}
]
def _patch_registry(monkeypatch, versions):
"""Patch the registry client to serve a canned version list (no network).
Only ``fetch_registry_package`` is faked; the real
``get_compatible_registry_versions`` / ``pick_best_registry_version`` run on
the canned data so the intersection logic is exercised for real.
"""
registry = esphome.espidf.component._make_registry_client()
monkeypatch.setattr(
registry,
"fetch_registry_package",
lambda spec: {
"owner": {"username": spec.owner or "owner"},
"name": spec.name,
"versions": [
{"name": v, "files": [{"download_url": f"http://x/{v}.tar.gz"}]}
for v in versions
],
},
)
monkeypatch.setattr(
esphome.espidf.component, "_make_registry_client", lambda: registry
)
def test_resolve_registry_version_intersects_constraints(monkeypatch):
_patch_registry(monkeypatch, ["1.10018.1", "1.10021.0", "1.10021.1"])
owner, name, version, url = _resolve_registry_version(
"esphome", "libsodium", {"==1.10021.0", "^1.10018.1"}
)
assert (owner, name, version) == ("esphome", "libsodium", "1.10021.0")
assert url == "http://x/1.10021.0.tar.gz"
def test_resolve_registry_version_picks_highest_satisfying(monkeypatch):
_patch_registry(monkeypatch, ["1.0.0", "1.5.0", "2.0.0"])
_owner, _name, version, _url = _resolve_registry_version("o", "p", {"^1.0.0"})
assert version == "1.5.0"
def test_resolve_registry_version_conflict_raises(monkeypatch):
_patch_registry(monkeypatch, ["1.0.0", "2.0.0"])
with pytest.raises(RuntimeError, match="satisfies all requirements"):
_resolve_registry_version("o", "p", {"==1.0.0", "==2.0.0"})
def test_generate_idf_components_dedupes_shared_dependency(
tmp_path: Path,
monkeypatch: pytest.MonkeyPatch,
esp32_idf_core: None,
) -> None:
# A and B both depend on shared C under different version specs. The batch
# must resolve C once with BOTH requirements collected, wire a single C
# instance into both, and regenerate (overwrite) each library's build files.
manifests = {
"esphome/A": {
"name": "A",
"dependencies": [
{"owner": "esphome", "name": "C", "version": "==1.10021.0"}
],
},
"esphome/B": {
"name": "B",
"dependencies": [
{"owner": "esphome", "name": "C", "version": "^1.10018.1"}
],
},
"esphome/C": {"name": "C"},
}
def fake_download(self, force=False):
self.path = tmp_path / self.get_sanitized_name().replace("/", "__")
(self.path / "src").mkdir(parents=True, exist_ok=True)
(self.path / "src" / "x.c").write_text("int x;")
(self.path / "library.json").write_text(json.dumps(manifests[self.name]))
(self.path / "CMakeLists.txt").write_text("# TRIPWIRE\n")
monkeypatch.setattr(IDFComponent, "download", fake_download)
captured: dict[str, set[str]] = {}
resolve_calls: list[str] = []
def fake_resolve(owner, pkgname, requirements):
resolve_calls.append(pkgname)
captured[f"{owner}/{pkgname}"] = set(requirements)
version = "1.10021.0" if pkgname == "C" else "1.0.0"
return owner, pkgname, version, f"http://x/{pkgname}.tar.gz"
monkeypatch.setattr(
esphome.espidf.component, "_resolve_registry_version", fake_resolve
)
top = generate_idf_components(
[Library("esphome/A", "1.0.0", None), Library("esphome/B", "1.0.0", None)]
)
# C resolved once (not once per consumer) with BOTH requirements gathered.
assert captured["esphome/C"] == {"==1.10021.0", "^1.10018.1"}
assert resolve_calls.count("C") == 1
# Top-level components returned in request order.
assert [c.name for c in top] == ["esphome/A", "esphome/B"]
# A and B reference the SAME single C instance (deduped).
a_dep = top[0].dependencies[0]
b_dep = top[1].dependencies[0]
assert a_dep.name == "esphome/C"
assert a_dep is b_dep
# The bundled CMakeLists was overwritten with generated content.
generated = (a_dep.path / "CMakeLists.txt").read_text()
assert "TRIPWIRE" not in generated
assert "idf_component_register" in generated
def test_generate_idf_components_handles_dependency_cycle(
tmp_path: Path,
monkeypatch: pytest.MonkeyPatch,
esp32_idf_core: None,
) -> None:
# A -> B -> A. Must terminate (not recurse forever) and wire the cycle with
# a single instance per component.
manifests = {
"esphome/A": {
"name": "A",
"dependencies": [{"owner": "esphome", "name": "B", "version": "1.0.0"}],
},
"esphome/B": {
"name": "B",
"dependencies": [{"owner": "esphome", "name": "A", "version": "1.0.0"}],
},
}
def fake_download(self, force=False):
self.path = tmp_path / self.get_sanitized_name().replace("/", "__")
(self.path / "src").mkdir(parents=True, exist_ok=True)
(self.path / "src" / "x.c").write_text("int x;")
(self.path / "library.json").write_text(json.dumps(manifests[self.name]))
monkeypatch.setattr(IDFComponent, "download", fake_download)
monkeypatch.setattr(
esphome.espidf.component,
"_resolve_registry_version",
lambda owner, pkgname, requirements: (
owner,
pkgname,
"1.0.0",
f"http://x/{pkgname}.tar.gz",
),
)
top = generate_idf_components([Library("esphome/A", "1.0.0", None)])
assert [c.name for c in top] == ["esphome/A"]
component_a = top[0]
component_b = component_a.dependencies[0]
assert component_b.name == "esphome/B"
# The cycle is wired back to the same A instance, not a duplicate.
assert component_b.dependencies[0] is component_a
def test_generate_idf_components_git_overrides_registry_warns(
tmp_path: Path,
monkeypatch: pytest.MonkeyPatch,
esp32_idf_core: None,
caplog: pytest.LogCaptureFixture,
) -> None:
# A pulls shared as a registry pin; B pulls the same component from a git
# source. The git source wins, but the dropped registry pin must be warned
# about (not silently discarded).
manifests = {
"esphome/A": {
"name": "A",
"dependencies": [
{"owner": "esphome", "name": "shared", "version": "==1.0.0"}
],
},
"esphome/B": {
"name": "B",
"dependencies": [
{
"owner": "esphome",
"name": "shared",
"version": "https://github.com/esphome/shared.git#main",
}
],
},
"esphome/shared": {"name": "shared"},
}
def fake_download(self, force=False):
self.path = tmp_path / self.get_sanitized_name().replace("/", "__")
(self.path / "src").mkdir(parents=True, exist_ok=True)
(self.path / "src" / "x.c").write_text("int x;")
(self.path / "library.json").write_text(json.dumps(manifests[self.name]))
monkeypatch.setattr(IDFComponent, "download", fake_download)
monkeypatch.setattr(
esphome.espidf.component,
"_resolve_registry_version",
lambda owner, pkgname, requirements: (
owner,
pkgname,
"1.0.0",
f"http://x/{pkgname}.tar.gz",
),
)
top = generate_idf_components(
[Library("esphome/A", "1.0.0", None), Library("esphome/B", "1.0.0", None)]
)
# shared resolved from the git source (version "*"), not the registry pin.
shared = top[0].dependencies[0]
assert shared.name == "esphome/shared"
assert isinstance(shared.source, GitSource)
assert "using the git source" in caplog.text
assert "==1.0.0" in caplog.text
def test_generate_idf_components_missing_manifest_raises(
tmp_path: Path,
monkeypatch: pytest.MonkeyPatch,
esp32_idf_core: None,
) -> None:
# A library with neither library.json nor library.properties is invalid;
# fail loudly rather than silently generating build files for it.
def fake_download(self, force=False):
self.path = tmp_path / self.get_sanitized_name().replace("/", "__")
(self.path / "src").mkdir(parents=True, exist_ok=True)
# no library.json / library.properties written
monkeypatch.setattr(IDFComponent, "download", fake_download)
monkeypatch.setattr(
esphome.espidf.component,
"_resolve_registry_version",
lambda owner, pkgname, requirements: (
owner,
pkgname,
"1.0.0",
f"http://x/{pkgname}.tar.gz",
),
)
with pytest.raises(RuntimeError, match="missing library.json"):
generate_idf_components([Library("esphome/A", "1.0.0", None)])
def test_generate_idf_components_warns_on_noncanonical_duplicate(
tmp_path: Path,
monkeypatch: pytest.MonkeyPatch,
esp32_idf_core: None,
caplog: pytest.LogCaptureFixture,
) -> None:
# A references "shared" (bare) and B references "owner/shared"; both resolve
# to the same canonical name but as distinct graph nodes, so they aren't
# deduplicated -- warn about it.
manifests = {
"esphome/A": {
"name": "A",
"dependencies": [{"name": "shared", "version": "1.0.0"}],
},
"esphome/B": {
"name": "B",
"dependencies": [{"owner": "owner", "name": "shared", "version": "1.0.0"}],
},
"owner/shared": {"name": "shared"},
}
def fake_download(self, force=False):
self.path = tmp_path / self.get_sanitized_name().replace("/", "__")
(self.path / "src").mkdir(parents=True, exist_ok=True)
(self.path / "src" / "x.c").write_text("int x;")
(self.path / "library.json").write_text(json.dumps(manifests[self.name]))
monkeypatch.setattr(IDFComponent, "download", fake_download)
# Bare "shared" and "owner/shared" both resolve to canonical owner/shared.
monkeypatch.setattr(
esphome.espidf.component,
"_resolve_registry_version",
lambda owner, pkgname, requirements: (
owner or "owner",
pkgname,
"1.0.0",
f"http://x/{pkgname}.tar.gz",
),
)
generate_idf_components(
[Library("esphome/A", "1.0.0", None), Library("esphome/B", "1.0.0", None)]
)
assert "referenced under multiple names" in caplog.text
def test_generate_idf_components_incompatible_top_level_raises(
tmp_path: Path,
monkeypatch: pytest.MonkeyPatch,
esp32_idf_core: None,
) -> None:
# A top-level library that isn't ESP-IDF/esp32 compatible must fail fast,
# not be silently dropped.
def fake_download(self, force=False):
self.path = tmp_path / self.get_sanitized_name().replace("/", "__")
(self.path / "src").mkdir(parents=True, exist_ok=True)
(self.path / "library.json").write_text(
json.dumps({"name": "A", "platforms": ["espressif8266"]})
)
monkeypatch.setattr(IDFComponent, "download", fake_download)
monkeypatch.setattr(
esphome.espidf.component,
"_resolve_registry_version",
lambda owner, pkgname, requirements: (
owner,
pkgname,
"1.0.0",
f"http://x/{pkgname}.tar.gz",
),
)
with pytest.raises(RuntimeError, match="not compatible with ESP-IDF"):
generate_idf_components([Library("esphome/A", "1.0.0", None)])
def test_generate_idf_components_incompatible_dependency_skipped(
tmp_path: Path,
monkeypatch: pytest.MonkeyPatch,
esp32_idf_core: None,
) -> None:
# An incompatible *transitive* dependency is skipped (not fatal): A is fine,
# its esp8266-only dep B is dropped and not wired.
manifests = {
"esphome/A": {
"name": "A",
"dependencies": [{"owner": "esphome", "name": "B", "version": "1.0.0"}],
},
"esphome/B": {"name": "B", "platforms": ["espressif8266"]},
}
def fake_download(self, force=False):
self.path = tmp_path / self.get_sanitized_name().replace("/", "__")
(self.path / "src").mkdir(parents=True, exist_ok=True)
(self.path / "library.json").write_text(json.dumps(manifests[self.name]))
monkeypatch.setattr(IDFComponent, "download", fake_download)
monkeypatch.setattr(
esphome.espidf.component,
"_resolve_registry_version",
lambda owner, pkgname, requirements: (
owner,
pkgname,
"1.0.0",
f"http://x/{pkgname}.tar.gz",
),
)
top = generate_idf_components([Library("esphome/A", "1.0.0", None)])
assert [c.name for c in top] == ["esphome/A"]
# The incompatible dependency was dropped, not wired in.
assert top[0].dependencies == []
@@ -148,12 +148,3 @@ def test_get_idedata_regenerates_on_corrupted_cache(setup_core: Path) -> None:
mock_transform.assert_called_once()
assert result == {"cxx_path": "regen"}
def test_get_core_framework_version_from_core_data():
"""The version is read from CORE.data when validation populated it."""
from esphome.components.esp32.const import KEY_ESP32, KEY_IDF_VERSION
import esphome.config_validation as cv
CORE.data = {KEY_ESP32: {KEY_IDF_VERSION: cv.Version(5, 5, 4)}}
assert toolchain._get_core_framework_version() == "5.5.4"
-82
View File
@@ -471,88 +471,6 @@ def test_command_config__show_secrets_skips_redaction(
assert "\\033[8m" not in output
def test_command_config__no_defaults_dumps_user_snapshot(
tmp_path: Path, capfd: CaptureFixture[str]
) -> None:
"""``--no-defaults`` dumps ``config.user_config`` instead of the
validated config, so schema defaults don't leak into the output."""
from esphome.config import Config
setup_core(tmp_path=tmp_path, config={"esphome": {"name": "test"}})
args = MockArgs()
args.show_secrets = True
args.no_defaults = True
validated = Config()
validated["esphome"] = {"name": "test", "build_path": "build/test"}
validated["wifi"] = {"ssid": "MyNet", "reboot_timeout": "15min"}
validated.user_config = {
"esphome": {"name": "test"},
"wifi": {"ssid": "MyNet"},
}
result = command_config(args, validated)
assert result == 0
output = capfd.readouterr().out
assert "ssid: MyNet" in output
# Defaults present on the validated config must not appear.
assert "reboot_timeout" not in output
assert "build_path" not in output
def test_command_config__no_defaults_warns_when_snapshot_missing(
tmp_path: Path,
capfd: CaptureFixture[str],
caplog: pytest.LogCaptureFixture,
) -> None:
"""If the snapshot is unavailable (e.g. a plain dict was passed in),
``--no-defaults`` logs a warning and falls back to the input config."""
setup_core(tmp_path=tmp_path, config={"esphome": {"name": "test"}})
args = MockArgs()
args.show_secrets = True
args.no_defaults = True
with caplog.at_level(logging.WARNING, logger="esphome.__main__"):
result = command_config(args, {"wifi": {"ssid": "MyNet"}})
assert result == 0
output = capfd.readouterr().out
assert "ssid: MyNet" in output
assert any(
"user-only config snapshot is unavailable" in rec.message
for rec in caplog.records
)
def test_command_config__no_defaults_skips_strip_default_ids(
tmp_path: Path, capfd: CaptureFixture[str]
) -> None:
"""When ``--no-defaults`` is set, ``strip_default_ids`` isn't run --
the user snapshot is already free of schema-injected IDs."""
from esphome.config import Config
setup_core(tmp_path=tmp_path, config={"esphome": {"name": "test"}})
args = MockArgs()
args.show_secrets = True
args.no_defaults = True
validated = Config()
validated["sensor"] = [{"name": "x", "id": "auto_generated"}]
validated.user_config = {"sensor": [{"name": "x"}]}
with patch(
"esphome.__main__.strip_default_ids", side_effect=AssertionError
) as mock_strip:
result = command_config(args, validated)
assert result == 0
mock_strip.assert_not_called()
output = capfd.readouterr().out
assert "name: x" in output
assert "auto_generated" not in output
def test_choose_upload_log_host_with_string_default() -> None:
"""Test with a single string default device."""
setup_core()
+2 -54
View File
@@ -8,7 +8,7 @@ from unittest.mock import MagicMock, Mock, patch
import pytest
from esphome import config_validation as cv, storage_json
from esphome import storage_json
from esphome.const import CONF_DISABLED, CONF_MDNS, Toolchain
from esphome.core import CORE
@@ -206,7 +206,6 @@ def test_storage_json_as_dict() -> None:
framework="arduino",
core_platform="esp32",
area="Living Room",
framework_version="5.3.1",
)
result = storage.as_dict()
@@ -236,7 +235,6 @@ def test_storage_json_as_dict() -> None:
assert result["framework"] == "arduino"
assert result["core_platform"] == "esp32"
assert result["area"] == "Living Room"
assert result["framework_version"] == "5.3.1"
def test_storage_json_to_json() -> None:
@@ -315,12 +313,8 @@ def test_storage_json_from_esphome_core(setup_core: Path) -> None:
mock_core.toolchain = Toolchain.ESP_IDF
mock_core.area = "Living Room"
with (
patch("esphome.components.esp32.get_esp32_variant") as mock_variant,
patch("esphome.components.esp32.idf_version") as mock_idf_version,
):
with patch("esphome.components.esp32.get_esp32_variant") as mock_variant:
mock_variant.return_value = "ESP32-C3"
mock_idf_version.return_value = cv.Version(5, 3, 1)
result = storage_json.StorageJSON.from_esphome_core(mock_core, old=None)
@@ -339,7 +333,6 @@ def test_storage_json_from_esphome_core(setup_core: Path) -> None:
assert result.core_platform == "esp32"
assert result.toolchain == "esp-idf"
assert result.area == "Living Room"
assert result.framework_version == "5.3.1"
def test_storage_json_from_esphome_core_mdns_enabled(setup_core: Path) -> None:
@@ -552,51 +545,6 @@ def test_storage_json_apply_to_core_ignores_unknown_toolchain(
assert CORE.toolchain is None
def test_storage_json_framework_version_round_trip(setup_core: Path) -> None:
"""Sidecar framework_version restores CORE.data[esp32][idf_version]."""
from esphome.components.esp32.const import KEY_ESP32, KEY_IDF_VERSION
storage = _make_storage_with_toolchain("esp-idf")
storage.framework_version = "5.3.1"
path = setup_core / "storage.json"
path.write_text(storage.to_json())
assert json.loads(path.read_text())["framework_version"] == "5.3.1"
loaded = storage_json.StorageJSON.load(path)
assert loaded is not None
assert loaded.framework_version == "5.3.1"
loaded.apply_to_core()
assert CORE.data[KEY_ESP32][KEY_IDF_VERSION] == cv.Version(5, 3, 1)
def test_storage_json_apply_to_core_without_framework_version(
setup_core: Path,
) -> None:
"""Older sidecars lacking framework_version don't populate idf_version."""
from esphome.components.esp32.const import KEY_ESP32, KEY_IDF_VERSION
loaded = _make_storage_with_toolchain("esp-idf")
assert loaded.framework_version is None
loaded.apply_to_core()
assert KEY_IDF_VERSION not in CORE.data[KEY_ESP32]
def test_storage_json_apply_to_core_raises_on_invalid_framework_version(
setup_core: Path,
) -> None:
"""A malformed version string fails with an actionable error at parse time."""
from esphome.core import EsphomeError
loaded = _make_storage_with_toolchain("esp-idf")
loaded.framework_version = "not-a-version"
with pytest.raises(EsphomeError, match="clean the build"):
loaded.apply_to_core()
def test_esphome_storage_json_as_dict() -> None:
"""Test EsphomeStorageJSON.as_dict returns correct dictionary."""
storage = storage_json.EsphomeStorageJSON(
-41
View File
@@ -361,47 +361,6 @@ def test_validate_config_without_command_line_substitutions_maintains_ordered_di
assert result[CONF_SUBSTITUTIONS]["var2"] == "value2"
def test_validate_config_captures_user_config_snapshot(tmp_path: Path) -> None:
"""validate_config stores a deep copy of the user's config -- with
substitutions re-added and no schema defaults applied -- on
``result.user_config`` for ``esphome config --no-defaults``.
"""
test_config = _get_test_minimal_valid_config(tmp_path)
result = config_module.validate_config(test_config, None)
# Snapshot is populated.
assert result.user_config is not None
# Substitutions are re-added and appear first.
assert list(result.user_config.keys())[0] == CONF_SUBSTITUTIONS
assert result.user_config[CONF_SUBSTITUTIONS]["var1"] == "value1"
# User-supplied keys are present without schema-default fields like
# ``build_path`` (which preload_core_config injects on the validated
# result's esphome section).
assert result.user_config["esphome"] == {"name": "test_device"}
assert "build_path" not in result.user_config["esphome"]
assert "min_version" not in result.user_config["esphome"]
assert result.user_config["esp32"] == {"board": "esp32dev"}
def test_validate_config_user_config_snapshot_is_deep_copy(tmp_path: Path) -> None:
"""The snapshot is independent of subsequent mutations to the result
config -- preload_core_config rewrites ``esphome:`` in place, but the
snapshot keeps the user's literal block.
"""
test_config = _get_test_minimal_valid_config(tmp_path)
result = config_module.validate_config(test_config, None)
assert result.user_config is not None
# preload_core_config injected build_path onto the validated config.
assert "build_path" in result["esphome"]
# The snapshot was taken before that and is unaffected.
assert "build_path" not in result.user_config["esphome"]
# And the two are not aliased.
assert result["esphome"] is not result.user_config["esphome"]
def test_merge_config_preserves_ordered_dict() -> None:
"""Test that merge_config preserves OrderedDict type.
+53 -102
View File
@@ -112,7 +112,6 @@ def create_storage() -> Callable[..., StorageJSON]:
framework=kwargs.get("framework", "arduino"),
core_platform=kwargs.get("core_platform", "esp32"),
toolchain=kwargs.get("toolchain", "platformio"),
framework_version=kwargs.get("framework_version"),
)
return _create
@@ -158,32 +157,6 @@ def test_storage_should_clean_when_toolchain_changes(
assert storage_should_clean(old, new) is True
def test_storage_should_clean_when_framework_changes(
create_storage: Callable[..., StorageJSON],
) -> None:
"""Test that clean is triggered when the framework changes.
Switching between arduino and esp-idf produces incompatible build trees
even on the same toolchain, so the build must be wiped.
"""
old = create_storage(loaded_integrations=["api", "wifi"], framework="arduino")
new = create_storage(loaded_integrations=["api", "wifi"], framework="esp-idf")
assert storage_should_clean(old, new) is True
def test_storage_should_clean_when_framework_version_changes(
create_storage: Callable[..., StorageJSON],
) -> None:
"""Test that clean is triggered when the framework version changes.
A different framework/ESP-IDF version compiles against a different SDK, so
the stale build tree must be wiped.
"""
old = create_storage(loaded_integrations=["api", "wifi"], framework_version="5.3.1")
new = create_storage(loaded_integrations=["api", "wifi"], framework_version="5.4.0")
assert storage_should_clean(old, new) is True
def test_storage_should_clean_when_component_removed(
create_storage: Callable[..., StorageJSON],
) -> None:
@@ -368,8 +341,8 @@ def test_update_storage_json_logging_when_old_is_none(
with caplog.at_level("INFO"):
update_storage_json()
# Verify clean_build was called with a full wipe (runs before src is written)
mock_clean_build.assert_called_once_with(clear_pio_cache=False, full=True)
# Verify clean_build was called
mock_clean_build.assert_called_once()
# Verify the correct log message was used (not the component removal message)
assert "Core config or version changed, cleaning build files..." in caplog.text
@@ -419,50 +392,60 @@ def test_update_storage_json_logging_components_removed(
new_storage.save.assert_called_once_with("/test/path")
def _mock_cmake_cache_paths(mock_core: MagicMock, tmp_path: Path) -> None:
"""Wire relative_pioenvs_path/relative_build_path to tmp_path subtrees."""
mock_core.name = "test_device"
mock_core.relative_pioenvs_path.side_effect = (tmp_path / ".pioenvs").joinpath
mock_core.relative_build_path.side_effect = tmp_path.joinpath
@patch("esphome.writer.CORE")
def test_clean_cmake_cache_platformio(
def test_clean_cmake_cache(
mock_core: MagicMock,
tmp_path: Path,
caplog: pytest.LogCaptureFixture,
) -> None:
"""Test clean_cmake_cache removes the PlatformIO CMakeCache.txt."""
_mock_cmake_cache_paths(mock_core, tmp_path)
cmake_cache_file = tmp_path / ".pioenvs" / "test_device" / "CMakeCache.txt"
cmake_cache_file.parent.mkdir(parents=True)
"""Test clean_cmake_cache removes CMakeCache.txt file."""
# Create directory structure
pioenvs_dir = tmp_path / ".pioenvs"
pioenvs_dir.mkdir()
device_dir = pioenvs_dir / "test_device"
device_dir.mkdir()
cmake_cache_file = device_dir / "CMakeCache.txt"
cmake_cache_file.write_text("# CMake cache file")
# Setup mocks
mock_core.relative_pioenvs_path.return_value = pioenvs_dir
mock_core.name = "test_device"
# Verify file exists before
assert cmake_cache_file.exists()
# Call the function
with caplog.at_level("INFO"):
clean_cmake_cache()
# Verify file was removed
assert not cmake_cache_file.exists()
# Verify logging
assert "Deleting" in caplog.text
assert "CMakeCache.txt" in caplog.text
@patch("esphome.writer.CORE")
def test_clean_cmake_cache_esp_idf(
def test_clean_cmake_cache_no_pioenvs_dir(
mock_core: MagicMock,
tmp_path: Path,
caplog: pytest.LogCaptureFixture,
) -> None:
"""Test clean_cmake_cache removes the native ESP-IDF build/CMakeCache.txt."""
_mock_cmake_cache_paths(mock_core, tmp_path)
cmake_cache_file = tmp_path / "build" / "CMakeCache.txt"
cmake_cache_file.parent.mkdir(parents=True)
cmake_cache_file.write_text("# CMake cache file")
"""Test clean_cmake_cache when pioenvs directory doesn't exist."""
# Setup non-existent directory path
pioenvs_dir = tmp_path / ".pioenvs"
with caplog.at_level("INFO"):
clean_cmake_cache()
# Setup mocks
mock_core.relative_pioenvs_path.return_value = pioenvs_dir
assert not cmake_cache_file.exists()
assert str(cmake_cache_file) in caplog.text
# Verify directory doesn't exist
assert not pioenvs_dir.exists()
# Call the function - should not crash
clean_cmake_cache()
# Verify directory still doesn't exist
assert not pioenvs_dir.exists()
@patch("esphome.writer.CORE")
@@ -470,11 +453,27 @@ def test_clean_cmake_cache_no_cmake_file(
mock_core: MagicMock,
tmp_path: Path,
) -> None:
"""Test clean_cmake_cache when no CMakeCache.txt exists -- should not crash."""
_mock_cmake_cache_paths(mock_core, tmp_path)
"""Test clean_cmake_cache when CMakeCache.txt doesn't exist."""
# Create directory structure without CMakeCache.txt
pioenvs_dir = tmp_path / ".pioenvs"
pioenvs_dir.mkdir()
device_dir = pioenvs_dir / "test_device"
device_dir.mkdir()
cmake_cache_file = device_dir / "CMakeCache.txt"
# Setup mocks
mock_core.relative_pioenvs_path.return_value = pioenvs_dir
mock_core.name = "test_device"
# Verify file doesn't exist
assert not cmake_cache_file.exists()
# Call the function - should not crash
clean_cmake_cache()
# Verify file still doesn't exist
assert not cmake_cache_file.exists()
@patch("esphome.writer.CORE")
def test_clean_build(
@@ -508,11 +507,6 @@ def test_clean_build(
managed_components_dir.mkdir()
(managed_components_dir / "espressif__arduino-esp32").mkdir()
# Converted-PIO-library cache (native ESP-IDF), under the data dir.
pio_components_dir = tmp_path / "pio_components"
pio_components_dir.mkdir()
(pio_components_dir / "abc12345").mkdir()
# Create PlatformIO cache directory
platformio_cache_dir = tmp_path / ".platformio" / ".cache"
platformio_cache_dir.mkdir(parents=True)
@@ -535,7 +529,6 @@ def test_clean_build(
assert idedata_cache.exists()
assert idf_build_dir.exists()
assert managed_components_dir.exists()
assert pio_components_dir.exists()
assert platformio_cache_dir.exists()
# Mock PlatformIO's ProjectConfig cache_dir
@@ -561,7 +554,6 @@ def test_clean_build(
assert not idedata_cache.exists()
assert not idf_build_dir.exists()
assert not managed_components_dir.exists()
assert not pio_components_dir.exists()
assert not platformio_cache_dir.exists()
# Verify logging
@@ -575,41 +567,6 @@ def test_clean_build(
assert "PlatformIO cache" in caplog.text
@patch("esphome.writer.CORE")
def test_clean_build_full_wipes_build_dir(
mock_core: MagicMock,
tmp_path: Path,
caplog: pytest.LogCaptureFixture,
) -> None:
"""full=True wipes the whole build dir (incl. src/) but keeps siblings."""
build_dir = tmp_path / "build" / "test"
(build_dir / "src").mkdir(parents=True)
(build_dir / "src" / "main.cpp").write_text("// generated")
(build_dir / "platformio.ini").write_text("[platformio]")
(build_dir / ".pioenvs").mkdir()
idedata_cache = tmp_path / "idedata" / "test.json"
idedata_cache.parent.mkdir()
idedata_cache.write_text("{}")
# A sibling of the build dir (under the data dir) must survive.
survivor = tmp_path / "keep_me.txt"
survivor.write_text("keep")
# build_path may be a str (e.g. set from config); clean_build must coerce.
mock_core.build_path = str(build_dir)
mock_core.name = "test"
mock_core.relative_internal_path.side_effect = tmp_path.joinpath
with caplog.at_level("INFO"):
clean_build(clear_pio_cache=False, full=True)
assert not build_dir.exists()
assert not idedata_cache.exists()
assert survivor.exists()
assert str(build_dir) in caplog.text
@patch("esphome.writer.CORE")
def test_clean_build_partial_exists(
mock_core: MagicMock,
@@ -629,7 +586,6 @@ def test_clean_build_partial_exists(
mock_core.relative_pioenvs_path.return_value = pioenvs_dir
mock_core.relative_piolibdeps_path.return_value = piolibdeps_dir
mock_core.relative_build_path.side_effect = lambda name: tmp_path / name
mock_core.relative_internal_path.side_effect = tmp_path.joinpath
# Verify only pioenvs exists
assert pioenvs_dir.exists()
@@ -667,7 +623,6 @@ def test_clean_build_nothing_exists(
mock_core.relative_pioenvs_path.return_value = pioenvs_dir
mock_core.relative_piolibdeps_path.return_value = piolibdeps_dir
mock_core.relative_build_path.side_effect = lambda name: tmp_path / name
mock_core.relative_internal_path.side_effect = tmp_path.joinpath
# Verify nothing exists
assert not pioenvs_dir.exists()
@@ -704,7 +659,6 @@ def test_clean_build_platformio_not_available(
mock_core.relative_pioenvs_path.return_value = pioenvs_dir
mock_core.relative_piolibdeps_path.return_value = piolibdeps_dir
mock_core.relative_build_path.side_effect = lambda name: tmp_path / name
mock_core.relative_internal_path.side_effect = tmp_path.joinpath
# Verify all exist before
assert pioenvs_dir.exists()
@@ -743,7 +697,6 @@ def test_clean_build_empty_cache_dir(
mock_core.relative_pioenvs_path.return_value = pioenvs_dir
mock_core.relative_piolibdeps_path.return_value = tmp_path / ".piolibdeps"
mock_core.relative_build_path.side_effect = lambda name: tmp_path / name
mock_core.relative_internal_path.side_effect = tmp_path.joinpath
# Verify pioenvs exists before
assert pioenvs_dir.exists()
@@ -1472,7 +1425,6 @@ def test_clean_build_handles_readonly_files(
mock_core.relative_pioenvs_path.return_value = pioenvs_dir
mock_core.relative_piolibdeps_path.return_value = tmp_path / ".piolibdeps"
mock_core.relative_build_path.side_effect = lambda name: tmp_path / name
mock_core.relative_internal_path.side_effect = tmp_path.joinpath
# Verify file is read-only
assert not os.access(readonly_file, os.W_OK)
@@ -1537,7 +1489,6 @@ def test_clean_build_reraises_for_other_errors(
mock_core.relative_pioenvs_path.return_value = pioenvs_dir
mock_core.relative_piolibdeps_path.return_value = tmp_path / ".piolibdeps"
mock_core.relative_build_path.side_effect = lambda name: tmp_path / name
mock_core.relative_internal_path.side_effect = tmp_path.joinpath
try:
# Mock os.access in writer module to return True (writable)