Avoid per-file HTTP HEAD requests during config validation when running
esphome logs against a previously-cached project. The skip_external_update
flag was already plumbed for git operations, but external_files.download_content
ignored it. Thread it through CORE so audio_file, micro_wake_word,
speaker/media_player, image, font, and bme68x_bsec2 reuse cached files
without per-URL network round-trips when the file already exists locally.
- Add `--` terminator before submodule paths (both clone and refresh
paths) so a path beginning with `-` cannot be parsed as a git option.
- Reword the refresh-fetch comment: this fetch also runs when ref is
None, in which case it pulls the remote default branch.
Fixesesphome/esphome#11550
When clone_or_update is called with a ref or submodules, the initial
clone uses --depth=1 but the subsequent git fetch and git submodule
update commands run deep, pulling the full history on every refresh.
This causes excess network traffic and disk usage for external_components,
packages, and dashboard imports.
Add --depth=1 to all four call sites (initial fetch, initial submodule
init, refresh fetch, refresh submodule update). Shallow fetches still
advance the local clone to the current remote tip when it moves, so the
refresh path keeps working as before.
- Add unit tests asserting cv.Invalid when `substitutions: !include list.yaml`
resolves to a non-mapping, covering both do_substitution_pass and
do_packages_pass.
- Note in resolve_substitutions_block that the resolve is single-shot and
chained top-level includes are not supported (matches _walk_packages for
`packages: !include`).
- Seed `resolve_include` context with `command_line_substitutions` so
`substitutions: !include ${var}.yaml` can reference CLI-provided vars
in the include filename (parallels the `packages: !include` path).
- Validate shape of resolved substitutions in `do_packages_pass` and raise
`cv.Invalid` under `CONF_SUBSTITUTIONS` instead of letting `UserDict()`
fail with a low-level exception on a non-mapping.
- Fixture 17 exercises the CLI-templated include filename.
Resolve a deferred IncludeFile before validating the substitutions shape in
do_substitution_pass, and before wrapping it in UserDict in do_packages_pass.
Fixesesphome/esphome#15848
The labels were there to help humans scanning the generated main.cpp
find component boundaries, but they were:
- Unreliable: CORE.flush_tasks can interleave coroutines on each
await, so a component's later statements can land in another
component's begin/end block.
- Load-bearing for a pile of complexity: a tuple return from
_wrap_in_iifes, a has_iife flag, a comment-only detector to
suppress trailing end-markers for comment-only components, and
a brittle `"[]()" in line` check that could false-positive on
YAML dumps containing lambda syntax.
- Not actually needed — generated main.cpp is a build artifact
rarely read by anyone, and cg.LineComment("name:") already puts
the component name at the start of its block.
ComponentMarker stays as a pure chunking sentinel — it tells
cpp_main_section where component boundaries are (for grouping) but
produces no C++ output. _wrap_in_iifes returns a plain list again.
Added a regression test for the now-defused case of a comment
containing "[]()" that was previously flagged by review.
- Count { and } characters per line instead of matching whole-line
tokens. Current codegen only emits scope braces as standalone lines
(from cg.with_local_variable()), but the defensive change is robust
against future codegen emitting inline control flow like
`if (cond) {` or `} else {` on one line.
- Add a regression test covering those inline-brace patterns.
- Fix stale docstrings on ComponentMarker and cpp_main_section that
still claimed "stack frame released on return" and described the
IIFEs as "noinline". The IIFEs have no noinline attribute and rely
on scope-based lifetime shortening rather than guaranteed frames.
Rename the bracket markers from "// === X ===" (same on both sides)
to "// === begin X ===" and "// === end X ===" so the generated
main.cpp reads unambiguously when scanning by component. Comment-only
components still get a single "begin X" marker since they have no
IIFE to close.
The marker comment was being emitted as the first line *inside* each
IIFE:
[]() {
// === logger ===
// logger:
// ...
...
}();
That works but buries the component label inside the lambda body, so
scanning generated main.cpp to find "where does component X's setup
live" is harder than it needs to be. Emit the marker before and after
the IIFE instead:
// === logger ===
[]() {
// logger:
// ...
...
}();
// === logger ===
Comment-only components (e.g. sha256, async_tcp, empty platforms like
binary_sensor:) don't grow a useless trailing duplicate marker —
when there's no IIFE to bracket, the marker is emitted once.
Some components (sha256, async_tcp, network, empty text_sensor:, etc.)
emit only a ComponentMarker plus config-dump comments and no actual
C++ statements. Wrapping those in a `[]() { ... }();` IIFE is pure
clutter in the generated main.cpp — the IIFE has no body.
When _wrap_in_iifes sees a chunk whose lines are all // comments,
emit them verbatim instead of wrapping. Peak stack and flash are
unchanged on apollo and neargaragedoor since GCC was already
eliding the empty IIFEs; this just makes the generated code read
cleanly to humans.
Additional measurements showed GCC's -Os inliner re-inlines most IIFE
chunks back into setup() by choice, and the structural scoping alone
captures nearly all of the peak-stack benefit on esp32 without the
flash cost of forcing all chunks to stay as real functions.
Apollo (esp32-s3, -Os) with vs without noinline:
peak setup stack 176 B (noinline) vs 304 B (scope-only)
flash delta +388 B (noinline) vs -504 B (scope-only)
chunks kept 86 vs 20
Issue #15796 is an LVGL-setup class of bug that has only surfaced on
esp32 after years in the field; the extra guarantee that noinline
provides is not worth the flash cost in practice. Also rename the
helper from _wrap_in_noinline_iifes to _wrap_in_iifes to match.
The C++ standard-attribute spelling [[gnu::noinline]] placed between a
lambda's parameter list and body binds to the return type, not the
call operator. GCC 14 silently ignores it and emits -Wattributes
warnings at every chunk site. Switch to GCC's __attribute__((...))
syntax which binds to operator() as intended.
Measured impact on apollo-r-pro-1-eth (esp32-s3, -Os) vs the broken
[[gnu::noinline]] version: setup() frame 160 B -> 32 B, peak stack
304 B -> 176 B (another -42%). Flash grows by 888 B because all 86
chunks now stay as separate functions instead of GCC inlining the
small ones (which it was free to do when the attribute was ignored).
Net vs baseline -Os: peak stack 1264 B -> 176 B (-86%); flash
+388 B (<0.05% of a typical esp32 partition).
Generated setup() is a single monolithic function whose stack frame
scales super-linearly with config size. On a 5,943-line apollo build
the frame reached 1,264 B at -Os; extrapolation onto larger configs
(e.g. the 16k-line LVGL config in #15796) plausibly overflows the
8 KB loop task stack before safe_mode can increment its boot counter.
Emit a ComponentMarker sentinel at the start of each component's
to_code output, then have cpp_main_section wrap each component's
block (and sub-splits of up to 50 statements within each block) in a
noinline IIFE lambda. Each lambda's ENTRY frame is released on
return, bounding peak stack to setup() frame + max chunk frame.
Measured on apollo-r-pro-1-eth (esp32-s3, -Os):
setup() frame 1264 B -> 160 B
max chunk frame n/a -> 144 B
peak setup stack 1264 B -> 304 B (-76%)
total flash 792,471 B -> 791,995 B (-476 B)
The brace-depth guard in _wrap_in_noinline_iifes ensures we never
split between the RawStatement("{") / RawStatement("}") pair emitted
by cg.with_local_variable() (currently only wifi), so scoped locals
stay intact.
PlatformIO's idedata may list flash images that do not exist on disk
(e.g. a tasmota tinyuf2.bin referenced by the adafruit_qtpy_esp32s3_n4r2
board). Previously the CLI passed every entry straight to esptool, which
aborted the entire flash with "No such file or directory". The dashboard
path is unaffected because it flashes the pre-merged firmware.factory.bin
produced by the post-build step, which already tolerates missing inputs.
Filter non-existent extra_flash_images with a warning so a stale or
incorrect platform-declared image no longer breaks esphome run.
Fixes https://github.com/esphome/esphome/issues/15634
J. Nick Koston