After the Phase A / Phase B split in this PR, an external producer that
called wake_loop_threadsafe() (MQTT RX, USB RX, BLE event, espnow,
camera, mWW, speakers, USB host/CDC, lwip socket, enable_loop_soon_any_context)
only got Phase A — the component phase stayed gated by loop_interval_,
so the producer's component loop() could be delayed by up to
loop_interval_ ms before draining its queued work. That breaks the
long-standing semantic of wake_loop_threadsafe().
Add a wake_request flag set by every wake_loop_* entry point and
exchange-cleared at the gate in Application::loop(). When the flag is
set, force Phase B regardless of loop_interval_.
Storage is conditional on the threading model:
- ESPHOME_THREAD_MULTI_ATOMICS: std::atomic<uint8_t> (uint8_t, not
bool, because GCC on Xtensa generates an indirect call for
atomic<bool> ops — same workaround as scheduler.h)
- ESPHOME_THREAD_SINGLE / ESPHOME_THREAD_MULTI_NO_ATOMICS: volatile
uint8_t (8-bit aligned loads/stores are atomic on every supported
MCU; the platform signal that follows wake_request_set provides the
cross-thread/cross-core memory barrier)
Helpers (wake_request_set / wake_request_take) are always_inline so
IRAM_ATTR call sites stay in IRAM. Set BEFORE the platform signal so the
consumer is guaranteed to see the flag on its next gate check.
Adds an integration test that raises loop_interval_ to 2s, snapshots a
counting component's loop count, spawns a std::thread that calls
App.wake_loop_threadsafe() after 50ms, and asserts the count increments
inside a 500ms observation window. Without the fix the count would not
move for ~2s.
- Add unit tests asserting cv.Invalid when `substitutions: !include list.yaml`
resolves to a non-mapping, covering both do_substitution_pass and
do_packages_pass.
- Note in resolve_substitutions_block that the resolve is single-shot and
chained top-level includes are not supported (matches _walk_packages for
`packages: !include`).
- Seed `resolve_include` context with `command_line_substitutions` so
`substitutions: !include ${var}.yaml` can reference CLI-provided vars
in the include filename (parallels the `packages: !include` path).
- Validate shape of resolved substitutions in `do_packages_pass` and raise
`cv.Invalid` under `CONF_SUBSTITUTIONS` instead of letting `UserDict()`
fail with a low-level exception on a non-mapping.
- Fixture 17 exercises the CLI-templated include filename.
Resolve a deferred IncludeFile before validating the substitutions shape in
do_substitution_pass, and before wrapping it in UserDict in do_packages_pass.
Fixesesphome/esphome#15848
Doc and test updates from a code review of this PR:
- Correct the `tail_us == 0 on Phase A-only ticks` claim in the
Application::loop() comment and the RuntimeStatsCollector::record_loop_active
docstring. `loop_tail_start_us` is set to `loop_before_end_us`, and
`loop_now_us` is sampled later, so `tail_us` on Phase A-only ticks is
the small gate-check + record prefix — tiny but non-zero.
(Also flagged by Copilot on application.h:623 and runtime_stats.h:45.)
- Call out ESP8266 as the floor case in the WDT_FEED_INTERVAL_MS margin
table. Its soft WDT (~1.6 s) is the tightest margin at ~5x, so future
changes to the constant need to preserve comfortable headroom there.
- Tighten the test lower bound at tests/integration/test_loop_interval_decoupling.py
from `2 <= loop_delta <= 6` to `3 <= loop_delta <= 6`. Allowing 2 would
let a >50% slowdown from the 4-in-2s nominal pass as CI jitter, which
undermines the regression signal. 3 keeps the test honest while still
absorbing realistic CI jitter.
- Add a second integration test
(test_loop_interval_default_not_pulled_forward) that covers the inverse
direction: at the default loop_interval_ with a fast scheduler item
(5 ms — well under the old delay_time/2 = 8 ms floor), the component
phase must still run at ~62 Hz, not the pre-fix ~128 Hz. This locks
down the original 128 Hz → 62 Hz regression that motivated the PR.
J. Nick Koston