Commit Graph
27620 Commits
Author SHA1 Message Date
J. Nick Koston a036a9eab3 Merge branch 'decouple_scheduler_loop_cadence' into integration 2026-04-18 17:33:39 -05:00
J. Nick Koston 7d12b984a8 [core] address review polish on main-loop decoupling
Doc and test updates from a code review of this PR:

- Correct the `tail_us == 0 on Phase A-only ticks` claim in the
  Application::loop() comment and the RuntimeStatsCollector::record_loop_active
  docstring. `loop_tail_start_us` is set to `loop_before_end_us`, and
  `loop_now_us` is sampled later, so `tail_us` on Phase A-only ticks is
  the small gate-check + record prefix — tiny but non-zero.
  (Also flagged by Copilot on application.h:623 and runtime_stats.h:45.)

- Call out ESP8266 as the floor case in the WDT_FEED_INTERVAL_MS margin
  table. Its soft WDT (~1.6 s) is the tightest margin at ~5x, so future
  changes to the constant need to preserve comfortable headroom there.

- Tighten the test lower bound at tests/integration/test_loop_interval_decoupling.py
  from `2 <= loop_delta <= 6` to `3 <= loop_delta <= 6`. Allowing 2 would
  let a >50% slowdown from the 4-in-2s nominal pass as CI jitter, which
  undermines the regression signal. 3 keeps the test honest while still
  absorbing realistic CI jitter.

- Add a second integration test
  (test_loop_interval_default_not_pulled_forward) that covers the inverse
  direction: at the default loop_interval_ with a fast scheduler item
  (5 ms — well under the old delay_time/2 = 8 ms floor), the component
  phase must still run at ~62 Hz, not the pre-fix ~128 Hz. This locks
  down the original 128 Hz → 62 Hz regression that motivated the PR.
2026-04-18 17:32:29 -05:00
J. Nick Koston 45344c181a [core] decouple status_led dispatch from WDT feed rate limit
Raising WDT_FEED_INTERVAL_MS to 300 ms in the previous commit also capped
the status_led update cadence to ~3 Hz, because status_led::loop() was
re-dispatched only from inside feed_wdt_slow_(). That distorts the error
blink pattern (ERROR_PERIOD_MS = 250 ms with 150 ms on-window in
status_led.cpp) — at a 300 ms dispatch interval, the LED can be sampled
entirely inside or entirely outside the on-window on any given period,
turning a readable blink into an aliased one.

Split the two rate limits so they evolve independently:

  WDT_FEED_INTERVAL_MS            = 300 ms — arch_feed_wdt() rate limit
  STATUS_LED_DISPATCH_INTERVAL_MS = 100 ms — status_led loop() dispatch

The feed_wdt_with_time() hot path now has two independent gate checks
(a load + sub + branch each). Fast path on both misses is the common case
and remains cheap. feed_wdt_slow_() no longer touches status_led; the
status_led re-dispatch moves into a new service_status_led_slow_() that's
compiled in only when USE_STATUS_LED is set.

No change to behavior on devices without status_led. Devices with
status_led get the intended LED cadence restored (100 ms dispatch sits
below the 150 ms error on-window and well below the 250 ms warning on-
window).

Flagged by Copilot review on application.h:242.
2026-04-18 17:32:11 -05:00
J. Nick Koston d3603679d1 Merge branch 'decouple_scheduler_loop_cadence' into integration 2026-04-18 17:05:54 -05:00
J. Nick Koston f29da222fe [core] raise WDT_FEED_INTERVAL_MS from 3 to 300
The 3 ms rate limit was tight enough that the outer feed_wdt_with_time()
at the top of Application::loop() hit the slow path on nearly every
iteration. On-device measurements showed 95-99.9% of iterations
triggering esp_task_wdt_reset(), costing ~9-12 us/hit (BT proxies worse
due to WDT spinlock contention with the BT task on the other core).

Much worse under wake-storm workloads: with loop_interval_ raised for
power saving and an external stack (OpenThread in the reported case)
posting frequent wake notifications, the main loop can spin at thousands
of Hz doing almost nothing but feeding the watchdog — burning ~26% CPU
in the extreme case reported on #15792.

Raising the threshold to 300 ms:
  - Normal 62 Hz loop: feeds once every ~19 iterations (~3 Hz) instead
    of every iteration.
  - Wake-storm case: feeds ~3 Hz regardless of wake rate.
  - Any operation exceeding 300 ms still triggers a real feed right
    after it finishes (the post-component and post-scheduler-item
    feeds naturally clear the gate).

Safety margins vs platform watchdog timeouts remain large: 16x on ESP32
(5 s task WDT), 5x on ESP8266 soft WDT (1.6 s), 20x on ESP8266 HW WDT.
2026-04-18 17:01:17 -05:00
J. Nick Koston ba775b7a0a Merge remote-tracking branch 'upstream/dev' into integration 2026-04-18 16:23:21 -05:00
J. Nick Koston 38d894dfe7 [ld2412] Fix flaky integration test race condition (#15833) 2026-04-18 08:17:22 -05:00
J. Nick Koston 629a6b4a03 Merge remote-tracking branch 'upstream/esp8266-pin-esptoolpy-5x' into integration 2026-04-18 08:12:51 -05:00
J. Nick Koston fe06684242 [esp8266] Clarify 0.0.1 is pioarduino registry tag, not tool version 2026-04-18 08:12:28 -05:00
J. Nick KostonandCopilot Autofix powered by AI b293be23b0 [ci] Honor CONFLICTS_WITH when grouping component tests (#15834)
Co-authored-by: Copilot Autofix powered by AI <175728472+Copilot@users.noreply.github.com>
2026-04-18 08:11:38 -05:00
J. Nick Koston addda4c48f [esp8266] Mirror tool-esptoolpy pin in dev platformio.ini
Keep local pio run -e esp8266-arduino / esp32-arduino invocations
(clang-tidy, IDE) in sync with the ESPHome codegen override so they
share the same installed tool-esptoolpy package.
2026-04-18 08:10:29 -05:00
J. Nick Koston 63606dd4bb [esp8266] Pin tool-esptoolpy to 5.x to match ESP32 platform
The platform-espressif8266 4.2.1 platform manifest pins tool-esptoolpy
to ~1.30000.0, while the pioarduino ESP32 platform uses 5.2.0. Because
PlatformIO stores tool-esptoolpy in a single shared package directory,
switching between esp8266 and esp32 builds forces PlatformIO to
uninstall and reinstall the package every time.

Override the pin on the ESP8266 side to the same pioarduino 5.2.0 build
so both platforms resolve to the same installed package. ESPHome already
invokes its own Python esptool 5.2.0 for uploads, and modern esptool
still accepts the legacy underscore CLI flags used by the ESP8266
platform's builder scripts.
2026-04-18 08:09:03 -05:00
J. Nick Koston ccb53e34ca [core] Default PollingComponent() to 1ms when codegen is bypassed (#15831) 2026-04-18 09:04:51 -04:00
dependabot[bot] ec9d59f3dc Bump aioesphomeapi from 44.16.0 to 44.16.1 (#15836)
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2026-04-18 12:32:36 +00:00
J. Nick Koston 6abb9fd815 Merge remote-tracking branch 'upstream/decouple_scheduler_loop_cadence' into integration
# Conflicts:
#	esphome/core/application.h
2026-04-18 07:12:36 -05:00
J. Nick Koston 4e05d39f4b [core] use post-scheduler timestamp for gate check
scheduler_tick_ now returns the scheduler's advanced timestamp (free via
PR #15830's Scheduler::call return). Previously we still used the
pre-scheduler millis() for `elapsed = now - last_loop_`, which
underestimated elapsed time by whatever the scheduler dispatch took.

Adopt the returned value as `now` so the gate check, WDT feed, runtime
stats, and sleep computation all see consistent post-scheduler time.
Drops the obsolete "we deliberately reuse pre-scheduler now" comment —
that rationale was predicated on saving a millis() call, which no longer
applies.
2026-04-18 07:09:52 -05:00
J. Nick Koston 309c9463aa Merge remote-tracking branch 'upstream/dev' into decouple_scheduler_loop_cadence
# Conflicts:
#	esphome/core/application.h
2026-04-18 07:06:28 -05:00
J. Nick Koston df72aa26c0 [core] Feed WDT unconditionally in main loop to fix empty-config panic (#15830) 2026-04-18 11:58:54 +00:00
J. Nick Koston e347d65b64 Merge remote-tracking branch 'upstream/sensor/throttle-with-priority-nan-specialize' into integration 2026-04-18 06:32:18 -05:00
J. Nick Koston c122fc8c78 [ci] Apply conflict split before CI batch distribution
Previously split_conflicting_groups ran only inside test_build_components.
The CI-side batcher (split_components_for_ci.py / determine-jobs.py)
still saw the pre-split group, so its weight budgeting assumed one build
where runtime produces two. Apply the split where the groups are formed
so batch distribution reflects actual build counts.
2026-04-18 06:30:03 -05:00
J. Nick Koston 897dd7cd5e [ci] Move conflict-aware grouping helpers into script/helpers.py with tests 2026-04-18 06:26:03 -05:00
J. Nick Koston 698dbbdcb5 [ci] Simplify conflict walk 2026-04-18 06:21:43 -05:00
J. Nick Koston 875de948e8 [ci] Scope conflict parsing to batch components only 2026-04-18 06:19:35 -05:00
J. Nick Koston 9e714a3942 [ci] Clean up AST-target dispatch in _component_conflicts 2026-04-18 06:18:02 -05:00
J. Nick Koston fe00a5445e [ci] Honor CONFLICTS_WITH when grouping component tests
The test-grouping pipeline merged components that share a bus signature
into a single config without checking CONFLICTS_WITH. When
bme68x_bsec2 declared CONFLICTS_WITH=["bme680_bsec"] (and vice versa),
the merged YAML containing both failed validation.

Statically parse AUTO_LOAD and CONFLICTS_WITH from every component
__init__.py, propagate conflicts through AUTO_LOAD, and split any
group that contains a conflicting pair into separate builds.
2026-04-18 06:15:40 -05:00
J. Nick Koston 64c4eb12bc Merge remote-tracking branch 'upstream/wdt-feed-empty-loop' into integration 2026-04-18 05:29:16 -05:00
J. Nick Koston 327f03fe4f [core] Feed WDT unconditionally in main loop
Fixes the task watchdog firing on configs with no looping components and
no scheduler work (e.g. a minimal esphome: + logger: config, or a device
where every looping component has called disable_loop()).

In 2026.4.0, scheduler.call() feeds the WDT per executed item and each
component feeds it after its loop() runs. When neither fires on a tick,
the main loop task sleeps in yield_with_select_() with nothing ever
reaching arch_feed_wdt(), so the task watchdog starves and panics.

Add one feed_wdt_with_time() call right after before_loop_tasks_() in
Application::loop(). Rate-limited inline fast path, nearly free when
the 3 ms floor has not elapsed.

To keep the timestamp argument monotonic with last_wdt_feed_ (advanced
by Scheduler::execute_item_() as items fire), Scheduler::call() now
returns its internal `now` (advanced via
`now = this->execute_item_(item, now);`), forwarded through
before_loop_tasks_(). No extra millis() call needed; when no items run
the returned value equals the input.
2026-04-18 05:23:22 -05:00
Clyde Stubbs d3691c7ca5 [lvgl] Fix crash with snow on rotated display (#15822) 2026-04-18 09:17:28 +10:00
J. Nick Koston 0a794625ea Merge branch 'dev' into sensor/throttle-with-priority-nan-specialize 2026-04-17 17:54:32 -05:00
J. Nick Koston 562ce541a0 [bme680_bsec] [bme68x_bsec2] Mark the two BSEC variants as mutually exclusive (#15826) 2026-04-17 17:54:24 -05:00
J. Nick Koston 6ebe1e92eb [ci] Scope local pylint pre-commit hook to esphome/ (#15818) 2026-04-17 17:54:12 -05:00
J. Nick Koston 147e2fe9cf Merge remote-tracking branch 'origin/ci-scope-pylint' into integration 2026-04-17 17:46:26 -05:00
J. Nick Koston b54b0c24fc Merge remote-tracking branch 'origin/bme-bsec/conflicts-with' into integration 2026-04-17 17:46:23 -05:00
J. Nick Koston 0acf1873e7 Merge remote-tracking branch 'origin/sensor/throttle-with-priority-nan-specialize' into integration 2026-04-17 17:46:20 -05:00
J. Nick Koston fa2a75acd0 Merge remote-tracking branch 'origin/core-chunked-setup' into integration 2026-04-17 17:46:08 -05:00
J. Nick Koston 998337e3b1 [bme680_bsec] [bme68x_bsec2] Mark the two BSEC variants as mutually exclusive
\`bme680_bsec\` pulls in the BSEC v1 library (single-instance API:
\`bsec_init\`, \`bsec_set_configuration\`, …) and \`bme68x_bsec2\` pulls in
the BSEC v2 library (multi-instance API: \`bsec_init_m\`,
\`bsec_set_configuration_m\`, …). The two libraries export different
symbol sets and can't coexist in one firmware image — linking them
together currently fails with undefined references to the \`*_m\`
symbols. This manifests in CI when the test-component grouping logic
merges \`bme680_bsec\` and \`bme68x_bsec2_i2c\` into a single build.

Declare \`CONFLICTS_WITH\` on both components so the config validator
rejects the combination at config time with a clear message instead of
leaving users to decode a linker failure.
2026-04-17 16:50:01 -05:00
J. Nick Koston e26ce59797 for progmem 2026-04-17 16:18:39 -05:00
J. Nick Koston efe7bb1eb9 [sensor] Guard NaN specialization against explicit empty value list
Preserves the prior behavior of `value: []` (behaves like plain throttle,
no NaN bypass) by requiring a non-empty list before treating the config as
NaN-only — `all([])` is vacuously true otherwise.
2026-04-17 15:52:04 -05:00
Clyde Stubbs 1bf455cfbb [runtime_image] Fix RGB order (#15813) 2026-04-18 06:42:45 +10:00
Clyde Stubbs 290e213cd0 [mipi_spi] Add Sunton ESP32-2424S012 (#15812) 2026-04-18 06:41:33 +10:00
J. Nick Koston a5cba91729 [sensor] Specialize throttle_with_priority NaN-only case
Short form `- throttle_with_priority: 1s` and explicit `value: nan`
previously generated a `ThrottleWithPriorityFilter<1>` with a lambda
returning NAN stored in a `TemplatableFn<float>` array. Detect this
common case at codegen time and emit a new `ThrottleWithPriorityNanFilter`
that inlines the `std::isnan()` check. Behavior is preserved: NaN
sensor readings still bypass the throttle.
2026-04-17 15:39:30 -05:00
J. Nick Koston 9fa6d224c2 [core] Tighten docstrings and inline comments 2026-04-17 15:35:53 -05:00
J. Nick Koston 91b238aa97 [core] Fix grammar in ComponentMarker docstring 2026-04-17 15:34:15 -05:00
J. Nick Koston 00f08ba6ed [core] Drop per-component begin/end labels from generated main.cpp
The labels were there to help humans scanning the generated main.cpp
find component boundaries, but they were:

- Unreliable: CORE.flush_tasks can interleave coroutines on each
  await, so a component's later statements can land in another
  component's begin/end block.
- Load-bearing for a pile of complexity: a tuple return from
  _wrap_in_iifes, a has_iife flag, a comment-only detector to
  suppress trailing end-markers for comment-only components, and
  a brittle `"[]()" in line` check that could false-positive on
  YAML dumps containing lambda syntax.
- Not actually needed — generated main.cpp is a build artifact
  rarely read by anyone, and cg.LineComment("name:") already puts
  the component name at the start of its block.

ComponentMarker stays as a pure chunking sentinel — it tells
cpp_main_section where component boundaries are (for grouping) but
produces no C++ output. _wrap_in_iifes returns a plain list again.
Added a regression test for the now-defused case of a comment
containing "[]()" that was previously flagged by review.
2026-04-17 15:19:48 -05:00
Jonathan Swoboda b1b0005574 [esp32] Downgrade unneeded ignore_pin_validation_error to a warning (#15811) 2026-04-17 16:14:54 -04:00
J. Nick Koston f82401a504 [core] Address Copilot review: robust brace depth, accurate docstrings
- Count { and } characters per line instead of matching whole-line
  tokens. Current codegen only emits scope braces as standalone lines
  (from cg.with_local_variable()), but the defensive change is robust
  against future codegen emitting inline control flow like
  `if (cond) {` or `} else {` on one line.
- Add a regression test covering those inline-brace patterns.
- Fix stale docstrings on ComponentMarker and cpp_main_section that
  still claimed "stack frame released on return" and described the
  IIFEs as "noinline". The IIFEs have no noinline attribute and rely
  on scope-based lifetime shortening rather than guaranteed frames.
2026-04-17 15:06:42 -05:00
J. Nick Koston 178f23a7aa [core] Use begin/end marker pairs around each component's IIFE
Rename the bracket markers from "// === X ===" (same on both sides)
to "// === begin X ===" and "// === end X ===" so the generated
main.cpp reads unambiguously when scanning by component. Comment-only
components still get a single "begin X" marker since they have no
IIFE to close.
2026-04-17 15:06:42 -05:00
J. Nick Koston 864d31aa65 [core] Put ComponentMarker outside the IIFE as a visual bracket
The marker comment was being emitted as the first line *inside* each
IIFE:

  []() {
    // === logger ===
    // logger:
    //   ...
    ...
  }();

That works but buries the component label inside the lambda body, so
scanning generated main.cpp to find "where does component X's setup
live" is harder than it needs to be. Emit the marker before and after
the IIFE instead:

  // === logger ===
  []() {
    // logger:
    //   ...
    ...
  }();
  // === logger ===

Comment-only components (e.g. sha256, async_tcp, empty platforms like
binary_sensor:) don't grow a useless trailing duplicate marker —
when there's no IIFE to bracket, the marker is emitted once.
2026-04-17 15:06:42 -05:00
J. Nick Koston 936694af2c [core] Don't emit IIFE for comment-only chunks
Some components (sha256, async_tcp, network, empty text_sensor:, etc.)
emit only a ComponentMarker plus config-dump comments and no actual
C++ statements. Wrapping those in a `[]() { ... }();` IIFE is pure
clutter in the generated main.cpp — the IIFE has no body.

When _wrap_in_iifes sees a chunk whose lines are all // comments,
emit them verbatim instead of wrapping. Peak stack and flash are
unchanged on apollo and neargaragedoor since GCC was already
eliding the empty IIFEs; this just makes the generated code read
cleanly to humans.
2026-04-17 15:06:42 -05:00
J. Nick Koston 6a7c9af870 [core] Drop noinline from IIFE chunks and rename helper
Additional measurements showed GCC's -Os inliner re-inlines most IIFE
chunks back into setup() by choice, and the structural scoping alone
captures nearly all of the peak-stack benefit on esp32 without the
flash cost of forcing all chunks to stay as real functions.

Apollo (esp32-s3, -Os) with vs without noinline:
  peak setup stack     176 B (noinline)  vs  304 B (scope-only)
  flash delta         +388 B (noinline)  vs   -504 B (scope-only)
  chunks kept          86               vs    20

Issue #15796 is an LVGL-setup class of bug that has only surfaced on
esp32 after years in the field; the extra guarantee that noinline
provides is not worth the flash cost in practice. Also rename the
helper from _wrap_in_noinline_iifes to _wrap_in_iifes to match.
2026-04-17 15:06:42 -05:00