Commit Graph

181 Commits

Author SHA1 Message Date
J. Nick Koston 78aabf257f [core] wake_loop_threadsafe() forces a component-phase iteration
After the Phase A / Phase B split in this PR, an external producer that
called wake_loop_threadsafe() (MQTT RX, USB RX, BLE event, espnow,
camera, mWW, speakers, USB host/CDC, lwip socket, enable_loop_soon_any_context)
only got Phase A — the component phase stayed gated by loop_interval_,
so the producer's component loop() could be delayed by up to
loop_interval_ ms before draining its queued work. That breaks the
long-standing semantic of wake_loop_threadsafe().

Add a wake_request flag set by every wake_loop_* entry point and
exchange-cleared at the gate in Application::loop(). When the flag is
set, force Phase B regardless of loop_interval_.

Storage is conditional on the threading model:
  - ESPHOME_THREAD_MULTI_ATOMICS: std::atomic<uint8_t> (uint8_t, not
    bool, because GCC on Xtensa generates an indirect call for
    atomic<bool> ops — same workaround as scheduler.h)
  - ESPHOME_THREAD_SINGLE / ESPHOME_THREAD_MULTI_NO_ATOMICS: volatile
    uint8_t (8-bit aligned loads/stores are atomic on every supported
    MCU; the platform signal that follows wake_request_set provides the
    cross-thread/cross-core memory barrier)

Helpers (wake_request_set / wake_request_take) are always_inline so
IRAM_ATTR call sites stay in IRAM. Set BEFORE the platform signal so the
consumer is guaranteed to see the flag on its next gate check.

Adds an integration test that raises loop_interval_ to 2s, snapshots a
counting component's loop count, spawns a std::thread that calls
App.wake_loop_threadsafe() after 50ms, and asserts the count increments
inside a 500ms observation window. Without the fix the count would not
move for ~2s.
2026-04-20 11:52:56 +02:00
J. Nick Koston 93e99efc2d Merge branch 'dev' into decouple_scheduler_loop_cadence 2026-04-18 17:38:24 -05:00
J. Nick Koston 7d12b984a8 [core] address review polish on main-loop decoupling
Doc and test updates from a code review of this PR:

- Correct the `tail_us == 0 on Phase A-only ticks` claim in the
  Application::loop() comment and the RuntimeStatsCollector::record_loop_active
  docstring. `loop_tail_start_us` is set to `loop_before_end_us`, and
  `loop_now_us` is sampled later, so `tail_us` on Phase A-only ticks is
  the small gate-check + record prefix — tiny but non-zero.
  (Also flagged by Copilot on application.h:623 and runtime_stats.h:45.)

- Call out ESP8266 as the floor case in the WDT_FEED_INTERVAL_MS margin
  table. Its soft WDT (~1.6 s) is the tightest margin at ~5x, so future
  changes to the constant need to preserve comfortable headroom there.

- Tighten the test lower bound at tests/integration/test_loop_interval_decoupling.py
  from `2 <= loop_delta <= 6` to `3 <= loop_delta <= 6`. Allowing 2 would
  let a >50% slowdown from the 4-in-2s nominal pass as CI jitter, which
  undermines the regression signal. 3 keeps the test honest while still
  absorbing realistic CI jitter.

- Add a second integration test
  (test_loop_interval_default_not_pulled_forward) that covers the inverse
  direction: at the default loop_interval_ with a fast scheduler item
  (5 ms — well under the old delay_time/2 = 8 ms floor), the component
  phase must still run at ~62 Hz, not the pre-fix ~128 Hz. This locks
  down the original 128 Hz → 62 Hz regression that motivated the PR.
2026-04-18 17:32:29 -05:00
J. Nick Koston 38d894dfe7 [ld2412] Fix flaky integration test race condition (#15833) 2026-04-18 08:17:22 -05:00
J. Nick Koston 43d98cb445 Merge remote-tracking branch 'upstream/dev' into decouple_scheduler_loop_cadence
# Conflicts:
#	esphome/core/scheduler.cpp
2026-04-17 09:33:59 -05:00
J. Nick Koston 523c6f2376 [core] coerce set_interval(0) / update_interval: 0ms to 1ms (#15799) 2026-04-17 02:45:50 -10:00
J. Nick Koston a0ac226e6c [core] decouple main loop cadence from scheduler wake timing
Restructures Application::loop() into two independent phases to stop the
scheduler from silently pulling the component loop cadence forward.

Before: Application::loop() bounded its sleep by
  min(loop_interval_ - elapsed, next_schedule_in())
with a delay_time/2 floor. Any scheduler item due sooner than
loop_interval_/2 dragged the whole component phase with it. On a typical
ESP32 config with default loop_interval_=16ms, combined scheduler
activity from api / esp32_ble / esp32_ble_tracker / debug was keeping
every component's loop() running at ~128 Hz instead of the documented
~62 Hz.

This has become more visible recently as more components convert to
PollingComponent (which uses set_interval internally) and more in-tree
code uses set_interval / set_timeout directly. Adding or removing any
scheduled item silently changed every other component's loop cadence.
App.set_loop_interval() for power savings was also silently defeated.

After:
- Phase A (every tick): drain wake notifications, run scheduler.call(),
  feed WDT
- Phase B (gated by loop_interval_ or HighFrequencyLoopRequester):
  iterate registered components and update last_loop_

Sleep = min(time-until-next-component-phase, next_schedule_in()). When
a scheduler event wakes us early, Phase A services it and the component
phase stays gated independently. loop_interval_ is now a true minimum
interval between component phases.

The delay_time/2 floor is removed. Any legitimate need to wake faster
than loop_interval_ has proper mechanisms:
- HighFrequencyLoopRequester for sustained fast-loop needs
- Application::wake_loop_threadsafe() from any context (new in 2026.4.0)
  for one-shot wake-on-event

Also guards against set_interval(0) misuse — it asks the main loop to
spin forever, which was never the intended API. Warns at creation time
pointing authors at HighFrequencyLoopRequester. set_timeout(0)/defer()
is unaffected; zero-delay one-shots remain legitimate.

Runtime stats: process_pending_stats is now called on every tick (not
just when the component phase runs) so log_interval_ isn't quantized to
the component-phase cadence. Added an inline fast-path gate in
runtime_stats.h that early-outs unless now >= next_log_time_, keeping
Application::loop() slim; the log_stats_ work stays out-of-line.

Ordering constraints preserved:
- defer() callbacks still FIFO before components same-tick (Phase A
  runs before Phase B)
- Scheduled items still execute before components when both due
- Scheduled callbacks still run on main thread only
- loop_component_start_time_ is still set fresh at each component's loop
- WDT is still fed at least once per tick
2026-04-16 14:27:39 -10:00
J. Nick Koston b232fc91ab [runtime_stats] Track main loop active time and report overhead (#15743) 2026-04-16 14:07:26 -10:00
J. Nick Koston e48c7165c5 [light] Avoid addressable transition stall at low gamma-corrected values (#15726) 2026-04-15 07:45:42 +12:00
J. Nick Koston da9fbb8044 [core] Fix app_state_ status bits clobbered for non-looping components (#15658) 2026-04-14 07:50:11 -10:00
J. Nick Koston 8e02d0a20e [fan] Store preset mode vector on Fan entity to eliminate heap allocation (#15209) 2026-04-09 10:25:37 +12:00
J. Nick Koston faa05031a7 [climate] Store custom mode vectors on Climate entity to eliminate heap allocation (#15206) 2026-04-09 10:25:29 +12:00
J. Nick Koston 5d31f4aeba [light] Use function-pointer fields in LightControlAction (#15132) 2026-04-07 12:00:17 -10:00
J. Nick Koston 674d030cbb [core] Reschedule fired intervals directly into heap (#15516) 2026-04-07 07:36:55 -10:00
Bonne Eggleston c6bb1fe141 [modbus] Add integration tests for server and server via controller (#14845)
Co-authored-by: J. Nick Koston <nick@home-assistant.io>
2026-04-03 20:24:02 +00:00
J. Nick Koston 9b97e95cf3 [binary_sensor] Add on_multi_click integration test (#15329) 2026-03-31 07:42:12 -10:00
J. Nick Koston 584807b039 [ld2410] Fix flaky integration test race condition (#15299) 2026-03-29 11:58:03 -10:00
J. Nick Koston f5cd1e5e76 [ld2450] Fix flaky integration test race condition (#15226) 2026-03-27 08:23:26 -10:00
J. Nick Koston 9fb5b6aa15 [light] Replace initial_state storage with flash-resident callback (#15133) 2026-03-24 14:03:18 -10:00
J. Nick Koston 5560c9eef7 [test] Fix flakey ld2412 integration test race condition (#15100) 2026-03-22 21:10:51 -10:00
Keith Burzinski 225330413a [uart] Rename FlushResult to UARTFlushResult with UART_FLUSH_RESULT_ prefix (#15101)
Co-authored-by: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-03-23 01:55:14 -05:00
J. Nick Koston 4d09eb2cec [tests] Fix flaky ld24xx integration tests by disabling API batching (#15050) 2026-03-22 12:29:28 -10:00
J. Nick Koston 5e68282519 [light] Fix constant_brightness broken by gamma LUT refactor (#15048)
Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>
2026-03-23 09:14:52 +13:00
Bonne Eggleston 92d5e7b18c [tests] Fix integration helper to match entities exactly (#14837)
Co-authored-by: J. Nick Koston <nick@home-assistant.io>
2026-03-15 13:02:23 -10:00
J. Nick Koston bd844fcd0a [template] Fix misleading 'Text value too long to save' warning (#14753) 2026-03-13 07:37:44 -10:00
Kjell Braden 326769e43c [runtime_image] fix BMP parsing (#14762) 2026-03-13 09:18:42 -04:00
J. Nick Koston 89719cf4b2 [water_heater] Set OPERATION_MODE feature flag when modes are configured (#14748) 2026-03-12 14:48:41 -10:00
J. Nick Koston 4d2ef09a29 [log] Detect early log calls before logger init and optimize hot path (#14538) 2026-03-10 09:12:10 -10:00
J. Nick Koston 9dd3ec258c [scheduler] Replace unique_ptr with raw pointers, add leak detection (#14620) 2026-03-10 09:11:28 -10:00
Jonathan Swoboda c31ac662bd [multiple] Fix crashes from malformed external input (#14643)
Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com>
Co-authored-by: J. Nick Koston <nick@home-assistant.io>
2026-03-09 20:39:58 -04:00
J. Nick Koston aef2d74e41 [ld2450] Add integration tests with mock UART (#14611) 2026-03-08 14:32:59 -10:00
J. Nick Koston 88536ff72b [modbus] Fix timeout for non-hardware UARTs (e.g., USB UART) (#14614)
Co-authored-by: Clyde Stubbs <2366188+clydebarrow@users.noreply.github.com>
2026-03-08 14:31:42 -10:00
Keith Burzinski 5e842a8b20 [uart] Return flush result, expose timeout via config (#14608)
Co-authored-by: J. Nick Koston <nick@koston.org>
Co-authored-by: pre-commit-ci-lite[bot] <117423508+pre-commit-ci-lite[bot]@users.noreply.github.com>
2026-03-08 05:23:13 +00:00
J. Nick Koston 888f3d804b [ld2420] Add integration tests with mock UART (#14471) 2026-03-07 13:22:50 -10:00
Bonne Eggleston b0be02e16d [modbus] Fix timing bugs and better adhere to spec (#8032)
Co-authored-by: brambo123 <52667932+brambo123@users.noreply.github.com>
Co-authored-by: Keith Burzinski <kbx81x@gmail.com>
Co-authored-by: J. Nick Koston <nick@koston.org>
Co-authored-by: J. Nick Koston <nick+github@koston.org>
Co-authored-by: J. Nick Koston <nick@home-assistant.io>
2026-03-05 20:54:17 +00:00
J. Nick Koston 5df4fd0a27 [tests] Fix flaky uart_mock integration tests (#14476) 2026-03-04 15:51:51 -10:00
J. Nick Koston b2e8544c58 [ld2412] Add integration tests with mock UART (#14448) 2026-03-04 07:18:31 -10:00
Jonathan Swoboda ee78d7a0c0 [tests] Fix integration test race condition in PlatformIO cache init (#14435)
Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com>
2026-03-03 17:42:41 -05:00
Clyde Stubbs cfde0613bb [const][uart][usb_uart][weikai][core] Move constants to components/const (#14430) 2026-03-03 07:53:18 -05:00
J. Nick Koston 2e623fd6c3 [tests] Fix flaky log assertion race in oversized payload tests (#14414) 2026-03-02 11:48:50 -10:00
J. Nick Koston 3615a7b90c [core] Eliminate __udivdi3 in millis() on ESP32 and RP2040 (#14409) 2026-03-02 11:42:25 -10:00
Bonne Eggleston 3160457ca6 Create integration tests for modbus (#14395)
Co-authored-by: J. Nick Koston <nick@home-assistant.io>
2026-03-01 22:51:27 -10:00
J. Nick Koston 80a2acca4f [ld2410] Add UART mock integration test for LD2410 component (#14377) 2026-03-01 18:19:32 -10:00
Jonathan Swoboda 0d5b7df77d [sensor] Fix delta filter percentage mode regression (#14302)
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
2026-02-25 19:32:02 -05:00
J. Nick Koston 8bb577de64 [api] Split ProtoVarInt::parse into 32-bit and 64-bit phases (#14039) 2026-02-25 12:23:13 -06:00
J. Nick Koston b539a5aa51 [water_heater] Fix device_id missing from state responses (#14212) 2026-02-22 23:07:56 +00:00
Jonathan Swoboda db7870ef5f [alarm_control_panel] Fix flaky integration test race condition (#13964)
Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com>
2026-02-12 16:04:39 -05:00
J. Nick Koston 483b7693e1 [api] Fix debug asserts in production code, encode_bool bug, and reduce flash overhead (#13936)
Co-authored-by: Jonathan Swoboda <154711427+swoboda1337@users.noreply.github.com>
2026-02-11 13:57:08 -06:00
tronikos e3141211c3 [water_heater] Add On/Off and Away mode support to template platform (#13839)
Co-authored-by: J. Nick Koston <nick@koston.org>
Co-authored-by: J. Nick Koston <nick@home-assistant.io>
2026-02-10 12:45:18 +00:00
J. Nick Koston e0712cc53b [scheduler] Make core timer ID collisions impossible with type-safe internal IDs (#13882)
Co-authored-by: Jonathan Swoboda <154711427+swoboda1337@users.noreply.github.com>
2026-02-09 13:16:22 -06:00