[scheduler] Snapshot cross-thread counters once per Scheduler::call()

On ESPHOME_THREAD_MULTI_NO_ATOMICS (BK72xx is the sole target — ARMv5TE,
no LDREX/STREX, so std::atomic is off), the per-helper _empty_() fast
paths fall back to "always take the lock". That means Scheduler::call()
paid a separate FreeRTOS mutex round-trip for each of:

  - process_defer_queue_   (defer_empty_)
  - process_to_add         (to_add_empty_)
  - cleanup_               (to_remove_empty_)

on every iteration, even on idle ticks where all three counters are
zero. Each round-trip is ~5-10us on BK72xx, adding ~75ms/min of
main-loop overhead at ~3100 iter/min. This matched the measured gap
between BK72xx (sched=125ms/min) and RTL87xx (sched=19ms/min) for
identical code.

Snapshot all three counters once at the top of Scheduler::call():

  - NO_ATOMICS: single LockGuard, read three plain uint32_t fields.
  - ATOMICS:    three relaxed atomic loads (free, same order as the
                per-helper fast paths).
  - SINGLE:     untouched — the existing direct container checks are
                already cheap with no concurrent writers.

The three skip-work gates below then branch on the snapshot instead of
each calling its own _empty_() (and re-locking on NO_ATOMICS). When a
gate fires, the slow path is invoked directly so it still acquires the
lock fresh to read container state.

After process_defer_queue_slow_path_ runs, resnapshot to_add_count_ and
to_remove_: callbacks dispatched by the defer queue can call
set_timeout/set_interval/cancel_*, which mutate those counters. Missing
that refresh would cause the subsequent process_to_add / cleanup_ gates
to skip freshly-queued work for one tick. The defer queue itself is
drained inside the slow path so snap_defer is consumed by the single
call.

Measured on BK7238/BK7231N while debugging overhead alongside
libretiny-eu/libretiny#360.
This commit is contained in:
J. Nick Koston
2026-04-23 05:23:25 -05:00
parent 43a371caab
commit ecbc249d7a
2 changed files with 51 additions and 16 deletions
+24 -12
View File
@@ -534,13 +534,29 @@ void HOT Scheduler::process_defer_queue_slow_path_(uint32_t &now) {
#endif /* not ESPHOME_THREAD_SINGLE */
uint32_t HOT Scheduler::call(uint32_t now) {
// Snapshot the skip-work counters once up front so the gates below don't
// each re-lock on NO_ATOMICS. See snapshot_counters_ for per-platform cost.
uint32_t snap_add;
uint32_t snap_remove;
#ifndef ESPHOME_THREAD_SINGLE
this->process_defer_queue_(now);
#endif /* not ESPHOME_THREAD_SINGLE */
uint32_t snap_defer;
this->snapshot_counters_(snap_defer, snap_add, snap_remove);
if (snap_defer > 0) {
this->process_defer_queue_slow_path_(now);
// Defer callbacks may set_timeout/set_interval/cancel_*, mutating the
// other two counters. Re-snapshot.
this->snapshot_counters_(snap_defer, snap_add, snap_remove);
}
#else
this->snapshot_counters_(snap_add, snap_remove);
#endif
// Extend the caller's 32-bit timestamp to 64-bit for scheduler operations
const auto now_64 = this->millis_64_from_(now);
this->process_to_add();
if (snap_add > 0)
this->process_to_add_slow_path_();
// Track if any items were added to to_add_ during callbacks
bool has_added_items = false;
@@ -582,13 +598,9 @@ uint32_t HOT Scheduler::call(uint32_t now) {
}
#endif /* ESPHOME_DEBUG_SCHEDULER */
// Cleanup removed items before processing
// First try to clean items from the top of the heap (fast path)
this->cleanup_();
// If we still have too many cancelled items, do a full cleanup
// This only happens if cancelled items are stuck in the middle/bottom of the heap
if (this->to_remove_count_() >= MAX_LOGICALLY_DELETED_ITEMS) {
// cleanup_slow_path_ returns the post-cleanup to_remove_ value (read
// under lock), so the MAX gate uses the fresh count for free.
if (snap_remove > 0 && this->cleanup_slow_path_() >= MAX_LOGICALLY_DELETED_ITEMS) {
this->full_cleanup_removed_items_();
}
// IMPORTANT: This loop uses index-based access (items_[0]), NOT iterators.
@@ -723,7 +735,7 @@ void HOT Scheduler::process_to_add_slow_path_() {
this->to_add_.clear();
this->to_add_count_clear_();
}
bool HOT Scheduler::cleanup_slow_path_() {
uint32_t HOT Scheduler::cleanup_slow_path_() {
// We must hold the lock for the entire cleanup operation because:
// 1. We're modifying items_ (via pop_raw_locked_) which requires exclusive access
// 2. We're decrementing to_remove_ which is also modified by other threads
@@ -740,7 +752,7 @@ bool HOT Scheduler::cleanup_slow_path_() {
this->to_remove_decrement_();
this->recycle_item_main_loop_(this->pop_raw_locked_());
}
return !this->items_.empty();
return this->to_remove_count_();
}
Scheduler::SchedulerItem *HOT Scheduler::pop_raw_locked_() {
std::pop_heap(this->items_.begin(), this->items_.end(), SchedulerItem::cmp);
+27 -4
View File
@@ -312,10 +312,11 @@ class Scheduler {
inline bool ESPHOME_ALWAYS_INLINE HOT cleanup_() {
if (this->to_remove_empty_())
return !this->items_.empty();
return this->cleanup_slow_path_();
this->cleanup_slow_path_();
return !this->items_.empty();
}
// Slow path for cleanup_() when there are items to remove - defined in scheduler.cpp
bool cleanup_slow_path_();
// Slow path for cleanup_() - returns the post-cleanup to_remove_ count.
uint32_t cleanup_slow_path_();
// Slow path for process_to_add() when there are items to merge - defined in scheduler.cpp
void process_to_add_slow_path_();
// Remove and return the front item from the heap as a raw pointer.
@@ -446,7 +447,29 @@ class Scheduler {
// IMPORTANT: Caller must hold the scheduler lock before calling this function.
// IMPORTANT: Must not be inlined - rare path, outlined to keep it out of the hot instruction cache lines.
void __attribute__((noinline)) compact_defer_queue_locked_();
#endif /* not ESPHOME_THREAD_SINGLE */
// Snapshot skip-work counters for Scheduler::call(). NO_ATOMICS: one lock
// for all three reads. ATOMICS: three relaxed loads, free.
inline void ESPHOME_ALWAYS_INLINE HOT snapshot_counters_(uint32_t &defer, uint32_t &add, uint32_t &remove) {
#ifdef ESPHOME_THREAD_MULTI_NO_ATOMICS
LockGuard guard{this->lock_};
defer = this->defer_count_;
add = this->to_add_count_;
remove = this->to_remove_;
#else /* ESPHOME_THREAD_MULTI_ATOMICS */
defer = this->defer_count_.load(std::memory_order_relaxed);
add = this->to_add_count_.load(std::memory_order_relaxed);
remove = this->to_remove_.load(std::memory_order_relaxed);
#endif
}
#else /* ESPHOME_THREAD_SINGLE */
// SINGLE form — two direct reads, no defer queue. Lets call() use the
// same snap_add / snap_remove gates on every platform.
inline void ESPHOME_ALWAYS_INLINE HOT snapshot_counters_(uint32_t &add, uint32_t &remove) {
add = static_cast<uint32_t>(this->to_add_.size());
remove = this->to_remove_;
}
#endif /* ESPHOME_THREAD_SINGLE */
// Helper to check if item is marked for removal (platform-specific)
// Returns true if item should be skipped, handles platform-specific synchronization