Sub-microsecond benchmarks are dominated by the ~60ns per-iteration
valgrind start/stop cost in CodSpeed simulation mode. Add kInnerIterations
(1000) inner loops to all fast benchmarks so the actual work dominates.
Move DoNotOptimize calls outside inner loops to prevent artificial overhead.
Also address review feedback:
- Use tokenless CodSpeed (public repo, no CODSPEED_TOKEN needed)
- Fix warning message to show component-specific path
- Fix stray ". :" in error message
- Verify pinned SHA on re-runs to prevent stale checkouts
- Call original_setup() in benchmark main.cpp so code-generated App
initialization (pre_setup, area/device registration, looping_components_
init) runs before benchmarks
- Restore ApplicationLoop_Empty benchmark that calls the actual App.loop()
- Remove BM_ prefix from all benchmark names (unnecessary convention)
- Remove bench_application_loop.cpp (App needs pre_setup/setup which
requires full code generation; will revisit as integration benchmark)
- Fix extra_include_dirs to use os.path.relpath for sibling directories
Core is not a component — its benchmarks belong in tests/benchmarks/core/
not tests/benchmarks/components/core/. Add extra_include_dirs parameter
to build_and_run to support non-component benchmark directories.
- Move scheduler/loop/helpers benchmarks to tests/benchmarks/components/core/
- Add random_float and random_uint32 benchmarks (from ol.yaml)
- Fix core pseudo-component crash: skip components where get_component()
returns None when adding dependencies to config
- Use CodSpeed simulation mode (CPU instruction counting) for reproducible
CI results instead of walltime
Add automated benchmarks using Google Benchmark to prevent performance
regressions in the API protobuf encoding/decoding and core loop paths.
Benchmarks cover:
- Protobuf encode: SensorState, BinarySensorState, HelloResponse,
LightState, DeviceInfoResponse (20 nested devices + 20 areas)
- Protobuf decode: HelloRequest, SwitchCommand, LightCommand
- Protobuf calculate_size and full calc+encode send path
- Varint parse/encode/size for various value ranges
- Scheduler call/next_schedule_in with idle and active timers
- Application loop component dispatch and blocking guard overhead
Infrastructure:
- Extract shared build logic from cpp_unit_test.py into test_helpers.py
- Add cpp_benchmark.py mirroring the unit test build pattern
- Support benchmark.yaml per component dir for declaring dependencies
- Add CodSpeed CI workflow triggered on api/core changes
- Fix ProtoMessage protected destructor on host platform
Co-authored-by: Copilot Autofix powered by AI <175728472+Copilot@users.noreply.github.com>
Co-authored-by: pre-commit-ci-lite[bot] <117423508+pre-commit-ci-lite[bot]@users.noreply.github.com>
J. Nick Koston