- Both audio_file and speaker.media_player now use functools.partial to
bind path_for=_compute_local_file_path instead of an inline lambda.
- download_content_many's dedup loop becomes a dict comprehension.
Last-URL wins now (dict-comp semantics) instead of first-URL; in
practice duplicate paths only arise when the URL itself is duplicated,
so the choice is meaningless. Test updated accordingly.
- download_content_many now de-duplicates by `path` so two callers
asking for the same cache file (e.g. the same URL referenced twice
in a config) can't race on download_content's non-atomic write. When
duplicates are present, the first-seen URL for that path wins.
- Clamp `max_workers` to at least 1 so an invalid caller value can't
raise ValueError out of ThreadPoolExecutor.
- _write_etag: suppress OSError (not just FileNotFoundError) when removing
a stale sidecar -- ETag persistence is best-effort and a permission
error there must not abort an otherwise-successful download.
- Atomic-write test: patch write_file directly so the failure happens
inside the helper that does the rename, instead of before
download_content even calls it.
audio_file and speaker.media_player had byte-identical helpers that
collected web URLs from a list-of-file-configs and handed them to
download_content_many. Move that logic into external_files as
`download_web_files_in_config(config, path_for)` -- callers only have
to pass the per-component cache-path callback now.
The ETag sidecar describes a specific snapshot of the cache file. If the
cache file is replaced or edited out-of-band (manual edit, restore from
backup, another tool overwriting it), the sidecar's recorded ETag no
longer matches the bytes on disk -- using it would cause the server to
return 304 and we'd serve the wrong content from cache.
- _write_etag now os.utime()'s the sidecar to share the cache file's
st_mtime_ns immediately after writing it.
- _read_etag compares the two mtimes; if they don't match it returns
None and removes the sidecar so subsequent calls don't re-check it.
Each component that uses external_files (audio_file, speaker
media_player, ...) currently calls download_content once per file
inline inside a per-item config validator. With ~24 cached audio
files in a Home Assistant Voice PE config, that means ~24 sequential
HEAD round-trips, even when every response is a 304.
This adds download_content_many(items, ...) which fans the per-file
checks out across a ThreadPoolExecutor (capped at 16 workers so
configs with hundreds of files don't open hundreds of sockets), then
refactors audio_file and speaker.media_player to collect URLs at the
list level and call the batch helper once instead of downloading
inside each per-item validator.
Wall time for the validation phase drops from sum(latency) to roughly
max(latency) when the cache is warm.
raw.githubusercontent.com ignores If-Modified-Since (always returns
200), but honors If-None-Match with ETag (returns 304). This caused
every esphome compile/config run to re-download every cached external
file (audio_file, micro_wake_word, image, font, bme68x_bsec2, etc.)
sourced from a /raw/ URL.
- Send If-None-Match with the cached ETag when present
- Persist the ETag from each download in a hidden sidecar file
(.{name}.etag) and refresh it when a 304 carries a new ETag
- Replace path.write_bytes() with helpers.write_file() so downloads are
written atomically and can no longer leave partially-written cache
files behind on crash
Avoid per-file HTTP HEAD requests during config validation when running
esphome logs against a previously-cached project. The skip_external_update
flag was already plumbed for git operations, but external_files.download_content
ignored it. Thread it through CORE so audio_file, micro_wake_word,
speaker/media_player, image, font, and bme68x_bsec2 reuse cached files
without per-URL network round-trips when the file already exists locally.
- Add `--` terminator before submodule paths (both clone and refresh
paths) so a path beginning with `-` cannot be parsed as a git option.
- Reword the refresh-fetch comment: this fetch also runs when ref is
None, in which case it pulls the remote default branch.
Fixesesphome/esphome#11550
When clone_or_update is called with a ref or submodules, the initial
clone uses --depth=1 but the subsequent git fetch and git submodule
update commands run deep, pulling the full history on every refresh.
This causes excess network traffic and disk usage for external_components,
packages, and dashboard imports.
Add --depth=1 to all four call sites (initial fetch, initial submodule
init, refresh fetch, refresh submodule update). Shallow fetches still
advance the local clone to the current remote tip when it moves, so the
refresh path keeps working as before.
Wrap the on_finished_playback CallbackManager storage, registration
method and call site behind USE_RTTTL_FINISHED_PLAYBACK_CALLBACK so
configs without an on_finished_playback automation pay zero RAM and
zero flash for it.
Keeps CallbackManager rather than switching to StaticCallbackManager
because rtttl is MULTI_CONF — a per-class compile-time N would force
every instance to reserve max(callbacks_per_instance) slots.
EsphomeLogsHandler.build_command now appends --no-states to the
spawned `esphome logs` argv when the WebSocket spawn message
includes `no_states: true`. This lets the dashboard frontend
suppress entity-state log lines for OTA log sessions without
requiring users to drop to the CLI.
Adds three unit tests covering the truthy, missing, and explicit-False
cases.
- script/build_helpers.py: when injecting a non-MULTI_CONF component
into the post-validation config, run its CONFIG_SCHEMA with {} so
defaults are populated. Without this, socket got config = {} and
socket.FILTER_SOURCE_FILES crashed with KeyError on
'implementation' (the schema's defaulted key was never filled in).
Falls back to {} if the schema can't validate empty input.
- tests/components/json/__init__.py: enable codegen for json so its
to_code runs during cpp unit test builds, registering the
ArduinoJson library. Required for any api dep test, since
json_util.cpp #includes <ArduinoJson.h>.
Locally verified 'script/cpp_unit_test.py api' now compiles and runs;
ProtoMacVarint test suite (9 cases) passes.
- Test file: declare proto_debug_end_ locally instead of misusing
PROTO_ENCODE_DEBUG_INIT (which expands to a comma+expression for
appending to a function call, not a standalone statement). Add
NOLINTNEXTLINE on the deterministic mt19937_64 seed so clang-tidy
cert-msc32-c stops failing the build (the seed is intentional for
reproducible test runs).
- socket FILTER_SOURCE_FILES: tolerate non-dict CORE.config['socket']
(e.g. C++ unit-test builds where socket isn't validated as a
mapping). Returning [] is safe -- all impl files are guarded by
USE_SOCKET_IMPL_* defines so only the selected one contributes
code.
Verifies encode_varint_raw_48bit and calc_uint64_48bit_force for the
8 corner-case MAC addresses requested in review:
00:00:00:00:00:00, 11:00:00:00:00:00, 00:AA:00:00:00:00,
00:00:BB:00:00:00, 00:00:00:CC:00:00, 00:00:00:00:DD:00,
00:00:00:00:00:EE, FF:FF:FF:FF:FF:FF
For each value the test asserts byte-identical output to the reference
encode_varint_raw_64 loop, the expected encoded byte length, agreement
with calc_uint64_48bit_force, and round-trip through a generic varint
decoder. Adds a 100-value deterministic-random sample across the full
48-bit space for additional coverage.
J. Nick Koston