The generated src/main.cpp.o was being skipped by _scan_source_symbols
(which only walked src/esphome/) and _source_file_to_component had no
rule for it. As a result setup()/loop() fell through to the
"app_framework" heuristic in const.py instead of being counted as
[esphome]core, both via the nm scan path and the linker map path.
Scan all of src/ and recognize .../src/main.cpp.o as core.
Replace `type(mock_response).content = property(...)` with a tiny
_BodyReadErrorResponse stub class. The previous form mutates
MagicMock's class globally, which can leak into every other
MagicMock-using test in this file.
`requests.Response.content` reads the body lazily, so chunked-decode,
gzip-decode, and mid-stream connection drops all surface as
RequestException subclasses on first access -- not from requests.get
itself. The previous code accessed `.content` outside the surrounding
try/except, so any of those (rare but real) errors would propagate
out of download_content instead of falling back to the cached file
or raising the user-friendly cv.Invalid.
Move `data = req.content` inside the try block so the existing error
path handles it. Two new tests cover the with-cache and no-cache
branches using a `ChunkedEncodingError` injected on `.content`.
- _read_etag collapses into a single try/except that handles both stat
and read failures (and the unlink) uniformly.
- _mtime_seconds() helper compares mtimes at whole-second resolution
instead of nanoseconds. Sub-second precision varies by filesystem
(FAT does 2s, NTFS 100ns, APFS/ext4 ns); whole seconds is the only
resolution every supported filesystem can preserve through a
set+read round-trip.
- has_remote_file_changed: walrus-assign `etag` and `new_etag` instead
of an explicit binding + immediate check.
- New test fixtures `mock_requests_head`, `mock_requests_get`,
`mock_has_remote_file_changed`, `mock_write_file` replace the
per-test `@patch(...)` decorator stacks. Tests are simpler and the
fixtures can be reused by future tests in the file.
- Both audio_file and speaker.media_player now use functools.partial to
bind path_for=_compute_local_file_path instead of an inline lambda.
- download_content_many's dedup loop becomes a dict comprehension.
Last-URL wins now (dict-comp semantics) instead of first-URL; in
practice duplicate paths only arise when the URL itself is duplicated,
so the choice is meaningless. Test updated accordingly.
- download_content_many now de-duplicates by `path` so two callers
asking for the same cache file (e.g. the same URL referenced twice
in a config) can't race on download_content's non-atomic write. When
duplicates are present, the first-seen URL for that path wins.
- Clamp `max_workers` to at least 1 so an invalid caller value can't
raise ValueError out of ThreadPoolExecutor.
- _write_etag: suppress OSError (not just FileNotFoundError) when removing
a stale sidecar -- ETag persistence is best-effort and a permission
error there must not abort an otherwise-successful download.
- Atomic-write test: patch write_file directly so the failure happens
inside the helper that does the rename, instead of before
download_content even calls it.
audio_file and speaker.media_player had byte-identical helpers that
collected web URLs from a list-of-file-configs and handed them to
download_content_many. Move that logic into external_files as
`download_web_files_in_config(config, path_for)` -- callers only have
to pass the per-component cache-path callback now.
The ETag sidecar describes a specific snapshot of the cache file. If the
cache file is replaced or edited out-of-band (manual edit, restore from
backup, another tool overwriting it), the sidecar's recorded ETag no
longer matches the bytes on disk -- using it would cause the server to
return 304 and we'd serve the wrong content from cache.
- _write_etag now os.utime()'s the sidecar to share the cache file's
st_mtime_ns immediately after writing it.
- _read_etag compares the two mtimes; if they don't match it returns
None and removes the sidecar so subsequent calls don't re-check it.
Each component that uses external_files (audio_file, speaker
media_player, ...) currently calls download_content once per file
inline inside a per-item config validator. With ~24 cached audio
files in a Home Assistant Voice PE config, that means ~24 sequential
HEAD round-trips, even when every response is a 304.
This adds download_content_many(items, ...) which fans the per-file
checks out across a ThreadPoolExecutor (capped at 16 workers so
configs with hundreds of files don't open hundreds of sockets), then
refactors audio_file and speaker.media_player to collect URLs at the
list level and call the batch helper once instead of downloading
inside each per-item validator.
Wall time for the validation phase drops from sum(latency) to roughly
max(latency) when the cache is warm.
raw.githubusercontent.com ignores If-Modified-Since (always returns
200), but honors If-None-Match with ETag (returns 304). This caused
every esphome compile/config run to re-download every cached external
file (audio_file, micro_wake_word, image, font, bme68x_bsec2, etc.)
sourced from a /raw/ URL.
- Send If-None-Match with the cached ETag when present
- Persist the ETag from each download in a hidden sidecar file
(.{name}.etag) and refresh it when a 304 carries a new ETag
- Replace path.write_bytes() with helpers.write_file() so downloads are
written atomically and can no longer leave partially-written cache
files behind on crash
Avoid per-file HTTP HEAD requests during config validation when running
esphome logs against a previously-cached project. The skip_external_update
flag was already plumbed for git operations, but external_files.download_content
ignored it. Thread it through CORE so audio_file, micro_wake_word,
speaker/media_player, image, font, and bme68x_bsec2 reuse cached files
without per-URL network round-trips when the file already exists locally.
J. Nick Koston