The ETag sidecar describes a specific snapshot of the cache file. If the
cache file is replaced or edited out-of-band (manual edit, restore from
backup, another tool overwriting it), the sidecar's recorded ETag no
longer matches the bytes on disk -- using it would cause the server to
return 304 and we'd serve the wrong content from cache.
- _write_etag now os.utime()'s the sidecar to share the cache file's
st_mtime_ns immediately after writing it.
- _read_etag compares the two mtimes; if they don't match it returns
None and removes the sidecar so subsequent calls don't re-check it.
Each component that uses external_files (audio_file, speaker
media_player, ...) currently calls download_content once per file
inline inside a per-item config validator. With ~24 cached audio
files in a Home Assistant Voice PE config, that means ~24 sequential
HEAD round-trips, even when every response is a 304.
This adds download_content_many(items, ...) which fans the per-file
checks out across a ThreadPoolExecutor (capped at 16 workers so
configs with hundreds of files don't open hundreds of sockets), then
refactors audio_file and speaker.media_player to collect URLs at the
list level and call the batch helper once instead of downloading
inside each per-item validator.
Wall time for the validation phase drops from sum(latency) to roughly
max(latency) when the cache is warm.
raw.githubusercontent.com ignores If-Modified-Since (always returns
200), but honors If-None-Match with ETag (returns 304). This caused
every esphome compile/config run to re-download every cached external
file (audio_file, micro_wake_word, image, font, bme68x_bsec2, etc.)
sourced from a /raw/ URL.
- Send If-None-Match with the cached ETag when present
- Persist the ETag from each download in a hidden sidecar file
(.{name}.etag) and refresh it when a 304 carries a new ETag
- Replace path.write_bytes() with helpers.write_file() so downloads are
written atomically and can no longer leave partially-written cache
files behind on crash
- Cache UART selection at setup time so each loop iteration no longer
dereferences global_logger and pays for a non-inlined Logger::get_uart()
call before the read switch.
- Use App.get_loop_component_start_time() once per loop instead of two
millis() calls (especially relevant on ESP8266 where millis() involves
interrupt-locked 64-bit timer access).
- Move read_byte_() to the header as ESPHOME_ALWAYS_INLINE so the call/ret
pair and optional<uint8_t> staging are elided at the call sites in loop().
Avoid per-file HTTP HEAD requests during config validation when running
esphome logs against a previously-cached project. The skip_external_update
flag was already plumbed for git operations, but external_files.download_content
ignored it. Thread it through CORE so audio_file, micro_wake_word,
speaker/media_player, image, font, and bme68x_bsec2 reuse cached files
without per-URL network round-trips when the file already exists locally.
The TEMPLATABLE_VALUE C++ fields are typed `int`, but Python codegen
was passing `cg.int32` (`int32_t`). On ESP-IDF/xtensa toolchains where
`int32_t` is `long`, the codegen-generated lambda's function-pointer
type `int32_t (*)(...)` does not match `int (*)(...)`, causing
`TemplatableFn` to select its deprecated casting-trampoline overload
and emit a `-Wdeprecated-declarations` warning at every call site.
Use `cg.int_` so the lambda's return type matches the field exactly.
- Add `--` terminator before submodule paths (both clone and refresh
paths) so a path beginning with `-` cannot be parsed as a git option.
- Reword the refresh-fetch comment: this fetch also runs when ref is
None, in which case it pulls the remote default branch.
Fixesesphome/esphome#11550
When clone_or_update is called with a ref or submodules, the initial
clone uses --depth=1 but the subsequent git fetch and git submodule
update commands run deep, pulling the full history on every refresh.
This causes excess network traffic and disk usage for external_components,
packages, and dashboard imports.
Add --depth=1 to all four call sites (initial fetch, initial submodule
init, refresh fetch, refresh submodule update). Shallow fetches still
advance the local clone to the current remote tip when it moves, so the
refresh path keeps working as before.
The base sensor already logs the published state via the [S]
publish line, so the component-level [D] line was redundant
log spam on every update. Drop it to ESP_LOGV.
J. Nick Koston