Commit Graph
282 Commits
Author SHA1 Message Date
J. Nick Koston f06e96685b [store_yaml] Refactor YAML discovery to share bundle.py's approach
The previous implementation extended `track_yaml_loads` across the entire
validation pass to catch deferred `!include` and package loads, but that also
captured framework YAML loaded internally by component validators (e.g.
LVGL's `hello_world.yaml`) and produced spurious "unresolved substitution"
warnings during the pre-validation force-load. bundle.py already had the
right pattern: a fresh post-validation re-parse plus `force_load_include_files`.

- Lift the discovery into `yaml_util.discover_user_yaml_files` and have both
  `bundle.py` and `config.py` use it (DRY).
- Capture `secrets.yaml` / `secrets.yml` by the *un-resolved* listener fname
  so a `secrets.yaml` symlinked to a non-secrets-named target is still
  flagged for redaction.
- Add a `warn_on_unresolved` flag to `force_load_include_files` so the
  discovery path (where substitutions haven't run) logs at debug instead of
  warning.
- Reject `store_yaml` configs with an unencrypted API via
  `FINAL_VALIDATE_SCHEMA`; an explicit `allow_unencrypted: true` opt-out
  keeps lab setups working (renamed from `allow_unencrypted_api` so the
  integration-test harness's naive `api:` string replacement doesn't
  clobber it).
- Annotate `store_yaml_chunk_buf` with the
  `cppcoreguidelines-avoid-non-const-global-variables` suppression that
  matches other intentional API-side globals.
- Update unit tests to exercise the new `DiscoveredYamlFiles` shape plus a
  symlink-secrets case.
2026-05-15 10:21:00 -07:00
J. Nick Koston 3d77e3f5dd [store_yaml] Address review: capability bit, dump suppression, paths, unit tests
- Advertise `has_store_yaml` in DeviceInfoResponse so recovery tooling can
  detect support without timing out against firmware built without
  USE_STORE_YAML.
- Suppress GetYamlResponse from the proto dump path. Every chunk would
  otherwise log embedded configuration (including opted-in secrets) on
  builds with HAS_PROTO_MESSAGE_DUMP, and bloat logs during recovery.
- Preserve the include graph for files outside the project root: use
  `os.path.relpath` instead of just the basename so e.g. two
  `../common.yaml` siblings don't collide on recovery.
- Keep `track_yaml_loads` open across `validate_config` so files loaded
  by remote packages and substitution-resolved includes are captured.
- Add focused unit tests for `_gather_files` (redaction, secrets.yml,
  opt-in, dedupe, external-path handling, missing sources) and
  `_pack_envelope` (round-trip, UTF-8 paths, overlong-path guard).
- Make the test_bundle assertion case-insensitive.
2026-05-15 10:20:45 -07:00
J. Nick Koston ec1826a6ed [yaml_util] Promote include-discovery helper, share it with bundle (#16447) 2026-05-15 10:17:50 -07:00
Jonathan Swoboda 56983f414f [espidf] Gate esp_idf_size --ng on IDF version (#16441) 2026-05-14 22:41:36 -04:00
Jonathan Swoboda a3b6f92433 [espidf] Regenerate bundled CMakeLists; auto-REQUIRE via IDF build properties (#16406) 2026-05-13 23:58:48 +00:00
Jonathan Swoboda 910cc38dd7 [writer] Clean ESP-IDF build artifacts in clean_build (#16410) 2026-05-13 19:25:35 -04:00
Jonathan Swoboda 1c6966b761 [espidf] Run PIO extraScript with SCons-env shim (#16404) 2026-05-13 13:07:59 -04:00
J. Nick Koston cb520cda6b [core] Retry PlatformIO downloads on transport-layer errors (#16397) 2026-05-13 10:01:42 -05:00
J. Nick Koston 8bce32ec35 [tests] Cover top-level !include failure path in track_yaml_loads (#16396) 2026-05-13 10:01:26 -05:00
J. Nick Koston 0e4922a340 [core] Cache validated config to skip re-validation on upload/logs (#16381) 2026-05-13 22:14:19 +12:00
65ea29b44a [core] Fix !include vars not being substituted in !lambda values (#16320)
Co-authored-by: J. Nick Koston <nick@koston.org>
Co-authored-by: J. Nick Koston <nick@home-assistant.io>
2026-05-12 22:41:30 -05:00
76d3433425 [cli] Add config-hash command (#15548)
Co-authored-by: J. Nick Koston <nick@koston.org>
Co-authored-by: J. Nick Koston <nick@home-assistant.io>
2026-05-12 20:43:48 +00:00
Clyde Stubbs 4ff946ac15 [cli] Add --no-states flag to run command (#16366) 2026-05-12 07:52:07 -04:00
b5e50144e3 [ota] Improve OTA error messages (#16327)
Co-authored-by: pre-commit-ci-lite[bot] <117423508+pre-commit-ci-lite[bot]@users.noreply.github.com>
Co-authored-by: Copilot Autofix powered by AI <175728472+Copilot@users.noreply.github.com>
Co-authored-by: J. Nick Koston <nick+github@koston.org>
2026-05-12 02:32:58 +00:00
J. Nick Koston 55ef66cc26 [helpers] Re-enable ProgressBar under --dashboard mode (#16357) 2026-05-11 13:12:55 -05:00
J. Nick Koston a52ca4f80a [ota] Implement host platform OTA backend with re-exec for integration testing (#16304) 2026-05-11 10:51:08 -05:00
J. Nick Koston 4d9d6e02e5 [core] Add stable esphome.upload_targets module for port classification (#16346) 2026-05-11 09:13:16 -04:00
J. Nick Koston b967adeb9d [wifi] Accept lowercase variant in variant_has_wifi (#16345) 2026-05-11 09:11:36 -04:00
Jonathan Swoboda e4d9786f00 [core] Move platformio files to subfolder (#16347) 2026-05-11 21:34:47 +12:00
e9cc10fedc [core] Native idf full support (#14678)
Co-authored-by: Jonathan Swoboda <154711427+swoboda1337@users.noreply.github.com>
Co-authored-by: Jonathan Swoboda <swoboda1337@users.noreply.github.com>
Co-authored-by: pre-commit-ci-lite[bot] <117423508+pre-commit-ci-lite[bot]@users.noreply.github.com>
Co-authored-by: Jesse Hills <3060199+jesserockz@users.noreply.github.com>
2026-05-11 02:12:07 +00:00
J. Nick Koston 66e2dcffc4 [cli] Tighten command_rename: scoped name rewrite, target-collision check (#16296) 2026-05-11 13:00:09 +12:00
J. Nick Koston 17080ddce6 [wifi][rp2040] Add stable wifi-capability helpers for device-builder (#16300) 2026-05-11 12:56:56 +12:00
J. Nick Koston 930d539969 [config_validation] Add a visibility UI-hint kwarg (#16267) 2026-05-11 12:07:15 +12:00
1365251365 [ota] Add bootloader update functionality to ota component (#16238)
Co-authored-by: pre-commit-ci-lite[bot] <117423508+pre-commit-ci-lite[bot]@users.noreply.github.com>
Co-authored-by: J. Nick Koston <nick@home-assistant.io>
2026-05-08 21:36:06 +00:00
90693fb39a [core] Fix WiFi connection in safe mode (#16269)
Co-authored-by: J. Nick Koston <nick@home-assistant.io>
Co-authored-by: pre-commit-ci-lite[bot] <117423508+pre-commit-ci-lite[bot]@users.noreply.github.com>
Co-authored-by: J. Nick Koston <nick@koston.org>
2026-05-06 09:56:33 -05:00
J. Nick Koston 6e1a59da3e [packages] Make package !include vars visible to its substitutions block (#16274) 2026-05-06 09:53:14 -05:00
J. Nick Koston caaa1aefc7 [substitutions] Fix sibling references inside dict-valued substitutions (#16273) 2026-05-06 08:41:17 -05:00
J. Nick Koston ff0c5f575e [bundle] Include secrets.yaml when !secret keys are quoted (#16271) 2026-05-06 07:32:35 -05:00
J. Nick Koston 85f33978e7 [core] Skip external component update on esphome clean (#16268) 2026-05-06 07:23:35 -04:00
J. Nick Koston f30ad588ea [cli] Add --ota-platform flag to pick web_server or native API OTA (#16207) 2026-05-05 18:25:53 -05:00
Jesse Hills 7c2a63bf82 [api] Use safe_print for log output and fix safe_print bytes-repr fallback (#16160) 2026-05-05 08:12:20 +12:00
Diorcet Yann 690a197346 [main] Move stacktrace handling out of platformio_api and FlashImage into platform components/util (#16186) 2026-05-04 15:07:31 -04:00
J. Nick Koston 9ddb828da3 [api] Don't tear down log connection on stack-trace decode failure (#16196) 2026-05-03 20:05:13 -05:00
J. Nick KostonandCopilot Autofix powered by AI b5eb444015 [dashboard] Stabilize device-builder dashboard backend's API surface (#16206)
Co-authored-by: Copilot Autofix powered by AI <175728472+Copilot@users.noreply.github.com>
2026-05-03 20:01:51 -05:00
7cfab58a05 [ota] Add partition table update functionality to ota component (#15780)
Co-authored-by: pre-commit-ci-lite[bot] <117423508+pre-commit-ci-lite[bot]@users.noreply.github.com>
Co-authored-by: J. Nick Koston <nick@home-assistant.io>
Co-authored-by: J. Nick Koston <nick@koston.org>
2026-05-03 19:42:10 -05:00
Mat931andJ. Nick Koston 58cb7effd4 [ota] Add extended OTA protocol (#16164)
Co-authored-by: J. Nick Koston <nick@home-assistant.io>
2026-05-01 10:40:14 -05:00
Jesse Hills e127268dac [core] Strip \\?\ prefix from sys.executable for PlatformIO subprocess (#16158) 2026-04-30 16:04:52 +12:00
J. Nick Koston 985dba9332 [core] Defer heavy module-scope imports in __main__, loader, and config (#15955) 2026-04-29 13:17:59 -05:00
J. Nick Koston 44cabc191d [core] Catch body-read errors in download_content (#16023) 2026-04-29 14:06:41 -04:00
J. Nick Koston 8ceada8d04 [core] Download external_files in parallel (#16021) 2026-04-29 14:32:30 +12:00
J. Nick Koston 592486ae9a [analyze_memory] Attribute main.cpp setup()/loop() to esphome core (#16033) 2026-04-29 14:06:54 +12:00
J. Nick Koston eec770d622 [core] Use ETag in external_files cache to fix re-downloads from raw.githubusercontent.com (#16020) 2026-04-29 13:52:09 +12:00
J. Nick Koston d7b21a84a3 [git] Make ref fetches and submodule updates shallow (#16014) 2026-04-29 13:49:51 +12:00
J. Nick Koston 0759a3c681 [core] Split wake.{h,cpp} into per-platform files (#15978) 2026-04-28 08:48:13 -05:00
J. Nick Koston 2e096bb036 [core] Combine set_component_source_ + register_component_ into one call (#16029) 2026-04-26 21:54:15 -05:00
J. Nick Koston 0f25d91e68 [core] Unify skip_external_update and honor it in external_files for faster esphome logs (#16016) 2026-04-26 07:24:33 -05:00
68625a1b76 [core] Isolate generated build metadata (#16007)
Co-authored-by: J. Nick Koston <nick@koston.org>
Co-authored-by: J. Nick Koston <nick@home-assistant.io>
2026-04-26 09:11:09 +00:00
b5ccd55f4e [packages] Fix premature substitution of vars in remote package files (#15997)
Co-authored-by: J. Nick Koston <nick+github@koston.org>
Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>
2026-04-25 17:06:58 +00:00
Paulus Schoutsen 9b45b046a8 [core] Allow finding all devices as target that match mac suffix (#13135) 2026-04-23 08:43:32 -05:00
Jesse Hills e1d629f0d2 [time] Handle Windows EINVAL when validating POSIX TZ strings (#15934) 2026-04-23 14:35:13 +12:00