Compare commits

...
Author SHA1 Message Date
J. Nick Koston d2b6fa0245 Derive both raw swap oracles from one helper 2026-09-03 23:58:47 +02:00
J. Nick Koston d5cab16b0b Diagnostic, case normalized number entity lookup 2026-09-03 23:58:47 +02:00
J. Nick Koston ad06db6ca8 Address review: exact object_id lookup for the write numbers 2026-09-03 23:58:47 +02:00
J. Nick Koston 59365beaf6 Address review: shared raw view constant, honest annotation, require_entity in the bits test 2026-09-03 23:58:47 +02:00
J. Nick Koston 92b57d1851 Apply simplify pass: derive write expectations from one table, reuse require_entity 2026-09-03 23:58:47 +02:00
J. Nick Koston 2bc5a695cc Address review: raw sensor pins the byte swap on the write path 2026-09-03 23:58:47 +02:00
J. Nick Koston c46365a95d Note the number schema bound cap on the write numbers 2026-09-03 23:58:46 +02:00
J. Nick Koston 5cf1ff9798 Address review: distinct fp32_r baseline, note the byte-swapped raw view 2026-09-03 23:58:46 +02:00
J. Nick Koston 0df09adebb Trim fixture comments 2026-09-03 23:58:46 +02:00
J. Nick Koston ad9ef273da Apply simplify pass: single source for the mesh baseline values 2026-09-03 23:58:46 +02:00
J. Nick Koston 22aa570b47 [core] Fold the modbus write and bits tests into the shared mesh fixture 2026-09-03 23:58:46 +02:00
J. Nick Koston 8d43616507 Address review: marker scan covers every usage form including multiline pytestmark lists 2026-09-03 23:58:43 +02:00
J. Nick Koston a5bbac084a Address review: marker scan also covers pytestmark assignments 2026-09-03 23:40:58 +02:00
J. Nick Koston 44cea5c0ed Tighten shared_yaml marker validation to exactly one literal arg 2026-09-03 23:05:38 +02:00
J. Nick Koston 28ab280c7b Address review: per decorator marker validation instead of count matching 2026-09-03 22:47:46 +02:00
J. Nick Koston f2f9b9cbff Address review: layout agnostic leftover ELF sweep 2026-09-03 22:18:21 +02:00
J. Nick Koston f32e3e75f4 Address review: guard utime against peer reap, decorator anchored use detector, invariant holds without a stamp, print for executor thread messages 2026-09-03 22:05:29 +02:00
J. Nick Koston 5494190d97 Address review: survive peer reap before lock, alias tolerant marker scan, document shared_yaml, visible edge warnings 2026-09-03 21:40:52 +02:00
J. Nick Koston 36468022ea Address review: prefs helper requires the redirect, visible stamp rejection, lock aware age re-probe, wider marker regexes 2026-09-03 21:01:49 +02:00
J. Nick Koston 19f5641706 Address review: decorator anchored guard, validated stamp path, max mtime age probe, stray file reclaim 2026-09-03 19:55:15 +02:00
J. Nick Koston 900dd63883 Address review: prefs seeding honors the redirect, age sweep re-probes under the lock 2026-09-03 18:47:38 +02:00
J. Nick Koston bca4c9dbba Apply simplify pass: delete ELF before compile, reuse esphome.helpers, hoist prefs isolation fixture 2026-09-03 17:44:36 +02:00
J. Nick Koston e1595cb7ef Address review: utf-8 everywhere, built stamp with self healing relink, per test prefs dir, visible prune warnings 2026-09-03 17:31:45 +02:00
J. Nick Koston 86c10ac0c8 Address review: relink freshness check, quarantine partial prunes, log age probe errors 2026-09-03 17:14:54 +02:00
J. Nick Koston 7b17651837 Address review: age sweep for orphaned build dirs, logger warnings, strict marker literal guard 2026-09-03 16:08:38 +02:00
J. Nick Koston cb7cfa5340 Address review: visible prune failures, lock wait progress, marker arg validation, cache hit exists check 2026-09-03 12:16:13 +02:00
J. Nick Koston 9ca77cd7c7 Address review: orphan fixture invariant, note shared prefs constraint on the marker 2026-09-03 11:24:49 +02:00
J. Nick Koston da5f43ce41 Address review: shared fixtures visible to CI test selection, checkout scoped prune, cancellable build lock 2026-09-03 10:54:05 +02:00
J. Nick Koston 736218e747 Address review: per checkout build keys, prune stale build dirs, guard against mutated shared configs 2026-09-03 10:19:09 +02:00
J. Nick Koston 6fce48f1d5 Trim fixture comments 2026-09-02 12:31:20 +02:00
J. Nick Koston e3820e62bc Apply simplify pass: key shared builds off the fixture source, hoist the cache root, dedup helpers 2026-09-02 12:20:13 +02:00
J. Nick Koston 92e663821c [core] Share compiled binaries across modbus integration tests 2026-09-02 12:02:33 +02:00
esphome[bot] da16c01351 [ci] Refresh integration test durations (#18944) 2026-09-02 09:32:47 +00:00
Jesse Hills 6099ac7b53 [core] Document C++ conventions in AGENTS.md that reviews keep catching (#18941) 2026-09-02 11:17:22 +02:00
dependabot[bot]anddependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> b8480b8424 Bump pylint from 4.0.7 to 4.0.8 (#18935)
Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2026-09-01 21:36:49 -04:00
dependabot[bot]anddependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> 8bef5b22e1 Bump zeroconf from 0.150.4 to 0.151.2 (#18934)
Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2026-09-01 21:36:04 -04:00
Jakepre-commit-ci-lite[bot] <117423508+pre-commit-ci-lite[bot]@users.noreply.github.com>Jesse Hills
379e077b5f [ds1603l] New sensor DS1603L V1.0 (#13133)
Co-authored-by: pre-commit-ci-lite[bot] <117423508+pre-commit-ci-lite[bot]@users.noreply.github.com>
Co-authored-by: Jesse Hills <3060199+jesserockz@users.noreply.github.com>
2026-09-02 10:05:56 +12:00
J. Nick KostonandJesse Hills 3f68930001 [ota] Add Noise encryption to the OTA platform (#18489)
Co-authored-by: Jesse Hills <3060199+jesserockz@users.noreply.github.com>
2026-09-02 08:11:57 +12:00
0aff9e1c54 [d01] add D01 pm2.5 sensor support (#17788)
Co-authored-by: Andrej Walilko <awalilko@liquidweb.com>
Co-authored-by: Jesse Hills <3060199+jesserockz@users.noreply.github.com>
2026-09-02 07:33:35 +12:00
ZebbleandClaude f9824ee83f [core] Move CONF_KEYS to the shared component constants (#18933)
Co-authored-by: Claude <noreply@anthropic.com>
2026-09-01 14:23:21 -04:00
Oliver KleineckeOliver Kleineckepre-commit-ci-lite[bot] <117423508+pre-commit-ci-lite[bot]@users.noreply.github.com>Jonathan Swoboda
68ffd5a773 [safe_mode] Uncover silent error in safe-mode (#18749)
Co-authored-by: Oliver Kleinecke <kleinecke.oliver@googlemail.com>
Co-authored-by: pre-commit-ci-lite[bot] <117423508+pre-commit-ci-lite[bot]@users.noreply.github.com>
Co-authored-by: Jonathan Swoboda <154711427+swoboda1337@users.noreply.github.com>
2026-09-01 09:12:49 -04:00
fe3788ff47 [esp32][mipi_rgb] Add ESP32-S31 support for execute_from_psram (#18929)
Co-authored-by: Claude Sonnet 5 <noreply@anthropic.com>
Co-authored-by: Copilot Autofix powered by AI <175728472+Copilot@users.noreply.github.com>
2026-09-01 08:56:03 -04:00
J. Nick Koston d6758377d1 [core] Clone git libraries in parallel in the library prefetch (#18836) 2026-09-01 14:25:58 +12:00
Jesse Hills 5dbc8ffe4c [epaper_spi] Add UC8179 mono driver and Seeed reTerminal E1001 model (#17568) 2026-09-01 14:17:52 +12:00
J. Nick Koston 0f982f03b2 [core] Prefetch tool-scons by PlatformIO's core spec (#18831) 2026-09-01 11:55:53 +12:00
Bonne Eggleston afb0022dd0 [core] Lint: require braces around single ESP_LOG control-statement bodies (#18727) 2026-09-01 11:53:39 +12:00
124 changed files with 4342 additions and 1830 deletions
+96 -2
View File
@@ -44,6 +44,16 @@ This document provides essential context for AI models interacting with this pro
## 4. Coding Conventions & Style Guide
**Read the developer documentation before writing a component.** https://developers.esphome.io covers the
component lifecycle, the main loop, and the reasoning behind the rules below in far more depth than this
file does, and it is the authority when they disagree. The most useful starting points:
* https://developers.esphome.io/architecture/components/ - component lifecycle, `setup()`, `loop()`,
setup priorities, and how a component is registered.
* https://developers.esphome.io/architecture/components/advanced/ - choosing between `loop()`,
`set_interval`, `set_timeout` and `defer`; waking the loop from another thread; the RAM cost of each.
* https://developers.esphome.io/contributing/code/ - contribution rules, public API and breaking changes.
* **Formatting:**
* **Python:** Uses `ruff` and `flake8` for linting and formatting. Configuration is in `pyproject.toml`.
* **C++:** Uses `clang-format` for formatting. Configuration is in `.clang-format`.
@@ -142,6 +152,47 @@ This document provides essential context for AI models interacting with this pro
* **Indentation:** Use spaces (two per indentation level), not tabs
* **Type aliases:** Prefer `using type_t = int;` over `typedef int type_t;`
* **Line length:** Wrap lines at no more than 120 characters
* **Timing in `loop()`:** Never call `millis()` in a `loop()` body. The current tick's timestamp is
already cached - use `App.get_loop_component_start_time()` (from `esphome/core/application.h`).
Only reach for `millis()` when you genuinely need sub-tick resolution inside a long operation.
* **The main loop runs every 16 ms.** A rate-limit gate shorter than that does nothing: the check
passes on essentially every pass of the loop, so it costs a comparison and buys nothing. Pick an
interval comfortably coarser than 16 ms, or drop the gate entirely and accept running every loop.
```cpp
// Bad - a 10ms gate against a 16ms loop never holds anything back
static constexpr uint32_t POLL_INTERVAL_MS = 10;
const uint32_t now = millis();
if (now - this->last_poll_ < POLL_INTERVAL_MS)
return;
this->last_poll_ = now;
```
```cpp
// Good - an interval that actually rate limits, off the cached timestamp
static constexpr uint32_t POLL_INTERVAL_MS = 100;
const uint32_t now = App.get_loop_component_start_time();
if (now - this->last_poll_ < POLL_INTERVAL_MS)
return;
this->last_poll_ = now;
```
Pick the primitive by cadence: under 250 ms use a gated `loop()`; 500 ms and above use
`set_interval`. Full reasoning, including why `set_interval` costs more below 500 ms:
https://developers.esphome.io/architecture/components/advanced/#quick-rule-of-thumb
* **Don't override a default with the same value:** if a base class method already returns what you
want, do not override it. `Component::get_setup_priority()` returns `setup_priority::DATA`, so a
component that wants `DATA` should simply leave it alone.
```cpp
// Bad - this is exactly what the base class already does
float get_setup_priority() const override { return setup_priority::DATA; }
```
* **Logging string literals:** wrap literals passed as `%s` arguments in `LOG_STR_LITERAL()` so they
can be stored in flash rather than RAM.
```cpp
// Bad
ESP_LOGV(TAG, "Key %u %s", key, pressed ? "pressed" : "released");
// Good
ESP_LOGV(TAG, "Key %u %s", key, pressed ? LOG_STR_LITERAL("pressed") : LOG_STR_LITERAL("released"));
```
* **Constructor parameters vs setters:** Component properties that are both **required** and **invariant**
(never change after construction) should be constructor parameters rather than set via setter methods.
This makes the dependency explicit and prevents use of the object in an incompletely-initialized state.
@@ -562,6 +613,33 @@ This document provides essential context for AI models interacting with this pro
Use `cg.add_define("MAX_SERVICES", count)` to set the size from Python configuration.
Like `std::array` but with vector-like API (`push_back()`, `size()`) and no STL reallocation code.
**Listener and child-entity registration lists are the most common case, and the most commonly
missed.** A `register_*()` method called once per child at code generation time has a count that
is known at compile time, so it should never be a `std::vector`. Use `cg.slot_counter()`: it
returns a function that each consumer calls once per slot it will occupy, and after every
`to_code` has run it emits the define with the final count. When nothing registers, no define is
emitted and the storage plus its registration method compile out entirely.
```python
# hub component's __init__.py
_request_listener_slot = cg.slot_counter("MY_COMPONENT_LISTENER_COUNT")
async def register_listener(hub: MockObj, var: MockObj) -> None:
_request_listener_slot()
cg.add(hub.register_listener(var))
```
```cpp
#ifdef MY_COMPONENT_LISTENER_COUNT
void register_listener(MyComponentListener *listener);
#endif
protected:
#ifdef MY_COMPONENT_LISTENER_COUNT
StaticVector<MyComponentListener *, MY_COMPONENT_LISTENER_COUNT> listeners_;
#endif
```
Request slots from `to_code`, not from a job that runs after `CoroPriority.FINAL` - a late
request raises rather than silently undercounting.
3. **Runtime-known sizes:** Use `FixedVector` from `esphome/core/helpers.h` when the size is only known at runtime initialization.
```cpp
// Bad - generates STL realloc code (_M_realloc_insert)
@@ -599,9 +677,25 @@ This document provides essential context for AI models interacting with this pro
```
Linear search on small datasets (1-16 elements) is often faster than hashing/tree overhead, but this depends on lookup frequency and access patterns. For frequent lookups in hot code paths, the O(1) vs O(n) complexity difference may still matter even for small datasets. `std::vector` with simple structs is usually fine—it's the heavy containers (`map`, `set`, `unordered_map`) that should be avoided for small datasets unless profiling shows otherwise.
5. **Avoid `std::deque`:** It allocates in 512-byte blocks regardless of element size, guaranteeing at least 512 bytes of RAM usage immediately. This is a major source of crashes on memory-constrained devices.
5. **Strings set once from configuration:** Use `StringRef` (`esphome/core/string_ref.h`) rather than
`std::string`. Code generation passes a string literal that lives in flash for the life of the
program, so storing a `std::string` copies it onto the heap for nothing. `StringRef` is a
non-owning pointer plus length; it does not copy, and it must only ever refer to storage that
outlives it (a string literal, or a buffer owned elsewhere).
```cpp
// Bad - heap copy of a literal that is already in flash
void set_keys(std::string keys) { this->keys_ = std::move(keys); }
std::string keys_;
```
```cpp
// Good - no allocation
void set_keys(const char *keys) { this->keys_ = StringRef(keys); }
StringRef keys_;
```
6. **Detection:** Look for these patterns in compiler output:
6. **Avoid `std::deque`:** It allocates in 512-byte blocks regardless of element size, guaranteeing at least 512 bytes of RAM usage immediately. This is a major source of crashes on memory-constrained devices.
7. **Detection:** Look for these patterns in compiler output:
- Large code sections with STL symbols (vector, map, set)
- `alloc`, `realloc`, `dealloc` in symbol names
- `_M_realloc_insert`, `_M_default_append` (vector reallocation)
+2
View File
@@ -131,6 +131,7 @@ esphome/components/cst816/* @clydebarrow
esphome/components/cst9220/* @clydebarrow
esphome/components/ct_clamp/* @jesserockz
esphome/components/current_based/* @djwmarcx
esphome/components/d01/* @ch604
esphome/components/dac7678/* @NickB1
esphome/components/daikin_arc/* @MagicBear
esphome/components/daikin_brc/* @hagak
@@ -148,6 +149,7 @@ esphome/components/display_menu_base/* @numo68
esphome/components/dlms_meter/* @latonita @PolarGoose @SimonFischer04 @Tomer27cz
esphome/components/dps310/* @kbx81
esphome/components/ds1307/* @badbadc0ffee
esphome/components/ds1603l/* @JakeLC15
esphome/components/ds2484/* @mrk-its
esphome/components/ds248x/* @tomwellnitz
esphome/components/dsmr/* @glmnet @PolarGoose
+35 -3
View File
@@ -23,7 +23,8 @@ For this repository there are two trusted inputs by design:
1. **The configuration.** Anyone who can supply or edit a YAML config is trusted
(see below).
2. **Authenticated peers of a running device** — clients holding the device's
API encryption key / password, OTA password, or web server credentials.
API/OTA encryption key, API password, OTA password, or web server
credentials.
The security boundary is therefore **unauthenticated network traffic vs. those
trusted inputs.** A bug that lets an unauthenticated attacker cross it is a
@@ -76,8 +77,8 @@ These *are* security bugs in this repo, and we want to hear about them privately
captive portal, etc.) **without** valid credentials.
- Authentication or encryption bypass on the device — reaching API calls, OTA
updates, or the web server without the configured key/password.
- Flaws that weaken the device's API encryption (Noise), OTA, or web server auth
below their documented guarantees.
- Flaws that weaken the device's API or OTA encryption (Noise), OTA auth, or
web server auth below their documented guarantees.
## The web server is an open HTTP API by design
@@ -121,6 +122,37 @@ and any memory-safety or protocol bug in the server reachable without credential
This section documents the current design and scope; it is not a judgment that the
design is optimal or that it will not change.
## OTA update encryption
The `esphome` OTA platform optionally encrypts updates with the same Noise
`NNpsk0` pattern the native API uses; one key protects the device. With an
`encryption:` block configured the guarantees are: the firmware image is
confidential in transit, the uploader is authenticated by the pre-shared key,
and the plaintext negotiation preceding the handshake is bound into the
handshake prologue, so stripping or tampering with it fails the first MAC.
Both ends fail closed with no override: a device built with a key refuses
plaintext uploads, and the CLI refuses to send plaintext when a key is
configured.
Defeating any of that without the key is in scope: a keyed device accepting a
plaintext or downgraded upload, getting past the MAC, or recovering image
contents from captured traffic.
The following are **not** vulnerabilities, by design:
- Plaintext OTA on a device with no `encryption:` block. That is the
documented default, authenticated (if at all) by the OTA password.
- The enablement window: turning encryption on takes one last upload of the
encryption-enabled firmware over the existing plaintext channel, with the
pre-existing plaintext exposure.
- The web OTA `/update` endpoint alongside encryption. The `web_server`
component keeps it always reachable, and `captive_portal:` auto-loads it
for the fallback AP window; validation warns about both combinations, and
the operator keeps the recovery path.
- CLI retry behavior on transport or MAC failures; every attempt renegotiates
a fresh handshake with fresh ephemerals, so retrying does not weaken
authentication.
## Explicitly out of scope
- Local attackers who already have shell access on the host that runs `esphome`.
+28 -1
View File
@@ -26,7 +26,9 @@ from esphome.const import (
CONF_DEASSERT_RTS_DTR,
CONF_DISABLED,
CONF_DISCOVER_IP,
CONF_ENCRYPTION,
CONF_ESPHOME,
CONF_KEY,
CONF_LEVEL,
CONF_LOG,
CONF_LOG_TOPIC,
@@ -1336,6 +1338,19 @@ def _upload_via_native_api(
remote_port = int(ota_conf[CONF_PORT])
password = ota_conf.get(CONF_PASSWORD)
# Fail closed: an encryption block whose key did not resolve must never
# fall back to a plaintext upload
noise_psk = None
if (encryption_conf := ota_conf.get(CONF_ENCRYPTION)) is not None:
noise_psk = encryption_conf.get(CONF_KEY)
if not noise_psk:
raise EsphomeError(
"OTA encryption is configured but no key was resolved; "
"set the key under 'ota: encryption:' or 'api: encryption:'"
)
# Ensure the key is a string, as required by the underlying OTA implementation.
# It arrives here as a SensitiveStr which aioesphomeapi rejects.
noise_psk = str(noise_psk)
def check_partition_access(option_string: str) -> None:
if not ota_conf.get("allow_partition_access"):
@@ -1366,7 +1381,9 @@ def _upload_via_native_api(
if ota_type == espota2.OTA_TYPE_UPDATE_BOOTLOADER:
_validate_bootloader_binary(binary)
return espota2.run_ota(network_devices, remote_port, password, binary, ota_type)
return espota2.run_ota(
network_devices, remote_port, password, binary, ota_type, noise_psk
)
def _upload_via_web_server(
@@ -1375,6 +1392,16 @@ def _upload_via_web_server(
from esphome import web_server_ota
from esphome.web_server_helpers import get_web_server_connection
if any(
ota_item.get(CONF_PLATFORM) == CONF_ESPHOME
and ota_item.get(CONF_ENCRYPTION) is not None
for ota_item in config.get(CONF_OTA, [])
):
_LOGGER.warning(
"This config has OTA encryption, but the web_server OTA path sends "
"the image over plaintext HTTP; use the esphome OTA platform to "
"keep it confidential"
)
remote_port, username, password = get_web_server_connection(config)
return web_server_ota.run_ota(
network_devices, remote_port, username, password, binary
+2 -1
View File
@@ -162,8 +162,9 @@ void Alpha3::send_request_(uint8_t *request, size_t len) {
auto status =
esp_ble_gattc_write_char(this->parent_->get_gattc_if(), this->parent_->get_conn_id(), this->geni_handle_, len,
request, ESP_GATT_WRITE_TYPE_NO_RSP, ESP_GATT_AUTH_REQ_NONE);
if (status)
if (status) {
ESP_LOGW(TAG, "[%s] esp_ble_gattc_write_char failed, status=%d", this->parent_->address_str(), status);
}
}
void Alpha3::update() {
+2 -1
View File
@@ -2391,8 +2391,9 @@ void APIConnection::process_batch_() {
} else if (payload_size == 0) {
// payload_size == 0 with remove set means encoding hit OOM and the
// connection is being dropped; warn only for a genuinely oversized message
if (!this->flags_.remove)
if (!this->flags_.remove) {
ESP_LOGW(TAG, "Message too large to send: type=%u", item.message_type);
}
this->clear_batch_();
}
return;
+2 -1
View File
@@ -62,8 +62,9 @@ BdkActivityState bdk_scan_state(uint8_t activity_idx) {
uint8_t bdk_scan_acquire_activity() {
uint8_t idx = app_ble_get_idle_actv_idx_handle(SCAN_ACTV);
if (idx == INVALID_ACTIVITY_IDX)
if (idx == INVALID_ACTIVITY_IDX) {
ESP_LOGE(TAG, "Scan start failed: no idle activity handle");
}
return idx;
}
+10 -5
View File
@@ -181,8 +181,9 @@ void BK72xxBLE::enable() {
break;
}
}
if (!bdaddr_live)
if (!bdaddr_live) {
ESP_LOGW(TAG, "Controller address still unset after init; BLE stack may not have started");
}
#endif
this->state_ = BLEComponentState::ACTIVE;
@@ -210,8 +211,9 @@ void BK72xxBLE::loop() {
// Re-check a settled scan; scan_start() refills the bring-up budget.
// WARN: the only report of a drop that recovers inside its budget.
if (this->scan_start(this->requested_.interval, this->requested_.window, this->requested_.active) !=
ScanOpResult::SETTLED)
ScanOpResult::SETTLED) {
ESP_LOGW(TAG, "Controller dropped the scan; restarting");
}
}
// Drain the lock-free ring filled by the BLE task; all per-report work runs
@@ -230,8 +232,9 @@ void BK72xxBLE::loop() {
// Log dropped reports — only reachable when reports were processed; drops can
// only occur while the queue is full, and only this loop drains it.
uint16_t dropped = this->report_queue_.get_and_reset_dropped_count();
if (dropped > 0)
if (dropped > 0) {
ESP_LOGW(TAG, "Dropped %u scan reports due to queue overflow", dropped);
}
}
void BK72xxBLE::get_mac_lsb_first(uint8_t out[MAC_ADDRESS_SIZE]) const {
@@ -449,8 +452,9 @@ ScanOpResult BK72xxBLE::advance_stop_(BdkActivityState state, bool ready) {
if (!ready) {
// Acting mid-operation could delete an activity whose start lands
// afterwards, leaking the slot with the radio on; wait.
if (this->last_result_ == ScanOpResult::SETTLED)
if (this->last_result_ == ScanOpResult::SETTLED) {
ESP_LOGD(TAG, "Scan stop deferred (controller busy)");
}
return ScanOpResult::PENDING;
}
// Settled, so CREATED unambiguously means "never started".
@@ -474,8 +478,9 @@ ScanOpResult BK72xxBLE::advance_start_(BdkActivityState state, bool ready) {
return ScanOpResult::PENDING;
}
if (!ready) {
if (this->last_result_ == ScanOpResult::SETTLED)
if (this->last_result_ == ScanOpResult::SETTLED) {
ESP_LOGD(TAG, "Scan start deferred (controller busy)");
}
return ScanOpResult::PENDING;
}
if (state == BdkActivityState::CREATED) {
@@ -69,8 +69,9 @@ void BK72xxBLETracker::on_ota_global_state(ota::OTAState state, float progress,
this->stop_scan();
// The transfer starves the loop; a deferred stop would leave the radio
// scanning for the whole update, so drain it here, bounded.
if (!this->parent_->flush_pending_stop(OTA_STOP_FLUSH_MS))
if (!this->parent_->flush_pending_stop(OTA_STOP_FLUSH_MS)) {
ESP_LOGE(TAG, "Scan still stopping at OTA start; the radio may contend with the update");
}
} else if (state == ota::OTA_ERROR || state == ota::OTA_ABORT) {
// On success the device reboots, so restore only on a failed/aborted update;
// loop() restarts the scan on its next iteration (continuous idle branch).
@@ -80,8 +80,9 @@ void BLEBinaryOutput::write_state(bool state) {
esp_err_t err =
esp_ble_gattc_write_char(this->parent()->get_gattc_if(), this->parent()->get_conn_id(), this->char_handle_,
sizeof(state_as_uint), &state_as_uint, this->write_type_, ESP_GATT_AUTH_REQ_NONE);
if (err != ESP_GATT_OK)
if (err != ESP_GATT_OK) {
ESP_LOGW(TAG, "[%s] Write error, err=%d", this->char_uuid_.to_str(char_buf), err);
}
}
} // namespace esphome::ble_client
+4 -2
View File
@@ -327,10 +327,12 @@ void BME680Component::read_data_() {
ESP_LOGD(TAG, "Got temperature=%.1f°C pressure=%.1fhPa humidity=%.1f%% gas_resistance=%.1fΩ", temperature, pressure,
humidity, gas_resistance);
if (!gas_valid)
if (!gas_valid) {
ESP_LOGW(TAG, "Gas measurement unsuccessful, reading invalid!");
if (!heat_stable)
}
if (!heat_stable) {
ESP_LOGW(TAG, "Heater unstable, reading invalid! (Normal for a few readings after a power cycle)");
}
if (this->temperature_sensor_ != nullptr)
this->temperature_sensor_->publish_state(temperature);
+12 -6
View File
@@ -749,33 +749,39 @@ void Climate::dump_traits_(const char *tag) {
}
if (!traits.get_supported_modes().empty()) {
ESP_LOGCONFIG(tag, " Supported modes:");
for (ClimateMode m : traits.get_supported_modes())
for (ClimateMode m : traits.get_supported_modes()) {
ESP_LOGCONFIG(tag, " - %s", LOG_STR_ARG(climate_mode_to_string(m)));
}
}
if (!traits.get_supported_fan_modes().empty()) {
ESP_LOGCONFIG(tag, " Supported fan modes:");
for (ClimateFanMode m : traits.get_supported_fan_modes())
for (ClimateFanMode m : traits.get_supported_fan_modes()) {
ESP_LOGCONFIG(tag, " - %s", LOG_STR_ARG(climate_fan_mode_to_string(m)));
}
}
if (!traits.get_supported_custom_fan_modes().empty()) {
ESP_LOGCONFIG(tag, " Supported custom fan modes:");
for (const char *s : traits.get_supported_custom_fan_modes())
for (const char *s : traits.get_supported_custom_fan_modes()) {
ESP_LOGCONFIG(tag, " - %s", s);
}
}
if (!traits.get_supported_presets().empty()) {
ESP_LOGCONFIG(tag, " Supported presets:");
for (ClimatePreset p : traits.get_supported_presets())
for (ClimatePreset p : traits.get_supported_presets()) {
ESP_LOGCONFIG(tag, " - %s", LOG_STR_ARG(climate_preset_to_string(p)));
}
}
if (!traits.get_supported_custom_presets().empty()) {
ESP_LOGCONFIG(tag, " Supported custom presets:");
for (const char *s : traits.get_supported_custom_presets())
for (const char *s : traits.get_supported_custom_presets()) {
ESP_LOGCONFIG(tag, " - %s", s);
}
}
if (!traits.get_supported_swing_modes().empty()) {
ESP_LOGCONFIG(tag, " Supported swing modes:");
for (ClimateSwingMode m : traits.get_supported_swing_modes())
for (ClimateSwingMode m : traits.get_supported_swing_modes()) {
ESP_LOGCONFIG(tag, " - %s", LOG_STR_ARG(climate_swing_mode_to_string(m)));
}
}
}
+2
View File
@@ -14,6 +14,7 @@ CONF_CHANNEL_COLORS = "channel_colors"
CONF_CLIMATE_ID = "climate_id"
CONF_CO2_EQUIVALENT = "co2_equivalent"
CONF_COLOR_DEPTH = "color_depth"
CONF_COLUMNS = "columns"
CONF_CRC_ENABLE = "crc_enable"
CONF_DATA_BITS = "data_bits"
CONF_DESCRIPTION = "description"
@@ -25,6 +26,7 @@ CONF_GYROSCOPE_RANGE = "gyroscope_range"
CONF_IAQ = "iaq"
CONF_IGNORE_NOT_FOUND = "ignore_not_found"
CONF_IS_WRGB = "is_wrgb"
CONF_KEYS = "keys"
CONF_LABEL = "label"
CONF_LIBRETINY = "libretiny"
CONF_LOOP = "loop"
View File
+45
View File
@@ -0,0 +1,45 @@
#include "d01.h"
#include "esphome/core/log.h"
// uart specification for d01 sensor from https://manuals.plus/ae/1005006417362019:
//
// A frame of serial output data includes 4 bytes, formatted as follows:
// __Characteristic byte: Fixed value 0xA5.
// __Data byte: DATAH is the high 7 bits of the concentration value, and DATAL is the low 7 bits of the concentration
// value.
// __Check byte: The low 7 bits of the sum of all bytes before the check byte.
//
// If the serial output is 4 bytes of data: 0*A5 0*01 0*2C 0*52, then DATAH = 0*01 = 1, DATAL = 0*2C = 44.
// Concentration value = 1*128 + 44 = 172 µg/m³.
//
// The PM2.5 dust concentration value obtained from the dust sensor needs to be calibrated with a K value coefficient
// based on the TSI instrument's photometric method. It is generally recommended to use 0.4.
namespace esphome::d01 {
static const char *const TAG = "d01";
static const uint8_t D01_FRAME_HEADER = 0xA5;
void D01SensorComponent::dump_config() { LOG_SENSOR(" ", "D01 PM2.5", this); }
void D01SensorComponent::loop() {
uint8_t buf[4];
while (this->available() >= 4) {
if (this->peek() != D01_FRAME_HEADER) {
this->read();
continue;
}
this->read_array(buf, 4);
uint8_t sum = (buf[0] + buf[1] + buf[2]) & 0x7F;
if (sum != buf[3]) {
ESP_LOGW(TAG, "checksum mismatch");
continue;
}
uint16_t latest_concentration = (buf[1] & 0x7F) * 128 + (buf[2] & 0x7F);
ESP_LOGV(TAG, "Unadjusted PM2.5 Concentration: %d µg/m³", latest_concentration);
this->publish_state(latest_concentration);
}
}
} // namespace esphome::d01
+14
View File
@@ -0,0 +1,14 @@
#pragma once
#include "esphome/core/component.h"
#include "esphome/components/sensor/sensor.h"
#include "esphome/components/uart/uart.h"
namespace esphome::d01 {
class D01SensorComponent final : public sensor::Sensor, public Component, public uart::UARTDevice {
public:
void dump_config() override;
void loop() override;
};
} // namespace esphome::d01
+45
View File
@@ -0,0 +1,45 @@
import esphome.codegen as cg
from esphome.components import sensor, uart
import esphome.config_validation as cv
from esphome.const import (
DEVICE_CLASS_PM25,
ICON_BLUR,
STATE_CLASS_MEASUREMENT,
UNIT_MICROGRAMS_PER_CUBIC_METER,
)
from esphome.types import ConfigType
CODEOWNERS = ["@ch604"]
DEPENDENCIES = ["uart"]
d01_ns = cg.esphome_ns.namespace("d01")
D01SensorComponent = d01_ns.class_(
"D01SensorComponent", sensor.Sensor, uart.UARTDevice, cg.Component
)
CONFIG_SCHEMA = (
sensor.sensor_schema(
D01SensorComponent,
unit_of_measurement=UNIT_MICROGRAMS_PER_CUBIC_METER,
icon=ICON_BLUR,
accuracy_decimals=0,
device_class=DEVICE_CLASS_PM25,
state_class=STATE_CLASS_MEASUREMENT,
)
.extend(cv.COMPONENT_SCHEMA)
.extend(uart.UART_DEVICE_SCHEMA)
)
FINAL_VALIDATE_SCHEMA = uart.final_validate_device_schema(
"d01",
baud_rate=9600,
require_rx=True,
require_tx=False,
)
async def to_code(config: ConfigType) -> None:
var = await sensor.new_sensor(config)
await cg.register_component(var, config)
await uart.register_uart_device(var, config)
+2 -1
View File
@@ -154,8 +154,9 @@ bool HOT IRAM_ATTR DHT::read_sensor_(float *temperature, float *humidity, bool r
}
}
if (error_code != 0) {
if (report_errors)
if (report_errors) {
ESP_LOGW(TAG, ESP_LOG_MSG_COMM_FAIL);
}
return false;
}
+68
View File
@@ -0,0 +1,68 @@
#include "ds1603l.h"
#include <cstring>
#include "esphome/core/helpers.h"
#include "esphome/core/log.h"
namespace esphome::ds1603l {
static const char *const TAG = "ds1603l.sensor";
void DS1603L::loop() {
// Assemble frames one byte at a time so a stream that starts mid-frame can realign
uint8_t byte;
while (this->available() > 0 && this->read_byte(&byte)) {
if (this->rx_count_ == 0 && byte != HEADER_BYTE) {
ESP_LOGV(TAG, "Skipping byte 0x%02X while looking for header", byte);
continue;
}
this->rx_buffer_[this->rx_count_++] = byte;
if (this->rx_count_ < FRAME_SIZE) {
continue;
}
if (this->parse_data_()) {
this->rx_count_ = 0;
} else {
// The header byte was part of the payload of a misaligned frame, so realign instead of dropping everything
this->resync_();
}
}
}
void DS1603L::dump_config() { LOG_SENSOR("", "DS1603L", this); }
bool DS1603L::parse_data_() {
uint8_t header = this->rx_buffer_[0];
uint8_t data_h = this->rx_buffer_[1];
uint8_t data_l = this->rx_buffer_[2];
uint8_t checksum = this->rx_buffer_[3];
uint8_t computed_checksum = (header + data_h + data_l) & 0xFF;
ESP_LOGV(TAG, "Data: Header=0x%02X, Data_H=0x%02X, Data_L=0x%02X, Checksum=0x%02X", header, data_h, data_l, checksum);
if (checksum != computed_checksum) {
ESP_LOGW(TAG, "Checksum mismatch: received 0x%02X, expected 0x%02X", checksum, computed_checksum);
return false;
}
this->publish_state(encode_uint16(data_h, data_l));
return true;
}
void DS1603L::resync_() {
// Drop the byte that was treated as the header, then look for the next candidate header in what is left
size_t start = 1;
while (start < this->rx_count_ && this->rx_buffer_[start] != HEADER_BYTE) {
start++;
}
this->rx_count_ -= start;
if (this->rx_count_ > 0) {
memmove(this->rx_buffer_, this->rx_buffer_ + start, this->rx_count_);
}
}
} // namespace esphome::ds1603l
+30
View File
@@ -0,0 +1,30 @@
#pragma once
#include <cstddef>
#include <cstdint>
#include "esphome/components/sensor/sensor.h"
#include "esphome/components/uart/uart.h"
#include "esphome/core/component.h"
namespace esphome::ds1603l {
class DS1603L final : public sensor::Sensor, public Component, public uart::UARTDevice {
public:
void loop() override;
void dump_config() override;
protected:
static constexpr uint8_t HEADER_BYTE = 0xFF;
static constexpr size_t FRAME_SIZE = 4;
// Validates the checksum of the frame in rx_buffer_ and publishes it. Returns false if the frame is invalid.
bool parse_data_();
// Drops the first buffered byte and realigns the buffer on the next possible header byte.
void resync_();
uint8_t rx_buffer_[FRAME_SIZE]; // Buffer for the frame being assembled
size_t rx_count_{0}; // Number of bytes currently in rx_buffer_
};
} // namespace esphome::ds1603l
+43
View File
@@ -0,0 +1,43 @@
import esphome.codegen as cg
from esphome.components import sensor, uart
import esphome.config_validation as cv
from esphome.const import (
DEVICE_CLASS_DISTANCE,
STATE_CLASS_MEASUREMENT,
UNIT_MILLIMETER,
)
from esphome.types import ConfigType
CODEOWNERS = ["@JakeLC15"]
DEPENDENCIES = ["uart"]
ds1603l_ns = cg.esphome_ns.namespace("ds1603l")
DS1603L = ds1603l_ns.class_("DS1603L", sensor.Sensor, cg.Component, uart.UARTDevice)
CONFIG_SCHEMA = (
sensor.sensor_schema(
DS1603L,
unit_of_measurement=UNIT_MILLIMETER,
accuracy_decimals=0,
device_class=DEVICE_CLASS_DISTANCE,
state_class=STATE_CLASS_MEASUREMENT,
)
.extend(uart.UART_DEVICE_SCHEMA)
.extend(cv.COMPONENT_SCHEMA)
)
FINAL_VALIDATE_SCHEMA = uart.final_validate_device_schema(
"ds1603l",
baud_rate=9600,
require_tx=False,
require_rx=True,
data_bits=8,
stop_bits=1,
)
async def to_code(config: ConfigType) -> None:
var = await sensor.new_sensor(config)
await cg.register_component(var, config)
await uart.register_uart_device(var, config)
@@ -0,0 +1,139 @@
#include "epaper_spi_uc8179.h"
#include <algorithm>
#include "esphome/core/log.h"
namespace esphome::epaper_spi {
static constexpr const char *const TAG = "epaper_spi.uc8179";
bool EPaperUC8179::initialise(bool partial) {
EPaperBase::initialise(partial); // send the model init sequence
this->partial_ = partial;
ESP_LOGV(TAG, "Power on");
// POWER ON must precede the waveform/mode registers and the data transfer
// (the original driver powers on and busy-waits before writing them).
// The state machine busy-waits before entering TRANSFER_DATA.
this->command(0x04);
// Give the busy line time to assert before the state machine polls it
this->next_delay_ = 100;
return true;
}
// Set up the refresh mode. Must be called after power-on has completed.
void EPaperUC8179::set_refresh_mode_() {
if (!this->is_using_partial_update_()) {
return; // plain full refresh uses the mode set by the init sequence
}
// Fast and partial refresh use flipped data polarity and a floating border
this->cmd_data(0x50, {0xA9, 0x07});
// Force the waveform via the temperature registers: 0x5A selects the fast
// full-refresh waveform, 0x6E the partial-refresh waveform
this->cmd_data(0xE0, {0x02});
if (this->partial_) {
this->cmd_data(0xE5, {0x6E});
this->command(0x91); // enter partial mode
// Set the partial window to the full screen
const uint16_t x_end = this->width_ - 1;
const uint16_t y_end = this->height_ - 1;
this->cmd_data(0x90, {0x00, 0x00, static_cast<uint8_t>(x_end >> 8), static_cast<uint8_t>(x_end & 0xFF), 0x00, 0x00,
static_cast<uint8_t>(y_end >> 8), static_cast<uint8_t>(y_end & 0xFF), 0x01});
} else {
this->cmd_data(0xE5, {0x5A});
this->command(0x92); // exit partial mode
}
}
bool HOT EPaperUC8179::transfer_data() {
const uint32_t start_time = millis();
const size_t buffer_length = this->buffer_length_;
if (this->current_data_index_ == 0) {
this->set_refresh_mode_();
}
// Fast full refresh sends the previous-image plane as well, so that every pixel transitions
const bool two_pass = this->is_using_partial_update_() && !this->partial_;
// Plain full refresh sends inverted data (buffer is 1=white, the wire wants 0=white);
// in fast/partial mode the data polarity is flipped via the VCOM/data-interval
// register instead, so the new-image plane is sent unmodified
const bool invert_new_data = !this->is_using_partial_update_();
uint8_t bytes_to_send[MAX_TRANSFER_SIZE];
// Phase 1 (fast full refresh only): previous image via 0x10 (DTM1), inverse of the new image
if (two_pass && this->current_data_index_ < buffer_length) {
if (this->current_data_index_ == 0) {
this->command(0x10); // DATA START TRANSMISSION 1 (previous image)
}
this->start_data_();
while (this->current_data_index_ < buffer_length) {
const size_t bytes_to_copy = std::min(MAX_TRANSFER_SIZE, buffer_length - this->current_data_index_);
for (size_t i = 0; i < bytes_to_copy; i++) {
bytes_to_send[i] = ~this->buffer_[this->current_data_index_ + i];
}
this->write_array(bytes_to_send, bytes_to_copy);
this->current_data_index_ += bytes_to_copy;
if (millis() - start_time > MAX_TRANSFER_TIME) {
this->disable();
return false;
}
}
this->disable();
}
// Phase 2: new image via 0x13 (DTM2)
const size_t offset = two_pass ? buffer_length : 0;
const size_t total = offset + buffer_length;
if (this->current_data_index_ < total) {
if (this->current_data_index_ == offset) {
this->command(0x13); // DATA START TRANSMISSION 2 (new image)
}
this->start_data_();
while (this->current_data_index_ < total) {
const size_t bytes_to_copy = std::min(MAX_TRANSFER_SIZE, total - this->current_data_index_);
const size_t data_idx = this->current_data_index_ - offset;
for (size_t i = 0; i < bytes_to_copy; i++) {
const uint8_t byte = this->buffer_[data_idx + i];
bytes_to_send[i] = invert_new_data ? ~byte : byte;
}
this->write_array(bytes_to_send, bytes_to_copy);
this->current_data_index_ += bytes_to_copy;
if (millis() - start_time > MAX_TRANSFER_TIME) {
this->disable();
return false;
}
}
this->disable();
}
this->current_data_index_ = 0;
return true;
}
void EPaperUC8179::power_on() {
// Power-on is sent at the end of initialise() instead, because the
// waveform/mode registers and the data transfer must follow it
}
void EPaperUC8179::refresh_screen(bool /*partial*/) {
ESP_LOGV(TAG, "Refresh");
this->command(0x12); // DISPLAY REFRESH
// Delay the next busy poll: the busy line takes a short time to assert after
// the refresh command, and polling too early would read it as already idle
this->next_delay_ = 100;
}
void EPaperUC8179::power_off() {
ESP_LOGV(TAG, "Power off");
this->command(0x02); // POWER OFF
}
void EPaperUC8179::deep_sleep() {
// Deep sleep loses the previous-image RAM that partial refresh compares against
if (!this->is_using_partial_update_()) {
ESP_LOGV(TAG, "Deep sleep");
this->cmd_data(0x07, {0xA5}); // DEEP SLEEP with check code
}
}
} // namespace esphome::epaper_spi
@@ -0,0 +1,52 @@
#pragma once
#include "epaper_spi.h"
namespace esphome::epaper_spi {
/**
* Monochrome e-paper displays using the UC8179 controller.
* Supports: 7.5" V2 (EPD_7in5_V2), 800x480 pixels, as used by the
* Waveshare 7.5" V2 HAT and the Seeed reTerminal E1001.
*
* Buffer layout: 1 bit per pixel, 1=white, 0=black (the base class default).
*
* The INITIALISE state sends the panel configuration followed by power-on
* (0x04); the state machine busy-waits for power-on to complete before
* TRANSFER_DATA, which first writes the waveform/mode registers (these are
* only accepted while powered) and then the image data. The state machine
* busy-waits again before triggering REFRESH_SCREEN (0x12).
*
* Three refresh modes are used, following the Waveshare EPD_7in5_V2 examples:
* - full_update_every == 1: plain full refresh. The new image is sent
* inverted to DTM2 (0x13) and the controller uses its normal waveform.
* - full_update_every > 1, full update: fast full refresh. The data polarity
* is flipped via the VCOM/data-interval register, a fast waveform is forced
* via the temperature registers, and the image is sent to both DTM1 (0x10,
* inverted) and DTM2 (0x13) so that every pixel transitions.
* - full_update_every > 1, partial update: partial refresh. A partial-update
* waveform is forced, partial mode is entered with a full-screen window and
* only DTM2 is sent; the controller compares against its previous-image RAM.
*/
class EPaperUC8179 final : public EPaperBase {
public:
EPaperUC8179(const char *name, uint16_t width, uint16_t height, const uint8_t *init_sequence,
size_t init_sequence_length)
: EPaperBase(name, width, height, init_sequence, init_sequence_length, DISPLAY_TYPE_BINARY) {
this->buffer_length_ = this->row_width_ * height;
}
protected:
bool initialise(bool partial) override;
bool transfer_data() override;
void refresh_screen(bool partial) override;
void power_on() override;
void power_off() override;
void deep_sleep() override;
void set_refresh_mode_();
// Set by initialise() so transfer_data() knows which planes to send
bool partial_{};
};
} // namespace esphome::epaper_spi
@@ -0,0 +1,93 @@
"""Monochrome e-paper displays using the UC8179 controller.
Supported models:
- waveshare-7.5in-v2: 7.5" mono display, 800x480 pixels (EPD_7in5_V2)
- seeed-reterminal-e1001: Seeed reTerminal E1001, which uses the same
7.5" 800x480 panel on an integrated ESP32-S3 board
Panel configuration and power-on (0x04) are both sent during the INITIALISE
state; the state machine's built-in busy wait then covers the power-on delay
before the waveform/mode registers and image data are transferred.
These displays support fast full and partial refresh: set ``full_update_every``
greater than 1 to enable it. Every ``full_update_every``-th update is a fast
full refresh, with partial refreshes in between.
"""
from typing import Any
from esphome.const import CONF_DATA_RATE
from . import EpaperModel
class UC8179(EpaperModel):
"""EpaperModel class for monochrome displays using the UC8179 controller."""
def __init__(
self,
name: str,
class_name: str = "EPaperUC8179",
data_rate: str = "10MHz",
**defaults: Any,
) -> None:
defaults.setdefault(CONF_DATA_RATE, data_rate)
super().__init__(name, class_name, **defaults)
def get_init_sequence(self, config: dict) -> tuple:
"""Generate the initialization sequence for UC8179 mono displays.
Panel configuration only the driver appends power-on (0x04) at the
end of the INITIALISE state, and the state machine busy-waits for it
to complete before the data transfer starts.
"""
width, height = self.get_dimensions(config)
return (
# POWER SETTING
(0x01, 0x07, 0x07, 0x3F, 0x3F),
# BOOSTER SOFT START
(0x06, 0x17, 0x17, 0x28, 0x17),
# PANEL SETTING (black/white mode, LUT from OTP)
(0x00, 0x1F),
# RESOLUTION SETTING (width x height)
(
0x61,
(width >> 8) & 0xFF,
width & 0xFF,
(height >> 8) & 0xFF,
height & 0xFF,
),
# DUAL SPI MODE (disabled)
(0x15, 0x00),
# VCOM AND DATA INTERVAL SETTING
(0x50, 0x10, 0x07),
# TCON SETTING
(0x60, 0x22),
)
uc8179 = UC8179("uc8179")
# Waveshare 7.5" V2 mono (EPD_7in5_V2) — 800x480, UC8179 controller
waveshare_7_5_v2 = uc8179.extend(
"waveshare-7.5in-v2",
width=800,
height=480,
)
# Seeed reTerminal E1001 — 7.5" mono e-paper (800x480), same panel as the
# Waveshare 7.5" V2, driven by an integrated ESP32-S3 board
waveshare_7_5_v2.extend(
"seeed-reterminal-e1001",
cs_pin=10,
dc_pin=11,
reset_pin=12,
busy_pin={
"number": 13,
"inverted": True,
"mode": {
"input": True,
"pullup": True,
},
},
)
+9 -8
View File
@@ -182,6 +182,13 @@ SIGNED_OTA_V1_ECDSA_VARIANTS = {
VARIANT_ESP32,
}
# Variants that support execution from PSRAM
PSRAM_XIP_VARIANTS = {
VARIANT_ESP32S3,
VARIANT_ESP32P4,
VARIANT_ESP32S31,
}
# NVS encryption (HMAC peripheral scheme) is only available on variants that
# expose the HMAC peripheral (SOC_HMAC_SUPPORTED in soc_caps.h). The original
# ESP32 and ESP32-C2 do not have it. New variants with an HMAC peripheral
@@ -1523,7 +1530,7 @@ def final_validate(config) -> None:
)
)
if advanced[CONF_EXECUTE_FROM_PSRAM]:
if config[CONF_VARIANT] not in {VARIANT_ESP32S3, VARIANT_ESP32P4}:
if config[CONF_VARIANT] not in PSRAM_XIP_VARIANTS:
errs.append(
cv.Invalid(
f"'{CONF_EXECUTE_FROM_PSRAM}' is not available on this esp32 variant",
@@ -2727,13 +2734,7 @@ async def to_code(config):
_configure_lwip_max_sockets(conf)
if advanced[CONF_EXECUTE_FROM_PSRAM]:
if variant == VARIANT_ESP32S3:
add_idf_sdkconfig_option("CONFIG_SPIRAM_FETCH_INSTRUCTIONS", True)
add_idf_sdkconfig_option("CONFIG_SPIRAM_RODATA", True)
elif variant == VARIANT_ESP32P4:
add_idf_sdkconfig_option("CONFIG_SPIRAM_XIP_FROM_PSRAM", True)
else:
raise ValueError("Unhandled ESP32 variant")
add_idf_sdkconfig_option("CONFIG_SPIRAM_XIP_FROM_PSRAM", True)
# Apply LWIP core locking for better socket performance
# This is already enabled by default in Arduino framework, where it provides
@@ -210,8 +210,9 @@ esp_err_t CameraWebServer::streaming_handler_(struct httpd_req *req) {
if (!image) {
// A shutdown is not a lost frame: wait_for_image_() returns empty as soon
// as running_ clears, and the loop condition below ends the stream anyway.
if (this->running_)
if (this->running_) {
ESP_LOGW(TAG, "STREAM: failed to acquire frame");
}
res = ESP_FAIL;
}
if (res == ESP_OK) {
+133 -2
View File
@@ -1,12 +1,20 @@
import logging
import esphome.codegen as cg
from esphome.components.noise import (
decode_encryption_key,
encryption_schema,
is_reserved_key,
)
from esphome.components.ota import BASE_OTA_SCHEMA, OTAComponent, ota_to_code
from esphome.config_helpers import merge_config
import esphome.config_validation as cv
from esphome.const import (
CONF_API,
CONF_ENCRYPTION,
CONF_ESPHOME,
CONF_ID,
CONF_KEY,
CONF_NUM_ATTEMPTS,
CONF_OTA,
CONF_PASSWORD,
@@ -15,6 +23,7 @@ from esphome.const import (
CONF_REBOOT_TIMEOUT,
CONF_SAFE_MODE,
CONF_VERSION,
CONF_WEB_SERVER,
)
from esphome.core import CORE, coroutine_with_priority
from esphome.coroutine import CoroPriority
@@ -22,6 +31,7 @@ import esphome.final_validate as fv
from esphome.types import ConfigType
CONF_ALLOW_PARTITION_ACCESS = "allow_partition_access"
CONF_CAPTIVE_PORTAL = "captive_portal"
_LOGGER = logging.getLogger(__name__)
@@ -30,7 +40,15 @@ CODEOWNERS = ["@esphome/core"]
DEPENDENCIES = ["network"]
AUTO_LOAD = ["sha256", "socket"]
def AUTO_LOAD(config: ConfigType) -> list[str]:
"""Auto-load noise only when encryption is configured."""
base = ["sha256", "socket"]
# A falsy config is a tooling probe for the maximal set (None from
# dependency resolution, {} from the components-graph platform probe);
# a validated config always carries defaults, never empty
if not config or CONF_ENCRYPTION in config:
return base + ["noise"]
return base
esphome = cg.esphome_ns.namespace("esphome")
@@ -67,11 +85,24 @@ def ota_esphome_final_validate(config: ConfigType) -> None:
CONF_PASSWORD in merged_ota_esphome_configs_by_port[conf_port]
and CONF_PASSWORD in ota_conf
and merged_ota_esphome_configs_by_port[conf_port][CONF_PASSWORD]
!= ota_conf.get(CONF_PASSWORD)
!= ota_conf[CONF_PASSWORD]
):
raise cv.Invalid(
f"Found multiple configurations but {CONF_PASSWORD} is inconsistent"
)
# Encryption blocks conflict only when both pin a key; a bare
# `encryption:` (a package/device split) is compatible with a
# keyed one, and merge_config yields the keyed result
merged_key = (
merged_ota_esphome_configs_by_port[conf_port]
.get(CONF_ENCRYPTION, {})
.get(CONF_KEY)
)
other_key = ota_conf.get(CONF_ENCRYPTION, {}).get(CONF_KEY)
if merged_key and other_key and merged_key != other_key:
raise cv.Invalid(
f"Found multiple configurations but {CONF_ENCRYPTION} is inconsistent"
)
ports_with_merged_configs.append(conf_port)
merged_ota_esphome_configs_by_port[conf_port] = merge_config(
@@ -94,6 +125,20 @@ def ota_esphome_final_validate(config: ConfigType) -> None:
new_ota_conf.extend(merged_ota_esphome_configs_by_port.values())
api_conf = full_conf.get(CONF_API) or {}
for ota_conf in merged_ota_esphome_configs_by_port.values():
# Merging same-port blocks can combine a password from one block with
# encryption from another; re-check the exclusion on the merged result.
_validate_no_password_with_encryption(ota_conf)
if (encryption_conf := ota_conf.get(CONF_ENCRYPTION)) is not None:
_resolve_encryption_key(encryption_conf, api_conf)
if any(
conf.get(CONF_PLATFORM) == CONF_WEB_SERVER for conf in full_ota_conf
) and any(
CONF_ENCRYPTION in conf for conf in merged_ota_esphome_configs_by_port.values()
):
_warn_web_server_ota(full_conf)
full_conf[CONF_OTA] = new_ota_conf
fv.full_config.set(full_conf)
@@ -107,6 +152,73 @@ def ota_esphome_final_validate(config: ConfigType) -> None:
)
def _warn_web_server_ota(full_conf: ConfigType) -> None:
"""The web_server ota platform accepts the same image over plaintext HTTP
with basic auth, bypassing the encryption; warn rather than fail so the
operator keeps the recovery path."""
if CONF_CAPTIVE_PORTAL in full_conf and CONF_WEB_SERVER not in full_conf:
# The captive_portal auto-load: the endpoint only exists while the
# fallback AP is active
_LOGGER.warning(
"OTA encryption does not cover the %s OTA platform (auto-loaded "
"by captive_portal); the plaintext /update endpoint stays "
"reachable while the fallback AP is active",
CONF_WEB_SERVER,
)
else:
_LOGGER.warning(
"OTA encryption does not cover the %s OTA platform; its "
"plaintext /update endpoint accepts the same image",
CONF_WEB_SERVER,
)
def _resolve_encryption_key(encryption_conf: ConfigType, api_conf: ConfigType) -> None:
"""Resolve the one encryption key per device into the ota block.
An explicit ota key must match the api key, a bare block inherits it,
a runtime provisioned api key cannot be inherited, and the all-zeros
provisioning sentinel is rejected (the device treats it as no key).
"""
api_key = api_conf.get(CONF_ENCRYPTION, {}).get(CONF_KEY)
if ota_key := encryption_conf.get(CONF_KEY):
if api_key and ota_key != api_key:
raise cv.Invalid(
f"'{CONF_OTA}' {CONF_ENCRYPTION} {CONF_KEY} must match the "
f"'{CONF_API}' {CONF_ENCRYPTION} {CONF_KEY}; omit the "
f"'{CONF_OTA}' {CONF_KEY} to use the '{CONF_API}' one"
)
elif not api_key:
if CONF_ENCRYPTION in api_conf:
raise cv.Invalid(
f"the '{CONF_API}' {CONF_ENCRYPTION} {CONF_KEY} is provisioned at "
f"runtime and cannot be inherited at build time; set an explicit "
f"'{CONF_OTA}' {CONF_ENCRYPTION} {CONF_KEY}"
)
raise cv.Invalid(
f"'{CONF_OTA}' {CONF_ENCRYPTION} has no {CONF_KEY} and there is no "
f"'{CONF_API}' {CONF_ENCRYPTION} {CONF_KEY} to inherit; set one of them"
)
else:
encryption_conf[CONF_KEY] = api_key
if is_reserved_key(encryption_conf[CONF_KEY]):
raise cv.Invalid(
f"The all-zeros {CONF_KEY} is reserved and provides no protection; "
f"generate a real key with: openssl rand -base64 32"
)
# Also called on merged same-port configs in final validate, where schemas
# do not run
def _validate_no_password_with_encryption(config: ConfigType) -> ConfigType:
if CONF_PASSWORD in config and CONF_ENCRYPTION in config:
raise cv.Invalid(
f"'{CONF_PASSWORD}' cannot be combined with '{CONF_ENCRYPTION}'; the "
f"encryption key already authenticates the uploader, remove '{CONF_PASSWORD}'"
)
return config
def _consume_ota_sockets(config: ConfigType) -> ConfigType:
"""Register socket needs for OTA component."""
from esphome.components import socket
@@ -134,6 +246,7 @@ CONFIG_SCHEMA = cv.All(
): cv.port,
cv.Optional(CONF_ALLOW_PARTITION_ACCESS, default=False): cv.boolean,
cv.Optional(CONF_PASSWORD): cv.sensitive(),
cv.Optional(CONF_ENCRYPTION): encryption_schema,
cv.Optional(CONF_NUM_ATTEMPTS): cv.invalid(
f"'{CONF_SAFE_MODE}' (and its related configuration variables) has moved from 'ota' to its own component. See https://esphome.io/components/safe_mode"
),
@@ -147,12 +260,24 @@ CONFIG_SCHEMA = cv.All(
)
.extend(BASE_OTA_SCHEMA)
.extend(cv.COMPONENT_SCHEMA),
_validate_no_password_with_encryption,
_consume_ota_sockets,
)
FINAL_VALIDATE_SCHEMA = ota_esphome_final_validate
def FILTER_SOURCE_FILES() -> list[str]:
"""Filter out the noise transport when no ota entry configures encryption."""
for ota_conf in CORE.config.get(CONF_OTA, []):
if (
ota_conf.get(CONF_PLATFORM) == CONF_ESPHOME
and ota_conf.get(CONF_ENCRYPTION) is not None
):
return []
return ["ota_esphome_noise.cpp"]
@coroutine_with_priority(CoroPriority.OTA_UPDATES)
async def to_code(config: ConfigType) -> None:
var = cg.new_Pvariable(config[CONF_ID])
@@ -171,6 +296,12 @@ async def to_code(config: ConfigType) -> None:
if config.get(CONF_ALLOW_PARTITION_ACCESS):
cg.add_define("USE_OTA_PARTITIONS")
if (encryption_conf := config.get(CONF_ENCRYPTION)) is not None:
# A missing key was resolved from the api component in final validate.
key = encryption_conf[CONF_KEY]
cg.add_define("USE_OTA_ENCRYPTION")
cg.add(var.set_noise_psk(list(decode_encryption_key(key))))
# Build flag so lwip_fast_select.c (a .c file that can't include defines.h) sees it.
cg.add_build_flag("-DUSE_OTA_PLATFORM_ESPHOME")
+95 -26
View File
@@ -27,7 +27,6 @@ namespace esphome {
static const char *const TAG = "esphome.ota";
static constexpr uint16_t OTA_BLOCK_SIZE = 8192;
static constexpr size_t OTA_BUFFER_SIZE = 1024; // buffer size for OTA data transfer
static constexpr uint32_t OTA_SOCKET_TIMEOUT_HANDSHAKE = 20000; // milliseconds for initial handshake
static constexpr uint32_t OTA_SOCKET_TIMEOUT_DATA = 90000; // milliseconds for data transfer
@@ -105,6 +104,11 @@ void ESPHomeOTAComponent::dump_config() {
ESP_LOGCONFIG(TAG, " Password configured");
}
#endif
#ifdef USE_OTA_ENCRYPTION
if (this->noise_ctx_.has_psk()) {
ESP_LOGCONFIG(TAG, " Encryption configured");
}
#endif
#ifdef USE_OTA_PARTITIONS
ESP_LOGCONFIG(TAG,
" Partition access allowed\n"
@@ -149,8 +153,10 @@ void ESPHomeOTAComponent::loop() {
static constexpr uint8_t CLIENT_FEATURE_SUPPORTS_COMPRESSION = 0x01;
static constexpr uint8_t CLIENT_FEATURE_SUPPORTS_SHA256_AUTH = 0x02;
static constexpr uint8_t CLIENT_FEATURE_SUPPORTS_EXTENDED_PROTOCOL = 0x04;
static constexpr uint8_t CLIENT_FEATURE_SUPPORTS_NOISE = 0x08;
static constexpr uint8_t SERVER_FEATURE_SUPPORTS_COMPRESSION = 0x01;
static constexpr uint8_t SERVER_FEATURE_SUPPORTS_PARTITION_ACCESS = 0x02;
static constexpr uint8_t SERVER_FEATURE_SUPPORTS_NOISE = 0x04;
void ESPHomeOTAComponent::handle_handshake_() {
/// Handle the OTA handshake and authentication.
@@ -202,8 +208,7 @@ void ESPHomeOTAComponent::handle_handshake_() {
}
// Validate magic bytes
static const uint8_t MAGIC_BYTES[5] = {0x6C, 0x26, 0xF7, 0x5C, 0x45};
if (memcmp(this->handshake_buf_, MAGIC_BYTES, 5) != 0) {
if (memcmp(this->handshake_buf_, MAGIC_BYTES, sizeof(MAGIC_BYTES)) != 0) {
ESP_LOGW(TAG, "Magic bytes mismatch! 0x%02X-0x%02X-0x%02X-0x%02X-0x%02X", this->handshake_buf_[0],
this->handshake_buf_[1], this->handshake_buf_[2], this->handshake_buf_[3], this->handshake_buf_[4]);
this->send_error_and_cleanup_(ota::OTA_RESPONSE_ERROR_MAGIC);
@@ -235,6 +240,19 @@ void ESPHomeOTAComponent::handle_handshake_() {
}
this->ota_features_ = this->handshake_buf_[0];
ESP_LOGV(TAG, "Features: 0x%02X", this->ota_features_);
#ifdef USE_OTA_ENCRYPTION
// Fail closed: with a PSK configured the client must negotiate encryption
// (which requires the extended protocol); refuse plaintext uploads.
static constexpr uint8_t NOISE_REQUIRED_FEATURES =
CLIENT_FEATURE_SUPPORTS_NOISE | CLIENT_FEATURE_SUPPORTS_EXTENDED_PROTOCOL;
if (this->noise_ctx_.has_psk() && (this->ota_features_ & NOISE_REQUIRED_FEATURES) != NOISE_REQUIRED_FEATURES) {
ESP_LOGW(TAG, "Client does not support encryption");
this->send_error_and_cleanup_(ota::OTA_RESPONSE_ERROR_ENCRYPTION_REQUIRED);
return;
}
#endif
this->transition_ota_state_(OTAState::FEATURE_ACK);
const bool supports_compression =
@@ -250,6 +268,11 @@ void ESPHomeOTAComponent::handle_handshake_() {
this->handshake_buf_[1] = (supports_compression ? SERVER_FEATURE_SUPPORTS_COMPRESSION : 0);
#ifdef USE_OTA_PARTITIONS
this->handshake_buf_[1] |= SERVER_FEATURE_SUPPORTS_PARTITION_ACCESS;
#endif
#ifdef USE_OTA_ENCRYPTION
if (this->noise_ctx_.has_psk()) {
this->handshake_buf_[1] |= SERVER_FEATURE_SUPPORTS_NOISE;
}
#endif
} else {
this->handshake_buf_[0] =
@@ -265,6 +288,20 @@ void ESPHomeOTAComponent::handle_handshake_() {
if (!this->try_write_(ack_size, LOG_STR("ack feature"))) {
return;
}
#ifdef USE_OTA_ENCRYPTION
// With a PSK configured the rest of the session runs inside the noise
// transport; the client sends the first handshake frame next, so there
// is nothing to do until data arrives.
if (this->noise_ctx_.has_psk()) {
// handshake_buf_ still holds the feature ack composed above; a
// would-block re-entry lands here without rebuilding it
if (!this->noise_start_session_(this->handshake_buf_[1])) {
return;
}
this->transition_ota_state_(OTAState::NOISE_HANDSHAKE);
return;
}
#endif
#ifdef USE_OTA_PASSWORD
// If password is set, move to auth phase
if (!this->password_.empty()) {
@@ -302,6 +339,16 @@ void ESPHomeOTAComponent::handle_handshake_() {
this->handle_data_();
return;
#ifdef USE_OTA_ENCRYPTION
case OTAState::NOISE_HANDSHAKE:
if (!this->handle_noise_handshake_()) {
return;
}
this->transition_ota_state_(OTAState::DATA);
this->handle_data_();
return;
#endif
default:
break;
}
@@ -340,6 +387,8 @@ void ESPHomeOTAComponent::handle_data_() {
/// Raw TCP (8266, RP2040): setblocking is no-op; SO_RCVTIMEO uses
/// wakeable_delay() in read();
/// write() always returns immediately
// Backend calls overwrite this with OK; reset to UNKNOWN before any
// goto error that follows a successful begin()/write()
ota::OTAResponseTypes error_code = ota::OTA_RESPONSE_ERROR_UNKNOWN;
size_t total = 0;
uint32_t last_progress = 0;
@@ -361,11 +410,11 @@ void ESPHomeOTAComponent::handle_data_() {
this->client_->setblocking(true);
// Acknowledge auth OK - 1 byte
this->write_byte_(ota::OTA_RESPONSE_AUTH_OK);
this->data_write_byte_(ota::OTA_RESPONSE_AUTH_OK);
if (this->extended_proto_) {
// Read ota type, 1 byte
if (!this->readall_(buf, 1)) {
if (!this->data_readall_(buf, 1)) {
this->log_read_error_(LOG_STR("OTA type"));
goto error; // NOLINT(cppcoreguidelines-avoid-goto)
}
@@ -374,7 +423,7 @@ void ESPHomeOTAComponent::handle_data_() {
ESP_LOGV(TAG, "OTA type is 0x%02x", ota_type);
// Read size, 4 bytes MSB first
if (!this->readall_(buf, 4)) {
if (!this->data_readall_(buf, 4)) {
this->log_read_error_(LOG_STR("size"));
goto error; // NOLINT(cppcoreguidelines-avoid-goto)
}
@@ -405,11 +454,12 @@ void ESPHomeOTAComponent::handle_data_() {
goto error; // NOLINT(cppcoreguidelines-avoid-goto)
// Acknowledge prepare OK - 1 byte
this->write_byte_(ota::OTA_RESPONSE_UPDATE_PREPARE_OK);
this->data_write_byte_(ota::OTA_RESPONSE_UPDATE_PREPARE_OK);
// Read binary MD5, 32 bytes
if (!this->readall_(buf, 32)) {
if (!this->data_readall_(buf, 32)) {
this->log_read_error_(LOG_STR("MD5 checksum"));
error_code = ota::OTA_RESPONSE_ERROR_UNKNOWN;
goto error; // NOLINT(cppcoreguidelines-avoid-goto)
}
sbuf[32] = '\0';
@@ -417,7 +467,7 @@ void ESPHomeOTAComponent::handle_data_() {
this->backend_->set_update_md5(sbuf);
// Acknowledge MD5 OK - 1 byte
this->write_byte_(ota::OTA_RESPONSE_BIN_MD5_OK);
this->data_write_byte_(ota::OTA_RESPONSE_BIN_MD5_OK);
// Track when we last received data so a silently-vanished peer (no FIN/RST
// delivered, e.g. uploader killed mid-transfer or NAT/router dropped state)
@@ -433,19 +483,35 @@ void ESPHomeOTAComponent::handle_data_() {
}
size_t remaining = ota_size - total;
size_t requested = remaining < OTA_BUFFER_SIZE ? remaining : OTA_BUFFER_SIZE;
ssize_t read = this->client_->read(buf, requested);
if (read == -1) {
const int err = errno;
if (this->would_block_(err)) {
// read() already waited up to SO_RCVTIMEO for data, just feed WDT
App.feed_wdt();
continue;
ssize_t read;
#ifdef USE_OTA_ENCRYPTION
if (this->noise_ != nullptr) {
// One frame per call; noise_read_data_ waits internally (readall_), so
// there is no would-block retry here and failures are already logged.
read = this->noise_read_data_(buf, requested);
if (read <= 0) {
error_code = ota::OTA_RESPONSE_ERROR_UNKNOWN;
goto error; // NOLINT(cppcoreguidelines-avoid-goto)
}
} else
#endif
{
read = this->client_->read(buf, requested);
if (read == -1) {
const int err = errno;
if (this->would_block_(err)) {
// read() already waited up to SO_RCVTIMEO for data, just feed WDT
App.feed_wdt();
continue;
}
ESP_LOGW(TAG, "Read err %d", err);
error_code = ota::OTA_RESPONSE_ERROR_UNKNOWN;
goto error; // NOLINT(cppcoreguidelines-avoid-goto)
} else if (read == 0) {
ESP_LOGW(TAG, "Remote closed");
error_code = ota::OTA_RESPONSE_ERROR_UNKNOWN;
goto error; // NOLINT(cppcoreguidelines-avoid-goto)
}
ESP_LOGW(TAG, "Read err %d", err);
goto error; // NOLINT(cppcoreguidelines-avoid-goto)
} else if (read == 0) {
ESP_LOGW(TAG, "Remote closed");
goto error; // NOLINT(cppcoreguidelines-avoid-goto)
}
last_data_ms = millis();
@@ -457,7 +523,7 @@ void ESPHomeOTAComponent::handle_data_() {
total += read;
#if USE_OTA_VERSION == 2
while (size_acknowledged + OTA_BLOCK_SIZE <= total || (total == ota_size && size_acknowledged < ota_size)) {
this->write_byte_(ota::OTA_RESPONSE_CHUNK_OK);
this->data_write_byte_(ota::OTA_RESPONSE_CHUNK_OK);
size_acknowledged += OTA_BLOCK_SIZE;
}
#endif
@@ -476,7 +542,7 @@ void ESPHomeOTAComponent::handle_data_() {
}
// Acknowledge receive OK - 1 byte
this->write_byte_(ota::OTA_RESPONSE_RECEIVE_OK);
this->data_write_byte_(ota::OTA_RESPONSE_RECEIVE_OK);
error_code = this->backend_->end();
if (error_code != ota::OTA_RESPONSE_OK) {
@@ -485,10 +551,10 @@ void ESPHomeOTAComponent::handle_data_() {
}
// Acknowledge Update end OK - 1 byte
this->write_byte_(ota::OTA_RESPONSE_UPDATE_END_OK);
this->data_write_byte_(ota::OTA_RESPONSE_UPDATE_END_OK);
// Read ACK
if (!this->readall_(buf, 1) || buf[0] != ota::OTA_RESPONSE_OK) {
if (!this->data_readall_(buf, 1) || buf[0] != ota::OTA_RESPONSE_OK) {
this->log_read_error_(LOG_STR("ack"));
// do not go to error, this is not fatal
}
@@ -511,7 +577,7 @@ void ESPHomeOTAComponent::handle_data_() {
App.safe_reboot();
error:
this->write_byte_(static_cast<uint8_t>(error_code));
this->data_write_byte_(static_cast<uint8_t>(error_code));
// Abort backend before cleanup - cleanup_connection_() destroys the backend.
// Always call abort() unconditionally: backends register external partitions before
@@ -678,6 +744,9 @@ void ESPHomeOTAComponent::cleanup_connection_() {
this->backend_ = nullptr;
#ifdef USE_OTA_PASSWORD
this->cleanup_auth_();
#endif
#ifdef USE_OTA_ENCRYPTION
this->noise_ = nullptr;
#endif
// Intentionally no disable_loop() — letting loop() run one more iteration catches
// any connection that queued on the listener mid-session (otherwise the wake flag,
+69 -1
View File
@@ -4,6 +4,9 @@
#ifdef USE_OTA
#include "esphome/components/ota/ota_backend_factory.h"
#include "esphome/components/socket/socket.h"
#ifdef USE_OTA_ENCRYPTION
#include "esphome/components/noise/noise_handshake.h"
#endif
#include "esphome/core/helpers.h"
#include "esphome/core/log.h"
#include "esphome/core/preferences.h"
@@ -24,7 +27,10 @@ class ESPHomeOTAComponent final : public ota::OTAComponent {
AUTH_SEND, // Sending authentication request
AUTH_READ, // Reading authentication data
#endif // USE_OTA_PASSWORD
DATA, // BLOCKING! Processing OTA data (update, etc.)
#ifdef USE_OTA_ENCRYPTION
NOISE_HANDSHAKE, // Exchanging Noise handshake frames
#endif
DATA, // BLOCKING! Processing OTA data (update, etc.)
};
#ifdef USE_OTA_PASSWORD
void set_auth_password(const std::string &password) { password_ = password; }
@@ -38,6 +44,10 @@ class ESPHomeOTAComponent final : public ota::OTAComponent {
}
#endif // USE_OTA_PASSWORD
#ifdef USE_OTA_ENCRYPTION
void set_noise_psk(noise::psk_t psk) { this->noise_ctx_.set_psk(psk); }
#endif
/// Manually set the port OTA should listen on
void set_port(uint16_t port) { this->port_ = port; }
@@ -63,6 +73,48 @@ class ESPHomeOTAComponent final : public ota::OTAComponent {
bool writeall_(const uint8_t *buf, size_t len);
inline bool write_byte_(uint8_t byte) { return this->writeall_(&byte, 1); }
#ifdef USE_OTA_ENCRYPTION
// Heap-allocated only while an encrypted OTA session is active.
struct NoiseSession {
~NoiseSession();
noise::NoiseResponderHandshake handshake;
NoiseCipherState *send_cipher{nullptr};
NoiseCipherState *recv_cipher{nullptr};
uint16_t frame_len{0}; // total frame size once the header is parsed, 0 until then
uint16_t frame_pos{0}; // bytes read or written so far
bool writing{false}; // a produced handshake frame is still being flushed
uint8_t frame_buf[noise::FRAME_HEADER_SIZE + 1 + noise::MAX_HANDSHAKE_SIZE];
};
bool noise_start_session_(uint8_t server_feature_flags);
bool handle_noise_handshake_();
bool noise_try_read_frame_();
bool noise_try_write_frame_();
void noise_send_reject_(const LogString *reason);
ssize_t noise_decrypt_(uint8_t *buf, size_t len);
ssize_t noise_read_frame_blocking_(uint8_t *buf, size_t min_ciphertext, size_t max_ciphertext);
bool noise_readall_(uint8_t *buf, size_t len);
ssize_t noise_read_data_(uint8_t *buf, size_t capacity);
bool noise_write_byte_(uint8_t byte);
#endif // USE_OTA_ENCRYPTION
// Data-phase I/O dispatch: through the noise transport when a session is
// active, straight to the socket otherwise.
inline bool data_write_byte_(uint8_t byte) {
#ifdef USE_OTA_ENCRYPTION
if (this->noise_ != nullptr)
return this->noise_write_byte_(byte);
#endif
return this->write_byte_(byte);
}
// When encrypted, buf must have room for len + noise::MAC_SIZE bytes.
inline bool data_readall_(uint8_t *buf, size_t len) {
#ifdef USE_OTA_ENCRYPTION
if (this->noise_ != nullptr)
return this->noise_readall_(buf, len);
#endif
return this->readall_(buf, len);
}
bool try_read_(size_t to_read, const LogString *desc);
bool try_write_(size_t to_write, const LogString *desc);
@@ -91,6 +143,10 @@ class ESPHomeOTAComponent final : public ota::OTAComponent {
std::string password_;
std::unique_ptr<uint8_t[]> auth_buf_;
#endif // USE_OTA_PASSWORD
#ifdef USE_OTA_ENCRYPTION
noise::NoiseContext noise_ctx_;
std::unique_ptr<NoiseSession> noise_;
#endif // USE_OTA_ENCRYPTION
socket::ListenSocket *server_{nullptr};
std::unique_ptr<socket::Socket> client_;
@@ -98,6 +154,18 @@ class ESPHomeOTAComponent final : public ota::OTAComponent {
uint32_t client_connect_time_{0};
static constexpr size_t HANDSHAKE_BUF_SIZE = 5;
// Buffer size for OTA data transfer. The upload client derives its maximum
// encrypted frame plaintext from this (espota2.NOISE_MAX_PLAINTEXT is this
// minus the 16-byte MAC); both must change together.
static constexpr size_t OTA_BUFFER_SIZE = 1040;
#ifdef USE_OTA_ENCRYPTION
// espota2.NOISE_MAX_PLAINTEXT; shrinking the buffer would reject every
// frame a current CLI sends
static constexpr size_t NOISE_CLIENT_MAX_PLAINTEXT = 1024;
static_assert(OTA_BUFFER_SIZE >= NOISE_CLIENT_MAX_PLAINTEXT + noise::MAC_SIZE,
"OTA_BUFFER_SIZE must fit a full encrypted data frame");
#endif
static constexpr uint8_t MAGIC_BYTES[5] = {0x6C, 0x26, 0xF7, 0x5C, 0x45};
#ifdef USE_OTA_PARTITIONS
uint32_t running_app_offset_{0};
size_t running_app_size_{0};
@@ -0,0 +1,279 @@
#include "ota_esphome.h"
#ifdef USE_OTA
#ifdef USE_OTA_ENCRYPTION
#include "esphome/components/noise/noise.h"
#include "esphome/components/ota/ota_backend.h"
#include "esphome/core/log.h"
#include <cstring>
#include <new>
#ifdef USE_ESP8266
#include <pgmspace.h>
#endif
namespace esphome {
static const char *const TAG = "esphome.ota";
#ifdef USE_ESP8266
static constexpr char OTA_NOISE_PROLOGUE_INIT[] PROGMEM = "NoiseOTAInit";
#else
static constexpr char OTA_NOISE_PROLOGUE_INIT[] = "NoiseOTAInit";
#endif
static constexpr size_t OTA_NOISE_PROLOGUE_INIT_LEN = sizeof(OTA_NOISE_PROLOGUE_INIT) - 1;
ESPHomeOTAComponent::NoiseSession::~NoiseSession() {
if (this->send_cipher != nullptr) {
noise_cipherstate_free(this->send_cipher);
}
if (this->recv_cipher != nullptr) {
noise_cipherstate_free(this->recv_cipher);
}
}
/** Allocate the session and start the responder handshake.
*
* The prologue binds the whole plaintext preamble, so any tampering with the
* negotiation (a stripped feature flag, a changed version) breaks the first
* handshake MAC on either side:
* "NoiseOTAInit" | magic(5) | OK,version | client_features | FEATURE_FLAGS,server_flags
*/
bool ESPHomeOTAComponent::noise_start_session_(uint8_t server_feature_flags) {
// NOLINTNEXTLINE(clang-analyzer-cplusplus.NewDeleteLeaks)
this->noise_ = std::unique_ptr<NoiseSession>(new (std::nothrow) NoiseSession());
if (this->noise_ == nullptr) {
ESP_LOGW(TAG, "Session allocation failed");
this->cleanup_connection_();
return false;
}
static constexpr size_t PROLOGUE_ACK_LEN = 2; // OTA_RESPONSE_OK + version
static constexpr size_t PROLOGUE_CLIENT_FEATURES_LEN = 1;
static constexpr size_t PROLOGUE_FEATURE_ACK_LEN = 2; // OTA_RESPONSE_FEATURE_FLAGS + server flags
uint8_t prologue[OTA_NOISE_PROLOGUE_INIT_LEN + sizeof(MAGIC_BYTES) + PROLOGUE_ACK_LEN + PROLOGUE_CLIENT_FEATURES_LEN +
PROLOGUE_FEATURE_ACK_LEN];
#ifdef USE_ESP8266
memcpy_P(prologue, OTA_NOISE_PROLOGUE_INIT, OTA_NOISE_PROLOGUE_INIT_LEN);
#else
std::memcpy(prologue, OTA_NOISE_PROLOGUE_INIT, OTA_NOISE_PROLOGUE_INIT_LEN);
#endif
uint8_t *p = prologue + OTA_NOISE_PROLOGUE_INIT_LEN;
// Magic bytes, already validated in MAGIC_READ
std::memcpy(p, MAGIC_BYTES, sizeof(MAGIC_BYTES));
p += sizeof(MAGIC_BYTES);
// Our magic ack
*p++ = ota::OTA_RESPONSE_OK;
*p++ = USE_OTA_VERSION;
// The feature byte the client sent
*p++ = this->ota_features_;
// The feature ack we sent (noise requires the extended protocol)
*p++ = ota::OTA_RESPONSE_FEATURE_FLAGS;
*p++ = server_feature_flags;
int err = this->noise_->handshake.init(this->noise_ctx_.get_psk(), prologue, sizeof(prologue));
if (err != 0) {
ESP_LOGW(TAG, "Handshake init: %s", LOG_STR_ARG(noise::noise_err_to_logstr(err)));
this->cleanup_connection_();
return false;
}
return true;
}
/** Drive the non-blocking handshake from loop(); returns true once the
* transport ciphers are ready. A would-block returns false and the next
* loop() resumes from the NoiseSession cursors; on failure the connection
* is cleaned up.
*/
bool ESPHomeOTAComponent::handle_noise_handshake_() {
NoiseSession &s = *this->noise_;
while (true) {
if (s.writing) {
if (!this->noise_try_write_frame_()) {
return false; // would block, or errored and cleaned up
}
s.writing = false;
s.frame_pos = 0;
s.frame_len = 0;
}
switch (s.handshake.action()) {
case noise::NoiseResponderHandshake::Action::ACTION_READ: {
if (!this->noise_try_read_frame_()) {
return false;
}
const uint16_t payload_len = s.frame_len - noise::FRAME_HEADER_SIZE;
s.frame_pos = 0;
s.frame_len = 0;
if (s.frame_buf[noise::FRAME_HEADER_SIZE] != noise::HANDSHAKE_STATUS_OK) {
ESP_LOGW(TAG, "Bad handshake error byte: %u", s.frame_buf[noise::FRAME_HEADER_SIZE]);
this->cleanup_connection_();
return false;
}
int err = s.handshake.read_message(s.frame_buf + noise::FRAME_HEADER_SIZE + 1, payload_len - 1);
if (err != 0) {
ESP_LOGW(TAG, "Handshake read: %s", LOG_STR_ARG(noise::noise_err_to_logstr(err)));
this->noise_send_reject_(noise::reject_reason_for(err));
this->cleanup_connection_();
return false;
}
break;
}
case noise::NoiseResponderHandshake::Action::ACTION_WRITE: {
size_t msg_len = 0;
int err =
s.handshake.write_message(s.frame_buf + noise::FRAME_HEADER_SIZE + 1, noise::MAX_HANDSHAKE_SIZE, msg_len);
if (err != 0) {
ESP_LOGW(TAG, "Handshake write: %s", LOG_STR_ARG(noise::noise_err_to_logstr(err)));
this->cleanup_connection_();
return false;
}
const uint16_t payload_len = msg_len + 1;
noise::write_frame_header(s.frame_buf, payload_len);
s.frame_buf[noise::FRAME_HEADER_SIZE] = noise::HANDSHAKE_STATUS_OK;
s.frame_len = noise::FRAME_HEADER_SIZE + payload_len;
s.frame_pos = 0;
s.writing = true;
break;
}
case noise::NoiseResponderHandshake::Action::ACTION_SPLIT: {
int err = s.handshake.split(s.send_cipher, s.recv_cipher);
if (err != 0) {
ESP_LOGW(TAG, "Handshake split: %s", LOG_STR_ARG(noise::noise_err_to_logstr(err)));
this->cleanup_connection_();
return false;
}
ESP_LOGD(TAG, "Noise handshake complete");
return true;
}
default: {
ESP_LOGW(TAG, "Bad handshake state");
this->cleanup_connection_();
return false;
}
}
}
}
/// Non-blocking read of one handshake frame into the session buffer.
bool ESPHomeOTAComponent::noise_try_read_frame_() {
NoiseSession &s = *this->noise_;
while (s.frame_pos < noise::FRAME_HEADER_SIZE) {
ssize_t read = this->client_->read(s.frame_buf + s.frame_pos, noise::FRAME_HEADER_SIZE - s.frame_pos);
if (!this->handle_read_error_(read, LOG_STR("read noise header"))) {
return false;
}
s.frame_pos += read;
}
if (s.frame_len == 0) {
const uint16_t payload_len = encode_uint16(s.frame_buf[1], s.frame_buf[2]);
if (s.frame_buf[0] != noise::FRAME_INDICATOR || payload_len < 1 || payload_len > 1 + noise::MAX_HANDSHAKE_SIZE) {
ESP_LOGW(TAG, "Bad handshake frame: 0x%02X, %u bytes", s.frame_buf[0], payload_len);
this->cleanup_connection_();
return false;
}
s.frame_len = noise::FRAME_HEADER_SIZE + payload_len;
}
while (s.frame_pos < s.frame_len) {
ssize_t read = this->client_->read(s.frame_buf + s.frame_pos, s.frame_len - s.frame_pos);
if (!this->handle_read_error_(read, LOG_STR("read noise frame"))) {
return false;
}
s.frame_pos += read;
}
return true;
}
/// Non-blocking write of the pending session-buffer frame.
bool ESPHomeOTAComponent::noise_try_write_frame_() {
NoiseSession &s = *this->noise_;
while (s.frame_pos < s.frame_len) {
ssize_t written = this->client_->write(s.frame_buf + s.frame_pos, s.frame_len - s.frame_pos);
if (!this->handle_write_error_(written, LOG_STR("write noise frame"))) {
return false;
}
s.frame_pos += written;
}
return true;
}
/// Best-effort explicit reject frame so the client can log a readable reason.
void ESPHomeOTAComponent::noise_send_reject_(const LogString *reason) {
// Every reason here comes from noise::reject_reason_for(), so the exported
// floor is the exact capacity needed
uint8_t data[noise::FRAME_HEADER_SIZE + noise::MAC_FAILURE_PAYLOAD_SIZE];
const size_t payload_len =
noise::format_reject_payload(data + noise::FRAME_HEADER_SIZE, sizeof(data) - noise::FRAME_HEADER_SIZE, reason);
noise::write_frame_header(data, payload_len);
this->client_->write(data, noise::FRAME_HEADER_SIZE + payload_len); // Best effort, non-blocking
}
/// Decrypt a ciphertext in place; returns the plaintext size or -1.
ssize_t ESPHomeOTAComponent::noise_decrypt_(uint8_t *buf, size_t len) {
NoiseBuffer mbuf;
noise_buffer_init(mbuf);
noise_buffer_set_inout(mbuf, buf, len, len);
int err = noise_cipherstate_decrypt(this->noise_->recv_cipher, &mbuf);
if (err != 0) {
ESP_LOGW(TAG, "Decrypt: %s", LOG_STR_ARG(noise::noise_err_to_logstr(err)));
return -1;
}
return mbuf.size;
}
/** Blocking read of one frame whose ciphertext size must be within the given
* bounds, decrypted in place; returns the plaintext size, or -1 on error.
* buf needs max_ciphertext capacity.
*/
ssize_t ESPHomeOTAComponent::noise_read_frame_blocking_(uint8_t *buf, size_t min_ciphertext, size_t max_ciphertext) {
uint8_t header[noise::FRAME_HEADER_SIZE];
if (!this->readall_(header, sizeof(header))) {
return -1;
}
const size_t ciphertext_len = encode_uint16(header[1], header[2]);
if (header[0] != noise::FRAME_INDICATOR || ciphertext_len < min_ciphertext || ciphertext_len > max_ciphertext) {
ESP_LOGW(TAG, "Bad frame: 0x%02X, %zu bytes", header[0], ciphertext_len);
return -1;
}
if (!this->readall_(buf, ciphertext_len)) {
return -1;
}
return this->noise_decrypt_(buf, ciphertext_len);
}
/** Blocking read of one frame whose plaintext must be exactly len bytes
* (control units are one unit per frame). buf needs len + noise::MAC_SIZE
* capacity; the plaintext lands at buf[0..len).
*/
bool ESPHomeOTAComponent::noise_readall_(uint8_t *buf, size_t len) {
return this->noise_read_frame_blocking_(buf, len + noise::MAC_SIZE, len + noise::MAC_SIZE) == (ssize_t) len;
}
/** Blocking read of one data-phase frame, decrypted in place; returns the
* plaintext size, or -1 on error. buf is the OTA_BUFFER_SIZE data buffer.
* The ciphertext must fit that buffer and its plaintext must fit what the
* caller accepts (the remaining image bytes).
*/
ssize_t ESPHomeOTAComponent::noise_read_data_(uint8_t *buf, size_t capacity) {
const size_t max_ciphertext = std::min(capacity + noise::MAC_SIZE, OTA_BUFFER_SIZE);
return this->noise_read_frame_blocking_(buf, noise::MAC_SIZE + 1, max_ciphertext);
}
/// Blocking write of one response byte as an encrypted frame.
bool ESPHomeOTAComponent::noise_write_byte_(uint8_t byte) {
uint8_t frame[noise::FRAME_HEADER_SIZE + 1 + noise::MAC_SIZE];
frame[noise::FRAME_HEADER_SIZE] = byte;
NoiseBuffer mbuf;
noise_buffer_init(mbuf);
noise_buffer_set_inout(mbuf, frame + noise::FRAME_HEADER_SIZE, 1, 1 + noise::MAC_SIZE);
int err = noise_cipherstate_encrypt(this->noise_->send_cipher, &mbuf);
if (err != 0) {
ESP_LOGW(TAG, "Encrypt: %s", LOG_STR_ARG(noise::noise_err_to_logstr(err)));
return false;
}
noise::write_frame_header(frame, mbuf.size);
return this->writeall_(frame, noise::FRAME_HEADER_SIZE + mbuf.size);
}
} // namespace esphome
#endif // USE_OTA_ENCRYPTION
#endif // USE_OTA
+2 -1
View File
@@ -334,8 +334,9 @@ void Fan::dump_traits_(const char *tag, const char *prefix) {
}
if (traits.supports_preset_modes()) {
ESP_LOGCONFIG(tag, "%s Supported presets:", prefix);
for (const char *s : traits.supported_preset_modes())
for (const char *s : traits.supported_preset_modes()) {
ESP_LOGCONFIG(tag, "%s - %s", prefix, s);
}
}
}
@@ -29,8 +29,9 @@ void HBridgeSwitch::dump_config() {
LOG_PIN(" On Pin: ", this->on_pin_);
LOG_PIN(" Off Pin: ", this->off_pin_);
ESP_LOGCONFIG(TAG, " Pulse length: %" PRId32 " ms", this->pulse_length_);
if (this->wait_time_)
if (this->wait_time_) {
ESP_LOGCONFIG(TAG, " Wait time %" PRId32 " ms", this->wait_time_);
}
}
void HBridgeSwitch::write_state(bool state) {
+4 -2
View File
@@ -44,8 +44,9 @@ void HE60rCover::dump_config() {
" Close Duration: %.1fs",
this->open_duration_ / 1e3f, this->close_duration_ / 1e3f);
auto restore = this->restore_state_();
if (restore.has_value())
if (restore.has_value()) {
ESP_LOGCONFIG(TAG, " Saved position %d%%", (int) (restore->position * 100.f));
}
}
void HE60rCover::endstop_reached_(CoverOperation operation) {
@@ -77,8 +78,9 @@ void HE60rCover::process_rx_(uint8_t data) {
ESP_LOGV(TAG, "Process RX data %X", data);
if (!this->query_seen_) {
this->query_seen_ = data == QUERY_BYTE;
if (!this->query_seen_)
if (!this->query_seen_) {
ESP_LOGD(TAG, "RX Byte %02X", data);
}
return;
}
switch (data) {
@@ -257,8 +257,9 @@ void HoermannHcp::on_state_reg_(uint16_t value) {
}
}
// The low byte can change on its own, so only report a state we cannot decode once.
if (state != (previous >> 8))
if (state != (previous >> 8)) {
ESP_LOGW(TAG, "Unknown door state 0x%02X", state);
}
}
// Low byte of register 6: bit 0x10 is the lamp, bit 0x04 the relay. The reference implementation records
@@ -16,26 +16,33 @@ void KeyCollector::loop() {
void KeyCollector::dump_config() {
#if ESPHOME_LOG_LEVEL >= ESPHOME_LOG_LEVEL_CONFIG
ESP_LOGCONFIG(TAG, "Key Collector:");
if (this->min_length_ > 0)
if (this->min_length_ > 0) {
ESP_LOGCONFIG(TAG, " min length: %d", this->min_length_);
if (this->max_length_ > 0)
}
if (this->max_length_ > 0) {
ESP_LOGCONFIG(TAG, " max length: %d", this->max_length_);
if (!this->back_keys_.empty())
}
if (!this->back_keys_.empty()) {
ESP_LOGCONFIG(TAG, " erase keys '%s'", this->back_keys_.c_str());
if (!this->clear_keys_.empty())
}
if (!this->clear_keys_.empty()) {
ESP_LOGCONFIG(TAG, " clear keys '%s'", this->clear_keys_.c_str());
if (!this->start_keys_.empty())
}
if (!this->start_keys_.empty()) {
ESP_LOGCONFIG(TAG, " start keys '%s'", this->start_keys_.c_str());
}
if (!this->end_keys_.empty()) {
ESP_LOGCONFIG(TAG,
" end keys '%s'\n"
" end key is required: %s",
this->end_keys_.c_str(), ONOFF(this->end_key_required_));
}
if (!this->allowed_keys_.empty())
if (!this->allowed_keys_.empty()) {
ESP_LOGCONFIG(TAG, " allowed keys '%s'", this->allowed_keys_.c_str());
if (this->timeout_ > 0)
}
if (this->timeout_ > 0) {
ESP_LOGCONFIG(TAG, " entry timeout: %0.1f", this->timeout_ / 1000.0);
}
#endif
}
+2 -1
View File
@@ -333,8 +333,9 @@ void LN882HBLE::loop() {
// the queue empty — from the very first report on. Checking here keeps that
// failure visible instead of producing a scanner that is silently dead.
uint16_t dropped = this->report_queue_.get_and_reset_dropped_count();
if (dropped > 0)
if (dropped > 0) {
ESP_LOGW(TAG, "Dropped %u scan reports (queue full or out of memory for a report slot)", dropped);
}
// Drain the lock-free ring filled by the rw task; all per-report work runs
// here on the main task, then the report returns to the pool.
BLEScanReport *report = this->report_queue_.pop();
+2 -1
View File
@@ -1059,8 +1059,9 @@ static void *lv_alloc_draw_buf(size_t size, bool internal) {
void *buffer;
size = LV_ROUND_UP(size, LV_DRAW_BUF_ALIGN);
buffer = heap_caps_aligned_alloc(LV_DRAW_BUF_ALIGN, size, internal ? MALLOC_CAP_8BIT : cap_bits); // NOLINT
if (buffer == nullptr)
if (buffer == nullptr) {
ESP_LOGW(esphome::lvgl::TAG, "Failed to allocate %zu bytes for %sdraw buffer", size, internal ? "internal " : "");
}
return buffer;
}
+1 -2
View File
@@ -2,7 +2,7 @@ from contextlib import ExitStack
from esphome import automation
import esphome.codegen as cg
from esphome.components.const import CONF_ROWS
from esphome.components.const import CONF_COLUMNS, CONF_ROWS
import esphome.config_validation as cv
from esphome.const import CONF_ID, CONF_ITEMS, CONF_ROW, CONF_TEXT, CONF_WIDTH
from esphome.core import ID
@@ -20,7 +20,6 @@ from .label import CONF_LABEL
CONF_TABLE = "table"
CONF_CELLS = "cells"
CONF_COLUMNS = "columns"
CONF_ROW_COUNT = "row_count"
CONF_COLUMN_COUNT = "column_count"
CONF_MERGE_RIGHT = "merge_right"
+1 -3
View File
@@ -1,7 +1,7 @@
from esphome import automation, pins
import esphome.codegen as cg
from esphome.components import key_provider
from esphome.components.const import CONF_ROWS
from esphome.components.const import CONF_COLUMNS, CONF_KEYS, CONF_ROWS
import esphome.config_validation as cv
from esphome.const import CONF_ID, CONF_ON_KEY, CONF_PIN, CONF_TRIGGER_ID
from esphome.types import ConfigType
@@ -21,8 +21,6 @@ MatrixKeyTrigger = matrix_keypad_ns.class_(
)
CONF_KEYPAD_ID = "keypad_id"
CONF_COLUMNS = "columns"
CONF_KEYS = "keys"
CONF_DEBOUNCE_TIME = "debounce_time"
CONF_HAS_DIODES = "has_diodes"
CONF_HAS_PULLDOWNS = "has_pulldowns"
+2 -1
View File
@@ -237,8 +237,9 @@ void MipiDsi::write_to_display_(int x_start, int y_start, int w, int h, const ui
xSemaphoreTake(this->io_lock_, portMAX_DELAY);
}
}
if (err != ESP_OK)
if (err != ESP_OK) {
ESP_LOGE(TAG, "lcd_lcd_panel_draw_bitmap failed: %s", esp_err_to_name(err));
}
}
bool MipiDsi::check_buffer_() {
+3 -7
View File
@@ -5,7 +5,7 @@
#include "esphome/core/helpers.h"
#include "esphome/core/log.h"
#include <driver/gpio.h>
#include <esp_lcd_panel_rgb.h>
#include <esp_lcd_panel_ops.h>
#include <span>
namespace esphome::mipi_rgb {
@@ -177,11 +177,6 @@ void MipiRgb::common_setup_() {
ESP_LOGCONFIG(TAG, "MipiRgb setup complete");
}
void MipiRgb::loop() {
if (this->handle_ != nullptr)
esp_lcd_rgb_panel_restart(this->handle_);
}
void MipiRgb::update() {
if (this->is_failed())
return;
@@ -243,8 +238,9 @@ void MipiRgb::write_to_display_(int x_start, int y_start, int w, int h, const ui
ptr += stride; // next line
}
}
if (err != ESP_OK)
if (err != ESP_OK) {
ESP_LOGE(TAG, "lcd_lcd_panel_draw_bitmap failed: %s", esp_err_to_name(err));
}
}
bool MipiRgb::check_buffer_() {
+7 -2
View File
@@ -3,7 +3,7 @@
#if defined(USE_ESP32_VARIANT_ESP32S3) || defined(USE_ESP32_VARIANT_ESP32P4) || defined(USE_ESP32_VARIANT_ESP32S31)
#include "esphome/core/gpio.h"
#include "esphome/components/display/display.h"
#include "esp_lcd_panel_ops.h"
#include <esp_lcd_panel_rgb.h>
#ifdef USE_SPI
#include "esphome/components/spi/spi.h"
#endif
@@ -25,7 +25,12 @@ class MipiRgb : public display::Display {
public:
MipiRgb(int width, int height) : width_(width), height_(height) {}
void setup() override;
void loop() override;
#ifdef USE_ESP32_VARIANT_ESP32S3
void loop() override {
if (this->handle_ != nullptr)
esp_lcd_rgb_panel_restart(this->handle_);
}
#endif
void update() override;
void fill(Color color) override;
void draw_pixels_at(int x_start, int y_start, int w, int h, const uint8_t *ptr, display::ColorOrder order,
+6 -3
View File
@@ -31,12 +31,15 @@ void internal_dump_config(const char *model, int width, int height, int offset_w
LOG_PIN(" CS Pin: ", cs);
LOG_PIN(" Reset Pin: ", reset);
LOG_PIN(" DC Pin: ", dc);
if (offset_width != 0)
if (offset_width != 0) {
ESP_LOGCONFIG(TAG, " Offset width: %d", offset_width);
if (offset_height != 0)
}
if (offset_height != 0) {
ESP_LOGCONFIG(TAG, " Offset height: %d", offset_height);
if (brightness.has_value())
}
if (brightness.has_value()) {
ESP_LOGCONFIG(TAG, " Brightness: %u", brightness.value());
}
}
} // namespace esphome::mipi_spi
+4 -2
View File
@@ -1199,15 +1199,17 @@ void ModbusServerHub::send_raw_(const uint8_t *payload, uint16_t len) {
this->set_timeout("deferred_send", (this->tx_delay_remaining() + US_PER_MS - 1) / US_PER_MS, [this]() {
ModbusFrame frame(this->deferred_payload_[0], this->deferred_payload_.data() + 1,
this->deferred_payload_len_ - 1);
if (!this->send_frame_(frame))
if (!this->send_frame_(frame)) {
ESP_LOGE(TAG, "Deferred server reply dropped: transmission still blocked");
}
});
return;
}
ModbusFrame frame(payload[0], payload + 1, len - 1);
if (!this->send_frame_(frame))
if (!this->send_frame_(frame)) {
ESP_LOGE(TAG, "Server reply dropped: a frame arrived during the send delay");
}
}
void Modbus::clear_rx_buffer_(const LogString *reason, bool warn, size_t bytes_to_clear) {
+4 -2
View File
@@ -39,10 +39,12 @@ inline char *append_char(char *p, char c) {
// Function implementation of LOG_MQTT_COMPONENT macro to reduce code size
void log_mqtt_component(const char *tag, MQTTComponent *obj, bool state_topic, bool command_topic) {
char buf[MQTT_DEFAULT_TOPIC_MAX_LEN];
if (state_topic)
if (state_topic) {
ESP_LOGCONFIG(tag, " State Topic: '%s'", obj->get_state_topic_to_(buf).c_str());
if (command_topic)
}
if (command_topic) {
ESP_LOGCONFIG(tag, " Command Topic: '%s'", obj->get_command_topic_to_(buf).c_str());
}
}
void MQTTComponent::set_qos(uint8_t qos) { this->qos_ = qos; }
+9
View File
@@ -45,6 +45,15 @@ def decode_encryption_key(value: str) -> bytes:
return decoded
def is_reserved_key(value: str) -> bool:
"""Whether the key is the reserved all-zeros provisioning sentinel.
The device treats it as no key configured, so consumers that require a
real key must reject it.
"""
return not any(decode_encryption_key(value))
ENCRYPTION_SCHEMA = cv.Schema(
{
cv.Optional(CONF_KEY): cv.sensitive(validate_encryption_key),
+2 -1
View File
@@ -18,8 +18,9 @@ const std::vector<uint64_t> &OneWireBus::get_devices() { return this->devices_;
bool OneWireBus::reset_() {
int res = this->reset_int();
if (res == -1)
if (res == -1) {
ESP_LOGE(TAG, "1-wire bus is held low");
}
return res == 1;
}
+1
View File
@@ -49,6 +49,7 @@ enum OTAResponseTypes {
OTA_RESPONSE_ERROR_BOOTLOADER_VERIFY = 0x91,
OTA_RESPONSE_ERROR_BOOTLOADER_UPDATE = 0x92,
OTA_RESPONSE_ERROR_VERSION_DOWNGRADE = 0x93,
OTA_RESPONSE_ERROR_ENCRYPTION_REQUIRED = 0x94,
OTA_RESPONSE_ERROR_UNKNOWN = 0xFF,
};
@@ -551,12 +551,14 @@ void PacketTransport::dump_config() {
" Ping-pong: %s",
this->platform_name_, YESNO(this->is_encrypted_()), YESNO(this->ping_pong_enable_));
#ifdef USE_SENSOR
for (const auto &sensor : this->sensors_)
for (const auto &sensor : this->sensors_) {
ESP_LOGCONFIG(TAG, " Sensor: %s", sensor.id);
}
#endif
#ifdef USE_BINARY_SENSOR
for (const auto &sensor : this->binary_sensors_)
for (const auto &sensor : this->binary_sensors_) {
ESP_LOGCONFIG(TAG, " Binary Sensor: %s", sensor.id);
}
#endif
for (const auto &host : this->providers_) {
ESP_LOGCONFIG(TAG, " Remote host: %s", host.first.c_str());
@@ -564,15 +566,17 @@ void PacketTransport::dump_config() {
#ifdef USE_SENSOR
auto rs = this->remote_sensors_.find(host.first.c_str());
if (rs != this->remote_sensors_.end()) {
for (const auto &key : rs->second | std::views::keys)
for (const auto &key : rs->second | std::views::keys) {
ESP_LOGCONFIG(TAG, " Sensor: %s", key.c_str());
}
}
#endif
#ifdef USE_BINARY_SENSOR
auto rbs = this->remote_binary_sensors_.find(host.first.c_str());
if (rbs != this->remote_binary_sensors_.end()) {
for (const auto &key : rbs->second | std::views::keys)
for (const auto &key : rbs->second | std::views::keys) {
ESP_LOGCONFIG(TAG, " Binary Sensor: %s", key.c_str());
}
}
#endif
}
+2 -1
View File
@@ -124,8 +124,9 @@ void QwiicPIRComponent::dump_config() {
void QwiicPIRComponent::clear_events_() {
// Clear event status register
if (!this->write_byte(QWIIC_PIR_EVENT_STATUS, 0x00))
if (!this->write_byte(QWIIC_PIR_EVENT_STATUS, 0x00)) {
ESP_LOGW(TAG, "Failed to clear events");
}
}
} // namespace esphome::qwiic_pir
@@ -75,8 +75,9 @@ void RpiDpiRgb::draw_pixels_at(int x_start, int y_start, int w, int h, const uin
break;
}
}
if (err != ESP_OK)
if (err != ESP_OK) {
ESP_LOGE(TAG, "lcd_lcd_panel_draw_bitmap failed: %s", esp_err_to_name(err));
}
}
int RpiDpiRgb::get_width() {
+11 -4
View File
@@ -255,12 +255,17 @@ bool SafeModeComponent::should_enter_safe_mode(uint8_t num_attempts, uint32_t en
}
void SafeModeComponent::write_rtc_(uint32_t val) {
this->rtc_.save(&val);
global_preferences->sync();
if (!this->rtc_.save(&val)) {
ESP_LOGE(TAG, "Failed to set rtc value (%" PRIu32 ")", val);
return;
}
if (!global_preferences->sync()) {
ESP_LOGE(TAG, "Failed to persist rtc value (%" PRIu32 ")", val);
}
}
uint32_t SafeModeComponent::read_rtc_() {
uint32_t val;
uint32_t val = 0;
if (!this->rtc_.load(&val))
return 0;
return val;
@@ -272,7 +277,9 @@ void SafeModeComponent::clean_rtc() {
// before sync, the boot wasn't really successful anyway and the counter should
// remain incremented.
uint32_t val = 0;
this->rtc_.save(&val);
if (!this->rtc_.save(&val)) {
ESP_LOGE(TAG, "Failed to clear boot loop counter");
}
}
void SafeModeComponent::on_safe_shutdown() {
@@ -629,8 +629,9 @@ stm32_unique_ptr stm32_init(uart::UARTDevice *stream, const uint8_t flags, const
stm->pid = (buf[1] << 8) | buf[2];
if (returned > 2) {
ESP_LOGD(TAG, "This bootloader returns %d extra bytes in PID:", returned);
for (auto i = 2; i <= returned; i++)
for (auto i = 2; i <= returned; i++) {
ESP_LOGD(TAG, " %02x", buf[i]);
}
}
if (stm32_get_ack(stm) != STM32_ERR_OK) {
return make_stm32_with_deletor(nullptr);
+2 -1
View File
@@ -406,8 +406,9 @@ class SPIClient {
this->release_device_, this->write_only_);
#ifdef USE_SPI_PSRAM_DMA
this->delegate_->set_psram_dma(this->psram_dma_);
if (this->psram_dma_)
if (this->psram_dma_) {
esph_log_config("spi_device", "PSRAM DMA: enabled");
}
#endif
}
+6 -3
View File
@@ -42,8 +42,9 @@ class SPIDelegateHw : public SPIDelegate {
if (this->release_device_)
this->add_device_();
if (this->is_ready()) {
if (spi_device_acquire_bus(this->handle_, portMAX_DELAY) != ESP_OK)
if (spi_device_acquire_bus(this->handle_, portMAX_DELAY) != ESP_OK) {
ESP_LOGE(TAG, "Failed to acquire SPI bus");
}
SPIDelegate::begin_transaction();
} else {
ESP_LOGW(TAG, "SPI device not ready, cannot begin transaction");
@@ -63,8 +64,9 @@ class SPIDelegateHw : public SPIDelegate {
~SPIDelegateHw() override {
esp_err_t const err = spi_bus_remove_device(this->handle_);
if (err != ESP_OK)
if (err != ESP_OK) {
ESP_LOGE(TAG, "Remove device failed - err %X", err);
}
}
// do a transfer. either txbuf or rxbuf (but not both) may be null.
@@ -284,8 +286,9 @@ class SPIBusHw : public SPIBus {
}
buscfg.max_transfer_sz = MAX_TRANSFER_SIZE;
auto err = spi_bus_initialize(channel, &buscfg, SPI_DMA_CH_AUTO);
if (err != ESP_OK)
if (err != ESP_OK) {
ESP_LOGE(TAG, "Bus init failed - err %X", err);
}
}
SPIDelegate *get_delegate(uint32_t data_rate, SPIBitOrder bit_order, SPIMode mode, GPIOPin *cs_pin,
+2 -1
View File
@@ -78,8 +78,9 @@ void ST7701S::draw_pixels_at(int x_start, int y_start, int w, int h, const uint8
break;
}
}
if (err != ESP_OK)
if (err != ESP_OK) {
esph_log_e(TAG, "lcd_lcd_panel_draw_bitmap failed: %s", esp_err_to_name(err));
}
}
void ST7701S::draw_pixel_at(int x, int y, Color color) {
+1 -1
View File
@@ -1,6 +1,7 @@
from esphome import automation, pins
import esphome.codegen as cg
from esphome.components import i2c, key_provider
from esphome.components.const import CONF_KEYS
import esphome.config_validation as cv
from esphome.const import (
CONF_ID,
@@ -19,7 +20,6 @@ from esphome.cpp_generator import MockObj
from esphome.types import ConfigType
CONF_KEYPAD = "keypad"
CONF_KEYS = "keys"
CONF_KEY_ROWS = "key_rows"
CONF_KEY_COLUMNS = "key_columns"
CONF_SLEEP_TIME = "sleep_time"
@@ -177,14 +177,18 @@ water_heater::WaterHeaterMode TuyaWaterHeater::default_on_mode_() const {
void TuyaWaterHeater::dump_config() {
LOG_WATER_HEATER("", "Tuya Water Heater", this);
if (this->switch_id_.has_value())
if (this->switch_id_.has_value()) {
ESP_LOGCONFIG(TAG, " Switch has datapoint ID %u", *this->switch_id_);
if (this->mode_id_.has_value())
}
if (this->mode_id_.has_value()) {
ESP_LOGCONFIG(TAG, " Mode has datapoint ID %u", *this->mode_id_);
if (this->target_temperature_id_.has_value())
}
if (this->target_temperature_id_.has_value()) {
ESP_LOGCONFIG(TAG, " Target Temperature has datapoint ID %u", *this->target_temperature_id_);
if (this->current_temperature_id_.has_value())
}
if (this->current_temperature_id_.has_value()) {
ESP_LOGCONFIG(TAG, " Current Temperature has datapoint ID %u", *this->current_temperature_id_);
}
}
} // namespace esphome::tuya
+6 -3
View File
@@ -129,8 +129,9 @@ void UDPComponent::dump_config() {
" Listen Port: %u\n"
" Broadcast Port: %u",
this->listen_port_, this->broadcast_port_);
for (const char *address : this->addresses_)
for (const char *address : this->addresses_) {
ESP_LOGCONFIG(TAG, " Address: %s", address);
}
if (this->listen_address_.has_value()) {
char addr_buf[network::IP_ADDRESS_BUFFER_SIZE];
ESP_LOGCONFIG(TAG, " Listen address: %s", this->listen_address_.value().str_to(addr_buf));
@@ -145,8 +146,9 @@ void UDPComponent::send_packet(const uint8_t *data, size_t size) {
#if defined(USE_SOCKET_IMPL_BSD_SOCKETS) || defined(USE_SOCKET_IMPL_LWIP_SOCKETS)
for (const auto &saddr : this->sockaddrs_) {
auto result = this->broadcast_socket_->sendto(data, size, 0, &saddr, sizeof(saddr));
if (result < 0)
if (result < 0) {
ESP_LOGW(TAG, "sendto() error %d", errno);
}
}
#endif
#ifdef USE_SOCKET_IMPL_LWIP_TCP
@@ -155,8 +157,9 @@ void UDPComponent::send_packet(const uint8_t *data, size_t size) {
if (this->udp_client_.beginPacketMulticast(saddr, this->broadcast_port_, iface, 128) != 0) {
this->udp_client_.write(data, size);
auto result = this->udp_client_.endPacket();
if (result == 0)
if (result == 0) {
ESP_LOGW(TAG, "udp.write() error");
}
}
}
#endif
@@ -110,8 +110,9 @@ bool UponorSmatrixComponent::parse_byte_(uint8_t byte) {
// Handle packet
size_t data_len = (packet_len - 6) / 3;
if (data_len == 0) {
if (packet[4] == UPONOR_ID_REQUEST)
if (packet[4] == UPONOR_ID_REQUEST) {
ESP_LOGVV(TAG, "Ignoring request packet for device 0x%08" PRIX32 "", device_address);
}
return true;
}
+2 -1
View File
@@ -194,8 +194,9 @@ std::vector<CdcEps> USBUartTypePL2303::parse_descriptors(usb_device_handle_t dev
}
}
if (cdc_devs.empty())
if (cdc_devs.empty()) {
ESP_LOGE(TAG, "PL2303: failed to find bulk IN+OUT endpoints");
}
return cdc_devs;
}
@@ -40,8 +40,9 @@ void WakeOnLanButton::press_action() {
memcpy(buffer + i * sizeof(this->macaddr_) + sizeof(PREFIX), this->macaddr_, sizeof(this->macaddr_));
}
if (this->broadcast_socket_->sendto(buffer, sizeof(buffer), 0, reinterpret_cast<const sockaddr *>(&saddr),
addr_len) <= 0)
addr_len) <= 0) {
ESP_LOGW(TAG, "sendto() error %d", errno);
}
#else
IPAddress broadcast = IPAddress(255, 255, 255, 255);
for (auto ip : esphome::network::get_ip_addresses()) {
+10 -5
View File
@@ -348,14 +348,18 @@ size_t WeikaiChannel::rx_in_fifo_() {
uint8_t const fsr = this->reg(WKREG_FSR);
if (fsr & (FSR_RFOE | FSR_RFLB | FSR_RFFE | FSR_RFPE)) {
char bin_buf[9];
if (fsr & FSR_RFOE)
if (fsr & FSR_RFOE) {
ESP_LOGE(TAG, "Receive data overflow FSR=%s", format_bin_to(bin_buf, fsr));
if (fsr & FSR_RFLB)
}
if (fsr & FSR_RFLB) {
ESP_LOGE(TAG, "Receive line break FSR=%s", format_bin_to(bin_buf, fsr));
if (fsr & FSR_RFFE)
}
if (fsr & FSR_RFFE) {
ESP_LOGE(TAG, "Receive frame error FSR=%s", format_bin_to(bin_buf, fsr));
if (fsr & FSR_RFPE)
}
if (fsr & FSR_RFPE) {
ESP_LOGE(TAG, "Receive parity error FSR=%s", format_bin_to(bin_buf, fsr));
}
}
if ((available == 0) && (fsr & FSR_RFDAT)) {
// here we should be very careful because we can have something like this:
@@ -495,8 +499,9 @@ void print_buffer(std::vector<uint8_t> buffer) {
hex_buffer[(3 * 32) + 1] = 0;
for (size_t i = 0; i < buffer.size(); i++) {
snprintf(&hex_buffer[3 * (i % 32)], sizeof(hex_buffer), "%02X ", buffer[i]);
if (i % 32 == 31)
if (i % 32 == 31) {
ESP_LOGI(TAG, " %s", hex_buffer);
}
}
if (buffer.size() % 32) {
// null terminate if incomplete line
+1
View File
@@ -242,6 +242,7 @@
#define USE_RUNTIME_IMAGE_QOI
#define USE_RUNTIME_STATS
#define USE_OTA
#define USE_OTA_ENCRYPTION
#define USE_OTA_PASSWORD
#define USE_OTA_VERSION 2
#define USE_TIME_TIMEZONE
+193 -5
View File
@@ -53,6 +53,7 @@ RESPONSE_ERROR_PARTITION_TABLE_UPDATE = 0x90
RESPONSE_ERROR_BOOTLOADER_VERIFY = 0x91
RESPONSE_ERROR_BOOTLOADER_UPDATE = 0x92
RESPONSE_ERROR_VERSION_DOWNGRADE = 0x93
RESPONSE_ERROR_ENCRYPTION_REQUIRED = 0x94
RESPONSE_ERROR_UNKNOWN = 0xFF
OTA_VERSION_1_0 = 1
@@ -63,8 +64,20 @@ MAGIC_BYTES = [0x6C, 0x26, 0xF7, 0x5C, 0x45]
CLIENT_FEATURE_SUPPORTS_COMPRESSION = 0x01
CLIENT_FEATURE_SUPPORTS_SHA256_AUTH = 0x02
CLIENT_FEATURE_SUPPORTS_EXTENDED_PROTOCOL = 0x04
CLIENT_FEATURE_SUPPORTS_NOISE = 0x08
SERVER_FEATURE_SUPPORTS_COMPRESSION = 0x01
SERVER_FEATURE_SUPPORTS_PARTITION_ACCESS = 0x02
SERVER_FEATURE_SUPPORTS_NOISE = 0x04
NOISE_FRAME_INDICATOR = 0x01
NOISE_HANDSHAKE_OK = 0x00
# The device decrypts frames in its transfer buffer (OTA_BUFFER_SIZE, sized
# as this plus the 16-byte ChaCha20-Poly1305 MAC). 1024 divides the 8192-byte
# upload block exactly, so blocks tile into full frames with no runt.
NOISE_MAX_PLAINTEXT = 1024
# Wire contract: the device sends exactly this reject reason for a bad MAC
NOISE_MAC_FAILURE_REASON = "Handshake MAC failure"
NOISE_PROLOGUE_INIT = b"NoiseOTAInit"
# OTA types this client knows how to send. Future PRs that add bootloader/partition
# updates extend this set. Anything outside the set is rejected up front so callers
@@ -171,6 +184,12 @@ _ERROR_MESSAGES: dict[int, str] = {
"enabled: the new firmware's version must be newer than the version the "
"device is currently running."
),
RESPONSE_ERROR_ENCRYPTION_REQUIRED: (
"The device requires an encrypted OTA connection but this upload has no "
"encryption key. Add 'encryption:' to the 'ota: platform: esphome' section "
"of the YAML this upload uses, or update your esphome installation if it "
"predates OTA encryption."
),
RESPONSE_ERROR_UNKNOWN: "Unknown error from ESP",
}
@@ -305,16 +324,149 @@ def send_check(
raise OTANetworkError(f"sending {msg}: {err}") from err
class NoiseSocketWrapper:
"""Runs the OTA session inside a Noise (ChaCha20-Poly1305) transport.
Exposes the socket subset perform_ota uses. Frames are indicator 0x01,
16-bit big-endian length, ciphertext; recv() drains one decrypted frame
at a time, sendall() keeps control units in one frame and splits data
at NOISE_MAX_PLAINTEXT.
"""
def __init__(self, sock: socket.socket, psk: str, prologue: bytes) -> None:
# Deliberately lazy: the noise stack (noiseprotocol, cryptography) is
# only imported when an encrypted upload actually runs.
try:
from aioesphomeapi.noise import NoiseHandshake
except ImportError as err:
raise OTAError(
"OTA encryption requires a newer aioesphomeapi; update your "
"esphome installation (pip install -U esphome) and retry"
) from err
# The aioesphomeapi import above already loaded cryptography; bind
# the exception once so recv() pays no per-frame import lookup
from cryptography.exceptions import InvalidTag
self._invalid_tag = InvalidTag
self._sock = sock
try:
self._handshake = NoiseHandshake(psk, prologue)
except ValueError as err:
raise OTAError(f"Invalid OTA encryption key: {err}") from err
self._encrypt = None
self._decrypt = None
self._buffer = b""
# Only harmless socket controls pass through; byte-moving methods are
# deliberately absent so plaintext cannot leak past the transport.
def settimeout(self, timeout: float | None) -> None:
self._sock.settimeout(timeout)
def setsockopt(self, level: int, optname: int, value: int) -> None:
self._sock.setsockopt(level, optname, value)
def close(self) -> None:
self._sock.close()
def do_handshake(self) -> None:
"""Run the two-message NNpsk0 handshake and set up the transport ciphers."""
try:
self._send_frame(
bytes([NOISE_HANDSHAKE_OK]) + self._handshake.write_message()
)
payload = self._recv_frame()
except OSError as err:
raise OTANetworkError(f"noise handshake: {err}") from err
if not payload:
raise OTANetworkError("Device closed connection during the noise handshake")
if payload[0] != NOISE_HANDSHAKE_OK:
reason = payload[1:].decode("utf-8", "replace")
if reason == NOISE_MAC_FAILURE_REASON:
raise OTAError(
"Device rejected the handshake; is the OTA encryption key correct?"
)
raise OTAError(f"Device rejected the noise handshake: {reason}")
try:
self._handshake.read_message(payload[1:])
except (ValueError, self._invalid_tag) as err:
# InvalidTag is a wrong key; ValueError covers a device sending an
# invalid curve point, which cryptography rejects during the DH
raise OTAError(
"Noise handshake failed; is the OTA encryption key correct?"
) from err
self._encrypt, self._decrypt = self._handshake.get_ciphers()
def sendall(self, data: bytes) -> None:
frames: list[bytes] = []
for offset in range(0, len(data), NOISE_MAX_PLAINTEXT):
ciphertext = self._encrypt.encrypt(
data[offset : offset + NOISE_MAX_PLAINTEXT]
)
frames.append(self._frame_header(len(ciphertext)))
frames.append(ciphertext)
self._sock.sendall(b"".join(frames))
def recv(self, amount: int) -> bytes:
if not self._buffer:
ciphertext = self._recv_frame()
if not ciphertext:
return b"" # connection closed at a frame boundary
try:
self._buffer = self._decrypt.decrypt(ciphertext)
except self._invalid_tag as err:
# Retryable: a fresh connection renegotiates the session
raise OTANetworkError(
"Noise decryption failed (MAC mismatch); frame corrupted or tampered"
) from err
if not self._buffer:
# Reject MAC-only frames so b"" always means the peer closed
raise OTANetworkError("Device sent an empty noise frame")
data = self._buffer[:amount]
self._buffer = self._buffer[amount:]
return data
@staticmethod
def _frame_header(length: int) -> bytes:
return bytes([NOISE_FRAME_INDICATOR, (length >> 8) & 0xFF, length & 0xFF])
def _send_frame(self, payload: bytes) -> None:
self._sock.sendall(self._frame_header(len(payload)) + payload)
def _recv_frame(self) -> bytes:
header = self._recv_exact(3, closed_ok=True)
if not header:
return b"" # connection closed at a frame boundary
# A malformed frame is a broken transport, not a device error;
# retryable so a fresh session is tried
if header[0] != NOISE_FRAME_INDICATOR:
raise OTANetworkError(f"Bad noise frame indicator 0x{header[0]:02X}")
length = (header[1] << 8) | header[2]
if length == 0:
raise OTANetworkError("Device sent an empty noise frame")
return self._recv_exact(length)
def _recv_exact(self, amount: int, closed_ok: bool = False) -> bytes:
data = b""
while len(data) < amount:
chunk = self._sock.recv(amount - len(data))
if not chunk:
if closed_ok and not data:
return b""
raise OSError("connection closed inside a noise frame")
data += chunk
return data
def perform_ota(
sock: socket.socket,
password: str | None,
file_handle: io.IOBase,
filename: Path,
ota_type: int = OTA_TYPE_UPDATE_APP,
noise_psk: str | None = None,
) -> None:
# Validate ota_type up front. It travels as a single byte on the wire, and
# passing an out-of-range value would only surface as a ValueError from
# bytes([ota_type]) deep inside send_check, bypassing OTAError handling.
# Validate up front; an out-of-range value would only surface as a
# ValueError deep inside send_check, bypassing OTAError handling
if not isinstance(ota_type, int) or not 0 <= ota_type <= 0xFF:
raise OTAError(
f"Invalid ota_type {ota_type!r}; expected an integer in range 0-255"
@@ -325,6 +477,11 @@ def perform_ota(
f"Unsupported OTA type 0x{ota_type:02X}; this ESPHome supports: {supported}"
)
if noise_psk is not None and not noise_psk:
raise OTAError(
"An empty OTA encryption key was provided; refusing to upload in plaintext"
)
file_contents = file_handle.read()
file_size = len(file_contents)
_LOGGER.info("Uploading %s (%s bytes)", filename, file_size)
@@ -347,6 +504,8 @@ def perform_ota(
| CLIENT_FEATURE_SUPPORTS_SHA256_AUTH
| CLIENT_FEATURE_SUPPORTS_EXTENDED_PROTOCOL
)
if noise_psk:
features_to_send |= CLIENT_FEATURE_SUPPORTS_NOISE
send_check(sock, features_to_send, "features")
features = receive_exactly(
sock,
@@ -369,6 +528,31 @@ def perform_ota(
else:
features = 0
if noise_psk:
# Fail closed: never fall back to a plaintext upload when an
# encryption key is configured, an active attacker could otherwise
# strip the feature flag and capture the image (it contains the wifi
# credentials and the api encryption key).
if not (extended_proto and features & SERVER_FEATURE_SUPPORTS_NOISE):
raise OTAError(
"An OTA encryption key is configured but the device did not "
"offer encryption; refusing to send the image in plaintext. "
"If the running firmware predates OTA encryption, first update "
"it without the 'ota: encryption:' block (over a trusted "
"network or via USB), then restore the block and upload again."
)
# The prologue binds every negotiation byte both sides saw, so any
# tampering with the plaintext preamble breaks the handshake.
prologue = (
NOISE_PROLOGUE_INIT
+ bytes(MAGIC_BYTES)
+ bytes([RESPONSE_OK, version, features_to_send])
+ bytes([RESPONSE_FEATURE_FLAGS, features])
)
sock = NoiseSocketWrapper(sock, noise_psk, prologue)
sock.do_handshake()
_LOGGER.info("Encrypted connection established")
if ota_type != OTA_TYPE_UPDATE_APP:
# Any non-app OTA type requires the extended protocol and the
# partition-access server feature. Reject up front so the user gets
@@ -572,6 +756,7 @@ def run_ota_impl_(
password: str | None,
filename: Path,
ota_type: int = OTA_TYPE_UPDATE_APP,
noise_psk: str | None = None,
) -> tuple[int, str | None]:
from esphome.core import CORE
@@ -636,7 +821,7 @@ def run_ota_impl_(
reached_device = True
with contextlib.closing(sock), Path(filename).open("rb") as file_handle:
try:
perform_ota(sock, password, file_handle, filename, ota_type)
perform_ota(sock, password, file_handle, filename, ota_type, noise_psk)
except OTANetworkError as err:
# Transient network failure; retry
last_error = str(err)
@@ -661,9 +846,12 @@ def run_ota(
password: str | None,
filename: Path,
ota_type: int = OTA_TYPE_UPDATE_APP,
noise_psk: str | None = None,
) -> tuple[int, str | None]:
try:
return run_ota_impl_(remote_host, remote_port, password, filename, ota_type)
return run_ota_impl_(
remote_host, remote_port, password, filename, ota_type, noise_psk
)
except OTAError as err:
_LOGGER.error(err)
return 1, None
+9
View File
@@ -457,6 +457,15 @@ def _clone_complete_marker_path(repo_dir: Path) -> Path:
return repo_dir / ".git" / _CLONE_COMPLETE_MARKER
def has_complete_clone(
url: str, ref: str | None, domain: str, subpath: Path | None = None
) -> bool:
"""Lock-free probe for a complete clone; can go stale immediately, so
best-effort decisions only, never a substitute for ``clone_or_update``."""
repo_dir = _repo_entry_dir(_cache_key(url, ref), domain, subpath)
return _clone_complete_marker_path(repo_dir).is_file()
def _clear_clone_complete_marker(repo_dir: Path) -> None:
"""Best-effort removal of the completion marker.
+77 -26
View File
@@ -13,7 +13,7 @@ regardless of which toolchain consumes the result.
"""
from collections import deque
from collections.abc import Callable, Iterable
from collections.abc import Callable, Hashable, Iterable
from dataclasses import dataclass, field
from functools import partial
import glob
@@ -99,6 +99,17 @@ class Source:
) -> Path:
raise NotImplementedError
def prefetch_key(self, dir_suffix: str) -> Hashable | None:
"""Prefetch dedup identity; None = not prefetchable. Sources that
could write one cache dir must return equal keys (workers must never
share a dir); a coarser key only skips a prefetch."""
return None
def is_cached(self, dir_suffix: str, salt: str = "", namespace: str = "") -> bool:
"""Whether a completed fetch exists; only consulted when
``prefetch_key()`` is not None, True is the safe default."""
return True
def source_root(self, build_path: Path) -> Path:
"""Directory holding the library's own files (manifest + sources).
@@ -127,6 +138,9 @@ class URLSource(Source):
h.update(salt.encode())
return base_dir / h.hexdigest()[:8] / dir_suffix
def prefetch_key(self, dir_suffix: str) -> Hashable | None:
return self.url if self.size else None
def is_cached(self, dir_suffix: str, salt: str = "", namespace: str = "") -> bool:
"""Whether a completed extraction already exists for this source."""
return (
@@ -177,14 +191,29 @@ class GitSource(Source):
self.url = url
self.ref = ref
def download(
self, dir_suffix: str, force: bool = False, salt: str = "", namespace: str = ""
) -> Path:
@staticmethod
def _domain(salt: str, namespace: str) -> str:
domain = DOMAIN
if namespace:
domain = f"{domain}/{namespace}"
if salt:
domain = f"{domain}/{salt}"
return domain
def prefetch_key(self, dir_suffix: str) -> Hashable | None:
# The clone target dir is hash(url@ref)/<dir_suffix>
return (self.url, self.ref, dir_suffix)
def is_cached(self, dir_suffix: str, salt: str = "", namespace: str = "") -> bool:
"""Whether a completed clone already exists for this source."""
return git.has_complete_clone(
self.url, self.ref, self._domain(salt, namespace), Path(dir_suffix)
)
def download(
self, dir_suffix: str, force: bool = False, salt: str = "", namespace: str = ""
) -> Path:
domain = self._domain(salt, namespace)
path, _ = git.clone_or_update(
url=self.url,
ref=self.ref,
@@ -988,56 +1017,78 @@ def _fetch_source(
)
def _clone_source(
component: ConvertedLibrary,
salt: str,
namespace: str,
tracker: Callable[[int], None],
) -> None:
# No byte progress from git; one tick so a cancelled batch stops here
tracker(0)
component.source.download(
component.get_sanitized_name(), salt=salt, namespace=namespace
)
def _prefetch_wave(
wave: list[tuple[str, ConvertedLibrary]], salt: str, namespace: str
) -> None:
"""Best-effort parallel download of a wave's registry archives.
"""Best-effort parallel fetch of a wave's registry archives and git clones.
The walk's own ``download()`` stays authoritative; duplicate URLs
The walk's own ``download()`` stays authoritative; duplicate sources
prefetch once so two threads never share a cache directory. Archives
whose size the registry did not report are left to the sequential
loop, whose per-file bars don't interleave. A node a sibling in the
same wave supersedes has its archive fetched in vain (knowing better
same wave supersedes has its source fetched in vain (knowing better
would need the manifests being downloaded).
"""
try:
components: list[ConvertedLibrary] = []
seen: set[str] = set()
archives: list[ConvertedLibrary] = []
clones: list[ConvertedLibrary] = []
seen: set[Hashable] = set()
for _key, component in wave:
source = component.source
if not isinstance(source, URLSource) or not source.size:
name = component.get_sanitized_name()
dedup_key = source.prefetch_key(name)
if dedup_key is None or dedup_key in seen:
continue
if source.url in seen:
continue
seen.add(source.url)
seen.add(dedup_key)
try:
cached = source.is_cached(
component.get_sanitized_name(), salt=salt, namespace=namespace
)
cached = source.is_cached(name, salt=salt, namespace=namespace)
except OSError as err:
# Best-effort, but visibly: a systematic probe failure makes
# every warm build re-download every archive
# every warm build re-fetch every source
_LOGGER.warning("Cache probe for %s failed: %s", component.name, err)
cached = False
if cached:
# A warm build must stay silent
continue
components.append(component)
if not components:
(archives if isinstance(source, URLSource) else clones).append(component)
if not archives and not clones:
return
# Single-item waves (a dependency chain discovers one archive per
# wave) go through the same runner: one download method, one bar
_LOGGER.info(
"Downloading %d library archive(s): %s",
len(components),
", ".join(c.name for c in components),
)
if archives:
_LOGGER.info(
"Downloading %d library archive(s): %s",
len(archives),
", ".join(c.name for c in archives),
)
if clones:
_LOGGER.info(
"Cloning %d library repo(s): %s",
len(clones),
", ".join(c.name for c in clones),
)
failures = run_batch_downloads(
"Downloading libraries",
[
(c.name, c.source.size, partial(_fetch_source, c, salt, namespace))
for c in components
],
for c in archives
]
# Size 0: clones share the worker pool without skewing the
# byte bar, whose total stays the archive sum
+ [(c.name, 0, partial(_clone_source, c, salt, namespace)) for c in clones],
)
# The sequential call below retries and raises the real error
warn_prefetch_failures(
+9 -9
View File
@@ -832,16 +832,16 @@ def _prefetch(build_dir: Path, env: str) -> None:
for name, opts in p.packages.items()
if not opts.get("optional")
]
# PIO's build engine installs outside the platform package list;
# skipped when the platform lists it itself
if not any(s.name == "tool-scons" for s in specs):
specs.append(
PackageSpec(
owner="platformio",
name="tool-scons",
requirements=get_core_dependencies()["tool-scons"],
)
# PIO's build engine installs tool-scons by its own registry spec at build
# start; a platform URL copy has no owner to match it, so prefetch that spec
specs = [s for s in specs if s.name != "tool-scons"]
specs.append(
PackageSpec(
owner="platformio",
name="tool-scons",
requirements=get_core_dependencies()["tool-scons"],
)
)
lib_deps = config.get(f"env:{env}", "lib_deps", [])
# pio run's storage dir for this env, with its compatibility
# qualifiers: an unqualified library install could land a different
+1 -1
View File
@@ -14,7 +14,7 @@ esptool==5.3.1
click==8.3.3
aioesphomeapi==46.3.0
aiohappyeyeballs==2.7.1 # Happy Eyeballs for requests downloads; already pulled in by aioesphomeapi
zeroconf==0.150.4
zeroconf==0.151.2
puremagic==2.2.0
ruamel.yaml==0.19.1 # dashboard_import
ruamel.yaml.clib==0.2.15 # dashboard_import
+1 -1
View File
@@ -1,4 +1,4 @@
pylint==4.0.7
pylint==4.0.8
flake8==7.3.0 # also change in .pre-commit-config.yaml when updating
ruff==0.16.5 # also change in .pre-commit-config.yaml when updating
pyupgrade==3.21.2 # also change in .pre-commit-config.yaml when updating
+148
View File
@@ -319,6 +319,154 @@ def lint_no_long_delays(fname, match):
)
# An if/else/for/while whose only body is an unbraced ESP_LOG*() call. When the build's compile-time
# log level drops that macro, the body expands to nothing and the compiler warns (-Wempty-body).
# clang-tidy's brace check does not catch these (ShortStatementLines allows short unbraced bodies), so
# this fills that gap. Matched against comment/string-masked content, so commented-out or quoted code
# is ignored. Both spellings are covered: core/log.h defines the uppercase ESP_LOG*() macros and
# the lowercase esph_log_*() ones, and both expand to nothing below their log level.
# 'for' allows ';' inside its parentheses (the classic C-style header); 'if'/'while' do not, so their
# condition cannot run past the statement it guards. The 'for' header permits one level of nested
# parens so it stays bounded to its own statement: without that, it can run past the loop body and
# latch onto a later ')', mis-reporting the line and skipping the '#' preprocessor check below.
ESP_LOG_NEEDS_BRACES_RE = re.compile(
r"(?:\bif\s*\([^{};]*\)|\bwhile\s*\([^{};]*\)|\bfor\s*\((?:[^{}()]|\([^{}()]*\))*\)|\belse\b)"
r"[ \t]*\n?[ \t]*(?:ESP_LOG[A-Z]*|esph_log_[a-z]+)\s*\(",
re.MULTILINE,
)
def _mask_cpp_comments_strings(s):
"""Return s with // and /* */ comments and string/char/raw-string literals blanked to spaces
(length and newlines preserved) so a regex only matches real code. Parentheses in real code are
kept, so callers can still balance them on the masked text."""
out = list(s)
i = 0
n = len(s)
while i < n:
c = s[i]
# Raw string literal: an optional encoding prefix, then R"delim( ... )delim". The body may
# contain quotes, //, /* and unbalanced parens, so it must be consumed as one unit.
if c == "R" and i + 1 < n and s[i + 1] == '"':
j = i + 2
delim = ""
while j < n and s[j] not in "( \t\r\n\\" and len(delim) < 16:
delim += s[j]
j += 1
if j < n and s[j] == "(":
closing = ")" + delim + '"'
end = s.find(closing, j + 1)
end = n if end == -1 else end + len(closing)
for k in range(i, end):
if s[k] != "\n":
out[k] = " "
i = end
continue
i += 1
elif c == "/" and i + 1 < n and s[i + 1] == "/":
while i < n and s[i] != "\n":
out[i] = " "
i += 1
elif c == "/" and i + 1 < n and s[i + 1] == "*":
out[i] = out[i + 1] = " "
i += 2
while i < n and not (s[i] == "*" and i + 1 < n and s[i + 1] == "/"):
if s[i] != "\n":
out[i] = " "
i += 1
if i < n:
out[i] = " "
if i + 1 < n:
out[i + 1] = " "
i += 2
# A "'" after an alphanumeric or '_' is a C++ digit separator (1'000), not a literal opener.
elif c == '"' or (
c == "'" and not (i and (s[i - 1].isalnum() or s[i - 1] == "_"))
):
quote = c
out[i] = " "
i += 1
while i < n:
if s[i] == "\\":
out[i] = " "
if i + 1 < n:
out[i + 1] = " "
i += 2
continue
if s[i] == quote:
out[i] = " "
i += 1
break
if s[i] != "\n":
out[i] = " "
i += 1
else:
i += 1
return "".join(out)
def _log_statement_end(masked, open_paren):
"""Index of the ';' ending the ESP_LOG call whose '(' is at open_paren, or None. Balanced on the
masked text so quotes/comments inside the arguments do not confuse the paren count."""
depth = 0
i = open_paren
n = len(masked)
while i < n:
ch = masked[i]
if ch == "(":
depth += 1
elif ch == ")":
depth -= 1
if depth == 0:
j = i + 1
while j < n and masked[j] != ";":
if not masked[j].isspace():
return None
j += 1
return j if j < n else None
i += 1
return None
@lint_content_check(include=cpp_include)
def lint_esp_log_needs_braces(fname, content):
# Cheap bailout: no log call means nothing to flag, and skips masking the file entirely.
if "ESP_LOG" not in content and "esph_log_" not in content:
return []
masked = _mask_cpp_comments_strings(content)
errors = []
for match in ESP_LOG_NEEDS_BRACES_RE.finditer(masked):
pos = match.start()
line_start = content.rfind("\n", 0, pos) + 1
# Skip preprocessor conditionals (#if/#else/#elif): not C++ control statements.
if content[line_start:pos].lstrip().startswith("#"):
continue
# A '// NOLINT' may sit at the end of the log line (where the message says to put it) or on the
# control-statement line, so scan the whole statement rather than only up to the ESP_LOG token.
stmt_end = _log_statement_end(masked, match.end() - 1)
nolint_end = (
content.find("\n", stmt_end) if stmt_end is not None else match.end()
)
if nolint_end == -1:
nolint_end = len(content)
if "NOLINT" in content[pos:nolint_end]:
continue
snippet = content[pos : match.end()].replace("\n", " ").strip()
errors.append(
(
content.count("\n", 0, pos) + 1,
pos - line_start + 1,
(
f"{highlight(snippet)} - an if/else/for/while body that is a single log "
"call must be wrapped in braces. When the log level compiles the macro out, the "
"body becomes empty and the compiler warns (-Wempty-body). Add { } around the "
"log call (or a '// NOLINT' comment if this is genuinely intended)."
),
)
)
return errors
@lint_content_check(
include=[
"esphome/const.py",
+17
View File
@@ -1104,6 +1104,10 @@ def get_components_per_integration_fixture() -> dict[str, set[str]]:
_TEST_FUNC_RE = re.compile(r"async def (test_\w+)")
# Any usage form (decorator, pytestmark assignment or list element); only
# test_*.py files are scanned, so the marker docs elsewhere cannot false-hit
_SHARED_YAML_USE_RE = re.compile(r"\bmark\.shared_yaml")
_SHARED_YAML_ARG_RE = re.compile(r"\(\s*[\"'](\w+)[\"']\s*\)")
@cache
@@ -1123,6 +1127,19 @@ def get_fixture_to_test_files() -> dict[str, frozenset[str]]:
for func in _TEST_FUNC_RE.findall(content):
base_name = func.replace("test_", "").partition("[")[0]
result.setdefault(base_name, set()).add(rel_path)
# Shared fixtures are named by marker, not by a test function; each
# decorator must carry a string literal or its fixture would silently
# map to no tests
for use in _SHARED_YAML_USE_RE.finditer(content):
arg = _SHARED_YAML_ARG_RE.match(content, use.end())
if arg is None:
line = content.count("\n", 0, use.start()) + 1
raise ValueError(
f"{rel_path}:{line}: shared_yaml marker must take a "
"single-line string literal so CI test selection can map "
"its fixture"
)
result.setdefault(arg.group(1), set()).add(rel_path)
return {k: frozenset(v) for k, v in result.items()}
@@ -0,0 +1,15 @@
esphome:
name: test
esp32:
board: esp32-s3-devkitc-1
variant: esp32s3
spi:
clk_pin: GPIO7
mosi_pin: GPIO9
display:
- platform: epaper_spi
id: epaper_display
model: seeed-reterminal-e1001
@@ -439,6 +439,23 @@ def test_enable_pin_multiple(
assert all(pin["mode"]["output"] is True for pin in enable_pins)
def test_uc8179_e1001_code_generation(
generate_main: Callable[[str | Path], str],
component_config_path: Callable[[str], Path],
) -> None:
"""Test that the reTerminal E1001 model generates the UC8179 driver and init sequence."""
main_cpp = generate_main(component_config_path("uc8179_e1001_test.yaml"))
# The model must instantiate the UC8179 driver class with the panel dimensions
assert "epaper_spi::EPaperUC8179" in main_cpp
assert re.search(r'"SEEED-RETERMINAL-E1001",\s*800,\s*480', main_cpp)
# The generated init sequence must contain the UC8179 resolution setting
# for 800x480: command 0x61, 4 data bytes 0x03 0x20 0x01 0xE0
# (rendered as decimal in the generated array)
assert "97, 4, 3, 32, 1, 224" in main_cpp
def test_enable_pin_code_generation(
generate_main: Callable[[str | Path], str],
component_config_path: Callable[[str], Path],
@@ -0,0 +1,13 @@
esphome:
name: test
esp32:
variant: esp32s31
board: esp32-s31-devkitc
framework:
type: esp-idf
advanced:
execute_from_psram: true
psram:
mode: octal
+28 -4
View File
@@ -203,6 +203,18 @@ def test_esp32_rejects_unsupported_cli_toolchain(
r"'execute_from_psram' requires PSRAM to be configured @ data\['framework'\]\['advanced'\]\['execute_from_psram'\]",
id="execute_from_psram_requires_psram_p4_config",
),
pytest.param(
{
"variant": "esp32s31",
"board": "esp32-s31-devkitc",
"framework": {
"type": "esp-idf",
"advanced": {"execute_from_psram": True},
},
},
r"'execute_from_psram' requires PSRAM to be configured @ data\['framework'\]\['advanced'\]\['execute_from_psram'\]",
id="execute_from_psram_requires_psram_s31_config",
),
pytest.param(
{
"variant": "esp32s3",
@@ -422,12 +434,12 @@ def test_execute_from_psram_s3_sdkconfig(
generate_main: Callable[[str | Path], str],
component_config_path: Callable[[str], Path],
) -> None:
"""Test that execute_from_psram on ESP32-S3 sets the correct sdkconfig options."""
"""Test that execute_from_psram on ESP32-S3 sets the correct sdkconfig option."""
generate_main(component_config_path("execute_from_psram_s3.yaml"))
sdkconfig = CORE.data[KEY_ESP32][KEY_SDKCONFIG_OPTIONS]
assert sdkconfig.get("CONFIG_SPIRAM_FETCH_INSTRUCTIONS") is True
assert sdkconfig.get("CONFIG_SPIRAM_RODATA") is True
assert "CONFIG_SPIRAM_XIP_FROM_PSRAM" not in sdkconfig
assert sdkconfig.get("CONFIG_SPIRAM_XIP_FROM_PSRAM") is True
assert "CONFIG_SPIRAM_FETCH_INSTRUCTIONS" not in sdkconfig
assert "CONFIG_SPIRAM_RODATA" not in sdkconfig
def test_execute_from_psram_p4_sdkconfig(
@@ -442,6 +454,18 @@ def test_execute_from_psram_p4_sdkconfig(
assert "CONFIG_SPIRAM_RODATA" not in sdkconfig
def test_execute_from_psram_s31_sdkconfig(
generate_main: Callable[[str | Path], str],
component_config_path: Callable[[str], Path],
) -> None:
"""Test that execute_from_psram on ESP32-S31 sets the correct sdkconfig option."""
generate_main(component_config_path("execute_from_psram_s31.yaml"))
sdkconfig = CORE.data[KEY_ESP32][KEY_SDKCONFIG_OPTIONS]
assert sdkconfig.get("CONFIG_SPIRAM_XIP_FROM_PSRAM") is True
assert "CONFIG_SPIRAM_FETCH_INSTRUCTIONS" not in sdkconfig
assert "CONFIG_SPIRAM_RODATA" not in sdkconfig
def test_nvs_encryption_sdkconfig(
generate_main: Callable[[str | Path], str],
component_config_path: Callable[[str], Path],
@@ -5,7 +5,11 @@ from __future__ import annotations
import pytest
from esphome import config_validation as cv
from esphome.components.noise import decode_encryption_key, validate_encryption_key
from esphome.components.noise import (
decode_encryption_key,
is_reserved_key,
validate_encryption_key,
)
KEY = "AAECAwQFBgcICQoLDA0ODxAREhMUFRYXGBkaGxwdHh8="
@@ -35,3 +39,8 @@ def test_decode_encryption_key_rejects_short_decode() -> None:
a zero padded PSK on the device."""
with pytest.raises(cv.Invalid, match="32 bytes"):
decode_encryption_key("AAECAw==")
def test_is_reserved_key() -> None:
assert is_reserved_key("AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA=")
assert not is_reserved_key(KEY)
+312 -2
View File
@@ -8,17 +8,25 @@ from typing import Any
import pytest
from esphome import config_validation as cv
from esphome.components.esphome.ota import ota_esphome_final_validate
from esphome.components.esphome.ota import (
AUTO_LOAD,
FILTER_SOURCE_FILES,
_validate_no_password_with_encryption,
ota_esphome_final_validate,
)
from esphome.const import (
CONF_API,
CONF_ENCRYPTION,
CONF_ESPHOME,
CONF_ID,
CONF_KEY,
CONF_OTA,
CONF_PASSWORD,
CONF_PLATFORM,
CONF_PORT,
CONF_VERSION,
)
from esphome.core import ID
from esphome.core import CORE, ID
import esphome.final_validate as fv
@@ -103,3 +111,305 @@ def test_non_esphome_ota_unaffected() -> None:
assert len(updated[CONF_OTA]) == 3
finally:
fv.full_config.reset(token)
API_KEY = "AAECAwQFBgcICQoLDA0ODxAREhMUFRYXGBkaGxwdHh8="
OTHER_KEY = "AQIDBAUGBwgJCgsMDQ4PEBESExQVFhcYGRobHB0eHyA="
ZEROS_KEY = "AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA="
def test_encryption_key_inherited_from_api() -> None:
"""A bare encryption block resolves to the api encryption key."""
full_conf = {
CONF_API: {CONF_ENCRYPTION: {CONF_KEY: API_KEY}},
CONF_OTA: [_make_ota_config(port=3232, **{CONF_ENCRYPTION: {}})],
}
token = fv.full_config.set(full_conf)
try:
ota_esphome_final_validate({})
updated = fv.full_config.get()
assert updated[CONF_OTA][0][CONF_ENCRYPTION][CONF_KEY] == API_KEY
finally:
fv.full_config.reset(token)
def test_encryption_explicit_key_matching_api_accepted() -> None:
"""An explicit ota key equal to the api key validates."""
full_conf = {
CONF_API: {CONF_ENCRYPTION: {CONF_KEY: API_KEY}},
CONF_OTA: [
_make_ota_config(port=3232, **{CONF_ENCRYPTION: {CONF_KEY: API_KEY}})
],
}
token = fv.full_config.set(full_conf)
try:
ota_esphome_final_validate({})
updated = fv.full_config.get()
assert updated[CONF_OTA][0][CONF_ENCRYPTION][CONF_KEY] == API_KEY
finally:
fv.full_config.reset(token)
def test_encryption_key_differing_from_api_rejected() -> None:
"""There is one key per device; an ota key differing from the api key raises."""
full_conf = {
CONF_API: {CONF_ENCRYPTION: {CONF_KEY: API_KEY}},
CONF_OTA: [
_make_ota_config(port=3232, **{CONF_ENCRYPTION: {CONF_KEY: OTHER_KEY}})
],
}
token = fv.full_config.set(full_conf)
try:
with pytest.raises(cv.Invalid, match="must match the 'api' encryption key"):
ota_esphome_final_validate({})
finally:
fv.full_config.reset(token)
def test_encryption_explicit_key_without_api_encryption_accepted() -> None:
"""An explicit ota key with a plaintext api has nothing to match; it stands."""
full_conf = {
CONF_API: {},
CONF_OTA: [
_make_ota_config(port=3232, **{CONF_ENCRYPTION: {CONF_KEY: OTHER_KEY}})
],
}
token = fv.full_config.set(full_conf)
try:
ota_esphome_final_validate({})
updated = fv.full_config.get()
assert updated[CONF_OTA][0][CONF_ENCRYPTION][CONF_KEY] == OTHER_KEY
finally:
fv.full_config.reset(token)
def test_encryption_without_any_key_rejected() -> None:
"""A bare encryption block with no api key to inherit raises."""
full_conf = {
CONF_API: {},
CONF_OTA: [_make_ota_config(port=3232, **{CONF_ENCRYPTION: {}})],
}
token = fv.full_config.set(full_conf)
try:
with pytest.raises(cv.Invalid, match="no 'api' encryption key to inherit"):
ota_esphome_final_validate({})
finally:
fv.full_config.reset(token)
def test_encryption_explicit_all_zeros_key_rejected() -> None:
"""The all-zeros key is the provisioning sentinel; the device would treat
it as no PSK and accept plaintext, so it must fail validation."""
full_conf = {
CONF_OTA: [
_make_ota_config(port=3232, **{CONF_ENCRYPTION: {CONF_KEY: ZEROS_KEY}})
],
}
token = fv.full_config.set(full_conf)
try:
with pytest.raises(cv.Invalid, match="all-zeros key is reserved"):
ota_esphome_final_validate({})
finally:
fv.full_config.reset(token)
def test_encryption_inherited_all_zeros_key_rejected() -> None:
"""An all-zeros api key must not silently disable ota encryption either."""
full_conf = {
CONF_API: {CONF_ENCRYPTION: {CONF_KEY: ZEROS_KEY}},
CONF_OTA: [_make_ota_config(port=3232, **{CONF_ENCRYPTION: {}})],
}
token = fv.full_config.set(full_conf)
try:
with pytest.raises(cv.Invalid, match="all-zeros key is reserved"):
ota_esphome_final_validate({})
finally:
fv.full_config.reset(token)
def test_encryption_key_mismatch_between_merged_configs_rejected() -> None:
"""Same-port configs with different encryption keys raise."""
full_conf = {
CONF_OTA: [
_make_ota_config(port=3232, **{CONF_ENCRYPTION: {CONF_KEY: API_KEY}}),
_make_ota_config(port=3232, **{CONF_ENCRYPTION: {CONF_KEY: OTHER_KEY}}),
]
}
token = fv.full_config.set(full_conf)
try:
with pytest.raises(cv.Invalid, match="encryption is inconsistent"):
ota_esphome_final_validate({})
finally:
fv.full_config.reset(token)
@pytest.mark.parametrize("keyed_first", [True, False])
def test_encryption_bare_and_keyed_blocks_merge(keyed_first: bool) -> None:
"""A bare encryption block (package/device split) is compatible with a
keyed one on the same port; the merge resolves to the keyed result."""
keyed = _make_ota_config(port=3232, **{CONF_ENCRYPTION: {CONF_KEY: OTHER_KEY}})
bare = _make_ota_config(port=3232, **{CONF_ENCRYPTION: {}})
full_conf = {
CONF_OTA: [keyed, bare] if keyed_first else [bare, keyed],
}
token = fv.full_config.set(full_conf)
try:
ota_esphome_final_validate({})
updated = fv.full_config.get()
assert len(updated[CONF_OTA]) == 1
assert updated[CONF_OTA][0][CONF_ENCRYPTION][CONF_KEY] == OTHER_KEY
finally:
fv.full_config.reset(token)
def test_encryption_runtime_provisioned_api_key_not_inheritable() -> None:
"""A keyless api encryption block provisions its key at runtime; a bare
ota encryption block cannot inherit it and the message says so."""
full_conf = {
CONF_API: {CONF_ENCRYPTION: {}},
CONF_OTA: [_make_ota_config(port=3232, **{CONF_ENCRYPTION: {}})],
}
token = fv.full_config.set(full_conf)
try:
with pytest.raises(cv.Invalid, match="provisioned at runtime"):
ota_esphome_final_validate({})
finally:
fv.full_config.reset(token)
def test_encryption_explicit_key_with_runtime_provisioned_api_accepted() -> None:
"""The documented remedy for a runtime-provisioned api key: set an
explicit ota key."""
full_conf = {
CONF_API: {CONF_ENCRYPTION: {}},
CONF_OTA: [
_make_ota_config(port=3232, **{CONF_ENCRYPTION: {CONF_KEY: OTHER_KEY}})
],
}
token = fv.full_config.set(full_conf)
try:
ota_esphome_final_validate({})
updated = fv.full_config.get()
assert updated[CONF_OTA][0][CONF_ENCRYPTION][CONF_KEY] == OTHER_KEY
finally:
fv.full_config.reset(token)
def test_encryption_with_web_server_ota_warns(
caplog: pytest.LogCaptureFixture,
) -> None:
"""With the web_server component the plaintext /update endpoint is always
on; the combination validates with a warning."""
full_conf = {
"web_server": {},
CONF_OTA: [
_make_ota_config(port=3232, **{CONF_ENCRYPTION: {CONF_KEY: OTHER_KEY}}),
{CONF_PLATFORM: "web_server", CONF_ID: ID("ota_ws", is_manual=False)},
],
}
token = fv.full_config.set(full_conf)
try:
with caplog.at_level(logging.WARNING):
ota_esphome_final_validate({})
assert any("plaintext /update" in record.message for record in caplog.records)
finally:
fv.full_config.reset(token)
def test_encryption_with_captive_portal_web_server_ota_warns(
caplog: pytest.LogCaptureFixture,
) -> None:
"""captive_portal auto-loads the web_server ota platform without the
web_server component; encryption stays usable and only warns, so the
fallback AP recovery path is not lost."""
full_conf = {
"captive_portal": {},
CONF_OTA: [
_make_ota_config(port=3232, **{CONF_ENCRYPTION: {CONF_KEY: OTHER_KEY}}),
{CONF_PLATFORM: "web_server", CONF_ID: ID("ota_ws", is_manual=False)},
],
}
token = fv.full_config.set(full_conf)
try:
with caplog.at_level(logging.WARNING):
ota_esphome_final_validate({})
assert any("captive_portal" in record.message for record in caplog.records)
esphome_conf = next(
conf
for conf in fv.full_config.get()[CONF_OTA]
if conf.get(CONF_PLATFORM) == CONF_ESPHOME
)
assert esphome_conf[CONF_ENCRYPTION][CONF_KEY] == OTHER_KEY
finally:
fv.full_config.reset(token)
def test_web_server_ota_without_encryption_unaffected() -> None:
"""web_server ota stays valid alongside an unencrypted esphome entry."""
full_conf = {
CONF_OTA: [
_make_ota_config(port=3232),
{CONF_PLATFORM: "web_server", CONF_ID: ID("ota_ws", is_manual=False)},
],
}
token = fv.full_config.set(full_conf)
try:
ota_esphome_final_validate({})
assert len(fv.full_config.get()[CONF_OTA]) == 2
finally:
fv.full_config.reset(token)
def test_auto_load_pulls_noise_only_for_encryption() -> None:
"""A plain ota entry must never pull noise-c into the build."""
assert AUTO_LOAD({CONF_PORT: 3232}) == ["sha256", "socket"]
assert "noise" in AUTO_LOAD({CONF_ENCRYPTION: {}})
# Tooling probes must get the maximal set: None from dependency
# resolution, {} from the components-graph platform probe
assert "noise" in AUTO_LOAD(None)
assert "noise" in AUTO_LOAD({})
def test_filter_source_files_excludes_noise_without_encryption() -> None:
"""The noise transport source compiles only for encrypted builds."""
old_config = CORE.config
try:
CORE.config = {CONF_OTA: [_make_ota_config(port=3232)]}
assert FILTER_SOURCE_FILES() == ["ota_esphome_noise.cpp"]
CORE.config = {
CONF_OTA: [
_make_ota_config(port=3232, **{CONF_ENCRYPTION: {CONF_KEY: API_KEY}})
]
}
assert FILTER_SOURCE_FILES() == []
finally:
CORE.config = old_config
def test_password_with_encryption_rejected() -> None:
"""The password and encryption options are mutually exclusive."""
config = {CONF_PASSWORD: "pw", CONF_ENCRYPTION: {CONF_KEY: API_KEY}}
with pytest.raises(cv.Invalid, match="cannot be combined"):
_validate_no_password_with_encryption(config)
def test_password_alone_accepted() -> None:
"""A password without encryption still validates."""
config = {CONF_PASSWORD: "pw"}
assert _validate_no_password_with_encryption(config) is config
def test_merged_password_and_encryption_rejected() -> None:
"""A password block and an encryption block merged on one port raise."""
full_conf = {
CONF_OTA: [
_make_ota_config(port=3232, **{CONF_PASSWORD: "pw"}),
_make_ota_config(port=3232, **{CONF_ENCRYPTION: {CONF_KEY: API_KEY}}),
]
}
token = fv.full_config.set(full_conf)
try:
with pytest.raises(cv.Invalid, match="cannot be combined"):
ota_esphome_final_validate({})
finally:
fv.full_config.reset(token)
+3
View File
@@ -0,0 +1,3 @@
sensor:
- platform: d01
name: D01 PM2.5 Concentration
+7
View File
@@ -0,0 +1,7 @@
substitutions:
tx_pin: GPIO4
rx_pin: GPIO5
packages:
uart: !include ../../test_build_components/common/uart/esp32-idf.yaml
d01: !include common.yaml
@@ -0,0 +1,7 @@
substitutions:
tx_pin: GPIO0
rx_pin: GPIO2
packages:
uart: !include ../../test_build_components/common/uart/esp8266-ard.yaml
d01: !include common.yaml
@@ -0,0 +1,7 @@
substitutions:
tx_pin: GPIO4
rx_pin: GPIO5
packages:
uart: !include ../../test_build_components/common/uart/rp2040-ard.yaml
d01: !include common.yaml
+3
View File
@@ -0,0 +1,3 @@
sensor:
- platform: ds1603l
name: ds1603l Distance
@@ -0,0 +1,7 @@
substitutions:
tx_pin: GPIO1
rx_pin: GPIO3
packages:
uart: !include ../../test_build_components/common/uart/esp32-idf.yaml
ds1603l: !include common.yaml
@@ -0,0 +1,7 @@
substitutions:
tx_pin: GPIO0
rx_pin: GPIO2
packages:
uart: !include ../../test_build_components/common/uart/esp8266-ard.yaml
ds1603l: !include common.yaml
@@ -255,3 +255,45 @@ display:
it.filled_rectangle(0, 0, it.get_width(), it.get_height(), Color::WHITE);
it.circle(it.get_width() / 2, it.get_height() / 2, 100, Color::BLACK);
it.circle(it.get_width() / 2, it.get_height() / 2, 60, Color(255, 0, 0));
# Waveshare 7.5" V2 mono (800x480, UC8179 controller, EPD_7in5_V2)
# full_update_every > 1 exercises the fast/partial refresh paths
- platform: epaper_spi
spi_id: spi_bus
model: waveshare-7.5in-v2
full_update_every: 4
cs_pin:
allow_other_uses: true
number: GPIO5
dc_pin:
allow_other_uses: true
number: GPIO17
reset_pin:
allow_other_uses: true
number: GPIO16
busy_pin:
allow_other_uses: true
number: GPIO4
inverted: true
lambda: |-
it.filled_rectangle(0, 0, it.get_width(), it.get_height(), Color::WHITE);
it.circle(it.get_width() / 2, it.get_height() / 2, 100, Color::BLACK);
# Seeed reTerminal E1001 - 7.5" mono e-paper (800x480, UC8179)
# Pins overridden to avoid conflicts with the E1002 defaults above
- platform: epaper_spi
spi_id: spi_bus
model: seeed-reterminal-e1001
cs_pin:
allow_other_uses: true
number: GPIO5
dc_pin:
allow_other_uses: true
number: GPIO17
reset_pin:
allow_other_uses: true
number: GPIO16
busy_pin:
allow_other_uses: true
number: GPIO4
inverted: true
+9
View File
@@ -0,0 +1,9 @@
wifi:
ssid: MySSID
password: password1
ota:
- platform: esphome
port: 3288
encryption:
key: "AAECAwQFBgcICQoLDA0ODxAREhMUFRYXGBkaGxwdHh8="
@@ -0,0 +1,12 @@
wifi:
ssid: MySSID
password: password1
api:
encryption:
key: "AAECAwQFBgcICQoLDA0ODxAREhMUFRYXGBkaGxwdHh8="
ota:
- platform: esphome
port: 3289
encryption:
@@ -0,0 +1,2 @@
packages:
ota: !include encryption.yaml
@@ -0,0 +1,2 @@
packages:
ota: !include encryption.yaml
@@ -0,0 +1,2 @@
packages:
ota: !include encryption.yaml
@@ -0,0 +1,2 @@
packages:
ota: !include encryption_inherit.yaml
+7
View File
@@ -21,6 +21,13 @@ The `yaml_config` fixture automatically loads YAML configurations based on the t
- The fixture file must exist or the test will fail with a clear error message
- The fixture automatically injects a dynamic port number into the API configuration
Tests marked `@pytest.mark.shared_yaml("name")` load `fixtures/name.yaml` instead
of the test-named file and compile it in a shared, hash-keyed build directory, so
the whole group pays one full compile and each test only a relink. The marker
argument must be a single-line string literal (CI test selection maps fixtures to
test files by scanning for it), and marked tests must hand the `yaml_config`
content to `run_compiled` unmodified.
### Key Fixtures
- `run_compiled` - Combines write, compile, and run operations into a single context manager
+335 -74
View File
@@ -4,17 +4,22 @@ from __future__ import annotations
import asyncio
from collections.abc import AsyncGenerator, Callable, Generator
from contextlib import AbstractAsyncContextManager, asynccontextmanager
from contextlib import AbstractAsyncContextManager, asynccontextmanager, suppress
import fcntl
from functools import cache
import hashlib
import logging
import os
from pathlib import Path
import platform
import re
import shutil
import signal
import socket
import subprocess
import sys
import tempfile
import time
from typing import TextIO
from aioesphomeapi import APIClient, APIConnectionError, LogParser, ReconnectLogic
@@ -23,7 +28,13 @@ import pytest_asyncio
import esphome.config
from esphome.core import CORE
from esphome.helpers import get_usable_cpu_count
from esphome.helpers import (
get_usable_cpu_count,
read_file,
rmtree,
write_file,
write_file_if_changed,
)
from esphome.platformio.toolchain import get_idedata
from .const import (
@@ -56,6 +67,21 @@ import pty # not available on Windows
pytest.register_assert_rewrite("tests.integration.entity_utils")
def pytest_configure(config: pytest.Config) -> None:
config.addinivalue_line(
"markers",
"shared_yaml(name): load fixtures/<name>.yaml and compile it in a shared, "
"hash-keyed incremental build directory",
)
FIXTURES_DIR = Path(__file__).parent / "fixtures"
REPO_ROOT = Path(__file__).resolve().parent.parent.parent
# CI caches parts of this path; keep in sync with ci.yml integration-tests.
INTEGRATION_TESTS_ROOT = Path.home() / ".esphome-integration-tests"
def _get_platformio_env(cache_dir: Path) -> dict[str, str]:
"""Get environment variables for PlatformIO with shared cache."""
env = os.environ.copy()
@@ -78,7 +104,7 @@ def _get_platformio_env(cache_dir: Path) -> dict[str, str]:
)
# Compile with THIS tree's esphome sources, not wherever the venv's editable
# install points (which may be a different git worktree or checkout).
repo_root = str(Path(__file__).resolve().parent.parent.parent)
repo_root = str(REPO_ROOT)
existing = env.get("PYTHONPATH")
env["PYTHONPATH"] = f"{repo_root}{os.pathsep}{existing}" if existing else repo_root
return env
@@ -88,8 +114,7 @@ def _get_platformio_env(cache_dir: Path) -> dict[str, str]:
def shared_platformio_cache() -> Generator[Path]:
"""Initialize a shared PlatformIO cache for all integration tests."""
# Use a dedicated directory for integration tests to avoid conflicts.
# CI caches parts of this path; keep in sync with ci.yml integration-tests.
test_cache_dir = Path.home() / ".esphome-integration-tests"
test_cache_dir = INTEGRATION_TESTS_ROOT
cache_dir = test_cache_dir / "platformio"
# Use a lock file in the home directory to ensure only one process initializes the cache
@@ -112,7 +137,9 @@ def shared_platformio_cache() -> Generator[Path]:
init_dir = Path(tmpdir)
fixture_path = Path(__file__).parent / "fixtures" / "cache_init.yaml"
config_path = init_dir / "cache_init.yaml"
config_path.write_text(fixture_path.read_text())
config_path.write_text(
fixture_path.read_text(encoding="utf-8"), encoding="utf-8"
)
# Run compilation to populate the cache
# We must succeed here to avoid race conditions where multiple
@@ -181,21 +208,29 @@ def unused_tcp_port(reserved_tcp_port: tuple[int, socket.socket]) -> int:
return reserved_tcp_port[0]
@pytest.fixture(autouse=True)
def isolated_preferences(monkeypatch: pytest.MonkeyPatch, tmp_path: Path) -> Path:
"""Give every test its own host prefs dir; prefs are keyed only by device
name, which tests sharing a fixture also share."""
prefdir = tmp_path / "prefs"
monkeypatch.setenv("ESPHOME_PREFDIR", str(prefdir))
return prefdir
@pytest_asyncio.fixture
async def yaml_config(request: pytest.FixtureRequest, unused_tcp_port: int) -> str:
"""Load YAML configuration based on test name."""
# Get the test function name
test_name: str = request.node.name
# Extract the base test name (remove test_ prefix and any parametrization)
base_name = test_name.replace("test_", "").partition("[")[0]
shared_name = _shared_yaml_name(request)
# Base test name: test_ prefix and any parametrization stripped
base_name = shared_name or request.node.name.replace("test_", "").partition("[")[0]
# Load the fixture file
fixture_path = Path(__file__).parent / "fixtures" / f"{base_name}.yaml"
fixture_path = FIXTURES_DIR / f"{base_name}.yaml"
if not fixture_path.exists():
raise FileNotFoundError(f"Fixture file not found: {fixture_path}")
loop = asyncio.get_running_loop()
content = await loop.run_in_executor(None, fixture_path.read_text)
content = await loop.run_in_executor(None, read_file, fixture_path)
# Replace the port in the config if it contains api section
if "api:" in content:
@@ -219,11 +254,13 @@ async def yaml_config(request: pytest.FixtureRequest, unused_tcp_port: int) -> s
# Replace external component path placeholder if present
if "EXTERNAL_COMPONENT_PATH" in content:
external_components_path = str(
Path(__file__).parent / "fixtures" / "external_components"
)
external_components_path = str(FIXTURES_DIR / "external_components")
content = content.replace("EXTERNAL_COMPONENT_PATH", external_components_path)
if shared_name is not None:
# _compile verifies the marked test compiles this content unmodified
request.node._shared_yaml_content = content
return content
@@ -233,24 +270,218 @@ async def write_yaml_config(
) -> AsyncGenerator[ConfigWriter]:
"""Write YAML configuration to a file."""
# Get the test name for default filename
test_name = request.node.name
base_name = test_name.replace("test_", "").split("[")[0]
base_name = request.node.name.replace("test_", "").partition("[")[0]
async def _write_config(content: str, filename: str | None = None) -> Path:
if filename is None:
filename = f"{base_name}.yaml"
config_path = integration_test_dir / filename
loop = asyncio.get_running_loop()
await loop.run_in_executor(None, config_path.write_text, content)
await loop.run_in_executor(None, write_file, config_path, content)
return config_path
yield _write_config
# Deliberately not CI-cached (ci.yml caches only platformio/ subpaths); stale
# dirs for a fixture are pruned when its content hash changes.
SHARED_BUILDS_ROOT = INTEGRATION_TESTS_ROOT / "builds"
# In the dir name (not just the hash) so pruning stays inside this checkout
_REPO_KEY = hashlib.sha256(str(REPO_ROOT).encode()).hexdigest()[:8]
# Give a contended shared build lock time for a full cold compile ahead of us
_SHARED_LOCK_TIMEOUT_S = 900
_SHARED_LOCK_POLL_S = 0.1
_SHARED_LOCK_REPORT_S = 30
# Reclaims dirs orphaned by fixture renames or deleted checkouts
_STALE_BUILD_MAX_AGE_S = 30 * 24 * 3600
# ELF path per shared build dir; constant once compiled, so resolve it only once
_shared_elf_paths: dict[Path, Path] = {}
# Dirs this process already swept; pruning is session-scoped work
_pruned_dirs: set[Path] = set()
def _shared_yaml_name(request: pytest.FixtureRequest) -> str | None:
"""Name passed to the shared_yaml marker, or None when unmarked."""
marker = request.node.get_closest_marker("shared_yaml")
if marker is None:
return None
# Exactly one \w+ positional arg: the name doubles as a build dir
# component, and CI test selection (script/helpers.py) parses the same shape
if (
len(marker.args) != 1
or marker.kwargs
or not re.fullmatch(r"\w+", str(marker.args[0]))
):
raise ValueError(
"shared_yaml marker requires exactly one \\w+ fixture name literal"
)
return marker.args[0]
def _shared_build_prefix(name: str) -> str:
return f"{name}-{_REPO_KEY}-"
@cache
def _shared_build_dir(name: str) -> Path:
"""Dir keyed by checkout and fixture source, before per-test injections."""
key = hashlib.sha256((FIXTURES_DIR / f"{name}.yaml").read_bytes()).hexdigest()[:16]
return SHARED_BUILDS_ROOT / (_shared_build_prefix(name) + key)
def _read_stamp(stamp: Path, shared_dir: Path) -> Path | None:
"""ELF path recorded by the last completed compile, or None."""
try:
text = stamp.read_text(encoding="utf-8").strip()
except FileNotFoundError:
return None
except OSError as err:
print(f"Cannot read {stamp}: {err}")
return None
if not text:
print(f"Ignoring empty stamp {stamp}")
return None
built = Path(text)
# Never trust a stamp pointing outside its own build dir as an unlink target
if shared_dir.resolve() in built.resolve().parents:
return built
print(f"Ignoring stamp {stamp} pointing outside {shared_dir}")
return None
def _unused_since(stale: Path, cutoff: float) -> bool:
"""Whether a build dir looks untouched since cutoff; unknown counts as used."""
# Newest of the .built stamp (rewritten by every completed compile) and the
# dir itself (freshened by a worker claiming the dir before locking)
newest: float | None = None
for probe in (stale / ".built", stale):
try:
mtime = probe.stat().st_mtime
except FileNotFoundError:
continue
except NotADirectoryError:
return True # a stray file where a dir should be; reclaimable
except OSError as err:
print(f"Cannot age-probe {stale}: {err}")
return False # unknown never authorizes deletion
newest = mtime if newest is None else max(newest, mtime)
return newest is not None and newest < cutoff
def _prune_stale_builds(name: str, keep: Path) -> None:
"""Remove outdated build dirs (blocking, run in executor): this checkout's
other dirs for the fixture, plus anything untouched for 30 days. Tolerates
other workers pruning the same dirs concurrently."""
cutoff = time.time() - _STALE_BUILD_MAX_AGE_S
prefix = _shared_build_prefix(name)
for stale in SHARED_BUILDS_ROOT.iterdir():
if stale == keep:
continue
same_fixture = stale.name.startswith(prefix)
if not same_fixture and not _unused_since(stale, cutoff):
continue
# Creating .lock bumps the dir mtime, so remember whether the re-probe
# under the lock can trust it
lock_preexisting = (stale / ".lock").exists()
try:
lock_file = (stale / ".lock").open("w")
except FileNotFoundError:
continue # pruned by another worker meanwhile
except NotADirectoryError:
print(f"Removing stray file {stale}")
stale.unlink(missing_ok=True)
continue
except OSError as err:
print(f"Cannot prune {stale}: {err}")
continue
with lock_file:
try:
fcntl.flock(lock_file.fileno(), fcntl.LOCK_EX | fcntl.LOCK_NB)
except BlockingIOError:
continue # still in use by another run
# Re-probe under the lock: a worker freshens its dir before
# locking, so a just-claimed dir no longer looks unused. A dir
# whose .lock we just created cannot be held by anyone, and our
# own open bumped its mtime, so its pre-open probe stands
if (
lock_preexisting
and not same_fixture
and not _unused_since(stale, cutoff)
):
continue
# rmtree tolerates races; a leftover partial tree only costs a
# rebuild, since the ELF is deleted before every compile
try:
rmtree(stale)
except OSError as err:
print(f"Failed to prune {stale}: {err}")
async def _run_esphome_compile(
config_path: Path, cwd: Path, env: dict[str, str]
) -> None:
"""Run `esphome compile`, retrying up to 3 times on a segfault."""
max_retries = 3
for attempt in range(max_retries):
# Compile using subprocess, inheriting stdout/stderr to show progress
proc = await asyncio.create_subprocess_exec(
sys.executable,
"-m",
"esphome",
"compile",
str(config_path),
cwd=cwd,
stdout=None, # Inherit stdout
stderr=None, # Inherit stderr
stdin=asyncio.subprocess.DEVNULL,
# Start in a new process group to isolate signal handling
start_new_session=True,
env=env,
close_fds=False,
)
await proc.wait()
if proc.returncode == 0:
break
if proc.returncode == -11 and attempt < max_retries - 1:
# Segfault (-11 = SIGSEGV), retry
print(
f"Compilation segfaulted (attempt {attempt + 1}/{max_retries}), retrying..."
)
await asyncio.sleep(1) # Brief pause before retry
continue
raise RuntimeError(
f"Failed to compile {config_path}, return code: {proc.returncode}. "
f"Run with 'pytest -s' to see compilation output."
)
def _resolve_compiled_binary(config_path: Path) -> Path:
"""Load the config to learn the compiled ELF path (blocking, run in executor)."""
CORE.reset() # Reset CORE state between test runs
CORE.config_path = config_path
config = esphome.config.read_config(
{"command": "compile", "config": str(config_path)}
)
if config is None:
raise RuntimeError(f"Failed to read config from {config_path}")
idedata = get_idedata(config)
binary_path = Path(idedata.firmware_elf_path)
if not binary_path.exists():
raise RuntimeError(f"Compiled binary not found at {binary_path}")
return binary_path
@pytest_asyncio.fixture
async def compile_esphome(
integration_test_dir: Path,
shared_platformio_cache: Path,
request: pytest.FixtureRequest,
) -> AsyncGenerator[CompileFunction]:
"""Compile an ESPHome configuration and return the binary path."""
@@ -258,66 +489,96 @@ async def compile_esphome(
# Use the shared PlatformIO cache for faster compilation
# This avoids re-downloading dependencies for each test
env = _get_platformio_env(shared_platformio_cache)
# Retry compilation up to 3 times if we get a segfault
max_retries = 3
for attempt in range(max_retries):
# Compile using subprocess, inheriting stdout/stderr to show progress
proc = await asyncio.create_subprocess_exec(
sys.executable,
"-m",
"esphome",
"compile",
str(config_path),
cwd=integration_test_dir,
stdout=None, # Inherit stdout
stderr=None, # Inherit stderr
stdin=asyncio.subprocess.DEVNULL,
# Start in a new process group to isolate signal handling
start_new_session=True,
env=env,
close_fds=False,
)
await proc.wait()
if proc.returncode == 0:
# Success!
break
if proc.returncode == -11 and attempt < max_retries - 1:
# Segfault (-11 = SIGSEGV), retry
print(
f"Compilation segfaulted (attempt {attempt + 1}/{max_retries}), retrying..."
)
await asyncio.sleep(1) # Brief pause before retry
continue
# Other error or final retry
raise RuntimeError(
f"Failed to compile {config_path}, return code: {proc.returncode}. "
f"Run with 'pytest -s' to see compilation output."
)
# Load the config to get idedata (blocking call, must use executor)
loop = asyncio.get_running_loop()
def _read_config_and_get_binary():
CORE.reset() # Reset CORE state between test runs
CORE.config_path = config_path
config = esphome.config.read_config(
{"command": "compile", "config": str(config_path)}
name = _shared_yaml_name(request)
if name is None:
await _run_esphome_compile(config_path, integration_test_dir, env)
return await loop.run_in_executor(
None, _resolve_compiled_binary, config_path
)
if config is None:
raise RuntimeError(f"Failed to read config from {config_path}")
# Get the compiled binary path
idedata = get_idedata(config)
return Path(idedata.firmware_elf_path)
binary_path = await loop.run_in_executor(None, _read_config_and_get_binary)
if not binary_path.exists():
raise RuntimeError(f"Compiled binary not found at {binary_path}")
return binary_path
# Shared fixture: build in a hash-keyed dir so tests sharing a config
# pay one full compile and later only a main.cpp (port) rebuild + relink
shared_dir = _shared_build_dir(name)
shared_dir.mkdir(parents=True, exist_ok=True)
# Freshen the dir before locking so a concurrent age sweep, which
# re-probes under the lock, never reaps a dir a worker just claimed;
# if a peer reaped it already, the guarded lock open recreates it
with suppress(FileNotFoundError):
os.utime(shared_dir)
if shared_dir not in _pruned_dirs:
_pruned_dirs.add(shared_dir)
await loop.run_in_executor(None, _prune_stale_builds, name, shared_dir)
shared_config = shared_dir / f"{name}.yaml"
private_binary = integration_test_dir / f"{name}.elf"
content = await loop.run_in_executor(None, read_file, config_path)
if content != getattr(request.node, "_shared_yaml_content", None):
# The dir is keyed by the fixture source; a mutated config would be
# cached under a hash that does not describe it
raise RuntimeError(
"shared_yaml tests must compile the yaml_config content unmodified"
)
# flock serializes concurrent xdist workers; closing the fd releases it.
# Hand-rolled rather than filelock.FileLock: non-blocking retries keep
# the wait cancellable, while a blocking acquire in an executor thread
# would survive test cancellation holding the fd
try:
lock_file = (shared_dir / ".lock").open("w")
except FileNotFoundError:
# A peer run pruning divergent hashes reaped the dir between our
# mkdir and this open; recreate it and pay a full rebuild
shared_dir.mkdir(parents=True, exist_ok=True)
lock_file = (shared_dir / ".lock").open("w")
with lock_file:
start = time.monotonic()
last_report = start
while True:
try:
fcntl.flock(lock_file.fileno(), fcntl.LOCK_EX | fcntl.LOCK_NB)
break
except BlockingIOError:
now = time.monotonic()
if now - start > _SHARED_LOCK_TIMEOUT_S:
raise RuntimeError(
f"Timed out waiting for the {shared_dir} lock"
) from None
if now - last_report >= _SHARED_LOCK_REPORT_S:
last_report = now
print(
f"Waited {now - start:.0f}s for another worker's "
f"build of {shared_dir.name}"
)
await asyncio.sleep(_SHARED_LOCK_POLL_S)
# .built carries the ELF path of the last completed compile, so
# later workers skip the config re-read in _resolve_compiled_binary
stamp = shared_dir / ".built"
if (built := _shared_elf_paths.get(shared_dir)) is None:
built = await loop.run_in_executor(None, _read_stamp, stamp, shared_dir)
# Delete the ELF before compiling: whatever exists afterwards is
# this compile's output, so no staleness check is ever needed.
# With no usable stamp, sweep any leftover at the known layout
if built is not None:
built.unlink(missing_ok=True)
else:
# Layout-agnostic: ESPHOME_BUILD_PATH can move the build tree
for leftover in shared_dir.rglob("program"):
if leftover.is_file():
leftover.unlink()
await loop.run_in_executor(
None, write_file_if_changed, shared_config, content
)
await _run_esphome_compile(shared_config, shared_dir, env)
if built is None or not built.exists():
built = await loop.run_in_executor(
None, _resolve_compiled_binary, shared_config
)
_shared_elf_paths[shared_dir] = built
await loop.run_in_executor(None, write_file, stamp, str(built))
# Copy out before unlocking: another worker may relink firmware.elf
# while this test is still running its private copy
await loop.run_in_executor(None, shutil.copy2, built, private_binary)
return private_binary
yield _compile
@@ -0,0 +1,11 @@
esphome:
name: host-ota-test
host:
api:
ota:
- platform: esphome
port: __OTA_PORT__
encryption:
key: "AAECAwQFBgcICQoLDA0ODxAREhMUFRYXGBkaGxwdHh8="
logger:
level: DEBUG
@@ -1,58 +0,0 @@
esphome:
name: test-batch-window-filters
host:
api:
batch_delay: 0ms # Disable batching to receive all state updates
logger:
level: DEBUG
# Template sensor that we'll use to publish values
sensor:
- platform: template
name: "Source Sensor"
id: source_sensor
accuracy_decimals: 2
# Batch window filters (window_size == send_every) - use streaming filters
- platform: copy
source_id: source_sensor
name: "Min Sensor"
id: min_sensor
filters:
- min:
window_size: 5
send_every: 5
send_first_at: 1
- platform: copy
source_id: source_sensor
name: "Max Sensor"
id: max_sensor
filters:
- max:
window_size: 5
send_every: 5
send_first_at: 1
- platform: copy
source_id: source_sensor
name: "Moving Avg Sensor"
id: moving_avg_sensor
filters:
- sliding_window_moving_average:
window_size: 5
send_every: 5
send_first_at: 1
# Button to trigger publishing test values
button:
- platform: template
name: "Publish Values Button"
id: publish_button
on_press:
- lambda: |-
// Publish 10 values: 1.0, 2.0, ..., 10.0
for (int i = 1; i <= 10; i++) {
id(source_sensor).publish_state(float(i));
}
@@ -1,111 +0,0 @@
esphome:
name: uart-mock-modbus-cli-rw
host:
api:
logger:
level: VERBOSE
external_components:
- source:
type: local
path: EXTERNAL_COMPONENT_PATH
# Dummy uart entry to satisfy modbus's DEPENDENCIES = ["uart"]
# The actual UART bus used is the uart_mock component below
uart:
baud_rate: 115200
port: /dev/null
# Two virtual buses looped back to each other: the client's transmissions reach the server and the
# server's replies reach the client. auto_start so forwarding is active before the button fires.
uart_mock:
- id: virtual_uart_server
baud_rate: 9600
auto_start: true
debug:
on_tx:
- then:
- uart_mock.inject_rx:
id: virtual_uart_client
data: !lambda return data;
- id: virtual_uart_client
baud_rate: 9600
auto_start: true
debug:
on_tx:
- then:
- uart_mock.inject_rx:
id: virtual_uart_server
data: !lambda return data;
globals:
- id: stored_1
type: uint16_t
initial_value: "0"
modbus:
- uart_id: virtual_uart_server
id: virtual_modbus_server
role: server
- uart_id: virtual_uart_client
id: virtual_modbus_client
role: client
turnaround_time: 10ms
modbus_server:
- address: 1
modbus_id: virtual_modbus_server
registers:
# Writable + readable register: the read publishes what it returns, so the test can confirm the
# write half of the 0x17 ran before the read half (Modbus 6.17).
- address: 0x01
value_type: U_WORD
read_lambda: |-
id(srv_read_1).publish_state(id(stored_1));
return id(stored_1);
write_lambda: |-
id(stored_1) = x;
id(srv_write_1).publish_state(x);
return true;
# Read-only register, returned together with 0x01 by the 2-register read half.
- address: 0x02
value_type: U_WORD
read_lambda: return 0x00AA;
sensor:
# Server-side observations.
- platform: template
name: "srv_write_1"
id: srv_write_1
- platform: template
name: "srv_read_1"
id: srv_read_1
# Client-side read-back: the values the client's on_response received.
- platform: template
name: "client_read_0"
id: client_read_0
- platform: template
name: "client_read_1"
id: client_read_1
button:
- platform: template
name: "Start Scenario"
id: start_scenario_btn
on_press:
# FC 0x17: write reg 0x0001 = 0x1234, then read regs 0x0001..0x0002 back in the same transaction.
- modbus_client.read_write_multiple_registers:
address: 0x01
read_address: 0x0001
read_count: 2
write_address: 0x0001
values: [0x1234]
on_response:
then:
- lambda: |-
// values is the read-back block: reg 0x0001 (must be the just-written 0x1234) and reg 0x0002.
if (values.size() >= 2) {
id(client_read_0).publish_state(values[0]);
id(client_read_1).publish_state(values[1]);
}
@@ -1,88 +0,0 @@
esphome:
name: uart-mock-modbus-custom-pdu
host:
api:
logger:
level: VERBOSE
external_components:
- source:
type: local
path: EXTERNAL_COMPONENT_PATH
# Dummy uart entry to satisfy modbus's DEPENDENCIES = ["uart"]
# The actual UART bus used is the uart_mock component below
uart:
baud_rate: 115200
port: /dev/null
uart_mock:
- id: virtual_uart_server
baud_rate: 9600
auto_start: true
debug:
on_tx:
- then:
- uart_mock.inject_rx:
id: virtual_uart_controller
data: !lambda return data;
- id: virtual_uart_controller
baud_rate: 9600
auto_start: true
debug:
on_tx:
- then:
- uart_mock.inject_rx:
id: virtual_uart_server
data: !lambda return data;
modbus:
- uart_id: virtual_uart_server
id: virtual_modbus_server
role: server
- uart_id: virtual_uart_controller
id: virtual_modbus_controller
role: client
turnaround_time: 10ms
modbus_controller:
- address: 1
modbus_id: virtual_modbus_controller
id: modbus_controller_1
update_interval: 1s
modbus_server:
- address: 1
modbus_id: virtual_modbus_server
id: modbus_server_1
registers:
- address: 0x01
value_type: U_WORD
read_lambda: return 259;
sensor:
# Plain read to confirm the controller <-> server link is up.
- platform: modbus_controller
modbus_controller_id: modbus_controller_1
name: "plain_read"
address: 0x01
register_type: holding
value_type: U_WORD
# Custom PDU: read holding register 0x0001, count 1. The PDU is
# {function code, address hi, address lo, count hi, count lo}; the device
# address and CRC are added by the hub. The lambda parses the response payload
# (the register value, big-endian).
- platform: modbus_controller
modbus_controller_id: modbus_controller_1
name: "custom_read"
custom_pdu: [0x03, 0x00, 0x01, 0x00, 0x01]
lambda: |-
if (data.size() < 2) return {};
return (float) ((data[0] << 8) | data[1]);
button:
- platform: template
name: "Start Scenario"
id: start_scenario_btn
# This test does not have anything to start (mock is autostart)
@@ -1,106 +0,0 @@
esphome:
name: uart-mock-modbus-dep-buffer
host:
api:
logger:
level: VERBOSE
external_components:
- source:
type: local
path: EXTERNAL_COMPONENT_PATH
# Dummy uart entry to satisfy modbus's DEPENDENCIES = ["uart"]
# The actual UART bus used is the uart_mock component below
uart:
baud_rate: 115200
port: /dev/null
uart_mock:
- id: virtual_uart_server
baud_rate: 9600
auto_start: true
debug:
on_tx:
- then:
- uart_mock.inject_rx:
id: virtual_uart_controller
data: !lambda return data;
- id: virtual_uart_controller
baud_rate: 9600
auto_start: true
debug:
on_tx:
- then:
- uart_mock.inject_rx:
id: virtual_uart_server
data: !lambda return data;
globals:
- id: reg10
type: uint16_t
initial_value: "0"
modbus:
- uart_id: virtual_uart_server
id: virtual_modbus_server
role: server
- uart_id: virtual_uart_controller
id: virtual_modbus_controller
role: client
turnaround_time: 10ms
modbus_controller:
- address: 1
modbus_id: virtual_modbus_controller
id: modbus_controller_1
update_interval: 1s
modbus_server:
- address: 1
modbus_id: virtual_modbus_server
id: modbus_server_1
registers:
- address: 0x10
value_type: U_WORD
read_lambda: return id(reg10);
write_lambda: |-
id(reg10) = x;
return true;
# A number whose write_lambda uses the DEPRECATED buffer parameter (fills `payload` with a legacy raw
# frame as words: device address + function code + data) instead of the new item->write_* API. The write
# must still land with its legacy semantics, and the one-time deprecation warning must fire only once per
# entity no matter how many writes happen.
number:
- platform: modbus_controller
modbus_controller_id: modbus_controller_1
name: "buf_number"
id: buf_number
address: 0x10
register_type: holding
value_type: U_WORD
min_value: 0
max_value: 1000
step: 1
write_lambda: |-
// Legacy raw frame as words: [addr 0x01 | fc 0x06], register 0x0010, value.
payload.push_back(0x0106);
payload.push_back(0x0010);
payload.push_back((uint16_t) x);
return {};
# Reports the server-side register so the test can observe that the deprecated buffer write landed.
sensor:
- platform: template
name: "written_value"
id: written_value
update_interval: 0.5s
lambda: "return id(reg10);"
button:
- platform: template
name: "Start Scenario"
id: start_scenario_btn
# The test drives the writes via number_command; the mock is autostart.

Some files were not shown because too many files have changed in this diff Show More