Compare commits

..
Author SHA1 Message Date
Jesse Hills f7f52dc6ef Merge remote-tracking branch 'origin/dev' into jesserockz-2026-584
# Conflicts:
#	esphome/components/api/__init__.py
#	esphome/platformio/library.py
#	platformio.ini
#	tests/unit_tests/test_espidf_clang_tidy.py
2026-09-01 14:35:37 +12:00
Jesse Hills 63336ed377 Merge remote-tracking branch 'origin/dev' into jesserockz-2026-584
# Conflicts:
#	esphome/components/api/__init__.py
#	platformio.ini
2026-08-18 12:00:59 +12:00
Jesse Hills eb53ed5558 [api] Cover the PlatformIO toolchain in the managed component tests
The ESP-IDF framework can also be built with the PlatformIO toolchain, and the
managed components are used on both. Record why that choice is deliberately
toolchain-independent: wireguard splits on the same condition, and if the two
ever disagree one of them converts a second libsodium next to the managed one.

The test now sets the toolchain explicitly and asserts the PlatformIO one takes
the managed path too, so the condition cannot narrow without a test failing.
2026-08-18 07:24:25 +12:00
Jesse Hills 6d14778123 Merge remote-tracking branch 'origin/dev' into jesserockz-2026-584
# Conflicts:
#	esphome/components/api/__init__.py
#	platformio.ini
2026-08-18 07:11:48 +12:00
Jesse Hills eec17043bc Merge remote-tracking branch 'origin/dev' into jesserockz-2026-584 2026-08-13 23:12:59 +12:00
Jesse Hills 041123b14c [api] Import noise-c and libsodium as ESP-IDF managed components
Both libraries now ship their own CMakeLists.txt, so on ESP-IDF they can be
pulled straight from the component registry (noise-c 0.1.15, libsodium
1.10021.2) instead of going through ESPHome's PlatformIO library converter.

A library must not be both converted and managed, or IDF refuses component
discovery, so the converter now takes a set of names the toolchain already
provides. Arduino keeps the converted path: arduino-esp32 brings its own
espressif/libsodium and IDF cannot pick between two managed components whose
names differ only by namespace.
2026-08-13 23:12:47 +12:00
82 changed files with 1839 additions and 3201 deletions
+2 -96
View File
@@ -44,16 +44,6 @@ This document provides essential context for AI models interacting with this pro
## 4. Coding Conventions & Style Guide
**Read the developer documentation before writing a component.** https://developers.esphome.io covers the
component lifecycle, the main loop, and the reasoning behind the rules below in far more depth than this
file does, and it is the authority when they disagree. The most useful starting points:
* https://developers.esphome.io/architecture/components/ - component lifecycle, `setup()`, `loop()`,
setup priorities, and how a component is registered.
* https://developers.esphome.io/architecture/components/advanced/ - choosing between `loop()`,
`set_interval`, `set_timeout` and `defer`; waking the loop from another thread; the RAM cost of each.
* https://developers.esphome.io/contributing/code/ - contribution rules, public API and breaking changes.
* **Formatting:**
* **Python:** Uses `ruff` and `flake8` for linting and formatting. Configuration is in `pyproject.toml`.
* **C++:** Uses `clang-format` for formatting. Configuration is in `.clang-format`.
@@ -152,47 +142,6 @@ file does, and it is the authority when they disagree. The most useful starting
* **Indentation:** Use spaces (two per indentation level), not tabs
* **Type aliases:** Prefer `using type_t = int;` over `typedef int type_t;`
* **Line length:** Wrap lines at no more than 120 characters
* **Timing in `loop()`:** Never call `millis()` in a `loop()` body. The current tick's timestamp is
already cached - use `App.get_loop_component_start_time()` (from `esphome/core/application.h`).
Only reach for `millis()` when you genuinely need sub-tick resolution inside a long operation.
* **The main loop runs every 16 ms.** A rate-limit gate shorter than that does nothing: the check
passes on essentially every pass of the loop, so it costs a comparison and buys nothing. Pick an
interval comfortably coarser than 16 ms, or drop the gate entirely and accept running every loop.
```cpp
// Bad - a 10ms gate against a 16ms loop never holds anything back
static constexpr uint32_t POLL_INTERVAL_MS = 10;
const uint32_t now = millis();
if (now - this->last_poll_ < POLL_INTERVAL_MS)
return;
this->last_poll_ = now;
```
```cpp
// Good - an interval that actually rate limits, off the cached timestamp
static constexpr uint32_t POLL_INTERVAL_MS = 100;
const uint32_t now = App.get_loop_component_start_time();
if (now - this->last_poll_ < POLL_INTERVAL_MS)
return;
this->last_poll_ = now;
```
Pick the primitive by cadence: under 250 ms use a gated `loop()`; 500 ms and above use
`set_interval`. Full reasoning, including why `set_interval` costs more below 500 ms:
https://developers.esphome.io/architecture/components/advanced/#quick-rule-of-thumb
* **Don't override a default with the same value:** if a base class method already returns what you
want, do not override it. `Component::get_setup_priority()` returns `setup_priority::DATA`, so a
component that wants `DATA` should simply leave it alone.
```cpp
// Bad - this is exactly what the base class already does
float get_setup_priority() const override { return setup_priority::DATA; }
```
* **Logging string literals:** wrap literals passed as `%s` arguments in `LOG_STR_LITERAL()` so they
can be stored in flash rather than RAM.
```cpp
// Bad
ESP_LOGV(TAG, "Key %u %s", key, pressed ? "pressed" : "released");
// Good
ESP_LOGV(TAG, "Key %u %s", key, pressed ? LOG_STR_LITERAL("pressed") : LOG_STR_LITERAL("released"));
```
* **Constructor parameters vs setters:** Component properties that are both **required** and **invariant**
(never change after construction) should be constructor parameters rather than set via setter methods.
This makes the dependency explicit and prevents use of the object in an incompletely-initialized state.
@@ -613,33 +562,6 @@ file does, and it is the authority when they disagree. The most useful starting
Use `cg.add_define("MAX_SERVICES", count)` to set the size from Python configuration.
Like `std::array` but with vector-like API (`push_back()`, `size()`) and no STL reallocation code.
**Listener and child-entity registration lists are the most common case, and the most commonly
missed.** A `register_*()` method called once per child at code generation time has a count that
is known at compile time, so it should never be a `std::vector`. Use `cg.slot_counter()`: it
returns a function that each consumer calls once per slot it will occupy, and after every
`to_code` has run it emits the define with the final count. When nothing registers, no define is
emitted and the storage plus its registration method compile out entirely.
```python
# hub component's __init__.py
_request_listener_slot = cg.slot_counter("MY_COMPONENT_LISTENER_COUNT")
async def register_listener(hub: MockObj, var: MockObj) -> None:
_request_listener_slot()
cg.add(hub.register_listener(var))
```
```cpp
#ifdef MY_COMPONENT_LISTENER_COUNT
void register_listener(MyComponentListener *listener);
#endif
protected:
#ifdef MY_COMPONENT_LISTENER_COUNT
StaticVector<MyComponentListener *, MY_COMPONENT_LISTENER_COUNT> listeners_;
#endif
```
Request slots from `to_code`, not from a job that runs after `CoroPriority.FINAL` - a late
request raises rather than silently undercounting.
3. **Runtime-known sizes:** Use `FixedVector` from `esphome/core/helpers.h` when the size is only known at runtime initialization.
```cpp
// Bad - generates STL realloc code (_M_realloc_insert)
@@ -677,25 +599,9 @@ file does, and it is the authority when they disagree. The most useful starting
```
Linear search on small datasets (1-16 elements) is often faster than hashing/tree overhead, but this depends on lookup frequency and access patterns. For frequent lookups in hot code paths, the O(1) vs O(n) complexity difference may still matter even for small datasets. `std::vector` with simple structs is usually fine—it's the heavy containers (`map`, `set`, `unordered_map`) that should be avoided for small datasets unless profiling shows otherwise.
5. **Strings set once from configuration:** Use `StringRef` (`esphome/core/string_ref.h`) rather than
`std::string`. Code generation passes a string literal that lives in flash for the life of the
program, so storing a `std::string` copies it onto the heap for nothing. `StringRef` is a
non-owning pointer plus length; it does not copy, and it must only ever refer to storage that
outlives it (a string literal, or a buffer owned elsewhere).
```cpp
// Bad - heap copy of a literal that is already in flash
void set_keys(std::string keys) { this->keys_ = std::move(keys); }
std::string keys_;
```
```cpp
// Good - no allocation
void set_keys(const char *keys) { this->keys_ = StringRef(keys); }
StringRef keys_;
```
5. **Avoid `std::deque`:** It allocates in 512-byte blocks regardless of element size, guaranteeing at least 512 bytes of RAM usage immediately. This is a major source of crashes on memory-constrained devices.
6. **Avoid `std::deque`:** It allocates in 512-byte blocks regardless of element size, guaranteeing at least 512 bytes of RAM usage immediately. This is a major source of crashes on memory-constrained devices.
7. **Detection:** Look for these patterns in compiler output:
6. **Detection:** Look for these patterns in compiler output:
- Large code sections with STL symbols (vector, map, set)
- `alloc`, `realloc`, `dealloc` in symbol names
- `_M_realloc_insert`, `_M_default_append` (vector reallocation)
-2
View File
@@ -131,7 +131,6 @@ esphome/components/cst816/* @clydebarrow
esphome/components/cst9220/* @clydebarrow
esphome/components/ct_clamp/* @jesserockz
esphome/components/current_based/* @djwmarcx
esphome/components/d01/* @ch604
esphome/components/dac7678/* @NickB1
esphome/components/daikin_arc/* @MagicBear
esphome/components/daikin_brc/* @hagak
@@ -149,7 +148,6 @@ esphome/components/display_menu_base/* @numo68
esphome/components/dlms_meter/* @latonita @PolarGoose @SimonFischer04 @Tomer27cz
esphome/components/dps310/* @kbx81
esphome/components/ds1307/* @badbadc0ffee
esphome/components/ds1603l/* @JakeLC15
esphome/components/ds2484/* @mrk-its
esphome/components/ds248x/* @tomwellnitz
esphome/components/dsmr/* @glmnet @PolarGoose
+3 -35
View File
@@ -23,8 +23,7 @@ For this repository there are two trusted inputs by design:
1. **The configuration.** Anyone who can supply or edit a YAML config is trusted
(see below).
2. **Authenticated peers of a running device** — clients holding the device's
API/OTA encryption key, API password, OTA password, or web server
credentials.
API encryption key / password, OTA password, or web server credentials.
The security boundary is therefore **unauthenticated network traffic vs. those
trusted inputs.** A bug that lets an unauthenticated attacker cross it is a
@@ -77,8 +76,8 @@ These *are* security bugs in this repo, and we want to hear about them privately
captive portal, etc.) **without** valid credentials.
- Authentication or encryption bypass on the device — reaching API calls, OTA
updates, or the web server without the configured key/password.
- Flaws that weaken the device's API or OTA encryption (Noise), OTA auth, or
web server auth below their documented guarantees.
- Flaws that weaken the device's API encryption (Noise), OTA, or web server auth
below their documented guarantees.
## The web server is an open HTTP API by design
@@ -122,37 +121,6 @@ and any memory-safety or protocol bug in the server reachable without credential
This section documents the current design and scope; it is not a judgment that the
design is optimal or that it will not change.
## OTA update encryption
The `esphome` OTA platform optionally encrypts updates with the same Noise
`NNpsk0` pattern the native API uses; one key protects the device. With an
`encryption:` block configured the guarantees are: the firmware image is
confidential in transit, the uploader is authenticated by the pre-shared key,
and the plaintext negotiation preceding the handshake is bound into the
handshake prologue, so stripping or tampering with it fails the first MAC.
Both ends fail closed with no override: a device built with a key refuses
plaintext uploads, and the CLI refuses to send plaintext when a key is
configured.
Defeating any of that without the key is in scope: a keyed device accepting a
plaintext or downgraded upload, getting past the MAC, or recovering image
contents from captured traffic.
The following are **not** vulnerabilities, by design:
- Plaintext OTA on a device with no `encryption:` block. That is the
documented default, authenticated (if at all) by the OTA password.
- The enablement window: turning encryption on takes one last upload of the
encryption-enabled firmware over the existing plaintext channel, with the
pre-existing plaintext exposure.
- The web OTA `/update` endpoint alongside encryption. The `web_server`
component keeps it always reachable, and `captive_portal:` auto-loads it
for the fallback AP window; validation warns about both combinations, and
the operator keeps the recovery path.
- CLI retry behavior on transport or MAC failures; every attempt renegotiates
a fresh handshake with fresh ephemerals, so retrying does not weaken
authentication.
## Explicitly out of scope
- Local attackers who already have shell access on the host that runs `esphome`.
+1 -28
View File
@@ -26,9 +26,7 @@ from esphome.const import (
CONF_DEASSERT_RTS_DTR,
CONF_DISABLED,
CONF_DISCOVER_IP,
CONF_ENCRYPTION,
CONF_ESPHOME,
CONF_KEY,
CONF_LEVEL,
CONF_LOG,
CONF_LOG_TOPIC,
@@ -1338,19 +1336,6 @@ def _upload_via_native_api(
remote_port = int(ota_conf[CONF_PORT])
password = ota_conf.get(CONF_PASSWORD)
# Fail closed: an encryption block whose key did not resolve must never
# fall back to a plaintext upload
noise_psk = None
if (encryption_conf := ota_conf.get(CONF_ENCRYPTION)) is not None:
noise_psk = encryption_conf.get(CONF_KEY)
if not noise_psk:
raise EsphomeError(
"OTA encryption is configured but no key was resolved; "
"set the key under 'ota: encryption:' or 'api: encryption:'"
)
# Ensure the key is a string, as required by the underlying OTA implementation.
# It arrives here as a SensitiveStr which aioesphomeapi rejects.
noise_psk = str(noise_psk)
def check_partition_access(option_string: str) -> None:
if not ota_conf.get("allow_partition_access"):
@@ -1381,9 +1366,7 @@ def _upload_via_native_api(
if ota_type == espota2.OTA_TYPE_UPDATE_BOOTLOADER:
_validate_bootloader_binary(binary)
return espota2.run_ota(
network_devices, remote_port, password, binary, ota_type, noise_psk
)
return espota2.run_ota(network_devices, remote_port, password, binary, ota_type)
def _upload_via_web_server(
@@ -1392,16 +1375,6 @@ def _upload_via_web_server(
from esphome import web_server_ota
from esphome.web_server_helpers import get_web_server_connection
if any(
ota_item.get(CONF_PLATFORM) == CONF_ESPHOME
and ota_item.get(CONF_ENCRYPTION) is not None
for ota_item in config.get(CONF_OTA, [])
):
_LOGGER.warning(
"This config has OTA encryption, but the web_server OTA path sends "
"the image over plaintext HTTP; use the esphome OTA platform to "
"keep it confidential"
)
remote_port, username, password = get_web_server_connection(config)
return web_server_ota.run_ota(
network_devices, remote_port, username, password, binary
-2
View File
@@ -14,7 +14,6 @@ CONF_CHANNEL_COLORS = "channel_colors"
CONF_CLIMATE_ID = "climate_id"
CONF_CO2_EQUIVALENT = "co2_equivalent"
CONF_COLOR_DEPTH = "color_depth"
CONF_COLUMNS = "columns"
CONF_CRC_ENABLE = "crc_enable"
CONF_DATA_BITS = "data_bits"
CONF_DESCRIPTION = "description"
@@ -26,7 +25,6 @@ CONF_GYROSCOPE_RANGE = "gyroscope_range"
CONF_IAQ = "iaq"
CONF_IGNORE_NOT_FOUND = "ignore_not_found"
CONF_IS_WRGB = "is_wrgb"
CONF_KEYS = "keys"
CONF_LABEL = "label"
CONF_LIBRETINY = "libretiny"
CONF_LOOP = "loop"
View File
-45
View File
@@ -1,45 +0,0 @@
#include "d01.h"
#include "esphome/core/log.h"
// uart specification for d01 sensor from https://manuals.plus/ae/1005006417362019:
//
// A frame of serial output data includes 4 bytes, formatted as follows:
// __Characteristic byte: Fixed value 0xA5.
// __Data byte: DATAH is the high 7 bits of the concentration value, and DATAL is the low 7 bits of the concentration
// value.
// __Check byte: The low 7 bits of the sum of all bytes before the check byte.
//
// If the serial output is 4 bytes of data: 0*A5 0*01 0*2C 0*52, then DATAH = 0*01 = 1, DATAL = 0*2C = 44.
// Concentration value = 1*128 + 44 = 172 µg/m³.
//
// The PM2.5 dust concentration value obtained from the dust sensor needs to be calibrated with a K value coefficient
// based on the TSI instrument's photometric method. It is generally recommended to use 0.4.
namespace esphome::d01 {
static const char *const TAG = "d01";
static const uint8_t D01_FRAME_HEADER = 0xA5;
void D01SensorComponent::dump_config() { LOG_SENSOR(" ", "D01 PM2.5", this); }
void D01SensorComponent::loop() {
uint8_t buf[4];
while (this->available() >= 4) {
if (this->peek() != D01_FRAME_HEADER) {
this->read();
continue;
}
this->read_array(buf, 4);
uint8_t sum = (buf[0] + buf[1] + buf[2]) & 0x7F;
if (sum != buf[3]) {
ESP_LOGW(TAG, "checksum mismatch");
continue;
}
uint16_t latest_concentration = (buf[1] & 0x7F) * 128 + (buf[2] & 0x7F);
ESP_LOGV(TAG, "Unadjusted PM2.5 Concentration: %d µg/m³", latest_concentration);
this->publish_state(latest_concentration);
}
}
} // namespace esphome::d01
-14
View File
@@ -1,14 +0,0 @@
#pragma once
#include "esphome/core/component.h"
#include "esphome/components/sensor/sensor.h"
#include "esphome/components/uart/uart.h"
namespace esphome::d01 {
class D01SensorComponent final : public sensor::Sensor, public Component, public uart::UARTDevice {
public:
void dump_config() override;
void loop() override;
};
} // namespace esphome::d01
-45
View File
@@ -1,45 +0,0 @@
import esphome.codegen as cg
from esphome.components import sensor, uart
import esphome.config_validation as cv
from esphome.const import (
DEVICE_CLASS_PM25,
ICON_BLUR,
STATE_CLASS_MEASUREMENT,
UNIT_MICROGRAMS_PER_CUBIC_METER,
)
from esphome.types import ConfigType
CODEOWNERS = ["@ch604"]
DEPENDENCIES = ["uart"]
d01_ns = cg.esphome_ns.namespace("d01")
D01SensorComponent = d01_ns.class_(
"D01SensorComponent", sensor.Sensor, uart.UARTDevice, cg.Component
)
CONFIG_SCHEMA = (
sensor.sensor_schema(
D01SensorComponent,
unit_of_measurement=UNIT_MICROGRAMS_PER_CUBIC_METER,
icon=ICON_BLUR,
accuracy_decimals=0,
device_class=DEVICE_CLASS_PM25,
state_class=STATE_CLASS_MEASUREMENT,
)
.extend(cv.COMPONENT_SCHEMA)
.extend(uart.UART_DEVICE_SCHEMA)
)
FINAL_VALIDATE_SCHEMA = uart.final_validate_device_schema(
"d01",
baud_rate=9600,
require_rx=True,
require_tx=False,
)
async def to_code(config: ConfigType) -> None:
var = await sensor.new_sensor(config)
await cg.register_component(var, config)
await uart.register_uart_device(var, config)
-68
View File
@@ -1,68 +0,0 @@
#include "ds1603l.h"
#include <cstring>
#include "esphome/core/helpers.h"
#include "esphome/core/log.h"
namespace esphome::ds1603l {
static const char *const TAG = "ds1603l.sensor";
void DS1603L::loop() {
// Assemble frames one byte at a time so a stream that starts mid-frame can realign
uint8_t byte;
while (this->available() > 0 && this->read_byte(&byte)) {
if (this->rx_count_ == 0 && byte != HEADER_BYTE) {
ESP_LOGV(TAG, "Skipping byte 0x%02X while looking for header", byte);
continue;
}
this->rx_buffer_[this->rx_count_++] = byte;
if (this->rx_count_ < FRAME_SIZE) {
continue;
}
if (this->parse_data_()) {
this->rx_count_ = 0;
} else {
// The header byte was part of the payload of a misaligned frame, so realign instead of dropping everything
this->resync_();
}
}
}
void DS1603L::dump_config() { LOG_SENSOR("", "DS1603L", this); }
bool DS1603L::parse_data_() {
uint8_t header = this->rx_buffer_[0];
uint8_t data_h = this->rx_buffer_[1];
uint8_t data_l = this->rx_buffer_[2];
uint8_t checksum = this->rx_buffer_[3];
uint8_t computed_checksum = (header + data_h + data_l) & 0xFF;
ESP_LOGV(TAG, "Data: Header=0x%02X, Data_H=0x%02X, Data_L=0x%02X, Checksum=0x%02X", header, data_h, data_l, checksum);
if (checksum != computed_checksum) {
ESP_LOGW(TAG, "Checksum mismatch: received 0x%02X, expected 0x%02X", checksum, computed_checksum);
return false;
}
this->publish_state(encode_uint16(data_h, data_l));
return true;
}
void DS1603L::resync_() {
// Drop the byte that was treated as the header, then look for the next candidate header in what is left
size_t start = 1;
while (start < this->rx_count_ && this->rx_buffer_[start] != HEADER_BYTE) {
start++;
}
this->rx_count_ -= start;
if (this->rx_count_ > 0) {
memmove(this->rx_buffer_, this->rx_buffer_ + start, this->rx_count_);
}
}
} // namespace esphome::ds1603l
-30
View File
@@ -1,30 +0,0 @@
#pragma once
#include <cstddef>
#include <cstdint>
#include "esphome/components/sensor/sensor.h"
#include "esphome/components/uart/uart.h"
#include "esphome/core/component.h"
namespace esphome::ds1603l {
class DS1603L final : public sensor::Sensor, public Component, public uart::UARTDevice {
public:
void loop() override;
void dump_config() override;
protected:
static constexpr uint8_t HEADER_BYTE = 0xFF;
static constexpr size_t FRAME_SIZE = 4;
// Validates the checksum of the frame in rx_buffer_ and publishes it. Returns false if the frame is invalid.
bool parse_data_();
// Drops the first buffered byte and realigns the buffer on the next possible header byte.
void resync_();
uint8_t rx_buffer_[FRAME_SIZE]; // Buffer for the frame being assembled
size_t rx_count_{0}; // Number of bytes currently in rx_buffer_
};
} // namespace esphome::ds1603l
-43
View File
@@ -1,43 +0,0 @@
import esphome.codegen as cg
from esphome.components import sensor, uart
import esphome.config_validation as cv
from esphome.const import (
DEVICE_CLASS_DISTANCE,
STATE_CLASS_MEASUREMENT,
UNIT_MILLIMETER,
)
from esphome.types import ConfigType
CODEOWNERS = ["@JakeLC15"]
DEPENDENCIES = ["uart"]
ds1603l_ns = cg.esphome_ns.namespace("ds1603l")
DS1603L = ds1603l_ns.class_("DS1603L", sensor.Sensor, cg.Component, uart.UARTDevice)
CONFIG_SCHEMA = (
sensor.sensor_schema(
DS1603L,
unit_of_measurement=UNIT_MILLIMETER,
accuracy_decimals=0,
device_class=DEVICE_CLASS_DISTANCE,
state_class=STATE_CLASS_MEASUREMENT,
)
.extend(uart.UART_DEVICE_SCHEMA)
.extend(cv.COMPONENT_SCHEMA)
)
FINAL_VALIDATE_SCHEMA = uart.final_validate_device_schema(
"ds1603l",
baud_rate=9600,
require_tx=False,
require_rx=True,
data_bits=8,
stop_bits=1,
)
async def to_code(config: ConfigType) -> None:
var = await sensor.new_sensor(config)
await cg.register_component(var, config)
await uart.register_uart_device(var, config)
+14 -10
View File
@@ -182,13 +182,6 @@ SIGNED_OTA_V1_ECDSA_VARIANTS = {
VARIANT_ESP32,
}
# Variants that support execution from PSRAM
PSRAM_XIP_VARIANTS = {
VARIANT_ESP32S3,
VARIANT_ESP32P4,
VARIANT_ESP32S31,
}
# NVS encryption (HMAC peripheral scheme) is only available on variants that
# expose the HMAC peripheral (SOC_HMAC_SUPPORTED in soc_caps.h). The original
# ESP32 and ESP32-C2 do not have it. New variants with an HMAC peripheral
@@ -1530,7 +1523,7 @@ def final_validate(config) -> None:
)
)
if advanced[CONF_EXECUTE_FROM_PSRAM]:
if config[CONF_VARIANT] not in PSRAM_XIP_VARIANTS:
if config[CONF_VARIANT] not in {VARIANT_ESP32S3, VARIANT_ESP32P4}:
errs.append(
cv.Invalid(
f"'{CONF_EXECUTE_FROM_PSRAM}' is not available on this esp32 variant",
@@ -2734,7 +2727,13 @@ async def to_code(config):
_configure_lwip_max_sockets(conf)
if advanced[CONF_EXECUTE_FROM_PSRAM]:
add_idf_sdkconfig_option("CONFIG_SPIRAM_XIP_FROM_PSRAM", True)
if variant == VARIANT_ESP32S3:
add_idf_sdkconfig_option("CONFIG_SPIRAM_FETCH_INSTRUCTIONS", True)
add_idf_sdkconfig_option("CONFIG_SPIRAM_RODATA", True)
elif variant == VARIANT_ESP32P4:
add_idf_sdkconfig_option("CONFIG_SPIRAM_XIP_FROM_PSRAM", True)
else:
raise ValueError("Unhandled ESP32 variant")
# Apply LWIP core locking for better socket performance
# This is already enabled by default in Arduino framework, where it provides
@@ -3343,7 +3342,12 @@ def _write_idf_component_yml():
# Don't process arduino libraries
if name not in ARDUINO_DISABLED_LIBRARIES
]
for component in generate_idf_components(libraries):
# A library that is also declared as a managed component must not be
# converted as well, or IDF sees the same requirement from two
# components and refuses to build. Converted components still link
# against it via ${ESPHOME_PROJECT_MANAGED_COMPONENTS}.
managed = set(CORE.data[KEY_ESP32].get(KEY_COMPONENTS, {}))
for component in generate_idf_components(libraries, managed=managed):
dependencies[component.get_sanitized_name()] = {
"override_path": str(component.path)
}
+2 -133
View File
@@ -1,20 +1,12 @@
import logging
import esphome.codegen as cg
from esphome.components.noise import (
decode_encryption_key,
encryption_schema,
is_reserved_key,
)
from esphome.components.ota import BASE_OTA_SCHEMA, OTAComponent, ota_to_code
from esphome.config_helpers import merge_config
import esphome.config_validation as cv
from esphome.const import (
CONF_API,
CONF_ENCRYPTION,
CONF_ESPHOME,
CONF_ID,
CONF_KEY,
CONF_NUM_ATTEMPTS,
CONF_OTA,
CONF_PASSWORD,
@@ -23,7 +15,6 @@ from esphome.const import (
CONF_REBOOT_TIMEOUT,
CONF_SAFE_MODE,
CONF_VERSION,
CONF_WEB_SERVER,
)
from esphome.core import CORE, coroutine_with_priority
from esphome.coroutine import CoroPriority
@@ -31,7 +22,6 @@ import esphome.final_validate as fv
from esphome.types import ConfigType
CONF_ALLOW_PARTITION_ACCESS = "allow_partition_access"
CONF_CAPTIVE_PORTAL = "captive_portal"
_LOGGER = logging.getLogger(__name__)
@@ -40,15 +30,7 @@ CODEOWNERS = ["@esphome/core"]
DEPENDENCIES = ["network"]
def AUTO_LOAD(config: ConfigType) -> list[str]:
"""Auto-load noise only when encryption is configured."""
base = ["sha256", "socket"]
# A falsy config is a tooling probe for the maximal set (None from
# dependency resolution, {} from the components-graph platform probe);
# a validated config always carries defaults, never empty
if not config or CONF_ENCRYPTION in config:
return base + ["noise"]
return base
AUTO_LOAD = ["sha256", "socket"]
esphome = cg.esphome_ns.namespace("esphome")
@@ -85,24 +67,11 @@ def ota_esphome_final_validate(config: ConfigType) -> None:
CONF_PASSWORD in merged_ota_esphome_configs_by_port[conf_port]
and CONF_PASSWORD in ota_conf
and merged_ota_esphome_configs_by_port[conf_port][CONF_PASSWORD]
!= ota_conf[CONF_PASSWORD]
!= ota_conf.get(CONF_PASSWORD)
):
raise cv.Invalid(
f"Found multiple configurations but {CONF_PASSWORD} is inconsistent"
)
# Encryption blocks conflict only when both pin a key; a bare
# `encryption:` (a package/device split) is compatible with a
# keyed one, and merge_config yields the keyed result
merged_key = (
merged_ota_esphome_configs_by_port[conf_port]
.get(CONF_ENCRYPTION, {})
.get(CONF_KEY)
)
other_key = ota_conf.get(CONF_ENCRYPTION, {}).get(CONF_KEY)
if merged_key and other_key and merged_key != other_key:
raise cv.Invalid(
f"Found multiple configurations but {CONF_ENCRYPTION} is inconsistent"
)
ports_with_merged_configs.append(conf_port)
merged_ota_esphome_configs_by_port[conf_port] = merge_config(
@@ -125,20 +94,6 @@ def ota_esphome_final_validate(config: ConfigType) -> None:
new_ota_conf.extend(merged_ota_esphome_configs_by_port.values())
api_conf = full_conf.get(CONF_API) or {}
for ota_conf in merged_ota_esphome_configs_by_port.values():
# Merging same-port blocks can combine a password from one block with
# encryption from another; re-check the exclusion on the merged result.
_validate_no_password_with_encryption(ota_conf)
if (encryption_conf := ota_conf.get(CONF_ENCRYPTION)) is not None:
_resolve_encryption_key(encryption_conf, api_conf)
if any(
conf.get(CONF_PLATFORM) == CONF_WEB_SERVER for conf in full_ota_conf
) and any(
CONF_ENCRYPTION in conf for conf in merged_ota_esphome_configs_by_port.values()
):
_warn_web_server_ota(full_conf)
full_conf[CONF_OTA] = new_ota_conf
fv.full_config.set(full_conf)
@@ -152,73 +107,6 @@ def ota_esphome_final_validate(config: ConfigType) -> None:
)
def _warn_web_server_ota(full_conf: ConfigType) -> None:
"""The web_server ota platform accepts the same image over plaintext HTTP
with basic auth, bypassing the encryption; warn rather than fail so the
operator keeps the recovery path."""
if CONF_CAPTIVE_PORTAL in full_conf and CONF_WEB_SERVER not in full_conf:
# The captive_portal auto-load: the endpoint only exists while the
# fallback AP is active
_LOGGER.warning(
"OTA encryption does not cover the %s OTA platform (auto-loaded "
"by captive_portal); the plaintext /update endpoint stays "
"reachable while the fallback AP is active",
CONF_WEB_SERVER,
)
else:
_LOGGER.warning(
"OTA encryption does not cover the %s OTA platform; its "
"plaintext /update endpoint accepts the same image",
CONF_WEB_SERVER,
)
def _resolve_encryption_key(encryption_conf: ConfigType, api_conf: ConfigType) -> None:
"""Resolve the one encryption key per device into the ota block.
An explicit ota key must match the api key, a bare block inherits it,
a runtime provisioned api key cannot be inherited, and the all-zeros
provisioning sentinel is rejected (the device treats it as no key).
"""
api_key = api_conf.get(CONF_ENCRYPTION, {}).get(CONF_KEY)
if ota_key := encryption_conf.get(CONF_KEY):
if api_key and ota_key != api_key:
raise cv.Invalid(
f"'{CONF_OTA}' {CONF_ENCRYPTION} {CONF_KEY} must match the "
f"'{CONF_API}' {CONF_ENCRYPTION} {CONF_KEY}; omit the "
f"'{CONF_OTA}' {CONF_KEY} to use the '{CONF_API}' one"
)
elif not api_key:
if CONF_ENCRYPTION in api_conf:
raise cv.Invalid(
f"the '{CONF_API}' {CONF_ENCRYPTION} {CONF_KEY} is provisioned at "
f"runtime and cannot be inherited at build time; set an explicit "
f"'{CONF_OTA}' {CONF_ENCRYPTION} {CONF_KEY}"
)
raise cv.Invalid(
f"'{CONF_OTA}' {CONF_ENCRYPTION} has no {CONF_KEY} and there is no "
f"'{CONF_API}' {CONF_ENCRYPTION} {CONF_KEY} to inherit; set one of them"
)
else:
encryption_conf[CONF_KEY] = api_key
if is_reserved_key(encryption_conf[CONF_KEY]):
raise cv.Invalid(
f"The all-zeros {CONF_KEY} is reserved and provides no protection; "
f"generate a real key with: openssl rand -base64 32"
)
# Also called on merged same-port configs in final validate, where schemas
# do not run
def _validate_no_password_with_encryption(config: ConfigType) -> ConfigType:
if CONF_PASSWORD in config and CONF_ENCRYPTION in config:
raise cv.Invalid(
f"'{CONF_PASSWORD}' cannot be combined with '{CONF_ENCRYPTION}'; the "
f"encryption key already authenticates the uploader, remove '{CONF_PASSWORD}'"
)
return config
def _consume_ota_sockets(config: ConfigType) -> ConfigType:
"""Register socket needs for OTA component."""
from esphome.components import socket
@@ -246,7 +134,6 @@ CONFIG_SCHEMA = cv.All(
): cv.port,
cv.Optional(CONF_ALLOW_PARTITION_ACCESS, default=False): cv.boolean,
cv.Optional(CONF_PASSWORD): cv.sensitive(),
cv.Optional(CONF_ENCRYPTION): encryption_schema,
cv.Optional(CONF_NUM_ATTEMPTS): cv.invalid(
f"'{CONF_SAFE_MODE}' (and its related configuration variables) has moved from 'ota' to its own component. See https://esphome.io/components/safe_mode"
),
@@ -260,24 +147,12 @@ CONFIG_SCHEMA = cv.All(
)
.extend(BASE_OTA_SCHEMA)
.extend(cv.COMPONENT_SCHEMA),
_validate_no_password_with_encryption,
_consume_ota_sockets,
)
FINAL_VALIDATE_SCHEMA = ota_esphome_final_validate
def FILTER_SOURCE_FILES() -> list[str]:
"""Filter out the noise transport when no ota entry configures encryption."""
for ota_conf in CORE.config.get(CONF_OTA, []):
if (
ota_conf.get(CONF_PLATFORM) == CONF_ESPHOME
and ota_conf.get(CONF_ENCRYPTION) is not None
):
return []
return ["ota_esphome_noise.cpp"]
@coroutine_with_priority(CoroPriority.OTA_UPDATES)
async def to_code(config: ConfigType) -> None:
var = cg.new_Pvariable(config[CONF_ID])
@@ -296,12 +171,6 @@ async def to_code(config: ConfigType) -> None:
if config.get(CONF_ALLOW_PARTITION_ACCESS):
cg.add_define("USE_OTA_PARTITIONS")
if (encryption_conf := config.get(CONF_ENCRYPTION)) is not None:
# A missing key was resolved from the api component in final validate.
key = encryption_conf[CONF_KEY]
cg.add_define("USE_OTA_ENCRYPTION")
cg.add(var.set_noise_psk(list(decode_encryption_key(key))))
# Build flag so lwip_fast_select.c (a .c file that can't include defines.h) sees it.
cg.add_build_flag("-DUSE_OTA_PLATFORM_ESPHOME")
+26 -95
View File
@@ -27,6 +27,7 @@ namespace esphome {
static const char *const TAG = "esphome.ota";
static constexpr uint16_t OTA_BLOCK_SIZE = 8192;
static constexpr size_t OTA_BUFFER_SIZE = 1024; // buffer size for OTA data transfer
static constexpr uint32_t OTA_SOCKET_TIMEOUT_HANDSHAKE = 20000; // milliseconds for initial handshake
static constexpr uint32_t OTA_SOCKET_TIMEOUT_DATA = 90000; // milliseconds for data transfer
@@ -104,11 +105,6 @@ void ESPHomeOTAComponent::dump_config() {
ESP_LOGCONFIG(TAG, " Password configured");
}
#endif
#ifdef USE_OTA_ENCRYPTION
if (this->noise_ctx_.has_psk()) {
ESP_LOGCONFIG(TAG, " Encryption configured");
}
#endif
#ifdef USE_OTA_PARTITIONS
ESP_LOGCONFIG(TAG,
" Partition access allowed\n"
@@ -153,10 +149,8 @@ void ESPHomeOTAComponent::loop() {
static constexpr uint8_t CLIENT_FEATURE_SUPPORTS_COMPRESSION = 0x01;
static constexpr uint8_t CLIENT_FEATURE_SUPPORTS_SHA256_AUTH = 0x02;
static constexpr uint8_t CLIENT_FEATURE_SUPPORTS_EXTENDED_PROTOCOL = 0x04;
static constexpr uint8_t CLIENT_FEATURE_SUPPORTS_NOISE = 0x08;
static constexpr uint8_t SERVER_FEATURE_SUPPORTS_COMPRESSION = 0x01;
static constexpr uint8_t SERVER_FEATURE_SUPPORTS_PARTITION_ACCESS = 0x02;
static constexpr uint8_t SERVER_FEATURE_SUPPORTS_NOISE = 0x04;
void ESPHomeOTAComponent::handle_handshake_() {
/// Handle the OTA handshake and authentication.
@@ -208,7 +202,8 @@ void ESPHomeOTAComponent::handle_handshake_() {
}
// Validate magic bytes
if (memcmp(this->handshake_buf_, MAGIC_BYTES, sizeof(MAGIC_BYTES)) != 0) {
static const uint8_t MAGIC_BYTES[5] = {0x6C, 0x26, 0xF7, 0x5C, 0x45};
if (memcmp(this->handshake_buf_, MAGIC_BYTES, 5) != 0) {
ESP_LOGW(TAG, "Magic bytes mismatch! 0x%02X-0x%02X-0x%02X-0x%02X-0x%02X", this->handshake_buf_[0],
this->handshake_buf_[1], this->handshake_buf_[2], this->handshake_buf_[3], this->handshake_buf_[4]);
this->send_error_and_cleanup_(ota::OTA_RESPONSE_ERROR_MAGIC);
@@ -240,19 +235,6 @@ void ESPHomeOTAComponent::handle_handshake_() {
}
this->ota_features_ = this->handshake_buf_[0];
ESP_LOGV(TAG, "Features: 0x%02X", this->ota_features_);
#ifdef USE_OTA_ENCRYPTION
// Fail closed: with a PSK configured the client must negotiate encryption
// (which requires the extended protocol); refuse plaintext uploads.
static constexpr uint8_t NOISE_REQUIRED_FEATURES =
CLIENT_FEATURE_SUPPORTS_NOISE | CLIENT_FEATURE_SUPPORTS_EXTENDED_PROTOCOL;
if (this->noise_ctx_.has_psk() && (this->ota_features_ & NOISE_REQUIRED_FEATURES) != NOISE_REQUIRED_FEATURES) {
ESP_LOGW(TAG, "Client does not support encryption");
this->send_error_and_cleanup_(ota::OTA_RESPONSE_ERROR_ENCRYPTION_REQUIRED);
return;
}
#endif
this->transition_ota_state_(OTAState::FEATURE_ACK);
const bool supports_compression =
@@ -268,11 +250,6 @@ void ESPHomeOTAComponent::handle_handshake_() {
this->handshake_buf_[1] = (supports_compression ? SERVER_FEATURE_SUPPORTS_COMPRESSION : 0);
#ifdef USE_OTA_PARTITIONS
this->handshake_buf_[1] |= SERVER_FEATURE_SUPPORTS_PARTITION_ACCESS;
#endif
#ifdef USE_OTA_ENCRYPTION
if (this->noise_ctx_.has_psk()) {
this->handshake_buf_[1] |= SERVER_FEATURE_SUPPORTS_NOISE;
}
#endif
} else {
this->handshake_buf_[0] =
@@ -288,20 +265,6 @@ void ESPHomeOTAComponent::handle_handshake_() {
if (!this->try_write_(ack_size, LOG_STR("ack feature"))) {
return;
}
#ifdef USE_OTA_ENCRYPTION
// With a PSK configured the rest of the session runs inside the noise
// transport; the client sends the first handshake frame next, so there
// is nothing to do until data arrives.
if (this->noise_ctx_.has_psk()) {
// handshake_buf_ still holds the feature ack composed above; a
// would-block re-entry lands here without rebuilding it
if (!this->noise_start_session_(this->handshake_buf_[1])) {
return;
}
this->transition_ota_state_(OTAState::NOISE_HANDSHAKE);
return;
}
#endif
#ifdef USE_OTA_PASSWORD
// If password is set, move to auth phase
if (!this->password_.empty()) {
@@ -339,16 +302,6 @@ void ESPHomeOTAComponent::handle_handshake_() {
this->handle_data_();
return;
#ifdef USE_OTA_ENCRYPTION
case OTAState::NOISE_HANDSHAKE:
if (!this->handle_noise_handshake_()) {
return;
}
this->transition_ota_state_(OTAState::DATA);
this->handle_data_();
return;
#endif
default:
break;
}
@@ -387,8 +340,6 @@ void ESPHomeOTAComponent::handle_data_() {
/// Raw TCP (8266, RP2040): setblocking is no-op; SO_RCVTIMEO uses
/// wakeable_delay() in read();
/// write() always returns immediately
// Backend calls overwrite this with OK; reset to UNKNOWN before any
// goto error that follows a successful begin()/write()
ota::OTAResponseTypes error_code = ota::OTA_RESPONSE_ERROR_UNKNOWN;
size_t total = 0;
uint32_t last_progress = 0;
@@ -410,11 +361,11 @@ void ESPHomeOTAComponent::handle_data_() {
this->client_->setblocking(true);
// Acknowledge auth OK - 1 byte
this->data_write_byte_(ota::OTA_RESPONSE_AUTH_OK);
this->write_byte_(ota::OTA_RESPONSE_AUTH_OK);
if (this->extended_proto_) {
// Read ota type, 1 byte
if (!this->data_readall_(buf, 1)) {
if (!this->readall_(buf, 1)) {
this->log_read_error_(LOG_STR("OTA type"));
goto error; // NOLINT(cppcoreguidelines-avoid-goto)
}
@@ -423,7 +374,7 @@ void ESPHomeOTAComponent::handle_data_() {
ESP_LOGV(TAG, "OTA type is 0x%02x", ota_type);
// Read size, 4 bytes MSB first
if (!this->data_readall_(buf, 4)) {
if (!this->readall_(buf, 4)) {
this->log_read_error_(LOG_STR("size"));
goto error; // NOLINT(cppcoreguidelines-avoid-goto)
}
@@ -454,12 +405,11 @@ void ESPHomeOTAComponent::handle_data_() {
goto error; // NOLINT(cppcoreguidelines-avoid-goto)
// Acknowledge prepare OK - 1 byte
this->data_write_byte_(ota::OTA_RESPONSE_UPDATE_PREPARE_OK);
this->write_byte_(ota::OTA_RESPONSE_UPDATE_PREPARE_OK);
// Read binary MD5, 32 bytes
if (!this->data_readall_(buf, 32)) {
if (!this->readall_(buf, 32)) {
this->log_read_error_(LOG_STR("MD5 checksum"));
error_code = ota::OTA_RESPONSE_ERROR_UNKNOWN;
goto error; // NOLINT(cppcoreguidelines-avoid-goto)
}
sbuf[32] = '\0';
@@ -467,7 +417,7 @@ void ESPHomeOTAComponent::handle_data_() {
this->backend_->set_update_md5(sbuf);
// Acknowledge MD5 OK - 1 byte
this->data_write_byte_(ota::OTA_RESPONSE_BIN_MD5_OK);
this->write_byte_(ota::OTA_RESPONSE_BIN_MD5_OK);
// Track when we last received data so a silently-vanished peer (no FIN/RST
// delivered, e.g. uploader killed mid-transfer or NAT/router dropped state)
@@ -483,35 +433,19 @@ void ESPHomeOTAComponent::handle_data_() {
}
size_t remaining = ota_size - total;
size_t requested = remaining < OTA_BUFFER_SIZE ? remaining : OTA_BUFFER_SIZE;
ssize_t read;
#ifdef USE_OTA_ENCRYPTION
if (this->noise_ != nullptr) {
// One frame per call; noise_read_data_ waits internally (readall_), so
// there is no would-block retry here and failures are already logged.
read = this->noise_read_data_(buf, requested);
if (read <= 0) {
error_code = ota::OTA_RESPONSE_ERROR_UNKNOWN;
goto error; // NOLINT(cppcoreguidelines-avoid-goto)
}
} else
#endif
{
read = this->client_->read(buf, requested);
if (read == -1) {
const int err = errno;
if (this->would_block_(err)) {
// read() already waited up to SO_RCVTIMEO for data, just feed WDT
App.feed_wdt();
continue;
}
ESP_LOGW(TAG, "Read err %d", err);
error_code = ota::OTA_RESPONSE_ERROR_UNKNOWN;
goto error; // NOLINT(cppcoreguidelines-avoid-goto)
} else if (read == 0) {
ESP_LOGW(TAG, "Remote closed");
error_code = ota::OTA_RESPONSE_ERROR_UNKNOWN;
goto error; // NOLINT(cppcoreguidelines-avoid-goto)
ssize_t read = this->client_->read(buf, requested);
if (read == -1) {
const int err = errno;
if (this->would_block_(err)) {
// read() already waited up to SO_RCVTIMEO for data, just feed WDT
App.feed_wdt();
continue;
}
ESP_LOGW(TAG, "Read err %d", err);
goto error; // NOLINT(cppcoreguidelines-avoid-goto)
} else if (read == 0) {
ESP_LOGW(TAG, "Remote closed");
goto error; // NOLINT(cppcoreguidelines-avoid-goto)
}
last_data_ms = millis();
@@ -523,7 +457,7 @@ void ESPHomeOTAComponent::handle_data_() {
total += read;
#if USE_OTA_VERSION == 2
while (size_acknowledged + OTA_BLOCK_SIZE <= total || (total == ota_size && size_acknowledged < ota_size)) {
this->data_write_byte_(ota::OTA_RESPONSE_CHUNK_OK);
this->write_byte_(ota::OTA_RESPONSE_CHUNK_OK);
size_acknowledged += OTA_BLOCK_SIZE;
}
#endif
@@ -542,7 +476,7 @@ void ESPHomeOTAComponent::handle_data_() {
}
// Acknowledge receive OK - 1 byte
this->data_write_byte_(ota::OTA_RESPONSE_RECEIVE_OK);
this->write_byte_(ota::OTA_RESPONSE_RECEIVE_OK);
error_code = this->backend_->end();
if (error_code != ota::OTA_RESPONSE_OK) {
@@ -551,10 +485,10 @@ void ESPHomeOTAComponent::handle_data_() {
}
// Acknowledge Update end OK - 1 byte
this->data_write_byte_(ota::OTA_RESPONSE_UPDATE_END_OK);
this->write_byte_(ota::OTA_RESPONSE_UPDATE_END_OK);
// Read ACK
if (!this->data_readall_(buf, 1) || buf[0] != ota::OTA_RESPONSE_OK) {
if (!this->readall_(buf, 1) || buf[0] != ota::OTA_RESPONSE_OK) {
this->log_read_error_(LOG_STR("ack"));
// do not go to error, this is not fatal
}
@@ -577,7 +511,7 @@ void ESPHomeOTAComponent::handle_data_() {
App.safe_reboot();
error:
this->data_write_byte_(static_cast<uint8_t>(error_code));
this->write_byte_(static_cast<uint8_t>(error_code));
// Abort backend before cleanup - cleanup_connection_() destroys the backend.
// Always call abort() unconditionally: backends register external partitions before
@@ -744,9 +678,6 @@ void ESPHomeOTAComponent::cleanup_connection_() {
this->backend_ = nullptr;
#ifdef USE_OTA_PASSWORD
this->cleanup_auth_();
#endif
#ifdef USE_OTA_ENCRYPTION
this->noise_ = nullptr;
#endif
// Intentionally no disable_loop() — letting loop() run one more iteration catches
// any connection that queued on the listener mid-session (otherwise the wake flag,
+1 -69
View File
@@ -4,9 +4,6 @@
#ifdef USE_OTA
#include "esphome/components/ota/ota_backend_factory.h"
#include "esphome/components/socket/socket.h"
#ifdef USE_OTA_ENCRYPTION
#include "esphome/components/noise/noise_handshake.h"
#endif
#include "esphome/core/helpers.h"
#include "esphome/core/log.h"
#include "esphome/core/preferences.h"
@@ -27,10 +24,7 @@ class ESPHomeOTAComponent final : public ota::OTAComponent {
AUTH_SEND, // Sending authentication request
AUTH_READ, // Reading authentication data
#endif // USE_OTA_PASSWORD
#ifdef USE_OTA_ENCRYPTION
NOISE_HANDSHAKE, // Exchanging Noise handshake frames
#endif
DATA, // BLOCKING! Processing OTA data (update, etc.)
DATA, // BLOCKING! Processing OTA data (update, etc.)
};
#ifdef USE_OTA_PASSWORD
void set_auth_password(const std::string &password) { password_ = password; }
@@ -44,10 +38,6 @@ class ESPHomeOTAComponent final : public ota::OTAComponent {
}
#endif // USE_OTA_PASSWORD
#ifdef USE_OTA_ENCRYPTION
void set_noise_psk(noise::psk_t psk) { this->noise_ctx_.set_psk(psk); }
#endif
/// Manually set the port OTA should listen on
void set_port(uint16_t port) { this->port_ = port; }
@@ -73,48 +63,6 @@ class ESPHomeOTAComponent final : public ota::OTAComponent {
bool writeall_(const uint8_t *buf, size_t len);
inline bool write_byte_(uint8_t byte) { return this->writeall_(&byte, 1); }
#ifdef USE_OTA_ENCRYPTION
// Heap-allocated only while an encrypted OTA session is active.
struct NoiseSession {
~NoiseSession();
noise::NoiseResponderHandshake handshake;
NoiseCipherState *send_cipher{nullptr};
NoiseCipherState *recv_cipher{nullptr};
uint16_t frame_len{0}; // total frame size once the header is parsed, 0 until then
uint16_t frame_pos{0}; // bytes read or written so far
bool writing{false}; // a produced handshake frame is still being flushed
uint8_t frame_buf[noise::FRAME_HEADER_SIZE + 1 + noise::MAX_HANDSHAKE_SIZE];
};
bool noise_start_session_(uint8_t server_feature_flags);
bool handle_noise_handshake_();
bool noise_try_read_frame_();
bool noise_try_write_frame_();
void noise_send_reject_(const LogString *reason);
ssize_t noise_decrypt_(uint8_t *buf, size_t len);
ssize_t noise_read_frame_blocking_(uint8_t *buf, size_t min_ciphertext, size_t max_ciphertext);
bool noise_readall_(uint8_t *buf, size_t len);
ssize_t noise_read_data_(uint8_t *buf, size_t capacity);
bool noise_write_byte_(uint8_t byte);
#endif // USE_OTA_ENCRYPTION
// Data-phase I/O dispatch: through the noise transport when a session is
// active, straight to the socket otherwise.
inline bool data_write_byte_(uint8_t byte) {
#ifdef USE_OTA_ENCRYPTION
if (this->noise_ != nullptr)
return this->noise_write_byte_(byte);
#endif
return this->write_byte_(byte);
}
// When encrypted, buf must have room for len + noise::MAC_SIZE bytes.
inline bool data_readall_(uint8_t *buf, size_t len) {
#ifdef USE_OTA_ENCRYPTION
if (this->noise_ != nullptr)
return this->noise_readall_(buf, len);
#endif
return this->readall_(buf, len);
}
bool try_read_(size_t to_read, const LogString *desc);
bool try_write_(size_t to_write, const LogString *desc);
@@ -143,10 +91,6 @@ class ESPHomeOTAComponent final : public ota::OTAComponent {
std::string password_;
std::unique_ptr<uint8_t[]> auth_buf_;
#endif // USE_OTA_PASSWORD
#ifdef USE_OTA_ENCRYPTION
noise::NoiseContext noise_ctx_;
std::unique_ptr<NoiseSession> noise_;
#endif // USE_OTA_ENCRYPTION
socket::ListenSocket *server_{nullptr};
std::unique_ptr<socket::Socket> client_;
@@ -154,18 +98,6 @@ class ESPHomeOTAComponent final : public ota::OTAComponent {
uint32_t client_connect_time_{0};
static constexpr size_t HANDSHAKE_BUF_SIZE = 5;
// Buffer size for OTA data transfer. The upload client derives its maximum
// encrypted frame plaintext from this (espota2.NOISE_MAX_PLAINTEXT is this
// minus the 16-byte MAC); both must change together.
static constexpr size_t OTA_BUFFER_SIZE = 1040;
#ifdef USE_OTA_ENCRYPTION
// espota2.NOISE_MAX_PLAINTEXT; shrinking the buffer would reject every
// frame a current CLI sends
static constexpr size_t NOISE_CLIENT_MAX_PLAINTEXT = 1024;
static_assert(OTA_BUFFER_SIZE >= NOISE_CLIENT_MAX_PLAINTEXT + noise::MAC_SIZE,
"OTA_BUFFER_SIZE must fit a full encrypted data frame");
#endif
static constexpr uint8_t MAGIC_BYTES[5] = {0x6C, 0x26, 0xF7, 0x5C, 0x45};
#ifdef USE_OTA_PARTITIONS
uint32_t running_app_offset_{0};
size_t running_app_size_{0};
@@ -1,279 +0,0 @@
#include "ota_esphome.h"
#ifdef USE_OTA
#ifdef USE_OTA_ENCRYPTION
#include "esphome/components/noise/noise.h"
#include "esphome/components/ota/ota_backend.h"
#include "esphome/core/log.h"
#include <cstring>
#include <new>
#ifdef USE_ESP8266
#include <pgmspace.h>
#endif
namespace esphome {
static const char *const TAG = "esphome.ota";
#ifdef USE_ESP8266
static constexpr char OTA_NOISE_PROLOGUE_INIT[] PROGMEM = "NoiseOTAInit";
#else
static constexpr char OTA_NOISE_PROLOGUE_INIT[] = "NoiseOTAInit";
#endif
static constexpr size_t OTA_NOISE_PROLOGUE_INIT_LEN = sizeof(OTA_NOISE_PROLOGUE_INIT) - 1;
ESPHomeOTAComponent::NoiseSession::~NoiseSession() {
if (this->send_cipher != nullptr) {
noise_cipherstate_free(this->send_cipher);
}
if (this->recv_cipher != nullptr) {
noise_cipherstate_free(this->recv_cipher);
}
}
/** Allocate the session and start the responder handshake.
*
* The prologue binds the whole plaintext preamble, so any tampering with the
* negotiation (a stripped feature flag, a changed version) breaks the first
* handshake MAC on either side:
* "NoiseOTAInit" | magic(5) | OK,version | client_features | FEATURE_FLAGS,server_flags
*/
bool ESPHomeOTAComponent::noise_start_session_(uint8_t server_feature_flags) {
// NOLINTNEXTLINE(clang-analyzer-cplusplus.NewDeleteLeaks)
this->noise_ = std::unique_ptr<NoiseSession>(new (std::nothrow) NoiseSession());
if (this->noise_ == nullptr) {
ESP_LOGW(TAG, "Session allocation failed");
this->cleanup_connection_();
return false;
}
static constexpr size_t PROLOGUE_ACK_LEN = 2; // OTA_RESPONSE_OK + version
static constexpr size_t PROLOGUE_CLIENT_FEATURES_LEN = 1;
static constexpr size_t PROLOGUE_FEATURE_ACK_LEN = 2; // OTA_RESPONSE_FEATURE_FLAGS + server flags
uint8_t prologue[OTA_NOISE_PROLOGUE_INIT_LEN + sizeof(MAGIC_BYTES) + PROLOGUE_ACK_LEN + PROLOGUE_CLIENT_FEATURES_LEN +
PROLOGUE_FEATURE_ACK_LEN];
#ifdef USE_ESP8266
memcpy_P(prologue, OTA_NOISE_PROLOGUE_INIT, OTA_NOISE_PROLOGUE_INIT_LEN);
#else
std::memcpy(prologue, OTA_NOISE_PROLOGUE_INIT, OTA_NOISE_PROLOGUE_INIT_LEN);
#endif
uint8_t *p = prologue + OTA_NOISE_PROLOGUE_INIT_LEN;
// Magic bytes, already validated in MAGIC_READ
std::memcpy(p, MAGIC_BYTES, sizeof(MAGIC_BYTES));
p += sizeof(MAGIC_BYTES);
// Our magic ack
*p++ = ota::OTA_RESPONSE_OK;
*p++ = USE_OTA_VERSION;
// The feature byte the client sent
*p++ = this->ota_features_;
// The feature ack we sent (noise requires the extended protocol)
*p++ = ota::OTA_RESPONSE_FEATURE_FLAGS;
*p++ = server_feature_flags;
int err = this->noise_->handshake.init(this->noise_ctx_.get_psk(), prologue, sizeof(prologue));
if (err != 0) {
ESP_LOGW(TAG, "Handshake init: %s", LOG_STR_ARG(noise::noise_err_to_logstr(err)));
this->cleanup_connection_();
return false;
}
return true;
}
/** Drive the non-blocking handshake from loop(); returns true once the
* transport ciphers are ready. A would-block returns false and the next
* loop() resumes from the NoiseSession cursors; on failure the connection
* is cleaned up.
*/
bool ESPHomeOTAComponent::handle_noise_handshake_() {
NoiseSession &s = *this->noise_;
while (true) {
if (s.writing) {
if (!this->noise_try_write_frame_()) {
return false; // would block, or errored and cleaned up
}
s.writing = false;
s.frame_pos = 0;
s.frame_len = 0;
}
switch (s.handshake.action()) {
case noise::NoiseResponderHandshake::Action::ACTION_READ: {
if (!this->noise_try_read_frame_()) {
return false;
}
const uint16_t payload_len = s.frame_len - noise::FRAME_HEADER_SIZE;
s.frame_pos = 0;
s.frame_len = 0;
if (s.frame_buf[noise::FRAME_HEADER_SIZE] != noise::HANDSHAKE_STATUS_OK) {
ESP_LOGW(TAG, "Bad handshake error byte: %u", s.frame_buf[noise::FRAME_HEADER_SIZE]);
this->cleanup_connection_();
return false;
}
int err = s.handshake.read_message(s.frame_buf + noise::FRAME_HEADER_SIZE + 1, payload_len - 1);
if (err != 0) {
ESP_LOGW(TAG, "Handshake read: %s", LOG_STR_ARG(noise::noise_err_to_logstr(err)));
this->noise_send_reject_(noise::reject_reason_for(err));
this->cleanup_connection_();
return false;
}
break;
}
case noise::NoiseResponderHandshake::Action::ACTION_WRITE: {
size_t msg_len = 0;
int err =
s.handshake.write_message(s.frame_buf + noise::FRAME_HEADER_SIZE + 1, noise::MAX_HANDSHAKE_SIZE, msg_len);
if (err != 0) {
ESP_LOGW(TAG, "Handshake write: %s", LOG_STR_ARG(noise::noise_err_to_logstr(err)));
this->cleanup_connection_();
return false;
}
const uint16_t payload_len = msg_len + 1;
noise::write_frame_header(s.frame_buf, payload_len);
s.frame_buf[noise::FRAME_HEADER_SIZE] = noise::HANDSHAKE_STATUS_OK;
s.frame_len = noise::FRAME_HEADER_SIZE + payload_len;
s.frame_pos = 0;
s.writing = true;
break;
}
case noise::NoiseResponderHandshake::Action::ACTION_SPLIT: {
int err = s.handshake.split(s.send_cipher, s.recv_cipher);
if (err != 0) {
ESP_LOGW(TAG, "Handshake split: %s", LOG_STR_ARG(noise::noise_err_to_logstr(err)));
this->cleanup_connection_();
return false;
}
ESP_LOGD(TAG, "Noise handshake complete");
return true;
}
default: {
ESP_LOGW(TAG, "Bad handshake state");
this->cleanup_connection_();
return false;
}
}
}
}
/// Non-blocking read of one handshake frame into the session buffer.
bool ESPHomeOTAComponent::noise_try_read_frame_() {
NoiseSession &s = *this->noise_;
while (s.frame_pos < noise::FRAME_HEADER_SIZE) {
ssize_t read = this->client_->read(s.frame_buf + s.frame_pos, noise::FRAME_HEADER_SIZE - s.frame_pos);
if (!this->handle_read_error_(read, LOG_STR("read noise header"))) {
return false;
}
s.frame_pos += read;
}
if (s.frame_len == 0) {
const uint16_t payload_len = encode_uint16(s.frame_buf[1], s.frame_buf[2]);
if (s.frame_buf[0] != noise::FRAME_INDICATOR || payload_len < 1 || payload_len > 1 + noise::MAX_HANDSHAKE_SIZE) {
ESP_LOGW(TAG, "Bad handshake frame: 0x%02X, %u bytes", s.frame_buf[0], payload_len);
this->cleanup_connection_();
return false;
}
s.frame_len = noise::FRAME_HEADER_SIZE + payload_len;
}
while (s.frame_pos < s.frame_len) {
ssize_t read = this->client_->read(s.frame_buf + s.frame_pos, s.frame_len - s.frame_pos);
if (!this->handle_read_error_(read, LOG_STR("read noise frame"))) {
return false;
}
s.frame_pos += read;
}
return true;
}
/// Non-blocking write of the pending session-buffer frame.
bool ESPHomeOTAComponent::noise_try_write_frame_() {
NoiseSession &s = *this->noise_;
while (s.frame_pos < s.frame_len) {
ssize_t written = this->client_->write(s.frame_buf + s.frame_pos, s.frame_len - s.frame_pos);
if (!this->handle_write_error_(written, LOG_STR("write noise frame"))) {
return false;
}
s.frame_pos += written;
}
return true;
}
/// Best-effort explicit reject frame so the client can log a readable reason.
void ESPHomeOTAComponent::noise_send_reject_(const LogString *reason) {
// Every reason here comes from noise::reject_reason_for(), so the exported
// floor is the exact capacity needed
uint8_t data[noise::FRAME_HEADER_SIZE + noise::MAC_FAILURE_PAYLOAD_SIZE];
const size_t payload_len =
noise::format_reject_payload(data + noise::FRAME_HEADER_SIZE, sizeof(data) - noise::FRAME_HEADER_SIZE, reason);
noise::write_frame_header(data, payload_len);
this->client_->write(data, noise::FRAME_HEADER_SIZE + payload_len); // Best effort, non-blocking
}
/// Decrypt a ciphertext in place; returns the plaintext size or -1.
ssize_t ESPHomeOTAComponent::noise_decrypt_(uint8_t *buf, size_t len) {
NoiseBuffer mbuf;
noise_buffer_init(mbuf);
noise_buffer_set_inout(mbuf, buf, len, len);
int err = noise_cipherstate_decrypt(this->noise_->recv_cipher, &mbuf);
if (err != 0) {
ESP_LOGW(TAG, "Decrypt: %s", LOG_STR_ARG(noise::noise_err_to_logstr(err)));
return -1;
}
return mbuf.size;
}
/** Blocking read of one frame whose ciphertext size must be within the given
* bounds, decrypted in place; returns the plaintext size, or -1 on error.
* buf needs max_ciphertext capacity.
*/
ssize_t ESPHomeOTAComponent::noise_read_frame_blocking_(uint8_t *buf, size_t min_ciphertext, size_t max_ciphertext) {
uint8_t header[noise::FRAME_HEADER_SIZE];
if (!this->readall_(header, sizeof(header))) {
return -1;
}
const size_t ciphertext_len = encode_uint16(header[1], header[2]);
if (header[0] != noise::FRAME_INDICATOR || ciphertext_len < min_ciphertext || ciphertext_len > max_ciphertext) {
ESP_LOGW(TAG, "Bad frame: 0x%02X, %zu bytes", header[0], ciphertext_len);
return -1;
}
if (!this->readall_(buf, ciphertext_len)) {
return -1;
}
return this->noise_decrypt_(buf, ciphertext_len);
}
/** Blocking read of one frame whose plaintext must be exactly len bytes
* (control units are one unit per frame). buf needs len + noise::MAC_SIZE
* capacity; the plaintext lands at buf[0..len).
*/
bool ESPHomeOTAComponent::noise_readall_(uint8_t *buf, size_t len) {
return this->noise_read_frame_blocking_(buf, len + noise::MAC_SIZE, len + noise::MAC_SIZE) == (ssize_t) len;
}
/** Blocking read of one data-phase frame, decrypted in place; returns the
* plaintext size, or -1 on error. buf is the OTA_BUFFER_SIZE data buffer.
* The ciphertext must fit that buffer and its plaintext must fit what the
* caller accepts (the remaining image bytes).
*/
ssize_t ESPHomeOTAComponent::noise_read_data_(uint8_t *buf, size_t capacity) {
const size_t max_ciphertext = std::min(capacity + noise::MAC_SIZE, OTA_BUFFER_SIZE);
return this->noise_read_frame_blocking_(buf, noise::MAC_SIZE + 1, max_ciphertext);
}
/// Blocking write of one response byte as an encrypted frame.
bool ESPHomeOTAComponent::noise_write_byte_(uint8_t byte) {
uint8_t frame[noise::FRAME_HEADER_SIZE + 1 + noise::MAC_SIZE];
frame[noise::FRAME_HEADER_SIZE] = byte;
NoiseBuffer mbuf;
noise_buffer_init(mbuf);
noise_buffer_set_inout(mbuf, frame + noise::FRAME_HEADER_SIZE, 1, 1 + noise::MAC_SIZE);
int err = noise_cipherstate_encrypt(this->noise_->send_cipher, &mbuf);
if (err != 0) {
ESP_LOGW(TAG, "Encrypt: %s", LOG_STR_ARG(noise::noise_err_to_logstr(err)));
return false;
}
noise::write_frame_header(frame, mbuf.size);
return this->writeall_(frame, noise::FRAME_HEADER_SIZE + mbuf.size);
}
} // namespace esphome
#endif // USE_OTA_ENCRYPTION
#endif // USE_OTA
+2 -1
View File
@@ -2,7 +2,7 @@ from contextlib import ExitStack
from esphome import automation
import esphome.codegen as cg
from esphome.components.const import CONF_COLUMNS, CONF_ROWS
from esphome.components.const import CONF_ROWS
import esphome.config_validation as cv
from esphome.const import CONF_ID, CONF_ITEMS, CONF_ROW, CONF_TEXT, CONF_WIDTH
from esphome.core import ID
@@ -20,6 +20,7 @@ from .label import CONF_LABEL
CONF_TABLE = "table"
CONF_CELLS = "cells"
CONF_COLUMNS = "columns"
CONF_ROW_COUNT = "row_count"
CONF_COLUMN_COUNT = "column_count"
CONF_MERGE_RIGHT = "merge_right"
+3 -1
View File
@@ -1,7 +1,7 @@
from esphome import automation, pins
import esphome.codegen as cg
from esphome.components import key_provider
from esphome.components.const import CONF_COLUMNS, CONF_KEYS, CONF_ROWS
from esphome.components.const import CONF_ROWS
import esphome.config_validation as cv
from esphome.const import CONF_ID, CONF_ON_KEY, CONF_PIN, CONF_TRIGGER_ID
from esphome.types import ConfigType
@@ -21,6 +21,8 @@ MatrixKeyTrigger = matrix_keypad_ns.class_(
)
CONF_KEYPAD_ID = "keypad_id"
CONF_COLUMNS = "columns"
CONF_KEYS = "keys"
CONF_DEBOUNCE_TIME = "debounce_time"
CONF_HAS_DIODES = "has_diodes"
CONF_HAS_PULLDOWNS = "has_pulldowns"
+6 -1
View File
@@ -5,7 +5,7 @@
#include "esphome/core/helpers.h"
#include "esphome/core/log.h"
#include <driver/gpio.h>
#include <esp_lcd_panel_ops.h>
#include <esp_lcd_panel_rgb.h>
#include <span>
namespace esphome::mipi_rgb {
@@ -177,6 +177,11 @@ void MipiRgb::common_setup_() {
ESP_LOGCONFIG(TAG, "MipiRgb setup complete");
}
void MipiRgb::loop() {
if (this->handle_ != nullptr)
esp_lcd_rgb_panel_restart(this->handle_);
}
void MipiRgb::update() {
if (this->is_failed())
return;
+2 -7
View File
@@ -3,7 +3,7 @@
#if defined(USE_ESP32_VARIANT_ESP32S3) || defined(USE_ESP32_VARIANT_ESP32P4) || defined(USE_ESP32_VARIANT_ESP32S31)
#include "esphome/core/gpio.h"
#include "esphome/components/display/display.h"
#include <esp_lcd_panel_rgb.h>
#include "esp_lcd_panel_ops.h"
#ifdef USE_SPI
#include "esphome/components/spi/spi.h"
#endif
@@ -25,12 +25,7 @@ class MipiRgb : public display::Display {
public:
MipiRgb(int width, int height) : width_(width), height_(height) {}
void setup() override;
#ifdef USE_ESP32_VARIANT_ESP32S3
void loop() override {
if (this->handle_ != nullptr)
esp_lcd_rgb_panel_restart(this->handle_);
}
#endif
void loop() override;
void update() override;
void fill(Color color) override;
void draw_pixels_at(int x_start, int y_start, int w, int h, const uint8_t *ptr, display::ColorOrder order,
+31 -15
View File
@@ -5,12 +5,18 @@ from typing import Any
import esphome.codegen as cg
import esphome.config_validation as cv
from esphome.const import CONF_KEY
from esphome.core import CORE
from esphome.types import ConfigType
CODEOWNERS = ["@esphome/core"]
noise_ns = cg.esphome_ns.namespace("noise")
# Keep in sync with platformio.ini and esphome/idf_component.yml.
# LIBSODIUM_VERSION must match the version noise-c pins in its manifests.
NOISE_C_VERSION = "0.1.21"
LIBSODIUM_VERSION = "1.10021.4"
CONFIG_SCHEMA = cv.Schema({})
@@ -45,15 +51,6 @@ def decode_encryption_key(value: str) -> bytes:
return decoded
def is_reserved_key(value: str) -> bool:
"""Whether the key is the reserved all-zeros provisioning sentinel.
The device treats it as no key configured, so consumers that require a
real key must reject it.
"""
return not any(decode_encryption_key(value))
ENCRYPTION_SCHEMA = cv.Schema(
{
cv.Optional(CONF_KEY): cv.sensitive(validate_encryption_key),
@@ -72,12 +69,31 @@ def encryption_schema(config: ConfigType | None) -> ConfigType:
async def to_code(config: ConfigType) -> None:
cg.add_define("USE_NOISE")
cg.add_library("esphome/noise-c", "0.1.21")
# noise-c depends on libsodium, but declaring it here too lets the
# library manager see the full set up front instead of discovering
# libsodium only after noise-c has downloaded, so the two can download
# in parallel. The version must match noise-c's library.json.
cg.add_library("esphome/libsodium", "1.10021.4")
# Both libraries build themselves as ESP-IDF components, so on ESP32 they
# are pulled straight from the component registry instead of going through
# ESPHome's PlatformIO-library converter. Deliberately not conditional on
# the toolchain: wireguard splits on the same condition, and if the two
# disagree one of them converts a second libsodium next to the managed one.
#
# Not on the Arduino framework though: arduino-esp32 depends on
# espressif/libsodium of its own (on IDF < 6.0), so the component manager
# would see two managed components whose names match once the namespace is
# stripped, and refuse to pick between them.
#
# libsodium is declared alongside noise-c rather than left to noise-c's own
# manifest either way: it lets the library manager see the full set up front
# instead of discovering libsodium only after noise-c has downloaded, and it
# keeps other components that depend on it (wireguard) from converting a
# second copy next to the managed one. The version must match the one
# noise-c pins.
if CORE.is_esp32 and not CORE.using_arduino:
from esphome.components.esp32 import add_idf_component
add_idf_component(name="esphome/noise-c", ref=NOISE_C_VERSION)
add_idf_component(name="esphome/libsodium", ref=LIBSODIUM_VERSION)
else:
cg.add_library("esphome/noise-c", NOISE_C_VERSION)
cg.add_library("esphome/libsodium", LIBSODIUM_VERSION)
# Enable optimized memzero/memcmp in libsodium instead of volatile byte loops
cg.add_build_flag("-DHAVE_WEAK_SYMBOLS=1")
cg.add_build_flag("-DHAVE_INLINE_ASM=1")
-1
View File
@@ -49,7 +49,6 @@ enum OTAResponseTypes {
OTA_RESPONSE_ERROR_BOOTLOADER_VERIFY = 0x91,
OTA_RESPONSE_ERROR_BOOTLOADER_UPDATE = 0x92,
OTA_RESPONSE_ERROR_VERSION_DOWNGRADE = 0x93,
OTA_RESPONSE_ERROR_ENCRYPTION_REQUIRED = 0x94,
OTA_RESPONSE_ERROR_UNKNOWN = 0xFF,
};
+4 -11
View File
@@ -255,17 +255,12 @@ bool SafeModeComponent::should_enter_safe_mode(uint8_t num_attempts, uint32_t en
}
void SafeModeComponent::write_rtc_(uint32_t val) {
if (!this->rtc_.save(&val)) {
ESP_LOGE(TAG, "Failed to set rtc value (%" PRIu32 ")", val);
return;
}
if (!global_preferences->sync()) {
ESP_LOGE(TAG, "Failed to persist rtc value (%" PRIu32 ")", val);
}
this->rtc_.save(&val);
global_preferences->sync();
}
uint32_t SafeModeComponent::read_rtc_() {
uint32_t val = 0;
uint32_t val;
if (!this->rtc_.load(&val))
return 0;
return val;
@@ -277,9 +272,7 @@ void SafeModeComponent::clean_rtc() {
// before sync, the boot wasn't really successful anyway and the counter should
// remain incremented.
uint32_t val = 0;
if (!this->rtc_.save(&val)) {
ESP_LOGE(TAG, "Failed to clear boot loop counter");
}
this->rtc_.save(&val);
}
void SafeModeComponent::on_safe_shutdown() {
+1 -1
View File
@@ -1,7 +1,6 @@
from esphome import automation, pins
import esphome.codegen as cg
from esphome.components import i2c, key_provider
from esphome.components.const import CONF_KEYS
import esphome.config_validation as cv
from esphome.const import (
CONF_ID,
@@ -20,6 +19,7 @@ from esphome.cpp_generator import MockObj
from esphome.types import ConfigType
CONF_KEYPAD = "keypad"
CONF_KEYS = "keys"
CONF_KEY_ROWS = "key_rows"
CONF_KEY_COLUMNS = "key_columns"
CONF_SLEEP_TIME = "sleep_time"
-1
View File
@@ -242,7 +242,6 @@
#define USE_RUNTIME_IMAGE_QOI
#define USE_RUNTIME_STATS
#define USE_OTA
#define USE_OTA_ENCRYPTION
#define USE_OTA_PASSWORD
#define USE_OTA_VERSION 2
#define USE_TIME_TIMEZONE
+21 -8
View File
@@ -238,6 +238,17 @@ def _parse_lib_deps(platformio_ini: Path, framework: str):
return libs
def _esphome_manifest_deps() -> set[str]:
"""Names of the managed components declared in ``esphome/idf_component.yml``."""
import yaml
esphome_dir = Path(__file__).resolve().parent.parent
manifest = yaml.safe_load(
(esphome_dir / "idf_component.yml").read_text(encoding="utf-8")
)
return set(manifest.get("dependencies") or {})
def _convert_pio_libs(
platformio_ini: Path, framework: str
) -> dict[str, dict[str, str]]:
@@ -250,12 +261,20 @@ def _convert_pio_libs(
The whole library set is resolved as a single batch so a shared transitive
dependency (e.g. esphome/libsodium pulled by both noise-c and esp_wireguard)
is deduplicated to one component instead of clashing override_path entries.
Libraries ESPHome's own manifest already provides as managed components
(noise-c, libsodium, ...) are skipped, mirroring what the real esp32 build
does -- converting them too would make IDF see the same requirement twice.
On Arduino those entries are rule-disabled in the manifest (arduino-esp32
brings its own libsodium), so nothing provides them there and they have to
go through the converter as before.
"""
from esphome.espidf.component import generate_idf_components
libraries = _parse_lib_deps(platformio_ini, framework)
managed = set() if framework == "arduino" else _esphome_manifest_deps()
deps: dict[str, dict[str, str]] = {}
for component in generate_idf_components(libraries):
for component in generate_idf_components(libraries, managed=managed):
deps[component.get_sanitized_name()] = {"override_path": str(component.path)}
return deps
@@ -273,19 +292,13 @@ def _arduino_excluded_stubs(work_dir: Path) -> dict[str, dict]:
ethernet) are NOT stubbed -- those are real deps we need, and arduino-esp32
resolves to the same component rather than conflicting.
"""
import yaml
from esphome.components.esp32 import (
ARDUINO_EXCLUDED_IDF_COMPONENTS,
_idf_component_dep_name,
_idf_component_stub_name,
)
esphome_dir = Path(__file__).resolve().parent.parent
base_manifest = yaml.safe_load(
(esphome_dir / "idf_component.yml").read_text(encoding="utf-8")
)
esphome_deps = set(base_manifest.get("dependencies") or {})
esphome_deps = _esphome_manifest_deps()
stubs_dir = work_dir / "component_stubs"
stubs_dir.mkdir(parents=True, exist_ok=True)
+13 -3
View File
@@ -287,12 +287,22 @@ def _emit_idf_component(component: IDFComponent) -> None:
)
def generate_idf_components(libraries: list[Library]) -> list[IDFComponent]:
"""Resolve and convert a batch of PlatformIO libraries to IDF components."""
def generate_idf_components(
libraries: list[Library], managed: set[str] | None = None
) -> list[IDFComponent]:
"""Resolve and convert a batch of PlatformIO libraries to IDF components.
``managed`` names the registry components already declared in the project
manifest (via ``add_idf_component``). Those are skipped by the converter --
a library must not be both converted and managed, or IDF fails component
discovery with "Requirement <owner>__<name> and requirement <name> are both
added as project_managed_components". Converted components pick the managed
one up through ``${ESPHOME_PROJECT_MANAGED_COMPONENTS}`` in their REQUIRES.
"""
backend = LibraryBackend(
platform=ESP32_PLATFORM,
framework=_idf_framework(),
emit=_emit_idf_component,
cache_key="idf",
)
return convert_libraries(libraries, backend)
return convert_libraries(libraries, backend, provided=managed)
+5 -193
View File
@@ -53,7 +53,6 @@ RESPONSE_ERROR_PARTITION_TABLE_UPDATE = 0x90
RESPONSE_ERROR_BOOTLOADER_VERIFY = 0x91
RESPONSE_ERROR_BOOTLOADER_UPDATE = 0x92
RESPONSE_ERROR_VERSION_DOWNGRADE = 0x93
RESPONSE_ERROR_ENCRYPTION_REQUIRED = 0x94
RESPONSE_ERROR_UNKNOWN = 0xFF
OTA_VERSION_1_0 = 1
@@ -64,20 +63,8 @@ MAGIC_BYTES = [0x6C, 0x26, 0xF7, 0x5C, 0x45]
CLIENT_FEATURE_SUPPORTS_COMPRESSION = 0x01
CLIENT_FEATURE_SUPPORTS_SHA256_AUTH = 0x02
CLIENT_FEATURE_SUPPORTS_EXTENDED_PROTOCOL = 0x04
CLIENT_FEATURE_SUPPORTS_NOISE = 0x08
SERVER_FEATURE_SUPPORTS_COMPRESSION = 0x01
SERVER_FEATURE_SUPPORTS_PARTITION_ACCESS = 0x02
SERVER_FEATURE_SUPPORTS_NOISE = 0x04
NOISE_FRAME_INDICATOR = 0x01
NOISE_HANDSHAKE_OK = 0x00
# The device decrypts frames in its transfer buffer (OTA_BUFFER_SIZE, sized
# as this plus the 16-byte ChaCha20-Poly1305 MAC). 1024 divides the 8192-byte
# upload block exactly, so blocks tile into full frames with no runt.
NOISE_MAX_PLAINTEXT = 1024
# Wire contract: the device sends exactly this reject reason for a bad MAC
NOISE_MAC_FAILURE_REASON = "Handshake MAC failure"
NOISE_PROLOGUE_INIT = b"NoiseOTAInit"
# OTA types this client knows how to send. Future PRs that add bootloader/partition
# updates extend this set. Anything outside the set is rejected up front so callers
@@ -184,12 +171,6 @@ _ERROR_MESSAGES: dict[int, str] = {
"enabled: the new firmware's version must be newer than the version the "
"device is currently running."
),
RESPONSE_ERROR_ENCRYPTION_REQUIRED: (
"The device requires an encrypted OTA connection but this upload has no "
"encryption key. Add 'encryption:' to the 'ota: platform: esphome' section "
"of the YAML this upload uses, or update your esphome installation if it "
"predates OTA encryption."
),
RESPONSE_ERROR_UNKNOWN: "Unknown error from ESP",
}
@@ -324,149 +305,16 @@ def send_check(
raise OTANetworkError(f"sending {msg}: {err}") from err
class NoiseSocketWrapper:
"""Runs the OTA session inside a Noise (ChaCha20-Poly1305) transport.
Exposes the socket subset perform_ota uses. Frames are indicator 0x01,
16-bit big-endian length, ciphertext; recv() drains one decrypted frame
at a time, sendall() keeps control units in one frame and splits data
at NOISE_MAX_PLAINTEXT.
"""
def __init__(self, sock: socket.socket, psk: str, prologue: bytes) -> None:
# Deliberately lazy: the noise stack (noiseprotocol, cryptography) is
# only imported when an encrypted upload actually runs.
try:
from aioesphomeapi.noise import NoiseHandshake
except ImportError as err:
raise OTAError(
"OTA encryption requires a newer aioesphomeapi; update your "
"esphome installation (pip install -U esphome) and retry"
) from err
# The aioesphomeapi import above already loaded cryptography; bind
# the exception once so recv() pays no per-frame import lookup
from cryptography.exceptions import InvalidTag
self._invalid_tag = InvalidTag
self._sock = sock
try:
self._handshake = NoiseHandshake(psk, prologue)
except ValueError as err:
raise OTAError(f"Invalid OTA encryption key: {err}") from err
self._encrypt = None
self._decrypt = None
self._buffer = b""
# Only harmless socket controls pass through; byte-moving methods are
# deliberately absent so plaintext cannot leak past the transport.
def settimeout(self, timeout: float | None) -> None:
self._sock.settimeout(timeout)
def setsockopt(self, level: int, optname: int, value: int) -> None:
self._sock.setsockopt(level, optname, value)
def close(self) -> None:
self._sock.close()
def do_handshake(self) -> None:
"""Run the two-message NNpsk0 handshake and set up the transport ciphers."""
try:
self._send_frame(
bytes([NOISE_HANDSHAKE_OK]) + self._handshake.write_message()
)
payload = self._recv_frame()
except OSError as err:
raise OTANetworkError(f"noise handshake: {err}") from err
if not payload:
raise OTANetworkError("Device closed connection during the noise handshake")
if payload[0] != NOISE_HANDSHAKE_OK:
reason = payload[1:].decode("utf-8", "replace")
if reason == NOISE_MAC_FAILURE_REASON:
raise OTAError(
"Device rejected the handshake; is the OTA encryption key correct?"
)
raise OTAError(f"Device rejected the noise handshake: {reason}")
try:
self._handshake.read_message(payload[1:])
except (ValueError, self._invalid_tag) as err:
# InvalidTag is a wrong key; ValueError covers a device sending an
# invalid curve point, which cryptography rejects during the DH
raise OTAError(
"Noise handshake failed; is the OTA encryption key correct?"
) from err
self._encrypt, self._decrypt = self._handshake.get_ciphers()
def sendall(self, data: bytes) -> None:
frames: list[bytes] = []
for offset in range(0, len(data), NOISE_MAX_PLAINTEXT):
ciphertext = self._encrypt.encrypt(
data[offset : offset + NOISE_MAX_PLAINTEXT]
)
frames.append(self._frame_header(len(ciphertext)))
frames.append(ciphertext)
self._sock.sendall(b"".join(frames))
def recv(self, amount: int) -> bytes:
if not self._buffer:
ciphertext = self._recv_frame()
if not ciphertext:
return b"" # connection closed at a frame boundary
try:
self._buffer = self._decrypt.decrypt(ciphertext)
except self._invalid_tag as err:
# Retryable: a fresh connection renegotiates the session
raise OTANetworkError(
"Noise decryption failed (MAC mismatch); frame corrupted or tampered"
) from err
if not self._buffer:
# Reject MAC-only frames so b"" always means the peer closed
raise OTANetworkError("Device sent an empty noise frame")
data = self._buffer[:amount]
self._buffer = self._buffer[amount:]
return data
@staticmethod
def _frame_header(length: int) -> bytes:
return bytes([NOISE_FRAME_INDICATOR, (length >> 8) & 0xFF, length & 0xFF])
def _send_frame(self, payload: bytes) -> None:
self._sock.sendall(self._frame_header(len(payload)) + payload)
def _recv_frame(self) -> bytes:
header = self._recv_exact(3, closed_ok=True)
if not header:
return b"" # connection closed at a frame boundary
# A malformed frame is a broken transport, not a device error;
# retryable so a fresh session is tried
if header[0] != NOISE_FRAME_INDICATOR:
raise OTANetworkError(f"Bad noise frame indicator 0x{header[0]:02X}")
length = (header[1] << 8) | header[2]
if length == 0:
raise OTANetworkError("Device sent an empty noise frame")
return self._recv_exact(length)
def _recv_exact(self, amount: int, closed_ok: bool = False) -> bytes:
data = b""
while len(data) < amount:
chunk = self._sock.recv(amount - len(data))
if not chunk:
if closed_ok and not data:
return b""
raise OSError("connection closed inside a noise frame")
data += chunk
return data
def perform_ota(
sock: socket.socket,
password: str | None,
file_handle: io.IOBase,
filename: Path,
ota_type: int = OTA_TYPE_UPDATE_APP,
noise_psk: str | None = None,
) -> None:
# Validate up front; an out-of-range value would only surface as a
# ValueError deep inside send_check, bypassing OTAError handling
# Validate ota_type up front. It travels as a single byte on the wire, and
# passing an out-of-range value would only surface as a ValueError from
# bytes([ota_type]) deep inside send_check, bypassing OTAError handling.
if not isinstance(ota_type, int) or not 0 <= ota_type <= 0xFF:
raise OTAError(
f"Invalid ota_type {ota_type!r}; expected an integer in range 0-255"
@@ -477,11 +325,6 @@ def perform_ota(
f"Unsupported OTA type 0x{ota_type:02X}; this ESPHome supports: {supported}"
)
if noise_psk is not None and not noise_psk:
raise OTAError(
"An empty OTA encryption key was provided; refusing to upload in plaintext"
)
file_contents = file_handle.read()
file_size = len(file_contents)
_LOGGER.info("Uploading %s (%s bytes)", filename, file_size)
@@ -504,8 +347,6 @@ def perform_ota(
| CLIENT_FEATURE_SUPPORTS_SHA256_AUTH
| CLIENT_FEATURE_SUPPORTS_EXTENDED_PROTOCOL
)
if noise_psk:
features_to_send |= CLIENT_FEATURE_SUPPORTS_NOISE
send_check(sock, features_to_send, "features")
features = receive_exactly(
sock,
@@ -528,31 +369,6 @@ def perform_ota(
else:
features = 0
if noise_psk:
# Fail closed: never fall back to a plaintext upload when an
# encryption key is configured, an active attacker could otherwise
# strip the feature flag and capture the image (it contains the wifi
# credentials and the api encryption key).
if not (extended_proto and features & SERVER_FEATURE_SUPPORTS_NOISE):
raise OTAError(
"An OTA encryption key is configured but the device did not "
"offer encryption; refusing to send the image in plaintext. "
"If the running firmware predates OTA encryption, first update "
"it without the 'ota: encryption:' block (over a trusted "
"network or via USB), then restore the block and upload again."
)
# The prologue binds every negotiation byte both sides saw, so any
# tampering with the plaintext preamble breaks the handshake.
prologue = (
NOISE_PROLOGUE_INIT
+ bytes(MAGIC_BYTES)
+ bytes([RESPONSE_OK, version, features_to_send])
+ bytes([RESPONSE_FEATURE_FLAGS, features])
)
sock = NoiseSocketWrapper(sock, noise_psk, prologue)
sock.do_handshake()
_LOGGER.info("Encrypted connection established")
if ota_type != OTA_TYPE_UPDATE_APP:
# Any non-app OTA type requires the extended protocol and the
# partition-access server feature. Reject up front so the user gets
@@ -756,7 +572,6 @@ def run_ota_impl_(
password: str | None,
filename: Path,
ota_type: int = OTA_TYPE_UPDATE_APP,
noise_psk: str | None = None,
) -> tuple[int, str | None]:
from esphome.core import CORE
@@ -821,7 +636,7 @@ def run_ota_impl_(
reached_device = True
with contextlib.closing(sock), Path(filename).open("rb") as file_handle:
try:
perform_ota(sock, password, file_handle, filename, ota_type, noise_psk)
perform_ota(sock, password, file_handle, filename, ota_type)
except OTANetworkError as err:
# Transient network failure; retry
last_error = str(err)
@@ -846,12 +661,9 @@ def run_ota(
password: str | None,
filename: Path,
ota_type: int = OTA_TYPE_UPDATE_APP,
noise_psk: str | None = None,
) -> tuple[int, str | None]:
try:
return run_ota_impl_(
remote_host, remote_port, password, filename, ota_type, noise_psk
)
return run_ota_impl_(remote_host, remote_port, password, filename, ota_type)
except OTAError as err:
_LOGGER.error(err)
return 1, None
+13
View File
@@ -106,3 +106,16 @@ dependencies:
version: d44c800a9e876a8394caefc2ce4915dd96dac77b
rules:
- if: "$ESPHOME_ARDUINO_COMPONENT == 1"
# api. Not on Arduino: arduino-esp32 pulls espressif/libsodium, and IDF
# refuses to build two managed components whose names differ only by
# namespace. The Arduino envs get noise-c as a PlatformIO library instead.
esphome/noise-c:
version: 0.1.21
rules:
- if: "$ESPHOME_ARDUINO_COMPONENT == 0"
# Declared even though noise-c depends on it, so that the PlatformIO-library
# converter knows to skip the copy esp_wireguard would otherwise pull in.
esphome/libsodium:
version: 1.10021.4
rules:
- if: "$ESPHOME_ARDUINO_COMPONENT == 0"
+16 -4
View File
@@ -1102,7 +1102,9 @@ def _prefetch_wave(
def convert_libraries(
libraries: list[Library], backend: LibraryBackend
libraries: list[Library],
backend: LibraryBackend,
provided: set[str] | None = None,
) -> list[ConvertedLibrary]:
"""Resolve and convert a batch of PlatformIO libraries for ``backend``.
@@ -1123,14 +1125,24 @@ def convert_libraries(
``lib_ignore`` from ``esphome->platformio_options`` excludes libraries by
short name (part after the ``/``), matched against both the top-level
libraries and every dependency discovered during the graph walk.
``provided`` names libraries the toolchain already supplies by other means
(for ESP-IDF: registry-managed components declared via
``add_idf_component``). They are excluded exactly like ``lib_ignore``, so a
library is never both converted and managed -- ESP-IDF refuses to build when
two components claim the same requirement.
"""
nodes: dict[str, _LibNode] = {}
lib_ignore = lib_ignore_set()
# Libraries the toolchain supplies by other means are excluded exactly like
# lib_ignore, so every is_lib_ignored() call site honors both.
lib_ignore = lib_ignore_set() | {
name.split("/")[-1].lower() for name in provided or ()
}
# The generated build files inside the shared cache bake in the dependency
# wiring, which lib_ignore changes; salt the cache path so configs with
# different lib_ignore values don't fight over (and constantly rewrite) the
# wiring, which the exclusion set changes; salt the cache path so configs
# with different exclusions don't fight over (and constantly rewrite) the
# same converted component files.
salt = (
hashlib.sha256(",".join(sorted(lib_ignore)).encode()).hexdigest()[:8]
+3 -1
View File
@@ -45,7 +45,6 @@ lib_deps_base =
lib_deps =
${common.lib_deps_base}
https://github.com/dudanov/MideaUART.git#eeea6c3e9b4474f067054592b435be1c4e466815 ; midea
esphome/noise-c@0.1.21 ; noise (api, ota)
improv/Improv@1.2.7 ; improv_serial / esp32_improv
kikuchan98/pngle@1.1.0 ; online_image
; Using the repository directly, otherwise ESP-IDF can't use the library
@@ -77,6 +76,9 @@ lib_compat_mode = strict
extends = common
lib_deps =
${common.lib_deps}
; api -- on the ESP-IDF framework this comes from the component registry
; instead (see esphome/idf_component.yml), so it is not in [common].
esphome/noise-c@0.1.21 ; api
SPI ; spi (Arduino built-in)
Wire ; i2c (Arduino built-int)
heman/AsyncMqttClient-esphome@1.0.0 ; mqtt
+1 -1
View File
@@ -14,7 +14,7 @@ esptool==5.3.1
click==8.3.3
aioesphomeapi==46.3.0
aiohappyeyeballs==2.7.1 # Happy Eyeballs for requests downloads; already pulled in by aioesphomeapi
zeroconf==0.151.2
zeroconf==0.150.4
puremagic==2.2.0
ruamel.yaml==0.19.1 # dashboard_import
ruamel.yaml.clib==0.2.15 # dashboard_import
+1 -1
View File
@@ -1,4 +1,4 @@
pylint==4.0.8
pylint==4.0.7
flake8==7.3.0 # also change in .pre-commit-config.yaml when updating
ruff==0.16.5 # also change in .pre-commit-config.yaml when updating
pyupgrade==3.21.2 # also change in .pre-commit-config.yaml when updating
-17
View File
@@ -1104,10 +1104,6 @@ def get_components_per_integration_fixture() -> dict[str, set[str]]:
_TEST_FUNC_RE = re.compile(r"async def (test_\w+)")
# Any usage form (decorator, pytestmark assignment or list element); only
# test_*.py files are scanned, so the marker docs elsewhere cannot false-hit
_SHARED_YAML_USE_RE = re.compile(r"\bmark\.shared_yaml")
_SHARED_YAML_ARG_RE = re.compile(r"\(\s*[\"'](\w+)[\"']\s*\)")
@cache
@@ -1127,19 +1123,6 @@ def get_fixture_to_test_files() -> dict[str, frozenset[str]]:
for func in _TEST_FUNC_RE.findall(content):
base_name = func.replace("test_", "").partition("[")[0]
result.setdefault(base_name, set()).add(rel_path)
# Shared fixtures are named by marker, not by a test function; each
# decorator must carry a string literal or its fixture would silently
# map to no tests
for use in _SHARED_YAML_USE_RE.finditer(content):
arg = _SHARED_YAML_ARG_RE.match(content, use.end())
if arg is None:
line = content.count("\n", 0, use.start()) + 1
raise ValueError(
f"{rel_path}:{line}: shared_yaml marker must take a "
"single-line string literal so CI test selection can map "
"its fixture"
)
result.setdefault(arg.group(1), set()).add(rel_path)
return {k: frozenset(v) for k, v in result.items()}
@@ -1,13 +0,0 @@
esphome:
name: test
esp32:
variant: esp32s31
board: esp32-s31-devkitc
framework:
type: esp-idf
advanced:
execute_from_psram: true
psram:
mode: octal
+4 -28
View File
@@ -203,18 +203,6 @@ def test_esp32_rejects_unsupported_cli_toolchain(
r"'execute_from_psram' requires PSRAM to be configured @ data\['framework'\]\['advanced'\]\['execute_from_psram'\]",
id="execute_from_psram_requires_psram_p4_config",
),
pytest.param(
{
"variant": "esp32s31",
"board": "esp32-s31-devkitc",
"framework": {
"type": "esp-idf",
"advanced": {"execute_from_psram": True},
},
},
r"'execute_from_psram' requires PSRAM to be configured @ data\['framework'\]\['advanced'\]\['execute_from_psram'\]",
id="execute_from_psram_requires_psram_s31_config",
),
pytest.param(
{
"variant": "esp32s3",
@@ -434,12 +422,12 @@ def test_execute_from_psram_s3_sdkconfig(
generate_main: Callable[[str | Path], str],
component_config_path: Callable[[str], Path],
) -> None:
"""Test that execute_from_psram on ESP32-S3 sets the correct sdkconfig option."""
"""Test that execute_from_psram on ESP32-S3 sets the correct sdkconfig options."""
generate_main(component_config_path("execute_from_psram_s3.yaml"))
sdkconfig = CORE.data[KEY_ESP32][KEY_SDKCONFIG_OPTIONS]
assert sdkconfig.get("CONFIG_SPIRAM_XIP_FROM_PSRAM") is True
assert "CONFIG_SPIRAM_FETCH_INSTRUCTIONS" not in sdkconfig
assert "CONFIG_SPIRAM_RODATA" not in sdkconfig
assert sdkconfig.get("CONFIG_SPIRAM_FETCH_INSTRUCTIONS") is True
assert sdkconfig.get("CONFIG_SPIRAM_RODATA") is True
assert "CONFIG_SPIRAM_XIP_FROM_PSRAM" not in sdkconfig
def test_execute_from_psram_p4_sdkconfig(
@@ -454,18 +442,6 @@ def test_execute_from_psram_p4_sdkconfig(
assert "CONFIG_SPIRAM_RODATA" not in sdkconfig
def test_execute_from_psram_s31_sdkconfig(
generate_main: Callable[[str | Path], str],
component_config_path: Callable[[str], Path],
) -> None:
"""Test that execute_from_psram on ESP32-S31 sets the correct sdkconfig option."""
generate_main(component_config_path("execute_from_psram_s31.yaml"))
sdkconfig = CORE.data[KEY_ESP32][KEY_SDKCONFIG_OPTIONS]
assert sdkconfig.get("CONFIG_SPIRAM_XIP_FROM_PSRAM") is True
assert "CONFIG_SPIRAM_FETCH_INSTRUCTIONS" not in sdkconfig
assert "CONFIG_SPIRAM_RODATA" not in sdkconfig
def test_nvs_encryption_sdkconfig(
generate_main: Callable[[str | Path], str],
component_config_path: Callable[[str], Path],
@@ -5,11 +5,7 @@ from __future__ import annotations
import pytest
from esphome import config_validation as cv
from esphome.components.noise import (
decode_encryption_key,
is_reserved_key,
validate_encryption_key,
)
from esphome.components.noise import decode_encryption_key, validate_encryption_key
KEY = "AAECAwQFBgcICQoLDA0ODxAREhMUFRYXGBkaGxwdHh8="
@@ -39,8 +35,3 @@ def test_decode_encryption_key_rejects_short_decode() -> None:
a zero padded PSK on the device."""
with pytest.raises(cv.Invalid, match="32 bytes"):
decode_encryption_key("AAECAw==")
def test_is_reserved_key() -> None:
assert is_reserved_key("AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA=")
assert not is_reserved_key(KEY)
+2 -312
View File
@@ -8,25 +8,17 @@ from typing import Any
import pytest
from esphome import config_validation as cv
from esphome.components.esphome.ota import (
AUTO_LOAD,
FILTER_SOURCE_FILES,
_validate_no_password_with_encryption,
ota_esphome_final_validate,
)
from esphome.components.esphome.ota import ota_esphome_final_validate
from esphome.const import (
CONF_API,
CONF_ENCRYPTION,
CONF_ESPHOME,
CONF_ID,
CONF_KEY,
CONF_OTA,
CONF_PASSWORD,
CONF_PLATFORM,
CONF_PORT,
CONF_VERSION,
)
from esphome.core import CORE, ID
from esphome.core import ID
import esphome.final_validate as fv
@@ -111,305 +103,3 @@ def test_non_esphome_ota_unaffected() -> None:
assert len(updated[CONF_OTA]) == 3
finally:
fv.full_config.reset(token)
API_KEY = "AAECAwQFBgcICQoLDA0ODxAREhMUFRYXGBkaGxwdHh8="
OTHER_KEY = "AQIDBAUGBwgJCgsMDQ4PEBESExQVFhcYGRobHB0eHyA="
ZEROS_KEY = "AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA="
def test_encryption_key_inherited_from_api() -> None:
"""A bare encryption block resolves to the api encryption key."""
full_conf = {
CONF_API: {CONF_ENCRYPTION: {CONF_KEY: API_KEY}},
CONF_OTA: [_make_ota_config(port=3232, **{CONF_ENCRYPTION: {}})],
}
token = fv.full_config.set(full_conf)
try:
ota_esphome_final_validate({})
updated = fv.full_config.get()
assert updated[CONF_OTA][0][CONF_ENCRYPTION][CONF_KEY] == API_KEY
finally:
fv.full_config.reset(token)
def test_encryption_explicit_key_matching_api_accepted() -> None:
"""An explicit ota key equal to the api key validates."""
full_conf = {
CONF_API: {CONF_ENCRYPTION: {CONF_KEY: API_KEY}},
CONF_OTA: [
_make_ota_config(port=3232, **{CONF_ENCRYPTION: {CONF_KEY: API_KEY}})
],
}
token = fv.full_config.set(full_conf)
try:
ota_esphome_final_validate({})
updated = fv.full_config.get()
assert updated[CONF_OTA][0][CONF_ENCRYPTION][CONF_KEY] == API_KEY
finally:
fv.full_config.reset(token)
def test_encryption_key_differing_from_api_rejected() -> None:
"""There is one key per device; an ota key differing from the api key raises."""
full_conf = {
CONF_API: {CONF_ENCRYPTION: {CONF_KEY: API_KEY}},
CONF_OTA: [
_make_ota_config(port=3232, **{CONF_ENCRYPTION: {CONF_KEY: OTHER_KEY}})
],
}
token = fv.full_config.set(full_conf)
try:
with pytest.raises(cv.Invalid, match="must match the 'api' encryption key"):
ota_esphome_final_validate({})
finally:
fv.full_config.reset(token)
def test_encryption_explicit_key_without_api_encryption_accepted() -> None:
"""An explicit ota key with a plaintext api has nothing to match; it stands."""
full_conf = {
CONF_API: {},
CONF_OTA: [
_make_ota_config(port=3232, **{CONF_ENCRYPTION: {CONF_KEY: OTHER_KEY}})
],
}
token = fv.full_config.set(full_conf)
try:
ota_esphome_final_validate({})
updated = fv.full_config.get()
assert updated[CONF_OTA][0][CONF_ENCRYPTION][CONF_KEY] == OTHER_KEY
finally:
fv.full_config.reset(token)
def test_encryption_without_any_key_rejected() -> None:
"""A bare encryption block with no api key to inherit raises."""
full_conf = {
CONF_API: {},
CONF_OTA: [_make_ota_config(port=3232, **{CONF_ENCRYPTION: {}})],
}
token = fv.full_config.set(full_conf)
try:
with pytest.raises(cv.Invalid, match="no 'api' encryption key to inherit"):
ota_esphome_final_validate({})
finally:
fv.full_config.reset(token)
def test_encryption_explicit_all_zeros_key_rejected() -> None:
"""The all-zeros key is the provisioning sentinel; the device would treat
it as no PSK and accept plaintext, so it must fail validation."""
full_conf = {
CONF_OTA: [
_make_ota_config(port=3232, **{CONF_ENCRYPTION: {CONF_KEY: ZEROS_KEY}})
],
}
token = fv.full_config.set(full_conf)
try:
with pytest.raises(cv.Invalid, match="all-zeros key is reserved"):
ota_esphome_final_validate({})
finally:
fv.full_config.reset(token)
def test_encryption_inherited_all_zeros_key_rejected() -> None:
"""An all-zeros api key must not silently disable ota encryption either."""
full_conf = {
CONF_API: {CONF_ENCRYPTION: {CONF_KEY: ZEROS_KEY}},
CONF_OTA: [_make_ota_config(port=3232, **{CONF_ENCRYPTION: {}})],
}
token = fv.full_config.set(full_conf)
try:
with pytest.raises(cv.Invalid, match="all-zeros key is reserved"):
ota_esphome_final_validate({})
finally:
fv.full_config.reset(token)
def test_encryption_key_mismatch_between_merged_configs_rejected() -> None:
"""Same-port configs with different encryption keys raise."""
full_conf = {
CONF_OTA: [
_make_ota_config(port=3232, **{CONF_ENCRYPTION: {CONF_KEY: API_KEY}}),
_make_ota_config(port=3232, **{CONF_ENCRYPTION: {CONF_KEY: OTHER_KEY}}),
]
}
token = fv.full_config.set(full_conf)
try:
with pytest.raises(cv.Invalid, match="encryption is inconsistent"):
ota_esphome_final_validate({})
finally:
fv.full_config.reset(token)
@pytest.mark.parametrize("keyed_first", [True, False])
def test_encryption_bare_and_keyed_blocks_merge(keyed_first: bool) -> None:
"""A bare encryption block (package/device split) is compatible with a
keyed one on the same port; the merge resolves to the keyed result."""
keyed = _make_ota_config(port=3232, **{CONF_ENCRYPTION: {CONF_KEY: OTHER_KEY}})
bare = _make_ota_config(port=3232, **{CONF_ENCRYPTION: {}})
full_conf = {
CONF_OTA: [keyed, bare] if keyed_first else [bare, keyed],
}
token = fv.full_config.set(full_conf)
try:
ota_esphome_final_validate({})
updated = fv.full_config.get()
assert len(updated[CONF_OTA]) == 1
assert updated[CONF_OTA][0][CONF_ENCRYPTION][CONF_KEY] == OTHER_KEY
finally:
fv.full_config.reset(token)
def test_encryption_runtime_provisioned_api_key_not_inheritable() -> None:
"""A keyless api encryption block provisions its key at runtime; a bare
ota encryption block cannot inherit it and the message says so."""
full_conf = {
CONF_API: {CONF_ENCRYPTION: {}},
CONF_OTA: [_make_ota_config(port=3232, **{CONF_ENCRYPTION: {}})],
}
token = fv.full_config.set(full_conf)
try:
with pytest.raises(cv.Invalid, match="provisioned at runtime"):
ota_esphome_final_validate({})
finally:
fv.full_config.reset(token)
def test_encryption_explicit_key_with_runtime_provisioned_api_accepted() -> None:
"""The documented remedy for a runtime-provisioned api key: set an
explicit ota key."""
full_conf = {
CONF_API: {CONF_ENCRYPTION: {}},
CONF_OTA: [
_make_ota_config(port=3232, **{CONF_ENCRYPTION: {CONF_KEY: OTHER_KEY}})
],
}
token = fv.full_config.set(full_conf)
try:
ota_esphome_final_validate({})
updated = fv.full_config.get()
assert updated[CONF_OTA][0][CONF_ENCRYPTION][CONF_KEY] == OTHER_KEY
finally:
fv.full_config.reset(token)
def test_encryption_with_web_server_ota_warns(
caplog: pytest.LogCaptureFixture,
) -> None:
"""With the web_server component the plaintext /update endpoint is always
on; the combination validates with a warning."""
full_conf = {
"web_server": {},
CONF_OTA: [
_make_ota_config(port=3232, **{CONF_ENCRYPTION: {CONF_KEY: OTHER_KEY}}),
{CONF_PLATFORM: "web_server", CONF_ID: ID("ota_ws", is_manual=False)},
],
}
token = fv.full_config.set(full_conf)
try:
with caplog.at_level(logging.WARNING):
ota_esphome_final_validate({})
assert any("plaintext /update" in record.message for record in caplog.records)
finally:
fv.full_config.reset(token)
def test_encryption_with_captive_portal_web_server_ota_warns(
caplog: pytest.LogCaptureFixture,
) -> None:
"""captive_portal auto-loads the web_server ota platform without the
web_server component; encryption stays usable and only warns, so the
fallback AP recovery path is not lost."""
full_conf = {
"captive_portal": {},
CONF_OTA: [
_make_ota_config(port=3232, **{CONF_ENCRYPTION: {CONF_KEY: OTHER_KEY}}),
{CONF_PLATFORM: "web_server", CONF_ID: ID("ota_ws", is_manual=False)},
],
}
token = fv.full_config.set(full_conf)
try:
with caplog.at_level(logging.WARNING):
ota_esphome_final_validate({})
assert any("captive_portal" in record.message for record in caplog.records)
esphome_conf = next(
conf
for conf in fv.full_config.get()[CONF_OTA]
if conf.get(CONF_PLATFORM) == CONF_ESPHOME
)
assert esphome_conf[CONF_ENCRYPTION][CONF_KEY] == OTHER_KEY
finally:
fv.full_config.reset(token)
def test_web_server_ota_without_encryption_unaffected() -> None:
"""web_server ota stays valid alongside an unencrypted esphome entry."""
full_conf = {
CONF_OTA: [
_make_ota_config(port=3232),
{CONF_PLATFORM: "web_server", CONF_ID: ID("ota_ws", is_manual=False)},
],
}
token = fv.full_config.set(full_conf)
try:
ota_esphome_final_validate({})
assert len(fv.full_config.get()[CONF_OTA]) == 2
finally:
fv.full_config.reset(token)
def test_auto_load_pulls_noise_only_for_encryption() -> None:
"""A plain ota entry must never pull noise-c into the build."""
assert AUTO_LOAD({CONF_PORT: 3232}) == ["sha256", "socket"]
assert "noise" in AUTO_LOAD({CONF_ENCRYPTION: {}})
# Tooling probes must get the maximal set: None from dependency
# resolution, {} from the components-graph platform probe
assert "noise" in AUTO_LOAD(None)
assert "noise" in AUTO_LOAD({})
def test_filter_source_files_excludes_noise_without_encryption() -> None:
"""The noise transport source compiles only for encrypted builds."""
old_config = CORE.config
try:
CORE.config = {CONF_OTA: [_make_ota_config(port=3232)]}
assert FILTER_SOURCE_FILES() == ["ota_esphome_noise.cpp"]
CORE.config = {
CONF_OTA: [
_make_ota_config(port=3232, **{CONF_ENCRYPTION: {CONF_KEY: API_KEY}})
]
}
assert FILTER_SOURCE_FILES() == []
finally:
CORE.config = old_config
def test_password_with_encryption_rejected() -> None:
"""The password and encryption options are mutually exclusive."""
config = {CONF_PASSWORD: "pw", CONF_ENCRYPTION: {CONF_KEY: API_KEY}}
with pytest.raises(cv.Invalid, match="cannot be combined"):
_validate_no_password_with_encryption(config)
def test_password_alone_accepted() -> None:
"""A password without encryption still validates."""
config = {CONF_PASSWORD: "pw"}
assert _validate_no_password_with_encryption(config) is config
def test_merged_password_and_encryption_rejected() -> None:
"""A password block and an encryption block merged on one port raise."""
full_conf = {
CONF_OTA: [
_make_ota_config(port=3232, **{CONF_PASSWORD: "pw"}),
_make_ota_config(port=3232, **{CONF_ENCRYPTION: {CONF_KEY: API_KEY}}),
]
}
token = fv.full_config.set(full_conf)
try:
with pytest.raises(cv.Invalid, match="cannot be combined"):
ota_esphome_final_validate({})
finally:
fv.full_config.reset(token)
-3
View File
@@ -1,3 +0,0 @@
sensor:
- platform: d01
name: D01 PM2.5 Concentration
-7
View File
@@ -1,7 +0,0 @@
substitutions:
tx_pin: GPIO4
rx_pin: GPIO5
packages:
uart: !include ../../test_build_components/common/uart/esp32-idf.yaml
d01: !include common.yaml
@@ -1,7 +0,0 @@
substitutions:
tx_pin: GPIO0
rx_pin: GPIO2
packages:
uart: !include ../../test_build_components/common/uart/esp8266-ard.yaml
d01: !include common.yaml
@@ -1,7 +0,0 @@
substitutions:
tx_pin: GPIO4
rx_pin: GPIO5
packages:
uart: !include ../../test_build_components/common/uart/rp2040-ard.yaml
d01: !include common.yaml
-3
View File
@@ -1,3 +0,0 @@
sensor:
- platform: ds1603l
name: ds1603l Distance
@@ -1,7 +0,0 @@
substitutions:
tx_pin: GPIO1
rx_pin: GPIO3
packages:
uart: !include ../../test_build_components/common/uart/esp32-idf.yaml
ds1603l: !include common.yaml
@@ -1,7 +0,0 @@
substitutions:
tx_pin: GPIO0
rx_pin: GPIO2
packages:
uart: !include ../../test_build_components/common/uart/esp8266-ard.yaml
ds1603l: !include common.yaml
-9
View File
@@ -1,9 +0,0 @@
wifi:
ssid: MySSID
password: password1
ota:
- platform: esphome
port: 3288
encryption:
key: "AAECAwQFBgcICQoLDA0ODxAREhMUFRYXGBkaGxwdHh8="
@@ -1,12 +0,0 @@
wifi:
ssid: MySSID
password: password1
api:
encryption:
key: "AAECAwQFBgcICQoLDA0ODxAREhMUFRYXGBkaGxwdHh8="
ota:
- platform: esphome
port: 3289
encryption:
@@ -1,2 +0,0 @@
packages:
ota: !include encryption.yaml
@@ -1,2 +0,0 @@
packages:
ota: !include encryption.yaml
@@ -1,2 +0,0 @@
packages:
ota: !include encryption.yaml
@@ -1,2 +0,0 @@
packages:
ota: !include encryption_inherit.yaml
-7
View File
@@ -21,13 +21,6 @@ The `yaml_config` fixture automatically loads YAML configurations based on the t
- The fixture file must exist or the test will fail with a clear error message
- The fixture automatically injects a dynamic port number into the API configuration
Tests marked `@pytest.mark.shared_yaml("name")` load `fixtures/name.yaml` instead
of the test-named file and compile it in a shared, hash-keyed build directory, so
the whole group pays one full compile and each test only a relink. The marker
argument must be a single-line string literal (CI test selection maps fixtures to
test files by scanning for it), and marked tests must hand the `yaml_config`
content to `run_compiled` unmodified.
### Key Fixtures
- `run_compiled` - Combines write, compile, and run operations into a single context manager
+74 -335
View File
@@ -4,22 +4,17 @@ from __future__ import annotations
import asyncio
from collections.abc import AsyncGenerator, Callable, Generator
from contextlib import AbstractAsyncContextManager, asynccontextmanager, suppress
from contextlib import AbstractAsyncContextManager, asynccontextmanager
import fcntl
from functools import cache
import hashlib
import logging
import os
from pathlib import Path
import platform
import re
import shutil
import signal
import socket
import subprocess
import sys
import tempfile
import time
from typing import TextIO
from aioesphomeapi import APIClient, APIConnectionError, LogParser, ReconnectLogic
@@ -28,13 +23,7 @@ import pytest_asyncio
import esphome.config
from esphome.core import CORE
from esphome.helpers import (
get_usable_cpu_count,
read_file,
rmtree,
write_file,
write_file_if_changed,
)
from esphome.helpers import get_usable_cpu_count
from esphome.platformio.toolchain import get_idedata
from .const import (
@@ -67,21 +56,6 @@ import pty # not available on Windows
pytest.register_assert_rewrite("tests.integration.entity_utils")
def pytest_configure(config: pytest.Config) -> None:
config.addinivalue_line(
"markers",
"shared_yaml(name): load fixtures/<name>.yaml and compile it in a shared, "
"hash-keyed incremental build directory",
)
FIXTURES_DIR = Path(__file__).parent / "fixtures"
REPO_ROOT = Path(__file__).resolve().parent.parent.parent
# CI caches parts of this path; keep in sync with ci.yml integration-tests.
INTEGRATION_TESTS_ROOT = Path.home() / ".esphome-integration-tests"
def _get_platformio_env(cache_dir: Path) -> dict[str, str]:
"""Get environment variables for PlatformIO with shared cache."""
env = os.environ.copy()
@@ -104,7 +78,7 @@ def _get_platformio_env(cache_dir: Path) -> dict[str, str]:
)
# Compile with THIS tree's esphome sources, not wherever the venv's editable
# install points (which may be a different git worktree or checkout).
repo_root = str(REPO_ROOT)
repo_root = str(Path(__file__).resolve().parent.parent.parent)
existing = env.get("PYTHONPATH")
env["PYTHONPATH"] = f"{repo_root}{os.pathsep}{existing}" if existing else repo_root
return env
@@ -114,7 +88,8 @@ def _get_platformio_env(cache_dir: Path) -> dict[str, str]:
def shared_platformio_cache() -> Generator[Path]:
"""Initialize a shared PlatformIO cache for all integration tests."""
# Use a dedicated directory for integration tests to avoid conflicts.
test_cache_dir = INTEGRATION_TESTS_ROOT
# CI caches parts of this path; keep in sync with ci.yml integration-tests.
test_cache_dir = Path.home() / ".esphome-integration-tests"
cache_dir = test_cache_dir / "platformio"
# Use a lock file in the home directory to ensure only one process initializes the cache
@@ -137,9 +112,7 @@ def shared_platformio_cache() -> Generator[Path]:
init_dir = Path(tmpdir)
fixture_path = Path(__file__).parent / "fixtures" / "cache_init.yaml"
config_path = init_dir / "cache_init.yaml"
config_path.write_text(
fixture_path.read_text(encoding="utf-8"), encoding="utf-8"
)
config_path.write_text(fixture_path.read_text())
# Run compilation to populate the cache
# We must succeed here to avoid race conditions where multiple
@@ -208,29 +181,21 @@ def unused_tcp_port(reserved_tcp_port: tuple[int, socket.socket]) -> int:
return reserved_tcp_port[0]
@pytest.fixture(autouse=True)
def isolated_preferences(monkeypatch: pytest.MonkeyPatch, tmp_path: Path) -> Path:
"""Give every test its own host prefs dir; prefs are keyed only by device
name, which tests sharing a fixture also share."""
prefdir = tmp_path / "prefs"
monkeypatch.setenv("ESPHOME_PREFDIR", str(prefdir))
return prefdir
@pytest_asyncio.fixture
async def yaml_config(request: pytest.FixtureRequest, unused_tcp_port: int) -> str:
"""Load YAML configuration based on test name."""
shared_name = _shared_yaml_name(request)
# Base test name: test_ prefix and any parametrization stripped
base_name = shared_name or request.node.name.replace("test_", "").partition("[")[0]
# Get the test function name
test_name: str = request.node.name
# Extract the base test name (remove test_ prefix and any parametrization)
base_name = test_name.replace("test_", "").partition("[")[0]
# Load the fixture file
fixture_path = FIXTURES_DIR / f"{base_name}.yaml"
fixture_path = Path(__file__).parent / "fixtures" / f"{base_name}.yaml"
if not fixture_path.exists():
raise FileNotFoundError(f"Fixture file not found: {fixture_path}")
loop = asyncio.get_running_loop()
content = await loop.run_in_executor(None, read_file, fixture_path)
content = await loop.run_in_executor(None, fixture_path.read_text)
# Replace the port in the config if it contains api section
if "api:" in content:
@@ -254,13 +219,11 @@ async def yaml_config(request: pytest.FixtureRequest, unused_tcp_port: int) -> s
# Replace external component path placeholder if present
if "EXTERNAL_COMPONENT_PATH" in content:
external_components_path = str(FIXTURES_DIR / "external_components")
external_components_path = str(
Path(__file__).parent / "fixtures" / "external_components"
)
content = content.replace("EXTERNAL_COMPONENT_PATH", external_components_path)
if shared_name is not None:
# _compile verifies the marked test compiles this content unmodified
request.node._shared_yaml_content = content
return content
@@ -270,218 +233,24 @@ async def write_yaml_config(
) -> AsyncGenerator[ConfigWriter]:
"""Write YAML configuration to a file."""
# Get the test name for default filename
base_name = request.node.name.replace("test_", "").partition("[")[0]
test_name = request.node.name
base_name = test_name.replace("test_", "").split("[")[0]
async def _write_config(content: str, filename: str | None = None) -> Path:
if filename is None:
filename = f"{base_name}.yaml"
config_path = integration_test_dir / filename
loop = asyncio.get_running_loop()
await loop.run_in_executor(None, write_file, config_path, content)
await loop.run_in_executor(None, config_path.write_text, content)
return config_path
yield _write_config
# Deliberately not CI-cached (ci.yml caches only platformio/ subpaths); stale
# dirs for a fixture are pruned when its content hash changes.
SHARED_BUILDS_ROOT = INTEGRATION_TESTS_ROOT / "builds"
# In the dir name (not just the hash) so pruning stays inside this checkout
_REPO_KEY = hashlib.sha256(str(REPO_ROOT).encode()).hexdigest()[:8]
# Give a contended shared build lock time for a full cold compile ahead of us
_SHARED_LOCK_TIMEOUT_S = 900
_SHARED_LOCK_POLL_S = 0.1
_SHARED_LOCK_REPORT_S = 30
# Reclaims dirs orphaned by fixture renames or deleted checkouts
_STALE_BUILD_MAX_AGE_S = 30 * 24 * 3600
# ELF path per shared build dir; constant once compiled, so resolve it only once
_shared_elf_paths: dict[Path, Path] = {}
# Dirs this process already swept; pruning is session-scoped work
_pruned_dirs: set[Path] = set()
def _shared_yaml_name(request: pytest.FixtureRequest) -> str | None:
"""Name passed to the shared_yaml marker, or None when unmarked."""
marker = request.node.get_closest_marker("shared_yaml")
if marker is None:
return None
# Exactly one \w+ positional arg: the name doubles as a build dir
# component, and CI test selection (script/helpers.py) parses the same shape
if (
len(marker.args) != 1
or marker.kwargs
or not re.fullmatch(r"\w+", str(marker.args[0]))
):
raise ValueError(
"shared_yaml marker requires exactly one \\w+ fixture name literal"
)
return marker.args[0]
def _shared_build_prefix(name: str) -> str:
return f"{name}-{_REPO_KEY}-"
@cache
def _shared_build_dir(name: str) -> Path:
"""Dir keyed by checkout and fixture source, before per-test injections."""
key = hashlib.sha256((FIXTURES_DIR / f"{name}.yaml").read_bytes()).hexdigest()[:16]
return SHARED_BUILDS_ROOT / (_shared_build_prefix(name) + key)
def _read_stamp(stamp: Path, shared_dir: Path) -> Path | None:
"""ELF path recorded by the last completed compile, or None."""
try:
text = stamp.read_text(encoding="utf-8").strip()
except FileNotFoundError:
return None
except OSError as err:
print(f"Cannot read {stamp}: {err}")
return None
if not text:
print(f"Ignoring empty stamp {stamp}")
return None
built = Path(text)
# Never trust a stamp pointing outside its own build dir as an unlink target
if shared_dir.resolve() in built.resolve().parents:
return built
print(f"Ignoring stamp {stamp} pointing outside {shared_dir}")
return None
def _unused_since(stale: Path, cutoff: float) -> bool:
"""Whether a build dir looks untouched since cutoff; unknown counts as used."""
# Newest of the .built stamp (rewritten by every completed compile) and the
# dir itself (freshened by a worker claiming the dir before locking)
newest: float | None = None
for probe in (stale / ".built", stale):
try:
mtime = probe.stat().st_mtime
except FileNotFoundError:
continue
except NotADirectoryError:
return True # a stray file where a dir should be; reclaimable
except OSError as err:
print(f"Cannot age-probe {stale}: {err}")
return False # unknown never authorizes deletion
newest = mtime if newest is None else max(newest, mtime)
return newest is not None and newest < cutoff
def _prune_stale_builds(name: str, keep: Path) -> None:
"""Remove outdated build dirs (blocking, run in executor): this checkout's
other dirs for the fixture, plus anything untouched for 30 days. Tolerates
other workers pruning the same dirs concurrently."""
cutoff = time.time() - _STALE_BUILD_MAX_AGE_S
prefix = _shared_build_prefix(name)
for stale in SHARED_BUILDS_ROOT.iterdir():
if stale == keep:
continue
same_fixture = stale.name.startswith(prefix)
if not same_fixture and not _unused_since(stale, cutoff):
continue
# Creating .lock bumps the dir mtime, so remember whether the re-probe
# under the lock can trust it
lock_preexisting = (stale / ".lock").exists()
try:
lock_file = (stale / ".lock").open("w")
except FileNotFoundError:
continue # pruned by another worker meanwhile
except NotADirectoryError:
print(f"Removing stray file {stale}")
stale.unlink(missing_ok=True)
continue
except OSError as err:
print(f"Cannot prune {stale}: {err}")
continue
with lock_file:
try:
fcntl.flock(lock_file.fileno(), fcntl.LOCK_EX | fcntl.LOCK_NB)
except BlockingIOError:
continue # still in use by another run
# Re-probe under the lock: a worker freshens its dir before
# locking, so a just-claimed dir no longer looks unused. A dir
# whose .lock we just created cannot be held by anyone, and our
# own open bumped its mtime, so its pre-open probe stands
if (
lock_preexisting
and not same_fixture
and not _unused_since(stale, cutoff)
):
continue
# rmtree tolerates races; a leftover partial tree only costs a
# rebuild, since the ELF is deleted before every compile
try:
rmtree(stale)
except OSError as err:
print(f"Failed to prune {stale}: {err}")
async def _run_esphome_compile(
config_path: Path, cwd: Path, env: dict[str, str]
) -> None:
"""Run `esphome compile`, retrying up to 3 times on a segfault."""
max_retries = 3
for attempt in range(max_retries):
# Compile using subprocess, inheriting stdout/stderr to show progress
proc = await asyncio.create_subprocess_exec(
sys.executable,
"-m",
"esphome",
"compile",
str(config_path),
cwd=cwd,
stdout=None, # Inherit stdout
stderr=None, # Inherit stderr
stdin=asyncio.subprocess.DEVNULL,
# Start in a new process group to isolate signal handling
start_new_session=True,
env=env,
close_fds=False,
)
await proc.wait()
if proc.returncode == 0:
break
if proc.returncode == -11 and attempt < max_retries - 1:
# Segfault (-11 = SIGSEGV), retry
print(
f"Compilation segfaulted (attempt {attempt + 1}/{max_retries}), retrying..."
)
await asyncio.sleep(1) # Brief pause before retry
continue
raise RuntimeError(
f"Failed to compile {config_path}, return code: {proc.returncode}. "
f"Run with 'pytest -s' to see compilation output."
)
def _resolve_compiled_binary(config_path: Path) -> Path:
"""Load the config to learn the compiled ELF path (blocking, run in executor)."""
CORE.reset() # Reset CORE state between test runs
CORE.config_path = config_path
config = esphome.config.read_config(
{"command": "compile", "config": str(config_path)}
)
if config is None:
raise RuntimeError(f"Failed to read config from {config_path}")
idedata = get_idedata(config)
binary_path = Path(idedata.firmware_elf_path)
if not binary_path.exists():
raise RuntimeError(f"Compiled binary not found at {binary_path}")
return binary_path
@pytest_asyncio.fixture
async def compile_esphome(
integration_test_dir: Path,
shared_platformio_cache: Path,
request: pytest.FixtureRequest,
) -> AsyncGenerator[CompileFunction]:
"""Compile an ESPHome configuration and return the binary path."""
@@ -489,96 +258,66 @@ async def compile_esphome(
# Use the shared PlatformIO cache for faster compilation
# This avoids re-downloading dependencies for each test
env = _get_platformio_env(shared_platformio_cache)
# Retry compilation up to 3 times if we get a segfault
max_retries = 3
for attempt in range(max_retries):
# Compile using subprocess, inheriting stdout/stderr to show progress
proc = await asyncio.create_subprocess_exec(
sys.executable,
"-m",
"esphome",
"compile",
str(config_path),
cwd=integration_test_dir,
stdout=None, # Inherit stdout
stderr=None, # Inherit stderr
stdin=asyncio.subprocess.DEVNULL,
# Start in a new process group to isolate signal handling
start_new_session=True,
env=env,
close_fds=False,
)
await proc.wait()
if proc.returncode == 0:
# Success!
break
if proc.returncode == -11 and attempt < max_retries - 1:
# Segfault (-11 = SIGSEGV), retry
print(
f"Compilation segfaulted (attempt {attempt + 1}/{max_retries}), retrying..."
)
await asyncio.sleep(1) # Brief pause before retry
continue
# Other error or final retry
raise RuntimeError(
f"Failed to compile {config_path}, return code: {proc.returncode}. "
f"Run with 'pytest -s' to see compilation output."
)
# Load the config to get idedata (blocking call, must use executor)
loop = asyncio.get_running_loop()
name = _shared_yaml_name(request)
if name is None:
await _run_esphome_compile(config_path, integration_test_dir, env)
return await loop.run_in_executor(
None, _resolve_compiled_binary, config_path
def _read_config_and_get_binary():
CORE.reset() # Reset CORE state between test runs
CORE.config_path = config_path
config = esphome.config.read_config(
{"command": "compile", "config": str(config_path)}
)
if config is None:
raise RuntimeError(f"Failed to read config from {config_path}")
# Shared fixture: build in a hash-keyed dir so tests sharing a config
# pay one full compile and later only a main.cpp (port) rebuild + relink
shared_dir = _shared_build_dir(name)
shared_dir.mkdir(parents=True, exist_ok=True)
# Freshen the dir before locking so a concurrent age sweep, which
# re-probes under the lock, never reaps a dir a worker just claimed;
# if a peer reaped it already, the guarded lock open recreates it
with suppress(FileNotFoundError):
os.utime(shared_dir)
if shared_dir not in _pruned_dirs:
_pruned_dirs.add(shared_dir)
await loop.run_in_executor(None, _prune_stale_builds, name, shared_dir)
shared_config = shared_dir / f"{name}.yaml"
private_binary = integration_test_dir / f"{name}.elf"
content = await loop.run_in_executor(None, read_file, config_path)
if content != getattr(request.node, "_shared_yaml_content", None):
# The dir is keyed by the fixture source; a mutated config would be
# cached under a hash that does not describe it
raise RuntimeError(
"shared_yaml tests must compile the yaml_config content unmodified"
)
# flock serializes concurrent xdist workers; closing the fd releases it.
# Hand-rolled rather than filelock.FileLock: non-blocking retries keep
# the wait cancellable, while a blocking acquire in an executor thread
# would survive test cancellation holding the fd
try:
lock_file = (shared_dir / ".lock").open("w")
except FileNotFoundError:
# A peer run pruning divergent hashes reaped the dir between our
# mkdir and this open; recreate it and pay a full rebuild
shared_dir.mkdir(parents=True, exist_ok=True)
lock_file = (shared_dir / ".lock").open("w")
with lock_file:
start = time.monotonic()
last_report = start
while True:
try:
fcntl.flock(lock_file.fileno(), fcntl.LOCK_EX | fcntl.LOCK_NB)
break
except BlockingIOError:
now = time.monotonic()
if now - start > _SHARED_LOCK_TIMEOUT_S:
raise RuntimeError(
f"Timed out waiting for the {shared_dir} lock"
) from None
if now - last_report >= _SHARED_LOCK_REPORT_S:
last_report = now
print(
f"Waited {now - start:.0f}s for another worker's "
f"build of {shared_dir.name}"
)
await asyncio.sleep(_SHARED_LOCK_POLL_S)
# .built carries the ELF path of the last completed compile, so
# later workers skip the config re-read in _resolve_compiled_binary
stamp = shared_dir / ".built"
if (built := _shared_elf_paths.get(shared_dir)) is None:
built = await loop.run_in_executor(None, _read_stamp, stamp, shared_dir)
# Delete the ELF before compiling: whatever exists afterwards is
# this compile's output, so no staleness check is ever needed.
# With no usable stamp, sweep any leftover at the known layout
if built is not None:
built.unlink(missing_ok=True)
else:
# Layout-agnostic: ESPHOME_BUILD_PATH can move the build tree
for leftover in shared_dir.rglob("program"):
if leftover.is_file():
leftover.unlink()
await loop.run_in_executor(
None, write_file_if_changed, shared_config, content
)
await _run_esphome_compile(shared_config, shared_dir, env)
if built is None or not built.exists():
built = await loop.run_in_executor(
None, _resolve_compiled_binary, shared_config
)
_shared_elf_paths[shared_dir] = built
await loop.run_in_executor(None, write_file, stamp, str(built))
# Copy out before unlocking: another worker may relink firmware.elf
# while this test is still running its private copy
await loop.run_in_executor(None, shutil.copy2, built, private_binary)
return private_binary
# Get the compiled binary path
idedata = get_idedata(config)
return Path(idedata.firmware_elf_path)
binary_path = await loop.run_in_executor(None, _read_config_and_get_binary)
if not binary_path.exists():
raise RuntimeError(f"Compiled binary not found at {binary_path}")
return binary_path
yield _compile
@@ -1,11 +0,0 @@
esphome:
name: host-ota-test
host:
api:
ota:
- platform: esphome
port: __OTA_PORT__
encryption:
key: "AAECAwQFBgcICQoLDA0ODxAREhMUFRYXGBkaGxwdHh8="
logger:
level: DEBUG
@@ -0,0 +1,58 @@
esphome:
name: test-batch-window-filters
host:
api:
batch_delay: 0ms # Disable batching to receive all state updates
logger:
level: DEBUG
# Template sensor that we'll use to publish values
sensor:
- platform: template
name: "Source Sensor"
id: source_sensor
accuracy_decimals: 2
# Batch window filters (window_size == send_every) - use streaming filters
- platform: copy
source_id: source_sensor
name: "Min Sensor"
id: min_sensor
filters:
- min:
window_size: 5
send_every: 5
send_first_at: 1
- platform: copy
source_id: source_sensor
name: "Max Sensor"
id: max_sensor
filters:
- max:
window_size: 5
send_every: 5
send_first_at: 1
- platform: copy
source_id: source_sensor
name: "Moving Avg Sensor"
id: moving_avg_sensor
filters:
- sliding_window_moving_average:
window_size: 5
send_every: 5
send_first_at: 1
# Button to trigger publishing test values
button:
- platform: template
name: "Publish Values Button"
id: publish_button
on_press:
- lambda: |-
// Publish 10 values: 1.0, 2.0, ..., 10.0
for (int i = 1; i <= 10; i++) {
id(source_sensor).publish_state(float(i));
}
@@ -0,0 +1,111 @@
esphome:
name: uart-mock-modbus-cli-rw
host:
api:
logger:
level: VERBOSE
external_components:
- source:
type: local
path: EXTERNAL_COMPONENT_PATH
# Dummy uart entry to satisfy modbus's DEPENDENCIES = ["uart"]
# The actual UART bus used is the uart_mock component below
uart:
baud_rate: 115200
port: /dev/null
# Two virtual buses looped back to each other: the client's transmissions reach the server and the
# server's replies reach the client. auto_start so forwarding is active before the button fires.
uart_mock:
- id: virtual_uart_server
baud_rate: 9600
auto_start: true
debug:
on_tx:
- then:
- uart_mock.inject_rx:
id: virtual_uart_client
data: !lambda return data;
- id: virtual_uart_client
baud_rate: 9600
auto_start: true
debug:
on_tx:
- then:
- uart_mock.inject_rx:
id: virtual_uart_server
data: !lambda return data;
globals:
- id: stored_1
type: uint16_t
initial_value: "0"
modbus:
- uart_id: virtual_uart_server
id: virtual_modbus_server
role: server
- uart_id: virtual_uart_client
id: virtual_modbus_client
role: client
turnaround_time: 10ms
modbus_server:
- address: 1
modbus_id: virtual_modbus_server
registers:
# Writable + readable register: the read publishes what it returns, so the test can confirm the
# write half of the 0x17 ran before the read half (Modbus 6.17).
- address: 0x01
value_type: U_WORD
read_lambda: |-
id(srv_read_1).publish_state(id(stored_1));
return id(stored_1);
write_lambda: |-
id(stored_1) = x;
id(srv_write_1).publish_state(x);
return true;
# Read-only register, returned together with 0x01 by the 2-register read half.
- address: 0x02
value_type: U_WORD
read_lambda: return 0x00AA;
sensor:
# Server-side observations.
- platform: template
name: "srv_write_1"
id: srv_write_1
- platform: template
name: "srv_read_1"
id: srv_read_1
# Client-side read-back: the values the client's on_response received.
- platform: template
name: "client_read_0"
id: client_read_0
- platform: template
name: "client_read_1"
id: client_read_1
button:
- platform: template
name: "Start Scenario"
id: start_scenario_btn
on_press:
# FC 0x17: write reg 0x0001 = 0x1234, then read regs 0x0001..0x0002 back in the same transaction.
- modbus_client.read_write_multiple_registers:
address: 0x01
read_address: 0x0001
read_count: 2
write_address: 0x0001
values: [0x1234]
on_response:
then:
- lambda: |-
// values is the read-back block: reg 0x0001 (must be the just-written 0x1234) and reg 0x0002.
if (values.size() >= 2) {
id(client_read_0).publish_state(values[0]);
id(client_read_1).publish_state(values[1]);
}
@@ -0,0 +1,88 @@
esphome:
name: uart-mock-modbus-custom-pdu
host:
api:
logger:
level: VERBOSE
external_components:
- source:
type: local
path: EXTERNAL_COMPONENT_PATH
# Dummy uart entry to satisfy modbus's DEPENDENCIES = ["uart"]
# The actual UART bus used is the uart_mock component below
uart:
baud_rate: 115200
port: /dev/null
uart_mock:
- id: virtual_uart_server
baud_rate: 9600
auto_start: true
debug:
on_tx:
- then:
- uart_mock.inject_rx:
id: virtual_uart_controller
data: !lambda return data;
- id: virtual_uart_controller
baud_rate: 9600
auto_start: true
debug:
on_tx:
- then:
- uart_mock.inject_rx:
id: virtual_uart_server
data: !lambda return data;
modbus:
- uart_id: virtual_uart_server
id: virtual_modbus_server
role: server
- uart_id: virtual_uart_controller
id: virtual_modbus_controller
role: client
turnaround_time: 10ms
modbus_controller:
- address: 1
modbus_id: virtual_modbus_controller
id: modbus_controller_1
update_interval: 1s
modbus_server:
- address: 1
modbus_id: virtual_modbus_server
id: modbus_server_1
registers:
- address: 0x01
value_type: U_WORD
read_lambda: return 259;
sensor:
# Plain read to confirm the controller <-> server link is up.
- platform: modbus_controller
modbus_controller_id: modbus_controller_1
name: "plain_read"
address: 0x01
register_type: holding
value_type: U_WORD
# Custom PDU: read holding register 0x0001, count 1. The PDU is
# {function code, address hi, address lo, count hi, count lo}; the device
# address and CRC are added by the hub. The lambda parses the response payload
# (the register value, big-endian).
- platform: modbus_controller
modbus_controller_id: modbus_controller_1
name: "custom_read"
custom_pdu: [0x03, 0x00, 0x01, 0x00, 0x01]
lambda: |-
if (data.size() < 2) return {};
return (float) ((data[0] << 8) | data[1]);
button:
- platform: template
name: "Start Scenario"
id: start_scenario_btn
# This test does not have anything to start (mock is autostart)
@@ -0,0 +1,106 @@
esphome:
name: uart-mock-modbus-dep-buffer
host:
api:
logger:
level: VERBOSE
external_components:
- source:
type: local
path: EXTERNAL_COMPONENT_PATH
# Dummy uart entry to satisfy modbus's DEPENDENCIES = ["uart"]
# The actual UART bus used is the uart_mock component below
uart:
baud_rate: 115200
port: /dev/null
uart_mock:
- id: virtual_uart_server
baud_rate: 9600
auto_start: true
debug:
on_tx:
- then:
- uart_mock.inject_rx:
id: virtual_uart_controller
data: !lambda return data;
- id: virtual_uart_controller
baud_rate: 9600
auto_start: true
debug:
on_tx:
- then:
- uart_mock.inject_rx:
id: virtual_uart_server
data: !lambda return data;
globals:
- id: reg10
type: uint16_t
initial_value: "0"
modbus:
- uart_id: virtual_uart_server
id: virtual_modbus_server
role: server
- uart_id: virtual_uart_controller
id: virtual_modbus_controller
role: client
turnaround_time: 10ms
modbus_controller:
- address: 1
modbus_id: virtual_modbus_controller
id: modbus_controller_1
update_interval: 1s
modbus_server:
- address: 1
modbus_id: virtual_modbus_server
id: modbus_server_1
registers:
- address: 0x10
value_type: U_WORD
read_lambda: return id(reg10);
write_lambda: |-
id(reg10) = x;
return true;
# A number whose write_lambda uses the DEPRECATED buffer parameter (fills `payload` with a legacy raw
# frame as words: device address + function code + data) instead of the new item->write_* API. The write
# must still land with its legacy semantics, and the one-time deprecation warning must fire only once per
# entity no matter how many writes happen.
number:
- platform: modbus_controller
modbus_controller_id: modbus_controller_1
name: "buf_number"
id: buf_number
address: 0x10
register_type: holding
value_type: U_WORD
min_value: 0
max_value: 1000
step: 1
write_lambda: |-
// Legacy raw frame as words: [addr 0x01 | fc 0x06], register 0x0010, value.
payload.push_back(0x0106);
payload.push_back(0x0010);
payload.push_back((uint16_t) x);
return {};
# Reports the server-side register so the test can observe that the deprecated buffer write landed.
sensor:
- platform: template
name: "written_value"
id: written_value
update_interval: 0.5s
lambda: "return id(reg10);"
button:
- platform: template
name: "Start Scenario"
id: start_scenario_btn
# The test drives the writes via number_command; the mock is autostart.
@@ -0,0 +1,95 @@
esphome:
name: uart-mock-modbus-lambda-invert
host:
api:
logger:
level: VERBOSE
external_components:
- source:
type: local
path: EXTERNAL_COMPONENT_PATH
# Dummy uart entry to satisfy modbus's DEPENDENCIES = ["uart"]
# The actual UART bus used is the uart_mock component below
uart:
baud_rate: 115200
port: /dev/null
uart_mock:
- id: virtual_uart_server
baud_rate: 9600
auto_start: true
debug:
on_tx:
- then:
- uart_mock.inject_rx:
id: virtual_uart_controller
data: !lambda return data;
- id: virtual_uart_controller
baud_rate: 9600
auto_start: true
debug:
on_tx:
- then:
- uart_mock.inject_rx:
id: virtual_uart_server
data: !lambda return data;
globals:
- id: reg40
type: uint16_t
initial_value: "5"
modbus:
- uart_id: virtual_uart_server
id: virtual_modbus_server
role: server
- uart_id: virtual_uart_controller
id: virtual_modbus_controller
role: client
turnaround_time: 10ms
modbus_controller:
- address: 1
modbus_id: virtual_modbus_controller
id: modbus_controller_1
update_interval: 1s
modbus_server:
- address: 1
modbus_id: virtual_modbus_server
id: modbus_server_1
registers:
- address: 0x40
value_type: U_WORD
read_lambda: return id(reg40);
write_lambda: id(reg40) = x; return true;
# An active-low holding switch: the write_lambda inverts the wire value, but the entity must still
# report the REQUESTED state. assumed_state keeps the register unpolled, so the published state comes
# only from write_state() - turning ON writes 0x0000 yet the switch shows ON.
switch:
- platform: modbus_controller
modbus_controller_id: modbus_controller_1
name: "invert_switch"
register_type: holding
address: 0x40
assumed_state: true
write_lambda: |-
return !x;
sensor:
- platform: modbus_controller
modbus_controller_id: modbus_controller_1
name: "reg_40"
address: 0x40
register_type: holding
value_type: U_WORD
button:
- platform: template
name: "Start Scenario"
id: start_scenario_btn
# This test does not have anything to start (mock is autostart)
@@ -0,0 +1,97 @@
esphome:
name: uart-mock-modbus-lambda-write
host:
api:
logger:
level: VERBOSE
external_components:
- source:
type: local
path: EXTERNAL_COMPONENT_PATH
# Dummy uart entry to satisfy modbus's DEPENDENCIES = ["uart"]
# The actual UART bus used is the uart_mock component below
uart:
baud_rate: 115200
port: /dev/null
uart_mock:
- id: virtual_uart_server
baud_rate: 9600
auto_start: true
debug:
on_tx:
- then:
- uart_mock.inject_rx:
id: virtual_uart_controller
data: !lambda return data;
- id: virtual_uart_controller
baud_rate: 9600
auto_start: true
debug:
on_tx:
- then:
- uart_mock.inject_rx:
id: virtual_uart_server
data: !lambda return data;
globals:
- id: reg30
type: uint16_t
initial_value: "0"
modbus:
- uart_id: virtual_uart_server
id: virtual_modbus_server
role: server
- uart_id: virtual_uart_controller
id: virtual_modbus_controller
role: client
turnaround_time: 10ms
modbus_controller:
- address: 1
modbus_id: virtual_modbus_controller
id: modbus_controller_1
update_interval: 1s
modbus_server:
- address: 1
modbus_id: virtual_modbus_server
id: modbus_server_1
registers:
- address: 0x30
value_type: U_WORD
read_lambda: return id(reg30);
write_lambda: id(reg30) = x; return true;
# A COIL-type switch (assumed_state, write-only) whose write_lambda ignores its own coil type and instead
# drives a HOLDING-REGISTER write on the mock server through the entity itself: `item` IS the command, so
# item->write_single_register() sends a register write from a coil entity (cross-type). Returning nothing
# (an empty optional) tells the write path the lambda already dispatched the frame - no default coil write.
switch:
- platform: modbus_controller
modbus_controller_id: modbus_controller_1
name: "cross_switch"
register_type: coil
address: 0x00
assumed_state: true
write_lambda: |-
item->write_single_register(0x30, x ? 1234 : 0);
return {};
sensor:
- platform: modbus_controller
modbus_controller_id: modbus_controller_1
name: "reg_30"
address: 0x30
register_type: holding
value_type: U_WORD
button:
- platform: template
name: "Start Scenario"
id: start_scenario_btn
# This test does not have anything to start (mock is autostart)
@@ -1,233 +0,0 @@
esphome:
name: uart-mock-modbus-loopback
host:
api:
logger:
level: VERBOSE
external_components:
- source:
type: local
path: EXTERNAL_COMPONENT_PATH
# Dummy uart entry to satisfy modbus's DEPENDENCIES = ["uart"]
# The actual UART bus used is the uart_mock component below
uart:
baud_rate: 115200
port: /dev/null
# Shared loopback fixture (see the shared_yaml markers in the test file);
# register spaces are disjoint so each test only observes its own entities.
uart_mock:
- id: virtual_uart_server
baud_rate: 9600
auto_start: true
debug:
on_tx:
- then:
- uart_mock.inject_rx:
id: virtual_uart_controller
data: !lambda return data;
- id: virtual_uart_controller
baud_rate: 9600
auto_start: true
debug:
on_tx:
- then:
- uart_mock.inject_rx:
id: virtual_uart_server
data: !lambda return data;
globals:
- id: reg10
type: uint16_t
initial_value: "100"
- id: reg11
type: uint16_t
initial_value: "200"
- id: reg12
type: uint16_t
initial_value: "300"
- id: reg13
type: uint16_t
initial_value: "0xABCD"
- id: reg30
type: uint16_t
initial_value: "0"
- id: reg40
type: uint16_t
initial_value: "5"
- id: reg50
type: uint16_t
initial_value: "0"
modbus:
- uart_id: virtual_uart_server
id: virtual_modbus_server
role: server
- uart_id: virtual_uart_controller
id: virtual_modbus_controller
role: client
turnaround_time: 10ms
modbus_controller:
- address: 1
modbus_id: virtual_modbus_controller
id: modbus_controller_1
update_interval: 1s
modbus_server:
- address: 1
modbus_id: virtual_modbus_server
registers:
- address: 0x01
value_type: U_WORD
read_lambda: return 259;
- address: 0x10
value_type: U_WORD
read_lambda: return id(reg10);
write_lambda: id(reg10) = x; return true;
- address: 0x11
value_type: U_WORD
read_lambda: return id(reg11);
write_lambda: id(reg11) = x; return true;
- address: 0x12
value_type: U_WORD
read_lambda: return id(reg12);
write_lambda: id(reg12) = x; return true;
- address: 0x13
value_type: U_WORD
read_lambda: return id(reg13);
- address: 0x30
value_type: U_WORD
read_lambda: return id(reg30);
write_lambda: id(reg30) = x; return true;
- address: 0x40
value_type: U_WORD
read_lambda: return id(reg40);
write_lambda: id(reg40) = x; return true;
- address: 0x50
value_type: U_WORD
read_lambda: return id(reg50);
write_lambda: id(reg50) = x; return true;
# Byte-based offset: 2 bytes -> register 0x11 (the old code folded it in as a
# register count, hitting 0x12). assumed_state keeps the switch write-only.
switch:
- platform: modbus_controller
modbus_controller_id: modbus_controller_1
name: "offset_switch"
register_type: holding
address: 0x10
offset: 2
assumed_state: true
# Reading switch, byte offset 6 -> register 0x13; the pre-fix resolution (0x16)
# would draw ILLEGAL_DATA_ADDRESS and never publish.
- platform: modbus_controller
modbus_controller_id: modbus_controller_1
name: "read_offset_switch"
register_type: holding
address: 0x10
offset: 6
bitmask: 0x1
# Coil switch whose write_lambda dispatches a holding-register write via `item`;
# returning an empty optional suppresses the default coil write.
- platform: modbus_controller
modbus_controller_id: modbus_controller_1
name: "cross_switch"
register_type: coil
address: 0x00
assumed_state: true
write_lambda: |-
item->write_single_register(0x30, x ? 1234 : 0);
return {};
# Active-low: the write_lambda inverts the wire value but the entity must still
# report the requested state (assumed_state keeps the register unpolled).
- platform: modbus_controller
modbus_controller_id: modbus_controller_1
name: "invert_switch"
register_type: holding
address: 0x40
assumed_state: true
write_lambda: |-
return !x;
# Uses the deprecated buffer parameter (legacy raw frame as words); the write
# must land and the deprecation warning must fire only once per entity.
number:
- platform: modbus_controller
modbus_controller_id: modbus_controller_1
name: "buf_number"
id: buf_number
address: 0x50
register_type: holding
value_type: U_WORD
min_value: 0
max_value: 1000
step: 1
write_lambda: |-
// Legacy raw frame as words: [addr 0x01 | fc 0x06], register 0x0050, value.
payload.push_back(0x0106);
payload.push_back(0x0050);
payload.push_back((uint16_t) x);
return {};
sensor:
- platform: modbus_controller
modbus_controller_id: modbus_controller_1
name: "plain_read"
address: 0x01
register_type: holding
value_type: U_WORD
# Custom PDU: read holding register 0x0001; device address and CRC are added
# by the hub. The lambda parses the big-endian register value.
- platform: modbus_controller
modbus_controller_id: modbus_controller_1
name: "custom_read"
custom_pdu: [0x03, 0x00, 0x01, 0x00, 0x01]
lambda: |-
if (data.size() < 2) return {};
return (float) ((data[0] << 8) | data[1]);
- platform: modbus_controller
modbus_controller_id: modbus_controller_1
name: "reg_10"
address: 0x10
register_type: holding
value_type: U_WORD
- platform: modbus_controller
modbus_controller_id: modbus_controller_1
name: "reg_11"
address: 0x11
register_type: holding
value_type: U_WORD
- platform: modbus_controller
modbus_controller_id: modbus_controller_1
name: "reg_12"
address: 0x12
register_type: holding
value_type: U_WORD
- platform: modbus_controller
modbus_controller_id: modbus_controller_1
name: "reg_30"
address: 0x30
register_type: holding
value_type: U_WORD
- platform: modbus_controller
modbus_controller_id: modbus_controller_1
name: "reg_40"
address: 0x40
register_type: holding
value_type: U_WORD
# Reports the server-side register so the test can observe that the deprecated buffer write landed.
- platform: template
name: "written_value"
id: written_value
update_interval: 0.5s
lambda: "return id(reg50);"
button:
- platform: template
name: "Start Scenario"
id: start_scenario_btn
# Nothing to start (mock is autostart); tests drive entities directly
@@ -0,0 +1,138 @@
esphome:
name: uart-mock-modbus-reg-offset
host:
api:
logger:
level: VERBOSE
external_components:
- source:
type: local
path: EXTERNAL_COMPONENT_PATH
# Dummy uart entry to satisfy modbus's DEPENDENCIES = ["uart"]
# The actual UART bus used is the uart_mock component below
uart:
baud_rate: 115200
port: /dev/null
uart_mock:
- id: virtual_uart_server
baud_rate: 9600
auto_start: true
debug:
on_tx:
- then:
- uart_mock.inject_rx:
id: virtual_uart_controller
data: !lambda return data;
- id: virtual_uart_controller
baud_rate: 9600
auto_start: true
debug:
on_tx:
- then:
- uart_mock.inject_rx:
id: virtual_uart_server
data: !lambda return data;
globals:
- id: reg10
type: uint16_t
initial_value: "100"
- id: reg11
type: uint16_t
initial_value: "200"
- id: reg12
type: uint16_t
initial_value: "300"
- id: reg13
type: uint16_t
initial_value: "0xABCD"
modbus:
- uart_id: virtual_uart_server
id: virtual_modbus_server
role: server
- uart_id: virtual_uart_controller
id: virtual_modbus_controller
role: client
turnaround_time: 10ms
modbus_controller:
- address: 1
modbus_id: virtual_modbus_controller
id: modbus_controller_1
update_interval: 1s
modbus_server:
- address: 1
modbus_id: virtual_modbus_server
id: modbus_server_1
registers:
- address: 0x10
value_type: U_WORD
read_lambda: return id(reg10);
write_lambda: id(reg10) = x; return true;
- address: 0x11
value_type: U_WORD
read_lambda: return id(reg11);
write_lambda: id(reg11) = x; return true;
- address: 0x12
value_type: U_WORD
read_lambda: return id(reg12);
write_lambda: id(reg12) = x; return true;
- address: 0x13
value_type: U_WORD
read_lambda: return id(reg13);
write_lambda: id(reg13) = x; return true;
# A holding-register switch at 0x10 with a 2-BYTE offset. offset is byte-based, so the write must target
# register 0x10 + 2/2 = 0x11. The old (pre-fix) behavior folded offset into the address as a register
# count, hitting 0x12 instead. assumed_state keeps the switch write-only so it does not read any register.
switch:
- platform: modbus_controller
modbus_controller_id: modbus_controller_1
name: "offset_switch"
register_type: holding
address: 0x10
offset: 2
assumed_state: true
# A holding-register switch that READS its state. Byte offset 6 -> register 0x10 + 6/2 = 0x13. Post-fix
# the switch itself resolves to 0x13 (the even byte offset folds into the address as whole registers) and
# joins the 0x10..0x13 range, so no separate 0x13 sensor is needed. Pre-fix the whole byte offset folds
# into the address (0x16), where the server answers ILLEGAL_DATA_ADDRESS and the switch never publishes.
- platform: modbus_controller
modbus_controller_id: modbus_controller_1
name: "read_offset_switch"
register_type: holding
address: 0x10
offset: 6
bitmask: 0x1
sensor:
- platform: modbus_controller
modbus_controller_id: modbus_controller_1
name: "reg_10"
address: 0x10
register_type: holding
value_type: U_WORD
- platform: modbus_controller
modbus_controller_id: modbus_controller_1
name: "reg_11"
address: 0x11
register_type: holding
value_type: U_WORD
- platform: modbus_controller
modbus_controller_id: modbus_controller_1
name: "reg_12"
address: 0x12
register_type: holding
value_type: U_WORD
button:
- platform: template
name: "Start Scenario"
id: start_scenario_btn
# This test does not have anything to start (mock is autostart)
@@ -0,0 +1,124 @@
esphome:
name: uart-mock-modbus-server-test
host:
api:
logger:
level: VERBOSE
external_components:
- source:
type: local
path: EXTERNAL_COMPONENT_PATH
# Dummy uart entry to satisfy modbus's DEPENDENCIES = ["uart"]
# The actual UART bus used is the uart_mock component below
uart:
baud_rate: 115200
port: /dev/null
uart_mock:
- id: virtual_uart_dev
baud_rate: 9600
rx_full_threshold: 120
rx_timeout: 2
auto_start: false
debug:
injections:
- delay: 100ms
inject_rx: [0x01, 0x03, 0x00, 0x03, 0x00, 0x01, 0x74, 0x0A] # Read holding register 3 on device 1 (basic_read)
- delay: 100ms
# Read holding register 7 on device 2
# Reply from device 2
# Read holding register 5 on device 1 (read_after_peer_response)
inject_rx:
[
0x02,
0x03,
0x00,
0x07,
0x00,
0x01,
0x35,
0xF8,
0x02,
0x03,
0x02,
0x00,
0xF0,
0xFC,
0x00,
0x01,
0x03,
0x00,
0x05,
0x00,
0x01,
0x94,
0x0B,
]
- delay: 100ms
inject_rx: [0x02, 0x03, 0x00, 0x07, 0x00, 0x01, 0x35, 0xF8] # Read holding register 7 on device 2, with no response
- delay: 100ms
# Read holding register 7 on device 2, with no response
# Read holding register A on device 1 (read_after_peer_timeout)
inject_rx:
[
0x02,
0x03,
0x00,
0x07,
0x00,
0x01,
0x35,
0xF8,
0x01,
0x03,
0x00,
0x0A,
0x00,
0x01,
0xA4,
0x08,
]
modbus:
uart_id: virtual_uart_dev
role: server
modbus_server:
- address: 1
registers:
- address: 0x03
value_type: U_WORD
read_lambda: |-
id(basic_read).publish_state(1);
return 1;
- address: 0x05
value_type: U_WORD
read_lambda: |-
id(read_after_peer_response).publish_state(1);
return 1;
- address: 0x0A
value_type: U_WORD
read_lambda: |-
id(read_after_peer_timeout).publish_state(1);
return 1;
sensor:
- platform: template
name: "basic_read"
id: basic_read
- platform: template
name: "read_after_peer_response"
id: read_after_peer_response
- platform: template
name: "read_after_peer_timeout"
id: read_after_peer_timeout
button:
- platform: template
name: "Start Scenario"
id: start_scenario_btn
on_press:
- lambda: "id(virtual_uart_dev).start_scenario();"
@@ -1,5 +1,5 @@
esphome:
name: uart-mock-modbus-mesh
name: uart-mock-modbus-server-contro
host:
api:
@@ -17,14 +17,13 @@ uart:
baud_rate: 115200
port: /dev/null
# Shared 3-bus mesh (see the shared_yaml markers): addr 1 = typed read-only
# registers, addr 5 = the read/write 0x17 target, addr 2/3 on the second
# server hub. auto_start everywhere: the controller polls at boot, so the
# forwarding must already be live or early requests generate warnings.
# Every test presses Start Scenario, so all merged actions fire in every test.
uart_mock:
- id: virtual_uart_server
baud_rate: 9600
# auto_start must be true for loopback fixtures: the modbus controller
# polls on its update_interval immediately at boot, so the uart_mock
# forwarding must already be active or early requests are lost and
# generate modbus warnings.
auto_start: true
debug:
on_tx:
@@ -32,68 +31,35 @@ uart_mock:
- uart_mock.inject_rx:
id: virtual_uart_controller
data: !lambda return data;
- uart_mock.inject_rx:
id: virtual_uart_server_2
data: !lambda return data;
- id: virtual_uart_server_2
baud_rate: 9600
auto_start: true
debug:
on_tx:
- then:
- uart_mock.inject_rx:
id: virtual_uart_server
data: !lambda return data;
- uart_mock.inject_rx:
id: virtual_uart_controller
data: !lambda return data;
- id: virtual_uart_controller
baud_rate: 9600
auto_start: true
auto_start: true # See comment on virtual_uart_server above
debug:
on_tx:
- then:
- uart_mock.inject_rx:
id: virtual_uart_server
data: !lambda return data;
- uart_mock.inject_rx:
id: virtual_uart_server_2
data: !lambda return data;
globals:
- id: stored_1
type: uint16_t
initial_value: "0"
modbus:
- uart_id: virtual_uart_server
id: virtual_modbus_server
role: server
- uart_id: virtual_uart_server_2
id: virtual_modbus_server_2
role: server
- uart_id: virtual_uart_controller
id: virtual_modbus_client
id: virtual_modbus_controller
role: client
turnaround_time: 10ms
modbus_controller:
- address: 1
modbus_id: virtual_modbus_client
modbus_id: virtual_modbus_controller
id: modbus_controller_1
update_interval: 1s
- address: 2
modbus_id: virtual_modbus_client
id: modbus_controller_2
update_interval: 1s
- address: 3
modbus_id: virtual_modbus_client
id: modbus_controller_3
update_interval: 1s
modbus_server:
- address: 1
modbus_id: virtual_modbus_server
id: modbus_server_1
registers:
- address: 0x01
value_type: U_WORD
@@ -137,34 +103,6 @@ modbus_server:
- address: 0x28
value_type: FP32_R
read_lambda: return 3.14;
- address: 5
modbus_id: virtual_modbus_server
registers:
# Writable + readable register: srv_write_1 plus the client's read-back
# confirm the write half of the 0x17 ran before the read half (Modbus 6.17).
- address: 0x01
value_type: U_WORD
read_lambda: return id(stored_1);
write_lambda: |-
id(stored_1) = x;
id(srv_write_1).publish_state(x);
return true;
# Read-only register, returned together with 0x01 by the 2-register read half.
- address: 0x02
value_type: U_WORD
read_lambda: return 0x00AA;
- address: 2
modbus_id: virtual_modbus_server_2
registers:
- address: 0x01
value_type: U_WORD
read_lambda: return 919;
- address: 3
modbus_id: virtual_modbus_server_2
registers:
- address: 0x01
value_type: U_WORD
read_lambda: return 929;
sensor:
- platform: modbus_controller
@@ -257,46 +195,9 @@ sensor:
address: 0x28
register_type: holding
value_type: FP32_R
- platform: modbus_controller
modbus_controller_id: modbus_controller_2
name: "multi_reg_a"
address: 0x01
register_type: holding
value_type: U_WORD
- platform: modbus_controller
modbus_controller_id: modbus_controller_3
name: "multi_reg_b"
address: 0x01
register_type: holding
value_type: U_WORD
# client_read_write observations, server- and client-side.
- platform: template
name: "srv_write_1"
id: srv_write_1
- platform: template
name: "client_read_0"
id: client_read_0
- platform: template
name: "client_read_1"
id: client_read_1
button:
- platform: template
name: "Start Scenario"
id: start_scenario_btn
on_press:
# FC 0x17: write reg 0x0001 = 0x1234, then read regs 0x0001..0x0002 back in the same transaction.
- modbus_client.read_write_multiple_registers:
address: 5
read_address: 0x0001
read_count: 2
write_address: 0x0001
values: [0x1234]
on_response:
then:
- lambda: |-
// values is the read-back block: reg 0x0001 (must be the just-written 0x1234) and reg 0x0002.
if (values.size() >= 2) {
id(client_read_0).publish_state(values[0]);
id(client_read_1).publish_state(values[1]);
}
# This test does not have anything to start (mock is autostart)
@@ -0,0 +1,116 @@
esphome:
name: uart-mock-modbus-server-mult
host:
api:
logger:
level: VERBOSE
external_components:
- source:
type: local
path: EXTERNAL_COMPONENT_PATH
# Dummy uart entry to satisfy modbus's DEPENDENCIES = ["uart"]
# The actual UART bus used is the uart_mock component below
uart:
baud_rate: 115200
port: /dev/null
uart_mock:
- id: virtual_uart_server
baud_rate: 9600
# auto_start must be true for loopback fixtures: the modbus controller
# polls on its update_interval immediately at boot, so the uart_mock
# forwarding must already be active or early requests are lost and
# generate modbus warnings.
auto_start: true
debug:
on_tx:
- then:
- uart_mock.inject_rx:
id: virtual_uart_controller
data: !lambda return data;
- uart_mock.inject_rx:
id: virtual_uart_server_2
data: !lambda return data;
- id: virtual_uart_server_2
baud_rate: 9600
auto_start: true # See comment on virtual_uart_server above
debug:
on_tx:
- then:
- uart_mock.inject_rx:
id: virtual_uart_server
data: !lambda return data;
- uart_mock.inject_rx:
id: virtual_uart_controller
data: !lambda return data;
- id: virtual_uart_controller
baud_rate: 9600
auto_start: true # See comment on virtual_uart_server above
debug:
on_tx:
- then:
- uart_mock.inject_rx:
id: virtual_uart_server
data: !lambda return data;
- uart_mock.inject_rx:
id: virtual_uart_server_2
data: !lambda return data;
modbus:
- uart_id: virtual_uart_server
id: virtual_modbus_server
role: server
- uart_id: virtual_uart_server_2
id: virtual_modbus_server_2
role: server
- uart_id: virtual_uart_controller
id: virtual_modbus_client
role: client
turnaround_time: 10ms
modbus_controller:
- address: 1
modbus_id: virtual_modbus_client
update_interval: 1s
id: modbus_controller_1
- address: 2
modbus_id: virtual_modbus_client
update_interval: 1s
id: modbus_controller_2
modbus_server:
- address: 1
modbus_id: virtual_modbus_server
registers:
- address: 0x01
value_type: U_WORD
read_lambda: return 919;
- address: 2
modbus_id: virtual_modbus_server_2
registers:
- address: 0x01
value_type: U_WORD
read_lambda: return 929;
sensor:
- platform: modbus_controller
modbus_controller_id: modbus_controller_1
name: "reg_u_word"
address: 0x01
register_type: holding
value_type: U_WORD
- platform: modbus_controller
modbus_controller_id: modbus_controller_2
name: "reg_u_word_2"
address: 0x01
register_type: holding
value_type: U_WORD
button:
- platform: template
name: "Start Scenario"
id: start_scenario_btn
# This test does not have anything to start (mock is autostart)
@@ -1,5 +1,5 @@
esphome:
name: uart-mock-modbus-srv-injected
name: uart-mock-modbus-srv-rw
host:
api:
@@ -17,8 +17,6 @@ uart:
baud_rate: 115200
port: /dev/null
# Shared server-role fixture (see the shared_yaml markers in the test file);
# the injections concatenate and each test waits only on its own sensors.
uart_mock:
- id: virtual_uart_dev
baud_rate: 9600
@@ -27,31 +25,18 @@ uart_mock:
auto_start: false
debug:
injections:
- delay: 100ms
inject_rx: [0x01, 0x03, 0x00, 0x03, 0x00, 0x01, 0x74, 0x0A] # Read holding register 3 on device 1 (basic_read)
- delay: 100ms
# Read holding register 7 on device 2, its reply, then read holding
# register 5 on device 1 (read_after_peer_response)
inject_rx: [0x02, 0x03, 0x00, 0x07, 0x00, 0x01, 0x35, 0xF8,
0x02, 0x03, 0x02, 0x00, 0xF0, 0xFC,
0x00, 0x01, 0x03, 0x00, 0x05, 0x00, 0x01, 0x94, 0x0B]
- delay: 100ms
inject_rx: [0x02, 0x03, 0x00, 0x07, 0x00, 0x01, 0x35, 0xF8] # Read holding register 7 on device 2, with no response
- delay: 100ms
# Read holding register 7 on device 2 with no response, then read
# holding register A on device 1 (read_after_peer_timeout)
inject_rx: [0x02, 0x03, 0x00, 0x07, 0x00, 0x01, 0x35, 0xF8,
0x01, 0x03, 0x00, 0x0A, 0x00, 0x01, 0xA4, 0x08]
# FC 0x17 on device 1: write reg 0x0001 = 0x1234 then read 0x0001..0x0002;
# per Modbus 6.17 the write runs first, so 0x0001 must read back 0x1234.
# FC 0x17 Read/Write Multiple Registers on device 1:
# write reg 0x0001 = 0x1234 (qty 1), then read regs 0x0001..0x0002 (qty 2).
# Per Modbus 6.17 the write is performed before the read, so reg 0x0001 must
# read back the just-written 0x1234 in the same request.
- delay: 100ms
inject_rx:
[0x01, 0x17, 0x00, 0x01, 0x00, 0x02, 0x00, 0x01, 0x00, 0x01, 0x02, 0x12, 0x34, 0x49, 0xD8]
# FC 0x17: write reg 0x0006 = 0x5678 (qty 1), then read reg 0x0006 (qty 1) -
# FC 0x17: write reg 0x0003 = 0x5678 (qty 1), then read reg 0x0003 (qty 1) -
# a write and read targeting a different register block.
- delay: 100ms
inject_rx:
[0x01, 0x17, 0x00, 0x06, 0x00, 0x01, 0x00, 0x06, 0x00, 0x01, 0x02, 0x56, 0x78, 0x8B, 0x55]
[0x01, 0x17, 0x00, 0x03, 0x00, 0x01, 0x00, 0x03, 0x00, 0x01, 0x02, 0x56, 0x78, 0x9B, 0x10]
globals:
- id: stored_1
@@ -85,18 +70,8 @@ modbus_server:
read_lambda: |-
id(rw_read_2).publish_state(0x00AA);
return 0x00AA;
# Second writable + readable register, targeted by the second request.
- address: 0x03
value_type: U_WORD
read_lambda: |-
id(basic_read).publish_state(1);
return 1;
- address: 0x05
value_type: U_WORD
read_lambda: |-
id(read_after_peer_response).publish_state(1);
return 1;
# Second writable + readable register, targeted by the second FC 0x17 request.
- address: 0x06
value_type: U_WORD
read_lambda: |-
id(rw_read_3).publish_state(id(stored_3));
@@ -105,22 +80,8 @@ modbus_server:
id(stored_3) = x;
id(rw_write_3).publish_state(x);
return true;
- address: 0x0A
value_type: U_WORD
read_lambda: |-
id(read_after_peer_timeout).publish_state(1);
return 1;
sensor:
- platform: template
name: "basic_read"
id: basic_read
- platform: template
name: "read_after_peer_response"
id: read_after_peer_response
- platform: template
name: "read_after_peer_timeout"
id: read_after_peer_timeout
- platform: template
name: "rw_write_1"
id: rw_write_1
+3 -11
View File
@@ -1,7 +1,7 @@
"""Helpers for manipulating the host platform's preferences file.
ESPHome's host platform stores preferences in
``$ESPHOME_PREFDIR/<app_name>.prefs`` using a simple binary layout that
``~/.esphome/prefs/<app_name>.prefs`` using a simple binary layout that
mirrors ``HostPreferences::sync()``:
``[uint32_t key][uint8_t len][uint8_t data[len]]`` per entry.
@@ -11,21 +11,13 @@ boot (e.g. forcing safe mode) or to clear stale state between runs.
from __future__ import annotations
import os
from pathlib import Path
import struct
def host_prefs_path(device_name: str) -> Path:
"""Return the on-disk prefs file path for a host-platform device.
Requires ESPHOME_PREFDIR, which the autouse isolated_preferences fixture
sets; refusing the ~/.esphome/prefs fallback keeps tests off real user
data if the fixture is ever bypassed."""
prefdir = os.environ.get("ESPHOME_PREFDIR")
if not prefdir:
raise RuntimeError("ESPHOME_PREFDIR is not set; refusing the real prefs dir")
return Path(prefdir) / f"{device_name}.prefs"
"""Return the on-disk prefs file path for a host-platform device."""
return Path.home() / ".esphome" / "prefs" / f"{device_name}.prefs"
def clear_host_prefs(device_name: str) -> None:
+140 -141
View File
@@ -1,143 +1,142 @@
{
"tests/integration/test_action_concurrent_reentry.py": 57.91,
"tests/integration/test_addressable_light_transition.py": 21.25,
"tests/integration/test_alarm_control_panel_state_transitions.py": 70.71,
"tests/integration/test_api_action_metadata.py": 66.6,
"tests/integration/test_api_action_responses.py": 36.1,
"tests/integration/test_api_action_timeout.py": 68.86,
"tests/integration/test_api_conditional_memory.py": 15.48,
"tests/integration/test_api_custom_services.py": 18.77,
"tests/integration/test_api_get_time_response_timezone.py": 21.08,
"tests/integration/test_api_homeassistant.py": 65.59,
"tests/integration/test_api_homeassistant_action_no_subscriber.py": 18.44,
"tests/integration/test_api_homeassistant_binary_sensor_initial_state.py": 15.05,
"tests/integration/test_api_list_entities_backpressure.py": 13.88,
"tests/integration/test_api_message_size_batching.py": 29.98,
"tests/integration/test_api_reboot_timeout.py": 16.05,
"tests/integration/test_api_string_lambda.py": 15.31,
"tests/integration/test_api_vv_logging.py": 19.28,
"tests/integration/test_api_zero_psk_provisioning.py": 31.5,
"tests/integration/test_areas_and_devices.py": 24.95,
"tests/integration/test_automation_wait_actions.py": 20.92,
"tests/integration/test_automations.py": 35.19,
"tests/integration/test_batch_delay_zero_rapid_transitions.py": 17.99,
"tests/integration/test_binary_sensor_autorepeat_filter.py": 20.39,
"tests/integration/test_binary_sensor_invalidate_state.py": 18.41,
"tests/integration/test_blocking_warning_log_time_not_charged_to_next_operation.py": 24.69,
"tests/integration/test_build_info.py": 18.7,
"tests/integration/test_camera_mock.py": 16.23,
"tests/integration/test_climate_control_action.py": 21.14,
"tests/integration/test_climate_custom_modes.py": 20.74,
"tests/integration/test_continuation_actions.py": 16.81,
"tests/integration/test_cover_control_action.py": 20.34,
"tests/integration/test_crc8_helper.py": 9.36,
"tests/integration/test_device_id_in_state.py": 44.67,
"tests/integration/test_duplicate_entities.py": 23.58,
"tests/integration/test_entity_icon.py": 34.35,
"tests/integration/test_fan_turn_on_action.py": 24.23,
"tests/integration/test_fnv1_hash_object_id.py": 16.21,
"tests/integration/test_fnv1a_hash.py": 13.38,
"tests/integration/test_gpio_expander_cache.py": 13.06,
"tests/integration/test_host_logger_thread_safety.py": 23.66,
"tests/integration/test_host_mode_basic.py": 8.01,
"tests/integration/test_host_mode_batch_delay.py": 21.0,
"tests/integration/test_host_mode_climate_basic_state.py": 22.14,
"tests/integration/test_host_mode_climate_control.py": 19.39,
"tests/integration/test_host_mode_empty_string_options.py": 21.76,
"tests/integration/test_host_mode_entity_fields.py": 29.61,
"tests/integration/test_host_mode_fan_preset.py": 20.01,
"tests/integration/test_host_mode_many_entities.py": 39.08,
"tests/integration/test_host_mode_many_entities_multiple_connections.py": 23.92,
"tests/integration/test_host_mode_noise_encryption.py": 42.42,
"tests/integration/test_host_mode_reconnect.py": 3.41,
"tests/integration/test_host_mode_sensor.py": 22.96,
"tests/integration/test_host_ota.py": 29.5,
"tests/integration/test_host_preferences.py": 16.06,
"tests/integration/test_host_preferences_suspend_resume.py": 18.71,
"tests/integration/test_improv_serial_uart.py": 20.22,
"tests/integration/test_large_message_batching.py": 26.56,
"tests/integration/test_legacy_area.py": 22.72,
"tests/integration/test_legacy_climate_compat.py": 14.13,
"tests/integration/test_legacy_fan_compat.py": 14.33,
"tests/integration/test_light_automations.py": 18.81,
"tests/integration/test_light_binary_effect_off_phase.py": 8.38,
"tests/integration/test_light_calls.py": 21.88,
"tests/integration/test_light_constant_brightness.py": 59.45,
"tests/integration/test_light_control_action.py": 31.91,
"tests/integration/test_light_dim_relative_action.py": 14.43,
"tests/integration/test_light_effect_zero_brightness.py": 25.05,
"tests/integration/test_light_initial_state.py": 18.97,
"tests/integration/test_light_toggle_action.py": 17.44,
"tests/integration/test_lock_automations.py": 18.9,
"tests/integration/test_logger_buffered_recursion_guard.py": 18.2,
"tests/integration/test_loop_disable_enable.py": 63.35,
"tests/integration/test_loop_interval_decoupling.py": 17.7,
"tests/integration/test_loop_interval_default_not_pulled_forward.py": 21.56,
"tests/integration/test_micros_to_millis.py": 15.89,
"tests/integration/test_multi_click_trigger.py": 17.23,
"tests/integration/test_multi_device_preferences.py": 19.4,
"tests/integration/test_noise_encryption_key_protection.py": 72.59,
"tests/integration/test_object_id_api_verification.py": 19.22,
"tests/integration/test_object_id_friendly_name_no_mac_suffix.py": 16.77,
"tests/integration/test_object_id_no_friendly_name.py": 45.8,
"tests/integration/test_online_image_auto_detects_image_bmp_mime.py": 86.73,
"tests/integration/test_online_image_auto_detects_redirected_image_bmp_mime.py": 40.4,
"tests/integration/test_online_image_bmp.py": 37.24,
"tests/integration/test_oversized_payloads.py": 55.75,
"tests/integration/test_preference_key_stability.py": 25.49,
"tests/integration/test_runtime_stats.py": 29.81,
"tests/integration/test_safe_mode_loop_runs.py": 6.26,
"tests/integration/test_scheduler_blocking_warning.py": 37.98,
"tests/integration/test_scheduler_bulk_cleanup.py": 18.67,
"tests/integration/test_scheduler_defer_cancel.py": 18.46,
"tests/integration/test_scheduler_defer_cancel_regular.py": 16.34,
"tests/integration/test_scheduler_defer_fifo_simple.py": 18.26,
"tests/integration/test_scheduler_defer_stress.py": 17.74,
"tests/integration/test_scheduler_heap_stress.py": 3.89,
"tests/integration/test_scheduler_internal_id_no_collision.py": 20.01,
"tests/integration/test_scheduler_interval_reschedule.py": 16.29,
"tests/integration/test_scheduler_interval_zero_coerced.py": 16.09,
"tests/integration/test_scheduler_null_name.py": 14.69,
"tests/integration/test_scheduler_numeric_id_test.py": 17.08,
"tests/integration/test_scheduler_pool.py": 19.88,
"tests/integration/test_scheduler_rapid_cancellation.py": 4.42,
"tests/integration/test_scheduler_recursive_timeout.py": 4.3,
"tests/integration/test_scheduler_removed_item_race.py": 15.49,
"tests/integration/test_scheduler_self_keyed.py": 25.77,
"tests/integration/test_scheduler_simultaneous_callbacks.py": 14.84,
"tests/integration/test_scheduler_string_test.py": 15.42,
"tests/integration/test_script_array_params.py": 12.73,
"tests/integration/test_script_delay_params.py": 12.69,
"tests/integration/test_script_queued.py": 20.38,
"tests/integration/test_script_queued_idle_loop.py": 25.06,
"tests/integration/test_script_wait_on_boot.py": 15.67,
"tests/integration/test_select_stringref_trigger.py": 19.48,
"tests/integration/test_sensor_filters_delta.py": 27.62,
"tests/integration/test_sensor_filters_ring_buffer.py": 20.27,
"tests/integration/test_sensor_filters_sliding_window.py": 56.28,
"tests/integration/test_sensor_filters_value_list.py": 20.6,
"tests/integration/test_sensor_timeout_filter.py": 22.21,
"tests/integration/test_socket_wake_gate_tcp.py": 16.37,
"tests/integration/test_status_flags.py": 29.68,
"tests/integration/test_strftime_to.py": 17.42,
"tests/integration/test_syslog.py": 18.39,
"tests/integration/test_template_alarm_control_panel_many_sensors.py": 25.61,
"tests/integration/test_template_text_save.py": 19.16,
"tests/integration/test_text_command.py": 16.43,
"tests/integration/test_text_sensor_raw_state.py": 17.19,
"tests/integration/test_uart_mock_ld2410.py": 37.0,
"tests/integration/test_uart_mock_ld2412.py": 40.82,
"tests/integration/test_uart_mock_ld2420.py": 32.7,
"tests/integration/test_uart_mock_ld2450.py": 32.84,
"tests/integration/test_uart_mock_modbus.py": 548.87,
"tests/integration/test_udp.py": 16.67,
"tests/integration/test_use_address_runtime.py": 27.26,
"tests/integration/test_valve_control_action.py": 24.58,
"tests/integration/test_varint_five_byte_device_id.py": 22.5,
"tests/integration/test_wait_until_mid_loop_timing.py": 22.05,
"tests/integration/test_wait_until_on_boot.py": 10.37,
"tests/integration/test_wait_until_ordering.py": 18.23,
"tests/integration/test_wait_until_reentrant_restart.py": 19.35,
"tests/integration/test_wake_loop_forces_phase_b.py": 17.83,
"tests/integration/test_water_heater_template.py": 25.7
"tests/integration/test_action_concurrent_reentry.py": 45.23,
"tests/integration/test_addressable_light_transition.py": 74.47,
"tests/integration/test_alarm_control_panel_state_transitions.py": 74.1,
"tests/integration/test_api_action_metadata.py": 62.1,
"tests/integration/test_api_action_responses.py": 71.08,
"tests/integration/test_api_action_timeout.py": 21.64,
"tests/integration/test_api_conditional_memory.py": 13.72,
"tests/integration/test_api_custom_services.py": 24.16,
"tests/integration/test_api_get_time_response_timezone.py": 23.48,
"tests/integration/test_api_homeassistant.py": 37.87,
"tests/integration/test_api_homeassistant_action_no_subscriber.py": 14.38,
"tests/integration/test_api_list_entities_backpressure.py": 26.85,
"tests/integration/test_api_message_size_batching.py": 33.36,
"tests/integration/test_api_reboot_timeout.py": 13.63,
"tests/integration/test_api_string_lambda.py": 25.04,
"tests/integration/test_api_vv_logging.py": 16.6,
"tests/integration/test_api_zero_psk_provisioning.py": 43.14,
"tests/integration/test_areas_and_devices.py": 25.98,
"tests/integration/test_automation_wait_actions.py": 21.91,
"tests/integration/test_automations.py": 42.43,
"tests/integration/test_batch_delay_zero_rapid_transitions.py": 16.65,
"tests/integration/test_binary_sensor_autorepeat_filter.py": 28.67,
"tests/integration/test_binary_sensor_invalidate_state.py": 23.69,
"tests/integration/test_blocking_warning_log_time_not_charged_to_next_operation.py": 22.99,
"tests/integration/test_build_info.py": 24.96,
"tests/integration/test_camera_mock.py": 14.47,
"tests/integration/test_climate_control_action.py": 31.07,
"tests/integration/test_climate_custom_modes.py": 28.59,
"tests/integration/test_continuation_actions.py": 14.96,
"tests/integration/test_cover_control_action.py": 26.14,
"tests/integration/test_crc8_helper.py": 10.92,
"tests/integration/test_device_id_in_state.py": 64.97,
"tests/integration/test_duplicate_entities.py": 30.81,
"tests/integration/test_entity_icon.py": 32.85,
"tests/integration/test_fan_turn_on_action.py": 24.91,
"tests/integration/test_fnv1_hash_object_id.py": 12.54,
"tests/integration/test_fnv1a_hash.py": 21.8,
"tests/integration/test_gpio_expander_cache.py": 5.2,
"tests/integration/test_host_logger_thread_safety.py": 21.7,
"tests/integration/test_host_mode_basic.py": 13.62,
"tests/integration/test_host_mode_batch_delay.py": 14.56,
"tests/integration/test_host_mode_climate_basic_state.py": 30.95,
"tests/integration/test_host_mode_climate_control.py": 29.06,
"tests/integration/test_host_mode_empty_string_options.py": 27.22,
"tests/integration/test_host_mode_entity_fields.py": 30.95,
"tests/integration/test_host_mode_fan_preset.py": 14.44,
"tests/integration/test_host_mode_many_entities.py": 54.13,
"tests/integration/test_host_mode_many_entities_multiple_connections.py": 32.17,
"tests/integration/test_host_mode_noise_encryption.py": 42.77,
"tests/integration/test_host_mode_reconnect.py": 4.06,
"tests/integration/test_host_mode_sensor.py": 13.47,
"tests/integration/test_host_ota.py": 21.4,
"tests/integration/test_host_preferences.py": 25.43,
"tests/integration/test_host_preferences_suspend_resume.py": 19.2,
"tests/integration/test_improv_serial_uart.py": 31.52,
"tests/integration/test_large_message_batching.py": 15.64,
"tests/integration/test_legacy_area.py": 22.63,
"tests/integration/test_legacy_climate_compat.py": 26.13,
"tests/integration/test_legacy_fan_compat.py": 24.05,
"tests/integration/test_light_automations.py": 30.86,
"tests/integration/test_light_binary_effect_off_phase.py": 23.19,
"tests/integration/test_light_calls.py": 32.35,
"tests/integration/test_light_constant_brightness.py": 29.89,
"tests/integration/test_light_control_action.py": 29.06,
"tests/integration/test_light_dim_relative_action.py": 29.61,
"tests/integration/test_light_effect_zero_brightness.py": 18.68,
"tests/integration/test_light_initial_state.py": 24.49,
"tests/integration/test_light_toggle_action.py": 26.46,
"tests/integration/test_lock_automations.py": 23.28,
"tests/integration/test_logger_buffered_recursion_guard.py": 24.29,
"tests/integration/test_loop_disable_enable.py": 45.28,
"tests/integration/test_loop_interval_decoupling.py": 28.35,
"tests/integration/test_loop_interval_default_not_pulled_forward.py": 21.97,
"tests/integration/test_micros_to_millis.py": 20.79,
"tests/integration/test_multi_click_trigger.py": 26.2,
"tests/integration/test_multi_device_preferences.py": 16.87,
"tests/integration/test_noise_encryption_key_protection.py": 77.05,
"tests/integration/test_object_id_api_verification.py": 73.51,
"tests/integration/test_object_id_friendly_name_no_mac_suffix.py": 62.33,
"tests/integration/test_object_id_no_friendly_name.py": 43.47,
"tests/integration/test_online_image_auto_detects_image_bmp_mime.py": 32.21,
"tests/integration/test_online_image_auto_detects_redirected_image_bmp_mime.py": 56.86,
"tests/integration/test_online_image_bmp.py": 50.9,
"tests/integration/test_oversized_payloads.py": 53.2,
"tests/integration/test_preference_key_stability.py": 26.09,
"tests/integration/test_runtime_stats.py": 18.34,
"tests/integration/test_safe_mode_loop_runs.py": 10.07,
"tests/integration/test_scheduler_blocking_warning.py": 40.91,
"tests/integration/test_scheduler_bulk_cleanup.py": 23.14,
"tests/integration/test_scheduler_defer_cancel.py": 24.54,
"tests/integration/test_scheduler_defer_cancel_regular.py": 13.48,
"tests/integration/test_scheduler_defer_fifo_simple.py": 26.86,
"tests/integration/test_scheduler_defer_stress.py": 27.23,
"tests/integration/test_scheduler_heap_stress.py": 24.02,
"tests/integration/test_scheduler_internal_id_no_collision.py": 24.57,
"tests/integration/test_scheduler_interval_reschedule.py": 13.12,
"tests/integration/test_scheduler_interval_zero_coerced.py": 22.91,
"tests/integration/test_scheduler_null_name.py": 23.46,
"tests/integration/test_scheduler_numeric_id_test.py": 24.54,
"tests/integration/test_scheduler_pool.py": 25.0,
"tests/integration/test_scheduler_rapid_cancellation.py": 14.68,
"tests/integration/test_scheduler_recursive_timeout.py": 25.35,
"tests/integration/test_scheduler_removed_item_race.py": 26.19,
"tests/integration/test_scheduler_self_keyed.py": 23.43,
"tests/integration/test_scheduler_simultaneous_callbacks.py": 22.16,
"tests/integration/test_scheduler_string_test.py": 15.22,
"tests/integration/test_script_array_params.py": 14.67,
"tests/integration/test_script_delay_params.py": 15.65,
"tests/integration/test_script_queued.py": 24.93,
"tests/integration/test_script_queued_idle_loop.py": 5.04,
"tests/integration/test_script_wait_on_boot.py": 13.08,
"tests/integration/test_select_stringref_trigger.py": 29.6,
"tests/integration/test_sensor_filters_delta.py": 28.01,
"tests/integration/test_sensor_filters_ring_buffer.py": 25.04,
"tests/integration/test_sensor_filters_sliding_window.py": 71.5,
"tests/integration/test_sensor_filters_value_list.py": 16.94,
"tests/integration/test_sensor_timeout_filter.py": 29.48,
"tests/integration/test_socket_wake_gate_tcp.py": 20.36,
"tests/integration/test_status_flags.py": 37.42,
"tests/integration/test_strftime_to.py": 22.61,
"tests/integration/test_syslog.py": 16.34,
"tests/integration/test_template_alarm_control_panel_many_sensors.py": 29.81,
"tests/integration/test_template_text_save.py": 25.43,
"tests/integration/test_text_command.py": 23.34,
"tests/integration/test_text_sensor_raw_state.py": 69.57,
"tests/integration/test_uart_mock_ld2410.py": 37.95,
"tests/integration/test_uart_mock_ld2412.py": 93.22,
"tests/integration/test_uart_mock_ld2420.py": 43.24,
"tests/integration/test_uart_mock_ld2450.py": 31.75,
"tests/integration/test_uart_mock_modbus.py": 667.4,
"tests/integration/test_udp.py": 9.38,
"tests/integration/test_use_address_runtime.py": 37.05,
"tests/integration/test_valve_control_action.py": 24.47,
"tests/integration/test_varint_five_byte_device_id.py": 25.03,
"tests/integration/test_wait_until_mid_loop_timing.py": 23.73,
"tests/integration/test_wait_until_on_boot.py": 9.16,
"tests/integration/test_wait_until_ordering.py": 13.3,
"tests/integration/test_wait_until_reentrant_restart.py": 25.23,
"tests/integration/test_wake_loop_forces_phase_b.py": 23.34,
"tests/integration/test_water_heater_template.py": 17.67
}
@@ -24,6 +24,12 @@ NEW_KEY = base64.b64encode(b"n" * 32)
KEY_ACTIVATION_DELAY = 0.5
@pytest.fixture(autouse=True)
def isolated_preferences(monkeypatch: pytest.MonkeyPatch, tmp_path) -> None:
"""Keep host preferences per-test so every run starts unprovisioned."""
monkeypatch.setenv("ESPHOME_PREFDIR", str(tmp_path / "prefs"))
@pytest.mark.asyncio
async def test_api_zero_psk_provisioning(
yaml_config: str,
-57
View File
@@ -10,7 +10,6 @@ from __future__ import annotations
import asyncio
from collections.abc import Generator
from contextlib import contextmanager
import functools
import socket
import pytest
@@ -112,62 +111,6 @@ async def test_host_ota_self_update(
assert proc.pid == pid_before
@pytest.mark.asyncio
async def test_host_ota_encrypted(
yaml_config: str,
write_yaml_config: ConfigWriter,
compile_esphome: CompileFunction,
reserved_tcp_port: tuple[int, socket.socket],
) -> None:
"""Encrypted self-OTA succeeds; a plaintext upload to the same device fails."""
pytest.importorskip("aioesphomeapi.noise")
noise_psk = "AAECAwQFBgcICQoLDA0ODxAREhMUFRYXGBkaGxwdHh8="
api_port, api_socket = reserved_tcp_port
with _reserve_port() as (ota_port, ota_socket):
yaml_config = yaml_config.replace("__OTA_PORT__", str(ota_port))
config_path = await write_yaml_config(yaml_config)
binary_path = await compile_esphome(config_path)
api_socket.close()
ota_socket.close()
loop = asyncio.get_running_loop()
rebooted = loop.create_future()
def on_log(line: str) -> None:
if not rebooted.done() and "Rebooting safely" in line:
rebooted.set_result(True)
async with run_binary(binary_path, line_callback=on_log) as (proc, _lines):
await _wait_for_port(LOCALHOST, api_port, PORT_WAIT_TIMEOUT)
pid_before = proc.pid
# A plaintext upload must be refused with the device unharmed
rc, _ = await loop.run_in_executor(
None, espota2.run_ota, LOCALHOST, ota_port, None, binary_path
)
assert rc == 1, "plaintext upload to an encrypted device must fail"
await asyncio.sleep(0.5)
assert proc.returncode is None, "process died on rejected plaintext OTA"
# The encrypted upload goes through and the device re-execs
rc, _ = await loop.run_in_executor(
None,
functools.partial(
espota2.run_ota,
LOCALHOST,
ota_port,
None,
binary_path,
noise_psk=noise_psk,
),
)
assert rc == 0, "encrypted OTA reported failure"
await asyncio.wait_for(rebooted, timeout=10.0)
await _wait_for_port(LOCALHOST, api_port, PORT_WAIT_TIMEOUT)
assert proc.returncode is None, "process exited instead of execing"
assert proc.pid == pid_before
@pytest.mark.asyncio
async def test_host_ota_rejects_garbage(
yaml_config: str,
@@ -41,6 +41,15 @@ async def _poll_until_exists(path: Path) -> None:
await asyncio.sleep(0.05)
@pytest.fixture(autouse=True)
def isolated_preferences(monkeypatch: pytest.MonkeyPatch, tmp_path) -> Path:
"""Keep host preferences per-test so this test never touches the real
~/.esphome/prefs and never races other tests over ESPHOME_PREFDIR."""
prefdir = tmp_path / "prefs"
monkeypatch.setenv("ESPHOME_PREFDIR", str(prefdir))
return prefdir / f"{DEVICE_NAME}.prefs"
@pytest.mark.asyncio
async def test_host_preferences_suspend_resume(
yaml_config: str,
@@ -49,7 +58,7 @@ async def test_host_preferences_suspend_resume(
isolated_preferences: Path,
) -> None:
"""Test that a running syncer flushes, a suspended one doesn't, and resume restores flushing."""
pref_file = isolated_preferences / f"{DEVICE_NAME}.prefs"
pref_file = isolated_preferences
loop = asyncio.get_running_loop()
saved_in_memory = loop.create_future()
@@ -11,6 +11,14 @@ from .state_utils import InitialStateHelper, require_entity
from .types import APIClientConnectedFactory, RunCompiledFunction
@pytest.fixture(autouse=True)
def isolated_preferences(monkeypatch: pytest.MonkeyPatch, tmp_path) -> None:
"""Keep host preferences per-test so RESTORE_AND_ON never loads a stale value left
behind by a previous run (host preferences otherwise persist to ~/.esphome/prefs,
keyed only by device name)."""
monkeypatch.setenv("ESPHOME_PREFDIR", str(tmp_path / "prefs"))
@pytest.mark.asyncio
async def test_light_initial_state(
yaml_config: str,
+7 -16
View File
@@ -173,7 +173,6 @@ async def test_uart_mock_modbus_no_threshold(
_assert_no_modbus_errors(error_log_lines, warning_log_lines)
@pytest.mark.shared_yaml("uart_mock_modbus_server_injected")
@pytest.mark.asyncio
async def test_uart_mock_modbus_server(
yaml_config: str,
@@ -204,7 +203,6 @@ async def test_uart_mock_modbus_server(
_assert_no_modbus_errors(error_log_lines, warning_log_lines)
@pytest.mark.shared_yaml("uart_mock_modbus_server_injected")
@pytest.mark.asyncio
async def test_uart_mock_modbus_server_read_write(
yaml_config: str,
@@ -233,8 +231,8 @@ async def test_uart_mock_modbus_server_read_write(
"rw_write_1": 4660, # 0x1234 written to reg 0x0001
"rw_read_1": 4660, # reg 0x0001 reads back the just-written value
"rw_read_2": 170, # 0x00AA read from reg 0x0002 in the same request
"rw_write_3": 22136, # 0x5678 written to reg 0x0006
"rw_read_3": 22136, # reg 0x0006 reads back the just-written value
"rw_write_3": 22136, # 0x5678 written to reg 0x0003
"rw_read_3": 22136, # reg 0x0003 reads back the just-written value
}
)
@@ -243,8 +241,7 @@ async def test_uart_mock_modbus_server_read_write(
api_client_connected() as client,
):
await tracker.setup_and_start_scenario(client)
# The FC 0x17 injections fire last, behind four earlier 100ms delays
await tracker.await_all(futures, timeout=4.0)
await tracker.await_all(futures)
_assert_no_modbus_errors(error_log_lines, warning_log_lines)
@@ -299,7 +296,6 @@ async def test_uart_mock_modbus_server_read_write_invalid(
)
@pytest.mark.shared_yaml("uart_mock_modbus_mesh")
@pytest.mark.asyncio
async def test_uart_mock_modbus_server_controller(
yaml_config: str,
@@ -489,7 +485,6 @@ async def test_uart_mock_modbus_server_controller_bits(
_assert_no_modbus_errors(error_log_lines, warning_log_lines)
@pytest.mark.shared_yaml("uart_mock_modbus_mesh")
@pytest.mark.asyncio
async def test_uart_mock_modbus_server_controller_multiple(
yaml_config: str,
@@ -500,7 +495,7 @@ async def test_uart_mock_modbus_server_controller_multiple(
line_callback, error_log_lines, warning_log_lines = _make_modbus_line_callback()
expected_values = {"multi_reg_a": 919, "multi_reg_b": 929}
expected_values = {"reg_u_word": 919, "reg_u_word_2": 929}
tracker = SensorTracker(list(expected_values.keys()))
futures = tracker.expect_all(expected_values)
@@ -711,7 +706,6 @@ async def test_uart_mock_modbus_shared_address(
_assert_no_modbus_errors(error_log_lines, warning_log_lines)
@pytest.mark.shared_yaml("uart_mock_modbus_loopback")
@pytest.mark.asyncio
async def test_uart_mock_modbus_custom_pdu(
yaml_config: str,
@@ -938,7 +932,6 @@ async def test_uart_mock_modbus_broadcast_write(
_assert_no_modbus_errors(error_log_lines, warning_log_lines)
@pytest.mark.shared_yaml("uart_mock_modbus_mesh")
@pytest.mark.asyncio
async def test_uart_mock_modbus_client_read_write(
yaml_config: str,
@@ -954,7 +947,9 @@ async def test_uart_mock_modbus_client_read_write(
"""
line_callback, error_log_lines, warning_log_lines = _make_modbus_line_callback()
tracker = SensorTracker(["srv_write_1", "client_read_0", "client_read_1"])
tracker = SensorTracker(
["srv_write_1", "srv_read_1", "client_read_0", "client_read_1"]
)
futures = tracker.expect_all(
{
"srv_write_1": 4660, # server wrote 0x1234 to reg 0x0001
@@ -972,7 +967,6 @@ async def test_uart_mock_modbus_client_read_write(
_assert_no_modbus_errors(error_log_lines, warning_log_lines)
@pytest.mark.shared_yaml("uart_mock_modbus_loopback")
@pytest.mark.asyncio
async def test_uart_mock_modbus_register_offset(
yaml_config: str,
@@ -1028,7 +1022,6 @@ async def test_uart_mock_modbus_register_offset(
)
@pytest.mark.shared_yaml("uart_mock_modbus_loopback")
@pytest.mark.asyncio
async def test_uart_mock_modbus_lambda_write(
yaml_config: str,
@@ -1065,7 +1058,6 @@ async def test_uart_mock_modbus_lambda_write(
await tracker.await_change(wrote_30, "reg_30", timeout=4.0)
@pytest.mark.shared_yaml("uart_mock_modbus_loopback")
@pytest.mark.asyncio
async def test_uart_mock_modbus_lambda_invert(
yaml_config: str,
@@ -1121,7 +1113,6 @@ async def test_uart_mock_modbus_lambda_invert(
)
@pytest.mark.shared_yaml("uart_mock_modbus_loopback")
@pytest.mark.asyncio
async def test_uart_mock_modbus_deprecated_write_buffer(
yaml_config: str,
-28
View File
@@ -2122,34 +2122,6 @@ def test_get_cpp_changed_components_independent_of_cwd(
) == ["time"]
def test_fixture_map_includes_shared_yaml_markers() -> None:
"""Fixtures named only by shared_yaml markers must map to their test file."""
helpers.get_fixture_to_test_files.cache_clear()
mapping = helpers.get_fixture_to_test_files()
for fixture in (
"uart_mock_modbus_loopback",
"uart_mock_modbus_mesh",
"uart_mock_modbus_server_injected",
):
assert mapping[fixture] == frozenset(
{"tests/integration/test_uart_mock_modbus.py"}
)
def test_no_orphan_integration_fixtures() -> None:
"""Every fixture must reach CI test selection; an orphan selects nothing."""
helpers.get_fixture_to_test_files.cache_clear()
mapping = helpers.get_fixture_to_test_files()
fixtures_dir = (Path(__file__).parent.parent / "integration" / "fixtures").resolve()
fixtures = list(fixtures_dir.glob("*.yaml"))
assert fixtures, f"no fixtures found under {fixtures_dir}"
# cache_init is covered via INTEGRATION_TESTS_TRIGGER_FILES instead
orphans = [
f.stem for f in fixtures if f.stem != "cache_init" and f.stem not in mapping
]
assert not orphans, f"fixtures invisible to CI test selection: {orphans}"
def test_lpt_partition_balances_skewed_weights() -> None:
"""Heavy items spread across groups instead of clustering."""
items = [f"i{n}" for n in range(6)]
@@ -0,0 +1,131 @@
"""Tests for the noise-c/libsodium library wiring in the noise component.
On ESP32 (but not the Arduino framework) both libraries build themselves as
native ESP-IDF managed components, so they are declared via add_idf_component()
instead of going through ESPHome's PlatformIO-library converter, on either
toolchain. Elsewhere they still go through that converter via cg.add_library():
on the Arduino framework because arduino-esp32 depends on espressif/libsodium
of its own, and off ESP32 because there are no IDF components at all. This
drives the real to_code() coroutine so every branch of that decision is
exercised end to end, not just mocked.
"""
from __future__ import annotations
import asyncio
import pytest
import esphome.codegen as cg
from esphome.components import esp32, noise
from esphome.const import (
KEY_CORE,
KEY_TARGET_FRAMEWORK,
KEY_TARGET_PLATFORM,
Framework,
Platform,
Toolchain,
)
from esphome.core import CORE
def _setup_core(platform: Platform, framework: Framework, toolchain: Toolchain) -> None:
CORE.reset()
CORE.toolchain = toolchain
CORE.data[KEY_CORE] = {
KEY_TARGET_PLATFORM: str(platform),
KEY_TARGET_FRAMEWORK: str(framework),
}
if platform == Platform.ESP32:
CORE.data[esp32.KEY_ESP32] = {esp32.KEY_VARIANT: "ESP32"}
def _record_calls(
monkeypatch: pytest.MonkeyPatch,
) -> tuple[list[dict], list[tuple]]:
"""Capture both wiring paths so each test can assert one ran and one did not."""
idf_calls: list[dict] = []
lib_calls: list[tuple] = []
monkeypatch.setattr(
esp32, "add_idf_component", lambda **kwargs: idf_calls.append(kwargs)
)
monkeypatch.setattr(
cg,
"add_library",
lambda name, version, repository=None: lib_calls.append((name, version)),
)
return idf_calls, lib_calls
@pytest.mark.parametrize("toolchain", [Toolchain.ESP_IDF, Toolchain.PLATFORMIO])
def test_to_code_esp32_idf_uses_managed_idf_components(
toolchain: Toolchain,
monkeypatch: pytest.MonkeyPatch,
) -> None:
"""On ESP32 + ESP-IDF both libraries are declared as managed IDF components
rather than converted PlatformIO libraries. The choice is deliberately the
same on either toolchain, because wireguard splits on the same condition."""
_setup_core(Platform.ESP32, Framework.ESP_IDF, toolchain)
idf_calls, lib_calls = _record_calls(monkeypatch)
asyncio.run(noise.to_code({}))
assert idf_calls == [
{"name": "esphome/noise-c", "ref": noise.NOISE_C_VERSION},
{"name": "esphome/libsodium", "ref": noise.LIBSODIUM_VERSION},
]
assert lib_calls == []
def test_to_code_esp32_arduino_uses_add_library(
monkeypatch: pytest.MonkeyPatch,
) -> None:
"""On the Arduino framework arduino-esp32 depends on espressif/libsodium of
its own, so declaring esphome/libsodium as a managed component too would
leave the component manager unable to pick between them."""
_setup_core(Platform.ESP32, Framework.ARDUINO, Toolchain.ESP_IDF)
idf_calls, lib_calls = _record_calls(monkeypatch)
asyncio.run(noise.to_code({}))
assert lib_calls == [
("esphome/noise-c", noise.NOISE_C_VERSION),
("esphome/libsodium", noise.LIBSODIUM_VERSION),
]
assert idf_calls == []
def test_to_code_non_esp32_uses_add_library(
monkeypatch: pytest.MonkeyPatch,
) -> None:
"""Off ESP32 entirely (e.g. host) there are no IDF components at all."""
_setup_core(Platform.HOST, Framework.NATIVE, Toolchain.PLATFORMIO)
idf_calls, lib_calls = _record_calls(monkeypatch)
asyncio.run(noise.to_code({}))
assert lib_calls == [
("esphome/noise-c", noise.NOISE_C_VERSION),
("esphome/libsodium", noise.LIBSODIUM_VERSION),
]
assert idf_calls == []
def test_versions_match_the_repo_manifests() -> None:
"""The pins are duplicated in platformio.ini and esphome/idf_component.yml;
a bump that misses one would ship two different libsodium versions."""
from pathlib import Path
import yaml
repo_root = Path(__file__).resolve().parents[4]
manifest = yaml.safe_load(
(repo_root / "esphome" / "idf_component.yml").read_text(encoding="utf-8")
)
deps = manifest["dependencies"]
assert deps["esphome/noise-c"]["version"] == noise.NOISE_C_VERSION
assert deps["esphome/libsodium"]["version"] == noise.LIBSODIUM_VERSION
assert f"esphome/noise-c@{noise.NOISE_C_VERSION}" in (
repo_root / "platformio.ini"
).read_text(encoding="utf-8")
@@ -0,0 +1,107 @@
"""Tests for esp32's _write_idf_component_yml() managed-component wiring.
A library that is already declared as a managed IDF component (via
add_idf_component(), e.g. api's noise-c/libsodium) must not also be converted
from a PlatformIO library, or ESP-IDF sees the same requirement declared by
two components and refuses to build. _write_idf_component_yml() passes the
set of already-managed component names to generate_idf_components() so the
converter excludes them.
"""
from __future__ import annotations
from pathlib import Path
from unittest.mock import MagicMock
import pytest
from esphome.components import esp32
from esphome.const import (
KEY_CORE,
KEY_TARGET_FRAMEWORK,
KEY_TARGET_PLATFORM,
Framework,
Platform,
Toolchain,
)
from esphome.core import CORE
def _setup_core(tmp_path: Path) -> None:
CORE.reset()
CORE.name = "testdevice"
CORE.build_path = tmp_path
CORE.toolchain = Toolchain.ESP_IDF
CORE.data[KEY_CORE] = {
KEY_TARGET_PLATFORM: str(Platform.ESP32),
KEY_TARGET_FRAMEWORK: str(Framework.ESP_IDF),
}
def test_write_idf_component_yml_passes_managed_components(
tmp_path: Path,
monkeypatch: pytest.MonkeyPatch,
) -> None:
"""The names already registered via add_idf_component (e.g. noise-c from
api's encryption config) are passed through as ``managed`` so the
PlatformIO-library converter skips them."""
_setup_core(tmp_path)
CORE.data[esp32.KEY_ESP32] = {
esp32.KEY_COMPONENTS: {
"esphome/noise-c": {
esp32.KEY_REPO: None,
esp32.KEY_REF: "0.1.15",
esp32.KEY_PATH: None,
},
},
}
captured: dict[str, set[str] | None] = {}
# A converted (non-managed) library the batch still resolves, so the loop
# wiring its override_path into the manifest is exercised for real too.
converted = MagicMock()
converted.get_sanitized_name.return_value = "esphome/other-lib"
converted.path = tmp_path / "pio_components" / "other-lib"
def fake_generate_idf_components(libraries, managed=None):
captured["managed"] = managed
return [converted]
monkeypatch.setattr(esp32, "generate_idf_components", fake_generate_idf_components)
esp32._write_idf_component_yml()
assert captured["managed"] == {"esphome/noise-c"}
# The managed component itself is still written into the manifest deps
# directly (from KEY_COMPONENTS), just not converted a second time.
yml_path = tmp_path / "src" / "idf_component.yml"
assert yml_path.is_file()
contents = yml_path.read_text(encoding="utf-8")
assert "esphome/noise-c" in contents
assert "0.1.15" in contents
# The converted library the batch DID return is still wired in.
assert "esphome/other-lib" in contents
assert str(converted.path) in contents
def test_write_idf_component_yml_empty_managed_when_no_components(
tmp_path: Path,
monkeypatch: pytest.MonkeyPatch,
) -> None:
"""No managed components registered yet (no add_idf_component calls) ->
an empty managed set, matching the pre-existing (unfiltered) behavior."""
_setup_core(tmp_path)
CORE.data[esp32.KEY_ESP32] = {esp32.KEY_COMPONENTS: {}}
captured: dict[str, set[str] | None] = {}
def fake_generate_idf_components(libraries, managed=None):
captured["managed"] = managed
return []
monkeypatch.setattr(esp32, "generate_idf_components", fake_generate_idf_components)
esp32._write_idf_component_yml()
assert captured["managed"] == set()
+113 -1
View File
@@ -3,12 +3,22 @@
import json
import os
from pathlib import Path
from types import SimpleNamespace
from unittest.mock import patch
import pytest
import yaml
from esphome.espidf import clang_tidy
from esphome.espidf.clang_tidy import _Settings, _setup_core, _write_tidy_project
from esphome.espidf.clang_tidy import (
_arduino_excluded_stubs,
_convert_pio_libs,
_esphome_manifest_deps,
_Settings,
_setup_core,
_write_tidy_project,
)
import esphome.espidf.component as espidf_component
REPO_ROOT = Path(__file__).resolve().parents[2]
@@ -69,6 +79,108 @@ def test_setup_core_sets_arduino_env(
assert os.environ["ESPHOME_ARDUINO_COMPONENT"] == expected
def test_esphome_manifest_deps_reads_repo_manifest() -> None:
"""Returns the top-level dependency names from esphome/idf_component.yml,
independent of any per-dependency framework rules."""
manifest = yaml.safe_load(
(REPO_ROOT / "esphome" / "idf_component.yml").read_text(encoding="utf-8")
)
deps = _esphome_manifest_deps()
assert isinstance(deps, set)
assert "esphome/noise-c" in deps
assert "esphome/libsodium" in deps
# Cross-check against a fresh parse instead of hardcoding the manifest's
# whole key list, so this doesn't need updating whenever a dependency is
# added or removed.
assert deps == set(manifest["dependencies"])
def test_convert_pio_libs_arduino_framework_passes_empty_managed(
tmp_path: Path,
monkeypatch: pytest.MonkeyPatch,
) -> None:
"""On Arduino, ESPHome's manifest entries for noise-c/libsodium are
rule-gated off (arduino-esp32 brings its own libsodium), so nothing
provides them there -- managed must be empty and they go through the
PlatformIO-library converter as before."""
monkeypatch.setattr(clang_tidy, "_parse_lib_deps", lambda ini, framework: [])
captured: dict[str, set[str] | None] = {}
# A converted library the batch resolves, so the loop wiring its
# override_path into the returned deps mapping is exercised for real too.
converted = SimpleNamespace(
get_sanitized_name=lambda: "esphome/other-lib",
path=tmp_path / "other-lib",
)
def fake_generate_idf_components(libraries, managed=None):
captured["managed"] = managed
return [converted]
monkeypatch.setattr(
espidf_component, "generate_idf_components", fake_generate_idf_components
)
result = _convert_pio_libs(tmp_path / "platformio.ini", "arduino")
assert captured["managed"] == set()
assert result == {
"esphome/other-lib": {"override_path": str(tmp_path / "other-lib")}
}
def test_convert_pio_libs_espidf_framework_passes_manifest_deps(
tmp_path: Path,
monkeypatch: pytest.MonkeyPatch,
) -> None:
"""On ESP-IDF, libraries ESPHome's own manifest already provides as
managed components (noise-c, libsodium, ...) must be passed through as
``managed`` so the converter skips them -- converting them too would make
IDF see the same requirement twice."""
monkeypatch.setattr(clang_tidy, "_parse_lib_deps", lambda ini, framework: [])
captured: dict[str, set[str] | None] = {}
def fake_generate_idf_components(libraries, managed=None):
captured["managed"] = managed
return []
monkeypatch.setattr(
espidf_component, "generate_idf_components", fake_generate_idf_components
)
result = _convert_pio_libs(tmp_path / "platformio.ini", "espidf")
assert captured["managed"] == _esphome_manifest_deps()
assert "esphome/noise-c" in captured["managed"]
assert result == {}
def test_arduino_excluded_stubs_skips_components_esphome_manifest_provides(
tmp_path: Path,
) -> None:
"""A component ESPHome's own idf_component.yml declares for real (e.g.
espressif/lan867x for ethernet) must not be stubbed away -- stubbing it
would silently disable ethernet on Arduino. A component that is only ever
bundled by arduino-esp32 (never in ESPHome's own manifest) still gets a
stub so the arduino-bundled copy doesn't clash with noise-c's libsodium."""
deps = _arduino_excluded_stubs(tmp_path)
# lan867x is a real ESPHome dependency (esphome/idf_component.yml), so it
# must be excluded from the stub set.
assert "espressif/lan867x" not in deps
# espressif/libsodium (arduino-esp32's bundled copy) is a different
# package from ESPHome's own esphome/libsodium, so it's still stubbed.
assert "espressif/libsodium" in deps
stub_info = deps["espressif/libsodium"]
assert stub_info["version"] == "*"
stub_path = Path(stub_info["override_path"])
assert (stub_path / "CMakeLists.txt").is_file()
def test_idedata_from_tidy_project(tmp_path) -> None:
"""The tidy TU's compile entry is assembled into consumer-shaped idedata."""
compile_commands = tmp_path / "compile_commands.json"
+106
View File
@@ -803,6 +803,112 @@ def test_generate_idf_components_lib_ignore_filters_top_level_and_dependencies(
assert download_salts == [hashlib.sha256(b"b,c").hexdigest()[:8]]
def test_generate_idf_components_managed_filters_top_level_and_dependencies(
tmp_path: Path,
monkeypatch: pytest.MonkeyPatch,
esp32_idf_core: None,
) -> None:
# managed (e.g. noise-c/libsodium already declared via add_idf_component)
# must drop B at the top level and C when discovered as a dependency of A,
# exactly like lib_ignore -- neither may be resolved, downloaded, or wired
# into a manifest.
manifests = {
"esphome/A": {
"name": "A",
"dependencies": [
{"owner": "esphome", "name": "C", "version": "==1.10021.0"}
],
},
"esphome/B": {"name": "B"},
}
download_salts: list[str] = []
def fake_download(self, force=False, salt="", namespace=""):
download_salts.append(salt)
self.path = tmp_path / self.get_sanitized_name().replace("/", "__")
(self.path / "src").mkdir(parents=True, exist_ok=True)
(self.path / "src" / "x.c").write_text("int x;")
(self.path / "library.json").write_text(json.dumps(manifests[self.name]))
monkeypatch.setattr(IDFComponent, "download", fake_download)
resolve_calls: list[str] = []
def fake_resolve(owner, pkgname, requirements):
resolve_calls.append(pkgname)
return owner, pkgname, "1.0.0", f"http://x/{pkgname}.tar.gz", None
monkeypatch.setattr(
esphome.platformio.library, "_resolve_registry_version", fake_resolve
)
top = generate_idf_components(
[Library("esphome/A", "1.0.0", None), Library("esphome/B", "1.0.0", None)],
managed={"esphome/B", "esphome/C"},
)
assert [c.name for c in top] == ["esphome/A"]
# Managed libraries were never resolved (and therefore never downloaded).
assert resolve_calls == ["A"]
# The managed dependency is not wired into A's manifest.
assert top[0].dependencies == []
# managed changes the generated wiring just like lib_ignore, so the cache
# path is salted the same way.
assert download_salts == [hashlib.sha256(b"b,c").hexdigest()[:8]]
def test_generate_idf_components_lib_ignore_and_managed_combine_into_salt(
tmp_path: Path,
monkeypatch: pytest.MonkeyPatch,
esp32_idf_core: None,
) -> None:
# lib_ignore and managed both contribute to the same exclusion set, so a
# config using both gets a salt reflecting the union of the two sources
# rather than either alone.
manifests = {
"esphome/A": {"name": "A"},
"esphome/D": {"name": "D"},
"esphome/E": {"name": "E"},
}
download_salts: list[str] = []
def fake_download(self, force=False, salt="", namespace=""):
download_salts.append(salt)
self.path = tmp_path / self.get_sanitized_name().replace("/", "__")
(self.path / "src").mkdir(parents=True, exist_ok=True)
(self.path / "src" / "x.c").write_text("int x;")
(self.path / "library.json").write_text(json.dumps(manifests[self.name]))
monkeypatch.setattr(IDFComponent, "download", fake_download)
resolve_calls: list[str] = []
def fake_resolve(owner, pkgname, requirements):
resolve_calls.append(pkgname)
return owner, pkgname, "1.0.0", f"http://x/{pkgname}.tar.gz", None
monkeypatch.setattr(
esphome.platformio.library, "_resolve_registry_version", fake_resolve
)
monkeypatch.setattr(CORE, "platformio_options", {"lib_ignore": ["D"]})
top = generate_idf_components(
[
Library("esphome/A", "1.0.0", None),
Library("esphome/D", "1.0.0", None),
Library("esphome/E", "1.0.0", None),
],
managed={"esphome/E"},
)
assert [c.name for c in top] == ["esphome/A"]
assert resolve_calls == ["A"]
# The salt reflects BOTH lib_ignore's "D" and managed's "E" together.
assert download_salts == [hashlib.sha256(b"d,e").hexdigest()[:8]]
def test_generate_idf_components_handles_dependency_cycle(
tmp_path: Path,
monkeypatch: pytest.MonkeyPatch,
-407
View File
@@ -1,407 +0,0 @@
"""Unit tests for encrypted OTA uploads in esphome.espota2.
A fake device implementing the responder side of the wire protocol (via
noiseprotocol, which esphome already has through aioesphomeapi) serves a real
TCP loopback connection, so these exercise the actual handshake, framing, and
cipher interop of the client code. Tests that need the client-side crypto skip
when the installed aioesphomeapi predates the noise module.
"""
from __future__ import annotations
import base64
import hashlib
import io
from pathlib import Path
import socket
import sys
import threading
from unittest.mock import Mock, patch
import pytest
from esphome import espota2
PSK = base64.b64encode(bytes(range(32))).decode()
OTHER_PSK = base64.b64encode(bytes(range(1, 33))).decode()
MAGIC = bytes(espota2.MAGIC_BYTES)
def _recv_exact(sock: socket.socket, amount: int) -> bytes:
data = b""
while len(data) < amount:
chunk = sock.recv(amount - len(data))
if not chunk:
raise ConnectionError("client closed")
data += chunk
return data
def _frame(payload: bytes) -> bytes:
return (
bytes([espota2.NOISE_FRAME_INDICATOR, len(payload) >> 8, len(payload) & 0xFF])
+ payload
)
def _send_frame(sock: socket.socket, payload: bytes) -> None:
sock.sendall(_frame(payload))
def _recv_frame(sock: socket.socket) -> bytes:
header = _recv_exact(sock, 3)
assert header[0] == 0x01
return _recv_exact(sock, (header[1] << 8) | header[2])
class FakeEncryptedDevice(threading.Thread):
"""Responder side of the encrypted OTA wire protocol."""
def __init__(
self,
psk: str = PSK,
version: int = 2,
offer_noise: bool = True,
require_noise: bool = True,
prologue_features_override: int | None = None,
) -> None:
super().__init__(daemon=True)
self.psk = psk
self.version = version
self.offer_noise = offer_noise
self.require_noise = require_noise
self.prologue_features_override = prologue_features_override
self.received: bytes | None = None
self.error: Exception | None = None
self.listener = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
self.listener.bind(("127.0.0.1", 0))
self.listener.listen(1)
self.port = self.listener.getsockname()[1]
def run(self) -> None:
try:
sock, _ = self.listener.accept()
sock.settimeout(10)
with sock:
self._serve(sock)
except Exception as err: # noqa: BLE001 - surfaced via join_and_check
self.error = err
finally:
self.listener.close()
def join_and_check(self) -> None:
self.join(timeout=10)
assert not self.is_alive(), "fake device did not finish"
if self.error is not None:
raise self.error
def _serve(self, sock: socket.socket) -> None:
assert _recv_exact(sock, 5) == MAGIC
sock.sendall(bytes([espota2.RESPONSE_OK, self.version]))
features = _recv_exact(sock, 1)[0]
noise_negotiated = bool(
features & espota2.CLIENT_FEATURE_SUPPORTS_NOISE
and features & espota2.CLIENT_FEATURE_SUPPORTS_EXTENDED_PROTOCOL
)
if self.require_noise and not noise_negotiated:
sock.sendall(bytes([espota2.RESPONSE_ERROR_ENCRYPTION_REQUIRED]))
return
server_flags = espota2.SERVER_FEATURE_SUPPORTS_NOISE if self.offer_noise else 0
sock.sendall(bytes([espota2.RESPONSE_FEATURE_FLAGS, server_flags]))
if not (self.offer_noise and noise_negotiated):
return # the client fails closed; nothing further arrives
from cryptography.exceptions import InvalidTag
from noise.connection import NoiseConnection
prologue_features = (
features
if self.prologue_features_override is None
else self.prologue_features_override
)
prologue = (
espota2.NOISE_PROLOGUE_INIT
+ MAGIC
+ bytes([espota2.RESPONSE_OK, self.version, prologue_features])
+ bytes([espota2.RESPONSE_FEATURE_FLAGS, server_flags])
)
proto = NoiseConnection.from_name(b"Noise_NNpsk0_25519_ChaChaPoly_SHA256")
proto.set_as_responder()
proto.set_psks(base64.b64decode(self.psk))
proto.set_prologue(prologue)
proto.start_handshake()
msg1 = _recv_frame(sock)
assert msg1[0] == 0x00
try:
proto.read_message(msg1[1:])
except InvalidTag:
_send_frame(sock, b"\x01" + espota2.NOISE_MAC_FAILURE_REASON.encode())
return
_send_frame(sock, b"\x00" + bytes(proto.write_message()))
def send_byte(byte: int) -> None:
_send_frame(sock, proto.encrypt(bytes([byte])))
def recv_unit(length: int) -> bytes:
plaintext = proto.decrypt(_recv_frame(sock))
assert len(plaintext) == length, "control units must be one per frame"
return plaintext
send_byte(espota2.RESPONSE_AUTH_OK)
recv_unit(1) # ota type
size = int.from_bytes(recv_unit(4), "big")
send_byte(espota2.RESPONSE_UPDATE_PREPARE_OK)
md5_hex = recv_unit(32)
send_byte(espota2.RESPONSE_BIN_MD5_OK)
received = b""
acked = 0
while len(received) < size:
plaintext = proto.decrypt(_recv_frame(sock))
assert 0 < len(plaintext) <= espota2.NOISE_MAX_PLAINTEXT
received += plaintext
if self.version >= espota2.OTA_VERSION_2_0:
while acked + espota2.UPLOAD_BLOCK_SIZE <= len(received) or (
len(received) == size and acked < size
):
send_byte(espota2.RESPONSE_CHUNK_OK)
acked += espota2.UPLOAD_BLOCK_SIZE
assert hashlib.md5(received).hexdigest().encode() == md5_hex
send_byte(espota2.RESPONSE_RECEIVE_OK)
send_byte(espota2.RESPONSE_UPDATE_END_OK)
assert recv_unit(1) == bytes([espota2.RESPONSE_OK])
self.received = received
def _upload(
device: FakeEncryptedDevice, firmware: bytes, noise_psk: str | None
) -> None:
device.start()
sock = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
sock.settimeout(10)
sock.connect(("127.0.0.1", device.port))
try:
espota2.perform_ota(
sock, None, io.BytesIO(firmware), Path("firmware.bin"), noise_psk=noise_psk
)
finally:
sock.close()
def test_encrypted_upload_success() -> None:
"""A full encrypted v2 upload spanning several 8192-byte blocks."""
pytest.importorskip("aioesphomeapi.noise")
firmware = bytes(range(256)) * 80 # 20480 bytes, crosses chunk-ack boundaries
device = FakeEncryptedDevice()
with patch("time.sleep"):
_upload(device, firmware, PSK)
device.join_and_check()
assert device.received == firmware
def test_encrypted_upload_version_1() -> None:
"""Version 1 protocol (no chunk acks) works through the noise transport."""
pytest.importorskip("aioesphomeapi.noise")
firmware = b"v1 firmware image" * 100
device = FakeEncryptedDevice(version=1)
with patch("time.sleep"):
_upload(device, firmware, PSK)
device.join_and_check()
assert device.received == firmware
def test_wrong_key_fails_with_clear_error() -> None:
"""A key mismatch surfaces the device's handshake reject readably."""
pytest.importorskip("aioesphomeapi.noise")
device = FakeEncryptedDevice(psk=OTHER_PSK)
with pytest.raises(espota2.OTAError, match="encryption key correct"):
_upload(device, b"firmware", PSK)
device.join_and_check()
def test_tampered_negotiation_breaks_handshake() -> None:
"""A negotiation byte differing between the sides breaks the prologue MAC."""
pytest.importorskip("aioesphomeapi.noise")
device = FakeEncryptedDevice(
prologue_features_override=espota2.CLIENT_FEATURE_SUPPORTS_EXTENDED_PROTOCOL
)
with pytest.raises(espota2.OTAError, match="encryption key correct"):
_upload(device, b"firmware", PSK)
device.join_and_check()
def test_client_fails_closed_when_device_lacks_encryption() -> None:
"""With a key configured, a device not offering noise aborts the upload."""
device = FakeEncryptedDevice(offer_noise=False, require_noise=False)
with pytest.raises(espota2.OTAError, match="refusing to send the image"):
_upload(device, b"firmware", PSK)
device.join_and_check()
def test_plaintext_client_gets_encryption_required_error() -> None:
"""A client without a key gets the device's 0x94 error message."""
device = FakeEncryptedDevice()
with pytest.raises(espota2.OTAError, match="requires an encrypted OTA"):
_upload(device, b"firmware", None)
device.join_and_check()
def test_missing_aioesphomeapi_noise_module_message() -> None:
"""An aioesphomeapi without the noise module produces a clear error."""
with (
patch.dict(sys.modules, {"aioesphomeapi.noise": None}),
pytest.raises(espota2.OTAError, match="requires a newer aioesphomeapi"),
):
espota2.NoiseSocketWrapper(Mock(), PSK, b"prologue")
class ScriptedSocket:
"""Serves scripted recv chunks; b"" means the peer closed."""
def __init__(self, *chunks: bytes | Exception) -> None:
self.chunks = list(chunks)
self.sent: list[bytes] = []
def sendall(self, data: bytes) -> None:
self.sent.append(data)
def settimeout(self, timeout: float) -> None:
pass
def recv(self, amount: int) -> bytes:
if not self.chunks:
return b""
chunk = self.chunks[0]
if isinstance(chunk, Exception):
self.chunks.pop(0)
raise chunk
take, rest = chunk[:amount], chunk[amount:]
if rest:
self.chunks[0] = rest
else:
self.chunks.pop(0)
return take
def _wrapper(*chunks: bytes | Exception) -> espota2.NoiseSocketWrapper:
pytest.importorskip("aioesphomeapi.noise")
return espota2.NoiseSocketWrapper(ScriptedSocket(*chunks), PSK, b"prologue")
def test_wrapper_rejects_malformed_psk() -> None:
pytest.importorskip("aioesphomeapi.noise")
with pytest.raises(espota2.OTAError, match="Invalid OTA encryption key"):
espota2.NoiseSocketWrapper(ScriptedSocket(), "not-base64!!!", b"prologue")
def test_handshake_socket_error_is_network_error() -> None:
wrapper = _wrapper(OSError("boom"))
with pytest.raises(espota2.OTANetworkError, match="noise handshake"):
wrapper.do_handshake()
def test_handshake_closed_at_frame_boundary() -> None:
wrapper = _wrapper()
with pytest.raises(espota2.OTANetworkError, match="closed connection during"):
wrapper.do_handshake()
def test_handshake_reject_with_other_reason() -> None:
wrapper = _wrapper(_frame(b"\x01Handshake error"))
with pytest.raises(
espota2.OTAError, match="rejected the noise handshake: Handshake error"
):
wrapper.do_handshake()
def test_handshake_garbage_second_message() -> None:
"""A valid-looking point with a garbage MAC fails cleanly."""
wrapper = _wrapper(_frame(b"\x00" + bytes(range(48))))
with pytest.raises(
espota2.OTAError, match="handshake failed; is the OTA encryption key"
):
wrapper.do_handshake()
def test_handshake_invalid_curve_point() -> None:
"""An all-zero x25519 point is rejected as a clean error, not a crash."""
wrapper = _wrapper(_frame(b"\x00" + bytes(48)))
with pytest.raises(
espota2.OTAError, match="handshake failed; is the OTA encryption key"
):
wrapper.do_handshake()
def test_recv_closed_at_frame_boundary_returns_empty() -> None:
wrapper = _wrapper()
assert wrapper.recv(1) == b""
def test_recv_corrupt_frame_is_retryable_network_error() -> None:
from cryptography.exceptions import InvalidTag
wrapper = _wrapper(_frame(b"ciphertext"))
wrapper._decrypt = Mock(decrypt=Mock(side_effect=InvalidTag()))
with pytest.raises(espota2.OTANetworkError, match="decryption failed"):
wrapper.recv(1)
def test_wrapper_blocks_unencrypted_socket_methods() -> None:
"""Byte-moving socket methods must not bypass the encrypted transport."""
wrapper = _wrapper()
# The harmless socket controls pass through to the wrapped socket
wrapper._sock = Mock()
wrapper.settimeout(1)
wrapper._sock.settimeout.assert_called_once_with(1)
wrapper.setsockopt(6, 1, 1)
wrapper._sock.setsockopt.assert_called_once_with(6, 1, 1)
wrapper.close()
wrapper._sock.close.assert_called_once_with()
with pytest.raises(AttributeError):
_ = wrapper.send
with pytest.raises(AttributeError):
_ = wrapper.recv_into
def test_recv_empty_plaintext_frame_is_protocol_error() -> None:
"""A MAC-only frame decrypts to nothing; b'' from recv must mean close."""
wrapper = _wrapper(_frame(bytes(16)))
wrapper._decrypt = Mock(decrypt=Mock(return_value=b""))
with pytest.raises(espota2.OTANetworkError, match="empty noise frame"):
wrapper.recv(1)
def test_recv_frame_bad_indicator_is_retryable() -> None:
wrapper = _wrapper(b"\x02\x00\x01x")
with pytest.raises(espota2.OTANetworkError, match="Bad noise frame indicator"):
wrapper._recv_frame()
def test_recv_frame_zero_length_is_retryable() -> None:
wrapper = _wrapper(bytes([espota2.NOISE_FRAME_INDICATOR, 0, 0]))
with pytest.raises(espota2.OTANetworkError, match="empty noise frame"):
wrapper._recv_frame()
def test_perform_ota_blank_key_refuses_plaintext() -> None:
with pytest.raises(espota2.OTAError, match="empty OTA encryption key"):
espota2.perform_ota(
ScriptedSocket(), None, io.BytesIO(b"x"), Path("f.bin"), noise_psk=""
)
def test_recv_exact_closed_mid_frame() -> None:
wrapper = _wrapper(_frame(b"partial")[:5])
with pytest.raises(OSError, match="closed inside a noise frame"):
wrapper._recv_frame()
def test_recv_serves_buffered_plaintext_without_new_frame() -> None:
"""A second recv drains the decrypted buffer without reading another frame."""
wrapper = _wrapper(_frame(b"ciphertext"))
wrapper._decrypt = Mock(decrypt=Mock(return_value=b"AB"))
assert wrapper.recv(1) == b"A" # reads and decrypts one frame
assert wrapper.recv(1) == b"B" # served from the buffer, no new frame
wrapper._decrypt.decrypt.assert_called_once()
+5 -103
View File
@@ -86,9 +86,7 @@ from esphome.const import (
CONF_BROKER,
CONF_DISABLED,
CONF_DISCOVER_IP,
CONF_ENCRYPTION,
CONF_ESPHOME,
CONF_KEY,
CONF_LEVEL,
CONF_LOG,
CONF_LOG_TOPIC,
@@ -2108,65 +2106,10 @@ def test_upload_program_ota_success(
tmp_path / ".esphome" / "build" / "test" / ".pioenvs" / "test" / "firmware.bin"
)
mock_run_ota.assert_called_once_with(
["192.168.1.100"], 3232, "secret", expected_firmware, OTA_TYPE_UPDATE_APP, None
["192.168.1.100"], 3232, "secret", expected_firmware, OTA_TYPE_UPDATE_APP
)
def test_upload_program_ota_encryption_key(
mock_run_ota: Mock,
mock_get_port_type: Mock,
tmp_path: Path,
) -> None:
"""The resolved encryption key is passed through to run_ota."""
setup_core(platform=PLATFORM_ESP32, tmp_path=tmp_path)
mock_get_port_type.return_value = "NETWORK"
mock_run_ota.return_value = (0, "192.168.1.100")
key = "AAECAwQFBgcICQoLDA0ODxAREhMUFRYXGBkaGxwdHh8="
config = {
CONF_OTA: [
{
CONF_PLATFORM: CONF_ESPHOME,
CONF_PORT: 3232,
CONF_ENCRYPTION: {CONF_KEY: key},
}
]
}
exit_code, host = upload_program(config, MockArgs(), ["192.168.1.100"])
assert exit_code == 0
assert host == "192.168.1.100"
expected_firmware = (
tmp_path / ".esphome" / "build" / "test" / ".pioenvs" / "test" / "firmware.bin"
)
mock_run_ota.assert_called_once_with(
["192.168.1.100"], 3232, None, expected_firmware, OTA_TYPE_UPDATE_APP, key
)
def test_upload_program_ota_encryption_without_key_fails_closed(
mock_run_ota: Mock,
mock_get_port_type: Mock,
tmp_path: Path,
) -> None:
"""An encryption block with no resolved key must never upload plaintext."""
setup_core(platform=PLATFORM_ESP32, tmp_path=tmp_path)
mock_get_port_type.return_value = "NETWORK"
config = {
CONF_OTA: [
{
CONF_PLATFORM: CONF_ESPHOME,
CONF_PORT: 3232,
CONF_ENCRYPTION: {},
}
]
}
with pytest.raises(EsphomeError, match="no key was resolved"):
upload_program(config, MockArgs(), ["192.168.1.100"])
mock_run_ota.assert_not_called()
def test_upload_program_ota_with_file_arg(
mock_run_ota: Mock,
mock_get_port_type: Mock,
@@ -2194,7 +2137,7 @@ def test_upload_program_ota_with_file_arg(
assert exit_code == 0
assert host == "192.168.1.100"
mock_run_ota.assert_called_once_with(
["192.168.1.100"], 3232, None, Path("custom.bin"), OTA_TYPE_UPDATE_APP, None
["192.168.1.100"], 3232, None, Path("custom.bin"), OTA_TYPE_UPDATE_APP
)
@@ -2249,7 +2192,6 @@ def test_upload_program_ota_partition_table_with_file_arg(
None,
partition_file,
OTA_TYPE_UPDATE_PARTITION_TABLE,
None,
)
@@ -2311,7 +2253,6 @@ def test_upload_program_ota_partition_table_mqttip(
None,
partition_file,
OTA_TYPE_UPDATE_PARTITION_TABLE,
None,
)
@@ -2499,7 +2440,6 @@ def test_upload_program_ota_bootloader_with_file_arg(
None,
bootloader_file,
OTA_TYPE_UPDATE_BOOTLOADER,
None,
)
@@ -2662,42 +2602,6 @@ def test_has_web_server_logging_respects_log_disabled() -> None:
assert has_web_server_logging() is False
def test_upload_program_web_server_warns_when_encryption_configured(
mock_run_web_server_ota: Mock,
mock_run_ota: Mock,
mock_get_port_type: Mock,
tmp_path: Path,
caplog: pytest.LogCaptureFixture,
) -> None:
"""Explicitly picking web_server OTA on an encrypted config warns about
the plaintext upload path."""
setup_core(platform=PLATFORM_ESP32, tmp_path=tmp_path)
mock_get_port_type.return_value = "NETWORK"
mock_run_web_server_ota.return_value = (0, "192.168.1.100")
config = {
CONF_OTA: [
{
CONF_PLATFORM: CONF_ESPHOME,
CONF_PORT: 3232,
CONF_ENCRYPTION: {CONF_KEY: "test_key"},
},
{CONF_PLATFORM: CONF_WEB_SERVER},
],
CONF_WEB_SERVER: {
CONF_PORT: 80,
CONF_AUTH: {CONF_USERNAME: "admin", CONF_PASSWORD: "pw"},
},
}
args = MockArgs(ota_platform=CONF_WEB_SERVER)
with caplog.at_level(logging.WARNING):
exit_code, _ = upload_program(config, args, ["192.168.1.100"])
assert exit_code == 0
assert any("plaintext HTTP" in record.message for record in caplog.records)
mock_run_ota.assert_not_called()
def test_upload_program_web_server_only_auto_dispatches(
mock_run_web_server_ota: Mock,
mock_run_ota: Mock,
@@ -2988,7 +2892,7 @@ def test_upload_program_ota_with_mqtt_resolution(
tmp_path / ".esphome" / "build" / "test" / ".pioenvs" / "test" / "firmware.bin"
)
mock_run_ota.assert_called_once_with(
["192.168.1.100"], 3232, None, expected_firmware, OTA_TYPE_UPDATE_APP, None
["192.168.1.100"], 3232, None, expected_firmware, OTA_TYPE_UPDATE_APP
)
@@ -3038,7 +2942,7 @@ def test_upload_program_ota_with_mqtt_empty_broker(
tmp_path / ".esphome" / "build" / "test" / ".pioenvs" / "test" / "firmware.bin"
)
mock_run_ota.assert_called_once_with(
["192.168.1.50"], 3232, None, expected_firmware, OTA_TYPE_UPDATE_APP, None
["192.168.1.50"], 3232, None, expected_firmware, OTA_TYPE_UPDATE_APP
)
# Verify warning was logged
assert "MQTT IP discovery failed" in caplog.text
@@ -5210,7 +5114,6 @@ def test_upload_program_ota_static_ip_with_mqttip(
None,
expected_firmware,
OTA_TYPE_UPDATE_APP,
None,
)
@@ -5260,7 +5163,6 @@ def test_upload_program_ota_multiple_mqttip_resolves_once(
None,
expected_firmware,
OTA_TYPE_UPDATE_APP,
None,
)
@@ -5438,7 +5340,7 @@ def test_upload_program_ota_mqtt_timeout_fallback(
tmp_path / ".esphome" / "build" / "test" / ".pioenvs" / "test" / "firmware.bin"
)
mock_run_ota.assert_called_once_with(
["192.168.1.100"], 3232, None, expected_firmware, OTA_TYPE_UPDATE_APP, None
["192.168.1.100"], 3232, None, expected_firmware, OTA_TYPE_UPDATE_APP
)