[api] Outline the fixed32 writers on ARM cores without unaligned access too

Cortex-M0+ and ARM9 turn the four byte unaligned store into a memcpy call with a stack
temporary at every fixed32 field, and the outlined helper itself became a memcpy call
there, so the helper now spells out the byte stores. Xtensa and host objects are byte for
byte unchanged; on the RP2040 bench config the api object loses 28 bytes and the fixed32
memcpy calls.

AGENTS.md

@@ -553,6 +553,7 @@ file does, and it is the authority when they disagree. The most useful starting
     4.  **Lint:** Run `prek` to ensure code is compliant.
     5.  **Commit:** Commit your changes. There is no strict format for commit messages.
     6.  **Pull Request:** Submit a PR against the `dev` branch. The Pull Request title must start with a `[tag]` prefix. For component work, use the component name (e.g., `[display] Fix bug`, `[abc123] Add new component`); for changes to shared/core code that isn't tied to a single component, use `[core]` (e.g., `[core] Add validator`). Update documentation, examples, and add `CODEOWNERS` entries as needed. Pull requests should always be made using the `.github/PULL_REQUEST_TEMPLATE.md` template - fill out all sections completely without removing any parts of the template.
+    7.  **Comments:** When commenting on GitHub PRs or issues, don't tag contributors, especially bots. Avoid referring to list items (e.g. from reviews) with the form #nn - this will be interpreted by GitHub as a reference to issue or PR nn. Keep comments short and exclude irrelevant details, backstories, restatement of previous comments and anything that is already obvious to the reader.
 
 *   **Documentation Contributions:**
     *   Documentation is hosted in the separate `esphome/esphome.io` repository.

THREAT_MODEL.md

@@ -125,30 +125,47 @@ design is optimal or that it will not change.
 ## OTA update encryption
 
 The `esphome` OTA platform optionally encrypts updates with the same Noise
-`NNpsk0` pattern the native API uses; one key protects the device. With an
-`encryption:` block configured the guarantees are: the firmware image is
-confidential in transit, the uploader is authenticated by the pre-shared key,
-and the plaintext negotiation preceding the handshake is bound into the
-handshake prologue, so stripping or tampering with it fails the first MAC.
-Both ends fail closed with no override: a device built with a key refuses
+`NNpsk0` pattern the native API uses; one key protects the device. A device
+whose `api:` block has an encryption key, static in the YAML or provisioned at
+runtime, compiles in the transport and offers it on every OTA connection once
+it holds a key, so an uploader presenting that key gets the guarantees below
+even without an `ota: encryption:` block; only that block makes the device
+require encryption. The guarantees are: the firmware image is confidential in
+transit, the uploader is authenticated by the pre-shared key, and the plaintext
+negotiation preceding the handshake is bound into the handshake prologue, so
+stripping or tampering with it fails the first MAC. With `ota: encryption:`
+configured both ends fail closed with no override: the device refuses
 plaintext uploads, and the CLI refuses to send plaintext when a key is
-configured.
+configured. Without that block the CLI tries a static api key when the device
+offers and, until 2027.3.0, falls back to plaintext with a warning when the
+offer is missing or the handshake fails; a runtime provisioned key never
+reaches the CLI, so those uploads stay plaintext.
 
-Defeating any of that without the key is in scope: a keyed device accepting a
-plaintext or downgraded upload, getting past the MAC, or recovering image
-contents from captured traffic.
+Defeating any of that without the key is in scope: a device that requires
+encryption accepting a plaintext or downgraded upload, getting past the MAC,
+or recovering image contents from captured traffic.
 
 The following are **not** vulnerabilities, by design:
 
-- Plaintext OTA on a device with no `encryption:` block. That is the
-  documented default, authenticated (if at all) by the OTA password.
-- The enablement window: turning encryption on takes one last upload of the
-  encryption-enabled firmware over the existing plaintext channel, with the
-  pre-existing plaintext exposure.
-- The web OTA `/update` endpoint alongside encryption. The `web_server`
-  component keeps it always reachable, and `captive_portal:` auto-loads it
-  for the fallback AP window; validation warns about both combinations, and
-  the operator keeps the recovery path.
+- Plaintext OTA on a device with no `ota: encryption:` block, including one
+  that offers encryption because it has an api key. That is the documented
+  default, authenticated (if at all) by the OTA password. An uploader that
+  takes the offer skips the password; the key authenticates it. With a
+  runtime provisioned key and no `provisioning:` window, whoever provisions
+  the key gains that upload path too; validation warns about the pair.
+- The CLI plaintext fallback until 2027.3.0: without `ota: encryption:` an
+  active attacker who strips the offer or breaks the handshake can make a
+  keyed CLI upload plaintext, with the pre-existing plaintext exposure. A
+  device that requires encryption still refuses that upload.
+- The enablement window: firmware built with a static api key already offers
+  encryption, so turning on `ota: encryption:` is itself an encrypted upload.
+  Older firmware needs one last plaintext upload of an offering build, with
+  the pre-existing plaintext exposure.
+- The web OTA `/update` endpoint alongside encryption. With the `web_server`
+  or `prometheus` component the shared listener is always up, so the endpoint
+  stays reachable and validation warns about that combination;
+  `captive_portal:` alone brings the listener up only for the fallback AP
+  window, which is the intended recovery path, so that is not warned about.
 - CLI retry behavior on transport or MAC failures; every attempt renegotiates
   a fresh handshake with fresh ephemerals, so retrying does not weaken
   authentication.

docker/Dockerfile

@@ -22,7 +22,7 @@ RUN \
     -r /requirements.txt
 
 # Install the ESPHome Device Builder dashboard.
-RUN uv pip install --no-cache-dir esphome-device-builder==1.14.0
+RUN uv pip install --no-cache-dir esphome-device-builder==1.14.4
 
 RUN \
     platformio settings set enable_telemetry No \

esphome/__main__.py

@@ -1335,12 +1335,14 @@ def _upload_via_native_api(
             break
 
     from esphome import espota2
+    from esphome.components.noise import static_encryption_key
 
     remote_port = int(ota_conf[CONF_PORT])
     password = ota_conf.get(CONF_PASSWORD)
     # Fail closed: an encryption block whose key did not resolve must never
     # fall back to a plaintext upload
     noise_psk = None
+    plaintext_fallback = False
     if (encryption_conf := ota_conf.get(CONF_ENCRYPTION)) is not None:
         noise_psk = encryption_conf.get(CONF_KEY)
         if not noise_psk:
@@ -1351,6 +1353,10 @@ def _upload_via_native_api(
         # Ensure the key is a string, as required by the underlying OTA implementation.
         # It arrives here as a SensitiveStr which aioesphomeapi rejects.
         noise_psk = str(noise_psk)
+    elif api_key := static_encryption_key(config.get(CONF_API) or {}):
+        # Remove before 2027.3.0: the api key is tried, falling back to plaintext
+        noise_psk = str(api_key)
+        plaintext_fallback = True
 
     def check_partition_access(option_string: str) -> None:
         if not ota_conf.get("allow_partition_access"):
@@ -1382,7 +1388,13 @@ def _upload_via_native_api(
         _validate_bootloader_binary(binary)
 
     return espota2.run_ota(
-        network_devices, remote_port, password, binary, ota_type, noise_psk
+        network_devices,
+        remote_port,
+        password,
+        binary,
+        ota_type,
+        noise_psk,
+        plaintext_fallback=plaintext_fallback,
     )
 
 

esphome/components/api/__init__.py

@@ -14,6 +14,7 @@ from esphome.components.noise import (  # noqa: F401
     ENCRYPTION_SCHEMA,
     decode_encryption_key,
     encryption_schema,
+    new_psk_progmem,
     validate_encryption_key,
 )
 from esphome.config_helpers import filter_source_files_from_defines, get_logger_level
@@ -589,8 +590,7 @@ async def to_code(config: ConfigType) -> None:
 
     if (encryption_config := config.get(CONF_ENCRYPTION, None)) is not None:
         if key := encryption_config.get(CONF_KEY):
-            decoded = decode_encryption_key(key)
-            cg.add(var.set_noise_psk(list(decoded)))
+            cg.add(var.set_noise_psk(new_psk_progmem(config[CONF_ID], key)))
             cg.add_define("USE_API_NOISE_PSK_FROM_YAML")
         else:
             # No key provided, but encryption desired

esphome/components/api/api_connection.cpp

@@ -2161,7 +2161,10 @@ void APIConnection::on_homeassistant_action_response(const HomeassistantActionRe
 bool APIConnection::send_noise_encryption_set_key_response_(const NoiseEncryptionSetKeyRequest &msg) {
   NoiseEncryptionSetKeyResponse resp;
   resp.success = false;
-
+#ifdef USE_API_NOISE_PSK_FROM_YAML
+  // A yaml key cannot be changed at runtime, so no decode or save path is built
+  ESP_LOGW(TAG, "Key set in YAML");
+#else
 #ifdef USE_PROVISIONING
   // Refuse to set a key once the provisioning window has closed (defense in depth;
   // such connections are already rejected at hello).
@@ -2196,6 +2199,7 @@ bool APIConnection::send_noise_encryption_set_key_response_(const NoiseEncryptio
     }
 #endif
   }
+#endif  // USE_API_NOISE_PSK_FROM_YAML
 
   return this->send_message(resp);
 }

esphome/components/api/api_frame_helper_noise.cpp

@@ -548,7 +548,7 @@ APIError APINoiseFrameHelper::write_frame_(const uint8_t *data, uint16_t len) {
  * @return 0 on success, -1 on error (check errno)
  */
 APIError APINoiseFrameHelper::init_handshake_() {
-  int err = this->handshake_.init(this->ctx_.get_psk(), prologue_.data(), prologue_.size());
+  int err = this->handshake_.init(this->ctx_, prologue_.data(), prologue_.size());
   APIError aerr = handle_noise_error_(err, LOG_STR("noise_handshake_init"), APIError::HANDSHAKESTATE_SETUP_FAILED);
   if (aerr != APIError::OK)
     return aerr;

esphome/components/api/api_server.cpp

@@ -41,13 +41,13 @@ void APIServer::setup() {
   ControllerRegistry::register_controller(this);
 
 #ifdef USE_API_NOISE
+  // Always reserve the slot: flash preferences are positional on esp8266, so
+  // a yaml key build must keep the layout of a runtime key build
   uint32_t hash = 88491486UL;
-
   this->noise_pref_ = global_preferences->make_preference<SavedNoisePsk>(hash, true);
-
 #ifndef USE_API_NOISE_PSK_FROM_YAML
-  // Only load saved PSK if not set from YAML
-  if (this->load_and_apply_noise_psk_()) {
+  // A cleared record loads fine but holds no key
+  if (this->load_and_apply_noise_psk_() && this->noise_ctx_.has_psk()) {
     ESP_LOGD(TAG, "Loaded saved Noise PSK");
   }
 #endif
@@ -550,6 +550,7 @@ const std::vector<APIServer::HomeAssistantStateSubscription> &APIServer::get_sta
 #endif
 
 #ifdef USE_API_NOISE
+#ifndef USE_API_NOISE_PSK_FROM_YAML
 bool APIServer::update_noise_psk_(const SavedNoisePsk &new_psk, const LogString *save_log_msg,
                                   const LogString *fail_log_msg, bool make_active) {
   if (!this->noise_pref_.save(&new_psk)) {
@@ -583,22 +584,19 @@ bool APIServer::update_noise_psk_(const SavedNoisePsk &new_psk, const LogString
 }
 
 bool APIServer::load_and_apply_noise_psk_() {
-  SavedNoisePsk saved{};
-  if (!this->noise_pref_.load(&saved))
+  // Load into a temp so a failed read cannot disturb the key in use
+  SavedNoisePsk loaded{};
+  if (!this->noise_pref_.load(&loaded))
     return false;
-  this->set_noise_psk(saved.psk);
+  this->saved_psk_ = loaded;
+  // An unprovisioned device stores the reserved all-zeros key, which is no key
+  const bool has_key = !noise::NoiseContext::is_all_zeros(this->saved_psk_.psk);
+  this->noise_ctx_.set_psk(has_key ? this->saved_psk_.psk.data() : nullptr);
   return true;
 }
 
 bool APIServer::save_noise_psk(noise::psk_t psk, bool make_active) {
-#ifdef USE_API_NOISE_PSK_FROM_YAML
-  // When PSK is set from YAML, this function should never be called
-  // but if it is, reject the change
-  ESP_LOGW(TAG, "Key set in YAML");
-  return false;
-#else
-  auto &old_psk = this->noise_ctx_.get_psk();
-  if (std::equal(old_psk.begin(), old_psk.end(), psk.begin())) {
+  if (this->saved_psk_.psk == psk) {
     ESP_LOGW(TAG, "New PSK matches old");
     return true;
   }
@@ -614,15 +612,8 @@ bool APIServer::save_noise_psk(noise::psk_t psk, bool make_active) {
   }
 #endif
   return result;
-#endif
 }
 bool APIServer::clear_noise_psk(bool make_active) {
-#ifdef USE_API_NOISE_PSK_FROM_YAML
-  // When PSK is set from YAML, this function should never be called
-  // but if it is, reject the change
-  ESP_LOGW(TAG, "Key set in YAML");
-  return false;
-#else
   SavedNoisePsk empty_psk{};
   bool result = this->update_noise_psk_(empty_psk, LOG_STR("Noise PSK cleared"), LOG_STR("Failed to clear Noise PSK"),
                                         make_active);
@@ -634,8 +625,8 @@ bool APIServer::clear_noise_psk(bool make_active) {
   }
 #endif
   return result;
-#endif
 }
+#endif  // USE_API_NOISE_PSK_FROM_YAML
 #endif
 
 #ifdef USE_HOMEASSISTANT_TIME

esphome/components/api/api_server.h

@@ -76,9 +76,14 @@ class APIServer final : public Component,
   APIBuffer &get_shared_buffer_ref() { return shared_write_buffer_; }
 
 #ifdef USE_API_NOISE
+#ifndef USE_API_NOISE_PSK_FROM_YAML
+  // Runtime key changes exist for the provisioning path only (not lambdas);
+  // with a yaml key they compile out
   bool save_noise_psk(noise::psk_t psk, bool make_active = true);
   bool clear_noise_psk(bool make_active = true);
-  void set_noise_psk(noise::psk_t psk) { this->noise_ctx_.set_psk(psk); }
+#endif
+  /// psk points at 32 bytes that live in flash for the life of the program
+  void set_noise_psk(const uint8_t *psk) { this->noise_ctx_.set_psk(psk); }
   noise::NoiseContext &get_noise_ctx() { return this->noise_ctx_; }
 #endif  // USE_API_NOISE
 
@@ -275,10 +280,12 @@ class APIServer final : public Component,
 #endif
 
 #ifdef USE_API_NOISE
+#ifndef USE_API_NOISE_PSK_FROM_YAML
   bool update_noise_psk_(const SavedNoisePsk &new_psk, const LogString *save_log_msg, const LogString *fail_log_msg,
                          bool make_active);
   // Load saved PSK from preferences and apply it. Returns true on success.
   bool load_and_apply_noise_psk_();
+#endif  // USE_API_NOISE_PSK_FROM_YAML
 #endif  // USE_API_NOISE
 #ifdef USE_API_HOMEASSISTANT_STATES
   // Helper methods to reduce code duplication
@@ -358,6 +365,9 @@ class APIServer final : public Component,
 
 #ifdef USE_API_NOISE
   noise::NoiseContext noise_ctx_;
+#ifndef USE_API_NOISE_PSK_FROM_YAML
+  SavedNoisePsk saved_psk_{};  // backs noise_ctx_ for a runtime provisioned key
+#endif
   ESPPreferenceObject noise_pref_;
 #endif  // USE_API_NOISE
 };

esphome/components/api/proto.h

@@ -287,19 +287,31 @@ class ProtoWriteBuffer {
   uint8_t *pos_;
 };
 
+// A four byte unaligned store is a memcpy call on ESP-IDF (-fno-builtin-memcpy) and on ARM cores without
+// unaligned access (Cortex-M0+, ARM9), so those targets share one outlined byte store helper per fixed32
+// field. Elsewhere the write inlines to a single store, or on ESP8266 to a few stores that measured
+// faster than a call, so it stays inline.
+#if defined(USE_ESP32) || (defined(__arm__) && !defined(__ARM_FEATURE_UNALIGNED))
+#define PROTO_OUTLINE_FOR_SIZE __attribute__((noinline))
+#define PROTO_FIXED32_BYTE_STORES true
+#else
+#define PROTO_OUTLINE_FOR_SIZE inline
+#define PROTO_FIXED32_BYTE_STORES false
+#endif
+
 // Varint encoding thresholds — used by both proto_encode_* free functions and ProtoSize.
 constexpr uint32_t VARINT_MAX_1_BYTE = 1 << 7;   // 128
 constexpr uint32_t VARINT_MAX_2_BYTE = 1 << 14;  // 16384
 
-/// Static encode helpers for generated encode() functions.
-/// Generated code hoists buffer.pos_ into a local uint8_t *__restrict__ pos,
-/// then calls these methods which take pos by reference. No struct, no overhead.
-/// For sub-messages, pos is synced back to buffer before the call and reloaded after.
+/// Static encode helpers for the generated encode bodies. Each takes the write cursor by value and
+/// returns it advanced, so outlined calls at -Os chain through the return register instead of a
+/// stack slot. Helpers without a _force suffix skip fields holding the proto3 default.
 class ProtoEncode {
  public:
   /// Write a multi-byte varint directly through a pos pointer.
   template<typename T>
-  static inline void encode_varint_raw_loop(uint8_t *__restrict__ &pos PROTO_ENCODE_DEBUG_PARAM, T value) {
+  [[nodiscard]] static inline uint8_t *encode_varint_raw_loop(uint8_t *__restrict__ pos PROTO_ENCODE_DEBUG_PARAM,
+                                                              T value) {
     do {
       PROTO_ENCODE_CHECK_BOUNDS(pos, 1);
       *pos++ = static_cast<uint8_t>(value | 0x80);
@@ -307,48 +319,49 @@ class ProtoEncode {
     } while (value > 0x7F);
     PROTO_ENCODE_CHECK_BOUNDS(pos, 1);
     *pos++ = static_cast<uint8_t>(value);
+    return pos;
   }
-  static inline void ESPHOME_ALWAYS_INLINE encode_varint_raw(uint8_t *__restrict__ &pos PROTO_ENCODE_DEBUG_PARAM,
-                                                             uint32_t value) {
+  [[nodiscard]] static inline uint8_t *ESPHOME_ALWAYS_INLINE
+  encode_varint_raw(uint8_t *__restrict__ pos PROTO_ENCODE_DEBUG_PARAM, uint32_t value) {
     if (value < VARINT_MAX_1_BYTE) [[likely]] {
       PROTO_ENCODE_CHECK_BOUNDS(pos, 1);
       *pos++ = static_cast<uint8_t>(value);
-      return;
+      return pos;
     }
-    encode_varint_raw_loop(pos PROTO_ENCODE_DEBUG_ARG, value);
+    return encode_varint_raw_loop(pos PROTO_ENCODE_DEBUG_ARG, value);
   }
   /// Encode a varint that is expected to be 1-2 bytes (e.g. zigzag RSSI, small lengths).
-  static inline void ESPHOME_ALWAYS_INLINE encode_varint_raw_short(uint8_t *__restrict__ &pos PROTO_ENCODE_DEBUG_PARAM,
-                                                                   uint32_t value) {
+  [[nodiscard]] static inline uint8_t *ESPHOME_ALWAYS_INLINE
+  encode_varint_raw_short(uint8_t *__restrict__ pos PROTO_ENCODE_DEBUG_PARAM, uint32_t value) {
     if (value < VARINT_MAX_1_BYTE) [[likely]] {
       PROTO_ENCODE_CHECK_BOUNDS(pos, 1);
       *pos++ = static_cast<uint8_t>(value);
-      return;
+      return pos;
     }
     if (value < VARINT_MAX_2_BYTE) [[likely]] {
       PROTO_ENCODE_CHECK_BOUNDS(pos, 2);
       *pos++ = static_cast<uint8_t>(value | 0x80);
       *pos++ = static_cast<uint8_t>(value >> 7);
-      return;
+      return pos;
     }
-    encode_varint_raw_loop(pos PROTO_ENCODE_DEBUG_ARG, value);
+    return encode_varint_raw_loop(pos PROTO_ENCODE_DEBUG_ARG, value);
   }
-  static inline void ESPHOME_ALWAYS_INLINE encode_varint_raw_64(uint8_t *__restrict__ &pos PROTO_ENCODE_DEBUG_PARAM,
-                                                                uint64_t value) {
+  [[nodiscard]] static inline uint8_t *ESPHOME_ALWAYS_INLINE
+  encode_varint_raw_64(uint8_t *__restrict__ pos PROTO_ENCODE_DEBUG_PARAM, uint64_t value) {
     if (value < VARINT_MAX_1_BYTE) [[likely]] {
       PROTO_ENCODE_CHECK_BOUNDS(pos, 1);
       *pos++ = static_cast<uint8_t>(value);
-      return;
+      return pos;
     }
-    encode_varint_raw_loop(pos PROTO_ENCODE_DEBUG_ARG, value);
+    return encode_varint_raw_loop(pos PROTO_ENCODE_DEBUG_ARG, value);
   }
   /// Encode a 48-bit MAC address (stored in a uint64) as varint.
   /// Real MAC addresses occupy the full 48 bits (OUI in upper 24), so the
   /// fast path -- any non-zero bit in the top 6 of 48 -- emits exactly 7 bytes
   /// with no per-byte branch. Falls back to the general loop otherwise.
   /// Caller must guarantee value fits in 48 bits (checked in debug builds).
-  static inline void ESPHOME_ALWAYS_INLINE encode_varint_raw_48bit(uint8_t *__restrict__ &pos PROTO_ENCODE_DEBUG_PARAM,
-                                                                   uint64_t value) {
+  [[nodiscard]] static inline uint8_t *ESPHOME_ALWAYS_INLINE
+  encode_varint_raw_48bit(uint8_t *__restrict__ pos PROTO_ENCODE_DEBUG_PARAM, uint64_t value) {
 #ifdef ESPHOME_DEBUG_API
     assert(value < (1ULL << (MAC_ADDRESS_SIZE * 8)) && "encode_varint_raw_48bit: value exceeds 48 bits");
 #endif
@@ -363,38 +376,39 @@ class ProtoEncode {
       pos[4] = static_cast<uint8_t>((value >> 28) | 0x80);
       pos[5] = static_cast<uint8_t>((value >> 35) | 0x80);
       pos[6] = static_cast<uint8_t>(value >> 42);
-      pos += 7;
-      return;
+      return pos + 7;
     }
-    encode_varint_raw_64(pos PROTO_ENCODE_DEBUG_ARG, value);
+    return encode_varint_raw_64(pos PROTO_ENCODE_DEBUG_ARG, value);
   }
-  static inline void ESPHOME_ALWAYS_INLINE encode_field_raw(uint8_t *__restrict__ &pos PROTO_ENCODE_DEBUG_PARAM,
-                                                            uint32_t field_id, uint32_t type) {
-    encode_varint_raw(pos PROTO_ENCODE_DEBUG_ARG, (field_id << 3) | type);
+  [[nodiscard]] static inline uint8_t *ESPHOME_ALWAYS_INLINE
+  encode_field_raw(uint8_t *__restrict__ pos PROTO_ENCODE_DEBUG_PARAM, uint32_t field_id, uint32_t type) {
+    return encode_varint_raw(pos PROTO_ENCODE_DEBUG_ARG, (field_id << 3) | type);
   }
   /// Write a single precomputed tag byte. Tag must be < 128.
-  static inline void ESPHOME_ALWAYS_INLINE write_raw_byte(uint8_t *__restrict__ &pos PROTO_ENCODE_DEBUG_PARAM,
-                                                          uint8_t b) {
+  [[nodiscard]] static inline uint8_t *ESPHOME_ALWAYS_INLINE
+  write_raw_byte(uint8_t *__restrict__ pos PROTO_ENCODE_DEBUG_PARAM, uint8_t b) {
     PROTO_ENCODE_CHECK_BOUNDS(pos, 1);
     *pos++ = b;
+    return pos;
   }
   /// Reserve one byte for later backpatch (e.g., sub-message length).
   /// Advances pos past the reserved byte without writing a value.
-  static inline void ESPHOME_ALWAYS_INLINE reserve_byte(uint8_t *__restrict__ &pos PROTO_ENCODE_DEBUG_PARAM) {
+  [[nodiscard]] static inline uint8_t *ESPHOME_ALWAYS_INLINE
+  reserve_byte(uint8_t *__restrict__ pos PROTO_ENCODE_DEBUG_PARAM) {
     PROTO_ENCODE_CHECK_BOUNDS(pos, 1);
-    pos++;
+    return pos + 1;
   }
   /// Write raw bytes to the buffer (no tag, no length prefix).
-  static inline void ESPHOME_ALWAYS_INLINE encode_raw(uint8_t *__restrict__ &pos PROTO_ENCODE_DEBUG_PARAM,
-                                                      const void *data, size_t len) {
+  [[nodiscard]] static inline uint8_t *ESPHOME_ALWAYS_INLINE
+  encode_raw(uint8_t *__restrict__ pos PROTO_ENCODE_DEBUG_PARAM, const void *data, size_t len) {
     PROTO_ENCODE_CHECK_BOUNDS(pos, len);
     std::memcpy(pos, data, len);
-    pos += len;
+    return pos + len;
   }
   /// Encode tag + 1-byte length + raw string data. For strings with max_data_length < 128.
   /// Tag must be a single-byte varint (< 128). Always encodes (no zero check).
-  static inline void encode_short_string_force(uint8_t *__restrict__ &pos PROTO_ENCODE_DEBUG_PARAM, uint8_t tag,
-                                               const StringRef &ref) {
+  [[nodiscard]] static inline uint8_t *encode_short_string_force(uint8_t *__restrict__ pos PROTO_ENCODE_DEBUG_PARAM,
+                                                                 uint8_t tag, const StringRef &ref) {
 #ifdef ESPHOME_DEBUG_API
     assert(ref.size() < 128 && "encode_short_string_force: string exceeds max_data_length < 128");
 #endif
@@ -402,137 +416,187 @@ class ProtoEncode {
     pos[0] = tag;
     pos[1] = static_cast<uint8_t>(ref.size());
     std::memcpy(pos + 2, ref.c_str(), ref.size());
-    pos += 2 + ref.size();
+    return pos + 2 + ref.size();
   }
-  /// Write a precomputed tag byte + 32-bit value in one operation.
-  static inline void ESPHOME_ALWAYS_INLINE write_tag_and_fixed32(uint8_t *__restrict__ &pos PROTO_ENCODE_DEBUG_PARAM,
-                                                                 uint8_t tag, uint32_t value) {
+  /// Unaligned little-endian store; __builtin_memcpy stays inline even under -fno-builtin-memcpy.
+  static inline void ESPHOME_ALWAYS_INLINE write_fixed32_le(uint8_t *__restrict__ pos, uint32_t value) {
+    if constexpr (PROTO_FIXED32_BYTE_STORES) {
+      // Spelled out so the outlined helper does not itself become a memcpy call
+      pos[0] = static_cast<uint8_t>(value);
+      pos[1] = static_cast<uint8_t>(value >> 8);
+      pos[2] = static_cast<uint8_t>(value >> 16);
+      pos[3] = static_cast<uint8_t>(value >> 24);
+    } else {
+      const uint32_t le = convert_little_endian(value);
+      __builtin_memcpy(pos, &le, 4);
+    }
+  }
+  /// Write a precomputed tag byte + 32-bit value. Outlined on embedded: one copy beats inline stores per field.
+  [[nodiscard]] static PROTO_OUTLINE_FOR_SIZE uint8_t *write_tag_and_fixed32(
+      uint8_t *__restrict__ pos PROTO_ENCODE_DEBUG_PARAM, uint8_t tag, uint32_t value) {
     PROTO_ENCODE_CHECK_BOUNDS(pos, 5);
     pos[0] = tag;
-#if __BYTE_ORDER__ == __ORDER_LITTLE_ENDIAN__
-    std::memcpy(pos + 1, &value, 4);
-#else
-    pos[1] = static_cast<uint8_t>(value & 0xFF);
-    pos[2] = static_cast<uint8_t>((value >> 8) & 0xFF);
-    pos[3] = static_cast<uint8_t>((value >> 16) & 0xFF);
-    pos[4] = static_cast<uint8_t>((value >> 24) & 0xFF);
-#endif
-    pos += 5;
+    write_fixed32_le(pos + 1, value);
+    return pos + 5;
   }
-  static inline void encode_string(uint8_t *__restrict__ &pos PROTO_ENCODE_DEBUG_PARAM, uint32_t field_id,
-                                   const char *string, size_t len, bool force = false) {
-    if (len == 0 && !force)
-      return;
-    encode_field_raw(pos PROTO_ENCODE_DEBUG_ARG, field_id, 2);  // type 2: Length-delimited string
+  [[nodiscard]] static inline uint8_t *encode_string_force(uint8_t *__restrict__ pos PROTO_ENCODE_DEBUG_PARAM,
+                                                           uint32_t field_id, const char *string, size_t len) {
+    pos = encode_field_raw(pos PROTO_ENCODE_DEBUG_ARG, field_id, 2);  // type 2: Length-delimited string
     // NOLINTNEXTLINE(readability-inconsistent-ifelse-braces) -- false positive on [[likely]] attribute
     if (len < VARINT_MAX_1_BYTE) [[likely]] {
       PROTO_ENCODE_CHECK_BOUNDS(pos, 1 + len);
       *pos++ = static_cast<uint8_t>(len);
     } else {
-      encode_varint_raw_loop(pos PROTO_ENCODE_DEBUG_ARG, len);
+      pos = encode_varint_raw_loop(pos PROTO_ENCODE_DEBUG_ARG, len);
       PROTO_ENCODE_CHECK_BOUNDS(pos, len);
     }
     std::memcpy(pos, string, len);
-    pos += len;
+    return pos + len;
   }
-  static inline void encode_string(uint8_t *__restrict__ &pos PROTO_ENCODE_DEBUG_PARAM, uint32_t field_id,
-                                   const std::string &value, bool force = false) {
-    encode_string(pos PROTO_ENCODE_DEBUG_ARG, field_id, value.data(), value.size(), force);
+  [[nodiscard]] static inline uint8_t *encode_string(uint8_t *__restrict__ pos PROTO_ENCODE_DEBUG_PARAM,
+                                                     uint32_t field_id, const char *string, size_t len) {
+    if (len == 0)
+      return pos;
+    return encode_string_force(pos PROTO_ENCODE_DEBUG_ARG, field_id, string, len);
   }
-  static inline void encode_string(uint8_t *__restrict__ &pos PROTO_ENCODE_DEBUG_PARAM, uint32_t field_id,
-                                   const StringRef &ref, bool force = false) {
-    encode_string(pos PROTO_ENCODE_DEBUG_ARG, field_id, ref.c_str(), ref.size(), force);
+  [[nodiscard]] static inline uint8_t *encode_string_force(uint8_t *__restrict__ pos PROTO_ENCODE_DEBUG_PARAM,
+                                                           uint32_t field_id, const std::string &value) {
+    return encode_string_force(pos PROTO_ENCODE_DEBUG_ARG, field_id, value.data(), value.size());
   }
-  static inline void encode_bytes(uint8_t *__restrict__ &pos PROTO_ENCODE_DEBUG_PARAM, uint32_t field_id,
-                                  const uint8_t *data, size_t len, bool force = false) {
-    encode_string(pos PROTO_ENCODE_DEBUG_ARG, field_id, reinterpret_cast<const char *>(data), len, force);
+  [[nodiscard]] static inline uint8_t *encode_string(uint8_t *__restrict__ pos PROTO_ENCODE_DEBUG_PARAM,
+                                                     uint32_t field_id, const StringRef &ref) {
+    return encode_string(pos PROTO_ENCODE_DEBUG_ARG, field_id, ref.c_str(), ref.size());
   }
-  static inline void encode_uint32(uint8_t *__restrict__ &pos PROTO_ENCODE_DEBUG_PARAM, uint32_t field_id,
-                                   uint32_t value, bool force = false) {
-    if (value == 0 && !force)
-      return;
-    encode_field_raw(pos PROTO_ENCODE_DEBUG_ARG, field_id, 0);
-    encode_varint_raw(pos PROTO_ENCODE_DEBUG_ARG, value);
+  [[nodiscard]] static inline uint8_t *encode_string_force(uint8_t *__restrict__ pos PROTO_ENCODE_DEBUG_PARAM,
+                                                           uint32_t field_id, const StringRef &ref) {
+    return encode_string_force(pos PROTO_ENCODE_DEBUG_ARG, field_id, ref.c_str(), ref.size());
   }
-  static inline void encode_uint64(uint8_t *__restrict__ &pos PROTO_ENCODE_DEBUG_PARAM, uint32_t field_id,
-                                   uint64_t value, bool force = false) {
-    if (value == 0 && !force)
-      return;
-    encode_field_raw(pos PROTO_ENCODE_DEBUG_ARG, field_id, 0);
-    encode_varint_raw_64(pos PROTO_ENCODE_DEBUG_ARG, value);
+  [[nodiscard]] static inline uint8_t *encode_bytes(uint8_t *__restrict__ pos PROTO_ENCODE_DEBUG_PARAM,
+                                                    uint32_t field_id, const uint8_t *data, size_t len) {
+    return encode_string(pos PROTO_ENCODE_DEBUG_ARG, field_id, reinterpret_cast<const char *>(data), len);
   }
-  static inline void encode_bool(uint8_t *__restrict__ &pos PROTO_ENCODE_DEBUG_PARAM, uint32_t field_id, bool value,
-                                 bool force = false) {
-    if (!value && !force)
-      return;
-    encode_field_raw(pos PROTO_ENCODE_DEBUG_ARG, field_id, 0);
+  [[nodiscard]] static inline uint8_t *encode_bytes_force(uint8_t *__restrict__ pos PROTO_ENCODE_DEBUG_PARAM,
+                                                          uint32_t field_id, const uint8_t *data, size_t len) {
+    return encode_string_force(pos PROTO_ENCODE_DEBUG_ARG, field_id, reinterpret_cast<const char *>(data), len);
+  }
+  [[nodiscard]] static inline uint8_t *encode_uint32_force(uint8_t *__restrict__ pos PROTO_ENCODE_DEBUG_PARAM,
+                                                           uint32_t field_id, uint32_t value) {
+    pos = encode_field_raw(pos PROTO_ENCODE_DEBUG_ARG, field_id, 0);
+    return encode_varint_raw(pos PROTO_ENCODE_DEBUG_ARG, value);
+  }
+  [[nodiscard]] static inline uint8_t *encode_uint32(uint8_t *__restrict__ pos PROTO_ENCODE_DEBUG_PARAM,
+                                                     uint32_t field_id, uint32_t value) {
+    if (value == 0)
+      return pos;
+    return encode_uint32_force(pos PROTO_ENCODE_DEBUG_ARG, field_id, value);
+  }
+  [[nodiscard]] static inline uint8_t *encode_uint64_force(uint8_t *__restrict__ pos PROTO_ENCODE_DEBUG_PARAM,
+                                                           uint32_t field_id, uint64_t value) {
+    pos = encode_field_raw(pos PROTO_ENCODE_DEBUG_ARG, field_id, 0);
+    return encode_varint_raw_64(pos PROTO_ENCODE_DEBUG_ARG, value);
+  }
+  [[nodiscard]] static inline uint8_t *encode_uint64(uint8_t *__restrict__ pos PROTO_ENCODE_DEBUG_PARAM,
+                                                     uint32_t field_id, uint64_t value) {
+    if (value == 0)
+      return pos;
+    return encode_uint64_force(pos PROTO_ENCODE_DEBUG_ARG, field_id, value);
+  }
+  [[nodiscard]] static inline uint8_t *encode_bool_force(uint8_t *__restrict__ pos PROTO_ENCODE_DEBUG_PARAM,
+                                                         uint32_t field_id, bool value) {
+    pos = encode_field_raw(pos PROTO_ENCODE_DEBUG_ARG, field_id, 0);
     PROTO_ENCODE_CHECK_BOUNDS(pos, 1);
     *pos++ = value ? 0x01 : 0x00;
+    return pos;
   }
-  static inline void encode_fixed32(uint8_t *__restrict__ &pos PROTO_ENCODE_DEBUG_PARAM, uint32_t field_id,
-                                    uint32_t value, bool force = false) {
-    if (value == 0 && !force)
-      return;
-    encode_field_raw(pos PROTO_ENCODE_DEBUG_ARG, field_id, 5);
+  [[nodiscard]] static inline uint8_t *encode_bool(uint8_t *__restrict__ pos PROTO_ENCODE_DEBUG_PARAM,
+                                                   uint32_t field_id, bool value) {
+    if (!value)
+      return pos;
+    return encode_bool_force(pos PROTO_ENCODE_DEBUG_ARG, field_id, value);
+  }
+  /// Tag + fixed32 for multi-byte tags; single-byte tags use write_tag_and_fixed32.
+  [[nodiscard]] static PROTO_OUTLINE_FOR_SIZE uint8_t *encode_fixed32_force(
+      uint8_t *__restrict__ pos PROTO_ENCODE_DEBUG_PARAM, uint32_t field_id, uint32_t value) {
+    pos = encode_field_raw(pos PROTO_ENCODE_DEBUG_ARG, field_id, 5);
     PROTO_ENCODE_CHECK_BOUNDS(pos, 4);
-#if __BYTE_ORDER__ == __ORDER_LITTLE_ENDIAN__
-    std::memcpy(pos, &value, 4);
-    pos += 4;
-#else
-    *pos++ = (value >> 0) & 0xFF;
-    *pos++ = (value >> 8) & 0xFF;
-    *pos++ = (value >> 16) & 0xFF;
-    *pos++ = (value >> 24) & 0xFF;
-#endif
+    write_fixed32_le(pos, value);
+    return pos + 4;
+  }
+  [[nodiscard]] static inline uint8_t *encode_fixed32(uint8_t *__restrict__ pos PROTO_ENCODE_DEBUG_PARAM,
+                                                      uint32_t field_id, uint32_t value) {
+    if (value == 0)
+      return pos;
+    return encode_fixed32_force(pos PROTO_ENCODE_DEBUG_ARG, field_id, value);
   }
   // NOTE: Wire type 1 (64-bit fixed: double, fixed64, sfixed64) is intentionally
   // not supported to reduce overhead on embedded systems. All ESPHome devices are
   // 32-bit microcontrollers where 64-bit operations are expensive. If 64-bit support
   // is needed in the future, the necessary encoding/decoding functions must be added.
-  static inline void encode_float(uint8_t *__restrict__ &pos PROTO_ENCODE_DEBUG_PARAM, uint32_t field_id, float value,
-                                  bool force = false) {
-    uint32_t raw = float_to_raw(value);
-    if (raw == 0 && !force)
-      return;
-    encode_fixed32(pos PROTO_ENCODE_DEBUG_ARG, field_id, raw);
+  [[nodiscard]] static inline uint8_t *encode_float(uint8_t *__restrict__ pos PROTO_ENCODE_DEBUG_PARAM,
+                                                    uint32_t field_id, float value) {
+    return encode_fixed32(pos PROTO_ENCODE_DEBUG_ARG, field_id, float_to_raw(value));
   }
-  static inline void encode_int32(uint8_t *__restrict__ &pos PROTO_ENCODE_DEBUG_PARAM, uint32_t field_id, int32_t value,
-                                  bool force = false) {
+  [[nodiscard]] static inline uint8_t *encode_float_force(uint8_t *__restrict__ pos PROTO_ENCODE_DEBUG_PARAM,
+                                                          uint32_t field_id, float value) {
+    return encode_fixed32_force(pos PROTO_ENCODE_DEBUG_ARG, field_id, float_to_raw(value));
+  }
+  [[nodiscard]] static inline uint8_t *encode_int32_force(uint8_t *__restrict__ pos PROTO_ENCODE_DEBUG_PARAM,
+                                                          uint32_t field_id, int32_t value) {
     if (value < 0) {
       // negative int32 is always 10 byte long
-      encode_uint64(pos PROTO_ENCODE_DEBUG_ARG, field_id, static_cast<uint64_t>(value), force);
-      return;
+      return encode_uint64_force(pos PROTO_ENCODE_DEBUG_ARG, field_id, static_cast<uint64_t>(value));
     }
-    encode_uint32(pos PROTO_ENCODE_DEBUG_ARG, field_id, static_cast<uint32_t>(value), force);
+    return encode_uint32_force(pos PROTO_ENCODE_DEBUG_ARG, field_id, static_cast<uint32_t>(value));
   }
-  static inline void encode_int64(uint8_t *__restrict__ &pos PROTO_ENCODE_DEBUG_PARAM, uint32_t field_id, int64_t value,
-                                  bool force = false) {
-    encode_uint64(pos PROTO_ENCODE_DEBUG_ARG, field_id, static_cast<uint64_t>(value), force);
+  [[nodiscard]] static inline uint8_t *encode_int32(uint8_t *__restrict__ pos PROTO_ENCODE_DEBUG_PARAM,
+                                                    uint32_t field_id, int32_t value) {
+    if (value == 0)
+      return pos;
+    return encode_int32_force(pos PROTO_ENCODE_DEBUG_ARG, field_id, value);
   }
-  static inline void encode_sint32(uint8_t *__restrict__ &pos PROTO_ENCODE_DEBUG_PARAM, uint32_t field_id,
-                                   int32_t value, bool force = false) {
-    encode_uint32(pos PROTO_ENCODE_DEBUG_ARG, field_id, encode_zigzag32(value), force);
+  [[nodiscard]] static inline uint8_t *encode_int64(uint8_t *__restrict__ pos PROTO_ENCODE_DEBUG_PARAM,
+                                                    uint32_t field_id, int64_t value) {
+    return encode_uint64(pos PROTO_ENCODE_DEBUG_ARG, field_id, static_cast<uint64_t>(value));
   }
-  static inline void encode_sint64(uint8_t *__restrict__ &pos PROTO_ENCODE_DEBUG_PARAM, uint32_t field_id,
-                                   int64_t value, bool force = false) {
-    encode_uint64(pos PROTO_ENCODE_DEBUG_ARG, field_id, encode_zigzag64(value), force);
+  [[nodiscard]] static inline uint8_t *encode_int64_force(uint8_t *__restrict__ pos PROTO_ENCODE_DEBUG_PARAM,
+                                                          uint32_t field_id, int64_t value) {
+    return encode_uint64_force(pos PROTO_ENCODE_DEBUG_ARG, field_id, static_cast<uint64_t>(value));
   }
-  /// Sub-message encoding: sync pos to buffer, delegate, get pos from return value.
+  [[nodiscard]] static inline uint8_t *encode_sint32(uint8_t *__restrict__ pos PROTO_ENCODE_DEBUG_PARAM,
+                                                     uint32_t field_id, int32_t value) {
+    return encode_uint32(pos PROTO_ENCODE_DEBUG_ARG, field_id, encode_zigzag32(value));
+  }
+  [[nodiscard]] static inline uint8_t *encode_sint32_force(uint8_t *__restrict__ pos PROTO_ENCODE_DEBUG_PARAM,
+                                                           uint32_t field_id, int32_t value) {
+    return encode_uint32_force(pos PROTO_ENCODE_DEBUG_ARG, field_id, encode_zigzag32(value));
+  }
+  [[nodiscard]] static inline uint8_t *encode_sint64(uint8_t *__restrict__ pos PROTO_ENCODE_DEBUG_PARAM,
+                                                     uint32_t field_id, int64_t value) {
+    return encode_uint64(pos PROTO_ENCODE_DEBUG_ARG, field_id, encode_zigzag64(value));
+  }
+  [[nodiscard]] static inline uint8_t *encode_sint64_force(uint8_t *__restrict__ pos PROTO_ENCODE_DEBUG_PARAM,
+                                                           uint32_t field_id, int64_t value) {
+    return encode_uint64_force(pos PROTO_ENCODE_DEBUG_ARG, field_id, encode_zigzag64(value));
+  }
+  /// Sub-message encoding: sync pos to buffer, delegate, read the cursor back.
   template<typename T>
-  static inline void encode_sub_message(uint8_t *__restrict__ &pos PROTO_ENCODE_DEBUG_PARAM, ProtoWriteBuffer &buffer,
-                                        uint32_t field_id, const T &value) {
+  [[nodiscard]] static inline uint8_t *encode_sub_message(uint8_t *__restrict__ pos PROTO_ENCODE_DEBUG_PARAM,
+                                                          ProtoWriteBuffer &buffer, uint32_t field_id, const T &value) {
     buffer.set_pos(pos);
     buffer.encode_sub_message(field_id, value);
-    pos = buffer.get_pos();
+    return buffer.get_pos();
   }
   template<typename T>
-  static inline void encode_optional_sub_message(uint8_t *__restrict__ &pos PROTO_ENCODE_DEBUG_PARAM,
-                                                 ProtoWriteBuffer &buffer, uint32_t field_id, const T &value) {
+  [[nodiscard]] static inline uint8_t *encode_optional_sub_message(uint8_t *__restrict__ pos PROTO_ENCODE_DEBUG_PARAM,
+                                                                   ProtoWriteBuffer &buffer, uint32_t field_id,
+                                                                   const T &value) {
     buffer.set_pos(pos);
     buffer.encode_optional_sub_message(field_id, value);
-    pos = buffer.get_pos();
+    return buffer.get_pos();
   }
 };
+#undef PROTO_OUTLINE_FOR_SIZE
+#undef PROTO_FIXED32_BYTE_STORES
 
 #ifdef HAS_PROTO_MESSAGE_DUMP
 /**

esphome/components/climate/__init__.py

@@ -125,6 +125,19 @@ CLIMATE_SWING_MODES = {
 
 validate_climate_swing_mode = cv.enum(CLIMATE_SWING_MODES, upper=True)
 
+ClimateAction = climate_ns.enum("ClimateAction")
+CLIMATE_ACTIONS = {
+    "OFF": ClimateAction.CLIMATE_ACTION_OFF,
+    "COOLING": ClimateAction.CLIMATE_ACTION_COOLING,
+    "HEATING": ClimateAction.CLIMATE_ACTION_HEATING,
+    "IDLE": ClimateAction.CLIMATE_ACTION_IDLE,
+    "DRYING": ClimateAction.CLIMATE_ACTION_DRYING,
+    "FAN": ClimateAction.CLIMATE_ACTION_FAN,
+    "DEFROSTING": ClimateAction.CLIMATE_ACTION_DEFROSTING,
+}
+
+validate_climate_action = cv.enum(CLIMATE_ACTIONS, upper=True)
+
 CONF_MIN_HUMIDITY = "min_humidity"
 CONF_MAX_HUMIDITY = "max_humidity"
 CONF_TARGET_HUMIDITY = "target_humidity"

esphome/components/esp32_ble/ble.cpp

@@ -100,21 +100,38 @@ void ESP32BLE::disable() {
 #ifdef USE_ESP32_BLE_ADVERTISING
 void ESP32BLE::advertising_start() {
   this->advertising_init_();
-  if (!this->is_active())
+  this->advertising_ref_count_++;
+  this->advertising_refresh();
+}
+
+void ESP32BLE::advertising_stop() {
+  if (this->advertising_ref_count_ == 0)
     return;
-  this->advertising_->start();
+  this->advertising_ref_count_--;
+  this->advertising_refresh();
+}
+
+void ESP32BLE::advertising_refresh() {
+  if (this->advertising_ == nullptr || !this->is_active())
+    return;
+  // Advertise while any component still needs it, otherwise stop
+  if (this->advertising_ref_count_ == 0) {
+    this->advertising_->stop();
+  } else {
+    this->advertising_->start();
+  }
 }
 
 void ESP32BLE::advertising_set_service_data(const std::vector<uint8_t> &data) {
   this->advertising_init_();
   this->advertising_->set_service_data(data);
-  this->advertising_start();
+  this->advertising_refresh();
 }
 
 void ESP32BLE::advertising_set_manufacturer_data(const std::vector<uint8_t> &data) {
   this->advertising_init_();
   this->advertising_->set_manufacturer_data(data);
-  this->advertising_start();
+  this->advertising_refresh();
 }
 
 void ESP32BLE::advertising_set_service_data_and_name(std::span<const uint8_t> data, bool include_name) {
@@ -136,7 +153,7 @@ void ESP32BLE::advertising_set_service_data_and_name(std::span<const uint8_t> da
     this->advertising_->set_service_data(data);
   }
 
-  this->advertising_start();
+  this->advertising_refresh();
 }
 
 void ESP32BLE::advertising_register_raw_advertisement_callback(std::function<void(bool)> &&callback) {
@@ -147,13 +164,13 @@ void ESP32BLE::advertising_register_raw_advertisement_callback(std::function<voi
 void ESP32BLE::advertising_add_service_uuid(ESPBTUUID uuid) {
   this->advertising_init_();
   this->advertising_->add_service_uuid(uuid);
-  this->advertising_start();
+  this->advertising_refresh();
 }
 
 void ESP32BLE::advertising_remove_service_uuid(ESPBTUUID uuid) {
   this->advertising_init_();
   this->advertising_->remove_service_uuid(uuid);
-  this->advertising_start();
+  this->advertising_refresh();
 }
 #endif
 
@@ -575,6 +592,10 @@ void ESP32BLE::loop_handle_state_transition_not_active_() {
     }
 
     this->state_ = BLE_COMPONENT_STATE_ACTIVE;
+#ifdef USE_ESP32_BLE_ADVERTISING
+    // Requests made before the stack was up (or before it was re-enabled) take effect now
+    this->advertising_refresh();
+#endif
   }
 }
 

esphome/components/esp32_ble/ble.h

@@ -114,7 +114,17 @@ class ESP32BLE final : public Component {
   void set_name(const char *name) { this->name_ = name; }
 
 #ifdef USE_ESP32_BLE_ADVERTISING
+  /** Request advertising on behalf of a component.
+   *
+   * Requests are reference counted: advertising runs until every component that called
+   * advertising_start() has released it again with advertising_stop(). Each component must
+   * pair its calls, so nothing advertises until something actually asks for it.
+   */
   void advertising_start();
+  /// Release a request made with advertising_start(); advertising stops at the last release.
+  void advertising_stop();
+  /// Apply the current payload and request count: advertise while requested, otherwise stop.
+  void advertising_refresh();
   void advertising_set_service_data(const std::vector<uint8_t> &data);
   void advertising_set_manufacturer_data(const std::vector<uint8_t> &data);
   void advertising_set_appearance(uint16_t appearance) { this->appearance_ = appearance; }
@@ -226,6 +236,9 @@ class ESP32BLE final : public Component {
   // 1-byte aligned members (grouped together to minimize padding)
   BLEComponentState state_{BLE_COMPONENT_STATE_OFF};  // 1 byte (uint8_t enum)
   bool enable_on_boot_{};                             // 1 byte
+#ifdef USE_ESP32_BLE_ADVERTISING
+  uint8_t advertising_ref_count_{0};  // 1 byte, number of components requesting advertising
+#endif
 
 #ifdef ESPHOME_ESP32_BLE_EXTENDED_AUTH_PARAMS
   optional<esp_ble_auth_req_t> auth_req_mode_;

esphome/components/esp32_ble_beacon/esp32_ble_beacon.cpp

@@ -67,6 +67,8 @@ void ESP32BLEBeacon::setup() {
       this->on_advertise_();
     }
   });
+  // A beacon always needs the device to advertise, and never releases the request
+  global_ble->advertising_start();
 }
 
 void ESP32BLEBeacon::on_advertise_() {

esphome/components/esp32_ble_server/__init__.py

@@ -596,6 +596,18 @@ async def to_code(config):
     cg.add(var.set_parent(parent))
     cg.add(parent.advertising_set_appearance(config[CONF_APPEARANCE]))
     cg.add(var.set_max_clients(config[CONF_MAX_CLIENTS]))
+    # Only advertise for the server itself when the configuration gives clients something to
+    # find. A server that is auto-loaded purely to host a runtime service (esp32_improv) stays
+    # silent until that service asks for advertising.
+    cg.add(
+        var.set_advertising_required(
+            CONF_MANUFACTURER_DATA in config
+            or any(
+                not uuid_is(service_config[CONF_UUID], DEVICE_INFORMATION_SERVICE_UUID)
+                for service_config in config[CONF_SERVICES]
+            )
+        )
+    )
     if CONF_MANUFACTURER_DATA in config:
         cg.add(var.set_manufacturer_data(config[CONF_MANUFACTURER_DATA]))
     for service_config in config[CONF_SERVICES]:

esphome/components/esp32_ble_server/ble_server.cpp

@@ -81,6 +81,7 @@ void BLEServer::loop() {
       if (this->device_information_service_->is_running()) {
         this->state_ = RUNNING;
         this->restart_advertising_();
+        this->request_advertising_();
         ESP_LOGD(TAG, "BLE server setup successfully");
       } else if (this->device_information_service_->is_created()) {
         this->device_information_service_->start();
@@ -98,6 +99,20 @@ void BLEServer::restart_advertising_() {
   }
 }
 
+void BLEServer::request_advertising_() {
+  if (!this->advertising_required_ || this->advertising_requested_)
+    return;
+  this->advertising_requested_ = true;
+  this->parent_->advertising_start();
+}
+
+void BLEServer::release_advertising_() {
+  if (!this->advertising_requested_)
+    return;
+  this->advertising_requested_ = false;
+  this->parent_->advertising_stop();
+}
+
 BLEService *BLEServer::create_service(ESPBTUUID uuid, bool advertise, uint16_t num_handles) {
 #if ESPHOME_LOG_LEVEL >= ESPHOME_LOG_LEVEL_VERBOSE
   char uuid_buf[esp32_ble::UUID_STR_LEN];
@@ -170,7 +185,7 @@ void BLEServer::gatts_event_handler(esp_gatts_cb_event_t event, esp_gatt_if_t ga
       this->add_client_(param->connect.conn_id);
       // Resume advertising so additional clients can discover and connect
       if (this->client_count_ < this->max_clients_) {
-        this->parent_->advertising_start();
+        this->parent_->advertising_refresh();
       }
       this->dispatch_callbacks_(CallbackType::ON_CONNECT, param->connect.conn_id);
       break;
@@ -178,7 +193,7 @@ void BLEServer::gatts_event_handler(esp_gatts_cb_event_t event, esp_gatt_if_t ga
     case ESP_GATTS_DISCONNECT_EVT: {
       ESP_LOGD(TAG, "BLE Client disconnected");
       this->remove_client_(param->disconnect.conn_id);
-      this->parent_->advertising_start();
+      this->parent_->advertising_refresh();
       this->dispatch_callbacks_(CallbackType::ON_DISCONNECT, param->disconnect.conn_id);
       break;
     }
@@ -226,6 +241,8 @@ void BLEServer::remove_client_(uint16_t conn_id) {
 }
 
 void BLEServer::ble_before_disabled_event_handler() {
+  // Advertising is re-requested once the server is running again after BLE is re-enabled
+  this->release_advertising_();
   // Delete all clients
   this->client_count_ = 0;
   // Delete all services

esphome/components/esp32_ble_server/ble_server.h

@@ -38,6 +38,13 @@ class BLEServer final : public Component, public Parented<ESP32BLE> {
     this->restart_advertising_();
   }
 
+  /** Whether this server needs the device to advertise so clients can find and connect to it.
+   *
+   * False for a server that only hosts services created at runtime (e.g. esp32_improv), which
+   * request advertising themselves for as long as they need it.
+   */
+  void set_advertising_required(bool required) { this->advertising_required_ = required; }
+
   void set_max_clients(uint8_t max_clients) { this->max_clients_ = max_clients; }
   uint8_t get_max_clients() const { return this->max_clients_; }
 
@@ -82,6 +89,8 @@ class BLEServer final : public Component, public Parented<ESP32BLE> {
   };
 
   void restart_advertising_();
+  void request_advertising_();
+  void release_advertising_();
 
   int8_t find_client_index_(uint16_t conn_id) const;
   void add_client_(uint16_t conn_id);
@@ -93,6 +102,8 @@ class BLEServer final : public Component, public Parented<ESP32BLE> {
   std::vector<uint8_t> manufacturer_data_{};
   esp_gatt_if_t gatts_if_{0};
   bool registered_{false};
+  bool advertising_required_{true};
+  bool advertising_requested_{false};
 
   uint16_t clients_[USE_ESP32_BLE_MAX_CONNECTIONS]{};
   uint8_t client_count_{0};

esphome/components/esp32_hosted/__init__.py

@@ -37,6 +37,25 @@ CONF_HANDSHAKE_PIN = "handshake_pin"
 CONF_SDIO_FREQUENCY = "sdio_frequency"
 CONF_SPI_MODE = "spi_mode"
 
+# ESP-NOW-over-hosted shim (esp_now_hosted.cpp). esp-hosted proxies esp_wifi.h
+# but not esp_now.h (espressif/esp-hosted-mcu#19), and esp_wifi_remote injects
+# the esp_now.h header on the ESP32-P4 host with no implementation, leaving the
+# esp_now_* symbols undefined at link. On a P4 host, esp_now_hosted.cpp DEFINES
+# those symbols and forwards each call to the co-processor over esp-hosted's
+# CustomRpc "peer data transfer" channel, so ESPHome's `espnow` component links
+# and runs unchanged (proven on a Tab5, 2026-07-20). The .cpp is guarded to
+# CONFIG_IDF_TARGET_ESP32P4 so it compiles to nothing on hosts with a native
+# ESP-NOW stack. CustomRpc needs these two host-side Kconfig options. Host
+# registers 3 handlers (RESP, RECV, SEND); the coprocessor registers 1 (REQ);
+# we ask for 8 to leave room for other CustomRpc extensions alongside.
+#
+# The coprocessor must run the matching custom firmware (a parallel effort in
+# esphome/esp-hosted-firmware). esp_now_hosted_rpc.h here is the canonical copy
+# of the wire contract and MUST stay byte-identical to the copy that coprocessor
+# firmware uses — the packed structs are the on-wire layout, so any divergence
+# silently corrupts every ESP-NOW frame.
+_MAX_CUSTOM_MSG_HANDLERS = 8
+
 # Shared fields for both transport modes
 BASE_SCHEMA = cv.Schema(
     {
@@ -262,6 +281,23 @@ async def to_code(config: ConfigType) -> None:
     else:
         _configure_spi(config)
 
+    # ESP-NOW-over-hosted shim: only the radio-less ESP32-P4 host needs it (see
+    # the note by _MAX_CUSTOM_MSG_HANDLERS). Enabled for every P4 host, not
+    # gated on the `espnow` component being present: the shim is tiny and the
+    # esp_now_* symbols/CustomRpc calls it defines require these Kconfig options
+    # to link whenever esp_now_hosted.cpp compiles (which is on any P4 host), so
+    # coupling the two keeps the build consistent. When `espnow` is absent the
+    # symbols are simply unused and never register a callback at runtime.
+    if esp32.get_esp32_variant() == esp32.VARIANT_ESP32P4:
+        add_define("USE_ESP_NOW_HOSTED")
+        # esp-hosted's CustomRpc ("peer data transfer") path — off by default.
+        esp32.add_idf_sdkconfig_option(
+            "CONFIG_ESP_HOSTED_ENABLE_PEER_DATA_TRANSFER", True
+        )
+        esp32.add_idf_sdkconfig_option(
+            "CONFIG_ESP_HOSTED_MAX_CUSTOM_MSG_HANDLERS", _MAX_CUSTOM_MSG_HANDLERS
+        )
+
     # Place the transport mempool in PSRAM. Required on memory-tight host
     # configurations (e.g. P4 with a large LVGL UI) where the internal-RAM
     # mempool allocation fails at boot with `sdio_mempool_create` assert.

esphome/components/esp32_hosted/esp_now_hosted.cpp

@@ -0,0 +1,467 @@
+/*
+ * esp_now_hosted — host-side shim implementing <esp_now.h> over esp-hosted
+ * CustomRpc, so ESPHome's `espnow` component can run on a radio-less host
+ * (e.g. the ESP32-P4) whose radio lives on an esp-hosted co-processor.
+ *
+ * A radio-less host has no native ESP-NOW. esp_wifi_remote INJECTS the full
+ * esp_now.h header (types + declarations) but ships NO implementation, so every
+ * esp_now_* symbol is an undefined reference at link time. This translation
+ * unit provides those definitions; each forwards to the co-processor over
+ * CustomRpc (see esphome/esp-hosted-firmware for the matching coprocessor
+ * handlers). No esp-hosted or esp_wifi_remote source is patched, and there is no
+ * duplicate-symbol clash because nothing else defines these symbols here.
+ *
+ * See esp_now_hosted_rpc.h for the wire protocol.
+ */
+
+#include "sdkconfig.h"
+
+// Only build the shim on the radio-less host. On chips with a native ESP-NOW
+// stack (S3, C6, …) the real symbols exist and this file must stay empty to
+// avoid duplicate definitions.
+#if defined(CONFIG_IDF_TARGET_ESP32P4)
+
+#include <cstring>
+
+#include "freertos/FreeRTOS.h"
+#include "freertos/semphr.h"
+
+#include "esp_idf_version.h"
+#include "esp_log.h"
+#include "esp_timer.h"
+
+#include <esp_now.h>         // injected declarations we are now DEFINING
+#include <esp_wifi_types.h>  // wifi_pkt_rx_ctrl_t, wifi_tx_info_t
+
+// esp_hosted_misc.h (host) ships WITHOUT an extern "C" guard, so including it
+// from C++ would give its declarations C++ linkage and the real C symbols in
+// libesp_hosted would go unresolved at link. Wrap it. (Verified vs
+// esp_hosted 2.12.9.)
+extern "C" {
+#include "esp_hosted_misc.h"  // esp_hosted_{send_custom_data,register_custom_callback}
+}
+
+#include "esp_now_hosted_rpc.h"
+
+namespace {
+
+const char *const TAG = "esp_now_hosted";
+
+// One outstanding request at a time. ESPHome drives esp_now_* from the main
+// loop; the matching response and the async RECV/SEND events all arrive on the
+// single esp-hosted RPC RX thread. Serializing requests keeps the shared
+// response slot race-free; a sequence number stops a late/stale response from
+// being mistaken for ours.
+SemaphoreHandle_t g_req_mutex = nullptr;
+SemaphoreHandle_t g_resp_sem = nullptr;  // given when the matching RESP lands
+bool g_setup_done = false;               // set only after setup fully succeeds
+uint8_t g_seq = 0;
+volatile uint8_t g_expect_seq = 0;
+volatile int32_t g_resp_status = 0;
+uint8_t g_resp_ret[16];
+volatile uint16_t g_resp_ret_len = 0;
+
+// Written from the main loop (register/unregister/deinit), read from the
+// esp-hosted RX thread (on_recv/on_send). volatile for the same reason the
+// g_resp_* globals are: force the RX thread to observe an updated pointer
+// (e.g. a nulling by esp_now_deinit) rather than a cached one.
+volatile esp_now_recv_cb_t g_recv_cb = nullptr;
+volatile esp_now_send_cb_t g_send_cb = nullptr;
+
+// Local mirror of the co-processor's peer table. ESPHome's espnow component
+// calls esp_now_is_peer_exist() on the main loop for every received frame
+// (twice) and every send; forwarding each as a blocking RPC round-trip stalls
+// the loop. The shim is the only path that mutates the co-processor peer table
+// (add/del/deinit all go through here), so this mirror is authoritative and
+// esp_now_is_peer_exist() can answer from it with no round-trip.
+//
+// esp_now_* are public C symbols: any component or user lambda may call them,
+// and although ESPHome's espnow touches peers only from the main loop today
+// (its RX/TX callbacks merely enqueue), the shim cannot rely on that. A short
+// spinlock keeps the mirror consistent from any task/core, matching native
+// esp_now_*'s own internal thread-safety. The critical sections are a bounded
+// (<=20-entry) scan, so they stay tiny. ESP_NOW_MAX_TOTAL_PEER_NUM is 20.
+constexpr size_t ESP_NOW_HOSTED_MAX_PEERS = 20;
+uint8_t g_peer_cache[ESP_NOW_HOSTED_MAX_PEERS][6];
+size_t g_peer_count = 0;
+portMUX_TYPE g_peer_lock = portMUX_INITIALIZER_UNLOCKED;
+
+// Caller must hold g_peer_lock.
+int peer_cache_find_locked(const uint8_t *mac) {
+  for (size_t i = 0; i < g_peer_count; i++) {
+    if (memcmp(g_peer_cache[i], mac, 6) == 0)
+      return static_cast<int>(i);
+  }
+  return -1;
+}
+
+bool peer_cache_contains(const uint8_t *mac) {
+  portENTER_CRITICAL(&g_peer_lock);
+  const bool found = peer_cache_find_locked(mac) >= 0;
+  portEXIT_CRITICAL(&g_peer_lock);
+  return found;
+}
+
+void peer_cache_add(const uint8_t *mac) {
+  portENTER_CRITICAL(&g_peer_lock);
+  if (peer_cache_find_locked(mac) < 0 && g_peer_count < ESP_NOW_HOSTED_MAX_PEERS)
+    memcpy(g_peer_cache[g_peer_count++], mac, 6);
+  portEXIT_CRITICAL(&g_peer_lock);
+}
+
+void peer_cache_remove(const uint8_t *mac) {
+  portENTER_CRITICAL(&g_peer_lock);
+  const int idx = peer_cache_find_locked(mac);
+  if (idx >= 0) {
+    g_peer_count--;
+    if (static_cast<size_t>(idx) != g_peer_count)  // move the last entry into the gap
+      memcpy(g_peer_cache[idx], g_peer_cache[g_peer_count], 6);
+  }
+  portEXIT_CRITICAL(&g_peer_lock);
+}
+
+void peer_cache_clear() {
+  portENTER_CRITICAL(&g_peer_lock);
+  g_peer_count = 0;
+  portEXIT_CRITICAL(&g_peer_lock);
+}
+
+// ── CustomRpc event handlers (run on the esp-hosted RPC RX thread) ──────────
+// Keep them short and non-blocking. In particular they MUST NOT call back into
+// any esp_now_* shim function: that would try to take g_req_mutex / wait on the
+// RX thread that delivers the response, and deadlock.
+
+void on_resp(uint32_t /*msg_id*/, const uint8_t *data, size_t len, void * /*ctx*/) {
+  if (len < sizeof(esp_now_hosted_resp_t)) {
+    ESP_LOGW(TAG, "RESP too short: %u bytes", static_cast<unsigned>(len));
+    return;
+  }
+  const auto *r = reinterpret_cast<const esp_now_hosted_resp_t *>(data);
+  if (r->seq != g_expect_seq) {  // late response from a timed-out request (expected)
+    ESP_LOGV(TAG, "dropping stale RESP seq %u (want %u)", r->seq, g_expect_seq);
+    return;
+  }
+  g_resp_status = r->status;
+  uint16_t rl = r->ret_len;
+  if (rl > sizeof(g_resp_ret)) {
+    // Larger than any real opcode return — a likely wire-format drift signal.
+    ESP_LOGW(TAG, "RESP ret_len %u exceeds buffer, clamping (wire drift?)", rl);
+    rl = sizeof(g_resp_ret);
+  }
+  if (len >= sizeof(esp_now_hosted_resp_t) + rl) {
+    memcpy(g_resp_ret, r->ret, rl);
+  } else {
+    // Truncated frame: fail closed. Never hand the caller stale bytes left in
+    // g_resp_ret by a previous response, and don't let request() report a
+    // zeroed payload as success — override the status to an error.
+    ESP_LOGW(TAG, "RESP truncated: claims %u ret bytes, frame too short", rl);
+    rl = 0;
+    g_resp_status = ESP_ERR_INVALID_RESPONSE;
+  }
+  g_resp_ret_len = rl;
+  xSemaphoreGive(g_resp_sem);
+}
+
+void on_recv(uint32_t /*msg_id*/, const uint8_t *data, size_t len, void * /*ctx*/) {
+  // Read the volatile pointer once: esp_now_unregister_recv_cb()/deinit() (via
+  // the espnow component's disable()) can null it on the main loop between the
+  // guard and the call, which would otherwise turn the call into a null-deref.
+  const esp_now_recv_cb_t cb = g_recv_cb;
+  if (cb == nullptr)
+    return;
+  if (len < sizeof(esp_now_hosted_recv_evt_t)) {
+    ESP_LOGW(TAG, "RECV too short: %u bytes", static_cast<unsigned>(len));
+    return;
+  }
+  const auto *e = reinterpret_cast<const esp_now_hosted_recv_evt_t *>(data);
+  if (len < sizeof(esp_now_hosted_recv_evt_t) + e->data_len) {
+    ESP_LOGW(TAG, "RECV data_len %u exceeds frame", e->data_len);
+    return;
+  }
+
+  // ESPHome dereferences info->rx_ctrl->{rssi,timestamp}; give it a real one.
+  wifi_pkt_rx_ctrl_t rx_ctrl;
+  memset(&rx_ctrl, 0, sizeof(rx_ctrl));
+  rx_ctrl.rssi = e->rssi;
+  rx_ctrl.channel = e->channel;
+  rx_ctrl.timestamp = static_cast<uint32_t>(esp_timer_get_time());
+
+  esp_now_recv_info_t info;
+  info.src_addr = const_cast<uint8_t *>(e->src_addr);
+  info.des_addr = const_cast<uint8_t *>(e->des_addr);
+  info.rx_ctrl = &rx_ctrl;
+  cb(&info, e->data, static_cast<int>(e->data_len));
+}
+
+void on_send(uint32_t /*msg_id*/, const uint8_t *data, size_t len, void * /*ctx*/) {
+  // Read the volatile pointer once (see on_recv): disable()/deinit() can null it
+  // on the main loop concurrently with this RX-thread callback.
+  const esp_now_send_cb_t cb = g_send_cb;
+  if (cb == nullptr)
+    return;
+  if (len < sizeof(esp_now_hosted_send_evt_t)) {
+    ESP_LOGW(TAG, "SEND evt too short: %u bytes", static_cast<unsigned>(len));
+    return;
+  }
+  const auto *e = reinterpret_cast<const esp_now_hosted_send_evt_t *>(data);
+#if ESP_IDF_VERSION >= ESP_IDF_VERSION_VAL(5, 5, 0)
+  // IDF >= 5.5: esp_now_send_cb_t takes esp_now_send_info_t (== wifi_tx_info_t),
+  // whose des_addr is a POINTER (not an inline array). Point it at the event's
+  // MAC (valid for this callback) — do NOT memcpy into it (that writes NULL and
+  // faults). ESPHome reads only info->des_addr.
+  esp_now_send_info_t si;
+  memset(&si, 0, sizeof(si));
+  si.des_addr = const_cast<uint8_t *>(e->des_addr);
+  cb(&si, static_cast<esp_now_send_status_t>(e->status));
+#else
+  cb(e->des_addr, static_cast<esp_now_send_status_t>(e->status));
+#endif
+}
+
+esp_err_t ensure_setup() {
+  // Gate on g_setup_done, not on g_req_mutex: a failure part-way through (a
+  // semaphore that did not allocate, a callback that did not register) must not
+  // leave a later call thinking setup completed. Semaphore creation is guarded
+  // so a retry after a partial failure does not leak the earlier handles.
+  if (g_setup_done)
+    return ESP_OK;
+  if (g_req_mutex == nullptr)
+    g_req_mutex = xSemaphoreCreateMutex();
+  if (g_resp_sem == nullptr)
+    g_resp_sem = xSemaphoreCreateBinary();
+  if (g_req_mutex == nullptr || g_resp_sem == nullptr)
+    return ESP_ERR_NO_MEM;
+  esp_err_t err;
+  if ((err = esp_hosted_register_custom_callback(ESP_NOW_HOSTED_MSG_RESP, on_resp, nullptr)) != ESP_OK)
+    return err;
+  if ((err = esp_hosted_register_custom_callback(ESP_NOW_HOSTED_MSG_RECV, on_recv, nullptr)) != ESP_OK)
+    return err;
+  if ((err = esp_hosted_register_custom_callback(ESP_NOW_HOSTED_MSG_SEND, on_send, nullptr)) != ESP_OK)
+    return err;
+  g_setup_done = true;
+  return ESP_OK;
+}
+
+// Send one request envelope. With wait=true (default) block until the matching
+// response (or timeout); with wait=false return as soon as the frame is handed
+// to the transport (fire-and-forget, used by esp_now_send).
+//
+// `tail` is an optional second chunk written straight after `payload`. Callers
+// with a fixed header plus a bulk body (esp_now_send) pass the two separately
+// so they never need a build buffer of their own: both chunks are laid into the
+// request buffer here, under g_req_mutex, which keeps concurrent callers from
+// racing and saves a full copy of the body on every transmit.
+esp_err_t request(uint8_t opcode, const void *payload, uint16_t plen, void *ret, uint16_t ret_cap, uint16_t *ret_len,
+                  bool wait = true, const void *tail = nullptr, uint16_t tail_len = 0) {
+  esp_err_t err = ensure_setup();
+  if (err != ESP_OK)
+    return err;
+  if (plen > ESP_NOW_HOSTED_MAX_PAYLOAD || tail_len > ESP_NOW_HOSTED_MAX_PAYLOAD - plen)
+    return ESP_ERR_INVALID_SIZE;
+  const uint16_t total_len = static_cast<uint16_t>(plen + tail_len);
+
+  if (xSemaphoreTake(g_req_mutex, portMAX_DELAY) != pdTRUE)
+    return ESP_FAIL;
+
+  static uint8_t buf[sizeof(esp_now_hosted_req_t) + ESP_NOW_HOSTED_MAX_PAYLOAD];  // guarded by g_req_mutex
+  auto *req = reinterpret_cast<esp_now_hosted_req_t *>(buf);
+  req->opcode = opcode;
+  req->seq = ++g_seq;
+  req->payload_len = total_len;
+  if (plen != 0)
+    memcpy(req->payload, payload, plen);
+  if (tail_len != 0)
+    memcpy(req->payload + plen, tail, tail_len);
+  g_expect_seq = req->seq;
+
+  xSemaphoreTake(g_resp_sem, 0);  // drain any stale signal before sending
+  err = esp_hosted_send_custom_data(ESP_NOW_HOSTED_MSG_REQ, buf, sizeof(esp_now_hosted_req_t) + total_len);
+  if (err != ESP_OK) {
+    xSemaphoreGive(g_req_mutex);
+    return err;
+  }
+  if (!wait) {
+    // Fire-and-forget (esp_now_send): the co-processor enqueues the frame and
+    // reports the real TX result later via the async SEND event, exactly like
+    // native esp_now_send. Returning here keeps the main loop off the ~100 ms+
+    // RPC round-trip. The matching RESP is ignored (seq won't match the next
+    // waited request, so on_resp drops it).
+    xSemaphoreGive(g_req_mutex);
+    return ESP_OK;
+  }
+  if (xSemaphoreTake(g_resp_sem, pdMS_TO_TICKS(ESP_NOW_HOSTED_TIMEOUT_MS)) != pdTRUE) {
+    ESP_LOGW(TAG, "opcode %u timed out", opcode);
+    xSemaphoreGive(g_req_mutex);
+    return ESP_ERR_TIMEOUT;
+  }
+
+  const int32_t status = g_resp_status;
+  if (ret != nullptr && ret_cap != 0) {
+    uint16_t n = g_resp_ret_len < ret_cap ? g_resp_ret_len : ret_cap;
+    memcpy(ret, const_cast<const uint8_t *>(g_resp_ret), n);
+    if (ret_len != nullptr)
+      *ret_len = n;
+  }
+  xSemaphoreGive(g_req_mutex);
+  return static_cast<esp_err_t>(status);
+}
+
+}  // namespace
+
+// ── The <esp_now.h> surface, defined for the radio-less host ────────────────
+extern "C" {
+
+esp_err_t esp_now_init(void) { return request(ESP_NOW_HOSTED_OP_INIT, nullptr, 0, nullptr, 0, nullptr); }
+
+esp_err_t esp_now_deinit(void) {
+  g_recv_cb = nullptr;
+  g_send_cb = nullptr;
+  peer_cache_clear();  // the co-processor drops all peers on deinit
+  return request(ESP_NOW_HOSTED_OP_DEINIT, nullptr, 0, nullptr, 0, nullptr);
+}
+
+esp_err_t esp_now_get_version(uint32_t *version) {
+  uint32_t v = 0;
+  uint16_t rl = 0;
+  esp_err_t err = request(ESP_NOW_HOSTED_OP_GET_VERSION, nullptr, 0, &v, sizeof(v), &rl);
+  if (version != nullptr)
+    *version = v;
+  return err;
+}
+
+esp_err_t esp_now_register_recv_cb(esp_now_recv_cb_t cb) {
+  // Only arm the callback once the CustomRpc handlers are actually registered,
+  // so a failed setup leaves g_recv_cb null rather than falsely "registered".
+  esp_err_t err = ensure_setup();
+  if (err != ESP_OK)
+    return err;
+  g_recv_cb = cb;
+  return ESP_OK;
+}
+esp_err_t esp_now_unregister_recv_cb(void) {
+  g_recv_cb = nullptr;
+  return ESP_OK;
+}
+esp_err_t esp_now_register_send_cb(esp_now_send_cb_t cb) {
+  esp_err_t err = ensure_setup();
+  if (err != ESP_OK)
+    return err;
+  g_send_cb = cb;
+  return ESP_OK;
+}
+esp_err_t esp_now_unregister_send_cb(void) {
+  g_send_cb = nullptr;
+  return ESP_OK;
+}
+
+static esp_err_t add_or_mod_peer(uint8_t opcode, const esp_now_peer_info_t *peer, bool wait) {
+  if (peer == nullptr)
+    return ESP_ERR_ESPNOW_ARG;
+  esp_now_hosted_peer_t p;
+  memset(&p, 0, sizeof(p));
+  memcpy(p.peer_addr, peer->peer_addr, 6);
+  memcpy(p.lmk, peer->lmk, 16);
+  p.channel = peer->channel;
+  p.ifidx = static_cast<uint8_t>(peer->ifidx);
+  p.encrypt = peer->encrypt ? 1 : 0;
+  return request(opcode, &p, sizeof(p), nullptr, 0, nullptr, wait);
+}
+esp_err_t esp_now_add_peer(const esp_now_peer_info_t *peer) {
+  // Fire-and-forget (wait=false): adding a peer is a blocking RPC round-trip,
+  // and ESPHome's espnow calls it on the main loop when a device joins the mesh
+  // — under co-processor load that stalls the UI (peer-churn stutter). Issue it
+  // without waiting and mirror it locally. Safe against a following
+  // esp_now_send to the same peer: both ride the same in-order CustomRpc
+  // channel (mutex-serialized on the host) and the co-processor processes REQs
+  // FIFO, so ADD_PEER is applied before the SEND. Trade-off: a co-processor-side
+  // failure (e.g. peer table full) is no longer reported synchronously — the
+  // same limitation as esp_now_send — but ESPHome only adds peers it validated.
+  esp_err_t err = add_or_mod_peer(ESP_NOW_HOSTED_OP_ADD_PEER, peer, /*wait=*/false);
+  if (err == ESP_OK)
+    peer_cache_add(peer->peer_addr);  // keep the local mirror in sync
+  return err;
+}
+esp_err_t esp_now_mod_peer(const esp_now_peer_info_t *peer) {
+  // mod_peer changes a peer's parameters, not its existence, so the cache is
+  // unaffected. Kept synchronous — it is not on any hot path (espnow never
+  // calls it), so the extra round-trip does not matter and the status is useful.
+  return add_or_mod_peer(ESP_NOW_HOSTED_OP_MOD_PEER, peer, /*wait=*/true);
+}
+
+esp_err_t esp_now_del_peer(const uint8_t *peer_addr) {
+  if (peer_addr == nullptr)
+    return ESP_ERR_ESPNOW_ARG;
+  // Fire-and-forget for the same reason as add_peer (peer churn on the main
+  // loop). Removal is order-independent, so this is strictly safe.
+  esp_err_t err = request(ESP_NOW_HOSTED_OP_DEL_PEER, peer_addr, 6, nullptr, 0, nullptr, /*wait=*/false);
+  if (err == ESP_OK)
+    peer_cache_remove(peer_addr);  // keep the local mirror in sync
+  return err;
+}
+
+bool esp_now_is_peer_exist(const uint8_t *peer_addr) {
+  if (peer_addr == nullptr)
+    return false;
+  // Answered from the local mirror — no RPC round-trip. ESPHome's espnow calls
+  // this on the main loop for every received frame and every send, so a
+  // blocking round-trip here would stall rendering under mesh traffic.
+  return peer_cache_contains(peer_addr);
+}
+
+esp_err_t esp_now_send(const uint8_t *peer_addr, const uint8_t *data, size_t len) {
+  if (len > ESP_NOW_HOSTED_MAX_FRAME)
+    return ESP_ERR_ESPNOW_ARG;
+  if (data == nullptr && len != 0)  // native esp_now_send treats this as an arg error
+    return ESP_ERR_ESPNOW_ARG;
+  // Only the small fixed header is built here; the caller's frame goes over as
+  // the request tail, so request() lays both into its own buffer under
+  // g_req_mutex. esp_now_send is a public C symbol and may be called from any
+  // task, and a shared build buffer here would let two callers corrupt each
+  // other's frame. Passing the body through also drops a full-frame copy per
+  // transmit, on the path this shim exists to keep quick.
+  uint8_t hdr[sizeof(esp_now_hosted_send_req_t)];
+  auto *s = reinterpret_cast<esp_now_hosted_send_req_t *>(hdr);
+  s->has_addr = peer_addr != nullptr ? 1 : 0;
+  if (peer_addr != nullptr)
+    memcpy(s->peer_addr, peer_addr, 6);
+  else
+    memset(s->peer_addr, 0, 6);
+  s->data_len = static_cast<uint16_t>(len);
+  // Fire-and-forget (wait=false): native esp_now_send returns once the frame is
+  // queued, with the real TX result delivered later through the send callback.
+  // The co-processor mirrors that — it acks enqueue immediately and reports the
+  // outcome via the async SEND event (on_send -> on_send_report). Waiting for
+  // the RPC RESP here would block the main loop for the full round-trip on
+  // every transmit.
+  return request(ESP_NOW_HOSTED_OP_SEND, hdr, sizeof(hdr), nullptr, 0, nullptr, /*wait=*/false, data,
+                 static_cast<uint16_t>(len));
+}
+
+esp_err_t esp_now_set_pmk(const uint8_t *pmk) {
+  if (pmk == nullptr)
+    return ESP_ERR_ESPNOW_ARG;
+  return request(ESP_NOW_HOSTED_OP_SET_PMK, pmk, 16, nullptr, 0, nullptr);
+}
+
+// Remainder of the <esp_now.h> surface. Not used by ESPHome's espnow component
+// today; provided so the whole header links and future callers get a defined
+// (if unimplemented) symbol rather than a link error. Wire them through
+// CustomRpc if a use case appears.
+esp_err_t esp_now_get_peer(const uint8_t * /*peer_addr*/, esp_now_peer_info_t * /*peer*/) {
+  return ESP_ERR_NOT_SUPPORTED;
+}
+esp_err_t esp_now_fetch_peer(bool /*from_head*/, esp_now_peer_info_t * /*peer*/) { return ESP_ERR_NOT_SUPPORTED; }
+esp_err_t esp_now_get_peer_num(esp_now_peer_num_t * /*num*/) { return ESP_ERR_NOT_SUPPORTED; }
+esp_err_t esp_now_set_wake_window(uint16_t /*window*/) {
+  return ESP_ERR_NOT_SUPPORTED;  // power-save wake window is not forwarded; don't claim success
+}
+esp_err_t esp_now_set_peer_rate_config(const uint8_t * /*peer_addr*/, esp_now_rate_config_t * /*cfg*/) {
+  return ESP_ERR_NOT_SUPPORTED;
+}
+esp_err_t esp_wifi_config_espnow_rate(wifi_interface_t /*ifx*/, wifi_phy_rate_t /*rate*/) {
+  return ESP_ERR_NOT_SUPPORTED;
+}
+
+}  // extern "C"
+
+#endif  // CONFIG_IDF_TARGET_ESP32P4

esphome/components/esp32_hosted/esp_now_hosted_rpc.h

@@ -0,0 +1,128 @@
+/*
+ * esp_now_hosted — ESP-NOW-over-CustomRpc wire protocol.
+ *
+ * Shared, byte-for-byte-identical contract between:
+ *   - the host shim         (esphome/components/esp32_hosted/esp_now_hosted.cpp)
+ *   - the coprocessor firmware (esphome/esp-hosted-firmware)
+ *
+ * It rides esp-hosted's CustomRpc channel (RPC ID 388, "peer data transfer",
+ * available since esp-hosted v2.8.1), teaching the radio-less host <-> radio
+ * co-processor link to carry esp_now.h, which esp-hosted itself does not proxy
+ * (Espressif issue espressif/esp-hosted-mcu#19).
+ *
+ * KEEP THE TWO COPIES IN SYNC. The canonical copy lives here; the coprocessor
+ * firmware uses a verbatim copy. Both sides are little-endian, so these packed
+ * structs are wire-compatible with no byte-swapping.
+ */
+
+#ifndef ESP_NOW_HOSTED_RPC_H
+#define ESP_NOW_HOSTED_RPC_H
+
+#ifdef __cplusplus
+#include <cstdint>
+#else
+#include <stdint.h>
+#endif
+
+#ifdef __cplusplus
+extern "C" {
+#endif
+
+/* ── CustomRpc message IDs (any uint32_t except 0xFFFFFFFF) ──────────────────
+ * One REQ handler slot on the device; three event handler slots on the host.
+ * The bytes spell "now" + index, a private range unlikely to clash with other
+ * CustomRpc users (e.g. the stock peer_data_transfer example's 1..6).         */
+#define ESP_NOW_HOSTED_MSG_REQ 0x6E6F7701u  /* host   -> device : request envelope */
+#define ESP_NOW_HOSTED_MSG_RESP 0x6E6F7702u /* device -> host   : reply to a REQ   */
+#define ESP_NOW_HOSTED_MSG_RECV 0x6E6F7703u /* device -> host   : async RX frame   */
+#define ESP_NOW_HOSTED_MSG_SEND 0x6E6F7704u /* device -> host   : async TX status  */
+
+/* ── Request opcodes ────────────────────────────────────────────────────── */
+enum {
+  ESP_NOW_HOSTED_OP_INIT = 1,          /* esp_now_init + register device recv/send cbs */
+  ESP_NOW_HOSTED_OP_DEINIT = 2,        /* unregister cbs + esp_now_deinit             */
+  ESP_NOW_HOSTED_OP_ADD_PEER = 3,      /* payload: esp_now_hosted_peer_t              */
+  ESP_NOW_HOSTED_OP_DEL_PEER = 4,      /* payload: 6-byte peer MAC                    */
+  ESP_NOW_HOSTED_OP_IS_PEER_EXIST = 5, /* payload: 6-byte MAC; ret: 1 byte bool       */
+  ESP_NOW_HOSTED_OP_SEND = 6,          /* payload: esp_now_hosted_send_req_t          */
+  ESP_NOW_HOSTED_OP_GET_VERSION = 7,   /* ret: uint32 version                        */
+  ESP_NOW_HOSTED_OP_SET_PMK = 8,       /* payload: 16-byte PMK                        */
+  ESP_NOW_HOSTED_OP_MOD_PEER = 9,      /* payload: esp_now_hosted_peer_t              */
+};
+
+/* Largest ESP-NOW payload we forward. ESP-NOW v2 (IDF >= 5.4) is 1470 B; well
+ * under esp-hosted's 8166 B CustomRpc cap, so the shim never truncates.        */
+#define ESP_NOW_HOSTED_MAX_FRAME 1470u
+/* Envelope slack for the largest opcode payload (a SEND req wrapping a frame). */
+#define ESP_NOW_HOSTED_MAX_PAYLOAD (ESP_NOW_HOSTED_MAX_FRAME + 16u)
+/* Host request/response round-trip timeout over the transport. Generous:
+ * normal RTT is sub-millisecond, but Wi-Fi/BLE contention on the co-processor
+ * can stall the RX thread.                                                     */
+#define ESP_NOW_HOSTED_TIMEOUT_MS 2000
+
+/* ── Envelopes ──────────────────────────────────────────────────────────── */
+
+/* These payloads are shared verbatim with the C co-processor firmware, so they
+ * use C's `typedef struct {...} name;` idiom rather than C++ `using` aliases,
+ * which would not compile there. Silence clang-tidy's modernize-use-using for
+ * the shared struct block. */
+// NOLINTBEGIN(modernize-use-using)
+typedef struct {
+  uint8_t opcode;       /* one of ESP_NOW_HOSTED_OP_*                        */
+  uint8_t seq;          /* wraps 0..255; echoed in the response for matching */
+  uint16_t payload_len; /* bytes of opcode-specific payload that follow      */
+  uint8_t payload[];    /* flexible                                          */
+} __attribute__((packed)) esp_now_hosted_req_t;
+
+typedef struct {
+  uint8_t opcode;   /* echoes the request opcode                         */
+  uint8_t seq;      /* echoes the request seq                            */
+  int32_t status;   /* esp_err_t from the native call on the co-processor */
+  uint16_t ret_len; /* bytes of return payload that follow               */
+  uint8_t ret[];    /* flexible (e.g. version u32, is_peer_exist bool)   */
+} __attribute__((packed)) esp_now_hosted_resp_t;
+
+/* ── Opcode payloads ────────────────────────────────────────────────────── */
+
+/* esp_now_peer_info_t minus the host-only `priv` pointer, which is meaningless
+ * across the transport and never set by ESPHome's espnow component.           */
+typedef struct {
+  uint8_t peer_addr[6];
+  uint8_t lmk[16];
+  uint8_t channel; /* 0 = current channel                              */
+  uint8_t ifidx;   /* wifi_interface_t (0=STA, 1=AP)                    */
+  uint8_t encrypt; /* bool                                             */
+} __attribute__((packed)) esp_now_hosted_peer_t;
+
+typedef struct {
+  uint8_t has_addr; /* 0 => peer_addr is NULL (broadcast to all peers)   */
+  uint8_t peer_addr[6];
+  uint16_t data_len;
+  uint8_t data[]; /* flexible, up to ESP_NOW_HOSTED_MAX_FRAME          */
+} __attribute__((packed)) esp_now_hosted_send_req_t;
+
+/* ── Async events (device -> host) ──────────────────────────────────────── */
+
+/* Reconstructed on the host into an esp_now_recv_info_t + a minimal
+ * wifi_pkt_rx_ctrl_t. ESPHome's espnow reads info->src_addr, info->des_addr,
+ * info->rx_ctrl->rssi and info->rx_ctrl->timestamp.                           */
+typedef struct {
+  uint8_t src_addr[6];
+  uint8_t des_addr[6];
+  int8_t rssi;
+  uint8_t channel;
+  uint16_t data_len;
+  uint8_t data[]; /* flexible                                          */
+} __attribute__((packed)) esp_now_hosted_recv_evt_t;
+
+typedef struct {
+  uint8_t des_addr[6];
+  uint8_t status; /* esp_now_send_status_t (0 = success)               */
+} __attribute__((packed)) esp_now_hosted_send_evt_t;
+// NOLINTEND(modernize-use-using)
+
+#ifdef __cplusplus
+}
+#endif
+
+#endif /* ESP_NOW_HOSTED_RPC_H */

esphome/components/esp32_improv/esp32_improv_component.cpp

@@ -112,6 +112,7 @@ void ESP32ImprovComponent::loop() {
       this->state_callback_.call(this->state_, this->error_state_);
 #endif
     }
+    this->release_advertising_();
     this->incoming_data_.clear();
     return;
   }
@@ -143,8 +144,9 @@ void ESP32ImprovComponent::loop() {
           ESP_LOGV(TAG, "Starting with device name advertising");
           this->advertising_device_name_ = true;
           this->last_name_adv_time_ = App.get_loop_component_start_time();
+          // Set the payload before requesting, so advertising starts exactly once
           esp32_ble::global_ble->advertising_set_service_data_and_name(std::span<const uint8_t>{}, true);
-          esp32_ble::global_ble->advertising_start();
+          this->request_advertising_();
 
           // Set initial state based on whether we have an authorizer
           this->set_state_(this->get_initial_state_(), false);
@@ -326,6 +328,8 @@ void ESP32ImprovComponent::stop() {
   this->set_timeout("end-service", STOP_ADVERTISING_DELAY, [this] {
     if (this->state_ == improv::STATE_STOPPED || this->service_ == nullptr)
       return;
+    // Release first so removing the service UUID does not restart advertising on the way out
+    this->release_advertising_();
     this->service_->stop();
     this->set_state_(improv::STATE_STOPPED);
   });
@@ -520,6 +524,20 @@ void ESP32ImprovComponent::update_advertising_type_() {
   }
 }
 
+void ESP32ImprovComponent::request_advertising_() {
+  if (this->advertising_requested_)
+    return;
+  this->advertising_requested_ = true;
+  esp32_ble::global_ble->advertising_start();
+}
+
+void ESP32ImprovComponent::release_advertising_() {
+  if (!this->advertising_requested_)
+    return;
+  this->advertising_requested_ = false;
+  esp32_ble::global_ble->advertising_stop();
+}
+
 improv::State ESP32ImprovComponent::get_initial_state_() const {
 #ifdef USE_BINARY_SENSOR
   // If we have an authorizer, start in awaiting authorization state

esphome/components/esp32_improv/esp32_improv_component.h

@@ -104,8 +104,11 @@ class ESP32ImprovComponent final : public Component, public improv_base::ImprovB
   bool status_indicator_state_{false};
   uint32_t last_name_adv_time_{0};
   bool advertising_device_name_{false};
+  bool advertising_requested_{false};
   void set_status_indicator_state_(bool state);
   void update_advertising_type_();
+  void request_advertising_();
+  void release_advertising_();
 
   void set_state_(improv::State state, bool update_advertising = true);
   void set_error_(improv::Error error);

esphome/components/esphome/ota/__init__.py

@@ -2,12 +2,12 @@ import logging
 
 import esphome.codegen as cg
 from esphome.components.noise import (
-    decode_encryption_key,
     encryption_schema,
-    is_reserved_key,
+    new_psk_progmem,
+    static_encryption_key,
 )
 from esphome.components.ota import BASE_OTA_SCHEMA, OTAComponent, ota_to_code
-from esphome.config_helpers import merge_config
+from esphome.config_helpers import filter_source_files_from_defines, merge_config
 import esphome.config_validation as cv
 from esphome.const import (
     CONF_API,
@@ -31,7 +31,6 @@ import esphome.final_validate as fv
 from esphome.types import ConfigType
 
 CONF_ALLOW_PARTITION_ACCESS = "allow_partition_access"
-CONF_CAPTIVE_PORTAL = "captive_portal"
 
 _LOGGER = logging.getLogger(__name__)
 
@@ -41,11 +40,10 @@ DEPENDENCIES = ["network"]
 
 
 def AUTO_LOAD(config: ConfigType) -> list[str]:
-    """Auto-load noise only when encryption is configured."""
+    """Auto-load noise only when encryption is configured; the api key offer
+    inherits it from the api component."""
     base = ["sha256", "socket"]
-    # A falsy config is a tooling probe for the maximal set (None from
-    # dependency resolution, {} from the components-graph platform probe);
-    # a validated config always carries defaults, never empty
+    # A falsy config is a tooling probe for the maximal set
     if not config or CONF_ENCRYPTION in config:
         return base + ["noise"]
     return base
@@ -132,12 +130,56 @@ def ota_esphome_final_validate(config: ConfigType) -> None:
         _validate_no_password_with_encryption(ota_conf)
         if (encryption_conf := ota_conf.get(CONF_ENCRYPTION)) is not None:
             _resolve_encryption_key(encryption_conf, api_conf)
-    if any(
-        conf.get(CONF_PLATFORM) == CONF_WEB_SERVER for conf in full_ota_conf
-    ) and any(
-        CONF_ENCRYPTION in conf for conf in merged_ota_esphome_configs_by_port.values()
+        elif CONF_PASSWORD in ota_conf and static_encryption_key(api_conf) is not None:
+            _LOGGER.warning(
+                "'%s' %s wastes significant flash and RAM (about 3.5 KB and 60 "
+                "bytes plus the password on the heap): the device already offers "
+                "encryption with the '%s' %s %s, which authenticates any uploader "
+                "that takes it, and a password only matters for uploaders without "
+                "encryption support; remove '%s' and add '%s' under '%s' so "
+                "uploads use the key and encryption is required",
+                CONF_OTA,
+                CONF_PASSWORD,
+                CONF_API,
+                CONF_ENCRYPTION,
+                CONF_KEY,
+                CONF_PASSWORD,
+                CONF_ENCRYPTION,
+                CONF_OTA,
+            )
+        elif (
+            CONF_PASSWORD in ota_conf
+            and CONF_ENCRYPTION in api_conf
+            and not api_conf[CONF_ENCRYPTION].get(CONF_KEY)
+        ):
+            # The CLI still needs the password; whoever provisions the key skips it
+            _LOGGER.warning(
+                "The '%s' %s %s provisioned at runtime also authenticates OTA "
+                "uploads once provisioned; '%s' %s then only guards plaintext "
+                "uploads. Whoever provisions the key can upload firmware "
+                "without the password, so add a 'provisioning:' block to limit "
+                "when that is possible",
+                CONF_API,
+                CONF_ENCRYPTION,
+                CONF_KEY,
+                CONF_OTA,
+                CONF_PASSWORD,
+            )
+    # web_server and prometheus keep the shared listener up; the captive
+    # portal's copy only exists on the fallback AP and is the recovery path
+    if (
+        (CONF_WEB_SERVER in full_conf or "prometheus" in full_conf)
+        and any(conf.get(CONF_PLATFORM) == CONF_WEB_SERVER for conf in full_ota_conf)
+        and any(
+            CONF_ENCRYPTION in conf
+            for conf in merged_ota_esphome_configs_by_port.values()
+        )
     ):
-        _warn_web_server_ota(full_conf)
+        _LOGGER.warning(
+            "OTA encryption does not cover the %s OTA platform; its "
+            "plaintext /update endpoint accepts the same image",
+            CONF_WEB_SERVER,
+        )
 
     full_conf[CONF_OTA] = new_ota_conf
     fv.full_config.set(full_conf)
@@ -152,33 +194,11 @@ def ota_esphome_final_validate(config: ConfigType) -> None:
         )
 
 
-def _warn_web_server_ota(full_conf: ConfigType) -> None:
-    """The web_server ota platform accepts the same image over plaintext HTTP
-    with basic auth, bypassing the encryption; warn rather than fail so the
-    operator keeps the recovery path."""
-    if CONF_CAPTIVE_PORTAL in full_conf and CONF_WEB_SERVER not in full_conf:
-        # The captive_portal auto-load: the endpoint only exists while the
-        # fallback AP is active
-        _LOGGER.warning(
-            "OTA encryption does not cover the %s OTA platform (auto-loaded "
-            "by captive_portal); the plaintext /update endpoint stays "
-            "reachable while the fallback AP is active",
-            CONF_WEB_SERVER,
-        )
-    else:
-        _LOGGER.warning(
-            "OTA encryption does not cover the %s OTA platform; its "
-            "plaintext /update endpoint accepts the same image",
-            CONF_WEB_SERVER,
-        )
-
-
 def _resolve_encryption_key(encryption_conf: ConfigType, api_conf: ConfigType) -> None:
     """Resolve the one encryption key per device into the ota block.
 
     An explicit ota key must match the api key, a bare block inherits it,
-    a runtime provisioned api key cannot be inherited, and the all-zeros
-    provisioning sentinel is rejected (the device treats it as no key).
+    a runtime provisioned api key cannot be inherited.
     """
     api_key = api_conf.get(CONF_ENCRYPTION, {}).get(CONF_KEY)
     if ota_key := encryption_conf.get(CONF_KEY):
@@ -201,11 +221,6 @@ def _resolve_encryption_key(encryption_conf: ConfigType, api_conf: ConfigType) -
         )
     else:
         encryption_conf[CONF_KEY] = api_key
-    if is_reserved_key(encryption_conf[CONF_KEY]):
-        raise cv.Invalid(
-            f"The all-zeros {CONF_KEY} is reserved and provides no protection; "
-            f"generate a real key with: openssl rand -base64 32"
-        )
 
 
 # Also called on merged same-port configs in final validate, where schemas
@@ -267,15 +282,9 @@ CONFIG_SCHEMA = cv.All(
 FINAL_VALIDATE_SCHEMA = ota_esphome_final_validate
 
 
-def FILTER_SOURCE_FILES() -> list[str]:
-    """Filter out the noise transport when no ota entry configures encryption."""
-    for ota_conf in CORE.config.get(CONF_OTA, []):
-        if (
-            ota_conf.get(CONF_PLATFORM) == CONF_ESPHOME
-            and ota_conf.get(CONF_ENCRYPTION) is not None
-        ):
-            return []
-    return ["ota_esphome_noise.cpp"]
+FILTER_SOURCE_FILES = filter_source_files_from_defines(
+    {"ota_esphome_noise.cpp": "USE_OTA_ENCRYPTION"}
+)
 
 
 @coroutine_with_priority(CoroPriority.OTA_UPDATES)
@@ -296,11 +305,24 @@ async def to_code(config: ConfigType) -> None:
     if config.get(CONF_ALLOW_PARTITION_ACCESS):
         cg.add_define("USE_OTA_PARTITIONS")
 
-    if (encryption_conf := config.get(CONF_ENCRYPTION)) is not None:
-        # A missing key was resolved from the api component in final validate.
-        key = encryption_conf[CONF_KEY]
+    # One key per device: an api encryption block supplies it (static or
+    # runtime) and offers; the ota block only adds the requirement
+    api_conf = CORE.config.get(CONF_API) or {}
+    encryption_conf = config.get(CONF_ENCRYPTION)
+    own_key = None
+    if encryption_conf is not None and static_encryption_key(api_conf) is None:
+        own_key = encryption_conf[CONF_KEY]
+    if own_key is not None:
         cg.add_define("USE_OTA_ENCRYPTION")
-        cg.add(var.set_noise_psk(list(decode_encryption_key(key))))
+        cg.add(var.set_noise_psk(new_psk_progmem(config[CONF_ID], own_key)))
+    elif CONF_ENCRYPTION in api_conf:
+        cg.add_define("USE_OTA_ENCRYPTION")
+        cg.add_define("USE_OTA_ENCRYPTION_FROM_API")
+        if static_encryption_key(api_conf) is None:
+            # The key arrives at runtime, so the offer has to look for it
+            cg.add_define("USE_OTA_ENCRYPTION_PROVISIONED")
+    if encryption_conf is not None:
+        cg.add_define("USE_OTA_ENCRYPTION_REQUIRED")
 
     # Build flag so lwip_fast_select.c (a .c file that can't include defines.h) sees it.
     cg.add_build_flag("-DUSE_OTA_PLATFORM_ESPHOME")

esphome/components/esphome/ota/ota_esphome.cpp

@@ -1,4 +1,7 @@
 #include "ota_esphome.h"
+#ifdef USE_OTA_ENCRYPTION_FROM_API
+#include "esphome/components/api/api_server.h"
+#endif
 #ifdef USE_OTA
 #ifdef USE_OTA_PASSWORD
 #include "esphome/components/sha256/sha256.h"
@@ -26,6 +29,16 @@
 namespace esphome {
 
 static const char *const TAG = "esphome.ota";
+
+#ifdef USE_OTA_ENCRYPTION
+const noise::NoiseContext &ESPHomeOTAComponent::noise_context_() const {
+#ifdef USE_OTA_ENCRYPTION_FROM_API
+  return api::global_api_server->get_noise_ctx();
+#else
+  return this->noise_ctx_;
+#endif
+}
+#endif
 static constexpr uint16_t OTA_BLOCK_SIZE = 8192;
 static constexpr uint32_t OTA_SOCKET_TIMEOUT_HANDSHAKE = 20000;  // milliseconds for initial handshake
 static constexpr uint32_t OTA_SOCKET_TIMEOUT_DATA = 90000;       // milliseconds for data transfer
@@ -97,18 +110,30 @@ void ESPHomeOTAComponent::dump_config() {
   ESP_LOGCONFIG(TAG,
                 "Over-The-Air updates:\n"
                 "  Address: %s:%u\n"
-                "  Version: %d",
-                network::get_use_address_to(addr_buf), this->port_, USE_OTA_VERSION);
+                "  Version: %d"
+#ifdef USE_OTA_ENCRYPTION
+                "\n  Encryption: %s"
+#endif
+                ,
+                network::get_use_address_to(addr_buf), this->port_, USE_OTA_VERSION
+#ifdef USE_OTA_ENCRYPTION_REQUIRED
+                ,
+                LOG_STR_LITERAL("required")
+#elif defined(USE_OTA_ENCRYPTION_PROVISIONED)
+                // A runtime provisioned key may not exist yet
+                ,
+                this->noise_context_().has_psk() ? LOG_STR_LITERAL("offered, plaintext accepted")
+                                                 : LOG_STR_LITERAL("offered once the api key is provisioned")
+#elif defined(USE_OTA_ENCRYPTION)
+                ,
+                LOG_STR_LITERAL("offered, plaintext accepted")
+#endif
+  );
 #ifdef USE_OTA_PASSWORD
   if (!this->password_.empty()) {
     ESP_LOGCONFIG(TAG, "  Password configured");
   }
 #endif
-#ifdef USE_OTA_ENCRYPTION
-  if (this->noise_ctx_.has_psk()) {
-    ESP_LOGCONFIG(TAG, "  Encryption configured");
-  }
-#endif
 #ifdef USE_OTA_PARTITIONS
   ESP_LOGCONFIG(TAG,
                 "  Partition access allowed\n"
@@ -154,10 +179,22 @@ static constexpr uint8_t CLIENT_FEATURE_SUPPORTS_COMPRESSION = 0x01;
 static constexpr uint8_t CLIENT_FEATURE_SUPPORTS_SHA256_AUTH = 0x02;
 static constexpr uint8_t CLIENT_FEATURE_SUPPORTS_EXTENDED_PROTOCOL = 0x04;
 static constexpr uint8_t CLIENT_FEATURE_SUPPORTS_NOISE = 0x08;
+// Noise needs the extended protocol: the prologue binds the 2-byte feature ack
+static constexpr uint8_t CLIENT_NOISE_FEATURES =
+    CLIENT_FEATURE_SUPPORTS_NOISE | CLIENT_FEATURE_SUPPORTS_EXTENDED_PROTOCOL;
 static constexpr uint8_t SERVER_FEATURE_SUPPORTS_COMPRESSION = 0x01;
 static constexpr uint8_t SERVER_FEATURE_SUPPORTS_PARTITION_ACCESS = 0x02;
 static constexpr uint8_t SERVER_FEATURE_SUPPORTS_NOISE = 0x04;
 
+inline bool ESPHomeOTAComponent::extended_proto_() const {
+#ifdef USE_OTA_ENCRYPTION_REQUIRED
+  // FEATURE_READ already refused every client without the extended protocol
+  return true;
+#else
+  return (this->ota_features_ & CLIENT_FEATURE_SUPPORTS_EXTENDED_PROTOCOL) != 0;
+#endif
+}
+
 void ESPHomeOTAComponent::handle_handshake_() {
   /// Handle the OTA handshake and authentication.
   ///
@@ -241,12 +278,9 @@ void ESPHomeOTAComponent::handle_handshake_() {
       this->ota_features_ = this->handshake_buf_[0];
       ESP_LOGV(TAG, "Features: 0x%02X", this->ota_features_);
 
-#ifdef USE_OTA_ENCRYPTION
-      // Fail closed: with a PSK configured the client must negotiate encryption
-      // (which requires the extended protocol); refuse plaintext uploads.
-      static constexpr uint8_t NOISE_REQUIRED_FEATURES =
-          CLIENT_FEATURE_SUPPORTS_NOISE | CLIENT_FEATURE_SUPPORTS_EXTENDED_PROTOCOL;
-      if (this->noise_ctx_.has_psk() && (this->ota_features_ & NOISE_REQUIRED_FEATURES) != NOISE_REQUIRED_FEATURES) {
+#ifdef USE_OTA_ENCRYPTION_REQUIRED
+      // `ota: encryption:` requires the client to negotiate encryption
+      if ((this->ota_features_ & CLIENT_NOISE_FEATURES) != CLIENT_NOISE_FEATURES) {
         ESP_LOGW(TAG, "Client does not support encryption");
         this->send_error_and_cleanup_(ota::OTA_RESPONSE_ERROR_ENCRYPTION_REQUIRED);
         return;
@@ -261,18 +295,21 @@ void ESPHomeOTAComponent::handle_handshake_() {
       // Compose the feature-ack response. When the client negotiates the extended protocol we emit
       // a 2-byte response (marker + server feature flags); otherwise we emit the single-byte
       // legacy response.
-      this->extended_proto_ = (this->ota_features_ & CLIENT_FEATURE_SUPPORTS_EXTENDED_PROTOCOL) != 0;
-      if (this->extended_proto_) {
+      if (this->extended_proto_()) {
         static_assert(HANDSHAKE_BUF_SIZE >= 2, "handshake_buf_ must hold the 2-byte extended-protocol feature ack");
         this->handshake_buf_[0] = ota::OTA_RESPONSE_FEATURE_FLAGS;
         this->handshake_buf_[1] = (supports_compression ? SERVER_FEATURE_SUPPORTS_COMPRESSION : 0);
 #ifdef USE_OTA_PARTITIONS
         this->handshake_buf_[1] |= SERVER_FEATURE_SUPPORTS_PARTITION_ACCESS;
 #endif
-#ifdef USE_OTA_ENCRYPTION
-        if (this->noise_ctx_.has_psk()) {
+#ifdef USE_OTA_ENCRYPTION_PROVISIONED
+        // A runtime provisioned key may not exist yet
+        if (this->noise_context_().has_psk()) {
           this->handshake_buf_[1] |= SERVER_FEATURE_SUPPORTS_NOISE;
         }
+#elif defined(USE_OTA_ENCRYPTION)
+        // A yaml key always exists: validation rejects the all-zeros key
+        this->handshake_buf_[1] |= SERVER_FEATURE_SUPPORTS_NOISE;
 #endif
       } else {
         this->handshake_buf_[0] =
@@ -284,15 +321,15 @@ void ESPHomeOTAComponent::handle_handshake_() {
     case OTAState::FEATURE_ACK: {
       static constexpr size_t STANDARD_PROTO_ACK_SIZE = 1;
       static constexpr size_t EXTENDED_PROTO_ACK_SIZE = 2;
-      const size_t ack_size = this->extended_proto_ ? EXTENDED_PROTO_ACK_SIZE : STANDARD_PROTO_ACK_SIZE;
+      const size_t ack_size = this->extended_proto_() ? EXTENDED_PROTO_ACK_SIZE : STANDARD_PROTO_ACK_SIZE;
       if (!this->try_write_(ack_size, LOG_STR("ack feature"))) {
         return;
       }
 #ifdef USE_OTA_ENCRYPTION
-      // With a PSK configured the rest of the session runs inside the noise
-      // transport; the client sends the first handshake frame next, so there
-      // is nothing to do until data arrives.
-      if (this->noise_ctx_.has_psk()) {
+      // Latch the offer actually sent: a key activating between the two
+      // states must not start a session the client never expects
+      if ((this->handshake_buf_[1] & SERVER_FEATURE_SUPPORTS_NOISE) != 0 &&
+          (this->ota_features_ & CLIENT_NOISE_FEATURES) == CLIENT_NOISE_FEATURES) {
         // handshake_buf_ still holds the feature ack composed above; a
         // would-block re-entry lands here without rebuilding it
         if (!this->noise_start_session_(this->handshake_buf_[1])) {
@@ -412,7 +449,7 @@ void ESPHomeOTAComponent::handle_data_() {
   // Acknowledge auth OK - 1 byte
   this->data_write_byte_(ota::OTA_RESPONSE_AUTH_OK);
 
-  if (this->extended_proto_) {
+  if (this->extended_proto_()) {
     // Read ota type, 1 byte
     if (!this->data_readall_(buf, 1)) {
       this->log_read_error_(LOG_STR("OTA type"));

esphome/components/esphome/ota/ota_esphome.h

@@ -44,8 +44,9 @@ class ESPHomeOTAComponent final : public ota::OTAComponent {
   }
 #endif  // USE_OTA_PASSWORD
 
-#ifdef USE_OTA_ENCRYPTION
-  void set_noise_psk(noise::psk_t psk) { this->noise_ctx_.set_psk(psk); }
+#if defined(USE_OTA_ENCRYPTION) && !defined(USE_OTA_ENCRYPTION_FROM_API)
+  /// psk points at 32 bytes that live in flash for the life of the program
+  void set_noise_psk(const uint8_t *psk) { this->noise_ctx_.set_psk(psk); }
 #endif
 
   /// Manually set the port OTA should listen on
@@ -85,9 +86,12 @@ class ESPHomeOTAComponent final : public ota::OTAComponent {
     bool writing{false};    // a produced handshake frame is still being flushed
     uint8_t frame_buf[noise::FRAME_HEADER_SIZE + 1 + noise::MAX_HANDSHAKE_SIZE];
   };
+  // The api server's live context when the api has encryption, else our own
+  const noise::NoiseContext &noise_context_() const;
   bool noise_start_session_(uint8_t server_feature_flags);
   bool handle_noise_handshake_();
   bool noise_try_read_frame_();
+  size_t noise_frame_payload_len_(const uint8_t *header, size_t min_len, size_t max_len);
   bool noise_try_write_frame_();
   void noise_send_reject_(const LogString *reason);
   ssize_t noise_decrypt_(uint8_t *buf, size_t len);
@@ -144,7 +148,9 @@ class ESPHomeOTAComponent final : public ota::OTAComponent {
   std::unique_ptr<uint8_t[]> auth_buf_;
 #endif  // USE_OTA_PASSWORD
 #ifdef USE_OTA_ENCRYPTION
+#ifndef USE_OTA_ENCRYPTION_FROM_API
   noise::NoiseContext noise_ctx_;
+#endif
   std::unique_ptr<NoiseSession> noise_;
 #endif  // USE_OTA_ENCRYPTION
 
@@ -166,6 +172,8 @@ class ESPHomeOTAComponent final : public ota::OTAComponent {
                 "OTA_BUFFER_SIZE must fit a full encrypted data frame");
 #endif
   static constexpr uint8_t MAGIC_BYTES[5] = {0x6C, 0x26, 0xF7, 0x5C, 0x45};
+  // Derived from the feature byte; storing it would pad the trailing bytes
+  bool extended_proto_() const;
 #ifdef USE_OTA_PARTITIONS
   uint32_t running_app_offset_{0};
   size_t running_app_size_{0};
@@ -179,7 +187,6 @@ class ESPHomeOTAComponent final : public ota::OTAComponent {
   uint8_t auth_buf_pos_{0};
   uint8_t auth_type_{0};  // Store auth type to know which hasher to use
 #endif                    // USE_OTA_PASSWORD
-  bool extended_proto_{false};
 };
 
 }  // namespace esphome

esphome/components/esphome/ota/ota_esphome_noise.cpp

@@ -3,6 +3,7 @@
 #ifdef USE_OTA_ENCRYPTION
 #include "esphome/components/noise/noise.h"
 #include "esphome/components/ota/ota_backend.h"
+#include "esphome/core/hal.h"
 #include "esphome/core/log.h"
 
 #include <cstring>
@@ -40,24 +41,17 @@ ESPHomeOTAComponent::NoiseSession::~NoiseSession() {
  *   "NoiseOTAInit" | magic(5) | OK,version | client_features | FEATURE_FLAGS,server_flags
  */
 bool ESPHomeOTAComponent::noise_start_session_(uint8_t server_feature_flags) {
+  // A provisioned key cleared between the offer and here is not guarded: the
+  // session runs on the zero key load_psk fills in and fails the client's MAC.
+  // Default-init: the frame buffer is written before it is read
   // NOLINTNEXTLINE(clang-analyzer-cplusplus.NewDeleteLeaks)
-  this->noise_ = std::unique_ptr<NoiseSession>(new (std::nothrow) NoiseSession());
-  if (this->noise_ == nullptr) {
-    ESP_LOGW(TAG, "Session allocation failed");
-    this->cleanup_connection_();
-    return false;
-  }
-
+  this->noise_ = std::unique_ptr<NoiseSession>(new (std::nothrow) NoiseSession);
   static constexpr size_t PROLOGUE_ACK_LEN = 2;  // OTA_RESPONSE_OK + version
   static constexpr size_t PROLOGUE_CLIENT_FEATURES_LEN = 1;
   static constexpr size_t PROLOGUE_FEATURE_ACK_LEN = 2;  // OTA_RESPONSE_FEATURE_FLAGS + server flags
   uint8_t prologue[OTA_NOISE_PROLOGUE_INIT_LEN + sizeof(MAGIC_BYTES) + PROLOGUE_ACK_LEN + PROLOGUE_CLIENT_FEATURES_LEN +
                    PROLOGUE_FEATURE_ACK_LEN];
-#ifdef USE_ESP8266
-  memcpy_P(prologue, OTA_NOISE_PROLOGUE_INIT, OTA_NOISE_PROLOGUE_INIT_LEN);
-#else
-  std::memcpy(prologue, OTA_NOISE_PROLOGUE_INIT, OTA_NOISE_PROLOGUE_INIT_LEN);
-#endif
+  progmem_memcpy(prologue, OTA_NOISE_PROLOGUE_INIT, OTA_NOISE_PROLOGUE_INIT_LEN);
   uint8_t *p = prologue + OTA_NOISE_PROLOGUE_INIT_LEN;
   // Magic bytes, already validated in MAGIC_READ
   std::memcpy(p, MAGIC_BYTES, sizeof(MAGIC_BYTES));
@@ -71,9 +65,13 @@ bool ESPHomeOTAComponent::noise_start_session_(uint8_t server_feature_flags) {
   *p++ = ota::OTA_RESPONSE_FEATURE_FLAGS;
   *p++ = server_feature_flags;
 
-  int err = this->noise_->handshake.init(this->noise_ctx_.get_psk(), prologue, sizeof(prologue));
+  // The caller only starts a session when the context holds a key
+  int err = this->noise_ == nullptr ? NOISE_ERROR_NO_MEMORY
+                                    : this->noise_->handshake.init(this->noise_context_(), prologue, sizeof(prologue));
   if (err != 0) {
-    ESP_LOGW(TAG, "Handshake init: %s", LOG_STR_ARG(noise::noise_err_to_logstr(err)));
+    // Raw noise codes throughout: the name table would cost flash in builds
+    // where only the OTA uses noise
+    ESP_LOGW(TAG, "Session init: %d", err);
     this->cleanup_connection_();
     return false;
   }
@@ -105,14 +103,16 @@ bool ESPHomeOTAComponent::handle_noise_handshake_() {
         s.frame_pos = 0;
         s.frame_len = 0;
         if (s.frame_buf[noise::FRAME_HEADER_SIZE] != noise::HANDSHAKE_STATUS_OK) {
-          ESP_LOGW(TAG, "Bad handshake error byte: %u", s.frame_buf[noise::FRAME_HEADER_SIZE]);
+          ESP_LOGW(TAG, "Client rejected the handshake: %u", s.frame_buf[noise::FRAME_HEADER_SIZE]);
           this->cleanup_connection_();
           return false;
         }
         int err = s.handshake.read_message(s.frame_buf + noise::FRAME_HEADER_SIZE + 1, payload_len - 1);
         if (err != 0) {
-          ESP_LOGW(TAG, "Handshake read: %s", LOG_STR_ARG(noise::noise_err_to_logstr(err)));
-          this->noise_send_reject_(noise::reject_reason_for(err));
+          // A MAC failure here almost always means the uploader has a different key
+          const LogString *reason = noise::reject_reason_for(err);
+          ESP_LOGW(TAG, "Handshake read: %s (%d)", LOG_STR_ARG(reason), err);
+          this->noise_send_reject_(reason);
           this->cleanup_connection_();
           return false;
         }
@@ -123,7 +123,7 @@ bool ESPHomeOTAComponent::handle_noise_handshake_() {
         int err =
             s.handshake.write_message(s.frame_buf + noise::FRAME_HEADER_SIZE + 1, noise::MAX_HANDSHAKE_SIZE, msg_len);
         if (err != 0) {
-          ESP_LOGW(TAG, "Handshake write: %s", LOG_STR_ARG(noise::noise_err_to_logstr(err)));
+          ESP_LOGW(TAG, "Handshake write: %d", err);
           this->cleanup_connection_();
           return false;
         }
@@ -138,7 +138,7 @@ bool ESPHomeOTAComponent::handle_noise_handshake_() {
       case noise::NoiseResponderHandshake::Action::ACTION_SPLIT: {
         int err = s.handshake.split(s.send_cipher, s.recv_cipher);
         if (err != 0) {
-          ESP_LOGW(TAG, "Handshake split: %s", LOG_STR_ARG(noise::noise_err_to_logstr(err)));
+          ESP_LOGW(TAG, "Handshake split: %d", err);
           this->cleanup_connection_();
           return false;
         }
@@ -154,33 +154,41 @@ bool ESPHomeOTAComponent::handle_noise_handshake_() {
   }
 }
 
+/// Payload length from a frame header, or 0 (logged) when the indicator or
+/// the length is out of range. Callers pass min_len >= 1 so 0 is never valid.
+size_t ESPHomeOTAComponent::noise_frame_payload_len_(const uint8_t *header, size_t min_len, size_t max_len) {
+  const size_t payload_len = encode_uint16(header[1], header[2]);
+  if (header[0] != noise::FRAME_INDICATOR || payload_len < min_len || payload_len > max_len) {
+    ESP_LOGW(TAG, "Bad frame: 0x%02X, %zu bytes", header[0], payload_len);
+    return 0;
+  }
+  return payload_len;
+}
+
 /// Non-blocking read of one handshake frame into the session buffer.
 bool ESPHomeOTAComponent::noise_try_read_frame_() {
   NoiseSession &s = *this->noise_;
-  while (s.frame_pos < noise::FRAME_HEADER_SIZE) {
-    ssize_t read = this->client_->read(s.frame_buf + s.frame_pos, noise::FRAME_HEADER_SIZE - s.frame_pos);
-    if (!this->handle_read_error_(read, LOG_STR("read noise header"))) {
-      return false;
+  while (true) {
+    // The header first, then the body once the header says how long it is
+    const uint16_t want = s.frame_len == 0 ? noise::FRAME_HEADER_SIZE : s.frame_len;
+    if (s.frame_pos < want) {
+      ssize_t read = this->client_->read(s.frame_buf + s.frame_pos, want - s.frame_pos);
+      if (!this->handle_read_error_(read, LOG_STR("read noise"))) {
+        return false;
+      }
+      s.frame_pos += read;
+      continue;
     }
-    s.frame_pos += read;
-  }
-  if (s.frame_len == 0) {
-    const uint16_t payload_len = encode_uint16(s.frame_buf[1], s.frame_buf[2]);
-    if (s.frame_buf[0] != noise::FRAME_INDICATOR || payload_len < 1 || payload_len > 1 + noise::MAX_HANDSHAKE_SIZE) {
-      ESP_LOGW(TAG, "Bad handshake frame: 0x%02X, %u bytes", s.frame_buf[0], payload_len);
+    if (s.frame_len != 0) {
+      return true;
+    }
+    const size_t payload_len = this->noise_frame_payload_len_(s.frame_buf, 1, 1 + noise::MAX_HANDSHAKE_SIZE);
+    if (payload_len == 0) {
       this->cleanup_connection_();
       return false;
     }
     s.frame_len = noise::FRAME_HEADER_SIZE + payload_len;
   }
-  while (s.frame_pos < s.frame_len) {
-    ssize_t read = this->client_->read(s.frame_buf + s.frame_pos, s.frame_len - s.frame_pos);
-    if (!this->handle_read_error_(read, LOG_STR("read noise frame"))) {
-      return false;
-    }
-    s.frame_pos += read;
-  }
-  return true;
 }
 
 /// Non-blocking write of the pending session-buffer frame.
@@ -214,7 +222,7 @@ ssize_t ESPHomeOTAComponent::noise_decrypt_(uint8_t *buf, size_t len) {
   noise_buffer_set_inout(mbuf, buf, len, len);
   int err = noise_cipherstate_decrypt(this->noise_->recv_cipher, &mbuf);
   if (err != 0) {
-    ESP_LOGW(TAG, "Decrypt: %s", LOG_STR_ARG(noise::noise_err_to_logstr(err)));
+    ESP_LOGW(TAG, "Decrypt: %d", err);
     return -1;
   }
   return mbuf.size;
@@ -229,9 +237,8 @@ ssize_t ESPHomeOTAComponent::noise_read_frame_blocking_(uint8_t *buf, size_t min
   if (!this->readall_(header, sizeof(header))) {
     return -1;
   }
-  const size_t ciphertext_len = encode_uint16(header[1], header[2]);
-  if (header[0] != noise::FRAME_INDICATOR || ciphertext_len < min_ciphertext || ciphertext_len > max_ciphertext) {
-    ESP_LOGW(TAG, "Bad frame: 0x%02X, %zu bytes", header[0], ciphertext_len);
+  const size_t ciphertext_len = this->noise_frame_payload_len_(header, min_ciphertext, max_ciphertext);
+  if (ciphertext_len == 0) {
     return -1;
   }
   if (!this->readall_(buf, ciphertext_len)) {
@@ -267,7 +274,7 @@ bool ESPHomeOTAComponent::noise_write_byte_(uint8_t byte) {
   noise_buffer_set_inout(mbuf, frame + noise::FRAME_HEADER_SIZE, 1, 1 + noise::MAC_SIZE);
   int err = noise_cipherstate_encrypt(this->noise_->send_cipher, &mbuf);
   if (err != 0) {
-    ESP_LOGW(TAG, "Encrypt: %s", LOG_STR_ARG(noise::noise_err_to_logstr(err)));
+    ESP_LOGW(TAG, "Encrypt: %d", err);
     return false;
   }
   noise::write_frame_header(frame, mbuf.size);

esphome/components/espnow/__init__.py

@@ -3,6 +3,7 @@ from typing import Any
 from esphome import automation, core
 import esphome.codegen as cg
 from esphome.components import wifi
+from esphome.components.esp32 import VARIANT_ESP32P4, get_esp32_variant
 from esphome.components.udp import CONF_ON_RECEIVE
 import esphome.config_validation as cv
 from esphome.const import (
@@ -17,6 +18,7 @@ from esphome.const import (
 )
 from esphome.core import CORE, HexInt
 from esphome.cpp_generator import MockObj, TemplateArgsType
+import esphome.final_validate as fv
 from esphome.types import ConfigType
 
 CODEOWNERS = ["@jesserockz"]
@@ -132,6 +134,24 @@ CONFIG_SCHEMA = cv.All(
 )
 
 
+def _validate_variant(config: ConfigType) -> ConfigType:
+    # ESP-NOW rides the Wi-Fi PHY. Radio-less esp32 variants have no native
+    # ESP-NOW; only the ESP32-P4 has a path, via the esp32_hosted shim that
+    # supplies the esp_now_* symbols. Fail here with a clear message instead of
+    # letting the build reach an "undefined reference to esp_now_*" link error.
+    variant = get_esp32_variant()
+    if wifi.variant_has_wifi(variant):
+        return config
+    if variant != VARIANT_ESP32P4:
+        raise cv.Invalid(f"ESP-NOW is not supported on {variant} (no Wi-Fi radio)")
+    if "esp32_hosted" not in fv.full_config.get():
+        raise cv.Invalid(f"ESP-NOW on {variant} requires the esp32_hosted component")
+    return config
+
+
+FINAL_VALIDATE_SCHEMA = _validate_variant
+
+
 async def _trigger_to_code(config: ConfigType) -> MockObj:
     if address := config.get(CONF_ADDRESS):
         address = address.parts

esphome/components/mdns/__init__.py

@@ -192,6 +192,8 @@ async def to_code(config: ConfigType) -> None:
     if CORE.using_arduino:
         if CORE.is_esp8266:
             cg.add_library("ESP8266mDNS", None)
+            # No MDNS global in the build; mdns_esp8266.cpp owns a guarded MDNSResponder
+            cg.add_build_flag("-DNO_GLOBAL_MDNS")
         elif CORE.is_rp2:
             cg.add_library("LEAmDNS", None)
 

esphome/components/mdns/mdns_esp8266.cpp

@@ -13,8 +13,47 @@
 
 namespace esphome::mdns {
 
+// Main-loop calls into LEAmDNS that send (update() and close(); begin(), addService() and
+// the scheduled restart never reach a send) can yield inside UdpContext::sendTimeout(); a
+// packet arriving then re-enters LEAmDNS from lwIP on the same UdpContext and both sides
+// free the same tx pbufs (#18760). Received packets stay queued during such a call and are
+// processed from the main loop afterwards.
+class GuardedMDNSResponder : public ::esp8266::MDNSImplementation::MDNSResponder {
+ public:
+  void update_guarded() { this->run_guarded_(&GuardedMDNSResponder::update); }
+  void close_guarded() { this->run_guarded_(&GuardedMDNSResponder::close); }
+
+ private:
+  void run_guarded_(bool (GuardedMDNSResponder::*fn)()) {
+    UdpContext *ctx = this->m_pUDPContext;
+    if (ctx == nullptr) {
+      (this->*fn)();
+      return;
+    }
+    // Set every time: a restart replaces the context together with its stock handler. Only
+    // begin() and the scheduled netif callback restart, never update() or close(), so the
+    // context cannot change underneath this call.
+    ctx->onRx([this]() {
+      if (!this->in_loop_call_) {
+        this->_callProcess();
+      }
+    });
+    this->in_loop_call_ = true;
+    (this->*fn)();
+    // close() releases the context; a yield in here queues further packets for this loop too
+    while (this->m_pUDPContext != nullptr && this->m_pUDPContext->next()) {
+      this->_parseMessage();
+    }
+    this->in_loop_call_ = false;
+  }
+
+  volatile bool in_loop_call_{false};
+};
+
+static GuardedMDNSResponder mdns_responder;  // NOLINT(cppcoreguidelines-avoid-non-const-global-variables)
+
 static void register_esp8266(MDNSComponent *, StaticVector<MDNSService, MDNS_SERVICE_COUNT> &services) {
-  MDNS.begin(App.get_name().c_str());
+  mdns_responder.begin(App.get_name().c_str());
 
   for (const auto &service : services) {
     // Strip the leading underscore from the proto and service_type. While it is
@@ -30,10 +69,10 @@ static void register_esp8266(MDNSComponent *, StaticVector<MDNSService, MDNS_SER
       service_type++;
     }
     uint16_t port = service.port.value();
-    MDNS.addService(FPSTR(service_type), FPSTR(proto), port);
+    mdns_responder.addService(FPSTR(service_type), FPSTR(proto), port);
     for (const auto &record : service.txt_records) {
-      MDNS.addServiceTxt(FPSTR(service_type), FPSTR(proto), FPSTR(MDNS_STR_ARG(record.key)),
-                         FPSTR(MDNS_STR_ARG(record.value)));
+      mdns_responder.addServiceTxt(FPSTR(service_type), FPSTR(proto), FPSTR(MDNS_STR_ARG(record.key)),
+                                   FPSTR(MDNS_STR_ARG(record.value)));
     }
   }
 }
@@ -52,7 +91,7 @@ void MDNSComponent::start_polling_window_() {
     if (wifi->is_roaming() || (!wifi->is_connected() && !wifi->is_ap_active()))
       return;
 #endif
-    MDNS.update();
+    mdns_responder.update_guarded();
   });
   this->set_timeout(MDNS_POLL_STOP_ID, MDNS_POLL_WINDOW_MS, [this]() { this->cancel_interval(MDNS_POLL_ID); });
 }
@@ -81,7 +120,7 @@ void MDNSComponent::on_ip_state(const network::IPAddresses &ips, const network::
 #endif
 
 void MDNSComponent::on_shutdown() {
-  MDNS.close();
+  mdns_responder.close_guarded();
   delay(10);
 }
 

esphome/components/mipi_dsi/mipi_dsi.cpp

@@ -35,8 +35,8 @@ void MipiDsi::setup() {
       .bus_id = 0,  // index from 0, specify the DSI host to use
       .num_data_lanes =
           this->lanes_,  // Number of data lanes to use, can't set a value that exceeds the chip's capability
-      .phy_clk_src = MIPI_DSI_PHY_CLK_SRC_DEFAULT,  // Clock source for the DPHY
-      .lane_bit_rate_mbps = this->lane_bit_rate_,   // Bit rate of the data lanes, in Mbps
+      // phy_clk_src left at 0 to enable runtime auto-select.
+      .lane_bit_rate_mbps = this->lane_bit_rate_,  // Bit rate of the data lanes, in Mbps
   };
   auto err = esp_lcd_new_dsi_bus(&bus_config, &this->bus_handle_);
   if (err != ESP_OK) {

esphome/components/noise/__init__.py

@@ -4,7 +4,9 @@ from typing import Any
 
 import esphome.codegen as cg
 import esphome.config_validation as cv
-from esphome.const import CONF_KEY
+from esphome.const import CONF_ENCRYPTION, CONF_KEY
+from esphome.core import ID
+from esphome.cpp_generator import MockObj
 from esphome.types import ConfigType
 
 CODEOWNERS = ["@esphome/core"]
@@ -23,6 +25,14 @@ def validate_encryption_key(value: Any) -> str:
 
     if len(decoded) != 32:
         raise cv.Invalid("Encryption key must be base64 and 32 bytes long")
+    if not any(decoded):
+        # The device treats the all-zeros key as no key at all (it is the
+        # provisioning sentinel), so it must never reach a build
+        raise cv.Invalid(
+            f"The all-zeros {CONF_KEY} is reserved and provides no protection; "
+            f"omit the {CONF_KEY} to provision it at runtime, or generate a real "
+            "key with: openssl rand -base64 32"
+        )
 
     # Return original data for roundtrip conversion
     return value
@@ -45,15 +55,6 @@ def decode_encryption_key(value: str) -> bytes:
     return decoded
 
 
-def is_reserved_key(value: str) -> bool:
-    """Whether the key is the reserved all-zeros provisioning sentinel.
-
-    The device treats it as no key configured, so consumers that require a
-    real key must reject it.
-    """
-    return not any(decode_encryption_key(value))
-
-
 ENCRYPTION_SCHEMA = cv.Schema(
     {
         cv.Optional(CONF_KEY): cv.sensitive(validate_encryption_key),
@@ -61,6 +62,21 @@ ENCRYPTION_SCHEMA = cv.Schema(
 )
 
 
+def static_encryption_key(conf: ConfigType) -> str | None:
+    """The build time key of a component config; None without one or when
+    the key is provisioned at runtime."""
+    return (conf.get(CONF_ENCRYPTION) or {}).get(CONF_KEY) or None
+
+
+def new_psk_progmem(parent_id: ID, key: str) -> MockObj:
+    """Emit the decoded key as a PROGMEM array; the component keeps a pointer
+    so the key never occupies RAM."""
+    return cg.progmem_array(
+        ID(f"{parent_id.id}_psk", is_declaration=True, type=cg.uint8),
+        list(decode_encryption_key(key)),
+    )
+
+
 def encryption_schema(config: ConfigType | None) -> ConfigType:
     # A bare `encryption:` block is valid; a missing key means the consumer
     # falls back to its keyless behavior (api provisioning, ota inheriting
@@ -72,12 +88,12 @@ def encryption_schema(config: ConfigType | None) -> ConfigType:
 
 async def to_code(config: ConfigType) -> None:
     cg.add_define("USE_NOISE")
-    cg.add_library("esphome/noise-c", "0.1.21")
+    cg.add_library("esphome/noise-c", "0.1.24")
     # noise-c depends on libsodium, but declaring it here too lets the
     # library manager see the full set up front instead of discovering
     # libsodium only after noise-c has downloaded, so the two can download
     # in parallel. The version must match noise-c's library.json.
-    cg.add_library("esphome/libsodium", "1.10021.4")
+    cg.add_library("esphome/libsodium", "1.10021.6")
     # Enable optimized memzero/memcmp in libsodium instead of volatile byte loops
     cg.add_build_flag("-DHAVE_WEAK_SYMBOLS=1")
     cg.add_build_flag("-DHAVE_INLINE_ASM=1")

esphome/components/noise/noise.cpp

@@ -1,5 +1,6 @@
 #include "noise.h"
 #ifdef USE_NOISE
+#include "esphome/core/hal.h"
 #include "esphome/core/log.h"
 
 #include <algorithm>
@@ -15,6 +16,14 @@ namespace esphome::noise {
 
 static const char *const TAG = "noise";
 
+void NoiseContext::load_psk(psk_t &out) const {
+  if (this->psk_ == nullptr) {
+    out.fill(0);
+    return;
+  }
+  progmem_memcpy(out.data(), this->psk_, out.size());
+}
+
 const LogString *noise_err_to_logstr(int err) {
   if (err == NOISE_ERROR_NO_MEMORY)
     return LOG_STR("NO_MEMORY");

esphome/components/noise/noise.h

@@ -23,16 +23,16 @@ class NoiseContext {
     }
     return acc == 0;
   }
-  void set_psk(psk_t psk) {
-    this->psk_ = psk;
-    this->has_psk_ = !is_all_zeros(psk);
-  }
-  const psk_t &get_psk() const { return this->psk_; }
-  bool has_psk() const { return this->has_psk_; }
+  /// psk points at 32 bytes that outlive the context (PROGMEM or caller owned
+  /// RAM); nullptr means no key. Runtime callers map the all-zeros key to
+  /// nullptr themselves; validation keeps it out of yaml.
+  void set_psk(const uint8_t *psk) { this->psk_ = psk; }
+  /// Copy the key out (flash-aware on ESP8266); all zeros when none is set.
+  void load_psk(psk_t &out) const;
+  bool has_psk() const { return this->psk_ != nullptr; }
 
  protected:
-  psk_t psk_{};
-  bool has_psk_{false};
+  const uint8_t *psk_{nullptr};
 };
 
 /// Convert a noise error code to a readable error

esphome/components/noise/noise_handshake.cpp

@@ -20,7 +20,7 @@ NoiseResponderHandshake::~NoiseResponderHandshake() {
   }
 }
 
-int NoiseResponderHandshake::init(const psk_t &psk, const uint8_t *prologue, size_t prologue_len) {
+int NoiseResponderHandshake::init(const NoiseContext &ctx, const uint8_t *prologue, size_t prologue_len) {
   if (this->handshake_ != nullptr) {
     noise_handshakestate_free(this->handshake_);
     this->handshake_ = nullptr;
@@ -44,6 +44,9 @@ int NoiseResponderHandshake::init(const psk_t &psk, const uint8_t *prologue, siz
     HANDSHAKE_STEP_LOG("noise_handshakestate_new_by_id", err);
     return err;
   }
+  // noise-c keeps its own copy, so the key only passes through the stack here
+  psk_t psk;
+  ctx.load_psk(psk);
   err = noise_handshakestate_set_pre_shared_key(this->handshake_, psk.data(), psk.size());
   if (err != 0) {
     HANDSHAKE_STEP_LOG("noise_handshakestate_set_pre_shared_key", err);

esphome/components/noise/noise_handshake.h

@@ -36,9 +36,9 @@ class NoiseResponderHandshake {
   NoiseResponderHandshake(const NoiseResponderHandshake &) = delete;
   NoiseResponderHandshake &operator=(const NoiseResponderHandshake &) = delete;
 
-  /// Create and start the handshake with the given PSK and prologue. A
-  /// repeated call frees the previous handshake state and starts over.
-  [[nodiscard]] int init(const psk_t &psk, const uint8_t *prologue, size_t prologue_len);
+  /// Create and start the handshake with the context's PSK and the prologue.
+  /// A repeated call frees the previous handshake state and starts over.
+  [[nodiscard]] int init(const NoiseContext &ctx, const uint8_t *prologue, size_t prologue_len);
   /// ACTION_FAILED is the catch-all: returned before init(), after split()
   /// has released the state, and when noise-c reports a failed handshake.
   [[nodiscard]] Action action() const;

esphome/components/template/climate/__init__.py

@@ -0,0 +1,465 @@
+from esphome import automation
+import esphome.codegen as cg
+from esphome.components import climate, sensor
+from esphome.components.climate import climate_ns
+import esphome.config_validation as cv
+from esphome.const import (
+    CONF_ACTION,
+    CONF_CURRENT_TEMPERATURE,
+    CONF_CUSTOM_FAN_MODE,
+    CONF_CUSTOM_FAN_MODES,
+    CONF_CUSTOM_PRESET,
+    CONF_CUSTOM_PRESETS,
+    CONF_FAN_MODE,
+    CONF_HUMIDITY_SENSOR,
+    CONF_ID,
+    CONF_INITIAL_STATE,
+    CONF_MODE,
+    CONF_OPTIMISTIC,
+    CONF_PRESET,
+    CONF_RESTORE_MODE,
+    CONF_SENSOR,
+    CONF_SUPPORTED_FAN_MODES,
+    CONF_SUPPORTED_MODES,
+    CONF_SUPPORTED_PRESETS,
+    CONF_SUPPORTED_SWING_MODES,
+    CONF_SWING_MODE,
+    CONF_TARGET_TEMPERATURE,
+    CONF_TARGET_TEMPERATURE_HIGH,
+    CONF_TARGET_TEMPERATURE_LOW,
+)
+from esphome.core import ID
+from esphome.cpp_generator import MockObj, TemplateArgsType
+from esphome.types import ConfigType
+
+from .. import template_ns
+
+CONF_CURRENT_HUMIDITY = "current_humidity"
+CONF_TARGET_HUMIDITY = "target_humidity"
+CONF_SUPPORTS_ACTION = "supports_action"
+CONF_SUPPORTS_TWO_POINT_TARGET_TEMPERATURE = "supports_two_point_target_temperature"
+CONF_SUPPORTS_TARGET_HUMIDITY = "supports_target_humidity"
+CONF_SUPPORTS_CURRENT_TEMPERATURE = "supports_current_temperature"
+CONF_SUPPORTS_CURRENT_HUMIDITY = "supports_current_humidity"
+CONF_SET_MODE_ACTION = "set_mode_action"
+CONF_SET_TARGET_TEMPERATURE_ACTION = "set_target_temperature_action"
+CONF_SET_TARGET_TEMPERATURE_LOW_ACTION = "set_target_temperature_low_action"
+CONF_SET_TARGET_TEMPERATURE_HIGH_ACTION = "set_target_temperature_high_action"
+CONF_SET_TARGET_HUMIDITY_ACTION = "set_target_humidity_action"
+CONF_SET_FAN_MODE_ACTION = "set_fan_mode_action"
+CONF_SET_CUSTOM_FAN_MODE_ACTION = "set_custom_fan_mode_action"
+CONF_SET_SWING_MODE_ACTION = "set_swing_mode_action"
+CONF_SET_PRESET_ACTION = "set_preset_action"
+CONF_SET_CUSTOM_PRESET_ACTION = "set_custom_preset_action"
+
+TemplateClimate = template_ns.class_("TemplateClimate", climate.Climate, cg.Component)
+TemplateClimatePublishAction = template_ns.class_(
+    "TemplateClimatePublishAction",
+    automation.Action,
+    cg.Parented.template(TemplateClimate),
+)
+
+TemplateClimateRestoreMode = template_ns.enum(
+    "TemplateClimateRestoreMode", is_class=True
+)
+CLIMATE_RESTORE_MODES = {
+    "NO_RESTORE": TemplateClimateRestoreMode.TEMPLATE_CLIMATE_RESTORE_MODE_NO_RESTORE,
+    "RESTORE": TemplateClimateRestoreMode.TEMPLATE_CLIMATE_RESTORE_MODE_RESTORE,
+}
+
+# Per-field actions that forward a requested value on. The third item is the type of `x`.
+SET_ACTIONS = (
+    (CONF_SET_MODE_ACTION, "get_set_mode_trigger", climate.ClimateMode),
+    (
+        CONF_SET_TARGET_TEMPERATURE_ACTION,
+        "get_set_target_temperature_trigger",
+        cg.float_,
+    ),
+    (
+        CONF_SET_TARGET_TEMPERATURE_LOW_ACTION,
+        "get_set_target_temperature_low_trigger",
+        cg.float_,
+    ),
+    (
+        CONF_SET_TARGET_TEMPERATURE_HIGH_ACTION,
+        "get_set_target_temperature_high_trigger",
+        cg.float_,
+    ),
+    (CONF_SET_TARGET_HUMIDITY_ACTION, "get_set_target_humidity_trigger", cg.float_),
+    (CONF_SET_FAN_MODE_ACTION, "get_set_fan_mode_trigger", climate.ClimateFanMode),
+    (
+        CONF_SET_CUSTOM_FAN_MODE_ACTION,
+        "get_set_custom_fan_mode_trigger",
+        cg.StringRef,
+    ),
+    (
+        CONF_SET_SWING_MODE_ACTION,
+        "get_set_swing_mode_trigger",
+        climate.ClimateSwingMode,
+    ),
+    (CONF_SET_PRESET_ACTION, "get_set_preset_trigger", climate.ClimatePreset),
+    (CONF_SET_CUSTOM_PRESET_ACTION, "get_set_custom_preset_trigger", cg.StringRef),
+)
+
+# supports_* keys have no default so that an omitted key can mean "derive it from the sensor or
+# set action that makes the trait useful", which is not expressible once a default fills it in.
+DERIVED_SUPPORTS = (
+    (CONF_SUPPORTS_CURRENT_TEMPERATURE, (CONF_SENSOR,)),
+    (CONF_SUPPORTS_CURRENT_HUMIDITY, (CONF_HUMIDITY_SENSOR,)),
+    (
+        CONF_SUPPORTS_TWO_POINT_TARGET_TEMPERATURE,
+        (
+            CONF_SET_TARGET_TEMPERATURE_LOW_ACTION,
+            CONF_SET_TARGET_TEMPERATURE_HIGH_ACTION,
+        ),
+    ),
+    (CONF_SUPPORTS_TARGET_HUMIDITY, (CONF_SET_TARGET_HUMIDITY_ACTION,)),
+)
+
+
+# Custom fan modes/presets are opaque user-defined strings with no build-time correctness check
+# elsewhere (Climate::set_supported_custom_fan_modes()/set_supported_custom_presets() don't block
+# empty entries), so reject empty ones here -- they could never be selected at runtime anyway.
+validate_custom_climate_string = cv.All(cv.string_strict, cv.Length(min=1))
+
+
+def _validate_two_point(config: ConfigType) -> ConfigType:
+    has_low = CONF_TARGET_TEMPERATURE_LOW in config
+    has_high = CONF_TARGET_TEMPERATURE_HIGH in config
+    if has_low != has_high:
+        raise cv.Invalid(
+            f"'{CONF_TARGET_TEMPERATURE_LOW}' and '{CONF_TARGET_TEMPERATURE_HIGH}' must be used together"
+        )
+    if (has_low or has_high) and CONF_TARGET_TEMPERATURE in config:
+        raise cv.Invalid(
+            f"'{CONF_TARGET_TEMPERATURE}' cannot be used together with "
+            f"'{CONF_TARGET_TEMPERATURE_LOW}'/'{CONF_TARGET_TEMPERATURE_HIGH}'"
+        )
+    return config
+
+
+def _validate_set_actions(config: ConfigType) -> ConfigType:
+    has_low = CONF_SET_TARGET_TEMPERATURE_LOW_ACTION in config
+    has_high = CONF_SET_TARGET_TEMPERATURE_HIGH_ACTION in config
+    if has_low != has_high:
+        raise cv.Invalid(
+            f"'{CONF_SET_TARGET_TEMPERATURE_LOW_ACTION}' and "
+            f"'{CONF_SET_TARGET_TEMPERATURE_HIGH_ACTION}' must be used together"
+        )
+    if (has_low or has_high) and CONF_SET_TARGET_TEMPERATURE_ACTION in config:
+        raise cv.Invalid(
+            f"'{CONF_SET_TARGET_TEMPERATURE_ACTION}' cannot be used together with "
+            f"'{CONF_SET_TARGET_TEMPERATURE_LOW_ACTION}'/'{CONF_SET_TARGET_TEMPERATURE_HIGH_ACTION}'"
+        )
+    return config
+
+
+def _resolve_supports(config: ConfigType) -> ConfigType:
+    # An explicit true stays valid without either, since climate.template.publish can report the
+    # value; an explicit false that contradicts the configuration is an error, not a silent override.
+    for key, sources in DERIVED_SUPPORTS:
+        configured = [source for source in sources if source in config]
+        if key not in config:
+            config[key] = bool(configured)
+        elif not config[key] and configured:
+            raise cv.Invalid(
+                f"'{key}' cannot be false while '{configured[0]}' is configured",
+                path=[key],
+            )
+    return config
+
+
+def _validate_initial_state(config: ConfigType) -> ConfigType:
+    # Climate keeps target_temperature and target_temperature_low in a union, so writing the wrong
+    # one of the pair corrupts the setpoint with no runtime complaint.
+    if (initial_state := config.get(CONF_INITIAL_STATE)) is None:
+        return config
+
+    two_point = config[CONF_SUPPORTS_TWO_POINT_TARGET_TEMPERATURE]
+    if two_point and CONF_TARGET_TEMPERATURE in initial_state:
+        raise cv.Invalid(
+            f"'{CONF_TARGET_TEMPERATURE}' is not available while "
+            f"'{CONF_SUPPORTS_TWO_POINT_TARGET_TEMPERATURE}' is enabled; use "
+            f"'{CONF_TARGET_TEMPERATURE_LOW}'/'{CONF_TARGET_TEMPERATURE_HIGH}' instead",
+            path=[CONF_INITIAL_STATE, CONF_TARGET_TEMPERATURE],
+        )
+    if not two_point:
+        for key in (CONF_TARGET_TEMPERATURE_LOW, CONF_TARGET_TEMPERATURE_HIGH):
+            if key in initial_state:
+                raise cv.Invalid(
+                    f"'{key}' requires '{CONF_SUPPORTS_TWO_POINT_TARGET_TEMPERATURE}' to be enabled",
+                    path=[CONF_INITIAL_STATE, key],
+                )
+    if (
+        CONF_TARGET_HUMIDITY in initial_state
+        and not config[CONF_SUPPORTS_TARGET_HUMIDITY]
+    ):
+        raise cv.Invalid(
+            f"'{CONF_TARGET_HUMIDITY}' requires '{CONF_SUPPORTS_TARGET_HUMIDITY}' to be enabled",
+            path=[CONF_INITIAL_STATE, CONF_TARGET_HUMIDITY],
+        )
+    return config
+
+
+# Same settable fields as climate.template.publish, minus current_temperature/current_humidity/
+# action: those are reported values (from a sensor or the device), not meaningful static defaults.
+INITIAL_STATE_SCHEMA = cv.All(
+    cv.Schema(
+        {
+            cv.Optional(CONF_MODE): climate.validate_climate_mode,
+            cv.Optional(CONF_TARGET_TEMPERATURE): cv.temperature,
+            cv.Optional(CONF_TARGET_TEMPERATURE_LOW): cv.temperature,
+            cv.Optional(CONF_TARGET_TEMPERATURE_HIGH): cv.temperature,
+            cv.Optional(CONF_TARGET_HUMIDITY): cv.percentage_int,
+            cv.Exclusive(CONF_FAN_MODE, "fan_mode"): climate.validate_climate_fan_mode,
+            cv.Exclusive(
+                CONF_CUSTOM_FAN_MODE, "fan_mode"
+            ): validate_custom_climate_string,
+            cv.Optional(CONF_SWING_MODE): climate.validate_climate_swing_mode,
+            cv.Exclusive(CONF_PRESET, "preset"): climate.validate_climate_preset,
+            cv.Exclusive(CONF_CUSTOM_PRESET, "preset"): validate_custom_climate_string,
+        }
+    ),
+    _validate_two_point,
+)
+
+CONFIG_SCHEMA = cv.All(
+    climate.climate_schema(TemplateClimate)
+    .extend(
+        {
+            cv.Optional(CONF_SENSOR): cv.use_id(sensor.Sensor),
+            cv.Optional(CONF_HUMIDITY_SENSOR): cv.use_id(sensor.Sensor),
+            # action only ever arrives through climate.template.publish, so unlike the other
+            # supports_* keys there is no set action to derive it from.
+            cv.Optional(CONF_SUPPORTS_ACTION, default=False): cv.boolean,
+            cv.Optional(CONF_SUPPORTS_CURRENT_TEMPERATURE): cv.boolean,
+            cv.Optional(CONF_SUPPORTS_CURRENT_HUMIDITY): cv.boolean,
+            cv.Optional(CONF_SUPPORTS_TWO_POINT_TARGET_TEMPERATURE): cv.boolean,
+            cv.Optional(CONF_SUPPORTS_TARGET_HUMIDITY): cv.boolean,
+            cv.Required(CONF_SUPPORTED_MODES): cv.All(
+                cv.ensure_list(climate.validate_climate_mode), cv.Unique()
+            ),
+            cv.Optional(CONF_SUPPORTED_FAN_MODES): cv.All(
+                cv.ensure_list(climate.validate_climate_fan_mode), cv.Unique()
+            ),
+            cv.Optional(CONF_CUSTOM_FAN_MODES): cv.All(
+                cv.ensure_list(validate_custom_climate_string), cv.Unique()
+            ),
+            cv.Optional(CONF_SUPPORTED_SWING_MODES): cv.All(
+                cv.ensure_list(climate.validate_climate_swing_mode), cv.Unique()
+            ),
+            cv.Optional(CONF_SUPPORTED_PRESETS): cv.All(
+                cv.ensure_list(climate.validate_climate_preset), cv.Unique()
+            ),
+            cv.Optional(CONF_CUSTOM_PRESETS): cv.All(
+                cv.ensure_list(validate_custom_climate_string), cv.Unique()
+            ),
+            cv.Optional(CONF_OPTIMISTIC, default=True): cv.boolean,
+            cv.Optional(CONF_RESTORE_MODE, default="RESTORE"): cv.enum(
+                CLIMATE_RESTORE_MODES, upper=True
+            ),
+            cv.Optional(CONF_INITIAL_STATE): INITIAL_STATE_SCHEMA,
+            cv.Optional(CONF_SET_MODE_ACTION): automation.validate_automation(
+                single=True
+            ),
+            cv.Optional(
+                CONF_SET_TARGET_TEMPERATURE_ACTION
+            ): automation.validate_automation(single=True),
+            cv.Optional(
+                CONF_SET_TARGET_TEMPERATURE_LOW_ACTION
+            ): automation.validate_automation(single=True),
+            cv.Optional(
+                CONF_SET_TARGET_TEMPERATURE_HIGH_ACTION
+            ): automation.validate_automation(single=True),
+            cv.Optional(
+                CONF_SET_TARGET_HUMIDITY_ACTION
+            ): automation.validate_automation(single=True),
+            cv.Optional(CONF_SET_FAN_MODE_ACTION): automation.validate_automation(
+                single=True
+            ),
+            cv.Optional(
+                CONF_SET_CUSTOM_FAN_MODE_ACTION
+            ): automation.validate_automation(single=True),
+            cv.Optional(CONF_SET_SWING_MODE_ACTION): automation.validate_automation(
+                single=True
+            ),
+            cv.Optional(CONF_SET_PRESET_ACTION): automation.validate_automation(
+                single=True
+            ),
+            cv.Optional(CONF_SET_CUSTOM_PRESET_ACTION): automation.validate_automation(
+                single=True
+            ),
+        }
+    )
+    .extend(cv.COMPONENT_SCHEMA),
+    _validate_set_actions,
+    _resolve_supports,
+    _validate_initial_state,
+)
+
+
+async def to_code(config: ConfigType) -> None:
+    var = cg.new_Pvariable(config[CONF_ID])
+    await cg.register_component(var, config)
+    await climate.register_climate(var, config)
+
+    if (sens := config.get(CONF_SENSOR)) is not None:
+        cg.add(var.set_sensor(await cg.get_variable(sens)))
+
+    if (sens := config.get(CONF_HUMIDITY_SENSOR)) is not None:
+        cg.add(var.set_humidity_sensor(await cg.get_variable(sens)))
+
+    for key, flag in (
+        (CONF_SUPPORTS_ACTION, climate_ns.CLIMATE_SUPPORTS_ACTION),
+        (
+            CONF_SUPPORTS_CURRENT_TEMPERATURE,
+            climate_ns.CLIMATE_SUPPORTS_CURRENT_TEMPERATURE,
+        ),
+        (CONF_SUPPORTS_CURRENT_HUMIDITY, climate_ns.CLIMATE_SUPPORTS_CURRENT_HUMIDITY),
+        (
+            CONF_SUPPORTS_TWO_POINT_TARGET_TEMPERATURE,
+            climate_ns.CLIMATE_SUPPORTS_TWO_POINT_TARGET_TEMPERATURE,
+        ),
+        (CONF_SUPPORTS_TARGET_HUMIDITY, climate_ns.CLIMATE_SUPPORTS_TARGET_HUMIDITY),
+    ):
+        if config[key]:
+            cg.add(var.add_feature_flags(flag))
+
+    for mode in config[CONF_SUPPORTED_MODES]:
+        cg.add(var.add_supported_mode(mode))
+
+    for mode in config.get(CONF_SUPPORTED_FAN_MODES, []):
+        cg.add(var.add_supported_fan_mode(mode))
+
+    if CONF_CUSTOM_FAN_MODES in config:
+        cg.add(
+            var.set_supported_custom_fan_modes(
+                cg.ArrayInitializer(*config[CONF_CUSTOM_FAN_MODES])
+            )
+        )
+
+    for mode in config.get(CONF_SUPPORTED_SWING_MODES, []):
+        cg.add(var.add_supported_swing_mode(mode))
+
+    for preset in config.get(CONF_SUPPORTED_PRESETS, []):
+        cg.add(var.add_supported_preset(preset))
+
+    if CONF_CUSTOM_PRESETS in config:
+        cg.add(
+            var.set_supported_custom_presets(
+                cg.ArrayInitializer(*config[CONF_CUSTOM_PRESETS])
+            )
+        )
+
+    for key, trigger_getter, arg_type in SET_ACTIONS:
+        if (conf := config.get(key)) is not None:
+            await automation.build_automation(
+                getattr(var, trigger_getter)(), [(arg_type, "x")], conf
+            )
+
+    cg.add(var.set_optimistic(config[CONF_OPTIMISTIC]))
+    cg.add(var.set_restore_mode(config[CONF_RESTORE_MODE]))
+
+    if (initial_state := config.get(CONF_INITIAL_STATE)) is not None:
+        if (v := initial_state.get(CONF_MODE)) is not None:
+            cg.add(var.set_mode(v))
+        if (v := initial_state.get(CONF_TARGET_TEMPERATURE)) is not None:
+            cg.add(var.set_target_temperature(v))
+        if (v := initial_state.get(CONF_TARGET_TEMPERATURE_LOW)) is not None:
+            cg.add(var.set_target_temperature_low(v))
+        if (v := initial_state.get(CONF_TARGET_TEMPERATURE_HIGH)) is not None:
+            cg.add(var.set_target_temperature_high(v))
+        if (v := initial_state.get(CONF_TARGET_HUMIDITY)) is not None:
+            cg.add(var.set_target_humidity(v))
+        if (v := initial_state.get(CONF_FAN_MODE)) is not None:
+            cg.add(var.set_fan_mode(v))
+        if (v := initial_state.get(CONF_CUSTOM_FAN_MODE)) is not None:
+            cg.add(var.set_custom_fan_mode(v))
+        if (v := initial_state.get(CONF_SWING_MODE)) is not None:
+            cg.add(var.set_swing_mode(v))
+        if (v := initial_state.get(CONF_PRESET)) is not None:
+            cg.add(var.set_preset(v))
+        if (v := initial_state.get(CONF_CUSTOM_PRESET)) is not None:
+            cg.add(var.set_custom_preset(v))
+
+
+CLIMATE_TEMPLATE_PUBLISH_ACTION_SCHEMA = cv.All(
+    cv.Schema(
+        {
+            cv.GenerateID(): cv.use_id(TemplateClimate),
+            cv.Optional(CONF_CURRENT_TEMPERATURE): cv.templatable(cv.temperature),
+            cv.Optional(CONF_CURRENT_HUMIDITY): cv.templatable(cv.percentage_int),
+            cv.Optional(CONF_TARGET_TEMPERATURE): cv.templatable(cv.temperature),
+            cv.Optional(CONF_TARGET_TEMPERATURE_LOW): cv.templatable(cv.temperature),
+            cv.Optional(CONF_TARGET_TEMPERATURE_HIGH): cv.templatable(cv.temperature),
+            cv.Optional(CONF_TARGET_HUMIDITY): cv.templatable(cv.percentage_int),
+            cv.Optional(CONF_MODE): cv.templatable(climate.validate_climate_mode),
+            cv.Optional(CONF_ACTION): cv.templatable(climate.validate_climate_action),
+            cv.Exclusive(CONF_FAN_MODE, "fan_mode"): cv.templatable(
+                climate.validate_climate_fan_mode
+            ),
+            cv.Exclusive(CONF_CUSTOM_FAN_MODE, "fan_mode"): cv.templatable(
+                validate_custom_climate_string
+            ),
+            cv.Optional(CONF_SWING_MODE): cv.templatable(
+                climate.validate_climate_swing_mode
+            ),
+            cv.Exclusive(CONF_PRESET, "preset"): cv.templatable(
+                climate.validate_climate_preset
+            ),
+            cv.Exclusive(CONF_CUSTOM_PRESET, "preset"): cv.templatable(
+                validate_custom_climate_string
+            ),
+        }
+    ),
+    _validate_two_point,
+)
+
+
+@automation.register_action(
+    "climate.template.publish",
+    TemplateClimatePublishAction,
+    CLIMATE_TEMPLATE_PUBLISH_ACTION_SCHEMA,
+    synchronous=True,
+)
+async def climate_template_publish_to_code(
+    config: ConfigType,
+    action_id: ID,
+    template_arg: cg.TemplateArguments,
+    args: TemplateArgsType,
+) -> MockObj:
+    var = cg.new_Pvariable(action_id, template_arg)
+    await cg.register_parented(var, config[CONF_ID])
+
+    if (v := config.get(CONF_CURRENT_TEMPERATURE)) is not None:
+        cg.add(var.set_current_temperature(await cg.templatable(v, args, cg.float_)))
+    if (v := config.get(CONF_CURRENT_HUMIDITY)) is not None:
+        cg.add(var.set_current_humidity(await cg.templatable(v, args, cg.float_)))
+    if (v := config.get(CONF_TARGET_TEMPERATURE)) is not None:
+        cg.add(var.set_target_temperature(await cg.templatable(v, args, cg.float_)))
+    if (v := config.get(CONF_TARGET_TEMPERATURE_LOW)) is not None:
+        cg.add(var.set_target_temperature_low(await cg.templatable(v, args, cg.float_)))
+    if (v := config.get(CONF_TARGET_TEMPERATURE_HIGH)) is not None:
+        cg.add(
+            var.set_target_temperature_high(await cg.templatable(v, args, cg.float_))
+        )
+    if (v := config.get(CONF_TARGET_HUMIDITY)) is not None:
+        cg.add(var.set_target_humidity(await cg.templatable(v, args, cg.float_)))
+    if (v := config.get(CONF_MODE)) is not None:
+        cg.add(var.set_mode(await cg.templatable(v, args, climate.ClimateMode)))
+    if (v := config.get(CONF_ACTION)) is not None:
+        cg.add(var.set_action(await cg.templatable(v, args, climate.ClimateAction)))
+    if (v := config.get(CONF_FAN_MODE)) is not None:
+        cg.add(var.set_fan_mode(await cg.templatable(v, args, climate.ClimateFanMode)))
+    if (v := config.get(CONF_CUSTOM_FAN_MODE)) is not None:
+        cg.add(var.set_custom_fan_mode(await cg.templatable(v, args, cg.std_string)))
+    if (v := config.get(CONF_SWING_MODE)) is not None:
+        cg.add(
+            var.set_swing_mode(await cg.templatable(v, args, climate.ClimateSwingMode))
+        )
+    if (v := config.get(CONF_PRESET)) is not None:
+        cg.add(var.set_preset(await cg.templatable(v, args, climate.ClimatePreset)))
+    if (v := config.get(CONF_CUSTOM_PRESET)) is not None:
+        cg.add(var.set_custom_preset(await cg.templatable(v, args, cg.std_string)))
+
+    return var

esphome/components/template/climate/automation.h

@@ -0,0 +1,57 @@
+#pragma once
+
+#include "template_climate.h"
+#include "esphome/core/automation.h"
+
+namespace esphome::template_ {
+
+template<typename... Ts>
+class TemplateClimatePublishAction final : public Action<Ts...>, public Parented<TemplateClimate> {
+ public:
+  TEMPLATABLE_VALUE(float, current_temperature)
+  TEMPLATABLE_VALUE(float, current_humidity)
+  TEMPLATABLE_VALUE(float, target_temperature)
+  TEMPLATABLE_VALUE(float, target_temperature_low)
+  TEMPLATABLE_VALUE(float, target_temperature_high)
+  TEMPLATABLE_VALUE(float, target_humidity)
+  TEMPLATABLE_VALUE(climate::ClimateMode, mode)
+  TEMPLATABLE_VALUE(climate::ClimateAction, action)
+  TEMPLATABLE_VALUE(climate::ClimateFanMode, fan_mode)
+  TEMPLATABLE_VALUE(std::string, custom_fan_mode)
+  TEMPLATABLE_VALUE(climate::ClimateSwingMode, swing_mode)
+  TEMPLATABLE_VALUE(climate::ClimatePreset, preset)
+  TEMPLATABLE_VALUE(std::string, custom_preset)
+
+  void play(const Ts &...x) override {
+    if (this->current_temperature_.has_value())
+      this->parent_->current_temperature = this->current_temperature_.value(x...);
+    if (this->current_humidity_.has_value())
+      this->parent_->current_humidity = this->current_humidity_.value(x...);
+    if (this->target_temperature_.has_value())
+      this->parent_->set_target_temperature(this->target_temperature_.value(x...));
+    if (this->target_temperature_low_.has_value())
+      this->parent_->set_target_temperature_low(this->target_temperature_low_.value(x...));
+    if (this->target_temperature_high_.has_value())
+      this->parent_->set_target_temperature_high(this->target_temperature_high_.value(x...));
+    if (this->target_humidity_.has_value())
+      this->parent_->set_target_humidity(this->target_humidity_.value(x...));
+    if (this->mode_.has_value())
+      this->parent_->set_mode(this->mode_.value(x...));
+    if (this->action_.has_value())
+      this->parent_->action = this->action_.value(x...);
+    if (this->fan_mode_.has_value())
+      this->parent_->set_fan_mode(this->fan_mode_.value(x...));
+    if (this->custom_fan_mode_.has_value())
+      this->parent_->set_custom_fan_mode(StringRef(this->custom_fan_mode_.value(x...)));
+    if (this->swing_mode_.has_value())
+      this->parent_->set_swing_mode(this->swing_mode_.value(x...));
+    if (this->preset_.has_value())
+      this->parent_->set_preset(this->preset_.value(x...));
+    if (this->custom_preset_.has_value())
+      this->parent_->set_custom_preset(StringRef(this->custom_preset_.value(x...)));
+
+    this->parent_->publish_state();
+  }
+};
+
+}  // namespace esphome::template_

esphome/components/template/climate/template_climate.cpp

@@ -0,0 +1,164 @@
+#include "template_climate.h"
+#include "esphome/core/log.h"
+
+namespace esphome::template_ {
+
+static const char *const TAG = "template.climate";
+
+void TemplateClimate::setup() {
+  if (this->restore_mode_ == TemplateClimateRestoreMode::TEMPLATE_CLIMATE_RESTORE_MODE_RESTORE) {
+    auto restore = this->restore_state_();
+    if (restore.has_value()) {
+      restore->apply(this);
+    }
+  }
+
+  // Sensors publish every reading, not just changes, so only re-publish when the value moved.
+  // NAN means the sensor went unavailable and is passed through rather than dropped; the second
+  // check stops an unavailable sensor re-publishing forever, since NAN never equals NAN.
+#ifdef USE_SENSOR
+  if (this->sensor_ != nullptr) {
+    this->current_temperature = this->sensor_->state;
+    this->sensor_->add_on_state_callback([this](float state) {
+      if (state != this->current_temperature && !(std::isnan(state) && std::isnan(this->current_temperature))) {
+        this->current_temperature = state;
+        this->publish_state();
+      }
+    });
+  }
+
+  if (this->humidity_sensor_ != nullptr) {
+    this->current_humidity = this->humidity_sensor_->state;
+    this->humidity_sensor_->add_on_state_callback([this](float state) {
+      if (state != this->current_humidity && !(std::isnan(state) && std::isnan(this->current_humidity))) {
+        this->current_humidity = state;
+        this->publish_state();
+      }
+    });
+  }
+#endif
+}
+
+void TemplateClimate::dump_config() {
+  LOG_CLIMATE("", "Template Climate", this);
+  ESP_LOGCONFIG(TAG, "  Optimistic: %s", YESNO(this->optimistic_));
+}
+
+void TemplateClimate::control(const climate::ClimateCall &call) {
+  // Each field present fires its set_*_action; on_control sees the whole call. optimistic: true
+  // also applies the values right away, false waits for a climate.template.publish report.
+  if (auto mode = call.get_mode()) {
+    if (this->optimistic_)
+      this->mode = *mode;
+    this->set_mode_trigger_.trigger(*mode);
+  }
+
+  if (auto target_temp = call.get_target_temperature()) {
+    if (this->optimistic_)
+      this->target_temperature = *target_temp;
+    this->set_target_temperature_trigger_.trigger(*target_temp);
+  }
+
+  if (auto target_temp_low = call.get_target_temperature_low()) {
+    if (this->optimistic_)
+      this->target_temperature_low = *target_temp_low;
+    this->set_target_temperature_low_trigger_.trigger(*target_temp_low);
+  }
+
+  if (auto target_temp_high = call.get_target_temperature_high()) {
+    if (this->optimistic_)
+      this->target_temperature_high = *target_temp_high;
+    this->set_target_temperature_high_trigger_.trigger(*target_temp_high);
+  }
+
+  if (auto target_humidity = call.get_target_humidity()) {
+    if (this->optimistic_)
+      this->target_humidity = *target_humidity;
+    this->set_target_humidity_trigger_.trigger(*target_humidity);
+  }
+
+  if (auto fan_mode = call.get_fan_mode()) {
+    if (this->optimistic_)
+      this->set_fan_mode_(*fan_mode);
+    this->set_fan_mode_trigger_.trigger(*fan_mode);
+  }
+
+  if (call.has_custom_fan_mode()) {
+    if (this->optimistic_)
+      this->set_custom_fan_mode_(call.get_custom_fan_mode());
+    this->set_custom_fan_mode_trigger_.trigger(call.get_custom_fan_mode());
+  }
+
+  if (auto swing_mode = call.get_swing_mode()) {
+    if (this->optimistic_)
+      this->swing_mode = *swing_mode;
+    this->set_swing_mode_trigger_.trigger(*swing_mode);
+  }
+
+  if (auto preset = call.get_preset()) {
+    if (this->optimistic_)
+      this->set_preset_(*preset);
+    this->set_preset_trigger_.trigger(*preset);
+  }
+
+  if (call.has_custom_preset()) {
+    if (this->optimistic_)
+      this->set_custom_preset_(call.get_custom_preset());
+    this->set_custom_preset_trigger_.trigger(call.get_custom_preset());
+  }
+
+  if (this->optimistic_)
+    this->publish_state();
+}
+
+// A climate.template.publish report (and initial_state:) never goes through ClimateCall::validate_(),
+// so check here instead -- otherwise a typo is published as state the receiving end will reject.
+void TemplateClimate::set_mode(climate::ClimateMode mode) {
+  if (!this->traits_.supports_mode(mode)) {
+    ESP_LOGW(TAG, "'%s' - Unsupported mode %u", this->get_name().c_str(), static_cast<unsigned>(mode));
+    return;
+  }
+  this->mode = mode;
+}
+
+void TemplateClimate::set_swing_mode(climate::ClimateSwingMode swing_mode) {
+  if (!this->traits_.supports_swing_mode(swing_mode)) {
+    ESP_LOGW(TAG, "'%s' - Unsupported swing mode %u", this->get_name().c_str(), static_cast<unsigned>(swing_mode));
+    return;
+  }
+  this->swing_mode = swing_mode;
+}
+
+void TemplateClimate::set_fan_mode(climate::ClimateFanMode fan_mode) {
+  if (!this->traits_.supports_fan_mode(fan_mode)) {
+    ESP_LOGW(TAG, "'%s' - Unsupported fan mode %u", this->get_name().c_str(), static_cast<unsigned>(fan_mode));
+    return;
+  }
+  this->set_fan_mode_(fan_mode);
+}
+
+void TemplateClimate::set_preset(climate::ClimatePreset preset) {
+  if (!this->traits_.supports_preset(preset)) {
+    ESP_LOGW(TAG, "'%s' - Unsupported preset %u", this->get_name().c_str(), static_cast<unsigned>(preset));
+    return;
+  }
+  this->set_preset_(preset);
+}
+
+void TemplateClimate::set_custom_fan_mode(StringRef mode) {
+  if (this->find_custom_fan_mode_(mode.c_str(), mode.size()) == nullptr) {
+    ESP_LOGW(TAG, "'%s' - Unsupported custom fan mode '%s'", this->get_name().c_str(), mode.c_str());
+    return;
+  }
+  this->set_custom_fan_mode_(mode);
+}
+
+void TemplateClimate::set_custom_preset(StringRef preset) {
+  if (this->find_custom_preset_(preset.c_str(), preset.size()) == nullptr) {
+    ESP_LOGW(TAG, "'%s' - Unsupported custom preset '%s'", this->get_name().c_str(), preset.c_str());
+    return;
+  }
+  this->set_custom_preset_(preset);
+}
+
+}  // namespace esphome::template_

esphome/components/template/climate/template_climate.h

@@ -0,0 +1,92 @@
+#pragma once
+
+#include "esphome/core/automation.h"
+#include "esphome/core/component.h"
+#include "esphome/components/climate/climate.h"
+#ifdef USE_SENSOR
+#include "esphome/components/sensor/sensor.h"
+#endif
+
+namespace esphome::template_ {
+
+enum class TemplateClimateRestoreMode {
+  TEMPLATE_CLIMATE_RESTORE_MODE_NO_RESTORE,
+  TEMPLATE_CLIMATE_RESTORE_MODE_RESTORE,
+};
+
+class TemplateClimate final : public climate::Climate, public Component {
+ public:
+  void setup() override;
+  void dump_config() override;
+
+  climate::ClimateTraits traits() override { return this->traits_; }
+
+  void add_feature_flags(uint32_t flags) { this->traits_.add_feature_flags(flags); }
+
+#ifdef USE_SENSOR
+  // The matching feature flag is added from codegen, so the configuration alone decides it.
+  void set_sensor(sensor::Sensor *sensor) { this->sensor_ = sensor; }
+  void set_humidity_sensor(sensor::Sensor *sensor) { this->humidity_sensor_ = sensor; }
+#endif
+
+  void add_supported_mode(climate::ClimateMode mode) { this->traits_.add_supported_mode(mode); }
+  void add_supported_fan_mode(climate::ClimateFanMode mode) { this->traits_.add_supported_fan_mode(mode); }
+  void add_supported_swing_mode(climate::ClimateSwingMode mode) { this->traits_.add_supported_swing_mode(mode); }
+  void add_supported_preset(climate::ClimatePreset preset) { this->traits_.add_supported_preset(preset); }
+
+  void set_optimistic(bool optimistic) { this->optimistic_ = optimistic; }
+  void set_restore_mode(TemplateClimateRestoreMode restore_mode) { this->restore_mode_ = restore_mode; }
+
+  // Fired from control() for each field the call carries, so a device-backed config can forward
+  // it on. Which of these are configured also decides the two-point/target-humidity traits.
+  Trigger<climate::ClimateMode> *get_set_mode_trigger() { return &this->set_mode_trigger_; }
+  Trigger<float> *get_set_target_temperature_trigger() { return &this->set_target_temperature_trigger_; }
+  Trigger<float> *get_set_target_temperature_low_trigger() { return &this->set_target_temperature_low_trigger_; }
+  Trigger<float> *get_set_target_temperature_high_trigger() { return &this->set_target_temperature_high_trigger_; }
+  Trigger<float> *get_set_target_humidity_trigger() { return &this->set_target_humidity_trigger_; }
+  Trigger<climate::ClimateFanMode> *get_set_fan_mode_trigger() { return &this->set_fan_mode_trigger_; }
+  Trigger<StringRef> *get_set_custom_fan_mode_trigger() { return &this->set_custom_fan_mode_trigger_; }
+  Trigger<climate::ClimateSwingMode> *get_set_swing_mode_trigger() { return &this->set_swing_mode_trigger_; }
+  Trigger<climate::ClimatePreset> *get_set_preset_trigger() { return &this->set_preset_trigger_; }
+  Trigger<StringRef> *get_set_custom_preset_trigger() { return &this->set_custom_preset_trigger_; }
+
+  // Used by TemplateClimatePublishAction, which is not a Climate subclass and so cannot reach the
+  // protected setters, and by codegen to apply `initial_state:` before setup() runs.
+  void set_target_temperature(float value) { this->target_temperature = value; }
+  void set_target_temperature_low(float value) { this->target_temperature_low = value; }
+  void set_target_temperature_high(float value) { this->target_temperature_high = value; }
+  void set_target_humidity(float value) { this->target_humidity = value; }
+  void set_mode(climate::ClimateMode mode);
+  void set_swing_mode(climate::ClimateSwingMode mode);
+  void set_fan_mode(climate::ClimateFanMode mode);
+  void set_custom_fan_mode(const char *mode) { this->set_custom_fan_mode(StringRef(mode)); }
+  void set_custom_fan_mode(StringRef mode);
+  void set_preset(climate::ClimatePreset preset);
+  void set_custom_preset(const char *preset) { this->set_custom_preset(StringRef(preset)); }
+  void set_custom_preset(StringRef preset);
+
+ protected:
+  void control(const climate::ClimateCall &call) override;
+
+  climate::ClimateTraits traits_;
+  bool optimistic_{false};
+  TemplateClimateRestoreMode restore_mode_{TemplateClimateRestoreMode::TEMPLATE_CLIMATE_RESTORE_MODE_NO_RESTORE};
+
+#ifdef USE_SENSOR
+  sensor::Sensor *sensor_{nullptr};
+  sensor::Sensor *humidity_sensor_{nullptr};
+#endif
+
+  Trigger<climate::ClimateMode> set_mode_trigger_;
+  Trigger<float> set_target_temperature_trigger_;
+  Trigger<float> set_target_temperature_low_trigger_;
+  Trigger<float> set_target_temperature_high_trigger_;
+  Trigger<float> set_target_humidity_trigger_;
+  Trigger<climate::ClimateFanMode> set_fan_mode_trigger_;
+  Trigger<StringRef> set_custom_fan_mode_trigger_;
+  Trigger<climate::ClimateSwingMode> set_swing_mode_trigger_;
+  Trigger<climate::ClimatePreset> set_preset_trigger_;
+  Trigger<StringRef> set_custom_preset_trigger_;
+};
+
+}  // namespace esphome::template_

esphome/components/usb_uart/usb_uart.cpp

@@ -434,11 +434,12 @@ void USBUartTypeCdcAcm::on_connected() {
       auto err_comm = usb_host_interface_claim(this->handle_, this->device_handle_,
                                                channel->cdc_dev_.interrupt_interface_number, 0);
       if (err_comm != ESP_OK) {
+        // Continue anyway: the interface number stays valid for CDC request addressing
         ESP_LOGW(TAG, "Could not claim comm interface %d: %s", channel->cdc_dev_.interrupt_interface_number,
                  esp_err_to_name(err_comm));
-        channel->cdc_dev_.interrupt_interface_number = 0xFF;  // Mark as unavailable, but continue anyway
       } else {
         ESP_LOGD(TAG, "Claimed comm interface %d", channel->cdc_dev_.interrupt_interface_number);
+        channel->cdc_dev_.interrupt_interface_claimed = true;
       }
     }
     auto err =
@@ -465,14 +466,15 @@ void USBUartTypeCdcAcm::on_disconnected() {
       usb_host_endpoint_halt(this->device_handle_, channel->cdc_dev_.out_ep->bEndpointAddress);
       usb_host_endpoint_flush(this->device_handle_, channel->cdc_dev_.out_ep->bEndpointAddress);
     }
-    if (channel->cdc_dev_.notify_ep != nullptr) {
+    // Only tear down the notify pipe when we claimed its interface ourselves;
+    // no transfer is ever submitted on it, so there is nothing else to cancel.
+    if (channel->cdc_dev_.notify_ep != nullptr && channel->cdc_dev_.interrupt_interface_claimed) {
       usb_host_endpoint_halt(this->device_handle_, channel->cdc_dev_.notify_ep->bEndpointAddress);
       usb_host_endpoint_flush(this->device_handle_, channel->cdc_dev_.notify_ep->bEndpointAddress);
     }
-    if (channel->cdc_dev_.interrupt_interface_number != 0xFF &&
-        channel->cdc_dev_.interrupt_interface_number != channel->cdc_dev_.bulk_interface_number) {
+    if (channel->cdc_dev_.interrupt_interface_claimed) {
       usb_host_interface_release(this->handle_, this->device_handle_, channel->cdc_dev_.interrupt_interface_number);
-      channel->cdc_dev_.interrupt_interface_number = 0xFF;
+      channel->cdc_dev_.interrupt_interface_claimed = false;
     }
     usb_host_interface_release(this->handle_, this->device_handle_, channel->cdc_dev_.bulk_interface_number);
     // Reset the input and output started flags to their initial state to avoid the possibility of spurious restarts

esphome/components/usb_uart/usb_uart.h

@@ -34,7 +34,10 @@ struct CdcEps {
   const usb_ep_desc_t *in_ep;
   const usb_ep_desc_t *out_ep;
   uint8_t bulk_interface_number;
+  // Also the wIndex target for CDC class requests (SET_LINE_CODING etc.), so it
+  // must remain valid even when the interface itself is not claimed.
   uint8_t interrupt_interface_number;
+  bool interrupt_interface_claimed{false};
 };
 
 enum CH34xChipType : uint8_t {

esphome/components/wifi/__init__.py

@@ -66,13 +66,14 @@ from esphome.const import (
 )
 from esphome.core import (
     CORE,
+    ID,
     CoroPriority,
     EsphomeError,
     HexInt,
     coroutine_with_priority,
 )
 import esphome.final_validate as fv
-from esphome.types import ConfigType
+from esphome.types import ConfigType, TemplateArgsType
 
 from . import wpa2_eap
 
@@ -208,6 +209,7 @@ WiFiEnabledCondition = wifi_ns.class_("WiFiEnabledCondition", Condition)
 WiFiAPActiveCondition = wifi_ns.class_("WiFiAPActiveCondition", Condition)
 WiFiEnableAction = wifi_ns.class_("WiFiEnableAction", automation.Action)
 WiFiDisableAction = wifi_ns.class_("WiFiDisableAction", automation.Action)
+WiFiRoamAction = wifi_ns.class_("WiFiRoamAction", automation.Action)
 WiFiConfigureAction = wifi_ns.class_(
     "WiFiConfigureAction", automation.Action, cg.Component
 )
@@ -820,6 +822,18 @@ async def wifi_disable_to_code(config, action_id, template_arg, args):
     return cg.new_Pvariable(action_id, template_arg)
 
 
+@automation.register_action(
+    "wifi.roam", WiFiRoamAction, cv.Schema({}), synchronous=True
+)
+async def wifi_roam_to_code(
+    config: ConfigType,
+    action_id: ID,
+    template_arg: cg.TemplateArguments,
+    args: TemplateArgsType,
+) -> cg.MockObj:
+    return cg.new_Pvariable(action_id, template_arg)
+
+
 KEEP_SCAN_RESULTS_KEY = "wifi_keep_scan_results"
 RUNTIME_POWER_SAVE_KEY = "wifi_runtime_power_save"
 RUNTIME_ROAMING_SUPPRESSION_KEY = "wifi_runtime_roaming_suppression"

esphome/components/wifi/automation.h

@@ -31,6 +31,11 @@ template<typename... Ts> class WiFiDisableAction final : public Action<Ts...> {
   void play(const Ts &...x) override { global_wifi_component->disable(); }
 };
 
+template<typename... Ts> class WiFiRoamAction final : public Action<Ts...> {
+ public:
+  void play(const Ts &...x) override { global_wifi_component->force_roam_check(); }
+};
+
 template<typename... Ts> class WiFiConfigureAction final : public Action<Ts...>, public Component {
  public:
   TEMPLATABLE_VALUE(std::string, ssid)

esphome/components/wifi/wifi_component.cpp

@@ -846,17 +846,18 @@ void WiFiComponent::loop() {
           this->notify_connect_state_listeners_();
 #endif
 
-          // Post-connect roaming: check for better AP
-          if (this->post_connect_roaming_) {
-            if (this->is_roaming_scan_active()) {
-              if (this->scan_done_) {
-                this->process_roaming_scan_();
-              }
-              // else: scan in progress, wait
-            } else if (this->roaming_state_ == RoamingState::IDLE && this->roaming_attempts_ < ROAMING_MAX_ATTEMPTS &&
-                       now - this->roaming_last_check_ >= ROAMING_CHECK_INTERVAL && !this->roaming_suppressed_()) {
-              this->check_roaming_(now);
+          // Post-connect roaming: check for better AP. A scan may have been started by an
+          // explicit force_roam_check() even when post_connect_roaming_ is disabled, so the
+          // scan must always be consumed here to avoid leaving roaming_state_ stuck.
+          if (this->is_roaming_scan_active()) {
+            if (this->scan_done_) {
+              this->process_roaming_scan_();
             }
+            // else: scan in progress, wait
+          } else if (this->post_connect_roaming_ && this->roaming_state_ == RoamingState::IDLE &&
+                     this->roaming_attempts_ < ROAMING_MAX_ATTEMPTS &&
+                     now - this->roaming_last_check_ >= ROAMING_CHECK_INTERVAL && !this->roaming_suppressed_()) {
+            this->check_roaming_(now);
           }
         }
         break;
@@ -2463,6 +2464,17 @@ void WiFiComponent::notify_scan_results_listeners_() {
 }
 #endif  // USE_WIFI_SCAN_RESULTS_LISTENERS
 
+void WiFiComponent::force_roam_check() {
+  if (!this->is_connected() || this->roaming_state_ != RoamingState::IDLE || this->roaming_suppressed_()) {
+    ESP_LOGD(TAG, "Roam check requested, but not able to check now");
+    return;
+  }
+  // Reset the attempt counter so a prior run of failed roams doesn't block this explicit request
+  // Note that this re-arms automatic roaming if enabled.
+  this->roaming_attempts_ = 0;
+  this->check_roaming_(millis());
+}
+
 void WiFiComponent::check_roaming_(uint32_t now) {
   // Guard: not for hidden networks (may not appear in scan)
   const WiFiAP *selected = this->get_selected_sta_();
@@ -2484,7 +2496,11 @@ void WiFiComponent::check_roaming_(uint32_t now) {
 
   ESP_LOGD(TAG, "Roam scan (%d dBm, attempt %u/%u)", rssi, this->roaming_attempts_, ROAMING_MAX_ATTEMPTS);
   this->roaming_state_ = RoamingState::SCANNING;
-  this->wifi_scan_start_(this->passive_scan_);
+  if (!this->wifi_scan_start_(this->passive_scan_)) {
+    // Scan failed to start (e.g. busy) - don't get stuck in SCANNING forever
+    ESP_LOGD(TAG, "Roam scan failed to start");
+    this->roaming_state_ = RoamingState::IDLE;
+  }
 }
 
 void WiFiComponent::process_roaming_scan_() {

esphome/components/wifi/wifi_component.h

@@ -565,6 +565,12 @@ class WiFiComponent final : public Component {
   void set_keep_scan_results(bool keep_scan_results) { this->keep_scan_results_ = keep_scan_results; }
   void set_post_connect_roaming(bool enabled) { this->post_connect_roaming_ = enabled; }
 
+  /** Force an immediate post-connect roaming check, bypassing the periodic interval and the
+   * per-connection attempt limit. Does nothing (besides a debug log) if not connected, if a
+   * roam scan or connect is already in progress, or if roaming is currently suppressed.
+   */
+  void force_roam_check();
+
 #ifdef USE_WIFI_CONNECT_TRIGGER
   Trigger<> *get_connect_trigger() { return &this->connect_trigger_; }
 #endif

esphome/config_validation.py

@@ -133,6 +133,7 @@ Upper = vol.Upper
 Length = vol.Length
 Exclusive = vol.Exclusive
 Inclusive = vol.Inclusive
+Unique = vol.Unique
 ALLOW_EXTRA = vol.ALLOW_EXTRA
 UNDEFINED = vol.UNDEFINED
 RequiredFieldInvalid = vol.RequiredFieldInvalid

esphome/core/defines.h

@@ -71,6 +71,7 @@
 #define USE_ESP32_HOSTED
 #define USE_ESP32_HOSTED_HTTP_UPDATE
 #define USE_ESP32_IMPROV_STATE_CALLBACK
+#define USE_ESP_NOW_HOSTED
 #define USE_EVENT
 #define USE_FAN
 #define USE_GPIO_BINARY_SENSOR_INTERRUPT
@@ -244,6 +245,9 @@
 #define USE_RUNTIME_STATS
 #define USE_OTA
 #define USE_OTA_ENCRYPTION
+#define USE_OTA_ENCRYPTION_FROM_API
+#define USE_OTA_ENCRYPTION_PROVISIONED
+#define USE_OTA_ENCRYPTION_REQUIRED
 #define USE_OTA_PASSWORD
 #define USE_OTA_VERSION 2
 #define USE_TIME_TIMEZONE

esphome/espota2.py

@@ -202,6 +202,49 @@ class OTANetworkError(OTAError):
     """Network-level OTA failure (timeout, reset, closed connection); retrying may succeed."""
 
 
+# Remove before 2027.3.0
+class OTAEncryptionFallback(OTAError):
+    """The encrypted attempt failed and the caller may retry in plaintext."""
+
+
+# Remove before 2027.3.0
+PLAINTEXT_FALLBACK_NOTICE = (
+    "A device with an api encryption key offers encryption after this "
+    "install; add 'encryption:' under 'ota: platform: esphome' to require it. "
+    "This plaintext fallback is removed in 2027.3.0."
+)
+
+
+# Remove before 2027.3.0
+class _EncryptionAttempt:
+    """The key an upload tries and whether it may fall back to plaintext;
+    a rejected handshake falls back at once, a transport fault only on repeat."""
+
+    def __init__(self, noise_psk: str | None, plaintext_fallback: bool) -> None:
+        self.noise_psk = noise_psk
+        self.plaintext_fallback = plaintext_fallback
+        self.handshake_faults = 0
+
+    def handshake_fault_falls_back(self) -> bool:
+        self.handshake_faults += 1
+        return self.plaintext_fallback and self.handshake_faults >= 2
+
+    def downgrade(self, reason: str) -> None:
+        _LOGGER.warning(
+            "%s. Retrying in plaintext; a device that requires encryption "
+            "refuses it. %s",
+            reason,
+            PLAINTEXT_FALLBACK_NOTICE,
+        )
+        self.noise_psk = None
+        self.plaintext_fallback = False
+
+
+# Remove before 2027.3.0: only the fallback decision needs this distinction
+class OTAHandshakeNetworkError(OTANetworkError):
+    """A transport failure inside the noise handshake; retrying encrypted may succeed."""
+
+
 def _committed_error(err: OTANetworkError) -> OTAError:
     """Wrap a network failure that happened once the device had the full image.
 
@@ -464,6 +507,7 @@ def perform_ota(
     filename: Path,
     ota_type: int = OTA_TYPE_UPDATE_APP,
     noise_psk: str | None = None,
+    plaintext_fallback: bool = False,
 ) -> None:
     # Validate up front; an out-of-range value would only surface as a
     # ValueError deep inside send_check, bypassing OTAError handling
@@ -528,19 +572,28 @@ def perform_ota(
     else:
         features = 0
 
-    if noise_psk:
-        # Fail closed: never fall back to a plaintext upload when an
-        # encryption key is configured, an active attacker could otherwise
-        # strip the feature flag and capture the image (it contains the wifi
-        # credentials and the api encryption key).
-        if not (extended_proto and features & SERVER_FEATURE_SUPPORTS_NOISE):
+    if noise_psk and not (extended_proto and features & SERVER_FEATURE_SUPPORTS_NOISE):
+        if plaintext_fallback:
+            # Remove before 2027.3.0: older firmware that cannot encrypt still
+            # gets its update on this connection
+            _LOGGER.warning(
+                "The device did not offer OTA encryption; continuing in plaintext. %s",
+                PLAINTEXT_FALLBACK_NOTICE,
+            )
+            noise_psk = None
+        else:
+            # Fail closed: an attacker could otherwise strip the offer and
+            # capture the image (wifi credentials, api key)
             raise OTAError(
                 "An OTA encryption key is configured but the device did not "
                 "offer encryption; refusing to send the image in plaintext. "
-                "If the running firmware predates OTA encryption, first update "
-                "it without the 'ota: encryption:' block (over a trusted "
-                "network or via USB), then restore the block and upload again."
+                "The running firmware predates ESPHome 2026.9.0 or has no "
+                "'api: encryption: key'. With an api key, install once "
+                "without the 'ota: encryption:' block (that build offers "
+                "encryption), then restore it; otherwise flash by serial or "
+                "the web_server OTA platform."
             )
+    if noise_psk:
         # The prologue binds every negotiation byte both sides saw, so any
         # tampering with the plaintext preamble breaks the handshake.
         prologue = (
@@ -549,8 +602,18 @@ def perform_ota(
             + bytes([RESPONSE_OK, version, features_to_send])
             + bytes([RESPONSE_FEATURE_FLAGS, features])
         )
+        # Built outside the try: a local failure must never downgrade the upload
         sock = NoiseSocketWrapper(sock, noise_psk, prologue)
-        sock.do_handshake()
+        try:
+            sock.do_handshake()
+        except OTANetworkError as err:
+            # A transport fault: retry encrypted before considering plaintext
+            raise OTAHandshakeNetworkError(str(err)) from err
+        except OTAError as err:
+            # Remove before 2027.3.0
+            if plaintext_fallback:
+                raise OTAEncryptionFallback(str(err)) from err
+            raise
         _LOGGER.info("Encrypted connection established")
 
     if ota_type != OTA_TYPE_UPDATE_APP:
@@ -757,6 +820,7 @@ def run_ota_impl_(
     filename: Path,
     ota_type: int = OTA_TYPE_UPDATE_APP,
     noise_psk: str | None = None,
+    plaintext_fallback: bool = False,
 ) -> tuple[int, str | None]:
     from esphome.core import CORE
 
@@ -795,7 +859,9 @@ def run_ota_impl_(
     total_attempts = len(res) + EXTRA_UPLOAD_ATTEMPTS
     last_error = ""
     reached_device = False
-    for attempt in range(total_attempts):
+    attempt = 0
+    encryption = _EncryptionAttempt(noise_psk, plaintext_fallback)
+    while attempt < total_attempts:
         af, socktype, _, _, sa = res[attempt % len(res)]
         if reached_device or attempt >= len(res):
             _LOGGER.info(
@@ -815,17 +881,40 @@ def run_ota_impl_(
             sock.close()
             _LOGGER.warning("Connecting to %s port %s failed: %s", sa[0], sa[1], err)
             last_error = f"connecting to {sa[0]} failed: {err}"
+            attempt += 1
             continue
 
         _LOGGER.info("Connected to %s", sa[0])
         reached_device = True
         with contextlib.closing(sock), Path(filename).open("rb") as file_handle:
             try:
-                perform_ota(sock, password, file_handle, filename, ota_type, noise_psk)
+                perform_ota(
+                    sock,
+                    password,
+                    file_handle,
+                    filename,
+                    ota_type,
+                    encryption.noise_psk,
+                    encryption.plaintext_fallback,
+                )
+            except OTAEncryptionFallback as err:
+                # Same address and attempt budget: not a network retry
+                last_error = str(err)
+                encryption.downgrade(last_error)
+                continue
+            except OTAHandshakeNetworkError as err:
+                last_error = str(err)
+                if encryption.handshake_fault_falls_back():
+                    encryption.downgrade(last_error)
+                    continue
+                _LOGGER.warning("%s", last_error)
+                attempt += 1
+                continue
             except OTANetworkError as err:
                 # Transient network failure; retry
                 last_error = str(err)
                 _LOGGER.warning("%s", last_error)
+                attempt += 1
                 continue
             except OTAError as err:
                 # Device-reported error (wrong password, wrong flash size, ...);
@@ -847,10 +936,17 @@ def run_ota(
     filename: Path,
     ota_type: int = OTA_TYPE_UPDATE_APP,
     noise_psk: str | None = None,
+    plaintext_fallback: bool = False,
 ) -> tuple[int, str | None]:
     try:
         return run_ota_impl_(
-            remote_host, remote_port, password, filename, ota_type, noise_psk
+            remote_host,
+            remote_port,
+            password,
+            filename,
+            ota_type,
+            noise_psk,
+            plaintext_fallback,
         )
     except OTAError as err:
         _LOGGER.error(err)

esphome/framework_helpers.py

@@ -23,6 +23,7 @@ from esphome.net_retry import (
 )
 
 if TYPE_CHECKING:
+    from filelock import FileLock
     import requests
 
 PathType = str | os.PathLike
@@ -909,6 +910,61 @@ def _part_path(dest: Path) -> Path:
     return dest.with_name(dest.name + ".part")
 
 
+def downloaded_bytes(dest: Path, size: int | None = None) -> int:
+    """Bytes of ``dest`` on disk (its ``.part`` while streaming), capped at ``size``."""
+    done = 0
+    for candidate in (_part_path(dest), dest):
+        try:
+            done = candidate.stat().st_size
+            break
+        except FileNotFoundError:
+            continue
+    return done if size is None else min(done, size)
+
+
+# Short lock-acquire slices so a waiting worker still observes Ctrl-C
+_DOWNLOAD_LOCK_POLL = 1
+
+# Waiting on another process's download; past this the caller leaves the
+# file to its holder (the later sequential install waits on the same lock)
+DOWNLOAD_LOCK_TIMEOUT = 60
+
+
+class DownloadLockUnavailable(OSError):
+    """The lock file cannot be used at all (a lock-less filesystem)."""
+
+
+def wait_for_download_lock(
+    lock: "FileLock",
+    tracker: Callable[[int], None],
+    on_disk: Callable[[], int],
+    name: str,
+) -> None:
+    """Acquire ``lock``, reporting ``on_disk()`` to ``tracker`` each poll so the
+    bar follows the holder's download. Raises filelock's ``Timeout`` once
+    ``DOWNLOAD_LOCK_TIMEOUT`` seconds pass."""
+    from filelock import Timeout
+
+    deadline = time.monotonic() + DOWNLOAD_LOCK_TIMEOUT
+    waiting = False
+    while True:
+        try:
+            lock.acquire(timeout=_DOWNLOAD_LOCK_POLL)
+            return
+        except Timeout:
+            pass
+        except OSError as err:
+            # Distinct from an OSError out of on_disk(), which must not
+            # read as "locks unsupported"
+            raise DownloadLockUnavailable(*err.args) from err
+        if not waiting:
+            waiting = True
+            _LOGGER.info("Waiting for another process downloading %s", name)
+        tracker(on_disk())  # raises when the batch is cancelled
+        if time.monotonic() >= deadline:
+            raise Timeout(lock.lock_file)
+
+
 def discard_partial_download(dest: Path) -> None:
     """Remove ``dest`` and the resume sidecars of an abandoned download."""
     part = _part_path(dest)
@@ -1319,10 +1375,7 @@ def download_from_mirrors(
             )
             # Tick with the bytes already on disk so a combined bar holds
             # steady during the backoff instead of rewinding to zero
-            done = 0
-            if progress is not None:
-                part = _part_path(path_target)
-                done = part.stat().st_size if part.is_file() else 0
+            done = downloaded_bytes(path_target) if progress is not None else 0
             _cancellable_sleep(delay, progress, done)
 
     # 3. Report every attempted URL if all mirrors failed. failures spans

esphome/platformio/prefetch.py

@@ -33,11 +33,14 @@ import time
 from typing import Any, NamedTuple
 
 from esphome.framework_helpers import (
+    DownloadLockUnavailable,
     content_length,
     discard_partial_download,
+    downloaded_bytes,
     failure_reason,
     resume_fetch_job,
     run_batch_downloads,
+    wait_for_download_lock,
     warn_prefetch_failures,
 )
 from esphome.helpers import get_bool_env, get_usable_cpu_count, rmtree
@@ -61,16 +64,10 @@ _RESOLVE_WORKERS = 8
 # A hung child must not block the build; downloads resume on the next run
 _PREFETCH_TIMEOUT = 20 * 60
 
-# Waiting on another process's URL download; past this, leave it to pio
-_DOWNLOAD_LOCK_TIMEOUT = 60
-
 # Child exit for a handled, already-warned failure; 1 would collide with
 # the interpreter's own import-failure exit
 _EXIT_HANDLED = 3
 
-# Short lock-acquire slices so a waiting worker still observes Ctrl-C
-_URI_LOCK_POLL = 1
-
 # Resolution errored (vs a clean skip); suppresses the warm sentinel
 _RESOLVE_FAILED = object()
 
@@ -462,51 +459,54 @@ def _uri_jobs(
 
 
 def _serialized_fetch_job(
-    dl_path: Path, lock_path: str, body: Any, unlocked_ok: bool = True
+    dl_path: Path,
+    lock_path: str,
+    body: Any,
+    size: int,
+    stream_dest: Path | None = None,
+    unlocked_ok: bool = True,
 ) -> Any:
-    """Wrap ``body`` so the shared destination is single-writer.
-
-    Interleaved writers truncate each other's ``.part`` bytes (see
-    registry.py). The bounded poll observes Ctrl-C via the tracker; a
-    blown deadline is a clean skip (the holder's copy is what the build
-    needs). On a lock-less filesystem a sha256-verified body runs
-    unlocked with one warning; a checksum-less one
-    (``unlocked_ok=False``) is a counted failure instead.
+    """Wrap ``body`` so the shared destination is single-writer (interleaved
+    writers truncate each other's ``.part``, see registry.py). A blown deadline
+    is a clean skip. On a lock-less filesystem a sha256-verified body runs
+    unlocked with one warning; a checksum-less one (``unlocked_ok=False``) fails.
     """
 
+    def on_disk() -> int:
+        # A URL job's holder streams beside the staging path until it
+        # promotes; after that only dl_path is left
+        done = downloaded_bytes(dl_path, size)
+        if not done and stream_dest is not None:
+            done = downloaded_bytes(stream_dest, size)
+        return done
+
     def run(tracker: Any) -> None:
         from filelock import FileLock, Timeout
 
         # fallback_to_soft would leave a stale marker on lock-less
         # filesystems that blocks every later build (see git.py)
         lock = FileLock(lock_path, fallback_to_soft=False)
-        deadline = time.monotonic() + _DOWNLOAD_LOCK_TIMEOUT
-        while True:
-            try:
-                lock.acquire(timeout=_URI_LOCK_POLL)
-                break
-            except Timeout:
-                tracker(0)  # raises when the batch is cancelled
-                if time.monotonic() >= deadline:
-                    # Another process is fetching this same file; its copy
-                    # is what the build needs (a large framework archive
-                    # can hold the lock far longer than this deadline)
-                    _LOGGER.debug("Leaving %s to its current downloader", dl_path.name)
-                    return
-            except OSError as err:
-                if not unlocked_ok:
-                    # A body with no checksum to catch interleaved corruption
-                    raise
-                lock = None
-                _LOGGER.warning(
-                    "Could not lock %s (%s); downloading unlocked",
-                    dl_path.name,
-                    err,
-                )
-                break
+        try:
+            wait_for_download_lock(lock, tracker, on_disk, dl_path.name)
+        except Timeout:
+            # The holder's copy is what the build needs (a large
+            # framework archive can outlast this deadline)
+            _LOGGER.debug("Leaving %s to its current downloader", dl_path.name)
+            return
+        except DownloadLockUnavailable as err:
+            if not unlocked_ok:
+                # A body with no checksum to catch interleaved corruption
+                raise
+            lock = None
+            _LOGGER.warning(
+                "Could not lock %s (%s); downloading unlocked",
+                dl_path.name,
+                err,
+            )
         try:
             if dl_path.is_file():
-                return  # another process finished it while we waited
+                tracker(size)  # another process finished it while we waited
+                return
             body(tracker)
         finally:
             if lock is not None:
@@ -540,6 +540,7 @@ def _registry_fetch_job(
         dl_path,
         f"{dl_path}.esphome.lock",
         resume_fetch_job(url, dl_path, sha256=checksum, size=size),
+        size,
     )
 
     def run(tracker: Any) -> None:
@@ -571,9 +572,9 @@ def _uri_fetch_job(manager: Any, url: str, dl_path: Path, size: int) -> Any:
         tmp.replace(dl_path)
 
     def run(tracker: Any) -> None:
-        _serialized_fetch_job(dl_path, f"{tmp}.lock", promote, unlocked_ok=False)(
-            tracker
-        )
+        _serialized_fetch_job(
+            dl_path, f"{tmp}.lock", promote, size, tmp, unlocked_ok=False
+        )(tracker)
         if dl_path.is_file():
             # Won or lost, the race is over; staging files left behind
             # are dead weight PlatformIO's cache never prunes

esphome/platformio/registry.py

@@ -17,8 +17,10 @@ from esphome.framework_helpers import (
     archive_extract_all,
     download_from_mirrors,
     download_with_resume,
+    downloaded_bytes,
     rmdir,
     run_batch_downloads,
+    wait_for_download_lock,
 )
 from esphome.net_retry import fetch_with_retry, http_request
 
@@ -164,11 +166,17 @@ class _PendingArchive(NamedTuple):
     name: str
     version: str
     dest: Path
+    archive: Path
     url: str
     sha256: str
     size: int
 
 
+def _archive_path(downloads_dir: Path, name: str, version: str) -> Path:
+    """The one archive path the prefetch and the sequential install share."""
+    return downloads_dir / f"{name}-{version}"
+
+
 def _already_installed(dest: Path) -> bool:
     """Whether ``dest`` holds a completed install (extraction marker)."""
     return (dest / ".esphome_extracted").is_file()
@@ -187,18 +195,18 @@ def prefetch_packages(
     lock as ``install_package``: the archive's ``.part`` file is shared, and
     two concurrent writers would truncate each other's bytes.
     """
-    from filelock import FileLock
+    from filelock import FileLock, Timeout
 
     pending: list[_PendingArchive] = []
-    seen: set[str] = set()
+    seen: set[Path] = set()
     for name, version, dest, mirrors in packages:
         if mirrors or (dest / ".esphome_extracted").is_file():
             continue
-        archive_name = f"{name}-{version}"
-        if archive_name in seen:
+        archive = _archive_path(downloads_dir, name, version)
+        if archive in seen:
             # A duplicate entry would race itself between two workers
             continue
-        seen.add(archive_name)
+        seen.add(archive)
         try:
             url, sha256, size = registry_download(name, version)
         except EsphomeError as err:
@@ -207,10 +215,9 @@ def prefetch_packages(
             continue
         if not size:
             continue
-        archive = downloads_dir / archive_name
         if archive.is_file() and archive.stat().st_size == size:
             continue
-        pending.append(_PendingArchive(name, version, dest, url, sha256, size))
+        pending.append(_PendingArchive(name, version, dest, archive, url, sha256, size))
     if len(pending) < 2:
         return
     downloads_dir.mkdir(parents=True, exist_ok=True)
@@ -222,20 +229,36 @@ def prefetch_packages(
 
     def _fetch(entry: _PendingArchive, tracker: Callable[[int], None]) -> None:
         entry.dest.parent.mkdir(parents=True, exist_ok=True)
-        with FileLock(f"{entry.dest}.lock", fallback_to_soft=False):
-            # Marker re-check: a concurrent build may have installed (and
-            # deleted the archive of) this package while we waited;
-            # re-downloading would orphan a fresh copy in downloads_dir
-            # no branch: the thread tracer misses the skip edge; both
-            # arms of _already_installed are pinned directly
-            if not _already_installed(entry.dest):  # pragma: no branch
-                download_with_resume(
-                    entry.url,
-                    downloads_dir / f"{entry.name}-{entry.version}",
-                    sha256=entry.sha256,
-                    size=entry.size,
-                    progress=tracker,
-                )
+
+        def on_disk() -> int:
+            if done := downloaded_bytes(entry.archive, entry.size):
+                return done
+            # The holder deletes the archive once it has installed it
+            return entry.size if _already_installed(entry.dest) else 0
+
+        lock = FileLock(f"{entry.dest}.lock", fallback_to_soft=False)
+        try:
+            wait_for_download_lock(lock, tracker, on_disk, entry.name)
+        except Timeout:
+            # install_package waits on this same lock and verifies the
+            # holder's copy
+            _LOGGER.debug("Leaving %s to its current downloader", entry.name)
+            return
+        try:
+            if _already_installed(entry.dest):
+                # A concurrent build installed it while we waited; a
+                # re-download would orphan a fresh copy in downloads_dir
+                tracker(entry.size)
+                return
+            download_with_resume(
+                entry.url,
+                entry.archive,
+                sha256=entry.sha256,
+                size=entry.size,
+                progress=tracker,
+            )
+        finally:
+            lock.release()
 
     failures = run_batch_downloads(
         "Downloading packages",
@@ -288,7 +311,7 @@ def install_package(
         rmdir(dest, msg=f"Clean up incomplete {name} install")
         # Persistent location so an interrupted download resumes across runs.
         downloads_dir.mkdir(parents=True, exist_ok=True)
-        archive = downloads_dir / f"{name}-{version}"
+        archive = _archive_path(downloads_dir, name, version)
         _LOGGER.info("Downloading %s %s ...", name, version)
         if mirrors:
             _LOGGER.warning(

esphome/wizard.py

@@ -148,11 +148,13 @@ def wizard_file(**kwargs: Unpack[WizardFileKwargs]) -> str:
     if "api_encryption_key" in kwargs:
         config += f'  encryption:\n    key: "{kwargs["api_encryption_key"]}"\n'
 
-    # Configure OTA
+    # The api key also secures OTA; a password only serves older uploaders
     config += "\nota:\n"
     config += "  - platform: esphome\n"
     if "ota_password" in kwargs:
         config += f'    password: "{kwargs["ota_password"]}"'
+    elif "api_encryption_key" in kwargs:
+        config += "    encryption:"
 
     # Configuring wifi
     config += "\n\nwifi:\n"
@@ -529,20 +531,9 @@ def wizard(path: Path) -> int:
         safe_print()
         safe_print("You'll need this key when adding the device to Home Assistant.")
         sleep(1)
-
-        safe_print()
-        safe_print(
-            f"Do you want to set a {color(AnsiFore.GREEN, 'password')} for OTA updates? "
-            "This can be insecure if you do not trust the WiFi network."
-        )
-        safe_print()
-        sleep(0.25)
-        safe_print("Press ENTER for no password")
-        ota_password = safe_input(color(AnsiFore.BOLD_WHITE, "(password): "))
     else:
         ssid, psk = "", ""
         api_encryption_key = None
-        ota_password = ""
 
     kwargs = {
         "path": path,
@@ -553,10 +544,9 @@ def wizard(path: Path) -> int:
         "psk": psk,
         "type": "basic",
     }
+    # The api key also secures OTA updates, so the wizard sets no OTA password
     if api_encryption_key:
         kwargs["api_encryption_key"] = api_encryption_key
-    if ota_password:
-        kwargs["ota_password"] = ota_password
 
     if not wizard_write(**kwargs):
         return 1

platformio.ini

@@ -45,7 +45,7 @@ lib_deps_base =
 lib_deps =
     ${common.lib_deps_base}
     https://github.com/dudanov/MideaUART.git#eeea6c3e9b4474f067054592b435be1c4e466815 ; midea
-    esphome/noise-c@0.1.21                 ; noise (api, ota)
+    esphome/noise-c@0.1.24                 ; noise (api, ota)
     improv/Improv@1.2.7                    ; improv_serial / esp32_improv
     kikuchan98/pngle@1.1.0                 ; online_image
     ; Using the repository directly, otherwise ESP-IDF can't use the library
@@ -244,7 +244,7 @@ lib_deps =
     ${common:idf-component-libs.lib_deps}
     ESP32Async/ESPAsyncWebServer@3.9.6    ; web_server_base
     droscy/esp_wireguard@0.4.5            ; wireguard
-    esphome/noise-c@0.1.21                ; noise (api, ota)
+    esphome/noise-c@0.1.24                ; noise (api, ota)
     ESP32Async/AsyncTCP@3.4.5             ; async_tcp
     DNSServer                             ; captive_portal
     heman/AsyncMqttClient-esphome@2.0.0   ; mqtt
@@ -641,7 +641,7 @@ build_unflags =
 extends = common
 platform = platformio/native
 lib_deps =
-    esphome/noise-c@0.1.21  ; used by noise (api, ota)
+    esphome/noise-c@0.1.24  ; used by noise (api, ota)
     lvgl/lvgl@9.5.0         ; lvgl
 build_flags =
     ${common.build_flags}

requirements.txt

@@ -27,7 +27,7 @@ bleak==3.0.2
 smpclient==7.2.0
 requests==2.34.2
 py7zr==1.1.3
-platformdirs==4.11.5  # native esp-idf toolchain global cache dir
+platformdirs==4.11.7  # native esp-idf toolchain global cache dir
 ninja==1.13.2  # native esp8266 arduino toolchain build driver
 filelock==3.32.5  # inter-process locks (PlatformIO cache heal, git clone cache); >=3.32 for FileLock(fallback_to_soft=...), older versions silently drop the kwarg
 

requirements_test.txt

@@ -2,7 +2,7 @@ pylint==4.0.8
 flake8==7.3.0  # also change in .pre-commit-config.yaml when updating
 ruff==0.16.5  # also change in .pre-commit-config.yaml when updating
 pyupgrade==3.21.2  # also change in .pre-commit-config.yaml when updating
-prek==0.5.0  # also change in .github/workflows/ci.yml when updating
+prek==0.5.1  # also change in .github/workflows/ci.yml when updating
 
 # Unit tests
 pytest==9.1.1

script/api_protobuf/api_protobuf.py

@@ -131,6 +131,12 @@ def force_str(force: bool) -> str:
     return str(force).lower()
 
 
+def _encode_call(func: str, *args: str, force: bool = False) -> str:
+    """Emit one ProtoEncode call; every helper takes the cursor and returns it advanced."""
+    suffix = "_force" if force else ""
+    return f"pos = ProtoEncode::{func}{suffix}({', '.join(('pos', *args))});"
+
+
 class TypeInfo(ABC):
     """Base class for all type information."""
 
@@ -264,14 +270,16 @@ class TypeInfo(ABC):
     # write_raw_byte(tag) + raw encode instead of the full encode_* method,
     # eliminating the zero-check branch and encode_field_raw indirection.
     # {value} is replaced with the actual field expression.
-    RAW_ENCODE_MAP: dict[str, str] = {
-        "encode_uint32": "ProtoEncode::encode_varint_raw(pos, {value});",
-        "encode_uint64": "ProtoEncode::encode_varint_raw_64(pos, {value});",
-        "encode_sint32": "ProtoEncode::encode_varint_raw_short(pos, encode_zigzag32({value}));",
-        "encode_sint64": "ProtoEncode::encode_varint_raw_64(pos, encode_zigzag64({value}));",
-        "encode_int64": "ProtoEncode::encode_varint_raw_64(pos, static_cast<uint64_t>({value}));",
-        "encode_bool": "ProtoEncode::write_raw_byte(pos, {value} ? 0x01 : 0x00);",
+    RAW_ENCODE_MAP: dict[str, tuple[str, str]] = {
+        "encode_uint32": ("encode_varint_raw", "{value}"),
+        "encode_uint64": ("encode_varint_raw_64", "{value}"),
+        "encode_sint32": ("encode_varint_raw_short", "encode_zigzag32({value})"),
+        "encode_sint64": ("encode_varint_raw_64", "encode_zigzag64({value})"),
+        "encode_int64": ("encode_varint_raw_64", "static_cast<uint64_t>({value})"),
+        "encode_bool": ("write_raw_byte", "{value} ? 0x01 : 0x00"),
     }
+    # Fixed32 value expression for the shared tag+fixed32 writer; None for other wire types
+    fixed32_value_template: str | None = None
 
     def _encode_with_precomputed_tag(self, value_expr: str) -> str | None:
         """Try to emit a precomputed-tag encode for a field.
@@ -288,12 +296,17 @@ class TypeInfo(ABC):
             return None
         max_val = self.max_value
         # Only use RAW_ENCODE_MAP for forced fields or fields with max_value
-        raw_expr = None
+        raw = None
         if self.force or max_val is not None:
-            raw_expr = self.RAW_ENCODE_MAP.get(self.encode_func)
-        if raw_expr is None:
+            raw = self.RAW_ENCODE_MAP.get(self.encode_func)
+        if raw is None:
             return None
-        body = f"ProtoEncode::write_raw_byte(pos, {tag});\n{raw_expr.format(value=value_expr)}"
+        func, arg = raw
+        body = (
+            _encode_call("write_raw_byte", str(tag))
+            + "\n"
+            + _encode_call(func, arg.format(value=value_expr))
+        )
         if self.force:
             return body
         # Non-forced with max_value: inline zero-check + raw encode
@@ -314,23 +327,43 @@ class TypeInfo(ABC):
             return None
         # When max_len < 128, length varint is always 1 byte
         len_encode = (
-            f"ProtoEncode::write_raw_byte(pos, static_cast<uint8_t>({len_expr}));"
+            _encode_call("write_raw_byte", f"static_cast<uint8_t>({len_expr})")
             if max_len is not None and max_len < 128
-            else f"ProtoEncode::encode_varint_raw(pos, {len_expr});"
+            else _encode_call("encode_varint_raw", len_expr)
         )
+        return "\n".join(
+            (
+                _encode_call("write_raw_byte", str(tag)),
+                len_encode,
+                _encode_call("encode_raw", data_expr, len_expr),
+            )
+        )
+
+    def _encode_fixed32_with_precomputed_tag(self, value_expr: str) -> str | None:
+        """Single-byte tag fixed32 write, or None for multi-byte tags."""
+        tag = self.calculate_tag()
+        if tag >= 128:
+            return None
+        if self.force:
+            return _encode_call("write_tag_and_fixed32", str(tag), value_expr)
         return (
-            f"ProtoEncode::write_raw_byte(pos, {tag});\n"
-            f"{len_encode}\n"
-            f"ProtoEncode::encode_raw(pos, {data_expr}, {len_expr});"
+            f"if (uint32_t raw = {value_expr}; raw != 0) [[likely]] {{\n"
+            f"  {_encode_call('write_tag_and_fixed32', str(tag), 'raw')}\n"
+            "}"
         )
 
     @property
     def encode_content(self) -> str:
-        if result := self._encode_with_precomputed_tag(f"this->{self.field_name}"):
+        value = f"this->{self.field_name}"
+        if result := self._encode_with_precomputed_tag(value):
             return result
-        if self.force:
-            return f"ProtoEncode::{self.encode_func}(pos, {self.number}, this->{self.field_name}, true);"
-        return f"ProtoEncode::{self.encode_func}(pos, {self.number}, this->{self.field_name});"
+        if self.fixed32_value_template is not None and (
+            result := self._encode_fixed32_with_precomputed_tag(
+                self.fixed32_value_template.format(value=value)
+            )
+        ):
+            return result
+        return _encode_call(self.encode_func, str(self.number), value, force=self.force)
 
     encode_func = None
 
@@ -635,6 +668,8 @@ class FloatType(FixedSizeTypeMixin, TypeInfo):
     encode_func = "encode_float"
     wire_type = WireType.FIXED32  # Uses wire type 5
 
+    fixed32_value_template = "float_to_raw({value})"
+
     def dump(self, name: str) -> str:
         o = f'snprintf(buffer, sizeof(buffer), "%g", {name});\n'
         o += "out.append(buffer);"
@@ -697,11 +732,11 @@ class UInt64Type(VarintTypeMixin, TypeInfo):
         return self._get_simple_size_calculation(name, force, "uint64")
 
     @property
-    def RAW_ENCODE_MAP(self) -> dict[str, str]:  # noqa: N802
+    def RAW_ENCODE_MAP(self) -> dict[str, tuple[str, str]]:  # noqa: N802
         if self.mac_address:
             return {
                 **TypeInfo.RAW_ENCODE_MAP,
-                "encode_uint64": "ProtoEncode::encode_varint_raw_48bit(pos, {value});",
+                "encode_uint64": ("encode_varint_raw_48bit", "{value}"),
             }
         return TypeInfo.RAW_ENCODE_MAP
 
@@ -769,15 +804,7 @@ class Fixed32Type(FixedSizeTypeMixin, TypeInfo):
         o += "out.append(buffer);"
         return o
 
-    @property
-    def encode_content(self) -> str:
-        tag = self.calculate_tag()
-        if self.force and tag < 128:
-            # Emit combined tag+value write: precomputed tag + direct memcpy
-            return f"ProtoEncode::write_tag_and_fixed32(pos, {tag}, this->{self.field_name});"
-        if self.force:
-            return f"ProtoEncode::{self.encode_func}(pos, {self.number}, this->{self.field_name}, true);"
-        return f"ProtoEncode::{self.encode_func}(pos, {self.number}, this->{self.field_name});"
+    fixed32_value_template = "{value}"
 
     def get_size_calculation(self, name: str, force: bool = False) -> str:
         field_id_size = self.calculate_field_id_size()
@@ -851,9 +878,12 @@ class StringType(TypeInfo):
             f"this->{self.field_name}_ref_.size()",
         ):
             return result
-        if self.force:
-            return f"ProtoEncode::encode_string(pos, {self.number}, this->{self.field_name}_ref_, true);"
-        return f"ProtoEncode::encode_string(pos, {self.number}, this->{self.field_name}_ref_);"
+        return _encode_call(
+            "encode_string",
+            str(self.number),
+            f"this->{self.field_name}_ref_",
+            force=self.force,
+        )
 
     def dump(self, name):
         # If name is 'it', this is a repeated field element - always use string
@@ -951,7 +981,9 @@ class MessageType(TypeInfo):
     @property
     def encode_content(self) -> str:
         # Sub-message encoding needs buffer for backpatch/sync
-        return f"ProtoEncode::{self.encode_func}(pos, buffer, {self.number}, this->{self.field_name});"
+        return _encode_call(
+            self.encode_func, "buffer", str(self.number), f"this->{self.field_name}"
+        )
 
     @property
     def decode_length(self) -> str:
@@ -1058,9 +1090,13 @@ class BytesType(TypeInfo):
             f"this->{self.field_name}_ptr_", f"this->{self.field_name}_len_"
         ):
             return result
-        if self.force:
-            return f"ProtoEncode::encode_bytes(pos, {self.number}, this->{self.field_name}_ptr_, this->{self.field_name}_len_, true);"
-        return f"ProtoEncode::encode_bytes(pos, {self.number}, this->{self.field_name}_ptr_, this->{self.field_name}_len_);"
+        return _encode_call(
+            "encode_bytes",
+            str(self.number),
+            f"this->{self.field_name}_ptr_",
+            f"this->{self.field_name}_len_",
+            force=self.force,
+        )
 
     def dump(self, name: str) -> str:
         ptr_dump = f"format_hex_pretty(this->{self.field_name}_ptr_, this->{self.field_name}_len_)"
@@ -1170,9 +1206,13 @@ class PointerToBytesBufferType(PointerToBufferTypeBase):
             f"this->{self.field_name}", f"this->{self.field_name}_len"
         ):
             return result
-        if self.force:
-            return f"ProtoEncode::encode_bytes(pos, {self.number}, this->{self.field_name}, this->{self.field_name}_len, true);"
-        return f"ProtoEncode::encode_bytes(pos, {self.number}, this->{self.field_name}, this->{self.field_name}_len);"
+        return _encode_call(
+            "encode_bytes",
+            str(self.number),
+            f"this->{self.field_name}",
+            f"this->{self.field_name}_len",
+            force=self.force,
+        )
 
     @property
     def decode_length_content(self) -> str | None:
@@ -1224,16 +1264,19 @@ class PointerToStringBufferType(PointerToBufferTypeBase):
         if max_len is not None and max_len < 128 and self.force:
             tag = self.calculate_tag()
             if tag < 128:
-                return f"ProtoEncode::encode_short_string_force(pos, {tag}, this->{self.field_name});"
+                return _encode_call(
+                    "encode_short_string_force", str(tag), f"this->{self.field_name}"
+                )
         if result := self._encode_bytes_with_precomputed_tag(
             f"this->{self.field_name}.c_str()",
             f"this->{self.field_name}.size()",
         ):
             return result
-        if self.force:
-            return f"ProtoEncode::encode_string(pos, {self.number}, this->{self.field_name}, true);"
-        return (
-            f"ProtoEncode::encode_string(pos, {self.number}, this->{self.field_name});"
+        return _encode_call(
+            "encode_string",
+            str(self.number),
+            f"this->{self.field_name}",
+            force=self.force,
         )
 
     @property
@@ -1421,9 +1464,13 @@ class FixedArrayBytesType(TypeInfo):
             f"this->{self.field_name}", f"this->{self.field_name}_len", max_len=max_len
         ):
             return result
-        if self.force:
-            return f"ProtoEncode::encode_bytes(pos, {self.number}, this->{self.field_name}, this->{self.field_name}_len, true);"
-        return f"ProtoEncode::encode_bytes(pos, {self.number}, this->{self.field_name}, this->{self.field_name}_len);"
+        return _encode_call(
+            "encode_bytes",
+            str(self.number),
+            f"this->{self.field_name}",
+            f"this->{self.field_name}_len",
+            force=self.force,
+        )
 
     def dump(self, name: str) -> str:
         return f"out.append(format_hex_pretty({name}, {name}_len));"
@@ -1520,9 +1567,9 @@ class EnumType(VarintTypeMixin, TypeInfo):
     @property
     def encode_content(self) -> str:
         value_expr = f"static_cast<uint32_t>(this->{self.field_name})"
-        if self.force:
-            return f"ProtoEncode::{self.encode_func}(pos, {self.number}, {value_expr}, true);"
-        return f"ProtoEncode::{self.encode_func}(pos, {self.number}, {value_expr});"
+        return _encode_call(
+            self.encode_func, str(self.number), value_expr, force=self.force
+        )
 
     def dump(self, name: str) -> str:
         return f"out.append_p(proto_enum_to_string<{self.cpp_type}>({name}));"
@@ -1701,9 +1748,9 @@ def _generate_inline_encode_block(
 
     lines = []
     lines.append(f"auto &sub_msg = {element};")
-    lines.append(f"ProtoEncode::write_raw_byte(pos, {tag});")
+    lines.append(_encode_call("write_raw_byte", str(tag)))
     lines.append("uint8_t *len_pos = pos;")
-    lines.append("ProtoEncode::reserve_byte(pos);")
+    lines.append(_encode_call("reserve_byte"))
 
     # Generate inline field encoding for each sub-message field
     for field in sub_desc.field:
@@ -1775,17 +1822,22 @@ class FixedArrayRepeatedType(TypeInfo):
     def _encode_element(self, element: str) -> str:
         """Helper to generate encode statement for a single element."""
         if isinstance(self._ti, EnumType):
-            return f"ProtoEncode::{self._ti.encode_func}(pos, {self.number}, static_cast<uint32_t>({element}), true);"
+            return _encode_call(
+                self._ti.encode_func,
+                str(self.number),
+                f"static_cast<uint32_t>({element})",
+                force=True,
+            )
         # Repeated message elements use encode_sub_message (force=true is default)
         if isinstance(self._ti, MessageType):
             if _is_inline_encode(self._ti.cpp_type):
                 return _generate_inline_encode_block(
                     self.number, self._ti.cpp_type, element
                 )
-            return f"ProtoEncode::encode_sub_message(pos, buffer, {self.number}, {element});"
-        return (
-            f"ProtoEncode::{self._ti.encode_func}(pos, {self.number}, {element}, true);"
-        )
+            return _encode_call(
+                "encode_sub_message", "buffer", str(self.number), element
+            )
+        return _encode_call(self._ti.encode_func, str(self.number), element, force=True)
 
     @property
     def cpp_type(self) -> str:
@@ -2137,13 +2189,18 @@ class RepeatedTypeInfo(TypeInfo):
     def _encode_element_call(self, element: str) -> str:
         """Helper to generate encode call for a single element."""
         if isinstance(self._ti, EnumType):
-            return f"ProtoEncode::{self._ti.encode_func}(pos, {self.number}, static_cast<uint32_t>({element}), true);"
+            return _encode_call(
+                self._ti.encode_func,
+                str(self.number),
+                f"static_cast<uint32_t>({element})",
+                force=True,
+            )
         # Repeated message elements use encode_sub_message (force=true is default)
         if isinstance(self._ti, MessageType):
-            return f"ProtoEncode::encode_sub_message(pos, buffer, {self.number}, {element});"
-        return (
-            f"ProtoEncode::{self._ti.encode_func}(pos, {self.number}, {element}, true);"
-        )
+            return _encode_call(
+                "encode_sub_message", "buffer", str(self.number), element
+            )
+        return _encode_call(self._ti.encode_func, str(self.number), element, force=True)
 
     @property
     def encode_content(self) -> str:
@@ -2152,7 +2209,7 @@ class RepeatedTypeInfo(TypeInfo):
             # Special handling for const char* elements (when container_no_template contains "const char")
             if "const char" in self._container_no_template:
                 o = f"for (const char *it : *this->{self.field_name}) {{\n"
-                o += f"  ProtoEncode::{self._ti.encode_func}(pos, {self.number}, it, strlen(it), true);\n"
+                o += f"  {_encode_call(self._ti.encode_func, str(self.number), 'it', 'strlen(it)', force=True)}\n"
             else:
                 o = f"for (const auto &it : *this->{self.field_name}) {{\n"
                 o += f"  {self._encode_element_call('it')}\n"

script/ci-custom.py

@@ -294,6 +294,9 @@ def highlight(s):
         "esphome/components/socket/headers.h",
         "esphome/core/defines.h",
         "esphome/components/http_request/httplib.h",
+        # Shared C wire header (byte-identical with the co-processor firmware);
+        # these are protocol constants and constexpr is C++-only.
+        "esphome/components/esp32_hosted/esp_now_hosted_rpc.h",
     ],
 )
 def lint_no_defines(fname, match):
@@ -816,6 +819,10 @@ def lint_relative_py_import(fname: Path, line, col, content):
         "esphome/components/host/helpers.cpp",
         "esphome/components/zephyr/helpers.cpp",
         "esphome/components/http_request/httplib.h",
+        # Global extern "C" esp_now_* linker symbols + shared C wire header;
+        # neither can live in a C++ namespace.
+        "esphome/components/esp32_hosted/esp_now_hosted.cpp",
+        "esphome/components/esp32_hosted/esp_now_hosted_rpc.h",
     ],
 )
 def lint_namespace(fname: Path, content: str) -> str | None:
@@ -841,7 +848,15 @@ def lint_esphome_h(fname, line, col, content):
     )
 
 
-@lint_content_check(include=["*.h"], exclude=["esphome/core/entity_types.h"])
+@lint_content_check(
+    include=["*.h"],
+    exclude=[
+        "esphome/core/entity_types.h",
+        # Shared C wire header; uses a classic #ifndef guard for portability
+        # across the co-processor firmware repo it stays byte-identical with.
+        "esphome/components/esp32_hosted/esp_now_hosted_rpc.h",
+    ],
+)
 def lint_pragma_once(fname, content):
     if "#pragma once" not in content:
         return (

tests/component_tests/esp32_ble_server/config/improv_only.yaml

@@ -0,0 +1,13 @@
+esphome:
+  name: test
+
+esp32:
+  variant: esp32
+
+wifi:
+  ssid: MySSID
+  password: password1
+
+# esp32_ble_server is only auto-loaded here, so it has no services of its own.
+esp32_improv:
+  authorizer: none

tests/component_tests/esp32_ble_server/config/manufacturer_data_only.yaml

@@ -0,0 +1,9 @@
+esphome:
+  name: test
+
+esp32:
+  variant: esp32
+
+esp32_ble_server:
+  id: ble_server
+  manufacturer_data: [0x72, 0x04, 0x00, 0x23]

tests/component_tests/esp32_ble_server/config/own_service.yaml

@@ -0,0 +1,14 @@
+esphome:
+  name: test
+
+esp32:
+  variant: esp32
+
+esp32_ble_server:
+  id: ble_server
+  services:
+    - uuid: 2a24b789-7aab-4535-af3e-ee76a35cc12d
+      characteristics:
+        - uuid: cad48e28-7fbe-41cf-bae9-d77a6c233423
+          read: true
+          value: [1, 2, 3, 4]

tests/component_tests/esp32_ble_server/test_esp32_ble_server.py

@@ -1,5 +1,10 @@
 """Tests for esp32_ble_server configuration helpers."""
 
+from __future__ import annotations
+
+from collections.abc import Callable
+from pathlib import Path
+
 import pytest
 
 from esphome.components.esp32_ble_server import (
@@ -45,3 +50,26 @@ def test_uuid_is_matches_descriptor_short_strings(uuid16) -> None:
     assert uuid_is(uuid16, uuid16)
     assert uuid_is(f"{uuid16:04X}", uuid16)
     assert uuid_is(f"{uuid16:08X}", uuid16)
+
+
+@pytest.mark.parametrize(
+    ("config_file", "required"),
+    [
+        # Auto-loaded by esp32_improv only: nothing to find until Improv asks for it
+        ("improv_only.yaml", False),
+        # The configuration defines a service clients are meant to connect to
+        ("own_service.yaml", True),
+        # Manufacturer data is only useful if it is actually broadcast
+        ("manufacturer_data_only.yaml", True),
+    ],
+)
+def test_advertising_required(
+    generate_main: Callable[[str | Path], str],
+    component_config_path: Callable[[str], Path],
+    config_file: str,
+    required: bool,
+) -> None:
+    """The server only requests advertising when the configuration needs it."""
+    main_cpp = generate_main(component_config_path(config_file))
+
+    assert f"set_advertising_required({str(required).lower()})" in main_cpp

tests/component_tests/noise/test_encryption_key.py

@@ -5,11 +5,7 @@ from __future__ import annotations
 import pytest
 
 from esphome import config_validation as cv
-from esphome.components.noise import (
-    decode_encryption_key,
-    is_reserved_key,
-    validate_encryption_key,
-)
+from esphome.components.noise import decode_encryption_key, validate_encryption_key
 
 KEY = "AAECAwQFBgcICQoLDA0ODxAREhMUFRYXGBkaGxwdHh8="
 
@@ -41,6 +37,8 @@ def test_decode_encryption_key_rejects_short_decode() -> None:
         decode_encryption_key("AAECAw==")
 
 
-def test_is_reserved_key() -> None:
-    assert is_reserved_key("AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA=")
-    assert not is_reserved_key(KEY)
+def test_validate_encryption_key_rejects_all_zeros() -> None:
+    """The all-zeros key is the provisioning sentinel the device treats as no
+    key, so it never reaches a build."""
+    with pytest.raises(cv.Invalid, match="all-zeros key is reserved"):
+        validate_encryption_key("AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA=")

tests/component_tests/ota/test_esphome_ota.py

@@ -2,6 +2,7 @@
 
 from __future__ import annotations
 
+from collections.abc import Callable
 import logging
 from typing import Any
 
@@ -14,6 +15,7 @@ from esphome.components.esphome.ota import (
     _validate_no_password_with_encryption,
     ota_esphome_final_validate,
 )
+from esphome.components.noise import static_encryption_key
 from esphome.const import (
     CONF_API,
     CONF_ENCRYPTION,
@@ -115,7 +117,6 @@ def test_non_esphome_ota_unaffected() -> None:
 
 API_KEY = "AAECAwQFBgcICQoLDA0ODxAREhMUFRYXGBkaGxwdHh8="
 OTHER_KEY = "AQIDBAUGBwgJCgsMDQ4PEBESExQVFhcYGRobHB0eHyA="
-ZEROS_KEY = "AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA="
 
 
 def test_encryption_key_inherited_from_api() -> None:
@@ -197,36 +198,6 @@ def test_encryption_without_any_key_rejected() -> None:
         fv.full_config.reset(token)
 
 
-def test_encryption_explicit_all_zeros_key_rejected() -> None:
-    """The all-zeros key is the provisioning sentinel; the device would treat
-    it as no PSK and accept plaintext, so it must fail validation."""
-    full_conf = {
-        CONF_OTA: [
-            _make_ota_config(port=3232, **{CONF_ENCRYPTION: {CONF_KEY: ZEROS_KEY}})
-        ],
-    }
-    token = fv.full_config.set(full_conf)
-    try:
-        with pytest.raises(cv.Invalid, match="all-zeros key is reserved"):
-            ota_esphome_final_validate({})
-    finally:
-        fv.full_config.reset(token)
-
-
-def test_encryption_inherited_all_zeros_key_rejected() -> None:
-    """An all-zeros api key must not silently disable ota encryption either."""
-    full_conf = {
-        CONF_API: {CONF_ENCRYPTION: {CONF_KEY: ZEROS_KEY}},
-        CONF_OTA: [_make_ota_config(port=3232, **{CONF_ENCRYPTION: {}})],
-    }
-    token = fv.full_config.set(full_conf)
-    try:
-        with pytest.raises(cv.Invalid, match="all-zeros key is reserved"):
-            ota_esphome_final_validate({})
-    finally:
-        fv.full_config.reset(token)
-
-
 def test_encryption_key_mismatch_between_merged_configs_rejected() -> None:
     """Same-port configs with different encryption keys raise."""
     full_conf = {
@@ -295,13 +266,14 @@ def test_encryption_explicit_key_with_runtime_provisioned_api_accepted() -> None
         fv.full_config.reset(token)
 
 
+@pytest.mark.parametrize("component", ["web_server", "prometheus"])
 def test_encryption_with_web_server_ota_warns(
-    caplog: pytest.LogCaptureFixture,
+    caplog: pytest.LogCaptureFixture, component: str
 ) -> None:
-    """With the web_server component the plaintext /update endpoint is always
-    on; the combination validates with a warning."""
+    """web_server and prometheus keep the shared listener up, so the
+    plaintext /update endpoint is always on and the combination warns."""
     full_conf = {
-        "web_server": {},
+        component: {},
         CONF_OTA: [
             _make_ota_config(port=3232, **{CONF_ENCRYPTION: {CONF_KEY: OTHER_KEY}}),
             {CONF_PLATFORM: "web_server", CONF_ID: ID("ota_ws", is_manual=False)},
@@ -316,12 +288,12 @@ def test_encryption_with_web_server_ota_warns(
         fv.full_config.reset(token)
 
 
-def test_encryption_with_captive_portal_web_server_ota_warns(
+def test_encryption_with_captive_portal_does_not_warn(
     caplog: pytest.LogCaptureFixture,
 ) -> None:
     """captive_portal auto-loads the web_server ota platform without the
-    web_server component; encryption stays usable and only warns, so the
-    fallback AP recovery path is not lost."""
+    web_server component; its endpoint only exists while the fallback AP is
+    active and is the intended recovery path, so there is no warning."""
     full_conf = {
         "captive_portal": {},
         CONF_OTA: [
@@ -333,7 +305,10 @@ def test_encryption_with_captive_portal_web_server_ota_warns(
     try:
         with caplog.at_level(logging.WARNING):
             ota_esphome_final_validate({})
-        assert any("captive_portal" in record.message for record in caplog.records)
+        assert not any(
+            "OTA encryption does not cover" in record.message
+            for record in caplog.records
+        )
         esphome_conf = next(
             conf
             for conf in fv.full_config.get()[CONF_OTA]
@@ -344,6 +319,100 @@ def test_encryption_with_captive_portal_web_server_ota_warns(
         fv.full_config.reset(token)
 
 
+def test_password_with_api_key_warns(caplog: pytest.LogCaptureFixture) -> None:
+    """A static api key makes the device offer encryption and the CLI take
+    it, so the password is dead weight; the config validates with a warning."""
+    full_conf = {
+        CONF_API: {CONF_ENCRYPTION: {CONF_KEY: API_KEY}},
+        CONF_OTA: [_make_ota_config(port=3232, **{CONF_PASSWORD: "pw"})],
+    }
+    token = fv.full_config.set(full_conf)
+    try:
+        with caplog.at_level(logging.WARNING):
+            ota_esphome_final_validate({})
+        assert any("wastes significant flash" in r.message for r in caplog.records)
+    finally:
+        fv.full_config.reset(token)
+
+
+def test_password_with_runtime_api_key_warns_differently(
+    caplog: pytest.LogCaptureFixture,
+) -> None:
+    """The CLI still needs the password, but the provisioned key also
+    authenticates uploads; the warning says so without the flash advice."""
+    full_conf = {
+        CONF_API: {CONF_ENCRYPTION: {}},
+        CONF_OTA: [_make_ota_config(port=3232, **{CONF_PASSWORD: "pw"})],
+    }
+    token = fv.full_config.set(full_conf)
+    try:
+        with caplog.at_level(logging.WARNING):
+            ota_esphome_final_validate({})
+        messages = [r.message for r in caplog.records]
+        assert any("provisioned at runtime also authenticates" in m for m in messages)
+        assert not any("wastes significant flash" in m for m in messages)
+    finally:
+        fv.full_config.reset(token)
+
+
+def test_password_without_api_key_no_warning(
+    caplog: pytest.LogCaptureFixture,
+) -> None:
+    """Without an api key there is no offer, so nothing to warn about."""
+    full_conf = {
+        CONF_API: {},
+        CONF_OTA: [_make_ota_config(port=3232, **{CONF_PASSWORD: "pw"})],
+    }
+    token = fv.full_config.set(full_conf)
+    try:
+        with caplog.at_level(logging.WARNING):
+            ota_esphome_final_validate({})
+        assert not any("authenticates" in r.message for r in caplog.records)
+    finally:
+        fv.full_config.reset(token)
+
+
+def test_web_server_component_without_ota_platform_does_not_warn(
+    caplog: pytest.LogCaptureFixture,
+) -> None:
+    """The web_server component alone has no /update endpoint."""
+    full_conf = {
+        "web_server": {},
+        CONF_OTA: [
+            _make_ota_config(port=3232, **{CONF_ENCRYPTION: {CONF_KEY: OTHER_KEY}})
+        ],
+    }
+    token = fv.full_config.set(full_conf)
+    try:
+        with caplog.at_level(logging.WARNING):
+            ota_esphome_final_validate({})
+        assert not any(
+            "OTA encryption does not cover" in r.message for r in caplog.records
+        )
+    finally:
+        fv.full_config.reset(token)
+
+
+def test_web_server_ota_platform_alone_does_not_warn(
+    caplog: pytest.LogCaptureFixture,
+) -> None:
+    """Only the web_server component starts the shared listener, so the ota
+    platform on its own never exposes /update."""
+    full_conf = {
+        CONF_OTA: [
+            _make_ota_config(port=3232, **{CONF_ENCRYPTION: {CONF_KEY: OTHER_KEY}}),
+            {CONF_PLATFORM: "web_server", CONF_ID: ID("ota_ws", is_manual=False)},
+        ],
+    }
+    token = fv.full_config.set(full_conf)
+    try:
+        with caplog.at_level(logging.WARNING):
+            ota_esphome_final_validate({})
+        assert not any("plaintext /update" in r.message for r in caplog.records)
+    finally:
+        fv.full_config.reset(token)
+
+
 def test_web_server_ota_without_encryption_unaffected() -> None:
     """web_server ota stays valid alongside an unencrypted esphome entry."""
     full_conf = {
@@ -370,20 +439,87 @@ def test_auto_load_pulls_noise_only_for_encryption() -> None:
     assert "noise" in AUTO_LOAD({})
 
 
-def test_filter_source_files_excludes_noise_without_encryption() -> None:
-    """The noise transport source compiles only for encrypted builds."""
-    old_config = CORE.config
-    try:
-        CORE.config = {CONF_OTA: [_make_ota_config(port=3232)]}
-        assert FILTER_SOURCE_FILES() == ["ota_esphome_noise.cpp"]
-        CORE.config = {
-            CONF_OTA: [
-                _make_ota_config(port=3232, **{CONF_ENCRYPTION: {CONF_KEY: API_KEY}})
-            ]
-        }
-        assert FILTER_SOURCE_FILES() == []
-    finally:
-        CORE.config = old_config
+def test_static_encryption_key() -> None:
+    """Only a build-time key counts; a runtime provisioned one does not."""
+    assert static_encryption_key({}) is None
+    assert static_encryption_key({CONF_ENCRYPTION: {}}) is None
+    assert static_encryption_key({CONF_ENCRYPTION: {CONF_KEY: API_KEY}}) == API_KEY
+
+
+@pytest.mark.parametrize(
+    ("yaml_name", "defines_present", "defines_absent"),
+    [
+        # An api key alone compiles the transport in without requiring it;
+        # the device uses the api server's key, not a copy
+        (
+            "api_key_offer",
+            {"USE_OTA_ENCRYPTION", "USE_OTA_ENCRYPTION_FROM_API"},
+            {"USE_OTA_ENCRYPTION_REQUIRED", "USE_OTA_ENCRYPTION_PROVISIONED"},
+        ),
+        # A password still guards plaintext uploads on an offering device
+        (
+            "api_key_offer_password",
+            {"USE_OTA_ENCRYPTION", "USE_OTA_ENCRYPTION_FROM_API", "USE_OTA_PASSWORD"},
+            {"USE_OTA_ENCRYPTION_REQUIRED", "USE_OTA_ENCRYPTION_PROVISIONED"},
+        ),
+        # The ota encryption block is what makes the device refuse plaintext
+        (
+            "encryption_required",
+            {
+                "USE_OTA_ENCRYPTION",
+                "USE_OTA_ENCRYPTION_REQUIRED",
+                "USE_OTA_ENCRYPTION_FROM_API",
+            },
+            {"USE_OTA_ENCRYPTION_PROVISIONED"},
+        ),
+        # Without api encryption the ota key is the device's own
+        (
+            "own_key",
+            {"USE_OTA_ENCRYPTION", "USE_OTA_ENCRYPTION_REQUIRED"},
+            {"USE_OTA_ENCRYPTION_FROM_API", "USE_OTA_ENCRYPTION_PROVISIONED"},
+        ),
+        # A key provisioned at runtime lives in the api server; the device
+        # offers with it once provisioned and never requires it
+        (
+            "runtime_api_key",
+            {
+                "USE_OTA_ENCRYPTION",
+                "USE_OTA_ENCRYPTION_FROM_API",
+                "USE_OTA_ENCRYPTION_PROVISIONED",
+            },
+            {"USE_OTA_ENCRYPTION_REQUIRED"},
+        ),
+        # No api encryption at all keeps the noise glue out of the build
+        (
+            "plain",
+            set(),
+            {
+                "USE_OTA_ENCRYPTION",
+                "USE_OTA_ENCRYPTION_REQUIRED",
+                "USE_OTA_ENCRYPTION_FROM_API",
+                "USE_OTA_ENCRYPTION_PROVISIONED",
+            },
+        ),
+    ],
+)
+def test_encryption_offer_codegen(
+    generate_main: Callable[[str], str],
+    yaml_name: str,
+    defines_present: set[str],
+    defines_absent: set[str],
+) -> None:
+    main_cpp = generate_main(
+        f"tests/component_tests/ota/test_esphome_ota_{yaml_name}.yaml"
+    )
+    defines = {define.name for define in CORE.defines}
+    assert defines_present <= defines
+    assert not (defines_absent & defines)
+    encrypted = "USE_OTA_ENCRYPTION" in defines_present
+    own_key = encrypted and "USE_OTA_ENCRYPTION_FROM_API" not in defines_present
+    assert ("esphome_esphomeotacomponent_id->set_noise_psk(" in main_cpp) is own_key
+    assert ("set_auth_password(" in main_cpp) is ("USE_OTA_PASSWORD" in defines_present)
+    # The noise transport source compiles only when the define is set
+    assert FILTER_SOURCE_FILES() == ([] if encrypted else ["ota_esphome_noise.cpp"])
 
 
 def test_password_with_encryption_rejected() -> None:

tests/component_tests/ota/test_esphome_ota_api_key_offer.yaml

@@ -0,0 +1,11 @@
+esphome:
+  name: ota-offer
+
+host:
+
+api:
+  encryption:
+    key: "AAECAwQFBgcICQoLDA0ODxAREhMUFRYXGBkaGxwdHh8="
+
+ota:
+  - platform: esphome

tests/component_tests/ota/test_esphome_ota_api_key_offer_password.yaml

@@ -0,0 +1,12 @@
+esphome:
+  name: ota-offer-password
+
+host:
+
+api:
+  encryption:
+    key: "AAECAwQFBgcICQoLDA0ODxAREhMUFRYXGBkaGxwdHh8="
+
+ota:
+  - platform: esphome
+    password: "superlongpasswordthatnoonewillknow"

tests/component_tests/ota/test_esphome_ota_encryption_required.yaml

@@ -0,0 +1,12 @@
+esphome:
+  name: ota-encryption-required
+
+host:
+
+api:
+  encryption:
+    key: "AAECAwQFBgcICQoLDA0ODxAREhMUFRYXGBkaGxwdHh8="
+
+ota:
+  - platform: esphome
+    encryption:

tests/component_tests/ota/test_esphome_ota_own_key.yaml

@@ -0,0 +1,11 @@
+esphome:
+  name: ota-own-key
+
+host:
+
+api:
+
+ota:
+  - platform: esphome
+    encryption:
+      key: "AAECAwQFBgcICQoLDA0ODxAREhMUFRYXGBkaGxwdHh8="

tests/component_tests/ota/test_esphome_ota_plain.yaml

@@ -0,0 +1,9 @@
+esphome:
+  name: ota-plain
+
+host:
+
+api:
+
+ota:
+  - platform: esphome

tests/component_tests/ota/test_esphome_ota_runtime_api_key.yaml

@@ -0,0 +1,10 @@
+esphome:
+  name: ota-runtime-key
+
+host:
+
+api:
+  encryption:
+
+ota:
+  - platform: esphome

tests/component_tests/template/test_template_climate.py

@@ -0,0 +1,145 @@
+"""Tests for template climate config validation."""
+
+import pytest
+
+from esphome import config_validation as cv
+from esphome.components.template.climate import (
+    CONF_SET_TARGET_HUMIDITY_ACTION,
+    CONF_SET_TARGET_TEMPERATURE_ACTION,
+    CONF_SET_TARGET_TEMPERATURE_HIGH_ACTION,
+    CONF_SET_TARGET_TEMPERATURE_LOW_ACTION,
+    CONF_SUPPORTS_CURRENT_HUMIDITY,
+    CONF_SUPPORTS_CURRENT_TEMPERATURE,
+    CONF_SUPPORTS_TARGET_HUMIDITY,
+    CONF_SUPPORTS_TWO_POINT_TARGET_TEMPERATURE,
+    CONF_TARGET_HUMIDITY,
+    _resolve_supports,
+    _validate_initial_state,
+    _validate_set_actions,
+)
+from esphome.const import (
+    CONF_HUMIDITY_SENSOR,
+    CONF_INITIAL_STATE,
+    CONF_SENSOR,
+    CONF_TARGET_TEMPERATURE,
+    CONF_TARGET_TEMPERATURE_HIGH,
+    CONF_TARGET_TEMPERATURE_LOW,
+)
+from esphome.types import ConfigType
+
+
+def test_supports_current_temperature_derived_from_sensor() -> None:
+    config: ConfigType = {CONF_SENSOR: "some_sensor"}
+    assert _resolve_supports(config)[CONF_SUPPORTS_CURRENT_TEMPERATURE] is True
+
+
+def test_supports_current_temperature_false_without_sensor() -> None:
+    assert _resolve_supports({})[CONF_SUPPORTS_CURRENT_TEMPERATURE] is False
+
+
+def test_supports_current_temperature_explicit_true_without_sensor_allowed() -> None:
+    # The value can still be reported with climate.template.publish.
+    config: ConfigType = {CONF_SUPPORTS_CURRENT_TEMPERATURE: True}
+    assert _resolve_supports(config)[CONF_SUPPORTS_CURRENT_TEMPERATURE] is True
+
+
+def test_supports_current_temperature_false_with_sensor_rejected() -> None:
+    config: ConfigType = {
+        CONF_SENSOR: "some_sensor",
+        CONF_SUPPORTS_CURRENT_TEMPERATURE: False,
+    }
+    with pytest.raises(cv.Invalid, match="cannot be false"):
+        _resolve_supports(config)
+
+
+def test_supports_current_humidity_false_with_sensor_rejected() -> None:
+    config: ConfigType = {
+        CONF_HUMIDITY_SENSOR: "some_sensor",
+        CONF_SUPPORTS_CURRENT_HUMIDITY: False,
+    }
+    with pytest.raises(cv.Invalid, match="cannot be false"):
+        _resolve_supports(config)
+
+
+def test_two_point_derived_from_set_actions() -> None:
+    config: ConfigType = {
+        CONF_SET_TARGET_TEMPERATURE_LOW_ACTION: [{}],
+        CONF_SET_TARGET_TEMPERATURE_HIGH_ACTION: [{}],
+    }
+    assert _resolve_supports(config)[CONF_SUPPORTS_TWO_POINT_TARGET_TEMPERATURE] is True
+
+
+def test_two_point_false_with_set_action_rejected() -> None:
+    config: ConfigType = {
+        CONF_SET_TARGET_TEMPERATURE_LOW_ACTION: [{}],
+        CONF_SUPPORTS_TWO_POINT_TARGET_TEMPERATURE: False,
+    }
+    with pytest.raises(cv.Invalid, match="cannot be false"):
+        _resolve_supports(config)
+
+
+def test_target_humidity_derived_from_set_action() -> None:
+    config: ConfigType = {CONF_SET_TARGET_HUMIDITY_ACTION: [{}]}
+    assert _resolve_supports(config)[CONF_SUPPORTS_TARGET_HUMIDITY] is True
+
+
+def test_set_target_temperature_low_requires_high() -> None:
+    config: ConfigType = {CONF_SET_TARGET_TEMPERATURE_LOW_ACTION: [{}]}
+    with pytest.raises(cv.Invalid, match="must be used together"):
+        _validate_set_actions(config)
+
+
+def test_set_target_temperature_conflicts_with_two_point_actions() -> None:
+    config: ConfigType = {
+        CONF_SET_TARGET_TEMPERATURE_ACTION: [{}],
+        CONF_SET_TARGET_TEMPERATURE_LOW_ACTION: [{}],
+        CONF_SET_TARGET_TEMPERATURE_HIGH_ACTION: [{}],
+    }
+    with pytest.raises(cv.Invalid, match="cannot be used together"):
+        _validate_set_actions(config)
+
+
+def test_initial_state_target_temperature_rejected_with_two_point() -> None:
+    config: ConfigType = {
+        CONF_SUPPORTS_TWO_POINT_TARGET_TEMPERATURE: True,
+        CONF_SUPPORTS_TARGET_HUMIDITY: False,
+        CONF_INITIAL_STATE: {CONF_TARGET_TEMPERATURE: 21.0},
+    }
+    with pytest.raises(cv.Invalid, match="is not available"):
+        _validate_initial_state(config)
+
+
+def test_initial_state_two_point_values_rejected_without_two_point() -> None:
+    config: ConfigType = {
+        CONF_SUPPORTS_TWO_POINT_TARGET_TEMPERATURE: False,
+        CONF_SUPPORTS_TARGET_HUMIDITY: False,
+        CONF_INITIAL_STATE: {
+            CONF_TARGET_TEMPERATURE_LOW: 18.0,
+            CONF_TARGET_TEMPERATURE_HIGH: 24.0,
+        },
+    }
+    with pytest.raises(cv.Invalid, match="requires"):
+        _validate_initial_state(config)
+
+
+def test_initial_state_target_humidity_rejected_without_support() -> None:
+    config: ConfigType = {
+        CONF_SUPPORTS_TWO_POINT_TARGET_TEMPERATURE: False,
+        CONF_SUPPORTS_TARGET_HUMIDITY: False,
+        CONF_INITIAL_STATE: {CONF_TARGET_HUMIDITY: 50},
+    }
+    with pytest.raises(cv.Invalid, match="requires"):
+        _validate_initial_state(config)
+
+
+def test_initial_state_matching_two_point_accepted() -> None:
+    config: ConfigType = {
+        CONF_SUPPORTS_TWO_POINT_TARGET_TEMPERATURE: True,
+        CONF_SUPPORTS_TARGET_HUMIDITY: True,
+        CONF_INITIAL_STATE: {
+            CONF_TARGET_TEMPERATURE_LOW: 18.0,
+            CONF_TARGET_TEMPERATURE_HIGH: 24.0,
+            CONF_TARGET_HUMIDITY: 50,
+        },
+    }
+    assert _validate_initial_state(config) is config

tests/components/api/test_proto_mac_varint.cpp

@@ -59,7 +59,7 @@ static void verify_mac(uint64_t mac, size_t expected_bytes) {
 #ifdef ESPHOME_DEBUG_API
   uint8_t *proto_debug_end_ = api_buf.data() + api_buf.size();
 #endif
-  ProtoEncode::encode_varint_raw_48bit(pos PROTO_ENCODE_DEBUG_ARG, mac);
+  pos = ProtoEncode::encode_varint_raw_48bit(pos PROTO_ENCODE_DEBUG_ARG, mac);
   size_t new_len = pos - api_buf.data();
 
   EXPECT_EQ(new_len, expected_bytes) << "mac=0x" << std::hex << mac << std::dec;

tests/components/climate/common.yaml

@@ -30,8 +30,7 @@ climate:
       - switch.turn_on: climate_heater_switch
       - switch.turn_off: climate_cooler_switch
   # Thermostat-based climate so climate.control: action variants get build
-  # coverage (bang_bang doesn't support fan modes, presets, etc.). Climate
-  # has no template platform, so thermostat is the right vehicle.
+  # coverage (bang_bang doesn't support fan modes, presets, etc.).
   - platform: thermostat
     id: climate_test_thermostat
     name: Test Thermostat

tests/components/esp32_hosted/test-espnow.esp32-p4-idf.yaml

@@ -0,0 +1,5 @@
+# Exercises the ESP-NOW-over-hosted shim: on the ESP32-P4 host, esp32_hosted
+# supplies the esp_now_* symbols that the espnow component links against.
+packages:
+  esp32_hosted: !include common.yaml
+  espnow: !include ../espnow/common.yaml

tests/components/noise/test_noise_handshake.cpp

@@ -68,6 +68,14 @@ class Initiator {
 
 static const uint8_t PROLOGUE[] = {'t', 'e', 's', 't', 'p', 'r', 'o', 'l', 'o', 'g', 'u', 'e'};
 
+// The context only points at the key and init() copies it before returning,
+// so a temporary context over a temporary key is safe within one call
+static NoiseContext ctx_for(const psk_t &psk) {
+  NoiseContext ctx;
+  ctx.set_psk(psk.data());
+  return ctx;
+}
+
 static psk_t make_psk(uint8_t seed) {
   psk_t psk;
   for (size_t i = 0; i < psk.size(); i++) {
@@ -102,7 +110,7 @@ TEST(NoiseResponderHandshakeTest, MessageMethodsErrorBeforeInit) {
 TEST(NoiseResponderHandshakeTest, FullHandshakeAndTransportRoundTrip) {
   const psk_t psk = make_psk(7);
   NoiseResponderHandshake responder;
-  ASSERT_EQ(responder.init(psk, PROLOGUE, sizeof(PROLOGUE)), 0);
+  ASSERT_EQ(responder.init(ctx_for(psk), PROLOGUE, sizeof(PROLOGUE)), 0);
   EXPECT_EQ(responder.action(), Action::ACTION_READ);
 
   Initiator initiator(psk, PROLOGUE, sizeof(PROLOGUE));
@@ -155,8 +163,8 @@ TEST(NoiseResponderHandshakeTest, ReInitRestartsHandshake) {
   // proves the restart took effect; the old state surviving would fail the
   // MAC here.
   NoiseResponderHandshake responder;
-  ASSERT_EQ(responder.init(make_psk(7), PROLOGUE, sizeof(PROLOGUE)), 0);
-  ASSERT_EQ(responder.init(make_psk(9), PROLOGUE, sizeof(PROLOGUE)), 0);
+  ASSERT_EQ(responder.init(ctx_for(make_psk(7)), PROLOGUE, sizeof(PROLOGUE)), 0);
+  ASSERT_EQ(responder.init(ctx_for(make_psk(9)), PROLOGUE, sizeof(PROLOGUE)), 0);
   EXPECT_EQ(responder.action(), Action::ACTION_READ);
 
   Initiator initiator(make_psk(9), PROLOGUE, sizeof(PROLOGUE));
@@ -168,7 +176,7 @@ TEST(NoiseResponderHandshakeTest, ReInitRestartsHandshake) {
 
 TEST(NoiseResponderHandshakeTest, WrongPskFailsWithMacFailure) {
   NoiseResponderHandshake responder;
-  ASSERT_EQ(responder.init(make_psk(7), PROLOGUE, sizeof(PROLOGUE)), 0);
+  ASSERT_EQ(responder.init(ctx_for(make_psk(7)), PROLOGUE, sizeof(PROLOGUE)), 0);
 
   Initiator initiator(make_psk(200), PROLOGUE, sizeof(PROLOGUE));
   uint8_t msg[MAX_HANDSHAKE_SIZE];
@@ -185,7 +193,7 @@ TEST(NoiseResponderHandshakeTest, MismatchedPrologueFailsWithMacFailure) {
   // tampered preamble must fail even with the right key.
   const psk_t psk = make_psk(7);
   NoiseResponderHandshake responder;
-  ASSERT_EQ(responder.init(psk, PROLOGUE, sizeof(PROLOGUE)), 0);
+  ASSERT_EQ(responder.init(ctx_for(psk), PROLOGUE, sizeof(PROLOGUE)), 0);
 
   static const uint8_t TAMPERED[] = {'x'};
   Initiator initiator(psk, TAMPERED, sizeof(TAMPERED));

tests/components/noise/test_noise_primitives.cpp

@@ -17,12 +17,17 @@ TEST(NoiseContextTest, AllZerosPskIsReserved) {
   EXPECT_FALSE(NoiseContext::is_all_zeros(psk));
 
   NoiseContext ctx;
+  psk_t loaded;
   EXPECT_FALSE(ctx.has_psk());
-  ctx.set_psk(zeros);
-  EXPECT_FALSE(ctx.has_psk());
-  ctx.set_psk(psk);
+  ctx.load_psk(loaded);
+  EXPECT_EQ(loaded, zeros);
+  ctx.set_psk(psk.data());
   EXPECT_TRUE(ctx.has_psk());
-  EXPECT_EQ(ctx.get_psk(), psk);
+  ctx.load_psk(loaded);
+  EXPECT_EQ(loaded, psk);
+  // Callers map the reserved key to nullptr; the context just stores what it is given
+  ctx.set_psk(nullptr);
+  EXPECT_FALSE(ctx.has_psk());
 }
 
 TEST(WireFormatTest, FrameHeaderIsIndicatorPlusBigEndianLength) {

tests/components/ota/api_key_offer.yaml

@@ -0,0 +1,12 @@
+wifi:
+  ssid: MySSID
+  password: password1
+
+api:
+  encryption:
+    key: "AAECAwQFBgcICQoLDA0ODxAREhMUFRYXGBkaGxwdHh8="
+
+ota:
+  - platform: esphome
+    port: 3290
+    password: "superlongpasswordthatnoonewillknow"

tests/components/ota/api_runtime_key.yaml

@@ -0,0 +1,10 @@
+wifi:
+  ssid: MySSID
+  password: password1
+
+api:
+  encryption:
+
+ota:
+  - platform: esphome
+    port: 3291

tests/components/ota/test-api_key_offer.esp32-idf.yaml

@@ -0,0 +1,2 @@
+packages:
+  ota: !include api_key_offer.yaml

tests/components/ota/test-api_key_offer.esp8266-ard.yaml

@@ -0,0 +1,2 @@
+packages:
+  ota: !include api_key_offer.yaml

tests/components/ota/test-api_runtime_key.esp32-idf.yaml

@@ -0,0 +1,2 @@
+packages:
+  ota: !include api_runtime_key.yaml

tests/components/ota/test-api_runtime_key.esp8266-ard.yaml

@@ -0,0 +1,2 @@
+packages:
+  ota: !include api_runtime_key.yaml

tests/components/template/common-base.yaml

@@ -25,6 +25,27 @@ esphome:
         away: !lambda "return true;"
         is_on: !lambda "return false;"
 
+    - climate.template.publish:
+        id: template_climate
+        current_temperature: 21.0
+        mode: HEAT
+        fan_mode: AUTO
+        swing_mode: "OFF"
+        preset: NONE
+        target_temperature: 22.0
+
+    # Templated
+    - climate.template.publish:
+        id: template_climate
+        current_temperature: !lambda "return 21.5f;"
+        mode: !lambda "return climate::CLIMATE_MODE_COOL;"
+        target_temperature: !lambda "return 23.0f;"
+
+    - climate.template.publish:
+        id: template_climate_custom_modes
+        custom_fan_mode: "turbo"
+        custom_preset: "eco_plus"
+
     # Test C++ API: set_template() with stateless lambda (no captures)
     # NOTE: set_template() is not intended to be a public API, but we test it to ensure it doesn't break.
     - lambda: |-
@@ -513,6 +534,98 @@ alarm_control_panel:
     codes:
       - "1234"
 
+climate:
+  - platform: template
+    id: template_climate
+    name: "Template Climate"
+    optimistic: true
+    sensor: template_template_sens
+    supports_action: true
+    supports_current_humidity: true
+    restore_mode: NO_RESTORE
+    initial_state:
+      mode: HEAT
+      target_temperature: 21.0
+      fan_mode: LOW
+    supported_modes:
+      - "OFF"
+      - HEAT
+      - COOL
+    supported_fan_modes:
+      - AUTO
+      - LOW
+      - HIGH
+    supported_swing_modes:
+      - "OFF"
+      - VERTICAL
+    supported_presets:
+      - NONE
+      - ECO
+    visual:
+      min_temperature: 16.0
+      max_temperature: 30.0
+      temperature_step: 0.5
+    set_mode_action:
+      - logger.log:
+          format: "set_mode_action %d"
+          args: ["(int) x"]
+    set_target_temperature_action:
+      - logger.log:
+          format: "set_target_temperature_action %.1f"
+          args: ["x"]
+    set_target_humidity_action:
+      - logger.log:
+          format: "set_target_humidity_action %.1f"
+          args: ["x"]
+    set_fan_mode_action:
+      - logger.log:
+          format: "set_fan_mode_action %d"
+          args: ["(int) x"]
+    set_swing_mode_action:
+      - logger.log:
+          format: "set_swing_mode_action %d"
+          args: ["(int) x"]
+    set_preset_action:
+      - logger.log:
+          format: "set_preset_action %d"
+          args: ["(int) x"]
+    on_control:
+      - logger.log: "on_control fired"
+    on_state:
+      - logger.log: "on_state fired"
+
+  - platform: template
+    id: template_climate_custom_modes
+    name: "Template Climate Custom Modes"
+    optimistic: true
+    sensor: template_template_sens
+    supported_modes:
+      - "OFF"
+      - HEAT
+    custom_fan_modes:
+      - turbo
+      - silent
+      - eco
+    custom_presets:
+      - eco_plus
+      - power_save
+      - max
+    set_custom_fan_mode_action:
+      - logger.log:
+          format: "set_custom_fan_mode_action %s"
+          args: ["x.c_str()"]
+    set_custom_preset_action:
+      - logger.log:
+          format: "set_custom_preset_action %s"
+          args: ["x.c_str()"]
+    initial_state:
+      custom_fan_mode: eco
+      custom_preset: max
+    visual:
+      min_temperature: 16.0
+      max_temperature: 30.0
+      temperature_step: 0.5
+
 water_heater:
   - platform: template
     id: template_water_heater

tests/components/wifi/common.yaml

@@ -14,6 +14,7 @@ esphome:
           condition: wifi.ap_active
           then:
             - logger.log: "WiFi AP is active!"
+      - wifi.roam
 
 wifi:
   networks:

tests/integration/conftest.py

@@ -162,6 +162,13 @@ def integration_test_dir() -> Generator[Path]:
         yield Path(tmpdir)
 
 
+@pytest.fixture
+def isolated_preferences(monkeypatch: pytest.MonkeyPatch, tmp_path: Path) -> None:
+    """Host preferences persist per device name; give the test its own so a
+    provisioned key never leaks into another run."""
+    monkeypatch.setenv("ESPHOME_PREFDIR", str(tmp_path / "prefs"))
+
+
 @pytest.fixture
 def reserved_tcp_port() -> Generator[tuple[int, socket.socket]]:
     """Reserve an unused TCP port by holding the socket open."""

tests/integration/const.py

@@ -9,6 +9,13 @@ API_CONNECTION_TIMEOUT = 30.0  # seconds
 PORT_WAIT_TIMEOUT = 30.0  # seconds
 PORT_POLL_INTERVAL = 0.1  # seconds
 
+# The well-known all-zeros provisioning PSK, a key to provision over it, and
+# the time the device takes to activate a newly saved key (100 ms timer plus
+# margin)
+ZERO_PSK = "AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA="
+PROVISIONING_PSK = b"bm5ubm5ubm5ubm5ubm5ubm5ubm5ubm5ubm5ubm5ubm4="
+KEY_ACTIVATION_DELAY = 0.5  # seconds
+
 # Process shutdown timeouts
 SIGINT_TIMEOUT = 5.0  # seconds
 SIGTERM_TIMEOUT = 2.0  # seconds

tests/integration/fixtures/api_encode_boundaries.yaml

@@ -0,0 +1,58 @@
+esphome:
+  name: api-encode-boundaries-test
+  # Top-level area fills DeviceInfoResponse.suggested_area (field 16, a two-byte tag)
+  area:
+    id: kitchen_area
+    name: Kitchen
+  on_boot:
+    - sensor.template.publish:
+        id: zero_then_value
+        state: 0.0
+
+host:
+api:
+logger:
+  level: DEBUG
+
+sensor:
+  - platform: template
+    name: "Zero Then Value"
+    id: zero_then_value
+    # Negative int32 takes the ten byte varint path
+    accuracy_decimals: -2
+    update_interval: never
+
+text_sensor:
+  - platform: template
+    name: "Long Text"
+    id: long_text
+    update_interval: never
+
+number:
+  - platform: template
+    name: "Negative Number"
+    optimistic: true
+    min_value: -1000
+    max_value: 1000
+    step: 0.5
+    initial_value: -123.5
+
+select:
+  - platform: template
+    name: "Long Option Select"
+    optimistic: true
+    options:
+      - short
+      - "option-with-a-name-long-enough-that-its-length-prefix-needs-two-varint-bytes-when-the-list-entities-response-is-encoded-xxxxxxxxxx"
+    initial_option: short
+
+button:
+  - platform: template
+    name: "Publish Values"
+    on_press:
+      - sensor.template.publish:
+          id: zero_then_value
+          state: 12.5
+      - text_sensor.template.publish:
+          id: long_text
+          state: !lambda return std::string(200, 'y');

tests/integration/fixtures/host_ota_api_key_offer_with_password.yaml

@@ -0,0 +1,12 @@
+esphome:
+  name: host-ota-test
+host:
+api:
+  encryption:
+    key: "AAECAwQFBgcICQoLDA0ODxAREhMUFRYXGBkaGxwdHh8="
+ota:
+  - platform: esphome
+    port: __OTA_PORT__
+    password: "hunter2"
+logger:
+  level: DEBUG

tests/integration/fixtures/host_ota_provisioned_api_key.yaml

@@ -0,0 +1,10 @@
+esphome:
+  name: host-ota-test
+host:
+api:
+  encryption:
+ota:
+  - platform: esphome
+    port: __OTA_PORT__
+logger:
+  level: DEBUG

tests/integration/fixtures/template_climate_basic.yaml

@@ -0,0 +1,72 @@
+esphome:
+  name: tmpl-clim-basic
+  on_boot:
+    - climate.template.publish:
+        id: test_climate
+        action: IDLE
+host:
+api:
+logger:
+
+climate:
+  - platform: template
+    id: test_climate
+    name: Test Basic Climate
+    optimistic: true
+    sensor: test_climate_current_temperature
+    humidity_sensor: test_climate_current_humidity
+    supports_action: true
+    supported_modes:
+      - "OFF"
+      - HEAT
+      - COOL
+    supported_fan_modes:
+      - AUTO
+      - LOW
+      - HIGH
+    supported_swing_modes:
+      - "OFF"
+      - VERTICAL
+    supported_presets:
+      - NONE
+      - ECO
+    visual:
+      min_temperature: 16.0
+      max_temperature: 30.0
+      temperature_step: 0.5
+    on_control:
+      - lambda: |-
+          if (x.get_mode().has_value())
+            ESP_LOGD("test", "on_control mode=%d", (int) *x.get_mode());
+          if (x.get_target_temperature().has_value())
+            ESP_LOGD("test", "on_control target_temperature=%.1f", *x.get_target_temperature());
+          if (x.get_fan_mode().has_value())
+            ESP_LOGD("test", "on_control fan_mode=%d", (int) *x.get_fan_mode());
+          if (x.get_swing_mode().has_value())
+            ESP_LOGD("test", "on_control swing_mode=%d", (int) *x.get_swing_mode());
+          if (x.get_preset().has_value())
+            ESP_LOGD("test", "on_control preset=%d", (int) *x.get_preset());
+
+sensor:
+  - platform: template
+    id: test_climate_current_temperature
+    name: Test Climate Current Temperature
+    lambda: "return 22.5f;"
+    update_interval: 10ms
+  - platform: template
+    id: test_climate_current_humidity
+    name: Test Climate Current Humidity
+    lambda: "return 55.0f;"
+    update_interval: 10ms
+
+button:
+  - platform: template
+    id: simulate_device_report
+    name: Simulate Device Report
+    on_press:
+      - climate.template.publish:
+          id: test_climate
+          mode: "OFF"
+          fan_mode: AUTO
+          swing_mode: "OFF"
+          preset: NONE

tests/integration/fixtures/template_climate_custom_modes.yaml

@@ -0,0 +1,47 @@
+esphome:
+  name: tmpl-clim-custom
+host:
+api:
+logger:
+
+climate:
+  - platform: template
+    id: test_climate
+    name: Test Custom Mode Climate
+    optimistic: true
+    sensor: test_climate_current_temperature
+    supported_modes:
+      - "OFF"
+      - HEAT
+      - COOL
+    custom_fan_modes:
+      - turbo
+      - silent
+      - eco
+    custom_presets:
+      - eco_plus
+      - power_save
+      - max
+    on_control:
+      - lambda: |-
+          if (x.has_custom_fan_mode())
+            ESP_LOGD("test", "on_control custom_fan_mode=%s", x.get_custom_fan_mode().c_str());
+          if (x.has_custom_preset())
+            ESP_LOGD("test", "on_control custom_preset=%s", x.get_custom_preset().c_str());
+
+sensor:
+  - platform: template
+    id: test_climate_current_temperature
+    name: Test Climate Current Temperature
+    lambda: "return 22.5f;"
+    update_interval: 10ms
+
+button:
+  - platform: template
+    id: simulate_device_report
+    name: Simulate Device Report
+    on_press:
+      - climate.template.publish:
+          id: test_climate
+          custom_fan_mode: "eco"
+          custom_preset: "max"

tests/integration/fixtures/template_climate_nonoptimistic.yaml

@@ -0,0 +1,56 @@
+esphome:
+  name: tmpl-clim-nonopt
+host:
+api:
+logger:
+
+climate:
+  - platform: template
+    id: test_climate
+    name: Test Template Climate Nonoptimistic
+    optimistic: false
+    supported_modes:
+      - "OFF"
+      - HEAT
+      - COOL
+      - FAN_ONLY
+    supported_fan_modes:
+      - AUTO
+      - LOW
+      - HIGH
+    supported_swing_modes:
+      - "OFF"
+      - VERTICAL
+    supported_presets:
+      - NONE
+      - ECO
+      - AWAY
+    visual:
+      min_temperature: 16.0
+      max_temperature: 30.0
+      temperature_step: 0.5
+    on_control:
+      - lambda: |-
+          if (x.get_mode().has_value())
+            ESP_LOGD("test", "on_control mode=%d", (int) *x.get_mode());
+          if (x.get_target_temperature().has_value())
+            ESP_LOGD("test", "on_control target_temperature=%.1f", *x.get_target_temperature());
+          if (x.get_fan_mode().has_value())
+            ESP_LOGD("test", "on_control fan_mode=%d", (int) *x.get_fan_mode());
+          if (x.get_swing_mode().has_value())
+            ESP_LOGD("test", "on_control swing_mode=%d", (int) *x.get_swing_mode());
+          if (x.get_preset().has_value())
+            ESP_LOGD("test", "on_control preset=%d", (int) *x.get_preset());
+
+button:
+  - platform: template
+    id: simulate_device_confirmation
+    name: Simulate Device Confirmation
+    on_press:
+      - climate.template.publish:
+          id: test_climate
+          mode: HEAT
+          target_temperature: 22.5
+          fan_mode: HIGH
+          swing_mode: VERTICAL
+          preset: AWAY

tests/integration/fixtures/template_climate_on_control_ordering.yaml

@@ -0,0 +1,26 @@
+esphome:
+  name: tmpl-clim-oc-order
+host:
+api:
+logger:
+
+# on_control fires with the full ClimateCall (arg `x`) from the base Climate component's
+# ClimateCall::perform(), before validate_()/control() run -- so when the lambda action below
+# runs, the entity's own .mode is still the OLD value, even though x.get_mode() already reports
+# the NEW requested value. on_state fires afterward, once control() has applied it.
+climate:
+  - platform: template
+    id: test_climate
+    name: Test On Control Ordering
+    optimistic: true
+    supported_modes:
+      - "OFF"
+      - HEAT
+    on_control:
+      - lambda: |-
+          ESP_LOGD("test", "on_control requested_mode=%d current_mode_before_apply=%d",
+                   x.get_mode().has_value() ? (int) *x.get_mode() : -1,
+                   (int) id(test_climate).mode);
+    on_state:
+      - lambda: |-
+          ESP_LOGD("test", "on_state mode=%d", (int) x.mode);

tests/integration/fixtures/template_climate_publish_all_fields.yaml

@@ -0,0 +1,63 @@
+esphome:
+  name: tmpl-clim-publish-all
+host:
+api:
+logger:
+
+climate:
+  - platform: template
+    id: test_climate
+    name: Test Publish All Fields
+    optimistic: true
+    # current_temperature/current_humidity/action are only sent over the API at all if their
+    # trait is advertised: current_temperature/current_humidity because a sensor/humidity_sensor
+    # is referenced below, action because supports_action is set. The sensors' fixed readings
+    # match what climate.template.publish pushes, so the sensor callback (guarded to only publish
+    # on an actual change) doesn't produce an extra, unexpected state update of its own.
+    sensor: test_climate_current_temperature
+    humidity_sensor: test_climate_current_humidity
+    supports_action: true
+    supported_modes:
+      - "OFF"
+      - HEAT
+    supported_fan_modes:
+      - AUTO
+      - HIGH
+    supported_swing_modes:
+      - "OFF"
+      - VERTICAL
+    supported_presets:
+      - NONE
+      - ECO
+    on_control:
+      # Should never fire in this test: climate.template.publish is a pure bypass and must not
+      # re-trigger on_control as if the entity were freshly commanded.
+      - logger.log: "on_control fired"
+
+sensor:
+  - platform: template
+    id: test_climate_current_temperature
+    name: Test Climate Current Temperature
+    lambda: "return 20.0f;"
+    update_interval: 10ms
+  - platform: template
+    id: test_climate_current_humidity
+    name: Test Climate Current Humidity
+    lambda: "return 60.0f;"
+    update_interval: 10ms
+
+button:
+  - platform: template
+    id: publish_all
+    name: Publish All
+    on_press:
+      - climate.template.publish:
+          id: test_climate
+          current_temperature: 20.0
+          current_humidity: 60.0
+          target_temperature: 23.0
+          mode: HEAT
+          action: HEATING
+          fan_mode: HIGH
+          swing_mode: VERTICAL
+          preset: ECO

tests/integration/fixtures/template_climate_sensor_push.yaml

@@ -0,0 +1,49 @@
+esphome:
+  name: tmpl-clim-sensor-push
+host:
+api:
+logger:
+
+# No lambda/update_interval: these sensors only ever report a value when a button below
+# publishes one (standing in for e.g. a BLE scan callback in a real config).
+sensor:
+  - platform: template
+    id: room_temperature
+    name: Room Temperature
+  - platform: template
+    id: room_humidity
+    name: Room Humidity
+
+climate:
+  - platform: template
+    id: test_climate
+    name: Test Sensor Push Climate
+    optimistic: true
+    sensor: room_temperature
+    humidity_sensor: room_humidity
+    supported_modes:
+      - "OFF"
+      - HEAT
+
+button:
+  - platform: template
+    id: publish_temperature
+    name: Publish Temperature
+    on_press:
+      - sensor.template.publish:
+          id: room_temperature
+          state: 24.0
+  - platform: template
+    id: publish_temperature_same
+    name: Publish Temperature Same Value
+    on_press:
+      - sensor.template.publish:
+          id: room_temperature
+          state: 24.0
+  - platform: template
+    id: publish_humidity
+    name: Publish Humidity
+    on_press:
+      - sensor.template.publish:
+          id: room_humidity
+          state: 65.0

tests/integration/fixtures/template_climate_set_actions.yaml

@@ -0,0 +1,89 @@
+esphome:
+  name: tmpl-clim-set-act
+host:
+api:
+logger:
+
+# Every settable field forwards its requested value to a set_*_action. supports_two_point and
+# supports_target_humidity are not declared here: they are derived from the low/high and humidity
+# set actions being present.
+climate:
+  - platform: template
+    id: test_climate
+    name: Test Set Actions
+    optimistic: false
+    restore_mode: NO_RESTORE
+    supported_modes:
+      - "OFF"
+      - HEAT
+      - COOL
+    supported_fan_modes:
+      - AUTO
+      - LOW
+    supported_swing_modes:
+      - "OFF"
+      - VERTICAL
+    supported_presets:
+      - NONE
+      - ECO
+    custom_fan_modes:
+      - turbo
+    custom_presets:
+      - eco_plus
+    visual:
+      min_temperature: 16.0
+      max_temperature: 30.0
+      temperature_step: 0.5
+    set_mode_action:
+      - logger.log:
+          format: "set_mode_action %d"
+          args: ["(int) x"]
+    set_target_temperature_low_action:
+      - logger.log:
+          format: "set_target_temperature_low_action %.1f"
+          args: ["x"]
+    set_target_temperature_high_action:
+      - logger.log:
+          format: "set_target_temperature_high_action %.1f"
+          args: ["x"]
+    set_target_humidity_action:
+      - logger.log:
+          format: "set_target_humidity_action %.0f"
+          args: ["x"]
+    set_fan_mode_action:
+      - logger.log:
+          format: "set_fan_mode_action %d"
+          args: ["(int) x"]
+    set_custom_fan_mode_action:
+      - logger.log:
+          format: "set_custom_fan_mode_action %s"
+          args: ["x.c_str()"]
+    set_swing_mode_action:
+      - logger.log:
+          format: "set_swing_mode_action %d"
+          args: ["(int) x"]
+    set_preset_action:
+      - logger.log:
+          format: "set_preset_action %d"
+          args: ["(int) x"]
+    set_custom_preset_action:
+      - logger.log:
+          format: "set_custom_preset_action %s"
+          args: ["x.c_str()"]
+
+button:
+  - platform: template
+    id: report_device_state
+    name: Report Device State
+    on_press:
+      - climate.template.publish:
+          id: test_climate
+          mode: HEAT
+
+  - platform: template
+    id: report_unsupported_mode
+    name: Report Unsupported Mode
+    on_press:
+      - climate.template.publish:
+          id: test_climate
+          mode: DRY

tests/integration/fixtures/template_climate_two_point_temperature.yaml

@@ -0,0 +1,52 @@
+esphome:
+  name: tmpl-clim-two-point
+host:
+api:
+logger:
+
+climate:
+  - platform: template
+    id: test_climate
+    name: Test Two-Point Heatpump
+    optimistic: true
+    sensor: test_climate_current_temperature
+    supports_two_point_target_temperature: true
+    supports_target_humidity: true
+    supported_modes:
+      - "OFF"
+      - HEAT_COOL
+      - HEAT
+      - COOL
+    visual:
+      min_temperature: 16.0
+      max_temperature: 30.0
+      temperature_step: 0.5
+    on_control:
+      - lambda: |-
+          if (x.get_mode().has_value())
+            ESP_LOGD("test", "on_control mode=%d", (int) *x.get_mode());
+          if (x.get_target_temperature_low().has_value())
+            ESP_LOGD("test", "on_control target_temperature_low=%.1f", *x.get_target_temperature_low());
+          if (x.get_target_temperature_high().has_value())
+            ESP_LOGD("test", "on_control target_temperature_high=%.1f", *x.get_target_temperature_high());
+          if (x.get_target_humidity().has_value())
+            ESP_LOGD("test", "on_control target_humidity=%.1f", *x.get_target_humidity());
+
+sensor:
+  - platform: template
+    id: test_climate_current_temperature
+    name: Test Climate Current Temperature
+    lambda: "return 21.0f;"
+    update_interval: 10ms
+
+button:
+  - platform: template
+    id: simulate_device_report
+    name: Simulate Device Report
+    on_press:
+      - climate.template.publish:
+          id: test_climate
+          mode: HEAT_COOL
+          target_temperature_low: 18.0
+          target_temperature_high: 24.0
+          target_humidity: 50.0

tests/integration/state_utils.py

@@ -57,6 +57,43 @@ async def wait_for_state(
     return await asyncio.wait_for(future, timeout=timeout)
 
 
+class StateWaiter:
+    """Route one state subscription to any number of predicate waits."""
+
+    def __init__(self) -> None:
+        self._waiters: list[
+            tuple[Callable[[EntityState], bool], asyncio.Future[EntityState]]
+        ] = []
+
+    def on_state(self, state: EntityState) -> None:
+        for predicate, future in self._waiters:
+            if future.done():
+                continue
+            try:
+                matched = predicate(state)
+            except Exception as exc:  # noqa: BLE001  the wait re-raises it, the callback must not die
+                future.set_exception(exc)
+                continue
+            if matched:
+                future.set_result(state)
+
+    async def expect(
+        self, predicate: Callable[[EntityState], bool], timeout: float = 5.0
+    ) -> EntityState:
+        """Wait for the next state matching ``predicate``; states seen before this call do not count."""
+        entry = (predicate, asyncio.get_running_loop().create_future())
+        self._waiters.append(entry)
+        try:
+            async with asyncio.timeout(timeout):
+                return await entry[1]
+        except TimeoutError:
+            raise TimeoutError(
+                f"no state matched {predicate} within {timeout}s"
+            ) from None
+        finally:
+            self._waiters.remove(entry)
+
+
 def find_entity[T: EntityInfo](
     entities: list[EntityInfo],
     object_id_substring: str,

tests/integration/test_api_encode_boundaries.py

@@ -0,0 +1,76 @@
+"""Encode paths at their branch boundaries: zero skipped float, fixed32 state, negative int32,
+length prefixes of two varint bytes and two byte field tags."""
+
+from __future__ import annotations
+
+import asyncio
+
+from aioesphomeapi import (
+    NumberState,
+    SelectInfo,
+    SensorInfo,
+    SensorState,
+    TextSensorState,
+)
+import pytest
+
+from .state_utils import InitialStateHelper, StateWaiter, require_entity
+from .types import APIClientConnectedFactory, RunCompiledFunction
+
+LONG_OPTION = (
+    "option-with-a-name-long-enough-that-its-length-prefix-needs-two-varint-bytes-"
+    "when-the-list-entities-response-is-encoded-xxxxxxxxxx"
+)
+
+
+@pytest.mark.asyncio
+async def test_api_encode_boundaries(
+    yaml_config: str,
+    run_compiled: RunCompiledFunction,
+    api_client_connected: APIClientConnectedFactory,
+) -> None:
+    async with run_compiled(yaml_config), api_client_connected() as client:
+        device_info, (entities, _) = await asyncio.gather(
+            client.device_info(), client.list_entities_services()
+        )
+        assert device_info.suggested_area == "Kitchen"
+
+        sensor = require_entity(entities, "zero_then_value", SensorInfo)
+        assert sensor.accuracy_decimals == -2
+        select = require_entity(entities, "long_option_select", SelectInfo)
+        assert len(LONG_OPTION) >= 128
+        assert select.options == ["short", LONG_OPTION]
+        text = require_entity(entities, "long_text")
+        number = require_entity(entities, "negative_number")
+        button = require_entity(entities, "publish_values")
+
+        initial = InitialStateHelper(entities)
+        waiter = StateWaiter()
+        client.subscribe_states(initial.on_state_wrapper(waiter.on_state))
+        await initial.wait_for_initial_states()
+
+        # A float of exactly zero is skipped on the wire and must still read as 0.0, not missing
+        first = initial.initial_states[sensor.key]
+        assert isinstance(first, SensorState)
+        assert first.state == 0.0 and not first.missing_state
+        first_number = initial.initial_states[number.key]
+        assert isinstance(first_number, NumberState)
+        assert first_number.state == -123.5
+
+        client.button_command(button.key)
+        await asyncio.gather(
+            waiter.expect(
+                lambda s: (
+                    isinstance(s, SensorState)
+                    and s.key == sensor.key
+                    and s.state == 12.5
+                )
+            ),
+            waiter.expect(
+                lambda s: (
+                    isinstance(s, TextSensorState)
+                    and s.key == text.key
+                    and s.state == "y" * 200
+                )
+            ),
+        )

tests/integration/test_api_zero_psk_provisioning.py

@@ -10,34 +10,40 @@ from __future__ import annotations
 
 import asyncio
 import base64
+import socket
 
 from aioesphomeapi import InvalidEncryptionKeyAPIError, RequiresEncryptionAPIError
 import pytest
 
-from .types import APIClientConnectedFactory, RunCompiledFunction
+from .conftest import run_binary_and_wait_for_port
+from .const import KEY_ACTIVATION_DELAY, LOCALHOST, PROVISIONING_PSK, ZERO_PSK
+from .types import (
+    APIClientConnectedFactory,
+    CompileFunction,
+    ConfigWriter,
+    RunCompiledFunction,
+)
 
-# The well-known provisioning PSK: base64 of 32 zero bytes
-ZERO_PSK = base64.b64encode(bytes(32)).decode()
-# A real key to provision
-NEW_KEY = base64.b64encode(b"n" * 32)
-# Time for the device to activate a newly saved key (100ms timer plus margin)
-KEY_ACTIVATION_DELAY = 0.5
-
-
-@pytest.fixture(autouse=True)
-def isolated_preferences(monkeypatch: pytest.MonkeyPatch, tmp_path) -> None:
-    """Keep host preferences per-test so every run starts unprovisioned."""
-    monkeypatch.setenv("ESPHOME_PREFDIR", str(tmp_path / "prefs"))
+pytestmark = pytest.mark.usefixtures("isolated_preferences")
+NEW_KEY = PROVISIONING_PSK
 
 
 @pytest.mark.asyncio
 async def test_api_zero_psk_provisioning(
     yaml_config: str,
-    run_compiled: RunCompiledFunction,
+    write_yaml_config: ConfigWriter,
+    compile_esphome: CompileFunction,
+    reserved_tcp_port: tuple[int, socket.socket],
     api_client_connected: APIClientConnectedFactory,
 ) -> None:
-    """Exercise the reject paths, then provision a key over the zero-PSK channel."""
-    async with run_compiled(yaml_config):
+    """Exercise the reject paths, provision a key over the zero-PSK channel,
+    and check the key comes back from preferences on the next boot."""
+    port, port_socket = reserved_tcp_port
+    config_path = await write_yaml_config(yaml_config)
+    binary_path = await compile_esphome(config_path)
+    port_socket.close()
+
+    async with run_binary_and_wait_for_port(binary_path, LOCALHOST, port):
         # --- Pre-provisioning reject paths (device state is unchanged) ---
 
         # A wrong (non-zero) PSK fails against the zero provisioning PSK
@@ -97,6 +103,19 @@ async def test_api_zero_psk_provisioning(
             async with api_client_connected(timeout=5) as client:
                 await client.device_info()
 
+    # The key is loaded from preferences on the next boot
+    lines: list[str] = []
+    async with run_binary_and_wait_for_port(
+        binary_path, LOCALHOST, port, line_callback=lines.append
+    ):
+        async with api_client_connected(noise_psk=NEW_KEY.decode()) as client:
+            device_info = await client.device_info()
+            assert device_info.api_encryption_provisionable is False
+        with pytest.raises(InvalidEncryptionKeyAPIError):
+            async with api_client_connected(noise_psk=ZERO_PSK, timeout=5) as client:
+                await client.device_info()
+    assert any("Loaded saved Noise PSK" in line for line in lines)
+
 
 @pytest.mark.asyncio
 async def test_api_zero_psk_provisioning_plaintext(