Compare commits

..
Author SHA1 Message Date
J. Nick Koston fd9760d8e3 [socket] Apply SO_REUSEADDR to the pcb and unmap IPv4-mapped IPv6 destinations 2026-08-28 11:23:43 -05:00
J. Nick Koston a4d2a4ad4b [socket] Zero lwip address struct and narrow dual-stack bind promotion 2026-08-28 10:31:20 -05:00
J. Nick Koston 7cbc81ad66 [socket] Simplify UDP implementation and deduplicate lwip helpers 2026-08-28 10:26:05 -05:00
J. Nick Koston c71a599f5a [socket] Address review feedback on UDP implementation 2026-08-28 10:08:58 -05:00
J. Nick Koston e6f9d9c369 Merge remote-tracking branch 'origin/dev' into socket-lwip-raw-udp
# Conflicts:
#	esphome/components/socket/__init__.py
2026-08-28 09:33:29 -05:00
J. Nick Koston 6d2e1f8658 Merge branch 'dev' into socket-lwip-raw-udp 2026-05-13 01:36:28 -05:00
J. Nick Koston e855ddb1f1 Merge branch 'dev' into socket-lwip-raw-udp 2026-04-22 08:22:37 +02:00
J. Nick Koston e191f5fb4b Merge branch 'dev' into socket-lwip-raw-udp 2026-04-21 15:08:29 +02:00
J. Nick Koston e7c126d3dc [socket] Rename UDP socket types so UDPSocket is the full send+recv type
Swap the UDP type naming so the unsuffixed UDPSocket is the full
send+recv socket and the send-only variant becomes UDPSendSocket. The
previous naming inverted reader expectations (UDPSocket sounded complete
but was send-only, UDPRecvSocket sounded limited but was the full one).

Public:
  UDPSocket       (was UDPRecvSocket)
  UDPSendSocket   (was UDPSocket)
  socket_udp                     (was socket_udp_recv)
  socket_ip_udp                  (was socket_ip_udp_recv)
  socket_udp_loop_monitored      (was socket_udp_recv_loop_monitored)
  socket_ip_udp_loop_monitored   (was socket_ip_udp_recv_loop_monitored)
  socket_udp_send                (was socket_udp)
  socket_ip_udp_send             (was socket_ip_udp)

Internal:
  LWIPRawUDPImpl       (was LWIPRawUDPRecvImpl)
  LWIPRawUDPSendImpl   (was LWIPRawUDPImpl)

No consumers exist yet, so this is a clean rename with no migration.
2026-04-10 18:40:14 -10:00
J. Nick Koston af7b3821b8 Merge branch 'dev' into socket-lwip-raw-udp 2026-04-10 18:31:45 -10:00
J. Nick Koston d6c48e2d64 [socket] Validate source address before copying data in recvfrom
Move pbuf_copy_partial after ip2sockaddr_ validation so the caller
buffer is not modified when the address conversion fails.
2026-04-09 09:21:48 -10:00
J. Nick Koston c01699f2a4 [socket] Move socket_loop_monitored declaration before UDP factories
Fix forward declaration ordering: socket_udp_recv_loop_monitored
calls socket_loop_monitored, so the latter must be declared first.
2026-04-09 09:20:36 -10:00
J. Nick Koston abc4069657 [socket] Restore LWIP thread safety documentation to common header 2026-04-09 09:09:38 -10:00
J. Nick Koston c3827423ba [socket] Move thin UDP and listen factory wrappers inline into socket.h
All the _ip_ variants and non-LWIP_TCP delegators are one-liners
that just forward to other factory functions. Move them inline to
reduce socket.cpp and keep the dispatch logic visible in one place.
2026-04-09 09:09:17 -10:00
J. Nick Koston 41a8e7f61b [socket] Split lwip raw impl into tcp, udp, and common files
Extract UDP classes and shared helpers from lwip_raw_tcp_impl into
separate files for better organization:

- lwip_raw_common_impl.{h,cpp}: shared helpers (lwip_ip_to_sockaddr,
  sockaddr_to_lwip, lwip_bind_err)
- lwip_raw_udp_impl.{h,cpp}: LWIPRawUDPImpl, LWIPRawUDPRecvImpl,
  and UDP factory functions
- lwip_raw_tcp_impl.{h,cpp}: TCP-only code (LWIPRawCommon,
  LWIPRawImpl, LWIPRawListenImpl, TCP factories)

Update __init__.py FILTER_SOURCE_FILES to exclude all three lwip
raw files when not using the lwip_tcp implementation.
2026-04-09 09:06:04 -10:00
J. Nick Koston 0a3f8c6d67 [socket] Add unified UDP recv loop-monitored factory functions
Add socket_udp_recv_loop_monitored() and socket_ip_udp_recv_loop_monitored()
so consumers can create UDP recv sockets with loop wake support using a single
API across all platforms, without #ifdef guards.
2026-04-09 08:56:11 -10:00
J. Nick Koston 2375faee88 fixes, update 2026-04-09 08:54:08 -10:00
J. Nick Koston 3da3a66d09 Merge branch 'dev' into socket-lwip-raw-udp 2026-04-09 07:55:07 -10:00
J. Nick Koston cf22559af0 Merge branch 'dev' into socket-lwip-raw-udp 2026-04-02 09:56:54 -10:00
J. Nick Koston 17c2557cca Merge branch 'dev' into socket-lwip-raw-udp 2026-04-01 18:47:13 -10:00
J. Nick Koston 012dadbf77 Merge branch 'dev' into socket-lwip-raw-udp 2026-03-22 21:27:55 -10:00
J. Nick Koston 97e4bb71c3 Merge branch 'dev' into socket-lwip-raw-udp 2026-03-18 19:31:21 -10:00
J. Nick Koston b333bb76e4 Merge branch 'dev' into socket-lwip-raw-udp 2026-03-16 16:36:46 -10:00
J. Nick Koston 273637b6d7 tweak 2026-03-16 16:35:32 -10:00
J. Nick Koston 75efdd8662 tweaks 2026-03-16 11:01:21 -10:00
J. Nick Koston 86e4341a52 tweaks 2026-03-16 11:00:57 -10:00
J. Nick Koston 402398b389 tweaks 2026-03-16 10:55:33 -10:00
J. Nick Koston dde81d3f63 tweaks 2026-03-16 10:51:54 -10:00
J. Nick Koston eef806c806 Merge branch 'dev' into socket-lwip-raw-udp 2026-03-16 10:44:13 -10:00
J. Nick Koston cdbbcfb87d [socket] Extract lwip_bind_err() to deduplicate bind error handling
TCP bind and UDP bind_internal_ had identical ERR_USE/ERR_VAL/ERR_OK
to errno mapping. Extract into a shared helper.
2026-03-14 16:12:39 -10:00
J. Nick Koston 71da3dc2de [socket] Fix RP2040 socket_delay race: don't clear wake flag before sleep
Restore the comment explaining why s_socket_woke must not be cleared
between the early-return check and the __wfe() loop, and restore the
s_socket_woke = false after the loop to consume the wake for the next
call. Both were lost during conflict resolution.
2026-03-14 16:09:24 -10:00
J. Nick Koston 0176305d24 [socket] Restore read_locked_, SO_RCVTIMEO, and wait_for_data_ lost in merge
These features from upstream/dev were dropped when resolving conflicts
with the PR's remote branch: read_locked_/wait_for_data_ (blocking read
with SO_RCVTIMEO timeout support), recv_timeout_cs_ field, SO_RCVTIMEO
and SO_SNDTIMEO setsockopt/getsockopt handling, and the setblocking()
implementation that accepts blocking mode for SO_RCVTIMEO.
2026-03-14 16:07:15 -10:00
J. Nick Koston c81e9fd154 [socket] Deduplicate sockaddr parsing between TCP and UDP bind
Extract sockaddr_to_lwip() from LWIPRawUDPImpl static method to a
shared file-level function. Refactor LWIPRawCommon::bind() to use it
instead of inline address parsing, removing ~35 lines of duplicated
sockaddr-to-ip_addr_t conversion code.
2026-03-14 16:03:24 -10:00
J. Nick Koston 556ef1894f Merge branch 'socket-lwip-raw-udp' of https://github.com/esphome/esphome into socket-lwip-raw-udp
# Conflicts:
#	esphome/components/socket/lwip_raw_tcp_impl.cpp
#	esphome/components/socket/lwip_raw_tcp_impl.h
2026-03-14 16:00:31 -10:00
J. Nick KostonandClaude Opus 4.6 519be06e73 [socket] Add LWIP_LOCK to UDP socket methods for RP2040 safety
On RP2040, lwip callbacks run from a low-priority IRQ context and can
preempt main-loop code. All lwip API calls from the main loop must
hold the async_context lock (LWIP_LOCK) to prevent races on shared
lwip state (PCB lists, pbuf pools, IGMP groups).

The TCP implementation was already correct; the UDP methods were
missing the lock.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-03-14 15:57:57 -10:00
J. Nick Koston 1e935c128a Merge remote-tracking branch 'upstream/dev' into socket-lwip-raw-udp
# Conflicts:
#	esphome/components/socket/lwip_raw_tcp_impl.cpp
2026-03-14 15:53:39 -10:00
J. Nick Koston a0e162912c safety 2026-03-11 00:00:52 -10:00
J. Nick Koston 1131af1690 safety 2026-03-10 23:59:25 -10:00
J. Nick Koston 6fb00baa29 Merge remote-tracking branch 'upstream/dev' into socket-lwip-raw-udp 2026-03-10 23:57:51 -10:00
J. Nick Koston fccfab8083 [socket] Switch UDP rx queue from lock-free SPSC to count-based with LWIP_LOCK
With the lwip lock infrastructure in place from the TCP race fix,
the lock-free SPSC ring buffer's wasted slot is no longer needed.
Switch to a simple rx_count_ approach that uses all 4 queue slots.

Also add LWIP_LOCK() to all UDP methods that call lwip APIs
(bind, close, sendto, setsockopt, getsockopt, recvfrom, factories).
2026-03-10 01:15:57 -10:00
J. Nick KostonandJ. Nick Koston f54756ae2d [socket] Address review comments: doc comments and (void) flags
- Document port_host byte order convention in lwip_ip_to_sockaddr
- Note intentional method hiding in LWIPRawUDPRecvImpl
- Note recvfrom truncation differs from POSIX MSG_TRUNC
- Add (void) flags in sendto to clarify flags are ignored

Co-Authored-By: J. Nick Koston <nick@koston.org>
2026-03-10 01:12:15 -10:00
J. Nick Koston 49ba08cec9 [socket] Add lwip raw UDP socket implementation
Add native UDP support to the lwip raw TCP socket layer used by
ESP8266 and RP2040, eliminating the need for Arduino WiFiUDP fallback.

Two new classes:
- LWIPRawUDPImpl: send-only UDP (8 bytes overhead)
- LWIPRawUDPRecvImpl: send+recv with fixed-size ring buffer (no heap
  allocation in recv callback)

Factory functions: socket_udp(), socket_udp_recv(), socket_ip_udp(),
socket_ip_udp_recv() with UDPSocket/UDPRecvSocket type aliases.

Additive only — no consumer migration in this PR.
2026-03-10 01:12:15 -10:00
J. Nick Koston 81d12fd14a [socket] Hold lwip lock for entire write() operation
Same pattern as writev — write() calls internal_write_() then
internal_output_(), each acquiring the lock separately. Hold
the lock at the outer scope so inner calls just bump the
recursion counter.
2026-03-10 00:57:36 -10:00
J. Nick Koston cc05bf3ed2 [socket] Add LWIP_LOCK to socket factory functions
tcp_new() is an lwip core API call that must be bracketed with
the lwip lock on RP2040 per pico-sdk docs. Add LWIP_LOCK() to
socket() and socket_listen() factory functions.
2026-03-10 00:55:34 -10:00
J. Nick Koston c182c0c74f [socket] Hold lwip lock for entire readv/writev scatter-gather operation
Avoid repeated lock acquire/release cycles per iovec element.
The recursive mutex re-entry in inner calls is nearly free (counter
bump), while the outer lock prevents the expensive IRQ disable/enable
on each iteration.
2026-03-10 00:53:01 -10:00
J. Nick Koston a88e9b8146 [socket] Fix RP2040 TCP race condition between lwip callbacks and main loop
On RP2040 (Pico W), arduino-pico sets PICO_CYW43_ARCH_THREADSAFE_BACKGROUND=1,
which means lwip callbacks (recv_fn, accept_fn, err_fn) run from a PendSV
interrupt — not the main loop. This allows them to preempt read(), write(),
close(), and accept() at any point, causing race conditions on shared state
like the rx_buf_ pbuf chain.

The most critical race: recv_fn calls pbuf_cat(rx_buf_, pb) while read() is
freeing nodes in the same chain, leading to use-after-free and lwip's
"Creating an infinite loop" assertion panic. This is the root cause of #10681.

Fix: implement RP2040's LwIPLock (previously a no-op) to call
cyw43_arch_lwip_begin/end, which acquires the pico-sdk async_context recursive
mutex. Add LWIP_LOCK() guards to all main-loop lwip API call sites in the
socket layer.

On ESP8266, lwip callbacks run cooperatively from the main loop, so
LwIPLock remains a no-op.

Closes #10681
2026-03-10 00:34:20 -10:00
J. Nick KostonandJ. Nick Koston 7dea3756e9 [socket] Address review comments: doc comments and (void) flags
- Document port_host byte order convention in lwip_ip_to_sockaddr
- Note intentional method hiding in LWIPRawUDPRecvImpl
- Note recvfrom truncation differs from POSIX MSG_TRUNC
- Add (void) flags in sendto to clarify flags are ignored

Co-Authored-By: J. Nick Koston <nick@koston.org>
2026-03-09 23:56:37 -10:00
J. Nick Koston fa0bff3374 [socket] Add lwip raw UDP socket implementation
Add native UDP support to the lwip raw TCP socket layer used by
ESP8266 and RP2040, eliminating the need for Arduino WiFiUDP fallback.

Two new classes:
- LWIPRawUDPImpl: send-only UDP (8 bytes overhead)
- LWIPRawUDPRecvImpl: send+recv with fixed-size ring buffer (no heap
  allocation in recv callback)

Factory functions: socket_udp(), socket_udp_recv(), socket_ip_udp(),
socket_ip_udp_recv() with UDPSocket/UDPRecvSocket type aliases.

Additive only — no consumer migration in this PR.
2026-03-09 23:48:07 -10:00
17 changed files with 764 additions and 513 deletions
+5 -8
View File
@@ -90,10 +90,9 @@ def get_project_cmakelists(
"""
idf_target = variant_to_idf_target(get_esp32_variant())
# esp_idf_size 2.x (IDF >=6.0) made NG the default and removed --ng;
# 1.x (IDF 5.5) needs --ng for --format=json2. 1.x json2 also lacks
# total_size, hence the ELF fallback in espidf/size_summary.py; both
# go away together when 1.x support is dropped.
# esp_idf_size 2.x (bundled with IDF >=6.0) made NG the default and
# removed the --ng flag; on 1.x (IDF 5.5) --ng is required to get
# --format=raw because the legacy mode doesn't support it.
size_ng_flag = "--ng" if idf_version() < cv.Version(6, 0, 0) else ""
# Project-wide compile options: -D defines and -W warning flags (skip
@@ -212,12 +211,10 @@ include($ENV{{IDF_PATH}}/tools/cmake/project.cmake)
project({CORE.name})
# Emit per-memory-type JSON size data for ESPHome to read post-build.
# json2 stays small; raw dumps every symbol (~2s on a large map) and
# this command runs inside the link edge, blocking everything downstream.
# Emit raw JSON size data for ESPHome to read post-build.
add_custom_command(
TARGET ${{CMAKE_PROJECT_NAME}}.elf POST_BUILD
COMMAND ${{PYTHON}} -m esp_idf_size {size_ng_flag} --format=json2
COMMAND ${{PYTHON}} -m esp_idf_size {size_ng_flag} --format=raw
-o ${{CMAKE_BINARY_DIR}}/esp_idf_size.json
${{CMAKE_PROJECT_NAME}}.map
WORKING_DIRECTORY ${{CMAKE_BINARY_DIR}}
+2
View File
@@ -187,7 +187,9 @@ async def to_code(config: ConfigType) -> None:
# for the selected implementation.
FILTER_SOURCE_FILES = filter_source_files_from_defines(
{
"lwip_raw_common_impl.cpp": "USE_SOCKET_IMPL_LWIP_TCP",
"lwip_raw_tcp_impl.cpp": "USE_SOCKET_IMPL_LWIP_TCP",
"lwip_raw_udp_impl.cpp": "USE_SOCKET_IMPL_LWIP_TCP",
"bsd_sockets_impl.cpp": "USE_SOCKET_IMPL_BSD_SOCKETS",
"lwip_sockets_impl.cpp": "USE_SOCKET_IMPL_LWIP_SOCKETS",
}
@@ -144,6 +144,9 @@ class BSDSocketImpl {
int get_fd() const { return this->fd_; }
/// UDP rx drop counter parity with LWIPRawUDPImpl; drops are not counted here.
uint16_t get_rx_dropped() const { return 0; }
protected:
// fd_ < 0 means "not open" — used both pre-open (initial state) and post-close. This
// replaces a separate closed_ flag: close() sets fd_ = -1 after ::close(), and the
+10
View File
@@ -20,6 +20,16 @@
#define IPPROTO_IP 0
#define IPPROTO_TCP 6
#define IPPROTO_UDP 17
#define IP_ADD_MEMBERSHIP 3
#define IP_DROP_MEMBERSHIP 4
// NOLINTNEXTLINE(readability-identifier-naming)
struct ip_mreq {
struct in_addr imr_multiaddr;
struct in_addr imr_interface;
};
#if LWIP_IPV6
#define AF_INET6 10
@@ -0,0 +1,109 @@
#include "lwip_raw_common_impl.h"
#include "esphome/core/defines.h"
#ifdef USE_SOCKET_IMPL_LWIP_TCP
#include <cerrno>
#include <cstring>
namespace esphome::socket {
int lwip_ip_to_sockaddr(sa_family_t family, const ip_addr_t *ip, uint16_t port_host, struct sockaddr *name,
socklen_t *addrlen) {
if (family == AF_INET) {
if (*addrlen < sizeof(struct sockaddr_in)) {
errno = EINVAL;
return -1;
}
auto *addr = reinterpret_cast<struct sockaddr_in *>(name);
addr->sin_family = AF_INET;
*addrlen = addr->sin_len = sizeof(struct sockaddr_in);
addr->sin_port = htons(port_host);
inet_addr_from_ip4addr(&addr->sin_addr, ip_2_ip4(ip));
return 0;
}
#if LWIP_IPV6
if (family == AF_INET6) {
if (*addrlen < sizeof(struct sockaddr_in6)) {
errno = EINVAL;
return -1;
}
auto *addr = reinterpret_cast<struct sockaddr_in6 *>(name);
addr->sin6_family = AF_INET6;
*addrlen = addr->sin6_len = sizeof(struct sockaddr_in6);
addr->sin6_port = htons(port_host);
// AF_INET6 sockets may receive IPv4 packets; convert to IPv4-mapped IPv6.
if (IP_IS_V4(ip)) {
ip_addr_t mapped;
ip4_2_ipv4_mapped_ipv6(ip_2_ip6(&mapped), ip_2_ip4(ip));
inet6_addr_from_ip6addr(&addr->sin6_addr, ip_2_ip6(&mapped));
} else {
inet6_addr_from_ip6addr(&addr->sin6_addr, ip_2_ip6(ip));
}
return 0;
}
#endif
errno = EAFNOSUPPORT;
return -1;
}
bool sockaddr_to_lwip(const struct sockaddr *addr, socklen_t addrlen, ip_addr_t *ip, uint16_t *port) {
// headers.h defines sockaddr and sockaddr_in with the same size, so this covers AF_INET
if (addrlen < sizeof(struct sockaddr))
return false;
// Zero the whole struct — the IPv6 zone byte would otherwise be stack garbage
memset(ip, 0, sizeof(*ip));
if (addr->sa_family == AF_INET) {
auto *addr4 = reinterpret_cast<const sockaddr_in *>(addr);
*port = ntohs(addr4->sin_port);
IP_SET_TYPE_VAL(*ip, IPADDR_TYPE_V4);
ip_2_ip4(ip)->addr = addr4->sin_addr.s_addr;
return true;
}
#if LWIP_IPV6
if (addr->sa_family == AF_INET6) {
if (addrlen < sizeof(sockaddr_in6))
return false;
auto *addr6 = reinterpret_cast<const sockaddr_in6 *>(addr);
*port = ntohs(addr6->sin6_port);
IP_SET_TYPE_VAL(*ip, IPADDR_TYPE_V6);
memcpy(&ip_2_ip6(ip)->addr, &addr6->sin6_addr.un.u8_addr, 16);
// Unmap ::ffff:a.b.c.d so replies to recvfrom addresses route as IPv4
if (ip6_addr_isipv4mappedipv6(ip_2_ip6(ip))) {
unmap_ipv4_mapped_ipv6(ip_2_ip4(ip), ip_2_ip6(ip));
IP_SET_TYPE_VAL(*ip, IPADDR_TYPE_V4);
}
return true;
}
#endif
return false;
}
bool sockaddr_to_lwip_bind(sa_family_t family, const struct sockaddr *addr, socklen_t addrlen, ip_addr_t *ip,
uint16_t *port) {
if (!sockaddr_to_lwip(addr, addrlen, ip, port))
return false;
#if LWIP_IPV6
// Only promote wildcard binds — a specific address must keep filtering
if (family == AF_INET6 && ip_addr_isany_val(*ip))
IP_SET_TYPE_VAL(*ip, IPADDR_TYPE_ANY);
#endif
return true;
}
int lwip_bind_err(err_t err) {
if (err == ERR_OK)
return 0;
if (err == ERR_USE) {
errno = EADDRINUSE;
} else if (err == ERR_VAL) {
errno = EINVAL;
} else {
errno = EIO;
}
return -1;
}
} // namespace esphome::socket
#endif // USE_SOCKET_IMPL_LWIP_TCP
@@ -0,0 +1,53 @@
#pragma once
#include "esphome/core/defines.h"
#ifdef USE_SOCKET_IMPL_LWIP_TCP
#include "headers.h"
#include "lwip/ip.h"
namespace esphome::socket {
// ---- LWIP thread safety ----
//
// On RP2040 (Pico W), arduino-pico sets PICO_CYW43_ARCH_THREADSAFE_BACKGROUND=1.
// This means lwip callbacks (recv_fn, accept_fn, err_fn) run from a low-priority
// user IRQ context, not the main loop (see low_priority_irq_handler() in pico-sdk
// async_context_threadsafe_background.c). They can preempt main-loop code at any point.
//
// Without locking, this causes race conditions between recv_fn and read() on the
// shared rx_buf_ pbuf chain — recv_fn calls pbuf_cat() while read() is freeing
// nodes, leading to use-after-free and infinite-loop crashes. See esphome#10681.
//
// On ESP8266, lwip callbacks run from the SYS context which cooperates with user
// code (CONT context) — they never preempt each other, so no locking is needed.
//
// esphome::LwIPLock is the platform-provided RAII guard (see helpers.h/helpers.cpp).
// On RP2040, it acquires cyw43_arch_lwip_begin/end (WiFi) or ethernet_arch_lwip_begin/end
// (Ethernet). On ESP8266, it's a no-op.
//
// Each .cpp file that needs locking defines its own LWIP_LOCK() macro:
// #define LWIP_LOCK() esphome::LwIPLock lwip_lock_guard
// This is a per-TU convenience macro, not defined here to avoid macro leaking.
/// Convert lwip ip_addr_t + host-order port to sockaddr, based on the socket's address family.
/// TCP callers pass ntohs(pcb port) to preserve historical getpeername/getsockname output.
int lwip_ip_to_sockaddr(sa_family_t family, const ip_addr_t *ip, uint16_t port_host, struct sockaddr *name,
socklen_t *addrlen);
/// Convert sockaddr to lwip ip_addr_t and host-order port.
/// For IPv6, sets type to IPADDR_TYPE_V6 — correct for sendto destinations.
/// Bind paths must use sockaddr_to_lwip_bind() instead.
bool sockaddr_to_lwip(const struct sockaddr *addr, socklen_t addrlen, ip_addr_t *ip, uint16_t *port);
/// sockaddr_to_lwip variant for bind: promotes AF_INET6 sockets to
/// IPADDR_TYPE_ANY so they accept both IPv4 and IPv6 (dual-stack).
bool sockaddr_to_lwip_bind(sa_family_t family, const struct sockaddr *addr, socklen_t addrlen, ip_addr_t *ip,
uint16_t *port);
/// Map lwip bind error to errno. Returns 0 on success, -1 on error with errno set.
int lwip_bind_err(err_t err);
} // namespace esphome::socket
#endif // USE_SOCKET_IMPL_LWIP_TCP
+13 -106
View File
@@ -10,6 +10,7 @@
#include "esphome/core/helpers.h"
#include "esphome/core/wake.h"
#include "esphome/core/log.h"
#include "lwip_raw_common_impl.h"
#ifdef USE_OTA_PLATFORM_ESPHOME
extern "C" void esphome_wake_ota_component_any_context();
@@ -24,23 +25,9 @@ extern "C" void esphome_wake_ota_component_any_context();
namespace esphome::socket {
// ---- LWIP thread safety ----
//
// On RP2040 (Pico W), arduino-pico sets PICO_CYW43_ARCH_THREADSAFE_BACKGROUND=1.
// This means lwip callbacks (recv_fn, accept_fn, err_fn) run from a low-priority
// user IRQ context, not the main loop (see low_priority_irq_handler() in pico-sdk
// async_context_threadsafe_background.c). They can preempt main-loop code at any point.
//
// Without locking, this causes race conditions between recv_fn and read() on the
// shared rx_buf_ pbuf chain — recv_fn calls pbuf_cat() while read() is freeing
// nodes, leading to use-after-free and infinite-loop crashes. See esphome#10681.
//
// On ESP8266, lwip callbacks run from the SYS context which cooperates with user
// code (CONT context) — they never preempt each other, so no locking is needed.
//
// esphome::LwIPLock is the platform-provided RAII guard (see helpers.h/helpers.cpp).
// On RP2040, it acquires cyw43_arch_lwip_begin/end (WiFi) or ethernet_arch_lwip_begin/end
// (Ethernet). On ESP8266, it's a no-op.
// LWIP thread safety — see lwip_raw_common_impl.h for full explanation.
// esphome::LwIPLock is the platform-provided RAII guard.
// On RP2040, it acquires cyw43_arch_lwip_begin/end. On ESP8266, it's a no-op.
#define LWIP_LOCK() esphome::LwIPLock lwip_lock_guard // NOLINT
static const char *const TAG = "socket";
@@ -112,59 +99,14 @@ int LWIPRawCommon::bind(const struct sockaddr *name, socklen_t addrlen) {
return -1;
}
ip_addr_t ip;
in_port_t port;
#if LWIP_IPV6
if (this->family_ == AF_INET) {
if (addrlen < sizeof(sockaddr_in)) {
errno = EINVAL;
return -1;
}
auto *addr4 = reinterpret_cast<const sockaddr_in *>(name);
port = ntohs(addr4->sin_port);
ip.type = IPADDR_TYPE_V4;
ip.u_addr.ip4.addr = addr4->sin_addr.s_addr;
LWIP_LOG("tcp_bind(%p ip=%s port=%u)", this->pcb_, ip4addr_ntoa(&ip.u_addr.ip4), port);
} else if (this->family_ == AF_INET6) {
if (addrlen < sizeof(sockaddr_in6)) {
errno = EINVAL;
return -1;
}
auto *addr6 = reinterpret_cast<const sockaddr_in6 *>(name);
port = ntohs(addr6->sin6_port);
ip.type = IPADDR_TYPE_ANY;
memcpy(&ip.u_addr.ip6.addr, &addr6->sin6_addr.un.u8_addr, 16);
LWIP_LOG("tcp_bind(%p ip=%s port=%u)", this->pcb_, ip6addr_ntoa(&ip.u_addr.ip6), port);
} else {
uint16_t port;
if (!sockaddr_to_lwip_bind(this->family_, name, addrlen, &ip, &port)) {
errno = EINVAL;
return -1;
}
#else
if (this->family_ != AF_INET) {
errno = EINVAL;
return -1;
}
auto *addr4 = reinterpret_cast<const sockaddr_in *>(name);
port = ntohs(addr4->sin_port);
ip.addr = addr4->sin_addr.s_addr;
LWIP_LOG("tcp_bind(%p ip=%u port=%u)", this->pcb_, ip.addr, port);
#endif
err_t err = tcp_bind(this->pcb_, &ip, port);
if (err == ERR_USE) {
LWIP_LOG(" -> err ERR_USE");
errno = EADDRINUSE;
return -1;
}
if (err == ERR_VAL) {
LWIP_LOG(" -> err ERR_VAL");
errno = EINVAL;
return -1;
}
if (err != ERR_OK) {
LWIP_LOG(" -> err %d", err);
errno = EIO;
return -1;
}
return 0;
LWIP_LOG(" -> err %d", err);
return lwip_bind_err(err);
}
int LWIPRawCommon::close() {
@@ -349,43 +291,8 @@ int LWIPRawCommon::setsockopt(int level, int optname, const void *optval, sockle
}
int LWIPRawCommon::ip2sockaddr_(ip_addr_t *ip, uint16_t port, struct sockaddr *name, socklen_t *addrlen) {
if (this->family_ == AF_INET) {
if (*addrlen < sizeof(struct sockaddr_in)) {
errno = EINVAL;
return -1;
}
struct sockaddr_in *addr = reinterpret_cast<struct sockaddr_in *>(name);
addr->sin_family = AF_INET;
*addrlen = addr->sin_len = sizeof(struct sockaddr_in);
addr->sin_port = port;
inet_addr_from_ip4addr(&addr->sin_addr, ip_2_ip4(ip));
return 0;
}
#if LWIP_IPV6
else if (this->family_ == AF_INET6) {
if (*addrlen < sizeof(struct sockaddr_in6)) {
errno = EINVAL;
return -1;
}
struct sockaddr_in6 *addr = reinterpret_cast<struct sockaddr_in6 *>(name);
addr->sin6_family = AF_INET6;
*addrlen = addr->sin6_len = sizeof(struct sockaddr_in6);
addr->sin6_port = port;
// AF_INET6 sockets are bound to IPv4 as well, so we may encounter IPv4 addresses that must be converted to IPv6.
if (IP_IS_V4(ip)) {
ip_addr_t mapped;
ip4_2_ipv4_mapped_ipv6(ip_2_ip6(&mapped), ip_2_ip4(ip));
inet6_addr_from_ip6addr(&addr->sin6_addr, ip_2_ip6(&mapped));
} else {
inet6_addr_from_ip6addr(&addr->sin6_addr, ip_2_ip6(ip));
}
return 0;
}
#endif
return -1;
// lwip pcb ports are host order; ntohs preserves historical byte-swapped sin_port output
return lwip_ip_to_sockaddr(this->family_, ip, ntohs(port), name, addrlen);
}
// ---- LWIPRawImpl methods ----
@@ -887,11 +794,11 @@ err_t LWIPRawListenImpl::accept_fn_(struct tcp_pcb *newpcb, err_t err) {
return ERR_OK;
}
// ---- Factory functions ----
// ---- TCP Factory functions ----
std::unique_ptr<Socket> socket(int domain, int type, int protocol) {
if (type != SOCK_STREAM) {
ESP_LOGE(TAG, "UDP sockets not supported on this platform, use WiFiUDP");
ESP_LOGE(TAG, "Use socket_udp() for UDP sockets on this platform");
errno = EPROTOTYPE;
return nullptr;
}
@@ -911,7 +818,7 @@ std::unique_ptr<Socket> socket_loop_monitored(int domain, int type, int protocol
std::unique_ptr<ListenSocket> socket_listen(int domain, int type, int protocol) {
if (type != SOCK_STREAM) {
ESP_LOGE(TAG, "UDP sockets not supported on this platform, use WiFiUDP");
ESP_LOGE(TAG, "Use socket_udp() for UDP sockets on this platform");
errno = EPROTOTYPE;
return nullptr;
}
@@ -0,0 +1,328 @@
#include "socket.h"
#include "esphome/core/defines.h"
#ifdef USE_SOCKET_IMPL_LWIP_TCP
#include <cerrno>
#include <cstring>
#include "esphome/core/helpers.h"
#include "esphome/core/log.h"
#include "esphome/core/wake.h"
#include "lwip_raw_common_impl.h"
#include "lwip/igmp.h"
#include "lwip/pbuf.h"
#include "lwip/udp.h"
namespace esphome::socket {
// LWIP thread safety — see lwip_raw_common_impl.h for full explanation.
// esphome::LwIPLock is the platform-provided RAII guard.
// On RP2040, it acquires cyw43_arch_lwip_begin/end. On ESP8266, it's a no-op.
#define LWIP_LOCK() esphome::LwIPLock lwip_lock_guard // NOLINT
// ---- LWIPRawUDPSendImpl (send-only) methods ----
LWIPRawUDPSendImpl::~LWIPRawUDPSendImpl() {
// Guard avoids acquiring the lwip lock when already closed
if (this->pcb_ != nullptr)
this->close();
}
int LWIPRawUDPSendImpl::bind(const struct sockaddr *name, socklen_t addrlen) {
LWIP_LOCK();
if (this->pcb_ == nullptr) {
errno = EBADF;
return -1;
}
if (name == nullptr) {
errno = EINVAL;
return -1;
}
ip_addr_t ip;
uint16_t port;
if (!sockaddr_to_lwip_bind(this->family_, name, addrlen, &ip, &port)) {
errno = EINVAL;
return -1;
}
return lwip_bind_err(udp_bind(this->pcb_, &ip, port));
}
int LWIPRawUDPSendImpl::close() {
LWIP_LOCK();
return this->close_internal_locked_();
}
int LWIPRawUDPSendImpl::close_internal_locked_() {
// Caller must hold LWIP_LOCK
if (this->pcb_ == nullptr) {
errno = EBADF;
return -1;
}
udp_remove(this->pcb_);
this->pcb_ = nullptr;
return 0;
}
int LWIPRawUDPSendImpl::ip2sockaddr_(const ip_addr_t *ip, uint16_t port, struct sockaddr *name, socklen_t *addrlen) {
// UDP recv callback provides port in host byte order
return lwip_ip_to_sockaddr(this->family_, ip, port, name, addrlen);
}
ssize_t LWIPRawUDPSendImpl::sendto(const void *buf, size_t len, int flags, const struct sockaddr *dest_addr,
socklen_t addrlen) {
(void) flags; // Flags (MSG_DONTWAIT, etc.) are ignored; raw lwip is always non-blocking
if (buf == nullptr || dest_addr == nullptr) {
errno = EINVAL;
return -1;
}
// pbuf_alloc takes u16_t length; reject oversized packets
if (len > UINT16_MAX) {
errno = EMSGSIZE;
return -1;
}
ip_addr_t dst_ip;
uint16_t dst_port;
if (!sockaddr_to_lwip(dest_addr, addrlen, &dst_ip, &dst_port)) {
errno = EINVAL;
return -1;
}
LWIP_LOCK();
if (this->pcb_ == nullptr) {
errno = EBADF;
return -1;
}
// Allocate pbuf and copy data
struct pbuf *pb = pbuf_alloc(PBUF_TRANSPORT, (uint16_t) len, PBUF_RAM);
if (pb == nullptr) {
errno = ENOMEM;
return -1;
}
memcpy(pb->payload, buf, len);
err_t err = udp_sendto(this->pcb_, pb, &dst_ip, dst_port);
pbuf_free(pb);
if (err != ERR_OK) {
errno = err == ERR_MEM ? ENOMEM : EIO;
return -1;
}
return (ssize_t) len;
}
int LWIPRawUDPSendImpl::setsockopt(int level, int optname, const void *optval, socklen_t optlen) {
LWIP_LOCK();
if (this->pcb_ == nullptr) {
errno = EBADF;
return -1;
}
if (level == SOL_SOCKET && optname == SO_REUSEADDR) {
if (optval == nullptr || optlen < sizeof(int)) {
errno = EINVAL;
return -1;
}
// Effective only where lwip is built with SO_REUSE=1 (ESP8266 yes, RP2040 currently no)
if (*reinterpret_cast<const int *>(optval)) {
ip_set_option(this->pcb_, SOF_REUSEADDR);
} else {
ip_reset_option(this->pcb_, SOF_REUSEADDR);
}
return 0;
}
if (level == SOL_SOCKET && optname == SO_BROADCAST) {
if (optval == nullptr || optlen < sizeof(int)) {
errno = EINVAL;
return -1;
}
int val = *reinterpret_cast<const int *>(optval);
if (val) {
ip_set_option(this->pcb_, SOF_BROADCAST);
} else {
ip_reset_option(this->pcb_, SOF_BROADCAST);
}
return 0;
}
if (level == IPPROTO_IP && (optname == IP_ADD_MEMBERSHIP || optname == IP_DROP_MEMBERSHIP)) {
if (optval == nullptr || optlen < sizeof(struct ip_mreq)) {
errno = EINVAL;
return -1;
}
auto *mreq = reinterpret_cast<const struct ip_mreq *>(optval);
ip4_addr_t multiaddr{mreq->imr_multiaddr.s_addr};
ip4_addr_t ifaddr{mreq->imr_interface.s_addr};
err_t err =
optname == IP_ADD_MEMBERSHIP ? igmp_joingroup(&ifaddr, &multiaddr) : igmp_leavegroup(&ifaddr, &multiaddr);
if (err != ERR_OK) {
errno = EIO;
return -1;
}
return 0;
}
errno = ENOPROTOOPT;
return -1;
}
int LWIPRawUDPSendImpl::getsockopt(int level, int optname, void *optval, socklen_t *optlen) {
LWIP_LOCK();
if (this->pcb_ == nullptr) {
errno = EBADF;
return -1;
}
if (level == SOL_SOCKET && optname == SO_REUSEADDR) {
if (optval == nullptr || optlen == nullptr || *optlen < sizeof(int)) {
errno = EINVAL;
return -1;
}
*reinterpret_cast<int *>(optval) = ip_get_option(this->pcb_, SOF_REUSEADDR) ? 1 : 0;
*optlen = sizeof(int);
return 0;
}
errno = ENOPROTOOPT;
return -1;
}
int LWIPRawUDPSendImpl::setblocking(bool blocking) {
if (blocking) {
// blocking operation not supported on raw lwip
errno = EINVAL;
return -1;
}
return 0;
}
// ---- LWIPRawUDPImpl methods ----
LWIPRawUDPImpl::LWIPRawUDPImpl(sa_family_t family, struct udp_pcb *pcb) : LWIPRawUDPSendImpl(family, pcb) {
// Registered here (not in bind) so unbound client sockets can receive replies
udp_recv(this->pcb_, LWIPRawUDPImpl::s_recv_fn, this);
}
LWIPRawUDPImpl::~LWIPRawUDPImpl() {
// Flush rx queue and unregister callback before base destructor removes pcb
if (this->pcb_ != nullptr)
this->close();
}
int LWIPRawUDPImpl::close() {
LWIP_LOCK();
// Unregister recv callback before removing pcb
if (this->pcb_ != nullptr) {
udp_recv(this->pcb_, nullptr, nullptr);
}
// Flush queued rx packets; slots within rx_count_ always hold a live pbuf
for (; this->rx_count_ > 0; this->rx_count_--) {
pbuf_free(this->rx_queue_[this->rx_read_idx_].pb);
this->rx_read_idx_ = (this->rx_read_idx_ + 1) & UDP_RX_MASK;
}
// close_internal_locked_() returns EBADF if already closed, which is fine from destructor
return this->close_internal_locked_();
}
ssize_t LWIPRawUDPImpl::read(void *buf, size_t len) { return this->recvfrom(buf, len, nullptr, nullptr); }
ssize_t LWIPRawUDPImpl::recvfrom(void *buf, size_t len, struct sockaddr *src_addr, socklen_t *addrlen) {
if (buf == nullptr && len > 0) {
errno = EINVAL;
return -1;
}
LWIP_LOCK();
if (this->pcb_ == nullptr) {
errno = EBADF;
return -1;
}
if (this->rx_count_ == 0) {
errno = EWOULDBLOCK;
return -1;
}
auto &pkt = this->rx_queue_[this->rx_read_idx_];
// On address conversion failure, still consume the packet — the failure is
// deterministic (family_ and *addrlen), so keeping it would wedge the queue
ssize_t ret = -1;
if (src_addr == nullptr || addrlen == nullptr ||
this->ip2sockaddr_(&pkt.src_addr, pkt.src_port, src_addr, addrlen) == 0) {
ret = (ssize_t) std::min(len, (size_t) pkt.pb->tot_len);
pbuf_copy_partial(pkt.pb, buf, ret, 0);
}
pbuf_free(pkt.pb);
this->rx_read_idx_ = (this->rx_read_idx_ + 1) & UDP_RX_MASK;
this->rx_count_--;
return ret;
}
void LWIPRawUDPImpl::s_recv_fn(void *arg, struct udp_pcb *pcb, struct pbuf *p, const ip_addr_t *addr, u16_t port) {
auto *self = reinterpret_cast<LWIPRawUDPImpl *>(arg);
self->recv_fn_(p, addr, port);
}
// LWIP CALLBACK — runs from IRQ context on RP2040 (low-priority user IRQ).
// No heap allocation allowed — malloc is not IRQ-safe (see #14687).
// No LWIP_LOCK() needed — lwip core already holds the async_context lock.
void LWIPRawUDPImpl::recv_fn_(struct pbuf *p, const ip_addr_t *addr, u16_t port) {
if (p == nullptr)
return;
// Check if queue is full
if (this->rx_count_ >= UDP_RX_QUEUE_SIZE) {
// Drop packet — queue full. Can't log from IRQ context, so count it
// (saturating) for consumers to surface via get_rx_dropped().
if (this->rx_dropped_ != UINT16_MAX)
this->rx_dropped_++;
pbuf_free(p);
return;
}
// Enqueue the packet
uint8_t write_idx = (this->rx_read_idx_ + this->rx_count_) & UDP_RX_MASK;
auto &slot = this->rx_queue_[write_idx];
slot.pb = p;
slot.src_addr = *addr;
slot.src_port = port;
this->rx_count_++;
esphome::wake_loop_any_context();
}
// ---- UDP Factory functions ----
static struct udp_pcb *new_udp_pcb(int domain) {
#if LWIP_IPV6
return udp_new_ip_type(domain == AF_INET6 ? IPADDR_TYPE_ANY : IPADDR_TYPE_V4);
#else
return udp_new();
#endif
}
std::unique_ptr<UDPSendSocket> socket_udp_send(int domain, int protocol) {
(void) protocol; // Raw lwip UDP ignores protocol; kept for API compatibility
LWIP_LOCK();
auto *pcb = new_udp_pcb(domain);
if (pcb == nullptr) {
errno = ENOMEM;
return nullptr;
}
return make_unique<LWIPRawUDPSendImpl>((sa_family_t) domain, pcb);
}
std::unique_ptr<UDPSocket> socket_udp(int domain, int protocol) {
(void) protocol; // Raw lwip UDP ignores protocol; kept for API compatibility
LWIP_LOCK();
auto *pcb = new_udp_pcb(domain);
if (pcb == nullptr) {
errno = ENOMEM;
return nullptr;
}
// Ctor registers the recv callback under the lock held here
return make_unique<LWIPRawUDPImpl>((sa_family_t) domain, pcb);
}
#undef LWIP_LOCK
} // namespace esphome::socket
#endif // USE_SOCKET_IMPL_LWIP_TCP
@@ -0,0 +1,113 @@
#pragma once
#include "esphome/core/defines.h"
#ifdef USE_SOCKET_IMPL_LWIP_TCP
#include <array>
#include <cstdint>
#include "headers.h"
#include "lwip/ip.h"
#include "lwip/udp.h"
namespace esphome::socket {
/// Send-only UDP socket implementation for LWIP raw API.
/// Non-virtual, concrete type. Uses lwip/udp.h raw API.
/// No receive capability — use LWIPRawUDPImpl for sockets that need to receive.
class LWIPRawUDPSendImpl {
public:
/// The pcb is allocated by the factory (like the TCP impl); never null here.
LWIPRawUDPSendImpl(sa_family_t family, struct udp_pcb *pcb) : pcb_(pcb), family_(family) {}
~LWIPRawUDPSendImpl();
LWIPRawUDPSendImpl(const LWIPRawUDPSendImpl &) = delete;
LWIPRawUDPSendImpl &operator=(const LWIPRawUDPSendImpl &) = delete;
int bind(const struct sockaddr *name, socklen_t addrlen);
int close();
/// Send a UDP packet to the specified destination.
ssize_t sendto(const void *buf, size_t len, int flags, const struct sockaddr *dest_addr, socklen_t addrlen);
int setsockopt(int level, int optname, const void *optval, socklen_t optlen);
int getsockopt(int level, int optname, void *optval, socklen_t *optlen);
int setblocking(bool blocking);
bool ready() const { return false; }
int get_fd() const { return -1; }
protected:
/// Convert lwip ip_addr_t and port to sockaddr.
int ip2sockaddr_(const ip_addr_t *ip, uint16_t port, struct sockaddr *name, socklen_t *addrlen);
/// Shared close logic — removes the udp pcb. Caller must hold LWIP_LOCK.
int close_internal_locked_();
struct udp_pcb *pcb_;
sa_family_t family_;
};
/// UDP socket with receive support for LWIP raw API.
/// Extends LWIPRawUDPSendImpl with a fixed-size ring buffer for incoming packets.
/// Inheritance is private (base dtor is non-virtual; converting to a base
/// pointer would leak queued pbufs on destruction).
class LWIPRawUDPImpl : private LWIPRawUDPSendImpl {
public:
/// Caller (the factory) must hold the lwip lock; registers the recv callback.
LWIPRawUDPImpl(sa_family_t family, struct udp_pcb *pcb);
~LWIPRawUDPImpl();
using LWIPRawUDPSendImpl::bind;
using LWIPRawUDPSendImpl::get_fd;
using LWIPRawUDPSendImpl::getsockopt;
using LWIPRawUDPSendImpl::sendto;
using LWIPRawUDPSendImpl::setblocking;
using LWIPRawUDPSendImpl::setsockopt;
/// Close the socket, flushing any queued rx packets first.
int close();
/// Read the next queued packet, discarding source address info.
/// If buf is smaller than the packet, data is silently truncated (returns bytes copied).
/// Note: unlike POSIX MSG_TRUNC, this does not return the original packet length on truncation.
ssize_t read(void *buf, size_t len);
/// Read the next queued packet and return the source address.
/// If buf is smaller than the packet, data is silently truncated (returns bytes copied).
/// Note: unlike POSIX MSG_TRUNC, this does not return the original packet length on truncation.
ssize_t recvfrom(void *buf, size_t len, struct sockaddr *src_addr, socklen_t *addrlen);
/// Returns true if there are packets available to read.
/// Intentionally unlocked — same rationale as LWIPRawImpl::ready().
bool ready() const { return this->rx_count_ > 0; }
/// Number of packets dropped because the rx queue was full (saturating).
uint16_t get_rx_dropped() const { return this->rx_dropped_; }
protected:
static void s_recv_fn(void *arg, struct udp_pcb *pcb, struct pbuf *p, const ip_addr_t *addr, u16_t port);
void recv_fn_(struct pbuf *p, const ip_addr_t *addr, u16_t port);
/// Ring buffer for received UDP packets.
/// Both producer (recv callback) and consumer (main loop) are serialized by the
/// lwip lock — the callback runs under lwip core lock, and consumer methods hold
/// LWIP_LOCK(). All 4 slots are usable (no wasted slot for full/empty distinction).
/// No heap allocation in the recv callback — packets are dropped if the queue is full.
static constexpr uint8_t UDP_RX_QUEUE_SIZE = 4;
static constexpr uint8_t UDP_RX_MASK = UDP_RX_QUEUE_SIZE - 1;
static_assert((UDP_RX_QUEUE_SIZE & UDP_RX_MASK) == 0, "UDP_RX_QUEUE_SIZE must be power of 2");
// Fields are written by recv_fn_ before rx_count_ makes a slot visible
struct UDPRxPacket {
ip_addr_t src_addr;
struct pbuf *pb;
uint16_t src_port;
};
std::array<UDPRxPacket, UDP_RX_QUEUE_SIZE> rx_queue_{};
uint16_t rx_dropped_{0};
uint8_t rx_read_idx_{0};
uint8_t rx_count_{0};
};
} // namespace esphome::socket
#endif // USE_SOCKET_IMPL_LWIP_TCP
@@ -84,6 +84,9 @@ class LwIPSocketImpl {
int get_fd() const { return this->fd_; }
/// UDP rx drop counter parity with LWIPRawUDPImpl; drops are not counted here.
uint16_t get_rx_dropped() const { return 0; }
protected:
// fd_ < 0 means "not open" — used both pre-open (initial state) and post-close. This
// replaces a separate closed_ flag: close() sets fd_ = -1 after lwip_close(), and the
+1 -19
View File
@@ -116,25 +116,7 @@ size_t format_sockaddr_to(const struct sockaddr *addr_ptr, socklen_t len, std::s
return 0;
}
std::unique_ptr<Socket> socket_ip(int type, int protocol) {
#if USE_NETWORK_IPV6
return socket(AF_INET6, type, protocol);
#else
return socket(AF_INET, type, protocol);
#endif /* USE_NETWORK_IPV6 */
}
#ifdef USE_SOCKET_IMPL_LWIP_TCP
// LWIP_TCP has separate Socket/ListenSocket types — needs out-of-line factory.
// BSD and LWIP_SOCKETS define this inline in socket.h.
std::unique_ptr<ListenSocket> socket_ip_loop_monitored(int type, int protocol) {
#if USE_NETWORK_IPV6
return socket_listen_loop_monitored(AF_INET6, type, protocol);
#else
return socket_listen_loop_monitored(AF_INET, type, protocol);
#endif /* USE_NETWORK_IPV6 */
}
#endif
std::unique_ptr<Socket> socket_ip(int type, int protocol) { return socket(IP_DOMAIN, type, protocol); }
socklen_t set_sockaddr(struct sockaddr *addr, socklen_t addrlen, const char *ip_address, uint16_t port) {
#if USE_NETWORK_IPV6
+49 -8
View File
@@ -20,6 +20,7 @@
#include "lwip_sockets_impl.h"
#elif defined(USE_SOCKET_IMPL_LWIP_TCP)
#include "lwip_raw_tcp_impl.h"
#include "lwip_raw_udp_impl.h"
#endif
namespace esphome::socket {
@@ -27,17 +28,32 @@ namespace esphome::socket {
// Type aliases — only one implementation is active per build.
// Socket is the concrete type for connected sockets.
// ListenSocket is the concrete type for listening/server sockets.
// On BSD and LWIP_SOCKETS, both aliases resolve to the same type.
// UDPSocket is the concrete type for UDP sockets (send + receive).
// UDPSendSocket is the concrete type for send-only UDP sockets.
// On BSD and LWIP_SOCKETS, all aliases resolve to the same type.
// On LWIP_TCP, they are different types (no virtual dispatch between them).
#ifdef USE_SOCKET_IMPL_BSD_SOCKETS
using Socket = BSDSocketImpl;
using ListenSocket = BSDSocketImpl;
using UDPSendSocket = BSDSocketImpl;
using UDPSocket = BSDSocketImpl;
#elif defined(USE_SOCKET_IMPL_LWIP_SOCKETS)
using Socket = LwIPSocketImpl;
using ListenSocket = LwIPSocketImpl;
using UDPSendSocket = LwIPSocketImpl;
using UDPSocket = LwIPSocketImpl;
#elif defined(USE_SOCKET_IMPL_LWIP_TCP)
using Socket = LWIPRawImpl;
using ListenSocket = LWIPRawListenImpl;
using UDPSendSocket = LWIPRawUDPSendImpl;
using UDPSocket = LWIPRawUDPImpl;
#endif
// Domain used by the socket_ip_* helpers: newest available IP domain.
#if USE_NETWORK_IPV6
inline constexpr int IP_DOMAIN = AF_INET6;
#else
inline constexpr int IP_DOMAIN = AF_INET;
#endif
#ifdef USE_LWIP_FAST_SELECT
@@ -104,6 +120,36 @@ std::unique_ptr<Socket> socket_ip(int type, int protocol);
/// File descriptors >= FD_SETSIZE will not be monitored and will log an error.
std::unique_ptr<Socket> socket_loop_monitored(int domain, int type, int protocol);
/// Create a send-only UDP socket (socket_udp_send), a UDP socket with receive
/// support (socket_udp), or a UDP socket monitored for data in the main loop
/// (socket_udp_loop_monitored).
#ifdef USE_SOCKET_IMPL_LWIP_TCP
std::unique_ptr<UDPSendSocket> socket_udp_send(int domain, int protocol);
std::unique_ptr<UDPSocket> socket_udp(int domain, int protocol);
// Wake is built into the recv callback, so monitoring needs nothing extra.
inline std::unique_ptr<UDPSocket> socket_udp_loop_monitored(int domain, int protocol) {
return socket_udp(domain, protocol);
}
#else
inline std::unique_ptr<UDPSendSocket> socket_udp_send(int domain, int protocol) {
return esphome::socket::socket(domain, SOCK_DGRAM, protocol);
}
inline std::unique_ptr<UDPSocket> socket_udp(int domain, int protocol) {
return esphome::socket::socket(domain, SOCK_DGRAM, protocol);
}
// Registers the socket with the Application's select() loop.
inline std::unique_ptr<UDPSocket> socket_udp_loop_monitored(int domain, int protocol) {
return socket_loop_monitored(domain, SOCK_DGRAM, protocol);
}
#endif
/// socket_udp* variants using the newest available IP domain.
inline std::unique_ptr<UDPSendSocket> socket_ip_udp_send(int protocol) { return socket_udp_send(IP_DOMAIN, protocol); }
inline std::unique_ptr<UDPSocket> socket_ip_udp(int protocol) { return socket_udp(IP_DOMAIN, protocol); }
inline std::unique_ptr<UDPSocket> socket_ip_udp_loop_monitored(int protocol) {
return socket_udp_loop_monitored(IP_DOMAIN, protocol);
}
/// Create a listening socket of the given domain, type and protocol.
/// Create a listening socket and monitor it for data in the main loop.
/// Create a listening socket in the newest available IP domain and monitor it.
@@ -111,7 +157,6 @@ std::unique_ptr<Socket> socket_loop_monitored(int domain, int type, int protocol
// LWIP_TCP has separate Socket/ListenSocket types — needs distinct factory functions.
std::unique_ptr<ListenSocket> socket_listen(int domain, int type, int protocol);
std::unique_ptr<ListenSocket> socket_listen_loop_monitored(int domain, int type, int protocol);
std::unique_ptr<ListenSocket> socket_ip_loop_monitored(int type, int protocol);
#else
// BSD and LWIP_SOCKETS: Socket == ListenSocket, so listen variants just delegate.
inline std::unique_ptr<ListenSocket> socket_listen(int domain, int type, int protocol) {
@@ -120,14 +165,10 @@ inline std::unique_ptr<ListenSocket> socket_listen(int domain, int type, int pro
inline std::unique_ptr<ListenSocket> socket_listen_loop_monitored(int domain, int type, int protocol) {
return socket_loop_monitored(domain, type, protocol);
}
#endif
inline std::unique_ptr<ListenSocket> socket_ip_loop_monitored(int type, int protocol) {
#if USE_NETWORK_IPV6
return socket_loop_monitored(AF_INET6, type, protocol);
#else
return socket_loop_monitored(AF_INET, type, protocol);
#endif
return socket_listen_loop_monitored(IP_DOMAIN, type, protocol);
}
#endif
/// Set a sockaddr to the specified address and port for the IP version used by socket_ip().
/// @param addr Destination sockaddr structure
+16 -81
View File
@@ -9,19 +9,16 @@ byte-identical to PlatformIO's output:
Flash: [=== ] 48.4% (used 888511 bytes from 1835008 bytes)
The format matches ``script/ci_memory_impact_extract.py`` so CI memory
analysis works unchanged on native ESP-IDF builds. RAM usage comes from
the DRAM (or unified DIRAM) region of the linker map. Flash used is the
exact image size matching the ``Total image size`` line: json2
``total_size`` when present, otherwise derived from the ELF (see
``_image_size_from_elf``). Flash total is taken from
analysis works unchanged on native ESP-IDF builds. RAM total is the
DRAM region size from the linker map; Flash total is taken from
``partitions.csv`` using PlatformIO's rule (first app partition whose
subtype is ``factory`` or ``ota_0``; see
``platform-espressif32/builder/main.py::_update_max_upload_size``).
Structured size data is produced at link time by a CMake POST_BUILD
custom command (see ``build_gen/espidf.py``) which writes
``esp_idf_size.json`` (``--format=json2``, a per-memory-type summary)
next to the ELF; we read that rather than re-running ``esp_idf_size``.
``esp_idf_size.json`` next to the ELF. We read that file here rather
than re-running ``esp_idf_size`` from Python.
"""
from __future__ import annotations
@@ -30,7 +27,6 @@ import csv
import json
import logging
from pathlib import Path
import struct
from esphome.build_helpers.size_summary import print_size_line
@@ -73,43 +69,11 @@ def _find_app_partition_size(partitions_csv: Path) -> int:
raise ValueError(f"No app+factory or app+ota_0 partition in {partitions_csv}")
def _image_size_from_elf(elf: Path) -> int:
"""Sum the allocated PROGBITS section sizes from an ELF32 file.
Matches ``esp_idf_size.ng.memorymap._get_image_size`` byte for byte;
esptool's ``ELFFile`` filters sections differently and would not.
Raises ``ValueError`` for anything but a well-formed ELF32 LE file.
"""
with elf.open("rb") as f:
header = f.read(52) # ELF32 header
if len(header) < 52 or header[:6] != b"\x7fELF\x01\x01":
raise ValueError(f"{elf} is not a 32-bit little-endian ELF")
(e_shoff,) = struct.unpack_from("<I", header, 0x20) # e_shoff
e_shentsize, e_shnum = struct.unpack_from("<HH", header, 0x2E)
if e_shentsize < 40: # sizeof(Elf32_Shdr)
raise ValueError(f"{elf} has an invalid section header size")
f.seek(e_shoff)
table = f.read(e_shnum * e_shentsize)
if len(table) < e_shnum * e_shentsize:
raise ValueError(f"{elf} has a truncated section header table")
total = 0
for off in range(0, e_shnum * e_shentsize, e_shentsize):
sh_type, sh_flags = struct.unpack_from("<II", table, off + 4)
(sh_size,) = struct.unpack_from("<I", table, off + 20)
if sh_type == 1 and sh_flags & 0x2: # SHT_PROGBITS with SHF_ALLOC
total += sh_size
if total == 0:
# A used-0-bytes Flash line would read as a real measurement
raise ValueError(f"{elf} has no allocated PROGBITS sections")
return total
def print_summary(size_json: Path, partitions_csv: Path, firmware_elf: Path) -> None:
def print_summary(size_json: Path, partitions_csv: Path | None) -> None:
"""Print PlatformIO-shaped RAM and Flash one-liners.
Failures are non-fatal: the build has already succeeded, we just couldn't
summarize. Anomalies (missing region, unreadable ELF) warn; expected
optional inputs (no size json, no partitions.csv) log at debug.
summarize. Logs the cause at debug level.
"""
if not size_json.is_file():
_LOGGER.debug("Skipping size summary: %s not found", size_json)
@@ -119,49 +83,20 @@ def print_summary(size_json: Path, partitions_csv: Path, firmware_elf: Path) ->
except (OSError, json.JSONDecodeError) as e:
_LOGGER.debug("Skipping size summary: %s", e)
return
if not isinstance(data, dict):
_LOGGER.warning("Skipping size summary: unexpected json shape in %s", size_json)
return
layout = data.get("layout")
regions = {
entry.get("name"): entry
for entry in (layout if isinstance(layout, list) else [])
if isinstance(entry, dict)
}
# Every chip has a DRAM or DIRAM region, so a warning here usually
# means the esp_idf_size json schema changed
ram_region = regions.get("DRAM") or regions.get("DIRAM")
if ram_region is None:
_LOGGER.warning("Skipping RAM summary: no DRAM/DIRAM region in %s", size_json)
elif (
isinstance(ram_total := ram_region.get("total"), int)
and ram_total > 0
and isinstance(ram_used := ram_region.get("used"), int)
):
memory_types = data.get("memory_types", {})
ram_region = memory_types.get("DRAM") or memory_types.get("DIRAM") or {}
ram_used = ram_region.get("used")
ram_total = ram_region.get("size")
if ram_total and ram_used is not None:
print_size_line("RAM", ram_used, ram_total)
else:
_LOGGER.warning(
"Skipping RAM summary: unusable region %s in %s", ram_region, size_json
)
# esp-idf-size >= 2.1 (IDF >= 6.0) reports the exact image size in
# json2; older 1.x omits it, so derive the same figure from the ELF.
flash_used = data.get("total_size")
if not (isinstance(flash_used, int) and flash_used > 0):
_LOGGER.debug("No total_size in %s, deriving from %s", size_json, firmware_elf)
try:
flash_used = _image_size_from_elf(firmware_elf)
except (OSError, ValueError) as e:
# The ELF must be present and well formed after a successful build
_LOGGER.warning("Skipping Flash summary: %s", e)
return
image_size = data.get("image_size")
if image_size is None or partitions_csv is None:
return
try:
app_size = _find_app_partition_size(partitions_csv)
except (OSError, ValueError) as e:
except ValueError as e:
_LOGGER.debug("Skipping Flash summary: %s", e)
return
if app_size <= 0:
_LOGGER.debug("Skipping Flash summary: app partition size is 0")
return
print_size_line("Flash", flash_used, app_size)
print_size_line("Flash", image_size, app_size)
+3 -12
View File
@@ -542,7 +542,7 @@ def run_compile(config, verbose: bool) -> int:
if rc == 0:
size_json = CORE.relative_build_path("build", "esp_idf_size.json")
partitions = CORE.relative_build_path("partitions.csv")
print_summary(size_json, partitions, get_built_elf_path())
print_summary(size_json, partitions if partitions.is_file() else None)
return rc
@@ -579,16 +579,6 @@ def get_ota_firmware_path() -> Path:
return build_dir / "firmware.ota.bin"
def get_built_elf_path() -> Path:
"""Path to the ELF idf.py writes directly, ``<build>/<name>.elf``.
Exists as soon as the build finishes, unlike the ``firmware.elf``
copy that ``create_elf_copy`` makes later.
"""
build_dir = CORE.relative_build_path("build")
return build_dir / f"{CORE.name}.elf"
def get_elf_path() -> Path:
"""Get the path to the firmware ELF file.
@@ -716,7 +706,8 @@ def create_elf_copy() -> bool:
"download ELF" link requests the literal filename ``firmware.elf``
(PlatformIO convention), so copy it to that name.
"""
src_elf = get_built_elf_path()
build_dir = CORE.relative_build_path("build")
src_elf = build_dir / f"{CORE.name}.elf"
dst_elf = get_elf_path()
if not src_elf.is_file():
-12
View File
@@ -163,18 +163,6 @@ def test_has_discovered_components_after_configure(tmp_path: Path) -> None:
assert has_discovered_components()
def test_get_project_cmakelists_size_command_uses_json2() -> None:
"""The POST_BUILD size command uses the cheap json2 format, with --ng
only on the 1.x tool bundled with IDF < 6."""
content = _render()
assert "-m esp_idf_size --ng --format=json2" in content
CORE.data[KEY_ESP32][KEY_IDF_VERSION] = cv.Version(6, 0, 0)
content = _render()
assert "--ng" not in content
assert "--format=json2" in content
def test_get_project_cmakelists_uses_supplied_builtin_components() -> None:
"""A cached list replaces project_description.json and is still filtered
by EXCLUDE_COMPONENTS."""
-37
View File
@@ -638,43 +638,6 @@ def test_run_compile_passes_compile_process_limit(setup_core: Path) -> None:
mock_run.assert_called_once_with("build", "size", jobs=1)
def test_run_compile_passes_size_summary_paths(setup_core: Path) -> None:
"""print_summary receives the size json, partitions.csv, and the built
ELF from get_built_elf_path, which must stay in lockstep with the
project() name in the generated CMakeLists."""
_setup_build(setup_core)
config = {CONF_ESPHOME: {}}
with (
patch.object(toolchain, "need_reconfigure", return_value=False),
patch.object(toolchain, "run_idf_py", return_value=0),
patch.object(toolchain, "print_summary") as mock_summary,
):
assert toolchain.run_compile(config, verbose=False) == 0
mock_summary.assert_called_once_with(
CORE.relative_build_path("build", "esp_idf_size.json"),
CORE.relative_build_path("partitions.csv"),
CORE.relative_build_path("build", f"{CORE.name}.elf"),
)
def test_create_elf_copy(setup_core: Path) -> None:
"""The built <name>.elf is copied to the firmware.elf dashboard name."""
_setup_build(setup_core)
src = toolchain.get_built_elf_path()
src.parent.mkdir(parents=True, exist_ok=True)
src.write_bytes(b"elf")
assert toolchain.create_elf_copy() is True
assert toolchain.get_elf_path().read_bytes() == b"elf"
def test_create_elf_copy_missing_source(setup_core: Path) -> None:
"""A missing built ELF is a warning and False, not a crash."""
_setup_build(setup_core)
assert toolchain.create_elf_copy() is False
def test_run_compile_without_compile_process_limit(setup_core: Path) -> None:
"""When no compile_process_limit is set, no job limit is passed to idf.py."""
_setup_build(setup_core)
+56 -230
View File
@@ -4,8 +4,6 @@ from __future__ import annotations
import json
from pathlib import Path
import struct
from unittest.mock import patch
import pytest
@@ -19,106 +17,64 @@ def _write_size_json(tmp_path: Path, data: dict) -> Path:
return out
def _write_partitions(tmp_path: Path) -> Path:
"""Drop a partitions.csv with a 0x1C0000 (1835008 byte) app slot."""
out = tmp_path / "partitions.csv"
out.write_text(
"# name, type, subtype, offset, size, flags\n"
"app0, app, ota_0, 0x10000, 0x1C0000,\n"
)
return out
def _elf_bytes(sections: list[tuple[int, int, int]], shentsize: int = 40) -> bytes:
"""Build a minimal ELF32 LE whose section headers carry the given
(sh_type, sh_flags, sh_size) triples."""
out = bytearray(52)
out[0:4] = b"\x7fELF"
out[4] = out[5] = 1 # 32-bit, little-endian
struct.pack_into("<I", out, 0x20, 52) # e_shoff
struct.pack_into("<HH", out, 0x2E, shentsize, len(sections))
for sh_type, sh_flags, sh_size in sections:
shdr = bytearray(40)
struct.pack_into("<II", shdr, 4, sh_type, sh_flags)
struct.pack_into("<I", shdr, 20, sh_size)
out += shdr
return bytes(out)
def _esp32_size_data() -> dict:
"""Synthetic json2 for the original ESP32 (split IRAM/DRAM), in the
esp-idf-size >= 2.1 shape that carries ``total_size``."""
"""Synthetic esp_idf_size.json for the original ESP32 (split IRAM/DRAM)."""
return {
"version": "1.1",
"total_size": 827455,
"layout": [
{
"name": "DRAM",
"total": 180736,
"image_size": 827455,
"memory_types": {
"DRAM": {
"size": 180736,
"used": 47332,
"free": 133404,
"parts": {
".bss": {"size": 30616},
".data": {"size": 16716},
"sections": {
".dram0.bss": {"abbrev_name": ".bss", "size": 30616},
".dram0.data": {"abbrev_name": ".data", "size": 16716},
},
},
{
"name": "IRAM",
"total": 131072,
"IRAM": {
"size": 131072,
"used": 80351,
"free": 50721,
"parts": {
".text": {"size": 79323},
".vectors": {"size": 1028},
"sections": {
".iram0.text": {"abbrev_name": ".text", "size": 79323},
".iram0.vectors": {"abbrev_name": ".vectors", "size": 1028},
},
},
],
},
}
def _s3_size_data() -> dict:
"""Synthetic json2 for ESP32-S3 (unified DIRAM), in the esp-idf-size 1.x
shape without ``total_size``."""
"""Synthetic esp_idf_size.json for ESP32-S3 (unified DIRAM)."""
return {
"version": "1.1",
"layout": [
{
"name": "DIRAM",
"total": 341760,
"image_size": 724215,
"memory_types": {
"DIRAM": {
"size": 341760,
"used": 104999,
"free": 236761,
"parts": {
".text": {"size": 58051},
".bss": {"size": 27088},
".data": {"size": 19708},
".noinit": {"size": 152},
"sections": {
".iram0.text": {"abbrev_name": ".text", "size": 58051},
".dram0.bss": {"abbrev_name": ".bss", "size": 27088},
".dram0.data": {"abbrev_name": ".data", "size": 19708},
".noinit": {"abbrev_name": ".noinit", "size": 152},
},
},
{
"name": "IRAM",
"total": 16384,
"IRAM": {
"size": 16384,
"used": 16384,
"free": 0,
"parts": {
".text": {"size": 15356},
".vectors": {"size": 1028},
"sections": {
".iram0.text": {"abbrev_name": ".text", "size": 15356},
".iram0.vectors": {"abbrev_name": ".vectors", "size": 1028},
},
},
],
},
}
def _print_summary_ram_only(tmp_path: Path, size_json: Path) -> None:
"""Call print_summary with no partitions.csv or ELF on disk."""
print_summary(size_json, tmp_path / "partitions.csv", tmp_path / "firmware.elf")
def test_print_summary_esp32_uses_dram(
tmp_path: Path, capsys: pytest.CaptureFixture[str]
) -> None:
"""Original ESP32: RAM = DRAM.used / DRAM.total."""
"""Original ESP32: DRAM has no ``.text``, so RAM = DRAM.used / DRAM.size unchanged."""
size_json = _write_size_json(tmp_path, _esp32_size_data())
_print_summary_ram_only(tmp_path, size_json)
print_summary(size_json, partitions_csv=None)
out = capsys.readouterr().out
assert "RAM:" in out
assert "used 47332 bytes from 180736 bytes" in out
@@ -127,193 +83,63 @@ def test_print_summary_esp32_uses_dram(
def test_print_summary_s3_falls_back_to_diram(
tmp_path: Path, capsys: pytest.CaptureFixture[str]
) -> None:
"""ESP32-S3 with no DRAM entry falls back to DIRAM and reports raw region usage."""
"""ESP32-S3 with no DRAM key falls back to DIRAM and reports raw region usage."""
size_json = _write_size_json(tmp_path, _s3_size_data())
_print_summary_ram_only(tmp_path, size_json)
print_summary(size_json, partitions_csv=None)
out = capsys.readouterr().out
assert "used 104999 bytes from 341760 bytes" in out
def test_print_summary_skips_when_diram_total_collapses(
tmp_path: Path,
capsys: pytest.CaptureFixture[str],
caplog: pytest.LogCaptureFixture,
tmp_path: Path, capsys: pytest.CaptureFixture[str]
) -> None:
"""A zero-size region drops the RAM line rather than divide by zero."""
size_json = _write_size_json(
tmp_path,
{
"version": "1.1",
"layout": [{"name": "DIRAM", "total": 0, "used": 0}],
"memory_types": {
"DIRAM": {
"size": 0,
"used": 0,
"sections": {},
},
},
},
)
_print_summary_ram_only(tmp_path, size_json)
print_summary(size_json, partitions_csv=None)
out = capsys.readouterr().out
assert "RAM:" not in out
assert "unusable region" in caplog.text
def test_print_summary_handles_missing_json(
tmp_path: Path, capsys: pytest.CaptureFixture[str]
) -> None:
"""Missing size json is non-fatal and prints nothing."""
_print_summary_ram_only(tmp_path, tmp_path / "does_not_exist.json")
print_summary(tmp_path / "does_not_exist.json", partitions_csv=None)
assert capsys.readouterr().out == ""
def test_print_summary_handles_no_layout(
tmp_path: Path,
capsys: pytest.CaptureFixture[str],
caplog: pytest.LogCaptureFixture,
) -> None:
"""A size json without ``layout`` warns so schema drift is visible."""
size_json = _write_size_json(tmp_path, {"version": "1.1"})
_print_summary_ram_only(tmp_path, size_json)
assert capsys.readouterr().out == ""
assert any(
r.levelname == "WARNING" and "no DRAM/DIRAM region" in r.message
for r in caplog.records
)
def test_print_summary_flash_line_prefers_total_size(
def test_print_summary_handles_no_memory_types(
tmp_path: Path, capsys: pytest.CaptureFixture[str]
) -> None:
"""With ``total_size`` in the json, that figure wins without reading the
ELF, in the exact shape script/ci_memory_impact_extract.py greps."""
size_json = _write_size_json(tmp_path, _esp32_size_data())
partitions = _write_partitions(tmp_path)
print_summary(size_json, partitions, tmp_path / "firmware.elf")
out = capsys.readouterr().out
assert "Flash: " in out
assert "(used 827455 bytes from 1835008 bytes)" in out
def test_print_summary_flash_line_derives_from_elf(
tmp_path: Path, capsys: pytest.CaptureFixture[str]
) -> None:
"""A 1.x json without ``total_size`` sums the ELF's loadable PROGBITS
sections; NOBITS and non-alloc sections are excluded."""
size_json = _write_size_json(tmp_path, _s3_size_data())
partitions = _write_partitions(tmp_path)
firmware_elf = tmp_path / "firmware.elf"
firmware_elf.write_bytes(
_elf_bytes(
[
(1, 0x6, 700000), # PROGBITS, alloc+exec: counted
(1, 0x2, 24215), # PROGBITS, alloc: counted
(8, 0x2, 50000), # NOBITS (.bss): excluded
(1, 0x0, 12345), # PROGBITS, no alloc (.debug_*): excluded
]
)
)
print_summary(size_json, partitions, firmware_elf)
out = capsys.readouterr().out
assert "(used 724215 bytes from 1835008 bytes)" in out
@pytest.mark.parametrize(
"data",
[
pytest.param([1, 2], id="top_level_list"),
pytest.param({"version": "1.1", "layout": None}, id="layout_null"),
pytest.param({"version": "1.1", "layout": 7}, id="layout_scalar"),
],
)
def test_print_summary_handles_unexpected_shapes(
data: object, tmp_path: Path, capsys: pytest.CaptureFixture[str]
) -> None:
"""A foreign-schema size json degrades to a warning, never a traceback."""
size_json = _write_size_json(tmp_path, data)
_print_summary_ram_only(tmp_path, size_json)
"""A size json without ``memory_types`` still doesn't crash."""
size_json = _write_size_json(tmp_path, {"image_size": 0})
print_summary(size_json, partitions_csv=None)
assert capsys.readouterr().out == ""
def test_print_summary_skips_flash_on_zero_app_partition(
def test_print_summary_flash_line(
tmp_path: Path, capsys: pytest.CaptureFixture[str]
) -> None:
"""A zero-size app partition skips the Flash line rather than printing
a from-0-bytes figure CI would record."""
"""A partition table with an app row yields the Flash line in the exact
padded shape script/ci_memory_impact_extract.py greps."""
size_json = _write_size_json(tmp_path, _esp32_size_data())
partitions = tmp_path / "partitions.csv"
partitions.write_text(
"# name, type, subtype, offset, size, flags\napp0, app, ota_0, 0x10000, 0x0,\n"
"# name, type, subtype, offset, size, flags\n"
"app0, app, ota_0, 0x10000, 0x1C0000,\n"
)
print_summary(size_json, partitions, tmp_path / "firmware.elf")
print_summary(size_json, partitions)
out = capsys.readouterr().out
assert "Flash:" not in out
def test_print_summary_skips_flash_on_unreadable_partitions(
tmp_path: Path, capsys: pytest.CaptureFixture[str]
) -> None:
"""An unreadable partitions.csv is non-fatal (chmod tricks don't work
for root in CI containers, so simulate the OSError instead)."""
size_json = _write_size_json(tmp_path, _esp32_size_data())
partitions = _write_partitions(tmp_path)
with patch(
"esphome.espidf.size_summary._find_app_partition_size",
side_effect=PermissionError("denied"),
):
print_summary(size_json, partitions, tmp_path / "firmware.elf")
assert "Flash:" not in capsys.readouterr().out
def test_print_summary_flash_falls_back_on_bad_total_size(
tmp_path: Path, capsys: pytest.CaptureFixture[str]
) -> None:
"""A zero or non-int total_size falls back to the ELF instead of
printing a used-0-bytes line CI would read as a real measurement."""
data = _s3_size_data()
data["total_size"] = 0
size_json = _write_size_json(tmp_path, data)
partitions = _write_partitions(tmp_path)
firmware_elf = tmp_path / "firmware.elf"
firmware_elf.write_bytes(_elf_bytes([(1, 0x2, 4096)]))
print_summary(size_json, partitions, firmware_elf)
out = capsys.readouterr().out
assert "(used 4096 bytes from 1835008 bytes)" in out
_GOOD_ELF = _elf_bytes([(1, 0x2, 1024)])
@pytest.mark.parametrize(
("elf_bytes", "with_partitions"),
[
pytest.param(None, True, id="missing_elf"),
pytest.param(b"junk", True, id="not_an_elf"),
pytest.param(
_elf_bytes([(1, 0x2, 1024)], shentsize=0), True, id="bad_shentsize"
),
pytest.param(_GOOD_ELF[:60], True, id="truncated_table"),
pytest.param(_elf_bytes([]), True, id="no_sections"),
pytest.param(_elf_bytes([(8, 0x2, 50000)]), True, id="no_progbits"),
pytest.param(_GOOD_ELF, False, id="missing_partitions"),
],
)
def test_print_summary_skips_flash_on_bad_input(
elf_bytes: bytes | None,
with_partitions: bool,
tmp_path: Path,
capsys: pytest.CaptureFixture[str],
caplog: pytest.LogCaptureFixture,
) -> None:
"""An unusable ELF or missing partitions.csv skips the Flash line, not the RAM line."""
size_json = _write_size_json(tmp_path, _s3_size_data())
firmware_elf = tmp_path / "firmware.elf"
if elf_bytes is not None:
firmware_elf.write_bytes(elf_bytes)
if with_partitions:
_write_partitions(tmp_path)
print_summary(size_json, tmp_path / "partitions.csv", firmware_elf)
out = capsys.readouterr().out
assert "RAM:" in out
assert "Flash:" not in out
# ELF problems warn (anomaly after a successful build); a missing
# partitions.csv stays at debug
warned = any(
r.levelname == "WARNING" and "Skipping Flash summary" in r.message
for r in caplog.records
)
assert warned == with_partitions
assert "Flash: " in out
assert "(used 827455 bytes from 1835008 bytes)" in out