Compare commits

...
Author SHA1 Message Date
J. Nick Koston 205318ee1a Merge remote-tracking branch 'origin/web-server-sse-listing-per-session' into web-server-event-source-shared 2026-11-15 19:04:55 -06:00
J. Nick Koston 48d8df449a Merge remote-tracking branch 'origin/web-server-idf-stream-response' into web-server-sse-listing-per-session 2026-11-15 19:04:52 -06:00
J. Nick Koston 383ac1b41a Merge remote-tracking branch 'origin/dev' into web-server-idf-stream-response 2026-11-15 19:04:42 -06:00
J. Nick Koston 7acd023079 [web_server_base] Bump ESPAsyncWebServer to 3.12.1 (#22682) 2026-11-16 13:43:15 +13:00
Clyde Stubbs 5623a5d511 [sx127x] Add async sends and failure recovery (#22733) 2026-11-15 23:52:31 +00:00
J. Nick Koston 4f8c2239dd [components] Use put_be16 and the other byte helpers in more drivers (#22771) 2026-11-15 23:32:08 +00:00
J. Nick Koston 0ac2299482 Merge remote-tracking branch 'origin/web-server-sse-listing-per-session' into HEAD 2026-11-14 08:40:18 -06:00
J. Nick Koston b58038af63 Merge remote-tracking branch 'origin/web-server-idf-stream-response' into HEAD 2026-11-14 08:39:50 -06:00
J. Nick Koston c741de7a47 Merge remote-tracking branch 'origin/dev' into HEAD 2026-11-14 08:38:08 -06:00
J. Nick Koston 8a5e07e29f [web_server] Keep the server out of host builds, which only need sse_chunk.cpp for unit tests
tests/components/web_server now pulls web_server into the --all host
build, where another test defines USE_WEBSERVER.
2026-11-14 00:19:08 -06:00
J. Nick Koston f7b52be670 Merge remote-tracking branch 'origin/web-server-sse-listing-per-session' into web-server-event-source-shared 2026-11-13 23:15:51 -06:00
J. Nick Koston d5317669c9 Merge remote-tracking branch 'origin/web-server-idf-stream-response' into web-server-sse-listing-per-session 2026-11-13 23:15:42 -06:00
J. Nick Koston 5d0bd88bc6 Merge remote-tracking branch 'origin/dev' into web-server-idf-stream-response 2026-11-13 23:13:08 -06:00
pre-commit-ci-lite[bot] b714d8f69b apply automatic formatting fixes 2026-11-14 04:33:03 +00:00
J. Nick Koston ecabdbb24b [web_server] Keep the shared /events layer free of IDF-only types
The gather type comes from stream_response.h, the default header count is
defined on every platform, the session takes a non-const request, and the
fd in log lines goes through log_id(). Also drops a dead forward declaration
and an unused alias, and narrows the source filter.
2026-11-13 22:28:19 -06:00
J. Nick Koston ffbc4148fb [web_server] Move the ESP32 /events layer to web_server/event_source
The event source and its sessions move out of web_server_idf into
web_server/event_source.h/.cpp, sse_chunk moves to web_server and
stream_head.h to web_server_base, so the ESPAsyncWebServer platforms can
share them later. web_server_base/stream_response.h picks the stream
transport. No behaviour change; off ESP32 the build is unchanged.
2026-11-13 22:16:53 -06:00
J. Nick Koston 2169167908 [web_server] Name the iterator's session member session_ 2026-11-13 22:00:50 -06:00
J. Nick Koston 900574c03a [web_server_idf] List entities to the new /events session only
The listing iterator held the whole event source, so each new client's
detail_all listing went to every open session. Hold the session instead,
as the Arduino path already does.
2026-11-13 21:55:42 -06:00
J. Nick Koston 7b1f9c6bf1 Merge remote-tracking branch 'origin/dev' into web-server-idf-stream-response
# Conflicts:
#	.agents/skills/hardware-test/reference/recipes/web_server.md
2026-11-07 22:09:36 -06:00
J. Nick Koston 3587e29725 Merge branch 'dev' into web-server-idf-stream-response 2026-11-04 22:21:12 -06:00
J. Nick Koston d4244c5388 [web_server_idf] Correct the session close order comment and point stall docs at the stream 2026-11-04 21:47:51 -06:00
J. Nick Koston bec53baf1d Merge branch 'dev' into web-server-idf-stream-response 2026-11-04 21:29:11 -06:00
J. Nick Koston 98fd3786dd [web_server_idf] Create the stream send mutex at boot and test that a failed head batch stops the rest 2026-11-04 21:15:36 -06:00
J. Nick Koston 1f531ff022 Merge branch 'dev' into web-server-idf-stream-response 2026-11-04 20:58:44 -06:00
J. Nick Koston 65e7eb6b06 [web_server_idf] Note that sess_ctx is reserved for AsyncStreamResponse 2026-11-04 20:41:33 -06:00
J. Nick Koston 7538864e1e [web_server_idf] Keep a stream from sending on a socket httpd has closed 2026-11-04 20:36:27 -06:00
J. Nick Koston c0f97475d3 [web_server_idf] Document the stream head body as a C string 2026-11-04 20:07:37 -06:00
J. Nick Koston df6fcaa6fe Merge remote-tracking branch 'origin/dev' into web-server-idf-stream-response
# Conflicts:
#	esphome/components/web_server_idf/web_server_idf.cpp
#	esphome/components/web_server_idf/web_server_idf.h
2026-11-03 18:47:25 -06:00
J. Nick Koston eb1b793af9 Merge branch 'dev' into web-server-idf-stream-response 2026-10-30 15:46:21 -10:00
J. Nick Koston b8ce904e0c [web_server_idf] Let a stream response header replace a default header of the same name 2026-10-30 14:24:31 -10:00
J. Nick Koston 6befa73b00 [web_server_idf] Shut the socket down when the stream head is not fully sent 2026-10-30 13:09:28 -10:00
J. Nick Koston 02211e15d8 [web_server_idf] Skip the stall clock for empty writes and clear it on send errors 2026-10-30 12:52:36 -10:00
J. Nick Koston dbbe02f423 [web_server_idf] Include stream_head.h without web_server, for HttpHeader 2026-10-30 12:35:17 -10:00
J. Nick Koston 97a1aa781e [web_server_idf] Simplify the stream response stall clock and head batching
Log the no-memory warning once per stall, free the closing session's buffers once,
add is_stalled() and check that the event stream head fits one write.
2026-10-30 12:22:07 -10:00
J. Nick Koston 0b67a754cd [web_server_idf] Build the stream head tests only for host 2026-10-30 12:15:18 -10:00
J. Nick Koston 320b6a8221 [web_server_idf] Factor the stream response head, iovec advance and stall clock out for host tests
Moves the pure parts of AsyncStreamResponse into stream_head.h (no httpd or
lwIP includes) and adds gtests: the event stream head is compared byte for
byte against what httpd_resp_send_chunk() sent, plus raw heads, batching,
short writes and the stall clock including a millis() rollover.
2026-10-30 12:06:59 -10:00
J. Nick Koston 4e5a5ec32d [web_server_idf] Send the stream response head in one write and tidy the stall clock
Gather the status line, headers and optional body into one iovec list
and send it with a single blocking sendmsg, instead of one httpd_send per
piece. Split the stall clock into start_stall_(), stall_expired_() and
clear_stall_(), and use is_connected() and is_closing() in the event
stream.
2026-10-30 11:40:30 -10:00
J. Nick Koston 09ce8d7d6e [web_server_idf] Add a long-lived streaming response that components can use
Move the session handling of AsyncEventSourceResponse into a new
AsyncStreamResponse base: the response head and session context set up
in the request handler, the non-blocking send override, main loop gather
writes with the send stall clock, the identity-checked close through
httpd_queue_work and the LRU refresh. The event stream now derives from
it, so other components can stream raw or chunked bytes from the main
loop without their own tasks or raw socket calls.

The response head is written directly instead of through
httpd_resp_send_chunk(), so a stream does not have to start with a data
chunk. The event stream still sends the same bytes on the wire.
2026-10-30 11:35:09 -10:00
48 changed files with 2629 additions and 968 deletions

No files matched your search

@@ -278,10 +278,9 @@ APIError APINoiseFrameHelper::state_action_client_hello_() {
state_ = State::FAILED;
return APIError::OUT_OF_MEMORY;
}
dst[0] = (uint8_t) (rx_size >> 8);
dst[1] = (uint8_t) rx_size;
uint8_t *payload = put_be16(dst, rx_size);
if (rx_size > 0) {
std::memcpy(dst + 2, this->rx_buf_.data(), rx_size);
std::memcpy(payload, this->rx_buf_.data(), rx_size);
}
#ifdef USE_API_OUTGOING_CONNECTION
@@ -464,10 +463,8 @@ APIError APINoiseFrameHelper::encrypt_noise_message_(uint8_t *buf_start, uint16_
// Write message header (to be encrypted)
constexpr uint8_t msg_offset = noise::FRAME_HEADER_SIZE;
buf_start[msg_offset] = static_cast<uint8_t>(message_type >> 8); // type high byte
buf_start[msg_offset + 1] = static_cast<uint8_t>(message_type); // type low byte
buf_start[msg_offset + 2] = static_cast<uint8_t>(payload_size >> 8); // data_len high byte
buf_start[msg_offset + 3] = static_cast<uint8_t>(payload_size); // data_len low byte
put_be16(buf_start + msg_offset, message_type);
put_be16(buf_start + msg_offset + 2, payload_size);
// payload data is already in the buffer starting at offset + 7
// Encrypt the message in place
@@ -1,6 +1,7 @@
#include "api_overflow_buffer.h"
#ifdef USE_API
#include <cstring>
#include "esphome/core/helpers.h"
namespace esphome::api {
@@ -17,7 +18,7 @@ ssize_t APIOverflowBuffer::try_drain(socket::Socket *socket) {
while (this->count_ > 0) {
uint8_t *msg = this->buf_.data() + this->head_;
size_t len = msg[0] | (msg[1] << 8);
size_t len = encode_value_le<uint16_t>(msg);
ssize_t sent = socket->write(msg + LEN_PREFIX, len);
if (sent <= 0)
+5 -1
View File
@@ -35,7 +35,11 @@ async def to_code(config: ConfigType) -> None:
add_idf_component(name="esp32async/asynctcp", ref="3.4.91")
elif CORE.is_libretiny:
# https://github.com/ESP32Async/AsyncTCP
cg.add_library("ESP32Async/AsyncTCP", "3.4.5")
cg.add_library("ESP32Async/AsyncTCP", "3.5.0")
# AsyncTCP 3.5.0 debug logs call xPortGetCoreID(), which LibreTiny (single core) lacks; they are compiled
# in by LibreTiny's debug level or CONFIG_ASYNC_TCP_DEBUG, so the call always expands to int(), which is 0
# (a function-like -D does not survive PlatformIO's build flags)
cg.add_build_flag("-DxPortGetCoreID=int")
elif CORE.is_esp8266:
# https://github.com/ESP32Async/ESPAsyncTCP
cg.add_library("ESP32Async/ESPAsyncTCP", "2.0.0")
@@ -2,6 +2,7 @@
#include <algorithm>
#include <cstring>
#include "esphome/core/helpers.h"
namespace esphome::ble_device_base {
@@ -163,8 +164,7 @@ bool aes_ccm_auth_decrypt(const uint8_t key[16], const uint8_t *nonce, size_t no
if (aad_len > 0) {
// Only the < 2^16-2^8 encoding is needed for BLE-sized AAD.
uint8_t blk[16] = {0};
blk[0] = static_cast<uint8_t>((aad_len >> 8) & 0xff);
blk[1] = static_cast<uint8_t>(aad_len & 0xff);
put_be16(blk, aad_len);
size_t ai = 0;
size_t pos = 2;
while (pos < 16 && ai < aad_len)
+2 -2
View File
@@ -1,6 +1,7 @@
#include "cse7761.h"
#include "esphome/core/log.h"
#include "esphome/core/helpers.h"
namespace esphome::cse7761 {
@@ -104,8 +105,7 @@ void CSE7761Component::write_(uint8_t reg, uint16_t data) {
buffer[2] = data & 0xFF;
len = 3;
} else {
buffer[2] = (data >> 8) & 0xFF;
buffer[3] = data & 0xFF;
put_be16(buffer + 2, data);
len = 4;
}
buffer[len] = ~sum_bytes(buffer, len);
+1 -1
View File
@@ -173,7 +173,7 @@ void DFPlayer::loop() {
}
// Parse valid received command
uint8_t cmd = this->read_buffer_[3];
uint16_t argument = (this->read_buffer_[5] << 8) | this->read_buffer_[6];
uint16_t argument = encode_uint16(this->read_buffer_[5], this->read_buffer_[6]);
ESP_LOGV(TAG, "Received message cmd: %#02x arg %#04x", cmd, argument);
+1 -1
View File
@@ -22,7 +22,7 @@ void EE895Component::setup() {
uint8_t serial_number[20];
this->read(serial_number, 20);
crc16_check = (serial_number[19] << 8) + serial_number[18];
crc16_check = encode_value_le<uint16_t>(serial_number + 18);
if (!this->setup_check(crc16_check == calc_crc16_(serial_number, 18), LOG_STR("CRC check failed")))
return;
#if ESPHOME_LOG_LEVEL >= ESPHOME_LOG_LEVEL_VERBOSE
@@ -4,6 +4,7 @@
#include <cstring>
#include "esphome/core/application.h"
#include "esphome/core/log.h"
#include "esphome/core/helpers.h"
namespace esphome::entity_transport {
@@ -373,8 +374,7 @@ size_t EntityTransport::write_record_(const LocalEntity &entity, uint8_t *record
#endif
uint16_t tag =
entity.number | (type << TAG_TYPE_SHIFT) | (payload << TAG_PAYLOAD_SHIFT) | (value ? TAG_VALUE_BIT : 0);
record[0] = tag;
record[1] = tag >> 8;
put_le16(record, tag);
return size;
}
@@ -1,5 +1,6 @@
#include "inkbird_ibsth1_mini.h"
#include "esphome/core/log.h"
#include "esphome/core/helpers.h"
namespace esphome::inkbird_ibsth1_mini {
@@ -82,7 +83,7 @@ bool InkbirdIbstH1Mini::parse_device(const ble_device_base::ESPBTDevice &device)
}
auto battery_level = mnf_data.data[5];
auto humidity = ((mnf_data.data[1] << 8) + mnf_data.data[0]) / 100.0f;
auto humidity = encode_value_le<uint16_t>(mnf_data.data.data()) / 100.0f;
// Send temperature only if the value is set
if (!std::isnan(temperature) && this->temperature_sensor_ != nullptr) {
+2 -1
View File
@@ -1,4 +1,5 @@
#include "modbus_helpers.h"
#include "esphome/core/helpers.h"
#include "esphome/core/log.h"
#include <algorithm>
@@ -471,7 +472,7 @@ PduBuffer create_client_pdu(FunctionCode function_code, uint16_t start_address,
values[1]);
return pdu;
}
append_pdu_header(pdu, function_code, start_address, uint16_t((values[0] << 8) | values[1]));
append_pdu_header(pdu, function_code, start_address, encode_uint16(values[0], values[1]));
return pdu;
}
// The quantity is spec-bounded above, so the data length just has to agree with it exactly
@@ -283,7 +283,7 @@ void HdlcReliability::receive_(const PacketBuffer &data, const PacketMetaData &m
size_t length = frame.size() - FRAME_OVERHEAD;
uint8_t control = frame[0];
const uint8_t *payload = frame.data() + 1;
uint16_t crc = frame[length + 1] | (frame[length + 2] << 8);
uint16_t crc = encode_value_le<uint16_t>(frame.data() + length + 1);
if (frame_crc(control, payload, length) != crc) {
ESP_LOGV(TAG, "%s: CRC error, frame dropped", this->name_);
this->note_received_(false);
+18 -29
View File
@@ -434,42 +434,31 @@ bool RTSPSession::queue_next_packet_() {
const size_t rtp_len = header_len - INTERLEAVED_HEADER_LEN + qtables_len + payload_len;
auto *buf = reinterpret_cast<uint8_t *>(this->send_buf_.data());
size_t pos = 0;
buf[pos++] = '$';
buf[pos++] = 0; // interleaved channel 0
buf[pos++] = static_cast<uint8_t>(rtp_len >> 8);
buf[pos++] = static_cast<uint8_t>(rtp_len);
uint8_t *p = buf;
*p++ = '$';
*p++ = 0; // interleaved channel 0
p = put_be16(p, rtp_len);
buf[pos++] = 0x80; // V=2, P=0, X=0, CC=0
buf[pos++] = static_cast<uint8_t>((last_fragment ? 0x80 : 0x00) | RTP_PAYLOAD_TYPE_JPEG);
buf[pos++] = static_cast<uint8_t>(this->sequence_number_ >> 8);
buf[pos++] = static_cast<uint8_t>(this->sequence_number_);
*p++ = 0x80; // V=2, P=0, X=0, CC=0
*p++ = static_cast<uint8_t>((last_fragment ? 0x80 : 0x00) | RTP_PAYLOAD_TYPE_JPEG);
p = put_be16(p, this->sequence_number_);
this->sequence_number_++;
buf[pos++] = static_cast<uint8_t>(this->rtp_timestamp_ >> 24);
buf[pos++] = static_cast<uint8_t>(this->rtp_timestamp_ >> 16);
buf[pos++] = static_cast<uint8_t>(this->rtp_timestamp_ >> 8);
buf[pos++] = static_cast<uint8_t>(this->rtp_timestamp_);
buf[pos++] = static_cast<uint8_t>(this->ssrc_ >> 24);
buf[pos++] = static_cast<uint8_t>(this->ssrc_ >> 16);
buf[pos++] = static_cast<uint8_t>(this->ssrc_ >> 8);
buf[pos++] = static_cast<uint8_t>(this->ssrc_);
p = put_be32(p, this->rtp_timestamp_);
p = put_be32(p, this->ssrc_);
// RFC 2435 main JPEG header
buf[pos++] = 0; // type-specific
buf[pos++] = static_cast<uint8_t>(this->frame_sent_ >> 16);
buf[pos++] = static_cast<uint8_t>(this->frame_sent_ >> 8);
buf[pos++] = static_cast<uint8_t>(this->frame_sent_);
buf[pos++] = this->jpeg_.type;
buf[pos++] = 255; // Q >= 128: the quantization tables follow in the first packet
buf[pos++] = static_cast<uint8_t>(this->jpeg_.width / 8);
buf[pos++] = static_cast<uint8_t>(this->jpeg_.height / 8);
*p++ = 0; // type-specific
p = put_be24(p, this->frame_sent_);
*p++ = this->jpeg_.type;
*p++ = 255; // Q >= 128: the quantization tables follow in the first packet
*p++ = static_cast<uint8_t>(this->jpeg_.width / 8);
*p++ = static_cast<uint8_t>(this->jpeg_.height / 8);
uint8_t count = 0;
if (first_fragment) {
buf[pos++] = 0; // MBZ
buf[pos++] = 0; // 8-bit precision
buf[pos++] = static_cast<uint8_t>(qtables_len >> 8);
buf[pos++] = static_cast<uint8_t>(qtables_len);
*p++ = 0; // MBZ
*p++ = 0; // 8-bit precision
put_be16(p, qtables_len);
this->iov_[0] = {buf, header_len};
this->iov_[1] = {const_cast<uint8_t *>(this->jpeg_.qtable0), QTABLE_LEN};
this->iov_[2] = {const_cast<uint8_t *>(this->jpeg_.qtable1), QTABLE_LEN};
@@ -145,11 +145,9 @@ void RuntimeImage::draw_pixel(int x, int y, const Color &color) {
this->map_chroma_key(mapped_color);
uint16_t rgb565 = display::ColorUtil::color_to_565(mapped_color);
if (this->big_endian_) {
this->buffer_[pos + 0] = static_cast<uint8_t>((rgb565 >> 8) & 0xFF);
this->buffer_[pos + 1] = static_cast<uint8_t>(rgb565 & 0xFF);
put_be16(this->buffer_ + pos, rgb565);
} else {
this->buffer_[pos + 0] = static_cast<uint8_t>(rgb565 & 0xFF);
this->buffer_[pos + 1] = static_cast<uint8_t>((rgb565 >> 8) & 0xFF);
put_le16(this->buffer_ + pos, rgb565);
}
if (this->transparency_ == image::TRANSPARENCY_ALPHA_CHANNEL) {
const size_t alpha_pos = pos / 2 + this->buffer_width_ * this->buffer_height_ * 2;
@@ -48,17 +48,16 @@ bool SensirionI2CDevice::write_command_(uint16_t command, CommandLen command_len
temp[raw_idx++] = command & 0xFF;
} else {
// command is 2 bytes
temp[raw_idx++] = command >> 8;
temp[raw_idx++] = command & 0xFF;
put_be16(temp, command);
raw_idx = 2;
}
// add parameters followed by crc
// skipped if len == 0
for (size_t i = 0; i < data_len; i++) {
temp[raw_idx++] = data[i] >> 8;
temp[raw_idx++] = data[i] & 0xFF;
put_be16(temp + raw_idx, data[i]);
// Use MSB first since Sensirion devices use CRC-8 with MSB first
uint8_t crc = crc8(&temp[raw_idx - 2], 2, 0xFF, CRC_POLYNOMIAL, true);
temp[raw_idx++] = crc;
temp[raw_idx + 2] = crc8(temp + raw_idx, 2, 0xFF, CRC_POLYNOMIAL, true);
raw_idx += 3;
}
this->last_error_ = this->write(temp, raw_idx);
return this->last_error_ == i2c::ERROR_OK;
@@ -611,7 +611,7 @@ stm32_unique_ptr stm32_init(uart::UARTDevice *stream, const uint8_t flags, const
ESP_LOGD(TAG, "Only %d bytes sent in the PID, unknown/unsupported device", returned);
return make_stm32_with_deletor(nullptr);
}
stm->pid = (buf[1] << 8) | buf[2];
stm->pid = encode_uint16(buf[1], buf[2]);
if (returned > 2) {
ESP_LOGD(TAG, "This bootloader returns %d extra bytes in PID:", returned);
for (auto i = 2; i <= returned; i++) {
@@ -952,7 +952,7 @@ stm32_err_t stm32_crc_memory(const stm32_unique_ptr &stm, const uint32_t address
if (buf[4] != xor_bytes(buf, 4))
return STM32_ERR_UNKNOWN;
*crc = (buf[0] << 24) | (buf[1] << 16) | (buf[2] << 8) | buf[3];
*crc = encode_uint32(buf[0], buf[1], buf[2], buf[3]);
}
return STM32_ERR_OK;
@@ -123,8 +123,7 @@ void HOT SSD1331::draw_absolute_pixel_internal(int x, int y, Color color) {
const uint32_t color565 = display::ColorUtil::color_to_565(color);
// where should the bits go in the big buffer array? math...
uint16_t pos = (x + y * this->get_width_internal()) * SSD1331_BYTESPERPIXEL;
this->buffer_[pos++] = (color565 >> 8) & 0xff;
this->buffer_[pos] = color565 & 0xff;
put_be16(this->buffer_ + pos, color565);
}
void SSD1331::fill(Color color) {
// If clipping is active, fall back to base implementation
@@ -155,8 +155,7 @@ void HOT SSD1351::draw_absolute_pixel_internal(int x, int y, Color color) {
const uint32_t color565 = display::ColorUtil::color_to_565(color);
// where should the bits go in the big buffer array? math...
uint16_t pos = (x + y * this->get_width_internal()) * SSD1351_BYTESPERPIXEL;
this->buffer_[pos++] = (color565 >> 8) & 0xff;
this->buffer_[pos] = color565 & 0xff;
put_be16(this->buffer_ + pos, color565);
}
void SSD1351::fill(Color color) {
// If clipping is active, fall back to base implementation
+23 -48
View File
@@ -61,9 +61,7 @@ ConfigurationResult SX126xCommands::configure(const RadioProfile &profile, std::
if (profile.tcxo_voltage != TCXO_CTRL_NONE) {
uint32_t delay = profile.tcxo_delay >> 6;
buf[0] = profile.tcxo_voltage;
buf[1] = (delay >> 16) & 0xFF;
buf[2] = (delay >> 8) & 0xFF;
buf[3] = (delay >> 0) & 0xFF;
put_be24(buf + 1, delay);
this->write_opcode_(RADIO_SET_TCXOMODE, buf, 4);
buf[0] = 0x7F;
this->write_opcode_(RADIO_CALIBRATE, buf, 1);
@@ -99,10 +97,7 @@ ConfigurationResult SX126xCommands::configure(const RadioProfile &profile, std::
// set frequency
uint64_t freq = ((uint64_t) profile.frequency << 25) / XTAL_FREQ;
buf[0] = (uint8_t) ((freq >> 24) & 0xFF);
buf[1] = (uint8_t) ((freq >> 16) & 0xFF);
buf[2] = (uint8_t) ((freq >> 8) & 0xFF);
buf[3] = (uint8_t) (freq & 0xFF);
put_be32(buf, static_cast<uint32_t>(freq));
this->write_opcode_(RADIO_SET_RFFREQUENCY, buf, 4);
// configure pa
@@ -152,23 +147,17 @@ ConfigurationResult SX126xCommands::configure(const RadioProfile &profile, std::
// set modulation params
uint32_t bitrate = ((uint64_t) XTAL_FREQ * 32) / profile.bitrate;
uint32_t fdev = ((uint64_t) profile.deviation << 25) / XTAL_FREQ;
buf[0] = (bitrate >> 16) & 0xFF;
buf[1] = (bitrate >> 8) & 0xFF;
buf[2] = (bitrate >> 0) & 0xFF;
put_be24(buf, bitrate);
buf[3] = profile.shaping;
buf[4] = progmem_read_byte(&BW_FSK[profile.bandwidth - SX126X_BW_4800]);
buf[5] = (fdev >> 16) & 0xFF;
buf[6] = (fdev >> 8) & 0xFF;
buf[7] = (fdev >> 0) & 0xFF;
put_be24(buf + 5, fdev);
this->write_opcode_(RADIO_SET_MODULATIONPARAMS, buf, 8);
// set crc params
if (profile.crc_enable) {
buf[0] = profile.crc_initial >> 8;
buf[1] = profile.crc_initial & 0xFF;
put_be16(buf, profile.crc_initial);
this->write_register_(REG_CRC_INITIAL, buf, 2);
buf[0] = profile.crc_polynomial >> 8;
buf[1] = profile.crc_polynomial & 0xFF;
put_be16(buf, profile.crc_polynomial);
this->write_register_(REG_CRC_POLYNOMIAL, buf, 2);
}
@@ -203,8 +192,7 @@ void SX126xCommands::encode_packet_params(const RadioProfile &profile, AppliedPr
auto &buf = applied.packet_params;
const uint8_t payload_length = profile.max_packet_size();
if (profile.modulation == PACKET_TYPE_LORA) {
buf[0] = (profile.preamble_size >> 8) & 0xFF;
buf[1] = (profile.preamble_size >> 0) & 0xFF;
put_be16(buf.data(), profile.preamble_size);
buf[2] = (profile.payload_length > 0) ? 0x01 : 0x00;
buf[3] = payload_length;
buf[4] = (profile.crc_enable) ? 0x01 : 0x00;
@@ -212,8 +200,7 @@ void SX126xCommands::encode_packet_params(const RadioProfile &profile, AppliedPr
applied.packet_params_size = 6;
} else {
uint16_t preamble_size = profile.preamble_size * 8;
buf[0] = (preamble_size >> 8) & 0xFF;
buf[1] = (preamble_size >> 0) & 0xFF;
put_be16(buf.data(), preamble_size);
buf[2] = (profile.preamble_detect > 0) ? ((profile.preamble_detect - 1) | 0x04) : 0x00;
buf[3] = profile.sync_size * 8;
buf[4] = 0x00;
@@ -259,20 +246,20 @@ void SX126xCommands::calibrate_image(uint32_t frequency) {
}
}
void SX126xCommands::receive() {
// The IRQ mask also routes to DIO1; DIO2 and DIO3 stay unused.
void SX126xCommands::set_dio_irq_params_(uint16_t irq) {
uint8_t buf[8];
// configure irq params
uint16_t irq = IRQ_RX_DONE | IRQ_RX_TX_TIMEOUT | IRQ_CRC_ERROR;
buf[0] = (irq >> 8) & 0xFF;
buf[1] = (irq >> 0) & 0xFF;
buf[2] = (irq >> 8) & 0xFF;
buf[3] = (irq >> 0) & 0xFF;
buf[4] = (IRQ_RADIO_NONE >> 8) & 0xFF;
buf[5] = (IRQ_RADIO_NONE >> 0) & 0xFF;
buf[6] = (IRQ_RADIO_NONE >> 8) & 0xFF;
buf[7] = (IRQ_RADIO_NONE >> 0) & 0xFF;
put_be16(buf, irq);
put_be16(buf + 2, irq);
put_be16(buf + 4, IRQ_RADIO_NONE);
put_be16(buf + 6, IRQ_RADIO_NONE);
this->write_opcode_(RADIO_SET_DIOIRQPARAMS, buf, 8);
}
void SX126xCommands::receive() {
this->set_dio_irq_params_(IRQ_RX_DONE | IRQ_RX_TX_TIMEOUT | IRQ_CRC_ERROR);
uint8_t buf[3];
// set timeout to 0
buf[0] = 0x00;
@@ -286,24 +273,12 @@ void SX126xCommands::receive() {
}
void SX126xCommands::transmit(uint32_t timeout_ms) {
uint8_t buf[8];
this->set_dio_irq_params_(IRQ_TX_DONE | IRQ_RX_TX_TIMEOUT);
// configure irq params
uint16_t irq = IRQ_TX_DONE | IRQ_RX_TX_TIMEOUT;
buf[0] = (irq >> 8) & 0xFF;
buf[1] = (irq >> 0) & 0xFF;
buf[2] = (irq >> 8) & 0xFF;
buf[3] = (irq >> 0) & 0xFF;
buf[4] = (IRQ_RADIO_NONE >> 8) & 0xFF;
buf[5] = (IRQ_RADIO_NONE >> 0) & 0xFF;
buf[6] = (IRQ_RADIO_NONE >> 8) & 0xFF;
buf[7] = (IRQ_RADIO_NONE >> 0) & 0xFF;
this->write_opcode_(RADIO_SET_DIOIRQPARAMS, buf, 8);
uint8_t buf[3];
const uint32_t ticks = timeout_ms * 64;
buf[0] = ticks >> 16;
buf[1] = ticks >> 8;
buf[2] = ticks;
put_be24(buf, ticks);
this->write_opcode_(RADIO_SET_TX, buf, 3);
}
@@ -41,6 +41,7 @@ class SX126xCommands {
private:
void read_opcode_(uint8_t opcode, uint8_t *data, uint8_t size);
void write_opcode_(uint8_t opcode, const uint8_t *data, uint8_t size);
void set_dio_irq_params_(uint16_t irq);
void read_register_(uint16_t reg, uint8_t *data, uint8_t size);
void write_register_(uint16_t reg, const uint8_t *data, uint8_t size);
static void encode_packet_params(const RadioProfile &profile, AppliedProfile &applied);
+313 -94
View File
@@ -1,12 +1,27 @@
#include "sx127x.h"
#include "esphome/core/application.h"
#include "esphome/core/hal.h"
#include "esphome/core/helpers.h"
#include "esphome/core/log.h"
#include "esphome/core/progmem.h"
#include <cmath>
#include <utility>
namespace esphome::sx127x {
ESPHOME_LOG_TAG(TAG, "sx127x");
static const uint32_t FXOSC = 32000000u;
static constexpr uint32_t TRANSMIT_TIMEOUT_MS = 4000;
static constexpr uint32_t MAX_TRANSMIT_TIMEOUT_MS = 262143; // the same range as SX126x
static constexpr uint32_t MODE_TIMEOUT_MS = 20;
static constexpr uint32_t RECONFIGURE_INTERVAL_ID = 0;
static constexpr uint32_t RECONFIGURE_INTERVAL_MS = 1000;
// Indexed by SX127xError
PROGMEM_STRING_TABLE(SX127xErrorStrings, "none", "timeout", "invalid parameters", "radio not ready", "radio busy",
"cancelled");
static const LogString *set_mode_failure() { return LOG_STR("Set mode failure"); }
static constexpr uint16_t RAMP[16] PROGMEM = {3400, 2000, 1000, 500, 250, 125, 100, 62, 50, 40, 31, 25, 20, 15, 12, 10};
// Aligned 32 bit loads from flash are safe on ESP8266, so this table is read directly
static constexpr uint32_t BW_HZ[22] PROGMEM = {2604, 3125, 3906, 5208, 6250, 7812, 10416, 12500,
@@ -56,26 +71,72 @@ void SX127x::write_fifo_(const uint8_t *data, size_t len) {
this->disable();
}
void IRAM_ATTR SX127x::gpio_intr(SX127x *arg) { arg->enable_loop_soon_any_context(); }
void SX127x::setup() {
// setup reset
this->rst_pin_->setup();
// setup dio0
if (this->dio0_pin_) {
if (this->dio0_pin_ != nullptr) {
this->dio0_pin_->setup();
this->dio0_pin_->attach_interrupt(&SX127x::gpio_intr, this, gpio::INTERRUPT_RISING_EDGE);
if (this->dio0_pin_->is_internal()) {
static_cast<InternalGPIOPin *>(this->dio0_pin_)
->attach_interrupt(&SX127x::gpio_intr, this, gpio::INTERRUPT_RISING_EDGE);
}
}
// start spi
this->spi_setup();
// configure rf
this->configure();
const LogString *reason = this->configure_radio_();
if (reason != nullptr) {
this->mark_failed(reason);
return;
}
this->state_ = RadioState::RADIO_STATE_IDLE;
// Sized once so later packet reads do not reallocate
this->packet_.reserve(this->get_max_packet_size());
}
void SX127x::configure() {
// Nothing to configure before setup() or after a failed setup
if (!this->is_ready()) {
return;
}
if (this->is_transmitting()) {
ESP_LOGW(TAG, "Radio operation in progress, configuration ignored");
return;
}
const LogString *reason = this->configure_radio_();
if (reason != nullptr) {
this->fail_(reason);
return;
}
this->state_ = RadioState::RADIO_STATE_IDLE;
if (this->reconfigure_pending_) {
this->reconfigure_pending_ = false;
this->cancel_interval(RECONFIGURE_INTERVAL_ID);
}
this->status_clear_warning();
this->enable_loop();
}
void SX127x::fail_(const LogString *reason) {
this->state_ = RadioState::RADIO_STATE_NEEDS_CONFIGURATION;
// Nothing else configures the radio again
if (!this->reconfigure_pending_) {
this->reconfigure_pending_ = true;
this->set_interval(RECONFIGURE_INTERVAL_ID, RECONFIGURE_INTERVAL_MS, [this]() { this->configure(); });
}
// Last: a log listener may call configure(), which cancels the retry
this->status_set_warning(reason);
}
const LogString *SX127x::configure_radio_() {
// Until this succeeds; a held packet is discarded
this->state_ = RadioState::RADIO_STATE_NEEDS_CONFIGURATION;
this->applied_ = {this->frequency_, this->payload_length_, this->modulation_, this->packet_mode_, this->rx_start_};
// toggle chip reset
this->rst_pin_->digital_write(false);
delayMicroseconds(1000);
@@ -84,12 +145,13 @@ void SX127x::configure() {
// check silicon version to make sure hw is ok
if (this->read_register_(REG_VERSION) != 0x12) {
this->mark_failed(LOG_STR("Unsupported chip version"));
return;
return LOG_STR("Unsupported chip version");
}
// enter sleep mode
this->set_mode_(MOD_FSK, MODE_SLEEP);
if (!this->set_mode_(MOD_FSK, MODE_SLEEP)) {
return set_mode_failure();
}
// set freq
uint64_t frf = ((uint64_t) this->frequency_ << 19) / FXOSC;
@@ -98,21 +160,27 @@ void SX127x::configure() {
this->write_register_(REG_FRF_LSB, (uint8_t) ((frf >> 0) & 0xFF));
// enter standby mode
this->set_mode_(MOD_FSK, MODE_STDBY);
if (!this->set_mode_(MOD_FSK, MODE_STDBY)) {
return set_mode_failure();
}
// run image cal
this->run_image_cal();
const LogString *reason = this->image_cal_();
if (reason != nullptr) {
return reason;
}
// go back to sleep
this->set_mode_sleep();
if (!this->set_mode_(this->modulation_, MODE_SLEEP)) {
return set_mode_failure();
}
// config pa
if (this->pa_pin_ == PA_PIN_BOOST) {
this->pa_power_ = std::max(this->pa_power_, (uint8_t) 2);
this->pa_power_ = std::min(this->pa_power_, (uint8_t) 17);
this->pa_power_ = clamp(this->pa_power_, (uint8_t) 2, (uint8_t) 17);
this->write_register_(REG_PA_CONFIG, (this->pa_power_ - 2) | this->pa_pin_ | PA_MAX_POWER);
} else {
this->pa_power_ = std::min(this->pa_power_, (uint8_t) 14);
this->pa_power_ = clamp_at_most(this->pa_power_, (uint8_t) 14);
this->write_register_(REG_PA_CONFIG, (this->pa_power_ - 0) | this->pa_pin_ | PA_MAX_POWER);
}
if (this->modulation_ != MOD_LORA) {
@@ -129,11 +197,10 @@ void SX127x::configure() {
}
// switch to rx or sleep
if (this->rx_start_) {
this->set_mode_rx();
} else {
this->set_mode_sleep();
if (!this->restore_()) {
return set_mode_failure();
}
return nullptr;
}
void SX127x::configure_fsk_ook_() {
@@ -252,70 +319,144 @@ void SX127x::configure_lora_() {
}
size_t SX127x::get_max_packet_size() {
if (this->payload_length_ > 0) {
return this->payload_length_;
if (this->applied_.payload_length > 0) {
return this->applied_.payload_length;
}
if (this->modulation_ == MOD_LORA) {
if (this->applied_.modulation == MOD_LORA) {
return SX127X_MAX_PACKET_SIZE;
}
return 64;
}
SX127xError SX127x::transmit_packet(const uint8_t *data, size_t len) {
SX127xError result = this->start_transmit_(nullptr, data, len, TRANSMIT_TIMEOUT_MS);
if (result == SX127xError::SX127X_ERROR_NONE) {
// Without a listener this send cannot be cancelled or replaced
result = SX127xError::SX127X_ERROR_NOT_READY;
while (this->is_transmitting() && !this->poll_transmit_(millis(), result)) {
App.feed_wdt();
}
}
if (result != SX127xError::SX127X_ERROR_NONE) {
ESP_LOGW(
TAG, "Packet send failed: %s",
LOG_STR_ARG(SX127xErrorStrings::get_log_str(static_cast<uint8_t>(result), SX127xErrorStrings::LAST_INDEX)));
}
return result;
}
SX127xError SX127x::start_transmit_async(SX127xTransmitListener *listener, const uint8_t *data, size_t len,
uint32_t timeout_ms) {
if (listener == nullptr || timeout_ms == 0 || timeout_ms > MAX_TRANSMIT_TIMEOUT_MS) {
return SX127xError::SX127X_ERROR_INVALID_PARAMS;
}
return this->start_transmit_(listener, data, len, timeout_ms);
}
SX127xError SX127x::start_transmit_(SX127xTransmitListener *listener, const uint8_t *data, size_t len,
uint32_t timeout_ms) {
// Callers such as a template number's on_value can run before setup() or after a failed setup
if (!this->is_ready()) {
ESP_LOGW(TAG, "Radio not ready, packet dropped");
if (!this->is_ready() || this->state_ == RadioState::RADIO_STATE_NEEDS_CONFIGURATION) {
return SX127xError::SX127X_ERROR_NOT_READY;
}
if (this->payload_length_ > 0 && this->payload_length_ != len) {
ESP_LOGE(TAG, "Packet size does not match config");
return SX127xError::INVALID_PARAMS;
if (this->state_ != RadioState::RADIO_STATE_IDLE) {
return SX127xError::SX127X_ERROR_BUSY;
}
if (len == 0 || len > this->get_max_packet_size()) {
ESP_LOGE(TAG, "Packet size out of range");
return SX127xError::INVALID_PARAMS;
if (data == nullptr || len == 0 || len > this->get_max_packet_size() ||
(this->applied_.payload_length > 0 && this->applied_.payload_length != len)) {
return SX127xError::SX127X_ERROR_INVALID_PARAMS;
}
if (this->dio0_pin_ == nullptr) {
ESP_LOGE(TAG, "DIO0 pin not configured, cannot wait for transmit completion");
return SX127xError::INVALID_PARAMS;
return SX127xError::SX127X_ERROR_INVALID_PARAMS;
}
SX127xError ret = SX127xError::NONE;
if (this->modulation_ == MOD_LORA) {
this->set_mode_standby();
if (this->payload_length_ == 0) {
if (!this->set_mode_(this->applied_.modulation, MODE_STDBY)) {
this->fail_(set_mode_failure());
return SX127xError::SX127X_ERROR_NOT_READY;
}
// A packet received before standby stays in the FIFO for loop()
if (this->packet_waiting_()) {
this->state_ = RadioState::RADIO_STATE_HOLDING_RX;
this->enable_loop();
return SX127xError::SX127X_ERROR_BUSY;
}
if (this->applied_.modulation == MOD_LORA) {
if (this->applied_.payload_length == 0) {
this->write_register_(REG_PAYLOAD_LENGTH, len);
}
this->write_register_(REG_IRQ_FLAGS, 0xFF);
this->write_register_(REG_FIFO_ADDR_PTR, 0);
this->write_fifo_(data, len);
this->set_mode_tx();
} else {
this->set_mode_standby();
if (this->payload_length_ == 0) {
this->write_register_(REG_FIFO, len);
}
this->write_fifo_(data, len);
this->set_mode_tx();
} else if (this->applied_.payload_length == 0) {
this->write_register_(REG_FIFO, len);
}
this->write_fifo_(data, len);
if (!this->enter_tx_()) {
this->fail_(set_mode_failure());
return SX127xError::SX127X_ERROR_NOT_READY;
}
this->tx_ = {listener, millis(), timeout_ms};
this->state_ = RadioState::RADIO_STATE_TRANSMITTING;
this->enable_loop();
return SX127xError::SX127X_ERROR_NONE;
}
// wait until transmit completes, typically the delay will be less than 100 ms
uint32_t start = millis();
while (!this->dio0_pin_->digital_read()) {
App.feed_wdt();
if (millis() - start > 4000) {
ESP_LOGE(TAG, "Transmit packet failure");
ret = SX127xError::TIMEOUT;
break;
}
bool SX127x::poll_transmit_(uint32_t now, SX127xError &result) {
if (!this->is_transmitting()) {
return false;
}
if (this->rx_start_) {
this->set_mode_rx();
// DIO0 is TX_DONE in LoRa, and PacketSent in FSK and OOK while transmitting
if (this->dio0_pin_->digital_read()) {
result = SX127xError::SX127X_ERROR_NONE;
} else if (static_cast<int32_t>(now - this->tx_.started_ms) >= static_cast<int32_t>(this->tx_.timeout_ms)) {
// Signed, as the cached loop time can precede the start
result = SX127xError::SX127X_ERROR_TIMEOUT;
} else {
this->set_mode_sleep();
return false;
}
return ret;
result = this->finish_transmit_(result);
return true;
}
bool SX127x::cancel_transmit_async(SX127xTransmitListener *listener) {
if (!this->is_transmitting() || listener == nullptr || listener != this->tx_.listener) {
return false;
}
this->finish_transmit_(SX127xError::SX127X_ERROR_CANCELLED);
return true;
}
// The only place a send ends; state is committed before the listener runs
SX127xError SX127x::finish_transmit_(SX127xError result) {
SX127xTransmitListener *listener = std::exchange(this->tx_.listener, nullptr);
if (this->restore_()) {
this->state_ = RadioState::RADIO_STATE_IDLE;
} else {
this->fail_(set_mode_failure());
}
if (listener != nullptr) {
listener->on_transmit_complete(result);
}
return result;
}
bool SX127x::restore_() {
// Via standby: the chip ignores RX while a packet is still going out
if (!this->set_mode_(this->applied_.modulation, MODE_STDBY)) {
return false;
}
if (this->applied_.modulation == MOD_LORA) {
this->write_register_(REG_IRQ_FLAGS, 0xFF);
}
return this->applied_.rx_start ? this->enter_rx_() : this->set_mode_(this->applied_.modulation, MODE_SLEEP);
}
bool SX127x::packet_waiting_() {
if (this->applied_.modulation == MOD_LORA) {
const uint8_t flags = this->read_register_(REG_IRQ_FLAGS);
return (flags & RX_DONE) != 0 && (flags & PAYLOAD_CRC_ERROR) == 0;
}
return this->applied_.packet_mode && (this->read_register_(REG_IRQ_FLAGS2) & PAYLOAD_READY) != 0;
}
void SX127x::call_listeners_(const std::vector<uint8_t> &packet, float rssi, float snr) {
@@ -326,43 +467,81 @@ void SX127x::call_listeners_(const std::vector<uint8_t> &packet, float rssi, flo
}
void SX127x::loop() {
this->disable_loop();
if (this->dio0_pin_ == nullptr || !this->dio0_pin_->digital_read()) {
return;
switch (this->state_) {
case RadioState::RADIO_STATE_TRANSMITTING: {
SX127xError result;
this->poll_transmit_(App.get_loop_component_start_time(), result);
return;
}
case RadioState::RADIO_STATE_HOLDING_RX:
this->receive_();
return;
case RadioState::RADIO_STATE_IDLE:
if (this->dio0_pin_ == nullptr) {
this->disable_loop();
return;
}
// DIO0's interrupt wakes the loop
if (this->dio0_pin_->is_internal()) {
this->disable_loop();
}
if (this->dio0_pin_->digital_read()) {
this->receive_();
}
return;
default:
// configure() wakes the loop
this->disable_loop();
return;
}
}
if (this->modulation_ == MOD_LORA) {
void SX127x::receive_() {
const bool holding = this->state_ == RadioState::RADIO_STATE_HOLDING_RX;
bool received = false;
float rssi = 0.0f;
float snr = NAN; // FSK and OOK have no SNR
if (this->applied_.modulation == MOD_LORA) {
uint8_t status = this->read_register_(REG_IRQ_FLAGS);
this->write_register_(REG_IRQ_FLAGS, 0xFF);
if ((status & PAYLOAD_CRC_ERROR) == 0) {
if ((status & RX_DONE) != 0 && (status & PAYLOAD_CRC_ERROR) == 0) {
uint8_t bytes = this->read_register_(REG_NB_RX_BYTES);
uint8_t addr = this->read_register_(REG_FIFO_RX_CURR_ADDR);
uint8_t rssi = this->read_register_(REG_PKT_RSSI_VALUE);
int8_t snr = (int8_t) this->read_register_(REG_PKT_SNR_VALUE);
uint8_t raw_rssi = this->read_register_(REG_PKT_RSSI_VALUE);
int8_t raw_snr = (int8_t) this->read_register_(REG_PKT_SNR_VALUE);
this->packet_.resize(bytes);
this->write_register_(REG_FIFO_ADDR_PTR, addr);
this->read_fifo_(this->packet_);
if (this->frequency_ > 700000000) {
this->call_listeners_(this->packet_, (float) rssi - RSSI_OFFSET_HF, (float) snr / 4);
} else {
this->call_listeners_(this->packet_, (float) rssi - RSSI_OFFSET_LF, (float) snr / 4);
}
rssi = (float) raw_rssi - (this->applied_.frequency > 700000000 ? RSSI_OFFSET_HF : RSSI_OFFSET_LF);
snr = (float) raw_snr / 4;
received = true;
}
} else if (this->packet_mode_) {
uint8_t payload_length = this->payload_length_;
} else if (this->applied_.packet_mode && (this->read_register_(REG_IRQ_FLAGS2) & PAYLOAD_READY) != 0) {
uint8_t payload_length = this->applied_.payload_length;
if (payload_length == 0) {
payload_length = this->read_register_(REG_FIFO);
}
this->packet_.resize(payload_length);
this->read_fifo_(this->packet_);
this->call_listeners_(this->packet_, 0.0f, 0.0f);
received = true;
}
if (holding) {
if (this->restore_()) {
this->state_ = RadioState::RADIO_STATE_IDLE;
} else {
this->fail_(set_mode_failure());
}
}
if (received) {
this->call_listeners_(this->packet_, rssi, snr);
}
}
void SX127x::run_image_cal() {
if (this->modulation_ == MOD_LORA) {
this->set_mode_(MOD_FSK, MODE_SLEEP);
this->set_mode_(MOD_FSK, MODE_STDBY);
const LogString *SX127x::image_cal_() {
if (this->applied_.modulation == MOD_LORA) {
if (!this->set_mode_(MOD_FSK, MODE_SLEEP) || !this->set_mode_(MOD_FSK, MODE_STDBY)) {
return set_mode_failure();
}
}
if (this->auto_cal_) {
this->write_register_(REG_IMAGE_CAL, IMAGE_CAL_START | AUTO_IMAGE_CAL_ON | TEMP_THRESHOLD_10C);
@@ -372,17 +551,29 @@ void SX127x::run_image_cal() {
uint32_t start = millis();
while (this->read_register_(REG_IMAGE_CAL) & IMAGE_CAL_RUNNING) {
if (millis() - start > 20) {
this->mark_failed(LOG_STR("Image cal failure"));
break;
return LOG_STR("Image cal failure");
}
}
if (this->modulation_ == MOD_LORA) {
this->set_mode_(this->modulation_, MODE_SLEEP);
this->set_mode_(this->modulation_, MODE_STDBY);
if (this->applied_.modulation == MOD_LORA) {
if (!this->set_mode_(this->applied_.modulation, MODE_SLEEP) ||
!this->set_mode_(this->applied_.modulation, MODE_STDBY)) {
return set_mode_failure();
}
}
return nullptr;
}
void SX127x::run_image_cal() {
if (!this->can_change_mode_()) {
return;
}
const LogString *reason = this->image_cal_();
if (reason != nullptr) {
this->fail_(reason);
}
}
void SX127x::set_mode_(uint8_t modulation, uint8_t mode) {
bool SX127x::set_mode_(uint8_t modulation, uint8_t mode) {
uint32_t start = millis();
this->write_register_(REG_OP_MODE, modulation | mode);
while (true) {
@@ -391,34 +582,62 @@ void SX127x::set_mode_(uint8_t modulation, uint8_t mode) {
if (mode == MODE_SLEEP) {
this->write_register_(REG_OP_MODE, modulation | mode);
}
break;
return true;
}
if (millis() - start > 20) {
this->mark_failed(LOG_STR("Set mode failure"));
return;
if (millis() - start > MODE_TIMEOUT_MS) {
return false;
}
}
}
void SX127x::set_mode_rx() {
this->set_mode_(this->modulation_, MODE_RX);
if (this->modulation_ == MOD_LORA) {
bool SX127x::enter_rx_() {
if (this->applied_.modulation == MOD_LORA) {
this->write_register_(REG_IRQ_FLAGS_MASK, 0x00);
this->write_register_(REG_DIO_MAPPING1, DIO0_MAPPING_00);
}
return this->set_mode_(this->applied_.modulation, MODE_RX);
}
void SX127x::set_mode_tx() {
this->set_mode_(this->modulation_, MODE_TX);
if (this->modulation_ == MOD_LORA) {
bool SX127x::enter_tx_() {
if (this->applied_.modulation == MOD_LORA) {
this->write_register_(REG_IRQ_FLAGS_MASK, 0x00);
this->write_register_(REG_DIO_MAPPING1, DIO0_MAPPING_01);
}
return this->set_mode_(this->applied_.modulation, MODE_TX);
}
void SX127x::set_mode_standby() { this->set_mode_(this->modulation_, MODE_STDBY); }
bool SX127x::can_change_mode_() {
if (this->state_ == RadioState::RADIO_STATE_IDLE) {
return true;
}
ESP_LOGW(TAG, "Radio unavailable, mode change ignored");
return false;
}
void SX127x::set_mode_sleep() { this->set_mode_(this->modulation_, MODE_SLEEP); }
void SX127x::change_mode_(uint8_t mode) {
if (!this->can_change_mode_()) {
return;
}
bool ok;
if (mode == MODE_RX) {
ok = this->enter_rx_();
} else if (mode == MODE_TX) {
ok = this->enter_tx_();
} else {
ok = this->set_mode_(this->applied_.modulation, mode);
}
if (!ok) {
this->fail_(set_mode_failure());
}
}
void SX127x::set_mode_rx() { this->change_mode_(MODE_RX); }
void SX127x::set_mode_tx() { this->change_mode_(MODE_TX); }
void SX127x::set_mode_standby() { this->change_mode_(MODE_STDBY); }
void SX127x::set_mode_sleep() { this->change_mode_(MODE_SLEEP); }
void SX127x::dump_config() {
ESP_LOGCONFIG(TAG, "SX127x:");
+79 -7
View File
@@ -36,10 +36,35 @@ enum SX127xBw : uint8_t {
SX127X_BW_500_0,
};
enum class SX127xError { NONE = 0, TIMEOUT, INVALID_PARAMS, SX127X_ERROR_NOT_READY };
enum class SX127xError {
SX127X_ERROR_NONE = 0,
SX127X_ERROR_TIMEOUT,
SX127X_ERROR_INVALID_PARAMS,
SX127X_ERROR_NOT_READY,
SX127X_ERROR_BUSY,
SX127X_ERROR_CANCELLED,
};
class SX127xTransmitListener {
public:
virtual void on_transmit_complete(SX127xError result) = 0;
};
// Driver state, not the chip's mode
enum class RadioState : uint8_t {
// Before setup or after a failed mode change; configure() is retried every second
RADIO_STATE_NEEDS_CONFIGURATION,
// Receiving or sleeping as configured, or sending after the TX mode action (untracked)
RADIO_STATE_IDLE,
// A send is on air
RADIO_STATE_TRANSMITTING,
// A send found a received packet waiting; loop() delivers it
RADIO_STATE_HOLDING_RX,
};
class SX127xListener {
public:
// Called from loop(); the packet is only valid until this returns
virtual void on_packet(const std::vector<uint8_t> &packet, float rssi, float snr) = 0;
};
@@ -60,8 +85,9 @@ class SX127x final : public Component,
void set_coding_rate(uint8_t coding_rate) { this->coding_rate_ = coding_rate; }
void set_crc_enable(bool crc_enable) { this->crc_enable_ = crc_enable; }
void set_deviation(uint32_t deviation) { this->deviation_ = deviation; }
void set_dio0_pin(InternalGPIOPin *dio0_pin) { this->dio0_pin_ = dio0_pin; }
void set_dio0_pin(GPIOPin *dio0_pin) { this->dio0_pin_ = dio0_pin; }
void set_frequency(uint32_t frequency) { this->frequency_ = frequency; }
// Ignored unless the radio is idle
void set_mode_rx();
void set_mode_tx();
void set_mode_standby();
@@ -76,26 +102,68 @@ class SX127x final : public Component,
void set_preamble_polarity(uint8_t preamble_polarity) { this->preamble_polarity_ = preamble_polarity; }
void set_preamble_size(uint16_t preamble_size) { this->preamble_size_ = preamble_size; }
void set_preamble_detect(uint8_t preamble_detect) { this->preamble_detect_ = preamble_detect; }
void set_rst_pin(InternalGPIOPin *rst_pin) { this->rst_pin_ = rst_pin; }
void set_rst_pin(GPIOPin *rst_pin) { this->rst_pin_ = rst_pin; }
void set_rx_floor(float floor) { this->rx_floor_ = floor; }
void set_rx_start(bool start) { this->rx_start_ = start; }
void set_shaping(uint8_t shaping) { this->shaping_ = shaping; }
void set_spreading_factor(uint8_t spreading_factor) { this->spreading_factor_ = spreading_factor; }
void set_sync_value(const std::vector<uint8_t> &sync_value) { this->sync_value_ = sync_value; }
void run_image_cal();
// Ignored while a send is on air; discards a held packet
void configure();
SX127xError transmit_packet(const uint8_t *data, size_t len);
SX127xError transmit_packet(const std::vector<uint8_t> &packet) {
return this->transmit_packet(packet.data(), packet.size());
}
// On SX127X_ERROR_NONE, the listener is called once from loop()
[[nodiscard]] SX127xError start_transmit_async(SX127xTransmitListener *listener, const uint8_t *data, size_t len,
uint32_t timeout_ms = 4000);
[[nodiscard]] SX127xError start_transmit_async(SX127xTransmitListener *listener, const std::vector<uint8_t> &packet,
uint32_t timeout_ms = 4000) {
return this->start_transmit_async(listener, packet.data(), packet.size(), timeout_ms);
}
// Completes the send with SX127X_ERROR_CANCELLED; false if the listener has none
bool cancel_transmit_async(SX127xTransmitListener *listener);
bool is_transmitting() const { return this->state_ == RadioState::RADIO_STATE_TRANSMITTING; }
void register_listener(SX127xListener *listener) { this->listeners_.push_back(listener); }
Trigger<std::vector<uint8_t>, float, float> *get_packet_trigger() { return &this->packet_trigger_; }
protected:
static void gpio_intr(SX127x *arg);
// Settings as last applied by configure(), used by sends and reception
struct AppliedSettings {
uint32_t frequency{0};
uint32_t payload_length{0};
uint8_t modulation{0};
bool packet_mode{false};
bool rx_start{false};
};
struct TxOperation {
SX127xTransmitListener *listener{nullptr};
uint32_t started_ms{0};
uint32_t timeout_ms{0};
};
static void IRAM_ATTR gpio_intr(SX127x *arg) { arg->enable_loop_soon_any_context(); }
// Each returns the failure reason, or nullptr
const LogString *configure_radio_();
const LogString *image_cal_();
void configure_fsk_ook_();
void configure_lora_();
void set_mode_(uint8_t modulation, uint8_t mode);
// False if the chip did not enter the mode
bool set_mode_(uint8_t modulation, uint8_t mode);
bool enter_rx_();
bool enter_tx_();
bool can_change_mode_();
void change_mode_(uint8_t mode);
// Puts the radio back in RX or sleep after a send or a held packet
bool restore_();
// Retries configure() every second
void fail_(const LogString *reason);
bool packet_waiting_();
void receive_();
SX127xError start_transmit_(SX127xTransmitListener *listener, const uint8_t *data, size_t len, uint32_t timeout_ms);
// True once the send has ended, with its outcome in result
bool poll_transmit_(uint32_t now, SX127xError &result);
SX127xError finish_transmit_(SX127xError result);
void write_fifo_(const uint8_t *data, size_t len);
void read_fifo_(std::vector<uint8_t> &packet);
void write_register_(uint8_t reg, uint8_t value);
@@ -105,8 +173,10 @@ class SX127x final : public Component,
std::vector<SX127xListener *> listeners_;
std::vector<uint8_t> packet_;
std::vector<uint8_t> sync_value_;
InternalGPIOPin *dio0_pin_{nullptr};
InternalGPIOPin *rst_pin_{nullptr};
AppliedSettings applied_;
TxOperation tx_;
GPIOPin *dio0_pin_{nullptr};
GPIOPin *rst_pin_{nullptr};
SX127xBw afc_bandwidth_{SX127X_BW_50_0};
SX127xBw bandwidth_;
uint32_t bitrate_;
@@ -130,6 +200,8 @@ class SX127x final : public Component,
bool crc_enable_{false};
bool packet_mode_{false};
bool rx_start_{false};
bool reconfigure_pending_{false};
RadioState state_{RadioState::RADIO_STATE_NEEDS_CONFIGURATION};
};
} // namespace esphome::sx127x
+6
View File
@@ -46,6 +46,7 @@ enum SX127xReg : uint8_t {
REG_PAYLOAD_LENGTH_LSB = 0x32,
REG_FIFO_THRESH = 0x35,
REG_IMAGE_CAL = 0x3B,
REG_IRQ_FLAGS2 = 0x3F,
// LoRa registers
REG_FIFO_ADDR_PTR = 0x0D,
REG_FIFO_TX_BASE_ADDR = 0x0E,
@@ -260,6 +261,11 @@ enum SX127xIrqFlags : uint8_t {
CAD_DETECTED = 0x01,
};
enum SX127xIrqFlags2 : uint8_t {
PACKET_SENT = 0x08,
PAYLOAD_READY = 0x04,
};
enum SX127xModemCfg1 : uint8_t {
BW_7_8 = 0x00,
BW_10_4 = 0x10,
@@ -1,6 +1,7 @@
#include "esphome/core/log.h"
#include "esphome/core/application.h"
#include "uart_transport.h"
#include "esphome/core/helpers.h"
namespace esphome::uart {
@@ -23,7 +24,7 @@ void UARTTransport::loop() {
if (this->rx_started_ && this->receive_buffer_.size() > 6) {
auto len = this->receive_buffer_.size();
auto crc = crc16(this->receive_buffer_.data(), len - 2);
if (crc != (this->receive_buffer_[len - 2] | (this->receive_buffer_[len - 1] << 8))) {
if (crc != encode_value_le<uint16_t>(this->receive_buffer_.data() + len - 2)) {
ESP_LOGD(TAG, "CRC mismatch, discarding packet");
this->rx_started_ = false;
this->receive_buffer_.clear();
+4 -4
View File
@@ -45,9 +45,9 @@ void VBus::loop() {
this->buffer_.push_back(c);
if (this->buffer_.size() == 7) {
this->protocol_ = this->buffer_[4];
this->source_ = (this->buffer_[3] << 8) + this->buffer_[2];
this->dest_ = (this->buffer_[1] << 8) + this->buffer_[0];
this->command_ = (this->buffer_[6] << 8) + this->buffer_[5];
this->source_ = encode_value_le<uint16_t>(this->buffer_.data() + 2);
this->dest_ = encode_value_le<uint16_t>(this->buffer_.data());
this->command_ = encode_value_le<uint16_t>(this->buffer_.data() + 5);
}
if ((this->protocol_ == 0x20) && (this->buffer_.size() == 15)) {
this->state_ = 0;
@@ -56,7 +56,7 @@ void VBus::loop() {
return;
}
septet_spread(this->buffer_.data(), 7, 6, this->buffer_[13]);
uint16_t id = (this->buffer_[8] << 8) + this->buffer_[7];
uint16_t id = encode_value_le<uint16_t>(this->buffer_.data() + 7);
uint32_t value = encode_uint32(this->buffer_[12], this->buffer_[11], this->buffer_[10], this->buffer_[9]);
ESP_LOGV(TAG, "P1 C%04x %04x->%04x: %04x %04" PRIx32 " (%" PRIu32 ")", this->command_, this->source_,
this->dest_, id, value, value);
+17 -1
View File
@@ -614,9 +614,25 @@ async def to_code(config: ConfigType) -> None:
def FILTER_SOURCE_FILES() -> list[str]:
"""Filter out web_server_v1.cpp when version is not 1."""
"""Filter out web_server_v1.cpp when version is not 1, the ESP32 event stream elsewhere, and the server on host."""
files_to_filter: list[str] = []
# The host has no web server; it builds only sse_chunk.cpp, for its unit tests
if CORE.is_host:
files_to_filter.extend(
(
"event_source.cpp",
"list_entities.cpp",
"web_server.cpp",
"web_server_v1.cpp",
)
)
return files_to_filter
# The /events implementation is ESP32 only
if not CORE.is_esp32:
files_to_filter.extend(("event_source.cpp", "sse_chunk.cpp"))
# web_server_v1.cpp is only needed when version is 1
config = CORE.config.get(DOMAIN, {})
if config.get(CONF_VERSION, 2) != 1:
@@ -0,0 +1,491 @@
#include "event_source.h"
#if defined(USE_ESP32) && defined(USE_WEBSERVER)
#include <algorithm>
#include <cstring>
#include <iterator>
#include "esphome/core/application.h"
#include "esphome/core/helpers.h"
#include "esphome/core/log.h"
#include "web_server.h"
namespace esphome::web_server {
ESPHOME_LOG_TAG(TAG, "web_server_idf");
// Only prepare_json_() may hold the JSON arena: every other frame in this file is capped below one
// arena. Measured at -Os on GCC 14; newer toolchains stay checked on purpose, older ones skip it.
#if defined(__GNUC__) && !defined(__clang__) && __GNUC__ >= 14 && defined(__OPTIMIZE_SIZE__)
#pragma GCC diagnostic error "-Wstack-usage=2048"
#endif
AsyncEventSource::~AsyncEventSource() {
LockGuard guard{this->pending_mutex_};
for (auto *vec : {&this->sessions_, &this->pending_sessions_}) {
for (auto *ses : *vec) {
delete ses; // NOLINT(cppcoreguidelines-owning-memory)
}
}
}
void AsyncEventSource::handleRequest(AsyncWebServerRequest *request) {
// Httpd task: set up the live httpd_req_t and park the session; main loop does the rest.
// NOLINTNEXTLINE(cppcoreguidelines-owning-memory,clang-analyzer-cplusplus.NewDeleteLeaks)
auto *rsp = new AsyncEventSourceResponse(request, this, this->web_server_);
{
LockGuard guard{this->pending_mutex_};
this->pending_sessions_.push_back(rsp);
this->has_pending_sessions_.store(true, std::memory_order_release);
}
this->web_server_->enable_loop_soon_any_context();
}
// clang-analyzer traces a false-positive leak path from loop() through
// adopt_pending_sessions_main_loop_() into start_session_main_loop_() and
// finally ArduinoJson. Suppress along the entire in-our-code call chain.
// NOLINTBEGIN(clang-analyzer-cplusplus.NewDeleteLeaks)
bool AsyncEventSource::loop() {
// Fast path: one atomic load per tick. Slow path is out-of-line on connect.
if (this->has_pending_sessions_.load(std::memory_order_acquire)) {
this->adopt_pending_sessions_main_loop_();
}
// Clean up dead sessions safely
// This follows the ESP-IDF pattern where free_ctx marks resources as dead
// and the main loop handles the actual cleanup to avoid race conditions
for (size_t i = 0; i < this->sessions_.size();) {
auto *ses = this->sessions_[i];
// If the session has a dead socket (marked by destroy callback)
if (ses->safe_to_delete()) {
// destroy() already logged the close with the fd; don't double-log here.
delete ses; // NOLINT(cppcoreguidelines-owning-memory)
// Remove by swapping with last element (O(1) removal, order doesn't matter for sessions)
this->sessions_[i] = this->sessions_.back();
this->sessions_.pop_back();
} else {
ses->loop();
++i;
}
}
// A session still waiting for httpd to commit its context keeps the loop alive too
return !this->sessions_.empty() || this->has_pending_sessions_.load(std::memory_order_acquire);
}
void AsyncEventSource::adopt_pending_sessions_main_loop_() {
std::vector<AsyncEventSourceResponse *> incoming;
{
LockGuard guard{this->pending_mutex_};
incoming.swap(this->pending_sessions_);
this->has_pending_sessions_.store(false, std::memory_order_relaxed);
}
for (auto *rsp : incoming) {
// Already disconnected? Drop it; skip session start on a dead session.
if (rsp->safe_to_delete()) {
delete rsp; // NOLINT(cppcoreguidelines-owning-memory)
continue;
}
// httpd commits the session context only after the creating handler returns, so stay
// pending until then; httpd_req_cleanup() always commits it or calls destroy()
if (!rsp->is_attached()) {
LockGuard guard{this->pending_mutex_};
this->pending_sessions_.push_back(rsp);
this->has_pending_sessions_.store(true, std::memory_order_release);
continue;
}
this->sessions_.push_back(rsp);
rsp->start_session_main_loop_();
}
}
// NOLINTEND(clang-analyzer-cplusplus.NewDeleteLeaks)
void AsyncEventSource::try_send_nodefer(const char *message, size_t message_len, const char *event, uint32_t id,
uint32_t reconnect) {
for (auto *ses : this->sessions_) {
if (ses->is_connected()) {
ses->try_send_nodefer(message, message_len, event, id, reconnect);
}
}
}
void AsyncEventSource::deferrable_send_state(void *source, bool detail_all, message_generator_t *message_generator) {
// Skip if no connected clients to avoid unnecessary processing
if (this->empty())
return;
for (auto *ses : this->sessions_) {
if (ses->is_connected()) {
ses->deferrable_send_state(source, detail_all, message_generator);
}
}
}
AsyncEventSourceResponse::AsyncEventSourceResponse(AsyncWebServerRequest *request, AsyncEventSource *server,
WebServer *ws)
: server_(server), web_server_(ws), entities_iterator_(ws, this) {
// Httpd task only. start_session_main_loop_() sends the greeting and starts the iterator.
this->begin(request, SSE_STATUS, SSE_CONTENT_TYPE, SSE_HEADERS, SSE_FIRST_CHUNK);
}
static_assert(web_server_base::stream_head_iov_count(std::size(SSE_HEADERS) + WEB_SERVER_DEFAULT_HEADERS_COUNT) <=
web_server_base::STREAM_HEAD_MAX_IOV,
"the event stream head must go out in one write");
// NOLINTBEGIN(clang-analyzer-cplusplus.NewDeleteLeaks) false positive with ArduinoJson
void AsyncEventSourceResponse::start_session_main_loop_() {
auto *ws = this->web_server_;
// The tcp send buffer is empty on connect. A refusal is a closing session or a failed tail
// allocation; nothing retries the greeting, so close and let the client reconnect.
auto message = ws->get_config_json();
if (!this->try_send_nodefer(message.c_str(), message.size(), "ping", millis(), 30000)) {
ESP_LOGW(TAG, "Config not sent to fd %d", this->log_id());
this->close();
return;
}
#ifdef USE_WEBSERVER_SORTING
for (auto &group : ws->sorting_groups_) {
json::JsonBuilder builder;
JsonObject root = builder.root();
root["name"] = json::linked(group.second.name);
root["sorting_weight"] = group.second.weight;
message = builder.serialize();
// a (very) large number of these should be able to be queued initially without defer
// since the only thing in the send buffer at this point is the initial ping/config.
// A refusal means the socket is full or closing; the remaining groups are not sent. The
// session stays up on purpose: partial grouping beats a reconnect loop on a slow link.
if (!this->try_send_nodefer(message.c_str(), message.size(), "sorting_group")) {
ESP_LOGW(TAG, "Sorting groups not sent to fd %d", this->log_id());
break;
}
}
#endif
this->entities_iterator_.begin(ws->include_internal_);
}
// NOLINTEND(clang-analyzer-cplusplus.NewDeleteLeaks)
// helper for allowing only unique entries in the queue
void AsyncEventSourceResponse::deq_push_back_with_dedup_(void *source, message_generator_t *message_generator) {
DeferredEvent item(source, message_generator);
// Use range-based for loop instead of std::find_if to reduce template instantiation overhead and binary size
for (auto &event : this->deferred_queue_) {
if (event == item) {
return; // Already in queue, no need to update since items are equal
}
}
this->deferred_queue_.push_back(item);
}
void AsyncEventSourceResponse::process_deferred_queue_() {
if (this->is_closing()) {
return;
}
while (!deferred_queue_.empty()) {
DeferredEvent &de = deferred_queue_.front();
if (this->send_json_(de.source_, de.message_generator_)) {
if (this->is_closing() || deferred_queue_.empty()) {
return;
}
// O(n) but memory efficiency is more important than speed here which is why std::vector was chosen
deferred_queue_.erase(deferred_queue_.begin());
} else {
break;
}
}
}
void AsyncEventSourceResponse::release_buffers_() {
std::vector<DeferredEvent>().swap(this->deferred_queue_);
this->tail_.reset();
this->tail_cap_ = 0;
this->tail_len_ = 0;
this->tail_sent_ = 0;
}
void AsyncEventSourceResponse::drain_tail_() {
if (this->sending_ || this->is_closing() || this->tail_len_ == 0) {
return;
}
SendGuard guard{*this};
const size_t remaining = this->tail_len_ - this->tail_sent_;
const ssize_t sent = this->write(this->tail_.get() + this->tail_sent_, remaining);
if (sent <= 0) {
return; // write() owns the stall clock and the close
}
this->no_memory_logged_ = false;
this->tail_sent_ += sent;
if (this->tail_sent_ < this->tail_len_) {
ESP_LOGV(TAG, "Partial send: %zd/%zu bytes (total: %u/%u)", sent, remaining, this->tail_sent_, this->tail_len_);
return;
}
// Fully sent; the storage stays for the next stall
this->tail_len_ = 0;
}
bool AsyncEventSourceResponse::reserve_tail_(size_t len) {
if (this->tail_cap_ >= len) {
return true;
}
if (len > TAIL_MAX_SIZE) {
return false;
}
// Nothing is pending while the tail grows, so free the old block first. PREFER_INTERNAL keeps
// the tail where plain new put it.
this->tail_.reset();
this->tail_ = RAMAllocator<uint8_t>(RAMAllocator<uint8_t>::PREFER_INTERNAL).make_unique_array_for_overwrite(len);
this->tail_cap_ = this->tail_ ? len : 0;
return this->tail_cap_ != 0;
}
bool AsyncEventSourceResponse::stash_chunk_(const char *prefix, size_t prefix_len, const char *message,
size_t message_len, size_t total, size_t sent) {
// A log event of nothing but line breaks is the largest chunk; reserve_tail_() must never
// refuse it for size, only for memory
static_assert(SSE_SEP_LEN * LOG_EVENT_MAX_LEN + PREFIX_BUF_SIZE + SSE_SUFFIX_LEN <= TAIL_MAX_SIZE,
"the log cut in web_server.h must keep a worst case log event inside the tail ceiling");
if (!this->reserve_tail_(total)) {
if (sent != 0) {
// Part of the chunk is on the wire, so the stream is broken and the client has to go
ESP_LOGW(TAG, "Cannot buffer a %zu byte chunk, closing", total);
this->close();
} else {
this->tail_alloc_failed_(total); // nothing on the wire, the caller retries on the stall clock
}
return false;
}
uint8_t *dst = this->tail_.get();
std::memcpy(dst, prefix, prefix_len);
dst += prefix_len;
for_each_chunk_piece(
message, message_len,
[](void *ctx, const char *piece, size_t len) {
auto &out = *static_cast<uint8_t **>(ctx);
std::memcpy(out, piece, len);
out += len;
},
&dst);
this->tail_len_ = total;
this->tail_sent_ = sent;
return true;
}
void AsyncEventSourceResponse::loop() {
AsyncStreamResponse::loop();
if (this->is_closing()) {
if (this->tail_cap_ != 0 || this->deferred_queue_.capacity() != 0) {
this->release_buffers_();
}
return;
}
drain_tail_();
process_deferred_queue_();
if (this->is_closing())
return;
// One step per loop; refusals retry next pass
this->entities_iterator_.try_advance(1);
}
#if defined(__GNUC__) && !defined(__clang__) && __GNUC__ >= 14 && defined(__OPTIMIZE_SIZE__)
#pragma GCC diagnostic push
#pragma GCC diagnostic ignored "-Wstack-usage=" // the one frame that holds the JSON arena
#endif
AsyncEventSourceResponse::JsonPrepareResult AsyncEventSourceResponse::prepare_json_(void *source,
message_generator_t *generator,
char *buffer, size_t *size_out) {
// Log lines emitted while the generator runs are dropped for this session: on the large path they would fill
// tail_ before it is overwritten, and on the small path they would go out ahead of the event being built.
SendGuard guard{*this};
json::JsonArena<json::JSON_ARENA_SIZE> arena;
json::JsonBuilder builder(&arena);
generator(this->web_server_, source, builder);
*size_out = builder.serialize_to(buffer, JSON_BUF_SIZE);
if (*size_out < JSON_BUF_SIZE) {
return JsonPrepareResult::JSON_PREPARE_RESULT_BUFFER;
}
char prefix[PREFIX_BUF_SIZE];
const size_t prefix_len = build_chunk_prefix(prefix, sizeof(prefix), "state", 0, 0, true);
// Nothing reaches the wire until the complete document and its framing are owned by the caller.
size_t json_len = 0;
size_t cap = std::max<size_t>(JSON_BUF_SIZE * 2, this->tail_cap_);
for (;;) {
if (!this->reserve_tail_(cap)) {
*size_out = cap;
return JsonPrepareResult::JSON_PREPARE_RESULT_NO_MEMORY;
}
const size_t room = cap - prefix_len - SSE_SUFFIX_LEN;
json_len = builder.serialize_to(reinterpret_cast<char *>(this->tail_.get()) + prefix_len, room);
if (json_len < room) {
break;
}
if (cap >= TAIL_MAX_SIZE) {
*size_out = room;
return JsonPrepareResult::JSON_PREPARE_RESULT_TOO_LARGE;
}
cap = std::min<size_t>(cap * 2, TAIL_MAX_SIZE);
}
const size_t total = prefix_len + json_len + SSE_SUFFIX_LEN;
write_chunk_header(prefix, total - CHUNK_HDR_LEN - CHUNK_END_LEN);
uint8_t *dst = this->tail_.get();
std::memcpy(dst, prefix, prefix_len);
std::memcpy(dst + prefix_len + json_len, SSE_SUFFIX, SSE_SUFFIX_LEN);
this->tail_len_ = total;
this->tail_sent_ = 0;
return JsonPrepareResult::JSON_PREPARE_RESULT_TAIL;
}
#if defined(__GNUC__) && !defined(__clang__) && __GNUC__ >= 14 && defined(__OPTIMIZE_SIZE__)
#pragma GCC diagnostic pop
#endif
bool AsyncEventSourceResponse::send_json_(void *source, message_generator_t *generator) {
if (!this->ready_to_send_()) {
return false;
}
char buffer[JSON_BUF_SIZE];
size_t size;
const auto result = this->prepare_json_(source, generator, buffer, &size);
switch (result) {
case JsonPrepareResult::JSON_PREPARE_RESULT_BUFFER:
return this->try_send_nodefer(buffer, size, "state");
case JsonPrepareResult::JSON_PREPARE_RESULT_TAIL:
this->drain_tail_();
return true;
case JsonPrepareResult::JSON_PREPARE_RESULT_NO_MEMORY: {
SendGuard guard{*this};
this->tail_alloc_failed_(size);
return false;
}
case JsonPrepareResult::JSON_PREPARE_RESULT_TOO_LARGE: {
SendGuard guard{*this};
ESP_LOGW(TAG, "State event does not fit %zu bytes, dropped", size);
this->tail_.reset();
this->tail_cap_ = 0;
this->stall_.clear();
this->no_memory_logged_ = false;
return true;
}
}
return false;
}
void AsyncEventSourceResponse::tail_alloc_failed_(size_t cap) {
// Same stall clock as a socket that stops draining, so a session cannot retry forever. A refused write may
// already have started it, so the warning keeps its own flag.
if (!this->no_memory_logged_) {
this->no_memory_logged_ = true;
ESP_LOGW(TAG, "No memory for a %zu byte chunk", cap);
}
const uint32_t now = App.get_loop_component_start_time();
if (!this->stall_.is_running()) {
this->stall_.start(now);
return;
}
if (this->stall_.is_expired(now, SEND_STALL_TIMEOUT_MS)) {
ESP_LOGW(TAG, "Closing EventSource after %" PRIu32 " ms without memory", this->stall_.elapsed(now));
this->close();
}
}
bool AsyncEventSourceResponse::ready_to_send_() {
if (this->sending_ || !this->is_connected() || this->is_closing()) {
return false;
}
drain_tail_();
return !this->is_closing() && this->tail_len_ == 0;
}
bool AsyncEventSourceResponse::try_send_nodefer(const char *message, size_t message_len, const char *event, uint32_t id,
uint32_t reconnect) {
if (!this->ready_to_send_()) {
return false;
}
SendGuard guard{*this};
// Everything after the prefix goes out straight from the caller's buffer
char prefix[PREFIX_BUF_SIZE];
const size_t prefix_len = build_chunk_prefix(prefix, sizeof(prefix), event, id, reconnect, message != nullptr);
if (message == nullptr && prefix_len == CHUNK_HDR_LEN) {
return true; // Match ESPAsyncWebServer: nothing to send
}
if (prefix_len >= PREFIX_BUF_SIZE - 1) {
// The appenders truncate silently, which would put a malformed event on the wire
ESP_LOGW(TAG, "Event name too long, dropped");
return true;
}
// Gather list: the prefix, then the data lines and their separators from the message
struct Gather {
web_server_base::StreamIovec iov[MAX_SEND_IOV]; // left uninitialized on purpose
int iovcnt{1};
size_t total{0};
bool fits{true};
} g;
g.total = prefix_len;
for_each_chunk_piece(
message, message_len,
[](void *ctx, const char *piece, size_t len) {
auto &g = *static_cast<Gather *>(ctx);
g.total += len;
if (len == 0) {
return;
}
if (g.iovcnt == MAX_SEND_IOV) {
g.fits = false;
return;
}
g.iov[g.iovcnt++] = {const_cast<char *>(piece), len};
},
&g);
// The header and the terminator are not part of the chunk length
write_chunk_header(prefix, g.total - CHUNK_HDR_LEN - CHUNK_END_LEN);
g.iov[0] = {prefix, prefix_len};
// A message with more lines than the list holds skips straight to the tail
const ssize_t sent = g.fits ? this->write(g.iov, g.iovcnt) : 0;
if (sent < 0) {
return false;
}
if (sent > 0) {
this->no_memory_logged_ = false;
}
if (static_cast<size_t>(sent) == g.total) {
return true;
}
// The caller's buffers do not outlive this call, so keep the chunk and continue from loop()
return this->stash_chunk_(prefix, prefix_len, message, message_len, g.total, sent);
}
void AsyncEventSourceResponse::deferrable_send_state(void *source, bool detail_all,
message_generator_t *message_generator) {
// Let the detail_all listing finish before bare state events, so the web UI never shows unnamed entries and
// initial connect sends less
if (!detail_all && !this->entities_iterator_.completed())
return;
drain_tail_();
process_deferred_queue_();
if (this->is_closing()) {
return;
}
if (this->tail_len_ != 0 || !deferred_queue_.empty()) {
// outgoing event buffer or deferred queue still not empty which means downstream tcp send buffer full, no point
// trying to send first
deq_push_back_with_dedup_(source, message_generator);
} else {
// A send error closes the session and clears the queue; nothing is queued after that
if (!this->send_json_(source, message_generator) && !this->is_closing()) {
deq_push_back_with_dedup_(source, message_generator);
}
}
}
} // namespace esphome::web_server
#endif // USE_ESP32 && USE_WEBSERVER
@@ -0,0 +1,176 @@
#pragma once
#include "esphome/core/defines.h"
#if defined(USE_ESP32) && defined(USE_WEBSERVER)
#include "esphome/components/json/json_util.h"
#include "esphome/components/web_server_base/stream_response.h"
#include "esphome/core/helpers.h"
#include "esphome/core/string_ref.h"
#include "list_entities.h"
#include "sse_chunk.h"
#include <atomic>
#include <cstdint>
#include <vector>
namespace esphome::web_server {
class WebServer;
class AsyncEventSource;
class AsyncEventSourceResponse;
using message_generator_t = void(WebServer *, void *, json::JsonBuilder &);
/*
This class holds a pointer to the source component that wants to publish a state event, and a pointer to a function
that will lazily generate that event. The two pointers allow dedup in the deferred queue if multiple publishes for
the same component are backed up, and take up only two pointers of memory. The entry in the deferred queue (a
std::vector) is the DeferredEvent instance itself (not a pointer to one elsewhere in heap) so still only two pointers
per entry (and no heap fragmentation). Even 100 backed up events (you'd have to have at least 100 sensors publishing
because of dedup) would take up only 0.8 kB.
*/
struct DeferredEvent {
friend class AsyncEventSourceResponse;
protected:
void *source_;
message_generator_t *message_generator_;
public:
DeferredEvent(void *source, message_generator_t *message_generator)
: source_(source), message_generator_(message_generator) {}
bool operator==(const DeferredEvent &test) const {
return (source_ == test.source_ && message_generator_ == test.message_generator_);
}
};
static_assert(sizeof(DeferredEvent) == sizeof(void *) + sizeof(message_generator_t *),
"DeferredEvent should have no padding");
class AsyncEventSourceResponse : public web_server_base::AsyncStreamResponse {
friend class AsyncEventSource;
public:
bool try_send_nodefer(const char *message, size_t message_len, const char *event = nullptr, uint32_t id = 0,
uint32_t reconnect = 0);
// detail_all events go out while entities are still being listed; state events wait until listing is done
void deferrable_send_state(void *source, bool detail_all, message_generator_t *message_generator);
// The old event name argument would otherwise convert to true
void deferrable_send_state(void *source, const char *event_type, message_generator_t *message_generator) = delete;
void loop();
protected:
AsyncEventSourceResponse(AsyncWebServerRequest *request, AsyncEventSource *server, WebServer *ws);
// Main-loop only: sends initial ping/config/sorting_groups, starts entity iterator.
void start_session_main_loop_();
void deq_push_back_with_dedup_(void *source, message_generator_t *message_generator);
void process_deferred_queue_();
// A new chunk may go out: not re-entered from a log line, session alive, tail empty
bool ready_to_send_();
// Push what is left of the chunk in tail_ to the socket
void drain_tail_();
// Grow tail_ to hold len bytes, kept at its high-water mark. False on OOM.
bool reserve_tail_(size_t len);
// Keep the whole chunk in tail_ and continue from sent; false when the tail cannot be allocated.
bool stash_chunk_(const char *prefix, size_t prefix_len, const char *message, size_t message_len, size_t total,
size_t sent);
// Send a state event; JSON too large for the stack buffer is serialized into tail_ instead
bool send_json_(void *source, message_generator_t *generator);
enum class JsonPrepareResult : uint8_t {
JSON_PREPARE_RESULT_BUFFER,
JSON_PREPARE_RESULT_TAIL,
JSON_PREPARE_RESULT_NO_MEMORY,
JSON_PREPARE_RESULT_TOO_LARGE,
};
// Keep the arena out of the caller's frame so it is gone before the state send.
// size_out: BUFFER = JSON length, NO_MEMORY = requested tail capacity, TOO_LARGE = available JSON room;
// unused for TAIL.
JsonPrepareResult __attribute__((noinline)) prepare_json_(void *source, message_generator_t *generator, char *buffer,
size_t *size_out);
// Warn once per stall, and close the session once the stall timeout passes with no memory for the tail
void tail_alloc_failed_(size_t cap);
// A log line emitted inside a send re-enters try_send_nodefer on this session; refuse it
struct SendGuard {
AsyncEventSourceResponse &owner;
explicit SendGuard(AsyncEventSourceResponse &owner) : owner(owner) { owner.sending_ = true; }
~SendGuard() { this->owner.sending_ = false; }
};
// Frees the tail and the queue's storage once the session is closing
void release_buffers_();
AsyncEventSource *server_;
std::vector<DeferredEvent> deferred_queue_;
WebServer *web_server_;
ListEntitiesIterator entities_iterator_;
// One chunk the socket did not take whole, allocated on the first stall; the only heap use
// on the send path
RAMUniquePtr<uint8_t[]> tail_;
uint16_t tail_cap_{0};
uint16_t tail_len_{0}; // Zero means nothing pending
uint16_t tail_sent_{0};
bool sending_{false};
// The no-memory warning went out during the current stall; cleared on send progress
bool no_memory_logged_{false};
// The longest multi line log message in the tree (a climate dump_config) has 22 lines; a
// longer one goes through the tail
static constexpr size_t MAX_SEND_LINES = 22;
static constexpr size_t MAX_SEND_IOV = 1 + 2 * MAX_SEND_LINES;
// Chunk header, retry/id/event lines and the first "data: "
static constexpr size_t PREFIX_BUF_SIZE = 128;
// Stack buffer for a state event's JSON; a larger document is serialized into the tail
static constexpr size_t JSON_BUF_SIZE = 1024;
// Same ceiling JsonBuilder::serialize() applies (max_heap_size in json_util.cpp); a larger
// document is dropped before anything is on the wire
static constexpr size_t JSON_MAX_SIZE = 5120;
// Most RAM a stalled session keeps: the largest state document plus any accepted framing
static constexpr size_t TAIL_MAX_SIZE = JSON_MAX_SIZE + PREFIX_BUF_SIZE + SSE_SUFFIX_LEN;
};
class AsyncEventSource : public AsyncWebHandler {
friend class AsyncEventSourceResponse;
public:
AsyncEventSource(StringRef url, WebServer *ws) : url_(url), web_server_(ws) {}
~AsyncEventSource() override;
// NOLINTNEXTLINE(readability-identifier-naming)
bool canHandle(AsyncWebServerRequest *request) const override {
if (request->method() != HTTP_GET)
return false;
char url_buf[AsyncWebServerRequest::URL_BUF_SIZE];
return request->url_to(url_buf) == this->url_;
}
// NOLINTNEXTLINE(readability-identifier-naming)
void handleRequest(AsyncWebServerRequest *request) override;
void try_send_nodefer(const char *message, size_t message_len, const char *event = nullptr, uint32_t id = 0,
uint32_t reconnect = 0);
void deferrable_send_state(void *source, bool detail_all, message_generator_t *message_generator);
// The old event name argument would otherwise convert to true
void deferrable_send_state(void *source, const char *event_type, message_generator_t *message_generator) = delete;
/// Returns true if there are sessions remaining (including pending cleanup).
bool loop();
bool empty() { return this->count() == 0; }
size_t count() const { return this->sessions_.size(); }
protected:
// Cold path: move sessions from pending_sessions_ into sessions_ and greet each one.
void __attribute__((noinline, cold)) adopt_pending_sessions_main_loop_();
StringRef url_; // Must outlive this object (string literal)
// Main-loop only. Vector: SSE sessions are 1-5 connections, linear search beats set.
std::vector<AsyncEventSourceResponse *> sessions_;
// Httpd-task intake; guarded by pending_mutex_, gated by has_pending_sessions_.
std::vector<AsyncEventSourceResponse *> pending_sessions_;
Mutex pending_mutex_;
WebServer *web_server_;
std::atomic<bool> has_pending_sessions_{false};
};
} // namespace esphome::web_server
#endif // USE_ESP32 && USE_WEBSERVER
@@ -8,14 +8,8 @@
namespace esphome::web_server {
#ifdef USE_ESP32
ListEntitiesIterator::ListEntitiesIterator(const WebServer *ws, AsyncEventSource *es) : web_server_(ws), events_(es) {}
#elif USE_ARDUINO
ListEntitiesIterator::ListEntitiesIterator(const WebServer *ws, DeferredUpdateEventSource *es)
: web_server_(ws), events_(es) {}
#endif
void ListEntitiesIterator::send_all_(void *source, message_generator_t *generator) {
this->events_->deferrable_send_state(source, true, generator);
this->session_->deferrable_send_state(source, true, generator);
}
#ifdef USE_BINARY_SENSOR
+9 -18
View File
@@ -1,33 +1,28 @@
#pragma once
#include "esphome/core/defines.h"
#ifdef USE_WEBSERVER
#if defined(USE_WEBSERVER) && (defined(USE_ESP32) || defined(USE_ARDUINO))
#include "esphome/core/component.h"
#include "esphome/core/component_iterator.h"
namespace esphome::json {
class JsonBuilder;
} // namespace esphome::json
namespace esphome::web_server_idf {
#ifdef USE_ESP32
class AsyncEventSource;
#endif
} // namespace esphome::web_server_idf
namespace esphome::web_server {
#if !defined(USE_ESP32) && defined(USE_ARDUINO)
// The /events session the iterator lists entities to
#ifdef USE_ESP32
class AsyncEventSourceResponse;
using EventSession = AsyncEventSourceResponse;
#elif defined(USE_ARDUINO)
class DeferredUpdateEventSource;
using EventSession = DeferredUpdateEventSource;
#endif
class WebServer;
class ListEntitiesIterator final : public ComponentIterator {
public:
#ifdef USE_ESP32
ListEntitiesIterator(const WebServer *ws, esphome::web_server_idf::AsyncEventSource *es);
#elif defined(USE_ARDUINO)
ListEntitiesIterator(const WebServer *ws, DeferredUpdateEventSource *es);
#endif
ListEntitiesIterator(const WebServer *ws, EventSession *session) : web_server_(ws), session_(session) {}
// Entity overrides (generated from entity_types.h).
// Implementations live in list_entities.cpp.
@@ -45,11 +40,7 @@ class ListEntitiesIterator final : public ComponentIterator {
void send_all_(void *source, void (*generator)(WebServer *, void *, json::JsonBuilder &));
const WebServer *web_server_;
#ifdef USE_ESP32
esphome::web_server_idf::AsyncEventSource *events_;
#elif USE_ARDUINO
DeferredUpdateEventSource *events_;
#endif
EventSession *session_;
};
} // namespace esphome::web_server
@@ -5,7 +5,7 @@
#include "esphome/core/helpers.h"
namespace esphome::web_server_idf {
namespace esphome::web_server {
void for_each_chunk_piece(const char *message, size_t message_len, ChunkPieceSink sink, void *ctx) {
if (message == nullptr) {
@@ -63,4 +63,4 @@ void write_chunk_header(char *buf, size_t chunk_len) {
buf[9] = '\n';
}
} // namespace esphome::web_server_idf
} // namespace esphome::web_server
@@ -3,7 +3,9 @@
#include <cstddef>
#include <cstdint>
namespace esphome::web_server_idf {
#include "esphome/components/web_server_base/stream_head.h"
namespace esphome::web_server {
// Wire framing of one Server-Sent-Events event inside the chunked /events response:
// "%08x\r\n" + fields + "data: " line ["\r\ndata: " line]... "\r\n" + "\r\n" + "\r\n"
@@ -22,6 +24,17 @@ constexpr size_t SSE_SEP_LEN = sizeof(SSE_SEP) - 1;
constexpr char SSE_SUFFIX[] = "\r\n\r\n\r\n";
constexpr size_t SSE_SUFFIX_LEN = sizeof(SSE_SUFFIX) - 1;
// Status, Content-Type and headers of the /events response head
constexpr char SSE_STATUS[] = "200 OK";
constexpr char SSE_CONTENT_TYPE[] = "text/event-stream";
constexpr web_server_base::HttpHeader SSE_HEADERS[] = {
{"Transfer-Encoding", "chunked"},
{"Cache-Control", "no-cache"},
{"Connection", "keep-alive"},
};
// Hand-framed first chunk holding an empty event (one blank line), as httpd_resp_send_chunk() sent it before
constexpr char SSE_FIRST_CHUNK[] = "2\r\n\r\n\r\n";
// Receives one piece of the chunk; ctx is whatever the caller passed to for_each_chunk_piece()
using ChunkPieceSink = void (*)(void *ctx, const char *piece, size_t len);
@@ -37,4 +50,4 @@ size_t build_chunk_prefix(char *buf, size_t size, const char *event, uint32_t id
// Fills the header placeholder at the start of buf once the chunk length is known
void write_chunk_header(char *buf, size_t chunk_len);
} // namespace esphome::web_server_idf
} // namespace esphome::web_server
+37 -2
View File
@@ -220,6 +220,34 @@ EntityMatchResult UrlMatch::match_entity(EntityBase *entity) const {
}
#if !defined(USE_ESP32) && defined(USE_ARDUINO)
// BEGIN ESPAsyncWebServer 3.12.1 workaround: remove once a release containing upstream commit
// 880c97068abb0f0e63f803cd3c0cac84fa5cd7bb is out and we bump to it.
// 3.12.1 creates the SSE client before the head is ACKed and credits the head's ACK to its first queued messages,
// dropping or truncating them. Create the client once the head is ACKed instead, as 3.9.6 did.
class HeadAckedEventSourceResponse final : public AsyncEventSourceResponse {
public:
HeadAckedEventSourceResponse(DeferredUpdateEventSourceList *list, WebServer *ws)
: AsyncEventSourceResponse(nullptr), list_(list), ws_(ws) {}
void _respond(AsyncWebServerRequest *request) override {
String head;
this->_assembleHead(head, request->version());
request->client()->write(head.c_str(), this->_headLength);
}
// Every ACK until clientRelease() is the head's, and clientRelease() unhooks this. _state must stay
// RESPONSE_SETUP: past RESPONSE_WAIT_ACK the request would close its released (null) client after this returns.
size_t _ack(AsyncWebServerRequest *request, size_t len, uint32_t time) override {
this->_ackedLength += len;
if (this->_ackedLength >= this->_headLength)
this->list_->add_client_(this->ws_, request->clientRelease());
return 0;
}
private:
DeferredUpdateEventSourceList *list_;
WebServer *ws_;
};
// END ESPAsyncWebServer 3.12.1 workaround
// helper for allowing only unique entries in the queue
void __attribute__((flatten)) DeferredUpdateEventSource::deq_push_back_with_dedup_(
void *source, message_generator_t *message_generator) {
@@ -245,7 +273,7 @@ void DeferredUpdateEventSource::process_deferred_queue_() {
deferred_queue_.erase(deferred_queue_.begin());
this->consecutive_send_failures_ = 0; // Reset failure count on successful send
} else {
// NOTE: Similar logic exists in web_server_idf/web_server_idf.cpp in AsyncEventSourceResponse::drain_tail_().
// NOTE: Similar logic exists in web_server_idf/web_server_idf.cpp in AsyncStreamResponse::write().
// The close mechanisms are platform-specific (this path calls close() directly; the IDF path is time-based and
// closes through HTTPD to preserve session ownership), but both drop a client after roughly 20 seconds without
// send progress. Keep that stall policy in sync when changing either side.
@@ -328,7 +356,13 @@ void DeferredUpdateEventSourceList::try_send_nodefer(const char *message, size_t
}
}
// BEGIN ESPAsyncWebServer 3.12.1 workaround (see HeadAckedEventSourceResponse): once removed, add_new_client creates
// the source as add_client_ does and ends with es->handleRequest(request); add_client_ goes away.
void DeferredUpdateEventSourceList::add_new_client(WebServer *ws, AsyncWebServerRequest *request) {
request->send(new HeadAckedEventSourceResponse(this, ws)); // NOLINT
}
void DeferredUpdateEventSourceList::add_client_(WebServer *ws, AsyncClient *client) {
DeferredUpdateEventSource *es = new DeferredUpdateEventSource(ws, ESPHOME_F("/events"));
this->push_back(es);
@@ -336,9 +370,10 @@ void DeferredUpdateEventSourceList::add_new_client(WebServer *ws, AsyncWebServer
es->onDisconnect([this, es](AsyncEventSourceClient *client) { this->on_client_disconnect_(es); });
es->handleRequest(request);
new AsyncEventSourceClient(client, es); // NOLINT: owned by es
ws->enable_loop_soon_any_context();
}
// END ESPAsyncWebServer 3.12.1 workaround
void DeferredUpdateEventSourceList::on_client_connect_(DeferredUpdateEventSource *source) {
WebServer *ws = source->web_server_;
+8 -1
View File
@@ -5,7 +5,11 @@
#include "esphome/components/json/json_util.h"
#include "esphome/components/web_server_base/web_server_base.h"
#include "esphome/components/web_server_base/captive_dns.h"
#ifdef USE_WEBSERVER
// The host only builds sse_chunk.cpp for unit tests, but other tests can define USE_WEBSERVER there
#if defined(USE_WEBSERVER) && (defined(USE_ESP32) || defined(USE_ARDUINO))
#ifdef USE_ESP32
#include "event_source.h"
#endif
#include "esphome/core/component.h"
#include "esphome/core/entity_base.h"
#include "esphome/core/entity_includes.h"
@@ -171,9 +175,12 @@ class DeferredUpdateEventSource final : public AsyncEventSource {
};
class DeferredUpdateEventSourceList final : public std::list<DeferredUpdateEventSource *> {
friend class HeadAckedEventSourceResponse; // ESPAsyncWebServer 3.12.1 workaround, remove with it
protected:
void on_client_connect_(DeferredUpdateEventSource *source);
void on_client_disconnect_(DeferredUpdateEventSource *source);
void add_client_(WebServer *ws, AsyncClient *client); // ESPAsyncWebServer 3.12.1 workaround, remove with it
public:
/// Returns true if there are event sources remaining (including pending cleanup).
@@ -105,9 +105,9 @@ async def to_code(config: ConfigType) -> None:
var = cg.new_Pvariable(config[CONF_ID])
cg.add(cg.RawExpression(f"{web_server_base_ns}::global_web_server_base = {var}"))
# Default headers added in init(): web_server_idf's StaticVector, or the library's DefaultHeaders
cg.add_define("WEB_SERVER_DEFAULT_HEADERS_COUNT", 1)
if CORE.is_esp32:
# Count for StaticVector in web_server_idf - matches headers added in init()
cg.add_define("WEB_SERVER_DEFAULT_HEADERS_COUNT", 1)
return
# ESP32 uses IDF web server (early return above), so this is for other Arduino platforms
@@ -132,7 +132,7 @@ async def to_code(config: ConfigType) -> None:
)
cg.add_platformio_option("extra_scripts", ["pre:fix_rp2040_hash.py"])
# https://github.com/ESP32Async/ESPAsyncWebServer/blob/main/library.json
cg.add_library("ESP32Async/ESPAsyncWebServer", "3.9.6")
cg.add_library("ESP32Async/ESPAsyncWebServer", "3.12.1")
# The DNS server used for captive portals on ESP32; other platforms use the Arduino
@@ -0,0 +1,126 @@
#pragma once
#include <cstddef>
#include <cstdint>
#include <cstring>
#include <span>
#include <strings.h>
// Pure helpers of AsyncStreamResponse, kept free of httpd and lwIP so they build on the host.
// The iovec type is a template parameter: lwIP defines its own on the device, POSIX on the host.
namespace esphome::web_server_base {
struct HttpHeader {
const char *name;
const char *value;
};
/// Drops the first sent bytes from a gather list after a write; a short write resumes inside the
/// element it stopped in. count reaches 0 once everything went out.
template<typename Iovec> void advance_iovecs(Iovec *&iov, int &count, size_t sent) {
while (count > 0 && sent >= iov->iov_len) {
sent -= iov->iov_len;
++iov;
--count;
}
if (count > 0) {
iov->iov_base = static_cast<char *>(iov->iov_base) + sent;
iov->iov_len -= sent;
}
}
/// Pieces in a head: 4 for the status line and Content-Type, 4 per header, the blank line and the body
constexpr size_t stream_head_iov_count(size_t header_count) { return 4 + 4 * header_count + 2; }
/// Pieces per batch: a head with up to four headers goes out in one write; 192 bytes on 32-bit
static constexpr int STREAM_HEAD_MAX_IOV = 24;
/// True if headers holds a header named name, compared case insensitively
inline bool has_header(std::span<const HttpHeader> headers, const char *name) {
for (const auto &header : headers) {
if (strcasecmp(header.name, name) == 0) {
return true;
}
}
return false;
}
/// Gathers the head of a stream response: "HTTP/1.1 <status>", Content-Type, headers, the default_headers that
/// headers does not replace, the blank line and body (may be null). send(Iovec *iov, int count) gets each full batch of
/// STREAM_HEAD_MAX_IOV pieces and then the rest; once it returns false the later pieces are skipped. Returns false if
/// any send failed.
template<typename Iovec, typename Send>
bool write_stream_head(Send &&send, const char *status, const char *content_type, std::span<const HttpHeader> headers,
std::span<const HttpHeader> default_headers, const char *body) {
struct Batch {
Send &send;
int count{0};
bool ok{true};
Iovec iov[STREAM_HEAD_MAX_IOV]; // left uninitialized on purpose
void flush() {
if (this->ok && this->count > 0) {
this->ok = this->send(this->iov, this->count);
}
this->count = 0;
}
void add(const char *buf, size_t len) {
if (this->count == STREAM_HEAD_MAX_IOV) {
this->flush();
}
this->iov[this->count++] = {const_cast<char *>(buf), len};
}
void add(const char *str) { this->add(str, strlen(str)); }
void add_header(const HttpHeader &header) {
this->add("\r\n", 2);
this->add(header.name);
this->add(": ", 2);
this->add(header.value);
}
} batch{send};
batch.add("HTTP/1.1 ", 9);
batch.add(status);
batch.add("\r\nContent-Type: ", 16);
batch.add(content_type);
for (const auto &header : headers) {
batch.add_header(header);
}
for (const auto &header : default_headers) {
if (!has_header(headers, header.name)) {
batch.add_header(header);
}
}
batch.add("\r\n\r\n", 4);
if (body != nullptr) {
batch.add(body);
}
batch.flush();
return batch.ok;
}
/// How long sends have made no progress. Zero means not running, so a start at millis() == 0 records 1.
/// The comparison is wraparound safe across a millis() rollover.
class StallClock {
public:
/// Starts the clock; a running clock keeps its start time.
void start(uint32_t now) {
if (this->started_ms_ == 0) {
this->started_ms_ = now != 0 ? now : 1;
}
}
void clear() { this->started_ms_ = 0; }
bool is_running() const { return this->started_ms_ != 0; }
/// True once the clock has run for timeout_ms
bool is_expired(uint32_t now, uint32_t timeout_ms) const {
return this->started_ms_ != 0 && static_cast<int32_t>(now - (this->started_ms_ + timeout_ms)) >= 0;
}
uint32_t elapsed(uint32_t now) const { return now - this->started_ms_; }
protected:
uint32_t started_ms_{0};
};
} // namespace esphome::web_server_base
@@ -0,0 +1,16 @@
#pragma once
#include "esphome/core/defines.h"
// The long lived stream response the /events layer writes to, for the platform's web server
#ifdef USE_ESP32
#include "esphome/components/web_server_idf/web_server_idf.h"
#include <sys/socket.h>
namespace esphome::web_server_base {
using web_server_idf::AsyncStreamResponse;
// One entry of the gather list AsyncStreamResponse::write() takes
using StreamIovec = ::iovec;
} // namespace esphome::web_server_base
#endif
@@ -17,7 +17,6 @@
#include <freertos/task.h>
#include "utils.h"
#include "sse_chunk.h"
#include "web_server_idf.h"
#ifdef USE_WEBSERVER_AUTH_DIGEST
@@ -30,17 +29,15 @@
#include "multipart.h" // For parse_multipart_boundary and other utils
#endif
#ifdef USE_WEBSERVER
#include "esphome/components/web_server/web_server.h"
#include "esphome/components/web_server/list_entities.h"
#endif // USE_WEBSERVER
// Include socket headers after Arduino headers to avoid IPADDR_NONE/INADDR_NONE macro conflicts
#include <cerrno>
#include <sys/socket.h>
namespace esphome::web_server_idf {
using web_server_base::advance_iovecs;
using web_server_base::write_stream_head;
// Status strings not provided by esp_http_server.h
#ifndef HTTPD_401
#define HTTPD_401 "401 Unauthorized"
@@ -55,13 +52,10 @@ namespace esphome::web_server_idf {
#define HTTPD_422 "422 Unprocessable Entity"
#endif
#define CRLF_STR "\r\n"
#define CRLF_LEN (sizeof(CRLF_STR) - 1)
ESPHOME_LOG_TAG(TAG, "web_server_idf");
// Only prepare_json_() may hold the JSON arena: every other frame in this file is capped below one
// arena. Measured at -Os on GCC 14; newer toolchains stay checked on purpose, older ones skip it.
// Every frame in this file is capped below one JSON arena.
// Measured at -Os on GCC 14; newer toolchains stay checked on purpose, older ones skip it.
#if defined(__GNUC__) && !defined(__clang__) && __GNUC__ >= 14 && defined(__OPTIMIZE_SIZE__)
#pragma GCC diagnostic error "-Wstack-usage=2048"
#endif
@@ -140,6 +134,8 @@ void AsyncWebServer::safe_close_with_shutdown(httpd_handle_t hd, int sockfd) {
//
// See: https://github.com/esphome/esphome-webserver/issues/163
AsyncStreamResponse::socket_closing(hd, sockfd);
// Attempt shutdown - ignore errors as socket may already be closed
shutdown(sockfd, SHUT_RD);
@@ -736,224 +732,84 @@ void AsyncResponseStream::printf(const char *fmt, ...) {
this->print(str);
}
#ifdef USE_WEBSERVER
AsyncEventSource::~AsyncEventSource() {
LockGuard guard{this->pending_mutex_};
for (auto *vec : {&this->sessions_, &this->pending_sessions_}) {
for (auto *ses : *vec) {
delete ses; // NOLINT(cppcoreguidelines-owning-memory)
}
}
}
void AsyncEventSource::handleRequest(AsyncWebServerRequest *request) {
// Httpd task: set up the live httpd_req_t and park the session; main loop does the rest.
// NOLINTNEXTLINE(cppcoreguidelines-owning-memory,clang-analyzer-cplusplus.NewDeleteLeaks)
auto *rsp = new AsyncEventSourceResponse(request, this, this->web_server_);
{
LockGuard guard{this->pending_mutex_};
this->pending_sessions_.push_back(rsp);
this->has_pending_sessions_.store(true, std::memory_order_release);
}
this->web_server_->enable_loop_soon_any_context();
}
// clang-analyzer traces a false-positive leak path from loop() through
// adopt_pending_sessions_main_loop_() into start_session_main_loop_() and
// finally ArduinoJson. Suppress along the entire in-our-code call chain.
// NOLINTBEGIN(clang-analyzer-cplusplus.NewDeleteLeaks)
bool AsyncEventSource::loop() {
// Fast path: one atomic load per tick. Slow path is out-of-line on connect.
if (this->has_pending_sessions_.load(std::memory_order_acquire)) {
this->adopt_pending_sessions_main_loop_();
}
// Clean up dead sessions safely
// This follows the ESP-IDF pattern where free_ctx marks resources as dead
// and the main loop handles the actual cleanup to avoid race conditions
for (size_t i = 0; i < this->sessions_.size();) {
auto *ses = this->sessions_[i];
// If the session has a dead socket (marked by destroy callback)
if (ses->safe_to_delete_()) {
// destroy() already logged the close with the fd; don't double-log here.
delete ses; // NOLINT(cppcoreguidelines-owning-memory)
// Remove by swapping with last element (O(1) removal, order doesn't matter for sessions)
this->sessions_[i] = this->sessions_.back();
this->sessions_.pop_back();
} else {
ses->loop();
++i;
}
}
// A session still waiting for httpd to commit its context keeps the loop alive too
return !this->sessions_.empty() || this->has_pending_sessions_.load(std::memory_order_acquire);
}
void AsyncEventSource::adopt_pending_sessions_main_loop_() {
std::vector<AsyncEventSourceResponse *> incoming;
{
LockGuard guard{this->pending_mutex_};
incoming.swap(this->pending_sessions_);
this->has_pending_sessions_.store(false, std::memory_order_relaxed);
}
for (auto *rsp : incoming) {
// Already disconnected? Drop it; skip session start on a dead session.
if (rsp->safe_to_delete_()) {
delete rsp; // NOLINT(cppcoreguidelines-owning-memory)
continue;
}
// httpd commits the session context only after the creating handler returns, so stay
// pending until then; httpd_req_cleanup() always commits it or calls destroy()
if (httpd_sess_get_ctx(rsp->hd_, rsp->fd_.load()) != rsp) {
LockGuard guard{this->pending_mutex_};
this->pending_sessions_.push_back(rsp);
this->has_pending_sessions_.store(true, std::memory_order_release);
continue;
}
this->sessions_.push_back(rsp);
rsp->start_session_main_loop_();
}
}
// NOLINTEND(clang-analyzer-cplusplus.NewDeleteLeaks)
void AsyncEventSource::try_send_nodefer(const char *message, size_t message_len, const char *event, uint32_t id,
uint32_t reconnect) {
for (auto *ses : this->sessions_) {
if (ses->fd_.load() != 0) { // Skip dead sessions
ses->try_send_nodefer(message, message_len, event, id, reconnect);
}
}
}
void AsyncEventSource::deferrable_send_state(void *source, bool detail_all, message_generator_t *message_generator) {
// Skip if no connected clients to avoid unnecessary processing
if (this->empty())
return;
for (auto *ses : this->sessions_) {
if (ses->fd_.load() != 0) { // Skip dead sessions
ses->deferrable_send_state(source, detail_all, message_generator);
}
}
}
AsyncEventSourceResponse::AsyncEventSourceResponse(const AsyncWebServerRequest *request,
esphome::web_server_idf::AsyncEventSource *server,
esphome::web_server::WebServer *ws)
: server_(server), web_server_(ws), entities_iterator_(ws, server) {
// Httpd task only. start_session_main_loop_() sends the greeting and starts the iterator.
void AsyncStreamResponse::begin(AsyncWebServerRequest *request, const char *status, const char *content_type,
std::span<const HttpHeader> headers, const char *body) {
httpd_req_t *req = *request;
httpd_resp_set_status(req, HTTPD_200);
httpd_resp_set_type(req, "text/event-stream");
httpd_resp_set_hdr(req, "Cache-Control", "no-cache");
httpd_resp_set_hdr(req, "Connection", "keep-alive");
DefaultHeaders::Instance().apply_(req);
httpd_resp_send_chunk(req, CRLF_STR, CRLF_LEN);
// Same head httpd_resp_send_chunk() writes, without its first chunk. Blocking sends: the session's non-blocking
// send override is not installed yet.
const int fd = httpd_req_to_sockfd(req);
const auto &defaults = DefaultHeaders::Instance().headers_;
const bool head_sent = write_stream_head<struct iovec>(
[fd](struct iovec *iov, int count) {
while (count > 0) {
struct msghdr msg{};
msg.msg_iov = iov;
msg.msg_iovlen = count;
const ssize_t sent = sendmsg(fd, &msg, 0);
if (sent <= 0) {
return false;
}
advance_iovecs(iov, count, sent);
}
return true;
},
status, content_type, headers, std::span<const HttpHeader>(defaults.data(), defaults.size()), body);
req->sess_ctx = this;
req->free_ctx = AsyncEventSourceResponse::destroy;
req->free_ctx = AsyncStreamResponse::destroy;
this->hd_ = req->handle;
this->fd_.store(httpd_req_to_sockfd(req));
this->fd_.store(fd);
// Use non-blocking send to prevent watchdog timeouts when TCP buffers are full
httpd_sess_set_send_override(this->hd_, this->fd_.load(), nonblocking_send);
httpd_sess_set_send_override(this->hd_, fd, nonblocking_send);
if (!head_sent) {
// A partly sent head must not be continued by later writes. close() is main loop only, so shut the socket
// down here; the first write() then fails and closes the stream, and httpd frees the session as usual.
shutdown(fd, SHUT_RDWR);
}
}
// NOLINTBEGIN(clang-analyzer-cplusplus.NewDeleteLeaks) false positive with ArduinoJson
void AsyncEventSourceResponse::start_session_main_loop_() {
auto *ws = this->web_server_;
// Held around a stream's ownership check and send, and while its socket is marked closed.
// Created at boot, so no allocation can fail once requests arrive.
static Mutex stream_send_mutex; // NOLINT(cppcoreguidelines-avoid-non-const-global-variables)
// The tcp send buffer is empty on connect. A refusal is a closing session or a failed tail
// allocation; nothing retries the greeting, so close and let the client reconnect.
auto message = ws->get_config_json();
if (!this->try_send_nodefer(message.c_str(), message.size(), "ping", millis(), 30000)) {
ESP_LOGW(TAG, "Config not sent to fd %d", this->fd_.load());
this->request_close_();
void AsyncStreamResponse::socket_closing(httpd_handle_t hd, int fd) {
// Only streams set a session context
auto *rsp = static_cast<AsyncStreamResponse *>(httpd_sess_get_ctx(hd, fd));
if (rsp == nullptr)
return;
}
#ifdef USE_WEBSERVER_SORTING
for (auto &group : ws->sorting_groups_) {
json::JsonBuilder builder;
JsonObject root = builder.root();
root["name"] = json::linked(group.second.name);
root["sorting_weight"] = group.second.weight;
message = builder.serialize();
// a (very) large number of these should be able to be queued initially without defer
// since the only thing in the send buffer at this point is the initial ping/config.
// A refusal means the socket is full or closing; the remaining groups are not sent. The
// session stays up on purpose: partial grouping beats a reconnect loop on a slow link.
if (!this->try_send_nodefer(message.c_str(), message.size(), "sorting_group")) {
ESP_LOGW(TAG, "Sorting groups not sent to fd %d", this->fd_.load());
break;
}
}
#endif
this->entities_iterator_.begin(ws->include_internal_);
LockGuard guard{stream_send_mutex};
rsp->socket_closed_ = true;
}
// NOLINTEND(clang-analyzer-cplusplus.NewDeleteLeaks)
void AsyncEventSourceResponse::destroy(void *ptr) {
auto *rsp = static_cast<AsyncEventSourceResponse *>(ptr);
void AsyncStreamResponse::destroy(void *ptr) {
auto *rsp = static_cast<AsyncStreamResponse *>(ptr);
int fd = rsp->fd_.exchange(0); // Atomically get and clear fd
ESP_LOGD(TAG, "Event source connection closed (fd: %d)", fd);
// Mark as dead - will be cleaned up in the main loop
// Note: We don't delete or remove from set here to avoid race conditions
ESP_LOGD(TAG, "Stream connection closed (fd: %d)", fd);
// Mark as dead - the owner deletes it from the main loop once safe_to_delete().
// httpd will call our custom close_fn (safe_close_with_shutdown) which handles
// shutdown() before close() to prevent lwIP race conditions
}
// helper for allowing only unique entries in the queue
void AsyncEventSourceResponse::deq_push_back_with_dedup_(void *source, message_generator_t *message_generator) {
DeferredEvent item(source, message_generator);
// Use range-based for loop instead of std::find_if to reduce template instantiation overhead and binary size
for (auto &event : this->deferred_queue_) {
if (event == item) {
return; // Already in queue, no need to update since items are equal
}
}
this->deferred_queue_.push_back(item);
}
void AsyncEventSourceResponse::process_deferred_queue_() {
if (this->close_requested_) {
return;
}
while (!deferred_queue_.empty()) {
DeferredEvent &de = deferred_queue_.front();
if (this->send_json_(de.source_, de.message_generator_)) {
if (this->close_requested_ || deferred_queue_.empty()) {
return;
}
// O(n) but memory efficiency is more important than speed here which is why std::vector was chosen
deferred_queue_.erase(deferred_queue_.begin());
} else {
break;
}
}
}
void AsyncEventSourceResponse::request_close_() {
void AsyncStreamResponse::close() {
if (!this->close_requested_) {
this->close_requested_ = true;
this->deferred_queue_.clear();
this->tail_.reset();
this->tail_cap_ = 0;
this->tail_len_ = 0;
this->tail_sent_ = 0;
this->next_close_attempt_ms_ = App.get_loop_component_start_time();
}
this->process_close_();
}
void AsyncEventSourceResponse::process_close_() {
void AsyncStreamResponse::loop() {
if (this->close_requested_) {
this->process_close_();
} else {
this->refresh_lru_();
}
}
void AsyncStreamResponse::process_close_() {
if (!this->close_requested_ || this->close_work_queued_.load(std::memory_order_acquire)) {
return;
}
@@ -971,7 +827,7 @@ void AsyncEventSourceResponse::process_close_() {
// httpd_sess_trigger_close() queues only a reusable fd/session slot and can
// therefore close a new client if the original peer disconnects meanwhile.
this->close_work_queued_.store(true, std::memory_order_release);
const esp_err_t err = httpd_queue_work(this->hd_, &AsyncEventSourceResponse::close_session_work, this);
const esp_err_t err = httpd_queue_work(this->hd_, &AsyncStreamResponse::close_session_work, this);
this->next_close_attempt_ms_ = now + (err == ESP_OK ? CLOSE_CONFIRM_INTERVAL_MS : CLOSE_RETRY_INTERVAL_MS);
if (err == ESP_OK) {
return;
@@ -979,17 +835,17 @@ void AsyncEventSourceResponse::process_close_() {
this->close_work_queued_.store(false, std::memory_order_release);
if (!this->close_retry_warning_logged_) {
ESP_LOGW(TAG, "Failed to queue EventSource close (%s); retrying", esp_err_to_name(err));
ESP_LOGW(TAG, "Failed to queue stream close (%s); retrying", esp_err_to_name(err));
this->close_retry_warning_logged_ = true;
}
}
void AsyncEventSourceResponse::refresh_lru_() {
// httpd only advances a session's LRU counter when a request arrives, and an event stream receives one
void AsyncStreamResponse::refresh_lru_() {
// httpd only advances a session's LRU counter when a request arrives, and a stream receives one
// request ever, so the purge that frees a socket for a new client would evict the live streams first.
// The HTTPD task owns the session table, so the refresh runs there.
// A stalled stream is the socket the purge should take, and a zeroed fd has no session left to refresh.
if (this->send_failure_started_ms_ != 0 || this->fd_.load() == 0)
if (this->stall_.is_running() || this->fd_.load() == 0)
return;
const uint32_t now = App.get_loop_component_start_time();
if (now - this->last_lru_refresh_ms_ < LRU_REFRESH_INTERVAL_MS ||
@@ -998,13 +854,13 @@ void AsyncEventSourceResponse::refresh_lru_() {
}
this->last_lru_refresh_ms_ = now; // a full queue also waits for the next interval
this->lru_work_queued_.store(true, std::memory_order_release);
if (httpd_queue_work(this->hd_, &AsyncEventSourceResponse::refresh_lru_work, this) != ESP_OK) {
if (httpd_queue_work(this->hd_, &AsyncStreamResponse::refresh_lru_work, this) != ESP_OK) {
this->lru_work_queued_.store(false, std::memory_order_release);
}
}
void AsyncEventSourceResponse::refresh_lru_work(void *arg) {
auto *response = static_cast<AsyncEventSourceResponse *>(arg);
void AsyncStreamResponse::refresh_lru_work(void *arg) {
auto *response = static_cast<AsyncStreamResponse *>(arg);
const int fd = response->fd_.load();
if (response->owns_session_(fd)) {
httpd_sess_update_lru_counter(response->hd_, fd);
@@ -1013,8 +869,8 @@ void AsyncEventSourceResponse::refresh_lru_work(void *arg) {
response->lru_work_queued_.store(false, std::memory_order_release);
}
void AsyncEventSourceResponse::close_session_work(void *arg) {
auto *response = static_cast<AsyncEventSourceResponse *>(arg);
void AsyncStreamResponse::close_session_work(void *arg) {
auto *response = static_cast<AsyncStreamResponse *>(arg);
const int fd = response->fd_.load();
if (response->owns_session_(fd)) {
// The HTTPD task remains the session owner. Shutting the socket down makes
@@ -1026,320 +882,61 @@ void AsyncEventSourceResponse::close_session_work(void *arg) {
response->close_work_queued_.store(false, std::memory_order_release);
}
ssize_t AsyncEventSourceResponse::send_(struct iovec *iov, int iovcnt) {
// httpd frees a session before closing its socket, so the fd may already be a new client's.
ssize_t AsyncStreamResponse::write(struct iovec *iov, int iovcnt) {
if (this->close_requested_) {
return -1;
}
// Nothing to send is not a stall
size_t total = 0;
for (int i = 0; i < iovcnt; i++) {
total += iov[i].iov_len;
}
if (total == 0) {
return 0;
}
// httpd closes the socket before it clears the session, so the fd number may already be a new client's.
// Treated as would-block; the stall timer ends a session that never becomes ours again.
const int fd = this->fd_.load();
if (!this->owns_session_(fd)) {
return 0;
ssize_t sent = 0;
int err = 0;
{
// Held across the check and the send, so socket_closing() cannot close the socket in between
LockGuard guard{stream_send_mutex};
if (!this->socket_closed_ && this->owns_session_(fd)) {
struct msghdr msg{};
msg.msg_iov = iov;
msg.msg_iovlen = iovcnt;
sent = sendmsg(fd, &msg, MSG_DONTWAIT);
err = errno;
}
}
struct msghdr msg{};
msg.msg_iov = iov;
msg.msg_iovlen = iovcnt;
const ssize_t sent = sendmsg(fd, &msg, MSG_DONTWAIT);
if (sent >= 0) {
if (sent > 0) {
this->stall_.clear();
return sent;
}
const int err = errno;
if (err == EAGAIN || err == EWOULDBLOCK) {
return 0;
if (sent < 0 && err != EAGAIN && err != EWOULDBLOCK) {
ESP_LOGD(TAG, "send error: errno %d", err);
// A hard error is not a timeout, so is_stalled() must not report one
this->stall_.clear();
this->close();
return -1;
}
ESP_LOGD(TAG, "send error: errno %d", err);
this->request_close_();
return -1;
}
void AsyncEventSourceResponse::drain_tail_() {
if (this->sending_ || this->close_requested_ || this->tail_len_ == 0) {
return;
}
SendGuard guard{*this};
const size_t remaining = this->tail_len_ - this->tail_sent_;
struct iovec iov = {this->tail_.get() + this->tail_sent_, remaining};
const ssize_t sent = this->send_(&iov, 1);
if (sent < 0) {
return;
}
if (sent == 0) {
// Socket buffer full, try again later
// NOTE: Similar logic exists in web_server/web_server.cpp in DeferredUpdateEventSource::process_deferred_queue_().
// The IDF path is intentionally time-based and closes through HTTPD to preserve session ownership.
const uint32_t now = App.get_loop_component_start_time();
if (this->send_failure_started_ms_ == 0) {
this->send_failure_started_ms_ = now != 0 ? now : 1; // Reserve zero for no stall.
}
if (static_cast<int32_t>(now - (this->send_failure_started_ms_ + SEND_STALL_TIMEOUT_MS)) >= 0) {
ESP_LOGW(TAG, "Closing stuck EventSource connection after %" PRIu32 " ms without send progress",
now - this->send_failure_started_ms_);
this->request_close_();
}
return;
}
this->send_failure_started_ms_ = 0;
this->tail_sent_ += sent;
if (this->tail_sent_ < this->tail_len_) {
ESP_LOGV(TAG, "Partial send: %zd/%zu bytes (total: %u/%u)", sent, remaining, this->tail_sent_, this->tail_len_);
return;
}
// Fully sent; the storage stays for the next stall
this->tail_len_ = 0;
}
bool AsyncEventSourceResponse::reserve_tail_(size_t len) {
if (this->tail_cap_ >= len) {
return true;
}
if (len > TAIL_MAX_SIZE) {
return false;
}
// Nothing is pending while the tail grows, so free the old block first. PREFER_INTERNAL keeps
// the tail where plain new put it.
this->tail_.reset();
this->tail_ = RAMAllocator<uint8_t>(RAMAllocator<uint8_t>::PREFER_INTERNAL).make_unique_array_for_overwrite(len);
this->tail_cap_ = this->tail_ ? len : 0;
return this->tail_cap_ != 0;
}
bool AsyncEventSourceResponse::stash_chunk_(const char *prefix, size_t prefix_len, const char *message,
size_t message_len, size_t total, size_t sent) {
// A log event of nothing but line breaks is the largest chunk; reserve_tail_() must never
// refuse it for size, only for memory
static_assert(SSE_SEP_LEN * web_server::LOG_EVENT_MAX_LEN + PREFIX_BUF_SIZE + SSE_SUFFIX_LEN <= TAIL_MAX_SIZE,
"the log cut in web_server.h must keep a worst case log event inside the tail ceiling");
if (!this->reserve_tail_(total)) {
if (sent != 0) {
// Part of the chunk is on the wire, so the stream is broken and the client has to go
ESP_LOGW(TAG, "Cannot buffer a %zu byte chunk, closing", total);
this->request_close_();
} else {
this->tail_alloc_failed_(total); // nothing on the wire, the caller retries on the stall clock
}
return false;
}
uint8_t *dst = this->tail_.get();
std::memcpy(dst, prefix, prefix_len);
dst += prefix_len;
for_each_chunk_piece(
message, message_len,
[](void *ctx, const char *piece, size_t len) {
auto &out = *static_cast<uint8_t **>(ctx);
std::memcpy(out, piece, len);
out += len;
},
&dst);
this->tail_len_ = total;
this->tail_sent_ = sent;
return true;
}
void AsyncEventSourceResponse::loop() {
if (this->close_requested_) {
this->process_close_();
return;
}
this->refresh_lru_();
drain_tail_();
process_deferred_queue_();
if (this->close_requested_)
return;
// One step per loop; refusals retry next pass
this->entities_iterator_.try_advance(1);
}
#if defined(__GNUC__) && !defined(__clang__) && __GNUC__ >= 14 && defined(__OPTIMIZE_SIZE__)
#pragma GCC diagnostic push
#pragma GCC diagnostic ignored "-Wstack-usage=" // the one frame that holds the JSON arena
#endif
AsyncEventSourceResponse::JsonPrepareResult AsyncEventSourceResponse::prepare_json_(void *source,
message_generator_t *generator,
char *buffer, size_t *size_out) {
// Log lines emitted while the generator runs are dropped for this session: on the large path they would fill
// tail_ before it is overwritten, and on the small path they would go out ahead of the event being built.
SendGuard guard{*this};
json::JsonArena<json::JSON_ARENA_SIZE> arena;
json::JsonBuilder builder(&arena);
generator(this->web_server_, source, builder);
*size_out = builder.serialize_to(buffer, JSON_BUF_SIZE);
if (*size_out < JSON_BUF_SIZE) {
return JsonPrepareResult::JSON_PREPARE_RESULT_BUFFER;
}
char prefix[PREFIX_BUF_SIZE];
const size_t prefix_len = build_chunk_prefix(prefix, sizeof(prefix), "state", 0, 0, true);
// Nothing reaches the wire until the complete document and its framing are owned by the caller.
size_t json_len = 0;
size_t cap = std::max<size_t>(JSON_BUF_SIZE * 2, this->tail_cap_);
for (;;) {
if (!this->reserve_tail_(cap)) {
*size_out = cap;
return JsonPrepareResult::JSON_PREPARE_RESULT_NO_MEMORY;
}
const size_t room = cap - prefix_len - SSE_SUFFIX_LEN;
json_len = builder.serialize_to(reinterpret_cast<char *>(this->tail_.get()) + prefix_len, room);
if (json_len < room) {
break;
}
if (cap >= TAIL_MAX_SIZE) {
*size_out = room;
return JsonPrepareResult::JSON_PREPARE_RESULT_TOO_LARGE;
}
cap = std::min<size_t>(cap * 2, TAIL_MAX_SIZE);
}
const size_t total = prefix_len + json_len + SSE_SUFFIX_LEN;
write_chunk_header(prefix, total - CHUNK_HDR_LEN - CHUNK_END_LEN);
uint8_t *dst = this->tail_.get();
std::memcpy(dst, prefix, prefix_len);
std::memcpy(dst + prefix_len + json_len, SSE_SUFFIX, SSE_SUFFIX_LEN);
this->tail_len_ = total;
this->tail_sent_ = 0;
return JsonPrepareResult::JSON_PREPARE_RESULT_TAIL;
}
#if defined(__GNUC__) && !defined(__clang__) && __GNUC__ >= 14 && defined(__OPTIMIZE_SIZE__)
#pragma GCC diagnostic pop
#endif
bool AsyncEventSourceResponse::send_json_(void *source, message_generator_t *generator) {
if (!this->ready_to_send_()) {
return false;
}
char buffer[JSON_BUF_SIZE];
size_t size;
const auto result = this->prepare_json_(source, generator, buffer, &size);
switch (result) {
case JsonPrepareResult::JSON_PREPARE_RESULT_BUFFER:
return this->try_send_nodefer(buffer, size, "state");
case JsonPrepareResult::JSON_PREPARE_RESULT_TAIL:
this->drain_tail_();
return true;
case JsonPrepareResult::JSON_PREPARE_RESULT_NO_MEMORY: {
SendGuard guard{*this};
this->tail_alloc_failed_(size);
return false;
}
case JsonPrepareResult::JSON_PREPARE_RESULT_TOO_LARGE: {
SendGuard guard{*this};
ESP_LOGW(TAG, "State event does not fit %zu bytes, dropped", size);
this->tail_.reset();
this->tail_cap_ = 0;
this->send_failure_started_ms_ = 0;
return true;
}
}
return false;
}
void AsyncEventSourceResponse::tail_alloc_failed_(size_t cap) {
// Same stall clock as a socket that stops draining, so a session cannot retry forever
// Socket buffer full, try again later; a socket that stays full past the stall timeout is closed
const uint32_t now = App.get_loop_component_start_time();
if (this->send_failure_started_ms_ == 0) {
this->send_failure_started_ms_ = now != 0 ? now : 1; // Reserve zero for no stall.
ESP_LOGW(TAG, "No memory for a %zu byte chunk", cap);
return;
}
if (static_cast<int32_t>(now - (this->send_failure_started_ms_ + SEND_STALL_TIMEOUT_MS)) >= 0) {
ESP_LOGW(TAG, "Closing EventSource after %" PRIu32 " ms without memory", now - this->send_failure_started_ms_);
this->request_close_();
this->stall_.start(now);
if (this->stall_.is_expired(now, SEND_STALL_TIMEOUT_MS)) {
ESP_LOGW(TAG, "Closing stuck stream after %" PRIu32 " ms without send progress", this->stall_.elapsed(now));
this->close();
return -1;
}
return 0;
}
bool AsyncEventSourceResponse::ready_to_send_() {
if (this->sending_ || this->fd_.load() == 0 || this->close_requested_) {
return false;
}
drain_tail_();
return !this->close_requested_ && this->tail_len_ == 0;
ssize_t AsyncStreamResponse::write(const void *data, size_t len) {
struct iovec iov = {const_cast<void *>(data), len};
return this->write(&iov, 1);
}
bool AsyncEventSourceResponse::try_send_nodefer(const char *message, size_t message_len, const char *event, uint32_t id,
uint32_t reconnect) {
if (!this->ready_to_send_()) {
return false;
}
SendGuard guard{*this};
// Everything after the prefix goes out straight from the caller's buffer
char prefix[PREFIX_BUF_SIZE];
const size_t prefix_len = build_chunk_prefix(prefix, sizeof(prefix), event, id, reconnect, message != nullptr);
if (message == nullptr && prefix_len == CHUNK_HDR_LEN) {
return true; // Match ESPAsyncWebServer: nothing to send
}
if (prefix_len >= PREFIX_BUF_SIZE - 1) {
// The appenders truncate silently, which would put a malformed event on the wire
ESP_LOGW(TAG, "Event name too long, dropped");
return true;
}
// Gather list: the prefix, then the data lines and their separators from the message
struct Gather {
struct iovec iov[MAX_SEND_IOV]; // left uninitialized on purpose
int iovcnt{1};
size_t total{0};
bool fits{true};
} g;
g.total = prefix_len;
for_each_chunk_piece(
message, message_len,
[](void *ctx, const char *piece, size_t len) {
auto &g = *static_cast<Gather *>(ctx);
g.total += len;
if (len == 0) {
return;
}
if (g.iovcnt == MAX_SEND_IOV) {
g.fits = false;
return;
}
g.iov[g.iovcnt++] = {const_cast<char *>(piece), len};
},
&g);
// The header and the terminator are not part of the chunk length
write_chunk_header(prefix, g.total - CHUNK_HDR_LEN - CHUNK_END_LEN);
g.iov[0] = {prefix, prefix_len};
// A message with more lines than the list holds skips straight to the tail
const ssize_t sent = g.fits ? this->send_(g.iov, g.iovcnt) : 0;
if (sent < 0) {
return false;
}
if (static_cast<size_t>(sent) == g.total) {
this->send_failure_started_ms_ = 0; // progress, whichever stall clock was running
return true;
}
// The caller's buffers do not outlive this call, so keep the chunk and continue from loop()
return this->stash_chunk_(prefix, prefix_len, message, message_len, g.total, sent);
}
void AsyncEventSourceResponse::deferrable_send_state(void *source, bool detail_all,
message_generator_t *message_generator) {
// Let the detail_all listing finish before bare state events, so the web UI never shows unnamed entries and
// initial connect sends less
if (!detail_all && !this->entities_iterator_.completed())
return;
drain_tail_();
process_deferred_queue_();
if (this->close_requested_) {
return;
}
if (this->tail_len_ != 0 || !deferred_queue_.empty()) {
// outgoing event buffer or deferred queue still not empty which means downstream tcp send buffer full, no point
// trying to send first
deq_push_back_with_dedup_(source, message_generator);
} else {
// A send error closes the session and clears the queue; nothing is queued after that
if (!this->send_json_(source, message_generator) && !this->close_requested_) {
deq_push_back_with_dedup_(source, message_generator);
}
}
}
#endif
#ifdef USE_WEBSERVER_OTA
esp_err_t AsyncWebServer::handle_multipart_upload_(httpd_req_t *r, const char *content_type) {
// Parse boundary and create reader
@@ -15,21 +15,14 @@
#include <utility>
#include <vector>
#ifdef USE_WEBSERVER
#include "esphome/components/json/json_util.h"
#include "esphome/components/web_server/list_entities.h"
#include "sse_chunk.h"
#endif
#include "esphome/components/web_server_base/stream_head.h"
struct iovec; // NOLINT(readability-identifier-naming) - forward decl of lwip's gather list entry
namespace esphome {
#ifdef USE_WEBSERVER
namespace web_server {
class WebServer;
}; // namespace web_server
#endif
namespace web_server_idf {
namespace esphome::web_server_idf {
using web_server_base::HttpHeader;
using web_server_base::StallClock;
class AsyncWebParameter {
public:
@@ -191,6 +184,7 @@ class AsyncWebServerRequest {
void onDisconnect(std::function<void()> fn) { this->on_disconnect_ = std::move(fn); }
#endif
/// sess_ctx is reserved for AsyncStreamResponse: the server's close_fn treats any session context as one.
operator httpd_req_t *() const { return this->req_; }
optional<std::string> get_header(const char *name) const;
/// Copy a header value into buf without heap allocation.
@@ -269,204 +263,85 @@ class AsyncWebHandler {
virtual bool isRequestHandlerTrivial() const { return true; }
};
#ifdef USE_WEBSERVER
class AsyncEventSource;
class AsyncEventSourceResponse;
using message_generator_t = void(esphome::web_server::WebServer *, void *, json::JsonBuilder &);
/*
This class holds a pointer to the source component that wants to publish a state event, and a pointer to a function
that will lazily generate that event. The two pointers allow dedup in the deferred queue if multiple publishes for
the same component are backed up, and take up only two pointers of memory. The entry in the deferred queue (a
std::vector) is the DeferredEvent instance itself (not a pointer to one elsewhere in heap) so still only two pointers
per entry (and no heap fragmentation). Even 100 backed up events (you'd have to have at least 100 sensors publishing
because of dedup) would take up only 0.8 kB.
*/
struct DeferredEvent {
friend class AsyncEventSourceResponse;
protected:
void *source_;
message_generator_t *message_generator_;
/// A response that keeps its connection open and is written from the main loop, such as an event stream or
/// an audio stream. Use it directly or derive from it. Call loop() and delete the object only through the type
/// it was created as: subclasses hide loop() instead of overriding it, and the destructor is not virtual.
///
/// Lifecycle:
/// 1. Httpd task, inside handleRequest(): create it and call begin(), then hand the pointer to the main loop.
/// 2. Main loop: once is_attached() is true, write() may go through. Call loop() on every pass.
/// 3. Main loop: delete it once safe_to_delete() is true. httpd owns the socket throughout.
class AsyncStreamResponse {
public:
DeferredEvent(void *source, message_generator_t *message_generator)
: source_(source), message_generator_(message_generator) {}
bool operator==(const DeferredEvent &test) const {
return (source_ == test.source_ && message_generator_ == test.message_generator_);
}
};
static_assert(sizeof(DeferredEvent) == sizeof(void *) + sizeof(message_generator_t *),
"DeferredEvent should have no padding");
/// Httpd task, once. Sends the status line (for example HTTPD_200), Content-Type, headers, the default headers
/// and body (a C string or null, sent as is), then makes this object the session context. A chunked response
/// passes {"Transfer-Encoding", "chunked"} as its first header and frames each chunk itself.
void begin(AsyncWebServerRequest *request, const char *status, const char *content_type,
std::span<const HttpHeader> headers, const char *body = nullptr);
class AsyncEventSourceResponse {
friend class AsyncEventSource;
public:
bool try_send_nodefer(const char *message, size_t message_len, const char *event = nullptr, uint32_t id = 0,
uint32_t reconnect = 0);
// detail_all events go out while entities are still being listed; state events wait until listing is done
void deferrable_send_state(void *source, bool detail_all, message_generator_t *message_generator);
// The old event name argument would otherwise convert to true
void deferrable_send_state(void *source, const char *event_type, message_generator_t *message_generator) = delete;
/// Main loop. Non-blocking gather write. Returns the bytes accepted, which may be fewer than offered: the caller
/// keeps the rest and continues from there. 0 means the socket is full or not ours yet; -1 means the stream failed
/// and is closing. Refused writes for SEND_STALL_TIMEOUT_MS in a row also close it.
ssize_t write(struct iovec *iov, int iovcnt);
ssize_t write(const void *data, size_t len);
/// Main loop. Shuts the connection down through httpd; safe to call more than once.
void close();
/// Main loop, every pass. Keeps the session fresh in httpd's LRU and retries a pending close.
void loop();
/// Httpd task, from the server's close_fn: no write() reaches the socket after this returns.
static void socket_closing(httpd_handle_t hd, int fd);
protected:
AsyncEventSourceResponse(const AsyncWebServerRequest *request, esphome::web_server_idf::AsyncEventSource *server,
esphome::web_server::WebServer *ws);
// Main-loop only: sends initial ping/config/sorting_groups, starts entity iterator.
void start_session_main_loop_();
void deq_push_back_with_dedup_(void *source, message_generator_t *message_generator);
void process_deferred_queue_();
// A new chunk may go out: not re-entered from a log line, session alive, tail empty
bool ready_to_send_();
// Non-blocking gather write. Returns bytes written, 0 on would-block or a socket that is not
// ours, -1 after requesting the close on any other error.
ssize_t send_(struct iovec *iov, int iovcnt);
// Push what is left of the chunk in tail_ to the socket; owns the stall timer.
void drain_tail_();
// Grow tail_ to hold len bytes, kept at its high-water mark. False on OOM.
bool reserve_tail_(size_t len);
// Keep the whole chunk in tail_ and continue from sent; false when the tail cannot be allocated.
bool stash_chunk_(const char *prefix, size_t prefix_len, const char *message, size_t message_len, size_t total,
size_t sent);
// Send a state event; JSON too large for the stack buffer is serialized into tail_ instead
bool send_json_(void *source, message_generator_t *generator);
enum class JsonPrepareResult : uint8_t {
JSON_PREPARE_RESULT_BUFFER,
JSON_PREPARE_RESULT_TAIL,
JSON_PREPARE_RESULT_NO_MEMORY,
JSON_PREPARE_RESULT_TOO_LARGE,
};
// Keep the arena out of the caller's frame so it is gone before the state send.
// size_out: BUFFER = JSON length, NO_MEMORY = requested tail capacity, TOO_LARGE = available JSON room;
// unused for TAIL.
JsonPrepareResult __attribute__((noinline)) prepare_json_(void *source, message_generator_t *generator, char *buffer,
size_t *size_out);
// Warn once, and close the session once the stall timeout passes with no memory for the tail
void tail_alloc_failed_(size_t cap);
void request_close_();
// A log line emitted inside a send re-enters try_send_nodefer on this session; refuse it
struct SendGuard {
AsyncEventSourceResponse &owner;
explicit SendGuard(AsyncEventSourceResponse &owner) : owner(owner) { owner.sending_ = true; }
~SendGuard() { this->owner.sending_ = false; }
};
void process_close_();
static void close_session_work(void *arg);
void refresh_lru_();
static void refresh_lru_work(void *arg);
// httpd frees a session before closing its socket, so the fd may already belong to a new client
bool owns_session_(int fd) const { return fd != 0 && httpd_sess_get_ctx(this->hd_, fd) == this; }
// Deletable only after destroy() zeroed fd_ and no queued HTTPD work still references this object.
bool safe_to_delete_() const {
/// httpd holds this response as the session context; begin() has sent the whole head by then.
bool is_attached() const { return this->owns_session_(this->fd_.load()); }
/// The client is still connected; httpd has not freed the session.
bool is_connected() const { return this->fd_.load() != 0; }
bool is_closing() const { return this->close_requested_; }
/// The socket fd, for log lines only
int log_id() const { return this->fd_.load(); }
/// Sends are making no progress; after write() returns -1 this tells a send timeout from a send error.
bool is_stalled() const { return this->stall_.is_running(); }
/// The session is gone and no queued httpd work still references this object.
bool safe_to_delete() const {
return this->fd_.load() == 0 && !this->close_work_queued_.load(std::memory_order_acquire) &&
!this->lru_work_queued_.load(std::memory_order_acquire);
}
static constexpr uint32_t SEND_STALL_TIMEOUT_MS = 20000;
protected:
void process_close_();
static void close_session_work(void *arg);
void refresh_lru_();
static void refresh_lru_work(void *arg);
// httpd closes the socket before it clears the session, so this can still match a closed fd; write() also checks
// socket_closed_
bool owns_session_(int fd) const { return fd != 0 && httpd_sess_get_ctx(this->hd_, fd) == this; }
static void destroy(void *p);
AsyncEventSource *server_;
httpd_handle_t hd_{};
std::atomic<int> fd_{};
std::vector<DeferredEvent> deferred_queue_;
esphome::web_server::WebServer *web_server_;
esphome::web_server::ListEntitiesIterator entities_iterator_;
// One chunk the socket did not take whole, allocated on the first stall; the only heap use
// on the send path
RAMUniquePtr<uint8_t[]> tail_;
uint32_t send_failure_started_ms_{0}; // Zero means no send stall in progress.
// Runs while sends make no progress; a running clock keeps its start time
StallClock stall_;
uint32_t next_close_attempt_ms_{0};
uint32_t last_lru_refresh_ms_{0};
uint16_t tail_cap_{0};
uint16_t tail_len_{0}; // Zero means nothing pending
uint16_t tail_sent_{0};
// Set on the main loop before queueing close work, cleared by the HTTPD-task callback when done.
std::atomic<bool> close_work_queued_{false};
std::atomic<bool> lru_work_queued_{false};
// Guarded by the stream send mutex
bool socket_closed_{false};
// Main-loop only; the HTTPD task never reads or writes these flags.
bool close_requested_{false};
bool close_retry_warning_logged_{false};
bool sending_{false};
// The longest multi line log message in the tree (a climate dump_config) has 22 lines; a
// longer one goes through the tail
static constexpr size_t MAX_SEND_LINES = 22;
static constexpr size_t MAX_SEND_IOV = 1 + 2 * MAX_SEND_LINES;
// Chunk header, retry/id/event lines and the first "data: "
static constexpr size_t PREFIX_BUF_SIZE = 128;
// Stack buffer for a state event's JSON; a larger document is serialized into the tail
static constexpr size_t JSON_BUF_SIZE = 1024;
// Same ceiling JsonBuilder::serialize() applies (max_heap_size in json_util.cpp); a larger
// document is dropped before anything is on the wire
static constexpr size_t JSON_MAX_SIZE = 5120;
// Most RAM a stalled session keeps: the largest state document plus any accepted framing
static constexpr size_t TAIL_MAX_SIZE = JSON_MAX_SIZE + PREFIX_BUF_SIZE + SSE_SUFFIX_LEN;
static constexpr uint32_t SEND_STALL_TIMEOUT_MS = 20000;
static constexpr uint32_t CLOSE_RETRY_INTERVAL_MS = 250;
static constexpr uint32_t CLOSE_CONFIRM_INTERVAL_MS = 1000;
static constexpr uint32_t LRU_REFRESH_INTERVAL_MS = 1000;
};
using AsyncEventSourceClient = AsyncEventSourceResponse;
class AsyncEventSource : public AsyncWebHandler {
friend class AsyncEventSourceResponse;
public:
AsyncEventSource(StringRef url, esphome::web_server::WebServer *ws) : url_(url), web_server_(ws) {}
~AsyncEventSource() override;
// NOLINTNEXTLINE(readability-identifier-naming)
bool canHandle(AsyncWebServerRequest *request) const override {
if (request->method() != HTTP_GET)
return false;
char url_buf[AsyncWebServerRequest::URL_BUF_SIZE];
return request->url_to(url_buf) == this->url_;
}
// NOLINTNEXTLINE(readability-identifier-naming)
void handleRequest(AsyncWebServerRequest *request) override;
void try_send_nodefer(const char *message, size_t message_len, const char *event = nullptr, uint32_t id = 0,
uint32_t reconnect = 0);
void deferrable_send_state(void *source, bool detail_all, message_generator_t *message_generator);
// The old event name argument would otherwise convert to true
void deferrable_send_state(void *source, const char *event_type, message_generator_t *message_generator) = delete;
/// Returns true if there are sessions remaining (including pending cleanup).
bool loop();
bool empty() { return this->count() == 0; }
size_t count() const { return this->sessions_.size(); }
protected:
// Cold path: move sessions from pending_sessions_ into sessions_ and greet each one.
void __attribute__((noinline, cold)) adopt_pending_sessions_main_loop_();
StringRef url_; // Must outlive this object (string literal)
// Main-loop only. Vector: SSE sessions are 1-5 connections, linear search beats set.
std::vector<AsyncEventSourceResponse *> sessions_;
// Httpd-task intake; guarded by pending_mutex_, gated by has_pending_sessions_.
std::vector<AsyncEventSourceResponse *> pending_sessions_;
Mutex pending_mutex_;
esphome::web_server::WebServer *web_server_;
std::atomic<bool> has_pending_sessions_{false};
};
#endif // USE_WEBSERVER
struct HttpHeader {
const char *name;
const char *value;
};
class DefaultHeaders {
friend class AsyncWebServerRequest;
friend class AsyncStreamResponse;
friend class AsyncWebServerResponse;
#ifdef USE_WEBSERVER
friend class AsyncEventSourceResponse;
#endif
static_assert(WEB_SERVER_DEFAULT_HEADERS_COUNT <= 8, "AsyncWebServerResponse::replaced_defaults_ holds 8 bits");
public:
@@ -484,8 +359,7 @@ class DefaultHeaders {
void apply_(httpd_req_t *req, uint8_t skip_mask = 0) const;
};
} // namespace web_server_idf
} // namespace esphome
} // namespace esphome::web_server_idf
using namespace esphome::web_server_idf; // NOLINT(google-global-names-in-headers)
+3 -6
View File
@@ -32,12 +32,9 @@ bool random_bytes(uint8_t *data, size_t len) {
#ifdef USE_NRF52
void get_mac_address_raw(uint8_t *mac) { // NOLINT(readability-non-const-parameter)
mac[0] = ((NRF_FICR->DEVICEADDR[1] & 0xFFFF) >> 8) | 0xC0;
mac[1] = NRF_FICR->DEVICEADDR[1] & 0xFFFF;
mac[2] = NRF_FICR->DEVICEADDR[0] >> 24;
mac[3] = NRF_FICR->DEVICEADDR[0] >> 16;
mac[4] = NRF_FICR->DEVICEADDR[0] >> 8;
mac[5] = NRF_FICR->DEVICEADDR[0];
// The top two bits of a random static address are set
put_be16(mac, NRF_FICR->DEVICEADDR[1] | 0xC000);
put_be32(mac + 2, NRF_FICR->DEVICEADDR[0]);
}
#endif
} // namespace esphome
+4 -4
View File
@@ -125,7 +125,7 @@ lib_deps =
ESP8266WiFi ; wifi (Arduino built-in)
Update ; ota (Arduino built-in)
ESP32Async/ESPAsyncTCP@2.0.0 ; async_tcp
ESP32Async/ESPAsyncWebServer@3.9.6 ; web_server_base
ESP32Async/ESPAsyncWebServer@3.12.1 ; web_server_base
makuna/NeoPixelBus@2.7.3 ; neopixelbus
ESP8266HTTPClient ; http_request (Arduino built-in)
ESP8266mDNS ; mdns (Arduino built-in)
@@ -217,7 +217,7 @@ lib_deps =
${common:arduino.lib_deps}
${common:idf-component-libs.lib_deps}
https://github.com/esphome-libs/RPAsyncTCP.git#1.3.2-esphome.1 ; async_tcp
ESP32Async/ESPAsyncWebServer@3.9.6 ; web_server_base
ESP32Async/ESPAsyncWebServer@3.12.1 ; web_server_base
WiFi ; wifi (arduino-pico built-in)
lwIP_CYW43 ; wifi (arduino-pico built-in, WiFi dependency)
HTTPClient ; http_request (arduino-pico built-in)
@@ -245,10 +245,10 @@ lib_compat_mode = soft
lib_deps =
${common.lib_deps_base}
${common:idf-component-libs.lib_deps}
ESP32Async/ESPAsyncWebServer@3.9.6 ; web_server_base
ESP32Async/ESPAsyncWebServer@3.12.1 ; web_server_base
esphome/wireguard@0.4.8 ; wireguard
esphome/noise-c@0.1.31 ; noise (api, ota)
ESP32Async/AsyncTCP@3.4.5 ; async_tcp
ESP32Async/AsyncTCP@3.5.0 ; async_tcp
DNSServer ; captive_portal
heman/AsyncMqttClient-esphome@2.0.0 ; mqtt
improv/Improv@1.2.7 ; improv_serial
Whitespace-only changes.
@@ -0,0 +1,20 @@
"""LibreTiny always gets the xPortGetCoreID stand-in, whatever its log level."""
from collections.abc import Callable
from pathlib import Path
from esphome.core import CORE
def test_libretiny_adds_core_id_flag(
generate_main: Callable[[str | Path], str], tmp_path: Path
) -> None:
config = tmp_path / "test.yaml"
config.write_text(
"esphome:\n name: test\n\n"
"bk72xx:\n board: generic-bk7231n-qfn32-tuya\n\n"
"wifi:\n ssid: test\n password: testtest12\n\n"
"async_tcp:\n"
)
generate_main(config)
assert "-DxPortGetCoreID=int" in CORE.build_flags
+16
View File
@@ -0,0 +1,16 @@
"""Build the SX127x C++ tests against a software SPI bus, with real logger callbacks."""
import esphome.codegen as cg
from esphome.components.logger import request_log_listener
from esphome.types import ConfigType
from tests.testing_helpers import ComponentManifestOverride
def override_manifest(manifest: ComponentManifestOverride) -> None:
async def to_code(config: ConfigType) -> None:
request_log_listener()
# The tests build a software SPI bus in C++, which no configured bus asks for
cg.add_define("USE_SPI_SOFTWARE")
manifest.multi_conf = False
manifest.to_code = to_code
@@ -0,0 +1,347 @@
#include "sx127x_test_helpers.h"
#include <cmath>
#include <tuple>
namespace esphome::sx127x::testing {
static std::vector<uint8_t> expected_fifo(const RadioHardware &hw, bool lora, bool fixed) {
std::vector<uint8_t> fifo;
if (!lora && !fixed) {
fifo.push_back(hw.packet.size()); // A variable length FSK packet starts with its length
}
fifo.insert(fifo.end(), hw.packet.begin(), hw.packet.end());
return fifo;
}
class SX127xAsyncModes : public ::testing::TestWithParam<std::tuple<bool, bool, bool>> {};
TEST_P(SX127xAsyncModes, ReturnsBeforeCompletionAndRestoresMode) {
const auto [lora, rx, fixed] = GetParam();
RadioHardware hw;
TransmitListener listener;
hw.configure(lora, rx, fixed);
ASSERT_FALSE(hw.radio.is_failed());
hw.auto_complete = false;
ASSERT_EQ(hw.radio.start_transmit_async(&listener, hw.packet, 1234), SX127xError::SX127X_ERROR_NONE);
EXPECT_TRUE(hw.radio.is_transmitting());
EXPECT_EQ(hw.mode(), MODE_TX);
ASSERT_EQ(hw.sent.size(), 1u);
EXPECT_EQ(hw.sent[0], expected_fifo(hw, lora, fixed));
hw.radio.loop();
EXPECT_TRUE(listener.results.empty());
hw.complete_send();
hw.radio.loop();
EXPECT_EQ(listener.results, std::vector<SX127xError>{SX127xError::SX127X_ERROR_NONE});
EXPECT_FALSE(hw.radio.is_transmitting());
EXPECT_EQ(hw.mode(), rx ? MODE_RX : MODE_SLEEP);
if (lora) {
EXPECT_EQ(hw.irq, 0); // TX_DONE is cleared, so DIO0 is low again
}
}
TEST_P(SX127xAsyncModes, BlockingSendWaitsForCompletion) {
const auto [lora, rx, fixed] = GetParam();
RadioHardware hw;
hw.configure(lora, rx, fixed);
EXPECT_EQ(hw.radio.transmit_packet(hw.packet), SX127xError::SX127X_ERROR_NONE);
EXPECT_FALSE(hw.radio.is_transmitting());
ASSERT_EQ(hw.sent.size(), 1u);
EXPECT_EQ(hw.sent[0], expected_fifo(hw, lora, fixed));
EXPECT_EQ(hw.mode(), rx ? MODE_RX : MODE_SLEEP);
}
INSTANTIATE_TEST_SUITE_P(Modes, SX127xAsyncModes,
::testing::Combine(::testing::Bool(), ::testing::Bool(), ::testing::Bool()));
// Modulation and fixed length; only a receiving radio has a packet waiting
class SX127xHeldPacket : public ::testing::TestWithParam<std::tuple<bool, bool>> {};
TEST_P(SX127xHeldPacket, PacketWaitingWhenASendStartsIsDeliveredFirst) {
const auto [lora, fixed] = GetParam();
const bool rx = true;
RadioHardware hw;
PacketListener packets;
TransmitListener listener;
hw.radio.register_listener(&packets);
hw.configure(lora, rx, fixed);
hw.receive({0x44, 0x55, 0x66});
EXPECT_EQ(hw.radio.start_transmit_async(&listener, hw.packet), SX127xError::SX127X_ERROR_BUSY);
EXPECT_TRUE(hw.sent.empty());
hw.radio.set_mode_sleep(); // Ignored while the packet is held
EXPECT_EQ(hw.mode(), MODE_STDBY);
hw.radio.loop();
ASSERT_EQ(packets.packets.size(), 1u);
EXPECT_EQ(packets.packets[0], (std::vector<uint8_t>{0x44, 0x55, 0x66}));
EXPECT_TRUE(hw.receiving());
EXPECT_EQ(hw.radio.start_transmit_async(&listener, hw.packet), SX127xError::SX127X_ERROR_NONE);
hw.radio.loop();
EXPECT_EQ(listener.results, std::vector<SX127xError>{SX127xError::SX127X_ERROR_NONE});
EXPECT_EQ(hw.sent.size(), 1u);
}
INSTANTIATE_TEST_SUITE_P(Modes, SX127xHeldPacket, ::testing::Combine(::testing::Bool(), ::testing::Bool()));
TEST(SX127xAsync, OneSendAtATime) {
RadioHardware hw;
TransmitListener first, second;
hw.configure(true, true);
hw.auto_complete = false;
ASSERT_EQ(hw.radio.start_transmit_async(&first, hw.packet), SX127xError::SX127X_ERROR_NONE);
EXPECT_EQ(hw.radio.start_transmit_async(&second, hw.packet), SX127xError::SX127X_ERROR_BUSY);
EXPECT_EQ(hw.radio.transmit_packet(hw.packet), SX127xError::SX127X_ERROR_BUSY);
hw.radio.set_mode_rx();
hw.radio.run_image_cal();
hw.radio.configure();
EXPECT_EQ(hw.mode(), MODE_TX);
EXPECT_EQ(hw.sent.size(), 1u);
hw.complete_send();
hw.radio.loop();
EXPECT_EQ(first.results, std::vector<SX127xError>{SX127xError::SX127X_ERROR_NONE});
EXPECT_TRUE(second.results.empty());
}
TEST(SX127xAsync, SendWithoutCompletionTimesOut) {
RadioHardware hw;
TransmitListener listener;
hw.configure(true, true);
hw.auto_complete = false;
const uint32_t start = millis();
ASSERT_EQ(hw.radio.start_transmit_async(&listener, hw.packet, 1000), SX127xError::SX127X_ERROR_NONE);
{
LoopBlockingGuard clock{nullptr, nullptr, start + 500};
hw.radio.loop();
}
EXPECT_TRUE(listener.results.empty());
{
LoopBlockingGuard clock{nullptr, nullptr, start + 1100};
hw.radio.loop();
}
EXPECT_EQ(listener.results, std::vector<SX127xError>{SX127xError::SX127X_ERROR_TIMEOUT});
EXPECT_FALSE(hw.radio.status_has_warning());
EXPECT_TRUE(hw.receiving());
}
// The chip leaves a send that is still going out for standby or sleep, but not for RX
class SX127xCancel : public ::testing::TestWithParam<std::tuple<bool, bool>> {};
TEST_P(SX127xCancel, CancelEndsOnlyTheListenersOwnSend) {
const auto [lora, rx] = GetParam();
RadioHardware hw;
TransmitListener owner, other;
hw.configure(lora, rx);
hw.auto_complete = false;
ASSERT_EQ(hw.radio.start_transmit_async(&owner, hw.packet), SX127xError::SX127X_ERROR_NONE);
EXPECT_FALSE(hw.radio.cancel_transmit_async(&other));
EXPECT_FALSE(hw.radio.cancel_transmit_async(nullptr));
EXPECT_TRUE(hw.radio.cancel_transmit_async(&owner));
EXPECT_EQ(owner.results, std::vector<SX127xError>{SX127xError::SX127X_ERROR_CANCELLED});
EXPECT_FALSE(hw.radio.is_transmitting());
EXPECT_FALSE(hw.radio.status_has_warning());
EXPECT_EQ(hw.mode(), rx ? MODE_RX : MODE_SLEEP);
EXPECT_FALSE(hw.radio.cancel_transmit_async(&owner));
hw.radio.loop();
EXPECT_EQ(owner.results.size(), 1u);
EXPECT_TRUE(other.results.empty());
}
INSTANTIATE_TEST_SUITE_P(Modes, SX127xCancel, ::testing::Combine(::testing::Bool(), ::testing::Bool()));
TEST(SX127xAsync, CompletionMayStartTheNextSend) {
RadioHardware hw;
TransmitListener listener;
hw.configure(true, true);
listener.on_complete = [&](SX127xError result) {
if (listener.results.size() == 1) {
EXPECT_EQ(hw.radio.start_transmit_async(&listener, hw.packet), SX127xError::SX127X_ERROR_NONE);
}
};
ASSERT_EQ(hw.radio.start_transmit_async(&listener, hw.packet), SX127xError::SX127X_ERROR_NONE);
hw.radio.loop();
EXPECT_TRUE(hw.radio.is_transmitting());
hw.radio.loop();
EXPECT_EQ(listener.results.size(), 2u);
EXPECT_EQ(hw.sent.size(), 2u);
EXPECT_TRUE(hw.receiving());
}
TEST(SX127xAsync, PacketListenerMaySend) {
RadioHardware hw;
PacketListener packets;
hw.radio.register_listener(&packets);
hw.configure(true, true);
packets.on_receive = [&]() { EXPECT_EQ(hw.radio.transmit_packet(hw.packet), SX127xError::SX127X_ERROR_NONE); };
hw.receive({0x01, 0x02});
hw.radio.loop();
EXPECT_EQ(packets.packets.size(), 1u);
EXPECT_EQ(hw.sent.size(), 1u);
EXPECT_TRUE(hw.receiving());
}
TEST(SX127xAsync, InvalidRequestsAreRefused) {
RadioHardware hw;
TransmitListener listener;
hw.configure(true, true, true);
EXPECT_EQ(hw.radio.start_transmit_async(nullptr, hw.packet), SX127xError::SX127X_ERROR_INVALID_PARAMS);
EXPECT_EQ(hw.radio.start_transmit_async(&listener, hw.packet, 0), SX127xError::SX127X_ERROR_INVALID_PARAMS);
EXPECT_EQ(hw.radio.start_transmit_async(&listener, hw.packet, 262144), SX127xError::SX127X_ERROR_INVALID_PARAMS);
EXPECT_EQ(hw.radio.start_transmit_async(&listener, nullptr, 3), SX127xError::SX127X_ERROR_INVALID_PARAMS);
EXPECT_EQ(hw.radio.transmit_packet(hw.packet.data(), 0), SX127xError::SX127X_ERROR_INVALID_PARAMS);
EXPECT_EQ(hw.radio.transmit_packet(hw.packet.data(), 2), SX127xError::SX127X_ERROR_INVALID_PARAMS);
EXPECT_TRUE(hw.sent.empty());
EXPECT_TRUE(listener.results.empty());
EXPECT_EQ(hw.radio.transmit_packet(hw.packet), SX127xError::SX127X_ERROR_NONE);
}
TEST(SX127xAsync, SendWithoutDio0IsRefused) {
RadioHardware hw;
hw.radio.set_dio0_pin(nullptr);
hw.configure(true, true);
EXPECT_EQ(hw.radio.transmit_packet(hw.packet), SX127xError::SX127X_ERROR_INVALID_PARAMS);
EXPECT_TRUE(hw.sent.empty());
}
TEST(SX127xReceive, LoRaPacketHasRssiAndSnr) {
RadioHardware hw;
PacketListener packets;
hw.radio.register_listener(&packets);
hw.configure(true, true);
hw.receive({0x01, 0x02, 0x03, 0x04});
hw.radio.loop();
ASSERT_EQ(packets.packets.size(), 1u);
EXPECT_EQ(packets.packets[0], (std::vector<uint8_t>{0x01, 0x02, 0x03, 0x04}));
EXPECT_FLOAT_EQ(packets.rssi, 100.0f - 164.0f);
EXPECT_FLOAT_EQ(packets.snr, 5.0f);
EXPECT_EQ(hw.irq, 0);
}
TEST(SX127xReceive, FskPacketHasNoSnr) {
RadioHardware hw;
PacketListener packets;
hw.radio.register_listener(&packets);
hw.configure(false, true);
hw.receive({0x01, 0x02, 0x03, 0x04});
hw.radio.loop();
ASSERT_EQ(packets.packets.size(), 1u);
EXPECT_EQ(packets.packets[0], (std::vector<uint8_t>{0x01, 0x02, 0x03, 0x04}));
EXPECT_TRUE(std::isnan(packets.snr));
}
TEST(SX127xReceive, PacketThatFailedItsCrcIsDroppedAndDoesNotHoldUpASend) {
RadioHardware hw;
PacketListener packets;
hw.radio.register_listener(&packets);
hw.configure(true, true);
hw.receive({0x01}, true);
EXPECT_EQ(hw.radio.transmit_packet(hw.packet), SX127xError::SX127X_ERROR_NONE);
hw.radio.loop();
EXPECT_TRUE(packets.packets.empty());
}
// The TX mode action sends whatever the FIFO holds without tracking it; its completion
// is not taken for a received packet
TEST(SX127xReceive, RawTxModeIsNotAPacket) {
for (const bool lora : {true, false}) {
RadioHardware hw;
PacketListener packets;
hw.radio.register_listener(&packets);
hw.configure(lora, true);
hw.radio.set_mode_tx();
EXPECT_EQ(hw.mode(), MODE_TX);
EXPECT_TRUE(hw.dio0.digital_read());
hw.radio.loop();
EXPECT_TRUE(packets.packets.empty()) << (lora ? "LoRa" : "FSK");
hw.radio.set_mode_rx();
EXPECT_EQ(hw.radio.transmit_packet(hw.packet), SX127xError::SX127X_ERROR_NONE);
}
}
TEST(SX127xRecovery, FailedModeChangeIsReconfiguredWithoutBeingAsked) {
RadioHardware hw;
hw.configure(true, true);
hw.stuck = true;
hw.radio.set_mode_standby();
EXPECT_FALSE(hw.radio.is_failed());
EXPECT_TRUE(hw.radio.status_has_warning());
EXPECT_EQ(hw.radio.transmit_packet(hw.packet), SX127xError::SX127X_ERROR_NOT_READY);
hw.radio.set_mode_sleep(); // Ignored until configured again
hw.stuck = false;
EXPECT_EQ(hw.mode(), MODE_RX);
run_scheduler(1100);
EXPECT_FALSE(hw.radio.status_has_warning());
EXPECT_TRUE(hw.receiving());
EXPECT_EQ(hw.radio.transmit_packet(hw.packet), SX127xError::SX127X_ERROR_NONE);
// Once configured, the retry stops
hw.radio.set_mode_sleep();
run_scheduler(1100);
EXPECT_EQ(hw.mode(), MODE_SLEEP);
}
TEST(SX127xRecovery, FailedSendStartIsReconfiguredWithoutBeingAsked) {
RadioHardware hw;
TransmitListener listener;
hw.configure(true, true);
hw.stuck = true;
EXPECT_EQ(hw.radio.start_transmit_async(&listener, hw.packet), SX127xError::SX127X_ERROR_NOT_READY);
EXPECT_FALSE(hw.radio.is_transmitting());
EXPECT_TRUE(listener.results.empty());
hw.stuck = false;
run_scheduler(1100);
EXPECT_EQ(hw.radio.start_transmit_async(&listener, hw.packet), SX127xError::SX127X_ERROR_NONE);
}
TEST(SX127xRecovery, UnsupportedChipFailsSetup) {
RadioHardware hw;
hw.version = 0x22;
hw.configure(true, true);
EXPECT_TRUE(hw.radio.is_failed());
EXPECT_EQ(hw.radio.transmit_packet(hw.packet), SX127xError::SX127X_ERROR_NOT_READY);
}
TEST(SX127xRecovery, NothingHappensBeforeSetup) {
RadioHardware hw;
unsigned resets = 0;
hw.rst.on_write = [&](bool high) { resets += high ? 0 : 1; };
hw.radio.configure();
hw.radio.set_mode_rx();
EXPECT_EQ(hw.radio.transmit_packet(hw.packet), SX127xError::SX127X_ERROR_NOT_READY);
EXPECT_EQ(resets, 0u);
EXPECT_EQ(hw.mode(), MODE_STDBY);
}
} // namespace esphome::sx127x::testing
namespace esphome::sx127x::testing {
// configure() is refused during a send, so settings changed then are not used until it succeeds
TEST(SX127xSettings, SettingsChangedDuringASendWaitForConfigure) {
RadioHardware hw;
TransmitListener listener;
PacketListener packets;
hw.radio.register_listener(&packets);
hw.configure(false, true, true);
hw.auto_complete = false;
ASSERT_EQ(hw.radio.start_transmit_async(&listener, hw.packet), SX127xError::SX127X_ERROR_NONE);
hw.radio.set_payload_length(4);
hw.radio.set_rx_start(false);
hw.radio.set_modulation(MOD_LORA);
hw.radio.configure(); // Refused: a send is on air
hw.complete_send();
hw.radio.loop();
EXPECT_EQ(listener.results, std::vector<SX127xError>{SX127xError::SX127X_ERROR_NONE});
EXPECT_TRUE(hw.receiving()); // Still the applied rx_start
EXPECT_FALSE(hw.lora());
EXPECT_EQ(hw.radio.get_max_packet_size(), 3u);
const std::vector<uint8_t> four{0x01, 0x02, 0x03, 0x04};
EXPECT_EQ(hw.radio.transmit_packet(four), SX127xError::SX127X_ERROR_INVALID_PARAMS);
hw.receive({0x07, 0x08, 0x09});
hw.radio.loop();
ASSERT_EQ(packets.packets.size(), 1u);
EXPECT_EQ(packets.packets[0], (std::vector<uint8_t>{0x07, 0x08, 0x09}));
// Once applied, they are used
hw.radio.configure();
EXPECT_TRUE(hw.lora());
EXPECT_EQ(hw.mode(), MODE_SLEEP);
EXPECT_EQ(hw.radio.get_max_packet_size(), 4u);
}
} // namespace esphome::sx127x::testing
@@ -0,0 +1,54 @@
#include "sx127x_test_helpers.h"
#include "esphome/components/logger/logger.h"
#include <string>
namespace esphome::sx127x::testing {
// Runs a callback for each warning logged, as a logger on_message automation would
class LogCallback {
public:
LogCallback() : previous_(logger::global_logger) {
this->logger_.pre_setup();
this->logger_.set_log_level(ESPHOME_LOG_LEVEL_WARN);
this->logger_.add_log_callback(this,
[](void *instance, uint8_t level, const char *tag, const char *message, size_t len) {
auto *self = static_cast<LogCallback *>(instance);
if (self->callback_) {
self->callback_(std::string(message, len));
}
});
}
~LogCallback() { logger::global_logger = this->previous_; }
void set_callback(std::function<void(const std::string &)> callback) { this->callback_ = std::move(callback); }
private:
std::function<void(const std::string &)> callback_;
logger::Logger *previous_;
logger::Logger logger_{0};
};
// A configure() from the warning's log callback recovers the radio, and no retry is left
// behind to reset it again later
TEST(SX127xReentry, ConfigureFromTheFailureWarningLeavesNoRetry) {
RadioHardware hw;
LogCallback log;
hw.configure(true, true);
bool reentered = false;
log.set_callback([&](const std::string &message) {
if (!reentered && message.find("set Warning flag") != std::string::npos) {
reentered = true;
hw.stuck = false;
hw.radio.configure();
}
});
hw.stuck = true;
hw.radio.set_mode_standby();
ASSERT_TRUE(reentered);
EXPECT_FALSE(hw.radio.status_has_warning());
EXPECT_TRUE(hw.receiving());
hw.radio.set_mode_sleep();
run_scheduler(1100);
EXPECT_EQ(hw.mode(), MODE_SLEEP);
}
} // namespace esphome::sx127x::testing
@@ -0,0 +1,329 @@
#pragma once
#include <gtest/gtest.h>
#include <array>
#include <functional>
#include <vector>
#include "esphome/components/sx127x/sx127x.h"
#include "esphome/core/application.h"
namespace esphome::sx127x::testing {
class TestPin : public GPIOPin {
public:
void setup() override {}
void pin_mode(gpio::Flags flags) override {}
gpio::Flags get_flags() const override { return gpio::FLAG_NONE; }
size_t dump_summary(char *buffer, size_t len) const override { return snprintf(buffer, len, "test"); }
bool digital_read() override { return this->on_read ? this->on_read() : this->level; }
void digital_write(bool level) override {
this->level = level;
if (this->on_write) {
this->on_write(level);
}
}
bool level{false};
std::function<bool()> on_read;
std::function<void(bool)> on_write;
};
// An SX127x register file behind the real software SPI bus. The first byte of each
// transaction is the address, with bit 7 set for a write; FIFO reads and writes stream.
class RadioHardware {
public:
RadioHardware() {
this->reset_();
this->cs.level = true;
this->cs.on_write = [this](bool high) {
if (!high) {
this->selected_ = true;
this->pos_ = 0;
this->bit_ = 0;
this->byte_ = 0;
} else {
this->selected_ = false;
}
};
this->miso.on_read = [this]() {
EXPECT_TRUE(this->selected_);
if (this->bit_ == 0) {
this->reply_ = this->reply_byte_();
}
const bool bit = (this->reply_ >> (7 - this->bit_)) & 1;
this->byte_ = (this->byte_ << 1) | this->mosi.level;
if (++this->bit_ == 8) {
this->byte_done_(this->byte_);
this->byte_ = 0;
this->bit_ = 0;
this->pos_++;
}
return bit;
};
this->dio0.on_read = [this]() {
this->finish_pending_send_();
return this->dio0_level();
};
this->rst.on_write = [this](bool high) {
if (!high) {
this->reset_();
}
};
this->spi.set_clk(&this->clk);
this->spi.set_mosi(&this->mosi);
this->spi.set_miso(&this->miso);
// call() runs setup() and marks the bus set up, so it accepts devices
this->spi.call();
this->radio.set_spi_parent(&this->spi);
this->radio.set_cs_pin(&this->cs);
this->radio.set_dio0_pin(&this->dio0);
this->radio.set_rst_pin(&this->rst);
this->radio.set_frequency(433920000);
this->radio.set_pa_pin(PA_PIN_BOOST);
this->radio.set_pa_power(17);
this->radio.set_pa_ramp(PA_RAMP_40);
this->radio.set_spreading_factor(7);
this->radio.set_coding_rate(CODING_RATE_4_5);
this->radio.set_preamble_size(8);
this->radio.set_crc_enable(true);
this->radio.set_sync_value({0x12});
}
~RadioHardware() {
// The configuration retry interval (id 0) must not outlive the radio
App.scheduler.cancel_interval(&this->radio, uint32_t{0});
this->radio.spi_teardown();
}
void configure(bool lora, bool rx, bool fixed = false) {
this->radio.set_modulation(lora ? MOD_LORA : MOD_FSK);
this->radio.set_bandwidth(lora ? SX127X_BW_125_0 : SX127X_BW_50_0);
this->radio.set_bitrate(4800);
this->radio.set_deviation(5000);
this->radio.set_packet_mode(true);
this->radio.set_rx_start(rx);
this->radio.set_payload_length(fixed ? 3 : 0);
this->radio.call(); // The normal component lifecycle runs setup().
}
bool lora() const { return (this->regs[REG_OP_MODE] & MOD_LORA) != 0; }
uint8_t mode() const { return this->regs[REG_OP_MODE] & MODE_MASK; }
bool receiving() const { return this->mode() == MODE_RX; }
// A packet arrives, as the chip reports it in its current modulation
void receive(const std::vector<uint8_t> &data, bool crc_error = false) {
this->rx_packet = data;
this->rx_pos_ = 0;
if (this->lora()) {
this->irq |= RX_DONE | (crc_error ? PAYLOAD_CRC_ERROR : 0);
} else if (!crc_error) {
this->payload_ready = true;
// A variable length FSK packet starts with its length
this->fsk_length_pending_ = (this->regs[REG_PACKET_CONFIG_1] & VARIABLE_LENGTH) != 0;
}
}
bool dio0_level() const {
if (this->lora()) {
const uint8_t mapping = this->regs[REG_DIO_MAPPING1] & DIO0_MAPPING_11;
return (this->irq & (mapping == DIO0_MAPPING_01 ? TX_DONE : RX_DONE)) != 0;
}
return this->mode() == MODE_TX ? this->packet_sent : this->payload_ready;
}
SX127x radio;
spi::SPIComponent spi;
TestPin cs, dio0, rst, clk, mosi, miso;
std::array<uint8_t, 128> regs{};
std::vector<uint8_t> rx_packet{0x11, 0x22, 0x33};
std::vector<uint8_t> packet{0x11, 0x22, 0x33};
std::vector<uint8_t> fifo; // bytes written since the last send
std::vector<std::vector<uint8_t>> sent;
uint8_t irq{0};
uint8_t version{0x12};
bool payload_ready{false};
bool packet_sent{false};
// The chip completes a send as soon as it enters TX
bool auto_complete{true};
// Writes to the mode register are ignored, so set_mode_() times out
bool stuck{false};
protected:
void reset_() {
this->regs.fill(0);
this->regs[REG_OP_MODE] = MODE_STDBY;
this->irq = 0;
this->payload_ready = false;
this->packet_sent = false;
this->fifo.clear();
}
uint8_t reply_byte_() {
if (this->pos_ == 0) {
return 0;
}
const uint8_t addr = this->addr_ & 0x7F;
if (this->addr_ & 0x80) {
return 0;
}
if (addr == REG_FIFO) {
if (this->fsk_length_pending_) {
this->fsk_length_pending_ = false;
return this->rx_packet.size();
}
const uint8_t value = this->rx_pos_ < this->rx_packet.size() ? this->rx_packet[this->rx_pos_] : 0;
this->rx_pos_++;
if (!this->lora() && this->rx_pos_ >= this->rx_packet.size()) {
this->payload_ready = false;
}
return value;
}
if (addr == REG_VERSION) {
return this->version;
}
if (addr == REG_IRQ_FLAGS2 || (addr == REG_IRQ_FLAGS && this->lora())) {
this->finish_pending_send_();
}
if (addr == REG_IRQ_FLAGS2) {
return (this->payload_ready ? PAYLOAD_READY : 0) | (this->packet_sent ? PACKET_SENT : 0);
}
if (this->lora()) {
switch (addr) {
case REG_IRQ_FLAGS:
return this->irq;
case REG_NB_RX_BYTES:
return this->rx_packet.size();
case REG_FIFO_RX_CURR_ADDR:
return 0;
case REG_PKT_RSSI_VALUE:
return 100;
case REG_PKT_SNR_VALUE:
return 20;
default:
break;
}
}
return this->regs[addr];
}
void byte_done_(uint8_t value) {
if (this->pos_ == 0) {
this->addr_ = value;
return;
}
if (!(this->addr_ & 0x80)) {
return;
}
const uint8_t addr = this->addr_ & 0x7F;
if (addr == REG_FIFO) {
this->fifo.push_back(value);
return;
}
if (addr == REG_OP_MODE) {
this->write_mode_(value);
return;
}
if (addr == REG_IRQ_FLAGS && this->lora()) {
this->irq &= ~value;
return;
}
if (addr == REG_FIFO_ADDR_PTR) {
this->rx_pos_ = 0;
}
this->regs[addr] = value;
}
void write_mode_(uint8_t value) {
if (this->stuck) {
return;
}
const uint8_t previous = this->mode();
const uint8_t mode = value & MODE_MASK;
// As on an SX1276, a send that is still going out can be stopped by standby or sleep only
if (previous == MODE_TX && !this->send_done_() && mode != MODE_STDBY && mode != MODE_SLEEP) {
return;
}
this->regs[REG_OP_MODE] = value;
if (previous == MODE_TX && mode != MODE_TX) {
this->packet_sent = false;
this->send_pending_ = false; // Stopped before it finished
}
if (mode == MODE_SLEEP) {
this->payload_ready = false;
}
if (mode == MODE_TX && previous != MODE_TX) {
this->sent.push_back(this->fifo);
this->fifo.clear();
// Completes on the next poll, not before the mode reads back as TX
this->send_pending_ = this->auto_complete;
}
}
bool send_done_() const { return this->lora() ? (this->irq & TX_DONE) != 0 : this->packet_sent; }
void finish_pending_send_() {
if (this->send_pending_ && this->mode() == MODE_TX) {
this->complete_send();
}
this->send_pending_ = false;
}
public:
// In LoRa the chip returns to standby by itself; in FSK it stays in TX
void complete_send() {
if (this->lora()) {
this->irq |= TX_DONE;
this->regs[REG_OP_MODE] = (this->regs[REG_OP_MODE] & ~MODE_MASK) | MODE_STDBY;
} else {
this->packet_sent = true;
}
}
protected:
size_t pos_{0};
size_t rx_pos_{0};
uint8_t addr_{0};
uint8_t bit_{0};
uint8_t byte_{0};
uint8_t reply_{0};
bool selected_{false};
bool fsk_length_pending_{false};
bool send_pending_{false};
};
class PacketListener : public SX127xListener {
public:
void on_packet(const std::vector<uint8_t> &packet, float rssi, float snr) override {
this->packets.push_back(packet);
this->rssi = rssi;
this->snr = snr;
if (this->on_receive) {
this->on_receive();
}
}
std::vector<std::vector<uint8_t>> packets;
float rssi{};
float snr{};
std::function<void()> on_receive;
};
class TransmitListener : public SX127xTransmitListener {
public:
void on_transmit_complete(SX127xError result) override {
this->results.push_back(result);
if (this->on_complete) {
this->on_complete(result);
}
}
std::vector<SX127xError> results;
std::function<void(SX127xError)> on_complete;
};
// Runs the scheduler for ms milliseconds of real time, as the main loop would
inline void run_scheduler(uint32_t ms) {
const uint32_t start = millis();
while (millis() - start < ms) {
App.scheduler.call(millis());
delay(5);
}
}
} // namespace esphome::sx127x::testing
@@ -6,9 +6,9 @@
#include <random>
#include <string>
#include "esphome/components/web_server_idf/sse_chunk.h"
#include "esphome/components/web_server/sse_chunk.h"
namespace esphome::web_server_idf::testing {
namespace esphome::web_server::testing {
// The chunk as try_send_nodefer lays it out: header, fields, first "data: ", then the pieces
static std::string build_chunk(const char *message, size_t message_len, const char *event, uint32_t id,
@@ -128,4 +128,4 @@ TEST(SseChunk, MatchesReferenceOnRandomMessages) {
}
}
} // namespace esphome::web_server_idf::testing
} // namespace esphome::web_server::testing
@@ -0,0 +1,300 @@
#ifdef USE_HOST
#include <gtest/gtest.h>
#include <sys/uio.h>
#include <cstdint>
#include <cstdio>
#include <string>
#include <vector>
#include "esphome/components/web_server/sse_chunk.h"
#include "esphome/components/web_server_base/stream_head.h"
namespace esphome::web_server::testing {
using namespace web_server_base; // NOLINT(google-build-using-namespace)
// Collects every batch write_stream_head() sends
struct Capture {
std::string bytes;
std::vector<int> batch_sizes;
int attempts{0};
int fail_after_batches{-1};
bool operator()(struct iovec *iov, int count) {
this->attempts++;
if (static_cast<int>(this->batch_sizes.size()) == this->fail_after_batches) {
return false;
}
this->batch_sizes.push_back(count);
for (int i = 0; i < count; i++) {
this->bytes.append(static_cast<const char *>(iov[i].iov_base), iov[i].iov_len);
}
return true;
}
};
static std::string build_head(Capture &cap, const char *status, const char *content_type,
std::span<const HttpHeader> headers, std::span<const HttpHeader> defaults,
const char *body) {
EXPECT_TRUE(write_stream_head<struct iovec>(cap, status, content_type, headers, defaults, body));
return cap.bytes;
}
// What httpd_resp_send_chunk() put on the wire for its first chunk (esp_http_server/src/httpd_txrx.c in
// ESP-IDF 5.5): the chunked head with Transfer-Encoding, each httpd_resp_set_hdr() header, the blank line,
// then "%lx\r\n", the chunk data and "\r\n".
static std::string reference_send_chunk(const char *status, const char *content_type,
const std::vector<HttpHeader> &resp_hdrs, const char *buf, size_t buf_len) {
char head[256];
std::string out(
head, snprintf(head, sizeof(head), "HTTP/1.1 %s\r\nContent-Type: %s\r\nTransfer-Encoding: chunked\r\n", status,
content_type));
for (const auto &hdr : resp_hdrs) {
out.append(hdr.name).append(": ").append(hdr.value).append("\r\n");
}
out.append("\r\n");
char len_str[10];
out.append(len_str, snprintf(len_str, sizeof(len_str), "%lx\r\n", static_cast<long>(buf_len)));
out.append(buf, buf_len);
out.append("\r\n");
return out;
}
static constexpr HttpHeader DEFAULTS[] = {
{"Access-Control-Allow-Origin", "*"},
{"Access-Control-Allow-Private-Network", "true"},
};
TEST(StreamHead, SseHeadMatchesHttpdRespSendChunk) {
// The old event source set Cache-Control and Connection, then the default headers, and sent "\r\n" as a chunk
const std::string expected = reference_send_chunk(
"200 OK", "text/event-stream",
{{"Cache-Control", "no-cache"}, {"Connection", "keep-alive"}, DEFAULTS[0], DEFAULTS[1]}, "\r\n", 2);
Capture cap;
EXPECT_EQ(build_head(cap, SSE_STATUS, SSE_CONTENT_TYPE, SSE_HEADERS, DEFAULTS, SSE_FIRST_CHUNK), expected);
}
TEST(StreamHead, HeaderReplacesDefaultOfSameName) {
// A default the caller also sets, in any case, goes out once with the caller's value
static constexpr HttpHeader HEADERS[] = {
{"access-control-allow-origin", "http://example.com"},
};
Capture cap;
EXPECT_EQ(build_head(cap, "200 OK", "text/plain", HEADERS, DEFAULTS, nullptr),
"HTTP/1.1 200 OK\r\nContent-Type: text/plain\r\naccess-control-allow-origin: http://example.com"
"\r\nAccess-Control-Allow-Private-Network: true\r\n\r\n");
}
TEST(StreamHead, RawHeadWithCustomHeadersAndNoBody) {
static constexpr HttpHeader HEADERS[] = {
{"Cache-Control", "no-store"},
{"X-Stream", "audio"},
};
Capture cap;
EXPECT_EQ(build_head(cap, "200 OK", "audio/wav", HEADERS, {}, nullptr), "HTTP/1.1 200 OK\r\n"
"Content-Type: audio/wav\r\n"
"Cache-Control: no-store\r\n"
"X-Stream: audio\r\n"
"\r\n");
}
TEST(StreamHead, BareHead) {
Capture cap;
EXPECT_EQ(build_head(cap, "204 No Content", "text/plain", {}, {}, nullptr),
"HTTP/1.1 204 No Content\r\nContent-Type: text/plain\r\n\r\n");
EXPECT_EQ(cap.batch_sizes, std::vector<int>{5});
}
TEST(StreamHead, ManyHeadersSplitIntoBatchesWithSameBytes) {
// 4 + 12 * 4 + 2 = 54 pieces, more than two batches hold
static constexpr HttpHeader HEADERS[] = {
{"X-Header-0", "value-0"}, {"X-Header-1", "value-1"}, {"X-Header-2", "value-2"}, {"X-Header-3", "value-3"},
{"X-Header-4", "value-4"}, {"X-Header-5", "value-5"}, {"X-Header-6", "value-6"}, {"X-Header-7", "value-7"},
{"X-Header-8", "value-8"}, {"X-Header-9", "value-9"},
};
const std::span<const HttpHeader> headers{HEADERS};
Capture batched;
const std::string bytes = build_head(batched, "200 OK", "text/plain", headers, DEFAULTS, "body");
EXPECT_EQ(batched.batch_sizes, (std::vector<int>{STREAM_HEAD_MAX_IOV, STREAM_HEAD_MAX_IOV, 6}));
std::string expected = "HTTP/1.1 200 OK\r\nContent-Type: text/plain";
const auto append_headers = [&expected](std::span<const HttpHeader> list) {
for (const auto &h : list) {
expected.append("\r\n").append(h.name).append(": ").append(h.value);
}
};
append_headers(headers);
append_headers(DEFAULTS);
expected.append("\r\n\r\nbody");
EXPECT_EQ(bytes, expected);
}
TEST(StreamHead, BatchBoundary) {
// 4 + 4 * 4 + 1 + 1 = 22 pieces fit one batch; a fifth header makes 4 + 5 * 4 = 24 before the blank line
static constexpr HttpHeader HEADERS[] = {{"A", "1"}, {"B", "2"}, {"C", "3"}, {"D", "4"}, {"E", "5"}};
Capture four;
build_head(four, "200 OK", "text/plain", std::span(HEADERS, 4), {}, "x");
EXPECT_EQ(four.batch_sizes, std::vector<int>{22});
EXPECT_EQ(stream_head_iov_count(4), 22u);
Capture five;
const std::string bytes = build_head(five, "200 OK", "text/plain", HEADERS, {}, "x");
EXPECT_EQ(five.batch_sizes, (std::vector<int>{STREAM_HEAD_MAX_IOV, 2}));
EXPECT_EQ(bytes, "HTTP/1.1 200 OK\r\nContent-Type: text/plain\r\nA: 1\r\nB: 2\r\nC: 3\r\nD: 4\r\nE: 5\r\n\r\nx");
}
TEST(StreamHead, FailedSendSkipsLaterBatches) {
// Three batches; the second fails, so the third is never sent
std::vector<HttpHeader> headers(12, HttpHeader{"X", "y"});
Capture full;
build_head(full, "200 OK", "text/plain", headers, {}, nullptr);
ASSERT_EQ(full.batch_sizes.size(), 3u);
Capture cap;
cap.fail_after_batches = 1;
EXPECT_FALSE(write_stream_head<struct iovec>(cap, "200 OK", "text/plain", headers, {}, nullptr));
EXPECT_EQ(cap.attempts, 2);
EXPECT_EQ(cap.batch_sizes.size(), 1u);
}
// Three elements of 3, 4 and 5 bytes
struct IovFixture {
char a[4] = "abc";
char b[5] = "defg";
char c[6] = "hijkl";
struct iovec iov[3] = {{a, 3}, {b, 4}, {c, 5}};
struct iovec *cur = iov;
int count = 3;
std::string rest() const {
std::string out;
for (int i = 0; i < this->count; i++) {
out.append(static_cast<const char *>(this->cur[i].iov_base), this->cur[i].iov_len);
}
return out;
}
};
TEST(AdvanceIovecs, ZeroLengthWriteChangesNothing) {
IovFixture f;
advance_iovecs(f.cur, f.count, 0);
EXPECT_EQ(f.cur, f.iov);
EXPECT_EQ(f.count, 3);
EXPECT_EQ(f.rest(), "abcdefghijkl");
}
TEST(AdvanceIovecs, PartialInsideFirstElement) {
IovFixture f;
advance_iovecs(f.cur, f.count, 2);
EXPECT_EQ(f.cur, f.iov);
EXPECT_EQ(f.count, 3);
EXPECT_EQ(f.cur->iov_len, 1u);
EXPECT_EQ(f.rest(), "cdefghijkl");
}
TEST(AdvanceIovecs, ExactlyOnElementBoundary) {
IovFixture f;
advance_iovecs(f.cur, f.count, 3);
EXPECT_EQ(f.cur, &f.iov[1]);
EXPECT_EQ(f.count, 2);
EXPECT_EQ(f.rest(), "defghijkl");
}
TEST(AdvanceIovecs, AcrossElementBoundaries) {
IovFixture f;
advance_iovecs(f.cur, f.count, 9);
EXPECT_EQ(f.cur, &f.iov[2]);
EXPECT_EQ(f.count, 1);
EXPECT_EQ(f.rest(), "jkl");
}
TEST(AdvanceIovecs, RepeatedShortWrites) {
IovFixture f;
std::string sent;
while (f.count > 0) {
const std::string before = f.rest();
advance_iovecs(f.cur, f.count, 2);
sent += before.substr(0, before.size() - f.rest().size());
}
EXPECT_EQ(sent, "abcdefghijkl");
}
TEST(AdvanceIovecs, EverythingWritten) {
IovFixture f;
advance_iovecs(f.cur, f.count, 12);
EXPECT_EQ(f.count, 0);
EXPECT_EQ(f.rest(), "");
}
TEST(AdvanceIovecs, SkipsZeroLengthElements) {
char a[] = "ab";
char c[] = "cd";
struct iovec iov[3] = {{a, 2}, {nullptr, 0}, {c, 2}};
struct iovec *cur = iov;
int count = 3;
advance_iovecs(cur, count, 2);
EXPECT_EQ(cur, &iov[2]);
EXPECT_EQ(count, 1);
}
static constexpr uint32_t TIMEOUT = 20000;
TEST(StallClock, NotStarted) {
StallClock clock;
EXPECT_FALSE(clock.is_running());
EXPECT_FALSE(clock.is_expired(0, TIMEOUT));
EXPECT_FALSE(clock.is_expired(UINT32_MAX, TIMEOUT));
}
TEST(StallClock, ExpiresAtExactlyTheTimeout) {
StallClock clock;
clock.start(1000);
EXPECT_TRUE(clock.is_running());
EXPECT_FALSE(clock.is_expired(1000, TIMEOUT));
EXPECT_FALSE(clock.is_expired(1000 + TIMEOUT - 1, TIMEOUT));
EXPECT_TRUE(clock.is_expired(1000 + TIMEOUT, TIMEOUT));
EXPECT_TRUE(clock.is_expired(1000 + TIMEOUT + 1, TIMEOUT));
EXPECT_EQ(clock.elapsed(1000 + TIMEOUT), TIMEOUT);
}
TEST(StallClock, RunningClockKeepsItsStart) {
StallClock clock;
clock.start(1000);
clock.start(5000);
EXPECT_TRUE(clock.is_expired(1000 + TIMEOUT, TIMEOUT));
}
TEST(StallClock, Cleared) {
StallClock clock;
clock.start(1000);
clock.clear();
EXPECT_FALSE(clock.is_running());
EXPECT_FALSE(clock.is_expired(1000 + TIMEOUT, TIMEOUT));
clock.start(50000);
EXPECT_FALSE(clock.is_expired(50000 + TIMEOUT - 1, TIMEOUT));
EXPECT_TRUE(clock.is_expired(50000 + TIMEOUT, TIMEOUT));
}
TEST(StallClock, StartAtZeroStillRuns) {
StallClock clock;
clock.start(0);
EXPECT_TRUE(clock.is_running());
// Recorded as 1, so it expires one millisecond late
EXPECT_FALSE(clock.is_expired(TIMEOUT, TIMEOUT));
EXPECT_TRUE(clock.is_expired(TIMEOUT + 1, TIMEOUT));
}
TEST(StallClock, AcrossMillisWraparound) {
StallClock clock;
const uint32_t start = UINT32_MAX - 5000;
clock.start(start);
EXPECT_FALSE(clock.is_expired(UINT32_MAX, TIMEOUT));
EXPECT_FALSE(clock.is_expired(start + TIMEOUT - 1, TIMEOUT)); // wrapped to 14998
EXPECT_TRUE(clock.is_expired(start + TIMEOUT, TIMEOUT));
EXPECT_EQ(clock.elapsed(start + TIMEOUT), TIMEOUT);
}
} // namespace esphome::web_server::testing
#endif // USE_HOST