[ci] Add the speaker media player, web_server_idf and MQTT WebSocket flows to the hardware-test skill (#21229)

This commit is contained in:
J. Nick Koston
2026-10-20 15:29:45 -10:00
committed by GitHub
parent d24cadbc8f
commit a66faec424
19 changed files with 776 additions and 5 deletions
+7 -2
View File
@@ -115,7 +115,7 @@ Per-topic scripts, each described in the matching `reference/recipes/<component>
(host driver, Zephyr driver probe), `rtsp_server/` (protocol client, raw rate reader, measurement set,
API image grab), `cse7766/` (API observer with ping), `zephyr_coredump/` (reader flow, decode with the
Zephyr tools), `libretiny_prefs/` (rtc preference harness header), `libretiny_crash/` (BK72xx crash probe header and
capture decoder), `api_outgoing/` (wake cycle and Wi-Fi drop timing readers), `ld2450/` (zone number driver), `dfrobot_sen0395/` (API driver and case runner), `homeassistant_light/` (direct bulb driver, held-report case driver), `usb_audio/` (URL player).
capture decoder), `api_outgoing/` (wake cycle and Wi-Fi drop timing readers), `ld2450/` (zone number driver), `dfrobot_sen0395/` (API driver and case runner), `homeassistant_light/` (direct bulb driver, held-report case driver), `usb_audio/` (URL player), `speaker_media_player/` (API player driver, pipeline task case runner), `web_server/` (SSE client, request mix, SSE and HTTPD case runners), `mqtt/` (broker config with WebSocket listeners, test CA, transport case runner).
## Templates in `$SKILL/templates`
@@ -131,7 +131,8 @@ capture decoder), `api_outgoing/` (wake cycle and Wi-Fi drop timing readers), `l
`homeassistant-light-dut.yaml`, `deep-sleep-outgoing-dut.yaml`, `outgoing-wifi-drop-dut.yaml`,
`globals-storage-dut.yaml`, `usb-audio-waveshare-s3-eth-dut.yaml`, `usb-audio-sendspin-s3-eth-dut.yaml`,
`cast-pro-1-sendspin-control.yaml`, `aux-2-preprod-stock-dut.yaml`, `aux-2-preprod-usb-audio-dut.yaml`,
`dfrobot-sen0395-dut.yaml`.
`dfrobot-sen0395-dut.yaml`, `aux-2-preprod-speaker-player-dut.yaml`, `web-server-sse-dut.yaml`,
`mqtt-transport-dut.yaml`.
## When something looks wrong
@@ -145,6 +146,10 @@ capture decoder), `api_outgoing/` (wake cycle and Wi-Fi drop timing readers), `l
preference and repeat it in a periodic alive line, as `scripts/libretiny_prefs/ltprefs.hpp` does.
- Multicast from the host never arrives on some access points: test it device to device. Broadcast
is lossy under Wi-Fi power save, unicast is not. Throttle UDP floods to a fixed packet rate.
- The merge base comes from a freshly fetched `origin/dev`; a stale local `dev` puts dependency bumps into the
PR diff and builds the control from the wrong commit.
- A `.local` name can stop resolving in the middle of a run; keep the address from the boot log and drive by it.
- `wc -l` pads its count on macOS, so `tail -n +$n0` needs `$((n0+0))` or an arithmetic expansion.
- `api:` reboots a board after 15 min without a client: `reboot_timeout: 0s` on `api:` and `mqtt:` for
soak tests.
@@ -11,7 +11,7 @@ One file per component, named after it:
- [`hdmi_cec.md`](hdmi_cec.md): two-board wire
- [`ld2450.md`](ld2450.md): polygon zones, zone number round trip through the radar's readback
- [`libretiny.md`](libretiny.md): rtc preference emulation, BK72xx crash report on the next boot
- [`mqtt.md`](mqtt.md): broker bench, QoS, reconnect, crash log over MQTT
- [`mqtt.md`](mqtt.md): broker bench, QoS, reconnect, crash log over MQTT, WebSocket transports with a local and a public broker
- [`nrf52.md`](nrf52.md): dongle board support
- [`online_image.md`](online_image.md): stalled and malformed downloads
- [`openthread.md`](openthread.md): border router on the S3 host plus H2 RCP
@@ -28,3 +28,5 @@ One file per component, named after it:
- [`homeassistant.md`](homeassistant.md): light platform mirrored from a local Home Assistant, held reports versus later calls
- [`usb_audio.md`](usb_audio.md): USB audio speaker and microphone, the usb_host FIFO split, and Music Assistant playback through sendspin
- [`dfrobot_sen0395.md`](dfrobot_sen0395.md): SEN0395 mmWave command flow, module quirks and overlong lines through its error reply
- [`speaker.md`](speaker.md): media player pipeline tasks on the Connect AUX-2 with the `speaker` platform player
- [`web_server.md`](web_server.md): state SSE stress and the HTTPD task stack on web_server_idf
@@ -10,3 +10,22 @@ after bulk subscribe and unsubscribe; read the heap seconds later over several r
leak. Spawned subscriptions need about 2.5 s to reach the broker before the next publish.
`esphome logs --device MQTT` only uses the MQTT transport when the config has no `api:`. Driver pattern: a
paho script publishes and polls the serial capture file.
## WebSocket transports on ESP-IDF (#16056)
One build per transport from `templates/mqtt-transport-dut.yaml`, with `scripts/mqtt/mosquitto-ws.conf` as the
local broker (mosquitto 2.x from brew has `websockets` support; `log_type all` prints `New connection from <ip>:<port>
on port <listener>`, which proves which listener each client used) and `scripts/mqtt/make_test_ca.sh BROKER_IP` for
the TLS listener's certificate, both run in the scratch directory so keys, logs and outputs stay out of the checkout. `scripts/mqtt/cases.sh NAME KEY BROKER_HOST SUB_PORT CAPTURE [DEV_HOST] [restart]` subscribes
to `NAME/#`, sends a switch command and an `on_message` topic, presses the 3000 byte publish button over the API,
restarts the local broker when the last argument is `restart` (the PID in `mosquitto.pid`, written when the broker
was started with `mosquitto -c mosquitto-ws.conf & echo $! > mosquitto.pid` in the working directory next to the
certificates), checks its three ports answer again, and prints the broker's message counts with the device log. `mosquitto_sub` cannot speak
WebSocket, so it subscribes on the plain listener of the same broker. Cases: `tcp` on 1883, `ws` on 9001, `wss` on
9002 with `certificate_authority` (the certificate names an address, so `skip_cert_cn_check: true`), `wss` to
test.mosquitto.org on 8081 with no CA for the certificate bundle, and two negatives that must never connect: a
different CA (`mbedtls_ssl_handshake returned -0x2700`) and the self-signed server through the bundle
(`esp-x509-crt-bundle: Failed to verify certificate`, `-0x3000`). `CONFIG_MQTT_TRANSPORT_WEBSOCKET` is on by
default in the generated sdkconfig. The TLS session costs about 29 kB of internal heap while connected; the public
broker carries other devices' `esphome/discover` traffic, which is noise. Results:
https://github.com/esphome/esphome/pull/16056#issuecomment-6301034019
@@ -0,0 +1,19 @@
# speaker
## Media player pipeline tasks (#21203)
The `speaker` media player platform (`speaker/media_player/`) creates `ann_read`, `ann_decode`, `med_read` and
`med_decode` FreeRTOS tasks on every play and deletes them after a stop. The AUX-2 and sendspin packages use the
`speaker_source` player instead, which has its own pipeline and never runs that code, so a test of
`speaker/media_player/*` has to swap the player: `templates/aux-2-preprod-speaker-player-dut.yaml` removes the
package's `external_media_player` with `id: !remove` and adds a `speaker` platform player on the package's resampler
speakers, plus a "Task names" button whose lambda calls `xTaskGetHandle()` on the four names and logs internal free
heap. Serve FLAC files from the host (`ffmpeg -f lavfi -i "sine=frequency=440:duration=90" -ac 2 -ar 48000 -c:a flac
media90.flac`, a 3 s mono one for announcements, `python3 -m http.server 8765 --directory www`); `audio_http`
refuses WAV and MP3. `scripts/speaker_media_player/cases.sh HOST KEY MEDIA_URL ANN_URL CAPTURE LABEL` runs three
cycles through `drive.py` (`play:URL | ann:URL | stop | press:NAME | wait:S` on one API connection) with the probe
pressed idle, during media, during the announcement, after it and after the stop. Pass: idle none, media `med_*`
only, announcement all four, after stop none, every cycle. API commands land a second or two late over Wi-Fi, so read
the probe against the `State changed to` lines rather than the driver's timestamps. Internal free heap after a stop
moved by 2.5 kB between two boots of the same build, so a heap delta needs more than one boot per build. Results:
https://github.com/esphome/esphome/pull/21203#issuecomment-6299581943
@@ -0,0 +1,31 @@
# web_server
## State SSE on web_server_idf (#21109) and the HTTPD task stack (#21038)
`templates/web-server-sse-dut.yaml` (ESP32 classic, ESP-IDF, `web_server` version 3) publishes eight sensors every
200 ms (small state events on the 1024 byte stack buffer) and three text sensors whose JSON carries the string twice
(`value` and `state`): 600 chars for about 1.3 kB (the 2048 byte tail), 1800 chars for about 3.7 kB (tail grown to
4096) and 2800 chars for about 5.7 kB, over the 5120 byte JSON room, which must log `State event does not fit`
and never put a truncated frame on the wire. An `interval` lambda logs `uxTaskGetStackHighWaterMark()` of the loop
task (`nullptr`) and of the HTTPD task (`xTaskGetHandle("httpd")`) every 5 s; the ESPHome loop task is `loopTask`
on ESP32. Keep `power_save_mode: none`, and drive the device by address: mDNS names dropped out mid-run once.
`scripts/web_server/sse.py HOST SECONDS [--clients N] [--stall] [--churn CYCLES]` opens raw TCP connections to
`/events`, decodes the chunked body and parses every `state` event as JSON; `--stall` adds a connection with a 2 kB
receive buffer that never reads (the device must log `Closing stuck EventSource connection` after the 20 s stall
timeout), `--churn` opens and resets a connection CYCLES times. `sse_cases.sh HOST KEY LABEL CAPTURE` runs the
set: 1 client 60 s, 4 clients 60 s, a reader plus a stalled client 45 s, a reader plus 10 abrupt closes, then two
clients with the "Burst" button (50 sensor and 50 text publishes in one loop pass). Pass: 0 invalid JSON on every
client, the oversize sensor dropped with its warning on every publish, the stalled session closed after about
20 s, no reset. The loop task watermark is a minimum since boot, so compare the value after the first client, after
four clients and after the burst between the builds (the SSE split in #21109 left 1280 B more on the ESP32 classic).
For the HTTPD task, `web_server: auth: type: digest` (the default `auth` is Basic) and
`scripts/web_server/httpd_cases.sh HOST USER PASS LABEL CAPTURE`: `httpmix.sh` sends a `curl --anyauth` request
mix per round (page, `/sensor/<Name>`, `/text_sensor/<Name>`, `/switch/<Name>`, a `POST` with `Content-Length: 0`
to `/switch/<Name>/toggle`, an unknown path, which the IDF server closes without a response, an unauthenticated
request answered 401, and a 2 s `/events` stream), 20 rounds alone and 12 rounds next to two `ssecurl.sh` SSE
clients. Entity URLs use the entity name, URL encoded, not an object id. The HTTPD watermark follows the deepest
request of the boot; compare after the mix and after the SSE round, with two boots when the numbers are close.
Results: https://github.com/esphome/esphome/pull/21109#issuecomment-6300582328 and
https://github.com/esphome/esphome/pull/21038#issuecomment-6301310182
@@ -1,6 +1,6 @@
"""Press a template button on the test device over the API, or list its buttons.
usage: api_button.py HOST KEY list | press "Button name" [...]
usage: api_button.py HOST KEY list | press "Button name" [...] (KEY of - means a plaintext API)
"""
import asyncio
@@ -13,7 +13,7 @@ CMD, ARGS = sys.argv[3], sys.argv[4:]
async def main() -> None:
c = APIClient(HOST, 6053, None, noise_psk=KEY)
c = APIClient(HOST, 6053, None, noise_psk=None if KEY == "-" else KEY)
await c.connect(login=True)
ents, _ = await c.list_entities_services()
buttons = {e.name: e for e in ents if isinstance(e, ButtonInfo)}
+42
View File
@@ -0,0 +1,42 @@
#!/bin/sh
# cases.sh NAME KEY BROKER_HOST SUB_PORT CAPTURE [DEV_HOST] [restart] : subscribe to NAME/# on the broker, send a
# switch command and an on_message topic, press the 3000 byte publish button over the API (KEY, - for plaintext),
# optionally restart the local broker (its PID from mosquitto.pid in the working directory, written when it was started
# with `mosquitto -c mosquitto-ws.conf & echo $! > mosquitto.pid`), and show the device log. Outputs (sub-NAME.txt, mosquitto.out) go to the working directory, which should be the scratch directory.
N=$1; KEY=$2; B=$3; SP=$4; LOG=$5; DEV=${6:-$1.local}; RESTART=$7; HERE=$(cd "$(dirname "$0")" && pwd); PY=${PYTHON:-python3}; WORK=$PWD
BTN=$HERE/../api_button.py
[ -n "$KEY" ] && [ -r "$LOG" ] || { echo "usage: cases.sh NAME KEY BROKER_HOST SUB_PORT CAPTURE [DEV_HOST] [restart] (CAPTURE must be a readable file)"; exit 2; }
show() { tail -n +$(($1+1)) "$LOG" | sed 's/\x1b\[[0-9;]*m//g' | grep -a 'mqtt\|\[probe:\|cmd payload\|Relay\|\[W\]\|\[E\]\|Guru\|abort\|Backtrace\|ESPHome version' | grep -a -v 'took a long time\|Publish.*Counter\|sensor/counter' | cut -c1-170; }
pub() { mosquitto_pub -h "$B" -p "$SP" -t "$1" -m "$2" || { echo "mosquitto_pub to $1 failed"; exit 1; }; }
n0=$(($(wc -l < "$LOG")+0))
timeout 75 mosquitto_sub -h "$B" -p "$SP" -v -t "$N/#" > "$WORK/sub-$N.txt" 2> "$WORK/sub-$N.err" &
sub_pid=$!
sleep 20
echo "=== $N: command to the switch and the 3000 byte publish ($(date +%T))"
pub "$N/switch/relay/command" ON; sleep 2
pub "$N/cmd" "hello-over-$N"; sleep 2
$PY "$BTN" "$DEV" "$KEY" press "Publish big" || { echo "button press failed"; exit 1; }; sleep 3
pub "$N/switch/relay/command" OFF; sleep 2
if [ "$RESTART" = restart ]; then
[ -f "$WORK/mosquitto-ws.conf" ] || { echo "no mosquitto-ws.conf in $WORK, cannot restart the broker"; exit 1; }
echo "=== $N: broker restart ($(date +%T))"; n1=$(($(wc -l < "$LOG")+0))
# Only this run's broker is signalled: its PID comes from mosquitto.pid, written when it was started
pid=$(cat "$WORK/mosquitto.pid" 2>/dev/null); [ -n "$pid" ] || { echo "no mosquitto.pid in $WORK, cannot restart the broker"; exit 1; }
kill "$pid" || { echo "broker PID $pid is not running, nothing to restart"; exit 1; }; sleep 8
nc -z -w 3 "$B" 1883 && { echo "the old broker still answers on 1883 after the kill"; exit 1; }
(cd "$WORK" && mosquitto -c mosquitto-ws.conf > mosquitto.out 2>&1 & echo $! > "$WORK/mosquitto.pid"); sleep 25
for port in 1883 9001 9002; do nc -z -w 3 "$B" "$port" || { echo "broker port $port not listening after the restart"; exit 1; }; done
tail -n +$((n1+1)) "$LOG" | grep -a -q "MQTT_EVENT_CONNECTED\|\[mqtt:[0-9]*\]: Connected" || { echo "the device did not reconnect after the broker restart"; exit 1; }
else
echo "=== $N: no broker restart (pass 'restart' as the 7th argument for a local broker)"
fi
wait $sub_pid; sub_rc=$?
[ "$sub_rc" = 124 ] || { echo "subscriber ended with status $sub_rc: $(cat "$WORK/sub-$N.err")"; exit 1; }
echo "--- broker saw ($(grep -c . "$WORK/sub-$N.txt") messages): $(cut -c1-60 "$WORK/sub-$N.txt" | awk '{print $1}' | sort | uniq -c | tr '\n' ';')"
big=$(grep "^$N/big " "$WORK/sub-$N.txt" | awk '{print length($2)}' | tr '\n' ' ')
echo "--- big payload bytes seen: $big"
echo "--- device log:"; show "$n0"
case " $big " in *" 3000 "*) ;; *) echo "the 3000 byte payload did not reach the broker"; exit 1;; esac
for want in "Relay ON" "cmd payload: hello-over-$N" "Relay OFF"; do
tail -n +$((n0+1)) "$LOG" | grep -a -q "$want" || { echo "the device never logged '$want': the broker to device path failed"; exit 1; }
done
+13
View File
@@ -0,0 +1,13 @@
#!/bin/sh
# make_test_ca.sh BROKER_IP : a throwaway CA and a server certificate for the broker address (30 days), written into
# the working directory; run it in the scratch directory where a copy of mosquitto-ws.conf lives. The device
# config takes ca.crt as certificate_authority (skip_cert_cn_check: true, the name check does not match an address);
# other-ca.crt is a second CA for the negative case.
set -e
IP=$1
openssl req -x509 -newkey rsa:2048 -nodes -keyout ca.key -out ca.crt -days 30 -subj "/CN=hwtest-ca"
openssl req -newkey rsa:2048 -nodes -keyout server.key -out server.csr -subj "/CN=hwtest-broker"
printf "subjectAltName=IP:%s\nextendedKeyUsage=serverAuth\n" "$IP" > san.ext
openssl x509 -req -in server.csr -CA ca.crt -CAkey ca.key -CAcreateserial -out server.crt -days 30 -extfile san.ext
openssl req -x509 -newkey rsa:2048 -nodes -keyout other-ca.key -out other-ca.crt -days 30 -subj "/CN=other-ca"
openssl verify -CAfile ca.crt server.crt
@@ -0,0 +1,14 @@
# Copy next to the certificates in the scratch directory and start it there with
# `mosquitto -c mosquitto-ws.conf & echo $! > mosquitto.pid` (cases.sh restarts that PID): plain MQTT, MQTT over WebSocket, and MQTT over WebSocket with TLS (server.crt and
# server.key from make_test_ca.sh in the same directory). log_type all gives the listener port per connection.
listener 1883
protocol mqtt
listener 9001
protocol websockets
listener 9002
protocol websockets
certfile server.crt
keyfile server.key
allow_anonymous true
log_type all
log_dest file mosquitto.log
@@ -0,0 +1,12 @@
#!/bin/sh
# cases.sh HOST KEY MEDIA_URL ANN_URL CAPTURE LABEL : three play, announce, stop cycles on a speaker media player with the
# "Task names" probe button pressed idle, during media, during the announcement, after it and after the stop.
# KEY of - is plaintext. Serve FLAC files with `python3 -m http.server 8765 --directory www` (audio_http refuses WAV and MP3).
H=$1; K=$2; MEDIA=$3; ANN=$4; LOG=$5; LABEL=$6; PY=${PYTHON:-python3}; HERE=$(dirname "$0")
[ -r "$LOG" ] || { echo "usage: cases.sh HOST KEY MEDIA_URL ANN_URL CAPTURE LABEL (CAPTURE must be a readable file)"; exit 2; }
show() { tail -n +$(($1+1)) "$LOG" | sed 's/\x1b\[[0-9;]*m//g' | grep -a '\[tasks:\|speaker_media_player\|speaker_mixer\|\[W\]\|\[E\]\|Guru\|abort\|Backtrace' | grep -a -v 'took a long time' | cut -c1-180; }
for cycle in 1 2 3; do
n0=$(wc -l < "$LOG"); echo "=== $LABEL cycle $cycle ($(date +%T))"
$PY "$HERE/drive.py" "$H" "$K" "press:Task names" wait:1 "play:$MEDIA" wait:5 "press:Task names" wait:1 "ann:$ANN" wait:1.5 "press:Task names" wait:5 "press:Task names" wait:1 stop wait:3 "press:Task names" || { rc=$?; echo "driver failed ($rc), aborting"; exit $rc; }
sleep 1; echo "--- device log:"; show "$n0"
done
@@ -0,0 +1,50 @@
"""Drive the speaker platform media player (the first media player found) over one API connection.
usage: drive.py HOST KEY op [op ...]
ops: play:URL | ann:URL | stop | press:BUTTON | wait:S
"""
import asyncio
import sys
from aioesphomeapi import APIClient, ButtonInfo, MediaPlayerCommand, MediaPlayerInfo
HOST, KEY, OPS = sys.argv[1], sys.argv[2], sys.argv[3:]
async def main() -> None:
c = APIClient(HOST, 6053, None, noise_psk=None if KEY == "-" else KEY)
await c.connect(login=True)
try:
await run_ops(c)
finally:
await c.disconnect()
async def run_ops(c: APIClient) -> None:
ents, _ = await c.list_entities_services()
player = next((e for e in ents if isinstance(e, MediaPlayerInfo)), None)
if player is None:
sys.exit("no media player on this device")
buttons = {e.name: e for e in ents if isinstance(e, ButtonInfo)}
for op in OPS:
kind, _, rest = op.partition(":")
print(f"{asyncio.get_running_loop().time():.1f} {op}", flush=True)
if kind == "play":
c.media_player_command(player.key, media_url=rest)
elif kind == "ann":
c.media_player_command(player.key, media_url=rest, announcement=True)
elif kind == "stop":
c.media_player_command(player.key, command=MediaPlayerCommand.STOP)
elif kind == "press":
if rest not in buttons:
sys.exit(f"no button {rest!r}; have {sorted(buttons)}")
c.button_command(buttons[rest].key)
elif kind == "wait":
await asyncio.sleep(float(rest))
else:
sys.exit(f"unknown op {op!r}")
await asyncio.sleep(0.5)
asyncio.run(main())
@@ -0,0 +1,17 @@
#!/bin/sh
# httpd_cases.sh HOST USER PASS LABEL CAPTURE : HTTPD task stack load for the web_server_idf task size; probe lines from
# CAPTURE, SSE client output into the working directory. A failed step ends the run with its status.
H=$1; U=$2; P=$3; LABEL=$4; LOG=$5; HERE=$(dirname "$0")
[ -r "$LOG" ] || { echo "usage: httpd_cases.sh HOST USER PASS LABEL CAPTURE (CAPTURE must be a readable file)"; exit 2; }
probes() { tail -n +$(($1+1)) "$LOG" | sed 's/\x1b\[[0-9;]*m//g' | grep -a -o "\[[0-9:]*\]\[I\]\[probe:[0-9]*\]: .*httpd min free stack [0-9]*\|\[[0-9:]*\]\[[WE]\]\[[a-z_.]*:[0-9]*\].*\|Guru.*" | grep -a -v "took a long time\|does not fit\|\]\[httpd\]: /" | sed 's/\[I\]\[probe:[0-9]*\]: loopTask min free stack \([0-9]*\), httpd min free stack \([0-9]*\)/ loop \1 httpd \2/' | uniq -f1 | cut -c1-140; }
step() { n0=$(wc -l < "$LOG"); echo "=== $LABEL: $1 ($(date +%T))"; shift; "$@" || { rc=$?; echo "step failed ($rc)"; exit $rc; }; echo "--- device log:"; probes "$n0"; }
step "idle 15 s" sleep 15
step "request mix, 20 rounds" sh "$HERE/httpmix.sh" "$H" "$U" "$P" 20
# Two SSE clients next to the request mix; arguments stay quoted, nothing is reparsed as shell code
concurrent() {
sh "$HERE/ssecurl.sh" "$H" "$U" "$P" 40 "$PWD/sse1-$LABEL.txt" & p1=$!
sh "$HERE/ssecurl.sh" "$H" "$U" "$P" 40 "$PWD/sse2-$LABEL.txt" & p2=$!
sh "$HERE/httpmix.sh" "$H" "$U" "$P" 12 && wait $p1 && wait $p2
}
step "2 SSE clients 40 s with the request mix alongside" concurrent
step "idle 15 s after" sleep 15
+26
View File
@@ -0,0 +1,26 @@
#!/bin/sh
# httpmix.sh HOST USER PASS ROUNDS : authenticated request mix (curl --anyauth follows the server's Basic or Digest
# challenge): page, two json endpoints, a switch POST, a 404, an unauthenticated 401 and a 2 s SSE stream per round
H=$1; U=$2; P=$3; N=$4; ok=0; bad=0
# chk EXPECTED LABEL curl-args... : only LABEL is printed on a miss, never the argument list with the credentials
chk() { exp=$1; label=$2; shift 2; c=$(curl -s -m 6 -o /dev/null -w "%{http_code}" "$@"); if [ "$c" = "$exp" ]; then ok=$((ok+1)); else bad=$((bad+1)); echo "expected $exp got $c: $label"; fi; }
sse_tmp=$(mktemp)
for i in $(seq 1 $N); do
page=$(curl -s -m 6 -o /dev/null -w "%{http_code}" -u "$U:$P" --anyauth "http://$H/")
if [ "$page" = 200 ]; then ok=$((ok+1)); else bad=$((bad+1)); echo "expected 200 got $page: /"; fi
chk 200 "GET /sensor/Fast%201" -u "$U:$P" --anyauth "http://$H/sensor/Fast%201"
chk 200 "GET /text_sensor/Big" -u "$U:$P" --anyauth "http://$H/text_sensor/Big"
chk 200 "GET /switch/Toggle" -u "$U:$P" --anyauth "http://$H/switch/Toggle"
chk 200 "POST /switch/Toggle/toggle" -u "$U:$P" --anyauth -X POST -H "Content-Length: 0" "http://$H/switch/Toggle/toggle"
# The IDF server closes an unknown path without a response: curl ends with 52 (empty reply) or 56 (reset), never a
# timeout (28) or a refused connection (7); only trust the close when the page just answered
c=$(curl -s -m 6 -o /dev/null -w "%{http_code}" -u "$U:$P" --anyauth "http://$H/nonexistent"); rc=$?
if [ "$c" = 000 ] && [ "$page" = 200 ] && { [ "$rc" = 52 ] || [ "$rc" = 56 ]; }; then ok=$((ok+1)); else bad=$((bad+1)); echo "unknown path: got $c, curl status $rc, with the page at $page"; fi
chk 401 "GET / without credentials" "http://$H/"
timeout 2 curl -s -N -u "$U:$P" --anyauth -o "$sse_tmp" "http://$H/events" 2>/dev/null; rc=$?
if [ "$rc" = 124 ] && grep -q "^event: state" "$sse_tmp"; then ok=$((ok+1)); else bad=$((bad+1)); echo "SSE stream: curl status $rc, $(grep -c '^event: state' "$sse_tmp") state events"; fi
sleep 1
done
rm -f "$sse_tmp"
echo "httpmix: $ok ok, $bad unexpected"
[ "$bad" = 0 ]
@@ -0,0 +1,187 @@
"""SSE client(s) for a web_server /events stream: counts state events and checks each is valid JSON.
usage: sse.py HOST SECONDS [--clients N] [--stall] [--churn CYCLES]
--stall opens one extra connection that never reads (the device's socket buffer fills); --churn opens
and abruptly closes a connection CYCLES times, 2 s apart, next to the reading clients.
"""
import argparse
from collections import Counter
import contextlib
import json
import socket
import sys
import threading
import time
ap = argparse.ArgumentParser()
ap.add_argument("host")
ap.add_argument("seconds", type=float)
ap.add_argument("--clients", type=int, default=1)
ap.add_argument("--stall", action="store_true")
ap.add_argument("--churn", type=int, default=0)
args = ap.parse_args()
REQ = b"GET /events HTTP/1.1\r\nHost: %s\r\nAccept: text/event-stream\r\nConnection: keep-alive\r\n\r\n"
def read_chunked(sock, deadline, on_line):
"""Decode HTTP chunked transfer encoding and feed complete event-stream lines to on_line."""
buf = b""
body = b""
headers_done = False
sock.settimeout(1.0)
while time.monotonic() < deadline:
try:
data = sock.recv(65536)
except TimeoutError:
continue
if not data:
return "closed"
buf += data
if not headers_done:
i = buf.find(b"\r\n\r\n")
if i < 0:
continue
headers_done = True
buf = buf[i + 4 :]
while True:
i = buf.find(b"\r\n")
if i < 0:
break
size = int(buf[:i].split(b";")[0], 16)
if len(buf) < i + 2 + size + 2:
break
body += buf[i + 2 : i + 2 + size]
buf = buf[i + 2 + size + 2 :]
if size == 0:
return "closed"
while b"\n" in body:
line, body = body.split(b"\n", 1)
on_line(line.rstrip(b"\r"))
return "deadline"
class Client(threading.Thread):
def __init__(self, idx, seconds):
super().__init__(daemon=True)
self.idx = idx
self.seconds = seconds
self.events = Counter()
self.ids = Counter()
self.invalid = 0
self.max_data = 0
self.bytes = 0
self.end = "?"
self.cur_event = None
def on_line(self, line):
self.bytes += len(line) + 1
if line.startswith(b"event:"):
self.cur_event = line[6:].strip().decode()
elif line.startswith(b"data:"):
ev = self.cur_event or "message"
self.events[ev] += 1
data = line[5:].strip()
self.max_data = max(self.max_data, len(data))
if ev in ("state", "state_detail_all"):
try:
obj = json.loads(data)
self.ids[obj.get("id", "?")] += 1
except ValueError:
self.invalid += 1
elif line == b"":
self.cur_event = None
def run(self):
try:
s = socket.create_connection((args.host, 80), timeout=5)
s.sendall(REQ % args.host.encode())
self.end = read_chunked(s, time.monotonic() + self.seconds, self.on_line)
s.close()
except (OSError, ValueError) as e:
self.end = f"error: {e}"
clients = [Client(i, args.seconds) for i in range(args.clients)]
for c in clients:
c.start()
time.sleep(0.3)
stall_sock = None
if args.stall:
# The window scale is negotiated in the SYN, so the small buffer must be set before connect()
stall_sock = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
stall_sock.setsockopt(socket.SOL_SOCKET, socket.SO_RCVBUF, 2048)
stall_sock.settimeout(5)
stall_sock.connect((args.host, 80))
stall_sock.sendall(REQ % args.host.encode())
print("stalled client opened (never reads)", flush=True)
churn_done = 0
for _ in range(args.churn):
try:
cs = socket.create_connection((args.host, 80), timeout=5)
cs.sendall(REQ % args.host.encode())
cs.settimeout(1.0)
with contextlib.suppress(TimeoutError):
cs.recv(4096)
cs.setsockopt(
socket.SOL_SOCKET, socket.SO_LINGER, b"\x01\x00\x00\x00\x00\x00\x00\x00"
) # RST on close
cs.close()
churn_done += 1
except OSError as e:
print("churn error", e, flush=True)
time.sleep(2)
if args.churn:
print(f"churn: {churn_done}/{args.churn} abrupt closes", flush=True)
for c in clients:
c.join()
stall_closed = None
if stall_sock is not None:
# Drain what the small buffer holds: a close shows as b"" or a reset after it. A session the device left
# open keeps feeding events, so the drain has its own deadline and data past it means "still open".
stall_sock.settimeout(2.0)
drain_deadline = time.monotonic() + 5.0
try:
while stall_sock.recv(65536):
if time.monotonic() > drain_deadline:
break
else:
stall_closed = True
except TimeoutError:
pass
except OSError:
stall_closed = True
if stall_closed is None:
stall_closed = False
print(
"stalled client: closed by device"
if stall_closed
else "stalled client: still open",
flush=True,
)
stall_sock.close()
for c in clients:
states = c.events["state"]
print(
f"client {c.idx}: {c.end}, {states} state events, {c.invalid} invalid JSON, max data {c.max_data} B, "
f"{c.bytes} B, other events { {k: v for k, v in c.events.items() if k != 'state'} }",
flush=True,
)
print(" per id:", dict(sorted(c.ids.items())), flush=True)
failed = [
c for c in clients if c.end != "deadline" or c.invalid or not c.events["state"]
]
if failed:
sys.exit(
f"{len(failed)} client(s) did not run clean to the deadline with valid state events"
)
if stall_closed is False:
sys.exit("stalled session was not closed by the device")
if churn_done < args.churn:
sys.exit(f"only {churn_done} of {args.churn} churn connections were made")
@@ -0,0 +1,19 @@
#!/bin/sh
# sse_cases.sh HOST KEY LABEL CAPTURE : the SSE cases against templates/web-server-sse-dut.yaml with a console capture
# running into CAPTURE; KEY is the API key (- for plaintext). The burst client output goes to the working directory.
H=$1; K=$2; LABEL=$3; LOG=$4; PY=${PYTHON:-python3}; HERE=$(dirname "$0"); BTN=$HERE/../api_button.py
[ -r "$LOG" ] || { echo "usage: sse_cases.sh HOST KEY LABEL CAPTURE (CAPTURE must be a readable file)"; exit 2; }
show() { tail -n +$(($1+1)) "$LOG" | sed 's/\x1b\[[0-9;]*m//g' | grep -a '\[probe:\|\[W\]\|\[E\]\|web_server\|Guru\|abort\|Backtrace\|ESPHome version' | grep -a -v 'took a long time' | cut -c1-200; }
step() { n0=$(wc -l < "$LOG"); echo "=== $LABEL: $1 ($(date +%T))"; shift; "$@" || { rc=$?; echo "client failed ($rc), aborting"; exit $rc; }; echo "--- device log:"; show "$n0"; }
step "idle 15 s, no client" sleep 15
step "1 client 60 s" $PY "$HERE/sse.py" "$H" 60 --clients 1
step "4 clients 60 s" $PY "$HERE/sse.py" "$H" 60 --clients 4
step "1 reader plus a stalled client 45 s" $PY "$HERE/sse.py" "$H" 45 --clients 1 --stall
step "1 reader plus 10 abrupt closes 30 s" $PY "$HERE/sse.py" "$H" 30 --clients 1 --churn 10
n0=$(wc -l < "$LOG"); echo "=== $LABEL: 2 clients 25 s with the Burst button at 5 s ($(date +%T))"
$PY "$HERE/sse.py" "$H" 25 --clients 2 > "$PWD/burst-$LABEL.txt" 2>&1 &
pid=$!
sleep 5; $PY "$BTN" "$H" "$K" press "Burst" || { echo "button press failed"; exit 1; }
wait $pid || { cat "$PWD/burst-$LABEL.txt"; echo "SSE clients failed during the burst"; exit 1; }
cat "$PWD/burst-$LABEL.txt"; echo "--- device log:"; show "$n0"
step "idle 15 s after" sleep 15
+25
View File
@@ -0,0 +1,25 @@
#!/bin/sh
# ssecurl.sh HOST USER PASS SECONDS OUT : one authenticated SSE client through curl; counts state events and checks JSON
H=$1; U=$2; P=$3; T=$4; OUT=$5
timeout "$T" curl -sS -f -N -u "$U:$P" --anyauth "http://$H/events" > "$OUT"; rc=$?
[ "$rc" = 124 ] || { echo "sse: curl ended with status $rc before the $T s deadline"; exit 1; }
python3 - "$OUT" <<'PY'
import json, sys
states = invalid = 0; biggest = 0; ev = None
data = open(sys.argv[1], "rb").read()
lines = data.split(b"\n")
if not data.endswith(b"\n"):
lines = lines[:-1] # the deadline can cut the last line in half; only complete lines are checked
for line in lines:
line = line.rstrip(b"\r")
if line.startswith(b"event:"):
ev = line[6:].strip()
elif line.startswith(b"data:") and ev == b"state":
states += 1; d = line[5:].strip(); biggest = max(biggest, len(d))
try: json.loads(d)
except ValueError: invalid += 1
elif line == b"":
ev = None
print(f"sse: {states} state events, {invalid} invalid JSON, largest {biggest} B")
sys.exit(1 if invalid or not states else 0)
PY
@@ -0,0 +1,66 @@
# Needs packages/aux-2-pre-prod.yaml from NabuCasa/aux-2 at commit 283f2e64 next to this file (see
# reference/recipes/usb_audio.md). The AUX-2 package with the package's speaker_source player swapped for the speaker
# platform player, which is the one built on speaker/media_player/audio_pipeline.cpp, plus the task name probe.
packages:
core: !include aux-2-pre-prod.yaml
esphome:
name: aux-2-dut
name_add_mac_suffix: false
wifi:
ssid: !secret wifi_ssid
password: !secret wifi_password
api:
encryption:
key: !secret api_key
logger:
level: DEBUG
sendspin:
unpaired_access: true
media_player:
- id: !remove external_media_player
- platform: speaker
id: speaker_media_player
name: Media Player
volume_increment: 0.05
volume_min: 0.4
volume_max: 0.85
announcement_pipeline:
speaker: announcement_resampling_speaker
format: FLAC
num_channels: 1
sample_rate: 48000
media_pipeline:
speaker: media_resampling_speaker
format: FLAC
num_channels: 2
sample_rate: 48000
on_announcement:
- mixer_speaker.apply_ducking:
id: media_mixing_input
decibel_reduction: 20
duration: 0.0s
button:
- platform: template
name: Task names
on_press:
- lambda: |-
static const char *const NAMES[] = {"ann_read", "ann_decode", "med_read", "med_decode"};
for (const char *n : NAMES) {
TaskHandle_t h = xTaskGetHandle(n);
if (h == nullptr) {
ESP_LOGI("tasks", "%s: missing", n);
} else {
ESP_LOGI("tasks", "%s: found as '%s', stack high water %u", n, pcTaskGetName(h),
(unsigned) uxTaskGetStackHighWaterMark(h));
}
}
ESP_LOGI("tasks", "internal free %u, largest block %u", (unsigned) heap_caps_get_free_size(MALLOC_CAP_INTERNAL),
(unsigned) heap_caps_get_largest_free_block(MALLOC_CAP_INTERNAL));
@@ -0,0 +1,88 @@
# MQTT transport test (#16056, needs that PR's tree for the transport option): one build per transport, name and mqtt block per file (see reference/recipes/mqtt.md).
esp32:
board: esp32dev
framework:
type: esp-idf
logger:
level: VERBOSE
logs:
wifi: DEBUG
api: DEBUG
api.connection: DEBUG
scheduler: DEBUG
component: DEBUG
sensor: DEBUG
esp32: DEBUG
wifi:
ssid: !secret wifi_ssid
password: !secret wifi_password
power_save_mode: none
api:
encryption:
key: !secret api_key
reboot_timeout: 0s
ota:
- platform: esphome
interval:
- interval: 10s
then:
- lambda: |-
ESP_LOGI("probe", "mqtt connected %d, internal free %u, largest block %u", id(mq).is_connected(),
(unsigned) heap_caps_get_free_size(MALLOC_CAP_INTERNAL),
(unsigned) heap_caps_get_largest_free_block(MALLOC_CAP_INTERNAL));
sensor:
- platform: template
name: Counter
update_interval: 2s
accuracy_decimals: 0
lambda: |-
static uint32_t n = 0;
return n++;
switch:
- platform: template
name: Relay
id: relay
optimistic: true
on_turn_on:
- logger.log: "Relay ON"
on_turn_off:
- logger.log: "Relay OFF"
button:
- platform: template
name: Publish big
on_press:
- mqtt.publish:
topic: ${name}/big
payload: !lambda |-
return std::string(3000, 'q');
substitutions:
name: mqtt-dut
esphome:
name: ${name}
mqtt:
id: mq
broker: !secret broker_ip # test.mosquitto.org for the public bundle run
port: 9002 # 1883 tcp, 9001 ws, 9002 wss, 8081 public wss
transport: wss # drop for tcp
ws_path: /mqtt # ws and wss only
certificate_authority: !secret mqtt_ca # drop for the bundle runs
skip_cert_cn_check: true # with the local CA only: the certificate names an address
reboot_timeout: 0s
discovery: false
on_message:
- topic: ${name}/cmd
then:
- logger.log:
format: "cmd payload: %s"
args: ["x.c_str()"]
@@ -0,0 +1,136 @@
# web_server_idf state SSE and HTTPD task stress (#21109, #21038). Add `auth:` with `type: digest` for the HTTPD
# stack case; the entity endpoints are named after the entities (/sensor/Fast%201, /switch/Toggle/toggle).
esphome:
name: web-server-dut
esp32:
board: esp32dev
framework:
type: esp-idf
logger:
level: DEBUG
wifi:
ssid: !secret wifi_ssid
password: !secret wifi_password
power_save_mode: none
api:
encryption:
key: !secret api_key
reboot_timeout: 0s
ota:
- platform: esphome
web_server:
port: 80
version: 3
# Stack and heap probe from the main loop task (loopTask) and the HTTPD task.
interval:
- interval: 5s
then:
- lambda: |-
TaskHandle_t httpd = xTaskGetHandle("httpd");
if (httpd == nullptr) {
ESP_LOGW("probe", "httpd task not found, loopTask min free stack %u",
(unsigned) uxTaskGetStackHighWaterMark(nullptr));
return;
}
ESP_LOGI("probe", "loopTask min free stack %u, httpd min free stack %u, internal free %u, largest block %u",
(unsigned) uxTaskGetStackHighWaterMark(nullptr), (unsigned) uxTaskGetStackHighWaterMark(httpd),
(unsigned) heap_caps_get_free_size(MALLOC_CAP_INTERNAL),
(unsigned) heap_caps_get_largest_free_block(MALLOC_CAP_INTERNAL));
# Small state events: eight sensors at 200 ms, well under the 1024 byte stack buffer.
sensor:
- platform: template
name: Fast 1
id: fast1
update_interval: 200ms
accuracy_decimals: 3
lambda: return random_float() * 100.0f;
- platform: template
name: Fast 2
update_interval: 200ms
accuracy_decimals: 3
lambda: return random_float() * 100.0f;
- platform: template
name: Fast 3
update_interval: 200ms
accuracy_decimals: 3
lambda: return random_float() * 100.0f;
- platform: template
name: Fast 4
update_interval: 200ms
accuracy_decimals: 3
lambda: return random_float() * 100.0f;
- platform: template
name: Fast 5
update_interval: 200ms
accuracy_decimals: 3
lambda: return random_float() * 100.0f;
- platform: template
name: Fast 6
update_interval: 200ms
accuracy_decimals: 3
lambda: return random_float() * 100.0f;
- platform: template
name: Fast 7
update_interval: 200ms
accuracy_decimals: 3
lambda: return random_float() * 100.0f;
- platform: template
name: Fast 8
update_interval: 200ms
accuracy_decimals: 3
lambda: return random_float() * 100.0f;
# Large state events: the text sensor JSON carries the string twice (value and state).
# mid ~1.3 kB goes to the 2048 byte tail, big ~3.7 kB grows the tail to 4096, huge ~5.7 kB exceeds the
# 5120 byte JSON room and must be dropped with "State event does not fit".
text_sensor:
- platform: template
name: Mid
id: mid
update_interval: 1s
lambda: |-
static uint32_t n = 0;
std::string s = std::to_string(n++) + ":";
s.append(600 - s.size(), 'm');
return s;
- platform: template
name: Big
id: big
update_interval: 2s
lambda: |-
static uint32_t n = 0;
std::string s = std::to_string(n++) + ":";
s.append(1800 - s.size(), 'b');
return s;
- platform: template
name: Huge
id: huge
update_interval: 7s
lambda: |-
static uint32_t n = 0;
std::string s = std::to_string(n++) + ":";
s.append(2800 - s.size(), 'h');
return s;
switch:
- platform: template
name: Toggle
optimistic: true
button:
- platform: template
name: Burst
on_press:
- lambda: |-
for (int i = 0; i < 50; i++) {
id(fast1).publish_state(i);
id(mid).publish_state(std::string(600, 'x'));
}