The registry response already carries each package file's size, so the
combined bar is sized from URLSource instead of a HEAD sweep; the probe,
its all-or-nothing fallback, and its duplicate Content-Length parsing are
gone, and archives without a reported size simply stay on the sequential
path. run_batch_downloads now takes (name, size, fetch) jobs and builds
the (now private) progress bar itself, collecting failures from future
results instead of a shared list; both call sites use functools.partial
instead of lambda factories. The prefetch touches only URLSource, so the
progress parameter comes off Source/Git/Local/ConvertedLibrary, and the
wave prefetch is wrapped best-effort like its ESP-IDF twin, which also
dedupes by dest only after the verifiability check. ProgressBar.done()
itself now skips the newline when no frame was drawn or the 100% frame
already ended the line, collapsing the batch wrapper's guard and fixing
the stray blank line after OTA uploads.
Split out of #18662: the converter/manifest hardening without the
download parallelization, which stays there.
The PIO extra-script shim moves from espidf/ to platformio/ and gains
type checks for every manifest field, capture of the Append variants and
CPPPATH, warn-once diagnostics for unmodelled env reads, and uniform
shlex quoting so captured values survive the lex_build_flags round-trip.
The dependency walk normalizes every spelling PIO accepts, splits the
routine cross-platform skip (IncompatiblePlatform) from real manifest
problems, validates manifest shape once, and the shared flag lexer
warns and drops trailing or empty glued arguments.
A failed stderr-sidecar write no longer stamps the cache: the next build
re-runs -E and re-derives the preprocessor diagnostic honestly instead
of losing it for the life of the build dir. The sidecar unlink joins the
best-effort contract. The plain-linker denylist gains the
-nodefaultlibs/-nostdlib/-rdynamic family and the -fuse-ld=/--specs=
prefixes, with a comment naming it best-effort rather than complete.
The damaged-file helper test now lives only in test_helpers.py beside
the OSError-branch coverage.
shlex.split('-D ""') left a bare -D that gcc resolves by eating the
next flag; the funnel now warns and drops it for every consumer, which
also retires raise_on_empty_arg_flags, whose staged ESP8266 caller no
longer has anything to catch.
cancel_futures only drops queued jobs, and the non-daemon workers are
joined at interpreter exit, so a Ctrl-C during the ESP-IDF toolchain
fetch waited out every in-flight archive. The tracker each job reports
through now checks a cancellation flag set when the wait raises, so
downloads abort at their next progress tick and the resumable .part
files keep the bytes already fetched.
The wave-deferral test stubs _content_lengths instead of issuing real
HEAD requests that passed only by failing, the dead hasattr branch in
the HEAD-probe test is gone, and the .C suffix comment names the SCons
divergence instead of claiming parity.
Prepend/AppendUnique/PrependUnique write the same captured keys and now
alias Append instead of no-op'ing away a link directory. env.get on an
unmodelled var warns once so a script taking the default branch is
diagnosable. LIBS/LINKFLAGS/CPPFLAGS entries are shlex-quoted like the
path buckets, since each is one argv token in SCons and re-lexing must
not split a spaced value. The HEAD probes follow the happy-eyeballs
convention, and the tool-prefetch backstop keeps its traceback at debug.
J. Nick Koston