Abort in-flight batch downloads on Ctrl-C, make the deferral test hermetic

cancel_futures only drops queued jobs, and the non-daemon workers are
joined at interpreter exit, so a Ctrl-C during the ESP-IDF toolchain
fetch waited out every in-flight archive. The tracker each job reports
through now checks a cancellation flag set when the wait raises, so
downloads abort at their next progress tick and the resumable .part
files keep the bytes already fetched.

The wave-deferral test stubs _content_lengths instead of issuing real
HEAD requests that passed only by failing, the dead hasattr branch in
the HEAD-probe test is gone, and the .C suffix comment names the SCons
divergence instead of claiming parity.
This commit is contained in:
J. Nick Koston
2026-08-23 12:04:49 -05:00
parent 7c0e03f309
commit ea3ff0e372
5 changed files with 67 additions and 9 deletions
+22 -3
View File
@@ -755,15 +755,25 @@ def run_batch_downloads(
collected (list.append is atomic under the GIL) and returned after the
bar is done, so the caller's warnings never land on the bar's row; a
failed job credits its tracker 0 so the bar can still complete. Ctrl-C
drops queued jobs instead of downloading them all before the process
can exit; in-flight ones still finish. ``jobs`` must be non-empty.
drops queued jobs and aborts in-flight ones at their next progress
tick; resumable ``.part`` files keep the bytes already fetched.
``jobs`` must be non-empty.
"""
failures: list[tuple[str, Exception]] = []
cancelled = threading.Event()
def _run(name: str, fetch: Callable[[Callable[[int], None]], None]) -> None:
tracker = progress.tracker()
def checked(done: int) -> None:
if cancelled.is_set():
raise _BatchDownloadCancelled
tracker(done)
try:
fetch(tracker)
fetch(checked)
except _BatchDownloadCancelled:
tracker(0)
except Exception as err: # noqa: BLE001 # pylint: disable=broad-exception-caught
failures.append((name, err))
tracker(0)
@@ -772,12 +782,21 @@ def run_batch_downloads(
try:
for future in [ex.submit(_run, name, fetch) for name, fetch in jobs]:
future.result()
except BaseException:
# Without this the non-daemon workers download to completion before
# the interpreter can exit, making Ctrl-C ineffective for minutes
cancelled.set()
raise
finally:
ex.shutdown(wait=True, cancel_futures=True)
progress.done()
return failures
class _BatchDownloadCancelled(Exception):
"""Raised inside a download job to abandon it after Ctrl-C."""
class BatchDownloadProgress:
"""One progress bar across several concurrent ``download_with_resume`` calls.
+2 -1
View File
@@ -59,7 +59,8 @@ DEFAULT_BUILD_FLAGS = []
# "asm" merges SCons's AS and ASPP sets: all compile as assembler-with-cpp.
# The kind values drive the ESP8266 native ninja rules (later in this
# chain); existing backends consume only the keys. Note .C/.C++ join the
# suffix set here, matching PlatformIO's CXXSUFFIXES.
# suffix set here per CXXSUFFIXES; SCons demotes .C to C on
# case-insensitive filesystems, we always treat it as C++.
SOURCE_KIND_FOR_SUFFIX: dict[str, str] = {
".c": "c",
".cpp": "cxx",