Commit Graph
16968 Commits
Author SHA1 Message Date
J. Nick Koston 3b2fc9f59a Close the owner-qualified drop gap and the duplicate-archive gap in the library backend
An owner-qualified version-less dependency was skipped by both the
arduino backend (owner set) and the reconciliation (provides() knew the
short name), so nothing logged the drop; provides() now only satisfies
owner-less names, mirroring the walk's backend-provided guard. A dict
dependency entry with no name at all now warns in the normalizer like
any other malformed entry.

Bundled dependency additions are deferred until conversion finishes and
skipped when a converted library's manifest already provides the name,
so a name that is both bundled and registry-resolved cannot build twice.
External short names come from the request spec (custom Name=url form
included) instead of the URL tail. The empty-bundled-library probe now
walks the whole library tree against a shared header-suffix constant,
and the dead versioned-dependency branch is folded into the surviving
check.
2026-08-22 11:24:49 -05:00
J. Nick Koston 57e738adf8 Simplify the library backend: share the dependency filter, drop the request-key drop diff, dedupe test scaffolding
Validate manifest shape once in the converter (every backend dereferences
data/build), share the InvalidLibrary filter as dependency_is_usable(),
and let the walk's reconciliation own version-less drop reporting so the
arduino backend's request-key diff, owner-no-version warning, and
node_key plumbing all go away. One memoized _provided() predicate now
answers bundled-name checks at all three sites. Tests gain shared
scaffold helpers and lose the assertions that pinned deleted messages.
2026-08-22 11:04:35 -05:00
J. Nick Koston 70b8573110 Move version-less drop reconciliation into the walk, harden the name guards
The shared walk now defers every version-less skip and reconciles after
emit against the final resolution set (request keys, resolved manifest
names, and backend.provides(name) per name), so drop visibility is
correct for every backend by construction instead of by a comment-level
contract, and the arduino backend's duplicate pending_drops pass and its
short-name suppression heuristics are deleted. The provides lambda
applies _is_safe_library_name so an unsafe manifest name is simply not
provided, and the guard also rejects drive-colon names that would
escape the tree on Windows.
2026-08-22 10:46:50 -05:00
J. Nick Koston 735d4076c1 Keep the top-level suppression total and honest
The check guards URL-ish names before _node_key (a malformed entry
belongs to the drop warning, not a RuntimeError) and matches against
top_level_keys so the message says exactly what it verifies; a
transitive-only node can still fail compatibility later and must not
suppress the drop report.
2026-08-22 10:03:43 -05:00
J. Nick Koston a48fbd1cd1 Suppress the top-level-requested false positive, guard names once, fail empty bundled libraries
The walk's version-less warning stays quiet for a name the config also
requests top-level, and the provides-backend debug level carries the
ownership rationale (the arduino backend defers drops and suppresses
walk-resolved names this layer cannot know yet). _is_safe_library_name
guards both the top-level bundled check and manifest dependencies, so a
YAML libraries entry of '.' or a backslash name cannot join under the
framework tree. The drop suppression strips only the sanitized owner
prefix, keeping a library's own double underscores. An empty bundled
directory raises naming the fix instead of warning into an unlinkable
build, and the second real-converter test pins ESPHOME_DATA_DIR.
2026-08-22 09:41:35 -05:00
J. Nick Koston e47ec9acad Cover the routine version-less skip through the real walk 2026-08-22 09:29:49 -05:00
J. Nick Koston 8e6af6aa8c Name the manifest in every dependency diagnostic, warn where no backend can recover
normalize_dependencies' parameter is manifest_name (the dict branch
already binds owner to a package owner), and the arduino call site
passes the library's name so its unrecognized-entry warning stops
saying 'of manifest'. A version-less dependency warns when the backend
declares no provides tree (espidf/zephyr/nrf52 have no post-emit
pickup), and the shared walk mirrors the typed IncompatiblePlatform
branch: routine platform skips stay at debug, any other InvalidLibrary
cause warns naming the component. The two real-converter tests pin
ESPHOME_DATA_DIR to tmp_path so an ambient data dir cannot leak in.
2026-08-22 09:20:57 -05:00
J. Nick Koston 9bb899d8df Never shred a string dependency list, defer and de-duplicate drop warnings
A plain-string dependencies value comma-splits into names instead of
iterating into one-character libraries; entries that normalize to
nothing warn naming the manifest, and the walk's version-less skip logs
at debug for every backend. The not-bundled warning is deferred until
after the walk and stays quiet for names resolved transitively, so a
false skipping message cannot teach users to ignore the real one. An
explicitly requested bundled library that resolves to no sources and no
headers warns that the install may be incomplete.
2026-08-22 00:48:30 -05:00
J. Nick Koston 35c77165ca Surface every dependency-drop path the walk can take
The converted path warns for a properties-only depends= spelling (the
walk reads the JSON key). A dependency name is validated before
becoming a path component: non-strings and separator names are
malformed entries, never joined (apply_extra_script already guards the
same shape). normalize_dependencies coerces PIO's bare string-list form
to name dicts instead of dropping it before every visibility warning. A
non-* version pin discarded for a backend-provided bundled copy warns
naming the substitution. The -I global-include promotion and the
start-group link-order contract are documented.
2026-08-22 00:18:35 -05:00
J. Nick Koston 44b14d8655 Merge branch 'esp8266-native-framework-installer' into esp8266-native-library-backend 2026-08-21 21:27:59 -05:00
J. Nick Koston 4eb31c45da Isolate the preresolved-ccache test from ambient CCACHE_DIR, future-tense the sentinel comment 2026-08-21 21:27:49 -05:00
J. Nick Koston 227522364a Merge branch 'esp8266-native-framework-installer' into esp8266-native-library-backend 2026-08-21 21:01:47 -05:00
J. Nick Koston 4bea2dd058 Merge branch 'esp8266-native-build-infra' into esp8266-native-framework-installer 2026-08-21 21:01:47 -05:00
J. Nick Koston db0e52a36b Cover shell_token's Windows branch on the coverage-uploading runner 2026-08-21 21:01:37 -05:00
J. Nick Koston a05a392fdd Merge branch 'esp8266-native-framework-installer' into esp8266-native-library-backend 2026-08-21 20:56:37 -05:00
J. Nick Koston 6a963ba794 Merge branch 'esp8266-native-build-infra' into esp8266-native-framework-installer 2026-08-21 20:56:37 -05:00
J. Nick Koston cbf9e0ab14 Merge branch 'esp8266-native-toolchain-plumbing' into esp8266-native-build-infra 2026-08-21 20:56:37 -05:00
J. Nick Koston a527b21761 Merge branch 'esp8266-native-build-surgery' into esp8266-native-toolchain-plumbing 2026-08-21 20:56:36 -05:00
J. Nick Koston 4d63564cdd Phrase the board-table regen comment imperatively 2026-08-21 20:56:27 -05:00
J. Nick Koston 42e255c1e5 Merge branch 'esp8266-native-framework-installer' into esp8266-native-library-backend 2026-08-21 20:55:19 -05:00
J. Nick Koston 012b571a2e Make the ccache sentinel public; sibling modules pass it as a default 2026-08-21 20:55:07 -05:00
J. Nick Koston a1d59049b4 Merge branch 'esp8266-native-framework-installer' into esp8266-native-library-backend 2026-08-21 20:51:57 -05:00
J. Nick Koston 4a16365f13 Merge branch 'esp8266-native-toolchain-plumbing' into esp8266-native-build-infra 2026-08-21 20:51:56 -05:00
J. Nick Koston 587873a2da Merge branch 'esp8266-native-build-infra' into esp8266-native-framework-installer 2026-08-21 20:51:56 -05:00
J. Nick Koston aad89cd939 Merge branch 'esp8266-native-framework-installer' into esp8266-native-library-backend 2026-08-21 20:46:18 -05:00
J. Nick Koston 4101ad4fb1 One owner for the toolchain tool layout, thread a pre-resolved ccache
toolchain_tool carries the bin/xtensa-lx106-elf-<name> pattern and the
Windows suffix that four call sites previously spelled out (only one of
which handled the suffix). ccache_env and get_build_env accept the
already-resolved ccache path so run_compile can resolve once instead of
paying the PATH scan and runnability probe three times per build.
2026-08-21 20:46:10 -05:00
J. Nick Koston 048025e9dd Form the local-library URL with as_uri so the test passes on Windows 2026-08-21 20:44:49 -05:00
J. Nick Koston e9d6132c61 Name the honored native-arduino pio options once
NATIVE_ARDUINO_PIO_OPTIONS is the single list the routing reads; the
backend's ignored-option warning consumes it too, so honoring a new key
is one edit instead of two synchronized ones.
2026-08-21 20:43:43 -05:00
J. Nick Koston 8ad1ed98b7 Skip backend-provided dependencies in the graph walk, validate manifests before extra scripts
The dict-shorthand bundled preference ran after convert_libraries had
already resolved the dependency through the registry, so a real
{"Wire": "*"} manifest still fetched or failed on a registry Wire (the
mocked test hid it). LibraryBackend gains an optional provides hook the
walk consults for owner-less dependencies; the arduino backend answers
from the framework tree and the bundled copy is added after emit. A
real-converter test with a local library pins that the registry is
never touched. _emit also validates the manifest shape by name before
apply_extra_script dereferences it.
2026-08-21 20:38:23 -05:00
J. Nick Koston 7298eed5b3 Merge branch 'esp8266-native-toolchain-plumbing' into esp8266-native-build-infra 2026-08-21 20:27:43 -05:00
J. Nick Koston 78a1e88915 Merge branch 'esp8266-native-build-infra' into esp8266-native-framework-installer 2026-08-21 20:27:43 -05:00
J. Nick Koston b87875881e Merge branch 'esp8266-native-framework-installer' into esp8266-native-library-backend 2026-08-21 20:27:43 -05:00
J. Nick Koston d8c02221cc Route board_build.ldscript through to the native ESP8266 generator too
Published configs override it to reserve a filesystem region
(SmartIntercom's eagle.flash.4m2m.ld) or to correct a board's assumed
flash size (2 MB ESP8285 plugs pinning eagle.flash.2m.ld); dropping it
changes the flash layout under them.
2026-08-21 20:27:33 -05:00
J. Nick Koston 76d8c9eb46 Merge branch 'esp8266-native-toolchain-plumbing' into esp8266-native-build-infra 2026-08-21 20:25:32 -05:00
J. Nick Koston a703acf85f Merge branch 'esp8266-native-build-infra' into esp8266-native-framework-installer 2026-08-21 20:25:32 -05:00
J. Nick Koston 7beda110d4 Merge branch 'esp8266-native-framework-installer' into esp8266-native-library-backend 2026-08-21 20:25:32 -05:00
J. Nick Koston 7fb18f395f Honor board_build.f_cpu under the native ESP8266 Arduino toolchain
Dropping it was a real regression: many published ESP8266 configs pin
board_build.f_cpu: 160000000L for timing-sensitive integrations
(MHI-AC-Ctrl documents the 160 MHz requirement in its example), and the
warn-and-drop left those devices at 80 MHz. The option now routes into
CORE.platformio_options under toolchain: arduino for the generator to
consume; other native toolchains keep the warning.
2026-08-21 20:25:23 -05:00
J. Nick Koston c1a7c87a64 Cover the non-dict bundled-manifest branch 2026-08-21 17:44:57 -05:00
J. Nick Koston d96ddf3612 Catch the depends= spelling, warn on bundled extraScripts, trust top-level requests
The bundled-dependency visibility warning also reads the
library.properties depends= key the shared parser returns raw, since
properties is the format bundled libraries actually ship. A bundled
manifest declaring an extraScript warns that it never runs (a new
Known-deviations entry). A version-less manifest dependency the config
separately requests as a top-level library is already in the build, so
the skipping warning no longer fires for it and cannot teach users to
ignore the real drop.
2026-08-21 17:38:25 -05:00
J. Nick Koston e18852e084 Fail on dropped requests, guard the last manifest fields, parse dot_a_linkage strictly
A top-level library the converter dropped always makes the firmware
wrong, so resolve_libraries raises naming the missing requests instead
of warning toward link errors, and a resolved component without its
node_key fails as a converter bug rather than silently substituting the
mismatched canonical name. includeDir and srcFilter join srcDir and
flags in raising a named error on malformed values, and dot_a_linkage
parses through the same strict table as libArchive so a typo warns and
keeps the archive default instead of flipping link semantics.
2026-08-21 17:19:01 -05:00
J. Nick Koston 1f868d22fb Expose request_key as the public pair of ConvertedLibrary.node_key
The arduino backend consumed the private _node_key across packages to
reconstruct the request-side key; the thin public wrapper keeps both
sides of the contract on the same API surface.
2026-08-21 17:04:03 -05:00
J. Nick Koston 2ecf7af416 Name malformed manifests, diff drop-warnings by node key, surface bundled deps
A non-dict manifest or build section raises an EsphomeError naming the
library instead of an AttributeError. ConvertedLibrary remembers the
request-side node key it resolved from, so the converter-drop warning
diffs requests against those keys; a bare name resolving to a canonical
owner__pkg is no longer falsely listed missing. A bundled library whose
manifest declares dependencies warns that the walk never runs for it (a
no-op on the ESP8266 core, not on every core this module serves).
2026-08-21 16:52:32 -05:00
J. Nick Koston 91a4eb0f57 Merge branch 'esp8266-native-framework-installer' into esp8266-native-library-backend 2026-08-21 16:07:40 -05:00
J. Nick Koston 97ed253c38 Say the floor's config-validation guard is future wiring, not present
Nothing at this commit enforces MIN_FRAMEWORK_VERSION during
validation; the native backend's validator arrives with the wiring PR.
The comment no longer invites skipping it.
2026-08-21 16:07:30 -05:00
J. Nick Koston be2395da66 Merge branch 'esp8266-native-framework-installer' into esp8266-native-library-backend 2026-08-21 15:34:00 -05:00
J. Nick Koston cbf1241401 Resolve the ccache path on every call, matching the espidf backend
functools.cache froze the ESPHOME_CCACHE_ENABLE/PATH decision for the
process lifetime, which a long-lived host like device-builder never
restarts, and forced a cache_clear fixture in the tests. The docstring
and test now pin the uncached delegation.
2026-08-21 15:33:51 -05:00
J. Nick Koston ce87ca2e11 Merge branch 'esp8266-native-framework-installer' into esp8266-native-library-backend 2026-08-21 15:19:10 -05:00
J. Nick Koston 33aea420ac Merge branch 'esp8266-native-build-infra' into esp8266-native-framework-installer 2026-08-21 15:19:10 -05:00
J. Nick Koston e3d29e9fff Name a non-list registry system field instead of a TypeError
file["system"] normalized None and str but let an int or dict fall
through to the in test, where an int raises TypeError and a dict
becomes a key test; both now raise the same Unexpected-response error
as the other shape guards.
2026-08-21 15:18:54 -05:00
J. Nick Koston 03bf037cb8 Merge branch 'esp8266-native-framework-installer' into esp8266-native-library-backend 2026-08-21 14:50:47 -05:00