The route gained a 500 PRUNE_FAILED branch and the seeder method gained a None return, both so a prune that did not run cannot be reported as a clean one. Neither contract was written down.
MemoryError is an Exception, so the per-spec and per-batch handlers stored it alongside ordinary faults and carried on - allocating for every remaining spec and then every remaining batch while the process was already out of memory. Both handlers now let it through, and the scan records a failure and stops.
Commit 1dbcdcd7 and the comment-cleanup pass 8205022f reduced this to "All
database reads and writes, including the legacy import, run under the lock",
dropping the part that did the work: upstream master locks after migrating and
justifies it with "Alembic uses its own connection, so we must wait until it's
done before locking -- otherwise our own lock blocks the migration". That is
false, the lock is on a separate <db>.lock file, and the surviving sentence
said nothing to stop a contributor "fixing" the ordering back.
Restored and adapted rather than pasted: the legacy copy and the db_exists
probe now happen inside the lock, which the original text predates, so both
are named in the list of things the ordering makes mutually exclusive.
scanner.py reached past the package's own re-export surface to import
_is_live_path_conflict directly out of records.py. The underscore said
module-private while the import said otherwise, and records.py deliberately
publishes its public names through app.assets.database.queries -- which the
same import block three lines above was already using.
The use is correct and unchanged; only the name and the route change. Renamed
to is_live_path_conflict, listed in the package __init__ import and __all__
alongside its siblings, and scanner.py now takes it from the package like
everything else it imports from there.
get_name_and_tags_from_asset_path raises ValueError by contract when a path
stops resolving to a configured root, and it sat outside the guard, as did
compute_loader_path and mimetypes.guess_type. An escape skipped the
_WATCH_LIST[:] = remaining write at the end, so drained entries stayed on the
list and were re-attempted every tick while entries past the fault never
reached the increment _WATCH_SCAN_RETRIES needs to retire them. The list
wedged permanently.
Spec construction is now inside a guard that drops just the offending entry,
and the list write moved into a finally so no future escape can skip it. The
loop walks an iterator rather than the list, so the finally can put back the
entries it never reached instead of discarding them.
New event name rather than reusing one: scanner.watch_seed_failed is emitted
only when seed_asset_specs returns an error, and widening it to also mean
"never got as far as seeding" would make it lie -- a consumer treating it as a
database-health signal would get false positives from what is really a path
layout problem. scanner.watch_spec_failed is registered in ALLOWED_EVENTS and
in the static call-site manifest.
Every other per-file emit on this scan path is gated -- mark_emitted(
"stat_failed:enrich"), "hash_discarded_modified", "hash_failed",
"enrich_failed" -- but scanner.invalid_mtime fired per file, so a restored
archive or a FAT volume of pre-epoch mtimes put one structured event per file
into the stream the closed vocabulary exists to keep parseable.
Counted and emitted once, carrying the count. seed_asset_specs receives no
_ScanProgress object and neither does insert_asset_specs above it, so routing
this through mark_emitted would mean changing both signatures plus the seeder
call site; the count form needs neither and the event is now strictly more
informative than N identical fieldless lines. The per-file logging.warning is
unchanged, and the emit stays inside seed_asset_specs so the static call-site
manifest still matches.
test_seed_skips_negative_fresh_mtime_with_warning_and_telemetry now pins the
full list of invalid_mtime lines to exactly ["... count=1"] instead of
asserting one such line exists -- a strictly stronger assertion, and the only
change the new field required.
prepare_file_db_path no longer copies the legacy database - that moved inside the process lock in _init_file_db - so patching copy_legacy_default_db here did nothing. Leaving it implied a side effect the function does not have, and would have masked one if it were reintroduced.
Multipart parsing writes the uploaded bytes to a temporary file before it
validates the remaining form fields, so a request rejected after its file
part had already been read left the temp file and its uuid directory on
disk.
Routing those removals through delete_temp_file_if_exists also changes the
success path. The previous helper returned early when the temp file was
already gone, so it never reached the parent rmdir; the shared helper
attempts the rmdir unconditionally. Moving the upload to its destination
leaves the temp path absent, so a successful upload now also discards its
empty uuid directory, closing a pre-existing leak.
The fixture wrapped importlib.import_module in a bare except that called
pytest.skip, so a circular import, a missing dependency or a syntax error in
app/prompt_worker.py would retire all four resume-contract tests while CI
stayed green. The whole premise of extracting the module is that main.py can
import it, so an import failure has to be a collection error.
The CPU guard is genuinely load-bearing — comfy.model_management selects its
device at import time and a CUDA build with no driver raises there — so it is
kept, but as a precondition rather than an exception handler, matching the
args.cpu-before-import convention already used by the comfy_test and
comfy_api_nodes_test modules. Nothing is caught now.
The assertion asked for the flag with no argument, so it read the parameter default rather than anything the manager reported - in a test whose subject is the two agreeing. Passing the manager's own state keeps it honest if the setup ever yields an enabled manager.
The cursor holds at the last row a batch actually attempted, so a pause ends a batch early and the rows behind it are selected again when the scan resumes. Only the attempt is capped at once per pass.
* review-stack 1/4: code (37 files, +3217/-3958)
Review-and-land stack for synap5e/feat/asset-record-content-split, generated by review-stack.py. Once approved,
merges DOWN into the layer below (a fast-forward); only the bottom layer
squash-merges into the real base. See ~/adocs/review-stack.md.
Rule: path not under tests-unit/ or tests/
Question: Is the logic change right?
Source tip: 7007d18582
Merge-base: 783545f689
* review-stack 2/4: tests-removed (24 files, +274/-8220)
Review-and-land stack for synap5e/feat/asset-record-content-split, generated by review-stack.py. Once approved,
merges DOWN into the layer below (a fast-forward); only the bottom layer
squash-merges into the real base. See ~/adocs/review-stack.md.
Rule: test file deleted, or modified with deleted/(added+deleted) >= 0.9
Question: For each dropped assertion: obsolete by a ruling, or covered by a tests-new test?
Source tip: 7007d18582
Merge-base: 783545f689
* review-stack 3/4: tests-changed (13 files, +1043/-1218)
Review-and-land stack for synap5e/feat/asset-record-content-split, generated by review-stack.py. Once approved,
merges DOWN into the layer below (a fast-forward); only the bottom layer
squash-merges into the real base. See ~/adocs/review-stack.md.
Rule: remaining modified test files (incl. conftest.py / helpers)
Question: Did the edits weaken an existing check?
Source tip: 7007d18582
Merge-base: 783545f689
* review-stack 4/4: tests-new (46 files, +8601/-0)
Review-and-land stack for synap5e/feat/asset-record-content-split, generated by review-stack.py. Once approved,
merges DOWN into the layer below (a fast-forward); only the bottom layer
squash-merges into the real base. See ~/adocs/review-stack.md.
Rule: test file added
Question: Is the code layer well covered?
Source tip: 7007d18582
Merge-base: 783545f689
* review-stack 5/6: code (13 files, +351/-104)
Review-and-land stack for synap5e/feat/assets-di, generated by review-stack.py. Once approved,
merges DOWN into the layer below (a fast-forward); only the bottom layer
squash-merges into the real base. See ~/adocs/review-stack.md.
Rule: path not under tests-unit/ or tests/
Question: Is the logic change right?
Source tip: eca2c74bff
Merge-base: 20d59d2a5f
* review-stack 6/6: tests (8 files, +753/-238)
Review-and-land stack for synap5e/feat/assets-di, generated by review-stack.py. Once approved,
merges DOWN into the layer below (a fast-forward); only the bottom layer
squash-merges into the real base. See ~/adocs/review-stack.md.
Rule: every changed file under tests-unit/ or tests/ (added, modified, or deleted)
Question: Is the code layer well covered, and did any edit weaken an existing check?
Source tip: eca2c74bff
Merge-base: 20d59d2a5f
* review-stack 7/8: ported-fixes (42 files, +1361/-180)
Review-and-land stack for synap5e/feat/assets-di-v2, generated by
review-stack.py conventions (hand-built continuation layer; see the PR body).
Once approved, merges DOWN into the layer below (a fast-forward); only the
bottom layer squash-merges into the real base. See ~/adocs/review-stack.md.
Rule: the 11 base-branch fix/docs commits 595cd6e4..94d7185b cherry-picked across the DI refactor (7efdd1d7 excluded, superseded by layer 8)
Question: was each base fix ported faithfully across the DI refactor?
Source tip: 6841881069284803b902b4a9e33bdcda13126771
Merge-base: 7fdfb40f4b
* review-stack 8/8: defensive-parity (4 files, +36/-3)
Review-and-land stack for synap5e/feat/assets-di-v2, generated by
review-stack.py conventions (hand-built continuation layer; see the PR body).
Once approved, merges DOWN into the layer below (a fast-forward); only the
bottom layer squash-merges into the real base. See ~/adocs/review-stack.md.
Rule: match-or-improve master's dependency defenses — NoAssets selection when DB deps unavailable (7efdd1d7's outcome via the DI seam), requirements warning before assets imports, blake3 in the guarded dependency set
Question: does each degradation path now match or improve master's behavior?
Source tip: ebc2cfeebc
Merge-base: 7fdfb40f4b
* fix(assets): only discard content rows this operation actually inserted
CR-9: Enumerated all six create_content call sites. Only scanner seeding and the three ingest registration paths track IDs for failure cleanup.
* fix(assets): reject hash-only uploads with FEATURE_DISABLED when hashing is off
CodeRabbit finding CR-2: reject hash-only multipart uploads before create_from_hash when hashing is disabled.
* fix(assets): seed persists the stat it verified
CR-7: persist the fresh seed-time restat instead of walk-time spec values.
* fix(assets): route database lock failures to the lock guidance
CR-16: route file-lock startup failures through the existing lock guidance and exit path.
* fix(assets): drop the inaccurate temp-cleanup claim from the shutdown warning
References CR-10.
* fix(assets): walk the output root after execution so undeclared outputs register promptly
Custom nodes that write files into the output directory without declaring
them in output_ui only became assets when the next full walk happened - a
frontend GET /object_info or a restart. Headless and API-only sessions never
trigger either, so those files never converged into the asset database.
The post-execution hook now requests a FULL scan of the output root instead of
an enrich-only pass. The seeder's pending-request queue was generalised from
enrich-specific to carrying a scan phase, so the request starts immediately
when the seeder is idle and coalesces (escalating to FULL on a phase mismatch)
when a scan is already running. queue_output_enrichment is renamed to
queue_output_scan across the protocol, the NoAssets no-op and the call site.
References FIX-6.
* chore(assets): remove seeder paths orphaned by the output-scan change
45c2f96e rerouted both former enrich call sites to start()/enqueue_scan(),
leaving two seeder methods that look live but are not. Review round F2
raised this along with four smaller items; the user's disposition was to
fix all six here.
- Delete start_enrich: zero callers repo-wide after 45c2f96e.
- Delete enqueue_enrich: no production callers; its ~18 call sites in
tests/test_asset_seeder.py move to enqueue_scan(phase=ScanPhase.ENRICH)
with their semantics unchanged. The deletion forces the half-done class
renames (TestEnqueueEnrich* -> TestEnqueueScan*, consistent with the
already-renamed TestPendingScanDrain) and restores the module docstring
that was dropped rather than reworded.
- Document at manager.queue_output_scan that ScanPhase.FULL per debounce
window is the deliberate, user-ratified trade, so it is not optimised
back to ENRICH without revisiting the decision.
- Document that SeedAssetSpec.size_bytes/mtime_ns are walk-time
diagnostics only - production persists the seed-time restat since CR-7.
- Export create_content_reporting_insert from the queries facade and fold
scanner.py's direct-module import into the existing facade block.
- Harden test_queue_output_scan_does_not_duplicate_declared_output against
a vacuous pass: it now asserts the seeder finished without errors and
that an undeclared sibling written into the same directory WAS
registered by the same scan, proving the walk actually ran.
No production behaviour changes beyond the two deletions.
References F2-cleanup.
* chore: comment cleanup
Comment-Gate: 18 quarantined
* fix(assets): preserve pause across the seeder's pending-scan drain
pause() runs before every prompt, while pending-scan enqueue and resume only run inside the debounced gc-interval gate. If the active scan finishes just after the next prompt's pause, its finally block resets the seeder to idle and the pending drain starts a replacement with the run gate open, so resume becomes a no-op.
Capture pausedness under the lock before resetting to idle, then start the drained scan already paused. Setting the state and gate before launching the thread avoids the start-then-reclear window and lets resume release the existing scan checkpoints.
Ultraworked with [Sisyphus](https://github.com/code-yeongyu/oh-my-openagent)
Co-authored-by: Sisyphus <clio-agent@sisyphuslabs.ai>
* test(assets): pin job_id absence for scan-discovered assets
Owner ruling, recorded 2026-09-03 in the stack-9-hardening planning notepad: scan-discovered assets — including undeclared outputs found by the post-execution walk — carry job_id = None, always; only emission-time registration (output_ui declaration) attributes a job; attributing walk finds to the most recent prompt would be a temporal-correlation guess that is wrong exactly when prompts interleave; None is honest provenance. Do NOT add proximity-based attribution heuristics to the scanner. Ratified against Jacob Segal's cross-job-attribution concern (2026-09-08 review meeting) — a wrongly-attributed asset could mean one user's cloud job sees another user's asset.
Ultraworked with [Sisyphus](https://github.com/code-yeongyu/oh-my-openagent)
Co-authored-by: Sisyphus <clio-agent@sisyphuslabs.ai>
* [review-stack 10/10] assets-tests (#16218)
* test(execution): run the battery with assets enabled and assert asset-system health at teardown
* test(execution): cover list-shaped outputs registering assets
Ultraworked with [Sisyphus](https://github.com/code-yeongyu/oh-my-openagent)
Co-authored-by: Sisyphus <clio-agent@sisyphuslabs.ai>
---------
Co-authored-by: Sisyphus <clio-agent@sisyphuslabs.ai>
* [review-stack 11/11] review-fixes (#16261)
* fix(assets): only exit on database file-lock timeout when assets are enabled
* test(assets): pin live_contents_under_prefixes path-filtering semantics
* perf(assets): push live-content prefix filtering into SQL
* test(assets): declare per-entry intent in the path-prefix corpus
* test(assets): normalize POSIX-literal path expectations for Windows
* test(assets): force observable stat changes and close-before-mutate on Windows-sensitive rewrites
* test(assets): force an observable mtime change in the hash-mode split test
---------
Co-authored-by: Sisyphus <clio-agent@sisyphuslabs.ai>
Co-authored-by: guill <jacob.e.segal@gmail.com>
preview_url was assembled from a /api/view link whose type was chosen by
matching the asset's tags against "input" then "output". Anything written
anywhere else - temp above all, where preview nodes put their images - fell
off the end of that chain and came back with no preview at all. Tags are
user-editable, so removing one also silently destroyed the URL.
Derive the URL from where the file actually sits instead. That covers every
root /api/view serves, temp included, and no longer depends on tags or on a
filename in user_metadata. A file outside those roots, or content no client
can render from its own bytes, gets no preview URL rather than one that
cannot work.
Nominated previews are resolved a page at a time rather than per row, so a
list costs one extra query however long it is. A preview that is soft-deleted
or not visible to the caller drops out of that lookup and is no longer
advertised.
Co-authored-by: guill <jacob.e.segal@gmail.com>
* Keep temp-directory assets visible while their files exist
Assets written to the temp directory were flagged as missing and dropped
from GET /api/assets, even with the file sitting on disk. One list of
directories was answering two different questions -- where the scanner
looks for new files, and which files ComfyUI considers its own -- and
temp belongs only in the second, so every temp reference was disowned by
the prune that runs at startup and on POST /api/assets/prune.
Ownership now covers temp. Discovery still does not: the temp directory
is wiped before the scan runs, and assets written there are already
registered with a hash, mime type and dimensions, so walking it would
find nothing. Temp references are instead reconciled against the
filesystem directly, so a temp file that really is gone is still retired
rather than lingering as a broken entry.
get_prefixes_for_root becomes get_scan_prefixes_for_root so the two
questions are told apart by name rather than by comment.
* Cover the unhashed temp asset in the reconciliation tests
The existing temp tests all registered hashed assets, so they never
exercised the path an unhashed asset takes when its file is gone: the
orphaned rows are removed rather than kept as missing, exactly as under
any other root.
* Implement tags_all/tags_any/tags_none on the assets list API (BE-6600)
Adds the three canonically-named tag filter params to GET /api/assets and
GET /api/assets/tags/refine:
- tags_all: asset carries every tag (replaces include_tags)
- tags_any: asset carries at least one tag (new)
- tags_none: asset carries no tag (replaces exclude_tags)
Clauses intersect; tags_none always wins. include_tags/exclude_tags remain
as permanent deprecated aliases and behave exactly as before when used on
their own.
Invalid combinations return 400 INVALID_TAG_FILTER, but only when the
request uses at least one new-name parameter (non-empty after
normalisation):
- mixed spellings of one slot (include_tags with tags_all, exclude_tags
with tags_none)
- the same tag in the effective all-list and none-list (query can never
match)
Old-names-only requests gain no new error paths: include_tags=a&exclude_tags=a
still returns an empty 200. tags_any/tags_none overlap stays valid (dead
term, not a dead query).
* Address review findings: positional-compat, deprecation metadata, test matrix
- Move any_tags to the end of the four touched signatures: inserting it
mid-signature silently misbound pre-existing positional callers (e.g.
a caller passing name_contains positionally would have it consumed as
any_tags).
- Mark include_tags/exclude_tags Field(deprecated=True) on both list
schemas so generated schema metadata matches the contract, not just a
comment (schemas_out.py already uses this form for Asset.name).
- Add tests: legal cross-slot old/new combinations, repeated query-key
concatenation (pins Core behavior; outside the cross-platform
contract), tags_any two-page cursor consistency (total/has_more/
no-overlap), refine-route mixed-spelling rejection + legacy-conflict
preservation, and schema deprecation metadata.
* Pin tag-value opacity: case-sensitive matching, byte-exact conflict check
The prod tag survey (~/comfy/prod-model-tag-shape.md) found live
case-distinct tag pairs (SEEDVR2/seedvr2) that resolve differently, so
the contract now states tag values are opaque byte-strings. Pin that:
case-distinct tags filter separately, and a case-distinct all/none pair
is not an INVALID_TAG_FILTER conflict.
* Document tags_all/tags_any/tags_none in openapi.yaml, deprecate aliases
Add the three tag-filter parameters to both listAssets and
getAssetTagHistogram parameter blocks and mark include_tags/exclude_tags
deprecated: true, keeping the spec in step with the runtime schemas so
generated clients can discover the new filters while the aliases stay
present for existing consumers.
* Move schemas_in import to module scope in test_list_filter
Review feedback: no import cycle requires the local import.
* Silence per-request DeprecationWarning in the tag-filter remap shim
Reading the deprecated include_tags/exclude_tags fields by attribute
fires pydantic's DeprecationWarning on every list/refine request even
for callers using only the new names. The warning is aimed at API
clients, not the server's own remap; read via model_dump instead.
* Cap tag-filter lists at 100 entries, all spellings
Review finding: unbounded tag lists fan out into one correlated EXISTS
per tag on both page and count statements. Cap each list at 100
normalized entries with 400 INVALID_TAG_FILTER naming the parameter.
Applies to the legacy spellings as well — a deliberate, decided
exception to the old-names-behave-identically rule, since a cap only on
new names would leave the same fan-out reachable through the aliases.
* Strip process narration from comments
Comments carried decision dates, contract cross-references, and review
context. Keep only the constraints the code cannot show, one line each.
Each folder in the listing now carries its registered extension
allowlist verbatim; an empty array means the folder accepts any
extension (match-all), mirroring filter_files_extensions semantics.
Gives consumers the filtering rule itself rather than just its output:
/models/{folder} lists files by the per-folder rule but the rule is not
exposed anywhere, and /experiment/models/{folder} filters everything by
the global supported_pt_extensions regardless of registration.
Presentation-level filtering of match-all folders (e.g. hiding
README/config noise that repository-downloading custom nodes leave in
model directories) is deliberately left to the consumer.
Co-authored-by: guill <jacob.e.segal@gmail.com>
* feat(assets): add namespaced model type tags
* fix(assets): mark path-derived upload tags automatic
* fix(assets): merge duplicate scan specs
* test(assets): make duplicate path normalization portable
* feat(assets): add loader_path as the authoritative loader locator (#14796)
* fix(assets): filter model_type tags by bucket extension sets
Buckets sharing a base directory (e.g. diffusion_models and a custom
unet_gguf) tagged every file in the directory regardless of whether the
bucket could load it, so .safetensors files were tagged
model_type:unet_gguf and vice versa. Carry each bucket's registered
extension set through get_comfy_models_folders and only emit a
model_type tag when the file extension matches, keeping the empty-set
match-all convention from folder_paths.filter_files_extensions.
Files under a model base matching no bucket now keep only the models
tag instead of every directory-matching model_type tag.
* feat(assets): replace response file_path with persisted loader_path
The old file_path response field was a namespaced storage locator
(models/checkpoints/foo.safetensors): not an absolute path, not unique
identity, and not the value a loader consumes. Nothing needs that shape
on the wire (hash/ID-based locating is the long-term direction), so it
is dropped rather than renamed; the storage-root matching stays internal,
powering display_name.
What loaders DO need is the in-root loader path (category dropped:
models/checkpoints/foo/bar.safetensors -> foo/bar.safetensors). Serve it
as a first-class loader_path field, persisted on asset_references
(migration 0006) and written by every ingest pipeline at insert, so
responses read the column verbatim.
Like the model_type tags, loader_path is a seed-time derivative of the
model folder registry, maintained by the same scan lifecycle (new files seed
fresh values, pruning retires rows whose bucket disappeared). Rows
predating the column serve a null loader_path; databases from before
this stack already need recreating for the base branch's tag changes.
loader_path resolves every registered base including extra_model_paths
entries; display_name only the canonical storage roots. A file can
therefore be loadable with no display name (extra-path models) or the
reverse (unregistered files under the models root), and loader_path is
null exactly when no loader can resolve the file.
* test(assets): lock loader_path matrix (asymmetry, null, persist/read)
Cover the behaviour that has no production change but is easy to regress:
the extra-path asymmetry (loadable but no storage namespace), null
loader_path persistence for orphan files, and the response reading the
stored column with a compute fallback for un-backfilled rows.
* fix(assets): persist subfolder-qualified loader_path for ingested outputs
ingest_existing_file built its seed spec with the file's basename, so
outputs saved into a subfolder persisted loader_path (and the
user_metadata filename that preview URLs split for their subfolder
param) as just the basename: the served locator pointed at a file that
does not exist at that path. Scanner and seeder specs already derive
fname via compute_loader_path; use the same derivation here.
* fix(assets): only extension-matching buckets contribute a loader_path
The model-base match in get_asset_category_and_relative_path ignored
each bucket's extension set, so a file inside a registered base whose
extension the bucket cannot load (e.g. a .txt uploaded into
model_type:checkpoints) advertised a loader_path that no loader list
would ever resolve, while the tag side of the same stack already
excluded it. Apply the extension check used for backend tags (empty set
accepts any extension), keeping loader_path null exactly when no loader
can resolve the file.
* fix(assets): refresh loader_path when re-ingesting an existing reference
upsert_reference only wrote loader_path on the INSERT branch, so
re-ingesting an existing reference (an output overwritten in place, or a
file re-registered after its loader_path derivation changed) kept the
stale or NULL value forever. Write it on the UPDATE branch too, with a
null-safe change guard so a loader_path difference alone is enough to
trigger the update, and identical values stay a no-op.
* fix(assets): repair semantic merge breakage from #14796 and master
Two textually-clean but semantically-broken merges:
- routes.py lost its folder_paths import when #14796's import block
superseded the base's, while the content-type hardening added via the
base's master merge still calls folder_paths.is_dangerous_content_type.
- master's SVG download-hardening test uploads with the pre-namespacing
bare checkpoints tag, which this branch's destination validation
rejects; use model_type:checkpoints.
---------
Co-authored-by: guill <jacob.e.segal@gmail.com>
Simon Pinfold