Commit Graph
201 Commits
Author SHA1 Message Date
Simon Pinfold e9af384a9e fix(assets): keep spec construction failures from wedging the watch list
get_name_and_tags_from_asset_path raises ValueError by contract when a path
stops resolving to a configured root, and it sat outside the guard, as did
compute_loader_path and mimetypes.guess_type. An escape skipped the
_WATCH_LIST[:] = remaining write at the end, so drained entries stayed on the
list and were re-attempted every tick while entries past the fault never
reached the increment _WATCH_SCAN_RETRIES needs to retire them. The list
wedged permanently.

Spec construction is now inside a guard that drops just the offending entry,
and the list write moved into a finally so no future escape can skip it. The
loop walks an iterator rather than the list, so the finally can put back the
entries it never reached instead of discarding them.

New event name rather than reusing one: scanner.watch_seed_failed is emitted
only when seed_asset_specs returns an error, and widening it to also mean
"never got as far as seeding" would make it lie -- a consumer treating it as a
database-health signal would get false positives from what is really a path
layout problem. scanner.watch_spec_failed is registered in ALLOWED_EVENTS and
in the static call-site manifest.
2026-09-17 19:47:37 -07:00
Simon Pinfold e0286a695e fix(assets): emit the invalid-mtime event once per scan
Every other per-file emit on this scan path is gated -- mark_emitted(
"stat_failed:enrich"), "hash_discarded_modified", "hash_failed",
"enrich_failed" -- but scanner.invalid_mtime fired per file, so a restored
archive or a FAT volume of pre-epoch mtimes put one structured event per file
into the stream the closed vocabulary exists to keep parseable.

Counted and emitted once, carrying the count. seed_asset_specs receives no
_ScanProgress object and neither does insert_asset_specs above it, so routing
this through mark_emitted would mean changing both signatures plus the seeder
call site; the count form needs neither and the event is now strictly more
informative than N identical fieldless lines. The per-file logging.warning is
unchanged, and the emit stays inside seed_asset_specs so the static call-site
manifest still matches.

test_seed_skips_negative_fresh_mtime_with_warning_and_telemetry now pins the
full list of invalid_mtime lines to exactly ["... count=1"] instead of
asserting one such line exists -- a strictly stronger assertion, and the only
change the new field required.
2026-09-17 19:47:37 -07:00
Simon Pinfold 64d884aef0 refactor(assets): reap empty upload directories without importing the API layer 2026-09-17 19:47:37 -07:00
Simon Pinfold c0e5ecc1d4 fix(assets): reject duplicate file parts instead of stranding the first upload 2026-09-17 19:47:37 -07:00
Simon Pinfold d071895286 refactor(assets): collapse the duplicated batch fault deferral into seed_asset_specs 2026-09-17 19:47:37 -07:00
Simon Pinfold e9b3587ae7 fix(assets): distinguish partial batch insert failures 2026-09-17 19:47:36 -07:00
Simon Pinfold 1f34110261 fix(assets): report specs committed before a batch fault and preserve the fault itself 2026-09-17 19:47:36 -07:00
Simon Pinfold 1eac7422e5 test(db): drop the inert legacy-copy patch from the path preparation tests
prepare_file_db_path no longer copies the legacy database - that moved inside the process lock in _init_file_db - so patching copy_legacy_default_db here did nothing. Leaving it implied a side effect the function does not have, and would have masked one if it were reintroduced.
2026-09-17 19:47:36 -07:00
Simon Pinfold 80682d076a fix(assets): preserve successful specs when a scan batch fault propagates 2026-09-17 19:47:36 -07:00
Simon Pinfold d496e23bb5 fix(assets): remove temporary uploads on non-UploadError failures 2026-09-17 19:47:36 -07:00
Simon Pinfold c605a9be77 fix(assets): keep unreadable filesystem metadata from failing a whole scan batch 2026-09-17 19:47:35 -07:00
Simon Pinfold e13f0e1f43 test(assets): make the keyset tie-breaker and temp-exclusion tests falsifiable 2026-09-17 19:47:35 -07:00
Simon Pinfold 68c7893381 fix(assets): correct event-log status snapshots and failure telemetry 2026-09-17 19:47:35 -07:00
Simon Pinfold 9b8e766edc fix(assets): keep updated_at stable on no-op renames 2026-09-17 19:47:34 -07:00
Simon Pinfold 417532d548 fix(assets): clean up temp uploads on validation failures
Multipart parsing writes the uploaded bytes to a temporary file before it
validates the remaining form fields, so a request rejected after its file
part had already been read left the temp file and its uuid directory on
disk.

Routing those removals through delete_temp_file_if_exists also changes the
success path. The previous helper returned early when the temp file was
already gone, so it never reached the parent rmdir; the shared helper
attempts the rmdir unconditionally. Moving the upload to its destination
leaves the temp path absent, so a successful upload now also discards its
empty uuid directory, closing a pre-existing leak.
2026-09-17 19:47:34 -07:00
Simon Pinfold 9c6749ab38 fix(assets): drop watch-list entries on stat errors instead of aborting the scan 2026-09-17 19:47:34 -07:00
Simon Pinfold 6bc286c0e6 fix(db): copy the legacy database inside the process lock 2026-09-17 19:47:34 -07:00
Simon Pinfold d698d2a27b fix(assets): only absorb duplicate-path races when seeding scanned assets 2026-09-17 19:47:33 -07:00
Simon Pinfold fba73d181e test(assets): let a broken prompt worker import fail instead of skipping
The fixture wrapped importlib.import_module in a bare except that called
pytest.skip, so a circular import, a missing dependency or a syntax error in
app/prompt_worker.py would retire all four resume-contract tests while CI
stayed green. The whole premise of extracting the module is that main.py can
import it, so an import failure has to be a collection error.

The CPU guard is genuinely load-bearing — comfy.model_management selects its
device at import time and a CUDA build with no driver raises there — so it is
kept, but as a precondition rather than an exception handler, matching the
args.cpu-before-import convention already used by the comfy_test and
comfy_api_nodes_test modules. Nothing is caught now.
2026-09-17 19:47:33 -07:00
Simon Pinfold 1f30b3ab33 test(api): derive the expected assets flag from the manager under test
The assertion asked for the flag with no argument, so it read the parameter default rather than anything the manager reported - in a test whose subject is the two agreeing. Passing the manager's own state keeps it honest if the setup ever yields an enabled manager.
2026-09-17 19:47:33 -07:00
Simon Pinfold a787e251cc chore(assets): address review follow-ups in the hashing guard, feature flags, and pagination pin 2026-09-17 19:47:32 -07:00
Simon Pinfold 3af6baf6b2 fix(assets): track the scan pause across prompt worker iterations 2026-09-17 19:47:32 -07:00
Simon Pinfold e30769a9f8 fix(assets): advance the enrichment cursor only past rows the batch attempted 2026-09-17 19:47:32 -07:00
Simon Pinfold 5ce874200c refactor(assets): test the blake3 import guard in-process instead of via subprocess 2026-09-17 19:47:32 -07:00
Simon Pinfold ac1b25e02f refactor(assets): extract prompt_worker so its resume contract is testable in-process 2026-09-17 19:47:31 -07:00
Simon Pinfold 0297d729a3 fix(assets): paginate enrichment by id cursor so failures cannot starve or overflow the query 2026-09-17 19:47:31 -07:00
Simon Pinfold 9f2b470216 fix(api): derive the assets feature flag from the selected manager 2026-09-17 19:47:31 -07:00
Simon Pinfold 8fffd7da73 fix(assets): always resume background scanning when prompt handling fails 2026-09-17 19:47:31 -07:00
Simon Pinfold 560a1ffa56 fix(assets): guard the hashing dependency and chain the real import error 2026-09-17 19:47:31 -07:00
Simon Pinfold 898348bc20 refactor(tests): hoist migration-0007 test imports to module scope 2026-09-17 19:47:30 -07:00
Simon Pinfold d9d78b99da test(assets): cover asset system state index parity 2026-09-17 19:47:30 -07:00
Simon Pinfold 7eaabb5ca0 test(assets): assert alembic and ORM index parity for the surviving asset tables 2026-09-17 19:47:30 -07:00
Simon Pinfold 02842ce6bd fix(assets): drop asset_meta when downgrading a database that already created it 2026-09-17 19:47:30 -07:00
Simon Pinfold b9c428b6c5 chore(assets): drop the unused asset_meta table from migration 0007 2026-09-17 19:47:29 -07:00
rattus 7a0b5eede3 Aimdo 0.5.5 + Auto-detect and enable --fast-disk when the disk is fast (CORE-440) (#16333) 2026-09-15 17:30:01 -04:00
rattus b2da2b4247 Declare loop boundaries in node schema (#16347) 2026-09-15 15:27:07 -04:00
Lukas Buck a84f954bc6 Carry every item of a heterogeneous list through a loop (#16345)
Start Loop took the first delivered value of `initial_iteration_value` and
LoopIteration took the first value of `current_iteration_value`, so a Create
List holding mixed state reached the loop body with only its first item. Get
Item From List then raised `IndexError` for every index past 0, which is the
opposite of what the generic loop advertises: one carried value holding an
image and its prompt, unpacked inside the iteration.

Pass the opener's own input into the expanded graph so the first iteration
receives the carried value through the same link the later ones use, and let
LoopIteration re-emit it whole.
2026-09-15 15:26:30 -04:00
guill 36da3ff763 Temporarily disable testing with --enable-assets (#16334)
This is just to unblock other work and reduce frustration while
we address the flaky tests.
2026-09-14 21:32:24 -07:00
rattus 50ab50c15a Implement Generic Loops (Candidate III - implemented ✅) (CORE-14) (#16227) 2026-09-14 16:56:34 -04:00
f42b24efbe feat: structured event log lines for the assets system (#16306)
* feat(assets): structured event log lines for the assets system

* test(assets): AST check that tagged event lines stay inside the closed vocabulary

* feat(assets): structured event logging for the seeder lifecycle

* feat(assets): structured event logging for scanner and ingest failure paths

* feat(assets): structured event logging for API request failures

* fix(seeder): finalize checkpoint no longer parks completed scans

* fix(scanner): enrich_failed counts exceptions, not benign races

* fix(seeder): idle reset survives emit/assert failures

* refactor(assets): private emit-once bookkeeping helper; clean Progress DTO

* test(event-log): call-site registry as frozenset; drop dead pending machinery

* feat(event-log): scanner.stat_failed with emit-once discipline; drop dead vocabulary

* refactor(assets): drop API request failure events

* refactor(assets): drop ingest failure events

* refactor(assets): narrow event vocabulary to scan pipeline

* test(assets): per-call-site event tests collapse to mechanism-once

* refactor(assets): event log lines as logfmt, matching the log's own idiom

* test(assets): adapt hash-failure log assertion to privacy-safe scan-error logging

The rebase onto master picked up the privacy-safe _log_scan_error from
PR 16096, which no longer includes the file path in the log message.
Assert on the generic 'Asset scan error' message and that the path does
not leak, instead of asserting the path is present.

* fix(assets): enforce event vocabulary and scan counters

---------

Co-authored-by: Simon Pinfold <synap5e@users.noreply.github.com>
Co-authored-by: guill <jacob.e.segal@gmail.com>
2026-09-13 22:24:33 -07:00
Simon Pinfold eecbfb4046 test(assets): keep test typing 3.10-compatible (#16305) 2026-09-13 21:31:09 -07:00
19e1058f4c feat(assets): split asset records from content (#16295)
* review-stack 1/4: code (37 files, +3217/-3958)

Review-and-land stack for synap5e/feat/asset-record-content-split, generated by review-stack.py. Once approved,
merges DOWN into the layer below (a fast-forward); only the bottom layer
squash-merges into the real base. See ~/adocs/review-stack.md.
Rule: path not under tests-unit/ or tests/
Question: Is the logic change right?
Source tip: 7007d18582
Merge-base: 783545f689

* review-stack 2/4: tests-removed (24 files, +274/-8220)

Review-and-land stack for synap5e/feat/asset-record-content-split, generated by review-stack.py. Once approved,
merges DOWN into the layer below (a fast-forward); only the bottom layer
squash-merges into the real base. See ~/adocs/review-stack.md.
Rule: test file deleted, or modified with deleted/(added+deleted) >= 0.9
Question: For each dropped assertion: obsolete by a ruling, or covered by a tests-new test?
Source tip: 7007d18582
Merge-base: 783545f689

* review-stack 3/4: tests-changed (13 files, +1043/-1218)

Review-and-land stack for synap5e/feat/asset-record-content-split, generated by review-stack.py. Once approved,
merges DOWN into the layer below (a fast-forward); only the bottom layer
squash-merges into the real base. See ~/adocs/review-stack.md.
Rule: remaining modified test files (incl. conftest.py / helpers)
Question: Did the edits weaken an existing check?
Source tip: 7007d18582
Merge-base: 783545f689

* review-stack 4/4: tests-new (46 files, +8601/-0)

Review-and-land stack for synap5e/feat/asset-record-content-split, generated by review-stack.py. Once approved,
merges DOWN into the layer below (a fast-forward); only the bottom layer
squash-merges into the real base. See ~/adocs/review-stack.md.
Rule: test file added
Question: Is the code layer well covered?
Source tip: 7007d18582
Merge-base: 783545f689

* review-stack 5/6: code (13 files, +351/-104)

Review-and-land stack for synap5e/feat/assets-di, generated by review-stack.py. Once approved,
merges DOWN into the layer below (a fast-forward); only the bottom layer
squash-merges into the real base. See ~/adocs/review-stack.md.
Rule: path not under tests-unit/ or tests/
Question: Is the logic change right?
Source tip: eca2c74bff
Merge-base: 20d59d2a5f

* review-stack 6/6: tests (8 files, +753/-238)

Review-and-land stack for synap5e/feat/assets-di, generated by review-stack.py. Once approved,
merges DOWN into the layer below (a fast-forward); only the bottom layer
squash-merges into the real base. See ~/adocs/review-stack.md.
Rule: every changed file under tests-unit/ or tests/ (added, modified, or deleted)
Question: Is the code layer well covered, and did any edit weaken an existing check?
Source tip: eca2c74bff
Merge-base: 20d59d2a5f

* review-stack 7/8: ported-fixes (42 files, +1361/-180)

Review-and-land stack for synap5e/feat/assets-di-v2, generated by
review-stack.py conventions (hand-built continuation layer; see the PR body).
Once approved, merges DOWN into the layer below (a fast-forward); only the
bottom layer squash-merges into the real base. See ~/adocs/review-stack.md.
Rule: the 11 base-branch fix/docs commits 595cd6e4..94d7185b cherry-picked across the DI refactor (7efdd1d7 excluded, superseded by layer 8)
Question: was each base fix ported faithfully across the DI refactor?
Source tip: 6841881069284803b902b4a9e33bdcda13126771
Merge-base: 7fdfb40f4b

* review-stack 8/8: defensive-parity (4 files, +36/-3)

Review-and-land stack for synap5e/feat/assets-di-v2, generated by
review-stack.py conventions (hand-built continuation layer; see the PR body).
Once approved, merges DOWN into the layer below (a fast-forward); only the
bottom layer squash-merges into the real base. See ~/adocs/review-stack.md.
Rule: match-or-improve master's dependency defenses — NoAssets selection when DB deps unavailable (7efdd1d7's outcome via the DI seam), requirements warning before assets imports, blake3 in the guarded dependency set
Question: does each degradation path now match or improve master's behavior?
Source tip: ebc2cfeebc
Merge-base: 7fdfb40f4b

* fix(assets): only discard content rows this operation actually inserted

CR-9: Enumerated all six create_content call sites. Only scanner seeding and the three ingest registration paths track IDs for failure cleanup.

* fix(assets): reject hash-only uploads with FEATURE_DISABLED when hashing is off

CodeRabbit finding CR-2: reject hash-only multipart uploads before create_from_hash when hashing is disabled.

* fix(assets): seed persists the stat it verified

CR-7: persist the fresh seed-time restat instead of walk-time spec values.

* fix(assets): route database lock failures to the lock guidance

CR-16: route file-lock startup failures through the existing lock guidance and exit path.

* fix(assets): drop the inaccurate temp-cleanup claim from the shutdown warning

References CR-10.

* fix(assets): walk the output root after execution so undeclared outputs register promptly

Custom nodes that write files into the output directory without declaring
them in output_ui only became assets when the next full walk happened - a
frontend GET /object_info or a restart. Headless and API-only sessions never
trigger either, so those files never converged into the asset database.

The post-execution hook now requests a FULL scan of the output root instead of
an enrich-only pass. The seeder's pending-request queue was generalised from
enrich-specific to carrying a scan phase, so the request starts immediately
when the seeder is idle and coalesces (escalating to FULL on a phase mismatch)
when a scan is already running. queue_output_enrichment is renamed to
queue_output_scan across the protocol, the NoAssets no-op and the call site.

References FIX-6.

* chore(assets): remove seeder paths orphaned by the output-scan change

45c2f96e rerouted both former enrich call sites to start()/enqueue_scan(),
leaving two seeder methods that look live but are not. Review round F2
raised this along with four smaller items; the user's disposition was to
fix all six here.

- Delete start_enrich: zero callers repo-wide after 45c2f96e.
- Delete enqueue_enrich: no production callers; its ~18 call sites in
  tests/test_asset_seeder.py move to enqueue_scan(phase=ScanPhase.ENRICH)
  with their semantics unchanged. The deletion forces the half-done class
  renames (TestEnqueueEnrich* -> TestEnqueueScan*, consistent with the
  already-renamed TestPendingScanDrain) and restores the module docstring
  that was dropped rather than reworded.
- Document at manager.queue_output_scan that ScanPhase.FULL per debounce
  window is the deliberate, user-ratified trade, so it is not optimised
  back to ENRICH without revisiting the decision.
- Document that SeedAssetSpec.size_bytes/mtime_ns are walk-time
  diagnostics only - production persists the seed-time restat since CR-7.
- Export create_content_reporting_insert from the queries facade and fold
  scanner.py's direct-module import into the existing facade block.
- Harden test_queue_output_scan_does_not_duplicate_declared_output against
  a vacuous pass: it now asserts the seeder finished without errors and
  that an undeclared sibling written into the same directory WAS
  registered by the same scan, proving the walk actually ran.

No production behaviour changes beyond the two deletions.

References F2-cleanup.

* chore: comment cleanup

Comment-Gate: 18 quarantined

* fix(assets): preserve pause across the seeder's pending-scan drain

pause() runs before every prompt, while pending-scan enqueue and resume only run inside the debounced gc-interval gate. If the active scan finishes just after the next prompt's pause, its finally block resets the seeder to idle and the pending drain starts a replacement with the run gate open, so resume becomes a no-op.

Capture pausedness under the lock before resetting to idle, then start the drained scan already paused. Setting the state and gate before launching the thread avoids the start-then-reclear window and lets resume release the existing scan checkpoints.

Ultraworked with [Sisyphus](https://github.com/code-yeongyu/oh-my-openagent)

Co-authored-by: Sisyphus <clio-agent@sisyphuslabs.ai>

* test(assets): pin job_id absence for scan-discovered assets

Owner ruling, recorded 2026-09-03 in the stack-9-hardening planning notepad: scan-discovered assets — including undeclared outputs found by the post-execution walk — carry job_id = None, always; only emission-time registration (output_ui declaration) attributes a job; attributing walk finds to the most recent prompt would be a temporal-correlation guess that is wrong exactly when prompts interleave; None is honest provenance. Do NOT add proximity-based attribution heuristics to the scanner. Ratified against Jacob Segal's cross-job-attribution concern (2026-09-08 review meeting) — a wrongly-attributed asset could mean one user's cloud job sees another user's asset.

Ultraworked with [Sisyphus](https://github.com/code-yeongyu/oh-my-openagent)
Co-authored-by: Sisyphus <clio-agent@sisyphuslabs.ai>

* [review-stack 10/10] assets-tests (#16218)

* test(execution): run the battery with assets enabled and assert asset-system health at teardown

* test(execution): cover list-shaped outputs registering assets

Ultraworked with [Sisyphus](https://github.com/code-yeongyu/oh-my-openagent)

Co-authored-by: Sisyphus <clio-agent@sisyphuslabs.ai>

---------

Co-authored-by: Sisyphus <clio-agent@sisyphuslabs.ai>

* [review-stack 11/11] review-fixes (#16261)

* fix(assets): only exit on database file-lock timeout when assets are enabled

* test(assets): pin live_contents_under_prefixes path-filtering semantics

* perf(assets): push live-content prefix filtering into SQL

* test(assets): declare per-entry intent in the path-prefix corpus

* test(assets): normalize POSIX-literal path expectations for Windows

* test(assets): force observable stat changes and close-before-mutate on Windows-sensitive rewrites

* test(assets): force an observable mtime change in the hash-mode split test

---------

Co-authored-by: Sisyphus <clio-agent@sisyphuslabs.ai>
Co-authored-by: guill <jacob.e.segal@gmail.com>
2026-09-13 12:04:51 -07:00
Christian Byrne d43a5fa20c Log typed asset scanner filesystem errors (#16096) 2026-09-12 16:54:50 -07:00
Jialong(Bruce) Li a20738f1d3 Fix linear_input_act to respect _full_precision_mm fallback (#16285) 2026-09-12 18:45:38 -04:00
Christian Byrne 7ba217d6b5 Don't add noise to the alpha channel in the Add Noise to Image node (#15626) 2026-09-12 15:28:16 -07:00
Alexis Rolland c40c94e1f5 Fix unit tests (#16271) 2026-09-11 14:27:22 -04:00
rattus d537de93a0 Implement Video Concatenate (CORE-436) (#16267) 2026-09-11 11:06:33 -07:00
Terry Jia 3074d0e331 Report the file saved by Save 3D (Advanced) as a standard 3d output item (#16171) 2026-09-08 21:50:21 -07:00
poorpaper 421a1c245c Fix MiniMax H3 denoise masks (#15988) 2026-09-08 14:25:33 -07:00
Alex Artyomov 0d0b6b5694 Add LTXVAddLatentGuide for pinning a pre-encoded latent as a guide (#16176) 2026-09-08 14:12:47 -07:00