Pack schemas use it for input bounds, and the module that has always
defined it (nodes.py) is a host module a sandboxed pack cannot import.
Same value by definition: 16384 is frozen into every workflow that
ever serialized a bound.
The V2 additions the KJNodes completion needed on the core side:
- _model_transforms: the closed, core-owned transform vocabulary
behind ModelRef.patch — 29 named transforms, declaratively
parameterized, validated host-side, immutable and stacking. No
function ever crosses the boundary; a pack cannot register one.
- structured-vs-live split: value()/from_value() only on structured
data refs (LATENT, AUDIO, TRACKS...). MODEL/CLIP/VAE/asset refs are
handles in every execution mode — in-process identity resolution no
longer hands a live model to node code.
- preview overrides (tiny-VAE, LTX factors), triton VAE seam, memory
attention, and profiling surfaces backing the corresponding closed
brokers in the overlay.
- torch_compile/model_patcher/model_management: compiled-view
aliasing recognized by the model manager (no double-counted
weights); shared state-dict loading path so the native loader and
the V2 broker cannot drift.
VaeRef.decode/encode, ClipRef.tokenize/encode_from_tokens_scheduled/encode,
CondRef.combine/concat, with in-process implementations behind the existing
named-op registry so an overlay extends the vocabulary without touching the
contract.
These keep the old API's shape on purpose — you still write vae.decode(latent)
— and change only what the call means: the node holds a handle, awaits, and the
decode runs on the trusted plane against weights it never sees. That is what
lets a node DECLARE a VAE input and still be sandboxable. Compatible in shape so
conversion is mechanical across a corpus nobody here maintains; different in
substance so a converted node is sandboxable by construction.
Two mirroring defects fixed before shipping: encode() sliced its input to three
channels (core's VAEEncode does not, so it silently dropped alpha), and ClipRef
offered only a combined encode(text) (CLIPTextEncodeSDXL builds one token dict
from two prompts, and the ACE nodes pass a dozen tokenizer kwargs, so the
collapse made both inexpressible).
InProcessRefResolver now records a ref's kind at creation and checks it at
resolve. Ref tokens cross as {kind, id, cls} and the host rebuilt from what
arrived, so the holder chose its own ref's type: an ImageRef id could be
presented as a VaeRef. Unguessable ids already stopped a guest reaching a handle
it was never given; possessing a handle is not the same as labelling it.
Release is explicit, not collected. InProcessRefResolver.clear() drops the
table's strong references at a known point, called from execution.py in a
finally so it also covers the path where the node raised or its guest died —
which is exactly when nothing else will run. A ref table can hold multi-gigabyte
tensors, and refcount timing neither crosses a process boundary nor is bounded
under reference cycles.
OpsProvider.apply is annotated (op, subject: Ref, params) -> Any; it claimed
ImageRef in and out while handle ops take a VaeRef and may return a LatentRef,
a CondRef, or a plain token dict.
Two generic gaps that blocked an out-of-process node from being a sampler.
1. wrap_inputs only wrapped tensors and latents. A MODEL or CONDITIONING is a
live engine object, so it passed straight through — and an out-of-process
backend cannot serialize a ModelPatcher, so any node taking one simply could
not run in a guest. The rule is now by capability rather than an enumerated
type list: a value that can cross as data does, and anything else becomes a
handle. That is the correct boundary rule anyway — objects do not cross,
handles do — and it is what lets a node take a MODEL and still run isolated.
2. ExecutionPlan.permissions was never populated, so the seam could not tell a
backend what a node needs. A node now declares SDK_PERMISSIONS and the seam
copies it onto the plan. Declaring is not granting: the backend decides, and
an out-of-process one still gates every call at the wire. Nodes that declare
nothing — the overwhelming majority — get nothing.
Behaviour-preserving in-process: 54 core seam tests and 26 overlay tests pass.
`unwrap_outputs` rebuilds a node's NodeOutput in order to swap output refs back
for real objects. It rebuilt it from the results alone — `NodeOutput(*resolved)`
— silently discarding `ui`, `expand` and `block_execution`.
The practical effect: no SDK_REFS node could be an output node. ComfyUI only
emits the `executed` websocket event, the one that delivers a node's results to
the frontend, for nodes that return ui data (`if len(output_ui) > 0`). So a
converted PreviewImage-style node executed perfectly and then displayed
nothing, with no error anywhere to explain it. `expand` (subgraph expansion)
and `block_execution` were lost the same way.
Resolving refs is a transport concern and has no business changing what the
node said. Generic fix, not specific to any backend: it applies equally to the
in-process path.
Replaces enumerated invert/scale methods on OpsProvider with generic dispatch:
apply(op, image, params) + supports(op) + a built-in registry {invert, scale}
+ register_op. ImageRef.op(name, **params) is the untyped transport seam;
invert()/scale() remain as built-in convenience. Adds OpNotSupported (carries
the capability name) so a node can fall back to the raw tier. An overlay now
extends the op vocabulary without touching OSS core.
Statically-typed op methods live on the secure-lib side (per guidance), not
here — core stays a generic seam.
Generic out-of-process enablement: the ExecutionPlan now ships the node's
module spec and ref-wrapped inputs, and dispatch receives the per-node host
runtime (refs/ctx/ops) so an external backend can execute the node elsewhere
and broker guest calls against the same ref table. In-process default ignores
all of it; legacy nodes unaffected (seam tests green).
Nodes operate on assets (image.invert()) and never receive buffers; compute
runs on the trusted plane via the OpsProvider seam. Raw buffer access becomes
a permissioned, discouraged escape hatch (raw(); forces dedicated tier under
the overlay). The execution seam wraps heavy inputs as refs for SDK_REFS
nodes and resolves output refs for downstream legacy nodes. The .pyi contract
no longer imports torch.
POC stand-ins (interface debt, ledgered in the overlay repo DEBT.md):
invert/scale enumerated on OpsProvider; duck-typed wrap_inputs; SDK_REFS
class-attr opt-in.
- execution.py: route V3 node dispatch through providers.execution_backend
with a behavior-preserving local_call closure (exact sync/async-task
semantics retained); bind per-node ctx+refs inside the invocation scope so
the concurrent-async path is correct. V1 nodes untouched. Default backend =
in-process => byte-identical to today.
- comfy_api/latest/_sdk_public.pyi: authoritative type contract for the secure
SDK (backend analog of the frontend comfy-api.d.ts): refs, ctx + domains,
ctx() accessor; host/overlay seam separated.
- custom_nodes/comfy_sdk_poc: SandboxInvert POC node authored to the SDK.
- tests-unit: seam regression (sync+async SDK nodes through the real engine;
provider-swap intercept). Verified PASS.
Establishes the open-source 'key Python API' for secure custom nodes:
- comfy_api/latest/_sdk.py: opaque typed refs (ImageRef/ModelRef/AssetRef...),
a brokered ctx surface (assets/progress/scratch/events/storage + stubs),
and a Providers registry (ExecutionBackend/CtxProvider/RefResolver) with
in-process DEFAULTS so OSS behaves exactly as today (a ref wraps the real
object; ctx is a passthrough; zero-copy, zero-overhead).
- load_overlay(): env-var (COMFY_OVERLAY_MODULE) path loader that lets a
separable, proprietary cloud overlay register isolated implementations at
the seam. Unset => pure OSS. Wired guarded into main.startup.
- comfy_api/v0_0_3: new API version exposing sdk; registered in version_list.
Nothing isolation-specific lives in OSS: this is the seam, not the engine.
* feat(assets): add namespaced model type tags
* fix(assets): mark path-derived upload tags automatic
* fix(assets): merge duplicate scan specs
* test(assets): make duplicate path normalization portable
* feat(assets): add loader_path as the authoritative loader locator (#14796)
* fix(assets): filter model_type tags by bucket extension sets
Buckets sharing a base directory (e.g. diffusion_models and a custom
unet_gguf) tagged every file in the directory regardless of whether the
bucket could load it, so .safetensors files were tagged
model_type:unet_gguf and vice versa. Carry each bucket's registered
extension set through get_comfy_models_folders and only emit a
model_type tag when the file extension matches, keeping the empty-set
match-all convention from folder_paths.filter_files_extensions.
Files under a model base matching no bucket now keep only the models
tag instead of every directory-matching model_type tag.
* feat(assets): replace response file_path with persisted loader_path
The old file_path response field was a namespaced storage locator
(models/checkpoints/foo.safetensors): not an absolute path, not unique
identity, and not the value a loader consumes. Nothing needs that shape
on the wire (hash/ID-based locating is the long-term direction), so it
is dropped rather than renamed; the storage-root matching stays internal,
powering display_name.
What loaders DO need is the in-root loader path (category dropped:
models/checkpoints/foo/bar.safetensors -> foo/bar.safetensors). Serve it
as a first-class loader_path field, persisted on asset_references
(migration 0006) and written by every ingest pipeline at insert, so
responses read the column verbatim.
Like the model_type tags, loader_path is a seed-time derivative of the
model folder registry, maintained by the same scan lifecycle (new files seed
fresh values, pruning retires rows whose bucket disappeared). Rows
predating the column serve a null loader_path; databases from before
this stack already need recreating for the base branch's tag changes.
loader_path resolves every registered base including extra_model_paths
entries; display_name only the canonical storage roots. A file can
therefore be loadable with no display name (extra-path models) or the
reverse (unregistered files under the models root), and loader_path is
null exactly when no loader can resolve the file.
* test(assets): lock loader_path matrix (asymmetry, null, persist/read)
Cover the behaviour that has no production change but is easy to regress:
the extra-path asymmetry (loadable but no storage namespace), null
loader_path persistence for orphan files, and the response reading the
stored column with a compute fallback for un-backfilled rows.
* fix(assets): persist subfolder-qualified loader_path for ingested outputs
ingest_existing_file built its seed spec with the file's basename, so
outputs saved into a subfolder persisted loader_path (and the
user_metadata filename that preview URLs split for their subfolder
param) as just the basename: the served locator pointed at a file that
does not exist at that path. Scanner and seeder specs already derive
fname via compute_loader_path; use the same derivation here.
* fix(assets): only extension-matching buckets contribute a loader_path
The model-base match in get_asset_category_and_relative_path ignored
each bucket's extension set, so a file inside a registered base whose
extension the bucket cannot load (e.g. a .txt uploaded into
model_type:checkpoints) advertised a loader_path that no loader list
would ever resolve, while the tag side of the same stack already
excluded it. Apply the extension check used for backend tags (empty set
accepts any extension), keeping loader_path null exactly when no loader
can resolve the file.
* fix(assets): refresh loader_path when re-ingesting an existing reference
upsert_reference only wrote loader_path on the INSERT branch, so
re-ingesting an existing reference (an output overwritten in place, or a
file re-registered after its loader_path derivation changed) kept the
stale or NULL value forever. Write it on the UPDATE branch too, with a
null-safe change guard so a loader_path difference alone is enough to
trigger the update, and identical values stay a no-op.
* fix(assets): repair semantic merge breakage from #14796 and master
Two textually-clean but semantically-broken merges:
- routes.py lost its folder_paths import when #14796's import block
superseded the base's, while the content-type hardening added via the
base's master merge still calls folder_paths.is_dangerous_content_type.
- master's SVG download-hardening test uploads with the pre-namespacing
bare checkpoints tag, which this branch's destination validation
rejects; use model_type:checkpoints.
---------
Co-authored-by: guill <jacob.e.segal@gmail.com>
a1d95f3f padded the decode width to the next multiple of 32 with the pad filter to fix libswscale's float YUV->GBR edge corruption, but kept the pad target height equal to the source height. The pad filter requires the target height to be a multiple of the input's vertical chroma subsampling factor, so a chroma-subsampled input such as yuv420p (the format the gbrpf32le float branch decodes) with an odd height makes the filter round the target below the input height and fail to configure: 'Padded dimensions cannot be smaller than input dimensions' (Errno 22). This is reachable from LoadImage, which routes static images through VideoFromFile, on a lossy WebP whose width is not a multiple of 32 and whose height is odd.
The pad filter also fills the added border with black, and chroma upsampling bleeds that black into the cropped edge of every unaligned-width subsampled decode.
Pad both axes to the next multiple of 32 (32 is a multiple of every vertical subsampling factor, including yuv410p's 4 that a plain even rounding misses) and run fillborders mode=smear to replicate the real edge into the padding so it never bleeds into the cropped output, then crop both axes back to the source size. Aligned-width and uint8 paths run the identical to_ndarray call as before and are byte-identical to master; only unaligned-width subsampled inputs change, from a crash or edge artifact to a clean, deterministic decode.
Create Video gets a bit_depth option (8-bit/10-bit); the selected depth is carried by the video and applied when it gets encoded. Save Video and Video Slice now keep the source bit depth instead of always quantizing to 8-bit, so 10-bit videos stay 10-bit. 10-bit uses h264 with the yuv420p10le pixel format,so there's no new codec or container.
Signed-off-by: bigcat88 <bigcat88@icloud.com>
Split GLB save logic out of nodes_hunyuan3d.py into a new nodes_save_3d.py, and extend the writer to support UVs, per-vertex colors, and embedded baseColor textures.
Extend the MESH type with optional uvs, vertex_colors, and texture fields so meshes can carry texture data through the graph.
Add pack_variable_mesh_batch / get_mesh_batch_item helpers and switch VoxelToMesh / VoxelToMeshBasic to use them so batches with differing vertex/face counts no longer fail at torch.stack.
benjcooley