fix: use sd_get_preview_interval() (#1907)

.github/workflows/build.yml

@@ -23,6 +23,7 @@ on:
         "**/*.c",
         "**/*.cpp",
         "**/*.cu",
+        "ggml",
         "examples/server/frontend",
         "examples/server/frontend/**",
       ]
@@ -40,6 +41,7 @@ on:
         "**/*.c",
         "**/*.cpp",
         "**/*.cu",
+        "ggml",
         "examples/server/frontend",
         "examples/server/frontend/**",
       ]

README.md

@@ -152,6 +152,7 @@ For runtime and parameter backend placement, see the [backend selection guide](.
 - [LCM/LCM-LoRA](./docs/lcm.md)
 - [Docker](./docs/docker.md)
 - [Quantization and GGUF](./docs/quantization_and_gguf.md)
+- [INT8 convrot safetensors](./docs/int8_convrot.md)
 - [Inference acceleration via caching](./docs/caching.md)
 
 ## Bindings

docs/int8_convrot.md

@@ -0,0 +1,102 @@
+# INT8 Convrot Safetensors
+
+sd.cpp can load and execute ComfyUI `int8_tensorwise` safetensors with `convrot` metadata directly. The stored INT8 weights are not converted to another weight type at load time.
+
+## Checkpoint format
+
+Each quantized linear module contains the following tensors:
+
+- `<module>.weight`: an I8 weight matrix.
+- `<module>.weight_scale`: one floating-point scale for each output row. ComfyUI's two-dimensional `[out_features, 1]` representation is normalized to a one-dimensional tensor while loading.
+- `<module>.comfy_quant`: a U8 tensor containing the JSON quantization configuration.
+
+A supported configuration has this form:
+
+```json
+{
+  "format": "int8_tensorwise",
+  "convrot": true,
+  "convrot_groupsize": 256
+}
+```
+
+The convrot group size must be a power of four and must divide the input feature dimension. The commonly used configuration is H256, with `convrot_groupsize` set to `256`.
+
+## How INT8 convrot works
+
+Convrot combines an offline rotation of the weights with the same rotation of the activations at runtime. The rotation uses a normalized regular Hadamard matrix constructed recursively from
+
+```text
+     [ 1  1  1 -1 ]
+     [ 1  1 -1  1 ]
+H4 = [ 1 -1  1  1 ] / 2
+     [-1  1  1  1 ]
+```
+
+For a group size `G = 4^n`, the transform is the normalized Kronecker power of `H4`. It is applied independently to every contiguous group of `G` input features. The resulting block-diagonal rotation matrix `R` is orthogonal and symmetric, so `R R^T = I`.
+
+For an original floating-point linear layer
+
+```text
+Y = X W^T + b
+```
+
+the checkpoint stores a rotated weight matrix `W_rot = W R`, quantized per output row. At runtime sd.cpp computes `X_rot = X R`. Ignoring quantization error,
+
+```text
+X_rot W_rot^T = X R (W R)^T = X R R^T W^T = X W^T
+```
+
+The rotation therefore preserves the linear operation. Its purpose is to spread isolated large values across each feature group, reducing the effect of outliers on tensorwise INT8 quantization.
+
+### Weight quantization
+
+The rotated weights are quantized offline with one scale per output row:
+
+```text
+s_w[o]      = max_i(abs(W_rot[o, i])) / 127
+Q_w[o, i]   = clamp(round(W_rot[o, i] / s_w[o]), -127, 127)
+```
+
+`Q_w` is stored in `<module>.weight`, and `s_w` is stored in `<module>.weight_scale`.
+
+### Runtime activation quantization
+
+For every activation row, sd.cpp applies the group-wise Hadamard rotation and then calculates one dynamic scale across the entire rotated row:
+
+```text
+s_x[r]      = max_i(abs(X_rot[r, i])) / 127
+Q_x[r, i]   = clamp(round(X_rot[r, i] / s_x[r]), -127, 127)
+```
+
+The matrix multiplication accumulates into signed 32-bit integers:
+
+```text
+A[r, o] = sum_i(Q_x[r, i] * Q_w[o, i])
+```
+
+The floating-point output is reconstructed as
+
+```text
+Y[r, o] ~= A[r, o] * s_x[r] * s_w[o] + b[o]
+```
+
+The packed runtime activation tensor contains the I8 activation rows and their floating-point row scales. Linear layers that share the same input and convrot group size reuse this packed tensor, avoiding repeated rotation and activation quantization within the graph.
+
+## Backend support
+
+- CPU provides the portable regular Hadamard, activation quantization, INT8 matrix multiplication, and scale restoration implementations.
+- NVIDIA CUDA devices with compute capability 7.5 or newer use the native accelerated path. For H256, CUDA fuses the rotation, row-wise maximum reduction, and activation quantization. It uses cuBLAS for I8 x I8 to I32 GEMM and a CUDA kernel for scale restoration and bias addition.
+- Vulkan and other GPU backends do not currently have dedicated INT8 convrot kernels. They use the backend scheduler to fall back to CPU, which is expected to be substantially slower than the CUDA path.
+
+LoRA adapters are applied at runtime without modifying the INT8 weights. The INT8 convrot path computes the base linear output, while LoRA, LoHa, LoKr, and raw weight-difference adapters compute their output corrections from the original, unrotated activation and add them to the base output. `--lora-apply-mode auto` selects this path for models containing INT8 tensorwise weights. If `immediately` is requested, sd.cpp falls back to runtime application because merging an adapter would require dequantizing and rotating its weight update, then recalculating the per-row scales and requantizing the result.
+
+The dedicated CUDA convrot activation path currently requires a group size of `256`; other supported group sizes use CPU execution.
+
+## Example
+
+ComfyUI INT8 convrot safetensors can be passed to `--diffusion-model` without conversion:
+
+```powershell
+.\bin\Release\sd-cli.exe --diffusion-model ..\models\diffusion_models\krea2_turbo_int8_convrot.safetensors --llm ..\models\text_encoders\Qwen3-VL-4B-Instruct-Q4_K_M.gguf --vae ..\models\vae\wan_2.1_vae.safetensors -p "a lovely cat holding a sign says 'krea2.cpp'" --steps 8 --cfg-scale 1 --diffusion-fa -v --offload-to-cpu
+```

examples/common/common.cpp

@@ -1008,7 +1008,7 @@ ArgOptions SDGenerationParams::get_options() {
          &hires_upscaler},
         {"",
          "--extra-sample-args",
-         "extra sampler/scheduler/guidance args, key=value list. CFG supports guidance_schedule; APG supports apg_eta, apg_momentum, apg_norm_threshold, apg_norm_threshold_smoothing; SLG supports slg_uncond; lcm supports noise_clip_std, noise_scale_start, noise_scale_end; flux supports base_shift, max_shift; ltx2 supports max_shift, base_shift, stretch, terminal; euler_ge supports gamma; beta scheduler supports alpha, beta; logit_normal supports mu, std, logsnr_min, logsnr_max, resolution_aware; lms supports lms_divisions",
+         "extra sampler/scheduler/guidance args, key=value list. CFG supports guidance_schedule; APG supports apg_eta, apg_momentum, apg_norm_threshold, apg_norm_threshold_smoothing; SLG supports slg_uncond; lcm supports noise_clip_std, noise_scale_start, noise_scale_end; flux supports base_shift, max_shift; ltx2 supports max_shift, base_shift, stretch, terminal; euler_ge supports gamma; beta scheduler supports alpha, beta; logit_normal supports mu, std, logsnr_min, logsnr_max, resolution_aware; lms supports lms_max_order, lms_shift, lms_divisions",
          (int)',',
          &extra_sample_args},
         {"",
@@ -1555,6 +1555,16 @@ ArgOptions SDGenerationParams::get_options() {
         return 1;
     };
 
+    std::string sample_methods = sample_method_to_str[0];
+    for (int i = 1; i < SAMPLE_METHOD_COUNT; i++) {
+        sample_methods += ", " + std::string(sample_method_to_str[i]);
+    }
+
+    std::string schedulers = scheduler_to_str[0];
+    for (int i = 1; i < SCHEDULER_COUNT; i++) {
+        schedulers += ", " + std::string(scheduler_to_str[i]);
+    }
+
     options.manual_options = {
         {"-s",
          "--seed",
@@ -1562,17 +1572,18 @@ ArgOptions SDGenerationParams::get_options() {
          on_seed_arg},
         {"",
          "--sampling-method",
-         "sampling method, one of [euler, euler_a, heun, dpm2, dpm++2s_a, dpm++2m, dpm++2mv2, dpm++2m_sde, dpm++2m_sde_bt, ipndm, ipndm_v, lcm, ddim_trailing, tcd, res_multistep, res_2s, er_sde, euler_cfg_pp, euler_a_cfg_pp, lms]"
-         "(default: euler for Flux/SD3/Wan, euler_a otherwise)",
+         "sampling method, one of [" + sample_methods + "], "
+                                                        "default: euler for Flux/SD3/Wan, euler_a otherwise",
          on_sample_method_arg},
         {"",
          "--high-noise-sampling-method",
-         "(high noise) sampling method, one of [euler, euler_a, heun, dpm2, dpm++2s_a, dpm++2m, dpm++2mv2, dpm++2m_sde, dpm++2m_sde_bt, ipndm, ipndm_v, lcm, ddim_trailing, tcd, res_multistep, res_2s, er_sde, euler_cfg_pp, euler_a_cfg_pp, lms]"
-         " default: euler for Flux/SD3/Wan, euler_a otherwise",
+         "(high noise) sampling method, one of [" + sample_methods + "], "
+                                                                     "default: euler for Flux/SD3/Wan, euler_a otherwise",
          on_high_noise_sample_method_arg},
         {"",
          "--scheduler",
-         "denoiser sigma scheduler, one of [discrete, karras, exponential, ays, gits, smoothstep, sgm_uniform, simple, kl_optimal, lcm, bong_tangent, ltx2, logit_normal, flux2, flux, beta], alias: normal=discrete, default: model-specific",
+         "denoiser sigma scheduler, one of [" + schedulers + "], "
+                                                             "alias: normal=discrete, default: model-specific",
          on_scheduler_arg},
         {"",
          "--sigmas",

include/stable-diffusion.h

@@ -60,6 +60,8 @@ enum sample_method_t {
     SAMPLE_METHOD_COUNT
 };
 
+extern SD_API const char* sample_method_to_str[];
+
 enum scheduler_t {
     DISCRETE_SCHEDULER,
     KARRAS_SCHEDULER,
@@ -80,6 +82,8 @@ enum scheduler_t {
     SCHEDULER_COUNT
 };
 
+extern SD_API const char* scheduler_to_str[];
+
 enum prediction_t {
     EPS_PRED,
     V_PRED,

scripts/merge_safetensors.py

@@ -23,16 +23,16 @@ from typing import BinaryIO
 # Configuration
 # -----------------------------------------------------------------------------
 
-OUTPUT_PATH = Path(r"..\models\diffusion_models\minimax_h3_ref2va_pruned_bf16.safetensors")
+OUTPUT_PATH = Path(r".minimax_h3_fl2va_pruned_bf16.safetensors")
 
 SOURCE_RULES = [
     {
-        "path": Path(r"..\models\diffusion_models\minimax_h3_ref2va_bf16.safetensors"),
+        "path": Path(r".minimax_h3_fl2va_bf16.safetensors"),
         "include": [r".*"],
         "exclude": [r".*adaln_proj\.linear.*", r"time_embedder.*"],
     },
     {
-        "path": Path(r"..\models\diffusion_models\minimax_h3_ref2va_pruned_int8_convrot.safetensors"),
+        "path": Path(r".minimax_h3_fl2va_pruned_int8_convrot.safetensors"),
         "include": [r"^.*adaln_proj\.linear.*", "adaln_t_table"],
         "exclude": [],
     },

src/core/ggml_extend.hpp

@@ -1039,6 +1039,38 @@ __STATIC_INLINE__ ggml_tensor* ggml_ext_linear(ggml_context* ctx,
     return x;
 }
 
+__STATIC_INLINE__ ggml_tensor* ggml_ext_linear_i8_tensorwise(ggml_context* ctx,
+                                                             ggml_tensor* x,
+                                                             ggml_tensor* w,
+                                                             ggml_tensor* weight_scale,
+                                                             ggml_tensor* b,
+                                                             int convrot_group_size,
+                                                             float scale = 1.f) {
+    GGML_ASSERT(x->type == GGML_TYPE_F32 || (x->type == GGML_TYPE_I8 && scale == 1.f));
+    if (scale != 1.f) {
+        x = ggml_ext_scale(ctx, x, scale);
+    }
+
+    ggml_tensor* fused_bias = scale == 1.f ? b : nullptr;
+    if (x->ne[2] * x->ne[3] > 1024) {
+        int64_t ne2 = x->ne[2];
+        int64_t ne3 = x->ne[3];
+        x           = ggml_reshape_2d(ctx, x, x->ne[0], x->ne[1] * x->ne[2] * x->ne[3]);
+        x           = ggml_mul_mat_i8_tensorwise(ctx, w, x, weight_scale, fused_bias, convrot_group_size);
+        x           = ggml_reshape_4d(ctx, x, x->ne[0], x->ne[1] / ne2 / ne3, ne2, ne3);
+    } else {
+        x = ggml_mul_mat_i8_tensorwise(ctx, w, x, weight_scale, fused_bias, convrot_group_size);
+    }
+
+    if (scale != 1.f) {
+        x = ggml_ext_scale(ctx, x, 1.f / scale);
+        if (b != nullptr) {
+            x = ggml_add_inplace(ctx, x, b);
+        }
+    }
+    return x;
+}
+
 __STATIC_INLINE__ ggml_tensor* ggml_ext_pad_ext(ggml_context* ctx,
                                                 ggml_backend_t backend,
                                                 ggml_tensor* x,
@@ -1679,6 +1711,13 @@ struct WeightAdapter {
                                            ggml_tensor* b,
                                            const std::string& prefix,
                                            ForwardParams forward_params)                                                              = 0;
+    virtual ggml_tensor* add_lora_to_output(ggml_context* ctx,
+                                            ggml_backend_t backend,
+                                            ggml_tensor* x,
+                                            ggml_tensor* w,
+                                            ggml_tensor* output,
+                                            const std::string& prefix,
+                                            ForwardParams forward_params)                                                             = 0;
     virtual size_t get_extra_graph_size()                                                                                             = 0;
 };
 
@@ -1696,6 +1735,7 @@ struct GGMLRunnerContext {
     std::function<ggml_tensor*(const std::string&)> get_cache_tensor;
     std::function<void(const std::string&, ggml_tensor*)> cache_tensor;
     std::function<void(ggml_tensor*, const void*)> set_backend_tensor_data;
+    std::map<std::pair<ggml_tensor*, int>, ggml_tensor*> int8_convrot_cache;
 
     void capture_tensor(const std::string& name, ggml_tensor* tensor) {
         if (debug_tensors == nullptr || tensor == nullptr) {
@@ -1754,7 +1794,8 @@ protected:
 
     std::vector<ggml_backend_t> extra_runtime_backends;  // borrowed (SDBackendManager-owned)
     ggml_backend_sched_t sched             = nullptr;    // owned
-    ggml_backend_t cpu_fallback_backend    = nullptr;    // owned, sched requires a trailing CPU backend
+    size_t sched_graph_capacity            = 0;
+    ggml_backend_t cpu_fallback_backend    = nullptr;  // owned, sched requires a trailing CPU backend
     bool multi_device_eval_callback_warned = false;
 
     std::shared_ptr<WeightAdapter> weight_adapter = nullptr;
@@ -2040,9 +2081,19 @@ protected:
     // Pass explicit buffer types: synthesized defaults can make CUDA devices
     // report supporting each other's buffers and skip a required copy.
     bool ensure_sched(ggml_cgraph* gf) {
-        if (sched != nullptr) {
+        const size_t required_graph_size = gf != nullptr
+                                               ? std::max<size_t>(1,
+                                                                  (size_t)ggml_graph_n_nodes(gf) +
+                                                                      sd::ggml_graph_cut::leaf_count(gf))
+                                               : 1;
+        if (sched != nullptr && sched_graph_capacity >= required_graph_size) {
             return true;
         }
+        if (sched != nullptr) {
+            ggml_backend_sched_free(sched);
+            sched                = nullptr;
+            sched_graph_capacity = 0;
+        }
         std::vector<ggml_backend_t> backends;
         backends.reserve(extra_runtime_backends.size() + 2);
         backends.push_back(runtime_backend);
@@ -2070,20 +2121,17 @@ protected:
             bufts.push_back(buft);
         }
 
-        size_t graph_size = MAX_GRAPH_SIZE;
-        if (gf != nullptr) {
-            graph_size = std::max<size_t>(graph_size, (size_t)ggml_graph_n_nodes(gf));
-        }
         sched = ggml_backend_sched_new(backends.data(),
                                        bufts.data(),
                                        (int)backends.size(),
-                                       graph_size,
+                                       required_graph_size,
                                        /*parallel=*/false,
                                        /*op_offload=*/false);
         if (sched == nullptr) {
             LOG_ERROR("%s: failed to create backend sched", get_desc().c_str());
             return false;
         }
+        sched_graph_capacity = required_graph_size;
         return true;
     }
 
@@ -2408,13 +2456,14 @@ protected:
         GGML_ASSERT(gf != nullptr);
 
         size_t effective_budget = max_graph_vram_bytes;
+        size_t free_clamp       = SIZE_MAX;
         if (stream_layers_enabled && max_graph_vram_bytes > 0 && runtime_backend != nullptr) {
             ggml_backend_dev_t dev = ggml_backend_get_device(runtime_backend);
             if (dev != nullptr && ggml_backend_dev_type(dev) != GGML_BACKEND_DEVICE_TYPE_CPU) {
                 size_t free_vram = 0, total_vram = 0;
                 ggml_backend_dev_memory(dev, &free_vram, &total_vram);
                 constexpr size_t safety_margin = 512ull * 1024 * 1024;
-                size_t free_clamp              = (free_vram > safety_margin) ? (free_vram - safety_margin) : 0;
+                free_clamp                     = (free_vram > safety_margin) ? (free_vram - safety_margin) : 0;
                 if (free_clamp < effective_budget) {
                     LOG_DEBUG("%s clamping streaming budget: actual free VRAM %.2f MB < user cap %.2f MB",
                               get_desc().c_str(),
@@ -2431,7 +2480,9 @@ protected:
                 observed_max_effective_budget_ = effective_budget;
                 budget_increased               = true;
             } else {
-                effective_budget = observed_max_effective_budget_;
+                // Keep the plan cache stable, but never plan above what is free now:
+                // another model or process can take VRAM after the first measurement.
+                effective_budget = std::min(observed_max_effective_budget_, free_clamp);
             }
         }
 
@@ -3030,7 +3081,8 @@ public:
         }
         if (sched != nullptr) {
             ggml_backend_sched_free(sched);
-            sched = nullptr;
+            sched                = nullptr;
+            sched_graph_capacity = 0;
         }
     }
 
@@ -3356,14 +3408,18 @@ protected:
     bool force_f32;
     bool force_prec_f32;
     bool allow_weight_scale;
-    bool has_weight_scale = false;
+    bool has_weight_scale       = false;
+    bool int8_convrot           = false;
+    int int8_convrot_group_size = 0;
     float scale;
     std::string prefix;
 
     void init_params(ggml_context* ctx, const String2TensorStorage& tensor_storage_map = {}, const std::string prefix = "") override {
-        this->prefix         = prefix;
-        has_weight_scale     = false;
-        enum ggml_type wtype = get_type(prefix + "weight", tensor_storage_map, GGML_TYPE_F32);
+        this->prefix            = prefix;
+        has_weight_scale        = false;
+        int8_convrot            = false;
+        int8_convrot_group_size = 0;
+        enum ggml_type wtype    = get_type(prefix + "weight", tensor_storage_map, GGML_TYPE_F32);
         if (in_features % ggml_blck_size(wtype) != 0 || force_f32) {
             wtype = GGML_TYPE_F32;
         }
@@ -3372,10 +3428,18 @@ protected:
             enum ggml_type wtype = GGML_TYPE_F32;
             params["bias"]       = ggml_new_tensor_1d(ctx, wtype, out_features);
         }
-        if (allow_weight_scale && tensor_storage_map.find(prefix + "weight_scale") != tensor_storage_map.end()) {
+        auto weight_storage           = tensor_storage_map.find(prefix + "weight");
+        const bool is_int8_tensorwise = weight_storage != tensor_storage_map.end() && weight_storage->second.is_int8_tensorwise;
+        if ((allow_weight_scale || is_int8_tensorwise) && tensor_storage_map.find(prefix + "weight_scale") != tensor_storage_map.end()) {
             params["weight_scale"] = ggml_new_tensor_1d(ctx, GGML_TYPE_F32, out_features);
             has_weight_scale       = true;
         }
+        if (is_int8_tensorwise) {
+            GGML_ASSERT(wtype == GGML_TYPE_I8);
+            GGML_ASSERT(has_weight_scale);
+            int8_convrot            = weight_storage->second.int8_convrot;
+            int8_convrot_group_size = weight_storage->second.int8_convrot_group_size;
+        }
     }
 
 public:
@@ -3410,6 +3474,49 @@ public:
         }
         ggml_tensor* linear_bias = has_weight_scale ? nullptr : b;
         ggml_tensor* out         = nullptr;
+        if (w->type == GGML_TYPE_I8) {
+            if (x->type != GGML_TYPE_F32) {
+                x = ggml_ext_cast_f32(ctx->ggml_ctx, ctx->backend, x);
+            }
+            if (!ggml_is_contiguous(x)) {
+                x = ggml_cont(ctx->ggml_ctx, x);
+            }
+            ggml_tensor* lora_input = x;
+            if (ctx->weight_adapter && b != nullptr) {
+                b = ctx->weight_adapter->patch_weight(ctx->ggml_ctx, ctx->backend, b, prefix + "bias");
+            }
+            if (int8_convrot && scale == 1.f) {
+                const auto cache_key = std::make_pair(x, int8_convrot_group_size);
+                auto cached          = ctx->int8_convrot_cache.find(cache_key);
+                if (cached == ctx->int8_convrot_cache.end()) {
+                    x = ggml_quantize_i8_convrot(ctx->ggml_ctx, x, int8_convrot_group_size);
+                    ctx->int8_convrot_cache.emplace(cache_key, x);
+                } else {
+                    x = cached->second;
+                }
+            }
+            out = ggml_ext_linear_i8_tensorwise(ctx->ggml_ctx,
+                                                x,
+                                                w,
+                                                params["weight_scale"],
+                                                b,
+                                                int8_convrot ? int8_convrot_group_size : 0,
+                                                scale);
+            if (ctx->weight_adapter) {
+                WeightAdapter::ForwardParams forward_params;
+                forward_params.op_type               = WeightAdapter::ForwardParams::op_type_t::OP_LINEAR;
+                forward_params.linear.force_prec_f32 = force_prec_f32;
+                forward_params.linear.scale          = scale;
+                out                                  = ctx->weight_adapter->add_lora_to_output(ctx->ggml_ctx,
+                                                                                               ctx->backend,
+                                                                                               lora_input,
+                                                                                               w,
+                                                                                               out,
+                                                                                               prefix,
+                                                                                               forward_params);
+            }
+            return out;
+        }
         if (ctx->weight_adapter) {
             WeightAdapter::ForwardParams forward_params;
             forward_params.op_type               = WeightAdapter::ForwardParams::op_type_t::OP_LINEAR;

src/core/ggml_graph_cut.cpp

@@ -453,11 +453,12 @@ namespace sd::ggml_graph_cut {
         if (tensor == nullptr || tensor->name[0] == '\0') {
             return false;
         }
-        return std::strncmp(tensor->name, GGML_RUNNER_CUT_PREFIX, std::strlen(GGML_RUNNER_CUT_PREFIX)) == 0;
+        return starts_with(tensor->name, GGML_RUNNER_CUT_PREFIX) &&
+               ends_with(tensor->name, GGML_RUNNER_CUT_SUFFIX);
     }
 
     std::string make_graph_cut_name(const std::string& group, const std::string& output) {
-        return std::string(GGML_RUNNER_CUT_PREFIX) + group + "|" + output;
+        return std::string(GGML_RUNNER_CUT_PREFIX) + group + "|" + output + GGML_RUNNER_CUT_SUFFIX;
     }
 
     void mark_graph_cut(ggml_tensor* tensor, const std::string& group, const std::string& output) {
@@ -603,7 +604,43 @@ namespace sd::ggml_graph_cut {
         GGML_ASSERT(gf != nullptr);
         GGML_ASSERT(graph_ctx_out != nullptr);
 
-        const size_t graph_size = segment.internal_node_indices.size() + segment.input_refs.size() + 8;
+        // Collect leaf inputs and internal nodes, then any tensor they
+        // reference that is not already represented, notably the view_src of a
+        // view-typed input leaf. ggml_gallocr sizes its hash set from
+        // n_nodes + n_leafs (plus a 25% margin that rounds down to zero for a
+        // one-node segment), so every distinct tensor it will hash must be
+        // counted here or a tiny segment overflows the hash set and aborts.
+        std::vector<ggml_tensor*> leaves;
+        std::unordered_set<ggml_tensor*> represented;
+        for (const auto& input : segment.input_refs) {
+            ggml_tensor* current_input = input_tensor(gf, input);
+            if (current_input == nullptr) {
+                continue;
+            }
+            if (represented.insert(current_input).second) {
+                leaves.push_back(current_input);
+            }
+        }
+        for (int node_idx : segment.internal_node_indices) {
+            represented.insert(ggml_graph_node(gf, node_idx));
+        }
+        auto add_reference = [&](ggml_tensor* tensor) {
+            if (tensor != nullptr && represented.insert(tensor).second) {
+                leaves.push_back(tensor);
+            }
+        };
+        for (int node_idx : segment.internal_node_indices) {
+            ggml_tensor* node = ggml_graph_node(gf, node_idx);
+            for (int src_idx = 0; src_idx < GGML_MAX_SRC; ++src_idx) {
+                add_reference(node->src[src_idx]);
+            }
+            add_reference(node->view_src);
+        }
+        for (size_t i = 0; i < leaves.size(); ++i) {
+            add_reference(leaves[i]->view_src);
+        }
+
+        const size_t graph_size = segment.internal_node_indices.size() + leaves.size() + 8;
         ggml_init_params params = {
             /*.mem_size   =*/ggml_graph_overhead_custom(graph_size, false) + 1024,
             /*.mem_buffer =*/nullptr,
@@ -614,13 +651,9 @@ namespace sd::ggml_graph_cut {
         ggml_cgraph* segment_graph = ggml_new_graph_custom(graph_ctx, graph_size, false);
         GGML_ASSERT(segment_graph != nullptr);
 
-        for (const auto& input : segment.input_refs) {
-            ggml_tensor* current_input = input_tensor(gf, input);
-            if (current_input == nullptr) {
-                continue;
-            }
+        for (ggml_tensor* leaf : leaves) {
             GGML_ASSERT(segment_graph->n_leafs < segment_graph->size);
-            segment_graph->leafs[segment_graph->n_leafs++] = current_input;
+            segment_graph->leafs[segment_graph->n_leafs++] = leaf;
         }
 
         for (int output_node_index : segment.output_node_indices) {
@@ -751,7 +784,9 @@ namespace sd::ggml_graph_cut {
 
             plan.has_cuts = true;
             std::string full_name(node->name);
-            std::string payload = full_name.substr(std::strlen(GGML_RUNNER_CUT_PREFIX));
+            size_t prefix_len   = std::strlen(GGML_RUNNER_CUT_PREFIX);
+            size_t suffix_len   = std::strlen(GGML_RUNNER_CUT_SUFFIX);
+            std::string payload = full_name.substr(prefix_len, full_name.size() - prefix_len - suffix_len);
             size_t sep          = payload.find('|');
             std::string group   = sep == std::string::npos ? payload : payload.substr(0, sep);
 

src/core/ggml_graph_cut.h

@@ -68,6 +68,7 @@ namespace sd::ggml_graph_cut {
     };
 
     static constexpr const char* GGML_RUNNER_CUT_PREFIX = "ggml_runner_cut:";
+    static constexpr const char* GGML_RUNNER_CUT_SUFFIX = "|";
 
     struct MaxVramAssignment {
         float default_gib = 0.f;

src/model/adapter/lora.hpp

@@ -1072,6 +1072,34 @@ public:
         return out;
     }
 
+    ggml_tensor* add_lora_to_output(ggml_context* ctx,
+                                    ggml_backend_t backend,
+                                    ggml_tensor* x,
+                                    ggml_tensor* w,
+                                    ggml_tensor* output,
+                                    const std::string& prefix,
+                                    WeightAdapter::ForwardParams forward_params) override {
+        for (auto& lora_model : lora_models) {
+            ggml_tensor* weight_diff = lora_model->get_weight_diff(prefix + "weight", backend, ctx, w, false);
+            if (weight_diff != nullptr) {
+                GGML_ASSERT(forward_params.op_type == ForwardParams::op_type_t::OP_LINEAR);
+                ggml_tensor* out_diff = ggml_ext_linear(ctx,
+                                                        x,
+                                                        weight_diff,
+                                                        nullptr,
+                                                        forward_params.linear.force_prec_f32,
+                                                        forward_params.linear.scale);
+                output                = ggml_add_inplace(ctx, output, out_diff);
+            }
+
+            ggml_tensor* out_diff = lora_model->get_out_diff(ctx, backend, x, w, forward_params, prefix + "weight");
+            if (out_diff != nullptr) {
+                output = ggml_add_inplace(ctx, output, out_diff);
+            }
+        }
+        return output;
+    }
+
     size_t get_extra_graph_size() override {
         size_t lora_tensor_num = 0;
         for (auto& lora_model : lora_models) {

src/model/vae/tae.hpp

@@ -426,10 +426,10 @@ class TinyVideoDecoder : public UnaryBlock {
     static const int num_layers  = 3;
     int channels[num_layers + 1] = {256, 128, 64, 64};
     int patch_size               = 1;
-    int t_upscale                = 1;
     bool is_wide                 = false;
 
 public:
+    int t_upscale = 1;
     TinyVideoDecoder(int z_channels = 4, int patch_size = 1, std::vector<bool> time_upscale = {false, true, true}, bool is_wide = false)
         : z_channels(z_channels), patch_size(patch_size), is_wide(is_wide) {
         t_upscale = 1;
@@ -536,6 +536,10 @@ public:
             patch          = 4;
             time_downscale = {true, true, true};
             time_upscale   = {true, true, true};
+        } else if (sd_version_is_minimax_h3(version)) {
+            z_channels     = 24;
+            patch          = 2;
+            time_downscale = {true, true, false};
         }
         blocks["decoder"] = std::shared_ptr<GGMLBlock>(new TinyVideoDecoder(z_channels, patch, time_upscale, is_wide));
         if (!decode_only) {
@@ -545,24 +549,123 @@ public:
 
     ggml_tensor* decode(GGMLRunnerContext* ctx, ggml_tensor* z) {
         auto decoder = std::dynamic_pointer_cast<TinyVideoDecoder>(blocks["decoder"]);
-        if (sd_version_is_wan(version) || sd_version_is_hunyuan_video(version) || sd_version_is_ltxav(version)) {
+        if (sd_version_is_wan(version) || sd_version_is_hunyuan_video(version) || sd_version_is_ltxav(version) || sd_version_is_minimax_h3(version)) {
             // (W, H, C, T) -> (W, H, T, C)
             z = ggml_cont(ctx->ggml_ctx, ggml_permute(ctx->ggml_ctx, z, 0, 1, 3, 2));
         }
         auto result = decoder->forward(ctx, z);
-        if (sd_version_is_wan(version) || sd_version_is_hunyuan_video(version) || sd_version_is_ltxav(version)) {
+
+        if (sd_version_is_minimax_h3(version)) {
+            int64_t num_frames   = result->ne[3];
+            int64_t chunk_frames = 5 * decoder->t_upscale;
+            int64_t pad          = (chunk_frames - (num_frames % chunk_frames)) % chunk_frames;
+
+            result = ggml_ext_pad_ext(ctx->ggml_ctx, ctx->backend, result, 0, 0, 0, 0, 0, 0, 0, pad, false, false);
+
+            int64_t num_chunks                  = (num_frames + pad) / chunk_frames;
+            auto to_trim                        = decoder->t_upscale - 1;
+            std::vector<ggml_tensor*> to_concat = {};
+            for (int i = 0; i < num_chunks; i++) {
+                auto chunk = ggml_view_4d(ctx->ggml_ctx, result,
+                                          result->ne[0], result->ne[1], result->ne[2], chunk_frames - to_trim,
+                                          result->nb[1], result->nb[2], result->nb[3],
+                                          i * chunk_frames * result->nb[3]);
+                to_concat.push_back(chunk);
+            }
+            result = ggml_ext_vec_concat(ctx->ggml_ctx, to_concat, 3);
+            result = ggml_view_4d(ctx->ggml_ctx, result,
+                                  result->ne[0], result->ne[1], result->ne[2],
+                                  result->ne[3] - decoder->t_upscale * 3,
+                                  result->nb[1], result->nb[2], result->nb[3], 0);
+        }
+
+        if (sd_version_is_wan(version) || sd_version_is_hunyuan_video(version) || sd_version_is_ltxav(version) || sd_version_is_minimax_h3(version)) {
             // (W, H, T, C) -> (W, H, C, T)
             result = ggml_cont(ctx->ggml_ctx, ggml_permute(ctx->ggml_ctx, result, 0, 1, 3, 2));
         }
         return result;
     }
 
-    ggml_tensor* encode(GGMLRunnerContext* ctx, ggml_tensor* x) {
+    ggml_tensor* encode_h3(GGMLRunnerContext* ctx, ggml_tensor* x) {
         auto encoder = std::dynamic_pointer_cast<TinyVideoEncoder>(blocks["encoder"]);
-        if (sd_version_is_wan(version) || sd_version_is_hunyuan_video(version) || sd_version_is_ltxav(version)) {
+
+        int64_t num_frames = x->ne[3];
+        int64_t pad        = (17 - (num_frames % 17)) % 17;
+
+        if (pad > 0) {
+            auto last_frame = ggml_view_4d(ctx->ggml_ctx, x,
+                                           x->ne[0], x->ne[1], x->ne[2], 1,
+                                           x->nb[1], x->nb[2], x->nb[3],
+                                           (num_frames - 1) * x->nb[3]);
+            for (int i = 0; i < pad; i++) {
+                x = ggml_concat(ctx->ggml_ctx, x, last_frame, 3);
+            }
+        }
+
+        int64_t T_padded   = x->ne[3];
+        int64_t num_chunks = T_padded / 17;
+
+        auto zero_frame = ggml_view_4d(ctx->ggml_ctx, x,
+                                       x->ne[0], x->ne[1], x->ne[2], 1,
+                                       x->nb[1], x->nb[2], x->nb[3], 0);
+        auto zeros_1    = ggml_scale(ctx->ggml_ctx, ggml_cont(ctx->ggml_ctx, zero_frame), 0.0f);
+        auto zeros_3    = zeros_1;
+        for (int i = 1; i < 3; i++) {
+            zeros_3 = ggml_concat(ctx->ggml_ctx, zeros_3, zeros_1, 3);
+        }
+        ggml_tensor* out = nullptr;
+        if (false) {
+            std::vector<ggml_tensor*> to_concat = {};
+            for (int i = 0; i < num_chunks; i++) {
+                auto chunk = ggml_view_4d(ctx->ggml_ctx, x,
+                                          x->ne[0], x->ne[1], x->ne[2], 17,
+                                          x->nb[1], x->nb[2], x->nb[3],
+                                          i * 17 * x->nb[3]);
+
+                auto chunk_padded = ggml_concat(ctx->ggml_ctx, zeros_3, chunk, 3);
+
+                to_concat.push_back(chunk_padded);
+            }
+            ggml_tensor* x_in = ggml_ext_vec_concat(ctx->ggml_ctx, to_concat, 3);
+            out               = encoder->forward(ctx, x_in);
+        } else {
+            std::vector<ggml_tensor*> to_concat = {};
+            for (int i = 0; i < num_chunks; i++) {
+                auto chunk = ggml_view_4d(ctx->ggml_ctx, x,
+                                          x->ne[0], x->ne[1], x->ne[2], 17,
+                                          x->nb[1], x->nb[2], x->nb[3],
+                                          i * 17 * x->nb[3]);
+
+                auto chunk_padded = ggml_concat(ctx->ggml_ctx, zeros_3, chunk, 3);
+
+                auto chunk_out = encoder->forward(ctx, chunk_padded);
+                // auto chunk_out = encoder->forward_seq(ctx, chunk_padded); // ~same vram usage, and straight-up slower. it's already sequential enough
+
+                to_concat.push_back(chunk_out);
+            }
+            out = ggml_ext_vec_concat(ctx->ggml_ctx, to_concat, 3);
+        }
+
+        // Return x[:, :-3] - drop the last 3 elements in the T dimension
+        int64_t out_T = out->ne[3];
+        out           = ggml_view_4d(ctx->ggml_ctx, out,
+                                     out->ne[0], out->ne[1], out->ne[2], out_T - 3,
+                                     out->nb[1], out->nb[2], out->nb[3], 0);
+
+        return ggml_cont(ctx->ggml_ctx, ggml_permute(ctx->ggml_ctx, out, 0, 1, 3, 2));
+    }
+
+    ggml_tensor* encode(GGMLRunnerContext* ctx, ggml_tensor* x) {
+        if (sd_version_is_wan(version) || sd_version_is_hunyuan_video(version) || sd_version_is_ltxav(version) || (sd_version_is_minimax_h3(version) && x->ne[3] > 1)) {
             // (W, H, T, C) -> (W, H, C, T)
             x = ggml_cont(ctx->ggml_ctx, ggml_permute(ctx->ggml_ctx, x, 0, 1, 3, 2));
         }
+        if (sd_version_is_minimax_h3(version)) {
+            return encode_h3(ctx, x);
+        }
+
+        auto encoder = std::dynamic_pointer_cast<TinyVideoEncoder>(blocks["encoder"]);
+
         int64_t num_frames = x->ne[3];
         if (num_frames % encoder->t_downscale) {
             // pad to multiple of encoder->t_downscale at the end

src/model/vae/vae.hpp

@@ -156,7 +156,7 @@ public:
             output = _compute(n_threads, input, false);
         }
 
-        free_compute_buffer();
+        runner_done();
 
         if (output.empty()) {
             LOG_ERROR("vae encode compute failed");
@@ -207,7 +207,7 @@ public:
             output = _compute(n_threads, input, true);
         }
 
-        free_compute_buffer();
+        runner_done();
 
         if (output.empty()) {
             LOG_ERROR("vae decode compute failed");

src/model_io/safetensors_io.cpp

@@ -7,6 +7,7 @@
 #include <fstream>
 #include <ostream>
 #include <string>
+#include <unordered_map>
 #include <unordered_set>
 #include <vector>
 
@@ -93,10 +94,52 @@ static ggml_type safetensors_dtype_to_ggml_type(const std::string& dtype) {
         ttype = GGML_TYPE_I32;
     } else if (dtype == "I64") {
         ttype = GGML_TYPE_I32;
+    } else if (dtype == "I8") {
+        ttype = GGML_TYPE_I8;
     }
     return ttype;
 }
 
+struct ComfyQuantConfig {
+    std::string format;
+    bool convrot   = false;
+    int group_size = 0;
+};
+
+static bool read_comfy_quant_config(std::ifstream& file,
+                                    const std::string& file_path,
+                                    const std::string& tensor_name,
+                                    size_t offset,
+                                    size_t size,
+                                    ComfyQuantConfig& config,
+                                    std::string* error) {
+    static constexpr size_t MAX_COMFY_QUANT_CONFIG_SIZE = 64 * 1024;
+    if (size == 0 || size > MAX_COMFY_QUANT_CONFIG_SIZE) {
+        set_error(error, "invalid ComfyUI quantization metadata tensor '" + tensor_name + "' in '" + file_path + "'");
+        return false;
+    }
+
+    std::vector<char> data(size + 1, '\0');
+    file.clear();
+    file.seekg((std::streamoff)offset, std::ios::beg);
+    file.read(data.data(), (std::streamsize)size);
+    if (!file) {
+        set_error(error, "read ComfyUI quantization metadata tensor failed: '" + tensor_name + "'");
+        return false;
+    }
+
+    try {
+        const nlohmann::json json = nlohmann::json::parse(data.data(), data.data() + size);
+        config.format             = json.value("format", "");
+        config.convrot            = json.value("convrot", false);
+        config.group_size         = json.value("convrot_groupsize", 0);
+    } catch (const std::exception&) {
+        set_error(error, "parsing ComfyUI quantization metadata tensor failed: '" + tensor_name + "'");
+        return false;
+    }
+    return true;
+}
+
 // https://huggingface.co/docs/safetensors/index
 bool read_safetensors_file(const std::string& file_path,
                            std::vector<TensorStorage>& tensor_storages,
@@ -163,6 +206,33 @@ bool read_safetensors_file(const std::string& file_path,
         }
     }
 
+    std::unordered_map<std::string, ComfyQuantConfig> comfy_quant_configs;
+    for (const auto& item : header_.items()) {
+        const std::string& name = item.key();
+        if (name == "__metadata__" || !ends_with(name, ".comfy_quant")) {
+            continue;
+        }
+
+        const nlohmann::json& tensor_info = item.value();
+        if (tensor_info.value("dtype", "") != "U8") {
+            set_error(error, "invalid dtype for ComfyUI quantization metadata tensor '" + name + "'");
+            return false;
+        }
+        const size_t begin = tensor_info["data_offsets"][0].get<size_t>();
+        const size_t end   = tensor_info["data_offsets"][1].get<size_t>();
+        if (begin > end || end > file_size_ - data_start) {
+            set_error(error, "data offsets out of bounds for tensor '" + name + "'");
+            return false;
+        }
+
+        ComfyQuantConfig config;
+        if (!read_comfy_quant_config(file, file_path, name, data_start + begin, end - begin, config, error)) {
+            return false;
+        }
+        const std::string module_name = name.substr(0, name.size() - std::string(".comfy_quant").size());
+        comfy_quant_configs.emplace(module_name, std::move(config));
+    }
+
     tensor_storages.clear();
     for (auto& item : header_.items()) {
         std::string name           = item.key();
@@ -220,6 +290,39 @@ bool read_safetensors_file(const std::string& file_path,
         TensorStorage tensor_storage(name, type, ne, n_dims, 0, data_start + begin);
         tensor_storage.reverse_ne();
 
+        if (ends_with(name, ".weight")) {
+            const std::string module_name = name.substr(0, name.size() - std::string(".weight").size());
+            auto config                   = comfy_quant_configs.find(module_name);
+            if (config != comfy_quant_configs.end() && config->second.format == "int8_tensorwise") {
+                if (type != GGML_TYPE_I8) {
+                    set_error(error, "ComfyUI int8_tensorwise weight is not I8: '" + name + "'");
+                    return false;
+                }
+                if (config->second.convrot) {
+                    int group_size_remainder = config->second.group_size;
+                    while (group_size_remainder > 1 && group_size_remainder % 4 == 0) {
+                        group_size_remainder /= 4;
+                    }
+                    if (group_size_remainder != 1 || tensor_storage.ne[0] % config->second.group_size != 0) {
+                        set_error(error, "invalid ComfyUI convrot group size for tensor '" + name + "'");
+                        return false;
+                    }
+                }
+                tensor_storage.is_int8_tensorwise      = true;
+                tensor_storage.int8_convrot            = config->second.convrot;
+                tensor_storage.int8_convrot_group_size = config->second.group_size;
+            }
+        } else if (ends_with(name, ".weight_scale")) {
+            const std::string module_name = name.substr(0, name.size() - std::string(".weight_scale").size());
+            auto config                   = comfy_quant_configs.find(module_name);
+            if (config != comfy_quant_configs.end() && config->second.format == "int8_tensorwise" &&
+                tensor_storage.n_dims == 2 && tensor_storage.ne[0] == 1) {
+                tensor_storage.ne[0]  = tensor_storage.ne[1];
+                tensor_storage.ne[1]  = 1;
+                tensor_storage.n_dims = 1;
+            }
+        }
+
         size_t tensor_data_size = end - begin;
 
         bool tensor_size_ok;

src/model_io/tensor_storage.h

@@ -15,14 +15,17 @@
 
 struct TensorStorage {
     std::string name;
-    ggml_type type          = GGML_TYPE_F32;
-    ggml_type expected_type = GGML_TYPE_COUNT;
-    bool is_f8_e4m3         = false;
-    bool is_f8_e5m2         = false;
-    bool is_f64             = false;
-    bool is_i64             = false;
-    int64_t ne[SD_MAX_DIMS] = {1, 1, 1, 1, 1};
-    int n_dims              = 0;
+    ggml_type type              = GGML_TYPE_F32;
+    ggml_type expected_type     = GGML_TYPE_COUNT;
+    bool is_f8_e4m3             = false;
+    bool is_f8_e5m2             = false;
+    bool is_f64                 = false;
+    bool is_i64                 = false;
+    bool is_int8_tensorwise     = false;
+    bool int8_convrot           = false;
+    int int8_convrot_group_size = 0;
+    int64_t ne[SD_MAX_DIMS]     = {1, 1, 1, 1, 1};
+    int n_dims                  = 0;
 
     std::string storage_key;
     size_t file_index = 0;

src/model_loader.cpp

@@ -1508,6 +1508,9 @@ bool ModelLoader::load_tensors(std::map<std::string, ggml_tensor*>& tensors,
 
 bool ModelLoader::tensor_should_be_converted(const TensorStorage& tensor_storage, ggml_type type) {
     const std::string& name = tensor_storage.name;
+    if (tensor_storage.is_int8_tensorwise) {
+        return false;
+    }
     if (type != GGML_TYPE_COUNT) {
         if (ggml_is_quantized(type) && tensor_storage.ne[0] % ggml_blck_size(type) != 0) {
             // Pass, do not convert

src/runtime/denoiser.hpp

@@ -2582,27 +2582,49 @@ static sd::Tensor<float> sample_lms(denoise_cb_t model,
                                     sd::Tensor<float> x,
                                     const std::vector<float>& sigmas,
                                     const SamplerExtraArgs& extra_sample_args) {
-    // Linear Multi-Step from https://github.com/crowsonkb/k-diffusion
-
+    // Linear Multi-Step from https://github.com/crowsonkb/k-diffusion,
+    // modified with "history shift" value, which seemingly needs less steps
     int divisions = 1000;
+    int max_order = 4;
+    int shift     = 1;  // 4, 0 - original; 4, 1 - PR #1843; 3, 1 - smoother image
     for (const auto& [key, value] : extra_sample_args) {
         int parsed = 0;
+        if (key == "lms_max_order") {
+            if (!parse_strict_int(value, parsed)) {
+                LOG_WARN("ignoring invalid lms extra sample arg '%s=%s'", key.c_str(), value.c_str());
+                continue;
+            }
+            max_order = std::max(1, parsed);
+            // smaller values make the result softer, closer to Euler
+            // higher values need more steps
+            // values above 12 can produce NaNs, depending on steps and scheduler
+        }
+        if (key == "lms_shift") {
+            if (!parse_strict_int(value, parsed)) {
+                LOG_WARN("ignoring invalid lms extra sample arg '%s=%s'", key.c_str(), value.c_str());
+                continue;
+            }
+            shift = std::max(0, parsed);
+            // for a low number of steps, the value 1 works best
+        }
         if (key == "lms_divisions") {
             if (!parse_strict_int(value, parsed)) {
                 LOG_WARN("ignoring invalid lms extra sample arg '%s=%s'", key.c_str(), value.c_str());
                 continue;
             }
             divisions = parsed;  // std::max(1, parsed);
-            // values above 35M produce noise, can be fixed by double precision
             // values < 1 always produce noise
+            // values above 30M require double precision in the integrator
+            // (they are needless and just slow the integration down, but
+            //  with single precision they softly produce noise
+            //  near the 35M, it can be used for distorted generations)
         }
     }
-    LOG_DEBUG("linear multi-step sampler: integrating using %i division%s", divisions, (divisions == 1) ? "" : "s");
 
     auto linear_multistep_coeff = [=](const int order, const int m, const int j) -> float {
         if (!divisions)
             return sigmas[m + 1] - sigmas[m];  // delta / 0 * 0
-#define LMS_PRECISION float                    // double
+#define LMS_PRECISION float                    // when divisions > 30 millions, the double precision fixes noise
         const LMS_PRECISION a = sigmas[m], dx = (sigmas[m + 1] - a) / divisions, s = sigmas[m - j];
         const LMS_PRECISION b0 = a + 0.5f * dx;  // using Riemann middle integral
         LMS_PRECISION sum      = 0.0f;
@@ -2622,11 +2644,12 @@ static sd::Tensor<float> sample_lms(denoise_cb_t model,
         return sum * dx;
     };
 
-    const int max_order = 4;
-    float lms_coeff[max_order];
+    int steps = static_cast<int>(sigmas.size()) - 1;
+    max_order = std::min(max_order, steps);  // history can not be larger than steps
+    LOG_DEBUG("linear multi-step sampler: lms_max_order = %i, lms_shift = %i, lms_divisions = %i", max_order, shift, divisions);
+    std::vector<float> lms_coeff(max_order);
     std::vector<sd::Tensor<float>> hist = {};
 
-    int steps = static_cast<int>(sigmas.size()) - 1;
     for (int i = 0; i < steps; i++) {
         const float sigma = sigmas[i];
 
@@ -2637,25 +2660,26 @@ static sd::Tensor<float> sample_lms(denoise_cb_t model,
         sd::Tensor<float> denoised = std::move(denoised_opt.pred);
 
         const int order = std::min(max_order, i + 1);
+
         for (int c = 0; c < order; c++)  // computing coefficients
             lms_coeff[c] = linear_multistep_coeff(order, i, c);
 
         sd::Tensor<float> d_cur = (x - denoised) / sigma;
-        switch (order) {
-            case 4:  // derivative + 3 history points
-                x += hist[hist.size() - 2] * lms_coeff[3];
-            case 3:
-                x += hist[hist.size() - 1] * lms_coeff[2];
-            case 2:
-                x += hist.back() * lms_coeff[1];
-            case 1:
-                x += d_cur * lms_coeff[0];
+        x += d_cur * lms_coeff[0];
+        if (max_order > 1) {  // if max_order == 1, the history is not used (order always < 2)
+            int hist_size_p1 = hist.size() + 1;
+            if (i) {  // history does not exist at 1st step
+                int hist_max = hist.size() - 1;
+                for (int c = 2; c <= order; c++)
+                    x += hist[std::min(hist_max, hist_size_p1 - c + shift)] * lms_coeff[c - 1];
+                // max_order == 4  =>  hist[] index = 2, 1, 0
+                // shift == 1      =>  hist[] index = 2, 2, 1
+            }
+            if (hist_size_p1 == max_order) {
+                hist.erase(hist.begin());
+            }
+            hist.push_back(std::move(d_cur));
         }
-
-        if (hist.size() == static_cast<size_t>(max_order - 1)) {
-            hist.erase(hist.begin());
-        }
-        hist.push_back(std::move(d_cur));
     }
     return x;
 }

src/stable-diffusion.cpp

@@ -150,6 +150,9 @@ const char* sampling_methods_str[] = {
     "LMS",
 };
 
+static_assert(SAMPLE_METHOD_COUNT == sizeof(sampling_methods_str) / sizeof(sampling_methods_str[0]),
+              "\nnumber of elements in sampling_methods_str[] != SAMPLE_METHOD_COUNT");
+
 /*================================================== Helper Functions ================================================*/
 
 static bool sd_version_supports_ref_latent_img_cfg(SDVersion version) {
@@ -960,8 +963,16 @@ public:
 
         LOG_DEBUG("ggml tensor size = %d bytes", (int)sizeof(ggml_tensor));
 
+        bool have_int8_tensorwise = false;
+        for (const auto& [_, tensor_storage] : model_loader.get_tensor_storage_map()) {
+            if (tensor_storage.is_int8_tensorwise) {
+                have_int8_tensorwise = true;
+                break;
+            }
+        }
+
         if (sd_ctx_params->lora_apply_mode == LORA_APPLY_AUTO) {
-            bool have_quantized_weight = false;
+            bool have_quantized_weight = have_int8_tensorwise;
             for (const auto& [type, _] : wtype_stat) {
                 if (ggml_is_quantized(type)) {
                     have_quantized_weight = true;
@@ -977,12 +988,19 @@ public:
                 apply_lora_immediately = true;
             }
         } else if (sd_ctx_params->lora_apply_mode == LORA_APPLY_IMMEDIATELY) {
-            if (row_split_active()) {
+            if (have_int8_tensorwise) {
+                LOG_WARN(
+                    "INT8 tensorwise weights do not support the immediately LoRA apply mode; "
+                    "using at_runtime instead");
+                apply_lora_immediately = false;
+            } else if (row_split_active()) {
                 LOG_WARN(
                     "row-split tensors do not support the immediately LoRA apply mode; "
                     "LoRAs will not be applied to them (use --lora-apply-mode at_runtime)");
+                apply_lora_immediately = false;
+            } else {
+                apply_lora_immediately = true;
             }
-            apply_lora_immediately = true;
         } else {
             apply_lora_immediately = false;
         }
@@ -1008,11 +1026,11 @@ public:
             tae_preview_only = false;
             use_tae          = true;
         }
-        if (sd_version_is_minimax_h3(version) && use_tae) {
-            LOG_WARN("MiniMax-H3 does not have a compatible TAE; ignoring --taesd");
-            tae_preview_only = false;
-            use_tae          = false;
-        }
+        // if (sd_version_is_minimax_h3(version) && use_tae) {
+        //     LOG_WARN("MiniMax-H3 does not have a compatible TAE; ignoring --taesd");
+        //     tae_preview_only = false;
+        //     use_tae          = false;
+        // }
 
         auto& tensor_storage_map = model_loader.get_tensor_storage_map();
 
@@ -1387,7 +1405,7 @@ public:
             }
 
             auto create_tae = [&](bool decode_only) -> std::shared_ptr<VAE> {
-                if (sd_version_uses_wan_vae(version) || sd_version_is_hunyuan_video(version) || sd_version_is_ltxav(version)) {
+                if (sd_version_uses_wan_vae(version) || sd_version_is_hunyuan_video(version) || sd_version_is_ltxav(version) || sd_version_is_minimax_h3(version)) {
                     return std::make_shared<TinyVideoAutoEncoder>(backend_for(SDBackendModule::VAE),
                                                                   tensor_storage_map,
                                                                   "decoder",
@@ -2297,7 +2315,7 @@ public:
                     return;
                 }
             } else if (channels == 24) {
-                if(sd_version_is_minimax_h3(version)){
+                if (sd_version_is_minimax_h3(version)) {
                     latent_rgb_proj = minimax_latent_rgb_proj;
                     latent_rgb_bias = minimax_latent_rgb_bias;
                 } else {
@@ -2652,7 +2670,9 @@ public:
                     denoised = denoised * denoise_mask + init_latent * (1.0f - denoise_mask);
                 }
                 if (sd_should_preview_denoised() && preview.callback != nullptr) {
-                    preview_image(step, denoised, version, preview.mode, preview.callback, preview.data, false);
+                    if (step % sd_get_preview_interval() == 0) {
+                        preview_image(step, denoised, version, preview.mode, preview.callback, preview.data, false);
+                    }
                 }
                 report_sample_progress(step, steps, &last_progress_us);
                 sd::guidance::GuiderOutput output;
@@ -2661,7 +2681,9 @@ public:
             }
 
             if (sd_should_preview_noisy() && preview.callback != nullptr) {
-                preview_image(step, noised_input, version, preview.mode, preview.callback, preview.data, true);
+                if (step % sd_get_preview_interval() == 0) {
+                    preview_image(step, noised_input, version, preview.mode, preview.callback, preview.data, true);
+                }
             }
 
             sd::Tensor<float> cond_out;
@@ -2871,7 +2893,9 @@ public:
                 denoised = denoised * denoise_mask + init_latent * (1.0f - denoise_mask);
             }
             if (sd_should_preview_denoised() && preview.callback != nullptr) {
-                preview_image(step, denoised, version, preview.mode, preview.callback, preview.data, false);
+                if (step % sd_get_preview_interval() == 0) {
+                    preview_image(step, denoised, version, preview.mode, preview.callback, preview.data, false);
+                }
             }
             report_sample_progress(step, steps, &last_progress_us);
             output.pred = denoised;
@@ -3291,6 +3315,9 @@ const char* sample_method_to_str[] = {
     "lms",
 };
 
+static_assert(SAMPLE_METHOD_COUNT == sizeof(sample_method_to_str) / sizeof(sample_method_to_str[0]),
+              "\nnumber of elements in sample_method_to_str[] != SAMPLE_METHOD_COUNT");
+
 const char* sd_sample_method_name(enum sample_method_t sample_method) {
     if (sample_method < SAMPLE_METHOD_COUNT) {
         return sample_method_to_str[sample_method];
@@ -3326,6 +3353,9 @@ const char* scheduler_to_str[] = {
     "beta",
 };
 
+static_assert(SCHEDULER_COUNT == sizeof(scheduler_to_str) / sizeof(scheduler_to_str[0]),
+              "\nnumber of elements in scheduler_to_str[] != SCHEDULER_COUNT");
+
 const char* sd_scheduler_name(enum scheduler_t scheduler) {
     if (scheduler < SCHEDULER_COUNT) {
         return scheduler_to_str[scheduler];
@@ -5584,6 +5614,18 @@ SD_API bool generate_image(sd_ctx_t* sd_ctx,
         return false;
     }
 
+    // MiniMax-H3 is video-only. Its denoiser always splits the packed latent into a video and an
+    // audio half, and only generate_video ever computes the audio length, so reaching this
+    // function with an H3 checkpoint is guaranteed to die on
+    // GGML_ASSERT(!audio_input_cache.empty()) with a core dump, after the several minutes it
+    // takes to load the weights, and with nothing in the output pointing at the missing --mode.
+    // (The AnimateDiff path below routes vid_gen back through here, but that is SD1.5 plus a
+    // motion module, never H3.)
+    if (sd_version_is_minimax_h3(sd_ctx->sd->version)) {
+        LOG_ERROR("MiniMax-H3 is a video model and cannot be run in img_gen mode; use --mode vid_gen");
+        return false;
+    }
+
     sd_ctx->sd->reset_cancel_flag();
 
     int64_t t0                    = ggml_time_ms();