Skip to main content

Release Notes 1.4.0

Release date: 2026-07-05

Skulk 1.4.0 brings multi-node GGUF inference: a model that fits no single GPU node can now pool the GPU memory of several nodes and serve as one instance. Two AMD Strix Halo boxes (128 GB + 64 GB) can together serve a quant that neither could load alone, with placement, recovery, and the dashboard all treating the pooled instance as a first-class citizen. The release also adds a plugin API for serving-path extensions, cleaner model-card semantics, and a wave of placement-recovery hardening found by running the full end-to-end battery against a live five-node cluster until it passed clean.

Highlights​

  • Multi-node GGUF inference (GPU memory pooling). A GGUF model that fits no single GPU node but fits the combined GPU memory of several now places as a driver-plus-donors instance: the driver node runs llama-server --rpc donor:port,... and holds the model file, each donor runs a small ggml-rpc-server that lends its GPU memory, and llama.cpp splits the weights and KV across the pooled devices itself. Single-node placement is always preferred when the model fits one node; pooling is a capacity feature, not a speedup (expect roughly 15% slower decode than the same model on one sufficient node, and plan pooled placements for models that have no single-node home). A donor or driver death tears the instance down cleanly and re-places it. Donor traffic prefers a USB4/Thunderbolt link when one exists, which speeds model load noticeably. Requirements: a llama.cpp build with -DGGML_RPC=ON (the bundled AMD install script now builds this) and SKULK_LLAMA_SERVER_BIN set; the donor binary is found next to it or via SKULK_RPC_SERVER_BIN. See the AMD / Strix Halo node guide.

  • Extension (plugin) API. Separately installed Python packages can hook the serving path: transform a chat request before cluster dispatch, observe the completed response, and call the cluster's embedding serving in-process. Extensions are discovered via the skulk.extensions entry-point group, are version-gated against the running Skulk, and are fully guarded: a raising extension is logged and skipped, never degrading inference, and no extension installed means Skulk is unchanged. SKULK_EXTENSIONS_DISABLE=1 is a node-local kill switch. See the Extensions guide for the contract and a complete worked example.

  • Model cards now declare model truth only. A card's compatible_backends records which engines the model's artifacts run on; limitations of Skulk's current runners (for example, the served llama.cpp engine cannot yet load a vision model's projector) are enforced by a code-level capability table instead of card edits. Models never land where an advertised capability would silently degrade, and cards need no sweep when the platform catches up.

  • Linux network interfaces are now typed. Interface classification previously existed only on macOS; Linux NICs all reported "unknown", so the Thunderbolt-first transport preference could never fire between two Linux boxes. Interfaces are now classified from sysfs (thunderbolt, ethernet, wifi), so a USB4 link between two GPU nodes is detected and preferred automatically.

Fixes​

  • Joining a long-lived cluster no longer triggers a gossip storm. Workers used to re-emit connectivity readings every tick, building a huge replay tail that a joining node had to ingest at once, saturating its send queues and flapping the join (worst for a GPU node joining a mostly-Mac fleet). Connectivity now gossips only on change, and node liveness rides telemetry freshness.

  • A failed model download no longer poisons future placements. Recovery from a terminal download failure (for example a full disk) now clears the failure record; previously it lingered and condemned every later placement of that model touching that node until a fleet restart.

  • Placement refusals recover on heterogeneous clusters. When a node refuses its shard and no wider split exists, the master now retries the model single-node excluding the refuser instead of giving up, bounded so the retry chain can never oscillate; a truly doomed placement is torn down with its in-flight downloads cancelled.

  • Cluster listener ports moved out of the OS ephemeral range, eliminating rare bind collisions between instance listeners and short-lived OS connections on both Linux and macOS.

  • A served engine or RPC donor process that dies between requests is now detected and the runner is failed with the subprocess's log tail, instead of the next request hanging against a zombie.

  • Placement previews report the instance shape that would actually launch for each combination, so a dashboard or script can trust the preview's instance_meta verbatim.

  • Every bundled model card audited and corrected. All 136 bundled cards were checked end to end: schema, cross-field consistency, capability resolution, and a live check of every referenced Hugging Face repo and file. The Ornith 1.0-35B MLX card pointed at a repo its author had deleted and now uses the official mlx-community conversion; several Qwen3 instruct-only variants no longer advertise a thinking mode they do not have; and a handful of cards gained their real context lengths and KV-head counts so context admission and memory estimation use exact values. The audit's checks now run as tests, so a bad bundled card fails the build instead of shipping.

Upgrading​

All nodes in a cluster must run the same Skulk version. Upgrade the whole fleet together; mixed-version clusters are unsupported.