PlaceInstanceParams

model_idModel Id (string)required

shardingSharding (string)

Possible values: [Tensor, Pipeline]

Default value: Pipeline

instance_metaInstanceMeta (string)

Possible values: [MlxRing, MlxJaccl]

Default value: MlxRing

min_nodesMin Nodes (integer)

Default value: 1

excluded_nodesstring[]

Optional. Node IDs the master should treat as if absent when scoring candidate cycles for this placement. Empty list = consider all nodes. Already-running instances on the listed nodes are not affected — exclusion is per-placement, not cluster-wide.

PlaceInstanceParams

{
  "excluded_nodes": [],
  "instance_meta": "MlxRing",
  "min_nodes": 1,
  "model_id": "mlx-community/Llama-3.2-1B-Instruct-4bit",
  "sharding": "Pipeline"
}