Skip to main content

PlacementIntent

One ordinary placement as the operator asked for it, free of live IDs.

modelIdModelid (string)required

Model to place again.

shardingSharding (string)required

Sharding strategy it was placed with.

Possible values: [Tensor, Pipeline]

instanceMetaInstanceMeta (string)required

Serving topology it used.

Possible values: [MlxRing, MlxJaccl, LlamaRpc]

minNodesMinnodes (integer)required

Width: how many machines it spanned.

Possible values: >= 1

requestedContextTokens object

Context window the operator asked for, if any.

anyOf
integer
hostInstallIdsstring<uuid>[]required

Stable install IDs of the machines it ran on, in rank order.

excludedInstallIdsstring<uuid>[]

Machines the operator excluded, by install ID.

Default value: []
PlacementIntent
{
  "modelId": "string",
  "sharding": "Tensor",
  "instanceMeta": "MlxRing",
  "minNodes": 0,
  "requestedContextTokens": 0,
  "hostInstallIds": [
    "3fa85f64-5717-4562-b3fc-2c963f66afa6"
  ],
  "excludedInstallIds": [
    "3fa85f64-5717-4562-b3fc-2c963f66afa6"
  ]
}