ClusterTimelineRunner

Compact current-state synopsis for one runner in the cluster timeline view.

Designed for at-a-glance debugging of distributed deadlocks: each runner's rank, current phase, and time-stuck-in-phase make a rank-disagreement pattern visible without having to cross-reference per-node payloads.

nodeIdNodeid (string)required

Node owning this runner.

runnerIdRunnerid (string)required

Runner ID.

instanceIdInstanceid (string)required

Instance ID.

modelIdModelid (string)required

Model assigned to this runner.

deviceRankDevicerank (integer)required

Distributed device rank.

worldSizeWorldsize (integer)required

Distributed world size.

pid object

Runner subprocess PID, when started.

anyOf

• integer
• null

integer

processAliveProcessalive (boolean)required

Whether the runner subprocess is alive.

statusKindStatuskind (string)required

Current runner status variant.

phasePhase (string)required

Last runner phase reported.

Possible values: [created, idle, connect_group, load_model, warmup, task_submission, task_agreement, prompt_build, vision_preprocess, kv_cache_lookup, prefill_barrier, prefill_pipeline, prefill_stream, decode_barrier, decode_wait_first_token, decode_stream, parser, cancel_requested, cancel_observed, completion, error, shutdown_cleanup]

phaseDetail object

Compact human-readable detail for the current phase.

anyOf

• string
• null

string

secondsInPhaseSecondsinphase (number)required

Wall-clock seconds spent in the current phase.

lastProgressAt object

UTC timestamp for the last flight-recorder update.

anyOf

• string
• null

string

activeTaskId object

Task ID associated with the current phase, when known.

anyOf

• string
• null

string

activeCommandId object

Command ID associated with the current phase, when known.

anyOf

• string
• null

string

lastMlxMemory object

Most recent MLX memory snapshot reported by the runner.

anyOf

• MlxMemorySnapshot
• null

generatedAtGeneratedat (string)required

UTC timestamp when the snapshot was taken.

active object

Currently active MLX memory, when the runtime exposes it.

anyOf

• Memory
• null

inBytesInbytes (integer)

Default value: 0

cache object

MLX cache memory, when the runtime exposes it.

anyOf

• Memory
• null

inBytesInbytes (integer)

Default value: 0

peak object

Peak MLX memory since the last reset, when available.

anyOf

• Memory
• null

inBytesInbytes (integer)

Default value: 0

wiredLimit object

Configured MLX wired memory limit when known. Current MLX releases do not expose a getter on all platforms, so this may be null.

anyOf

• Memory
• null

inBytesInbytes (integer)

Default value: 0

sourceSource (string)required

Runtime module that supplied the measurement, such as mlx.core.

ClusterTimelineRunner

{
  "nodeId": "string",
  "runnerId": "string",
  "instanceId": "string",
  "modelId": "string",
  "deviceRank": 0,
  "worldSize": 0,
  "pid": 0,
  "processAlive": true,
  "statusKind": "string",
  "phase": "created",
  "phaseDetail": "string",
  "secondsInPhase": 0,
  "lastProgressAt": "string",
  "activeTaskId": "string",
  "activeCommandId": "string",
  "lastMlxMemory": {
    "generatedAt": "string",
    "active": {
      "inBytes": 0
    },
    "cache": {
      "inBytes": 0
    },
    "peak": {
      "inBytes": 0
    },
    "wiredLimit": {
      "inBytes": 0
    },
    "source": "string"
  }
}