Skip to main content

ClusterTimelineEntry

One flight-recorder entry annotated with cluster identity for merge.

Identical in spirit to RunnerFlightRecorderEntry but with redundant node_id / device_rank / world_size lifted onto the entry so a chronologically merged list across all ranks reads top-to-bottom as a single distributed timeline.

atAt (string)required

UTC timestamp when the runner emitted the update.

nodeIdNodeid (string)required

Node owning the runner that emitted this entry.

runnerIdRunnerid (string)required

Runner ID that emitted this entry.

deviceRankDevicerank (integer)required

Distributed device rank for this entry.

worldSizeWorldsize (integer)required

Distributed world size for this entry.

phasePhase (string)required

Runner phase at this entry.

Possible values: [created, idle, connect_group, load_model, warmup, task_submission, task_agreement, prompt_build, vision_preprocess, kv_cache_lookup, prefill_barrier, prefill_pipeline, prefill_stream, decode_barrier, decode_wait_first_token, decode_stream, parser, cancel_requested, cancel_observed, completion, error, shutdown_cleanup]

eventEvent (string)required

Short event name within the phase.

detail object

Compact human-readable detail for diagnostics.

anyOf
string
attrs object

Structured low-cardinality diagnostic attributes.

property name* object
anyOf
string
taskId object

Task ID associated with the entry, when known.

anyOf
string
commandId object

Command ID associated with the entry, when known.

anyOf
string
mlxMemory object

MLX memory snapshot captured with this entry, when present.

anyOf
generatedAtGeneratedat (string)required

UTC timestamp when the snapshot was taken.

active object

Currently active MLX memory, when the runtime exposes it.

anyOf
inBytesInbytes (integer)
Default value: 0
cache object

MLX cache memory, when the runtime exposes it.

anyOf
inBytesInbytes (integer)
Default value: 0
peak object

Peak MLX memory since the last reset, when available.

anyOf
inBytesInbytes (integer)
Default value: 0
wiredLimit object

Configured MLX wired memory limit when known. Current MLX releases do not expose a getter on all platforms, so this may be null.

anyOf
inBytesInbytes (integer)
Default value: 0
sourceSource (string)required

Runtime module that supplied the measurement, such as mlx.core.

ClusterTimelineEntry
{
  "at": "string",
  "nodeId": "string",
  "runnerId": "string",
  "deviceRank": 0,
  "worldSize": 0,
  "phase": "created",
  "event": "string",
  "detail": "string",
  "attrs": {},
  "taskId": "string",
  "commandId": "string",
  "mlxMemory": {
    "generatedAt": "string",
    "active": {
      "inBytes": 0
    },
    "cache": {
      "inBytes": 0
    },
    "peak": {
      "inBytes": 0
    },
    "wiredLimit": {
      "inBytes": 0
    },
    "source": "string"
  }
}