Skip to main content

InferenceAssessment

Estimated feasibility, independently of observed serving readiness.

modelIdModelid (string)required
cardDigestCarddigest (string)required

Canonical authorized-card content identity, excluding catalog snapshot.

sourceRepositorySourcerepository (string)required
sourceRevision objectrequired
anyOf
string
selectedFile objectrequired
anyOf
string
quantizationQuantization (string)required
workload objectrequired

Requested text workload; context includes input and generated tokens.

contextTokensContexttokens (integer)

Input plus output context tokens per simultaneous request.

Possible values: >= 256 and <= 1048576

Default value: 8192
concurrentRequestsConcurrentrequests (integer)

Requested simultaneous serving width, without a performance SLO.

Possible values: >= 1 and <= 16

Default value: 1
statusStatus (string)required

Possible values: [local, external_required, blocked, insufficient_evidence]

readinessReadiness (string)

Possible values: [not_observed, loaded_not_workload_verified]

Default value: not_observed
readAtReadat (string)required

API-side UTC assessment time, not payload measurement freshness.

reasonsstring[]required
assumptionsstring[]required
uncertaintystring[]required
weightBytesWeightbytes (integer)required
weightEvidenceWeightevidence (string)

Weight provenance is the bound card, not a measured resident-memory claim.

Constant value: card_declared
Default value: card_declared
localReplicas object[]
  • Array [
    • shardingSharding (string)required
    shards object[]required
  • Array [
    • nodeIdNodeid (string)required
    backend objectrequired
    anyOf
    string
    estimatedMemoryBytes objectrequired

    Shared estimated footprint; null for runtime-selected RPC partitions.

    anyOf
    integer
    usableMemoryBytesUsablememorybytes (integer)required

    Memory available after current commitments and working-set caps.

    memoryPoolMemorypool (string)required

    Possible values: [ram, vram]

  • ]
    • concurrentRequestsConcurrentrequests (integer)required
    contextPoolTokensContextpooltokens (integer)required

    Aggregate reserved context for requests assigned to this replica.

    existingInstanceId object
    anyOf
    string
  • ]
    • externalConfigurations object[]
  • Array [
    • backendBackend (string)required
    engineBuild objectrequired
    anyOf
    string
    hardwareClassesstring[]required
    replicaCountReplicacount (integer)required
    requestsPerReplicaRequestsperreplica (integer)required
    contextPoolTokensContextpooltokens (integer)required
    nodesPerReplicaNodesperreplica (integer)
    Constant value: 1
    Default value: 1
    gpusPerNodeGpuspernode (integer)required

    Discrete GPU requirement; zero includes CPU and Apple unified-memory hosts.

    Possible values: [0, 1]

    memoryPoolMemorypool (string)required

    Possible values: [ram, vram]

    estimatedMemoryBytesPerReplicaEstimatedmemorybytesperreplica (integer)required
    minimumMemoryBytesPerNodeMinimummemorybytespernode (integer)required

    Physical RAM or VRAM needed to expose the estimated usable working set.

    minimumWeightDiskBytesPerNodeMinimumweightdiskbytespernode (integer)required

    Selected weights and projector only; no installation or staging allowance.

    hostRamBytes object

    Unknown for discrete GPU configurations; staging/runtime need extra RAM.

    anyOf
    integer
    offerAvailableOfferavailable (boolean)

    No live offer, stock or price was read for this hypothetical specification.

    Constant value: false
    Default value: false
  • ]
    • provisioningProvisioning (string)

    This interface performs no acquisition and grants no execution authority.

    Constant value: recommendation_only
    Default value: recommendation_only
    InferenceAssessment
    {
      "modelId": "string",
      "cardDigest": "string",
      "sourceRepository": "string",
      "sourceRevision": "string",
      "selectedFile": "string",
      "quantization": "string",
      "workload": {
        "contextTokens": 8192,
        "concurrentRequests": 1
      },
      "status": "local",
      "readiness": "not_observed",
      "readAt": "string",
      "reasons": [
        "string"
      ],
      "assumptions": [
        "string"
      ],
      "uncertainty": [
        "string"
      ],
      "weightBytes": 0,
      "weightEvidence": "card_declared",
      "localReplicas": [
        {
          "sharding": "string",
          "shards": [
            {
              "nodeId": "string",
              "backend": "string",
              "estimatedMemoryBytes": 0,
              "usableMemoryBytes": 0,
              "memoryPool": "ram"
            }
          ],
          "concurrentRequests": 0,
          "contextPoolTokens": 0,
          "existingInstanceId": "string"
        }
      ],
      "externalConfigurations": [
        {
          "backend": "string",
          "engineBuild": "string",
          "hardwareClasses": [
            "string"
          ],
          "replicaCount": 0,
          "requestsPerReplica": 0,
          "contextPoolTokens": 0,
          "nodesPerReplica": 1,
          "gpusPerNode": 0,
          "memoryPool": "ram",
          "estimatedMemoryBytesPerReplica": 0,
          "minimumMemoryBytesPerNode": 0,
          "minimumWeightDiskBytesPerNode": 0,
          "hostRamBytes": 0,
          "offerAvailable": false
        }
      ],
      "provisioning": "recommendation_only"
    }