GenerationStats

prompt_tpsPrompt Tps (number)required

generation_tpsGeneration Tps (number)required

prompt_tokensPrompt Tokens (integer)required

generation_tokensGeneration Tokens (integer)required

peak_memory_usage objectrequired

inBytesInbytes (integer)

Default value: 0

GenerationStats

{
  "prompt_tps": 0,
  "generation_tps": 0,
  "prompt_tokens": 0,
  "generation_tokens": 0,
  "peak_memory_usage": {
    "inBytes": 0
  }
}