Optional
Context window, in tokens, that llama-server, in-process llama.cpp and vLLM placements get when the placement names none. Those engines reserve the whole window's memory when the model loads. Absent means the server default (32768).
Context window, in tokens, that llama-server, in-process llama.cpp and vLLM placements get when the placement names none. Those engines reserve the whole window's memory when the model loads. Absent means the server default (32768).