The engine decides its checkpoint spacing per sequence and fits within a memory budget: --checkpoint-min-step auto, --fit-budget - #94

The engine decides its checkpoint spacing per sequence and fits within a memory budget: --checkpoint-min-step auto, --fit-budget#94

joelteply merged 4 commits into

889580fc/seq-state-across-stream-countsCambrianTech/llama.cpp:889580fc/seq-state-across-stream-countsfrom

fable/fit-derivedCambrianTech/llama.cpp:fable/fit-derivedCopy head branch name to clipboard