Up to 60% of GPU compute goes to recomputing context. Flash storage can put that capacity back to work

GPUs running large language model inference can spend up to 60% of their compute recalculating context they already processed, says Alan McSeveney, field CTO at WEKA.