Posts tagged “local-inference”

• llama.cpp b11413: How to Test Vulkan Sparse FlashAttention with Quantized KV

  Check eligibility for llama.cpp’s Vulkan sparse-attention change, understand the 16× cache threshold, and compare performance with the follow-up fix included.

• llama.cpp b11393 Fixes Tool-Call Parser Use-After-Free

  llama.cpp b11393 clears a stale tool-call pointer. See which parser configurations warrant an upgrade and why the earlier b11390 CUDA fix is separate.

• llama.cpp OpenVINO 2026.4.1 Update: Intel MoE Upgrade Guide

  What llama.cpp b11374 changes for Intel OpenVINO users, how to interpret the Gemma MoE benchmark, and what to validate before a production upgrade.

• llama.cpp /v1/systemone: Running Typed Decision Models Locally

  Serve local routing and classification models with llama.cpp’s new endpoint, choose a checkpoint, and check compatibility before replacing a hosted API.

← Back to blog