Thinking Machines released Inkling 2 Mini on 20 November 2026, with support for Apple-silicon Macs that have at least 32GB of memory. For a first local setup, use Llama App with the 17.6GB IQ4_NL download; MLX offers an alternative using the native checkpoint. The advertised million-token context does not establish that a 32GB Mac can use that full capacity.
Inkling 2 Mini accepts text and images and replies in text. This guide starts with a short text conversation, then adds an image and, optionally, a local client.
Choose the download before filling your disk
Confirm that your Mac has Apple silicon and at least 32GB memory. That published target does not establish support for Intel Macs or smaller-memory configurations. Close unnecessary memory-heavy applications before the first run.
The model card and GGUF repository list these sizes. GGUF is the model-file format used by llama.cpp, the engine behind Llama App; its quantized versions store weights at reduced numerical precision.
Download | Listed size | How to use it |
|---|---|---|
GGUF IQ4_NL | 17.6GB | Starting choice for the Llama route below. |
Native checkpoint | About 20GB | Alternative for an MLX build supporting this model. |
GGUF Q8_0 | 27.8GB | A larger download; do not assume it leaves enough runtime headroom on 32GB. |
GGUF BF16 | 52.2GB | Not the starting choice for this 32GB setup. |
For image input, the GGUF repository also lists a matching vision encoder: 346MB in Q8_0 or 642MB in BF16. Allow for that additional file.
Choosing IQ4_NL here is a memory-planning decision, not a claim that its answers match the other versions. A file smaller than total memory is not a complete fit calculation: conversation caches, working buffers, macOS and other applications also need room. The MLX memory guidance explains why long prompts and caches add to memory demands.
Nor does the 52.2GB BF16 download recover an undistributed higher-precision original: the GGUF maintainers say it expands the native low-precision expert weights into BF16. For another example of separating advertised capacity from a whole-machine budget, see what Nemotron Bolt’s 24GB target means for local AI; those figures are not Inkling requirements.
Install Llama and start with explicit limits
Download the Mac app from Llama’s installation page. It also installs the command-line tool. Open a new terminal and check that it is available:
llama --versionUse a current build with Inkling 2 Mini support. A minimum supporting build number has not been established, so a version printout alone does not prove that an older installation can load this release.
The following combines the maintainer’s model identifier with documented Llama CLI controls:
llama cli -hf ggml-org/Inkling-2-Mini-GGUF:IQ4_NL -c 4096 -n 1024-hfselects the Hub download, cached after first use.-c 4096sets a 4,096-token context window—the working space for the conversation.-n 1024limits generation to 1,024 tokens. Tokens are units of text, not necessarily whole words.
These are illustrative first-run settings for a short question, not vendor-prescribed optima or a measured guarantee of fit. Try a simple request such as asking for three bullet points about a short paragraph you provide. Check memory pressure and responsiveness before increasing the workload.
The output cap may stop the model’s reasoning before it reaches a final answer. If that happens, inspect the response and adjust deliberately. Do not jump to -c 0, which requests the model’s full context.
Keep output limits separate from agent-turn limits
Thinking Machines’ model card reports that Mini sometimes repeated its reasoning without progress in internal testing, especially while debugging. It recommends limits on both generated output and agent turns. Llama’s documented CLI default is unlimited generation, making the explicit cap above useful.
An agent-turn limit belongs in the application that repeatedly calls the model and executes tools. Set a finite value there before handing it a multi-step job; the provider gives no universal number. A token cap bounds a response, while a turn cap bounds the repeated model-and-tool cycle.
Reducing reasoning_effort is not a substitute. The model card says minimum effort does not eliminate reasoning, and Mini still uses three passes through its middle layers. If adding file-editing tools, follow its recommendation to sandbox the workload and require confirmation for destructive actions.
Add an image after the text check
Use a local PNG or JPEG whose contents you can check. In the interactive CLI, Llama documents adding the image before the question:
/image image.pngThen ask what is visible in that image. This is image understanding with a text response, not image generation.
The multimodal loading instructions distinguish two routes: Hub loading can retrieve the matching image assets, while manually loaded GGUF files require the correct --mmproj file. Check that the vision component loads; the language-model file alone is not the complete image setup.
The model card recommends each image dimension be between 40 and 4,096 pixels. Start with one image and a short question; larger visual inputs may require revisiting the initial context budget.
Use a browser or connect a local client
For a browser interface, stop the CLI session before starting the server instead of loading a second copy. Make sure another Llama server is not already using the same port:
llama serve -hf ggml-org/Inkling-2-Mini-GGUF:IQ4_NL -c 4096 -np 1 -n 1024The server guide documents -np 1 as one concurrent request slot and the default browser address as http://127.0.0.1:9931. The server reference documents the generated-token cap.
A compatible client can use the local API base http://localhost:9931/v1. Check its per-request output limit and agent-turn budget too; a server default is not a substitute for the client’s settings. Protocol compatibility alone does not prove that a particular agent’s tool calls work with Mini.
If you maintain an application embedding the runtime rather than calling its HTTP server, the separate llama.cpp application-compatibility guide explains why library upgrades and server-client changes are different tasks.
When to choose MLX instead
Hugging Face’s MLX walkthrough documents loading thinkingmachines/Inkling-2-Mini directly, retaining the original mixed-precision weights without conversion. Choose this alternative if you already use MLX and want the native checkpoint; it is not an extra step required by Llama.
The first published MLX package version supporting Mini remains unverified. Follow the linked integration instructions for a compatible build rather than assuming any existing installation—or a generic package upgrade—supports the new architecture. The walkthrough also supplies its model-specific image example.
What local use does—and does not—cost
Llama App is free and open source. Running the model on your Mac does not incur a hosted inference token bill, but storage, electricity and any connected paid tools remain costs. Thinking Machines’ paid Tinker service is a separate hosted option, not a prerequisite for this local route.
Local model computation also does not make an entire connected assistant offline. Web searches, remote tools and cloud-routed clients have their own data paths. Check those connections separately.
The model card identifies an Apache 2.0 licence and links a Model Acceptable Use Policy. Review both before building a product around the weights; the licence label alone is not a promise of unrestricted use.
For an existing 32GB Mac, the useful first milestone is a bounded conversation and one image check on your own machine. Thinking Machines’ launch demonstration used a 48GB MacBook Pro; it does not establish speed on your 32GB configuration. Increase context or add tools only after checking the memory use and answers for your actual task.
