Ollama: run your first local AI model

Start with a small, inspectable task

Ollama provides a local model runtime and tools for downloading and running supported models. Its code is MIT-licensed. The models are separate artifacts with their own licenses, download sizes and hardware needs.

A useful first experiment is rewriting a short, non-sensitive note into three bullets. You can compare the result with the original yourself. That gives you a better starting point than immediately wiring a model into customer records or unattended actions.

This walkthrough uses macOS and the small qwen3:0.6b model as a starting example. Documentation and model details were reviewed on November 2, 2026; no benchmark or new araNET deployment is being claimed.

Install the native Mac application

Download Ollama from the official site, open the disk image and move the application to Applications. Launch it and complete the local command-line setup when prompted. The macOS documentation requires Sonoma 14 or later and supports Apple Silicon; Intel Macs use CPU execution.

Use the native application for this walkthrough. Docker Desktop on macOS does not provide the same GPU access. Windows and Linux have separate official setup paths; commands for one platform should not be pasted into another without checking them.

Open a new terminal and confirm the command is available:

ollama --version

Choose local execution deliberately

Ollama also supports cloud features. Selecting a local model matters if your goal is local inference. In the server configuration at ~/.ollama/server.json, merge the following setting with any existing settings rather than overwriting the file:

{
  "disable_ollama_cloud": true
}

Restart Ollama after saving. The FAQ documents this local-only option and the corresponding log confirmation. It disables cloud models and web search. It does not make your entire computer, extensions or other applications offline, and downloading models still uses the network.

Download one model and try a prompt

The Qwen3 0.6B listing identifies an Apache-2.0 model artifact with a download size of about 523 MB at review time. That is the file size, not the total memory required to run it. Longer context and concurrent work need additional memory. Check available disk space before downloading.

ollama pull qwen3:0.6b
ollama run qwen3:0.6b

At the prompt, try: “Turn this note into three bullets. Do not add facts: The sample website has five pages. Photos are still missing. The owner needs to review the contact form.” Compare every bullet with the note. A small model can omit details, invent connections or produce a weak answer; a successful download does not establish useful accuracy.

Type /bye to leave the interactive session. These commands let you inspect the downloaded models and current runtime state:

ollama list
ollama ps

Call the local API

Ollama’s local API normally listens at 127.0.0.1:11434. With the app running and the model downloaded, send a small request from another terminal:

curl http://127.0.0.1:11434/api/chat \
  -H 'Content-Type: application/json' \
  -d '{"model":"qwen3:0.6b","messages":[{"role":"user","content":"Name three ordinary desk items."}],"stream":false}'

Look for the answer in message.content. Setting stream to false returns a completed JSON response instead of a sequence of streamed chunks. The API introduction describes the endpoints and response format.

If the connection fails, confirm the app is running. If the model is missing, inspect ollama list and pull the exact name used in your request. Keep the default loopback binding for this first experiment; exposing the API is a separate access-control task.

Give the experiment a fair check

Use a handful of short examples with answers you can verify. Keep an unchanged copy of the inputs, the exact model name and the outputs. Note whether information was preserved, unsupported details were added and the result actually helped with the task.

Repeat the same inputs before changing models. Larger downloads can improve some tasks while also increasing memory use and latency. Avoid turning one impressive response into a claim about every kind of work.

Know what stays on the machine

For local model execution, the prompt is processed by the local runtime. That boundary changes if you choose a cloud model, connect a remote integration or enable a network tool. Review each component rather than treating “local AI” as a blanket privacy guarantee.

When finished, release the loaded model’s memory without deleting the downloaded artifact:

ollama stop qwen3:0.6b

The first milestone is modest: a local model you can start, inspect, call and stop, with a task you can judge yourself. From there, a larger model or a simple local application has a clear foundation.