Ollama is an orchestration tool for self-hosting some of the larger available public models. Although not all GPUs will run or support all models, you can often get excellent performance from many of the medium-sized models.

When I first ran Junie with the JetBrains model, I was using up my free credits pretty quickly. Instead of burning through API tokens, I decided to connect it to Ollama. All I got was dead air. I would type a prompt and hit enter; rather than a response, it would spin for about a minute then return a timeout message.

I ran through several debugging iterations and came up with a set of suggestions:

• Decide on the model you want to use. qwen3-coder has been fine for my needs.
• Run ollama run qwen3-coder to verify it will run on your system.
• Run curl http://127.0.0.1:11434/api/generate -d '{ "model": "qwen3-coder", "prompt": "Write a Fibonacci sequence function in C", "stream": false }' will tell you whether Ollama is responding.
• Request a Junie API token. Obviously, be sure the Junie CLI is actually using the local model.
• Point Junie to Ollama in the settings. Usually it is at (assuming the same computer): https://127.0.0.1:11434/.
• If you don't see any response, try giving Junie a query and running your GPU monitor to see if power consumption rises. nvidia-smi is helpful for this on Nvidia GPUs. Sometimes the problem is Ollama doesn't get the query and sometimes it's that Junie doesn't understand the response. If you see power going up, Ollama probably got the query.

If that doesn't work, you probably have one of two issues I encountered:

Permissions issues: Ollama (when run as a system daemon) doesn't use the same repository as when you run from the command line. I updated my system repository: and added You can also run to trace Ollama calls from Junie.

Misunderstood responses: Junie is querying an API with calls and may not understand the API responses. Different models respond to different queries differently, including doing things like wrapping explanation text around compilable code which Junie doesn't know how to separate.

I solved that with:

Juniefile

FROM qwen3-coder-next:latest # Drop creativity to enforce strict syntax matches PARAMETER temperature 0.0 PARAMETER top_p 0.1 SYSTEM """You are the backend engine for the Junie CLI application. Your output is parsed programmatically. You must only return raw code blocks, requested structured tool call JSON layouts, or explicit data patches without any chat pleasantries, descriptions, introductions, or conversational filler."""

I didn't write this but found it online.