Local AI Models

SupaSidebar supports three ways to run AI models for tag suggestions without sending your links to a hosted cloud service: built-in MLX models, Ollama, and any OpenAI-compatible server you point it at. The first two need no API key at all, and all three let you pick a model and see live connection status before you rely on them.

Option 1: Built-in Models (MLX)

The simplest option. Models download and run directly inside SupaSidebar using Apple’s MLX framework.

Requirements: Apple Silicon Mac (M1, M2, M3, or M4)

Setup:

1. Open Preferences → AI Tags
2. Set AI Provider to Local AI (MLX)
3. Click Download on your preferred model
4. Once downloaded, click Use This to activate it

Available Models:

RAM Guide:

• 8GB Mac - Use models up to 3B parameters (Gemma 2 2B, Phi-4 Mini)
• 16GB+ Mac - Can use all models including Qwen 2.5 7B

Option 2: Ollama (Any Local Model)

Use any model you want through Ollama. This gives you access to thousands of models and works on both Apple Silicon and Intel Macs.

Step 1: Install Ollama

1. Go to ollama.com  and download the macOS app
2. Open Ollama - it runs quietly in the background

Step 2: Download a Model

Open Terminal and run:

ollama pull qwen2.5:3b

This downloads the Qwen 2.5 3B model, which we recommend for tag suggestions. You can also pull other models - any text generation model works.

Step 3: Configure in SupaSidebar

1. Open Preferences → AI Tags
2. Set AI Provider to Ollama (Local)
3. Click Check to verify the connection
4. Select your model from the dropdown (it auto-populates from Ollama)
5. Enable Auto-suggest tags if you want automatic suggestions

Recommended Ollama Models

You can also use larger models for better accuracy:

ollama pull qwen2.5:7b    # 4.4 GB, very accurate
ollama pull llama3.1:8b   # 4.7 GB, excellent quality

Tips

• Model names are not case sensitive - qwen2.5:3b and Qwen2.5:3B both work
• Any text generation model works - if you already have models in Ollama, they’ll appear in the dropdown
• The endpoint defaults to http://localhost:11434 - only change this if you’re running Ollama on another machine
• You can use different models for Voice (in Voice settings) and Tags (in AI Tags settings)

Option 3: OpenAI-Compatible Server

Many local and self-hosted model runners (LM Studio, llama.cpp, vLLM, text-generation-webui) and most hosted gateways expose an OpenAI-compatible API. SupaSidebar can talk to any of them, so you can use a model running on another machine on your network or behind your own endpoint.

Setup

1. Open Preferences → AI Tags
2. Set AI Provider to OpenAI-Compatible
3. Enter the server’s base URL (for example http://localhost:1234/v1)
4. Add an API key if your server requires one. Leave it blank for local servers that don’t.
5. Click Check to verify the connection and see live status
6. Pick your model from the dropdown, or type the model name if the server doesn’t list one

The model dropdown auto-populates from servers that report their available models. If your server doesn’t, the model field accepts a name you type in.

MLX vs Ollama vs OpenAI-Compatible

Our recommendation: Start with MLX + Phi-4 Mini for the easiest setup. Switch to Ollama if you want more model choices or have an Intel Mac, or use OpenAI-Compatible to reach a model running on another machine or your own endpoint.