Ollama

Ollama is an installable runtime for running LLMs locally.

Basic commands:

• ollama pull xxx
• ollama run xxx
• ollama serve
• ollama ls

An alternative to Ollama is LM Studio.

Ollama Python interface

You can run Ollama and connect a Python script to it, creating a programmatic interface: