Run the Mythos Enhanced Coding Mannequin Domestically with llama.cpp and Pi

# Introduction

 
Qwythos-9B-Claude-Mythos-5-1M is a 9B reasoning and coding mannequin primarily based on Qwen3.5, designed for native coding workflows, agentic growth, and long-context duties. What makes it fascinating is that it’s sufficiently small to run on shopper {hardware}, whereas nonetheless being succesful sufficient to assist with sensible coding duties.

On this information, I’ll present you the way to run the Mythos-enhanced Qwythos mannequin domestically utilizing llama.cpp, then join it to Pi so you should utilize it as a neighborhood coding agent. I might be utilizing an RTX 4070 Ti Tremendous with 16GB of VRAM, which permits me to run the Q6_K MTP quantization comfortably. When you have an 8GB GPU, I like to recommend beginning with the Q4_K_M variant as a result of it gives a greater stability between high quality, velocity, and reminiscence utilization.

# Putting in llama.cpp

 
First, set up the official llama.cpp command-line interface (CLI). This gives you entry to the llama command, together with llama serve, which we’ll use to run the mannequin domestically.

curl -LsSf https://llama.app/set up.sh | sh

Subsequent, add the set up listing to your shell path so your terminal can discover the llama command:

echo 'export PATH="$HOME/.native/bin:$PATH"' >> ~/.bashrc
supply ~/.bashrc

To substantiate that the set up labored, run:

If every little thing is put in accurately, you need to see the accessible llama.cpp instructions and choices.

# Setting a Hugging Face Cache Listing

 
Set the Hugging Face cache someplace with sufficient free storage. That is particularly helpful on cloud machines the place the default dwelling listing has restricted disk area.

export HF_HOME="/workspace/huggingface"
mkdir -p "$HF_HOME"

To persist it throughout new terminal classes, add it to your shell configuration:

echo 'export HF_HOME="/workspace/huggingface"' >> ~/.bashrc
supply ~/.bashrc

# Beginning the Qwythos MTP Mannequin

 
Run the Q6_K MTP GGUF mannequin with all accessible GPU layers:

llama serve 
  -hf "empero-ai/Qwythos-9B-Claude-Mythos-5-1M-GGUF:MTP-Q6_K" 
  --alias "qwythos-9b-mtp" 
  --host 0.0.0.0 
  --port 8910 
  --n-gpu-layers all 
  --ctx-size 100000 
  --parallel 1 
  --batch-size 1024 
  --ubatch-size 512 
  --flash-attn on 
  --cache-type-k q8_0 
  --cache-type-v q8_0 
  --spec-type draft-mtp 
  --spec-draft-n-max 6 
  --threads 12 
  --threads-batch 24 
  --temp 0.6 
  --top-p 0.95 
  --top-k 20 
  --repeat-penalty 1.05 
  --jinja 
  --perf

The primary time you run this command, llama.cpp will obtain the mannequin information from Hugging Face.

After that, it can load the mannequin into GPU reminiscence and begin a neighborhood server.

As soon as the mannequin is working, open the native interface in your browser:

This additionally provides you an OpenAI-compatible native API endpoint at:

The Q6_K variant provides higher output high quality, however it additionally makes use of extra reminiscence. In the event you run into VRAM or RAM points, scale back --ctx-size first. If that’s nonetheless not sufficient, attempt the Q4_K_M variant as an alternative.

Listed here are an important flags within the command:

On my RTX 4070 Ti Tremendous, I used to be getting round 81.74 tokens per second. In my check, the mannequin was additionally in a position to motive by way of the duty, name the required instruments, and return the newest gold worth when used contained in the native agent workflow.

# Putting in Pi and the llama.cpp Integration

 
Pi can hook up with the native llama.cpp server and use the mannequin as a coding agent. The pi-llama plugin is designed to attach Pi with a working native llama.cpp server, without having an exterior API key.

Set up Pi:

curl -fsSL https://pi.dev/set up.sh | sh

Set up the llama.cpp plugin for Pi:

pi set up git:github.com/huggingface/pi-llama

Create a small check challenge:

mkdir new-project
cd new-project

By default, the plugin appears for llama.cpp on port 8080. On this information, our native server is working on port 8910, so we have to set the right native API deal with earlier than beginning Pi:

export LLAMA_BASE_URL="http://127.0.0.1:8910/v1"

Now launch Pi:

Inside Pi, sort:

Choose the native mannequin alias:

You must see the mannequin listed inside Pi, and as soon as chosen, you can begin utilizing it as a neighborhood coding agent.

# Testing the Mythos-Enhanced Coding Mannequin with Pi

 
Now it’s time to check the native mannequin on actual coding duties inside Pi. I used two easy however sensible duties: a browser sport and a small Python CLI device.

// Constructing a Easy Browser Sport

Begin with a immediate that asks Pi to create a small browser sport referred to as “Beat the AI.”

Create a easy browser sport referred to as “Beat the AI”.

The participant has 30 seconds to reply brief pattern-recognition questions. Every right reply will increase the rating by one. Present a countdown timer, rating show, progress bar, and a remaining outcomes display screen.

Necessities:
– Use HTML, CSS, and vanilla JavaScript solely.
– Generate a minimum of three varieties of questions, equivalent to quantity patterns, fast math, and phrase logic.
– Add a restart button after the sport ends.
– Make the interface really feel playful and polished.
– Hold all code straightforward to know and keep away from pointless information.

Take a look at the sport within the browser earlier than ending.

In my check, Pi created the complete sport in a single HTML file with embedded CSS and JavaScript, which retains the challenge easy and straightforward to examine.

Pi then summarized what it constructed, together with the 30-second countdown, rating monitoring, progress bar, query sorts, and restart button.

After that, I opened the sport within the browser to confirm that it labored accurately.

The consequence was a sophisticated little sport with:

• A countdown timer
• A rating show
• A progress bar
• A number of query sorts
• A restart movement after the sport ends

This can be a good instance of how the mannequin can deal with a whole front-end process domestically without having an exterior API.

// Constructing a CSV-to-Excel Python CLI

For the second check, ask Pi to construct a small Python CLI that converts a CSV file into an Excel .xlsx file.

Construct a easy Python CLI that converts a CSV file into an Excel .xlsx file, accepts enter and output file paths as arguments, preserves column headers, validates lacking information, and prints clear success or error messages. Earlier than ending, create a small dummy CSV file with pattern information, use it to check the CLI, confirm that the Excel file is created accurately, after which summarize the check consequence.

Pi created a Python script named csv2excel.py, generated a pattern CSV file, ran the check command, after which summarized the outcomes.

python csv2excel.py sample_data.csv output.xlsx

In my run, the abstract confirmed that the script:

• Accepted enter and output file paths.
• Preserved the column headers.
• Accurately transformed the pattern information.
• Dealt with lacking information with clear error messages.
• Dealt with lacking output paths correctly.

I additionally opened the generated Excel file to substantiate that the output was right.

The pattern output preserved the rows and headers accurately, which confirmed that the CLI labored as anticipated.

# Closing Ideas

 
I actually like this small native coding mannequin. It’s quick, correct sufficient for on a regular basis coding duties, and doesn’t want an enormous quantity of VRAM to be helpful. You need to use it with Pi, Claude Code, OpenCode, or any coding setup that helps native OpenAI-compatible or Anthropic-compatible endpoints.

For primary front-end apps, Python scripts, CLI instruments, and fast prototypes, it really works surprisingly nicely. You’ll be able to construct helpful tasks domestically with out relying on exterior APIs or paying for each request.

To get even higher outcomes, I extremely advocate including net search abilities, Context7, and different helpful Pi integrations. You can too take a look at my full information on optimizing your Pi coding agent setup right here: The best way to Set Up Kimi K2.7 Code with Pi: The Final AI Coding Setting.
 
 

Abid Ali Awan (@1abidaliawan) is a licensed information scientist skilled who loves constructing machine studying fashions. Presently, he’s specializing in content material creation and writing technical blogs on machine studying and information science applied sciences. Abid holds a Grasp’s diploma in know-how administration and a bachelor’s diploma in telecommunication engineering. His imaginative and prescient is to construct an AI product utilizing a graph neural community for college kids fighting psychological sickness.