TinyMobileLLM

TinyMobileLLM is a research-style project that benchmarks tiny language models (0.5B–2B parameters) on both PC and Mobile hardware.
The purpose is to understand:

• how fast tiny LLMs run on real smartphones
• how quantization affects speed & memory
• which architectures (Transformer vs Recurrent) perform better
• how multi-threading scales on mobile CPUs
• whether tiny LLMs are usable for real offline apps

All tests use llama.cpp with GGUF models.

---

Project Structure

tinyMobileLLM/
│
├── README.md
├── LICENSE
├── .gitignore
│
├── models/                # GGUF models (NOT committed)
├── llama.cpp/             # Windows or Termux build
│
├── docs/
│   ├── 01_overview.md
│   ├── 02_pc_setup.md
│   ├── 03_model_inventory.md
│   ├── 04_benchmark_methodology.md
│   ├── 05_results_summary.md
│   └── 06_future_work.md
│   ├── experiments_pc/
│   └── experiments_mobile/
│
├── benchmarks/
│   ├── pc_logs/
│   └── mobile_logs/
│
├── scripts/
│   ├── pc_benchmark.ps1
│   └── termux_benchmark.sh
│   
│
└── media/
    ├── screenshots/
    └──recordings/
    

---

Requirements

PC

• Windows 10
• i5-12400F
• 16GB DDR4
• llama.cpp b7109

Mobile

• Snapdragon 855
• 6GB RAM
• Termux
• Android 12

---

Download Required Models (GGUF)

You must download the same models used in our benchmarks.

Qwen2.5 Models (0.5B & 1.5B)

https://huggingface.co/Qwen/Qwen2.5-0.5B-Instruct-GGUF
https://huggingface.co/Qwen/Qwen2.5-1.5B-Instruct-GGUF/tree/main

Gemma e2B Q3_K_M

https://huggingface.co/gleidsonnunes/gemma-3n-E2B-it-Q3_K_M-GGUF/tree/main

RecurrentGemma 2B Q2_K

https://huggingface.co/archaeus06/RLPR-Gemma2-2B-it-Q2_K-GGUF/tree/main

Place them inside:

tinyMobileLLM/models/<model-family>/

(Full structure shown in Model Inventory.)

Quickstart

PC Inference

.\llama-cli.exe -m "models/qwen2.5/qwen2.5-0.5b-instruct-q5_k_m.gguf" -p "Hello" -n 200

Mobile Inference

./llama-cli -m "/data/.../qwen2.5-0.5b-instruct-q5_k_m.gguf" -p "Hello" -n 100

---

Summary Tables

PC Decode Speed (tokens/s)

Model	Quant	TPS	Memory
Qwen0.5B	Q5_K_M	80.58	852 MB
Qwen1.5B	Q3_K_M	39.79	1290 MB
Qwen1.5B	Q4_K_M	33.85	1474 MB
Qwen1.5B	Q5_K_M	33.44	1635 MB
Gemma e2B	Q3_K_M	22.29	2770 MB
RecurrentGemma 2B	Q2_K	26.00	2087 MB

---

Mobile Decode Speed (Thread = 1)

Model	Quant	TPS	Memory
Qwen0.5B	Q5_K_M	16.25	852 MB
Qwen1.5B	Q3_K_M	7.60	1290 MB
Qwen1.5B	Q4_K_M	6.29	1474 MB
Qwen1.5B	Q5_K_M	5.98	1635 MB
RecurrentGemma 2B	Q2_K	5.10	2087 MB
Gemma e2B	Q3_K_M	3.65	2770 MB

---

Mobile Multi-Thread Scaling (t1 → t4)

Model	t1 TPS	t4 TPS	Scaling
Qwen0.5B Q5	16.25	15.45	↓ none
Qwen1.5B Q3	7.60	13.81	↑ good
Qwen1.5B Q5	5.98	11.11	↑ good
RecurrentGemma 2B	5.10	8.88	↑ very good
Gemma e2B Q3	3.65	N/A	—

---

Recommended Tiny Models for Mobile

Rank	Model	Why
#1	Qwen1.5B Q3_K_M	Best speed/quality balance
#2	RecurrentGemma 2B Q2_K	Best large model for phones
#3	Qwen0.5B Q5_K_M	Extremely fast & lightweight

---

Experiment Documentation

• All PC experiments → docs/experiments_pc/
• All Mobile experiments → docs/experiments_mobile/
• Raw logs → benchmarks/{pc_logs,mobile_logs}

Each experiment includes:

• commands
• raw logs
• extracted metrics
• sample output
• interpretation

---

Future Work

• more models (Phi-2, MiniCPM, RWKV)
• more devices (Snapdragon 8 Gen 1/2)
• thermal profiling
• quality scoring
• automated benchmark scripts

---

Youtube video representation language (hindi)

🤝Contributions

PRs are welcome — especially additional mobile devices and models.