Turn your spare Android phone into a distributed local LLM inference server with a single click, completely bypassing complicated Termux terminal configurations.
Why Let Old Phones Collect Dust in a Drawer?
Modern Android flagships and mid-range devices pack surprisingly capable NPUs and multi-core processors. Yet, most developers keep their old phones in a desk drawer while paying monthly cloud fees to run small quantized language models.
Servllama (github.com/milkboyinchina/servllama) solves this by packaging an end-to-end local LLM inference proxy server into an intuitive mobile interface. No manual Termux installation, no compiling from scratch, and no root permissions required. Just tap start, point your LAN tools to your phone IP, and enjoy off-grid AI inference.
Core Features
- 1-Click Launch: Zero command-line setup required; launches an OpenAI-compatible API endpoint directly on your local Wi-Fi network.
- No Termux or Root Required: Runs purely within native Android application sandboxes with automated background wake locks.
- Distributed Inference Hub: Distributes light completion workloads across connected edge nodes to ease desktop CPU loads.
- Thermal & Battery Watchdog: Automatically throttles batch context lengths when device temperatures climb to protect phone batteries.
- OpenAI API Compatible: Drop-in compatible with standard client SDKs, Continue.dev, Chatbox, and web frontends.
Quick Start
# Clone the repository
git clone https://github.com/milkboyinchina/servllama.git
cd servllama
# Install dependencies and test server proxy
npm install
npm run devProject source and mobile releases on GitHub: milkboyinchina/servllama.
