uni-llm-api: The Unified LLM API Layer

One interface, consistent results, powered by Rust.

Background

After using the free deepseek-r1 API provided by various cloud vendors and integrating it into the frontend, I noticed that the display process was not smooth and occasionally froze. However, when I directly called the stream API using curl, this issue did not occur. Upon investigation, I found that the problem was caused by inconsistencies in how different APIs handle and return the chain-of-thought results. This led me to start developing this tool to address the issue.

Here are a few platforms I'm currently using that offer free API access:

• deepseek-platform: Offers a 10 CNY credit
• siliconflow: Offers a 14 CNY credit
• alibaba cloud: Offers 10 million tokens, expiring after six months
• bytedance cloud: Offers 500,000 tokens per model
• tencent cloud: Free access until February 26
• google gemini: Totally free ...

Features

• Unified Interface : Standardizes APIs to match the Ollama API,and specifically tailored the experience for OpenWebUI.
• High Performance : After comparing the performance and speed of directly adapting plugins through OpenWebUI's API with this tool, we've observed a performance boost of approximately 50%.
• API Proxying : This feature allows for custom proxy configurations for each individual API provider, which is incredibly useful for users within China accessing APIs from services like Gemini or OpenAI.
• Easy to manage API calls : By storing API keys and preferred models for various vendors in a single JSON file, and then exposing a unified API for external tools to call, we've significantly streamlined model integration for local AI tools.

QuickStart

Install

1. By downloading the precompiled binary: releases

2. Install via the cargo install:

cargo install --git https://github.com/acking-you/uni-llm-api.git --bin uni-llm

Usage

Configuration file

Start the service

Integrated into other AI tools

OpenWebUi