feat: Major features v1.3.0 - Cost Tracking, Caching, Multi-User, A/B Testing #7
Conversation
## New Features ### Core Modules Added - **Cost Tracking** (metrics.py): Per-request cost calculation, budget limits, alerts - **Request Caching** (cache.py): LRU cache with TTL, invalidation strategies - **Multi-User Support** (users.py): Per-user token/cost limits, model access control - **A/B Testing** (ab_testing.py): Model comparison with statistical analysis ### Bug Fixes - OAuth graceful fallback when credentials missing - Router initialization race condition fixed - Request metadata store memory leak fixed (TTL + max size) - Model reload thrashing fixed with cooldown - Default config now works out of the box ### Enhancements - Health check endpoint: ccproxy status --health - Shell integration: ccproxy shell-integration --shell [bash|zsh|fish] - OAuth token background refresh - Configuration validation at startup - Global tokenizer cache for performance - Request retry with exponential backoff ### Documentation - docs/troubleshooting.md: Common issues and solutions - docs/architecture.md: System design with ASCII diagrams - docs/examples.md: Configuration examples for various use cases - README.md updated with all new features ### Tests - 356 tests passing - New test files: test_metrics, test_cache, test_users, test_ab_testing, etc. - Coverage improved to 71%
|
Before reviewing 6,000 lines: can you write up, in your own words, what problem you're solving and what value this adds? |
|
Hey, You're right, 6k lines was way too much. My bad. Been thinking about this - wanna break it into small PRs instead. Each one solving a specific thing. First one would be config validation with helpful errors. Fixes that "default config doesn't work" issue. Pretty small change, maybe couple hundred lines, no breaking stuff. Basically when someone messes up the YAML they get an actual helpful message instead of a stack trace. After that got some other stuff lined up (oauth race condition, connection pooling etc) but lets start small. Want me to go ahead with the config one? |
|
Happy new year! actually yea config validation would be great, but the "default" behavior is what, normal claude code usage? users interacting through their llm harness may not be aware of adding ccproxy as a dep, add the alias from the readme and their harness stops working. However, a config assist would be too useful to pass up for what's coming in the pre-release I'm going post today, it's all on the dev branch already. It has some important new features, including some that I would only talk about off-platform besides linking this Also, sorry, it likely will break your in-progress version. |
|
Hey @starbaser, thanks for the heads up! I've switched to the dev branch as suggested to avoid conflicts with the new features. I've implemented the config validation logic we discussed. It now catches I've pushed the changes to my fork. Should I open a new PR for this feature (feat/config-validation) targeting dev, or would you prefer to handle it differently? |
2 participants
Summary
This PR adds major new features to ccproxy v1.3.0, including cost tracking, request caching, multi-user support, and A/B testing framework. All features are fully tested with 356 tests passing.
New Modules
metrics.pycache.pyusers.pyab_testing.pyBug Fixes
Enhancements
ccproxy status --healthfor health metricsccproxy shell-integrationfor shell aliasesDocumentation
docs/troubleshooting.md- Common issuesdocs/architecture.md- System designdocs/examples.md- Configuration examplesTests