[Feature] Open scaling query API to everyone

[marcelroed]

The current API is hosted on Stanford's intranet and requires an API key per user to limit query budgets.

There are at least two different options for how we open this up to everyone, with trade-offs:

1. Package results into a Python package that can be queried, and store a local file with usage to determine how much of the budget they've used. Issues: How do we ship the training information without leaking all the data points? How do we prevent people from resetting their budget? Do we want to have a hard limit per user?
2. Open access to the current API to anyone. Issues: How do we distribute API keys? What amount of manual labor is involved? Need to optimize the model to prevent server overload (the current code is inefficient and already times out when users submit too many previous queries).

[ByzanTine]

Thanks for considering it for public, I am quite interested in playing around with this. I would vouch for option 1, that would be great for educational purpose. Given we are not chasing for grades, leaking/budget resetting is probably less a concern?

The downside would be Stanford staff may need to change the model architecture etc for next class session (likely newer architecture variants would show up next year anyway?).

[asdf93074]

Yes, do consider making it public somehow so I can do this assignment properly as well. What exactly is underneath the training API?

[philharmonic77]

Is there a conclusion now? Thank you for considering this — it’s exactly what I need right now.

[marcelroed]

I'll be working on this shortly to prepare for the 2026 iteration of the class. The current direction is to create a Python package with the data bundled. This means it needs to release after we wrap up leaderboards for the class, which will still use an API with user API keys distributed to the class. If all goes well I can release the package with last years data before the class starts.