New Skeelz

[kylebernhardy]

Add skills documentation for vector indexing, caching, JWT auth, and using blob data type across templates

[dawsontoth]

Very nice, Kyle! We can work through a few things on this. I should be able to work on it on Monday if you don't have a chance to get to my comments.

[kriszyp]

I'd love to get more of an overall understanding of why this is going in here instead of our documentation (and pointing the agents at our documentation)? I don't understand how the things in this PR are AI specific?

[dawsontoth]

By having it accessible in the file system, the AI is much more likely to get access to them, across agents. Internet access can be very agent specific. In the case of Hairper, yes, we could put these into documentation hosted somewhere, and the agent wouldn't be any the wiser. But for fast iteration, I've been placing them in here.

[dawsontoth]

(And it wouldn't help other agents if we did.)

[kriszyp]

So why not just submodule the docs then? Or publish it as a package for a dep? What are we iterating on here that is different than just bringing in all the docs?

[dawsontoth]

@kriszyp this has a different focus than the docs, and it's here for quick iteration with a strong feedback loop with Hairper. It might hint at a fundamental shift in how the docs should be written, but I don't know that yet.

Much of this is based on my experience writing tools and agents over the last several years, and so this is hunches that I'm working on getting out of my head and into the world so we can consider and measure them.

Here are a few of the grounding principles that I am operating upon:

1. Some people will look at the examples themselves as a standalone thing to help them understand Harper. I've done this constantly with other platforms, frameworks, etc. So I want https://github.com/HarperFast/create-harper/tree/main/template-react-ts to be something you can navigate to and look at with simple files there.
2. People will use agents other than Hairper to work with Harper. The present industry trend is towards skills.
3. Agents prioritize code that's on the filesystem, right in front of them.
4. Web retrievals cost more to execute, and have a lot more fluff around them to strip away before they can be meaningful.
5. Web content, instead, tends to be a part of the foundational models corpus of training data. This data lags behind by 3 months to 1 year, depending on the model and the training times. It's also heavily processed and filtered by the goals of the company training the model. So you're programming with a feedback loop of a quarter of a year.
6. Every single token costs someone money, and impacts the performance of the agent (in accuracy or latency or monetary spend).
7. Agents don't know what they can do with Harper. The skills bridge the gap from a broader notion (like real-time communication) to very succinct guidance, and very simple code examples that could be dropped in with minor customizations. The goal isn't to teach, the goal is to hand the AI a fishing pole with a fish hooked on it so it can yank it out of the water and understand the system by seeing it working.
8. Those succinct examples combine with TypeScript imports and strong types, which gives a thread that the agents follow to look at the types.

Long term, where does this live? How do we maintain and build it? Good questions.

1. We could build and tag these off of ours docs, once we have a better idea of what's missing from the docs or what could be better explained. We can measure the performance differences (tokens, accuracy of generated code, etc) to quantify what we need to accomplish, and what form it needs to take.
2. We'll want to publish these to skill directories so that we can meet others where they are. https://skills.sh/ for example. Maybe OpenClaw.
3. Those published artifacts can be pulled in here, in Studio, in Hairper, wherever we need them.

Bear with me.

[kriszyp]

Agents prioritize code that's on the filesystem

Ok, but submodules definitely put files on the filesystem...

Every single token costs someone money

But putting all the docs in the filesystem for the agent to review may be too expensive? So this is about narrowing down the docs to the most interesting bits?

it's here for quick iteration with a strong feedback loop with Hairper.

Ok, so by individually putting docs (or modified versions of docs) in here, you are trying to assess which docs are most helpful to the agents. And so at some point this information will give us more clarity about what needs to be in the local filesystem vs what is accepted as general background knowledge. And then at that point we might be able to develop a more maintainable system of pulling in individual docs, rather than trying to continually update/maintain our skills as the product and docs evolve and change?

[dawsontoth]

@kriszyp yup!

[dawsontoth]

Oh, the conflicts are from my renaming HarperDB to Harper. I can deal with that. I'll clean up the commit prefixes and formatting (whitespace) a tiny bit too once you're happy with where things are at, Kyle. Thanks again!

[dawsontoth]

Merged! After a rebase and compaction.