What transcription engine does this use?

whisper.cpp

This is a great point as well. I have no accent and I get stuck w/ macOS transcribing.

The first goal was just to have fun with it. I wanted to learn more about audio streaming and chunking. I was talking with a friend and said it's "super easy" and so that's how I got here.

The issue I have with the macOS transcribing is that it can never do things like learn slang, change the punctuation based on the input box, filter filler words, etc... It does work and it's primarily what I use on mobile but I just wanted to go a different direction here.

One thing I wanted to do with this project (which I likely won't do) is make it so you can tell it a set of instructions like "Go to Slack and send Tom a Direct message saying that I am going to be 5 minutes late" and it would be able to do that. That's really close to the behavior of Siri but I thought that might be neat.

But this will only ever be as good as the iOS / macOS dictation prediction. Which is OK but not really the point of what I was trying to do.

If you wanted that, you could just use accessibility mode built in to your mac.

I didn't know about this - this is neat!

Please let me know what you think :D

Super curious to hear feedback or suggestions. We have a lot of different directions we're going to explore, the next big things we're looking at are:

Model Router - Routing between different models seamlessly without killing cache rate

Incremental Progress - Add the ability for the AI system to tweak tools/prompts seamlessly to improve cost/efficiency/response

Skill / API Aggregation - Avoiding the MiTM route and using a skill so the AI reports its usage. The idea is that this would enable it to work with things like Claude Review, etc...

We added a comparison mode today so that you can see how your system reacts to the same prompt with different tools!

We made a simple profiler to help optimize AI token spend. Generally speaking anytime you want to optimize your app, whether it's for memory or otherwise you typically start with a profiler. There are a ton of MiTM Gateways but there aren't many true profilers, so I thought I'd make one. Just sign up, create a "system", instrument it, and watch. Great way to see where your tokens are going.