Is anybody aware of any self hosted alternatives to Parrot.ai or Otter.ai? I’ve tried these services and I’m finding them very useful, but the price tag is a little steep. It seems like something that the open source community could solve. Anybody know of any projects, either existing or upcoming? Thanks!
You can run a transcription model and a language model (the AI you talk to) locally however you will need a beefy GPU especially if you want to run the large models for better results.
OpenAI’s Whisper is open source and does transcription, and you can run inference on language models like LLaMa (+variants) or GPT4all locally. To store information long term (“AI memory”) you could find an open source vector database but I don’t have experience with this.
Thank you! I’ll make see if I can string together a few things to come up with my own homebrew version of these services. Honestly, for what they’re charging I think I can justify a new dedicated GPU. I’ve got a few other dockers/services which could take advantage of it anyway, so maybe this is the excuse I’ve been needing to pull the trigger on that purchase.