Deploying vLLM’s openai compatible endpoint (you interact with it in the same way as you would the openai API) is a single line of code! You can set it up on serverless GPUs on Modal or another cloud provider. OSS AI in 5 lines of code. Experiment w/ Sampling Params, beam search, structured gen etc.

C’est vrai que ecogiquement, cette branche de progres pourra cree exponentiellement plus de problemes que les modèles avant la série O-n puisqu’on utilise bcp plus d’electricité a chaque fois qu’on utilise le model 😳.

Are you also working on a 7-8b model 👀👀

Llama 3.1 8b right? I thought I missed a drop for second 🙃

Congrats on Pleias results!

I’ve always wondered what programme you make these on?

Would you use it for code documentation, is it easy to deploy to github pages?

Spacy always write the most intuitive code, it always looks so elegant and easy to build with. Excited to give this a go 🙌🏼

So if you work in a startup, complete all goals for the year in the first 6 weeks… Sounds about right 🥲