Google's sloppy Gemma launch

Yesterday, Google launched their ‘Gemma’ family of models (their open-source contribution to LLM) kudos!

I’ll say, as an active member of one community it felt more “thrown over the fence” than a proper launch. I’ll explain.

I tried to run it on my local server.

Sadly, it wasn’t working because of an issue with a dependency in my stack.

Before I could try, Ollama required me to install a pre-release build, so I put in a PR to make that easier for everyone.

After getting the correct Ollama version installed, we realized that there was an issue in a dependency that was causing Gemma to fail.

llama.cpp was that dependency, which acts as a local runner within the Ollama ecosystem.

The feedback from the community was sad: “Gemma is spewing garbage” and “Gemma is slow.”

These first impressions matter and you can’t re-create that excitement the community had to play with this new model.

It’s really important for Google, and others, who are going to engage with the open-source community to recognize that Ollama, VLLM, and others like them are channels and they need to make sure their models work on launch day.

To my friends at Google, the founder/maintainer of Ollama is Jeffrey Morgan and the key maintainers of VLLM are Zhuohan Li and Woosuk Kwon. Llama.cpp is maintained and created by Georgi Gerganov.

Get to know them, and, please include their projects and communities before you launch new things.

They can help get your work running faster, act as your biggest advocate and your first feedback.