Making slow progress experimenting with local LLMs in the Mac app. Smaller models are a challenge because they are so much worse and unpredictable than what I’m used to with frontier cloud models. I’ve tested a few flavors of Gemma 3, think I’m settling on 4 billion params, Q5.