I finished upcycling of gemma4-12B

Excerpt

Add 4 experts into Dense model and confirmed recovering model's ability up to "general level". Hey, google. Please releae official 124B MoE model!!!!!!! submitted by /u/Desperate-Sir-5088 [link] [comments]