Aurora1.0-150M Releases!

Excerpt

The first generation of our 150M model has just been released Its performance is similar to that of GPT2-Small The benchmarks: PIQA: 62.24% Hellaswag: 32.20% Arc-Easy: 44.91% Arc-Challenge: 25.00% Arithmark 3.0: 33.90% CapitalBench: 36.55% It was trained on 7B tokens, using an RTX Pro 6000 an example inference script to try it out yourself is available in the Huggingface repo If there's any question, I'll gladly answer them! submitted by /u/Tall_Abrocoma_3533 [link] [comments]