Ling-3.0-tiny is a very interesting model. Run on NVIDIA Orin Nano Super 8GB at 128K context with IQ4_NL quant.

Excerpt

I have been searching for suitable model to run on my 8GB RAM toy, NVIDIA Orin Nano Super 8GB. This little toy was priced at $249 earlier this year (not any more), and pulls very little power when idle. It was an interesting device that suitable for an agent to host on. It is likely won't do any coding job in meaningful way, but likely can handle simple things just fine. When the llama.cpp support showed up on github, I asked my Hermes to setup it up and run some basic test for me. The result is