Anyone else tried out KV cache blending?

Excerpt

Idea is simple-ish in abstract: instead of running normal prefill over all of a given prompt, split it into parts - generates caches for part A and part B in isolation, concatenate the result, feed it into decode like normal. I honestly thought it'd totally fail. But I've been trying it out on Ling3-tiny for the non KDA layers - and if I give them some overlap between chunks, the model actually maintains full needle-in-haystack retrieval. Can even synthesise across the split parts. Currently run