Extracting Concepts from GPT-4

Excerpt

Using new techniques for scaling sparse autoencoders, we automatically identified 16 million patterns in GPT-4's computations.