The stacked sparse autoencoder is a powerful deep learning architecture composed of multiple autoencoder layers, with each layer responsible for extracting features at different levels.

One promising approach is the sparse autoencoder (SAE), a deep learning ... JumpReLU makes it easier to identify and track individual features in LLM activations, which can be a step toward ...

This operation of 'capturing the whole from a small part of the firing and finding features' is performed by a 'sparse autoencoder', but the existing sparse autoencoder development method had the ...

including a 16 million feature autoencoder on GPT-4,” OpenAI wrote. So far, they can’t interpret all of GPT-4’s behaviors: “Currently, passing GPT-4’s activations through the sparse ...