Why AI black boxes matter In many cases, there is good reason to be wary of black box machine-learning algorithms and models. Suppose a machine-learning model has made a diagnosis about your health.

All in all, not all advanced machine learning models are black box, and for most applications, a degree of explainability is sufficient to meet legal and regulatory requirements.

Anthropic’s team isn’t the only one working to crack open the black box of LLMs. There’s a group at DeepMind also working on the problem, run by a researcher who used to work with Olah.