In the Engine Room: Transformer Models
摘要
What is the machinery that makes LLMs work? Are there alternatives? The paper Elhage, N., Nanda, N., Olsson, C., Henighan, T., Joseph, N., Mann, B., Askell, A., Bai, Y., Chen, A., Conerly, T., et al. (2021) A mathematical framework for transformer circuits, provides a starting point for this discussion.