Recommended Readings
Most of my blog posts have prerequisites stated at the beginning of the post. If you are unfamiliar with one or another, please refer to the following table for recommended readings.
I only list references that I found helpful, so it is by no means exhaustive nor objective, and you might find better references. If you do, please let me know! I update the table regularly.
| Topic | Light introduction | Deep dive |
|---|---|---|
| A/B testing | Vincent Vanhoucke's blog post on A/B testing | |
| AI alignment problem | See "Friendly AI" | See "Friendly AI" |
| AI safety | See "Friendly AI" | See "Friendly AI" |
| automatic differentiation | Ari Seff's video | |
| Bayesian statistics | MacKay | |
| classical mechanics | Landau and Lifshitz | |
| convex optimization | Intelligent Systems Lab's video | |
| deep implicit layers | The NeurIPS 2020 tutorial | |
| finite factored sets | Scott Garrabant's lecture | |
| friendly AI | Bostrom Eliezer Yudkowski's presentation | Leike et al. Everitt et al. alignmentforum.org * |
| functional decision theory | Yudkowski and Soares | |
| game theory | Leyton-Brown and Shoham * | |
| Gaussian processes | David MacKay's lecture (slides and alternative upload here) | Rasmussen and Williams * |
| information theory | MacKay | |
| integral equations | Wazwaz * | |
| LSTMs | Christopher Olah's blog post | |
| model compression | Sam Sučík's blog post | |
| neural differential equations | Kidger * | |
| neural networks | 3blue1brown’s series of videos | Roberts, Yaida, and Hanin Off the Convex Path blog * |
| neural processes | Garnelo et al. | |
| object-oriented programming | Graham's blog post | |
| reinforcement learning | David Silver's lecture series Berkley deep reinforcement learning lecture series* | Sutton and Barto |
| relativity | Einstein's wonderful nearly-equation-free book | Wald Misner et al. Penrose and Rindler |
| transformers | Dinan, et al. | |
| writing | George Orwell's essay John Wentworth's post on quick and rigorous writing Michael Nielsen's post on Discovery Fiction | Douglas |
Sources with a “*” are those which I have read only partially.
References
1.
Dinan, E., Yaida, S. & Zhang, S. Effective Theory of Transformers at Initialization. Preprint at https://doi.org/10.48550/arXiv.2304.02034 (2023).
1.
Roberts, D. A., Yaida, S. & Hanin, B. The Principles of Deep Learning Theory. (2021).
1.
Wazwaz, A.-M. Linear and nonlinear integral equations: methods and applications. (Higher Education Press, 2011).
1.
Kidger, P. On Neural Differential Equations. arXiv:2202.02435 [cs, math, stat] (2022).
1.
Douglas, Y. The Reader’s Brain: How Neuroscience Can Make You a Better Writer. (Cambridge University Press, 2015).
1.
Wald, R. M. General relativity. (University of Chicago Press, 1984).
1.
Misner, C. W., Thorne, K. S. & Wheeler, J. A. Gravitation. (W. H. Freeman, 1973).
1.
Einstein, A. Relativity : the special and general theory. (2005).
1.
Penrose, R. & Rindler, W. Spinors and space-time. vol. 1 (Cambridge University Press, 1984).
1.
Landau, L. D. & Lifshitz, E. M. Mechanics. (Elsevier, 1982).
1.
Leyton-Brown, K. & Shoham, Y. Essentials of Game Theory: A Concise Multidisciplinary Introduction. Synthesis Lectures on Artificial Intelligence and Machine Learning 2, 1–88 (2008).
1.
Rasmussen, C. E. & Williams, C. K. I. Gaussian processes for machine learning. (MIT Press, 2008).
1.
Everitt, T., Lea, G. & Hutter, M. AGI Safety Literature Review. arXiv:1805.01109 [cs] (2018).
1.
Bostrom, N. Superintelligence: paths, dangers, strategies. (Oxford University Press, 2014).
1.
Leike, J. et al. Scalable agent alignment via reward modeling: a research direction. arXiv:1811.07871 [cs, stat] (2018).
1.
Garnelo, M. et al. Neural Processes. arXiv:1807.01622 [cs, stat] (2018).
1.
Sutton, R. S. & Barto, A. G. Reinforcement Learning: An Introduction.
1.
Yudkowsky, E. & Soares, N. Functional Decision Theory: A New Theory of Instrumental Rationality. arXiv:1710.05060 (2017).
1.
MacKay, D. J. C. Information Theory, Inference, and Learning Algorithms. (Cambridge University Press, 2003).