Readings & resources: Representing and modeling meaning

L5 (September 8, 2026)
back to schedule

• semantle.com
• AI paint colors
• Mikolov, T., Chen, K., Corrado, G., Dean, J. “Efficient Estimation of Word Representations in Vector Space.” 2013.
• “Word Embedding using Word2Vec.” GeeksforGeeks.
• Vaswani, A. et al. “Attention Is All You Need.” NIPS 2017.
• Alammar, J. “The Illustrated Transformer.” June 27, 2018.
• Huang, A., Subramanian, S., Sum, J., Almubarak, K., Biderman, S. “The Annotated Transformer.” v2022; original by Sasha Rush.
• Devlin, J., Chang, M.-W., Lee, K., Toutanova, K. “BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding.” NAACL 2019.
• Radford, A., Narasimhan, K., Salimans, T., Sutskever, I. “Improving Language Understanding by Generative Pre-Training.” OpenAI.
• Liu, A., Hayase, J., Hofmann, V., Oh, S., Smith, N. A., Choi, Y. “SuperBPE: Space travel for language models.” COLM 2025.
• Huang, H., Zhu, D., Wu, B., Zeng, Y., Wang, Y., Min, Q., Zhou, X. “Over-Tokenized Transformer: Vocabulary is Generally Worth Scaling.” 2025.