This page contains my favorite survey papers and tutorials related to my research.

Video tutorials

NLP Foundations

• UMass CS685 Advanced NLP by Mohit Iyyer (28h 24min)
• CMU Advanced NLP 2021 by Graham Neubig (30h 17min)

Explainability Foundations

• Interpreting Predictions of NLP Models (Wallace et al., 2020) (EMNLP 2020) (4h 27min)
• Interpretability and Analysis in Neural NLP (Belinkov et al., 2020) (ACL 2020) (2h 56min)
• Human-Centered Evaluation of Explanations (Boyd-Graber et al., 2022) (NAACL 2022) (2h 4min)
• Explanations in the Era of Large Language Models (Zhu et al., 2024) (NAACL 2024)

Recommended literature

Explainable NLP

• Methods: Post-hoc Interpretability for Neural NLP: A Survey (Madsen et al., 2022) (ACM CSUR 2022)
• Component Analysis and Mechanistic Interpretability: A Primer on the Inner Workings of Transformer-based Language Models (Ferrando et al., 2024)
• Datasets: Teach Me to Explain: A Review of Datasets for Explainable NLP (Wiegreffe & Marasović, 2021) (NeurIPS 2021)
• Paper summaries: Opinions on Interpretable Machine Learning and 70 Summaries of Recent Papers (Hase & Shen, 2021)
• Social sciences: Explanation in Artificial Intelligence: Insights from the Social Sciences (Miller, 2019) (Artificial Intelligence 2019)
• Probing: Probing Classifiers: Promises, Shortcomings, and Advances (Belinkov, 2021) (CL 2021)

Adjacent areas

• NLG evaluation: Repairing the Cracked Foundation: A Survey of Obstacles in Evaluation Practices for Generated Text (Gehrmann et al., 2022)
• Causal Inference: Causal Inference in Natural Language Processing: Estimation, Prediction, Interpretation and Beyond (Feder et al., 2022) (TACL 2022)
• Question Answering: QA Dataset Explosion: A Taxonomy of NLP Resources for Question Answering and Reading Comprehension (Rogers et al., 2022) (ACM CSUR 2022)
• Prompting: Pre-train, Prompt, and Predict: A Systematic Survey of Prompting Methods in Natural Language Processing (Liu et al., 2022) (ACM CSUR 2022)
• BERTology: A Primer in BERTology: What We Know About How BERT Works (Rogers et al., 2020) (TACL 2020)

Writing

• Tips for the new writer (Choshen, 2022)

Research

• Coming up with Research Ideas (Ribeiro, 2022)
• The Tao of PhD

Tools I love to work with 🧰

PyTorch, Hugging Face datasets + transformers and Captum : My “Explainable NLP toolbox”
PyCharm + Atom : Preferred editors for writing code
Obsidian.md (+ dataview), Zotero and Semantic Scholar (API) : Paper management