Open Topics
Explaining Knowledge Conflicts and Factual Errors (of Temporal Generalization) in LLM Generations
How can we expose and express knowledge conflicts in LLMs resulting from poor temporal generalization?
[1] DYNAMICQA (Marjanović et al., EMNLP 2024 Findings)
[2] Survey on Factuality Challenges (Augenstein et al., 2023)
[3] Unfaithful Explanations in CoT Prompting (Turpin et al., NeurIPS 2023)
[4] Interventions for Explaining Factual Associations (Geva et al., EMNLP 2023)
[5] Self-Bias in LLMs (Xu et al., 2024)
[6] Mismatches between Token Probabilities and LLM Outputs (Wang et al., ACL 2024 Findings)
[7] Resolving Knowledge Conflicts (Wang et al., COLM 2024)
[8] SAT Probe (Yuksekgonul et al., ICLR 2024)
[9] MONITOR metric (Wang et al., NAACL 2024)
Conversational Model Refinement
- Can we elicit expert human feedback using targeted question generation in a mixed-initiative dialogue setting?
- Can we use human feedback to natural language explanations to improve the model performance and align it to user preferences?
[1] Compositional Explanations (Yao et al., NeurIPS 2021)
[2] Digital Socrates (Gu et al., ACL 2024)
[3] Explanation Formats (Malaviya et al., NAACL 2024)
[4] FeedbackQA (Li et al., ACL 2022 Findings)
[5] Synthesis Step by Step (Wang et al., EMNLP 2023 Findings)