Causal Parrots: Large Language Models May Talk Causality But Are Not Causal - 2023

Title : Causal Parrots: Large Language Models May Talk Causality But Are Not Causal Author(s): Zečević, Matej and Willig, Moritz and Dhami, Devendra Singh and Kersting, Kristian Link(s) : https://openreview.net/forum?id=tv46tCzs83

This paper defines and shows a new subgroup of SCMs (meta-SCMs) that can encode causal facts about other SCMs in their variables. (#TODO Define variables explicitly in the SCM definition). Conjecture: When LLMs succeed in doing causal inference (#TODO Define what causal inference means here), there is an underlying meta-SCM that shows correlation between causal facts in the training data. Empirical analysis suggests that LLMs are weak causal parrots.

• Starts by assuming a pro-causal AI stance.
• Transformers (thus modern LLMs) are not parametrized SCMs, they do not specify structural equations mimicking causal mechanisms.
• LLMs sometimes answer causal questions correctly.
• LLMs encounter causal facts during training, since there is so much data.
• LLMs can be trained to produce desired content, in this case the right causal answers.
• Figure 1: Nature generates the true SCM, in this case altitude \(A\) causes temperature \(T\). We do not know this apriori, so we would take measurements and try to see if A->T, A<-T, A<->T, A T. Now, that is a causal model, which if known can answer queries which we cannot answer with just observational data. On the other hand, we could have this information encoded in textual format, so if we ask a query like "Does A cause T" or "What is the causal relationship between A and T" it would happen to give the correct answers.
• Being universal does not imply its easy to make LLMs causal (#TODO What makes an LLM causal?).
• CHT: Observational data alone cannot answer causal statements (unless given causal assumptions). This is a wall for the scaling hypothesis. UNLESS: Causal assumptions (needed for causal inference as per CHT) are in the observational data itself.
• SCM definition
• 3 types of causal relationships: Direct causes (direct edge), causes (path), confounded variables (bidirectional edge).
• CHT: Three languages (spaces of queries - observational queries, interventional queries, counterfactual queries).
• Insight 1: Knowledge about structural equations and the causal graph is knowledge about answering L2, L3 queries.
• an ML model is causal wrt some query if it can answer that query with the right L2 fact (#TODO What does that mean). (#TODO Reread sentence just before Insight 2)
• Insight 2: Variables of SCMs doesn't have to be typical measurable concepts, they can be meta-concepts such as causal fact, e.g. knowledge about L2, L3.
• (#TODO Example showing difference between p(y|x) and p(y|do(x)) - namely P(Alt|Temp) neq P(Alt|do(Temp)) )
• An SCM M_meta is meta wrt M if observational distribution of M_meta is enough to answer interventional queries of M.
• #TODO What does a "language" mean in the context of L1,L2,L3.
• "Namely, some machine learning model is ‘causal’ w.r.t. some query if it can answer that query with the right L2 fact." #TODO What does L2 fact mean here?