Causal Discovery Evaluation Framework in the Absence of Ground-Truth Causal Graph
In causal learning, discovering the causal graph of the underlying generative mechanism from observed data is crucial. However, real-world data for causal discovery is scarce and expensive, leading researchers to rely on synthetic datasets, which may not accurately reflect real-world performance. To...
Saved in:
| Main Authors: | , , , , , |
|---|---|
| Format: | Article |
| Language: | English |
| Published: |
IEEE
2024-01-01
|
| Series: | IEEE Access |
| Subjects: | |
| Online Access: | https://ieeexplore.ieee.org/document/10669554/ |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1841536152457183232 |
|---|---|
| author | Tingpeng Li Lei Wang Danhua Peng Jun Liao Li Liu Zhendong Liu |
| author_facet | Tingpeng Li Lei Wang Danhua Peng Jun Liao Li Liu Zhendong Liu |
| author_sort | Tingpeng Li |
| collection | DOAJ |
| description | In causal learning, discovering the causal graph of the underlying generative mechanism from observed data is crucial. However, real-world data for causal discovery is scarce and expensive, leading researchers to rely on synthetic datasets, which may not accurately reflect real-world performance. To address this, we propose a novel method for evaluating causal discovery algorithms without needing real causal graphs. Specifically, our method employs deep learning evaluation strategies and ensemble learning techniques to robustly assess the performance of causal discovery methods. To elaborate, our approach emulates deep learning validation strategies by dividing the data into training and testing sets. We perform causal discovery on the training set and subsequently use the testing set to conduct Markov blanket tests on the node set and causal direction determination on the edge set. Moreover, we employ multiple ensemble strategies to ensure a comprehensive evaluation of the algorithms. Furthermore, experiments on both synthetic and real datasets demonstrate our method’s effectiveness in accurately and comprehensively validating causal discovery algorithms. Our results show that our proposed method can reflect the performance of causal discovery methods in practice with reasonable error. |
| format | Article |
| id | doaj-art-0d9c60d322634aca864dcaa6e72f88f9 |
| institution | Kabale University |
| issn | 2169-3536 |
| language | English |
| publishDate | 2024-01-01 |
| publisher | IEEE |
| record_format | Article |
| series | IEEE Access |
| spelling | doaj-art-0d9c60d322634aca864dcaa6e72f88f92025-01-15T00:03:37ZengIEEEIEEE Access2169-35362024-01-011213650213651410.1109/ACCESS.2024.345623310669554Causal Discovery Evaluation Framework in the Absence of Ground-Truth Causal GraphTingpeng Li0Lei Wang1https://orcid.org/0009-0009-4538-3387Danhua Peng2Jun Liao3Li Liu4https://orcid.org/0000-0002-4776-5292Zhendong Liu5https://orcid.org/0000-0002-0865-3079State Key Laboratory of Complex Electromagnetic Environmental Effects on Electronics and Information System, Luoyang, ChinaSchool of Big Data and Software Engineering, Chongqing University, Chongqing, ChinaState Key Laboratory of Complex Electromagnetic Environmental Effects on Electronics and Information System, Luoyang, ChinaSchool of Big Data and Software Engineering, Chongqing University, Chongqing, ChinaSchool of Big Data and Software Engineering, Chongqing University, Chongqing, ChinaiFLYTEK Big Data College, Chongqing City Vocational College, Chongqing, ChinaIn causal learning, discovering the causal graph of the underlying generative mechanism from observed data is crucial. However, real-world data for causal discovery is scarce and expensive, leading researchers to rely on synthetic datasets, which may not accurately reflect real-world performance. To address this, we propose a novel method for evaluating causal discovery algorithms without needing real causal graphs. Specifically, our method employs deep learning evaluation strategies and ensemble learning techniques to robustly assess the performance of causal discovery methods. To elaborate, our approach emulates deep learning validation strategies by dividing the data into training and testing sets. We perform causal discovery on the training set and subsequently use the testing set to conduct Markov blanket tests on the node set and causal direction determination on the edge set. Moreover, we employ multiple ensemble strategies to ensure a comprehensive evaluation of the algorithms. Furthermore, experiments on both synthetic and real datasets demonstrate our method’s effectiveness in accurately and comprehensively validating causal discovery algorithms. Our results show that our proposed method can reflect the performance of causal discovery methods in practice with reasonable error.https://ieeexplore.ieee.org/document/10669554/Causal discoveryMarkov blanketcausal graphical modelscause effect identificationcondition independence testing |
| spellingShingle | Tingpeng Li Lei Wang Danhua Peng Jun Liao Li Liu Zhendong Liu Causal Discovery Evaluation Framework in the Absence of Ground-Truth Causal Graph IEEE Access Causal discovery Markov blanket causal graphical models cause effect identification condition independence testing |
| title | Causal Discovery Evaluation Framework in the Absence of Ground-Truth Causal Graph |
| title_full | Causal Discovery Evaluation Framework in the Absence of Ground-Truth Causal Graph |
| title_fullStr | Causal Discovery Evaluation Framework in the Absence of Ground-Truth Causal Graph |
| title_full_unstemmed | Causal Discovery Evaluation Framework in the Absence of Ground-Truth Causal Graph |
| title_short | Causal Discovery Evaluation Framework in the Absence of Ground-Truth Causal Graph |
| title_sort | causal discovery evaluation framework in the absence of ground truth causal graph |
| topic | Causal discovery Markov blanket causal graphical models cause effect identification condition independence testing |
| url | https://ieeexplore.ieee.org/document/10669554/ |
| work_keys_str_mv | AT tingpengli causaldiscoveryevaluationframeworkintheabsenceofgroundtruthcausalgraph AT leiwang causaldiscoveryevaluationframeworkintheabsenceofgroundtruthcausalgraph AT danhuapeng causaldiscoveryevaluationframeworkintheabsenceofgroundtruthcausalgraph AT junliao causaldiscoveryevaluationframeworkintheabsenceofgroundtruthcausalgraph AT liliu causaldiscoveryevaluationframeworkintheabsenceofgroundtruthcausalgraph AT zhendongliu causaldiscoveryevaluationframeworkintheabsenceofgroundtruthcausalgraph |