An interpretive evaluation of entity summarization system
The task of entity summarization (ES) is to select an optimum subset from a large set of triples describing an entity in a knowledge graph.ES systems often integrate many and various ES features in a complex way.While state-of-the-art ES systems have been evaluated and compared by recent benchmarkin...
Saved in:
| Main Authors: | , , |
|---|---|
| Format: | Article |
| Language: | zho |
| Published: |
China InfoCom Media Group
2021-05-01
|
| Series: | 大数据 |
| Subjects: | |
| Online Access: | http://www.j-bigdataresearch.com.cn/thesisDetails#10.11959/j.issn.2096-0271.2021023 |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1850212455302561792 |
|---|---|
| author | Qingxia LIU Junyou LI Gong CHENG |
| author_facet | Qingxia LIU Junyou LI Gong CHENG |
| author_sort | Qingxia LIU |
| collection | DOAJ |
| description | The task of entity summarization (ES) is to select an optimum subset from a large set of triples describing an entity in a knowledge graph.ES systems often integrate many and various ES features in a complex way.While state-of-the-art ES systems have been evaluated and compared by recent benchmarking efforts, it was unclear whether and how much each constituent ES feature had contributed to the performance of an ES system.An interpretive evaluation of ES systems was proposed.Two novel evaluation metrics were proposed, feature effectiveness ratio and feature significance ratio, to characterize how much ground-truth summaries and machine-generated summaries exhibit each ES feature.Their comparison would help to interpret the performance of an ES system.Based on three benchmarks, metrics with six popular ES features were implemented, and an interpretive evaluation of nine unsupervised ES systems and two supervised ES systems were presented.The code and data are open source. |
| format | Article |
| id | doaj-art-4fdda210826e446bba55b5e7a6ea199b |
| institution | OA Journals |
| issn | 2096-0271 |
| language | zho |
| publishDate | 2021-05-01 |
| publisher | China InfoCom Media Group |
| record_format | Article |
| series | 大数据 |
| spelling | doaj-art-4fdda210826e446bba55b5e7a6ea199b2025-08-20T02:09:21ZzhoChina InfoCom Media Group大数据2096-02712021-05-017202102359537830An interpretive evaluation of entity summarization systemQingxia LIUJunyou LIGong CHENGThe task of entity summarization (ES) is to select an optimum subset from a large set of triples describing an entity in a knowledge graph.ES systems often integrate many and various ES features in a complex way.While state-of-the-art ES systems have been evaluated and compared by recent benchmarking efforts, it was unclear whether and how much each constituent ES feature had contributed to the performance of an ES system.An interpretive evaluation of ES systems was proposed.Two novel evaluation metrics were proposed, feature effectiveness ratio and feature significance ratio, to characterize how much ground-truth summaries and machine-generated summaries exhibit each ES feature.Their comparison would help to interpret the performance of an ES system.Based on three benchmarks, metrics with six popular ES features were implemented, and an interpretive evaluation of nine unsupervised ES systems and two supervised ES systems were presented.The code and data are open source.http://www.j-bigdataresearch.com.cn/thesisDetails#10.11959/j.issn.2096-0271.2021023knowledge graph;entity summarization;benchmark |
| spellingShingle | Qingxia LIU Junyou LI Gong CHENG An interpretive evaluation of entity summarization system 大数据 knowledge graph;entity summarization;benchmark |
| title | An interpretive evaluation of entity summarization system |
| title_full | An interpretive evaluation of entity summarization system |
| title_fullStr | An interpretive evaluation of entity summarization system |
| title_full_unstemmed | An interpretive evaluation of entity summarization system |
| title_short | An interpretive evaluation of entity summarization system |
| title_sort | interpretive evaluation of entity summarization system |
| topic | knowledge graph;entity summarization;benchmark |
| url | http://www.j-bigdataresearch.com.cn/thesisDetails#10.11959/j.issn.2096-0271.2021023 |
| work_keys_str_mv | AT qingxialiu aninterpretiveevaluationofentitysummarizationsystem AT junyouli aninterpretiveevaluationofentitysummarizationsystem AT gongcheng aninterpretiveevaluationofentitysummarizationsystem AT qingxialiu interpretiveevaluationofentitysummarizationsystem AT junyouli interpretiveevaluationofentitysummarizationsystem AT gongcheng interpretiveevaluationofentitysummarizationsystem |