Text similarity detection method based on NLP
Current text similarity detection methods that ignore document structure information and lack semantic relevance.To solve these problems, a text-oriented similarity detection method was proposed.First, analytic hierarchy process (AHP) was used to calculate word position weight to extract feature wor...
Saved in:
| Main Authors: | , , |
|---|---|
| Format: | Article |
| Language: | zho |
| Published: |
Editorial Department of Journal on Communications
2021-10-01
|
| Series: | Tongxin xuebao |
| Subjects: | |
| Online Access: | http://www.joconline.com.cn/zh/article/doi/10.11959/j.issn.1000-436x.2021192/ |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1841539250506432512 |
|---|---|
| author | Xiaoli DAI Shifeng LIU Daqing GONG |
| author_facet | Xiaoli DAI Shifeng LIU Daqing GONG |
| author_sort | Xiaoli DAI |
| collection | DOAJ |
| description | Current text similarity detection methods that ignore document structure information and lack semantic relevance.To solve these problems, a text-oriented similarity detection method was proposed.First, analytic hierarchy process (AHP) was used to calculate word position weight to extract feature words.Second, the Pearson correlation coefficient was used to measure semantic correlation between words which was the weight of generalized Dice coefficient to calculate similarity.Experimental results show that the proposed method can improve the precision of feature word extraction and the accuracy of similarity calculation results. |
| format | Article |
| id | doaj-art-c5af08ce28ea4aac903af94ee27f40f6 |
| institution | Kabale University |
| issn | 1000-436X |
| language | zho |
| publishDate | 2021-10-01 |
| publisher | Editorial Department of Journal on Communications |
| record_format | Article |
| series | Tongxin xuebao |
| spelling | doaj-art-c5af08ce28ea4aac903af94ee27f40f62025-01-14T07:22:59ZzhoEditorial Department of Journal on CommunicationsTongxin xuebao1000-436X2021-10-014217318159745598Text similarity detection method based on NLPXiaoli DAIShifeng LIUDaqing GONGCurrent text similarity detection methods that ignore document structure information and lack semantic relevance.To solve these problems, a text-oriented similarity detection method was proposed.First, analytic hierarchy process (AHP) was used to calculate word position weight to extract feature words.Second, the Pearson correlation coefficient was used to measure semantic correlation between words which was the weight of generalized Dice coefficient to calculate similarity.Experimental results show that the proposed method can improve the precision of feature word extraction and the accuracy of similarity calculation results.http://www.joconline.com.cn/zh/article/doi/10.11959/j.issn.1000-436x.2021192/text similarityword position weightanalytic hierarchy process,feature word extractionPearson correlation coefficient |
| spellingShingle | Xiaoli DAI Shifeng LIU Daqing GONG Text similarity detection method based on NLP Tongxin xuebao text similarity word position weight analytic hierarchy process, feature word extraction Pearson correlation coefficient |
| title | Text similarity detection method based on NLP |
| title_full | Text similarity detection method based on NLP |
| title_fullStr | Text similarity detection method based on NLP |
| title_full_unstemmed | Text similarity detection method based on NLP |
| title_short | Text similarity detection method based on NLP |
| title_sort | text similarity detection method based on nlp |
| topic | text similarity word position weight analytic hierarchy process, feature word extraction Pearson correlation coefficient |
| url | http://www.joconline.com.cn/zh/article/doi/10.11959/j.issn.1000-436x.2021192/ |
| work_keys_str_mv | AT xiaolidai textsimilaritydetectionmethodbasedonnlp AT shifengliu textsimilaritydetectionmethodbasedonnlp AT daqinggong textsimilaritydetectionmethodbasedonnlp |