Generation of Syntax Parser on South Indian Language using Bottom-Up Parsing Technique and PCFG
In our research, we provide a statistical syntax parsing method experimented on Kannada texts, which is an official language of Karnataka, India. The dataset is downloaded from TDIL website. Using the Cocke-Younger-Kasami (CYK) parsing technique, we generated Kannada Treebank dataset from 1000 annot...
Saved in:
| Main Authors: | , |
|---|---|
| Format: | Article |
| Language: | English |
| Published: |
University of Tehran
2023-01-01
|
| Series: | Journal of Information Technology Management |
| Subjects: | |
| Online Access: | https://jitm.ut.ac.ir/article_89408_c4080ccc6b50e91acff6f3b11641df86.pdf |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1850237847030726656 |
|---|---|
| author | M. Rajani Shree Shambhavi B. R. |
| author_facet | M. Rajani Shree Shambhavi B. R. |
| author_sort | M. Rajani Shree |
| collection | DOAJ |
| description | In our research, we provide a statistical syntax parsing method experimented on Kannada texts, which is an official language of Karnataka, India. The dataset is downloaded from TDIL website. Using the Cocke-Younger-Kasami (CYK) parsing technique, we generated Kannada Treebank dataset from 1000 annotated sentences in the first stage. The Treebank generated in this stage contains 1000 syntactically structured sentences and it is used as input to train the syntax parser model in the second stage. We have adopted Probabilistic Context Free Grammar (PCFG) while training the parser model and extracting the Chmosky Normal Form (CNF) grammar from a Treebank dataset. The developed syntax parser model is tested on 150 raw Kannada sentences. It outputs with the most likely parse tree for each sentence and this is verified with golden Treebank. The syntax parser model generated 74.2% precision, 79.4% recall, and 75.3% F1-score respectively. The similar technique may be adopted for other low resource languages. |
| format | Article |
| id | doaj-art-1d3553019e094664b2a1f73d95042ee3 |
| institution | OA Journals |
| issn | 2008-5893 2423-5059 |
| language | English |
| publishDate | 2023-01-01 |
| publisher | University of Tehran |
| record_format | Article |
| series | Journal of Information Technology Management |
| spelling | doaj-art-1d3553019e094664b2a1f73d95042ee32025-08-20T02:01:39ZengUniversity of TehranJournal of Information Technology Management2008-58932423-50592023-01-0115Special Issue193310.22059/jitm.2022.8940889408Generation of Syntax Parser on South Indian Language using Bottom-Up Parsing Technique and PCFGM. Rajani Shree0Shambhavi B. R.1Research Scholar, Visvesvaraya Technological University, Belagavi, Karnataka, Assistant Professor, BNMIT, Bengaluru, Karnataka.Associate Professor, Department of Information Science and Engineering, BMSCE, Bengaluru, Karnataka.In our research, we provide a statistical syntax parsing method experimented on Kannada texts, which is an official language of Karnataka, India. The dataset is downloaded from TDIL website. Using the Cocke-Younger-Kasami (CYK) parsing technique, we generated Kannada Treebank dataset from 1000 annotated sentences in the first stage. The Treebank generated in this stage contains 1000 syntactically structured sentences and it is used as input to train the syntax parser model in the second stage. We have adopted Probabilistic Context Free Grammar (PCFG) while training the parser model and extracting the Chmosky Normal Form (CNF) grammar from a Treebank dataset. The developed syntax parser model is tested on 150 raw Kannada sentences. It outputs with the most likely parse tree for each sentence and this is verified with golden Treebank. The syntax parser model generated 74.2% precision, 79.4% recall, and 75.3% F1-score respectively. The similar technique may be adopted for other low resource languages.https://jitm.ut.ac.ir/article_89408_c4080ccc6b50e91acff6f3b11641df86.pdfnatural language processingartificial intelligencesyntax parsercyk parsing algorithmprobabilistic context free grammar |
| spellingShingle | M. Rajani Shree Shambhavi B. R. Generation of Syntax Parser on South Indian Language using Bottom-Up Parsing Technique and PCFG Journal of Information Technology Management natural language processing artificial intelligence syntax parser cyk parsing algorithm probabilistic context free grammar |
| title | Generation of Syntax Parser on South Indian Language using Bottom-Up Parsing Technique and PCFG |
| title_full | Generation of Syntax Parser on South Indian Language using Bottom-Up Parsing Technique and PCFG |
| title_fullStr | Generation of Syntax Parser on South Indian Language using Bottom-Up Parsing Technique and PCFG |
| title_full_unstemmed | Generation of Syntax Parser on South Indian Language using Bottom-Up Parsing Technique and PCFG |
| title_short | Generation of Syntax Parser on South Indian Language using Bottom-Up Parsing Technique and PCFG |
| title_sort | generation of syntax parser on south indian language using bottom up parsing technique and pcfg |
| topic | natural language processing artificial intelligence syntax parser cyk parsing algorithm probabilistic context free grammar |
| url | https://jitm.ut.ac.ir/article_89408_c4080ccc6b50e91acff6f3b11641df86.pdf |
| work_keys_str_mv | AT mrajanishree generationofsyntaxparseronsouthindianlanguageusingbottomupparsingtechniqueandpcfg AT shambhavibr generationofsyntaxparseronsouthindianlanguageusingbottomupparsingtechniqueandpcfg |