Augmenting Training Data for a Virtual Character Using GPT-3.5
This paper compares different methods of using a large language model (GPT-3.5) for creating synthetic training data for a retrieval-based conversational character. The training data are in the form of linked questions and answers, which allow a classifier to retrieve a pre-recorded answer to an uns...
Saved in:
| Main Authors: | , |
|---|---|
| Format: | Article |
| Language: | English |
| Published: |
LibraryPress@UF
2024-05-01
|
| Series: | Proceedings of the International Florida Artificial Intelligence Research Society Conference |
| Subjects: | |
| Online Access: | https://journals.flvc.org/FLAIRS/article/view/135552 |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1849736777731932160 |
|---|---|
| author | Elizabeth Chen Ron Artstein |
| author_facet | Elizabeth Chen Ron Artstein |
| author_sort | Elizabeth Chen |
| collection | DOAJ |
| description | This paper compares different methods of using a large language model (GPT-3.5) for creating synthetic training data for a retrieval-based conversational character. The training data are in the form of linked questions and answers, which allow a classifier to retrieve a pre-recorded answer to an unseen question; the intuition is that a large language model could predict what human users might ask, thus saving the effort of collecting real user questions as training data. Results show small improvements in test performance for all synthetic datasets. However, a classifier trained on only small amounts of collected user data resulted in a higher F-score than the classifiers trained on much larger amounts of synthetic data generated using GPT-3.5. Based on these results, we see a potential in using large language models for generating training data, but at this point it is not as valuable as collecting actual user data for training. |
| format | Article |
| id | doaj-art-c2c758a7e70c4fb291e674494a9cb971 |
| institution | DOAJ |
| issn | 2334-0754 2334-0762 |
| language | English |
| publishDate | 2024-05-01 |
| publisher | LibraryPress@UF |
| record_format | Article |
| series | Proceedings of the International Florida Artificial Intelligence Research Society Conference |
| spelling | doaj-art-c2c758a7e70c4fb291e674494a9cb9712025-08-20T03:07:10ZengLibraryPress@UFProceedings of the International Florida Artificial Intelligence Research Society Conference2334-07542334-07622024-05-013710.32473/flairs.37.1.13555271931Augmenting Training Data for a Virtual Character Using GPT-3.5Elizabeth Chen0https://orcid.org/0009-0006-7188-6859Ron Artstein1https://orcid.org/0009-0005-5187-6381Wellesley CollegeInstitute for Creative Technologies, University of Southern CaliforniaThis paper compares different methods of using a large language model (GPT-3.5) for creating synthetic training data for a retrieval-based conversational character. The training data are in the form of linked questions and answers, which allow a classifier to retrieve a pre-recorded answer to an unseen question; the intuition is that a large language model could predict what human users might ask, thus saving the effort of collecting real user questions as training data. Results show small improvements in test performance for all synthetic datasets. However, a classifier trained on only small amounts of collected user data resulted in a higher F-score than the classifiers trained on much larger amounts of synthetic data generated using GPT-3.5. Based on these results, we see a potential in using large language models for generating training data, but at this point it is not as valuable as collecting actual user data for training.https://journals.flvc.org/FLAIRS/article/view/135552large language modelssynthetic datadialog based systems |
| spellingShingle | Elizabeth Chen Ron Artstein Augmenting Training Data for a Virtual Character Using GPT-3.5 Proceedings of the International Florida Artificial Intelligence Research Society Conference large language models synthetic data dialog based systems |
| title | Augmenting Training Data for a Virtual Character Using GPT-3.5 |
| title_full | Augmenting Training Data for a Virtual Character Using GPT-3.5 |
| title_fullStr | Augmenting Training Data for a Virtual Character Using GPT-3.5 |
| title_full_unstemmed | Augmenting Training Data for a Virtual Character Using GPT-3.5 |
| title_short | Augmenting Training Data for a Virtual Character Using GPT-3.5 |
| title_sort | augmenting training data for a virtual character using gpt 3 5 |
| topic | large language models synthetic data dialog based systems |
| url | https://journals.flvc.org/FLAIRS/article/view/135552 |
| work_keys_str_mv | AT elizabethchen augmentingtrainingdataforavirtualcharacterusinggpt35 AT ronartstein augmentingtrainingdataforavirtualcharacterusinggpt35 |