Research and application of defense mechanism for prompt injection attack of large language model in financial industry
The large language models had a broad application prospect in the financial sector, and they were expected to play an important role in both asset management and wealth management. With the rapid development and wide application of large language models such as ChatGPT and GPT-4, attention to the se...
Saved in:
| Main Authors: | , , , , |
|---|---|
| Format: | Article |
| Language: | English |
| Published: |
POSTS&TELECOM PRESS Co., LTD
2024-10-01
|
| Series: | 网络与信息安全学报 |
| Subjects: | |
| Online Access: | http://www.cjnis.com.cn/thesisDetails#10.11959/j.issn.2096-109x.2024071 |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| Summary: | The large language models had a broad application prospect in the financial sector, and they were expected to play an important role in both asset management and wealth management. With the rapid development and wide application of large language models such as ChatGPT and GPT-4, attention to the security of large language models increased. The financial industry, characterized by strict regulations and supervision, demanded heightened security measures. Consequently, a comprehensive study on prompt injection attacks and a security defense framework was delved into in large language models within the financial sector. A risk taxonomy encompassing eight forms of input prompt injection attacks and five categories of safety scenarios on the output side was developed, and a financial domain large model prompt injection attack dataset, FIN-CSAPrompts, was collected using a human-machine adversarial approach. An end-to-end security defense framework against prompt injection attacks in large language models was proposed and tested, and comparative evaluations were performed using prevalent open-source large language models. The research indicated that in the financial industry, the application of the proposed security defense framework significantly enhanced the defensive capabilities of Chinese large language models, effectively reducing the generation of inappropriate content and improving their resilience against prompt injection attacks. This research provided a reference and foundation for further research on the security issues of Chinese large language models in the financial domain, offering datasets, evaluation metrics, and solutions for consideration and adaptation. |
|---|---|
| ISSN: | 2096-109X |