AI Response Quality in Public Services: Temperature Settings and Contextual Factors
This study investigated how generative Artificial Intelligence (AI) systems—now increasingly integrated into public services—respond to different technical configurations, and how these configurations affect the <i>perceived quality</i> of the outputs. Drawing on an experimental evaluati...
Saved in:
| Main Authors: | , , |
|---|---|
| Format: | Article |
| Language: | English |
| Published: |
MDPI AG
2025-05-01
|
| Series: | Societies |
| Subjects: | |
| Online Access: | https://www.mdpi.com/2075-4698/15/5/127 |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| Summary: | This study investigated how generative Artificial Intelligence (AI) systems—now increasingly integrated into public services—respond to different technical configurations, and how these configurations affect the <i>perceived quality</i> of the outputs. Drawing on an experimental evaluation of <i>Govern-AI</i>, a chatbot designed for professionals in the social, educational, and labor sectors, we analyzed the impact of the <i>temperature</i> parameter—which controls the degree of creativity and variability in the responses—on two key dimensions: <i>accuracy</i> and <i>comprehensibility</i>. This analysis was based on 8880 individual evaluations collected from five professional profiles. The findings revealed the following: (1) the high-temperature responses were generally more comprehensible and appreciated, yet less accurate in strategically sensitive contexts; (2) professional groups differed significantly in their assessments, where trade union representatives and regional policy staff expressed more critical views than the others; (3) the <i>type of question</i>—whether operational or informational—significantly influenced the perceived output quality. This study demonstrated that the AI performance was far from neutral: it depended on technical settings, usage contexts, and the profiles of the end users. Investigating these “behind-the-scenes” dynamics is essential for fostering the <i>informed governance</i> of AI in public services, and for avoiding the risk of technology functioning as an opaque <i>black box</i> within decision-making processes. |
|---|---|
| ISSN: | 2075-4698 |