Normative conflict resolution through human–autonomous agent interaction
We have become increasingly reliant on the decision-making capabilities of autonomous agents. These decisions are often executed under non-ideal conditions, offer significant moral risk, and directly affect human well-being. Such decisions may involve the choice to optimise one value over another: p...
Saved in:
| Main Authors: | , , , , |
|---|---|
| Format: | Article |
| Language: | English |
| Published: |
Elsevier
2025-03-01
|
| Series: | Journal of Responsible Technology |
| Subjects: | |
| Online Access: | http://www.sciencedirect.com/science/article/pii/S2666659625000101 |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1850027539687276544 |
|---|---|
| author | Beverley Townsend Katie J. Parnell Sinem Getir Yaman Gabriel Nemirovsky Radu Calinescu |
| author_facet | Beverley Townsend Katie J. Parnell Sinem Getir Yaman Gabriel Nemirovsky Radu Calinescu |
| author_sort | Beverley Townsend |
| collection | DOAJ |
| description | We have become increasingly reliant on the decision-making capabilities of autonomous agents. These decisions are often executed under non-ideal conditions, offer significant moral risk, and directly affect human well-being. Such decisions may involve the choice to optimise one value over another: promoting safety over human autonomy, or ensuring accuracy over fairness, for example. All too often decision-making of this kind requires a level of normative evaluation involving ethically defensible moral choices and value judgements, compromises, and trade-offs. Guided by normative principles such decisions inform the possible courses of action the agent may take and may even change a set of established actionable courses.This paper seeks to map the decision-making processes in normative choice scenarios wherein autonomous agents are intrinsically linked to the decision process. A care-robot is used to illustrate how a normative choice - underpinned by normative principles - arises, where the agent must ‘choose’ an actionable path involving the administration of critical or non-critical medication. Critically, the choice is dependent upon the trade-off involving two normative principles: respect for human autonomy and the prevention of harm. An additional dimension is presented, that of the inclusion of the urgency of the medication to be administered, which further informs and changes the course of action to be followed.We offer a means to map decision-making involving a normative choice within a decision ladder using stakeholder input, and, using defeasibility, we show how specification rules with defeaters can be written to operationalise such choice. |
| format | Article |
| id | doaj-art-e92c9fbdc07f410fb41ea16b171c03a6 |
| institution | DOAJ |
| issn | 2666-6596 |
| language | English |
| publishDate | 2025-03-01 |
| publisher | Elsevier |
| record_format | Article |
| series | Journal of Responsible Technology |
| spelling | doaj-art-e92c9fbdc07f410fb41ea16b171c03a62025-08-20T03:00:07ZengElsevierJournal of Responsible Technology2666-65962025-03-012110011410.1016/j.jrt.2025.100114Normative conflict resolution through human–autonomous agent interactionBeverley Townsend0Katie J. Parnell1Sinem Getir Yaman2Gabriel Nemirovsky3Radu Calinescu4University of York, York, England, UKUniversity of Southampton, Southampton, England UK; Corresponding author.University of York, York, England, UKUniversity of York, York, England, UKUniversity of York, York, England, UKWe have become increasingly reliant on the decision-making capabilities of autonomous agents. These decisions are often executed under non-ideal conditions, offer significant moral risk, and directly affect human well-being. Such decisions may involve the choice to optimise one value over another: promoting safety over human autonomy, or ensuring accuracy over fairness, for example. All too often decision-making of this kind requires a level of normative evaluation involving ethically defensible moral choices and value judgements, compromises, and trade-offs. Guided by normative principles such decisions inform the possible courses of action the agent may take and may even change a set of established actionable courses.This paper seeks to map the decision-making processes in normative choice scenarios wherein autonomous agents are intrinsically linked to the decision process. A care-robot is used to illustrate how a normative choice - underpinned by normative principles - arises, where the agent must ‘choose’ an actionable path involving the administration of critical or non-critical medication. Critically, the choice is dependent upon the trade-off involving two normative principles: respect for human autonomy and the prevention of harm. An additional dimension is presented, that of the inclusion of the urgency of the medication to be administered, which further informs and changes the course of action to be followed.We offer a means to map decision-making involving a normative choice within a decision ladder using stakeholder input, and, using defeasibility, we show how specification rules with defeaters can be written to operationalise such choice.http://www.sciencedirect.com/science/article/pii/S2666659625000101Normative conflictAutonomous agentsDecision-makingChoiceDefeasibilityDecision ladder |
| spellingShingle | Beverley Townsend Katie J. Parnell Sinem Getir Yaman Gabriel Nemirovsky Radu Calinescu Normative conflict resolution through human–autonomous agent interaction Journal of Responsible Technology Normative conflict Autonomous agents Decision-making Choice Defeasibility Decision ladder |
| title | Normative conflict resolution through human–autonomous agent interaction |
| title_full | Normative conflict resolution through human–autonomous agent interaction |
| title_fullStr | Normative conflict resolution through human–autonomous agent interaction |
| title_full_unstemmed | Normative conflict resolution through human–autonomous agent interaction |
| title_short | Normative conflict resolution through human–autonomous agent interaction |
| title_sort | normative conflict resolution through human autonomous agent interaction |
| topic | Normative conflict Autonomous agents Decision-making Choice Defeasibility Decision ladder |
| url | http://www.sciencedirect.com/science/article/pii/S2666659625000101 |
| work_keys_str_mv | AT beverleytownsend normativeconflictresolutionthroughhumanautonomousagentinteraction AT katiejparnell normativeconflictresolutionthroughhumanautonomousagentinteraction AT sinemgetiryaman normativeconflictresolutionthroughhumanautonomousagentinteraction AT gabrielnemirovsky normativeconflictresolutionthroughhumanautonomousagentinteraction AT raducalinescu normativeconflictresolutionthroughhumanautonomousagentinteraction |