To further investigate the impact of domain expertise on interaction with automated decision aids, this study assigned individuals with self-reported Spanish proficiency to complete a translation task assisted by two decision aids. This study aimed to examine whether or not previously identified antecedents of trust would be significant in a specific use case of language translation decision aids. We hypothesized that individuals who report higher language proficiency will be more attuned to nuanced differences in automated suggestions and will calibrate trust more appropriately to accuracy than less proficient individuals. We hypothesized that characterization of the decision aids as AI or human and negative attitudes toward automation would impact trust calibration. Participants (n = 58) engaged in a translation task in which they were tasked with submitting the correct English translation of sentences written in Spanish, while receiving suggestions from two aids which varied in correctness. Participants rated the quality of suggestions that were presented to them and rated their confidence in their submitted translations. High proficiency individuals were more attuned to nuanced differences in automated suggestions, opting to write in their own translations rather than taking a suggestion more often than Low proficiency individuals. Task performance did not differ significantly between High and Low proficiency participants. The results suggest that characterization of the decision aids impacted engagement, with confidence and quality ratings being higher on average for the Spanish-Instructor translations than the AI-generated translations. Attitudes toward automation were not found to correlate with confidence ratings or quality ratings.

错误:搜索内容不能为空,请输入英文关键词
错误:关键词超出字数限制,请精简
高级检索

Examining the Influence of Domain Expertise and Other Antecedents of Trust in a Decision-Aid Assisted Language Translation Task

  • Eve Vazquez,
  • Christine Shahan Brugh

摘要

To further investigate the impact of domain expertise on interaction with automated decision aids, this study assigned individuals with self-reported Spanish proficiency to complete a translation task assisted by two decision aids. This study aimed to examine whether or not previously identified antecedents of trust would be significant in a specific use case of language translation decision aids. We hypothesized that individuals who report higher language proficiency will be more attuned to nuanced differences in automated suggestions and will calibrate trust more appropriately to accuracy than less proficient individuals. We hypothesized that characterization of the decision aids as AI or human and negative attitudes toward automation would impact trust calibration. Participants (n = 58) engaged in a translation task in which they were tasked with submitting the correct English translation of sentences written in Spanish, while receiving suggestions from two aids which varied in correctness. Participants rated the quality of suggestions that were presented to them and rated their confidence in their submitted translations. High proficiency individuals were more attuned to nuanced differences in automated suggestions, opting to write in their own translations rather than taking a suggestion more often than Low proficiency individuals. Task performance did not differ significantly between High and Low proficiency participants. The results suggest that characterization of the decision aids impacted engagement, with confidence and quality ratings being higher on average for the Spanish-Instructor translations than the AI-generated translations. Attitudes toward automation were not found to correlate with confidence ratings or quality ratings.