ABSTRACT

In an increasingly technological environment, security-critical and safety-critical situations often give a system operator information from both human and automated sources simultaneously.

This study was conducted to understand the personal characteristics and situational variables that affect reliance on human or automated decision aids, when the decision aids are in conflict.

Based on the experimental protocol used by Lyons & Stokes1, this study compares two samples, military and civilian. The civilian sample (N = 126, mean age = 19) was taken from an undergraduate population at a large Southeastern university. The military sample (N = 29, mean age = 38) was taken from a group of special operations members enrolled in an education program at a U.S. Army base.

The experimental protocol used the Convoy Leader software platform, tasking participants to choose an optimal convoy route based on provided decision aid information. Participants marked their route decision, which indicated reliance on the automated tool recommendation, the human decision aid recommendation, or their own reasoning (the route not recommended by a decision aid). Participants also reported their perceived risk of the situation and perceived workload (NASA TLX) of the overall task.

Results indicated that there were significant differences among civilian and military samples on reliance measures overall, based on a Chi-Square test (X2 = .028 [2, 155]). To evaluate which proportions were significantly different, three two-sample z tests were conducted. The significance criterion was constrained .0125 to control for family-wise error across three consecutive z tests. A significantly higher proportion of the military sample relied upon themselves (on the non-recommended route) compared to the undergraduate sample (z = 2.5, p = .011). Non-significantly higher proportions of the civilian sample, compared to the military sample, relied on the automated tool (z = 1.6, p = .11) and human decision aid (z = .3, p = .7).

Results also showed differences in situational perceptions. The military sample (M = 3.14, SD = 2.17) perceived significantly less risk in the situation than the civilian sample (M = 4.52, SD = 1.79); t = 3.6, p = .003. The military sample (M = 32.45, SD = 18.17) also experienced significantly less workload than the civilian sample (M = 46.14, SD = 16.03); t = 4.04, p = .001.

The included measures indicate that the military sample experienced the situation differently than the civilian sample and came to different decisions. The background of participants in experimental simulations is vitally important, as it appears different life experiences and specialized training can drastically shift situational perceptions as well as situational outcomes in this context. This should be considered when designing experiments, especially in the population that conclusions are meant to generalize to. Designing decision support systems for both civilian and military populations should consider that the same situation will likely be perceived differently, and the same given information may not lead to the same decision output.

[1] Lyons, J., & Stokes, C. (2011). Human-Human Reliance in the Context of Automation. Human Factors: The Journal of the Human Factors and Ergonomics Society, 112-121.