Interactive Signal Detection Theory explainer

Signal Detection Theory has broad applicability and is a central concept in understanding human performance and the inevitability of “human error”.  Its central concepts also apply to understanding the performance of machine learning algorithms and other decision aids that might augment or replace human detection and decision making.

Staff at airport security checkpoints face a signal detection problem.  They inspect x-ray images to detect guns, water bottles, and other dangerous items.  Signal Detection Theory described their performance in terms of four outcomes:

• Hits–correctly detecting a dangerous item
• Misses–failing to detect a dangerous item
• False Alarms–incorrectly thinking there is a dangerous item when none exists
• Correct Rejection–correctly thinking the luggage is free of dangerous items

Signal Detection Theory explains these outcomes in terms of separating a signal from noise. Two parameters define performance in this situation.  First, d’ is the separation of the noise and and signal+noise distributions. Second is the threshold for detection, beta.  The idea of a noisy signal, separation of distributions, decision thresholds, and their relationship to the four possible outcomes can be hard to understand.  Most challenging is the Receiver Operator Characteristic curve that shows how the changing Beta allows an operator to trade off probability of a Hit with that of a False Alarm.

A interactive Signal Detection Theory explainer helps address this challenge.