SD 142 Solution
Signal Detection Theory
Question 1:
Airport security people are responsible for watching carryon bags as they travel through an xray machine. The xray of the bag appears on a video monitor where they inspect it for potentially dangerous items, primarily guns and explosive devices. Their shifts can last four hours.
How would you set this up as a signal detection task.
(1 mark) What is the "signal"?________ the signal is a bag with a bomb in it ______
(1 mark) What is the "noise"?____ noise is bags without bombs in them ___
(8 marks) There are four possible categories of responses. Explain what they are and what they mean in this case.
Marks are given 1 for the category and 1 for the explanation of what it means.
1. Hit – inspector identifies a bag with a bomb in it.
2. Miss – a bag with a bomb in it goes through
3. False Alarm – inspector thinks the bag has a bomb in it and it doesn’t
4. Correct Rejection – bag is ok and inspector lets it through
(4 marks) In terms of the four response categories above, what changes would you expect to find in performance after one hour? Indicate in terms of increases and decreases.
Because the "signals" would presumably be infrequent, after one hour you should assume that these operators are now performing a vigilance task. Their performance will now shift in accordance to vigilance theory, you can assume this is a beta criterion shift to the right on your signal detection graphs. This means that you would expect
1. P(Hit) to decrease
2. P(Miss) to increase
3. P(False Alarm) to decrease
4. P(Correct Rejection) to increase
2x3 Airport inspectors typically have a relatively high false alarm rate compared to other detection type occupations.
Why?
Airport inspectors have a high P(FA) because they need to be risky responders (1). They need to have a very high p(Hit) and very low p(Miss) because a p(Miss) is very costly/dangerous means a plane might crash or be hijacked. (1) A high p(Hit) will cause a high p(FA) (1).
Question 2:
John is working at the Davis Centre Library as a bag inspector. His job is to make sure students don’t take books out of the library that haven’t been signed out. John started his day at 8am. From 8 am to 8:30 am John checked 21 bags and found four library books that hadn’t been signed out. One student managed to get a book out of the library. He stopped two students who looked like they had books in their bags but on a closer inspection did not. From 8:30 to 9:00 am John checked 32 bags and found three books which hadn’t been signed out. He mistakenly stopped one student. Five students managed to get books out of the library without John noticing.
1. Discuss how you would set this up as a signal detection problem. What is the signal? What is the noise?
2. For the two time periods, 8:30-9:00 am and 9:00am to 9:30 am calculate the probability of hits, misses, correct rejections and false alarms.
3. Explain the change in John’s performance. What has happened over the two time periods?
4. Make a plot with the probability of a hit on the x axis and the probability of a false alarm on the y axis. Plot the data from the problem. Draw the line where p(hit) = p(false alarm). Discuss what the distance of the data points from this line indicates.
1. A signal is a bag with a book in it. Noise is a bag without a book in it. 2 marks
2.
At time 1 --
P(hit) = 4/5
P(miss) = 1/5
In calculating P(hit) and P(miss), the denominator is the number of "signal" events. In this case that is the number of bags with books in them. The P(Hit) is the ratio over signals of the number of correctly stopped books. The P(Miss) is the ratio over this of the number of bags with books that were not stopped. Hint: P(Hit)+P(Miss)=1.
P(FA) = 2/16
P(CR)=14/16
At time 2 --
P(hit) = 3/8
p(miss) = 5/8
p(FA) = 1/24
p(CR) = 23/24
Each calculation is worth one mark for a total of 8 marks.
3. P(hit) has decreased. John is probably suffering from vigilance decrement.
2 marks, 1 for mentioning P(Hit), one for the term "vigilance decrement".
4. The graph should look as follows
Axes 1 mark
Scale 0-1 on both axes 1 mark
Line P(FA) =p(hit) correctly drawn, 1 mark
Points1 and 2 correctly placed 2 marks
(graph adds up to 5 marks total when you consider all the elements)
The distance from the line p(hit) = p(false alarm) indicates how easy or difficult it is to discriminate signal from noise. When p(hit) = p(FA) noise and signal are indiscriminable. In this case at time 1 the signal is much more discriminable to John than at time 2, thereby showing his vigilance decrement. This explanation is worth 3 marks.