Primary Investigator:
Jamie Davis

Project Members
Andrew Rozema, Jamie Davis

Phishing remains a critical cybersecurity threat, often lead ing to operational incidents and data breaches. Prior research on the effectiveness of cybersecurity awareness training has yielded mixed results, especially concerning the impact of training on responses to phishing lures of varying diffi culty. This paper presents a large-scale measurement study (N ≈4000) conducted at a US-based international fintech f irm, evaluating the effectiveness of different phishing train ing modalities. We compared a control group (no training), traditional lecture-based training, and the same traditional training augmented with an interactive phishing exercise. We observed statistically significant differences in reporting rates following training. Although lure efficacy—approximated using the NIST Phish Scale—significantly impacted click-through rates, our analy sis indicates that interactive training resulted in a statistically significant improvement in reporting behavior. Specifically, the interactive training group reported phishing attempts 37% more often than the baseline group and 25% more frequently than those receiving traditional training. However, the effect size remains modest. While interactive training does enhance phishing reporting, its impact is limited. This large-scale study contributes by demonstrating the practical utility of the NIST Phish Scale and the limited benefits of interactive training exercises in bolstering organizational defenses against phishing