acm distinguished & tedx speaker linkedin/in/ponguru...
TRANSCRIPT
![Page 1: ACM Distinguished & TEDx Speaker Linkedin/in/ponguru ...precog.iiitd.edu.in/hcionnptel/2018/slides/HCIonNPTEL-Spring-2018-… · Linkedin/in/ponguru/ fb/ponnurangam.kumaraguru, @ponguru](https://reader033.vdocuments.net/reader033/viewer/2022042917/5f5a110cf1ef9a56ee125c72/html5/thumbnails/1.jpg)
Introduction to Human Computer Interaction
Course on NPTEL, Spring 2018Week 7
Usable Security
Ponnurangam Kumaraguru (“PK”)Associate Professor
ACM Distinguished & TEDx SpeakerLinkedin/in/ponguru/
fb/ponnurangam.kumaraguru, @ponguru 1
![Page 2: ACM Distinguished & TEDx Speaker Linkedin/in/ponguru ...precog.iiitd.edu.in/hcionnptel/2018/slides/HCIonNPTEL-Spring-2018-… · Linkedin/in/ponguru/ fb/ponnurangam.kumaraguru, @ponguru](https://reader033.vdocuments.net/reader033/viewer/2022042917/5f5a110cf1ef9a56ee125c72/html5/thumbnails/2.jpg)
Usability and Security
●Why should we study this?●Why is it important? ●Any experience / relationship?
2
![Page 3: ACM Distinguished & TEDx Speaker Linkedin/in/ponguru ...precog.iiitd.edu.in/hcionnptel/2018/slides/HCIonNPTEL-Spring-2018-… · Linkedin/in/ponguru/ fb/ponnurangam.kumaraguru, @ponguru](https://reader033.vdocuments.net/reader033/viewer/2022042917/5f5a110cf1ef9a56ee125c72/html5/thumbnails/3.jpg)
Everyday Security ProblemsSetting File Permissions
3
![Page 4: ACM Distinguished & TEDx Speaker Linkedin/in/ponguru ...precog.iiitd.edu.in/hcionnptel/2018/slides/HCIonNPTEL-Spring-2018-… · Linkedin/in/ponguru/ fb/ponnurangam.kumaraguru, @ponguru](https://reader033.vdocuments.net/reader033/viewer/2022042917/5f5a110cf1ef9a56ee125c72/html5/thumbnails/4.jpg)
Secure, but usable?
![Page 5: ACM Distinguished & TEDx Speaker Linkedin/in/ponguru ...precog.iiitd.edu.in/hcionnptel/2018/slides/HCIonNPTEL-Spring-2018-… · Linkedin/in/ponguru/ fb/ponnurangam.kumaraguru, @ponguru](https://reader033.vdocuments.net/reader033/viewer/2022042917/5f5a110cf1ef9a56ee125c72/html5/thumbnails/5.jpg)
5
Unusable security frustrates users
![Page 6: ACM Distinguished & TEDx Speaker Linkedin/in/ponguru ...precog.iiitd.edu.in/hcionnptel/2018/slides/HCIonNPTEL-Spring-2018-… · Linkedin/in/ponguru/ fb/ponnurangam.kumaraguru, @ponguru](https://reader033.vdocuments.net/reader033/viewer/2022042917/5f5a110cf1ef9a56ee125c72/html5/thumbnails/6.jpg)
Usable Privacy and Security“Give end-users security controls they can understand and privacy they can control for the dynamic, pervasive computing environments of the future.”
- Grand Challenges in Information Security & Assurance Computing Research Association (2003)
More research needed on how “cultural and social influences can affect how people use computers and electronic information in ways that increase the risk of cybersecurity breaches.”
- Grand Challenges for Engineering National Academy of Engineering (2008)
6
![Page 7: ACM Distinguished & TEDx Speaker Linkedin/in/ponguru ...precog.iiitd.edu.in/hcionnptel/2018/slides/HCIonNPTEL-Spring-2018-… · Linkedin/in/ponguru/ fb/ponnurangam.kumaraguru, @ponguru](https://reader033.vdocuments.net/reader033/viewer/2022042917/5f5a110cf1ef9a56ee125c72/html5/thumbnails/7.jpg)
Humans are weakest link● Most security breaches attributed to “human error”● Social engineering attacks proliferate
7
![Page 8: ACM Distinguished & TEDx Speaker Linkedin/in/ponguru ...precog.iiitd.edu.in/hcionnptel/2018/slides/HCIonNPTEL-Spring-2018-… · Linkedin/in/ponguru/ fb/ponnurangam.kumaraguru, @ponguru](https://reader033.vdocuments.net/reader033/viewer/2022042917/5f5a110cf1ef9a56ee125c72/html5/thumbnails/8.jpg)
How can we make secure systems more usable?
● Make it “just work”● Invisible security
● Make security/privacy understandable● Make it visible● Make it intuitive● Use metaphors that users can relate to
● Train the user
8
![Page 9: ACM Distinguished & TEDx Speaker Linkedin/in/ponguru ...precog.iiitd.edu.in/hcionnptel/2018/slides/HCIonNPTEL-Spring-2018-… · Linkedin/in/ponguru/ fb/ponnurangam.kumaraguru, @ponguru](https://reader033.vdocuments.net/reader033/viewer/2022042917/5f5a110cf1ef9a56ee125c72/html5/thumbnails/9.jpg)
Concerns may not be aligned
Security
Expert User
Keep the bad guys out
Don’t lock me out!
![Page 10: ACM Distinguished & TEDx Speaker Linkedin/in/ponguru ...precog.iiitd.edu.in/hcionnptel/2018/slides/HCIonNPTEL-Spring-2018-… · Linkedin/in/ponguru/ fb/ponnurangam.kumaraguru, @ponguru](https://reader033.vdocuments.net/reader033/viewer/2022042917/5f5a110cf1ef9a56ee125c72/html5/thumbnails/10.jpg)
Grey
●Smartphone based access-control system
●Used to open doors in the Carnegie Mellon CIC building
●Allows users to grant access to their doors remotely
L. Bauer, L.F. Cranor, R.W. Reeder, M.K. Reiter, and K. Vaniea. A User Study of Policy Creation in a Flexible Access-Control System. CHI 2008. http://www.robreeder.com/pubs/greyCHI2008.pdf
L. Bauer, L. F. Cranor, M. K. Reiter, and K. Vaniea. Lessons Learned from the Deployment of a Smartphone-Based Access-Control System. SOUPS 2007. http://cups.cs.cmu.edu/soups/2007/proceedings/p64_bauer.pdf
![Page 11: ACM Distinguished & TEDx Speaker Linkedin/in/ponguru ...precog.iiitd.edu.in/hcionnptel/2018/slides/HCIonNPTEL-Spring-2018-… · Linkedin/in/ponguru/ fb/ponnurangam.kumaraguru, @ponguru](https://reader033.vdocuments.net/reader033/viewer/2022042917/5f5a110cf1ef9a56ee125c72/html5/thumbnails/11.jpg)
Data collection
● Year long interview study● Recorded 30 hours of
interviews with Grey users● System was actively used: 29
users x 12 access per week
![Page 12: ACM Distinguished & TEDx Speaker Linkedin/in/ponguru ...precog.iiitd.edu.in/hcionnptel/2018/slides/HCIonNPTEL-Spring-2018-… · Linkedin/in/ponguru/ fb/ponnurangam.kumaraguru, @ponguru](https://reader033.vdocuments.net/reader033/viewer/2022042917/5f5a110cf1ef9a56ee125c72/html5/thumbnails/12.jpg)
Users complained about speed
● Users said Grey was slow● But Grey was as fast as keys● Videotaped a door to better
understand how doors are opened differently with Grey and keys
![Page 13: ACM Distinguished & TEDx Speaker Linkedin/in/ponguru ...precog.iiitd.edu.in/hcionnptel/2018/slides/HCIonNPTEL-Spring-2018-… · Linkedin/in/ponguru/ fb/ponnurangam.kumaraguru, @ponguru](https://reader033.vdocuments.net/reader033/viewer/2022042917/5f5a110cf1ef9a56ee125c72/html5/thumbnails/13.jpg)
13
“I find myself standing outside and everybody inside is looking at me standing outside while I am trying to futz with my phone and open the stupid door.”
![Page 14: ACM Distinguished & TEDx Speaker Linkedin/in/ponguru ...precog.iiitd.edu.in/hcionnptel/2018/slides/HCIonNPTEL-Spring-2018-… · Linkedin/in/ponguru/ fb/ponnurangam.kumaraguru, @ponguru](https://reader033.vdocuments.net/reader033/viewer/2022042917/5f5a110cf1ef9a56ee125c72/html5/thumbnails/14.jpg)
Train the user
![Page 15: ACM Distinguished & TEDx Speaker Linkedin/in/ponguru ...precog.iiitd.edu.in/hcionnptel/2018/slides/HCIonNPTEL-Spring-2018-… · Linkedin/in/ponguru/ fb/ponnurangam.kumaraguru, @ponguru](https://reader033.vdocuments.net/reader033/viewer/2022042917/5f5a110cf1ef9a56ee125c72/html5/thumbnails/15.jpg)
Why do humans fall for phish?● Not motivated to pay attention to training
● “Security is not my problem”
● Mental models inconsistent with reality● “If site looks professional it must be legitimate”
● Need actionable advice they can understand● Difficult to be alert if you don’t know what you’re looking for
![Page 16: ACM Distinguished & TEDx Speaker Linkedin/in/ponguru ...precog.iiitd.edu.in/hcionnptel/2018/slides/HCIonNPTEL-Spring-2018-… · Linkedin/in/ponguru/ fb/ponnurangam.kumaraguru, @ponguru](https://reader033.vdocuments.net/reader033/viewer/2022042917/5f5a110cf1ef9a56ee125c72/html5/thumbnails/16.jpg)
How do we get people trained?
Learning science principles +
Teachable moments +
Fun
P. Kumaraguru, S. Sheng, A. Acquisti, L. Cranor, and J. Hong. Teaching Johnny Not to Fall for Phish. ACM Trans. Internet Technol. 10, 2 (May 2010), 1-31.
![Page 17: ACM Distinguished & TEDx Speaker Linkedin/in/ponguru ...precog.iiitd.edu.in/hcionnptel/2018/slides/HCIonNPTEL-Spring-2018-… · Linkedin/in/ponguru/ fb/ponnurangam.kumaraguru, @ponguru](https://reader033.vdocuments.net/reader033/viewer/2022042917/5f5a110cf1ef9a56ee125c72/html5/thumbnails/17.jpg)
PhishGuru embedded training● Send email that look like phish● If recipient falls for it, train in succinct and engaging format● Study demonstrated effectiveness of PhishGuru and found that same training was
not effective sent as regular email
Learning science principles
+Teachable moments
+Fun
![Page 18: ACM Distinguished & TEDx Speaker Linkedin/in/ponguru ...precog.iiitd.edu.in/hcionnptel/2018/slides/HCIonNPTEL-Spring-2018-… · Linkedin/in/ponguru/ fb/ponnurangam.kumaraguru, @ponguru](https://reader033.vdocuments.net/reader033/viewer/2022042917/5f5a110cf1ef9a56ee125c72/html5/thumbnails/18.jpg)
Design rationale ● Paper and HTML prototypes● One page constraint● Analyzed instructions from most popular websites● Present the training materials when users click on the link
![Page 19: ACM Distinguished & TEDx Speaker Linkedin/in/ponguru ...precog.iiitd.edu.in/hcionnptel/2018/slides/HCIonNPTEL-Spring-2018-… · Linkedin/in/ponguru/ fb/ponnurangam.kumaraguru, @ponguru](https://reader033.vdocuments.net/reader033/viewer/2022042917/5f5a110cf1ef9a56ee125c72/html5/thumbnails/19.jpg)
Applies learning-by-doing and immediate feedback principles
![Page 20: ACM Distinguished & TEDx Speaker Linkedin/in/ponguru ...precog.iiitd.edu.in/hcionnptel/2018/slides/HCIonNPTEL-Spring-2018-… · Linkedin/in/ponguru/ fb/ponnurangam.kumaraguru, @ponguru](https://reader033.vdocuments.net/reader033/viewer/2022042917/5f5a110cf1ef9a56ee125c72/html5/thumbnails/20.jpg)
Applies story-based agent principle
![Page 21: ACM Distinguished & TEDx Speaker Linkedin/in/ponguru ...precog.iiitd.edu.in/hcionnptel/2018/slides/HCIonNPTEL-Spring-2018-… · Linkedin/in/ponguru/ fb/ponnurangam.kumaraguru, @ponguru](https://reader033.vdocuments.net/reader033/viewer/2022042917/5f5a110cf1ef9a56ee125c72/html5/thumbnails/21.jpg)
Applies contiguity principlePresents procedural knowledge
![Page 22: ACM Distinguished & TEDx Speaker Linkedin/in/ponguru ...precog.iiitd.edu.in/hcionnptel/2018/slides/HCIonNPTEL-Spring-2018-… · Linkedin/in/ponguru/ fb/ponnurangam.kumaraguru, @ponguru](https://reader033.vdocuments.net/reader033/viewer/2022042917/5f5a110cf1ef9a56ee125c72/html5/thumbnails/22.jpg)
Applies personalization principlePresents conceptual knowledge
![Page 23: ACM Distinguished & TEDx Speaker Linkedin/in/ponguru ...precog.iiitd.edu.in/hcionnptel/2018/slides/HCIonNPTEL-Spring-2018-… · Linkedin/in/ponguru/ fb/ponnurangam.kumaraguru, @ponguru](https://reader033.vdocuments.net/reader033/viewer/2022042917/5f5a110cf1ef9a56ee125c72/html5/thumbnails/23.jpg)
![Page 24: ACM Distinguished & TEDx Speaker Linkedin/in/ponguru ...precog.iiitd.edu.in/hcionnptel/2018/slides/HCIonNPTEL-Spring-2018-… · Linkedin/in/ponguru/ fb/ponnurangam.kumaraguru, @ponguru](https://reader033.vdocuments.net/reader033/viewer/2022042917/5f5a110cf1ef9a56ee125c72/html5/thumbnails/24.jpg)
Iterations
![Page 25: ACM Distinguished & TEDx Speaker Linkedin/in/ponguru ...precog.iiitd.edu.in/hcionnptel/2018/slides/HCIonNPTEL-Spring-2018-… · Linkedin/in/ponguru/ fb/ponnurangam.kumaraguru, @ponguru](https://reader033.vdocuments.net/reader033/viewer/2022042917/5f5a110cf1ef9a56ee125c72/html5/thumbnails/25.jpg)
First intervention
![Page 26: ACM Distinguished & TEDx Speaker Linkedin/in/ponguru ...precog.iiitd.edu.in/hcionnptel/2018/slides/HCIonNPTEL-Spring-2018-… · Linkedin/in/ponguru/ fb/ponnurangam.kumaraguru, @ponguru](https://reader033.vdocuments.net/reader033/viewer/2022042917/5f5a110cf1ef9a56ee125c72/html5/thumbnails/26.jpg)
Intervention: eBay
![Page 27: ACM Distinguished & TEDx Speaker Linkedin/in/ponguru ...precog.iiitd.edu.in/hcionnptel/2018/slides/HCIonNPTEL-Spring-2018-… · Linkedin/in/ponguru/ fb/ponnurangam.kumaraguru, @ponguru](https://reader033.vdocuments.net/reader033/viewer/2022042917/5f5a110cf1ef9a56ee125c72/html5/thumbnails/27.jpg)
![Page 28: ACM Distinguished & TEDx Speaker Linkedin/in/ponguru ...precog.iiitd.edu.in/hcionnptel/2018/slides/HCIonNPTEL-Spring-2018-… · Linkedin/in/ponguru/ fb/ponnurangam.kumaraguru, @ponguru](https://reader033.vdocuments.net/reader033/viewer/2022042917/5f5a110cf1ef9a56ee125c72/html5/thumbnails/28.jpg)
![Page 29: ACM Distinguished & TEDx Speaker Linkedin/in/ponguru ...precog.iiitd.edu.in/hcionnptel/2018/slides/HCIonNPTEL-Spring-2018-… · Linkedin/in/ponguru/ fb/ponnurangam.kumaraguru, @ponguru](https://reader033.vdocuments.net/reader033/viewer/2022042917/5f5a110cf1ef9a56ee125c72/html5/thumbnails/29.jpg)
![Page 30: ACM Distinguished & TEDx Speaker Linkedin/in/ponguru ...precog.iiitd.edu.in/hcionnptel/2018/slides/HCIonNPTEL-Spring-2018-… · Linkedin/in/ponguru/ fb/ponnurangam.kumaraguru, @ponguru](https://reader033.vdocuments.net/reader033/viewer/2022042917/5f5a110cf1ef9a56ee125c72/html5/thumbnails/30.jpg)
![Page 31: ACM Distinguished & TEDx Speaker Linkedin/in/ponguru ...precog.iiitd.edu.in/hcionnptel/2018/slides/HCIonNPTEL-Spring-2018-… · Linkedin/in/ponguru/ fb/ponnurangam.kumaraguru, @ponguru](https://reader033.vdocuments.net/reader033/viewer/2022042917/5f5a110cf1ef9a56ee125c72/html5/thumbnails/31.jpg)
![Page 32: ACM Distinguished & TEDx Speaker Linkedin/in/ponguru ...precog.iiitd.edu.in/hcionnptel/2018/slides/HCIonNPTEL-Spring-2018-… · Linkedin/in/ponguru/ fb/ponnurangam.kumaraguru, @ponguru](https://reader033.vdocuments.net/reader033/viewer/2022042917/5f5a110cf1ef9a56ee125c72/html5/thumbnails/32.jpg)
Focus group studies● One with age group 18 – 55 and another with age group greater than
65● All age groups will read the interventions ● Everybody liked the gold fish and the comic script format● Participants did not like the phisher character
![Page 33: ACM Distinguished & TEDx Speaker Linkedin/in/ponguru ...precog.iiitd.edu.in/hcionnptel/2018/slides/HCIonNPTEL-Spring-2018-… · Linkedin/in/ponguru/ fb/ponnurangam.kumaraguru, @ponguru](https://reader033.vdocuments.net/reader033/viewer/2022042917/5f5a110cf1ef9a56ee125c72/html5/thumbnails/33.jpg)
![Page 34: ACM Distinguished & TEDx Speaker Linkedin/in/ponguru ...precog.iiitd.edu.in/hcionnptel/2018/slides/HCIonNPTEL-Spring-2018-… · Linkedin/in/ponguru/ fb/ponnurangam.kumaraguru, @ponguru](https://reader033.vdocuments.net/reader033/viewer/2022042917/5f5a110cf1ef9a56ee125c72/html5/thumbnails/34.jpg)
![Page 35: ACM Distinguished & TEDx Speaker Linkedin/in/ponguru ...precog.iiitd.edu.in/hcionnptel/2018/slides/HCIonNPTEL-Spring-2018-… · Linkedin/in/ponguru/ fb/ponnurangam.kumaraguru, @ponguru](https://reader033.vdocuments.net/reader033/viewer/2022042917/5f5a110cf1ef9a56ee125c72/html5/thumbnails/35.jpg)
First lab study results
● Security notices are an ineffective medium for training users
● Users educated with embedded training make better decisions than those sent security notices
Kumaraguru, P., Rhee, Y., Acquisti, A., Cranor, L. F., Hong, J., and Nunge, E. Protecting people from phishing: the design and evaluation of an embedded training email system. CHI ’07, pp. 905-914.
![Page 36: ACM Distinguished & TEDx Speaker Linkedin/in/ponguru ...precog.iiitd.edu.in/hcionnptel/2018/slides/HCIonNPTEL-Spring-2018-… · Linkedin/in/ponguru/ fb/ponnurangam.kumaraguru, @ponguru](https://reader033.vdocuments.net/reader033/viewer/2022042917/5f5a110cf1ef9a56ee125c72/html5/thumbnails/36.jpg)
Second lab study results
● Users educated with PhishGuru retained knowledge after seven days
● Users trained with embedded did better than users trained with non-embedded
Kumaraguru, P., Rhee, Y., Sheng, S., Hasan, S., Acquisti, A., Cranor, L. F., and Hong, J. Getting users to pay attention to anti-phishing education: Evaluation of retention and transfer. e-Crime Researchers Summit, Anti-Phishing Working Group (2007).
![Page 37: ACM Distinguished & TEDx Speaker Linkedin/in/ponguru ...precog.iiitd.edu.in/hcionnptel/2018/slides/HCIonNPTEL-Spring-2018-… · Linkedin/in/ponguru/ fb/ponnurangam.kumaraguru, @ponguru](https://reader033.vdocuments.net/reader033/viewer/2022042917/5f5a110cf1ef9a56ee125c72/html5/thumbnails/37.jpg)
Real world study: Portuguese ISP
● PhishGuru is effective in training people in the real world
● Trained participants retained knowledge after 7 days of training
Kumaraguru, P., Sheng, S., Acquisti, A., Cranor, L. F., and Hong, J. Lessons from a real world evaluation of anti-phishing training. e-Crime Researchers Summit, 2008
![Page 38: ACM Distinguished & TEDx Speaker Linkedin/in/ponguru ...precog.iiitd.edu.in/hcionnptel/2018/slides/HCIonNPTEL-Spring-2018-… · Linkedin/in/ponguru/ fb/ponnurangam.kumaraguru, @ponguru](https://reader033.vdocuments.net/reader033/viewer/2022042917/5f5a110cf1ef9a56ee125c72/html5/thumbnails/38.jpg)
Real world study: CMU● Evaluate effectiveness of PhishGuru training in the real world● Investigate retention after 1 week, 2 weeks, and 4 weeks ● Compare effectiveness of 2 training messages with effectiveness of 1
training message
P. Kumaraguru, J. Cranshaw, A. Acquisti, L. Cranor, J. Hong, M. A. Blair, and T. Pham. School of Phish: A Real-World Evaluation of Anti-Phishing Training. 2009. Under review.
![Page 39: ACM Distinguished & TEDx Speaker Linkedin/in/ponguru ...precog.iiitd.edu.in/hcionnptel/2018/slides/HCIonNPTEL-Spring-2018-… · Linkedin/in/ponguru/ fb/ponnurangam.kumaraguru, @ponguru](https://reader033.vdocuments.net/reader033/viewer/2022042917/5f5a110cf1ef9a56ee125c72/html5/thumbnails/39.jpg)
Study design
● Sent email to all CMU students, faculty and staff to recruit participants to opt-in to study
● 515 participants in three conditions ● Control ● One training message ● Two training messages
● Emails sent over 28 day period● 7 simulated spear-phishing messages● 3 legitimate messages from ISO (cyber security
scavenger hunt)● Exit survey
![Page 40: ACM Distinguished & TEDx Speaker Linkedin/in/ponguru ...precog.iiitd.edu.in/hcionnptel/2018/slides/HCIonNPTEL-Spring-2018-… · Linkedin/in/ponguru/ fb/ponnurangam.kumaraguru, @ponguru](https://reader033.vdocuments.net/reader033/viewer/2022042917/5f5a110cf1ef9a56ee125c72/html5/thumbnails/40.jpg)
What study design? ● For 2 different solutions – PhishGuru & PhishX
40
![Page 41: ACM Distinguished & TEDx Speaker Linkedin/in/ponguru ...precog.iiitd.edu.in/hcionnptel/2018/slides/HCIonNPTEL-Spring-2018-… · Linkedin/in/ponguru/ fb/ponnurangam.kumaraguru, @ponguru](https://reader033.vdocuments.net/reader033/viewer/2022042917/5f5a110cf1ef9a56ee125c72/html5/thumbnails/41.jpg)
Comparing Two Alternatives
● Between groups experiment● two groups of test users● each group uses only 1 of the systems
● Within groups experiment● one group of test users
● each person uses both systems,randomized ordering
● can’t use the same tasks or order (learning)
● Between groups requires many more participants than within groups
41
![Page 42: ACM Distinguished & TEDx Speaker Linkedin/in/ponguru ...precog.iiitd.edu.in/hcionnptel/2018/slides/HCIonNPTEL-Spring-2018-… · Linkedin/in/ponguru/ fb/ponnurangam.kumaraguru, @ponguru](https://reader033.vdocuments.net/reader033/viewer/2022042917/5f5a110cf1ef9a56ee125c72/html5/thumbnails/42.jpg)
Implementation ● Unique hash in the URL for each participant● Demographic and department/status data linked to each hash ● Form does not POST login details● Campus help desks and all spoofed departments were notified before
messages were sent
![Page 43: ACM Distinguished & TEDx Speaker Linkedin/in/ponguru ...precog.iiitd.edu.in/hcionnptel/2018/slides/HCIonNPTEL-Spring-2018-… · Linkedin/in/ponguru/ fb/ponnurangam.kumaraguru, @ponguru](https://reader033.vdocuments.net/reader033/viewer/2022042917/5f5a110cf1ef9a56ee125c72/html5/thumbnails/43.jpg)
Study schedule
Day of the study
Control One training message
Two training messages
Day 0 Test and real Train and real Train and real
Day 2 Test Day 7 Test and real
Day 14 Test Test Train Day 16 TestDay 21 Test Day 28 Test and real Day 35 Post-study survey
![Page 44: ACM Distinguished & TEDx Speaker Linkedin/in/ponguru ...precog.iiitd.edu.in/hcionnptel/2018/slides/HCIonNPTEL-Spring-2018-… · Linkedin/in/ponguru/ fb/ponnurangam.kumaraguru, @ponguru](https://reader033.vdocuments.net/reader033/viewer/2022042917/5f5a110cf1ef9a56ee125c72/html5/thumbnails/44.jpg)
Simulated spear phishing message
URL is not hidden
Plain text email without graphics
![Page 45: ACM Distinguished & TEDx Speaker Linkedin/in/ponguru ...precog.iiitd.edu.in/hcionnptel/2018/slides/HCIonNPTEL-Spring-2018-… · Linkedin/in/ponguru/ fb/ponnurangam.kumaraguru, @ponguru](https://reader033.vdocuments.net/reader033/viewer/2022042917/5f5a110cf1ef9a56ee125c72/html5/thumbnails/45.jpg)
Simulated phishing website
http://andrewwebmail.org/password/change.htm?ID=9009
![Page 46: ACM Distinguished & TEDx Speaker Linkedin/in/ponguru ...precog.iiitd.edu.in/hcionnptel/2018/slides/HCIonNPTEL-Spring-2018-… · Linkedin/in/ponguru/ fb/ponnurangam.kumaraguru, @ponguru](https://reader033.vdocuments.net/reader033/viewer/2022042917/5f5a110cf1ef9a56ee125c72/html5/thumbnails/46.jpg)
Simulated phishing website
http://andrewwebmail.org/password/thankyou.html?ID=9009
![Page 47: ACM Distinguished & TEDx Speaker Linkedin/in/ponguru ...precog.iiitd.edu.in/hcionnptel/2018/slides/HCIonNPTEL-Spring-2018-… · Linkedin/in/ponguru/ fb/ponnurangam.kumaraguru, @ponguru](https://reader033.vdocuments.net/reader033/viewer/2022042917/5f5a110cf1ef9a56ee125c72/html5/thumbnails/47.jpg)
PhishGuru intervention
![Page 48: ACM Distinguished & TEDx Speaker Linkedin/in/ponguru ...precog.iiitd.edu.in/hcionnptel/2018/slides/HCIonNPTEL-Spring-2018-… · Linkedin/in/ponguru/ fb/ponnurangam.kumaraguru, @ponguru](https://reader033.vdocuments.net/reader033/viewer/2022042917/5f5a110cf1ef9a56ee125c72/html5/thumbnails/48.jpg)
Effect of PhishGuru
Condition N % who clicked on Day 0
% who clicked on Day 28
Control 172 52.3 44.2
Trained 343 48.4 24.5
![Page 49: ACM Distinguished & TEDx Speaker Linkedin/in/ponguru ...precog.iiitd.edu.in/hcionnptel/2018/slides/HCIonNPTEL-Spring-2018-… · Linkedin/in/ponguru/ fb/ponnurangam.kumaraguru, @ponguru](https://reader033.vdocuments.net/reader033/viewer/2022042917/5f5a110cf1ef9a56ee125c72/html5/thumbnails/49.jpg)
Results conditioned on participants who clicked on day 0
Trained participants less likely to fall for phish
![Page 50: ACM Distinguished & TEDx Speaker Linkedin/in/ponguru ...precog.iiitd.edu.in/hcionnptel/2018/slides/HCIonNPTEL-Spring-2018-… · Linkedin/in/ponguru/ fb/ponnurangam.kumaraguru, @ponguru](https://reader033.vdocuments.net/reader033/viewer/2022042917/5f5a110cf1ef9a56ee125c72/html5/thumbnails/50.jpg)
Results conditioned on participants who clicked on day 0
Trained participants less likely to fall for phish
Trained participants remember what they learned 28 days later
![Page 51: ACM Distinguished & TEDx Speaker Linkedin/in/ponguru ...precog.iiitd.edu.in/hcionnptel/2018/slides/HCIonNPTEL-Spring-2018-… · Linkedin/in/ponguru/ fb/ponnurangam.kumaraguru, @ponguru](https://reader033.vdocuments.net/reader033/viewer/2022042917/5f5a110cf1ef9a56ee125c72/html5/thumbnails/51.jpg)
Results conditioned on participants who clicked on day 0 and day 14
Two-train participants less likely than one-train participants to click on days 16 and 21
![Page 52: ACM Distinguished & TEDx Speaker Linkedin/in/ponguru ...precog.iiitd.edu.in/hcionnptel/2018/slides/HCIonNPTEL-Spring-2018-… · Linkedin/in/ponguru/ fb/ponnurangam.kumaraguru, @ponguru](https://reader033.vdocuments.net/reader033/viewer/2022042917/5f5a110cf1ef9a56ee125c72/html5/thumbnails/52.jpg)
Results conditioned on participants who clicked on day 0 and day 14
Two-train participants less likely than one-train participants to click on days 16 and 21
Two-train participants less likely than one-train participants to provide information on day 28
![Page 53: ACM Distinguished & TEDx Speaker Linkedin/in/ponguru ...precog.iiitd.edu.in/hcionnptel/2018/slides/HCIonNPTEL-Spring-2018-… · Linkedin/in/ponguru/ fb/ponnurangam.kumaraguru, @ponguru](https://reader033.vdocuments.net/reader033/viewer/2022042917/5f5a110cf1ef9a56ee125c72/html5/thumbnails/53.jpg)
Legitimate emails
Condition N Day 0 Day 7 Day 28 Clicked % Clicked % Clicked
% Control 90 50.0 41.1 38.9
One-train 89 39.3 42.7 32.3Two-train 77 48.1 44.2 35.1
No difference between the three conditions on day 0, 7, and 28
![Page 54: ACM Distinguished & TEDx Speaker Linkedin/in/ponguru ...precog.iiitd.edu.in/hcionnptel/2018/slides/HCIonNPTEL-Spring-2018-… · Linkedin/in/ponguru/ fb/ponnurangam.kumaraguru, @ponguru](https://reader033.vdocuments.net/reader033/viewer/2022042917/5f5a110cf1ef9a56ee125c72/html5/thumbnails/54.jpg)
Legitimate emails
No difference between the three conditions on day 0, 7, and 28
No difference within the three conditions for the three emails
Condition N Day 0 Day 7 Day 28 Clicked % Clicked % Clicked
% Control 90 50.0 41.1 38.9
One-train 89 39.3 42.7 32.3Two-train 77 48.1 44.2 35.1
![Page 55: ACM Distinguished & TEDx Speaker Linkedin/in/ponguru ...precog.iiitd.edu.in/hcionnptel/2018/slides/HCIonNPTEL-Spring-2018-… · Linkedin/in/ponguru/ fb/ponnurangam.kumaraguru, @ponguru](https://reader033.vdocuments.net/reader033/viewer/2022042917/5f5a110cf1ef9a56ee125c72/html5/thumbnails/55.jpg)
Most participants liked training, wanted more
● 280 complete post study responses ● 80% recommended that CMU continue PhishGuru training
● “I really liked the idea of sending CMU students fake phishing emails and then saying to them, essentially, HEY! You could've just gotten scammed! You should be more careful - here's how....”
● “I think the idea of using something fun, like a cartoon, to teach people about a serious subject is awesome!”
![Page 56: ACM Distinguished & TEDx Speaker Linkedin/in/ponguru ...precog.iiitd.edu.in/hcionnptel/2018/slides/HCIonNPTEL-Spring-2018-… · Linkedin/in/ponguru/ fb/ponnurangam.kumaraguru, @ponguru](https://reader033.vdocuments.net/reader033/viewer/2022042917/5f5a110cf1ef9a56ee125c72/html5/thumbnails/56.jpg)
Summary from this study● People trained with PhishGuru were less likely to click on phishing
links than those not trained● People retained their training for 28 days● Two training messages are better than one● PhishGuru training does not make people less likely to click on
legitimate links
![Page 57: ACM Distinguished & TEDx Speaker Linkedin/in/ponguru ...precog.iiitd.edu.in/hcionnptel/2018/slides/HCIonNPTEL-Spring-2018-… · Linkedin/in/ponguru/ fb/ponnurangam.kumaraguru, @ponguru](https://reader033.vdocuments.net/reader033/viewer/2022042917/5f5a110cf1ef9a56ee125c72/html5/thumbnails/57.jpg)
Summary of studies
Studies Results Lab study
I • Security notices are ineffective
• Users educated with PhishGuru made better decisionsLab study
II• Users in embedded condition retain and transfer
knowledge more effectively than other conditions even after 7 days
Real-world study I
• PhishGuru is effective in training people in the real world • Trained participants retained knowledge after 7 days of
trainingReal-world study II
• People trained with PhishGuru were less likely to click on phishing links than those not trained
• People retained their training for 28 days • Two training messages are better than one
• PhishGuru training does not make people less likely to click on legitimate links
![Page 58: ACM Distinguished & TEDx Speaker Linkedin/in/ponguru ...precog.iiitd.edu.in/hcionnptel/2018/slides/HCIonNPTEL-Spring-2018-… · Linkedin/in/ponguru/ fb/ponnurangam.kumaraguru, @ponguru](https://reader033.vdocuments.net/reader033/viewer/2022042917/5f5a110cf1ef9a56ee125c72/html5/thumbnails/58.jpg)
Training games: Anti-phishing Phil
Learning science principles
+Teachable moments
+Fun
![Page 59: ACM Distinguished & TEDx Speaker Linkedin/in/ponguru ...precog.iiitd.edu.in/hcionnptel/2018/slides/HCIonNPTEL-Spring-2018-… · Linkedin/in/ponguru/ fb/ponnurangam.kumaraguru, @ponguru](https://reader033.vdocuments.net/reader033/viewer/2022042917/5f5a110cf1ef9a56ee125c72/html5/thumbnails/59.jpg)
Takeaways
59
⚫Becoming an important problem to study ⚫Large number of projects are getting funded
into this area ⚫Less number of expertise available
![Page 60: ACM Distinguished & TEDx Speaker Linkedin/in/ponguru ...precog.iiitd.edu.in/hcionnptel/2018/slides/HCIonNPTEL-Spring-2018-… · Linkedin/in/ponguru/ fb/ponnurangam.kumaraguru, @ponguru](https://reader033.vdocuments.net/reader033/viewer/2022042917/5f5a110cf1ef9a56ee125c72/html5/thumbnails/60.jpg)
Ponnurangam Kumaraguru (“PK”)Associate Professor
Indraprastha Institute of Information TechnologyNew Delhi – 110078