exploiting machine learning to subvert your spam filter
DESCRIPTION
Exploiting Machine Learning to Subvert Your Spam Filter. Blaine Nelson / Marco Barreno / fuching Jack Chi / Anthony D. Joseph Benjamin I. P. Rubinstein / Udam Saini / Charles Sutton / J.D. Tygar / Kai Xia University of California, Berkeley April, 2008 Presented by: GyuYoung Lee. - PowerPoint PPT PresentationTRANSCRIPT
![Page 1: Exploiting Machine Learning to Subvert Your Spam Filter](https://reader035.vdocuments.net/reader035/viewer/2022062218/5681626f550346895dd2dcf2/html5/thumbnails/1.jpg)
Exploiting Machine Learning to Subvert Your Spam Filter
Blaine Nelson / Marco Barreno / fuching Jack Chi / Anthony D. JosephBenjamin I. P. Rubinstein / Udam Saini / Charles Sutton / J.D. Tygar / Kai Xia
University of California, BerkeleyApril, 2008
Presented by: GyuYoung Lee
![Page 2: Exploiting Machine Learning to Subvert Your Spam Filter](https://reader035.vdocuments.net/reader035/viewer/2022062218/5681626f550346895dd2dcf2/html5/thumbnails/2.jpg)
• Do you know ?
• ☞ Spam Filtering System
• ☞ Machine learning is applied to Spam Filtering
• ☞ Adversary can exploit machine learning
Making Spam Filter to be useless
User give up using Spam Filter
Introduction
![Page 3: Exploiting Machine Learning to Subvert Your Spam Filter](https://reader035.vdocuments.net/reader035/viewer/2022062218/5681626f550346895dd2dcf2/html5/thumbnails/3.jpg)
• Which part is most weak ?
• ☞ Machine Learning
• How can we attack ?
• ☞ Poisoning Training Set
Spam Filtering System
![Page 4: Exploiting Machine Learning to Subvert Your Spam Filter](https://reader035.vdocuments.net/reader035/viewer/2022062218/5681626f550346895dd2dcf2/html5/thumbnails/4.jpg)
Poisoning Training Set
FalseNegative
FalsePositive
![Page 5: Exploiting Machine Learning to Subvert Your Spam Filter](https://reader035.vdocuments.net/reader035/viewer/2022062218/5681626f550346895dd2dcf2/html5/thumbnails/5.jpg)
• If attacker wins at contaminating attack?
① High false positives
☞ User loses so many legitimate e-mails
② High false negatives
☞ User encounters so many spam e-mails
③ High unsure messages
☞ so many human decision required No time saving
▣ Finally, user gives up using spam filter
Poisoning Effect
![Page 6: Exploiting Machine Learning to Subvert Your Spam Filter](https://reader035.vdocuments.net/reader035/viewer/2022062218/5681626f550346895dd2dcf2/html5/thumbnails/6.jpg)
• Bayesian spam filter• Three classifications
① Spam② Ham(non-spam)③ Unsure
• Score• Spam filter generate one score for ham and another for spam
Bayesian Spam Filtering - Concept
message Spam score Ham score
spam high low
ham low high
unsure high high
unsure low low
Strength↓ false positives↓ false negatives
Weakness↑ unsures(need human decision)
![Page 7: Exploiting Machine Learning to Subvert Your Spam Filter](https://reader035.vdocuments.net/reader035/viewer/2022062218/5681626f550346895dd2dcf2/html5/thumbnails/7.jpg)
• Spamicity of words included in the e-mail
① Measure them respectively
② Combine them
③ Evaluate the possibility that the e-mail can be spam
Bayesian Spam Filtering - Steps
![Page 8: Exploiting Machine Learning to Subvert Your Spam Filter](https://reader035.vdocuments.net/reader035/viewer/2022062218/5681626f550346895dd2dcf2/html5/thumbnails/8.jpg)
• ① Measure Spamicity of words respectively
Bayesian Spam Filtering - Details
![Page 9: Exploiting Machine Learning to Subvert Your Spam Filter](https://reader035.vdocuments.net/reader035/viewer/2022062218/5681626f550346895dd2dcf2/html5/thumbnails/9.jpg)
• ② Combine Spamicity of words
Bayesian Spam Filtering - Steps
![Page 10: Exploiting Machine Learning to Subvert Your Spam Filter](https://reader035.vdocuments.net/reader035/viewer/2022062218/5681626f550346895dd2dcf2/html5/thumbnails/10.jpg)
• ③ Evaluate the possibility that the e-mail can be spam
☞ If (Pr > Threshold) then regard the e-mail as Spam
Bayesian Spam Filtering - Steps
![Page 11: Exploiting Machine Learning to Subvert Your Spam Filter](https://reader035.vdocuments.net/reader035/viewer/2022062218/5681626f550346895dd2dcf2/html5/thumbnails/11.jpg)
• Traditional attack
: modify spam emails evade spam filter
• Attack in this paper
: subvert the spam filter drop legitimate emails
Attack Strategies
![Page 12: Exploiting Machine Learning to Subvert Your Spam Filter](https://reader035.vdocuments.net/reader035/viewer/2022062218/5681626f550346895dd2dcf2/html5/thumbnails/12.jpg)
• Dictionary attack
① Include entire dictionary
☞ spam score of all tokens
② Legitimate email ☞ marked as spam
• Focused attack
① Include only tokens in a particular target e-mail
② Target message ☞ marked as spam
Attack Styles
![Page 13: Exploiting Machine Learning to Subvert Your Spam Filter](https://reader035.vdocuments.net/reader035/viewer/2022062218/5681626f550346895dd2dcf2/html5/thumbnails/13.jpg)
• We can find
① 1% attack emails
② Accuracy falls
significantly
③ Filter unusable
Experiments – dictionary attack
![Page 14: Exploiting Machine Learning to Subvert Your Spam Filter](https://reader035.vdocuments.net/reader035/viewer/2022062218/5681626f550346895dd2dcf2/html5/thumbnails/14.jpg)
• Probability of guessing
• Guessing p increase
Attack is more Effective
• We can find
• Success of target attack
depends on
prior knowledge
Experiments – focused attack #1
![Page 15: Exploiting Machine Learning to Subvert Your Spam Filter](https://reader035.vdocuments.net/reader035/viewer/2022062218/5681626f550346895dd2dcf2/html5/thumbnails/15.jpg)
• Condition
• Fix Guessing p to 0.5
• X-axis
• N of msgs in the attack
• Y-axis
• % of msgs misclassified
• We can find
• Target e-mail is quickly blocked by the filter
Experiments – focused attack #2
![Page 16: Exploiting Machine Learning to Subvert Your Spam Filter](https://reader035.vdocuments.net/reader035/viewer/2022062218/5681626f550346895dd2dcf2/html5/thumbnails/16.jpg)
• RONI (Reject on Negative Impact) defense
① Idea
☞ Measuring each email’s impact
☞ Removing deleterious messages from training set
② Method
☞ Measuring the effect of email
☞ Testing performance difference with and without that e-mail
③ Effect
☞ Perfectly identify all dictionary attacks
☞ Hard to identify focused attack emails
Defenses – RONI
![Page 17: Exploiting Machine Learning to Subvert Your Spam Filter](https://reader035.vdocuments.net/reader035/viewer/2022062218/5681626f550346895dd2dcf2/html5/thumbnails/17.jpg)
• Dynamic threshold defense
① Method
☞ Dynamically adjusts two spots of threshold
② Effect
☞ Compared to SpamBayes alone,
☞ Misclassification is significantly reduced
Defenses – dynamic threshold
![Page 18: Exploiting Machine Learning to Subvert Your Spam Filter](https://reader035.vdocuments.net/reader035/viewer/2022062218/5681626f550346895dd2dcf2/html5/thumbnails/18.jpg)
• Adversary can disable SpamBayes filter
• Dictionary attack
• Only 1% control 36% misclassification
• RONI can defense it effectively
• Dynamic threshold can mitigate it effectively
• Focused attackhard to defend by attack’s knowledge
• These Techniques can be effective
• Similar learning algorithms (ex) Bogo Filter
• Other learning system (ex) worm or intrusion detection
Conclusion