cheats, anti-cheats, and machine learning
DESCRIPTION
Talk about the current cheats in video games, and how we propose to fix it. Machine learning.TRANSCRIPT
‘THE MULTI-BILLION DOLLAR INDUSTRY THAT’S
IGNORED’PART I: CHEATS, ANTI-CHEATS, AND MACHINE LEARNING
SOLUTIONS
INTRO
HP ESS TVM – RESEARCHER / PERSON OF MANY HATS
OWNERS – WHITE WIZARD STUDIO LLC
FORMER PROFESSIONAL GAMER – CPL, CEVO
@RSEVEY
PRINCIPAL RESEARCHER, VERACODE
SECURITY ENGINEER, APP SEC SPECIALIST
SOFTWARE DEVELOPER
AUTHOR / INSTRUCTOR
@J_MONTY
THIS TALK
• THE TALK THAT ALMOST WASN’T
• BH USA 2014, PAX DEV
• VALVE, FACEPUNCH STUDIO
GAMING INDUSTRY – OPEN GAMING ALLIANCE
ACTIVE GAMERS – FROM THE OPEN GAMING ALLIANCE
GAMER
• 67% OF US HOUSEHOLDS PLAY VIDEO GAMES
• AVERAGE AGE: 35
• 54% MALE / 46% FEMALE
• 62% PLAY GAMES ONLINE
SECURITY ISSUES
CheatingAnti-Cheats, EULA, Invasive nature
Digital Distribution Systems
Steam, Origin, Battle.net
DRM
Online Economy
Micro transactions, stores, Twitch
Game Engine
Application Security
CHEATINGCheat Maker
Cheat Distribut
or
User
CHEATING ECONOMY
Cheat Distributor Registered Users Estimated Paid Users Estimated Monthly Gross10% - 20% active paid
Aimjunkies.com 172,315 155,084 $186,100.8 - $372,201.6
Tmcheats.com 171,833 154,649 $185,578.8 – $371,157.6
Artificialaiming.net 142,319 113,855 $136,626 - $273,252
Fpscheats.com 375,733 281,799 $338,158.8 - $676,317.6
Ilikecheats.com 276,871 207,653 $249,183.6 - $498,367.2
Catalyst-hax.com 118,448 82,913 $99,495.6 – $198,991.2
Callofdutyhacks.com 225,709 169,281 $203,137.2 - $406,274.4
Hackersadvantage.com 8,774 7,896 $9475.2 - $18,950.4
CHEAT ECONOMY
• ~$1,407,752 TO ~$2,815,509 PER MONTH
• ~$16,893,024 TO ~$33,786,108 ANNUALLY
MORE THEORY
• 2.2 MILLION VAC BANNED ACCOUNTS
• IF EACH CHEATED FOR 1 MONTH
$21.9 MILLION• AVERAGE GAME COST - $20
• VALVE MAKES IF EVERYONE RE-BUYS:
$43.7 MILLION
ANTI-CHEATS EXAMPLES
VAC, Warden, Punkbuster
Monitor RAM, Processes
Doesn’t work.
Signature
Fairfight
Statistical based
Easy to avoid – just don’t use aimbot
Server
Valve’s Overwatch
Humans review highlights, match games only
Humans are.. Human.
Human
“This specific VAC test for this specific round of cheats was effective for 13 days, which is fairly typical.”
https://www.reddit.com/r/gaming/comments/1y70ej/valve_vac_and_trust/
“For most cheat developers, social engineering might be a cheaper way to attack the system than continuing the code arms race, which means that there will be more Reddit posts trying to cast VAC in a sinister light.”
“Our response is to make it clear what we were actually doing and why with enough transparency that people can make their own judgements as to whether or not we are trustworthy.”
Brian Dye, Senior Vice President for Information Security at Symantec
“82 percent of all malware it detects stays active for a mere hour, and 70 percent of all threats only surface once, as malware authors rapidly change their software to skirt detection from traditional antivirus solutions. ‘The function signature-based AV serves has become more akin to ghost hunting than threat detection and prevention’…”
http://www.pcworld.com/article/2150743/antivirus-is-dead-says-maker-of-norton-antivirus.html
FireEye “Antivirus Is Dead”PC World article in May of 2014
“Anti-virus is dead.”
VALVE, BLIZZARD, AND OTHERS
• ANTI-CHEAT JUSTIFICATION IS EXTREME
• BLIZZARD – THE WARDEN
• MONITORS RAM, PROCESSES, BROWSER TABS
• VALVE – VALVE ANTI CHEAT
• SENDS DNS INFORMATION BACK TO VALVE
• BLIZZARD AND VALVE – “INFORMATION IS HASHED BEFORE SENT”
END USER LICENSE AGREEMENTS - VALVE
END USER LICENSE AGREEMENTS - BLIZZARD
PUNKBUSTER EULA
THE CHEATS (OR ‘H4X, HAX, HACKS’)
• PAID CHEATS HAVE DRM SYSTEM
• SOME ARE HOSTED ON AMAZON EC2
• AMAZON DOESN’T CARE
THE LOADERS
HOW THEY’RE WORKING
• TL;DR THEY’RE ROOTKITS
• OPERATE AT RING 0
• THUS ANTI-CHEATS ALSO OPERATE AT RING 0.
CS:GO CHEATING
DEMO
OUTCOME
• ACCOUNT STILL NOT VAC BANNED
• 10 COMPETITIVE MATCHES, OBVIOUS CHEATING
• OVERWATCH BANNED
• BOTTOM-LINE:
• VAC DOESN’T WORK, YET CAN BE HIGHLY INVASIVE
LET’S REVIEW
• PROBLEM: CHEATING STILL NOT EFFECTIVELY DETECTED
• ANTI-CHEATS ARE INVASIVE
• CHEAT MAKERS AND DISTRIBUTORS HAVE LOTS OF MONEY
PROPOSED SOLUTION
• ANTI-CHEAT BASED OFF MACHINE LEARNING
• DOESN’T HAVE TO BE PERFECT
• JUST HAS TO BE BETTER THAN CURRENT SYSTEM, AND HUMANS
ML LOVES DATA
• FIRST PROBLEM… GETTING DATA
• FACEPUNCH NOR VALVE WOULD GIVE US ANY DATA
• HARD TO HARVEST IT ALL OURSELVES.
• VACBANNED.COM
• CHEATER DATASET
• COMPETITIVE LEAGUES
• NON-CHEATER DATASET
UNAVOIDABLE DATASET ISSUES
• VAC BANNED FROM WHICH GAME?
• IMMEDIATE VAC BAN, NO STATS
• NON-CHEATING PLAYER SET CONTAINS CHEATERS
• STEAM WEB API LIMITATIONS
• 400% ACCURACY?? WUT?
• MISSING IMPORTANT FEATURES
ML ANTI-CHEAT SYSTEM
• CLASSIFICATION PROBLEM (CHEATING / NOT CHEATING)
• SUPERVISED LEARNING
• BOOSTED DECISION TREE
• CHOOSING FEATURES
• “INDIVIDUAL MEASURABLE HEURISTIC PROPERTY OF A PHENOMENON BEING OBSERVED”
• K:D, ACCURACY, # OF MVP AWARDED, ETC.
DECISION TREE – TITANIC EXAMPLE
DEMO
Q & A