Pushing Counter Strike logs into ELK

Daniel Lawrence

pushingCOUNTER STRIKE

events into ELK

Beer & BulletsEvery 4th Thursday

How it works

read logs from disk

PythonSimple “while True” loop.Read log lines straight from the line.

Logs to json

RegexParse the event lines into json

push to elasticsearch

requestsPush all the parsed json into an elasticsearch cluster.

under the coversWARNING: I suck at regex.

02/04/2016 - 22:53:13: "rborkows<109><STEAM_1:1:18222969><TERRORIST>" [604 -357 359] killed "Dustin<101><BOT><CT>" [770 -273 504] with "galilar" (headshot)L 02/04/2016 - 22:53:14: "JEST3R<105><STEAM_1:0:2414056><TERRORIST>" [-816 297 184] killed "Brad<45><STEAM_1:1:15941846><CT>" [-413 -665 224] with "tec9"L 02/04/2016 - 22:53:15: "Norm<100><BOT><TERRORIST>" [-258 68 71] killed "M4DDOG<104><STEAM_1:1:2415605><CT>" [-290 77 135] with "galilar"L 02/04/2016 - 22:53:21: "Irving<98><BOT><TERRORIST>" [535 -195 88] killed "M4DDOG<104><STEAM_1:1:2415605><CT>" [504 -701 98] with "galilar"L 02/04/2016 - 22:53:22: "3ulle+ C@+cher<106><STEAM_1:0:6158004><CT>" [-672 87 44] killed "JEST3R<105><STEAM_1:0:2414056><TERRORIST>" [-622 86 110] with "mag7"L 02/04/2016 - 22:53:23: "Brad<45><STEAM_1:1:15941846><CT>" [218 -750 8] killed "Irving<98><BOT><TERRORIST>" [477 -323 144] with "mag7"L 02/04/2016 - 22:53:23: "rborkows<109><STEAM_1:1:18222969><TERRORIST>" [746 -554 431] killed "squarel<108><STEAM_1:0:42240982><CT>" [585 -535 499] with "sawedoff"L 02/04/2016 - 22:53:25: "Brad<45><STEAM_1:1:15941846><CT>" [242 -940 3] killed "Art Vandelay<107><STEAM_1:0:74644593><TERRORIST>" [195 -749 124] with "g3sg1" (revenge)L 02/04/2016 - 22:53:28: "Tim<103><BOT><CT>" [-264 -263 20] killed "scurvy_lubber<2><STEAM_1:0:23714619><TERRORIST>" [-123 101 137] with "sawedoff"L 02/04/2016 - 22:53:29: "Brad<45><STEAM_1:1:15941846><CT>" [181 -746 12] killed "rborkows<109><STEAM_1:1:18222969><TERRORIST>" [120 -714 81] with "g3sg1" (domination)L 02/04/2016 - 22:53:30: "JEST3R<105><STEAM_1:0:2414056><TERRORIST>" [-561 -522 11] killed "popacai<110><STEAM_1:1:82066642><CT>" [-494 -814 123] with "hkp2000"L 02/04/2016 - 22:53:31: "3ulle+ C@+cher<106><STEAM_1:0:6158004><CT>" [-67 980 72] killed "Norm<100><BOT><TERRORIST>" [215 826 160] with "mag7" (revenge)L 02/04/2016 - 22:53:35: "Brandon<102><BOT><TERRORIST>" [-1221 -425 12] killed "Brad<45><STEAM_1:1:15941846><CT>" [-857 -438 79] with "m4a1" (headshot)L 02/04/2016 - 22:53:38: "M4DDOG<104><STEAM_1:1:2415605><CT>" [28 -1087 5] killed "Art Vandelay<107><STEAM_1:0:74644593><TERRORIST>" [-1 -1080 70] with "knife_default_ct"L 02/04/2016 - 22:53:39: "Dustin<101><BOT><CT>" [-243 1014 56] killed "Norm<100><BOT><TERRORIST>" [-312 689 120] with "sawedoff"L 02/04/2016 - 22:53:40: "squarel<108><STEAM_1:0:42240982><CT>" [-328 -1123 5] killed "JEST3R<105><STEAM_1:0:2414056><TERRORIST>" [-478 -984 77] with "mag7"L 02/04/2016 - 22:53:41: "rborkows<109><STEAM_1:1:18222969><TERRORIST>" [-260 -238 36] killed "Tim<103><BOT><CT>" [-89 214 142] with "sawedoff" (headshot)L 02/04/2016 - 22:53:43: "Brandon<102><BOT><TERRORIST>" [-552 -520 12] killed "M4DDOG<104><STEAM_1:1:2415605><CT>" [-547 -906 78] with "galilar"L 02/04/2016 - 22:53:44: "Irving<98><BOT><TERRORIST>" [-380 694 56] killed "Dustin<101><BOT><CT>" [-315 811 120] with "sawedoff"L 02/04/2016 - 22:53:46: "scurvy_lubber<2><STEAM_1:0:23714619><TERRORIST>" [-612 -502 4] killed "squarel<108><STEAM_1:0:42240982><CT>" [-557 -935 58] with "sawedoff"L 02/04/2016 - 22:53:47: "Brad<45><STEAM_1:1:15941846><CT>" [460 18 95] killed "rborkows<109><STEAM_1:1:18222969><TERRORIST>" [473 138 145] with "scar20"L 02/04/2016 - 22:53:50: "JEST3R<105><STEAM_1:0:2414056><TERRORIST>" [-873 -309 382] killed "Dustin<101><BOT><CT>" [676 -552 349] with "hkp2000"L 02/04/2016 - 22:53:51: "Art Vandelay<107><STEAM_1:0:74644593><TERRORIST>" [-365 -1079 10] killed "Tim<103><BOT><CT>" [-200 -1051 79] with "tec9"L 02/04/2016 - 22:53:53: "M4DDOG<104><STEAM_1:1:2415605><CT>" [407 258 85] killed "Norm<100><BOT><TERRORIST>" [376 261 152] with "knife_default_ct"L 02/04/2016 - 22:53:54: "Dustin<101><BOT><CT>" [220 -985 5] killed "Art Vandelay<107><STEAM_1:0:74644593><TERRORIST>" [171 -957 72] with "sawedoff"L 02/04/2016 - 22:53:58: "Dustin<101><BOT><CT>" [-36 -1061 5] killed "rborkows<109><STEAM_1:1:18222969><TERRORIST>" [-298 -1039 85] with "mag7"

txt2re.comregular expression generator

txt2re.comL 02/04/2016 - 22:53:14:

"JEST3R<105><z><TERRORIST>" [-816 297 184] killed

"Brad<45><x><CT>" [-413 -665 224] with "tec9"

kill_re = r'L (.*?) - (.*?): "(.+?)<.+?" \[(.+?)\].+?"' + \ r'(.+?)<.+?" \[(.+?)\].+?"(.+?)"(.*)'

kill_distance = math.sqrt(((x1-x2)**2) + ((y1-y2)**2) +((z1-z2)**2))

return { ‘type’: ‘kill’, ‘player’: killer, ‘victim’: victim, ‘weapon’: weapon_name, ‘distance’: distance, ‘tags’: tags, ‘kill_date’: time_stmap}

requestsis awesome

requests.post(‘http://elasticsearch/csgo’, data=kills)

So now we have thedata streaming into ELK

Last beer & bullets!

Anurag

Killed the most players

Daniel

Died the most… However knifed the most

amount of people

Brandon

Got stabbed the most.

Robert

Killed the most number of innocent chickens

Beer & BulletsEvery 4th Thursday

github.com/daniellawrence/steam2elk

Take Aways

kibana 4 is awesome

use requests

use re2text.com

come to the next event