the curious case of protobufs …
DESCRIPTION
De-mystifying Google’s hottest binary protocol. The Curious case of Protobufs …. Prasanna Kanagasabai Jovin Lobo. About us :. Prasanna Kanagasabai : Security Engineer @ T houghtWorks Member of null- The Open Security Community . Author of IronSAP a module over IronWASP . - PowerPoint PPT PresentationTRANSCRIPT
![Page 1: The Curious case of Protobufs …](https://reader036.vdocuments.net/reader036/viewer/2022081507/56813ea5550346895da8fb9b/html5/thumbnails/1.jpg)
THE CURIOUS CASE OF PROTOBUFS…
De-mystifying Google’s hottest binary protocol
Prasanna KanagasabaiJovin Lobo
![Page 2: The Curious case of Protobufs …](https://reader036.vdocuments.net/reader036/viewer/2022081507/56813ea5550346895da8fb9b/html5/thumbnails/2.jpg)
About us : Prasanna Kanagasabai :
Security Engineer @ ThoughtWorks Member of null- The Open Security Community . Author of IronSAP a module over IronWASP. Speaker @ nullcon-Delhi, Clubhack, IIT Guwahati and
various null meetups.
Jovin Lobo : Associate Consultant @ Aujas Networks Member of null- The Open Security Community. Author of GameOver – Linux distro for learning web
security. Spoken at nullCon, GNUnify before.
![Page 3: The Curious case of Protobufs …](https://reader036.vdocuments.net/reader036/viewer/2022081507/56813ea5550346895da8fb9b/html5/thumbnails/3.jpg)
Agenda
Introduction. Anatomy of Protobufs
Defining Message formats in .Proto files. Protobuf compiler Python API to read write messages.
Encoding Scheme Problem Statement. Decoding like-a-pro with IronWasp
‘Protobuf Decoder’.
![Page 4: The Curious case of Protobufs …](https://reader036.vdocuments.net/reader036/viewer/2022081507/56813ea5550346895da8fb9b/html5/thumbnails/4.jpg)
Introduction:
Protocol Buffers a.k.a Protobufs : Protobufs are Google's own way of
serializing structured data . Extensible, language-neutral and
platform-neutral . Smaller, faster and simpler to
implement. Java, C++ and Python
![Page 5: The Curious case of Protobufs …](https://reader036.vdocuments.net/reader036/viewer/2022081507/56813ea5550346895da8fb9b/html5/thumbnails/5.jpg)
Anatomy:
Over view :
![Page 6: The Curious case of Protobufs …](https://reader036.vdocuments.net/reader036/viewer/2022081507/56813ea5550346895da8fb9b/html5/thumbnails/6.jpg)
Defining a .Proto file.
#> less Example.protomessage Conference {required string conf_name = 1 ; required int32 no_of_days = 2 ; optional string email = 3 ;
}// * 1,2,3 are unique tags. These are used by
the fields in binary encoding.* For optimization use tags from 1-15 as higher nos. will use one more byte to encode.
![Page 7: The Curious case of Protobufs …](https://reader036.vdocuments.net/reader036/viewer/2022081507/56813ea5550346895da8fb9b/html5/thumbnails/7.jpg)
Compiling
Syntax: protoc –I=$_input_Dir --
python_out=$_out_Dir $_Path_ProtoFile
Eg: protoc –I=. --python_out=.
Example.proto
This will generate a Example_pb2.py file in the specified destination directory.
![Page 8: The Curious case of Protobufs …](https://reader036.vdocuments.net/reader036/viewer/2022081507/56813ea5550346895da8fb9b/html5/thumbnails/8.jpg)
$ProtoFile_pb2.py
The Protobuf compiler generates special descriptors for all your messages, enums, and fields.
It also generates empty classes, one for each message type:
Eg:
![Page 9: The Curious case of Protobufs …](https://reader036.vdocuments.net/reader036/viewer/2022081507/56813ea5550346895da8fb9b/html5/thumbnails/9.jpg)
Reading and writing messages using the Protobuf binary format :
SerializeToString() serializes the message and returns it as a
string. ParseFromString(data)
parses a message from the given string.
![Page 10: The Curious case of Protobufs …](https://reader036.vdocuments.net/reader036/viewer/2022081507/56813ea5550346895da8fb9b/html5/thumbnails/10.jpg)
Demo: Protobuf… how it wrks
![Page 11: The Curious case of Protobufs …](https://reader036.vdocuments.net/reader036/viewer/2022081507/56813ea5550346895da8fb9b/html5/thumbnails/11.jpg)
Encoding. example2.proto
message Ex1 { required int32 num = 1; // field tag }
Code snippet:obj = example2_pb2.Ex1();obj.num = 290; // field valueobj.SerializeToString();
Output : 08 A2 02 #hex000010001010001000000010 #binary
![Page 12: The Curious case of Protobufs …](https://reader036.vdocuments.net/reader036/viewer/2022081507/56813ea5550346895da8fb9b/html5/thumbnails/12.jpg)
Problem statement.
![Page 13: The Curious case of Protobufs …](https://reader036.vdocuments.net/reader036/viewer/2022081507/56813ea5550346895da8fb9b/html5/thumbnails/13.jpg)
This is what freaked him out
08 A2 02000010001010001000000010
![Page 14: The Curious case of Protobufs …](https://reader036.vdocuments.net/reader036/viewer/2022081507/56813ea5550346895da8fb9b/html5/thumbnails/14.jpg)
Lets Decode it ..
Step 1 : Find the wire type .
Step 2: Find the field number.
Step 3: Find the field tag.
![Page 15: The Curious case of Protobufs …](https://reader036.vdocuments.net/reader036/viewer/2022081507/56813ea5550346895da8fb9b/html5/thumbnails/15.jpg)
Step1: finding wire type.
0000 1000 1010 0010 0000 0010 To find wire type take the first
byte: 0000 1000 1010 0010 0000 0010
[0]000 1000 Drop MSB from First byte.
0001 000 The last 3 bits give wire type.
Wire type is 000 type = 0 is Varint.
![Page 16: The Curious case of Protobufs …](https://reader036.vdocuments.net/reader036/viewer/2022081507/56813ea5550346895da8fb9b/html5/thumbnails/16.jpg)
Wire types
![Page 17: The Curious case of Protobufs …](https://reader036.vdocuments.net/reader036/viewer/2022081507/56813ea5550346895da8fb9b/html5/thumbnails/17.jpg)
Step 2: Field tag.
What we already have is 0001000 Now we right shift value by 3 bits
and the remaining bits will give us the field tag. 0001000 0001 000 ‘0001 ‘ i.e. ‘ 1’
So we get the field tag = 1
![Page 18: The Curious case of Protobufs …](https://reader036.vdocuments.net/reader036/viewer/2022081507/56813ea5550346895da8fb9b/html5/thumbnails/18.jpg)
Step 3: Find the field value 0000 1000 1010 0010 0000 0010 We drop the 1st byte
1010 0010 0000 0010 Drop the MSB’s from each of these bytes
1010 0010 0000 0010 010 0010 000 0010
Reverse these bytes to obtain the field value. 000 0010 010 0010 000 0010 010 0010 i.e 256 + 32 + 2 = 290
So we finally get the value of the field = 290.
![Page 19: The Curious case of Protobufs …](https://reader036.vdocuments.net/reader036/viewer/2022081507/56813ea5550346895da8fb9b/html5/thumbnails/19.jpg)
So we successfully decoded example2.proto
message Ex1 { required int32 num = 1; }
Code snippet:obj = example2_pb2.Ex1();obj.num = 290;obj.SerializeToString();
Output : 08 A2 02 #hex000010001010001000000010 #binary
We successfully Decoded Value : “290”
![Page 20: The Curious case of Protobufs …](https://reader036.vdocuments.net/reader036/viewer/2022081507/56813ea5550346895da8fb9b/html5/thumbnails/20.jpg)
Demo : Lets do this live
![Page 21: The Curious case of Protobufs …](https://reader036.vdocuments.net/reader036/viewer/2022081507/56813ea5550346895da8fb9b/html5/thumbnails/21.jpg)
Automating all this with IronWasp Protobuf Decoder:
About IronWasp : IronWasp is an open-source web security
scanner. It is designed to be customizable to the
extent where users can create their own custom security scanners using it.
Author – Lavakumar Kuppan (@lavakumark)
Website : www.ironwasp.org
![Page 22: The Curious case of Protobufs …](https://reader036.vdocuments.net/reader036/viewer/2022081507/56813ea5550346895da8fb9b/html5/thumbnails/22.jpg)
ProtoBuf Decoder
![Page 23: The Curious case of Protobufs …](https://reader036.vdocuments.net/reader036/viewer/2022081507/56813ea5550346895da8fb9b/html5/thumbnails/23.jpg)
Road Map for Protobuf Decoder
![Page 24: The Curious case of Protobufs …](https://reader036.vdocuments.net/reader036/viewer/2022081507/56813ea5550346895da8fb9b/html5/thumbnails/24.jpg)
01101000001111010000010110111001111001001000000101000101110101011001010111001101110100011010010110111101101110011100110010000000111111
![Page 25: The Curious case of Protobufs …](https://reader036.vdocuments.net/reader036/viewer/2022081507/56813ea5550346895da8fb9b/html5/thumbnails/25.jpg)
01101000001111010000010110111001111001001000000101000101110101011001010111001101110100011010010110111101101110011100110010000000111111
![Page 26: The Curious case of Protobufs …](https://reader036.vdocuments.net/reader036/viewer/2022081507/56813ea5550346895da8fb9b/html5/thumbnails/26.jpg)
01101000001111010000010110111001111001001000000101000101110101011001010111001101110100011010010110111101101110011100110010000000111111
Hmmm … Decoding ……
![Page 27: The Curious case of Protobufs …](https://reader036.vdocuments.net/reader036/viewer/2022081507/56813ea5550346895da8fb9b/html5/thumbnails/27.jpg)
Any Questions ?
Done … It says ……
![Page 28: The Curious case of Protobufs …](https://reader036.vdocuments.net/reader036/viewer/2022081507/56813ea5550346895da8fb9b/html5/thumbnails/28.jpg)
Any Questions ?
Done … It says ……
![Page 29: The Curious case of Protobufs …](https://reader036.vdocuments.net/reader036/viewer/2022081507/56813ea5550346895da8fb9b/html5/thumbnails/29.jpg)
Thank You