Detecting Source Code Plagiarism with CodeMatch
Bob ZeidmanZeidman Consulting
Agenda
Source code plagiarismPrevious toolsCodeMatch
– Statement Matching– Comment Matching– Identifier Matching– Partial Identifier Matching– Instruction Sequence Matching
Conclusion
Source Code Plagiarism
Entities– Universities– Corporations
Reasons– Internet– Search engines– Open source movements– Mobile employees
Plague: Algorithm
Geoff Whale, University of New South WalesThree phases:– Create a sequence of tokens and a list of metrics to
describe each program.– Compare the structure metrics of files to find similar
code structures.– Compare token sequences within similar source
code structures.
Plague: Example
if (x == 5){
// Loop on j herefor (j = 0; j < Index; j++)
printf("x = %i", j);}else
while (i < 5) i++;
CONDITIONAL_BEGINLOOP_BEGINDISPLAYLOOP_ENDCONDITIONAL_ENDCONDITIONAL_BEGINLOOP_BEGINARITHMETICLOOP_ENDCONDITIONAL_END
Plague: Problems
Hard to adapt to new programming languages.The output needs interpretation.Uses slow UNIX shell tools for processing.Vulnerable to changing the order of code lines in the source code. Throws out useful information when it discards comments, variable names, function names, and other identifiers.
YAP, YAP2, YAP3: Algorithm
Michael Wise, University of Sydney, AustraliaTwo phases:– Remove whitespace, comments, and identifier
names, replace language statements with tokens.– Compare pairs of token files.
JPlag: Algorithm
Lutz Prechelt and Guido Malpohl, University KarlsruheMichael Philippsen, University of Erlangen-NurembergPhases:– Remove whitespace, comments, and identifier
names, replace language statements with tokens.– Compare tokens in different files.
YAP, JPlag: Problems
To decrease the run time, uses hashing and only considers matches of strings of a minimal length.Tokens are still dependent on knowledge of the programming language.Although less so than Plague, still vulnerable to changing the order of code lines.Throws out useful information when it discards comments, variable names, function names, and other identifiers.
MOSS: Algorithm
Alex Aiken, Stanford UniversityPhases:
– Remove all whitespace and punctuation, convert characters to lower case.
– Divide remaining characters into k-grams, which are contiguous substrings of length k, by sliding a window of size kthrough the file.
– Hash each k-gram and select a subset of all k-grams to be the fingerprints of the file.
– Compare file fingerprints to find similar files.
MOSS: Example
She loves you yeah, yeah, yeah.
Some textshelo helov elove loves ovesy vesyo esyousyouy youye ouyea uyeah yeahy eahye ahyeahyeah yeahy eahye ahyea hyeah
5-grams77 72 42 17 98 50 23 55 6 66 34 24 39 1184 24 39 11 84
Hypothetical hash72 24 84 24 84
Fingerprint
MOSS: Problems
Structural information is lost (e.g., whitespace, punctuation, uppercase characters, non-alphanumeric symbols).Larger k-grams decrease execution time, but decrease sensitivity.Most k-grams are also thrown out for faster processing, reducing accuracy.
CodeMatch: Algorithms
Statement MatchingComment MatchingIdentifier MatchingPartial Identifier MatchingInstruction Sequence Matching
CodeMatch: Statement Matching
File 1 File 2
1 /* begin routine */ 1 /* find the file extension */
2 void fdiv( 2 void file_divide(
3 char *fname, // file name 3 char *fname,
4 char *path) /* path */ 4 char *path)
5 { 5 {
6 int Index1, j; 6 int i, j; // begin routine
7 7 while (1) // loop here
8 while (1) 8 j = strlen(fname);
9 j = strlen(fname); 9
10 // find the file extension 10
CodeMatch: Comment Matching
File 1 File 2
1 /* begin routine */ 1 /* find the file extension */
2 void fdiv( 2 void file_divide(
3 char *fname, // file name 3 char *fname,
4 char *path) /* path */ 4 char *path)
5 { 5 {
6 int Index1, j; 6 int i, j; // begin routine
7 7 while (1) // loop here
8 while (1) 8 j = strlen(fname);
9 j = strlen(fname); 9
10 // find the file extension 10
CodeMatch: Identifier Matching
Counts the number of matching words that are not programming language keywords.Requires a list of keywords to exclude.Matching numerals given less weight that matching alphabeticals.Finds matching identifiers – routines, variables, constants, etc.
CodeMatch: Partial Identifier Matching
Counts the number of partially matching words that are not programming language keywords.Requires a list of keywords to exclude.Matching numerals given less weight that matching alphabeticals.Finds disguised identifiers – routines, variables, constants, etc.For example, abc partially matches abc1 and xxxabc.
CodeMatch: Instruction Sequence Matching
File 1 File 2
1 /* begin routine */ 1 /* find the file extension */
2 void fdiv( 2 void file_divide(
3 char *fname, // file name 3 char *fname,
4 char *path) /* path */ 4 char *path)
5 { 5 {
6 int Index1, j; 6 int i, j; // begin routine
7 7 while (1) // loop here
8 while (1) 8 j = strlen(fname);
9 j = strlen(fname); 9
10 // find the file extension 10
CodeMatch: Total Match Score
t = kww +kp p +ks s +kc c +kq q
CodeMatch Basic Report
Comparing files in folder D:\CodeMatch\Code Development\test\C test 2\files 1 To files in folder D:\CodeMatch\Code Development\test\C test 2\files 2
D:\CodeMatch\Code Development\test\C test 2\files 1\bpf_dump.c Match Score Compared To File
2910 D:\CodeMatch\Code Development\test\C test 2\files 2\bpf_dump.c 374 D:\CodeMatch\Code Development\test\C test 2\files 2\W32NReg.c 374 D:\CodeMatch\Code Development\test\C test 2\files 2\test\W32NReg (variable names changed).c374 D:\CodeMatch\Code Development\test\C test 2\files 2\test\W32NReg (no comments).c
D:\CodeMatch\Code Development\test\C test 2\files 1\bpf_filter.c Match Score Compared To File
606 D:\CodeMatch\Code Development\test\C test 2\files 2\W32NReg.c 606 D:\CodeMatch\Code Development\test\C test 2\files 2\test\W32NReg (no comments).c 572 D:\CodeMatch\Code Development\test\C test 2\files 2\test\W32NReg (variable names changed).c398 D:\CodeMatch\Code Development\test\C test 2\files 2\bpf_dump.c
CodeMatch Detailed Report
C o m p a r i n g f i l e 1 : D : \ C o d e M a t c h \ \ t e s t \ C t e s t 2 \ f i l e s 1 \ b p f _ d u m p .c T o f i l e 2 : D : \ C o d e M a t c h \ C t e s t 2 \ f i l e s 2 \ t e s t \ W 3 2 N r e g . c
M a t c h i n g s o u r c e l i n e s : F i l e 1 L i n e # F i le 2 L i n e # S o u r c e l i n e
2 1 1 # in c lu d e < w in d o w s . h >2 2 3 # in c lu d e < s t d io . h > 2 4 7 # in c lu d e " W iN D I S . h "
M a t c h i n g c o m m e n t l i n e s : F i l e 1 L i n e # F i le 2 L i n e # C o m m e n t l i n e
3 3 * T h e R e g e n t s o f t h e U n iv e r s it y o f C a l i fo r n ia . A l l r ig h t s r e s e r v e d . 1 0 5 * R e d is t r ib u t io n a n d u s e in s o u r c e a n d b in a r y fo r m s , w it h o r w it h o u t
L o n g e s t m a t c h i n g s e m a n t i c s e q u e n c e :
F i l e 1 L i n e # F i l e 2 L i n e # N u m b e r o f m a t c h i n g l i n e s
2 1 1 3
M a t c h i n g w o r d s :
s t d io W iN D I S w in d o w s
M a t c h i n g p a r t i a l w o r d s :
0 x w in d o w s
Competitive Evaluation: Test
GNU C compiler GCC version 3.3.2– Less than 100 lines– Between 100 and 1000 lines– Greater than 1000 lines
Modify files– Remove all comments– Rename all identifiers– Rearrange routines within the file– Rearrange lines of code within routines in the file– Do all of the above– Remove all the code but leave the comments– Create one file that has exactly one routine from each of the other
files in the same category
Competitive Evaluation: Accuracy
70%(146 of 210)
MOSS
80%(169 of 210)
JPlag
95%(200 of 210)
CodeMatch
Copied files foundProgram
Conclusion
Previous tools miss important matchesCodeMatch– Statement Matching– Comment Matching– Identifier Matching– Partial Identifier Matching– Instruction Sequence Matching
More accurate than other tools
Download CodeMatch For Free
www.ZeidmanConsulting.com