static analysis for perl

Static Code Analysis for Perl

@moznion

Taiki Kawakami a.k.a @moznion

Sever side engineer (Java and Perl)

Author of - Perl::Lint - go-setlock

Taiki Kawakami a.k.a @moznion

Sever side engineer (Java and Perl)

Author of - Perl::Lint - go-setlock

Taiki Kawakami a.k.a @moznion

Sever side engineer (Java and Perl)

Author of - Perl::Lint - go-setlock

Fundamental of Static Analysis

Static Analysis

A method of analysis source code WITHOUT execution

Static AnalysisExample of advantages: - Easy to detect - unused vars - irregular coding styles - Analyze dependencies between modules/classes

Static AnalysisExample of advantages: - Easy to detect - unused vars - irregular coding styles - Analyze dependencies between modules/classes

BORING!

Static AnalysisExample of advantages: - Easy to detect - unused vars - irregular coding styles - Analyze dependencies between modules/classes

Difficult…

Let's Exercise

This code has 5 traps

This code has 5 traps

This code has 5 traps

This code has 5 traps

This code has 5 traps

This code has 5 traps

It was fun?

This is ridiculous code ceview

Probably human overlooks

We should focus on advanced topic on code review

How?

It is necessary clean code

Destroy these

Be maintainable code!

Make computer analyze them!

How to make static analyzer?

Pre-Process

Lexical Analyze

Syntactic Analyze

Source code (String)

Result

Analyze

Pre-Process

Lexical Analyze

Syntactic Analyze

Source code (String)

Result

Analyze

PPI::Tokenizer

Pre-Process

Lexical Analyze

Syntactic Analyze

Source code (String)

Result

Analyze

PPI::Document

Provides PDOM Structure

Pre-Process

Lexical Analyze

Syntactic Analyze

Source code (String)

Result

Analyze

“Analyze” phase checks code with using AST and tokens in accordance with rules

Method of some languages are different; they look byte code (e.g. Java:findbugs)

Perl::Critic

Perl::Critic is the great tool!

Perl::Critic checks the code conform to PBP style or not

Perl::Critic uses PPI as a Lexer and Parser

Pre-Process

Lexical Analyze

Syntactic Analyze

Source code (String)

Result

Analyze

PPI

Perl::Lint

Perl::Lint is a yet another static analyser for perl

This project supported by TPF

Perl::Critic is enough. Why Perl::Lint?

I want to make it faster!!!

Mechanism of Perl::Lint

Pre-Process

Lexical Analyze

Syntactic Analyze

Source code (String)

Result

Analyze

Regex

Compiler::Lexer

Perl::Lint::Policy

Pre-Process

Lexical Analyze

Syntactic Analyze

Source code (String)

Result

Analyze

Regex

Compiler::Lexer

Perl::Lint::Policy

Pre-Processing

## no lint

## no lintTo retrieve this

Find where (what line) is “## no lint” by regex

Find where (what line) is “## no lint” by regex

And compare between line number of “## no lint” and violation’s one, if match them, ignore form result!

Compiler::Lexer can retrieve comments by verbose mode, but it makes slower about 4 times😢 So using regex

Pre-Process

Lexical Analyze

Syntactic Analyze

Source code (String)

Result

Analyze

Regex

Compiler::Lexer

Perl::Lint::Policy

Tokenize source code by Compiler::Lexer

Compiler::Lexer made of C++ Really fast!

Stable (nowadays)

But…

Perl-5.22………………

Pre-Process

Lexical Analyze

Syntactic Analyze

Source code (String)

Result

Analyze

Regex

Compiler::Lexer

Perl::Lint::Policy

Compiler::Parser exists, but that doesn’t work as expected

Pre-Process

Lexical Analyze

Syntactic Analyze

Source code (String)

Result

Analyze

Regex

Compiler::Lexer

Perl::Lint::Policy

Read token list sequentially and evaluate them. Each policies are responsible for those.

Like this

Like this

Like this…

And it is necessary to analyze contents of regex (m/here!/)

Using Regexp::Lexer This is a module to tokenize regex

Example;

Each policies are independent, so easy to write new policy (You can write your own policy)

Easy and Simple: Scan tokens and write validation processing according to scanned token sequentially

Perl::Lint has filter system

Perl::Lint executes all of the policies by default. Write a black list to ignore any policy.

Current Status

Almost policies of Perl::Critic are available on Perl::Lint

現状のステータス

Documentation is lacked…

Application

Test::Perl::Lint

Testing module like a Test::Perl::Critic

Perl::Lint::Git

Connect git and Perl::Lint to blame the right people for violations.Connect git and Perl::Lint to blame

the right people for violations.

Future works

I should have written a parser… Compiler::Lexer::PP (?)

Enhance documentation

Bug fix

Support new perl notations

Support code climate

CHEATING: Run each policies with pre-fork model

Any Q? (If I can answer…)