Pathologically Polluting Perl(with C, Java and Other Rubbish using Inline.pm)

Brian IngersonActiveState

Thursday June 14th, 19101

Leacock 132

Yet Another Perl Conference

Inline.pm

Pathologically Polluting Perl(with C, Python and Other Rubbish using Inline.pm)

Brian IngersonActiveState

Wednesday July 25th 3:45pm

Grande Ballroom C

The Perl Conference 5

Inline.pm

Part I• Introducing Inline.pm

• C Perl Run

• Whence Inline?

• One-Liners

• The Inline Syntax

• Inline::C

• Supported Platforms

Inline.pm

Introducing Inline.pm• Inline lets you write Perl subroutines ...

• ... in other programming languages!

• Functions, methods and classes too.

• Put some other code directly into your script,

• and simply run it like normal. It just works.

• Inline takes care of all the details!

Inline.pm

C Perl Run• Hello, world

• Hello, foo

• A 'Length'y Discussion

Inline.pm

hello_world.pluse Inline C => <<'END_C';

#include <stdio.h>

void greet() { printf("Hello, world\n");}

END_C

greet;

Inline.pm

hello_world.pluse Inline C => <<'END_C';

#include <stdio.h>

void greet() { printf("Hello, world\n");}

END_C

greet;

Inline.pm

hello_world.pluse Inline C => <<'END_C';

#include <stdio.h>

void greet() { printf("Hello, world\n");}

END_C

greet;

Inline.pm

AUTO_INCLUDE

#include "EXTERN.h"#include "perl.h"#include "XSUB.h"#include "INLINE.h"

Inline.pm

hello_foo.pluse Inline C;

greet('foo');

__END____C__

void greet(char* who) { printf("Hello, %s\n", who);}

Inline.pm

hello_foo.pluse Inline C;

greet('foo');

__END____C__

void greet(char* who) { printf("Hello, %s\n", who);}

Inline.pm

hello_foo.pluse Inline C;

greet('foo');

__END____C__

void greet(char* who) { printf("Hello, %s\n", who);}

Inline.pm

length.pluse Inline C;

open DICT, '/usr/dict/words' or die $!;undef $/;print "The length of /usr/dict/words is ", l(<DICT>), " characters\n";

__END____C__

int l(char* str) { return strlen(str);}

Inline.pm

length.pluse Inline C;

open DICT, '/usr/dict/words' or die $!;undef $/;print "The length of /usr/dict/words is ", len(<DICT>), " characters\n";

__END____C__

int len(char* str) { return strlen(str);}

Inline.pm

Whence Inline?• Inline came forth from Altered States:

Inline.pm

• TPC

• DDC

• PRD

• MDV

• The Perl Conf 4

• Dr. Damian Conway

• Parse::RecDescent

• Digest MD5

• Happy Birthday!

One-liners• Print all primes• perl -le 'while(($_||=1)++){print if(1x$_)!~/^(11+)\1+$/}'

• Crash your Pentium• perl -e 'require

DynaLoader;DynaLoader::dl_install_xsub("main::hangme",unpack("I", pack("P4", "\xF0\x0F\xC7\xC8")));hangme()'

• A scary JAPH• perl -e 'BEGIN{my$x="Knuth heals rare project\n";$^H

{integer}=sub{my$y=shift;$_=substr$x=>$y&0x1F,1;$y>32?uc:lc};$^H=hex join""=>2,1,1,0,0}print 52,2,10,23,16,8,1,19,3,6,15, 12,5,49,21,14,9,11,36,13,22,32,7,18,24;'

Inline.pm

Inline One-liners• JAPH• perl -e 'use Inline C=>q{void J(){printf("Just Another Perl

Hacker\n");}};J'

• JAxH• perl -le 'use Inline C=>q{SV*JAxH(char*x){return

newSVpvf("Just Another %s Hacker",x);}};print JAxH+Perl'

• The Muenzerbrot Setuse Inline C=>'void C(){int m,u,e=0;float l,_,I;for(;1840-e;putchar((+

+e>907 &&942>e?61-m:u)["\n)moc.isc@rezneumb(rezneuM drahnreB"]))for(u=_=l=0;79-(m =e%80)&&I*l+_*_<6&&26-++u;_=2*l*_+e/80*.09-1,l=I)I=l*l-_*_-2+m/27.;}';&C

Inline.pm

The Inline Syntax• Basic Syntax

use Inline C => "source code";

• "String"use Inline C => <<'END_C';

• 'DATA'use Inline C => 'DATA';

• 'FILE'use Inline::Files;

use Inline C => 'FILE';

Inline.pm

The Inline Syntax• Basic Configuration

use Inline C => "source code",

LIBS => '-lfoo',

PREFIX => 'foo_';

• General Config Options– NAME, DIRECTORY

• Language ('C') Config Options– LIBS, INC, AUTO_INCLUDE

Inline.pm

The Inline Syntax• Shorthand

use Inline C => 'DATA';

use Inline C => 'FILE';

use Inline C;

Inline.pm

The Inline Syntax

Inline.pm

• Inline has two functions: bind() and init()• Both functions are rarely used• bind() is like eval() for C code• Takes same args as 'use Inline ...'Inline->bind(C => $code);

• init() is the runtime equiv of INIT()Inline->init;

Inline::C• Basic Perl5 Internals

• Hacking Internals

• Wrapping External Libraries

• The Perl Stack

Inline.pm

Basic Perl5 Internals

Inline.pm

• SV*• The Perl STACK• Typemaps• perlapi.pod

Scalar Values (SV*)

Inline.pm

•http://gisle.aas.no/perl/illguts/

Perl5 API (perlapi.pod)

Inline.pm

SV* newSVpvf(const char* pat, ...)void sv_catpvf(SV* sv, const char* pat, ...)void SvGROW(SV* sv, STRLEN len)SV* newSViv(IV i)SV* sv_2mortal(SV* sv)char* SvPV(SV* sv, STRLEN len)char* SvPV_nolen(SV* sv)U32 SvREFCNT(SV* sv)bool SvROK(SV* sv)bool SvIOK(SV* sv)STRLEN SvLEN(SV* sv)SV* get_sv(const char* name, I32 create)I32 call_pv(const char* sub_name, I32 flags)SV* eval_pv(const char* p, I32 croak_on_err)void croak(const char* pat, ...)

Hacking Internals

Inline.pm

• Inline makes it easy to mess around with Perl internals• Caveat Scriptor!• READONLY• Lost and Found

writable.pl

Inline.pm

use Inline C;$str = \ "Ingy hates Inline\n";writable($$str);$$str =~ s/hate/love/g;print $$str;

__END____C__void writable(SV* x) {SvREADONLY_off(x);}

writable.pl

Inline.pm

use Inline C;$str = \ "Ingy hates Inline\n";writable($$str);$$str =~ s/hate/love/g;print $$str;

__END____C__void writable(SV* x) {SvREADONLY_off(x);}

lostfound.pl

Inline.pm

use Inline C;

$x = q{"Everybody Loves My Ingy"};$x = 42;find_string($x);print "My favorite number is $x\n";

__END____C__void find_string(SV* x) { SvPOK_on(x); }

lostfound.pl

Inline.pm

use Inline C;

$x = q{"Everybody Loves My Ingy"};$x = 42;find_string($x);print "My favorite number is $x\n";

__END____C__void find_string(SV* x) { SvPOK_on(x); }

Wrapping External Libraries

Inline.pm

• LIBS• INC• PREFIX• TYPEMAPS• AUTOWRAP

WinJAPH.pluse Inline C => DATA => LIBS => '-luser32', PREFIX => 'my_';

MessageBoxA('Inline Message Box', 'Just Another Perl Hacker');

__END____C__

#include <windows.h>int my_MessageBoxA(char* Caption, char* Text) { return MessageBoxA(0, Text, Caption, 0);}

Inline.pm

WinJAPH.pluse Inline C => DATA => LIBS => '-luser32', PREFIX => 'my_';

MessageBoxA('Inline Message Box', 'Just Another Perl Hacker');

__END____C__

#include <windows.h>int my_MessageBoxA(char* Caption, char* Text) { return MessageBoxA(0, Text, Caption, 0);}

Inline.pm

localtime.plprint map {"$_\n"} get_localtime(time);

use Inline C => <<'END_OF_C_CODE';#include <time.h>void get_localtime(int utc) { struct tm *ltime = localtime(&utc); Inline_Stack_Vars; Inline_Stack_Reset; Inline_Stack_Push(newSViv(ltime->tm_year)); Inline_Stack_Push(newSViv(ltime->tm_mon)); Inline_Stack_Push(newSViv(ltime->tm_mday)); Inline_Stack_Push(newSViv(ltime->tm_hour)); Inline_Stack_Push(newSViv(ltime->tm_min)); Inline_Stack_Push(newSViv(ltime->tm_sec)); Inline_Stack_Push(newSViv(ltime->tm_isdst)); Inline_Stack_Done;}END_OF_C_CODE

Inline.pm

localtime.plprint map {"$_\n"} get_localtime(time);

use Inline C => <<'END_OF_C_CODE';#include <time.h>void get_localtime(int utc) { struct tm *ltime = localtime(&utc); Inline_Stack_Vars; Inline_Stack_Reset; Inline_Stack_Push(newSViv(ltime->tm_year)); Inline_Stack_Push(newSViv(ltime->tm_mon)); Inline_Stack_Push(newSViv(ltime->tm_mday)); Inline_Stack_Push(newSViv(ltime->tm_hour)); Inline_Stack_Push(newSViv(ltime->tm_min)); Inline_Stack_Push(newSViv(ltime->tm_sec)); Inline_Stack_Push(newSViv(ltime->tm_isdst)); Inline_Stack_Done;}END_OF_C_CODE

Inline.pm

Supported Platforms for C• All Modern Unix Variants

• All Modern Windows & Cygwin

• Requires Perl's Build Environment

• Binary Distribution Support

• Works with Mingw32/gcc on Win32

• Working on VMS et al.

• Supports Perl 5.005_03 and up.

Inline.pm

Part II• Object Oriented C

• Inline C++

• How Inline Works

• Inline::Python

• Inline::Java

• Inline::ASM

Inline.pm

Object Oriented C• Perl can use C based objects

• The OO behaviour is handled by Perl

• The data is on accessible from C

Inline.pm

oopc.plmy $obj1 = Soldier->new('Benjamin', 'Private', 11111);my $obj2 = Soldier->new('Sanders', 'Colonel', 22222);my $obj3 = Soldier->new('Matt', 'Sergeant', 33333);

for my $obj ($obj1, $obj2, $obj3) { print ($obj->get_serial, ") ", $obj->get_name, " is a ", $obj->get_rank, "\n");}

Inline.pm

oopc.plpackage Soldier;use Inline C => <<'END';typedef struct { char* name; char* rank; long serial;} Soldier;

Inline.pm

oopc.pl

Inline.pm

SV* new(char* class, char* name, char* rank, long serial) { Soldier* soldier = malloc(sizeof(Soldier)); SV* obj_ref = newSViv(0); SV* obj = newSVrv(obj_ref, class); soldier->name = strdup(name); soldier->rank = strdup(rank); soldier->serial = serial;

sv_setiv(obj, (IV)soldier); SvREADONLY_on(obj); return obj_ref;}

oopc.pl

char* get_name(SV* obj) { return ((Soldier*)SvIV(SvRV(obj)))->name;}

char* get_rank(SV* obj) { return ((Soldier*)SvIV(SvRV(obj)))->rank;}

long get_serial(SV* obj) { return ((Soldier*)SvIV(SvRV(obj)))->serial;}

Inline.pm

oopc.plvoid DESTROY(SV* obj) { Soldier* soldier = (Soldier*)SvIV(SvRV(obj)); free(soldier->name); free(soldier->rank); free(soldier);}

Inline.pm

Some Ware Beyond the C• The primary goal of Inline is:

– "Make it easy to use other programming languages with Perl"

• This is not limited to C.

• Drumroll please…

Inline.pm

Enter Neil Watkiss!

Inline.pm

Inline::CPP

Inline.pm

use Inline 'C++';my $obj1 = Soldier->new('Benjamin', 'Private', 11111);__END____C++__class Soldier { public: Soldier(char *name, char *rank, int serial); char *get_name(); char *get_rank(); int get_serial(); private: char *name; char *rank; int serial;};

How Inline Works• Inline's implementation is simple.

• It makes use of existing Perl tools.

• It just does a *lot* of little things.

• It's magic.

Inline.pm

The Friends of Inline• Parse::RecDescent

• Digest::MD5

• Config.pm

• XS

• ExtUtils::MakeMaker

• DynaLoader

• perl !

Inline.pm

The C Parse::RecDescent Grammar

Inline.pm

c_code: part(s) {1}part: comment | function_definition { my $function = $item[1]->[0]; push @{$thisparser->{data}->{functions}}, $function; -- lines deleted -- } | anything_elsecomment: m{\s* // [^\n]* \n }x | m{\s* /\* (?:[^*]+|\*(?!/))* \*/ ([ \t]*)? }xfunction_definition: -- lines deleted --anything_else: /.*/

How Inline C Works1. Name, NAME, AUTONAME2. Dust for Fingerprints (.inl)3. Directory Assistance4. Build

1. Parse2. Glue3. Compile4. Cache

5. Load (and Bind)

Inline.pm

How Inline Foo Works1. Name, NAME, AUTONAME2. Dust for Fingerprints (.inl)3. Directory Assistance4. Build

1. Parse2. Glue3. Compile4. Cache

5. Load (and Bind)

Inline.pm

The 'config' file

Inline.pm

version : 0.40languages : % C : C C++ : CPP CPP : CPPtypes : % C : compiled CPP : compiledmodules : % C : Inline::C CPP : Inline::CPPsuffixes : % C : so CPP : so

The '.inl' file

Inline.pm

md5 : 2cdb20c55cc56102fd68db27620832aename : japhlanguage : Clanguage_id : Cdate_compiled : Fri Jun 1 01:15:55 2001inline_version : 0.40ILSM : % module : Inline::C suffix : so type : compiledConfig : % apiversion : 5.6.1 archname : i686-linux-thread-multi cc : gcc osname : linux osvers : 2.2.17 so : so version : 5.6.1

Inline::Python

Inline.pm

snake.pluse Inline Python;my $language = shift;print $language, (match($language, 'Perl') ? ' rules' : ' sucks'), "!\n";__END____Python__import sysimport redef match(str, regex): f = re.compile(regex); if f.match(str): return 1 return 0

Inline.pm

Inline::Java

Inline.pm

•Meet Patrick LeBoutillier

squarer.java

public class squarer { public squarer() {} public double square(double x) { return x * x ; }}

Inline.pm

squarer.pluse strict;use Inline Java => 'DATA', CLASSPATH => '/usr/local/j????';

my $s = new my_squarer();my $n = 6;

print "$n squared is ", $s->square($n), "\n";

__END____Java__class my_squarer extends squarer { public my_squarer() { super(); }}

Inline.pm

yasquarer.pluse strict;

use Inline Java => 'STUDY', CLASSPATH =>'/usr/local/j????', STUDY => ['squarer'];

my $s = new squarer();my $n = 7;

print "$n squared is ", $s->square($n), "\n" ;

Inline.pm

Inline::ASM• Inline Assembly Language

• Seriously

Inline.pm

asm_japh.pluse Inline ASM => DATA => PROTOTYPES => {JAxH => 'void(char*)'};print JAxH('Perl');__END____C__ BITS 32 GLOBAL JAxH EXTERN printf SECTION .textJAxH push ebp mov ebp,esp mov eax,[ebp+8] push dword eax push dword jaxhstr call printf mov esp,ebp pop ebp ret SECTION .datajaxhstr db "Just Another %s Hacker", 10, 0

Inline.pm

Part III• The Inline API

• Inline::Foo

• Inline::PERL

• Inline::Perl

• Inline::MakeMaker

• Inline::CPR

• Future Plans

Inline.pm

The Inline API• Rolling Your Own

• Almost any language can be Inlined

• 5 degrees of separation

• Actually 5 methods

• Inline-API.pod

Inline.pm

The Inline API• ILSMs are subclasses of Inline

• Named Inline::Foo

• register()

• validate()

• build()

• load()

• info()

Inline.pm

Inline::Foo

Inline.pm

•It really exists

•I use it to test Inline

•You're soaking in it

t/01usages.t

use Test; plan( tests => 1);

# test 5# Make sure language name aliases work ('foo'

instead of 'Foo')

ok(test5('test5'));

use Inline foo => <<'END_OF_FOO';foo-sub test5 { foo-return $_[0] foo-eq 'test5';}END_OF_FOO

Inline.pm

Inline/Foo.pm

package Inline::Foo;$VERSION = '0.01';require Inline;@ISA = qw(Inline);use strict;use Carp;

Inline.pm

Inline/Foo.pm

sub register { return { language => 'Foo', aliases => ['foo'], type => 'interpreted', suffix => 'foo', };}

Inline.pm

Inline/Foo.pm sub validate { my $o = shift; $o->{ILSM}{PATTERN} ||= 'foo-'; $o->{ILSM}{BAR} ||= 0; while (@_) { my ($key, $value) = splice @_, 0, 2; if ($key eq 'PATTERN') { $o->{ILSM}{PATTERN} = $value; next; } if ($key eq 'BAR') { croak usage_config_bar unless $value =~ /^[01]$/; $o->{ILSM}{BAR} = $value; next; } croak usage_config($key); }}

Inline.pm

Inline/Foo.pm sub build { my $o = shift; my $code = $o->{API}{code}; my $pattern = $o->{ILSM}{PATTERN}; $code =~ s/$pattern//g; $code =~ s/bar-//g if $o->{ILSM}{BAR}; sleep 1; # imitate compile delay { package Foo::Tester; eval $code; } croak "Foo build failed:\n$@" if $@; my $path = "$o->{API}{install_lib}/auto/". "$o->{API}{modpname}"; my $obj = $o->{API}{location}; $o->mkpath($path) unless -d $path; open FOO_OBJ, "> $obj" or croak "Can't open $obj for output\n$!"; print FOO_OBJ $code; close \*FOO_OBJ;}

Inline.pm

Inline/Foo.pm sub load { my $o = shift; my $obj = $o->{API}{location}; open FOO_OBJ, "< $obj" or croak "Can't open $obj for output\n$!"; my $code = join '', <FOO_OBJ>; close \*FOO_OBJ; eval "package $o->{API}{pkg};\n$code"; croak "Unable to load Foo module $obj:\n$@" if $@;}

Inline.pm

Inline::PERL

Inline.pm

•Bring the power of PERL programming,

•to your Perl programs!

•John McNamara

•Posted on perlmonks.net

•Available as Acme-Inline-Perl

•It's on its way to CPAN...

Inline/PERL.pm sub register { return { language => 'PERL', aliases => ['PEARL', 'CGI'], type => 'interpreted', suffix => 'PL', };}sub build { my $o = shift; my $code = $o->{API}{code}; if ($code =~ /strict/) { carp <<END;Your code contains "use strict;". Please be aware thatthis may provide you with too much of a clue.';END

Inline.pm

Inline::Perl

Inline.pm

•Bind another Perl to Perl

•Legacy Perls

•Like Perl 4

•or Perl 5.8

•or Perl 5+i

Inline CPAN Modules

Inline.pm

•Write an Inline Module

•Add a NAME and VERSION

•use Inline::MakeMaker;

•make dist

•Equivalent to XS

PerlAPI.pmpackage PerlAPI;

use strict;require Exporter;@PerlAPI::ISA = qw(Exporter);@PerlAPI::EXPORT = qw(SvREADONLY_off);$PerlAPI::VERSION = '0.42';use Inline C => DATA => PREFIX => 'my_', NAME => 'PerlAPI', VERSION => '0.42';

1;__DATA____C__void my_SvREADONLY_off(SV* x)

{SvREADONLY_off(x);}

Inline.pm

Makefile.PL use Inline::MakeMaker;

WriteInlineMakefile( NAME => 'PerlAPI', VERSION_FROM => 'PerlAPI.pm',);

Inline.pm

test.pluse Test;use PerlAPI;plan(tests => 1);

$str = \ "Ingy hates Inline\n";SvREADONLY_off($$str);$$str =~ s/hate/love/g;

ok($$str =~ /loves/);

Inline.pm

See Perl Run. Run Perl, Run!

Inline.pm

Soooo….• Inline is a great way to extend Perl w/ C

• But...

• How can we easily embed Perl in C?

Inline.pm

What if….• We could pass plain C to Perl,

• And Perl could pass it to Inline,

• And Inline could bind to main(),

• And just invoke main() from Perl.

Inline.pm

Introducing CPR• "C Perl Run"

• The C interpreter

• A brand new programming language

• That combines all of C

• With the guts of Perl

• #!usr/bin/cpr

Inline.pm

Hello_World.cpr

Inline.pm

#!/usr/bin/cprint main(void) { printf("Hello, world\n");}

Hello_Perl.cpr

Inline.pm

#!/usr/local/bin/cprint main(void) { printf("Hello World, I'm running under Perl

version %s\n", CPR_eval("use Config; $Config{version}") ); return 0;}

Hello_Cruel_World.cpr

Inline.pm

#!/usr/bin/cprint main(void) { CPR_eval("use Inline (C => q{ char* greet() { return \"Hello world\"; } })"); printf("%s, I'm running under Perl version %s\n", CPR_eval("&greet"), CPR_eval("use Config; $Config{version}")); return 0;}

Future Plans• A new frontier for Perl

• More ILSMs

• Inline 0.50

• Development Tools

• You!

Inline.pm

Planned ILSMs

Inline.pm

• Guile• Scheme• Fortran• Lisp• Ruby• R• Bash• Perl

Inline 0.50 Plans

Inline.pm

• XS like modules• Viral Inline stub• No core requirement• Support all versions equally• Encourage hacking and collaboration

Development Tools

Inline.pm

• The Inline Debugger• Module authoring automation

• Binary distributions

Inline Needs YOU!

Inline.pm

• inline@perl.org• http://inline.perl.org

Questions

Inline.pm