an implementation of python for racket · an implementation of python for racket pedro palma ramos...
TRANSCRIPT
![Page 1: An Implementation of Python for Racket · An Implementation of Python for Racket Pedro Palma Ramos António Menezes Leitão](https://reader030.vdocuments.net/reader030/viewer/2022040707/5e07e11afc786122af64a373/html5/thumbnails/1.jpg)
An Implementation of Python for Racket
Pedro Palma Ramos
António Menezes Leitão
![Page 2: An Implementation of Python for Racket · An Implementation of Python for Racket Pedro Palma Ramos António Menezes Leitão](https://reader030.vdocuments.net/reader030/viewer/2022040707/5e07e11afc786122af64a373/html5/thumbnails/2.jpg)
Contents
• Motivation
• Goals
• Related Work
• Solution
• Performance Benchmarks
• Future Work
Pedro Ramos & António Menezes Leitão | An Implementation of Python for Racket
2
![Page 3: An Implementation of Python for Racket · An Implementation of Python for Racket Pedro Palma Ramos António Menezes Leitão](https://reader030.vdocuments.net/reader030/viewer/2022040707/5e07e11afc786122af64a373/html5/thumbnails/3.jpg)
Racket + DrRacket
Pedro Ramos & António Menezes Leitão | An Implementation of Python for Racket
3
![Page 4: An Implementation of Python for Racket · An Implementation of Python for Racket Pedro Palma Ramos António Menezes Leitão](https://reader030.vdocuments.net/reader030/viewer/2022040707/5e07e11afc786122af64a373/html5/thumbnails/4.jpg)
Racket + DrRacket
Pedro Ramos & António Menezes Leitão | An Implementation of Python for Racket
4
![Page 5: An Implementation of Python for Racket · An Implementation of Python for Racket Pedro Palma Ramos António Menezes Leitão](https://reader030.vdocuments.net/reader030/viewer/2022040707/5e07e11afc786122af64a373/html5/thumbnails/5.jpg)
Racket + DrRacket
Pedro Ramos & António Menezes Leitão | An Implementation of Python for Racket
5
![Page 6: An Implementation of Python for Racket · An Implementation of Python for Racket Pedro Palma Ramos António Menezes Leitão](https://reader030.vdocuments.net/reader030/viewer/2022040707/5e07e11afc786122af64a373/html5/thumbnails/6.jpg)
Our goal...
Pedro Ramos & António Menezes Leitão | An Implementation of Python for Racket
6
![Page 7: An Implementation of Python for Racket · An Implementation of Python for Racket Pedro Palma Ramos António Menezes Leitão](https://reader030.vdocuments.net/reader030/viewer/2022040707/5e07e11afc786122af64a373/html5/thumbnails/7.jpg)
Our goal...
Pedro Ramos & António Menezes Leitão | An Implementation of Python for Racket
7
![Page 8: An Implementation of Python for Racket · An Implementation of Python for Racket Pedro Palma Ramos António Menezes Leitão](https://reader030.vdocuments.net/reader030/viewer/2022040707/5e07e11afc786122af64a373/html5/thumbnails/8.jpg)
Pedro Ramos & António Menezes Leitão | An Implementation of Python for Racket
8
Why Python?
Python is replacing Scheme in introductory programming courses
![Page 9: An Implementation of Python for Racket · An Implementation of Python for Racket Pedro Palma Ramos António Menezes Leitão](https://reader030.vdocuments.net/reader030/viewer/2022040707/5e07e11afc786122af64a373/html5/thumbnails/9.jpg)
Rosetta IDE
Pedro Ramos & António Menezes Leitão | An Implementation of Python for Racket
9
![Page 10: An Implementation of Python for Racket · An Implementation of Python for Racket Pedro Palma Ramos António Menezes Leitão](https://reader030.vdocuments.net/reader030/viewer/2022040707/5e07e11afc786122af64a373/html5/thumbnails/10.jpg)
Rosetta IDE
Pedro Ramos & António Menezes Leitão | An Implementation of Python for Racket
10
AUTOLISP
Rosetta (Racket)
Front ends: Back ends:
![Page 11: An Implementation of Python for Racket · An Implementation of Python for Racket Pedro Palma Ramos António Menezes Leitão](https://reader030.vdocuments.net/reader030/viewer/2022040707/5e07e11afc786122af64a373/html5/thumbnails/11.jpg)
• Borrows influences from Lisp
• High level, dynamically typed, GC’d
• Multiple paradigms
• Huge standard library + third-party libraries
Pedro Ramos & António Menezes Leitão | An Implementation of Python for Racket
11
![Page 12: An Implementation of Python for Racket · An Implementation of Python for Racket Pedro Palma Ramos António Menezes Leitão](https://reader030.vdocuments.net/reader030/viewer/2022040707/5e07e11afc786122af64a373/html5/thumbnails/12.jpg)
Goals
• Correctness + Completeness
• Performance
• DrRacket Integration
• Interoperability with Racket
Pedro Ramos & António Menezes Leitão | An Implementation of Python for Racket
12
![Page 13: An Implementation of Python for Racket · An Implementation of Python for Racket Pedro Palma Ramos António Menezes Leitão](https://reader030.vdocuments.net/reader030/viewer/2022040707/5e07e11afc786122af64a373/html5/thumbnails/13.jpg)
Related implementations
Pedro Ramos & António Menezes Leitão | An Implementation of Python for Racket
13
Language(s) written
Platform(s) targetted
Speedup (vs. CPython)
Std. library support
CPython C CPython’s VM 1x Full
![Page 14: An Implementation of Python for Racket · An Implementation of Python for Racket Pedro Palma Ramos António Menezes Leitão](https://reader030.vdocuments.net/reader030/viewer/2022040707/5e07e11afc786122af64a373/html5/thumbnails/14.jpg)
Related implementations
Pedro Ramos & António Menezes Leitão | An Implementation of Python for Racket
14
Language(s) written
Platform(s) targetted
Speedup (vs. CPython)
Std. library support
CPython C CPython’s VM 1x Full
Jython Java JVM ~1x Most
IronPython C# CLI ~1.8x Most
CLPython Common Lisp Common Lisp ~0.5x Most
![Page 15: An Implementation of Python for Racket · An Implementation of Python for Racket Pedro Palma Ramos António Menezes Leitão](https://reader030.vdocuments.net/reader030/viewer/2022040707/5e07e11afc786122af64a373/html5/thumbnails/15.jpg)
Related implementations
Pedro Ramos & António Menezes Leitão | An Implementation of Python for Racket
15
Language(s) written
Platform(s) targetted
Speedup (vs. CPython)
Std. library support
CPython C CPython’s VM 1x Full
Jython Java JVM ~1x Most
IronPython C# CLI ~1.8x Most
CLPython Common Lisp Common Lisp ~0.5x Most
PLT Spy Scheme, C Scheme ~0.001x Full
![Page 16: An Implementation of Python for Racket · An Implementation of Python for Racket Pedro Palma Ramos António Menezes Leitão](https://reader030.vdocuments.net/reader030/viewer/2022040707/5e07e11afc786122af64a373/html5/thumbnails/16.jpg)
Our solution...
Pedro Ramos & António Menezes Leitão | An Implementation of Python for Racket
16
![Page 17: An Implementation of Python for Racket · An Implementation of Python for Racket Pedro Palma Ramos António Menezes Leitão](https://reader030.vdocuments.net/reader030/viewer/2022040707/5e07e11afc786122af64a373/html5/thumbnails/17.jpg)
Pipeline
Pedro Ramos & António Menezes Leitão | An Implementation of Python for Racket
17
.py .rkt
Python → Racket (source-to-source compiler)
Racket → Bytecode (Racket compiler + JIT)
![Page 18: An Implementation of Python for Racket · An Implementation of Python for Racket Pedro Palma Ramos António Menezes Leitão](https://reader030.vdocuments.net/reader030/viewer/2022040707/5e07e11afc786122af64a373/html5/thumbnails/18.jpg)
Architecture
Pedro Ramos & António Menezes Leitão | An Implementation of Python for Racket
18
![Page 19: An Implementation of Python for Racket · An Implementation of Python for Racket Pedro Palma Ramos António Menezes Leitão](https://reader030.vdocuments.net/reader030/viewer/2022040707/5e07e11afc786122af64a373/html5/thumbnails/19.jpg)
Racket Modules
• reader module (for compilation)
– read: input-port (listof s-expression)
– read-syntax: input-port (listof syntax-object)
• python module (for runtime behaviour)
– Provides functions/macros used in compiled code
Pedro Ramos & António Menezes Leitão | An Implementation of Python for Racket
19
?
![Page 20: An Implementation of Python for Racket · An Implementation of Python for Racket Pedro Palma Ramos António Menezes Leitão](https://reader030.vdocuments.net/reader030/viewer/2022040707/5e07e11afc786122af64a373/html5/thumbnails/20.jpg)
Syntax-objects
• S-expression
• Source location information
– File, line number, column number, span
• Lexical-binding information
Pedro Ramos & António Menezes Leitão | An Implementation of Python for Racket
20
![Page 21: An Implementation of Python for Racket · An Implementation of Python for Racket Pedro Palma Ramos António Menezes Leitão](https://reader030.vdocuments.net/reader030/viewer/2022040707/5e07e11afc786122af64a373/html5/thumbnails/21.jpg)
Syntax-objects
Pedro Ramos & António Menezes Leitão | An Implementation of Python for Racket
21
(py-print (py-get-index arr 6)) line: 3, cols: 0-12
py-print (py-get-index arr 6) line: 3, cols: 6-12
py-get-index arr line: 3, cols: 6-9
6 line: 3, cols: 10-11
![Page 22: An Implementation of Python for Racket · An Implementation of Python for Racket Pedro Palma Ramos António Menezes Leitão](https://reader030.vdocuments.net/reader030/viewer/2022040707/5e07e11afc786122af64a373/html5/thumbnails/22.jpg)
Architecture
Pedro Ramos & António Menezes Leitão | An Implementation of Python for Racket
22
![Page 23: An Implementation of Python for Racket · An Implementation of Python for Racket Pedro Palma Ramos António Menezes Leitão](https://reader030.vdocuments.net/reader030/viewer/2022040707/5e07e11afc786122af64a373/html5/thumbnails/23.jpg)
How to implement Python’s behaviour?
Pedro Ramos & António Menezes Leitão | An Implementation of Python for Racket
23
![Page 24: An Implementation of Python for Racket · An Implementation of Python for Racket Pedro Palma Ramos António Menezes Leitão](https://reader030.vdocuments.net/reader030/viewer/2022040707/5e07e11afc786122af64a373/html5/thumbnails/24.jpg)
Runtime implementation
• Mapping to Python/C API
(via Racket Foreign Function Interface)
• Racket reimplementation
Pedro Ramos & António Menezes Leitão | An Implementation of Python for Racket
24
Two alternatives:
![Page 25: An Implementation of Python for Racket · An Implementation of Python for Racket Pedro Palma Ramos António Menezes Leitão](https://reader030.vdocuments.net/reader030/viewer/2022040707/5e07e11afc786122af64a373/html5/thumbnails/25.jpg)
Architecture
Pedro Ramos & António Menezes Leitão | An Implementation of Python for Racket
25
![Page 26: An Implementation of Python for Racket · An Implementation of Python for Racket Pedro Palma Ramos António Menezes Leitão](https://reader030.vdocuments.net/reader030/viewer/2022040707/5e07e11afc786122af64a373/html5/thumbnails/26.jpg)
FFI Approach
Pedro Ramos & António Menezes Leitão | An Implementation of Python for Racket
26
Racket CPython VM
Racket FFI (Foreign Function Interface)
Python/C API
foreign calls on C pointers
libpython module
![Page 27: An Implementation of Python for Racket · An Implementation of Python for Racket Pedro Palma Ramos António Menezes Leitão](https://reader030.vdocuments.net/reader030/viewer/2022040707/5e07e11afc786122af64a373/html5/thumbnails/27.jpg)
(define (py-add x y)
(PyObject_CallObject (PyObject_GetAttrString x "__add__")
(make-py-tuple y)))
(define (make-py-tuple . elems)
(let ([py-tuple (PyTuple_New (length elems))])
(for ([i (range (length elems))]
[elem elems])
(PyTuple_SetItem py-tuple i elem))
py-tuple))
FFI Runtime - Example
Pedro Ramos & António Menezes Leitão | An Implementation of Python for Racket
27
x + y
![Page 28: An Implementation of Python for Racket · An Implementation of Python for Racket Pedro Palma Ramos António Menezes Leitão](https://reader030.vdocuments.net/reader030/viewer/2022040707/5e07e11afc786122af64a373/html5/thumbnails/28.jpg)
FFI Runtime - Disadvantages
• Bad Performance
– Expensive type conversions + FFI calls
– Finalizers for GC
• Clumsy Interoperability with Racket
– Wrappers/Unwrappers
Pedro Ramos & António Menezes Leitão | An Implementation of Python for Racket
28
![Page 29: An Implementation of Python for Racket · An Implementation of Python for Racket Pedro Palma Ramos António Menezes Leitão](https://reader030.vdocuments.net/reader030/viewer/2022040707/5e07e11afc786122af64a373/html5/thumbnails/29.jpg)
What about implementing it over Racket data types?
Pedro Ramos & António Menezes Leitão | An Implementation of Python for Racket
We must first understand Python’s data model
29
![Page 30: An Implementation of Python for Racket · An Implementation of Python for Racket Pedro Palma Ramos António Menezes Leitão](https://reader030.vdocuments.net/reader030/viewer/2022040707/5e07e11afc786122af64a373/html5/thumbnails/30.jpg)
Python’s Data Model
• Every value is an object
• Every object has a reference to its type-object
• Type-objects hold hash-table for method dispatching
– Maps method names to function objects
• Operator behaviour is mapped to methods
Pedro Ramos & António Menezes Leitão | An Implementation of Python for Racket
30
![Page 31: An Implementation of Python for Racket · An Implementation of Python for Racket Pedro Palma Ramos António Menezes Leitão](https://reader030.vdocuments.net/reader030/viewer/2022040707/5e07e11afc786122af64a373/html5/thumbnails/31.jpg)
Optimizations
• Basic types mapped to Racket types
– int, long, float, complex, string, dict
– Avoids wrapping/unwrapping
• Early method dispatching for operators
– Avoids expensive method dispatching for common uses
Pedro Ramos & António Menezes Leitão | An Implementation of Python for Racket
31
![Page 32: An Implementation of Python for Racket · An Implementation of Python for Racket Pedro Palma Ramos António Menezes Leitão](https://reader030.vdocuments.net/reader030/viewer/2022040707/5e07e11afc786122af64a373/html5/thumbnails/32.jpg)
Racket Runtime - Example
Pedro Ramos & António Menezes Leitão | An Implementation of Python for Racket
32
(define (py-add x y)
(py-method-call x "__add__" y))
(define (py-add x y)
(cond
[(and (number? x) (number? y)) (+ x y)]
[(and (string? x) (string? y)) (string-append x y)]
[else (py-method-call x "__add__" y)]))
x + y
![Page 33: An Implementation of Python for Racket · An Implementation of Python for Racket Pedro Palma Ramos António Menezes Leitão](https://reader030.vdocuments.net/reader030/viewer/2022040707/5e07e11afc786122af64a373/html5/thumbnails/33.jpg)
How are modules imported?
Pedro Ramos & António Menezes Leitão | An Implementation of Python for Racket
33
![Page 34: An Implementation of Python for Racket · An Implementation of Python for Racket Pedro Palma Ramos António Menezes Leitão](https://reader030.vdocuments.net/reader030/viewer/2022040707/5e07e11afc786122af64a373/html5/thumbnails/34.jpg)
Python import system
• import <module>
– <module> is imported as a module object
• from <module> import <id>
– <id> is imported as a new binding
• from <module> import *
– All bindings from <module> are imported
Pedro Ramos & António Menezes Leitão | An Implementation of Python for Racket
34
module->exports +
dynamic-require
require
• Special syntax for Racket imports
![Page 35: An Implementation of Python for Racket · An Implementation of Python for Racket Pedro Palma Ramos António Menezes Leitão](https://reader030.vdocuments.net/reader030/viewer/2022040707/5e07e11afc786122af64a373/html5/thumbnails/35.jpg)
Import - Example
Pedro Ramos & António Menezes Leitão | An Implementation of Python for Racket
35
#lang python
import "racket" as rkt
def add_cons(c):
return rkt.car(c) + rkt.cdr(c)
c1 = rkt.cons(2, 3)
c2 = rkt.cons("abc", "def")
> add_cons(c1)
5
> add_cons(c2)
"abcdef"
![Page 36: An Implementation of Python for Racket · An Implementation of Python for Racket Pedro Palma Ramos António Menezes Leitão](https://reader030.vdocuments.net/reader030/viewer/2022040707/5e07e11afc786122af64a373/html5/thumbnails/36.jpg)
Import - Example
Pedro Ramos & António Menezes Leitão | An Implementation of Python for Racket
36
#lang python
from "racket" import cons, car, cdr
def add_cons(c):
return car(c) + cdr(c)
c1 = cons(2, 3)
c2 = cons("abc", "def")
> add_cons(c1)
5
> add_cons(c2)
"abcdef"
![Page 37: An Implementation of Python for Racket · An Implementation of Python for Racket Pedro Palma Ramos António Menezes Leitão](https://reader030.vdocuments.net/reader030/viewer/2022040707/5e07e11afc786122af64a373/html5/thumbnails/37.jpg)
Import - Example (Macros)
Pedro Ramos & António Menezes Leitão | An Implementation of Python for Racket
37
#lang python
from "racket/trace" import trace
def factorial(n):
if n == 0: return 1
else: return n * factorial(n-1)
trace(factorial)
> factorial(5) >(factorial 5) > (factorial 4) > >(factorial 3) > > (factorial 2) > > >(factorial 1) > > > (factorial 0) < < < 1 < < <1 < < 2 < <6 < 24 <120 120
![Page 38: An Implementation of Python for Racket · An Implementation of Python for Racket Pedro Palma Ramos António Menezes Leitão](https://reader030.vdocuments.net/reader030/viewer/2022040707/5e07e11afc786122af64a373/html5/thumbnails/38.jpg)
Other Features
• Class definitions
– class statement new type object
• Exception handling
– raise, try...except statements raise, with-handlers forms
• Flow control statements
– return, break, continue, yield escape continuations
Pedro Ramos & António Menezes Leitão | An Implementation of Python for Racket
38
![Page 39: An Implementation of Python for Racket · An Implementation of Python for Racket Pedro Palma Ramos António Menezes Leitão](https://reader030.vdocuments.net/reader030/viewer/2022040707/5e07e11afc786122af64a373/html5/thumbnails/39.jpg)
Benchmarks
• Ackermann
– computing the Ackermann function
• Mandelbrot
– computing if a complex sequence diverges after a limited number
of iterations
Pedro Ramos & António Menezes Leitão | An Implementation of Python for Racket
39
![Page 40: An Implementation of Python for Racket · An Implementation of Python for Racket Pedro Palma Ramos António Menezes Leitão](https://reader030.vdocuments.net/reader030/viewer/2022040707/5e07e11afc786122af64a373/html5/thumbnails/40.jpg)
Ackermann
Pedro Ramos & António Menezes Leitão | An Implementation of Python for Racket
40
(define (ackermann m n) (cond [(= m 0) (+ n 1)] [(and (> m 0) (= n 0)) (ackermann (- m 1) 1)] [else (ackermann (- m 1) (ackermann m (- n 1)))])) (ackermann 3 9)
def ackermann(m,n): if m == 0: return n+1 elif m > 0 and n == 0: return ackermann(m-1,1) else: return ackermann(m-1, ackermann(m,n-1)) print ackermann(3,9)
![Page 41: An Implementation of Python for Racket · An Implementation of Python for Racket Pedro Palma Ramos António Menezes Leitão](https://reader030.vdocuments.net/reader030/viewer/2022040707/5e07e11afc786122af64a373/html5/thumbnails/41.jpg)
Mandelbrot
Pedro Ramos & António Menezes Leitão | An Implementation of Python for Racket
41
(define (mandelbrot limit c) (let loop ([i 0] [z 0+0i]) (cond [(> i limit) i] [(> (magnitude z) 2) i] [else (loop (add1 i) (+ (* z z) c))]))) (mandelbrot 1000000 .2+.3i)
def mandelbrot(limit, c): z = 0+0j for i in range(limit+1): if abs(z) > 2: return i z = z*z + c return i+1 print mandelbrot(1000000, .2+.3j)
![Page 42: An Implementation of Python for Racket · An Implementation of Python for Racket Pedro Palma Ramos António Menezes Leitão](https://reader030.vdocuments.net/reader030/viewer/2022040707/5e07e11afc786122af64a373/html5/thumbnails/42.jpg)
Ackermann - Results
Pedro Ramos & António Menezes Leitão | An Implementation of Python for Racket
42
93 2641
54351
115557
11357
811 0
20.000
40.000
60.000
80.000
100.000
120.000
(a) (b) (c.1) (c.2) (d.1) (d.2)
Mili
seco
nd
s
■ (a) Racket on Racket
■ (b) Python on CPython
■ (c.1) Python on Racket (FFI)
■ (c.2) Python on Racket (FFI + finalizers)
■ (d.1) Python on Racket (native)
■ (d.2) Python on Racket (native + early dispatch)
![Page 43: An Implementation of Python for Racket · An Implementation of Python for Racket Pedro Palma Ramos António Menezes Leitão](https://reader030.vdocuments.net/reader030/viewer/2022040707/5e07e11afc786122af64a373/html5/thumbnails/43.jpg)
Mandelbrot - Results
Pedro Ramos & António Menezes Leitão | An Implementation of Python for Racket
43
406 359
7145
19126
1045 765
0
2.000
4.000
6.000
8.000
10.000
12.000
14.000
16.000
18.000
20.000
(a) (b) (c.1) (c.2) (d.1) (d.2)
Mili
seco
nd
s
■ (a) Racket on Racket
■ (b) Python on CPython
■ (c.1) Python on Racket (FFI)
■ (c.2) Python on Racket (FFI + finalizers)
■ (d.1) Python on Racket (native)
■ (d.2) Python on Racket (native + early dispatch)
![Page 44: An Implementation of Python for Racket · An Implementation of Python for Racket Pedro Palma Ramos António Menezes Leitão](https://reader030.vdocuments.net/reader030/viewer/2022040707/5e07e11afc786122af64a373/html5/thumbnails/44.jpg)
Future Work
• Fully implement compilation process
• Implement behaviour for built-in types
• Integrate FFI calls with current data model
• Formal testing for correctness
Pedro Ramos & António Menezes Leitão | An Implementation of Python for Racket
44
![Page 45: An Implementation of Python for Racket · An Implementation of Python for Racket Pedro Palma Ramos António Menezes Leitão](https://reader030.vdocuments.net/reader030/viewer/2022040707/5e07e11afc786122af64a373/html5/thumbnails/45.jpg)
Thank you for listening! Questions? Comments?
Pedro Ramos & António Menezes Leitão | An Implementation of Python for Racket
45