Uniconfrom Application to Evolution

Clint Jeffery

University of Idaho

jeffery@uidaho.edu

In this talk...

A “cookie talk” presented at AT&T Labs Research in Florham Park NJ, 1/27/10 nperf, an important application

Progress, plans, and the grand vision for the Unicon programming language research opportunities evolving the language and its tools

Implementing a Network Analysis Tool Using a Very High

Level Language

Kostas Oikonomou Clint Jeffery

AT&T Labs Research University of Idaho

ko@research.att.com jeffery@uidaho.edu

Outline (for the cookie talk)

Motivation - nperf Background - Unicon Performance discussion Results of AT&T – U Idaho collaboration Next steps Conclusions

nperf: software block diagram

nperf – the program

Kostas Oikonomou, Rakesh Sinha A literate program written in Icon• migrating gradually to Unicon for OO• C libraries for key components

Successful experimental capabilities Now, how to scale it up• Speed, even with tuned C critical parts• Belady-Lehman pain curve

nperf Implementation

“small”• 21 KLOC of noweb files• 14 KLOC Icon + 700 LOC C

Numeric-intensive• Not surprising, but not Icon's forté

Not just matrices• Matrices of structs (or objects)

From Snobol to Icon to Unicon

“Very high level” programming languages enable and accelerate experimental research

Successive domain generalizations Snobol, AT&T, 1960-• pattern matching language

Icon, U. of Arizona, 1980-• goal-directed imperative

Unicon, Unicon.org, 2000-• OO and IO

Unicon vs. Perl vs. Pythonadapted from a comp.lang.python posting

class MyClass:AnotherClass(name)

method greet()

write(“Hi, I'm “, name)

end

end

procedure main()

m := MyClass(“Bob”)

m.greet()

end

class MyClass(AnotherClass):

def __init__(self, name):

self.name = name

def greet(self):

print “Hi, I'm “, self.name

m = MyClass('Bob')

m.greet()

package MyClass;

@ISA = ('AnotherClass');

sub new {

($class, $name) = @_;

$self = {};

bless $self, $class;

$self->{'name'} = $name;

return $self;

}

sub greet {

($self) = @_;

Print “Hi, I'm “, $self->{'name'}, “\n”;

}

$a = MyClass->new('Bob');

$a->greet();

Unicon vs. Perl vs. Pythonselected underlying issues

a better Icon goal-directed

evaluation ! classes and

packages are optional

compile-step

a better ABC Lisp, Icon

OO highly interactive reference count Indent-syntax !

a better sh, awk regex'es++ readability caters to lone

wolves scripting !

Some other Unicon Applications

The SSEUS system at NLM/NIH Database peer review management

MKR (Knowledge Representation Lang) Richard McCullough, AT&T (ret)

CVE (Collaborative Virtual Environment) 3D, online, social IDE

Unicon translator Merr (Error table generator for Yacc) Ivib (GUI builder) Udb (Unicon debugger)

Framework for automatic debugging agents

AT&T – UIdaho Collaboration

Started late 08 K.O. + Jeffery + J. Al-Gharaibeh Improve nperf's two scalability issues Performance• Will it run faster on diamond?• (not until Unicon is concurrent)

• How bad is the Icon VM bottleneck? Code maintainability• Will OO refactoring help?

Nperf08 Performance

Carefully instrumented by K.O. T1=25.63 T3=2.15 T4=19.78 T5=5.60

T6=54.61 T7=4.80 T8=14.67 sec Most time spent reformatting matrices to

get in and out of C Idea: modify Unicon to use native C

arrays Idea: extend and streamline Unicon-C

interface

IntArray and RealArray

n-dimensional, C-compatible Initially, new data types prototyped for

Unicon in summer 09 (Jeffery)• minimal nperf tweaks: ~10% speedup• could modify nperf's matrix-of-structs representation for more speed

Melted implicitly into the list type in Fall 09 (Al-Gharaibeh)

Unicon Optimizing Compiler

“unicon -C” based on iconc (K. Walker) Scalable type inferencing (M. Wilder) Able to handle nperf?• Yes! ... after numerous workarounds

2X-2.5X overall speedup for nperf• despite critical sections already in C

Complements array speedups

Next Steps

Fix nperf-related compiler bugs Standardize array representations within

the list data type Modify nperf to use array representations

more aggressively where needed Radically enhance the Unicon-C interface• Inline C• type-rich autowrapper

Review nperf code for OO opportunities to increase its flexibility

Longer term: concurrent Unicon

Conclusions

We have improved nperf performance without extensive rewrites

We can improve it more with “modest” changes that leave research flexibility

Improving the underlying language is sometimes better than overtuning

Key: better Unicon – C collaboration

Unicon Evolution

Clinton Jeffery

Jafar Al Gharaibeh Hani Bani Salameh

your name may be here someday

Changing Plans?

The bosses at AT&T ignored our proposed plans, and jumped all over the two lines mentioning “concurrent Unicon” Maybe this is because their machine diamond

is a 128 processor shared memory machine with, like, 256GB of shared memory.

Maybe it is because nperf runs many hours on current simulations, after heavy tuning

Round 2 of the AT&T collaboration is mostly about concurrent Unicon

Actually, this was “bait” in my cookie talk, because I wanted to go after it.

Unicon Evolution: the Big Picture

Languages that don't evolve are dead The past 10 years of evolution have been

about Input/Output (3D, DB, IP) Multicore support is now essential C/C++ interoperation is also vital Research goal: get there without

“ruining” Unicon!

Concurrency in high level languages

Scripting languages are late to the party Explicit

Threads Forks and joins Communication and synchronization Mostly considered “too hard” for mainstream

Implicit “Free” concurrency Vectorizing compilers Lisp and Prolog Difficult to get much benefit for mainstream

languages & applications

Concurrency in Unicon

Co-expressions are synchronous threads Under what circumstances is it safe to

turn them loose? How do we make the VM thread-safe?

Native co-exprs vs. pthread coexprs Local co-exprs vs. loaded programs

The C Calling Interface

Completely uninteresting to many customers; completely essential to some

Icon's C Calling Interface Write a wrapper wrapperfunc() to convert data

values and call func() P := load(“foo.so”, “wrapperfunc”)

Idea #1: wrappers from prototypes Idea #2: in-line C Idea #3: Unicon classes wrap C++ classes

SWIG

David Beazely; LANL, UChicago, and ?? C/C++ → HLL External interface file, external tool Great for applications developers,

suboptimal for a language vendor

SWIG Example

%module example %{/* Put header files here or function declarations like

below */ %}extern double My_variable;extern int fib(int n);extern int my_mod(int x, int y); extern char

*get_time();

…

unix % swig -perl5 example.iunix % gcc -c example.c example_wrap.c \-I/usr/lib/perl/solaris/5.003/COREunix % ld -G example.o example_wrap.o -o example.so

Unicon, Batchu example

procedure main() write(“Fibonacci of 4 is “, fib(4))end

$c fibonacci.o { int fib(int) }$endc