Computational Computational Methods in Methods in

AstrophysicsAstrophysics ASTR 5210 ASTR 5210Dr Rob Thacker (AT319E)Dr Rob Thacker (AT319E)

thacker@apthacker@ap

11stst Homework Homework Will put up on website at lunchWill put up on website at lunch Questions on Questions on

Flynn’s taxonomyFlynn’s taxonomy 2’s complement2’s complement RoundingRounding Floating point formatsFloating point formats Testing rounding modes of fortran Testing rounding modes of fortran

compilerscompilers

Today’s LectureToday’s Lecture Programming paradigms and high Programming paradigms and high

level languageslevel languages What is expected?What is expected?

You develop some knowledge of the You develop some knowledge of the different paradigms and what they different paradigms and what they meanmean Pros and cons etcPros and cons etc

You aren’t expected to be a whiz at You aren’t expected to be a whiz at programming in all programming stylesprogramming in all programming styles

Programming ParadigmsProgramming Paradigms ParadigmParadigm is used in many different ways, is used in many different ways,

according to Webster’s:according to Webster’s: ““a philosophical and theoretical framework of a a philosophical and theoretical framework of a

scientific school or discipline within which theories, scientific school or discipline within which theories, laws, and generalizations and the experiments laws, and generalizations and the experiments performed in support of them are formulated; performed in support of them are formulated; broadly broadly :: a philosophical or theoretical framework of a philosophical or theoretical framework of any kindany kind” ”

A A programming paradigmprogramming paradigm is a way of is a way of conceptualizing what it means to perform conceptualizing what it means to perform computation, and how tasks that are to be computation, and how tasks that are to be carried out on a computer should be structured carried out on a computer should be structured and organized.and organized. The same paradigm can be encapsulated in many The same paradigm can be encapsulated in many

different languagesdifferent languages

How could we look at How could we look at program execution?program execution?

We can view computation in many We can view computation in many different ways, for example:different ways, for example: A list of instructions to be executedA list of instructions to be executed A set of expressions to be evaluatedA set of expressions to be evaluated A list of rules to be appliedA list of rules to be applied A set of objects to interactA set of objects to interact A set of messages to be sent/receivedA set of messages to be sent/received

Why look at programming Why look at programming paradigms?paradigms?

Different paradigms provide different Different paradigms provide different features and benefitsfeatures and benefits Understanding the commonality makes it easier Understanding the commonality makes it easier

to learn new languagesto learn new languages Some paradigms are more appropriate for one Some paradigms are more appropriate for one

problem than another, allows you to select best problem than another, allows you to select best approachapproach

Object oriented great for database designObject oriented great for database design Imperative great for scientific programmingImperative great for scientific programming

Address problems that are not easily tractable in Address problems that are not easily tractable in one paradigm versus anotherone paradigm versus another

This will help develop the link between your algorithm This will help develop the link between your algorithm knowledge and your programming skillsknowledge and your programming skills

Influences of the Influences of the programming paradigmprogramming paradigm

Programming techniqueProgramming technique Frequently influenced by algorithmic Frequently influenced by algorithmic

approaches to solving a particular approaches to solving a particular problemproblem

Programming styleProgramming style How are ideas expressed within the How are ideas expressed within the

program?program? Can we encapsulate ideas elegantly (or not?)Can we encapsulate ideas elegantly (or not?)

Programming cultureProgramming culture

Four main programming Four main programming paradigmsparadigms

Although there are actually many Although there are actually many programming paradigms, a reasonable programming paradigms, a reasonable division is as followsdivision is as follows

The imperative paradigmThe imperative paradigm Includes the procedural paradigmIncludes the procedural paradigm Includes the object-oriented paradigmIncludes the object-oriented paradigm

The declarative paradigmThe declarative paradigm Includes the functional paradigmIncludes the functional paradigm Includes the logical paradigmIncludes the logical paradigm

Wikipedia has a long list of other possibilities Wikipedia has a long list of other possibilities if you are interestedif you are interested

Imperative programmingImperative programming This was the first programming paradigm developed This was the first programming paradigm developed

and represents a common view of programmingand represents a common view of programming e.g. e.g. First First do thisdo this and next and next do thatdo that

Similar in approach to instructions on how to make Similar in approach to instructions on how to make things, cook a meal, repair a car and so onthings, cook a meal, repair a car and so on

Strongly motivated by von Neumann architecture Strongly motivated by von Neumann architecture and a program counter stepping through a list of and a program counter stepping through a list of instructions instructions

Can only support a low level of abstraction because Can only support a low level of abstraction because the model is closely associated with computer the model is closely associated with computer designdesign The idea of a procedure is the most natural abstractionThe idea of a procedure is the most natural abstraction

A group of commands is abstracted into a single commandA group of commands is abstracted into a single command Commonly called Commonly called procedural programmingprocedural programming

Declarative: Functional Declarative: Functional ProgrammingProgramming

Derived from mathematics and theory of Derived from mathematics and theory of functionsfunctions e.g. e.g. evaluate an evaluate an expressionexpression and use the and use the resultresult

Programming is about Programming is about defining relationshipsdefining relationships between data, rather than the steps involvedbetween data, rather than the steps involved Very different feel to imperative programmingVery different feel to imperative programming Functional programming is atemporalFunctional programming is atemporal

Concepts derived from work by Alonzo Concepts derived from work by Alonzo Church in 1930s (Lambda calculus)Church in 1930s (Lambda calculus)

Beginning to filter into the mainstream, Beginning to filter into the mainstream, the the Linspire OS team use the Linspire OS team use the HaskellHaskell language language for for developmentdevelopment

Contrasting functional and Contrasting functional and imperative programmingimperative programming

# imperative style (Python)target = [] # create empty listfor item in source_list: # iterate over each thing in source trans1 = G(item) # transform the item with the G() function trans2 = F(trans1) # second transform with the F() function target.append(trans2) # add transformed item to target

A functional version has a different feel to it:

# functional style (Python)def compose2(F, G): # FP-oriented languages often have standard compose() def C(item): # here we create utility-function for composition return F(G(item)) return Ctarget = map(compose2(F,G), source_list)

From Wikipedia

Logical programmingLogical programming Strongly oriented Strongly oriented

toward AI, and toward AI, and designed around the designed around the formal concepts of formal concepts of proofs and axiomsproofs and axioms Can ask for a given Can ask for a given

hypothesis whether it hypothesis whether it is consistent with the is consistent with the underlying systemunderlying system

i.e. we give a i.e. we give a series of facts and series of facts and relationships, and relationships, and then ask whether then ask whether another another relationship is truerelationship is true

is_a(abel,male).is_a(abel,male).is_a(adam,male).is_a(adam,male). ... etc ...... etc ...

is_a(amanda,female).is_a(amanda,female).is_a(anjali,female).is_a(anjali,female). ... etc ...... etc ...

is_father_of(abel,bert).is_father_of(abel,bert).is_father_of(abel,belinda).is_father_of(abel,belinda). ... etc ...... etc ...

is_mother_of(anjali,bill).is_mother_of(anjali,bill).is_mother_of(anjali,bridget).is_mother_of(anjali,bridget). ... etc ...... etc ...

is_grandparent_of(X, Z) :-is_grandparent_of(X, Z) :- is_parent_of(X, Y),is_parent_of(X, Y), is_parent_of(Y, Z).is_parent_of(Y, Z).

is_parent_of(X, Y) :-is_parent_of(X, Y) :- is_mother_of(X, Y).is_mother_of(X, Y).

is_parent_of(X, Y) :-is_parent_of(X, Y) :- is_father_of(X, Y).is_father_of(X, Y).

Example of a Prologprogram

Object oriented Object oriented programming (OOP)programming (OOP)

No single agreed definition on what constitutes No single agreed definition on what constitutes OOPOOP Based upon concepts and how humans interact with Based upon concepts and how humans interact with

real world phenomenareal world phenomena If you simulate a system then each physical object in the If you simulate a system then each physical object in the

system should be represented by a computational system should be represented by a computational objectobject Shifts the idea of stepping through commands Shifts the idea of stepping through commands

(instruction flow) to describing interactions between (instruction flow) to describing interactions between objects and objects and methodsmethods (which are effectively (which are effectively functions) that describe an objects actionsfunctions) that describe an objects actions

In this paradigm there is a distinct relationship In this paradigm there is a distinct relationship between data and functions (both are between data and functions (both are encapsulated within the object)encapsulated within the object) In much imperative-style programming this isn’t In much imperative-style programming this isn’t

truetrue

Common OOP Common OOP terminologyterminology

How do we define an object?How do we define an object? Objects are a single Objects are a single instanceinstance of a of a classclass Think of your dog “Fido” being an instance of the Think of your dog “Fido” being an instance of the

class of the dogsclass of the dogs The dog class will include all information about The dog class will include all information about

dogs and the messages they can send (by the dogs and the messages they can send (by the bark method for example)bark method for example)

Classes allow us to define inheritanceClasses allow us to define inheritance Once you’ve defined dog, you might want to Once you’ve defined dog, you might want to

define the sub-class of poodlesdefine the sub-class of poodles This is efficient because you can reuse much of This is efficient because you can reuse much of

the informationthe information This is the reason why object oriented methods are This is the reason why object oriented methods are

effective for very large software developmentseffective for very large software developments

DiagramaticallyDiagramaticallyClass

Methods

Datadeclarations

Data specifications

Objecte.g. tree class

Has a height of xmetres

Pineobject

5m height

OOP is sufficiently important that we will come back to it later

Set-oriented view of the Set-oriented view of the paradigmsparadigms

Imperative

Procedural

Object- oriented

Fortran Basic

Pascal C

C++ Java

Declarative

Functional

Logic

LispHaskell

Prolog

Assembler

Algol

Simula

Excel

Have programming Have programming paradigms influenced paradigms influenced

machine design?machine design? Not as much as you would thinkNot as much as you would think

By using translators to increase By using translators to increase abstraction from the machine code level abstraction from the machine code level sophisticated logic systems can be sophisticated logic systems can be implemented quite “easily”implemented quite “easily”

However, the LISP language did However, the LISP language did spawn a number of dedicated spawn a number of dedicated computers because of its huge computers because of its huge memory requirementsmemory requirements However, increasing availability of cheap However, increasing availability of cheap

memory has made this less worthwhilememory has made this less worthwhile

MIT LISP machine from Dan Keller’s homepage

High-level LanguagesHigh-level Languages High-level languages were the natural progression from High-level languages were the natural progression from

machine code and assembly languagemachine code and assembly language Early in their development it was very clear that performance Early in their development it was very clear that performance

was an issuewas an issue FORTRAN, LISP & COBOL were the first three “big” FORTRAN, LISP & COBOL were the first three “big”

HLLs HLLs Idea: take a higher-level description of what we want to Idea: take a higher-level description of what we want to

do, and let the computer do, and let the computer translate translate it into the machine it into the machine language as specified beforelanguage as specified before

Called Called compiler compiler because it might take a single high-level because it might take a single high-level command, and compile a sequence of low-level command, and compile a sequence of low-level commandscommands Input high-level language as text, store binary commands in Input high-level language as text, store binary commands in

executable fileexecutable file Of course, we now also interpret commands on the flyOf course, we now also interpret commands on the fly

The designers of the very first high level The designers of the very first high level programming language were aware that their programming language were aware that their success depended on acceptable performance success depended on acceptable performance of the generated target programs:of the generated target programs:

John Backus (1957)John Backus (1957): “… It was our belief that if FORTRAN … : “… It was our belief that if FORTRAN … were to translate any reasonable scientific source program were to translate any reasonable scientific source program into an object program only half as fast as its hand-coded into an object program only half as fast as its hand-coded counterpart, then acceptance of our system would be in counterpart, then acceptance of our system would be in serious danger …”serious danger …”

The Emergence of High-Level Sequential Languages

High-level algorithmic languages became generallyHigh-level algorithmic languages became generally accepted standards foraccepted standards for programming programming as as

their advantages outweighed any performance drawbackstheir advantages outweighed any performance drawbacks

Humour: How to shoot Humour: How to shoot yourself in the footyourself in the foot

FORTRAN FORTRAN You shoot yourself in each toe, iteratively, until You shoot yourself in each toe, iteratively, until

you run out of toes, then you read in the next foot you run out of toes, then you read in the next foot and repeat. If you run out of bullets, you continue and repeat. If you run out of bullets, you continue anyway because you have no exception handling anyway because you have no exception handling ability.ability.

COBOLCOBOL USEing a COLT 45 HANDGUN, AIM gun at USEing a COLT 45 HANDGUN, AIM gun at

LEG.FOOT, THEN place ARM.HAND.FINGER on LEG.FOOT, THEN place ARM.HAND.FINGER on HANDGUN.TRIGGER and SQUEEZE. THEN HANDGUN.TRIGGER and SQUEEZE. THEN return HANDGUN to HOLSTER. CHECK whether return HANDGUN to HOLSTER. CHECK whether shoelace needs to be retied.shoelace needs to be retied.

Humour: couple of other Humour: couple of other examplesexamples

JAVAJAVA You attempt to shoot yourself in the foot You attempt to shoot yourself in the foot

using a bullet that will work in any gun using a bullet that will work in any gun in the world. But you discover that the in the world. But you discover that the "Microsoft Gun" is actually a cross bow. "Microsoft Gun" is actually a cross bow.

FORTH FORTH Foot in yourself shoot.Foot in yourself shoot.

TimelineTimeline

FORTRANFORTRAN FORTRAN (1953-1958, led by John FORTRAN (1953-1958, led by John

Backus)Backus) FORTRAN is a shortened form of FORmula TRANslaterFORTRAN is a shortened form of FORmula TRANslater First compiled high-level languageFirst compiled high-level language Fortran I (1956)Fortran I (1956) 50 years of FORTRAN!50 years of FORTRAN!

I/O could formatted easily (ie print to screen, printer, file)I/O could formatted easily (ie print to screen, printer, file) 6 characters variable names6 characters variable names The type of variables (The type of variables (i.e.i.e. floating point, integer) were implicitly floating point, integer) were implicitly

declared by the first letter in the name of the variable declared by the first letter in the name of the variable Close to the speed of assembly in many casesClose to the speed of assembly in many cases

Forced a simple programming style thoughForced a simple programming style though Fortran II (1958)Fortran II (1958)

Added subroutines & fixed a few bugs Added subroutines & fixed a few bugs

Evolution of FORTRANEvolution of FORTRAN The development of other languages occurred The development of other languages occurred

while FORTRAN was still being used – why?while FORTRAN was still being used – why? FORTRAN was initially very limited in terms of the data FORTRAN was initially very limited in terms of the data

structures it could supportstructures it could support Handling of anything other than numbers was clumsy at Handling of anything other than numbers was clumsy at

best, absolutely hideous at worst (converting a numeric value best, absolutely hideous at worst (converting a numeric value to a string remains problematic)to a string remains problematic)

FORTRAN IV (1962), 77 (1977), 90 (1990) all added FORTRAN IV (1962), 77 (1977), 90 (1990) all added new features, and latest versions include newer new features, and latest versions include newer programming paradigms (such as object oriented programming paradigms (such as object oriented programming)programming) New standards are still being defined (e.g. FORTRAN 2005)New standards are still being defined (e.g. FORTRAN 2005) Extremely efficient and readableExtremely efficient and readable

Once you’ve written one program it all seems easyOnce you’ve written one program it all seems easy

COBOL: The world’s most COBOL: The world’s most common languagecommon language

COBOL (1959, CODASYL committee)COBOL (1959, CODASYL committee) Common Business LanguageCommon Business Language Designed by committee, but heavily influenced by Designed by committee, but heavily influenced by

the ideas of (Rear Admiral) Grace Murray Hopper the ideas of (Rear Admiral) Grace Murray Hopper It was estimated in that late 1990’s that 80% of all It was estimated in that late 1990’s that 80% of all

the lines of programming ever written were COBOL the lines of programming ever written were COBOL about 240 about 240 billionbillion lines lines

Designed to be easy to use, and uses words Designed to be easy to use, and uses words rather than formulasrather than formulas

Many useful features like macros, hierarchical Many useful features like macros, hierarchical data structuresdata structures

C: Bringing together C: Bringing together good ideas good ideas

C (1972, Dennis Ritchie)C (1972, Dennis Ritchie) Developed at Bell LabsDeveloped at Bell Labs Derived from earlier languages (BCPL & B) Derived from earlier languages (BCPL & B)

using a best practices type methodologyusing a best practices type methodology No new features but the design was well No new features but the design was well

thought outthought out C is closely tied to UNIX, and second C is closely tied to UNIX, and second

generations of UNIX were written in Cgenerations of UNIX were written in C Portable, which has lead to it being very widely usedPortable, which has lead to it being very widely used Some times referred to as a “mid” level languageSome times referred to as a “mid” level language Interface to the operating system is easy to useInterface to the operating system is easy to use

C++: Objects go C++: Objects go mainstreammainstream

C++ (1982-1985, C++ (1982-1985, Bjarne StroustroupBjarne Stroustroup)) Another development from Bell LabsAnother development from Bell Labs

Motivation: a better way to deal with codes that had Motivation: a better way to deal with codes that had hundreds of thousands of lineshundreds of thousands of lines

Windows 2000 has more than 30 million lines of code!Windows 2000 has more than 30 million lines of code! Layered OOP ideas implemented in the Layered OOP ideas implemented in the

simula67 language onto Csimula67 language onto C Initially developed as a translator from Initially developed as a translator from

C++ to CC++ to C Later versions compiled directly to linkableLater versions compiled directly to linkable

object filesobject files

Is there a “best” Is there a “best” language to use?language to use?

We first need to set some criteria that we can use to We first need to set some criteria that we can use to evaluate languages withevaluate languages with 1. Simplicity and Clarity:1. Simplicity and Clarity: Can the language be expressed & read easily?Can the language be expressed & read easily? Does an operator always mean the same thing?Does an operator always mean the same thing? How easy is the language to learn/teach?How easy is the language to learn/teach?2. Level of Abstraction:2. Level of Abstraction: Is the level of abstraction sufficient to enable new Is the level of abstraction sufficient to enable new

and powerful expressions?and powerful expressions? Machine languages are very limited (only support Machine languages are very limited (only support

“numbers”)“numbers”)

Efficiency & SpecializationEfficiency & Specialization3. Efficiency:3. Efficiency: Can the language be implemented efficiently, Can the language be implemented efficiently,

or for that matter, can it even be implemented or for that matter, can it even be implemented at allat all

Object oriented languages have tended to be very Object oriented languages have tended to be very difficult to implement efficiently (C++ in difficult to implement efficiently (C++ in particular) particular)

4. Specialization:4. Specialization: What kind of facilities does the language What kind of facilities does the language

provide for application specific operations?provide for application specific operations? FORTRAN90 has support for vector operationsFORTRAN90 has support for vector operations

Reliability & robustnessReliability & robustness5. Reliability & robustness:5. Reliability & robustness: Does the program behave as anticipated under Does the program behave as anticipated under

all circumstances?all circumstances? Error checking is the key hereError checking is the key here Does the compiler (or interpreter) check for errors Does the compiler (or interpreter) check for errors

(such as assigning the wrong type to a variable or an (such as assigning the wrong type to a variable or an array address out of bounds)array address out of bounds)

What about dealing with exceptions? Can you make What about dealing with exceptions? Can you make corrective action and continue or does the program corrective action and continue or does the program need to exit?need to exit?

This is a key issue, think about the Arianne 5 This is a key issue, think about the Arianne 5 programming bug that we mentioned earlier!programming bug that we mentioned earlier!

Choose the best Choose the best languagelanguage for the for the applicationapplication

This is why FORTRAN is still with us after 50 years!This is why FORTRAN is still with us after 50 years! It is still very efficient for many classes of scientific It is still very efficient for many classes of scientific

computationcomputation C really doesn’t have that many advantages in this areaC really doesn’t have that many advantages in this area

What about business applications?What about business applications? Visual basic, COBOLVisual basic, COBOL

During my time at J.P. Morgan some of the traders wrote their During my time at J.P. Morgan some of the traders wrote their option pricing software in VBasicoption pricing software in VBasic

What about the internet?What about the internet? Then you need something that deals with information Then you need something that deals with information

processing easily: HTML, javascript, XMLprocessing easily: HTML, javascript, XML What about AI?What about AI?

Then you need a language that deals with logic and Then you need a language that deals with logic and abstraction: LISP or Prologabstraction: LISP or Prolog

What is the “fastest” What is the “fastest” language?language?

Depends what you mean by fastest…Depends what you mean by fastest…

Programming speed

Cod

e ex

ecut

ion

spee

d

FORTRANC

C++Java

Matlab Python

Perl

Time to solution should be the guide

Why did Python become so Why did Python become so popular?popular?

It isn’t new (1991!)It isn’t new (1991!) Offers a lot of flexibilityOffers a lot of flexibility It has a comparatively simple programming It has a comparatively simple programming

style (not lots of weird characters to style (not lots of weird characters to remember)remember)

Libraries often can add “good” performanceLibraries often can add “good” performance

Main issue: starting to be adopted Main issue: starting to be adopted outside of academic => useful skilloutside of academic => useful skill

SummarySummary At highest level programming paradigms At highest level programming paradigms

separate into imperative and declarative modelsseparate into imperative and declarative models Imperative model is strongly influenced by machine Imperative model is strongly influenced by machine

designdesign Declarative models are influenced by the desire to Declarative models are influenced by the desire to

solve logic and arithmetic problemssolve logic and arithmetic problems Optimal solutions to problem requires an Optimal solutions to problem requires an

appreciation for the available paradigms and appreciation for the available paradigms and languageslanguages

There is no one best language, but we can There is no one best language, but we can choose on the basis of clarity, abstraction level, choose on the basis of clarity, abstraction level, efficiency, specialization and reliabilityefficiency, specialization and reliability

High level languages began with FORTRAN, High level languages began with FORTRAN, which continues to evolve 50 years laterwhich continues to evolve 50 years later