Slide 1

Parallel and Distributed Computing Early(and Often)Steven Bogaerts (speaker), Kyle Burke, Brian ShelburneDepartment of Mathematics & Computer ScienceWittenberg UniversitySpringfield, OHMelissa SmithDepartment of Electrical & Computer EngineeringClemson UniversityClemson, SCEric StahlbergCCR Bioinformatics CoreNational Cancer InstituteBethesda, MDFirst NSF/TCPP Workshop on Parallel and Distributed Computing Education (EduPar-11)May 16th, 2011NSF grant CCF-0915805, SHF:Small:RUI:Collaborative Research: Accelerators to Applications Supercharging the Undergraduate Computer Science CurriculumPIs: Eric Stahlberg, Melissa Smith, Steven BogaertsPDC Concepts Throughout the CS Curriculum Steven BogaertsAlgorithmsProgramming LanguagesComputational ModelingCS2CS1CS0 or Computer LiteracyComputer Literacy

Steven BogaertsScratch

Computer Literacy

Steven Bogaerts

Computer Literacy PDC Overview Steven BogaertsConceptsThe world is obviously parallel.Big-picture descriptions of some applications.Terminology, historyPhysical activitiesLow-level: binary adderHigher-level: card sortingCommunicationShared memory vs. message passingComputer Literacy Curriculum Initiative Connections Steven BogaertsCross-Cutting Why and what is PDC?Concurrency topics : Non-determinismPowerCurrent/Hot/Advanced TopicsClusterCloud/GridAlgorithms : Algorithmic Problems : BroadcastAsynchronyProgramming : Semantics and Correctness Issues : Tasks and ThreadsArchitecture : Classes : MulticoreCS1

Steven BogaertsParallelism/Concurrency can be used as a medium for teaching core computer science topics.Analogy:Object-Oriented Programming Steven BogaertsYears ago:OOP was a new paradigmShould OOP be taught in introductory computer science courses?Should OOP topics be sprinkled into existing courses?If so, which topics should be taken out to make room?

Now:Not just an advanced OOP courseCS1Data structuresGraphics

Topics taught through the medium of OOGoals(CS1 and Beyond) Steven BogaertsTo prepare students for the impending ordinariness (Doug Lea) of parallelism and concurrency.Minimize the amount of material that must be removed to make room.Prevent the necessity of adding and staffing another course.CS1

Steven BogaertsIntroduction similar to Literacy courseTerminology, high-level concepts, physical activitiesBasic use of Python multiprocessing moduleFlexible application to many kinds of problems Minimal additional syntax

Python Multiprocessing Module

Steven Bogaerts

Python Multiprocessing Module

Steven BogaertslockVar = Lock()lockVar.acquire()lockVar.release()

procVar.join()

Parallelism/Concurrency as a Medium Steven BogaertsClasses and objectsParameter passingModularity and abstraction

Sandbox for exploration of other concepts:Computing terms of the quadratic formulaSearching or sorting a listPattern matchingClusteringSimulations of parallel processes in natureKey idea: Do what you usually do, just do (some of) it in parallel.

13

Computational Modeling MotivationsComputational models are increasing in complexityData available for constructing computational models is rapidly growingParallel computing platforms are readily availableExtreme Scale computing (exascale) platforms require new algorithms built from the ground up using parallel computing conceptsIndustry needs better performing models and parallel and accelerated computing are the only real options

Computational Modeling Enhancement ObjectivesPrimaryProvide examples of easy parallel computing for models (embarrassingly parallel)Provide introduction to use of 2D arrays in parallel computing environmentsKeep it straightforward and accessibleSecondaryProvide introduction to contention and overhead

Computational Modeling ApproachUse Computational Models and Methods course200 level (sophomore level) Pre-requisite: Calculus, CS1Implementation using Mathematica Parallel Computing ToolkitMathematica is proven effective for computational models and methodsFunctional programmingReady with built-in parallel programming primitives starting with version 7Machine-independentSelect examples and provide a lab experience using primitivesComputational Modeling Trial Lessons LearnedStudents can master easy primitivesStudents can even become comfortable with parallelizing matrix access by indexMathematica more difficult for high-level parallelismEmpty Kernel starting point requires significant distribution of data and functions to start parallelismComputational Modeling Simple Parallel ExamplesData reduction (simple)Sum[a,b,c,] ParallelSum[a,b,c,]Product[a,b,c,] ParallelProduct[a,b,c,]Matrices (data sharing)Shared[{a,b,c}]Other new fun options to tryParallelPlot3D[]ParallelDensityPlot[]ParallelContourPlot[]

Programming Language CoursesExploration of programming paradigmsInclude HPC languages!Language:Supports multi-threadingAlleviates common concurrency issuesSimplifies vital parallel patterns Steven Bogaerts19Computer Organization CoursesAlready implicitly include parallelism!Combinatorial circuitsPipelines, super-scalar architecturesI/O with interrupts and DMA

Make the inherent parallelism more explicitConsider VHDLExample:Carry look-ahead adder (parallel propagation of carrys) vs. ripple carry adder Steven BogaertsConclusionEarly work in applying parallelism/concurrency as a medium for learning computer science.Developments in:CS1Programming languagesComputer organization Steven Bogaerts