educon16 "comparing computational thinking development assessment scores with software...

EDUCON 2016, Abu Dhabi

Comparing Computational Thinking DevelopmentAssessment Scores with Software Complexity

Metrics

Jesus Moreno-Leon, Gregorio Robles, Marcos Roman-Gonzalez

[email protected], [email protected], [email protected], Universidad Rey Juan Carlos

IEEE EDUCON 2016, Abu Dhabi, April 11th 2016

Jesus Moreno-Leon, Gregorio Robles, Marcos Roman-Gonzalez Comparing Computational Thinking Development Assessment Scores with Software Complexity Metrics


(cc) 2016 Jesus Moreno-Leon, Gregorio Robles and Marcos Roman-GonzalezSome rights reserved. This work licensed under Creative Commons

Attribution-ShareAlike License. To view a copy of full license, seehttp://creativecommons.org/licenses/by-sa/3.0/ or write to

Creative Commons, 559 Nathan Abbott Way, Stanford,California 94305, USA.

Some of the figures have been taken from the InternetSource, and author and licence if known, is specified.

For those images, fair use applies.



Code to learn (I)

Logo programming language

Developed in the 1960s

Its educational impact wasintensively investigated inthe 70s and 80s

Students’ improvements inmaths (and otherdisciplines) were proved

“Disappeared” from theeducational landscape sincemid-90s

Seymour Papert’s picture: jgora.net



Code to learn (and II)

Computational Thinking is back in town

Alice, Greenfoot, Kodu, Scratch...

Code.org, EU Code Week, Africa Code Week, ArabCode.org...

If there is no evidence showing educational impact ofprogramming, this resurgence of programming in schoolscould disappear in a few years.



Dr. Scratch: Analysis of CT skills

Measuring logic development skills




Dr. Scratch allows learners to evaluate their projects toreceive a computational thinking score

Gamified feedback with tips and tricks

The computational thinking score ranges from 0 to 21 points

It is based on the degree of development of differentdimensions of the computational thinking competence:

1 abstraction and problem decomposition2 logical thinking3 synchronization4 parallelism5 algorithmic notions of flow control6 user interactivity7 and data representation




Example Dr. Scratch feedback: http://www.drscratch.org



Classic Software Complexity Metrics

Cyclomatic Complexity (CC)

is a graph-theoretic complexity measure that can be used tomanage and control program complexity.based on the number of linear independent paths in a program,and can be used to establish the number of test cases in thebasis path testing methodology.

Halstead’s metrics

identifiy certain properties of a program that can be measuredand the relationships between them to assess softwarecomplexity.widely used in software engineering to estimate maintenanceefforts and guide software testing by identifying complex, hardto maintain modules.



Example Scratch program

Figure: Example Scratch program



Software Metrics

Figure: Software metrics for the previous program



Research question

Is the Computational Thinkingscore given by Dr. Scratch acomplexity value that could becompared with the classic

software engineering metrics?

Background picture: rebel-performance.com



Methodology

Random selection of 25 projects of each of the followingcategories:

1 stories2 animations3 games4 and art creations.

Although 100 projects were downloaded, 5 of them producedan error while being analyzed, which limits the sample size to95 projects.

We developed a plug-in to obtain CC and Halstead fromScratch projects

The mean score was 13.75, while both median and mode were15.



Correlation between Dr. Scratch and CC and Halstead

Figure: Correlation between Dr. Scratch and CC and Halstead



Correlation by dimension

Figure: Correlation by dimension



In short...

There is correlation between Dr. Scratch CT score andMcCabe’s Cyclomatic Complexity and Halstead’s metrics

Provides validation of the complexity assessment process ofDr. Scratch

The range of Dr. Scratch CT scores, from 0 to 21 points,could not be flexible enough to represent the differences ofcomplexity for complex projectsAn increment in the range of the evaluation of dimensionscould enhance correlation

But beware... not everything is (computational) complexity!

Background picture: flamingcow.co.uk



Comparing Computational Thinking DevelopmentAssessment Scores with Software Complexity

Metrics

Jesus Moreno-Leon, Gregorio Robles, Marcos Roman-Gonzalez

[email protected], [email protected], [email protected], Universidad Rey Juan Carlos

IEEE EDUCON 2016, Abu Dhabi, April 11th 2016


educon16 "comparing computational thinking development assessment scores with software...

Education