Cost Estimation & Metrics Project Report:

Measuring The Code Changes When Ported

From Platform To Another

By: Fawaz Alturbaq Siddhesh Palav

1

Summary:

Many softwares are moved to different operating systems and devices. We are introducing a model that consists of aggregated metrics to measure the changes are required to move a system to another platform. The results still empirical. However, we came up with a model that could be updated with a new results to be generalized. if the same metrics applied to different projects, we could use these results to update the weights we have used to measure the changes on the source code when a system ported to a new platform.

Pledge “I pledge my honor that I have abided by the Stevens Honor System”

2

INDEX Introduction 4

Doom 4 Terms 4

Proposed Model 5 GQM 5 Metrics 5

Number of Files 6 Average Complexity 6 Line Of Code (LOC) 8 Methods Reused Percentage 9 Number of Methods 10 Percentage of Comments 11

Algorithmic Model 11 Usage 12

Results and Discussion 12

Limitations 12

Conclusions 12

Reflection 13

References 13

Appendix A 13

3

1. Introduction Many scenarios lead to porting a system to another platform running different operating

system on a different hardware architecture. Sometimes this process is straightforward requiring only recompiling. However, sometimes this process require a lot of changes on the code to run the same system. We are introducing a metric measuring what a new platform could impose on the code to be changed. We used a popular video game that have been ported twice. We pulled the code and run it through Cppcheck and CMM tools to get get the data about the three different version of the same video game. We analyzed the data and notice some patterns when the video game ported to a new platform.

1.1 Doom Doom is a popular PC video published in 1993, and sold 10 millions copy. It was popular

at that time and still until this day. It was one of the first video games that mimic the 3D world using a 2D graphic engine. The same video game was ported to Apple IOS and Apple IOS2 using the same graphic engine. Because it’s the same video game, the same graphic engine, and its source code is available on github for Doom, Doom IOS and DoomI IOS2, we chose it as an example to build our metric.

https://github.com/id-Software/DOOM https://github.com/id-Software/DOOM-iOS https://github.com/id-Software/DOOM-IOS2

1.2 Terms Doom: A popular video game on PC. The classic doom has been ported to IOS Port: The process of moving a system to run on a different platform Platform: Combination of an operating system and hardware architecture. IOS: iPhone Operating System.

4

2. Proposed Model The model we are proposing is aiming to measure the required modification when a

system ported to another platform. These modifications are not a new functionality or bugs being fixed. They are the required modifications so that the system can run on a different platform. We are measuring the modification using aggregated change metrics. We applied these metrics to measure the changes in the code and documentation (comments) that were required during the porting process. This model could be used in the future to measure the cost of porting a system from platform to another. We applied our metrics to spot a pattern of change. Some of the metrics produced an interesting pattern and left us wondering if this pattern could show up on different projects that have been ported on different platforms. On the other hand, some of the metrics didn’t show a pattern or didn’t change at all.

1.1 GQM

● The goal here to measure the modifications that the new platform impose. We designed our questions first that serve

our goal. Then we came up with metrics that answering them.

1.1 Metrics In this section, we are going in depths and graphs explaining our metrics and how we

used them to spot if there’s a pattern of change. I included our analysis in the appendices as you can view the data and follow the analysis step by step.

5

i. Number of Files First Metric we analyzed is the number of files. As you can see in Figure 1, there is no pattern of

increase in number of files. Also you can view that there’s no pattern in Table 1.

Doom ported→ Doom IOS ported→ Doom IOS2

All Files

146 52%↑ 223 121%↑ 494

C Files 72 41.6%↑ 102 117%↑ 222

Header Files

74 60.8%↑ 119 123%↑ 266

Metric Conclusion: We didn’t notice any pattern of the change in number of files either totally, header or C files.

ii. Average Complexity To remove conversion, the tool we used provided us with average complexity number for

each file. For example, in Doom IOS we have 119 average complexity numbers. The tool calculated the complexity for each method in the file and returned the average for each file. To remove confusion, I’m going to call these numbers “File Complexity” because later we are measuring the average of these 119 numbers (the average of File Complexity).

6

We were curious to measure how complexity will change when the video game was ported to

IOS then later IOS 2. We discarded many files with 0 complexity to remove the noise because there are many files just an interface. We were interested only in the files with complexity more than zero. and interestingly that the average of the File Complexity is almost the same with different variance as you can see in Figure 2.

Doom Doom IOS Doom IOS2

Mean of File Complexity 7.6 8.1 7.5

File Complexity Standard Deviation

6.6 9.1 7.4

Breaking the code complexity even further, we took another features in our data which is the number of the statements in each nested block level in each file. Because we need three axis (Files, number of statements, and block level), we chose for a clearer view to show that in a heatmap diagram which it shows three different values in a 2D diagram instead of plotting it in a 3D graph which it will makes it hard to see the patterns of change. As you can see in Figure 3, there’s no pattern of change noticed. Most of statements in the three projects are in the nested block levels from 2 to 5. Block level 2 has the most then it decreases to level 5 with some exceptional files that reached higher levels. However, these exceptional files exists in all three projects inconsistently.

7

Metric Conclusion: We didn’t notice any change in File Complexity when the system ported to another platform

iii. Line Of Code (LOC) In this metric we tried to view lines of code in each file to see if there’s pattern or not.

Then we compared the total number lines of code. We used Boxplot to vide the lines of code for each file and since the whole majority of files have few lines of code, the what made this plot show that the three projects share the same 4 quartiles as you can see in Figure 4.

8

Doom Doom IOS Doom IOS2

Mean of Files LOC 514 670 1114

Standard Deviation of Files LOC

335 417 448

Platforms Total LOC

Doom 48963

Doom IOS 93178

Doom IOS 2 221463

Metric Conclusions: We noticed that every time the video game has been ported to a new platform, where was increase in the total LOC of the project on an average 2.2 due to the added outliers. We considered this a pattern that we can add in our model to calculate the code modification when the project is ported to a new platform

iv. Methods Reused Percentage In this metric we are aim to measure the re-usage of methods when the video game

ported from PC to IOS to IOS2. Because there was prefixes added in IOS and IOS2 projects, we used a built-in function in Python “SequenceMatcher ” that measures the similarity of a list of strings and returns a ratio. The result were:

Platforms Doom Doom IOS Doom IOS2

Doom 1

Doom IOS 0.436 1

Doom IOS2 0.28 0.7 1

9

Metric Conclusion: We concluded that there was around 40% and 70% methods re-usage in Doom IOS and IOS2 respectively. Therefore, the added new methods were 60% in Doom IOS and 30% in Doom IOS2.

v. Number of Methods This metric measure the total number of methods in each projects and compare them to

see if there’s a pattern has been noticed. We graphed that data as you can see in Figure 5.

Platforms Methods/Files

Doom 5.31

Doom IOS 11.54

Doom IOS 2 20.55

Conclusion: We noticed the number of methods is doubled every time the system is ported to a new platform.

10

vi. Percentage of Comments We measured the change pattern in the documentation (Comments). We calculated the

lines of comments in LOC and we graphed them to compare (Figure 6).

Platforms Avg % of comments

Doom 17.22

Doom IOS 21.68

Doom IOS 2 25.33

Metric Conclusion: We noticed that there’s increase 4% in the comments when the video game was ported to another platform.

1.2 Algorithmic Model Since we found a solid consistent change in Doom when it was ported to IOS and again in to IOS 2. We converted these changes to an algorithmic model to freeze our results and give opportunity to update this equation in the future when another project is measured and different patterns were found.

11

1.3 Usage This model could be used to calculate the cost when a system is ported to another platform. There are many scenarios this model could be useful.

● Moving a legacy system to a newer platform ● Deploy a system on a mobile platform ● Switch to another Operating System for better security or support

These scenarios could require a lot of changes and our metric could help to calculate the cost for better decision.

3.Results and Discussion We found solid change patterns in this project and we created our aggregated change metrics. However, these are only empirical results even though these patterns happened twice. They are not solid enough to form a new model. If the same patterns happened in different projects then a new model could be formed. However, if there’s patterns in other projects but with different quantity, the weights in the equation could be updated for more generalized formula. The more the weight are update, the more solid the equation will be. The weights we have found are 0.4, 0.45, and 1 for comments, number of methods, and KLOC respectfully.

4.Limitations ● We had a hard time finding a good and free tool to analyze Doom C code. ● We were limited by the metrics the tools provided us.

5.Conclusions In this project we learnt how to measure and analyze different aspects in the

software code. It’s indeed a valuable skill for software cost estimation. We also realized how hard and important software measurement is. It’s hard because ,unlike other engineering fields, softwares can’t be seen or touched. Therefore, they are hard to measure and always underestimated. However, it’s very important to measure as close as we can get for more successful projects.

12

6.Reflection This model could be extended and applied on other projects that have been ported to

different platform without changing the functionality. That will solid our results.

7.References Almond, C., Brown, M., Davis, C., Dy, W., Ionescu, P., Richardson, J., ... & Williamson, R. (2006). Solaris to Linux Migration: A Guide for System Administrators . IBM Redbooks. Al-Najdawi, N. (2007). Introduction to visualization using game engines. In AHRC Methods Network Workshop .

8. Appendix A We have used IPython and Jupyter Notebook to analyze the data. I included all the

needed files to walk through the same data analysis we did. You need to install python first to run the code. http://jupyter.readthedocs.io/en/latest/install.html Once Jupyter is installed, run the Jupyter notebook from our data directory.

You will get this page

Choose the Doom Analysis notebook “DoomAnalysis.ipynb”

13

And that will open the notebook to run the commands step by step. Also we included the python code file that you can run but that will pop all the graphs at once.

Appendix B Data Description This is a view of our data. We pulled project details and methods details data. Both of them are attached as a csv files with the report. Also you can download it from GitHub (https://github.com/falturba/DoomAnalysis.git)

Project Details Data Points:

● File Name ● Lines ● Percent Lines with Comments

14

● Functions ● Average Block Depth ● Average Complexity ● Statements at block level 0 ● Statements at block level 1 ● Statements at block level 2 ● Statements at block level 3 ● Statements at block level 4 ● Statements at block level 5 ● Statements at block level 6 ● Statements at block level 7 ● Statements at block level 8 ● Statements at block level 9

Sample of the first 5 rows of the data:

Methods Details Data Points

● Method ● Complexity ● Statements ● Maximum Depth ● Calls

Sample of the first 5 rows of the data:

15