Download - CSI518 – Group 1
![Page 1: CSI518 – Group 1](https://reader036.vdocuments.net/reader036/viewer/2022062302/56816455550346895dd621fb/html5/thumbnails/1.jpg)
Evaluating Web Software Reliability Based on Workload and Failure
Data Extracted FromServer Logs
CSI518 – Group 1
By Zumrut Akcam, Kim Gero, Allen Chestoski,Javian Li and Rohan Warkad
![Page 2: CSI518 – Group 1](https://reader036.vdocuments.net/reader036/viewer/2022062302/56816455550346895dd621fb/html5/thumbnails/2.jpg)
Definition of Reliability
The increasing usage of Web software systems and attraction of society to the Web makes reliability of Web systems more important.
What is reliability for Web applications? The reliability for Web applications can be
defined as the probability of failure-free Web operation completions.[1]
Failure is “the event of a system deviating from its specified behavior like obtaining or delivering information”.[2]
![Page 3: CSI518 – Group 1](https://reader036.vdocuments.net/reader036/viewer/2022062302/56816455550346895dd621fb/html5/thumbnails/3.jpg)
Failure Sources
Failures are caused from the following sources: Host, network or browser failures: computer
systems, network or software failures, etc. Source content failures: missing, inaccessible
files, JavaScript errors, etc. User errors: improper usage, mistyped URLs.[1]
![Page 4: CSI518 – Group 1](https://reader036.vdocuments.net/reader036/viewer/2022062302/56816455550346895dd621fb/html5/thumbnails/4.jpg)
Project Goal
This project concentrates on the minimizing of source content failures to strengthen the reliability of Web applications.
With this project, we try to accomplish the following goals as a team:
Attempt to extend work on testing the reliability of websites.
Gain experience doing a research project
![Page 5: CSI518 – Group 1](https://reader036.vdocuments.net/reader036/viewer/2022062302/56816455550346895dd621fb/html5/thumbnails/5.jpg)
Workload Analysis
To analyze the reliability of web systems, we're going to use the access logs and error logs under the title of server logs.
Failure information alone is not enough for assessing the reliability of a system so measuring the workload is also necessary.
The measurements for workload are byte count, user count, session count and number of hits.
![Page 6: CSI518 – Group 1](https://reader036.vdocuments.net/reader036/viewer/2022062302/56816455550346895dd621fb/html5/thumbnails/6.jpg)
Workload Measures Hit Count: Each hit shows the specific request
to a web server. Misleading because individual hits show high variability.
Byte Count: Number of bytes transferred gives finer granularity than hit count.
User Count: Treat each client IP address as one user. Disadvantages: coarse granularity.
Session Count: Number of user sessions can be calculated by IP address and access times using time limits per user[3].
![Page 7: CSI518 – Group 1](https://reader036.vdocuments.net/reader036/viewer/2022062302/56816455550346895dd621fb/html5/thumbnails/7.jpg)
Sprint 1 Goals Read relevant research papers Identify factors that may effect reliability analysis Determine a system to analyze reliability on Identify a metric to analyze reliability Gather access and error logs
![Page 8: CSI518 – Group 1](https://reader036.vdocuments.net/reader036/viewer/2022062302/56816455550346895dd621fb/html5/thumbnails/8.jpg)
Relevant research papers Toan Huynh and James Miller. 2009. Another viewpoint on
"evaluating web software reliability based on workload and failure data extracted from server logs". Empirical Softw. Engg. 14, 4 (August 2009), 371-396. DOI=10.1007/s10664-008-9084-6 http://dx.doi.org/10.1007/s10664-008-9084-6
Jeff Tian, Sunita Rudraraju, and Zhao Li. 2004. Evaluating Web Software Reliability Based on Workload and Failure Data Extracted from Server Logs. IEEE Trans. Softw. Eng. 30, 11 (November 2004), 754-769. DOI=10.1109/TSE.2004.87 http://dx.doi.org/10.1109/TSE.2004.87
We hope to extend work on these papers
![Page 9: CSI518 – Group 1](https://reader036.vdocuments.net/reader036/viewer/2022062302/56816455550346895dd621fb/html5/thumbnails/9.jpg)
Factors that may effect reliability analysis Byte count User count Session count Error count
![Page 10: CSI518 – Group 1](https://reader036.vdocuments.net/reader036/viewer/2022062302/56816455550346895dd621fb/html5/thumbnails/10.jpg)
System to analyze reliability on Reliability analysis via error logs Variety of reliability requirements Commercial and non-commercial We will try to record the technologies the
websites employ (Apache, DNN, ISS, PHP, ColdFusion, etc..)
![Page 11: CSI518 – Group 1](https://reader036.vdocuments.net/reader036/viewer/2022062302/56816455550346895dd621fb/html5/thumbnails/11.jpg)
The Nelson Method R = (n-f)/n = 1 – (f/n) = 1 – r R: Reliability f: Total number of failures n: Number of workload units r: Failure rate
Mean Time Between Failures (MTBF) MTBF = (1/f) Σi ti
MTBF = n/f
Identify a metric to analyze reliability
![Page 12: CSI518 – Group 1](https://reader036.vdocuments.net/reader036/viewer/2022062302/56816455550346895dd621fb/html5/thumbnails/12.jpg)
Access and error logs Universities and companies refusing to provide
us with access and error logs. Confidentiality reasons Outsourcing server management to external
companies
![Page 13: CSI518 – Group 1](https://reader036.vdocuments.net/reader036/viewer/2022062302/56816455550346895dd621fb/html5/thumbnails/13.jpg)
Sprint 2 Goals Collect enough log files for calculation Automate processes to extra data (user, session, byte, and error counts) and convert them into excel format Log Parser
![Page 14: CSI518 – Group 1](https://reader036.vdocuments.net/reader036/viewer/2022062302/56816455550346895dd621fb/html5/thumbnails/14.jpg)
Sprint 2 Progress A web developer agreed to send all logs he has (ASP.NET / DNN)
![Page 15: CSI518 – Group 1](https://reader036.vdocuments.net/reader036/viewer/2022062302/56816455550346895dd621fb/html5/thumbnails/15.jpg)
What is DotNetNuke (DNN)
Founded 2006 .NET version of Drupal An open source platform for building web sites and web applications based on Microsoft .NET technology. Leading open source ASP.NET web content management system and .NET development framework ~100 employees Has been downloaded over 6 million times 5th Version
![Page 16: CSI518 – Group 1](https://reader036.vdocuments.net/reader036/viewer/2022062302/56816455550346895dd621fb/html5/thumbnails/16.jpg)
Our DNN Logs 10 Websites Window Server (Same Server) SQL Server 2008~1000 unique visiters per dayLogs contain
User count Little Error count
Doesn't contain Session count Byte count
![Page 17: CSI518 – Group 1](https://reader036.vdocuments.net/reader036/viewer/2022062302/56816455550346895dd621fb/html5/thumbnails/17.jpg)
Sprint 2 Problems Still looking for logs and may have to consider generating our information To create our own logs is under discussion
![Page 18: CSI518 – Group 1](https://reader036.vdocuments.net/reader036/viewer/2022062302/56816455550346895dd621fb/html5/thumbnails/18.jpg)
LogParser
Microsoft tool designed for parsing text-based logs
Flexible Support for common log file formats SQL style queries allow for targeted data extraction
Can output to .csv Ease of bulk parsing
Operates off Windows command prompt
![Page 19: CSI518 – Group 1](https://reader036.vdocuments.net/reader036/viewer/2022062302/56816455550346895dd621fb/html5/thumbnails/19.jpg)
Unique Client-IP’s for One Log Set
![Page 20: CSI518 – Group 1](https://reader036.vdocuments.net/reader036/viewer/2022062302/56816455550346895dd621fb/html5/thumbnails/20.jpg)
Sprint 3 and Beyond
![Page 21: CSI518 – Group 1](https://reader036.vdocuments.net/reader036/viewer/2022062302/56816455550346895dd621fb/html5/thumbnails/21.jpg)
Information Extraction
Plot graphs and charts on parsed data Mine the data and derive relations Reliability models
Why Nelson Model? Calculate Operational Reliability
R = (n-f)/n = 1-r MTBF = (1/f)Σti
![Page 22: CSI518 – Group 1](https://reader036.vdocuments.net/reader036/viewer/2022062302/56816455550346895dd621fb/html5/thumbnails/22.jpg)
Conclusion
Derive key factors affecting reliability Provide Inputs
Validating previous research Pointers for topics to explore for future research Detailed documentation and publishing
![Page 23: CSI518 – Group 1](https://reader036.vdocuments.net/reader036/viewer/2022062302/56816455550346895dd621fb/html5/thumbnails/23.jpg)
References
[1] J.Tian, S.Rudraraju, Z.Li, “Evaluating Web Software Reliability Based on Workload and Failure Data Extracted from Server Logs”,2004.
[2] T.Huynh, J.Miller, “Another viewpoint on 'Evaluating Web Software Reliability Based on Workload and Failure Data Extracted from Server Logs'”,2008.
[3] G. Albeanu, A. Averian, I. Duda, “Web Software Reliability Engineering”,2009.