cs898 project -- actionable alert detectionmli/xinye-actionablealertdetection.pdf · build project...
TRANSCRIPT
![Page 1: CS898 PROJECT -- Actionable Alert Detectionmli/Xinye-ActionableAlertDetection.pdf · Build Project PRESENTATION TITLE PAGE 16 Build subject project for each revision. Delete the revisions](https://reader034.vdocuments.net/reader034/viewer/2022051911/60021f5452ec2d2549651d8f/html5/thumbnails/1.jpg)
CS898 PROJECT-- Actionable Alert Detection
Presented by: Xinye Tang
![Page 2: CS898 PROJECT -- Actionable Alert Detectionmli/Xinye-ActionableAlertDetection.pdf · Build Project PRESENTATION TITLE PAGE 16 Build subject project for each revision. Delete the revisions](https://reader034.vdocuments.net/reader034/viewer/2022051911/60021f5452ec2d2549651d8f/html5/thumbnails/2.jpg)
OUTLINE▪ Background
▪ Automatic static analysis(ASA)▪ Actionable alert(AA) & Unactionable alert(UA)
▪ Motivation
▪ Related Work
▪ Method
▪ Experiment
▪ Summary
PRESENTATION TITLE PAGE 2
![Page 3: CS898 PROJECT -- Actionable Alert Detectionmli/Xinye-ActionableAlertDetection.pdf · Build Project PRESENTATION TITLE PAGE 16 Build subject project for each revision. Delete the revisions](https://reader034.vdocuments.net/reader034/viewer/2022051911/60021f5452ec2d2549651d8f/html5/thumbnails/3.jpg)
Automatic Static Analysis▪ Static analysis is the process of evaluating a system or component
based on its form, structure, content or documentation.
▪ ASA can identify common coding problems early in the development process via a tool that automates the inspection of source code.
▪ ASA tools: Findbugs, Lint, Checkstyle.
PRESENTATION TITLE PAGE 3
![Page 4: CS898 PROJECT -- Actionable Alert Detectionmli/Xinye-ActionableAlertDetection.pdf · Build Project PRESENTATION TITLE PAGE 16 Build subject project for each revision. Delete the revisions](https://reader034.vdocuments.net/reader034/viewer/2022051911/60021f5452ec2d2549651d8f/html5/thumbnails/4.jpg)
Alert▪ Alerts: potential source code anomalies reported by ASA.
▪ Null pointer dereference
▪ Buffer overflows
▪ Style inconsistencies
PRESENTATION TITLE PAGE 4
![Page 5: CS898 PROJECT -- Actionable Alert Detectionmli/Xinye-ActionableAlertDetection.pdf · Build Project PRESENTATION TITLE PAGE 16 Build subject project for each revision. Delete the revisions](https://reader034.vdocuments.net/reader034/viewer/2022051911/60021f5452ec2d2549651d8f/html5/thumbnails/5.jpg)
Alert▪ Actionable Alerts: if a developer determines the alert is an
important, fixable anomaly.
▪ Unactionable Alerts: When an alert is not an indication of an actual code anomaly or the alert is deemed unimportant to the developer.
PRESENTATION TITLE PAGE 5
![Page 6: CS898 PROJECT -- Actionable Alert Detectionmli/Xinye-ActionableAlertDetection.pdf · Build Project PRESENTATION TITLE PAGE 16 Build subject project for each revision. Delete the revisions](https://reader034.vdocuments.net/reader034/viewer/2022051911/60021f5452ec2d2549651d8f/html5/thumbnails/6.jpg)
Alert▪
AletSTCAL: Sharing a single instance across thread boundaries without proper synchronization will result in erratic behaviour of the application.
7 times in revision 1497967 of Tomcat.
PRESENTATION TITLE PAGE 6
![Page 7: CS898 PROJECT -- Actionable Alert Detectionmli/Xinye-ActionableAlertDetection.pdf · Build Project PRESENTATION TITLE PAGE 16 Build subject project for each revision. Delete the revisions](https://reader034.vdocuments.net/reader034/viewer/2022051911/60021f5452ec2d2549651d8f/html5/thumbnails/7.jpg)
Alert
AlertThis method might ignore an exception.
PRESENTATION TITLE PAGE 7
![Page 8: CS898 PROJECT -- Actionable Alert Detectionmli/Xinye-ActionableAlertDetection.pdf · Build Project PRESENTATION TITLE PAGE 16 Build subject project for each revision. Delete the revisions](https://reader034.vdocuments.net/reader034/viewer/2022051911/60021f5452ec2d2549651d8f/html5/thumbnails/8.jpg)
OUTLINE▪ Background
▪ Motivation
▪ Related Work
▪ Method
▪ Experiment
▪ Summary
PRESENTATION TITLE PAGE 8
![Page 9: CS898 PROJECT -- Actionable Alert Detectionmli/Xinye-ActionableAlertDetection.pdf · Build Project PRESENTATION TITLE PAGE 16 Build subject project for each revision. Delete the revisions](https://reader034.vdocuments.net/reader034/viewer/2022051911/60021f5452ec2d2549651d8f/html5/thumbnails/9.jpg)
MotivationAlert density: 40 alerts/KLOC35 - 91 % of alerts are UA.Lots of UAs may lead developers and managers to reject ASA due to the overhead of alert inspection.
Suppose,1000 alerts, 5 min/alertneed 10.4 workdays to inspect all alertsidentify UAs can save 3.6 - 9.5 days
PRESENTATION TITLE PAGE 9
![Page 10: CS898 PROJECT -- Actionable Alert Detectionmli/Xinye-ActionableAlertDetection.pdf · Build Project PRESENTATION TITLE PAGE 16 Build subject project for each revision. Delete the revisions](https://reader034.vdocuments.net/reader034/viewer/2022051911/60021f5452ec2d2549651d8f/html5/thumbnails/10.jpg)
MotivationActionable Alert Identification Techniques(AAIT):use the alerts with other information to classify or prioritize alerts.
classification: divide alerts into two groups, UA and AA.prioritization: order alerts by the likelihood an alert is AA.
PRESENTATION TITLE PAGE 10
![Page 11: CS898 PROJECT -- Actionable Alert Detectionmli/Xinye-ActionableAlertDetection.pdf · Build Project PRESENTATION TITLE PAGE 16 Build subject project for each revision. Delete the revisions](https://reader034.vdocuments.net/reader034/viewer/2022051911/60021f5452ec2d2549651d8f/html5/thumbnails/11.jpg)
OUTLINE▪ Background
▪ Motivation
▪ Related Work
▪ Method
▪ Experiment
▪ Summary
PRESENTATION TITLE PAGE 11
![Page 12: CS898 PROJECT -- Actionable Alert Detectionmli/Xinye-ActionableAlertDetection.pdf · Build Project PRESENTATION TITLE PAGE 16 Build subject project for each revision. Delete the revisions](https://reader034.vdocuments.net/reader034/viewer/2022051911/60021f5452ec2d2549651d8f/html5/thumbnails/12.jpg)
Related WorkHeckman and Williams use alert characteristics and machine learning to predict actionable FindBugs alerts. This is one of the most comprehensive actionable alert prediction studies today.
Bodden, Lam and Hendren use static analysis to deduce run-time properties of program. They use decision trees with code characteristics to decrease false positives.
Quinn et al. use alert characteristics and machine learning to find code patterns in static analysis alerts.
PRESENTATION TITLE PAGE 12
![Page 13: CS898 PROJECT -- Actionable Alert Detectionmli/Xinye-ActionableAlertDetection.pdf · Build Project PRESENTATION TITLE PAGE 16 Build subject project for each revision. Delete the revisions](https://reader034.vdocuments.net/reader034/viewer/2022051911/60021f5452ec2d2549651d8f/html5/thumbnails/13.jpg)
OUTLINE▪ Background
▪ Motivation
▪ Related Work
▪ Method
▪ Revision History & Build Project
▪ Run ASA Tool
▪ Alert Statement Slice
▪ Tokenization
▪ Extract Features
▪ ModelPRESENTATION TITLE PAGE 13
![Page 14: CS898 PROJECT -- Actionable Alert Detectionmli/Xinye-ActionableAlertDetection.pdf · Build Project PRESENTATION TITLE PAGE 16 Build subject project for each revision. Delete the revisions](https://reader034.vdocuments.net/reader034/viewer/2022051911/60021f5452ec2d2549651d8f/html5/thumbnails/14.jpg)
Method
PRESENTATION TITLE PAGE 14
Revision History
Run ASA Tool
Alert Statement Slicer
Tokenization N-Gram Features
Subversion Information
Actionable Alert DetectionModel
Build Project
SourceCode Metrics
Alert Information
![Page 15: CS898 PROJECT -- Actionable Alert Detectionmli/Xinye-ActionableAlertDetection.pdf · Build Project PRESENTATION TITLE PAGE 16 Build subject project for each revision. Delete the revisions](https://reader034.vdocuments.net/reader034/viewer/2022051911/60021f5452ec2d2549651d8f/html5/thumbnails/15.jpg)
Revision History
PRESENTATION TITLE PAGE 15
Generate subject revision history from source code repository, like CVS or SVN
![Page 16: CS898 PROJECT -- Actionable Alert Detectionmli/Xinye-ActionableAlertDetection.pdf · Build Project PRESENTATION TITLE PAGE 16 Build subject project for each revision. Delete the revisions](https://reader034.vdocuments.net/reader034/viewer/2022051911/60021f5452ec2d2549651d8f/html5/thumbnails/16.jpg)
Build Project
PRESENTATION TITLE PAGE 16
Build subject project for each revision. Delete the revisions which can not build successfully.
![Page 17: CS898 PROJECT -- Actionable Alert Detectionmli/Xinye-ActionableAlertDetection.pdf · Build Project PRESENTATION TITLE PAGE 16 Build subject project for each revision. Delete the revisions](https://reader034.vdocuments.net/reader034/viewer/2022051911/60021f5452ec2d2549651d8f/html5/thumbnails/17.jpg)
Run ASA Tool
PRESENTATION TITLE PAGE 17
Run Automatic Static Analysis Tool at each revision to get alert information.In our method, we use FindBug.
![Page 18: CS898 PROJECT -- Actionable Alert Detectionmli/Xinye-ActionableAlertDetection.pdf · Build Project PRESENTATION TITLE PAGE 16 Build subject project for each revision. Delete the revisions](https://reader034.vdocuments.net/reader034/viewer/2022051911/60021f5452ec2d2549651d8f/html5/thumbnails/18.jpg)
Alert Statement Slices
PRESENTATION TITLE PAGE 18
program slicer included in the IBM T.J. Watson Libraries for Analysis (WALA)
▪ We select the SA alerts as seed statements▪ The slicer use the source code and seed statements to build a call
graph and pointer analysis▪ Construct backwards slices for each alert
![Page 19: CS898 PROJECT -- Actionable Alert Detectionmli/Xinye-ActionableAlertDetection.pdf · Build Project PRESENTATION TITLE PAGE 16 Build subject project for each revision. Delete the revisions](https://reader034.vdocuments.net/reader034/viewer/2022051911/60021f5452ec2d2549651d8f/html5/thumbnails/19.jpg)
Alert Statement Slices
PRESENTATION TITLE PAGE 19
Original:
int i;int sum=0;int product = 1;for(i=1; i<N; ++i){
sum = sum +i;product = product *i;
}write(sum);write(product);
After slicing:
int i;int sum=0;
for(i=1; i<N; ++i){sum = sum +i;
}write(sum);
![Page 20: CS898 PROJECT -- Actionable Alert Detectionmli/Xinye-ActionableAlertDetection.pdf · Build Project PRESENTATION TITLE PAGE 16 Build subject project for each revision. Delete the revisions](https://reader034.vdocuments.net/reader034/viewer/2022051911/60021f5452ec2d2549651d8f/html5/thumbnails/20.jpg)
TokenizationControl flow information is important
Granularity: High level. consider method, control flow as tokens
Tool: Eclipse JDT Core
PRESENTATION TITLE PAGE 20
![Page 21: CS898 PROJECT -- Actionable Alert Detectionmli/Xinye-ActionableAlertDetection.pdf · Build Project PRESENTATION TITLE PAGE 16 Build subject project for each revision. Delete the revisions](https://reader034.vdocuments.net/reader034/viewer/2022051911/60021f5452ec2d2549651d8f/html5/thumbnails/21.jpg)
Extract N-Gram FeaturesCreate a dictionary of N-gram from the program slice.Use information gain to reduce data dimensionality.Identify 320 features with most information value.
PRESENTATION TITLE PAGE 21
![Page 22: CS898 PROJECT -- Actionable Alert Detectionmli/Xinye-ActionableAlertDetection.pdf · Build Project PRESENTATION TITLE PAGE 16 Build subject project for each revision. Delete the revisions](https://reader034.vdocuments.net/reader034/viewer/2022051911/60021f5452ec2d2549651d8f/html5/thumbnails/22.jpg)
Other FeaturesThere are three potential features for static analysis actionable alerts:
▪ Alert Information▪ Source code metrics▪ Subversion Information
The characteristic for each alert may be different for each alert type, so we need the alert information.Alert Information is retrieved from FindBug for each revision.
PRESENTATION TITLE PAGE 22
![Page 23: CS898 PROJECT -- Actionable Alert Detectionmli/Xinye-ActionableAlertDetection.pdf · Build Project PRESENTATION TITLE PAGE 16 Build subject project for each revision. Delete the revisions](https://reader034.vdocuments.net/reader034/viewer/2022051911/60021f5452ec2d2549651d8f/html5/thumbnails/23.jpg)
Other Features
PRESENTATION TITLE PAGE 23
Group Features
Alert Information(9)
Alert Category
Alert Type
Project Name
Package Name
File Name
Class Name
Method Signature
Priority
Total Alerts for Revision
![Page 24: CS898 PROJECT -- Actionable Alert Detectionmli/Xinye-ActionableAlertDetection.pdf · Build Project PRESENTATION TITLE PAGE 16 Build subject project for each revision. Delete the revisions](https://reader034.vdocuments.net/reader034/viewer/2022051911/60021f5452ec2d2549651d8f/html5/thumbnails/24.jpg)
Other FeaturesCode complexity metrics correlate with failure-prone modeules. Previous work have used code size metrics to predict fault counts.
Use JavaNCSS to generate metrics at the file, package, and project levels.
These tools provide information about the size of source code by lines and the complexity of the programs.
Non commenting source statements (NCSS) counts the number of all statements excluding comments, empty statements, empty blocks, closing brackets or semicolons after closing brackets. PRESENTATION TITLE PAGE 24
![Page 25: CS898 PROJECT -- Actionable Alert Detectionmli/Xinye-ActionableAlertDetection.pdf · Build Project PRESENTATION TITLE PAGE 16 Build subject project for each revision. Delete the revisions](https://reader034.vdocuments.net/reader034/viewer/2022051911/60021f5452ec2d2549651d8f/html5/thumbnails/25.jpg)
Other Features
PRESENTATION TITLE PAGE 25
Group Features
Software Metrics(7) Classes in Package
Functions in Package
Functions in Class
Cyclomatic complexity in Function
Class NCSS
Function NCSS
Package NCSS
![Page 26: CS898 PROJECT -- Actionable Alert Detectionmli/Xinye-ActionableAlertDetection.pdf · Build Project PRESENTATION TITLE PAGE 16 Build subject project for each revision. Delete the revisions](https://reader034.vdocuments.net/reader034/viewer/2022051911/60021f5452ec2d2549651d8f/html5/thumbnails/26.jpg)
Other FeaturesSource code repository help determine how the set of alerts generated by static analysis and how the code base has changed over time.
We use the log files of the subversion repositories to analyze the code history.
PRESENTATION TITLE PAGE 26
![Page 27: CS898 PROJECT -- Actionable Alert Detectionmli/Xinye-ActionableAlertDetection.pdf · Build Project PRESENTATION TITLE PAGE 16 Build subject project for each revision. Delete the revisions](https://reader034.vdocuments.net/reader034/viewer/2022051911/60021f5452ec2d2549651d8f/html5/thumbnails/27.jpg)
Other Features
PRESENTATION TITLE PAGE 27
Subversion (15)
Alert Open Revision
Developers
File Creation Revision
File Last Modified Revision
File Age
Project Added Lines
Project Delete Lines
Project Growth
File Added Lines
File Deleted Lines
File Growth
Package Total Modified Lines
Package Percent Modified Lines
File Total Modified Lines
File Percent Modified Lines
![Page 28: CS898 PROJECT -- Actionable Alert Detectionmli/Xinye-ActionableAlertDetection.pdf · Build Project PRESENTATION TITLE PAGE 16 Build subject project for each revision. Delete the revisions](https://reader034.vdocuments.net/reader034/viewer/2022051911/60021f5452ec2d2549651d8f/html5/thumbnails/28.jpg)
ModelTake input from the 351 featuresAdd two fully-connected layerone dropout layerFinal Layer is Sigmoid layer
Grid Search to tune hyperparameter
PRESENTATION TITLE PAGE 28
![Page 29: CS898 PROJECT -- Actionable Alert Detectionmli/Xinye-ActionableAlertDetection.pdf · Build Project PRESENTATION TITLE PAGE 16 Build subject project for each revision. Delete the revisions](https://reader034.vdocuments.net/reader034/viewer/2022051911/60021f5452ec2d2549651d8f/html5/thumbnails/29.jpg)
OUTLINE▪ Related Work
▪ Method
▪ Experiment
▪ Dataset
▪ Configuration
▪ Ground Truth
▪ Baseline
▪ Evaluation
▪ Results
▪ SummaryPRESENTATION TITLE PAGE 29
![Page 30: CS898 PROJECT -- Actionable Alert Detectionmli/Xinye-ActionableAlertDetection.pdf · Build Project PRESENTATION TITLE PAGE 16 Build subject project for each revision. Delete the revisions](https://reader034.vdocuments.net/reader034/viewer/2022051911/60021f5452ec2d2549651d8f/html5/thumbnails/30.jpg)
Data Set
PRESENTATION TITLE PAGE 30
JDOM Log4jDomain Data Format Logging Library
Size(KLOC) 9-13 12-19
Time Frame 05/2000-12/2008 08/2001-06/2007
# Built Revisions 30 11
Total Alerts 489 237
Actionable Alerts 200 97
Unactionable Alerts 254 112
Deleted Alerts 35 28
![Page 31: CS898 PROJECT -- Actionable Alert Detectionmli/Xinye-ActionableAlertDetection.pdf · Build Project PRESENTATION TITLE PAGE 16 Build subject project for each revision. Delete the revisions](https://reader034.vdocuments.net/reader034/viewer/2022051911/60021f5452ec2d2549651d8f/html5/thumbnails/31.jpg)
Configuration• Gram Size - The size of an n-gram model. 3-gram model• Minimum Token Occurrence - The minimum number of times a token must occur in the software to be included in an n-gram model. 3
PRESENTATION TITLE PAGE 31
![Page 32: CS898 PROJECT -- Actionable Alert Detectionmli/Xinye-ActionableAlertDetection.pdf · Build Project PRESENTATION TITLE PAGE 16 Build subject project for each revision. Delete the revisions](https://reader034.vdocuments.net/reader034/viewer/2022051911/60021f5452ec2d2549651d8f/html5/thumbnails/32.jpg)
Ground TruthBy definition, AA is an alert that a developer resolves by modifying the program. It will disappear from static analysis at some point.
If it is UA, the alert will never disappear. If an alert is removed because the file containing the alert is deleted, we consider the alert status as unknown and remove it from the list.
PRESENTATION TITLE PAGE 32
![Page 33: CS898 PROJECT -- Actionable Alert Detectionmli/Xinye-ActionableAlertDetection.pdf · Build Project PRESENTATION TITLE PAGE 16 Build subject project for each revision. Delete the revisions](https://reader034.vdocuments.net/reader034/viewer/2022051911/60021f5452ec2d2549651d8f/html5/thumbnails/33.jpg)
Ground TruthFaultBench method by Heckman and Williams to accurately classify alerts as AA or UA.
▪ Generate revision history through the source code repository log.▪ Data collection for each project. download all files associated
with a revision.▪ Run ASA at each revision and determine which alerts were
closed during the alert history.▪ Create features for each alert.
PRESENTATION TITLE PAGE 33
![Page 34: CS898 PROJECT -- Actionable Alert Detectionmli/Xinye-ActionableAlertDetection.pdf · Build Project PRESENTATION TITLE PAGE 16 Build subject project for each revision. Delete the revisions](https://reader034.vdocuments.net/reader034/viewer/2022051911/60021f5452ec2d2549651d8f/html5/thumbnails/34.jpg)
BaselineFindBugs assigns a priority measure to each alert. We assume high priority alerts are more actionable than low priority alerts.
Default FindBugs priority ranking:Sort alerts according to the priority measure and randomize the order of alerts with the same priority.
PRESENTATION TITLE PAGE 34
![Page 35: CS898 PROJECT -- Actionable Alert Detectionmli/Xinye-ActionableAlertDetection.pdf · Build Project PRESENTATION TITLE PAGE 16 Build subject project for each revision. Delete the revisions](https://reader034.vdocuments.net/reader034/viewer/2022051911/60021f5452ec2d2549651d8f/html5/thumbnails/35.jpg)
EvaluationTen-fold cross validation to evaluate models.
randomly separate alerts into ten equal sets.nine of the sets train the model and test the model use the last set.repeat the process ten times.
PRESENTATION TITLE PAGE 35
![Page 36: CS898 PROJECT -- Actionable Alert Detectionmli/Xinye-ActionableAlertDetection.pdf · Build Project PRESENTATION TITLE PAGE 16 Build subject project for each revision. Delete the revisions](https://reader034.vdocuments.net/reader034/viewer/2022051911/60021f5452ec2d2549651d8f/html5/thumbnails/36.jpg)
Evaluation MetricsPrecision and Recall to evaluate how well our method performs.
Precision is the percentage of alerts classified as actionable that were actionable.
Recall is the percentage of alerts classified as actionable out of all actual actionable alerts.
PRESENTATION TITLE PAGE 36
![Page 37: CS898 PROJECT -- Actionable Alert Detectionmli/Xinye-ActionableAlertDetection.pdf · Build Project PRESENTATION TITLE PAGE 16 Build subject project for each revision. Delete the revisions](https://reader034.vdocuments.net/reader034/viewer/2022051911/60021f5452ec2d2549651d8f/html5/thumbnails/37.jpg)
Results
PRESENTATION TITLE PAGE 37
Project Average Precision
Average Recall
JDOM 90.3% 86.2%
Log4j 91.4% 85.0%
![Page 38: CS898 PROJECT -- Actionable Alert Detectionmli/Xinye-ActionableAlertDetection.pdf · Build Project PRESENTATION TITLE PAGE 16 Build subject project for each revision. Delete the revisions](https://reader034.vdocuments.net/reader034/viewer/2022051911/60021f5452ec2d2549651d8f/html5/thumbnails/38.jpg)
Results
PRESENTATION TITLE PAGE 38
ExampleIn Log4j, when x=25, it means from the top 25% of the alerts, 74% of actionable alerts are found in our method, while 26% of actionable alerts are found using FindBugs priority ranking.
![Page 39: CS898 PROJECT -- Actionable Alert Detectionmli/Xinye-ActionableAlertDetection.pdf · Build Project PRESENTATION TITLE PAGE 16 Build subject project for each revision. Delete the revisions](https://reader034.vdocuments.net/reader034/viewer/2022051911/60021f5452ec2d2549651d8f/html5/thumbnails/39.jpg)
Results
PRESENTATION TITLE PAGE 39
Our method outperforms FindBugs priority ranking,it can help enhance alert ranking.
![Page 40: CS898 PROJECT -- Actionable Alert Detectionmli/Xinye-ActionableAlertDetection.pdf · Build Project PRESENTATION TITLE PAGE 16 Build subject project for each revision. Delete the revisions](https://reader034.vdocuments.net/reader034/viewer/2022051911/60021f5452ec2d2549651d8f/html5/thumbnails/40.jpg)
OUTLINE▪ Background
▪ Motivation
▪ Related Work
▪ Method
▪ Experiment
▪ Summary
PRESENTATION TITLE PAGE 40
![Page 41: CS898 PROJECT -- Actionable Alert Detectionmli/Xinye-ActionableAlertDetection.pdf · Build Project PRESENTATION TITLE PAGE 16 Build subject project for each revision. Delete the revisions](https://reader034.vdocuments.net/reader034/viewer/2022051911/60021f5452ec2d2549651d8f/html5/thumbnails/41.jpg)
SummaryAfter introducing different methods of actionable alert detection, this project presents the following contributions:1) present a deep learning method to detect actionable alerts.2) use N-Gram feature combining with other alert features.3) apply our method to JDOM and Log4j projects.
In our experiment, our method reach precision up to 91.4% and recall up to 86.2%.Our method outperforms FindBugs priority ranking in alert ranking.
PRESENTATION TITLE PAGE 41
![Page 42: CS898 PROJECT -- Actionable Alert Detectionmli/Xinye-ActionableAlertDetection.pdf · Build Project PRESENTATION TITLE PAGE 16 Build subject project for each revision. Delete the revisions](https://reader034.vdocuments.net/reader034/viewer/2022051911/60021f5452ec2d2549651d8f/html5/thumbnails/42.jpg)
THANKS FOR YOUR TIME!
PAGE 42