nist computer security research, development, and outreach 12/22/2011 rick kuhn
TRANSCRIPT
NIST Computer Security Research, Development, and
Outreach
12/22/2011
Rick Kuhn
Talk Overview
• Who we are, what we do, including some examples of NIST contributions to computer security
• In-depth review of technology that saves a lot of companies a lot of money
12/22/2011● Computer Security Division ●2
Please keep the dull stuff to a minimum!
Conduct research, development and outreach necessary to provide
standards and guidelines, mechanisms, tools, metrics and practices to protect our nation’s
information and information systems.
12/22/2011
3
COMPUTER SECURITY DIVISION (CSD)
● Computer Security Division ●
NIST Computer Security- examples
• Cryptographic standards– Data Encryption Standard (DES) – publicly available encryption
algorithm; developed by IBM to meet NIST requirements– Advanced encryption standard (AES)– SHA family of hash functions– FIPS 140 Cryptographic Module Validation Program
• Role Based Access Control (RBAC) – Used in most large organizations; NIST formal model and standard
• Personal Identity Verification (PIV) and CAC card specs– Standard development and interoperabilty testing
• Security automation, SCAP standard – managing security updates• National Vulnerability Database – publicly known vulnerabilities• Quantum cryptography research – single photon source and detector• Quantum-resistant crypto – resist cryptanalysis using quantum computer
12/22/2011● Computer Security Division ●4
Computer Security Division Groups
• CRYPTOGRAPHIC TECHNOLOGY GROUP (773.01): Research, develop, engineer, and standardize cryptographic algorithms, methods, and protocols.
• SECURITY COMPONENTS AND MECHANISMS GROUP (773.02): Research, develop and standardize foundational security mechanisms, protocols and services.
• SECURE SYSTEMS AND APPLICATIONS GROUP (773.03): Integrate and apply security technologies, standards and guidelines for computing platforms and information systems.
• SECURITY OUTREACH AND INTEGRATION GROUP (773.04): Develop, integrate and promote the mission-specific application of information security standards, guidelines, best practices and technologies.
• SECURITY TESTING, VALIDATION AND MEASUREMENT GROUP (773.05): Advance information security testing, measurement science, and conformance.
12/22/2011
5
● Computer Security Division ●
Core Focus Areas
• Research, Development, Standards, Guidelines, Reference Materials– Security Mechanisms (e.g. protocols, cryptographic, access control,
auditing/logging)– Security Mechanism Applications
• Confidentiality• Integrity• Availability• Authentication• Non-Repudiation• Accessibility• Safety
• Secure System and Component configuration• Conformance validation and assurance of security properties of
products and systems
12/22/2011
6
● Computer Security Division ●
Delivery Mechanisms
1) Standards – FIPS, International Consensus, National Consensus
2) Guidelines – NIST SPs, IRs
3) Journal & Conference papers
4) Reference Materials
5) Workshops & Conferences – hosting and participation in
6) Consortia & Forums
7) Training
8) Reference Implementations & Demonstrations
9) Conformance Verification Activities
10) Test, Tools and other conformance determination tools
11) Standards Development Organization Participation
12/22/2011● Computer Security Division ●7
Community Engagement
• Industry- Accessing Expertise and Leveraging Resources- Coordinating Standards and Initiatives
• Academia- Accessing Expertise and Leveraging Resources- Representative Institutions and Consortia
• International- Formal Standards Groups- Accessing Expertise and Leveraging Resources
• Federal, State, and Local Government- Interdepartmental- Department of Commerce- State and Local Governments
12/22/2011● Computer Security Division ●8
● Chief Information Officers (CIO) Council
● Federal Systems Security Governance Board
Member
● National Cyber Study Group (NCSG) Member
● Cyber Security and Information Assurance
Interagency Working Group
● Information Security Research Council
● Common Terrorism Information Security Standards Working Group
● Committee for National Security Systems
(Observer)
● Information Sharing Environment Enterprise Architecture Security Working Group
● Supply Chain Risk Management Working Group
● Federal Information Systems Security
Educators' Association
● Software Assurance Forum
● IT Entrepreneurs' Forum
● Governance Coordinating Council
● Federal Enterprise Architecture Security and
Privacy Profile Working Group
● Interagency C&A Transformation Working Group
● Internet Engineering Task Force (IETF) Security
Chair
● International Organization for Standardization
(Chair/Convener several Committees, Work
Groups, and Task Forces)
● American National Standards Institute
● International Committee for Information
Technology Standards (Biometrics Chair)
● Biometrics Consortium Co-Chair
● National Science &Technology Council Committee
on Biometrics and Identity Management (Co-
Chair)
12/22/2011● Computer Security Division ●9
Community Engagement Examples
NIST Responsibilities for Cybersecurity• NIST is responsible for developing standards and guidelines, including minimum requirements, that
provide adequate information security for all agency operations and assets in furtherance of its statutory responsibilities under the Federal Information Security Management Act (FISMA) of 2002, Public Law 107-347, but such standards and guidelines shall not apply to national security systems.
• Under FISMA NIST shall “conduct research, as needed, to determine the nature and extent of information security vulnerabilities and techniques for providing cost-effective information security.”
• NIST develops guidelines consistent with the requirements of the Office of Management and Budget (OMB) Circular A-130, Section 8b(3), Securing Agency Information Systems, as analyzed in A-130, Appendix IV: Analysis of Key Sections. Supplemental information is provided in A-130,
• In accordance with the Cyber Security Research and Development Act, The National Institute of Standards and Technology
develops, and revises as necessary, checklists setting forth settings and option selections that minimize the security risks associated with each computer hardware or software system that is, or is likely to become, widely used within the Federal Government.
• Homeland Security Presidential Directive 7; “The Department of Commerce will work with private sector, research, academic, and government organizations to improve technology for cyber systems and promote other critical infrastructure efforts, including using its authority under the Defense Production Act to assure the timely availability of industrial products, materials, and services to meet homeland security requirements.”
• Homeland Security Presidential Directive 12: “The Secretary of Commerce shall promulgate in accordance with applicable
law a Federal standard for secure and reliable forms of identification (the "Standard")”
12/22/2011● Computer Security Division ●10
NIST Responsibilities for Cybersecurity
• Cloud Computing– America COMPETES Reauthorization Act of 2010– Federal Cloud Computing Strategy (February 2011)
• Healthcare– American Recovery and Reinvestment Act of 2009
• Identity Management– National Strategy for Trusted Identities in Cyberspace
• Smart Grid– Energy Independence and Security Act (EISA) of 2007 – American Recovery and Reinvestment Act of 2009
• Voluntary Voting System Standards– Military and Overseas Voter Empowerment (MOVE) Act of 2009– Help America Vote Act
• Cyber Security Education– National Initiative for
Cyber Education (NICE)
12/22/2011● Computer Security Division ●11
When is the part about saving money?
Combinatorial Methods in Software Testing
What is NIST and why are we doing this?• US Government agency, whose mission is to support US industry
through developing better measurement and test methods
• Project goals – reduce testing cost, improve cost-benefit ratio for testing
• Over 26,000 downloads of tutorial
• Users include hundreds of major tech organizations
Need• Exploitable vulnerabilities often arise from
implementation flaws • Analysis of reports in the NVD shows more than half are
buffer errors, memory management, numeric errors, other flaws that are not uniquely security related.
• Testing is typically half of software cost
buffe
r erro
r
SQL in
jectio
n
resou
rce m
gmt
perm
issio
ns, a
cces
s ctl
inpu
t vali
datio
n
code
injec
tion
numeri
c erro
rsot
her
auth
entic
ation
path
trav
ersal
desig
n erro
r
OS com
mand i
njec
tion
crede
ntial
s mgm
g
crypt
ograp
hic i
ssue
conf
igur
ation
race c
ondi
tions
form
at str
ing
info
rmati
on le
ak
cross
site r
eque
st fo
rge
link f
ollo
wing
cross
site s
cript
0
200
400
600
800
1000
1200
Our approach - benefits
• Cooperative R&D Agreement w/ Lockheed Martin• 2.5 year study, 8 Lockheed Martin pilot projects in
aerospace software• Results announced Feb. 2013 • Results: “Our initial estimate is that this method
supported by the technology can save up to 20% of test planning/design costs if done early on a program while increasing test coverage by 20% to 50%.”
Our approach - benefits(cont.)
• Rockwell Collins• “used this approach to automate parts of the unit and
integration testing of a 196 KSLOC avionics system.” • “The goal was to see if it might cost-effectively reduce
rework by reducing the number of software defects escaping into system test”
• “Overcoming scalability issues required moderate effort, but in general it was effective – e.g., generating 47,040 test cases (input vectors, expected outputs) in 75 seconds, executing and analyzing them in 2.6 hours. It subsequently detected all seeded defects, and achieved nearly 100% structural coverage.”
How did we get here?
• NIST studied software failures in 15 years of FDA medical device recall data
• What causes software failures?• logic errors? calculation errors? inadequate
input checking? interaction faults? Etc.
Interaction faults: e.g., failure occurs ifpressure < 10 && volume>300 (interaction between 2 factors)
Example from FDA failure analysis:
Failure when “altitude adjustment set on 0 meters and total flow volume set at delivery rate of less than 2.2 liters per minute.”
Examples from the National Vulnerability Database
• Single variable, 1-way interactionexample: Heap-based buffer overflow in the SFTP protocol handler for Panic Transmit … allows remote attackers to execute arbitrary code via a long ftps:// URL.
• 2-way interactionexample: single character search string in conjunction with a single character replacement string, which causes an "off by one overflow"
• 3-way interactionexample: Directory traversal vulnerability when register_globals is enabled and magic_quotes is disabled and .. (dot dot) in the page parameter
12/22/2011● Computer Security Division ●18
What does a 2-way fault look like in code?How does an interaction fault manifest itself in code?Example: altitude_adj == 0 && volume < 2.2 (2-way interaction)
if (altitude_adj == 0 ) {
// do something
if (volume < 2.2) { faulty code! BOOM! }
else { good code, no problem}
} else {
// do something else
}A test with altitude_adj == 0 and volume = 1 would find this
How are interaction faults distributed?• Interactions e.g., failure occurs if pressure < 10 (1-way interaction) pressure < 10 & volume > 300 (2-way interaction) pressure < 10 & volume > 300 & velocity = 5 (3-way interaction)• Surprisingly, no one had looked at interactions beyond 2-way before • The most complex medical device failure reported required 4-way
interaction to trigger.
0
10
20
30
40
50
60
70
80
90
100
1 2 3 4
Interaction
% d
etec
ted
Interesting, but that's just one kindof application!
Number of factors involved in faults
What about other applications?
Server (green)
0
10
20
30
40
50
60
70
80
90
100
1 2 3 4 5 6
Interactions
% d
ete
cte
d
These faults more complex than medical device software!!
Why?
Number of factors involved in faults
Others?
Browser (magenta)
0
10
20
30
40
50
60
70
80
90
100
1 2 3 4 5 6
Interactions
% d
ete
cte
d
0
10
20
30
40
50
60
70
80
90
100
1 2 3 4 5 6
Interactions
% d
ete
cte
d
Number of factors involved in faults
Still more?
NASA Goddard distributed database (light blue)
0
10
20
30
40
50
60
70
80
90
100
1 2 3 4 5 6
Interactions
% d
ete
cte
d
0
10
20
30
40
50
60
70
80
90
100
1 2 3 4 5 6
Interactions
% d
ete
cte
d
0
10
20
30
40
50
60
70
80
90
100
1 2 3 4 5 6
Interactions
% d
ete
cte
d
Number of factors involved in faults
Even more?
FAA Traffic Collision Avoidance System module (seeded errors) (purple)
0
10
20
30
40
50
60
70
80
90
100
1 2 3 4 5 6
Interactions
% d
ete
cte
d
0
10
20
30
40
50
60
70
80
90
100
1 2 3 4 5 6
Interactions
% d
ete
cte
d
0
10
20
30
40
50
60
70
80
90
100
1 2 3 4 5 6
Interactions
% d
ete
cte
d
0
10
20
30
40
50
60
70
80
90
100
1 2 3 4 5 6
Interactions
% d
ete
cte
d
Number of factors involved in faults
Finally
Network security (Bell, 2006) (orange)
0
10
20
30
40
50
60
70
80
90
100
1 2 3 4 5 6
Interactions
% d
ete
cte
d
0
10
20
30
40
50
60
70
80
90
100
1 2 3 4 5 6
Interactions
% d
ete
cte
d
0
10
20
30
40
50
60
70
80
90
100
1 2 3 4 5 6
Interactions
% d
ete
cte
d
0
10
20
30
40
50
60
70
80
90
100
1 2 3 4 5 6
Interactions
% d
ete
cte
d
Curves appear to be similar across a variety of application domains.
Number of factors involved in faults
Number of factors involved in faults
• New algorithms make it practical to test these combinations• We test large number of combinations with very few tests
• Number of factors involved in failures is small
• Refers to how many parameters are involved in faults: Interaction rule: most failures are triggered by one or two parameters, and progressively fewer by three, four, or more parameters, and the maximum interaction degree is small.
• Maximum interactions for fault triggering was 6• Popular “pairwise testing” not enough • More empirical work needed• Reasonable evidence that maximum interaction strength for
fault triggering is relatively small
Interaction Rule
How does it help me to know this?
How does this knowledge help?
If all faults are triggered by the interaction of t or fewer variables, then testing all t-way combinations can provide strong assurance.
(taking into account: value propagation issues, equivalence partitioning, timing issues, more complex interactions, . . . )
Still no silver bullet. Rats!
Let’s see how to use this knowledge in testing. A simple example:
How Many Tests Would It Take?
There are 10 effects, each can be on or off All combinations is 210 = 1,024 tests What if our budget is too limited for these tests? Instead, let’s look at all 3-way interactions …
There are = 120 3-way interactions. Naively 120 x 23 = 960 tests. Since we can pack 3 triples into each test, we need
no more than 320 tests. Each test exercises many triples:
Now How Many Would It Take?
OK, OK, what’s the smallest number of tests?
103
0 1 1 0 0 0 0 1 1 0
A covering array
Each row is a test:
Each column is a parameter:
• Developed 1990s• Extends Design of Experiments concept• Difficult mathematically but good algorithms now
All triples in only 13 tests, covering 23 = 960 combinations 103
Suppose we have a system with on-off switches. Software must produce the right response for any combination of switch settings:
A larger example
34 switches = 234 = 1.7 x 1010 possible inputs = 1.7 x 1010 tests
How do we test this?
• 34 switches = 234 = 1.7 x 1010 possible inputs = 1.7 x 1010 tests• If only 3-way interactions, need only 33 tests• For 4-way interactions, need only 85 tests
What if we knew no failure involves more than 3 switch settings interacting?
33 tests for this range of fault detection
85 tests for this range of fault detection
That’s way better than 17 billion!
Number of factors involved in faults
Example: Network Deadlock Detection
• “Simured” network simulator• Kernel of ~ 5,000 lines of C++ (not including GUI)
• Objective: detect configurations that can produce deadlock:
• Prevent connectivity loss when changing network• Attacks that could lock up network
• Compare effectiveness of random vs. combinatorial inputs
• Deadlock combinations discovered
Detection rates Deadlocks Detected:
combinatorial
t Tests500 pkts
1000 pkts
2000 pkts
4000 pkts
8000 pkts
2 28 0 0 0 0 03 161 2 3 2 3 34 752 14 14 14 14 14
Average Deadlocks Detected: random
t Tests500 pkts
1000 pkts
2000 pkts
4000 pkts
8000 pkts
2 28 0.63 0.25 0.75 0. 50 0. 753 161 3 3 3 3 34 752 10.13 11.75 10.38 13 13.25
Results
Detected 14 configurations that can cause deadlock: 14/ 31,457,280 = 4.4 x 10-7
Combinatorial testing found more deadlocks than random, including some that might never have been found with random testing
Why do this testing? Risks:• accidental deadlock configuration: low• deadlock config discovered by attacker: much higher
(because they are looking for it)
Example: Laptop application testing
Problem: connect many peripherals, order of connection may prevent proper operation
Sequence Covering Array• With 6 events, all sequences = 6! = 720 tests• Only 10 tests needed for all 3-way sequences, even better for larger numbers of events;
• Results: problems detected with far fewer tests• Example: .*c.*f.*b.* covered. Any such 3-way seq covered.
Test Sequence1 a b c d e f2 f e d c b a3 d e f a b c4 c b a f e d5 b f a d c e6 e c d a f b7 a e f c b d8 d b c f e a9 c e a d b f
10 f b d a e c
What tools are available?• Covering array generator – basic tool for test input or
configurations;
• Sequence covering array generator – new concept; applies combinatorial methods to event sequence testing
• Combinatorial coverage measurement – detailed analysis of combination coverage; automated generation of supplemental tests; helpful for integrating c/t with existing test methods
• Domain/application specific tools:• Access control policy tester• .NET config file generator
• Smaller test sets faster, with a more advanced user interface• First parallelized covering array algorithm• More information per test
126001070048>1 dayNA47011625>1 dayNA65.03109416
1549313056>1 dayNA43.544580>1
dayNA18s42265
12764696>21 hour14763.541536540014843.0513634
3.079158>12 hour4720.71413102023880.364003
2.75101>1 hour1080.0011080.731200.81002
TimeSizeTimeSizeTimeSizeTimeSizeTimeSize
TVG (Open Source)TConfig (U. of Ottawa)Jenny (Open Source)ITCH (IBM)IPOGT-Way
New algorithms
Traffic Collision Avoidance System (TCAS): 273241102
Times in seconds
ACTS - Defining a new system
Constraints
Variable interaction strength
Covering array output
ACTS Users: > 1,000 organizations
Information Technology
Defense
Finance
Telecom
Output optionsMappable values
Degree of interaction coverage: 2Number of parameters: 12Number of tests: 100
-----------------------------
0 0 0 0 0 0 0 0 0 0 0 0 1 1 1 1 1 1 1 0 1 1 1 1 2 0 1 0 1 0 2 0 2 2 1 0 0 1 0 1 0 1 3 0 3 1 0 1 1 1 0 0 0 1 0 0 4 2 1 0 2 1 0 1 1 0 1 0 5 0 0 1 0 1 1 1 0 1 2 0 6 0 0 0 1 0 1 0 1 0 3 0 7 0 1 1 2 0 1 1 0 1 0 0 8 1 0 0 0 0 0 0 1 0 1 0 9 2 1 1 1 1 0 0 1 0 2 1 0 1 0 1 Etc.
Human readable
Degree of interaction coverage: 2Number of parameters: 12Maximum number of values per parameter: 10Number of configurations: 100-----------------------------------Configuration #1:
1 = Cur_Vertical_Sep=2992 = High_Confidence=true3 = Two_of_Three_Reports=true4 = Own_Tracked_Alt=15 = Other_Tracked_Alt=16 = Own_Tracked_Alt_Rate=6007 = Alt_Layer_Value=08 = Up_Separation=09 = Down_Separation=010 = Other_RAC=NO_INTENT11 = Other_Capability=TCAS_CA12 = Climb_Inhibit=true
Rick Kuhn Raghu Kacker [email protected] [email protected]
http://csrc.nist.gov/acts
Please contact us if you are interested.