considerations from an ipv6 product developer thomas narten [email protected] may 4, 2007, nist
TRANSCRIPT
2 Thomas Narten
Background
IBM has long history of supporting IPv6
Active contributors to IETF IPv6 effort
AIX shipped IPv6 product in 1997
Currently ship IPv6 in i5/OS, AIX, z/OS (all have IPv6 Ready Logo Phase I certification)
Significant developer of IPv6 functionality in Linux
Number of our products support IPv6
–http://www-306.ibm.com/software/info/ipv6/compliance.jsp
IBM is a strong supporter of IPv6!
3 Thomas Narten
Overview
Three flavors of testing
Cost issues for vendors
Product & logistical issues
Harmonization of USG testing efforts
Leverage existing testing programs
Self-certification
Publication of Test Criteria
Ensure adequate accountability
4 Thomas Narten
Three Flavors of Testing
USG operated
3rd Party operated
Self-certification
Questions for each:–How quickly can it ramp up service?
–At what rate can it evaluate products? (e.g., number per month?)
–Can testing be timely?
–What are scaling properties (impacts almost every product)?
–Where does product expertise come from?
–Who bears cost?
–In practice, what will actual cost be?
5 Thomas Narten
Significant Money May Be At Stake
Testing not free; someone bears cost (both direct and indirect)–Assumption: cost will fall on product vendors
If cost too high, some vendors will simply opt out–Consequence: reduced product choice for USG
Business built on providing testing service can be self-serving–Predictable revenue stream needed for business plan
–“required” testing potentially attractive
–Avoid creating a “business” in IPv6 testing
6 Thomas Narten
Offsite & Third Party Testing Costs
Requires hardware to be shipped to test site–Not practical for large servers, high-end configurations
–Not always trivial to acquire actual hardware
–Shipping fees
Direct fee costs to third party–Membership fee
–Per-product fee
–Facilities space
–Third party training (to setup/use/test product)
Travel for testing engineer –Travel cost
–Time away from office
7 Thomas Narten
Product Considerations
Vendor may have 100s of products
Need to avoid/minimize redundant testing–Many releases of (essentially) same product
–Different configurations of same products
–Many applications share code
–Products may contain OEM components that have already been tested
–Not desirable to test/certify each one separately; need way to leverage results of prior testing
Some products are operating system agnostic–Run on top of OS provided by another vendor
–Product may be sold independent of underlying hardware/OS
8 Thomas Narten
Harmonize USG Testing Requirements
Cannot afford to go through same testing process multiple
times for different USG agencies
Ideally, harmonize USG testing requirements...
Even if final profiles differ, 80% of the RFCs overlap
Thus, 80% of testing should also overlap
9 Thomas Narten
Leverage/Reuse Existing Testing Programs
IPv6 Ready Logo (Phase I) already covers core IPv6 protocols–RFCs: 2460 (IPv6), 2461 (ND), 2462 (addrconf), 4443 (ICMPv6).
1981 (PMTU-D)
–Additional Phases as well (e.g., DHCPv6, IPsec, etc.)
–For those already certified, what is benefit of additional testing?
Interoperability testing of IPsec has already been done by ICSA
or VPN Consortium
10 Thomas Narten
Make Test Criteria & Test Suites Publicly Available
Provides transparency w.r.t. details of actual functionality tested
Vendors can test in own labs, as part of product development and test cycle–Facilitates pre-release testing (can be problematical to have outside
party test pre-release software)
–Significantly reduced cost to vendor
Allows vendor to prepare in advance of an external test (where efficiency is important)
Must be free of IPR concerns
Wide availability of TAHI suites has benefited community
11 Thomas Narten
Self-Certification Highly Desirable
Has worked well in practice (e.g., IPv6 Ready Logo, Y2K
preparation, all of TCP/IP, etc.)
Increasingly necessary as one moves higher up the stack
(e.g., into applications)–Significant application-specific expertise needed to test the product
–Infeasible for outside party test the number and range of products
Self-certification to a publicly available criteria–Most efficient
–Scales well
–Good balance between cost and benefit
Neutral third party can verify claims if needed
12 Thomas Narten
Accountability of Testing Program
Any testing/certification suite must provide accountability
IETF defines SHOULD as follows:–“SHOULD This word, or the adjective "RECOMMENDED", mean that
there may exist valid reasons in particular circumstances to ignore a particular item, but the full implications must be understood and carefully weighed before choosing a different course.”
Cannot simply require implementation of all SHOULDs
Need a workable process to resolve disagreements between IPv6 tester and product developer
Need an open process to review test criteria
Need an open process to fix “bugs” in test criteria
13 Thomas Narten
Questions?