it infrastructure partnership program (itp) 0 it infrastructure partnership program fred duball and...
TRANSCRIPT
1
IT Infrastructure Partnership Program (ITP)
IT Infrastructure Partnership Program
Fred Duball and Joe Fay
Program Directors
July 18, 2007ITIB IT Infrastructure Committee
2
IT Infrastructure Partnership Program (ITP)
Agenda
• Service Delivery
– Service Delivery Metrics
– Customer Satisfaction
– Request for Service
– P2P Procurement
• Transformation
• Partnership Financials
• Independent Program Maturity Assessment
3
IT Infrastructure Partnership Program (ITP)
Service Delivery
4
IT Infrastructure Partnership Program (ITP)
Jun
• Significant Incidents:
– Controller switch failure disrupted communication between mainframe and tape drives- mainframe services down all day 6/20, primary impact DMV, DSS, VEC + 12 other agencies – Interim fix 6/20 Final fix 6/24
– User executable caused the Unisys mainframe multiple brief outages on 6/22, 7/3 and 7/4. Primary Impact DSS. Final patch applied 7/15.
– DCR call center bandwidth not adequate for peak period VOIP Calls –customer calls garbled for 1.5 weeks. Resolved 6/8 with new DS3 circuit.
– VDOT Bristol Email and BlackBerry E-mail Failure – e-mail down 610 users for 5 days, BB e-mail down 16 users for 22 days. Resolved 6/12
– Symantec uninstall at Library of Virginia did not work– 40 PCs down 1 day, 7 PCs down 1.5 days. Resolved 5/15
– Combination of server and network configuration issues results in DFP slow network – response for primary applications slow for 3 months. Planned remediation complete 9/14
TargetGreen -
Meet or exceed target within 10%
Blue -Exceed target by > 10% or reach maximum of 100%
Yellow -Fail to reach target
within 10%
Red -Fail to reach target
by > 10%
Legend
Central Metrics
Service Delivery Dashboard
Field Metrics
7.1%
0%
64.3%
28.6%
Apr
0%
0%
57.1%
42.9%
May
0%
7.1%
42.8%
50.0%
Apr May Jun
R
GY
B
0.0%
0.0%
68.8%
31.25%
0.0%
6.25%
62.5%
31.25%
R
GY
B
3.9%
1.7%
88.2%
6.2%
5
IT Infrastructure Partnership Program (ITP)
Domain MeasureMOU-SLO
A S O N D J F M A M J
End User
Average Speed to Answer
<30 sec 33 22 14 22 18 26 28 47 47 28 18
Call Abandon Rate < 5% 6.1% 2.1% 0.8% 2.0% 1.5% 2.2% 2.1% 3.1% 3.3% 3.7% 2.5%
Email Response <60 mins
15 16 18 16 16 18 18 25 22 28 46
Voicemail Response <30 mins
15 16 16 16 16 18 18 25 22 28 28
First Call Resolution >70% 20% 21% 61% 67% 73% 74% 77% 79% 83% 86% 83%
VITA Messaging System Availability
>99.0% 99.9% 100% 99.9% 100% 99.9% 100% 99.7% 99.8% 100% 100% 99.9%
Shared Messaging System Availability
>99.0% 100% 99.9% 100% 100% 99.9% 100% 100% 99.4% 100% 100% 99.5%
Data Center
IBM Mainframe Availability
>99.9% 99.9% 100% 100% 100% 100% 100% 100% 100% 99.9% 99.9% 98.6%
Unisys Mainframe Availability
>99.9% 100% 99.9% 100% 100% 100% 100% 100% 100% 100% 100% 99.9%
UNIX Server Availability
>99% 99.8% 99.9% 99.8% 99.9% 99.9% 99.9% 99.8% 99.9% 99.9% 99.9% 99.9%
Windows Server Availability
>99% 99.9% 99.3% 100% 99.9% 99.4% 99.9% 99.9% 99.8% 99.9% 99.9% 99.9%
Network Circuits Availability 99.2% 99.5% 99.4% 99.8% 99.7% 99.7% 99.6% 99.7% 99.4 99.6% 99.6% N/A
Security
ACF2 Logon Requests
95% 100% 99% 100% 100% 100% 100% 100% 100% 100% 100%
Security Incident Reporting
95% 100% 100% 100% 100% 100% 100% 100% 100% 100% 100%
FCR – Start Oct measure is (Resolved on First Call/Can be Resolved on First Call. Previous measure was (Resolved on First Call/Total Calls). May NG/VITA agreed to First Call Resolvable categories
TargetGreen -
Meet or exceed target within 10%
Blue -Exceed target by > 10% or reach maximum of 100%
Yellow -Fail to reach target
within 10%
Red -Fail to reach target
by > 10%
Legend
Central Operation Metrics
6
IT Infrastructure Partnership Program (ITP)
Speed to Answer
0
5
10
15
20
1/27
2/10
2/24
3/10
3/24
4/7
4/21
5/5
5/19
6/2
6/16
6/30
AgentsStaffing
Slots
Actions to address problem:
• Apr/May – incentives to agents to meet goals
• May/Jun –Added interim support through overtime and temp assignment of up to 3 field techs
• May/Jun/Jul – For longer term fix, increased agent staffing from 11 (Apr) to 21 (Jul)
• 11 Jun – Jan Weiner, Lafayette QA Manager, is Interim VCCC Manager to standardize processes/procedures and optimize Peregrine utilization.
0
200
400
600
800
1000
1200
1400
1600
1/27
2/10
2/24
3/10
3/24
4/7
4/21
5/5
5/19
6/2
6/16
6/30
Call VolumeCalls
0
10
20
30
40
50
60
70
80
1/27
2/10
2/24
3/10
3/24
4/7
4/21
5/5
5/19
6/2
6/16
6/30
Seconds
Week Average
Month AverageGoal
Actual
Plan
Speed to Answer
7
IT Infrastructure Partnership Program (ITP)
SLO#
AgenciesMOU S O N D J F M A M J
Average Speed to Answer 3 30 Sec 16 16 20 20 22 21 16 22 22
Call Abandonment Rate 3 5.0% 7.0% 8.4% 9.4% 5.3% 4.3% 6.4
First Call Resolution 12 70.0% 76.0% 79.3% 84.6% 84.7% 86.9% 87.7% 84.7% 88.0% 92.8% 91.9
End User
Average Time On-hold 2 90 Sec 5 4.5 5.5 5 3 5 6.5 3 3 3
Help Desk Password Resets
24 90.0% 87.6% 94.8% 93.9% 96.8% 92.8% 98.1% 94.6% 96.8% 97.5% 97.4
Service via Incident Ticket 22 78.0% 81.8% 82.7% 79.2% 82.6% 77.8% 79.1% 82.6% 85.9% 90.0% 85.4
Service via Service Request
25 80.0% 86.6% 86.9% 89.7% 91.4% 90.7% 92.5% 89.6% 94.2% 95.3% 96
Incident Repair 23 80.0% 84.3% 83.4% 82.6% 88.1% 86.4% 89.4% 88.1% 91.0% 92.0% 92.1
Messaging Service 27 99.0% 99.6% 100% 99.9% 100% 99.9% 100% 100% 99.9% 99.9% 99.9
Windows Mission Critical Servers
41 99.0% 99.1% 100% 100% 100% 100% 99.9% 100% 99.9% 99.8% 99.9
RISC/Unix Mission Critical Servers
14 99.0% 100% 99.8% 100% 99.9% 100% 100% 100% 99.9% 99.8% 100
Data Center
Windows Other Server 34 90.0% 100% 100% 100% 100% 100% 100% 100% 100% 100% 100
RISC/Unix Other Servers 11 90.0% 100% 100% 100% 100% 100% 99.9% 100% 100% 100% 100
QA/Test Systems and Servers
11 90.0% 100% 100% 100% 100% 100% 100% 100% 100% 100% 100
Development Servers 17 90.0% 100% 99.9% 100% 100% 100% 100% 99.5% 100% 100% 100
Network Internet Access 16 99.8% 99.5% 100.0% 100% 100% 99.9% 100% 99.9% 99.9% 99.9% 99.9
Call Abandonment Rate recalculated in Apr to account for calls received only during Help Desk operating hours and not count abandons during first 30 seconds. Dec-Mar identifies previous calculation above, recalculated below.
11.2%
4.0% 6.6% 5.3%
12.7% 11.0% 12.7%
5.5%
TargetGreen -
Meet or exceed target within 10%
Blue -Exceed target by > 10% or reach maximum of 100%
Yellow -Fail to reach target
within 10%
Red -Fail to reach target
by > 10%
Legend
Field Metrics
8
IT Infrastructure Partnership Program (ITP)
Partnership continues to reinforce importance of customer satisfaction and customer service mind-set
• Service delivery - Measuring and improving– Reduced customer wait time - average speed-to-answer for VCCC at 18 seconds for June
– Reduced down time - Verizon mean-time-to-repair on circuit outages under four hour SLA/4th month running
– 93.2% on-time installation - processed 929 telecommunication service requests in May
• Request for Service (RFS) and Procure to Pay (P2P) improvements reduced process time
– Combined tasks in parallel
– Consolidating desktop orders through Product Support Center (PSC)
– Drill down reporting of tasks and steps
• Actively engaging customers
9
IT Infrastructure Partnership Program (ITP)
Question 2: Timeliness74.9%
16.0%
2.3%
2.8%
2.1%
1.9%
0% 10% 20% 30% 40% 50% 60% 70% 80%
Strongly Agree
Agree
Slightly Agree
Slightly Disagree
Disagree
Strongly DisagreeCu
sto
me
r A
gre
em
en
t
Overall
Question 4: Staff Knowledge76.6%
18.0%
2.4%
1.7%
0.7%
0.7%
0% 10% 20% 30% 40% 50% 60% 70% 80%
Strongly Agree
Agree
Slightly Agree
Slightly Disagree
Disagree
Strongly DisagreeCu
sto
me
r A
gre
em
en
t
Overall
Question 3: Staff Professionalism78.9%
17.8%
1.2%
0.9%
0.0%
1.2%
0% 10% 20% 30% 40% 50% 60% 70% 80%
Strongly Agree
Agree
Slightly Agree
Slightly Disagree
Disagree
Strongly DisagreeCu
sto
me
r A
gre
em
en
t
Overall
Question 5: Overall Satisfaction76.1%
17.1%
2.1%
2.1%
0.9%
1.6%
0% 10% 20% 30% 40% 50% 60% 70% 80%
Strongly Agree
Agree
Slightly Agree
Slightly Disagree
Disagree
Strongly DisagreeCu
sto
me
r A
gre
em
en
t
Overall
Question 1: Effectiveness76.0%
17.7%
1.9%
1.9%
0.9%
1.6%
0% 10% 20% 30% 40% 50% 60% 70% 80%
Strongly Agree
Agree
Slightly Agree
Slightly Disagree
Disagree
Strongly DisagreeCu
sto
me
r A
gre
em
en
t
Overall
Q1 Avg. = 5.61
Customer satisfaction is high for desktop support
Q2 Avg. = 5.53
Q5 Avg. = 5.60
Q3 Avg. = 5.71
Q4 Avg. = 5.66
• Period of Analysis: 04/07/07 – 06/29/07
• Number of Responses: 435
• Number of Resolved Tickets: 3807
• Response Rate: 11.4%
10
IT Infrastructure Partnership Program (ITP)
Question 4: Staff Knowledge60.1%
25.7%
7.9%
2.2%
1.7%
2.4%
0% 10% 20% 30% 40% 50% 60% 70% 80%
Strongly Agree
Agree
Slightly Agree
Slightly Disagree
Disagree
Strongly DisagreeCu
sto
me
r A
gre
em
en
t
Overall
Question 3: Staff Professionalism69.5%
24.1%
3.4%
1.0%
0.4%
1.6%
0% 10% 20% 30% 40% 50% 60% 70% 80%
Strongly Agree
Agree
Slightly Agree
Slightly Disagree
Disagree
Strongly DisagreeCu
sto
me
r A
gre
em
en
t
Overall
Question 5: Overall Satisfaction63.4%
23.1%
5.6%
2.0%
2.2%
3.7%
0% 10% 20% 30% 40% 50% 60% 70% 80%
Strongly Agree
Agree
Slightly Agree
Slightly Disagree
Disagree
Strongly DisagreeCu
sto
me
r A
gre
em
en
t
Overall
Question 2: Timeliness61.8%
24.9%
5.1%
2.0%
2.7%
3.4%
0% 10% 20% 30% 40% 50% 60% 70% 80%
Strongly Agree
Agree
Slightly Agree
Slightly Disagree
Disagree
Strongly DisagreeCu
sto
me
r A
gre
em
en
t
Overall
Question 1: Effectiveness62.4%
26.1%
4.8%
1.8%
2.3%
2.7%
0% 10% 20% 30% 40% 50% 60% 70% 80%
Strongly Agree
Agree
Slightly Agree
Slightly Disagree
Disagree
Strongly DisagreeCu
sto
me
r A
gre
em
en
t
Overall
Q1 Avg. = 5.36
Customer satisfaction is high for help desk services
Q2 Avg. = 5.31
Q5 Avg. = 5.32
Q3 Avg. = 5.57
Q4 Avg. = 5.33
• Period of Analysis: 02/09/07 – 06/29/07
• Number of Responses: 1282
• Number of Resolved Tickets: 27172
• Response Rate: 4.7%
11
IT Infrastructure Partnership Program (ITP)
Customer satisfaction is high for messaging services
E-mail Easy to Use26.1%
58.7%
8.8%
4.4%
1.8%
0.3%
0% 10% 20% 30% 40% 50% 60%
Strongly Agree
Agree
Slightly Agree
Slightly Disagree
Disagree
Strongly DisagreeCu
sto
me
r A
gre
em
en
t
Q5 Avg. = 5.02(341 Responses)
Availability of E-mail Support/Resources
12.0%
49.2%
21.3%
8.7%
7.5%
1.2%
0% 10% 20% 30% 40% 50% 60%
Strongly Agree
Agree
Slightly Agree
Slightly Disagree
Disagree
Strongly DisagreeCu
sto
me
r A
gre
em
en
t
Q11 Avg. = 4.46(333 Responses)
Overall Satisfaction with E-mail16.5%
57.6%
16.2%
5.0%
4.1%
0.6%
0% 10% 20% 30% 40% 50% 60%
Strongly Agree
Agree
Slightly Agree
Slightly Disagree
Disagree
Strongly DisagreeCu
sto
me
r A
gre
em
en
t
Q12 Avg. = 4.76(340 Responses)
• Survey Period: 04/30/07 – 05/14/07
• Number of Responses: 348
• Number of Recipients: 3,567
• Response Rate: 9.8%
Webmail Meets Needs21.6%
43.1%
20.3%
5.2%
8.5%
1.3%
0% 10% 20% 30% 40% 50% 60%
Strongly Agree
Agree
Slightly Agree
Slightly Disagree
Disagree
Strongly DisagreeCu
sto
me
r A
gre
em
en
t
Security of E-mail
17.3%
44.9%
20.8%
7.6%
6.2%
3.2%
0% 10% 20% 30% 40% 50% 60%
Strongly Agree
Agree
Slightly Agree
Slightly Disagree
Disagree
Strongly DisagreeCu
sto
me
r A
gre
em
en
t
Q10 Avg. = 4.50(341 Responses)
Q14A Avg. = 4.60(153 Responses)
12
IT Infrastructure Partnership Program (ITP)Request for Service (RFS) Process Monthly Totals
Request for Service Monthly Totals
45
6 9 14 917
21
3 25
20
0
10
20
30
40
50
60
70
Jul-Dec 2006 Jan-07 Feb-07 Mar-07 Apr-07 May-07
Time Period
Quantity
Status Totals Total Value
Total RFS Requests 67
Proposals in development 22
Proposals submitted to customer 45 $9.42M
Proposals under review by customer 18 $2.99M
Proposals authorized to proceed 27 $6.43M
RFS projects in progress 11 $1.68M
RFS projects completed 16 $4.75M
• RFS is new “project” work, incremental to the contract, such as:
– Providing infrastructure in a new Commonwealth facility or supporting an agency move
– Standing up infrastructure to support a new information system
13
IT Infrastructure Partnership Program (ITP)
Improved P2P (Procure to Pay) Process
• Near-term Improvements: Communicate standards to the agency community
Streamline approvals; parallel vs. sequential
Move Northrop Grumman Procurement workflow to work parallel with eVA workflow
Use Help Desk to initiate and verify installation – June 15
- Utilize Peregrine to track PRs/POs from submission to installation and generate automated reports
• Long-term Improvement Strategy:– Develop and deploy service catalog
– Consolidate contracts - 4Q07
*Assumption is that no approvals are escalated
14
IT Infrastructure Partnership Program (ITP)
P2P end-to-end processing time reduced
5.6
Note: The end-to-end processing time includes the time from submission of the eVA request until the proof of delivery is received from the vendor.
65.771.5
77.2
51.9
36.3
15.8
0
10
20
30
40
50
60
70
80
Average Number of Days
January February March April May June
Month
TOTAL NUMBER OF LINE ITEMS
Jan 531
Feb 967
Mar 623
Apr 332
May 603
June 1006
Line items = quantity of products (1 to n) on a single line of PO
15
IT Infrastructure Partnership Program (ITP)
End-to-end processing for standard products
Total number of items procured
Product May June
PCs 591 439
Monitors 430 206
Printers 84 82
Note: The end-to-end processing time includes the time from submission of the eVA request until the proof of delivery is received from the vendor.
31.9
13.0 14.816.7
0
5
10
15
20
25
30
35
Average Days
PCs &Monitors Std Printers
Product
May June
16
IT Infrastructure Partnership Program (ITP)
End-to-end processing for non-standard products
Total number of items procured
Product May JuneServers 61 42
Network 193 1,275
Software --- 3,837
Maintenance 902 506
Misc. 3,060 3,533
17.115.4
23.0
14.0 14.1
7.4
15.4
39.7
15.5
0
5
10
15
20
25
30
35
40
Average Days
Products
May June
Note: The end-to-end processing time includes the time from submission of the eVA request until the proof of delivery is received from the vendor.
17
IT Infrastructure Partnership Program (ITP)
Service Catalog• When
– Initial production phase start: November 15, 2007
• Benefit– Ability for end user to select “pre-packaged” bundles for quick order
– Allows standardization across the Commonwealth
– Easy approval process
– For common systems “extra” and “additional” features easily identified and selected
• Impact– End user training for a new order process
– Agency mapping for approval process
– Custom orders still may require current P2P process
– Possible changes to firewall to allow access to Web-based tool
18
IT Infrastructure Partnership Program (ITP)Service Delivery ImprovementsService Delivery Agency Focused
Improvements
‾ Library of Virginia now using Enterprise E-mail
‾ Migrated Tax i-File to new server hardware/software – hosted in RPB – can accommodate more concurrent users when filing taxes.
‾ Migrated VDOT to new e-mail software – more reliable and many new features.
Transformation Deployed to Operations – Desktops (5000+ deployed)
• Standard platform with anti virus and firewall security
• Reduces support calls required for older equipment
– Global Address List
• Single e-mail address list for the Commonwealth of VA reducing support calls
– Peregrine live at 7 sites (14 waiting agency signoff)
• Centralized ticket reporting to support teams
• Easier metrics consolidation and monitoring
– Server consolidation
• Standardizing hardware platforms and added redundancy to the architecture allows for fewer problem outages.
– Disk Consolidation – storage
• Removes older hardware from the architecture.
• Rapid deployment of new storage requirements
• Data replication to improve recovery timeframes
– Piloted ITIL
• Integrated change management to increase successful change and coordination for infrastructure teams
– Network MPLS Points of presence
• Provides redundancy for main core infrastructure.
• Less down time and allows for maintenance to be performed while keeping network available
19
IT Infrastructure Partnership Program (ITP)
Enterprise Exchange/Outlook EmailEnterprise Collaboration Tools
Active Directory, DNS$25M Investment
Messaging ServicesDesktop
Mass Desktop Refresh ProjectsNetwork Printer Consolidation and RefreshEnterprise Desktop Management Systems
$35M Investment
Enterprise Help Desk in Russell and ChesterfieldField Based Agents and Technicians for Level 3
Enterprise Help Desk System (Peregrine)$10M Investment
Help Desk
New IBM and Unisys Mainframes Consolidation and Refresh of Servers
Migration of servers to the Data Center$50M Investment
Mainframeand Servers
Enterprise Security Operations CenterComputer Security Incident Response Center
Secure Internet Gateway$10M Investment
Security
New Data Center/Office Building in ChesterfieldNew Disaster Recovery Center and Help Desk
in Russell County$60M Investment
Facilities Tier 3 and Tier 2
New Commonwealth-wide MPLS Core WANLAN upgrades to Local Switches/Routers as Needed
Network Re-addressing of IP, DHCP$60M Investment
Network
Voice over IP Network Optimized for Voice and Video Traffic
$20M Investment
Voice / Video
People – Process – ToolsReliable, High Performance,
Enterprise-Wide IT Infrastructure
$270 Million Investment
20
IT Infrastructure Partnership Program (ITP)
2006 2007 2008 2009+SAJ DNO FJ M SAJM DNOJA
Service Commencement Date 7/1/06General
Transformation Phase (36 Months to June 2009)Domains
ESOC Complete SCD+23 (6/1/08)
Security Interim SecurityIncident trackingand Mgmt SystemSCD+3 (10/1/06)
Enterprise Vulnerability Assessment Program Operational SCD+20 (3/1/08)
Enterprise Vulnerability Assessment Program Operational SCD+20 (3/1/08)
CSIRC CompleteSCD+20 (3/1/08)CSIRC CompleteSCD+20 (3/1/08)
MPLS Core Complete SCD+14 (9/1/07)
Complete Agency LAN migration (90%)SCD+30 (1/1/09)
Enterprise NOCSCD+16 (11/1/07)
DataNetwork
Temp. NOC(11/1/06)
VoiceVoice VoIP Architecture DesignAnd Recommendations
SCD+9 (4/1/07)
VoIP Architecture DesignAnd Recommendations
SCD+9 (4/1/07)
VoIP Completion (90%)SCD+63 10/01/11
VoIP Completion (90%)SCD+63 10/01/11
Procedures Manual (10/1/06)Procedures Manual (10/1/06)
ITIL Process Optimization Complete SCD+23 (6/1/08)ITIL Process Optimization
Complete SCD+23 (6/1/08)
DR Test at SWESC SCD+22 (5/1/08)DR Test at SWESC SCD+22 (5/1/08)
FJ M SAJM DNOJA FJ M A
Messaging
monthsmonths
+24+24 +30+30
DC
SD
CS
NV
SN
VS
SS
SS
RPB MigrationComplete
SCD+19 (2/1/08)
CESC Ready for Occupancy
SCD+12 (7/1/07)
SWESC ReadyFor Occupancy
SCD+16 (11/1/07)
FacilitiesFacilities
Server Consolidation90% Complete
SCD+35 (6/1/09)
Mainframe /Server
Mainframe / server workload migration from RPB to CESC
SCD+18 (1/1/08)
Help Desk Production Incident Mgmt System / SPOC Help Desk (SWESC)
SCD+24 (7/1/08)
Complete Desktop RefreshSCD+32 (3/1/09)
Desktop
Single Statewide Address List SCD+9 (P4/1/07-A3/22/07)
Enterprise messaging 90% complete
SCD+ 35 (6/1/09)
DNS / WINS InfrastructureSCD+13 (9/1/07)
EU
SE
US
Infrastructure Ops Center(Interim) 11/1/06)
Begin Desktop Refresh (P3/01/07-A3/15/07)
Incident Mgmt. Web Accessible (8/1/06)
Knowledge Mgt. System Operational P10/1/06-A4/01/07
Procedures Manual Plan (8/1/06)
Process Cutover (Internal Apps)
Desktop & asset mgmt system operational (P1/1/07-A1/29/07)
Arch Network Blueprint Addressing Plan (2/1/07)
+12+12 +18+18+6+6SCDSCD
= Delivered, awaiting final VITA acceptance
= Delivered = Delayed
21
IT Infrastructure Partnership Program (ITP)
Internal AppsGeneral Services Cross
Functional
SecuritySecurity Services
Voice
DataNetwork Services
Mainframe and Server
FacilitiesData Center Services
Messaging
Desktop
Help Desk
End User Services
Q4 07Q3 07Q2 07Technology Domain
Service Area
Qtr 2 Desktop Refresh (7/1)
Qtr 3 Desktop Refresh (10/1)
RPB Server consolidation complete(5/15/07)
CSIRC System Detailed Design (5/15/07)
ESOC System detailed Design (6/1/07
CESC Ready for Occupancy (7/1/07)
MPLS Core Complete Begin Agency Migration (9/1/07)
Change Management “go live”
15% LAN MigrationSCD+15 (10/1/07)
Begin Messaging Refresh 10/1)
Standup Messaging Back End (8/30)
Connectivity to CESC (7/1/07)
VITA security Dashboard (09/10/07)
Installation of Telephony and Dedicated Incident Mgt System and Agent workstations installed (12/07)
Qtr 1 Desktop Refresh (4/1)
Qtr 4 Desktop Refresh (01/08)
Commonwealth GAL Update (4/1/07)
SWESC Ready for Occupancy (11/1/07)
ESOC Transitional Complete (11/1/07)
Enterprise NOC (11/1/07)
Connectivity to SWESC (11/1/07)
Move Mainframe/Server workload from RPB to CESC (1/1/08)
Commonwealth Server Consolidation 25% complete
(11/1/07)
RPB Server testing begins (6/06/07)
Phase 1 Change, Config and Release Mgt begins (4/01/07) Phase 2 Incident Mgt Begins
(8/20/07)
1st Mainframe Mock cutover test (10/1/07)
Service Catalog Production Demo (9/30/07)
Service Catalog Production Demo (11/15/07)
22
IT Infrastructure Partnership Program (ITP)Transformation Dashboard
EUS
Desktop
1) Moved from red to yellow May 21 – Will move from yellow to green with achievement of scheduled refresh rate
2) Asset management system delivered and accepted
3) Q1 desktop refresh completed. Target 3484+ systems
4) Q2 desktop refresh on target. Target additional 2500+ systems
Helpdesk
1) Knowledge management system in post live test. Target July 15th for testing complete
2) Agency migration schedule to Peregrine system has been defined and started
3) Project upgrade to green status based on achievement of scheduled rate, planned by August 1
Messaging
1) Change in storage solution to EMC will impact stand-up of backend infrastructure in CESC
2) AITR approval to proceed with pilot migration for Department for the Aging and Virginia Department of Health
3) Project will go from yellow to green when timeline from storage solution change is brought back onto baseline
DCS
Server
1) Phase 1 server consolidation complete at RPB
2) Phase 2 server consolidation begins for greater Richmond area data centers
3) Monitor server move into CESC. Will move from yellow to green after successful execution of first phase moves
Mainframe1) New mainframe deployment at CESC in progress
2) Monitor mainframe test at CESC. Will move from yellow to green after successful execution of first phase of testing
Facilities 1) CESC on schedule. SWESC on schedule
NWS
Data Network
1) Pilot deployments began – June 30
2) Modifying network rollout schedule for ABC and VEC requirements
3) Will move from yellow to green after successful execution of first agency to the MPLS infrastructure
Voice Network 1) Awaiting business decision on VoIP Engineering Change Proposal for CESC
Security 1) Two week delay on Internet security gateway beta agency testing
2) Beta testing for three agencies schedule in progress; will move from yellow to green after successful testing
GEN
Internal Apps 1) Delay in briefing VITA on the “managed service” chargeback offering warrants yellow status
Cross-Functional1) ITP change management training goals achieved2) VITA change management pilot in progress3) Phase 2 process design complete for incident, problem, capacity, availability processes
23
IT Infrastructure Partnership Program (ITP)
Desktop Refresh• When
– Q3 refresh targeting 4500+ additional systems
• Benefit– New PCs for workers
– Standardized environment
– Altiris system management tool deployed
– Standard antivirus software deployed
– Firewall for desktops deployed
• Impact– Potential of 2-4 hours of end user down time during PC upgrade
– Agency testing of application packages prior to deployment
– Site operations staff assistance during refresh activity
– Equipment storage and agency logistic support may be required
24
IT Infrastructure Partnership Program (ITP)
Desktop Quarterly Run Rate
Desktop Refresh Plan
0
10000
20000
30000
40000
50000
60000
Milestone Quarters
Cu
mu
lati
ve
To
tal
Un
its
Proposed Plan
Actual Run Rate
Proposed Plan 3484 5984 10484 18484 26884 35284 43284 51284
Actual Run Rate 3484 6056
Q107Q207 2500
Q307 4500
Q407 8000
Q108 8400
Q208 8400
Q308 8000
Q4+ Q1-09 8000
Q207 “Actual Run Rate” as of June 30, 2007
Quantity
25
IT Infrastructure Partnership Program (ITP)
Desktop Weekly Run Rate
Desktop Refresh Weekly Progress
0
500
1000
1500
2000
2500
3000
Week of
PC
Co
un
ts
Planned
Actual
26
IT Infrastructure Partnership Program (ITP)
Helpdesk Peregrine Tool Rollout• When
– Q3 rollout targeting 12,942+ additional supported users
– Q3 rollout targeting 25 additional agencies
• Benefit– Standardized environment for problem/change management
– Allows multiple support groups to cover multiple agency/systems from a single management tool
– Allows for centralized metrics reporting / trend analysis
– Centralized SLA management
• Impact– No impact to end users
– Agency participation for retained application support encouraged
– Operations staff availability for training
– Documentation of environment and critical level for systems to be defined
27
IT Infrastructure Partnership Program (ITP)
Peregrine Rollout(Incident Management Tool)
0
10000
20000
30000
40000
50000
60000
70000
En
d U
sers
Su
pp
ort
ed
0
5
10
15
20
25
Ag
en
cie
s i
n t
hat
Qu
art
er
Forecast Agencies 6 21 15 11 6 4
Actual Agencies 0 9
Actual Desktops 628 4138
Forecast Desktops 628 5143 10649 28711 32334 57736
Q1 CY07
Q2 CY07
Q3 CY07
Q4 CY07
Q1 CY08
Q2 CY08
Helpdesk – Peregrine Tool Coverage
Dept. of Mental Health (pilot site) VITA Dept. of Veterans Services Virginia Museum of Natural History Dept. of Minority Business
Enterprise Dept. of Historic Resources Virginia Museum of Fine Arts
Dept. of Mental Health Office of the Governor Dept. of Employment Dispute
Resolution Dept. of General Services Dept. of Business Assistance Virginia Department of Health
Agencies Complete
Larger Agencies in Progress
28
IT Infrastructure Partnership Program (ITP)
Messaging Client Rollout• When
– Q3-Q4 Pilot for 5,029 users at Virginia Department of Health and Virginia Department for the Aging
• Benefit– Standardized environment for e-mail systems
– Redundant architecture with complete failover capability
– Disaster recovery planned for SWESC to support e-mail
– Centralized e-mail environment for finding addresses, rooms, etc.
• Impact– End users will get a new mail client installed
– Agency participation for communicating mail policies
– Service delivery staff participation
– Agency documentation of application environment that uses e-mail systems
29
IT Infrastructure Partnership Program (ITP)
Messaging Quarterly Run RateExchange End User Migration Approach
0
10000
20000
30000
40000
50000
60000
70000
En
d U
sers
Mig
rate
d
Planned 5029 5029 5401 7562 23097 34855 47213 57213
Q3 CY07 Q4 CY07 Q1 CY08 Q2 CY08 Q3 CY08 Q4 CY08 Q1 CY09 Q2 CY09
Q3 Pilots
Q4 No ActivityMessaging follows
Network Refresh
Q108 Starts M
essaging Full R
ollout
30
IT Infrastructure Partnership Program (ITP)
Network Refresh Rollout• When
– Began Q3 with pilot at Department of Juvenile Justice (DJJ)
– Planned Virginia Department of Transportation and Alcohol Beverage Control headquarters
– Alcohol Beverage Control stores (230+) begin refresh process
• Benefit– Standardized equipment for reliability and management
– Redundant architecture with complete failover capability
– Connectivity to new network allows logical extension of agency connection to new data centers
– Higher security, monitoring, speed and allows rapid expansion
• Impact– End users get new IP addresses automatically
– Agency participation for ensuring application readiness
– Service delivery staff participation
– Agency documentation of application environment that uses hard-coded IP addresses
31
IT Infrastructure Partnership Program (ITP)
Server Consolidation Rollout• When
– Began Q3 server consolidation site surveys
– Working with VDOT to assist in virtualizations
• Benefit
– Standardized equipment for reliability and management
– Redundant architecture with complete failover capability
– Expand the use of tape robotics to improve data access performance
– Upgrade to current disk technology will enable data replication of virtual tape and disk
• Impact
– Agency participation for ensuring application readiness
– Service delivery staff participation
– Agency documentation of application environment that uses hard-coded IP addresses
32
IT Infrastructure Partnership Program (ITP)
Server Consolidation Metrics
– Scheduling the following RPB servers for virtualization• 37 Federally-funded servers• 34 Agency released servers• 12 VITA servers
– Working with VDOT to consolidate 70 servers – Planning to consolidate and move 10 Dept. of Environmental Quality servers to RPB – Developing weekly view for server consolidation
Server Consolidated
0
500
1000
1500
2000
2500
3000
Se
rve
r c
ou
nt
Planned Target 0 250 500 750 1000 1250 1500 1750 2000 2250 2500
Examined Forecast 0 250 600 750 1000 1250 1500 1750 2000 2250 2500
Servers Consolidated 0 49 400
Jan-07 Mar-07 Jun-07 Sep-07 Dec-07 Mar-08 Jun-08 Sep-08 Dec-08 Mar-09 Jun-09
33
IT Infrastructure Partnership Program (ITP)
Data Center Move
34
IT Infrastructure Partnership Program (ITP)Data Center Move
June June
7/1/07 8/1/07 9/1/07 10/1/07 11/1/07 12/1/07 1/1/08 2/1/08 3/1/08 4/1/08 5/1/08 6/1/08
7/15/07Mainframe
Install CESC
6/1/07
MPLS Cnctd CESC
Network/SecurityMainframe/ServerFacility/Move
7/13/07NG Move
into CESC
7/24/07VITA Move into CESC
11/11/07
Enterprise NOC/TSOC Oper CESC
3/11/08Agency Server
Consldtn Comp
11/12/07Mainframe/
Server Prod Cutover
CESC
9/1/07
MPLS Core Active CESC
& SWESC
10/30/07Server Install CESC
12/31/07Mainframes/Servers out
of RPB
1/29/08Servers Install
SWESC
1/4/08Mainframes
Install SWESC
4/17/08DR Test Comp
11/1/07
Help Desk Oper
Cutover11/16/07
ESOC Oper
SWESC
11/1/07SWESC Ready Occup
• Moving the data center from RPB to new, secure CESC facility is complex
– Significant coordination with agencies, minimizing business disruptions, technical complexity
• Robust, detailed data center move plan in place
– Move strategy, test objectives, risk mitigation, communication plan with agencies
35
IT Infrastructure Partnership Program (ITP)
Data Center Move Strategy• Mainframe Move Strategy
– Replicated mainframe and related hardware required for full application testing
– Coordinate all mainframe users (15 agencies)
– Provide agencies isolated external connectivity to CESC test environment
– Test environment available for 60-90 days
– Multiple mock cutover test prior to final go-live
• Server Move Strategy
– Virtualize/consolidate servers at RPB
– Consolidate multiple SAN/DISK storage systems at RPB
– Replicate RPB internal network (LAN) at CESC
– A two-phased cutover approach will be utilized
• Phase 1 is movement of servers onto an extended duplicate RPB network at CESC
• Phase 2 will require that servers be moved from duplicate RPB network to “New” network at CESC
– Servers will be moved in 7 logical groups, based primarily on agency usage
36
IT Infrastructure Partnership Program (ITP)
Data Center Move Test Objectives• Operations Team objectives and control
Test
– All systems will IPL/boot and communicate with peripherals
– Administrative functions (monitoring and management) operate as expected
– Data replication between CESC and RPB functions properly
– Internal CESC network (LAN) and firewalls function properly
– Print infrastructure functions properly
– Tape backup infrastructure functions properly
– Control-M infrastructure functions properly for support of batch operations
– Point-to-point connections function properly
• Retained Application support Team objectives
– Applications will initiate and connect with database(s)
– Applications will update data and print reports as expected
– Regression test of all applications components on the mainframe systems
37
IT Infrastructure Partnership Program (ITP)
Data Center Move Risk MitigationTop 5 risks identified during the server and mainframe move from RPB and CESC.
• Standup of an Isolated Test Environment
– Replicate mainframe hardware and software infrastructure
– Replicate servers running tier 2 applications that interface with mainframes
– Replicate DASD and tape storage infrastructure and data via high speed data links
• Extended Test Period
– Provide agencies with at least 60 days to complete application testing
– Mock move weekends scheduled to accommodate thorough integration testing of complex, interdependent applications
• Test Coordination / Command Center Support
– Provide rapid response team to quickly address problems that surface during testing
– Testing coordination teams will be assigned to each key mainframe using agency
– Weekly reporting of testing progress by agency and application to be shared with agency managers
• Stand-by Hardware
– Mission critical application hardware can be made available if hardware problems arise due to move
• Fallback Contingency
– RPB processing infrastructure will remain intact for at least 2-3 weeks following the move to provide fall-back capability
38
IT Infrastructure Partnership Program (ITP)
Mainframe and Server Communication Plan• Communications is key to the successes of the RPB Move to CESC.
• Detailed weekly planning meetings with all Service Delivery Teams – started May 28
• Overview and kick-off meeting with each affected agency – started June 1
– Review move plans and get feedback on proposed move dates and process
– Agencies need to assign POC for application testing and coordination
• Detailed planning meetings with agency application teams to develop test scenarios
• Detailed weekly status reviews with all agency/VITA test teams throughout test window
• Monthly status/update meetings with agency management directly affected by the RPB move
• Dedicated test coordinators from the Transformation Team, Current Ops Team, and the agency
• 24x7 command center setup before, during, and post move/cutover
– Multiple locations linked by phone and/or video conferencing (agency, RPB, CESC)
– Participation by agency application staff, Service Delivery, VITA, Transformation, and vendors (as needed)
– Representation by network, security, mainframe, server, applications, etc
39
IT Infrastructure Partnership Program (ITP)
ITP Financials
40
IT Infrastructure Partnership Program (ITP)
$0.0
$50.0
$100.0
$150.0
$200.0
$250.0
Funded Obligation $36.1 $51.0 $65.0 $81.6 $108.4 $122.4 $136.4 $152.7 $172.4 $196.0 $210.5 $230.0
ACTL+FCST $31.8 $48.6 $63.3 $80.6 $106.9 $122.3 $139.2 $156.8 $171.4 $191.1 $211.9 $230.0
Jul '06 Aug Sep Oct Nov Dec Jan '07 Feb Mar Apr May Jun
$M
Actuals
Forecast
Partnership Budget and Forecast – Year 1
$230.0M Forecast Total:
• $81.8M Current Ops
• $1.4M Facilities
• $57.5M Milestones
• $89.3M Managed Exp
Cumulative expenses through April continue to track closely to the budget; this is largely due to the delay of several milestones
$57.5M for milestones based on Northrop Grumman’s proposed schedule, which includes performance bond, relaxed refresh and service level agreement (help desk & messaging) adjustments
41
IT Infrastructure Partnership Program (ITP)
Independent Program Maturity Assessment
42
IT Infrastructure Partnership Program (ITP)
Independent Program Maturity Assessment
• To date CACI has completed 3 of 4 currently scheduled assessments
– July and November 2006 with net 192 positive findings and 115 minor negative findings – no major negative findings relative to program best practices
– May 2007 with net 70 consolidated minor negative findings and 45 minor positive findings – no major negative findings relative to program best practices
– Remaining assessment scheduled for November.
– Slower than expected rate by which the program governance and stakeholder management processes should have matured.
• ITP Program Governance Council next steps
– Focus on scope, timing and targets relative to best practices for program management maturation
– Balance demands of tactical program performance, management and best practices
– Determine long-term focus for program assessment
43
IT Infrastructure Partnership Program (ITP)
IT Infrastructure Partnership Program
Fred Duball and Joe Fay
Program Directors
July 18, 2007ITIB IT Infrastructure Committee