document: the complete report, pdf, 68 pages, 7...

(b)(6)

(b)(6)

(b)(6)

(b)(6)

(b)(6)

(b)(6)

(b)(6)

(b)(6)

(b)(6)

(b)(6)

(b)(6)

(b)(6)

Subj: COMMAND INVESTIGATION OF DIESEL ENGINE AND RELATED MAINTENANCE AND QUALITY ASSURANCE ISSUES ABOARD USS SAN ANTONIO (LPD 17)

2

(31) Summary of Interview (32) NR 1B MPDE Photos (33) Work Item 233-11-001, Reservation Task Request 33 (34) SAN ANTONIO Engineering Log of Nov 09 (35) NR 2A MPDE Summary of Inspection (36) NR 2A MPDE Photos (37) MARMC Lab Rpt #2009NN03572 of 4 Dec 09 (38) MARMC Lab Rpt #2010NN00038 of 7 Jan 10 (39) MARMC Lab Rpt #2009NN03466 of 23 Nov 09 (40) MPDE Lube Oil Wetted Parts List (41) LO Service System Description and Diagram (42) E-mail of 10 Nov 09 (43) LO Piping Material Specification (44) LO Pump Parts List (45) LO System Valve Assembly (46) LO Service System 6 Inch Valve Diagram (47) LO Service System H Valve Diagram (48) S9233-DL-010 (49) MRC 2331 W-1R (50) Investigator Notes (CoS)(3M Program Review) (51) Maintenance Bucket Pictures (52) SAN ANTONIO LO Log Aug 09 (53) SAN ANTONIO Engineering Log of Sep 09 (54) SAN ANTONIO LO Log Sep 09 (55) SAN ANTONIO LO Log Oct 09 (56) SAN ANTONIO LO Log Nov 09 (57) memo 5830 of 14 Jan 10 (Muslin Bag) (58) Muslin Bag LO Flush Photos (59) LO Flush Manual (60) NSTM 233 (61) CLARUS Reports (62) Summary of Interview (63) Fairbanks Morse Condition Reports (64) memo 5830 of 14 Jan 10 (Bearing Inspection) (65) Bearing Damage Photos (66) E-mail of 15 Jan 10 (67) JFMM Volume IV Chapter 4 (68) memo 5830 of 17 Dec 09 (MPDE Material History Logs) (69) EM01 Quarterly Schedule (70) MRC Card 2640 M-2 (71) MRC 2331 18M-12R (72) EM02 Quarterly PMS Schedule (73) MRC 2331 R-12W

FOR OFFICIAL USE ONLY

(b)(6)

(b)(6)

(b)(6)

(b)(6)

(b)(6)

(b)(6)

(b)(6)


3

(74) memo 5830 of 17 Dec 09 (3M Program) (75) MRC 2331 R-13W (76) MRC 2331 R-14M (77) MRC 2331 U-35 (78) MRC 2620 U-1 (79) MRC 2620 U-3 (80) memo 5830 of 15 Jan 10 (LOQM) (81) SAN ANTONIO SAMM Oil Sample DB 6 Dec 09 (82) EOCC Procedures (83) memo 5830 of 15 Jan 10 (Eng Dept Trng) (84) Investigator Notes (CoS) (Training Program Review) (85) E-mail of 22 Dec 09 (86) E-mail of 29 Dec 09 (87) Summary of Interview (88) Summary of Interview (89) Summary of Interview (90) ECS PP Brief (91) E-mail 13 Oct 06 (92) PRESINSURV NORFOLK VA 132000Z Apr 07 (93) PRESINSURV NORFOLK VA 061500Z Dec 07 (94) CASREP 07075 08099 Summary (95) e-mail of 16 Nov 07 (96) CRO Predeployment PowerPoint of Aug 08 (97) LPD 17 Deployment Status Report 27 Nov 08 to 12 Dec 08 (98) ail of 15 Dec 08 (99) E-mail of 12 Oct 06 (100) E-mail of 22 Oct 06 (101) Post FOA Support PowerPoint (102) e-mail 5 Sep 07 (103) SAN ANTONIO TSO List (104) E-mail of 5 Jan 10 (105) E-mail 8 Jan 10 (106) NAVSEA Philadelphia Code 911 LPD 17 Class ICAS Status Update (107) ICAS PP Brief (108) memo 5830 of 15 Jan 10 (109) LPD 17 Class NTSP 1 Jul 09 (Final-Approved) (110) Letter of Promulgation for the Management Responsibilities (111) LPD Training POM 12 PP Brief (112) PSMD LPD 17 6 Jan 04 (113) PSMD LPD 17 Change One 25 Feb 05 (114) PSMD LPD 17 Change Two 9 Apr 07 (115) E-mail of 11 Jan 10


(b)(6)

(b)(6)

(b)(6)

(b)(6)

(b)(6)

(b)(6)

(b)(6)

(b)(6)

(b)(6)

(b)(6)

(b)(6)(b)(6)

(b)(6)

(b)(6)

(b)(6)

(b)(6)

(b)(6)

(b)(6)


4

(116) PEO Washington DC 301804Z Mar 09 LPD 17 Class FY09 Interim Training Plan for Main Propulsion (117) PEO Washington DC 171810Z Oct 09 LPD 17 Class FY10 Interim Training Plan for Main Propulsion (118) LPD 17 Class MPDE-SSDG-ECS Interim Training PP (119) E-mail of 18 Dec 09 (120) EOOW PQS NAVEDTRA 43104 (121) Engineering Collaterals and Qualifications PQS NAVEDTRA 43704 (122) CO, USS SAN ANTONIO (LPD 17) ltr 5100 Ser 00 of 26 Mar 08 (123) CO, USS SAN ANTONIO (LPD 17) ltr 5100 Ser 00 of 3 Jun 09 (124) CO, USS SAN ANTONIO (LPD 17) ltr 5100 Ser 00 of 1 Sep 09

PRELIMINARY STATEMENT

1. Pursuant to enclosure (1), and in accordance with reference (a), I conducted a command investigation into the facts and circumstances surrounding the discovery of metal shavings on the oil side of three of four of USS SAN ANTONIO (LPD 17) Main Propulsion Diesel Engines (MPDE) and a review of the maintenance performance and quality control efforts by shipyard facilities, intermediate maintenance activities, and SAN ANTONIO. Enclosure (2), a list of relevant acronyms and abbreviations, is included to assist in a review of the investigation. 2. Executive Summary a. Metal Shavings (1) In October 2008, MARMC/NNSY voyage repair team conducted significant repairs to the MPDE lube oil service system piping onboard SAN ANTONIO due to failure of several welded joints. The repair included a partial use of carbon steel piping. Starting June 2009 during CNO PMA, NNSY conducted permanent repairs of the piping that included the replacement of approximately 80% of the existing external lube oil service piping with new 300 series stainless steel piping. After the completion of the piping repairs, CLARUS using the NAVSEA procedure conducted a flush of the piping. However, as designed the flushing procedure was inadequate to completely remove contaminants from the lube oil service system.


(b)(6)


5

(2) In November and December 2009, NSSA Diesel Inspectors and SAN ANTONIO personnel discovered foreign matter in the lube oil service system for NR 1A, 1B, and 2A MPDE. The foreign matter included 300 series stainless steel. The probable source of the 300 series stainless steel found in NR 1A, 1B, and 2A MPDE lube oil service systems was the byproducts of the production and installation of stainless steel lube oil service system piping. The cause of the stainless steel was potentially the result of poor weld joint fit up during new construction that created contaminate traps that could allow for an intermittent release of contaminates. (3) In November 2009, NSSA determined that depot level maintenance was required to address excessive crankshaft main bearing and thrust bearing wear in 1A MPDE, 1B MPDE, and 2A MPDE. This bearing damage is the primary component of MPDE corrective maintenance since completion of the PMA in October 2009. This damage is not related to the metal shavings found in the MPDE lube oil service systems. Bearing failure root cause is lube oil contamination that occurred during the Shipbuilding process to include all repairs and maintenance prior to the PMA. Total estimated cost of this additional MPDE corrective maintenance is $7.5M. The operational impact is such that repairs may preclude SAN ANTONIO executing her next scheduled deployment. (4) An alternate means of contamination causing bearing damage based on technical evaluation of bearing failures on LPD 17 and LPD 21 has been proposed by NAVSEA during a recent video teleconference with Fleet Forces Command, Second Fleet, Commander Naval Surfaces Forces Atlantic and Expeditionary Strike Group Two on 8 January. This hypothesis suggests that the lube oil contaminates that are circulated back to the MPDE sump via the self-cleaning pre-filter are continuously reduced in physical size by the macerating action of the lube oil service pumps until the contaminates are small enough to pass through the filter and duplex strainers. Contaminates that pass through the filter and strainer complex could still be large enough to cause bearing damage. NAVSEA is considering an emergent alteration to eliminate this mechanism of contaminate introduction into the diesel complex by filtering out contamination discharged by the filter back to the sump. (5) Ship’s Force was slow to recognize lube oil contamination for a variety of long term issues. Command



6

leadership failed to effectively execute basic engineering program, specifically the Lube Oil Quality Management Program, which was determined to be ineffective. c. LPD 17 Engineering Systems (1) SAN ANTONIO was designed to be manned and operated presuming that all automated control and monitoring systems were properly operating. (2) With ECS, ICAS, and SAMMS not functioning as designed, the ship was unable to effectively operate and maintain the engineering plant. 2. Evidence a. The Investigating Officer collected all reasonably available evidence in order to complete this investigation. b. All documentary evidence included herein, is hereby certified to be a true copy that is a true and accurate representation of the original document. c. During the course of this investigation, the Investigating Officer and participating members performed the following: (1) An audit of all Norfolk Naval Shipyard (NNSY) production work on the MPDEs, (2) An audit of contracted depot level work completed during the CNO PMA by a local Master Ship Repair (MSR) company and associated subcontractors to determine if pre or post flush production work could have contaminated the MPDE lube oil systems, (3) Interviews with appropriate Norfolk Ship Support Activity (NSSA) PMA project team personnel and on scene Diesel Inspectors, (4) A technical evaluation of the adequacy and execution of the MPDE flush procedure, (5) An external and internal evaluation of ship's force maintenance, operational procedures and training. This evaluation focused on the ability of ship's force to operate, monitor and maintain MPDE lube oil and associated systems. This was done in part to determine the likelihood that ship's force introduced contamination or was slow to recognize if lube oil quality was jeopardized,



7

(6) An external review to validate if Sailor/Officer training and NAVSEA conditioned based maintenance reach back systems were in place to support the lead ship of a new class that had a crew that was "right sized" with these external assumptions in place, (7) A review of the automated systems designed to support shipboard operations including the Engineering Control System (ECS), Integrated Condition Assessment System (ICAS) and Total Ship Training System (TSTS) was conducted to evaluate the effectiveness of these systems to support ship's force's ability to operate the propulsion plant per the current ship's manning documents, (8) An audit by using the appropriate Technical Work Documents (TWD) and associated Objective Quality Evidence (OQE) against Joint Fleet Maintenance Manual (JFMM), NAVSEA Standard Items, Specification Items, and other appropriate technical guidance to assess Depot and MSR corrective maintenance performance, (9) A review of ship's force corrective and preventive maintenance by examining preventive maintenance scheduling and execution, (10) Oral interviews with the chain of command, (11) A review of NAVSEA technical documentation describing how SAN ANTONIO class ships are to manned, trained and provided external support was reviewed to determine completeness and effectiveness of execution, (12) A briefing from NAVSEA representatives on the Engineering Control System (ECS) and Integrated Condition Assessment System (ICAS), and (13) A review of the syllabus for training provided by Surface Warfare Officer School (SWOS) to prospective SAN ANTONIO class Commanding Officers, Executive Officers and Chief Engineers. 4. The Investigating Officer has the following prior engineering/maintenance experience: USS HUNLEY (AS 31) Planning and Estimating Officer (12 months), Repair Duty Officer (12 months), and Auxiliaries Officer (11 months); USS LEXINGTON (AVT 16) Main Engines Officer (19 months); Steam Engineering Officer of the Watch (EOOW) Instructor and Course Coordinator (30



8

months); and USS MOUNT HOOD (AE 32), Chief Engineer (CHENG) (20 months). 5. The following personnel assisted in the conduct of the investigation (prior engineering/maintenance experience listed): a. Chief of Staff, Expeditionary Strike Group TWO: BS Mechanical Engineering, University of Wisconsin 1983; MA Systems Engineering, Naval Postgraduate School 1991; Nuclear Power School 1984; Nuclear Power Training Unit, Idaho Falls (A1W Prototype) 1985; USS LONG BEACH (CGN 9), Number Two Engine Room Division Officer (36 months); USS ARKANSAS (CGN 41), Main Propulsion Assistant (MPA) (16 months); USS TRUXTUN (CGN 35), CHENG (22 months); Atlantic Fleet Nuclear Propulsion Examination Team (NPEB), Junior Team Member (28 months); and USS EISENHOWER (CVN 69) Reactor Officer (28 months). b. N4, Expeditionary Strike Group TWO: USS SCOTT (DDG 995), MPA (30 months); USS CHINOOK (PC 9), CHENG (36 months); Mk 5 Special Operations Craft, Officer in Charge/Assistant Officer in Charge (38 months); USS KEARSARGE (LHD 3), MPA (24 months); USS WASP (LHD 1), CHENG (18 months); and Auxiliary Floating Dry Dock Dynamic, Commanding Officer (23 months). c. N43, Naval Surface Force Atlantic: USS NASSAU (LHA 4), Assault Officer (24 months), Auxiliaries Officer (18 months), and Repair Officer (six months); Afloat Training Group, Atlantic, Training Liaison Officer (24 months); and USS OAK HILL (LSD 51), CHENG (13 months). d. Engineering Support, Expeditionary Strike Group TWO: OAK HILL, Electrical Officer (28 months). e. Engineering Support, Expeditionary Strike Group TWO: NASSAU, Electrical Officer (17 months). f. Diesel Engine Inspector (Code 216.5) Diesel Engine Inspector, Norfolk Naval Shipyard: USS MESA VERDE (LPD 19), Top Snipe (24 months); USS SAN ANTONIO, Pre-Commissioning Unit (PCU) for MESA VERDE (six months); USS


(b)(6)

(b)(6)

(b)(6)

(b)(6)

(b)(6)

(b)(6)


9

NEW ORLEANS, PCU for MESA VERDE (six months); USS CARTER HALL (LSD 50), PCU for MESA VERDE; OAK HILL, M-Division Leading Chief Petty Officer (eight months); and HSV-X-1, Lead Engineman (12 months). g. Maintenance and Material Management System Chief Petty Officer, Amphibious Squadron SIX: USS REID (FFG-30) (36 months); USS MCKEE (AS-41) (24 months); and USS CORMORANT (MHC 57), EA01 Leading Petty Officer (48 months). h. , Norfolk Ship Support Activity (NSSA): Merchant Marine Officer (9yrs)– U.S Coast Guard licensed Engineering Officer on medium speed diesel powered ships; U.S. Navy Port Engineer (12 years) – DDG Class/CG Class Class maintenance management; NSSA LSD/LPD Class Team Leader (three years) – Managed RMC Waterfront Operations Department LSD/LPD 4/LPD 17 Class Maintenance Teams; NSSA CVN/Amphibious Ship Division Manager (one year) - Principal Waterfront Operations Department advisor in matters relating to CVN and amphibious ship maintenance. i. , Code 212.5, Nondestructive Testing Level III, Quality Assurance Supervisor, NSSA: Retired HTC; engineering plant operator/watch stander (MMOW) (four years); ship/submarine maintenance, repair and quality assurance (22 years); and contractor oversight (seven years).

j. Quality Assurance Specialist with Code 212.5, Norfolk Naval Shipyard Engineering and Technical Fleet Support Department: Retired HTC; NAVSEA certified NDT Level II Inspector (22 years); and NAVSEA certified NDT Level III Examiner (eight years). k. Project Team Leader, Norfolk Naval Shipyard (NNSY), USS BATAAN (LHD 5): prior experience as Supervisory Assistant Project Superintendent: SAN ANTONIO, WASP, USS NASSAU, and USS GEORGE WASHINGTON (CVN 73); NNSY Production Department (30 years). 6. Expeditionary Strike Group TWO, and Expeditionary Strike Group TWO provided administrative support in the course of this investigation.


(b)(6)

(b)(6)

(b)(6)

(b)(6)

(b)(6)

(b)(6)

(b)(6)


10

7. provided legal guidance in the course of this investigation.

FINDINGS OF FACT

Metal Shavings Discovery and Causation Introduction 1. During a deployment to Fifth Fleet Area of Operations between 9 October and 25 October 2008, SAN ANTONIO suffered repetitive weld joint failures in the MPDE lube oil service system piping. [Encls. 3, 4, 5)] 2. The weld joint failures were a result of poor weld techniques. [Encls. (5, 6)] 3. The poor weld techniques during new construction caused excessive oxidation and stainless steel globules inside the lube oil service system piping. [Encl. (6)] 4. The poor weld joint fit up during new construction created traps. [Encls. (5, 6)] 5. From 2 to 23 November 2008, MARMC/NNSY voyage repair team conducted repairs on the lube oil piping inport Bahrain. [Encls. (3, 4)] 6. The repair included piping replacement using 304 stainless steel and carbon steel piping. [Encls. (3, 4)] 7. The repairs did not include a flush of the lube oil system. [Encls. (3, 4)] 8. Upon completion of repairs, the lube oil service system serving each engine was operated in normal configuration. [Encl. (7)] 9. Ships force continued to monitor lube oil duplex strainer content until the baskets were clear of foreign material and particulate. [Encl. (7)] 10. On 27 March 2009, SAN ANTONIO returned from the deployment. [Encl. (8)]


(b)(6)


11

11. On 17 June 2009, SAN ANTONIO began a Chief of Naval Operations Planned Maintenance Availability (CNO PMA). [Encl. (8)] 12. During the CNO PMA, NNSY replaced approximately 80% of the existing external lube oil service system piping serving all four MPDEs with new 300 series stainless steel piping. [Encls. (9-15)] 13. After completion of the lube oil service system piping repairs, a flush of the lube oil system occurred. [Encl. (9)] 14. During the CNO PMA, contract personnel conducted the following MPDE repairs: NR 2A1 lube oil service pump, NR 1A and NR 1B MPDEs valve cover software replacement, NR 1B MPDE crankcase fan oil seal, and NR 2A MPDE low pressure air line line. [Encl. (9)] 15. On 2 October 2009, SAN ANTONIO completed a light off assessment (LOA). [Encl. (8)] 16. On 7 October 2009, SAN ANTONIO conducted sea trials. [Encl. (8)] NR 1A MPDE Foreign Matter Discovery Timeline 17. On 29 July 2009, work started to replace NR 1A MPDE cylinder head (16 total) valve cover gaskets and software crankcase in accordance with NSTM S9233-DL-MMM-020 (Chapter 7) and NSTM S9233-DL-MMM-010. [Encl. (16)] 18. On 19 August 2009, CHENG, closed out NR 1A MPDE lube oil sump. [Encl. (17)] 19. On 11 September 2009, work completed to replace NR 1A MPDE cylinder head (16 total) valve cover gaskets and software crankcase in accordance with NSTM S9233-DL-MMM-020 (Chapter 7) and NSTM S9233-DL-MMM-010. [Encl. (16)] 20. If normal cleanliness standards are adhered to during the execution of the work, a comprehensive lube oil service system flush is not required. [Encl. (18, 19)] 21. Normal cleanliness standards were met prior to commencement of production work. [Encl. (20)]


(b)(6)


12

22. On 7 October 2009 at 1903 while operating, NR 1A MPDE suffered an engineering casualty: unusual noise and vibration on NR 9 cylinder. [Encl. (21)] 23. On 7 October 2009 at 1955, a lube oil casualty sample was taken from NR 1A MPDE. [Encl. (21)] 24. On 14 October 2009, CASREP submitted on NR 1A MPDE regarding loud metallic noise. [Encl. (22)] 25. The CASREP indicated no unusual conditions were noted on NR 9 cylinder. [Encl. (22)] 26. On 21 October 2009, CASCOR submitted on NR 1A MPDE regarding loud metallic noise. [Encl. (22)] 27. On 22 November 2009, Naval Surface Warfare Center Carderock Division-Ship Systems Engineering Station Philadelphia (NSWCCD-SSES Philly) reported abnormal wear levels in NR 1A MPDE based on a lube oil sample submitted on 7 October 2009. [Encl. (23)] 28. Due to the abnormal wear levels, NSWCCD-SSES Philly recommended an inspection of NR 1A MPDE. [Encl. (23)] 29. On 25 November 2009, the Commanding Officer, SAN ANTONIO, requested Port Engineer, and , CHENG, consult with NSWCCD-SSES Philly, for MPDE inspection recommendations. [Encl. (23)] 30. On 25 November 2009, requested

and Norfolk Naval Shipyard Diesel Inspectors, provide inspection recommendations. [Encl. (23)] 31. On 25 November 2009, conducted the inspection of the lube oil service system and an inspection of NR 1A MPDE. [Encls. (23-25)] 32. During the inspection of NR 1A MPDE, the engine remained static and was not barred over. [Encl. (25)] 33. The inspection of NR 1A MPDE included a visual inspection of cam bearing and journals, and rod bearing and journals, and main bearing and journals. [Encl. (25)]


(b)(6) (b)(6)

(b)(6)

(b)(6) (b)(6)

(b)(6) (b)(6)

(b)(6)


13

34. The online strainer exhibited no particulates. [Encls. (25, 26)] 35. The off-line strainer exhibited ferrous particulate (fine) on the magnet that if accumulated would be less than 1% of the magnet cylinder surface. [Encls. (25, 26)] 36. The back-flushing filter exhibited particulate: metal shavings, paint chips, gasket material, fine debris located in remnants of fibrous material in 20 candles of 66. [Encls. (25, 26)] 37. The cam covers accesses exhibited no debris or particulates. [Encls. (25, 26)] 38. The dry sump exhibited no solid material. [Encl. (25)] 39. In the dry sump, there was free water standing in the oil. [Encl. (25)] 40. recommended the wet sump be pumped out and certified gas free for further inspection. [Encl. (25)] 41. On 1 December 2009, wet sump cleaned and gas freed. [Encls. (25)] 42. During the initial visual inspection of the wet sump, the wet sump exhibited a higher than expected amount of babbitt material. [Encl. (25)] 43. In the wet sump, there was free water standing in the residual oil. [Encl. (25)] 44. On 15 December 2009, on-line and off-line Rocker Duplex Strainers removed by Ships Force and inspected by DEI

. [Encl. (24)] 45. On-line and Off-line Rocker Strainers exhibited free water, dirt, and paint chips (white, terracotta, and yellow). [Encl. (24)] 46. Rocker Sump opened by Ships Force and inspected by DEI

[Encl. (24)]


(b)(6)

(b)(6)

(b)(6)

(b)(6)


14

47. Rocker Sump exhibited free water, dirt, and paint chips. [Encl. (24)] NR 1B MPDE Foreign Matter Discovery Timeline 48. On 21 July 2009, work tasked to repair NR 1B MPDE crankcase fan oil seal. This repair had an estimated start date of 27 July 2009 and was signed complete on 25 August 2009. [Encl. (27)] 49. On 27 July 2009, work started to repair NR 1B MPDE crankcase fan oil seal. [Encl. (27)] 50. On 29 July 2009, work started to replace NR 1B MPDE cylinder head (16 total) valve cover gaskets and software crankcase in accordance with NSTM S9233-DL-MMM-020 (Chapter 7) and NSTM S9233-DL-MMM-010. [Encl. (28)] 51. On 25 August 2009, work completed on NR 1B MPDE crankcase fan oil seal. [Encl. (27)] 52. On 26 August 2009 at 1140, Chief Engineer, closed out NR 1B MPDE lube oil sump. [Encl. (17)] 53. On 11 September 2009, work completed to replace NR 1B MPDE cylinder head (16 total) valve cover gaskets and software crankcase in accordance with NSTM S9233-DL-MMM-020 (Chapter 7) and NSTM S9233-DL-MMM-010. [Encl. (28)] 54. If normal cleanliness standards are adhered to during the execution of the work, a comprehensive lube oil service system flush is not required. [Encl. (18, 19)] 55. Work would not have commenced prior to cleanliness standards being met. [Encls. (29-31)] 56. On 7 October 2009 at 1010 while operating NR 1B MPDE, NR 1B MPDE suffered an engineering casualty: hot bearing on NR5 main bearing (172 degrees F). [Encl. (21)] 57. The hot bearing was a 21 degree spread from the next highest bearing temperature. [Encl. (21)] 58. On 7 October 2009 at 1010, NR 5 main bearing (172 degrees F) was 21 degrees F higher than the next highest bearing temperature. [Encl. (21)]


(b)(6)


15

59. On 1 December 2009, filter removed for NR 1B MPDE due to issues with NR 1A and 2A MPDE. [Encl. (24)] 60. Lube oil results for NR 1B had been satisfactory (did not indicate need to conduct inspection). [Encl. (24)] 61. Filter element removed. Set on work bench. Amount of babbitt material on filter and filter candles. Had fibrous material as well. No further inspection. [Encl. (24)] 62. On 15 December 2009, on-line and off-line strainers were removed by ships force and inspected by DEI [Encl. (24)] 63. On-line strainer exhibited fine ferrous particulate on the magnet, accumulated debris did not exceed the circumference of a dime. [Encls. (24, 32)] 64. Off-line strainer exhibited fine ferrous particulate on the magnet, accumulated debris did not exceed the circumference of a dime. [Encls. (24, 32)] 65. On 15 December 2009, on-line and off-line Rocker Duplex Strainers removed by Ships Force and inspected by DEI

[Encl. (24)] 66. On-line and Off-line Rocker Strainers exhibited dirt, gasket material, paint chips and free water. [Encls. (24, 32)] 67. Rocker Sump opened by Ships Force and inspected by DEI

[Encl. (24)] 68. Rocker Sump exhibited free water, dirt, and paint chips. Sump bottom was coated with a large amount of fine bronze material and non-ferrous matter similar to babbitt. [Encls. (24, 32)] 69. On 16 December 2009, 1B MPDE lube oil sump pumped, 600 gallons of lube oil remained at the pump end of the sump. [Encl. (24)] 70. On 17 December 2009, 1B MPDE lube oil sump opened for inspection. [Encl. (24)]


(b)(6)

(b)(6)

(b)(6)

(b)(6)


16

71. At the sump cover opening ferrous particulate, paint chips, dirt and carbon were immediately visualized before entering the sump. [Encls. (24, 32)] 72. The DEI hand skimmed the oil layer at the sump cover opening; this revealed fine particulate (bronze in color), non ferrous particulate resembling babbit. [Encls. (24, 32)] NR 2A MPDE Foreign Matter Discovery Timeline 73. On 22 July 2009, work tasked to replace NR 2A MPDE low pressure air line serving the engine barring gear (turning gear). [Encl. (33)] 74. On 25 July 2009, work started and completed to replace NR 2A MPDE low pressure air lube oil line serving the engine barring gear (turning gear). [Encl. (33)] 75. The low pressure air line consisted of 3/8 inch stainless steel tube. [Encl. (33)] 76. On 13 August 2009, , Chief Engineer, closed out NR 2A MPDE lube oil sump. [Encl. (17)] 77. On 14 August 2009 at 1815, NR 2A MPDE lube oil strainer check point complete by Engineering Duty Officer in preparation for engine flush. [Encl. (17)] 78. On 6 October 2009 at 0750, NR 2A MPDE emergency stopped due to NR 8 hot bearing. [Encl. (21)] 79. On 6 October 2009 at 0822, casualty sample taken for NR 2A MPDE. [Encl. (21)] 80. On 6 October 2009 at 1210 NR 2A MPDE lube oil strainer opened and inspected SAT. [Encl. (21)] 81. On 6 October 2009 at 1250, NR 2A MPDE lube oil service system duplex strainer tested satisfactory. [Encl. (21)] 82. On 6 October 2009 at 1315, DEI, and verified positive lube oil flow on NR 2A MPDE NR 8 bearing. [Encl. (21)] 83. On 6 October 2009 at 1734, bearing inspection on NR 2A MPDE NR8 bearing revealed no apparent damage. [Encl. (21)]


(b)(6)

(b)(6) (b)(6)


17

84. DEI, and the CHENG closed out NR 8 bearing. [Encl. (21)] 85. On 2 November 2009 at 1043, NR 2A MPDE stopped due to hot bearing casualty. [Encl. (34)] 86. On 2 November 2009 at 1102, casualty sample drawn on NR 2A MPDE NR 2 bearing. [Encl. (34)] 87. On 2 November 2009 at 1154, Diesel Engine Inspector

, CHENG, and inspected NR 2A MPDE NR 1 and NR 2 main bearing. [Encl. (35)] 88. SAN ANTONIO personnel and found a quarter of an inch ferrous metal sliver and several non-ferrous particles. [Encls. (35, 36)] 89. On 2 November 2009, NR 2A MPDE lube oil service system tagged out for inspection of strainer and back-flushing filter. [Encls. (35, 36)] 90. The strainer candles were dislodged. [Encl. (35)] 91. The strainer exhibited ferrous particulate (fine) on the magnet that if accumulated would be not exceed the area of a dime. [Encls. (35, 36)] 92. The back-flushing filter exhibited ferrous and non-ferrous over 75% of the candles. [Encl. (35)] 93. The ferrous particles exhibited on the back-flushing filter measured under one half inch when stretched. [Encls. (35, 36)] Identification of Foreign Matter NR 1A MPDE 94. The foreign debris removed from NR 1A MPDE sump included organic and metallic materials. [Encl. (37)] 95. The organic materials consisted of items similar to paint, textile floor coverings, synthetic polymer Nylon, rubber, hydraulic oil, Kevlar, glass fibers, hair, polypropylene, and cellulose fibers. [Encl. (37)] 96. The metallic materials consisted of material similar to a 300 Series Stainless Steel alloy. [Encl. (37)]


(b)(6)

(b)(6)

(b)(6) (b)(6)

(b)(6)


18

97. Three of the metallic samples provided appear to be from a machining operation. [Encl. (37)] 98. For comparison MARMC personnel analyzed known standards of 304 Series Stainless Steel, 316 Series Stainless Steel, 321 Series Stainless Steel, and 347 Series Stainless Steel alloys. [Encl. (37)] 99. MARMC personnel cannot identify the precise composition of the metallic materials due to possible contamination. [Encl. (37)] NR 1B MPDE 100. The debris removed from NR 1B MPDE are consistent with carbon steel of undetermined origin, and 300 Series Stainless Steel alloys from a machining operation. [Encl. (38)] NR 2A MPDE 101. The metallic debris removed from NR 2A MPDE lube oil service system back-flushing filter consisted of a material similar to a 300 Series Stainless Steel alloy from a machining peration. [Encl. (39)] o Internal Source of the Foreign Matter MPDE 102. There are no parts of the MPDE wetted by lube oil that contain stainless steel. [Encl. (40)] LO Service System 103. Each MPDE has its own lube oil sump, lube oil service pumps, self-cleaning filter, heat exchanger and duplex strainer. Lube oil is drawn from the sump via the installed positive displacement lube oil pumps. [Encl. (41)] 104. Lube oil discharged from the lube oil service pumps enters a self-cleaning filter. [Encl. (41)] 105. The self-cleaning filter filtration is 30 micron absolute and is equipped with an external (15PSID) and internal (29PSID) bypass that ensures the MPDE is not starved for oil if a high differential pressure condition occurs in the filter. [Encl. (421)] 106. The external bypass valve has been recently directed by appropriate technical authority to be disabled via a blank



19

flange to prevent contaminates bypassing the filter. [Encl. (41)] 107. The filter has a rotating set of internal elements that alternately filter the lube oil when in service. [Encl. (41)] 108. When the internal elements are not in service, they are flushed with lube oil discharged from the lube oil service pumps. [Encl. (41)] 109. The filter effluent is discharged directly back to the sump, which results in the total mass of contaminates remaining constant in the MPDE sump. [Encl. (6, 41)] 110. The filtered lube oil discharged from the self-cleaning filter flows through a heat exchanger, header pressure regulating valve, a duplex strainer (55 micron absolute), and then into the diesel complex where the lube oil is ultimately discharged into the MPDE lube oil sump. [Encl. (41)] 111. The attached transfer pump and purifier bowl assembly of the lube oil purifier come into contact with the lube oil. [Encl. (42)] 112. The transfer pump has carbon steel rotors and a bronze housing. [Encl. (42)] 113. The purifier bowl is made of a special stainless steel alloy. [Encl. (42)] 114. The paring disc is made of 316 Series Stainless Steel. [Encl. (42)] 115. If the paring disc and purifier bowl come into contact, this would cause noticeable sound and vibration. [Encl. (42)] 116. Any contact will be evident on the paring disc top and bottom surfaces. [Encl. (42)] 117. The lube oil piping system contains 304 Series Stainless Steel, 316 Series Stainless Steel alloy and Carbon Steel (from voyage repair). [Encl. (43)]



20

118. MPDE lube oil service pump special washers used to fasten the pump casing to its suction inlet bell and mechanical seal cover assembly are made from stainless steel. [Encl. (44)] 119. MPDE lube oil service pump special washers do not come in contact with lube oil during the normal operation of the lube oil service system. [Encl. (44)] 120. The valves in the lube oil service system do not contain stainless steel. [Encls. (45-47)] MPDE Bearings 121. The MPDE bearings consist of a steel backing, top layer in overlay, babbit, and copper. [Encl. (48)] t Internal Cause of Foreign Matter Introduction Into the LO System LO Service System Filling/Strikedown Procedures 122. MRC 2331 W-1R requires the drain and refill of the MPDE lube oil loop seals. [Encl. (49)] 123. During the performance of MRC 2331 W-1R, SAN ANTONIO personnel indicated the use of a blue plastic bucket to perform the maintenance check. [Encls. (50, 51)] 124. The blue plastic bucket is used to perform other tasks in the propulsion spaces. [Encls. (50, 51)] NR 1A MPDE 125. On 18 August 2009, the lube oil quality management (LOQM) log annotated zero gallons in the NR 1A MPDE lube oil sump. [Encl. (52)] 126. On 9 September 2009 at 1738, the prior to transfer (PTT) sample on lube oil tank 6-65-2-F was satisfactory. [Encl. (53)] 127. On 9 September 2009 at 1855, a lube oil transfer from lube oil tank 6-65-2-F to NR 1A MPDE lube oil sump commenced. [Encl. (53)] 128. On 9 September 2009 at 2113, a lube oil transfer from lube oil tank 6-65-2-F to NR 1A MPDE lube oil sump of 1647 gallons completed. [Encl. (53)] 129. On 10 September 2009, the LOQM log annotated 2000 gallons in the NR 1A MPDE lube oil sump. [Encl. (54)]



21

130. On 15 September 2009, the LOQM annotated 1750 gallons in the NR 1A MPDE lube oil sump. [Encl. (54)] 131. On 18 September 2009 at 0915, the PTT sample on lube oil tank 6-68-2-F was satisfactory. [Encl. (53)] 132. On 18 September 2009 at 1350, there was 538 gallons of oil transferred to NR 1A MDPE lube oil sump from lube oil tank 6-68-2-F. [Encl. (53)] 133. On 8 October 2009 at 0329, the PTT sample from lube oil tank 6-122-1-F satisfactory. [Encl. (21)] 134. On 21 October 2009 at 1340, the PTT sample on lube oil tank 6-68-2-F was satisfactory. [Encl. (21)] 135. On 22 October 2009, the LOQM log annotated 1640 gallons in the NR 1A MPDE lube oil sump. [Encl. (55)] 136. A lube oil transfer is not documented in the engineering log. [Encl. (21)] 137. On 23 October 2009 at 0639, the PTT sample on lube oil tank 6-121-1-F was satisfactory. [Encl. (21)] 138. On 4 November 2009 at 1548, NR 2 40WT lube oil purifier started purifying NR 1A MPDE sump to 6-117-1-F IAW PTLO. [Encl. (34)] 139. On 19 November 2009, the LOQM log annotated 1657 gallons in the NR 1A MPDE lube oil sump. [Encl. (56)] 140. A lube oil transfer is not documented in the engineering log. [Encl. (34)] 141. On 30 November 2009 at 0914, NR 1 40WT lube oil purifier transfer pump started taking suction from NR 1A MPDE lube oil sump discharging to lube oil tank 6-63-2-F IAW PTLO. [Encl. (34)] 142. On 30 November 2009 at 1453, NR 1 40WT lube oil purifier transfer pump secured IAW PTLO. [Encl. (34)]



22

143. On 30 November 2009 at 1453, a 1355 gallon transfer from NR 1A MPDE lube oil sump to lube oil tank 6-63-2-F completed. [Encl. (34)] NR 1B MPDE 144. On 18 August 2009, there was a 10 gallon increase in 1B MPDE sump (280 gallons to 290 gallons). No Prior to Transfer sample taken. [Encl. (52)] 145. A lube oil transfer is not documented in the engineering log. [Encl. (17)] 146. On 9 September 2009 at 1247, the PTT sample on lube oil tank 6-68-2-F was satisfactory. [Encl. (53)] 147. On 9 September 2009 at 1309, NR 1 40 WT lube oil purifier taking suction from lube oil tank 6-68-2-F to NR 1B MPDE. [Encl. (53)] 148. On 9 September 2009 at 1328, the PTT sample on lube oil tank 6-46-2-F was satisfactory. [Encl. (53)] 149. On 9 September 2009 at 1329, a lube oil transfer commenced from lube oil tank 6-68-2-F to NR 1B MPDE sump. [Encl. (53)] 150. On 9 September 2009 at 1738, the PTT sample on lube oil tank 6-65-2-F was satisfactory. [Encl. (53)] 151. On 9 September 2009 at 1729, a transfer of 2118 gallons of lube oil to NR 1B MPDE was completed. [Encl. (53)] 152. On 10 September 2009, the LOQM log annotated 2025 gallons in NR 1B MPDE sump. [Encl. (54)] 153. On 14 September 2009, the LOQM log annotated 2225 gallons in NR 1B MPDE sump. [Encl. (54)] 154. A lube oil transfer is not documented in the engineering log. [Encl. (53)] 155. On 15 September 2009, the LOQM log annotated 1769 gallons in NR 1B MPDE sump. [Encl. (54)] 156. A lube oil transfer is not documented in the engineering log. [Encl. (53)]



23

157. On 22 October 2009, the LOQM log annotated 1940 gallons in NR 1B MPDE lube oil sump. [Encl. (55)] 158. A lube oil transfer is not documented in the engineering log. [Encl. (21)] 159. On 23 October 2009 at 0639, the sample on lube oil tank 6-121-1-F was satisfactory. [Encl. (21)] 160. On 23 October 2009, the LOQM log annotated 1769 gallons in NR 1B MPDE lube oil sump. [Encl. (55)] 161. A lube oil transfer is not documented in the engineering log. [Encl. (21)] NR 2A MPDE 162. On 4 August 2009, the LOQM log annotated 380 gallons in NR 2A MPDE lube oil sump. [Encl. (52)] 163. A lube oil transfer is not documented in the engineering log. [Encl. (17)] 164. On 8 August 2009, the LOQM log annotated 280 gallons in NR 2A MPDE lube oil sump. [Encl. (52)] 165. A lube oil transfer is not documented in the engineering log. [Encl. (17)] 166. On 11 August 2009, the LOQM log annotated zero gallons in NR 2A MPDE lube oil sump. [Encl. (52)] 167. A lube oil transfer is not documented in the engineering log. [Encl. (17)] 168. On 15 September 2009 at 0850, the PTT sample on lube oil tank 6-122-1-F was satisfactory. [Encl. (53)] 169. On 15 September 2009 at 0923, NR 2 40WT lube oil purifier aligned and started filling NR 2A MPDE. [Encl. (53)] 170. On 15 September 2009, the LOQM log annotated 1416 gallons in NR 2A MPDE lube oil sump. [Encl. (54)] 171. A lube oil transfer is not documented in the engineering log. [Encl. (53)]



24

172. On 26 September 2009, the LOQM log annotated 1657 gallons in NR 2A MPDE lube oil sump. [Encl. (54)] 173. A lube oil transfer is not documented in the engineering log. [Encl. (53)] 174. On 7 October 2009, the LOQM annotated 1421 gallons in NR 2A MPDE lube oil sump. [Encl. (55)] 175. A lube oil transfer is not documented in the engineering log. [Encl. (21)] 176. On 8 October 2009 at 0329, the PTT sample on lube oil tank 6-122-1-F is satisfactory. [Encl. (21)] 177. On 8 October 2009 at 0340, NR 2 40WT lube oil transfer pump aligned and started taking suction from 6-122-1-F and discharging to NR 2A MPDE sump IAW PTLO. [Encl. (21)] 178. On 8 October 2009 at 0407, secured from transfer of lube oil to NR 2A MPDE Sump IAW PTLO. [Encl. (21)] 179. On 9 October 2009, 1657 gal annotated in LOQM Logs, +237 gal from previous day. [Encl. (55)] 180. The lube oil transfer is documented in the engineering log as occurring two days earlier and accounts for the 179 gallon increase. [Encl. (21)] 181. On 16 October 2009 at 1236, a PTT sample on lube oil tank 6-122-1-F was satisfactory. [Encl. (21)] 182. On 23 October 2009 at 0639, a PTT sample on lube oil tank on 6-121-1F was satisfactory. [Encl. (21)] 183. On 23 October 2009 at 0702, NR 2 40WT lube oil purifier aligned and started suction from 6-122-1-F discharging to NR 2A MPDE sump. [Encl. (21)] 184. On 23 October 2009 at 0742, secured from transferring 40WT lube oil to NR 2A MPDE Sump. Transferred 445 gal. [Encl. (21)]



25


Possible External Source of the Foreign Matter LO System Flushing Components 185. The flushing procedure involves the use of muslin bags that contain kevlar to catch debris. [Encls. (57, 58)] External Cause of the Foreign Matter Introduction Into the LO System Lube Oil Service System Flushing Process 186. The flushing process for the LPD 17 Class is NAVSEA S9233-DM-HBK-010-007 Diesel Engine Maintenance Standard Flush LPD 17, MPDE Lube Oil System and Engine. [Encl. (59)] 187. The flushing procedure consists of the following steps: a. The MPDE sump is drained and refilled with a high detergent, low viscosity special diesel lube oil. b. The installed lube oil service pumps, self-cleaning filter element and duplex strainer elements are removed to prevent a low flow area that could allow the accumulation of contaminants. c. A flush rig with pumps, heaters, filters and associated manifolds are installed as a temporary system. d. The pumps provide a pressure source for the lube oil, the heaters maintain lube oil temperature/viscosity at the required value and the manifolds aligned flushing lube oil to the appropriate components. e. Muslin bags are installed in the temporary filter and are used to capture any contaminants. f. The individual flushes are timed flushes that are complete once the contamination found in the muslin bags meets acceptance criteria. g. The temporary pumps take suction from the main sump and in series with the filter discharge clean oil into the external and/or internal piping where the contaminants in the piping are discharged into the sump. h. The general sequence is that the external lube oil piping is flushed, then the MPDE internal lube oil


26

piping/components are flushed followed by a combined (external and internal) flush. i. Upon completion of the flushes the sump is pumped down and cleaned. j. The LO Cooler flush can be accomplished at anytime during the flushing evolution. k. In addition to the temporary manifold installed to align flushing lube oil to the external and/or internal piping, two temporary valves were installed to bypass both pressure regulating valves serving the lube oil system. l. The government approved procedure fails to indicate the required position of these valves at the start of the external piping flush. [Encl. (60)] 188. NAVSEA S9233-DM-HBK-010-007 Note 2 (page 6) states the manual exceeds the requirements of NAVSEA S9086-HB-STM-010 NSTM Chapter 233 Diesel Engines. [Encl. (59)] 189. NAVSEA S9233-DM-HBK-010-007 does not contain a requirement for the lube oil sump to be cleaned prior to commencing the flush. [Encl. (59)] 190. NSTM Chapter 233 paragraph 233-8.21.12.f states that prior to flushing the engine sump shall be drained and wiped clean with lint free rags. [Encl. (60)] 191. NAVSEA S9233-DM-HBK-010-007 does not indicate location or amount of quadraplex filters required or installed during flushing. [Encl. (59)] 192. NSTM Chapter 233, Figures 233-8-1 and 233-8-2 depict filters installed immediately downstream of the flushing pump and immediately upstream of the sump. [Encl. (60)] 193. NAVSEA S9233-DM-HBK-010-007 Note 18 states that all checkpoints shall be verified, logged on enclosure (4) of the procedure and signed by performing activity representative and DEI before the next step can be performed. Verify with ship’s force representative only if required. [Encl. (59)]



27

194. NAVSEA S9233-DM-HBK-010-007 Checkpoint 2 states that DEI or Ship Representative and the contractor shall inspect all accessible engine areas to verify all foreign material and debris has been removed. [Encl. (59)] 195. NAVSEA S9233-DM-HBK-010-007 Temporary valves (TV 1 and 2) are installed in the external piping, however the alignment of these valves are not indicated in Table 1 (Valve Alignment). Paragraph 3-1.13.1 requires that TVs be opened for a minimum of 5 minutes; however this occurs during the flushing process allowing the potential for the valves to be improperly aligned at the start of the flush. [Encl. (59)] 196. NAVSEA S9233-DM-HBK-010-007, paragraphs 1-5.1 and 1-5.4 require removal of number 1 and 2 LO service pumps respectively, prior to commencing flush. [Encl. (59)] 197. The lube oil pumps were not removed prior to the flush. [Encls. (61, 62)] 198. NAVSEA S9233-DM-HBK-010-007 does not require the contractor to document the foreign material found during the flush of the system. [Encl. (60)] Damage Caused to the MPDE Bearings (to include cost) Bearing Inspection Results 199. An inspection of the following bearings has been conducted: NR 1A MPDE-Nr 2, 4, 6, and 8 main bearings and thrust bearing; NR 1B MPDE-all main bearings and thrust bearing; and NR 2A MPDE-all main bearings and thrust bearing. [Encl. (63, 64)] 200. The bearing inspections revealed that all bearing surfaces showed debris passed through. [Encl. (63-65)] 201. The bearing surfaces reflected debris similar to the following items: industrial, bead blasting material, material consistent with pipe welding process, dirt and sand, and water. [Encl. (63-65)] 202. All main bearing backings reflected fretting at the shell back ends with oil was found on the lower bearing steel-backs during the removal due to inadequate bearing crush. [Encl. (63-65)]



28

203. NR 1A MPDE thrust bearing lower shell exhibited high heat marks on the steel backing, which reflect a previous hot bearing during operation. [Encl. (63-65)] 204. The date and time of the damage to NR 1A thrust bearing cannot be determined due to no ICAS records, i.e., system inoperable or switched off. [Encl. (63, 64)] 205. NR 1B MPDE thrust bearing surfaces exhibited the passing of large and small debris. [Encl. (63-65)] 206. A 3 millimeter wide and 2 millimeter deep groove was visualized in the babbitt surface and embedded debris remained in the bearing surface and journal. [Encl. (63-65)] 207. The thrust journal has grooves and cuts matching the debris tracks in the bearing surface. [Encl. (63-65)] 208. The thrust bearing face showed wear consistent with Geisling mis-alignment/improper spacing. [Encl. (63-65)] 209. NR 2A MPDE thrust bearing surfaces exhibited the passing of small debris. [Encl. (63-65)] 210. The wear patterns were not consistent with the low operating hours. [Encl. (63, 64)] 211. All bearings removed and inspected meet the discard criteria in Chapter 4 of the LPD17 Class PC2.5 STC Volume 1 Technical Manual. [Encl. (63, 64)] 212. The current cost estimate to repair NR 1A, 1B, and 2A MPDE is $7.5 million dollars. [Encl. (66)] Maintenance Performance and Quality Control Efforts Shipboard Engineering Programs MPDE Material History 213. The Joint Fleet Maintenance Manual, Volume IV, Chapter 4 Diesel Engines, paragraph 4.4b requires that the Commanding Officer shall “ensure that each inspection/assessment is entered in the applicable machinery history with a brief description of any repair before operating conditions found at the conclusion of the assessment/inspection.” [Encl. (67)]



29

214. The Joint Fleet Maintenance Manual, Volume IV, Chapter 4 Diesel Engines, paragraph 4.5d requires that “Engine Operation Hours since Overhaul and Engine Operating Hours since Commissioning, at a minimum, shall be kept for all MPDEs, SSDGs, and EDGs. For Colt Pielstick Engines on LSD-41/49, LPD-17, and LHD-8 Class ships, operating hours since cylinder head refurbishment and individual power pack change outs shall also be kept.” [Encl. (67)] 215. The Joint Fleet Maintenance Manual, Volume IV, Chapter 4 Diesel Engines, Paragraph 4.6b(4) Commanding Officers shall ensure all diesel records are readily available to include “Diesel Maintenance/history records to include all maintenance and significant items accomplished on the engine since last engine overhaul.” [Encl. (67)] 216. SAN ANTONIO maintained material history logs using standard green hardback log books. [Encl. (68)] 217. The material history documented hours for each MPDE. [Encl. (68)] 218. The material history did not document inspection or ssessment for each MPDE. [Encl. (68)] a 3M Program 219. On 19 September 2009, EM01 conducted PMS Check: 2331-017-49 U-11 Change Main Engine Lube Oil System Lubricating Oil. [Encl. (69)] 220. The 13 Week Accountability Log does not clearly record the individual that conducted the check. [Encl. (69)] 221. On 20 September 2009, EM01 conducted PMS Check: 2331-017-49 U-35 Clean and Inspect Rocker Arm Lube Oil Strainer. [Encl. (69)] 222. The 13 Week Accountability Log does not clearly record the individual that conducted the check. [Encl. (69)] 223. On 20 September 2009, EM01 completed PMS Check: 2331-017-49 U-11 Change Main Engine Lube Oil System Lubricating Oil. [Encl. (69)]



30

224. The 13 Week Accountability Log does not clearly record the individual that conducted the check. [Encl. (69)] 225. On 20 September 2009, EM01 completed PMS Check: 2331-017-49 U-35 Clean and Inspect Rocker Arm Lube Oil Strainer. [Encl. (69)] 226. The 13 Week Accountability Log does not clearly record the individual that conducted the check. [Encl. (69)] 227. On 22 September 2009, EM01 conducted PMS Check: 2640/006-98 M-2 Clean and Inspect Diesel Engine Purifier Bowl. [Encl. (69)] 228. The 13 Week Accountability Log does not clearly record the individual that conducted the check. [Encl. (69)] 229. The check requires a tag out. [Encl. (70)] 230. On 1 October 2009, EM01 completed PMS Check: 2331-017-49 18M-8R Measure and Record Crankshaft Deflection Readings. [Encl. (69)] 231. The 13 Week Accountability Log does not clearly record the individual that conducted the check. [Encl. (69)] 232. On 29 October 2009, EM01 completed PMS Check: 2640/006-98 M-2 Clean and Inspect Diesel Engine Purifier Bowl (Tag Out # E-1281). [Encl. (69)] 233. The 13 Week Accountability Log does not clearly record the individual that conducted the check. [Encl. (69)] 234. On 29 October 2009, EM01 conducted PMS Check: 2640/006-98 M-2 Clean and Inspect Diesel Engine Purifier Bowl. [Encl. (69)] 235. The 13 Week Accountability Log does not clearly record the individual that conducted the check. [Encl. (69)] 236. On 12 November 2009, EM01 conducted PMS Check: 2331/017-49 18M-12R Test Injector Nozzle. [Encl. (69)] 237. The 13 Week Accountability Log does not clearly record the individual that conducted the check. [Encl. (69)]



31

238. The check requires a tag out. [Encl. (71)] 239. The 13 Week Accountability Log has “N/A” annotated for the tag out number. [Encl. (69)] 240. On 25 November 2009, EM01 conducted PMS Check: 2640/006-98 M-2 Clean and Inspect Diesel Engine Purifier Bowl. [Encl. (69)] 241. The 13 Week Accountability Log does not clearly record the individual that conducted the check. [Encl. (69)] 242. On 27 October 2009, EM02 conducted PMS Check: 2640/006-98 M-2 Clean and Inspect Diesel Engine Purifier Bowl. [Encl. (72)] 243. The 13 Week Accountability Log does not clearly record the individual that conducted the check. [Encl. (72)] 244. MRC 2331 R-12W MPDE Sump Samples is required: weekly in use. [Encl. (73)] 245. SAN ANTONIO personnel last performed the check in March 2009. [Encl. (50, 74)] 246. SAN ANTONIO personnel lined out MRC 2331 R-12W on MIP. [Encl. (50, 74)] 247. MRC 2331 R-13W MPDE Rocker Arm LO Viscosity Sample using OTC is required: weekly in use. [Encl. (75)] 248. SAN ANTONIO personnel last performed the check in March 2009. [Encl. (50, 74)] 249. SAN ANTONIO personnel lined out MRC 2331 R-13W on MIP. [Encl. (50, 74)] 250. MRC 2331 R-14M MPDE Sump Shore Lab Samples is required: Monthly in use. [Encl. (76)] 251. SAN ANTONIO personnel typically performed the check but did not take credit for performance of the check for PMS purposes. [Encl. (50)] 252. MRC 2331 U-35 C/I Rocker Arm LO Strainer is required: (a) >5 PSID; and (b) Change Rocker Arm LO. [Encl. (77)]



32

253. SAN ANTONIO personnel did not document the check for NR 2A MPDE, following change out of lube oil in August 2009, in 3M Program. [Encl. (50)] 254. MRC 2620 U-1 C/I Duplex LO Strainer is required: (a) >15 PSID; (b) LO Change; and (c) 1/watch 1st 48 hours after maintenance. [Encl. (78)] 255. SAN ANTONIO personnel did not perform check after lube oil change out or after maintenance. [Encl. (50)] 256. MRC 2620 U-3 Renew LO Filter Elements is required: (a) >15 PSID; (b) As directed by results of LO analysis; and (c) Change LO. [Encl. (79)] 257. SAN ANTONIO personnel did not document the check in conjunction with a lube oil change out. [Encl. (50)] 258. Ship’s Force provided Memorandum of Record reporting completion of MRC. [Encl. (50)] Lube Oil Quality Management Program 259. SAN ANTONIO personnel made no meaningful attempt to monitor JOAP/NOAP monthly preventive maintenance lube oil samples. [Encl. (81)] 260. Chain of command review of the LOQM logs were cursory and not performed on a daily basis. [Encl. (81)] 261. Quarterly calibration of the onboard KITTEWAKE lube oil analysis equipment was not performed. [Encl. (81)] 262. Over the period of February to May 2009, the SAMM Database consistently revealed high levels of engine wear products to include Iron (Fe) in NR 1A, 1B, and 2A MPDE lube oil. [Encl. (80, 81)] 263. EOCC Procedure Hot Bearing in a Main Engine (MHMEB) does not require a visual inspection of the strainers or filter media for the presence of babbit which would provide an immediate indication of bearing failure. [Encl. (82)] 264. EOCC Procedure Unusual Noise and Vibration in a Main Propulsion Diesel Engine (MNVME) does not require a visual inspection of the strainers or filter media for the presence of



33

babbit which would provide an immediate indication of bearing failure. [Encl. (82)] Training Program 265. An audit of FLTMPS TYCOM required courses associated with lube oil systems was conducted and the following deficiencies were noted: a. One of fifty required Sailors have completed the LPD 17 Class Lube Oil System Familiarization Course (LPD-005) b. Zero of two required Sailors have completed the Propulsion Alarm and Indication Course (A-651-0047) c. One of fifty required Sailors have completed the LPD 17 Class Fuel Oil System Familiarization Course (LPD-004) d. Five of twenty-three required Sailors have completed the LPD 17 Class Electrical Systems Familiarization Course (LPD-006) e. Two of sixty-five required Sailors have completed the LPD 17 Class Electrical Generation and Distribution Course (LPD-007) f. One of twenty-three required Sailors have completed the LPD 17 Class AC and Refrigeration Course (LPD-014) g. Two of fifty-eight required Sailors have completed the LPD 17 Class Reverse Osmosis Familiarization Course (LPD-017) h. Six of eighty-nine required Sailors have completed the LPD 17 Class Compressed Air System Familiarization Course (LPD-019) i. Five of eighty-eight required Sailors have completed the LPD 17 Class Engineering Control System (ECS) Course (LPD-032) j. Two of forty-five required Sailors have completed the LPD 17 Class Shipboard Status Monitor System Course (LPD-036) k. Three of forty-two required Sailors have completed the LPD 17 Class Main Propulsion Diesel Engine Operation Course (LPD-103A)



34

l. Two of fifty-eight required Sailors have completed the LPD 17 Class Ships Service Diesel Generator Operation Course (LPD-103B) m. One of seventy-four required Sailors have completed the LPD 17 Class Steering Gear Machinery System Operation Course (LPD-103D) n. Two of twenty-five required Sailors have completed the LPD 17 Class Controllable Pitch Propeller Operation Course (LPD-103E) o. Two of twenty-five required Sailors have completed the LPD 17 Class Fuel Oil Purifier Operation Course (LPD-104) p. Two of twenty-five required Sailors have completed the LPD 17 Class MPDE LO Purifier Operation Course (LPD-105A) q. Two of twenty-five required Sailors have completed the LPD 17 Class Reduction Gear LO Purifier Operation Course (LPD-105B) r. Two of seventeen required Sailors have completed the LPD 17 Class Electrical System Operation Course (LPD-106) s. Three of fifteen required Sailors have completed the LPD 17 Class Refrigeration Plant Operation Course (LPD-114B) t. One of twenty-five required Sailors have completed the LPD 17 Class Reverse Osmosis Operation Course (LPD-117) u. Three of forty-five required Sailors have completed the LPD 17 Class Shipboard Status Monitor Subsystem Operation Course (LPD-136) v. One of thirty-eight required Sailors have completed the LPD 17 Class Steering Gear Machinery Maintenance Course (LPD-203D) w. Three of forty-eight required Sailors have completed the LPD 17 Class Controllable Pitch Propeller Maintenance Course (LPD-203E) x. Zero of thirty-five required Sailors have completed the LPD 17 Class Fuel Oil Purifier Maintenance Course (LPD-204)



35

y. One of thirty-five required Sailors have completed the LPD 17 Class Main Reduction Gear LO Purifier Maintenance Course (LPD-205A) z. Two of twenty-five required Sailors have completed the LPD 17 Class Fuel Oil Purifier Operation Course (LPD-104) [Encl. (83)] 266. Engineering Department has not completed many required NKO courses including courses associated with ship control systems (ECS/MCS), main propulsion and lubricating oils. [Encl. (84)] 267. Three months of training records for the period of February to December 2009 were unavailable. [Encl. (84)] 268. Despite a post deployment leave period in March/April 2009, MP Division reported 100% attendance at all documented training (typical reports stated that 25 of 25 Sailors attended training). [Encl. (84)] 269. All MP Division training conducted since February 2009 was documented as being conducted by the same second class petty officer. [Encl. (84)] 270. The SAN ANTONIO Training Instruction requires that a Subject Matter Expert conduct the training. [Encl. (84)] 271. Since February 2009, MP Division conducted training three times on a system that did not exist on SAN ANTONIO: Diesel Engine Brake System. [Encl. (84)] 272. From February to December 2009, training was not conducted on MPDE bearing high temperature casualties, loss of lube oil pressure or loss of lube oil cooling casualties. [Encl. (84)] 273. Two lesson plans (Lubricating Oil Storage, Transfer and Purification and Diesel Engine Clutch and Brake System) were dated December 2000. [Encl. (84)] 274. Five lesson plans (Lubricating Oil Quality Management, Diesel Main Reduction Gear, Bearing Records, Air System Casualties and Diesel Engine Lube Oil System were dated 2005 (various months). [Encl. (84)] 275. The Diesel Lube Oil System lesson plan:



36

a. Did not address the self-cleaning filter or strainer inspection criteria and potential sources/types of contamination, or typical lube oil casualties (low level, low pressure, loss of cooling (high temperature) b. Had incorrect or incomplete criteria for the shift clean and inspection of the lube oil strainers [Encl. (84)] 276. The Lube Oil Storage, Transfer and Purification System lesson plan did not address required samples or appropriate lube oil log entries in support of transfers. [Encl. (84)] 277. The Lube Oil Quality Management lesson plan did not address NSTM requirement to periodically test lube oil analysis equipment, or weekly lube oil sample analysis when in operation, or the means to obtain the lube oil test results via the SAMMs database and the requirement to maintain lube oil in specification. [Encl. (84)] SAN ANTONIO Class Engineering Systems Shipboard Automated Maintenance Management System 278. Shipboard Automated Maintenance Management (SAMM) System provides electronic means for ship’s force to track lube oil analysis results. Ship’s force samples are sent to the lab. Results are posted on the internet in the SAMMS folder. [Encl. (85, 86)] 279. Based on oil sample results, recommendations to ship's force are developed by NSWC Philadelphia and forwarded to the ship via email. [Encl. (85, 86)] 280. PREDICT is contracted by the Navy to conduct lube oil sample analysis. The lab is required to provide analysis of samples two working days after the sample is received at the lab. [Encl. (85, 86)] 281. Currently NSWC Philadelphia has tasked PREDICT to conduct ferrography analysis on samples for LPD 17 class ships. [Encl. (85, 86)] 282. Lube oil sample shipping costs are paid for by the program under NSWC Philadelphia. [Encl. (85, 86)]



37

283. Based on location of the ship, and the methodology used to ship samples (transfer to support ship, inport, etc) samples may be received in the PREDICT lab out of sequence. [Encl. (85, 86)] 284. SAN ANTONIO has a stand alone version of SAMMS. Instead of automatically sending data to SAMMS, ship’s force must download information to a disc, and send information via the internet. [Encl. (85, 86)] 285. With a stand alone system, ship’s force does not have direct access to the SAMMS folder or lube oil analysis trend data. [Encl. (85, 86)] 286. NSWC Philadelphia calibrates Kittiwake equipment onboard ships every two years. [Encl. (85, 86)] 287. Training on SAMMS is scheduled for ships, and additional training is available upon request. [Encl. (85, 86)] 288. Crew members from SAN ANTONIO received SAMMS training in April 2008, July 2008, and August 2008 prior to the ship’s deployment. [Encl. (85, 86)] 289. Ship’s force had the capability via their Port Engineer to gain access to the monthly lube oil samples but did not attempt to do so because of their level of knowledge of the system. [Encl. (87-89)] 290. The current work around is that based on oil sample results recommendations to ship's force are developed by NSWC Philadelphia and forwarded to the ship via email. [Encl. (87-89)] Engineering Control System 291. LPD 17 class Engineering Control System (ECS) is a microprocessor-based real-time distributed data acquisition, control, processing, and display system that: a. provides remote monitoring and control of propulsion, electrical, auxiliary, fuel, damage control, and ballast systems b. consists of 20 operator Multi-Function Workstations (MFWS) and 35 Data Acquisition Units (DAUs)



38

c. uses the Shipboard Wide Area Network (SWAN) to distribute data to and from ECS consoles [Encl. (90)] 292. SWAN and ECS are commercial off the shelf systems integrated and maintained by Raytheon Incorporated during the new construction process. [Encl. (90)] 293. DAU circuit boards, also called VME Remote Terminal Units (VRTU), are the interface between engineering plant sensors and ECS and are designed to accept inputs from multiple sensors. [Encl. (90)] 294. The DAU configuration allows interruption of multiple sensor signals if one sensor electrically fails. [Encl. (90)] 295. Troubleshooting of the DUA is complex and tedious because the technician must search each signal wired to the VRTU for the failed sensor. [Encl. (90)] 296. After ship delivery in 2006, SAN ANTONIO and an outside maintenance community documented problematic ECS operation, troubleshooting, and repair. [Encl. (91)] 297. PREINSURV findings during Final Contract Trials held from 26-30 March 2007 included: a. Significant deficiencies associated with the SWAN that negatively impacted ECS operation. b. LPD 17 Class integrated support was marginal at best. [Encl. (92)] 298. PREINSURV findings during Material Inspection (MI) completed 26-30 November 2007 included: a. Ships force lacked the fundamental troubleshooting tools to attain self-sufficiency in effectively maintaining the SWAN and ECS because of system software upgrades and installation of alterations without supporting technical documentation b. Technical documentation was not available because of proprietary rights (i.e., MPDE engine controls technical documentation)



39

c. Integrated logistic support necessary to troubleshoot analog and digital signals between installed equipment and ECS DAUs did not exist d. At the start of the MI, ships force and NSSA technicians were actively troubleshooting 55 out of service faults and 71 unknown faults displayed at ECS consoles e. Optimal manning, existing level of knowledge, and lack of technical documentation had combined to exceed available ships force resources to operate the engineering plant f. During underway portion the MI, an average of 20 alarms per minute displayed by ECS during underway operations. g. Watchstanders had become conditioned to ignore alarms. [Encl. (93)] 299. ECS maintenance after INSURV focused on correcting shore power interface problems and restoring DAU circuit boards to reduce the number of ECS faults in support of the February 2008 OPEVAL. [Encl. (94, 95)] 300. Significant post OPEVAL maintenance included the intermittent loss of speed and heading at the Bridge and Steering Helm console. [Encl. (94)] 301. Technical community approach was to complete a system wide assessment of SWAN to determine root cause of signal failure. [Encl. (96)] 302. NAVSEA PMS 317 funded an onboard SWAN subject matter expert for the entire deployment to assist ships force with troubleshooting/repair and defining SWAN/ECS interface problems. [Encl. (97, 98)] 303. LPD 17 Class precommissioning crews received SWAN and ECS training directly from Raytheon during the commissioning process. [Encl. (91)] 304. Upon ship arrival in Norfolk, no RMC technicians were trained in the ECS system. [Encl. (91)] 305. NAVSEA-Philadelphia Code 955 has been designated as In Service Engineering Agent for ECS. [Encl. (91)]



40

306. Access to Raytheon for shipboard ECS technical support was convoluted due lack of a contract vehicle that could be used by the Regional Maintenance Centers (RMCs) for emergent troubleshooting and repair. Raytheon support is funded by NAVSEA PMS 317. [Encl. (100)] 307. A post delivery support assist process was used by the RMC Maintenance Team to interface with PMS 317 and Raytheon on a case by case basis. This process resulted in delays in system restoration. [Encls. (101, 102)] 308. ECS reliability issues caused SAN ANTONIO to set on average three additional engineering watch standers during normal operation. [Encl. (87, 88)] 309. In mid-December 2009, the Commanding Officer had approved approximately fifty Temporary Standing Orders (TSOs) that authorized ship’s force to modify EOSS or add additional duties to watch standers to locally monitor, operate or verify the proper operation of equipment vice remote operation and monitoring via the Engineering Control Systems (ECS). [Encl. (87, 89, 90, 103)] ICAS 310. ICAS provides a distributed set of six personal computers (PC) workstations and associated Portable Data Acquisition (PDA), Portable Data Terminal (PDT) and Data Acquisition Set-64 (DAS-64) capable of obtaining and displaying engineering plant equipment status data. [Encl. (104)] 311. ICAS maintains records of individual plant equipment in a ship unique Configuration Data Set (CDS), for monitoring and trend analysis functions. [Encl. (104)] 312. ICAS is a non-mission critical subsystem that only runs as a stand alone application on the unclassified side of the SWAN network. [Encl. (104)] 313. The Channel B connection (dual hone) to SWAN modifications for each ICAS work station were completed during the Post Shakedown Availability (2 April 2007 – 25 June 2007). [Encl. (105)] 314. During the PSA sea trials, the Main Machinery Room NR 1 ECS console and ICAS console desynched a total of three times:



41

ECS console desynched one time; ICAS console desynched two times. [Encl. (105)] 315. The desynchronization of one console did not affect the other console. [Encl. (105)] 316. NAVSEA ICAS and ECS representatives were onboard during sea trials. [Encl. (105)] 317. On 18 July 2007, NAVSEA ICAS representatives left SAN ANTONIO with all five ICAS workstations and one ICAS viewport operational. [Encl. (105)] 318. In October 2007, NAVSEA ICAS representative participated in an ICMP assessment aboard SAN ANTONIO. [Encl. (105)] 319. In October 2007, NAVSEA ICAS representative visited SAN ANTONIO to update logsheets and install new ECS software. [Encl. (105)] 320. On 1 March 2008, NAVSEA ICAS representative received email from stating ICAS was up and running, but needed to install a new logsheet. [Encl. (105)] 321. On 7 March 2008, NAVSEA ICAS representative received email stating that during an OPEVAL: "ICAS fault caused ECS to desync during ballast operations with a LCU in the well delaying the evolution." [Encl. (105)] 322. In July 2008, NAVSEA ICAS representative attempted to further assist ship via distance support (email) but lost communication. [Encl. (105)] 323. The LPD 17 Class de-synchronization issue is caused by the Ship Wide Area Network’s (SWAN) handling of network communications amongst the various ECS consoles. [Encl. (106)] 324. The de-synchronization problem was initially encountered during SAN ANTONIO’s builder trials (prior to ICAS installation). [Encl. (106)] 325. The recommended solution was the five ICAS workstations become dual honed with a Channel B connection and network card. [Encl. (106)]


(b)(6)

(b)(6)

(b)(6)

(b)(6)

(b)(6)

(b)(6)


42

326. The installed solution differed in that a “shared” Channel B connection with the neighboring ECS console was implemented. [Encl. (106)] 327. The five ICAS consoles thus shared a pre-existing channel B connection with a neighboring ECS console. [Encl. (106)] 328. The installation of the GIG E network onboard USS NEW ORLEANS (LPD 18) in Spring 2008 has further mitigated the potential of de-synchronization since it is more robust than the original ATM solution. [Encl. (106)] 329. NEW ORLEANS has been using ICAS since Spring 2008. [Encl. (106)] 330. Although SAN ANTONIO received the GIG E upgrade in (fall 2009) she is reluctant to utilize the ICAS consoles for fear of desynchronizing the entire control system. [Encl. (106)] 331. NAVSEA ICAS representatives believe that SAN ANTONIO has lost confidence in ICAS and will provide her training and a ship rider during her upcoming trial in January 2010. [Encl. (106)] 332. Ship’s force has mitigated the risk associated with ICAS being secured on SAN ANTONIO by setting additional watches and manually logging and trending propulsion plant parameters. [Encl. (88)] 333. Operating parameters and other equipment information monitored by ICAS is designed to be automatically sent off ship. [Encl. (107)] 334. This interface has not been enabled on LPD 17, so ship’s force would conduct a manual push of the data off ship on a periodic basis if ICAS was operational. [Encl. (107)] 335. Off ship subject matter experts would review the data and make recommendations with regard to operations and condition based maintenance. [Encl. (107)] 336. Until recently, only automated reports associated for air conditioning systems has been funded and available for ship’s force although ICAS is designed to monitor the entire propulsion plant. [Encl. (108)]



43

337. ICAS support would be in place to monitor MPDE performance for LPD 19 deployment. [Encl. (5)] Navy Training System Plan 338. The NTSP for USS San Antonio (LPD 17) Class Amphibious Transport Dock Ship is NSTP N85-NTSP-S-30-9603B (01 Jul 09). [Encl. (109)] 339. NSTP N85-NTSP-S-30-9603B page xii states that the 01 July 2009 version was prepared to document changes since the previously approved NSTP in May 2003. [Encl. (109)] 340. NSTP N85-NTSP-S-30-9603B pages I-3 to I-4 states: “LPD 17 Class ships were introduced into the Fleet beginning with the commissioning of USS SAN ANTONIO in January 2006. Despite the relative newness of the Class, modest improvements, modernization and changes have already been made that necessitate updating this NTSP. These ships were built with the primary goal of reducing the total ownership cost (TOC) while maintaining and improving readiness and reducing manning through greater emphasis on technological enhancements, distance learning, continuous learning systems, job/training aids, distributed/joint learning, virtual reality, and multi-media training. Introduction of these technologies provides the crew and embarked forces with training tools that provide the flexibility to adjust to meet mission objectives. The use of electronic media has greatly improved the overall capacity for the ship to train itself. This advance “training technology insertion, combined with implementation of advance Condition Based Maintenance technologies and the use of Integrated Condition Assessment System (ICAS) to eliminate work and increase system readiness, provides the LPD 17 class with a comprehensive program that reduces TOC and meets reduced manning requirements.” [Encl. (109)] 341. NSTP N85-NTSP-S-30-9603B page I-5 states: “TSTS. The TSTS provides the LPD 17 Class ships with centralized coordination and management of training. It maintains the individual and team training records including courses/events required and completed, qualifications attained, and evaluations of training effectiveness. ICW courses can be taken from any work station computer or tactical console with the results interfacing directly with TSTS. TSTS also supports and records the results of stand alone training exercises using the embedded training features of specific systems. The Navy Integrated



44

Application Product Suite (NIAPS) and Navy Knowledge Online (NKO) Afloat will replace TSTS functionality once the LPD 17 Class ICWs and IETMs are compatible with and hosted on NKO.” [Encl. (109)] 342. The Ship’s Electronic Classroom was never finished and is currently used as the ship’s gym. [Encl. (83, 88)] 343. The TSTS Server is located in the Learning Resource Center (LRC) but is not connected to the ship’s LAN or desk top computers in the LRC. [Encl. (83, 88)] 344. The work around for TSTS is that classes that should be resident on the TSTS are available via Navy Knowledge Online (KNO). Course completion is automatically posted to FLTMPS database. Qualifications are manually tracked using onboard data bases. [Encl. (83, 87, 88)] 345. Engineering Department has a low completion rate of NKO courses. [Encl. (83)] 346. In the opinion of the Training Officer,

the training department is too small (approximately four Officers/Sailors) to track and record training. [Encl. (83)] 347. The ship’s training instruction was signed by the current Commanding Officer in February 2009. [Encl. (83)] 348. The Training Officer was unfamiliar with several aspects of the ship’s training instruction including lesson plan approval, record keeping requirements and monthly training reports. [Encl. (83)] 349. NSTP N85-NTSP-S-30-9603B page I-7 under “Organizational Level states: “The LPD 17 maintenance strategy calls for the maximum use of onboard O-Level maintenance capabilities through state of the art maintenance aids such as the Module Test and Repair capability, ICAS, ETMs, and IETMs. PM is accomplished in accordance with the ship’s Planned Maintenance System (PMS) Schedule. The performance of PM maximizes the service life and minimizes equipment down time. PM tasks include condition directed and time directed tasks. Every effort should be made to reduce the number of time driven tasks and make optimum use of condition monitoring equipment instead…” [Encl. (109)]


(b)(6)

(b)(6)


45

350. NSTP N85-NTSP-S-30-9603B page I-7 under “Corrective Maintenance states: “CM incorporates the use of ETMs/IETMs and interactive diagnosis to reduce MTTR. The installation of ICAS affords “shore based” knowledge aboard ship, thereby reducing trouble shooting time and maintenance-induced failures and improving the operational efficiency of the ship.” [Encl. (109)] 351. NSTP N85-NTSP-S-30-9603B Appendix A discusses a “LPD 17 Class Stakeholder meeting and training summit” held 22 January 2008. [Encl. (109)] 352. “The Stakeholder’s meeting identified five major systems aboard the LPD 17 Class ships that presented critical operation, maintenance and training concerns. These systems included the Main Propulsion Diesel Engines (MPDE), the Ship Service Diesel Generators (SSDG), the Engineering Control System (ECS), the Ship Wide Area Network (SWAN), and the MK 46 MOD ½ 30mm Gun Weapon System(GWS). All of these systems except the MK 46 GWS are commercial off the shelf systems that are unique to LPD 17 Class and for which no Equipment/System/Subsystem (E/S/S) Navy Training System Plan has yet been developed and no formal Navy schoolhouse/pipeline training currently exists. A major outcome of this meeting was that TEAM SHIPS and all the Stakeholders established, in accordance with OPNAVINST 1500.76A, a Training Planning Process Methodology (TRPPM) Advisory Board to plan the way ahead to determine the long term training solution and update this LPD 17 Class NTSP.” [Encl. (109)] 353. “The LPD 17 Class training summit was called to address the action items identified during the Stakeholder’s Meeting. The four main tasks assigned were to: a. Describe the immediate training requirements for USS SAN ANTONIO (LPD 17) b. Describe initial training for LPD 17 Class Pre-Commissioning Crews c. Determine what the interim training plan for LPD 17 Class follow-on crews should be



46

d. Develop the process for determining the formal long term training solution for the LPD 17 Class and brief the “Way Ahead.” [Encl. (109)]

354. “The following paragraphs provide the individual working group’s recommended short-term training solutions. These remain recommendations until vetted by the LPD 17 Class TRPPM Advisory Board. To determine the Follow-on/Life Cycle Training Solutions Front End Analysis (FEA) must be funded and performed for each of the below systems and vetted by the TRPPM Advisory Board to implementation… [Encl. (109)] 355. Recommendations of the HM&E working group: the consensus of the working group membership is to create a training pipeline for the LPD 17 Class propulsion system that is similar to the existing one for LSD 41/49 Class leading to the assignment of a Navy Enlisted Classification (NEC) codes for LPD 17 Class propulsion technicians and LPD 17 Class Engineering Control System (ECS) Operation and Maintenance Technicians…” [Encl. (109)] LPD 17 Title 10 Training/Budget 356. 17 March 2009 Memorandum of Agreement for LPD 17 Manpower, Personnel, and Training (MPT) requirements was signed by representatives from OPNAV N85, OPNAV N151, PMS 470, PMS 317, and LSD/LPD CLASSRON. [Encl. (110)] 357. Per the MOA, PMS 317 shall maintain the management, execution and fiscal responsibility for developing and implementing initial new construction crew familiarization, indoctrination, including counterpart training. PMS 317 shall also be responsible for interim training for SWAN, ECS, MPDE, SSDG, and the MK 46 GWS, as well as other systems as funded by OPNAV through FY 11. PMS 317 shall coordinate with PMS 470 and the LSD/LPD 17 CLASSRON to ensure lifecycle/follow-on training plan is developed that meets the MPT requirements of the fleet. [Encl. (110)] 358. Per the MOA, PMS 470 shall maintain the management, execution and fiscal responsibility for developing and implementing the life cycle/follow-on training requirements, to include the development of any necessary system level Navy Training System Plans and the associated training solutions. [Encl. (110)]



47

359. Per the MOA, PMS 470 shall coordinate with Training Planning Process Methodology (TRPPM) stakeholders to ensure lifecycle/follow-on training plan is developed that meets the MPT requirements of the fleet. [Encl. (110)] 360. Per the MOA, starting in FY12, PMS 470 shall assume all LPD 17 interim MPT requirements. [Encl. (110)] 361. According to POM 12 Budget (internal CNO staff documents) formal or school house training does not exist for Mission Critical Hull Material Electrical Systems, SWAN GIG E, IC systems, Video Systems, Mine Signature Control System, and Navigation Data Distribution System. [Encl. (111)] 362. Proposed POM 12 submission by N85 proposes additional funding to meet life cycle training requirements for SWAN, Navigation Data Distribution System, Engineering Control System, Ship Service Diesel Generator, Main Propulsion Diesel Generator, Electric Plant, Steering Control System, Interior Voice Communications, and Magnetic Signature Control System. [Encl. [Encl. (111)] LPD 17 Class Manpower and Training 363. Initial manpower estimates for a ship class are developed by the program office. The Preliminary Ship Manpower Document (PSMD) generated by the program office is refined by NAVMAC into a Ship Manpower Document after the ship's first deployment (1-2 years after commission). [Encl. (112-114)] 364. The Navy Training System Plan (NTSP) for LPD 17 was based on the premise that training would occur onboard the individual ships in lieu of traditional classroom/schoolhouse training. However, all the new systems, manpower, and training were not linked in the current NTSP even though the new systems and onboard training were factors in reducing total ownership cost and manpower. [Encl. (112-114)] 365. The resource sponsor (N85) recommended a review of the current NTSP due to insufficient detail to execute the described shipboard training for all systems and sub-systems. [Encl. (112-114)] 366. PMS 317 is conducting a training requirements solution analysis using Training Planning Process methodology (TRPPM) to generate new NTSPs for each subsystem (ECS, MPDE, SSDG, MK-46,



48

MPDE, SWAN, etc). (Expect delivery to N85 in Mar 2010) [Encl. (112-114)] 367. LPD 17's Preliminary Ship's Manpower Document was released by PMS 317 in January 2004 and was updated in February 2005(Change 1), April 2007(Change 2), August 2008 (Change 3), and November 2009 (Change 4). [Encl. (112-114)] 368. NAVMAC validation process of the Ship's Manpower Documents is in progress. The document is expected to be ready for review by the Fleet in June/July of 2010. [Encl. (112-114)] 369. NAVMACs validation does not include the manpower associated with the requirement for the crew to self-train. Counting "self-train" time would conflict with the afloat wartime Navy Standard Workweek (NSWW) which already contains seven hours per week per requirement provided for sustainment training to maintain readiness. This time is not earmarked for "initial" training and qualification/certification. To add in classroom time would result in a double-counting of training hours and increase manpower requirements by including unqualified personnel in the document. NAVMAC builds manpower documents that reflect fully qualified requirements in a wartime environment - not a training environment. [Encl. (112-114)] NAVSEA Training 370. NAVSEA provides training to PRECOM crews on ICAS, ECS, SWAN, SSDG, MPDEs, and the MK 46 Gun Weapon System. Courses are taught either by NAVSEA or with Original Equipment Manufacturer (OEM) contracted support. [Encl. (115-118)] 371. NAVSEA interim training requires students complete prerequisite Interactive Course Ware (ICW) before attending the courses. Interim training is referred to as “Delta.” Delta courses are geared to students who are expected to be more senior in ratings with the associated experience base to benefit from the training. [Encl. (115-118)] 372. MPDE/SSDG/ECS interim training consists of a combined one week course conducted simultaneously using two classrooms ashore plus shipboard systems for hands-on training. The MPDE/SSDG interim training session is for the Engineman rating. It includes one and one-half days on SSDG, one-half day on ECS and two days on MPDE followed by one day hands-on aboard ship. The electric plant/ECS training session is for the EM, ET and IC



49

ratings. It includes two days on electric plant, one day on ECS and two days hands-on aboard ship. [Encl. (115-118)] 373. SWAN interim training is conducted in two stages. Stage 1/SWAN operations and maintenance hands-on orientation course: two weeks of hands-on orientation training conducted by Raytheon in their sustaining engineering environment (see) lab in San Diego, California. Stage 2/SPAWAR SWAN system administrator Delta course is two weeks in length. [Encl. (115-118)] 374. Mk46 Mod2 GWS interim training course is three weeks. The course consists of an overview, operations and basic maintenance training for GM/FC ratings. [Encl. (115-118)] 375. Interim training is provided on both coasts. The intent of the interim training is to cover the rotation of initially trained crews of the LPD 17 class until the pipeline training is either ready for training (RFT) or the necessary updates have been incorporated. Classes are open to all LPD 17 commands. [Encl. (115-118)] 376. Twenty San Antonio Sailors completed MPDE/SSDG/ECS interim training in June 2009. [Encl. (115-118)] LPD 17 Training and Qualification Pipeline Training 377. PCO and PXO pipeline training does not include LPD 17 ECS and ICAS. [Encl. (104)] 378. LPD 17 Class Chief Engineers are second tour 1110 Department Heads. [Encl. (104)] 379. As of 15 December 2009, with the exception of one officer, all other LPD 17 Class Chief Engineers’ first department head tour was aboard a diesel platform. [Encl. (104)] 380. The excepted officer’s first department head tour was aboard a gas turbine platform. [Encl. (104)] 381. Prior to reporting, LPD 17 Class Chief Engineers complete Surface Warfare Officer Prospective Engineering Officer Diesel Course (A-4H-0063). [Encl. (104)]



50

382. Prior to reporting, LPD 17 Class Main Propulsion Assistants and Auxiliary Officers complete Advanced Diesel Engineering Course (A-4H-0160). [Encl. (104)] 383. The curriculum for Surface Warfare Officer Prospective Engineering Officer Diesel Course (A-4H-0063) and Advanced Diesel Engineering Course (A-4H-0160) are similar. [Encl. (104)] 384. Weeks one through three (Common Core) cover an in-depth explanation of engineering theory and engineering management programs including Maintenance University (two days) and Advanced Shipboard Firefighting (two days). [Encl. (104)] 385. Weeks four through seven (Specialty) cover Main Propulsion, Auxiliaries, and Electrical systems and includes the use of a LPD-17 desk top trainer in the execution of engineering evolutions and casualty control procedures. [Encl. (104)] 386. Prior to week eight’s Engineering Assessment with Afloat Training Group, students are required to complete an oral board. [Encl. (104)] 387. During Week 8 (Engineering Assessment), students complete a mock engineering assessment. [Encl. (104)] 388. To complete the engineering assessment, the student must satisfactory perform drills and evolutions within the LPD-17 trainer. [Encl. (104)] 389. Satisfactory performance is 50% for drills and 65% for evolutions and is in accordance with the SFTM. [Encl. (104)] 390. Week 9 (Ship Ride) consists of a SWOS N74 Instructor led space walk-through, equipment familiarization, and material self assessment training on platforms that correspond to their ultimate duty station. [Encl. (104)] 391. Throughout the course, assessors evaluate every students level of knowledge through a review of operating logs, records and management program questions as they conduct evolutions and drills. [Encl. (104)] 392. The assessors evaluate student performance in a "watch team" mode noting the students ability and willingness to provide "forceful backup". [Encl. (104)]



51

393. Instruction for ICAS and ECS is approximately 65 hours of Instructor Led Training (ILT), and a two hour lab/lesson from

NAVSEA Philadelphia Code 951. [Encl. (104)] 394. Instruction includes the traditional methods of main propulsion system operation and plant monitoring as a back-up to ICAS. [Encl. (104)] 395. Successful completion of Surface Warfare Officer Prospective Engineering Officer Diesel Course (A-4H-0063) and Advanced Diesel Engineering Course (A-4H-0160) includes passing multiple written tests and level of knowledge oral boards; and completing homework and the mock Engineering Assessment. [Encl. (104)] 396. The minimum passing GPA for the Surface Warfare Officer Prospective Engineering Officer Diesel Course (A-4H-0063) and Advanced Diesel Engineering Course (A-4H-0160)is 80%. [Encl. (104)] 397. USS SAN ANTONIO (LPD 17) personnel completed the following courses of instruction: Surface Warfare Officer Prospective Engineering Officer Diesel Course (A-4H-0063)-

on July 2007; Advanced Diesel Engineering Course (A-4H-0160) on July 2007, and on November 2009. [Encl. (104)] 398. USS NEW ORLEANS (LPD 18) personnel completed the following course of instruction: Surface Warfare Officer Prospective Engineering Officer Diesel Course (A-4H-0063)

4/2007 MGT PEO; Advanced Diesel Engineering Course (A-4H-0160)- on April 2007. [Encl. (104)] 399. USS MESA VERDE (LPD 19) personnel completed the following course of instruction: Surface Warfare Officer Prospective Engineering Officer Diesel Course (A-4H-0063)- on July 2007; Advanced Diesel Engineering Course (A-4H-0160)-CWO3

on July 2009, and on September 2009. [Encl. (104)] 400. USS GREEN BAY (LPD 20) personnel completed the following course of instruction: Surface Warfare Officer Prospective Engineering Officer Diesel Course (A-4H-0063)- on May 2009 (first tour MGT engineer); Advanced Diesel Engineering Course (A-4H-0160)- on February


(b)(6)

(b)(6)

(b)(6) (b)(6)

(b)(6)

(b)(6)

(b)(6)

(b)(6)

(b)(6)

(b)(6) (b)(6)

(b)(6)

(b)(6)


52

2008, on December 2007, and on July 2007. [Encl. (104)]

401. USS NEW YORK (LPD 21) personnel completed the following course of instruction: Surface Warfare Officer Prospective Engineering Officer Diesel Course (A-4H-0063)- on September 2006; Advanced Diesel Engineering Course (A-4H-0160)- on July 2009, on May 2008, on December 2008, and on December 2008. [Encl. (104)] ICAS Training 402. On 7 December 2009, the following USS SAN ANTONIO (LPD 17) personnel attended ICAS Basic Operator (CIN# A-191-0800):

, , , and [Encl. (119)]

403. On 8 December 2009, the following USS SAN ANTONIO (LPD 17) personnel attended ICAS Engineering Supervisor (CIN# A-191-3001): , and

. [Encl. (119)] 404. From 9 to 11 December 2009, the following USS SAN ANTONIO (LPD 17) personnel attended ICAS Maintenance Manager (CIN# A-191-1002): and [Encl. (119)] Personnel Qualifications Standard 405. EOOW PQS NAVEDTRA 43104 is divided into three sections: 100 Introduction to Fundamentals; 200 Introduction to Systems; and 300 Introduction to Watchstations. [Encl. (120)] 406. EOOW PQS NAVEDTRA 43104 Section 100 Introduction to Fundamentals does not include the main reduction gear or ship service diesel generator. [Encl. (120)] 407. EOOW PQS NAVEDTRA 43104 Section 200 Introduction to Systems does not include the main reduction gear lube oil system or ship service diesel generator lube oil system. [Encl. (120)] 408. EOOW PQS NAVEDTRA 43104 Section 200 Introduction to Systems: Diesel Lube Oil Fill, Transfer, and Purification System does not include the filter or lube oil cooler components. [Encl. (120)]


(b)(6) (b)(6)

(b)(6)

(b)(6)

(b)(6) (b)(6)

(b)(6) (b)(6)

(b)(6)

(b)(6) (b)(6) (b)(6) (b)(6)

(b)(6)

(b)(6) (b)(6) (b)(6)

(b)(6)

(b)(6) (b)(6)


53

409. EOOW PQS NAVEDTRA 43104 Section 200 Introduction to Systems: MPDE System does not include the lube oil cooler. [Encl. (120)] 410. Engineering Collaterals and Qualifications PQS NAVEDTRA 43704 is divided into three sections: 100 Introduction to Fundamentals; 200 Introduction to Systems; and 300 Introduction to Watchstations. [Encl. (121)] 411. Engineering Collaterals and Qualifications PQS NAVEDTRA 43704 200 Introduction to Systems includes: 203 Fuel Oil (FO) Service; and 204 FO Filling, Transfer, and Purification System. [Encl. (121)] 412. Engineering Collaterals and Qualifications PQS NAVEDTRA 43704 Watchstation 320 Lube Oil King does not require any 200 Systems prerequisites. [Encl. (121)] 413. Engineering Collaterals and Qualifications PQS NAVEDTRA 43704 Watchstation 320 Lube Oil King does not require completion of any 200 Systems (e.g., 207 Lube Oil (LO) Storage, Transfer, and Purification System) as a prerequisite for the completion of the watchstation. [Encl. (121)] 414. Engineering Collaterals and Qualifications PQS NAVEDTRA 43704 Watchstation 335 Fuel Oil King requires completion of 200 Systems: 203 Fuel Oil (FO) Service; and 204 FO Filling, Transfer, and Purification System, as a prerequisite for the completion of the watchstation. [Encl. (121)] 415. On 26 March 2008, the Commanding Officer designated

as Assistant Oil King. [Encl. (122)] 416. On 3 June 2009, the Commanding Officer designated

as Oil King. [Encl. (123)] 417. On 1 September 2009, the Commanding Officer designated

as Oil King. [Encl. (124)] 418. The designation letters reference NAVEDTRA 43116-5G, the PQS for Oil King/Water King for a plant with main propulsion boilers. [Encl. (122-124)]


(b)(6)

(b)(6)

(b)(6)

(b)(6)

(b)(6)

(b)(6)


54

OPINIONS

Metal Shavings Discovery and Causation Source of the Foreign Material 1. The probable source of the 300 series stainless steel found in NR 1A, 1B, and 2A MPDE lube oil service systems is the byproducts of the production and installation of stainless steel lube oil service system piping. [FoF (1-6, 12, 94, 96-100, 102, 113-120)] 2. A possible, but unlikely, source of the 300 series stainless steel found in NR 1A, 1B, and 2A MPDE lube oil service systems is the lube oil service pump. [FoF (1-6, 12, 94, 96-100, 102, 113-120)] 3. Until removal and inspection, the lube oil service pump cannot be eliminated as a source of the 300 series stainless steel found in NR 1A, 1B, and 2A MPDE lube oil service systems. [FoF (1-6, 12, 94, 96-100, 102, 113-120)] 4. A possible, but improbable, source of the 300 series stainless steel found in NR 1A, 1B, and 2A MPDE lube oil service systems is the lube oil purifier. [FoF (1-6, 12, 94, 96-100, 102, 113-120)] 5. The source of the babbitt found in NR 1A, 1B, and 2A MPDE lube oil service systems is from the MPDE’s bearings. [FoF (121)] 6. The probable source of the ferrous materials found in NR 1A, 1B, and 2A MPDE lube oil service systems is engine wear products. [FoF (5, 27-29, 35, 36, 63, 64, 71, 88, 91-94, 96-101)] 7. The possible, but unlikely, source of the ferrous materials found in NR 1A, 1B, and 2A MPDE lube oil service systems is the byproducts of the production and installation of carbon steel lube oil service piping. [FoF (5, 27-29, 35, 36, 63, 64, 71, 88, 91-94, 96-101)] 8. The probable source of the fibrous materials and kevlar found in NR 1A, 1B, and 2A MPDE lube oil service systems is the byproducts of MPDE lube oil flush materials, i.e., muslin bags. [FoF (36, 61, 95, 183, 185)]



55

Introduction of the Foreign Material 9. The poor weld joint fit up during new construction created contaminate traps that could allow for an intermittent release of contaminates. [FoF (2-4)] 10. The poor weld joint fit up during new construction contributed to the introduction of the foreign material into the lube oil service system. [FoF (2-4)] 11. The poor weld joints have shown visual indications of poor weld techniques that result in stainless steel globules that could break off and propagate through the lube oil service system. [FoF (2-4)] 12. If these poor joints exist between the lube oil strainers and the diesel complex, then the bearings would not be protected from damage. [FoF (2-4, 102-111)] 13. Neglecting to flush the lube oil service system following the Fall 2008 voyage repairs and an inadequate flush during the CNO PMA contributed to the introduction of the foreign material into the lube oil service system. [FoF (4, 6, 7, 184-196)] 14. The post-flush production work that occurred during the CNO PMA (NR 1A and NR 1B MPDEs valve cover software replacement, NR 1B MPDE crankcase fan oil seal, and NR 2A MPDE low pressure air line) is unlikely to have introduced foreign material in the lube oil service system. [FoF (18-21, 48-50, 52-55)] 15. The blue bucket used for the performance of MRC 2331 W-1R Loop Seal Fill may have contributed to the introduction of foreign material in the lube oil service system. [FoF (120-122)] 16. Until the lube oil storage tanks are inspected, the transfer procedures used by SAN ANTONIO personnel of lube oil from storage tanks to the lube oil service system cannot be eliminated as a contributing factor of the introduction of foreign material in the lube oil service system. [FoF (123-182)] 17. There is no evidence to support a malicious introduction of foreign material into the lube oil service system. [FoF (1-196)] Removal of the Foreign Material Diesel Engine Maintenance Standard Flush LPD 17, MPDE Lube Oil System and Engine (NAVSEA S9233-DM-HBK-010-007)



56

16. All foreign material will not be removed from the lube oil service system because the procedure discharges the flushing medium into the main engine lube oil sump without being filtered. [FoF (184, 188, 189)] 17. The failure to designate the alignment of temporary valves prior to commencing the flush potentially resulted in diminished flow to piping or caused external piping dead legs with no lube oil flow resulting in contamination being released upon MPDE operation. [FoF (184)] 18. The procedure did not meet the requirements of NSTM Chapter 233. [FoF (187-190)] 19. The procedure does not provide a true indication of the level of contamination remaining in the lube oil service system piping at the conclusion of the flush. [FoF (184, 185)] 20. The procedure as executed during the CNO PMA was inadequate as the permanent lube oil service pumps were not removed. [FoF (192, 193)] 21. The procedure should require documentation of the contamination in muslin bags and sump to assess procedure effectiveness. [FoF (190, 194)] 22. The permanent lube oil service pumps were not removed prior to flush commencement preventing thorough inspection and cleaning of pump parts and draw box potentially leaving contamination in these areas. [FoF (192-193)] Lube Oil Service System Design 23. The location of the self-cleaning filters upstream of the duplex strainers will dump contaminated oil back to the MPDE sump. [FoF (59-61, 70-72, 101, 104-110)] 24. Particulate smaller than 25 microns will pass through the self-cleaning filter. [FoF (59-61, 70-72, 101, 104-110)] 25. The lube oil contaminates that are circulated back to the MPDE sump via the self-cleaning filter are continuously reduced in physical size by the macerating action of the lube oil service pumps until the contaminates are small enough to pass through the filter and duplex strainers. [FoF (102-111)]



57

26. Contaminates that pass through the filter and strainer complex could still be large enough to cause bearing damage. [FoF (102-111, 199-212)] Bearing Damage 27. The damage to the main and thrust bearings occurred prior to CNO PMA. [FoF (34-39, 42-45, 47, 63, 64, 66, 68, 71, 72, 85, 86, 88, 90-101, 199-212, 259-262)] 28. The primary cause of the damage to the main and thrust bearing was the result of a lack of adherence to cleanliness standards during the assembly and installation of the MPDEs and lube oil support systems during new construction. [FoF (34-39, 42-45, 47, 63, 64, 66, 68, 71, 72, 85, 86, 88, 90-101, 199-212, 259-262)] 29. The 300 series stainless steel found in NR 1A, 1B, and 2A MPDE lube oil service system did not cause the bearing damage. [FoF (34-39, 42-45, 47, 63, 64, 66, 68, 71, 72, 85, 86, 88, 90-101, 199-212, 259-262)] Maintenance Performance and Quality Control Efforts Shipboard Engineering Programs 30. SAN ANTONIO MPDE material history program is ineffective. [FoF (213-218)] 31. SAN ANTONIO 3M program is partially effective. [FoF (219-258)] 32. SAN ANTONIO Lube Oil Quality Management program is ineffective. [FoF (122-184, 259-262)] 33. SAN ANTONIO Engineering Department training program is not effective. [FoF (265-277)] 34. Had the Lube Oil Quality Management program been effective, SAN ANTONIO personnel may have recognized the damage to the MPDE main and thrust bearings sooner. [FoF (34-39, 42-45, 47, 63, 64, 66, 68, 71, 72, 85, 86, 88, 90-101, 122-184, 199-212, 259-262)] LPD 17 Class Engineering Systems 35. SAMMS as a standalone system creates obstacles for ship’s force to obtain and integrate lube oil sampling data into material history. [FoF (216-218, 262, 278-290)]



58

36. Ship’s force’s ability to operate the plant during normal and casualty operations is severely handicapped by the failure of installed monitoring and control systems to operate properly. [FoF (278-355)] 37. EOCC procedures for MPDE high bearing temperature casualties are lacking in immediate and supplemental actions. [FoF (263, 264)] 38. Several systems: ICAS, ECS, and TSTS have not reached full maturity or are not functioning. In addition, the ship’s crew has not demonstrated the wherewithal to properly mitigate the impact of operating a ship with diminished systems. The size of the crew is a significant factor in the capacity to address system challenges. The pre-commissioning training pipeline and interim training courses appear insufficient to bring the crew to the requisite level of knowledge to manage the propulsion plant correctly. The reduction of manpower and the incompleteness of designed support are contributing factors in the operational unavailability of the LPD 17 class. [FoF (278-377, 393, 394, 402-404)] 39. Several stakeholders recognize the lack of depth in concerning the NTSP approach of self training, and have recommended returning to traditional schoolhouse training as a resolution. The process of how the crew will self-train is inadequately discussed in the NTSP, and NAVMAC is not charted to account for self-training in the accounting of personnel work load. The stakeholders way ahead is to budget for the infrastructure and instructor base in POM 12. [FoF (346, 349-376)] 40. Training for the LPD17 class has not seen the rigor and follow-up that is necessary to support long term sustainability of level of knowledge necessary to support the life of the ship sailor and maintenance. [FoF (265-277, 289, 303, 304, 338-418)] 41. Engineering PQS across all watchstations requires further review and tailoring to account for major LPD 17 components and systems. [FoF (405-418)] 42. EOOW PQS NAVEDTRA 43104 and Engineering Collaterals and Qualifications PQS NAVEDTRA 43704 do not adequately address the level of knowledge of engineering watchstanders for lube oil systems and the associated engineering equipment supported by the lube oil system. [FoF (405-418)]



59

RECOMMENDATIONS 1. SAN ANTONIO develop a comprehensive POAM in accordance with EDORM to assess, correct and train engineering department in critical programs to include QA, LOQM and LEGAL records by 1 February 2010. 2. SAN ANTONIO require all Lube Oil Kings re-qualify their watchstations by 15 March 2010. Until complete all lube oil analysis will be monitored by the ISIC. 3. SAN ANTONIO add roving watches to log and monitor plant operations until ECS and ICAS operation has returned to normal operation. 4. Revise SAN ANTONIO Engineering Department training program to conform with the Surface Force Training Manual and Ship’s Instruction by 15 March 2010. 5. Temporarily increase LPD 17 ship’s force manning/training for ECS/ICAS/SAAMS/TSTS, until off-ship subject matter expert support of propulsion plant systems is in place to support ICAS. 6. ISIC audit and correct SAN ANTONIO Engineering Department preventive maintenance system deficiencies by 15 March 2010. 7. LSD/LPD CLASSRON audit engineering watch station PQS for completeness and adequacy for the LPD 17 Class with recommendations for changes to the Chief Readiness Officer (CRO) by 15 March 2010. 8. Recommend scheduling of a diesel engine lube oil flush completion only when all repairs and maintenance have been completed. 9. LSD/LPD CLASSRON audit all ECS and ICAS alarm set points against EOSS/Tech Manual Guidance/Class Advisories by 15 March 2010. 10. LPD CLASSRON standardize guidance on watchstation manning. 11. LSD/LPD CLASSRON in coordination with the Afloat Training Group develop a LPD 17 master lesson topic guide list that includes standandardized lesson plans.



60

12. Revise the Surface Force Training Manual providing TYCOM guidance on long/short term training schedules; lesson plan development, authorization and retention; use of subject matter experts (especially Chief Petty Officers) for divisional training; monitoring requirements; etc. that uses SUBLANT Joint Training Manual and the AIRLANT Reactor Training Manual requirements as guidelines. 13. Increase the number of DEIs commensurate to support additional diesel plant platforms operation and maintenance. 14. NAVY QA Representative and Diesel Engine Inspector be present during testing for verification prior to delivery to the Ship Builder to create a build sheet of the engine assembly process providing pertinent data will provided for engine material history (web deflections, bearing presses, thrust, saddle torque pressure, main bearing cap pressure, fuel nozzle data are examples). 15. Schedule Certified Diesel Inspectors to inspect diesel engines and engine support systems prior to the Shipbuilder Engine Light-off and again prior to Ship Delivery in accordance with COMFLTFORCOMIST 4790.3 REV-A Change-6 3.3.10. 16. NAVSEA and the LSD/LPD CLASSRON conduct a thorough review/validation of all LPD-17 CLASS EOSS (EOCC, EOP) procedures to include recent class feedback reports, class advisories, etc. by 1 May 2010. 17. NAVSEA S9233-DM-HBK-010-007 Diesel Engine Maintenance Standard Flush LPD 17 be rewritten to NSTM Chapter 233 requirements prior to the next MPDE lube oil flush onboard a LPD 17 Class ship. 18. NAVSEA 04Y research technical documents related to diesel engine maintenance and repair for any existing foreign material exclusion requirements; where requirements are found take action to insure that contracted maintenance adheres to these requirements. 19. NAVSEA 04Y reiterate Project Execution Maintenance Team roles and responsibilities to Project Manager, Port Engineer, Ship Building Specialist, Quality Assurance Specialist (QAS), and Diesel Engine Inspector personnel. Emphasis should be placed on oversight, checkpoints, and product verification



61

inspection of diesel engine maintenance and repair. Additionally, determine requisite knowledge required of SBS/QAS personnel to provide effective oversight of contracted diesel engine maintenance. 20. Improve the training and quantity of RMC ECS/ICAS technicians. 21. Develop fleet concentration area training for LPD17 class to cover ECS/ICAS training and other key areas. 22. Review manning of LPD 17 class to include possible development of a new (integrated systems) division to maintain the ECS/ICAS system. 23. Expeditiously complete manning review of LPD 17 class to include possible development of a new (integrated systems) division to maintain the ECS/ICAS system. Direct NAVMAC to consider and account for NTSP self -training premise in the manpower analysis. 24. NAVSEA/Technical Warranty Holder follow-up on ECS/ICAS assessment held in December 2009/January 2009 and provide a clear path for completion of remaining outstanding items of priority by 28 February 2010. 25. NAVSEA collaborate with NETWARCOM and SPAWAR to ensure SAMMS can function as intended on LPD 17 class ships. 26. NAVSEA determine the causation of LPD 17 Class MPDE bearing failures. 27. Stakeholders fund traditional school house and instructor training for LPD 17 critical systems. 28. Improve LPD 17 Class oversight to ensure the maturation of the training program, and ensure rigor is reinstated to the process of ship building. 29. Conduct bottom up, top down review of SUPSHIP Gulf Coast quality control process, and implementation of necessary changes to increase rigor and develop a product that supports fleet operations upon delivery.
