order 3140.20b - flight standards information system …fsims.faa.gov/wdocs/orders/3140_20.doc ·...

U.S. DEPARTMENT OF TRANSPORTATIONFEDERAL AVIATION ADMINISTRATION

National PolicyORDER

3140.20B

Effective Date:1/6/12

SUBJ: Flight Standards Service National Training Program

This order establishes procedures and processes for the Flight Standards Service (AFS) National Training Program and defines the roles of organizations and individuals responsible for administering, managing, and supporting the program. It includes two primary appendices, the AFS On-the-Job Training (OJT) Program, and the Course Mentor Program. The OJT Program appendix describes processes and tools for planning, delivering, and evaluating the AFS OJT Program. The Course Mentor Program provides course mentors with the basic tools and reference information to help them make AFS training courses effective.

ORIGINAL SIGNED BY/s/ John W. McGraw for

John M. Allen,Director, Flight Standards Service

Distribution: Electronic Only Initiated By: AFS-500

[THIS PAGE INTENTIONALLY LEFT BLANK]

1/6/12 3140.20B

Table of Contents

Paragraph PageChapter 1. General Information..............................................................................................1-11. Purpose of This Order.......................................................................................................1-12. Audience...........................................................................................................................1-13. Where You Can Find This Order.....................................................................................1-14. Cancellation......................................................................................................................1-15. Explanation of Changes....................................................................................................1-16. Scope................................................................................................................................1-17. Training Program Administration....................................................................................1-28. Training Development and Revision................................................................................1-79. Call for Training...............................................................................................................1-910. Flight Standards Master Plan for Training.....................................................................1-1211. Evaluation.......................................................................................................................1-12

Chapter 2. Administrative Information........................................................................................2-11. Distribution.......................................................................................................................2-12. Background.......................................................................................................................2-13. Related Publications.........................................................................................................2-14. Acronyms and Abbreviations...........................................................................................2-2

Chapter 3. Terms and Definitions.................................................................................................2-1

Appendix A. On-the-Job Training Program (36 pages)......................................................A-1Appendix B. Course Mentor Program (26 pages)...............................................................B-1Appendix C. Quick Start Guide to Using the FAA Post Course Evaluation System

Web Site (12 pages).......................................................................................C-1Appendix D. Course Mentor Annual Course Status Report Format (2 page).....................D-1Appendix E. Flight Standards Service Observer Checklist for

Subject Matter Experts (4 pages)...................................................................E-1Appendix F. End-of-Course Evaluation (2 page)................................................................F-1Directive Feedback Information

iiiUNCONTROLLED COPY WHEN DOWNLOADED

Check with FSIMS to verify current version before using

1/6/12 3140.20B

List of Figures

Figure Page

A-1. Phase 1, Planning.........................................................................................................A-14A-2. OJT PTRS Tracking Decision Aid..............................................................................A-16A-3. Phase 2, Delivery.........................................................................................................A-21A-4. Phase 3, Evaluation.....................................................................................................A-31A-5. OJT Program Evaluation.............................................................................................A-34A-6. Conducting and Validating OJT..................................................................................A-24A-7. Conducting and Validating OJT—Level I..................................................................A-25A-8. Conducting and Validating OJT—Level II.................................................................A-26A-9. Conducting and Validating OJT—Level III................................................................A-28B-1. How to Review and Process New or Revised Directives............................................B-10B-2. Monitoring/Revising Inspector Training.....................................................................B-18C-1. Login Page.....................................................................................................................C-2C-2. The Post Course Evaluation System Home Page..........................................................C-3C-3. Editor and Viewer..........................................................................................................C-3C-4. Question Viewer Screen................................................................................................C-3C-5. Courses View.................................................................................................................C-4C-6. Course Profile Information............................................................................................C-4C-7. Reports View.................................................................................................................C-5C-8. Survey Reports View.....................................................................................................C-5C-9. Survey Analysis View....................................................................................................C-6C-10. Comments Report View...............................................................................................C-6C-11. Comments List Screen.................................................................................................C-7C-12. Participant’s Evaluation Response...............................................................................C-8C-13. Notifications Report.....................................................................................................C-8C-14. Report Creator View....................................................................................................C-9C-15. Class Comparison Screen............................................................................................C-9C-16. Class Comparison Data................................................................................................C-9C-17. Search Function Screen..............................................................................................C-10C-18. Change Password Screen...........................................................................................C-10C-19. Sample E-mail—Student...........................................................................................C-11C-20. Student Questions......................................................................................................C-11C-21. Sample E-mail—FLM...............................................................................................C-12C-22. FLM Questions..........................................................................................................C-12E-1. Flight Standards Service Observer Checklist for Subject Matter Experts.....................E-3F-1. End-of-Course Evaluation..............................................................................................F-1

ivUNCONTROLLED COPY WHEN DOWNLOADED


1/6/12 3140.20B

Chapter 1. General Information

1. Purpose of This Order. This order establishes procedures and processes for the Flight Standards Service (AFS) National Training Program and defines the roles of organizations and individuals responsible for administering, managing, and supporting the program.

2. Audience. This order applies to Flight Standards course sponsors, course mentors (CM), stakeholders, regional training coordinators, managers, and employees.

3. Where You Can Find This Order. You can find this order on the MyFAA Web site at https://employees.faa.gov/tools_resources/orders_notices/. Inspectors can access this order through the Flight Standards Information Management System (FSIMS) at http://fsims.avs.faa.gov. Operators and the public may find this information at: http://fsims.faa.gov.

4. Cancellation. This order cancels Federal Aviation Administration (FAA) Order 3140.20A CHG 1, Flight Standards Service National Training Program, dated December 2, 2009.

5. Explanation of Changes. This order does the following:

Note: Due to the dynamics of Web site URLs, the term “Web site” as used in this order refers to the established AFS-500 sites: Home Web site, Quality Management System (QMS) site, and/or SharePoint site(s).

a. Revises guidance and incorporates additional training program information.

b. Identifies the role of the Curriculum Oversight Steering Committee (COSC), Curriculum Oversight Board (COB), and the Curriculum Oversight Teams (COT), in providing direction for the AFS training program.

c. Identifies the individual responsibilities of course sponsors, course mentors (CM), and stakeholders and defines their interaction;

d. Includes updated information on the On-the-Job Training (OJT) Program and Course Mentor Program.

e. Updates the definitions and appendices.

6. Scope. This order pertains to all training and personnel development activities that AFS designs, sponsors, funds, or conducts. Training and personnel development activities encompass all products designed to enhance the knowledge, skills, and abilities that employees need to accomplish the AFS mission. Those formal courses or information dissemination products include, but are not limited to, formal classroom training, out-of-agency training (OAT), computer-based instruction (CBI), web-based training (WBT), interactive video teletraining (IVT), OJT, briefings, videotapes, and correspondence courses. Orientation, coaching, job aids, and performance support are also included.

1-1UNCONTROLLED COPY WHEN DOWNLOADED


1/6/12 3140.20B

7. Training Program Administration. AFS training is developed and administered through a collaborative process that links AFS regulatory responsibilities, directive guidance, job tasks, and competencies to workforce training. Training program administration includes the identification of training requirements, training development, training delivery, training evaluation, and training revision. AFS will ensure that all employees have complete training in the job tasks, skills, and knowledge needed to accomplish the AFS mission. The Flight Standards Training Division (AFS-500), will oversee and manage a single, integrated program that ensures the delivery of valid, systematic, and cost-effective training. AFS-500 will work closely with the COSC, COB, COTs, sponsors, CMs, stakeholders, and training providers to ensure that the training program meets the needs of the AFS workforce.

a. Roles and Responsibilities.

(1) Director of Flight Standards Service (AFS-1). AFS-1 is responsible for preparing the workforce to accomplish the AFS mission. AFS-1 supports special programs such as the Executive Leadership Program, Federal Executive Institute, and Executive Potential Program. Under AFS-1’s leadership, both the organization and individuals have integral roles in the administration and implementation of the National Training Program.

(2) COSC. The COSC provides corporate leadership on training strategies and policies that affect AFS’ overall training program and oversee the work of individual COTs. AFS-1 charters the COSC, and AFS-2 chairs it. Additionally, the COSC:

(a) Sets the strategic direction of the AFS training program to ensure training is supportive of AFS strategic goals and business requirements and is consistent with the Department of Transportation/FAA Human Capital Plan.

(b) Approves COT-defined curricula and ensures the curricula are aligned with COSC direction.

(c) Charter COTs, including identifying the lead and members of each COT and identifying COT objectives and tasking. AFS-500, in coordination with AFS-2P or AFS-2F, facilitates the chartering of COTs.

(d) Charters additional subcommittees and working groups as needed to accomplish the above tasks.

(3) COB. The COB manages the work of the various COTs, oversees the training courses, and training competencies that affect the target populations of multiple COTs. The COB provides input on larger training strategies and policies that affect the AFS overall training program. The COB oversees components of training curricula that impact multiple target populations, to include:

o Defining competencies and training requirements that impact several curricula, ando Developing plans and strategies for meeting the training requirements that affect multiple criteria.



1/6/12 3140.20B

(4) COT. The COT manages the AFS training curriculum associated with a specific occupational specialty such as the air carrier/Airworthiness aviation safety inspector (ASI). The COT oversees the common curriculum requirements that affect multiple courses in a specific curriculum and ensures that the COSC understands the needs of this curriculum. Representatives of the appropriate policy division, region, field office, or national subject matter experts may populate COTs. The COT also has the authority to solicit the support necessary to accomplish the objectives of the COT from the division managers who are sponsors or stakeholders of courses within this curriculum and AFS-500. The COT has the responsibility for providing the strategic direction for managing their curriculum. The sponsoring managers are responsible for providing the direction to their COT and CMs. In addition, the COTs:

(a) Define the training that falls within their specialty and receive approval for the associated curriculum from the COSC.

(b) Define job tasks, technical competencies, and training requirements for the target population in line with the AFS strategic goals and objectives.

(c) Define overall strategies and plans for ensuring the target population’s training requirements are met.

(d) Report to the COSC on training or other human capital management issues related to the curriculum.

(e) Identify the need for developing new training within the curriculum for the target population and identify the sponsoring organization and any stakeholders.

(f) Identify the need for revising the curriculum and coordinating needed changes with sponsoring managers.

(g) Monitor training evaluation reports provided by the Quality Assurance Branch (AFS-530) for training in the curriculum for technical accuracy and currency.

(h) Provide direction on training within the curriculum to the sponsoring managers as necessary and ensure that sponsoring managers and their CMs are coordinating effectively with their stakeholders.

(i) Monitor the AFS training profiles for the curriculum and collaborate with AFS-500 and the sponsoring managers to ensure the profiles reflect the needs of the curriculum.

(j) Charter workgroups as necessary to help the COT in completing its assigned tasks and to perform specialized projects in support of the COT.

(k) Review and sign off on all training development requests (TDR) within their curriculum.

(5) AFS-500. The current edition of Order FS 1100.1, Flight Standards Services Organizational Handbook, defines AFS-500 division and branch responsibilities.



1/6/12 3140.20B

(6) Headquarters, Field Divisions, and Staff Offices. AFS headquarters divisions oversee policy in specific regulatory and operational areas. They develop guidance and define job task performance associated with those areas. Divisions and staff offices may be both sponsors for courses associated with their regulatory and operational areas of responsibility as well as stakeholders in related regulatory areas. Divisions and staff offices:

(a) Provide required support for courses they sponsor.

(b) Initiate new course development and request major course revisions of existing courses through AFS-500.

(c) Initiate new profile development and request revisions of existing profiles through AFS-500.

(d) Validate job tasks and competencies, and maintain job tasks in FSIMS.

(e) Support the curriculum development and revision processes by appointing CMs and providing subject matter experts for the course design, development, and revision.

(f) Use CMs to work with AFS-500 and training providers to ensure that course content is consistent with regulatory policy/guidance and participate in course evaluations to verify course materials meet job performance requirements.

(g) The sponsoring manager, with AFS-500 management, resolves any development issues upon which the workgroup cannot agree. If the Sponsoring manager and AFS-500 management cannot agree, the issue can be raised to AFS-1, 2P or 2F for resolution. (See Appendix B, Course Mentor Program Guide, Chapter 5, New Course Development.)"

(h) Identify, proactively, potential future training requirements to facilitate effective service resource planning and programming.

(i) Identify division training requirements for technical, leadership, management, automation, and general training through the call for training process.

(j) Participate as a member of the COSC and the appropriate COT.

(k) Participate in quarterly program reviews with AFS-500 and the Regulatory Standards Division, AMA-200.

(7) CM. The CM is the individual responsible for monitoring the technical accuracy of one or more of the sponsor’s courses. The sponsoring manager appoints the CM. With the exception of signing a TDR or Course Revision Request (CRR), the CM has the authority to represent the sponsor for all course-related issues to ensure that the technical content of an assigned course meets the needs of AFS. The CM is responsible for working with AFS-500 and the appropriate training provider to ensure that each assigned course’s content is correct, current, and meets AFS’ and the sponsor’s program needs. For more information, refer to appendix B.



1/6/12 3140.20B

(8) Managers. Managers at all levels of the service identify employees’ training needs through the Call for Training process. They guide employees in prioritizing appropriate courses, adjusting work assignments to ensure course attendance, and facilitating employees’ application of newly learned knowledge and job skills. In addition, managers:

(a) Implement AFS training policies.

(b) Determine when additional training is needed and provide applicable justification.

(c) Notify training coordinators about any enrollment changes or cancellations as soon as possible.

(d) Ensure employees meet all training prerequisites.

(e) Communicate training requirements to each employee and, when asked, assist with the development of an individual development plan (IDP).

(f) Encourage employees to participate in self-development activities.

(g) Provide subject matter experts (SME) for training program workgroups.

(h) Support training program evaluations through feedback to the CMs.

(i) Administer the OJT Program in accordance with national guidance and ensure employees accomplish assigned OJT tasks, as required.

(9) Employees. Employees work with their managers to identify training needs and relate those needs to job requirements and career development goals. Employees provide critical information on how training relates to their job tasks. In addition, employees:

(a) Actively participate in training and complete end-of-course and post course evaluations.

(b) During training, adhere to FAA standards of conduct, Aviation Safety (AVS) Consistency and Standardization Initiative guidelines, and AFS professionalism guidelines.

(c) Apply the knowledge and skills gained in training to the job.

(d) Develop IDPs that identify career goals and outline the formal training, developmental activities, and assignments necessary to achieve those goals.

(e) Participate in self-development activities such as details, special projects, and volunteer programs.

(f) Serve as SMEs for training program workgroups and/or as course instructors, when selected.



1/6/12 3140.20B

(10) Regional Training Coordinators. Regional training coordinators manage and administer training in their respective regions and FAA headquarters. They work closely with AFS-500, training providers, and the managers and employees in their regions. In addition, training coordinators:

(a) Assist AFS-500 in implementing training policy and procedures.

(b) Advise managers and employees on training processes and ensure the region’s compliance with training requirements.

(c) Maintain regional training resources, e.g., training videos, lending library, and provide information to employees.

(d) Administer regional training quotas and ensure timely enrollments.

(e) Support the region’s training budget processes.

(f) Coordinate and distribute locally arranged and conducted training (LACT) funds and, if necessary, assist offices in procuring training.

(g) Review and validate the region’s annual training requests.

(h) Coordinate distance learning training delivery for their region.

(i) Participate in training evaluations at the local level for LACT.

(j) Process Memoranda of Understanding (MOU) and equivalent training requests.

(k) Process pop-up training requests with the regions’ representatives and AFS-500.

b. OJT Program. Appendix A, Flight Standards OJT Program is the official guidance for the OJT Program. The following summarizes key responsibilities:

(1) Regional Division Managers. Regional division managers are responsible for ensuring that the Flight Standards OJT Program is implemented efficiently and effectively.

(2) Field Office Managers. Field office managers are responsible for providing work environment orientation and the OJT Program information to all inspectors in their office’s jurisdiction. They are also responsible for ensuring that the OJT program manager has established the OJT Program in the field office in accordance with national AFS guidelines.

(3) Frontline Managers (FLM). FLMs are responsible for implementing the provisions of the OJT Program training requirements for their employees. They review the trainee’s progress on a regular basis and initiate corrective action if necessary. When there are OJT problems and/or disagreements between OJT trainers and trainees, management will serve as mediators.

(4) OJT Program Managers. OJT program managers are responsible for implementing and managing the OJT Program in their field offices. Additionally, OJT program managers are



1/6/12 3140.20B

responsible for ensuring that OJT instruction is provided to trainees in accordance with applicable directives, appendix A, and the trainee’s OJT plan.

(5) OJT Trainers. OJT trainers are responsible for ensuring that trainees satisfactorily accomplish all elements of their OJT plan. OJT trainers provide instruction on specific tasks at levels I, II, and III, in accordance with established procedures which reflect national policies and practices. OJT trainers will notify the OJT program manager and the trainee’s FLM when all required training has been completed and the trainee is able to satisfactorily perform the required task(s).

(6) Trainees. Trainees are responsible for working with their FLMs to identify developmental needs and plan training activities. They are also responsible for fulfilling their OJT requirements according to their OJT plan within the established timeframes, constructively participating in their training progress reviews, and providing feedback to help ensure continuous improvement.

8. Training Development and Revision. AFS-500 is responsible for overseeing all phases of course development/revision and ensuring that proposed training is linked to employee job performance before course development begins. Training course sponsors and stakeholders have integral, interdependent roles in the course development/revision process. The sponsor’s role is equivalent to that of an office of principal interest. In some cases, the sponsoring policy division and the stakeholders identified may have common interests and expectations for the course, while in other instances there might be differing interests between the sponsors and stakeholders. The CM, as the sponsor’s representative, plays a critical role in the development/revision process and works with AFS-500 to ensure that the performance objectives and other course materials actually meet the training needs of sponsors and stakeholders. The CM has a leadership role in coordinating the needs of sponsors and any course stakeholders with AFS-500 in the development/revision process. The CM is the primary focal point for decisions regarding a course’s technical content. New training development or revision projects are typically initiated by the sponsoring organization with a properly-signed TDR or CRR submitted to AFS-500. A TDR is used to request new course development. A CRR is used to request a major revision to an existing course. Requests include a description of the need and job performance requirement, audience identification, and a description of how the requested training relates to the target audience.

a. Statutory and Regulatory Requirements. Congressional training regulations establish requirements and limitations on the funding of employee training. Public Law 109-115 requires that training developers use an instructional systems design process, relate training to job functions, and base the course design on a needs assessment. This public law also requires that course participants receive advance notice of training content and methodologies before training begins, and that they complete end-of-course written evaluations. In accordance with legislative requirements, Federal employee training and educational programs must provide employees the opportunity to perform at their highest potential and advance in their careers. The current edition of FAA Order 3200.9, Federal Aviation Personnel Manual, and the FAA Human Resource Policy Manual, delineate responsibilities for FAA employee training and development. Additionally, FAA/employee bargaining unit agreements may contain provisions related to the training of bargaining unit employees.



1/6/12 3140.20B

b. Requests for New Course Development. Any regional division manager, headquarters division manager, or headquarters staff office manager may request the development of a new course through the submission of a TDR to AFS-500. TDR forms and instructions are available on the AFS-500 Web site on the FAA employees’ intranet.

(1) Regional Requests. AMA-200 and regional divisions may also identify training development needs through a TDR. The training may be critical to the originating region and/or have national implications. The regional division manager submits the completed TDR to AFS-500. AFS-500 will work to determine the appropriate training sponsor and coordinate the TDR with the sponsor for review and approval. If it is not clear which organization should sponsor the training, AFS-500 will discuss the sponsorship with AFS-1/AFS-2.

(2) Headquarters TDR Development and Submission. The headquarters division/staff that develops and submits the TDR must complete the TDR in accordance with the instructions posted on the AFS-500 Web site. If other organizations have an interest/stake in the training described in the TDR, the developer of the TDR should contact them and coordinate their input during the development of the TDR. All TDRs must be signed by the originating organization’s manager and sent to AFS-500 for coordination. The completed TDR form:

(a) Identifies the training need and how this need will contribute to the AFS mission and goals.

(b) Describes the job performance requirement.

(c) Identifies the target audience.

(d) Describes how the training relates to the audience.

(e) Identifies the CM, stakeholders, and their points of contact.

(f) Describes the impact if the training is not developed.

(3) TDR Coordination. Upon receipt of a TDR, AFS-500 coordinates with the sponsor’s CM to gather any additional information needed to address the request. AFS-500 provides a copy of the TDR to the appropriate COT for concurrence.

(4) Prioritization. The AFS-500 project manager, using input from the assigned CM, prioritizes training development projects and sets a start date in agreement with the CM. If the project manager and CM are unable to agree on an appropriate start date, the issue is forwarded to the AFS-500 branch manager for resolution. If the AFS-500 branch manager is unable to resolve the issue with the CM, the matter is forwarded to AFS-500 division management for resolution with the course sponsor.

c. Requests for Course Revisions. The course sponsor, AFS-500, or the training provider may request a major course revision by completing and submitting a CRR form, available on the AFS-500 Web site. Other people who identify a need for course revision may request a revision by submitting a draft CRR to the CM. If the CM concurs with the draft CRR, the CM must



1/6/12 3140.20B

forward the draft CRR to the sponsoring division manager for approval. The approved and signed CRR will be sent to AFS-500 for processing. If the person submitting a request believes the revision to be minor as defined in International Organization for Standardization (ISO) Process AFS-500-004, Revise Existing Curriculum, the request may be submitted to the CM by way of e-mail with an explanation of the proposed revisions. Minor revisions may also be submitted to the CM by furnishing an annotated copy of the affected lesson plans. Proposals for minor changes will be sent to the CM with a copy to the responsible AFS-500 branch. Major and minor revisions are made in accordance with the approved course revision process, in ISO Process AFS-500-004.

d. Instructional Systems Design (ISD). AFS-500 uses the ISD process to develop training. The ISD process encompasses five phases: analysis, design, development, implementation, and evaluation. CMs and stakeholders provide technical input through each phase of the process and ensure the training is consistent with regulations, policies, and applicable job tasks.

(1) Analysis Phase. During the analysis phase the sponsor analyzes the training need and job performance requirements. The CM, appointed development workgroup members, as appropriate, and AFS-500 define training content by identifying the tasks, skills, and knowledge required to perform the job function. The CM approves the Task Analysis Report.

(2) Design Phase. During the design phase, AFS-500 conducts the necessary research and develops a Course Design Guide (CDG) that is used as a blueprint in developing course materials. The CDG defines the course and supporting objectives, sequence of topics, learning strategies, learning activities (practice exercises, case studies, etc.), and methods of evaluation. The CM approves the CDG.

(3) Development Phase. In the development phase, AFS-500 works with the development workgroup to develop instructional materials, validate the materials, and revise them if necessary. After the training materials are completed, the workgroup conducts a course walk-through. After the walk-through, the CM approves the course to go to prototype with the target audience as students. The next step in development is conducting the prototype.

(4) Implementation Phase. During the implementation phase the training provider completes all of the tasks necessary to conduct the course. In this phase the course instructors are trained and classes are conducted.

(5) Evaluation Phase. The purpose of the evaluation phase is to determine whether or not the training achieves the desired course objectives. AFS-530 coordinates evaluation activities for AFS. The evaluation phase is mandated by public law and addresses all aspects of the course. Evaluation results are used to revise course material and determine if the training improves job performance.

9. Call for Training. The Call for Training is a process for identifying, requesting, and justifying training requirements. AFS-500 uses the Call for Training process to plan and administer national training requirements.



1/6/12 3140.20B

a. Call for Training Web Site and Guidance. The Web site gives managers and employees access to extensive training information. The Call for Training guidance, available on the AFS-500 Web site, provides guidance for identifying training needs. It explains the process field offices and regions must use to request training.

b. Identifying Training Requirements. Offices, regions, and headquarters divisions will identify formal and locally arranged training requirements for technical, leadership, management, automation, and general training through the Call for Training process on an annual basis. Newly appointed FLMs should complete an assessment of their individual training needs and develop a personal 12-month learning plan within 30 days of assuming a management position. (Refer to MSC 10.4, FAA Managerial Workforce Planning.)

c. Formal Flight Training Requirements. Inspectors must meet FAA flight training requirements in order to perform job functions in aircraft and simulators. Training criteria are contained in FAA Order 8900.1, Flight Standards Information Management System (FSIMS). The following paragraphs explain the general policy on flight and systems training.

(1) Normally, no more than one flight course in category every 24 months is allowed for general aviation operations inspectors conducting certification duties in aircraft that do not require a type rating by type design.

(2) Normally, no more than one flight course in category every 12 months is allowed for general aviation and/or air carrier operations inspectors that meet the threshold table in aircraft that require a type rating by type design.

(3) Normally, no more then one systems course in any fiscal year is allowed for maintenance and avionics inspectors for aircraft that meet the threshold table. Unlike flight courses, there is no 12- or 24-month regulatory requirement for systems courses. In addition, once the office has the skill, a different person should not be submitted for that training in subsequent years.

(4) No flight courses are allowed for regional or headquarters staff unless they:

(a) Serve as a national or regional resource identified through the Call for Training process.

(b) Serve as an internal FAA check pilot identified through the FAA Aircraft Management Program.

(c) Conduct certification checks for the Light-Sport Aviation Branch (AFS-610); or

(d) Conduct simulator testing in the Flight Technologies and Procedures Division (AFS-400) (B-737-800 Initial Qualification only).

d. Formal Flight Training Waivers. The process for requesting flight-training waivers is outlined in the AFS QMS document AFS-500-006, Administer Training Programs, available on the AFS-500 Web site. The required training waiver forms are available as well.



1/6/12 3140.20B

(1) AFS-520, in coordination with the Flight Program Office (AFS-60), may issue a waiver extending formal flight training requirements only if the region has requested the training identified in the waiver, the course is unavailable at the required time, and the inspector has received a confirmed class date for the course.

(2) Turboprop Initial Waiver. AFS-520, in coordination with AFS-60, may issue a waiver only if the regional Flight Standards division (RFSD) manager submits a memorandum request to waive turboprop initial formal training requirement when an inspector has logged at least 200 hours as a pilot in a turbo-propeller-powered airplane within the last 5 years.

Note: Per FAA Order 8900.1, inspectors who hold a type rating in a turboprop-powered airplane do not need to complete turboprop initial formal training; therefore, a waiver is not required.

e. Equivalent Training. Offices requesting certification of equivalent training will complete the Equivalent Training Form located in the forms section of the Call for Training Web site. Offices complete the form online and forward it to the appropriate regional training coordinator.

(1) Criteria. Equivalent training is technical training that an employee receives through sources outside of the FAA training procurement process. Offices can request certification of equivalent training for training courses and/or other coursework. Training coordinators forward the requests to AFS-520.

(2) Approval. AFS-500 approves equivalent training credit for training that is equal to or more comprehensive in scope and content than the FAA course for which credit is requested. Additionally, AFS-500 authorizes credit for training that an employee receives through the military or other organizations, even if there is no equivalent FAA course. AFS-520 will advise the requesting region how approved equivalent training should be entered in the applicant’s training history.

(3) Formal Flight Training. AFS-520, in coordination with AFS-60, approves equivalent training credit for training that is equal to or more comprehensive in scope and content than the FAA course for which credit is requested.

f. MOU Training. Offices must document the training employees receive under the provisions of an MOU with the Memorandum of Understanding Certification Form located in the forms section of the Call for Training Web site. Offices complete the form online and forward it to the appropriate regional training coordinator.

(1) Criteria. MOU training is training that an Aircrew Program Manager, Training Center Program Manager, Advance Qualification Program Manager, or Flight Standardization Board member receives under the provisions of an approved MOU.

(2) Approval. The regional division manager approves MOU training. When the region approves MOU training, the training coordinator requests that the training be entered in the applicant’s permanent training history.



1/6/12 3140.20B

10. Flight Standards Master Plan for Training. The Flight Standards Master Plan for Training states the strategic vision, goals, and indicators of success for the national training program in accordance with the AFS mission.

11. Evaluation. AFS-500 ensures that all AFS training is evaluated in accordance with Public Law 105-61, Treasury and General Government Appropriates Act of 1998, section 622, and policy delineated in the Human Resource Policy Manual. Evaluation measures training’s current results, identifies areas for improvement to increase training’s future effectiveness, and enables AFS to determine if training has benefited the organization. AFS-500 coordinates course evaluations with the appropriate CM.

a. End-of-Course Evaluations. Students should complete an end-of-course evaluations that rate their reactions to:

Course content, Organization and sequence of the course, Clarity of objectives, and Instructor/vendor effectiveness.

b. Quality Assurance.

(1) AFS-530 uses appropriate evaluation techniques and tools to evaluate AFS training courses and the processes that impact the national training program. AFS-530 works with CMs to determine when course evaluations need to be conducted. Training course evaluations measure:

o Participant reactions,o Learning outcomes,o Transfer of learning, ando Organizational results.

(2) One of the tools available to the CMs in determining the overall health of a course is the Web-based Post Course Evaluation System. The Level III evaluation data stored in this system provide the extent to which the training is transferred to the job.

c. Feedback. AFS-500 and the CMs assess the perceived and measurable results of the training program in terms of performance improvement, quality improvement, and cost/benefit to the organization. This is accomplished through feedback from course sponsors, CMs, and students, as well as continuous evaluation of training provided. AFS-500 keeps CMs informed of any feedback received on their courses and involves them in formal course evaluations.



1/6/12 3140.20B

Chapter 2. Administrative Information

1. Distribution. We will distribute this order to all Flight Standards Service (AFS) employees in Washington headquarters and regions and to all AFS field offices. Distribution is to the branch level in Washington headquarters, the regional Human Resource Management divisions, and the Regulatory Standards Division at the Mike Monroney Aeronautical Center. Inspectors can access this order through FSIMS.

2. Background. AFS strategic goals include the implementation of a Quality Management System (QMS). The initiative requires AFS to clarify responsibilities, identify products, and define processes. This order aligns applicable guidance with AFS National Training Program processes and provides cohesive information on procedures and requirements.

3. Related Publications. These documents provide complementary information to the guidance in this order and are available on the AFS-500 Web site on the FAA employee intranet.

Note: Due to the dynamics of Web site URLs, the term “Web site” as used in this order refers to the established AFS-500 sites: Home Web site, QMS site, and/or SharePoint site(s).

a. Flight Standards Master Plan for Training 2010–2014. States the strategic vision, goals, and indicators of success for the Flight Standards National Training Program for Fiscal Years 2010–2014.

b. Call for Training Web Site and Guidance. The Web site gives managers and employees access to extensive training information. The Call for Training guidance, available on the AFS-500 Web site, provides guidance for identifying training needs. It explains the process field offices and regions must use to request training.

c. AFS-500-00a, Organization Chart. Illustrates the organizational structure of the Flight Standards Training Division, AFS-500.

d. AFS-500-00b, Roles & Responsibilities. Defines roles and responsibilities of the AFS-500 branches and personnel.

e. AFS-500-001, Evaluate Training for Revision. Describes the processes involved in evaluating training courses.

f. AFS-500-003, Develop Curriculum. Defines the processes and procedures to develop training curriculum, from the identification of training needs to completing of the training development project.

g. AFS-500-004, Revise Existing Curriculum. Defines the processes and procedures to revise existing training curriculum.

h. AFS-500-005, Plan Training Program. Describes the procedure and processes to identify requirements and plan for training of Flight Standards personnel.

1UNCONTROLLED COPY WHEN DOWNLOADED


1/6/12 3140.20B

i. AFS-500-006, Administer Training Program. Describes the procedure and processes to implement and administer training for Flight Standards personnel.

j. AFS-500-007, Production of Briefings. Describes the procedure and processes involved in the production of briefings for Flight Standards Service.

4. Acronyms and Abbreviations.

a. CBI. Computer-Based Instruction.

b. CDG. Course Design Guide.

c. CM. Course Mentor.

d. COB. Curriculum Oversight Board.

e. COSC. Curriculum Oversight Steering Committee.

f. COT. Curriculum Oversight Teams.

g. COTR. Contracting Officers Technical Representative.

h. CRR. Course Revision Request.

i. FLM. Frontline Manager.

j. FSDO. Flight Standards District Office.

k. IDP. Individual Development Plan.

l. ISD. Instructional Systems Design.

m.IVT. Interactive Video Training.

n. JTA. Job Task Analysis.

o. LACT. Locally Arranged and Conducted Training.

p. M&G. Management and General.

q. MOU. Memorandum of Understanding.

r. NPR. New Profile Request.

s. OAT. Out-of-Agency Training.

t. OJT. On-the-Job Training.

u. QPR. Quarterly Program Review.

v. SME. Subject Matter Expert.



1/6/12 3140.20B

w. TDR. Training Development Request.

x. WBT. Web-Based Training.



1/6/12 3140.20B

Chapter 3. Terms and Definitions.

a. Call for Training. The Call for Training is the process for identifying, requesting, and justifying AFS training requirements.

b. Competencies. Competencies are a cluster of related knowledge, skills, and attitudes that affect a major part of a job (such as a duty or responsibility) that can be improved with training and development. Competencies are described by associated behaviors based on documented job tasks.

c. Contracting Officers Technical Representative (COTR). The COTR is responsible for overseeing the development and delivery of contract tasks by an outside entity providing products and services to AFS. The COTR represents the FAA’s contracting officer in dealing with contract training providers on technical issues. Coordinate through the COTR all communications or correspondence with training contractors.

d. Course Design Guide (CDG). The CDG defines the course and supporting objectives, sequence of topics, learning strategies, learning activities, and methods of evaluation. It is used as a blueprint when developing course materials.

e. Course Manager. The course manager is responsible for the administration and oversight of course delivery. The specific functions include recruiting/assigning instructors, arranging for classrooms, participating in the scheduling of training, ensuring that instructional materials are duplicated and available in sufficient quantities ensuring that the course runs smoothly and that needed resources are provided.

f. Course Mentor (CM). The CM, appointed by the sponsoring manager, is responsible for monitoring the quality and content of one or more of the sponsor’s courses. The CM has the authority to represent the sponsor for all course-related issues to ensure that the technical content of an assigned course meets the needs of AFS. The CM is responsible for working with AFS-500 and the appropriate training provider to ensure that each assigned course’s content is correct, current, and meets AFS’ and the sponsor’s program needs. The CM does not sign training development requests (TDR) or course revision requests (CRR), nor does he or she formally approve major revisions or new TDRs. When required, the CM ensures that the stakeholders’ interests are represented.

g. Course Monitoring. Course monitoring includes activities performed to ensure that a course is kept current with FAA policy, guidance, regulations, and job functions, and meets the needs of AFS.

h. Course Revision Request (CRR). A CRR is a form submitted by an individual requesting major revisions to an existing course.

i. Curriculum Oversight Board (COB). The COB manages the work of the various Curriculum Oversight Teams (COT), oversees the training courses or training competencies that impact the target populations of multiple COTs, and provides corporate leadership on larger training strategies and policies that impact the AFS overall training program.



1/6/12 3140.20B

j. Curriculum Oversight Steering Committee (COSC). Chartered by AFS-1, the COSC sets the strategic direction necessary to best meet the training needs and coordinates common training needs of the Flight Standards workforce.

k. Curriculum Oversight Teams (COT). Provide strategic direction to the training and development of specific occupational groups within the AFS workforce.

l. Distance Learning. For the purposes of this order, distance learning means learning that is accomplished by any training method not including an instructor and a gathering of trainees colocated in a traditional classroom. Examples include computer-based instruction (CBI), Web-based training (WBT), interactive video teletraining (IVT), correspondence courses, videotapes, etc.

m.Electronic Learning Management System (eLMS). A Department of Transportation (DOT)-wide system used for managing employee learning and training records.

n. End-of-Course Student Evaluation. Students provide feedback on the training at the end of a class.

o. Formal Course. Formal courses are numbered and include flight and non-flight courses. They may be delivered via classroom, Internet, or IVT broadcasts.

p. Formal Course Evaluation. Formal course evaluation is a structured evaluation AFS-530 coordinates involving the training triad. Formal course evaluations are scheduled on a recurring basis (routine) or as requested by the course mentor (special).

q. Frontline Manager (FLM). The FLM provides first-level supervision to subordinate employees and manages the activities of one operating unit, project, or program area. FLMs report to middle or senior managers.

r. Informational/Instructional Briefing. Typically delivered via the Internet or IVT, briefings are either informational (focusing on what the recipient needs to know) or instructional (addressing how the recipient implements and integrates the information into their work).

s. Instructional Systems Design (ISD). ISD is the systematic and reflective process of translating principles of learning and instruction into plans for instructional materials, activities, information resources, and evaluation.

t. Job Task Analysis (JTA). JTA is a procedure for identifying the component parts or tasks comprising a specific job. Tasks identified through job analysis are further analyzed, using a process called task analysis, to include the subtasks or steps comprising each task. A JTA helps define the content of required training by identifying the knowledge, skills, and abilities required to accomplish job task objectives and perform a particular job function.



1/6/12 3140.20B

u. Locally Arranged and Conducted Training (LACT). LACT includes management and general (M&G) courses, technical training, and certain out-of-agency training (OAT) courses. M&G courses may be funded either nationally or locally. AFS-500 initiates distribution of national funding to the regions for LACT M&G and technical LACT and is accountable for the application of funding to training.

v. Major Revisions. A revision may be considered major when one or more of the following conditions occur to an approved training course:

(1) The length of the course increases by more than 4 hours;

(2) The training outcomes and/or instructional objectives are added or deleted;

(3) The design of the course is changed (adding or deleting exercises, simulations, etc.); or

(4) The source or method of training delivery changes (classroom, computer-based instruction, interactive video teletraining, Web-based training, etc.). Refer to Appendix B, Chapter 7.

w. Minor Revisions. A revision may be considered minor when it does not fall into the category of a major revision. Routine revisions are necessary to maintain the currency of the course. Minor revisions should not affect the training outcomes/instructional objectives, course length, or method of course delivery. Refer to Appendix B, Chapter 7.

x. On-the-Job Training (OJT). OJT is a planned, structured training event conducted at a worksite and validated by a designated OJT trainer. This type of training provides direct experience in the work environment in which the employee is performing or will be performing on the job.

y. Operational Tryout. The operational tryout gives each instructor an opportunity to deliver proposed course material to receive feedback, and to assess the timing of the delivery.

z. Out-of-Agency Training (OAT). OAT is training for flight, systems, or other training provided through a contract with a vendor.

aa. Performance-Based Training. Performance-based training teaches job performance and includes hands-on exercises, simulations, scenarios, etc., that are designed to reflect the job tasks.

bb. Pop-Up Training. Pop-up training is out-of-agency or resident training that is needed during the year but was not identified in the annual call for training.

cc. Programming Conference. Both the FAA Academy and the Center for Management and Executive Leadership hold conferences to review projected training requirements for the next fiscal year.



1/6/12 3140.20B

dd. Prototype Class. The first regularly scheduled class delivered in its final developmental form to its intended target audience.

ee. Regional Training Coordinators. Regional training coordinators manage and administer training in their respective regions and FAA headquarters. They work closely with AFS-500, training providers, and the managers and employees in their regions.

ff. Sponsor. The sponsor is the AFS policy organization with overall responsibility for ensuring the technical content of an assigned course meets the needs of AFS. The sponsor approves new Training Development Requests (TDR) and Course Revision Requests (CRR).

gg. Stakeholder. A stakeholder is an organization with a stake in the presentation regarding quality and technical content of any portion of a course. When required, stakeholders will provide the CM with a point of contact.

hh. Storyboard. A storyboard provides a textual and visual description of content, graphics, animation, and other elements for a Web-based training course.

ii. Subject Matter Expert (SME). An SME is any employee of the AFS workforce or outside consultant with the knowledge and experience essential to produce a final product that meets the training needs of the respective audience.

jj. Training Development Request (TDR). The TDR is the formal request for the development of a new AFS training course.

kk. Training Profile. A profile is a prioritized list of courses recommended or required for a unique combination of a specialty and a position.

ll. Training Triad. The members of the training triad are the CM, AFS-500, and the training provider. This triad works together to monitor the currency, effectiveness, and relevance of AFS courses.

mm. Walk-Through. A session in the course development or major revision process in which the course is read aloud to the workgroup in its entirety to facilitate validation of the course.

nn. Web-Based Post Course Evaluation System. The Post Course Evaluation System is an evaluation tool that uses the intranet for gathering data on whether the trainee is applying the completed training on the job and whether the training has improved work performance.



1/6/12 3140.20BAppendix A

Appendix A. On-the-Job Training Program

1. Introduction. On-the-job training (OJT) is planned training a designated trainer conducts at a worksite. This type of training provides direct experience in the work environment in which the employee is, or will be working. OJT is a career-long process to ensure maximum employee development in the dynamic industry of aviation. OJT is never truly completed; it is a journey toward the mastery of knowledge and skill. Though there may be a finite initial task that may be accomplished based on an employee’s assignment, continuing development throughout one’s career requires continuing OJT. Flight Standards Service (AFS) training consists of both formal training and OJT. It is through the accomplishment of both types that AFS ensures a well-trained and qualified workforce. Employees are expected to complete OJT task training requirements pertinent to their duty assignment.

2. Link to Master Plan for Training. Structured OJT is a critical component of an effective performance-based training program. It is a core training process used in every AFS office. An effective OJT program contributes to the vision and goals of the AFS Master Plan for Training. To achieve this vision, AFS-500 established the following four goals:

a. Goal 1: Provide a complete, effective training curriculum that helps employees across all levels and disciplines perform their aviation safety job responsibilities well, and in accordance with the overall policies and direction of AFS, and that develops critical capabilities of stakeholder groups in accordance with AFS business strategy.

b. Goal 2: Establish processes to ensure training is current and well designed that are tailorable to the needs of individual employees, and are administered in a fast and flexible way in response to changing needs.

c. Goal 3: Build and maintain a training infrastructure—including technology, organization, and facilities—that efficiently provide the services and support that managers and employees need to optimize training.

d. Goal 4: Gain support from AFS and other organizational leaders, managers, and employees (e.g., Department of Transportation (DOT), FAA, and Aviation Safety) for the commitments of money, time, and other resources necessary to ensure an effective training program.

3. On-the-Job Training Program. This appendix supersedes all previous versions of the OJT Guide. This appendix describes processes and tools for planning, delivering, and evaluating the AFS OJT Program. The OJT Program, including this appendix, is a dynamic ongoing program that will change to meet the needs of AFS and its employees.

a. Applicability of OJT. The OJT Program in this appendix applies to AFS employees in new knowledge and skills training, requalification training, and training in new programs and tasks. OJT is the vehicle to maintain proficiency as job requirements change. The OJT framework in this appendix is directly applicable to positions that have established job tasks in the Flight Standards Information Management System (FSIMS). This same framework may also be used for those positions that do not have established job tasks in FSIMS.

A-1UNCONTROLLED COPY WHEN DOWNLOADED



b. Value of OJT. The OJT Program adds value to the overall AFS training effort through the following:

(1) Flexibility. The AFS OJT Program is a process for implementation and management of a structured OJT system at the local level using national guidelines. The program can be tailored to any AFS employee requiring training, to include tasks unique to the office.

(2) Timeliness. OJT can be provided immediately when the need or opportunity arises.

(3) Skills Application. By applying knowledge and skills learned, the trainee completes the learning process. At the same time, the local AFS office gains confidence in the trainee’s capabilities and better understands how to utilize the employee on the job.

(4) Cost Effectiveness. OJT is relatively inexpensive compared to classroom training. Primarily, OJT requires an investment of time by the OJT trainers, trainees, their front line managers, and the OJT program manager (PM) in the office.

(5) Locally Managed. OJT empowers managers and employees to develop needed skills. OJT is provided at the local level when it is identified as the best method for delivering the needed training or if no other means to receive the training is available.

(6) Career Broadening. Throughout an employee’s career in AFS, OJT remains a valuable tool for broadening technical skills and capabilities. Cross training in tasks of other disciplines may not be possible through other training means due to resource limitations, but may be more easily attainable through a structured OJT program.

4. OJT Basics in AFS.

a. Background.

(1) In 1985, AFS conducted a complete analysis of the work performed by its inspector workforce. This analysis resulted in a Job Task Listing (JTL), which listed the job tasks performed by inspectors. It also resulted in an analysis of each task to include details such as subtasks, standards, and applicable references.

(2) In 1995, AFS updated and expanded its job task analysis. This job task analysis also resulted in a listing of all tasks performed by inspectors, as well as detailed information related to each task. The 1995 listing was much larger and more detailed than the 1985 listing. For this reason, AFS incorporated the 1995 JTL into its training documentation.

(3) In 2000, AFS-530 used the 1985 and 1995 tasks lists and began working with the policy divisions to conduct a Job Task Analysis (JTA) for each division. Once the JTA work was complete, the 1985 and 1995 lists of job tasks were retired. FSIMS hosts these validated job tasks and the authorized task lists. Each AFS policy division is responsible for keeping their job tasks current.




(4) In 2007, AFS-500 and the OJT Steering Committee responded to the feedback from the field offices and made changes to the OJT Program, e.g., using an Excel spreadsheet as the means to track OJT activity.

(5) In 2009, AFS-500 and the OJT Steering Committee responded to feedback from the field offices and the 2008 OJT Program Evaluation Report by revising this appendix and the OJT tracking system. One recommendation was “to establish a national, automated system to track OJT completions and to monitor the OJT Program.” Two components comprise the official tracking system for the AFS OJT: the Program Tracking and Reporting System (PTRS) for data entry, and the Safety Performance Analysis System (SPAS) to view the data in a standardized format. PTRS was chosen because of its national accessibility as a common database with permanent data retention. SPAS was chosen due to its national accessibility and ability to retrieve and generate reports.

b. OJT Configuration.

(1) Task Structure and Assignment.

(a) This appendix introduces tasks that are categorized first by job specialty (e.g., Air Carrier Operations, General Aviation Airworthiness), then by duty (investigation, surveillance, certification, and general technical), and then by function (e.g., aircraft surveillance, airman certification). This structure emulates that of FSIMS. Only those tasks that a trainee performs in a specific office environment should be included in the individual’s OJT training plan. Management decides the tasks for which an employee needs training based on the employee’s work assignment and local environment needs.

(b) Pursuant to Aviation Safety Inspector Credentials Program, the issuance of Aviation Safety Inspector’s Credentials (110A) is predicated on completion of OJT task training requirements pertinent to the duty assignment of an ASI as determined by the manager or immediate FLM. As stated previously, there is a difference between the completion of a finite set of tasks for initial qualifications, as opposed to the OJT Program as a whole, which is a career-long process. Promotions are not solely predicated on OJT task completions; however, OJT progress may be considered.

(2) Training Structure.

(a) The Training Structure of the Flight Standards OJT Program follows a logical progression of adult learning through three levels:

Level I—Knowledge. Level II—Understanding. Level III—Performance.

(b) Any task assigned to a trainee will be trained to all three levels. Credit may be granted for OJT Levels I and/or II based on completion of training outside of OJT when appropriately validated. Both formal training and OJT are integral parts of a well-developed training program and should be scheduled to complement each other.




(c) The three distinct levels are in no way indicative of how many times a task must be performed to achieve competency. The number of iterations for each task is based on each individual’s demonstration of knowledge/skill in the subject area.

5. Definitions. The following are definitions of terms related to the AFS OJT Program:

a. Certification. The certification work activities validate the competency of an air operator, air agency, or airman and their compliance with appropriate statutory and regulatory requirements before active performance in the aviation industry.

b. General Technical. General Technical are functions trainees perform that do not fit in surveillance, investigations, or certification (e.g., aviation education and promoting aviation safety to all segments of the aviation community).

c. Individual OJT Plan. An individual OJT plan is created from the OJT Office Task Profile. It is a comprehensive list of Job tasks an individual employee is responsible for accomplishing within the office’s environment.

d. Investigations. Investigations are the means by which the FAA determines causal factors of potential or actual problem areas, and is the vehicle to implement appropriate corrective action. These work activities are generated on an “as required” or “as discovered” basis.

e. Level I Training. Level I training is related to that body of knowledge associated with a specific job task. This knowledge is contained in orders, rules, guidance, and standards. Level I training typically involves a review of all reference materials applicable to the job tasks for which training has been identified. Level I training may be satisfied through classroom training or other delivery methods that provide the opportunity for the trainee to achieve all aspects of the task and the objectives of Level I. An OJT trainer or management must validate all Level I training.

f. Level II Training. Level II training usually involves observation of the performance of specific job tasks to achieve a level of understanding. This training typically involves the trainee observing and/or assisting the OJT trainer, or other technical resources, in the performance of those specific job tasks for which the trainee will be held accountable. Level II training may be satisfied through appropriate classroom training that provides the opportunity for the trainee to achieve all aspects of the task and the objectives of Level II. An OJT trainer or management must validate all Level II training.

g. Level III Training. Level III training involves the application of knowledge and skills to the performance of specific job tasks. Typically, the trainee performs the job task under the observation of a qualified OJT trainer. The trainer assesses the performance of the task and indicates on the OJT record when the trainee achieves Level III performance.

h. OJT PM. The OJT PM is the employee designated in writing by the office manager and trained to establish and maintain the OJT Program for the local office. This is a key role in the implementation of the OJT Program. In some offices this can be a significant task for one person to handle. Management can designate additional resources as necessary.




Note: The OJT program is designed and dependent upon the establishment and use of PMs. In some offices the OJT PM may also be the local training program manager (TPM).

i. OJT Steering Committee. The OJT Steering Committee is a group of employees from the field and headquarters who have oversight of the OJT Program. The committee may include office and regional management, OJT PMs, policy division representatives, Professional Aviation Safety Specialists representatives, OJT trainers, and AFS-500 representatives. Members of the steering committee serve for 3 years.

j. OJT Trainer. An OJT trainer is a highly skilled and experienced employee with advanced knowledge, skill, and experience who has been designated in writing by the office manager to provide and/or validate OJT instruction to trainees on specific tasks at Levels I, II, and III, in accordance with established procedures and reflective of national policies and practices.

k. OJT Training Record. PTRS is used to record the trainee’s OJT plan, progress, and task accomplishment.

l. Regional OJT Program Manager. The regional OJT PM is the employee designated and trained to evaluate the OJT program for the region. This is another key role in ensuring compliance with the OJT program guidance. Management can designate additional resources, as necessary.

m.Simulation. Simulation is a method for training inspectors when the “real-life” task situation is not feasible and/or advisable, such as an en route inspection. When a task is simulated, the environment, conditions, equipment, and performance of the task must be as near the “real-life” situation as possible. Simulation is not prohibited, but if the office selects OJT tasks for inspectors that are normally done in that office, there should be minimal simulation required.

n. Surveillance. Surveillance is a tool to provide information for performance assessments and risk management. It is an observation to validate a certificate holder’s active systems to ensure they continue to meet their intended regulatory and safety objectives. Validation is the oversight function that ensures continuing operational safety. Such assessments also validate that a certificate holder’s operating systems produce intended results, which include control of hazards and associated risk.

o. Task. A task is a unit of work that contains logical and necessary steps in the performance of a job duty, typically with a defined beginning and end. The task must produce a meaningful result.

p. Technical Resource. A technical resource is designated in writing by the office manager, with the technical skill and knowledge in particular job task areas and specialized at the advanced level. The office manager must evaluate and designate this resource to provide specific training.




q. Trainee. A trainee is any AFS employee who receives on-the-job training. (Depending on the context, the word “employee” can be synonymous with “trainee” because all employees may receive OJT regardless of any “new-hire” or “trainee” status.)

r. Validate/Validation. The validation process helps determine that an acceptable level of competency has been achieved. The Validation tools are required to be used and can be found in this appendix and at the OJT PM SharePoint site.

6. Roles and Responsibilities. This section describes the roles and responsibilities of those involved in the AFS OJT Program. This information supplements information on roles and responsibilities provided in this order.

a. Trainee. The trainee is responsible for:

(1) Participating with the FLM to identify developmental needs and to plan training activities.

(2) Requesting OJT credit from their FLM for prior training and/or experience for Levels I and/or II. Level III must be accomplished through AFS OJT.

(3) Fulfilling their OJT requirements as established within the office.

(4) Participating in the feedback process to help ensure continual improvement, including feedback on the performance of the trainer.

(5) Participating, in a constructive manner, in his or her training progress reviews and checking the accuracy of completed tasks during the review meetings.

(6) Recording time and activities spent on OJT in appropriate tracking program(s), e.g., Labor Distribution Reporting (LDR), PTRS.

b. Technical Resource. The technical resource is responsible for:

(1) OJT update training in eLMS.

(2) Obtaining designation, in writing, from the office manager (template found on the OJT SharePoint site).

(3) Following AFS’ standardized procedures prescribed in paragraph 7, The OJT Process, in this appendix.

(4) Exhibiting objective, constructive, empathetic, and other behaviors conducive to supporting all inspector trainees regarding OJT.

(5) Assessing the trainee level of knowledge and skill on specific tasks and providing feedback to the assigned trainer.




(6) Providing structured, well-planned OJT training with stated objectives and expected levels of performance.

(7) Communicating with the OJT trainer about trainee’s task performance.

(8) Ensuring that the trainee has accomplished all elements of OJT training associated with a particular task in an acceptable manner.

(9) Ensuring that OJT training is consistent with applicable national regulations and practices.

(10) Recording time and activities spent on OJT in appropriate tracking program(s), e.g., LDR, PTRS.

c. OJT Trainer. The OJT trainer is responsible for:

(1) Completing the Flight Standards OJT Techniques Training (formerly 25703, 25702, or 10530), Managing the Office OJT Program Training, or AFS-500 approved training.


(3) Ensuring that OJT instruction is consistent with applicable national regulations and practices.

(4) Following AFS’ standardized procedures for training delivery prescribed in the current OJT Techniques courses or other approved AFS-500 training.

(5) Exhibiting objective, constructive, empathetic, and other professional behaviors conducive to supporting OJT with all trainees.

(6) Conducting OJT according to the trainee’s individual training plan as developed by the trainee and his or her FLM.

(7) Assessing the trainee level of knowledge and skill on specific tasks.

(8) Providing structured, well-planned, and documented OJT training with stated objectives and expected levels of performance in accordance with Order 8900.1, Flight Standards Information Management System (FSIMS), JTA data and OJT training guidance.

(9) Communicating with FLMs about the trainee’s progress.

(10) Informing the OJT PM of the trainee’s progress via e-mail or other locally implemented procedures, indicating date of completion of OJT in specific tasks and appropriate levels.




(11) Validating that the trainee has accomplished all elements of OJT instruction associated with a particular task in an acceptable manner before notifying the OJT PM and FLM that the trainee is able to perform the task as required.

(12) Notifying the OJT PM that tasks have been completed at each level. If the FLM accepts prior experience or training for any level of OJT, the FLM should sign off at the accepted levels after validation.

(13) Recording time and activities spent on OJT in appropriate tracking program(s), e.g., LDR, PTRS.

d. OJT PM. The OJT PM is responsible for:

(1) Implementing and managing the OJT Program in the office.

(2) Ensuring that the local OJT Program is consistent with national policies and guidance.

(3) Educating and keeping office management, FLMs, and employees current on the AFS OJT Program.

(4) Continually evaluating the effectiveness of the local OJT Program and recommending improvements to management. When authorized, the office (and/or designee) will implement improvements to ensure the office develops the skills and capabilities it needs.

(5) Completing the Flight Standards OJT Techniques Training (formerly 25703, 25702, or 10530), Managing the Office OJT Program Training, or AFS-500 approved training


(7) Facilitating the resolution of problems or issues that may impede the effective delivery of OJT.

(8) Participating in the quarterly regional teleconferences with other OJT PMs to discuss the OJT Program’s best practices, suggested changes, and program issues.

(9) In coordination with office management, interact with AFS-500 to implement national OJT policies, objectives, and priorities, and to communicate AFS office requirements or issues that may require a national response.

(10) Assisting the office managers and FLMs in identifying tasks performed in the office environment for which OJT is required.

(11) Ensuring the currency of the profile against the FSIMS job task list.

(12) Facilitating and supporting management and trainees in developing OJT individual training plans and schedules.




(13) Ensuring OJT instruction is provided to trainees in accordance with applicable directives and the trainee’s OJT plan.

(14) Conducting reviews of each trainee’s OJT records (in PTRS), and ensuring that after Level III, completed tasks have been entered correctly into PTRS and the record has the “sign off” by the FLM (also in PTRS).

(15) Ensuring that new hires begin their OJT Program as soon as possible after their date of employment. (Completion of classroom indoctrination training at the FAA Academy is not a prerequisite to OJT instruction.)

(16) Recommending OJT trainer candidates to the office manager.

(17) Verifying that, before conducting OJT, selected OJT trainer candidates have successfully completed the Flight Standards OJT Techniques Training (formerly 25703, 25702, or 10530), Managing the Office OJT Program Training, or AFS-500 approved training; and that they hold a letter of designation from the office manager.

(18) Monitoring OJT trainer performance and coaching OJT trainers on effective methods and techniques.

e. FLM. The FLM is responsible for:


(2) Implementing the provisions of the OJT Program training requirements for all trainees for whom direct supervisory oversight is provided.

(3) Ensuring that task training is tracked in appropriate training records and oversight of their employees OJT Program.

(4) Applying the guidance found on the FAA Human Resource Management (AHR) intranet site if employee performance does not meet expectations.

(5) Adhering to provisions of all applicable collective bargaining agreements.

(6) Ensuring that new hires begin their OJT Program as soon as possible after their date of employment. (Completion of classroom indoctrination training at the FAA Academy is not a prerequisite to OJT instruction.)

(7) Forecasting and informing the office manager of their employees’ OJT needs on a periodic basis so that management may determine the necessary resources for the program.

(8) Assist in establishing standardized baseline tasks (profiles) for each position (e.g., GA Operations, AC Airworthiness) in collaboration with the OJT PM and, depending upon office environment, identifying the tasks that require OJT.




(9) Meeting with each direct-report employee to develop an OJT Plan. And assigning to each trainee, tasks required based on his/her work assignment

(10) Authorizing OJT levels of credit granted to an employee. (Credit may be given for previous training or work experience for Levels I and/or II, provided each level is appropriately validated. Level III must be accomplished through OJT training.)

(11) Determining and recording satisfactory accomplishment of each OJT task assigned to each subordinate employee.

(12) Participating in the meeting to review the trainee’s task performance and progress.

(13) Deciding when OJT is conducted, including start and end time, as well as the amount of time required for completing OJT training on specific tasks.

(14) Reviewing with each OJT trainer, on a regular basis, the progress of assigned trainees and initiating any corrective action necessary to improve performance and/or training deficiencies.

(15) Recommending to the office manager those experienced employees who may be qualified to serve as OJT trainers and providing feedback on OJT trainer performance evaluations, including recommending removal of authorization as an OJT trainer.

(16) Evaluating OJT trainer performance during normal performance review cycle based on:

o Feedback from trainees and the OJT PM,o The trainer’s ability to meet training plans, ando The selection criteria.

(17) Assuming the role of mediator and decisionmaker when there are OJT problems and/or disagreements involving OJT trainers and inspector trainees.

(18) Acting upon feedback from trainees concerning the OJT Program, either resolving the situation at their level or ensuring that the information is forwarded to an appropriate party for action.

(19) Implementing program improvements, in conjunction with the OJT PM.

(20) Ensuring personnel designated as an OJT trainer or OJT PM meet all established requirements for those roles. (The use of management personnel as an OJT PM or trainer is highly discouraged except for unique and unusual circumstances. Coordination with upper management is strongly recommended.)

f. Office Manager. The office manager is responsible for:





(2) Ensuring that OJT orientation and OJT training is provided to assigned AFS employees within the jurisdiction of their office.

(3) Emphasizing the OJT Program in appropriate budget submissions made to the AFS division manager.

(4) Determining the number and types of OJT program resources required to support their office OJT Program.

(5) Applying the guidance found on the AHR Web site if employee performance does not meet expectations.

(6) Adhering to provisions of all applicable collective bargaining agreements.

(7) Identifying standard baseline tasks for each specialty in the office (refer to Phase 1, Step 1.1—Establish Local OJT Infrastructure), and ensuring that the baseline tasks are approved in accordance with the local office procedures. This should be done in coordination with the FLMs and the OJT PM.

(8) Establishing a standardized method to ensure that trainees are provided adequate time and resources required for completing OJT training on specific tasks.

(9) Selecting and designating, in writing, the individual(s) who meet(s) all of the requirements and assigned the role of OJT PM.

(10) Ensuring that the OJT PM has proper training and has established the OJT Program in the office in accordance with national AFS guidelines.

(11) Selecting and designating in writing the individual(s) who meet(s) all of the requirements and assigned the role of OJT trainer.

(12) Determine if existing OJT trainers will remain in that role, based on projected training needs and performance. (See Resource Requirements in Phase I, Step 1.1 for recommendations.)

(13) Selecting and designating, in writing, the individual(s) who meet(s) all of the requirements and assigned the role of Technical Resource.

(14) Ensuring that inspectors assigned to perform OJT instructional duties meet all of the selection criteria, as outlined in the Phase I—Planning section of this appendix, and are proficient in the job functions and tasks for which they will provide instruction.

(15) Obtaining assistance from an OJT trainer located at another office when a training requirement cannot be fulfilled locally due to the lack of internal instructional expertise.

(16) Ensuring that employees have successfully accomplished and are able to perform the tasks assigned and that tasks are recorded.

(17) Ensuring any accreditation for Level I and/or II OJT has been properly validated.




g. Regional OJT PM. The regional OJT PM is responsible for:

(1) Completing the Flight Standards OJT Techniques Training (formerly 25703, 25702, or 10530), Managing the Office OJT Program Training, or AFS-500 approved training

(2) Completing the AFS OJT Briefing for Managers training course.

(3) Having experience as an OJT trainer.

(4) Communicating between the office OJT PM and AFS-500.

(5) Performing an annual audit of the individual offices’ OJT program and communication any findings to AFS-500.

(6) Briefing division management on the status of the offices’ OJT programs.

(7) Performing onsite evaluations.

(8) Contacting each office and branch in the region to evaluate the OJT program.

(9) Reporting results of the evaluation to the office, regional management, and AFS-500.

(10) Participating in periodic teleconferences between the office, regional program managers, and AFS-500.

(11) Coordinating teleconferences, quarterly or as needed, for OJT PMs across the region.

(12) Attending the National OJT teleconference with AFS-500 or designating an OJT PM attendee.

(13) Ensuring that offices/region/headquarters OJT baseline tasks are in compliance and post to OJT PM SharePoint site.

h. Division Manager. The Regional AFS division manager is responsible for:

(1) Ensuring that the AFS OJT Program is implemented efficiently and effectively.

(2) Planning and budgeting to ensure that the OJT Program receives the resources necessary for the effective accomplishment of its stated purpose.

(3) Selecting and designating in writing the individual(s) who meet(s) all of the requirements and assigned the role of regional OJT PM.

(4) Ensuring local and regional evaluations are completed.




i. The OJT Steering Committee. This national team composed of field and headquarters personnel is responsible for:

(1) Overseeing the OJT Program.

(2) Providing guidance for the development and implementation of the program.

(3) Monitoring and assessing the accomplishment of program objectives and recommending changes to the national program.

(4) Meeting on an annual basis and more often as required by program needs.

j. Flight Standards Training Division (AFS-500). AFS-500 has responsibility for:

(1) Developing and managing the program.

(2) Working with the OJT Steering Committee and stakeholders to develop OJT policy and guidance.

(3) Coordinating with division contacts and the OJT Steering Committee to implement program policy and guidance.

(4) Revising the OJT Program guidance and OJT courses to reflect new policy and/or guidance.

(5) Integrating OJT into the overall training program.

7. The OJT Process. Three phases compose the OJT Process: Planning, Delivery, and Evaluation. While these phases are separate and have distinct actions and roles, they are cyclical and continual in nature. For example, planning for the office needs is continual as the office dynamics change. An evaluation of the program may lead to a change in the office needs, thus requiring additional planning.

8. Phase 1—Planning. Figure A-1, Phase 1, Planning, depicts the steps and key participants in the OJT Planning phase. This phase consists of planning for all aspects of the program—from the overall office needs to the individual trainee.




Figure A-1. Phase 1, Planning

Step 1.1—Establish the Local OJT Infrastructure.

Key Participants

Office Manager Front Line Manager OJT PM

a. Overview.

(1) The first step in the Planning phase is to establish the local OJT infrastructure. This includes establishing the resources needed for the program and determining the tasks that the office needs as an initial profile. During this step of the Planning phase, the office manager will designate an OJT PM who will then conduct activities to establish the OJT Program in the office or to implement changes in the OJT Program.




(2) All program participants are encouraged to provide feedback in the form of questions and comments through the individual office OJT PMs who can, when required, escalate those concerns to the OJT Steering Committee. Questions, concerns, or comments about the OJT Program can be submitted via the OJT SharePoint site.

b. Process. Select and designate the OJT PM.

(1) The OJT PM is very important to the success of the OJT Program within the office. He or she has the responsibility of implementing the program according to national policy and this order. When selecting an OJT PM, the office manager should consider the following demonstrated criteria:

o Desire to be a PM.o Effective level of communication and interpersonal skills with people at all levels.o Ability to make presentations to groups.o Ability to set up a program and to oversee its implementation.o Knowledge of OJT instruction.o Willingness to track OJT for each employee in the office.o Attention to detail.

(2) Unless previously trained, the OJT PM must complete the OJT Techniques and the OJT PM training courses. In writing the office manager will state that the individual meets the criteria to be an OJT PM. An example of a PM designation is on the OJT SharePoint site. The letter lists the specific roles and responsibilities assigned to the OJT PM.

c. Develop the OJT Office Task Profile.

(1) The OJT framework in this appendix is directly applicable to positions that have established job tasks in FSIMS. This same framework may also be used for those positions that do not have currently established job tasks in FSIMS.

(2) A master OJT job task listing was taken from FSIMS. This list is in the OJT Planning Module. This module is used to build initial task profiles and is accessible on the OJT SharePoint site along with instructions for how to use the module.

(3) As part of establishing the OJT Program in the office, a profile for each specialty in that office (e.g., General Aviation Operations, Cabin Safety, Air Carrier Airworthiness) must be developed. This profile includes a list of common tasks that the local management determines most employees should be able to perform, driven by the local office environment. This profile must be posted to the OJT SharePoint site. Offices will annually review their local environment to determine if any changes to the profile are necessary. This profile serves as a starting point in Step 1.3 for establishing a task list specific to individual employees. Use the OJT Planning Module to document training tasks not currently established in FSIMS, e.g., job tasks for ADMIN, HQ personnel.




(4) The OJT PM ensures the currency of the profile against the FSIMS job task list. The FLMs, office managers, and OJT PMs must be involved in establishing the profiles. Once developed, any changes must be coordinated with the office manager, FLM, and OJT PM in accordance with the local office procedures.

d. PTRS Tracks AFS OJT. Employees assigned OJT tasks must use PTRS and the associated guidance, accessible on the Sharepoint site. There is no requirement to recapture tasks already accomplished in the previous OJT tracking systems.

Figure A-2. OJT PTRS Tracking Decision Aid

Note: Before 2001, there was no mandate for a national tracking system. Employees onboard before 2001 are considered to have met all OJT task training requirements for their duty assignments at that time.




e. Select OJT Trainers. The office manager/FLM and the OJT PM should estimate their trainer requirements in planning the OJT Program. At a minimum, there should be one trainer for each represented occupational specialty in the office. Use the Training Needs Assessment process (including pop-ups) to plan for additional requirements or unusual circumstances. To determine the appropriate resources, consider the following:

(1) How many employees, including new hires, are predicted to need OJT for the planning period?

(2) What knowledge and skills are required for the OJT trainers?

(3) What specialties are represented in the needed training? This should come from the profiles developed earlier.

(4) How can the trainer resources be best utilized?

(5) Will it be necessary to adjust an employee’s workload to ensure the quality of the OJT instruction?

(6) Does the OJT trainer candidate meet the following criteria?

o Have qualification in the job specialty and job tasks he or she intends to teach.o Have advanced knowledge, skill, and experience that match the identified training needs along with the necessary skills to support and enhance training and create an effective learning environment.o Possess the ability to demonstrate a task in a clear and logical order.o Willing to prepare training, and instruct and coach trainees on performance of tasks being trained.o Possess the ability to communicate technical information, concepts, and procedures clearly, concisely, and positively in a variety of ways.o Desire to be a trainer.o Have effective level of communication and interpersonal skills with people at all levels.

(7) OJT trainers must attend an AFS OJT Techniques course or some other approved AFS-500 training to ensure consistency in delivering OJT and in evaluating the trainee’s progress. The OJT PM working with the trainer will reinforce concepts taught in the course by conveying the value of providing a structured, planned training activity for each trainee.

(8) Once individuals are appropriately trained to be OJT trainers, the office manager will state, in writing, that the individual attended the AFS OJT Techniques training, and meets the criteria to be a trainer. An example of an OJT trainer designation letter is on the OJT Sharepoint site. The letter will list the specific roles and responsibilities assigned to the OJT trainer from among those roles and responsibilities of OJT trainers listed in this appendix. Only those OJT trainers who are so designated are considered to be OJT trainers under the AFS OJT Program. In




assigning trainers to trainees, the FLM may choose to designate a primary trainer for each trainee; any other qualified trainer may also contribute to fulfilling the OJT Profile.

f. Designation of Technical Resources. In unique circumstance where an OJT trainer lacks the experience or mastery of a specific job task, the office manager may designate, in writing by task(s), a technical resource to conduct the training of that particular job function. Adjustment/balancing of employee workload may be necessary to ensure the quality of the OJT instruction.

(1) Use the following criteria to identify/nominate technical resources:

o Knowledge and skill required to be a technical resource.o Ability to adjust workload to ensure the quality of the OJT instruction.o Qualification in the job specialty and job tasks he or she is intended to teach.o Advanced knowledge, skill, and experience that matches the identified training needs along with the necessary skills to support and enhance training and create an effective learning environment.o Ability to demonstrate a task in a clear and logical order.o Willingness to prepare training and instruct trainees on performance of tasks in which trainees are being trained.o Ability to communicate technical information, concepts, and procedures clearly, concisely, and positively in a variety of ways.o Compliance with the standards and definitions of professionalism.

(2) Notes:

(a) There are both technical and instructional criteria requirements in the designation of a technical resource. The technical resource must have a mastery of technical skill and experience in giving instruction, (e.g., certificated flight instructor (CFI), 147 instructor, demonstration of instructional skill set observed by management.)

(b) When technical resources are used to accomplish OJT training by other than a designated OJT trainer, a designated OJT trainer must validate the training to ensure compliance with the training plan and other OJT objectives.

(c) If a resource possesses the technical requirements but lacks the instructional experience, an OJT trainer must observe the training.

Step 1.2—Communicate OJT Program to Local Office

Key Participants

OJT PM




a. Overview. A successful implementation of OJT in the office depends on communication between all of the people involved in the program.

b. Process. In coordination with office management, the OJT PM will schedule and facilitate any communication regarding the OJT Program in the office, as deemed appropriate by the office management. The OJT PM will conduct an initial meeting with trainees. The OJT PM will communicate with individuals, or small groups, as needed. Annually, there should be a meeting to discuss changes/status in the OJT Program. The OJT PM will:

Describe the AFS OJT Program. Introduce key participants and describe their roles and responsibilities. Discuss how OJT profiles are developed. Describe how OJT will be tracked and recorded as part of each employee’s training record. Answer questions about the OJT Program.

Step 1.3—Develop Individual OJT Profiles

Key Participants

Trainee Front Line Manager OJT PM

a. Overview. The OJT PM coordinates and facilitates a meeting between the trainee and the trainee’s FLM. This meeting is important to ensure that OJT Program expectations and responsibilities are understood. The focus of the meeting is to assess the OJT needs of the trainee and to develop the trainee’s OJT plan.

b. Process.

(1) Assess Needs to Develop the OJT Plan. The meeting should begin with the FLM/OJT PM reviewing the information from Step 1.2 such as:

o The importance and goals of OJT.o The roles of the trainee, OJT trainers, FLM, office manager, and OJT PM.o The OJT process.

(2) Inform the trainee that OJT is a means of receiving training but does not substitute for required classroom training. The OJT PM should aid the FLM and trainee/employee in assessing the employee’s training needs. This leads to the development of the individual’s OJT Plan.

(a) Ask and/or answer questions related to the employee’s experience and needed OJT.

(b) Starting with the office profile established in Step 1.1, the FLM will decide which tasks should be completed by the employees they supervise. There may be some tasks that an




employee does not have to complete because his or her work assignments do not include those tasks.

(c) Determine if any credit for classroom training (such as indoctrination) and/or prior experience can be applied in any tasks. In cases where the trainee has received formal training (FAA Academy classes, computer-based instruction (CBI), Web-based training) on specific tasks, or has previous directly related experience, the employee should request OJT credit for this prior training/experience. The trainee’s FLM may decide to credit up to the first two levels of OJT for certain tasks after assessing that the trainee’s knowledge is satisfactory and appropriately validated on those specific tasks. Level III can never be attained from formal training and/or prior experience; it must be trained, demonstrated, and validated through the OJT task training process.

(d) Establish a timeframe for when the OJT training tasks should be accomplished. This should list the tasks that should be accomplished by the next review meeting (see Step 2.5).

(e) At the end of the meeting, the trainee should know which tasks he or she needs to accomplish as OJT and a timeframe for getting them accomplished. If meeting participants are not able to reach agreement on the tasks to be trained, the FLM will make the decision. Either during the meeting or soon after, the trainee should be assigned a trainer to begin the OJT training. The trainee may have several trainers during OJT, but each should follow the training plan and work on the identified tasks to be accomplished.

c. Recording the OJT Plan. The OJT PM will finalize the list of tasks and provide the list to the trainee. The trainee will create a PTRS record (X023) for each OJT task (refer to the OJT SharePoint site for instructions). The OJT PM will explain to the trainee that there will be a quarterly review of the OJT record for accuracy.

Note: PTRS is used for tracking AFS OJT. Upon the effective date of this order, employees assigned OJT tasks must use PTRS and the associated guidance, accessible on the Sharepoint site. There is no requirement to recapture tasks already accomplished in previous OJT tracking systems.

9. Phase 2—Delivery. The diagram below depicts the steps and key participants in the OJT Delivery phase. This phase consists of the actual conduct of the training from its initial scheduling through to its progress review. Steps 2.4 and 2.5 may be completed simultaneously.




Figure A-3. Phase 2, Delivery

Step 2.1—Schedule OJT

Key Participants

Front Line Manager OJT PM OJT Trainer Trainee

a. Overview. Allotting sufficient time for the OJT to take place is an important part of scheduling OJT. It is also important to review the scheduling during Step 2.5 to ensure that the plan established in Step 1.3 is still reasonable.




b. Process. The trainee and OJT trainer will jointly develop a proposed schedule for providing OJT according to the approved OJT plan (as developed in Step 1.3). The FLM will allot sufficient time for the OJT to take place. The trainee should take initiative to ensure the planned OJT provides the needed training. Adjust the trainee’s work program to accommodate the trainer’s schedule, when practical. If a scheduling conflict arises, the trainee’s FLM will decide. Consider the following in planning time for OJT for the different levels of OJT:

(1) Typically, Level I training is mostly a self-study effort on the part of the trainee with guided discussion and validation conducted by the OJT trainer afterward. The complexity of the task should be a factor of the time allowed for OJT.

(2) Because Levels II and III usually involve the actual performance of the task (the trainer performs Level II; the trainee performs Level III), a good guideline is to take the normal amount of time to conduct the task and add 50 percent of that time to allow for instruction and questions (e.g., a normally 1-hour task becomes 1.5 hours for OJT).

(3) As this process of scheduling OJT is continuous in nature, and as opportunities for OJT on a task unfold, the schedule for delivering OJT should also change.

Step 2.2 – Prepare to Deliver OJT

Key Participants

OJT Trainer OJT PM

a. Overview. As mentioned previously, OJT trainers, or other technical resources, need to be experienced in the task(s) they are assigned to teach to trainees. The trainer needs to plan each lesson carefully.

b. Process.

(1) The main guidance documents that a trainer needs to prepare for and deliver any OJT are:

o Job Task document located at on FSIMS, or the source guidance for non-FSIMS job tasks.o Guidance on Conducting and Validating OJT, (refer to Figures A-6 through A-9 of this appendix), ando Order 8900.1 guidance for the task being taught.

(2) With these documents, the trainer will have the objectives and tips for delivering the information to the trainees. It is important for the trainer to prepare the lesson to ensure that all relevant information is included and presented logically.

(3) Using these documents, the trainer should:




(a) Review all reference material relevant to the task as found in 8900.1 and the job task document (e.g., orders, guidance in Order 8900.1, Codes of Federal Regulations (CFR), and job task lists).

(b) Identify and gather all needed equipment, software, and hardware.

(c) Determine if assistance from other resources is needed regarding the task and how it should be performed. If personnel other than a designated OJT trainer are used as a technical resource, an authorized OJT trainer must validate the training to ensure compliance with the training plan and other OJT objectives.

(d) Finalize logistical arrangements for training in office or off site. This may depend on what level session the trainer is preparing.

Step 2.3—Deliver OJT.

Key Participants

OJT Trainer OJT Trainee

a. Overview. OJT trainers should ensure OJT is delivered in accordance with the schedule developed in Step 2.1. Guidance to deliver OJT for trainers is on the OJT Sharepoint site and based on Flight Standards OJT Techniques.

b. Process.

(1) Teach the Task: Depending on whether a session is for OJT Level I, II, or III, delivery can include such activities as:

o Establishing a training environment,o Developing rapport with the trainee,o Stating learning objectives and expected outcome of performance,o Assessing the trainee’s existing knowledge and skill in performing the task before the scheduled training,o Providing explanations,o Questioning to check for understanding,o Demonstrating tasks,o Allowing sufficient time for the trainee to practice the task,o Observing the trainee performing the tasks,o Motivating the trainee,o Reviewing and summarizing information,o Providing feedback and evaluating the trainee’s performance, ando Retraining when necessary.




(2) The training guidance document provides specifics about each level of training and includes the objectives, techniques for delivery, and how to validate that training is successful.

c. Validate the Trainee Achievement of Objectives. At the end of a training session for any OJT level, the trainer will validate that the trainee has successfully completed that session before notifying the OJT PM that training is complete. The training guidance document has a standard assessment form for each OJT Level.

Figure A-6. Conducting and Validating OJT

1. Conducting On-the-Job Training (OJT) — Levels I, II, & III

2. Before conducting any level of OJT, the trainer should complete or obtain the associated Job Task Analysis (JTA) document to use in conjunction with this order. Download JTA documents from the Flight Standards Information Management System (FSIMS). They provide the related references and steps for performing the task.

3. OJT is presented in three stages or levels of learning. Level I OJT is the basic background and knowledge of a task, Level II is a further understanding of the function and conduct of the task, including observation or assistance of the task being performed, and Level III is where trainees actually perform the task themselves. These three levels provide for a progressive and structured field training experience for anyone who needs to learn a new job task.

4. OJT trainers will use this order to conduct OJT in a consistent manner from task to task, and from trainee to trainee. It provides guidance on the delivery of the training for all three levels and presents a standardized way to validate learning and competency at all three levels so that the FLM can sign off on the task as completed.

5. Below are tips in planning for training of any given task:

a. OJT training Levels I and II may be covered in the same session.

b. OJT trainers should allow sufficient time between Levels II and III for the trainee(s) to practice the task, since they will be required to perform it on their own for Level III.

c. With the consent of the office manager and the trainee’s FLM, prior experience and completion of FAA-approved training may be credited for the first two levels of OJT.




Figure A-7. Conducting and Validating OJT—Level I

1. Purpose of Level I Training: Level I training focuses on the knowledge required for the task. The knowledge is contained in orders, rules, guidance, standards, and definitions.

2. Level I Performance Objectives. Following this training, the trainee will be able to:

a. Identify appropriate materials associated with the task,

b. Define key terms and definitions associated with the task,

c. Describe how the task is documented,

d. Explain how the task is initiated, and

e. Explain the task outcome(s).

3. Conducting Level I. Begin by:

a. Putting trainees(s) at ease and establishing rapport,

b. Reviewing the prerequisites (if any) to determine what the trainee already knows about the task, and

c. Reviewing the performance objectives for the task and the purpose for Level I training.

(1) The accomplishment of Level I should include a review of appropriate regulations, guidance, and forms required for the task as found in the job task documents in FSIMS. This may be done in any of the following (or other similar) methods:

d. Have the trainee read through the materials on their own. Then you (the trainer) review and discuss the material with the trainee prior to validating Level I completion.

e. Give the trainee the JTA documentation for the given task and have him/her gather and research the resources from the JTA and any additional current guidance and discuss with you (the trainer).

f. Review the guidance with the trainee and/or present to the trainee (allowing for questions), then validate Level I completion.

g. For two or more trainees at a time (with the same task), have the trainees research and review the guidance together and then you (the trainer) review and discuss the material with the trainees prior to validating Level I completion.




Figure A-8. Conducting and Validating OJT—Level II

1. Purpose of Level II Training. Level II training usually involves a demonstration by the trainer of the specific job task steps and procedures with trainee observation and/or assistance to achieve a level of understanding.

2. Level II Performance Objectives. Following this training, the trainee will be able to:

a. Describe the sequence of steps to accomplish the task (as applicable),

b. Describe how appropriate materials are used to accomplish the task (e.g., forms and equipment),

c. Describe interactions among other FAA personnel required to accomplish the task, and

d. Describe coordination with the operator required to accomplish the task.

3. Conducting Level II. Begin by:

a. Gathering any materials needed to perform the task (the trainee may do this as well),

b. Reviewing the performance objectives for the task and the purpose for Level II training, and

c. Reviewing what was covered in Level I training.

4. Demonstration of the Task. The accomplishment of Level II should include a demonstration of the task itself, as follows:

a. Explain what you will be doing by briefly reviewing the task steps.

b. Solicit any questions about the task before you begin.A-26

UNCONTROLLED COPY WHEN DOWNLOADEDCheck with FSIMS to verify current version before using


c. Based upon the task and the comfort level of the trainee, determine whether or not it is appropriate for the trainee to assist in the task or simply observe you doing the task. You may also simply ask the trainee their preference—assist or observe.

d. Be sure that the environment is conducive to learning. For example, if you are performing the task in the field, can the trainee(s) sufficiently see and hear you?

e. Ask the trainee for the next step(s) as you demonstrate the task.

f. Ask questions about how the step is performed.

(1) For tasks that are largely document-based, actual demonstration may not be applicable. Therefore, Level II may be based on the review and discussion of sample or completed documentation. For example, Level II for reviewing a manual could include the trainee reviewing a manual for which an experienced aviation safety inspector (ASI) has already completed a review, and then comparing the trainee’s review to the ASIs completed review.




Figure A-9. Conducting and Validating OJT—Level III

1. Purpose of Level III Training: Level III training involves the trainee performing the task independently and accurately under the observation of the OJT trainer. The trainee may need more than one attempt to complete the task successfully. After each attempt, the trainer should provide feedback and suggestions for improvement for the next time.

2. Level III Performance Objectives. Following this training, the trainee will be able to do the following:

a. Demonstrate sufficient knowledge to complete the task proficiently,

b. Complete all steps necessary to accurately complete the task,

c. Complete steps in the proper order (as applicable),

d. Perform the task without assistance, and

e. Perform the task in a timely manner without undue hesitation.

3. Conducting Level III. The OJT trainer should sign off on Level III training only when the trainee meets all requirements, including specific prerequisite courses, are met. Begin as follows:

a. Review what was covered in Level II training,

b. Review the performance objectives for the task and the purpose for Level III training,

c. Ask the trainee(s) if he/she has all of the materials necessary to perform the task, and

d. Explain expectations—that the trainee will complete the task accurately and without assistance.

(1) Use the following task observation guidelines as the trainee performs the task:

e. Check the steps as you observe using the materials provided in the JTA documentation from FSIMS.

f. Assist only if it is required as part of the task to have a second person. Do not offer assistance.

g. Circle omitted or incorrect steps to address with the trainee once the task is complete.

h. Stop for unsafe or illegal actions. Discuss them with the trainee immediately before completing the task.

(1) Simulation is a method for training inspectors when the “real-life” task situation is not feasible and/or advisable, such as an en route inspection. When a task is simulated, the environment, conditions, equipment, and performance of the task must be as near the “real-life”




situation as possible. Simulation is not prohibited, but if the office selects OJT tasks for inspectors that are normally done in that office, there should be minimal simulation required. The trainer may need to develop a brief scenario to let the trainee know what they are looking for and why.

4. Validating Level III. To validate Level III OJT, the trainer must be able to answer “Yes” to all of the following:

Yes No

Did the trainee demonstrate sufficient knowledge to complete the task proficiently?

Did the trainee complete all steps necessary to (accurately) complete the task?

Were the steps completed in the proper order (as applicable)?

Did the trainee perform the task without assistance?

Did the trainee perform the task in a timely manner without undue hesitation?

Note: Recording OJT. OJT PMs, trainers, and trainees are required to record time spent on OJT in the appropriate tracking program (e.g., LDR and PTRS). Refer to instructions accessible on the OJT SharePoint site.

Step 2.4—Update OJT Records

Key Participants

OJT PM Frontline Manager OJT Trainer OJT Trainee

a. Overview. The OJT trainee creates and updates OJT records. These records track the trainee’s progress in his/her training plan and are updated as training is delivered.

b. Recordkeeping Requirements. Data entry is in PTRS. Records review is available through SPAS. Refer to instructions accessible on the OJT SharePoint site for recordkeeping procedures.

c. Process. The OJT trainer notifies the OJT PM and the trainee’s FLM via e-mail, or other locally implemented procedure that will provide a record, that a trainee has completed training on a task. The notification should include the PTRS Record ID. The OJT PM will review the records for accuracy and completeness.




Step 2.5—Conduct Review

Key Participants

OJT PM Frontline Manager Trainee

a. Overview. The purpose of the review meeting is to review the trainee’s progress in their OJT training plan and discuss what should be done next. This review will be conducted on a quarterly basis, at a minimum. The frequency of these reviews may depend on various factors such as the trainee’s volume of assigned OJT, whether the trainee or trainer is having any trouble with their OJT, or the changing needs of the office requiring changing assignments.

b. Process.

(1) The OJT PM should schedule a meeting with the trainee and the trainee’s FLM. The following areas should be discussed:

o Review of OJT since last review.o Feedback on OJT by trainee, since last review.o Update of trainee’s OJT plan, as needed.o Identification of next or new tasks that require training.o Identification of opportunities for OJT.o Accuracy of completed tasks.

(2) During this review meeting, if the trainee provides negative feedback on the OJT trainer, the OJT PM should meet with the OJT trainer separately and provide coaching as needed. The OJT PM or the trainee’s FLM (who was in the meeting) should meet with the trainer’s FLM to inform him or her of the negative feedback. The FLM decides whether an OJT trainer should continue in that role.

(3) The OJT PM records the date of the quarterly meeting in the training record along with any comments, if necessary, for review by the FLM. The OJT PM will record the meeting in an X025 and include comments.

10. Phase 3—Evaluation. The diagram below depicts the steps and key participants in the OJT Evaluation phase. This phase consists of evaluating the effectiveness of the OJT Program at the local, regional, and national levels. This phase is cyclical in nature in that it is a continuous process of evaluation, communication, and improvement.




Figure A-4. Phase 3, Evaluation




Step 3.1—Evaluate the OJT Program

Key Participants

OJT PM OJT Instructor Trainee Office Manager

Frontline Manager Regional OJT PM AFS-500

c. Overview.

(1) The evaluation process is a significant contributor to the quality management of OJT. Continuous improvement can only be accomplished through continuous evaluation. The evaluation process monitors the following key areas:

(a) Measurable Standards—Quantifiable accomplishment of stated procedural standards.

(b) Cultural Application—Qualitative accomplishment of stated policy.

(c) Overall Effectiveness—The success of the output product.

(2) Established evaluation tools are used at all levels of the organization and are accessible on the OJT SharePoint site.

d. Process. Quality audits will occur at the local and regional level.

(1) Local. The OJT PM will evaluate the local program with the input of OJT trainers, trainees, and management. This evaluation is required annually. The OJT PM will evaluate the OJT Program through audits, meetings, and observation. The review meeting (refer to Step 2.5) is one way of determining if the OJT Program is working properly. The feedback and suggestions should be analyzed and discussed with the office manager.

(2) Regional. Annually review the local offices self-audits for compliance and taking the appropriate corrective action. Additionally the Regional OJT PM will perform onsite evaluations of each office once every three years. Use the evaluation tools located on the OJT PM Share Point.

Step 3.2—Communicate Findings.

Key Participants

OJT PM Regional OJT PM AFS-500

a. Overview. Communication of evaluation findings occurs across all levels in evaluation. Findings from an evaluation may indicate a localized issue to be addressed individually, or may indicate a systemic issue that needs to be communicated broadly.




b. Process. The analyzed feedback and suggestions will be discussed with the office, division manager, and AFS-500. Continuous evaluations are encouraged throughout the application and conduct of OJT.

(1) Local. The local feedback will be analyzed and any suggested changes discussed with the office manager and forwarded to the regional OJT PM.

(2) Regional. The regional OJT PM will report the findings of their office evaluations to division and headquarters management. The regional OJT PM will communicate these findings through teleconferences, meetings, or visits.

(3) National. The National OJT Program office (AFS-500) will communicate OJT Program status to AFS senior management. AFS-500 will promote the OJT Program throughout AFS via briefings, newsletters, e-mail and other methods, as necessary.

Step 3.3—Implement Improvements

Key Participants

OJT PM AFS-500

a. Overview. The result of any evaluation should be to identify possible areas of improvement, where feasible. Program improvements at the local and regional levels must remain in accordance with this order. Program improvements at the national level may require change(s) to this order. Changes regarding this order affecting national policies, guidance, and procedures may be implemented only when approved by AFS-500.

b. Process.

(1) The OJT PM is critical in implementing changes as needed to ensure the local office develops the skills and capabilities it needs. The OJT PM, in concert with local management, will develop an implementation plan for needed improvements, answering the following questions:

(a) What improvements are needed to the OJT Program?

(b) What are the benefits of these improvements?

(c) What are the competing needs?

(d) Do these improvements affect national standards and policies?

(e) What approvals are needed?

(f) What funding is needed? Who will provide the funding?

(g) What is the plan for implementing improvements?

(h) Who is responsible for carrying out the improvements?A-33

UNCONTROLLED COPY WHEN DOWNLOADEDCheck with FSIMS to verify current version before using


(i) What is the estimated timeline?

(2) The office manager may be able to provide assistance to the OJT PM in answering some of these questions. The plan should be approved by the office manager and then shared with the regional office. If desired, the OJT PM can ask that AFS-500 review the plan and provide feedback.

Figure A-5. OJT Program Evaluation




Evaluation Instructions.

a. The evaluation process is a significant contributor to the quality management of OJT. Program improvements can only be accomplished through continuous evaluation. The purpose of these evaluations is threefold.

(1) First, the evaluations determine if the local program is meeting the stated procedural standards of the AFS OJT guidance.

(2) Second, they determine if the local program is meeting all guidance policy.

(3) Finally, the evaluations should focus on how successful the local program is at ensuring a well-trained and qualified workforce.

b. To aid in conducting evaluations, an OJT evaluation tool can be accessed on the OJT Program Managers (OJT PM) SharePoint site. In accordance with AFS OJT guidance, each office/branch OJT program will be evaluated annually. Normally, this evaluation will consist of an office self audit conducted by the local office/branch OJT PM. The results of these self audits will be forwarded to the Regional OJT PM.

c. The Regional OJT PM will conduct a desk audit of all offices’ OJT programs self audits. All offices must receive an onsite evaluation by the Regional OJT PM at least once every 3 years. These evaluations are to be conducted regardless of whether there are new hires in the office. The results of the Regional OJT PM’s desktop audits and onsite evaluations will be forwarded, in report form, annually to regional/branch management and AFS-500.

d. A worksheet tab has been created to provide evaluation questions for the key participants in the OJT process. Specific instructions for the use of the questions are included at the beginning of each tab. However, to ensure the most objective evaluation possible, it is expected that the evaluation will normally be conducted personally by the OJT PM and/or Regional OJT PM

e. The evaluation methodology is to include (but not be limited to), the sampling of PTRS records, interviews with key program participants, and, if possible, observation of actual OJT training being conducted. The evaluator should use the responses gathered to form an objective opinion as to the office’s effectiveness at meeting the evaluation goals stated above. Sufficient data should be sampled to accomplish this objective.

f. Program strengths and best practices should also be identified and captured and used to contribute to overall program improvement. Yes and no answers to the evaluation questions should be avoided to the maximum extent possible.

g. The distribution of evaluation forms for key program participants to complete on their own should also be avoided. Instead, a greater emphasis should be placed on personal observation on how a particular program requirement is accomplished by the office or why a particular procedure is followed. If a program requirement is not surveyed or observed, it




should not be rated.



1/6/12 3140.20BAppendix B

Appendix B. Course Mentor Program

Chapter 1. Course Mentor Overview

1. General. Sponsoring organizations have the overall responsibility to ensure that assigned training programs meet AFS needs. Managers of sponsoring organizations will designate course mentors (CM) to act on their behalf in managing each training course their organization sponsors.

a. CM Responsibility.

(1) CMs are the sponsor’s representative, and act on the sponsor’s behalf for all course matters, except formal approval of major revisions (course revision requests (CRR)) and approval of new training development requests (TDR). The manager of the sponsoring organization must sign these requests.

(2) While the manager of the sponsoring organization retains the primary responsibility for training, you, as the CM, are delegated the daily responsibilities of monitoring the content of your assigned courses. Because you will be acting on behalf of the sponsoring organization, you must keep the manager of the sponsoring organization informed of the status of their course. This communication may be more challenging, but is no less essential, especially if you are not assigned to the sponsoring organization.

b. Course Oversight. CMs are assigned oversight of AFS resident courses both at the FAA Academy and at the Center for Management and Executive Leadership. CMs have oversight of Web-based training, computer-based training (CBT), and out-of-agency courses that have been modified specifically for the Flight Standards Service (AFS). Off-the-shelf training purchases in the Title 14 of the Code of Federal Regulations (14 CFR) parts 121, 135, 141 and 142 schools, are under the authority of the local Flight Standards District Office and certificate management office (CMO). These 14 CFR courses are assigned to training program managers who provide surveillance and oversight.

2. Appointing CMs. The manager of a policy organization that is responsible for sponsoring an AFS training course is also responsible for appointing the CM. When a sponsoring organization develops a TDR for a new course, it will normally appoint a CM to assist with the training development. If the sponsoring organization does not appoint a CM, its manager will appoint a CM when the TDR is approved. When a CM is changed, the sponsoring organization will notify the training triad. The Flight Standards Training Division (AFS-500) will then post the change on the CM Web site.

B-1UNCONTROLLED COPY WHEN DOWNLOADED



Chapter 2. Getting Started as a Course Mentor

1. Introduction. This appendix provides background information and an outline of your roles and responsibilities as a CM. It will familiarize you with the particulars of your assigned course(s), if you are assigned to existing courses. This guide is not all-inclusive, nor can it provide answers to all of the questions you may have as a CM. It will provide you with the basic tools and reference information to enable you to accomplish the goal of making AFS training courses effective. Additionally, this document explains the guidelines for coordination, recurrent course evaluations, course revisions, and when required, new course development.

2. Where to Start.

a. Study This Appendix. Reading this appendix is your first step. Once you become familiar with your roles and responsibilities and coordination activities, familiarize yourself with the particulars of your assigned course(s). We recommend the following steps to help you understand the documents and procedures that you will use regularly as a CM.

(1) Become familiar with the CM Web site and the information available to assist you with your CM responsibilities (https://avssharepoint.faa.gov/afs/500/CM/default.aspx). Once you are assigned as a CM by your division manager, you will automatically be given access to the CM Web site. You will use your AFS logon and password to gain access to the site.

(2) Make an introductory call to the appropriate course manager and establish your lines of communication.

(3) Collect the following supporting data for all of the courses you mentor:

(a) Determine which specific job tasks your course will cover, and how recently they were reviewed and/or updated (course sponsor).

(b) Obtain a Course Design Guide (CDG) for each course. Contact AFS-540, Curriculum Development Branch, to obtain CDGs. Review to determine that it is consistent with the job tasks.

(4) Review course lesson plans and/or storyboards. Contact AFS-500 for these materials.

(5) Become familiar with the performance objectives.

(6) Obtain the proposed schedule for course delivery. Contact AFS-520, Plans and Programs Branch, to obtain schedule.

(7) Request a copy of the last formal evaluation report. AFS-530, Quality Assurance Branch, conducts periodic course evaluations and maintains them on file.

(a) Determine what actions were recommended and what actions have been completed.



https://avssharepoint.faa.gov/afs/500/CM/default.aspx


(b) If a formal evaluation report is not available, determine when the next formal evaluation is scheduled. If one is not scheduled, work with AFS-530 to get one scheduled.

(8) Review and analyze the student course critique reports for the last few classes. You can obtain these reports from AFS-530.

(9) Discuss with the course manager their perspective of the course.

Note: You may find that some of the above documents are not available for all courses. If this is the case, contact AFS-500 and/or the Regulatory Standards Division, AMA-200, course manager to assist you with the missing item and/or to obtain the status/explanation.

b. Begin Initial Course Review. After you have collected the supporting data listed in paragraph 2a(3), you may begin your initial review of the course. The first step is to ensure the job tasks for the course are current and consistent with the sponsor’s needs. Secondly, determine if the CDG, lesson plans, and storyboards support the job tasks and meet the course design objectives. If the CDG is not available, begin your review by evaluating available lesson plans to ensure they meet the course objectives and are in support of the applicable job tasks.

Note: In the past, training courses were developed and revised using different standards. For this reason, you might notice that the documentation for one course may not match that of a similar course.




Chapter 3. Debriefing.

Note: Throughout and following your review, be sure to keep the appropriate individuals briefed on the status of your findings. Please review Chapter 6, paragraph 3b(2)c, Debriefing, for required coordination. Course Mentor’s Roles and Responsibilities

1. Introduction. The CM, whom the sponsoring manager appoints, is responsible for monitoring the quality and content of one or more of the sponsor’s courses. The CM has the authority to represent the sponsor for all course-related issues to ensure that the technical content of an assigned course meets the needs of AFS. To accomplish this task, CMs are asked to perform a variety of tasks.

2. CM Responsibilities for New Courses Under Development.

a. Coordinate TDR Requirements. Coordinate course TDR requirements, as outlined in the TDR, with potential stakeholders.

b. Identify Selection Criteria. Identify selection criteria for subject matter experts (SME) and ensure AFS-500 has the appropriate criteria for the memorandum to all regional division managers, regional business agents (RBA), and regional training coordinators.

(1) Participate with your project manager from AFS-500 to validate that the nominee meets the identified selection criteria.

(2) Select your SMEs from all participants that meet the selection criteria.

c. Collaborate to Validate SME Selections. Collaborate with AFS-500 and the Professional Aviation Safety Specialists (PASS) point of contact (POC) to validate SME selections.

d. Participate in Training Development Work Group. Participate as a member of the training development workgroup and represent the sponsor’s and stakeholders’ concerns and interests.

e. Review the CDG. Review the CDG with the training triad.

(1) Approve the final Job Task Analysis (JTA) report.

(2) Ensure that student performance objectives (e.g., knowledge, skills, and abilities) for the course are documented accurately and meet the needs of the sponsor and stakeholders.

(3) Review the content of the course and approve or disapprove suggestions the workgroup members make.

f. Resolve Development Issues. Work with the appropriate training triad members to resolve development issues.




g. Approve Training Readiness. Approve training readiness to proceed to a prototype delivery. Mentor should attend prototype delivery.

h. Provide Feedback. Observe the training provider’s operational tryout delivery and provide feedback to the provider and AFS-500 to ensure the material is presented accurately and in a manner consistent with the sponsor’s/stakeholders’ policy and guidance. If significant changes are required, AFS-500 will revise the course material.

i. Observe the Prototype Delivery.

(1) Observe the course prototype and work with the training triad to evaluate the class delivery and make any necessary adjustments.

(2) Review the lesson and end-of-course evaluations for the prototype, and work with the training triad to resolve issues identified during the prototype.

(3) When all adjustments to the new course based on the prototype delivery have been completed, approve and sign off on the course, including the final version of the CDG, as a formal part of the AFS training program.

3. CM Responsibilities for Existing or Successfully Developed Courses Added to the AFS Training Program.

a. Ensure Courses’ Continued Attainment of Performance Objectives. Ensure that assigned courses continue to meet their performance objectives by continually observing the delivery of the courses.

b. Respond to the Annual Call for Evaluations. Respond to the annual call for evaluations each fiscal year and coordinate with AFS-530 to ensure periodic course evaluations are scheduled and completed. A formal evaluation should be conducted every 3 years if the course is not under development or undergoing a major revision.

c. Act as the Sponsor’s SME on a Course Evaluation Team.

d. Monitor the End-of-Course Evaluation Results. This process will vary depending on the training provider. With Academy courses, this will include monitoring periodic updates to the electronic database used to record such data.

e. Solicit Input. Frequently solicit and review feedback from course providers, field office(s), sponsor/stakeholder(s), students, and other sources. Work with the training triad to resolve any issues.

f. Review Policy and Course Related Materials. Actively review appropriate directives, policies, JTA, and other course-related materials to determine if course materials need to be revised.

g. Determine if a Course Revision or Special Evaluation is Warranted. Review all available data to determine if a special evaluation or course revision is warranted. If a special




evaluation is required, coordinate with AFS-530 to have an appropriate evaluation scheduled. If a course revision is believed to be appropriate:

(1) Determine if the proposed revision is major or minor. (Refer to the definitions of minor and major revisions in Chapter 7.)

(2) Coordinate course revisions with the stakeholders to ensure they are aware of planned revisions, and verify that their issues and concerns are satisfied.

(3) If a major revision requires a workgroup, the CM will participate in the workgroup and have responsibilities similar to those for a new development effort.

(4) If minor revisions are required for Academy courses, notify AFS-500 of the desired changes and coordinate with the Academy’s course manager to implement the changes.

(5) If minor revisions are required for courses not delivered or contracted by the Academy, work with AFS-500 to implement the changes.

h. Provide Annual Status Report. Provide an annual status report on each course to your division manager by October 30 each year (see appendix D). Forward a copy of the report the appropriate stakeholders, and to AFS-530 for tracking purposes. AFS-500 will provide a summary of these reports to the Director, Flight Standards Service (AFS-1).

(1) AFS-530 provides a yearly CM information dissemination service. The purpose of this program is to provide evaluation data to AFS CMs to assist them in preparing their annual course status report.

(2) An e-mail is sent to the CMs around the month of July each year, providing evaluation data that reflect the percentage of positive responses (favorable rate) for the current fiscal year. This information is valuable to the CMs in responding to the call for evaluation request, distributed around the same time as the information dissemination.

4. Other CM Responsibilities.

a. Keep the Sponsor or Stakeholder Informed. Keep the sponsor and stakeholder(s) informed of issues related to the development or revision of the mentor’s assigned courses.

b. Resolve Course Issues if Possible. Coordinate and resolve course issues at the working level, and only elevate them if consensus cannot be reached at the lowest level.

c. Provide Annual Status Report. Provide an annual status report to the sponsoring manager, the stakeholders, and the training triad. Submit this report (Appendix D) to your division manager and AFS-530 by October 30 each year.

d. Participate in Meetings. Participate in the division’s Quarterly Program Reviews (QPR) and the Curriculum Oversight Team (COT) meetings with AFS-500, as required.




e. Foster Working Relationships. Facilitate an effective working relationship among sponsor, stakeholders, AFS-500, the Academy, and other training providers.

f. Seek Assistance From an SME When Necessary. From time to time, a CM may need to rely on a SME for assistance. This is particularly important when a mentor cannot attend a scheduled formal evaluation.

g. Evaluate Feedback and its Relevance to Course. Evaluate feedback and assess its relevance and the impact it may have on your course. In one situation, feedback might indicate a need to revise training; in another, it might indicate a need to clarify FAA policy.

h. Delegate Work. Identify a designee to act on your behalf if you will be unable to carry out your CM responsibilities for some period.




Chapter 4. Course Mentor’s Coordination Activities

1. Coordinating With the Sponsor of a Course.

a. Coordination with the Sponsoring Manager. The CM keeps the sponsoring manager informed of the course’s status and any pertinent issues, and provides an annual status report. Ensure that you fully understand the manager’s vision for the course and any future revisions that he or she believes might be necessary.

b. Internal Coordination. You may need to consult with the manager if you believe that other activities within the sponsoring organization or in other parts of AFS are inconsistent with what the course teaches. Maintaining a broad awareness of the issues, new initiatives, and planned changes to FAA regulations and guidance will assist you in keeping your course(s) current. Effective internal coordination will minimize the likelihood that a course is revised in a way that is inconsistent with other activities taking place in the sponsoring organization. Identify and resolve any inconsistencies.

c. Assessing the Impact of New or Revised Directives. In reviewing the flow of technical information that might impact your assigned course, you will determine if the information requires changes to the course. If there is a need for changes, you will follow the process discussed in Chapter 7. If you believe that the information will not directly impact the course but may require the development of a new training course, you should notify your sponsoring manager. Your manager will evaluate the new or revised directives to determine if their organization or another sponsoring organization might need to develop a new course. The following flowchart (Figure B-1, How to Review and Process New or Revised Directives) provides a pictorial description of how to review and process new or revised directives.




Figure B-1. How to Review and Process New or Revised Directives




2. Coordination with Outside Organizations.

a. Stakeholder Organizations. Identify points of contact (POC) for your assigned course in each stakeholder organization and develop an effective working relationship with those individuals. Inform the stakeholder of pertinent issues. Consider issues and concerns the stakeholder’s POC raises. Understand each stakeholder’s perspective so that you can represent his or her positions in the initial development effort and any future coursework. Stakeholders and triad members should receive copies of your annual status report that you provide to the sponsor.

b. Other Organizations. Regional and field offices can provide valuable feedback related to your course. Be aware of any input or issues that industry, international, military, or other FAA organizations provide.

c. AFS-500 Quarterly Program Reviews (QPR). If your assigned course will be discussed at an AFS-500 QPR with your sponsoring organization, you may need to participate.

d. AFS-500. For non-FAA Academy resident courses, AFS-500 will assign a course manager.

3. Coordination with the FAA Academy.

a. Course Administration and Monitoring. For Academy resident courses, the Academy will appoint a course manager. The course manager is responsible for managing the delivery and administrative aspects of the course. The course manager will assist the CM in monitoring the currency of the course, and coordinating/recommending changes to the course material.

b. Enhancing Course Instructor Knowledge. The Academy has a program that involves instructors in regular AFS field and headquarters activities to enhance their understanding of the courses they are charged with delivering. As the CM, your assistance in identifying areas that would enhance the instructors’ knowledge in a particular area would be appreciated. Your ideas and/or suggestions should be passed to the appropriate Regulatory Standards Division, AMA-200, branch manager.

4. Labor Organizations.

a. SME Selection. Labor organizations play an important role in FAA’s training processes. When subject matter experts (SME) are selected to participate in the development of training courses, consult the appropriate labor organizations(s). The AFS-500 project manager coordinates with the key stakeholders (CM, appropriate labor organization) so that proper procedures are followed in accordance with AFS-500-003, Develop Curriculum. Also, the request for participation on a training workgroup must be communicated to the participant’s manager.

b. Sensitivity to Bargaining Agreements. It is important to understand that labor union bargaining agreements may impact how the training content is communicated. Close coordination between the members of the training triad, and possibly the respective labor union representative(s), may help avoid potential conflicts.




Chapter 5. New Course Development

1. Creating Training Development Requests (TDR).

a. Development of a TDR. You may be actively involved in helping the sponsoring organization develop a TDR. You should use TDR forms available on the AFS-500 Web site. Your name as course mentor (CM) and the names of any stakeholder POCs should be included on the TDR.

b. Identifying Potential Course Stakeholders. In developing a new TDR, think about which organizations might be stakeholders in the proposed new course. Once potential stakeholders are identified, contact the appropriate division manager to discuss the proposed course. Request that the manager authorize someone to be the stakeholders’ POC for that course. Coordinate the development of the TDR with the stakeholders’ representatives to obtain their input and perspective. If disagreements exist between you and any stakeholder POCs, elevate the issues to the sponsoring manager for resolution before finalizing the TDR.

c. TDR Submittal. Once the new TDR is completed and signed by the manager of the sponsoring organization, forward it to AFS-500 for processing.

2. Initial Coordination with AFS-500.

a. TDR Prioritization. Upon receipt of a TDR, AFS-500 coordinates with the sponsor’s CM to gather any additional information needed to address the request. AFS-500 provides a copy of the TDR to the appropriate COT for concurrence. The AFS-500 project manager, using input from the assigned CM, prioritizes training development projects and sets a start date in agreement with the CM.

b. AFS-500 Project Manager. Once an AFS-500 project manager is assigned to the TDR for development, the project manager will contact the CM and the training provider to discuss the development effort.

3. Development Process.

a. Training Development Process. The development process that will be used for resident and Web-based training is outlined in AFS-500-003, Develop Curriculum, and the associated AFS-540 Policy and Procedures Handbook and Style Guide. Before participating on a course development workgroup, you should review these documents. There may be occasions that a standard development template might not meet the needs of the workgroup. If there are reasons to deviate from the standard templates, refer them to the training triad for discussion.

b. Important Design/Development Events. You will make decisions in design and development events. These events include:

Identification of SME selection criteria for the workgroup. Creating task lists. Creating the CDG.




Creating a lesson plan and if needed, storyboard development and review. Performing a course walk-through. Facilitating an operational tryout (resident training only). Creating a prototype.

c. Additional Considerations for FAA Academy Out-of-Agency Training (OAT) Courses. In collaboration with the sponsor, AFS-500 may determine an OAT course is the best way to meet the training need. The Contracts and Program Administration Branch, AMA-260, is involved in procuring the new OAT training. CMs will work with AFS-500 to establish a development workgroup to assist AMA-260 in the development of the contract statement of work.

d. Developing Performance-Based Training. The FAA is committed to developing effective performance-based training tied to job tasks. A critical part of the development effort is to ensure that the job tasks addressed in new training reflect current work processes. The Flight Standards Information Management System (FSIMS) contains a listing of job tasks. AFS-500 ensures that the development workgroup reviews available job task information to identify which job tasks need to be addressed in the course being developed.

e. Other Considerations. Course development/revision workgroups consider what delivery methods would be most effective for a proposed course. For example, some distance learning methods might give students access to course material before they attend a class. A blended delivery might be appropriate when some course material is compatible with distance learning and some needs to be delivered in a classroom. The appropriateness of integrating new course performance objectives into the AFS OJT program should also be considered.

4. Development Roles.

a. CMs. Details of the CM’s role are in Chapter 3 of this appendix.

b. Training Triad. The training triad representatives on the development workgroup act as the group’s steering committee. The training triad guides the group throughout the development process and assists the workgroup to achieve consensus on development questions that may arise. The training triad forwards for resolution to the sponsoring manager and AFS-500 manager any development issues upon which the workgroup cannot agree.

c. AFS-500. Details of the roles of the AFS-500 project manager, instructional systems designer, and course developer are in the AFS-500 Policy and Procedures Handbook. AFS-500 is responsible for the training development process and issues related to the instructional system design of training, including delivery method.

d. SMEs. Subject matter experts (SME) may include any person with knowledge and experience essential to producing a final product that meets the training needs of the sponsor. SMEs may be requested to do the following:

(1) Participate in the analysis of job tasks and technical material.




(2) Provide technical expertise to ensure course content is accurate.

(3) Assist in the development of the technical instructional and participant materials (includes reviewing and providing materials).

(4) Participate in the walk-through presentation of the course materials.

(5) Assist in the update and/or revision of existing course materials.

(6) Instruct the course when applicable. Future instructors may attend development activities at the discretion of AFS-500.

e. Training Provider. For courses that will be provided by the Academy, a Regulatory Standards Division, AMA-200, representative will be assigned to help the workgroup. Usually the representative will be the course manager for the proposed course. As a member of the training triad, the Academy representative’s role in the workgroup is to help the workgroup develop the best training possible. The Academy representative brings to the workgroup knowledge of training delivery techniques, and knowledge of technical issues raised by AFS students or other Academy instructors.




Chapter 6. Course Monitoring

1. Overview.

a. Purpose. Once a course is successfully prototyped and signed off by the course mentor (CM), it becomes a permanent part of the Flight Standards Service (AFS) training program. It comprises existing job tasks, regulations, policy and guidance for covered programs, and performance objectives. Unfortunately, these components do not remain static; they change over time. Ensure that your assigned course is kept current and meets the overall needs of the AFS by monitoring all aspects of the course. Stay aware of changes in the technical work the course covers. By monitoring the course, you can see how the material is delivered, gauge student interaction, and determine whether the course continues to meet the needs of the sponsor.

b. Course Indicators. There are varieties of course indicators available for monitoring the effectiveness and relevance of your course. You can review course material, monitor the course itself, participate in an evaluation of the course, and review evaluation reports. You can compare your observations with those of other training triad members and evaluate information from other sources.

c. Flowchart. The following flowchart (Figure B-2) depicts the steps involved in monitoring and revising inspector training. In places, the flowchart references going to A, B, or C. These are elements of Figure B-2.




Figure B-2. Monitoring/Revising Inspector Training

2. Monitoring Roles.

a. CM. As CM, you bring the sponsor’s perspective to the monitoring process. Be aware of problems that regions and field offices experience in the program area(s) your course covers.




Evaluate feedback about your course. In one situation, feedback might indicate a need to revise training; in another, it may indicate a need to clarify FAA policy.

b. SME. In some situations, subject matter experts (SME) may be requested to assist a CM in monitoring a course. While a CM is expected to observe assigned course(s), a CM might not have all of the expertise needed.

c. AFS-500. AFS-500 supports employee training in many ways, including providing assistance to CMs and other members of the training triad in monitoring and evaluating courses.

d. Training Providers.

(1) FAA Academy—Resident Training. The Academy has a well-established relationship with AFS. For decades, the Academy developed and delivered the majority of AFS training. Most Academy instructors are AFS aviation safety inspectors (ASI) with field experience, and are trained to develop and deliver AFS training. The Academy continues in that relationship today by working with specific policy offices such as AFS-200, AFS-300, AFS-800, etc.; course sponsors; CMs; and AFS-500 to provide support for employee training.

(2) FAA Academy—Out-of-Agency Training (OAT). The training triad works through the AMA-260 contracting officer technical representative (COTR) when OAT is involved. For example, student end-of-course evaluation data for OAT courses are available through the AMA-260 COTR.

(3) Other Training Providers. When the Academy or an Academy-contracted OAT provider does not deliver your course, coordinate with AFS-500.

3. Monitoring a Course.

a. Current Material. Be sure you have the current job task information and associated course materials before you begin. Do not assume that previous copies of the course material are current. Course material changes as revisions are made.

b. Course Observations.

(1) How Often to Observe a Course. In determining how often to observe a course, consider the following:

(a) How often the course is offered.

(b) How recently the course was evaluated.

(c) How rapidly technical material is changed.

(d) Any trends in comments in End-of-Course Evaluations.

(e) Any trends in Post Course Evaluations.




(f) Any other feedback about the course.

(2) Observation Process.

(a) Scheduling. Course observation sessions are scheduled in advance to ensure classroom space and training materials are available. Coordinate scheduling with AFS-530, which will contact the appropriate Academy course manager, Academy COTR, or other training provider. Your observation session includes a verbal debriefing session that also should be scheduled in advance with the appropriate person(s) representing the training provider.

(b) Checklist. AFS-500 has developed a Flight Standards Observer Checklist for Subject Matter Experts that can be used when observing a course to document observations of a course in a thorough and systematic way. This checklist is located in Appendix C and on the CM Web site. Contact AFS-530 for help in using this checklist.

(c) Debriefing. Conduct a verbal debriefing before leaving the provider’s facility highlighting your observations, as follows:

1. FAA Academy Resident Course. Debriefing should include the course manager and AFS-530.

2. FAA Academy OAT Course. Debriefing should include the AMA-260 COTR and AFS-530. This debriefing is normally conducted by telephone because the COTR and AFS-530 are located in Oklahoma City, while the OAT provider could be located anywhere.

3. Courses Not Delivered or Contracted by the FAA Academy. Debriefing should include AFS-500’s course manager and a representative from AFS-530. AFS-500’s and AFS-530’s participation may be via telephone.

c. Formal Course Evaluations (Routine or Special). AFS-530 conducts formal evaluations of all AFS training courses every 3 years. A formal course evaluation is a more rigorous look at a course than observations, or normal end-of-course student evaluations. If you believe that there is a need to do a special formal course evaluation outside the recurring cycle, contact AFS-530 and explain your reasoning. Formal course evaluation teams look at design, delivery, and other aspects of a course to determine if it is effective and continues to meet the needs of classroom students, the sponsor, and stakeholders. The evaluation team develops an evaluation report that documents the team’s findings and recommendations, including any course revisions the team believes are necessary. You will be involved in post-evaluation discussions. When the evaluation team approves the final course evaluation report, it is distributed to the training triad.

d. Automated Student End-of-Course Evaluation Reports. The Academy has a standard student end-of-course evaluation system that collects data and student comments at the end of every class. This information is collected for Academy-resident training and Academy OAT. Information is entered into a database and comments are stored for tracking. Upon request, AFS-530 will provide you a report on your course so you can monitor students’ reactions. In the case of an OAT course, the training provider mails the evaluation data to the Academy for entry




and processing. This means that there will be a slight delay in getting the end-of-course evaluation report.

e. Post Course Evaluation Data. As part of the Post Course Evaluation process, each student and their FLM receives an evaluation via e-mail 3 to 6 months after the student completes training. This evaluation asks a number of questions about the relevance of the training course to job tasks, and how much the training has enhanced the student’s ability to perform their job functions. In responding to the Post Course Evaluation, students or their FLMs can indicate that they want to be contacted for further discussion. AFS-530 conducts Post Course Evaluations and records the information in the FAA Post Course Evaluation System. For information on how to retrieve this information, see the Quick Start Guide in Appendix C. If you need additional information about this system or wish to discuss your course information, contact AFS-530.

f. FAA Academy Resident Course Manager. A course manager can give you perspective on the accuracy of course material, the kinds of feedback the course has received from students, and feedback the course manager has received from the other instructors. This is a way to get a quick reading of the effectiveness and relevancy of a course. A course manager will also be able to share with you student feedback that might indicate instances of course content that does not follow standard policy, or is interpreting guidance in a nonstandard way. A course manager reviews student end-of-course evaluations and conducts feedback sessions with students at the end of every class.

g. FAA Academy OAT Course. Before contacting a training provider delivering an OAT course, contact the assigned AMA-260 COTR. Discuss with the COTR the nature of your proposed discussion with the contractor. The COTR will give you the name and phone number of the contractor’s organization point of contact, and may include instructions relating to terms of the contract or other restrictions that must be followed when communicating with the contractor. A COTR may want to participate in any conversation between you and a contractor.

h. Non-FAA Academy Training Provider. Generally, AFS-500 designates a course manager who can provide useful information about a course. Contact AFS-500 to determine who the training provider is and what procedures should be followed when contacting the provider and discussing the course.

i. Feedback from the Regions and Field Offices. You may receive and/or solicit feedback from regions and field offices about problems in the program areas covered by your course. You may receive direct feedback from students and AFS regional and field managers. In some cases, feedback might suggest topics that you can consider for inclusion in the course. In other cases, feedback might identify concerns with how course material is being interpreted by the students.

j. Negative Feedback. Negative feedback comes in many forms. If you receive serious complaints involving actionable issues, you should immediately convey those complaints to an appropriate management official(s). When you want assistance in evaluating the root cause of a complaint, you may contact AFS-530. AFS-530 is not involved in either the development or the delivery of any AFS course; it can serve as an objective third party to assist in the evaluation.




4. Training Triad Coordination. All three members of the training triad monitor the effectiveness and relevancy of AFS courses. You are encouraged to discuss your course with other members of the triad to gather input. The reverse also applies. Keep other members of the triad informed of your views of the course’s effectiveness and relevance.




Chapter 7. Course Revisions

1. Overview. Courses should be readily revisable to reflect a changing environment. The training triad works together to ensure that courses are kept relevant and useful.

2. Identifying Needed Changes. A number of factors can affect the need to revise a course, including:

Changes in sponsor’s direction for the course/program. Changes in technical documentation. Changes in job tasks/functions. Formal evaluations. Course mentor (CM) observations. Course feedback. Course revision request from any concerned individual. Changes in policy or regulations. Congressional mandates.

3. Course Revision—Roles.

a. CM. As the sponsor’s representative, a CM ensures that a course is technically up-to-date and continues to meet program design objectives and the needs of the target audience. How you work with the training provider and AFS-500 depends on whether a revision is major or minor, and on who is providing the training.

b. AFS-500. AFS-500 coordinates instruction systems design (ISD) and administrative support to assist with required revisions. AFS-500 ensures that AFS courses are consistent with current ISD techniques and meet AFS standards for course development, and is responsible for the course design and delivery methods. In developing a wide range of AFS training, AFS-500 can share with the CM insight on new elements being developed for other courses that may enhance the proposed revision.

c. Training Providers.

(1) FAA Academy Resident Training. The Academy’s course manager collects and reviews student end-of-course evaluations and other input provided to the Academy. Course managers often have valuable insight relating to a course’s content and suggestions for proposed course revisions. When a course manager identifies a needed revision, they should discuss their concern with the CM. Following this discussion if the CM believes a major revision is appropriate, they will coordinate a course revision request (CRR) with AFS-500 and forward suggested changes to the sponsoring manager for review and approval. If the proposed revisions are minor, the CM and course manager will notify AFS-500 and coordinate necessary revisions in accordance with paragraph 5. It is important to note that all verbal approvals to revise a course should be followed up in writing. In most cases, an e-mail describing the approved revision and forwarded to the course manager and AFS-500 will be sufficient.




(2) FAA Academy OAT. There are no Academy course managers for OAT; however, OAT courses have an FAA-appointed contracting officers technical representative (COTR) from AMA-260. Speak with the course’s COTR to determine if the course is an AFS-approved training program. If so, revisions to the course are coordinated with the appropriate FAA office. If not, coordinate any course revisions with the COTR and AFS-500.

4. Determining If Revisions are Minor or Major.

a. Guidelines for Revision Processes. The triad recognizes two basic revision processes, major and minor. The guidelines below are based on such things as impact on the course and impact on resources. In some situations, the line between a major and minor revision may not be clear. In those situations, members of the triad will discuss the nature of the revisions and reach consensus on whether the proposed revision is major or minor.

(1) Guidelines for Major Revisions. A revision may be considered major when one or more of the following conditions occur to an approved training course:

(a) The length of the course increases by more than 4 hours.

(b) The training outcomes and/or instructional objectives are added or deleted.

(c) The design of the course is changed (adding or deleting exercises, simulations, etc.).

(d) The source or method of training delivery changes (classroom, computer-based instruction, interactive video teletraining, Web-based training, etc.).

(2) Guidelines for Minor Revisions. A revision may be considered minor when it does not fall into the category of a major revision. Routine revisions are necessary to maintain the currency of the course. Minor revisions should not affect the training outcomes/instructional objectives, course length, or method of course delivery.

b. Consider the Impact of a Proposed Course Change. In addition to the above guidelines, the triad needs to consider the impact of the proposed course change before going through the time- and resource-intensive process of a major revision. Generally, a major revision will be a request to add, remove, or revise a significant amount of course material. The triad must consider the following when adding material to or removing material from a course:

(1) Is the material pertinent to the overall goal of the course?

(2) Will the material affect the achievement of the course objective(s)?

(3) Is any new material duplicated in other courses, or contradictory to other courses?

(4) Does the material fall within the job responsibility of the course’s job tasks?

(5) Is the material superfluous to the job tasks of the course?




c. Reaching Consensus on Major or Minor Revisions. If the triad is unable to agree about whether a change is a major or minor revision, the issue will be elevated to the respective sponsoring division manager and AFS-500 division manager for resolution.

5. Revision Coordination.

a. FAA Academy Resident Courses—Minor Revisions. The course sponsor, AFS-500, or the training provider may request a major course revision by completing and submitting a CRR form, available on the AFS-500 Web site. Other people who identify a need for course revision may request a revision by submitting a draft CRR to the course mentor. If the CM concurs with the draft CRR, it must be forwarded to the sponsoring division manager for approval, and then sent to AFS-500 for processing. If the person submitting a request believes the revision to be minor as defined in Quality Management System (QMS) process AFS-500-004, Revise Existing Curriculum, the request may be submitted to the CM by way of e-mail with an explanation of the proposed revisions. Minor revisions may also be submitted to the CM by submitting an annotated copy of the affected lesson plans. Proposals for minor changes will be sent to the CM with a copy to AFS-500. Major and minor revisions will be made in accordance with the approved course revision process, as shown in QMS process AFS-500-004.

b. Other Providers. AFS-500 sometimes uses training providers other than the Academy or an Academy OAT provider. When working with those providers, the CM will coordinate all course revisions through the AFS-500 course manager.



1/6/12 3140.20BAppendix C

Appendix C. Quick Start Guide to Using the FAA Post Course Evaluation System Web Site

1. Purpose. This is a reference guide on how to obtain Post Course Evaluation data. To assist course mentors (CM) in their roles and responsibilities of the newly developed Flight Standards Course Mentor Program, access has been granted to the FAA Web-based Post Course Evaluation System. The information in the system is offered as a tool in determining the overall “health” of particular course(s).

2. Background. The FAA Post Course Evaluation System is a Level III evaluation (Kirkpatrick Model of training) that provides measurement data on whether the employees are applying the training to their job, if the training has improved work performance, and if the work environment supports that transfer of knowledge. The automated system is a Web-based instrument that allows data input from remote locations. The Level III evaluation data are stored within the Web-based system and is used as an aid in course evaluation, revision and design, or re-design for all Flight Standards Service (AFS) training.

a. Student Training Evaluation.

(1) Students and their FLMs are notified, via e-mail that they have the opportunity to go to the Web to participate in the evaluation. Notifications are delivered to the participants approximately 60–90 days after completion of training. The delay in delivery of the evaluation is to allow ample time for the student to use the skills obtained in training on the job, thereby yielding accurate and credible data to measure the extent learning has been applied to the job.

(2) The e-mail notification provides a brief purpose of the questionnaire and a confidentiality statement, along with a direct link to the questionnaire. Participants receive the opportunity to provide comments, their name, e-mail address and/or phone number, should they prefer to be contacted about their response. The comments submitted on-line by the participants are stored electronically within the system. Participation is voluntary and is in compliance with the current edition of FAA Order 3140.20, Flight Standards Service National Training Program, Public Law 109-115, and Section 508 of the Americans with Disabilities Act of 1964. The responses given by the employee remains confidential and anonymous within the system unless the participant requests to be contacted about his/her responses or comments.

b. System Validation. This system has been coordinated and has received full acceptance with Professional Aviation Safety Specialists (PASS) and the American Federation for State, County and Municipal Employees (AFSCME).

3. Current System.

a. Data Collection. The FAA evaluation system began officially collecting data in June 2003. Currently, the overall return rate is 50 percent without additional followup/contact.

b. Implementing and Administering the Evaluation System. In October 2004, the FAA Academy implemented its evaluation system to include an automated End-of-Course Evaluation and Post-Course Evaluation. AFS and the Academy have agreed to combine both systems into

C-1UNCONTROLLED COPY WHEN DOWNLOADED



one, in which the Academy will administer both the End-of-Course and Post-Course Evaluations for all lines of business at the Academy including AFS. This includes using e-mail to distribute questionnaires to AFS employees and their FLMs, and storing the data collected within a central Web site. AFS-530, the Quality Assurance Branch, will remain the point of contact for responses to Post-Course Evaluations offered to AFS employees.

4. Overview—Course Monitoring. As discussed in Chapter 3, CMs are responsible for monitoring their course(s) to insure they are up to date and meet the course sponsor’s program requirements, and the needs of the target audience. The Web-based Post Course Evaluation is one of the tools available to the mentor in determining the overall health of his/her course(s). The following are step-by-step instructions for accessing the Post Course Evaluation System and information contained in the Reports section of the database.

5. Accessing the Post Course Evaluation System. You may access the FAA Post Course Evaluation System from any computer that has an Internet connection and a Web browser by entering the following Web address in the browser’s address bar: http://www.academy.faa.gov/afs/admin/

a. Login ID and Password. Your Login ID and Password is the first initial of your first name and your full last name. The password is faa for your first login to the system. The password is case sensitive. Example: Login ID and Password for Vanessa Cobb.

LOGIN ID: VCOBB PASSWORD: faa

b. Change Passwords. You are encouraged to change your password upon your first visit to the system to conform with security procedures and policies. Instructions on how to change your password can be found in this Appendix page C-9 and C-10. Enter your user ID and password in the space provided. Remember the password is case sensitive.”

c. Contact People. If you have trouble accessing the site, please contact the manager, Quality Assurance Branch (AFS-530).

Figure C-1. Login Page




Figure C-2. The Post Course Evaluation System Home Page

d. Logging In. When you successfully log on to the system, the first page you will see is the home page (Figure C-2). At the left of the page, you will see a blue menu bar. This is the main menu bar for accessing the various features of the site. When you click on one of the available choices, such as “Viewer,” your browser will open that page.

Figure C-3. Editor and Viewer

6. Viewer. This provides the questions presented to the students and their FLM, along with the course(s) you have been assigned. When you select “Questions” in this drop-down menu, you will see the following screen (Figure C-4).

7. Questions. Basic questions are included in all course evaluation questionnaires, along with course-specific questions for the selected course. For example, currently, all courses in the database include the basic questions. The on-the-job training (OJT) Instructor Training Course 25702 has an additional 16 questions specifically for OJT in the field. Computer-based instruction (CBI) course evaluations include basic questions and on additional course specific questions.

Figure C-4. Question Viewer Screen

8. Courses. When you select “Courses” from the “Viewer” drop-down menu, the course(s) assigned to you will be displayed.




Figure C-5. Courses View

a. By clicking on the course number from the previous screen, you will see the following (Figure C-6):

Figure C-6. Course Profile Information

b. This screen is for information purposes only. It contains the course title, status of class schedule, mentor’s name, course manager’s name, and the waiting period from the completion of the training to the distribution of the evaluation to the student and their FLM.

9. Reports. This selection provides the results of the evaluation data submitted by the participants in the following categories: Survey, Comments, Notification, Report Creator, Class Comparison; and under the “Favorable Rate Report” section, Course Comparison. Detailed descriptions and copies of the individual screens follow.




Figure C-7. Reports View

a. Survey Report.

(1) This report provides the number of surveys that have been completed for the mentor’s course(s). (To see the number of surveys by course/class, select the “Show course/class report” link.) The responses are grouped by student/FLM, course, and course/class. Each group provides the percentage completed as compared to the total surveys. You can also view the percentage results in a bar chart format by selecting the “Show Graph” button at the bottom of the report.

Figure C-8. Survey Reports View

(2) By selecting “Display All Surveys” the individual questions will be displayed for both students and FLMs, along with the aggregate response for each question.




Figure C-9. Survey Analysis View

Note: These are the total responses for all course(s) you are assigned. You can access individual course statistics by selecting the desired course number under “Group by Course.”

b. Comments Report. This report displays the total count of comments for the course(s) you are assigned. The comments are grouped by category and the number of participants requesting to be contacted.

Figure C-10. Comments Report View

(1) When one of the categories is selected, all of the comments for that group are listed on a separate screen.




Figure C-11. Comments List Screen

(2) Clicking the ID number in the left column will display the audit trail AFS-530 uses to document conversations that have taken place for that particular comment. The participant’s response to the evaluation is also included on this screen, as shown on the figure below.




Figure C-12. Participant’s Evaluation Response

Note: As the course mentor, you are encouraged to contact participants that provide contact information to clarify any portion of the evaluation submitted.

c. Notification Report. This report displays the number of notifications issued and completed for your course(s) (Response Rate). The percentage of completion is also included in this report and grouped by the date of issue and by course/class.

Figure C-13. Notifications Report




d. Report Creator. This tool generates special, ad-hoc reports of the categories shown in Figure C-14. Each type of search contains a step-by-step process for ease and simplicity. When “Survey,” “Comments,” or “Notification” is selected, a report is generated specific to the data selected for each screen.

Figure C-14. Report Creator View

e. Class Comparison.

(1) By selecting a course and the appropriate fiscal year from this screen, a report will be generated (see Figure C-15).

Figure C-15. Class Comparison Screen

Figure C-16. Class Comparison Data




(2) The graph above represents a comparison of how the student and FLM rate the course from class to class based on the course content, which is tied to question 4 in the evaluation. The percentage is calculated on the total average participation.

10. Search. This function allows you to search for specific data stored in the system. Follow the prompts through each function to generate the required report.

Figure C-17. Search Function Screen

11. Password. The first time you log on to the site, you are encouraged to change your password. The screen below is displayed be selecting “Password” on the left toolbar. Fill in the blanks as presented, and select “Submit” to establish your new password.

Figure C-18. Change Password Screen

12. Logout. When you are ready to exit the system, select “Logout” in the left toolbar. You can then close your Web browser, or select “FAA Post Course Evaluation System” to log on to the system again.

13. Sample Student and FLM E-mail, and Respective Post Course Questionnaires. Page two of this guide discusses the procedure used to notify students and their first-line FLMs of the opportunity to participate in the Post Course Evaluation. The following figures contain samples of the student and FLM e-mails and respective post-course questionnaires.




Figure C-19. Sample E-mail—Student

Figure C-20. Student Questions




Figure C-21. Sample E-mail—FLM

Figure C-22. FLM Questions



1/6/12 3140.20BAppendix D

Appendix D. Course Mentor Annual Course Status Report Format

Note: (Due October 30 each year to your division manager)

1. Course Title and Number:

2. Are you currently assigned as Course Mentor? Yes No

3. (If no, please list the name of the correct Course Mentor.) Yes No

4. Do you have a course evaluation schedule this fiscal year?

5. End-of-Course Evaluation favorable rate:

6. Post Course Evaluation favorable rate:

7. Coordination of Critical Revision: (Briefly describe any critical revisions you need made to the course this fiscal year due to guidance/policy changes, etc. Please remember to coordinate with your course manager and contact AFS-530 at (405) 954-1950.)

8. Other course related issues that would require revision in the coming year (Explain in detail):

D-1

1/6/12 3140.20BAppendix E

Appendix E. Flight Standards Service Observer Checklist for Subject Matter Experts

Using the Observer Checklist

1. Observer Checklist for Subject Matter Experts.

a. The following are the procedures to follow when using the Flight Standards Service Observer Checklist for Subject Matter Experts to observe training. Following the procedures is a copy of the checklist, which can be photocopied as needed. This checklist is one source of data for evaluating training. It may be used in conjunction with other forms of evaluation.

b. The checklist is designed to help in the evaluation of technical accuracy and course content. The checklist should be used while observing course delivery. The observer will evaluate 10 criteria and rate them as either satisfactory (3), needs improvement (2), or unsatisfactory (1). Space for the observer to make specific notes or comments is provided in the far right column of the checklist. It is recommended that an individual checklist be used for each lesson or module of training.

2. Pre-Observation. Before observing the training, the observer should prepare by doing the following:

a. Contact AFS-530, Quality Assurance Branch, who will communicate with the Regulatory Standards Division, AMA-200.

b. AMA-200 will contact the appropriate course manager to coordinate the observation.

c. It is recommended that you reproduce the checklist to assess each lesson in the course curriculum.

d. Review the checklist to ensure that all criteria to be rated are clearly understood.

e. If available, course materials will be provided to you by the AFS-530 training analyst who is also assigned to evaluate the course. Review the materials to increase your awareness of training objectives and materials.

3. Observe the Training. During the actual observation, use the following protocol.

a. Bring the checklist, writing paper, a writing instrument, and instructor and participant materials, if they have been provided to you.

b. Arrive early at the training location. Coordinate with the instructor and the analyst/evaluator from AFS-530 to ensure that you have all of the materials necessary for the course observation and to clarify your role/function in the evaluation process.

E-1UNCONTROLLED COPY WHEN DOWNLOADED



c. The instructor will tell the students where you are from and that you will be observing the training, not the participants. The instructor also will ensure that observer seating allows for full viewing and hearing of all classroom activities. Make sure to not interfere with the normal class proceedings.

d. Take notes during the training in the space provided on the checklist. At the end of the training, record your observations on the checklist. Provide as much detail in the comment column as possible. The last page provides space for additions and deletions to course content and additional comments.

4. Post-Observation. After the observation of training has been completed, the following steps should be taken.

a. Ensure that all comments on the checklist are legible, coherent, and complete.

b. Thank the course manager and the instructors for their cooperation. Before your departure, schedule time to debrief the AFS-530 evaluator as to your specific observations and overall assessment of the course.

c. After returning to your duty station, if you have any additional comments about the course, send them to the AFS-530 evaluator via e-mail no later than 10 days after your course observation. Your comments will be incorporated into the evaluation report and forwarded to the course sponsor.

Figure E-1. Flight Standards Service Observer Checklist for Subject Matter Experts

a. Course Status (Indicate with a check mark): Prototype Established

b. Course Title:

c. Course Number:

d. Class Number:

e. Lesson Title:

f. Date Observed:

g. Name of Observer:

h. Observation Site:

i. Directions: This checklist is for subject matter experts to use while observing course delivery. It contains criteria pertinent to currency/accuracy of course content, FAA and Flight Standards regulations, policies and procedures.




j. Ratings. Rate each item by circling the appropriate number. An Unsatisfactory rating (1) or Needs Improvement rating (2) must have an accompanying explanation in the comments column. There is a section for additional comments at the end of the checklist. Depending on the length or extent of your comments, you may use additional paper.

k. Checklist Rating Scale: 1 = Unsatisfactory; 2 = Needs Improvement; 3 = Meets Requirements

Criteria Rating Comments

Content reflects current FAA/Flight Standards orders, regulations, policies, and procedures.

1 2 3

Content is accurate, complete, and relevant to current job requirements and technology.

1 2 3

Content is organized in a logical learning sequence.

1 2 3

Activities such as practice exercises, labs, case studies, and simulations, etc., give participants the opportunity to practice/apply new information/skills.

1 2 3

Solutions to learning activities, practice exercises, and written examinations convey accurate information in accordance with current regulations, policies, and procedures.

1 2 3

Participant/Student Guide facilitated comprehension of technical content/information.

1 2 3

Duration of the lesson is appropriate for the subject matter.

1 2 3

(WBT) The graphics were beneficial to the instruction.

1 2 3

(WBT) The navigation tools were effective (e.g., minimal delay for screens, links, buttons, icons).

1 2 3

l. Additional Content. What content, if any, should be added to this course? (Please cite the regulatory and/or procedural doctrine that supports the additional content.)




m.Deleted Content. What content, if any, should be deleted from this course? (Please justify your reasons for deletion of specific content.)

n. Additional Comments. (Please be specific.)



1/6/12 3140.20BAppendix F

Appendix F. End-of-Course Evaluation

Figure F-1. End-of-Course Evaluation

F-1UNCONTROLLED COPY WHEN DOWNLOADED


U.S. Department of TransportationFederal Aviation Administration

Directive Feedback Information

Please submit any written comments or recommendations for improving this directive, or suggest new items or subjects to be added to it. Also, if you find an error, please tell us about it,

Subject: Order 3140.20B

To: Directive Management Officer, ______________________________

(Please check all appropriate line items)

An error (procedural or typographical) has been noted in paragraph _____________ on page _______.

Recommend paragraph ________ on page ________ be changed as follows:(attach separate sheet if necessary)

In a future change to this directive, please include coverage on the following subject(briefly describe what you want added):

Other comments:

I would like to discuss the above. Please contact me.

Submitted by: __________________________________________Date: _______________

FTS Telephone Number: _____________________ Routing Symbol: ___________________

FAA Form 1320-19 (8-89)

order 3140.20b - flight standards information system …fsims.faa.gov/wdocs/orders/3140_20.doc ·...

Documents