NCOA Leadership Immersive Training Workshop

NCOES Interim Assessment ReportJohn Sparks, Director, INCOPD

26 Jan 2010

Agenda

• Purpose• Data Collection Strategy• Elements Evaluated

– Training Analysis– Learning & Instructional Design– Instructor Strategies– Learning-to-Learn (L2L) Strategies

• The Way Ahead

Needs AssessmentPurpose

• Examine ways to improve NCOES instructor effectiveness

– Are evidence-based learning practices used in NCO course analysis, design, development and implementation?– What root causes prevent us from using evidence-based learning practices?

TrainingAnalysis

InstructionalDesign

InstructorStrategies

Learningto Learn

On-LineSurvey

LessonEvaluations

On-siteInterviews

On-siteObservations

Needs AssessmentData Collection

Element Evaluated

Criteria: • Job-Performance based • Multiple expert SMES with recent (3-6 mos) field experience

Results: • Approximately 30% of instruction is designed without accurate task analysis data

Root Causes:• Access to SMEs is very limited• Task content changes are too frequent to conduct analysis• Workforce is not highly skilled at analysis and does not perform it often• Military developers receive little or no training on task analysis

Training Analysis

Element Evaluated

Criteria:• NCOES Course/Lesson Design Checklist (evidence-based principles)

Results: • Fundamental principles of instruction are not applied in most lessons

– Lessons primarily deliver information to students, lack demonstration and opportunities to practice new information in job relevant situations

Root Causes:• Most courseware revisions are based on existing courseware design rather than on learning principles• Military and civilian personnel are not aware of current learning research• Training developers have limited experience with complex scenario development, technology and multimedia design• Training is not available for training developers

Learning & Instructional Design

Element Evaluated

Criteria:• Observations/NCOES Needs Assessment Instructor Checklist

Results: • Military personnel perform exceptionally well despite limited instructor training , short rotations, and limited knowledge of concepts, principles, and theories of teaching and learning, but……..

Root Causes:• Military instructors are primarily SMEs• ABIC and SGI are not enough; need additional training• Instructors are not trained well enough for administrative functions (e.g. counseling, academic reports, etc.)• Instructor selection process is not always successful

Instructor Strategies

Element Evaluated

Criteria:• NCOES Course/Lesson Design Checklist (evidence-based principles)

Results: • Little or no awareness of L2L strategies

Root Causes:• Not enough training time to consider the bigger picture --development of lifelong learning skills• Until recently, little emphasis was placed on strategies to become better learners• L-2-L training is not available for instructors and designers • Perception that it is the instructor’s responsibility and L-2-L strategies do not need to be included in the lesson design• Students don’t think they are accountable to become better learners

–Poor grades are blamed on the instructor or the courseware

Learning-to-Learn (L2L) Strategies

Way Ahead

Contact Information

Mr. John Sparksjohn.sparks1@us.army.mil(757) 788-5526DSN: 680-5526

Dr. Millie Abellmillie.abell@us.army.mil(757) 788-5530DSN: 680-5530

Ms. Roberta Sparkmanroberta.sparkman@us.army.mil(757) 788-5531DSN: 680-5531

Mr. Gary Rauchfussgary.rauchfuss@us.army.mil(757) 788-5570DSN: 680-5570

Backup Slides

Phase II Site Visits Include:• USASMA• Combat Arms: Fort Sill, Fort Benning• Combat Service Support: Fort Lee, Fort Eustis• Combat Support: Fort Leonard-Wood, Fort

Huachuca, Fort Rucker

Site Visits

Phase I

Develop instruments• Course/lesson checklist• Survey• Instructor observation checklist• Phase II interview questions

Pilot instruments

Phase I Data collection/coding• Evaluate sample lessons• Conduct survey

Phase I data analysis

Revise interview questions

Phase II data collection• Observe instructors• Conduct interviews

Phase II data analysis

Produce reportDevelop action planPhase II

NCOES Front End Analysis Timeline

9 Oct 09

16 Oct

15 FEB

25 Nov

20 Nov

10 NOV

8 MAR

26 MAR

Revise instruments 20 Oct

Course/lesson data call 9 Oct 09

Pilot interview questions 20 Nov

Produce interim report 25 Nov