sbe meeting 1/2012 attachment : lfi 1 · to prepare students for end of course exams ......

32
SBE Meeting 1/2012 Attachment : LFI 1 EXECUTIVE SUMMARY Title: NCVPS Process for Approval of Online Courses: Addendum to Standards for Approval of Online Courses (LFI 2, December 2011-SBE) Type of Executive Summary: Action Action on First Reading Discussion Information Policy Implications: Constitution General Statute #S.L. 2011-145 SECTION 7.22.(h).(2) SBE Policy #GCS-M-001 SBE Policy Amendment SBE Policy (New) APA # APA Amendment APA (New) Other Presenter(s): Mr. Ross White (Executive Director, NC Virtual Public School) Description: Language in Session Law 2011-145 SECTION 7.22 (h).(2) requires local education agencies and charter schools to obtain permission from NCVPS before offering any virtual course for credit other than ones administered through NCVPS. Therefore, it is not legal to offer non-NCVPS courses to any public school student without written permission from NCVPS. Standards for Online Courses were approved by the SBE at the December 2011 meeting. This process clearly defines the scope of online course materials which must be approved by NCVPS. Per e- Learning Commision's requests: (1) NCVPS must be given access to each third party course providing for random, periodic spot-checks and (2) vendors must require students to complete an end-of-course student survey to substantiate student satisfaction and high-quality teaching. Resources: NCVPS and NC DPI staff time Input Process: Input from e-Learning Commission. Stakeholders: LEAs, Students, Parents Timeline For Action: Spring 2012 Recommendations: ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- Audiovisual equipment requested for the presentation: Data Projector/Video (Videotape/DVD and/or Computer Data, Internet, Presentations-PowerPoint preferred) Specify: Audio Requirements (computer or other, except for PA system which is provided) Specify: Document Camera (for transparencies or paper documents white paper preferred)

Upload: phamtram

Post on 17-Aug-2018

213 views

Category:

Documents


0 download

TRANSCRIPT

SBE Meeting 1/2012 Attachment : LFI 1

EXECUTIVE SUMMARY

Title: NCVPS Process for Approval of Online Courses: Addendum to Standards for Approval of Online

Courses (LFI 2, December 2011-SBE)

Type of Executive Summary: Action Action on First Reading Discussion Information

Policy Implications:

Constitution

General Statute #S.L. 2011-145 SECTION 7.22.(h).(2)

SBE Policy #GCS-M-001

SBE Policy Amendment

SBE Policy (New)

APA #

APA Amendment

APA (New)

Other

Presenter(s): Mr. Ross White (Executive Director, NC Virtual Public School)

Description: Language in Session Law 2011-145 SECTION 7.22 (h).(2) requires local education agencies and charter schools to

obtain permission from NCVPS before offering any virtual course for credit other than ones administered through

NCVPS. Therefore, it is not legal to offer non-NCVPS courses to any public school student without written

permission from NCVPS. Standards for Online Courses were approved by the SBE at the December 2011 meeting.

This process clearly defines the scope of online course materials which must be approved by NCVPS. Per e-

Learning Commision's requests: (1) NCVPS must be given access to each third party course providing for random,

periodic spot-checks and (2) vendors must require students to complete an end-of-course student survey to

substantiate student satisfaction and high-quality teaching.

Resources: NCVPS and NC DPI staff time

Input Process: Input from e-Learning Commission.

Stakeholders: LEAs, Students, Parents

Timeline For Action:

Spring 2012

Recommendations:

-------------------------------------------------------------------------------------------------------------------------------------------------------------------------------

Audiovisual equipment requested for the presentation:

Data Projector/Video (Videotape/DVD and/or Computer Data, Internet, Presentations-PowerPoint preferred)

Specify:

Audio Requirements (computer or other, except for PA system which is provided)

Specify:

Document Camera (for transparencies or paper documents – white paper preferred)

-------------------------------------------------------------------------------------------------------------------------------------------------------------------------------

Motion By: ______________________________ Seconded By: ______________________________

Vote: Yes __________ No __________ Abstain __________

Approved __________ Disapproved __________ Postponed __________ Revised __________

-------------------------------------------------------------------------------------------------------------------------------------------------------------------------------

*Person responsible for SBE agenda materials and SBE policy updates: Susan Auton, 807-3435

1201_LFI 01_Attach

1

LFI 1 (January 2012): Standards for Approval of Online Courses

NCVPS Process for Vendor Approval

Introduction

The North Carolina Virtual Public School (NCVPS), in conjunction with the North Carolina Department of Public Instruction (NC DPI), will

implement a process for vendor approval of individual online courses. This is a process for approving first-time credit courses or

repeating a course for credit in accordance with S.L. 2011-145 Section 7.22 (h)(2) and SBE Policy # GCS-M-001 (“Course for Credit”).

The process was presented to the Superintendents at their quarterly meeting on October 18, 2011 and their feedback was requested

by November 1, 2011.

As a greater number of schools in North Carolina find a need to include online learning as an option for earning course credit, the State

Board of Education (SBE) must ensure the courses taken through a variety of online vendors meet high quality standards. The

standards outlined in this process are taken from iNACOL and SREB, national and regional organizations and leaders for online learning.

It is a core belief high quality online course for credit should be teacher-led. Therefore, both the standards for high quality online

courses, as well as high quality online teaching are included. The evaluation tool for online courses includes several categories for

evaluation which align to both iNACOL and SREB categories and are considered standard areas of evaluation for online courses. The

categories and a rationale for their use are listed below.

Content: The content of any approved online course should be aligned to the North Carolina Standard Course of Study and help

to prepare students for End of Course Exams or VoCATS tests if applicable. Students should be able to clearly determine the

objectives and standards within the course. Students should have access to course syllabus as well as teacher contact

information. Online courses will be of sufficient rigor and should be copyright compliant, protect student privacy, and address

information literacy skills.

Instructional Design: This category addresses how courses should be organized in order to maximize student learning.

Students should have a variety of ways to learn the content which is at the appropriate readability level for the student as well

as interact with the course instructor and other students.

Student Assessment: Student learning should be measured through both summative and formative assessments, with

ongoing, frequent feedback to the student. The assessments should be aligned with the course standards and objectives.

Technology: The technology requirements of the course should be clear and at a level so most students in the state have

reasonable access to the hardware and software needed to complete the course. Technical support should be available to the

1201_LFI 01_Attach

2

student and the online vendor should provide information or should make available information for technical support to the

student and parent.

Course Evaluation and Management: Online vendors should have processes in place in order to evaluate their own courses

and keep them up-to-date.

21st Century Skills: Online courses should intentionally address 21st century skills.

Vendors will be required to provide artifacts to address the rubric criteria for each standard of quality online courses and teaching for

each course submitted for approval. Artifacts are listed under “Artifact Indicators” on the evaluation tool.

Scope

As described in SBE Policy GCS-M-001, courses for credit, whether for first-time or repeating a course, must meet three criterion:

“must consist of 150 clock hours of instruction in a traditional schedule or [1.1]; must consist of a minimum of 135 clock hours of

instruction in a block schedule; developed curriculum guides, or Advanced Placement syllabi in which high school students are enrolled

[1.2] ; and must be directed by a teacher [1.3].” Likewise, SBE Policy GCS-M-001 further defines the responsibility of using online

courses for credit; “Enrollment in an e-learning “for credit course” shall count toward satisfying local board requirements related to

minimum instructional days, seat time policies, student attendance, athletic and/or extracurricular obligations. Furthermore, LEAs are

instructed to be purposeful in establishing processes and procedures to enroll and manage such e-learning students in an environment

where they can be successful. [10.5].”

Credit Recovery is defined as “a block of instruction that is less than the entirety of the Standard Course of Study for that course. Credit

Recovery therefore, delivers a subset of the Standard Course of Study or Blueprint of the original course in order to specifically address

deficiencies in a student’s mastery of the course and target specific components of a course necessary for completion” (SBE Policy #

GCS-M-001 (“Course for Credit”). When students are recovering a credit only according to the definition above, a designation of pass

or fail is used on the student’s transcript. These courses are not part of this approval process, unless the teacher making the

recommendation for credit is an employee of the content provider, rather than the Local Education Agency (LEA.)

This approval process does not include approval of programs to be used as Credit Recovery or to supplement instruction, as approved

courses must be paired with an online instructor.

Repeating a complete course for credit refers to “a high school course repeated via any delivery method when the entire Standard

Course of Study for that course is being taught to the student for a second time” (SBE Policy # GCS-M-001 (“Course for Credit”).

Therefore, these types of courses are repeated for credit and a final numerical grade is earned for the student transcript. These

courses are part of this approval process.

Process

Step 1:

1201_LFI 01_Attach

3

A Web page dedicated to the Vendor Approval Process will be created at www.ncvps.org. Vendors will be able to complete the approval

process for first-time credit courses or repeating a complete course for credit by accessing the following resources on the Web page:

process steps, online application, and evaluation tools.

Appendix 1: Spring 2012 Priority Reviews provides a list of courses to be evaluated. The list of courses is limited due to limited capacity

to review courses. Eventually, the evaluation process will include courses not offered in the NCVPS course catalog. If a LEA would like

to request additional courses be added to this list for priority review, they may contact NCVPS directly. Upon completing reviews of

these courses, NCVPS will begin soliciting other courses for review as determined by needs expressed by LEAs (including courses

already in the NCVPS catalog).

The application process will include a request for course login information so reviewers may review vendor courses submitted for

approval. This Web page will be developed by NCVPS.

Vendors will submit the application and all supporting artifacts to a Google folder created for each individual vendor. Access to the

Google folder will be given to the vendor and a message will be sent to the vendor indicating the availability of the folder.

Step 2:

Once an application is accepted, designated NCVPS and/or NC DPI reviewers will complete the course review process in no more than

120 days. Priority will be given to courses not currently part of the North Carolina course catalog via NCVPS.

1. Review Process

Reviewers come from both NCVPS and NC DPI to ensure quality. Once an application is submitted, the application will be given

to the reviewers so the designated section of the evaluation tool may be completed by reviewers using the artifacts submitted.

2. Quality Online Course Evaluation Rubric

The rubric lists the standards from SREB/iNACOL, contains “Artifacts Indicator”, identifies the evaluation process, and provides

an area for reviewer notes. Vendors will be responsible for providing artifacts to demonstrate the standards being met as

identified under the “Artifacts Indicator” section. Each artifact must be labeled with the corresponding standard being

addressed.

3. Quality Online Teacher Evaluation Rubric

The rubric lists the standards from SREB/iNACOL, identifies the “Indicators”, and provides an area for review notes. Each

standard is listed first and is followed by several “Indicators”. Vendors will be responsible for providing a narrative for each

standard demonstrating how each of the “Indicators” listed are met by the specific course instructor. Additional artifacts may be

requested. A narrative may include, but are not limited to: a process, procedure, policy, or guideline that demonstrates how the

standard(s) is being met by a highly qualified teacher.

An example is included below for Standard A: The teacher meets the professional teaching standards established by North

Carolina and the teacher has academic credentials in the field in which he or she is teaching.

1201_LFI 01_Attach

4

All teachers are required to be highly qualified under the No Child Left Behind Act -

http://www.ncpublicschools.org/nclb/highly/faqs/. NCVPS requires a copy of the teacher’s NC teaching certification to be

kept on file as it indicates the teacher’s highly qualified status on the certification to verify teacher meets Standard A.

NCVPS utilizes NC Wise data cross-reference with list of NCVPS teachers as a secondary verification of highly qualified

status through a School Activity Report (SAR.)

NC requires Standard Professional II (SPII) Professional Educator's Licenses. NCVPS verifies teacher meets Standard A by

requiring a copy of current teaching certificate showing the teacher has SPII status.

Teachers who have teaching certification outside of NC are required to work with the NC DPI Licensure Division to obtain a

NC teaching certification. A copy of the current NC teaching certificate is required to be kept on file as it indicates the

teacher’s NC teaching status and to verify teacher meets Standard A.

Note: This standard is important because all NC online teachers are required to be highly qualified by NC standards.

Step 3:

Results of application review are communicated to vendor by the NCVPS Chief Academic Officer with possible next steps.

1. Evaluations will be published on NCVPS Web page.

2. Approval will last for two years unless a significant change has been made to a course(s). In the case of a significant change to

a course(s), the vendor will need to reapply.

3. A weighting system will be developed using the evaluation tool to determine “cut-off” scores for approval. However, not aligning

to North Carolina curriculum standards would be a basis to not approve a course.

4. Once the review is complete, NCVPS will provide the evaluation to the vendor. If vendors wish to resubmit additional artifacts

based on the review, they must wait 90 days before resubmitting. The application process must be completed again as indicated

in Steps 1 and 2 showing evidence the course has been revised. Resubmission will follow the same initial submission process.

5. This approval process shall replace previous approvals. All vendors must re-submit their courses for inclusion.

Timeline:

1. Vendor should review all information on NCVPS Web page concerning Vendor Approval Process.

2. Vendor should complete online application

3. NCVPS Chief Academic Officer (CAO) will contact vendor with the next steps for submission once application is received.

4. Vendor should begin to collect data (see “Artifacts Indicator” on Evaluation Tool).

5. Vendor should follow directions from NCVPS CAO on submission procedures.

6. Once all required documents are received by the NCVPS CAO from the vendor, the NCVPS CAO will provide the vendor with a

timeline for the review process.

7. Upon review completion, the NCVPS CAO will share evaluation results with the vendor. All evaluation reviews are subject to

1201_LFI 01_Attach

5

publication on the NCVPS website.

Sources for the Evaluation Tools/Rubrics:

National Standards of Quality for Online Courses (2010)

National Standards for Quality Online Teaching (2010)

SREB Checklist for Evaluating Online Courses (2006)

SREB Standards for Quality Online Teaching (2006)

1201_LFI 01_Attach

6

NCVPS Evaluation Tool for Online Courses

Introduction

This evaluation checklist is based on the SREB Standards for Quality Online Courses and iNACOL National Standards of Quality for Online

Courses and is designed to determine the quality and effectiveness of online courses. It is suggested each course be rated on the extent to

which it meets the criteria, with “3” indicating a course does not meet the criteria and “1” indicating it does. This evaluation checklist is

key to ensuring online courses provide students with access to quality instruction and resources.

Course

Reviewer

Course Provider Email

URL Evaluation Completion Date

DIRECTIONS: Rate each item based on the extent to which the item meets the criteria.

A Content Artifact Indicators Agency Responsible

for Evaluation Process

Evidence of Artifact

1. The course goals and objectives are measurable and

clearly state what the participants will know or be able

to do at the end of the course.

Crosswalk linking the following information

to NC Standards i.e. Common

Core/Essential Standards: Course goals and

objectives are measureable, clearly stated,

matched to content requirements and to the

grade and skill levels of the intended

audience.

DPI C&I

2. The course content and assignments are aligned with

state’s content state’s content standards i.e. Common

Core and Essential Standards for nationally accepted

content standards set for Advanced Placement (AP)

courses not included in state standards.

Crosswalk linking the following information

to NC Standards: Course objectives,

student learning outcomes and assignments

are listed and align with established content

standards i.e. Common Core and Essential

Standards and the content and format

requirements of required local school

system, state or national assessments i.e.

AP.

DPI C&I

1201_LFI 01_Attach

7

3. Course tasks and assessments align with the required

local, state, and national assessments that are

associated with the course.

Crosswalk linking the following information

to NC/AP College Board Standards:

Advanced Placement testing requirements;

national assessments; state assessments;

local school system assessments;

technology certification assessments.

DPI C&I

4. The course content and assignments are of

sufficient rigor, depth, and breadth to teach the

standards being addressed.

Documentation of several exemplars from

course content providing the following

information: Clear and explicit alignment

exists between objectives, assessments,

instructional strategies, content and

technology; and the content, activities

and assignments provide multiple learning

opportunities to master the standards.

DPI C&I

5. Information literacy and communication skills are

incorporated and taught as an integral part of

the curriculum.

Documentation of several exemplars

from course content providing the

following information: Promotes student-

to-student communication as part of

lesson activities; utilizes communication

forums such as threaded discussion

forums, chats, etc.; information literacy is

addressed.

DPI C&I

6. Sufficient learning resources and materials to

increase student success are available to students

before the course begins.

Documentation of several exemplars

from course content providing the

following information: Self-assessments

or checklists that assess prerequisite

knowledge and skills; orientations,

tutorials, activities, policies and

procedures to help students acquire the

prerequisite knowledge and skills.

DPI C&I

1201_LFI 01_Attach

8

7. A clear, complete course overview and syllabus are included in the course.

Documentation of several exemplars

from course content providing the

following information: The syllabus and

overview to include: course objectives and

student learning outcomes; assignments;

student expectations; time requirements;

required materials; the grading policy;

teacher- student, teacher-parent con-

tact policies; the intended audience; and

the content scope and sequence. Other

important information includes a

rationale indicating how this course is

related to other courses, the date of

original publication and last revision, the

instructor availability and response time,

and a description of the final product or

other activities that are central to the

course content.

NCVPS C&I

8. Course requirements are consistent with course

goals, representative of the scope of the course,

and clearly stated.

Documentation of several exemplars from

course content providing the following

information: The course requirements to

include: a timeframe for participation; an

approximate time required for individual

activities; expectations for

communications, such as online

discussion, synchronous chats and e-mail;

activity and other assignment

expectations; and survey and/or exam

expectations.

DPI C&I

9. Information is provided to students, parents and

mentors on how to communicate with the online

teacher and course provider, including

information on the process for these

communications.

Documentation of several exemplars from

course content providing the following

information: Information included in the

course materials or in the orientation to

the course materials showing how

students, parents and mentors

communicate with the online teacher

and course provider.

NCVPS C&I

10. Issues associated with the use of copyrighted materials are addressed.

Documentation indicating all course

materials is in compliance with copyright

laws. NCVPS C&I

1201_LFI 01_Attach

9

11. Academic integrity and netiquette (Internet

etiquette) expectations regarding lesson

activities, discussions, e-mail communications and

plagiarism are clearly stated.

Documentation of several exemplars from

course to include guidelines for students

on the use of copyrighted materials,

Internet etiquette, appropriate use of

networked resources, and academic

honesty.

NCVPS C&I

12. Privacy policies are clearly stated.

Documentation of privacy posted within

the course. This could be included in the

course materials or in the orientation to

the course materials.

NCVPS C&I

13. Instructor resources and notes are included.

Documentation of several exemplars

from course of instructor resources and

notes to include: additional guidance for

teachers; possible scenarios for course

module sequence; suggestions for

guiding students on paths; teaching

strategies for diverse students; suggestions

for fostering student interaction and

participation in discussions; additional

content materials, such as optional

simulated lab materials or activities or

extra-credit assignments; ideas for

sensitivity to background and student

needs; and guidelines for using the

learning paths.

DPI C&I

14. Assessment and assignment answers and

explanations are included.

Documentation of several exemplars

from course such as built-in course

assessments, including answers and

explanations about each item.

DPI C&I

B Instructional Design SREB Indicators Agency Responsible

for Evaluation Process

Artifact

1. Course design reflects a clear understanding of

student needs, and incorporates varied ways to

learn and multiple levels of mastery of the

curriculum.

Documentation of several exemplars from

course showing how the course design

meets the needs of a student with low

literacy skills differently than a student

who is taking Advanced Placement; for

example, evidence of more scaffolding

and chunking of content, or more audio

scripts of written materials.

NCVPS C&I

1201_LFI 01_Attach

10

2. The course is organized into units and lessons. Documentation of the course outline to

include how the course is organized by

units and the associated lessons.

NCVPS C&I

3. The course unit overview describes the objectives,

activities and resources that frame the unit. It

includes a description of the activities and

assignments that are central to the unit.

Documentation of the unit overview

including the objectives, lessons and

resources in the unit; and the activities

and assignments that are central to the

unit.

NCVPS C&I

4. Each lesson includes a lesson overview, content

and activities, assignments, and assessments to

provide multiple learning opportunities for

students to master the content.

Documentation of the lesson overview

including the objectives, activities,

assignments, assessments, estimated

timeframe, and resources included in the

lesson.

NCVPS C&I

5. The course is designed to teach concepts and skills that students will retain over time.

Documentation supporting the course is

designed to teach concepts/skills students

will retain overtime to include: backward

design (focus on expectations for student

learning); essential questions; broad-

based concepts; important and

worthwhile concepts or skills that are

relevant to subject matter and to the

“real world” in which the content may be

applied; and meaningful and authentic

learning experiences that help students

apply course concepts and achieve course

objectives.

NCVPS C&I

6. The course instruction includes activities that engage students in active learning.

Documentation of several exemplars from

course showing evidence of students

engaged in active learning. Evidence of

activities for students who need or want

remediation or enrichment, additional

learning activities; resources for students

who need more foundational skills or

explanations for very difficult concepts.

Documentation to support course also

affords the opportunity for students to

learn at their own pace, with clear

directions and structure as to

expectations and time- lines with teacher

input and monitoring.

NCVPS C&I

1201_LFI 01_Attach

11

7. Instruction provides students with multiple

learning paths to master the content, based on

student needs.

Documentation of several exemplars from

course showing evidence of students

being given choices for alternative

assignments and assessments to make

the best use of their talents and skills, as

appropriate.

NCVPS C&I

8. The teacher engages students in learning

activities that address a variety of learning styles

and preferences.

Documentation of several exemplars from

course showing evidence a variety of

learning styles and preferences for

students. The learning activities may

include: examples; case studies;

simulations; research; a variety of reading

and writing activities; visuals and graphics;

discussions and labs; podcasts; streaming

video; and audio-scripted text; a learning

style inventory; strategies that consider

the different learning styles of students are

used.

NCVPS C&I

9. The course provides opportunities for students to

engage in higher-order thinking, critical-reasoning

activities and thinking in increasingly complex

ways.

Documentation of several exemplars from

course showing evidence of students are

engaged in higher-order thinking,

critical-reasoning activities and thinking

in increasingly complex ways. Exemplars

can include students supporting opinions

with evidence from test and experience;

higher-order thinking inquiries with a

variety of questions; interpersonal and

self-directional skills; critical and systems

thinking (understanding the

interconnections among systems);

activities that require students to frame,

analyze and solve problems, form opinions

and developmental models.

NCVPS C&I

10. The course reflects multicultural education and is accurate, current and free of bias.

Documentation showing evidence of accurate,

current and bias-free images; language and

content and selection of print and non-print

resources reflecting multicultural

education.

NCVPS C&I

11. The teacher can adapt learning activities to accommodate students’ needs.

Documentation showing evidence of learning

activities that accommodate students with

disabilities; 504 plans; ELL; low skill or literacy

levels; enrichment and gifted.

NCVPS C&I

1201_LFI 01_Attach

12

12. Readability levels, written language assignments

and mathematical requirements are appropriate

for the course content and the students.

Documentation showing evidence of sensitivity

to readability issues, written language skills

and mathematical requirements are being

demonstrated.

NCVPS C&I

13. The course design provides opportunities for

appropriate instructor-student interaction,

including timely and frequent feedback about

student progress.

Documentation showing evidence of teachers

responding to students in a timely fashion.

Evidence can include feedback to

emails/messages, discussions and portfolios;

teacher-graded assignments; feedback using

LMS tools (e.g., drop box, quizzes and grades;

phone calls; webinars; regular progress

reports; and end-of- semester/term grades)

NCVPS C&I

14. The course provides opportunities for appropriate

instructor-student and student- student

interaction to foster mastery and application of

the material and a plan for monitoring that

interaction.

Documentation showing evidence of

collaborative learning opportunities between

instructor-student and student-student

interaction. Evidence can be shared from

email, discussions, oral assessments,

synchronous chats, webinars, simulations, lab

activities and other group projects.

NCVPS C&I

15. The course provides opportunities for appropriate

student interaction with the content to foster

mastery and application of the material.

Documentation showing how course structure

is designed to facilitate, to the greatest extent

possible, student interaction with course

content and learning resources designed to

foster mastery and application of course

material.

NCVPS C&I

16. Students have access to resources that enrich the

course content.

Documentation of several exemplars

from course to include materials and/or

Web links selected for appropriateness

aligned with course objectives and

specifications and enrich course content.

NCVPS C&I

1201_LFI 01_Attach

13

C Student Assessment SREB Indicators Agency Responsible

for Evaluation Process

Artifact

1. Student evaluation strategies are consistent

with course goals and objectives,

representative of the scope of the course and

clearly stated.

Documentation of several exemplars from

course showing evidence of strategies

aligned with course goals and objectives.

Additional exemplars can include, but are

not limited to: student evaluations that

provide ongoing and frequent diagnostics to

verify student’s readiness for next modules;

evidence of student being aware of his/her

progress in class and mastery of content

beyond grades; clear expectations and

criteria for assignments, including rubrics

for projects and written assignments; step-

by-step instructions for each evaluative

exercise; guidelines for submitting

assignment; a variety of feasible and

content relevant assignments or evaluative

exercises; accommodations for English

Language Learner (ELL) and Special

Education requirements.

NCVPS C&I

2. The course structure includes adequate and

appropriate methods and procedures to assess

students’ mastery of content.

Documentation of several exemplars from

course showing evidence of valid and

reliable assessments that are frequent and

timely and include online or proctored

testing, performance assessments,

projects, demonstrations, oral assessments,

case studies and portfolios; assessment

and measurement strategies including

“self-check” or practice types of

assignments to provide automated,

immediate feedback to students, including

content that explains the error or alludes

to the content presentation; multi-layered

approach to promote academic integrity;

student participation in the evaluation of

their work; and assignments resubmitted

to demonstrate mastery.

NCVPS C&I

1201_LFI 01_Attach

14

3. Ongoing and frequent assessments are

conducted to verify each student’s readiness for

the next lesson.

Documentation of several exemplars from

course showing evidence of imbedded

assessments and other assessment

strategies found throughout the course to

ensure accurate and up-to-date assessment

of student performance.

NCVPS C&I

4. Assessment strategies and tools make the

student continuously aware of his/her progress

in class and mastery of the content beyond

letter grades.

Documentation of several exemplars from

course showing the use of feedback tools

and procedures built into the course;

students being able to continuously self-

monitor their academic progress.

NCVPS C&I

5. Assessment materials provide the teacher with

the flexibility to assess students in a variety of

ways.

Documentation of several exemplars from

course showing evidence of multiple

versions of the same test; test bank

available for the teacher’s use; alternative

evaluation methods available; test/

question item banks organized by the

learning outcomes they are intended to

assess.

NCVPS C&I

6. Grading rubrics and models of partially to fully

completed assignments are provided to the

teacher.

Documentation of several exemplars from

course showing evidence of grading

rubrics specifying the criteria for the

assignments, and models of partially

completed assignments demonstrating the

characteristics of the quality expected.

NCVPS C&I

7. Grading policy and practices are easy to understand.

Documentation of several exemplars from

course showing the grade values given to

assessments and assignments are easy to

understand.

NCVPS C&I

1201_LFI 01_Attach

15

D Technology SREB Indicators Agency Responsible

for Evaluation Process

Artifact

1. The course architecture permits the online

teacher to add content, activities and

assessments to extend learning

opportunities.

Documentation sharing how online teacher

can add content, activities and

assessments to extend learning

opportunities

NCVPS C&I/Tech

2. The course accommodates multiple school

calendars; e.g., block, 4X4 and traditional

schedules

Documentation sharing how the course

design allows users to adapt to various

school calendars and schedules.

NCVPS C&I

3. The course is easy to navigate.

Evidence of consistent and predictable

navigation methods ensuring students can

move logically and easily between areas of

the course; color, graphics and icons used

to guide the student through the course;

and a consistent look and feel that exist

throughout the course (consistent text,

colors, bullets, and heading styles).

NCVPS C&I

4. The course makes maximum use of the

capabilities of the online medium and makes

resources available by alternative means;

e.g., video, CDs and pod casts.

Evidence of rich media is provided in

multiple formats for ease of use and

accessibility i.e. video, podcasts, etc.

NCVPS C&I

5. Hardware, Web browser and software requirements are specified.

Documentation specifying operating

system, processor speed, plug-ins and

connection speed.

NCVPS C&I

6. Prerequisite skills in the use of technology are identified.

Documentation describing what skills and

knowledge of Internet and software

applications are needed.

NCVPS C&I

7. The course utilizes appropriate content-specific tools and software.

Documentation listing content-specific tools

or software needed such as: graphing

calculators, probes, microscopes, word

processing, spreadsheets, presentation

software, and audio recordings.

NCVPS C&I

1201_LFI 01_Attach

16

8. Interoperability technical standards allow

sharing content among different learning

management systems.

Documentation indicating content is

SCORM 1.2 conformant at a minimum

(includes IMS Content Packaging 1.1.2);

IMS QTI conformant assessments.

NCVPS C&I

9. Interoperability technical standards ensure

sharing of questions, assessments and

results with others.

Documentation indicating content is

SCORM 1.2 conformant at a minimum

(includes IMS Content Packaging 1.1.2);

IMS QTI conformant assessments.

NCVPS C&I

10. The course meets universal design

principles, Section 508 standards and W3C guidelines to ensure access for all students.

Documentation showing evidence the

course meets universal design principles,

Section 508 standards and W3C

guidelines. Evidence of content being

accessible by all students, including

students who use assistive technology to

see, hear and move the information.

NCVPS C&I

11. Online textbooks used in a course meet

nationally endorsed standards (NIMAS) for

publishers to ensure distribution of

accessible, alternative versions of textbooks

and other instructional materials.

Documentation showing evidence all electronic

resources meet nationally endorsed

standards (NIMAS).

NCVPS C&I

12. The course provider offers the course

teacher, school coordinator assistance with

technical support and course management.

Documentation showing evidence of

product/course updates, a help desk,

special assistance and answers to frequently

asked questions.

NCVPS C&I

13. The course provider offers orientation training.

Documentation showing evidence of

adequate training provided to ensure user

understanding and knowledge to use each

course. Evidence showing training is

available when and in a format the user

needs it.

NCVPS C&I

1201_LFI 01_Attach

17

E Course Evaluation and

Management SREB Indicators

Agency Responsible for Evaluation

Process

Artifact

1. The results of peer review and student evaluations of courses are available.

Documentation of peer review and student review of courses such as student and/or peer surveys.

NCVPS LFI

2. Course provider uses multiple ways of assessing course effectiveness.

Documentation of course evaluation using

multiple criteria: student completion rates;

student performance on in-course

assessments and on county, state, and/or

national assessments or satisfaction

surveys.

NCVPS LFI

3. The course is evaluated regularly for

effectiveness, and the findings are used

as a basis for improvement.

Documentation of ongoing evaluations and

associated improvements.

NCVPS LFI

4. The course is updated periodically to ensure timeliness.

Documentation of course updates. This

information may be in a separate

publication from the course publisher. It

also could be included in the course

overview; e.g., course updated June 2006.

NCVPS LFI

5. The course provider is authorized to

operate in the state in which the course is

offered.

State review of courses and accreditation of

providers are ways to ensure the course

provider is of quality.

NOTE: This specific criterion is being evaluated by the

completion of this process.

6. The teacher meets the professional

teaching standard established by a state

licensing agency or the teacher has

academic credentials in the field in which

he or she is teaching and has been

trained to teach online and to use the

course.

NOTE: See the NCVPS Standards for

Quality Online Teaching for additional

considerations

NCVPS LFI

7. Student information remains confidential,

as required by the Family Educational

Rights and Privacy Act (FERPA).

Documentation showing evidence student information is kept secure and confidential as required by the Family Educational Rights and Privacy Act (FERPA).

NCVPS LFI

1201_LFI 01_Attach

18

F

21st

Century Skills

SREB Indicators

Agency Responsible

for Evaluation Process

Artifact

1. The course intentionally emphasizes 21st

century skills in the course, including using 21st century skills in the core subjects, 21st century content, learning and thinking skills, ICT literacy, self-directed learning, global awareness, and includes 21st century assessments, as identified by the Partnership for 21st Century Skills.

Documentation showing evidence of 21st century skills in the course, 21st century content, learning and thinking skills, ICT literacy, self-directed learning, global awareness, and 21st century assessments, as identified by the Partnership for 21st Century Skills.

NCVPS C&I/DPI C&I

SREB Educational Technology Cooperative, . " Standards for Quality Online Courses." SREB SREB Educational Technology Cooperative.

SREB, November 2006. Web. 12 Sep 2011. <http://publications.sreb.org/2006/06T05_Standards_quality_online_courses.pdf >.

iNACOL International Association for K-12 Online Learning, . " iNACOL National Standards of Quality for Online Courses." iNACOL

International Association for K-12 Online Learning. iNACOL International Association for K-12 Online Learning, August 2010. Web. 12

Sep 2011. <http://www.inacol.org/research/nationalstandards/NACOL%20Standards%20Quality%20Online%20Courses%202007.pdf

>.

NCVPS Evaluation Tool for Online Teachers

Introduction

“Each student needs a quality teacher. That means someone who knows how to teach — and can show it. It also means

someone who knows the subject matter well — and can prove it. Most of all, it means someone who is successful in raising student achievement.”

Goals for Education: Challenge to Lead, Southern Regional Education Board, 2002

This checklist is based on the SREB report Standards for Quality Online Teachers and iNACOL National Standards of Quality for Online

Teachers and is designed to determine the quality and effectiveness of teachers who are teaching online courses.

1201_LFI 01_Attach

19

Each standard is listed first and is followed by several indicators. A narrative should be provided for each standard demons trating

how each of the indicators listed are met by the specific course instructor who makes up the teaching pool of the online provider.

Based on the narratives, additional documentation may be requested.

Each indicator should be rated on the extent to which it meets the criteria, with “3” indicating the narrative and/or document does not

meet the criteria and “1” indicating that it does meet the criteria. This evaluation is key to ensuring online teachers provide students with

access to quality instruction.

Course

Reviewer

Course Provider Email

URL Evaluation Completion Date

A

The teacher meets the professional teaching standards established by North

Carolina and the teacher has academic credentials in the field in which he or

she is teaching.

Evidence Provided

Through

Narrative

Meets the state’s professional teaching standards or has academic credentials in the field in which he or she is

teaching.

Provides evidence that he or she has credentials in the field of study to be taught.

Knows the content of the subject to be taught and understands how to teach the content to students.

Facilitates the construction of knowledge through an understanding of how students learn in specific subject areas.

Continues to update academic knowledge and skills.

1201_LFI 01_Attach

20

B The teacher has the prerequisite technology skills to teach online.

Evidence Provided

Through Narrative

Demonstrates the ability to effectively use word-processing, spreadsheet and presentation software.

Demonstrates effective use of Internet browsers, e-mail applications and appropriate online etiquette.

Utilizes synchronous and asynchronous tools (e.g., discussion boards, chat tools, electronic whiteboards)

effectively

Troubleshoots typical software and hardware problems (i.e. change passwords, download plug-ins, etc).

Demonstrates growth in technology knowledge and skills in order to stay current with emerging technologies and

trends.

C

The teacher plans, designs and incorporates strategies to encourage active

learning, interaction, participation and collaboration in the online

environment.

Evidence Provided

Through

Narrative

Demonstrates effective strategies and techniques that actively engage students in the learning process (e.g., team

problem-solving, in-class writing, analysis, synthesis and evaluation instead of passive lectures).

Facilitates and monitors appropriate interaction among students.

Builds and maintains a community of learners by creating a relationship of trust, demonstrating effective

facilitation skills, establishing consistent and reliable expectations, and supporting and encouraging independence

and creativity.

Promotes learning through group interaction.

1201_LFI 01_Attach

21

Leads online instruction groups that are goal-oriented, focused, project-based and inquiry-oriented.

Demonstrates knowledge and responds appropriately to the cultural background and learning needs of non-native

English speakers.

Differentiates instruction based on students’ learning styles and needs and assists students in assimilating

information to gain understanding and knowledge.

Demonstrates growth in teaching strategies in order to benefit from current research and practice.

Creates a warm and inviting atmosphere that promotes the development of a sense of community among participants.

Encourages students to bring real-life examples into the online classroom.

Mandates participation by setting limits if participation wanes or if the conversation is headed in the wrong direction.

Provides structure for students but allows for flexibility and negotiation.

Uses best practices to promote participation.

Begins each lesson with a short, student-friendly, summary statement indicating the goal of the lesson and the

primary benchmarks that will be covered.

Provides extended resources and activities to increase achievement levels.

1201_LFI 01_Attach

22

D The teacher provides online leadership in a manner that promotes student

success through regular feedback, prompt response and clear expectations.

Evidence Provided

Through

Narrative

Models effective communication skills and maintains records of applicable communications with students.

Encourages interaction and cooperation among students, encourages active learning, provides prompt feedback,

communicates high expectations, and respects diverse talents and learning styles.

Persists, in a consistent and reasonable manner, until students are successful.

Establishes and maintains ongoing and frequent teacher-student interaction, student- student interaction and teacher-

parent interaction.

Provides an online syllabus that defines objectives, concepts and learning outcomes in a clearly written, concise format.

Provides an online syllabus that defines the terms of class interaction for both teacher and students, defines clear

expectations for both teacher and students, defines the grading criteria, establishes inappropriate behavior criteria for

both teacher and students, and explains the course organization to students.

Uses student data to inform instruction, guides and monitors students’ management of their time, monitors learner

progress with available tools and develops an intervention plan for unsuccessful learners.

Provides timely, constructive feedback to students about assignments and questions.

Gives students clear expectations about teacher response time.

Contacts students who are not participating.

1201_LFI 01_Attach

23

Recognizes that student interaction with the lesson has instructional value and therefore encourages students to

participate in leading the instruction and /or demonstrating mastery of the content in other appropriate ways.

Personalizes feedback (support, growth and encouragement).*

Communicates high expectations.*

E The teacher models, guides and encourages legal, ethical, safe, and healthy

behavior related to technology use.

Evidence Provided

Through

Narrative

Facilitates student investigations of the legal and ethical issues related to technology and society; teaches students that copyright laws are

created for a reason.

Facilitates student investigations of the legal and ethical issues related to technology and society; teaches students that copyright laws are created for a reason.

Identifies the risks of academic dishonesty for students.

Uses course content that complies with intellectual property rights policies and fair use standards.

Provides students with an understanding of the importance of Acceptable Use Policies (AUP).

1201_LFI 01_Attach

24

F The teacher has experienced online learning from the perspective of a

student.

Evidence Provided

Through

Narrative

Has taken an online course and applies experiences as an online student to develop and implement

successful strategies for online teaching.

Demonstrates the ability to anticipate challenges and problems in the online classroom.

Demonstrates an understanding of the perspective of the online student through appropriate responsiveness

and a supportive attitude toward students.

G The teacher understands and is responsive to students with special needs in the

online classroom.

Evidence Provided

Through

Narrative

Understands that students have varied talents and skills and uses appropriate strategies designed to include all

students.

Provides activities, modified as necessary, that are relevant to the needs of all students.

Adapts and adjusts instruction to create multiple paths to learning objectives.

Encourages collaboration and interaction among all students.

Exhibits the ability to assess student knowledge and instruction in a variety of ways.

1201_LFI 01_Attach

25

Provides student-centered lessons and activities that are based on concepts of active learning and that are connected to

real-world applications.

Demonstrates ability to identify students struggling with ELL or literacy issues and delivers specific strategies.

Identifies options to expand student thinking, address styles of learning and avenues for enrichment or intervention.

Knows how to implement a team teaching concept.

H

The teacher demonstrates competencies in creating and implementing

assessments in online learning environments in ways that assure validity and

reliability of instruments and procedures.

Evidence Provided

Through

Narrative

Creates or selects fair, adequate and appropriate assessment instruments to measure online learning that reflect

sufficient content validity (i.e., that adequately cover the content they are designed to measure), reliability and

consistency over time.

Implements online assessment measures and materials in ways that ensure instrument validity and reliability.

I

The teacher develops and delivers assessments, projects, and assignments that

meet standards-based learning goals and assesses learning progress by

measuring student achievement of learning goals.

Evidence Provided

Through

Narrative

Includes authentic assessment (i.e., the opportunity to demonstrate understanding of acquired knowledge and skills as

opposed to testing isolated skills or retained facts)

as part of the evaluation process; assesses student knowledge in a forum beyond multiple guess.

1201_LFI 01_Attach

26

Provides continuous evaluation of students to include pre- and post-testing and student input throughout the

course.

Demonstrates an understanding of the relationships between and among the assignments, assessments and

standards-based learning goals.

J The teacher demonstrates competencies in using data and findings from

assessments and other data sources to modify instructional methods and

content and to guide student learning.

Evidence Provided

Through

Narrative

Assesses each student’s background and content knowledge and uses these data to plan instruction.

Reviews student responses to test items to identify issues related to test validity or instructional effectiveness.

Uses observational data (e.g., tracking data in electronic courses, Web logs, e-mail) to monitor course progress and

effectiveness.

Creates opportunities for self-reflection or assessment of teaching effectiveness within the online environment (e.g., classroom

assessment techniques, teacher evaluations, teacher peer reviews).

Addresses multiple intelligences and levels of ability through a variety of alternative interventions such as adjusting lessons

based upon re-teaching and using varied assessment strategies.

Provides evidence of effective learning strategies that worked for the individual student and details specific changes in

future instruction based upon assessment results and research study (data-driven and research- based).

Evaluates instructional strategies to determine their accuracy and usefulness for presenting specific ideas and concepts.

1201_LFI 01_Attach

27

K The teacher demonstrates frequent and effective strategies that enable both

teacher and students to complete self- and pre-assessments.

Evidence Provided

Through

Narrative

Employs ways to assess student readiness for course content and method of delivery.

Employs ways for students to effectively evaluate and assess their own readiness for course content and method of

delivery.

Understands that student success (e.g., grade, level of participation, mastery of content, completion percentage) is

an important measure of teaching and course success.

Provides opportunities for student self-assessment within courses.

Empowers students to independently define short- and long-term learning goals and monitors their personal progress.

L The teacher collaborates with colleagues.

Evidence Provided

Through

Narrative

Networks with others involved in online education.

Leads collaborative efforts to create common assessments among grade-level and /or content-area teachers and share

assessment results with colleagues to collaboratively plan instruction that will best meet individual student needs.

Instructional Design

The following section outlines standards for instructional design skills for the online teacher of record, where applicable. These

standards are considered optional, as instructional design does not always fall under online teaching responsibilities.

1201_LFI 01_Attach

28

M The teacher arranges media and content to help students and teachers transfer

knowledge most effectively in the online environment.

Evidence Provided

Through

Narrative

Demonstrates the ability to modify and add content and assessment, using an online Learning Management System (LMS).

Incorporates multimedia and visual resources into an online module.

Demonstrates the ability to effectively use and incorporate subject-specific and

developmentally appropriate software in an online learning module.

Reviews all materials and Web resources for their alignment with course objectives

and state and local standards and for their appropriateness on a continuing basis.

Creates assignments, projects and assessments that are aligned with students’ different visual, auditory and hands-on ways of learning.

Arranges media and content to help transfer knowledge most effectively in the online

environment.

SREB Educational Technology Cooperative, . "SREB Standards for Quality Online Teaching." SREB SREB Educational Technology

Cooperative. SREB, August 2006. Web. 12 Sep 2011. <http://publications.sreb.org/2006/06T02_Standards_Online_Teaching.pdf>.

iNACOL International Association for K-12 Online Learning, . "National Standards for Quality Online Teaching." iNACOL International

Association for K-12 Online Learning. iNACOL International Association for K-12 Online Learning, August 2010. Web. 12 Sep 2011.

<http://www.inacol.org/research/nationalstandards/NACOL%20Standards%20Quality%20Online%20Teaching.pdf>.

1201_LFI 01_Attach

29

Appendix 1: Spring 2012 Priority Reviews

Due to limited capacity to review courses, NCVPS must limit the scope of courses it will review. Based on needs assessment from

schools and district learning advisors, NCVPS has determined that the following courses (which will not be available through NCVPS in

Fall 2012) will be the review priorities in Spring 2012.

For Review – Spring 2012*• AP Biology

• AP Calculus AB

• AP European History

• AP Latin Vergil

• AP Music Theory

• AP Statistics

• AP US History

• Advanced Functions and Modeling

• Calculus Honors

• Computer Programming I

• French III Honors

• French IV Honors

• Accounting I

*proposed

If schools would like to request additional courses be added to this list for priority review, they may contact NCVPS directly. As reviews

are completed, NCVPS will request additional courses from vendors based on LEA- and charter school-identified needs.

1201_LFI 01_Attach

30

Recommendations from the e-Learning Commission

The e-Learning Commission provided feedback on the draft presented to the State Board of Education in December, 2011. Feedback

included the following concerns:

Online teacher quality assurance. While the standards approved by the State Board of Education reflect national standards

and best practices, the process established relies heavily on vendors to provide assurances about teacher quality through a

submitted narrative. When courses are approved, teachers may not yet be employed by the vendor; as demand becomes clear,

vendors may hire additional teachers. The e-Learning Commission recommended several steps to ensure continued teacher

quality: access to courses in progress and required student satisfaction surveys.

Course access and observation. The e-Learning Commission recommends that access all vendor courses in which North

Carolina students will be enrolled must be made available to NCVPS staff for the purposes of random or periodic review of

pedagogical strategies. This policy would need to be determined very carefully, as the Family Educational Rights and Privacy Act

(FERPA) limits the amount of access that can be given to personally identifiable student information in many circumstances.

Student surveying. To supplement course access policies, all students should be given the same end-of-course survey that

NCVPS students currently take. Aggregating and analyzing these results to determine the quality of vendor courses could

ensure that high-quality teaching practices are utilized and students view the vendor courses as worthwhile educational

opportunities.

Process for removing vendors. The proposed approval process does not currently provide a set of criteria for removing

vendor courses that prove to be of lower quality because it is a one-time process with no follow-up. The e-Learning Commission

recommended the development of baseline performance criteria and a clear process for removing vendors who do not meet

those criteria, as evidenced by course observation and student surveys.

Capacity and cost. The Commission recognized that continued evaluation of teacher quality (course observation and student

surveying) would require capacity beyond what NCVPS could provide under current funding models. They recommended an

application fee of up to $1,000 to evaluate the course and up to ten teachers, with an additional per-teacher fee for each

additional teacher. Such a fee would require legislation, and could not be included in the proposed process at this time.

Members of the Teaching and Learning subgroup of the e-Learning Commission ask that the State Board of Education fund the course

approval process at a level that would allow NCVPS to perform the services above, in addition to the currently proposed process. The

cost would be roughly $1,000 per vendor course submitted for review.