sbe meeting 1/2012 attachment : lfi 1 · to prepare students for end of course exams ......
TRANSCRIPT
SBE Meeting 1/2012 Attachment : LFI 1
EXECUTIVE SUMMARY
Title: NCVPS Process for Approval of Online Courses: Addendum to Standards for Approval of Online
Courses (LFI 2, December 2011-SBE)
Type of Executive Summary: Action Action on First Reading Discussion Information
Policy Implications:
Constitution
General Statute #S.L. 2011-145 SECTION 7.22.(h).(2)
SBE Policy #GCS-M-001
SBE Policy Amendment
SBE Policy (New)
APA #
APA Amendment
APA (New)
Other
Presenter(s): Mr. Ross White (Executive Director, NC Virtual Public School)
Description: Language in Session Law 2011-145 SECTION 7.22 (h).(2) requires local education agencies and charter schools to
obtain permission from NCVPS before offering any virtual course for credit other than ones administered through
NCVPS. Therefore, it is not legal to offer non-NCVPS courses to any public school student without written
permission from NCVPS. Standards for Online Courses were approved by the SBE at the December 2011 meeting.
This process clearly defines the scope of online course materials which must be approved by NCVPS. Per e-
Learning Commision's requests: (1) NCVPS must be given access to each third party course providing for random,
periodic spot-checks and (2) vendors must require students to complete an end-of-course student survey to
substantiate student satisfaction and high-quality teaching.
Resources: NCVPS and NC DPI staff time
Input Process: Input from e-Learning Commission.
Stakeholders: LEAs, Students, Parents
Timeline For Action:
Spring 2012
Recommendations:
-------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
Audiovisual equipment requested for the presentation:
Data Projector/Video (Videotape/DVD and/or Computer Data, Internet, Presentations-PowerPoint preferred)
Specify:
Audio Requirements (computer or other, except for PA system which is provided)
Specify:
Document Camera (for transparencies or paper documents – white paper preferred)
-------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
Motion By: ______________________________ Seconded By: ______________________________
Vote: Yes __________ No __________ Abstain __________
Approved __________ Disapproved __________ Postponed __________ Revised __________
-------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
*Person responsible for SBE agenda materials and SBE policy updates: Susan Auton, 807-3435
1201_LFI 01_Attach
1
LFI 1 (January 2012): Standards for Approval of Online Courses
NCVPS Process for Vendor Approval
Introduction
The North Carolina Virtual Public School (NCVPS), in conjunction with the North Carolina Department of Public Instruction (NC DPI), will
implement a process for vendor approval of individual online courses. This is a process for approving first-time credit courses or
repeating a course for credit in accordance with S.L. 2011-145 Section 7.22 (h)(2) and SBE Policy # GCS-M-001 (“Course for Credit”).
The process was presented to the Superintendents at their quarterly meeting on October 18, 2011 and their feedback was requested
by November 1, 2011.
As a greater number of schools in North Carolina find a need to include online learning as an option for earning course credit, the State
Board of Education (SBE) must ensure the courses taken through a variety of online vendors meet high quality standards. The
standards outlined in this process are taken from iNACOL and SREB, national and regional organizations and leaders for online learning.
It is a core belief high quality online course for credit should be teacher-led. Therefore, both the standards for high quality online
courses, as well as high quality online teaching are included. The evaluation tool for online courses includes several categories for
evaluation which align to both iNACOL and SREB categories and are considered standard areas of evaluation for online courses. The
categories and a rationale for their use are listed below.
Content: The content of any approved online course should be aligned to the North Carolina Standard Course of Study and help
to prepare students for End of Course Exams or VoCATS tests if applicable. Students should be able to clearly determine the
objectives and standards within the course. Students should have access to course syllabus as well as teacher contact
information. Online courses will be of sufficient rigor and should be copyright compliant, protect student privacy, and address
information literacy skills.
Instructional Design: This category addresses how courses should be organized in order to maximize student learning.
Students should have a variety of ways to learn the content which is at the appropriate readability level for the student as well
as interact with the course instructor and other students.
Student Assessment: Student learning should be measured through both summative and formative assessments, with
ongoing, frequent feedback to the student. The assessments should be aligned with the course standards and objectives.
Technology: The technology requirements of the course should be clear and at a level so most students in the state have
reasonable access to the hardware and software needed to complete the course. Technical support should be available to the
1201_LFI 01_Attach
2
student and the online vendor should provide information or should make available information for technical support to the
student and parent.
Course Evaluation and Management: Online vendors should have processes in place in order to evaluate their own courses
and keep them up-to-date.
21st Century Skills: Online courses should intentionally address 21st century skills.
Vendors will be required to provide artifacts to address the rubric criteria for each standard of quality online courses and teaching for
each course submitted for approval. Artifacts are listed under “Artifact Indicators” on the evaluation tool.
Scope
As described in SBE Policy GCS-M-001, courses for credit, whether for first-time or repeating a course, must meet three criterion:
“must consist of 150 clock hours of instruction in a traditional schedule or [1.1]; must consist of a minimum of 135 clock hours of
instruction in a block schedule; developed curriculum guides, or Advanced Placement syllabi in which high school students are enrolled
[1.2] ; and must be directed by a teacher [1.3].” Likewise, SBE Policy GCS-M-001 further defines the responsibility of using online
courses for credit; “Enrollment in an e-learning “for credit course” shall count toward satisfying local board requirements related to
minimum instructional days, seat time policies, student attendance, athletic and/or extracurricular obligations. Furthermore, LEAs are
instructed to be purposeful in establishing processes and procedures to enroll and manage such e-learning students in an environment
where they can be successful. [10.5].”
Credit Recovery is defined as “a block of instruction that is less than the entirety of the Standard Course of Study for that course. Credit
Recovery therefore, delivers a subset of the Standard Course of Study or Blueprint of the original course in order to specifically address
deficiencies in a student’s mastery of the course and target specific components of a course necessary for completion” (SBE Policy #
GCS-M-001 (“Course for Credit”). When students are recovering a credit only according to the definition above, a designation of pass
or fail is used on the student’s transcript. These courses are not part of this approval process, unless the teacher making the
recommendation for credit is an employee of the content provider, rather than the Local Education Agency (LEA.)
This approval process does not include approval of programs to be used as Credit Recovery or to supplement instruction, as approved
courses must be paired with an online instructor.
Repeating a complete course for credit refers to “a high school course repeated via any delivery method when the entire Standard
Course of Study for that course is being taught to the student for a second time” (SBE Policy # GCS-M-001 (“Course for Credit”).
Therefore, these types of courses are repeated for credit and a final numerical grade is earned for the student transcript. These
courses are part of this approval process.
Process
Step 1:
1201_LFI 01_Attach
3
A Web page dedicated to the Vendor Approval Process will be created at www.ncvps.org. Vendors will be able to complete the approval
process for first-time credit courses or repeating a complete course for credit by accessing the following resources on the Web page:
process steps, online application, and evaluation tools.
Appendix 1: Spring 2012 Priority Reviews provides a list of courses to be evaluated. The list of courses is limited due to limited capacity
to review courses. Eventually, the evaluation process will include courses not offered in the NCVPS course catalog. If a LEA would like
to request additional courses be added to this list for priority review, they may contact NCVPS directly. Upon completing reviews of
these courses, NCVPS will begin soliciting other courses for review as determined by needs expressed by LEAs (including courses
already in the NCVPS catalog).
The application process will include a request for course login information so reviewers may review vendor courses submitted for
approval. This Web page will be developed by NCVPS.
Vendors will submit the application and all supporting artifacts to a Google folder created for each individual vendor. Access to the
Google folder will be given to the vendor and a message will be sent to the vendor indicating the availability of the folder.
Step 2:
Once an application is accepted, designated NCVPS and/or NC DPI reviewers will complete the course review process in no more than
120 days. Priority will be given to courses not currently part of the North Carolina course catalog via NCVPS.
1. Review Process
Reviewers come from both NCVPS and NC DPI to ensure quality. Once an application is submitted, the application will be given
to the reviewers so the designated section of the evaluation tool may be completed by reviewers using the artifacts submitted.
2. Quality Online Course Evaluation Rubric
The rubric lists the standards from SREB/iNACOL, contains “Artifacts Indicator”, identifies the evaluation process, and provides
an area for reviewer notes. Vendors will be responsible for providing artifacts to demonstrate the standards being met as
identified under the “Artifacts Indicator” section. Each artifact must be labeled with the corresponding standard being
addressed.
3. Quality Online Teacher Evaluation Rubric
The rubric lists the standards from SREB/iNACOL, identifies the “Indicators”, and provides an area for review notes. Each
standard is listed first and is followed by several “Indicators”. Vendors will be responsible for providing a narrative for each
standard demonstrating how each of the “Indicators” listed are met by the specific course instructor. Additional artifacts may be
requested. A narrative may include, but are not limited to: a process, procedure, policy, or guideline that demonstrates how the
standard(s) is being met by a highly qualified teacher.
An example is included below for Standard A: The teacher meets the professional teaching standards established by North
Carolina and the teacher has academic credentials in the field in which he or she is teaching.
1201_LFI 01_Attach
4
All teachers are required to be highly qualified under the No Child Left Behind Act -
http://www.ncpublicschools.org/nclb/highly/faqs/. NCVPS requires a copy of the teacher’s NC teaching certification to be
kept on file as it indicates the teacher’s highly qualified status on the certification to verify teacher meets Standard A.
NCVPS utilizes NC Wise data cross-reference with list of NCVPS teachers as a secondary verification of highly qualified
status through a School Activity Report (SAR.)
NC requires Standard Professional II (SPII) Professional Educator's Licenses. NCVPS verifies teacher meets Standard A by
requiring a copy of current teaching certificate showing the teacher has SPII status.
Teachers who have teaching certification outside of NC are required to work with the NC DPI Licensure Division to obtain a
NC teaching certification. A copy of the current NC teaching certificate is required to be kept on file as it indicates the
teacher’s NC teaching status and to verify teacher meets Standard A.
Note: This standard is important because all NC online teachers are required to be highly qualified by NC standards.
Step 3:
Results of application review are communicated to vendor by the NCVPS Chief Academic Officer with possible next steps.
1. Evaluations will be published on NCVPS Web page.
2. Approval will last for two years unless a significant change has been made to a course(s). In the case of a significant change to
a course(s), the vendor will need to reapply.
3. A weighting system will be developed using the evaluation tool to determine “cut-off” scores for approval. However, not aligning
to North Carolina curriculum standards would be a basis to not approve a course.
4. Once the review is complete, NCVPS will provide the evaluation to the vendor. If vendors wish to resubmit additional artifacts
based on the review, they must wait 90 days before resubmitting. The application process must be completed again as indicated
in Steps 1 and 2 showing evidence the course has been revised. Resubmission will follow the same initial submission process.
5. This approval process shall replace previous approvals. All vendors must re-submit their courses for inclusion.
Timeline:
1. Vendor should review all information on NCVPS Web page concerning Vendor Approval Process.
2. Vendor should complete online application
3. NCVPS Chief Academic Officer (CAO) will contact vendor with the next steps for submission once application is received.
4. Vendor should begin to collect data (see “Artifacts Indicator” on Evaluation Tool).
5. Vendor should follow directions from NCVPS CAO on submission procedures.
6. Once all required documents are received by the NCVPS CAO from the vendor, the NCVPS CAO will provide the vendor with a
timeline for the review process.
7. Upon review completion, the NCVPS CAO will share evaluation results with the vendor. All evaluation reviews are subject to
1201_LFI 01_Attach
5
publication on the NCVPS website.
Sources for the Evaluation Tools/Rubrics:
National Standards of Quality for Online Courses (2010)
National Standards for Quality Online Teaching (2010)
SREB Checklist for Evaluating Online Courses (2006)
SREB Standards for Quality Online Teaching (2006)
1201_LFI 01_Attach
6
NCVPS Evaluation Tool for Online Courses
Introduction
This evaluation checklist is based on the SREB Standards for Quality Online Courses and iNACOL National Standards of Quality for Online
Courses and is designed to determine the quality and effectiveness of online courses. It is suggested each course be rated on the extent to
which it meets the criteria, with “3” indicating a course does not meet the criteria and “1” indicating it does. This evaluation checklist is
key to ensuring online courses provide students with access to quality instruction and resources.
Course
Reviewer
Course Provider Email
URL Evaluation Completion Date
DIRECTIONS: Rate each item based on the extent to which the item meets the criteria.
A Content Artifact Indicators Agency Responsible
for Evaluation Process
Evidence of Artifact
1. The course goals and objectives are measurable and
clearly state what the participants will know or be able
to do at the end of the course.
Crosswalk linking the following information
to NC Standards i.e. Common
Core/Essential Standards: Course goals and
objectives are measureable, clearly stated,
matched to content requirements and to the
grade and skill levels of the intended
audience.
DPI C&I
2. The course content and assignments are aligned with
state’s content state’s content standards i.e. Common
Core and Essential Standards for nationally accepted
content standards set for Advanced Placement (AP)
courses not included in state standards.
Crosswalk linking the following information
to NC Standards: Course objectives,
student learning outcomes and assignments
are listed and align with established content
standards i.e. Common Core and Essential
Standards and the content and format
requirements of required local school
system, state or national assessments i.e.
AP.
DPI C&I
1201_LFI 01_Attach
7
3. Course tasks and assessments align with the required
local, state, and national assessments that are
associated with the course.
Crosswalk linking the following information
to NC/AP College Board Standards:
Advanced Placement testing requirements;
national assessments; state assessments;
local school system assessments;
technology certification assessments.
DPI C&I
4. The course content and assignments are of
sufficient rigor, depth, and breadth to teach the
standards being addressed.
Documentation of several exemplars from
course content providing the following
information: Clear and explicit alignment
exists between objectives, assessments,
instructional strategies, content and
technology; and the content, activities
and assignments provide multiple learning
opportunities to master the standards.
DPI C&I
5. Information literacy and communication skills are
incorporated and taught as an integral part of
the curriculum.
Documentation of several exemplars
from course content providing the
following information: Promotes student-
to-student communication as part of
lesson activities; utilizes communication
forums such as threaded discussion
forums, chats, etc.; information literacy is
addressed.
DPI C&I
6. Sufficient learning resources and materials to
increase student success are available to students
before the course begins.
Documentation of several exemplars
from course content providing the
following information: Self-assessments
or checklists that assess prerequisite
knowledge and skills; orientations,
tutorials, activities, policies and
procedures to help students acquire the
prerequisite knowledge and skills.
DPI C&I
1201_LFI 01_Attach
8
7. A clear, complete course overview and syllabus are included in the course.
Documentation of several exemplars
from course content providing the
following information: The syllabus and
overview to include: course objectives and
student learning outcomes; assignments;
student expectations; time requirements;
required materials; the grading policy;
teacher- student, teacher-parent con-
tact policies; the intended audience; and
the content scope and sequence. Other
important information includes a
rationale indicating how this course is
related to other courses, the date of
original publication and last revision, the
instructor availability and response time,
and a description of the final product or
other activities that are central to the
course content.
NCVPS C&I
8. Course requirements are consistent with course
goals, representative of the scope of the course,
and clearly stated.
Documentation of several exemplars from
course content providing the following
information: The course requirements to
include: a timeframe for participation; an
approximate time required for individual
activities; expectations for
communications, such as online
discussion, synchronous chats and e-mail;
activity and other assignment
expectations; and survey and/or exam
expectations.
DPI C&I
9. Information is provided to students, parents and
mentors on how to communicate with the online
teacher and course provider, including
information on the process for these
communications.
Documentation of several exemplars from
course content providing the following
information: Information included in the
course materials or in the orientation to
the course materials showing how
students, parents and mentors
communicate with the online teacher
and course provider.
NCVPS C&I
10. Issues associated with the use of copyrighted materials are addressed.
Documentation indicating all course
materials is in compliance with copyright
laws. NCVPS C&I
1201_LFI 01_Attach
9
11. Academic integrity and netiquette (Internet
etiquette) expectations regarding lesson
activities, discussions, e-mail communications and
plagiarism are clearly stated.
Documentation of several exemplars from
course to include guidelines for students
on the use of copyrighted materials,
Internet etiquette, appropriate use of
networked resources, and academic
honesty.
NCVPS C&I
12. Privacy policies are clearly stated.
Documentation of privacy posted within
the course. This could be included in the
course materials or in the orientation to
the course materials.
NCVPS C&I
13. Instructor resources and notes are included.
Documentation of several exemplars
from course of instructor resources and
notes to include: additional guidance for
teachers; possible scenarios for course
module sequence; suggestions for
guiding students on paths; teaching
strategies for diverse students; suggestions
for fostering student interaction and
participation in discussions; additional
content materials, such as optional
simulated lab materials or activities or
extra-credit assignments; ideas for
sensitivity to background and student
needs; and guidelines for using the
learning paths.
DPI C&I
14. Assessment and assignment answers and
explanations are included.
Documentation of several exemplars
from course such as built-in course
assessments, including answers and
explanations about each item.
DPI C&I
B Instructional Design SREB Indicators Agency Responsible
for Evaluation Process
Artifact
1. Course design reflects a clear understanding of
student needs, and incorporates varied ways to
learn and multiple levels of mastery of the
curriculum.
Documentation of several exemplars from
course showing how the course design
meets the needs of a student with low
literacy skills differently than a student
who is taking Advanced Placement; for
example, evidence of more scaffolding
and chunking of content, or more audio
scripts of written materials.
NCVPS C&I
1201_LFI 01_Attach
10
2. The course is organized into units and lessons. Documentation of the course outline to
include how the course is organized by
units and the associated lessons.
NCVPS C&I
3. The course unit overview describes the objectives,
activities and resources that frame the unit. It
includes a description of the activities and
assignments that are central to the unit.
Documentation of the unit overview
including the objectives, lessons and
resources in the unit; and the activities
and assignments that are central to the
unit.
NCVPS C&I
4. Each lesson includes a lesson overview, content
and activities, assignments, and assessments to
provide multiple learning opportunities for
students to master the content.
Documentation of the lesson overview
including the objectives, activities,
assignments, assessments, estimated
timeframe, and resources included in the
lesson.
NCVPS C&I
5. The course is designed to teach concepts and skills that students will retain over time.
Documentation supporting the course is
designed to teach concepts/skills students
will retain overtime to include: backward
design (focus on expectations for student
learning); essential questions; broad-
based concepts; important and
worthwhile concepts or skills that are
relevant to subject matter and to the
“real world” in which the content may be
applied; and meaningful and authentic
learning experiences that help students
apply course concepts and achieve course
objectives.
NCVPS C&I
6. The course instruction includes activities that engage students in active learning.
Documentation of several exemplars from
course showing evidence of students
engaged in active learning. Evidence of
activities for students who need or want
remediation or enrichment, additional
learning activities; resources for students
who need more foundational skills or
explanations for very difficult concepts.
Documentation to support course also
affords the opportunity for students to
learn at their own pace, with clear
directions and structure as to
expectations and time- lines with teacher
input and monitoring.
NCVPS C&I
1201_LFI 01_Attach
11
7. Instruction provides students with multiple
learning paths to master the content, based on
student needs.
Documentation of several exemplars from
course showing evidence of students
being given choices for alternative
assignments and assessments to make
the best use of their talents and skills, as
appropriate.
NCVPS C&I
8. The teacher engages students in learning
activities that address a variety of learning styles
and preferences.
Documentation of several exemplars from
course showing evidence a variety of
learning styles and preferences for
students. The learning activities may
include: examples; case studies;
simulations; research; a variety of reading
and writing activities; visuals and graphics;
discussions and labs; podcasts; streaming
video; and audio-scripted text; a learning
style inventory; strategies that consider
the different learning styles of students are
used.
NCVPS C&I
9. The course provides opportunities for students to
engage in higher-order thinking, critical-reasoning
activities and thinking in increasingly complex
ways.
Documentation of several exemplars from
course showing evidence of students are
engaged in higher-order thinking,
critical-reasoning activities and thinking
in increasingly complex ways. Exemplars
can include students supporting opinions
with evidence from test and experience;
higher-order thinking inquiries with a
variety of questions; interpersonal and
self-directional skills; critical and systems
thinking (understanding the
interconnections among systems);
activities that require students to frame,
analyze and solve problems, form opinions
and developmental models.
NCVPS C&I
10. The course reflects multicultural education and is accurate, current and free of bias.
Documentation showing evidence of accurate,
current and bias-free images; language and
content and selection of print and non-print
resources reflecting multicultural
education.
NCVPS C&I
11. The teacher can adapt learning activities to accommodate students’ needs.
Documentation showing evidence of learning
activities that accommodate students with
disabilities; 504 plans; ELL; low skill or literacy
levels; enrichment and gifted.
NCVPS C&I
1201_LFI 01_Attach
12
12. Readability levels, written language assignments
and mathematical requirements are appropriate
for the course content and the students.
Documentation showing evidence of sensitivity
to readability issues, written language skills
and mathematical requirements are being
demonstrated.
NCVPS C&I
13. The course design provides opportunities for
appropriate instructor-student interaction,
including timely and frequent feedback about
student progress.
Documentation showing evidence of teachers
responding to students in a timely fashion.
Evidence can include feedback to
emails/messages, discussions and portfolios;
teacher-graded assignments; feedback using
LMS tools (e.g., drop box, quizzes and grades;
phone calls; webinars; regular progress
reports; and end-of- semester/term grades)
NCVPS C&I
14. The course provides opportunities for appropriate
instructor-student and student- student
interaction to foster mastery and application of
the material and a plan for monitoring that
interaction.
Documentation showing evidence of
collaborative learning opportunities between
instructor-student and student-student
interaction. Evidence can be shared from
email, discussions, oral assessments,
synchronous chats, webinars, simulations, lab
activities and other group projects.
NCVPS C&I
15. The course provides opportunities for appropriate
student interaction with the content to foster
mastery and application of the material.
Documentation showing how course structure
is designed to facilitate, to the greatest extent
possible, student interaction with course
content and learning resources designed to
foster mastery and application of course
material.
NCVPS C&I
16. Students have access to resources that enrich the
course content.
Documentation of several exemplars
from course to include materials and/or
Web links selected for appropriateness
aligned with course objectives and
specifications and enrich course content.
NCVPS C&I
1201_LFI 01_Attach
13
C Student Assessment SREB Indicators Agency Responsible
for Evaluation Process
Artifact
1. Student evaluation strategies are consistent
with course goals and objectives,
representative of the scope of the course and
clearly stated.
Documentation of several exemplars from
course showing evidence of strategies
aligned with course goals and objectives.
Additional exemplars can include, but are
not limited to: student evaluations that
provide ongoing and frequent diagnostics to
verify student’s readiness for next modules;
evidence of student being aware of his/her
progress in class and mastery of content
beyond grades; clear expectations and
criteria for assignments, including rubrics
for projects and written assignments; step-
by-step instructions for each evaluative
exercise; guidelines for submitting
assignment; a variety of feasible and
content relevant assignments or evaluative
exercises; accommodations for English
Language Learner (ELL) and Special
Education requirements.
NCVPS C&I
2. The course structure includes adequate and
appropriate methods and procedures to assess
students’ mastery of content.
Documentation of several exemplars from
course showing evidence of valid and
reliable assessments that are frequent and
timely and include online or proctored
testing, performance assessments,
projects, demonstrations, oral assessments,
case studies and portfolios; assessment
and measurement strategies including
“self-check” or practice types of
assignments to provide automated,
immediate feedback to students, including
content that explains the error or alludes
to the content presentation; multi-layered
approach to promote academic integrity;
student participation in the evaluation of
their work; and assignments resubmitted
to demonstrate mastery.
NCVPS C&I
1201_LFI 01_Attach
14
3. Ongoing and frequent assessments are
conducted to verify each student’s readiness for
the next lesson.
Documentation of several exemplars from
course showing evidence of imbedded
assessments and other assessment
strategies found throughout the course to
ensure accurate and up-to-date assessment
of student performance.
NCVPS C&I
4. Assessment strategies and tools make the
student continuously aware of his/her progress
in class and mastery of the content beyond
letter grades.
Documentation of several exemplars from
course showing the use of feedback tools
and procedures built into the course;
students being able to continuously self-
monitor their academic progress.
NCVPS C&I
5. Assessment materials provide the teacher with
the flexibility to assess students in a variety of
ways.
Documentation of several exemplars from
course showing evidence of multiple
versions of the same test; test bank
available for the teacher’s use; alternative
evaluation methods available; test/
question item banks organized by the
learning outcomes they are intended to
assess.
NCVPS C&I
6. Grading rubrics and models of partially to fully
completed assignments are provided to the
teacher.
Documentation of several exemplars from
course showing evidence of grading
rubrics specifying the criteria for the
assignments, and models of partially
completed assignments demonstrating the
characteristics of the quality expected.
NCVPS C&I
7. Grading policy and practices are easy to understand.
Documentation of several exemplars from
course showing the grade values given to
assessments and assignments are easy to
understand.
NCVPS C&I
1201_LFI 01_Attach
15
D Technology SREB Indicators Agency Responsible
for Evaluation Process
Artifact
1. The course architecture permits the online
teacher to add content, activities and
assessments to extend learning
opportunities.
Documentation sharing how online teacher
can add content, activities and
assessments to extend learning
opportunities
NCVPS C&I/Tech
2. The course accommodates multiple school
calendars; e.g., block, 4X4 and traditional
schedules
Documentation sharing how the course
design allows users to adapt to various
school calendars and schedules.
NCVPS C&I
3. The course is easy to navigate.
Evidence of consistent and predictable
navigation methods ensuring students can
move logically and easily between areas of
the course; color, graphics and icons used
to guide the student through the course;
and a consistent look and feel that exist
throughout the course (consistent text,
colors, bullets, and heading styles).
NCVPS C&I
4. The course makes maximum use of the
capabilities of the online medium and makes
resources available by alternative means;
e.g., video, CDs and pod casts.
Evidence of rich media is provided in
multiple formats for ease of use and
accessibility i.e. video, podcasts, etc.
NCVPS C&I
5. Hardware, Web browser and software requirements are specified.
Documentation specifying operating
system, processor speed, plug-ins and
connection speed.
NCVPS C&I
6. Prerequisite skills in the use of technology are identified.
Documentation describing what skills and
knowledge of Internet and software
applications are needed.
NCVPS C&I
7. The course utilizes appropriate content-specific tools and software.
Documentation listing content-specific tools
or software needed such as: graphing
calculators, probes, microscopes, word
processing, spreadsheets, presentation
software, and audio recordings.
NCVPS C&I
1201_LFI 01_Attach
16
8. Interoperability technical standards allow
sharing content among different learning
management systems.
Documentation indicating content is
SCORM 1.2 conformant at a minimum
(includes IMS Content Packaging 1.1.2);
IMS QTI conformant assessments.
NCVPS C&I
9. Interoperability technical standards ensure
sharing of questions, assessments and
results with others.
Documentation indicating content is
SCORM 1.2 conformant at a minimum
(includes IMS Content Packaging 1.1.2);
IMS QTI conformant assessments.
NCVPS C&I
10. The course meets universal design
principles, Section 508 standards and W3C guidelines to ensure access for all students.
Documentation showing evidence the
course meets universal design principles,
Section 508 standards and W3C
guidelines. Evidence of content being
accessible by all students, including
students who use assistive technology to
see, hear and move the information.
NCVPS C&I
11. Online textbooks used in a course meet
nationally endorsed standards (NIMAS) for
publishers to ensure distribution of
accessible, alternative versions of textbooks
and other instructional materials.
Documentation showing evidence all electronic
resources meet nationally endorsed
standards (NIMAS).
NCVPS C&I
12. The course provider offers the course
teacher, school coordinator assistance with
technical support and course management.
Documentation showing evidence of
product/course updates, a help desk,
special assistance and answers to frequently
asked questions.
NCVPS C&I
13. The course provider offers orientation training.
Documentation showing evidence of
adequate training provided to ensure user
understanding and knowledge to use each
course. Evidence showing training is
available when and in a format the user
needs it.
NCVPS C&I
1201_LFI 01_Attach
17
E Course Evaluation and
Management SREB Indicators
Agency Responsible for Evaluation
Process
Artifact
1. The results of peer review and student evaluations of courses are available.
Documentation of peer review and student review of courses such as student and/or peer surveys.
NCVPS LFI
2. Course provider uses multiple ways of assessing course effectiveness.
Documentation of course evaluation using
multiple criteria: student completion rates;
student performance on in-course
assessments and on county, state, and/or
national assessments or satisfaction
surveys.
NCVPS LFI
3. The course is evaluated regularly for
effectiveness, and the findings are used
as a basis for improvement.
Documentation of ongoing evaluations and
associated improvements.
NCVPS LFI
4. The course is updated periodically to ensure timeliness.
Documentation of course updates. This
information may be in a separate
publication from the course publisher. It
also could be included in the course
overview; e.g., course updated June 2006.
NCVPS LFI
5. The course provider is authorized to
operate in the state in which the course is
offered.
State review of courses and accreditation of
providers are ways to ensure the course
provider is of quality.
NOTE: This specific criterion is being evaluated by the
completion of this process.
6. The teacher meets the professional
teaching standard established by a state
licensing agency or the teacher has
academic credentials in the field in which
he or she is teaching and has been
trained to teach online and to use the
course.
NOTE: See the NCVPS Standards for
Quality Online Teaching for additional
considerations
NCVPS LFI
7. Student information remains confidential,
as required by the Family Educational
Rights and Privacy Act (FERPA).
Documentation showing evidence student information is kept secure and confidential as required by the Family Educational Rights and Privacy Act (FERPA).
NCVPS LFI
1201_LFI 01_Attach
18
F
21st
Century Skills
SREB Indicators
Agency Responsible
for Evaluation Process
Artifact
1. The course intentionally emphasizes 21st
century skills in the course, including using 21st century skills in the core subjects, 21st century content, learning and thinking skills, ICT literacy, self-directed learning, global awareness, and includes 21st century assessments, as identified by the Partnership for 21st Century Skills.
Documentation showing evidence of 21st century skills in the course, 21st century content, learning and thinking skills, ICT literacy, self-directed learning, global awareness, and 21st century assessments, as identified by the Partnership for 21st Century Skills.
NCVPS C&I/DPI C&I
SREB Educational Technology Cooperative, . " Standards for Quality Online Courses." SREB SREB Educational Technology Cooperative.
SREB, November 2006. Web. 12 Sep 2011. <http://publications.sreb.org/2006/06T05_Standards_quality_online_courses.pdf >.
iNACOL International Association for K-12 Online Learning, . " iNACOL National Standards of Quality for Online Courses." iNACOL
International Association for K-12 Online Learning. iNACOL International Association for K-12 Online Learning, August 2010. Web. 12
Sep 2011. <http://www.inacol.org/research/nationalstandards/NACOL%20Standards%20Quality%20Online%20Courses%202007.pdf
>.
NCVPS Evaluation Tool for Online Teachers
Introduction
“Each student needs a quality teacher. That means someone who knows how to teach — and can show it. It also means
someone who knows the subject matter well — and can prove it. Most of all, it means someone who is successful in raising student achievement.”
Goals for Education: Challenge to Lead, Southern Regional Education Board, 2002
This checklist is based on the SREB report Standards for Quality Online Teachers and iNACOL National Standards of Quality for Online
Teachers and is designed to determine the quality and effectiveness of teachers who are teaching online courses.
1201_LFI 01_Attach
19
Each standard is listed first and is followed by several indicators. A narrative should be provided for each standard demons trating
how each of the indicators listed are met by the specific course instructor who makes up the teaching pool of the online provider.
Based on the narratives, additional documentation may be requested.
Each indicator should be rated on the extent to which it meets the criteria, with “3” indicating the narrative and/or document does not
meet the criteria and “1” indicating that it does meet the criteria. This evaluation is key to ensuring online teachers provide students with
access to quality instruction.
Course
Reviewer
Course Provider Email
URL Evaluation Completion Date
A
The teacher meets the professional teaching standards established by North
Carolina and the teacher has academic credentials in the field in which he or
she is teaching.
Evidence Provided
Through
Narrative
Meets the state’s professional teaching standards or has academic credentials in the field in which he or she is
teaching.
Provides evidence that he or she has credentials in the field of study to be taught.
Knows the content of the subject to be taught and understands how to teach the content to students.
Facilitates the construction of knowledge through an understanding of how students learn in specific subject areas.
Continues to update academic knowledge and skills.
1201_LFI 01_Attach
20
B The teacher has the prerequisite technology skills to teach online.
Evidence Provided
Through Narrative
Demonstrates the ability to effectively use word-processing, spreadsheet and presentation software.
Demonstrates effective use of Internet browsers, e-mail applications and appropriate online etiquette.
Utilizes synchronous and asynchronous tools (e.g., discussion boards, chat tools, electronic whiteboards)
effectively
Troubleshoots typical software and hardware problems (i.e. change passwords, download plug-ins, etc).
Demonstrates growth in technology knowledge and skills in order to stay current with emerging technologies and
trends.
C
The teacher plans, designs and incorporates strategies to encourage active
learning, interaction, participation and collaboration in the online
environment.
Evidence Provided
Through
Narrative
Demonstrates effective strategies and techniques that actively engage students in the learning process (e.g., team
problem-solving, in-class writing, analysis, synthesis and evaluation instead of passive lectures).
Facilitates and monitors appropriate interaction among students.
Builds and maintains a community of learners by creating a relationship of trust, demonstrating effective
facilitation skills, establishing consistent and reliable expectations, and supporting and encouraging independence
and creativity.
Promotes learning through group interaction.
1201_LFI 01_Attach
21
Leads online instruction groups that are goal-oriented, focused, project-based and inquiry-oriented.
Demonstrates knowledge and responds appropriately to the cultural background and learning needs of non-native
English speakers.
Differentiates instruction based on students’ learning styles and needs and assists students in assimilating
information to gain understanding and knowledge.
Demonstrates growth in teaching strategies in order to benefit from current research and practice.
Creates a warm and inviting atmosphere that promotes the development of a sense of community among participants.
Encourages students to bring real-life examples into the online classroom.
Mandates participation by setting limits if participation wanes or if the conversation is headed in the wrong direction.
Provides structure for students but allows for flexibility and negotiation.
Uses best practices to promote participation.
Begins each lesson with a short, student-friendly, summary statement indicating the goal of the lesson and the
primary benchmarks that will be covered.
Provides extended resources and activities to increase achievement levels.
1201_LFI 01_Attach
22
D The teacher provides online leadership in a manner that promotes student
success through regular feedback, prompt response and clear expectations.
Evidence Provided
Through
Narrative
Models effective communication skills and maintains records of applicable communications with students.
Encourages interaction and cooperation among students, encourages active learning, provides prompt feedback,
communicates high expectations, and respects diverse talents and learning styles.
Persists, in a consistent and reasonable manner, until students are successful.
Establishes and maintains ongoing and frequent teacher-student interaction, student- student interaction and teacher-
parent interaction.
Provides an online syllabus that defines objectives, concepts and learning outcomes in a clearly written, concise format.
Provides an online syllabus that defines the terms of class interaction for both teacher and students, defines clear
expectations for both teacher and students, defines the grading criteria, establishes inappropriate behavior criteria for
both teacher and students, and explains the course organization to students.
Uses student data to inform instruction, guides and monitors students’ management of their time, monitors learner
progress with available tools and develops an intervention plan for unsuccessful learners.
Provides timely, constructive feedback to students about assignments and questions.
Gives students clear expectations about teacher response time.
Contacts students who are not participating.
1201_LFI 01_Attach
23
Recognizes that student interaction with the lesson has instructional value and therefore encourages students to
participate in leading the instruction and /or demonstrating mastery of the content in other appropriate ways.
Personalizes feedback (support, growth and encouragement).*
Communicates high expectations.*
E The teacher models, guides and encourages legal, ethical, safe, and healthy
behavior related to technology use.
Evidence Provided
Through
Narrative
Facilitates student investigations of the legal and ethical issues related to technology and society; teaches students that copyright laws are
created for a reason.
Facilitates student investigations of the legal and ethical issues related to technology and society; teaches students that copyright laws are created for a reason.
Identifies the risks of academic dishonesty for students.
Uses course content that complies with intellectual property rights policies and fair use standards.
Provides students with an understanding of the importance of Acceptable Use Policies (AUP).
1201_LFI 01_Attach
24
F The teacher has experienced online learning from the perspective of a
student.
Evidence Provided
Through
Narrative
Has taken an online course and applies experiences as an online student to develop and implement
successful strategies for online teaching.
Demonstrates the ability to anticipate challenges and problems in the online classroom.
Demonstrates an understanding of the perspective of the online student through appropriate responsiveness
and a supportive attitude toward students.
G The teacher understands and is responsive to students with special needs in the
online classroom.
Evidence Provided
Through
Narrative
Understands that students have varied talents and skills and uses appropriate strategies designed to include all
students.
Provides activities, modified as necessary, that are relevant to the needs of all students.
Adapts and adjusts instruction to create multiple paths to learning objectives.
Encourages collaboration and interaction among all students.
Exhibits the ability to assess student knowledge and instruction in a variety of ways.
1201_LFI 01_Attach
25
Provides student-centered lessons and activities that are based on concepts of active learning and that are connected to
real-world applications.
Demonstrates ability to identify students struggling with ELL or literacy issues and delivers specific strategies.
Identifies options to expand student thinking, address styles of learning and avenues for enrichment or intervention.
Knows how to implement a team teaching concept.
H
The teacher demonstrates competencies in creating and implementing
assessments in online learning environments in ways that assure validity and
reliability of instruments and procedures.
Evidence Provided
Through
Narrative
Creates or selects fair, adequate and appropriate assessment instruments to measure online learning that reflect
sufficient content validity (i.e., that adequately cover the content they are designed to measure), reliability and
consistency over time.
Implements online assessment measures and materials in ways that ensure instrument validity and reliability.
I
The teacher develops and delivers assessments, projects, and assignments that
meet standards-based learning goals and assesses learning progress by
measuring student achievement of learning goals.
Evidence Provided
Through
Narrative
Includes authentic assessment (i.e., the opportunity to demonstrate understanding of acquired knowledge and skills as
opposed to testing isolated skills or retained facts)
as part of the evaluation process; assesses student knowledge in a forum beyond multiple guess.
1201_LFI 01_Attach
26
Provides continuous evaluation of students to include pre- and post-testing and student input throughout the
course.
Demonstrates an understanding of the relationships between and among the assignments, assessments and
standards-based learning goals.
J The teacher demonstrates competencies in using data and findings from
assessments and other data sources to modify instructional methods and
content and to guide student learning.
Evidence Provided
Through
Narrative
Assesses each student’s background and content knowledge and uses these data to plan instruction.
Reviews student responses to test items to identify issues related to test validity or instructional effectiveness.
Uses observational data (e.g., tracking data in electronic courses, Web logs, e-mail) to monitor course progress and
effectiveness.
Creates opportunities for self-reflection or assessment of teaching effectiveness within the online environment (e.g., classroom
assessment techniques, teacher evaluations, teacher peer reviews).
Addresses multiple intelligences and levels of ability through a variety of alternative interventions such as adjusting lessons
based upon re-teaching and using varied assessment strategies.
Provides evidence of effective learning strategies that worked for the individual student and details specific changes in
future instruction based upon assessment results and research study (data-driven and research- based).
Evaluates instructional strategies to determine their accuracy and usefulness for presenting specific ideas and concepts.
1201_LFI 01_Attach
27
K The teacher demonstrates frequent and effective strategies that enable both
teacher and students to complete self- and pre-assessments.
Evidence Provided
Through
Narrative
Employs ways to assess student readiness for course content and method of delivery.
Employs ways for students to effectively evaluate and assess their own readiness for course content and method of
delivery.
Understands that student success (e.g., grade, level of participation, mastery of content, completion percentage) is
an important measure of teaching and course success.
Provides opportunities for student self-assessment within courses.
Empowers students to independently define short- and long-term learning goals and monitors their personal progress.
L The teacher collaborates with colleagues.
Evidence Provided
Through
Narrative
Networks with others involved in online education.
Leads collaborative efforts to create common assessments among grade-level and /or content-area teachers and share
assessment results with colleagues to collaboratively plan instruction that will best meet individual student needs.
Instructional Design
The following section outlines standards for instructional design skills for the online teacher of record, where applicable. These
standards are considered optional, as instructional design does not always fall under online teaching responsibilities.
1201_LFI 01_Attach
28
M The teacher arranges media and content to help students and teachers transfer
knowledge most effectively in the online environment.
Evidence Provided
Through
Narrative
Demonstrates the ability to modify and add content and assessment, using an online Learning Management System (LMS).
Incorporates multimedia and visual resources into an online module.
Demonstrates the ability to effectively use and incorporate subject-specific and
developmentally appropriate software in an online learning module.
Reviews all materials and Web resources for their alignment with course objectives
and state and local standards and for their appropriateness on a continuing basis.
Creates assignments, projects and assessments that are aligned with students’ different visual, auditory and hands-on ways of learning.
Arranges media and content to help transfer knowledge most effectively in the online
environment.
SREB Educational Technology Cooperative, . "SREB Standards for Quality Online Teaching." SREB SREB Educational Technology
Cooperative. SREB, August 2006. Web. 12 Sep 2011. <http://publications.sreb.org/2006/06T02_Standards_Online_Teaching.pdf>.
iNACOL International Association for K-12 Online Learning, . "National Standards for Quality Online Teaching." iNACOL International
Association for K-12 Online Learning. iNACOL International Association for K-12 Online Learning, August 2010. Web. 12 Sep 2011.
<http://www.inacol.org/research/nationalstandards/NACOL%20Standards%20Quality%20Online%20Teaching.pdf>.
1201_LFI 01_Attach
29
Appendix 1: Spring 2012 Priority Reviews
Due to limited capacity to review courses, NCVPS must limit the scope of courses it will review. Based on needs assessment from
schools and district learning advisors, NCVPS has determined that the following courses (which will not be available through NCVPS in
Fall 2012) will be the review priorities in Spring 2012.
For Review – Spring 2012*• AP Biology
• AP Calculus AB
• AP European History
• AP Latin Vergil
• AP Music Theory
• AP Statistics
• AP US History
• Advanced Functions and Modeling
• Calculus Honors
• Computer Programming I
• French III Honors
• French IV Honors
• Accounting I
*proposed
If schools would like to request additional courses be added to this list for priority review, they may contact NCVPS directly. As reviews
are completed, NCVPS will request additional courses from vendors based on LEA- and charter school-identified needs.
1201_LFI 01_Attach
30
Recommendations from the e-Learning Commission
The e-Learning Commission provided feedback on the draft presented to the State Board of Education in December, 2011. Feedback
included the following concerns:
Online teacher quality assurance. While the standards approved by the State Board of Education reflect national standards
and best practices, the process established relies heavily on vendors to provide assurances about teacher quality through a
submitted narrative. When courses are approved, teachers may not yet be employed by the vendor; as demand becomes clear,
vendors may hire additional teachers. The e-Learning Commission recommended several steps to ensure continued teacher
quality: access to courses in progress and required student satisfaction surveys.
Course access and observation. The e-Learning Commission recommends that access all vendor courses in which North
Carolina students will be enrolled must be made available to NCVPS staff for the purposes of random or periodic review of
pedagogical strategies. This policy would need to be determined very carefully, as the Family Educational Rights and Privacy Act
(FERPA) limits the amount of access that can be given to personally identifiable student information in many circumstances.
Student surveying. To supplement course access policies, all students should be given the same end-of-course survey that
NCVPS students currently take. Aggregating and analyzing these results to determine the quality of vendor courses could
ensure that high-quality teaching practices are utilized and students view the vendor courses as worthwhile educational
opportunities.
Process for removing vendors. The proposed approval process does not currently provide a set of criteria for removing
vendor courses that prove to be of lower quality because it is a one-time process with no follow-up. The e-Learning Commission
recommended the development of baseline performance criteria and a clear process for removing vendors who do not meet
those criteria, as evidenced by course observation and student surveys.
Capacity and cost. The Commission recognized that continued evaluation of teacher quality (course observation and student
surveying) would require capacity beyond what NCVPS could provide under current funding models. They recommended an
application fee of up to $1,000 to evaluate the course and up to ten teachers, with an additional per-teacher fee for each
additional teacher. Such a fee would require legislation, and could not be included in the proposed process at this time.
Members of the Teaching and Learning subgroup of the e-Learning Commission ask that the State Board of Education fund the course
approval process at a level that would allow NCVPS to perform the services above, in addition to the currently proposed process. The
cost would be roughly $1,000 per vendor course submitted for review.