assessment platform development recommendations building the next generation assessment platform –...

Assessment Platform Development

Recommendations

Building the Next Generation Assessment Platform – the Consortia

Opportunity

January 2011

www.assessmentgroup.org 2

Contents

Background Assessment Platform Opportunity

Potential Project Management Approach Structure Advisory Board

Potential Service Offerings / Consortia Needs Conclusion

Appendix – Advisory Board and Team Bios

January 2011

Important changes are on the way in Education

“I am calling on our nation‘s Governors and state education chiefs to develop standards and assessments that don‘t simply measure whether students can fill in a bubble on a test, but whether they possess 21st century skills like problem-solving and critical thinking, entrepreneurship and creativity.” -President Barack Obama, March 10, 2009

“Our education system at all levels will leverage the power of technology to measure what matters and use assessment data for continuous improvement.” - National Educational Technology Plan 2010 (Draft), U.S. Department of Education

“This new generation of state assessments will be an absolute game-changer in public education . . . For the first time, many teachers will have the assessments they have longed for.” – Arne Duncan

January 2011 www.assessmentgroup.org 3

www.assessmentgroup.org 4December 13, 2010

These changes will have significant impacts on the current “system”…

Legislation / RTTT / Federal Funding

Legislation / RTTT / Federal Funding

District / School /

Teacher / Student Trends

District / School /

Teacher / Student Trends

CCSS / Marketplace

Trends / Innovations / Technology

CCSS / Marketplace

Trends / Innovations / Technology

Children Learning

Experiences

Children Learning

Experiences

Educator / Administration Practices

Educator / Administration Practices

Curriculum /

Assessments

Curriculum /

Assessments

Technologies in

Schools

Technologies in

Schools

EducationEco-System

21st CenturyEduc.

System

Consortia will help 100% of their member states implement new assessments by 2014/2015

The leadership role of the consortia will help drive: A new generation of assessments

Aligned to the Common Core standards High quality Iterative with instruction/curriculum Test 21st century and other key skills (e.g., critical thinking, creativity) Flexibly built for tomorrow – e.g., online, interactive

A comprehensive, flexible next generation assessment platform

Transition of stakeholders to new processes and technologies Cost affordability and reliable implementation for all

stakeholders


The consortia will play a key role in driving a transformation in Education

Technology will be a central component of the Assessment transformation


Next Generation Assessment Platform


Technology Characteristics National scale – huge trxn. volumes / storage Flexible to meet diverse state needs

Technology-enabled Capabilities Linkage between curriculum and assessment Flexible to deliver online

(multi-device/interoperability) Configurable for various assessment types/

purposes (summative, formative, adaptive) New item/content types – e.g., simulations,

graphics, games Content sharing/repurposing Scoring - near real-time; AI; distributed Enhanced reporting for all stakeholders Linkage to Education communities of interest

Next Generation Education Platform

Next Generation Education Platform

New technology trends can benefit development of the new Platform


Open system/open source approach Leverage existing “best of breed” technology solutions / vendors

(including tying into “clouds” of major software providers) Source content openly and collaborate with a broad variety of

industries and disciplines (e.g., crowd sourcing) Continual maturing/refresh of the “system” over time

“App Store” / Software as a Service (SaaS) approach (potential ongoing revenue source for the consortia)

Device independence – mobile, tablet, PC, e-blackboards Maturing of learning management systems Large-scale data management New hosting/infrastructure models (vendor, university,

consortia)

Most states still administer assessments in PPT mode

Success with online testing has been mixed State and district technology resources are very

thin Implementation costs have been unpredictable

and high A comprehensive “how to implement online

testing” set of guidelines does not exist Cost savings from similar efforts have not met

expectations

www.assessmentgroup.org 8January 2011

Change will not come easy given the current state of online assessment

Success on this complex effort can be achieved by leveraging proven methods…

December 13, 2010 www.assessmentgroup.org 9

Involve Education’s best and brightest early in the process Get clear buy-in and consensus from all stakeholders

early with respect to platform “business requirements” Learn from others’ Assessment program mistakes Leverage successes in other Education transformations

Approaches – timeframes, resources, key success factors Technologies – leverage what already exists in the market today

Stage implementation to help States be successful Share know-how/resources where possible to ensure the right

support Develop a transparent, well-understood, reliable cost model up front

Anticipate the needs of the future and build in flexibility Employ the right resources and processes to help get

there

Key Success Factors

And by following a structured approach


Strategy / Planning

Strategy / Planning DesignDesign Implementati

on / RolloutImplementation / Rollout



Recommended Management Approach


A combination of Education and Technology skillsets/expertise will be needed


Business / Tech. Expertise•Strategy / Planning•Ed Technology Trends•Technology Innovations•Software Industry experience•Program Management•Requirements / Design•Application Development•Architecture•Infrastructure

Educational Expertise•Assessment / measurement•Standards (common core, 21st century, etc.)•Education trends•Classroom experience•School system experience•Administration experience

• School data systems• School technology

systems•Education trends

The skills required can be effectively provided be an external partner…

Risk of failure is too high if the needed skill are not obtained and proper business requirements are not developed

Combination of skillsets and focused attention needed to manage the process is likely best procured outside the consortia due to consortia resource constraints

A neutral 3rd party (non assessment platform vendor, non technology developer) should be considered to manage the process

Key vendor characteristics Familiar with both technology and educational assessment Able to leverage technology resources as needed Able to bring the right business, management and technology skills Experience in managing similar projects with a proven methodology


… that can bring the right skills at the right time


Educational Expertise•Assessment / measurement•Standards (common core, 21st century, etc.)•Education trends•Classroom experience•School system experience•Administration experience•Education trends

Business / Tech. Expertise•Strategy / Planning•Ed Technology Trends•Technology Innovations•Software Industry experience•Program Management•Requirements / Design•Application Development•Architecture•Infrastructure

ASG recommends a dynamic Staffing Model: Core team of key staff involved throughout for oversight and continuity Individual representatives will rotate on and off the team based on the skills required for that phase Consultants hired to fill short term skill gaps

There are significant benefits to this approach Enables the consortia to get the most qualified people and expertise Minimizes overhead, reduces costs and results in the best product

Technical / Tactical / Implementation Skills

Strategic / Planning / Business AcumenStart End

• Planning• Innovatio

n

• Strategy• Stakeholder Alignment

• Design• Technical

Evaluation

• Development

• Integration

• Test / Rollout• Change

Mgmt

Key Skills Required

Key Skills Required

Role of the Advisory Board: Provide Education industry and/or technology innovation

insight and subject matter expertise Assist in aligning key stakeholders Oversee the progress / direction of the overall initiative and

provide support to the Consortia Constituents:

Senior education technology experts from world class software and hardware companies

Representatives from Cisco, Microsoft, IBM, Google, Booz Allen, etc.

These companies all have focused efforts in the Education space All see the greater value in improving the current educational system in the

US


An Advisory Board of key experts will ensure the best solution is delivered

A Powerful Team

16

Advisory Board - Education Leaders from Top Software/Technology

Companies

CSCO

Booz Other

MSFT GOOG

IBM

Strategic Technologis

tsCore Technology Team

Dat

abas

e

Architectu

re

Next GenerationAssessment System

Program Management

Advisory Board members

Charles Fadel, Global Education Lead, Cisco Systems Cisco board member at Partnership for 21st Century Skills,

Innovate/Educate and Change the Equation Vice-chair of the Education committee of the Business and Industry

Advisory Committee (BIAC) to the Organization for Economic Co-operation and Development (OECD)

Worked on education projects with more than thirty countries and states

Visiting Scholar at Wharton/UPenn where he recently taught a class on Technologies for Learning

Co-authored 21st Century Skills: Learning for Life in our Times Recently served on the Massachusetts Governor’s Readiness

Project as well as its 21st Century Skills task force Visiting Practitioner, Harvard Graduate School of Education



Hans Van Delden, VP Technology Practice, Booz and Company Specializes in the strategy, operations and technology for educational assessment

businesses Led 10+ strategy, operations and technology programs for several education

companies including: Technology Strategy / Architecture for an Online Assessment solution

(Educational Assessment Provider) – included an evaluation of the technology providers in the educational assessment market

Architecture Review (functional and technical) of custom-developed Online Assessment Platform (High-stakes State Educational Assessment Provider)

Assessment, Strategy and Roadmap for the Online Assessment Solutions (Educational Services company)

Experience working in 12 countries across North America, Europe and Asia Earned five degrees from the Massachusetts Institute of Technology after five years

of study Twice awarded Booz & Company’s Professional Excellence Award, highest award

for client service



Steve Katz, Technology Strategist and Entrepreneur Thought leader in technology innovation and strategy Management roles in advancing innovation at IBM, Apple and Sony

Pictures Entertainment Executive strategy consultant to leading technology firms including

Microsoft, Sony Corporation, SAP and AT&T Serial entrepreneur; founded a number of pioneering ventures backed by

Kleiner Perkins and other top VCs Currently, founding partner in a strategy consulting boutique, O8 Partners -

helps drive major partnerships and strategy for emerging technology companies

Currently entrepreneur-in-residence at MHS Capital, an early stage venture firm

Involved in several non-profit and philanthropic activities, including serving as a founding member of the Full Circle Fund (partners with outstanding social innovators to explore different approaches to Educational reform)


Advisory Board Members

John Behrens, Director Networking Academy Learning Systems Development, Cisco, Inc.

Responsible for the design and development of the Networking Academy’s global on-line instructional offerings that integrate interactive curricular and assessment technologies with hands on activities. These curriculum, assessment, and gaming tools serve more than 700,000 students per semester in 165 countries across more than 10 languages. Over 100 million assessments have been delivered to date through the Cisco Networking Academy.

Dr. Behrens’ team integrates complex simulations in instruction and assessment and developed authoring tools to enable instructor-led visualization of abstract networking concepts.

John’s research interests focus on the intersection of cognitive, statistical and computing issues in instruction, assessment, and on-line learning.

Adjunct Assistant Research Professor, Department of Psychology, University of Notre Dame (Indiana, USA).



Ed Roeber, Professor, College of Education, Michigan State University; Partner and Co-Founder, Assessment Solutions Group

John Olson, Partner and Co-Founder, Assessment Solutions Group

Barry Topol, Managing Partner and Co-Founder, Assessment Solutions Group

(full bios on all advisory board members are located in the appendix)


Potential core technology team members

Jay Whitchurch (Program Lead/Executive) CEO and co-founder of campus2careers, an online career community that

helps college students prepare for, find, and launch their careers Senior Vice President, Educational Assessment (K-12), Pearson Education

Assessment portfolio included classroom-based (formative), skills (diagnostic/readiness), comparability (achievement), and reporting products/services

Chief Information Officer and Senior Vice President at Harcourt Assessment Led a team of 150+ Technology professionals in: Strategy/ Planning, Architecture,

Applications, Infrastructure, Program Management, and Vendor Management 13 years in management consulting driving strategic business

transformations for global Healthcare, Education, Nonprofit, and Public Sector clients

Principal, Booz & Company, Strategy/Technology Practice – experience included working with a leading Education company to define an online strategy and associated business cases, priorities, required investments, and transition approach

Senior Consultant, Accenture – Led teams of client and Accenture staff on major technology change initiatives including the design and implementation of large-scale Public Sector systems



Technology Strategist / Innovator Recognized as a thought leader on next-generation assessment systems Over 20 years of technology leadership experience and over 6 years of

experience in K-12 online assessment Currently a Technology Director for a major assessment/education co.

Responsible for developing and maintaining the strategic direction and architectural road map for PK-12 online assessment

Leads the technology development for the current employer’s primary initiative around Common Core and the Consortia

Well established relationships with multiple online assessment companies and is considered a subject matter expert on the capabilities of various providers and publishers.

Led the technology due diligence process for evaluating potential online assessment partners / acquisitions for two major Education companies

Provided technology leadership to multiple, established companies and held CTO/CIO positions in various entities during his career



Program Manager Currently a lead in a top Technology company 12+ years of experience in general management consulting

for Fortune 500 clients Focused on strategy based transformation for the Media & Technology

industries Strong experience in business / technology strategy, business case

development, financial planning, and “go-no-go” decision analysis Demonstrated track record for leading and helping with the

implementation of growth strategies and new product development Led large transformational teams – delivering against tight timelines

Entrepreneur in the education space, owns two Montessori schools that were built ground up - concept to operational

Experienced in design and development of applications Education:

MBA, Duke University, M.S. in Materials Science (Biomechanics) Michigan State University, M.S. in Mechanical Engineering (Robotics) VillanovaJanuary 2011 www.assessmentgroup.org 24


Milan Malkani (Architect / Technology lead) Technologist and entrepreneur Involved with the design and implementation of complex software

systems for established companies (e.g., Dell, TheKnot.com) and startups (e.g., ActiveVendors.com and campus2careers.com)

Spent 3 years travelling the US and abroad to train the software development groups of IT departments for large organizations (e.g., Wells Fargo, US Navy and Army, MD Anderson, etc)

Proven track record of bringing applications to market in record time


Potential ASG Service Offerings


ASG can support the Consortia across all key phases of the initiative


Strategy / Planning

Strategy / Planning DesignDesign Implementati

on / RolloutImplementation / Rollout

Initial Key Elements

Next Generation Assessment

Platform

Requirements Definition

Requirements Definition

Functional and Technical

Design

Functional and Technical

Design

State Financial Models

State Financial Models

Technology EvaluationTechnology Evaluation Build vs. BuyBuild vs. Buy Systems

IntegrationSystems

Integration

Plan Development

Plan Development

Program ManagementProgram Management

Rollout Plan DevelopmentRollout Plan

Development

RFP Preparation / Evaluation

RFP Preparation / Evaluation

Tasks by Project Phase

The Strategy / Planning Phase will form the foundation for success


Define/Confirm Common

Core Assessment

Vision

Define/Confirm Common

Core Assessment

VisionDevelop High-level

Technology Requiremen

ts

Develop High-level

Technology Requiremen

ts

Develop Plan for Design and Implementatio

n

Develop Plan for Design and Implementatio

n

Week 0

Week 4

Week 13

Week 16

• Project launch

• Stakeholder identification

• Stakeholder interviews

• Objectives / expectations

• Constraints

• Application• Data• Infrastructure

• Stakeholder review/input sessions

• Business Requirement iterations

• Roadmap for process, people, technology rollout and transitions

• Top-down estimates on costs/timeframe

• Priorities• Resource / skills

requirements

Survey Current

Technology Marketplace

Survey Current

Technology Marketplace

• Structured capabilities/gap analysis against Requirements

Develop High-level Business

Requirements

Develop High-level Business

Requirements

• “Must haves”• “Nice to

haves”• Out-of-scope

Align Stakeholder

s

Align Stakeholder

s

Identify Process and

Change Mgmt.

Impacts

Identify Process and

Change Mgmt.

Impacts• Keep/start/

stop activities

• Transition planning

More on the requirements definition component

Key to a successful project is being able to manage the requirements development process & achieve stakeholder buy in

Experience, focus and a proven methodology for generating requirements is critical

Need to get requirements at the right level of detail/complexity Need to be able to drive out ambiguity in the defined requirements

Define what the project is Define what the project is not

Need to get all stakeholders aligned Experience in leading major technology transformations is

invaluable in managing such a project Finally, all the above must be wrapped up into a carefully thought

out plan with time, cost, technology and resource estimates Doing this right typically takes 12-16 weeks of focused effort by a

skilled, experienced team


Key considerations for the Strategy / Planning phase

End-state objectives and expectations should be determined up front and all key stakeholders aligned

Weeks of doing this up front will save years in aiming at moving targets Stakeholders almost always not aligned on even going-in assumptions Propensity to “jump to answers” without the right information is a common

source of failed projects Technology is not “the answer” – it is an enabler

Business requirements and change management implications are the key pieces to get right - technology needs are by-products

Most frequent pitfall in these efforts is undefined and/or unclear business reqs. Business requirements should be clearly defined and “approved” before

technology discussions even begin

Getting the right answers in the shortest time and at the lowest cost requires leveraging different skillsets throughout

Strategic vs. Implementation Education and business expertise vs. program management vs. technical


ASG can bring key expertise to the Strategy / Planning phase

Requirements Definition Collect and synthesize requirements from key stakeholders (structured

interviews, research) Define and support the process for aligning stakeholders Translate Business Requirements into Technical Requirements

Technology Evaluation Evaluate existing technologies and assessment platforms to gain an

understanding of current capabilities, technologies, architectures, fit with the requirements

Surface and explore existing technology options/vendors to potentially leverage as a starting point to save time and costs

Example options: ITS, CAL, Vantage/McAnn, RANDA, Nimble Tools, AIR

Go Forward Plan Development Develop structured plan for implementing the Next-Generation Platform

Roadmap that identifies key activities, priorities, and timing Business case that identifies estimated costs/benefits and resources required


Strategy / Planning Phase

Program and Vendor management Oversight of vendor(s) selected to develop the platform per RFP

specifications – e.g., project management, cost control and technical management

Maintain stakeholder involvement at the right points in the process Future changes to the platform are carefully managed


ASG can also provide support services for the “Design” phase

Functional and technical design and analysis Develop Functional and Technical design / architecture based on the

analysis of current platforms and guidance from the Consortia and advisory board

Basis for the Technology systems RFP Current and future requirements Open system/open source; Software as a Service/Applications store Content and data management Scalability and extensibility Initial set of features and functionality/interoperability

Build vs. Buy analysis / recommendations Provide recommendations based on business requirements, evaluation

of the existing assessment platforms, and high level technical design Explore different build alternatives (open source, university, vendor,

other) and identify pros and cons for the Consortia Findings/recommendations to be reflected in the RFP for vendor services


ASG “Design” Support Services (continued)

RFP preparation and evaluation Develop the RFP for the assessment technology system

Prepared and informed by lessons learned from previous evaluations A variety of potential bidders to be solicited based on previous analyses

Evaluate/score vendor responses, both technical and cost, against a pre-determined rubric

Provide recommendations on the go-forward plan and vendor


ASG “Implementation / Rollout” Support Services

Preparation of individual state financial models The costs of converting to online assessment may be significant for

many states. A comprehensive model and analysis detailing state-specific costs, benefits, breakeven points and return on investment in converting to OLA will help state departments of education and appropriate constituents understand the costs and issues in moving online

Systems Integration Coordinate among different vendors/applications to ensure the

development of a unified system Investigate potential new technologies and applications Develop specifications for applications interfaces Manage the overall development of the system Identify solutions for open source code management and

administration


ASG “Implementation / Rollout” Support Services (continued)

Rollout Plan Development A comprehensive roll out plan for delivering the new assessment

system should be developed well before actual completion of the system itself. Resource requirements, costs, time frames, a training plan and documentation along with a state by state implementation plan should be produced to ensure the rollout proceeds as smoothly as possible.

OLA Implementation “How To” Manual Compilation of a detailed, step by step, approach to implementing

online assessment including network, bandwidth and help desk requirements/activities

Lessons learned from previous state OLA implementations Minimum system requirements Minimum people requirements for implementation and ongoing support Networks and bandwidth requirements


Conclusions/Recommendations

Developing the next-generation assessment platform is a once in a generation opportunity and should result in a transformative change in the way assessment is thought about and delivered

To be successful, a significant and focused effort around planning, requirements, technology evaluation/design/ implementation and management of the effort is needed

ASG offers a sound approach for supporting the Consortia in successfully delivering the next-generation assessment platform

Core strategic and tactical technologists with deep expertise gained from within and outside of the Education industry

An advisory board consisting of representatives of brand name technology development companies

Skillful use of outside consultants offers a sound approach to successfully delivering the next generation assessment system


Why ASG?

While the consortia are familiar with ASG’s cost modeling and technical assessment capabilities, the team assembled for this effort (Topol, Whitchurch & Co.) has built and/or overseen the building of many complex Technology projects

ASG understands the complexities of the consortia assessment systems as well as, assessment technology requirements

The group is independent and will bring an independent perspective and analysis of competing alternatives

The management approach to the effort minimizes overhead and maximizes value. Funding will go entirely to the project

The team and structured approach to managing the platform development effort will result in a powerful next generation assessment system


Appendix – Advisory Board and Team Bios


Advisory Board Bios – Charles Fadel

Global Education Lead at Cisco Systems Cisco board member at the Partnership for 21st Century Skills (Cisco was

a founding member), Innovate/Educate and Change the Equation (two organizations dedicated to advocacy of Science, Technology, Engineering and Math (STEM)

Vice-chair of the Education committee of the Business and Industry Advisory Committee (BIAC) to the Organization for Economic Co-operation and Development (OECD) and works with several teams at the OECD - PISA, AHELO, PIAAC, CELE, and TALIS

Worked with education ministries and organizations in more than thirty countries and states including Massachusetts, Canada (Federal, New Brunswick and PEI), France, Finland, Sweden, Italy, Chile, Brazil, Costa Rica, Tunisia, the Dominican Republic and others

Co-authored 21st Century Skills: Learning for Life in our Times (book) and frequently lectures on this topic, STEM, and Education Technology


Advisory Board Bios – Charles Fadel (continued)

Work spans the continuum of Schools, Higher Education, and Workforce Development/Lifelong Learning

Visiting Practitioner at Harvard Graduate School of Education, exploring curriculum redesign concepts

Visiting Scholar at Wharton/UPenn where he recently taught a class on Technologies for Learning (receiving highest student rating)

Distinguished Guest to the President’s Council of Olin College Incubating DigData, a non-profit addressing the convergence of 21st Century

Skills and psychology to better teach Statistics and Probabilities Creative Advisor to the Hayden planetarium of the Boston Museum of Science

on their new implementations Advising innovative school systems in Brazil (Lumiar) and Chile (Innova100) Recently served on the Massachusetts Governor’s Readiness Project as well

as its 21st Century Skills task force Contributed many articles and has been interviewed in publications

such as Technology & Learning, New Media Consortium, Massachusetts Association for Supervision and Curriculum Development, eSchool News, Education Week, University Business, EETimes, and others


Advisory Board Bios – Charles Fadel (continued)

Spoke at numerous Education conferences, including the Consortium for School Networking (COSN), the National School Boards Association (NSBA), the National Center for Technology Innovation (NCTI), and the Masie Center’s learning conferences

Awarded five patents on video, content, and communication technologies

Holds a Bachelor of Science in electronics with course concentration in quantum and solid-state physics with a minor in neuroscience, and a Master of Business Administration in international marketing

An avid reader, he has auto-didactically learned disciplines such evolutionary psychology and comparative linguistics. He also enjoys the lessons of classical history


Advisory Board Bios – Hans Van Delden

Partner, Vice President, Technology Practice at Booz & Company 15 years in Management Consulting focused on the consumer products,

education services, media and retail industries Led 10+ strategy, operations and technology programs for several leading

Education companies – e.g.,: Technology Strategy / Architecture for an Online Assessment solution (Educational

Assessment Provider) – included an evaluation of the technology providers in the educational assessment market

Architecture Review (functional and technical) of custom-developed Online Assessment Platform (High-stakes State Educational Assessment Provider)

Assessment, Strategy and Roadmap for the Online Assessment Solutions (Educational Services company) – included developing a technology strategy and investment roadmap for a newly acquired online assessment platform

Information Business Strategy (Educational Services company) - evaluated several potential new information-based businesses for our client

Latin American Growth Strategy (High-stakes Educational Assessment Company) Operations Diagnostic for Puerto Rican division (High-stakes Educational

Assessment Company) - included an end-to-end assessment of the operations to identify organizational, process and automation improvements


Advisory Board Bios – Hans Van Delden (continued)

Experience working in 12 countries across North America, Europe and Asia

Twice awarded Booz & Company’s Professional Excellence Award, our highest award for client service

Earned five degrees from the Massachusetts Institute of Technology after five years of study

Holds an MBA and a Master of Science in Civil Engineering Holds Bachelor of Science degrees in Mathematics, Civil Engineering and

Management MIT’s sole Rhodes Scholar candidate and Texas Rhodes Scholar finalist

in 1995 Recognized for achieving the highest score in North America on the

first general actuary exam in 1993


Advisory Board Bios – Steve Katz

20+ years as a driving force and thought leader in technology innovation and strategy

Held a wide range of management roles, advancing innovation with IBM, Apple, and Sony Pictures Entertainment

Also served as an executive strategy consultant to leading technology firms including Microsoft, Sony Corporation and SAP

Serial entrepreneur and founder of a number of pioneering ventures backed by Kleiner Perkins and other top VCs

Currently, Steve is founding partner in a strategy consulting boutique called O8 Partners, based in Silicon Valley, which helps drive major partnerships and business strategy for emerging technology companies

Currently, entrepreneur-in-residence at MHS Capital, an early stage venture firm funded by prominent Silicon Valley founders and executives

Involved in non-profit and philanthropic activities, including serving as a founding member of the Full Circle Fund. The Full Circle Fund partnered with outstanding social innovators to explore different approaches to Educational reform

Holds BS an MS degrees in Computer Science from Columbia


Advisory Board Bios -John Behrens

Director, Networking Academy Learning Systems Development, Cisco, Inc. Responsible for the design and development of the Networking Academy’s

global on-line instructional offerings that integrate interactive curricular and assessment technologies with hands on activities. These curriculum, assessment, and gaming tools serve more than 700,000 students per semester in 165 countries across more than 10 languages. Over 100 million assessments have been delivered to date through the Cisco Networking Academy.

Dr. Behrens’ team integrates complex simulations in instruction and assessment and developed authoring tools to enable instructor-led visualization of abstract networking concepts.

John’s research interests focus on the intersection of cognitive, statistical and computing issues in instruction, assessment, and on-line learning. Dr. Behrens has published chapters in a number of widely used psychology reference books along with articles in a range of educational, psychological, and statistical journals.

Adjunct Assistant Research Professor, Department of Psychology, University of Notre Dame (Indiana, USA).


Advisory Board Bios -John Behrens (continued)

Master’s Degree in Special Education and Ph.D. in Educational Psychology, Arizona State University.

B.A. Psychology and Philosophy, University of Notre Dame. Prior to joining Cisco, John was a tenured Associate Professor of

Psychology in Education, Arizona State University.


Advisory Board Bios – Barry Topol, CPA, MBA

CPA, MBA Managing Partner, Assessment Solutions Group

Designed and co-developed ASG’s assessment cost model Chief Financial Officer, Harcourt Assessment

Drove the review, analysis and ultimate restructuring/outsourcing of Harcourt Assessment’s online assessment platform

Established relationships with outside “best of breed” technology partners

Chief Financial Officer for several technology companies Served as CIO/CFO for internet based start-up of a Fortune 500

Corporation; successfully developed the company’s website. Company won a “Webby” award for best online yellow pages site.

C-Level experience in educational publishing, high technology, internet and other industries

Managed technology departments and key technology initiatives in several companies


Advisory Board Bios – Edward Roeber, Ph.D.

40 years of experience in Assessment Adjunct Professor of Education, Measurement and Quantitative

Methods, College of Education, Michigan State University Teaches courses on educational measurement Works on projects to improve the assessment skills of prospective

and current educators Provides support for faculty and students on assessment

Senior Executive Director in the Michigan Department of Education OEAA

Responsible for overseeing the assessments of general education students, students with disabilities and ELLs, as well as the accreditation and accountability programs

Conducted statewide online assessment pilot in 2006


Advisory Board Bios – John F. Olson, Ph.D.

25+ years of experience providing services, technical assistance, and support to states, the USED, CCSSO, testing companies, researchers, and others

Served as a technical advisor and independent consultant on a variety of measurement and statistical issues for international, national, state, and local assessment programs

Served in key leadership roles on several Enhanced Assessment Grants from the U.S. Department of Education.

Vice President for Psychometrics and Research Services, Harcourt Assessment, Inc.

Deep experience with online assessment including: Conducting surveys of states on their use of CBT (for TILSA SCASS) Authoring two published reports on the comparability of CBT-PPT Serving as technical advisor to states implementing an OLA.


Program Lead / Executive Bio – Jay Whitchurch

CEO and co-founder of campus2careers (open source solution), an online career community that helps college students prepare for, find, and launch their careers

Extensive leadership experience as an executive with two large Education cos. Senior Vice President, Educational Assessment (K-12) at Pearson Education

Managed $80M P&L for K-12 Assessment portfolio that included classroom-based (formative), skills (diagnostic/readiness), comparability (achievement), and reporting/analysis product and services

Led the Strategy, Products, Services and Operations organizations and served on the Executive Leadership team

Chief Information Officer and Senior Vice President at Harcourt Assessment Led a team of 150+ professionals in delivering on all aspects of Technology services for a

$300M Assessment business: Strategy and Planning, Architecture, Applications, Infrastructure, Program Management, and Vendor Management

Oversaw a portfolio of applications and infrastructure that supported the Catalog and Custom State Contract Assessment businesses and Back-office business needs

Directed strategy for developing online assessment and digital supply chain capabilities Orchestrated significant turnarounds on two underperforming, multi-million dollar, core

application initiatives


Program Lead / Executive Bio – Jay Whitchurch (continued)

13 years in management consulting driving strategic business transformations for global Healthcare, Education, Nonprofit, and Public Sector clients

Principal, Booz & Company, Strategy/Technology Practice Managed executive relationships and led client engagements in defining new strategies

and planning/executing large-scale business transformation initiatives Worked with a leading Education company to define an online strategy with associated

business cases, priorities, required investments, and transition approach Led a major transformation initiative for one of the world’s largest Healthcare cos. Drove development of a customer applications strategy for a trillion dollar federal agency

Senior Consultant, Accenture Led teams of client and Accenture staff on major technology change initiatives Design and implemented large-scale, Public Sector systems

Continues to exercise his experience and passion for transforming the Education system through campus2careers and Education initiatives/speaking engagements

Holds an MBA and a Bachelor of Business Administration degree from the McCombs School of Business (The University of Texas at Austin)

Currently serves on the Boards of The Money Academy (a K-8 Financial Literacy curriculum) and Circular Energy (an alternative energy company)


assessment platform development recommendations building the next generation assessment platform –...

Documents

education slide

education system

new assessments

new generation of assessments

assessment flexible

new technology trends

assessment transformation

department of education