CITI Course Site Updates CITI Course Site Updates

Paul Braunschweiger Ph.D.Paul Braunschweiger Ph.D.

Update on CITI Software Update on CITI Software Implementation.Implementation.

Paul Braunschweiger Ph.D.Paul Braunschweiger Ph.D.

Course site UpdatesCourse site Updateswww.citiprogram.orgwww.citiprogram.org; ; www.irbtraining.orgwww.irbtraining.org

• Human Subjects Human Subjects – Basic and RefresherBasic and Refresher– Spanish versionsSpanish versions

• GCPGCP– English, Spanish English, Spanish

• Lab AnimalLab Animal– Basic CoursesBasic Courses– Species Specific CoursesSpecies Specific Courses

• RCRRCR– Courses for 4 disciplineCourses for 4 discipline– Public AccessPublic Access

• Health Information Privacy and Security (HIPS)Health Information Privacy and Security (HIPS)• International Course SiteInternational Course Site

– Public accessPublic access

CITI Web Site Support CITI Web Site Support

• Server CapacityServer Capacity– UM IT Server and Software consultants analyzed UM IT Server and Software consultants analyzed

the configuration and code. the configuration and code. • 2000 concurrent users2000 concurrent users

– Several server and software modifications since 5-Several server and software modifications since 5-07. 07.

• 2 New web servers added for load balancing (4).2 New web servers added for load balancing (4).• Update ASP version.Update ASP version.• Code optimization.Code optimization.

– >1100 concurrent users.>1100 concurrent users.• Safety limitation is 1500. Safety limitation is 1500.

– 22ndnd Database server added for back-up. Database server added for back-up.• Hired a SQL database programmer (40%) Hired a SQL database programmer (40%)

CITI Program GrowthCITI Program Growth

00

100100

200200

300300

400400

500500

600600

PA

RT

ICIP

AT

ING

OR

GA

NIZ

AT

ION

SP

AR

TIC

IPA

TIN

G O

RG

AN

IZA

TIO

NS

1-Aug-00

17-Apr-01

15-Aug-02

13-Aug-03

15-Aug-04

1-Aug-05

CITI PARTICIPANTS 2000 TO 2007CITI PARTICIPANTS 2000 TO 2007

1-Sept-06

800800

700700

1-April -07

900900

1-Sept-07

832832

Which Human Subjects Protection Course (s) did Which Human Subjects Protection Course (s) did you complete?you complete?

• Basic course for biomedical investigators.Basic course for biomedical investigators. 29%29%

• Basic course for social & behavioral Basic course for social & behavioral investigators.investigators. 53%53%

• A biomedical research Refresher Course.A biomedical research Refresher Course. 8%8%

• A social & behavioral research Refresher A social & behavioral research Refresher Course.Course. 10%10%

N= >11,000; 8-25-07 to 9-27-07N= >11,000; 8-25-07 to 9-27-07

Does your institution rely exclusively on the CITI Does your institution rely exclusively on the CITI Program to provide mandated instruction in the Program to provide mandated instruction in the

protection of human research subjects?protection of human research subjects?

• Yes, The CITI Program is the only Yes, The CITI Program is the only option available at my institution. option available at my institution. 71%71%

• No, my institution offers seminars, No, my institution offers seminars, classes, or other options as an classes, or other options as an alternative to CITI.alternative to CITI. 29%29%

N= >11,000; 8-25-07 to 9-27-07N= >11,000; 8-25-07 to 9-27-07

User Satisfaction SurveysUser Satisfaction Surveys

0%

5%

10%

15%

20%

25%

30%

FR

EQ

UE

NC

Y

1 2 3 4 5 6 7 8 9

RATING

The quiz questions address the relevant issues in the modules?

1 = Strongly disagree; 5 = Don't know; 9 = Strongly agree.

CITI 2 Software UpdateCITI 2 Software Update

Features, Modifications, and Planned Features, Modifications, and Planned Upgrades Upgrades

October 2007October 2007

What Does the CITI Software What Does the CITI Software include?include?

Tools to Tools to • Create and modify coursework.Create and modify coursework.• Create and modify quizzes.Create and modify quizzes.• Design custom curricula for learners in each institution, Design custom curricula for learners in each institution,

depending on their research activities.depending on their research activities.• Design and administer surveys.Design and administer surveys.• Help each learner track progress through their coursework.Help each learner track progress through their coursework.• Provide institutional administrators with pertinent reports- who Provide institutional administrators with pertinent reports- who

has completed the required training?has completed the required training?• Monitor site usage (for site administrators).Monitor site usage (for site administrators).

CITI "2" Features as of January CITI "2" Features as of January 20072007

• A single user account can be affiliated with multiple A single user account can be affiliated with multiple institutions.institutions.

• Learner can have multiple grade books (human subjects, Learner can have multiple grade books (human subjects, RCR, animal research, etc) in each institution.RCR, animal research, etc) in each institution.

• Enrollment questions are used to determine custom Enrollment questions are used to determine custom curriculum for each learner, per institutional wishes.curriculum for each learner, per institutional wishes.

• When learners earn a completion report, email can be When learners earn a completion report, email can be sent to one or all institutional administrators.sent to one or all institutional administrators.

One Learner One Learner Account, Multiple Account, Multiple Institutions on the Institutions on the

Main MenuMain MenuAffiliate with another institution

Other Significant EnhancementsOther Significant Enhancements

• Code continually rewritten to be more efficient and to Code continually rewritten to be more efficient and to support more concurrent users.support more concurrent users.

Monthly Quizzes Taken, 2005 to 2007

0

100000

200000

300000

400000

500000

600000

700000

Monthly Quizzes taken

CITI 2 Features as of August CITI 2 Features as of August 20072007

• In addition to training record set downloads available to In addition to training record set downloads available to institutional administrators, record sets can be downloaded institutional administrators, record sets can be downloaded automatically to local personnel management databases.automatically to local personnel management databases.

• Learners can be placed directly into the CITI Main Menu after Learners can be placed directly into the CITI Main Menu after learners login to the institution's host network, bypassing the learners login to the institution's host network, bypassing the usual CITI login process.usual CITI login process.

• Optionally, reminder emails sent to learners as completion Optionally, reminder emails sent to learners as completion reports expire.reports expire.

• Multiple learner accounts can be merged into one account (still Multiple learner accounts can be merged into one account (still undergoing beta testing).undergoing beta testing).

• Pre- and post-course completion surveys can be set up to collect Pre- and post-course completion surveys can be set up to collect feedback from learners. feedback from learners.

Automatic Downloads for Institutional Automatic Downloads for Institutional Administrators- Record Set Formats Available Administrators- Record Set Formats Available

(comma or tab-delimited files)(comma or tab-delimited files)

• All completion reports earned at the institution since All completion reports earned at the institution since ~January 2006 (+/- module names and quiz scores).~January 2006 (+/- module names and quiz scores).

• Completion reports earned since the last dataset Completion reports earned since the last dataset download (+/- module names and quiz scores).download (+/- module names and quiz scores).

• Most recent completion report earned by each learner Most recent completion report earned by each learner (+/- module names and quiz scores).(+/- module names and quiz scores).

• Completion reports that will expire within 30, 60, 90, Completion reports that will expire within 30, 60, 90, 120, or 180 days from today's date.120, or 180 days from today's date.

Data fields in reports can be added or removed and sequenced as needed for each institution.

Same person

??

Direct Placement into the CITI Main Menu and Direct Placement into the CITI Main Menu and Automatic Dataset Transfers to InstitutionsAutomatic Dataset Transfers to Institutions

To 1) allow learners to use their institutional network username and To 1) allow learners to use their institutional network username and password to enter the CITI main menu directly and 2) automate learner password to enter the CITI main menu directly and 2) automate learner completion dataset transfers into local personnel management completion dataset transfers into local personnel management software the learner's institutional records and the learner's CITI software the learner's institutional records and the learner's CITI records records must share a unique identifiermust share a unique identifier to ensure a perfect match. to ensure a perfect match.

CITI Website

-John Doe's records

Local institutional training database

-John Doe's records

Direct Learner Placement Direct Learner Placement into the CITI Main Menuinto the CITI Main Menu

• Reduced burden on learners and Reduced burden on learners and administratorsadministrators

Learner goes to CITI website and creates a username and password

Learner chooses one or more institutions and provides contact and profile information

Learner answers enrollment questions to create course grade books

At Main Menu Learner completes coursework, earns completion report(s)

Learner prints completion report(s), sends to admin office

Admin office downloads records (email or dataset download) and enters data into local database

Learner logs into local institutional network using their existing username and password

Learner visits a local network page that collects profile and coursework information

A unique learner ID is added, then data is encrypted and sent to CITI server to create a learner "session"

CITI server automatically sends encrypted dataset to institution's server, unique ID used to merge dataset with local training database

Learner goes to CITI website and creates a username and password

Learner chooses one or more institutions and provides contact and profile information

Learner answers enrollment questions to create course grade books

At Main Menu Learner completes coursework, earns completion report(s)

Learner prints completion report(s), sends to admin office

Admin office downloads records (email or dataset download) and enters data into local database

Learner logs into local institutional network using their existing username and password

Learner visits a local network page that collects profile and coursework information

A unique learner ID is added, then data is encrypted and sent to CITI server to create a learner "session"

CITI server automatically sends encrypted dataset to institution's server, unique ID used to merge dataset with local training database

University of Wisconsin Beta Test of University of Wisconsin Beta Test of Local Authentication and Direct Local Authentication and Direct Placement into CITI Main MenuPlacement into CITI Main Menu

• Calvin Chang at UW providing coding and development Calvin Chang at UW providing coding and development support in collaboration with Stephanie Manuel and Chris support in collaboration with Stephanie Manuel and Chris Simmons (CITI programmers)Simmons (CITI programmers)

• Preliminary testing in September successful- proof of Preliminary testing in September successful- proof of concept verifiedconcept verified

• Final coding and testing in late OctoberFinal coding and testing in late October• Template will be available to all CITI institutionsTemplate will be available to all CITI institutions

University of Wisconsin Sample University of Wisconsin Sample ScreensScreens

These links take a non-UW learner to the CITI home page

xxxxxxxxxxx

xxxxxxxxxxx

Once submitted, a unique ID and other profile info is sent to the CITI server and a learner "session" is set up.

In this case, the learner groups were not sent to the CITI server, so the learner must click to go to the enrollment questions to receive a curriculum.

The UW Learner now can be...The UW Learner now can be...

Absolutely identified in dataset transfers from the CITI server to a UW server, Absolutely identified in dataset transfers from the CITI server to a UW server, because of the learner unique ID number shared by CITI and UW.because of the learner unique ID number shared by CITI and UW.

A script can be written to automatically process the dataset received from CITI so A script can be written to automatically process the dataset received from CITI so that UW administrators do not have to spend time matching CITI and UW learner that UW administrators do not have to spend time matching CITI and UW learner records. records. and...

Automatic Downloads for Institutional Automatic Downloads for Institutional Administrators- Record Set Formats Administrators- Record Set Formats

Available (comma or tab-delimited files)Available (comma or tab-delimited files)

• All completion reports earned at the institution since All completion reports earned at the institution since ~January 2006 (+/- module names and quiz scores).~January 2006 (+/- module names and quiz scores).

• Completion reports earned since the last dataset download Completion reports earned since the last dataset download (+/- module names and quiz scores).(+/- module names and quiz scores).

• Most recent completion report earned by each learner (+/- Most recent completion report earned by each learner (+/- module names and quiz scores).module names and quiz scores).

• Completion reports that will expire within 30, 60, 90, 120, or Completion reports that will expire within 30, 60, 90, 120, or 180 days from today's date.180 days from today's date.

Data fields in reports can be added or removed and sequenced as needed for each institution.

User Interface and Site User Interface and Site UsabilityUsability

• Although overall ease of site navigation and Although overall ease of site navigation and registration process receive high survey registration process receive high survey satisfaction scores, about 15% of learners are satisfaction scores, about 15% of learners are dissatisfied with one or both.dissatisfied with one or both.

• Two focus areas for improvement Two focus areas for improvement – Choosing affiliated institutionsChoosing affiliated institutions– Course enrollment processCourse enrollment process

Problems in Choosing Affiliated Problems in Choosing Affiliated InstitutionsInstitutions

• Some learners are confused by the multiple drop-down selection Some learners are confused by the multiple drop-down selection boxes, and believe they are supposed to choose one institution from boxes, and believe they are supposed to choose one institution from each category. each category. A redesigned process is needed.A redesigned process is needed.

Difficulty in Navigating the Difficulty in Navigating the Enrollment QuestionsEnrollment Questions

• Some learners have a difficult time Some learners have a difficult time navigating through the enrollment questions.navigating through the enrollment questions.

• Some specific challenges:Some specific challenges:– Currently, all learners have to see all enrollment Currently, all learners have to see all enrollment

questions.questions.– Some institutions have 10 or more enrollment Some institutions have 10 or more enrollment

questions that all appear on one page.questions that all appear on one page.– It can be difficult for learners to understand which It can be difficult for learners to understand which

questions they should answer. questions they should answer.

Solution- One Enrollment Question per PageSolution- One Enrollment Question per Page

Q1 (ID#233). Which of the following apply to your research activities?

Human subjects research. [if chosen, turns on Q#234; no groups associated]Animal research . [if chosen, turns on Q#245; no groups associated]Use of hazardous agents in research. [if chosen, turns on Q#250; no groups associated]

Q4 (ID#245).Do you serve on the IACUC?

Yes. [Essentials for IACUC Members; stage 1]No.

Q2 (ID#234). Which of the following apply to your human research activities?

Biomedical studies. [if chosen, turns on Q#235; no groups associated]Social or Behavioral studies. [CITI Social and Behavioral Course; stage 1]GCP. [CITI GCP Course; stage 1]\

Q3 (ID#235).Do you need to take the Basic or Refresher Biomed Course?

Basic (stage 1). [CITI Biomedical Course; stage 1]Refresher (stage 2) . [CITI Biomed Course; stage 2]

Q5 (ID#250).Do you work with select agents?

Yes. [Select Agent Training; stage 1 and Basic Biosafety; stage 1]No. [Basic Biosafety; stage 1]

(Skipped)

(Skipped)

(Skipped)

DONE

Q1 (ID#233). Which of the following apply to your research activities?

Human subjects research. [if chosen, turns on Q#234; no groups associated]Animal research . [if chosen, turns on Q#245; no groups associated]Use of hazardous agents in research. [if chosen, turns on Q#250; no groups associated]

Q4 (ID#245).Do you serve on the IACUC?

Yes. [Essentials for IACUC Members; stage 1]No.

Q2 (ID#234). Which of the following apply to your human research activities?

Biomedical studies. [if chosen, turns on Q#235; no groups associated]Social or Behavioral studies. [CITI Social and Behavioral Course; stage 1]GCP. [CITI GCP Course; stage 1]\

Q3 (ID#235).Do you need to take the Basic or Refresher Biomed Course?

Basic (stage 1). [CITI Biomedical Course; stage 1]Refresher (stage 2) . [CITI Biomed Course; stage 2]

Q5 (ID#250).Do you work with select agents?

Yes. [Select Agent Training; stage 1 and Basic Biosafety; stage 1]No. [Basic Biosafety; stage 1]

(Skipped)

(Skipped)

DONE

Q1 (ID#233). Which of the following apply to your research activities?

Human subjects research. [if chosen, turns on Q#234; no groups associated]Animal research . [if chosen, turns on Q#245; no groups associated]Use of hazardous agents in research. [if chosen, turns on Q#250; no groups associated]

Q4 (ID#245).Do you serve on the IACUC?

Yes. [Essentials for IACUC Members; stage 1]No.

Q2 (ID#234). Which of the following apply to your human research activities?

Biomedical studies. [if chosen, turns on Q#235; no groups associated]Social or Behavioral studies. [CITI Social and Behavioral Course; stage 1]GCP. [CITI GCP Course; stage 1]\

Q3 (ID#235).Do you need to take the Basic or Refresher Biomed Course?

Basic (stage 1). [CITI Biomedical Course; stage 1]Refresher (stage 2) . [CITI Biomed Course; stage 2]

Q5 (ID#250).Do you work with select agents?

Yes. [Select Agent Training; stage 1 and Basic Biosafety; stage 1]No. [Basic Biosafety; stage 1]

(Skipped)

DONE

(Skipped)

(Skipped)

Ongoing Feature EnhancementsOngoing Feature Enhancements

• Conversion of entire website to support "on the fly" foreign Conversion of entire website to support "on the fly" foreign language translation of web pages (testing nearly complete)language translation of web pages (testing nearly complete)

• Conversion of entire website from ASP pages to ASP.Net Conversion of entire website from ASP pages to ASP.Net pages.pages.

• Sophisticated code to check and alert a learner when a Sophisticated code to check and alert a learner when a learner's previous coursework suggests that they should be learner's previous coursework suggests that they should be in a different course or course stage than what their in a different course or course stage than what their enrollment question choices would suggest.enrollment question choices would suggest.

• Solve the problems related to transferring completion credits Solve the problems related to transferring completion credits between organizations. between organizations.

• Implement additional tools for the CITI Office staff to speed Implement additional tools for the CITI Office staff to speed set-ups.set-ups.

• Implement additional survey tools and utilitiesImplement additional survey tools and utilities

Updates from Executive Advisory Updates from Executive Advisory Committee (EAC) -Committee (EAC) -

Ernie Prentice Ph.D.Ernie Prentice Ph.D.

Final CITI Structure and Processes Final CITI Structure and Processes DocumentsDocuments

Karen HansenKaren Hansen

CITI NewsletterCITI Newsletter

Susan Rose Ph.D.Susan Rose Ph.D.

Biomedical Course UpdateBiomedical Course Update

Sally MannSally Mann

Social and Behavioral Course Social and Behavioral Course UpdateUpdate

Lorna HicksLorna Hicks

International Research International Research Platform Updates Platform Updates

Karen Hansen & Michel EckersleyKaren Hansen & Michel Eckersley

International Collaborations and International Collaborations and Demonstration ProjectsDemonstration Projects

Paul Braunschweiger Ph.D.Paul Braunschweiger Ph.D.

International Collaborations and International Collaborations and Demonstration ProjectsDemonstration Projects

• IntroductionIntroduction• Update on Canadian CITI – Laura-Lee Balkwill, Denise Update on Canadian CITI – Laura-Lee Balkwill, Denise

StockleyStockley• Update of Taiwan Program – Benjamin KuoUpdate of Taiwan Program – Benjamin Kuo• Update of Africa Program – Ifeoma OkoyeUpdate of Africa Program – Ifeoma Okoye• Latin America/South America – Sergio Litewka, Mario Latin America/South America – Sergio Litewka, Mario

Tristan, Roxana Lescano and Carla DeSousa Tristan, Roxana Lescano and Carla DeSousa • Spanish Language Course SiteSpanish Language Course Site

Update on CITI Canada Update on CITI Canada

Laura-Lee Balkwill, Denise StockleyLaura-Lee Balkwill, Denise Stockley

Update of CITI TaiwanUpdate of CITI Taiwan

Benjamin Kuo MDBenjamin Kuo MD

Update of Africa ProgramUpdate of Africa Program

Ifeoma Okoye MDIfeoma Okoye MD

Update of CITI Latin America/South Update of CITI Latin America/South AmericaAmerica

Sergio Litewka, Mario Tristan, Sergio Litewka, Mario Tristan, Roxana Lescano and Carla deSousaRoxana Lescano and Carla deSousa

CITI Worldwide FootprintCITI Worldwide Footprint

CITI Non-English Language Course CITI Non-English Language Course SiteSite

• Funding from NAIDS June 2007 to translate Funding from NAIDS June 2007 to translate GCP Course to Spanish.GCP Course to Spanish.

• Vendor problems to overcome.Vendor problems to overcome.• Technical problems to overcome.Technical problems to overcome.

Spanish and Portuguese Spanish and Portuguese Language Course SitesLanguage Course Sites