new technology support for government and social research russian-u.s. workshop with ilya zaslavsky...

New Technology Support for

Government and Social Research

Russian-U.S. Workshop

with Ilya Zaslavsky

San Diego Supercomputer Center

University of California, San [email protected]

“American Speakers”, Russian-US Workshop, Moscow-Tomsk, 10-20 Feb 2003

Disclaimer

Besides my own slides and slides developed and commonly shared within the DAKS group of SDSC, this and other presentations given in Moscow and

Tomsk during the Russian-U.S. workshop Feb 10-20, 2003, partly use fragments of slide presentations

available from multiple sites on the Internet.

Copyright to these slides remains with their respective owners.

The Moscow and Tomsk presentations are not intended for Web publication.


Overview of Day 2

• Distance Education: Personal experiences

• Federal Statistics• Social Statistics Online• Sociology Workbench

– demo!!


What is DE?

• Learning that happens when the instructor and student are in different physical locations

• First ‘distance education technology’


DISTANCE EDUCATION FOR WHOM

• WORKING PEOPLE

• HANDICAPPED PEOPLE

• INTERCONTINENTAL STUDENTS

• CONTINUING EDUCATION DURING CARRIER

Distance Education brings the information to the student.

It is very useful for an individual who cannot go to an ordinary class.


SYNCHRONOUS ASYNCHRONOUS

Distance Education Tools

• Doesn’t require the simultaneous participation of all students and instructor.

• Students may chose their own instructional time frame

• More flexible than synchronous.

• Simultaneous participation of both student and instructor is required.

• Interaction is done in real time.

There are basically two categories of distance

education delivery system.

Television, Audiographics, Computer Conferencing, IRC can be considered as synchronous distance education tools.

E-mail, listservs, audiocasstte, videotape and WWW can be considered as asynchronous distance education tools.


• PRINT (1800s)

• POSTAL SERVICE (1850s)

• TELEPHONE (1875)

• RADIO (1895)

• AUDIO TAPE (1945)




• BROADCAST TELEVISION (1953)

• VIDEO TAPE (1960)

• AUDIO TELE-CONFERENCING (1960)

• CABLE TELEVISION (1965)



• COMPUTER ASSISTED INTRUCTION (1975)

• COMPUTER CONFERENCING (1980)• SATELLITE DELIVERY (1980)

• AUDIOGRAPHIC TELE-CONFERENCING (1980)


Distance Education Tools• FACSIMILE (1980)

• VIDEO CONFERENCING (1980)• VIDEO DISC (1984)

• CD-ROM COMPACT DISC (1985)• COMPRESSED VIDEO (1988)• MULTIMEDIA (1989)• WIRELESS WORKSTATIONS (1996)


INTERNET tools in Distance Education

• E-MAIL SOFTWARE: Free form text messages between lecturer and individual students.

• MAILING LISTS: Delivering messages from instructor to students.• TELNET, WEB PAGE: Delivery of learning materials and

assignments to student as needed.• IRC, COMPUTER CONFERENCE: Text based commentary between

students about the learning resources.• AUDIO CONFERENCE EQUIPMENT or INTERNET PHONE:

Synchronous audio communication students and lecturer• ANSWERING MACHINE: Asynchronous audio communication

students and lecturer.

Ref: Distance Learning Glossary

http://www.wested.org/tie/dlrn/course/glossary.html#internet

FedCon Alliance/NPACI Meeting

Synchronous Distance Teaching on a Low Budget:

Ed Center’s Experiences and Student Assessment

Ilya Zaslavsky, [email protected]

http://www.edcenter.sdsu.edu

http://www.edcenter.sdsu.edu/disted

voice: 619.594.0491

fax: 619.594.0433


The setting

The setting

This is the GIS lab (17 Pentium133, Windows 95, with Windows NT server) at WMU, right after a lecture

This is me (Ilya Zaslavsky) in San Diego, teaching a class

Two classes: 567 “Geodata Handling” and 569 “GIS”, both relatively small (8-14 students).

Additional hardware and software - not to exceed $200 per machine, or $500 total

Teaching assistant (Kathleen Baker) at WMU


Organization of the Distance Learning System (SDSU - WMU, Geography 567 &

569, 1997-98)

Lab Serverat WMU

TA’sWorkstation

Instructor’s Workstation

Lab DemoComputer

Web Serverat WMU

Web Serverat SDSU

San Diego

Kalamazoo

Tools used:- synchronous

NetMeeting(sharing applications,whiteboard,chat)

- asynchronousclass web pages

formatted in DocReview(web-baseddiscussions, comments, Q&A)

E-Mail


the classroom, with cameraon tripod, Kathleenin front of the class


Using VideoUsing Video

• Camera on the instructor’s side– pointing at instructor’s face, at active-matrix laptop

screen (not too clear), at scratch paper (possible, but slow)

– move as little as possible, for good overall signal quality

• Camera in the lab:– pointing at students or TA, often turned off (it’s important to

watch that something is going on on the other side - but not at the expense of audio!)

– useful for student presentationsVideo is still a “frosting”, so far most useful in one-to-one

interaction (with grad. students, etc.)


Using AudioUsing Audio

• turned out to be the major problem• full-duplex audio is a must, in half-duplex mode

microphone switches direction of transmission when it picks up extraneous noises

• stay up-to-date with sound and network drivers

• transmit signal in one direction (turn off one microphone, one or both videos)

• don’t breathe, hold the mike in your teeth• have a phone line as backup


Sharing chat, whiteboard, programs

Sharing chat, whiteboard, programs

• Chat window:– good for questions from audience, repeating important points,

or when connection is poor

• Whiteboard:– turned out very useful, though drawing with a mouse requires

some skill…– having a set of pre-built images, and pasting them into

whiteboard was the best

• Sharing applications:– very important; this is what makes it different from standard

videoconferencing– SPSS, ArcView with extensions, Netscape, Stella, etc. … but

slow...


The asynchronous part: web-based discussions

The asynchronous part: web-based discussions

• Just placing lectures on the Web is not enoughhttp://unix.cc.wmich.edu/zaslavsk/classes/567 and

/569

• E-mail discussion of lecture notes and assignments - first 4-5 weeks

– class e-mail lists;– have to forward personal messages to the list...

• DocReview, a system for Web-based collaborative text editing/commenting

– don’t have to switch to e-mail to do comments– boost in interaction in 567, more focused (contextual) questions


The chronology (weeks 1-6)


Week 1

Week 2

Week 3

Week 4

Week 5

Week 6

new techniques connection overall

sharing whiteboard

lectures in DocReview

unstable

bad or absent, on one or both sides

sporadic, in and out

chat window, voice, video of instructor, students

e-mail discussions, chat,video of chalkboard

total disaster

quite decent, nota circus anymore

decent at times

sharing applic. (SPSS), whiteboard graphics kit;569 students lead live discussion

bad in the firstlecture, good inthe afternoon

doing it from newsite, with a phonebackup

60/40

50/50

TA gets a TA unstable, but present for entire classes

sporadic




Week 7

Week 8

Week 9

Week 10

Week 11

Week 12

new techniques connection overall

567 present their maps live, video and audio on; sharing ArcView

students get folder pass-words (files wiped out…)

speakerphone for569, good in 567

** midterms **

good

excellent!!

good

finally… remote pointeron whiteboard...

both 567 and 569 goover Internet fine

** Ilya in K-zoo **

** midterms ** ** midterms **

Ilya in San Jose atSupercomputing’97

speakerphone for569, good in 567

connection is bad, sostudents do lab as planned;we try out Tango...


What do you do if you can’t do anything ?

What do you do if you can’t do anything ?

“… if a horde of guests suddenly appears at your doorstep, and you have but absolutely nothing to treat them with, go to the basement and pick up a mid-sized leg of lamb. This is what you do with it…” (the “Molokhovets book”, 1898, a famous Russian cookbook)

• try to connect to another network, even via a modem - forget about transmitting any useful signal, though

• type in the whole lecture using as many interaction tricks as possible, have your TA do standby commentary

• have a TA who can do it all without you


Student Assessment

Standard assessment• No significant difference in student performance compared to previous

years.

Student surveys• 3 surveys: at the beginning, middle, and end of each semester

– no previous distance learning experience for all, standard Web tools are fairly novel for many

• In the final survey (30 students, both factual and open-ended questions):

– all but three say that they are satisfied with what they have accomplished in this course

– all joined virtual meetings, used class web pages– 83% used whiteboard, 60% collaborated in shared applications (most

useful techniques); 30% used text chat– lack of communication with instructor, missed eye contact, asked less

questions, though interacted more with fellow students


Lessons Learned and Useful Strategies

• Advanced preparation and careful lecture scenarios are important, though some ad hoc freedom in using visual tools is necessary. Have a pre-built set of images to share in whiteboard.

• Involve students in presenting themselves (once they see that they can do it, it works!)

• Re-create or maintain traditional channels of student-teacher interaction as much as possible.

• Maintain all possible interaction channels active (e-mail, chat, whiteboard, DocReview, etc.)

• Archive discussions, whiteboards, chats.• Meet with students face to face, at least couple times.


Lessons Learned and Useful Strategies (2)

• Encourage student interaction with peers.• Have a set of back-up plans if network connection is bad,

including speakerphone.• Have a reliable support on the "receiving side" !!!!• Survey students and keep a class log, this will help in detecting

strategies that work.• Use a second pair of computers to transmit different

components of the conference.• Turn off video in one or both directions, to keep audio clear.• Accept that this approach is not necessarily good for every

student, and some won't be happy.• Re-think your teaching style and habits!


“Distance learning” in Internet times

“Distance learning” in Internet times

1. Lecture notes on the Web (everywhere, electronic textbooks; Geographer’s craft - best example)

2. Lecture notes with some interactivity (tests, e-mail)3. Notes + session recording and playback (WLS - best example)4. Web-based collaboration within the classroom5. Synchronous chat, supplemented by occasional audio/video

with CU-SeeMe (NASA AMES to U. of North Dakota, $170,000, others)

6. Collaborative environment, sharing Java applets: Habanero (NCSA) - several sites

7. Synchronous audio/video, applications sharing + interactive asynchronous mode: NPAC to Jackson State U. (“Tango”) and WMU (“NetMeeting”)

syn

chro

no

us

rem

ote


URLsURLs

• This talk: http://unix.cc.wmich.edu/zaslavsk/classes/nov25/

• Michigan GIS Institute: http://www.wmich.edu/geography/GISinstitute/

• Tango web collaboratory (NPAC, Syracuse): http://trurl.npac.syr.edu/tango/

• Habanero collaboratory (NCSA): http://www.ncsa.uiuc.edu/SDG/Software/Habanero/

• Web Lecture System (North Carolina St. U.): http://renoir.csc.ncsu.edu/WLS/

• Ed. Center on Computational Science and Engineering: http://www.edcenter.sdsu.edu/

• Geographer’s Craft: http://www.utexas.edu/depts/grg/gcraft/contents.htm


Distance LearningSingapore-MIT Alliance (SMA)

Program

• A very selective Master/Ph.D. joint program• Involves National University of Singapore (NUS), Nanyang

Technological University (NTU) & MIT• Launched in July 1999 • Students undergoes initial orientation in Singapore, then 1

month in MIT followed by continuation of program in Singapore.

• Students earn single degree with indigenous registered institution

• 3 programs totaling about 67.5 conferencing hours/week• Expand to 5 programs totaling about 100 hours/week in

August 2001


SMA DE - Configuration

Studentaccess via web forrevision

NUS Auditorium / NUS Auditorium / SMART ClassroomSMART Classroom

Online digitizationonto videoserver

Video ServerVideo Server

MITMITAuditoriumAuditorium

Internet 2Internet 2

Audio / Video ConferencingAudio / Video Conferencing

Application Sharing via Internet 2Application Sharing via Internet 2

ISDNISDN

View videoView video


SMA DE Connectivity

NUS Gateway

SingAREN*GigaPoP

NUS Classroom

LAPoP

MITGateway

MIT Classroom14 Mbps ILC

OC-3

Abilene

155 Mbps


SMA DE – Distance Counseling

• Primary instructor works with remote assistants• Multiple platforms:

– Email– Discussion forum– Desktop conferencing– Chat rooms

• Lecturers equipped with Distance Education kits:– Desktop conferencing– Scanner– Writing tablet mouse

• Students have access to desktop multimedia conferencing stations


SMA DE – Lessons Learnt• Synchronous teaching/learning:

– A challenge to maintain Line of Presence in large classroom

– Sustained audio quality a challenge (noise & breakups)– Single screen mode demands higher video resolution– Delay in camera tracking system poses problems for

Q&A sessions – improvements using push-to-talk system

– Secondary communication channels for inter-team session monitoring to ensure rapid problem resolution


SMA DE – Lessons Learnt (cont)

• Pedagogical Issues:– Faculty needs to understand technical limitations– Adaptation required– Conducting an effective Q&A session is challenging– Backup connectivity activation disrupts flow of lessons

• Support Issues:– Requires well understood inter-team operational protocols– Inter-technical team building essential– Core technical staff available on mobile/beepers– 1-man operation highly risky– Replicated services in local helpdesk a necessity for quality

service


Distance Education Myths

• Distance education is about technology• Distance education is about replacing

professors http://e-education.mtt.ca/• Distance education is about saving/making

money http://www.polycom.com/streams/streamstation_overview/index.htm

• Distance education is about administrationhttp://129.128.5.72/main/index.jhtml

http://e-education.mtt.ca/



http://www.polycom.com/streams/streamstation_overview/index.htm






http://129.128.5.72/main/index.jhtml

http://129.128.5.72/main/index.jhtml


Increasing Competition

• offers more than 1,000 online courses and has enrolled over 12,000 students.

UCLA’s OnlineLearning.net

• offers 1,000 American Airlines frequent flyer points when you enrol in a Spring 2000 online course.


Federal Statistics

Social Statistics Online


Federal statistics

• 70 agencies, Census and BLS are the largest, but many more: SSA (program-based), EPA (for regulations), BTS…

• Little coordination: Office of Management and Budget + Interagency Council on Stat Policy

• Population estimates, redistricting (Census, $180 bil to local administration..)

• Consumer Price Index: wages, pensions, social services

• Health, Crime, Unemployment, “Digital Divide”• General Social Surveys


U.S. Census


General Social Survey

• The General Social Survey (GSS) is bi-annual personal interview survey of U.S. households conducted by the National Opinion Research Center (NORC). The first survey took place in 1972 and since then more than 35,000 respondents have answered over 2500 different questions. The mission of the GSS is to make timely, high-quality, scientifically relevant data available to the social science research community.

95. Do you think the use of marijuana should be made legal or not?

Should Should not Don’t know No answer


Data Archives

• Inter-university Consortium for Political and Social Research (ICPSR)– international data collections

• Roper Center for Public Opinion Data– Latin American Survey Data Bank and Japanese

Data Archive

• Data archives around the world - CESSDA maps

• Differences in data access procedures


Inter-University Consortium for Political and Social Research

(ICPSR)http://www.icpsr.umich.edu


Micro-level Data read into SPSS

World Values Survey (R. Inglehart); distributed by ICPSR


Roper Center for Public Opinion

http://www.ropercenter.uconn.edu


Surveys, and IT• American Transportation Survey (CATI, 65 thous

hholds), National Health Survey (NCHS): 5000 respondents, interviews + medical exams, National DNA sample

• All stages of survey development:– Planning/design (What is general population; sampling

schemes…)– Data collection (CATI, CAPI, CASI, PAPI)– Processing (lack of standards…)– Analysis – Dissemination (for non-specialists, user-friendly, single entry

point: Fedstats, Ferret, etc.)


What is FirstGov?http://www.FirstGov.gov


What is FirstGov?

• Point of access (“Web portal”) to Government information and services

• Portal is a free service that indexes Web-enabled government information and transactions (links to “electronic commerce” sites)

• FirstGov “Partners” program


Information Discovery Requirements of FirstGov

Data-bases

GILS Interface

Locator Records

Bibliographic Records

Web pages

FirstGov is required to interface with GILS-compliant servers, finding information in various formats, distributed across the Internet in catalogs, directories, databases


The Sociology Workbench v2.0


SWB and Digital Government


SWB Architecture – XML related

SWB Interface

XML Parser

XML Processor

Oracle

Function Query

XML Survey Document

XML Tree

SQL statements for table creation and the insertion of all question information (via JDBC)

Table Querying SQL statements (via JDBC)

Table output in XML format

XML Output

XML Output with Table Format Stylesheet

XML Survey Document

Client


The DDI Initiative

• The Data Documentation Initiative• A Project to Develop an XML Document Type

Definition for Data Documentation• Maps to 15 elements of the Dublin Core• 30 other recommended elements for social

science research & data management

• http://www.icpsr.umich.edu/DDI/codebook.html


In a DDI DTD XML codebook you can integrate meta-information about...

• Intellectual content of a study

• Its scope

• Methodological details

• Retrieval and dissemination policies

• File location and format


(+) References to accompanying documents,

e.g.

• Reports on methodology,• Publications,• Classifications lists,• Questionnaires and similar,• Computer syntax files,• Tables of results,

etc.


Challenges working with the ddi– All tags in the Variable Description portion of the

DTD are optional. – Flexibility of the ddi DTD allows for extensive

personal judgment in tag usage.

• Our Solutions– Place restrictions on tags by making them

mandatory for our application. For a listing of SWB mandatory tags and their usage: http://edcenterdev.sdsu.edu/SOURCE/ddi.html

– Use XSL Stylesheets to preprocess XML documents to adhere to our restrictions

ddi in SWBddi in SWB

http://edcenterdev.sdsu.edu/SOURCE/ddi.html










Example of a ddi compliant XML document

<?xml version="1.0" ?> - <codebook>- <docDscr> <guide> Sample Data From SWB </guide> </docDscr>- <stdyDscr>- <stdyInfo> <abstract>Example Survey </abstract> </stdyInfo> </stdyDscr>- <dataDscr>- <var name="SPANKING" format="String" ID="q1">- <valrng> <range min="0" max="9" /> </valrng> <txt>Favor Spanking to Discipline Child</txt> - <catgry> <catValu>1</catValu> <txt>Agree</txt> </catgry>- <catgry> <catValu>2</catValu> <txt>Neutral</txt> </catgry>- <catgry> <catValu>3</catValu> <txt>Disagree</txt> </catgry>- </var </dataDscr> </codebook>


XML Output from Oracle

• API - oracle.xml.sql.query.OracleXMLQuery

• In order to produce XML output from Oracle the SWB team had to create SQL functions to organize the tables in a way in which we can easily convert from Oracle tables to an XML document. This document can either be displayed to the screen or stylized to create a table.

• Output Tags- The tags are determined by the SQL query, and vary with each type of table.


Converting SWB to Web Service

• Wrote SOAP wrappers for each SWB function that we wanted to be a service– Wrapper calls the servlet and parses the resulting XML.

Returns the root node of the XML document

<?xml version="1.0" ?>

- <ROWSET>

- <ROW num="1">

<DESCRIP>Missing Data</DESCRIP>

<AMOUNT>979</AMOUNT>

<AMOUNT_PERCENT>33.71</AMOUNT_PERCENT>

</ROW>…

</ROWSET>


Survey Documentation and Analysis (SDA)

Program

• Written at UC Berkeley• Used by ICPSR and others-- referred to

as DAS (Data Analysis System)• Data files must be converted to SDA

format before use. ICPSR has converted a number of data sets in their topical archives into SDA format and are converting more.


Sources of Data at ICPSR (http://www.icpsr.umich.e

du)

• ICPSR topical archives– National Archive of Computerized Data on Aging

(NACDA)– National Archive of Criminal Justice Data (NACJD)– International Archive of Education Data– Substance Abuse and Mental Health Data Archive

(SAMHSA)

• General Social Survey• National Election Study


What Can You Do?

• Browse codebook

• Subset data

• Download data and documentation

• Run statistical procedures


Statistical Procedures

• Frequencies

• Crosstabs

• Comparison of means

• Comparison of correlations


What Else Can You Do?

• Recode (temporarily)

• Use control variables

• Use filter variables

• Use weight variable


Frequencies Program -- Specify Variables

• Row variable (required)

• Filter variables

• Weight variable


Frequencies Program -- Select Statistics

• Percents

• Central tendency -- mean, median, mode

• Variability -- standard deviation, variance

• Coefficient of Variation

• Standard error of the mean


Crosstabs Program -- Select Statistics

• Percents -- vertical (row), horizontal (column), total

• Chi square (Pearson’s, Likelihood Ratio)• Eta• Gamma• Tau-b and Tau-c• Somer’s d


Filter Variables

• Can also use filter variables to select particular cases

• Variable name (____; ____; ___)– Where _____ stands for a range of values or a

particular value– E.g., sex (1)– E.g., age (65-89)

• Using more than one filter variable– E.g., sex (1), age (65-89) to select all those who are 1

on sex and age 65 to 89– Joins the two variables with an AND


Subsetting Data Sets

• Select the files you want to construct– Data file (ASCII)– Codebook (ASCII)– Data definitions for SPSS or STATA or SAS

• Select the cases to include (leave blank if you want all the cases)

• Select the variables to include

Choose “Groups of Variables”

Statistics on the Web


Chance News

• Newsletter on risk and uncertainty http://www.darmouth.edu/~chance/chance_news/recent_news/


General

• Internat. Stat. Institute (ISI): http://www.cbs.nl/isi

• Statlib: http://lib.stat.cmu.edu/

• The Probability Web: http://www.maths.uq.oz.au/~pkp/probweb/intro.html

– Links to WWW Virtual Library, etc.

• Hyperstat online: http://www.ruf.rice.edu/~lane/hyperstat/contents.html

• Info on statistical packages: MIDAShttp://midas.ac.uk/stats/– Packages: SPSS, BMDP, SAS, STATA, S-PLUS, GAUSS, MLn,

NAG, TSP, LIMDEP, GLIM, etc. – Includes data sets


Probability distributions

• http://www.marketsys.com/pde.htm– Introduction to distributions (def., etc.)

• http://www.astro.virginia.edu/~eww6n/math/– graph, formula, and math. characteristics (e.g.

moments) for Bernoulli, Binomial, Geometric, Negative Binomial, etc.


Interactive statistical calculation:

http://members.aol.com/johnp71/javastat.html

Collection of programs:

http://www.stat.wisc.edu/statistics/software.html


Textbooks

• Virtual Laboratories in Probability and Statisticshttp://www.math.uah.edu/~stat (univ. Alabama in Huntsville)

– Basic probability and statistics (e.g. definitions, problems, distributions)

– Relation exponential - Gamma - Poisson, and Poisson - Binomial (cf. The Poisson Process)

– Simulation of experiments!– Data sets– Tables – Resources on the Web


Textbooks

• UCLA Statistics Textbook (Jan de Leeuw): http://www.stat.ucla.edu/textbook/

– Includes Statistical Calculators for e.g. distributions (+ plots)

• Links to other electronic textbooks, electronic journals, software, data and code archives: http://www.stat.ruhr-uni-bochum.de/links.html (website maintained by G. Rohwer and U. Poetter)

• List of short courses: http://www.statistics.com/courses.html

new technology support for government and social research russian-u.s. workshop with ilya zaslavsky...

Documents

russianus workshop

distance education toolsprint

distance education toolsdoesnt

tomsk presentations

carrierdistance education

video conferencing

computer conferencing

video disc