2011 acsi survey summary

2011 ACSI Survey Summary HDF/HDF-EOS Workshop

Riverdale, MDApril 18, 2012

Project BackgroundMeasurement timetable

Finalized questionnaire August 1, 2011

Data collection via webSending invitations spanned the first two weeks.

Sending reminders spanned the last two weeks. The survey was in the field for a longer time this year for resending invitations.

September 12, 2011 – October 18, 2011

Topline results October 26, 2011

Results briefing November 29, 2011

Project BackgroundData collectionRespondents

• 3,996 responses were received• 3,996 responses were used for modeling

E-mail addresses from lists associated with some of the data centers were included to reach the large number of users who may have accessed data via anonymous ftp.

Data Center Description

Original CleanedEmailed a

Survey Invitation

Bounce Backs Responded Response Rate

ASDC–LaRC 2350 2350 2349 135 194 9%ASF SAR DAAC 1371 1370 1364 108 172 14%

CDDIS 1302 1275 1271 468 95 12%GES DISC 1551 1544 1533 357 97 8%

GHRC 678 674 670 81 69 12%LP DAAC 25503 25490 25475 1477 1849 8%

MODAPS LAADS 6939 6839 6805 482 484 8%NSIDC DAAC 5487 5487 5468 619 398 8%

OBPG/Ocean Color 4893 4893 4891 721 200 5%ORNL DAAC/FLUXNET 3988 3976 3966 197 229 6%

PO.DAAC-JPL 1361 1352 1348 103 85 7%SEDAC 2728 2728 2724 148 124 5%Total 58151 57978 57864 4896 3996 8%

NASA Survey Responses

Those who answered for

more than one data center:

Two: 103

Three: 14

Four: 2

NASA EOSDIS BenchmarksStrong performance continues …

ACSI (Overall) is updated on a quarterly basis, with specific industries/sectors measured annually.Federal Government (Overall) is updated on an annual basis and data collection is done in Q3.Quarterly scores are based on a calendar timeframe: Q1- Jan through March; Q2 – April through June; Q3 – July through Sept.; Q4 – Oct. through Dec.

75

77

65

76

30 40 50 60 70 80

News & Information Sites(Public Sector) 2011

NASA EOSDIS - Aggregate 2011

Federal Government (Overall) 2010

ACSI (Overall) Q2 2011

2005 2007 20082004 20092006 2010 2011

NASA EOSDISCustomer satisfaction remains steady

Ideal

How close does [DAAC] come to the ideal organization?

Overall satisfaction

How satisfied are you with the data products and services

provided by [DAAC]?

Expectations

To what extent have data products and services provided by [DAAC]

fallen short of or exceeded expectations?

ACSI

76

82

73

78 75

80

73

73

77

81

74

7571

79

73

75

(+/-) 0.9 (+/-) 0.7 (+/-) 0.6 (+/-) 0.5

N=1016 N=1263 N=2291 N=2601

77

81

73

75

(+/-) 0.4

N=3842

72

78

71

74

(+/-) 0.5

N=2857

77

81

74

75

(+/-) 0.4

N=4390

77

81

74

75

(+/-) 0.4

N=3996

NASA EOSDIS ModelProduct Search/Selection/Documentation most critical

The performance of each component on a 0 to 100 scale. Component scores are made up of the weighted average of the corresponding survey questions.

Scores

The change in target variable that results from a five point change in a component score. For example, a 5-point gain in Product Search would yield a 0.9-point improvement in Satisfaction.

Impacts

Customer Satisfaction

Index

Future Use

Recommend

Sample Size: 3996

77

89

87

Customer Support

86

1.7

Product Search

75

0.9

Product Quality78

0.4

Product Documentation

76

0.9

Product Selection and

Order

77

1.1

3.2

3.8

Delivery81

0.4

User background and interestsquestions


Have you searched, ordered,

downloaded data?


downloaded data?

Did you look for or get

documentation?


documentation?

Have you reported

a problem?

Have you reported

a problem?

Have you requested assistance

from customer services?



Search questionsSearch questions

Rate searchRate

search

Order questionsOrder questions

DeliveryquestionsDelivery

questions

Formatquestions

Formatquestions

Usagequestions

Usagequestions

Did you get help 1st time?Did you get

help 1st time?

Customer Service

questions

Customer Service

questions

Documentationquestions


Rate delivery

Rate delivery

Rate customer service


Rate problem resolution


Rate orderRate order

Rate formatRate

format

Rate documentation

Rate documentation

ACSI standard 3 questions


ACSI outcomes 2 questions


• Blue boxes designate general survey areas• White boxes indicate rating questions• Embedded skips are shown with arrows

2011 EOSDIS Survey Overview

Did not search

Thank you!Thank you!

no

Did not order

no

no

no




downloaded data?


downloaded data?


documentation?


documentation?

Have you reported

a problem?

Have you reported

a problem?





Search questionsSearch questions

Rate searchRate

search

Order questionsOrder questions

DeliveryquestionsDelivery

questions

Formatquestions

Formatquestions

Usagequestions

Usagequestions

Did you get help 1st time?Did you get

help 1st time?

Customer Service

questions

Customer Service

questions



Rate delivery

Rate delivery





Rate orderRate order

Rate formatRate

format

Rate documentation

Rate documentation





• Blue boxes designate general survey areas• White boxes indicate rating questions• Embedded skips are shown with arrows

2011 EOSDIS Survey Overview

Did not search

Thank you!Thank you!

no

Did not order

no

no

no

39963996

3673367329542954

NASA EOSDIS 2008 – 2011 Scores hold steady; no change more than one point

=Significant Difference vs. 2010

(+/-) 0.4

(+/-) 0.9

(+/-) 0.5

(+/-) 0.5

(+/-) 0.5

(+/-) 0.5

(+/-) 0.6

77

86

81

78

77

76

75

77

86

80

77

77

76

76

77

85

81

77

76

77

75

77

84

81

74

77

75

75

Customer Satisfaction Index

Customer Support

Delivery

Product Quality

Product Selection and Order

Product Documentation

Product Search

2011 2010 2009 2008

Product QualityOne-point gain from last year

Impact=0.4

78

78

77

77

77

77

74

74

Product Quality

Ease of using the data product in the delivered format

2011 2010 2009 2008

=Significant Difference vs. 2010

Product QualityPreferences somewhat in line with what provided

~Multiple responses allowed

Format data products were provided Format preferred~HDF-EOS/HDF 53% HDF-EOS/HDF 40%NetCDF 13% NetCDF 20%Binary 9% Binary 12%ASCII 17% ASCII 24%GeoTIFF 41% GeoTIFF 53%JPEG, GIF, PNG, TIFF 15% JPEG, GIF, PNG, TIFF 18%OGC Web services 1% OGC Web services 4%GIS 8% GIS 23%KML, KMZ 5% KML, KMZ 13%CEOS 2% CEOS 2%Don´t know 4% OPeNDAP 2%Other format 2% Other preferred format 3%Number of Respondents 3,673 Number of Respondents 3,673

In 2010, 57% said products were provided in HDF-EOS and HDF and 42% said they

were their preferred method.

GeoTIFF is most preferred format, while HDF-EOS/HDF is format in which products were provided the most. Only 8% of products provided in GIS although nearly one-quarter prefer that format.

2011 EOSDIS Survey Flow Overview - CLB

HDF-EOS/HDF FormatTools used when data was provided in HDF format

~Multiple responses allowed

Many of the respondents (687) selected ‘Other’ and listed alternate tool names or described custom approaches. Of these respondents 69 selected 'other‘ exclusively.

Tools used with HDF Number %ENVI 867 44%ArcGIS 818 42%ERDAS 493 25%IDL 509 26%MATLAB 512 26%MODIS Reprojection Tool 506 26%SeaDAS 163 8%Geomatica® 73 4%Global Mapper 123 6%IDRISI 144 22%HDFView 438 22%HEG 109 6%NCL 42 2%GrADS 96 5%Other (Please specify) 303 15%

Number of HDF-EOS/HDF respondents 1961


Experience with HDF Mostly high ratings but some “Ease of Use” problems

0

100

200

300

400

500

600

700

1 2 3 4 5 6 7 8 9 10

Num

ber o

f Res

pond

ents

Ratings (10 = Excellent)

HDF Users Experience Ratings

Ease of Use Quality of Product Usability of Data

Over 60% of the respondents rated all three areas as

8, 9 or 10..

• Survey respondents provided ~ 90 comments about their experience with HDF format, for example pertaining to– Search method

“I found all of the HDF-4 files I needed easily, and in small sizes too which was a plus.”

– Order processing“A mosaicking option for all data sets would be nice”

– Preferences“Please no more HDF4 with irritating custom extensions”

– What they are not finding“I need data in ASCII format . . . data from HDF is complicated”

– Looking for documentation“Format Conversion (HDF to netcdf).”

– Over half were voluntary comments or suggestions “ . . . size and complexity (HDF-format) of the data files . . . can be ameliorated

with web services . . . “• Verbatim comments are available for analysis


HDF User CommentsComments are both positive and negative

SummarySatisfaction with NASA EOSDIS has held at

77 for four years. NASA continues to meet data users needs.

HDF-EOS/HDF is a well supported format• Not all users are comfortable or satisfied with

HDF• Comments received provide insight into users

effective use and/or problems • Verbatim comments are supplied in separate

word documents.

Comments

Verbatim comments are supplied in separate word documents.

In what format(s) were your data products provided to you? (select any that apply)•Other (please specify and/or comment)

Did you use software tool(s) to work with the data (e.g., format conversion, analysis, visualization, etc.?)

•Yes (Please specify which tool or tools you used to work with the data.)•No, I couldn’t find what I needed (please specify what you were looking for)•No, I couldn’t understand how to use it (please specify what you were trying to use)

Do you have any additional comments or suggestion about possible improvements to data products, services, tools, documentation, or the websites that you would like to share? Are you finding what you need on our websites? (please comment)

2011 acsi survey summary

Documents