what does open source mean for hdf? mike folk the hdf group july 2012 esip summer meeting 20121

ESIP Summer Meeting 2012 1

What does Open Source Mean for HDF?

Mike FolkThe HDF Group

July 2012

ESIP Summer Meeting 2012 2

About HDF

What is HDF?• A data model

– Structures for data organization and specification

• Open file format– Designed for high volume or complex data

• Open source software– Works with data in the format

Today we just focus on softwareESIP Summer Meeting 2012 3

ESIP Summer Meeting 2012 4

HDF4 or HDF5?

Both

ESIP Summer Meeting 2012 5

The HDF Group

• At U of Illinois NCSA 1988-2006

• Non-profit company since 2006

• About 35 staff and $3.5M in revenues

ESIP Summer Meeting 2012 6

Mission of The HDF Group

• To provide the highest quality software for managing large complex data sets

• To provide outstanding services for users of HDF technologies

• To insure long-term access and usability of data that is stored using HDF technologies

ESIP Summer Meeting 2012 7

HDF Communities

• Academia, government, commercial• All applications involving complex or big data• Users range from highly proficient software

developers to naïve end users.

ESIP Summer Meeting 2012 8

What we do

• Support some large diverse projects, such as EOS and JPSS– Whatever they need

• Work for hire– Training, consulting– Development of HDF core software

• General maintenance, QA and support

ESIP Summer Meeting 2012 9

Distribution of revenues by sector

Commercial34%

NASA & NOAA39%

HPC17%

Other10%

2012 (est)

ESIP Summer Meeting 2012 10

What does OSS mean for HDF

ESIP Summer Meeting 2012 11

History of HDF as OSS

• Why HDF became FOSS– Default

• Why HDF stayed FOSS despite objections– Not very monetizable– Universal access to data

• Why HDF should remain FOSS– HDF preservation mission– All of the above

ESIP Summer Meeting 2012 12

Community-Maintained:

SingleMaintainer:

GOTS COTSProprietaryOSS

OpenGOTS

ClosedGOTS

Single Maintainer

OSS

CommunityMaintained

OSS

GatedSW

Typical proprietary

SW

Open Technology Development*

* Based on slide 41 from “Open Source Software (OSS or FLOSS), the U.S. Department of Defense (DoD), and NASA,” David A Wheeler, NASA Open Source Summit, March 2011.

ESIP Summer Meeting 2012 13

Intellectual property

• U of I original owner• Transferred to HDF Group for royalty on

commercial profits• BSD license

ESIP Summer Meeting 2012 14

Benefits of OSS, as it relates to HDF.

ESIP Summer Meeting 2012 15

TRY BEFORE ADOPT

ESIP Summer Meeting 2012 16

IF IT ALMOST WORKS, YOU CAN MODIFY IT TO MAKE IT WORK

ESIP Summer Meeting 2012 17

DEVELOPMENT ACTIVITIES ARE PUBLIC

ESIP Summer Meeting 2012 18

FREEDOM TO DEVELOP TOOLS THAT MAKE HDF MORE USABLE

ESIP Summer Meeting 2012 19

LONG TERM ACCESS

ESIP Summer Meeting 2012 20

Aspects of OSS we’re less sure about, as they relate to HDF

ESIP Summer Meeting 2012 21

UNPAID CONTRIBUTORS CAN DO MUCH CORE WORK

ESIP Summer Meeting 2012 22

"GIVEN ENOUGH EYEBALLS, ALL BUGS ARE SHALLOW".

ESIP Summer Meeting 2012 23

FREQUENT DEVELOPMENT CYCLES ARE GOOD

ESIP Summer Meeting 2012 24

OSS IS EASY TO USE

ESIP Summer Meeting 2012 25

OSS IS LOW COST

ESIP Summer Meeting 2012 26

IT IS EASY TO RUN A COMMUNITY-BASED OSS PROJECT

ESIP Summer Meeting 2012 27

OSS BUSINESS MODELS

ESIP Summer Meeting 2012 28

What next?

ESIP Summer Meeting 2012 29

INCREASE AND DIVERSIFY REVENUE TO OFFER A BETTER PRODUCT

ESIP Summer Meeting 2012 30

BETTER USE OF COMMUNITY!

ESIP Summer Meeting 2012 31

Thank you

mfolk@hdfgroup.org