distributed multi-source development with bill … · distributed multi-source development with...

Distributed Multi-Source Development with

Open Source:

How New Tools, Processes and Free Code are Redefining Software Development

LinuxCon 2010

Bill McQuaideEVP Products & Strategy

Black Duck Software, Inc.

Copyright © 2010 Black Duck Software, Inc. All Rights Reserved. 2

Agenda

Introduction

Market Trends

Development Challenges

Agility and Innovation via Multi-Source Development

Best Practices

Customer Use Cases


Market Dynamics – “Multi-Source”Software development has changed

Collaborative developmentComponentization & Search for re-useAgile methods

OSS is compellingGartner: 85% of enterprises use OSS, 45% use is mission-criticalAccenture: Top benefits are quality, reliability, bug fixing, cost

Distributed, “Multi-Source” development using Agile methods represent new pragmatism

Market Need – “Managing Abundance”< 40% of customers have any OSS PoliciesNeed: address challenges of Multi-Source development:

- Compliance/Management – IP, security, export- Management/Automation – policy, process, multi-source

451 Group Survey on OSS Use (December 2009)

• 87% of companies say OSS meets or exceeds cost savings expectations

• 39% of OSS users ranked Flexibility as the primary benefit


Source: Accenture, August 2010

Open Source is Changing the Way Business Operates its IT


Abundance of Open Source

SugarCRM

MyS

QL

zlibPentaho

BIRT

Xerceslog4j

Asterix

ACEGI

Hibernate

OpenSIP

Alfresco

OpenSSL

SpringOpenNMS

HipHop

Var

nish

Android

SphinxFileZilla

Nagios

Subversion

JBOSS

Tomcat

OpenVista

Mumble

OpenHeX

FreeNA

S

BaculaWordpress

Ganglia

Virtual Dub

Jython

TweetC

raft

OpenEMR

FreeMedPatientOS

Hudson

Ant

Bioclipse

ANTLR

Free

BSD

Webkit

Archiva


Today’s Development Org & Process

Cambridge, MA Los Gatos, CA

Bristol, UK Bangalore, India

Outsourced Code Development

OSS Community

Commercial ISV CodeOffshore

Code Development

Distributed Agile “Multi-Source”


Challenges: Multi-Source Development at-Scale

Management & Control

Find & leverage the right software from many internal, external sources

Get a handle on code base after years of ad hoc ‘ism

Encourage standardization of components & versions

Compliance & Security

Comply with company’s or organization’s policy

Manage licensing and associated obligations

Complying with regulations

Formal control of open source software lags adoption:

~60% of companies surveyed do not have formal polices or guidelines for OSS

Far fewer using tools for automation / management

Source: 451 Group, December 2009


…and Open Source is Playing a Big Role

Please estimate what percent of your code is?

.00

10.00

20.00

30.00

40.00

50.00

60.00

Open source Internally written 3rd party proprietary Other (pleaseexplain)

From recent study of commercial software projects:– Sampled hundreds of commercial projects

Millions of files, representing hundreds of GB of code

– 22% of typical application/project is open sourceAvg project size: ~ 700MB of codeDozens to hundreds of OSS components

From development projects in progress:

OSS is a significant portion of code in DevelopmentSource: Survey of Users from WWW.Koders.com(January-March 8, 2010)


What We’re Hearing from Customers

Concerns about OSS use evaporating– Budget realities trump lingering concerns about OSS security,

quality

CIO’s not “wasting a good crisis”…– Agile: broader OSS use easing evolution to Agile methods

Reduction in design cycle time of 10% to 75%– Goals for Re-Use/Standardization of code of up to 80%

Now recognizing new challenges – managing & controlling use of OSS at-scale in multi-source environments– Need for defined OSS Policy– Making good choices – Search, Selection, Validation– Managing complexity & “abundance”

ad hoc use of hundreds of OSS components has led to governance, tracking, support challenges

– “Provenance” & “software supply chain” visibilityWhat’s in the code being received


Benefits of Multi-Source Development when Best Practices are Followed

Innovate more, code less

Accelerate Time to Market

Open source software to avoid reinventing

the wheel,

Faster delivery of functionality

Increase Innovation & Product Capability

Readily available code to fill out

feature list

Focus resources on features, innovation

Control Development Costs

Re-use to lower development and

licensing costs

Improve development and group

productivity


Pro-Active, Managed Use of Open Source

Cost of defects– Minimal when issues are

detected early in lifecycle– Grows 100-1,000X late in

the lifecycle or after release

For OSS, invest in process and automation:– Make better OSS code

choices up front (via search/selection)

– Validate the in-bound code before pushing it into the development process

– Validate all code prior to release

Follow QA best practicesCapers Jones, Applied software measurement: assuring productivity and quality, 1999.


Multi-Source Development Best Practices

Manage as a cross-functional business process

Published OSS use policy

OSS review board, and process owner

Supply chain techniques

Code reduction/re-use initiatives

Defined approval processes & workflow

Agile methodology

Automated validation at acquisition and in development

Automated monitoring & tracking

Obligation verification

Policy Process Technology


Case Study: “Design in” Compliance

Strategic acquisition of OSS

Developer

Catalog

KnowledgeBase

Development

Security

IT

Legal

Management

Quality

Approval Board

SourceForgeRubyForgeEclipse.orgApache.org

etc…

Open Source

Approval FlowAlertsAlerts

Subversion Build Test/GASoftware Bill

ofMaterials:

Validation Engines

Component, Licensing

•Open source•Home grown•Commercial•Outsourced

Continuous Multi-Source Development


SPDXTM - Standard for Exchanging License Information

What:A data exchange standard to share OSS license and component information (metadata) for software packages and related content with the aim of facilitating license and other policy compliance

Why:Enable easy exchange of license information between companies reducing burden on both suppliers and consumersAvoids due diligence redundancy where the same source code package is analyzed multiple times by different receivers

Who:Participation from over 16 organizations including software, systems and tool vendors

Sponsors:Linux Foundation & FOSSBazaar (governance best practices group under Linux Foundation)

Software Package Data Exchange™ (SPDX™)


OSS 3rd Party

LegacyCode

OSS

Case Study: Multi-Source Management in a Software Supply Chain

OSS 3rd Party

LegacyCode

Typical Smartphonehas over 300 components

SoftwareAsset

Corporate-Owned IP

Proprietary/Licensed IP

XML

Security

Networking

Email

Graphics

Database

Web Services

Customer

Development/Integration

Out Source/Offshore

Your Company

Development/Integration

InternalCode

OSSOSS OSS


SAP – Complex Software Supply Chain Automated via Black Duck Multi-Source Management

Open Source ProjectOpen Source Project

SAP CustomerSAP Customer

SAP product

Open source

component

Embedded third-party softwareContribute

Integrate

SAP product

Embedded third-party software

Redistribute

Complementary third-party software

Custom development

SAP PartnerSAP Partner

Partner component Distribute

Integrate

2001-06: use of OSS in SAP products was seen as a risk and managed as an exception

2007: approvals delegated to product units; OSS

contributions were limitedto Eclipse Foundation

2008-09: shift to OSS as productivity enabler – OSS

contribution approvals now delegated to the product units

New OSS usage

New Contributions


SummarySoftware development is changing rapidly– New processes, new tools, new standards, multi-source

Benefits of distributed agile development are significant– Reduced operating costs ($M’s)– Reduced cycle times (up to 75%)– Increased innovation and leverage

Benefits of multi-source development are significant– Reduce operating costs– Speed innovation, supports Agile methods, increases flexibility

New processes and tools enable Dev teams to realize the full potential

17

Thank You!

distributed multi-source development with bill … · distributed multi-source development with...

Documents