Challenges, Prize Programs, and the Opportunity for Government

Challenges and prize competitions are a severely underused tool that should be systematically instituted by governments in order to rectify the considerable problems and market failures that exist within the innovation system. This paper presents common errors and examples of government and public sector Challenges, presents two case studies of successfully executed Challenges, and then makes a strong argument for outsourcing Challenges to experts.

W H I T E P A P E R

2

About InnoCentive

InnoCentive is the global pioneer in crowdsourced innovation. We help innovation-driven organizations solve their critical business, scientific and technical problems by crowdsourcing ideas and solutions, either from our global network of highly educated problem solvers or from their own internal networks. By accessing vast virtual workforces with InnoCentive, organizations have been able to innovate faster, with less risk, and at a lower cost. We offer our proven Challenge Driven InnovationTM methodology, unrivalled network of over 380,000 problem solvers and purpose-built technology, as well as accompanying training and program management services. To date, InnoCentive have conducted over 2,000 external Challenges for organizations including NASA, DARPA, Thomson Reuters, AstraZeneca, GSK, Anheuser-Busch InBev, and Ford Motors.

3

Introduction Challenges and prize competitions are a severely underused tool that should be systematically instituted by governments in order to rectify the considerable problems and market failures that exist within the innovation system. After enjoying a series of dramatic innovation success stories during the industrial era (including the steam engine, automobile, and aviation technology breakthroughs), the deployment of prize competitions languished markedly in the twentieth century. The impressive resurgence of this useful mechanism — seen with the X-Prize, DARPA Challenges and others — is due to new opportunities afforded by modern communications technology and the pressing need to compensate for the increasing failure of traditional innovation strategies. This development should be especially welcome at a time when much of the world faces such challenging economics. However, it should be noted that the increasing use of prize competitions carries a number of potential risks of which policy makers must be aware.

One of the greatest risks is that Challenges and competitions may be poorly designed. This could lead to taxpayer money being wasted with negative market consequences, rather than the desired positive results that aim to correct traditional cost structures and failures. Even worse, badly designed or executed Challenges could discredit the open innovation concept itself, resulting in many exciting opportunities for innovation being lost. Poor institutional Challenge design is not uncommon and the history of Challenges is littered with examples of badly-devised incentives that failed to meet their original targets. The primary mistake that is made is that people underestimate the complexity of designing an effective Challenge and then do not realize that their design is fatally flawed until it is too late. Balancing achievability and audacity of innovation targets, developing robust success metrics, and communicating competition rules with explicit clarity are a few examples of important design features that are all too often misjudged. This paper presents common errors and examples of

C H A L L E N G E D E S I G N

Examples of important Challenge design features include balancing achievability and audacity of innovation targets, developing robust success metrics, and communicating competition rules with explicit clarity.

4

government and public sector Challenges that have encountered these problems, presents two case studies of successfully executed Challenges, and then makes a strong argument for outsourcing Challenges to the experts.

Competition Design and Implementation Errors

Whether public or private, profit or non-profit, any organization tasked with designing and implementing an innovation prize competition runs the risk of making mistakes that could negatively affect Challenge success outcomes. One key element of design involves identifying an appropriate innovation target to focus the Challenge around. If errors are made when setting a suitable end goal, then the effectiveness of a Challenge can be fundamentally undermined from the outset. Great emphasis needs to be devoted to setting the correct innovation target for a specific Challenge and this may involve investing a considerable amount of research effort. In cases where this proves especially difficult, expert prize designers would be aware of several alternative options available that should work to minimize the risk of the competition failure. For example, a system of variable victory conditions could be implemented, so that the size of the prize purse would automatically adjust itself according to the level of innovation attained. Another option might involve breaking the innovation solution down into more manageable constituent parts and then structuring the overall Challenge so that these problems are tackled systematically (and simultaneously by multiple problem solvers).

One of the most significant advantages of utilizing Challenges is that they can facilitate outreach to innovators beyond the traditional field or sector of the problem. This creation of diversity and the ability to orchestrate networks of problem solvers to solve Challenges is a hallmark of InnoCentive. These unexpected innovators have historically enjoyed a high level of success in prize competitions, with some research suggesting as much as one third of prizes are won by such entrants.1 However, participation from these fresh, new problem solvers is certainly not guaranteed and Challenges need to be deliberately designed so as to enable and encourage these innovators to compete.

The H-Prize — a hydrogen-energy related competition financed by the US Department of Energy — is a good example of a Challenge that suffered from a debilitating narrowness of scope: only two entries were submitted, and neither of these was deemed to meet the competition’s scientific testing parameters. Evaluating the project, the Department of Energy lamented about an overly limited scope that made the H-Prize

N E T W O R K

Cultivating a broad problem solver network is highly beneficial, as roughly one third of prizes are won by innovators from outside the traditional field of the problem.

5

a niche competition, with few innovators able to approach the problem as they had formulated it.2 This design difficulty could have been resolved by carefully abstracting the innovation problem to be tackled and reformulating specialist questions in a way that invites solution suggestions from nonspecialists.

Another way of achieving this important goal could have been to structure the competition so that it included a number of relevant sub-competitions and prizes specifically crafted to encourage entry from innovators working in related areas.

Ensuring that a Challenge is sufficiently flexible and adaptable is another important design feature all too often overlooked. The Scottish Government’s Saltire Prize (striving toward innovations in marine-energy generation) is only in its early stages, but it is already being criticized for failing to achieve a good balance of flexibility and firmness in the overall design and implementation.3 The Saltire Prize Challenge Committee are now considering options of reshaping the prize. H-Prize suffered from the problem of inflexibility, as administrators did not respond quickly enough to adapt their faltering Challenge before it failed.4

The Importance and Value of Design

There is a careful balance to be struck between explicit concreteness of Challenge rules and flexibility that enables continual adaptation based on feedback and experience. Challenges that have a superior structure:

• Have a design process that explicitly seeks feedback from entrants in the early stages of a Challenge

• Adapt guidelines as appropriate• Evolve to produce better results in the end

There are numerous other pitfalls to be aware of and a lot of ways a Challenge can go wrong due to subtle mistakes in design. The devil is truly in the details and getting these details wrong can ultimately lead to a Challenge failing and the invested resources being wasted. This contrasts dramatically with the significant positive results observed in well-designed innovation Challenges, which generally inspire a diverse range of benefits (as detailed in numerous independent reports and studies)5. The considerable difference between the ROI (return-on-investment) gained from successful competitions and the losses incurred by unsuccessful ones suggest that prize design expertise is a skill worth paying for. The services of Challenge design specialists, who understand the subtle complexities involved in innovation prize development, therefore represents a good investment for any organization hoping to run a Challenge.

I N N O C E N T I V E W O R K S H O P S

C H A L L E N G E D E S I G N

Ensuring that a Challenge is sufficiently flexible and adaptable is another important design feature all too often overlooked.

6

I N N O C E N T I V E W O R K S H O P S

Case Study: Energy Storage Challenge

Business Challenge

InnoCentive worked closely with the US Office of Naval Research Global to find the best ideas from around the world for a new system for storing and transporting energy.

Project Details

A set of rigorous criteria and inducements were developed that would ensure the Challenge produced truly fresh approaches to storing and transporting electricity, not just variations on existing ideas. Proposed systems were restricted in size, with a range from mobile phone batteries to systems that could fit in multiple sixty foot containers, and they needed to be demonstrably different from existing systems.

InnoCentive was effective in creating a global energy storage community by reaching out to thousands of problem solvers in over 25 countries around the world. In addition to deploying an arsenal of social media and publishing tools to reach solvers, InnoCentive ran localized Challenges in India, China, and Israel to find and compare energy storage innovations happening there with what was being produced in the US and Europe.

Over 120 proposals were submitted to the Challenge, from which 7 finalists were selected to attend PitchLive and present their ideas before sponsors, investors, and a panel of independent judges. The winning entrant received a prize fund worth $250,000.

Results

UK-based Cella Energy was awarded the final prize fund for its proposal to develop lowcost hydrogen storage materials. Moreover, universities, private consultants, SMEs, and large corporations from North America, Asia, and the South Pacific all participated in the Challenge. The pool of entrants was richly diverse, including 20 ideas submitted from the Chinese Academy of Science, and 15 from groups in Israel. This provided the valuable insight that the work that was happening in China and Israel was in a totally different direction to what solvers were doing in Europe and North America.

7

Impact

The Challenge allowed the main sponsor (US Office of Naval Research Global), industry stakeholders, and researchers to see beyond their own networks and gain a global perspective on where the transportable energy storage industry is now, and more importantly, what it might look like in 10 or 20 years. There was huge value from the entrant’s perspective, which was exemplified as Challenge winner Cella Energy received a new round of financing led by a $1 million investment by Space Florida.

8

Challenge Expertise as the Solution

Having established the value of prize design expertise, the next question for expanding the utilization of the Challenge mechanism is: where should this expertise come from? The arguments supporting the further deployment of this interesting tool might suggest the systematic institution of Challenges in all government programs, perhaps implying that governments would do well to internalize Challenge expertise within their various agencies.6 This may seem preferable upon first glance, as it could give the respective agencies more direct control over the Challenges that they run.

This system may also enable better cooperation and coordination with other government agencies and departments concerning the prize project and in some ways might enable a more holistic innovation system approach to be implemented.7

However, there are stronger arguments supporting the employment of an outsourcing model for publicly-funded Challenges: outsourcing of design, network orchestration, and competition execution. The increasing outsourcing of specialist functions is a trend that can be observed in both the public and private sectors and the basic principle is well-aligned with mainstream economic theories concerning the division of labor within complex economic systems. In this model, governments and their agencies would commission independent organizations to design and implement their various Challenge programs.

One advantage of the outsourcing model concerns the agility advantages that independent companies and organizations enjoy relative to large public sector institutions. Due to the considerable variation in what constitutes appropriate prize design in differing contexts, there is often a need to be highly flexible with design options and highly adaptable during Challenge implementation. This agility is not generally associated with large public sector agencies, whose regulated procedures and bureaucratic organizational cultures make them generally unsuited to the development of individualized, tailor-made, and original innovation competitions. Furthermore, large agencies may find it difficult to achieve the speed and manoeuvrability that enables continuous evolution and development of competition design.

Another advantage of outsourcing is that independent organizations may generally be perceived as more neutral than a government institution. This can be especially significant when it comes to judging, selecting winners, and dealing with legal controversies and intellectual

O U T S O U R C I N G

Outsourcing publicly-funded Challenges can provide greater agility, neutrality, and global outreach.

9

property disputes. Public institutions would likely appreciate being one level removed from such issues and disagreements. InnoCentive’s Challenge management methodology has been carefully developed to largely avoid these problems, as intellectual property considerations are built into the prize program structure where appropriate.

It is also true that Challenges run by national governments will tend to find it more difficult to engage a truly global innovation community. For many problems, global scale crowdsourcing is the proven way to access the best new ideas and innovations and more localized competitions should be expected to have proportionally less success on this level. National governments find it difficult to justify giving large awards to foreign individuals or companies, and so are less likely to be bold in the development of their Challenges.

There are a small and select number of private sector Challenge specialists that have built up an impressive array of resources to improve their ability to deliver high quality Challenges. The InnoCentive Global Solver Network is a good example of such a resource, providing a dedicated community of more than 380,000+ problem solvers with reach to millions more around the world through the company’s partnerships. Governments would find it difficult to replicate such a specialist network independently but undoubtedly benefit greatly from utilizing such a marketplace in the Challenges that they fund.

Another advantage of Challenge outsourcing is that contractors have better incentives for success: success is rewarded with additional work and failure is not. This is important, as it has proven difficult to construct an efficient success-incentive system within large public sector organizations. It is logical that these private marketplace dynamics tend to make independent contractors more determined and focussed on successful outcomes than their public-sector counterparts.

S O L V E R N E T W O R K

Private sector specialists have built up impressive solver networks - InnoCentive has a dedicated community of 380,000+ Solvers, plus reach to millions more through its partners.

10

I N N O C E N T I V E W O R K S H O P S

Case Study: City of Boston “Street Bump” Challenge

Business Challenge

Help the City of Boston eliminate potholes by building a computer program that uses data collected on smartphones to predict the location of potholes throughout the city.

Project Details

In 2011, the City of Boston posted a Challenge with InnoCentive to predict where potholes and other dangerous street features are located, using data collected by the “Street Bump” application running on volunteers’ Android smartphones.

When placed in a moving car, the Street Bump app, developed by the City of Boston’s ofice of New Urban Mechanics, uses a phone’s GPS and accelerometer to detect bumps in the road. The application sends a signal to a database when the car hits a bump. Unfortunately, when the app was first created, all bumps were treated equally — Street Bump could not differentiate between a pothole and a manhole or other irregularity. So, with funding from Liberty Mutual, the City of Boston launched an InnoCentive Challenge to convert the data collected into actionable information. Using acceleration and position information collected by the Street Bump app running on volunteers’ Android smartphones, the Challenge sought an algorithm that could accurately identify the location and severity of potholes, while weeding out other road anomalies like bridge surfaces and manhole covers. More than 700 individuals participated in the Challenge and 19 solutions were received. In early 2012, the City selected three winners who split the total prize of $20,000.

Results

All three of the awarded solutions were able to increase the accuracy of predicting potholes in the city, with each taking a very different approach. The winning solutions included a submission by Edward Aboufadel, a professor of mathematics at Grand Valley State University and members of Sprout & Co, a non-profit organization based in Somerville, Mass. The final algorithm, developed using elements of all three winning solutions, will be released publicly for use by other cities.

11

Impact

The City of Boston plans to use the data collected to more efficiently utilize work crews, respond more quickly to problems, and generally improve delivery of services to citizens. They expect that the app will have expanded uses as well. For example, disability groups would like the ability to map all curb cuts and the quality of those curb cuts for people in wheelchairs. Phones will be able to report this data automatically, allowing people to advocate on their own behalf. Consistent with the original intent of the Challenge, the City plans to opensource the code and make it available publicly, with the goal of encouraging other cities to adopt the technology as well.

12

Calls to Action

It is clear that the use of Challenges, Challenge design, and network orchestration expertise needs to be expanded upon and invested in if humanity is going to make best use of this valuable and re-emerging capability. Governments need to think strategically about the innovation system, how it can be improved, and how Challenges should integrate into the overall innovation framework supported by government. They need professional guidance to achieve these goals and it would seem wise for governments to add to their innovation systems through proven Challenge market leaders. Building this expert capability internally would be asking too much for governments to attempt when there are Challenge management companies that have developed private sector capabilities on a global scale. This would be a highly complex task and counter to the prevailing currents of thought:

• Increased public-private partnerships• Outsourcing of publicly-funded services in appropriate areas• Increasing the need for speed and economies of procuring solutions

to the world’s most important Challenges

The fact that Challenge design and implementation is not instinctively outsourced to specialists is largely because of a failure to recognize the true difficulty and complexity of the task. Government institutions need innovation systems experts who are adept at identifying the specific areas where Challenges would most likely be useful. Challenges are a highly adaptable policy tool, but they are not appropriate in all circumstances and there are many areas where it would be more sensible to employ alternative mechanisms to inspire innovation. Government innovation personnel must examine the various tools they have available and decide whether to deploy:

• Inducement/Challenge prizes• Contract grants• Entrepreneurial incubation support• Focused micro-finance• Regulatory measures• Research tax-credits• Or some combination of these to improve outcomes

With global economic and social pressures on government spending, this highly economical and scalable capability must be embraced and embedded into the government toolset. Governments can then avoid the growing global public outcry on excess spending with no measurable benefit to its people and tax payers.

P U B L I C S E C T O R

Public institutions should identify the areas that are most appropriate for Challenge deployment and then outsource the design and execution to independent companies who specialize in these areas.

13

Conclusion

Governments and international public institutions should be actively seeking to improve their own innovation systems expertise, but they should also support the development of a strong, vibrant, and bold innovation management industry in the private sector. Ultimately, it is this vibrant marketplace of independent organizations, with their global diversity of perspectives and ideas, that will best drive forward the evolution and progress of innovation using Challenges. The world stands to benefit greatly from a cultural paradigm shift away from closed innovation and traditional problem solving strategies toward open innovation, crowdsourcing, prize competitions, and Challenge-centric thinking. Governments must attempt to embrace these improved innovation cultures within their various internal agencies and this means reaching out to the private companies who are leading the advance.

Endnotes

1. http://www.prizecapital.net/Prize_Capital/Home/Home.html

2. https://www.hydrogen.energy.gov/pdfs/progress11/iv_e_8_serfass_2011.pdf

3. http://www.holyrood.com/component/content/article/62-business-industry-

economy/673-

4. http://www.hydrogen.energy.gov/pdfs/progress11/iv_e_8_serfass_2011.pdf

5. Davis, L.N. (2004) How Effective are Prizes as Incentives to Innovation? Evidence from

Three 20th Century Contests. Paper presented to the DRUID Summer Conference, Elsinore/

Copenhagen, 14-16 June.

6. Hynek, P. (2008) The Hyper-Commons: How Open Science Prizes Can Expand and Level

the

Medical Research Playing Field. Rejuvenation Research, Volume 11, Issue 6, 1065-1073.

7. Stine, D.D. (2009) Federally Funded Inducement Prizes

14

For more information on how you can run your own Challenges to rapidly solve problems and accelerate your innovation outcomes:

C A L L U S

Speak directly to a consultant.

North America: 1-855-276-9366

F I L L I N O U R C O N T A C T F O R M

Submit our contact form online here. Our team will respond shortly.

V I S I T O U R W E B S I T E

Explore our full library of white papers and view live Challenges at:http://www.innocentive.com

Non-US: +44 (0) 207 224 0110