factors affecting participation of solvers in crowdsourcing: an empirical study from china

10
SPECIAL THEME Factors affecting participation of solvers in crowdsourcing: an empirical study from China Bingjia Shao & Lei Shi & Bo Xu & Li Liu Received: 14 February 2011 / Accepted: 16 April 2012 / Published online: 18 May 2012 # Institute of Information Management, University of St. Gallen 2012 Abstract Crowdsourcing comprises a variety of creative contests, and its success is closely related to the quantity and quality of solvers. The research model of factors influ- encing the quantity and quality of solvers with respect to contest arrangement attributes and market competition situ- ation has been developed in this paper, and the model has been tested with data from a crowdsourcing website in China. The results show that higher awards, easier tasks, longer duration and lower competition intensity lead to a higher number of solvers. Higher awards, longer duration and higher difficulty level of tasks lead to higher ability level of winners, but competition intensity and market price for other competing projects do not show significant corre- lation with the ability level of winners. Keywords Crowdsourcing . Innovation contest . Solvers . Competition situation . Contest attributes JEL classification M1: Business Administration Introduction With the rapid growth of the Internet and the rise of Web2.0 applications, more and more enterprises have begun to use external Internet resources to enhance their competitiveness. Business innovation has begun to shift from previously closed innovation to open innovation. Crowdsourcing is now an important way for enterprises to implement open innovation in the Internet era, which has also formed a new e-commerce business model. Crowdsourcing refers to a company or organization out- sourcing work that used to be performed by employees to the non-specific social network on the Internet (Howe 2006). InnoCentive, founded in the United States in 2001, was the first online marketplace to host open innovation projects in the form of contests (Allion 2003). World famous multinational companies such as Boeing, DuPont, and Procter & Gamble have thrown their most troublesome problems to InnoCentivewith the result of the problem solving rate increasing to 30 % in InnoCentive. Following the success of InnoCentive, a variety of innovation transac- tion websites have been established. Projects ranging from web application development to brand and construction design are posted to these innovation websites. The solution seekers could be individuals, firms, or government organ- izations, who use these crowdsourcing platforms for open innovation projects. In China, crowdsourcing platforms such as Zhubajie (zhubajie.com), TaskCN (taskcn.com) and K68 (k68.com) have attracted a lot of innovative talent and solution seekers, which greatly enhance business operations. Howe ( 2006) believed that crowdsourcing was a commercial revolution. The key prerequisite for its im- plementation is network platform construction and net- work connectivity of potential participants. According to Responsible editor: Xin Luo B. Shao : L. Shi School of Economics and Business Administration, Chongqing University, Chongqing, China B. Shao e-mail: [email protected] B. Xu (*) Fudan University, Shanghai 200433, China e-mail: [email protected] L. Liu The Fundamental Education College, Sichuan Normal University, Chengdu, Sichuan Province, China Electron Markets (2012) 22:7382 DOI 10.1007/s12525-012-0093-3

Upload: li-liu

Post on 25-Aug-2016

215 views

Category:

Documents


1 download

TRANSCRIPT

Page 1: Factors affecting participation of solvers in crowdsourcing: an empirical study from China

SPECIAL THEME

Factors affecting participation of solvers in crowdsourcing:an empirical study from China

Bingjia Shao & Lei Shi & Bo Xu & Li Liu

Received: 14 February 2011 /Accepted: 16 April 2012 /Published online: 18 May 2012# Institute of Information Management, University of St. Gallen 2012

Abstract Crowdsourcing comprises a variety of creativecontests, and its success is closely related to the quantityand quality of solvers. The research model of factors influ-encing the quantity and quality of solvers with respect tocontest arrangement attributes and market competition situ-ation has been developed in this paper, and the model hasbeen tested with data from a crowdsourcing website inChina. The results show that higher awards, easier tasks,longer duration and lower competition intensity lead to ahigher number of solvers. Higher awards, longer durationand higher difficulty level of tasks lead to higher abilitylevel of winners, but competition intensity and market pricefor other competing projects do not show significant corre-lation with the ability level of winners.

Keywords Crowdsourcing . Innovation contest . Solvers .

Competition situation . Contest attributes

JEL classification M1: Business Administration

Introduction

With the rapid growth of the Internet and the rise of Web2.0applications, more and more enterprises have begun to useexternal Internet resources to enhance their competitiveness.Business innovation has begun to shift from previouslyclosed innovation to open innovation. Crowdsourcing isnow an important way for enterprises to implement openinnovation in the Internet era, which has also formed a newe-commerce business model.

Crowdsourcing refers to a company or organization out-sourcing work that used to be performed by employees tothe non-specific social network on the Internet (Howe2006). InnoCentive, founded in the United States in 2001,was the first online marketplace to host open innovationprojects in the form of contests (Allion 2003). World famousmultinational companies such as Boeing, DuPont, andProcter & Gamble have thrown their most troublesomeproblems to “InnoCentive” with the result of the problemsolving rate increasing to 30 % in “InnoCentive”. Followingthe success of InnoCentive, a variety of innovation transac-tion websites have been established. Projects ranging fromweb application development to brand and constructiondesign are posted to these innovation websites. The solutionseekers could be individuals, firms, or government organ-izations, who use these crowdsourcing platforms foropen innovation projects. In China, crowdsourcing platformssuch as Zhubajie (zhubajie.com), TaskCN (taskcn.com)and K68 (k68.com) have attracted a lot of innovativetalent and solution seekers, which greatly enhance businessoperations.

Howe (2006) believed that crowdsourcing was acommercial revolution. The key prerequisite for its im-plementation is network platform construction and net-work connectivity of potential participants. According to

Responsible editor: Xin Luo

B. Shao : L. ShiSchool of Economics and Business Administration,Chongqing University,Chongqing, China

B. Shaoe-mail: [email protected]

B. Xu (*)Fudan University,Shanghai 200433, Chinae-mail: [email protected]

L. LiuThe Fundamental Education College, Sichuan Normal University,Chengdu, Sichuan Province, China

Electron Markets (2012) 22:73–82DOI 10.1007/s12525-012-0093-3

Page 2: Factors affecting participation of solvers in crowdsourcing: an empirical study from China

the 2010 China Witkey Industrial White Paper (iResearch2010), by the end of October 2010, there had been more than100 Internet platforms providing crowdsourcing services inChina, with more than 20 million registered membersand more than 300 million cumulative transactions. Thedevelopment of crowdsourcing effectively expands thescope of e-commerce and forms an “invisible goods” e-commerce model. Businesses may benefit from crowd-sourcing in terms of multiple solutions, reduced costs, shorterR&D time, and higher quality solutions. However, extantstudies have found that satisfaction of seekers with crowd-sourcing platforms is relatively low, mainly due to the smallnumber and low quality of solutions (iResearch 2010).Therefore, it is necessary to determine factors influencingthe quantity and quality of solvers involved in the projects,so as to attract more high-level solvers to participate incrowdsourcing.

Drawing on product development theory and innovationcontest theory, this study proposes a research model in aneffort to investigate the salient factors contributing to num-ber of solvers and level of winners in crowdsourcing proj-ects in the context of Chinese crowdsourcing websites. Theobjective of this study is thus to examine the factors influ-encing solvers’ behaviour on such websites. The crowd-sourcing projects are characterized with attributes likeaward, duration and difficulty level. Since many projectsmay exist within same time period on a crowdsourcingwebsite, there is competition among the projects for solvers.This study is to investigate the influence of project attributesand competition situation on numbers of solvers attractedand ability level of final winners in crowdsourcing projects.We use software development projects as targets for ourresearch, to seek answers to the following research questions:(1) How do the project attributes affect the number of solvers?(2) How the winner’s ability level relates to the project attrib-utes? (3) How does the competition situation affect the num-ber of solvers in a project? (4) How the winner’s ability levelrelates to the competition situation? The results should ad-vance our understanding of solvers’ behavior on crowdsourc-ing platforms, and offer practical suggestions for seekers toincrease their benefits from such platforms.

The rest of the paper is organized as follows: “Comparisonsbetween Chinese and foreign crowdsourcing” comprises com-parisons between Chinese and foreign crowdsourcing web-sites. “Literature review” is a literature review, followed by theresearch model and hypotheses development in “Researchmodel and hypotheses”. Research methodology and data anal-yses are discussed in “Research methodology and dataanalyses”. Research findings are presented in “Resultsand analysis”. In the last section implications and lim-itations of the study are discussed, and the study isconcluded with a summary of contributions and sugges-tions for future research.

Comparisons between Chinese and foreigncrowdsourcing

Crowdsourcing websites provide platforms for buyers andsellers to post and share demand and supply information andto support crowdsourcing transactions. They make profit bycharging commissions to seekers. There are some differ-ences between Chinese and other crowdsourcing platforms.

First, the transaction modes are different. Most crowd-sourcing websites adopt a tender trading model, where bid-ders submit proposals to the buyer, and the buyer selects awinner from the bidders, and then the winner completes thetask independently. In contrast, Chinese crowdsourcingwebsites typically adopt a reward trading model, in whichsolvers complete the project in accordance with the seeker’srequirements, and then submit completed solutions to theseeker who may select winning solutions (one or more)from the submissions and pay an award to the winner(s).In the context of the tender trading model in countries otherthan China, bidders’ information is kept confidential toprevent malicious competition. But under the context ofthe reward trading model in China, solvers’ information ispublic, thus solvers may take into account more factors(e.g., other solvers’ ability) when considering whether toparticipate in a project.

Second, the range of participants is different. Crowdsourcingwebsites in countries other than China, e.g. InnoCentive, mayoperate more globally, with buyers and sellers from all overthe world. In Chinese websites, most participants are fromwithin China due to the language barriers.

The third difference lies in project content. Non-Chinesewebsites contain a broader range of tasks than Chinesewebsites, and classification is more detailed. The Chinesewebsites usually contain projects that are more related todaily life. We use the Non-Chinese website of eLance andthe Chinese website of zhubajie as examples. eLance hasfour categories of projects, including management support,engineering and manufacturing, financial management, andlaw, which are not included in zhubajie which containsprojects for birthday messages to friends and projects fornovel gift ideas.

The differences in Chinese and other crowdsourcing plat-forms may lead to solvers’ different behaviors. Thus, it isnecessary to build a theory to specifically explain the num-ber of solvers and ability of winners for a crowdsourcingproject in the Chinese context.

At present, the crowdsourcing platforms in China mainlyconsist of eight functional modules: task posting, audit andmanagement, buyer payment, bidding, solution selection,pricing and time extension, payment to solver, and moneywithdrawal. Among the functional modules, the task postingmodule is at the beginning of the whole crowdsourcingtransaction process. When posting a task on the website,

74 B. Shao et al.

Page 3: Factors affecting participation of solvers in crowdsourcing: an empirical study from China

seekers need to give the task title, task content, specificrequirements, amount of award, and completion deadline.Then solvers need to decide whether or not to participate inthe task. If solvers decide to participate, they need to clickthe “I want to apply” button in the task page. People whoagree to participate in the project are called solvers (in Chinasolvers are termed ‘witkey’). However, not all solvers willfinally submit work due to various reasons. For example,some solvers may not complete the task before the deadline.Since only the final solutions submitted by the solvers areconsidered by the seeker, in this study we focus on analyzingparticipation behaviors of solvers who have actually submit-ted their work on the crowdsourcing website. As one of thefirst attempts at studying crowdsourcing projects in theChinese market, this paper aims to investigate how to improvethe quantity and quality of solvers in crowdsourcing projects.

Literature review

Solving problems with the crowdsourcing model can beseen as a new form of creative product development. Incrowdsourcing, companies outsource product developmentto external talents on the crowdsourcing platform. Similar tothe traditional product development process, the outcome ofcrowdsourcing projects depends on solvers’ quantity, abilityand initiatives. Other factors that influence the traditionalproduct development process, such as product attributes andmarket competition, may also affect crowdsourcing projects.Therefore, product development theory is a theoretical foun-dation of this study. Crowdsourcing platforms adopt contestmechanisms, and there exists competition among solvers,which is similar to innovation contests. Thus, the innovationcontest theory is another theoretical basis of this study.Although there may be other theoretical perspectives thatmay also explain solvers’ behavior in crowdsourcing, prod-uct development theory and innovation contest theory seemto be better suited for this paper.

New product development

Product development means a “transformation of market op-portunity and a set of assumptions about product technologyinto a product available for sale” (Krishnan and Ulrich 2001,p.1). The literature on physical product development mainlyfocuses on the attributes of a product itself (e.g., product size,shape, configuration, function and dimension) and the orga-nization of the development team (e.g., product developmentteam structure, development process sequence and schedule).Shocker and Srinivasan (1979) and other scholars (e.gBalachandra and Friar 1997; Brown and Eisenhardt 1995;Cusumano and Nobeoka 1992; Griffin 1997) indicated thatattributes of a new product such as its difficulty level, to a

large extent affect whether a smooth development process canbe obtained and a satisfactory resulting product is achieved.Krishnan and Ulrich (2001) emphasized the importance ofenvironmental variables, such as competitive environment inthe market, that influence the development of new products.However, these studies were conducted on product develop-ment in physical environments rather than in an open innova-tion context. Loch and Kavadias (2007) made an effort toextend research on product development to open innovation.Chesbrough (2003) stated that the core of open innovation isto seek out external resources with commercial potential.Terwiesch and Xu (2008) proposed that in an open innovationcontext, when more solvers are involved, more externalresources will be brought in, and thus the seeker can get bettersolutions. Terwiesch and Xu (2008) also found that moresolvers will lead to more creative products, and the bestsolution usually comes from submissions by solvers of thehighest quality. As a type of open innovation, open sourcesoftware depends on external developers, Lerner and Tirole(2002) found that programmers choose to participate when thegains outweigh the costs in the open source projects.

Innovation contest

The first innovation contest model was developed by Lazearand Rosen (1981). They proposed a simple model with onlytwo competitors in the pool to see how to arrange the optimalprize structure to stimulate the best output. Results showedthat higher rewards have positive effects on the output, be-cause they would attract more capable people to participate,therefore better final output is achieved (Lazear and Rosen1981). However, the emphasis on “prize structure” does notmean other attributes of the contest are independent of thefinal result. Frey (1997) considered the influence of difficultylevel from the perspective of utility maximization. He arguedthat people’s behavior is driven by external incentives. Thus,if a task is difficult and the ultimate profits are small, then it ishard to attract solvers to participate; conversely, if the oppor-tunity cost of participation is low, it will attract more solvers.Taylor (1995) and Fullerton and McAfee (1999) studied theoptimal attributes design of a contest with a sequential sto-chastic model. They also studied the competition amongsolvers, and found that seekers will benefit from buyer’spower as the solvers are competing against each other.Leimeister and Huber (2009) researched IT-based ideas com-petition, and found that the competitive situation influencessolvers’ behavior, and seekers may impact solvers’ incentivesby setting project parameters. Yang and Chen (2009) consid-ered the factors of award, duration, competing properties, andmarket maturity, and found that these factors have impacts onthe number of solvers participating.

Previous research on new product development indicatedthat product attributes (e.g. difficulty level) and competitive

Factors affecting participation of solvers in crowdsourcing 75

Page 4: Factors affecting participation of solvers in crowdsourcing: an empirical study from China

market environment impact the process and final outcome ofproduct development. Previous research on innovation con-tests also indicated that task attributes (e.g. reward, difficultylevel) and competition may influence the result of the contest.Crowdsourcing projects have the features of both new productdevelopment and innovation contests. Currently, research oncrowdsourcing mainly focuses on motivations of solvers toparticipate, for example, Lakhani (2007) studied solvers onthe InnoCentive website, and indicated that their main moti-vations are to obtain awards and gain a sense of achievement.Brabham (2008) studied solvers on the Istockphoto website,and found that the principal motivations for them to participatein the projects are to gain recognition and financial rewards.There has been no comprehensive study on how the task orproduct attributes and competition situation affect solvers’behavior in crowdsourcing projects. In addition, most of theprevious studies have the number of solvers as the dependentvariable, and studies on ability level of solvers is rather rare.Yang and Chen (2010) indicated that people with high level ofability will have greater chances of winning in innovationcontests, but how a project’s attributes and competition situa-tion affect the level of winners is still unknown. From theseekers’ point of view, it is important not only to attract moresolvers to participate, but also to attract solvers of higherability levels, because the quality of final solutions dependson both the quantity and quality of solvers that participate.Therefore, in this paper we study the impact of project attrib-utes and competition situation on both number of solvers andthe ability levels of winners.

Research model and hypotheses

Based on above discussion, we propose the research model aspresented in Fig. 1. The model is developed based on newproduct development literature and innovation contest litera-ture. Two types of factors, project attributes (award, duration,and difficulty level) and competition situation (competitionintensity and market price), are supposed to influence thenumber of solvers and ability level of winners in crowdsourc-ing projects.

Impacts of project attributes and competition situationon number of solvers

The innovation contest theory suggests that a higher awardprovides higher anticipated or potential benefit, and willattract more solvers to participate. It also suggests that ahigher award provides better compensation for the transac-tion cost or time cost, which would attract more solvers toparticipate. Lazear and Rosen’s research showed that higherrewards attract more people to participate in innovationcontests (Lazear and Rosen 1981). Lerner and Tirole

(2002) found that programmers are more likely to chooseto participate in the open source software projects when thegains outweigh the costs. This is similar to reverse auctions,where a higher value will attract more bids (Snir and Hitt2003). Thus, it is reasonable to believe that in crowdsourcing,to attract more solvers, a seeker should make his project morecompetitive in terms of higher awards than other projects. Wehypothesize:

H1a: Projects with higher awards will attract moresolvers.

Snir and Hitt (2003) considered the effect of duration inreverse auction and found that longer duration would lead tomore bids. Similarly, in the crowdsourcing context, whenthe duration of a project is longer, solvers will have moretime to prepare for the solutions. Thus the projects with longduration usually require less intensive effort from solversand solvers will be more likely to choose to participate. Wehypothesize:

H1b: Projects with longer duration will attract moresolvers.

A previous study has shown that people are less likely tochoose to participate in innovation contests where the projectsare complex or difficult (Sonsino and Benzion 2002). Shockerand Srinivasan (1979) and other scholars (e.g Balachandraand Friar 1997; Brown and Eisenhardt 1995; Cusumano andNobeoka 1992; Griffin 1997) also concluded that in newproduct development, the difficulty level of a new product,to a large extent, affects whether a smooth developmentprocess can be obtained and a satisfactory resulting productis achieved. Frey (1997) believed that if a task is difficult andthe ultimate profits are small, then it is hard to attract solvers toparticipate; conversely, if the opportunity cost of participation

H2e

H1e

H1d

H2d

H2c

H1c

H2b

H2a

H1b

H1a Award

Difficulty level

Number of solvers

Level of winner

Competition situation

Project’s

attributesDuration

Competition intensity

Market price

Fig. 1 Research model

76 B. Shao et al.

Page 5: Factors affecting participation of solvers in crowdsourcing: an empirical study from China

is low, it will attract more solvers. Easy projects require lowcost and effort from solvers. Thus, we hypothesize:

H1c: Less difficult projects will attract more solvers.

Taylor (1995) and Fullerton and McAfee (1999) studiedsolvers in innovation contests, and found solvers were com-peting against each other. Krishnan and Ulrich (2001) alsoemphasized the importance of competitive environment inthe market that influences the development of a new product.Leimeister and Huber (2009) researched IT-based idea com-petitions, and found that the competitive situation influencessolvers’ behavior. Projects posted in a similar time frame arecompeting with each other over the solver pool (Yang andChen 2009). However, due to the time and capacity con-straints, solvers can only participate in limited number ofprojects within the same period of time. Thus, we hypothesize:

H1d: When there are more competing projects during thesame time period, a project will attract fewer solvers.

In addition to the number of competing projects, therelative attractiveness of a project is also influenced by themarket price among the competing projects. The research ofLerner and Tirole (2002) found that programmers choose toparticipate when the gains outweigh the costs in open sourcesoftware projects. When the market price is high, whichmeans that other competing projects have high awards,solvers face higher opportunity costs when they choose toparticipate in a certain project because they lose certaincapacity to pursue other higher-award projects (Yang andChen 2009). Thus, we hypothesize:

H1e: Higher market price will make a project attractfewer solvers.

Impacts of project attributes and competition situationon ability level of winners

In the task selection phase, seekers need to select the mostsatisfactory work from a number of submissions. In general,seekers favour high-quality solutions. Thus, solvers ofhigher ability level are more likely to win since they mayprovide better solutions. An empirical study of Yang andChen (2010) also showed that solvers of higher ability levelhave greater chance of winning.

Lakhani (2007) studied the winners in InnoCentive, andfound that they are mainly motivated by awards. As statedabove, higher awards will attract more solvers to participate,including solvers of high ability levels. But low skilledsolvers are not competitive with high skilled solvers, espe-cially for tasks with high technical requirements. Thus, theyare not likely to win in high-award tasks. We hypothesize:

H2a: Skill level of the winner will be higher when aproject offers a higher award.

Snir and Hitt (2003) considered the effect of duration inreverse auctions and found that an auction with longerduration would have more bids. As stated above, whenduration of a project is longer, there will be more peopleto participate. Therefore, it will accumulate more experts.As stated by Yang and Chen (2010), solvers of higher abilitylevel have more chance to win. Thus we hypothesize:

H2b: Skill level of the winner will be higher for projectwith longer duration.

The research of Brabham (2008) showed that high-levelsolvers are inclined to participate in challenging tasks.Pragmatically speaking, easy projects have lower difficultylevel and require lower skills to accomplish than difficultprojects. In addition, easier projects usually offer lowawards. Therefore, easy projects may not be attractive tosolvers with high ability levels. Thus, we hypothesize:

H2c: Skill level of the winner will be lower when aproject is easy.

If there are many competing projects at the same time, it ishighly possible that such types of projects are popular on thecrowdsourcing market. Thus, the crowdsourcing market mayhave accumulated many solvers with expertise and experiencein this category, and the final winner in this type of projectshould be a solver of higher ability level. In addition, high-skilled solvers may be more likely to choose competitiveprojects because they may gain more reputation and recogni-tion by winning in such projects. Thus, we hypothesize:

H2d: Skill level of winners will be higher when there aremore competing projects during the same period of time.

The relative attractiveness of a project is also influencedby market price among the competing projects. When themarket price is high, which means that other competingprojects have high awards, according to cost-benefit analy-sis, solvers of high skill level may choose to participate inother higher-award projects for more potential return orbenefit (Yang and Chen 2009). Thus, solvers remaining inthe project may have lower skill level and the skill level ofthe final winner may also be lower. We hypothesize:

H2e: Skill level of the winner will be lower whenmarket price is high.

Research methodology and data analyses

Data collection

Data was collected from zhubajie.com, the largest crowd-sourcing platform in China founded in 2006. The main typesof projects in zhubajie.com are brand design, applicationdesign, software development, web design, naming, creative

Factors affecting participation of solvers in crowdsourcing 77

Page 6: Factors affecting participation of solvers in crowdsourcing: an empirical study from China

writing, and Internet marketing. By the end of October, 2010,it had hosted over 150,000 projects, and the trade volumehad exceeded 140 million Yuan RMB(Approximately 21.24million US dollars. The website has attracted over 4 millionregistered solvers, who may participate in any tasks to get anaward. Currently, the average number of posting projects perday is about 500 on the website, and each project containsapproximately 58 solvers.

Data for this study was collected during the period fromAugust 2009 to October 2010.We chose software developmenttasks as target for the study because they are typically technicalprojects, and the measurement criteria for such tasks are moreobjective and specific compared to other creative tasks, such aslogo design and naming. There were 3479 projects selected intotal, 27.22 % of which offered awards to multiple winners. Weeliminated these projects since optimal design of award struc-ture is not the concern of this study. Ultimately, 2524 projectswere selected as the sample data in our research.

Operationalization of variables

Award Amount (A): The award amount refers to the moneypaid by the seeker to the winner as an award. In China, theplatform has an “award-never-refunded” policy, and the fullamount of money is paid to the website before a project starts.The website charges 20 % of the money as service fees forevery project, thus the remaining money, which is 80 % of thetotal amount, is paid to the ultimate winner. Since the 20 %service fee rate is fixed, which does not affect the results, weuse the total amount as the award amount.

Project Duration (D): The number of days between starttime and end time set by seekers is used as measurement forproject duration. The start and end time is available fromzhubajie.com.

Market Price (P): Market price refers to the average priceof other competing tasks during the same period of time.Since the average duration of projects on zhubajie.com is10 days, we use the average price of other projects in thesame category during 10 days before and after the start dateas the market price for a task.

Competition Intensity (I): Since the average duration ofprojects on zhubajie.com is 10 days, for a specific project,the number of other projects in the same category within10 days before and after the start date can be used tomeasure competition intensity.

Number of Solvers (NS): Refers to the number of solverswho submitted work within the prescribed duration. Thisdata is available from zhubajie.com for each project.

Difficulty level (E): In this paper we use task completionrate (the ratio of number of solvers to number of subscribers)to measure the difficulty level of each project. If the ratio ofnumber of solvers to number of subscribers is low, it meansthe project is difficult and many of the solvers cannot finish it

before the deadline. Otherwise, when the ratio is high, itmeans most or all of the solvers can finish the project beforethe deadline, and the project is easy.

Ability Level of Winner (w): This data is available fromthe website, where 0 refers to the novice, higher valuesrepresent higher ability level of solvers.

The descriptive statistics of variables are shown in Table 1.The correlations between variables are shown in Table 2.

Model estimation

We can see from Table 2 that there exist high auto-correlationsamong some variables in the research model. To avoid auto-correlation, we have all of them natural log-transformed anduse a log-linear model to estimate. Equation 1 is to test hy-pothesis 1a to hypothesis 1e, and Eq. 2 is to test hypothesis 2ato hypothesis 2e. All of the above variables have been naturallog-transformed. Through the Durbin-Watson test we find dvalues (Durbin–Watson d statistics) of all variables are withinthe upper and lower threshold at the significant level of 1 %,which indicates that auto-correlation has been eliminated.Then we use OLS regression to test the hypotheses.

lnN :S ¼ ba0 þ ba1 lnAþ ba2 lnDþ ba3 lnE

þ ba4 ln I þ ba5 lnP þ xa ð1Þ

lnW ¼ bb0 þ bb1 lnAþ bb2 lnDþ bb3 lnE þ bb4 ln I

þ bb5 lnP þ xb ð2Þ

Results and analysis

The results of OLS Regression are shown in Table 3. Theresults of hypotheses tests are shown in Fig. 2

Table 1 Descriptive statistics

Variable Mean. Std Dev Max Min

Award Amount(A)(RMB Yuan)

763.44 2191.17 30000.00 50

Project Duration (D) (Day) 9.83 9.85 82 1

Number of Solvers (N.S) 6.97 6.69 95 1

Market Price (P)(RMB Yuan)

798.31 246.07 2164.67 267.37

Competition Intensity (I) 198.64 82.42 316 27

Difficulty Level (E) 0.62 0.28 1 0.04

Ability Level of Winner(W) 4704.25 9724.56 103950 0

78 B. Shao et al.

Page 7: Factors affecting participation of solvers in crowdsourcing: an empirical study from China

From Table 3, we see that market price does not havesignificant impact on number of solvers, while the influenceof competition intensity is significant at the level of (P<0.05); influence of award, duration and difficulty level onnumber of solvers is significant at the level of (P<0.01).Duration does not have significant influence on winnerability level; competition intensity and market price havesignificant influence on winner ability level at the level of(P<0.1), while award and difficulty level of projects havesignificant influence on winner ability level at the level of(P<0.01). The R2 values in the two equations are 11.23 %and 9.39 % respectively.

Figure 2 also shows that if a task is characterized with ahigher award, less difficulty, longer duration and fewer com-peting projects, then it will attract more solvers. Thus, hypoth-eses H1a-H1d are supported. However, hypothesis H1e is notsupported. This may be because in this research we onlyconsidered the number of solvers as the dependent variable,but did not consider the proportion of high-ability and low-ability solvers in the projects. Higher market price means thatother projects offer higher awards. However, projects withhigh awards also mean they are competitive because solversof high ability are more likely to compete for these projects.Thus, low-ability solvers may be willing to choose the proj-ects that are not so competitive although they may not offerhigh award as competitive projects, thereby increasing the

chance of winning. Therefore, market price may not signifi-cantly affect the number of solvers in a project.

The task attributes of a project will also affect the winner’sability level. Higher award and higher difficulty level lead tohigher ability levels of winners, which supports hypothesesH2a and H2c. This reflects that solvers of higher ability levelare more inclined to participate in tasks with high awards andin difficult tasks that are challenging. Duration does not sig-nificantly affect winner ability levels. This may be explainedas follows: low-ability solvers are inclined to participate intasks with long duration since they are given more time tofinish them, but high-ability solvers may not care about dura-tion much since they are more efficient in preparing solutions.Thus the long duration of a task may attract more low-abilitysolvers to participate than high-ability solvers. Therefore,longer duration may not lead to higher ability levels of finalwinners. Competition intensity also shows significant effecton the ability level of winners. This confirms that solvers ofhigh ability are more likely to participate in competitiveprojects, because winning in such projects may bring themmore self-recognition and satisfaction. Thus, hypothesis H2dis supported. This is consistent with the findings of Brabham(2008) that high level solvers are inclined to participate inchallenging tasks. However, it is surprising that higher marketprice will lead to winners of higher ability level, which is incontrast to hypothesis H2e. This may be due to the market

Table 2 Correlations between variables

Variables Number ofSolvers

AwardAmount

ProjectDuration

DifficultyLevel

CompetitionIntensity

MarketPrice

Ability Levelof Winner

Number of Solvers 1.00

Award Amount 0.72 1.00

Project Duration 0.84 0.87 1.00

Difficulty level 0.82 −0.73 −0.76 1.00

Competition Intensity −0.62 −0.86 −0.78 0.82 1.00

Market Price 0.78 0.84 0.78 −0.74 −0.78 1.00

Ability Level of Winner 0.70 0.03 0.83 −0.89 0.62 0.58 1. 00

Table 3 Results of OLS regression

Variable Equation 1 Equation 2

Symbol Coefficient T Value P Value Symbol Coefficient T Value P Value

Constant βa0 0.8220 2.1075 <0.05 βb0 3.4282 3.8255 <0.01

Ln (Award) βa1 0.1704 14.3423 <0.01 βb1 0.3667 13.4865 <0.01

Ln (Duration) βa2 0.0712 4.0874 <0.01 βb2 −0.0309 −0.7713 Not significant

Ln (Difficulty Level) βa3 0.2890 10.0127 <0.01 βb3 −0.2436 −3.6401 <0.01

Ln (Competition intensity) βa4 −0.0542 −2.0573 <0.05 βb4 0.0859 1.4235 <0.1

Ln (Market price) βa5 0.0301 0.5795 Not significant βb5 0.1887 1.5772 <0.1

R2 R2011.23 % R209.39 %

Factors affecting participation of solvers in crowdsourcing 79

Page 8: Factors affecting participation of solvers in crowdsourcing: an empirical study from China

expansion effect (Yang and Chen 2009). When the marketprice is high, the marketplace will become more attractive topotential solvers, thus more solvers are likely to join themarket. When the market expansion effect outweighs thecompeting effect, the number of high-level solvers will in-crease in the labor pool. It may cause the ability levels ofwinners to become higher.

Figure 3 shows that the numbers of solvers participatingin 500 Yuan–award tasks are significantly higher than that in50 Yuan–award tasks. Figure 4 demonstrates that winner’sability level is significantly higher in 500 Yuan-award tasksthan in 50 Yuan-award tasks. This is consistent with theregression analysis results that task award affects the num-ber of solvers and winner’s ability level.

Discussions, implications and limitations

Discussions

This study empirically reveals some meaningful findings,and demonstrates that different attributes of projects and

different situations of market competition will lead to differ-ences in solvers’ behavior and winners’ ability levels. Mostof the hypotheses have been verified by data from thezhubajie.com crowdsourcing website in China. The empiricalresults imply that tasks with higher awards, less difficulty,longer duration, and fewer competing projects will attractmore solvers. Results show that winner’s ability level is pos-itively related to award and difficulty of the task, and compe-tition intensity of the market. However, the results indicatethat market price does not significantly affect number ofsolvers in a project, and when market price is high, winner’sability level will be high, which is apparently contradictory tothe hypothesis. It may be interpreted by the fact that low-ability solvers may not like to switch to other high-awardprojects that are competitive, and high market price mayattract more high-ability solvers to enter the market due tothe market expansion effect. In addition, the crowdsourcingwebsite has limitations on the maximum number of projectseach solver is permitted to participate in within one month,and due to the time constraints, solvers can only participate inprojects as time and schedule allow, thus at many times, theymay not choose or change to other projects even when theyknow the projects offer higher awards. Duration of projectdoes not show significant influence on ability level of winners.This may be explained by the notion that low-level solvers aremore inclined to participant in tasks with long duration thanhigh-level solvers because low-level solvers need more timeto work on solutions..

Implications for research

This research has some theoretical implications. In this studywe integrate project attributes and market competition factorsto investigate their influences on solvers’ behavior in crowd-sourcing website. Previous studies have no comprehensiveresearch on how the task or product attributes and competitionsituation affect solvers’ behavior in crowdsourcing projects. Inaddition, most of the previous studies have number of solversas dependent variable, but studies on ability level of winnersare rather rare. In prior studies on innovation contests andopen innovation, the attributes of a project, such as award(Lazear and Rosen 1981), duration (Snir and Hitt 2003) anddifficulty level (Balachandra and Friar 1997; Brown and

*~p<0.1 **~p<0.05, ***~p<0.01

0.1887*

0.0301

-0.0542**

0.0859*

-0.2436***

0.2890***

-0.0309

0.3667***

0.0712**

0.1704*** Award

Difficulty level

Competition intensity

Market price

Number of solvers

Level of winner

Competition situation

R2=11.23%

R2=9.39%

Duration Project’s

attributes

Fig. 2 Results of hypotheses test

01020304050607080

2009

0804

2009

0913

2009

1118

2010

0105

2010

0216

2010

0306

2010

0312

2010

0322

2010

0402

2010

0413

2010

0419

2010

0427

2010

0518

2010

0528

2010

0603

2010

0610

2010

0617

2010

0623

2010

0703

2010

0721

2010

0727

50050

Fig. 3 Comparison on number of solvers

0

20000

40000

60000

80000

100000

120000

2009

0804

2009

0920

2009

1212

2010

0203

2010

0303

2010

0312

2010

0322

2010

0402

2010

0413

2010

0422

2010

0508

2010

0526

2010

0603

2010

0611

2010

0618

2010

0627

2010

0718

2010

0725

50050

Fig. 4 Comparison on winner’s ability level

80 B. Shao et al.

Page 9: Factors affecting participation of solvers in crowdsourcing: an empirical study from China

Eisenhardt 1995; Cusumano and Nobeoka 1992; Frey 1997;Griffin 1997) have been shown to have effects on participants’behaviors. This study extends the research, and further dem-onstrates the effects of project attributes in a crowdsourcingcontext. This study also confirms the effect of competitionintensity, which has been studied in previous research(Leimeister and Huber 2009), and tests the effect of marketprice on solvers’ behaviour in crowdsourcing. Besides thenumber of solvers attracted, this study uses ability level ofwinners as a new dependent variable to get a deeper under-standing of solvers’ behavior. Thus, this research is morecomprehensive compared to prior studies on crowdsourcing,and provides a better theory to explain the success of crowd-sourcing projects.

In this study, product development theory and innovationcontest theory were used as the basis for studying factorsaffecting solvers participation in crowdsourcing. Previousresearch on product development and innovation contest ismainly based on traditional physical environment. Thisstudy applied the theories to the corwdsourcing context,which is an Internet-based open innovation. Thus, this re-search has extended the application of product developmenttheory and innovation contest theory, and made theoreticalcontributions. Furthermore, most previous studies focus ei-ther on attributes or market competition. This study revealsthat attributes of a project and market competition situationtogether can give a deeper understanding of solvers’ behaviorson crowdsourcing websites. Furthermore, this study indicatesthat some factors such as project duration and market pricethat are effective in other product development or innovationcontexts, , may not play significant role in the crowdsourcingcontext.

This study also broadened our knowledge of open innova-tion. Previously, most open innovation research was targetedat open source software or wikipedia, which are types ofcollective intelligence. However, crowdsourcing is differentfrom open source software or wikipedia in that it includes amarket mechanism and seekers offer awards to solvers whowin the contests. In this study, factors related to contest, e.g.award, market price, are introduced, which enriches the theoryand literature on open innovation.

Practical implications

This study also provides implications for business professio-nals, particularly the seekers and solvers, to reach their prag-matic decisions more effectively and efficiently. In essence,results show that number of solvers will be influenced byattributes of a project. Thus seekers can achieve their goalsthrough deliberate design of project or task attributes in orderto receive more submissions. Awards have great influence onboth number of solvers and ability level of winners. Thus,when seekers decide the amount of award to offer, the task

difficulty level and project duration should be considered. Ifthe task is difficult or the project has short duration, higherawards should be offered to attract solvers. If seekers want toreceive better solutions from solvers, they should offer higherawards because higher award may help attract solvers of highability, who may provide better solutions, especially for diffi-cult tasks.

More submissions come with higher assessment cost of theseeker. Due to the seeker’s limited capacity and time con-straints, it is necessary to indicate that a greater number ofattracted solvers may not always be a good thing. Attentionbased theories (Laursen and Salter 2005; Ocasio 1997; Simon1997) suggest that a contest should have an optimal number ofsolvers, which just fits the seeker’s capability of exploitingexternal knowledge such as assessment and feedback.Therefore, seekers need to know the tradeoff between numberand quality of submissions. More attention needs to be paid toattract high-ability solvers and get high-quality submissionsrather than just to increase the number of submissions.

In addition, seekers should know the competition situationwhen posting projects on crowdsourcing platforms. Since thenumber and price of competing projects may impact thenumber and quality of solvers this project may attract, seekersshould be aware of how many competing projects exist on thesame platform and the award levels set by the competingprojects. When the competition is higher, it is necessary tooffer a higher award or set longer project duration in order toattract more solvers with high ability level to participate.

Finally, this study also provides suggestions for crowd-sourcing platform building. The crowdsourcing platform canhelp seekers’ decision-making by making the competitioninformation more accessible and clearer to seekers on theplatform. According to the zhubajie.comwebsite, seekers tendto consult customer service on task attributes setting becausethere is no guidance or direction for seekers when they setattributes of a task before posting. To help seekers set taskattributes, the crowdsourcing website may provide to seekersinformation about past projects, including their award amount,duration, difficulty level, and number of solvers and abilitylevel of final winners, to support their decision-making, thusmaking the platform more useful and helpful to crowdsourc-ing participants.

Limitations

This study has inevitably suffered from several limitations.For example, this study uses the solvers’ ability values fromzhubajie.com as measurement of ability level of winners.However, on zhubajie.com solver’s ability value is a dy-namic value, which is represented by the total awards asolver won in previous projects. Thus solvers may havehigh ability value just because they have been registeredon the website for a long time, and have participated in

Factors affecting participation of solvers in crowdsourcing 81

Page 10: Factors affecting participation of solvers in crowdsourcing: an empirical study from China

many projects. Similarly, solvers with low ability value mayhave entered the website only recently, it does not necessar-ily mean they have low ability. Furthermore, in this re-search, due to data limitation we use the number of solversand winner ability level as dependent variables, but have notconsidered the proportion of high-ability and low-abilitysolvers among the solvers. In addition, as we can see thatthe R2 values are low. This is another limitation of thispaper.

Conclusions

In this study, based on data from a Chinese crowdsourcingwebsite, we analyze how the quantity and quality of solvers inonline crowdsourcing projects are influenced by the projectattributes and competition situation. The results of this studyprovide implications for seekers on crowdsourcing platformson how to attract more high-level participants to achieve bettersolutions. Due to the limitations of the current research, infuture research we will use some more objective measure-ments to test the model.

Acknowledgement This research was supported by the NationalSocial Science Foundation of China (No. 09CTQ023), Natural ScienceFoundation Project of CQ CSTC (No.2008BB2042), and Science andTechnology Innovation Fund for individual Graduate Students ofChongqing University (CDJXS11020023).

References

Allion, R. J. (2003). CEO interview: the InnoCentive Model of openinnovation. Strategy & Leadership, 32(4), 4–9.

Balachandra, R., & Friar, J. H. (1997). Factors for success in R&Dprojects and new product innovation: a contextual framework.IEEE Transaction. On Engineering Management, 44(8), 276–287.

Brabham, D. C. (2008). Crowdsourcing as a model for problem solv-ing. The International Journal of Research into New Media Tech-nologies, 14(1), 75–90.

Brown, S. L., & Eisenhardt, K. M. (1995). Product development: pastresearch, present findings, and future directions. ManagementReview, 20(4), 343–378.

Chesbrough, H. (2003). Open innovation: The new imperative forcreating and profiting from technology. Cambridge: HarvardBusiness School Press.

Cusumano, M. A., & Nobeoka, K. (1992). Strategy, structure andperformance in product development-Observations from the autoindustry. Research Policy, 21(1), 265–293.

Frey, B. (1997). Not just for the money: An economic theory ofpersonal motivation. Brookfield: Edward Elgar PublishingCompany.

Fullerton, R. L., & McAfee, R. P. (1999). Auctioning entry into tourna-ments. Political Economy, 107(3), 573–605.

Griffin, A. (1997). The effect of project and process characteristics onproduct development cycle time.Marketing Research., 34(2), 24–35.

Howe, J. (2006). The rise of croudsourcing. Wired, 6, 227–264.iResearch. (2010). China Witkey Industrial White Paper, 11, 2010.Krishnan, V., & Ulrich, K. T. (2001). Product development decisions: a

review of the literature. Management Science, 47(1), 1–21.Lakhani., K. (2007). The principles of distributed innovation. Innova-

tions, 23, 97–112.Laursen, K., & Salter, A. (2005). The paradox of openness in innova-

tive search: appropriability and the use of external sources ofknowledge for innovation. Strategic Management Journal, 25,467–493.

Lazear, E. P., & Rosen, S. (1981). Rank-order tournaments as optimumlabor contracts. Journal of Political Economy, 89, 841–864.

Leimeister, J., & Huber, M. (2009). Leveraging crowdsourcing:activation-supporting components for IT-based ideas competition.Journal of Management Information Systems, 26(1), 197–224.

Lerner, J., & Tirole, J. (2002). Some simple economics of open source.Journal of Industrial Economics, 50, 197–234.

Loch, C. H., & Kavadias, S. (2007). Handbook of new product develop-ment management. Burlington: Butterworth Henineman-Elsevier.

Ocasio, W. (1997). Towards an attention-based view of the firm.Strategic Management Journal, 18, 187–206.

Shocker, A. D., & Srinivasan, V. (1979). Multiattribute approaches forproduct concept evaluation and generation: a critical review.Journal of Marketing Research, 16, 159–180.

Simon, H. A. (1997). Administrative behavior: A study of decision-making process in administrative organization. New York: FreePress.

Snir, E. M., & Hitt, L. M. (2003). Costly bidding in online markets forIT services. Management Science, 49, 1504–1520.

Sonsino, D., & Benzion, U. (2002). The complexity effects on choicewith uncertainty–experimental evidence. The Economic Journal,112, 936–965.

Taylor, W. C. (1995). Digging for golden carrots: an analysis ofresearch tournament. American Economics Research, 85(4),872–890.

Terwiesch, C., & Xu, Y. (2008). Innovation contest, open innovationand multi-agent problem solving.Management science, 54, 1529–1543.

Yang, Y., & Chen. P. (2009). Open Innovation: Strategic Design ofOnline Contests. Proceeding of International Conference on In-formation Systems 2009, Pheonix, Arizona.

Yang, Y., & Chen. P. (2010). Who Will Win? Secrets behind Innova-tion Contests. Proceedings of International Conference on Infor-mation Systems 2010, Saint Louis, Missouri.

82 B. Shao et al.