4 zait bertea fff

7/30/2019 4 Zait Bertea FFF

1/8

METHODS FOR TESTING DISCRIMINANT VALIDITY

Professor PhD Adriana ZAIUniversity A. I. Cuza, Iai, Romania

Email: [email protected] Student Patricea Elena BERTEAUniversity A. I. Cuza, Iai, Romania

Email: [email protected]

Abstract:The study presents three methods which can be used to assess discriminantvalidity for multi-item scales. Q-sorting is presented as a method that can beused in early stages of research, being more exploratory, while the chi-square difference test and the average variance extracted analysis arerecommended for the confirmatory stages of research. The paper describes

briefly the three methods and presents evidence from two surveys that aimedto develop a scale for measuring perceived risk in e-commerce.

Keywords: validity, discriminant validity, Q-sorting, confirmatory factorialanalysis

IntroductionScale development represents an

important area of research in Marketing.Since we deal with latent variables

which are not observable we have tocreate instruments in order to measurethem. Variables such as personality orperceived risk are measured throughmulti-item scales. When developingsuch a scale researchers generally lookat one important criteria which is thelevel of reliability given by alphaCronbach values. A value of more than0,7 for alpha Cronbach is consideredacceptable (Nunnaly, 1967). Scale

reliability is influenced by several factorsof research design (Bertea, 2010), thisis why it is important to apply othermethods in order to be sure that theinstrument presents reliability as well asvalidity.

Construct validity refers more tothe measurement of the variable. Theissue is that the items chosen to buildup a construct interact in such mannerthat allows the researcher to capture the

essence of the latent variable that hasto be measured.

It is important to make thedistinction between internal validity andconstruct validity. The first one refers toassuring a methodology that enablesthe research to rule out alternativeexplanations for the dependentvariables, while construct validity ismore concerned with the choice of theinstrument and its ability to capture thelatent variable. Internal validity becomesa problem in experimental studies,where each experimental group has tofollow the same methodology in order tobe able to correctly isolate the effect.Construct validity has three

components: convergent, discriminantand nomological validity. Discriminantvalidity assumes that items shouldcorrelate higher among them than theycorrelate with other items from otherconstructs that are theoreticallysupposed not to correlate.

Testing for discriminant validity canbe done using one of the followingmethods: O-sorting, chi-squaredifference test and the average variance

extracted analysis.


2/8

Management&Marketing, volume IX, issue 2/2011218

Q-sortingThe Q-sorting procedure aims to

separate items in a multi-dimensionalconstruct according to their specificdomain. There are two ways that it canbe done (Storey, et al., 1997):

- Exploratory, when respondentsare given the items and asked to groupand identify category labels for eachgroup of items.

- Confirmatory, when thecategories are already labeled andrespondents are asked to classify eachitem in one category.

Q-sorting is applied on experts andother persons of interest for theresearch. It helps eliminate items thatdo not discriminate well betweencategories of items.

For the confirmatory procedure,the analysis can be made by calculatinga percent for correct classification ofeach item from a construct. When thispercent has low values, that means wehave items with problems that do notdiscriminate well in relation with otheritems that form a different construct.

Chi-square difference testAnother method that can be used

to assess discriminant validity is to do achi-square difference test (Segars,1997) that allows the researcher tocompare two models, one in which theconstructs are correlated and one inwhich they are not. When the test issignificant the constructs presentdiscriminant validity. In order to do that

the constructs are analyzed using

Confirmatory Factor Analysis (CFA)which is commonly used for validityissues. The measurement modelsshould be reflective and should beintroduced in analysis in pairs of two.So, we will compare each time two

constructs that we suspect to haveproblems with items discriminatingamong them. The first model analyzedthrough CFA will a model where the twoconstructs are not correlated, while thesecond will be the one where we willallow for correlation. Each model will

present a value for Chi-square ( )

and degrees of freedom (df). After doingthe difference between the values of the

two models we can see if the test issignificant or not.

2

Average variance extracted

analysisIn order to establish discriminant

validity there is need for an appropriateAVE (Average Variance Extracted)analysis. In an AVE analysis, we test tosee if the square root of every AVEvalue belonging to each latent construct

is much larger than any correlationamong any pair of latent constructs.AVE measures the explained varianceof the construct. When comparing AVEwith the correlation coefficient weactually want to see if the items of theconstruct explain more variance than dothe items of the other constructs.

AVE, which is a test of discriminantvalidity, is calculated as:

[i2]

AVE = ,[i

2]+[Var(i)]

where i is the loading of eachmeasurement item on its corresponding

construct and i is the errormeasurement.

The rule says that the square rootof the AVE of each construct should bemuch larger than the correlation of thespecific construct with any of the other

constructs. The value of AVE for eachconstruct should be at least 0.50(Fornell and Larcker, 1981).


3/8

Management&Marketing, volume IX, issue 2/2011 219

Perceived risk in e-commerce

Marketing literature talks aboutperceived risk as a multi-dimensionalconstruct. That means each dimensionrepresents a construct in itself and ismeasured through multiple-items. Intraditional commerce there were definedsix dimensions of perceived risk:financial, physical, functional orperformance, psychological, social(Jacoby & Kaplan, 1972) and a time riskor convenience risk (Roselius, 1971).However, in the context of e-commerceit is important to notice that there arechanges as far as the risk concepts areconcerned. Crespo et al. (2009) study

the multi-dimensional perceived risk inrelation with a certain product boughtthrough the Internet. Nevertheless, it

makes sense to analyze perceived riskalso in relation with the shoppingchannel. Featherman and Pavlou(2003) talk about financial, social,psychological, time, privacy risk andperformance risk. They refer to the risk

of the shopping channel, not of theproduct. The authors argue thatadopting e-services involves a muchhigher risk than e-commerce adoptionas users are to engage in a long termrelationship. In analyzing the influenceof perceived risk on e-services adoptionintention, Featherman and Pavlou(2003) used the basic TAM (TechnologyAcceptance Model, Davis, 1989) andthe multi-dimensional perceived riskapproach (fig. 1). Each dimension ofperceived risk was measured throughmultiple-items.

Figure 1. Featherman and Pavlou (2003) research modelSource: Featherman, M. S. & Pavlou, P. A. (2003), 'Predicting e-services adoption: aperceived risk facets perspective', International Journal of Human-Computer Studies 59(4),p.457

The use of multi-item scales fomeasuring perceived risk wasrecommended by Mitchell (1999) whoconsidered that it is a better way to goinside the consumers mind and to find

out what really defines his behavior.

r Research MethodologyThe present study has used

perceived risk as a multi-dimensionalconstruct, having six dimensionsredefined in the context of e-commerce

(table 1).


4/8


Table 1

Perceived risk dimensions

Type of risk Definition Numberof items

Financial The risk of losing money when buying online. 5Product The risk of getting a product that it is not what

presented on the website.

6

Delivery The risk of having a delayed delivery. 4Security The risk that the personal data is stolen and

used for identity theft.3

Social The risk that the social group does not agreewith e-commerce.

4

Psychological The risk of feeling anxiety when shopping online. 4

For the Q-sorting study wedeveloped a questionnaire were we

included all items measuring perceivedrisk without showing which item belongsto which type of perceived risk.

Respondents had to classify items into6 categories: social, psychological,

financial, security, product and deliveryrisk (table 2).

Table 2Q-sorting questionnaire example

Risk Item Chek the risk typeappropriate for the item

SocialFinancialPsychological

SecurityDelivery

Online shopping gives me

a state of stress becauseit does not fit with my self-image. Product

The questionnaire was applied ona sample formed of 23 students. Thesmall sample was due to the fact thatthe research was in exploratory stage.As a quantitative indicator of the Q-sorting procedure we used the correctclassification percent, which describesthe percent of respondents that havecorrectly classified an item (Straub, etal., 2004).

For the Chi-square difference testand the AVE analysis we applied aquestionnaire that aimed to measureperceived risk in e-commerce on asample of 481 students. The largersample was necessary since theresearch stage was confirmatory. Thequestionnaire had items that used a 7point Likert scale, items which wereeither taken from the literature(Featherman & Pavlou, 2003; Pires et

al., 2004; Forsythe et al., 2006; Crespoet al., 2009) either from in-depthinterviews. The in-depth interview wasused as a qualitative method in order toobtain information that is common toRomanian consumers, users of Internet.The motivation is that the other scaleswere developed on different populationswhich are characterized by differentcultures and levels of economicdevelopment.

For the Chi-square difference testwe performed a confirmatory factoranalysis using AMOS 19, while for theAVE we did the correlation matrix forthe types of risk and calculated the AVEvalues for each type of risk.


5/8


ResultsSince this study aims to discuss

methods for assessing discriminantvalidity we will present only results thatwere obtained by applying the threemethods previously mentioned.

Q-sortingIn order to calculate the percent of

correct classification, we identified thefrequency of respondents that checkedthe correct category for each item. Wehad items that obtained a 100% correctclassification 3 items, items that had

percents higher than 70% 22 items,but also items with lower percents -4items. We considered items with a lowclassification percent those who werebelow 60% (table 3).

Taking into account that more than

80% of all 26 items were correctlyclassified, we can consider that thescale has a good level of discriminantvalidity. However, it is important tofurther analyze those items that werenot correctly recognized as belonging toa certain category of risk.

Table 3Q-sorting results (items with low classification)

Risk type Item Percent

PsychologicalOnline shopping does not fit my self-image.

0.52

Security/privacy

There is high chance that hackers takeover my personal account from a e-shop.

0.59

If I do my shopping online, There is a highrisk that I receive a different product thatthe one I ordered.

0.52Time/delivery

When I buy online I am sure that I willreceive exactly the product I ordered.

0.22

Chi-square difference testWe will exemplify the chi-square

difference test on two constructs thathad items which were suspected to

produce confusion among respondents.The two constructs are: product risk anddelivery risk. In order to test fordiscriminant validity we followed Segars(1997) recommendations:

Create a model in which the twoconstructs do not correlate and performCFA (fig. 2)

Create a model in which the twoconstructs correlate and perform CFA(fig. 2)

Do the chi-square difference

test and if the test is significant thandiscriminant validity exists.

We introduced the two models intoAMOS and performed the analysis. Weset correlation to 0 for the first model(left side of figure 2) and for the secondmodel we allowed free correlation (rightside of figure 2).


6/8


Figure 2 Product risk versus delivery risk

Afterwards we calculated the chi-square difference test to see if it is significantor not (table 4).

Table 4CFA results

Model 1 Model 2

Chi-square = 400.58Degrees of freedom =35Probability level = 0.000

Chi-square = 133.632Degrees of freedom =34Probability level = 0.000

= 21 266.948

= 21 dfdf 1

The difference test result wassignificant (p=0 < 0,05) which meansthat the two constructs presentdiscriminat validity.

AVE analysisAs said before, the AVE values

calculated according to the formulapresented must be compared with the

correlation coefficients of each constructwith the other constructs. So, first of allit is necessary to obtain a matrix werewe can see the correlation of each typeof risk with the other types. Afterwardson the diagonal we insert the AVE valuein order to compare it with the othercorrelation coefficient (table 5).

Table 5

AVE analysisDelivery

riskFinancial

riskProduct

riskPsychologicalrisk

Socialrisk

Securityrisk

Deliveryrisk

0,707

Financialrisk

,539** 0,707

Productrisk

,652** ,551** 0,707

Psychological risk

,357** ,325** ,559** 0,707

Social risk ,159** ,201** ,257** ,517** 0,707

Securityrisk ,541** ,616** ,507** ,243** ,133** 0,707


7/8


Table 5 shows results of the AVEanalysis. It can easily be seen that theAVE values are above 0.5 and,moreover, are above the correlationcoefficients for each type of perceivedrisk.

ConclusionsConclusions refer to the use of

assessment methods for discriminantvalidity. Since the study aimed topresent the methods and exemplify theiruse, conclusions will not regard whetherperceived risks dimensions proved ornot discriminant validity. Thus, thissection will be concerned more onrecommendations regarding themethods.

Therefore, we strongly recommendthe Q-sorting procedure should be usedin early stages of research, especially inthe exploratory stage because it canreveal those items that generateproblems for discriminant validty. If asmall percent of respondents cancorrectly classify an item to its rightfulcategory, than the researcher should

consider reformulating the item orrejecting it from the construct. Q-sorting

is an easy to apply procedure, howeverits weak point is that the sample usedshould be formed mainly by experts andsometimes experts are not available ortheir number is very low.

The other two procedures should

be used in the confirmatory stage ofresearch. It is not necessary to applyboth methods in one research, becauseboth of them are strong and give validresults. Nevertheless, it seems that theAVE analysis is more popular becauseit offers a more parsimoniousprocedure, having all constructsgrouped in one matrix. The Chi-squaredifference test is time consuming sincewe have to take 2 constructs each timeand in the case of perceived risksdimensions we would have performed15 tests for verifying discriminantvalidity of each construct.

In conclusion, the Q-sortingprocedure should be use in the phase ofexploratory research when developing ascale for measuring latent variables,while the AVE analysis and the Chi-square difference test must be used inthe confirmatory stage.

REFERENCES

Bertea, P. (2010), 'Scales For Measuring Perceived Risk In E-Commerce-TestingInfluences On Reliability', Management and Marketing JournalCraiova8(S1),S81--S92.

Crespo, .. H.; del Bosque, I. R. & de los Salmones Snchez, M. M. G. (2009),The Influence Of Perceived Risk On Internet Shopping Behavior: AMultidimensional Perspective, Journal of Risk Research12(2), 259277.

Davis, F. (1989), 'Perceived usefulness, perceived ease of use, and useracceptance of information technology', MIS quarterly, 319--340.

Featherman, M. S. & Pavlou, P. A. (2003), 'Predicting e-services adoption: aperceived risk facets perspective', International Journal of Human-ComputerStudies59(4), 451 - 474.

Forsythe, S.; Liu, C.; Shannon, D. & Gardner, L. C. (2006), 'Development Of A

Scale To Measure The Perceived Benefits And Risks Of Online Shopping',Journal Of Interactive Marketing20(2).


8/8

4 zait bertea fff

Documents