development of a regular csitc round trial axel drieling

1 A. Drieling, J. Knowlton – Beltwide Cotton Conferences 2007 / 01

Lab of the Bremen Cotton Exchange

Development of a Regular CSITC Round Trial

Axel Drieling Faserinstitut Bremen e.V. (Bremen Fibre Institute)

- Laboratory of the Bremen Cotton Exchange -

James L. KnowltonUSDA, AMS, Cotton Program

Memphis, TN

Beltwide Cotton ConferencesNew Orleans, LA, January 9-12, 2006

Cotton Quality Measurements Conference



CSITC Task Force

• ICAC Task Force on Commercial Standardization of Instrument Testing of Cotton, formed in December 2003

• 15 international members: cotton exporters, importers, all segments of the world cotton industry and research

• Based on the consensus that instrument testing is superior to traditional hand classing• Objective: Introduction of a worldwide acceptable, adoptable and reliable

instrument based cotton quality assessment – with defined test rules– based on reliable instrument test results– based on a worldwide cotton testing laboratory evaluation system

• Options for instrument testing– Fixed calibration with Universal Standard Material (HVICCS etc.)– Choice of suitable parameters: at present Mic, Strength (HVICCS), Length (UHML HVICCS),

Uniformity Index, Color Rd and Color +b– Only / All instruments are suitable

• that conform to Universal Standard Material results for the chosen parameters • that do not show systematic deviation • that do not show extended result variability



CSITC Round Trial

• It is essential to check the reliability of cotton testing laboratories and the test results provided by the laboratories

• Round Trial results will allow the laboratories to demonstrate their capability to meet recommended standards, but certification cannot guarantee the accuracy of individual tests

• Round Trial results will help laboratories to achieve more accurate results

(Participation in the CSITC Round Trial is anonymous and individual results will only be given to the individual participant)



Excursion: Definitions / Wording

• Wording is chosen based on ISO 5725

• Trueness refers to the closeness of agreement between the arithmetic mean of a large number of test results (of one laboratory) and the accepted reference value

• Precision refers to the closeness of agreement between (single) test results / variability of instrument results on tested cottons

• Accuracy includes trueness and precision



Attributes of Different International Round Trials

Attribute USDA HVI Checktest Bremen Round Trial CSITC Round Trial

Number of participants 50 to 80 SITC 90 to 110 SITC ??

Kinds of instruments High volume (SITC) Every kind High volume (SITC)or similar results

Standard material Universal Calib. Std. Various std. materials Universal Calib. Standards

Cottons: Origin and type

USA;Upland

World;broad range of prop.

USA / world;Upland or similar

Frequency 12 times/yeareach 2 samples

4 times/yeareach 1 sample

4 times/yeareach 4 to 5 samples

Number of tests per sample

Asked for 12 testsfor each cotton

Proposed: typical number in the lab for

testing of cotton types

Fixed number: 5 days with each 6 tests on each

cotton = 30 test per cotton

Aim Information for the laboratory

Information for the laboratory

Official laboratory evaluation and detailed analysis for the laboratory

Evaluation of Laboratory average Laboratory average Laboratory average and all single data

Evaluation of Trueness only Trueness only Trueness and precision



Aims of the CSITC Round Trial

a. Evaluation of the test methods / Test result variability• inter-laboratory variability

• in-laboratory variability

• possbible additions: variations between instrument types etc.

b. Evaluation / rating of the participating laboratories- based on the trueness of the results -

c. Detailed analysis of laboratory results to achieve more accurate results- based on trueness and precision -



Configuration of the Round Trial Tests

Cotton 1 Cotton 2 Cotton 3 Cotton 4 Cotton 5

day 1 6 tests 6 tests 6 tests 6 tests 6 tests





Sub Total 30 tests 30 tests 30 tests 30 tests 30 tests

Total 150 tests for each Round Trial

Only cotton 1 to cotton 4 are used to evaluate laboratoriesCotton 5 offers the chance to include cottons with different behavior / from different origins



Basis: Reference Results

• Laboratory results are compared to reference results based on the interlaboratory average of the participating laboratories

– In case that the number of participating laboratories is not satisfactory to calculate reliable reference results, the USDA Established Results will be added for consideration

• Possibilities for the consideration of outliers

– No exclusion of outliers

– 90% trimmed mean (exclusion of 5% of the lowest and 5% of the highest results)

– Statistical detection of outliers: (e.g.) Grubbs Method

Grubbs Method was chosen, as only a statistical detection of outliers allows both:

– to calculate a reference result that is not influenced by outliers and

– to calculate inter-laboratory variability results that are not systematically biased



Excursion:Reference Results Consideration of Outliers

Average Inter-Laboratory CVsbased on 30 tests

0

1

2

3

4

5

6

7

Micronaire Strength Length Uniformity Color Rd Color +b

Coe

ffici

ent o

f Var

iatio

n (C

V%

)

All results

OutliersGrubbs90% trimmed mean



Excursion:Reference Results Consideration of Outliers

Average Inter-Laboratory CVs for Micronaire

0

0,5

1

1,5

2

2,5

3

3,5

4

Micronairebased on 30 tests

Micronairebased on

6 tests

Micronairebased on

single tests

Coe

ffici

ent o

f Var

iatio

n (C

V%

)

All results

OutliersGrubbs90% trimmed mean



a) Test Result Variability

Micronaire based on 30 tests

02468

101214161820

3,1

3,2

3,3

3,4

3,5

3,6

3,7

3,8

3,9 4

Micronaire, Mic

Num

ber o

f Lab

s Micronaire based on 6 tests on single days

0

20

40

60

80

100

120

3,1

3,2

3,3

3,4

3,5

3,6

3,7

3,8

3,9 4

Micronaire, Mic

Num

ber

of L

abs

x Da

ys

Exemplary inter-laboratory distributions of the results:Micronaire for Cotton 1a) Based on 30 testsb) Based on 6 tests on one day




Cotton 1 Cotton 2 Cotton 3 Cotton 4 AverageAverage of Laboratories (Grubbs) 3,83 5,17 4,40 3,81Number Of Labs/Instruments 46 46 46 46 46

SD 0,07 0,06 0,06 0,07 0,06CV % 1,76 1,14 1,46 1,73 1,52SD 0,07 0,07 0,10 0,07 0,08CV % 1,93 1,38 2,20 1,95 1,87SD 0,09 0,09 0,11 0,09 0,09CV % 2,25 1,69 2,44 2,30 2,17SD 0,03 0,03 0,04 0,03 0,03CV % 0,78 0,67 0,80 0,82 0,77SD 0,04 0,05 0,04 0,05 0,05CV % 1,10 1,00 0,97 1,24 1,08SD 0,05 0,06 0,06 0,05 0,06CV % 1,34 1,17 1,28 1,37 1,29

Micronaire

Inter-Laboratory Variation (Grubbs)

Typical In-Laboratory Variation (Median of all labs)

based on 30 tests

based on 6 tests

based on single testsbetween different dayswith each 6 testsbetween single testson one daybetween all testson different days

Strength

Length

Uniformity

Color Rd

Color +b






Strength



based on 30 tests

based on 6 tests







Length



based on 30 tests

based on 6 tests







Uniformity



based on 30 tests

based on 6 tests







Color Rd



based on 30 tests

based on 6 tests







Color +b



based on 30 tests

based on 6 tests




b) Evaluation / Rating of Each Participating Laboratoryfor the example of laboratory 115

Micronaire Strength Length Uniformity Color Rd Color +bReference Values Cotton 1 3,83 32,82 1,207 82,42 76,31 12,14

Cotton 2 5,17 28,22 1,136 81,90 78,06 11,53Cotton 3 4,40 25,54 0,948 78,53 74,86 10,86Cotton 4 3,81 32,89 1,177 83,65 76,08 10,98

Laboratory Average of All Days Cotton 1 3,80 33,62 1,207 82,71 75,37 11,38Cotton 2 5,23 28,50 1,134 81,44 76,05 10,82Cotton 3 4,36 26,11 0,969 76,13 73,62 10,41Cotton 4 3,79 32,72 1,182 83,83 75,29 10,17

Rel. Distance to Reference Cotton 1 -0,03 0,80 0,000 0,29 -0,94 -0,76Cotton 2 0,06 0,28 -0,003 -0,46 -2,00 -0,71Cotton 3 -0,04 0,57 0,021 -2,40 -1,24 -0,45Cotton 4 -0,02 -0,18 0,005 0,18 -0,79 -0,81

Mean Absolute Distance to Reference 0,04 0,46 0,007 0,83 1,24 0,68

Scale Factor (Based on USDA Reproducibility Limits except Rd) 0,10 1,50 0,02 1,00 1,50 0,50Summary Evaluation for Each Property(=Mean Abs. Distance divided by Scale Factor)

0,38 0,31 0,36 0,83 0,83 1,37

Relevance of Property 1,00 1,00 1,00 1,00 1,00 1,00

Summary Evaluation of All Properties(=Average of all properties)

Performance of Laboratory 115

0,68



b) Evaluation / Rating of Each Participating Laboratoryfor the example of laboratory 115

Evaluation of the Laboratory 115 0,68

Average 0,67Median 0,53Best Lab. 0,32Worst Lab. 3,16

Statistics- All Laboratories -

Evaluation Combined Prop.

Laboratory No. Evaluation Combined Prop.

Evaluation Results- Combined Properties -

0

2

4

6

8

10

12

14

00,

10,

20,

30,

40,

50,

60,

70,

80,

9 11,

11,

21,

31,

41,

51,

61,

71,

81,

9 22,

12,

22,

32,

42,

52,

62,

72,

82,

9 33,

13,

2

Evaluation Results

No.

of L

abor

ator

ies

Same kind of presentation

is done for every single

property



c) Detailed Laboratory Analysisfor the example of laboratory 115

Laboratory 115 - Color Rd

R2 = 0,4498

-4

-3

-2

-1

0

1

2

70 71 72 73 74 75 76 77 78 79

Reference Value

Devi

atio

n fr

om R

efer

ence

Color Rd - AverageSingle Days

1. Detailed Trueness Evaluation to find systematical deviations / biases



c) Detailed Laboratory Analysisfor the example of laboratory 115

2. Precision Evaluation to show / compare laboratory test result consistency/variability

Cotton 1 Cotton 2 Cotton 3 Cotton 4 Average RemarkAverage Test Result 12,14 11,53 10,86 10,98 --- ---

between different days with each 6 tests SD 0,13 0,16 0,10 0,15 0,13 ---between single tests on one day SD 0,13 0,15 0,10 0,13 0,13 ---between all tests on different days SD 0,19 0,23 0,15 0,19 0,19 ---

between different days with each 6 tests SD 0,27 0,24 0,09 0,10 0,17higher than typical variation

between single tests on one day SD 0,30 0,27 0,17 0,21 0,24higher than typical variation

between all tests on different days SD 0,41 0,35 0,19 0,26 0,30higher than typical variation

Typical In-Lab. Variation (Median)

In-Lab. Variation (Your Lab.)

Laboratory 115 - Color +b



Conclusion

• CSITC Round Trial– Is based on the recommendations of the CSITC Task Force– Was developed in co-operation between the Bremen Fibre Institute and

USDA-AMS – Is headed by the International Cotton Advisory Committee (ICAC)– Will be conducted regularly in co-operation between the Bremen Fibre Institute and

USDA-AMS

• Timetable– 1. Pilot Round Trial was conducted in 2005– 2. Pilot Round Trial was conducted in 2006, with 46 laboratories/instruments– Regular Round Trials are starting now (2007-1)

• Samples are already shipped• Approx. 60 laboratories will participate with approx. 90 instruments

– For registration in future Round Trials, please contact the ICAC

development of a regular csitc round trial axel drieling

Documents