learning objectives: a systematic method to state objectives and analyse consistency of a science...

Upload: dr-samira-el-boudamoussi

Post on 30-May-2018

220 views

Category:

Documents


0 download

TRANSCRIPT

  • 8/14/2019 Learning objectives: A systematic method to state objectives and analyse consistency of a science program

    1/17

    E L S E V I E R Studies in Educational Evaluation 32 (2006) 83-99

    S t u d i e s i nE d u c a t i o n a lE v a l u a t i o n

    www.elsevier.com/stueduc

    A S Y S T E M A T I C M E T H O D O F S T A T I N G L E A R N I N G O B J E C T I V E S F O RA N A M B I T IO U S S C I E N C E P R O G R A M : T H E A P Q U A S C I E N C E P R O J E C T

    Sa m i ra E l B o u d a m o u s s i * , Ma g d a Me d i r * , R o b e r t G i l a b e r t * * a n d B o n i f a c i o J i m ( ~ n e z*

    *Facu l t y o f Educa t i on Sc ience s and Psycho logy , Un i ve rs i t y Rov i ra i V i rgi li , Ca ta lon ia , Spa in* *Depa r tmen t o f E lec t r i ca l an d Mech an ica l Eng ine e r i ng , U n i ve rs i t y Rov i ra i V i rg i li , Ca ta lonia , Spa in

    A b s t r a c t

    The educat ional goals and object ives provide a reference for curr iculum designers ,teachers , and educators , and they are requi red for evaluat ion or accredi ta t ionprocesses . This ar t ic le presents a systemat ic method to s ta te the learn ing object ivesand ana lyze the cons i s t ency o f a secondary schoo l sc i ence p rog ram deve loped inSpain . The method i s developed through seven s teps , s tar t ing wi th data col lec t ionand organizat ion , fo l lowed by induct ion and c ategorizat ion of the object ives , andfin ish ing wi th an in ternal consis tency analysis betw een s ta ted object ives and theprog ram 's learn ing act iv i ties and an external consis tency analysis betwe en theseobject ives and progra m designers and users ' perceptions.

    I n t r o d u c t i o nO v e r t h e l a s t t h r e e d e c a d e s , t e n d e n c i e s i n s c i e n c e t e a c h i n g a n d i n d e v e l o p m e n t a n d

    i m p l e m e n t a t i o n o f n e w e d u c a t i o n a l p r o g r a m s h a v e l e d to a n i n c r e a s i n g i n t e re s t i ne v a l u a t i o n r e s e a r c h a n d , m o r e s p e c i f i c a l l y , i n e d u c a t i o n a l e v a l u a t i o n . A l s o , t h er e s p o n s i b i l i t i e s a n d r e s o u r c e s a l l o c a t e d b y s o c i e t y t o e d u c a t o r s h a v e i n c r e a s e d t h e n e e d f o ra c c o u n t a b i l i t y .

    0191-491X/04/$ - see front matter 2006 Published by Elsevier Science Ltd.doi: 10.1016(i.stueduc.2006.04.002

  • 8/14/2019 Learning objectives: A systematic method to state objectives and analyse consistency of a science program

    2/17

    84 S. E1 Boudamou ssi et aL/ Studies' in Educat ional Evaluation 32 (2006) 83-99

    As par t o f evaluat ion processes , ob jec t ives genera lly need to be ident if ied and s ta ted(Tymitz-Wolf , 1982) and they are of ten essent ia l requi rements for accredi ta t ion (Micek,1979; Wit t ig , 1992) . Goals and objec t ives are a l so cons idered key e lements in des igninginstructional m odu les (Hashim , 1999). Therefore, i t is impo rtant for progra m s andinst itutions to have c learly defined goals and objec tives (Thier & Dav iss, 2002; Witt ig,1992). Def in ing goals and objec t ives is one o f the main tasks tha t he lps es tabl ish areference for des igners and def ine the m ethods tha t wi l l or wi l l not be used to achieve thosegoals (Thier & Daviss , 2002) . I t i s a l so a reference for teachers to he lp them plan coursesand assess s tudents ' progress and per formance (Bloom 1979; Stuf f lebeam & Webster ,2000; Tyler, 1998).Unfo r tuna te ly , t he p roces s o f i den t ify ing ou tcom es o r ob j ec t ives i nvo lves m anyproblems and d i f f icul t ies . As Micek (1979) ment ions , the " speci f ic in tended outcomes ofprograms are d i f f icul t to ident i fy and agree up on" . Fo r Miceck, "even i f a consensus i sreached" , there is "a pauci ty of adequate da ta col lec t ion procedures" and e ven " in caseswhere outcomes informat ion has been obta ined, major problems are of ten encountered inthe interpretat ion and use o f the information".Fur thermore , in prac t ice , most educat ional programs are based on contents tha t a restructured and developed in relat ion to a given discipl ine rather than a given goal . Also,som e o f these programs are d es igned, developed , and f ie ld- tes ted wi th s tudents.

    The present s tudy inves t iga tes the APQUA School Program 12-16, which i s par t ofthe APQUA Pro jec t (M ed i r & Abe l l6 , 1996) . APQUA, the Ca ta l an ac ronym fo r Learningabout Chemicals ', their Uses and Appl icat ions, has been implemented in Cata lonia s ince1989 as the adapta t ion of SEPUP (Science Educat ion for Publ ic Unders tanding Program) ,wh ich s tar ted in 198 1 a t the Lawren ce H al l of Science of the Un ivers i ty of Cal i fornia a tBe rke ley (Thier , 1985).

    The APQ UA P r o j ec tAPQUA is an educat ional pro jec t tha t develops mater ia l s for the communi ty andschools wi th the f inancial suppor t o f indus t ry as w el l as pr iva te and g overnm enta lfoundat ions . I t has been widely d i f fused in Spain , reaching more than 3000 teachers and

    150,000 p upi l s in about 900 schools , and more than 7 ,500 adul t s (APQ UA , 2004) .The m a in goa l o f AP QU A i s t o deve lop g rea t e r awareness , knowledge , andunders tanding about chem icals and the i r in terac t ion wi th peop le ' s l ives . The projec t in tendsto provide people w i th the knowled ge and tools tha t he lp them learn to obta in informat ionabout chemicals , to make ef fec t ive indiv idual decis ions , and to par t ic ipa te ac t ive ly asme mb ers o f a demo cra t ic socie ty . AP QU A, thus, a l so a ims to promo te the use of sc ient if icpr inc ip les , processes , and evidence in publ ic decis ion-making (Med ir & Abel l6 , 1996) .F igu re 1 shows the o rgan iza tion o f the A PQ UA pro jec t , wh ich cons i s ts o f t heAP Q UA Com mun i ty P rogram fo r adu lts , t he APQ UA Schoo l P rog ram 10-12 fo r p r imaryschoo l s and the APQ UA Schoo l P rog ram 12-16 fo r s econda ry schoo ls .

  • 8/14/2019 Learning objectives: A systematic method to state objectives and analyse consistency of a science program

    3/17

    S. E1Boudamo ussi et al./ Studies in Educat ional Evaluation 32 (2006) 83-99 8 5

    r o r a m &A P Q U A P R O J E C T

    . /PnblicProam >

    Figure 1: Structure of the AP QU A ProjectT h e A P Q U A Sc h o o l P r o g r a m 1 2 - 1 6 , w h i c h is th e s u b j e c t o f t h is s t u d y , i s m a d e u p

    o f m o d u l e s . E a c h m o d u l e c o n t a i n s s e v e r a l u n i ts , a n d e v e r y u n i t i n t e g r a te s s e v e r a l a c t i v it i e s.A l is t o f m o d u l e s i s i n c l u d e d i n T a b l e 1 (t h e o n e s s e l e c t e d f o r t h e p r e s e n t s t u d y a r e m a r k e dw i t h a n a st e ri s k ). A l l m o d u l e s d e a l w i t h t w o m a i n th e m e s : r i s k a n d c h e m i c a l s . T h e y m a yt a c k l e d i f f e r e n t c o n t e n t s a n d d i f f e r e n t s p e c i f i c o b j e c t i v e s , h o w e v e r , t h e y a l l a i m a ta c h i e v i n g th e g o a l s o f th e A P Q U A p r o j e c t ( A PQ U A , 2 0 0 3 ) . I n o r d e r t o d is t i n g u i s hb e t w e e n d i f f e r e n t l e v e l s o f le a r n i n g o b j e c t i v e s , w e w i l l u s e i n t h is s t u d y t h e w o r d s " g o a l s "f o r th e p r o j e c t , " g e n e r a l o b j e c t i v e s " f o r t h e p r o g r a m s o f t h e p r o j e c t , " s p e c i f i c o b j e c t i v e s "f o r th e m o d u l e s o f a p r o g r a m a n d " c o n c r e t e o b j e c t i v e s " f o r th e u n i t s o f a m o d u l e .Table 1 : Mod ules of the APQ UA School Program 12-16

    Modu les o f the AP QU A Schoo l Prog ram 12-16 Abbrev ia t ionSolutions and Pollution*Contam inat ion of Groundw ater in Vallf rondosa*Solid Waste Managem ent*Plastics in our Lives*W hat is a Chemical?Risk ComparisonHazardous Waste TreatmentTox ico logy

    D CC VG RP LPQR V

    T R EV U

    * M o d u l e s i n c l u d e d i n th e s a m p l e

    T h e m o d u l e s ' a c t i v i t i e s p r o p o s e s i m u l a t e d s i t u a t i o n s a n d e x p e r i m e n t s t o t e a c hc o n c e p t s a n d p r o c e d u r e s a s w e l l a s a t t i t u d e s , n o r m s , a n d v a l u e s . T h e y a l s o c o n t a i n o p e n -e n d e d q u e s t i o n s t o h e l p t e a c h e r s c o n d u c t d i s c u s s i o n s a n d e n h a n c e l e a r n e r s ' r e f l e c t i o n .S t u d e n t s m a k e d e c i s i o n s u s i n g t h e n e w k n o w l e d g e a n d c o n s i d e r i n g t h e t r a d e o f f s i n v o l v e dw i t h e a c h o p t i o n ( E l B o u d a m o u s s i , M e d i r , J i m i n~z , & G i l a b e r t , 2 0 0 1 ).

  • 8/14/2019 Learning objectives: A systematic method to state objectives and analyse consistency of a science program

    4/17

    8 6 S . E1 Bo u d a m o u s s i e t a l . / S t u d i e s i n Ed u c a t i o n a l Ev a l u a t i o n 3 2 ( 2 0 0 6 ) 8 3 - 9 9

    A l t h o u g h A P Q U A is d e s i g n e d a n d d e v e l o p e d f r o m c l e a r ly d e f i n e d g o al s , th e g e n e r a lob jec t ives of the AP Q U A School Program 12-16 , the spec i f ic ob jec t ives of i ts mod ules andthe concre te ob jec t ives of the un i ts were e i ther par tia l ly s ta ted or no t s ta ted as sh ow n inFigure 2 . This i s so because for adapt ing the SEP UP mater ia ls to the contex t o f Ca ta lans o c i e ty a n d s c h o o l s , th e A P Q U A P r o j e c t t e a m f ie l d -t e st s v a r io u s v e r s i o n s o f t h e A P Q U Amodules before a f inal one is edi ted and dis tr ibuted. Therefore, the learning act ivi t ies aremodi f ied many t imes and the spec i f ic and concre te ob jec t ives in i t i a l ly ta rge ted maybeco m e incomp le te , non-s t ruc tured or roughly s tated .

    A P Q U A P R O J E C T _ . ~ G o a l s o f t h e p r o j e c tc o m p l e t e l y s t a t e d~ h o o l P r o g r a m 1 2 - 1 6 ~

    . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

    r - - - M o d u l e s" . . . . . . . . . . . . . . . . . . . . . . . . . . Un i t s

    iI M odu le I . o b j e c t i v e s o f1 M o d u le 2 [ ] [ ] [ ] p a r ti a ll yi l iM o d u le 3 [ ] [ ] [ ] s t at e dII :

    ,' ~ Spec i f i c ob jec t i ve s o fi . . . . . . . . "~ . . . . . . . . . . . . . . . . mod u le s n o t s t a t e dI. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

    Genera l ob jec t i ve s o f theprogram not s tated

    Figure 2: Diagnosis of Objectives in the AP QU A School Program 12-16Research Purpose

    T h is s t u d y d e v e l o p s a s y st e m a t i c m e t h o d t o e v a lu a t e t he A P Q U A S c h o o l P r o g r a m12-16 by s tating the learning obje ct ives (o f the units , the mo dule s and the progra m ) an dana lyz ing the internal and ex te rna l co ns i s tency o f the program. The s tages o f thi s me thodare bu i l t up , s tep by s tep , fo l lowing a sys temat ic and s t ruc tured process , which can bedescr ibed as fo l lows:

    S ta tement o f the spec i f ic ob jec t ives of a represen ta tive sample of mod ules o f theA P Q U A S c h o o l P r o g r a m 1 2 - 1 6Sta tement o f the genera l ob jec t ives of the prog ram based o n the spec i f ic ob jec t iveso f a s a m p l e 's m o d u l e sAna lys i s o f the program's internal cons i s tency making sure tha t the ob jec t ives s tatedare actual ly l inked to learning act ivi t ies in the mod ules o f the program .Ana lys i s o f the program's ex te rna l cons i s tency by involv ing a sample o f users anddev elope rs in the pro cess o f s tat ing the learning objec t ives.

  • 8/14/2019 Learning objectives: A systematic method to state objectives and analyse consistency of a science program

    5/17

    S. E1 Boudamou ssi et aL/ Studies' in Educa tional Evaluation 32 (2006) 83-99 87

    A de ta i led descr ip t ion of the m ethod ' s deve lo pm ent wi l l be publ i shed in a fu turearticle.Program Evalua t ion

    The main go a l o f p rogram eva lua t ion i s to improve a new program or ac t iv i ty(Anderson , 1994) . However , there a re as many def in i t ions as au thors and models fo r th i sconcept (Cronbach , 2000; E isner , 2000; McDonald , 1989; Par le t t & Hami l ton , 1989;Scriven, 2000; Stake, 2000; Stu ff leb eam & We bster , 2000; Tyler , 2000). An analysiscar r ied ou t by E1 Boudamouss i (2002) def ines eva lua t ion as a p rocess o f co l lec t ingrelevant , val id and rel iable data , comparing i t with establ ished cr i ter ia to make evidence-base d dec is ions and , som et imes , a va lue judgm ent .

    The me thod o f s ta ting lea rn ing ob jec t ives for the A PQ U A School Program 12-16can be cons idered as an eva lua t ion process s ince i t t akes in to account a ll the aspec ts o f theabo ve m ention ed defini tion. The me thod is ba sed on col lect ing, organizing, analyzing, andres t ruc tur ing da ta , and a l though i t does no t inc lude com par ing in form at ion wi th es tab l i shedcriteria , i t uses an internal and external a nalysis o f cons is tency.

    M e t h o d o l o g yThe research me thodo logy fo l low ed cons is t s o f four s teps s ta ting the learn ingo b j e c t iv e s fo r th e A P Q U A S c h o o l P r o g r a m 1 2 -1 6 :Sample def in i tionInformat ion ga ther ingObjec t ives s ta tementInternal and external consis ten cy analyse s

    Samp le DefinitionThe par t icu la r s t ruc ture o f the A PQ U A School Program 12-16 , o rganized in tomodules and un i t s (F igure 2) has shaped , in genera l , the method deve loped and , in

    part icular , the sample select ion.Fou r of the e igh t ex is t ing mo dules w ere se lec ted : So lu t ions and Pol lu t ion ,Con tamina t ion o f Gro undw ater in Val l f rondosa , So l id W as te Managem ent , and P las t ics inour Lives (Table 1) . The c r i te r ia fo l lowed inc luded the top ic ' s re levance , the re f lec t ion ofthe program's educa t iona l approach , and the non-s imi la r i ty be tween lea rn ing ac t iv i t i es inthe se lec ted mo dules .

    The modules exc luded were : What i s a Chemica l? , Risk Compar i son , HazardousWaste Trea tment , and Toxico logy .Data Gathering

    The da ta w ere bas ica l ly ob ta ined us ing the fo l lowing procedure :

  • 8/14/2019 Learning objectives: A systematic method to state objectives and analyse consistency of a science program

    6/17

    88 S. E1Boudamo ussi et al./ Studies in Educa tional Evaluation 32 (2006) 83-99

    .

    .

    A na ly s i s o f documen t s p r ov ided by t he A PQ U A p r o j ec t t e am s uch a s t e ache r sguides , s tudent gu ides , b rochures , annual repor t s and o ther publ ica t ions on theproject .A na ly s i s o f documen t s p r od uced t h r oughou t t h is s t udy . Thes e d ocumen t s p r e s en t edmore and more s t ruc tured informat ion and became more and more usefu l as thew o r k p r og r e s s ed .

    Statement o f ObjectivesThe process of s ta ting the genera l ob jec t ives wa s the inver ted pa th o f the program

    design. I t s tar ted from l is t ing the learning act ivi t ies prese nted in the units o f the mod ules.Then, i t identif ied the educational contents (knowledge, abi l i t ies , values, and at t i tudes)tackled in these units and ind uced the conc rete objec t ives of the units an d the spe cif icobjec t ives of the m odules . F ina l ly , i t ended up a t induc ing the genera l ob jec t ives of theprogram (Figure 3) .

    L e a r n i n ga c t i v i t i e s

    C o n c r e t e S p e c i f i co n t e n t s o b j e c t i v e s o b j e c t i v e s

    G e n e r a lo b j e c t i v e

    S t a t e m e n t o f O b j e c t i v e s

    C o n s i s t e n c y A n a l y s i s

    Figure 3: The Process of Stating Learning Objectives for the Program

    The educa t iona l charac te r o f the program sugges ted a r ranging the educa t iona lconten ts accord ing to the c lass i fica t ion of the second ary sc ience cur r icu lum of Cata lonia(Depar tmen t o f Educa t ion , 1993) . In th is edu ca t iona l d is tr ic t, wh ere the A PQ U A prog ramis deve loped , th ree main types o f conten ts a re emphas ized:

    Fac ts , concepts and conceptua l sys temsPr oces s e sValues , a t t i tudes and norm sCons i s t ency A na ly s is

    The interna l and ex te rna l ana lyses o f cons is tency w ere car r ied out us ingspec i f ica t ion tab les (Roegiers , 2000) and exper t modera t ion (Thier & Daviss , 2001) ,respec t ive ly .

  • 8/14/2019 Learning objectives: A systematic method to state objectives and analyse consistency of a science program

    7/17

    ~ 5. L I H o u a a m o u s s t e t a l . / ~ 5 1 u a te s m L a u c a t t o n a l L v a l u a t t o n 3 Z ( Z U U O ) ~ 3 - !~ ! ~ 3 9

    T h e s p e c i f i c a t i o n t a b le s a r e g e n e r a l l y u s e d i n p r o g r a m d e s i g n t o h i g h l ig h t t h e l in k sb e t w e e n t h e g o a l s a n d t h e l e a r n i n g a c t i v i t ie s . I n t h is s t u d y , a s p e c i f i c a t i o n t a b l e w a s b u i l tf o r e a c h m o d u l e o f t h e s a m p l e i n o r d e r t o c h e c k t he c o h e r e n c e b e t w e e n e a c h s p e c i f i co b j e c t i v e s t a te d f o r t h e m o d u l e a n d t h e c o r r e s p o n d i n g l e a r n i n g a c t i v i t ie s th a t l e d t o s t a t in git . A n e x a m p l e o f a s p e c i f i c a t i o n t a bl e i s s h o w n i n T a b l e 2 .T a b l e 2 : E x a m p l e o f a S p e c i f i c a t i o n T a b l e

    C o n c re t e a n d c o m m o nSpeci f i c ob i ec t i ves o f un i t s Educa t i ona l con t en t s Learn ing ac t i v i t i es

    o b j e c t i v e s C o n c r e t e C o n c r e t eo b j e c t i v e s o b j e c t i v e s

    c o m m o n t o a l luni t s

    Concre t e Al l educa t i ona l Al l l earn ing ac t i v i t i es t ha tob j ec t i ve 1 con t en t s t ha t co r r espond t o educa t i ona l con t en to f u n i t 1 c o r r e s p o n d t o c o n c r e t e 1 . 1 . 1 n u n i t 1ob j ec t i ve 1 o f un i t 1 Al l l earn ing ac t i v i t i es t ha t

    co r r espond t o educa t i ona l con t en t1.1.2 in unit 1Etc.

    C o m m o n C o n c r e t eo b j e c t i v e 1 o b j e c t i v e 1

    o f u n i t 2A l l e d u c a t i o n a lcon t en t s t ha tco r r espond t o concre t eo b j e c t i v e 1 o f u n i t 2

    Al l l earn ing ac t i v i t i es tha tco r r espond t o educa t i ona l con t en t1.2.1 in unit 2Al l l earn ing ac t i v i t i es tha tco r r espond t o educa t i ona l con t en t1.2.2 in unit 2Etc.Etc. Etc. Etc.

    Spec i f i co b j e c t i v e C o m m o n1 ob j ec t i ve 2C o n c r e t e A l l e d u c a t i o n a lob j ec t i ve 2 con t en t s t ha to f un i t 1 co r r espond t o concre t e

    o b j e c t i v e 2 o f u n i t 1

    C o n c r e t e A l l e d u c a t i o n a lob j ec t i ve 2 con t en t s t ha to f un i t 2 co r r espond t o concre t eo b j e c t i v e 2 o f u n i t 2

    Al l l earn ing ac t i v i t i es tha tco r r espond t o educa t i ona l con t en t2.1.1 in un it 1Al l l earn ing ac t i v i t i es tha tco r r espond t o educa t i ona l con t en t2 .1 .2 in uni t 1Etc.Al l l earn ing ac t i v i t i es tha tco r r espond t o educa t i ona l con t en t2.2.1 in unit 2Al l l earn ing ac t i v i t i es tha tco r r espond t o educa t i ona l con t en t2 .2 .2 in uni t 2Etc.

    Etc. Etc. Etc.

  • 8/14/2019 Learning objectives: A systematic method to state objectives and analyse consistency of a science program

    8/17

    90 S. E1 Boudamou ssi et aL/ Studies' in Education al Evaluation 32 (2006) 83-99A genera l t ab le ( see , fo r an example , Tab le 3 ) , was bu i l t t o show how the gene ra l

    o b j e c ti v e s o f th e p r o g r a m we r e g e n e r a t e d f r o m t h e s p e c i f ic o b j e c t iv e s o f t h e s a mp l e ' smo d u l e s . Th i s g e n e r a l t a b l e , t o g e t h e r w i t h Ta b l e 2 , a l l o we d c h e c k i n g t h e c o h e r e n c ebe tw een ev e ry gene ra l ob jec t ive and the l ea rn ing ac t iv it i e s.Table 3: Example o f a General Table of Consistency

    General objectives Specific objectives o f mo dules (S.O)of the program(G.O) Modu le 1 Modu le 2 Modu le 3 Modu le 4G .O .I S.O I.l.1 S.O 1.2.1 S.O 1.3.1 S.O 1.4.1

    S.O 1.1.2 S.O 1.2.2 S.O 1.3.2 S.O 1.4.2Etc. Etc. Etc. Etc.

    G.O. II S.O II.l.1 S.O II.2.1 S.O II.3.1 S.O II.4.1S.O II.1.2 S.O II.2.2 S.O II.3.2 S.O II.4.2Etc. Etc. Etc. Etc.

    Th e e x p e r t mo d e r a t i o n i n v o l v e d 2 0 s e c o n d a r y s c h o o l t e a c h e r s a n d 6 p r o j e c t t e a mme mb e r s i n t h e p r o c e s s o f s t a t i n g t h e o b j e c t i v e s . Th e s e c o n d a r y s c h o o l t e a c h e r s we r ese lec ted accord ing to the i r expe r ience in us ing the p rogram, in gene ra l , and the modu les o ft h e s a mp l e , i n p a rt i cu l a r , a n d t h e p r o j e c t te a m me m b e r s a c c o r d i n g t o t h e i r in v o l v e me n t int h e p r o g r a m ' s d e v e l o p me n t , i n g e n e ra l , a n d t h e m o d u l e s o f th e s a mp l e , i n p a r ti c u la r . Th ee x p e r t s ' o p i n i o n s a n d s u g g e s t i o n s we r e o b t a i n e d t h r o u g h q u e s t i o n n a i r e s a n d i n t e r v i e ws .Five qu es t ionna i re s were des igned : fo u r o f the m were r e la ted to the spec i f i c ob jec t ives o ft h e mo d u l e s a n d o n e wa s r e l a t e d t o th e g e n e r a l o b j e c t iv e s o f t h e p r o g r a m. Th e e x p e r t s wh oansw ered a t lea s t one qu es t ionna i re were in te rv iewed .

    A g r o u p o f th r e e j u d g e s i n t e r v e n e d t h r o u g h o u t t h e wh o l e r e s e a rc h p r o c e s s i n o r d e rto eva lua te and seek consensu s fo r each o f the re su l t s ob ta ined .

    Re s u l t s a n d D i s c u s s io nTh e m e t h o d d e v e l o p e d ( s h o w n i n F i g u r e 4 ) c o n s i st s o f s e v e n s t ag e s , e ac h o f wh i c h

    h a s s e v e r a l st e p s a n d g e n e r a t e s v a r io u s v e r s i o n s o f t h e p r o g r a m ' s o b j e c ti v e s f o l l o wi n g asp i ra l p rocess . T he sev en s t ages o f the m e th od inc lud e da ta co l l ec tion ; da ta o rgan iza t ion ;ca tegor iza t ion ; in te rna l cons i s t ency ana lys i s (us ing spec i f i ca t ion t ab le s ) ; app l i ca t ion andt rans fe r; gene ra l i za t ion , and ex te rna l cons i s t ency ana lys i s (us ing expe r t mod era t ion ) .

  • 8/14/2019 Learning objectives: A systematic method to state objectives and analyse consistency of a science program

    9/17

    S . E 1 B o u d a m o u s s i e t a l . / S t u d i e s i n E d u c a t i o n a l E v a l u a t i o n 3 2 ( 2 0 0 6 ) 8 3 - 9 9 9 1

    S t a g e 1 . D a t a C o l l e c t i o nSt e p 1 .1 . Se l e ct a s a mpl e o f p rog ra m ' s mo dul e sS t e p 1 .2 . R e a d a nd a na l yz e doc um e nt sS t e p 1 .3 . L i s t l e a rn i ng a c t i v i t ie s fo r t he un i t s o f t he f i r s t modu l e o ft he s a mpl e (M 1)

    S t a g e 2 . D a t a O r g a n i z a t i o nSt e p 2 .1 . S t a te c on t e n t s fo r t he un i t s o fM 1S t e p 2 .2 . Induc e a nd s t a t e c onc re t e ob j e c t i ve s fo r t he un i t s o fM 1S t ep 2 . 3 . I d e n t i f y c o n c r e t e o b j e c ti v e s c o m m o n t o a l l u n i t s o f M 1S t e p 2 .4 . Induc e a nd s t a t e spe c i f i c ob j e c t i ve s fo r M 1

    4 ,S t a g e 3 . C a t e g o r i z a t i o nStep 3.1. Define the main categories o f s?ecific objectives statedfor M 1 and assemble them into these categories

    S t a g e 4 . I n t e r n a l C o n s i s t e n c y ( o f m o d u l e s )St e p 4 .1 . E nc o de t he l e a rn i ng a c t i v it i e s , t he c on t e n t s , the c onc re t ea n d c o m m o n o b j e c ti v e s o f t h e u n i ts a n d s p e c i fi co b j e c ti v e s o f M 1S t e p 4 .2 . B ui l d a spe c i f i c a t i on t a b l e fo rM 1S t e p 4 .3 . An a l yz e qua n t i t a t i ve l y a nd qua l i t a t ive l y thes p e c i f ic a t io n t a b l e o f M 1S t e p 4 . 4 . R e s t a t e t h e s p e c i f i c o b j e c t i v e s o f M 1

    S t a g e 5 . A p p l i c a t i o n a n d T r a n s f e rSt e p 5 .1 . Ap pl y s t a ge s 1 a nd 2 a nd t r a ns fe r r e l e va n t r e su l t s t o o t he r

    m o d u l e s o f t h e s a m p l eS t ep 5 . 2 . T r a n s f e r ca t e g o r ie s a n d e n c o d i n g s y s t e m d e f i n e d f o r M 1

    t o o t h e r m o d u l e s o f t h e s a m p l eS t ep 5 .3 . A p p l y sta~e 3 a n d s t a~e 4 t o o t h e r m o d u l e s o f th e s a m p l e

    4 ,S t a g e 6 . G e n e r a l i z a t i o nSt e p 6 .1 . Ide n t i fy spe c i f ic ob j e c t ive s c omm on t o a l l t he mod ul e s o f

    t he s a mpl eS t ep 6 . 2 . I n d u c e g e n e r a l o b j e c ti v e s f o r th e p r o g r a m f r o m t h es p e c i fi c o b j e ct i v e s o f t h e s a m p l e ' s m o d u l e s ( I n d u c ti o n )S t e p 6 .3 . Ar r a nge i nduc e d ob j e c t i ve s in t o syn t he s i z e d ob j e c t i ve s(Synt he s i s )S t e p 6 .4 . S t a t e ge ne ra l ob j e c t i ve s fo r t he p rog ra m ba se d onsyn t he s i z e d ob j e c t i ve s (Ve rs i on 1 )S t e p 6 .5 . E x t ra po l a t e s t a g e 4 " In t e rna l C ons i s t e nc y" t o a na l yz e t heg e n e r a l i n te r n a l c o n s i s t e n c y o f t h e p r o g r a m

    S t a g e 7 . E x t e r n a l C o n s i s t e n c y ( u s i n g E x p e r t M o d e r a t i o n )S t ep 7 . 1 . D e f i n e a s a m p l e o f m o d e r a t io n e x p e r t sS t e p 7 .2 . De s i gn a nd a pp l y e xpe r t mod e ra t i on ins t rume n t sS t e p 7 .3 . An a l yz e e xpe r t mode ra t i on r e su l tsS tep 7 .4 . R es ta te t i l e s p ec i f i c ob ject ives o f t i l e s amp le ' s mod u lesa n d t h e g e n e r a l o b j e c t i v e s o f t h e p r o g r a m

    ~-[ Lea rnin g ac t ivi t i es 1r r [ V e r s i o n 2 i iI I , . . . . . . . . * . . . . . . ~ e ; s i ; ~ i -

    I E duc a t i ona l c on t e n t s. . . . . . . . . . . II : * I. .. .. .. .. .. ' " ' " 'i ' " " C . . . . t e a nd . . . . . . I II I t 1 ob j e c t i ve s o f un i t s I~ _ _ 4 _ _ _ _ 1 _ . . . . . . i I

    . . . . . . . . . . iI ~ - . . . . . . . . . . . . . . . . . . . . i! i Version : i I

    " . . . L . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . ~ IL 2 __ "~ - - V e r s i o n 3 i . f fI

    IJ

    I

    V e r s i o n 4

    I . . . . . . . , ; . . . . . . . . ./ t I o . . . . . lo b je c t i . . . f : ,l ,_ L i e . . . . . . . . . . . . . - _ _ j , ,

    F i g u r e 4 : M e t h o d o f S t a t i n g L e a r n i n g O b j e c t i v e s f o r t h e A P Q U A S c h o o l P r o g r a m 1 2 - 1 6

  • 8/14/2019 Learning objectives: A systematic method to state objectives and analyse consistency of a science program

    10/17

    92 S. E1 Boudamoussi et aL/ Studies' in Educational Evaluation 32 (2006) 83-99

    The f irst s tage , Data Collection (S tage 1 i n F i gur e 4 ) , i nc l ude s thr e e s te ps whi c hc on s i s t o f s e l e c t i ng a r e pre sentati ve sam pl e o f m od ul e s ( i nd i c ate d on Ta bl e l ) , r e ad ing andanal yz i n g d oc um e nts o f the progr am , and l i s t ing the l e arn i ng ac t i v i t i e s for the f ir s t sam pl e ' sm o d u l e ( M 1 ) . A n e x a m p l e o f l ea r ni ng a c t iv i t ie s i s s h o w n i n T a bl e 4 .T a b l e 4 : E x a m p l e s o f L e a r n i n g A c t i v i t i e s L i s t e d f o r Module M1E x a m p l e s o f le a r n i n g a c t iv i t ie s f r o m m o d u l e M 1T e a c h e r e x p l a i n s t h e i m p o r t a n c e o f w a t e r a s a s o l v e n t.S t u d e n t s r e a d t h e worksheet.T e a c h e r a s k s s t u d e n t s w h e t h e r c o m m e r c i a l a m m o n i a i s a n a c i d o r b a s e ; t h e s t u d e n ts answer.S t u d e n t s w h o p l a y t h e r o l e o f p r e s e n te r s i n t h e s i m u l a t io n o f a p u b l i c m e e t i n g m a k e t h e i r p r e s e n t a ti o n s .

    T h e s e c o n d s t a g e , Data Organization (S tage 2 i n F i gur e 4 ) , c on s i s t i ng o f four s te ps ,l e ads to s ta t ing the fi rs t ve r s i o n o f the spe c i f i c obje c t i ve s for m odu l e M1 (Tabl e 8 ) . Th i ss tage s tar t s by i de nt i fy i ng the e d uc at i ona l c onte nts tac k l e d i n the l e ar n ing ac t i v i t ie s (ane x a m p l e i s s h o w n i n T a b le 5 ) a n d i n d u c e s f r o m t h e m t h e co n cr et e o b j e c t i v e s o f t h e u n i t s(an e xam pl e i s shown i n Tabl e 6 ) . The n , i t i de nt i f i e s the c onc r e te obje c t i ve s c om m on to a l lu n i t s ( a n e x a m p l e i s s h o w n i n T a b l e 7 ) a n d , f i n a l l y , b a s e d o n t h e c o n c r e t e a n d c o m m o nobje c t i ve s , S tage 2 ge ne r ate s the f ir s t ve r s i o n o f the spe c i f i c ob je c t i ve s for m odu l e M1 (ane x a m p l e i s s h o w n i n T a b le 8 ) .

    The c onc re te obje c t i ve s c om m on to a ll un i t s ar e ve r y i m por tant to de c i de wh i c h o fthe c oncr e te obje c t i ve s o f the un i t s w i l l c ontr ibute to the s ta te m e nt o f the spe c i f i co b j e c t i v e s o f th e m o d u l e . H o w e v e r , t he s t a t e m e n t o f a s p e c i f i c o b j e c t iv e r a t h e r u s e s t h ei n f o r m a t i o n p r o v i d e d i n t h e c o n c re t e o b j e c t i v e s o f t he u n i t s. A s s h o w n i n T a b l e 7 , o n l y p a r to f t h is i n f o r m a t i o n i s p r es e n t i n th e c o m m o n o b j e c t iv e s .

    The f ir s t ve r s i o n o f the e duc at i ona l conte nts and c oncr e te and c om m on ob je c t i ve s o fm od ul e M1 i s ge ne r ate d a t th i s S tage 2 (F i gur e 4 ) .T a b l e 5 : E x a m p l e o f a n E d u c a t i o n a l C o n t e n t R e s u l t i n g f r o m a Set of Learning A c t i v i t i e s i n

    Module M1

    E x a m p l e o f a s e t o f l e a rn i n g a c t i v it i e s R e s u l t i n g e d u c a t i o n a lc o n t e n t

    A student reads' the introduction in a worksheetA student reads' the experimentalprocedure described in worksheet.In groups of two, students' carry out the experiments' described in theworksheet and they record their observations in a table.The students' design and carry out new tests using various'combinations of water solutions.

    Reading and interpreting aworksheet, and describing anexperimental procedure.

  • 8/14/2019 Learning objectives: A systematic method to state objectives and analyse consistency of a science program

    11/17

    T a b l e 6 :

    S. E1 Boudamoussi et aL/ Studies in Educational Evaluation 32 (2006) 83-99 93

    E x a m p l e o f a C o n c r e t e O b j e c t i v e R e s u l t i n g f r o m a S e t o f E d u c a t i o n a l C o n t e n t s i nM o d u l e M 1

    E x a m p l e o f a s e t o f c o n t e n t s R e s u l t i n g c o n c r e t e o b i e c t i v eSolution, solvent and solutePreparing solutionsPreparing a diluted solution f?om a concentrated solution and vice versaPreparing a saturated solution f?om a concentrated solution

    To build an operationaldefinition o f solution, solventand so lute

    T a b l e 7 : E x a m p l e o f a C o m m o n O b j e c t i v e R e s u lt i n g fr o m R e c u r r in g C o n c r e t e O b j e c t i v e s i nM o d u l e M 1

    E x a m p l e o f r e c u r ri n g c o n c r e t e o b j e c t i v e s R e s u l t i n g c o m m o n o b j e c t i v eUn i t 1 : To record experimental data obtained f?om observing theexperiments' of dissolving diffbrent substances in water.U n i t 2 : To observe and record quantitative and qualitative data aboutthe concentration and color o f every solution obtainedJ?om successivedilutions' of a colored solution with a known concentration.U n i t 3 : To observe and record, in a table, exper imental data obtainedf?om a systematic observation ofan indicator's' behavior with differentsolutions and their mixtures'.U n i t 4 : To observe and record, in a table, exper imental data obtainedf?om observing the colors' of an indicator in the solutions obtainedf?om successive dilutions ofan acid or base.U n i t 5 : To observe the reproducibility of experimental results' byiterating the experiment ofneutralizing an acid or basic solution.U n i t 6 : To observe and record the results' obtained f?om neutralizingan acid "A" with a base "B" as wel l as' with ammonia.Unit 6: To observe and record the results obtained f?om neutralizing abase "B" with an acid "A" as' wel l as' with vinegar.U n i t 7 : To observe and experience the fbrm ation of a salt when an acid(or base) is' neutralized with a base (or an acid).

    To observe and recordexperimental data.

    T a b l e 8 : E x a m p l e o f a S p e c i f i c O b j e c t i v e o f M o d u l e M 1 R e s u l t i n g f ro m a S e t o f C o n c r e t eO b j e c t i v e s o f U n i t s

    E x a m p l e o f a s e t o f c o n c r e te o b j e c t i v e s o f u n it sTo communicate the results' obtained f?om theobservation of an indicator's' behavior with differentsolutions and mixtures o f acids and bases.To express and accept opinions on using dilution tosolve the problem of water pollution with acids' orbases.To communicate and discuss the results of the universalindicator's, behavior with successive dilutions ofan acidor base.To communicate and discuss the results' obtained f?omthe neutralization ofan acid or basic solution.To express and accept opinions on using neutralizationto solve the probl em of water pollution with acids' orbases.

    R e s u l t i n g s p e c i f i c o b i e c t i v e o f m o d u l e M 1

    To communicate and discuss the results'obtained f?om using successive dilutionsand acid-base neutralization to solve theproblem of water pollution with acids' orbases.

  • 8/14/2019 Learning objectives: A systematic method to state objectives and analyse consistency of a science program

    12/17

    94 S. E1 Boudamou ssi et aL/ Studies' in Educat ional Evaluation 32 (2006) 83-99

    The third stage, Categorization (Stage 3 in Figure 4), clusters the specific objectivesof module M1 into main categories (Table 9). This categorization leads to the secondversion of the specific objectives for this module.Table 9: Categories nd Examples of Specific Objectives in Module M1I . C onc e p t sE . g .: T o b u i l d o p e r a t i o n a l d e f i n i t i o n s o f b a s i c c o n c e p t s r e l a t e d t o s o l u t i o n s a n d n e u t r a l i z a t i o n .I I. S c i e n t i f i c m e t h o d o l o g y , i t s t r a n s f e r a n d a p p l i c a t i o n to r e a l li f e a n d d a i ly i s s u e sE . g .: T o o b s e r v e a n d r e g i s t er , i n a n o r g a n i z e d a n d s y s t e m a t i c w a y , q u a n t i t a t i v e a n d q u a l i t a t i v e d a t ac o n c e r n i n g t h e c o l o r a n d c o n c e n t r a t i o n o f p r e p a r e d s o l u t i o n s , o r s o l u t i o n s o b t a i n e d f r o m s u c c e s s i v ed i l u t i o n s a n d f r o m a c i d - b a s e n e u t r a l i z a t io n .I II . A w a r e n e s s a n d r e s p o n s i b i l i t yE . g .: T o b e a w a r e o f c l e a n i n g d i f f i c u l ti e s r e la t e d t o c o n t a m i n a t e d - n a t u r a l - w a t e r - r e s o u r c e .

    The fourth stage, Internal Consistency (Stage 4 in Figure 4), analyzes theconsistency between the specific objectives stated for module M1 and its learningactivities. A specification table, already illustrated in the methodology in Table 2,establishes the links between each specific objective and the corresponding learningactivities. This stage results in a third version of the specific objectives of module M1.The fifth stage, Application and TransJer (Stage 5 in Figure 4), applies the fourprevious stages to the other sample's modules (M2, M3, and M4) and transfers some of therelevant results to these modules. A list of the learning activities, and a first version of theeducational contents, concrete and common objectives and specific objectives of thesemodules are obtained. The categories for the specific objectives are transferred frommodule M1 as well as the form of the statements used to express the concrete and commonobjectives and the specific objectives.

    The sixth stage, Generalization (Stage 6 in Figure 4), states the general objectives ofthe program based on the specific objectives of the sample's modules. It starts byidentifying the specific objectives that are similar or common to all the sample's modulesand induces from them a draft of induced general objectives (see statements inside thecircles on Figure 5). These are then assembled into categories of synthesized objectives(see statements inside the rectangles on Figure 5) that lead to the statement of a first versionof the general objectives of the program.

  • 8/14/2019 Learning objectives: A systematic method to state objectives and analyse consistency of a science program

    13/17

    S. E1Bouda moussi et aL/ Studies' in Educat ional Evaluation 32 (2006) 83-99 9 5

    < GI . 1 To b u i ld o p e r a t i o n a l d e f in i t i o n s o f b a s i c OG. I . 1 To b u i ldco n ce p t s r e l a t ed t o ch em ica l s , t h e i r p r o p e r t i e s , o p e r a t i o n a l d e f in i t i o n sa p p l i c a t io n s a n d t h e i r i n t e ra c t i o n s w i t h p e o p l ea n d t h e e n v i r o n m e n t a s w e l l a s t h e r i s k th e yDo se f o r b o th .~ ~ G .II.I T o i n t e r p ~sh ee t an d to ca r r y o u t an ~e r im e n t a l p . . . . d u r e ~

    + I / O G S H ~ T o e m p h a s i z e t h e a c c u ra t e u s e o f I +OG. I I .6 To wo r k / ex p e r im en ta l m a te r i a l an d r e sp ec t f o r h o w i t OG. I I .1 To ca r r y o u ti n a g r o u p \ s l~ o u ld b e u s e d a n d m a i n t a i n e d t o c a r r y o u t e x p e r i m e n t a l. . . . . i m e n ts I p . . . . d . . .

    ~ t t it u d e t o w a r d s t h e r o l e s a s s u m e d O G . II .6 T o w o r k i n a g r o u p /\ w i th in a g r o u p ~ /~ a n d d e b t ~ o l g S v ie n t i t ic ~ J O G . I I. 4 + T o d e v e l o pQ ~ ~ ' i e ~ f i c I . . . . . . ic a t i o n ab ilities

    + 1 ~ T o b e c o ~O G I I 7 T o d e v e lo p". : . . [ ~ f / o f t he p ro bl em s g . . . . te d b y w a ste , t he " ~ +c r lt lc a lt h ln k ln g r / d ~ e s V I 2 d e c - n t a m i n ~ t ~ n d-t he ~n ee d I O G .II I.3 T o b . . . . . . . . . .I ( f o r i n f o r m a t i o n I o f t h e e n v i r o n m e n t a l i m p a c ti ~ o f h u m a n a c t i v it i e s/ ~ x , .O G \ II I. 3 T o b . . . . . . . . . . . f t h e ]~ , . . . . . . ~ e n v i r o n m e n t a l i m p a c t o f h u m a n a c ti v it ie s I

    / ~ . / U . l l . / 1 o a c q u i r e a N/ critica l t h in k in g ~ ~ n ~T o o b s e r v e a n d r e c o r d i+ / v _ ..L~ . . . . . . . N a n o r g a n i s e d a n d s y s t e m a t i c w a yI / e v id e n c e -b a s e d ~ . . . . ' IO G . II .1 0 T o m a k e ~ d e c i s i o n s " , ~ u a n t l t a t l v e a n d q u a l i ta t i v e d a t a Ie v i d e n c e - b a s e d d e c i s i o n s ] O G I I .1 1 T o i n t e g r a te i n f o r m a t i o n a b o u t ~ /O G I I 3 T o a n a l y z e a n d] d i f f e r en t o p t io n s i n an ev id en ce -l~ ased \ ' . . /i n t e rp r e t e x p e r i m e n t a l] d e c i s i o n m a k i n g p r o c e s s / /~ ~ d a t a a n d g r a p h i c s /

    \ O G I I I. 7 T o e m p h a s i z e t h e a d w a nt ag . . . d / /~ d i s a d v an t a g e s , t h e i m p o r t . . . . f t h . . . . . i c /~ ' ~ c t . . . . d t h e .t ra d e -o f fs . i . . . . i d . . . . b a s e d / . /~ m a k in g p r o ~ ~

    Figure 5: Assembling the Induced General Objectives into Synthesized Objectives

    A s y n t h e s i z e d o b j e c t i v e c o n s i s t s o f a s h o r t s e n t e n c e t h a t r e f e r s t o a g r o u p o f g e n e r a li n d u c e d o b j e c t i v e s b u t w h i c h d o e s n o t p r o v i d e a l l t h e i n f o r m a t i o n c o n t a i n e d i n t h e s eo b j e c t i v e s o r in th e s p e c i f i c o b j e c t i v e s o f t h e m o d u l e s . T h e s y n t h e s i z e d o b j e c t i v e s t u r n e d

  • 8/14/2019 Learning objectives: A systematic method to state objectives and analyse consistency of a science program

    14/17

    96 X E1 Boudamoussi et aL/ Studies' in Educational Evaluation 32 (2006) 83-99out to be a helpful step in the procedure of stating the general objectives of the program.They offered an efficient way o f gaining an idea about how the general objectives wouldlook and served as a basis on which the statement of each general objective was built.The second version of the general objectives is obtained after extrapolating, to theprogram, the internal analysis of consistency (step 6.5) previously applied to the sample'smodules. This procedure, which correlates the general objectives stated in the first versionto the learning activities o f the modules, via the specific objectives, has already beenillustrated in Tables 2 and 3.

    Finally, the seventh and last stage, External consistency (using expert moderation)(Stage 7 in Figure 4), considers the opinions of two groups of experts about the last statedversions of the specific and general objectives. These experts are the users and developersof the program. The qualification of the objectives by the experts from the two groups aswell as their comments and suggestions lead to stating the fourth and thus final version ofthe specific objectives of the modules, and the third and thus last version of the generalobjectives of the program.

    ConclusionsWe developed a systematic method of stating learning objectives for the APQUASchool Program 12-16. For its development, this method takes into account both themodular structure of the program and the basic aspects o f an evaluation process.The method consists of seven stages, five of them applied to the individual modules,

    one to the whole program, and one applied to both of them. The stages applied to theindividual modules are data collection, organization, and categorization as well as aninternal consistency analysis. A generalization stage is applied to all the sample's modulesand an external consistency analysis is applied to both the program and the sample'smodules.Several versions of specific objectives of the modules and general objectives of theprogram are generated throughout the method. The definitive versions are the onesobtained from the external consistency analysis after applying the expert moderation steps.

    The method produces four versions of the specific objectives of the modules andthree versions of the general objectives of the program. The final version always resultsfrom the external analysis of consistency.The systematic method of stating learning objectives is mainly based on the modularstructure of the APQUA School Program 12-16 and the possibility of inducing its generalobjectives from the specific objectives of a sample's modules. Thus, the method could beapplied to any program that has a modular structure with educational contents (concepts,skills and attitudes) repeated in more than one module.

    ReferencesAnderson, G. (1994).Fundamentals' of educational research. [3rd Ed.] Basingstoke:Falmer.APQUA (2003). Proyecto APQUA. Aprendizqje de los Productos Quimicos, sus Usos y

    Aplicaciones [APQUA project. Learningabout chemicals, heir uses and applications]. Universitat Rovira iVirgili.

  • 8/14/2019 Learning objectives: A systematic method to state objectives and analyse consistency of a science program

    15/17

    S. E1 Boudamoussi et aL/ Studies' in Educational Evaluation 32 (2006) 83-99 97A P Q U A ( 2004) . Proyecto APQUA. Injbrme 2004 [ A P Q U A p r o j e c t. R e por t 2004 ] . T a r r a gona .

    D e pa r t a me n t o de I nge n i e r i a Q u i mi c a . U n i ve r s i t a t R ov i r a i V i r g il i .B loom, B .S . (1979) . Taxonomia de los ol)jetivos de la educaci6n. Clasificaci6n de las Metas

    Educativas [ T a xonomy o f e duc a t i ona l ob j e c t i ve s . T he c l a s s i f i c a t i on o f e duc a t i ona l goa l s ] . ( A c a r r e t aArne do, I . , Trans . 3rd ed . ). A lcoy : Marf i l .

    Cronb ach, L.J. (2000) . C ourse impro vem ent through eva lua t ion . In G.F . Mad aus , M.S . Scr iven , &D.L. Stu ffleb eam (Eds.) , Evaluation models'. Viewpoints' on educational and human services evaluation(13th ed . , pp . 101-115). Bos ton: Kluw er-Ni jhoff .

    D e pa r t me n t o f E du c a t i on o f C a t a l on i a ( 1993) . Curriculum. Educaci6 Secundhria Obligat6ria. ~ireade ci~ncies experimentals [ C ur r i cu l um. C om pu l s o r y s e conda r y e duc a ti on . S e c t i on o f na t u r a l sc i enc e s ].Barce lona , : Gene ra l i t a t de Ca ta luny a .

    Ei sner , E .W. (2000) . Educa t iona l connois seurship and c r i t i c i sm: the i r form and func t ions ineduca t iona l eva lua t ion , In G.F . M adaus , M.S . Scr iven, & D.L. S tuf f l ebeam (Eds .) . Evaluation models'.Viewpoints' on educational and human services evaluation (13th ed ., pp . 335-347) . Bos ton: Kluw er-Ni jhof t :

    E1 Bou dam ouss i , S . , Medi r , M . , J imin6z , B ., Gi laber t , R .M. (2001) . Estudio comparativo entre elmodelo te6rico de un programa educativo de ciencias y su desarrollo en el aula [ C ur r ic u l um. C om pu l s o r ys e c onda r y e duc a t ion . S e c t i on o f na t u r a l s c ie nc e s ]. V I C ongr e s o I n t e ma c i on a l sob r e I nve s t i ga c i 6n e n l aDidfict ica de l as C ienc ias : Re tos de l a En sef ianza de l as C ienc ias en e l S ig lo X XI . V 1 , (pp . 229-230) .Barce lona .

    E1 Boudamouss i , S . (2002) . Evaluation des ol)jectijg, du Programme APQUA Scolaire 12-16.Formulation et analyse de coherence [ E va l ua t ion o f the ob j e c t i ve s o f the A P Q U A S c hoo l P r og r a m 12 -16 .S ta temen t and coherence ana lys i s ] . Unp ubl i sh ed Ph.D. thes i s , Tar ragona : U nivers i t a t Rov i ra i Vi rg i li .R e t r i e ve d f r om:

    H a s h i m, Y . ( 1999) . A r e i n s t r uc t iona l de s i gn e l e me n t s be i ng u s e d i n m odu l e w r i t ing? British Journalof Educationa l Technology. 30 (4), 341-358.

    M cDo nald , B . (1989) . L a eva luac i6n y e l cont ro l de l a educa c i6n [Eva lu a t ion and the cont ro l ofeduca t ion] . In S . J . Gimeno & G. Per6z La enseYmnza. Su teoria y su practica [Teaching. Its' theory andpractice] (3rd ed ., pp . 467-478) . M adr id : Aka l .

    Medi r , M . , & Ab el l6 , M. (1996) . APQUA: A program on sciences' and societal issues. P r oc e e d i ngso f t he 2nd s c i e n t i fi c c on f e r e nc e on : " T he F u t u r e o f S c i e nc e a nd M a t he ma t i cs T e a c h i ng a nd t he N e e ds o fArab Soc ie ty" (pp 500-506) . Tun i s : Arab D eve lopm ent Ins t i tu te .

    M i c e k S .S . ( 1979) . I de n t i f y i ng , me a s u r i ng a nd e va l ua t i ng e duc a t i ona l ou tc ome s . North CentralAssociation Quarterly, 53 (4) , 408-419.

    Par le tt , M. , & H ami l ton , D . (1989). La eva lu ac idn como i lum inac idn [Eva lua t ion as enl ighten me nt ] .I n S . J . G i me no & G . A . P 6 re z La ense~anza. Su teo riay suprac tica [Teaching. I t s theory and prac t i ce ] (pp.450-466) . Madr id : Aka l .

  • 8/14/2019 Learning objectives: A systematic method to state objectives and analyse consistency of a science program

    16/17

    98 S. E1 Boudamoussi et aL/ Studies' in Educational Evaluation 32 (2006) 83-99Roegiers , X. (2000). Une p~dagogie de l'int~gration. Comp~tences et integration des acquis dans

    l'enseignement [ A pe da gogy o f i n t e g r a t i on . C ompe t e nc e s a nd i n t e g r a t i on o f know l e dge i n t e a c h i ng ] .Brusse l s : De Boeck Univers i t6 .

    Scr iven, M.S . (2000) . E va lua t io n ideologies . In G.F . M adaus , M.S . Scr iven, & D.L. S tuf f l ebe am(Eds.). Evaluation models'. Viewpoints' on educational and human services evaluation (13th ed. , pp. 229-260) . Bos ton: Kluwer-Ni jhofI :

    S take , R .E. (2000). P rogram ev a lua t ion , par t i cula r ly respo ns ive eva lua t ion . In G.F . Mad aus , M.S .Scr iven, & D .L. S tuf f l ebeam (Eds .) . Evaluation models'. Viewpoints' on educational and human services'evaluation (13th ed . , pp . 287-310, ) Bo s ton: Kluw er-Ni jhofI :

    S tuf f l ebeam , D.L., & W ebs te r , W. J . (2000) . A n ana ly s i s of a l t e rna t ive approaches to eva lua t ion , InG.F . Ma daus , M.S . Scr iven, & D.L. S tuf f l ebeam (Eds .) . Evaluation models'. Viewpoints' on educationaland human services' evaluation (13th ed ., pp . 23-43) . Bos ton : Kluwer-N i jhoff .

    Thie r , Herber t D . (1985) . So c ie ta l i s sues and concerns : A new em phas i s for s c ience educa t ion .Science and Education, 69 (20), 155-162.

    Thier, H.D., & Daviss , B. (2001). Developing inquiry-based science materials'. Guide jb r educators'.A pr o j e c t f o r t he L a w r e nc e H a l l o f S c i e nc e C e n t e r f o r C ur r i c u l um I nnova t i on . N e w Y or k : T e a c he r s C o l l e gePress.

    Tyler, R.W. (1998). Principios bdsicos del curriculo [ B a si c p r i nc i p l e s o f t he c u r r i c u lum] . B ue nosAi res : Troque l .

    Tyle r , R .W. (2000) . A ra t iona le for program eva lua t ion . In G.F . Madaus , M.S . Scr iven, & D.L.S tuf f l ebeam (Eds. ). Evaluation models'. Viewpoints' on educational and human services evaluation (13thed., pp . 67-78) . Bos ton: Kluw er-Ni jhoff .

    T ym i t z - W ol f , B . ( 1982) . G u i de l i ne s fo r a s s e s s i ng i nd i v i dua l i z e d e duc a t i on p r og r a m g oa l s a ndobjec t ives . Teaching Exceptional Children, 14 (5), 198-201.

    w i t t i g , G . R . ( 1992). M a k i ng u s e o f goa l s a nd ob j e c t i ve s f o r i n t e r na l p r og r a m e va l ua t ion . Journal ojEducation jb r Library and Injbrmatio n Science. 33 (2), 129-140.

    A c k n o w l e d g e m e n t sTo the mem ory of the l a t e Dr. Boni fac io J im6ne z who m ade a grea t cont r ibut ion to th i s s tudy. The

    a u t ho r s w ou l d mor e ove r l i ke t o t ha nk t he A P Q U A p r o j e c t ' s me mbe r s , t he S E P U P me mbe r s , a nd t heteachers who answ ered the ques t ionn a i res and par t i c ipa ted to the in te rv iews . They a l so thank Dr . Herber tT h i e f a nd M a r l e ne T h i e r f o r t he i r ma n y a nd r e l e va n t s ugge s t ions t o a n e a r l ie r ve r s i on o f t he pa pe r .

    T h e A u t h o r sS A M I R A E L B O U D A M O U S S I i s a p o s td o c r e se a r c h e r at t he F a c u l t y o f E d u c a t i o n a n dP s y c h o l o g y o f t h e U n i v e r s i t y R o v i r a i V i r g i l i in T a r r a g o n a , S p a i n . S h e i s a c h e m i c a l

  • 8/14/2019 Learning objectives: A systematic method to state objectives and analyse consistency of a science program

    17/17

    S. E1 Boudamou ssi et aL/ Studies' in Education al Evaluation 32 (2006) 83-99 99

    engineer f rom the Nat iona l School o f M ines and Indus t ry a t Raba t (Moroc co) and ga ined aPh .D. wi th the Europea n Labe l o f the Univers i ty Rovi ra i Vi rg il i.M A G D A M E D I R i s a p r o f e s s o r at t h e D e p a r t m e n t o f C h e m i c a l E n g i n ee r in g , m e m b e r o ft h e S c h o o l o f C h e m i c a l E n g i n e er in g a n d o f th e F a c u l t y o f E d u c a t i o n a n d P s y c h o l o g y ,Universi ty Rovira i Virgi l i a t Tarragona, Catalonia, Spain. She has been the Director , s ince1988, o f the APQ U A Pro jec t "Learn ing ab out Chem ica ls , it s Use s and Appl ica tions" of theChem ica l Engineer ing D epar tment o f the Un ivers i ty Ro vi ra i Vi rg i li , and sc ien ti f icco l labora tor in the SEPUP Pro jec t (Sc ience Educa t ion for Publ ic Unders tanding Program)of the Law rence H al l o f Sc ience , Un ivers i ty o f Ca l iforn ia a t Berke ley , USA . He r main f ie ldof research i s the deve lop men t and assessment o f sc ience and technolog y educa t iona lprograms for the schools and the comm uni ty .R O B E R T M . G I L A B E R T is a p r o f e s s o r a t t h e D e p a r tm e n t o f M e c h a n i c a l E n g i n ee r in g ,m e m b e r o f t h e S c h o o l o f C h e m i c a l E n g i n e e ri n g o f th e U n i v e r s it y R o v i r a i V i rg i li a tTarragona, Catalonia, Spain. H e is a lso a m em be r and scient if ic col labora tor o f theAPQUA Pro jec t "Learn ing about Chemica ls , i t s Uses and Appl ica t ions" , and a researcherin the assessm ent o f sc ience and techn ology educa t iona l p rograms for the schools and thec o m m u n i t y .B O N I F A C I O J I M E N E Z w a s a p r o f e ss o r at th e D e p a r tm e n t o f P e d a g o g y o f t he F a c u l t y o fEduc a t ion and P sycho logy , Univers i ty Rov i ra i Vi rg i li a t Tarragona, Ca ta lon ia , Spa in .Au thor and co-au tho r o f var ious book s and publ ica t ions on educa t iona l eva lua t ion ;cur r icu lum des ign , deve lopm ent , and eva lua t ion ; and educ a t iona l innova tion . Dr . Bon i fac ioJ im6nez d ied in Aug us t 2003 .Correspondence: