Sur

vey

afte

r Res

earc

h M

etho

ds o

n Com

pute

r Sci

ence

Sem

inar

1/5

0.

Resp

on

se r

ate

20 o

ut o

f 23

ans

wer

ed

1.

Ho

w lo

ng

are

yo

u a

lread

y w

ork

ing

on

yo

ur

Ph

.D.

?

Mos

t w

ere

begi

nner

s o

bvio

us c

orre

lation

bet

wee

n tim

e an

d ap

prec

iation

2.

Wh

at'

s yo

ur

ove

rall a

pp

reci

ati

on

of

the s

em

inar

con

ten

ts ?

3.

Wh

at'

s yo

ur

ove

rall a

pp

reci

ati

on

of

the t

each

er

?

Sem

inar

is w

ell r

ecei

ved

Teac

hing

sty

le h

ighl

y ap

prec

iate

dN

obod

y se

lect

ed E

xcel

len;

how

ever

Exc

elle

nt w

as d

efin

ed a

s "i

n th

e to

p 5

of t

hing

s th

at w

ill in

fluen

ce t

he w

ay I

con

duct

res

earc

h"

!"#$

%$

#$

&"#$

!$

'$'$

'$&$($!$)$#$*$%$+$

,-./-$

&,-$01./$(2

3$01./$!/3$01./$)-4$01./$#

-4$01./$5$#$01./,$

'$

&6$

&$'$

'$'$($)$*$+$&'$

&($

&)$

&*$

&+$

('$

7891::12-$

;<<

3$=.>/$

?<</$

@<//>A:1$

+$

&($

'$'$

'$'$($)$*$+$&'$

&($

&)$

7891::12-$

;<<

3$=.>/$

?<</$

@<//>A:1$

Sur

vey

afte

r Res

earc

h M

etho

ds o

n Com

pute

r Sci

ence

Sem

inar

2/5

4.

Wh

ich

featu

res

did

yo

u a

pp

reci

ate

?

All

feat

ures

wer

e ap

prec

iate

d by

som

e pa

rtic

ipan

ts;

man

y ap

prec

iate

d al

lmos

t al

l fea

ture

sTo

p 3:

Rev

iew

pro

cess

; Th

reat

s to

Val

idity;

Com

pariso

n of

res

earc

h m

etho

dsBot

tom

3:

orig

ins

of c

ompu

ter

scie

nce

+ o

n-lin

e di

scus

sion

+ (

rese

arch

phi

loso

phy

& a

rt m

etap

hor

& s

pect

rum

of ca

ses)

6$

&($

&($

&+$

&($

&)$

&*$

&6$

&%$

&%$

&6$

&%$

&*$

&#$

&#$

&!$

&&$

&%$

'$($)$*$+$&'$

&($

&)$

&*$

&+$

('$

Sur

vey

afte

r Res

earc

h M

etho

ds o

n Com

pute

r Sci

ence

Sem

inar

3/5

5.

Wh

ich

featu

res

did

yo

u a

pp

reci

ate

th

e m

ost

?

Firs

t ch

oice

= 3

poi

nts;

2nd

cho

ice

= 2

poi

nts;

rd

choi

ce =

1 p

oint

s

Writing

adv

ice

was

app

reci

ated

mos

t; e

spec

ially

abs

trac

t w

riting

but

als

o fis

h m

odel

PC-m

eeting

app

aran

tly

trig

gere

d a

lot

of id

eas

==

> w

riting

adv

ice

from

rev

iew

per

spec

tive

app

aran

tly

was

key

'$&$

)$

&&$

!$*$

'$

6$

($#$

&&$

(+$

&)$

*$)$

'$!$

&!$

'$#$&'$

&#$

('$

(#$

!'$

Sur

vey

afte

r Res

earc

h M

etho

ds o

n Com

pute

r Sci

ence

Sem

inar

4/5

6.

Wh

ich

to

pic

s w

here

ove

rkill ?

Ver

y fe

w fea

ture

s w

ere

cons

ider

ed o

verk

ill;

mai

nly

the

rese

arch

phi

loso

phy

and

the

pape

rs t

hat

need

ed t

o be

rev

iew

edD

isse

nsus

--

"art

met

opho

r" a

nd "

spec

trum

" of

cas

es a

pper

as b

oth

in t

opic

s m

ost

and

leas

t ap

prec

iate

d

#$!$

!$&$

)$

'$'$

'$&$

'$'$

'$&$

&$&$

)$($

&$'$($)$*$+$&'$

&($

&)$

&*$

&+$

('$

Sur

vey

afte

r Res

earc

h M

etho

ds o

n Com

pute

r Sci

ence

Sem

inar

5/5

7.

Ho

w d

o y

ou

feel ab

ou

t th

e v

irtu

al P

C-m

eeti

ng

at

the e

nd

.

PC-m

eeting

was

ove

rall

wel

l-re

ceiv

ed;

mai

n pr

oble

m w

as t

hat

the

pape

rs w

ere

outs

ide

of t

heir e

xper

tise

.G

ener

ated

litt

le d

iscu

ssio

n w

hich

wha

t m

ost

peop

le fin

d di

sapp

oint

ing.

Sug

gest

ing

to h

ave

stud

ents

rev

iew

eac

hoth

ers

abst

ract

s !!

!

!$

&*$

#$)$

($'$

($'$($)$*$+$&'$

&($

&)$

&*$

&+$

941//0$<2$-41$

B>1$

>2,>C4DE:$

>2.BB/<B

/>.-1$

B.B1

/,$

-<<$FE94$G</H$

3>,.BB

<>2I

2C$

G.,-1$<J$I

F1$

<-41

/$

Survey after Research Methods on Computer Science Seminar(answers to open questions)

===============================================7. How do you feel about the virtual PC-meeting at the end? - other: a bit disappointing; I expected more discussion. - other: (please specify) I have never experiences participating a PC meeting before. Therefore this is a good experience to me.

===============================================8. Open comments about the things you liked.

It was interesting to know how reviewing and acceptance or rejecting works in practice.

All in all, the course was useful, especially experiencing the reviewprocess. However, since the papers to review for the PC meeting were a bitout of expertise area for some people, the participation was not verystrong. Perhaps, it might be good to have a group of people choose andsubmit papers within their interest under a wider range of topics and withinthe conditions the instructor provides. Although it might be more difficultto organize, it might produce stronger champions and make them review morerelated work.

I really liked the second and last part; practical and focused on results.

I liked the discussion of research methods and threats to validity, and that we explicitly had to think about them when reviewing the papers. I think it will be helpful for my own work.

I especially liked the art pictures. They nicely visualize the concepts behind the different research methods and make the problems clearer. Examples are always a good way to give a better insight.

- I will always remember the point about the championship of a paper and whether it gets accepted or not!- the lecturers questions during the pc-meeting (as well as meta comments) ->

short, but relevant- warm-up questions/talks of lectures- selection of papers was easy to read - even for a non-expert- review regarding threads to validity and unit of analysis- enthusiasm of lecturer- acceptance rate overview- generally I preferred the second course afternoon (from the topic selection)

I've liked that there were also papers which are controversial. One we could even rejected, there came up a lot of interesting points.

- Nice atmosphere in the class - It would be helpful if you could provide a few papers that are formally (not content-wise) really good examples. Maybe one for each kind of research method. - a really good and helpful class! All the best for future classes!

Given the time I think the topics were well covered. It was interesting and helpful. I appreciated more the second presentation since was offering concrete information about writing papers and reviewing.I liked the idea simulating a conference and seeing the steps of the PC meeting. It was a useful experience.

Overall I learn lot. Specially, structuring and writing research publications. Also the reviewing process which I have never experiences. If you had included more on technical writing it would have been much completed.

I have different feelings about the PC meeting. It was very long and hard to follow discussions about papers I never read before. Exactly that was an excellent experience. I can imagine, how long such PC meetings can be. A really nice experience that I would not miss. Even if the preparation was a lot of effort. But it was worth. I also recommend to keep 2 papers per person and a meeting of 2hours or more. I think the time of the pc meeting is well invested. It's not realy efficient but gets the right impression. Overall I liked the whole course very much. It was not that theoretical like other research methods seminars. What I missed was were other research methods like Ground Theory, lab experiments, etc. that are more related to life science research.

Aside from useful insight into the research process, and how to write better papers, I found the interaction with the others most useful. I think all the sessions should be organized in the "U" shape to encourage active discussions in order to enhance learning

The only pity party is that I had to wait 4 year for this lecture ;-) Thanks a lot

===============================================9. Open comments about the things you disliked.

In the first lecture I would like to have more hints on how to do research and how to tackle common problems.

Maybe a little bit too much work (paper reviews). But it was my first PhD seminar, so I don't know about the "standard" amount of work for PhD seminars.

I expected to learn more about paper and abstract writing, i.e. to gain more experience, nevertheless was the time frame too narrow to do more.

- I had the impression that the lecturer tried to cover too many topics -> took also more time to explain than planned. when a teacher/speaker takes more time then assigned to the lecture, I loose interest and also concentration. i.e., this last minutes often don't reach me anymore, even though I don't want this.- I prefer breaks (or at least a vote for a break) after one hour. as a phd student, I'm not used to listen to someone that long anymore. ;-)- length of papers: we generally submit papers from 8 to 12 pages, so I felt these papers were very long- The comparison of research methods was too extended, in particular the focus on case study did not match my needs/preferences

It's more a neutral point: It should be more explained what's to do. What to write in the not-normal-review part. Like unit of analysis, research method and so on.I think in phase of the online-discussion there was not so much of discussion. That should be more moderated or explained in the 2nd session what there is to do.

- the statistics part of the pair programming paper was an overkill, but this is not your fault ;-)

Maybe we could have discussed more and the students could express more often their opinions regarding the topics above.

Too little interaction, most of it was at the end during the PC meeting. Papers should be chosen to fit the students domain. A clearer taxonomy of research methods - applicable research problems/questions/outcomes.

===============================================10. suggestion for an alternative reviewing procedure (I have it from another course and I liked it very much): every student has to submit an abstract, and then the other students are assigned to read and review it