scientific method lecture 2 asst.prof. supakorn kungpisdan, ph.d. [email protected]

Scientific MethodScientific MethodLecture 2Lecture 2

Asst.Prof. Supakorn Kungpisdan, Ph.D.Asst.Prof. Supakorn Kungpisdan, [email protected]@mut.ac.th

MethodologyMethodology

ISEC0502 - NETE0501ISEC0502 - NETE0501 22

Nature of the scientific Nature of the scientific methodmethod

The The ““scientific methodscientific method”” attempts to minimize the attempts to minimize the influence of the researchers' bias on the outcome of an influence of the researchers' bias on the outcome of an experiment.experiment.

The researcher may have a preference for one The researcher may have a preference for one outcome or another, and it is important that this outcome or another, and it is important that this preference not bias the results or their interpretation.preference not bias the results or their interpretation.

Sometimes "common sense" and "logic" tempt us into Sometimes "common sense" and "logic" tempt us into believing that no test is needed.believing that no test is needed.

Another common mistake is to ignore or rule out data Another common mistake is to ignore or rule out data which do not support the hypothesis.which do not support the hypothesis.

But there is no single, universal formal But there is no single, universal formal ““scientific scientific methodmethod””. There are several variants and each . There are several variants and each researcher needs to tune the process to the nature of researcher needs to tune the process to the nature of the problem and his / her working methods.the problem and his / her working methods.


Research MethodResearch MethodClassical PhasesClassical Phases


Classical Phases (cont.)Classical Phases (cont.)


Other VariantsOther Variants


Other VariantOther Variant

1.1. Observe Observe an event.an event.

2.2. Develop a Develop a model model (or (or hypothesishypothesis) which ) which makes a makes a predictionprediction..

3.3. Test Test the prediction. the prediction.

4.4. Observe Observe the result. the result.

5.5. Revise Revise the the hypothesis. hypothesis.

6.6. Repeat Repeat as needed.as needed.

7.7. A successful A successful hypothesis becomes a hypothesis becomes a Scientific TheoryScientific Theory..

Ask Fred To Act Ask Fred To Act Dramatically CoolDramatically Cool

A- askA- ask

F- form a hypothesisF- form a hypothesis

T- test hypothesisT- test hypothesis

A- analyze the resultsA- analyze the results

D- draw conclusionsD- draw conclusions

C- communityC- community


Other Variant (cont.)Other Variant (cont.)


In PracticeIn Practice


Errors of experts who did Errors of experts who did not follow the Scientific not follow the Scientific

MethodMethod"Computers in the future may weigh no more than 1.5 tons." "Computers in the future may weigh no more than 1.5 tons."

Popular Mechanics, forecasting the relentless march of science, Popular Mechanics, forecasting the relentless march of science, 19491949

"I think there is a world market for maybe five computers." "I think there is a world market for maybe five computers." Thomas Watson, chairman of IBM, 1943Thomas Watson, chairman of IBM, 1943

"Airplanes are interesting toys but of no military value." "Airplanes are interesting toys but of no military value." Marechal Ferdinand Foch, Professor of Strategy, Ecole Marechal Ferdinand Foch, Professor of Strategy, Ecole

Superieure de Guerre.Superieure de Guerre.

"Louis Pasteur's theory of germs is ridiculous fiction". "Louis Pasteur's theory of germs is ridiculous fiction". Pierre Pachet, Professor of Physiology at Toulouse, 1872Pierre Pachet, Professor of Physiology at Toulouse, 1872

"Heavier-than-air flying machines are impossible."Heavier-than-air flying machines are impossible.““ Lord Kelvin, president, Royal Society, 1895.Lord Kelvin, president, Royal Society, 1895.


Steps of the Scientific Steps of the Scientific MethodMethod


Step 1: Research Step 1: Research QuestionQuestion

The most important step in research ! Often The most important step in research ! Often comes from the thought:comes from the thought:

““What we have now is not quite right/good What we have now is not quite right/good enough – we can do better ...enough – we can do better ...””

The research question defines the The research question defines the ““area of area of interestinterest”” but it is not a declarative statement but it is not a declarative statement like a hypothesis.like a hypothesis.

Research question must be capable of being Research question must be capable of being confirmed or refuted. The study must be confirmed or refuted. The study must be feasible.feasible.


Research Research Questions/ProblemsQuestions/Problems

EXAMPLE (1 single question)EXAMPLE (1 single question)

““Which methods and tools should be developed to make Which methods and tools should be developed to make current manufacturing control / supervision systems reusable current manufacturing control / supervision systems reusable and swiftly modifiable?and swiftly modifiable?””EXAMPLE (multiple questions)EXAMPLE (multiple questions)

““Q1: What are the main components of logistics costs that Q1: What are the main components of logistics costs that determine the logistics and transport network design? determine the logistics and transport network design?

Q2: To what extent are the existing network design and Q2: To what extent are the existing network design and evaluation models sufficient and how can collaboration be evaluation models sufficient and how can collaboration be incorporated in the network design methodology?incorporated in the network design methodology?

Q3: How can economies of scale and scope, present in the Q3: How can economies of scale and scope, present in the network, be taken into account in the network design? network, be taken into account in the network design?

Q4: Is it possible to set boundaries to the development path of Q4: Is it possible to set boundaries to the development path of the network, and search for a feasible path instead of the network, and search for a feasible path instead of searching solely for a feasible solution? searching solely for a feasible solution? ““


Research Questions Research Questions (cont.)(cont.)

Q1: Which are the main characteristics of a collaborative network and of a Q1: Which are the main characteristics of a collaborative network and of a

collaborative networked environment? collaborative networked environment?

Q2: How can be assessed the performance of a CN? Q2: How can be assessed the performance of a CN?

Q3: Which are the most relevant conceptual frameworks, architectures, Q3: Which are the most relevant conceptual frameworks, architectures,

reference models, independent and industry- specific initiatives, ICT reference models, independent and industry- specific initiatives, ICT

platforms and their underlying technologies, targeting interoperability in a platforms and their underlying technologies, targeting interoperability in a

collaborative networked environment?collaborative networked environment?

Q4: Which are the main requirements for interoperability in a networked Q4: Which are the main requirements for interoperability in a networked

environment? environment?

Q6: Which are the main differences and similarities between existing Q6: Which are the main differences and similarities between existing

conceptual frameworks?conceptual frameworks?

Q7: How can conceptual frameworks be compared, and which are the Q7: How can conceptual frameworks be compared, and which are the

criteria to support such an analysis and evaluation? criteria to support such an analysis and evaluation?

Q8: Do the conceptual frameworks and the technological solutions compete Q8: Do the conceptual frameworks and the technological solutions compete

or complement each other?or complement each other?

ISEC0502 - NETE0501ISEC0502 - NETE0501 1414Too many, no hierarchy, some redundancy.

Step 2: Background / Step 2: Background / ObservationObservation

How has the work been How has the work been done previously? What done previously? What similar work has been similar work has been leading up to this point?leading up to this point?

Study state of the art Study state of the art (literature review, (literature review, projects, informal projects, informal discussions, etc).discussions, etc).Optional realization of Optional realization of preliminary preliminary experiments.experiments.

What distinguishes What distinguishes previous work from what previous work from what you want to do?you want to do?

Who / What will be Who / What will be impacted by this impacted by this research?research?


Step 3: Formulate Step 3: Formulate hypothesishypothesis

A scientific hypothesis states the A scientific hypothesis states the ‘‘predictedpredicted’’ (educated (educated guess) relationship amongst variables.guess) relationship amongst variables.

Serve to bring clarity, specificity and focus to a research Serve to bring clarity, specificity and focus to a research problem problem

... But are not essential ... But are not essential

... You can conduct valid research without constructing a ... You can conduct valid research without constructing a hypothesis hypothesis ... On the other hand you can construct as many hypothesis as ... On the other hand you can construct as many hypothesis as appropriateappropriate

Stated in declarative form. Brief and up to the point. A Stated in declarative form. Brief and up to the point. A possible format (formalized):possible format (formalized):““If ...... then .... (because ....) If ...... then .... (because ....) ““

In the case of a MasterIn the case of a Master’’s thesis, s thesis, one hypothesis after tested one hypothesis after tested becomes a thesis being defendedbecomes a thesis being defended..

One PhD dissertation may include more than one thesis. One PhD dissertation may include more than one thesis.


Characteristics of a Characteristics of a HypothesisHypothesis

Should be simple, specific and conceptually clear. Should be simple, specific and conceptually clear. ... ambiguity would make verification almost impossible.... ambiguity would make verification almost impossible.

Should be capable of verification. Should be capable of verification. ... i.e. There are methods and techniques for data ... i.e. There are methods and techniques for data collection and analysis.collection and analysis.

Should be related to the existing body of knowledge. Should be related to the existing body of knowledge. ... i.e. Able to add to the existing knowledge.... i.e. Able to add to the existing knowledge.

Should be operationalisable Should be operationalisable ... i.e. Expressed in terms that can be measured.... i.e. Expressed in terms that can be measured.


Hypothesis ExampleHypothesis Example

Payment over Short Message Services Payment over Short Message Services (SMSs) can be feasible if messages (SMSs) can be feasible if messages transmitted among involved parties are transmitted among involved parties are designed to be minimal and it does not designed to be minimal and it does not require high computational tasks.require high computational tasks.


Step 4: Design Step 4: Design ExperimentExperiment

Includes planning in detail all the steps of the experimental phase. Includes planning in detail all the steps of the experimental phase. In engineering research it often includes the design of a prototype In engineering research it often includes the design of a prototype / system architecture./ system architecture.

Identify the variables that will be manipulated and measured – the Identify the variables that will be manipulated and measured – the research outcomes must be measurable. In other words:research outcomes must be measurable. In other words:

What needs to be controlled in order to get an unbiased answer to What needs to be controlled in order to get an unbiased answer to the research question.the research question.

Therefore: it is necessary to not only design a prototype / system Therefore: it is necessary to not only design a prototype / system but also the thesis validation method !but also the thesis validation method !

How to validate the thesis?How to validate the thesis?

The plan should allow others to repeat it. It should be feasible...!The plan should allow others to repeat it. It should be feasible...!

Plan intermediate milestones.Plan intermediate milestones.

If you fail to plan, you planned to fail !If you fail to plan, you planned to fail !


Step 5: Test hypothesis / Collect Step 5: Test hypothesis / Collect datadata

Doing it !Doing it !

Implementation of methods (e.g. prototyping) and Implementation of methods (e.g. prototyping) and auxiliary tools (e.g. simulation)auxiliary tools (e.g. simulation)

Pilot testing and refinement. Pilot testing and refinement.

Field vs. Laboratory work. Field vs. Laboratory work.

Any ethical considerations ? Any ethical considerations ?

Confirm results by retesting !Confirm results by retesting !


Test HypothesisTest Hypothesis


Step 6: Interpret / Analyze Step 6: Interpret / Analyze ResultsResults

What did your experiment show?What did your experiment show?

Qualitative data analysis. Quantitative data analysis.Qualitative data analysis. Quantitative data analysis.Descriptive and inferential statistics, clustering, ...Descriptive and inferential statistics, clustering, ...

What might weaken your confidence in the results What might weaken your confidence in the results (critical spirit)?(critical spirit)?

Discussion regarding Discussion regarding LiteratureLiteratureResearch objectives Research objectives Research questions.Research questions.

Consider next steps Consider next steps Recommendations for further research.Recommendations for further research.


Step 6: Interpret / Analyze results Step 6: Interpret / Analyze results (cont.)(cont.)


Young or old lady?

Consider multiple perspectives !

Step 7: Publish FindingsStep 7: Publish Findings

A research result is not a contribution to the field if A research result is not a contribution to the field if no one knows about it or can use it !no one knows about it or can use it !

Write scientific papers, make presentationsWrite scientific papers, make presentationsIntermediate results Intermediate results

ConferencesConferences

Collect feedbackCollect feedback

Consolidated results Consolidated results JournalsJournals

Be careful in selecting where you publish ! Be careful in selecting where you publish !

Write thesisWrite thesis


Attributes of A Good Attributes of A Good ThesisThesis

Contestable, proposing an arguable point with Contestable, proposing an arguable point with which people could reasonably disagree. which people could reasonably disagree. Specific and focused. Specific and focused. Clearly asserts your own conclusion based on Clearly asserts your own conclusion based on evidence. evidence. Provides the reader with a map to guide him/her Provides the reader with a map to guide him/her through your work. through your work. Avoids vague language (like "it seems"). Avoids vague language (like "it seems"). Avoids the first person. ("I believe," "In my Avoids the first person. ("I believe," "In my opinion")opinion")Pass the Pass the ““So what? or Who cares?So what? or Who cares?”” test test


Is It A Good Thesis?Is It A Good Thesis?

Does the thesis inspire a reasonable reader to ask, Does the thesis inspire a reasonable reader to ask, "How?" or Why?"How?" or Why?““

Would a reasonable reader NOT respond with "Duh!" Would a reasonable reader NOT respond with "Duh!" or "So what?" or "Gee, no kidding!" or "Who cares?or "So what?" or "Gee, no kidding!" or "Who cares?““

Does the thesis avoid general phrasing and/or Does the thesis avoid general phrasing and/or sweeping words such as "all" or "none" or "every"?sweeping words such as "all" or "none" or "every"?

Does the thesis lead the reader toward the topic Does the thesis lead the reader toward the topic sentences (the subtopics needed to prove the thesis)?sentences (the subtopics needed to prove the thesis)?

Can the thesis be adequately developed in the Can the thesis be adequately developed in the required length of the paper or dissertation?required length of the paper or dissertation?


Proof Of ConceptProof Of Concept

Proof of concept is a short and/or incomplete Proof of concept is a short and/or incomplete realization of a certain method or idea(s) to realization of a certain method or idea(s) to demonstrate its feasibility, or a demonstration demonstrate its feasibility, or a demonstration in principle, whose purpose is to verify that in principle, whose purpose is to verify that some concept or theory is probably capable of some concept or theory is probably capable of exploitation in a useful manner. A related exploitation in a useful manner. A related (somewhat synonymous) term is "(somewhat synonymous) term is "proof of proof of principleprinciple".".


Presentation LanguagesPresentation Languages

Is it necessary to include many formulas and Is it necessary to include many formulas and equations? Is it not equations? Is it not ““scientificscientific”” if not full of if not full of mathematics?mathematics?

There are different There are different ““languageslanguages”” used in used in different disciplines. E.g. Mathematical different disciplines. E.g. Mathematical formulas, Logical formulas / Set theory formulas, Logical formulas / Set theory formalism,formalism,

Formal specification languages (e.g. Z, Petri Nets), Formal specification languages (e.g. Z, Petri Nets), charts, semi-formal diagrams (e.g. UML), etc.charts, semi-formal diagrams (e.g. UML), etc.


SimulationSimulation

Simulation is an important tool in engineering and Simulation is an important tool in engineering and research. research.

In some areas it can cope for In some areas it can cope for unaffordable costsunaffordable costs with with physical experiments physical experiments

It can also help when the performance of the It can also help when the performance of the experiment in the real world would take experiment in the real world would take a long period a long period of timeof time (beyond the duration of the research project (beyond the duration of the research project

But be careful with its use: But be careful with its use:

How well does the simulation model How well does the simulation model reflect the reflect the realityreality? ?

You might be inferring conclusions based on You might be inferring conclusions based on ““artificial artificial worldsworlds”” ... ...


Exercise (individual)Exercise (individual)

Task 1Task 1

Select a research paper and find research Select a research paper and find research question(s).question(s).

Task 2Task 2

Select a Master’s thesisSelect a Master’s thesis

Identify what’s been done in the thesis and Identify what’s been done in the thesis and summarizesummarize

Research question(s)Research question(s)

HypothesisHypothesis

ExperimentExperiment

ConclusionsConclusions


[email protected]@mut.ac.th


scientific method lecture 2 asst.prof. supakorn kungpisdan, ph.d. [email protected]

Documents

isec0502 nete0501nature

isec0502 nete0501steps

isec0502 nete0501errors

scientific theory

scientific methodlecture

scientific methodcomputers

successful hypothesis

existing network design