1

Proceedings of the ASME 2014 International Design Engineering Technical Conferences & Computers and Information in Engineering Conference

IDETC/CIE 2014 August 17-20, 2014, Buffalo, New York, USA

DRAFT:DETC2014-34954

IDEATION METHODS: A FIRST STUDY ON MEASURED OUTCOMES WITH PERSONALITY TYPE

Pui Kun Choo Singapore University of Technology & Design

Zhi Ning Lou Singapore University of Technology & Design

Bradley A. Camburn Singapore University of Technology & Design

Kristin L. Wood Singapore University of Technology & Design

Ben Koo Tsinghua University

Beijing, China

Francois Grey Centre for Nano and Micro

Mechanics, Tsinghua University, Beijing

ABSTRACT The research reported here considers an experiment and subsequent data coding and analysis to extract correlations between personality type and ideation outcome from several methods. This article presents the background theory, research methodology, and empirical results associated with the experiment. The experiment is based on observations of designers developing a real product, and associated assessment tools, where the goal is to correlate the quality, quantity, and variety of design outcomes with respect to personality type. This approach lays the foundation for a tailored ideation method or a suite of ideation methods that takes advantage of the preferences and strengths of individuals. We find that there are significant correlations between type and ideation metrics and that these correlations are supported by related theory from psychology and business management.

Keywords: ideation methods, brainstorming, mind-

mapping, Method 6-3-5, C Sketch, personality type, Myers-Briggs Type Indicator, Six Thinking Hats, empirical study 1. INTRODUCTION Structured ideation methods are critical for the progress of many projects in engineering design. Technically accurate information on the effectiveness of these methods is equally critical [1]. It is important to assess the methods through formal research methodologies and obtain insights on how and why ideation methods can be made effective.

Design managers have been grouping or seeking to understand teams based on personality type for a long time [2,3]. One possible avenue for research is to elicit the relationships between personality types and ideation method.

Since different personality types have been shown to communicate differently [4,5]; and different ideation methods are based on different types of communication [6,7], it is only intuitive to infer that there may be differences in the effectiveness of an ideation method for different personality types. This information could potentially be used to develop a structured ideation method or a suite of ideation methods that is tailored for individuals or different compositions of teams.

This study follows the practice of designers engaged in a sponsored project. Procedures are employed to extract concrete data about their performance on a design task. These data are then post processed to make inferences about the correlation of personality type and ideation methods. The type of research employed in this study is often referred to as an empirical study [1]. Other studies have in fact contributed to examining personality types during design. Individuals from different fields tend to span a range of particular personality types, and different personality types prefer particular stages of the design process [8,9]. There has also been comparative research on the various ideation methods used in this paper. Design outcome metrics have been used to compare the quantitative results of different ideation methods [6,7,9,10]. However, the authors believe this may be the first time that different ideation methods and personality type are directly compared for the ideation phase of design.

The objective of empirical studies is to verify and record observable phenomena. This allows for the quantificaion of what may previosuly have been intuitive knowledge. The experimental cycle allows this at a high level of granularity. Along this line, the following research questions were used to guide and develop this study: 1. What are the statistically significant trends (if any) in

quality, quantity, novelty, and variety of design solutions

2

produced by different personality types across different ideation methods, and between personality types within each method?

2. Are there statistically significant differences in how participants of different personality types self-perceive the outcome of different ideation methods?

We address these questions by creating a structured

ideation environment as part of an active design group, and measure the outcome quantitatively using established metrics. These metrics include quality, quantity, novelty, and variety of the design outcomes as well as self-efficacy of the participant designers. The following sections describe the ideation and personality type assessments employed, the experimental setup, the design problem and its context, the participants, data encoding and analysis procedures, as well as results and conclusions of the study. 2. BACKGROUND THEORY 2.1 Brainstorming As an intuitive method [11] of idea generation, the brainstorming method by Osborn [12] encourages divergent thinking [11] in design problem solving. It can be employed individually or in a group typically including five to fifteen people. Individually, one may start by identifying a few concepts to build on and generate as many solutions as possible. In a group, members verbally communicate ideas to one another for thirty to forty-five minutes. Ideally, the members should not be inhibited in expressing any ideas to achieve a comprehensive range of solutions. Development of individual ideas result as members respond by making connections to others’ ideas. These connections will vary as each member differs in skill sets, experience and personality, creating diversity in solutions. However, individual brainstorming has been shown to be more productive than group brainstorming. In a group, domination by a single or a few group members may occur [13]. Inhibition could also happen in the presence of an expert [13], or when groups are unreceptive to new ideas, and may result in discussing only existing solutions. A facilitator could be appointed to ensure participation by all, while restricting negative criticism. A complementary mind-mapping or 6-3-5 / C-Sketch session combined with or held after initial brainstorming may lead to greater effectiveness. An example brainstorming sheet from the study is shown in Figure 1.

Figure 1 An individual Brainstorming sheet

2.2 Mind-mapping

To effectively maximize the results from brainstorming, one can use mind-mapping. Mind-mapping is an intuitive semantic and categorization technique that emulates a process similar to how we organizes ideas in long-term memory. First, a key idea is placed at the center of a piece of paper. Next, possible solution categories are added, branching off the key idea. Finally, specific solutions to the problem are added to these key categories. Thus, each solution generated is related to the original problem statement. Research has demonstrated that mind-mapping may significantly increase the number of ideas generated compared to the classic brainstorming approach. This result is attributed to the categorizing of ideas, which arranges concepts hierarchically, hence suggesting the difference between the design avenue or category of ideas and specific solutions. Moreover, mind mapping facilitates piggy-backing and leap-frogging of ideas due to its a two-dimensional graphical map structure. This structure opens up the opportunity to identify and fill in gaps in the possible design space; for example, upon creating the mind-map, one may notice a certain branch of solutions is less complete than another [14]. An example mind -map from the study is shown in Figure 2.

Figure 2 An individual Mind-map

2.3 C-Sketch As an extension of the 6-3-5 method, which uses written description for idea generation, the C-Sketch method is an intuitive method that uses graphical descriptions instead of written descriptions. It is usually employed after the problem definition and clarification stage of design [15]. In a team, each member is given a sheet of blank paper on which they are to sketch three solutions with respect to the design problem statement. After t minutes, the papers are to be passed on to the next person on the right. Another t’ minutes will be given to add modifications or additional ideas to each idea. The process of passing repeats until all members have contributed to every individual paper. The number of people in each team is typically six, although a range of three to eight members may work well. Likewise, the time duration is variable; it can be fifteen minutes initially followed by ten-minute alteration sessions. Each individual is usually given a uniquely colored pen or marker to encourage no elimination of ideas, and allow members to easily identify their contributions for later discussion. There is no verbal communication allowed to prevent domination of the session by a single or small group of members, while encouraging participants to make individual inferences of the sketches that may result in unanticipated ideas. Labeling should also be kept to a minimum, but instead focus only on main keywords. It is also important to refrain from negative criticism, but instead to

3

focus on further developing the ideas. Brainstorming may be combined with a C-Sketch session, either before or after, for development of ideas through verbal communication. C-Sketch facilitates leap frogging of ideas [7], and achieves diversity in design [15]. Provocative stimuli [15] from sketches of other members reduces design fixation, and helps to develop new ideas. Research has also shown that C-Sketch induces a forty percent increase in quantity of ideas produced over a variety of comparable methods [11]. An example of a solution produced in C-Sketch can be seen in Figure 3.

Figure 3 A concept developed during C-Sketch

Each of the above ideation methods was deployed in the experiment. A summary of the methods is provided in Table 1. There are many useful methods, these three were chosen for several reasons. First because they require relatively little introductory training and allowed for the full deployment of several methods in limited time. Secondly these methods represent a mixture of structured versus open-ended approaches to ideation, and individual versus group formatting. Finally, there was a rich compliment of empirical literature to evaluate the properties of these methods readily available for review. The remainder of Section 2 details the Myers Briggs Personality Type Indicators and Six Thinking Hats.

TABLE 1: SUMMARY & COMPARISON OF METHODS

Method Communication Style

Individual Brain-storming

Written word only Provocative stimuli, use of analogies Group

Brain-storming More verbal than written word

Individual Mind-mapping

Written word only Categories, structured and organized Group

Mind-mapping Verbal and written word

C Sketch Sketching primarily Imagery, graphical, provocative stimuli

2.4 MBTI The Myers-Briggs Type Indicator (MBTI) [4] assesses an individual’s level of preference in four categories that indicate aspects or approaches to problem solving, decision-making and communication of information or ideas. The categories are based on C.G. Jung's theory of psychological types (Table 2). A total of sixteen types result from the permutation of the categories. Research has shown that engineers are more likely

to be ISTJ, followed closely by ESTJ, then ENTP [9]. Jung’s theory of eight cognitive modes, representing problem solving approaches, is related to the two dominant modes or sub-types as shown in Table 3 [3].

MBTI can be used in team formation strategies [2,3,16,17] to achieve diverse teams [2,3,16]. It ensures a mixture of members with a variety of cognitive styles, providing groups with a spectrum of viewpoints and problem solving methods. Through identification of each individual’s type, it informs and encourages understanding amongst members [3].

TABLE 2: OVERVIEW OF MBTI

Ori

enta

tion

Extraversion (E)

OR

Introversion (I)

Prefer working in groups and through external interaction, often taking a breadth-of-knowledge approach.

More comfortable working alone reflectively, taking a depth-of-knowledge approach into ideas and concepts.

Per

cept

ion

Sensing (S)

OR

Intuition (N)

Gather information through practical experience, focusing on observable phenomena, facts and details.

Perceive through imagination and internal sensing, focusing on the big picture, theories, and new possibilities.

Judg

men

t Thinking (T)

OR

Feeling (F)

Analytical and logical, judging objectively through impersonal evaluations.

Subjective and weigh human factors, often making decisions based on personal values.

Styl

e

Judging (J) OR

Perceiving (P)

Decisive and planners, preferring structure and order.

Keep options open, are flexible, spontaneous and exploratory.

TABLE 3: JUNG’S COGNITIVE MODES [3]

Information Collection

Decision-Making

ES EN ET EF Experiment Ideation Organization Community

IS IN IT IF Knowledge Imagination Analysis Evaluation

2.5 Six Thinking Hats The Six Thinking Hats model by Edward de Bono distinguishes six modes of thinking, represented by six colored hats [5] (Table 4).

TABLE 4: OVERVIEW OF SIX HATS

White Hat Red Hat Concerned with facts, and objective information.

Utilizes emotions and intuition.

Black Hat Yellow Hat

An analyst, the “devil’s advocate” who gives negative but logical criticism, identifying why something might not work.

An optimist, giving logical positive criticism on why something might work.

Green Hat Blue Hat Creative, generates new possibilities and solves problems through lateral thinking.

Often the leader facilitating, overseeing and organizing thinking processes to achieve the agenda.

Role-playing hats in a group facilitates group-thinking processes, as multiple perspectives can be covered [18]. Otherwise, if each individual’s hat is known, those of different hats can be grouped together to achieve a balance of thinking types within a group [2,3,16]. By identifying one’s correlations, association, or preferences for particular hats, it focuses and amplifies the particular preferred mode of

4

thinking. This improves communication as thinking modes are used deliberately, giving greater freedom to the thinker to express thoughts through the mode chosen [18].

Research has shown that groups that are formed with just either MBTI or six hats are more effective than groups that do not meet the criteria of the respective team formation strategies [2]. Groups that are formed based on both MBTI and six hats results and team formation strategies are significantly more effective than groups formed with just either MBTI or six hats, under certain conditions [2]. 3. EXPERIMENT Previous research shows that there are distinct differences in the results of various ideation methods [6,7], and that a designer’s personality type is correlated with different behavioral preferences in problem solving and communication [17,20]. Cross-correlating these two variables could help to develop a tailored ideation method or suite of ideation methods that reinforces the proclivity of each individual on a team, or the team dynamics as a whole. However, to achieve such a method, an investigation and development of quantified model of correlations are needed.

An experiment is thus pursued to measure outcomes at the intersection of personality types and ideation methods. Personality tests are given before participants, composed of designers working on a coordinated project, arrive on site. These tests result in both MBTI and Six Hats type indicators. During the design challenge, participants are provided the opportunity to employ design ideation methods in a controlled environment. Ideation outcomes are recorded for post analysis. Several methods of ideation are employed: brainstorming, mind-mapping and the C-Sketch method. Brainstorming and mind-mapping are employed both individually and in groups. The following section describes the experimental procedures and subsequent analysis in detail. 3.1 Context This experiment involved deploying design ideation methods and tracking results during a product design challenge. The design challenge, known as Lego2Nano, was the third in a series of China-UK Summer Schools between Tsinghua University, Peking University and the University College London, held on the campus of Tsinghua University. A number of individuals from diverse educational backgrounds were selected to work together for five days to design and build a low-cost Atomic Force Microscope1 (AFM) suitable for use in Chinese high schools.

An aspirational theme of the challenge was to determine how a low-cost AFM might transform science teaching in schools. To make this part of the challenge much more

1Traditional optical microscopes are unable to resolve features smaller than 

about one micrometer – a thousandth of a millimeter – the wavelength of visible light. The AFM uses direct physical contact between a sharp tip and a surface  to detect features on a surface that are much smaller than a micrometer. Such microscopes can even sense single atoms. The AFM was invented in the 1980s. It has proved very useful for many fields of research, including studying new materials for energy storage, measuring important biological aspects of DNA molecules, and fabricating novel types of electronic devices. But this type of microscope typically costs $100,000 or more for a professional quality version, and even so‐called "educational models" are at least $20,000.  

concrete, students from local Chinese high schools were invited to participate in the event, and were interviewed by the Lego2Nano teams on the first day of the challenge, to understand context of the high school students’ needs and constraints.

The Lego2Nano challenge focused on teamwork. Teams were selected based on carefully balancing age, gender, nationality and their different technical backgrounds, as well as personality traits such as whether individuals are intuitive or critical, extrovert or introvert. An important focus in the first couple of days was on activities that helped the team members learn about each other and appreciate their complementary skills.

Professional scientists and engineers have spent many years trying to improve the AFM, so there’s no reason to expect that young scientists – many of whom didn’t even know what an AFM was at the outset of the event - could complete this challenge in just one week. Still, a significant step forward towards a low-cost AFM was made. And, at the same time, the event represented a radically new approach to teaching science and technology, promoting teamwork and encouraging internationalization and interdisciplinary as part of a Chinese system.

This new approach is also called XLP, short for “eXtreme Learning Process”. XLP is a learning activity design methodology intended to explore the boundaries of cognitive capabilities of groups of people with diverse talents. XLP activities divide participants into two groups, namely “Challenge Designers” (developers and organizers) and “Missionaries” (participants).

In general, Challengers first play out learning related tasks on themselves, months ahead of time, so that they can assess how much time and resources are needed to accomplish certain tasks. After trying out the tasks and identifying at least one feasible solution, then, Challenge Designers will prepare the event according to the necessary success factors for an intended audience, called the “Missionaries.” Challenge Designers will stand by Missionaries during the intensive workshop, usually four to five days. The purpose is to guide Missionaries when necessary, but not to do the design tasks for them. Some times, Challenge Designers will serve as technicians to help Missionaries perform certain implementation tasks, but for a “price”, usually measured in virtual currencies. A comprehensive XLP would consist of simulated Banks, Courts, and Patent Offices. More detailed explanation of XLP can be found in [21]. 3.2 Participants

5

The challenge participants were composed of PhD and post-doctoral students in engineering and physics. Some of the particiapnts specialize in the development and use of nanotechnology equipment, while others focus on graphic and industrial design. Additionally, some of the participants have considerable experience building and using nanotechnology tools. These nanotechnologists also had minimal experience with design methodologies – which is positive, as they remain unbiased towards to any particular method due to previous or personal experience. There were thirty-one (31) participants, coming from top universities around the world. The designers and nanotechnologists were distributed evenly between teams. We were able to collect a complete set of data from twenty-five of the participants, as the process was voluntary, and participants completed only those methods they chose to engage their efforts in. Figure 4 depicts the breakdown of personality types. 3.3 Design Problem The unique design problem for this challenge was developed as part of a goal to create a novel research collaboration between China and the UK. Participants are aware of the fact that their results will become very real and are thus highly motivated to produce a novel design. For the purposes of a challenge event, the technical challenge was divided into five aspects:

1. Resolution of the force scanning device 2. Creative engineering and technical design aspects of the

device 3. Scientifically meaningful applications as supported by the

device 4. Suitability of the device for use by high school students 5. Ingenuity in sharing and crowdsourcing the device

production and its applications

The participants were given one week in which to conceive a design and produce a working prototype. They were provided with a few basic prototyping components including LEGO construction sets, MindStorms, a few piezo crystals, and an AFM probe or cantilever tip. For context, although prototyping will not be discussed in this paper, the final prototype is shown seen under construction in Figure 5.

3.4 Description of the Workshop Tutorials The format in which the researchers engaged in this activity was to provide the participants with a series of instructional videos and brief information sessions on applying ideation methods, and then walk participants through completion of each method. There were videos for brainstorming, mind-mapping and C-Sketch methods. Each instructional period was timed at ten minutes. Participants were then given fifteen minutes to work through each of the methods.

Figure 5 Construction of a final prototype

3.5 Data Recording The methods were deployed in sequence during the challenge, and participant communication was controlled, as was the amount of time allotted to each method. The sequence and communication levels of each method are as shown in Table 1.

During individual methods, participants were not allowed to talk. During team methods, teams were not allowed to discuss technical issues with other teams but individuals were allowed to discuss freely within their own team. The exception to this approach is C-Sketch, in which communication only occurs by passing the sheets of paper even though it is a team exercise.

The participants were instructed to produce as many ideas as they could during the fifteen minutes allotted to each method. Data collection occurred at the beginning of the session and after each method, consisting of a self-efficacy survey and collection of all concept sketches. Each individual was provided with a uniquely colored and coded pen so that

Figure 4: Breakdown of percentage of participants by their identified dominant type (rounded to the nearest integer). The

three type sets above are indepedent, each individual is represented once in each chart.

6

his or her solutions could be tracked anonymously. It was still possible to correlate personality type to ideation solution as individuals provided this same code on their personality surveys.

The methods of self-efficacy assessment via surveys and raw ideation solution collection provide for an analysis of the effectiveness of each method not only in terms of design content but also self-perceived ability or satisfaction. Previous studies have shown that self-efficacy is positively related to actual performance both in the past and future [19, 20]. Taking a picture with a high resolution DSLR camera allowed for later reference and analysis of the drawings. 3.6 Solution Encoding The researchers worked to encode the solutions into a common format to remove any biases due to handwriting quality, or irrelevant aspects of the original drawing. The solutions were described using a common format and re-listed as entries in an excel table. Images were also translated into descriptions of solutions. A link was maintained to the original image file for reference. For those solutions that consist of multiple aspects, each aspect is listed as a unique entry. This approach provides an equal comparison between a solution that only covered a single aspect of the problem and those that combined multiple aspects in a single solution. For example, it would not be equitable to rate a solution that covers probe design and stage motion control in a single drawing equivalently to a solution that only includes a probe design.

3.7 Metrics The analysis of results utilizes a standard set of metrics employed in design science literature for the purpose of evaluating ideation outcome. These are: quality, novelty, variety, and quantity as first introduced by Shah and adapted by Chan et al. [1,10]. Self-efficacy was another metric used to assess the design outcomes. In parallel both the Myers Briggs Personality Type Indicator (MBTI) and Six Hats methods were used to record personality type. Three raters with background in the challenge and solutions encoded the data. Pearson’s correlation coefficient for inter-rater agreement for solution binning by function was calculated as a 0.73 raw score, and 1.0 after discussion and resolution of each mismatched specific solution.

The chosen metrics were first introduced as a generic tool to provide quantitative evaluation of creative results produced in ideation sessions and for design research [1,6,7,10]. The metrics provide information about the performance of the individuals during an ideation session, and overall from certain methods in a numerical form so that statistical analysis

can be employed to test for significance of these findings. 3.7.1 Quality. Quality is a measure of the feasibility of a developed design or system in question to satisfy design requirements. For example, the challenge assessment of quality might be a normalized measure of the resolution of a microscope, where a microscope with a higher relative resolution is rated with a higher quality. Since the designs in question were at a conceptual level, experienced nanotechnology researchers provided input on the potential quality of each solution according to the scale in Table 5. 3.7.2 Quantity. Quantity is a direct and basic measure of the number of ideas produced (either in total for a single method or by an individual). Quantity can be measured as either unique ideas, that is, ideas that a rater determines to be unique functionally with respect to other ideas; and raw quantity of ideas, which is the total number of ideas listed during an exercise, even including repeats. Repetition is identified through Novelty and Variety.

3.7.3 Novelty and Variety. Variety is a measure of the explored solution space during the idea generation process [1]. The generation of similar ideas indicates a lower variety, and therefore corresponds to a lower probability of finding better ideas in the possible solution space. We calculate variety, in this study using the equation adapted from Shah by Chan et al. for evaluation of design ideation methods [1,10]. The equation for applying this approach to a set of ideas is stated as

(1)

where is the novelty of specific solution i; is the total number of times a specific solution was generated for that sub-function of the problem in the given ideation method; and is the total number of times the specific solution to be evaluated was generated in the given ideation method.

∑

(2)

where n, is the total number of solutions generated by an individual with a particular ideation method.

Novelty is a measure of uniqueness of a solution [10]; and, in a complementary way, variety is a measure of the uniqueness of a set of design solutions. Mathematically, it is simply the average novelty for a set of solutions. In our case, solutions were collected in concept variant bins. The solutions in a bin all perform the same function with the same basic principle. These are considered a specific solution. For

TABLE 5: QUALITY SCORING RUBRIC EMPLOYED DURING EVALUATION

Score Level Sublevel Examples (Control of Probe Approach)

0 Not a valid concern or idea Use magic

1 Valid idea but not implementable Low Accuracy is challenging

2 Medium Probe angle is hard to control

3 High Probe angle is hard to control due to atomic reaction forces

4 Valid idea that is implementable Low Probe angle must be controlled

5 Medium Use the same controller for probe approach angle and tapping mode scanning

6 High Use a low power high voltage controller for probe

7 Specific implementable solution Low Use Arduino to control probe

8 Medium Use an Arduino with PID function to control probe

9 High Arduino controller, linked with USB microscope and z-piezo as sensors for PID control

7

instance for the subfunction of ‘stage scanning scheme’, two drawings from different participants depicting a probe that is free to move in x, y, and z alongside a stage which is fixed would be listed as the same specific solution. However, a solution depicting a probe that moves only in z and a stage that scans in x, and y would be a different unique solution under this subfunction.

3.7.4 Self Efficacy Surveys. A method is also required to determine the participant’s own perception of the results of each method. This is important not only to determine the participant’s psychological reaction to the method but also as a parallel test of the metrics. This is possible, as it has been established that self-efficacy is correlated with actual performance [19,20]. Paper surveys are used to establish self-efficacy. Each individual was given a single page multiple-choice survey across ideation methods, and five minutes to complete the survey. We asked participants to rate the effectiveness of each ideation session in terms of quantity, quality, novelty, and variety of the ideas they generated, as well as the over-all usefulness of the method from their perspective. The questions are structured as a five-point Likert scale that ranges from ‘Strongly Agree’ to ‘Strongly Disagree’. 4. RESULTS AND DISCUSSION Over all we find the session to have been largely productive. We found that the 25 participants who opted in to analysis for the study produced 1095 individual solutions and 321 unique solution bins. That means the average individual produced more than 43 individual solutions and more than 12 completely unique ideas in the total 75 minutes of brainstorming. The detailed results section consists of three segments detailing overall trends of the ideation session, MBTI results, and finally Six hat results. We applied paired t-test analyses for mean shift in the data results, correspondingly, all ‘p’ values reported in the results section are the significance estimates reported from this test. 4.1 GENERAL IDEATION RESULTS Before examining comparisons between types it is important to review results of the ideation methods as a whole. The average results across all participants can be seen in Figure 6. As would be expected from the literature, mind-mapping and C-Sketch were effective methods. Individual mind-mapping scores were higher in quantity than individual brainstorming (p = 0.008), and group brainstorming (p = 0.015). C-Sketch also significantly outperformed individual brainstorming (p = 0.058), and group brainstorming (p = 0.064) in quantity. C- sketch is more importantly known for permitting the refinement and advancement of ideas, accordingly C-Sketch saw a significant increase of the quality of ideas produced, with ideas produced in C-Sketch having a higher mean than all other methods in quality (p <= 0.0003).

We find a surprising result that individual methods generally outperformed group methods. The explanation for this may be that during sequential efforts of the same method in individual and then group efforts, the participants saturated the obvious results within a particular method. This explains why there is a jump up in scores when introducing individual mind-mapping or C-Sketch, but fewer ideas are introduced on the second effort of using a similar method, e.g. group

brainstorming, and group mind-mapping. This is clarified by the ‘New Solution Bins’ sub-chart in Figure 6.

8

Figure 6 depicts the average number of new solutions introduced per person. Comparisons must carefully account for this. For lateral comparisons, (between types within one method), the comparison can be direct. For longitudinal comparisons, (between different methods within one type) the comparison is relative to the average of each method. 4.2 MBTI COMPARISONS For MBTI analysis, Jung’s cognitive modes (Table 3) are cross-compared as in Wilde’s Teamology. Studies have found them to be the dominant indicators of individual performance in team dynamics. There are eight possible modes, however we had a small sample size of Introverted Feeler types (Evaluation) and Introverted Sensing types (Knowledge); therefore we do not report results on these types. This deficit occurred as participants elected to join this project of their own accord and we did not have the opportunity to screen for an even number of each type.

The results of this comparison can be seen in Figure 7. Each bar in Figure 7 is the average score for all individuals in a given cognitive mode. Additionally, each plot represents an independent data set. The ET-EF-IT results are separated from the EN-ES-IN graphs. For instance, there are no Introverted Thinkers that are also Extroverted Feelers, but there may be some Extroverted Feelers that are also Extroverted Intuitors; thus, those two sets are not directly comparable.

By examining the results seen in Figure 7, a number of insights can be found that relate the cognitive modes to ideation results and the related theory. Results of the ‘decision making’ types (ET-EF-IT) will be discussed first. It would be expected that the ET or Organizer types would score highly in mind-mapping. They do outperform IT or Analysts. IT or Analyst types would be expected to perform most highly in C-Sketch as it is the most analytical method. Indeed they significantly outperformed EF or Community types (p = 0.05). This supports the theory that Analysts excel in the solution refinement process of C-Sketch. Lastly, what would be expected from theory on the decision making types is for EF or Community types to perform well in group methods. In fact they are the only type that actually did better in group brainstorming than individual brainstorming, but not quite significantly so (p = 0.15). With regards to quality, EF or Community types produced the best ideas in group mind-mapping. This would be expected as group mind-mapping requires a lot of group integration, their scores were significantly higher than for IT or Analyst types (p = 0.01). Finally, for variety, IT or Analyst types had greater variety than ET or Organizer types in GBS (p = 0.01), it may be that Analyst types were more comfortable without an organizational structure.

Quantity

Quality

Variety

NewSolutionBins

Figure 6 Total averages for all participants across the ideation session. Error bars are ±1 standard error. The vertical axis is performance in each metric (see section

3.7) The horizontal axis is the method code: I = indivudual, G = group; BS = brainstorming, MM =

mindmapping, and 635 = C-Sketch

9

Similarly, comparisons of interest can be made to the theory for ‘information collection’ types (EN-ES-IN). The EN or Ideation types would generally be expected to perform well across the board in group or extroverted ideation methods. There were no particular methods in which EN types did better than other types, this may be because group methods were placed after individual methods. IN or Imagination types would be expected to perform well at individual methods. They indeed have the highest performance in C-Sketch among the information collection group comparisons, with significance (p = 0.01) that they produce more ideas. This indicates that the segmentation approach of C-Sketch allows for individual introspective or imagination type ideators to flourish. With regards to quality, some comparisons were significant also. ES or Experimenter types outperformed EN Ideation types in C-Sketch average quality (p = 0.02). This could indicate that the C-Sketch process of iteratively evaluating an idea has an aspect comparable to experimentation. ES or Experimenter types similarly outperformed EN or Ideation types in Variety of C-Sketch ideas also. Table 6: Self efficacy for MBTI. Range is from 1 to 5, where

5 is strong agreement. Information Collection Decision Making

BS MM 635

BS MM 635

p

EN/ES 0.171 0.130 0.281 EF/ET 0.482 0.400 0.088

ES/IN 0.058 0.313 0.500 ET/IT 0.001 0.144 0.049

EN/IN 0.001 0.031 0.166 EF/IT 0.012 0.090 0.417

Typ

e

EN 3.8 3.6 3.6 EF 3.7 3.6 3.04

ES 3.5 3.3 3.1 ET 3.7 3.5 3.9

IN 2.8 3.1 3.1 IT 2.9 3.2 3.2

Self-efficacy results for the cognitive modes appear to

align with the quantitative results for EF and ET, Introverts tended to self-assess results lower than extroverts, which may be why IT reports a lower effectiveness of their ideation, the same can be seen with IN types. Table 6 summarizes the average score for self-assessed high quality, novelty and variety of ideas in the method.

4.3 Six Hat Comparisons Similarly for the Six Hats type indicators, significant differences in performance results were found between different types. To ensure that the type sets were independent, a similar process of separating groups was employed before comparison. An individual can be evaluated as having a high score for multiple hats. Each individual must be assessed according to their strongest hat preference to ensure that comparisons are independent for inter-type comparisons. However, some individuals who participated scored a strong but equal indication for several hats. These individuals were removed from analysis in the six hats comparison. Once those individuals were removed from the set, there were only enough individuals remaining to make statistically significant comparisons between strong Yellow Hat thinkers and strong Blue Hat thinkers. The results of this comparison are shown in Figure 8.

Decision Making Personality Type Set

Quantity Quality Variety

Information Gathering Personality Type Set

Quantity Quality Variety

Figure 7 Comparison between type averages for Jung’s Cognitive modes. Error bars are ±1 standard error. The vertical axis is performance in each metric (see section 3.7) The horizontal axis is the method code: I = indivudual, G = group; BS =

brainstorming, MM = mindmapping, and 635 = C-Sketch

10

It is clear that yellow hats are generally productive in the brainstorming and mind-mapping techniques. As would be expected from the theory, indicating that the characteristic of yellow hats is to expand on existing ideas act as optimistic ideators. It is also interesting to note that yellow hats produce this high quantity in both individual and group methods. The statistical significance of Yellow Hats produced a higher quantity of ideas than Blue hats in group brainstorming was only p = 0.069. This falls below the .05 threshold, but given the small sample size, it is a suggestion that with a larger sample size the difference may prove significant. Furthermore, it was significant that Yellow hats produced a higher than average quality in group brainstorming (p=0.047). This is also in accordance with extant Six Hat theory on Yellow hats as idea supporters, since group brainstorming permits piggy backing and leap-frogging.

Blue hats are characterized by preference for process driven problem solving. It would be expected that Blue hats perform well in 6-3-5 or C-Sketch, because it is a very systematic method. Indeed, the mean performance of Blue hats is higher for quantity (p=0.49), and variety (p=0.25) in 635 than that of Yellow hats, but not significantly. This may be due to the fact that Yellow hats are generally strong ideators and thus comparison to the other hats is required in the future when data is available. However, there was significance (p = 0.036) to the difference of mean scores for self-efficacy with Blue hats reporting an average indication that that they ideated more effectively than Yellow hats in C-Sketch. On a five point Likert scale, the Blue hats listed an average ‘4.0’ equivalent to ‘Agree’ that their ideas had high quality, quantity, and novelty during C-Sketch. Yellow hats only listed ‘3’ or “Neutral” for C-Sketch performance. The remainder of self-efficacy results can be seen in Table 7. Other comparisons in Table 7 also agree with the quantitative results.

Table 7: Self efficacy for Six Hats. Range is from 1 to 5, where 5 is strong agreement.

Self-Efficacy for High Quality, Novelty and Variety of Ideas in the Method

P value Yellow Blue

Brainstorming Methods

0.360 3.1 3

Mind-Mapping methods

0.220 3.2 3

C-Sketch 0.035 3 4

5. CONCLUDING REMARKS The results of this paper provide a clear indication that there are significant differences in the ideation results of different personality types across a set of ideation methods. It is shown that these significant comparisons match what would be expected from the theory of types. For example, Jung’s cognitive modes for decision-making and information gathering can both be used to interpret the characteristics of ideation results. This type of comparison allows a deeper understanding of ideation suites. On one hand, an emphasis on one ideation technique will not fully explore the potential of a group of individuals with differing communication and decision skills, and their ideation preferences. On the other hand, the results open consideration of new ideation methods or sequencing of methods that would leverage the characteristics of each communication style simultaneously. 6. LIMITATIONS AND FUTURE WORK The personality type indicators have been applied in a variety of team formation, management, and psychology contexts; however, this study was only able to evaluate the characteristics of MBTI and Six hats indicators as compared to ideation method in a single design problem. In support of our approach is the consideration that this design problem was a highly multidisciplinary one, touching on nano-technology, controls, kinematics, interaction design, and programming. Additionally, the type indicators may have some imprecision in evaluating personality type. The objective of our study was to explore trends across a set of individuals and therefore attempt to reduce any effects that might be a reflection of the individual. In regards to participants, there were not enough

Quantity

Quality

Variety

Figure 8 Total averages for all participants across the ideation session. Error bars are ±1 standard error. The vertical axis is performance in each metric (see section

3.7) The horizontal axis is the method code: I = individual, G = group; BS = brainstorming, MM =

mindmapping, and 635 = C-Sketch

11

individuals of certain types and a number of unanswered questions remain regarding the properties of those types for which we did not have sufficient data. Finally, there are potentially a number of other analyses and cross-comparisons which could be developed using data from this workshop such as technical skill sets and other personality type assessments, but this would exceed the space allotted to this paper to properly examine.

As with any psychological study a primary objective of future work is to increase sample size. Additionally we hope to explore the inter-relations with other aspects of the design process (such as prototyping) and personality type. It could also be useful to allow participants significantly more time to ideate. It has been shown that some ideation methods permit continual production of ideas if given a longer span of time. In general, we find that this study was a fruitful and intriguing look into the comparison of personality and ideation and find this research area to be open for continued efforts. ACKNOWLEDGEMENTS This work is supported by the Singapore University of Technology and Design (SUTD) and the SUTD-MIT International Design Center (IDC, idc.sutd.edu.sg). This project is partially supported by China’s Natural Science Foundation, project number: 70971073. Additionally, this work is made possible by collaboration with Tsinghua University in Beijing, and Center for Nano and Micro Mechanics. The authors would also like to thank Peking University and University College London participants for their design efforts and patience in this study. REFERENCES [1] Shah, J. J., Smith, S. M., and Vargas-Hernandez, N., 2003,

“Metrics For Measuring Ideation Effectiveness,” Design Studies, 24(2), pp. 111-134.

[2] Jensen, D., Feland, J., Bowe, M., and Self, B., 2000, “A 6-Hats Based Team Formation Strategy: Development And Comparison With An MBTI Based Approach,” In Proceedings of the ASEE Annual Conference, Session 2425.

[3] Wilde, D. J., 2008, Teamology: The Construction And Organization Of Effective Teams, Springer.

[4] Myers, I. B., McCaulley, M. H., and Most, R., 1985, Manual: A Guide To The Development And Use Of The Myers-Briggs Type Indicator, Consulting Psychologists Press, Palo Alto, CA.

[5] DeBono, E., 1985, Six Hats Thinking, Little, Brown and Co, Boston, MA.

[6] Linsey, J., Green, M. G., Murphy, J. T., Wood, K. L., and Markman, A. B., 2005, “Collaborating To Success: An Experimental Study Of Group Idea Generation Techniques,” DETC2005-85351, In ASME Design Theory and Methodology Conference, pp. 24-28.

[7] Linsey, J. S., Clauss, E. F., Kurtoglu, T., Murphy, J. T., Wood, K. L., and Markman, A. B., 2011, “An Experimental Study Of Group Idea Generation Techniques: Understanding The Roles Of Idea Representation And Viewing Methods,” Journal of Mechanical Design, 133(3), 031008.

[8] Puccio, G., and Grivas, C., 2009, “Examining The Relationship Between Personality Traits And Creativity Styles,” Creativity and Innovation Management, 18(4), pp. 247-255.

[9] Durling, D., Cross, N., and Johnson, J., 1996, “Personality And Learning Preferences Of Students In Design And Design-Related Disciplines,” In IDATER 1996 Conference, Loughborough University, Loughborough.

[10] Chan, J., Fu, K., Schunn, C., Cagan, J., Wood, K., and Kotovsky, K., 2011, “On The Benefits And Pitfalls Of Analogies

For Innovative Design: Ideation Performance Based On Analogical Distance, Commonness, And Modality Of Examples,” Journal Of Mechanical Design, 133, 081004.

[11] White, C., Wood, K., and Jensen, D., 2012, “From Brainstorming to C-Sketch to Principles of Historical Innovators: Ideation Techniques to Enhance Student Creativity,” Journal of STEM Education, 13(5), pp. 12-25.

[12] Osborn, A. F., 1957, Applied Imagination, Scribner, New York. [13] Diehl, M., and Stroebe W., 1987. “Productivity Loss in

Brainstorming Groups: Toward the Solution of a Riddle,” Journal of Personality and Social Psychology, 53(3), pp. 497-509.

[14] Otto, K., and Wood, K., 2001, Product Design: Techniques In Reverse Engineering And New Product Development, Prentice Hall, Upper Saddle River, NJ.

[15] Shah, J. J., Vargas-Hernandez, N. O. E., Summers, J. D., and Kulkarni, S., 2001, “Collaborative Sketching (C-Sketch) – An Idea Generation Technique For Engineering Design,” The Journal of Creative Behavior, 35(3), pp. 168-198.

[16] Jensen, D., Wood, J., and Wood, K., 2003, “Hands-On Activities, Interactive Multimedia And Improved Team Dynamics For Enhancing Mechanical Engineering Curricula,” International Journal of Engineering Education, 19(6), pp. 874-884.

[17] Shen, S. T., Prior, S. D., White, A. S., and Karamanoglu, M., 2007, “Using Personality Type Differences To Form Engineering Design Teams,” Engineering Education, 2(2), pp. 54-66.

[18] Carl III, W. J., 1996, “Six Thinking Hats: Argumentativeness and Response to Thinking Model.”

[19] Bandura, A., 1971, Social Learning Theory, General Learning Corporation, New York.

[20] Locke, E. A., Frederick, E., Lee, C., and Bobko, P., 1984, “Effect Of Self-Efficacy, Goals, And Task Strategies On Task Performance,” Journal Of Applied Psychology, 69(2), pp. 241-251.

[21] Gu, X. Y., Wang, D, Y., Zhou, S. Y., Yang, F. F, Lu, D. R., 2013, “Distributed Learning Workflow: An Operating System that Integrates Information Technologies and the Real Campus” Research in Higher Education of Engineering, 2013(2), pp. 72-89.

Additional Resources: self efficacy Bandura, A., & Schunk, D. H. (1981). Cultivating competence, self-

efficacy, and intrinsic interest through proximal self-motivation. Journal of personality and social psychology, 41(3), 586.

Bandura, A., & Schunk, D. H. (1981). Cultivating competence, self-efficacy, and intrinsic interest through proximal self-motivation. Journal of personality and social psychology, 41(3), 586.

Bandura, A. (1982). Self-efficacy mechanism in human agency. American psychologist, 37(2), 122.

Bandura, A. (1994). Self-efficacy. John Wiley & Sons, Inc.. Carberry, A. R., Lee, H. S., & Ohland, M. W. (2010). Measuring

Engineering Design Self-Efficacy. Journal of Engineering Education, 99(1), 71-79.

Schunk, D. H. (1995). Self-efficacy, motivation, and performance. Journal of Applied Sport Psychology, 7(2), 112-137.

Zimmerman, B. J. (2000). Self-efficacy: An essential motive to learn. Contemporary educational psychology, 25(1), 82-91.

Pintrich, P. R., & De Groot, E. V. (1990). Motivational and self-regulated learning components of classroom academic performance. Journal of educational psychology, 82(1), 33.

Tsenn, J., McAdams, D., Linsey, J., “A Comparison of Design Self-Efficacy of Mechanical Engineering Freshmen, Sophomores, and Juniors”, In Proceedings of the 2013 American Society for Engineering Education Annual Conference and Exposition, Atlanta GA

ideation Blair, B. M., & Hölttä-Otto, K. (2012, August). Comparing the

Contribution of the Group to the Initial Idea in Progressive Idea

12

Generation. In ASME 2012 International Design Engineering Technical Conferences and Computers and Information in Engineering Conference (pp. 425-436). American Society of Mechanical Engineers.

Fu, Katherine, et al. "The Meaning of “Near” and “Far”: The Impact of Structuring Design Databases and the Effect of Distance of Analogy on Design Output." Journal of Mechanical Design 135 (2013): 021007.

Linsey, J. S., Markman, A. B., & Wood, K. L. (2012). Design by analogy: A study of the wordtree method for problem re-representation. Journal of Mechanical Design, 134, 041009

creativity metrics Oman, S. K., Tumer, I. Y., Wood, K., & Seepersad, C. (2013). A

comparison of creativity and innovation metrics and sample validation through in-class design projects. Research in Engineering Design, 24(1), 65-92.

Shah, J. J., Kulkarni, S. V., & Vargas-Hernandez, N. (2000). Evaluation of idea generation methods for conceptual design: effectiveness metrics and design of experiments. Journal of Mechanical Design, 122, 377.

interaction and personality traits Carr, P. G., De La Garza, J. M., & Vorster, M. C. (2002).

Relationship between personality traits and performance for engineering and architectural professionals providing design services. Journal of Management in Engineering, 18(4), 158-166.

Feijs, L., Kyffin, S., & Young, B. (2005, November). Design and semantics of form and movement. In DeSForM Workshop.

De Bono, E. (1995). Serious creativity. Harper Business. De Bono’s, E. (2012). Six Thinking Hats™. Meneely, J., & Portillo, M. (2005). The adaptable mind in design:

Relating personality, cognitive style, and creative performance. Creativity Research Journal, 17(2-3), 155-166.

Richardson, A. L. (2008). Tinkering interactions on freshman engineering design teams. In Proceedings of the 2008 American Society for Engineering Education Annual Conference and Exposition.

Ullman, D. G. (1992). The mechanical design process (Vol. 2). New York: McGraw-Hill.