While creativity is often stressed in the conceptual phases of design, it is rarely considered during the concept selection process. Before effective methods can be developed to aid in creative concept section, however, differences in the perceptions of creativity between expert and novice designers and the influence of creativity evaluation methods on the process must be considered. Therefore, this paper was developed to address these questions by studying 11 expert and 11 novice designers. Specifically the study was developed to understand if experts' and novices' perception of a concepts creativity aligned, to introduce and compare the utility of our tool for assessing semantic creativity (TASC) to existing creativity evaluation methods, and to identify if our TASC method could be used as a proxy for expert evaluators. Our findings reveal that experts and novices generally had similar perceptions of a concept's creativity and that the TASC method was tapping into similar constructs of human perceptions of concept creativity. The results of this study contribute to our understanding of the factors that influence the selection or filtering of creative ideas after idea generation and provide a framework for research in this field.

References

1.
Sarkar
,
P.
, and
Chakrabarti
,
A.
,
2011
, “
Assessing Design Creativity
,”
Des. Stud.
,
32
(
4
), pp.
348
383
.
2.
Chulvi
,
V.
,
Gonzalez-Cruz
,
M. C.
,
Mulet
,
E.
, and
Aguilar-Zambrano
,
J.
,
2012
, “
Influence of Type of Idea-Generation Method on the Creativity of Solutions
,”
Res. Eng. Des.
,
24
(
1
), pp.
33
41
.
3.
Yilmaz
,
S.
,
Seifert
,
C. M.
, and
Gonzalez
,
R.
,
2012
, “
Design Heuristics: Cognitive Strategies for Creativity in Idea Generation
,”
Design Computing and Cognition '10
, J. S. Gero, ed., Springer, New York, pp.
35
53
.
4.
Shah
,
J. J.
,
Vargas-Hernandez
,
N.
, and
Smith
,
S. M.
,
2003
, “
Metrics for Measuring Ideation Effectiveness
,”
Des. Stud.
,
24
(
1
), pp.
111
134
.
5.
Rietzchel
,
E. F.
,
Nijstad
,
B. A.
, and
Stroebe
,
W.
,
2006
, “
Productivity is Not Enough: A Comparison of Interactive and Nominal Groups in Idea Generation and Selection
,”
J. Exp. Soc. Psychol.
,
42
(
2
), pp.
244
251
.
6.
Snider
,
C.
,
Cash
,
P.
,
Dekoninck
,
E.
, and
Culley
,
S.
,
2012
, “
Variation in Creative Behaviour During the Later Stages of the Design Process
,”
ICDC2012
:
The 2nd International Conference on Design Creativity
, University of Bath, pp.
147
156
.
7.
Kudrowitz
,
B. M.
, and
Wallace
,
D.
,
2013
, “
Assessing the Quality of Ideas From Prolific, Early-Stage Product Ideation
,”
J. Eng. Des.
,
24
(
2
), pp.
120
139
.
8.
Fishburn
,
P. C.
,
1970
, “
Utility Theory for Decision Making
,” DTIC Document, Mclean, VA,
Report No. RAC-R-105
.
9.
Von Neumann
,
J.
, and
Morgenstern
,
O.
,
1953
,
Theory of Games and Economic Behavior
, 3rd ed.,
Princeton University Press
,
Princeton, NJ
.
10.
Saaty
,
T. L.
,
1988
,
What is the Analytic Hierarchy Process?
,
Springer
,
Berlin, Germany
.
11.
Saaty
,
T. L.
,
1980
,
The Analytic Hierarchy Process
,
McGraw Hill
,
New York
.
12.
Pugh
,
S.
,
1991
,
Total Design: Integrated Methods for Successful Product Engineering
,
Addison-Wesley
,
Workingham, UK
.
13.
Pugh
,
S.
,
1981
, “
Concept Selection—A Method That Works
,”
International Conference of Engineering Design
, Rome, pp.
497
506
.
14.
Ter Harr
,
S.
,
Clausling
,
D.
, and
Eppinger
,
S.
,
1993
, “
Integration of Quality Function Deployment in the Design Structure Matrix
,” MIT, Cambridge, MA, Working Paper.
15.
Hauser
,
J.
, and
Clausing
,
D.
,
1996
, “
The House of Quality
,”
IEEE Eng. Manage. Rev.
,
24
(
1
), pp.
24
32
.
16.
Ross
,
T.
,
1995
,
Fuzzy Logic With Engineering Applications
,
McGraw Hill
,
West Sussex, U.K
.
17.
Thurston
,
D. L.
, and
Carnahan
,
J. V.
,
1992
, “
Fuzzy Ratings and Utility Analysis in Preliminary Design Evaluation of Multiple Attributes
,”
ASME J. Mech. Des.
,
114
(
4
), pp.
648
658
.
18.
Okugan
,
G. E.
, and
Tauhid
,
S.
,
2008
, “
Concept Selection Methods—A Literature Review From 1980 to 2008
,”
Int. J. Des. Eng.
,
1
(
3
), pp.
243
277
.
19.
Oman
,
S. K.
,
Tumer
, I
. Y.
,
Wood
,
K.
, and
Seepersad
,
C.
,
2013
, “
A Comparison of Creativity and Innovation Metrics and Sample Validation Through In-Class Design Projects
,”
Res. Eng. Des.
,
24
(
1
), pp.
65
92
.
20.
Amabile
,
T. M.
,
1983
, “
The Social Psychology of Creativity: A Componential Conceptualization
,”
J. Pers. Soc. Psychol.
,
45
(
2
), pp.
357
376
.
21.
Brown
,
D.
,
2013
, “
Developing Computational Design Creativity Systems
,”
Int. J. Des. Creativity Innovation
,
1
(
1
), pp.
43
55
.
22.
Besemer
,
S. P.
, and
O'Quin
,
K.
,
1999
, “
Confirming the Three-Factor Creative Product Analysis Matrix Model in an American Sample
,”
Creativity Res. J.
,
12
(
4
), pp.
287
296
.
23.
Kaufman
,
J. C.
,
Baer
,
J.
,
Cole
,
J. C.
, and
Sexton
,
J. D.
,
2008
, “
A Comparison of Expert and Nonexpert Raters Using the Consensual Assessment Technique
,”
Creativity Res. J.
,
20
(
2
), pp.
171
178
.
24.
Fischer
,
G.
,
2013
, “
Learning, Social Creativity, and Cultures of Participation
,”
Learning and Collective Creativity: Activity–Theoretical and Sociocultural Studies
,
Routledge
,
New York
, p.
198
.
25.
Martin
,
M. W.
,
2006
, “
Moral Creativity in Science and Engineering
,”
Sci. Eng. Ethics
,
12
(
3
), pp.
421
433
.
26.
Nelson
,
B. A.
,
Wilson
,
J. O.
,
Rosen
,
D.
, and
Yen
,
J.
,
2009
, “
Refined Metrics for Measuring Ideation Effectiveness
,”
Des. Stud.
,
30
(
6
), pp.
737
743
.
27.
Chulvi
,
V.
,
Mulet
,
E.
,
Chakrabarti
,
A.
,
López-Mesa
,
B.
, and
González-Cruz
,
C.
,
2012
, “
Comparison of the Degree of Creativity in the Design Outcomes Using Different Design Methods
,”
J. Eng. Des.
,
23
(
4
), pp.
241
269
.
28.
Srivathsavai
,
R.
,
Genco
,
N.
,
Holtta-Otto
,
K.
, and
Seepersad
,
C.
,
2010
, “
Study of Existing Metrics Used in Measurement of Ideation Effectiveness
,”
ASME
Paper No. DETC2010-28802.
29.
Redmond
,
M. R.
,
Mumford
,
M. D.
, and
Teach
,
R.
,
1993
, “
Putting Creativity to Work: Effects of Leader Behavior on Subordinate Creativity
,”
Organ. Behav. Hum. Decis. Processes
,
55
(
1
), pp.
120
151
.
30.
Bedell-Avers
,
K.
,
Hunter
,
S. T.
,
Angie
,
A. D.
,
Eubanks
,
D. L.
, and
Mumford
,
M. D.
,
2009
, “
Charismatic, Ideological, and Pragmatic Leaders: An Examination of Leader–Leader Interactions
,”
Leadership Q.
,
20
(
3
), pp.
299
315
.
31.
Hunter
,
S. T.
,
Bedell
,
K. E.
,
Ligon
,
G. S.
,
Hunsicker
,
C. M.
, and
Mumford
,
M. D.
,
2008
, “
Applying Multiple Knowledge Structures in Creative Thought: Effects on Idea Generation and Problem-Solving
,”
Creativity Res. J.
,
20
(
2
), pp.
137
154
.
32.
Brown
,
D. C.
,
2015
, “
Computational Design Creativity Evaluation
,”
Design Computing and Cognition '14
, J. S. Gero and S. Hanna, eds., Springer International Publishing, New York, pp.
207
224
.
33.
Maher
,
M. L.
, and
Fisher
,
D. H.
,
2012
, “
Using AI to Evaluate Creative Designs
,”
2nd International Conference on Design Creativity
, Glasgow, UK, pp.
45
54
.
34.
Gero
,
J. S.
, and
Kannengiesser
,
U.
,
2007
, “
Locating Creativity in a Framework of Designing for Innovation
,”
Trends in Computer Aided Innovation
,
Springer
,
New York
, pp.
57
66
.
35.
Shiffrin
,
R. M.
, and
Schneider
,
W.
,
1977
, “
Controlled and Automatic Human Information Processing: II. Perceptual Learning, Automatic Attending and a General Theory
,”
Psychol. Rev.
,
84
(
2
), pp.
127
190
.
36.
Baddeley
,
A. D.
,
2002
, “
Is Working Memory Still Working?
,”
Eur. Psychol.
,
7
(
2
), pp.
85
97
.
37.
Licuanan
,
B. F.
,
Dailey
,
L. R.
, and
Mumford
,
M. D.
,
2007
, “
Idea Evaluation: Error in Evaluating Highly Original Ideas
,”
J. Creative Behav.
,
41
(
1
), pp.
1
27
.
38.
Green
,
M.
,
Seepersad
,
C.
, and
Hölttä-Otto
,
K.
,
2014
, “
Crowd-Sourcing the Evaluation of Creativity in Conceptual Design: A Pilot Study
,”
ASME
Paper No. DETC2014-34434.
39.
Cross
,
N.
,
2004
, “
Expertise in Design: An Overview
,”
Des. Stud.
,
25
(
5
), pp.
427
441
.
40.
Atman
,
C. J.
,
Adams
,
R. S.
,
Cardella
,
M. E.
,
Turns
,
J.
,
Mosborg
,
S.
, and
Saleem
,
J.
,
2007
, “
Engineering Design Processes: A Comparison of Students and Expert Practitioners
,”
J. Eng. Educ.
,
96
(
4
), pp.
359
379
.
41.
Worsley
,
M.
, and
Blikstein
,
P.
,
2011
, “
What's an Expert? Using Learning Analytics to Identify Emergent Markers of Expertise Through Automated Speech, Sentiment and Sketch Analysis
,”
EDM
, pp.
235
240
.
42.
Ericsson
,
K. A.
, and
Lehmann
,
A. C.
,
1996
, “
Expert and Exceptional Performance: Evidence of Maximal Adaptation to Task Constraints
,”
Annu. Rev. Psychol.
,
47
(
1
), pp.
273
305
.
43.
Jiao
,
J. R.
,
Zhang
,
Y.
, and
Helander
,
M.
,
2006
, “
A Kansei Mining System for Affective Design
,”
Expert Syst. Appl.
,
30
(
4
), pp.
658
673
.
44.
Han
,
S. H.
, and
Hong
,
S. W.
,
2003
, “
A Systematic Approach for Coupling User Satisfaction With Product Design
,”
Ergonomics
,
46
(
13–14
), pp.
1441
1461
.
45.
Chuang
,
M.-C.
, and
Ma
,
Y.-C.
,
2001
, “
Expressing the Expected Product Images in Product Design of Micro-Electronic Products
,”
Int. J. Ind. Ergon.
,
27
(
4
), pp.
233
245
.
46.
Benedek
,
J.
, and
Miner
,
T.
,
2002
, “
Measuring Desirability: New methods for Evaluating Desirability in a Usability Lab Setting
,”
Usability Professionals Association
,
2003
, pp.
8
12
.
47.
Korpershoek
,
H.
,
Kuyper
,
H.
,
Werf
,
G. v. d.
, and
Bosker
,
R.
,
2010
, “
Who ‘Fits’ the Science and Technology Profile? Personality Differences in Secondary Education
,”
J. Res. Pers.
,
44
(
5
), pp.
649
654
.
48.
Choi
,
K.
, and
Jun
,
C.
,
2007
, “
A Systematic Approach to the Kansei Factors of Tactile Sense Regarding the Surface Roughness
,”
Appl. Ergon.
,
38
(
1
), pp.
53
63
.
49.
Childs
,
T.
,
Agouridas
,
V.
,
Barnes
,
C.
, and
Henson
,
B.
,
2006
, “
Controlled Appeal Product Design: A Life Cycle Role for Affective (Kansei) Engineering
,”
LCE2006
, pp.
537
542
.
50.
Prabhakaran
,
R.
,
Green
,
A. E.
, and
Gray
,
J. R.
,
2013
, “
Thin Slices of Creativity: Using Single-Word Utterances to Assess Creative Cognition
,”
Behavior Research Methods
,
Springer
,
New York
, pp.
1
19
.
51.
Fabiani
,
M.
, and
Donchin
,
E.
,
1995
, “
Encoding Processes and Memory Organization: A Model of the Von Restorff Effect
,”
J. Exp. Psychol.
,
21
(
1
), pp.
224
240
.
52.
Snyder
,
K. A.
,
Blank
,
M. P.
, and
Marsolek
,
C. J.
,
2008
, “
What Form of Memory Underlies Novelty Preferences?
,”
Psychon. Bull. Rev.
,
15
(
2
), pp.
315
321
.
53.
Anderson
,
C. J.
,
Glassman
,
M.
,
McAfee
,
R. B.
, and
Pinelli
,
T.
,
2001
, “
An Investigation of Factors Affecting How Engineers and Scientists Seek Information
,”
J. Eng. Technol. Manage.
,
18
(
2
), pp.
131
155
.
54.
Peracchio
,
L. A.
, and
Tybout
,
A. M.
,
1996
, “
The Moderating Role of Prior Knowledge in Schema-Based Product Evaluation
,”
J. Consum. Res.
,
23
(
3
), pp.
177
192
.
55.
Kudrowitz
,
B.
, and
Dippo
,
C.
,
2013
, “
Getting to the Novel Ideas: Exploring the Alternative Uses Test of Divergent Thinking
,”
ASME
Paper No. DETC2013-13262.
56.
Nikander
,
J. B.
,
Liikkanen
,
L. A.
, and
Laakso
,
M.
,
2014
, “
The Preference Effect in Design Concept Evaluation
,”
Des. Stud.
,
35
(
5
), pp.
473
499
.
57.
Lu
,
C.-C.
, and
Luh
,
D.-B.
,
2012
, “
A Comparison of Assessment Methods and Raters in Product Creativity
,”
Creativity Res. J.
,
24
(
4
), pp.
331
337
.
58.
Von Hippel
,
E.
,
1986
, “
Lead Users: A Source of Novel Product Concepts
,”
Manage. Sci.
,
32
(
7
), pp.
791
805
.
59.
Dailey
,
L.
, and
Mumford
,
M. D.
,
2006
, “
Evaluative Aspects of Creative Thought: Errors in Appraising the Implications of New Ideas
,”
Creativity Res. J.
,
18
(
3
), pp.
385
390
.
60.
Toh
,
C.
, and
Miller
,
S. R.
,
2013
, “
Product Dissection or Visual Inspection? The Impact of Designer–Product Interactions on Engineering Design Creativity
,”
ASME Design Engineering Technical Conferences
, Portland, OR, p. V005T06A011.
61.
Toh
,
C.
,
Miller
,
S.
, and
Okudan Kremer
,
G.
,
2012
, “
Mitigating Design Fixation Effects in Engineering Design Through Product Dissection Activities
,”
Design Computing and Cognition '12
, J. S. Gero, ed., Springer, New York, pp.
95
113
.
62.
Miller
,
S. R.
,
Bailey
,
B. P.
, and
Kirlik
,
A.
,
2014
, “
Exploring the Utility of Bayesian Truth Serum for Assessing Design Knowledge
,”
Hum. Comput. Interact.
,
29
(
5–6
), pp.
487
515
.
63.
Williams
,
D.
,
Kelly
,
G.
, and
Anderson
,
L.
, “
MSN 9: New User-Centered Desirability Methods Produce Compelling Visual Design
,”
CHI'04
Extended Abstracts on Human Factors in Computing Systems
, ACM, New York, pp.
959
974
.
64.
Kolb
,
P.
, “
Experiments on the Difference Between Semantic Similarity and Relatedness
,”
17th Nordic Conference on Computational Linguistics
-
NODALIDA’09
, pp.
1
8
.
65.
Kolb
,
P.
,
2008
, “
DISCO: A Multilingual Database of Distributionally Similar Words
,”
KONVENS-2008
,
Berlin
, pp.
1
12
.
66.
Linsey
,
J. S.
,
Clauss
,
E. F.
,
Kurtoglu
,
T.
,
Murphy
,
J. T.
,
Wood
,
K. L.
, and
Markman
,
A. B.
,
2011
, “
An Experimental Study of Group Idea Generation Techniques: Understanding the Roles of Idea Representation and Viewing Methods
,”
ASME J. Mech. Des.
,
133
(
3
), p.
031008
.
67.
Wells
,
J. D.
,
Campbell
,
D. E.
,
Valacich
,
J. S.
, and
Featherman
,
M.
,
2010
, “
The Effect of Perceived Novelty on the Adoption of Information Technology Innovations: A Risk/Reward Perspective
,”
Decis. Sci.
,
41
(
4
), pp.
813
843
.
68.
Endicott
,
J.
,
Spitzer
,
R. L.
,
Fleiss
,
J. L.
, and
Cohen
,
J.
,
1976
, “
The Global Assessment Scale: A Procedure for Measuring Overall Severity of Psychiatric Disturbance
,”
Arch. Gen. Psychiatry
,
33
(
6
), pp.
766
771
.
69.
Chiou
,
C.-F.
,
Hay
,
J. W.
,
Wallace
,
J. F.
,
Bloom
,
B. S.
,
Neumann
,
P. J.
,
Sullivan
,
S. D.
,
Yu
,
H.-T.
,
Keeler
,
E. B.
,
Henning
,
J. M.
, and
Ofman
,
J. J.
,
2003
, “
Development and Validation of a Grading System for the Quality of Cost-Effectiveness Studies
,”
Med. Care
,
41
(
1
), pp.
32
44
.
70.
Joyce
,
M.
, and
Kirakowski
,
J.
,
2014
, “
Measuring Confidence in Internet Use: The Development of an Internet Self-efficacy Scale
,”
Design, User Experience, and Usability. Theories, Methods, and Tools for Designing the User Experience
,
Springer
,
New York
, pp.
250
260
.
71.
Landis
,
J. R.
, and
Koch
,
G. G.
,
1977
, “
The Measurement of Observer Agreement for Categorical Data
,”
Biometrics
,
33
(
1
), pp.
159
174
.
72.
Chan
,
S.
,
1982
, “
Expert Judgments Made Under Uncertainty: Some Evidence and Suggestions
,”
Soc. Sci. Q.
,
63
, pp.
428
444
.
73.
Anderson
,
J. R.
,
1987
, “
Skill Acquisition: Compilation of Weak-Method Problem Situations
,”
Psychol. Rev.
,
94
(
2
), pp.
192
210
.
74.
Bollegala
,
D.
,
Matsuo
,
Y.
, and
Ishizuka
,
M.
,
2011
, “
A Web Search Engine-Based Approach to Measure Semantic Similarity Between Words
,”
IEEE Trans. Knowl. Data Eng.
,
23
(
7
), pp.
977
990
.
75.
Wu
,
W.
,
Luther
,
K.
,
Pavel
,
A.
,
Hartmann
,
B.
,
Dow
,
S.
, and
Agrawala
,
M.
,
2013
,
CrowdCritter: Strategies for Crowdsourcing Visual Design Critique
.
76.
Burnap
,
A.
,
Ren
,
Y.
,
Papalambros
,
P. Y.
,
Gonzalez
,
R.
, and
Gerth
,
R.
,
2013
, “
A Simulation Based Estimation of Crowd Ability and Its Influence on Crowdsourced Evaluation of Design Concepts
,”
ASME
Paper No. DETC2013-13020.
77.
Rietzschel
,
E.
,
Nijstad
,
B. A.
, and
Stroebe
,
W.
,
2010
, “
The Selection of Creative Ideas After Individual Idea Generation: Choosing Between Creativity and Impact
,”
Br. J. Psychol.
,
101
(
1
), pp.
47
68
.
78.
Mueller
,
J. S.
,
Melwani
,
S.
, and
Goncalo
,
J. A.
,
2011
, “
The Bias Against Creativity: Why People Desire but Reject Creative Ideas
,”
Psychol. Sci.
,
2011
, p.
0956797611421018
.
You do not currently have access to this content.