encouraging students towards meaningful subjective comparisons
DESCRIPTION
A review of Subjective Comparison techniques commonly used in audio research, and an introduction to methodologies used in the food/beverage industries. These formal research tactics can help inform our classroom comparisons, and encourage our students to make careful comparisons throughout their careers.TRANSCRIPT
encouraging students towards meaningful subjective comparisonsBradford SwansonJuly 27, 2013 - Murfreesboro, TNAES 50th Conference on Audio Education
bradfordswanson.com
encouraging students TO:use their earsbeware of outside stimuliunderstand research methodsincorporate into daily practice
bradfordswanson.com
outlineaudio research methodspractices in other disciplinessample comparison experienceaudio specific recommendations
bradfordswanson.com
subjectivity research in audiocodecsup/downmixingmultichannel techniquesHRTF & spatial perception
bradfordswanson.com
subjectivity research in audiographical representationscomparisons ratings
bradfordswanson.com
sound qualityspeech intelligibility
localization & envelopment
audio research methods2:15 PM - CoE 164 - Paper #21“From Practice to Research and Back Again: Research Skills in Audio Engineering Education” Drs. Leffords & Berg
bradfordswanson.com
audio research methodsITU-R BS.1116-1ITU-R BS.1534-1 ITU-R BS.1284-1semantic differential
bradfordswanson.com
ITU-R BS.1116-1 (ABC/HR)
bradfordswanson.com
Methods for the subjective assessment of small impairments in audio systems
Three stimuliRated by impairmentSingle listen to eachReference Reference = one of the stimuli
ITU-R BS.1534-1 (MUSHRA)
bradfordswanson.com
Method for the subjective assessment of intermediate quality level of coding systems
MUltiple Stimulus with Hidden Reference and Anchor
Three or more stimuliRated by sound qualityMultiple listensReference Reference = one of the stimuliAnchor = one of the stimuli
ITU-R BS.1534-1 (MUSHRA)
bradfordswanson.com
Method for the subjective assessment of intermediate quality level of coding systems
MUltipleStimulus
and
Anchor
HiddenReference
Source: mushraJS
ITU-R BS.1284-1 (general)
General Methods for the Subjective Assessment of Sound Quality
bradfordswanson.com
Single stimuli
ITU-R BS.1284-1 (general)
General Methods for the Subjective Assessment of Sound Quality
bradfordswanson.com
Multiple stimuli
semantic differential
bradfordswanson.com
Source: Marui, A., Martens, W. - “Timbre of nonlinear distortion effects: Perception beyond aptitude beyond “sharpness”?”
ratings along bipolar adjective scales
semantic differential
bradfordswanson.com
9 stimuli randomized through all 12 adjective pairs
Stimuli One
Dark ---------------------- Bright
ratings along bipolar adjective scales
semantic differential
bradfordswanson.com
Stimuli Eight
Dark ---------------------- Bright
9 stimuli randomized through all 12 adjective pairs
ratings along bipolar adjective scales
semantic differential
bradfordswanson.com
Stimuli Eight
Sharp ---------------------- Dull
9 stimuli randomized through all 12 adjective pairs
ratings along bipolar adjective scales
semantic differential
bradfordswanson.com
Source: Marui, A., Martens, W. - “Timbre of nonlinear distortion effects: Perception beyond aptitude beyond “sharpness”?”
9 stimuli assessed using all 12
adjective pairings
ratings along bipolar adjective scales
practices in other disciplinesFormal Sensory Evaluation Psychological TendenciesCultural ConsiderationsCasual/Social Experiences
bradfordswanson.com
sensory evaluation methodsAffective Tests
bradfordswanson.com
“Untrained” ParticipantsQuantative Suvery QuestionairreQualitative Interview or focus group Free response description
sensory evaluation methodsDescriptive Analysis orAttribute Difference Tests
bradfordswanson.com
Highly trained participantsSensory lexicon of defined termsRating intensity or quality on a scale
sensory evaluation methodsDiscrimination or Overall Difference Tests
bradfordswanson.com
Highly trained participantsThree Stimuli (two similar)Whether there is a difference and/or
How much of a difference there is
sensory evaluation methodsDiscrimination or Overall Difference Tests
bradfordswanson.com
A - B - Awhich is the odd one out?
sensory evaluation methodsDiscrimination or Overall Difference Tests
bradfordswanson.com
A (ref.) - A - B or
B (ref.) - A - B
which is most/least like the reference?
sensory evaluation methodsDiscrimination or Overall Difference Tests
bradfordswanson.com
A (ref.) - A - B or
B (ref.) - A - B
how much difference is there between these samples?
psychological tendenciesExpectation ErrorsSuggestion EffectsDistraction ErrorsStimulus/Logical ErrorsHalo Effects/Proximity ErrorsCarry-over, Order or Contrast Effects
bradfordswanson.com
cultural considerationsPersonal PreferencesBackground ExperienceAccessibility
bradfordswanson.com
sensory evaluation methodsExperimental Design
bradfordswanson.com
Identify test objectivesDetermine statistical analysis methodDefine the terminology usedProvide a controlConsider screening order
subjective listening experience15 examples Varying timbral qualitiesMark your perception
bradfordswanson.com
SUBJECTIVE LISTENING EXPERIENCETest Objective
bradfordswanson.com
develop critical listening skillscompare timbral perception of equipment
SUBJECTIVE LISTENING EXPERIENCEStatistical Analysis
bradfordswanson.com
Not collected - for personal use
SUBJECTIVE LISTENING EXPERIENCEDefine the terminology
bradfordswanson.com
Timbe:Content = ContentPresentation = PresentationPitch = PitchLoudness = LoudnessSomething = Something
SUBJECTIVE LISTENING EXPERIENCEDefine the terminology
bradfordswanson.com
Timbe:Harmonic Content or Frequency Spectrum
SUBJECTIVE LISTENING EXPERIENCEDefine the terminology
bradfordswanson.com
Timbe:Content = ContentPresentation = PresentationPitch = PitchLoudness = LoudnessSomething = Something
SUBJECTIVE LISTENING EXPERIENCEConsider Screening Order
bradfordswanson.com
?
subjective listening experience15 examples Varying timbral qualitiesMark your perception
bradfordswanson.com
subjective listening experience
bradfordswanson.com
“uncolored”or
natural reproduction of the harmonic content of this voice
subjective listening experience
bradfordswanson.com
“colored”or
significant deviations from the natural harmonic content of this voice
subjective listening experience
bradfordswanson.com
colored uncolored
l------l------l------l------l------l
subjective listening experience
bradfordswanson.com
what the heck was that?
SUBJECTIVE LISTENING EXPERIENCEConsider Screening Order
bradfordswanson.com
RandomisationBalanced Test Design
*5 stimuli*each presented 3 times*in different orders
subjective listening experience
bradfordswanson.com
What the heck was that?Shakespeare’s 29th SonnetPerformed onceBob Kuhn (bkvoice.com)5 microphones - 3 times eachReferences included as stimuliSimilar signal flow
subjective listening experience
bradfordswanson.com
Colored referenceExamples 4, 8, and 15
Shure 55sw
subjective listening experience
bradfordswanson.com
Examples 3, 9, and 14
Beyer M-88
subjective listening experience
bradfordswanson.com
Examples 1, 7, and 10
Sennheiser MD-409
subjective listening experience
bradfordswanson.com
Examples 2, 6, and 12
AKG C-535
subjective listening experience
bradfordswanson.com
Uncolored referenceExamples 5, 11, and 13
DPA 4006
subjective listening experience
bradfordswanson.com
1) MD-4092) C-5353) M-884) 55sw5) 40066) C-5357) MD-4098) 55sw
9) M-8810) MD-40911) 400612) C-53513) 400614) M-8815) 55sw
Casual/Social experiences
“it’s a wine-tasting,
bradfordswanson.com
not a shootout” Tom Bates
recommendationsExperimental Design
bradfordswanson.com
Define GoalsDefine VariablesMinimize Distractors
recommendationsMatching non-variables
bradfordswanson.com
PerformancePresentationSignal PathGain
recommendationsListening Experience Design
bradfordswanson.com
Listening EnvironmentsDouble-Blind TestingConduct a pilot sessionListen past the technology
bradfordswanson.com/mic