windows speech platform and cortana extension · pdf file•two test criteria defined, with...

• Two test criteria defined, with focus on Standard at Win10 RTM

Standard – Device meets recommended functional guidelines for working great with Speech Recognition.

Premium – Device meets basic functional guidelines for working with Speech Recognition, such as working well in less optimal environments (e.g. background noise, distance)

• Cortana and the Microsoft Speech Platform features do not have an associated Windows Certification program


• Speech Recognition Quiet test

The quiet tests represent an ideal environment with minimal ambient noise (noise floor < 35 dBA

SPL).

• Speech Recognition ambient test

The ambient noise tests represent various levels and types of noisy environments, e.g. Café & Pub

• Speech Recognition Echo test

The echo noise tests represent various levels and types of render playback scenarios (e.g. media

playing).


http://aka.ms/es202-396-1

CORTANA TEST REQUIRE ETSI Test Room ETSI EG 202 396-1 (Section 6)

• Test room size should be in range between

2.7m X 3.7m and 3.5m X 4.4m, height should

be between 2.2m and 2.5.

• Noise floor must be < 35 dB SPL(A), target 28

dB SPL(A)

• Reverberation time of the room should be

0.4s<RT<0.7s in frequency range between

100Hz and 8kHz


http://www.etsi.org/deliver/etsi_eg/202300_202399/20239601/01.02.02_60/eg_20239601v010202p.pdf

Cortana Test ROOM-CONT’


DUT Test Position


DUT Test Position-CONT’


Test EquipmentItem Recommendations Example

Head and Torso Simulator

and/or Mouth Simulator

Compliant with ITU-T P.58

Compliant with ITU-T P.51

B&K 4128C

G.R.A.S. 44AB , B&K 4227

or NTi TalkBox

Amplifier (if HATS used) Compatible with HATS used B&K Nexus or B&K 2716C

Audio Generator/Analyzer Fulfil test signal requirements AP 585

Room loudspeakers &

Stands

JBL LSR2325P & SMS-6000

Reference Free Field

Microphone

Or Sound Level Meter

100 Hz – 12 kHz, < 1% THD 94 dB

SPL @ MRP, ± 2 dB on axis response

DPA 4007

B&K 2240 or NTi Xl2 with M4260

PC Audio Interface RME 9632 or Roland Octa-Capture


http://aka.ms/bk4128c

http://aka.ms/gras44ab

http://aka.ms/bk4227

http://aka.ms/bk2240

Test Signal, Files, SoftwareStimulus Length Level Frequencies Other Parameters

Input Sweep 3s 94 dB SPL @ MRP 100 – 12000 Hz Continuous, Log Sweep

Speech Input ~ 20 min 89 dB SPL @ MRP 100 – 12000 Hz LongCleanTalk-

CortanaSubset_48k_24bit.wav

Music File >= 70 dBA SPL @LRP Local playback for echo test

Ambient Noise >= 57 dBA SPL @ DUT Café, Pub

from ETSI ES 202 396-1

Calibration

(mouth)

100 – 12000 Hz LongCleanTalk-Calibration.wav

OEMDriverVerifytool Check Audio Driver Configuration

Recording Tool Record Audio Speech

Score Tool Score


http://aka.ms/es202-396-1

• Prepare the normal ETSI room with any objects needed for

test scenarios

• Calibrate the reference mic sensitivity according to

manufacturer guidelines if use reference microphone

• Calibrate the HATS (or mouth simulator) to flat magnitude

response according to manufacturer guidelines


Two Speech files are provided• LongCleanTalk-CortanaSubset_48k_24bit.wav is used in Cortana test• LongCleanTalk-Calibration.wav is near 8 minutes and used to calibrate

Talker Level

Calibration File

Clean Talk Speech Input File

Long

Clean TalkLongCleanTalk-

Calibration.wav

LongCleanTalk-

CortanaSubset_48k_24bit.wav


• Ambient Noise Calibration

Play the background noise file and increase the averaging time on the reference

mic (or sound level meter) until it is steady

Change the level of the background noise to be >= 57dB SPL

• Echo Noise Calibration

Play the music file and increase the averaging time on sound level meter (or the

reference mic) until it is steady.

Change the playback level on the DUT until >= 70dB SPL @ LRP is reached (or the

DUT max playback level is reached)


DriverConfigurationVerification


OEMVerification: Version 1.4 Target Device: Microphone Array (Realtek High Definition Audio) Driver Mode Support: Raw: supported Speech: supported Number of Channels: 2 Array Type: Linear Beam Frequency Range: 120 ~ 7500hz Beam Target H. Range: 30 ~ -30deg Beam Target V. Range: 90 ~ -90deg Microphone Count: 2 Mic0: Type=Omni Location: (0, -23, 0) mm H. Angle: 0 deg V. Angle 0 deg Mic1: Type=Omni Location: (0, 22, 0) mm H. Angle: 0 deg V. Angle 0 deg Distance between Mic0 and Mic1 is 4.50 cm Effects: AcousticEchoCancellation NoiseSuppression BeamForming Feedback: Pipeline Indication: OEM provided expected to be used Default Mic Gain (from registry): 64.01% (0x1901)

Get Score Result

Visual C++ Redistributable Packages for Visual Studio 2013


mailto:https://www.microsoft.com/en-us/download/details.aspx?id=40784

Transcription:Any news about mortgage

Recognition: ortona how far away is

Transcription:Can you play Billy Joel Piano Man

Transcription:Cortana how far away is the moon

Recognition: porn

Transcription:Average water bill in Belmont California

Recognition: nine zero five eight one nine five

Transcription:Change the current date and time

Recognition: zero zero zero

Transcription:nine zero five eight one nine five zero zero zero

Recognition: weather clear up on the big island

Transcription:Did the weather clear up on the big island

Recognition: a note to myself

Transcription:august ninth

Recognition: remind me to get butter

Transcription:A note to myself

Recognition: what time

Transcription:Remind me to get butter at Safeway

Recognition: one six two five four

Transcription:Cortana play some blues

Recognition: five six five zero

Transcription:one six two five four five six five zero

Transcription:Any news about mortgage

Recognition: any news about mortgage

Transcription:Can you play Billy Joel Piano Man

Recognition: can you play billy joel piano man

Transcription:Cortana how far away is the moon

Recognition: cortana how far away is the moon

Transcription:Average water bill in Belmont California

Recognition: average water bill in belmont california

Transcription:Change the current date and time

Recognition: change the current date and time

Transcription:nine zero five eight one nine five zero zero zero

Recognition: nine zero five eight one nine five zero zero zero

Transcription:Did the weather clear up on the big island

Recognition: did the weather clear up on the big island

Transcription:august ninth

Recognition: august night

Transcription:A note to myself

Recognition: a note to myself

Transcription:Remind me to get butter at Safeway

Recognition: remind me to get butter at safeway

Transcription:Cortana play some blues

Recognition: time to play some blues


Result Analysis - CONT’


(c) 2015 Microsoft Corporation. All rights reserved. This document is provided "as-is." Information and views

expressed in this document, including URL and other Internet Web site references, may change without notice. You

bear the risk of using it. This document does not provide you with any legal rights to any intellectual property in any

Microsoft product. You may copy and use this document for your internal, reference purposes.

Some information relates to pre-released product which may be substantially modified before it’s commercially

released. Microsoft makes no warranties, express or implied, with respect to the information provided here.

windows speech platform and cortana extension · pdf file•two test criteria defined, with...

Documents