The Hurricane Challenge

Increasing the intelligibility of natural and synthetic speech

purpose | procedure | data | important dates | organisers | why Hurricane?

Purpose

Do you have an algorithm for the modification of natural or synthetic speech which improves intelligibility in known noise conditions? If so, you are invited to take part in the Hurricane Challenge, a co-ordinated international evaluation of modified speech intelligibility.

While the listener evaluation for the Challenge is now complete, new participants are still welcome to download the data.

You will be provided with a corpus of recorded sentences along with separate noise signals at a number of signal-to-noise ratios. Your task is to modify the speech only in such a way to promote its intelligibility. Modifications will be expected to meet constraints on changes in RMS level as well as durational constraints.

Results of the Challenge will be disseminated at a Special Session of Interspeech 2013.

Outline procedure

The data

  • Unmodified ('plain') sentences from the Harvard corpus, spoken by a male British English talker
  • Two maskers: speech-shaped noise and competing speech from a single talker
  • Each masker/speech combination presented at 3 SNRs
  • 180 sentences in each of the 6 conditions (2 maskers x 3 SNRs)
  • Maskers lead and lag the plain speech by 0.5s

Your task and constraints

  • You may modify the speech signals in any way, including durational changes up to a maximum of one second (to fit within the lead/lag of the masker)
  • You may use the noise signals to decide on your speech modifications
  • You will return only the modified speech signals to us

What we will then do

  • We will rescale the speech to meet constraints on RMS energy and remix the scaled speech and maskers at the specified SNRs
  • Modifications will be presented to a large cohort of native British English listeners
  • We will return your individual evaluation results to you along with results for the unmodified ('plain') speech. Minimally, your results will include:
    • raw listener responses
    • keywords correct scores
    • gains expressed as dBs over unmodified speech

Special notes for synthetic speech entries

  • We can provide additional speech data to help you to train or adapt models for the target talker
  • We will provide text for the target sentences
  • If there are sufficient entries, we will evaluate synthetic speech separately from natural speech
  • For additional information on synthetic speech, please contact Cassia Valentini-Botinhao

Getting the data

To receive details of the plain speech and masker signals, as well as more detailed instructions, please send a request to Martin Cooke.

Important dates

8th October 2012Task materials available
31st October 2012Deadline for registration of participation
1st December 2012Deadline for receipt of modified speech
30th January 2013Listener evaluation results returned to participants
18th March 2013Interspeech paper submission closes
August 2013Special session or satellite workshop at Interspeech, Lyon, France

Organisers

Martin Cooke, Ikerbasque & University of the Basque Country, Spain | Catherine Mayo, CSTR, University of Edinburgh, UK | Bastian Sauert, Aachen University, Germany | Yannis Stylianou, FORTH Institute of Computer Science, Crete, Greece | Cassia Valentini-Botinhao, CSTR, University of Edinburgh, UK | Yan Tang, Language and Speech Laboratory, University of the Basque Country, Spain

Why Hurricane?

The name Hurricane was suggested as a logical extension to even-more-adverse conditions of the annual Blizzard Challenge for the evaluation of synthetic speech.

Last updated: 22nd February 2013