UPDATED-When good demos go (very, very) bad

July 28, 2006

(updated July 31) Microsoft’s failed voice recognition product demonstration at last Thursday’s Financial Analyst Meeting came full circle on Monday when Larry Osterman, a 22-year veteran Microsoft developer admitted on his blog that he was responsible for the bug that led to the on-stage meltdown. In a disarmingly confessional post entitled “Wait, that was my bug? Ouch!” Osterman describes the spectacle and then adds ”…and it was my fault.” Wait a second. Someone in the computer industry just took personal responsibility for messing up?Eleven years after Microsoft co-founder Bill Gates in his 1995 book “The Road Ahead“ predicted humans would one day talk to their computers rather than have to type, the future appeared to be at hand.At Microsoft’s annual Financial Analyst Meeting on Thursday, Vista product manager Shanen Boettcher set out to show just how easy to use the speech recognition technology built into upcoming Windows Vista software will be. Like, for example, dictating aloud a simple, heartfelt letter to mom, and having one’s voice automatically transcribed into a computer.Voice Rec demoThe result was a disaster.Several tries at making the computer understand the simple salutation “Dear Mom” was read by Microsoft software as “Dear Aunt, let’s set so double the killer delete select all.” Attempts to correct or undo or delete the error only deepened the mess.It was not just a perfect refutation of the problems of making machines understand human speech. What other features of Microsoft Windows Vista pose trouble, the audience was left to wonder? “The crashing demo didn’t do a lot to instill confidence in the new Windows product,” one Wall Street analyst, who was present at the demo, said.Windows Vista, already five years in the making, has been postponed by Microsoft several times. Delays have put off the consumer version of Windows until early 2007 — after the crucial holiday shopping season. Vista is scheduled to ship to corporate customers this November, that is, unless more problems are uncovered.Later, Microsoft CEO Steve Ballmer blamed the failed speech recognition product demonstration on “a little bit of echo” in the room, which confused the speech-to-text system. To be sure, a second demonstration during the meeting showed how effectively speech recognition can be for navigating around applications, like Microsoft Outlook.Structured menus appear to work fine. But recognizing random, natural speech still has quite a ways to go, by all appearances: “Let’s set so double the killer delete select all.”– Additional reporting by Daisuke Wakabayashi.Read the more than 2,800 comments on Digg…Buyer beware, but several posters are offering T-shirts to commemorate the event. (1), (2), (3) Here is the YouTube link to various videos of the demo gone awry.

Comments

This is something that is slowly becoming standard with Windows, things just don’t work. How many things have been ditched from Windows Vista since it’s announcement ? Way too many.

I switched to Mac almost two years ago and love every minute spend with it, the things like this simply don’t happen.

Rest in peace Microsoft, I thing you are well beyond the point of no return.

 

Honestly though, it’s not that random natural speech recognition isnt ready yet, its just not ready yet with microsoft’s stuff. Apple’s Macintosh can take speech regonition insanely well, and even on the PC, programs like Dragon Naturally Speaking work fine. Microsoft may be biting off more than it can chew..

 

[...] Reuters: At Microsofts annual Financial Analyst Meeting on Thursday, Vista product manager Shanen Boettcher set out to show just how easy to use the speech recognition technology built into upcoming Windows Vista software will be. Like, for example, dictating aloud a simple, heartfelt letter to mom, and having ones voice automatically transcribed into a computer. [...]

 

[...] At Micrsoft âs annual Financial Analyst Meeting on Thursday, Vista product manager Shanen Boettcher set out to show just how easy to use the speech recognition technology built into upcoming Windows Vista software will be. But something went very wrong…read more | digg story [...]

 

That’s awespme – it might be interesting to try telling the Speech Recognition program the same thing multiple times on multiple programs (like Word, Excel, OpenOffice.org, whatever) and see if it gives the same output every time in each program. You might get some really funny results!

 

cant stop laughing , lol

Posted by Dhruv | Report as abusive
 

Obviously the words I have echoed on my sit and others, continue to ring true. I have had more than my share and yours, of broken, infested, and completely obliterated Microsoft operating systems. I have one Windows installation and for only one reason; Games. Being an Unreal Tournament 2003 freak is my only excuse. As for my everyday routine, I use
Suse 10.1, Suse Enterprise Desktop 10, & of course PC-BSD Unix. Sorry Bill but your OS is flopping as does your support team when offered an Atheros wireless driver for your 64 bit XP Pro. Thats correct. They shrugged me of with the infamous,”we appreiate your comments and your email has been sent to the appropriate parties. Enjoy it now Bill, while you still can!!

 

[...] Read the full article here. [...]

 

oh, please, it’s always something with microsoft. the echo of the room made it goof up. seems that if you have a microphone 1 inch from your mouth, would not be affected by echo of the room.

sorry, steve, but, room dynamics won’t make your software goof-up.

good story stick with it, just admit that your software doesn’t work and you should have checked it out before making the demo live.

Posted by Bob | Report as abusive
 

BEAUTIFUL,

Now I like Microsoft, but I just can’t get over how they make mistakes like this. It’s like they unintentionally “National Lampoon” the presentation, I’ve grown to like these funny mistakes they make, yeah sounds wierd, but ever since the first “Lampoon” by Mr’ Gates I just couldn’t get enough. So let the records show…

Dear Aunt, lets set so double the killer delete select all.

Is now part of Microsoft history!

You know… that sorta sounds like a phrase from some up and coming homicidal computer tech murder movie!

PS.
What’s with the stupid “Please add 5 plus 7″ thing??!!!?? :P

Posted by Lawrence | Report as abusive
 
 

[...] When Good demos go (very, very) bad – Reuters Newsblogs [...]

 

This is SO going to be the cause of global thermonuclear war… ;)

 

[...] A recent Windows Vista voice recognition demo goes horribly wrong: Several tries at making the computer understand the simple salutation Dear Mom was read by Microsoft software as Dear Aunt, lets set so double the killer delete select all. Attempts to correct or undo or delete the error only deepened the mess. [...]

 

[...] Vista human speech More egg on Microsoft’s face ‘Delete select all’read more | digg story « Torvalds critical of new GPL draft [...]

 

[...] At Micrsoft’s annual Financial Analyst Meeting on Thursday, Vista product manager Shanen Boettcher set out to show just how easy to use the speech recognition technology built into upcoming Windows Vista software will be. But something went very wrong…read more | digg story [...]

 
 

well…about 4months ago i used windows vista beta 2 and it seemed to work well..but when i went to use the speech program., it seemed to work pretty well, but about 20 mins into using the program it would crash and “hang” windows for about 3mins o more, sad. but true

 

I am not enthralled by Windows; however, to be fair, it is a system that works pretty well most of the time and I’m also quite sure that when Vista does appear for sale to consumers, that it will be a system that those at Microsoft can be proud of.

A lot of people really love to try to crucify anything: Windows just is an excuse. Nothing is perfect, nor will it ever be.

Posted by Ed Evans | Report as abusive
 

whats this:

Please perform the addition of 3 and 4 (simply put the answer in the box to the left – thanks!)

 

oic – 3 + 4 and now 1 + 2

 

Ha ha! Reminds me of the early Newton handwriting recognition. Of course it got better, but all anyone remembers is the first generation.

 

yea, i’ve used the vista beta and tried the voice stuff. it actually works pretty well, but the documentation cause its a beta isn’t anywhere complete so finding out what the exact commands are to control stuff aren’t easily found. As with betas they change alot of stuff from version to version which goes undocumented so he might’ve gotten confused when the commands he was use to saying got picked up as natural speech and wrote them down. Stuff like this happens.. I think the voice, if they keep it updated and allow for an open (unlikely) developer interface to allow for addons it could really be a nice feature of the OS.

Posted by matt | Report as abusive
 

One more reason I own Apple products. Bloatware no body needs. Besides if I feel like running Microsoft software I’ll just fire up my MacBook. :)

Posted by Dave | Report as abusive
 
 

Have put a video of the whole thing on my blog, plus a hilariously overdubbed Italian Vista speech recognition demo.

Link Rob Chalmers explaining what actually went wrong also on the blog – it wasn’t an echo, apparently…

http://www.geekzone.co.nz/juha/946

 

At the end of the transcript of Ballmer’s speech:

“Due to the varying sound quality and subject matter of tapes, the information in this transcript may contain inaccuracies.”

 
 

[...] That’s the sentence that Windows Vista’s speech recognition wrote in response to an attempt to dictate it a simple “dear mom” letter. [...]

 

you would think windows would have thought of some sort of failsafe to make sure that their demo wouldnt screw up like it did. maybe as simple as restricting its dictionary to the words that would ultimately be used or something like that. it is just odd that vista would fail to recognize spoken words as it did when there are already programs that can do the same thing only much better.

Posted by mike | Report as abusive
 

Lets set so double the killer delete select all.

FINALLY a phase that can replace “All your base are belong to us!”

Posted by Joe | Report as abusive
 

[...] Based on this report from Reuters, I think we can agree that there’s still a tad of tweaking to be done… [...]

 

[...] http://blogs.reuters.com/2006/07/28/when -good-demos-go-very-very-bad/ Eleven years after Microsoft co-founder Bill Gates in his 1995 book The Road Ahead% predicted humans would one day talk to their computers rather than have to type, the future appeared to be at hand. At Microsoft’s annual Financia … [...]

 

Wow just release it in 2020 when its ready

 

This is why i use PCLinuxOS. Or Gentoo, depending on my mood.

Windows is only useful for gaming, but it doesn’t even do that right half of the time. When WINE is perfect, Windows will be useless.

Posted by sonicbhoc | Report as abusive
 

I still can’t stop laughing…

Posted by achan | Report as abusive
 

dragon dictate gave me this gem:
pushy pop-up people pops

Posted by russs | Report as abusive
 

[...] When good demos go (very, very) bad – Reuters Newsblogs [...]

 

[...] That’s what Windows Vista thought product manager Shanen Boettcher said: “Dear Aunt, lets set so double the killer delete select all.” What he DID say was “Dear mom”. Reuters has the story: Several tries at making the computer understand the simple salutation Dear Mom was read by Microsoft software as Dear Aunt, lets set so double the killer delete select all. Attempts to correct or undo or delete the error only deepened the mess. [...]

 

[...] Over the weekend, the wires were full with reports of a speech recognition demo at the Microsoft’s Financial Analysts Meeting here in Seattle that went horribly wrong.  Slashdot had it, Neowin had it,  Digg had it, Reuters had it.  It was everywhere. And it was all my fault.   Well, mostly.  Rob Chambers on the speech team has already written about this, here’s the same problem from my side of the fence. About a month ago (more-or-less), we got some reports from an IHV that sometimes when they set the volume on a capture stream the actual volume would go crazy (crazy, for those that don’t know, is a technical term).  Since volume is one of the areas in the audio subsystem that I own, the bug landed on my plate.  At the time, I was overloaded with bugs, so another of the developers on the audio team took over the investigation and root caused the bug fairly quickly.  The annoying thing about it was that the bug wasn’t reproducible – every time he stepped through the code in the debugger, it worked perfectly, but it kept failing when run without any traces.   If you’ve worked with analog audio, it’s pretty clear what’s happening here – there’s a timing issue that is causing a positive feedback loop that resulted from a signal being fed back into an amplifier. It turns out that one of the common causes of feedback loops in software is a concurrency issue with notifications – a notification is received with new data, which updates a value, updating the value causes a new notification to be generated, which updates a value, updating the value causes a new notification, and so-on… The code actually handled most of the feedback cases involving notifications, but there were two lower level bugs that complicated things.  The first bug was that there was an incorrect calculation that occurred when handling one of the values in the notification, and the second was that there was a concurrency issue – a member variable that should have been protected wasn’t (I’m simplifying what actually happened, but this suffices).    As a consequence of these two very subtle low level bugs, the speech recognition engine wasn’t able to correctly control the gain on the microphone, when it did, it hit the notification feedback loop, which caused the microphone to clip, which meant that the samples being received by the speech recognition engine weren’t accurate. There were other contributing factors to the problem (the bug was fixed on more recent Vista builds than the one they were using for the demo, there were some issues with way the speech recognition engine had been “trained”, etc), but it doesn’t matter – the problem wouldn’t have been nearly as significant. Mea Culpa. [...]

 

[...] At Micrsoft’s annual Financial Analyst Meeting on Thursday, Vista product manager Shanen Boettcher set out to show just how easy to use the speech recognition technology built into upcoming Windows Vista software will be. But something went very very wrong…read more | digg story [...]

 

I wonder if you tell it Lets set so double the killer delete select all. if it will type “Dear Mom,”

As Joe (comment #30) said, i thought of the All your base similarities right away. I can’t wait for the Youtube video and T-shirt.

Posted by faroth | Report as abusive
 

[...]     Microsoft bashers used a failed product demonstration Thursday at the company’s annual Financial Analyst Meeting as an easy excuse to mock the world’s largest software company.     But the response from within Microsoft was anything but that of a corporate monolith. First one developer, then another owned up to the problem in online postings on the company’s developers’ blog. They described in detail what Microsoft already had done to fix the issue.     Over the weekend, Rob Chambers, a member of Microsoft’s speech recognition team, posted a tentative explanation of what he thought might have gone wrong. He provided a more definitive technical answer Monday that pinned the problem on audio gain issue called “clipping.”     “Have you ever heard a car drive by that had the stereo blasting away, and the audio sounded absolutely horrible?” he writes. “Microphones and sound cards can have similar problems trying to convert the analog signal from the microphone element into a digital signal for use by software on the PC.”     Chambers said the glitch was well understood by the team. The demo had worked perfectly well in practice, but because the bug is intermittent, it only popped up during the main event.     “Rest assured that we have the issue under control here in Redmond, and when Vista ships later this year, this audio gain issue will be a thing of the past,” he said.     Larry Osterman, a 22-year Microsoft veteran who is part of the audio team owned up to being slow to detect the software bug when it first was detected about a month ago. “Mea Culpa,” he writes in a blog post entitled “Wait, that was my bug? Ouch!”  [...]

 

[...] Vista demo becomes embarassment [...]

 

here’s my problems with Mac. they control all the hardware. I have a dell laptop, my newborn son threw up into it as I was having a video call with my mom.(Yes it was stupid of me to lean my son over my computer). I have goldcare on my laptop. Dell fixed and replaced everything. I talked to an apple sales rep a few days later about corporate purchases. He said, that if that had happened to an apple I was would SOL. I don’t doubt Mac is a more stable OS, they don’t have to account for a million varieties of hardware that the OS will run on. have you ever had service on your iPod? It’s dealing with Apple’s service that makes me not a fan of apple.

Posted by Mike | Report as abusive
 

“Wait a second. Someone in the computer industry just took personal responsibility for messing up?”

No, noone took personal responsibility, this developer was setup. The responsibility lies with the presenter, who did not prepare for his presentation.

Most often speech recognition software requires the user to train it for a few minutes, before it can be used successfully. Obviously this sales/marketting moron did not do anything like that before the demo, otherwise if the problem was actually in software he would’ve known it.

As usual noone who matter is taking personal responsibility.

 

[...] Video: Vista’s "double the killer delete select all" Vista’s Voice Recognition gone really bad in a live demo in front of investors. Reuter’s report here. I sense a meme coming… Remember: "double the killer delete select all!" Keywords: funny, video, windows [...]

 

I was at this meeting and wrote up an account of what happened that counters the newscast here.

 

[...] It’s how history repeats itself Posted by backtick Filed in Binary Notes [...]

 

[...] At Micrsoft â??s annual Financial Analyst Meeting on Thursday, Vista product manager Shanen Boettcher set out to show just how easy to use the speech recognition technology built into upcoming Windows Vista software will be. But something went very wrong…read more | digg story [...]

 

[...] Page Summary: Copyright Reuters. Rest in peace Microsoft, I thing you are well beyond the point of no return. We are asking you to do this to help prevent comment spam. We moderate all comments and will publish everything that advances the post directly or with relevant tangential information.read more | digg story [...]

 

[...] Anyway, there are indeed improvements in Voice recognition in Windows Vista, and no it is not perfect yet. I’ll be interested to see if this increases others’ usage of voice recognition. After a particularly bad faux pas at a demonstration, a lot of folks where laughing at Microsoft’s efforts, but I’ve given the new voice-recognition in Vista a few work outs and I’ll say it is much improved. You still need a good quality microphone for the best results. For another look at Vista’s voice recognition take a look at this article by Reuben Schwartz. Also here’s a re-link to a screencast by Long Zheng that shows off some of the voice recognition features.   Software | Vista 11/03/2006 05:21:30 (Mountain Standard Time, UTC-07:00)  Vista Speech Recognition Ups The Volume     Comments [0]  |   |  Digg This |  del.icio.us |  Citations  [...]

 

[...] Bill Gates’ image shoots itself in the foot July 12th, 2006 First David Letterman takes a shot at Bill Gates, then Microsoft takes a shot at themselves. [...]

 

[...] Computer, Let Me Tell You Something…Voice Recognition Grows Up From a July 2006 Reuters article describing an early Vista demo: [...]