Day at the Office: I’m Optimizing UX with SPEECH

Note: Say the title of this post in the same way as you would  “She Blinded Me With Science”


The Problem

The customer has a screen, which is used for data entry.  The source of the entry is various pieces of papers with scribbles on them.  On observing the person doing the data entry, what I saw was:

  • Look at paper to see WHO it was for
  • Look at screen to find the same.  Usually its the next one in order
  • Switch to Mouse
  • Click once to select the who
  • Click to select the thingies inside the who
  • Click the button to start the data entry for the who
  • Look at paper
  • Find the number
  • Switch to keyboard
  • Type in the number
  • Look at screen
  • Verify number
  • Press Enter to save
  • Look at paper
  • Move Hands to Paper
  • Mark Paper as done.
  • Move Paper Aside
  • Look at next paper.

Ouch.  We could do better.

Our (Final) Proposed Solution

I think its pretty cool, I don’t know yet if the user(s) will adopt it, but this is what we came up with.

  • User can keep right hand on numeric keypad, and arrow up to the first entry.  (they sort paper and computer entries to match beforehand)
  • Numeric / (on the keypad) starts entering for the currently highlighted row (even if it not yet selected)
  • As the data entry screen opens, the computer “speaks” the relevant info about what is being entered.  This is so the user can keep eyes on paper, while listening to verify that the right row was selected
  • Still keeping eyes on paper, user can type in the numbers on the keypad, and then press numeric + (on the keypad) to “speak” the numbers that were entered, for verification, if they want.
  • In case the user wants to look at the computer screen, we also made the window bigger, text larger, and used color to highlight the various corroborating pieces of information.
  • If all is good, user hits enter on the numeric keypad to save, and now the next row is highlighted.


This is well and good, except, this is what happened in testing:

  • /  34.5 +  <enter> / 45 <enter>
  • When the user got to the second /, the hotkey did not fire.

I spent many hours trying to figure this one out.  Its complicated, I still don’t know the full answer, but it has to do with a DockedWindow, a control suite, a Busy indicator,  a Ribbon suite, sub-ribbons, and document panels, and when the modal popup went away, focus did not return to the correct window. 

Bottom Line: all my attempts to set focus were ignored.  Seriously.  I even bound a hotkey at the Shell level (the whole application) “F10” to fire code that should set the focus back to the correct window.. and.. nothing.

This problem also extended to any window.   Any window, when it first came up, did not have focus, so any hotkeys bound inside that window, would not fire.   Till it was clicked in.  Then it was okay. 

Hacked Solution

I could still capture hotkeys via the Shell, so this is what I did:

  • Shell catches PreviewKeyDown
  • Shell looks at current active docked window (since it hosts the dock control, this is easy)
  • If current active window implements IReallyWantKeyDownStuff, then call the YoHeresAKeyForYou(). (Names changed because i don’t have intellisense writing at home, but you get the idea).
  • My window receives this, and if it was Numeric/, then it fires the button that starts the data entry process

I used the same code to also wire some other hotkeys for some other screens.

Office Visit Still Needed

I hope to (soon) visit my client’s office in person, and sit down with the folks who would use this “enhancement”, and show them what I had thought.   As someone once told me, I have to be careful how many strings I have attached to this:  I have to be okay with them taking my awesome idea and putting it in the trash.  No strings attached.  So I’m blogging about its coolness now, in case its a flop, because right now, I think its pretty cool.

Author: sunnywiz


Leave a Reply

Your email address will not be published. Required fields are marked *