Day at the Office: I’m Optimizing UX with SPEECH

Note: Say the title of this post in the same way as you would  “She Blinded Me With Science”


The Problem

The customer has a screen, which is used for data entry.  The source of the entry is various pieces of papers with scribbles on them.  On observing the person doing the data entry, what I saw was:

  • Look at paper to see WHO it was for
  • Look at screen to find the same.  Usually its the next one in order
  • Switch to Mouse
  • Click once to select the who
  • Click to select the thingies inside the who
  • Click the button to start the data entry for the who
  • Look at paper
  • Find the number
  • Switch to keyboard
  • Type in the number
  • Look at screen
  • Verify number
  • Press Enter to save
  • Look at paper
  • Move Hands to Paper
  • Mark Paper as done.
  • Move Paper Aside
  • Look at next paper.

Ouch.  We could do better.

Our (Final) Proposed Solution

I think its pretty cool, I don’t know yet if the user(s) will adopt it, but this is what we came up with.

  • User can keep right hand on numeric keypad, and arrow up to the first entry.  (they sort paper and computer entries to match beforehand)
  • Numeric / (on the keypad) starts entering for the currently highlighted row (even if it not yet selected)
  • As the data entry screen opens, the computer “speaks” the relevant info about what is being entered.  This is so the user can keep eyes on paper, while listening to verify that the right row was selected
  • Still keeping eyes on paper, user can type in the numbers on the keypad, and then press numeric + (on the keypad) to “speak” the numbers that were entered, for verification, if they want.
  • In case the user wants to look at the computer screen, we also made the window bigger, text larger, and used color to highlight the various corroborating pieces of information.
  • If all is good, user hits enter on the numeric keypad to save, and now the next row is highlighted.


This is well and good, except, this is what happened in testing:

  • /  34.5 +  <enter> / 45 <enter>
  • When the user got to the second /, the hotkey did not fire.

I spent many hours trying to figure this one out.  Its complicated, I still don’t know the full answer, but it has to do with a DockedWindow, a control suite, a Busy indicator,  a Ribbon suite, sub-ribbons, and document panels, and when the modal popup went away, focus did not return to the correct window. 

Bottom Line: all my attempts to set focus were ignored.  Seriously.  I even bound a hotkey at the Shell level (the whole application) “F10” to fire code that should set the focus back to the correct window.. and.. nothing.

This problem also extended to any window.   Any window, when it first came up, did not have focus, so any hotkeys bound inside that window, would not fire.   Till it was clicked in.  Then it was okay. 

Hacked Solution

I could still capture hotkeys via the Shell, so this is what I did:

  • Shell catches PreviewKeyDown
  • Shell looks at current active docked window (since it hosts the dock control, this is easy)
  • If current active window implements IReallyWantKeyDownStuff, then call the YoHeresAKeyForYou(). (Names changed because i don’t have intellisense writing at home, but you get the idea).
  • My window receives this, and if it was Numeric/, then it fires the button that starts the data entry process

I used the same code to also wire some other hotkeys for some other screens.

Office Visit Still Needed

I hope to (soon) visit my client’s office in person, and sit down with the folks who would use this “enhancement”, and show them what I had thought.   As someone once told me, I have to be careful how many strings I have attached to this:  I have to be okay with them taking my awesome idea and putting it in the trash.  No strings attached.  So I’m blogging about its coolness now, in case its a flop, because right now, I think its pretty cool.

Tagged with: ,
Posted in Code

Leave a Reply

Fill in your details below or click an icon to log in: Logo

You are commenting using your account. Log Out /  Change )

Google+ photo

You are commenting using your Google+ account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )

Connecting to %s

.net 3d 3d-printing 4k abc15 algorithms ames android anonymous types audio editing aws backup basecamp beatunes biorhythm bittorrent blender blog boston marathon bpm c# caffeine campfire candycrush car carmax certification charity chiropractor cities-skylines clog clone codelouisville codepalousa coding coffee collaboration color run ComputerElbow ComputerVision configuration consulting cooking crash course crashplan crestwood cryptocurrency cycling dabda dan dapper DataSet ddl diabetes dictation dotnetcore dotnetmud downtown e-cycling elite excel exercise expiration facebook feature-branching firefall flipflops Flow FL Studio focus food forecastle fortresscraft franklinplanner gadgets game-design games git github google docs google maps gopro gps grandpa greenshot hack half marathon headless health heart rate hiren ignew integration testing interop inventory ios ipad itunes javascript jobs karma kdf keyboards keys kittens lamont laptop lavalamp leaf lego life lifehack linq linqtotwitter linux los angeles louisville mandelbulber massage therapy mastery-teaching maths merge metformin Minecraft miniature modeling monitor mud muhammad ali institute music mvc mycartracks netfabb nexus10 node nomanssky nostalgia nutrition nwipe oldham county grand slam opal openjscad openscad owin pacedj paper mockup pepakura performance photoscan politics pomodoro postgresql powershell premiere priorities prius process product-management project-management qa ReFS resharper review rmi roman road 5k RSI rubiks running samsung 700t sandals schedule scooter scribblelive selenium service shapeways sleep slic3r sneakersync snot software software-engineering solidoodle soylent spacegame speaking sql sqlite SSDT SSIS standing state-machine stayfocusd stonehearth studying sunset tablet teaching team teamcity teamtreehouse terraform tesla testing tfs time timelapse torque touch tracks trs80 Tuple tutor twitter ubuntu unit testing utilities video video editing visual studio vscode vsvim warp stabilizer windows 8 windows home server wordpress wpf xml
%d bloggers like this: