windows 10 eye control

Latest Windows 10 Preview Brings Eye Control & Console Overhaul – Mobile & PC Slow Ring Builds Also Released

Author Photo
Aug 2, 2017

Dona Sarkar and Gabe Aul have finally released the much anticipated Windows 10 Insider Preview build 16257 for PC Insiders in the Fast ring and build 15237 for Mobile Insiders.

Today’s PC Preview build brings a number of expected features, including:

Today’s releases also include Windows 10 Mobile Preview build 15237 and Windows 10 PC Preview build 16251 for Slow ring PC Insiders.

windows-10-21Related Windows 10 November Update That “Died” in October Gets a Few More Months of Life

Here’s the complete changelog of Windows 10 16257 now available to PC Insiders in the Slow ring.

Windows 10 Eye Control (beta)

Yesterday, we announced Eye Control, which makes Windows 10 more accessible by empowering people with disabilities to operate an on-screen mouse, keyboard, and text-to-speech experience using only their eyes. The experience requires a compatible eye tracker, like the Tobii Eye Tracker 4C, which unlocks access to the Windows operating system to be able to do the tasks one could previously accomplish with a physical mouse and keyboard. We are starting by supporting the EN-US keyboard layout, and we are looking to expand to more keyboard layouts in the future. We are excited to release this experience as a beta and would love your feedback!

Setting up Eye Control:

  • Own or purchase a Tobii Eye Tracker 4C (Coming next will be support for Tobii Dynavox PCEye Mini, PCEyePlus, EyeMobile Plus, and I-series.)
  • Download and update to Tobii’s Core eye tracking hot fix release 2.10.11.6458 and run calibration with your own profile
  • Check for Windows Updates; the new Tobii Eye Tracker HIDClass Driver should be found on Windows Update and installed automatically.
  • Make sure your Tobii eye tracker is connected to your PC and turn on Eye Control by going to Settings > Ease of Access > Other Options > Eye control.

Eye Control launchpad – When you turn on Eye Control, the launchpad will appear on the screen. This allows you to access the mouse, keyboard, text-to-speech, and to reposition the UI to the opposite side of the screen.

Eye Control interaction model – To interact with the UI for Eye Control, simply look at the UI with your eyes until the button activates. A visual affordance will appear around the UI that you are looking at.

Eye Control mouse – To control the mouse, select the mouse from the launchpad, position your eyes on the screen where you want the cursor to be placed, fine tune the position, and select what action you want to take (left click, double left click, right click, or cancel).

Eye Control keyboard – To use the keyboard, select the keyboard from the launchpad, and dwell at the characters you want to type. You can type numbers and symbols on the &123 page and function keys on the Fn page. We currently support the EN-US keyboard layout.

Eye Control shape-writing – Type faster with your eyes by shape-writing on the Eye Control keyboard. To use shape writing, turn it on from the keyboard settings (found on the Fn page). Once it is on, you can form words by dwelling at the first and last character of the word, and simply glancing at letters in between.  A hint of the word predicted will appear on the last key of the word. If the prediction was incorrect, you can simply select an alternative prediction provided.

Eye Control text-to-speech – Communicate with your family and friends in person by using text-to-speech. To use text-to-speech, select text-to-speech from the launchpad. From here, you can use the keyboard to type sentences and have it spoken aloud. At the top are phrases that are spoken aloud immediately and can be edited to say different words. This uses the default text-to-speech voices, which can be changed in Settings > Time & Language > Speech > Text-to-speech.

Eye Control settings – Access settings from the Fn keyboard page to adjust the dwell times, turn on/off shape-writing, and turn on/off the gaze cursor used to test hardware calibration.

Known Issues:

  • Eye tracking does not work well in direct sunlight. The device may require new calibration when moving to a location with different lighting conditions.
  • The launchpad partially blocks the Tobii UI during device calibration. To work around this, turn off Eye Control during calibration and turn it back on when you are done. You can also use touch or mouse to reposition the UI during calibration.
  • Sometimes shape-writing can get stuck on. You can fix this by dwelling at the shape-writing icon to turn it on and off again
  • Sometimes the reposition UI icon in the launchpad takes focus after exiting text-to-speech

Hardware support:

Currently, Eye Control works with select Tobii hardware. We are open to working with additional hardware vendors to provide customers a broader set of hardware options to enable this experience.

To learn more:

If you have questions or feedback on how we can continue to improve our products and services, you can also contact us through the Disability Answer Desk (now with ASL support) and Accessibility User Voice Forum. You can learn more about accessibility at Microsoft by visiting, http://microsoft.com/accessibility.

Microsoft Edge Improvements

  • We’re giving Microsoft Edge a refreshed and more modern look in the browser frame, inspired by the Fluent Design System. The use of Acrylic material provides depth and transparency to the tab bar and other controls, and we’ve improved button animations to feel more responsive and delightful.”
  • Based on your feedback we’ve adjusted the design of the address bar so now even if the address bar isn’t in focus to start with when you click and drag the text it will remain under the cursor. Previously the text would shift as the “http://” appeared – this change will make it easier to quickly edit parts of the URL.
  • We fixed an issue where right-clicking on an image in Microsoft Edge and selecting copy then later pasting the clipboard content would result in the image URL being pasted rather than the image itself.
  • We fixed an issue where if a tab had been opened while Microsoft Edge was in full screen mode, using Ctrl + W to close that tab while still in full screen mode would close the content but leave the tab in the frame.
  • We fixed an issue where if you had two unrelated tabs in Microsoft Edge and opened a link from the first in a new tab, the new tab would appear to the right of the second tab rather than the first.

Console Improvements

During the Creators Update development we updated the Windows Console to support full 24-bit RGB color and today the Windows Console’s default colors are getting their first overhaul in more than 20 years! Yay!

The default color values have been changed to improve legibility of darker colors on modern screens, and to give the Console a more modern look & feel.

To preserve your preferences, this new color scheme will only be visible in Properties if you clean install this build.

Input Improvements

  • We’ve improved the performance of launching the touch keyboard after tapping the touch keyboard button in the taskbar.
  • We fixed an issue where using WIN+H to dictate into a UWP app’s text field wasn’t working.
  • We fixed an issue resulting in the Japanese IME prediction candidates position being shifted and overlapping with typed text if you set the page zoom level to something other than “100%” in Microsoft Edge.
  • We’ve adjusted the design of the Japanese curve-flick touch keyboard to display numbers and English letter keys in a smaller font that’s more consistent with the size of the Japanese character keys.

Windows Defender Application Guard (WDAG) Improvements

  • We have added new status strings to the WDAG splash dialog to provide more information about the startup stages for WDAG. These new strings will be displayed to users when WDAG is starting up and when it’s being resumed from a paused state.
  • We have also made significant improvements to container launch times by optimizing background preparation of the WDAG container.
  • We fixed a number of issues that were impacting networking inside the WDAG container as well as a poor user experience when connecting to the WDAG container.
  • Fixed an issue where the WDAG launch resulted in Error Codes 0x02 / 0x00 or 0x02 / 0c.
  • Fixed an issue where only part of the WDAG window was being displayed after launch.
  • Added a registry key to allow users to optimize WDAG launch times during active browsing by not suspending the WDAG container when the WDAG window is closed.  You can set the registry key to optimize for performance here: Computer\HKLM\Software\Microsoft\HVSI\, Value name: SuspendOnContainerClosed, REG_DWORD, Value data: 1.  Note: Setting this key will result in the container not pausing and the container will not release committed memory.

For known issues and details about the Mobile build, visit the official blog post.