We believe experiences matter.™
UX Nuggets Thoughts and advice on usability and user experience
The Most Precise (or Most Accurate?) Eye Tracker
May 18, 2011 |
Aga Bojko
(Re-post from the “Eye Tracking the User Experience” Blog. Aga is currently writing Eye Tracking the User Experience, A Practical Guide, to be published by Rosenfeld Media in 2012.)
To keep up with the developments in research and technology, I have a Google Alert set up for "eye tracking" OR "eyetracking" OR "eye-tracking." The daily email comes to my Inbox at 11:30am, just in time for my browsing lunch (more fun than a working lunch, less fun than a non-working lunch). Today, nine out of the twenty results in the alert email mentioned Tobii Technology introducing the "most precise eye tracking solution" for mobile device testing:
Most precise! Who could resist that?
The solution (Tobii Mobile Device Stand) described in the articles is actually quite clever. I'm not sure why it made the news today because it's been available for a while now. Maybe it was just this morning when they found it was "most precise." I continued reading in suspense.
To my disappointment, no explanation was offered for how this conclusion was reached. What's more, I don't even know what was meant by "precise." I think the author was referring to the accuracy of the eye tracking solution but I can't be sure. And that's precisely where the problem lies - in the confusion between precision and accuracy (and people not realizing that there is confusion). Let me explain...
The accuracy of an eye tracker is the average difference between what the eye tracker recorded as the gaze position and what the gaze position actually was. We want this offset to be as small as possible but it is obviously unrealistic to expect it to be equal to zero.
Accuracy is measured in degrees of visual angle. Typical accuracy values fall in a range between 0.5 and 1 degree. To give you an idea of what that means, one degree corresponds to half an inch (1.2 cm) on a computer monitor viewed at a distance of 27 inches (68.6 cm). In other words, the actual gaze location could be anywhere within a radius of 0.5 inch (the blue circle below) from the gaze location recorded with an eye tracker with one degree of accuracy (the "X"):
Accuracy values reported in eye tracker manuals are measured under ideal conditions, which typically include, for example, testing participants with no corrective eyewear and taking the measurement immediately after calibration. During "real research," the difference between the reported and actual gaze locations can be larger for participants wearing glasses or contact lenses or those who moved at some point following the calibration procedure.
Precision (aka "spatial resolution"), on the other hand, is a measure of how well the eye tracker is able to reliably reproduce a measurement. Ideally, if the eye is in the same exact location in two successive measurements, the eye tracker should report the two locations as identical. That would be perfect precision.
In reality, precision values of currently available eye trackers range from 0.01 to 1 degree. These values are calculated as the root mean square of the distance (in degrees of visual angle) between successive samples. Because the precision values reported by manufacturers are measured using a motionless artificial eye (pretty cool, huh?), tracking real eyes will exhibit less precision.
The table below summarizes the relationship between eye tracking accuracy and precision. The cross indicates the actual gaze location, while the dots are gaze locations reported by the eye tracker.
All in all, the "most precise eye tracking solution" was probably just a poor choice of words but it gave me an excuse to talk about precision vs. accuracy and sound like I'm up to date on current events. I do what I can.
Associate Director Aga Bojko has been instrumental in establishing the eye tracking practice at GfK User Centric by integrating eye movement analysis into traditional UX research studies for software applications, websites, and product packaging. She has a MS in Human Factors from the University of Illinois and a MS in Human-Computer Interaction from DePaul University.
Contact Us for Answers
Related news & publications:
Our research has been published in books and professional journals, and is presented regularly at conferences. Many of our articles and publications can be read online, downloaded in PDF format, or requested by e-mail.
Names in italics indicate User Centric employees at the time of publication.
Comments
Why it is the most precise eye tracking solution
First of all thank you for this post on what accuracy and precision means for eye trackers, it´s an important thing to remember when evaluating eye trackers or eye tracking data.
I work for Tobii as a Trainer and I´m also the Product Specialist for the Mobile Device Testing Solution. Let me try to answer your question on why precision with this Solution is better than other solutions out there:
We have three major reasons why accuracy and precision are increased with this solution compared to other solutions in the market such as Head mounted eye trackers or competitive remote eye trackers:
First reason: The phone is closer, increasing relative accuracy and precision. Most remote eye trackers have an optimal operating distance of about 65cm, meaning the participant has to sit about 65 cm from the eye tracker (with some freedom to move of course). Usually a stimulus is also shown right above the eye tracker, for example on a monitor. However, in some cases the stimulus is placed further back, since most eye trackers can capture stimuli at a maximum radius of about 35 degree radius from the front of the eye tracker. Imagine a cone with a 35 degree radius at the eye tracker which shrink into a small point between the participants´ eyes and which expands behind the eye tracker, allowing larger stimuli to fit in this 35degree radius or “maximum gaze angle”. The opposite is also true: smaller stimuli can be placed closer to the participant while still fitting in this maximum gaze angle cone. Since accuracy and precision are measured in angles, this also means that these same angles will actually cover a much smaller area at a closer distance. Small items on a mobile device will cover a larger part of the total field of vision. The Mobile Device Stand holds the mobile device much closer to a participant, which will leads to increased relative accuracy and precision when compared to other solution where the mobile device is usually place right above a remote eye tracker (conventional method).
Second Reason: Tobii supports the use of the Bright Pupil eye tracking technique, noise is reduced and precision increases. Tobii is known for supporting both common CR (Corneal Reflection) eye tracking techniques: Dark Pupil AND Bright Pupil. Although Dark Pupil is the most commonly used eye tracking technique (especially in Head Mounted it is the standard) and almost all competitive systems ONLY support Dark Pupil. Bright Pupil is known to be a more robust eye tracking technique which produces less noise and higher precision than Dark Pupil. A part of the worlds´ population, mainly Asians, will track better using Dark Pupil and this is why only DP is often the preferred choice for many manufacturers of eye trackers, this compromise is not made by Tobii since it supports both techniques!
Third Reason: Head Mounted eye trackers traditionally have lower accuracy and precision and usually rely on relatively low resolution cameras, we use a Full HD camera for the Mobile Device Stand, which is placed close to the mobile device, providing the resolution required to see details such as very small text or icons that are common on a mobile devices. This may not affect the eye tracking itself but it does make the stimulus a lot more clear and makes it easier to “precisely” see what a participant was fixating on.