Future of Fitness: A Field Study on Usability and Personal Activity Monitors

User Centric recently completed the first longitudinal user research study of personal activity (fitness) monitors. The popularity of this technology prompted User Centric to evaluate the impact usability has on motivation and sustained use of this type of device over time.  How do people really use them outside of a laboratory setting?  How are people using them to ‘monitor their health?’  Do people trust these devices to even work?  These are the types of questions this study set about to answer.

A fitness monitor is a device worn on the body or attached to a piece of clothing that keeps track of your activity level in some way. Unlike their predecessors (simple pedometers), these monitors tell you more than just the number of steps you have taken. Competing forms of fitness technology are narrower in focus such as heart rate monitors, fitness watches and activity computers.  These all offer features competitive with fitness monitors, but they are not integrated. 

Fitness monitors bridge fitness technologies.  They represent a product paradigm shift that connects the user experiences of integrated health improvement hardware with fitness based software.  For this paradigm to succeed, it is absolutely critical to approach it with user-centered design. This is the platform that future, more complex health technologies will use.

The specific information provided by the monitors in this study varies, but may include calories burned, duration of activity, and intensity of activity. Additional features may also be available, such as goal setting and trend tracking.  All of the devices tested synced with an online database in some way.  It is important to note that this study was not conducted to challenge the science that has already established these devices’ accuracy, but rather to analyze how these products are used and perceived by actual users over time in real-world conditions.

While the study was sponsored by User Centric, all three devices were donated for use throughout the study and granted all participants access to the online database that typically comes with a purchased device. Each physical device was equipped to access, or sync, with web-based activity tracking software. Activity metrics transferred to an online tracking system and progress toward user-specified goals were presented through an interactive interface of graphically depicted data. Metrics include calories, duration and intensity of activity, and more depending on the model. Even as the study progressed, features for tracking additional metrics became available.

Study Design

Web based surveys were employed throughout a longitudinal study design that took place between June 1-August 31, 2010. A longitudinal methodology relying on self reports was chosen in order to capture a critical portion of the user experience: the entire spectrum of early product interaction, from out-of-the-box impressions through weekly user experiences and final thoughts after three months of use.

Each participant was provided with two activity monitors to use during the study. Data were collected each week through online surveys, which were used to allow participants to report in from wherever they used the fitness monitor, as the purpose of this study was to assess their performance “in the wild.” Quantitative measures were obtained through 7 point Likert Scales and qualitative measures were collected through open feedback paired to each rating.

Over the course of three months, User Centric collected user feedback from 32 participants. All participants were selected because they were at least ‘somewhat likely’ to purchase two of the three activity monitors and use them as intended. Pseudo-random assignment was used to determine which two devices each participant would wear during the study. Participants included 10 males and 22 females with a varied age distribution. Twelve of the participants were aged 22-30, twelve were aged 31-40, two were aged 41-50, and six were aged 51-60.

A number of incentives were provided to minimize attrition. Every week, one participant was randomly selected to win a gift card if both activity monitors registered activity for at least five days that week, and the weekly survey had been completed. A final lottery was held at the conclusion of the study in which each participant had an equal chance to win one of three gift cards. Additionally, participants were provided with the option to continue using their preferred fitness monitor at a reduced monthly price at the conclusion of the study.

Findings Overview and Recommendations

While there were many different questions that were tackled during this study, and clear “winners” in certain categories emerged, overall there are valuable recommendations for the industry as a whole. 

What ‘Natural’ Usage Has Revealed

• Trust is a key factor.  Because people did not always use the devices as instructed they lost trust in them, resulting in reduced motivation.  While participants believed they used the fitness monitors almost ‘completely as intended’ by the manufacturer, they reported experiencing measurement variations, which caused frustration and then resulted in distrust of the devices.  The variation in measurement may have been caused in part by users not wearing devices exactly as intended, leading to the perception of inaccurate measurements.  This demonstrates the difference between user and designer expectations: if participants used the monitors as manufacturers expected, most aspects of product satisfaction would be extremely high, which was not the case.
  If a participant carried out certain activities while wearing the devices (e.g., bicycling, yoga, rock climbing), their trust in the accuracy of the measurements was further shaken, as these activities were reportedly ‘not tracked correctly’ by the devices. Accurate measurement in real-world conditions becomes much more unmanageable when there is a lot of variance in actual usage.
  Participant feedback also uncovered that users were not aware of important features unless they were obvious.   The online activity tracking is a key feature that differentiates these products from existing activity computers and fitness watches, however, participants were unaware of a variety of features until they learned about them through our surveys or discovered a software attribute after several weeks of use.  Manufactures of these devices should focus on creating awareness in  the tracking website that is fully able to ‘reveal’ its information in an intuitive, easy, and interesting manner.

• Convenience is critical in predicting how the technology will be embraced.  Participants complained of device placement during certain high impact activities (many participants expressed concern about losing their devices when highly active), devices not retaining a secure fit throughout the duration of the study, constant charging, and slow syncing process.  Many users consciously removed their fitness monitors to engage in high impact sports, group fitness classes, and water related activities. If it is difficult to wear an activity monitor while highly active, what is the point?
• Design must accommodate customer by anticipating usage variance.  Differences in calorie expenditure, as well as trust in the number of calories burned between monitors reflects disparities between real world usage and R&D intentions/expectations.  How would the manufacturers/designers respond to this? Knowing exactly why a potential customer does not use a product can be as useful as understanding why one does choose a product. Countering shortcomings through design that anticipates real world user behavior is good for business.
• A major intended feature of fitness monitors is connecting with people online, but this is under developed.  Surprisingly, most people reported not using the community features and found them to be relatively uninteresting, even though two of the three devices’ communities were deemed “usable.” Additionally, many participants reported that online community features were not an important aspect of the product.

Human Factors Considerations

A number of problematic issues arose throughout three months of field testing.  Some problems may erroneously be deemed ‘user error’ but as user experience professionals, we know that is often hardly the case.  When it comes to exercise, internal motivation is key; if the mechanism for external motivation is not as streamlined as possible, its effect and usefulness decrease dramatically.  Factors for fitness monitor engineers to consider include:

• Placement of fitness monitor; some of the monitors required users to indicate, in software, where on the body the monitor is being worn.  This minimizes ability to accurately use the device with spontaneous changes in activity.  Could there be a hardware indication of where on the body the device is being worn, for example?
• Design for convenience as users must wear the monitor consistently for precise measurement.
• Allow more tracking customization to afford better accuracy.
• Activity tracking software is important – more attention should be paid to how this is used in motivating performance, including the social network.
• Battery life a major concern for consumers.
• Very common user suggestion was to increase durability (i.e. waterproof and shockproof).
• Device feedback is useful and important but users should be able to quickly and easily disable feedback via the hardware itself to avoid receiving feedback in unwanted settings.
• Consider displaying more information on the physical device; most users found this persistent visual feedback to be motivating.

Field tests like this are extremely helpful in understanding the wide differences in potential use that manufacturers could not possibly anticipate while also identifying what kinds of errors and problems to expect, so that design can accommodate and prevent them before they even happen.

The full study with analysis, ratings and recommendations will be available in first quarter, 2011.