Self-Efficacy as a Usability Metric

Julian Abich, Ph.D., Senior Human Factors Engineer

I've attended and presented at several conferences this year, such as the Human Systems Digital Experience, World Aviation Training Summit (WATS), and Applied Human Factors and Ergonomics (AHFE), and have noticed a simple yet powerful construct appearing over and over again…self-efficacy. Self-efficacy is "concerned with people’s beliefs in their capabilities to produce given attainments" (Bandura, 2006). In other words, it's the confidence  in the ability to exert control over one's own motivation, behavior, and social environment (Carey & Forsyth, 2009). Extensive evidence has shown self-efficacy to be a significant predictor across a variety of contexts and domains, such as college academic performance (Choi, 2005), pre-career pilot performance (Wilson, 2021), weight loss success (Armitage et al., 2014), health management (Arslan, 2012), second language skills (Raoofi, Tan, & Chan, 2012), and work burnout and engagement (Ventura, Salanova, & Lloren, 2015). While there are a host of other areas that have explored the role of self-efficacy, one of particular interest to me that has been gaining more attention is usability. 

---

Usability assessments typically focus on effectiveness, efficiency, and satisfaction, but research suggests the integration of self-efficacy can provide a robust assessment (Martin, 2007). As the preponderance of technological solutions continuously diffuses across all aspects of our personal and work lives, our dependency on them will impact our ability to complete tasks. Therefore, belief in our ability to complete a task with a technological solution should impact the way in which these solutions are designed. Plenty of evidence exists indicating the role of self-efficacy in the adoption of technology, such as mobile learning solutions (Bettayeb, Alshurideh, Al Kurdi, 2020), desktop virtual environments for learning (Makransky & Petersen, 2019), fitness devices (Rupp, Michaelis, McConnel, Smither, 2018), and medical support tools (Lindblom, Gregory, Wilson, Flight, & Zajac, 2012). 

Although some usability measures focus on or integrate the concept of self-efficacy, not all are implemented correctly based on Bandura's guidance (2006). One major flaw is using Likert-type bipolar ratings (e.g., strongly agree to strongly disagree) instead of unipolar ones  (e.g., 0  to 10). The issue is if you have zero confidence in your ability to complete a task, then negative ratings below zero make little sense and lead to skewed interpretations of the results. Further, when bipolar ratings are used, the mid-point (usually labeled as neither agree nor disagree) gets converted into a moderate-level of self-efficacy which is incorrect and not a true reflection of the construct (Bandura, 2012). Leveraging self-efficacy as a usability metric can provide valuable insight into the design and evaluation of technology, but it's critical that measures be developed, implemented, and interpreted appropriately.

​Bandura (2012) stated that "there is no single all-purpose measure of self-efficacy with a single validity coefficient." This indicates that it's expected for new measures of self-efficacy to be created for, as he puts it, "activity domains." Activity domains are the topic areas in which the tasks under evaluation are performed. For example, evaluating self-efficacy for driving a monster truck. Use these guidelines when developing your measures and scales for usability evaluations (or any evaluation for that matter):

• Use unipolar ratings with multiple gradations of strength (at least 0 - 10 rating)
• Use a set of questionnaire items within the scope of an activity domain
• Items should indicate the goals that should be achieved for the tasks
• Make sure the items have clearly defined levels of attainment or performance
• Items should be measured against varying levels of task demands or challenges to avoid ceiling or floor effects
• Avoid developing measures for specific performance on a specific task
• Use statements with "I can" as opposed to "I will" since the latter reflects intention, not self-efficacy
• Avoid items that request foretelling a future event, but instead focus on judging their belief in reaching a specific achievement
• When capturing collective efficacy of a group, be mindful of the level of interdependence of group members to meet a task goal
• Be prepared to develop multiple assessments of self-efficacy for complex, multidimensional activity domains

Have you been capturing self-efficacy as part of your usability assessments? Tell us how. 

References:
Armitage, C. J., Wright, C. L., Parfitt, G., Pegington, M., Donnelly, L. S., & Harvie, M. N. (2014). Self-efficacy for temptations is a better predictor of weight loss than motivation and global self-efficacy: Evidence from two prospective studies among overweight/obese women at high risk of breast cancer. Patient Education and Counseling, 95(2), 254-258.
Arslan, A. (2012). Predictive power of the sources of primary school students' self-efficacy beliefs on their self-efficacy beliefs for learning and performance. Educational Sciences: Theory and Practice, 12(3), 1915-1920.
Bandura, A. (2006). Guide to the construction of self-efficacy scales. In Pajares, F., Urdan, T. (Eds.), Self-efficacy beliefs of adolescents, Vol. 5: 307-337. Greenwich, CT: Information Age.
Bandura, A. (2012). On the functional properties of perceived self-efficacy revisited. Journal of Management, 38(1), 9–44.
Bettayeb, H., Alshurideh, M. T., & Al Kurdi, B. (2020). The effectiveness of mobile learning in UAE universities: a systematic review of motivation, self-efficacy, usability and usefulness. Control and Automation, 13(2), 1558-1579.
Carey, M.P. & Forsyth, A.D. (2009). Teaching tip sheet: Self-efficacy. American Psychological Association. https://www.apa.org/pi/aids/resources/education/self-efficacy.
Choi, N. (2005). Self‐efficacy and self‐concept as predictors of college students' academic performance. Psychology in the Schools, 42(2), 197-205.
Lindblom, K., Gregory, T., Wilson, C., Flight, I.H.K., & Zajac, I. (2012). The impact of computer self-efficacy, computer anxiety, and perceived usability and acceptability on the efficacy of a decision support tool for colorectal cancer screening, Journal of the American Medical Informatics Association, 19(3), 407–412.
Makransky, G. & Petersen, G. B. (2019). Investigating the process of learning with desktop virtual reality: A structural equation modeling approach. Computers & Education, 134, 15-30.
Martin, C. V. (2007). The Importance of self-efficacy to usability: Grounded theory analysis of a child’s toy assembly task. Proceedings of the Human Factors and Ergonomics Society Annual Meeting, 51(14), 865–868.
Raoofi, S., Tan, B. H., & Chan, S. H. (2012). Self-Efficacy in Second/Foreign Language Learning Contexts. English Language Teaching, 5(11), 60-73.
Rupp, M. A., Michaelis, J. R., McConnell, D. S., & Smither, J. A. (2018). The role of individual differences on perceptions of wearable fitness device trust, usability, and motivational impact. Applied Ergonomics, 70, 77-87.
Ventura, M., Salanova, M., & Llorens, S. (2015). Professional self-efficacy as a predictor of burnout and engagement: The role of challenge and hindrance demands. The Journal of Psychology, 149(3), 277-302.
Wilson, N. (2021). Pre-Career Pilots and Motivation – What is the Best Predictor of Performance? 23rd World Aviation Training Summit (WATS), Orlando, FL, June 15-16, 2021.