The Robots Are Taking Our Jobs!
As AI technology continues to advance, we are seeing more and more applications in the research and scientific fields. One area where AI is gaining traction is in the writing of research reports. AI algorithms can be trained to generate written content on a given topic, and some researchers are even using AI to write their research reports. While the use of AI to write reports may have some potential benefits, such as saving time and providing a starting point for researchers to build upon, there are also significant ethical concerns to consider.
One of the main ethical issues with using AI to write research is the potential for bias. AI algorithms are only as good as the data and information that is fed into them, and if the data is biased, the AI-generated content will be biased as well. This can lead to the dissemination of incorrect or misleading information, which can have serious consequences in the research and scientific fields. Another ethical concern with using AI to write research reports is the potential for plagiarism. AI algorithms can generate content that is similar to existing work, and researchers may accidentally or intentionally use this content without proper attribution. This can be a violation of copyright law and can also damage the reputation of the researcher and the institution they are associated with.
Additionally, using AI to write research reports raises questions about the ownership and control of written content. AI algorithms can generate content without the input or consent of the individuals who will ultimately be using it. This raises concerns about who has the right to control and profit from the content that is generated. Overall, while the use of AI to write research reports may have some potential benefits, there are also significant ethical concerns to consider. It is important to carefully weigh the potential benefits and drawbacks of using AI in the writing of research reports and to consider the potential ethical implications of this technology.
I didn't write a word of that. It was produced by ChatGPT ( https://chat.openai.com/), an AI chatbot that launched to the public on November 30. Since its launch, it's been at the forefront of the tech news cycle because it's both very good and very easy to use. Yesterday, I asked it to write a literature review on a couple of topics and pasted the results in QIC's Teams. They all got a weird, uneasy feeling reading it; we all joke about the day "the robots will take our jobs," but we hadn't realized that our jobs were on the list of those that could be so easily automated. Is the copy above particularly eloquent? No. Does it answer the mail for a lot of things? Yes. And sometimes, as they say, that's good enough for government work.
The QIC crew wasn't alone in their unease. Across the internet, authors are writing to minimize the impact of technology like this or demonize it. If you try hard enough, you can make it do racist things. It'll help kids cheat at school. But that's not really why we react to it the way we do, is it? It's the realization something we thought made us human - the ability to create - is not something only we can do. In fact, it's not even something we can do as efficiently as something that is not only inhuman, it's not even alive.
While I was playing with ChatGPT, many of my friends were posting AI-generated stylized selfies using the Lensa app. Its popularity reignited a similar discussion in the art community. Aside from the data privacy discussion we've been having since the Cambridge Analytica fiasco, artists are rightly concerned about ownership and the ability to make money from their work. At the core, though, it's the same fear that if AI can do your job, where does that leave you?
When robots were anticipated to take over the world, many of us expected they would take the jobs we didn't want, like fertilizing crops and driving trucks. These were supposed to be the jobs that are physically exhausting, dangerous, and monotonous. They weren't supposed to be the ones that we went into thousands of dollars of student loan debt to be qualified to do. We didn't think it would be so easy for a machine to do something that when we do it reflects our feelings and thoughts. The phrase “intellectual property” presupposes an intellect, and an intellect presupposes a person.
As one who tries to maintain cautious optimism about the future of technology, I find it exciting that I may live to see the day when AI is far more efficient at most things than I am. Obviously, there’s a lot we have to consider from an ethics perspective. I’ve watched the Avengers enough times to appreciate the potential for Ultron to make decisions we might not like as a species. But that day is coming, and it’s important to have those conversations now. On a broader level, it’s time we start thinking about what it means for us as creative people, what we value, and why we are special on this planet. Because I do believe that we are.
Unless you either live under a rock or don’t have a relative/friend/boss who’s a Swiftie, you know that Taylor Swift concert tickets went on sale this week. You probably also know it was a catastrophe, with the blood and tears of millions of Swifties on Ticketmaster’s hands. It was enough to divert an entire news cycle away from Elon Musk. I’m here to tell you how in the future, this sort of thing might not happen.
Here’s what happened: Prior to the presale, Swifties registered as “Verified Fans” through a partnership between Ticketmaster and Taylor’s management team, Taylor Nation. From Ticketmaster’s reports, over 4 million fans signed up. The night before the presale, 1.5 million of them were notified by text and email that they won the presale lottery and were given access codes to buy tickets the next day. If you bought merchandise from the Taylor Swift store using the same email address as your Ticketmaster account, this helped your chances. I am both simultaneously proud and embarrassed to say that I have dropped enough money on that website to score a presale code.
The events of the following morning will live forever as one of the darkest of days in the annals of Swiftie history. Over the course of the morning, 1.5 million fans tried to buy tickets simultaneously – along with 12.5 million other people…and resale bots. (If you want to learn about how these bots work, this is a good primer.) The site crashed in spectacular fashion, codes did not work, and many irate Swifties did not score tickets. Moments later, ticket resale sites like StubHub and Vividseats immediately showed plenty of inventory – marked up to $30,000 a ticket in some places. The whole point of fan verification was to prevent something like this from happening. So, what went wrong?
This is not a post about that. This is a post about how it could have gone.
Non-fungible tokens, or NFTs, have a bad rep in some circles because they are often associated with “tech bro” culture and extremely ugly AI-generated cartoon apes. However, they are useful for situations in which you want to tie a digital asset to its owner. If concert tickets are issued on NFTs, the organizer would have a record of the owner of the ticket on a blockchain, and more importantly, would know when that ticket is sold to someone else and how much it sold for. Keeping these records could limit price gouging, scalping, and ticket fraud. Because the ticket resides on a blockchain, the organizer could set resale cost limits and conditions.
There are other benefits to using NFTs for ticket sales. Because this technology moves quickly and efficiently, it may be less likely to crash a website. Artists could attach digital collectibles to tickets as souvenirs of the show. Tickets are potentially more securely stored digitally using ticket wallets tied to a single device. And paying for tickets could be more seamless, too. Ticketmaster sees the value, too, and has partnered with blockchain providers for other kinds of events.
It'll be a while before the world buys Taylor Swift tickets on blockchain, but there’s a good chance it’ll be better than not being able to buy Taylor Swift tickets at all. I, for one, am ready for it.
I have been brushing my teeth for 40 years, which is approximately 1,000 hours of practice. Not exactly the 10,000 hours to achieve mastery, according to K. Anders Ericsson, although seemingly sufficient for a menial task. I didn’t think much about my toothbrushing proficiency, but if pressed, I would have said it’s “good enough.” However, the last few trips to the dentist indicated otherwise. Nothing major, but clear signs that my technique was flawed. So, I had a choice to either continue to accept good enough or try to improve my performance. Yes, toothbrushing is a mundane task, but why not finally listen to the expert and do it better?
I am an underperforming teeth brusher not because I’m incapable of performing the task but because I formed unproductive habits and lacked the motivation and tools to overcome those habits. I would buy an inexpensive toothbrush and then mindlessly move it around in my mouth for some reasonable amount of time twice a day. As a major step toward improvement, I bought an electric toothbrush (spurred on by my dentist, of course).
At QIC, we know that in order to push yourself to the next level, you require insight into your performance. Not only do you need enough data, you need the right data. It must be presented to you in a way that’s easy for you to understand at a glance, and it must be coupled with feedback. Knowing you’re not performing at your peak level isn’t enough – you need to know what to do to fix it! I was pleased to see these features in the companion app to my toothbrush. The brush itself gives you brushing guidance through a timing feature and sensor. The downloadable app provides insights into your brushing habits, and importantly, gives you specific strategies you can use to improve. Feedback on brushing frequency, average duration, and performance is displayed after each session, along with corrective actions such as “slow down.” Taken together, this Cadillac of toothbrushes is designed to break unproductive habits, model and maintain the correct behaviors, assess performance, and provide feedback. From a human factors and training perspective, it’s quite impressive!
Whether government or private industry, our customers are regularly faced with the choice of seeking opportunities for improvement over accepting good enough. Not about teeth brushing, necessarily, but for job tasks on which they are trained, practiced, and may not have to think much about. The most elite performers are motivated to improve even on the tasks that have become mundane. Despite the desire to constantly improve, how to improve may not always be clear to them. We, as human factors and training professionals, know how. Analyzing job tasks allows us to understand the negative habits, well-designed training and performance support can help break those habits, active participation and demonstrable results can motivate, and assessment and proper feedback can maintain performance. The fighter pilot, CEO, Navy SEAL, etc. choose to shun the good enough, whether in the mundane or extraordinary. It is part of what makes them great, and we can help focus that motivation to make them even greater. In the meantime, let’s start with brushing our teeth… properly.
Rebounding on and off the court
I recently had the privilege of joining Coach Matt Doherty on his webcast where we talked about a variety of workplace training topics. My first conversation with him was probably one of the most memorable moments of my career to date, as it revolved around two of the things I care most about in the world: leadership and UNC basketball. For those of you who weren't blessed from birth to be a Tar Heel, he played on the 1982 National Championship team alongside Michael Jordan. He's coached at a variety of schools (including my alma mater, Davidson College), but he's best known for his three-year tenure as the head coach of the UNC men's basketball team.
After the webcast, Coach sent me a copy of his book Rebound: From Pain to Passion as a token of thanks. I read it cover to cover the next day. In it, he discusses his rise as a coach to one of the most prestigious jobs in the game, the loss of that job, and how he was able to emotionally and professionally recover in the aftermath. He describes the feelings of loss, pain, and betrayal associated with falling off a high pedestal in an extremely public forum. I do not know many people who could recover from something that ego-shattering with grace.
Inherent to leadership is learning to be comfortable with people watching and evaluating you. You're on a stage all the time, even on work-from-home Fridays and during happy hour. Even for those of us who like the limelight, it takes a toll. One thing we never talk about is how, as leaders, to manage emotional pain while we're on that stage. It's one thing for a team member to go through a hardship; leaders rally around them, give them the space and support they need, and are patient while life is uncertain. But when it's you? Do you tell your team what's going on, and risk freaking them out? Do you try to model strength in the face of adversity and act like nothing is wrong? Will they think you're making excuses? Will they be scared? Can you handle your whole world knowing that you're not OK?
Basketball teams play a lot of games, which is great for people like me who like to watch them. If you're the coach, that means you have the potential to take a lot of "Ls." In the government contracting space, we do too. I lose a lot more than I win, and that's hard. When you add personal loss on top of that, the cracks start to show. Throw in a pandemic, political tension, and everything else, and honestly, there are some bad days. How do I deal with it? I tend to share in the hopes that the crew will know when it's them, they're in a safe place to ask for help. I also think they have a right to know if I'm not at 100%, and they should understand that this job is hard because one day, it might be theirs. But it's also just as important for them to see me trying to take care of myself, giving myself some grace, and getting better. Is this the right way to handle it? I don't know. But I do know that the world isn't getting any kinder, and this should be a topic we talk about a lot more.
Thanks to Coach Doherty for having the courage to write about his experience, and for sharing the lessons he learned from it. You can order Rebound: From Pain to Passion here: https://amzn.to/3Tl8vFV and watch to the Rebound Live Webcast here: https://coachmattdoherty.com/webcast/
I recently attended the 13th Applied Human Factors and Ergonomics (AHFE) Conference in New York City. Like a busy NYC street, the topics covered were an eclectic mix of human factors solutions highlighting advances in training, performance, safety, and usability in nearly every human endeavor. The presentations I saw at AHFE emphasized that human factors has embraced new technologies as tools for investigation and as objects of study. Here are just some of the interesting things at AHFE 2022:
Like the massive screens in Times Square, the above items were certainly attention-grabbing and my first reaction to much of what I saw was “cool.” Then, questions began to go through my mind as I started to think about what’s beyond this first impression. Is the alphabet instructional game enjoyable and thus motivating children to stay engaged? Is it intended to supplement parent-child interaction? Or replace it? The AR laser-based project system may improve productivity, but does it improve safety? Do the workers understand the system and enjoy using it? Are autopsies currently being done poorly and, if so...how? You get the idea.
The AHFE keynote address by Dr. Michael van Lent, President and CEO of Soar Technology, discussed the 4th Industrial Revolution. This revolution is brought about by a combination of technologies that afford the opportunity to blur the lines between the physical, digital, and biological, allowing new information, products, and services to emerge. This led me to think about technology's intended and unintended uses and how humans are changed simply because of their interactions with technology. Often, we focus on the obvious, albeit “cool”, intended use without considering secondary and even tertiary human-centered benefits and consequences. For instance, I hope that a children’s game for teaching the alphabet is motivational and effective while also promoting rich social interaction among parents and children.
As human factors researchers, we should consider how to measure the unintended benefits and how humans are changed because of technology interaction. For example, how does adaptive lighting in automobiles influence not only driver emotions but also driver focus and reaction time? While no single study can examine every effect, intended or unintended, our methods and measures must be expanded to account for the ever-increasing complexity of human systems interaction. What are the types of questions you ask when you see new tech or research? Are you able to move beyond the cool?
The Olympics concluded last week and some of you may have watched the fascinating sport of curling. I don’t know about you, but when I watch curling, I have many questions. Questions such as” “What the heck is going on?” and “How do they keep score?” But the aspect of curling that intrigues me the most is how the curlers communicate. There are moments of silence punctuated by abrupt yelling, and soft talking that becomes full on screaming. It is exciting to watch! But it still begs the questions, what are they communicating and how does this communication help their performance?
Let’s start with some basics. Curling is played on a long, narrow sheet of ice with a marked target area, called the house, at each end. Teams typically consist of four players each and each team takes turns sliding heavy granite stones to the opposite end of the ice. The goal is to get your team’s stones closer to the center button of the “house” by pushing the stone and sweeping the ice in front of the stone to keep it moving at the speed and direction you want. The friction from the “sweepers” sweeping polishes the ice, which makes the stone travel farther and straighter (Reich, 2014). However, for sweepers to know how and when to sweep, effective communication is required from the thrower to the sweepers.
Before the stone is thrown, the skip (the team strategy leader) should communicate the delivery of the stone with the thrower and sweepers. It is key for the delivering player and the sweepers to make sure they understand this guidance from the skip. If needed, they will discuss and confirm with the skip their understanding of the upcoming shot. The thrower and the sweepers will typically discuss the speed of the ice prior to throws as well. Once the stone is moving, constant communication should take place between the sweepers and the thrower. The sweepers will be responsible for judging the weight of the stone and will regularly communicate their estimation to the skip. The thrower is responsible for judging the stone’s course and should regularly inform the sweepers of that course and their need to sweep or not. Various codes and methods of communication exist to describe the path and weight of the stone. Table 1 provides examples of phrases used in curling between the thrower and the sweepers.
So how do we know if a curling team is communicating effectively? We can begin analyzing the communication in terms of an established framework (Jeffcott & Mackenzie, 2008). QIC is currently doing this with Army squad communications during execution of a common battle drill. In this effort, communication between squad members is classified as one of five possible factors: communication quality, leadership, monitoring, cooperation, and coordination. See Table 2 for a description of each factor. Squad scores within each factor are analyzed relative to squad performance metrics.
A similar approach can be applied to curling team communications. Communication Quality can refer to the extent that team members are yelling clearly and sufficiently often. Leadership can be measured through whether the skip is making sound decisions and providing clear direction to the thrower. Monitoring may consist of whether the thrower is constantly monitoring the speed and position of the rock, and if the sweepers are actively listening, responding to, and acknowledging the thrower’s direction. Cooperation can be measured by the degree to which the sweepers are working together. Coordination can refer to the implicit coordination between the sweepers and if the skip is providing direction at the right time.
Once team communications are organized according to the five factors and validated, the formal analysis comparing communications to curling performance can begin. For instance, how does communication within each of the five factors explain differences in curling team’s performance such as points scored, wins and losses, and time to complete a match? With enough data from different curling teams, a predictive model could be developed to better understand how different types of communication, according to the five-factor framework, can predict actual curling performance. This can have major implications for training and team member selection.
What other Olympic sports did you watch where team communications appeared to be an important factor?
Jeffcott, S. A., & Mackenzie, C. F. (2008). Measuring team performance in healthcare: review of research and implications for patient safety. Journal of Critical Care, 23(2), 188-196.
Reich, B. (2014, February 13). Here's everything you ever wanted to know about curling. Mic. Retrieved February 17, 2022, from https://www.mic.com/articles/81947/here-s-everything-you-ever-wanted-to-know-about-curling
Sticking to my exercise regime. Keeping up with those Spanish exercises on DuoLingo. Stop binge-watching “The Office”. Eating healthily. It’s only a few weeks into the new year and l’m already re-thinking my new year’s resolutions.
Most of us would agree that behavioral change is elusive. Two thoughts come to mind, one, “an object at rest stays at rest and an object in motion stays motion, unless it is acted upon by a force” which is Newton’s 1st law of motion (Britannica, 2021). Two, "we are just creatures of habit." Creatures, i.e., driven more by instinct with minimal prefrontal cortex (PFC) action. The PFC being associated with executive functions such as cognitive control, attention, planning, decision-making, impulse inhibition, reasoning, and problem-solving (Fuster, 2015).
So it seems like short of a compelling external force, I can’t help but fall back on old behaviors without some purposeful activation of my higher-order cognitive functions to steer me away from my path of least resistance. It also doesn’t help that in today’s digital age, we all have too many options for things to do at any one time. This means staying on a task requires an intentional decision every time we’re tempted to do something else.
Resolutions can either be approach-oriented goals such as eating more vegetables and keeping up with exercise, or avoidance-oriented goals such as spending less or quitting a bad habit. Studies show that approach-oriented goals are more effective than avoidance-oriented ones (Oscarsson, Carlbring, Andersson, & Rozental, 2020), suggesting that when possible, we should perhaps reframe those “quit” goals as new habits to be formed instead. Another consideration for forming resolutions is applying one of the seven principles of highly effective people: begin with the end in mind (Covey, 2020). This means I shouldn’t just resolve simply to exercise more or eat more healthily. The goals need to be specific and tied to a realistic vision of what the end looks like. For the exercise goal, this could be an ongoing one like jogging 2 miles three times a week.
Here are some commonly cited pointers to help us new year resolvers along:
1. Watch those cues and lower the bar for the good habit to form
Many of our behaviors are based on and triggered by cues which can be an event, emotion, place or person. For instance, when I come home after work and turn on the TV, this is the end-of-the-day signal to wind down, and for me, it is a cue to take a trip to the fridge or pantry and reach for that bag of chips. Feelings of boredom can also cue mindless snacking. To counter this, I’ll need the cues to signal the new behavior. Instead of turning on the TV the minute I get home, I should do something else that takes me away from the TV, e.g., go out for a walk or tend to my plants. Cues should also be used to encourage us to form good habits. Some suggest putting running shoes by the bed as a cue to exercise early the next day. I’d also suggest lowering the bar for the new habit and wearing my running socks to bed, or replace the chips with readily accessible healthy snacks, so I don’t quit snacking, but snack more healthily.
2. Use frequent reinforcements and immediate feedback
Once we have started on the behavior, we need frequent reinforcements to see it through to completion. In the midst of my jog, I’m always tempted to quit and so must be fueled by frequent reinforcements such as encouraging self-talk, or immediate feedback of my incremental gains provided by the upward count of the number of steps or distance displayed on my fitness app. Listening to music to distract from the monotony of the jog also works well. The feeling of accomplishment at the end of the exercise can also be rewarding.
3. Get support or make a social commitment.
Sometimes we all need an extra push to stay on top of our resolutions. This is where others come in. In a large study conducted on new year’s resolutions, participants who had more support with their goals were more successful in attaining them. Compared to the no support group, these supported participants named at least one person that would support their progress and were given support in the form of follow-up emails and information and exercises on how to cope with potential hurdles when working towards their goals (Oscarsson, Carlbring, Andersson, & Rozental, 2020). If you can’t find anyone to exercise with, then at least create a social contract by letting a friend or family member know of your goals. This will count as support if you feel obliged to keep our word and follow through.
4. Just keep on keeping on, there is no fail
It is inevitable that we’ll have bad days, so we shouldn’t ditch the resolution just because we failed once, or twice, or thrice, or however many times. The important thing to do is just to keep moving. Besides, there are benefits in making new year’s resolutions:
So if you haven’t done so but still want to, it’s not too late to make some resolutions. Let us know how you're doing with your resolutions!
Britannica, T. Editors of Encyclopaedia (2021, July 23). Newton's laws of motion. Encyclopedia Britannica. https://www.britannica.com/science/Newtons-laws-of-motion
Covey, S. R., & Covey, S. (2020). The 7 habits of highly effective people. Simon & Schuster.
Dai, H., Milkman, K. L., & Riis, J. (2014). The fresh start effect: Temporal landmarks motivate aspirational behavior. Management Science, 60(10), 2563-2582.
Dai, H., Milkman, K. L., & Riis, J. (2015). Put your imperfections behind you: Temporal landmarks spur goal initiation when they signal new beginnings. Psychological science, 26(12), 1927-1936.
Fuster, J. (2015). The prefrontal cortex. Academic Press.
Oscarsson, M., Carlbring, P., Andersson, G., & Rozental, A. (2020). A large-scale experiment on New Year’s resolutions: Approach-oriented goals are more successful than avoidance-oriented goals. PLoS One, 15(12), e0234097.
Recognizing and Learning from Failure: Project Management Lessons from a Colorado Summit Bid
While failure is no fun, it is a part of life. Whether summitting mountains or managing projects, there are times when one must recognize the signs of failure, make the tough decision to call it a day, and learn from the experience to return smarter and stronger next time.
This past summer, I took a vacation to Colorado which was a blast. Three days on the agenda consisted of backpacking into the Chicago Basin, camping, and summiting 14ers (mountains over 14,000 feet). I had no prior mountaineering experience though was with someone who did, and we planned extensively. I had the right gear, was in good shape, and the conditions were ideal; I was feeling confident. On the second day, as I attempted a second summit, I began feeling fatigued, making mental errors, and falling behind. Though I tried to press on, I was aware that poor decisions and continued mistakes, especially near the summit, could be catastrophic (see Wickens, Keller, & Shaw, 2015).
I made the difficult call to abandon the summit bid and hiked down to lower elevation. Watching the beautiful sunrise over the basin lakes, I began to reflect on the failure, thinking about it in terms of project management given a summit bid, like a project, is “a temporary endeavor undertaken to create a unique…result” (Project Management Institute, 2008, p. 5). Though a project manager (PM) never plans to fail, they do identify the risks and notice the potential signs of failure so that they can correct course. As a last resort, a responsible PM may decide it is best to call it quits before stakeholder and company losses become too great.
Recognizing the Signs
In a Project Management Institute (PMI) published article "Managing Troubled Projects", Alaskar (2013) outlines signs that a project may be in jeopardy. Several of these apply to my summit bid:
Learning from Failure
In another PMI published article, Ranganath (2006) presents a learning from experience (LifE) cycle for project management that applies to dealing with project failure. The cycle involves:
Though it may be a bit painful, tell us about a project failure whether work related or not. If you like to climb mountains, tell us about that too!
Alaskar, A. H. (2013). Managing troubled projects. Paper presented at PMI® Global Congress 2013—North America, New Orleans, LA. Newtown Square, PA: Project Management Institute.
Project Management Institute. (2008). A guide to the project management body of knowledge (PMBOK guide) (4th ed.). Project Management Institute.
Ranganath, P. G. (2006). LIfE—learning and improving from experience. Paper presented at PMI® Global Congress 2006—North America, Seattle, WA. Newtown Square, PA: Project Management Institute.
Wickens, Christopher D.; Keller, John W.; and Shaw, Christopher (2015) "Human Factors in High-Altitude Mountaineering," Journal of Human Performance in Extreme Environments: Vol. 12 : Iss. 1 , Article 1.
This semester QIC welcomes Natalie Paquette as a Human Factors Intern! Natalie is a Ph.D. student in the Human Factors and Cognitive Psychology program at the University of Central Florida (UCF). Natalie earned her MA in Applied Experimental and Human Factors Psychology in 2020 at UCF and her MA in Psychology focused on Cognitive and Behavioral Neuroscience at George Mason University in 2017. Her work has examined performance issues related to mismatched expectations, reliance on visual working memory, and the effect of restricted time intervals on error processing. Natalie’s interests include examining the neurophysiological and perceptual aspects of cognition and performance in various environments to determine optimal parameters for successful task completion.
This semester QIC welcomes Nicolas Uszak as a Human Factors Intern! Nicolas is a Ph.D. student at the University of Central Florida’s Human Factors and Cognitive Psychology Program. Nicolas has a Master’s in Applied Experimental Human Factors and a graduate certificate for Design Usability in Industrial Engineering, both from UCF. Previously he graduated summa cum laude with his B.A. in Psychology from Cleveland State University. His interests lie in motivation, situational awareness, automation, multi-tasking, vigilance, and machine learning. Nicolas is currently working on his dissertation involving situational awareness while operating automated vehicles.
I've attended and presented at several conferences this year, such as the Human Systems Digital Experience, World Aviation Training Summit (WATS), and Applied Human Factors and Ergonomics (AHFE), and have noticed a simple yet powerful construct appearing over and over again…self-efficacy. Self-efficacy is "concerned with people’s beliefs in their capabilities to produce given attainments" (Bandura, 2006). In other words, it's the confidence in the ability to exert control over one's own motivation, behavior, and social environment (Carey & Forsyth, 2009). Extensive evidence has shown self-efficacy to be a significant predictor across a variety of contexts and domains, such as college academic performance (Choi, 2005), pre-career pilot performance (Wilson, 2021), weight loss success (Armitage et al., 2014), health management (Arslan, 2012), second language skills (Raoofi, Tan, & Chan, 2012), and work burnout and engagement (Ventura, Salanova, & Lloren, 2015). While there are a host of other areas that have explored the role of self-efficacy, one of particular interest to me that has been gaining more attention is usability.
Usability assessments typically focus on effectiveness, efficiency, and satisfaction, but research suggests the integration of self-efficacy can provide a robust assessment (Martin, 2007). As the preponderance of technological solutions continuously diffuses across all aspects of our personal and work lives, our dependency on them will impact our ability to complete tasks. Therefore, belief in our ability to complete a task with a technological solution should impact the way in which these solutions are designed. Plenty of evidence exists indicating the role of self-efficacy in the adoption of technology, such as mobile learning solutions (Bettayeb, Alshurideh, Al Kurdi, 2020), desktop virtual environments for learning (Makransky & Petersen, 2019), fitness devices (Rupp, Michaelis, McConnel, Smither, 2018), and medical support tools (Lindblom, Gregory, Wilson, Flight, & Zajac, 2012).
Although some usability measures focus on or integrate the concept of self-efficacy, not all are implemented correctly based on Bandura's guidance (2006). One major flaw is using Likert-type bipolar ratings (e.g., strongly agree to strongly disagree) instead of unipolar ones (e.g., 0 to 10). The issue is if you have zero confidence in your ability to complete a task, then negative ratings below zero make little sense and lead to skewed interpretations of the results. Further, when bipolar ratings are used, the mid-point (usually labeled as neither agree nor disagree) gets converted into a moderate-level of self-efficacy which is incorrect and not a true reflection of the construct (Bandura, 2012). Leveraging self-efficacy as a usability metric can provide valuable insight into the design and evaluation of technology, but it's critical that measures be developed, implemented, and interpreted appropriately.
Bandura (2012) stated that "there is no single all-purpose measure of self-efficacy with a single validity coefficient." This indicates that it's expected for new measures of self-efficacy to be created for, as he puts it, "activity domains." Activity domains are the topic areas in which the tasks under evaluation are performed. For example, evaluating self-efficacy for driving a monster truck. Use these guidelines when developing your measures and scales for usability evaluations (or any evaluation for that matter):
Have you been capturing self-efficacy as part of your usability assessments? Tell us how.
Armitage, C. J., Wright, C. L., Parfitt, G., Pegington, M., Donnelly, L. S., & Harvie, M. N. (2014). Self-efficacy for temptations is a better predictor of weight loss than motivation and global self-efficacy: Evidence from two prospective studies among overweight/obese women at high risk of breast cancer. Patient Education and Counseling, 95(2), 254-258.
Arslan, A. (2012). Predictive power of the sources of primary school students' self-efficacy beliefs on their self-efficacy beliefs for learning and performance. Educational Sciences: Theory and Practice, 12(3), 1915-1920.
Bandura, A. (2006). Guide to the construction of self-efficacy scales. In Pajares, F., Urdan, T. (Eds.), Self-efficacy beliefs of adolescents, Vol. 5: 307-337. Greenwich, CT: Information Age.
Bandura, A. (2012). On the functional properties of perceived self-efficacy revisited. Journal of Management, 38(1), 9–44.
Bettayeb, H., Alshurideh, M. T., & Al Kurdi, B. (2020). The effectiveness of mobile learning in UAE universities: a systematic review of motivation, self-efficacy, usability and usefulness. Control and Automation, 13(2), 1558-1579.
Carey, M.P. & Forsyth, A.D. (2009). Teaching tip sheet: Self-efficacy. American Psychological Association. https://www.apa.org/pi/aids/resources/education/self-efficacy.
Choi, N. (2005). Self‐efficacy and self‐concept as predictors of college students' academic performance. Psychology in the Schools, 42(2), 197-205.
Lindblom, K., Gregory, T., Wilson, C., Flight, I.H.K., & Zajac, I. (2012). The impact of computer self-efficacy, computer anxiety, and perceived usability and acceptability on the efficacy of a decision support tool for colorectal cancer screening, Journal of the American Medical Informatics Association, 19(3), 407–412.
Makransky, G. & Petersen, G. B. (2019). Investigating the process of learning with desktop virtual reality: A structural equation modeling approach. Computers & Education, 134, 15-30.
Martin, C. V. (2007). The Importance of self-efficacy to usability: Grounded theory analysis of a child’s toy assembly task. Proceedings of the Human Factors and Ergonomics Society Annual Meeting, 51(14), 865–868.
Raoofi, S., Tan, B. H., & Chan, S. H. (2012). Self-Efficacy in Second/Foreign Language Learning Contexts. English Language Teaching, 5(11), 60-73.
Rupp, M. A., Michaelis, J. R., McConnell, D. S., & Smither, J. A. (2018). The role of individual differences on perceptions of wearable fitness device trust, usability, and motivational impact. Applied Ergonomics, 70, 77-87.
Ventura, M., Salanova, M., & Llorens, S. (2015). Professional self-efficacy as a predictor of burnout and engagement: The role of challenge and hindrance demands. The Journal of Psychology, 149(3), 277-302.
Wilson, N. (2021). Pre-Career Pilots and Motivation – What is the Best Predictor of Performance? 23rd World Aviation Training Summit (WATS), Orlando, FL, June 15-16, 2021.
These posts are written or shared by QIC team members. We find this stuff interesting, exciting, and totally awesome! We hope you do too!