Why collect subjective data from the end-users when evaluating a training device? I was asked this question at a conference last month, almost as if subjectivity is a dirty word. The short answer is that if your users don't like the training device, they likely won't use it. If they don't use it, you won't get the objective data you may be looking for. The term "like" does not necessarily mean the training experience was pleasant. For instance, Soldiers, firefighters, surgeons, etc., must train under difficult, often intolerable conditions to prepare for the real-world challenges of their job. "Liking" the training, in such cases, means that the trainees see the value in preparing them to do their job successfully. When we collect subjective data, we also ask users if their expectations were met, what their emotional responses were from their experience, and most importantly, why. If we only gather objective data, such as the time to complete a task or the number of errors made, then we are only getting half the story. For example, why did it take so long to complete the task and why were certain errors made?
On a project for the U.S. Air Force, QIC's role was to conduct usability and user experience evaluations on a virtual flight simulator (Abich & Sikorski, 2022; Abich, Montalbano, & Sikorski, 2021). One of the most compelling responses I heard from an instructor pilot as he walked into the room was, "This looks like training." I reflected on that and thought, "What does it mean to look like training? How does the user's initial impression affect the feedback provided? How does the design impact their motivation to give the training device a chance? And why should any of this matter if the training device does what it's supposed to do?" The answers to these questions are subjective as end-users apply their unique expertise, experience, and expectations in evaluating the device. They are the ones that will have to use the devices and can see the value or potential shortcomings. If their lives depend on the skills, you can bet they will be particularly critical of any new training device. So next time you do a training device evaluation, embrace the subjective and don't worry about getting a little dirty. Otherwise, you may be missing out on some highly valuable feedback.
Happy International Women's Day! As a mother working in the defense industry for a female-owned company, I must admit that today is a mixed bag of emotions. For starters, this article from last month lists the high-profile glass-ceiling-breaking women that have chosen to leave the workforce, stating, “The pattern has the potential to unwind decades of progress toward gender equity and increased female leadership in the workplace." Add to that the recent setbacks to policy that further disadvantage women such as reductions in benefits to lower-income families primarily led by single mothers.
During the pandemic, over 2 million women left their careers to care for their families due to schools shutting down. One in 3 childcare centers permanently closed down. We are barely getting back into the stride of returning to pre-pandemic employment numbers, but the field is far from level. Eve Rodsky’s book and consequent documentary, Fair Play, further illustrate how the US has the worst family-friendly public policy in the developed world. For example, US & Papua New Guinea are the only two countries in the world with no federal paid maternity leave. The picture is rather stark.
It's no surprise that women are burned or burning out. Our society does not have the infrastructure to support women. According to lawyer and US Rep Katy Porter, "There are lots of things we do in government that cost a lot. There is a hidden message there; it's just too expensive to support women. Wouldn't it be just cheaper if we just keep letting women do it all for free?"
Thankfully, at QIC, I report to the COO who meets with me weekly to ensure not only that my workload is balanced, but that I have the resources I need to be successful, including the flexibility necessary to maintain a work-life balance. I wish I could say this took having tough conversations to achieve, but the truth is, our company culture not only encourages us to be upfront and authentic, it requires it.
While there is certainly more work to be done to support women, especially mothers, I am proud of the conversations I hear taking place around me. We are finally vocalizing so much of what has been deeply felt for too long. Looking ahead, I hope we can turn this one-day celebration into a year-round push for equality.
If any of this interests you, check out this study from the LeanIn organization for more information and this site for ways to get involved.
These posts are written or shared by QIC team members. We find this stuff interesting, exciting, and totally awesome! We hope you do too!