Summary: This post discusses using the pointing-and-calling method when conducting accessibility tests. I attempt to argue that this occupational safety technique can help expose accessibility barriers as well.
Introduction
Last week I had the opportunity to speak at Ad Hoc’s monthly Engineering All-Hands meeting about accessibility testing. Given that I was talking to all the engineers company-wide, I wanted to make sure I left them with a clear call-to-action.
The engineering practice at Ad Hoc is generally very supportive of accessibility—Ad Hoc in general is probably the most accessibility-aware place I’ve ever worked, which makes my job 1000 percent easier. But a large share of the audience didn’t have deep experience thinking about accessibility or doing much manual accessibility testing. I wanted to give them easy takeaways that they could make a part of their daily development immediately.
I landed on six action items:
- Use an automated testing tool (eg. axe by Deque) to check for detectable code issues.
- Use a manual testing tool helper (eg. ANDI) to better expose accessibility features when reviewing a page.
- Do basic keyboard navigation testing to make sure every interactive element can be reached by [Tab] and the order makes sense for completing tasks.
- Check content reflow at high-zoom (200 percent, 300 percent, and 400 percent) and verify there’s no horizontal scrolling and no functionality is lost.
- Find a friend or colleague and talk about what you’re doing as you test.
- Try a screen reader (NVDA or VoiceOver) and listen to what it sounds like to have your whole page read out loud.
The goal here is to better pay attention to the accessibility features and barriers of whatever product you’re working on—without expecting deep knowledge of WCAG as a prerequisite. It’s not a perfect list and if I give the same presentation again in six months it might look a little different. But I think it’s a decent start.
What I want to write about today is item five on that list: “Find a friend or colleague and talk about what you’re doing as you test.” I wanted to include something on the list about maintaining awareness of possible barriers and being skeptical of your own code or the design decisions behind them. But I couldn’t think of a way to say it concisely and make it actionable and intuitive.
And then I remembered pointing-and-calling.
Pointing and calling
I’ll let Wikipedia’s “Pointing and calling” article define this for me:
Pointing and calling is a method in occupational safety for avoiding mistakes by pointing at important indicators and verbally calling out their status. It is particularly common on Japanese railways, where it is referred to as shisa kanko (指差喚呼), shisa kakunin kanko (指差確認喚呼) or yubisashi koshō (指差呼称); and in Chinese railways, where it is called “指差呼唤” (Pinyin: zhǐchā hūhuàn). Gesturing at and verbalizing these indicators helps with focus.
I’m not going to pretend to understand the neurology of pointing and calling—co-actions and co-reactions with a deliberate physical gesture and spoken verbal statement makes your brain do something to engage with the task better, or something. But it apparently works! Wikipedia cites a 1994 study that found an 85 percent decrease in mistakes when performing tasks.
This is the best kind of safety technique: easy to understand, easy to implement, easy to internalize as habit.
Manual accessibility testing
When conducting an accessibility review, I find that there’s a lot of “I know it when I see it” involved. So much of accessibility is context-dependent: Does the alt text accurately describe what the image is trying to communicate? Is the right amount of information announced for screen reader users to understand what’s changed on a page? Does the keyboard tab order make sense? Can a voice command user pronounce that link text?
As a tester, you can identify barriers before they end up in front of users by testing with assistive technologies and tools that simulate disabilities that you don’t have. But those tools are only as good as the awareness that you bring to your testing. Testing with a screen reader, for example, doesn’t do much good if you’re not also making an effort to listen critically and think about the interactions you’re experiencing.
That’s genuinely difficult. For a designer or an engineer, it’s difficult to perform a critical self-assessment of your own design or code under any circumstances. But it takes even more focus and effort to step outside of yourself and evaluate an interaction pattern that doesn’t feel intuitive to you. And if we’re asking designers and engineers to incorporate accessibility into their day-to-day work, it’s easy to imagine well-intentioned people settling into checking boxes in mental checklists rather than really engaging with their product.
Pointing and calling as you test
Enter pointing-and-calling. In transportation and industrial settings, there’s good evidence that it improves focus. I hypothesize that this is also true in white-collar settings as well (although I’m not aware of any evidence to support this).
I’ve also tried this myself, and found that for my limited single-person sample size, it’s a useful testing technique.
In particular, I’ve found that this method:
- Helps me be more deliberate about my testing steps. Rather than multitasking (eg. “I can test keyboard navigation and assess cognitive load at the same time!”), pointing and calling forces me to focus on a specific testing task.
- Highlights confusing or ambiguous content and interfaces. Realizing as you say something out loud that you’re not sure where you’re supposed to be pointing—or, conversely, pointing and realizing that you’re not sure how to say what you’re about to do—immediately identifies a potential barrier.
- Exposes hard-to-pronounce elements. In general, I feel like voice command support doesn’t get a lot of attention when doing accessibility testing. But the verbal part of this method forces some minimal simulation of the voice command user experience.
But maybe most importantly (and most vaguely), I find that pointing-and-calling activates that “I know it when I see it” sense. I’ve had multiple moments where I’ve stopped myself and thought “wait, that’s not right.” And that, of course, is exactly the outcome the train operators who pioneered this method were looking for.
For a designer or engineer testing their work and relatively new to accessibility, this is easy to learn and to adopt as a regular practice. Moreover, it shouldn’t take a lot of coaching for this method to expose certain HTML semantics (“Click… wait, is that a link or a button?”), cognitive load issues, and hopefully more.
This is not nothing! I do accessibility testing every day for my work and, theoretically, I’m fairly competent at it. But just by combining a physical gesture with spoken verbal statements I’m finding it improves my work.
Testing with a friend
Besides just speaking out loud as you test, I recommended that Ad Hoc engineers do this while working with a friend or colleague. Truthfully, I haven’t tried this as extensively as I ought too. Often I’m finishing my reviews right before my deadline to log feedback, and in a remote workplace spread across a continent it’s not always easy to coordinate finding someone to test with.
But I suspect that, for a sufficiently-engaged testing buddy, having a colleague point-and-call with you further activates that “I know it when I see it” sense. I know that I have a tendency to seek validation and confirmation from others when performing tasks in front of them. I would expect that increases my awareness of the tasks I’m performing and literally doubles the number of people paying attention to potential errors.
Assessing this method further
I’m writing this post for two big reasons:
- I’ve never seen anyone else link pointing-and-calling to accessibility testing (or software testing in general), and I’ve found some immediate utility in adopting this method.
- I’m hoping other people will try this too and help refine and formalize this practice.
There’s a lot of hand-waving in this post that would benefit from more rigorous exploration. I’m going to be thinking about ways to better assess its efficacy, but I’m hopeful that anyone reading this will also give it a try and share their experience back to me.