How to Measure Diversity Actionably in Technology
Hamid, M. M., Chatterjee, A., Guizani, M., Anderson, A., Moussaoui, F., Yang, S., Escobar, I. T., Sarma, A., & Burnett, M.
In Equity, Diversity, and Inclusion in Software Engineering: Best Practices and Insights, Apress · 2024
Demographic measurements ("only 37% of women would recommend our software") can say what goes wrong and how often — but not why, and without the why there is no actionable fix. This chapter presents a diversity measurement method powered by the validated GenderMag facet survey, and — for the first time — the exact steps of its scoring and validation process.
The problem
Improving how well a software system supports diverse users requires diversity measurements that are truly actionable — not just demographic statistics. Demography-based measurements can point out which features disproportionately affect diverse users and how often, but they cannot explain why the issues exist in the first place. Those "whys" are the missing link for translating empirical findings into design fixes.
What's needed is a fine-grained measurement device that relates technology "misfires" to individuals' traits relevant to technology usage. The GenderMag facets provide exactly such traits: five cognitive styles — motivations, information processing style, computer self-efficacy, attitude toward risk, and learning style — whose differences statistically cluster by gender.
What the chapter provides
The chapter presents the GenderMag facet survey — a validated 9-point Likert-scale survey that measures a respondent's value on each of the five facets — and a four-step scoring process: (1) convert responses to numeric scores, complementing reverse-keyed questions; (2) sum each facet's questions per participant; (3) compute facet medians relative to the participant's peer group; and (4) tag each participant's facet value as "Abi-like" or "Tim-like" — the two personas anchoring each facet's spectrum.
The survey originated in a collaboration with a Microsoft product team who wanted to validate the GenderMag facet values within their own customer base — and it has since become a general-purpose instrument for fine-grained diversity measurement of technology and technology-related artifacts such as user interfaces, documentation, and manuals.
Key results
- The survey extracts information on who runs across which inclusivity bugs and why — connecting each usability failure to the cognitive styles it disproportionately affects.
- It enables before/after comparisons of a technology's diversity support, e.g., to check whether a redesign actually closed a diversity gap.
- It shows developers and designers how to make empirical results actionable: because each facet value points at concrete design directions, findings translate directly into fixes.