AI for Good
AI for Good
Applying AI to society’s biggest challenges
Frequently Asked Questions
Unconscious Bias Detector
The machine learning model is trained for each of the bias categories. For example, in the examination of unconscious gender bias, the model is looking for differences in structured information, such as quantitative scoring based on gender, and unstructured information, such as performance review commentary, for differences in language, such as situations where commentary refers to personality traits more frequently for one gender than another.
Ultimately, the users of the analysis are making these decisions. The AI is helping to flag patterns that are very difficult for humans to detect in the large volume of data processed. The Text IQ analysis helps users identify these trends and then make decisions about whether bias is present and how it should be mitigated.
In general, the information used in the analysis is already available to employees, including for example their performance reviews. Additional information is already available in HR systems. Organizations can also anonymize employee names in the input and still identify bias by department, geography, manager, etc.
Absolutely, given a large volume of data from which to develop the machine learning model. Today’s solution is designed for performance evaluations and reviews, however.
Many organizations are already looking for these types of bias through review of their available data and, from those assessments, putting in place mitigation strategies. With this new technology, these organizations get much deeper insight into bias through the analysis of all the unstructured information that exists, including review commentary, interview notes, messages, and more. Unconscious bias that’s identified is then mitigated through the same approaches, such as training, process changes, manager coaching, and more.
The approach is applicable to any type of organization large enough to have an appropriate dataset to analyze. This includes public-traded and private companies, public sector agencies, and large membership/nonprofit organizations.
The answer is a range depending on the application, but a good rule of thumb is that organizations need at least 10,000 documents (for example, historical performance reviews) to benefit from an accurate model.
No, each organization benefits from a unique detector model built just for them. From our work with companies in other areas of sensitive data identification and categorization, we’ve learned that every organization has its own language, job functions, job titles, organizational structure and other variables that make sharing machine learning across organizations impractical. This approach also ensures that no private data is ever shared amongst users.