AI for Good

Applying AI to society’s biggest challenges

Frequently Asked Questions

Unconscious Bias Detector

How is the AI discovering unconscious bias?

The machine learning model is trained for each of the bias categories. For example, in the examination of unconscious gender bias, the model is looking for differences in structured information, such as quantitative scoring based on gender, and unstructured information, such as performance review commentary, for differences in language, such as situations where commentary refers to personality traits more frequently for one gender than another.

How is the decision made as to whether bias is present?

Ultimately, the users of the analysis are making these decisions. The AI is helping to flag patterns that are very difficult for humans to detect in the large volume of data processed. The Text IQ analysis helps users identify these trends and then make decisions about whether bias is present and how it should be mitigated.

How is privacy protected for individuals within an organization?

In general, the information used in the analysis is already available to employees, including for example their performance reviews. Additional information is already available in HR systems. Organizations can also anonymize employee names in the input and still identify bias by department, geography, manager, etc.

Can unconscious bias in hiring also be identified?

Absolutely, given a large volume of data from which to develop the machine learning model. Today’s solution is designed for performance evaluations and reviews, however.

What actions should an organization take based on the results?

Many organizations are already looking for these types of bias through review of their available data and, from those assessments, putting in place mitigation strategies. With this new technology, these organizations get much deeper insight into bias through the analysis of all the unstructured information that exists, including review commentary, interview notes, messages, and more. Unconscious bias that’s identified is then mitigated through the same approaches, such as training, process changes, manager coaching, and more.

For what types of organizations is this suitable?

The approach is applicable to any type of organization large enough to have an appropriate dataset to analyze. This includes public-traded and private companies, public sector agencies, and large membership/nonprofit organizations.

How large of a dataset is needed to get started?

The answer is a range depending on the application, but a good rule of thumb is that organizations need at least 10,000 documents (for example, historical performance reviews) to benefit from an accurate model.

Are the unconscious bias detector models built from shared data across companies?

No, each organization benefits from a unique detector model built just for them. From our work with companies in other areas of sensitive data identification and categorization, we’ve learned that every organization has its own language, job functions, job titles, organizational structure and other variables that make sharing machine learning across organizations impractical. This approach also ensures that no private data is ever shared amongst users.

If you have additional questions or media/press inquiries, please contact us at

Interested in exploring options to use Text IQ AI for good in your organization?

Please share your information, and one of our Text IQ specialists will be in touch.