machine learning bias (algorithm bias or AI bias)

Contributor(s): Corinne Bernstein

Machine learning bias, also known as algorithm bias or AI bias, is a phenomenon that occurs when an algorithm produces results that are systematically prejudiced due to erroneous assumptions in the machine learning process. Algorithms can have built-in biases because they are created by individuals who have conscious or unconscious preferences that may go undiscovered until the algorithms are used, and potentially amplified, publically.

High bias is a reflection of problems related to the gathering or usage of data, where systems draw improper conclusions about data sets. This is often due to human intervention or the researchers' lack of cognitive assessment. Types of cognitive bias that can be inadvertently applied to algorithms are stereotyping, bandwagon effects, confirmation bias, priming and selective perception.

Machine learning, a subset of artificial intelligence, depends on the quality, objectivity and size of learning data sets. Since machine-learning algorithms and pattern recognition abilities operate in the world defined by the data used to calibrate them, a lack of truly random or complete data can conclude in bias.

Eliminating harmful biases is essential because machine learning is often applied to decisions with business implications, such as which individuals to approve for a loan and which applicants to offer a job interview, and personal implications, such as diagnostics in medical environments. One example of machine learning bias was observed in the initial rollout of Google's facial recognition feature as users of varying race were often incorrectly tagged as inhuman or ignored completely.

Organizations should check the data being used to train machine-learning models for comprehensiveness and bias. The data should be representative of different races, genders, backgrounds and cultures who could be adversely affected. Data scientists developing the algorithms should shape data samples in a way that minimizes bias and decision-makers should evaluate when it is appropriate, or inappropriate, to apply machine learning technology.

This was last updated in October 2018

Continue Reading About machine learning bias (algorithm bias or AI bias)

Dig Deeper on Pattern recognition and machine learning

Join the conversation

1 comment

Send me notifications when other members comment.

Please create a username to comment.

How does your organization work to detect and eliminate machine learning bias?


File Extensions and File Formats

Powered by: