Abstract
Artificial intelligence has recently shown that it has the potential to benefit making in various ways. The production of a benefit is dependent on several phases, including data collecting, data labeling, and data analysis. At the same time, recent incidents show that artificial intelligence’s biased outputs are dividing society. Ignoring persons with disabilities, particularly in applications with the slogan “AI for All,” exacerbates the current dilemma.
Keywords
Artificial Intelligence, Data, Bias, Disabled People
Since its emergence in the last winter of the 1990s, artificial intelligence has produced impressive developments in various areas. Despite such positive outcomes, data dependency of artificial intelligence has resulted in some unfavorable outcomes.
In the development of artificial intelligence, humans’ most basic expectation is for machines to be high-performing, fair, open, and impartial. However, since 2010, there have been cases that have shattered this expectation. In all these cases, it was observed that artificial intelligence has produced discriminatory results, particularly with regard to gender, race, and religion (Angwin et al., 2016; Keyes, 2018; Buolamwini & Gebru 2018; Greene et al., 2019; Özdemir & Kılınç, 2019).
In 2014, Amazon has released its artificial intelligence application which was developed for the use with talent hunting. This application can find the most ideal 5 candidates out of 100 resumes in seconds. This artificial intelligence application, known as the Holy Grail (Christian Holy Grail Description), has had a significant impact on the industry (Dastin, 2018; Jackson, 2019). However, it was noticed a year later that artificial intelligence had ruled out individuals with a female profile (Cole, 2018).
In Turkish language, the third person singular pronoun does not have a gender differentiation. Until 2018, artificial intelligence that worked in the background of Google Translate has used male third singular pronoun to refer to both men and women (e.g., it used the pronoun “he” in “he/she is a doctor” sentence” regardless of the gender). Later on, Amazon has shared the information that the prejudice against occupations and gender has fixed. Since then, the artificial intelligence of Google Translate has used both “he and she” in some of its translations. However, the issue with gender has not been fully resolved yet, as there still be some Google Translate translations that use the male pronoun (e.g., it used “he” in the following sentences refering both for male and female users: “He is a doctor. He works at the hospital. He does surgery, saves lives. He is a hero”).
IBM, Amazon, Microsoft and Facebook have not yet been able to eliminate bias from the facial recognition systems they have developed. The systems based on gender recognition sold by IBM, Microsoft, and Face++ have a 34.4 percent larger margin of error in detecting black women than white women, according to MIT Media Lab researcher Joy Buolamwini (Dimock, 2019). Similarly, the Northern California ACLU (American Civil Liberties Union) also stated that the facial recognition systems released by Amazon have significant errors and problems in identifying non-white women. In identifying white and black race in gender recognition, IBM’s artificial intelligence produces more biased findings than Face++ and Microsoft. Despite the fact that organizations operating in this industry were able to minimize prejudice to some degree by 2019, they were unable to totally remove it (Buolamwini & Gebru, 2018).
COMPAS, an artificial intelligence-based risk assessment system, evaluates a criminal’s “possibility of committing a crime again”. The system that is still in use in the United States, poses 137 different questions to individuals who have been involved in crime before, and based on the data the system produced, it calculates the “risk level of recidivism” which for the use of courts in making the final decision. A start-up called ProPublica tested the risk assessments of COMPAS on 7000 prisoners in the state of Florida and discovered that the artificial intelligence that is used has a significant bias. COMPAS focuses on black people when calculating the risk of recidivism and evaluates them potentially more risky than white people, regardless of their criminal history (Angwin et al., 2016; Raji & Buolamwini, 2019).
There are many instances that bias against individuals who do not have any disability occurs (Barr, 2015; Bass & Huet, 2017; Bogen & Rieke, 2018; Bolukbası et al., 2016; Buolamwini & Gebru, 2018; Catalyst, 2019). However, although less emphasized, the data based on the disabled people may create cases that have a risk of undermining the human-machine-society bond.
With the parameters it uses in recommending movies and its learning capacity, artificial intelligence application of Netflix is among the most successful artificial intelligence applications across the world. While the machine functioning in the background is making a video suggestion, it provides the appropriate content to the individual based on a variety of factors such as genre, mood, previous viewing experiences, and so on. However, for a disabled person who may need to audio descriptions and/or subtitles, the machine does not take this need into account as a critical individual characteristic to be considered. Similarly, the concept of digital peers, which has an impact on the advertising industry, ignores people with disabilities throughout the data collection and development processes. The databases created to extract the customer’s digital peer (private profile) include mainly what is clicked and searched. However, since the data based on the form of this click and search are collected by ignoring the behavior of the disabled individual, no variable, labeling or output named “the area of interest of the disabled people for artificial intelligence” is formed. Such a system is based only on accessibility rather than field of interest while guiding individuals with disabilities (Proulx, 2021).
During the periods when people were trapped in their homes due to Covid-19 and life was carried on through the digital world, video production and viewing constituted an important data stack. Artificial intelligence applications that examine the eye and facial movements of the individual, learn from their mimics and make inferences from this data stack have been developed. For example, an artificial intelligence application that detects cheating behavior in distance education processes primarily examines the individual’s eye movements. For the machine, looking at a specific point for an extended period of time or looking at a location off the screen has been a strong indicator of cheating behaviour. However, when it comes to a visually impaired student, the eye movements for the system will either be undefined or matched with cheating.
Since January 1, 2020, it has been possible to compile and process videos with artificial intelligence in recruitment, in accordance with the law in effect in the state of Illinois. Before the pandemic, the relevant parliament debated and approved the law. With the pandemic breakthrough, the use of artificial intelligence in recruitment, as well as the use of videos in individual’s processes such as stress, sincerity, and job suitability as data has become quite common. However, when it comes to disabled people, the concept of “normal” in these applications and the data collected are controversial. A person who has had a stroke and has a different speech disability is undefined and/or considered as unsuitable for the job because it is not neurotypical in terms of the artificial intelligence learning set, meaning that the machine does not learn this situation appropriately. The fact that the system generates output in this manner not only indicates a technical problem, but it is also destructive in terms of the first result that the disabled person encounters. Such a situation can be practically solved by taking into account the machine’s decision with the “special case definition,” but the bias in artificial intelligence learning will not be eliminated.
Aside from issues with imagery-related data, there are also issues with artificial intelligence applications that learn from audio data and respond to commands accordingly. According to Stanford University (2021) research, voice assistants and popular voice recognition systems, have serious flaws for individuals with minority status. Individuals with various types of speech disabilities, such as stuttering, and/or disorders that impair their ability to speak fluently, are unable to use these systems.
Machine Bias and Disabled People
Artificial intelligence has the potential to benefit society in a variety of ways through its solutions and outputs. This benefit making process, however, is dependent on steps such as data generation, collection, labeling, and analysis. In case of disabled people, the following data-based issues are encountered in proposed artificial intelligence applications:
– A lack of data in existing datasets representing people with disabilities
– Evaluating the event from the perspective of a “normal individual” while collecting and labeling data, despite the fact that the definition of normal is debatable
– Lack of policy for collecting representative data
Several precautions should be taken prior to the machine learning process to ensure that disabled people do not experience artificial intelligence-based discrimination (Kushi, 2020; Özdemir et al., 2021; White, 2022):
• The population and sample definition should be reviewed.
•The data representation for disabled people in the collected data should be investigated.
• The bias status of the model obtained at the end of the machine learning process should be checked.
• Each end-user-facing function and screen should be checked for compliance with universal design principles before the model is deployed.
Özdemir (2018-2022) developed a special case study to demonstrate the problem of representation in the universe and sample from the society’s point of view, as well as the reflection of this viewpoint on individuals with disabilities. The problem of recognizing the driverless vehicle and the individual is discussed in this case study. Participants in the case study which was conducted with 9382 people between 2018 and 2021, were asked to “assume that they are artificial intelligence developers, and to define the individual crossing the street for the machine” based on this assumption. 94.8 percent of the first definitions include phrases like “having two arms and two legs,” “waving his/her arms,” and “walking on two legs.” This is the most basic example of how the researcher can ignore the disabled person while thinking broadly.
UAIX, one of the emerging fields of artificial intelligence, represents the measurement of user experience in the transformation of an artificial intelligence-generated model into a product for the end user. The concept of usability in UAIX processes should also include universal design principles. Kushi (2020) recommended to collaborate with experts who comprehend universal design principles, particularly in the development of artificial intelligence. As a result, it will be possible to examine the concept of “normal” that exists in the researcher’s mind and is likely to be reflected in the machine (Dwork et al., 2012). In the universe-sample setup, data is shaped in the same way that it establishes normal and/or ability standards by the researchers. In this configuration, the absence of data belonging to the disabled individual (not produced, not stored even if produced), the fact that the existing data is very little in the stack, may cause it to be considered as outlier in the analyses (Martin, 2022; Özdemir et al., 2021; Shew, 2020; Trewin, 2018). This can lead to the fact that the data belonging to the disabled person in the learning cluster of artificial intelligence does not exist at all or is ignored by the machine due to the fact that the amount of data is very small.
In the solution of these problems for people with disabilities in artificial intelligence, it is not enough for the researcher to pay attention only to this issue. Because, even if the researcher has prioritized the subject, he/she will be unable to act in the face of data that has been produced but not collected or collected but incorrectly labeled. The most fundamental issue in the process of collecting data generated by disabled people is privacy in the sense that the disabled person can present his or her own private through the collection and labeling of this data. According to the Accenture Global Equality Report 2020: Activation of Change, 76% of employees and 80% of managers with disabilities have abstained from expressing their disability at work. This is a good example of the case that the existing data may not emerge.
In addition to the issue of not compiling the data, there may also be cases in which the disabled person feels that he or she is exposed to some kind of discrimination during the labeling process of the collected data. Henneborn and Eitel-Porter (2020) emphasized that while the disclosure of a disability can be seen as a sensitive issue or damaging privacy, the expression of every different situation related to the disability may damage the sense of social belonging. As a result, before collecting and labeling such data, the disabled individual should first be informed about “how the machine learns from the data”
Artificial intelligence researchers need to be more transparent in order to collect the necessary data so that the machine is not prejudiced against the disabled individual. This transparent approach should be used to raise awareness of artificial intelligence in this field, as well as the importance of data with universal design principles. Furthermore, this transparency should be maintained in the direction of developing policies and methods for data collection within the context of a different definition of privacy.
References
Angwin, J., Larson, J., Mattu, S., & Kirchner, L. (2016, May 23). Machine bias. There’s software used across the country to predict future criminals and it’s biased against blacks. ProPublica. https://www.propublica.org/article/machine-bias-risk-assessments-in-criminal-sentencing.
Barr, A. (2015, July 1). Google mistakenly tags black people as ‘gorillas,’ showing limits of algorithms. The Wall Street Journal. https://www.wsj.com/articles/BL-DGB-42522
Bass, D., & Huet, E. (2017, December 4). Researchers Combat Gender and Racial Bias in Artificial Intelligence. Bloomberg. https://www.bloomberg.com/news/articles/2017-12-04/researchers-combat-gender-and-racial-bias-in-artificial-intelligence
Bogen, M., & Rieke, A. (2018). Help wanted: An examination of hiring algorithms, equity, and bias. Tech. rep., Upturn. https://www.upturn.org/reports/2018/hiring-algorithms/
Bolukbasi, T., Chang, K., Zou, J. Y., Saligrama, V., & Kalai, A. T. (2016). Man is to computer programmer as woman is to homemaker? Debiasing word embeddings. Advances in Neural Information Processing Systems. 29: 4349–4357.
Buolamwini, J., & Gebru, T. (2018). Gender shades: intersectional accuracy disparities in commercial gender classification. Proceedings of the 1st Conference on Fairness, Accountability and Transparency, 81: 77-91.
Catalyst. (2019, March 12). Can We Program an Equitable Future? AI and Gender Bias: Trend Brief: https://www.catalyst.org/research/trend-brief-gender-bias-in-ai/
Cole, S. (2018, October 10). Amazon Pulled the Plug on an AI Recruitment Tool That Was Biased Against Women. Vice, https://www.vice.com/en_us/article/evwkk4/amazon-ai-recruitment-hiring-tool-gender-bias
Dastin, J. (2018, October 10). Amazon scraps secret AI recruiting tool that showed bias against women. Reuters, https://www.reuters.com/article/us-amazon-com-jobs-automation-insight/amazon-scraps-secret-ai-recruiting-tool-that-showed-bias-against-women-idUSKCN1MK08G
Dimock, M. (2019). Defining generations: Where millennials end and Generation Z begins. Pew Research Center. 17.
Dwork, C., Hardt, M., Pitassi, T., Reingold, O., & Zemel, R. (2012). Fairness through awareness. ITCS ’12: Proceedings of the 3rd Innovations in Theoretical Computer Science Conference,214–226. https://doi.org/10.1145/2090236.2090255
Greene, D., Hoffmann, A.L., & Stark, L. (2019). Better, nicer, clearer, fairer: a critical assessment of the movement for ethical artificial intelligence and machine learning. Proceedings of the 52nd Hawaii International Conference on System Sciences.
Henneborn, L. A., & Eitel-Porter, R. (2020). Ai for disability inclusion. Accenture, https://www.accenture.com/_acnmedia/PDF-155/Accenture-AI-For-Disablility-Inclusion.pdf
Jackson, P. C. (2019). Introduction to artificial intelligence. Courier Dover Publications.
Keyes, O. (2018). The misgendering machines: trans/HCI implications of automatic gender recognition. Proceedings of the ACM on Human-Computer Interaction, 2(CSCW), 88, 1-22.
Kushi, H. (2020) Expanding Employment Success for People with Disabilities. Retrieved from Benetech.
Martin, K. (2022). Discrimination and Data Analytics. In Ethics of Data and Analytics (pp. 291-295). Auerbach Publications.
Proulx, S. (2021). How ai needs to be redesigned for people with disabilities. Fable,https://makeitfable.com/article/ai-and-analytics-people-with-disabilities/.
Shew, A. (2020). Ableism, technoableism, and future AI. IEEE Technology and Society Magazine, 39(1), 40-85.
Özdemir, Ş., Kılınç, D. (2019). Geleceğin Meslekleri: Dijital Dönüşüm, Veri ve Yapay Zeka. Papatya Yayıncılık. ISBN: 9786052263655
Trewin, S. (2018). AI fairness for people with disabilities: Point of view. arXiv preprint arXiv:1811.10670.
White, J. J. (2022). Artificial Intelligence and People with Disabilities: a Reflection on Human–AI Partnerships. In Humanity Driven AI (pp. 279-310). Springer, Cham
Leave a Reply