{"id":266370,"date":"2020-08-25T05:00:52","date_gmt":"2020-08-25T09:00:52","guid":{"rendered":"https:\/\/canadianinquirer.net\/v1\/?p=266370"},"modified":"2020-08-25T10:06:14","modified_gmt":"2020-08-25T14:06:14","slug":"ai-technologies-like-police-facial-recognition-discriminate-against-people-of-colour","status":"publish","type":"post","link":"https:\/\/canadianinquirer.net\/v1\/2020\/08\/25\/ai-technologies-like-police-facial-recognition-discriminate-against-people-of-colour\/","title":{"rendered":"AI technologies \u2014 like police facial recognition \u2014 discriminate against people of colour"},"content":{"rendered":"<figure id=\"attachment_266374\" aria-describedby=\"caption-attachment-266374\" style=\"width: 864px\" class=\"wp-caption alignnone\"><a href=\"https:\/\/canadianinquirer.net\/v1\/wp-content\/uploads\/2020\/08\/flat-3252983_960_720.png\"><img loading=\"lazy\" decoding=\"async\" class=\"size-full wp-image-266374\" src=\"https:\/\/canadianinquirer.net\/v1\/wp-content\/uploads\/2020\/08\/flat-3252983_960_720.png\" alt=\"\" width=\"864\" height=\"720\" srcset=\"https:\/\/canadianinquirer.net\/v1\/wp-content\/uploads\/2020\/08\/flat-3252983_960_720.png 864w, https:\/\/canadianinquirer.net\/v1\/wp-content\/uploads\/2020\/08\/flat-3252983_960_720-300x250.png 300w, https:\/\/canadianinquirer.net\/v1\/wp-content\/uploads\/2020\/08\/flat-3252983_960_720-768x640.png 768w\" sizes=\"auto, (max-width: 864px) 100vw, 864px\" \/><\/a><figcaption id=\"caption-attachment-266374\" class=\"wp-caption-text\">Experts suggest that Williams is not alone, and that others have been subjected to similar injustices. The ongoing controversy about police use of Clearview AI certainly underscores the privacy risks posed by facial recognition technology. But it\u2019s important to realize that not all of us bear those risks equally. (Pixabay photo)<\/figcaption><\/figure>\n<p>Detroit police wrongfully arrested Robert Julian-Borchak Williams in January 2020 <a href=\"https:\/\/www.nytimes.com\/2020\/06\/24\/technology\/facial-recognition-arrest.html\">for a shoplifting incident that had taken place two years earlier<\/a>. Even though Williams had nothing to do with the incident, facial recognition technology used by Michigan State Police \u201cmatched\u201d his face with a grainy image obtained from an in-store surveillance video showing another African American man taking US$3,800 worth of watches.<\/p>\n<p>Two weeks later, the case was dismissed at the prosecution\u2019s request. However, relying on the faulty match, police had already handcuffed and arrested Williams in front of his family, forced him to provide a mug shot, fingerprints and a sample of his DNA, interrogated him and imprisoned him overnight.<\/p>\n<p>Experts suggest that Williams is not alone, and that others have been subjected to similar injustices. The ongoing controversy about police use of Clearview AI certainly underscores the privacy risks posed by facial recognition technology. But it\u2019s important to realize that <a href=\"https:\/\/www.nytimes.com\/2019\/12\/19\/technology\/facial-recognition-bias.html\">not all of us bear those risks equally<\/a>.<\/p>\n<hr \/>\n<p><em><br \/>\n<strong><br \/>\nRead more:<br \/>\n<a href=\"https:\/\/theconversation.com\/the-coronavirus-pandemic-highlights-the-need-for-a-surveillance-debate-beyond-privacy-137060\">The coronavirus pandemic highlights the need for a surveillance debate beyond &#8216;privacy&#8217;<\/a><br \/>\n<\/strong><br \/>\n<\/em><\/p>\n<hr \/>\n<h2>Training racist algorithms<\/h2>\n<p>Facial recognition technology that is <a href=\"https:\/\/www.wired.com\/story\/best-algorithms-struggle-recognize-black-faces-equally\/\">trained on and tuned to Caucasian faces<\/a> systematically misidentifies and mislabels racialized individuals: numerous studies report that facial recognition technology is \u201c<a href=\"https:\/\/www.aclu.org\/news\/privacy-technology\/wrongfully-arrested-because-face-recognition-cant-tell-black-people-apart\/\">flawed and biased, with significantly higher error rates when used against people of colour<\/a>.\u201d<\/p>\n<p>This <a href=\"https:\/\/www.mirror.co.uk\/tech\/apple-accused-racism-after-face-11735152\">undermines the individuality and humanity of racialized persons<\/a> who are more likely to be misidentified as criminal. The technology \u2014 and the identification errors it makes \u2014 reflects and further entrenches long-standing social divisions that are deeply entangled with racism, sexism, homophobia, settler-colonialism and other intersecting oppressions.<\/p>\n<figure><iframe loading=\"lazy\" src=\"https:\/\/www.youtube.com\/embed\/vSuDE6wvQlU?wmode=transparent&amp;start=0\" width=\"440\" height=\"260\" frameborder=\"0\" allowfullscreen=\"allowfullscreen\"><\/iframe><figcaption><span class=\"caption\">A France24 investigation into racial bias in facial recognition technology.<\/span><\/figcaption><\/figure>\n<h2>How technology categorizes users<\/h2>\n<p>In his game-changing 1993 book, <a href=\"https:\/\/eric.ed.gov\/?id=ED377817\"><em>The Panoptic Sort<\/em><\/a>, scholar Oscar Gandy warned that \u201ccomplex technology [that] involves the collection, processing and sharing of information about individuals and groups that is generated through their daily lives \u2026 is used to coordinate and control their access to the goods and services that define life in the modern capitalist economy.\u201d Law enforcement uses it to pluck suspects from the general public, and private organizations use it to determine whether we have access to things like <a href=\"https:\/\/thehill.com\/blogs\/congress-blog\/technology\/459455-making-equitable-access-to-credit-a-reality-in-the-age-of\">banking<\/a> and <a href=\"https:\/\/hbr.org\/2019\/05\/all-the-ways-hiring-algorithms-can-introduce-bias\">employment<\/a>.<\/p>\n<p>Gandy prophetically warned that, if left unchecked, this form of \u201ccybernetic triage\u201d would exponentially disadvantage members of equality-seeking communities \u2014 for example, groups that are racialized or socio-economically disadvantaged \u2014 both in terms of what would be allocated to them and how they might come to understand themselves.<\/p>\n<p>Some 25 years later, we\u2019re now living with the panoptic sort on steroids. And examples of its negative effects on equality-seeking communities abound, such as the false identification of Williams.<\/p>\n<h2>Pre-existing bias<\/h2>\n<p>This sorting using algorithms infiltrates the most fundamental aspects of everyday life, occasioning both direct and structural violence in its wake.<\/p>\n<p>The direct violence experienced by Williams is immediately evident in the events surrounding his arrest and detention, and the individual harms he experienced are obvious and can be traced to the actions of police who chose to rely on the technology\u2019s \u201cmatch\u201d to make an arrest. More insidious is the <a href=\"https:\/\/www.thoughtco.com\/structural-violence-4174956\">structural violence<\/a> perpetrated through facial recognition technology and <a href=\"https:\/\/github.com\/MimiOnuoha\/On-Algorithmic-Violence\">other digital technologies<\/a> that rate, match, categorize and sort individuals in ways that magnify pre-existing discriminatory patterns.<\/p>\n<p>Structural violence harms are less obvious and less direct, and cause injury to equality-seeking groups through systematic denial to power, resources and opportunity. Simultaneously, it increases direct risk and harm to individual members of those groups.<\/p>\n<p>Predictive policing uses <a href=\"http:\/\/affinitymagazine.us\/2020\/05\/07\/predictive-policing-threatens-civil-liberties\/\">algorithmic processing of historical data to predict when and where new crimes are likely to occur<\/a>, assigns police resources accordingly and embeds enhanced police surveillance into communities, usually in lower-income and racialized neighbourhoods. This increases the chances that any criminal activity \u2014 including less serious criminal activity that might otherwise prompt no police response \u2014 will be detected and punished, ultimately limiting the life chances of the people who live within that environment.<\/p>\n<div data-react-class=\"InstagramEmbed\" data-react-props=\"{&quot;url&quot;:&quot;https:\/\/www.instagram.com\/p\/CDKKDxmlqEQ&quot;}\"><\/div>\n<p>And the evidence of inequities in other sectors continues to mount. <a href=\"https:\/\/www.wired.co.uk\/article\/gcse-results-alevels-algorithm-explained\">Hundreds of students in the United Kingdom<\/a> protested on Aug. 16 against the disastrous results of <a href=\"https:\/\/www.gov.uk\/government\/organisations\/ofqual\">Ofqual<\/a>, a flawed algorithm the U.K. government used to determine which students would qualify for university. In 2019, Facebook\u2019s microtargeting ad service <a href=\"https:\/\/www.cbc.ca\/news\/politics\/facebook-employment-job-ads-discrimination-1.5086491\">helped dozens of public and private sector employers<\/a> exclude people from receiving job ads on the basis of age and gender. Research conducted by ProPublica has documented <a href=\"https:\/\/www.propublica.org\/article\/breaking-the-black-box-when-algorithms-decide-what-you-pay\">race-based price discrimination for online products<\/a>. And search engines regularly produce racist and sexist results.<\/p>\n<hr \/>\n<p><em><br \/>\n<strong><br \/>\nRead more:<br \/>\n<a href=\"https:\/\/theconversation.com\/googles-algorithms-discriminate-against-women-and-people-of-colour-112516\">Google&#8217;s algorithms discriminate against women and people of colour<\/a><br \/>\n<\/strong><br \/>\n<\/em><\/p>\n<hr \/>\n<h2>Perpetuating oppression<\/h2>\n<p>These outcomes matter because they perpetuate and deepen pre-existing inequalities based on characteristics like race, gender and age. They also matter because they deeply affect how we come to know ourselves and the world around us, sometimes by <a href=\"https:\/\/fs.blog\/2017\/07\/filter-bubbles\/\">pre-selecting the information<\/a> we receive in ways that reinforce stereotypical perceptions. Even technology companies themselves acknowledge the <a href=\"https:\/\/www.digitaltrends.com\/news\/google-execs-bias-algorithms\/\">urgency of stopping algorithms from perpetuating discrimination<\/a>.<\/p>\n<p>To date the success of ad hoc investigations, conducted by the tech companies themselves, has been inconsistent. Occasionally, corporations involved in producing discriminatory systems withdraw them from the market, such as when <a href=\"https:\/\/priv.gc.ca\/en\/opc-news\/news-and-announcements\/2020\/nr-c_200706\/\">Clearview AI announced it would no longer offer facial recognition technology in Canada<\/a>. But often such decisions result from regulatory scrutiny or public outcry only <em>after<\/em> members of equality-seeking communities have already been harmed.<\/p>\n<p>It\u2019s time to give our regulatory institutions the tools they need to address the problem. Simple privacy protections that hinge on obtaining individual consent to enable data to be captured and repurposed by companies cannot be separated from the discriminatory outcomes of that use. This is especially true in an era when most of us (<a href=\"https:\/\/www.technologyreview.com\/2017\/04\/11\/5113\/the-dark-secret-at-the-heart-of-ai\/\">including technology companies themselves<\/a>) cannot fully understand what algorithms do or why they produce specific results.<\/p>\n<h2>Privacy is a human right<\/h2>\n<p>Part of the solution entails breaking down the current regulatory silos that treat privacy and human rights as separate issues. Relying on a consent-based data protection model flies in the face of the basic principle that privacy and equality are both human rights that cannot be contracted away.<\/p>\n<p>Even <a href=\"https:\/\/www.ic.gc.ca\/eic\/site\/062.nsf\/eng\/h_00108.html\">Canada\u2019s Digital Charter<\/a> \u2014 the federal government\u2019s latest attempt to respond to the shortcomings of the current state of the digital environment \u2014 maintains these conceptual distinctions. It treats hate and extremism, control and consent, and strong democracy as separate categories.<\/p>\n<p>To address algorithmic discrimination, we must recognize and frame both privacy and equality as human rights. And we must create an infrastructure that is equally attentive to and expert in both. Without such efforts, the glossy sheen of math and science will continue to camouflage AI\u2019s discriminatory biases, and travesties such as that inflicted on Williams can be expected to multiply.<!-- Below is The Conversation's page counter tag. Please DO NOT REMOVE. --><img loading=\"lazy\" decoding=\"async\" style=\"border: none !important; box-shadow: none !important; margin: 0 !important; max-height: 1px !important; max-width: 1px !important; min-height: 1px !important; min-width: 1px !important; opacity: 0 !important; outline: none !important; padding: 0 !important; text-shadow: none !important;\" src=\"https:\/\/counter.theconversation.com\/content\/143227\/count.gif?distributor=republish-lightbox-basic\" alt=\"The Conversation\" width=\"1\" height=\"1\" \/><!-- End of code. If you don't see any code above, please get new code from the Advanced tab after you click the republish button. The page counter does not collect any personal data. More info: https:\/\/theconversation.com\/republishing-guidelines --><\/p>\n<p><a href=\"https:\/\/theconversation.com\/profiles\/jane-bailey-1019384\">Jane Bailey<\/a>, Professor of Law and Co-Leader of The eQuality Project, <em><a href=\"https:\/\/theconversation.com\/institutions\/luniversite-dottawa-university-of-ottawa-1165\">L\u2019Universit\u00e9 d\u2019Ottawa\/University of Ottawa<\/a><\/em>; <a href=\"https:\/\/theconversation.com\/profiles\/jacquelyn-burkell-442395\">Jacquelyn Burkell<\/a>, Associate Vice-President, Research, <em><a href=\"https:\/\/theconversation.com\/institutions\/western-university-882\">Western University<\/a><\/em>, and <a href=\"https:\/\/theconversation.com\/profiles\/valerie-steeves-1138872\">Valerie Steeves<\/a>, Full Professor, <em><a href=\"https:\/\/theconversation.com\/institutions\/luniversite-dottawa-university-of-ottawa-1165\">L\u2019Universit\u00e9 d\u2019Ottawa\/University of Ottawa<\/a><\/em><\/p>\n<p>This article is republished from <a href=\"https:\/\/theconversation.com\">The Conversation<\/a> under a Creative Commons license. Read the <a href=\"https:\/\/theconversation.com\/ai-technologies-like-police-facial-recognition-discriminate-against-people-of-colour-143227\">original article<\/a>.<\/p>\n","protected":false},"excerpt":{"rendered":"<p>Detroit police wrongfully arrested Robert Julian-Borchak Williams in January 2020 for a shoplifting incident that had taken place two years &hellip;<\/p>\n","protected":false},"author":33,"featured_media":266374,"comment_status":"open","ping_status":"open","sticky":false,"template":"","format":"standard","meta":{"footnotes":""},"categories":[54365,5],"tags":[],"class_list":["post-266370","post","type-post","status-publish","format-standard","has-post-thumbnail","category-instagram","category-technology","mauthors-jane-bailey-luniversite-dottawa-university-of-ottawa","mauthors-jacquelyn-burkell-western-university","mauthors-valerie-steeves-luniversite-dottawa-university-of-ottawa","mauthors-the-conversation"],"_links":{"self":[{"href":"https:\/\/canadianinquirer.net\/v1\/wp-json\/wp\/v2\/posts\/266370","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/canadianinquirer.net\/v1\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/canadianinquirer.net\/v1\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/canadianinquirer.net\/v1\/wp-json\/wp\/v2\/users\/33"}],"replies":[{"embeddable":true,"href":"https:\/\/canadianinquirer.net\/v1\/wp-json\/wp\/v2\/comments?post=266370"}],"version-history":[{"count":2,"href":"https:\/\/canadianinquirer.net\/v1\/wp-json\/wp\/v2\/posts\/266370\/revisions"}],"predecessor-version":[{"id":266375,"href":"https:\/\/canadianinquirer.net\/v1\/wp-json\/wp\/v2\/posts\/266370\/revisions\/266375"}],"wp:featuredmedia":[{"embeddable":true,"href":"https:\/\/canadianinquirer.net\/v1\/wp-json\/wp\/v2\/media\/266374"}],"wp:attachment":[{"href":"https:\/\/canadianinquirer.net\/v1\/wp-json\/wp\/v2\/media?parent=266370"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/canadianinquirer.net\/v1\/wp-json\/wp\/v2\/categories?post=266370"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/canadianinquirer.net\/v1\/wp-json\/wp\/v2\/tags?post=266370"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}