Navigating AI Accountability: Trusting Our Machine Collaborators
Written on
Chapter 1: The Dawn of Cognitive Collaboration
About a decade ago, I had the opportunity to speak with pivotal members of the Watson team at IBM, shortly after their groundbreaking achievement of creating a machine capable of defeating top human contestants on the game show Jeopardy!. At that time, I articulated in Forbes that we were entering a transformative phase of cognitive collaboration among humans, computers, and one another.
This moment resonated with me, reminiscent of aviation pioneer Chuck Yeager's description of the shift to fly-by-wire technology four decades prior. In that era, pilots transitioned from direct control of aircraft to interfacing with computers that managed flight. Many seasoned aviators struggled to place their trust in these machines and adapt to this new paradigm.
Fast forward to the present with the emergence of ChatGPT, and Bill Gates has heralded the onset of the AI age. Much like those veteran pilots, we face challenges in adapting. Our success hinges not just on learning new competencies and adjusting our work methods, but also on how much we can trust our AI partners. To unlock AI's full potential, we must ensure it is held accountable.
Section 1.1: Understanding Data Bias
In human contexts, we strive to create safe and conducive learning environments. We carefully curate educational materials, select instructors, and bring together students to foster a beneficial mix of information and social dynamics. This meticulous approach stems from our awareness that the learning environment significantly impacts the overall experience.
Similarly, machines operate within a learning framework known as a "corpus." For instance, to train an algorithm to identify cats, it is exposed to thousands of cat images. Over time, it learns to distinguish between a cat and a dog. Just like humans, algorithms derive value from these learning experiences.
However, this process can lead to grave missteps. A notorious example is Microsoft's Tay, a Twitter bot launched in 2016. Within just a day, Tay transformed from a friendly entity (“humans are super cool”) to a disturbing presence (“Hitler was right and I hate Jews”), highlighting the potential dangers of bias.
Bias in the training data is more prevalent than we often recognize. A search for "professional haircut" yields predominantly images of white men, while "unprofessional haircut" reflects greater racial and gender diversity. This disparity occurs because content creators often depict different genders and ethnicities in contrasting manners. When we query machines, our inherent biases are frequently mirrored back.
Subsection 1.1.1: The Challenge of Algorithmic Bias
A second significant source of bias arises from the design of decision-making models. Take the case of Sarah Wysocki, a fifth-grade teacher who, despite accolades from parents, students, and administrators, was dismissed by the D.C. school district due to an algorithm's unfavorable assessment of her performance. The reasoning remained unclear, as the system's complexity obscured understanding.
It's easy to speculate on how such situations arise. If a teacher's worth is gauged primarily through test scores, other vital aspects, like addressing students with learning differences or emotional challenges, may go unrecognized or even unjustly penalized. While adept human managers identify exceptional cases, algorithms are typically not programmed to do so.
In other instances, models are built based on readily available data, or they become overly tailored to specific scenarios and are then broadly applied. For example, in 2013, Google Flu Trends overestimated flu cases nearly twofold. The algorithm was likely influenced by heightened media coverage, which led to increased searches from people not actually experiencing illness. The model wasn’t equipped to account for its own influence.
Ultimately, algorithms must be crafted in a certain manner. It’s impossible to cover every eventuality, meaning choices must be made, and biases will inevitably seep in. Errors will occur. The focus should not be on eradicating mistakes but on ensuring accountability through explainability, auditability, and transparency.
Chapter 2: Cultivating Trust in AI
In 2020, Ofqual, the organization overseeing A-Level college entrance exams in the UK, found itself embroiled in controversy. Due to Covid-19, they were unable to conduct live examinations and resorted to an algorithm that calculated scores partially based on the historical performance of the students' schools. This resulted in disadvantaged students facing even further penalties through artificially lowered scores.
The backlash was swift, but in many ways, the Ofqual incident had a positive outcome. The agency's transparency about the algorithm's construction quickly illuminated the source of bias, allowing for prompt corrective measures and mitigating much of the damage. As Linus's Law suggests, “given enough eyeballs, all bugs are shallow.”
The dawn of artificial intelligence demands our collaboration with machines, harnessing their capabilities to enhance human service. However, for this partnership to flourish, it must occur within a framework of trust. Machines, like humans, must be held accountable; their decisions cannot remain a “black box.” We need insight into their reasoning and the processes guiding their choices.
Senator Schumer is reportedly pursuing legislation aimed at enhancing transparency, but that is merely a starting point. The true transformation must come from within ourselves and our perceptions of our relationships with the machines we create. As Marshall McLuhan noted, media are extensions of humanity, and the same holds true for technology. Our machines carry our human shortcomings and vulnerabilities, and we must acknowledge this reality.
Greg Satell is the Co-founder of ChangeOS, a transformation and change advisory firm, an international keynote speaker, and the bestselling author of Cascades: How to Create a Movement that Drives Transformational Change. His previous book, Mapping Innovation, was recognized as one of the best business books of 2017. More information about Greg can be found on his website, GregSatell.com, and you can follow him on Twitter @DigitalTonto.
The first video, The A.I. Dilemma - March 9, 2023, delves into the ethical challenges we face in this new era of AI, highlighting the need for accountability and trust in our interactions with technology.
The second video, Promise and Perils of Artificial Intelligence, explores both the benefits and risks associated with AI, emphasizing the importance of transparency and ethical considerations in its development and use.