Weapons of Math Destruction is one of those books that makes me want to buy a case and send a copy to every person I know. Upton Sinclair once wrote, “It is difficult to get a man to understand something, when his salary depends on his not understanding it.” That is true, but there are exceptional people like mathematician Cathy O’Neil who when they come to understand the pernicious effects of what they do, quit their jobs and dedicate themselves to raising the alarm and working to counter the damage.
Weapons of Math Destruction are defined by O’Neil as those algorithms and analytics that are used to make many of the daily decisions that affect people’s lives. She takes care to clarify #NotAllAlgorithms. What makes an algorithm a WMD is when the factors that comprise the algorithm are opaque, unknown to the people affected by it and often even by the people using it. It operates on a large scale, affecting large swaths of people. Worse, it is unaccountable. It does not gather data to see if it makes the right decisions, it does not self-correct. It just runs and lives are changed and no one is the wise to its operations. And most of the time, these algorithms make the poor poorer, the rich richer, and make life harder for those whose lives are already hard.
These algorithms are with us everywhere we turn. They evaluate our teachers, determine prison sentences and parole releases, determine whether we are hired, promoted, fired, insured, surveilled, and how much we pay for things. E-scores even determine how long you wait for customer service, a good score guiding you to a human and a poor one shunting you off to call center hell. Data that shows you don’t shop around results in insurance companies charging you hundreds of dollars more, never offering the discounts they will offer to people whose scores say they comparison-shop. Even in politics, your cookies reveal data that has a politician’s front page show you different pictures and issues than they show your neighbor.
When governments make it illegal for companies to use data like credit scores or race, they turn to other options like e-scores, an unregulated mass of data collection used to create profiles of all of us, sold and used without our knowledge. These e-scores could include false data from other people with similar names and yet result in higher car insurance, worse commercial offers from retailers, higher prices on cars, and a raft of other things.
I loved Weapons of Math Destruction. It is even-handed, pointing out that many of these processes began with good intentions. O’Neil shows that quantitative analysis can be used for good and gives examples. She is not anti-math nor anti-analytics. She wants them to be more transparent and be open to correction. People need to know when algorithms are used against them. We know about credit scores, but there’s so many others, hidden behind “Intellectual Property” rights that allow companies to hide what factors influence our scores. Often these factors have nothing to do with us individually, but with people who seem, on the surface, like us.
Read this book! It is accessible, explaining in easy-to-understand terms exactly what is happening and how it affects us. Accessibility does not sacrifice rigor and a full 30% of the book is devoted to citations proving the case. It is urgent, because every day algorithms encroach on our lives even more. It is fascinating, full of shocking and surprising stories of people whose lives are changed by this, that, or the other score. It has useful tips, such as clearing your cookies before shopping so you are not logged in will result in more discount offers. Most of all, read it because there is really are secret forces, operating under the surface, affecting our lives from birth to death, at work, at home, at play. They will only gain power so long as we are unaware of them.
I received a review copy of Weapons of Math Destruction from Blogging For Books