Book of the Week: Weapons of Math Destruction

20 Oct 2016

This week I read Weapons of Math Destruction by Cathy O’Neil, mathbabe, because she accuses me of building WMDs. O’Neil is a data scientist with a PhD in mathematics from Harvard, who was previously a professor and a quant for D.E. Shaw during the Financial Crisis of 2008. I have a few issues with the book other than it saying that I have fallen to the dark side of big data. The book goes over how models affect a person’s life through their lifetime starting with school, college, courts, work and voting. O’Neil’s main point is that correlation does not imply causation and unfortunate individuals are lumped into groups by association, which damage their prospects for life. For me, there are probably bigger underlying issues that need to be addressed. The models are merely a reflection of reality. Businesses treat people like cogs, because they are cogs. Weapons of Math Destruction

The privileged, we’ll see time and again, are processed more by people, the masses by machines.

Let’s say you have a sample of people and you’re trying to predict whether or not a person is a rapist. In your dataset is Brock Turner. Additionally, your language models have “Stanford rapist” as a high occurring bigram since there are tons of news article. The correlation between Stanford and rapist is high. Now you ask your trained algorithm, whether or not a new person, Alice, is a rapist or not. Your algorithm has never seen Alice, but based on her information, it tells you that Alice is a rapist. Alice is shocked. Now being a wealthy person going to Stanford, Alice gets her father’s lawyers to subpoena and do discovery on the algorithm. After a few months the data scientist comes back and says, she was labelled as a rapist, because she went to Stanford and Brock Turner went to Stanford. Because Alice was privileged, she was able to correct this oversight, the masses aren’t so lucky. There are three elements to a Weapon of Math Destruction (WMD): opacity, scale and damage. WMDs are opaque, because you don’t know what the inputs are and how those inputs are used in computing a score. You don’t know what you did to deserve such a bad score and you have no way of rectifying the situation even though it may be a case of bad data. WMDs are usually adopted at scale to deal with many people who are difficult to sort of individually as a cost saving measure. It becomes a competitive advantage. If things don’t go way according to the model, you can suffer damages, like higher interest rates, rejection from jobs, increased incarceration and denied education. The people who construct models created flawed models, because of their own biases. Models that allocate police use violent crime and nusiance crime, but if you look at the cost of crime, I’d say there is probably no bigger criminal than bankers. How much of our police force is dedicated to that? How many bankers went to jail for cheating people out of money. People have lost more money to bankers than they have to petty theft. Yet we don’t arrest bankers, because they are the ones who pay the bills. Where is the justice? That said I am thankful to the police who safeguard my assets. O’Neil says people get screwed, because of how big data puts you in a group of people like you and that group may be trash. I think this is antiquated. Algorithms I work on, deal with the individual person. Things are personalized to a specific person, not a group. Sometimes when I try to explain it to people, they don’t believe it. US News and Education The college rankings from US News are pretty pervasive, but they don’t mean a thing. They were something thought up to drive magazine sales. There is not scientific rigor involved and colleges routinely try to game the rankings. In 2014, the Saudi’s made King Adulaziz University’s math department rank just behind Harvard by paying highly-cited professors $72,000 to work for 3 weeks and change their affiliation on Thomson Reuters. The university was only in existence for 2 years. Instead of focusing on providing a quality affordable education, colleges focus on increasing their US news rank. Even donating $1 as an alumni helps, because percentage of alumni donations is used to ranking. You would think the cost of tuition would be an important factor, but it is left out. Another sad thing is that teachers were graded on how well your students improved, they called this valued-added. If you have a class filled with special education and smart students, they don’t really improve. The special education students will continue to score low and the smart students will continue to score high. What you want are some students in the middle, which you can improve. You can also be screwed by their teacher by the year before. Let’s say their previous teacher “corrected” their test answers, so it looks like the students improved. Now when you get them and when they test, they scored lowered than before, because you neglected to “correct” their answers. You get fired and the previous teacher gets a bonus. Doesn’t this sound like Wall Street banking? Even if there wasn’t any cheating, the student class sizes are so small, the results are not statistically meaningful due to sampling error. Predatory Behavior

They rake in lead generation fees by providing a superfluous service to people, many of whom are soon targeted for services they can ill afford.

There is a dirty industry for targeting specific people with misleading ads to generate leads. For-profit colleges will pay up to $150 for good leads. You can sell good car insurance leads for $20. Each sucker has a price on their head. With Facebook (FB), you can target the exact demographics that are likely to be vulnerable to predatory payday loans. When you use Facebook, you shouldn’t forget that you are the product that they are selling. Allstate insurance had a model for how likely you were to shop for lower prices. If you weren’t likely to shop around, they charged you more. It was part of their price optimization strategy. Low Wage Workers After reading about how low wage workers need to deal with last minute changes to scheduling due to optimization algorithms makes me think that Uber is great for the world. When there is uncertainity, it is difficult for you to plan for the future. With Uber, you can work when you want to. Well, sort of. Unless there is a surge going. Then you’ll be out there driving. Simpson’s Paradox A Nation at Risk has statistical errors that affect public policy by saying America’s schools were failing. It took researchers at Sandia National Laboratories to identify that the conclusion was false due to the Simpson’s Paradox. Each of the individual groups were improving, but there were more poor students and minorities taking the SATs. Blackboxes Systems will only become more opaque as AI becomes more prevalent. But are humans any better. In 2015, 43% of Republicans thought Obama was Muslim. If I had a choice, I might welcome our robot overlords.