Notre Dame Science 2019 - ACMS and Mathematics Excellence

Data science skeptic and mathematician warns about algorithmic bias

Mathematics conference draws scholars of geometric representation theory

Data science skeptic and mathematician warns about algorithmic bias

Notre Dame’s Master of Science in Data Science program hosts author and data expert Cathy O’Neil

By Deanna Csomo McCool

Organizations are increasingly counting on mathematical models to make hiring, admissions, lending, and other decisions. But according to mathematician Cathy O’Neil, over-reliance on big data and algorithms has become a destructive force in society, further causing divisions between the “haves” and the “have-nots.”

Cathy O'Neil

Moral and ethical decisions are embedded into algorithms that benefit organizations and often not the people they are claiming to assist, described O’Neil, the New York Times best-selling author of “Weapons of Math Destruction: How big data increases inequality and threatens democracy,” as she spoke to an audience at The University of Notre Dame in September.

“If you think about a life, starting with going to high school or college, or getting a job, or a mortgage, or insurance….during every moment you are funneled by algorithms as a winner or loser,” she said. “It’s not just destructive for the individual, but it’s also destructive for society.”

O’Neil earned her doctorate in mathematics from Harvard University, taught at Barnard college, worked in the private sector, and spearheaded the Lede Program at Columbia University School of Journalism, which is designed to equip journalists and storytellers with the computational skills needed to turn data into narrative. Sponsored by the Master of Science in Data Science program and Mendoza College of Business, O’Neil’s talk about ethics and data science fit perfectly with the data science program’s goals of training technically proficient and ethically focused graduates, according to Roger Woodard, director of the data science program.

Although a data scientist herself, O’Neil’s skepticism about how mathematical models are used is outlined in her book. By tracing subjects’ lives and describing how algorithms affected them, she melds social justice issues with what many think are the black-and-white answers of mathematical algorithms. However, “these are done in secret, like a scoring system,” she said. “The score gives you a bonus for certain types of information. If your score passes a certain threshold, you are given that opportunity. If you are not high enough, you aren’t.”

Cathy O'Neil Audience

During her talk she shared several examples. She described how algorithms more often allowed the release from prison of white inmates over black inmates, how certain financial programs for students were less beneficial for university rankings (making some universities hesitant to use them), and how skewed child abuse data more frequently caused children in lower-income homes be removed from their families than children higher-income homes.

Cathy O'Neil

“The algorithms destroy exactly what they were trying to improve,” O’Neil told the audience. “They undermine science in a very important way—and this is a claim I’m making that you can challenge me on—but my problem is it’s called ‘data science,’ but there’s very little science. Data scientists are asking us just to trust them, but science is based on the idea of testing, or evidence. And algorithms, because of their secrecy, are by definition not tested.”

Woodard agreed with the premise of O’Neil’s talk, noting that algorithms can have hidden biases from a variety of sources based on how data was collected and which potential audiences are left out. Notre Dame’s online Master of Science in Data Science program is designed to provide its graduates with the ethical training O’Neil feels is lacking, he noted. “How the data scientist prepares the data for the algorithms can also introduce bias, and we train our students to recognize these issues and consider the impact of their actions,” he said.

In addition to her talk, O’Neil recorded video examples that will be used in the data science program’s Data Ethics and Policy course, Woodard said. She also met with faculty across campus, including The Law School at Notre Dame and the College of Arts and Letters.

“Examples of Weapons of Math Destruction are everywhere,” O’Neil warned the audience. “Who gets into high school, who gets into college; who gets this job, who keeps this job. If you think about a bureaucracy and a difficult decision for a bureaucracy to make, it’s probably being replaced by an algorithm right now.”

Mathematics conference draws scholars of geometric representation theory

By Deanna Csomo McCool

Professors, graduate students, and undergraduates shared their knowledge about geometric representation theory for three weeks in June during the 8th consecutive Center for Mathematics Thematic Program.

Samuel Evens

The program, organized this year by Samuel Evens, professor in the Department of Mathematics, Thomas Nevins, ’93, now a professor of mathematics at the University of Illinois, and Kevin McGerty, associate professor of mathematics at Oxford University, began with a week-long undergraduate portion. That was followed by another week for graduate students, and capped off with a week set aside for active researchers to discuss their current work.

The yearly event showcases the different areas of mathematics research at Notre Dame, and gives mathematicians at all stages of study the chance to collaborate and learn new ideas. Most papers are now written in collaboration, and the thematic programs prepare students for that process by including time for students to work together on problems, Evens said.

“The pure math community is very small; in the U.S. it’s probably counted in thousands rather than tens of thousands,” Nevins said. “This conference is a good way to find out what others are researching, and is a great opportunity to meet young people and share with them what the more senior people are doing.”

Geometric representation theory is one of the major tools for understanding the role of symmetry in mathematics, physics, and nature. The thematic event covered all topics and current research. The field is related to the research done by Martha Precup, who earned her doctorate in mathematics at Notre Dame, just completed her postdoctoral work at Northwestern University, and will join the mathematics faculty at Washington University in St. Louis in August.

The conference was an ideal way for undergraduates and graduate students to learn a broad overview of the topic, Precup said. “Young people hear the experts give an idea, and provide a roadmap of where (the undergraduate or graduate students) may be going in mathematics. It’s a very valuable experience.”

The conference is a painless way for graduate students to enter a slightly different area than they have been studying, according to fifth-year Notre Dame graduate student Kostya Timchenko. And undergraduate students “get a chance learn and see some beautiful tricks in math,” he said.

Twenty-two undergraduates participated in the first week, 50 graduate students the second, about 50 researchers participated during the final week. Notes were produced from the lectures and problem sessions, and Evens plans to use the them in reading or advanced courses during the academic year.

“Our hope is that this will encourage students to work collaboratively in graduate school and beyond,” Evens said. “A postdoc told me that she felt like she had spent more time talking to her graduate school adviser at the conference than during her time in graduate school, and this gave her the opportunity to test out many of her current ideas.”

The summer thematic programs are funded in part by the National Science Foundation, and last year the program met a milestone with its 1,000th participant. In 2019, the graduate segment of the program will be extended to two weeks in order to further foster discussion, group work and presentations among young mathematicians.