How external researchers struggle to understand the ‘black box’ of Facebook

This story is from The Pulse, a weekly health and science podcast.

Find it on Apple PodcastsSpotify, or wherever you get your podcasts.

In 2021, computer scientist Laura Edelson got banned from Facebook.

She said that became a problem in her personal life.  Edelson lives in a small town where  people use Facebook to find out about school delays, lost pets, and town meetings.

Edelson’s  research colleague, Damon McCoy, also got kicked off.

“It’s kind of annoying,” Edelson said. “I used to say it was us and Trump but now — it’s just us.”

All this happened because of their research on Facebook, which the company says used data gathering methods that broke their terms of service.

Facebook remains one of the biggest social media networks in the U.S., and around the world.

Over the years, researchers, parents, and politicians have had many questions about the effects of Facebook: Is misinformation really more engaging than other posts? Are particular groups of people more likely to see harmful ads, like scams? How effective are the platforms at driving users to accurate information? Facebook, like other social media companies, control all their data, so researchers either work with Facebook to access what data they can, or struggle on the outside.

Edelson and McCoy got interested in Facebook after the 2016 election, when the platform was under a lot of scrutiny over whether the company had mishandled user data, and swayed the outcome of the U.S. presidential election. CEO Mark Zuckerberg testified before Congress in 2018, where he said, “We didn’t take a broad enough view of our responsibility, and that was a big mistake.”

After that, Facebook released a continually updated archive of political ads in the U.S. so people can see how much each advertiser spent on their ads, and whom the ads reached.

Edelson and McCoy, who were both at New York University at the time, wanted to use this data to understand Facebook’s powerful recommendation engine, and the effects the ads can have on society.

They quickly published some work in 2018 showing that then President Donald Trump was the biggest political advertiser on Facebook. Facebook welcomed their work, telling The New York Times this is exactly how they hoped people would use their tool.

But Edelson says they quickly realized there were some important details missing from the archive, like the ad targeting data that shows the specific ways that advertisers had targeted ads at specific audiences. This data is valuable because they wanted to study who the advertisers wanted to reach, and who they were actually reaching.

“If you want to understand patterns across the entire ad ecosystem, you need information about the entire ad ecosystem,” Edelson said.

They got around this by working with investigative news outlet ProPublica to make a research tool that allowed them to collect information that Facebook provides to users. Facebook users can click on an ad and see the targeting criteria the company used to show them the ad.

Edelson and McCoy created a tool that people could download and voluntarily send the researchers information about the ads they were seeing.

Facebook ordered them to stop in 2020, and cut off their access completely in 2021. The company said in a press release that they did it because Laura and Damon were collecting data about Facebook users in a way that broke the company’s terms of service.

Edelson is now at Northeastern University.  She and McCoy are still doing their research, but now they need to rely on a research partner to get them the data.

A spokesperson for Meta, the parent company of Facebook, Instagram, and WhatsApp, said he could not tell me more details about this specific case. He also pointed out that the company has a track record of working with outside researchers on a variety of topics.

But even if researchers work with Meta, getting access to data can still be a challenge.

“Researcher access to Meta systems has really been on a downward trajectory over the past decade,” said Deen Freelon, professor of communications at the University of Pennsylvania. He is part of a research collaboration with Meta called Social Science One.

He says it would be unreasonable to expect complete access. “If you have a fundamental problem with that, then you need to get out of social media research because there’s no way around that.”

However, he added that researchers can still do valuable work “under the assumption that the process that produces the data is a black box, but that the output of that black box can be evaluated productively and usefully.”

  • January 29, 2024
  • Articles,
  • This post was written by