Fake data, MAHA, AI, and Jobs

On actually doing your own research, and what kinds of jobs can support an AI future

Last week I tasked ChatGPT to write a full essay, supported by data and quotes.

It made up some wonderful reports that don’t exist, like a University of Tokyo research report on handwriting, and a Forrester report on enterprise content. I spent time checking the data, digging deep in the crevices of the internet to see if these reports were real and valid. I even asked ChatGPT to help me find the original source of the data it had given me. It couldn’t find a thing, but helpfully shared with me 12 other reports that do not contain the data point it originally wrote.

Here’s to being more accurate than MAHA.

I’m a just humble marketer fascinated by AI, and most likely no one is building national policy off of my writing. A Presidential report on children’s health, however, should be building evidence-based direction for national health for years to come.

Last week, the Presidential Make America Healthy Again Commission found itself in hot water when its signature report was riddled with “evidence” from research that did not exist.

I’m skeptical of data sources found on the internet, and consistently fact-checking myself and those who work with me to make sure a source is not just real, but reputable. MAHA doesn’t hold itself to the same bar, and it took a NOTUS, a news organization of young up-and-coming journalists, to uncover its falsehoods.

But I’m not (just) here to put MAHA on blast for not even meeting the bar I set for an entry level strategist. I’m also here to call out data on the internet in general. x`

The reliable data problem is bigger than AI.

In doing research for last week’s posts, I found this very compelling data point:

”Experts predict 90% of content online will be AI-generated by 2026.”

Now as far as data accuracy goes, this is small potatoes; it’s a prediction number which is already not evidence-based. There are ways to extrapolate on current data and forecast AI content growth with a number that’s rooted in real data. But easier for this unnamed expert to just throw out a gut-check number and have it surf the internet.

It was in all sorts of links—this one and this one and this one and in this video and this Reddit post debating it. They all cite a 2022 study from Europol study on deep fakes. Sounds reputable, right? The report looks very polished legit. However when you scour the report, there is no such prediction or data point.

But I’m not the first person to stumble on this number and into the rabbithole of determining data accuracy; Matt Klein, writer of futures at Zine and Reddit Head of Foresight, landed on the same head-scratcher a couple years ago and went far deeper than I did in understanding the source.

Which takes me to…

Doing the work.

Double-checking recanted data, mistakes, false data, much less AI slop, is a lot of work. You’d think this would be one of the easiest tasks for AI to take on—poring through documents, identifying discrepancies, chasing the dragon’s tail to find the source of the fire.

But it’s taking young journalists teaming up to find the holes in the MAHA report, or a Substack-side-gig writer to get to the bottom of a mystery data point that’s been haunting the internet.

At the same time, Hard Fork just reported on a looming jobpocoloypse, with less entry level roles and recent grads scrambling for work. So there seems to be a demand for data-literate detectives to navigate the AI future, and a supply of young people who grew up with ChatGPT as a college companion—and a sense of existential dread around what AI means for their job prospects.

The tension I’ll leave you with is, what are the viable jobs that can serve as data vigilantes, swooping in to save us from bad information? Is it in journalism, a profession that’s hanging on by a thread? More data-literacy influencers, unpacking information for follower tips? Teachers and trainers that educate everyone on discerning what’s true and what’s not? If it’s all AI checking AI’s homework, we need humans to check the AI.