IEEE Spectrum‘s most popular AI stories of the last year show a clear theme. In 2024, the world struggled to come to terms with generative AI’s capabilities and flaws—both of which are significant. Two of the year’s most read AI articles dealt with chatbots’ coding abilities, while another looked at the best way to prompt chatbots and image generators (and found that humans are dispensable). In the “flaws” column, one in-depth investigation found that the image generator Midjourney has a bad habit of spitting out images that are nearly identical to trademarked characters and scenes from copyrighted movies, while another investigation looked at how bad actors can use the image generator Stable Diffusion version 1.5 to make child sexual abuse material.
Two of my favorites from this best-of collection are feature articles that tell remarkable stories. In one, an AI researcher narrates how he helped gig workers gather and organize data in order to audit their employer. In another, a sociologist who embedded himself in a buzzy startup for 19 months describes how engineers cut corners to meet venture capitalists’ expectations. Both of these important stories bring readers inside the hype bubble for a real view of how AI-powered companies leverage human labor. In 2025, IEEE Spectrum promises to keep giving you the ground truth.
David Plunkert
Even as the generative AI boom brought fears that chatbots and image generators would take away jobs, some hoped that it would create entirely new jobs—like prompt engineering, which is the careful construction of prompts to get a generative AI tool to create exactly the desired output. Well, this article put a damper on that hope. Spectrum editor Dina Genkina reported on new research showing that AI models do a better job of constructing prompts than human engineers.
Gary Marcus and Reid Southen via Midjourney
The New York Times and other newspapers have already sued AI companies for text plagiarism, arguing that chatbots are lifting their copyrighted stories verbatim. In this important investigation, Gary Marcus and Reid Southen showed clear examples of visual plagiarism, using Midjourney to produce images that looked almost exactly like screenshots from major movies, as well as trademarked characters such as Darth Vader, Homer Simpson, and Sonic the Hedgehog. It’s worth taking a look at the full article just to see the imagery.
The authors write: “These results provide powerful evidence that Midjourney has trained on copyrighted materials, and establish that at least some generative AI systems may produce plagiaristic outputs, even when not directly asked to do so, potentially exposing users to copyright infringement claims.”
Getty Images
When OpenAI’s ChatGPT first came out in late 2022, people were amazed by its capacity to write code. But some researchers who wanted an objective measure of its ability evaluated its code in terms of functionality, complexity and security. They tested GPT-3.5 (a version of the large language model that powers ChatGPT) on 728 coding problems from the LeetCode testing platform in five programming languages. They found that it was pretty good on coding problems that had been on LeetCode before 2021, presumably because it had seen those problems in its training data. With more recent problems, its performance fell off dramatically: Its score on functional code for easy coding problems dropped from 89 percent to 52 percent, and for hard problems it dropped from 40 percent to 0.66 percent.
It’s worth noting, though, that the OpenAI models GPT-4 and GPT-4o are superior to the older model GPT-3.5. And while general-purpose generative AI platforms continue to improve at coding, 2024 also saw the proliferation of increasingly capable AI tools that are tailored for coding.
Alamy
That third story on our list perfectly sets up the fourth, which takes a good look at how professors are altering their approaches to teaching coding, given the aforementioned proliferation of coding assistants. Introductory computer science courses are focusing less on coding syntax and more on testing and debugging, so students are better equipped to catch mistakes made by their AI assistants. Another new emphasis is problem decomposition, says one professor: “This is a skill to know early on because you need to break a large problem into smaller pieces that an LLM can solve.” Overall, instructors say that their students’ use of AI tools is freeing them up to teach higher-level thinking that used to be reserved for advanced classes.
Mike McQuade
This feature story was authored by an AI researcher, Dana Calacci, who banded together with gig workers at Shipt, the shopping and delivery platform owned by Target. The workers knew that Shipt had changed its payment algorithm in some mysterious way, and many had seen their pay drop, but they couldn’t get answers from the company—so they started collecting data themselves. When they joined forces with Calacci, he worked with them to build a textbot so workers could easily send screenshots of their pay receipts. The tool also analyzed the data, and told each worker whether they were getting paid more or less under the new algorithm. It found that 40 percent of workers had gotten an unannounced pay cut, and the workers used the findings to gain media attention as they organized strikes, boycotts, and protests.
Calacci writes: “Companies whose business models rely on gig workers have an interest in keeping their algorithms opaque. This “information asymmetry” helps companies better control their workforces—they set the terms without divulging details, and workers’ only choice is whether or not to accept those terms…. There’s no technical reason why these algorithms need to be black boxes; the real reason is to maintain the power structure.”
IEEE Spectrum
Like a couple of Russian nesting dolls, here we have a list within a list. Every year Stanford puts out its massive AI Index, which has hundreds of charts to track trends within AI; chapters include technical performance, responsible AI, economy, education, and more. This year’s index. And for the past four years, Spectrum has read the whole thing and pulled out those charts that seem most indicative of the current state of AI. In 2024, we highlighted investment in generative AI, the cost and environmental footprint of training foundation models, corporate reports of AI helping the bottom line, and public wariness of AI.
iStock
Neural networks have been the dominant architecture in AI since 2012, when a system called AlexNet combined GPU power with a many-layered neural network to get never-before-seen performance on an image-recognition task. But they have their downsides, including their lack of transparency: They can provide an answer that is often correct, but can’t show their work. This article describes a fundamentally new way to make neural networks that are more interpretable than traditional systems and also seem to be more accurate. When the designers tested their new model on physics questions and differential equations, they were able to visually map out how the model got its (often correct) answers.
Edd Gent
The next story brings us to the tech hub of Bengaluru, India, which has grown faster in population than in infrastructure—leaving it with some of the most congested streets in the world. Now, a former chip engineer has been given the daunting task of taming the traffic. He has turned to AI for help, using a tool that models congestion, predicts traffic jams, identifies events that draw big crowds, and enables police officers to log incidents. For next steps, the traffic czar plans to integrate data from security cameras throughout the city, which would allow for automated vehicle counting and classification, as well as data from food delivery and ride sharing companies.
Mike Kemp/Getty Images
In another important investigation exclusive to Spectrum, AI policy researchers David Evan Harris and Dave Willner explained how some AI image generators are capable of making child sexual abuse material (CSAM), even though it’s against the stated terms of use. They focused particularly on the open-source model Stable Diffusion version 1.5, and on the platforms Hugging Face and Civitai that host the model and make it available for free download (in the case of Hugging Face, it was downloaded millions of times per month). They were building on prior research that has shown that many image generators were trained on a data set that included hundreds of pieces of CSAM. Harris and Willner contacted companies to ask for responses to these allegations and, perhaps in response to their inquiries, Stable Diffusion 1.5 promptly disappeared from Hugging Face. The authors argue that it’s time for AI companies and hosting platforms to take seriously their potential liability.
The Voorhes
What happens when a sociologist embeds himself in a San Francisco startup that has just received an initial venture capital investment of $4.5 million and quickly shot up through the ranks to become one of Silicon Valley’s “unicorns” with a valuation of more than $1 billion? Answer: You get a deeply engaging book called Behind the Startup: How Venture Capital Shapes Work, Innovation, and Inequality, from which Spectrumexcerpted a chapter. The sociologist author, Benjamin Shestakofsky, describes how the company that he calls AllDone (not its real name) prioritized growth at all costs to meet investor expectations, leading engineers to focus on recruiting both staff and users rather than doing much actual engineering.
Although the company’s whole value proposition was that it would automatically match people who needed local services with local service providers, it ended up outsourcing the matching process to a Filipino workforce that manually made matches. “The Filipino contractors effectively functioned as artificial artificial intelligence,” Shestakofsky writes, “simulating the output of software algorithms that had yet to be completed.”
From Your Site Articles
Related Articles Around the Web