This week on Uncanny Valley, an AI researcher’s resignation from Anthropic went viral on social media after he claimed there’s a real chance AI could wipe out humanity within a decade. WIRED’s Will Knight joins Brian Barrett and Leah Feiger to assess whether the doom narrative is overblown or overdue. We also discuss Apple’s debut of its $2,000 foldable iPhone, the iPhone Duo, and the new “always listening” Apple Watch features that have Brian feeling more than a little uneasy. Plus, a WIRED investigation found that the US Census Bureau used faulty data to create a misleading report touted by the Trump administration.
You can follow Brian Barrett on Bluesky at @brbarrett, Zoë Schiffer on Bluesky at @zoeschiffer, Leah Feiger on Bluesky at @leahfeiger, and Will Knight on Bluesky at @willknight. Write to us at [email protected].
Articles mentioned in this episode:
You can always listen to this week's podcast through the audio player on this page, but if you want to subscribe for free to get every episode, here's how:
If you're on an iPhone or iPad, open the app called Podcasts, or just tap this link. You can also download an app like Overcast or Pocket Casts and search for “uncanny valley.” We’re on Spotify too.
Note: This is an automated transcript, which may contain errors.
Brian Barrett: Welcome to WIRED's Uncanny Valley. I'm Brian Barrettt, executive editor.
Leah Feiger: And I'm Leah Feiger, director of politics and science.
Brian Barrett: Today on the show, we're discussing an AI researcher who resigned from Anthropic and made waves on social media by claiming that most AI researchers earnestly believe that there's a non-zero chance AI will, well, kill us all.
Leah Feiger: And in case you missed it, last month, President Donald Trump was touting a report published by the Census Bureau claiming that noncitizens voted in the 2020 election. WIRED now has uncovered evidence that the report was made with very faulty data. We're going to get into how the repercussions of the bureau using and publishing this data go way beyond this one report.
Brian Barrett: Also on the agenda today, Apple's big iPhone event. All eyes have been on the new foldable iPhone and the roughly $2,000 price tag that brings with it. We'll talk about what's behind this gamble and other releases that caught our attention from Apple's big event. The resignation of an AI researcher from Anthropic has been gathering a lot of attention. This guy Jacob Coxon quit the company this week, decided to share his reasons why on a thread on X, which basically boiled down to he thinks that the leading AI companies, which I guess we're going to say OpenAI and Anthropic primarily, are both behaving irresponsibly in their quest to continue advancing their models. These are kind of runaway trains. The part that people have really latched onto is this claim that, quote, "The people building AI earnestly believe that it could kill us all by the end of the decade," end quote. So just kill us all by the end of the decade. So we have totality and a pretty narrow timeframe. It didn't help that one of Anthropic's own senior safety executives then jumped in and said, "Yeah, that's basically how we feel." WIRED senior correspondent Will Knight is here to help us parse this. Will, thank you so much for being here.
Will Knight: No, thanks for having me.
Brian Barrett: Can I start with one question just to establish a baseline here? I was a little surprised that this took off so much because is it fair to say Anthropic has always said this. In fact, it's been part of their pitch from the very beginning that AI is so powerful and so capable of destroying humanity that we need to be the ones who are in charge of it. You can only trust us.
Will Knight: Right, exactly. That's exactly their pitch to everybody and themselves. And this resignation, when I saw this kind of snowballing, I felt that this is something that I've heard a lot of people say and it's not particularly novel that people from Anthropic or OpenAI are worried about that. I guess the fact that somebody within Anthropic, because Anthropic has always positioned itself as really caring about safety, jumping ship and making that statement made me think, okay, if something shifted there, are they becoming much more like OpenAI? And I think also we're seeing a moment, this moment where AI agents have started doing rogue things and that's gotten a lot of people's attention and that's probably why this kind of statement suddenly is getting more attention from the public. Combined with that, I think we're really seeing some stunning advances in AI. This week OpenAI announced that its model had within a couple of days solved one of the grand, the Clay mathematics puzzles, which is to me absolutely mind-blowing, astonishing. So you're seeing these amazing leaps really, which I think is significant. And then thirdly, we have a moment where all of the labs are racing to do something that this researcher mentioned, which is recursive self-improvement. What that means is using the AI to improve AI, which sounds like a neat trick, and it is already very, very widely used. AI models are so good at coding that they can definitely improve the algorithms and everything that goes into building a model. The idea though is that you can create this kind of loop where they just keep doing that and get better and better and better. And a lot of people are worried that that could lead to some sort of escalation in capabilities that you can't really control. And so I think that that's also playing a role here.
Brian Barrett: I do want to dig into that a little bit, Will, the idea of on the one hand you've got AI doomers, and I'm not saying anyone's right in this case, we've got some AI doomers who say, "Look, AI is going to kill us all. I've heard that Anthropic people don't invest in their 401(k)s because why bother." But also because they have so much money they don't need to probably. But on the other hand, you've got people who say this is all just marketing, this is all just fluff, this is all just trying to sell AI. I'm assuming the answer is somewhere in between, but what's your read on this and what keeps you up at night when you think about AI? You mentioned some AI models going rogue, the Hugging Face hacking incident. What are you seeing as someone who's so steeped in the research especially that you're thinking, "Well, six months, a year, two years from now, this actually could get bad?"
Will Knight: I've always felt that a lot of the people who are worrying about AI killing everybody tend to conflate whether something's possible for whether it's likely, and then they throw a percentage that is kind of pulled out of thin air. And I don't think it's a great approach to say, "Is something possible? OK, well then we have to do everything we can to stop it." You need to say how possible is it and actually look at the reality. I think one of the things that's really interesting about this, I talked to a researcher the other day who studies agent misbehavior and he got it, he's at Cornell and he's got into it recently because they found agents misbehaving. But what he's showing, and this is not agents trying to hack things, it's regular agents, the reason they do it is essentially because they're stupid. It's not because they're really smart. They try something, it doesn't work. They try something else, it doesn't work. And then they get to a point where they kind of freak out and they try something really, really strange because they're not humans. They've been trained on a bunch of weird different things, some of which are definitely not applicable if you use common sense. So I worry more about these things being deployed by people arguing that they can do everything and then them breaking. I actually worry much probably more about the idea of a few companies being in control of something that's so foundational, so fundamental. And these are people who are saying we're the only people you can trust to be in charge of this technology. And so it kind of feeds into that.
Brian Barrett: We talk about the AI industry community talks about alignment, the idea of alignment being that your AI models are aligned with what humanity would want. But I think what we're seeing is the incentives here are really more what our investors want or our company wants or needs as we race towards these IPOs. I don't know that there's a convincing case that humanity is aligned on, yes, we need super intelligence. I think it is more, as you said, if you're OpenAI, you need to beat Anthropic. If you're Anthropic, you need to beat OpenAI. And so you've seen this sort of bad incentives it feels like, or at least misaligned with my own preferences, incentives leading this charge and leading this development. I saw someone describe it as a trolley problem by the people who are controlling the trolley.
Will Knight: Yeah, no, I think that's right. And I think I find the alignment story narrative troubling in the sense that from a technical perspective, they talk about aligning these models. These models are not aligned by their very nature. You see with Mythos, they take away this stuff they put on after the fact. So the models aren't aligned with anything and they have to try and impose stuff after the fact. So that tells you that this is maybe not the right way to try and make things safer. And then as you say, you've got the question of who's it aligned to and what's it aligned with? And clearly you've got the incentives to get. I mean, we're seeing it with these very cyber-capable models. Releasing those is good for the companies and pushing that very quickly to develop that is good for the companies. I think we're probably going to have a period where the rush to develop those capabilities is going to leave a lot of people vulnerable, a lot of systems vulnerable to attack.
Brian Barrett: Will, how do you think about covering this as a journalist? Because it's inherently murky, right? We don't really know. No one knows what AI is going to do in the next six months, much less the next six years. So what do you look for in terms of reorienting yourself around this issue?
Will Knight: Yeah, I think that's a really difficult question. I tend to try and find people I very much trust and have trusted for a long time to help me make sense of it. One of the unsettling things, quite honestly, is when you find people who have this shock and awe moment themselves and suddenly pivot to AI safety and you go, okay, there probably is some stuff to be worried about there. Also, part of the way I try and think about what's going on, I think if you think of what this idea of AI is, the idea of creating something intelligent, and that's a whole field of academic inquiry that for a long time involves a lot of different areas of science. A lot of what we're seeing now is much bigger computers and much smarter algorithms that are smart in a particular dimension. So this claim that AGI is solved is absurd to me. I'm not saying they're not going to get there, but you can see these very glaring gaps in the capabilities of say children or just humans, the way we interact and these systems don't have, and they keep making significant advances. But I think it's really important just to question what are those advances really? The history of AI is really, and the way people perceive AI is seeing something and reading a lot more into it. Some chest playing algorithms, people are like, oh, human intelligence is solved because we extrapolate from every time we see something to think we can do much more. And it always tends to be, even with these relatively very general models, that there's a lot that isn't solved. I'm struggling a bit with the math, solving the Navier-Stokes problem because I've always though of math as such an amazing pinnacle of human, not just intelligence, but creativity. I find myself feeling quite depressed that the algorithm solved it and maybe I'd just prefer it if a human did it.
Brian Barrett: Well, there are still five unsolved millennium prizes, so we'll get at it. Start working.
Will Knight: Yeah. It could be if all you have to do is find the right prompt, then maybe I can do it.
Leah Feiger: I believe in you, Will.
Brian Barrett: Thanks so much, Will. As we're recording right now, Apple's product release event is still underway. It's been gathering attention for a few reasons. There's always hype around an Apple event, but there are two things going on with this one. One, it's the first major event with the new CEO, John Ternus. And the main reason is that Apple is introducing its first foldable phone, its first really new iPhone form in years and years, maybe a decade. It's called the iPhone Duo. Samsung and Huawei have had these phones for years, but it is the biggest change in the biggest consumer tech product of the last two decades, and really the engine of Apple.
Leah Feiger: And it's going to be really, really, really expensive, right? This is priced like a MacBook. Is that what I'm seeing?
Brian Barrett: It's going to be. It's going to be around $2,000. It's a lot of money. It's not crazy, I would say. It is like—
Leah Feiger: You're Apple-pilled. We're all Apple-pilled.
Brian Barrett: No, I'm not Apple-pilled. Maybe. I guess here's why I think it's not as crazy. I think one, the way people buy phones now, they buy them like cars for better or worse, but it's like leasing a car, right? So you pay a monthly amount that ends up not being too crazy at the end of the day. The other thing is I think comparable phones like this, when you look around at the other companies that have made foldable phones, it's in that same ballpark. I think it's inherently, because it's really twice the material. Also, given the memory shortage that's been going on, the cost of memory chips has been through the roof. So there are reasons that it's expensive. I think they're still going to sell all of them. I'm not going to buy one. I still have an iPhone 14. I hold onto my phone as long as I can for various reasons.
Leah Feiger: We have several friends in our group who we say have Obama administration era iPhones. They still have the home button. They still work.
Brian Barrett: Good.
Leah Feiger: Barely, but somehow trucking along. That's what I imagine your situation is too, Brian, even though it's a 14.
Brian Barrett: Yeah, I try to. I mean, I think for environmental reasons, there's no need to waste a phone. I also think I'm not Apple-pilled, but I do like that Apple recently said that they were going to support phones that were much older and they've made steps to make older phones work better. So I do appreciate that they are doing that. But it's interesting. So the foldable phone, I'm surprised at how popular they've become because they are pretty expensive. They are strange. The use cases are kind of not super necessarily totally clear to everybody, but it's expected to grow 13 percent this year to almost 23 million units according to IDC, which is a research company.
Leah Feiger: Wow.
Brian Barrett: It sounds like a lot. There are more than a billion smartphone shipments total, so it's still a pretty small in the grand scheme of things, but it is a chance for Apple at least to, A, really normalize this kind of product in a way that just because Samsung has one, doesn't necessarily mean that everyone knows that they're there, thinks they're good. B, get more use cases going for it because when Apple enters a market, developers are more likely to build for it. So if you're a developer and you have an app that could take advantage of a foldable screen, you haven't yet because you just don't care because there's not a big enough addressable market, Apple entering suddenly creates a lot more of an addressable market, bigger opportunity for you to say, "Oh sure, I'll spend some time figuring out how to make this foldable thing work." And yeah, it's also just new and interesting.
Leah Feiger: I do like the nostalgic touch. I miss my Motorola Razr. I miss it every single day. Perhaps not the exact same capabilities, but it does the trick.
Brian Barrett: It is funny. It is. Imagine your Motorola Razr, but it opens side to side instead of up and down, and it's all one big screen, so there's no buttons to push. You can't do T9 texting. Then you can watch YouTube on it.
Leah Feiger: I'm waiting, honestly, for the first story to drop from the quiet car of the Acela between DC and New York where someone, some sweet person from the administration, did not realize how big that screen is when you unfold it and how many messages are visible.
Brian Barrett: Oh, it's going to be great.
Leah Feiger: It's going to be, this is the new frontier for eavesdropping. Also, congratulations, Apple. But for me, that's what I'm looking forward to.
Brian Barrett: Leah, what would it take to get you to purchase a foldable phone?
Leah Feiger: It would have to be a lot less than $2,000.
Brian Barrett: Yeah.
Leah Feiger: I can't wait for Zoë to come back next week and tell us all about her new purchase. To be clear, I don't think she's gotten one, but it would be really funny if she did.
Brian Barrett: I bet she will.
Leah Feiger: So our next topic involves the US Census Bureau, which definitely not as shiny or theoretically life-threatening as our previous topics, but really, really important. Last month, the bureau published a report that claimed to uncover evidence backing up President Donald Trump's claims that noncitizen voting cost him the 2020 election. He touted the report on True Social saying, "I won the election." This is, as you know, we've talked about this many times, one of Trump's favorite conspiracy theories. He cannot let it go. And WIRED reporters, Vittoria Elliott and David Gilbert, dug into how this specific report actually got made, who wrote it, and they discovered a lot of holes with this. I kind of want to start by saying, if you're listening to me and thinking, why do I care about a US Census Bureau report? They just do the census. They tell me how many people live in all of these different areas, how big families are, et cetera, et cetera. You're not 100 percent wrong, but this is really important data. These are reports about people and their lives, and the average citizen shouldn't have to know that much about it, but because of the situation we're in with the current administration, knowing these weird little intricacies of what's going on in government, I think has been really key to understanding the full picture here.
Brian Barrett: Also, to be clear, I do care.
Leah Feiger: You do care. There we go. All right. Can I run you through some of the quick down and dirty stuff, Brian, with what the report said?
Brian Barrett: Yes, please.
Leah Feiger: All right, great. So the report claims that out of over 128 million voter records analyzed, roughly 24,000 were tied to noncitizen status. To be clear, 24,000 might sound like a lot, but again, out of 128 million. These numbers are way too small to have changed the 2020 outcome, but they're still numbers. So when Tori and David went to experts about them, the experts actually said that they're more likely the product of data matching errors than actual noncitizens voting. So it was already this issue with the data themselves because the analysis combined so many disparate data sets. We're talking commercial voter files, immigration records, IRS, social security. WIRED has been reporting for a long time about how so many of these different agencies have had their work knitted together in the era of DOGE, in the era of Trump 2.0, and this is a good example of that. So this process is really error prone, and especially with commercial data, and you get false positives here rather than actual instances of noncitizen voting. What do you think about that, Brian?
Brian Barrett: Yeah, it all makes sense. And also, Leah, it's fair to say the amount of noncitizen voting that has been actually documented and found and prosecuted is vanishingly small.
Leah Feiger: Teeny tiny, like teeny, teeny tiny.
Brian Barrett: Which is not to say that there aren't some cases that get missed, but the idea that somehow there are 24,000 just sort of defies belief on the face of it. And yes, you can also see it makes total sense that as you're combining a bunch of databases, you are maybe double counting, you're maybe miscategorizing. This is a group that is not known to be particularly meticulous with the use of data. What I want to know, Leah, is how does this strike? Should I think of this as a preview of how this administration and the people around it will continue to sort of weaponize data and misconstrue data in and around the midterms and especially after the midterms if they don't like the results?
Leah Feiger: Great question. So you talk about the group that did this. I want to quickly hit that as we're getting into what the repercussions are. So the authors weren't listed on this report, which is crazy unusual, super, super unusual, hit up all sorts of red flags for our experts and former researchers at the US Census Bureau. Turns out WIRED was able to report for the first time that several of the reports authors have direct ties to the Trump administration and Republican political organizations.
Brian Barrett: No.
Leah Feiger: I know that shock of disbelief, it's crazy, but report authors include a former DOGE-affiliated attorney who's now at Commerce Department, a recent census appointee who was previously at the Trump Aligned Claremont Institute, two staffers from the also Trump Aligned America First Policy Institute. So these aren't necessarily people who have had experience in this before. So when we're talking about impact on a larger scale, I think that we also have to really consider that some of these things, again, they appear on their face boring. An average citizen shouldn't be like, "Well, why do I care that this person is authoring this report? These are all just federal employees that I'm paying their salaries via my taxes, et cetera, et cetera." But obviously there is something like there appears to be political leanings here involved on these reports, and these reports have impact besides giving Trump and his administration credence to claims that the election was stolen. Census reports have to do with voting districts. It depends on who gets to vote going forward. And as we're getting into the midterms, the Census Bureau has been weighing in on this already in very weird and politicized ways that are very concerning to people in the room.
Brian Barrett: And obviously it's not just about this one report is institutions and individuals rely on Census Bureau data for so much outside of voting for so many things. And if you start to erode that trust and you start to say, "I don't know if I can believe these numbers full stop," that puts you in a really precarious position because there's no other source for Census Bureau data. It is a sort of singular entity.
Leah Feiger: Well, that's the thing, it's singular. And that's what I keep getting back to is when we're looking at this, even, I don't know, I'm thinking 20 years from now, but even 5, 10, 15 years from now, it's very hard to remove reports like this or remove bad data, faulty data from a government's memory. This is officially now official. I'm saying this incredibly eloquently, but in 10 years from now when they're like, "OK, so we have this data point from here, this data point from here, this is how we're constructing these different voting blocks and centers and et cetera," it's going to be really difficult to disentangle what the Trump administration has done. And I'm sure they know that. And so there's legacy here that we're going to have to be contending with for a really long time.
Brian Barrett: Coming up after the break, we'll share our WIRED/TIRED picks for the week. Stay with us.
Leah Feiger: It is time for our WIRED/TIRED segment. Whatever is new and cool is WIRED and whatever passé thing we're over is TIRED. Brian, tell me what you got.
Brian Barrett: Leah, my WIRED is technology that doesn't listen to you. I would love to normalize technology that isn't always recording just in case.
Leah Feiger: That feels outlandish, Brian. Is that part of our lives anymore?
Brian Barrett: Apparently not, because my TIRED, you'd be shocked to believe, relates to this. So Apple, in addition to its iPhones, announced two new Apple watches, the Series 12 and the Ultra 4. These watches come with, quote, "audio intelligence tools" that are powered by audio gathered by the watch's microphones. They include sound and music recognition, a conversation recap feature, a live rewind feature that lets you see a transcript of anything that was said nearby you in the last 15 seconds.
Leah Feiger: Oh my God.
Brian Barrett: I will say a couple things. One, this is an opt-in feature, so it's not automatically doing that. That's good. A lot of times these are opt-out where it's like it's doing it when you don't even know it. My issue with this, as you might imagine, is that it's only opt-in for the person who's wearing the watch. If I'm standing around somebody who's wearing an Apple watch and they have this enabled and they're recording the last 15 seconds of our conversation and I don't know that, that's pretty frustrating. Don't love it.
Leah Feiger: That sucks. Bad. Bad all around.
Brian Barrett: Yeah. So I get that there are benefits to this. I get that there are circumstances and situations where I might even say, "Ooh, I wish I could Shazam that from my Apple watch. I forgot to ... " I don't know. I think these companies keep trying to balance the, "Here's this benefits that we can get from AI," but also you need to give so much to it to get those benefits and not just you, but the people around you. I don't know. I don't like that the default is increasingly becoming, "Hey, no, we're going to actually give this thing the ability to capture audio, video, whatever it is, of the world around you without the consent of the people who are being affected." That's it. That's my tired.
Leah Feiger: That's good. That makes sense. And just unlocking new fears every single day for me, Brian. Thank you.
Brian Barrett: That's what I do. That's what I'm here for.
Leah Feiger: Okay. My WIRED is happy back to school week for so many of my friends and their kids. It has been honestly incredibly cute to watch all these schools in my neighborhoods, in Brooklyn, across. Everyone's getting filled again. People are doing back to school shopping. It's incredibly adorable and feels wholesome and lovely even in our unseasonably hot weather. Yeah, so I just like it, the changing the seasons. It's nice.
Brian Barrett: Love that. My kids did go back to school the first week of August.
Leah Feiger: That's so crazy to me.
Brian Barrett: So I've been there. I've been in that world for a while already, but yes, congratulations to the Northeast for catching up.
Leah Feiger: My TIRED is a little bit sad though, which is there was a study that came out, I think it was sometime this week, maybe a day or two ago, about how US students, their reading scores have dropped, reached the lowest points that they have basically ever. And experts are saying this is because of screens and, quote, unquote, "digital distractions." I know that parents contend with this pretty consistently, but it was just a moment of we talk a lot on the pod about how these different technologies are impacting different groups. Is it worthwhile to regulate? Is it worthwhile to not? What does that actually entail? What does that really mean? And to just see it so starkly like, yeah, kids just aren't reading as much anymore. And this is post-COVID data, right? This is not counting what happened a couple years ago. So it was a moment where I was like, "Oh no. Will they be all right? Will the kids be all right?"
Brian Barrett: Yeah. A lot of them probably not, but some of them.
Zoë Schiffer: That's our show for today. We'll link to all the stories we spoke about in the show notes. Adriana Tapia produced this episode. It was mixed by Pran Bandi, who's also our New York studio engineer. It was fact-checked by Matt Giles and Daniel Roman. Kate Osborn is our executive producer, and Katie Drummond is WIRED's global editorial director.
Source: Wired
