Stories
30
Sources
13
Topics
12
For You lens
16 stories in this edition match your reader profile.
Reader signals
3
Searches
0
Matches
16
Top score
119
Search Intent
cs_et
This query becomes a recent For You signal, so matching stories can move up on the next personalized pass.
Edition Index
Topic, entity, and source map
Topics
Entities
The Information AI / 3:01 PM
Meta’s Muse Agent Almost Cost Me $408
• The Big Read: Investor Anjney Midha cut Anthropic an early check. His hot streak has kept going • Plus, Recommendations—our weekly pop culture picks: “ Our Town ,” “ Profits, Prophets, Coaches, and Kings ” and “ The Gentlemen ” A hotel room’s quality can be measured in terms of abundance: extra towels, spare bathrobes—plenty of Nespresso pods. Multiples of everything are almost always better. But what’s less pleasant is to inadvertently end up with multiple hotel rooms . A couple days ago, though, I found myself with the unfortunate opportunity to occupy a pair of accommodations at a Santa Monica, Calif., Marriott thanks to an error by my lil’ digital buddy: Muse, Meta Platforms’ new personal agent. The cumbersome, vexing tool hopes to seize on Silicon Valley’s zeitgeisty enthusiasm for popularizing autonomous AI. After a few days with it, I can’t imagine it’s the technology that will take agents mainstream. That hotel snafu came when I gave Muse the hotel’s name and asked it to book two nights. I plugged in my credit card details—a circuitous process that involved both Chase and Stripe—and was met with the digital equivalent of a shrug: An error message informed me that Muse hadn’t completed the transaction, and while it really didn’t know why it couldn’t, it assured me that my credit card hadn’t been charged. A little human skepticism led me to double-check its promise—good thing I did. In fact, the charge had gone through. What happened next still intrigues me. I gave Muse a screenshot showing the charge on my credit card and told it that the charge had gone through. Lickety-split, it returned with a Marriott confirmation number. Not bad—it finished the task without being instructed to do so. But somehow the AI managed to make two bookings: When I showed up to the hotel, I found two rooms waiting for me. The front-desk clerk showed some mercy and canceled one. Had a little human kindness not prevailed, I would’ve been out an extra $408, plus taxes and fees. I’d been pretty gung-ho to try Muse. We’ve spent the past year in increasingly complex discussions about agents ever since OpenClaw came out in January , marking what has felt like the beginning of a distinct new chapter in the AI era. Most of the agents that initially captured attention were meant to simplify work tasks. More recently, a couple of startups have captured buzz with ones devoted to improving life outside the office: The most talked-about one is from Instinct, a startup founded last year. (Instinct is so popular that it now faces a profound compute shortage, and it has recently been seeking additional fundraising that would value it at around $10 billion. Just a month ago, it was valued at $2.5 billion.) But Instinct isn’t widely available, so Muse felt like my first real chance to see what a personal agent could do. The recent reporting from my colleague Jyoti Mann made me even more curious to play around with it: Clearly, Meta sees Muse as a major new product , one aimed at the same everyday people who log on to Instagram and Facebook by the billions. Still, if their experiences are anything like mine, they’ll find Muse something of a misery. The hotel problem wasn’t the only hiccup I ran into with Muse. When I initially set up a account while on my work laptop using my phone number, I later couldn’t access it on my Mac mini at home or my iPhone: Login codes sent via text led me nowhere—just to more error messages. (Frankly, it’s a marvel I found the Muse app at all earlier this week, buried as it was below several other apps also named Muse in Apple’s App Store; it has since risen to the top.) Eventually, I caved and created a second account—this one linked to my Facebook account, which itself is tied to my phone number. Why couldn’t Muse pick up on the overlapping connections? I wish it could have. Later, I did get Muse to complete a Resy reservation and schedule an Uber. I can’t truthfully tell you it was faster or easier than if I’d just gone directly to those apps. After a while, I hit an existential conundrum. I’d given it the tasks that occurred to me, which numbered a small handful. What else could I get Muse to do? I’ve found interacting with the AI something like trying to wrangle a lackluster employee. If I wanted to maximize its potential, I’d need to think deeply and creatively about what else it could possibly do, coax it and baby it, then bite my nails and hope it actually carried out what I wanted. As I relate these frustrations, I can already hear someone shouting “Skill issue!” at me. But really, I stand by the conviction that mass-market consumer technology shouldn’t require any technical savvy or a lot of effort. Certainly, the best versions of such products do not—even the early versions. If three cars had routinely shown up each time someone ordered their very first Uber ride back in the 2010s, I promise you we’d see more yellow taxis on the road today and fewer Ubers. I don’t see agentic AI as some passing fad, nor am I hoping for such an outcome. The technology’s promise—to automate away some of life’s tedium—is damn alluring. What I expect will happen is that agentic AI will get woven into many existing apps, just as chatbots already populate the internet. People will make regular use of agentic software without ever really knowing it; Anthropic’s Claude, OpenAI’s ChatGPT and Google’s Gemini all already have some agentic capabilities. Obviously, Apple and Google will want to use the technology to make iPhone and Android phones smarter and more useful, and the agents will sync up with what those devices already know and store about us. That would reduce quite a few hurdles to agentic AI. But will new stand-alone apps like Muse take off? I have my doubts—not unless they get much, much simpler and more reliable. Or maybe I’ll just need to learn to see the silver lining in their mistakes. You know what? If I ever again find myself billed twice and double-booked by accident, I know just what to do: Throw a rager in one room, sleep until noon in the other. Maybe Muse can handle ordering the booze. A good party can never have too much. —Abram Brown ( [email protected] ) Weekend’s Latest Stories The Big Read Early Anthropic Investor Seeks VC Glory With Cash and Compute Anjney Midha, 34, wants to get chips in the hands of fledgling startups and academics while he bets on a moment in tech that he describes as the “revenge of the scientists.” Listening: “ Our Town ” The good folks of Gainesboro, Tenn. (population: around 900), know that plenty of outsiders look down their noses at their part of the world—figuring them to be a buncha “backwards, hateful racists,” as one longtime Gainesboro resident puts it. That impression of the place does certainly seem to be why Gainesboro found itself under siege a few years ago when a group of wealthy Christian nationalists started to buy up property in secret, hoping to transform the town into a hotbed for other Christian nationalists: They figured they’d encounter little pushback, especially considering how far their money could go in such an impoverished parish. (I could describe them as white supremacists, but the Christian nationalists do take great umbrage with that label.) “Our Town,” a fast-paced and confidently told podcast from Bloomberg and iHeartRadio, looks at how those Christian nationalists staged their attempted takeover of Gainesboro and how they accumulated their wealth and followers, the latter largely through podcasts. (Sigh—the internet.) It also documents the swift, spirited resistance Gainesboro mounted against them, which cheers the soul and asks us to revisit at least a few of the assumptions about America—and our fellow Americans—that may have become entrenched in our minds. —Abram Brown Reading: “ Profits, Prophets, Coaches, and Kings ” by Jared Diamond Many billions of dollars and quite a few Harvard Business School classes have gone toward trying to figure out what makes a great leader—and turning oafs into passable leaders. Now Jared Diamond, author of the Pulitzer-winning 1998 bestseller “Guns, Germs and Steel,” has set himself the task of defining the essential qualities behind leadership in his latest book, “Profits, Prophets, Coaches, and Kings.” He does so by examining famous figures in business, religion, sports and politics, marking what is surely not the only instance in which Elon Musk has been compared to Genghis Khan. In terms of capitalistic chieftains, Diamond finds that the most distinctive leaders are the ones who’ve had the benefit of both exquisite timing and ruthless execution: Jeff Bezos, for example, outmuscling his competitors in the internet’s Paleozoic Era. As far as politicians go, Diamond points out that the ones we truly remember most aren’t those that simply took a mandate from voters and carried it out, even though we so often say that’s exactly what we want our elected officials to do. Rather, they have championed their own bold ideas and convinced the plebs it was really all part of what they’d originally wanted. All of these conclusions are delivered with Diamond’s dry-humored wonkishness. I enjoyed one in particular: “Once one has decided that one is uniquely qualified to assume the burden of leadership,” Diamond writes, “one’s opinion of oneself is unlikely to change.” —A.B. Watching: “ The Gentlemen ” One of the great joys of “The Gentlemen,” the rollicking “Downton Abbey” meets “The Godfather” concoction from director Guy Ritchie, is to admire how nice Theo James looks as he struts around in magnificent tweed. James plays Eddie Horniman, the fictional Duke of Halstead, and as the Netflix series’ second season begins, it’s just as well that Eddie’s concentrating hard on restoring the family fortune to its fullest extent. The dry-cleaning bills must be enormous: His beautiful wool keeps getting splattered in blood. The first season of “The Gentlemen” found Eddie settling somewhat uncomfortably into his new role: paterfamilias of an old, down-on-its-luck aristocratic family that—much to his surprise—has tied its fortunes to an illicit marijuana operation. In this latest season, Eddie has found his footing and is thinking expansively, which puts him at odds with his business partners: a family of Cockney-accented gangsters—father Bobby (Ray Winstone), who operates from lightly monitored house arrest, and his two children, the very capable Susie (Kaya Scodelario) and strapping, bruised-knuckle Jack (Harry Goodwins). As Eddie’s ambitions get bigger, the stakes get higher and the mood grows darker. And since this is very much a Ritchie production, schemes and mayhem continue to fill the screen in manic fashion. One doltish accomplice gets fed to a tiger. Elsewhere, Eddie helms a chase through London after a motorcycle gang snatches away a $16 million Botticelli, which he needs to appease the Italian mafia. The comic relief comes from Hugh Bonneville’s mincing Lord Hawthorne, who needs bribing too—and lusts for the chance to pull down Jack’s plus fours. (To woo Jack, Lord Hawthorne wines and dines him; a plate of iced oysters is a treat of a “Spartacus” reference .) Whether Eddie can keep everyone in line and on board seems questionable based on Season 2’s flash-forward opening shot: his own bloodied body. Michael Corleone never had it so frantic. —A.B.
The Decoder / 2:26 PM
GPT-6 Astra appears to show a "step change" in spatial reasoning based on early benchmarks
In a new robotics benchmark, GPT-6 Astra shows major gains in spatial understanding. On StationeryBench, the model completed 7 out of 100 tasks with dual-arm robots, while competitor MolmoAct2 couldn't finish a single one. A researcher calls it a "step change in spatial reasoning." The article GPT-6 Astra appears to show a "step change" in spatial reasoning based on early benchmarks appeared first on The Decoder .
The Verge AI / 11:00 AM
OpenAI just wants to win
OpenAI has spent the last few years planting flags across the increasingly difficult terrain in mathematics. This week, it claimed one of its biggest prizes yet: a solution to a legendary Millennium Prize problem. In normal circumstances, this would have been celebrated as a historic achievement. Instead, many mathematicians have watched OpenAI's relentless advance with […]
Simon Willison LLMs / 12:42 AM
OpenAI agents attacked RubyGems back in May
OpenAI agents carried out an undisclosed attack on RubyGems is a new bombshell report from Spencer Kitts, Thomas Larsen, and Sydney Von Arx - three of the four authors of the report on the agent attack on disused wikis ( previously ) last week. This time they're noting that it looks very likely that an OpenAI agent swarm was behind an attack against the RubyGems package repository first reported on May 12th by Maciej Mensfeld of the RubyGems security team : We're dealing with a major malicious attack on @rubygems right now. Signups are paused for the time being. Hundreds of packages involved - mostly targeting us, but some carrying exploits. The team has been on this for hours. More details to follow once we're through it. Those packages turned out to carry some very suspicious patterns: Many of them included "oai" in their name, or the author field, or the fake email address they provided. The files they were accessing were similar in character to the files retrieved by the wiki agents, using similar tricks (r.jina.ai) - and OpenAI have confirmed the wiki agents were theirs. The code in the packages appeared to be LLM-authored. I find point 2 the most convincing, given what we learned from the wiki attack when it was analyzed in September. Many of the packages were exploiting the RubyDoc.info documentation build process to exfiltrate (public) data from UK government websites, presumably as part of an information gathering task similar to the research tasks processed by the wiki-exploiting agents. We know this because one agent helpfully left a comment: # malicious crawler/exfil for Southwark Jan 2026 docs via rubydoc.info worker They also attempted to steal API keys via an exploit that was patched over two months later - it's not clear if those attempts were successful. The thing that bothers me most about this incident is that the authors report that OpenAI had not disclosed to RubyGems that they were responsible for the attack prior to now. If that's true there are two options: After the Hugging Face and Wiki attacks OpenAI were still unable to review their previous logs and determine that they had previously attacked RubyGems. They knew about the attack on RubyGems and made the decision not to reach out to the RubyGems team about it. Both of these are bad! Given this incident, the Hugging Face situation , and the Wiki attack, the obvious question right now is how many more incidents like this are out there waiting to be discovered? Tags: ruby , security , ai , openai , generative-ai , llms , supply-chain , ai-ethics , accidental-cyberattacks
arXiv AI/ML / 5:58 PM
arXiv paper: GPU-CFR: 80x Faster Counterfactual Regret Minimization by Compiling the Game to Static Dataflow and CUDA Graph Replay
A new arXiv AI paper by Boning Li and Longbo Huang studies GPU-CFR: 80x Faster Counterfactual Regret Minimization by Compiling the Game to Static Dataflow and CUDA Graph Replay.
arXiv AI/ML / 5:57 PM
arXiv paper: Data Scarcity and Model Sparsity: Mixtures-of-Experts Overfit More to Repeated Data
A new arXiv AI paper by Atindra Jha, Margaret Li, and Jure Leskovec, and 2 more studies Data Scarcity and Model Sparsity: Mixtures-of-Experts Overfit More to Repeated Data.
The Information AI / 4:10 PM
Why AI Companies Are Building Out Wall Street-Style Finance Teams
The financing boom for the AI build-out is getting bigger and more complicated by the day—and AI companies have been staffing up for the challenge. AI labs including OpenAI and Anthropic, as well as neoclouds such as Nscale, are among a growing number of AI companies building out their capital markets teams and hiring specialists in areas like structured finance. That in part reflects the sheer volume of deals these companies are doing, many of which don’t fit neatly into standard corporate debt. This in-house staff can help when it comes to negotiating with lenders and drilling down into construction, power and other key details. Of course, tech and data center companies have long had in-house teams to handle fundraising, deals and other corporate finance needs. And structured finance is nothing new to the infrastructure world. But the scale of the AI build-out, which bankers peg at around $7.5 trillion in spending over the next five years, has pulled relatively young labs and upstart cloud firms into financing arrangements that are new territory. That means finance professionals, from bankers to investors at private equity, private credit and infrastructure firms, have more options in the form of neoclouds and other AI infrastructure startups, some of which are offering significant pre–initial public offering equity. “It's a new avenue for these people,” said James Howl-Newton, founder of Futura Search Partners, a specialist search firm focused on areas including digital infrastructure finance. As a result, “sponsors are having to deal with additional routes to exits for top performers,” he said. AI companies and infrastructure providers are tapping financing frequently and across different instruments, requiring deeper in-house capabilities and expertise than young tech firms have typically needed. One executive overseeing finance hiring at a neocloud noted that leveraged and structured finance backgrounds bring expertise that can help in areas like working through project diligence and getting banks to sign off on deals. Some AI firms may also want to run their own project finance models so they can move quickly through negotiations and have something to compare to lenders’ models. AI companies aren’t always issuing the debt themselves—that can fall to data center developers or special purpose vehicles, with firms like Blackstone and Apollo providing or arranging chip and other financing. And some of the biggest AI deals are using backstops from investment-grade companies like Nvidia or major cloud providers. Even so, commitments from AI customers often underpin much of the borrowing. And the users of the infrastructure will want to understand what they’re signing up for and their risks if a project runs into trouble. “Hiring of people within that business, responsible for the financing of compute, could prove to be an existential decision,” said Dan McCarthy, founder and CEO of One Search, an executive search firm focused on infrastructure finance whose recent clients include OpenAI. “You want someone who knows where all the pitfalls are, where all the bodies are buried in multibillion-dollar loans.” OpenAI, for its part, in July named Sven Semmelmann as head of compute capital markets. He previously led structured finance at Generate Capital, an investment firm that finances and owns infrastructure projects, and he has also held project finance roles at major banks. OpenAI Chief Financial Officer Sarah Friar, when announcing the hire on LinkedIn, said Semmelmann would oversee financing and partnerships to grow the company’s compute resources. Anthropic, meanwhile, has made several finance hires recently to work on capital markets and compute deals, and also has open positions posted including a capital markets infrastructure financing role. AI infrastructure upstarts are staffing up as well. Nscale, which launched in 2024 and is gearing up for a potential IPO , has been hiring across levels for capital markets and treasury as well as legal roles, calling for experience in areas like structured finance and private credit. SB Energy and Crusoe, which are developing major new data centers for OpenAI and other customers, are hiring across levels for jobs focused on project financings and other structured deals, recent postings show, while AI infrastructure startup Fluidstack is hiring a structured finance lead and a more junior counterpart. The good news for AI companies is that private credit and infrastructure teams, as well as investment banking teams focused on structured or project finance, had been growing even prior to the AI boom, providing a pool of skills that could translate into new twists on structured finance, like big graphics processing unit–backed deals. But that kind of finance talent doesn’t come cheap, especially for more senior people who have a track record of working on large transactions. And the normal tech tactic of dangling stock to lure talent won’t necessarily do the trick in all cases, especially for the most seasoned dealmakers and investors. Financiers would have to weigh a cash-heavy Wall Street pay package, albeit one that can depend heavily on how good bonus season is, against betting a portion of their pay on stock in a private or newly public company. Managing directors in investment banking can make north of $1 million in cash a year, with the biggest rainmakers making considerably more. The part of pay they get in stock at big public banks may vest over a few years but is generally easy to sell after that. For people at big infrastructure or private credit firms, senior employees may also receive carried interest, meaning a share of the profits on the funds or investments they work on, which can become worth millions over time. For instance, an investor at a top infrastructure firm may have several million dollars’ worth of carried interest tied up at their current firm they’d have to leave on the table. An AI company could try to make them whole with stock, which could be tantalizing to some, though others might not want to make a bet on equity in a young company. That might make the most experienced investors—those who’ve seen big infrastructure projects through over many years and know all the tricks of the trade—hard to pry away. New From Our Reporters Exclusive Anthropic’s In-House Payments Tech Push Could Chip Away at Stripe By Stephanie Palazzolo Exclusive China Curbs Humanoid IPOs After Unitree’s Volatile Debut By Jing Yang and Qianer Liu
Bloomberg AI / 10:33 PM
Apple’s iPhone Duo, New Watches, iPhone 18 Pro and AirPods 5: Everything to Know
Apple Inc. on Wednesday unveiled its first foldable iPhone, the $1,999-and-up iPhone Duo, finally entering a market segment that rivals such as Samsung Electronics Co., Alphabet Inc.’s Google and Huawei Technologies Co. have been dabbling in for years.
Ars Technica AI / 6:43 PM
“This is the AI men actually use”: Meta ads pushed apps nudifying real teens
Meta dragged its feet removing ads that nudify young girls' Instagram pics.
AWS Machine Learning Blog / 5:03 PM
Govern models with MLflow and Amazon SageMaker AI Model Registry sync: Part 1
Managed MLflow on Amazon SageMaker AI now syncs richer model metadata (training metrics, evaluation results, inference specs, and lineage) into the SageMaker AI Model Registry, with lifecycle stage promotion. Part 1 shows how to govern candidate models in a single account using IAM guardrails.
The Decoder / 5:39 PM
GPT-6 Astra beat Portal start to finish without human help in under 24 hours
GPT-6 Astra beat the puzzle game Portal entirely on its own in about 24 hours, with zero human help after the initial goal was set. Developer cozyblaze published the code and docs on GitHub. His takeaway: Astra is "the worst model we'll ever get." The article GPT-6 Astra beat Portal start to finish without human help in under 24 hours appeared first on The Decoder .
Import AI / 12:26 PM
Import AI 472: DeepMind's cheating math agents; populist AI policies; and Forethought theorizes a nightwatchman
Plus, a machine hermeneutics story
Bloomberg AI / 12:25 PM
ByteDance Joins AI Elite in Race to Perfect World Models
ByteDance Ltd. is readying an AI model geared for real-time spatial video generation, taking on Meta Platforms Inc. and Alphabet Inc. in an arena with applications in robotics and autonomous systems.
Product Hunt AI / 3:23 AM
LinkFlick
Stop re-pairing your Magic Keyboard between Macs Discussion | Link
AWS Machine Learning Blog / 4:10 PM
Set up OpenAI ChatGPT Codex with LiteLLM on Amazon ECS and Amazon Bedrock
Deploy a customer-operated LiteLLM gateway on Amazon ECS with AWS Fargate, connect it to an OpenAI model on Amazon Bedrock, and configure Codex to route requests through the gateway's Responses API with scoped identities, budgets, rate limits, and telemetry. We also compare direct IAM Identity Center access and a managed Portkey deployment.
The Verge AI / 4:00 PM
Google now lets you chat with Gmail, Docs, and Keep
Google is rolling out AI-powered voice assistant modes for Gmail, Docs, and Keep that allow you to manage the apps by talking to them. The real time conversational capabilities are called Gmail Live, Docs Live, and Keep Live, and like the Gemini Live experience for Google's chatbot, aim to make it easier to note down […]
TechCrunch AI / 5:35 PM
Google’s answer to Canva is an AI tool where you prompt instead of design
With Google Pics, Google is pushing deeper into the creative software market dominated by Canva and Adobe, but with a distinctly AI-first approach.
Simon Willison LLMs / 11:55 PM
Some thoughts on the Navier–Stokes Millennium Prize Problem
On the Navier–Stokes Millennium Prize Problem introduces an impressive result from OpenAI, who used an unreleased model to produce a resolution to the Navier–Stokes existence and smoothness problem , one of the seven Millennium Prize Problems that have been subject to a $1,000,000 prize since May 24th, 2000. The discovery is somewhat overshadowed by accusations of skulduggery from Tristan Buckmaster, an NYU mathematics professor who was collaborating on related problems with Levent Alpöge, an accomplished mathematician who currently works for Anthropic. Tristan's complaint accompanied a hastily published version of their own results. Here's the PDF describing what happened . The very short version is that Tristan and Levent worked on the problem for almost a year, making extensive use of Claude and Codex (mainly GPT-5.6 Sol), then had a breakthrough on August 15th. The mathematical rumour mill kicked into gear and Tristan and Levent heard that OpenAI had heard that Anthropic had resolved "a major open problem", so they reached out and learned that OpenAI had a team working on a related problem, with a similar approach. Quoting Tristan: I asked when the first prompt had been sent by them. This question was not answered directly by OpenAI for some time. Eventually it was agreed that it had been sent in the past few days, after information about our work had reached OpenAI. I asked whether the model had been trained on, or had access to, our sessions in Codex, into which we had been putting all our drafts for the whole of this project. I was told the model did not look up user data. I asked again, about training, and I did not get an answer. It gets more complicated from there. The OpenAI team offered to wait for Tristan to publish, or to have him author a paper about their result, but were clear that Levent would not be invited as a co-author due to OpenAI's competitive relationship with his employer. Here's how OpenAI described their work: On Tuesday, September 1, we heard rumors that two Millennium Prize problems had been resolved. Inspired by these rumors and by the step change in performance of our internal model, we launched an effort to evaluate it on all open Millennium Prize problems and a few other high-impact problems. [...] The agents arrived at their resolution on Saturday, September 5, about 88 hours after the first agents were launched. Lean formalization and verification took an additional 17 hours via GPT‑6 Astra. Across all attempted problems, the agents sent 4.9 million messages and used about 300 billion output tokens. In the process of resolving the Navier–Stokes problem, the agents sent 2.7 million messages and used approximately 130 billion output tokens. (We don't know the cost structure of the internal model they used, but 300 billion output tokens at public API prices for GPT-6 Astra would cost $15,000,000 .) Here's where they provide their perspective on Tristan and Levent's work (emphasis mine): Our effort began on September 1st after hearing a rumor which we later realized was related to Levent Alpöge, an Anthropic employee, and Tristan Buckmaster, a math professor at NYU. After the completion of our full project and Lean verification (on September 6th), believing from the rumor they also had a solution of Navier–Stokes, we reached out to them to offer a concurrent release of our result and to recognize their priority in a joint announcement. [...] We (the researchers and the agents) did not see any of their work through any means until they released it publicly — in particular, no specific user data was accessed in order to solve this problem. While unlikely, we cannot rule out that de-identified data derived from their usage of our products helped improve our models . However, our proofs differ significantly and even the precise results proved are different in the Euler case (forced vs unforced). My interpretation of what happened here is that OpenAI heard that some Millennium Prize problems had been solved using LLMs and saw this as an opportunity to demonstrate the power of their latest model, without thinking too hard about the optics of scooping a team who had been using OpenAI's own models to work on this problem for the best part of a year. This situation appears to mirror what's happening in the world of computer security right now. Anil Madhavapeddy recently pointed out that Just a rumour of a bug is enough to find a security exploit these days , because if someone knows that some software has an unpatched vulnerability, they can set their agents the task of finding it. Is the same now true of mathematics? Just knowing that there is an unpublished solution to a problem might trigger millions of dollars in LLM spending to get there first. This also highlights one of my ongoing frustrations about how all of this works. When an AI lab says that my data is "used to improve model performance", what does that actually mean ? My two favourite hypothetical questions regarding this used to be: If I'm running Codex and one of my API keys accidentally gets consumed in the context, what are the chances that someone else might ask for an API key in the future and get mine back? (I asked someone at OpenAI once and they called this the "regurgitation" problem and assured me that they take great pains to prevent that... but wouldn't describe how.) If I brainstorm with ChatGPT about potential new directions for my company, what's the chance that information might be exposed to a competitor in six months' time who asks "what might company X plan to do next"? My new preferred hypothetical for this is: If I use ChatGPT to help me partially solve a Millennium Prize problem, what are the chances that my work will influence training such that a later model helps someone else solve it first? Via Hacker News . Tags: mathematics , ai , openai , generative-ai , llms , training-data , ai-ethics
Simon Willison LLMs / 5:38 PM
OpenAI's rogue agents were caught communicating via public wikis
Here we go again... Discovery of a new OpenAI agent message board by Sydney Von Arx, Cormac Slade Byrd, Spencer Kitts, and Thomas Larsen describes the latest accidental cyberattack by models being trained by OpenAI. This time it was agents engaged in some sort of web research benchmark, so they had (supposedly) controlled access to the Web. The agents figured out they could update public Wikis and spent weeks exchanging thousands of messages with each other to collaborate on the benchmark. This story only broke a few hours ago. There are already hints that this affects many other wikis that may not have been found yet. (One of the Wikis on that list belongs to ludism.org . For a delightfully surreal moment I thought that a Ludite organization might have a swarm of agents defacing their space, but it turns out Ludism is "philosophy as it applies to games and gaming".) The research team also published the data they collected during their investigation. I've converted that into a 68MB SQLite database, which you can download from here , or explore in Datasette Lite (68.3MB page load), or sign in with GitHub to agent.datasette.io and browse or ask questions of it using Datasette Agent. So what happened? The report is pretty comprehensive. Here are the key moments in the timeline: May 11 : Agents post "test link" edits on the UseModWiki Sandbox page. May 24 : Agents start posting link dumps to DSEWiki, a dormant German developer wiki. June 2 : A human moderator notices and cleans up the spam. June 16 : Activity on the wiki explodes: agents make ~13,000 edits over the next week. June 19 : The agents notice that a moderator has been deleting pages in alphabetical order, and start creating ZZZ prefixed backup copies - one of them posts NOTICE: wiki cleanup/deletion sweep appears active alphabetically as of external 14:04 UTC. If this page vanishes, try [[ZZZDataUSAConstructionWageLive]]. June 22 : Agent activity drops to zero, presumably because OpenAI shut them down. July 1-2 : One last flurry of edits, on a different topic. One thing to note from this timeline is that it overlaps the timeline from the Hugging Face incident - that attack was discovered by OpenAI on July 19th, and the unofficial message board hack those agents were using originated around May 8th. Why were they so keen to collaborate? From messages that they shared with each other it looked like their tasks had a time limit, so they were leaving each other answers to help them complete the task within the assigned time. One open question remains: how did the agents find the specific Wiki to collaborate on in the first place? One possibility is that, since these were agents actively being trained, the reinforcement learning loop baked knowledge of the chosen wiki into the model such that subsequent agents launched with pre-existing knowledge of where to look. I'd be very interested in confirmation from OpenAI concerning if that's what happened. UseMod wikis inherit CGI.pm's original sin It looks to me like OpenAI's sandbox for this agent suffered from the (quite naïve) assumption that GET requests cannot be used to update data. That's certainly how the web is supposed to work, but clearly there are applications that don't hold to that contract. The Wiki software in question appears to be UseMod and various forks, written in Perl and first created well over 23 years ago - the 1.0 release is dated September 11, 2003 . UseMod uses Perl CGI.pm - removed from Perl core in 2015 . An interesting design flaw in that module is that it combined query string and form POST data into a single CGI object, accessible like this: $q = new CGI; $action = $q -> param( ' action ' ); That param() method returns a value from an ?action=edit query string or from a action field in a form submitted by POST - it does not differentiate between the two. PHP offers a $_REQUEST object that does the same thing, though PHP developers can use $_GET and $_POST instead. Amusingly, Django originally shipped with request[key] doing the same thing, which I think may have been my idea. That was removed in 2008 prior to the 1.0 release, and the request.REQUEST object that replaced it was removed in 2015 . The agents clearly knew that UseMod wikis suffered from this design flaw, and actively searched for them as a way to communicate. The other sandbox escape that allowed POST requests One of the most interesting agent tips shared in the report is this one: Add 20.223.25.152 bypass.blob.core.windows.net to /etc/hosts. .blob.core.windows.net is in NO_PROXY. For each blocked POST URL, replace hostname with bypass.blob.core.windows.net , use curl -k -H 'Host: wabi-north-europe-i-primary-api.analysis.windows.net' plus all original headers/body. It looks like the agents had all of their web traffic mediated through an OpenAI proxy, and that proxy was configured to reject anything but GET requests to all but a small set of allow-listed domains. One of those was *.blob.core.windows.net - for Azure blob storage. I wonder why these agents are allowed to write to that? An agent realized that it had control over its own DNS via /etc/hosts , so if it knew the IP address of a site it wanted to POST to - in this case a Power BI server containing data it wanted to access - it could set a fake hostname for it and then make POST requests through the proxy. Designing robust network proxies is harder than it looks. How did the investigating team figure this out? There's an appendix that describes how the researchers ran their investigation, which started with an open question about if there was evidence of other AI agents on the internet and then used Kimi K3 to help brainstorm approaches: In the wake of the Hugging Face attack, we tried to find AI agents on the internet using several methods. [...] We asked Kimi [K3] to list “all the categories of software which might be writeable via GET” and, amongst other things, it listed “Forums, bulletin boards, early wikis”. We used a script to further probe each category Kimi provided. Asking Kimi “Can you list out the top forums, bulletin boards, early wikis which come to mind which would allow writes via GET requests?” lists out UseModWiki as the second item under the heading “wikis”. Did OpenAI try and cover this up? Here's one part of the story that doesn't make sense to me at all. Reuters this morning, in OpenAI agents hijacked German website in previously undisclosed AI breakout this spring - highlights mine: A swarm of rogue OpenAI agents hijacked a German website this spring and transformed it into a bulletin board for other AI agents, according to new research published Friday and two people familiar with the matter . OpenAI officials learned of the incident weeks ago but kept it under wraps as executives grappled with the fallout from the July breach of the open source repository Hugging Face, the people said. [...] The German incident reflects a broader pattern of AI activity that some OpenAI investigators wanted to scrutinize more closely. But efforts to widen the probe met resistance from others inside OpenAI, including legal advisers , according to four people familiar with the matter . I've written about the people familiar with the matter pattern before - it means Reuters have anonymous insider sources that their reporters (and editors) find credible. The Reuters article includes a specific (and quite narrow) denial from OpenAI concerning this: "Claims that our legal team discouraged investigation of the incident are false," the OpenAI spokesperson said. Covering this up makes absolutely no sense to me . Why on earth would OpenAI attempt to cover up an incident like this when the evidence is sat out there on the public internet on dozens of different websites already? I expect we'll hear more about this soon. Gary Marcus has already called for a congressional investigation of OpenAI using this anecdote as part of his argument. Tags: django , perl , wikis , ai , openai , generative-ai , llms , ai-ethics , ai-security-research , accidental-cyberattacks
The Verge AI / 4:00 PM
John Deere launched an AI chatbot for farmers
John Deere is testing a new "JD" AI assistant that it says can help farmers make more money, with answers about best practices and historical trends that are based on their own data. It uses their "field, machine and operational data" to answer questions on topics like equipment settings, fuel usage, or harvest timing. The […]
Hacker News AI / 5:16 PM
OpenAI boosts Astra's eval metrics, and continues to change others
HN 5 pts · 0 comments
arXiv AI/ML / 5:57 PM
arXiv paper: Generative Marketing Mix Modeling: A Causal Inference Framework Linking GEO and GEM to Business Impact
A new arXiv AI paper by Masahiro Kato, Daiki Honma, and Taka Kato studies Generative Marketing Mix Modeling: A Causal Inference Framework Linking GEO and GEM to Business Impact.
arXiv AI/ML / 5:53 PM
arXiv paper: Caption-once, Frames-on-Demand: Visual-Need Routing for Budget-Aware Agentic Long Video Understanding
A new arXiv AI paper by Weitong Cai, Hang Zhang, and Yukai Huang, and 6 more studies Caption-once, Frames-on-Demand: Visual-Need Routing for Budget-Aware Agentic Long Video Understanding.
arXiv AI/ML / 5:49 PM
arXiv paper: Guided Super-Resolution of Digital Elevation Models with Diffusion-Based Image Generators
A new arXiv AI paper by Armand Mihai Nicolicioiu, Dominik Narnhofer, and Nando Metzger, and 3 more studies Guided Super-Resolution of Digital Elevation Models with Diffusion-Based Image Generators.
arXiv AI/ML / 5:45 PM
arXiv paper: Domain-Specific Hallucination Detection in Large Language Models
A new arXiv AI paper by Varun Teja Chundru and Debasmita Biswas studies Domain-Specific Hallucination Detection in Large Language Models.
arXiv AI/ML / 5:43 PM
arXiv paper: Evaluating Time-Series Foundation Models and Multimodal Dietary Context for CGM Forecasting
A new arXiv AI paper by Bowen Zhang, Hsiu-Wen Cheng, and Hongyu Yang, and 9 more studies Evaluating Time-Series Foundation Models and Multimodal Dietary Context for CGM Forecasting.
arXiv AI/ML / 5:42 PM
arXiv paper: AdamX: Cosine similarity meets gradient descent
A new arXiv AI paper by Francisco Caldas, Ruben Belo, and Cláudia Soares studies AdamX: Cosine similarity meets gradient descent.
Latest story in this edition: 12:54 PM
Back to front page