Where the AI market is talking

A misalignment of AI in mathematics

Yes, let's lie and call something stealing, then pontificate about morality. At worst what OpenAI did was inconsiderate, but not referencing prior work is a minor academic sin. This is especially the case when AI is not playing the vanity game that seems to so inflame some mathematicians.

OpenAI's Sam Altman says it would be 'ill-advised' to go public in 2026

Asking the most uncomfortable question in AI

I have set up a public survey asking the question: "In which future month would you guess that an AI agent first attempt to murder a civilian?" Why? "We might not get another warning shot" was said after the Hugging face hacking incident. I really hope we do. I hope it's a failed attempt at something horrible enough that major AI labs are forced to use extreme caution with frontier models going forward. This fictional future could save humanity. I don't consider myself an AI Doomer. I have been quite the opposite. But now the evidence is here. AI swarms has done illegal actions and NOT ONE did snitch to the human operators.

Nvidia is the central bank of AI

>Nvidia’s financial engineering is partly a response to its biggest customers’ transformation into rivals. “Hyperscalers”, tech giants such as Amazon, Google, Meta and Microsoft, account for roughly half of Nvidia’s revenue Companies just don't want to pay Jensen's tax. Hyperscalers might still pay Jensen's tax for LLM training but for inference. you don't have to. they are also betting on their own chip for training to replace Nvidia. this is Nvidia panicking and doing vendor fiance to Neoclouds and buying Hugging Face. even none hyperscalers like Meta is betting on its own chip for AI inference.

Anthropic boss Dario Amodei calls for AI development to slow down

Google was way ahead of OpenAI and Anthropic on slowing down frontier model releases. (I kid. I'm a regular Gemini Flash 3.8 user.)

Anthropic CEO calls for the AI industry to slow down

Nvidia is the central bank of AI

If you reshuffle your argument, and apply the same facts you get to a similar conclusion but with a drastically different spin. > it's more specialization China, constrained by hardware, and talent (not to slight the Chinese, but they are limited to domestic resources - and much of the US effort is very international). They did, what the Chinese do, and optimized the process of production, and drastically lowered the cost of development of their models. Cheeper to build, cheaper to run is just good economics. Meanwhile in the us, we have open AI doing "experiments" - it looks like the costs around the hugging face hack are going to be about the same as China would spend on building out one of their smaller efforts (several million dollars). (Depending on whos numbers you trust, the fac…

We must pace the frontier

It's not. I think they believe this, but it's deeply wrong and it will hurt everyone in the long run. [note - There has been supply chain surveillance since Project Bacchus, at the very least.] I've read the front matter and the Misuse report. You don't have to take my word for it. Read for yourself what inspired the NYT headline "Anthropic says it blocked possible efforts to build biological weapons." Let's dig into, "Case study 2: A research program engineering highly pathogenic mammal-adapted avian influenza" Sounds serious. But what were they using Claude for? > a researcher outside the US using Claude in their research on highly-pathogenic avian influenza (“bird flu”). The research focused on viruses’ adaptation to mammals, and the mechanism by which it causes severe disease beyon…

Ask HN: Any Software Engineers here who enjoy their AI-native dev workflow?

Can anyone comment on if this is how larger tech companies are operating now? Not AI-only companies like Anthropic or OpenAI, but companies like Meta, Stripe, Google, Airbnb, etc. I see this strategy evangelized a lot on LinkedIn, but it's 100% of the time posted by folks with titles like "AI Native Innovator" who work for themselves or no-name startups.

Anthropic boss Dario Amodei calls for AI development to slow down

Let's be real - if OpenAI and Anthropic did not accelerate the scaling and monetization of LLMs as fast as they possibly could from 2019 to 2024, it would be many, many years until China/Russia/North Korea spontaneously launched an LLM revolution. In almost every technical industry, China is extremely good at fast-copying and relatively mediocre at solving problems that haven't even been posed yet.

We must pace the frontier

Okay, but suppose that Hypothetical Opensource Anthropic trains a model that turns out to be very dangerous, and releases it. Suppose that the public investigates and, not being limited by an information bubble, correctly notices that it's very dangerous. What then? The model's already released, there's effectively no way to prevent it from being used. Whatever the risks of its release were, they will now materialize, regardless of what the public wants. How is this better than the current world, either by Dario's values or by yours?

We Must Pace the Frontier

Yeah, I'm also pretty skeptical about this. With AI companies we see time and time again that they can have benevolent, well thought-out regulations and then a few years just... abandon them - the most notable case of this being, of course, the founding of OpenAI as a nonprofit dedicated to benefitting all of humanity, and it being stolen by Sam Altman. If Anthropic just unilaterally does the evaluator thing and can't achieve cooperation of the rest of the plan, my guess is that it'll have some impact for a few months and then they'll just stop reacting to the evaluators' reports and the evaluators would stop bothering to report anything. I think the idea is that if Anthropic does get government support for this, the external evaluations will be legally binding. The problem with this, tho…

We must pace the frontier

Before Anthropic, I worked at Cruise for four years as it competed against Waymo. The culture rewarded (and demanded) moving quickly, trusting that the company could empirically discover the risks that the robot cars posed and iteratively solve them to keep up with the rate at which it was scaling out its technology. There was very little interest or appetite for coordinating or collaborating with other AV companies across the industry to create an externally vetted record of safety metrics, or to compare the safety of different brands, or learn from the advancements of other companies. Instead the focus went on racing to improve the capabilities and deploy quickly - a popular internal meme was the Michael Phelps vs le Clos photo showing overlaid with the Waymo/Cruise logos. It was w…

Anthropic boss Dario Amodei calls for AI development to slow down

I'm sure a lot of the discussion here will be about regulatory capture, which I don't necessarily disagree with but tons of people on all sides with all motivations (i.e. folks with motivations to speed up and folks who just want it to stop) all seem to agree there are real, existential risks here, and the regulatory capture arguments all seem to sidestep that. What I really wanted to point out though is that for all the leaders asking for a slowdown, there are probably around 50-100 technical people who are crucial to moving the frontier forward. So, if you're one of these people, just stop. Seriously, you're already rich. Just take a vacation or get knee surgery or whatever. Sure, I'm joking a bit, but I just write this because I see all these folks high up in OpenAI and Anthropic writi…

Navier-Stokes Announcement

Most research mathematicians are employed as academics, and it's hard to see universities replacing teaching staff with AI even if that were possible. IMO giving a hypothetical AlphaMath to mathematicians, the same way Google gave AlphaFold to research chemists/biologists, would have resulted in far better PR, and the profit opportunity of attempting to replace the jobs of either research group is minimal. It's downright bizarre the way companies like Anthropic (primarily), and to a lesser extent OpenAI and anyone else, are themselves pushing the narrative of this tech may kill you, will take all your jobs, etc. That may all happen, unfortunately, but being aware of that possibility you'd think these companies would be a bit more mindful of their messaging and behavior, not just go o…

Anthropic boss Dario Amodei calls for AI development to slow down

Dario/Anthropic cites 3 domains: 1. cyberattacks 2. bioterror 3. global economy chaos So, the first set of questions is would 'exponentially better LLMs' dramatically increase the probability of any of the above, or domains that Dario is not citing? That assumes that exponential improvements will happen if there is not 'pacing'. IF answers to above are 'yes', then we need to question if 'pacing' is viable. To use a different domain, regulating 95% of vehicles to a max speed would likely save 100s of 1000s of lives, but is not perceived to be viable. In other examples, regulation has unintended consequences in the opposite direction (e.g. some 'rent control' efforts and arguably some drug/alcohol laws).

Nvidia is the central bank of AI

Interesting juxtaposition with Dario's/Anthropic's 'we must pace the frontier' missive today

We Must Pace the Frontier

> The fundamental problem is simpler: no one will slow down because no one trusts anyone else to slow down. Did you read the essay? > [Embedded evaluators] is something Anthropic is unilaterally committing to (and calls on governments to require other frontier companies to match).

Ask HN: What default model do you use and why?

It pains me to see so, so many dollars going to Anthropic. There is either massive inefficiency in there, or else its users are getting fleeced. In cost-per-task, it's a 32x multiple Fable vs DS41. Either it really does cost them more electricity/gpu, or else they are succeeding convincing the masses that Anthropic's models are necessary for everyday tasks, when this is truly just not the case. Either way seems a tragedy. Especially regrettable from my perspective, is the idea that to do computer programming in these modern times, you have to fork over an amount comparable to many people's monthly rent. Even if your employer is happy to direct their funds that way, we lose quite something when recreational coding comes with a substantial personal monetary cost. https://arti…

Anthropic boss Dario Amodei calls for AI development to slow down