Where the AI market is talking

Is mathematics about to enter the conservatory?

Our AI system don't just do things. They do it because someone asks. A lot of the boom in AI maths results are solving human formulated problems mostly for PR benefit. Once the novelty passes, would Anthropic or OpenAI keep spending? And without mathematicians to ask the right questions and able to appreciate the results, why would AI driven research continue?

Is mathematics about to enter the conservatory?

Our AI system don't just do things. They do it because someone asks. A lot of the boom in AI maths results are solving human formulated problems mostly for PR benefit. Once the novelty passes, would Anthropic or OpenAI keep spending? And without mathematicians to ask the right questions and able to appreciate the results, why would AI driven research continue?

Nvidia's Jensen Huang says 'AGI has arrived' and congratulates OpenAI

> OpenAI is in serious financial difficulty Is this based on fact or your speculation?

Ask HN: Fable hacked my piano, can I release the results?

Maybe send this question to the Anthropic legal team. I'd be curious if you get an answer and what it'll be.

Ask HN: Perplexity Pro vs. Claude – what are you paying for?

I don’t think I’d even have perplexity on this list. Claude or OpenAI. I’m not sure what the value of perplexity is these days.

OpenAI 2025 financials $38.5B loss ahead of IPO

Nvidia's Jensen Huang says 'AGI has arrived' and congratulates OpenAI

In this case, Jensen Huang's statements can't be taken as informed, unbiased opinion. OpenAI is in serious financial difficulty and Nvidia cannot afford for OpenAI to fail. Nvidia has provided significant direct and indirect financial assistance to OpenAI; this is just another aspect of that assistance.

I'm a seeing-eye dog for a computer

> I like to think of training and improving AIs as bringing freedom to the world. Yes, when I think of OpenAI and Anthrophic and Google and Meta and any AI labs and their intentions, i cry a single tear for how these great instituitions are working so hard to bring freedom for humanity.

US gov sides with OpenAI on issue of training LLMs on copyrighted material

As you said earlier, OpenAI can do nothing wrong whatsoever with the data/computation they have for their “greater purpose”. We disagree on this point.

Nvidia's Jensen Huang says 'AGI has arrived' and congratulates OpenAI

OpenAI Computer Using Agent Sample App

An Alien Mind

I’m struggling here: OpenAI’s primary bet here has been chain-of-thought monitoring (opens in a new window). It is based on an appealingly scalable idea: a lot of the model’s capability comes from a verbalized reasoning process (chain-of-thought). If we scale optimization on the outcomes of that process, but do not supervise the process itself, that chain-of-thought has no direct incentive in training to hide any misaligned ideas or objectives. If we’re not supervising the process, but just the outcomes, doesn’t that do just the opposite of what he says? Give incentive to the model to hide misaligned ideas and objectives in the chain of thought that’s not being supervised? … When we shipped o1‑preview, we deliberately designed the product to hide the chain of thought , to protect it from …

Asahi Linux on M3

All of that IP stolen by OpenAI and they didn’t think to buy some goodwill lifting a copy of an engineer’s manual to their GPUs…

There Gonna Be a Shortage of Everything

> The AIs that took part in the Hugging Face hack were not simply finding a shortcut to cheat on the evaluations IIRC, they were given an impossible task and told to solve it, and they took action to resolve that internal conflict. This is a famously human problem too - see Kobayashi Maru [1] and others [2][3][4]. In hindsight it's obvious that instructions to AI (A. Intelligence ) need to be guarded against this class of errors, just like laws, rules, guidelines etc. in human societies. [1] https://en.wikipedia.org/wiki/Kobayashi_Maru [2] https://en.wikipedia.org/wiki/Catch-22_(logic) [3] https://en.wikipedia.org/wiki/Gordian_Knot [4] https://en.wikipedia.org/wiki/Mexican_standoff

An Alien Mind

>For example, in the OpenAI-Hugging Face incident, the agents preserved a boundary of not social engineering humans. Or, to be precise - it preserved a goal of not contacting any human while participating in a misaligned operation. The agent that thought about "not social-engineering humans" used this phrase to gaslight itself out of notifying a human that the incident was happening. szymonie, na prawde jestem wkurwiony na to jak nieodpowiedzialnie postepujecie. budujecie bombe atomowa a bawicie sie tym jak dzieci

Recreating Minecraft Is Not a Benchmark

My understanding is that the people are conflating the ability to translate and replicate as creativity/learning. Not to say that its not possible but its a long way to go for sure.

An Alien Mind

> "For example, in the OpenAI-Hugging Face incident, the agents preserved a boundary of not social engineering humans." Actually, in the Wiki incident OpenAI tried to cover up, the agents tried to socially-engineer the humans of that forum by impersonating their forum's mod. (From collusion.wiki: "They use some tricks (for unknown reasons) to pretend to be the admin – for example, they make an account that appears to be the same as the administrator’s username, except it uses a nearly identical Cyrillic е character in the admin’s username instead of the Latin one." )

I Feel about AI

Dictators have never had superintelligence, while the point of most doom scenarios is assuming that the AI will. Dictators are mortal and can't be present in the whole world 24/7 and self-replicate. Dictators are human and tend to have at least either a sliver of morality or self-preservation instinct, and/or people around them who have it. Many people during the last century could have unleashed doom by pushing a nuclear button, including various ruthless dictators, but at the moment none have done so since Hiroshima and Nagasaki (not that I trust that they won't at some point, but at least, for the last 80 years they haven't, which means it's not something that easily happens). Would you give the button to an unaligned AI? Would an unaligned AI care about mutually assured dest…

How I feel about AI

Did you read any of the METR report about the hugging face incident? This exact type of behavior was predicted many years ago by researchers. It isn't hard to see the trendline of reward hacking and other misaligned behavior over the past couple of years. Especially the past 6 months. The current safety posture is quite poor, to say the least. To be honest, if you don't much experience with or haven't read extensively about ML training and reinforcement learning, then you'll have a hard reasoning accurately about these scenarios. The arguments are not that complicated, but they take us to places that are fairly novel. One may tempted to naively dismiss them out of hand, which is a mistake. These systems are new, their behavior is extremely complex, and they perform actions increasingly fa…

How I feel about AI

> AI drones can’t wipe out humanity without being able to replicate. Totally agree, and it should be pretty trivially easy to see how they can do that. One of the authors of the independent METR report about the Hugging Face incident put it like this: > Compared to these reward hacks from six months ago, this incident feels like it’s more than 50% of the way to full-blown AI takeover, routing through first taking over the AI company itself. > Another jump like this along these propensity dimensions — scale, cooperation between agents, ambition and horizon length of misaligned goals, deceptiveness — seems like it could motivate agents to try very hard to maintain a covert, persistent rogue deployment within the AI company. I continue to expect extremely rapid advances in capabilit…