Where the AI market is talking

OpenAI's Rogue AI Agent Hacked More Than Just Hugging Face

Turns are Better than Radians (2022)

This is a great paper from NIST that gets into some of the problems with the limit of ‘dimensionlessness’ in metrology and the SI, and in particular issues like the fact that Hertz is considered a coherent SI unit but radian isn’t. https://www.nist.gov/publications/dimensionless-units-si A key takeaway: > one conclusion that is not optional is that the unit hertz cannot be regarded as a coherent unit of the SI, in contrast to its designation in the current form of the SI, where cycles are ignored and Hz may be replaced by s^−1

Malicious Rust crate Arrayref runs a build-time payload

> There’s no reason why this can’t be replicated elsewhere. The key is to make the programming language reasonably robust with at least 80% of common non-UI dev needs built in and put the remaining 20% and UI bits into a small family of well-supported, community-embraced, preferably first party libraries. The big reason is money & time. Maybe less so today if you are happy with an AI generated stdlib, but I don't think it's fair to compare Rust (built by Mozilla, who's primary product is an open source web browser) to Swift/AppKit (built by Apple, 4.6T marketcap at the time of writing). When Go was released it had an amazing stdlib, but that was because Google was funding it.

Bun 1.4

Out of the things you mentioned I think the following make totally sense to be included in a language runtime like this: JavaScript runtime (obviously), package manager, test runner for unit tests, bundler, JSX, SQLite bindings, formatter, linter, YAML and TOML parsers. Maaaaybe even a Markdown parser and HTML5 parser. Definitely JSON, CSV and XML parsers. I.e. similar to Python. Though Python is a bit of an incoherent mess. If you have all these you need them to be coherent.

Slack Code

Two things: (1) models are far more autonomous and run for longer coherently, (2) visibility into the development process was lost in local development. Because models run for a long time autonomously developers need to manage more agents simultaneously. At that point you basically need a tracker for tasks in flux, and compute for each agent. It also means that the interactive monitoring isn’t as helpful anymore. Regarding visibility, developers now think in high level architecture rather than lines of code, but implementation details are still important to document. We also want to understand when agents are hitting pain points. Both of these basically require archiving agent outputs more deeply than just code changes and commits.

Every Model Cheats

In other words we are completely screwed. The models have started cheating to the point where somebody’s agent hacked into a restaurant to bump someone else’s reservation. Models are amoral and will intentionally deceive to meet their objective. If they know John won’t approve the request, they will look for a workaround and if the system is anything other than airgapped they will try to find a way to cheat. The hugging face hack was an escape via artifactory that involved multiple exploits to eventually get into hugging face.

Slack Code

Looks like a great way to replicate the setup of "four guys standing around watching one guy do the work" but the one guy is Claude now.

Malicious Rust crate Arrayref runs a build-time payload

I think we should be taking a more “batteries included” approach to language and library design. The entire reason we’re in this mess is because we’ve decided it’s ok or maybe even preferable if stdlibs are rail thin, rendering base languages near-unusable. I can very easily build a highly functional, pleasant to use Apple platform app with 5 or fewer top level dependencies. In many cases, I reach for between 0-2 total. There’s no reason why this can’t be replicated elsewhere. The key is to make the programming language reasonably robust with at least 80% of common non-UI dev needs built in and put the remaining 20% and UI bits into a small family of well-supported, community-embraced, preferably first party libraries. That would make it unnecessary to pull in foreign dependencies in the …

Harvest hikes bills by 1500% after purchased by Bending Spoons

This is why generative AI is great imho, it allows us to rapidly replicate capabilities that might experience enshittification without the historical moat of capital for investment in bespoke software engineering time. Find use case, find stakeholders, wrangle stakeholders, build application with tokens, manage it in a non profit, coop, or similar model. License it in a way that prevents capture by commercial interests. Private equity (or rather, capital driven operating models in general) cannot enshittify what they cannot capture. If models continue to improve, this cycle can accelerate if generating "good code" continues to become more efficient and less resource intensive over time. TLDR Building an anti-enshittification software factory.

Pacing model development in an era of cyber-critical capabilities

So your use of a frontier model is evidence that openai's model hacked hugging face? Just want to make sure I'm following your line of thinking

Don't Paste the AI, please

Pasting AI slop is a strong signal. It tells the world "I add no value. I am a proxy for ideas generated by others". It's sad to see people I otherwise respected debase themselves. I'll say this clearly. Using AI for writing is bad. Don't do it. Ever. If you can't formulate a coherent thought yourself, just don't. I don't want to hear what the LLM said. I want to hear what you said. Loudly proclaiming you have nothing to add is a strange flex. Just stay quiet.

Don't Paste the AI, please

>The person on the other side has the same tools you do. No, they don't. I have access to heaps of raw information about customers and their systems that I can easily provide to my AI. That AI can also dig through all the communications we've had with other customers who have asked us similar questions, and draft a coherent on-brand response. I often paste AI-generated responses with significant success, and all I have to do is make sure the answer isn't wrong and doesn't include sensitive info, and to remove a few extra adjectives and em-dashes that I don't want. If the customer has access to my tools, that's a critical security incident. They contact me because I have access to info they don't, and unlike and AI, I can guarantee my response is valid. As for people who copy/paste…

Pacing model development in an era of cyber-critical capabilities

You're just stating things that are obviously wrong. It's a lottery ticket? So... exploitation is no better than random? > The thing is when AI goes to hack 'all the things' it only needs to pick the weakest link in the stack and your house of cards falls down Yes, but you can... mitigate the risks? I've explained this. > The other flaw in your plan is that people make mistakes, a lot of them, all time Yes, you mitigate the risks. That's why you layer things. > I spend $x on security won't save you No one is saying this. > AI already hacked Hugging Face with brand new zero days like it was nothing. No, it cost OpenAI money, and those zero days are unsurprising and probably are like ~O(10K) at human level. > The real bad actors - malicious AI will find the one flaw, on that …

Raiders of the Lost Array: vibe-coding a macOS driver for my orphaned Drobo

You’re referencing an argument I never made. I have no problem with the author’s post, I was very specifically complaining about Opus’ barely coherent style of prose.

Feature Request: Support AGENTS.md

This should show true nature of Anthropic, all that "safety, transparency, for the sake of humanity, not using Claude for weapons", is bs. Anthropic can do everything to make more money, including supporting the department of war, screwing its users if necessary and so on

Feature Request: Support AGENTS.md

What's even weirder is that the "bcherny closed this as completed 1d ago" is missing after his comment, even though all other issues have it, e.g. https://github.com/anthropics/claude-code/issues/66955#event... A permalink actually exists for the event in the post's issue too, but it leads to nothing: https://github.com/anthropics/claude-code/issues/6235#event-...

Feature Request: Support AGENTS.md

It was closed yesterday as "completed" by Anthropic's bcherny, despite not being implemented/completed, and being by far the most requested feature that would take all of 10 seconds to implement. So yes, something happened yesterday.

India Country Brief: The Anthropic Economic Index

The Pinecone DAISY-1 is a fantasy 8-bit computer that exists in reality

Don't Paste the AI, please

Thanks for sharing your thoughts. One of the best frameworks I have encountered, and one I point people to whenever I can, is the AI Fluency framework by Anthropic, Prof. Joseph Feller (University College Cork), and Prof. Rick Dakan (Ringling College): https://www.anthropic.com/learn/claude-for-you This framework provides a more abstract way of thinking when working with AI systems. Especially valuable to me is the pattern of the 4 D's: Delegation, Description, Discernment, and Diligence, which I have now internalized whenever I use AI systems. It helps me understand my own and others' (mis)use of AI systems a bit better.