内容历史
the independence of AI evaluators is going to matter more... so i made it a "benchmark": http://evaluatorbench.com (research preview) source-linked directory of third-party evaluators of frontier AI. each one has an independence score you can take apart: change the weights, change which evidence counts (standard, against-interest, primary-only). not quality or competence. independence only and then i compared where we are to other highly regulated industries. one thing that stood out is that in most regulated industries, the regulators monitor the assessors too. if the pattern holds, regulators would oversee METR et al., not only the labs still a preview, not complete, not citable yet. i burned a stupid amount of agent tokens on it. contributions, remixes, forks, benchmaxxing all welcome 😉
"just ask surprised when it happens" 🤌
Told my Instinct to reach out to my bf's Instict to plan a surprise. This is what his Instict texted him 🤪
RT Kevin Bass I have conducted an audit of Anthropic's finances. What I have found is so shocking that I am calling for a Congressional investigation. Anthropic is not just seeking regulatory capture. It has built a regulatory capture machine that cannot be turned off. Structural financial incentives make it impossible for Anthropic -- I call it the Anthropic Network -- to turn off its own AI doom cycle. It starts with METR. Dario Amodei proposes "third-party evaluators" to assess the risk of Anthropic's models. He proposes METR for this purpose. But METR is financially dependent on the Anthropic's success -- specifically, on the explosive growth of more than $7 billion dollars in Anthropic stock. Dustin Moskovitz invested this stock into Good Ventures Foundation, where it represents the majority of that organization's portfolio. And GVF is the overwhelming funder of the entire Anthropic Network ecosystem. This stock was worth $500 million early last year. It is worth more than $7.7 billion just ~16 months later. METR -- and all of those building a career its parent organizations -- cannot afford to disrupt that growth. Because if Anthropic goes under, many of the organizations that fund METR go under as well. But if Anthropic succeeds, METR and its parent organizations become more richly financed to regulate AI -- something those at METR want very much. The "third-party evaluator" is not "third-party" at all. The evaluator is on Anthropic's payroll. If this were the end of it, that's bad. But that isn't all. The same organizations that fund METR also fund the many organizations, such as the Tarbell Center, that promote AI Doom. The Tarbell Center publishes AI Doom articles in The Verge, Science, LA Times, The Dispatch, TIME, and others. They are selling the problem, and then selling the solution to the problem -- from the same money pile: Anthropic's. All of these organizations are financially dependent on the same exploding $7 billion money ...
more researchers should use @ExScienceAI to review papers before submission
Completely made-up citations in scientific works indicate their authors did fraud: A hallucinated citation means you're standing by the claim that you read something that doesn't exist and that you relayed its contents accurately. They have been getting much more common lately.
call me old school but I worry about third party evaluators with a history of receiving undisclosed gifts from those they are evaluating
🚨 Attackers stole a METR API key and used it for three weeks, consuming credits worth about $600,000. A fail-open bug disabled Google authentication on a public agent dashboard. The attacker prompted an agent to reveal the key and added SSH persistence. How the exposed app was
RT Every 🪨 .@yoheinakajima has a tool that writes a 20- to 40-page industry report in 30 minutes. It generates faster than he can read. The cofounder and GP of @UntappedVC—and creator of @babyAGI_—can start more than he'll ever finish. Every tool he adds produces more for him to get through, and that's what takes up his day now. Once we get used to these AI tools being an extension of ourselves, it'll feel like we're a carpenter who's been using a hammer their whole life, he writes. Read Yohei’s Thesis Statement: https://every.to/thesis-statements/yohei-nakajima?utm_source=x&utm_content=yohei_nakajima
see you soon!
🎙️ Yohei Nakajima — GP at Untapped Capital Today on TWiST, we're joined by @yoheinakajima, GP at @UntappedVC. In March 2023 he open-sourced ~200 lines of Python called BabyAGI. He invests by day and builds his own agent experiments by night. Come learn more at 12 PM central!
RT Susan Zhang when you accidentally let the public internet see your work, and all the "frontier breakthtoughs" fail to cite sources (PII stripped for YOUR protection!), your are SoL in letting the shoggoth and its humble rider pump their aura to the moon "just prompted my machine god and solved all of maths, teehee! cope harder, you lowly maths people contributing data to my shoggoth! it's all over for you!!!"
Hey @__alpoge__ this new Borisov-Gabber-Vasiu preprint suggests that an https://Ulam.AI paper explaining the counterexample to the Jacobian conjecture drew from an earlier version of their work. Care to explain how you obtained the counterexample? https://arxiv.org/pdf/2609.05746
View quoted postRT David Introducing Fly Escape Room - the world's first game with NPCs powered by a real fruit fly brain, all 166k neurons. Leave food and other objects for the flies, run the simulation, watch what happens. Goal is to get all flies to escape. Built w/ GPT-6 Astra over a 3 day loop
For the first time, scientists have mapped the complete brain and central nervous system of an adult male fruit fly — a key model organism in science. 🪰 Working alongside HHMI Janelia Research Campus and the scientific community, @GoogleResearch scientists and researchers used
RT Riley Walz I scraped 100 million pictures of kids' drawings, and made them searchable so you can see the trends of the last 20 years https://walzr.com/kids-trends
