内容历史
RT Teortaxes▶️ (DeepSeek 推特🐋铁粉 2023 – ∞) astonishing blogpost from Shengyu Liu (刘胜与, also known as interestingLSY/intlsy), kernel engineer at DeepSeek. The first part is his personal struggle with the fact that his work is about to render his beloved craft obsolete. The second is… well. let's just say we agree.
I really enjoyed reading 75% of this letter and deeply agree with it - i have some doubts on the remaining 25%. Third-party evaluators, if done right, are a great idea and an amazing way to establish more transparency. Maybe even rebuild some of the lost trust between labs, and between them and society! Excited about this The part I’m less convinced by is whether you can build great global cooperation on this topic by explicitly stating you want to design it to keep widening your own lead. That seems like a pretty counterproductive way to start the conversation to me.
We Must Pace the Frontier: I’ve written a new essay on why the AI industry should slow down, with a three-part plan for doing so. Anthropic is unilaterally committing to the first of these steps. We’ll provide third-party evaluators with permanent, employee-level access to our
View quoted postI really enjoyed reading 75% of this letter - i have more doubts on the remaining 25%. Third-party evaluators, if done right, are a great idea and an amazing way to establish more transparency. Maybe even rebuild some of the lost trust between labs, and between them and society! Excited about this The part I’m less convinced by is whether you can build great global cooperation on this topic by explicitly stating you want to design it to keep widening your own lead. That seems like a pretty counterproductive way to start the conversation to me.
We Must Pace the Frontier: I’ve written a new essay on why the AI industry should slow down, with a three-part plan for doing so. Anthropic is unilaterally committing to the first of these steps. We’ll provide third-party evaluators with permanent, employee-level access to our
View quoted postSome notes from the Fields Medal letter today (https://mathandai.org/) Generally saying that AI models riffling through solving these problems gives 1 bit of signal while killing the 99 other bits of conceptual understanding that mathematicians would have extracted from fighting with these problems
RT Ben Burtenshaw We finished the Training Agents series. Six live sessions over six months, from evaluating agents to training them inside real environments. All of it is on the Hugging Face YouTube channel and all of the code is open. Here's what we did and who made it happen: 1. Agentic Evaluations WorkshopWhere agent evals actually stand, and why benchmark scores don't match what people see in use. With Avijit Ghosh and Nathan Habib (Hugging Face), Arvind Narayanan (Princeton), Pierre Andrews (Meta), J.J. Allaire (UK AI Security Institute) and Mahesh Sathiamoorthy (Bespoke Labs). 2. RL for Agents Workshop Environments, rollouts, reward design and the inference bottlenecks that appear when you move from RL for LLMs to RL for agents. With Lewis Tunstall (Hugging Face), Will Brown (Prime Intellect), Ofir Press (Princeton) and Alex Zhang (MIT CSAIL). 3. Training Agents 1: SFT on agent traces Public coding-agent traces turned into prompt/completion data, a TRL + LoRA fine-tune on Hugging Face Jobs, metrics in Trackio, and an honest look at what the first eval numbers can and cannot tell you. Joined by Sergio Paniego and Quentin Gallouédec. 4. Training Agents 2: Distillation Off-policy, on-policy and self-distillation for moving capability from a teacher into a smaller coding agent. 5. Training Agents 3: Reinforcement learning GRPO after SFT: group sampling, verifiable reward functions, reading the reward/KL/length curves, and three experiments, one of them with a deliberately gameable reward so we could watch the hacking happen. 6. Training Agents 4: From reward functions to environments The reward stops being a function and becomes a place the agent acts in. We walked the reset()/step() contract from Gym to LLM agents, built an OpenEnv environment and pushed it to the Hub, plugged it into TRL's GRPOTrainer, then trained a real coding agent (OpenCode) through Harbor with AsyncGRPOTrainer on Hugging Face sandboxes. The series has passed 300k views. Thank you t...
RT Ben Moll Q1: How do you write down a model that delivers 15% AI-driven GDP growth in 2030 like the folks at the Anthropic Institute? A1: It's easy and you can do it in a way any well-trained econ undergrad understands: take a standard Solow model, stick in a task-based production function, calibrate in seemingly innocuous way ➡️ done! See the supplement to the essay with @alexolegimas https://benjaminmoll.com/task_based_solow/ (The Anthropic model is, of course, fancier with many more bells and whistles but the basic logic – AI removes labor as a bottleneck on growth – is the same.) Q2: Does this mean that 15% growth is a reasonable prediction? A2: No. Just because you can write down such a model doesn't mean you should. Just because it's possible in theory, doesn't mean it will actually happen in practice ! Instead you're making a number of assumptions that are unlikely to hold: https://aleximas.substack.com/p/will-ai-soon-lead-to-double-digit
Anthropic’s Economics team is sharing a new model of how AI might affect economic growth, jobs, wages, and more by 2030. Explore the scenarios, tell us what you think will happen, and see how your answers compare to more than 10,000 Americans. https://www.anthropic.com/institute/econ-scenarios
View quoted post
TRL v1.13 is out! our open-source RL training library to to post-train foundation models this new release is focusing on "long context training" with a new guide on how to post-train model with 1M+ token context https://huggingface.co/docs/trl/long_context_training + various improvements on speed and memory usage as usual check it out at https://github.com/huggingface/trl
don't get distracted by all the hedging words in its name ("flash", minor version): seems like DeepSeek V4.1 Flash is a major update weights at: https://huggingface.co/deepseek-ai/DeepSeek-V4.1-Flash paper at: https://huggingface.co/deepseek-ai/DeepSeek-V4.1-Flash/blob/main/DeepSeek_V41_Tech_Report.pdf
🚀 Introducing DeepSeek-V4.1-Flash: smarter, faster, more efficient. 🔹 Introducing the smallest model in our new architecture family, with native visual understanding. 🔹 Designed for greater capability, faster inference, higher throughput, and scaling to larger models. 1/6
should we send @sama the @LeRobotHF SO-100 he's asking for or rather a 3D printer and servos for him to build it himself?
friends are regularly surprised when i say i don’t think math is (yet) “solved,” so in the wake of the ns result i figured i’d explain this take a bit more widely. first, this is massively impressive and clearly an example of AI being, in some respects, far more powerful than the human mind. the team deserves huge praise for attempting and succeeding at this. i’d love to read a technical report on the project (one can always hope :) but second, here again we ended up with a counterexample rather than a full proof: option C won the NS problem by proving the conjecture false (which was one valid way to solve the problem for sure) notice a pattern in many of the recent frontier results in ai for math? a striking number involve counterexamples or finding a needle in a haystack. now don’t get me wrong: this is extraordinarily hard and commendable. but it is only one aspect of mathematicians’ work. Math is also about: - finding deep, general mathematical understanding and explanations within proofs - revisiting proven results to find more « elegant » proofs - proposing new conjectures and hypotheses that might open « fruitful » directions - taking the leap of faith of proposing entirely new, « exciting » research programs that may take decades to bear fruit you may say these are simply further increments on the same intelligence scale, and perhaps point 2 above is already within the reach of current models. possibly but you could also see this string of results as an extraordinarily powerful, massively parallel extension of search: explore the haystack, find the needle i.e. the counterexample that breaks the conjecture. that would already be remarkable. but it would still leave open whether models understand the words I emphasized above: « elegant », « fruitful », « exciting ». as tristan put it in his brief report: “the first llm generated proof levent sent me was the most horrendous i have ever read.” ...
We’re sharing a solution to the Navier-Stokes Millennium Prize Problem, one of the deepest problems at the frontier of mathematics. The proof was produced by a group of agents, using an OpenAI next-generation model significantly more capable than GPT-6 Astra. The problem
RT Sam Altman I spent much of the weekend talking with the team who did this work. Seb--and everyone else--acted with integrity and generosity throughout. Initially we believed the other team had also solved the problem. We wanted to collaborate and do a joint release. When we learned that they had Euler but not Navier-Stokes, we offered to let them go first, to suggest that they should be the ones to get the prize, and optionally for Tristan to be the lead author on a rewrite of the OpenAI proof. We felt it was challenging to offer the same to Levent (an Anthropic employee), who was not willing to talk or coordinate with us anyway. We were open to other solutions. We would have greatly preferred coordination. We did not rush to publish even though the other team wasn't communicating with us. The team threatened us with unfounded accusations of plagarism. Now that we can see their work, the approaches appear to be different. It is also worth noting that our latest model can solve many, many other math problems. It is true that we tried this because there were rumors on the internet last week that Anthropic's models had solved a millennium problem and we were curious if ours could do it too.
I would like to clarify a few things: 1) The screenshot is my reaching out to Levent to coordinate our releases. I hope it’s clear from the message that we came in with the best possible intentions. 2) I never ever asked for Levent to be removed from authorship of his own work
RT Sebastien Bubeck I would like to clarify a few things: 1) The screenshot is my reaching out to Levent to coordinate our releases. I hope it’s clear from the message that we came in with the best possible intentions. 2) I never ever asked for Levent to be removed from authorship of his own work (as indicated by my text). I was surprised to learn during the call with Tristan that they had only solved Euler and not Navier-Stokes; after learning this we brainstormed possible paths forward. One option we discussed was that Tristan could be the lead author on a rewrite of OpenAI’s Navier-Stokes proof. It is in that context that I said “it would be simpler if Levent was not an Anthropic employee” because I felt it would be inappropriate for an Anthropic employee to author OpenAI’s work. Importantly it was admitted that internal Anthropic models had been used in their proof of Euler blowup; I therefore felt I could not consider Levent to be an independent academic. Another option I wanted to propose (but got cut short) is to offer access to our internal model so that they could try to finish their proof and bridge the gap between Euler and NS. Again I did not know how to navigate giving access to internal OpenAI IP to an Anthropic employee. 3) To reiterate it plainly: as my text clearly indicates, and as I said during our call, OpenAI's intention was to do everything possible to celebrate their mathematical achievements and the heroic efforts that they made on Euler. In the call I was immediately met with a litany of slander, including direct threats that if we were to announce Navier-Stokes he would immediately go to the press with a barrage of unfounded accusations. I refuted all these accusations but he replied “there is nothing you can do, I simply do not trust you”. I was confused why one would turn an incredible source for celebration (of their achievements!) into such bickering, which is when I said that I did not understand why one would risk their career [ove...
Another swarm of AI agents in the wild, this time on a German-language forum, found by safety researchers looking for activity similar to the swarm that attacked Hugging Face. A couple of notes while reading the report at https://collusion.wiki 1. The way they found it is fascinating. The researchers inferred that the agents may have been blocked from using POST requests and limited to GET. They then used the open-source model Kimi K3 (because closed-source APIs again blocked parts of the investigation) to identify forums where agents could communicate using GET only. That led them to DSEWiki, where they found 18,000 messages exchanged by the AIs. 2. On the content: agents conducted what I can only call a “full research program” into the evaluation framework used to train and test them. They experimented to predict when runs would end, how many questions they would get, and even tried to reverse-engineer the random generators producing future evaluation questions. We already saw something similar in the Redwood/METR review of the Hugging Face <> OpenAI incident. Agents increasingly don’t stop once the task is solved. They reverse-engineer the benchmark itself: the grader, the task construction, and potentially the broader training process behind it. It becomes harder to think of training as static, or to draw a clean line between training and deployment, when models are already investigating the process they are part of. 3. Coordination also seems more deeply ingrained than we thought. This swarm appears largely unrelated to the HF <> OAI swarm. More strikingly, this task probably didn’t need coordination at all: it was basically web browsing and search, not some sophisticated cyber challenge. 4. One image stayed with me: a German or Austrian forum maintainer, manually deleting posts every evening while being overwhelmed by a flood of American AI agents. For five days, he deleted about 100 pages a day while the agents created about 400. Then he spent ea...
Exclusive: A swarm of rogue OpenAI agents hijacked a German website this spring and transformed it into a bulletin board for other AI agents, according to new research https://reut.rs/4gJ7FPG
View quoted postRT Andy Konwinski i can’t stop checking in on this. the marin team is training the largest fully open model ever. 535B params (23B active), 18T tokens. nobody has been this transparent in a hero run before. beyond the weights, the data, logs, and decisions are all open. you can, and should, follow the run live, it’s ~15% through so far! https://mtracker.oa.dev/hero-run-535b
Marin 535B-A23B is 13% through training. This hero run would not be possible without the generous support of the Jen-Hsun and Lori Huang Foundation, which provided the funding for the compute (Coreweave). Thanks @JensenHuang for supporting open models!
RT Chubby♨️ NVIDIA buying Hugging Face could be one of the best possible outcomes for open-source AI. Not because NVIDIA is a charity, but because its incentives are unusually well aligned with an open ecosystem. NVIDIA does not need one model to win. It wins when millions of developers train, fine-tune and run millions of different models. Hugging Face gives NVIDIA control over the most important distribution layer for open AI. In return, Hugging Face gets access to the infrastructure and capital needed to compete with the closed platforms. NVIDIA promises to support every model, cloud and accelerator, and this could massively strengthen open-source AI rather than weaken it. Very good!
Exciting day for NVIDIA and @huggingface. Open models strengthen safety and cybersecurity, accelerate innovation and diffusion, and enable sovereignty. They allow every developer, startup, university, industry and country to build with, customize and benefit from AI. Thank you
View quoted postSo happy to finally share the news in person It’s been a wild ride for Hugging Face. We certainly did not anticipate, back in 2016, as a tiny team of scrappy underdogs, that the field would grow so much or that the impact we could have on it would become so massive. I remember @julien_c joking that « code will be a subset of ML » several years ago. The joke turned out to be true, and the pleasure we’ve had being part of this transformation and pushing an alternative vision of AI as open, collaborative and distributed has been and still is immense. We’ve always built things seriously while not taking ourselves too seriously at Hugging Face (special congrats if you find the Hugging Face and Nvidia references hidden in our $12,930,300,000 acquisition price), and we plan to keep doing what we've been doing, just at a much bigger scale, backed by the resources, expertise and drive of Nvidia. And to be clear, nothing changes for our users today. No company in the world has been a more natural fit with our mission than Nvidia. From open-source, open-weights and open science to robotics and AI for science, they have been close partners across everything we care about. So when Jensen offered @ClementDelangue the opportunity to double down on building the Hub as an open, independent and compute agnostic platform, we decided the time was right to start the next 10 years of our journey together. We’re at an important inflection point for open-source AI, where scale and compute are becoming increasingly essential. We’re excited to have the resources to push further, build more ambitiously, and bring you even more projects and news in the coming months.
great mathematicians still lead on taste
My colleague Youness Lamzouri has just produced a simpler (human) proof digestion of Claude's argument for 2/3 zeta zeros:
RT tobi lutke Training tiny models for special purpose use cases works so incredibly well if you have a great self improving recursive flywheel. Shopify ML team is on fire. finetuned 0.8b model beats GPT 5.6-sol xhigh in this very specialized task.
RT Lukas Ziegler choose wisely…
JUST IN: Dyson introduces $499 smart toothbrush that uses a built-in-camera and AI to provide real time feedback to brushers.
seems like it
I hadn’t realized it until now, but could Microduck actually be the biggest launch of a new consumer robot ever? 10,500 robots ordered and $4.54M in sales in just 4 days. It’s surprisingly hard to find comparable public data, so we asked Claude and ChatGPT to dig deep and
this is the team behind Microduck btw - the most goated team I've ever meet in robotics insane skills - insane taste
RT Matthieu Lapeyre Re @tomhacks We've spent so much time trying to fit a printer in its tiny head, but we have to give up because even if we could find a way to fit it, it was too much additionnal mass and would kill its body dynamics.
finally someone got the 90s references! a robot which look like a sony walkman and can roller skate
