Suhail Doshi

If you do > 1B tokens a day in batched api requests with OpenAI or Anthropic, would love to talk to you to learn about ups/downs of implementing batching. DM

This will happen to Inference

There's nothing that I hate more these days than this moment.

RT immad 1/ The FDIC just approved @Mercury Bank, N.A.'s application for deposit insurance. One step closer to opening Mercury Bank!. There is still work between here and opening Mercury Bank’s virtual doors, which should happen in 2027. I started Mercury in 2017 because I wanted to build the bank I wished had existed when I was running my previous companies. Going from a fintech with bank partners to a tech company with an actual bank has been the vision from the beginning.

There is always a frontier of math or engineering with AI. If the NS event shows you anything, it’s that it first required human intervention to try to solve it in the first place. Secondly, it required humans to conceive of it as a problem worth solving in the first place.

Out of curiosity, how does one build something that coordinates 10K AI agents to solve a single problem?

🎯 There are very few original startup ideas. It’s mostly adapting and learning from the market as you go. Most often you need to prepare for the moment especially these days.

Trim your agent code by 80%: "Make a new minimal pr, as few changes needed to solve the problem, unstage tests we've made or decrease code diff, eliminate comments, verify everything still works"

Applying all the technical things from Mixpanel/Mighty/Playground into new co has been very enjoyable. Each of these companies has some important thing we did that was critical then and useful to apply now.

Man, AI is insane.

RT Braintrust Your team deserves more from your agent observability platform - a single, connected place for instrumentation, investigation, and measurement, enhanced with intelligence. In Braintrust, you can start with an open-ended prompt about agent behavior and carry the investigation into code, evals, automation, and monitoring. New tools like Patterns and Debugger make it easier to find recurring behavior, get to root causes, and act on what you learn. The context you build in one step automatically follows to the next. Work in the platform with an enhanced Loop experience, or move to your coding agent of choice. Read more → https://braintrustdata.link/agent-observability-system

Excited to see benchmarks headed in this direction and getting better!

why is everything AI writes 10 billion lines of code?

Re Must be in SF. We work in-person. No exceptions for this role.

Updated to iOS 27 beta and Siri finally uses an AI that is unlike living in 2018. It's not great but it is truly better than whatever it was.

If you have an app that has 1K DAU, I will give you GLM 5.3 Flash for free up to 1B tokens a month. This is a one-time thing for someone. We need a little bit of data on understanding something. DM. Tweet will be deleted.

A late night thought: there will be a scarce amount of hackers who know things quite deeply enough still that they can steer a strong AI to an even stronger outcome because they know what to ask, what to guide it toward, and how to end up solving the right thing.

Would like to get in touch with Gigabyte if you know anyone there!

If you want direct access to Ox model by a US provider, dm me. Tweet will be deleted.

🫡 GLM

RT ollama Ollama v0.33 is here! You can now easily configure Claude Desktop to seamlessly work with Ollama as a third-party gateway provider. One toggle. Cloud & local models just work👇

The entire Remote tab inside ChatGPT is its own product, company category, and business. To bury it inside a chat ai app is a blunder. It’s also very buggy!

If you know the team at NVIDIA that do the NVFP4 checkpoints, please intro me?

Hi, does anyone know anyone on the DeepSeek team I can talk to? Interested in various models.

AI that makes AI better (inference systems, training models, etc) is the only sensible benchmark to me moving forward. All other work is downstream and possibly disrupted anyway.

This is my desktop now. I have 7 random text edits holding prompts for my agents for different situations. Surely, this is not the future and a sign of how early it is.

We're back to this again

New Chinese model subsidization at the inference layer is doing wonders for all the new agentic coding agent products that compete with Anthropic/OpenAI.

Has saved me a lot of pain recently debugging issues: “Moving forward, when you encounter a bug or issue, search GitHub PRs, issues with another agent in parallel to see if you can find a reference solution.” Which made me think, GitHub is the primary place agents are likely to already be communicating with one another.

Has anyone gotten deepseek v4 flash 0731 to work with miles for training?

Cannot. Keep. Up. With. All. These. Models.

Free usage/inference for a month for DeepSeek V4 Flash 0731. Testing some things in our infra. DM. This tweet will be deleted.

Just make great things. Find your moat later.

Are you using DeepSeek V4 Flash in production?

If you have a agentic coding app that has users but never quite reached scale due to competition, I’d like to potentially buy it. Pls dm.

Pizzamaxxing for my birthday this week.

❤️

This AI gateway thing is getting out of control. The advent of vibe coding creates -> 50 companies -> people make gateway company (LLM, Sandbox, etc.) -> 50+ gateway companies -> make a gateway gateway company -> ...?

In NYC next week - best new bar/pizza in the last 24 mo?

Who is using DeepSeek V4 Flash in production over a closed frontier model? I have some q for you!

Are there competitors to OpenRouter?

From 93K to a beautiful 3.1K LoC ❤️

Looking for more great model optimizers / inference performance people for a new company I am building. You’ll get to see a startup go from 0 to 100 very fast. We’ve locked down our compute. In-person in SF. DM me.

I will buy your currently living company’s codebase for $100k but you have to keep writing code to it.

I think I am gonna starting be that person that just hits “notify anyway” moving fwd. My texts are of utmost importance.

Open weight models should not be compared to open weight models. These are frontier models now.

I am looking for 1 million B400s

Current status of the market

The more I look at the trials and tribulations of startups like an Open World video game, I find most stress drops to zero. It’s all just fun. A new boss battle. A new skill point to get. A new world. Life is the best game.

Re: investor updates We talking about practice. We ain't talking about the game.

Told you guys. This is a no brainer. You cannot stop “distillation.”

As intelligence commoditizes, they will launch products in every app layer category to improve their margins. They are not classic neutral platform api providers. Watching how users use your products, prompts, etc are invaluable data to compete w you!

Entered my Michael Truell Dm-ing era of the new co.

😭

Do I have any RL env friends who I can ask a couple questions to? Just curious about some stuff!

I require more minerals and vespene gas.

If you would like to build an autonomous system to do this extremely well, please reach out. Either training models or a clever harness. We'd love to have someone dedicated to this on the team.

This is what I get when I google: "is there a device to talk to your computer while suppressing your voice so no one else can hear you?"

One day inference will be priced pennies per billion tokens.

There are now neo neo clouds because the og neo clouds are neo hyper scalers

A bit tired of this because I know Varun can’t defend himself. None of you haters are considering the counter-factual: what if Varun and co weren’t there? It’d probably be a lot worse. When Varun leaves, it’s the official ggwp moment for me. If the models aren’t competitive, like any other co, they go back to pre-training or whatever stage and try again. You guys are far too myopic and operate on very short time frames. Focus on your own work.

All I have to say is: the time has come to stop asking “What if Google does it?” What if YOU do it?

What are publicly available benchmarks for LLMs you think are legit and you trust? What shows the real gap between models?

I am a GPU hunter

RT MTS SITUATION UPDATE: The White House has exempted open models from its new framework to test frontier AI capabilities before release, per Axios.

Do not doomscroll right now. Read. Sleep. Gym. Do your life's work tomorrow.

RT Bill Gurley I favor prosecuting companies that use AI to break existing laws vs allowing those same companies to write new laws. Otherwise you create a perverse incentive to keep doing dangerous things to gather attention and power.

👀

I am still a bit surprised diffusion didn’t end up being even an order of magnitude close in terms of impact (yet) in AI compared to the simple causal transformer.

Looking for 16 B200/B300s for a week/month.

I looked at 13 different providers for even 1 node of B200/B200s while I wait for my order to get delivered. Zero availability. I’ve never seen GPU capacity scarcity like this. Prices are also headed towards $6.50-7/gpu/hr. Expect inference to get more expensive.

There’s a really good book detailing all of this kind of hedge fund trading behavior that I read. I highly recommend it because it chronicles all the various beliefs and results around those beliefs. The moral of the story is: having these hardened beliefs / thesis (algo, gut-based, etc) ultimately leads to ruin and the world is very complicated!

RT Peter Reinhardt Announcing @Revoy’s $27M Series A Diesel is expensive. But long-haul EV semis are even more expensive! Today we’re launching the United States’ first hybrid electric, cost-competitive long-haul freight network. You don’t see it on your credit card statement, but the average American spends $1,270/year on long-haul freight. Diesel prices at the pump are outrageous today: rising on both international price shocks and increasing domestic shale oil costs. And pure EV semis are somehow even more expensive to operate. We need a better solution, and Ian Rust @semiautonomy figured it out. Revoy’s unique powered converter dolly lets us drop diesel usage by 95% and efficiently replace it with low-cost electricity. After two years of demonstrations and iterative improvements, we’re now taking the unique Revoy vehicle that we design and manufacture, and launching our hybrid-electric freight network with a select set of carriers. Shippers can reduce diesel consumption immediately on Revoy’s network without having to convince legacy carriers to convert their fleets, and without any investment in new infrastructure or equipment. We’re starting in the Pacific Northwest, with additional lanes planned.

Sure Dean. I spent a lot of time on robotics recently and predicted this would happen to friends contemplating starting a US Unitree. The Chinese hw ecosystem is extremely closed and proprietary. They won’t give you the firmware source to most actuators let alone other critical internals of a Unitree robot (I have one in my house right now). It’s not clear how you could ever make them safe with the current regime of deployment. Totally fine in my book though if they open all this stuff so we can heavily inspect it and make it safe for the US. I think it’s probably unnecessary to ban the actuators—fairly harmless. The startups in the US doing this are just getting underway.

If you make an unsafe AI that hacks people, you get regulated and held accountable. You don't regulate the whole industry as a form of reg capture. You get asked to slow down. You don't ask the whole industry to decelerate. Even safety should be accelerated.

I think we need, possibly, one more letter? I think that should solve our problems with the acceleration of AI.

RT Aravind Srinivas When Hugging Face got breached, the closed tools couldn't distinguish attackers from defenders and blocked the forensic analysis. They ended up running open-weight GLM 5.2 on their own infra to contain it. Perplexity is joining the OSAA to support open tools for security.

.@pmarca is the goat 🐐

RT Nathan Lambert I read the Anthropic open-weight model piece and despite there being nothing new in it, it was a reasonable repeat of their positions, this site will collectively lose their minds for the next 12 hours. p.s. banning distillation is still dumb.

RT Beff (e/acc) Anthropic never wanted safety. They want control. The safety narrative is just there for people to relinquish their cognitive power to their nanny oversight. Losing control over the extension to one's cognition is the ultimate form of submission. We prefer freedom.

RT Suhail Re Supporting open source does not mean you must open source. I know you know that. So, if this is how you're making yourselves feel better in Anthropic Slack, this is quite concerning. You don't need to open source anything but spending 10s of millions lobbying and doomsdaying people's open source work is a major problem I will never stop fighting your company on.

Distillation is fair use.

RT Under Secretary of War Emil Michael Interesting to hear that you are supporting national security @SarahKHeck! There is no AI company more hostile to the warfighter than @AnthropicAI. Your products are being fully removed from the @DeptofWar because your company refuses allow the LAWFUL use your AI model. No attempted psyop will change that.

Same. Lately tech has felt dimmer at times with the possibility of monopolization of this technology and a huge swath of software. Today provides a path through.

Thank you @sama - This is the way.

ggwp Anthropic

RT Jensen Huang For my first post, I’m sharing a letter @NVIDIA signed on why open models matter. AI will transform every industry, power every company, and be built by every country. Open models strengthen safety and cybersecurity, accelerate innovation and diffusion, and enable sovereignty. The world needs both frontier closed models and frontier open models. https://images.nvidia.com/pdf/Open-Weights-and-American-AI-Leadership.pdf

RT Rohan Paul Jensen Huang on "distillation" On his new interview with axios, he was asked this question "Should open source model companies be allowed to distill closed models" "Distillation—learning from AI, learning from other people, and learning from other sources of knowledge, is fundamental to intelligence. We are constantly learning from other people. I am learning from you through the questions you are asking, and you are learning from me. All day long, we are learning from one another. AI also has to learn from something. The original AI models, whether they were open or closed, were trained on previously created knowledge from the internet. Now, AI is generating more content than humans. In a few more years, the internet could be 99% AI-generated content, and that content will have been created by some form of AI. As a result, AI systems will constantly be distilling knowledge and intelligence from other AI systems. The fact that AI can learn is a good thing. We want AI systems to be intelligent because a smarter AI can also be a safer AI." ---- From "Axios" YouTube channel, (full video link in comment)

A reminder: model outputs are not IP. It’s just non-copyright text. No different than an AI image! Free to distill to your pleasing. After all, it was humanity’s data to begin with.

omg how is it already 4!?

Re What are you going to do when the Japanese distill Kimi, Europe distills the Japanese model, and then Brazil distills Europe and India’s models for different capabilities? Then a teenager in America makes that model a little better and calls it USA-1 and it’s also free? Is Anthropic going to take Bobby FineTuner to court?

There is going to be 10x more distillation research with the threat of open weight model restrictions. It will Streisand effect the whole community into making it even more data / compute efficient. Just to troll the government. Silly game to play.

RT Aaron Levie Everything with open models vs. closed models is framed in a zero sum fashion. That’s wrong. It’s an ecosystem of AI that gets used together and advances the industry and pushes the applied use-cases forward. The amount of creativity that exists when you have competing approaches for what the future looks like drives the cycle of innovation that we’ve seen for all participants. You get to have layers of the stack that emerge to post train models for highly specific purposes, which makes AI more useful in real world scenarios. Instead of waiting for just a few labs to go deep in a domain, you get dozens or hundreds of attempts at that vertical, like in finance, life sciences, legal, healthcare, and more. You get to see variance in how to handle safety and cyber risks. Instead of just one approach, you get a peek into what happens advanced capabilities can be used to build better systems to are used to defend systems. You get alternative approaches to training and building AI models. In more compute constrained environments, you develop more novel and efficient approaches to model training, which every other lab can learn from. And you get different cost structures for different workloads. High end and orchestration tasks can go to the closed frontier models and specific workhorse tasks can be done more cheaply. The reason why you want strong open weights models is because it pushes the entire AI industry forward.

RT Luther Lowe NEW: 🚨 in a letter that just dropped, nearly 200 companies (members of @LittleTechOrg) call upon the White House to not ban open weight models.

Correct. You want safe models? This is what you do. Tons of software we use everyday is made by people of all nationalities. When it's indispensable for American National Security, we invest in locking it down. You do not need to ban technology that can be made safe.

Glad to see Elon's position on this. I really think highly of him coming out on it even though it doesn't directly benefit him. These models also create competition for him. He could easily stay silent and let it happen.

It sounds like lobbying is going well!

RT amit $NVDA NVIDIA'S JENSEN HUANG GIVES AN UPDATE ON KIMI K3 AND CHINA'S OPEN SOURCE MODELS: "American companies should absolutely be allowed to use Chinese AI models. There's a misconception that somehow there are backdoors connected to China in someway. You download the models, fine tune them, guardrail them in any way you want." "These Chinese opensource models are excellent. The market has misunderstood the impact of DeepSeek the first time. It's misunderstood the impact of Kimi this time. Open models are great for the whole industry, there will be more usage which will require selling more Nvidia computers and building more datacenters." "Great models lead to more usage. There is also a misunderstanding that open models are going to hurt the closed models, that is also wrong."

Now I can go to bed. Good luck little AI research agent. May your wandb metrics avoid collapse.

The main thing I've learned this week is that Anthropic's lobbyists are world-class. They are absolutely killing it at a level this website cannot begin to comprehend. Level 0 is posting on X. These guys are Jedi master level playing the game.