@LangChain Always hiring: https://t.co/D5Ut3loFO7
The hardest part of and “org harness” or “enterprise ai” (as opposed to personal ai” is auth, and managing auth in a multiplayer channel (like slack) We’re thinking a lot about how to do this right for managed deepagents
i love @cursor_ai but the slackbot auths as the user who started a thread (instead of the user who sent a message) meanwhile claude tag lets you bypass PR reviews bc the PR is opened as an external entity (claude itself) anyone have solutions that actually work here?
We will be following this strictly for all LangChain dinners going forward 🫡
RT LangChain 🔌 Every Managed Deep Agent is an MCP server. Let coding assistants and other agents delegate research, analysis, and more to your deployed agent as a tool. Deploy once. Reuse across compatible MCP clients through the built-in /mcp endpoint. Get started → https://docs.langchain.com/langsmith/python/managed-deep-agents-mcp-endpoint
memory has been a hot topic for the past ~2 years every time we bring it up or do anything there, gets ton of interest, but nothing sticks few takes on why below: - how to update and then use memory needs to be pretty tightly integrated with the harness. so its hard to build a standalone memory product - the hardest part of memory is the logic of deciding what to remember. storing and even querying memory is not necessarily hard. the logic of deciding what to remember is often (a) application specific, (b) just prompting, which makes it hard to productize - memory is not that useful in a general purpose agent, but more for when you are doing repeated tasks. more high usage ai agent are not just doing the same task repeatedly - memory hasnt really proven to be useful for coding agents, and most agent things that take off happen first in coding agents
I continue to believe memory is the most interesting and under invested space right now
View quoted postRT Sydney Runkle we just changed how deepagents read files! our evals show that this new format reduces edit_file errors by 15% AND total input token usage by 10% the little things really add up w/ agents; we're hard at work on these small but mighty optimizations at @LangChain
RT Derek Gilbert We have been doing this at Consensus with great success. By far the most complete agent platform end to end, Slack integration worked perfect perfectly. If you are working on applied AI in any serious fashion, I highly recommend giving managed deep agents a look.
PSA: You can connect a Managed Deep Agent to Slack. Mentions, DMs, and thread replies all start a run. Your agent posts its response back in the same conversation. Here’s how to get started ⤵️
View quoted postslack is where agents are doing work managed deepagents has a very tight slack integration - check out the tutorial on it below
PSA: You can connect a Managed Deep Agent to Slack. Mentions, DMs, and thread replies all start a run. Your agent posts its response back in the same conversation. Here’s how to get started ⤵️
View quoted postRT LangChain PSA: You can connect a Managed Deep Agent to Slack. Mentions, DMs, and thread replies all start a run. Your agent posts its response back in the same conversation. Here’s how to get started ⤵️
RT Caspar good overview on the managed agents landscape by @Sam_Witteveen we've seen many major ai companies ship a managed agent product over the past few months. this space will only continue to grow exponentially langchain first built a managed agent product two years ago - langsmith deployment - a self-hostable runtime for agent workloads. LSD has evolved with our open source ecosystem and is used today for hosting anything from simple langgraph workflows to complex apps built on deepagents harness this space has come a long way in two years. managed deployment was the first step. the next is to give devs a complete foundation for their agents, so they don't have to assemble it themselves. that's why we built managed deep agents: you bring the the agent's intelligence (its instructions, skills, tools, model) and we provide a managed runtime (LSD agent server, session management, credentials, tracing) out of all the options listed below, MDA is the only one built on an open source harness. if you're building agents, see how much time you could save: http://langch.in/mda
RT Shreyansh Patni Want to build a domain-specific agent harness? This is a great practical guide by @sydneyrunkle on designing and building your own custom agent harness. Worth reading if you’re exploring agentic systems and AI-native products. https://langchain.com/blog/how-to-build-a-custom-agent-harness
Either you die a system of record or you live long enough to become a domain-specific harness
View quoted postin case you want to build a domain specific harness: https://www.langchain.com/blog/how-to-build-a-custom-agent-harness great blog by @sydneyrunkle
Either you die a system of record or you live long enough to become a domain-specific harness
View quoted postRT Colin Francis Credit Genie uses openwiki to help their engineers and coding agents work more effectively within their code bases their framing of openwiki is as an "institutional knowledge insurance policy" check it out ⤵️
Credit Genie's coding agents used to guess at how a repo worked. Now they just check their OpenWiki.
View quoted postRT LangChain Credit Genie's coding agents used to guess at how a repo worked. Now they just check their OpenWiki.
RT Kyle Jeong the future of computer-use is no longer coordinate clicks, the best agents are writing and executing code to control their environments. we dove deep with @LangChain and wrote about how agents that can write code coincidentally make for amazing browser agents.
i expect computer use to get a lot better and more popular over the next few months! llms becoming great at writing code is actually a big catalyst for this
RT Caspar Connections offer a simple, elegant way to implement agent auth. We support agent and user (OBO) credentials in mda 0.7. Try it out, it’s magic https://langch.in/mda
New in Managed Deep Agents 0.7.0+: Connections. Agent-owned secrets for tools everyone shares, user-owned OAuth for anything that should carry a real person's identity. Store your credentials in your LangSmith workspace instead of .env. https://www.langchain.com/blog/connections-managed-credentials-and-per-caller-identity-for-managed-deep-agents
View quoted postRT LangChain In 10 minutes, we set up Connections in Managed Deep Agents. Watch as we configure: 1️⃣ A shared @tavilyai key every user pulls from 2️⃣ A @Linear connection, OAuth via just a server URL 3️⃣ A GitHub OAuth app that files tickets under the user's own login, not a bot
RT LangChain New in Managed Deep Agents 0.7.0+: Connections. Agent-owned secrets for tools everyone shares, user-owned OAuth for anything that should carry a real person's identity. Store your credentials in your LangSmith workspace instead of .env. https://www.langchain.com/blog/connections-managed-credentials-and-per-caller-identity-for-managed-deep-agents
I like this framing a lot: agent improvement is harness improvement, not model improvement The interesting interventions are often at the tool boundary: - what context you pass - when tools are provided - how you recover from failure - what gets measured afterward That’s the loop we’re building around deepagents (the orchestration logic) + LangSmith (how you measure)
Beyond Prompts: Measuring and Optimizing LLM Tool-Agent Harnesses — new paper (arXiv, 4 Sep 2026; accepted EMNLP 2026). https://arxiv.org/abs/2609.05736 Treats harness optimization around a fixed model as budgeted selection: edits are guarded intercepts at the tool boundary, not rewrites
RT LangChain You can use the LangSmith CLI to pull your flagged traces, classify each one by issue type, and build an eval dataset with a split for each. A 3 minute tutorial ⤵️
RT Syd Here’s how Row-Bot built using LangGraph at its core, make multi Agent orchestration easy and seamless: https://x.com/sydsachar/status/2068319612433817729
harnesses should make context engineering easy "forking" subagents is a useful context engineering trick that is now built into deepagents
View quoted postRT Sydney Runkle want to build a company brain like stripe? they built it on deepagents, an open source harness. it’s super easy to customize, you can connect to any data source under the sun, and its model agnostic! https://github.com/langchain-ai/deepagents
The team at @stripe is setting the standard for internal AI platforms: minion coding agents, a custom prototyping rig, and now their company brain, Kai. On today's episode of How I AI, Sharadh shows us how 1.5 engineers and 2 weeks got them a company brain, including: -
View quoted postmemory comes built into deepagents!
holy sh*t. this is f**king insane. langchain just documented the missing memory layer for agents that are supposed to improve after every run. instead of stuffing everything into one giant prompt, deep agents can keep long-term memory, reusable skills and lessons from previous
View quoted postan increasingly hard part of agents is handling auth does the agent act as itself, or on behalf of a user we're trying to make this as easy as possible in managed deepagents!
Agent auth is hard, but it shouldn't be! Your agent needs to act as someone: sometimes that's one shared identity, sometimes it's the person asking. To get either right you'll need an OAuth app for each service, a token store, refresh logic, and a consent flow Today we're
RT Caspar Agent auth is hard, but it shouldn't be! Your agent needs to act as someone: sometimes that's one shared identity, sometimes it's the person asking. To get either right you'll need an OAuth app for each service, a token store, refresh logic, and a consent flow Today we're fixing this with Managed Connections: define who your agent acts as in code, and the rest is handled for you: 𝚊𝚐𝚎𝚗𝚝 -> agent uses its own token 𝚞𝚜𝚎𝚛 -> agent uses the user's token, guiding them through the consent flow if they haven't connected yet With a single argument you avoid the whole OAuth dance. Available now, in 𝚖𝚊𝚗𝚊𝚐𝚎𝚍-𝚍𝚎𝚎𝚙𝚊𝚐𝚎𝚗𝚝𝚜 0.7
harnesses should make context engineering easy "forking" subagents is a useful context engineering trick that is now built into deepagents
RT Colin Francis subagents now support forking in deep agents! this gives your subagents the ability to start with all the relevant conversation history the supervisor agent has rather than starting blank. check it out ⤵️
New in Deep Agents: Subagents can now fork the supervisor's full conversation instead of starting from a blank context window. Everything you need to know from @t_bengre and @chester_curme on the @LangChain_OSS team ⤵️
View quoted postkai is very cool, and every company should have a kai this episode is a great resource on how stripe built it here's another blog for some additional context: https://www.langchain.com/blog/how-stripe-built-their-knowledge-ai-platform-on-deep-agents
The team at @stripe is setting the standard for internal AI platforms: minion coding agents, a custom prototyping rig, and now their company brain, Kai. On today's episode of How I AI, Sharadh shows us how 1.5 engineers and 2 weeks got them a company brain, including: -
View quoted postRT claire vo 🖤 The team at @stripe is setting the standard for internal AI platforms: minion coding agents, a custom prototyping rig, and now their company brain, Kai. On today's episode of How I AI, Sharadh shows us how 1.5 engineers and 2 weeks got them a company brain, including: - projects as governance - skill routing + telemetry - a skills platform that works for 10k teammates Plus, he and I debate the merits of gentle parenting your AI (esp when your company is running evals.) Full episode on YT: https://www.youtube.com/watch?v=AbZODZ_4VaM
What are the tools you’d want to see in “open router for tools”
Openrouter for tools is a great idea. I don't wanna sign up for 1000 services
View quoted postThis is very similar to how we think about context management in Deep Agents: compaction should reduce what is sent to the model, not erase the underlying work history. In Deep Agents, the logic is: - every agent gets a filesystem to work with (can be real or virtual) - when a tool result is too large, we write the full result to the filesystem (/large_tool_results) and replace the model-visible message with a preview - when the conversation gets long, we offload older messages to the filesystem (/conversation_history) history first, then generate a summary - the agent can also decide to compact proactively via a compact_conversation tool, which uses the same offload + summarize logic Summarization middleware with all this logic: https://github.com/langchain-ai/deepagents/blob/main/libs/deepagents/deepagents/middleware/summarization.py
This is GPT-6 Astra. Anything you can do on a computer, Astra can do for you. Fast.
View quoted postsmithdb is a database we built for agent trajectories from the ground up ankush has driven this from 0->1, we're now hiring a lead for it know anyone good?
We’re looking for someone to lead SmithDB at LangChain! SmithDB is the database we’ve built from the ground up to power LangSmith. It’s optimized specifically for the access patterns that emerge when you’re storing and querying enormous volumes of agent traces. It’s serving
View quoted postRT Ankush Gola We’re looking for someone to lead SmithDB at LangChain! SmithDB is the database we’ve built from the ground up to power LangSmith. It’s optimized specifically for the access patterns that emerge when you’re storing and querying enormous volumes of agent traces. It’s serving production traffic at significant scale with great performance, but it’s still early. The problems we are solving are genuinely fun and very hard: indexing, query execution, compaction, ingestion specifically for agent observability + making all of this fast and cost-efficient at massive scale. We’re looking for an exceptional engineering leader to take ownership of this critical project. You’re likely a great fit if you’ve built database or distributed systems, led strong engineering teams through technically demanding projects, and have exceptional execution and project management skills. Apply here or DM me directly: https://jobs.ashbyhq.com/langchain/d81c35c1-7243-45e4-bfe2-7fc8746c76b9 Learn more about SmithDB here: https://www.langchain.com/blog/introducing-smithdb
RT LangChain New from the LangSmith Signal. Over the last 2 weeks, we looked at which models teams reach for, and which ones are doing the work. ✅ Reach: gpt-4o-mini was used by 13% of orgs ✅ Work: gpt-4.1-mini sat at 7% of LLM calls 💡 DeepSeek V4 Flash: The only open-weight model to make either list, 2nd in reach at 9%.
RT @sydneyrunkle: we now support the new stateless MCP spec in @LangChain! this revamp includes moving MCP support into the main langchain…
This is very true, and something we still try to do for all products (and even features) we launch. Find a small group of customers who we think will push us the most on product and then build for them and make them happy
More on how early customers shape your business (@cursor_ai early work with Shopify, OpenAI, NVIDIA): "Not all ARR early on is created equal. It's not about just that number. It's about the genetics. I care far more that you've selected and are working with really thoughtful,
View quoted postAgent workspaces should be durable, inspectable, and swappable Deep Agents’ virtual file system is intentionally behind a BackendProtocol, so the agent code can keep using read/write/glob/grep while teams choose the storage layer that fits production Nice to see MongoDB support this!
MongoDB now powers the virtual file system behind @LangChain's Deep Agents. 🚀 This new integration lets you implement Deep Agents' BackendProtocol against MongoDB Atlas instead of building your own storage layer. And agent code calling read_file, write, glob, and grep don’t
we have an absolutely killer lineup for our Interrupt conference in NYC (sept 24th) in particular, im excited about the fireside chats we're doing with with founders at different parts of the agent stack: - @alexatallah: openrouter is a key part of how companies maintain model optionality and try out the newest models. they've also been innovative on smart routing and model councils - @bernhardsson: modal has one of the best DXs of any company i know, and plays a pivotal role at both the model inference layer as well as sandboxes (useful for coding agents, RL, and more) - @GabeStengel: rogo is the leading ai partner for financial institutions, and is a great example of ai native app companies in addition to these fireside chats, there are many more speakers (including langchain folks!) - so it will be informative and entertaining come join us! https://interrupt.langchain.com/nyc
RT Colin Francis openwiki 🤝 cursor we recently made openwiki more accessible than ever with coding agent integrations. our newest integration is cursor! two commands and you're ready to go: 1⃣ npm install -g openwiki@latest 2⃣ openwiki integrations install new integrations are incredibly easy to add. let me know what you want to see next! check it out ⤵️
Agent traces shouldn’t only be useful to infra teams For most builders, the fastest way to debug an agent is to replay the conversation + tool calls in the shape the agent experienced them That’s the idea behind Messages View in LangSmith
The new LangSmith “Messages View” dropped into my account (beta rollout) Makes following @LangChain traces as easy as following the chat!
View quoted postRT Sydney Runkle you can now build @LangChain agents w/ agentic video understanding! google is consistently at the forefront of multimodal processing; this is the video analog of "agentic vision" token usage + cost is also way down w/ this new approach! docs: https://docs.langchain.com/oss/python/integrations/chat/google_generative_ai#agentic-video-understanding
We’re introducing a new capability to our latest Gemini models: agentic video understanding. This allows developers to process long-form video content with more accuracy, while using up to 88% less tokens. See how it works 🧵
View quoted postDeep research is a complex, multi-step search and synthesis workflow This is exactly the kind of workflow Deep Agents is meant to make easier to build
A research agent is only as good as the search layer behind it! I built a Deep Research Agent using LangChain Deep Agents with a search-first architecture designed around grounded, source-backed research. The agent plans the research in real time, breaks the work into smaller
RT Colin Francis 🧵[1/3] openwiki v0.5.0 is out 🎉 q: what's worse than long running jobs that fail? a: long running jobs that fail and take all progress with it. wiki generation takes time and tokens. openwiki respects your time and tokens. the new architecture makes init and update durable and resumable: ➡️ completed pages are checkpointed as they finish ➡️ interrupted runs resume instead of restarting ➡️ partial progress survives ci failures ➡️ native openwiki and coding agent integrations all use the same resumable lifecycle
❤️ still early, lots to build, keep the feedback coming
I have been using langgraph for like a year and a half now and i can easily say the growth this company has had and the way they have listened to the community is quite remarkably unmatched. Awesome work @hwchase17
View quoted postwe're doing a meetup! "a summer evening on continual learning" partnering with our friends at prime intellect and baseten come join! https://luma.com/cwn8mze6
if only there was a company that helped you do both evals and harness engineering
Next to evals, harness engineering is quickly becoming one of the most important skills for AI engineers to have today.
View quoted postRT Sydney Runkle we are rolling support for the new MCP spec in langchain and need feedback on our new API! with this new pattern you connect langchain agents to multiple MCP servers, hook up oauth, use elicitation w/ langchain interrupts! we're building on top of FastMCP which is the easiest way to get started building MCP clients and servers https://github.com/langchain-ai/langchain/issues/40072
Call for agent infra
Request for infra: Who is going to build the open source version of the Codex-style in-app browser for agents. Said differently - Who wants to be an overnight hero?! @hwchase17 @danshipper @davemorin Call in the cavalry!
View quoted postRT Alex Request for infra: Who is going to build the open source version of the Codex-style in-app browser for agents. Said differently - Who wants to be an overnight hero?! @hwchase17 @danshipper @davemorin Call in the cavalry!
RT Caspar managed deep agents feels like magic. it handles the entire slack setup for you, and if you haven't picked an icon, it derives one from the agent's name. ~2M possible icons! inspired by dither-kit's DitherAvatar, thanks @grimcodes. https://langch.in/mda
cost reconciliation at the trace level For AI apps, "how much did we spend?" is less useful than "which workflow, tool call, prompt path, or retry pattern drove that spend?"
Clay uses LangSmith to reconcile 99.5% of its AI spend. Head of AI @jeffbarg on why cost observability finally clicked once LangSmith was in place.
View quoted postRT Dan Kornas Most agent tutorials stop before the hard parts. This one builds them step by step. Deep Agents from Scratch is a five-notebook LangGraph course for builders who want to understand how advanced AI agents are assembled. It helps you move from a basic ReAct loop to a research-agent workflow by implementing task planning, file-based context offloading, and sub-agent delegation in sequence. Key features: • Progressive notebook path – five lessons build from `create_agent` to a complete research agent • ReAct foundation – starts with the reason-act loop used as the base for the later lessons • TODO planning – adds status-tracked task lists to organize multi-step workflows and monitor progress • Virtual file system – stores information in agent state with read, write, edit, and list operations for context offloading • Sub-agent delegation – introduces specialized agents, focused tool sets, and parallel work for independent research streams It’s open-source (MIT license). Link in the reply 👇
Separation of model and harness One of the first commandments of agents building; don’t get locked in
We’re ending our partnership with Cursor following its acquisition by SpaceX. Under our proposal, Cursor’s direct access to our models would end on November 12. We know that the people most affected by this decision are the developers who rely on OpenAI models in Cursor. We care
View quoted postthe original and internal name for LangSmith Insights was CLIO, which was Anthropics name for the research they did that inspired us to build Insights now Anthropic has rebranded CLIO as Anthropic Insights 🤝 https://docs.langchain.com/langsmith/insights
Hi twitter, here is a lil explainer of Anthropic Insights, which is how we analyze usage data in a privacy-protective way. We just launched an exciting pilot enabling independent researchers to run studies with it (https://www.anthropic.com/research/enabling-independent-research) and have understandably gotten
View quoted postRT chris S. Nakamoto That is why I build using langchain.
Models labs will create great harnesses and ecosystems for their models but will block model access to harnesses of other labs Only choice for a harness that works across models is one not associated with a lab Long live LangChain
View quoted post“Pair this with the right observability” Langsmith 🔥
@hwchase17 Pair this with the right observability and the system is unstoppable
View quoted postRT Prakalp Choubey Re @hwchase17 Pair this with the right observability and the system is unstoppable
when software makes decisions at runtime instead of build time, a 200 OK can still be a complete failure. Here is why traditional SDLC will not work and we need a new ADLC. https://x.com/i/article/2093398626408304640
View quoted postModels labs will create great harnesses and ecosystems for their models but will block model access to harnesses of other labs Only choice for a harness that works across models is one not associated with a lab Long live LangChain
We’re ending our partnership with Cursor following its acquisition by SpaceX. Under our proposal, Cursor’s direct access to our models would end on November 12. We know that the people most affected by this decision are the developers who rely on OpenAI models in Cursor. We care
View quoted postFollow @colifran_ for all things wiki! Wikis are a great way to represent knowledge in a simple and easy to understand way
seems remarkably similar to langchain wiki? also interesting how everything is collapsing to some shape of a loop and instruction file
RT george salapa 🜁 seems remarkably similar to langchain wiki? also interesting how everything is collapsing to some shape of a loop and instruction file
Would love feedback on our support for the new mcp spec!
we're adding support for the new MCP spec in @LangChain open source! the new API is built on top of FastMCP, so you can take advantage of their excellent devx for building MCP servers and clients. an early version is available in langchain==1.4.0a2; let us know what you think!
RT LangChain In 13 minutes, @jeffbarg, Vyshu Khota, and Soroush Khadem walk through how Clay scaled agent evals agents at 300M+ runs a month. Topics covered: ✅ Their four quadrant eval framework ✅ Why closing the production-to-eval loop is the hardest part ✅ How a data lake and long context changed what agents can do with data
RT LangChain .@PodiumHQ's agents seemed broken. LangSmith showed the real story: the agent was behaving rationally based on the context it had. Principal Software Engineer Walker Ward on tracing agent reasoning end to end.
this is why benchling will create the best ai agent for science, not ant/oai strongly believe in a many-agent world, each one domain specific
openwiki now supports okf 2.0
the claims runtime in openwiki v0.4.0 gives your wiki a persistent source of truth to help it forget and self-correct. but how would a reader know: 1⃣ what sources a page was built from? 2⃣ if a page is fully verified? 3⃣ who generated the page in the first place? to answer
RT Colin Francis the claims runtime in openwiki v0.4.0 gives your wiki a persistent source of truth to help it forget and self-correct. but how would a reader know: 1⃣ what sources a page was built from? 2⃣ if a page is fully verified? 3⃣ who generated the page in the first place? to answer that we migrated openwiki to support OKF v0.2 the claims runtime powers page-level sources and verification and openwiki separately stamps who generated the page and when. claims=content-level truth okf=page-level trust special thank you to @seo_dev2142 for kicking off okf v0.2 support in openwiki!
RT Caspar 𝚍𝚎𝚎𝚙𝚊𝚐𝚎𝚗𝚝𝚜 is evolving
Deepagents is becoming a multiplayer harness Auth, memory, etc If you’re building an agent that you want to expose to multiple users in same thread and are thinking about these issues - reach out! Would love to chat
View quoted postRT Vapi We’re excited to announce that 𝗛𝗮𝗿𝗿𝗶𝘀𝗼𝗻 𝗖𝗵𝗮𝘀𝗲 @hwchase17, Co-Founder & CEO of @LangChain, will be speaking at VapiCon 2026! 🎙️ Harrison co-founded LangChain with the goal of making it easier to use LLMs to develop context-aware reasoning applications. LangChain also builds LangSmith, a platform for agent engineering. Prior to starting LangChain, Harrison led the ML team at Robust Intelligence and the entity linking team at Kensho. He also studied statistics and computer science at Harvard. Join us in San Francisco this November to hear from Harrison and other leaders shaping what’s next in AI and voice. 📍 Fort Mason, San Francisco 📅 November 11–12, 2026 🎟️ Early bird tickets are limited — get yours at https://www.vapicon.ai/
How we think about evaluating wikis Including whether wikis are actually even helpful!
RT Nick Hollon the evals grind continues! this time on OpenWiki. we built WikiBench to answer two things: 1. how good is a given wiki? 2. does the wiki help at all? for the second, we ran the same questions three ways: wiki only, source only, and both. read on to find out which won.
RT Colin Francis a question we were interested in answering for openwiki 0.4.0 was: how do we make openwiki available to more devs? our answer was simple: coding agent integrations! you can now drive openwiki init and update using claude, codex, and opencode with almost no set up needed: 1⃣ npm install -g openwiki@latest 2⃣ openwiki integrations install claude|codex|opencode 3⃣ "use openwiki to create a wiki for this repo" or "use openwiki to update my wiki for this repo" go try openwiki. it is easier than ever. watch me drive openwiki update using codex 👇
Deepagents is becoming a multiplayer harness Auth, memory, etc If you’re building an agent that you want to expose to multiple users in same thread and are thinking about these issues - reach out! Would love to chat
RT Viv word on the street is that Ben Franklin used LangSmith to mine his traces and turn them into environments to hill-climb agents for science
RT Benjamin Tannyhill Big step forward for Engine! Some things I'm excited about in this release that we've shipped for our most avid users: - Engine now points more obviously to the issue in a trace. Engine identifies and clusters errors, but now helps you more quickly confirm its findings by guiding you to places of interest in your agent's logs. - We find a lot of our customers living entirely out of Slack + Linear. Engine now sends alerts to Slack, and syncs with Linear tickets, allowing you to continue tracking issues in whatever system is best, use your coding agent of choice, etc. - Engine's proposed fixes are much more effective at solving the issues that Engine has identified. Our biggest users are using Engine not only to find problems but to resolve them quickly and avoid asking c-bro to TAL at yet another problem. - We've introduced Analysis Levels (Reduced, Standard, and Extended) so your team can get the "right" amount of Engine based on your budget. DM if you have feedback!
LangSmith Engine now offers >2x performance on key internal benchmarks. Engine has already helped identify tens of thousands of issues in our customers’ agents. Now it offers: ✅ More accurate issue detection, clustering, and remediation ✅ Support for SaaS and self-hosted
View quoted postRT LangChain LangSmith Engine now offers >2x performance on key internal benchmarks. Engine has already helped identify tens of thousands of issues in our customers’ agents. Now it offers: ✅ More accurate issue detection, clustering, and remediation ✅ Support for SaaS and self-hosted deployments ✅ Reduced Analysis mode for cost-sensitive customers ✅ Integrations with @slackhq and @Linear ✅ Automatic closing of stale issues Learn more → https://www.langchain.com/blog/new-in-langsmith-engine-2x-better-issue-detection
its relatively easy to generate a wiki the first time, but how do you update them reliably? openwiki v0.4.0 improves in this dimension - it "forgets" better
openwiki v0.4.0 just went live 🎉 the focus? forgetting, accessibility, trust 1⃣ stale docs detection with forgetting and self-correction 2⃣ drive init and update through claude, codex, and opencode 3⃣ okf v0.2 support for provenance and verification 🧵1/4
View quoted postRT Colin Francis openwiki v0.4.0 just went live 🎉 the focus? forgetting, accessibility, trust 1⃣ stale docs detection with forgetting and self-correction 2⃣ drive init and update through claude, codex, and opencode 3⃣ okf v0.2 support for provenance and verification 🧵1/4
RT Nick Hollon Re @Vtrivedy10 and I have spent a lot of time thinking about how to efficiently build evals and environments. our eval-engineering skill detailed here helps spin the environment generation loop very quickly!
trying to think about how to most easily create evals we launched a skill to help with this iterative process
RT Viv this is a practical guide (+ updated skill) on how we use real world data like traces + human feedback to make synthetic environments + evals so we can measure, harness engineer, and post-train our agents a few main components: - World Knowledge gathering of the systems our agent will interact with - Using Specs to coordinate what will be built —> World Spec & Task Spec - Human-Agent collaboration to iteratively edit a Spec and infuse human feedback - An execution pipeline that takes an agreed on Task Spec and generates an environment + task - Fixing design flaws in the task and environment by actually running agents in the environment & mining the verifier scoring + traces @harrison_chase, @nick_hollon and I spent a ton of time going back and forth on design decisions and the overall flow - how much a coding agent should prompt for human feedback - what specs do we need and what do they contain - how to create and update world knowledge this new flow is packaged in our updated eval-engineering skill. we want to help every team own their intelligence which means owning the pipeline to convert their valuable data into Tasks+Environments that improve their agent over time check out the skill and reach out if you’re thinking about this! https://www.skills.sh/langchain-ai/langchain-skills/eval-engineering
RT Eric Rea Agree 100%. We started building agents and quickly realized we needed to build the system of record too. Agents need all the context and permissions you’d give an employee to complete work end-to-end. It goes both ways: the system of record makes the agent powerful, and agents force you to rethink the system of record.
Prediction: systems of record will need to become AI harnesses or face replacement by agents
View quoted postEvals matter
OpenRouter is an LLM gateway, not a router. Some of the investment in routers may be missing the plot. 80% of the work in building a good router is building a good eval. The routing logic is the easy part.
View quoted postA hard part of agents is connections - seems easy but hard to get right/seamless Managed deepagents tackles it for you More to come!
Starting today, a @LangChain's Managed Deep Agent deploy provisions your Slack app for you 🤯 No manifest. No OAuth redirects. No bot tokens to copy around. One command, and your agent says 👋 in Slack.
View quoted postRT Caspar just released managed deep agents 0.6.0. we now automate the ugly parts of deploying your agent to slack. it's like terraform for agents
RT Christian Bromann Starting today, a @LangChain's Managed Deep Agent deploy provisions your Slack app for you 🤯 No manifest. No OAuth redirects. No bot tokens to copy around. One command, and your agent says 👋 in Slack.
RT will brown skills are code, traces are data, environments are data, benchmarks are code, configs are code, one-shotted apps are data
you always gotta be asking whether it's code or data. applies to everything. json. markdown. tweets. group chats. emails. blogs. calendar notifications. zoom meetings
View quoted postRT Christian Bromann An agent is 3️⃣ layers. 👉 Business logic you write. 👉 A harness that runs the loop. 👉 Infrastructure that survives production. @hwchase17 breaks the whole stack in one video. https://youtu.be/TUJmfeGTr1Q
RT Viv launching another update to this skill soon - open holy grail question if anyone wants to riff! what goes in “How-to-make-tasks-harder[.]md”? some options in a list below👇 part of making a good eval is making sure it captures the real world. another part is making the actual task hard so there’s something to hill climb one way to calibrate “hard” is running a weaker model and smarter model, if everything passes all the time that’s not great -> perfect pass @ k isn’t a good learning signal a list of things that can make tasks harder, would love to compile more and hear other strategies - add more data (not super bullish on this as models are great brute forcers) - add information that needs to be discovered by search (ex: look across multiple tables to find missing information) - completeness (add many cases that all need to be passed, this works well. it can incentivize bad behavior though like over checking) - cross-domain tasks that combine 2 abilities. this is the best imo, agents are not good at this but it can look artificial and toyish humans are still useful here but would love to chat on thoughts
RT Sydney Runkle last night i spent some time building a browser agent with stagehand from @browserbase and deepagents from @LangChain i wanted to build something that required pretty complex web navigation to prove the value of agentic browser use! my agent plays a game called "map tap" where you are assigned a city around the globe, and have to drag the map, zoom in, and click as close to the given city as possible. you get points based on how close you are. 1. one shotted the agent w/ browserbase + langchain docs, it scored ~300/1000 points 2. asked my coding agent to review the trace and make it faster (smaller model) and more accurate (higher score) 3. came back 10 minutes later to see a perfect score screenshot in my trace here's a guide on how to build your own browser agent! https://docs.langchain.com/oss/python/integrations/tools/stagehand
RT LangChain OSS What's new in LangChain? 🚀 🧯 Standard exceptions: chat models now raise standard exception types across providers, so you can distinguish retryable errors (timeouts, rate limits, etc.) in a consistent way. Fully backward compatible. 🧩 Agent middleware: custom token_counter in ContextEditingMiddleware, plus non-retryable exceptions now skip retries in ModelRetryMiddleware (http://github.com/syyy44, http://github.com/Yigtwxx) 🔥 Fireworks document reranking: a new reranker integration (http://github.com/11adyy) 📊 Token & usage accounting: bug fixes for xAI and DeepSeek (http://github.com/aryansk, http://github.com/nazsats) ⚡ Portability & perf: lazy transformers import, and tighter grep scope for Anthropic (http://github.com/jtoman, http://github.com/Haaaarry) This is your harness! Thanks to everyone who made it better. 🙏
Open bot!
🎉 Introducing 𝙾𝚙𝚎𝚗 𝙱𝚘𝚝 An open source Grok Bot that works with ANY agent harness, designed for real companies. It includes: - AI Coworkers - Generative UI - Computer use (remote/local) - Agent-human handoffs - Full data recording, owned by you Repo →
View quoted postRT LangChain We are excited to be hosting Alex Atallah, Co-Founder & CEO of OpenRouter, at Interrupt NYC next month on September 24th! See the agenda and get your tickets: https://interrupt.langchain.com/nyc
very cool launch we did a webinar with jeff on "wiki" style memory and it's clear he'd thought about this problem a lot (webinar here: https://www.youtube.com/watch?v=Lsut4TCfygw)
I’ve been looking forward to today for 3 years Today we’re announcing Foundation - Chroma’s solution to memory Our research preview of this technology builds self-improving memory from your agent sessions. Try it out at https://trychroma.com/foundation
View quoted postRT Christian Bromann Managed Deep Agents is just getting started 🚀 @hwchase17 just published a set for banger videos that get you up to speed how shipping production agents today looks like.
🎓New YouTube playlist: Managed Deep Agents Gives an overview of Managed Deep Agents, and then each video dives deep into core concepts. Launching with six videos! 1⃣ Intro: https://youtu.be/xdrB53bgpp0 2⃣ Conceptual Overview: https://youtu.be/TUJmfeGTr1Q 3⃣ Quickstart: