Parsing the world's hardest PDFs @llama_index. cofounder/CEO Careers: https://t.co/EUnMNmbCtx Enterprise: https://t.co/Ht5jwxSrQB
au revoir, Stainless Building production-level SDKs is extremely important for any developer product, and also really hard. This is a fun post from our engineering team on why there's value in tooling to help build good, maintainable SDKs. 1. You don't want your SDKs to look like an OpenAPI spec 2. You want to autogenerate per language 3. You want your APIs to be clean, deduplicated, explainable 4. You need to maintain the right documentation come check out the post! https://www.llamaindex.ai/blog/au-revoir-stainless
keeping our SDKs up to date can be a pain. every API change brings another round of updates, tests, and releases. and each SDK should still feel like it was written by someone who actually uses the language. Stainless helped us do that for LlamaParse. It also pushed us to
RT LlamaIndex 🦙 keeping our SDKs up to date can be a pain. every API change brings another round of updates, tests, and releases. and each SDK should still feel like it was written by someone who actually uses the language. Stainless helped us do that for LlamaParse. It also pushed us to improve the API itself. inconsistent names and schemas become harder to ignore when they show up in the code developers use. with the Stainless team joining Anthropic, George He and Yong Park wrote about what worked, what we learned, and why changing SDK generators takes more care than you might expect. -> https://www.llamaindex.ai/blog/au-revoir-stainless thanks to the Stainless team for saving us a lot of SDK work. 👋 au revoir, Stainless
i'm flattered by the shoutout, but there's no free lunch for document parsing Cohere parse starts at $1.50 / 1k pages, which is indeed on the cheaper end of doc OCR solutions, but it scores ~50% when averaged across all 5 ParseBench dimensions: * It lacks high-quality visual grounding, doing coarse segmentation for tables and images, but lacking finer-grained bboxes at the word/cell/line-level. this is quite important for any sensitive AI application that requires precise citations back to the source. * It lacks general visual / chart parsing capabilities, which are prevalent across finance, manufacturing, healthcare, and more * to its credit, it does a decent job over tables (87%) compared to other comparable solutions at the price point our cost-effective mode in LlamaParse starts at 0.375c / page and is more consistently strong across all dimensions. also there's various ways to bring it down with volume discounts if you're interested in learning more about various document parsing modes and comparing tradeoffs in price/accuracy, come talk to us! https://www.llamaindex.ai/contact LlamaParse: https://cloud.llamaindex.ai/
did a double take
Today, @jerrylu is joining @naturalpay as our first GTM hire after spending the last five years investing at @maveron. Your career is long. I first met Jerry in November 2022 while pitching Ivella. Different company, different vision, and, frankly, a very different level of
View quoted post“Just-in-Time” OCR The latest agent harnesses (e.g. Cowork, Codex, Grok) do 2 passes of ad-hoc document processing when executing over a user-uploaded ‘data room’ of documents: 1️⃣ A fast and light pass using a free/OSS doc parsing tool, can be cheaply run over 100+ docs 2️⃣ A “just-in-time” pass using a dedicated OCR tool (like LlamaParse) that does more accurate transcription over a subset of pages. This pattern is common enough that I wanted to give it a name: “Just-in-Time OCR”. It works surprisingly well over a small to medium batch of docs (e.g. 10-100 docs); it is accurate at the expense of cost and latency. It’s not a substitute for large-scale offline indexing and retrieval. The ‘default’ approach for any frontier agent harness is to use an OSS tool like pypdf for the first pass and its native VLM (e.g. Fable 5.1) as the second pass. This is both expensive and lacks accuracy and grounding. If you’re a builder looking to implement this type of harness of a document data room, you can try out our suite of tools: - Liteparse as the first pass (the best/fastest OSS text parser) - LlamaParse as the second pass (most Pareto-efficient, includes grounding) Check out the blog post below! https://www.llamaindex.ai/blog/just-in-time-agentic-ocr
just-in-time OCR is all the rage. most pipelines parse every page before anyone asks a question. for an agent working through an ad-hoc data room, that's slow, expensive, and most of those pages never get read. the better pattern is just-in-time OCR in two passes: ✅️
RT LlamaIndex 🦙 just-in-time OCR is all the rage. most pipelines parse every page before anyone asks a question. for an agent working through an ad-hoc data room, that's slow, expensive, and most of those pages never get read. the better pattern is just-in-time OCR in two passes: ✅️ LiteParse (free, OSS, Rust, 50+ formats) does a fast layout-aware first pass: spatial text, bounding boxes, headings, tables, and a per-page complexity flag. a full data room in 32 seconds. ✅️ LlamaParse zooms in on only the pages that need it, by page number, and returns cell-level tables, bounding boxes, and confidence scores. the rest fills in the background. pypdf and pdftotext can't do the first pass well. parsing everything up front can't do it cheaply. two passes gets you both. full breakdown with numbers: https://www.llamaindex.ai/blog/just-in-time-agentic-ocr
the biggest issue with trying to use all these personal assistants is having to reimport all of my context every time some of it can be made easier with shared storage (e.g. my file system), some tools have migrate functionality (e.g. codex import), but it's still a large amount of work i've tried/am trying codex, claude code, claude cowork, grok bot, instinct, muse....i ultimately want one assistant to rule them all
in a world where frontier models can generally do everything, it makes sense that every startup AI co is a "domain specific harness" * at an extremely basic level it helps for branding and distribution. you show that you care a lot more about a specific domain than the labs * re: tech you can define more domain-specific evals and hillclimb towards it with higher accuracy, lower cost, and lower latency vs. using the models * re: product you can define a more guided experience that's more intuitive to execute a workflow, store and manage data, etc.
I was just at YC demo day yesterday. Besides hardware/physical things, everyone is just basically just building a domain-specific harness
Introducing FlyOCR 🪰 - I trained a fly brain to read a PDF It uses the full MaleCNS v1.0 fruit fly connectcome. The architecture is inspired by doomfly by @nftechie_ The fly splits a pdf image into individual glyphs, maps pixels into receptor activations, runs simplified current-based dynamics across the 166k neurons and 25m edges in the circuit, applies a compact readout model on the downstream spikes, and concatenates everything into the parsed output. On reading an actual Microsoft 10-k, the fly gets ~86% over the balance sheet heading, but is largely able to read the numeric values correctly. Over 1.7k+ sampled glyphs (chars+digits) it gets 87% accuracy. With enough training it might match some of the latter-generation MNIST models! Maybe eventually we’ll replace our doc parsing VLMs with flies. Full video below. Repo with full code + report: https://github.com/jerryjliu/fly_ocr
imagine bringing the gaokao to american schools (it's 9 hours long, no student has ever gotten a perfect score, and is the only determinant of where you end up in college in China) it would cause absolute pandemonium
TBH once you score 1600 you should unlock a second harder test that gives you a second score on top of the 1600— we should have more and higher measures of excellence But instead they ban the SAT so that everything turns into random lottery and excellence cannot be recognized
View quoted postLast Friday night, our team hosted a lit event at the US Open 🎾 we gathered a group of enterprise and startup AI leaders + founders together to drink, talk about AI and document parsing, and watch tennis until 2am. we've got to do this every week and bring the energy back to SF. stay tuned for more events!! (I say 'we' in third-person, I had to stay back because I got covid 😭)
@llama_index hosted an executive suite at the US Open in NYC on Friday night and we shared the night session with AI Leaders building some of the most document-intensive AI apps 🎾 Our team got to watch Venus and Serena play doubles together (surreal to see the sisters out
There's something deeply philosophical about the importance of human understanding in a world where most work is done by AI. Terence Tao's whole point in the letter isn't that AI is universally bad, but that it should be used in a way that complements the advancement of human understanding rather than doing work (like solving math proofs) for its own sake. You can obviously use AI to help accelerate research and understanding, but the point here is that you also don't have to in order to get things done. As someone who's largely optimistic about AI, I don't have concrete opinions on this matter yet, but I empathize with the broader question of how deeply humans should strive to understand the world vs. fully offloading it to others.
Twenty-five Fields Medal winners have published a joint declaration warning about what they see as a severe misalignment between AI companies and the mathematics community.
One of the main issues with AI document parsing is that because no solution is 100% accuracy, it's hard to tell if a given page is parsed incorrectly: for instance a misaligned table, missing/hallucinated values, or messy scans over the past few months we've invested a lot in models that provide more calibrated confidence scores - it will give you a confidence level that the page is parsed correctly. it is correlated with the parsing mode and how complex the source page is. this allows you to bake in human-in-the-loop mechanisms where you can review and correct the outputs, or trigger automated fallback logic, for highly sensitive paperwork-heavy processes. come check out our high-effort scores! https://developers.llamaindex.ai/llamaparse/parse/guides/configuring-parse/#confidence-score-high-effort Come sign up to LlamaParse here: https://cloud.llamaindex.ai/
confidence scores in LlamaParse just got an upgrade 🦸♀️ our new high-effort mode provides granular page-level scores with text explanations to help you intimately understand parsing quality of your documents. we even refer back to the original document for an extra check when
RT Imaan Sultan 🇵🇰🇸🇦 since there's 800 of yall now, thought i'd do a re-introduction 🌟 hi i'm imaan!! 👋🏽 - high-maintenance eldest daughter - i do everything brand, growth hacking, community, and yapping @llama_index - started a social club called dosti that has 1.6K+ ppl - love fashion, art, interior design, f1 - proud brown girl raised in saudi arabia 🇸🇦 - always trying new coffee shops - need more female friends in sf 😔 ok thank u for coming to my ted talk !! <3
RT LlamaIndex 🦙 confidence scores in LlamaParse just got an upgrade 🦸♀️ our new high-effort mode provides granular page-level scores with text explanations to help you intimately understand parsing quality of your documents. we even refer back to the original document for an extra check when generating the score. use high-effort only when you need it, at 5 additional credits per page. try it on your docs: https://developers.llamaindex.ai/llamaparse/parse/guides/configuring-parse/#confidence-score-high-effort
agreed, one of the biggest bottlenecks right now is my own ability to internalize the information that these models convey the latest models (astra, fable 5.1) are either way too verbose, or try too hard to pack information density to be concise it comes up with ai-coded jargon that's in to deciphering a math proof
claude and codex still use way too many words it's exhausting just reading all the word vomit there has to be a better way
View quoted postAstra is one of the best models we've tested at parsing complex documents with tables, at 93.2% on ParseBench. So it seems well suited for ad-hoc analysis over table-heavy documents e.g. in finance. To be fair, it's also at 10c per page so not suitable for document-heavy OCR workloads. Fable 5.1 still performs a better at charts. Check out the full set of results on ParseBench: https://www.parsebench.ai/
We benchmarked GPT-6 Astra on hard document parsing and extraction tasks. It is the best frontier model for one-shotting document extraction over short and medium-sized documents: 📊 97.2% over short documents, achieving a new SOTA on our benchmark 📊 90.6% over medium
Lightning-fast extraction over any document. ICYMI, we launched Extract Turbo last week, that lets you process documents from 1 page to dozens of pages with lightning-fast speed. Come check it out! Check out the video sequence below on equipment orders. Get all the line items with bounding boxes/citations in seconds. (disclaimer: astra sped up the video a bit so it's not quite 1x, but it should still complete in seconds!) https://cloud.llamaindex.ai/
Document extraction just got a new gear. ⚡️ Introducing Turbo mode for Extract, our fastest way yet to pull structured data from documents. It runs roughly 4× faster than our Cost Effective Tier at comparable accuracy, with a median latency of 3.7 seconds a page. Turbo
We've trained new models for checkbox parsing ☑️ This lets you convert checkboxes of any size and shape, filled or unfilled, into structured JSON that's suitable for programmatic or agentic use. Use it on insurance applications, tax forms, compliance questionnaires, KYC forms, and more. Check it out in our forms mode in LlamaParse! https://cloud.llamaindex.ai/
Our agentic extraction harness can extract 54+ fields from an insurance policy with 100% accuracy ✅ coverage limits ✅ nested sublimits ✅ explicit noncoverage This includes a full result review UI where you can audit every output, see which values have lower confidence scores, and see the source bounding box for every single field. Check out the video below, using the "Agentic Plus" mode in our Extract API. It's a great option for extracting a massive amount of output tokens from any-sized document, whether it's 10 pages or 1k pages. If you're interested in giving it a spin, come check out LlamaParse: https://cloud.llamaindex.ai/
Astra is awesome, but if you're using it specifically to parse and extract a massive volume of documents, you will break the bank (10c+ per page on parsing and extraction!) We've added a native LlamaParse connector to ChatGPT that gives the model access to high-quality document OCR and extraction at the pareto frontier of accuracy and cost. It contains native access to our parse and extract endpoints. Upload your most complex documents, and you can do the following: * digitalize it into extremely accurate context for any QA task * perform large-scale extraction from a set of documents with accurate grounding and citations Offload the higher-level reasoning to Astra, leave specialized doc parsing and extraction to us. ChatGPT plugin: https://chatgpt.com/plugins Come sign up here: https://cloud.llamaindex.ai/
Last week we brought LlamaParse to Claude. Now it’s available in ChatGPT, too 🦙 ChatGPT can already read PDFs. But document work gets harder when the information you need is buried in messy scans, dense tables, spreadsheets, and charts. Strike the right balance between burning
View quoted postRT LlamaIndex 🦙 Last week we brought LlamaParse to Claude. Now it’s available in ChatGPT, too 🦙 ChatGPT can already read PDFs. But document work gets harder when the information you need is buried in messy scans, dense tables, spreadsheets, and charts. Strike the right balance between burning tokens and accuracy, with a specialized API for document understanding. Turn complex documents into clean, structured context and let Astra run its magic. Use our connector to: ✅ Parse documents into clean Markdown, JSON, or HTML. ✅ Extract specific fields into a schema you define. ✅ Search across document collections to find the information you need. ✅ Classify files and split documents into logical sections. Live in the ChatGPT plugin directory now! → https://chatgpt.com/plugins/plugin_asdk_app_6a8c892f334c8191a56023560ed46b4b?q=llamaparse
it is 2am ET and they're in the beginning of the 4th set, holy
we're giving away arsicault, $3k in llamaparse credits, and a limited edition swag box 🎁 if you're building something that depends on processing high volumes of unstructured documents, let us know!
happy to announce i've become an arsicault chocolate almond croissant dealer! (just for today and tomorrow.) 😁 doing a massive drop-off of pastries, $3000 (!!) in llamaparse credits for each team, and a ton of cool merch + goodies i designed for @llama_index . 🌟 if you'd
RT simon holy shit
We’re sharing a solution to the Navier-Stokes Millennium Prize Problem, one of the deepest problems at the frontier of mathematics. The proof was produced by a group of agents, using an OpenAI next-generation model significantly more capable than GPT-6 Astra. The problem
RT Imaan Sultan 🇵🇰🇸🇦 happy to announce i've become an arsicault chocolate almond croissant dealer! (just for today and tomorrow.) 😁 doing a massive drop-off of pastries, $3000 (!!) in llamaparse credits for each team, and a ton of cool merch + goodies i designed for @llama_index . 🌟 if you'd like me to stop by your office to deliver some sweet treats today or tmrw, PLS shoot me a message or comment :D would love to come say hi to your team and learn about what you're building 💌
RT Imaan Sultan 🇵🇰🇸🇦 ‼️are you a yc / a16z speedrun founder that wants free goodies that aren’t a tote bag? 🌟with demo day coming up, @llama_index is dropping a bunch of custom boxes tmrw and day after at offices!! lots of goodies but we’re also giving out $3,000 in credits to every team to try out llamaparse on your docs 🙈 if you want me to come to your office, shoot me a dm and i’ll pull up and say hi :) p.s. i come bearing sweet treats
RT Logan Markewich Easy to hate on a place you've only seen online or from the highway. Sask has a lot of gems if you go touch grass
Nathan's just prepping for The Rehearsal season 3
YOU CAN SEE EVERYTHING, a documentary by Nathan Fielder and Lance Oppenheim. Featuring Elizabeth Holmes. Only in theaters this October.
View quoted postcorrection: the switching costs weren't enough to stop my desire to try out gpt-6 astra i ported over all my skills and conversational context to codex this weekend. planning to use it as my daily driver for now if the new model is sufficiently good enough, the switching costs for early adopters might continue to be 0 😅
i think there's a real opportunity for labs to bake in even higher switching costs between claude code/codex/grok bot etc. whenever a new model release comes out, i already feel an inertia to switch between apps because of my existing skills, routines, system instructions,
View quoted postAstra might finally help me automate what I've spent ~hundreds of hours doing since the start of this company: Making live product demos through Loom or Screen Studio Check out the video walkthrough on our new Agentic Plus mode for document extraction - where we're able to populate 60 fields worth of financials from a sample earnings release. This entire video was fully generated by Astra. With its default set of tools it can already login to my LlamaParse account and piece together a set of slides with static screenshots. I then gave it full access to computer use and Chrome, and let it try out Loom and Screen Studio on my desktop and record the current chrome session. In the meantime, if you do have use cases for gnarly extraction from extremely complex documents, come check out our LlamaParse product demo below 🙂 https://cloud.llamaindex.ai/
RT simon I have crazy cognitive dissonance on this. It seems simultaneously completely natural that we arrived here, and also absolutely insane at the same time
Unlike other players in the OCR space, we don't block our competitors from accessing our platform. We're also fully committed towards open, reproducible benchmarks for document parsing and extraction - on ParseBench and ExtractBench respectively. It's important to give the industry a sense of how the pareto frontier of accuracy and cost evolves over time. ParseBench and ExtractBench are fully open, maintained benchmarks that we're constantly adding every day (e.g. 3.8 flash, fable 5.1, r-1) Sometimes we make mistakes on benchmarking, and we take into account your feedback on how to fix and improve the results! You can access both benchmarks from the links below, which also give you access to Github/Huggingface and more. ParseBench: https://www.parsebench.ai/ ExtractBench: https://www.extractbench.ai/ In the meantime, if you're looking to ship doc OCR workloads to production, we'd be happy to help you benchmark LlamaParse against other solutions you're considering over your data and compare the results. Sign up here: https://cloud.llamaindex.ai/
RT LlamaIndex 🦙 Does paying 5x more per page actually get you better document extraction? We ran the data to find out. We evaluated 14 frontier systems across 370 enterprise documents, then plotted their accuracy against cost per page. The biggest takeaway? Higher cost does not equal better extraction. • Agentic Plus hit the highest accuracy overall at <1/3 the cost of the runner-up. • Agentic & Cost Effective routinely beat systems costing several times more per page. More expensive doesn’t necessarily mean more accurate. Now we have the data to show it. Try Extract on your own documents with 10,000 free credits when you sign up for LlamaParse → https://cloud.llamaindex.ai?utm_medium=socials&utm_source=twitter&utm_campaign=2026-aug-
me waiting for GPT-6 Astra access on my ChatGPT Pro subscription
This is GPT-6 Astra. Anything you can do on a computer, Astra can do for you. Fast.
View quoted postI trust these vibe checks more than I trust most of the benchmark results GPT-6 astra seems more powerful for e2e automation and better at writing (a massively underrated gap for these models). But it also seems harder to steer
BREAKING: OpenAI just dropped GPT-6 ASTRA!!! 🚀✨ We’ve been testing it extensively at @every across coding, writing, and knowledge work. My take: it’s a big upgrade from 5.6-Sol, with some frustrating habits that keep it from matching Fable at the top end. Here’s your vibe
View quoted postWe got GPT-6 before GTA-6
This is GPT-6 Astra. Anything you can do on a computer, Astra can do for you. Fast.
View quoted postRT LlamaIndex 🦙 Document extraction just got a new gear. ⚡️ Introducing Turbo mode for Extract, our fastest way yet to pull structured data from documents. It runs roughly 4× faster than our Cost Effective Tier at comparable accuracy, with a median latency of 3.7 seconds a page. Turbo processes pages in parallel, so latency stays nearly flat as document size grows. Available in beta today. Read the full breakdown → https://www.llamaindex.ai/blog/introducing-turbo-our-fastest-extraction-tier
Vendor-neutral startups are uniquely positioned to solve any given task better than the frontier labs. They have access to the full set of open-weight and frontier models. They can still use the frontier models where needed, but can optimize the harness e2e for any given task. They can even posttrain the model itself because they have the domain expertise to create specialized evals.
What is required to get the most out of models? “To take the most advantage out of models, you need to do something a little different from just routing. You need your agent or system to dynamically understand the task it is working on and understand how to allocate
View quoted postRT Imaan Sultan 🇵🇰🇸🇦 monopoly monday (shoutout to @alameldinkhaled @bigdogbower @stephen_revamp for bringing us all together 💌) my fav part of growth events is that you meet new people and suddenly find yourself in situations like eating yemeni mandi in the mission (pic attatched)
We've benchmarked all the latest frontier models on hard document extraction tasks (with Fable 5.1 and Gemini 3.8 Flash coming soon!). ExtractBench is now live on @kaggle . 5.6 Sol leads the pack, followed by other OpenAI models, then Gemini 3 Flash and Opus 5. Come check it out! https://www.kaggle.com/benchmarks/llamaindex-org/extractbench-leaderboard/leaderboard If you're looking for a dedicated extraction tool with out of the box grounding, citations at the price/performance frontier, come check out LlamaParse: https://cloud.llamaindex.ai/
ExtractBench is now live on @Kaggle. It tests schema-guided document extraction on the documents most likely to break downstream agents and workflows, including long record lists, noisy scans, handwriting, and complex tables. The benchmark covers 370 enterprise documents across
Most document OCR solutions have parse (doc->markdown) and extract (doc + schema -> structured output) endpoints. Sometimes you want to extract information from a semi-structured document like a form, without spending the work defining on defining the exact schema. This is where LlamaParse form mode comes in. Simply upload any form, partially filled in or not, and we’ll not only extract out the markdown, but give you key-value pairs on the precise form fields and values. Docs: https://developers.llamaindex.ai/llamaparse/parse/examples/enriched_forms/ Signup to LlamaParse here! https://cloud.llamaindex.ai/
welcome to the team @imaanxsultan !! 🦙
SO excited to share my next chapter: i’m joining @llama_index to lead all things growth marketing, brand, and community! 🦙🩷 when i came along them, it clicked incredibly fast: a team building the solution that made life SO much easier for developers, startups, enterprises, and
Fable 5.1 didn't come with document OCR benchmarks, so we benchmarked it on ParseBench: a comprehensive parsing benchmark that's comprised of 2k+ real-world documents across finance, legal, insurance, and more. As a positive note, it is one of the first frontier models in recent memory where document OCR performance actually improved between model generations! ✅ It beats Fable 5 across the board on all of our metrics, from tables/content faithfulness/formatting/charts/visual grounding ✅ It is one of the best models out there for parsing tables This is great, because while frontier models have rapidly advanced on reasoning benchmarks, they have not advanced on visual understanding. Prior to this the latest models (Opus 5, 5.6-Sol, 3.7 Flash) have stagnated on visual understanding tasks. Obviously you shouldn't use Fable 5.1 as a document OCR tool on its own; besides missing needed metadata, it also costs 15c a page - which is 10-15x more expensive than our default agentic mode on LlamaParse (which performs similarly). If you're interested in tracking the 90+ solutions that we've benchmarked on doc OCR, check out ParseBench: https://www.parsebench.ai/ If you're looking for a dedicated, SOTA OCR tool to power your doc processing workloads, check out LlamaParse: https://cloud.llamaindex.ai/
RT Imaan Sultan 🇵🇰🇸🇦 SO excited to share my next chapter: i’m joining @llama_index to lead all things growth marketing, brand, and community! 🦙🩷 when i came along them, it clicked incredibly fast: a team building the solution that made life SO much easier for developers, startups, enterprises, and everyone in between. in case you’re wondering, “wtf does llamaindex do?” we're solving one of the biggest problems in infra to date. the world runs on documents, yet 90% of enterprise data is unstructured, disconnected, and inaccessible to your favorite models. that's what we exist to solve, and our tech continuously tops on benchmarks (and we've processed billions of pages to date). stay tuned for a ton of insane events, growth hacking, community building, and much more. if you're not using us yet, you will be very soon 👀 p.s. @jerryjliu0 is a natural at selfies
Yesterday I hosted a founder dinner with @GuangyuRobert from @Fundamental (the company behind @tryshortcutai) to talk about a key topic: How to build a moat as an AI startup in 2026 We didn’t arrive at a crystallized answer, but we touched on a lot of interesting conversation threads: ✅ While some companies were building products that were a bit more adjacent to the frontier labs (Shortcut, maybe us with document OCR), other companies were deeply focused on adjacent domains, from evals to infrastructure to vertical-specific services. So then the question evolved into not only how to compete against frontier labs, but how to build a moat against any startup ✅ It’s increasingly easier for any startup to duplicate your business, with everyone using AI and AI coding tools. There’s a question of whether everyone can split the pie, in which case businesses might make good money but not capture the entire market opportunity, which makes it less appealing for venture. Or can you truly build an outlier business. ✅ There’s a line of thinking that the main moats are not related to the inherent nature of the business strategy or product, but mostly around money, talent, execution. Get in first, hire the best people, raise enough $$, and it’s structurally hard for others to compete. ✅ On the other hand, there can be some legitimate moats around: - Domain expertise in the given space - In general, the context layer that’s locked up within an organization and not inherent to a model - Deep-tech investment that’s hard to vibe-code - Vendor-neutrality for application-layer companies where they can use any mixture of open-weight, posttrained, and frontier models to optimize for a given task. It was a fun conversation. The most important bit is to keep your head straight, find the fulfillment in the journey itself, eat healthy and exercise 🙂 Thanks to everyone who participated! This was founder dinner 002. We hope to do this every month and are hoping to host founder dinner ...
There’s generally a massive difference in quality between specialized OCR providers, “simple” open-weight OCR models, and free/OSS solutions. 1️⃣ Specialized OCR models (including LlamaParse) solve for the long-tail of complex documents, and make sure that everything is digitalized properly with lower hallucinations. They typically use posttrained VLMs to cover a wide range of real-world docs. They have tuned bounding boxes and annotations for each section, letting agents trace citations back to the source. They also usually come with additional endpoints like extraction and splitting. 2️⃣ Open-weight VLMs (e.g. Paddle, MinerU, UnlimitedOCR) are reasonable over relatively simple documents like text and tables and can do basic visual reasoning. They can seem somewhat cheap to host but can be unreliable in quality. 3️⃣ Free OSS libs (including liteparse) are meant to be universally accessible, fast text extractors. They’re not meant to do any sort of visual reasoning, so won’t perform any linearization, or reasoning over complex tables, or OCR over non-native docs. AI agents like Claude will by default use these tools to do a light pass over documents. But I would caution using these for retrieval, because they will drop entire sections that are not digitalized. At this point we’ve benchmarked over 92 tools on ParseBench. Come check it out! https://www.parsebench.ai/
We’re excited to launch the official LlamaParse connector on Claude! Yes Claude can already read PDFs, and it probably works fine for relatively simple queries over relatively simple docs. But if you want to: ✅ Parse complex tables, graphs, filled-in dense forms, redlines from docs ✅ Include visual citations/bounding boxes for auditability ✅ Extract a *lot* of docs (e.g. 1k+) at scale without breaking the bank Then you’re going to want a dedicated OCR tool that can handle any document at the Pareto frontier of accuracy and cost. Install the connector here: https://claude.ai/directory/llamaparse If you haven’t already, come sign up for a LlamaParse account! https://cloud.llamaindex.ai/
LlamaParse is officially a verified connector on Claude! 🦙 Claude is perfectly fine for the occasional text-only PDF. But most of the context your agent needs lives in messy PDFs, spreadsheets, scanned forms, dense tables, and charts. Feeding those docs straight to the model
View quoted postThanks @juliafedorin for a great conversation! Some tidbits from the podcast on the AI space: - We've pivoted from building a RAG framework towards building the document infrastructure for agents - Documents are 90% of the world's unstructured context, and solving the context layer is the most important bottleneck towards unlocking general intelligence - There's a lot of ways to hack building basic RAG, but building production retrieval systems is still hard
My conversation with @jerryjliu0. He's building @llama_index: document infrastructure for AI agents, the layer underneath everything models need to work. Did you know he's the guy whose tool half the industry used for RAG? Then he called RAG a hack, two years before the field
View quoted postwe're hosting a founder dinner on this exact topic tonight! is the moat in: - product - data - brand - workflows - network effects - working harder than everybody else - anything else? Event here: https://luma.com/llamai-8hry even if you can't make the dinner, we'd love to get your general thoughts. we'll also be hosting more events like this on a regular cadence!
there are no more moats anymore. moats are dead. was talking to a friend of mine who's a CEO of a rapidly growing startup (~$1B revenue). we're small investors. He told me that he has only taken one one-week vacation in the last eight years of his startup. I was like, "Hey this
View quoted postRT LlamaIndex 🦙 LlamaParse is officially a verified connector on Claude! 🦙 Claude is perfectly fine for the occasional text-only PDF. But most of the context your agent needs lives in messy PDFs, spreadsheets, scanned forms, dense tables, and charts. Feeding those docs straight to the model burns through tokens and can lead to hallucinations when layouts get scrambled, chart values are missed, or numbers are pulled from the wrong column. LlamaParse converts those documents into clean, structured context Claude can work with. With the connector you can ✅️ Parse documents into clean Markdown, JSON, or HTML ✅️ Extract specific fields directly into your schema ✅️ Search across document collections with filesystem-style tools ✅️ Classify documents or split them into predefined sections Live in the Claude Connectors Directory now ! https://claude.ai/directory/llamaparse
RT Julia Fedorin My conversation with @jerryjliu0. He's building @llama_index: document infrastructure for AI agents, the layer underneath everything models need to work. Did you know he's the guy whose tool half the industry used for RAG? Then he called RAG a hack, two years before the field agreed. Here's the problem he's pointing at: 90% of the world's information is locked inside PDFs, PowerPoints, and Word documents. The models are already smarter than humans. But until an agent can read and reason over all that, it just sits still. RAG was the hack everyone reached for to bridge that gap: chunk, embed, retrieve, done. The easiest thing that kind of worked, so people could ship demos. Anything complex, and it falls apart. We got into: •Why RAG has no first principles, and what's replacing it •Why context, not intelligence, is the real bottleneck now •Why even MCP doesn't fully solve context •How you get agents to follow a company's culture, not just its instructions •What he'd build if he started LlamaIndex today 𝐓𝐈𝐌𝐄𝐒𝐓𝐀𝐌𝐏𝐒 (00:00) The Context Bottleneck (02:00) One Commit, One Tweet: The Origin (05:00) Feeding GPT-3 Your Private Data, Pre-RAG (08:00) From Quora to Uber's Self-Driving Lab (11:00) Meeting His Co-Founder at Uber ATG (14:00) Document Infrastructure for AI Agents (18:00) "RAG Is a Hack" — What He Meant (22:00) Why Production Search Is Still Unsolved (24:00) How a Research Background Changes How You Build (33:00) The Open Problem: Context Across Sources (38:00) Specs, Cultural Context, and Underspecified Tasks (40:00) What He'd Build Differently Today
I had to write Java during my junior year internship in 2016. And my manager asked me what IDE I was using. I told him, I'm using vim with no plugins. My whole life, I wanted to see if I was a badass or not. Am I a fake badass that coded in IntelliJ / NetBeans / VSCode, or am I really a badass on the inside?
Sequoia partner Doug Leone reveals he got a root canal with no Novocaine to see if he was really a badass "I had to get a root canal. And I had to go to a meeting right after that. I told the dentist, drill without any Novocaine.” “My whole life, I wanted to see if I was a
View quoted postat the end of the day, we all have to stand by our company and the values it represents 🙂 * frontier labs will want to own their own application/harness/model e2e. or at least, openai will to start with, and then anthropic will do the opposite of whatever openai does * everybody else will push for democratized access to a wide mixture of proprietary and open-weight models, so that we can optimize performance and margin for any given task and we're not beholden to the whims of the provider (this of course includes us with LlamaParse and any other startup building verticalized agents)
We unfortunately have decided that we cannot continue providing access to our models through Cursor and are ending our partnership. It boils down to trust and we’ve asked that this takes effect on November 12 to give you some time to plan. Many have used the GPT models through
View quoted postopenai's hatred for elon must've outweighed any aura loss this would cause them
We’re ending our partnership with Cursor following its acquisition by SpaceX. Under our proposal, Cursor’s direct access to our models would end on November 12. We know that the people most affected by this decision are the developers who rely on OpenAI models in Cursor. We care
View quoted postAgentic spreadsheet extraction 📊 is hard to execute well from a price/performance standpoint because your agent needs to ingest a massive quantity of input tokens from a highly complex interface (e.g. the Excel file format), and emit a massive number of output tokens that accurate represents the source document. We've tuned an agentic harness within LlamaParse that can extract from even the most complex spreadsheets and schemas, and emit a large volume of structured output with extremely high accuracy. It can be used for any document, and now it can be used for spreadsheets too! Simply toggle "spreadsheet extraction" in the advanced processing options (see screenshots below) Come check it out: https://cloud.llamaindex.ai/
We've introduced native, agentic spreadsheet extraction 📊 into LlamaParse. Spreadsheets are a wildly different format from PDFs (or any other document format). They can span arbitrarily many rows/columns, lack any guarantees on tabular structure, and can have information
RT Logan Markewich Obligatory plug for static embeddings. Are you using static embeddings in your projects? Would love to hear how people are leveraging them, whether for retrieval and beyond.
Static embeddings offer unmatched throughput, but they also suffer from accuracy loss compared to traditional embedding models. Can we make them better for retrieval? We tried several things: ✅️ raw maxsim scoring on the per-token embeddings ✅️ training a small adapter
static embeddings are extremely fast/cheap compared to dense embeddings, and still quite powerful. one issue is that they lose accuracy over longer text, since the tokens get pooled and averaged late interaction models score with MaxSim instead of average pooling, and avoid the pooling accuracy degradation. Can we apply these concepts to static models? This is a fun writeup by @LoganMarkewich - come check it out! https://www.llamaindex.ai/blog/exploring-static-embedding-retrieval
Static embeddings offer unmatched throughput, but they also suffer from accuracy loss compared to traditional embedding models. Can we make them better for retrieval? We tried several things: ✅️ raw maxsim scoring on the per-token embeddings ✅️ training a small adapter
RT LlamaIndex 🦙 Static embeddings offer unmatched throughput, but they also suffer from accuracy loss compared to traditional embedding models. Can we make them better for retrieval? We tried several things: ✅️ raw maxsim scoring on the per-token embeddings ✅️ training a small adapter model ✅️ changing the distillation training target and teacher While none of these saw the results we wanted, it's an excellent dive into static embeddings and how they do (and don't) work! Blog: https://www.llamaindex.ai/blog/exploring-static-embedding-retrieval
The hardest road paid off
Excited for the new release of #HuggingFace http://durant.ly/huggingface - proud investor! http://durant.ly/hfrelease
View quoted postPDF parsing is fun because there's an infinite variety of enteprise documents 📑. For each document category, there's a long tail of work to build more precise bounding boxes, confidence scores, and domain-specific annotations so that you provide any downstream agent rich metadata without it having to reinvent this from scratch. Take forms for example. Besides simply outputting it into markdown, we put in the work to detect every annotation, field, checkbox, and section. That way you immediately get structured information as to whether a form is filled without a separate LLM extraction step. You also get source citations for free! Doing this well is hard. ✅ There's a rabbit-hole of optimizations you can do, including extracting annotations for every document type. ✅ It's hard to properly render visual formats like charts, handwriting into digitalized information. The more you skip this step, the more work you're creating for any downstream agent. ✅ Precise bounding boxes are a necessity for precise citations on any type of document. You can aggressively tune the model + harness so that the accuracy/cost on any document subtype is much more competitive than the frontier models. Whether you're parsing forms (see the enriched forms option in "processing options") or any other type of doc, check out LlamaParse ! https://cloud.llamaindex.ai/
We've introduced native, agentic spreadsheet extraction 📊 into LlamaParse. Spreadsheets are a wildly different format from PDFs (or any other document format). They can span arbitrarily many rows/columns, lack any guarantees on tabular structure, and can have information linked across multiple sheets. The native ways to deal with spreadsheets is usually through a code interpeter as opposed to OCR. We've introduced a tuned agentic engine (model+harness) that is equipped for large-scale schema guided extraction from spreadsheets. You can extract out dense sheets like balance sheets into clean structured fields. Check out the docs for how to integrate into the API: https://developers.llamaindex.ai/llamaparse/extract/guides/configuring-extract/#spreadsheet-mode On the UI, select "agentic plus", and see "spreadsheet options" in the advanced options section. Signup for LlamaParse here! http://cloud.llamaindex.ai/
Most extraction tools treat spreadsheets like PDFs. They flatten the file into text or markdown, then ask a model to infer the original structure. But spreadsheets depend on structure. Headers, formulas, merged cells, and hidden rows give every value its context. Strip that
RT LlamaIndex 🦙 Most extraction tools treat spreadsheets like PDFs. They flatten the file into text or markdown, then ask a model to infer the original structure. But spreadsheets depend on structure. Headers, formulas, merged cells, and hidden rows give every value its context. Strip that away, and you map the right number to the wrong metric or period. That's why we built native spreadsheet extraction into the LlamaParse platform. Instead of flattening your workbook to text, it reads the raw cells directly and maps the data to your schema. Available today in beta on the agentic_plus tier. Give it a spin on your messiest .xlsx, .xls, or .csv files. Docs: https://developers.llamaindex.ai/llamaparse/extract/guides/configuring-extract/#spreadsheet-mode
We benchmarked 20+ open-weight models on easy-to-hard document extraction tasks through ExtractBench. The results are all available on @huggingface 🤗 ExtractBench is a schema-guided extraction benchmark that contains 4.8k+ pages across 8 domains and 67 document types, with a mix of short, medium, long docs and simple/complex schema.s The results reported is a measure of “value accuracy” through unified F1. ✅ Qwen 3.8 leads the pack ✅ earlier generations of Qwen models are also quite strong ✅ kimi-k3 is the next best. GLM-5.3-flash and qwen 3.8 flash also just released today - hopefully will have results on these soon! Note : these results don’t include visual grounding (whether each value is mapped to the right bounding box) and confidence scores. When you include these, it is increasingly clear why specialized OCR tools (like LlamaParse) matter, since this is metadata that’s hard to DIY by prompting the raw model. Come check out our HF leaderboard: https://huggingface.co/datasets/llamaindex/ExtractBench?leaderboard_base_model=false Learn more about ExtractBench here: https://www.extractbench.ai/
RT simon Computer needs document parsing
Parsing runs entirely on-device, so sensitive documents never leave the machine. On ParseBench-100, Computer scores 65.1% vs 34.6% for Hermes and 13.9% for Pi, in least time with fewest tokens.
If you are looking to parse large volumes of PDFs (and other document formats) while maximizing accuracy and cost, we're offering $1k in credits if you signup to the Pro plan! We're guaranteeing high quality and low cost. If you run into issues on the Pro plan we're always happy to open a Slack channel to help out. https://www.llamaindex.ai/pricing
🦙☀️ It’s a hot Llama summer, and it pays to be a PRO at document parsing. For a limited time, upgrade to the LlamaParse Pro plan and we'll triple your credits: 400K → 1.2M credits in your first month. That's $1,000 (~80k pages) of parsing, free. No code, applied
How do startups differentiate and win against the frontier labs? - product UI/UX - brand - harness - data - workflow We'll talk about this with @GuangyuRobert over good food and good vibes 👇
Hosting the 2nd founder dinner in sf on August 31 with @GuangyuRobert at @Fundamental - the team behind @tryshortcutai Agents are increasingly capable of solving tasks for longer task horizons without human in the loop. This has rewarded "horizontal" products owned by the
View quoted postRT Robert Yang come hang out with me and @jerryjliu0 of @llama_index next monday in sf!
Hosting the 2nd founder dinner in sf on August 31 with @GuangyuRobert at @Fundamental - the team behind @tryshortcutai Agents are increasingly capable of solving tasks for longer task horizons without human in the loop. This has rewarded "horizontal" products owned by the
View quoted postHosting the 2nd founder dinner in sf on August 31 with @GuangyuRobert at @Fundamental - the team behind @tryshortcutai Agents are increasingly capable of solving tasks for longer task horizons without human in the loop. This has rewarded "horizontal" products owned by the frontier lab, like Codex and Claude Code, which let users specify a broad range of tasks through a single interface. For startups to survive, is the moat around: - Product (can you build a differentiated interface from Cowork) - Harness (can you build a more differentiated agent for a given set of tasks) - Data (can you accumulate a proprietary set of data that frontier models are not trained on) - Deep domain and workflow expertise (can you learn more about a business than OpenAI can) - Brand / mindshare (you're more well known for a particular thing than a frontier model) RSVP here: https://luma.com/llamai-8hry
RT CoreWeave Another special guest is hitting the stage at #FullyConnected26! @jerryjliu0 built @llama_index, the framework thousands of teams use to connect LLMs to their data. Come see Jerry and many more Sept 29th to Oct 1st in SF. Grab your spot here: https://www.utm.io/usfdT
I'm actually curious if it's that scary to be in the #1 camp * I still think humans will use software instead only doing work through agent apps. So humans will continue to use SaaS through the UI and agents will use SaaS through the APIs * as other folks have mentioned in the comments, users likely won't do general agentic work through this software, but they'll make use of specialized agents for specific tasks * software can charge per seat and also with a credit allocation on API calls * It takes real work to maintain and improve software that's good enough to run your production workflows through it. If you DIY your own system of record, you'll be eating human or token costs trying to maintain it for your evolving needs, which is basically the same thing as paying a subscription
My rough prediction here is there's going to be two types of companies: 1. Companies that are ok with basically being an API - they expose an MCP, skills, embrace your agents 2. Companies that charge a ton for API access, try to get you using their own agents, Companies will
View quoted postWe built one of the most comprehensive benchmarks for document extraction, and evaluated it across a lot of different systems: ✅ one-shot frontier VLMs ✅ frontier VLMs + coding agent harnesses ✅ one-shot open weight VLMs ✅ other document extraction tools (including LlamaParse) Document extraction is an extremely diverse task that covers many different types of docs. From simple schemas over short docs (e.g. resume extraction) to complex extraction docs (credit agreements, data room bundles). @disiok is leading this webinar. Come check it out! https://watch.getcontrast.io/register/llamaindex-inside-extractbench-benchmarking-document-extraction-for-agents
Every extraction API demos well on a clean invoice. But what about the scanned form, the nested table, the 40-page financial report with merged headers? We tested 14 frontier systems to find out. ExtractBench evaluates schema-guided extraction across 370 enterprise documents, 67
as an LAHS grad, this isn't that out of the ordinary los altos is like the 4th richest zipcode in the country
Just passed Los Altos High parking lot Who tf buys a new Porsche 911 for a high school kid (with student driver stickers and all)
View quoted postnot sure i fully get this. - i want my agents to use slack - i don't want my agents to build a new slack - i don't want to use slack's agent i think every piece of software/system of record needs to become agent-native and not just human-native, which will benefit them because agent consumption will be exponentially higher than humans. but that doesn't necessarily mean turning into agents themselves
Prediction: systems of record will need to become AI harnesses or face replacement by agents
View quoted postI did track (specifically long and triple jump) for 6 years in middle and high school. If the robot had better form on the landing (legs folded instead of trying to stand straight up) and they provided a proper sand pit, it could've easily cleared the 8.95m world record.
Humanoid robot from Team Tianjiao recorded a 7.97-meter long jump to win gold at the 2nd World Humanoid Robot Games in Beijing. Just 0.98 m shy of Mike Powell's 8.95 m men's world record. In 2025 the robot best was only 1.25 m
View quoted postThis summer, we brought in two first-year Waterloo interns (‘rising sophomores’ in 🇺🇸) and tasked them with shipping complex production features that help push the frontier of our document processing and extraction capabilities. Yash created a dashboard that allowed customers to better understand the distribution of documents flowing through the system - and gives us anonymized metrics that we can use to improve our own capabilities. Jishnu helped to do deep research into grounded confidence scores for all of our document extraction capabilities. These scores are a lot more calibrated to real uncertainty about any value being extracted compared to our earlier implementation, allowing customers to trigger HITL in the right places within their document workflow. More on this coming soon 🙂 Outside of work, I have fond memories of watching the Knicks final game 4 with them (the one where the Knicks came back down 29 points). We’ve historically brought on full-time engineers with a bit more experience, but Yash and Jishnu ramped up extremely quickly and exemplified the high-agency, high-energy scrappiness that we look for in anyone joining the @llama_index team. They weren’t old enough to drink but old enough to ship production code 😂. We’re so happy they spent time with us so early in their careers, and wish them the best with the next 4 years of school. If you’re young but cracked, we’d love to chat. We’re hiring across *a lot* of roles across product/engineering/research (as well as growth and GTM). Careers page: https://www.llamaindex.ai/careers
RT LlamaIndex 🦙 Our 2nd founder dinner in SF co-hosted by @jerryjliu0 and @GuangyuRobert at @Fundamental - the team behind @tryshortcutai Talking about existing moats in the AI era. Frontier labs are moving past model APIs into vertical agents - ChatGPT Health, Claude for Legal. So where does the moat sit now? ✅ Agent engineering ✅ Infra optimization ✅ Domain evals and data ✅ Workflow expertise ✅ GTM and brand If you're a founder or CTO shipping agents in production and want to know what other teams are doing to maintain their moat. Request a seat 👉️ https://luma.com/llamai-8hry
RT True Ventures Coming to Connected Stack in SF — @jerryjliu0, CEO & Co-founder of @llama_index, nominated by @GreylockVC. LlamaIndex is building document infrastructure for AI agents, with LlamaParse purpose-built for the complex documents that general-purpose models struggle with. See Jerry Liu on September 10 in San Francisco. Request an invite at http://connectedstack.ai.
The latest RAG trend for the current agent harnesses (Codex, Cowork) is to do two passes of document processing to solve a knowledge work task over a data room of documents: 1️⃣ A fast and light pass, oftentimes using a free/OSS doc parsing tool. This can be cheaply run across 10-100-1k’s of files, and enables the agent to then do retrieval (e.g. grep, semantic) to find relevant subsets of context. 2️⃣ A “just-in-time” VLM-based pass. Once the agent finds the relevant pages of context, it will screenshot the documents can call its own VLM (or write code) to dissect the pages. The issue with only using VLM-based OCR tools over massive ad-hoc customer file dumps is that it’s slow and expensive. Doing JIT VLM OCR allows the agent to filter through the data cheaply, but still preserve accuracy for the context that’s needed for the task. The agent harnesses do two-pass document processing by default using off the shelf-tools: pdf2text as the first pass, and using itself (Opus 5) as the second pass. See the below video where Cowork runs over a bunch of PDFs to answer a question about a benchmark graph in the Kimi k3 paper. The main issues here with the “out of the box” doc processing these agents offer are: * Opus 5 is not the best VLM for OCR. It is also way too expensive at scale and lacks grounding * The OSS tools like pypdf, pdf2text, may not be versatile enough as the first pass. * The agent will write a lot of throwaway code to rewrite things an OCR tool would’ve provided out of the box, like chart processing, bounding boxes, confidence scores, leading to increased cost and speed. We have all the tools within @llama_index to help any agent do two-pass document processing with higher accuracy and lower cost. 1️⃣ We have liteparse for the first pass - a free/OSS parser written in Rust that’s faster/more accurate than other OSS parsers, and supports 50+ document types 2️⃣ We have LlamaParse for the second pass - an agentic document engine that uses VLMs+harnesses t...
the difference in productivity between 0-1 mode vs. working within a big team at a large co seems universally true even pre-chatgpt. it's really easy to be "productive" when (a) you're the only person driving decision making and implementation, and (b) you're still <20% effort in the 80-20 rule. it becomes a lot harder to contribute at the same pace, almost by design, when you share decision making within a larger team and have to optimize for quality AI in the hands of smart people probably makes this contrast even more apparent. it might also explain why the fastest moving products at large companies start with extremely small pockets of people (e.g. boris with claude code)
There’s a phenomenon where talented individuals can achieve 10x their potential thanks to AI when working on their own thing But when the same individual is put into a large organization, they at most increase their potential by 20% (and sometimes it’s even decreased) This is
View quoted posti think there's a real opportunity for labs to bake in even higher switching costs between claude code/codex/grok bot etc. whenever a new model release comes out, i already feel an inertia to switch between apps because of my existing skills, routines, system instructions, project setup, and more. i do maintain an externally generated wiki / artifacts for each project, so technically i could point any app at the same wiki and get similar results, but this loses the nuances of conversation history. if memory is turned on, then these apps index and remember your context for subsequent sessions, alleviating the need to retype massive amounts of context for new sessions the biggest pain point for any of these tools is figuring out how to efficiently feed it the right context every time. maybe the answer really is around building a good, self-improving context graph
saas isn't dead, it just has to be repurposed (+ remonetized) for agent consumption
So many vendors are NOT getting this I have one or two agents I use and like. For anyone else: give me an MCP interface to connect these agents to so I can use your service Unless your a frontier AI lab, I prob don't want to use your agent, sorry
View quoted post