For investors and limited partners (LPs), understanding a portfolio starts with the documents behind it. Investor reports, capital statements, and financial statements explain how investments are performing. Capital call and distribution notices, along with subscription and redemption documents, help track how capital moves into and out of those investments. Together, they provide the information investors rely on to understand their portfolios and make decisions.
These documents form the information layer for wealth technology. Their contents feed the systems investors use for portfolio reporting, analysis, and day-to-day operations. The usefulness of those systems depends on how faithfully that information is captured. As AI takes on more of that extraction work, we need a way to measure how reliably it gets the names, dates, and numbers with the highest accuracy possible.
Today, we’re introducing AWE-Bench, the Assets & Wealth Extraction Benchmark, to help answer that question. AWE-bench evaluates how accurately AI models extract information from financial documents, with known answers and results you can inspect field by field. It’s available on Mantle Research, our home for research into AI for wealth and equity.
We built AWE-bench to give practitioners and technology teams a shared reference for evaluating the raw document extraction abilities in the wealth tech space. Our aim is to bring the industry together around tests, and help ground conversations around AI in this vertical.
Financial documents, with answers we can check
AWE-bench covers the document families described above, from investor reports and capital statements to subscriptions, redemptions, capital calls, and distributions. It also tests different layouts, including bullet lists, newsletter cards, formal financial statements, and longer reports with appendices.
Those differences matter. A number in a table, a date in a paragraph, lurking footnotes, and a name repeated across several sections each present a different extraction task. A useful evaluation needs to show how models perform across that variety.
Every benchmark document is synthetic and has an exact answer key. That gives us a controlled way to check the output without publishing private investor records. Names are evaluated with fuzzy matching, dates against the precision shown in the document, and numbers within a defined tolerance in the document’s own units.
See what’s behind the accuracy score
AWE-bench reports overall accuracy as the share of scored fields extracted correctly. If a document fails outright, all of its fields count as missing.
You can then examine results by field type, document family, layout, and section. The failure breakdown distinguishes missing values from errors such as incorrect signs, rounding, or amounts read at the wrong scale. Values and list entries invented by a model are reported separately; they are not included in the headline accuracy calculation.
For a team evaluating extraction, those details help make the results actionable. A model that misses dates in prose presents a different review problem from one that misreads amounts in tables. Knowing where errors occur gives you a more specific starting point for testing your own workflow.
AWE-bench also reports cost per document and processing time alongside accuracy. These measures help teams explore the tradeoffs between extraction quality, cost, and speed. The synthetic document set provides a controlled comparison; evaluating performance on your own document mix is still part of deciding how to use a model.
Help shape a benchmark the industry can use
The people working with these documents know which omissions are easy to catch, which errors can travel into another system, and which exceptions take the longest to resolve. We want that experience to help shape AWE-bench as it develops.
We’re inviting wealth and equity practitioners, operators, researchers, and technology teams to examine the benchmark and consider:
- Which document types and layouts should the benchmark represent next?
- Which extraction errors create the most review work for your team?
- What would make the results more useful when evaluating AI for your workflow?
Our ambition is for AWE-bench to become a shared reference the industry can use, question, and improve together. Clear tasks and visible scoring give us a place to start that conversation.
Explore the AWE benchmark to compare model results and examine how they perform on the documents behind your work.
Dwayne Forde is the co-founder and CTO of Mantle, bringing over 25 years of experience across startups and enterprise organizations. Celebrated for introducing innovative patterns in emerging technology domains, he remains a trusted and influential figure in the engineering community.