Every vendor in the investment management space is now telling you their product is AI-ready. What they usually mean is that they have added an AI button to their GUI. What they almost never mean is that the underlying data your firm generates — the positions, the transactions, the research notes, the compliance records — is in a state where AI models can actually do useful work with it.
I have been working inside investment management infrastructure for several years. Over the past eighteen months, I have been directly involved in efforts to get our data into a state where AI tooling can be applied to it. This post is about what that actually involves: the specific problems you encounter, the decisions you have to make, and the things nobody tells you before you start.
The gap between "we have data" and "we have AI-ready data"
Investment management firms generate enormous amounts of data. Positions, NAVs, trade blotters, attribution reports, risk metrics, compliance exceptions, research documents, meeting notes, email threads about portfolio decisions. Most of this data exists. Almost none of it is in a state where you can point a language model at it and get reliable, useful output.
The problems fall into a few categories, and understanding which category you're dealing with determines what you actually need to build.
Structural inconsistency is the most common. The same concept — say, "cash" as a position — is represented differently in your portfolio management system, your risk system, your accounting system, and your reporting system. They might use different instrument identifiers, different sign conventions, different accounting treatments. A model that ingests data from multiple systems without understanding these differences will produce outputs that look confident and are wrong.
Temporal confusion is subtler but equally serious. Investment data is deeply time-dependent. A position as of yesterday is a completely different fact from a position as of a week ago. A price at market close is different from a price mid-day. If your data pipeline doesn't carry rigorous timestamp and as-of-date metadata through every transformation step, you will eventually feed a model data that it treats as current when it is stale.
Missing provenance is the one that creates regulatory risk. In a regulated environment, it is not sufficient to have a number. You need to know where that number came from, who touched it, and when. If you can't answer those questions for the data you're feeding into an AI system, you cannot use that system's outputs in any context that a regulator might review. Which in investment management is most contexts.
What "AI-ready" actually means
I find it useful to think about this in terms of three properties. Data is AI-ready when it is clean, contextualised, and retrievable.
Clean means what it always meant in data engineering: consistent identifiers, no duplicates, validated against known constraints, missing values handled explicitly rather than silently. The specific wrinkle in financial data is that many of the validation rules are domain-specific and not obvious to engineers who haven't worked in the industry. A negative cash balance is sometimes valid and sometimes a data error depending on context. A 100% position weight is correct for some fund structures and wrong for others. You need the business logic encoded, not just generic data quality checks.
Contextualised is where most firms fall short, and where the interesting engineering work lives. For an AI model to do useful work with investment data, it needs to understand what the data means, not just what it says. This is the difference between a model that can tell you the current NAV of a fund and one that can tell you whether a recent NAV move is unusual given the fund's historical behaviour and the market environment. The latter requires that your data carries enough metadata and relationship information to support that kind of reasoning.
In practice, contextualisation usually means maintaining a semantic layer: a structured representation of how your data concepts relate to each other, what the valid values are, what normal ranges look like. Think of it as a machine-readable data dictionary with teeth.
Retrievable is the operational problem. Clean, contextualised data that you can't efficiently query is not AI-ready. This is where the vector database conversation comes in, but also where a lot of firms go wrong by jumping straight to the solution without solving the upstream problems first.
The data quality work you can't skip
Before you touch vector embeddings or retrieval-augmented generation or any of the model infrastructure, you need to solve the data quality problems. There is no shortcut here. A model operating on bad data produces bad outputs confidently, which is worse than producing no output at all.
The practical starting point is a data quality audit that is specifically oriented toward AI use cases. Standard data quality checks (null counts, duplicate rates, referential integrity) are necessary but not sufficient. You also need to check for what I call AI-specific quality issues.
Semantic drift is one. This is where the same field in your database has been used to store different things at different points in time. Maybe a field called instrument_type used values like EQ, FI, CASH until 2021 and then a system migration changed the vocabulary to Equity, Fixed Income, Cash Equivalent. A model trained or fine-tuned on this data will see what looks like noise when it is actually history. You need to identify these discontinuities and either clean them or ensure the model knows about them.
Coverage gaps are another. Investment management data often has gaps that are not random. Data quality tends to be worse for older records, for legacy instruments, for edge cases in fund structures. If your AI system will be used to answer questions that might touch these gaps, you need to know where they are and either fill them or bound the system's ability to answer questions in those areas.
Entity resolution is the one that takes the most time. Your firm almost certainly has multiple systems that store information about the same entities — the same securities, the same counterparties, the same clients — under different identifiers with no automated linkage. Getting to a point where you have a reliable entity graph that a model can use to understand that "Apple Inc", "AAPL US Equity", "US0378331005" (the ISIN), and "037833100" (the CUSIP) are all the same thing requires work that is fundamentally not glamorous and cannot be automated away.
Structured data vs unstructured data: two different problems
Most AI-in-finance discussions conflate two genuinely different problems: using AI to work with structured financial data (positions, prices, transactions) and using AI to work with unstructured content (research notes, earnings call transcripts, internal memos, email threads).
Both are worth solving. They require different approaches.
For structured financial data, the dominant pattern right now is text-to-SQL: you point a language model at your data schema, give it enough context to understand the domain, and let it translate natural language questions into SQL queries that you then execute against your actual data. This is more reliable than asking a model to answer directly from embeddings because the SQL is verifiable — you can inspect the query before running it, catch errors before they become wrong answers.
The challenges in financial services are that the schema is complex, the domain concepts are non-obvious, and the stakes of a wrong answer are real. A model that generates a query using the wrong as-of date for a position lookup, or that misunderstands a fund's accounting treatment, will produce results that look right and aren't. This means you need substantial prompt engineering, a well-documented schema, and a validation layer between the model's generated SQL and the result the user sees.
For unstructured content — the research notes, the meeting records, the analyst commentary — the standard approach is retrieval-augmented generation (RAG): you chunk your documents, embed them as vectors, store them in a vector database, and at query time you retrieve the most relevant chunks and give them to the model as context. This works well when your documents are well-structured and your queries are semantically clear. It works less well when your documents are poorly formatted, when the answer to a question spans multiple documents that weren't written to relate to each other, or when precision matters more than recall.
In investment management, both of these failure modes are common. Research notes are often poorly formatted. Questions about portfolio construction often require synthesising information across many sources. Precision matters a great deal.
The compliance and governance layer
This section is the one I see engineers skip and then regret.
Any AI system operating on investment management data in a regulated context needs a governance layer that is not an afterthought. Specifically, you need answers to the following questions before you deploy anything to production.
What data can this system access, and what can it not access? Material non-public information (MNPI) is the obvious concern: a system that can inadvertently surface MNPI in its responses creates regulatory exposure that no one at your firm wants. The controls you have on MNPI need to extend to any AI system that touches related data.
How is the system's output logged, and for how long? Regulators increasingly expect firms to be able to reconstruct decision-making processes. If an analyst used an AI system to inform a trade decision, you may need to produce the specific query, the specific response, and the model version that generated it. This is not a hypothetical future requirement; it is something compliance teams are starting to ask about now.
What is the process when the system produces incorrect output? This is not a question of if, it is when. You need a documented process for identifying errors, communicating them to affected users, and correcting the underlying cause. In financial services, an AI system without a clear error-handling process is not deployable in any consequential context.
The infrastructure underneath
On the infrastructure side, the stack for AI-ready financial data is more complex than most initial assessments suggest.
You need a data pipeline that carries full provenance metadata — source system, extraction timestamp, transformation steps — through every stage. This rules out a surprising number of standard ETL patterns that drop or flatten metadata as a matter of course.
You need a semantic layer that is maintained alongside your data. In practice this usually means a combination of a data catalog (something like DataHub or Collibra, depending on your scale and budget) and an in-house-maintained domain model that encodes the business logic your catalog won't know about.
For the vector database layer, the main contenders right now are Pinecone, Weaviate, and pgvector if you want to keep it inside your existing PostgreSQL infrastructure. I have worked with pgvector in production and it is a reasonable choice for moderate-scale use cases where you are already running PostgreSQL and want to minimise operational complexity. For larger scale or more sophisticated retrieval patterns, a dedicated vector database is worth the operational overhead.
On the model side: unless you have a specific reason to run models on-premise (and there are some in financial services — data residency requirements, latency requirements for certain use cases, reluctance to send sensitive data to third-party APIs), the major hosted providers are the practical choice for most firms. The security and compliance posture of the major providers has matured enough that most compliance teams can get comfortable with it, but you will need to work through a formal vendor assessment.
What a realistic timeline looks like
If you are starting from a typical investment management data environment — several source systems with inconsistent identifiers, limited data quality tooling, no semantic layer — here is a realistic timeline for getting to a state where AI tooling can be used in production on anything consequential.
Data quality audit and remediation: three to six months, depending on the number of source systems and the depth of the quality issues. This cannot be parallelised much because each issue you find often reveals a downstream issue you didn't know about.
Semantic layer and entity resolution: two to four months, heavily dependent on whether you have business analysts who know the domain well enough to encode the rules. The engineering work is not the bottleneck here; the knowledge capture is.
Governance framework and compliance sign-off: one to three months, depending on your firm's internal processes and the specific use cases you are targeting. Firms with mature model risk management frameworks (required for certain regulated entities) have a more defined process here, which is both slower and more defensible.
Infrastructure and prototype: one to two months, running in parallel with the governance work once you have enough data quality to prototype on.
Realistic total: eight to fifteen months from standing start to a production system you can stand behind in a regulated context. Anyone telling you they can do it faster is either starting from a better data position than you, cutting corners on governance, or selling you something.
The thing that actually matters
I want to end with something that doesn't fit cleanly into the technical sections above.
The firms I have seen make genuine progress on AI in investment management are the ones where data quality is treated as a first-class infrastructure problem, not a prerequisite that someone else will handle before the interesting work begins. The data quality work is the interesting work. The semantic layer is the intellectual contribution. The model is just the interface.
If your firm's AI initiative is structured as "build the AI stuff and fix the data later", it will not produce results you can use in production. If it is structured as "fix the data in a way that makes it AI-ready, then plug in the models", it might actually get somewhere useful.
The models have gotten good enough that data quality is now the binding constraint in almost every serious financial AI application. Treat it accordingly.