Why Law Firms Are Starting to Build Their Own AI Infrastructure
For the past few years, most of the conversation around artificial intelligence in the legal industry has focused on the tools law firms are using. That conversation is starting to expand.
Latham & Watkins recently disclosed that it has purchased Nvidia GPU servers and is developing its own internal AI capabilities, including customizing open-weight AI models for legal work. According to the Financial Times, the firm has been working on the approach for several years and now employs more than 900 technology professionals across areas that include software engineering, machine learning and AI.
For a law firm, that is a significant investment. It also provides an interesting look at how Legal AI may develop as firms become more sophisticated users of the technology.
The question is no longer limited to which AI platform a law firm should buy. Some firms are beginning to think about the infrastructure behind those platforms, where their data is processed, which models are being used and how much control they want over the technology.
What Does It Mean for a Law Firm to Build Its Own AI Infrastructure?
Building AI infrastructure does not necessarily mean creating a large language model from scratch. There is a substantial difference between developing a frontier AI model and taking an existing model and adapting it for a particular organization.
In Latham's case, the firm reportedly purchased its own Nvidia GPU servers and is fine-tuning Nvidia's Nemotron open-weight models. The servers are housed in secured data-center space accessible by the firm's personnel. Latham can therefore run certain AI workloads within infrastructure it controls rather than sending all of that work to an outside cloud provider.
The firm has not abandoned commercial AI products. It continues to use outside tools as well. That combination may be the more important part of the story. Instead of relying on one model or one vendor for every task, a firm can determine which technology makes sense depending on the work involved.
Why Would a Law Firm Build Its Own AI Systems?
There are several reasons a large law firm might want more control over its AI infrastructure. One is data.
Law firms routinely handle confidential client information, litigation strategy, transaction documents, intellectual property, internal investigations and other highly sensitive material.
Latham's chief information officer, Rene Mendoza, told the Financial Times that there are situations where client information may be so sensitive that the firm does not want to send it to any cloud vendor.
That concern isn't unique to Latham. The American Bar Association has already addressed confidentiality as one of the ethical issues lawyers must consider when using generative AI. ABA Formal Opinion 512 explains that lawyers using these tools must consider existing professional obligations involving competence, confidentiality, communication, supervision and fees.
We discussed this issue previously in AI and Attorney-Client Confidentiality: What Lawyers Should Understand Before Using AI Tools:
For firms handling particularly sensitive matters, having greater control over where information is processed can be valuable.
Law Firms May Not Want to Depend on One AI Provider
There is also a broader business issue. The Legal AI market is changing quickly. OpenAI, Anthropic, Google and other companies continue to release new models, while legal technology companies are building products on top of those systems.
The model that performs best for one type of legal work may not be the best choice for another. Research may require one set of capabilities. Contract analysis may require another. Internal knowledge retrieval, document review and administrative work may each present different requirements.
A firm with its own AI infrastructure has more flexibility to decide which model should handle a particular task. Latham's approach reportedly allows its lawyers to choose between internally operated models and commercially available products depending on the work involved. The firm has also cited flexibility around future AI usage costs and vendor terms as part of its reasoning.
That flexibility could become more important as firms use AI more heavily.
Open-Weight Models Give Firms Another Option
Open-weight models are an important part of this development. Many of the AI tools people use every day are based on proprietary models. Users interact with the model through a product or service, but the model itself remains controlled by the company that developed it.
Open-weight models provide organizations with greater ability to download, operate and customize the model within their own technical environment.
For a law firm with the necessary resources, that creates additional options. A firm may be able to adapt a model around particular workflows, integrate it with internal systems or operate it in an environment where it has greater control over data and security.
That does not automatically make an open-weight model better than a commercial system. Performance, security, cost, maintenance and the specific legal task still matter. It simply gives sophisticated firms another way to build their AI strategy.
A Law Firm's Internal Knowledge Could Become More Valuable
There is another part of this that may ultimately matter just as much as the underlying model: the information law firms already possess.
Large firms have decades of institutional knowledge spread across prior matters, briefs, contracts, deal documents, research, negotiated provisions and internal work product.
Historically, finding and reusing that information efficiently has been difficult. AI makes that knowledge considerably easier to search, organize and use within legal workflows.
As firms develop more advanced internal systems, the competitive advantage may not come from simply having access to a powerful AI model. Many firms can buy access to the same commercial technology.
The difference may increasingly come from how effectively a firm connects that technology to its own knowledge, workflows and experience. That is one reason AI infrastructure and knowledge management are beginning to overlap.
Most Law Firms Are Not Going to Build GPU Servers
Latham's approach is interesting, but it is important to keep the scale of the investment in perspective. Most law firms are not going to purchase Nvidia servers, hire teams of machine-learning engineers and operate private AI infrastructure.
They probably shouldn't.
The Financial Times reported that operating this type of infrastructure at scale can cost tens of millions of dollars annually, with costs potentially increasing substantially as the technology develops.
For small and midsize firms, commercial Legal AI products will remain the more practical option. But the broader questions Latham is addressing apply to firms of every size.
Where does the firm's data go?
Is client information used to train a model?
How long is information retained?
Which model powers the product?
What security controls are in place?
Can the firm control which information the AI system can access?
What happens if the vendor changes its pricing or terms?
How easily can the firm move its data or workflows to another platform?
Those questions should be part of evaluating an AI product whether a firm has 20 lawyers or 2,000. We covered many of these considerations in How Law Firms Should Evaluate AI Tools Before Buying Them:
Legal AI Is Becoming an Infrastructure Decision
Law firms initially approached generative AI largely as another category of legal software. That made sense. A firm could evaluate a product, purchase licenses and allow lawyers to begin using it. The technology is becoming more embedded in legal work now.
AI can be connected to document management systems, research platforms, internal knowledge, contract databases and increasingly complex legal workflows. Agentic systems can also perform multiple steps within a workflow rather than responding to a single prompt.
For more on that development, see What Is Agentic AI? What Lawyers and Law Firms Need to Know:
Once AI reaches that level of integration, decisions about models, data, security and infrastructure become more important.
NIST's work on AI risk management reflects many of the same concerns at the organizational level. Its Generative AI Profile addresses risks involving privacy, security and the management of AI systems, while its more recent work on securing AI systems includes controls addressing models, training and test data, model weights and AI agents.
Law firms do not need to become technology companies. They do need to understand enough about the technology they are adopting to make informed decisions about how it interacts with client information and legal work.
What This Means for the Legal Industry
Latham's investment is probably not a blueprint that most firms will follow literally. It is a useful indication of how seriously some firms are beginning to treat AI.
The largest firms are no longer only asking which Legal AI products their lawyers should use. They are also considering how much of the underlying technology they want to control themselves.
Other firms will make different choices. Some will rely heavily on established Legal AI vendors. Others will build internal tools on top of commercial models. A smaller group may operate their own models and infrastructure.
There is room for all of those approaches. What matters is whether the technology fits the firm's work, protects the information entrusted to it and provides enough flexibility to adapt as AI continues to change.
The firms that understand those questions will be in a much better position than firms that simply purchase whatever AI product happens to be popular at the time.
Frequently Asked Questions
What is law firm AI infrastructure?
Law firm AI infrastructure refers to the technology used to operate and support artificial intelligence within a firm. Depending on the firm, that can include commercial AI platforms, cloud services, internal databases, document systems, AI models, computing hardware and the security controls connecting those systems.
Are law firms building their own AI models?
Some firms are developing internal AI capabilities, but that does not necessarily mean building a foundation model from scratch. Firms can use existing commercial models, customize open-weight models or build applications that connect AI models with internal firm systems and knowledge.
Why would a law firm run AI internally?
Greater control over sensitive information is one reason. Internal infrastructure can also provide more flexibility over which models are used, how systems are customized and how dependent the firm becomes on individual technology providers.
Are open-weight AI models more secure for law firms?
Not automatically. Running an open-weight model internally can give a firm more control over its environment and data, but the firm also assumes responsibility for securing, maintaining and operating that infrastructure. Security depends on the entire system, not simply whether a model is open or proprietary.
Do smaller law firms need their own AI infrastructure?
For most small and midsize firms, probably not at the scale being used by firms such as Latham & Watkins. Commercial AI products are generally much more practical. Smaller firms should still understand how vendors handle client data, security, retention, model training and access to firm information.
What should law firms consider before adopting AI?
Firms should evaluate the actual legal use case, accuracy, confidentiality, security, data practices, integration with existing systems, human review requirements, vendor terms and cost. The appropriate level of review will depend on how the technology will be used and what information it can access.
Related Articles
AI and Attorney-Client Confidentiality: What Lawyers Should Understand Before Using AI Tools
How Law Firms Are Actually Integrating AI Into Legal Practice
Risks of Using AI in Legal Work: What Lawyers and Law Firms Should Know
AI Hallucinations in Court: What Lawyers Can Learn From Recent Sanctions and Citation Errors
Can Lawyers Reliably Use AI for Contract Review? Benefits, Risks, and Limitations
AI Regulation and the Legal Industry: What Lawyers and Law Firms Should Expect
Further Reading
Financial Times - Latham & Watkins buys Nvidia servers to set up in-house AI systems
American Bar Association - Formal Opinion 512: Generative Artificial Intelligence Tools
National Institute of Standards and Technology - Artificial Intelligence Risk Management Framework: Generative Artificial Intelligence Profile
--------------------------------------
About the Author
Cory D. Raines ("Cory Raines") is a Legal AI Consultant and Founder of Raines Legal Group, and PROTIPPZ, where he focuses on legal strategy, emerging technology, AI workflows, and the evolving intersection of law and artificial intelligence.
Posted by Cory D. Raines
Comments