How Lockheed Martin Ensures Traceability in MLOps with lakeFS

The company

Lockheed Martin is an American aerospace and defense company with over $130 billion market capitalisation. It’s among the biggest names in aerospace, military support, security, and technology.

To leverage artificial intelligence, the company created the Lockheed Martin AI Center, which provides foundational tooling and infrastructure in libraries for teams in the business areas. By establishing it, Lockheed Martin aims to empower its team to develop artifacts from a system that can satisfy the requirements for safe, secure, and trustworthy AI. These requirements concern security, governance, traceability, and transparency.

To this end, the company created AI Factory, a development ecosystem that enables MLOps with transparency, traceability, and repeatability in mind – enabling engineers to build and operate a trustworthy design, using open source systems like lakeFS. It was “really important that it’s composable and to avoid locking ourselves into a single vendor” said Greg Forrest, Director of AI Foundations at Lockheed Martin

The challenges

Experiment tracking and data lineage

The MLOps platform at Lockheed Martin needed to match traceability and transparency requirements. The artifacts in the platform offer engineers data and model versioning control so they can understand what data the models have been trained with. Engineers can also learn if they manipulated data over time and how that feeds into the model that they created. 

When doing experiment tracking and hyperparameter sweeps, engineers may get different types of models that they will ultimately deploy depending on the performance. Tracking that and having pipelines and repeatability in that process is what MLOps is all about for Lockheed Martin.

Security and compliance

Another challenge Lockheed Martin needed to address as part of the data centralization process was security and compliance. The company found it difficult to move data within and between classified environments, making the transferability of its AI Factory one of the most important requirements.

Scalability and collaboration

At Lockheed Martin, teams host models across all environments, resulting in a fragmentary AI landscape. The company was looking for the ability to centralize the process and allow team members to use a single API to access the model they need for scalability. The ability to put these services in the hands of citizen data scientists and non-technical team members was a key requirement of the system’s scalability.

Adopted solution

Challenge solved: Experiment tracking and data lineage

lakeFS allows engineers to easily track these different experiments, ensuring that every change, parameter adjustment, or data modification is captured and stored. If a particular experiment produced the best model, it could be reliably reproduced by referencing the exact versions of the data and models used.

lakeFS also enabled engineers to maintain a detailed record of every version of data and models used during the development process. This enabled them to track which datasets were used for training different models, ensuring full traceability of the input data that led to specific model outputs. This capability is critical in meeting transparency requirements, as it allows engineers to see how data evolved over time and how those changes impacted the final model.

Challenge solved: Security and compliance

The ability of the AI Factory to be transferable brought significant benefits to Lockheed Martin teams. Teams can run AI Factory in their environments and get access to the entire MLOps toolbox, including open-source models. The ability to control access by user from the data center to the edge is very important for a company since it relies on local teams to use the MLOps toolbox to retrain and deploy models in the field across disconnected environments and they must be able to adhere to all security regulations. Implementing a system like lakeFS helped the Lockheed Martin team ensure role-based access to their users and groups, thus guaranteeing security and compliance across their entire team, no matter what actions they were running.

Challenge solved: Scalability and collaboration

The generative AI platform Lockheed Martin built allows the company to scale AI across all teams while simultaneously saving costs. Moving toward low-code and no-code solutions has been a part of this effort. Such solutions allow team members to create their assistance using the provided infrastructure and tooling. The AI Factory platform provides consultancy and ready-made solutions such as RAG-as-a-service, which team members can fine-tune to match their unique requirements.

Results

Lockheed Martin combined best-of-breed, open-source point solutions into its MLOps pipeline to meet the critical requirements of transparency and traceability, using, among other solutions, the open-source data version control solution lakeFS.

Transparent, traceable and repeatable development of artificial intelligence is critical to us. What’s important for Lockheed Martin here is that we don’t just focus on what we are building but also on the how: having toolsets and putting the system engineering rigor around how we develop artificial intelligence. I see all these companies with awesome technology and we’re all coming together for a cause to support national security.

Greg Forrest

Director of AI Foundations, Lockheed Martin

Further Reading: NVIDIA AI SUMMIT DC – OCTOBER 2024

Enabling Trustworthy AI Across the Enterprise

Watch the full presentation taken from the NVIDIA AI Summit, October 2024 and presented by Greg Forrest, Director AI Foundations at Lockheed Martin.

Iddo Avneri Author

Iddo has a strong software development background. He started his career in the army, where he served for 8 years, eventually heading the main software development school. Following his service, Iddo built technical teams for several startups in the Observability, Cloud and data spaces. Prior to joining the lakeFS team Iddo built the technical enterprise field team at Turbonomic, from the ground up, as well as served as the Field CTO, and was the account executive for some of the company’s largest customers; up to the $1.9B IBM acquisition in 2021. In Treeverse, the company behind lakeFS, Iddo runs all customer engagements from sales to customer success.