lakeFS Acquires DVC, Uniting Data Version Control Pioneers to Accelerate AI-Ready Data
Apache airflow enables you to build multistep workflows across multiple technologies. The programmatic approach, allowing you to schedule and monitor workflows, helps users build complicated ETLs on their data that will be difficult to achieve automatically otherwise.This enabled the evolution of ETLs from simple single steps to complicated, parallelized, multi steps advance transformations: The challenge […]
Our community is full of people with incredible skills and know-how. And this nomination proves us right! Our community member @Leonard Aukea has been nominated for Machine Learning Professional of the year as part of the Nordic DAIR Awards. Congratulations, Leonard! Who is Leonard? Leonard Aukea has been Heading Machine Learning Engineering and Operations at
Table of Contents Developing ETL pipelines – what it looks like today Developing and testing ETL pipelines is very complicated. Since you typically don’t want to test against production data, you need to create your own personal space, like a bucket, and Then either sample or copy data out of production. Both approaches have their
It was 27th June 2022. San Francisco was bustling with 5000+ data folks from around the world to attend the Data & AI summit live after two years. Four days packed with tons of information from Keynotes, Speakers, Panels, Expo booths and Databricks trainings. Flurry of new product announcements followed. lakeFS cloud launch, Delta lake
What is lakeFS? As data practitioners, we use many different terms to talk about what we do – we call it business intelligence, analytics, data pipelines, or insights. But there’s one term that captures what we do really well: delivering products. When we were leading a large R&D organization, we couldn’t help but wonder about
If you think about it, lakeFS is about two things — version control and big data. We see ourselves as bringing version control to big data. This bridges a workflow gap that currently exists when working with data and working with code. This gap is purely artificial — there’s no conceptual reason why different workflows should be required for
Introducing Kubeflow and lakeFS Kubeflow is a cloud-native ML platform that simplifies the training and deployment of machine learning pipelines on Kubernetes. An ML project using Kubeflow will consist of isolated components for each stage of the ML lifecycle. And each component of a Kubeflow pipeline is packaged as a Docker image and executed in a
We use cookies to improve your experience and understand how our site is used.
Join our community of experts:
introduce yourself, share your knowledge and discover best practices from fellow peers
