lakeFS Acquires DVC, Uniting Data Version Control Pioneers to Accelerate AI-Ready Data
A few weeks ago, I was looking at a dashboard in our internal BI system. It’s a simple system. Redash over PostgreSQL that has just a few hundreds of thousands of rows. I noticed a change in one of my favourite metrics that calculates the number of new installations since the beginning of the quarter. […]
Organizations have practically always needed data analytics, and they jumped on the analytics bandwagon as soon as the first computers appeared on the scene. In the 80s, businesses built data warehouses using relational databases as their decision-support systems (DSS). However, as companies generated more diverse data at high velocity, relational databases showed their limitations. This
Data Versioning at Scale: Solutions Overview Back when I was a 23-year-old student, I worked at an Israeli networking company as a BI analyst in the Operations department. My job revolved around modeling the company’s inventory which was quite costly and needed optimization. At some point, I attended a meeting of the company’s management. When
A year has passed since we shared the State of Data Engineering 2021. And since we released that article last May, not much has changed in the data landscape. In fact, we had discussions internally about whether we should even do an update for 2022. We kid. It was another year worthy of its own
Trends in the data industry, how lakeFS fits into the new data stack, and a personal story of why I chose to join lakeFS. Open Source Is Everywhere! For the past decade, open-source has made a home in the ever-evolving data stack zoo known as the Hadoop ecosystem. To keep everything in order, ZooKeeper was
Who needs a role model? When we first launched lakeFS in August of 2020, we asked ourselves a simple question: What does success look like? And how will we know we’re doing the right things to get there? Of course with thousands of installations, a thriving user community, active developer contributions, and exponential growth on
The idea About two years ago, we left SimilarWeb, where we had the privilege of managing some of the most interesting and complex data architecture projects in the world. The architecture was centered around a Data Lake – 7 Petabytes of data on Amazon S3. On top, hundreds of hourly, daily and monthly jobs running,
Let’s start with the obvious: the lakeFS project doesn’t exist in isolation. It belongs to a larger ecosystem of data engineering tools and technologies adjacent and complementary to the problems we are solving. What better way to visualize our place in this ecosystem, I thought, than by creating a cross-sectional LUMAscape to depict it. What’s
The demand for infrastructure that contributes to the collection, storage, and analysis of data is growing with the increasing amounts of data managed by organizations. Every organization that manages data pipelines to extract insights from data encounters the need for reproducibility, safe experimentation, and means to ensure data quality. The path to answering these needs
lakeFS is an open source platform that delivers resilience and manageability to your existing object-storage based data lake. With lakeFS you can build repeatable, atomic and versioned data lake operations – from complex ETL jobs to data science and analytics.
Join our community of experts:
introduce yourself, share your knowledge and discover best practices from fellow peers
