For an optimal experience, provide your email below and one of our lifeguards will send you a link to start swimming in the lake!
We’re happy to introduce experimental support for running SQL queries, directly on objects (your files!) in lakeFS. All through the lakeFS UI. No need to install or configure anything. TLDR – You can now run SQL queries on Parquet and other tabular formats, directly from the lakeFS UI! Explore data, look at its schema, compare …
lakeFS ❤️ DuckDB: Embedding an OLAP database in the lakeFS UI Read More »
TLDR – lakeFS hooks just became so much easier to use.As of v0.87.1, lakeFS includes a Lua Virtual Machine that allows you to run hooks directly in lakeFS, without relying on any external component. What are lakeFS hooks lakeFS allows users to define logic that would happen before or after important events in the system. …
In the past year, words like recession, business slowdown and monetary cuttings are being heard more and more often. Not just in the economic press and in the media, these discussions are very much heard also in almost all companies – within boardrooms, in management meetings and when engaging with potential investors and customers. As …
What is lakeFS? As data practitioners, we use many different terms to talk about what we do – we call it business intelligence, analytics, data pipelines, or insights. But there’s one term that captures what we do really well: delivering products. When we were leading a large R&D organization, we couldn’t help but wonder about …
What if you could manage your data lake just like you manage code? With rollback, versioning, and branching capabilities on top of your existing data lake? lakeFS is an open-source project that provides a Git-like version control interface for data lakes, with seamless integration to most data tools and frameworks. lakeFS enables you to easily …
The lakeFS playground is now live and everybody can play! Read More »
This is the first post of a series (well, at least 2) about Data Products. In Part I, we’ll cover what data products are, how they differ from traditional services built by product teams, and conclude with a few requirements that help create successful data products. What Are Data Products? Data products are carefully curated …
This post is the second in a series about Hive Metastore. In the last post, Einat covered its history, the problems that it solves – and questioned whether or not it will survive the next revolution. For this post, I’d like to take a closer look at Hive Metastore at present time. Specifically how it …
The idea About two years ago, we left SimilarWeb, where we had the privilege of managing some of the most interesting and complex data architecture projects in the world. The architecture was centered around a Data Lake – 7 Petabytes of data on Amazon S3. On top, hundreds of hourly, daily and monthly jobs running, …
How to use lakeFS to keep multiple Delta tables consistent and guarantee data quality.
Introduction When building a data lake, there is perhaps no more consequential decision than the format data will be stored in. The outcome will have a direct effect on its performance, usability, and compatibility. It is inspiring that by simply changing the format data is stored in, we can unlock new functionality and improve the …
Hudi, Iceberg and Delta Lake: Data Lake Table Formats Compared Read More »
© Copyright 2022 | All Rights Reserved | Privacy Policy | Terms of Use
