lakeFS Acquires DVC, Uniting Data Version Control Pioneers to Accelerate AI-Ready Data
A step by step guide to the lakeFS Cloud playground environment In this document, you will learn the quickest way to get started with lakeFS, utilizing the playground experience in lakeFS Cloud. Then I will cover how to connect your own storage to lakeFS, so you can run lakeFS against your own data. Step 1: […]
lakeFS directly supports Apache Iceberg tables. Using straightforward table identifiers you can switch between branches when reading and writing data: lakeFS itself remains format agnostic, happily providing version control for data whether binary, tabular, or just a lump of text. The benefit of this new functionality is that the same table that you create and
We are pleased to announce that lakeFS Cloud is now available as a self service on Azure. lakeFS Cloud is a fully-managed lakeFS platform, providing version control for your data lake. As well as being secure and scalable, it includes enterprise features such as Single Sign On (SSO), managed garbage collection, and role-based access control
If you’re deploying lakeFS on Azure you’ll be keen to ensure that you’re using the optimal set of components in doing so. One of the pieces that lakeFS needs to run is somewhere to store its metadata, and in the latest v0.103 release of lakeFS we now support CosmosDB for this. In this blog post,
lakeFS supports hooks written in Lua, enabling actions to be executed without any external dependencies.
Data is a goldmine for every organization, no matter the industry. But to make the most of it, businesses need technology to maintain and manage transactional data like payments, inventory updates, and customer records. This is where OLTP databases come in. Online Transaction Processing (OLTP) databases are used to store and process large numbers of
lakeFS is celebrating its version release 0.100.0! We’re excited to announce the 0.100.0 version release! It’s thanks to you, the community that we’ve reached this milestone and: Managed to run 4,197 commits From 73 contributors, including (to name a few): Let’s celebrate this 0.100.0 release in a BIG way. Install the 100th lakeFS v0.100.0 and
Implementing CI/CD-inspired workflows built atop lakeFS operations prevents inconsistent data and brings increased reliability to our analytics platform. This article was originally published in Lior’s Medium Blog. At Windward, our Maritime Artificial Intelligence Analytics (MAIA™️) platform delivers predictive intelligence on global maritime conditions to hundreds of businesses. Customers across different industries — like oil &
A discussion of how Role-Based Access Control works in lakeFS with code examples for three different types of user.
This post was originally published in the Enigma blog. In every software engineering problem I’ve worked on, I’ve noticed a recurring tension between two highly desirable properties: flexibility and robustness. But in each situation, this tension manifests itself in different ways. At Enigma, our goal is to build a complete set of authoritative profiles of
What’s not to love about data pipeline testing? Adding user acceptance testing and contract acceptance testing to your data pipelines makes them less likely to cause errors and ensures that enough quality checks are done on the data before it’s sent to end users. Data pipeline testing involves two components of any data pipeline: data
What is lakeFS? lakeFS is a platform that helps data engineers build scalable and resilient data lakes running on object storage. It provides version control, branching, and merging capabilities for data at petabyte scale, on or off premises. lakeFS enables teams to collaborate and manage data effectively by applying engineering best practices to data management.
We use cookies to improve your experience and understand how our site is used.
Join our community of experts:
introduce yourself, share your knowledge and discover best practices from fellow peers
