For an optimal experience, provide your email below and one of our lifeguards will send you a link to start swimming in the lake!
VC surveys show that the MLOps category has significantly expanded in the past few years, with hundreds of companies defining themselves as part of this dynamically evolving niche. MLOps systems provide the infrastructure allowing ML practitioners to manage the lifecycle of their work from development to production in a robust and reproducible manner. An MLOps …
No time for the full article now? Read the abbreviated version here Many organizations and companies are rapidly moving from managing only structured data sets to managing both structured and unstructured data. This is due to the growth in the number of sources and data types, which are rooted in the new variety of use …
Managing Structured and Unstructured Data – a Guide for an Effective Synergy Read More »
Implementing CI/CD-inspired workflows built atop lakeFS operations prevents inconsistent data and brings increased reliability to our analytics platform. This article was originally published in Lior’s Medium Blog. At Windward, our Maritime Artificial Intelligence Analytics (MAIA™️) platform delivers predictive intelligence on global maritime conditions to hundreds of businesses. Customers across different industries — like oil & …
How Windward Leverages lakeFS for Resilient Data Ingestion Read More »
A discussion of how Role-Based Access Control works in lakeFS with code examples for three different types of user.
This post was originally published in the Enigma blog. In every software engineering problem I’ve worked on, I’ve noticed a recurring tension between two highly desirable properties: flexibility and robustness. But in each situation, this tension manifests itself in different ways. At Enigma, our goal is to build a complete set of authoritative profiles of …
What’s not to love about data pipeline testing? Adding acceptance tests to your data pipelines makes them less likely to make mistakes and makes sure that enough quality checks are done on the data before it is sent to end users. Testing data pipelines involves two components of any data pipeline: data and code used …
Acceptance Testing For Data Pipelines: Expert Guide Read More »
What is lakeFS? lakeFS is a platform that helps data engineers build scalable and resilient data lakes running on object storage. It provides version control, branching, and merging capabilities for data at petabyte scale, on or off premises. lakeFS enables teams to collaborate and manage data effectively by applying engineering best practices to data management. …
lakeFS Product Offerings Overview: Open Source vs. Enterprise vs. Cloud Read More »
Introduction Last month, the lakeFS team decided to move from the decoupled security authentication and access control features to enable you to plug your own authentication and security mechanism. Consequently, the team decided to change the architecture to a pluggable one which enables you to choose your preference without being dependent on the lakeFS solution. …
Authorization (RBAC) in lakeFS: Step-by-Step Configuration Tutorial Read More »
Introduction lakeFS makes creating isolated environments for data ingestion instantaneous so you can run data ingestion jobs without impacting your production data and merge ingested data atomically to your production data instantaneously. This frees you from spending time on environment maintenance and makes it possible to create as many environments as needed. If ingested data …
The Airflow and lakeFS Integration: Step-by-Step Configuration Tutorial Read More »
Enterprises use more and more data as the foundation for their decisions and operations. The sheer number of digital goods that collect, analyze, and use data to feed decision-making algorithms in order to improve future services is also rapidly increasing. Because of this, data quality has become the most important asset for businesses in almost …
© Copyright 2022 | All Rights Reserved | Privacy Policy | Terms of Use
The annual State of Data Engineering Report is now available. Find out what’s new in 2023 -
Get the report