For an optimal experience, provide your email below and one of our lifeguards will send you a link to start swimming in the lake!
4 Metadata Management Challenges And How To Solve Them Modern data architectures support an increasing number and variety of business use cases. Product creation, tailored customer experiences, fraud detection, regulatory compliance, and data monetization are some examples. To enable such use cases, a data-driven business needs modern solutions for accessing, managing, and processing data. One …
Organizations need data governance for many reasons, not just to comply with a rising number of data privacy and protection rules, such as the GDPR of the European Union and the California Consumer Privacy Act (CCPA). A lack of it can cause more pain than a fine. One of the most impactful areas of data …
Data Governance: Guide to Enterprise Data Architecture Read More »
In the traditional setup, organizations had a centralized infrastructure team responsible for managing data ownership across domains. But product-led companies started to approach this matter a little differently. Instead, they distribute the data ownership directly among producers (subject matter experts) using a data mesh architecture. This is a concept originally presented by Zhamak Dehghani in …
Data Mesh Architecture: Guide to Enterprise Data Architecture Read More »
Guide to Enterprise Data Architecture Part 3 If you look at where companies keep their analytical data, you’ll quickly see that this space has split into two major architectural and technology stacks: data warehouses and data lakes. What are their defining characteristics? What factors should you take into account when choosing a data warehouse vs. …
Organizations can accomplish more with their data than ever before thanks to advances in analytical data processing and data democratization initiatives led by the spread of visualization tools, low-code and no-code solutions, and innovations like data mesh. Advances in compute power, innovative data processing methods, and broader cloud adoption have accelerated these trends, placing data …
Analytical Data: Guide to Enterprise Data Architecture Read More »
Data is a goldmine for every organization, no matter the industry. But to make the most of it, businesses need technology to maintain and manage transactional data like payments, inventory updates, and customer records. This is where OLTP databases come in. Online Transaction Processing (OLTP) databases are used to store and process large numbers of …
OLTP: Guide to Enterprise Data Architecture Part 1 Read More »
A step by step guide to running pipelines on Bronze, Silver and Gold layers with lakeFS Introduction The Medallion Architecture is a software design pattern that organizes a data pipeline into three distinct tiers based on functionality: bronze, silver, and gold. The bronze tier represents the core functionality of the system, while the silver and …
Version Control Data Pipelines Using the Medallion Architecture Read More »
lakeFS enabled us to streamline and run 200+ dbt models in production, increase data deployment velocity, efficiently reproduce ML experiments, increase productivity of the data teams and adhere to FDA compliance requirements
The last decade saw an unprecedented rise in the number of organizations that base their decisions and operations on data. The number of digital products that collect and process data and use it to fuel decision-making algorithms for enhancing future services is also growing at a very fast pace. That’s why data and data quality …
Clearing the mess – How to ensure data quality with versioning Read More »
Modern data engineering practices lead more and more organizations to a broader use of object stores. This happens due to the rising scale and complexity of the data that they manage – along with the growing variety of use cases that these data warehouses need to cater: from machine learning and algorithm development, to analytics …
5 Painful mistakes data engineers make, and how to avoid them Read More »
Create a Dev/Test Environment for Data Pipelines Using Spark and Python in this LIVE WEBINAR -
Register here
Join our community of experts:
introduce yourself, share your knowledge and discover best practices from fellow peers
