Webinar with Seqera and CytoReason: Trace any model back to its data.
Introduction As 2022 is about to end, many engineering leaders use this time to reflect on the year that has passed and start planning ahead on the year upcoming. Data engineering teams are usually swamped with tasks and requirements, which are many times also accompanied by failures and issues that require immediate attention. Therefore using […]
A year ago, in November 2021, I took a bold decision to leave my role at Microsoft and join team lakeFS as VP Marketing. Upon joining I wrote an article on why I decided to join. But just before publishing the piece, I had a change of heart; it didn’t feel right to me. It
In the past year, words like recession, business slowdown and monetary cuttings are being heard more and more often. Not just in the economic press and in the media, these discussions are very much heard also in almost all companies – within boardrooms, in management meetings and when engaging with potential investors and customers. As
As the Covid-19 pandemic loosens its grip on the world, we’re all eager to start travelling and meeting in person again. The great news is that in persone conferences are back, so it’s time to make up for lost time! Attending conferences is a great way to learn, network, and engage with like-minded people. Not
A few weeks ago, I was looking at a dashboard in our internal BI system. It’s a simple system. Redash over PostgreSQL that has just a few hundreds of thousands of rows. I noticed a change in one of my favourite metrics that calculates the number of new installations since the beginning of the quarter.
Organizations have practically always needed data analytics, and they jumped on the analytics bandwagon as soon as the first computers appeared on the scene. In the 80s, businesses built data warehouses using relational databases as their decision-support systems (DSS). However, as companies generated more diverse data at high velocity, relational databases showed their limitations. This
Data Versioning at Scale: Solutions Overview Back when I was a 23-year-old student, I worked at an Israeli networking company as a BI analyst in the Operations department. My job revolved around modeling the company’s inventory which was quite costly and needed optimization. At some point, I attended a meeting of the company’s management. When
A year has passed since we shared the State of Data Engineering 2021. And since we released that article last May, not much has changed in the data landscape. In fact, we had discussions internally about whether we should even do an update for 2022. We kid. It was another year worthy of its own
Trends in the data industry, how lakeFS fits into the new data stack, and a personal story of why I chose to join lakeFS. Open Source Is Everywhere! For the past decade, open-source has made a home in the ever-evolving data stack zoo known as the Hadoop ecosystem. To keep everything in order, ZooKeeper was
Who needs a role model? When we first launched lakeFS in August of 2020, we asked ourselves a simple question: What does success look like? And how will we know we’re doing the right things to get there? Of course with thousands of installations, a thriving user community, active developer contributions, and exponential growth on
The idea About two years ago, we left SimilarWeb, where we had the privilege of managing some of the most interesting and complex data architecture projects in the world. The architecture was centered around a Data Lake – 7 Petabytes of data on Amazon S3. On top, hundreds of hourly, daily and monthly jobs running,
Let’s start with the obvious: the lakeFS project doesn’t exist in isolation. It belongs to a larger ecosystem of data engineering tools and technologies adjacent and complementary to the problems we are solving. What better way to visualize our place in this ecosystem, I thought, than by creating a cross-sectional LUMAscape to depict it. What’s
We use cookies to improve your experience and understand how our site is used.
Join our community of experts:
introduce yourself, share your knowledge and discover best practices from fellow peers
