lakeFS is an open source project that provides the object storage a manageability layer with version control capabilities such as branching, committing, merging and using pre-commit and pre-merge hooks. This allows data engineers to manage the data lake securely, just like the code repository they manage. We are on a mission to simplify the lives of data engineers building data-intensive applications.
We’re looking for an experienced Backend Engineer to join our team and help make lakeFS even more powerful.
What you'll do
BUILD OUR PRODUCT
- Develop high end open-source solutions to the technical challenges we will face when working to fulfill our mission using your deep knowledge of Spark and the Hadoop ecosystem
- Maintain the product in production to ensure the best possible service to our customers
- Collaborate with the product and lakeFS core development team to develop our offerings
- Refine “Big Data” concepts, like retention and garbage collection, in the world of “Big Data Versioning”
BUILD OUR TEAM
We are a trusted team. It is our responsibility to ensure you and others on the team all feel safe and trusted to do their jobs. This is a central value for us as the core developers of an open-source product.
Our job is to develop the system and also to maintain it in production, ensuring reliability, durability and performance so we can give our customers the best possible service
- 7+ years experience working on data engineering pipelines
- At least 3 years experience with Java/Scala and Spark
- Deep understanding of Spark’s internals (Query planning, Filesystem API, codegen, etc.)
- Experience with PySpark, Hive Metastore, Delta Lake, Apache Hudi or Apache Iceberg
- Strong analytical and problem-solving skills
- Passion for data engineering and for enabling others by making their data easier to access
About the company
Treeverse is the company behind the open source project lakeFS. It was founded by data engineering professionals (Dr. Einat Orr and Oz Katz) with a long track record of innovating and delivering bleeding edge technology, and is backed by top tier investors (NVP, Zeev Ventures, Dell Technologies Capital).
If you’re passionate about building powerful dev tools that will disrupt the world of Data & AI – you belong with us.