How To Get Started with lakeFS Enterprise: Step-by-Step Tutorial

What is lakeFS Enterprise?

lakeFS Enterprise is a commercially-supported version of lakeFS, offering additional features and functionalities that meet the needs of organizations from a production-grade system.

Why did we build lakeFS Enterprise?

lakeFS Enterprise was built for organizations that require the support, security standards and features required of a production-grade system and are not using public clouds, hence they cannot use lakeFS Cloud.

What is the value of using lakeFS Enterprise?

1. Support: the lakeFS team is committed to supporting you under an SLA for both issues and product enhancements.
2. Security: Full support for a suite of security features and additional lakeFS functionality.

What security features does lakeFS Enterprise provide?

With lakeFS Enterprise you’ll receive access to the security package containing the following features:

1. A rich Role-Based Access Control (RBAC) permission system that allows for fine-grained control by associating permissions with users and groups, granting them specific actions on specific resources. This ensures data security and compliance within an organization.
2. To easily manage users and groups, lakeFS Enterprise provides SSO integration (including support for SAML, OIDC, ADFS, Okta, and Azure AD), supporting existing credentials from a trusted provider, eliminating separate logins.
3. lakeFS Enterprise supports SCIM for automatically provisioning and deprovisioning users and group memberships to allow organizations to maintain a single source of truth for their user database.
4. STS Auth offers temporary, secure logins using an Identity Provider, simplifying user access and enhancing security.
5. Authentication with AWS IAM Roles allows authentication using AWS IAM roles instead of lakeFS credentials, removing the need to maintain static credentials for lakeFS Enterprise users running on AWS.
6. Auditing provides a detailed action log of events happening within lakeFS, including who performed which action, on which resource – and when.

What additional functionality does lakeFS Enterprise provide?

1. lakeFS Mount allows users to virtually mount a remote lakeFS repository onto a local directory. Once mounted, users can access the data as if it resides on their local filesystem, using any tool, library, or framework that reads from a local filesystem.
2. Transactional Mirroring – allows replicating lakeFS repositories into consistent read-only copies in remote locations.

lakeFS Enterprise Sample

The lakeFS Enterprise Sample is the quickest way to experience the value of lakeFS Enterprise features in a containerized environment. This Docker-based setup is ideal if you want to easily interact with lakeFS without the hassle of integration and experiment with lakeFS without writing code.

By running the lakeFS Enterprise Sample, you will be getting a ready-to-use environment including the following containers:

• lakeFS
• Fluffy (which includes lakeFS Enterprise features)
• Postgres: used by lakeFS and Fluffy as a shared Key-Value (KV) store
• MinIO container: used as the storage connected to lakeFS for this sample but lakeFS supports any other Object Storage.
• Jupyter notebooks setup: Pre-populated with notebooks that demonstrate lakeFS Enterprise’ capabilities
• Apache Spark: this is useful for interacting with data you’ll manage with lakeFS

Prerequisites

1. Request a Trial License: To start a 30-day free trial, please contact us. You will be granted a token that allows downloading a Docker image that includes lakeFS Enterprise features.
2. Docker and Git software installed on your computer.

Step 1 – Clone the Git repository

git clone https://github.com/treeverse/lakeFS-samples.git
cd lakeFS-samples/02_lakefs_enterprise

Step 2 – Login to Dockerhub

Login to Treeverse Dockerhub by using the granted token so Fluffy proprietary image can be retrieved:

docker login -u externallakefs

Step 3 – Run Docker container

docker compose up

Step 4 – Access lakeFS UI

Once the Docker containers are up and running, open lakeFS UI (http://localhost:8084/) and login using Access Key ID AKIAIOSFOLKFSSAMPLES and Secret Access Key wJalrXUtnFEMI/K7MDENG/bPxRfiCYEXAMPLEKEY

Step 5 – Access MinIO UI

MinIO is used as the storage connected to lakeFS for this sample but lakeFS supports any other Object Storage.

Open MinIO UI (http://localhost:9005/) and login using Username minioadmin and Password minioadmin

Step 6 – Run Jupyter Notebook Samples

Open the Jupyter Notebook UI (http://localhost:8894):

Open 00_index.ipynb notebook to explore and run different samples:

Step 7 – Run RBAC Sample Notebook

Also, check out the RBAC demo notebook and the blog to see lakeFS Enterprise RBAC capabilities in action.

Step 8 – Remove Docker container

Once you’ve finished, run the following to remove all the containers:

docker compose down

Summary

In this tutorial, we outlined the steps to install lakeFS Enterprise quickly and to run RBAC sample notebook.