An upcoming release of lakeFS will no longer include user management with ACLs. The code currently used to manage users and groups will remain open source, but will move into a contrib/ directory within the lakeFS Git repository, to be used as reference by those looking to implement their own auth server.
Primer: Pluggable Auth* in lakeFS
With the release of version 0.98.0, lakeFS authentication and authorization changed from being part of the lakeFS monolithic application, to being a pluggable component, with a pluggable OpenAPI spec, allowing users and operators to implement their own authentication and authorization logic, connect it to their identity provider or existing user management systems.
The lakeFS release included a “simple” user and authorization management system – allowing multiple users to belong to different groups: readers, writers, super users and administrators.
Doing one thing and doing it well
Our focus and mission for lakeFS has always been to build the world’s most scalable data version control system – enabling engineering best practices for Data and AI. To do this, we believe we need to focus our efforts as much as we can on the core problems of data versioning.
With that in mind, we don’t want to sacrifice security. There’s inherent risk in building an authentication and authorization service, especially when needing to protect organizations’ most valuable assets: their data.
Over time, the burden of doing this well has increased. Doing so requires effort and ongoing maintenance. Dealing with user requests, reported bugs, performance improvements, and of course due response to security vulnerabilities (as we’re committed to per our Security Policy) takes away resources from that core mission stated above.
Moving ACLs out of core lakeFS
To reduce that burden and allow the core lakeFS team to focus on its mission, we’ve decided to move the ACL server outside of core lakeFS Open Source. This doesn’t mean we’re closing the code or removing it! It just means it will move, frozen in its current form, to a separate location in the lakeFS repository – to be used as a reference implementation for those looking to implement their own auth solution.
This means that the lakeFS binaries released will not include an authorization implementation and authentication will be reduced to a single user.
lakeFS operators will then have the choice of either:
- Building the reference ACL server – modifying or patching it as needed – and connecting that to their lakeFS server. Doing so will result in a system functionally equivalent to current lakeFS versions (v1.29.0 at the time of writing this) with multiple user management and ACL groups
- Implementing a custom server based on the OpenAPI specification, allowing any custom authentication and authorization required by the organization
- Opt into lakeFS Cloud or lakeFS Enterprise, both of which include a built-in authentication and authorization server, as detailed above, with full RBAC, OIDC and SAML implementations
- Of course, users running in small scale can continue to use lakeFS Open Source as is out of the box, with a single user
We’re here to help the community
We understand that removing features or functionality is never a popular decision, and that many organizations are using lakeFS successfully in production with multiple users and access controls in place. To assist, we are also publishing this ACL auth server example for lakeFS, which you can build and use with your Open Source lakeFS Deployment.
While we won’t be able to provide any official support for the reference implementation, we care about the community and we won’t leave you hanging – reach out to us – we are committed to making sure you continue to enjoy lakeFS.
