Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Auth: AD auth broken >v0.6.2 #351

Open
4 tasks done
IncandescentChrysalis opened this issue May 6, 2024 · 22 comments
Open
4 tasks done

Auth: AD auth broken >v0.6.2 #351

IncandescentChrysalis opened this issue May 6, 2024 · 22 comments
Labels
area/auth App authentication related issues scope/backend Related to backend changes status/triage/completed Automatic triage completed type/bug Something isn't working

Comments

@IncandescentChrysalis
Copy link

Issue submitter TODO list

  • I've looked up my issue in FAQ
  • I've searched for an already existing issues here
  • I've tried running main-labeled docker image and the issue still persists there
  • I'm running a supported version of the application which is listed here

Describe the bug (actual behavior)

As hinted in #254 (comment), LDAP backend towards Active Directory in non-RBAC mode does not seem to work anymore.
The last version this is still effective with the used configuration is provectus/kafka-ui's v0.6.2.

Expected behavior

LDAP authentication is said to be seperate from RBAC par #254 (comment), and as such should continue working whatever state of implementation RBAC is.
RBAC for Active Directory is considered not implemented as of yet.

Your installation details

Previous test made long ago with Docker image compiled from provectus/kafka-ui's v0.7.1
Current test made with Docker image compiled from kafbat/kafka-ui's v1.0.0

Steps to reproduce

Minimal set of environment variables to reproduce:

#LOGGING_LEVEL_ROOT: debug

AUTH_TYPE: "LDAP"
SPRING_LDAP_URLS: "ldaps://<AD URL>"
SPRING_LDAP_ADMINUSER: "CN=<CN>,OU=<OU1>,DC=<DC>"
SPRING_LDAP_ADMINPASSWORD: "<admin password>"

SPRING_LDAP_USERFILTER_SEARCHBASE: "OU=<OU2>,DC=<DC>"
SPRING_LDAP_USERFILTER_SEARCHFILTER: "<AD filter>"

Screenshots

No response

Logs

Existing session from HTTP authentication is not resumed; those are the logs after filling up the Web auth form:

kafka-ui-config_kube-kafka-ui-demo-1  | 2024-05-06 07:39:52,977 DEBUG [reactor-http-epoll-4] o.s.h.c.FormHttpMessageReader: [613b0092-5] Read form fields [username, password] (content masked)
kafka-ui-config_kube-kafka-ui-demo-1  | 2024-05-06 07:39:52,982 DEBUG [boundedElastic-1] o.s.s.l.a.BindAuthenticator: Failed to bind with any user DNs []
kafka-ui-config_kube-kafka-ui-demo-1  | 2024-05-06 07:39:52,991 DEBUG [boundedElastic-1] o.s.s.w.s.a.AuthenticationWebFilter: Authentication failed: Les identifications sont erronées
kafka-ui-config_kube-kafka-ui-demo-1  | 2024-05-06 07:39:52,992 DEBUG [boundedElastic-1] o.s.s.w.s.DefaultServerRedirectStrategy: Redirecting to '<path>/login?error'

Additional context

Working logs (v0.6.2):

kafka-ui-config_kube-kafka-ui-demo-1  | 2024-05-06 07:33:46,381 DEBUG [boundedElastic-2] o.s.s.l.a.BindAuthenticator: Failed to bind with any user DNs []
[…]
kafka-ui-config_kube-kafka-ui-demo-1  | 2024-05-06 07:33:46,580 DEBUG [boundedElastic-2] o.s.l.c.s.AbstractContextSource: Got Ldap context on server 'ldaps://<AD>'
kafka-ui-config_kube-kafka-ui-demo-1  | 2024-05-06 07:33:46,621 DEBUG [boundedElastic-2] o.s.s.l.SpringSecurityLdapTemplate: Found DN: CN=<CN>,OU=<OU>,DC=<DC>
kafka-ui-config_kube-kafka-ui-demo-1  | 2024-05-06 07:33:46,623 DEBUG [boundedElastic-2] o.s.s.l.s.FilterBasedLdapUserSearch: Found user '<user>', with FilterBasedLdapUserSearch [searchFilter=<AD filter>; searchBase=OU=<OU>,DC=<DC>; scope=subtree; searchTimeLimit=0; derefLinkFlag=false ]
[…]
kafka-ui-config_kube-kafka-ui-demo-1  | 2024-05-06 07:33:46,860 DEBUG [boundedElastic-2] o.s.l.c.s.AbstractContextSource: Got Ldap context on server 'ldaps://<AD>'
kafka-ui-config_kube-kafka-ui-demo-1  | 2024-05-06 07:33:46,861 DEBUG [boundedElastic-2] o.s.s.l.a.BindAuthenticator: Bound cn=<CN>,ou=<OU>,dc=<DC>
kafka-ui-config_kube-kafka-ui-demo-1  | 2024-05-06 07:33:46,862 DEBUG [boundedElastic-2] o.s.s.l.u.LdapUserDetailsMapper: Mapping user details from context with DN cn=<CN>,ou=<OU>,dc=<DC>
kafka-ui-config_kube-kafka-ui-demo-1  | 2024-05-06 07:33:46,862 DEBUG [boundedElastic-2] o.s.s.l.a.LdapAuthenticationProvider: Authenticated user
@IncandescentChrysalis IncandescentChrysalis added status/triage Issues pending maintainers triage type/bug Something isn't working labels May 6, 2024
@kapybro kapybro bot added status/triage/manual Manual triage in progress area/auth App authentication related issues status/triage/completed Automatic triage completed and removed status/triage Issues pending maintainers triage labels May 6, 2024
@Haarolean Haarolean changed the title Non-RBAC Active Directory (LDAP) auth broken > v0.6.2 Auth: AD auth broken >v0.6.2 May 6, 2024
@Haarolean
Copy link
Member

Hi, you're missing two essential properties:

     OAUTH2.LDAP.ACTIVEDIRECTORY: true
     OAUTH2.LDAP.ACTIVEDIRECTORY.DOMAIN: "memelord.lol"

Can you try it with the suggested properties?

Copy link

kapybro bot commented May 6, 2024

Further user feedback is requested. Please reply within 7 days or we might close the issue.

@IncandescentChrysalis
Copy link
Author

I tried with those.
By domain, I suppose you meant the capitalised name of the AD notion, no the domain name used as combination of DC

Here is what I got:

kafka-ui-config_kube-kafka-ui-demo-1  | 2024-05-07 15:03:47,334 DEBUG [Thread-9] j.e.security: X509Certificate: Alg:SHA256withRSA, […], Cert Id:334072978,  […]
kafka-ui-config_kube-kafka-ui-demo-1  | 2024-05-07 15:03:47,335 DEBUG [Thread-9] j.e.security: X509Certificate: Alg:SHA256withRSA, […], Cert Id:-996611390, […]
kafka-ui-config_kube-kafka-ui-demo-1  | 2024-05-07 15:03:47,340 DEBUG [Thread-9] j.e.security: ValidationChain: 1128928943, -996611390
kafka-ui-config_kube-kafka-ui-demo-1  | 2024-05-07 15:03:47,345 DEBUG [Thread-9] j.e.security:  TLSHandshake: <AD>:636, TLSv1.3, TLS_AES_256_GCM_SHA384, -996611390
kafka-ui-config_kube-kafka-ui-demo-1  | 2024-05-07 15:03:47,497 DEBUG [boundedElastic-1] o.s.s.l.a.a.ActiveDirectoryLdapAuthenticationProvider: Authentication for <user>@@null failed:javax.naming.AuthenticationException: [LDAP: error code 49 - 80090308: LdapErr: DSID-0C09050E, comment: AcceptSecurityContext error, data 52e, v4f7c]
kafka-ui-config_kube-kafka-ui-demo-1  | 2024-05-07 15:03:47,499 INFO  [boundedElastic-1] o.s.s.l.a.a.ActiveDirectoryLdapAuthenticationProvider: Active Directory authentication failed: Supplied password was invalid

I N-checked the password which is the one working in v0.6.2


On a side note, I saw those environment variables in https://github.com/kafbat/kafka-ui/blob/main/documentation/compose/ui-ldap.yaml, but is there other documentation about them?

Copy link

kapybro bot commented May 7, 2024

Thanks for the additional feedback! We'll get back to your issue soon.

@Haarolean
Copy link
Member

@IncandescentChrysalis

As this issue is exclusively AD related and is not affecting LDAP, reproducing it becomes quite complicated, as we don't have any AD set up. Let's try to find the exact commit which breaks this for you.

Could you try these two?
provectuslabs/kafka-ui:541e4018ec251ea371fd64b198b916b9fd198d99
provectuslabs/kafka-ui:744bdb32a310306eefe8641923d712db697b1c70

Let me know if it works with the first one and doesn't with the latter.

Copy link

kapybro bot commented May 7, 2024

Further user feedback is requested. Please reply within 7 days or we might close the issue.

@IncandescentChrysalis
Copy link
Author

IncandescentChrysalis commented May 13, 2024

Something is worth mentioning about the two supplemental headers you indicated to me:

  • OAUTH2.LDAP.ACTIVEDIRECTORY
  • OAUTH2.LDAP.ACTIVEDIRECTORY.DOMAIN

Using them on our image compiled from provectus/kafka-ui v0.6.2 breaks authentication. It is working without them.
Are you sure about those?


I tested both:

  • provectuslabs/kafka-ui:541e4018ec251ea371fd64b198b916b9fd198d99
  • provectuslabs/kafka-ui:744bdb32a310306eefe8641923d712db697b1c70

with and without those extra headers, and authentication failed everytime.

Copy link

kapybro bot commented May 13, 2024

Thanks for the additional feedback! We'll get back to your issue soon.

@Haarolean
Copy link
Member

This makes little to no sense honestly.
541e4018ec251ea371fd64b198b916b9fd198d99 dates back to this issue, where the changes have been tested by the other users and it's confirmed that it did work back then.
744bdb32a310306eefe8641923d712db697b1c70, on the other hand, dates back to this issue, which is about supporting LDAP in RBAC. This is one of the only PRs that could affect this and it was my best guess.

But having both not working makes no sense, even less does the fact that adding properties required to make AD work it make it worse in your case.

As setting up AD for debugging purposes is virtually impossible, and adding the fact that this is the first time I'm receiving such a report, we'd need either a test AD server provided to understand which changes might've affected it (if any) and/or other user reports on this matter. I'm putting this on hold until we get at least one of those. Please let me know if there's anything else I could do here.

@Haarolean Haarolean added scope/backend Related to backend changes status/pending Further information is requested and removed status/triage/manual Manual triage in progress labels May 13, 2024
@IncandescentChrysalis
Copy link
Author

IncandescentChrysalis commented May 14, 2024

It might simply come down to the fact that the two environment variables you indicated to me are improperly configured on my end.

I wrote earlier what I thought they should contain, based on guess work from their name only. The only other mention which would serve the purpose of docs in the codebase I found was a Docker Compose definition, not very verbose in itself.
Maybe more insights on what is expected there could help.

What strikes me most is that they are breaking the 0.6.2 provectus/kafka-ui, otherwise working, environment.
I suppose it is not, but is this expected?


Considering this issue is on hold, I will send that matter further in our backlog too.

@Haarolean Haarolean self-assigned this Jul 3, 2024
@Haarolean Haarolean removed the status/pending Further information is requested label Jul 3, 2024
@Haarolean
Copy link
Member

One more case of AD auth broken after 0.6.2
https://discord.com/channels/897805035122077716/1257657156338057276/1257956792890163251

@tdudlak4918ab
Copy link

tdudlak4918ab commented Jul 29, 2024

Hey, I am struggling here, what's the correct configuration for 0.7.2 provectus kafka-ui for AD?
I am using the yaml config file.

oauth2:
  ldap:
    activeDirectory: false
    activeDirectory.domain: my.domain.net

is this section correct do i miss something? the ldaps connection is in

spring:
  ldap:
    urls: "ldaps://x.my.domain.net
    ...

@IncandescentChrysalis
Copy link
Author

@Haarolean I got a solution for you!
Here is a test Docker Active Directory environment with Kafka UI in which to test authentication: IncandescentChrysalis/KafkaUI-ActiveDirectory

I can reproduce the problem with it:

  • Authentication works with provectuslabs/kafka-ui:v0.6.2
  • Authentication fails with:
    • provectuslabs/kafka-ui:v0.7.1
    • kafbat/kafka-ui:v1.0.0

@IncandescentChrysalis
Copy link
Author

IncandescentChrysalis commented Sep 18, 2024

To be crystal clear, authentication works in 0.6.2 and is broken since 0.7.0: I originally reported 0.7.1, which is consequentially true, but omits part of the problem.

I found the commit breaking authentication between 0.6.2 & 0.7.0: it implements provectus/kafka-ui#3700

Several things are being done there:

  1. RBAC for Active Directory authentication, replacing the existing stub (not being used)
  2. Change of the way the Spring configuration is being loaded
  3. Change from HTTP Basic to form-based frontend

The first item is important: as shown in my previous comment, despite interacting with an Active Directory instance, th protocol being used is LDAP.
I strongly suspect LDAP communication with a pure LDAP authentication backend to be broken as the result since then… but I would then surprised about the lack of issues about that: does that mean no-one uses it so it went that unnoticed?

I am still fiddling with the details of the breaking commit, trying to understand what exactly broke things.
Not being fluent with the Spring framework (hack: Java!), I still see a lot of decorators popped up on top of the way configuration is loaded, all modifying the authentication flow.

If your expert eyes see something obvious to you, that could greatly speed things up!

@IncandescentChrysalis
Copy link
Author

IncandescentChrysalis commented Sep 18, 2024

I found out why my LDAP authentication was now failing.

Up to 0.6.2, the Spring LDAP user filters were loaded with:

  @Value("${spring.ldap.userFilter.searchBase:#{null}}")
  private String userFilterSearchBase;
  @Value("${spring.ldap.userFilter.searchFilter:#{null}}")
  private String userFilterSearchFilter;

meaning the corresponding environment variables were:

SPRING_LDAP_USERFILTER_SEARCHBASE
SPRING_LDAP_USERFILTER_SEARCHFILTER

Starting with 0.7.0, through the aforementioned commit, they are loaded with:

@ConfigurationProperties("spring.ldap")
@Data
public class LdapProperties {

  […]
  private String userFilterSearchBase;
  private String userFilterSearchFilter;

meaning the corresponding environment variables are:

SPRING_LDAP_USERFILTERSEARCHBASE
SPRING_LDAP_USERFILTERSEARCHFILTER

Those breaking changes were not documented in the 0.7.0 release.


As per the actual Active Directory configuration, refer to @tdudlak4918ab's question: so far I have never used RBAC.

@Haarolean
Copy link
Member

@IncandescentChrysalis

meaning the corresponding environment variables are:

not really, taking spring boot's relaxed binding rules into consideration, both variants are equal. We have this LDAP compose which works perfectly fine.

I'll take a look at your example repo once I get some spare time for this, thank you.

@Haarolean
Copy link
Member

@IncandescentChrysalis I couldn't quite run your compose on arm64 host:

activedirectory-1  | ERROR(<class 'samba.provision.ProvisioningError'>): Provision failed - ProvisioningError: Your filesystem or build does not support posix ACLs, which s3fs requires.  Try the mounting the filesystem with the 'acl' option.
activedirectory-1  |   File "/usr/lib/python3/dist-packages/samba/netcmd/domain.py", line 493, in run
activedirectory-1  |     result = provision(self.logger,
activedirectory-1  |              ^^^^^^^^^^^^^^^^^^^^^^
activedirectory-1  |   File "/usr/lib/python3/dist-packages/samba/provision/__init__.py", line 2325, in provision
activedirectory-1  |     provision_fill(samdb, secrets_ldb, logger, names, paths,
activedirectory-1  |   File "/usr/lib/python3/dist-packages/samba/provision/__init__.py", line 1965, in provision_fill
activedirectory-1  |     setsysvolacl(samdb, paths.netlogon, paths.sysvol, paths.root_uid,
activedirectory-1  |   File "/usr/lib/python3/dist-packages/samba/provision/__init__.py", line 1677, in setsysvolacl
activedirectory-1  |     raise ProvisioningError("Your filesystem or build does not support posix ACLs, which s3fs requires.  "
activedirectory-1  | + '[' -f /var/lib/samba/.setup ']'
activedirectory-1  | + echo '[ERROR] Samba is not setup yet, which should happen automatically. Look for errors!'
activedirectory-1  | [ERROR] Samba is not setup yet, which should happen automatically. Look for errors!
activedirectory-1  | + exit 127
activedirectory-1 exited with code 127

is that fixable or should I get an amd64 machine for this?

@IncandescentChrysalis
Copy link
Author

IncandescentChrysalis commented Sep 19, 2024

not really, taking spring boot's relaxed binding rules into consideration, both variants are equal. We have this LDAP compose which works perfectly fine.

I would kindly suggest to put the theory to the test (o:
Those examples clearly misled me and made me lose almost a year on that problem.

I confirm what I wrote in my comment, all my tests concur on that, and I even spent a lot of time debugging the code on the commit breaking the behaviour: those variables are not defined post-change. The configuration environment variables syntax change I highlighted (haven't tested the configuration file approach) is mandatory > 0.6.2.

It seems maybe the @Value decorators would have been necessary to keep for this configuration to be loaded as before, but would it break the new configuration syntax? To test if interested.
Those decorators are a proof Spring was not naturally loading them anyway.

I only managed to make the commit working on the old configuration syntax by modifying the LdapProperties class with a new userFilter property of a new UserFile type, which class contained searchBase & searchFilter properties.
I could always push a branch with that example change if you want what I described as the old syntax to work.

@IncandescentChrysalis
Copy link
Author

is that fixable or should I get an amd64 machine for this?

Make sure to have a Docker environment matching your kernel/architecture.

@IncandescentChrysalis
Copy link
Author

IncandescentChrysalis commented Sep 19, 2024

I testes futher with 1.0.0, since the behaviour seems to have broken with 0.6.2 (not documented), but consistent since 0.7.0.

This works indeed per the example you provided:

SPRING_LDAP_USER_FILTER_SEARCH_BASE
SPRING_LDAP_USER_FILTER_SEARCH_FILTER

As stated before, this too:

SPRING_LDAP_USERFILTERSEARCHBASE
SPRING_LDAP_USERFILTERSEARCHFILTER

But I confirm this does not:

SPRING_LDAP_USERFILTER_SEARCHBASE
SPRING_LDAP_USERFILTER_SEARCHFILTER

I am no pro at Spring nor its magic, so I'll leave you to understand what the Hell is going on 🤣

If you are interested, I know what code change could be done to support that <= 0.6.2 syntax in order to be backwards compatible with <= 0.6.2 (but accepting with syntaxes might be harder).
I would kindly insist on the fact people coming over to your docs with a <= 0.6.2 configuration should find some help on how to adapt it to work >= 0.7.0.

@Haarolean
Copy link
Member

Seems like the unexpected changes have been discussed here previously.

Regarding the documentation:

  1. currently documentation is not versioned and we store only the actual info
  2. we do not expect people to come over to kafbat with k-ui version <0.7, as 0.6 is at least one year old
  3. I wish I could update 0.7.0 release notes, but that's in the old repo which I don't have an access to
  4. our docs are open source as well, so perhaps if you find this important, feel free to raise a PR noting these unexpected changes somewhere

@Haarolean
Copy link
Member

Make sure to have a Docker environment matching your kernel/architecture.

Can you elaborate? I'm running the compose you provided which is built locally (for the ad part), so there's no image architecture to match.

@Haarolean Haarolean removed their assignment Sep 24, 2024
@Haarolean Haarolean moved this to Backlog in Housekeeping Sep 24, 2024
@Haarolean Haarolean removed this from Housekeeping Oct 17, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
area/auth App authentication related issues scope/backend Related to backend changes status/triage/completed Automatic triage completed type/bug Something isn't working
Projects
None yet
Development

No branches or pull requests

3 participants