Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

RBAC: Error in maintaining connection between kafka-ui and keycloak behind AWS NLB (HTTP ERROR 500 Connection reset:) #4388

Open
4 tasks done
kivadratik-1 opened this issue Feb 23, 2024 · 6 comments · May be fixed by #4392
Labels
status/triage Issues pending maintainers triage type/bug Something isn't working

Comments

@kivadratik-1
Copy link

kivadratik-1 commented Feb 23, 2024

Issue submitter TODO list

  • I've looked up my issue in FAQ
  • I've searched for an already existing issues here
  • I've tried running master-labeled docker image and the issue still persists there
  • I'm running a supported version of the application which is listed here

Describe the bug (actual behavior)

The problem occurs when installing kafka-ui with RBAC(keycloak). Most likely due to java web client does not send keepalive packets to maintain tcp connection, and without this AWS loadbalancer with timeout 350s silently closes the tcp connection. The web client in kafka-ui only learns about this when it tries to send a packet again after 350 seconds and got Connection reset:
Screenshot 2024-02-23 at 15 48 09
Similar behavior is described for java web clients Reactor Netty Reference Guide in which option(ChannelOption.SO_KEEPALIVE, true) must be set. But unfortunately in the kafka-ui client implementation this option is not present

Expected behavior

No response

Your installation details

  1. 56fa824 v0.7.1

Steps to reproduce

Set up using the scheme kafka-ui(RBAC-oauth2) <--> AWS NLB <--> keycloak

Screenshots

No response

Logs

ERROR [reactor-http-nio-4] o.s.b.a.w.r.e.AbstractErrorWebExceptionHandler: [ххх-ххх] 500 Server Error for HTTP GET "/login/oauth2/code/keycloak?state=ххх kafka-ui org.springframework.web.reactive.function.client.WebClientRequestException: Connection reset

Additional context

No response

@kivadratik-1 kivadratik-1 added status/triage Issues pending maintainers triage type/bug Something isn't working labels Feb 23, 2024
Copy link

Hello there kivadratik-1! 👋

Thank you and congratulations 🎉 for opening your very first issue in this project! 💖

In case you want to claim this issue, please comment down below! We will try to get back to you as soon as we can. 👀

@Haarolean
Copy link
Contributor

@kivadratik-1 can you elaborate on why kafka-ui should send keepalive packets at all in a non-keepalive session scenario? And why has it occur every 350s? I doubt an authentication session should elapse this long.

@kivadratik-1
Copy link
Author

kivadratik-1 commented Feb 26, 2024

It's hard to say, I haven't investigated your code in detail, but judging by the behavior of ui, I assume that when the token expires, ui tries to go get a new token (or refresh) through the same tcp session it opened during the last attempt and receives rst from nlb, as a result ui gives 500. Actually we tried to fix this behavior and locally made a patch that helped us to overcome this problem. Will send you pr

@mike-kolt mike-kolt linked a pull request Feb 29, 2024 that will close this issue
13 tasks
@mike-kolt
Copy link

mike-kolt commented Mar 4, 2024

I assume that when the token expires, ui tries to go get a new token (or refresh) through the same tcp session it
opened during the last attempt and receives rst from nlb, as a result ui gives 500

Yes, kafka-ui uses spring-boot-security for authentification purpose and follows described behavior.
I created PR to fix this issue by using properties for jetty http client
#4392
@Haarolean I would be glad to receive your comments to provide additional support.

@Haarolean
Copy link
Contributor

@mike-kolt I believe my comments won't help much as I won't be able to do anything with your PR (see #4255) :)
But I'll be happy to discuss this if you raise the same issue/PR in the other repo, you can find the link in my profile :)

@rebo0tt
Copy link

rebo0tt commented Dec 18, 2024

@kivadratik-1 Could you please share your solution?

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
status/triage Issues pending maintainers triage type/bug Something isn't working
Projects
None yet
Development

Successfully merging a pull request may close this issue.

4 participants