API OIDC authentication mechanism

[ErykKul]

What this PR does / why we need it:
It implements OIDC authentication mechanism for API usage/SPA

Which issue(s) this PR closes:

Closes #

Special notes for your reviewer:
This is very similar conceptually as the oauth2-proxy, as in IQSS/dataverse-frontend#504, however, it turns out that the needed functionality is already supported in Payara: https://docs.payara.fish/enterprise/docs/Technical%20Documentation/Public%20API/OpenID%20Connect%20Support.html
This PR eliminates any need for a proxy, etc. It turned out that the implementation is very simple and elegant i.m.h.o.

Suggestions on how to test this:
I have tried to configure the default dev docker-compose to work with it, however, it uses the unstable image, not the one built locally. I already have my own dev environment, so I did test it on mine. The configuration should be almost identical to the committed one (except maybe for the test user; I tried using the "nick" built-in user here).

How I did test it:

• go to http://localhost:8080/oidc/login (your port may be different, also, I configured the user "builtin-nick" with password "password", so you should probably use that, otherwise you need to configure your own user in the keycloack), it will redirect you to log in:

after clicking sign in it redirects you again and it shows you this JSON response:

• copy the "session" from that response, it will be needed in the next steps, in this case it was "2f55c31432963b5867964107c47d"
• That was it! You are logged in for the SPA/API calls in the browser! For example, go to http://localhost:8080/api/v1/users/:me

• Test that the API call is working without the browser with curl, replace the session-id with the session id copied in the previous steps and run:

curl -v --cookie "JSESSIONID=session-id" http://localhost:7008/api/v1/users/:me | jq .

For example:

[eryk@eryk-pc dataverse]$ curl -v --cookie "JSESSIONID=2f55c31432963b5867964107c47d" http://localhost:7008/api/v1/users/:me | jq .
{
  "status": "OK",
  "data": {
    "id": 4,
    "identifier": "@kulTestResearcher",
    "displayName": "KU Leuven Test Researcher",
    "firstName": "KU Leuven",
    "lastName": "Test Researcher",
    "email": "[email protected]",
    "superuser": false,
    "deactivated": false,
    "affiliation": "KU Leuven",
    "position": "Researcher",
    "persistentUserId": "kulTestResearcher",
    "createdTime": "2024-03-20T20:44:50Z",
    "lastLoginTime": "2024-10-03T13:54:09Z",
    "lastApiUseTime": "2024-10-01T13:11:20Z",
    "authenticationProviderId": "builtin"
  }
}

Does this PR introduce a user interface change? If mockups are available, please link/include them here:
No

Is there a release notes update needed for this change?:
Yes

Additional documentation:
https://docs.payara.fish/enterprise/docs/Technical%20Documentation/Public%20API/OpenID%20Connect%20Support.html

[coveralls]

coverage: 20.706% (-0.2%) from 20.869%
when pulling 004613f on authn-arch-api-oidc
into a0cb73d on develop.

[ErykKul]

I forgot to mention this: the log-in is a one time thing. The normal flow is to go to http://localhost:8080/api/v1/callback/session and only if you are not authenticated (I implemented returning better error message now), you go to http://localhost:8080/oidc/login.

@pdurbin thanks for the docker run command, I will try testing it with that i.s.o. my own dev environment.

[pdurbin]

Some initial feedback. I haven't done any real testing yet.

[pdurbin]

I'm just putting this here at the top but can we please get a release note snippet?

[pdurbin]

Docs please! (API Guide.)

[pdurbin]

Docs here too, please.

[pdurbin]

Another pass.

[pdurbin]

Can you please remind me why this change is necessary?

[ErykKul]

We talked about matching users from OIDC to Dataverse users multiple times at LIBIS. We feel like matching users by verified emails would be the best approach. The user with "[email protected]" does not exist in the dev env, so it did not work. Matching by email would mean that you can log in with google, facebook, whatever, as long as you use the same email.

[ErykKul]

I reverted it and implemented the regular UserRecordIdentifier lookup, however, user is not known:

I have also implemented the redirect to the first login page if user is not known:

After that, I can log in:

Is it as it supposed to be? I have also noticed that oauth tokens are stored in the DB. The side effect of creating a new user after OIDC authentication is that it inserts one token there...

[ErykKul]

I think I can just pass null as token data to prevent storing it in DB. I am not sure why we store them, I assume it is like the cache for bearer tokens? Something to investigate a little bit more.

[ErykKul]

I had now reimplemented the entire OIDC code with the new mechanism. In the process I had to drop the PKCE support, which makes sense i.m.o. since the entire OIDC is now handled by the Payara built-in feature and the frontends do not need to implement it anymore. Another unexpected outcome is that I had to reimplement the bearer token support. I feel like dropping it would make this PR even harder to accept. The final implementation is not extremely bad, it reuses the cache that was first meant for the PKCE support. However, it makes the bearer tokens being stored twice in the system: once by the sessions, and once by the cache for the bearer support. I still believe that not storing the cookies, but generating one with each request is not much different from using the bearer tokens (I would leave the bearer token support feature off). For example, when using curl and not storing the cookie, the requests look like this:

curl -v --cookie "JSESSIONID=short-session-id" http://localhost:8080/api/v1/users/:me | jq .

vs

curl -H "Authorization: Bearer very-long-bearer-token-encoded-in-base64" http://localhost:8080/api/v1/users/:me | jq .

I still need to reimplement the "first login" feature for the unknown users (when email address is not found in the DB) and debug the code. Nevertheless, the main ideas are already in the code.

[github-actions]

📦 Pushed preview images as

ghcr.io/gdcc/dataverse:authn-arch-api-oidc

ghcr.io/gdcc/configbaker:authn-arch-api-oidc

🚢 See on GHCR. Use by referencing with full name as printed above, mind the registry name.