3094bbc broke fragments checking: InvalidBaseJoin

[autoantwort]

Before and after:

➜  lychee git:(87d5b56) cargo run -- ../network-documentation/services/zfs.md --include-fragments --base ../network-documentation/ --mode=emoji --timeout=5
    Finished `dev` profile [unoptimized + debuginfo] target(s) in 0.87s
     Running `target/debug/lychee ../network-documentation/services/zfs.md --include-fragments --base ../network-documentation/ --mode=emoji --timeout=5`
  6/6 ━━━━━━━━━━━━━━━━━━━━ Finished extracting links                                                                                                                                                           
🔍 6 Total (in 0s) ✅ 6 OK 🚫 0 Errors
➜  lychee git:(87d5b56) git checkout -
Previous HEAD position was 87d5b56 Box Octocrab error as it is too large (#1543)
HEAD is now at 3094bbc Add support for relative links (#1489)
➜  lychee git:(3094bbc) cargo run -- ../network-documentation/services/zfs.md --include-fragments --base ../network-documentation/ --mode=emoji --timeout=5
    Finished `dev` profile [unoptimized + debuginfo] target(s) in 16.22s
     Running `target/debug/lychee ../network-documentation/services/zfs.md --include-fragments --base ../network-documentation/ --mode=emoji --timeout=5`
[WARN] Error creating request: InvalidBaseJoin("#scrubs-wichtig")
[WARN] Error creating request: InvalidBaseJoin("#unterschiedliche-platten-in-einem-vdevpool")
  4/4 ━━━━━━━━━━━━━━━━━━━━ Finished extracting links                                                                                                                                                            
🔍 4 Total (in 0s) ✅ 4 OK 🚫 0 Errors

You can also reproduce that with the fragments folder in this repo:
Before: cargo run -- fixtures/fragments/ --include-fragments --mode=emoji --base .

  15/15 ━━━━━━━━━━━━━━━━━━━━ Finished extracting links                                                                                                                                                          Issues found in 2 inputs. Find details below.

[fixtures/fragments/file1.md]:
❌ file:///Users/leanderSchulten/git_projekte/lychee/fixtures/fragments/file2.md#missing-fragment
❌ file:///Users/leanderSchulten/git_projekte/lychee/fixtures/fragments/file1.md#missing-fragment

[fixtures/fragments/file.html]:
❌ file:///Users/leanderSchulten/git_projekte/lychee/fixtures/fragments/file.html#in-the-end

🔍 15 Total (in 0s) ✅ 12 OK 🚫 3 Errors

After:

⠁ 0/0 ━━━━━━━━━━━━━━━━━━━━ Extracting links                                                                                                                                                                     [WARN] Error creating request: InvalidBaseJoin("file1.md#lets-wear-a-hat-être")
[WARN] Error creating request: InvalidBaseJoin("#fragment-2")
[WARN] Error creating request: InvalidBaseJoin("file2.md#fragment-1")
[WARN] Error creating request: InvalidBaseJoin("#missing-fragment")
[WARN] Error creating request: InvalidBaseJoin("file2.md#missing-fragment")
[WARN] Error creating request: InvalidBaseJoin("#code-heading")
[WARN] Error creating request: InvalidBaseJoin("#explicit-fragment")
[WARN] Error creating request: InvalidBaseJoin("file.html#a-word")
[WARN] Error creating request: InvalidBaseJoin("file2.md#custom-id")
[WARN] Error creating request: InvalidBaseJoin("#kebab-case-fragment")
[WARN] Error creating request: InvalidBaseJoin("#kebab-case-fragment-1")
[WARN] Error creating request: InvalidBaseJoin("empty_file#fragment")
[WARN] Error creating request: InvalidBaseJoin("file1.md#fragment-1")
[WARN] Error creating request: InvalidBaseJoin("#in-the-beginning")
[WARN] Error creating request: InvalidBaseJoin("#in-the-end")
  0/0 ━━━━━━━━━━━━━━━━━━━━ Finished extracting links                                                                                                                                                            🔍 0 Total (in 0s) ✅ 0 OK 🚫 0 Errors

[trask]

I suspect this issue relates to --base modeling Base URL, while probably what you want in this case is the upcoming --root-dir

[MichaIng]

Not only fragment URLs are broken, but all relative URL checks:

   [WARN ] Error creating request: InvalidBaseJoin("../../software/vpn/")
   [WARN ] Error creating request: InvalidBaseJoin("../../software/webserver_stack/")
   [WARN ] Error creating request: InvalidBaseJoin("../")
   [WARN ] Error creating request: InvalidBaseJoin("../software_installation/")
   [WARN ] Error creating request: InvalidBaseJoin("../system_configuration/")
   [WARN ] Error creating request: InvalidBaseJoin("../system_maintenance/")
   [WARN ] Error creating request: InvalidBaseJoin("./")
   [WARN ] Error creating request: InvalidBaseJoin("#useful-dietpi-shell-functions")
   [WARN ] Error creating request: InvalidBaseJoin("#dietpi-banner")
   [WARN ] Error creating request: InvalidBaseJoin("#dietpi-cpu-info")

We use --base build with the HTML file structure all in ./build. Previously an URL ../../software/vpn/ in the HTML file ./build/docs/releases/v9_7/index.html was correctly resolved to the directory ./build/docs/software/vpn/. I actually wonder why --base was/is required at all in such case, as relative URLs can be always just resolved relative to the directory of the checked document. The help/docs say --base is for relative URLs, but isn't such needed for absolute URLs only, at least when checking for local files and directories?
EDIT: Of is the wording "relative URL" used for absolute file paths, i.e. relative to the base URL/domain, not relative to the directory structure?

However, when checking from any directory on the system and aiming to check relative path URLs against an online https://-like URLs, indeed the solution to introduce --root-dir and rename --base to --base-url seems the best solution:

• --root-dir should allow to resolve all absolute path URLs like /path/to to local files and directories relative to this root directory.
• --base-url allows to resolve absolute path URLs against online/scheme URLs, and in combination with --root-dir also relative URLs, resolving the path first relative to the --root-dir, then appending it to the given --base-url.

Checking relative path URLs, which include fragment-only URLs, against local files/dirs however should not require any of the two options, but just checking for a local file/dir relative to the directory of the checked document.

Based on #1489 and the linked issues, where relative path URLs never checked before? I though they were 🤔.
EDIT: Ah I see now in #1480 the problem:

• OP uses a relative path, which was always resolved against the given --base URL, not taking into account the possible path elements of the input URL.
• I am not sure why, but the fix broke relative local file resolution instead, including pure fragment URLs. Is the --base supposed to be given differently after the recent change, or is it a bug?