fix: symlinks handling

[2ZeroSix]

fixes: #3143

• reverts to old behavior: symlinks to directories are handled as copy-paste Pub does not support publishing packages with directory symlinks #3143
• fix of pubignore/gitignore:
  • allow non-resolving symlinks if they are ignored
  • don't warn on ignored directory symlinks

[jonasfj]

I'm a bit unsure if we want to support this, how does it handle cycles?
What is the need for symlink directories?

Might it not be better to just ask advanced users to copy files if they need to include them in multiple locations. They can wrap their dart pub publish command with a bash script living in tool/ -- it's ugly, but how many people actually need this?

[2ZeroSix]

It’s common use case for macOS/iOS plug-ins, third party libraries in ffi plugins etc

In our case it forces us to remove directory with symlink to common scripts before publish

pubignore also does not solve it since this check is done prior to application of ignore rules

[2ZeroSix]

• added test for cycles
• fixed wrongly appended / to path consisted of single delimiter

The only failing test should be one which ensures lish is failing on package with symlink to directory.
This seems to be a regression and was introduced somewhere between flutter 2.5+ and 2.8- (dart 2.12 and 2.15)

Previous behavior (pub bundled in dart 2.12 and behavior in current PR):

• pub archives all filtered files in tar.gz

• consumer receives them as separate files after unpacking of tar.gz

• so, there is no OS-dependant problems for consumer and no headache for package developer

Current behavior:

• pub throws on any (even ignored) directory symlink beneath package directory
• package developers should apply workarounds to their pipelines
• consumers doesn't see any difference

If you insist this is expected behavior: I could try to move the check after resolving of ignored files and allow only ignored directory symlinks

[2ZeroSix]

Hmm, it seems that in windows it fails to list dirrectory with loops in symlinks

It differs from what is done on posix-like systems

02:34 +32 -1: test/package_list_files_test.dart: throws on symlink cycles
02:34 +32 -2: test/package_list_files_test.dart: throws on symlink cycles [E]
  Expected: throws <Instance of 'DataException'> with `message`: contains 'Pub does not support publishing packages with non-resolving symlink:'
    Actual: <Closure: () => List<String>>
     Which: threw FileSystemException:<FileSystemException: Directory listing failed, path = 'C:\Users\runneradmin\AppData\Local\Temp\dart_test_5e4ddcd9\myapp\subdir\symlink\subdir\symlink\subdir\symlink\subdir\symlink\subdir\symlink\subdir\symlink\subdir\symlink\subdir\symlink\subdir\symlink\subdir\symlink\subdir\symlink\subdir\symlink\subdir\symlink\*' (OS Error: The system cannot find the path specified.
                  , errno = 3)>
            stack dart:io                                   _Directory.listSync
                  package:pub/src/package.dart 248:48       Package.listFiles.<fn>
                  package:pub/src/ignore.dart 291:28        Ignore.listFiles
                  package:pub/src/package.dart 245:19       Package.listFiles
                  test\package_list_files_test.dart 115:30  main.<fn>.<fn>
                  package:test_api                          expect
                  test\package_list_files_test.dart 114:5   main.<fn>
                  
            which is not an instance of 'DataException'
  
  package:test_api                         expect
  test\package_list_files_test.dart 114:5  main.<fn>

[sigurdm]

Besides the cycle-handling this is looking quite good!

[2ZeroSix]

jfyi: I had no time to reslove windows-related bugs lately, will continue in a few days

[sigurdm]

Hmm - seems the CI doesn't agree quite with my research.

On mac it seems contents of sufficiently nested directories is not listed. And on windows sufficiently nested links exists as Link but not as a Directory...

[spydon]

Hmm - seems the CI doesn't agree quite with my research.

On mac it seems contents of sufficiently nested directories is not listed. And on windows sufficiently nested links exists as Link but not as a Directory...

Does that matter if it is documented? It would still work for all use cases where everything is set up correctly right?

[sigurdm]

Yeah - perhaps we don't need to care that much about symlink cycles. If you have them, you kind of set yourself up for trouble...

[sigurdm]

@jonasfj I reinstated some symlink detection. I think™ this works.

• For every visited directory x, it resolves canonicalize(x) to x'.
• Then for every parent of canonicalize(x), p:
  • it resolves p to p'. If x' equals p' it reports a cycle.

Could you do a sanity check?

[jonasfj]

I think this will work.