Linkfinder too fast to scan everythin?

[Rdago]

Hello there,

I just ran Linkfinder on 393.162 JS-URLs and it took less than a second to run and it found 2 Endpoints. I dont think that it checked all of them. What am I missing?

My code:

python3 linkfinder.py -i valid-js.txt -o cli

[EdOverflow]

What version of LinkFinder are you using?

[Rdago]

Hello @EdOverflow
Latest version. I dumped everythin I had and pulled it again. Running it with python3 (if this matters)

[Rdago]

I guess it does not work with lists so I ran it like this:

for i in `cat mydoms.txt`; do python3 linkfinder.py -i $i -o cli >>LF-out.txt; done

Moreover it would be nice if there where flags to just pull out absolute urls. I greped them manually like this:

cat LF-out.txt | grep -E "http[s]?://.*" > LF-abs.txt

[EdOverflow]

Would you be willing to share a few lines of your input text file?

[Rdago]

Sure.

mydoms.txt: 
https://github.githubassets.com/assets/environment-bootstrap-e3b72f9d.js
https://github.githubassets.com/assets/vendor-ed3c6d95.js
https://github.githubassets.com/assets/github-bootstrap-091a9cad.js

PS: glaube das tool kann stand jetzt keine Listen einlesen dementsprechend über ne For-Schleife gelöst, ist aber für meine Zwecke erstmal okey :)

danke für deine Hilfe @EdOverflow
Ich sag mal bis zum CCC ;)

[EdOverflow]

You are absolutely right, @Rdago. There does appear to be a problem with lists now. Lists used to work so I must have broken it with an update. I will try my best to fix this issue as soon as possible.

Übrigens du kannst auch GNU Parallel verwenden falls du die Links in parallel einlesen möchtest. Lauft schneller als eine Für-Schleife. :)

$ parallel -j 12 "python3 linkfinder.py -i {} -o cli 2> /dev/null" :::: mydoms.txt