Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Need to check the following site detections #770

Open
WebBreacher opened this issue Nov 17, 2023 · 15 comments
Open

Need to check the following site detections #770

WebBreacher opened this issue Nov 17, 2023 · 15 comments

Comments

@WebBreacher
Copy link
Owner

Ran a check on the JSON and the following sites have failed with a VPN, without a VPN, and from a cloud server. Please check the detections and submit PRs for them.

Please do small batches of 3 or less sites in PRs in case others begin working them or in case there are issues.

1001mem
Alloannonces
ArmorGames
ArtStation
BiggerPockets
Bimpos
Brickset
Code Project
Codeberg
DIBIZ
Discogs
Dissenter
EPORNER
Ello.co
Etsy
ExtraLunchMoney
Eyeem
F3
FanCentro
Filmweb
Fodors Forum
Fortnite Tracker
Friendweb
Garmin connect
GitHubPlus
HackerRank
Hackernoon
Hanime
Heylink
Internet Archive User Search
Kickstarter
LibraryThing
MAGA-CHAT
MAGABOOK
MANYVIDS
Marshmallow
Massage Anywhere
Mastodon-meow.social
Mastodon-mstdn.io
Mastodon.chasedem.dev (Mastodon Instance)
Minds
Minecraft List
Mix
Muck Rack
Musician.social (Mastodon Instance)
MyFitnessPal Community
Myspreadshop
NaturalNews
OpenWRT Forums
Our Freedom Book
PCPartPicker
Parler
Pettingzoo.co (Mastodon Instance)
Pokec
Pokerstrategy
Polchat.pl
Pornhub Porn Stars
Pravda.me
Psstaudio
Replit cli
Replit web
Researchgate
Shanii Writes
Skyrock
SoliKick
Subscribestar
Suzuri
TAPiTAG
Tagged
Teddygirls
Teknik
Tellonym
Thetattooforum
Threads.net
TotalWar
Tunefind
Twitcasting
Twitch
Ubisoft
Udemy
VK
Virustotal
Voice123
Voices.com
Wanelo
WeTransfer
Weasyl
Wikipedia
Wykop
Yahoo! JAPAN Auction
Zillow
aNobii
aaha_chat
aflam
boosty
cnet
contactos.sex
datezone
depop
fandalism
freelancer
freesound
gfycat
gpodder.net
grandprof
hiberworld
itch.io
karab.in
kik
love_ru
memrise
metacritic
nairaland
netvibes
nihbuatjajan
osu!
pokemonshowdown
popl
skeb
speedrun
steller
thoughts
tripadvisor
ulub.pl
utip.io
uwu.ai
watchmemore.com
weheartit
zmarsa.com
zoomitir
@grabowskiadrian
Copy link
Contributor

I started fixing polish websites.

Filmweb -> merged
Fixed PolChat.pl -> invalid test user
Fixed Wykop.pl -> site load data async, title is presented
Removed Ulub.pl -> Site not exits, domain is to sell

Fixes in #772 and #773

@WebBreacher
Copy link
Owner Author

Thanks @grabowskiadrian!

@grabowskiadrian
Copy link
Contributor

grabowskiadrian commented Nov 25, 2023

I looked for sites that stopped working due to domain deletion/suspension or other issue.

Here is the list. I think that those projects whose domains do not exist can be removed from the wmn.json file.

1001mem.  -> 502 site is down
Alloannonces -> timeout, site is down 
Bimpos -> bimpos works, but ask.bimpos.com return error 500
Ello.co -> Origin DNS error
Friendweb - cloudlfare blocked?
GitHubPlus -> website down -> domain for sale
MAGA-CHAT -> website down -> domain for sale
Our Freedom Book -> cloudlfare blocked?
Pettingzoo.co (Mastodon Instance) -> website down
Psstaudio -> project closed
SoliKick -> domain not exists
Teknik -> timeout
Wanelo -> website is down
fandalism -> project closed
gfycat -> domain removed 
ulub.pl -> domain to sell

@grabowskiadrian
Copy link
Contributor

grabowskiadrian commented Dec 12, 2023

@WebBreacher

Current status:

site status
1001mem REMOVED
Alloannonces REMOVED
Brickset FIXED
Code Project FIXED
Codeberg FIXED
DIBIZ FIXED
Etsy FIXED
FanCentro FIXED
fandalism REMOVED
Filmweb FIXED
Garmin connect FIXED
GitHubPlus REMOVED
MAGA-CHAT REMOVED
pettingzoo.co REMOVED
psstaudio.com REMOVED
Polchat.pl FIXED
SoliKick REMOVED
Wanelo REMOVED
Wykop FIXED
gfycat REMOVED
love_ru FIXED
ulub.pl REMOVED

I will try to fix a lot more!

@WebBreacher
Copy link
Owner Author

THIS
IS
EXCELLENT! Thank you @grabowskiadrian !

@grabowskiadrian
Copy link
Contributor

Removed some sites:

  • 1001mem - 502 site it down
  • Alloannonces - timeout, site is down
  • pettingzoo.co - project closed
  • psstaudio.com - project closed (message on website)
  • Wanelo - domain for sale
  • fandalism - project stoped

Fandalism in message about close project linking to new site distrokid.com (integration to verify)

#790

@grabowskiadrian
Copy link
Contributor

21/130 is the current fixing progress.

It's a bit difficult to manage the list of fixes and the status of why something isn't working.

@WebBreacher What do you think? Maybe it's better if I check each of the 109 sites and make separate issues with description for each site?

@WebBreacher
Copy link
Owner Author

@grabowskiadrian That may be better....or take groups of 5-10 sites? I've struggled with how to do this massive checking of resources in the past and never come up with a good strategy

@grabowskiadrian
Copy link
Contributor

grabowskiadrian commented Feb 7, 2024

@WebBreacher I can do it. I see I'm the only one fixing these sites :)

First, I will make a list of sites and causes, because maybe at this stage I will find some simple fixes. For more difficult cases, I will create new issues.

@WebBreacher
Copy link
Owner Author

Well...how about this....come up with a plan for doing the sites and you can start at the top and I'll work from the bottom and we will get through them all.

@grabowskiadrian
Copy link
Contributor

Ok, nice. So give me some time, I will prepare a list with reasons for each site. Some sites will require you to decide: fix or not.

@WebBreacher
Copy link
Owner Author

Sounds good and THANK YOU for doing this!!

I had a person that was going to run a nightly checker on all sites from cloud IP, VPN, and home IP and then create a chart of what sites could be accessed from where and what sites needed to be updated/removed. But, it was too good to be true and fell through.

@grabowskiadrian
Copy link
Contributor

@WebBreacher I'm working on it!

I checked first 25 sites:

  • 9 sites fixed - easy to fix in pull request Next fixes #792
  • described 16 sites - reasons that need to be verified: cloudflare blocking, etc.

84 sites todo... but it's going fast :)

@grabowskiadrian
Copy link
Contributor

There is some info about 16 sites, You have to decide.

=== 1. ArmorGames
Link: https://armorgames.com/user/{account}
Reason: On user page is robot verification

=== 2. ArtStation
Link: https://www.artstation.com/{account}
Reason: Sites loads user data async, Wait a moment.. preloader. There is API: https://www.artstation.com/users/kongaxl_design/quick.json but response is also: Wait a moment..

=== 3. BiggerPockets
Link: https://www.biggerpockets.com/users/{account}
Reason: passed test in my script. integration is ok - to verify

=== 4. Bimpos
Link: https://ask.bimpos.com/user/{account}
Reason: passed test in my script. integration is ok - to verify

=== 5. Discogs
Link: https://www.discogs.com/user/{account}
Reason: Sites loads user data async, Wait a moment.. preloader.

=== 6. Dissenter
Link: https://dissenter.com/user/{account}
Reason: It's probably some other site. There is no login option, I don't see users anywhere.

=== 7. Ello.co
Link: https://ello.co/{account}
Reason: Error 1016, Origin DNS ERROR - i think it's to remove.

=== 8. F3
Link: https://f3.cool/{account}
Reason: There is only one page with buttons to appstore to download app.

=== 9. Fortnite Tracker
Link: https://fortnitetracker.com/profile/all/{account}
Reason: Cloudflare robot verification

=== 10. Friendweb
Link: https://friendweb.nl/{account}
Reason: Cloudflare blocked request - "Sorry, you have been blocked"

=== 11. Hackernoon
Link: https://hackernoon.com/_next/data/foL6JC7ro2FEEMD-gMKgQ/u/{account}.json
Reason: In URL is something like version hash, i think this hash is changing in time. Current hash is ON_igFu0-dfsxsZQzr5-N, but in future this hash also will be changed.. We should find better way.

=== 12. Heylink
Link: https://heylink.me/{account}/
Reason: Cloudflare blocked request: This website is using a security service to protect itself from online attacks.

=== 13. Internet Archive User Search
Link: https://archive.org/search?query={account}
Reason: I don't know

=== 14. Kickstarter
Link: https://www.kickstarter.com/profile/{account}
Reason: Cloudflare blocked request: This website is using a security service to protect itself from online attacks.

=== 15. Mastodon-mstdn.io
Link: https://mstdn.io/@{account}
Reason: In my opinion this is not possible to find user on this site, usernames have different @Domain

=== 16. Mastodon.chasedem.dev (Mastodon Instance)
Link: mastodon.chasem.dev
Reason: Timeout

@WebBreacher
Copy link
Owner Author

Thanks for doing this work. In general, anything with Cloudflare is unusable for us unless we can find an API or other avenue to move around the CF. Same thing with non-CF robot verifications. Unless we can find an API, we cannot use the site.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

No branches or pull requests

2 participants