Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[Bug]: SourceForge sourced apps returning 403 #763

Open
2 tasks done
chezzer64 opened this issue Oct 28, 2024 · 7 comments
Open
2 tasks done

[Bug]: SourceForge sourced apps returning 403 #763

chezzer64 opened this issue Oct 28, 2024 · 7 comments
Labels
bug Something isn't working

Comments

@chezzer64
Copy link

What happened?

All Evergreen apps hosted on SourceForge are currently returning '403 Forbidden'

Version

2410.1527

What PowerShell edition/s are you running Evergreen on?

Windows PowerShell

Which operating system/s are you running Evergreen on?

Windows 10+

Have you reviewed the documentation?

Verbose output

Get-EvergreenApp -Name keepass -verbose
VERBOSE: Function path: C:\Program Files\WindowsPowerShell\Modules\Evergreen\2410.1527\Apps\Get-keepass.ps1
VERBOSE: Function exists: C:\Program Files\WindowsPowerShell\Modules\Evergreen\2410.1527\Apps\Get-keepass.ps1.
VERBOSE: Dot sourcing: C:\Program Files\WindowsPowerShell\Modules\Evergreen\2410.1527\Apps\Get-keepass.ps1.
VERBOSE: Get-FunctionResource: read application resource strings from [C:\Program Files\WindowsPowerShell\Modules\Evergreen\2410.1527\Manifests\keepass.json]
VERBOSE: Calling: Get-keepass.
VERBOSE: Invoke-EvergreenRestMethod: Invoke-RestMethod parameter: UseBasicParsing: True.
VERBOSE: Invoke-EvergreenRestMethod: Invoke-RestMethod parameter: MaximumRedirection: 2.
VERBOSE: Invoke-EvergreenRestMethod: Invoke-RestMethod parameter: DisableKeepAlive: True.
VERBOSE: Invoke-EvergreenRestMethod: Invoke-RestMethod parameter: UserAgent: Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/128.0.0.0 Safari/537.36 Edg/127.0.2651.105.
VERBOSE: Invoke-EvergreenRestMethod: Invoke-RestMethod parameter: Method: Default.
VERBOSE: Invoke-EvergreenRestMethod: Invoke-RestMethod parameter: Uri: https://sourceforge.net/projects/keepass/best_release.json.
VERBOSE: Invoke-EvergreenRestMethod: Invoke-RestMethod parameter: ContentType: application/json; charset=utf-8.
VERBOSE: GET with 0-byte payload
WARNING: Invoke-EvergreenRestMethod: The remote server returned an error: (403) Forbidden., with: https://sourceforge.net/projects/keepass/best_release.json.
WARNING: Invoke-EvergreenRestMethod: For troubleshooting steps see: https://stealthpuppy.com/evergreen/troubleshoot/.
WARNING: Run 'Get-EvergreenApp -Name "keepass" -Verbose' to review additional details for troubleshooting.
Invoke-RestMethod : Just a moment...*{box-sizing:border-box;margin:0;padding:0}html{line-height:1.15;-webkit-text-size-adjust:100%;color:#313131;font-family:system-ui,-apple-system,BlinkMacSystemFont,Segoe
UI,Roboto,Helvetica Neue,Arial,Noto Sans,sans-serif,Apple Color Emoji,Segoe UI Emoji,Segoe UI Symbol,Noto Color
Emoji}body{display:flex;flex-direction:column;height:100vh;min-height:100vh}.main-content{margin:8rem auto;max-width:60rem;padding-left:1.5rem}@media (width Enable JavaScript and cookies to
continue(function(){window._cf_chl_opt={cvId: '3',cZone: "sourceforge.net",cType: 'managed',cRay: '8d9aa8455c8520a6',cH:
'6Iku2Pm7upZOD.u.sSrNv48bLBUigY5W6tA7i_njR2k-1730115692-1.2.1.1-o3CSn369Bp8JHNPSbLAczOE45PnYF2pqVdErUQP6ZtWhf_L6RElVryto262PDN.N',cUPMDTk:
"\/projects\/keepass\/best_release.json?__cf_chl_tk=Id1r7.eFrKYslEc58PdkykMDHC0R_e5Hz_vUNSH3te4-1730115692-1.0.1.1-YEE4Uyds0s.Xi9k6rPNOSKItH3EFizcbB3WDvuM_lq0",cFPWv: 'b',cITimeS: '1730115692',cTTimeMs:
'1000',cMTimeMs: '390000',cTplV: 5,cTplB: 'cf',cK: "",fa:
"\/projects\/keepass\/best_release.json?__cf_chl_f_tk=Id1r7.eFrKYslEc58PdkykMDHC0R_e5Hz_vUNSH3te4-1730115692-1.0.1.1-YEE4Uyds0s.Xi9k6rPNOSKItH3EFizcbB3WDvuM_lq0",md: "cFcGXILbnKjqjgULz3Z2D57PLuYUucZQYn8F2y1fR
EA-1730115692-1.2.1.1-tzj2WwhgzgR1e8GphAgciF5HTZsHTD1NEzcextqngjTqOYq5p_kshKhEMSHYHtGszeFvq7CzU8lHSgeTMghxHp8Yi9oOLqNym8zAfUPfc9OXKH_ZKxDhaMzxpcOvCPt47c1x7IW.RzMre3fp7vlpqWoOk2IuC5BuZ1ppL7JfrLEoMR7X9D8Z3enbk7
JD_QcKsWxo_0Z3xnHu9d0faKy5xzrYw2kZGX7Dz5BjjdfAKy.uK.VCHvt6YqwXTU0eO3vXfA4ero12Z7Emr4V6CYf76umkoIkJl3RK83QT_3AqdjdAWlMU1xhE0nhMP7UObkKtq3Qd4oI4vOzLBCCW0EPCIH0LXtWj7TTqx_csgTXXskowyqIPJhrt7NMnKNZjJ2M6ccckzBkZyR
PXIRZUEOk2GFk10TYZKhrHynHAFeHH_OZO.sf1GMxAJfSTp6sESeixxQheOB7pGpP4w8M_SdNN08XePc4juq2nbz8AX23Iq013KMUDoJg7sGAG126uJ1YcZjqB7GdmykCwq0ZtBWXiuOrSwI7OMkLiZRtGEykUnNqgpjAOQXk665V4aWNkKmvQkHrgonGLXCF1h9joRJu2_0QkNJ
uyhvKAqOC.90zMRHZcZgMr76nWQJC9IPwnqwYQMosLZ6E2dcrQ3u4i4NjAwTalNvWBoWBP7EqubRUqqJg5Bu0mA2L7Ush43V0tdI9W9SzOuWxYENSTkV4Q1d1eODWdJFJK4jOYNALVD5Ck_ZhMn7y.AmvLGhuw2PDOcGmHMOQvxxFbXggIMb5p6so.T0nZYr648Y7QvCEIRTLA6Y
99Yqbs2dVvhXN1h8abEh0H0391.n_ZeHjml1LZzlA08tEnHdZiWEeydECwulwiE96M7HzVaZqMCIHi8tbs0BGGMT6wKlGaBFpyEYI.xSG5AiEYC92IEzIBK5Sj7Ic16ecaX.SjyDLKW_NxY2XH27sDzxEwmbRo91.WoS90vabSJN2vfqrUbACYvNt1Od2.iY4NvG.d2Fkcj1iB_z
mgjDSo1ERo4DP0L.6MfDiZU89EO.nUzpl1XwYQn5zAWt7Uz2Mw.ILZmDmhwHhhPyqsQByFWonfCTLcI4nUjlFdx.N7WLfucaUNXTqqwILLLkj.bGL5QFJYkXyxq3eEV6PI9vAjoHt34z__ZtcgXINkaByjuE8LGIubmVaX178BqMrAcgivquduCE1Fg7vV3pWxta6esBAaZ2eT_T
_v1vEQClGmJotC2EIdv.GZFDRMCderCfxi0DEvfvQGJyNlNRcOoAkmrIpk7TqksnpKTFodRu8Uhh2yQ_IRrYq9zQAZfHeUNWkA_Kjgvm62AF3KX7DUyggXDfE0CuAShSFm5WA5MqHszeQyBhp2xbU4fG9WsVsEtvp41HjzV1GL0M0nFY4eU88K89LgzL2WpXmtOsYOVtnDTrduw1
MaYMbiPTkR3jVHRZ42g8zujEctNW685CtSX1l1kRqWrcfkBlX32cNECfFAh9MdAr_BBvYC2HG8lLwsAlZhqNSbbC4ThwDjCSkTX.pzxUtSmtB9TwrkvdS92smCcK09VGG36xdGLMW_NnXj6WdXqRnj6WFCZvpwN9zwKmkF5L5grVHwkBQP5ZLZdVG0bbFXtuW1ZOcflm8yJUKtN8
GX._XB_71TZxfK7D2zDE3bTe1bVprpfZGmaubeMOf2MN.FK13YPeygQVhk1zQvzwP7RTHh10SbG3JbHLhJOOHSqkA4c.HHFLRmETYBh6sjSsr5GOPySHyvhjeMm2tWGsY2DVkd2OGIU2hVSCPSYfR.LTN4ufYDW4JkkuUWVKnJZOzOM2NpiyX_uqLyLFqNi4lS.ckHtAbdqnPEcH
Un_7qWvdb63HRe8N0f7RH2KMD5wGRM11vHrq3gcHTgqG1ZtrF0Ni_1MZKWpHLOQEThJ1px_E1DzkPW.2sxqoBDuNygRKYm4nyy3uEkEUtvHVyYbo8fkeJYuLnoE174AuvsyxmyS3LtffO6CIweHrT6dZtejU4PbUyEoUFN1EyGL6khBL.qNrgewcMFRiGIaM6PpfVsY1EiJgEsS7
pQyK2TVk0XRRaPKBCBIUUWzt7awVbqnWHxdpM2ISw8iiJvAauXvTyFq8YyJeajYc1dQsZx0.J7S05WXpS9LPOm8U5jc7KeuWs4h49_DeKQ_bQ7eHiWYxwEFIIZ1cR391axzqeBgZTpDttkv.8W8aPa3sx4ibdH6IhysxQqUmB_JdVNbVlD7veeoHMJDl7Gx9_ClUP.ZKr5ws.K37
OyDZ89z.D9cFR2hi9pcwu0QzJgN.tqEKwxHG_WH3d8meCsTPvt4dZ5bTqURKl9KPYLdmTbOlrFYAyP8Yyo5kQ_fLh8Ka5bE_QnSYga_ukOA9kofXeubNDUSXpEdJdYt.AXbBkpWmF5bBa0jyVufTIu6oKaFKC6WzA_a1k70fHsJVwLOAu_UNe8iq4RLDMQBhOcFNxyE_ze7kZNRk
44Bpm314j0rw9ldtc4Ppoo67B4sY8QinvDnQP_767w4_xPkjfUyaihzSFv.4GQ6axUa7_0BX4wCaEPouw2ISeSWDn5n9Gh80hdjk1dB_hpOv92n4lQCrbPyyubQRUu1Kehnfpm2QFvS.LEA6iBP37Vc9GQqmTUbq2SK2sVTi0iQ4GewYW8ufkK2u11pEg1KPdxbinvmG7AyJll.C
C2bWf3TJNY7l_Dtot4qjRTAv3QC9bTJ8S3dCoX87Gva2mBbXSspsmmj3uUZEmyHfgkr6LUT4WPgvarC5Xv376jpUzQUQgCzGxRmUg4uAiy8lDlv7eykhZxvIuBwRGQ_uV7gofANRpq5vGXVygsx43HYaunWKBvVRkucKZLvLp1lrSexvyCf2_1Ughj7MRR_6Fgix3T",mdrd: "l
Aycaml31tG.x4miRC0F8jkWYyog.P.we1E5H8ODPEU-1730115692-1.2.1.1-XeEhukXN8q8H7bUDHvLZT.esf6JdNJUJj6Qfv2muFCceZBPycEAzRmxducePNGLQXNc.N.UO84dRDBZUAcUwgVKB2m0INboZah.BxjzLg2zCGrLLaVLBWdgg4Bhwdf6iJeTxKonHwjevk_zqlI
owIgSmA3eCGT.DInv0uH7amSureUqSQNuqj9pPzo_UWz_Uutwkd5NIwNprSLRw1QVV1P4QrJWQnjRH.270B8U5ltlT.QutdQ2aeAmXPlIpWSXrVUudNURPgwiMtAJRRpUJvQCnfaPKZcXCGUdCPDSC0XSFU_sgkJJjPPB5MCV43EGu9v39hhf_0pjLeNvxj9YJ6DGwhsxLZd27uW
dhgfoW7AaW7T7Jv2XctduqYkRbTqTE4AgH81SW0Wh4qKFFZ.AAOyQOLieGWddmPxwHwHEYw9PiEb3Y7oXr4.BO8m3W7RL7nD2_2Xe5CLtoOerP.I0O470Pagg4Ct5.eH2lRcLP5uq7fl8Reb.QnhdWnlOqzKDPGnMybREP2dxZer07k7ooecvL399_Bs3IhAJEvzJJ0NkYJ1g0LK
upvqLpN9mUqSTIyhYiJAOSbwSrR8m1fnn2k2nGXQkJ74UpcW2Rz8flZb9DFxjQN6MVwZ3m1NzOUC66kbg0sd2rlgC89toxMddar9Z979hNOrxcFD2wkFDGCkjao85SzAy_l5rQny6p38A46LCYsjCS4VyGC3hC0MSkweeVyOhGUNSOKUX6qdqbrs7IidrkFLJdALYmfA1BYcC45i
9F52c.rvZkSW.Yw0QKeKv1rQbj95XgPhbpZxZFHDkUnWcocgZa9WEFmdUjenxH0BXAzyUvxQX3.otyUuck2YZ8CaBZMuCzJvKmzJbFKm2jdg6pO._iPcp2CuM9aGhDgWro5VagdM75vLxe8fvsK3rMTKxZyRJWIsNVR7uJGUwdht3ocSDqJw2ZQa9tGIVUlL5i3xGIy_KJopufpg
jiIZt.H6UFXGl38oRTfOYEQoEpqAYJC4RVJgrLDINQ2CIq9eEhPTW6IyP84lpgBTpVA1PaoD_kJbl1P637tPTAx6fDGezT.Ab6fNmRUqMz_9fYcB_RjSh.zFiI3mL7OCeJIn6ZmJiNCuOLOHJUvN_JsJDcW8a1RSiZCh4yV_GSoEHMSPe4MWUj2W4IhTEkG66Ff.jtLay0XJ9Lsp
ZsrPSN1CXBEZUpMr4Bo4PRzsiDeNiKCAHpIpMOgPQCY1muxxZKSOZw9h0kepxm63Sv27F2hYbCQpIc0hLiQaOmjtZVZuSeRkcfGmJ8BDbHFcpkTsfE6yj84_1rF7U1bZ950YNRF2P7cnjwvUaPllwwfv.chauX9HwhTM1zus38GrnkNK4kjPMF_JqMOzE4VnEzFe1LeZRtqv35.1
1Bsw.DRjG.NY.5z9_oHfMzmQ435gtWCsmD68c5C8QwFZI8ln15f_8AsKObSaDUtMFM6e2LK4Bqd0aShH4CwT47bpsaf5Ee2ves_DzvDEKuwVy1g2KasmOfhFZocnBerVhOUvfCXfqPdwZfyepk14C1KjssuujCPVo0SyolPIt4fIBO.T832vGOMt7UubDQMvGvhoVdfNyCpODz5r
Eoi865MsEoerBUoqKxWqacgUV2hwNb4_Rl9QKce8.XGqzXufi1io3sLjwdk_.p4.xR9GHhW9517z8en8gMCkzZxUXbpBiIZVDjZzEo_oT1hVnbgRxEPVYnVGOxDFNS6eE26DkwEWc21Jn2hGZP0HhvzjLV8dQmCX35UImBbGp6fTsZ8Scsr771ivV5PnFil_8Egpe6oEKsllJbHd
U3ZED4eokFx8McqQbmR_f6TxfP9nt90A9ttWmpTOEWKY6Kvc82CF4SvBLSlLH9fyioRVNUu7CClTV5Qs7DIHUsagmcvSDEfW3RqfxMpvddX2tmBz9clCf8ug4j6ipvZ34UVPfTxYGzM5sV8OwdI7iqkFaG641Jr5x7E2i6ysoDlBDyNoak.80I48tM8_ZRVqs_ZjK.6YO5uSzDrK
VoL1oTe9RebWLyOyGuB.gBLmbLv30Z4wPnZApFoRIT5cZkttDwUyLH47Fe.hvcRNl5YKY3rTpvFsF98FRH7jFR4VKrZVaFMuf.wibe7wPgcI5yOneZL.9L9_mAE9wZ6VlOJVvNNISrnwHlCmJvj3Qh.hJAPLS6kYeIe1Ms021XKT6qjkCcyRUzL_M3l2xkA9T44.cFTLtK3HDNFK
eTVQbyNmdDFqrr4j5.AIfdrjFStP4ZvfhR.g"};var cpo = document.createElement('script');cpo.src = '/cdn-cgi/challenge-platform/h/b/orchestrate/chl_page/v1?ray=8d9aa8455c8520a6';window._cf_chl_opt.cOgUHash =
location.hash === '' && location.href.indexOf('#') !== -1 ? '#' : location.hash;window._cf_chl_opt.cOgUQuery = location.search === '' && location.href.slice(0, location.href.length -
window._cf_chl_opt.cOgUHash.length).indexOf('?') !== -1 ? '?' : location.search;if (window.history && window.history.replaceState) {var ogU = location.pathname + window._cf_chl_opt.cOgUQuery +
window._cf_chl_opt.cOgUHash;history.replaceState(null, null,
"\/projects\/keepass\/best_release.json?__cf_chl_rt_tk=Id1r7.eFrKYslEc58PdkykMDHC0R_e5Hz_vUNSH3te4-1730115692-1.0.1.1-YEE4Uyds0s.Xi9k6rPNOSKItH3EFizcbB3WDvuM_lq0" + window._cf_chl_opt.cOgUHash);cpo.onload =
function() {history.replaceState(null, null, ogU);}}document.getElementsByTagName('head')[0].appendChild(cpo);}());
At C:\Program Files\WindowsPowerShell\Modules\Evergreen\2410.1527\Private\Invoke-EvergreenRestMethod.ps1:123 char:21
+         $Response = Invoke-RestMethod @params
+                     ~~~~~~~~~~~~~~~~~~~~~~~~~
    + CategoryInfo          : InvalidOperation: (System.Net.HttpWebRequest:HttpWebRequest) [Invoke-RestMethod], WebException
    + FullyQualifiedErrorId : WebCmdletWebResponseException,Microsoft.PowerShell.Commands.InvokeRestMethodCommand
@chezzer64 chezzer64 added the bug Something isn't working label Oct 28, 2024
@chezzer64
Copy link
Author

chezzer64 commented Oct 28, 2024

Think it looks like SourceForge has added some 'anti-web scraping' protection?

Have tested a workaround by adding some additional headers to the request but not sure how long they will continue to be successful?

$headers = @{
'Accept' = "text/html,application/xhtml+xml,application/xml;q=0.9"
'Accept-Language' = "en-US,en;q=0.9"
'Sec-Ch-Ua-Platform' = "Windows"
}

Update: Workaround appears to have already stopped working unfortunately.

@aaronparker
Copy link
Owner

aaronparker commented Oct 29, 2024

I don't have a kind word to say about SourceForge, so I'll attempt some restraint. I do however, wish developers would stop using it.

@andre-sicking
Copy link
Contributor

Hi everyone,

at the moment it works for me when I use TLS 1.3 in my Windows PowerShell (5.1) scripts.
So just adding "[Net.ServicePointManager]::SecurityProtocol = [Net.SecurityProtocolType]::Tls13" at the beginning should work already. I guess .NET 4.8 should be installed upfront when using this on older systems as Windows Server 2016. But I did not test this on older OS. My Server 2022 deployments just work fine now.

@aaronparker I absolutly agree about SF. But nobody knows what Microsoft will do, if everybody moves things to Github. Possible Vendor-lock-in incoming ....

Andre

@aaronparker
Copy link
Owner

Using something like the below code was working about two days ago, however, this too is failing.

I'll test with TLS 1.3

$session = New-Object Microsoft.PowerShell.Commands.WebRequestSession
$session.UserAgent = "Mozilla/5.0 (Macintosh; Intel Mac OS X 10_15_7) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/130.0.0.0 Safari/537.36 Edg/130.0.0.0"
$session.Cookies.Add((New-Object System.Net.Cookie("VISITOR", "010663f5-784c-4b89-95b3-b94caeb35ef9", "/", "sourceforge.net")))
$session.Cookies.Add((New-Object System.Net.Cookie("__cf_bm", "757znJGmuebOAc5M59hKsO28h_ubXpvGGcDYocIlwX0-1730205433-1.0.1.1-VpbSE2aiIXI17YraYEUFZL6BKHo73NgKSDsxGrYdMyjwIphBVXE43T05YKDzuRWP9sjXYzacGHBMHYrFVUOxTQ", "/", ".sourceforge.net")))
Invoke-WebRequest -UseBasicParsing -Uri "https://sourceforge.net/projects/keepass/best_release.json" `
-WebSession $session `
-Headers @{
"authority"="sourceforge.net"
  "method"="GET"
  "path"="/projects/keepass/best_release.json"
  "scheme"="https"
  "accept"="text/html,application/xhtml+xml,application/xml;q=0.9,image/avif,image/webp,image/apng,*/*;q=0.8,application/signed-exchange;v=b3;q=0.7"
  "accept-encoding"="gzip, deflate, br, zstd"
  "accept-language"="en-AU,en-GB;q=0.9,en;q=0.8,en-US;q=0.7"
  "cache-control"="max-age=0"
  "dnt"="1"
  "priority"="u=0, i"
  "sec-ch-ua"="`"Chromium`";v=`"130`", `"Microsoft Edge`";v=`"130`", `"Not?A_Brand`";v=`"99`""
  "sec-ch-ua-mobile"="?0"
  "sec-ch-ua-platform"="`"macOS`""
  "sec-fetch-dest"="document"
  "sec-fetch-mode"="navigate"
  "sec-fetch-site"="none"
  "sec-fetch-user"="?1"
  "upgrade-insecure-requests"="1"
}

@andre-sicking
Copy link
Contributor

Maybe they had too many issues with regular user requests using specific filters or probably a "scoring system".
I bet they saw in the logs that many PowerShell-Sessions used TLS 1.2 and use this as a filter now.

To be safe one could determine a current Chrome version to assemble a custom UserAgent (and setting $DownloadUserAgent upfront) and just use that with TLS 1.3 on top so they can't detect PowerShell directly :-)

aaronparker added a commit that referenced this issue Oct 29, 2024
@aaronparker
Copy link
Owner

I've pushed a change to the Apps branch that includes a custom useragent for SourceForge requests. Looks OK to me here in PowerShell 5.1 on Windows 10, and PowerShell on macOS.

@aaronparker
Copy link
Owner

I've added this change to a new module version to see whether it helps, so should be ready to test now

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
bug Something isn't working
Projects
None yet
Development

No branches or pull requests

3 participants