-
|
I've been using code-server for a couple of years now on my private server where I host some stuff for myself and friends/family. Yesterday, I noticed that I got hacked for the first time ever (I've been self-hosting stuff for ~12 years now), and the point of entry was code-server. Luckily, code-server was constrained to a docker container without root access, so the damage was somewhat contained, but I still have no answer to how the attacker actually gained access. So I'd like to share my insights and maybe we can figure out together what happened. the setup
I'm not using the built-in password authentication mechanism of code-server, since authelia already handles authentication. the incidentunexpected access to code-server on 2024-07-31 from an IP address in India. full `remoteagent.log` from that daythe attacker then runs these commands on the vscode terminal inside the docker container this installs a crypto miner that causes 100% load on all cpu cores and some kind of VPN that allows scammers to use your IP address to scam other people. point of entry
so, it seems like the attacker was able to completely avoid my authentication mechanism. They didn't even go through the webserver and still got access to code-server. How? 🤔 |
Beta Was this translation helpful? Give feedback.
Replies: 6 comments 12 replies
-
|
The only way I can come up with off the top of my head that would make this possible is if code-server was listening on some interface that is directly exposed to the internet, making it possible to bypass both Authelia and NGINX. Is Docker bound to 0.0.0.0 maybe? |
Beta Was this translation helpful? Give feedback.
-
|
Except your firewall would block 8080 so even if it was listening on all interfaces that could not be it either... |
Beta Was this translation helpful? Give feedback.
-
|
I'm using docker compose, 8443:8443 is the only port mapping in the compose file. not using the host network. 8443 is not listed in ufw, and the policy is to block everything that is not listed. so I would not expect this to be a problem. |
Beta Was this translation helpful? Give feedback.
-
|
there's one more part to the story: the attacker provided their email and password for packetshare on the CLI, so I was able to log in with their credentials; changed the password, of course ;) there were about 50 other infected devices connected to this account (which I can't shut down remotely, unfortunately). This confirms my suspicion that this was not a targeted attack, but a rather simple exploit. Someone must have been scanning IP addresses for this exploit, and they didn't go through my webserver. Still got no theory how, though. ok, I do, but those are not trivial explots: a zero day in vs code or code-server specifically, or malicious code that was placed in one of the few extensions I have installed, which allowed the attacker to connect through a TCP connection opened from my end of the network. I don't know, someone with an exploit like that could do way more damage than installing a crypto miner. |
Beta Was this translation helpful? Give feedback.
-
I really should have verified this 😵 |
Beta Was this translation helpful? Give feedback.
-
|
I've decided to set up a honeypot: code-server will run on the same machine, with the same config except
the only way to access code-server now is from the inside, e.g. if an extension phones home and leaves a TCP connection open. let's see if someone will take the bait. |
Beta Was this translation helpful? Give feedback.
-
https://docs.docker.com/network/packet-filtering-firewalls/#docker-and-ufw I didn't know this. I've been using docker for so many years now, and all this time, my ufw rules were useless. This was the point of entry, code-server was exposed on port 8443 all along, I just never checked because I was so certain that I can rely on my ufw rules. For now, I've changed the port mapping to |
Beta Was this translation helpful? Give feedback.
-
|
It's kinda both, I guess, since ufw takes control of iptables to make it simple to block traffic, and docker takes control of iptables to make it simple to route traffic. If they were to agree on a compatible approach, they could avoid this conflict, but so far, both the docker and ufw maintainers have shown little interest in doing so. |
Beta Was this translation helpful? Give feedback.
-
|
That is really unfortunate. 😞 |
Beta Was this translation helpful? Give feedback.
-
|
@klamann Never expose a container/service on Use a (more advanced) proxy like Træfik for your services. Check out https://demo.jupyter.b-data.ch which is a reference deployment of https://gitlab.com/b-data/docker/deployments/jupyter. |
Beta Was this translation helpful? Give feedback.
-
|
There are a couple of scripts to make this work: https://github.com/chaifeng/ufw-docker, not sure if it is still maintained though. |
Beta Was this translation helpful? Give feedback.
-
|
I considered using ufw-docker when I was investigating this topic, but ultimately decided against it. This incident made me question a lot of the assumptions I have about network security. Adding a bunch of code that I don't understand from a project on github that does not seem to be actively maintained does not feel like a good idea, at least for me. So I decided learn more about docker networking, iptables and ufw to avoid similar pitfalls from now on. Also, I set up a script to run scheduled port scans on my own servers to notify me about new open ports. And another script to notify me when that script fails. Hopefully that's sufficient for now 😅 |
Beta Was this translation helpful? Give feedback.
-
|
Interesting article; similar topic: https://www.math.cmu.edu/~gautam/sj/blog/20191102-docker-sshguard.html |
Beta Was this translation helpful? Give feedback.
-
|
@klamann Can you share your configuration to bind authelia and code-server? I couldn't find the correct way to integrate authelia with code-server myself, and have tried a bunch of ways without a go. Thanks a lot! |
Beta Was this translation helpful? Give feedback.
-
|
Hey, I run both code-server and authelia in docker and bind them to a local IP (to avoid getting pwnd again) with e.g. Then I configure a subdomain for code-server in nginx like so The snippets |
Beta Was this translation helpful? Give feedback.
-
|
Hello! I noticed that someone had also gained access to my server, through code-server, and had been mining. (I don't believe that they did something else than mining.) My setup was somewhat different:
However, port forwarding through subdomains was enabled without authentication. So in my head, even if someone managed to "create" a subdomain (in that period of time when my script was running):
However, I did not realise that:
I expected code-server to reject requests for subdomains that don't bind to any port, instead of loading the editor. However, it still makes sense to me that code-server should not accept requests to invalid subdomains. (This would improve security slightly, IMO) I hope by sharing this, others can avoid making similar mistakes :) |
Beta Was this translation helpful? Give feedback.
-
|
I am glad you got it figured out! Your proposal and assumption does make sense to me. I assume you are using the So, as an example, if your proxy domain is |
Beta Was this translation helpful? Give feedback.
https://docs.docker.com/network/packet-filtering-firewalls/#docker-and-ufw
I didn't know this. I've been using docker for so many years now, and all this time, my ufw rules were useless. This was the point of entry, code-server was exposed on port 8443 all along, I just never checked because I was so certain that I can rely on my ufw rules.
For …