Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Need help with nginx config #729

Open
narenarora opened this issue Jan 11, 2024 · 5 comments
Open

Need help with nginx config #729

narenarora opened this issue Jan 11, 2024 · 5 comments

Comments

@narenarora
Copy link

I have script-server setup as a docker container on Server_B. I am modifying the container with my custom Dockerfile for 2 issues -

  1. python couldn't find "requests" module when I included "callbacks" in my conf.json
  2. attempted to include the fix mentioned in Issue with hosting script-server on subpath using nginx  #640 (did not work)

Here is my Dockerfile and docker-compose.yaml

# Dockerfile
FROM bugy/script-server:latest
COPY code/tornado_utils.py /app/src/utils/
RUN apt-get update && apt-get install -y openssh-client
RUN pip3 install requests

# docker-compose.yaml
services:
  scriptserver:
    container_name: scriptserver
    hostname: scriptserver
    #image: bugy/script-server:latest
    build:
      context: .
    volumes:
      - ./config:/app/conf
      - ./scripts:/app/scripts
      - ./auth:/auth
      - ./ssh:/root/.ssh
    ports:
      - "5050:5000"
    extra_hosts:
      - "host.docker.internal:host-gateway"
    restart: unless-stopped
    healthcheck:
      disable: true

I have Nginx Proxy Manager running in a docker container on Server_A. This is where my ports 80/443 are forwarded to.
Below is my config for using a subpath

location ^~ /myscripts/ {
	proxy_pass_header Server;
	proxy_set_header Host $proxy_host;
	proxy_set_header X-Real-IP $remote_addr;
	proxy_set_header X-Forwarded-For   $proxy_add_x_forwarded_for;
        proxy_pass http://Server_B:5050/;
        
	
	# needed for websockets
	proxy_http_version 1.1;
	proxy_set_header Upgrade $http_upgrade;
	proxy_set_header Connection "upgrade";
	proxy_set_header Origin http://$proxy_host;
}

However, this doesn't work for me. When I go to https://mydomain.com/myscripts/, I see it redirect to https://mydomain.com/index.html and then gives a 404 error.

This is what I see in the container logs (just noticed the timezone difference in the container timestamps, but I digress)

2024-01-11 16:12:04,715 [root.INFO] Starting Script Server, v1.18.0
2024-01-11 16:12:04,716 [migrations.INFO] Applying migration add_execution_info_to_log_files
2024-01-11 16:12:04,716 [migrations.INFO] Applying migration introduce_access_config
2024-01-11 16:12:04,717 [migrations.INFO] Applying migration migrate_bash_formatting_to_output_format
2024-01-11 16:12:04,717 [migrations.INFO] Applying migration add_user_id_to_log_files
2024-01-11 16:12:04,718 [migrations.INFO] Applying migration migrate_repeat_param_and_same_arg_param
2024-01-11 16:12:04,718 [migrations.INFO] Applying migration migrate_output_files_parameters_substitution
2024-01-11 16:12:04,724 [script_server.HtpasswdAuthenticator.INFO] Using built-in encoder for password verification
2024-01-11 16:12:04,778 [asyncio.DEBUG] Using selector: EpollSelector
Server is running on: http://0.0.0.0:5000
2024-01-11 16:12:55,445 [tornado.access.INFO] 302 GET / (192.168.210.1) 0.55ms
2024-01-11 16:13:17,514 [tornado.access.INFO] 302 GET / (192.168.210.1) 0.70ms
2024-01-11 16:15:31,353 [asyncio.DEBUG] Using selector: EpollSelector
2024-01-11 16:15:31,354 [tornado.access.INFO] 302 GET /myscripts/ (192.168.210.1) 2.29ms
2024-01-11 16:16:48,330 [tornado.access.INFO] 302 GET / (192.168.210.1) 0.40ms
2024-01-11 16:16:48,400 [tornado.general.ERROR] Could not open static file '/app/conf/theme/theme.css'
2024-01-11 16:16:48,401 [tornado.access.INFO] 200 GET /theme/theme.css (192.168.210.1) 0.79ms
2024-01-11 16:16:48,549 [web_server.WARNING] 401 GET /auth/info: user is not authenticated
2024-01-11 16:16:48,550 [tornado.general.WARNING] 401 GET /auth/info (192.168.210.1): Not authenticated
2024-01-11 16:16:48,550 [tornado.access.WARNING] 401 GET /auth/info (192.168.210.1) 1.86ms
2024-01-11 16:16:48,554 [web_server.WARNING] 401 GET /conf: user is not authenticated
2024-01-11 16:16:48,554 [tornado.general.WARNING] 401 GET /conf (192.168.210.1): Not authenticated
2024-01-11 16:16:48,555 [tornado.access.WARNING] 401 GET /conf (192.168.210.1) 1.26ms
2024-01-11 16:16:48,556 [web_server.WARNING] 401 GET /scripts: user is not authenticated
2024-01-11 16:16:48,557 [tornado.general.WARNING] 401 GET /scripts (192.168.210.1): Not authenticated
2024-01-11 16:16:48,557 [tornado.access.WARNING] 401 GET /scripts (192.168.210.1) 1.33ms
2024-01-11 16:16:48,559 [web_server.WARNING] 401 GET /executions/active: user is not authenticated
2024-01-11 16:16:48,559 [tornado.general.WARNING] 401 GET /executions/active (192.168.210.1): Not authenticated
2024-01-11 16:16:48,560 [tornado.access.WARNING] 401 GET /executions/active (192.168.210.1) 1.26ms
2024-01-11 16:19:23,159 [tornado.access.INFO] 302 GET / (192.168.210.1) 0.64ms
2024-01-11 16:21:33,480 [tornado.access.INFO] 302 GET / (192.168.210.1) 0.50ms
2024-01-11 16:25:27,652 [tornado.access.INFO] 302 GET / (192.168.210.1) 0.70ms
2024-01-11 16:25:27,803 [tornado.access.INFO] 200 GET /theme/theme.css (192.168.210.1) 0.77ms
2024-01-11 16:25:27,946 [web_server.WARNING] 401 GET /auth/info: user is not authenticated
2024-01-11 16:25:27,946 [tornado.general.WARNING] 401 GET /auth/info (192.168.210.1): Not authenticated
2024-01-11 16:25:27,946 [tornado.access.WARNING] 401 GET /auth/info (192.168.210.1) 1.46ms
2024-01-11 16:25:27,948 [web_server.WARNING] 401 GET /conf: user is not authenticated
2024-01-11 16:25:27,949 [tornado.general.WARNING] 401 GET /conf (192.168.210.1): Not authenticated
2024-01-11 16:25:27,949 [tornado.access.WARNING] 401 GET /conf (192.168.210.1) 1.63ms
2024-01-11 16:25:27,950 [web_server.WARNING] 401 GET /scripts: user is not authenticated
2024-01-11 16:25:27,950 [tornado.general.WARNING] 401 GET /scripts (192.168.210.1): Not authenticated
2024-01-11 16:25:27,951 [tornado.access.WARNING] 401 GET /scripts (192.168.210.1) 1.00ms
2024-01-11 16:25:27,951 [web_server.WARNING] 401 GET /executions/active: user is not authenticated
2024-01-11 16:25:27,951 [tornado.general.WARNING] 401 GET /executions/active (192.168.210.1): Not authenticated
2024-01-11 16:25:27,951 [tornado.access.WARNING] 401 GET /executions/active (192.168.210.1) 1.09ms
2024-01-11 16:26:24,637 [tornado.access.INFO] 302 GET / (192.168.210.1) 0.68ms
2024-01-11 16:26:24,760 [tornado.access.INFO] 200 GET /theme/theme.css (192.168.210.1) 0.77ms
2024-01-11 16:26:24,830 [web_server.WARNING] 401 GET /auth/info: user is not authenticated
2024-01-11 16:26:24,830 [tornado.general.WARNING] 401 GET /auth/info (192.168.210.1): Not authenticated
2024-01-11 16:26:24,830 [tornado.access.WARNING] 401 GET /auth/info (192.168.210.1) 1.34ms
2024-01-11 16:26:24,832 [web_server.WARNING] 401 GET /conf: user is not authenticated
2024-01-11 16:26:24,832 [tornado.general.WARNING] 401 GET /conf (192.168.210.1): Not authenticated
2024-01-11 16:26:24,832 [tornado.access.WARNING] 401 GET /conf (192.168.210.1) 1.20ms
2024-01-11 16:26:24,834 [web_server.WARNING] 401 GET /scripts: user is not authenticated
2024-01-11 16:26:24,834 [tornado.general.WARNING] 401 GET /scripts (192.168.210.1): Not authenticated
2024-01-11 16:26:24,834 [tornado.access.WARNING] 401 GET /scripts (192.168.210.1) 1.02ms
2024-01-11 16:26:24,834 [web_server.WARNING] 401 GET /executions/active: user is not authenticated
2024-01-11 16:26:24,834 [tornado.general.WARNING] 401 GET /executions/active (192.168.210.1): Not authenticated
2024-01-11 16:26:24,835 [tornado.access.WARNING] 401 GET /executions/active (192.168.210.1) 1.18ms
2024-01-11 16:26:34,980 [tornado.access.INFO] 302 GET / (192.168.210.1) 0.75ms
2024-01-11 16:26:35,122 [tornado.access.INFO] 200 GET /theme/theme.css (192.168.210.1) 0.79ms
2024-01-11 16:26:35,266 [web_server.WARNING] 401 GET /auth/info: user is not authenticated
2024-01-11 16:26:35,267 [tornado.general.WARNING] 401 GET /auth/info (192.168.210.1): Not authenticated
2024-01-11 16:26:35,267 [tornado.access.WARNING] 401 GET /auth/info (192.168.210.1) 1.97ms
2024-01-11 16:26:35,267 [web_server.WARNING] 401 GET /conf: user is not authenticated
2024-01-11 16:26:35,267 [tornado.general.WARNING] 401 GET /conf (192.168.210.1): Not authenticated
2024-01-11 16:26:35,268 [tornado.access.WARNING] 401 GET /conf (192.168.210.1) 2.07ms
2024-01-11 16:26:35,269 [web_server.WARNING] 401 GET /scripts: user is not authenticated
2024-01-11 16:26:35,269 [tornado.general.WARNING] 401 GET /scripts (192.168.210.1): Not authenticated
2024-01-11 16:26:35,269 [tornado.access.WARNING] 401 GET /scripts (192.168.210.1) 1.21ms
2024-01-11 16:26:35,271 [web_server.WARNING] 401 GET /executions/active: user is not authenticated
2024-01-11 16:26:35,271 [tornado.general.WARNING] 401 GET /executions/active (192.168.210.1): Not authenticated
2024-01-11 16:26:35,271 [tornado.access.WARNING] 401 GET /executions/active (192.168.210.1) 1.23ms
2024-01-11 16:27:49,423 [tornado.access.INFO] 302 GET / (192.168.210.1) 0.38ms
2024-01-11 16:29:48,520 [tornado.access.INFO] 302 GET / (192.168.210.1) 0.47ms
2024-01-11 16:31:35,809 [tornado.access.INFO] 302 GET / (192.168.210.1) 0.61ms
2024-01-11 16:31:44,115 [tornado.access.INFO] 302 GET / (192.168.210.1) 0.65ms
2024-01-11 16:39:10,494 [tornado.access.INFO] 302 GET / (192.168.210.1) 0.68ms

192.168.210.1 above is my internal docker IP.

Any help to figure out the issue with my config would be great. Thank you!

@bugy
Copy link
Owner

bugy commented Jan 12, 2024

Hi @narenarora I'm not an nginx expert unfortunately :(
I tried to reproduce the issue, without Docker though, on script-server.net and everything worked fine for me.
I used 1.18 version build

This is my nginx config:

	location ^~ /test-nginx/ {
        proxy_pass_header Server;
        proxy_set_header Host $proxy_host;
        proxy_set_header X-Real-IP $remote_addr;
  #     proxy_set_header X-Scheme $scheme;
        proxy_set_header X-Forwarded-For   $proxy_add_x_forwarded_for;
        proxy_pass http://127.0.0.1:5000/;

        # needed for websockets
        proxy_http_version 1.1;
        proxy_set_header Upgrade $http_upgrade;
        proxy_set_header Connection "upgrade";
        proxy_set_header Origin http://$proxy_host;
        }

@bugy
Copy link
Owner

bugy commented Jan 12, 2024

I even tried it with docker:
sudo docker run -d -p 5001:5000 bugy/script-server:1.18.0

	location ^~ /test-nginx-docker/ {
        proxy_pass_header Server;
        proxy_set_header Host $proxy_host;
        proxy_set_header X-Real-IP $remote_addr;
  #     proxy_set_header X-Scheme $scheme;
        proxy_set_header X-Forwarded-For   $proxy_add_x_forwarded_for;
        proxy_pass http://127.0.0.1:5001/;

        # needed for websockets
        proxy_http_version 1.1;
        proxy_set_header Upgrade $http_upgrade;
        proxy_set_header Connection "upgrade";
        proxy_set_header Origin http://$proxy_host;
        }

Still works fine :(

@narenarora
Copy link
Author

narenarora commented Jan 12, 2024

Thank you @bugy for trying this out. Would you be able to try it with your reverse proxy and script server running on different hosts? That's the only difference I can think of between your setup and mine.

Meanwhile, I will try to move my container to the same host that has my proxy and see if it works there.

Feel free to call out if none of this makes sense. I am guessing it has something to do with $proxy_host and 127.0.0.1 essentially being the same in your case. So maybe the Host and Origin proxy headers have some play here when the proxy_pass is to a non-local location.

Edit : Not a proxy / nginx expert either. Just guessing.

@narenarora
Copy link
Author

Incase this is where the issue is, below is my conf.json

{
  "port": 5000,
  "address": "0.0.0.0",
  "title": "My Script Server",
  "auth": {
        "type": "htpasswd",
        "htpasswd_path": "/auth/.htpasswd"
  },
  "access": {
        "admin_users": [ "admin" ]
  },
  "callbacks": {
    "notify_on_start": false,
    "notify_on_finish": true,
    "notification_fields": [ "execution_id", "script_name", "user", "audit_command", "exit_code" ],
    "destinations": [
        {
            "type": "http",
            "url": "http://ntfy/topic"
        }
    ]
  },
  "logging": {
    "execution_file": "$DATE-$ID.log",
    "execution_date_format": "%y-%m-%d_%H-%M"
  },
  "security": {
    "xsrf_protection": "token"
  }
}

@narenarora
Copy link
Author

narenarora commented Jan 12, 2024

If this helps .. this is my redirection log. Looks like once it hits index.html , it is internally redirected to /login.html without the subpath

$ curl -vL https://mydomain.com/myscripts 2>&1 | grep -i location
< location: https://mydomain.com/myscripts/
< location: http://mydomain.com/login.html?next=%2Fmyscripts%2F
< Location: https://mydomain.com/login.html?next=%2Fmyscripts%2F

I did try to run the script-server container on the same host as the proxy, but same issue. All the proxy configs I have seen in the issues seem to be proxy running on the host directly, so http://127.0.0.1:5000 is a valid location. Since I have my proxy running as a container, 127.0.0.1 doesn't apply.

Using a subdomain works, as expected. But I'd really like to find a solution for this using a subpath.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

2 participants