All posts Reading time: 4 minutes

Configuring Rundeck with Nginx and SSL Termination on AWS

AWS

I spent the whole day troubleshooting some authentication/redirection problems in my Rundeck setup, I went to the pub and had a few beers then I went to bed and woke up at 4 AM with one of those productive insomnia feelings so I headed straight to the computer to find the solution for my problems. Let’s go through my setup so I can explain the problem a bit more and hopefully help you avoid the same issue.

The setup


My setup consists of Rundeck running behind a reverse proxy configured in NGINX and the whole box sits behind an Application Load Balancer (ALB). The whole shebang looks more or less like this:

Architecture diagram
Architecture diagram

The problem


I’m configuring SSL termination on the ALB level, which is way more convenient IMO, however, it was causing some weird problems that took me a while to catch. The main symptom was at the login form. There was no feedback for login attempts (failed or successful) and the login redirection wasn’t working. I was able to log in by clicking on the login button then manually going to the home URL.

Trying the login page with the Developer tools open on Chrome I noticed that it was calling the URL path /j_security_check but the request on Chrome was being shown as canceled in the Networking tab all the times (if you know why it behaves like that leave a comment). Then I decided to manually try the authentication with curl, the command I ran was:

curl https://rundeck.example.com/j_security_check -d 'j_username=user&j_password=pass' -X POST -v

And the result was interesting:

.... #curl verbose stuff here

< HTTP/2 302
< date: Sat, 16 Feb 2019 09:07:48 GMT
< location: https://rundeck.example.com:80/user/error
< server: nginx/1.14.0 (Ubuntu)
< set-cookie: grails_remember_me=;Path=/;Expires=Thu, 01-Jan-1970 00:00:00 GMT;Max-Age=0
< expires: Thu, 01 Jan 1970 00:00:00 GMT
< set-cookie: JSESSIONID=node01tl6fp5oi0qoh1x0g3r9kxwu83407.node0;Path=/;Secure;HttpOnly
<
* Connection #0 to host rundeck.example.com left intact

It was taking my standard HTTPS request and responding 302 with a location https://rundeck.example.com:80/user/error. The root cause was that Rundeck wasn’t picking the X-Forwarded-Proto header in the requests although I was forcing it on the NGINX config with the directive proxy_set_header X-Forwarded-Proto 'https';.

Back into de documentation page (which could have a full-text search tool) I discovered that Rundeck isn’t proxy aware by default but there is a way to configure it. Using their own wording:

You can tell Jetty to honor X-Forwarded-Proto, X-Forwarded-Host, X-Forwarded-Server and X-Forwarded-For headers in two ways:

In rundeck-config.properties you can set:

server.useForwardHeaders=true

— from https://docs.rundeck.com/docs/administration/security/configuring-ssl.html


The solution


Considering that you are configuring your server to respond to the URL https://rundeck.example.com the config files would look like that:

framework.properties

framework.server.port = 4440
framework.server.url = https://rundeck.example.com

... # Rest of the document goes here

rundeck-config.properties

server.useForwardHeaders = true
grails.serverURL = https://rundeck.example.com

... # Rest of the document goes here

NGINX vhost config

server {
  listen 80 default_server;
  server_name rundeck.example.com _;

  location / {
    proxy_set_header    Host 'rundeck.example.com';
    proxy_pass_header   X-Real-IP;
    proxy_pass_header   X-Forwarded-Proto;
    proxy_pass_header   X-Forwarded-Host;
    proxy_pass_header   X-Forwarded-Server;
    proxy_pass_header   X-Forwarded-For;

    proxy_pass          http://localhost:4440;
  }
}

This solution works for ALB and the Classic Load Balancer (a.k.a ELB). Note that I’m using port 80 so make sure that you set your load balancer health check on the appropriate port.

How about you? Feel free to share a battle story, feelings or suggestions in the comments section.