AWS ELB and Secure WebSockets with Play Framework

UPDATE 2016/08/10: AWS improved their ELB and they now support websocket and http/2 protocols. More on this link https://aws.amazon.com/blogs/aws/new-aws-application-load-balancer/.

AWS ELB does not support WSS protocol on its HTTPS endpoints. If you are using it for load balancing this becomes a blocker for scaling your service. Hopefully there is a way to overcome this limitation.

Switching ELB protocols to TCP/SSL will not be enough as the server will not receive X-Forwarded-For header anymore.

To solve this you will need to

• Add ProxyProtocol policy to ELB so it starts using proxy_protocol
• Enable proxy_protocol support on nginx (Play unfortunatelly cannot be configured to understand proxy protocol at this moment :( )

Proxy protocol adds additional header to requests to pass server client’s ip which can be used if there is a load balancer between your server and clients.

How it looks?

PROXY_STRING + single space + INET_PROTOCOL + single space + CLIENT_IP + single space + PROXY_IP + single space + CLIENT_PORT + single space + PROXY_PORT + "\r\n"

Configuring ELB

Adding policy to ELB is for now only available through aws-cli. You can download it from here.

1. Add Policy

aws elb create-load-balancer-policy \
    --load-balancer-name <AWS_ELB_NAME> \
    --policy-name My-proxy-protocol \
    --policy-type-name ProxyProtocolPolicyType \\
    --policy-attributes AttributeName=ProxyProtocol,AttributeValue=True

2. Configure ELB

3. Configure nginx on your instance

Enable proxy_protocol support on nginx on your instance (and use it as a template for any other instance in your scaling pool).

# ELB PROXY protocol handler
server {
        listen 81 proxy_protocol;
        listen [::]:81 proxy_protocol;

    server_name _;

    root /var/www/html;

    set_real_ip_from 172.31.48.0/20; # ELB CIDR

    real_ip_header proxy_protocol;
    proxy_set_header X-Real-IP       $proxy_protocol_addr;
    proxy_set_header X-Forwarded-For $proxy_protocol_addr;

    location / {
            proxy_pass http://127.0.0.1:9000/;
            proxy_http_version 1.1;
            proxy_set_header Upgrade $http_upgrade;
            proxy_set_header Connection "upgrade";
            proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
            proxy_set_header Host $host;
    }

}

Important notes:

• We use Nginx here to pass requests to Play running on port 9000 on the same instance.
• Port 80 is kept here to redirect requests to https and it cannot use proxy protocol as ELB will not enable proxy protocol on HTTP/HTTPS listeners (which is a must on port 80)

Here is a sample nginx configuration for port 80

# Redirect everything to https
server {
        listen 80 default_server;
        listen [::]:80 default_server;

    server_name _;

    return 301 https://$host$request_uri;

}

4. Attach policy to ELB

Attach policy to ELB so it adds proxy protocol to all requests that are passed to port 81 on the instance

aws elb set-load-balancer-policies-for-backend-server \
    --load-balancer-name <ELB_NAME> \
    --instance-port 81 \
    --policy-names My-proxy-protocol

5. Open port 81 for EC2 security group :)

Just a reminder :)

6. Test it out

Test your setup by using 'wss://' in your requests. Websocket.org gives you an easy way to test websockets.