Apache: How to get an A (or A+) rating on securityheaders.com

A commenter on my old post about configuring security headers on Nginx (and getting an A or A+ rating on securityheaders.io, now moved to securityheaders.com) asked for a version for Apache, so… here it is. It’s basically the same, just with the appropriate syntax changes:

Nginx setting Explanation
header always set Strict-Transport-Security “max-age=31536000; includeSubdomains; preload” Enforces HTTPS on the entire site. Don’t use if you still need to provide HTTP, of course.
header always set X-Content-Type-Options nosniff Prevents browsers from trying to “guess” MIME types and such, forcing them to use what the server tells them.
header always set X-Frame-Options SAMEORIGIN Stops your site from being included in iframes on other sites.
header always set X-Xss-Protection “1; mode=block” Activates cross-scripting (XSS) protection in browsers.
header always set Referrer-Policy “unsafe-url” Makes the site always send referrer information to other sites. NOTE: this is not the most secure setting, but it’s the one I prefer; see below.
header always set Content-Security-Policy “default-src https: data: ‘unsafe-inline’ ‘unsafe-eval'” Forces TLS (don’t use if you still need to provide HTTP); prevents mixed content warnings. NOTE: again, this is not the most secure setting; see below.

About the last two settings, I’ll copy from my old post:

I choose to send referring information to other sites for a simple reason: I like to see where my own visitors come from (I don’t sell, share or monetize that in any way, it’s just for curiosity’s sake), and I’m a firm believer in treating others as I want to be treated. If you don’t care about that, you may want to change this to “no-referrer-when-downgrade“, or “strict-origin“.

As for the Content Security Policy, anything more “secure” than the above prevents (or at least makes it a lot more headache-inducing) the use of inline scripts and/or external scripts, which would mean no external tools/scripts/content (or at least a lot of work in adding every external domain to a list of exceptions), and a lot of hacking on software such as WordPress (seriously: add the setting, but then remove the “unsafe-*” options and see what stops working…). You might use the most paranoid settings for an internally developed, mostly static site, but I fail to see the point. So, this and the paragraph above are the reasons for an A rating, instead of an A+.

Hope this was useful. 🙂

How to get a 100% score on SSL Labs (Red Hat/CentOS 7.x, Apache, Let’s Encrypt)

There’s an Nginx version of this tutorial already, but since some people prefer/have to use Apache, here’s how to get 100% on SSL Labs’ server test with it.

For extra fun, instead of Debian/Ubuntu, we’ll be using Red Hat/CentOS (more precisely, CentOS 7.6 with the latest updates as of January 10th, 2019, but this shouldn’t change for any other 7.x versions of either CentOS or RHCE). And we’ll keep firewalld and SELinux enabled 1, even though, as far as I know, in most companies (well, at least the ones I’ve worked at) it’s, typically, official policy to disable both. 🙂

So, let’s assume you already have a CentOS/RHEL machine, with internet access and the default repositories enabled. First, we’ll enable the EPEL repository (to provide Let’s Encrypt’s certbot):

yum install https://dl.fedoraproject.org/pub/epel/epel-release-latest-7.noarch.rpm

Now we’ll install certbot (which will also install Apache’s httpd and mod_ssl as dependencies) and wget (which we may need later):

yum install python2-certbot-apache wget

To be run certbot and allow it to verify your site, you should first create a simple vhost. Add this to your Apache configuration (e.g. a new file ending in .conf in /etc/httpd/conf.d/):

<VirtualHost *:80>
    DocumentRoot "/var/www/html"
    ServerName mysite.mydomain.com
</VirtualHost>

Now add Apache’s ports to firewalld, and start the service:

firewall-cmd --permanent --zone=public --add-service=http
firewall-cmd --permanent --zone=public --add-service=https
firewall-cmd --reload

systemctl enable httpd ; systemctl restart httpd

Run certbot to create and begin using the new certificate:

certbot --apache --rsa-key-size 4096 --no-redirect --staple-ocsp -d mysite.mydomain.com

You should now have the virtual host configured in /etc/httpd/conf.d/ssl.conf . Let’s add a few lines to it (inside the <VirtualHost></VirtualHost> section):

Header always set Strict-Transport-Security "max-age=15768000"
SSLProtocol             all -SSLv3 -TLSv1 -TLSv1.1
SSLCipherSuite          ECDHE-ECDSA-CHACHA20-POLY1305:ECDHE-RSA-CHACHA20-POLY1305:ECDHE-ECDSA-AES256-GCM-SHA384:ECDHE-RSA-AES256-GCM-SHA384::ECDHE-ECDSA-AES256-SHA384:ECDHE-RSA-AES256-SHA384
SSLCipherSuite          TLSv1.3 TLS_CHACHA20_POLY1305_SHA256:TLS_AES_256_GCM_SHA384
SSLHonorCipherOrder     on
SSLCompression          off

(Don’t mind the “TLSv1.3” line for now, it won’t be used — if at all  — until later in this post, and won’t have any effect if it isn’t.)

This should get you an A+ rating, with all bars at 100% except for Key Exchange. A good start, right? 🙂 However, that final bar can be a bit tricky. The problem here is that SSL Labs limits the Key Exchange rating to 90% if the default ECDH curve is either prime256v1 or X25519, the first of them being the OpenSSL default… and Apache 2.4.6 (the version included in all 7.x RHELs/CentOSes) doesn’t yet include the option to specify a different list of curves (of which secp384r1 will give a 100% rating, and won’t add any compatibility issues).

Assuming you don’t want to install a non-official Apache, then the only way (as far as I know) to specify a new curve is to add the results of running:

openssl ecparam -name secp384r1

to your certificate file. But Let’s Encrypt certificates are renewed often, so you may need some trickery here — such as adding something like this to root’s crontab:

*/5 * * * * grep -q "EC PARAMETERERS" /your/certificate/file.crt || openssl ecparam -name secp384r1 >> /your/certificate/file.crt

Ugly, I know. 🙂

Alternatively, you can upgrade Apache to a newer version. CodeIT (for instance) provides replacement packages for RHEL and CentOS, which you can use by doing:

cd /etc/yum.repos.d && wget https://repo.codeit.guru/codeit.el`rpm -q --qf "%{VERSION}" $(rpm -q --whatprovides redhat-release)`.repo
yum -y update httpd mod_ssl

This will update Apache from 2.4.6 to 2.4.37 (as of January 10th, 2019), and so you can add the following to the vhost:

SSLOpenSSLConfCmd     Curves secp384r1:X25519:prime256v1

However! This will still not give 100%, since CoreIT’s Apache appears to come with a newer OpenSSL (1.1.1) statically linked to it, which means it’ll enable TLS 1.3 by default. And there’s nothing wrong with it… except that, from all the tests I’ve performed, Apache ignores the specified order of the ECDH curves when using TLS 1.3. With the line above, it’ll use secp384r1 for TLS 1.2, but X25519 for TLS 1.3 — and that’s enough to lower the Key Exchange bar to 90% again. 🙁

Given this, there are now several options:

1- disable TLS 1.3 (it’s still relatively new, and most people are still using 1.2): change the line:

SSLProtocol             all -SSLv3 -TLSv1 -TLSv1.1

to:

SSLProtocol             all -SSLv3 -TLSv1 -TLSv1.1 -TLSv1.3

2- force secp384r1: change the “SSLOpenSSLConfCmd Curves” line to simply:

SSLOpenSSLConfCmd       Curves secp384r1

3- ignore this situation altogether: X25519 is considered to be as secure as secp384r1, if not more; even SSL Labs mentions not penalizing it in the future in their (early 2018) grading guide — they just have been a bit lazy in updating their test. It’s quite possible that a future update of their tool will give a full 100% score to X25519.

Anyway, most of this was done for fun, for the challenge of it.  🙂

How to get a 100% score on SSL Labs (Nginx, Let’s Encrypt)

This post should be titled “How to get a 100% score on SSL Labs (Nginx, Let’s Encrypt) as of April 2018“, since SSL Labs‘s test evolves all the time (and a good thing it does, too.) But that would be too long a title. 🙂

The challenge:

As we’ve seen before, it’s relatively easy to get an A+ rating on SSL Labs. However, even that configuration will only give a score like this:

Zurgl.com's current SSL Labs score
Zurgl.com’s current SSL Labs score

Certainly more than good enough (and, incidentally, almost certainly much better than whatever home banking site you use…), but your weirdo of a host can’t look at those less-than-100% bars and not see a challenge. 🙂

First, note that all of the following will be just for fun, as the settings to get 100% will demand recent browsers (and, in some cases, recent operating systems/devices), so you probably don’t want this in a world-accessible site. It would be more applicable, say, for a webmail used by you alone, or by half a dozen people. But the point here isn’t real-world use, it’s fun (OK, OK, and learning something new). So, let’s begin.

Getting that perfect SSL Labs score:

For this example, I’ll be using a just-installed Debian 9 server, in which I did something like:

apt -y install nginx
echo "deb http://ftp.debian.org/debian stretch-backports main" >> /etc/apt/sources.list.d/stretch-backports.list # if you don't have this repository active already
apt -y install python-certbot-nginx -t stretch-backports

(The reason for the above extra complexity is the fact that the default certbot in Debian Stretch is too old and tries to authenticate new certificates in an obsolete way, but there’s a newer one in stretch-backports.)

I also created a simple index.html file on /var/www/html/ that just says “Hello world!”. Right now, the site doesn’t even have HTTPS.

So, let’s create a new certificate and configure the HTTPS server in Nginx:

certbot --nginx --rsa-key-size 4096 --no-redirect --staple-ocsp -d mysite.mydomain.com

(replacing mysite.mydomain.com, obviously.)

Testing it on SSL Labs, it gives… exactly the same bars as in the image above, but with an A rating instead of A+. Right, we need HSTS to get A+ (and currently certbot doesn’t support configuring it automatically in Nginx), so we add this to the virtual host’s server section:

add_header Strict-Transport-Security "max-age=63072000; includeSubdomains; preload";

(Note: remove “includeSubDomains;” if you have HTTP sites on any subdomain. Also, HSTS makes your site HTTPS-only, so you can’t use it if you want to keep an HTTP version — in which case you can’t get better than an A rating.)

This upgrades the rating to A+, as expected, but the bars still didn’t change. Let’s make them grow, shall we?

Certificate:

It’s already at 100%, so yay. 🙂

Protocol Support:

Just disable any TLS lower than 1.2. In this case, edit /etc/letsencrypt/options-ssl-nginx.conf and replace the ssl_protocols line (or comment it and add a new one) with:

ssl_protocols TLSv1.2;

WARNING: any change you make to the file above will affect all virtual hosts on this Nginx server. If you need some of them to support older protocols, it’s better to just comment out that option in that file, and then include it in each virtual host’s server section.

EDIT: actually, it turns out that ssl_protocols affects the entire server, with the first option Nginx finds  (first in /etc/nginx.conf, then in the default website (which possibly includes /etc/letsencrypt/options-ssl-nginx.conf)) taking precedence. So just choose whether you want TLS 1.0 to 1.2 (maximum compatibility) or just 1.2 only (maximum security/rating); if you need both, use two separate servers.

Key Exchange:

We’ve already created the new certificate with a 4096-bit key (with “rsa-key-size 4096“, see above), and according to SSL Labs’ docs this should be enough… but it isn’t. After some googling, I found out that you also need to add the following option inside the server section in Nginx:

ssl_ecdh_curve secp384r1;

Since the default Nginx+OpenSSL/LibreSSL setting, either “X25519” or “secp256r1” (actually “prime256v1“), also lowers the score.

EDIT: again, ssl_ecdh_curve affects the entire server, so you can’t use different default curves for each virtual host. So I’d suggest (in /etc/letsencrypt/options-ssl-nginx.conf):

ssl_ecdh_curve secp384r1:X25519:prime256v1;

if you want the 100% rating, or:

ssl_ecdh_curve X25519:prime256v1:secp384r1:

otherwise. This one doesn’t affect compatibility, by the way; it’s just a question of the preferred order.

The certificate’s key size (4096 or 2048) is, like the certificate itself, specific to each virtual host.

Cipher Strength:

For 100% here, you need to disable not only any old protocols, but also any 128-bit ciphers. I started from Mozilla’s SSL tool (with the “Modern” option selected), then removed anything with “128” in its name, then moved ChaCha20 to the front just because, and ended up with this line, which I added to /etc/letsencrypt/options-ssl-nginx.conf (replacing the current setting with the same name):

ssl_ciphers 'ECDHE-ECDSA-CHACHA20-POLY1305:ECDHE-RSA-CHACHA20-POLY1305:ECDHE-ECDSA-AES256-GCM-SHA384:ECDHE-RSA-AES256-GCM-SHA384::ECDHE-ECDSA-AES256-SHA384:ECDHE-RSA-AES256-SHA384';

(This time, this setting can indeed be specified for each virtual host, though you’d need to comment it out in the file above (which affects all virtual hosts with Let’s Encrypt certificates) if you want to do so, otherwise they’ll conflict.)

The result:

100% on SSL Labs

As said near the beginning, this is not something you likely want to do (yet) for a production, world-accessible site, as it will require relatively modern browsers, and you typically can’t control what visitors use1. According to SSL Labs’ list, it’s not actually that bad: all versions of Chrome, Firefox, or Safari from the past couple of years are fine, but no Microsoft browser older than IE11 will work, nor will Android’s default browser before 7.0 (Chrome on Android, which is not the same thing, will do fine.) I’d suggest it for a site where you can “control” the users, such as a small or medium-sized company’s webmail or employee portal (where you, as a sysadmin, can and should demand up-to-date security from your users).

But the main point of this exercise was, of course, to see if I could do it. Challenging yourself is always good. 🙂 And if you can share what you learned with others, so much the better.

Red Hat/CentOS: How to add HTTPS to an existing Nginx website (both with and without Let’s Encrypt)

(Yes, you read the title correctly. For extra fun, and to prevent this blog from being too focused on Ubuntu/Debian, this time I’ll be using Red Hat Enterprise Linux / CentOS (and, I assume, Fedora as well.) Later on, I may post a Debian-based version.)

Configuring a basic HTTP site on Nginx

(Note: if you already have a working HTTP site, you can skip to the next section (“Adding encryption…”))

Yes, the post title mentions an “existing” website, which I believe will be the case in most “real world” situations, but installing a new one is actually very easy on CentOS1. First, do:

yum -y install epel-release; yum -y install nginx

Then create a very basic configuration file for the (non-HTTPS) site, as /etc/nginx/conf.d/mysite.conf :

server {
        listen 80;
        server_name mysite.mydomain.com;
        root /var/www/mysite;

        location / {
        }
}

Then, of course, create the /var/www/mysite directory (CentOS doesn’t use /var/www by default, but I’m far too used to it to change. 🙂 ) If you’d like, create an index.html text file in that directory, restart nginx (“service nginx restart” or “systemctl restart nginx“, depending on your system’s major version), and try browsing to http://mysite.mydomain.com . If it works, congratulations, you have a running web server and a basic site.

Adding encryption to the site (not using Let’s Encrypt):

First, you need a key/certificate pair. I have tutorials for creating an RSA certificate or an ECDSA certificate.

Second, edit the site’s configuration file (in the “starting from scratch” example above, it’s “/etc/nginx/conf.d/mysite.conf“), and copy the entire server section so that it appears twice on that text file (one after the other). Pick either the original or the copy (not both!), and, inside it, change the line:

listen 80;

to:

listen 443 ssl http2;

(Note: the “http2” option is only available in Nginx 1.9.5 or newer. If your version complains about it, just remove it, or upgrade.)

Inside that section, add the following options:

ssl_certificate /path/to/certificate.crt;
ssl_certificate_key /path/to/privatekey.crt;

(replacing the paths and file names, of course.)

This should be enough — restart Nginx and you should have an HTTPS site as well as the HTTP one.

And what if you want to disable HTTP for that site and use HTTPS only? Just edit the same configuration file, look for the server section you didn’t change (the one that still includes “listen 80;“), and replace the inside of that section with:

listen 80;
server_name mysite.domain.com;
return 301 https://mysite.domain.com$request_uri;

Afterward, while you’re at it, why not go for an A+ rating on SSL Labs (skip the certification creation part from that tutorial, you’ve done it already) and an A rating on securityheaders.io?

Adding encryption to the site (using Let’s Encrypt):

First, install certbot:

yum install python2-certbot-nginx

Then check if Nginx is running and your normal HTTP site is online (it’ll be needed in a short while, to activate the certificate). If so, then enter:

certbot --nginx --rsa-key-size 4096 -d mysite.mydomain.com

(replacing “mysite.mydomain.com” with yours, of course.)

Answer the questions it asks you: a contact email, whether you agree with the terms (you need to say yes to this one), if you want to share your email with the EFF, and finally if you want “No redirect” (i.e. keep the HTTP site) or “Redirect” (make your site HTTPS only).

And that’s it (almost — see the next paragraph) — when you get the shell prompt back, certbot will already have reconfigured Nginx in the way you chose in the paragraph above, and restarted it so that it’s running the new configuration. You may want to add “http2” to the “listen 443 ssl;” line in the configuration file (it’ll probably be the default someday, but as of this post’s date it isn’t), and don’t forget your options for improved security and security headers.

Only one thing is missing: automatically renewing certificates. Strangely, the certbot package configures that automatically on Ubuntu, but not on CentOS, from what I’ve seen (please correct me if I’m wrong). The official Let’s Encrypt docs recommend adding this (which includes some randomization so that entire timezones don’t attempt to renew their certificates at precisely the same time) to root’s crontab:

0 0,12 * * * python -c 'import random; import time; time.sleep(random.random() * 3600)' && certbot renew 

(Note: It’s possible to use Let’s Encrypt to create ECDSA certificates, but as of this writing you have to do most of the work manually (creating a CSR, etc.), and you lose the automatic renewal, so for the moment I suggest using RSA certificates. I hope this changes in the future.)

Nginx: How to prevent it from logging local hits

Like any web server, Nginx logs all accesses/hits by default, and if you have some kind of log-based analytics tool (such as AWstats) you probably have it already set to ignore hits from the host itself (usually by skipping both localhost (127.0.0.1) and the server’s public IP address). If you don’t do so, then your statistics will probably be inflated (“wow, my newly created site is surprisingly popular!“), and, unless your site has many regular users, the IP address at the top of the “visitors” table will almost certainly be your own. Local hits can come from several culprits: for instance, some web software such as WordPress or MyBB use special URLs as a form of cron replacement, and also you may have yourself be monitoring your site in some way (such as, in my case, benchmarking a particular URL and MRTGing the access speed).

OK, so your analytics don’t show it anymore — but what about the log files themselves? Maybe you don’t want a large percentage of them being composed of internal hits (especially in the case of those “let’s measure the average response speed for 100 hits every 5 minutes” benchmarks, which for that site made up some 95% of its access logs…). Maybe it even interferes with some other tools you’re using, such as something like DenyHosts or Fail2ban, to detect some abuse patterns, even after whitelisting your external IP. How about having the option not to log them at all? 1

On Nginx, this is actually pretty easy to do.

1- in the Nginx main configuration file, add the following:

map $remote_addr $notlocal {
        default 1;
        ~^YOUR_EXTERNAL_IP$ 0;
        ~^127.0.0.1$ 0;
}

Replacing YOUR_EXTERNAL_IP with… you can probably guess. 🙂

Important: if your virtual hosts’ configurations are in another directory that is “included” from nginx.conf, take care to add the above before the include lines (e.g. include /etc/nginx/conf.d/*.conf) . Otherwise, the virtual hosts won’t have the “$notlocal” variable defined yet, and Nginx won’t start because of it.

2- for each virtual host where you want to stop logging local hits, edit the access_log line, changing it from something like:

access_log /var/log/nginx/mysite-access.log;

to:

access_log /var/log/nginx/mysite-access.log combined if=$notlocal;

Note the addition of “combined“. “combined” is the default format for Nginx access logs, so specifying it isn’t usually needed, but apparently if you want to specify the “if” condition, it must come after the log format option (otherwise, I guess Nginx thinks you want a log format of “if“, which of course doesn’t exist and will prevent the server from starting.)

That it! Restart Nginx, and enjoy your much cleaner/smaller logs.