Charles Engelke's Blog

We finished setting up Apache web server on Google Compute Engine in the last post. Now we’re finally ready for the whole point of this exercise: getting a free certificate from Let’s Encrypt for the new site at https://www.bibliote.ch/. We’re going to use certbot, so we follow that link to their site, and see a pretty user-friendly starting point:

We’re using Apache on CentOS 7 (a.k.a. RHEL 7) so we make those choices and get our instructions. But rather than following them (which include a step not needed on our Google image), here are the steps: SSH to your instance and issue the following commands.

sudo yum install -y python-certbot-apache
sudo certbot --apache
sudo service httpd graceful

You’ll have to answer some questions when certbot runs, but the answers are obvious or the default just works. And that’s it. If you go to your site on a fresh browser window you should no longer see any security warnings. This seems almost anticlimactic.

What just happened? Let’s Encrypt issues domain validation (DV) certificates. That means that they need to verify that the certificate requester (that’s us, here) is the authorized controller of the domain(s) to put on the certificate. Some certificate authorities will have the requester add a special record to their DNS; others ask them to put a special file somewhere on a web server in the domain. That latter is what Let’s Encrypt does. But certbot handles that automatically: it creates virtual hosts for each domain name on the certificate, starts them up on non-standard ports, and waits for Let’s Encrypt to check them. Then it gets rid of the domains and fetches and installs the certificate. It edits your Apache configuration files as necessary to do these steps. It’s all automatic and, in my experience, trouble-free.

The other possible kinds of certificates are organization validation (OV) and extended validation (EV). These require the certificate authority to check out who the requester is in the real world, such as checking company registration or business directories. That requires human interaction, so it takes more time and costs more. It’s not clear that these kinds of certificates actually protect web users any more than DV ones. In any case, this is a personal site and DV fits our needs fine.

And that’s it. Well, almost it. Your new certificate will expire in 90 days and the scary browser messages will come back. To solve that, put the following two lines in the system crontab using the command sudo crontab -e to open it in the vi editor. Type an i to enter insert mode, type a variant of these lines, then hit the Escape key and ZZ (in caps) to get out of vi. (Yes, vi is really strange. The standard joke for it is “I’ve been using vi for years. Because I can’t figure out how to exit it.”)

17 6 * * * /bin/certbot renew --quiet
17 18 * * * /bin/certbot renew --quiet

This will make certbot check to see if the certificate is within 30 days of expiration, and it will automatically renew it if so. The check will run every day at 0617 and 1817 (UTC). Why those times? Let’s Encrypt suggests checking every 12 hours at a random starting time so everybody doesn’t hit their servers at the same time. I asked Google for random numbers between 1 and 12 for the hour and 0 and 59 for the minute and got 6 and 17.

We’ve done what we set out to do, but let’s make sure we not only have a secure web site, but a well-configured secure web site. The Qualys SSL Labs SSL Server Test site is a fantastic way to see how secure your server really is. Just go to that page and enter your URL. In a few minutes you’ll get a score:

Yech. That’s a passing score, but just barely. Reading the fine print, we see that the server supports the obsolete (and insecure) SSL 3 protocol, the now deprecated RC4 cipher, and doesn’t support new algorithms that enable forward secrecy. If you’d like to learn more about those things I strongly recommend the Bulletproof SSL and TLS book by Ivan Ristić. Ristić is one of the people behind the SSL test page we just used, and this book is authoritative and clear. A few years ago I spent several days studying TLS to work out the best configuration for my servers. But now there’s another option.

If you don’t want to study TLS in depth you can do what I do now: use the Mozilla SSL Configuration Generator. Fill in your web server (Apache), version number (2.4.6, displayed using httpd -v), and OpenSSL version number (1.0.1e, displayed via openssl version), and then select how aggressive you want to be on security. Older browsers can’t handle the most secure options, so if you want to support them you’ll select Old or Intermediate and live with lesser security. I selected Modern instead, to get the most secure options that work with pretty much anything at all recent. Based on what that shows, you’ll need to edit both /etc/httpd/conf.d/ssl.conf and /etc/httpd/conf.d/ssl-www.conf. For each of them, remove or comment out the old lines for SSLProtocol and SSLCipherSuite, and add the following lines:

SSLProtocol all -SSLv3 -TLSv1 -TLSv1.1
SSLCipherSuite ECDHE-ECDSA-AES256-GCM-SHA384:ECDHE-RSA-AES256-GCM-SHA384:ECDHE-ECDSA-CHACHA20-POLY1305:ECDHE-RSA-CHACHA20-POLY1305:ECDHE-ECDSA-AES128-GCM-SHA256:ECDHE-RSA-AES128-GCM-SHA256:ECDHE-ECDSA-AES256-SHA384:ECDHE-RSA-AES256-SHA384:ECDHE-ECDSA-AES128-SHA256:ECDHE-RSA-AES128-SHA256
SSLHonorCipherOrder on
SSLCompression off
Header always set Strict-Transport-Security "max-age=15768000"

Note that the next line after SSLCipherSuite should be SSLHonorCipherOrder. Your browser will likely split that long SSLCipherSuite line and you’ll need to put it back together.

What do these lines do? The SSLProtocol line essentially says “use any protocol version except for the known vulnerable ones SSL version 3 and TLS version 1 or version 1.1.” At the moment, that means use TLS version 1.2, though version 1.3 is getting close to being finalized. Note that there are even older versions of SSL that the line doesn’t exclude, but Apache and OpenSSL never support them, anyway.

The SSLCipherSuite and SSLHonorCipherOrder tell the server which encryption algorithms to allow, and their preferred order. This is the line that requires either many days of study to figure out, or the use of Mozilla’s great generator. These combinations of algorithms are the ones still considered secure as of today. Not every browser supports every algorithm, which is why there are so many choices listed.

Turning off SSLCompression seems like a bad idea for performance, but it really doesn’t do any harm there, and it eliminates one kind of cryptographic vulnerability. Finally, the Strict-Transport-Security header that’s now always being sent tells the browser to never again even attempt to connect to the site via plain HTTP, and to never accept an invalid certificate in the future.

Now, run sudo service httpd graceful to reload the configuration file, and test again (if you still have the old page open, just click the Clear cache link to do that). Now we get a much better score:

That’s more like it! We should check this periodically to make sure our certificate is getting renewed properly and that there are no newly discovered vulnerabilities we should address. Renewal is particularly important now, because the Strict-Transport-Security header will cause browsers to refuse to connect to your site again if the certificate expires.

After several steps needed to set up any web server, and some very simple ones to get a certificate, we have a good secure server that we can do almost anything with, and it’s costing less than $5 per month.

Now that we’ve got a server in the cloud, we can install and configure the Apache web server for our site. The first step is using the cloud console to SSH to the server as shown in the last post. Once there, we will use the yum package manager to install the web server, as shown below:

sudo yum -y update
sudo yum -y install httpd mod_ssl

Now let’s see if it works. Try to start the server with sudo systemctl start httpd and then point your browser to the IP address of your server. You should see a default welcome page. If you use https, you’ll see a browser warning because the installed certificate is self-signed. We’ll fix that latter problem in the next post using Let’s Encrypt and certbot.

Assuming the page fetch worked, issue one more command to make the web server start automatically after a reboot: sudo systemctl enable httpd.

We have the web server, but we still need to configure it. We’re going to want this to be our site’s primary web page. I’m using the domain bibliote.ch for this, so I’m going to want http://bibliote.ch, http://www.bibliote.ch, and https://bibliote.ch all to redirect to https://www.bibliote.ch, which is where I’ll serve all my content.

My first step is out of scope for these posts: I need to set up two DNS A records, both pointing to my site’s external IP address. Those records will be for the names @ (which is for the bare domain name) and www. I have my DNS hosted by my domain registrar, and each registrar has some way to do this through their web page. Once you set this up it will take a while for the changes to occur, but we can continue with configuration while waiting. We can’t use Let’s Encrypt, though, until the name records are live.

The second step is to configure the web server to deal with those redirects.

The configuration files are installed in /etc/httpd. The main server configuration is in /etc/httpd/conf/httpd.conf and the secure server configuration is in /etc/httpd/conf.d/ssl.conf. We will need to edit each of these files. Our regular account doesn’t have permission to write these files, so we will use sudo vi /etc/httpd/conf/httpd.conf and sudo vi /etc/httpd/conf.d/ssl.conf to edit them.

The regular, non-secure, configuration changes are easier. In fact, we just need to add two lines, which can go almost anywhere in the file. I put them near the end, just before the last line that includes other files. Here they are:

RewriteEngine on
RewriteRule "^/(.*)" "https://www.bibliote.ch/$1" [R,L]

The first line turns on the module that allows directives that rewrite browser requests. The second line says that any request for a URL starting (^) with a slash (/, which all start with from the server’s perspective), followed by anything (.*), should result in telling the browser to instead request the page https://www.bibliote.ch/$1, where the $1 will be replaced by the matched part of the URL in the parentheses. The [R,L] at the end of the line means to send a redirect response to the client and to stop looking for and applying other rewrite rules.

I also made an unnecessary, but recommended, change by uncommenting the line that starts with #ServerName and replacing it with ServerName www.bibliote.ch:80.

After making those changes, I gracefully restarted the service with the command sudo service httpd graceful. After that, pointing the browser to the external IP address caused the browser to redirect to https://www.bibliote.ch, which is what I want. Once the DNS changes propagated, pointing to http://bibliote.ch or http://www.bibliote.ch each also pointed correctly to https://www.bibliote.ch.

We just need to make a couple more Apache configuration tweaks. We want https://www.bibliote.ch to be the canonical URL for the site, so we need https://bibliote.ch to redirect there, too.

The secure site is controlled by a file in /etc/httpd/conf.d: ssl.conf. That provides a single virtual host running TLS on port 443, so requests to any name that resolves to the external IP will be served by it. We’re going to need two such virtual hosts, one for https://bibliote.ch that redirects all requests, and finally one for https://www.bibliote.ch that handles the web pages.

The easiest way I see to do this is to copy the ssl.conf file to ssl-www.conf in the same directory. They will both be loaded in alphabetical order. We will then edit ssl.conf to serve a redirect for https://bibliote.ch and edit ssl-www.conf to serve the canonical site https://www.bibliote.ch.

After copying the file, edit ssl.conf as follows:

• Uncomment the line starting with #ServerName, and change the hostname in that line to your bare domain name (bibliote.ch in this case).
• Near the end, just before the </VirtualServer line, add the two lines to serve the redirect, just as we did for the regular sites:

RewriteEngine on
RewriteRule "^/(.*)" "https://www.bibliote.ch/$1" [R,L]

Finally, edit ssl-www.conf. Get rid of all the lines before the <VirtualServer one, because they would repeat what’s in ssl.conf. Then uncomment the #ServerName line and change the host name to your canonical site’s name: www.bibliote.ch in this case. Finally, uncomment the #DocumentRoot line so that it will point to /var/www/html, which is where you’ll put your content. Use sudo vi /var/www/html/index.html to create a holder web page:

<!DOCTYPE html>
<html>
<head><title>Placeholder</title></head>
<body><h1>Placeholder</h1></body>
</html>

Now reload the configuration files with sudo service httpd graceful, and try each of your possible site names. All of them should redirect to https://www.bibliote.ch, though each secure site will show a security warning because we don’t have valid certificates yet. That’s the topic for the next post.

Last time: overview of a project to set up a webserver with Let’s Encrypt on Google Compute Engine.

My website is still old-style. It runs on a Linux server with Apache. So step one is getting a server using Google Compute Engine. You can use any other cloud or hosting service that offers an acceptable version of Linux if you don’t want to use Google.

Go to console.cloud.google.com and log in with a Google account. If you’ve never used any Google Cloud Platform services before you’ll have to agree to some terms right away. Google changes the layout of the console all the time, but there should be a menu on the side that has Compute Engine as a choice. Click that, and you’ll immediately be given the chance to Create a project:

Create a project start screen

Pick any name you want (I used home-page) and click Create. You’ll be told to set up a billing account if you don’t have one. On a new account you should be given an offer to try Cloud Platform for free by accepting the terms, and then fill in payment information. You shouldn’t actually get charged anything for the things we do here, but once the account exists and the free trial expires or is used up you could choose to use paid services, so Google requires you to set up payment.

After a minute or two, your account will be activated and you can get started. Click VM instances in the menu on the left, and then the Create button in the middle of the page.

Now you can fill in details of the instance type you want. I named mine webserver, more or less randomly picked the us-central1-c zone (which is in Ohio) with a micro machine type, and chose a 10 GB CentOS 7 boot disk. I picked CentOS 7 because it’s well supported by certbot, which I’ll use to get my server certificate later. I’ll also checked the boxes to allow HTTP and HTTPS traffic. When all is ready, the page looks like this:

Click Create, and Compute Engine starts setting up your new server. In a couple of minutes, you should see a green check mark indicating your server is up and running:

We’re just about done with this first step, except for the important question of just how we connect to and use this server. We can try using SSH on the command line to connect to the External IP, but what are the credentials we need?

Google actually makes connecting very easy. The Connect column is a drop-down menu, and one of the options is Open in browser window. Select that, and Google creates an SSH key pair for you, pushes the public key to the server, and opens an SSH terminal session in your browser. It couldn’t be easier. We can use this shell to configure Apache and certbot over the next two posts.

If you prefer to use your own SSH client, the drop down menu will take you to instructions. I thought that would be my preference, but it turns out I prefer the browser client or the gcloud command line. Either of those options handles SSH key pairs for you automatically, which is handy. The gcloud command line also makes it easy to transfer files to and from your instance as well as connect to a shell.

Next time: installing and configuring Apache web server.

About a year ago I decided I wanted my personal website to be secure. That meant getting a TLS certificate, installing and configuring a web server, and setting things up so that insecure requests were redirected to secure ones. I got a free StartSSL certificate, set everything up on Amazon Web Services, and practiced benign neglect toward it.

Everything was fine for about year, and then I got a notice that my certificate would soon expire and need to be renewed. I could have just renewed the certificate, but I found several new problems:

1. My site’s score on SSL Labs server test had dropped from an A to an F due to an Apache vulnerability I had not patched.
2. StartSSL may be free, but it’s a pain to deal with, and I’ve been hearing great things about Let’s Encrypt and wanted to try it out.
3. StartSSL is part of StartCom, and they have been making browser vendors upset with their security practices. So upset, in fact, that both Mozilla and Chrome will no longer treat their certificates as trusted.
4. A highly recommended tool to use with Let’s Encrypt, certbot, didn’t seem to fully support the AWS Linux environment my server was using.

So I went with my preferred way of updating systems: build a new one from scratch then throw the old one away. Working in the cloud makes this easy. And if I’m going to start from scratch, I want to try new things out in the process.

My new environment is on Google Compute Engine, running on a micro Linux server instance. The console tells me that will cost $4.28 per month, which is a great deal. But last week at Google Cloud Next they announced their new free tier, which includes one such instance at no cost. So hosting for my site will actually cost me nothing. The Let’s Encrypt certificate is also free, and all the software I’m using is free and open source, too. In fact, the engelke.com domain is the only thing I need to pay for.

This was a worthwhile exercise for me, so I’m going to use a few posts here to go over the steps it took to do it. First, I’ll go over how to create and connect to a new Linux instance on Compute Engine, then installing and configuring Apache web server, and finally setting up certbot to install and automatically renew certificates from Let’s Encrypt. Watch this space over the next few days for those posts.