Damir's Corner.

HTTPS in TeamCity with Let's Encrypt Certificate

When exposing your TeamCity build server to the internet, you'll want to use HTTPS so that users won't have to send their passwords over an unencrypted connection. Thanks to Let's Encrypt, you can now get the SSL certificate for free, but there is still some work involved to get everything configured correctly.

Obtaining the Certificate

I've depended a lot on another blog post to get Let's Encrypt working on Windows. You might want to check it for additional details. Below are the steps as I did them.

First, download letsencrypt-win-simple Windows client. There's an issue with auto renewal in the latest version, so you should use version 1.8 instead until it's fixed. Unpack the archive to a location of your choice, e.g. C:\Program Files (x86)\letsencrypt-win-simple.

Use the tool to generate a test certificate and make sure you have everything configured correctly before generating the real one. To verify your ownership of the domain, Let's Encrypt server will check some files generated by the tool. They need to be publicly accessible at http://<your-domain>/.well-known. I serve these files from IIS, which requires the following web.config in the root folder (e.g. C:\WWW\build) to allow serving files without extension:

<?xml version="1.0" encoding="UTF-8"?>
<configuration>
  <system.webServer>
    <staticContent>
      <mimeMap fileExtension="." mimeType="text/plain" />
    </staticContent>
  </system.webServer>
</configuration>

Once this is configured, you can invoke the tool:

letsencrypt.exe --manualhost <your-domain> --webroot "C:\WWW\build" --test

This will generate the required files in C:\WWW\build\.well-known. If domain verification succeeds, the certificate will be generated in the Local Computer Web Hosting store. This means that you are ready to run the above command without the --test flag to obtain the real certificate:

letsencrypt.exe --manualhost <your-domain> --webroot "C:\WWW\build"

TeamCity Configuration

TeamCity uses Tomcat as its web server. By default, you can find its configuration in C:\TeamCity\conf\server.xml. Open the file and find any <Connector elements. The one for port 80 should match the following:

<Connector port="80" protocol="org.apache.coyote.http11.Http11NioProtocol"
    connectionTimeout="60000"
    redirectPort="8543"
    useBodyEncodingForURI="true"
    socket.txBufSize="64000"
    socket.rxBufSize="64000"
    tcpNoDelay="1"
/>

Comment it out and insert the following two elements instead:

<Connector port="80" protocol="org.apache.coyote.http11.Http11NioProtocol"
    redirectPort="443"
/>

<Connector port="443" protocol="org.apache.coyote.http11.Http11NioProtocol"
    SSLEnabled="true"
    scheme="https" secure="true"
    connectionTimeout="60000"
    redirectPort="8543"
    clientAuth="false"
    sslProtocol="TLS"
    useBodyEncodingForURI="true"
    socket.txBufSize="64000"
    socket.rxBufSize="64000"
    keystoreFile="C:\TeamCity\conf\teamcity.pfx"
    keystorePass="teamcity"
    keystoreType="PKCS12"
/>

The first one redirects any HTTP requests to HTTPS, while the second one configures the HTTPS access. Its most important part are keystore attributes which specify the pfx file to read the certificate from. Of course, you'll need to export the certificate from the store into the specified file and set a matching password.

I prevented external HTTP access to the server by only exposing HTTPS port to the internet. Alternatively, you'll want to force HTTPS connection on server.

Pay attention when installing future TeamCity updates. You'll very likely have to reapply the above changes to server.xml.

Automatic Certificate Renewal

The issued certificates are only valid for 3 months. The tool will automatically renew them after 2 months when invoked with --renew flag:

letsencrypt.exe --renew"

It even creates a scheduled task to run this command daily. However, the above configuration requires additional steps:

• Export the certificate into a pfx to make it available to Tomcat.
• Restart TeamCity server to make it use the new certificate.

I wrote a PowerShell script to take care of all that:

# attempt to refresh the certificate
& 'C:\Program Files (x86)\letsencrypt-win-simple\letsencrypt.exe' --renew

# check if a new certificate has been obtained
$now = Get-Date
$nowWithBuffer = $now.AddHours(-1)
$newCertificatesMeasure = Get-ChildItem Cert:\LocalMachine\WebHosting |
    Where-Object { $_.NotBefore -gt $nowWithBuffer } |
    Tee-Object -Variable newCertificates
    Measure-Object
If ($newCertificatesMeasure.Count -ne 1) {
    exit
}

# export the certificate
$newCertificates | ForEach-Object {
    &CertUtil.exe @('-f', '-p', 'teamcity', '-exportpfx', 'WebHosting', $_.Thumbprint, "C:\TeamCity\conf\teamcity.pfx")
}

# restart TeamCity to use the new certificate
Restart-Service TeamCity

The only tricky part is detecting whether a new certificate has been obtained. Whenever that happens, the tool puts it into the Local Machine Web Hosting store next to existing certificates. The latest certificate can be most reliably detected based on its start of validity. That's why I'm checking if this value is within the last hour. If I find such a certificate, I know it's a new one, hence I export it and restart TeamCity. Otherwise, the script just exits.

I disabled the original scheduled task created by the tool and created a new one to run this script instead. To capture the output of the script into a log file for future diagnostics, I invoked PowerShell with the following arguments:

-NonInteractive -NoLogo -NoProfile -Command "& {C:\Users\Administrator\Documents\letsencrypt.ps1; Return $LASTEXITCODE}" 2>&1> C:\Users\Administrator\Documents\letsencrypt.log