If you are setting up your own Public Key Infrastructure (PKI) or Certificate Authortiy (CA), the EasyRSA utility from OpenVPN can be extremely useful.

This tool is particularly suitable for development or other non-production Neo4j deployments, but any well protected and carefully managed PKI structure can be used in production just as easily, and with the same level of security.

EasyRSA is essentially just a wrapper around OpenSSL, but takes care of the finer points of setting up and maintaining a private key infrastructure. Including key generation, certificate issuing and renewals, as well as more involved options such as Subject Alternative Names (SANs) and other X509v3 extensions.

Preparing the PKI Directory and Certificate Authority

Installing EasyRSA is failry straightforward, with packages available for most operating systems, including Homebrew as "easy-rsa". This article covers its usage in a Linux, Unix-like, or OSX environment.

Following installation, it is advisable to define a memorable base location for building certificates, through by the EASYRSA_PKI environment variable:

export EASYRSA_PKI=/srv/dev-ca

We then initialise the EasyRSA's PKI structure here:

easyrsa init-pki

  You are about to remove the EASYRSA_PKI at: /srv/dev-ca
  and initialize a fresh PKI here.

  Type the word 'yes' to continue, or any other input to abort
  Confirm removal: yes

  init-pki complete; you may now create a CA or requests.
  Your newly created PKI dir is: /srv/dev-ca

Above command will create a new PKI directory at the specified path. The next step is to initialise the certificate authority (CA)

For the focused scope of this article, we can create a flat, one-level Root CA, so we can simply "build" a CA and not have to worry about generating a request and getting it signed. It is best to add a memorable name for this CA. In the following example, we may use "Neo4j Dev Cluster CA":

easyrsa build-ca

  Common Name (eg: your user, host, or server name) [Easy-RSA CA]:Neo4j Dev Cluster CA

  CA creation complete and you may now import and sign cert requests.
  Your new CA certificate file for publishing is at:
  /srv/dev-ca/ca.crt

You could add the "nopass" keyword at the end of this command to avoid having a password on the CA private key, but this is less secure so would not be suitable for production PKI.

It is also possible to set up a Root CA and then then build an intermediate CA off the root, if preferred, albeit, that is beyond scope for this article.

Generating the Keys and Certificates

With the CA built, we may begin building certificates for our Neo4j deployment, which can be achieved by a single command with EasyRSA, however there are some important things to note:

• The example uses "nopass" to avoid having password-protected private keys
• We need to use build-serverClient-full to ensure the certificates will work as both server and client certificates, if we subsequently want to use the certificates for intra-cluster encryption
• EasyRSA will take the name given on the command line and use it as the filename and the certificate CN field
• To maximise the number of addresses each certificate will match, we'll use Subject Alt Names to add all the addresses we expect to have on each instance, including any:
  • Fully Qualified (Long) DNS Names
  • Short DNS Names
  • IP Addresses

The generic command therefore is:

easyrsa --subject-alt-name='SAN-LIST' build-serverClient-full SHORT-NAME nopass

Replacing SAN-LIST and SHORT-NAME as appropriate for each server.

For a 3-instance Neo4j cluster with DNS names and IP addresses in the certificate, we might do the following:

easyrsa --subject-alt-name='DNS:server1.neo4j.myorg.test,DNS:server1,IP:192.168.100.1' build-serverClient-full server1 nopass
easyrsa --subject-alt-name='DNS:server2.neo4j.myorg.test,DNS:server2,IP:192.168.100.2' build-serverClient-full server2 nopass
easyrsa --subject-alt-name='DNS:server3.neo4j.myorg.test,DNS:server3,IP:192.168.100.3' build-serverClient-full server3 nopass

And this is built into a well-structured PKI hierarchy

/srv/dev-ca
├── ca.crt
├── certs_by_serial
│   ├── 232411F00A0280A2E8908E6A7FF2AF00.pem
│   ├── 4AD7D23C65AA4B801D89AD95B0DAE9F2.pem
│   └── 8084F87D8D5DF8B9D4CE63354BFFC1FC.pem
├── index.txt
├── index.txt.attr
├── index.txt.attr.old
├── index.txt.old
├── issued
│   ├── server1.crt
│   ├── server2.crt
│   └── server3.crt
├── openssl-easyrsa.cnf
├── private
│   ├── ca.key
│   ├── server1.key
│   ├── server2.key
│   └── server3.key
├── renewed
│   ├── certs_by_serial
│   ├── private_by_serial
│   └── reqs_by_serial
├── reqs
│   ├── server1.req
│   ├── server2.req
│   └── server3.req
├── revoked
│   ├── certs_by_serial
│   ├── private_by_serial
│   └── reqs_by_serial
├── safessl-easyrsa.cnf
├── serial
└── serial.old

Most of these directories are internal to EasyRSA as they are essential to running a CA. The directories we are primarily concerned with, are "private" and "issued", as these are where the generated keys and certificates are located respectively.

Preparing the Keys and Certificates

EasyRSA generates keys in the newer PKCS#8 format, but we'll want to use PKCS#1 format.

It may be suitable to build another "prepared" directory here, and output all the certificates and keys there during the clean up and conversion process.

From within the PKI directory, simply create the output directory:

mkdir prepared

Then create a sub-directory, to help structure the files logically:

mkdir prepared/server1
mkdir prepared/server2
mkdir prepared/server3

We can then use the OpenSSL x509 command to copy through the certificates, including the CA certificate:

openssl x509 -in ca.crt -out prepared/ca.crt

openssl x509 -in issued/server1.crt -out prepared/server1/public.crt
openssl x509 -in issued/server2.crt -out prepared/server2/public.crt
openssl x509 -in issued/server3.crt -out prepared/server3/public.crt

Depending on how certificates need to be copied to servers, it may be preferable to copy the CA certificate once to the top location, or once, per server directory if these directories will be later zipped/compressed.

The OpenSSL RSA command can be used to convert the keys:

openssl rsa -in private/server1.key -out prepared/server1/private.key
openssl rsa -in private/server2.key -out prepared/server2/private.key
openssl rsa -in private/server3.key -out prepared/server3/private.key

At this point we have the following structure:

/srv/dev-ca/prepared
├── ca.crt
├── server1
│   ├── private.key
│   └── public.crt
├── server2
│   ├── private.key
│   └── public.crt
└── server3
    ├── private.key
    └── public.crt

Any of these certificates can be checked, using the -noout -text flags for the X509 command, e.g:

openssl x509 -in prepared/server1/public.crt -noout -text

Details there should match the certificate we expect:

• The issuer should be the name given to the CA:

  Issuer: CN=Neo4j Dev Cluster CA

• The subject should be the base name specified on the command line, in our case, the Short Name:

  Subject: CN=server1

• Both Web Server and Web Client usages should be defined:

  X509v3 Extended Key Usage:
      TLS Web Server Authentication, TLS Web Client Authentication

• The Subject Alternative Names should match all the additional names provided:

  X509v3 Subject Alternative Name:
      DNS:server1.neo4j.myorg.test, DNS:server1, IP Address:192.168.100.1

Structuring the SSL Certificate Directory

With the above mentioned files prepared, we can copy them to the right place within the Neo4j home directory on each Neo4j instance. There may only be a `trusted` and `revoked` structure for certificate-based client authentication:

/var
└── lib
    └── neo4j
        └── certificates
              ├── backup
              │   ├── private.key
              │   ├── public.crt
              │   ├── revoked
              │   └── trusted
              │       └── ca.crt
              ├── bolt
              │   ├── private.key
              │   ├── public.crt
              │   ├── revoked
              │   └── trusted
              │       └── ca.crt
              ├── cluster
              │   ├── private.key
              │   ├── public.crt
              │   ├── revoked
              │   └── trusted
              │       └── ca.crt
              └── https
                  ├── private.key
                  └── public.crt

In cases where there are multiple SSL policies, these can point to a "common" set of certificates, if preferred and there may be a single $NEO4J_HOME/certificates/common/ directory structure:

/var
└── lib
    └── neo4j
        └── certificates
                └── common
                    ├── private.key
                    ├── public.crt
                    ├── revoked
                    └── trusted
                        └── ca.crt

If so, in your configuration, set the base_directory to the same set of certificates.

For example:

dbms.ssl.policy.bolt.base_directory=/var/lib/neo4j/certificates/common
dbms.ssl.policy.https.base_directory=/var/lib/neo4j/certificates/common
dbms.ssl.policy.cluster.base_directory=/var/lib/neo4j/certificates/common

Intra-Cluster Encryption

If configuring intra-cluster encryption as part of a causal clustering configuration we need ensure that the certificates used on the cluster endpoints, supports both Server and Client usage, such that when connecting between the Neo4j instances for clustering, each instance will use its own certificate to authenticate as a client on the connection to another server.

This can be verified from the certificate details:

openssl x509 -in public.crt -noout -text

We should see that the X509v3 Extended Key Usage section shows both the usages listed:

X509v3 Extended Key Usage:
    TLS Web Server Authentication, TLS Web Client Authentication

By following these steps, you can effectively set up and manage a PKI for your Neo4j cluster using EasyRSA.