Useful OpenSSL Testing and Debugging Tips

When debugging SSL issues with any service, including Neo4j Server, the industry standard OpenSSL suite provides a helpful set of tools that help diagnose and solve the vast majority of SSL issues.

The OpenSSL suite is written for a Unix-like environment such as Linux or MacOS, but would also work with WSL in a modern Windows environment.

Viewing and Testing Certificates

Checking the contents of a certificate file

To check the contents of a certificate in a file, we can use the x509 subcommand within openssl:

openssl x509 -in INPUT_FILE -noout -text

The main things to check here are the Subject CN field, SAN address, and the certificate valid from/expiration dates.

Other displayed fields, including the certificate content itself, can be useful in various ways.

 

Checking the certificate being presented

To check the certificate provided on a Neo4j HTTPS connector, we can use the s_client subcommand:

openssl s_client -showcerts -connect YOUR_SERVER_ADDRESS:7473 <<< ''

The <<< '' part is used to cleanly close the connection by sending the end of file marker to the connection.
Other TLS-secured endpoints such as the Bolt connector port on port 7676, could be checked in the output for this command in the "Verify return code" status.

Ideally we want to see the following at the bottom:

Verify return code: 0 (ok)

Instead you might see:

Verify return code: 18 (self signed certificate)

If using Neo4j Server 3.5, this could indicate that the default self-signed certificate is being served. This was removed in version 4.x

You could also see:

Verify return code: 20 (unable to get local issuer certificate)

Which could indicate that the certificate chain presented by Neo4j Server does not include a known CA chain.

In this case you may want to check the certificate chain the command displays, to see if it includes the server certificate and all the expected CA certificates.

Specifying a custom CA certificate

If your certificate chain comes from a private CA, such as your corporate PKI / CA system, or a test PKI system you have created, then you can specify a CA certificate file, usually the root certificate, to verify against, by including the -CAfile parameter:

openssl s_client -showcerts -CAfile YOUR_CA_CERT_FILE \
  -connect YOUR_SERVER_ADDRESS:7473 <<< ''

Alternatively, you could include the certificates within the Neo4j installation's trusted folder for the relevant SSL policy, or any other folder of custom CA certificates, by using the -CApath parameter:

openssl s_client -showcerts -CApath /PATH/TO/NEO4J_HOME/certificates/https/trusted/ \
  -connect YOUR_SERVER_ADDRESS:7473 <<< ''

Subsequently, update /certificates/https/trusted/ for your particular policy name and path.

 

Converting Certificate Formats

Converting a PEM certificate to DER format

In Neo4j 3.5, you might need to convert your CA certificates from PEM format to DER format before placing them in the trusted directory. This can be done as:

openssl x509 -in INPUT_FILE -outform DER -out OUTPUT_FILE

Converting a DER certificate to PEM format

Likewise, if the organisation's CA passes provides a server certificate in DER format, then this would need to be converted back to PEM format, which can be done easily as:

openssl x509 -inform DER -in INPUT_FILE -out OUTPUT_FILE