This post provides an overview of RSA encryption implementation. Please, read my next post for detailed guidelines and code examples.

The purpose of this project was to protect communication between C++ (VCL) client application and PHP server script with encrypting and digital signing HTTP requests and responses.

Well, the simplest solution for the project task is a usage of SSL (HTTPS). However, this project is targeting shared hosting users that cannot afford HTTPS or SSL certificates.
That’s why we considered a possible usage of GnuPG but abandoned it as hard to implement. Instead, we decided to base our solution directly on the RSA algorithm PGP is based on.

At the beginning of the project I had no idea about cryptography and RSA in particular. I’m still not very familiar with it.
Thanks to the Wikipedia you can read all you need to know about RSA in a single place: http://en.wikipedia.org/wiki/RSA.

What I knew at the beginning is that cryptographic libraries provide tools for RSA encryption, decryption, signing and verification. Thus, I consider this project as a good work. However, it turns into completely nightmare.
Why? Because of incompatibility.
Thanks to standardization of SSL (TLS) all cryptographic libraries are compatible at the top level and can communicate without a problem. However, their low-level functions and key formats are just not compatible.

The conclusion is not very weird:

• If you can use SSL at both sides then better to use it.
• If you can use the same cryptographic library at the client and the server then use it.
• If you’re going to use low-level functions of different libraries then prepare to have a very hard work dealing with incompatibility.

In this post I will cover main issues and conclusions made during the development. I’ll give more details and code examples in the second part of the post.

Read my next post for details and code examples on implementing RSA encryption/decryption and digital signing.

Libraries

Our client is VCL application made with Embarcadero C++ Builder 2009 and the server is PHP5 script. We already have some communication protocol for them. We just want  to encrypt all communication messages.

For PHP there is a very good OpenSSL extension that provide access to RSA functions and it was selected. There are also some native PHP implementations but they are extremely slow.

There is an open-source LockBox VCL component library implementing RSA algorithm. This is a very old and unsupported library but there is a RAD Studio 2009 port for it. The library is very easy to use but… not compatible. Don’t ask me why but what was encrypted with LockBox can’t be decrypted with OpenSSL and vice versa.

I failed to compile OpenSSL with C++ Builder and decided to use Microsoft CryptoAPI at the client side. That’s were incompatibility starts.

Incompatibility

Most information on the problems and solutions I got from this good article (in Russian). But it’s useless for those who don’t know Russian and has some leaks I needed to cover on my own.

In brief, there are following incompatibilities between CryptoAPI and OpenSSL:

1. Different byte order. For all the long integers (modulus, exponent, encrypted data, signature) CryptoAPI uses little-endian byte order while OpenSSL uses big-endian.
2. Different key formats. CryptoAPI uses PRIVATEKEYBLOB and PUBLICKEYBLOB structures to import/export RSA keys. OpenSSL extension supports “PEM” format only.

Solutions

The first (byte order) problem is very easy to solve. You just need a function that will reverse byte order. Below is C++ example for VCL streams:

void __fastcall ReverseStream(TMemoryStream* Stream)
{
	TMemoryStream* buf = new TMemoryStream;
	buf->LoadFromStream(Stream);
	__int64 size = Stream->Size;
	Stream->Clear();
	for (__int64 i=size-1; i >= 0; i--) {
		buf->Position = i;
		Stream->CopyFrom(buf, 1);
	}
}

With PHP you can do the same simply using strrev() function.
The key format problem is not so easy.

You can use OpenSSL 1.0.x beta to convert keys to/from PEM and CryptoAPI blob formats. There is a very good article on this.

However, OpenSSL 1.0.x beta do not works on Windows now. You need Linux or Cygwin to generate and convert the keys. It wasn’t an option for us because we need to exchange public keys on each session initialization and we don’t want the user to install Cygwin just for key conversion. There was no solution available for the key conversion and we came up with our own.
In fact, OpenSSL executable can export keys in “PEM” and “DER” formats where PEM is a base64 encoded version of DER format. DER is a binary file format well explained in the standard: http://www.itu.int/ITU-T/studygroups/com17/languages/X.690-0207.pdf. You can view the structure of DER files using ASN.1 Editor.

PUBLICKEYBLOB format is also well explained in MSDN: http://msdn.microsoft.com/en-us/library/aa387459(VS.85).aspx.
We decided to export and exchange public keys in DER format and convert them to PEM and PUBLICKEYBLOB at server and client side respectively using our own conversion code. That resolved the issue. Read my post with code examples for reading, writing and converting RSA keys in PEM, DER, PRIVATEKEYBLOB and PUBLICKEYBLOB formats for more information.
Continued…
Read my next post for details and code examples on implementing RSA encryption/decryption and digital signing.