Reading, writing and converting RSA keys in PEM, DER, PUBLICKEYBLOB and PRIVATEKEYBLOB formats

This post finishes my epic about the implementation of RSA encryption. See the part I and part II of my post about RSA encryption for C++/Delphi (CryptoAPI) and PHP (OpenSSL) applications.

The main problem we faced was incompatibility of key formats. CryptoAPI uses PRIVATEKEYBLOB and PUBLICKEYBLOB formats to export and import RSA keys while OpenSSL extension for PHP uses PEM format. In order to use both libraries in communicating applications we needed some tool to convert keys from one format to another. The only tool we found for this was OpenSSL 1.0.x beta. Notice that earlier versions of OpenSSL do not support CryptoAPI BLOBs.

Update: It was found later that CryptoAPI has native functions for key conversion. See “Update” section at the bottom of the post.

Below is a command line syntax example for conversion of private key from PEM to PRIVATEKEYBLOB format:

openssl rsa -inform PEM -in private.pem -outform MS\ PRIVATEKEYBLOB -out private.blob

And this example converts PUBLICKEYBLOB to PEM format:

openssl rsa -pubin -inform MS\ PUBLICKEYBLOB -in public.blob -outform PEM -out public.pem

Notice that backslash (\) in format names. You need to type it as it actually escapes the space character.

However, we found some drawbacks in usage of OpenSSL 1.0.x beta:

  • There was no Windows build of it available at the time of the post but we wanted to convert keys on Windows.
  • We also wanted to convert keys directly in our code w/o any need for external application.

As far as PRIVATEKEYBLOBPUBLICKEYBLOB and PEM format structures are known, we decided to develop code that will read and write them using low-level functions. It actually took 1-2 days for me to develop that code so I don’t think it’s a really hard task.

Later we faced another problem: PHP versions prior to 5.2 don’t support openssl_pkey_get_details function. Once again, handling key formats directly helped us to resolve the issue by providing a replacement for the function.

So, let me explain how you can implement reading/writing PEM, DER, PRIVATEKEYBLOB and PUBLICKEYBLOB formats with some code examples in PHP for PEM and DER formats and in C++/VCL for CryptoAPI BLOBs. As the task was a part of a commercial project I cannot post a complete working example here. But I will do my best helping you to assemble such code on your own. You can also request our service at Pumka.net.

Understanding RSA Key Formats

The first thing you need to know is that any key format is actually a container for the set of long numbers. All other data could be considered “noise”.

  • Private key contains: modulus, private exponent, public exponent, prime 1, prime 2, exponent 1, exponent 2 and coefficient.
  • Public key contains only modulus and public exponent.

Once you can read these numbers from one format and put them to another, you can covert keys. You can also export public keys for private ones this way.

PEM format produced by OpenSSL is actually base64 encoded and wrapped key data in the binary format called DER. Thus, to work with PEM format you must actually work with DER.

DER format is based on Abstract Syntax Notation One (ASN.1) standard. The standard specifies encoding of tree-like data structures. Two predefined data structures are used for private and public RSA keys. Though I didn’t find a good parser for the format, with a couple of notes from the standard in hand I wrote a class for encoding and decoding ASN.1 values.

PRIVATEKEYBLOB and PUBLICKEYBLOB formats are actually C-style PRIVATEKEYBLOB and PUBLICKEYBLOB structures. You can use pointer manipulation and memory copy to parse them. Notice that these formats use little-endian byte order for integer values unlike DER format which uses big-endian order. Thus, you’ll need to reverse integer byte sequences when converting key data from BLOB to DER or vise versa.

Getting DER Data out of PEM

As I wrote, PEM format contains base64 encoded DER data. It also adds a header and footer to it. Below is an example of PEM file for a private key.

-----BEGIN RSA PRIVATE KEY-----
MIIBOwIBAAJBAKIQa3Oic8/OOxMpVNUPsKrsSCvbOlKVQXKVRPX85VNkcYqx+Cr6
Kij/bfnLFQ6E2Of7hoZZsYjFLNsKcH14R4MCAwEAAQJBAJIefdMfiI23YroPDOah
I+en7BZmrfohioXWIfwsDVQWj22gDtB7whni1TqVQYDC3tqq8OtreIBuH1IS2784
dmkCIQDSeOC/a5uZaJQgA54N+wGqOr3q8ZqgMHS1wFc527QFrQIhAMUe5RNLps1Y
x1ReFU2DWT7zjqKM5gr0ffUAAusQpEfvAiAF1DEtO/awNfQ8Or1q17PBGiVeV1iX
7R+eVPhVct82dQIgcIbmdlFlcywO/haHSqyEse8PqbONTwurK8VJ5S6m2XkCIQCg
TC59mfa6vHyTPBdUliMnD2l9mUJvVkcpwr7yyxHT6A==
-----END RSA PRIVATE KEY-----

Thus, you can get DER data by removing header, footer and base64 decoding:

function PemToDer($Pem) {
    //Split lines:
    $lines = explode("\n", trim($Pem));
    //Remove last and first line:
    unset($lines[count($lines)-1]);
    unset($lines[0]);
    //Join remaining lines:
    $result = implode('', $lines);
    //Decode:
    $result = base64_decode($result);

    return $result;
}

Great! Now you can touch naked DER data. 😉

Below is the function that does the opposite: encodes pure DER data into PEM format. Notice that lines with base64 encoded data must be 65 character wide.

function DerToPem($Der, $Private=false)
{
    //Encode:
    $Der = base64_encode($Der);
    //Split lines:
    $lines = str_split($Der, 65);
    $body = implode("\n", $lines);
    //Get title:
    $title = $Private? 'RSA PRIVATE KEY' : 'PUBLIC KEY';
    //Add wrapping:
    $result = "-----BEGIN {$title}-----\n";
    $result .= $body . "\n";
    $result .= "-----END {$title}-----\n";

    return $result;
}

Parsing DER Format

You can view the structure and contents of DER files with ASN.1 Editor. I highly recommend this tool for understanding DER format and debugging your code.

Let’s open some private key file with ASN.1 Editor:

private

I added green markers pointing to particular values.

Below is the corresponding public key:

public

Despite of less real data, the structure of public key file looks more complicated. But the header sequence is always the same for any RSA public key so you can simply replicate it when writing the file.

You can download my PHP class for decoding and encoding ASN.1 values here: ASNValue.class.php.
Use Decode method to parse DER record into Tag and Value properties. Alternatively, set Tag and Value properties (you can do this with the constructor) and call Encode method to compose DER record.

Notice that many integer values in DER begin with a zero byte. That’s because ASN.1 standard requires adding zero byte to the beginning of multi-byte integer if the first bit of it is set to 1. I added methods for proper encoding and decoding of integers as well as for parsing and composing sequences.

Wow! Now we can read private key values from DER data:

//Decode root sequence
$body = new ASNValue;
$body->Decode($PrivateDER);
$bodyItems = $body->GetSequence();

//Read key values:
$Modulus = $bodyItems[1]->GetIntBuffer();
$PublicExponent = $bodyItems[2]->GetInt();
$PrivateExponent = $bodyItems[3]->GetIntBuffer();
$Prime1 = $bodyItems[4]->GetIntBuffer();
$Prime2 = $bodyItems[5]->GetIntBuffer();
$Exponent1 = $bodyItems[6]->GetIntBuffer();
$Exponent2 = $bodyItems[7]->GetIntBuffer();
$Coefficient = $bodyItems[8]->GetIntBuffer();

And you can export public key DER data:

//Encode key sequence
$modulus = new ASNValue(ASNValue::TAG_INTEGER);
$modulus->SetIntBuffer($Modulus);
$publicExponent = new ASNValue(ASNValue::TAG_INTEGER);
$publicExponent->SetInt($PublicExponent);
$keySequenceItems = array($modulus, $publicExponent);
$keySequence = new ASNValue(ASNValue::TAG_SEQUENCE);
$keySequence->SetSequence($keySequenceItems);
//Encode bit string
$bitStringValue = $keySequence->Encode();
$bitStringValue = chr(0x00) . $bitStringValue; //Add unused bits byte
$bitString = new ASNValue(ASNValue::TAG_BITSTRING);
$bitString->Value = $bitStringValue;
//Encode body
$bodyValue = "\x30\x0d\x06\x09\x2a\x86\x48\x86\xf7\x0d\x01\x01\x01\x05\x00" . $bitString->Encode();
$body = new ASNValue(ASNValue::TAG_SEQUENCE);
$body->Value = $bodyValue;
//Get DER encoded public key:
$PublicDER = $body->Encode();

Well, you can also use my class to read public key data and write private keys. I leave this for your homework.

Parsing CryptoAPI BLOBs

Since PRIVATEKEYBLOB and PUBLICKEYBLOB formats correspond to C structures, they could be parsed with pointer manipulation and memory copy tricks. Our client application built with C++ Builder and we used VCL TMemoryStream class for memory manipulation.

Below code example demonstrates parsing PRIVATEKEYBLOB format:

//Set structure pointers:
BLOBHEADER* blobheader = (BLOBHEADER*)PrivateBLOB->Memory;
RSAPUBKEY* rsapubkey = (RSAPUBKEY*)((System::PByte) PrivateBLOB->Memory + sizeof(BLOBHEADER));
//Get key length:
unsigned int byteLength = rsapubkey->bitlen/8;
unsigned int halfLength = byteLen/2;
//Set value pointers:
System::PByte modulus = ((System::PByte)PrivateBLOB->Memory + sizeof(BLOBHEADER) + sizeof(RSAPUBKEY));
System::PByte prime1 = modulus + byteLen;
System::PByte prime2 = prime1 + halfLen;
System::PByte exponent1 = prime2 + halfLen;
System::PByte exponent2 = exponent1 + halfLen;
System::PByte coefficient = exponent2 + halfLen;
System::PByte privateExponent = coefficient + halfLen;
//Create streams
TMemoryStream* Modulus= new TMemoryStream;
TMemoryStream* Prime1 = new TMemoryStream;
TMemoryStream* Prime2 = new TMemoryStream;
TMemoryStream* Exponent1 = new TMemoryStream;
TMemoryStream* Exponent2 = new TMemoryStream;
TMemoryStream* Coefficient = new TMemoryStream;
TMemoryStream* PrivateExponent = new TMemoryStream;
//Read values:
unsigned long PublicExponent = rsapubkey->pubexp;
Modulus->WriteBuffer(modulus, byteLength);
Prime1->WriteBuffer(prime1, halfLength);
Prime2->WriteBuffer(prime2, halfLength);
Exponent1->WriteBuffer(exponent1, halfLength);
Exponent2->WriteBuffer(exponent2, halfLength);
Coefficient->WriteBuffer(coefficient, halfLength);
PrivateExponent->WriteBuffer(privateExponent, byteLength);

In a similar way you can export PUBLICKEYBLOB with known modulus and public exponent values:

//Prepare structures
BLOBHEADER blobheader;
	blobheader.bType = PUBLICKEYBLOB;
	blobheader.bVersion = 2;
	blobheader.reserved = 0;
	blobheader.aiKeyAlg = CALG_RSA_KEYX;
RSAPUBKEY rsapubkey;
	rsapubkey.magic = PUBLICKEYBLOB_MAGIC; //RSA1
	rsapubkey.bitlen = Modulus->Size*8;
	rsapubkey.pubexp = PublicExponent;
//Write structures
PublicBLOB->WriteBuffer((void*)&blobheader, sizeof(BLOBHEADER));
PublicBLOB->WriteBuffer((void*)&rsapubkey, sizeof(RSAPUBKEY));
//Write modulus
PublicBLOB->CopyFrom(Modulus, 0);

Once again, I left decoding PUBLICKEYBLOB and encoding PRIVATEKEYBLOB for your homework.

Conclusion

Low-level access to RSA key formats provides easy solution for key format incompatibility problem that could look as an irresistible hinder between OpenSSL and CryptoAPI.

We implemented this technique for on-fly conversion of key BLOBs exported by CryptoAPI into native OpenSSL format (PEM) and vise versa. This allowed as to establish compatible and reliable encrypted communication between our applications using different cryptographic libraries.

We also applied parsing of PEM format for replacement of openssl_pkey_get_details function, which is not available prior to PHP v5.2.

Download Source Code:

ASNValue.class.php PHP class for encoding and decoding ASN.1 values.

Update: Converting RSA keys with CryptoAPI

Thanks to Jason who recently contacted me about this post it was found that CryptoAPI has native functions for PEM/DER to BLOB conversion.
Just a quick sample I didn’t test myself yet but it works fine for him:

// Convert from PEM format to DER format
// removes header and footer and decodes from base64
if (!CryptStringToBinaryA((char*)pbPublicPEM, iPEMSize, CRYPT_STRING_ANY,
    pbPublicDER, &iDERSize, NULL, NULL)
) {
    // Error handling
}

// Decode from DER format to CERT_PUBLIC_KEY_INFO. This has the public key
// in ASN.1 encoded format called "SubjectPublicKeyInfo" ... szOID_RSA_RSA
if (!CryptDecodeObjectEx(X509_ASN_ENCODING, X509_PUBLIC_KEY_INFO, pbPublicDER,
    iDERSize, CRYPT_ENCODE_ALLOC_FLAG, NULL, &pbPublicKey, &iPBLOBSize)
) {
    // Error handling
}

// Decode the RSA Public key itself to a PUBLICKEYBLOB
if (!CryptDecodeObjectEx(X509_ASN_ENCODING, RSA_CSP_PUBLICKEYBLOB,
    pbPublicKey->PublicKey.pbData, pbPublicKey->PublicKey.cbData,
    CRYPT_ENCODE_ALLOC_FLAG, NULL, &pbPKEY, &iPKEYSize)
) {
    // Error handling
}
WordPress › Error

There has been a critical error on this website.

Learn more about troubleshooting WordPress.