Into the Black Box: A Case Study in Obtaining Visibility into Commercial Software

Decoding the key database was significantly more difficult than was the case for the certificate database. This difficulty was mainly due to the lack of documentation available, and the fact the private key record in the data are encrypted with a password. Unlike the certificate database, the Netscape NSS does not provide any information describing the format of this database or the encryption used.

In trying to decode this database, we first dumped all of the records in the database. We discovered that there are only four different types of records in the key database and only two records contained the common header mentioned in Section 3. Records that use the common header have the record types shown in Figure 7.

The other two records which do not contain the common header are the Version record and the Global Salt¹ record. These records can be easily identified by their access keys, "Version" and "global-salt" respectively. The key database can be identified by the existence of the version record. Additionally, if the key database contains any private key records it will also contain a password check record, which can be accessed using "password-check" for the database access key.

As in the certificate database, records in the key database are in big endian format. The key database record formats shown in Figure 8 were actually easy to determine. However, determining how to use this information to decrypt a private key was a different story. Determining the role of each record in the decryption of a private key was going to be a challenge.

We started this task by first dumping a private key record header and data (ASN.1 encoded) as shown in Figure 9. The software used to decode the ASN.1 encoded information was written by Peter Gutmann and may be download from his Web site at

http://www.cs.auckland.ac.nz/~pgut001/

Decoding the ASN.1 key data revealed the object identifier (OID)² of (06 0B 2A 86 48 86 F7 0D 01 0C 05 01 03) that has description string of

indicating the specific encryption technique used to encrypt the private key. This OID description specifies password-based encryption (PBE) with secure hash version one (SHA1) and the Triple Data Encryption Standard (DES) in cipher block chaining mode (CBC). The OCTET String and the integer contained in the sequence following the OID are the salt and iterator value for the PBE scheme. Finally, the last OCTET STRING is the encrypted private key.

typedef struct

  {

unsigned char GlobalSalt[16];

}GlobalSaltHeader;


typedef struct

{

  // Contains just the common header

} KeyVersionHeader;


#define KEYPASSCHKFIXEDSIZE 18

typedef struct

{ 

  unsigned char Salt[16];

  unsigned short CryptAlgLength;

  unsigned char *AlgInfo;

  unsigned char *EncryptedAccessKey; // "password-check" Encrypted 16 bytes

} PasswordCheckHeader;


#define KEYHEADERFIXEDSIZE 8

typedef struct


{

  unsigned char Salt[8];

  char * NickName;

  unsigned char * KeyInfoDER;

}KeyHeader;

Record:
 
      Size:    436 bytes
 
      Version: 3
 
      Type:    Private Key
 
      Flags:   0x23
 
      Initial Vector: 47 EB A8 CE FC 4B C0 6B
 
      Key Name:Scott A Hissam's VeriSign, Inc. ID
 
      Name Length:34
 
      Encrypted ASN.1 Private Key
 
         0 30 386: SEQUENCE {
 
         4 30 28 :   SEQUENCE {
 
         6 06 11 :      OBJECT IDENTIFIER
 
                 :        pkcs-12-PBEWithSha1AndTripleDESCBC (1 2 840 113549 1
12 5 1 3)

                 :        (PKCS #12 OID PBEID (1 2 840 113549 1 12 5 1). 
Deprecated, use the incompatible but similar (1 2 840 113549 1 12 1 3) or (1 2 840 
113549 1 12 1 4) instead)
 
        19 30  13:       SEQUENCE {
 
        21 04   8:         OCTET STRING
 
                 :           47 EB A8 CE FC 4B C0 6B
 
        31 02   1:         INTEGER 1
 
                 :         }
 
                 :       }
 
        34 04 352:     OCTET STRING
 
                 :       BF 3E 52 71 3E 07 94 73 25 F2 28 8D 06 D6 1E F8
 
                 :       B3 EC FA 59 17 06 EC F9 8F 92 19 FE 4C FF C3 81
 
                 :       F8 BE F0 12 A2 DD 6A D3 17 DA 56 5A B4 65 8B E8
 
                 :       5D 6F 4B AE 6F 5F 39 DC 1F EF BF 56 6E 79 D5 B4
 
                 :       2B 9A 6E 20 98 4D 66 98 79 4C 85 98 31 1D 4B E3
 
                 :       DE EF C3 07 54 76 86 50 A8 22 9E 94 C8 CB F9 F4
 
                 :       46 9E 52 26 F8 20 8C 51 E8 52 6E 95 16 CA 9D 4D
 
                 :       E6 7E 90 69 96 1E 1E DF CC 67 FE AB 96 5A D7 88
 
                 :       26 1A A9 CC 52 F6 97 0F 28 FC 52 96 DE FB FD F7
 
                 :       87 01 AE 71 E0 88 1B C6 7D 01 C8 83 27 40 36 A3
 
                 :       46 23 DD 64 86 64 F7 64 73 46 04 30 3A 96 71 33
 
                 :       7E 98 F1 BE 18 B9 8B 10 DA FF FA 32 AC 03 18 37
 
                 :       DA 87 32 5F EB F7 ED 0D 37 B2 1B 97 35 D6 38 F2
 
                 :       F8 CC 4E 2D 00 E2 43 F1 6F 02 B2 FD 94 53 9D 7B
 
                 :       78 00 4D FB 4D 47 63 6E B9 65 92 4C 03 C2 A6 9F
 
                 :       20 59 80 D5 A0 D4 B2 79 51 6E 31 B6 20 D4 A9 43
 
                 :       80 31 CE C6 93 0C B0 1E 2F 13 3F C3 C0 E0 7B 16
 
                 :       89 76 88 DD 38 D6 8F 2B 5F 6F 50 1D F7 48 D9 2E
 
                 :       89 C2 04 1F 78 6B AC 85 97 55 0F 71 BE 5D D2 C7
 
                 :       C8 22 41 B6 C9 A0 C9 81 CD 93 55 83 D2 9D E3 00
 
                 :       63 72 4F 79 D4 E9 AD 1D 1E CD 79 3F 89 9A 66 E4
 
                 :       F6 A2 1D EC A0 3E 61 35 81 CC B8 83 5C DF 87 24
 
                 :     }
 

We needed to find a document that described the PBEWithSha1AndTripleDESCBC password-based encryption technique. An initial search of the Web did not reveal any additional information about the OID. However, we located documentation that described the password-based encryption technique for a similar OID called PBEWithSha1And3-KeyTripleDESCBC in the RSA laboratories PKCS#12 Personal Information Exchange Standard [RSA 97]. We thought there was a good chance that both object identifiers used the same password-based encryption technique.

We performed a Web search for an encryption package that supported the hashing function SHA1 and Triple DES CBC encryption. This resulted in the discovery of a package called SSLEAY that contains cryptographic libraries and certificate support software. Additionally, we located a software package that enhanced the certificate support software in SSLEAY by adding support for the PKCS12 standard [RSA 97]. This was fantastic because we found all of the software needed to decrypt a Netscape private key record on the Web.

We examined the source code from the downloaded software and incorporated into our browsing tool the portions that were needed to decrypt a private key. We then attempted to decrypt a private key using the code extracted from the implementation of the PKCS12 standard. This attempt ended in failure.

Because of our failed attempt, we decided to take a closer look at the Netscape NSS software. Upon examination, we noticed the function call SECKEY_ChangeKeyDBPasswordAlg. This API call appeared to change the password-based encryption algorithm used to encrypt the database. This was a guess because the NSS documentation only describes the higher level API calls necessary for using SSL and NSPR, it does not include (other than undocumented C header files) any documentation describing the lower level APIs. Examination of the header files yielded two password-based encryption algorithm identifiers that were of particular interest:

1. SEC_OID_PKCS12_PBE_WITH_SHA1_AND_TRIPLE_DES_CBC
2. SEC_OID_PKCS12_V2_PBE_WITH_SHA1_AND_3KEY_TRIPLE_DES_CBC

The first algorithm identifier appeared to be the same as the OID that we were unable to find any information about, while the second algorithm appeared to be the same as the OID that we had obtained documentation as well was an implementation. Possibly, our assumption that both OID’s were compatible was incorrect.

We then proceeded to write a program to change the database encryption algorithm to SEC_OID_PKCS12_V2_PBE_WITH_SHA1_AND_3KEY_TRIPLE_DES_CBC. After much trial and error in trying to figure out the semantics of Netscape's undocumented interface, we were successful using code shown in Figure 10. This exercise turned out to be very informative. We learned that the global salt record was used in combination with the password (exact details were not known at this time) and that, contrary to what we had thought, the two OID’s were not compatible.

Next, we tried to decrypt a private key record in the converted base database. Initially we were unsuccessful, but after some trial and error, with different password formats (unicode or non-unicode), we discovered that we could decrypt a private key. The output from the NSS API call SECKEY_HashPassword needed to be the input password to the PBE PKCS12 decryption software that we obtained from the Web. After further trial and error (really a wild guess), we determined that the SECKEY_HashPassword actually performs the hashing function shown in Figure 11. This was determined by first noticing that all password were always 20 bytes long, indicating that the user input password and salt were most likely being used as input to SHA-1 (since SHA1 is a hashing function that always returns a twenty-byte digest).

On our first attempt, we used the SHA1_Update call in the hash function shown in Figure 11 to concatenate salt onto the password; this failed, however. Next we changed the order (salt then password); this worked.³

Almost incidentally, we also determined that key databases do not always contain a "Global Salt" record, which is reason for the HaveGlobalSalt flag in the password hashing function, which explains the "if" statement in the hash function. The hashed password, however, is always 20 bytes in length.

#include <stdio.h>
 
#include <string.h>
 
#include <secitem.h>
 
#include <key.h>
 
int main (int argc, char **argv) 
 
{
 
 SECKEYKeyDBHandle *Handle;
 
 SECItem           *st;
 
 char              passwd[512]; 
 
 if (argc!=2) {
 
  printf("usage: changedb <database file>\n");
 
  return -1;
 
}
 
if ((Handle=SECKEY_OpenKeyDBFilename(argv[1],0))==NULL) {
 
 printf("database open error\n");
 
 return -1;
 
 }
 
printf("Enter Password:");
 
fgets(passwd,sizeof(passwd),stdin);
 
if (strlen(passwd)) passwd[strlen(passwd)-1]='\0';
 
st=SECKEY_HashPassword(passwd,Handle->global_salt );
 
if (SECKEY_CheckKeyDBPassword(Handle,st)!=SECSuccess) {
 
  printf("Incorrect Password\n");
 
  SECKEY_CloseKeyDB(Handle);
 
  return -1; 
 
 }
 
 // Original Database format was SEC_OID_PKCS12_PBE_WITH_SHA1_AND_TRIPLE_DES_CBC
 
if (SECKEY_ChangeKeyDBPasswordAlg(Handle,st, st,
 
     SEC_OID_PKCS12_V2_PBE_WITH_SHA1_AND_3KEY_TRIPLE_DES_CBC)==SECSuccess)
 
     printf("Database Format Change Success\n");
 
else printf("Database Format Change Falied\n");
 

SECKEY_CloseKeyDB(Handle);
 
return 0;
 
}
 

[abstract] [chapter 1] [chapter 2] [chapter 3] [chapter 4] [chapter 5] [references] [DTIC] [PDF]