Skip to content

PKCS#11 Fuzzer Reference

Cryptosense Analyzer works by testing traces of exchanges between an application and a cryptographic library. In order to test a device, the Cryptosense PKCS#11 Fuzzer must be used to send commands to the device's PKCS#11 interface and log the responses. This reference manual documents all features of the Cryptosense Fuzzer tool.

Installation

Before you can use the PKCS#11 Fuzzer, see installation instructions.

Usage

Prerequisites

A standard test session uses about 1 GB of free memory. The longer the session lasts, the more memory will be required. Test results are stored in Cryptosense Test (CST) files. File size increases with the level set for each command in the test.

Before you get started, make sure the device you want to test is connected. For hardware PKCS#11 devices you need to know the location of the PKCS#11 DLL, as well as the user PIN.

Make sure that you have backups for all key materials before running Cryptosense Fuzzer on a device. Cryptosense Fuzzer should not delete preexisting keys, but it tends to reveal firmware and driver bugs and those bugs may in turn require you to reset the device.

Overview

Cryptosense PKCS#11 Fuzzer is a command-line tool, so it is supposed to be executed from a terminal, but it can also easily be called from scripts. In this manual we assume the path to the cs-fuzzer executable to be in the PATH environment variable.

CST Files

CST files contain the tests that were performed on a device and also the results of those tests. You need to upload a CST to the Cryptosense Analyzer platform in order to obtain compliance and vulnerability results. A CST is generated by cs-fuzzer. The file is a sequence of JSON objects.

Command-line Options

To run cs-fuzzer on a device:

cs-fuzzer --dll DLL --pin PIN --output myfile.cst [OPTIONS]

You can get more information about the options by running:

cs-fuzzer --help

The parameters and options are:

  • --dll (required) specifies the location to the PKCS#11 driver DLL for the device.
  • -p PIN, --pin PIN (required) is the PIN used to open a session.
  • --output OUTPUT (required) write CST file into file named OUTPUT
  • --commands COMMANDS, --only-commands COMMANDS: Only test the following commands. This option takes a comma separated list of commands and may appear multiple times. It only sets commands that use cryptographic objects, not the ones that create them. It can't be used with --except-commands. Example usage:
    $ cs-fuzzer --commands C_Sign,C_Encrypt --pin PIN \
--dll DLL --output myfile.cst
  • --direct Do not use C_GetFunctionList -- used when the DLL is buggy and gives the wrong output from this call, causing a later crash.
  • --duration=DURATION Maximum duration to run the fuzzer for (in seconds)
  • --except-commands COMMANDS : Test all but the following commands. This option takes a comma-separated list of commands and may appear multiple times. Can't be used with --commands.
  • --except-generation-commands=EXCEPT_GENERATION_COMMANDS Use all but the following EXCEPT_GENERATION_COMMANDS. This option takes a comma separated list of generation command name and may appear many times. Can't be used with --only-generation-commands.
  • --expanded Display an expanded view of PKCS#11 return values.
  • --filter-config Only test mechanisms the token reports as supporting.
  • --help[=FMT] (default= auto) Show the help in format FMT. The value FMT must be either auto, pager, groff or plain. With auto, the format is pager or plain whenever the TERM env var is dumb or undefined.
  • --indirect_or_direct Try to use C_GetFunctionList, and if it fails, try again without using it. This is the default behaviour.
  • --level LEVEL: The level of tests to be completed, the higher the level is, the more tests are executed. This overrides the default which for most commands is 2. Default levels are specified in a profile.
  • --log-calls Causes all API calls to be written to a log file. The log filename is api-calls.log.
  • --nss "configDir=sql:/path/to/NSS/database" When using the NSS library, use this NSS configuration
  • --only-generation-commands COMMANDS: Only use the following functions to create cryptographic objects.
  • -q, --quiet Print less information. Can be given several times. Opposite of -v.
  • --resume-from Resume testing from an existing trace file.
  • --unsafe Avoid running the fuzzer in an isolated process. This improves performance, but at the cost of crashing the application if an error is encountered.
  • -s INDEX, --slot-index INDEX, --slot INDEX Test the token in slot INDEX. The options --slot-index, --slot-id, -- -slot-description and --token-label are mutually exclusive.
  • --slot-description DESCRIPTION Test the slot with the description DESCRIPTION. DESCRIPTION must not exceed 64 characters. The options --slot-index, --slot-id, --slot-descriptio n and --token-label are mutually exclusive.
  • --slot-id ID Test the token in slot ID. The options --slot-index, --slot-id, --slot-description and --token-label are mutually exclusive.
  • --stop-on-key-regeneration-failure Stop testing if an error is hit while regenerating a key on the HSM using a known valid template
  • --token-label LABEL Test the token with label LABEL. LABEL must not exceed 32 characters. The options --slot-index, --slot-id, --slot-description and --token-label are mutually exclusive.
  • --user-type USER TYPE Define the USER TYPE to use for the fuzzing session. Care must be taken to supply the correct PIN for the given user type.
  • -v, --verbose Print more information. Can be given several times. Opposite of -q.
  • --version Show version information.

Note that long form options can also be given with the equals sign (=) instead of a space, i.e. --output=file.cst is equivalent to --output file.cst.

Troubleshooting

Device or Driver Crashes

An occasional issue with cs-fuzzer, particularly with a new PKCS#11 DLL, is that the fuzzer causes a segmentation fault in the PKCS#11 driver DLL. Detecting these bugs in the driver is a feature of the fuzzer.

To find out which call caused the segmentation fault, run cs-fuzzer as follows:

cs-fuzzer --dll DLL -p PIN --output myfile.cst --log-calls

This creates a file called api-calls.log. At the end of this file, the inputs of the command which caused the error can be found.

Example Here is an extract of such a log file for a PKCS#11 device:

cs-test: C_Initialize ((nil)) // -> 0
cs-test: C_GetSlotList (0, (nil), 0x1ed0010) // -> 0
cs-test: C_GetSlotList (0, 0x1ecfec0, 0x1ed0010) // -> 0
cs-test: C_GetSlotInfo (0, 0x1ecfee0) // -> 0
cs-test: C_OpenSession (0, 6, (nil), (nil), 0x1ecff60) // -> 0
cs-test: C_Login (1, 1, 0x1ecdce0, 4) // -> 0

Each call to a function of the PKCS#11 is written to the file. You can see which Analyzer command you logged the calls of and the function that is being logged. The next part i.e. (0,0x1ecfee0) are the parameters used. The final part // -> 0 is the return code that the function gives. Here it is 0 which means there was no error.

Here is an example of a failed call:

cs-test: C_Login (1, 1, 0x1ecdce0, 4) // -> 1

When the PKCS#11 driver DLL crashes, the last line of the log usually looks like:

cs-test: C_Login (1, 1, 0x1ecdce0, 4)

Note that the // -> 0 part is missing, which means that the end of the call has never been reached.

For further debugging information, the fuzzer can be run under a debugger such as gdb in order to get details of the stacktrace that triggered the error.

If a CST file was the result of a complete fuzzing test run, it will contain as a final comment line "Fuzzing end". If the DLL crashed during the run, no such entry will be present. This allows the easy identification of CSTs corresponding to crashed runs.

A trace file that terminates correctly at the end of the fuzzing process will have the string "Fuzzing End" in the last object in the trace.

How Fuzzing Tests are Generated

Cryptosense Fuzzer is a mutation-based fuzzing engine, i.e. it starts with a set of standard calls and adds and/or removes parameters in order to generate queries that are edge-cases in the standard. The specific parameters that are added and removed are tailored to the compliance and vulnerability tests that will be executed on the CST file by the Analyzer.

The number of tests executed in a fuzzing test run for a given --level parameter will vary considerably depending on the mechanisms and commands supported by the device under test. For example, if a call to C_Encrypt succeeds for a certain mechanism, this will result in the creation of a ciphertext. This ciphertext will then be used as an input for future operations by the fuzzer. If the call fails and no ciphertext is created, fewer tests will be executed.

Runs using --level 3 can already take several days to run on typical devices. Generally, higher levels are only useful when used in combination with other restrictions such as --only-commands and --only-generation-commands.

Resuming from a Partial Trace

A run of the fuzzer is comprised of two parts:

  • testing generation commands (C_CreateObject, C_GenerateKey, C_GenerateKeyPair)
  • testing cryptographic commands (C_Encrypt, etc).

Cryptographic commands require key objects to be created, and the fuzzer uses information gathered during the first step to create these objects. However, this can be time-consuming since that step needs to be repeated for every run. Instead, it is possible to create a trace that only contains information about generation commands, and start fuzzing from information in that trace.

To do that, first generate a trace containing only generation commands. This can be done by using --only-commands C_Digest. For example:

cs-fuzzer --dll DLL -p PIN --output generation.cst --only-commands C_Digest

For subsequent traces, one can pass --resume-from with that first trace to bypass testing of generation commands:

cs-fuzzer --dll DLL -p PIN --output commands.cst --resume-from generation.cst

However, there are some caveats with that approach:

  • Producing a trace using one set of parameters, and then resuming from it using different parameters (e.g. a different --level ) may not create a meaningful trace.
  • Traces produced using --resume only contain fuzzing information for the cryptographic commands, not the generation commands. As a result, analysis findings that depend on generation commands may not be detected in the resumed trace. To get full results, you should analyze the original generation trace as well.

Because of these limitations, it is generally preferable to work with a single trace. The --resume option is typically helpful if the device under test is slow and/or crashes often.