BENCHMARKING WOLFSSL AND WOLFCRYPT

Many users are curious about how the wolfSSL embedded SSL/TLS library will perform on a specific hardware device or in a specific environment.  Because of the wide variety of different platforms and compilers used today in embedded, enterprise, and cloud-based environments, it is hard to give generic performance calculations.

To help wolfSSL users and customers in determining performance for wolfSSL and wolfCrypt, a benchmark application is bundled with wolfSSL.  Because the underlying cryptography is a very performance-critical aspect of SSL/TLS, our benchmark application runs performance tests on wolfCrypt’s algorithms.

The benchmark utility is located in the “./wolfcrypt/benchmark” directory of the wolfSSL package.  After building wolfSSL and the associated examples and apps, the benchmark application can be run by issuing the following command from the package directory root:

./wolfcrypt/benchmark/benchmark

Typical output will look similar to the output below (showing throughput in MB/s as well as cycles per byte):

wolfCrypt Benchmark (block bytes 1048576, min 1.0 sec each)
RNG                100 MB took 1.047 seconds,   95.466 MB/s Cycles per byte =  22.92
AES-128-CBC-enc    200 MB took 1.020 seconds,  196.027 MB/s Cycles per byte =  11.16
AES-128-CBC-dec    215 MB took 1.008 seconds,  213.318 MB/s Cycles per byte =  10.26
AES-192-CBC-enc    175 MB took 1.016 seconds,  172.265 MB/s Cycles per byte =  12.70
AES-192-CBC-dec    180 MB took 1.009 seconds,  178.405 MB/s Cycles per byte =  12.27
AES-256-CBC-enc    150 MB took 1.007 seconds,  148.932 MB/s Cycles per byte =  14.69
AES-256-CBC-dec    160 MB took 1.026 seconds,  155.994 MB/s Cycles per byte =  14.03
AES-128-GCM-enc     60 MB took 1.010 seconds,   59.427 MB/s Cycles per byte =  36.82
AES-128-GCM-dec     65 MB took 1.070 seconds,   60.750 MB/s Cycles per byte =  36.02
AES-192-GCM-enc     60 MB took 1.050 seconds,   57.138 MB/s Cycles per byte =  38.30
AES-192-GCM-dec     60 MB took 1.024 seconds,   58.590 MB/s Cycles per byte =  37.35
AES-256-GCM-enc     55 MB took 1.029 seconds,   53.438 MB/s Cycles per byte =  40.95
AES-256-GCM-dec     60 MB took 1.090 seconds,   55.069 MB/s Cycles per byte =  39.74
CHACHA             360 MB took 1.001 seconds,  359.628 MB/s Cycles per byte =   6.09
CHA-POLY           285 MB took 1.014 seconds,  280.943 MB/s Cycles per byte =   7.79
MD5                450 MB took 1.010 seconds,  445.573 MB/s Cycles per byte =   4.91
POLY1305          1265 MB took 1.000 seconds, 1264.402 MB/s Cycles per byte =   1.73
SHA                475 MB took 1.000 seconds,  474.914 MB/s Cycles per byte =   4.61
SHA-224            210 MB took 1.018 seconds,  206.308 MB/s Cycles per byte =  10.61
SHA-256            210 MB took 1.018 seconds,  206.200 MB/s Cycles per byte =  10.61
SHA-384            280 MB took 1.016 seconds,  275.520 MB/s Cycles per byte =   7.94
SHA-512            275 MB took 1.000 seconds,  274.868 MB/s Cycles per byte =   7.96
SHA3-224           240 MB took 1.006 seconds,  238.506 MB/s Cycles per byte =   9.18
SHA3-256           225 MB took 1.007 seconds,  223.454 MB/s Cycles per byte =   9.79
SHA3-384           175 MB took 1.002 seconds,  174.610 MB/s Cycles per byte =  12.53
SHA3-512           125 MB took 1.031 seconds,  121.254 MB/s Cycles per byte =  18.05
HMAC-MD5           445 MB took 1.001 seconds,  444.651 MB/s Cycles per byte =   4.92
HMAC-SHA           470 MB took 1.009 seconds,  465.749 MB/s Cycles per byte =   4.70
HMAC-SHA224        200 MB took 1.001 seconds,  199.874 MB/s Cycles per byte =  10.95
HMAC-SHA256        205 MB took 1.004 seconds,  204.228 MB/s Cycles per byte =  10.72
HMAC-SHA384        290 MB took 1.009 seconds,  287.401 MB/s Cycles per byte =   7.61
HMAC-SHA512        290 MB took 1.013 seconds,  286.214 MB/s Cycles per byte =   7.65
RSA   2048 public       2800 ops took 1.014 sec, avg 0.362 ms, 2761.995 ops/sec
RSA   2048 private       300 ops took 1.308 sec, avg 4.359 ms, 229.402 ops/sec
DH    2048 key gen       735 ops took 1.001 sec, avg 1.361 ms, 734.608 ops/sec
DH    2048 key agree     800 ops took 1.123 sec, avg 1.404 ms, 712.131 ops/sec
ECC    256 key gen      1108 ops took 1.001 sec, avg 0.903 ms, 1107.306 ops/sec
ECDHE  256 agree        1200 ops took 1.043 sec, avg 0.869 ms, 1150.329 ops/sec
ECDSA  256 sign         1200 ops took 1.078 sec, avg 0.898 ms, 1113.279 ops/sec
ECDSA  256 verify       1700 ops took 1.045 sec, avg 0.615 ms, 1627.064 ops/sec

This application is especially useful for comparing the public key speed before and after changing the math library. You can test the results using the normal math library (./configure), the fastmath library (./configure --enable-fastmath), and the fasthugemath library (./configure --enable-fasthugemath).

Footprint sizes (compiled binary size) for wolfSSL range between 20-100kB depending on build options and the compiler being used.  Typically on an embedded system with an embedded and optimized compiler, build sizes will be around 60kB.  This will include a full-featured TLS 1.2 client and server.  For details on build options and ways to further customize wolfSSL, please see Chapter 2 of the CyaSSL Manual, or the wolfSSL Tuning Guide.

Regarding runtime memory usage, wolfSSL will generally consume between 1-36 kB per SSL/TLS session.  The RAM usage per connection will vary depending the size of the input/output buffers being used, public key algorithm, and key size.  The I/O buffers in wolfSSL default to 128 bytes and are controlled by the RECORD_SIZE define in ./wolfssl/internal.h.  The maximum size is 16 kB per buffer (as specified by the SSL/TLS RFC).  As an example, with standard 16kB buffers, the total runtime memory usage of wolfSSL with a single connection would be 3kB (the library) + 16kB (input buffer) + 16kB (output buffer) = around 35kB.

The TLS context (WOLFSSL_CTX) is shared between all TLS connections of either a client or server.  The runtime memory usage can vary depending on how many certificates are being loaded and what size the certificate files are.  It will also vary depending on the session cache and whether or not storing session certificates is turned on (--enable-session-certs).  If you are concerned with reducing the session cache size, you can define SMALL_SESSION_CACHE (reduce the default session cache from 33 session to 6 sessions) and save almost 2.5 kB.  You can disable the session cache by defining NO_SESSION_CACHE, reducing memory by nearly 3 kB.

If you want to use only a subset of ciphers, you can customize which specific cipher suites and/or ciphers wolfSSL uses when making an SSL/TLS connection.  For example, to force 128-bit AES, add the following line after the call to wolfSSL_CTX_new (SSL_CTX_new):

wolfSSL_CTX_set_cipher_list(ctx, “AES128-SHA”);

1. The processors native register size (32 vs 64-bit) can make a big difference when doing 1000+ bit public key operations.

2. fastmath (--enable-fastmath) reduces dynamic memory usage and speeds up public key operations.  If you are having trouble building on a 32-bit platform with fastmath, disable shared libraries so that PIC isn’t hogging a register (also see notes in the README):

   ./configure --enable-fastmath --disable-shared
   make clean
   make

   *NOTE: doing a “make clean” is good practice with wolfSSL when switching configure options

3. By default, fastmath tries to use assembly optimizations if possible.  If assembly optimizations don’t work, you can still use fastmath without them by adding TFM_NO_ASM to CFLAGS when building wolfSSL:

   ./configure --enable-fastmath CFLAGS=-DTFM_NO_ASM

4. Using fasthugemath can try to push fastmath even more for users who are not running on embedded platforms:

   ./configure --enable-fasthugemath

5. With the default wolfSSL build, we have tried to find a good balance between memory usage and performance.  If you are more concerned about one of the two, please see Chapter 2 of the wolfSSL manual for additional wolfSSL configuration options.

6. Bulk Transfers:  wolfSSL by default uses 128 byte I/O buffers since about 80% of SSL traffic falls within this size and to limit dynamic memory use.  It can be configured to use 16K buffers (the maximum SSL size) if bulk transfers are required.