Improving ssh/scp Performance by Choosing Suitable Ciphers

Update on Oct. 9, 2014: You should be aware of the possible security problems of blowfish and it is suggested not to be used. Instead, you may consider ChaCha20 as suggested by Tony Arcieri. To use this with OpenSSH, you need to specify the Ciphers in your *.ssh/config* files as chacha20-poly1305@openssh.com possibly with another default one since only newer vesion of OpenSSH supports it.


ssh/scp are convenient and handy tools on Linux. Is it possible to further improve its speed/performance, especially when transferring files that are large (GBs, not MBs or KBs)? The answer is of course yes.

ssh supports several ciphers. The ciphers used have a large impact on the performance. Jens Neuhalfen and Ivan Zahariev‘s data are roughly the same as my own experience (from faster ones to slower ones):

arcfour >> blowfish >> aes >> 3des

Note that SSH 2 supported ciphers have more variance:

Ciphers

Specifies the ciphers allowed for protocol version 2 in order of preference. Multiple ciphers must be comma-separated. The supported ciphers are 3des-cbc aes128-cbc aes192-cbc aes256-cbc aes128-ctr aes192-ctr aes256-ctr aes128-gcm@openssh.com aes256-gcm@openssh.com arcfour128 arcfour256 arcfour blowfish-cbc and cast128-cbc. The default is:

aes128-ctr,aes192-ctr,aes256-ctr,arcfour256,arcfour128,
aes128-gcm@openssh.com,aes256-gcm@openssh.com,
aes128-cbc,3des-cbc,blowfish-cbc,cast128-cbc,aes192-cbc,
aes256-cbc,arcfour

Although arcfour is the fastest, there are concerns about its security.

Hence, my default choice is blowfish for both speed and security. According to the OpenSSH ssh man page:

The supported values are 3des blowfish and des. 3des (triple-des) is an encrypt-decrypt-encrypt triple with three different keys. It is believed to be secure. blowfish is a fast block cipher; it appears very secure and is much faster than 3des.

It delivers around 40MB/s~50MB/s on a 1Gbps link in our cluster. For comparison, the speed is 10+MB/s with the default ciphers used. If you run ssh/scp inside a trusted cluster, arcfour may be also a good choice.

It is also a good idea to enable compression by default so that ssh performs better over a low-bandwidth link, such as a slow Internet connection.

Overall, I put these lines into my ~/.ssh/config:

Host *
  Ciphers blowfish-cbc
  Compression yes
  CompressionLevel 6

The first line tells ssh/scp that these configuration applies to all hosts. The Ciphers line tells ssh/scp of version 2 to use blowfish-cbc. The 3rd and 4th lines enable compression and set its level.

To check which ciphers your are using, run ssh with -v parameter and find out lines like this in the “debug1” outputs:

debug1: kex: server->client blowfish-cbc hmac-md5 zlib@openssh.com
debug1: kex: client->server blowfish-cbc hmac-md5 zlib@openssh.com

You can see here that blowfish-cbc is used.

Eric Zhiqiang Ma

Eric is interested in building high-performance and scalable distributed systems and related technologies. The views or opinions expressed here are solely Eric's own and do not necessarily represent those of any third parties.

3 comments:

  1. Below is a script I made that you can use to benchmark each cipher. I use /dev/zero, /dev/null, and localhost to eliminate potential i/o bottlenecks. The results should report realistic maximum transfer throughput. There is also the “openssl speed” benchmark, but I find that doesn’t produce accurate data transfer throughput results.

    The following code works on Linux, Mac OS X, and Solaris:
    for i in 3des-cbc aes128-cbc aes128-ctr aes128-gcm@openssh.com aes192-cbc aes192-ctr aes256-cbc aes256-ctr aes256-gcm@openssh.com arcfour arcfour128 arcfour256 blowfish-cbc cast128-cbc chacha20-poly1305@openssh.com rijndael-cbc@lysator.liu.se; do dd if=/dev/zero bs=1000000 count=1000 2> /dev/null | ssh -c $i localhost "(time -p cat) > /dev/null" 2>&1 | grep real | awk '{print "'$i': "1000 / $2" MB/s" }'; done

    With AES-NI acceleration, aes128-gcm@openssh.com is the fastest cipher on OpenSSH Intel systems… take a look:

    Intel Core i7-2635QM (MacBookPro8,2)/ Fedora 21:
    3des-cbc 18.34 MB/s
    aes128-cbc 167.54 MB/s
    aes128-ctr 230.26 MB/s
    aes128-gcm@openssh.com 328.37 MB/s
    aes192-cbc 158.13 MB/s
    aes192-ctr 228.28 MB/s
    aes256-cbc 148.89 MB/s
    aes256-ctr 222.80 MB/s
    aes256-gcm@openssh.com 317.49 MB/s
    arcfour 178.52 MB/s
    arcfour128 177.52 MB/s
    arcfour256 178.50 MB/s
    blowfish-cbc 66.75 MB/s
    cast128-cbc 58.68 MB/s
    chacha20-poly1305@openssh.com 131.70 MB/s
    rijndael-cbc@lysator.liu.se 149.04 MB/s

    Mac OS X 10.10 cipher support is abysmal compared to Linux, take a look at the results:
    Intel Core i7-2635QM (MacBookPro8,2)/ Mac OS X 10.10:
    3des-cbc 14.98 MB/s
    aes128-cbc 88.08 MB/s
    aes128-ctr 87.18 MB/s
    aes192-cbc 85.18 MB/s
    aes192-ctr 84.10 MB/s
    aes256-cbc 82.51 MB/s
    aes256-ctr 81.48 MB/s
    arcfour 133.93 MB/s
    arcfour128 133.99 MB/s
    arcfour256 133.93 MB/s
    blowfish-cbc 39.96 MB/s
    cast128-cbc 39.06 MB/s
    rijndael-cbc@lysator.liu.se 82.24 MB/s

  2. @Nikolas:
    You test are performed on the same hardware, just different OS ?
    If so, the difference is indeed quite impressive.

    Make sense that a CPU with AES hardware support performs better, no ?

    I think that you need to take into account both machine, a ssh is not usualy used to connect locally.

    I use ssh for two main purpose.
    – edit file remotely, using a screen and vi.
    – ssh -Y to forward some graphical application (wireshark for instance)

    the cipher, the MACs and the compression have huge and different impact according to the usecase.
    I use one dedicated ~/.ssh/config entry for each usecase, different parameters (to connect to the same server).
    I have different parameters per server, as some have aes hardware acceleration and some don’t.
    I also take into account the network, when I work on my laptop on wifi, I use different parameters than ethernet.

    Bref, it’s a very deep and non-deterministic science.
    I don’t think there is one config that rules them all.

Leave a Reply

Your email address will not be published. Required fields are marked *