Pure Storage overcomes dedupe blocking with end-to-end encryption in flash array

By
Chris Mellor
-

Pure Storage has enlisted the help of Thales to build an end-to-end (E2) encryption facility with no deduplication blocking for the company’s FlashArray//X.

The technology, called Vormetric Transparent Encryption (VTE), was introduced at the 2019 RSA Conference. It is “transparent” in the sense that encryption takes place on the host and is invisible to users or the application.

VTE resolves the difficulty of encrypting data on deduplicating storage arrays.Normally a deduplicating storage array is baffled when a stream of encrypted data comes its way. Both compression and deduplication can be rendered ineffective such that there are few or no space savings.

So how does VTE perform? A FlashArray//X was asked to store the the publicly available 5.3GB Enron email corpus. The array reduced that 79.1 per cent to 1.11GB, a 4.8:1 reduction ratio. It was then encrypted using VTE and stored on a volume in the array with no VTE integration. Result: no data reduction at all.

Th data was then written to the Pure array with VTE integration, and reduced to 1.11GB again – the same 4.8:1 reduction ratio.

How is it done?

  1. The Vormetric File System agent is installed on a LINUX host
  2. The host checks out an encryption key from the Vormetric Data Security Manager (DSM)
  3. The FlashArray registers as a KMIP client with the DSM and checks out the host encryption key
  4. The host writes encrypted data to the FlashArray
  5. The FlashArray decrypts the data using the host key, reduces it, and re-encrypts it with the FlashArray key before writing it to flash. The un-encryption of data with the host key is an added step introduced with the integration.
  6. When the host reads the data, the FlashArray decrypts the data using the FlashArray key and re-encrypts with the host key prior to sending the data to the host. The re-encryption of data is an added step introduced with the integration.

Note there are two added storage steps, which will add some time to operations.

VTE integration also provides granular access control, privileged user access policies and audit logs.  

FlashArray//X requires v.5.3 of the Purity OS for all this to work. The upgrade is coming soon.

RELATED ARTICLESMORE FROM AUTHOR