Dropbox loves shingled disk drives

Dropbox has used shingled magnetic recording (SMR) hard drives in its data centres for a year now and today has let the world know how it is getting on with the technology. To cut to the chase, it loves shingled disk drives, which have saved it money and enabled it to keep up with data growth.

Dropbox adopted 14TB shingled disk drives from Western Digital in June 2018 for Magic Pocket, its in-house multi-exabyte storage system. Since then the company has used other vendors’ SMR drives and has told all a blog about its shingled disk drive experience

In its year of SMR drive use, Dropbox said moving to SMR has yielded

  • 20% plus savings overall compared to non-SMR disk drives,
  • It has been able to increase the density of its disk capacity, faster than the growth of data,
  • It has increased the number of disks from 60 to 100 on a single machine while maintaining the same CPU and memory.
  • Made 13 SMR-related contributions to the open-sourced libzbc SMR write library.

Dropbox said close to 40 per cent of its data will be on SMR by the end of the year. It envisages moving to 18, 21 and 24 TB drives, which tells us that Western Digital or other suppliers will introduce such SMR drive capacities.

Mingles in the Shingles

Shingled disk drives take advantage of disk write tracks being wider than read tracks. They partially overlap write tracks and so enable more tracks to be written and more data to be stored on a shingle disk compared to an ordinary drive.

Although shingled media recording (SMR) disks hold more data they are slower to write as groups of overlapping  tracks must be written sequentially in fixed 256MB zones, not just one track at a time.

Dropbox said it added an SSD staging area for data in preparation for fixed zone and sequential writes. Also “the size of the write zones we’ve set up in Magic Pocket, with 1 GB extents of data, fit perfectly with the 256 MB zones used to split up SMR drives.”

The code Dropbox uses writes directly to the disks without a filesystem, and it uses the Libzbc open source library to do this.

In its blog today Dropbox forecast “future technologies will likely use the same or similar API as SMR, where disks will be able to achieve greater densities by writing continuously to a limited number of zones of the disk at any given time. They may be microwave-assisted magnetic recording (MAMR) drives or heat-assisted magnetic recording (HAMR) drives—but they will have the same interface and we’ll be able to use the same software architecture.”

This implies multiple read:write heads working in parallel.