Chunking files at 32MB

November 27, 2017,
improved

One of the lesser known features of Ahsay 7 is the change in how it slices up large files before transmitting them.

Previous versions would send a file in its entirety after compression to the backup destination. It was perfectly normal for us to see Hyper V backups of 2 TB being stored on our platform.
Anyone can see the pitfall of such a process. If a network outage is going to happen as 1.999TB has been transmitted, the backup will need to start again.

The issue isn’t unique to Ahsay and all backup vendors have had to ‘up their game’ recently to facilitate large backups. The common phrase is ‘chunking’

In version 7 Ahsay have re-worked the chunk process so that all files are chopped up into 32MB size blocks. On a quiet day and if you wanted to, you could watch the stack on Azure and see thousands of 32MB blocks come flying is and being put to rest.

Resellers can in version 7 (‘Your Storage service’) backup large Hyper V and VMware data sets to Dropbox, Google and other 3rd party clouds. Traditionally these vendors would never allow large files onto their networks. Now we are slicing large files into 32MB blocks, it is easy to back up very large files to these basic file sharing platforms.
Everyone has spare capacity on these platforms which is why they work very well as a free online backup destination.

With our software you will have all of the modules required to back up anything you find in Microsoft, MAC or VMware environments.

Quick comparison between version 6 and 7 block sizes
Looking at a 1TB file backup. In version 6 we would have seen one file 1TB in size, in version 7 we see 32,000 x 32MB files. In tests we found having such a large amount of files does add somewhat to the overhead (approx. 5%), however the benefits of having a better recovery system during large transfers outweighs this.

Has compression improved?
Compression is one of the most important features of any online backup system. The more we compress files, the quicker the backups and restores run and the lower the storage prices are. We estimate compression rates of 40% on generic office files and data. Databases normally have a lot of white space and they can compress by up to 90%, however binary files such as images generally don’t compress by more than 2%.

 

BOBcloud.net
The Old Sorting Office, Corsham, Wiltshire SN13 9AA
Tel: 0800 907 8238 https://www.bobcloud.net/wp-content/themes/bobcloud/images/logo.png