Recently we encountered a case on the help desk where a customer’s File Replication backup was taking more than 12 hours to complete. During an investigation of the issue we noted that the time required to complete a backup had been increasing gradually over a period of about 12 months:
|20th October, 2009||22 minutes|
|20th January, 2010||77 minutes|
|20th April, 2010||208 minutes|
|20th August, 2010||521 minutes|
|20th October, 2010||651 minutes|
|18th November, 2010||720 minutes|
The customer was running BackupAssist version 5.4.7, which was configured to run a File Replication backup with Single Instance Store (SIS) enabled to a 1TB external USB hard drive connected locally.
- About Single Instance Store (SIS) and hard links: if Single Instance Store is enabled, only one unique copy of each file selected for backup will be stored on your backup device. SIS is similar to an incremental backup: if files are modified or new ones created, such changes will be merged into a new and complete backup set containing what appears to be a complete list of all files selected for backup, even though only one unique copy of each file actually exists in terms of disk space on your backup device. In other words, the same file can appear within four different backup sets, but still only be stored once on the backup device, thanks to hard links, which point back to the location of the original version of the file on the drive.
- For more information about SIS and hard links please see the following blog post: http://www.backupassist.com/blog/support/single-instance-store-a-guide-to-hard-links
The customer had already tried a number of things to reduce the backup time, including deleting previous backups from the backup destination and re-formatting the backup drive. This did result in a decrease in the time taken to complete a backup, but it was not an ideal solution given the loss of backup data. A different solution was required.
We were aware that if the hard link count for a file continues to increase (i.e. there are many backup sets on your drive containing a file or files that hard link back to a unique original), it will take longer and longer to replicate these hard links on the backup destination; this invariably affects the time taken to complete a backup. It seemed likely that an excessive number of hard links was the reason why the last backup took so long to finish.
To demonstrate that this was the case, we ran the backup for a few nights with SIS disabled to see how long a full copy of the data would take. The backup took an average of 93 minutes to finish, which was a massive reduction. SIS was re-enabled to see what impact it would have and the result was an impressive 33 minutes to complete the backup.
The customer was happy with this solution as it achieved both requirements:
- No previous backups were lost.
- Backup times were greatly reduced.
We are considering adding a feature to BackupAssist to automatically archive old backups on the backup destination to a new folder after a user defined period, but for now, if you experience an increase in backup time for a job where SIS is enabled, it might be worth disabling SIS, running a full backup, and then re-enabling SIS.
For instructions on how to enable and disable SIS for a File Replication job click here.
If you need any further information or have any questions, please let us know by contacting us at firstname.lastname@example.org.