Disk drive lottery :-(

[corblimey]

I bought 3 x 3TB disks last week to use as backup destinations for BA and at the time I was blissfully unaware of the minefield known as Advanced Format Drives. That is until I tried to do a system image to one of the drives.

Turns out there is a build in limitation to the Windows imaging engine that prevents it writing images to drives that reports the newer 4kB sector size. Many of these new drives provide compatibility by emulating the older 512B sector size so it's not always a problem. It's widely discussed on the Internet but like lots of things, until you start looking you won't see it.

Unfortunately it's not very easy to find out what kind of drive you are buying (i.e.whether or not it supports emulation), it doesn't just apply to the +2TB drives and the emulation (512e) doesn't always work when the device is behind a USB<->Sata interface.

Support have suggested that the latest drives (+2TB) will be AF and should be avoided. This doesn't seem practical since with 4TB shipping now +2TB drives will be the norm in just the few months time. That could add up to a lot of people purchasing unsuitable drives without knowing it. The issue should at least be made more visible and ideally a list of compatible devices should be published.

Alternatively, is there another way these drives could be used. I've not tried and I don't know the technicalities but could a drive be made available as a network share or an iscsi target (for the forthcoming BA release) as a way of "masking" the drive's sector size?

[TimN]

Hi corblimey,

It's Tim from BackupAssist. Thanks for your post.

Currently the best workaround for this would be to emulate the 512B sector size, as you have mentioned, if this is supported by your drive .

Western Digital drives do also have an Advanced Format utility that will format the drive with a 512B sector size.

More information on this is available here: http://wdc.custhelp.com/app/answers/detail/a_id/6618/session/L3RpbWUvMTMxMzUzNzc5My9zaWQvT1hrMSpJQms%3D

Please note that you will still be able to use the File replication or Zip backup engines in BackupAssist to backup to these devices.

As for making the drive available as an iSCSI target, this may work, however, we cannot confirm this as we are yet to test this extensively in-house.

Feel free to post any further findings and queries here.


Kind regards,

Tim.

[corblimey]

Hi Tim

Thanks for getting back to me.

I have read about the wd format tool but there is also this conflicting kb article on the wd site http://wdc.custhelp.com/app/answers/detail/a_id/5354/session/L3RpbWUvMTMzMDY3Nzk3NC9zaWQvdkV6UzkzU2s%3D stating they don't support Windows backup.

That's my point really, without some kind of concerted effort to document and list compatible products, it's a complete lottery.

From my reading, it would appear the issue is not going away until MS release an updated imaging engine but who knows when that will be. If it doesn't come until Win 8 server then it could take a long time to trickle down to the masses. That's a long window of time during which the end user is stuck in lottery mode.

Whilst I accept the problem is not of your making I feel that since fundamental functionality of your product is founded on this core MS feature there is at least a duty on your part to try and make the problem more visible to users and perhaps look at ways to ease the issue. That may just be maintaining some compatibility documentation or perhaps testing alternative hardware setups (e.g an iscsi solution with AF drives).

So far the only info I have found on the site relating to this issue is a blog entry from last year (which I would never have seen unless support had sent me the link) and a similar post to mine, again from last and also asking for a compatibility list.

http://www.backupassist.com/blog/support/4kb-sector-sized-disks-and-the-seagate-freeagent-goflex/
http://www.backupassist.com/phpBB3/viewtopic.php?f=11&t=3920&p=10082&hilit=advanced+format#p10082

I'm happy to start the ball rolling and add my experiences with various hardware as I come across stuff if that is of help. It would be nice to see a more proactive stance from BA's side though.

What do others think?

My experience so far:

Hitachi Touro Desk DX3 3TB USB3 drives do not work (part # HT0LDX3EB30001ABB)

Hitachi's tech briefing on the subject states :

"In order to ease the transition, current Advanced Format drives provide 512-byte emulation (512e)1 at the drive interface for backward compatibility with legacy applications."

If this is true then it doesn't work behind the DX3 sata <-> usb3 interface
http://www.hitachigst.com/tech/techlib.nsf/techdocs/3D2E8D174ACEA749882577AE006F3F05/$file/AFtechbrief.pdf

Regards

Glenn

[keithy1959]

I feel your pain - I have just spent £3000 of my company's money on a 10TB iScsi box, configured it all up to 2TB Volumes, and am now stripping it all back to 1.5TB "Drives".

In fairness to BackupAssist, at least in the stuff in the UK's vendor, it does make it clear that there is a 2TB limitation, but I read that to be a limitation of BUA ( and Microsofts) to back up FROM >2TB, not TO partitions >2TB. Caveat emptor.

Incidentally the error messages I had from Windows via BUA made no mention of this message, but sent me down a 2 day road of 3rd party iSCSI Initiators, failing I/O and anything but a design limitation. Perhaps BUA could do a quick sniff at the destination, and if its >2TB, stop the job creation.

With regards your disks, you might want to look at wrapping them in a cheap PC and turn it into a NAS - : http://www.openfiler.com/. I haven't used this myself but the IT support team in our Dubai Office have set one up there - I'm going there next week, so if you are interested, I can have a play and update this thread then.

[corblimey]

In fairness to BackupAssist, at least in the stuff in the UK's vendor, it does make it clear that there is a 2TB limitation, but I read that to be a limitation of BUA ( and Microsofts) to back up FROM >2TB, not TO partitions >2TB. Caveat emptor

Yes you read it right, you can't create a VHD file any larger than 2TB hence you can't image a partition larger than 2TB. We have to wait for the new VHDX format for that and I assume that will also resolve the 512b issue. Thing is that doesn't stop you backing up a system larger that 2TB if you split the backup jobs and methods accordingly. The drive issue is different, if the drive doesn't work (regardless of size) you can't do bare metal and as we all know that is not recommended best practise!

I see on Zen's site there is a one liner about 4k sector disks. Not sure how long that's been there, maybe they added after I moaned about it to them last week

Still it's not that simple since most new drives will be 4k now anyway, it's that some work with 512e (512 b emulation) and some don't. Some have 512e but don't appear to work behind a sata <-> usb interface but are fine if directly connected to the machine's sata interface. How do we know what works and what doesn't? I for one are not in a position to keep purchasing drives until I find something that works.

This is where I think BA should step in and help by collating a list of compatible drives or finding a work around. It's extremely frustrating to simply be told by support to contact the drive manufacturer, guess what, I contacted the drive manufacturer and they said.... " contact your software vendor". "Hand washing" really is the worst kind of support, stepping up and taking ownership is the way to hang onto users. At present, if I want to be confident new drives are going to work as bare metal backup media I'm going to have to turn to a different product (like Storagecraft's ShadowProtect) for my solutions portfolio. This is not where I want to be, I like BA alot

Can anyone recommend 3TB usb drives that work?

[Sally]

Looks like MS will start providing full support for Advanced Formatted drives starting with Win8:

“Learning from some issues identified with prior versions of Windows, AF disks have been a key design point for new features and technologies in Windows 8; as a result, Windows 8 is the first OS with full support for both types of AF disks – 512e and 4K Native.”

…
“Issues we addressed included the following:
• Introduce new and enhance existing API to better enable applications to query for the physical sector size of a disk
• Enhancing large-sector awareness within the NTFS file system, including ensuring appropriate sector padding when performing extending writes (writing to the end of the file)
• Incorporating large-sector awareness in the new VHDx file format used by Hyper-V to fully support both types of AF disks
• Enhancing the Windows boot code to work correctly when booting from 4K native disks “
http://blogs.msdn.com/b/b8/archive/2011/11/29/enabling-large-disks-and-large-sectors-in-windows-8.aspx

We have not received any information on whether they will implement it back into previous versions of Windows or not but we are keeping an active eye on this situation.

[corblimey]

Hi Sally

Thanks for the info.

As I said, that's going to take a while to trickle down unless we all upgrade to Win8 on it's launch (whenever that may be) or they roll the imaging to previous win versions. In the meantime more and more drives will be released.

Having just spoken to my supplier I now find myself the the position of have to pay a 20% restocking fee for them to take back two of the drives which are still sealed so they can go back on the shelf to waiting to be sold to some other sucker. And, i get lumbered with the opened drive which is of no use to me. And, I have to take yet another gamble trying to find replacements.

After much battling they have said if I document everything I've told them in an email they will pass it on to distribution and if they deem it a problem that warrants them pulling the drive from sale they will fully refund me.

Really not sure how I move forward now.

If anyone can recommend drives that work I'd really appreciate it.

[corblimey]

After much digging I found this

http://social.technet.microsoft.com/wiki/contents/articles/1780.external-backup-drives-compatibility-list.aspx

Hope it's of use and helps people avoid the situation I've found myself in. Don't take the list as gospel though, there is at least one case in the thread below where someone has purchased an HP external drive because it was on the list and it didn't work for them.

If you want to get a flavour of how widespread the issue is this is worth a read even if it is a little long. The thread was started in 2009 and is still running now
http://social.technet.microsoft.com/Forums/en-US/windowsbackup/thread/5d9e2f23-ee70-4d41-8bfc-c9c4068ee4e2/

I also spoke to Hitachi again this afternoon and the chap said the drives I have support 512e. I used CrystalDiskInfo to discover that the drive inside housing is a Deskstar 7K3000(HDS723030ALA64) and the spec sheet says it has a sector size of 512. There is no mention of whether or not it's 512e of native 512b (must be 512e since it's 3TB) despite the very important impact on compatibility
http://www.hitachigst.com/tech/techlib.nsf/techdocs/155901D3B251D9A9862577D50023A20A/$file/DS7K3000_ds.pdf

It's all totally ridiculous!!!

[corblimey]

Still battling with my supplier

In the meantime I have spoken to Western Digital and they have confirmed that the "My Book Essentials" drives will work once the "Default" reformat trick has been performed using their format software.

http://wdc.custhelp.com/app/answers/detail/search/1/a_id/6618/c/130/p/208,296

At least it's acknowledged, documented and a solution given by someone which is reassuring.

I'm going to get a 3TB My Book Essential drive to test and will report back to share my findings.

Whilst I acknowledge the issue is no fault of BA and they have no control over the drive manufacturers or Microsoft, I do feel they should make this issue visible to their customers. What are the official thoughts on this?

[corblimey]

Two bits of good news. Shame I had to waste so much time sorting it out.

Finally got my drives returned with a full refund after my supplier had been in conversation with Hitachi.

Here's what Hitachi said:

The drive inside the Touro is a 512 byte/sector bare drive, and should in theory work with Windows 7 backup. The problem is that the actual enclosure is designed to report the drive as 4096 bytes if you query it, which makes it incompatible with the Windows 7 backup utility.

The reason that the enclosure was designed to report as 4096 bytes is to allow Windows XP users the ability to connect a large (3TB) harddrive. In general Customers that have this issue are advised to use the Hitachi Backup software, and not to use the software built into the Windows 7 OS. This is not unique to Hitachi, WD and Seagate Customers also have the same issue.

One alternative solution would be to move him to a G-DRIVE 3TB, this would not have the issue with Windows 7 backup.

The best bit of news is that I have purchased a WD MY BOOK ESSENTIAL USB3.0 3TB (part # WDBACW0030HBK-UESN) and followed WD's kb article and it works!
http://wdc.custhelp.com/app/answers/detail/search/1/a_id/6618/c/130/p/208,296

Think I'll be sticking to WD drives to the time being.

It would still be nice to hear from support on this, anyone?

Whilst I acknowledge the issue is no fault of BA and they have no control over the drive manufacturers or Microsoft, I do feel they should make this issue visible to their customers. What are the official (Cortex) thoughts on this?

[michael.jones]

Hello Craig,

Thanks for contacting us in relation to BackupAssist. I'm Michael Jones, one of the technical support engineers.
I'm glad to hear that you got it sorted out.


Should you have additional questions or concerns Craig, please don't hesitate to contact me - I'd be glad to assist you.

Thank You,
Michael R. Jones
BackupAssist Help Desk
US Support Office: New Albany, IN

[corblimey]

Hi Michael

Thanks for the reply.

Unfortunately you don't give much away with regard to your answer to my question.

Whilst I acknowledge the issue is no fault of BA and they have no control over the drive manufacturers or Microsoft, I do feel they should make this issue visible to their customers. What are the official (Cortex) thoughts on this?

If I were to read between the lines of your reply I'd come to the conclusion that your response would be "nothing"

I'm sure this issue is going to reoccur more and more. At least making a list of working drives a sticky post in this forum would go some way to help others not get caught out.

[Pab]

I've alos come accross the same situation as you CorBlimey! Not sure that it's BA's place to publish a list of supported hardware. What is a manufacturer changed the hardware without notifying the world? BA would be recommending a product that does not work and they'd be in for more, justified, flak.

I found a seagate drive that works and i've stuck to them ever since. I'm using seagate expansion 1TB portable hard disks.

[corblimey]

Pab, thanks for adding your experience.

I guess I'd see it more along the lines of a list held by BA and contributed to by BA users sharing their experiences with different models.

As I say, I'll happily acknowledge the issue is by no means of their making but since imaging is an inherent part of their product a useful compatibility list of some form is surely not an unreasonable request. It's the issue of visibility of the problem that bugs me, I wasted a lot of time on this and I'm sure I won't be the last. It's well know but not visible unless you have the problem and I'm sure it will become more of an issue so why not give us a heads up. Besides they already publish a hardware compatibility list it just hasn't been updated of late.


Reported to work:
Seagate expansion 1TB portable hard disk
WD external drives as documented here http://wdc.custhelp.com/app/answers/det ... /p/208,296

Reported not to work:
Hitachi Touro DX3 3TB

[scpm]

There is a community self-managed list of external drives that work located here:

http://social.technet.microsoft.com/wik ... -list.aspx

Please update the list with your own results