I connected a brand-new 500 gigabyte Western Digital "My Book" USB hard drive to the computer.  Then I started a backup to the drive.

It failed after an hour:  "End of Media encountered while backing up to non-removable media."  The backup file was 4 gigabytes in size.

I wonder why the drive can't hold a file bigger than 4 gigabytes?

Maybe because it is pre-formatted as ... FAT32?

 

Of course it is FAT32.  That's because that is what is most likely to work.  It works with NT, it works with the DOSoid Windows (ME, 98, even 95 if it is the version with USB support), it works with MacOS and it works with Linux, all without needing to format.

Now in your particular case, FAT32 isn't the right choice, so format it, already.

The real WTF is that you are complaining about something that makes this device more accessible to more users.  I think the 80/20 rule applies here.

 

Critter:

Of course it is FAT32.  That's because that is what is most likely to work.  It works with NT, it works with the DOSoid Windows (ME, 98, even 95 if it is the version with USB support), it works with MacOS and it works with Linux, all without needing to format.

Now in your particular case, FAT32 isn't the right choice, so format it, already.

The real WTF is that you are complaining about something that makes this device more accessible to more users.  I think the 80/20 rule applies here.

While I agree the 80/20 rules does apply, I don't think you are correct in your assessment that it will work under all those systems. I do believe that it will fail in this case under all those system for the reason he states.

Now my question is this, barring the single file of 4gig could he have create 5 files a gig in size each and have it work?  What was the size limit onFAT32 or was it simply sector size which USB devices ignore?  If this test failed then yes, having it formatted for FAT32 is a WTF because by default it would not support the 500gig size.

 

I'm always suspicious when hard drives come pre-formatted. I've bought 3 WD hard drives, none of them were formatted. If you didn't buy from a dealer with a solid reputation (however you want to define that), you should check the drive for previous data.

Could be that someone started copying their pr0n onto the drive, had some problem with it, and brought it back to the dealer where it was formatted and re-shelved.

- Eam 

 Edit: I bought SATA drives, so the USB drives may be targeted at less-awesome customers and come with shitty formatting for a legitimate reason.
 

File size limit is approximately 4GB. According to http://www.allensmith.net/Storage/HDDlimit/FAT32.htm, Windows 2000/XP can not format FAT32 over 32GB (but CAN use drives formatted FAT32 that are over 32GB), whereas Windows 98 could format FAT32 up to about 128GB. Windows ME had a theoretical limit of about 4TB using FAT32.

KattMan:

Now my question is this, barring the single file of 4gig could he have create 5 files a gig in size each and have it work? 

Absolutely, yes.  He could even create five files of 4GB each.  He just could not exceed 4GB per file. 

What was the size limit onFAT32 or was it simply sector size which USB devices ignore?  If this test failed then yes, having it formatted for FAT32 is a WTF because by default it would not support the 500gig size.

No, it hasn't got anything to do with USB.  FAT has (as I mentioned a second ago) a 4GB file size limit, and the filesystem size is accommodated by changing the block size, if I recall correctly.  While I don't have any first-hand experience that says FAT works at 500GB, I do have some that says it works at more than 4GB, 60GB to be specific.

FAT32 has one other annoying limitation, and that is 127 objects maximum in the root directory.  This limit does not affect subdirectories.  This is sometimes an obstacle with my MP3 player.

I have to call this not a WTF.  The WD MyBook is basically a user friendly (read dead easy to use) external drive, designed for the general populace to plug in and do backups.  It comes with some autorun backup programs for Windows, but works just fine on my Linux boxes as well (in case you couldn't tell I have one).

As already stated the 4GB limit is only for single files.   The only time you should really have a problem is if you are doing video archiving, or personal video editing.  Even then you should usually be below the 4GB limit.

As the original poster found out, the software they usually come with, Dantz (EMC?) Retrospect, won't split files into 4gb partitions. So if you just say "back it all up!" the first files it creates will be as large as your (compressed) files - overflowing 4gb.

Doesn't FAT32 have major slack space issues with partitions that big?

The main problem is that the only filesystem supported by Windows, Mac AND Linux is FAT32. If you format it as, say, ReiserFS, then it is only readable on Linux.

If you use NTFS, you can read/write to it on Windows and read-only on Linux (can Macs at least read NTFS?)

 

So you don't really have much option if you want compatibility across Win/Mac/Linux.

 

EXT2 or EXT3 might be a good choice. They are backwards/forwards compatible and support is built into linux. Fairly good EXT2 drivers (which are of course compatible with EXT3) exist for Windows and Mac as well.

 

H3SO5:

If you use NTFS, ... read-only on Linux (can Macs at least read NTFS?)

 

This is no longer true. 

mthamil:

H3SO5:

If you use NTFS, ... read-only on Linux (can Macs at least read NTFS?)

This is no longer true.

links or it didn't happen.

I was under the impression that, while you could theoretically write NTFS on linux, it was currently in pre-1.0 and known to occasionally break.

Have you seen the NTFS-3G driver ( http://www.ntfs-3g.org/ ) - it claims that "Most POSIX file system operations are supported, with the exception of full file ownership and access right support. ".

Irrelevant:

mthamil:

H3SO5:

If you use NTFS, ... read-only on Linux (can Macs at least read NTFS?)

This is no longer true.

links or it didn't happen.

I was under the impression that, while you could theoretically write NTFS on linux, it was currently in pre-1.0 and known to occasionally break.

http://www.ntfs-3g.org/

It was just released as stable a few months ago, so I'm not surprised you didn't hear about it.  It does have some performance issues - it hasn't been optimized.  It isn't 100% as it will fail in some corner cases like extending the MFT in certain scenarios. 

 

 

mthamil:

H3SO5:

If you use NTFS, ... read-only on Linux (can Macs at least read NTFS?)

 

This is no longer true. 

 

what version of what distro?  Is it stable or dev? 

lpope187:

Irrelevant:

mthamil:

H3SO5:

If you use NTFS, ... read-only on Linux (can Macs at least read NTFS?)

This is no longer true.

links or it didn't happen.

I was under the impression that, while you could theoretically write NTFS on linux, it was currently in pre-1.0 and known to occasionally break.

http://www.ntfs-3g.org/

It was just released as stable a few months ago, so I'm not surprised you didn't hear about it.  It does have some performance issues - it hasn't been optimized.  It isn't 100% as it will fail in some corner cases like extending the MFT in certain scenarios. 

 

 

 Thank you so much!
 

bobday:

I'm always suspicious when hard drives come pre-formatted. I've bought 3 WD hard drives, none of them were formatted. If you didn't buy from a dealer with a solid reputation (however you want to define that), you should check the drive for previous data.

Could be that someone started copying their pr0n onto the drive, had some problem with it, and brought it back to the dealer where it was formatted and re-shelved.

- Eam 

 Edit: I bought SATA drives, so the USB drives may be targeted at less-awesome customers and come with shitty formatting for a legitimate reason.
 

 

No, the USB Harddrives always seem to come pre-formatted as FAT 32.

shadowman:

No, the USB Harddrives always seem to come pre-formatted as FAT 32.

It's the only filesystem supported by windows by default that won't enthusiastically corrupt itself if you unplug the drive without unmounting it first (NTFS is notoriously bad at this).

It's also a terrible filesystem. Go with ext3 wherever possible.

great to hear about that NTSF thingy :)

I have a 300GB external maxtor usb drive methingy, and was getting sick of FAT32 as well.
Now i can finally transport those huge ummm linux dvd iso's on it.

Currently it's holding around about 250GB of anime :P

It's weird how it doesn't really matter how much space you have, it will always find a way to fill up.
When i first bought my drive i though i would be able to keep free space for atleast a year, about 2 months later is was 95% full :)
 

stratos:

great to hear about that NTSF thingy :)

I have a 300GB external maxtor usb drive methingy, and was getting sick of FAT32 as well.
Now i can finally transport those huge ummm linux dvd iso's on it.

Currently it's holding around about 250GB of anime :P

It's weird how it doesn't really matter how much space you have, it will always find a way to fill up.
When i first bought my drive i though i would be able to keep free space for atleast a year, about 2 months later is was 95% full :)
 

It's easy to fill drives when you download anime and porn all day. 

asuffield:

It's the only filesystem supported by windows by default that won't enthusiastically corrupt itself if you unplug the drive without unmounting it first (NTFS is notoriously bad at this).

I would say that any journaling filesystem set to write to disk only when buffer is full on a USB connection is an error waiting to happen.

asuffield:

It's also a terrible filesystem. Go with ext3 wherever possible.

Could you explain why you think NTFS is so bad?

bobday:

Could be that someone started copying their pr0n onto the drive, had some problem with it, and brought it back to the dealer where it was formatted and re-shelved.

Are you suggesting it's a bad thing to get somebody else's porn?

I mean, granted, the harddrive might not work. But free porn!
 

smbell:

I have to call this not a WTF.  The WD MyBook is basically a user friendly (read dead easy to use) external drive, designed for the general populace to plug in and do backups.  It comes with some autorun backup programs for Windows, but works just fine on my Linux boxes as well (in case you couldn't tell I have one).

As already stated the 4GB limit is only for single files.   The only time you should really have a problem is if you are doing video archiving, or personal video editing.  Even then you should usually be below the 4GB limit.

i'm pleased to see that you've also started tagging your posts with "tag abuse"...warm fuzzies

newfweiler:

I connected a brand-new 500 gigabyte Western Digital "My Book" USB hard drive to the computer.  Then I started a backup to the drive.

It failed after an hour:  "End of Media encountered while backing up to non-removable media."  The backup file was 4 gigabytes in size.

I wonder why the drive can't hold a file bigger than 4 gigabytes?

Maybe because it is pre-formatted as ... FAT32?

 

Of course I reformatted it before posting.  I used NTFS for convenience.

In my opinion, everything I buy should come out of the box as one of the following:

1.  Some preparation required, or

2.  No preparation required.

It should either be completely ready to use, or not ready to use.  Either just plug it in and it works, or attach the power cord, pull out the battery tab, format it and it works.  One or the other, please.

The disk came preformatted with FAT32 because that's the only format that can be read by almost any system.  At the same time, FAT32 on a 500GB drive is of no use to anyone because of the file size limitation.

The only reason the disk comes formatted at all is so that it can install the Google Toolbar and show you a few ads when you plug it in.  For Your Convenience of course.  Oh, yes, and so it can install some hard drive utilities you don't need.