How is fragmentation handled?

[jwortiz333]

Hey guys,

I was curious about something. I just became a new owner of a Dish DVR510 ($99.00 promo) and I'm loving it! A question did pop up when I was using it over the weekend and I was wondering if someone can answer it.

How is fragmentation being handled on the DVR's? I figure with all these recorded programs being deleted and new ones recorded, there's going to be some serious fragmentation as a result.

Thanks.

 

[A_Noland]

I had one of the first 501's and was told that a defragmenter was added to the software. I have know idea if it is true though.

 

[James]

From what I've read at TiVo Community, the TiVo handles the defrag on its own.

 

[Smith P.]

As I can see from reports of some third party HDD diag program - it's _never_ happen with all existing versions of Dish PVR software .

 

[sampatterson]

Sounds scary to me, the idea of Dish writing defrag software for their receivers...

 

[supp]

I don't see a need to worry about the fragmentation that occurs on a DVR hard disk for SD content. A DVR reads and writes SD content to a hard drive at a relatively constant and relatively low bit rate compared to the drives maximum transfer capacity. The delay introduced by fragementation is unlikely to impact a hard disk's ability to transfer data at the rate necessary to handle these video streams. Don't forget too that these drives can have as much as 8Mb of cache in them now at consumer costs so it becomes less and less of a concern as the driver performance increases.

Here's a white paper from Maxtor's web site that talks about DVR hard disk performance requirements.

http://www.maxtor.com/en/documentation/white_papers/hdd_considerations_white_paper.pdf

The paper is interesting because it shows the tradeoffs of capacity/performance vs heat/noise.

The benefits/risks associated with defragmenting a DVR hard disk probably haven't been worth it to this point. HDTV DVRs introduce some new peformance requirements that would benefit from defragmentation but we'll see.

 

[davhol]

Fragmentation, as we understand it from the MSDOS/MSWindows world is not so much an issue with DVRs. Fragmentation occurs on any file system; it just becomes problematic when, for instance, the filesystem is not optimally designed (i.e. FAT16, FAT32). The filesystem layout on Dish's PVRs doesn't "suffer" from fragmentation problems as do our Windows computers. Yes, there IS fragmentation, but it is not as severe as we see on Microsoft platforms.

 

[AllieVi]

When put in "standby", the drives on my 508's are active for a while and then shut off. I've always assumed that housecleaning was going on at that time, including any defragmentation to optimize disk storage. It might be interesting to check the available recording time just before going into standby and again after powering it on to see if the numbers are the same.

 

[Smith P.]

AllieVi,

Could you make just rough estimation for defrag your PVR disk, let say for half of 80 GB capacity ?
How long your PVR still active after switching in stanby mode ?
Now please compare and tell us about your feeling .

 

[James]

The filesystem layout on Dish's PVRs doesn't "suffer" from fragmentation problems as do our Windows computers.

Whoa, how do you make that assumption?

 

[Smith P.]

Usually ppl working with a product doing such statement - as Microsoft tell us all time .

 

[James]

I doubt the file system of a Dish DVR is any more advanced than FAT32. Such a file system has overhead that a DVR doesn't need. A disk defrag tool and the constant restarts that make a Dish DVR a Dish DVR is all you need.

 

[xgrep]

I doubt the file system of a Dish DVR is any more advanced than FAT32.

The 721 is based on Linux and uses a standard Linux filesystem. Modern UNIX filesystems (since around 1980) don't fragment. Fragmentation is a "feature" of "industry standard" filesystems, not Linux. Check it out.

I must say that I knew Cutler at DEC, and never understood why NTFS (never mind FAT32 that he wasn't responsible for) is so backward compared to any modern UNIX filesystem (ok, it has better error recovery than older ones, but poor performance). He's way smarter than that. Must something in the Redmond water.

x

 

[James]

The 721 is based on Linux and uses a standard Linux filesystem. Modern UNIX filesystems (since around 1980) don't fragment. Fragmentation is a "feature" of "industry standard" filesystems, not Linux. Check it out.

Don't be condensending. I use HP-UX every day to complete my work and I integrate UNIX servers for the U.S. Navy. There is nothing you can say that I don't already know about JFS, xfs, extfs or any other journaled file system technology. I would expect someone who thinks a username of "xgrep" is cool, not to jump on fellow XML user/developers.

We were discussing the 501. We all know that the 721 and TiVo use Linux.

 

[davhol]

The filesystem layout on Dish's PVRs doesn't "suffer" from fragmentation problems as do our Windows computers.

Whoa, how do you make that assumption?

I was going on some technical conversations I had with an Echostar engineer one day. As a Un*x kernel hacker for 25 years myself, I was interested in what the 501 did. He didn't go into details but gave me the impression that the 501 didn't use a micros*ft-like filesystem and thus didn't suffer from it's foibles.

 

[xgrep]

Don't be condensending.

My apologies, you are right, there's never a valid reason for that tone.

We were discussing the 501. We all know that the 721 and TiVo use Linux.

It's true that the topic was originally about the 501, but some replies didn't make that clear, and it was beginning to look like a blanket statement that all DVRs might have a fragmentation problem. Or it was beginning to look like readers without technical knowledge might conclude that from the discussion.

I don't know what the SW platform for the 501 is (proprietary?), and while we believe it's not Linux, it might not be a Windows OS, either. I don't think anyone who has contributed so far has specific, direct knowledge that 5xx DVRs have (or do not have) potential fragmentation. The fact that most computer users believe that all disks have potential fragmentation problems is unfortunate.

x ("grep" is my actual name)

 

[James]

I read the thread again and see how you came to your conclusion. I guess 50x was implied, but not specified. SCO aside, most future PVRs will be based upon Linux and hande fragmentation well.

The fact that most computer users believe that all disks have potential fragmentation problems is unfortunate.

True, but if planned for, any programmer can take that into consideration and run a defrag utility if needed.

x ("grep" is my actual name)

Really? Do you ever use the xgrep command? I assume you already use grep.

 

[Big Bob]

Dang, just when I thought a good flame war was about to break out.

I must say, I so enjoy this forum for its lack of snottyness and pettyness. Unlike some other forums. I hope it stays this way.

 

[mike123abc]

As speculation I would suspect that the block sizes are fairly large, on the order of a minute of record time. Without a need for small files, the file system can be geared for larger chunks of data. This would allow a big block to be read into memory, more than enough time to allow the disk to seek to the next big block.

I would be surprised if the block size of the file system was below 1 megabyte. 4-16 megabyte blocks would probably be ideal.

 

[xgrep]

x ("grep" is my actual name)

Really? Do you ever use the xgrep command? I assume you already use grep.

I mostly use grep. The original xgrep (i.e., before the xml version) has only a few advantages .

"grep" is shortened from my full name of polish origin that has insufficient vowels for adequate pronunciation .