System Requirements:
- Windows NT 3.51, 4.0, 2000, XP, 2003
The Problem:
Yes, I admit it, I have hard disks running in my current servers which have been with me almost as long as I have – that either says something about how old I am, or how tragic I am; probably the latter actually.
Ever since I got fed up with it all back in 1999 and had a 9x cull, everything – and I mean everything on my home network has run some form of NT or another, be it 3.51, 4.0, 2000, XP or 2003; even Longhorn (I’m still not calling it “that”).
Having suffered some major server related setbacks in the last week, the short version of a very long and depressing story has been that I’ve needed to reinstall a large number of servers, including two here at home. One of them is a nice cost SCSI320 RAID array that has just been rebuild, but server 3, with its numerous ATA disks, that’s a different story.
Thinking long and hard about it, I can conclude that at least two of the disks would have originally started life as FAT Volumes, and has been slave drives under 98, Millennium, NT4, 2000 Professional, XP Professional, 2000 server and now 2003 server. In between there have been numerous file system changes, back and forward from one to the other through the native conversion tools, or by the modern marvel that is Partition Magic.
The point is, the volumes have had a lot of work done on them, many partition dimension changes, many file system changes and last and not least, many operating system changes. So when it came down to a little impromptu data reorganisation when server 3 was retired from Intranet duties, imagine my surprise when Windows was reporting a completely empty 60GB NTFS partition as using well over 3GB of disk space.
All file systems will have some degree of overhead involved in storing their configuration information, NTFS partitions in particular will always have a certain overhead reserved for the on-disk MFT store. However on a 60GB partition this should be around 95.7MB (100,396,572 bytes) depending upon cluster size. So where has the data gone?
The Fix:
Tragically, this little delight is down to the way in which Windows tracks the recycle bin index for any given volume and every given user account on the system. Every user on the computer is assigned and identified by a GUID value by the SAM when the operating system is installed and when the user account is created. Such a GUID looks like:
Note the ending value of 500, which means this account is almost certainly the system Administrator account in this example.
For each GUID on the system a MFT wrapper is created under the super hidden folder <drive>:\RECYCLER which contains the file system entries for everything that user holds in the recycle bin.
The meaning for this is clear, Windows maintains a per-user recycle bin schema, not a per system. Therefore each user account can populate the recycle bin with whatever information that user deletes, and it remain hidden on the disk.
The trouble is when a volume gets old, the GUID’s easily become stale as operating systems are reinstalled, Active Directory is added and removed, disk’s are moved between systems and the file systems themselves are altered.
In the case of this particular volume, well over 3GB of data was trapped in the recycle bins of a rather large number of long dead user accounts, which unintelligently Windows and successive operating system reinstallations has never picked up upon.
There are two ways to deal with this, the first of which I am not going to go into much detail on.
- You can wipe out stray GUID entries from the RECYCLER folders by seizing ownership of the RECYCLER folder and literally deleting the wanted GUID’s. Remember though, the system will let you delete the information from any user account that is currently not loaded. You can check the GUID’s off against the HKU hive in the registry to be certain if they exist.
- The second way, and the way that I am advocating is to pull all the data off of the volume and perform a format using Disk Manager. The logic is very simple, not only does it wipe out everything from the disk, and perform a thorough bad sector check/retest & repair, it also will upgrade and reconstruct the NTFS / MFT on the volume.
The volumes in question in my example were all NTFS from NT 4.0, where as the standard has been updated through 2000, XP and 2003 with additional features such as Shadow Copy support and native OS level Indexing service support. I therefore am at pains to imagine what sort of state the MFT on the volume was in with all the in-place bolt-on’s added by successive Windows releases, not to mention MFT file fragmentation. Formatting the volume will therefore naturally clean the entire nightmare up quite eloquently.