CBlockFile was developed to keep many small files in one file, a little like a .tar or .zip file or a database holding many records.
Hard Disks are formatted with fixed Block sizes. One File may occupy many Blocks, but one Block can only hold the data for one File.
I had a Server with 4kB Block size which meant that every file that was just a few Bytes long would take up 4kB of Disk space...
The Server was supposed to be used with a few large files but ended up having thousands of tiny files for a particular application (ESSI files for a CNC Profiler).
I created an application which tokenised and compressed the ESSI files and stored them all in a single BlockFile.
That File currently holds more than 7000 small files and gets Defragmented and backed up at midnight every night.
It's been running now (with Uninterruptible Power Supply) for four years without a problem.
CBlockFile was modeled on the malloc and free routines used in the RAM Memory Allocation routines on all Operating Systems.
Instead of RAM, though, CBlockFile allocates a block in a Disk File (Code is included to make it operate on RAM for debugging).
This implementation uses a struct FileInterface to hold all the file handling routines and uses the standard Low-Level routines defined in io.h
MFC users can change this to use CFile if they prefer.
If any of the boolean routines that aren't called Is...() return false, you must assume that the File is corrupt.
When a Block is Freed, the class will make one large Free Block from adjacent Free Blocks where possible.
If a Freed Block is at the end of the File, the File is shortened.
The Defragment() function will remove all Free blocks from a file.
Defragment when the Application opens the File:
CopyFile("MyFile.bf", "MyFile.bak", FALSE);
if(!BlockFile.Open("MyFile.bf")) return false;
Say("Defragmenting");
if(!BlockFile.Defragment()) {
BlockFile.Close();
CopyFile("MyFile.bak", "MyFile.bf", FALSE);
if(!BlockFile.Open("MyFile.bf")) return false;
}
Say("Ready...");
This looks a little paranoid, but when you have many files within one file you have cause to be paranoid!
This code has been thoroughly tested using a "Thrasher" which randomly calls random functions with random parameters...
The code was considered finished when the Thrasher ran through 20,000 calls to its functions without failure.
The following is true for all files, but this is a good place to make sure you know:
It only takes one bit to flip in the File and you could loose all the data it held.
For this reason it is recommended that the File is closed as soon as possible to minimise the chance of power failure while the File is open (which is the most likely cause of corruption).
If you leave the File open and your Application hangs or the power fails you may end up with corrupt data.
BlockFileThrasher.zip is an MFC project used to test and prove the reliability of CBlockFile.
THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS 'AS IS'
AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE DISCLAIMED.
IN NO EVENT SHALL THE REGENTS OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES
(INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION)
HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE)
ARISING IN ANY WAY OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
