oaicat filesystem and update

Filesystem Structure

I guess I will go for a filesystem populated with many small xml files. Each file contains one record in native format and would be named according to the objId, e.g.

objId1234567.mpx
kueId1234.mpx
mulId12345.mpx

There should be only one record per objId, so probably I do not need several directories although this might get very big directory.

Updates

I guess I could write a little perl script to update the filesystem.

In this situation, there will be a comparison of import (whether one big file or one dir with many files) and existing filesystem (fs). It should do:

• backup current state of fs
• cmp ids of import and fs
• if import_id does not exist in fs, copy it
• copy import record if exportdate is newer in import than fs
• optionally: look for deleted files in import and mark them as deleted in fs

I guess I can write a perl script which does this in a relatively short time. I wonder at what point it would make sense to use a native xml db.