A possible [tclvfs] - [CMcC] 20041029

Storage of metadata (data about data) is an important and undersupported facility of file systems.

Tcl supports the [[[file] attributes] command to store some system-defined metadata, and [tclvfs] permits this to be redefined and extended.

What this allows is [vfs] which intercepts the [[file attributes]] command and loads/stores attribute (name,value) pairs in some parallel store.  Such a  vfs would be stackable over other filesystems, so as to provide generic metadata for any filesystem.  This would allow applications to simply assume that they could store arbitrary metadata.

The question is what form of parallel store would be best?

   * '''parallel hierarchy''' - metadata is stored in a file hierarchy which parallels the structure of the original, so there's a mapping from ${path} -> ${path}.metadata which contains an array dump of name->value pairs.  ''CONS'': expensive/slow, ''PROS'': persistence for free.

   * '''hidden directory''' - metadata stored in special files hidden in a per-directory ''.metadata'' directory.  ''CONS'': expensive/slow, invasive (less so than invasive-interpolation, below), permissions problems, ''PROS'': faster than parallel-hierarchy, metadata is joined with with data (less so than invasive-interpolation)

   * '''persistent array''' - metadata is stored in an array $metadata($path) as an [[array get]] form per file, to be loaded/stored once, then accessed from memory.  Could use [tie] for persistence.  ''CONS'': doesn't scale well, persistence needs work ''PROS'': fast.

   * '''metakit''' - metadata stored in a metakit file, loaded/stored as required.  ''CONS'': not pure tcl, slower than persistent-array, ''PROS'': scales, faster than parallel-FS.

   * '''invasive interpolation''' - metadata stored at the head of each file in (say) RFC822-style name:value lines.  Similar to Mac resource forks. ''PROS'': some data (e.g. emails) are already of this form, some data (e.g. caches) can be coerced to this form ''CONS'': invasive, wrecks general files for general (non-tcl) use.

   * (add more here)

----
[[[Category VFS]]]