The modio project works at abstracting the storage in a way suitable for other stores than "Unix file based".
Traditionally, squid has implemented its storage directory on top of a UFS or UFS-like filesystem namespace. This allows a high flexibility in what replacement methods can be used, but is far from optimal.
The initial aim of the modio branch was to abstract the storage directory/IO layer to allow individual storage directories to define storage and replacement policies. The squid storage manager should handle object indexing and retrieval, and not replacement policy or IO.
This work has been completed and committed to the main sources and supports the standard UFS filesystem, Stuart Forsters pthreads-based AUFS filesystem, Duane Wessels SysV Shared Memory/Message Queue based DISKD UFS filesystem, and Eric Sterns COSS (Cyclic Object Squid Storage) filesystem. The COSS code is considered highly experimental and unstable, and is not implemented using any asynchronous disk IO techniques.
The next part of the modio project is to push the object storage further back into the storage directories, leaving squid to deal with protocol-specific information, and each storage directory to deal with object storage, indexing, replacement, and memory caching.
Previous research done by various groups (I will chase up URLs later) has shown that "traditional" UNIX filesystem *and* buffering semantics are not suitable for high-throughput caches. Research has also indicated that an on-disk hash for directory lookups might provide a decent memory/disk tradeoff, freeing memory for more object caching and better directory caching.
modio notes will be published here soon.
For more information, please contact Adrian Chadd <adrian@creative.net.au> or the squid-dev <squid-dev@squid-cache.org> mailing list.
