Comments for post there is something between append only and reusing blocks
Daniel Waterworth writes: Hi, I wondered if you'd be interested in aodbm ( sf.net/projects/aodbm/ ). aodbm is an append only database manager that uses an immutable B+Tree internally.
forchenyun writes: I think you could consider the implement of Berkeley DB Java Edition.
The compaction would never be the bottleneck because data files are separated every 10M.
Didier writes: Hi again - RCU is to optimize locking in memory. In a disk-based database, people will rather call it MVCC (even if they are slight different - see this post from Paul E. McKenney http://www.mail-archive.com/kernelnewbies@nl.linux.org/msg06575.html ). But IMO, this is off-topic here. Redis is mostly single-threaded. Even if the I/Os will be done in a separate thread (or maybe several of them), I doubt concurrency in memory will be a bottleneck.
Anonymous writes: Seconding the recommendation for RCU. That allows readers to mark their presence in an incredibly lightweight way, and have writers put off reclamation until all readers have finished with the data.
Chris writes: In CouchDB we have a theoretical problem with compaction not keeping up with new writes and therefor never finishing. We've never seen it in real life. The real issue tends to be when people don't have enough free disk space to compact. For steady state performance and to avoid surprises, I know some users who are always compacting.
Pierre Habouzit writes: 60 seconds may not be enough if you're really stuck.
What you want is RCU.
Didier writes: Hi - maybe random/sequential I/Os should be considered as well. If you isolate an append-only file on a disk, you can guarantee only sequential I/Os will be done. On the other hand, if you rewrite blocks in place, you will likely perform random I/Os, which are much slower on non-SSD hardware. Regards, Didier.
home