lice! : Stupid DB

Stupid DB

No, no rant here. Just a little update. The other day I pushed version 0.2.0 of stupiddb a very simple database for clojure. The main goal was it to make a database that is very easy to add to a project and very easy to maintain.

Often I saw code like this:

(with-open [w (io/writer "some-file")]
  (binding [*out* w]
    (pr my-map-with-data)))

That is called either at the end of the program or after every change of the map. Neither are god since it either can lead to data loss when the program isn’t terminated properly or it causes a lot of file IO when every change is written.

Now one could go for some kind of DB, MySQL, haddop, mongodb. All work great with clojure but at times it isn’t really what people want. It means that you’ve to deploy another kind of software and some drivers for the database.

So I figured to make a stupidly simple solution. A minimal database that abstracts the code earlier in a nice way. So stupid db has 6 db commands and 2 ‘maintenance’ commands to open and close the database.

The commands are simple:

  • (db-get [db key]) – gets a value from a key.
  • (db-get-in [db [& k]]) – gets a key from nested data in a db.
  • (db-assoc [db k v]) – associates a value with a key.
  • (db-dissoc [db k]) – dissociates a key with a value.
  • (db-assoc-in [db [& ks] v]) – associates a key within a nested db.
  • (db-update-in [db [& ks] f]) – updates a nested value in the db.

What happens in the background? Stupidb store the data in a map that is hold in a ref to keep data up to date. It attempts to give a good compromise out of IO performance and data security.Every time a db-* function called a log is written, it is written directly and the file so even if the program crashes every change will be logged.

Every n seconds the entire db is written and the log emptied, if the db-init function is passed :gzip true the db file is written is gzipped to save space and increase IO performance.

The db can be closed to make the auto save stop and force a db write, but it is not necessary since every action is logged and when the db is opened while there is a non empty the DB file will be loaded and the logs are ‘replayed’ to get the DB back to the latest stand.

That said stupiddb isn’t a replacement for a real database if big data amounts are handled but if it is just to save some data in a client side it is likely better then reinventing the wheel :).

Have fun!

Posted by Heinz N. 'Licenser' Gies Fri, 14 May 2010 08:48:00 GMT


Trackbacks

Use the following link to trackback from your own site:
http://blog.licenser.net/trackbacks?article_id=69

Comments

  1. Avatar
    murphy about 14 hours later:
    Does it automatically support transactions because of Clojure's concurrency model?
  2. Avatar
    Licenser 2 days later:
    Yes, as silly as it sounds even the logs should be transaction save, since even when the transaction has to be redone the latest update wins. So I can't prove it since I'm not 100% suer how java IO works but it should be fine in most cases - I hope.
  3. Avatar
    Licenser 3 days later:
    I did some research and there is an easy way to make it entirely concurrency proof, which will be implemented in the next version.
  4. Avatar
    murphy 3 days later:
    Awesome! I think this should be highlighted as a main feature.