Every now and then I check the link reports for Project FiFo to see what people think and write about it. Recently I stumbled about an article that oddly enough made me both proud and sad. It actually was a rather negative one, which is a shame, but on the other hand a project isn't mature until people care enough to complain.
Yet even so it would be very easy to cast this aside as a ‘success’ in a strange manner, it still bothers me that someone is upset enough with FiFo to spend his time writing a longish blog article and write their own management software.
Read more →With 0.4.3 FiFo introduces support for LeoFS and this allows for some quite nice new features. Most importantly it decouples FiFo's operations from storing big amounts of data which makes maintaining either of this much more sensible and scaling storage much more easy.
Then again while nice that is not the important part, just storing datasets somewhere else does not make much of a difference for most users but what LoeFS allows FiFo to store much more data then would be good in the old setup.
Read more →Following the article about Asynchronous garbage collection with CRDTs I experimented with implementing the concept. The OR Set is a very nice data structure for this since it's rather simple and so is it's garbage!
To garbage collect the OR Set we do the following, we take some of the elements of the remove set, and delete them from both the add and the remove set - this way we save the space for them and generate a new baseline.
Read more →So CRDTs are very very nice data structures awesome for eventual consistent applications like riak, or the components of Project-FiFo. So they have one big drawback, most of them collect garbage, and over time that can sum up to a lot making them pretty unpractical in many cases. Collecting this garbage is a bit tricky, since usually it means synchronising the data - which going back to the eventual consistent stuff prevents either A or P.
Read more →Some might know it, some might not and some might not care but for what it's worth I'm the author of Project-FiFo (or most of it) and today is Project-FiFo's first birthday (since the domain registration) and I want to take this chance to look back to the past year and reflect, say thank you to all of you and take a look in the future.
When I started Project FiFo a year ago it was more of a tiny hobby project and I could have sworn it would stand in row with all the other little open source projects no one would ever give a damn about.
Read more →The reason The question ‘why would I want at least 5 nodes’ comes up very often in the #riak IRC channel, there is a good explanation. But that's boring, no one likes reading manuals, we, as engineers, like to try things out (aka. break stuff).
Only downside with that is that we need to set things up before we can break them, or even worst need to un-break it later to try out different things (aka.
Read more →As promised in a previous post I'll talk a bit about writing tests for riak_test. To start with the obvious it's pretty simple and pretty awesome. riak_test gives you the tools you've dreamed of when testing distributed riak_core applications:
a backchannel to communicate and execute commands on the nodes. a nice and way to bring up and tear down the test environment. helper functions to deal with the riak_core cluster.
Read more →If you don't know what riak_core is, or don't have a riak_core based application you'll probably not take too much practical use of this posts you might want to start with Ryan Zezeski's “working” blog try try try and the rebar plugin.
That said if you have a riak_core app this posts should get you started on how to test it with riak_test. We'll not go through the topic of how to write the tests itself, this might come in a later post also the riak_kv tests are a good point to start.
Read more →Preamble Lets start with this, Erlang releases are super useful they are one of the features I like most about Erlang - you get out an entirely self contained package you can deploy and forget, no library trouble, no wrong version of the VM no trouble at all.
BUT (this had to come didn't it) sometimes they are limiting and kind of inflexible, adding a tiny little feature means rolling out a new release, with automated builds that is not so bad but ‘not so bad’ isn't good either.
Read more →I've had some open source projects before, actually quite some, but Project FiFo is by far the most successful one. And aside from the technical perspective I've learned a tremendous amount of new things already, one of which I want so share since. It sounds simple but I never looked at it this way before: Tickets.
The project has caught on momentum so fast that at ‘rush hours’ people come with questions, bug reports and feature requests at rate that it's they pile up faster then we can help or answer, the channel and community already does a great job ‘filtering’ out easy to answer topics but enough are complicated to a point where a developer has to look at them.
Read more →