NoSQL - RavenDB on Linux

For a while now I've been on a quest to find a NoSQL database that met the following criteria:

1. Document Oriented Database
2. Supports Transactions across multiple Documents
3. .Net/Mono compatible drivers
4. Runs under Linux

And the journey has been anything but easy.

Ever since I first read about Document Oriented NoSQL Databases, I've been fasinated by them. I started looking into CouchDB [apache.org] at first, but MongoDB [mongodb.org] soon became my favorite.

MongoDB is a Document Oriented Database that is written in C++ with performance in mind. MongoDB allowed my to write applications using my favorite languages and ran on many Operating Systems. After doing some research, I also found a port of MongoDB for ARM. It now runs on a Debian server that I have setup on PogoPlug which runs on an arm chipset. The JSON style BSON representation of data was simple to follow and the ability store any object in MongoDB without having to implement special interfaces or inheirit from special classes made it that much more appealing to me. The .Net drivers were readily available on their website along with plenty of documentation on how to use them.

MongoDB met three out of the three criteria I had above. The was one that it didn't satisfy: MongoDB doesn't yet support Transactions [mongodb.org] and there is no word out yet that would promise availability of transactions any time soon.

I continued my research and came across RavenDB [ravendb.net]. RavenDB is a Document Oriented Database that is written completely in .Net. It is also Mono compatible. Unlike MongoDB, RavenDB does support transactions across multiple documents, however, it only runs under Windows. Though evn RavenDB doesn't meet all the four criteria, since it was written completely in .Net, that gives me something to tinker around and see if it could be made to run under Linux.

My first attempt was a few months ago when I first read about RavenDB. I downloaded the code from their GitHub git repository and fired up MonoDevelop [monodevelop.com]. The issues became immediately apparent: heavy reliance on Silverlight. But even then, there was .Net 4.0 code that my mono compiler could not make sense of. After spending few minutes over it, I gave up on it.

Few days ago, with the update of Ubuntu, I got the near latest version of Mono installed on my laptop. I decided to give RavenDB another try.

From my konsole, I issue the command to get the latest code for RavenDB and created a new branch - linux.

git clone git://github.com/ravendb/ravendb.git

cd ravendb

git branch linux

git checkout linux

As soon as I fired up MonoDevelop and loaded the solution, I was greeted with this error message:

Also, the projects Raven.Backup and Raven.Smuggler were not set to build under my current configuration. I noticed that upon Right Click -> Options -> Build -> Configuration on the Raven.Backup project, the only configurations that were available were for x86. I was able to select Debug, click Copy button and create another configuration for All CPU. I repeated this for Release and then the same steps for the other project as well.

Then from the Right Click -> Options -> Build -> Configurations -> Configuration Mapping tab, I selected the correct configuration for the two projects. now the projects were no longer marked as "unable to build under current configuration." Now, on to the issue with the three projects failing to load: Raven.Client.Silverlight, Raven.Studio, Raven.Tests.Silverlight.

While anything Silverlight under Linux (using Moonlight) was not a promising prospect to begin with, I still wanted to give it a shot before beginning the removal of non-compilable projects. I opened the project files in plain text format and searched for the GUIDs from the screen print above. On finding them, I commented them and attempted to reload the projects. The solution was now ready for the first compile.

First attempt:

95 build errors. I also noticed that at least a few of those errors were really warnings. This meant one thing - some or all the projects were setup to treat warnings as errors. That was an easy fix. On all the projects, I unchecked the "Treat warnings as errors" box from Right click -> Options -> Build -> Compiler screen. Time for the second compile attempt.

Second attempt:

100 build errors. One of the first errors was in the Raven.Client.Debug project on classes from the Microsoft.VisualStudio.DebuggerVisualizers namespace. Dependency on Visual Studio would be a problem under Linux. I decided to not dwell too much on this error and chose to remove this project all together from the solution. All the dependencies on Raven.Client.Debug would also have to be removed.

Third attempt:

95 build errors. This time it was the NLog namespace under the Raven.Tryouts project. Reviewing the error revealed that NLog dll was not compatible with the current Mono runtime. Fortunately, NLog had a Mono compatible binary available for download on Codeplex. I downloaded it and replaced the reference in the project.

Forth attempt:

94 build errors. Upon closer inspection, I observed that Raven.Web relied on System.Web.Entity which is not available under Mono. Also, Raven.Client.Silverlight System.Windows and System.Windows.Browser which are also not included with Mono. I decided to remove these two project from compilation along with Raven.Tests.Silverlight project. Raven.Studio project was the next one to go since it was a WPF application - lots of XAML files - and WPF is not fully implemented under Mono.

Fifth attempt:

0 build errors. Are we there yet? Let's give it a try. I ran the Raven.Server project. Bummer! Runtime exception. DllNotFoundException. Turns out that Raven.Storage.Esent project implements Microsoft's ISAM Esent storage which is proprietary and requires Windows in order to run. Since we also had Raven.Storage.Managed project, I decided to modify the configuration of the application to use the managed storage library Munin instead of UnManaged Esent. I modified the App.Config under Raven.Server and changed the value for "Raven/StorageEngine" to "Munin" from "Esent".

Sixth attempt:

0 build errors (expected). Yet another runtime error however. This time it was a MissingManifestResourceException with a very vague description and a long stack trace. However, right before the stack trace took me into obscurity, there was a hint - ravendb/Raven.Database/Server/HttpServer.cs:110. The SatisfyImportsOnce call was failing after some execution. After doing some reading into it, I found out that RavenDB was build on MEF which is a feature of .Net 4.0 and is supposed to provide easy plugin/extension functionality to .Net programs. That line was supposed to initialize the extensions once. Somewhere in the code below there would be a code block that would expect non null values. I commented that line and proceeded.

Seventh attempt:

0 build errors. As expected, NullReferenceException on Raven.Database/Server/HttpServer.cs:112. RequestResponders was supposed to be non-null. I did some more research to understand how MEF was supposed to implement the extensibility. I had to find classes that were inheirited from AbstractRequestResponder. May be manually loading objects into RequestResponders would help. And indeed I found plenty of classes that were inheirited from AbstractRequestResponder and RequestResponder classes. I added the following lines of code in HttpServer.cs

Eight attempt:

0 build errors. Another exception, this time NullReferenceException at Server/HttpServer.cs:193. This seemed similar to the previous issue but with a different object - ConfigureHttpListeners. Time to repeat the exersice with different base class - interface this time.

Ninth attempt:

0 build errors. I started seeing some output repeating and they didn't look like error messages:

Available commands: cls, reset, gc, q

Could not understand:

I decided to run the compiled application from konsole to see how it would behave there.

Raven is ready to process requests. Build 13, Version 1.0.0.0 / abcdef0

Server started in 847 ms

Data directory: /home/karim/Projects/RavenDB/ravendb/Raven.Server/bin/Debug/Data

HostName: <any> Port: 8080, Storage: Munin

Server Url: http://karim-laptop:8080/

Available commands: cls, reset, gc, q

Time to run some tests. I created a small console .Net application and decided to fire it up. It worked! Some issues did surface, but nonetheless, it worked. I was able to store and retrieve simple documents to the database. Since I had removed some of the critical projects such as Raven.Web and Raven.Studio, the web interface was gone with them. Dynamic indices did not work either.

Now that I have a semi functional database, I'll put some time in and try to create a simple Raven Studio replacement as well as work the other kinks out.

Overall, it helps that the application was written completely in .Net without any dependencies on the OS that could not be achieved via framework level abstraction. I am certain that RavenDB could be made to work under Linux at similar level of confidence as it does under Windows.

This article is part of the series NoSQL - RavenDB on Linux. The series contains the following articles:
NoSQL - RavenDB on Linux
Open Source Shines - RavenDB on Linux
RavenDB on Linux - Source Code
RavenDB on Linux - Update