RavenDB - An Introduction

using(var documentStore = new DocumentStore() { Url = "http://localhost:8080"}) {

using (var documentStore = new DocumentStore() { DataDirectory = "Data" }) {

   public class BasicEntity
    {
        public string Id
        {
            get;
            set;
        }

        public string SomeData
        {
            get;
            set;
        }

 public string SomeOtherData
 {
            get;
            set;
 }
    }

BasicEntity entity = new BasicEntity()
{
    SomeData = "Hello World",
    SomeOtherData = "This is just another property",
};
documentSession.Store(entity);

documentSession.SaveChanges();

Querying for Documents
An important feature of anything we store data in, is the ability to query the store for 'views' or indeed the actual entities themselves.

We've gotten used to in NHibernate and other ORMs simply executing ad-hoc queries against the database, and while you can do that with some document databases, that's not really what RavenDb is designed for.

In order to query documents in RavenDB it is necessary to create an index across the properties of the documents you wish to query. This is done up front and exists in the database. A few things of note:

1) Documents are processed against those indexes when they are added (eventually)
2) Queries taking place against those indexes are therefore cheap (relative to say, doing an ad-hoc query)

In my application, I create all of my indexes up front as part of my 'database creation script' (actually, they're the only part of my database creation script, because there is so little setup involved. There is nothing to stop you doing it at any point when the application is running though.

Indexes exist as Linq queries against the documents in the store, and are either be defined as the strings that will be sent up to the server and stored as they are written, or defined as linq queries that will be converted *into* strings and stored on the server.

The downside to using the actual linq queries, is that the indexes on the server won't look exactly as you wrote them, but the upside is you get type safety and intellisense. I'm currently choosing to use the strongly typed linq queries because I don't mind how they look on the server, just so long as they work. I write tests for all of my indexes and queries so I know they're cool.

The recommended practise is still currently to define your indexes separately to the application, as strings in the Web UI.

There are two major components of each index, the "Map" query and the "Reduce" query. I'll not go into detail on what this means, because once again you can get this information across the internet, and Oren has written a very good visual explanation of what Map/Reduce looks like in Raven here:

In order to do a query, we need at the very least to create a Map telling Raven which fields we want indexing for our queries. This means we only index the fields that are relevant to our query and keep things small and (hopefully) more efficient.

Indexes are created against the document store (not the session), and the syntax for that looks something like this:

documentStore.DatabaseCommands.PutIndex(
    "BasicEntityBySomeData",
    new IndexDefinition<BasicEntity, BasicEntity>()
    {
        Map = docs => from doc in docs
                        where doc.SomeData != null
                        select new
                        {
                            SomeData = doc.SomeData
                        },
           });

* "BasicEntityBySomeData" is the unique identifier of the index we have created
* The linq query is ran against "docs" which is (effectively) a collection of all the documents in the database (not just BasicEntities).
* SomeData is now a field that is being indexed with the name SomeData

There is nothing to stop you from indexing every field of the document in a single index and just using that index across all of your queries, but that would probably be unwise. The whole point of creating a map is you are limiting the data you are indexing to just the data you want to search on.

You can write *almost* any code you want in the linq statement, as on the server it will be converted into a proper linq query and executed as a function across the documents.

To use this Index, we invoke the Query method on the DocumentSession specifying the name of the index we wish to use and a Where clause (which is effectively a Lucene query) against that index.

BasicEntity[] documents = documentSession.LuceneQuery<BasicEntity>("BasicEntityBySomeData")
    .Where("SomeData:Hello~")
    .WaitForNonStaleResults()
    .ToArray();

This will return a collection of documents where "SomeData" contains some text that looks like "Hello". Clearly there is some scope here for yet more strongly typed usefulness, but once again I have tests for all of my queries so it's not presenting a problem in this area.

The WaitForNonStaleResults call means that the call will wait a (default) amount of time for the documents to finish indexing before returning data (or timeout). The use of kind of call should be thought about carefully, as the whole point of the document database is that it's "eventually consistent", and you don't always *need* the most up to date result possible. (For example: Displaying a list of documents on the front page of your website)

Retrieving only the data you need
This is all very well and good, but because you're storing entire documents and sometimes you only want small portions of those documents, and this is of course possible too.

I've defined a simple projection of the BasicEntity containing a single property like so:

    public class SomeDataProjection
    {
        public string SomeData
        {
            get;
            set;
        }
    }

In order to get the value from the index rather than fetching the entire document from the store, we need to modify the index slightly:

documentStore.DatabaseCommands.PutIndex(
    "BasicEntityBySomeData",
    new IndexDefinition<BasicEntity, BasicEntity>()
    {
        Map = docs => from doc in docs
                        where doc.SomeData != null
                        select new
                        {
                            SomeData = doc.SomeData
                        },
        Stores = { { x => x.SomeData, FieldStorage.Yes } }
    });

We are then telling RavenDb to store the value in the index so it can be easily retrieved using the following query.

SomeDataProjection[] projections = documentSession.LuceneQuery<BasicEntity>("BasicEntityBySomeData")
    .WaitForNonStaleResults()
    .SelectFields<SomeDataProjection>("SomeData")
    .ToArray();

This will mean only the data you want is transmitted across the wire and make the query much more performant. Note: You can only pull back fields that have been stored using the Store facility on the index.

Reporting on your data
So that's round tripping  to and from the data store, but in the real world you soon need to be able to perform more complex queries across your data.

This is a contrived example, but how about summing up the total lengths of all the strings stored in all the SomeData properties across the document store?

Let's add a new property to the Entity called "Category" so we can get all the lengths by category:

  public class BasicEntity
    {
        public string Id
        {
            get;
            set;
        }

        public string Category
        {
            get;
            set;
        }

        public string SomeData
        {
            get;
            set;
        }
    }

And let's add a load of entities to the document store like thus:

documentSession.Store(new BasicEntity()
{
    Id = "Document1",
    Category = "One",
    SomeData = "Text"
});
documentSession.Store(new BasicEntity()
{
    Id = "Document2",
    Category = "Two",
    SomeData = "More text"
});
documentSession.Store(new BasicEntity()
{
    Id = "Document3",
    Category = "One",
    SomeData = "And more"
});
documentSession.SaveChanges();

What we want to do, is index the *length* of the strings stored in the document, and index the category, before reducing the query across category to get the total lengths. If you don't understand what I mean by that, then go and read about Map/Reduce on Oren's blog linked above!

documentStore.DatabaseCommands.PutIndex(
"BasicEntityCountSomeDataLengthByCategory",
new IndexDefinition<BasicEntity, CategoryDataCountResult>()
{
    Map = docs => from doc in docs where doc.SomeData != null
                            select new
                            {
                                Category = doc.Category,
                                SomeDataLength = doc.SomeData.Length
                            },
    
    Reduce =  results => from result in results 
                group result by result.Category into g
                select new
                {
                    Category = g.Key,
                    SomeDataLength = g.Sum(x => x.SomeDataLength)
                }
});

It's as simple as that, this now means I can execute the query:

CategoryDataCountResult[] counts = documentSession.LuceneQuery<BasicEntity>("BasicEntityCountSomeDataLengthByCategory")
    .WaitForNonStaleResults()
    .SelectFields<CategoryDataCountResult>("SomeDataLength", "Category")
    .ToArray();

And that will give me the results as expected:

One: 12
Two: 9

The beautiful thing about this, is that was pretty much calculated when the documents were added, reading the data out was a really cheap operation - think about the cost of doing this in T-SQL :)

Summary
This was quite a lengthy blog post to cover some of the simple features of RavenDB, I'll start getting into more detail in my next post where I'll cover some more complicated reporting queries/indexes and talk about the excellent Web interface that Oren has created part of this project.

My two cents
My experiences with the project so far have been mostly positive, I've found a few issues but Oren has been *very* fast to fix them and publish fixes to Github (and these have as he says, been "edge cases") which most people won't come into contact with when playing with RavenDB.

It's definitely worth a gander, I see this project taking off in the .NET space as it matures.