• More NoSql stuff :: Couchbase from C#

    Published by on February 6th, 2013 6:33 pm under c#, Couchbase, DB, Distributed databases, MapReduce, NoSQL, ORM

    1 Comment

    Continuing with my incursion in the NoSql database world, these days I’ve been playing with Couchbase installed on Windows 8 (after tackling several issues). Finally, after downloaded and installed it successfully, I’ve created a simple console application in C# to test its basis (as I did in my previous articles ‘Using MongoDB from C#‘ and ‘A little of RavenDB‘ a few days ago).

    This turns to be a Document Oriented store working as a key-value DB when interacting with its HTTP RESTful API. It stores JSON documents, and support sharding, replication, and many other things that we can find these days in most NoSql databases. You can go to the Couchbase home site and learn more about it.

    As I said before, in order to try some of its features, I’ve created a console application (the client driver is very simple to add to your solution – just using NuGet).

    First at all you need to get a configured client instance before interact with the DB. So let’s start by downloading the client NuGet package into your project, and add a new section within your .config file for Couchbase:

    <configSections>
       <section name="couchbase" type="Couchbase.Configuration.CouchbaseClientSection, Couchbase"/>
    </configSections>
    
    <couchbase>
       <servers bucket="default" bucketPassword="">
          <add uri="http://127.0.0.1:8091/pools/default"/>
       </servers>
    </couchbase>
    

    Now you can get a client instance from your app just this easy:

    var client = new CouchbaseClient();
    

    Let’s perform some simple CRUD operations like adding and getting a document. Having the following Book class defined:

    [Serializable]
    public class Book
    {
     public string Title { get; set; }
     public string[] Authors { get; set; }
     public decimal Price { get; set; }
     public bool Recommended { get; set; }
    }
    

    How do we create and add a new book to the store? Simple, just like the following:

    var bookId = "f1637bb1-cd16-40bd-9ae7-e267d58ff62f";
    client.StoreJson(StoreMode.Add, bookId, new Book
    {
       Title = "For Whom the Bell Tolls",
       Authors = new string[] { "Ernest Hemingway" },
       Recommended = true,
       Price = 15.64M
    });
    

    In this example I have chosen the key to be a Guid, but you can use any string-set you want to define your own keys domain. I have also used a StoreJson() method that is not present at the CouchbaseClient class interface; well you can take advantage of it by using the Couchbase.Extensions namespace static classes containing some extension methods for this class.

    And what’s the code to retrieve? Some kind of GetJson()? Exactly! Actually a GetJson<T>() extension method:

    var book = client.GetJson<Book>(bookId);
    

    Note: the CouchbaseClient class contains simple Store() and Get<T>() methods, so why I didn’t use them before? Well, if you use these methods the client will store the document instance as binary within a JSON document in the server. Having a JSON readable document stored in the server will be helpful to perform server-side operations over them (basically MapReduce operations).

    But now, how you can retrieve or perform a custom operation, and get its results, over a set of documents? Well, as you may guessed, the answer here (as in many other NoSql databases) is to recall the MapReduce concept I’ve mentioned before. In Couchbase you have a concept that is very similar to relational DB’s Views to resolve this kind of things, and as happens in a RDBMS, it allows you to obtain a subset or full set of documents from the store (besides performing aggregation operations). Views in Couchbase iterate over a specific bucket, and they can be expensive to build if the dataset you selected is big, but once created they are automatically updated whenever data changes (having low performance impact). To store the views this DB has a design document concept (as we can find it in CouchDB – where you can store your views within a design document or run temporary views, resulting in performance degradation as temporary views execute when you sent them to the server – refer to CouchDB technical documentation for more info).

    To create a new view you can go to your administrative console at http://localhost:8091/ and click in the Views tab to create a new design document and add some views to it, but I’ll show you how you can achieve this from your client app later.

    Let’s first create a view that will be useful to search using book’s titles as I did on previous articles, a BookByTitle view/index. I’ve created a JavascriptFunctions class within my application to contain (as strings) the Javascript functions I need for the MapReduce operations, although you can store them whenever you want. Here it is, containing the mapping function for getting books by title:

    public static class JavascriptFunctions
    {
     public static string Dev_Books_ByTitle_Map
     {
       get
       {
          return @"function (doc, meta) {
             if (doc.title && doc.authors) {
                emit(doc.title, null);
             }
          }";
       }
     }
    }
    

    As you can see within the function I’m checking for documents that contain a title and authors, as the documents within the bucket can be of any kind and the map function will receive all the documents in the bucket (at least if they contain title and authors there is a big chance that they are books – you can split your documents within different buckets or choose to have a kind of ‘document type‘ identifier).

    If you add this view to the server you can use it from the client to get a range of books by title for example, as shown below:

    var booksByTitle = client.GetView<Book>("dev_books", "bytitle", true);
    foreach (var book in booksByTitle.StartKey("S"))
    {
       Console.WriteLine(string.Format("* Book retrieved: {0}", book));
    }
    

    Using the GetView<T>() method you can instruct the server that you want to get the results from the view (ie. the map or reduce functions results), but setting the last method parameter (shouldLookupDocById) to true you will get the documents either. And there I’m showing the books which titles start with ‘S’ until the end of the collection (remember that you have an index built below).

    But, what if I want now to count books by authors as I did in previous articles? I’ll just need to add another view that maps the authors names with their book count from each document, and reduce these results grouping by the key (the author in these case).. as with other NoSql DBs I worked with, obtaining this result is a piece of cake. So let’s add two new ‘functions’ to my JavascriptFunctions class that will map and reduce authors with their books count:

    public static class JavascriptFunctions
    {
     public static string Dev_Books_ByTitle_Map
     {
       get
       {
          return @"function (doc, meta) {
             if (doc.title && doc.authors) {
                emit(doc.title, null);
             }
          }";
       }
     }
    
     public static string Dev_Books_AuthorWithBookCount_Map
     {
       get
       {
          return @"function (doc, meta) {
             if (doc.authors && doc.recommended) {
                for (var i in doc.authors) {
                   emit(doc.authors[i], 1);
                }
             }
          }";
       }
     }
    
     public static string Dev_Books_AuthorWithBookCount_Reduce
     {
       get
       {
          return @"function (keys, values, rereduce) {
             if (!rereduce) {
                return sum(values);
             }
          }";
       }
     }
    }
    

    Now I have two new Javascript functions defined at this class: Dev_Books_AuthorWithBookCount (map & reduce versions). If you pay attention to the Map version, you can see that I’m only interested in books that are ‘recommended‘ (an again, to know that they are books I’m querying its authors property). From the client app you can get its results in the following way:

    var countPerAuthor = client.GetView("dev_books", "authorwithbookcount").Group(true);
    foreach (var row in countPerAuthor)
    {
       Console.WriteLine("* Author {0} has {1} books", row.Info["key"], row.Info["value"]);
    }
    

    Please note that I’m using Group() method from the IView interface there to indicate the server that the reduce function should operate on each one of the keys. You can get more information here (and see there how to handle rereduce parameter – something that I’m not doing here since I don’t expect any).

    Well… up to now the client code seems pretty simple in order to get view results, but how can we test it so far? Until now you could have been creating the respective views within the administration console (under the Views tab).

    At first glance, it seems the C# SDK doesn’t have an API to manage views. Nevertheless, as the design document that contains views details is just another document within the DB, I’ll show you how you can create with a PUT request. You can achieve the ‘trick‘ easily with the following code that relies on the WebClient class from .NET framework to do the job, but first let’s add the design document’s body (JSON) to the JavascriptFunction class to have it at hand:

    public static string Dev_Books_DesignDocument
    {
       get
       {
          return @"{""views"":{
             ""bytitle"":{
                ""map"":""function (doc, meta) {\n if (doc.title && doc.authors) { \n   emit(doc.title, null); \n }\n}""
             },
             ""authorwithbookcount"":{
                ""map"":""function (doc, meta) {\n  if (doc.authors && doc.recommended) {\n    for (var i in doc.authors) {\n      emit(doc.authors[i], 1); \n    }\n  }\n}"",
                ""reduce"":""function (keys, values, rereduce) {\n  if (!rereduce) {\n    return sum(values); \n  }\n}""}}
             }";
       }
    }
    

    Note: This is a temporary solution to have the design document body, but I recommend you to create a C# class instance that can be mapped to that JSON body and transform it before make the request. That will be a better, and more elegant, way to do this.

    Anyway, with the JSON body there we can make the PUT request now:

    var url = "http://localhost:8091/couchBase/default/_design/dev_books";
    var webRequest = new WebClient();
    try
    {
       webRequest.DownloadData(url);
    }
    catch (WebException ex)
    {
       if (ex.Response is HttpWebResponse && ((HttpWebResponse)ex.Response).StatusCode.Equals(HttpStatusCode.NotFound))
       {
          var data = Encoding.ASCII.GetBytes(JavascriptFunctions.Dev_Books_DesignDocument);
          webRequest.Headers.Add("Content-Type", "application/json");
          webRequest.UploadData(url, "PUT", data);
       }
       else
          throw;
    }
    

    Where are making a PUT request to the default bucket with the _design/dev_books id. If you go to your Views tab within the administration console you should see the design document and its views. At the code I’m trying to get the design document and if doesn’t exist within the DB (404 – not found) I’m creating it.

    You can find the sample code here. As usually, I recommend you go to the DB home site, download and try it yourself (you can do it from C#, using any other language found there or simply by using the REST interface the server has).

    That’s all folks!!

    Tags: , , , , , ,