Johnny Halife’s Blog

Lately, we’ve been working on a Windows Azure Project with huge load, really high peaks. During the project we got the following “gotcha moments” that I’ll be trying to summarize throughout the post and thru a series of post I expect to write.

If you are using WCF, you must tweak it

On iServiceOriented.com, there was this post about tweaking WCF that still valid up-to-day. WCF is broken by default, and if you were planning using it on Windows Azure (or even your own servers) you must tweak it with all the performance optimizations, except that you are fine with a lousy 10 requests per second.

Gotcha #1: Follow the performance on “WCF Gotchas 3: Configuring Performance Options“

RetryPolicy can be evil

RetryPolicy is the mechanism used by the Windows Azure Storage Client to prevent the users from its own service fails. As the idea itself rocks, implementation wasn’t necessarily done for your scalability needs.

When writing high traffic services you might want to keep the least number of threads or at least all of the identified, the built-in RetryPolicy hides the underlying complexity of performing retries when the service fails but it also hides the Thread usage from you which at this scale is critical.

Gotcha #2: Disable the built-in RetryPolicy

By using RetryPolicies.NoRetry you can prevent your app for creating threads just to ensure that an action has been executed, and if you need your app to retry for an eventual Service Availability issue, write your own policy.

If you need inspiration, you can check these snippets for RetryPolicy and another thing that you should consider when doing this type of things, like adding an Extension Method to identify whether an exception is “Retry-able” or not.

Let Windows Azure Storage handle it

“Cloud Computing” brought Computing up to a whole new level, nowadays developers are able to tell how much a design decision cost. This is stunning, now all those performance optimizations that you always wanted but never had the chance to implement have a strong economical justification (or not).

Whenever you’re exposing data to a client thru a (web)service in Windows Azure your paying for I/O and Compute Hours. This costs are distributed as transfer from/to Storage to the Service and from/to Service to the Client.

Now consider the following, the reference data of your application (data pretty much the same for all the users) can be consumed by the Client straight from Storage instead. Now the cost distribution radically changes to be I/O from Windows Azure Storage to Client, no more Compute Time nor scalability headaches.

Gotcha #3: Deflect load to Windows Azure Storage as much as you can

Redirecting the load to Windows Azure not only saves you some bucks from Computing Power and I/O but also take out the pain of having to scale up your services, since Microsoft is the responsible of scaling it up.

When doing this remember that Windows Azure Storage is RESTful and all the optimization that can be performed at the transport level (like Caching, Expiration, GZip, etc) perfectly fit here, and if it’s a javascript client take a look at JSON(P) (JSON is much more efficient than XML).

Livin’ on the edge

Experience has proven that if you live on the latest VM Image by Windows Azure Team your performance and stability will increase as they ship new images.

Gotcha #4: Configure your Hosted Service Deployment to use the latest-available Virtual Machine

Except your code has compatibility issues with .NET 4 or issues with .NET 3.5 SP(x) you should live always on the latest VM image. This can be configured using the Windows Azure MMC and setting the Virtual Machine Image Version to “*” (star)

I expect to continue writing about the different patterns, gotchas and stuff we figured out while working on Windows Azure, so stay tuned!

thanks,
~johnny

Taken from the rebase concept of git which is also used by GitHub to show they newsworthy and notable projects, I’m using the post to do the same with bunch of Open Source, shared, hacking projects I’ll be doing lately.

Since 2010 started, I didn’t blogged that often, but there were a couple of projects that I’ve been working lately. Throughout this post, I’ll describe each one and the futures.

Every piece of feedback will be welcome, as every other contribution too.

Enjoy the ride,
~johnny

Rack::Auth::WRAP, the OAuth WRAP Middleware

Yesterday, with Juan Pablo, we published our first version of Rack::Auth::WRAP the first version of the Rack. If you are familiar with the protocol, you can skip the next section if not, take a look at it. Extracted from the read me at github.com.

What the heck is WRAP?

Web Resource Authorization Protocol (WRAP) is a profile of OAuth, also called OAuth WRAP. While similar in pattern to OAuth 1.0A, the WRAP profile(s) have a number of important capabilities that were not available previously in OAuth. This specification is being contributed to the IETF OAuth WG.

Also this same group owns the specification for the SWT (Simple-Web-Token), for more information read wiki.oauth.net/OAuth-WRAP or visit the groups.google.com/group/oauth-wrap-wg.

The latest specification for the complete protocol can be found at Google Group as HTML (RFC properly formatted) on groups.google.com/group/oauth-wrap-wg/attach/981df73f2839b8ef/draft-hardt-oauth-wrap-01.html?part=5

Creating your first protected resource

As you might be thinking, our first resource will be a Sinatra application.

First of all we need to install the gem, as

[sudo] gem install rack-oauth-wrap

To make the sample easier let’s create our own shared key, we can all share this for demo purpose NjkzNTczOTAtMDA2MC0wMTJkLTQ1M2YtMDAyMzMyYjFmYWY4\n

So let’s start by creating the protected resource

require 'rubygems'
require 'sinatra'
require 'rack/auth/wrap'

use Rack::Auth::WRAP, :shared_secret =>t; "NjkzNTczOTAtMDA2MC0wMTJkLTQ1M2YtMDAyMzMyYjFmYWY4",
                      :audiences =>; "http://localhost:4567",
                      :trusted_issuers =>; "urn:demo-issuer"

get "/" do
    if @env["REMOTE_USER"]
        return “You are authenticated as #{@env["REMOTE_USER"]['Email']}”
    else
        return “You are an unauthenticated user”
    end
end

Now we can start this on a Terminal (cmd, or whatever) and let’s jump to the consumer, but first if you try it without sending a token, and using the client we are going to build, you will get:

?> curl http://localhost:45678
You are unauthenticated

Now lets create a client trying to access a protected resource with a token on the header (requires restclient)

require 'rubygems'
require 'cgi'
require 'base64'
require 'restclient'
require 'hmac/sha2'

SHARED_SECRET = "NjkzNTczOTAtMDA2MC0wMTJkLTQ1M2YtMDAyMzMyYjFmYWY4\n"

simple_web_token = {'Audience' =>; "http://localhost:4567",
                    'Issuer' =>; "urn:demo-issuer",
                    'ExpiresOn' =>; (Time.now.to_i + 60).to_s,
                    'Email' =>; 'johnny.halife@sample.com'}.map{|k, v| "#{k}=#{CGI.escape(v)}"}.join("&")

signature = Base64.encode64(HMAC::SHA256.new(Base64.decode64(SHARED_SECRET)).update(simple_web_token.toutf8).digest).strip
simple_web_token += "&HMACSHA256=#{CGI.escape(signature)}"

puts RestClient.get("http://localhost:4567/", "Authorization" =>; "WRAP access_token=#{CGI.escape(simple_web_token)}")

Now let’s try our client, and see if there’s any difference with the curl request:

?> ruby client.rb
You are authenticated as johnny.halife@sample.com

As you can see, we have our first end to end, Rack::Auth::WRAP Sample.

DISCLAIMER: On a real world application you won’t generate your own token as we are doing on the client code. We are doing it for demo purposed, but probably on you app you will get a token from an authorization server.

Both snippets are available as gits on github: Protected Resource / Client. We are assuming that this is running on localhost:4567

TODO’s and futures

On the upcoming days/weeks/months we are going to get on the middleware support for the other ways of getting the token, like Query String and/or method body. Also we would like to implement the Web Profile of WRAP, so stay tuned.

You can read the freshly published documentation at http://rack-oauth-wrap.heroku.com

Source Code available at: http://github.com/johnnyhalife/rack-oauth-wrap

OAuth WRAP 0.9 for Tcl

First of all, if you aren’t familiar with Tcl it’s “originally from “Tool Command Language”, but conventionally
rendered as Tcl is a scripting language created by John Ousterhout”. I encourage you to test it and also if you are interested read Where’s Tcl hiding?.

This project was born after half an hour spiking on how hard it will be to parse a token on a bare linux distro that only has Tcl. After we noticed that Tcl is really straightforward language for design, prototype and is fun to write, we packed this lib and make it available for anyone interested.

Here’s an snippet of the intended usage of the lib

package require ::oauth::wrap

set rawToken "access_token=something&other_parameters_to_ignore" #=> the token from the IP

# => creates a configuration dictionary for the values
dict set configuration signingKey {valid_key} # => signing key used by the Identity Provider
dict set configuration issuer {valid_issuer} # => the identity provider URI
dict set configuration audience {valid_audience}  # => my application audience URI

# this will return the token when it's valid else it will return false
set token [oauth::wrap::authenticate $configuration $rawToken]

# at this point if the token valid you can mess around with its claims
# that are returned on a dictionary form
set name [dict get $token name]

It’s fun to give it a shot, check out the source code at http://github.com/johnnyhalife/tcl-oauth-wrap

Windows Azure Storage for Ruby v1.0

On the 4th February, 2010 I’ve published the version 1.0 of ruby gem I wrote for Windows Azure Storage. This time it has the great contribution of my friend Juan Pablo Garcia Dalolla who has implemented the Windows Azure Tables support.

This version of the gem also includes support for the version 2009-09-19.

Here’re are some code snippets from the Windows Azure Tables support

require 'waz-storage'
require 'waz-tables'

# The same connection of Windows Azure Storage Core (Queues, Blobs) can be reused
WAZ::Storage::Base.establish_connection!(:account_name =>; account_name,
                                         :access_key =>; access_key)

# Grab the service instance
service = WAZ::Tables::Table.service_instance

# Query the customer table
service.query('customer_table', {:expression =>; "(PartitionKey eq 'customer') and (Age eq 23)", :top =>; 15} )

# Insert something into the customer table
serivce.query('customer_table', {:row_key =>; 'my_custom_id', :name =>; 'johnny'})

There’s also a DataMapper adapter effort going on for Windows Azure Storage Tables, I recommend you to check out Juan Pablo’s post about Windows Azure Tables Adapter for Datamapper

Source Code available at: http://github.com/johnnyhalife/waz-storage

RDoc available at: http://github.com/johnnyhalife/waz-storage

Hey Folks, after PDC 09′ I’m back on track doing some work on the Windows Azure Storage SDK for Ruby. On my Channel9 interview, I told you that Ray Ozzie announced a new version of Windows Azure Storage API (a.k.a 2009-09-19) which enables developers to perform new tasks on the storage components

These are the enhancements for the 0.5.6 version of the SDK, this release only covers the Windows Azure Queues Service 2009-09-19 version, Blobs support will be released end of this week or early next week.

What’s new on the 0.5.6 version?

• Added new shared key authentication support for 2009-09-19 Version of the Storage API
• Queues API has been migrated to the 2009-09-19 Version of the Storage API
• Added a new parameter for Listing Queues with Metadata
• Added support for DequeueCount on messages being retrieved from the Queue
• Known Issue: Creating a queue multiple times with same metadata throws 409.

Using the latest version of the Windows Azure Storage SDK for Poison Message detection

  WAZ::Storage::Base.establish_connection!(:account_name => 'name', :access_key => 'key')

  # Let's start by selecting a queue
  queue = WAZ::Queues::Queue.find('my-queue')

  # While the queue has messages, let's check the content
  while (queue.size > 0) do
    # Now let's dequeue a message as we usually do we the API
    message = queue.lock

    # Before processing the message we can now do a sanity check for the message
    # if the message has been dequeued more than 5 times we will destroy it since
    # it can be a poison message.
    if (message.dequeue_count > 5)
        message.destroy!
    else
        # We process the message as we usually do
    end
  end

The key feature on this version of the API is the dequeue_count property for Queues which serves the primary goal, as shown sample above, of Poison Message detection.

Stay tuned for further updates on the SDK…

thanks,
~johnny

Since, the first release of the Windows Azure Storage SDK for Ruby, I’ve received retweets and comments from Jean-Christophe Cimetiere (Sr. Technical Evangelist - Interop Strategy Team at Microsoft), we never met in person until Ezequiel Glinsky put us in contact.

I traveled to PDC 09′ to work on the Keynote and stuff for the event. Thanks to JC, I was able to record a short demo/interview with him for Channel9. The interview is for those that doesn’t know the work we have been doing and also to get you and idea of where this work is going to.

The SDK stills in beta, and we’ve already implemented a couple of features of the 2009-09-19 version of the API announced by Ray Ozzie on the Microsoft PDC 09′. I hope you like it and any comments will be always welcomed.

As usual thanks to my team: Juan Pablo Garcia and Ezequiel Morito major advisors and contributors of this project.

thanks,
~johnny

About a month ago, I left my home in Buenos Aires to start working with James Conard and his team on the “Cloud Convergence” Demo and some other PDC 09′ Stuff. Today since Ray Ozzie touched the stage we (backstage) were waiting for our demo to become true. Before that were a couple of demos, and specially (and I’ll talk about this later on this post) “Tailspin Travel”.

When we got to Redmond area, we met Jonathan Carter (a.k.a JC) who was working with PC (a fellow southy), they were working on the “Tailspin Travel” demo for the keynote that was shown by Cameron Skinner. Tailpin’s demo represents most of the “best practices” that you should apply when doing an ASP.NET MVC Web Application that has features like back-end support with Workflow and other WCF Services, and integrates with your Enterprise Identity using WS-Federation Protocol.

Imagine how good the demo was, that we took that same application to do our work on the “Future of the Platform”. Additionally, they are doing something great with the application that since the keynote was available for you to download it and learn the “goodies” of .NET 4/WIF/AppFabric development.

Tailspin Travel

The Tailspin Travel application covers a pretty substantial set of functionality, but ultimately seeks to provide a holistic perspective of how Visual Studio 2010, .NET Framework 4,

and the server platform can be used together. Among other technologies it leverages: Visual Studio 2010 (with the new fancy diagrams), .NET 4.0 (MVC2, WIF, WF, WCF, and so) andshow the hosting layer of your services using the brand-new “AppFabric”

I guess at this point of the post you should be rushing to get the bits, and if you don’t go new to get them at http://tailspintravel.codeplex.com/.

Meet me at PDC

I’m on PDC with other fellow Southies, we are all the time around, and if you want to talk and share a coffee, just look for me or DM message at @johnnyhalife on twitter and we will meet for sure.

Today I’ve been interviewed by Jean-Christophe Cimetiere about the work I’m doing with Ruby and Azure. You should stay tuned for more information that I’ll be announcing soon on these things, like Azure 2009-09-19 support of the API on the Ruby SDK.

Acknowledments

Although Matías , PC and I where backstage, there were lots of Southies that made these PDC events work great, I sincerely appreciate the work all the Southies were doing, and how do they helped us.

hope to see you around,

~johnny

I was working on the Ruby Windows Azure SDK and I started receiving HTTP/1.1 403 errors, I figured out which is the problem so here you can get a simple issue analysis

Symptom
While trying to connect to Windows Azure (Storage at least) from Argentina, you get 403 errors even dough the message seems to be formatted correctly.

Issue
Argentina was supposed to change the time by adjusting it to DTS (Daylight Time Saving), one day before that happens the Argentinean Government decided not to make that change, generating a lots of implications for computer systems adjusted to DTS.

Since Windows Azure relays on UTC Timestamp for making an assertion on the signature, the current time for GMT -3 isn’t what is expected from the server side causing the whole message to fail after the an assertion of the signature.

Workaround
There’s no apparent solution yet, but there are a couple of workarounds in order to properly develop against Windows Azure from a not changed GMT -3 time zone.

• Change your time zone to GMT -4. I switched my computer back to Halifax - Canada that has GMT -4 for an Windows Azure started accepting my requests
• Keep your computer with the clock one hour ahead . Keep the current time zone without tweaking the Date/Time, but remember that you will be out of sync with the country

Both of the workarounds listed above proved to be successfully working, I will stick with GMT -4 since I guide myself (eating, sleeping, and all that) in my computer clock.

Hope it helps,
~johnny

Today I was discussing with my friend Juampi Garcia who is writing a nice sample application that leverages the waz-storage gem and some other interesting things. While we were talking about his application, he asked me for some way of overriding the default connection for performing a bunch of operations as he needed it for his application.

I started thinking and looking around some samples from DataMapper and other ORM frameworks that have the ability to override the current context by calling a method or specifying connection name. I looked deeply into it, but end up thinking that it will be better to have a little stack of connections in order to handle different contexts.

I end up writing a new method to WAZ::Storage::Base that enables you to scope your code to some specific context and once that context is left you get back to the default connection.

Consider the following sample in order to better understand what I did

  WAZ::Storage::Base.establish_connection!(:account_name => 'name', :access_key => 'key')

  # any operation performed here will be on the default context expressed above. e.g.
  container = WAZ::Blobs::Container.find('my-container')

  # So, now let's suppose that I need to perform an operation on another context,
  # but those are just a few, and very scoped that isn't worth switching the whole
  # context to that. Leveraging the new method, I'll just do:
  WAZ::Storage::Base.establish_connection(:account_name => 'another', :access_key => 'key') do
     # any operation that I perform here, will be on this new context. e.g.
     another_container = WAZ::Blobs::Container.find('my-container')
     # Note that the operation above is performing a look-up of the container on another
     # storage account.
  end

  # Now that I've left the block, I'm back to my original context.

This new feature has been added to project at github.com/johnnyhalife/waz-storage and the new documentation is also available at waz-storage.heroku.com.

You can upgrade your gem, in order to have this new feature or just using the bleeding edge by cloning the repository on github (git://github.com/johnnyhalife/waz-storage.git)

Another small change

If you are (or tried) using waz-storage on Heroku.com without rails, you will realize that some exceptions about ‘String.start_with?’ being undefined may appear. I’ve added an snippet to implement this method, on the rails way, when loading the waz gem (if it was previously undefined).

thanks,
~johnny

Hey, after talking, talking and talking about the possibilities of building a Windows Azure Storage SDK for Ruby, I finally got together a couple of ideas and develop the v.0.5. I’m still needing to write a decent sample application but now it has more resources that previous gems had.

For the project source code and pretty basic documentation go to http://github.com/johnnyhalife/waz-storage.

For the whole API documentation refer to http://waz-storage.heroku.com.

What is Ruby Windows Azure Storage SDK (a.k.a waz-storage)?

A simple implementation of Windows Azure Storage API for Ruby, inspired by the S3 gems and self experience of dealing with queues. The major goal of the whole gem is to enable ruby developers, like me =), to leverage Windows Azure Storage features and have another option for cloud storage.

The whole gem is implemented based on Microsoft’s specs from the communication and underlying service description and protocol (REST). The API is for Ruby developers built by a ruby developer. I’m trying to follow idioms, patterns and fluent type of doing APIs on Ruby.

This work isn’t related at all with StorageClient Sample shipped with Microsoft SDK and written in .NET, the whole API is based on my own understanding, experience and values of elegance and ruby development.

Full documentation for the gem is available at waz-storage.heroku.com

Scenario Idea

The gem was thought having heroku.com in mind as it is my main Ruby hosting vendor. The basic idea can be expressed with the following diagram:

As you can see there’s no need (and no personal desire) of hosting your application in Windows Azure or even writing it on .NET (or Windows needed at all).

How does this differ from waz-blobs and waz-queues?

Well, this is a sum up of the whole experience of writing those gems and getting them to work together to simplify end user experience. Although there’re some breaking changes, it’s pretty backward compatible with existing gems.

Open Standards

The waz-storage model is built upon the REST API published by Microsoft for Windows Azure Storage Services following the HTTP Standard. It’s basically a bunch of HTTP Requests with a special signature and some custom headers. The same API can be ported to any platform supporting a decent HTTP Stack.

So, what?

As in the latest posts, I won’t include too much detail here since it’s all explained at github.com/johnnyhalife/waz-storage and the whole API is detailed (and documented) at waz-storage.heroku.com.

The future is bright, I strongly appreciate all your feedback, your tweets and the time spent on (at least) reading the code. I really love this whole experience, but I won’t stop here, I’ll be relaxing from this SDK for a week or so (maybe less) until I come back for doing Tables support and fixing the issues that users may have encounter.

TODOs’

There should be some samples or at least one Reference Implementation of the API, probably I’ll be requesting help from other people in order to accomplish this, but I promise that some sample will be soon online so you can see the unveiled power of the Windows Azure Storage when running on Ruby.

Also there’s some support for Blocks that will be added, and some other sugar features for the application.

Time will tell, meanwhile you can comment, tweet or say whatever you want since I’ll be listening to your feedback.

Acknowledgment

This time I’ll make an special acknowledgment, besides my team, teammates, friends, and collegues at Southworks, Ezequiel Morito and Juampi Garcia. I want to thank and recognize that this work won’t be online today if I didn’t receive the advice from Martin Salias who encouraged me to go further on spiking and publishing my spikes.

Thanks Martin, I really appreciate your support and guidance.

Hey, after giving lots of thoughts of what am I going to do with the whole set of Ruby APIs for Windows Azure Storage, I figured out that might be better to have them merged, tested and refactored as a whole thing. I thought about lot’s of pro’s and con’s regarding whether to do this or not and I came out with a the following list:

Pro’s

• Blobs and Queues handle the same type of security, connection and headers. They share the core service called WAZ::Storage::SharedKeyCoreService
• While I was doing some work on Queues I found some bugs on connection management and URI management on the CoreService, so those changes weren’t propagated to Blobs (yet).
• 99% use case of the gem is that people handle a single WAZ Account for everything (Queues, Blobs and Tables) so have to connect, configure and consume them from different gems seems lot of overhead.
• Maintainability Index really drops on having them split.
• This is a single man’s work and even worse is doing over my spare time (so I cannot keep them both up-to-date always).
• Total gem size is 50KB compared to two gems of 30KB (so storage isn’t a problem here)

Con’s

• At some point seemed cool and very aligned with YAGNI that you need to install only what you need (queues or blobs)
• Shared behavior could be inherited from a third gem called ‘waz-storage-core’ but that will require more maintainability work.

Those are the things that came to my mind while working on this, so I decided to glue them together, refactor them and generate a consisten API for both services (and the tables soon enough).

Towards the SDK

Ruby devs aren’t usually required to have SDKs that’s more of the Windows World. SDK is the name I decided the whole thing will have: documentation, sample, gems and source code (Yes! I’ll do more docs). I’m looking for a consumable way for Ruby Devs to get involved on Windows Azure (since it’s worth doing it) but also there’re a lot of .NET Developers that may want to have a look at Ruby’s features and opportunities outside Windows World.

This is work in progress

Although I didn’t officially released anything related to this new SDK, I started my own work towards completing the API, so every piece of valuable feedback you might have will be really appreciated! I really enjoy while doing this and probably will do better as time goes by but as you can see on the pictures, I already started, merged and have a pretty decent TODO’s to apply to this thing and probably after that I’ll be releasing another version.

From the picture on the right you can infer, that I’ve already merged the test suite, and the TODO’s. The whole project is now a single thing. These are the things that I’m considering for next version

As you can see, there are things that still need to be figured out, but with time (no much) , they will emerge and I’ll be releasing as soon as I have them.

Help wanted!

As I mentioned now and on the previous posts, this is a one man’s work but you can change it =). I’m looking for Developers (juniors, amateurs, pro’s, whatever) to contribute on the project as they wish. If you want to test the code base, write a sample, write some docs, or whatever you’re invited! Drop me a message at @johnnyhalife and we should get going soon.

that’s it, stay tuned for more! (very soon)

thanks,
~johnny

Hey Pals! As you might recall two days ago I’ve published an API called “waz-blobs” that enabled Ruby Developers to get the best from Windows Azure storage straight from their pure ruby code. I told you I won’t stop there, so now I’m proud to announce that I’ve created waz-queues. As you can imagine and given the fact that I wasn’t creative at the time I named this things (If I had a dog I’d name it dog), this API exposes Windows Azure Queues to Ruby Developers (and IronRuby Devs too!).

Like the previous API, it’s 100% pure ruby, relays on RestClient written by Adam Wiggins and was written and tested on Mac and Ubuntu.

As on the previous post I wrote about the Blobs API,  this post isn’t about API documentation. It’s about all those things that forced me to write the API and some other random thoughts about where this whole thing is going.

Oh, you did it again!

Yes! I love it, interoperability is my passion, I love to write things to demonstrate that they can work with other very different things, and given the fact that Microsoft did a great job when they published the Windows Azure Storage Queue API on line, I decided I’ll spend some time building this. Queues are great, async programming is cool and REST is something that people do not believe until they see it (for those skeptical devs this is fully implemented over HTTP Stack on pure Ruby)

There’s nothing worse than waiting inline. Bummer!

Welcome to the twentieth one century, we live surrounded by the buzz and the current one is called “Cloud Computing”. Although I won’t discuss the topic here, it relays on a very simple principle “pay-to-play”. If you want to handle more requests just increase the number of workers and decrease the number of money you’ve got on your pockets.

Not all the operations on a System need to be always consistent, Amazon cart is an example of that. When you don’t want to pay more just get a queue, do some work async, and voila! your problems are solved.

Some hosting platforms like Heroku o Windows Azure have a limited number of requests that they can handle with a single machine/worker/dyno/you_name_it, when the backlog becomes to big they start to deflect request with HTTP 500 messages. If you relay on a queue you will also have a way to increase responsiveness on your app, sites like digg and twitter.com already do something similar.

Why do I need Azure Queues?

It’s an extremely powerful alternative to having a File System Queue or archaic Queue Systems like MSMQ, it’s on the cloud, it’s unlimited storage growth and can be used from either your current hosting or Windows Azure.

The whole Queue Service provided by Windows Azure it’s really simple but not least powerful. Possibilities are endless, and you can do whatever you want and create a multi-party system that has your web app hosted on Heroku written in pure ruby, a back end service running on Windows Azure written on C# and a python app hosted on Google Apps engine that uses Windows Azure Queues to communicate with each other. There’s no limit! (Even your mainframe can handle WAZ-Queues with some C HTTP Lib + playing nice with sockets).

Ok, I buy it, now show me your magic

The API is pretty simple, simpler than anything I’ve written before. Here’s a snippet that illustrates basic API interaction (or in other words, everything you need from a queue)

Getting started

sudo gem install waz-queues --source http://gemcutter.org

Reference Code

  require 'waz-queues'

        service = WAZ::Queues::Base.establish_connection!(:account_name => account_name,
                                                          :access_key => access_key)

        # excepts that the metadata for the queue changes this method behaves as PUT
        # remarks: it performs a validation whether metadata changed or not (HTTP 409 conflict)
        queue = WAZ::Queues::Queue.create('my-queue')

        10.times do |m|
          # enqueue a receives string. Message content can be anything up to 8KB
          # you can serialize and send anything that serializes to UTF-8 string (JSON, XML, etc)
          queue.enqueue!("message##{m}")
        end

        while(queue.size > 0) do
          # Since WAZ implements the peek lock pattern we are locking messages (not dequeuing)
          # it has two parameters how many messages and for how long they are locked
          messages = queue.lock(10)

          puts "dequeued message: #{messages.size}"

          # deletes the message from the queue so other clients do not pick it after
          # visibility time out expires
          messages.each {|m| m.destroy!}
        end

What’s next

While doing this I figured out that there’s a common behavior shared across blobs and queues, you can see it on the code that I started doing some merging while creating a ruby module called WAZ::Storage::SharedKeyCoreService that handles everything related to connections, requests, headers and signing messages. I’ll be merging the whole thing into ‘waz-storage’ gem that I’m planning to start over the weekend and create a small reference application to illustrate better what does this do? (if you wanna join ping me via twitter @johnnnyhalife)

That’s it, I won’t say I’ll do tables API too, I guess that after this week you can figure it out by yourselves (although it might take more time)

Special Thanks

I’d like to thank the following people that help me with something more important and painful than designing the API or writing the code that this time was an individual experiment. The following people are listening to me in every spare time I have at work, talking about why I design something like I did, how cool it is, or how do I enjoy this, which can be even hard than putting this code together!

These people are: Matias Woloski, Ezequiel Jadib, Federico Boerr, Ezequiel Morito, Sebastian Renzi, Martín Salias and Pablo Costantini.

That’s all folks. Thanks,
~johnny