Steve Spencer's Blog

Internet of Things seems to bring together two of my favourite topics: Gadgeteer and Service Bus. Whilst researching IoT I came across an article in MSDN magazine written by Clemens Vasters (http://msdn.microsoft.com/en-us/magazine/jj190807.aspx). This article is from July 2012 and things have moved on a little since then, but the fact that he  has Gadgeteer talking to service bus meant that I had to give it a go myself. The first port of call was the previous article (http://msdn.microsoft.com/en-us/magazine/jj133819.aspx - note the link is wrong in the current article). This explains the architecture that the sample is based upon using service bus topics to send commands to the device and a different topic to allow the device to send data. There is also a provisioning service that allows the devices to be initialised with the correct configuration. this provisioning service also configures up the service bus access control service (ACS) to allow each device to have its own security key. This means you can turn off devices using ACS.

Before you start take a look at the Service Bus Explorer as this is a useful tool when you are trying to diagnose why things aren’t working.

As I’m using a GHI Electronics Fez Spider main board I am using the .Net Micro-Framework 4.2. Upgrading the project to 4.2 had a couple of errors which needed resolving. Firstly, you will need to change GetJoystickPosition to GetPosition; secondly, change ConvertBase.ToBase64String to Convert.ToBase64String. This allowed me to run the project on my Gadgeteer board. However, I kept getting an error whenever I tried to call the provisioning service. I kept getting Bad Request. I immediately assumed that my configuration was wrong but after a bit of searching and then turning WCF tracing on I found that the service could not load the service bus assembly so I removed and the re-added it to solve the problem. As I mention configuration its probably a good idea to say what each of the settings in the provisioning service is used for:

sharedSignature : Go to https://manage.windowsazure.com/ and login. Click on service bus and then select the service bus you are using. on the bottom menu click the Connecting Information button. This will popup a configuration window. there are two keys in here. The first is part of the connectionstring and is under the sas section. Copy the connection string and find the key. This is the sharedSignature for this configuration setting.

servicebusNamespace: This is the name of the service bus as it appears in the management portal.i.e. sb://<servicebusNamespace>.servicebus.windows.net

managementKey: In the same connection information popup where you found the shared signature there is a section at the bottom labelled ACS. the managementkey is the Default Key

Microsoft.ServiceBus.ConnectionString: I used the connection string that appears in the SAS section of the connection information popup.

The other configuration you need to do is to change the url for the provisioning service. This is hard coded on the Gadgeteer board and is located in Program.cs, serverAddress variable in the ServiceBusApp project.

The provisioning service should be ready to go. However, I had problems connecting to the service from the Gadgeteer board as I kept receiving NotSupportedException each time I called GetRequestStream. This was due to an issue with the Ethernet configuration when trying to connect over https. This can be solved by updating the ssl seed using the Fez Config tool (https://www.ghielectronics.com/community/forum/topic?id=13927). This is done by clicking the Deployment (Advanced) button and the clicking Update SSL Seed.

Once complete I could then connect to the provisioning service. The provisioning service should only be called once per device and it is up to the device to store its configuration in a persistent store. This did not appear to be working on my device. Some of the settings were being but the topic urls were not. I changed the type from a URI to a string and the persistence then seemed to work and I only went to provision once. Each time the provisioning service is called a new subscription is create and a new access control identity and rule are also created.

With all this fixed I could now send messages, but I could not see them. this was because I didn’t have a subscriber to the topic where the data was published. This is easily resolved by creating one, but it will only receive new messages. Any messages sent before the subscription is created will be lost.

The provisioning service also has a web page that allows you to send commands to each device. It will broadcast a message to all devices by putting a single message into the devices topic and it sets the Broadcast property to be true. During provisioning the subscription that is created has a SQL Filter applied which allows the subscription to only receive messages that are targeted specifically at the device or if they are broadcast. The web page puts a message into the topic to tell the device to set its temperature to a specific value.The device should be listening for messages to its subscription and will act on the command once it is received.

The device never seemed to receive the message even though the Service Bus Explorer showed that the message was waiting in the queue. Whenever we tried to connect to the subscription “Bad Request” was being returned. After investigation is turns out that the sample only ever sets the event topic uri and not the devices topic uri. When we try and retrieve the device commands we are trying to connect to the events topic which is not a subscription. The sample needs modifying in Microsoft.ServiceBus.Micro project in the MessagingClient class. I added an extra Uri to the constructor and modified the CreateReceiveRequest and CreateLockRequest methods to use this Uri.

The final thing I changed was the command that is sent from the web page and how it was received:

The sender code in Default.aspx.cs in the BackEndWebrole Project

deviceSender.Broadcast(new Dictionary<string, object> { { "Temperature", this.TextBox1.Text } }, "SetTemperature");

And the receiver code in Program.cs in the ServiceBusApp project:

switch (commandType)
{
      case "SetTemperature":
      if (cmd.Properties.Contains("Temperature"))
      {
               this.settings.TargetTemperature = double.Parse((string)cmd.Properties["Temperature"]);
               StoreSettings(this.settings);
     }
     break;
}

I now have a Gadgeteer device talking to the service bus with the ability to send data and receive commands. My next steps are to create a webjob to process the event data (see my previous post) and also look into event hubs.

After a good night in Hereford at the Smart Devs User Group and my presentation at DDDNorth

Here are the links from my presentation and some from questions asked:

Gadgeteer: http://www.netmf.com/gadgeteer/

Signal-R: http://www.asp.net/signalr/

Web API: http://www.asp.net/web-api

The Signal-R chat example can be found at: http://www.asp.net/signalr/overview/getting-started/tutorial-getting-started-with-signalr

Windows Azure Pricing Calculator : http://www.windowsazure.com/en-us/pricing/calculator/?scenario=full

Signal-R Scaleout using Service bus, SQL Server or Redis: http://www.asp.net/signalr/overview/performance-and-scaling/scaleout-in-signalr

The Windows Azure Training Kit: http://www.windowsazure.com/en-us/develop/net/other-resources/training-kit/

Gadgeteer Modules: http://proto-pic.co.uk/categories/development-boards/net.html

Fex Spider Starter Kit: http://proto-pic.co.uk/fez-spider-starter-kit/

In addition to these links I have more from my presentation at the DareDevs user group in Warrington

It is possible to drive a larger display from Gadgeteer using a VGA adapter. You use this the same way that the Display-T35 works using the SimpleGraphics interface for example.

VB eBook - Learn to Program with Visual Basic and Gadgeteer

Fez Cerberus Tinker Kit: https://www.ghielectronics.com/catalog/product/455 

I’ve been playing with Gadgeteer (http://www.netmf.com/gadgeteer/) for a while now and I am a big fan of the simple way we can build embedded hardware applications with high functionality. We have a proof of concept device that includes a Colour touch screen, RFID reader and an Ethernet connections. This device is capable of connecting to a web api REST service which we have hosted in Windows Azure and we can use this service to retrieve data from our service depending upon the RFID code that is read. This works well but there are times when we would like to notify the device when something has changed. SignalR seems to be the right technology for this as it removes the need to write polling code in your application.

Gadgeteer uses the .Net Micro framework which is a cut down .Net framework and doesn’t support the ASP.NET SignalR libraries. As we can use web api using the micro framework using the WebRequest classes,  I wondered what was involved to get SignalR working on my Gadgeteer device.

The first problem was to work out the protocol used by SignalR and after a short while trawling the web for details of the protocol I gave up and got my old friend fiddler out to see what was really happening.

After creating a SignalR service I connected my working example to the signalR hub running on my local IIS..

The first thing that pleased me was that the protocol looked fairly simple. It starts with a negotiate which is used to return a token which is needed for the actual connection.

GET /signalr/negotiate?_=1369908593886

Which returns some JSON:

{"Url":"/signalr","ConnectionToken":"xyxljdMWO9CZbAfoGRLxNu54GLHm7YBaSe5Ctv6RseIJpQPRJIquHQKF4heV4B_C2PbVab7OA2_8KA-AoowOEeWCqKljKr4pNSxuyxI0tLIZXqTFpeO7OrZJ4KSx12a30","ConnectionId":"9dbc33c2-0d5e-458f-9ca6-68e3f8ff423e","KeepAliveTimeout":20.0,"DisconnectTimeout":30.0,"TryWebSockets":true,"WebSocketServerUrl":null,"ProtocolVersion":"1.2"}

I used this JSON to pull out the connection id and connection token. This was the first tricky part with the .Net Micro framework. There is not the same support for JSON serialisation you get with the full framework plus the string functions are limited as well. For this I used basic string functions using Substring and IndexOf as follows:

int index = negJson.IndexOf("\""+token+"\":\"");
if (index != -1)
{
    // Extracts the exact JSON value for then name represented by token
    int startindex = index + token.Length + 4;
    int endindex = negJson.IndexOf("\"", startindex);
    if (endindex != -1)
    {
        int length = endindex - startindex;
        stringToExtract = negJson.Substring(startindex, length);
    }
}

With the correct token received Fiddler led me to the actual connection of signalR:

GET /signalr/connect?transport=webSockets&connectionToken=yourtoken&connectionData=%5B%7B%22name%22%3A%22chathub%22%7D%5D&tid=2 HTTP/1.1

Looking at this I could determine that I needed to pass in the token I retrieved from negotiate, the transport type and the name of the hub I want to connect to. After a bit of investigating I used the transport of longPolling.

Now as I think I understood the protocol, I tried to implement it in SignalR. The first issue that arose was what to send with the negotiate call. I figured that this was some sort of id of the client that is trying to connect so I decided to use the current tick count. This seemed to work and I guess that as long as my devices don’t connect at exactly the same time then Signal R would work. I’ve had no problems so far with this.

Upon connecting to the hub I needed to create a separate thread to handle signalR so that the main device wouldn't stop running whilst the connection to the SignalR hub was waiting for a response. Once a response is received the response returns with a block of JSON data appropriate to the SignalR message being received. This needs to be decoded and passed onto the application. You then need to reconnect back to the SignalR hub. The period between receiving data and then reconnecting back to the hub needs to be small. Whilst the message is being processed it cannot receive any more message and may miss some data. I retrieve the response stream and then pass the processing of the stream to a separate thread so that I can reconnect to the hub as fast as possible.

This is not a full implementation of SignalR on the .Net Micro-framework but it is the implementation of a simple client and can be used fairly successfully on the Gadgeteer device. I still need to do a little more work to try to speed up the connections as it is possible to miss some data.

The SignalR hub is hosted on a Windows Azure website along side the web api service which allows both web, Windows 8 and Gadgeteer applications to work side by side.

Gadgeteer has opened up another avenue for development and helps us to provide more variety of devices in a solution

I was having problems getting my Gadgeteer ethernet card initialised and running. I wanted to set it up to use DHCP but I never got an IP address assigned. I am using a GHI Electronics J11D ethernet card and browsing for examples seemed to pull up a lot of code but none of it seemed to work or the code didn’t seem to match what the libraries were providing. I eventually found the solution.

// Wire up the event handler to notify when the ip address has been assigned 
// and the port is ready to use
ethernet_J11D.Interface.NetworkAddressChanged += new
   NetworkInterfaceExtension.NetworkAddressChangedEventHandler(
					Interface_NetworkAddressChanged);

// Open the ethernet port
ethernet_J11D.Interface.Open();

// Assign the network stack to the ethernet card
if (!ethernet_J11D.Interface.IsActivated)
{
    NetworkInterfaceExtension.AssignNetworkingStackTo(ethernet_J11D.Interface);
}

// Turn on DHCP and Dynamic DNS
ethernet_J11D.Interface.NetworkInterface.EnableDhcp();
ethernet_J11D.Interface.NetworkInterface.EnableDynamicDns();

It was the line (NetworkInterfaceExtension.AssignNetworkingStackTo(ethernet_J11D.Interface); ) that was the issue, once that was in everything worked fine.

I can now connect to my Windows Azure Websites hosted web api/signalR service.

The code for this is fairly standard and once I got the connection it worked well. The code below shows you how to call the web api service from Gadgeteer. This method works for both GET (read) and PUT (update) requests.

private string CallWebservice(string fn, bool put, string data)
{

string responseFromServer ;
try
{
    // Create a request for the URL. 
    WebRequest request = WebRequest.Create(url + fn);

    // set a timeout of a nice big value - 10 minutes
    request.Timeout = 600000;
    if (put)
    {
        request.Method = "PUT";
        System.Text.UTF8Encoding encoding = new System.Text.UTF8Encoding();
        byte[] arr = encoding.GetBytes(data);
        request.ContentType = "application/json";
        request.ContentLength = arr.Length;
        Stream requestStream = request.GetRequestStream();
        requestStream.Write(arr, 0, arr.Length);
        requestStream.Close(); 

    }

    // Get the response.
    WebResponse response = request.GetResponse();

    // Get the stream containing content returned by the server.
    Stream dataStream = response.GetResponseStream();

    // Open the stream using a StreamReader for easy access.
    StreamReader reader = new StreamReader(dataStream);

    // Read the content.
    responseFromServer = reader.ReadToEnd();

    // Tidy up
    reader.Close();
    response.Close();
}
catch (Exception ex)
{
    Debug.Print(ex.Message);
}

return responseFromServer ;
}