Google Home, Harmony Remote, IFTTT and Verum Dezyne


A realistic example of what it means to simplify the internet of things (IoT) development and testing.

Click here to download a .pdf version of this paper.

In a recently series of published articles, I have argued that some concepts of safety critical design can help every day developers and testers. One of my points was to use modern and lightweight modeling to create specifications of your system before you code. Of course, unlike formal modeling, we suggested to use tools with as little as possible of learning time.  With these types of new “smart whiteboard”, you quickly loop back from code to white board to code and validate that no usage scenario or logical error has been missed. This will result in cost reductions in testing and validation because of lower number of logical error in designs, and accurate, complete and verifiable requirements for the QA team. This article is a down to earth example of what I meant using this sort of approach (within Verum Dezyne, a new generation modeler) and real-life IoT (Internet of Things) products.

Google Home, Philips Hue Light, Harmony Remote and the IoT

I have always been an early adopter of technologies. I recently acquired a Google Home to control the Philips Hue Lights in my kitchen. With this setup, I can now say things like “Ok Google! Can you please turn all the lights blue?” and all the kitchen lights turn blue. Incredible!

How does it work? The Hue Lights come a physical Hub that simply plug in your home network. The Hub exposes a simple web-based API that Google can use to control your light.

Encouraged by this success, I remembered I also had a Harmony Universal Remote at home, which also provides a hub plugged in my network. So, I was now on a mission to also turn my projector and movie theater setup on and off using Google Home. “Ok Google! Can we watch a movie!”.

There was, sadly, no direct integration between Harmony and Google. The IoT emerging market will have a lot of similar and compatible technologies not able to work together, not because of the technology, but because of licensing and competition.

A Innovative Solution: IFTTT

Luckily, I found a solution in the IFTTT Platform (https://platform.ifttt.com), an interesting technology that let you interconnect systems not designed to work together. It is a perfect example of a new technology that will help consumers and integrators enable their device in the IoT world.

As a user, you create “applets”, sort of visual programs that react to “triggers” and run “actions”. In my case, the trigger is saying “Ok Google! Can we watch a movie!” to Google Home, and the action is turning the “Projector” configuration on with my Harmony Universal Remote. It was a breeze to configure. Success!
So how does IFTTT work? Both the Harmony and the Google device provides cloud services (Channels) that IFTTT accesses to first obtain your permission to use your service, and second to receive “triggers” and execute “actions”. All those are made thru simple calls to a URL / HTTPs endpoints.

A developer point of view

The IFTTT approach illustrates perfectly some of the IoT technologies and challenges:

  • A cloud service approach
  • Behaviors you cannot fully cover by testing because the end-user will be the one deciding what the applets do
  • APIs and services provided by 3rd party on which you have no control, as well as some of the common security challenges due to user authorization.

Let’s see what it would take if we were a Google or Harmony developer, and we oversaw developing services that will be accessed through HTTPs to deliver your functionality in the IFTTT framework.

The API reference to IFTTT is here: https://platform.ifttt.com/docs/api_reference.
But I have attached a copy of the documentation here.

A lot is already provided for you, but you will need to create multiple endpoints:

  1. One for the oAuth2 Authorization (“Can I get access?”)
  2. One for managing the oAuth2 token (“Can I get a token now I have access?”)
  3. One for User Information (“Who am I?”)
  4. One for Triggers
  5. One for Actions.

Our goal is not to go in details on how to implement an IFTTT Service, but to illustrate how we can introduce a new methodology – a light-weight variant of formal modeling, with little ramp-up time – to help us:

  • Build a solid design for our service
  • Explain it accurately, unambiguously and completely to a tester or fellow developer.
  • Have a place to think when we add or modify features
  • Verify our design is free on errors

Today, if you wanted to implement an IFTTT service, you will read the documentation, come up with your own understanding, and start implementing.

We are going to do a different approach, and first build a model from the documentation, and then explain its benefits.

Our exploration will focus mostly around the authentication part of the oAuth2 mechanism used by IFTTT. If you are unfamiliar with oAuth2, a good introduction is here https://www.digitalocean.com/community/tutorials/an-introduction-to-oauth-2 (see following diagram).

We will be focusing on 1. And 2.

Remove your coder hat

Before we move forward, I want you to remove your coder hat for a minute. You have probably never done formal modeling (because between you and I, who wants to do “petri-net”?) so this is important.

Of course, I will highlight the places where you need to stop thinking like a coder, but take a deep breath! … and grasp the two following concepts of event driven state machines:

  1. Stop thinking about Implementation details: we don’t care about which language, library or socket mechanism you will program this. We are not at the implementation stage yet, as “we are only throwing ideas on paper”. Focus on extracting concepts rather than implementation details.
  2. Stop thinking in term of Data or Algorithm: We need to stop thinking about data and how they affect behaviors. You shall not think of a test as “if (data=’valid data’) then do something”, but prefer a two-state base thinking (the data is valid or the data is not valid).

The two rules are important. While I progress in the examples, it’s highly possible and encouraged that you will make a different choice when modelling or formalizing a concept or a behavior. It’s totally OK! We all draw things on a white board a different way!

Building the model (the interfaces)

We focus only on the authentication and authorization part, leaving the triggers and the actions for a later article.
In a first step, we define the interface that must to be supported by our IFTTT service.

The documentation explains General API requirements, including HTTPs endpoints, headers, responses, and payload. Based on our 2 rules we will ignore most of it.

But the document specifies some HTTP status codes:

Use the following set of HTTP response status codes:

  • 200 The request was a success.
  • 400 There was something wrong with incoming data from IFTTT. Provide an error response body to clarify what went wrong.
  • 401 IFTTT sent an OAuth2 access token that isn’t valid.
  • 404 IFTTT is trying to reach a URL that doesn’t exist.
  • 500 There was an error in your application logic.
  • 503 Your service is not available at the moment, but IFTTT should try again later.

While they seem like implementation details, those response status codes are actually “states” that an endpoint sends back. So they must be considered in our model. We will implement them as a enum:

enum HTTP_Response {e200, e400, e401, e404, e500, e503};

The service interface (we are naming it Interface_IFTTT_Service) must provide an oAuth2 Authorization endpoint (GET_authorize). In our model, we specify it like this:

interface Interface_IFTTT_Service
{
   in HTTP_Response GET_authorize();
}

It almost looks like java, but it isn’t. We have simply specified that our interface will need to provide and endpoint called “GET_authorize”. This event will return an HTTP_Response “state” that will let the consumer program decide of its path of action.

In the documentation, the authorize endpoint takes a clientid, an anti-forgery token and a redirect URL as input parameters (I ignored the response_type and scope as they are constant). They are data, and as explained earlier they should not influence the state machine or its behavior so we could do without them. But, it remains important to know they are there. To allow the developer to remember, the model lets you add parameters to “events” but they are only “placeholders” or “hints” (In fact they are only useful if you decide to code-generate the model, why their types is specified in between the $ signs). Here is how it is done:

extern clientid $clientid$;
extern antiforgerytoken $antiforgerytoken$;
extern URL $URL$
interface Interface_IFTTT_Service
{
      in HTTP_Response GET_authorize(in clientid id, in antiforgerytoken state, in URL redirect);
}

Then, the documentation explains what happens when IFTTT calls the “authorize” endpoint:

After the user is redirected to your authorization request endpoint, you should authenticate the user and prompt to grant IFTTT access to the user’s resources on your service. <…> Once a user authorizes IFTTT, you should redirect the user to IFTTT’s channel authorization URL along with an authorization code which IFTTT can exchange for a bearer token in the next step. <…> Should the user deny IFTTT access to your service, you should redirect them to IFTTT indicating access was denied.

This means we need to set up 2 things:

  1. We need to call the IFTTT’s channel authorization URL to either instruct IFTTT that the channel was accepted or was refused.
  2. We need to have a component to authenticate the user and prompt him/her to authorize IFTTT to access their resources.

The same URL is used for the IFTTT’s channel authorization regardless if the user accepts the request or not. As a coder, we would immediately think of something like “void authorize(bool bIsAuthorized)”, but this is one of the places where we need to lose our coder hat and forget about the implementation. We cannot use “parameters” to affect behaviors. Looking at it differently, only 2 possible “events” that can be fired: Authorized, or not Authorized. So, we will specify the IFTTT’s channel authorization interface with 2 separate events.

interface Interface_IFTTT_Channel
{
     in HTTP_Response GET_authorize_accepted(in authcode code, in antiforgerytoken URL);
     in HTTP_Response GET_authorize_refused();  
}

The parameters to the first events are only “placeholders” and the have no role in the logic.

Similarly, the interface for the GUI in charge of authorizing the user is simple and return an OK, NOT_OK Status. The Web_GUI interface is implemented by another component that we will have to write. For now, we only define its external behavior by a single interface Web_GUI.

enum Status {OK, NOT_OK};
interface Web_GUI
{
    in Status askforIFTTTauthorization();

Building the model (the behaviors)

If we sum up where we are, we have defined 3 interfaces.

  1. Interface_IFTTT_Service: The interface that our Service will need to implement.
  2. Interface_IFTTT_Channel: The interface we must redirect to when we have authorize the user (or not)
  3. Web_GUI: The interface that we will use to attempt to authorize the user.

With the coder hat on, we would be done. The behavior would be determined at a later stage by the implementation of the interfaces or by the documentations.

With models, we want to specify behavior before we have the implementation and before the final documentation is available, so we must specify how those interfaces behave. This is a complete new concept for a coder! We are not going to implement them, but we are going to specify their behavior!

Let’s look at the Web_GUI: We have defined only one event: askforIFTTTauthorization. Because of its definition, we know it returns a status: OK or NOT_OK.

But will it ever return OK? Or NOT_OK? Is it possible that it returns only OK? Or NOT_OK? What is it? What can I expect from askforIFTTTauthorization?

The designer intent is that OK is returned for “access granted” and NOT_OK is returned for “access denied”. Both outputs are possible.  In a model, you must describe this behavior.

This will be easily done by adding a “behaviour” clause to the interface:

interface Web_GUI
{
    in Status askforIFTTTauthorization();
    behaviour
    {
        on askforIFTTTauthorization: reply(Status.OK);
        on askforIFTTTauthorization: reply(Status.NOT_OK);
     }
}

Think of “behaviour” as a list of all the possible events.

In this case, we simply say that askforIFTTTauthorization may return OK and may return NOT_OK. And again, we are not implementing anything, we are just stating what is possible and what is not and under which conditions. If askforIFTTTauthorization was supposed to return only OK all the time, we would just remove the second line in “behaviour”. More advanced constructs can let you specify more advanced behaviors, but this will be another paper.

Now we know about “behaviour”, let’s revisit the Interface_IFTTT_Channel, the interface provided by IFFT to which you must redirect to.

interface Interface_IFTTT_Channel
{
    in HTTP_Response GET_authorize_accepted(in authcode code, in antiforgerytoken state);
    in HTTP_Response GET_authorize_refused();
}

This actually is a good example of where a documentation is not enough, and a typical scenario where developers and testers may have to make an educated guess to decide how to develop or how to test.

Here is a problem: What sort of response will GET_authorize_accepted or GET_authorize_refused send back? The documentation doesn’t specify it. Will it send only 200? Can it send a 503 or a 500? It is not clear (and a probable reason why the list is missing is that a complete documentation of all possibilities would be too extensive and would overwhelm the reader).

Unlike plain English, this is possible with a model, we can specify the list of possible reply as we did for the Web_GUI: We add the “behaviour” of both events and specify that they both will only return the 200 response and nothing else.

interface Interface_IFTTT_Channel
{
    in HTTP_Response GET_authorize_accepted(in authcode code, in antiforgerytoken state);
    in HTTP_Response GET_authorize_refused();
behaviour
    {
        on GET_authorize_accepted: { reply(HTTP_Response.e200); }
        on GET_authorize_refused: { reply(HTTP_Response.e200); }
    }
}

We have now modeled the complete three interfaces that we need to communicate between “end points” or components. While we never documented any implementation details, we have completely defined the behaviors of the interfaces, and left no room for interpretation.

We have defined the “Interface_IFTTT_Service” interface that defines services, and the two interfaces we will rely on: Interface_IFTTT_Channel, to forward requests to IFTTT and Web_GUI, to let us interact with our application GUI.

There is still one thing we must get done: we must generate an authorization code for the requested ID. The documentation states:

Once a user authorizes IFTTT, you should redirect the user to IFTTT’s channel authorization URL along with an authorization code which IFTTT can exchange for a bearer token in the next step.

As a coder, you would write the code to generate the code from the ID, probably using look up table or random number generation.

As a modeler, you don’t. We must think in term of states and events again: What can happen when you generate a code? It will work or it won’t. That’s it!

Accordingly, an interface to create a code from a clientid is simply specified as followed:

interface Interface_Token_Manager
{
    in Status generateAuthCode(in clientid id, out authcode code);
    behaviour
    {
        on generateAuthCode: reply(Status.OK); // You Do!
        on generateAuthCode: reply(Status.NOT_OK); // Or you don’t!
    }
}

This is a standard approach to represent an “algorithm problem” with a state machine. Focus on the states or the events, and not the data or the algorithm.

Building the model (the component)

It is now time to decide how to implement the service itself. We have not build any “implementation” yet as we have only created interfaces and specified how those interfaces function. This is an important distinction that must be understood. Interfaces are contract, by which a component will behave, and sometimes they can be misunderstood as the way a component is implemented. They are not.

We can begin implementing the service (My_IFTTT_Service) that will provides the Interface_IFTTT_Service, and relies on 3 components defined as a Interface_IFTTT_Channel, Web_GUI and Interface_Token_Manager.

component My_IFTTT_Service
{
     provides Interface_IFTTT_Service service;
     requires Interface_IFTTT_Channel channel;
     requires Web_GUI GUI;
     requires Interface_Token_Manager tokenmanager;
     <…>

When describing the implementation, please keep in mind you are not writing “Code” but you should focus on the logical progression of the state machine. The logic in our example is:

  • ask for authorization (through Web_GUI)
  • generate an authorization token (through Interface_Token_Manager)
  • finally “redirect” the call to the Interface_IFTTT_Channel.

The “implementation” will look like this:

behaviour
{
    on service.GET_authorize(id,state,URL):
    {
        Status authok = GUI.askforIFTTTauthorization();
        if (authok == Status.OK)
       {
            authcode code;
            Status codeok = tokenmanager.generateAuthCode(id,code);
            if (codeok == Status.OK)
            {
                HTTP_Response httpcode = channel.GET_authorize_accepted(code,state);
               reply(httpcode);
            }
            else
            {
                HTTP_Response httpcode = channel.GET_authorize_refused();
                reply(httpcode);
            }       
        }
        else
        {
            HTTP_Response httpcode = channel.GET_authorize_refused();
            reply(httpcode);
        }
}
<…>

While this looks like code, it isn’t…

If needed, Verum Dezyne lets you generate code from the model. Verum guarantees that a correct model will generate correct code. But you may decide to use it or not.

If you don’t want to generate code, the model can produce an “event table view” or a “state chart view”. They can guide the developer when writing the code, or the tester when verifying a behavior.

An Event table view can lead the developer or the tester

This is where it starts to pay off

So far, we have spent time creating a model, but what have we gained?

  1. We have replaced a 10-page documentation by a 1-page specification model: True, we have lost some implementation details about the structure of the HTTP payloads, but our goal is to document, to test and to validate the logic before the implementation.
  2. We have removed all room for interpretation or missing information. We have noticed that the documentation was weak about the type of error code that can be returned (we will address this later in more details).
  3. We have created a model that can be used to code generate an implementation skeleton, or to produce an “event table view” or a “state chart view” helping developer and tester with a complete specification.

But there are two more things we can do:

You can inject test scenarios in the system, deciding at each step what event will happen next. Here are a few diagrams created by Verum Dezyne simulator.
Remember that you have not written any code at that point, but you can visually operate the system as if it was built.

Someone calls GET_authorize, and the system as the GUI to confirm authorization

The GUI authorizes the user, then the system requests an Authorization code to tokenmanager.

Simply put, you are now testing a system that doesn’t exist yet!!

You can validate the system. This is a unique Dezyne feature. In a single click, you can ask Dezyne if there is anything wrong with your system. In our example, Dezyne will show the following:

Something is wrong with our system. The diagram shows that if GET_authorize is called, the service will return a HTTP_Response_e200. But this is not allowed by the system.

Why? We forgot to specify that the Interface_IFTTT_Service can return a HTTP Status of 200. This is an oversight from us. Can you guess what could have been the impact in a real scenario?

I want you to stop for a moment and realized what just happened. Dezyne has run all the possible scenarios in your system and has validated that no scenario can lead to your implementation breaking down! It validated that no action can break the specifications of all interfaces and their behavior or requirements.

Verum tests for a lot more, Completeness, Deadlock, Live Lock, Illegal actions… etc. and the errors don’t end up being caught in the QA phases.

To resolve the issue, We will simply modify our interface “behaviour” to specify the 200 is an possible response code.

interface Interface_IFTTT_Service
{
    in HTTP_Response GET_authorize();
    behaviour
    {
        on GET_authorize: { reply(HTTP_Response.e200); }
    }
}

The big moment: Validation of changes

Remember that every single endpoint in the IFTTT framework is an HTTP call and each call can return any of the HTTP Response code. But which one? Have we handle all of them? What if an endpoint starts sending 500 and never did before? What will be the impact?

This paper is using a simple example, but even here, the complexity is a lot more than first thought. The limitation of documentation is leaving a few holes and imprecisions that can lead to personal interpretation and conflicting opinions. Sadly, they will create coding, but also testing and validation problems.

Here are 2 examples to scratch your head with, regardless if you are a designer, a coder or a tester.

  1. What happens if the Interface_IFTTT_Channel interface returns error 503 (“Your service is not available now, but IFTTT should try again later”). Will you retry? How many times? Or will you simply pass it back through the Interface_IFTTT_Service interface (which seems to be what the documentation suggests)
  2. What about starting and shutting down your service? Can you build the strategy around it? This will force your service to return error 503 errors itself? Will it break anything?

A “formal” specification with a tool like Verum Dezyne let you write it all down, explore options, test and validate them and manage the impact of change.

Conclusion

This article is targeted toward organization interested in finding new ways to reduce or limit the increase in testing and validation costs. While most invest a subsequent amount of resources into DevOps to streamline the mechanics of quality insurance, we reviewed a complementary approach that targets designer and developer. It is based on a lightweight, but easy to learn, formal modeling product.

This gem is Verum Dezyne. Verum Dezyne let you:

  • Replace overwhelming documentations with a concise and complete specification of logics
  • Validate concepts and logics before you implement them
  • Easily address and validate change
  • Unambiguously let you share the definition and behavior of defined interface between designers, developers and testers.

If you are interested to learn more, please contact me at Stephane.Raynaud[AT]Emenda.com or download Dezyne at www.verum.com. You can download full example of the IFTTT Authorization mechanism here to test it out. I will soon produce the full IFTTT API and include it in the article.

Finally, here is Verum Dezyne showing the complete oAuth2 Protocol Flow (as seen in a picture earlier):



Get in touch

Full Name (required)

Work Email (required)

Title

Company

Contact number

Priority
HighResearch

Product
KlocworkUnderstandPerforceLattixSecurity InnovationSemiosCloneTracker

Platform
CC++C#JavaPythonPerlPHPOther Language

Certificate Standard(s)

Additional information