Charles Engelke’s Blog

December 11, 2011

Chrome Web App Bookshelf – Part 3

Filed under: Uncategorized — Charles Engelke @ 11:21 am

Note: this is part 3 of the Bookshelf Project I’m working on. Part 1 was a Chrome app “Hello, World” equivalent, and part 2 added basic functionality. This part will finally call an Amazon Web Service via JavaScript.

I’ve been approaching this project from the top down so far, starting with creating a nearly empty shell as a Chrome app, then putting in the necessary logic to make it perform a minimal function. The next thing to add is actually calling the Amazon Web Service that looks up an ISBN and returns information about the product. For that, I’m going to switch to a more bottom-up point of view, focusing at first on just that web service call.

I’m going to put the JavaScript for talking to AWS into a separate file called aws.js. That file needs to be loaded into the web page before any file that references it, and after any file it references. I’ll be using jQuery, so the script tags in the main.html page need to look like this:

   <script src="jquery.js"></script>
   <script src="aws.js"></script>
   <script src="main.js"></script>

Within aws.js I’m going to declare a single function that will be used as a constructor for an Amazon Web Services accessing object. That object will have methods to perform the actual calls. Any access to AWS requires credentials. The REST API (which is what I’ll use) requires an access key ID and a secret access key, so I’ll pass those as parameters to the constructor. The overall code will look like this:

var AWS = function(accessKeyId, secretAccessKey){
   var self = this;

   self.itemLookup = function(itemId, onSuccess, onError){
      // code to call AWS Product Advertising API ItemLookup function
   }
}

I could have just used this throughout, instead of defining self as a copy of it, but the JavaScript this variable is kind of tricky in what it references at various times. During the initial call to the constructor it definitely refers to the new object being created, so I’ll save it and use that saved value from then on. This library can be invoked with code like the following (with real access credentials and a sample ISBN in place of the examples):

var aws = new AWS('my access id', 'my secret key');
aws.itemLookup('1234567890',
               function(){alert('it worked');},
               function(){alert('it failed');});

Now, what does that missing code look like? AWS REST API calls use various HTTP methods, but most of them (including this one) just use GET with no special HTTP headers. So if we can build the right URL it will be easy to invoke it. The form of that URL is endpoint?parameters, where endpoint is a web address specific to the API family, and parameters is a normal query string of the form name1=value1&name2=value2&…namen=valuen where the names and values depend on the specific function.

The ItemLookup function I want to use is part of the AWS Product Advertising API. For that API, the endpoint is https://webservices.amazon.com/onca/xml (you can use the http version instead, but I always use the secure version if at all possible). Regardless of the function called, the parameters must always include:

  • Service – the value is always AWSECommerceService for this API
  • AWSAccessKeyId – the accessKeyId part of the credentials
  • AssociateTag – this is a new requirement since November 2011; I’m going to have to add this to either the code, the constructor call, or the method call
  • Operation – the name of the function, ItemLookup in this case
  • Timestamp – when the request was created; AWS will only honor it for 15 minutes to prevent future “replay” attacks
  • Signature – a cryptographic signature created from all the other parameters and the secret access key

The ItemLookup function requires additional parameters:

  • ItemId – identifies the item to find, or a comma-separated list of up to ten items
  • ResponseGroup – tells how much detail we want in the response; I’m going to have to experiment with the various possibilities to see which groups include the data I want

Instead of just creating the query string directly out of these parameters, I’ll use an array of name/value pairs in my code, create the signature from that, then build the query string to use. The code shapes up as follows:

      var params = [];
      params.push({name: "Service", value: "AWSECommerceService"});
      params.push({name: "AWSAccessKeyId", value: accessKeyId});
      params.push({name: "AssociateTag", value: associateTag});
      params.push({name: "Operation", value: "ItemLookup"});
      params.push({name: "Timestamp", value: formattedTimestamp()});
      params.push({name: "ItemId", value: itemId});
      params.push({name: "ResponseGroup", value: "ItemAttributes"});

      var signature = computeSignature(params, secretAccessKey);
      params.push({name: "Signature", value: signature});

      var queryString = createQueryString(params);
      var url = "https://webservices.amazon.com/onca/xml?"+queryString;

This code assumes that a variable named associateTag already exists. I’m going to add it as a parameter to the main constructor function to make that happen. This code also invokes several helper functions: formattedTimestamp, computeSignature, and createQueryString. I’m going to have to write them inside of this library. The code then needs to make an HTTP GET request to that URL and (if the call is successful) pull the desired data out of the response body, passing that to the onSuccess handler.

I’ll tackle the new functions first, from easiest to hardest. formattedTimestamp just needs to return the current time in a standard format: YYYY-MM-DDTHH:MM:SSZ (the T is a separator between date and time, and the Z indicates UTC time). Actually, I could cheat here if I wanted to. I’ve found that any date in the future is accepted by AWS, so I could hard code the result of this function as 9999-12-31T23:59:59Z. But that strikes me as a loophole in the service that may be closed in the future, so I’ll play fair here.

   function formattedTimestamp(){
      var now = new Date();

      var year = now.getUTCFullYear();

      var month = now.getUTCMonth()+1; // otherwise gives 0..11 instead of 1..12
      if (month < 10) { month = '0' + month; } // leading 0 if needed

      var day = now.getUTCDate();
      if (day < 10) { day = '0' + day; }

      var hour = now.getUTCHours();
      if (hour < 10) { hour = '0' + hour; }

      var minute = now.getUTCMinutes();
      if (minute < 10) { minute = '0' + minute; }

      var second = now.getUTCSeconds();
      if (second < 10) { second = '0' + second; }

      return year+'-'+month+'-'+day+'T'+hour+':'+minute+':'+second+'Z';
   }

createQueryString is a bit trickier, but not much. I just need to build a query string in the standard format. However, I have to remember to URI encode the names and values, in case they include any special characters. And I’m going to add the parameters in sorted order by name, because that will be useful later when computing a signature according to AWS’s rules.

   function createQueryString(params){
      var queryPart = [];
      var i;

      params.sort(byNameField);

      for(i=0; i<params.length; i++){
         queryPart.push(encodeURIComponent(params[i].name) +
                        '=' +
                        encodeURIComponent(params[i].value));
      }

      return queryPart.join("&");

      function byNameField(a, b){
         if (a.name < b.name) { return -1; }
         if (a.name > b.name) { return 1; }
         return 0;
      }
   }

This function actually changes the parameter it is passed: it sorts the array it is given. It would be better behaved to make a copy and sort the copy, but instead I’ll just note this fact and keep it simpler.

Now it’s time for the hard one, computeSignature. Actually, with the steps already taken it’s not that hard any more. The AWS signature is a 256-bit SHA HMAC of a special string that includes the HTTP method, the host name of the end point, the path of the request, and the unsigned query string (as created above), signed using the secret access key. Of course, doing that cryptographic operation would be pretty hard, but I don’t have to. I can use the Stanford JavaScript Crypto Library. I downloaded the minified version of it and put it in my project folder in the file sjcl.js, and loaded it in the main page with a script tag before the aws.js reference there. With that in place, the computeSignature function is not too hard:

   function computeSignature(params, secretAccessKey){

      var stringToSign = 'GET\nwebservices.amazon.com\n/onca/xml\n' +
                         createQueryString(params);

      var key = sjcl.codec.utf8String.toBits(secretAccessKey);
      var hmac = new sjcl.misc.hmac(key, sjcl.hash.sha256);
      var signature = hmac.encrypt(stringToSign);
      signature = sjcl.codec.base64.fromBits(signature);

      return signature;
   }

The signing looks more complicated than it is because the hmac.encrypt function operates on bit strings, not normal JavaScript character strings, so there are extra steps to convert those back and forth.

With those preliminaries out of the way the code can create the URL to use to call the service. I’ll use jQuery to make the Ajax call:

      jQuery.ajax({
         type : "GET",
         url: url,
         data: null,
         success: extractAndReturnResult,
         error: returnErrorMessage
      });

This will call the URL and send the successful response to extractAndReturnResult or an unsuccessful one to returnErrorMessage. I’ve got to write those two functions, and then should be done.

      function extractAndReturnResult(data, status, xhr){
         onSuccess(xhr.responseText);
      }

      function returnErrorMessage(xhr, status, error){
         onError('Ajax request failed with status message '+status);
      }

Both these functions need a lot of work! In particular, extractAndReturnResult doesn’t do what its name says at all. It just returns the raw response from Amazon. But that’s going to be useful for exploring the different options on the call, so I’m keeping it that way for now.

Putting all the above together (and adding the necessary associateTag parameter to the constructor), the aws.js file is:

var AWS = function(accessKeyId, secretAccessKey, associateTag){
   var self = this;

   self.itemLookup = function(itemId, onSuccess, onError){
      var params = [];
      params.push({name: "Service", value: "AWSECommerceService"});
      params.push({name: "AWSAccessKeyId", value: accessKeyId});
      params.push({name: "AssociateTag", value: associateTag});
      params.push({name: "Operation", value: "ItemLookup"});
      params.push({name: "Timestamp", value: formattedTimestamp()});
      params.push({name: "ItemId", value: itemId});
      params.push({name: "ResponseGroup", value: "ItemAttributes"});

      var signature = computeSignature(params, secretAccessKey);
      params.push({name: "Signature", value: signature});

      var queryString = createQueryString(params);
      var url = "https://webservices.amazon.com/onca/xml?"+queryString;

      jQuery.ajax({
         type : "GET",
         url: url,
         data: null,
         success: extractAndReturnResult,
         error: returnErrorMessage
      });

      function extractAndReturnResult(data, status, xhr){
         onSuccess(xhr.responseText);
      }

      function returnErrorMessage(xhr, status, error){
         onError('Ajax request failed with status message '+status);
      }
   }

   function formattedTimestamp(){
      var now = new Date();

      var year = now.getUTCFullYear();

      var month = now.getUTCMonth()+1; // otherwise gives 0..11 instead of 1..12
      if (month < 10) { month = '0' + month; } // leading 0 if needed

      var day = now.getUTCDate();
      if (day < 10) { day = '0' + day; }

      var hour = now.getUTCHours();
      if (hour < 10) { hour = '0' + hour; }

      var minute = now.getUTCMinutes();
      if (minute < 10) { minute = '0' + minute; }

      var second = now.getUTCSeconds();
      if (second < 10) { second = '0' + second; }

      return year+'-'+month+'-'+day+'T'+hour+':'+minute+':'+second+'Z';
   }

   function createQueryString(params){
      var queryPart = [];
      var i;

      params.sort(byNameField);

      for(i=0; i<params.length; i++){
         queryPart.push(encodeURIComponent(params[i].name) +
                        '=' +
                        encodeURIComponent(params[i].value));
      }

      return queryPart.join("&");

      function byNameField(a, b){
         if (a.name < b.name) { return -1; }
         if (a.name > b.name) { return 1; }
         return 0;
      }
   }

   function computeSignature(params, secretAccessKey){

      var stringToSign = 'GET\nwebservices.amazon.com\n/onca/xml\n' +
                         createQueryString(params);

      var key = sjcl.codec.utf8String.toBits(secretAccessKey);
      var hmac = new sjcl.misc.hmac(key, sjcl.hash.sha256);
      var signature = hmac.encrypt(stringToSign);
      signature = sjcl.codec.base64.fromBits(signature);

      return signature;
   }
}

The main.js file needs a little tweaking now to call this properly. The new version is:

$(document).ready(function(){
   var aws = new AWS('my access id', 'my secret key', 'my associate id');
   $("#lookup").click(lookupIsbn);

   function lookupIsbn(){
      var isbn = $("#isbn").attr("value");
      aws.itemLookup(isbn,
                     function(message){
                        message = message.replace(/&/g, "&amp;");
                        message = message.replace(/</g, "&lt;");
                        message = message.replace(/>/g, "&gt;");
                        $("#results").append(message);
                     },
                     function(message){
                        alert("Something went wrong: "+message);
                     }
                     );
   }
});

There are only two real changes here. First, the constructor is called first, to get an object for working with AWS before anything else happens. Second, instead of just dumping the response message in the web page the code first replaces all special HTML characters with their equivalent character entities. That way, the message will be shown as text instead of interpreted as HTML, possibly including code.

And now I’m ready to go. I put an ISBN in the text box and pressed the button… and got this:

Error message: Something went wrong: Ajax request failed with status message error

That doesn’t tell me much, though. But the JavaScript console (opened with Control-Shift-J) is more helpful:

Origin is not allowed by Access-Control-Allow-Origin

Web browsers do not allow pages to make requests to other addresses, so this request was disallowed. That’s a security restriction. There is a new Cross-Origin Resource Sharing specification that allows this when the target web site decides it is safe to do, but AWS doesn’t support it. Not yet, anyway; I’m still hoping. However, Chrome apps can bypass this restriction if they ask. The manifest.json file needs to be changed to request this:

{
   "name": "Books to Buy",
   "description": "Keep a list of books to buy on Amazon, with their price and availability",
   "version": "1",
   "app": {
      "launch": {
         "local_path": "main.html"
      }
   },
   "icons": {
      "16":    "icon_16.png",
      "128":   "icon_128.png"
   },
   "permissions": [
      "https://webservices.amazon.com/*"
   ]
}

The permissions entry tells Chrome to allow this app to make requests to any URL matching the wild card given. I removed and reinstalled the app after making this change, and tried again. And got this:

Page showing XML response from AWS

Success! Sort of. A lot of XML came back from the request, and I need to pull the necessary data out of it. I also need to explore various response groups to get the data I need. And all that will be the subject of the next post in this series.

About these ads

4 Comments

  1. [...] on. Part 1 was a Chrome app “Hello, World” equivalent, part 2 added basic functionality, and part 3 finally called an Amazon web service. This part will finally parse the web service result and [...]

    Pingback by Chrome Web App Bookshelf – Part 4 « Charles Engelke’s Blog — December 28, 2011 @ 4:25 pm

  2. [...] on. Part 1 was a Chrome app “Hello, World” equivalent, part 2 added basic functionality, part 3 finally called an Amazon web service, and part 4 parsed the web service result. This part will [...]

    Pingback by Chrome Web App Bookshelf – Part 5 « Charles Engelke’s Blog — January 1, 2012 @ 6:22 pm

  3. [...] on. Part 1 was a Chrome app “Hello, World” equivalent, part 2 added basic functionality, part 3 finally called an Amazon web service, part 4 parsed the web service result, and part 5 actually [...]

    Pingback by Chrome Web App Bookshelf – Part 6 « Charles Engelke’s Blog — January 7, 2012 @ 3:52 pm

  4. [...] on. Part 1 was a Chrome app “Hello, World” equivalent, part 2 added basic functionality, part 3 finally called an Amazon web service, part 4 parsed the web service result, part 5 was useful [...]

    Pingback by Chrome Web App Bookshelf – Part 7 of 7 « Charles Engelke’s Blog — January 8, 2012 @ 1:37 pm


RSS feed for comments on this post.

The Rubric Theme. Blog at WordPress.com.

Follow

Get every new post delivered to your Inbox.

Join 44 other followers

%d bloggers like this: