Charles Engelke's Blog

May 9, 2011

Handling Large Data

Filed under: Uncategorized — Charles Engelke @ 8:48 pm
Tags: , ,

This is my final session at Google IO Bootcamp this year, and the one I know least about going in.  We’re working with larger datasets and trying to get meaning from them, so there’s a lot of potential for us here.

This is the only session I’ve been in that wasn’t packed.  There are plenty of folks here, but there are empty seats and nobody sitting on the floors.  I’m sure people don’t find this as sexy as Android, Chrome, or App Engine.

We’re starting with the setup, which is pretty complicated.  We have to sign in as a special Google user, then join a group, then download and unpack some files, then go to some web pages…  And I’ve done all that.  Now I guess I’m ready.

We start with Google Storage for Developers, which is geared toward developers, not end users.  You can store unlimited objects within a whole lot of buckets, via a RESTful API.

We do an exercise where we create a text file, upload it, and make it readable to the world.  Then we fetch our neighbor’s file.

Next on to Big Query.  Which, for me, is a disaster.  Getting the tools set up and working is a mess under Windows, even with instructions.  And the meaning of the data we’re querying isn’t clear, making the exercise difficult.  But I got a few things to work.

Finally, we’ll use the Prediction API.  As for the exercises, I’ll try each one once then give up if it doesn’t work.  Messing with the installation and configuration takes my attention away from the actual tools.  Well, I think I’ve set it all up; it says it’s running.  I learned a lot of mechanics here, but don’t really understand what’s going on.  It should take about 5 minutes to do the prediction run I’m trying, and then I’ll see if I can make sense of the result.

Well, the result was “error” after 10 minutes of crunching.  I guess I’ll try it again, perhaps from a Linux box, someday.

That concludes IO Bootcamp this year.  All in all, it was well worth attending, even though I already knew some of the material.


HTML5 Development with the Chrome Dev Tools

Filed under: Uncategorized — Charles Engelke @ 6:47 pm
Tags: , ,

This is the session I’ve been most looking forward to here at IO Bootcamp.  I use these tools all the time, but I know I’m missing out on a lot that’s available.  We will be working with the examples at  That includes a TODO web application, and the slides from today’s talk.

We start with using Chrome to change CSS styles in the app.  I have no sense of design, so I just make things uglier.  But the various transforms and transitions are cool.  I delete items by having them slowly shrink to nothingness.

Moving on to feature detection, seeing what APIs are available to your web page.  The next exercise uses Modernizr to see if the Geolocation API is available (of course, it is in Chrome), and then we add use of it to the JavaScript for the page.  Just edit the script in the dev tools, click Ctrl-S to “save” the changes, and refresh the page.  When you’re done, go to the Resources tab, look at every version of the resource you’ve saved, and right-click to save the modified script to a file on your PC.  Very nice.

Next, profiling.  We use the Profiles tab in the dev tools and start by taking snapshots and comparing their differences.  Then start profiling by clicking on the circle, do some things, and stop the profiler by clicking the circle again.  You get a list of how much time each part of your code was running during the test.  The half second busy loop inserted in the example really eats up a lot of the application’s time.

You can set breakpoints in your code the normal way (by selecting the code to break on), even with conditional breaks.  But you can also set breakpoints on code that handles events, callbacks for XHR, and even on changes to DOM elements.  That last is really going to be useful.  But, things are going fast here and I’m not fully keeping up.  Still, knowing what’s there will let me seek out the details later.

This has been a nice session, even if I didn’t always keep up with the presenter.  I eventually covered it all, and have more tools available to me when I get back to work.

Google App Engine Workshop

Filed under: Uncategorized — Charles Engelke @ 4:44 pm
Tags: , ,

Our first lab of IO Bootcamp.  We have a dual-screen display up front, Java on the left, Python on the right.


Well, that kept me busy!  A good introduction leaving no time to blog.  See for the labs I did.

Google TV talk

Filed under: Uncategorized — Charles Engelke @ 2:34 pm
Tags: , ,

The Google TV session is just a talk, not a lab.  Based on a show of hands, it seems that the ratio of Android to web developers in the room is about three to one.  It’s clear that this talk is much more focused on Android development for Google TV, which has not been possible for regular developers yet.  And they aren’t going to announce any way to do it today.  Guess that’s for one of the keynotes at Google IO this week.  The Wednesday session on developing Android apps for Google TV is sure to tell how to deploy these apps.

Google TV isn’t intended to replace your cable connection; it’s to bring new content to your TV.  (Of course, lots of people I know want to replace the cable connection for existing content, and I think that’s inevitable.  But I guess Google doesn’t want to pick a fight with the entrenched providers.  Yet.)

We’re not going to hear about the next version of Google TV, but there are allusions to it, and how it has taken the feedback from the first year to heart.  The current Google TV is like the G1 Android phone – a first generation to prove what does and does not work.

So what’s different in developing for Google TV instead of mobile platforms?  There’s no touch screen, but there is a mouse (or mouse-like device).  There are only two important resolutions (1920 by 1080 and 1280 by 720), and only a landscape orientation.  And large icons, controls, and especially, large font sizes work best.  Don’t overload the users with too much information.  Even though we will focus on developing Android apps for Google TV, the speakers emphasize that Web apps are very often an excellent choice there.

You develop Android apps for Google TV with the same tools as developing for mobile.  You can configure the project parameters and emulators to have the same characteristics as a TV, using version 7 of the Android APIs and HDPI or XHDPI for screen resolution.  The “abstracted density” they recommend is 231 dpi.  The real density is much lower, but you view the screen from much further away so the density appears higher.  And thanks to TV overscan (a holdover from CRTs) you probably won’t see the whole screen, losing perhaps 10% of the screen at the edges.

Google App Engine Overview

Filed under: Uncategorized — Charles Engelke @ 1:23 pm
Tags: , ,

I’m going to do a little bit of Google IO blogging this year.  Probably not during the main event (there’s too much going on to stop and write about it, and I probably won’t even bring a notebook to it), but during Bootcamp I’ll be doing hands-on exercises and may have time to take a few notes here.

Here at Google IO Bootcamp, my first session is an App Engine overview (or is it AppEngine?).  I’ve used it before, but not for two or three years.  Well, actually my personal web site ( is hosted on it, but doesn’t use any of its capabilities.  There should be some new stuff here to learn.

App Engine is a cloud computing offering, specifically Platform as a Service (PaaS).  Other kinds of offerings would be Software as a Service (SaaS) such as Google Apps or, and Infrastructure as a Service (Iaas) like Amazone EC2 or Rackspace.  I love the idea of PaaS, and really want to find ways to use it, both personally and in my company, but the endpoints (SaaS and Iaas) are easier to get into.

App Engine is getting pretty heavily used.  In fact, it serves 1.5 billion page views per day.  I know one of the Royal Wedding sites was run on it.

You can create an App Engine application using either Python (the first supported language on it) or Java.  I know there are people who have run other languages on App Engine’s JVM (such as JRuby, Scala, Groovy, and others, even Jython if you want to use Python but access Java classes), but that’s too tricky for me to want to mess with.  Personally, I’ve always used Python while trying out App Engine.  It’s not a strong suit for me, but it’s a nice language and a good environment for it.

There’s an Eclipse plug-in for App Engine development, which I have installed, but I don’t much like IDEs.  I prefer a command line and text editor, and that’s how I went through the on-line tutorial to prepare for today.  Actually, I used the GUI Launcher they now offer, but still kept in my preferred text editor.

June 2, 2008

Conference Materials are now available

Filed under: Google IO 2008,RailsConf 2008 — Charles Engelke @ 1:57 pm
Tags: , , , , ,

RailsConf 2008 has made most of its presentation materials available for free download.

Google IO is beginning to make videos of its sessions available.

A lot of this stuff is really great, well worth your time.

May 29, 2008

Today at Google IO

Filed under: Google IO 2008,Uncategorized — Charles Engelke @ 5:17 pm
Tags: , , ,

…is going better for me than yesterday.  Sessions are mostly running a bit short, so it’s not nearly as frantic running from room to room.  I even had time to eat a sandwich, and as a result had a nice chat with another attendee.  It turned out she grew up in Gainesville, where I lived and still work.  And she’s working on technology very relevant to what our applications need, and is going to send me some information on new functionality as soon as it’s announced.

That kind of ad hoc meeting is something I like about conferences, and Google IO’s ultra-tight schedule with no break for lunch gets in the way.  I hope they change it next year.

Keynote, Day 2

Filed under: Google IO 2008,Uncategorized — Charles Engelke @ 5:12 pm
Tags: , , ,

Today started with a talk by Marissa Mayer, Google VP of Search and User Experience.  It was again very well done, and she’s an engaging speaker.

We heard some interesting things about how Google designs their pages.  For instance, they added the copyright notice at the bottom of the page not for legal reasons, but because in early user tests people kept waiting after the page was displayed before they’d enter a query.  Why?  They were waiting for the “rest of it”; the page couldn’t be loaded, it was too sparse.  So the copyright notice was added “as punctuation” to signal folks that the page was loaded and ready.

They do a lot of A/B (or A/B/C…) testing, where different users get slightly different pages from Google, and Google gathers and analyzes data about user behavior as a result.  They often find that very tiny changes changes can have a big effect.  The amount of white space between the Google logo and the separator bar on a results page?  The small amount they use results in greater user satisfaction and more Google searches than larger gaps.  Text ads with yellow backgrounds instead of blue?  Measurably better results.

What I took away from this was that you should listen to, or observe, what your users do, not what they say.  Mayer referenced a Henry Ford quote I hadn’t heard before:

“If I had asked people what they wanted, they would have said faster horses.”

After Hours at Google IO

Filed under: Google IO 2008,Uncategorized — Charles Engelke @ 1:33 am
Tags: , , ,

This evening Google had a reception with lots of food and drink, and music by Flight of the Conchords.

State of Ajax

Filed under: Google IO 2008,Uncategorized — Charles Engelke @ 1:27 am
Tags: , , ,

Dion Almaer and Ben Galbraith of Ajaxian told us about Ajax tools and frameworks, and the direction they see Ajax going.  Their talk was really well done, and very interesting.  Google taped all the talks and said they will post them sometime, probably on, so you’ll be able to see for yourselves.

Highlights I took down:

There are lots of Ajax frameworks and toolkits, which were created to do different kinds of things. But over time, the leaders all evolved to cover similar broad spectra of functionality. The four families that really matter now are Prototype, Dojo, jQuery, and GWT. (I wonder if including GWT is partly just an acknowledgment of the conference sponsor.)

Future directions are to make the browser as capable as your PC, and will eventually be strong competition for native GUI applications. Tools for that include Fluid (which I hadn’t heard of), Adobe Air, Mozilla Prism, and Google Gears.

Google IOKO

Filed under: Google IO 2008,Uncategorized — Charles Engelke @ 1:07 am
Tags: , , ,

Google’s logo for Google IO is the binary values of ASCII IO, with white circles for 1 and black circles for 0. Their slides do the same thing, but with large and small circles:

The t-shirts they’ve given us use the same coding to spell out Google IO. Except they spell Google KO instead:

Their own t-shirts say the same thing, but in black on white instead of white on black.

A mistake? Or a threat to potential competitors?

May 28, 2008

Keynote talk

Filed under: Google IO 2008,Uncategorized — Charles Engelke @ 7:59 pm
Tags: , , ,

This morning’s keynote for Google IO was the just about the best talk of that kind I’ve ever attended.  It was a broad overview of the topics from Vic Gundotra, a Google Vice President of Engineering, with several short talks on specific topics by relevant staff members.  It was full of useful information and whetted my appetite for the upcoming breakout sessions, and the speakers were all very polished and clearly rehearsed.

Some highlighted topics:

  • Google Javascript APIs.  Google has really opened its services up, and provides easy to use RESTful libraries for getting them to them in Javascript.  I’m not real interested right now in any single one of them, but the breadth of what’s available now is impressive.
  • Android.  Some very nice demos of mobile phones running Android, but I’m not clear on when this technology will actually be available for people like me to use.  I want it, but I bet today’s cellular provides, who always want to lock their users down, probably don’t want it.
  • AppEngine.  This is the technology that triggered my decision to attend the conference.  I’ve done some development in it, like it, and see great uses for it.  The big news for AppEngine is that it is now (as of today) open for anyone to sign up to use.  They also showed some approximate pricing for when it becomes a fully supported product, but committed to making it always free for low volume users.  “Low volume” will be defined in terms of storage, CPU, data transfers, and so on, but the free level will be enough to cover an average of 5 million page views per month.
  • OpenSocial.  Google’s supporting open APIs for social networking in a big way.  Personally, I’m not currently very interested in it.
  • Google Web Toolkit.  In addition to the native Javascript APIs, Google supports client-side development with GWT.  You write the applications in Java, and GWT compiles it to Javascript for deployment to the browser.  I don’t get it.  The speaker made a big deal of how Java was a much better language for this, and had grown-up development tools, but didn’t convince me.  I remember vendors pushing Cobol for developing in OS/2 and Windows for the same kinds of reasons.  Have you seen a lot of Cobol GUI programs?
  • Google Gears.  Now just called Gears.  This is a browser plug-in that lets you do all sorts of great things with Ajax, like store persistent data on the client and access client resources.  They are looking at this as a bleeding edge early preview of HTML 5, and I think it’s going to be important.

That’s not exhaustive, but it’s fairly complete.  I’m glad I came.

Google doesn’t scale

Filed under: Google IO 2008,Uncategorized — Charles Engelke @ 3:37 pm
Tags: , , ,

That is, the Google IO conference starting today in San Francisco isn’t scaling very well.  Registration was very, very slow for some reason, even though all that seemed to be happening was finding preprinted badges and giving them to attendees.  They simply couldn’t get people registered in the 90 minutes from when they opened at 8:00 until the keynote at 9:30, so they decided to let people attend sessions most of today without badges.  People need to go to the desks between sessions and get their badges by the end of the day.

Agenda scheduling doesn’t seem very practical.  Sessions go on non-stop all day with 15 minute gaps between them.  People are supposed to grab food during those breaks, but the food’s on a separate floor, and with such large crowds I don’t think you can even get to the other floor and back in that time.  The other choice would be to skip a session to eat, but the agenda is very strong.  I’m glad I ate a big breakfast.

The conference content has been great so far, with the most polished yet technical talks I’ve ever seen.  I hope that keeps up for the whole time.  Notes on sessions later as I get some breaks.  I’m not going to write during the sessions themselves.

Blog at