Charles Engelke's Blog

March 4, 2009

Update: Web Hosting with Google AppEngine

Filed under: Uncategorized — Charles Engelke @ 9:55 pm
Tags: ,

I’ve been getting some comments recently on last summer’s post on Google AppEngine for web hosting.  I guess I’m not the only one who wants to add some simple static web pages to my Google Apps hosted services.

Most commenters are reporting success with the method, but Peter G has a problem getting the pages to work right at the naked domain name.  That is, pages at http://www.example.com work fine, but the same pages at example.com don’t.  Why did it work for me, but not him?  I had no idea.

I had to set up a new domain for a friend anyway, so I paid close attention to how to set this up.  And found that I couldn’t make the naked domain name work, either.  A little clicking around found the problem:

I’d like to map my app to http://myurl.com (also known as a naked domain).

Due to recent changes, Google App Engine no longer supports mapping your app to a naked domain.

Well, that’s just great.  I don’t know what recent changes caused this, but you apparently can no longer make AppEngine serve pages for a naked domain name.

The documentation does describe one solution: URL forwarding.  For this to work you need your domain name service provider (which is probably your domain registrar) to return an HTTP redirect response to any requests at the naked domain.  Then any web request to example.com/some/page gets redirected to http://www.example.com/some/page, which AppEngine will serve up fine.

The new domain I was setting up was registered with eNom, so I used their DNS dashboard to add a record saying that requests to @ (the symbol for a naked domain) should generate a URL redirect to http://www.example.com, and it all just works.

Hope this is helps anyone having trouble with this.  There is one potential problem, though.  The solution depends on services from your domain registrar, not Google Apps, so you may be out of luck if your registrar doesn’t offer URL redirection.

July 31, 2008

AppEngine “Rewrite Rules”

Filed under: How To — Charles Engelke @ 8:43 pm
Tags: ,

Yesterday I showed how to get a basic static web site hosted on Google AppEngine.  But I need a little bit more than a purely static site.  My blog used to be hosted at engelke.com/blog and http://www.engelke.com/blog, but now it’s at blog.engelke.com.  So any existing links to my blog entries will break.  That is, they’ll break unless requests to engelke.com/blog/something get redirected to blog.engelke.com/something.

I did this with a rewrite rule when my site was being served with the Apache httpd server.  The specific rule I had was:

RewriteRule ^/blog(.*)$  http://blog.engelke.com$1  [R]

That says that a request that starts with /blog will be redirected to one at blog.engelke.com followed by whatever followed the word blog.  Which is actually kind of wrong; for example, this will redirect a request made to engelke.com/blogging to blog.engelke.comging, which is nonsense.  I really should have given two rules:

RewriteRule ^/blog/(.*)$  https://blog.engelke.com/$1 [R]
RewriteRule ^/blog$  https://blog.engelke.com/        [R]

The first rule says that anything starting with /blog/ will be redirected to blog.engelke.com/ followed by whatever was after blog/.  That handles everything but a request to engelke.com/blog just by itself.  The second rule handles that.

AppEngine’s url: mapping rules in app.yaml kind of look like rewrite rules.  For example, the mapping:

- url: (.*)/
  static_files: static\1/index.html

That says any request that ends with a slash should be served with the file at that relative directory followed by index.html.  But that’s not a redirect.  The browser still requests the URL ending with the slash; the server just returns the contents of a particular file.

No, if we want redirection similar to the rewrite rules I had, we’ll need to write an AppEngine script.  First we add two url: mappings to app.yaml.  These should be placed before the existing ones:

- url: /blog/.*
  script: redirector.py

- url: /blog
  script: redirector.py

These correspond to the first part of the old rewrite rules, but they don’t tell what the response should be.  Instead, they just tell AppEngine to run the redirector.py script and let it figure out what to do.

I started writing the redirector.py script with a standard skeleton from the AppEngine documentation:

from google.appengine.ext import webapp
from google.appengine.ext.webapp.util import run_wsgi_app

application = webapp.WSGIApplication(
            [
            # pattern to handler mapping pairs go here
            ])

def main():
   run_wsgi_app(application)

if __name__ == "__main__":
    main()

The “pattern to hander mapping pairs” are each a string that’s a regular expression to match and the name of a class to handle those request paths.  The following two lines match both patterns and invoke the BlogHandler class for them (the ‘r’ in front a string just tells Python it’s a “raw” string, so Python doesn’t interpret any characters in any special way):

     (r'^/blog/(.*)', BlogHandler),
     (r'^/blog$',     BlogHandler)

The BlogHandler class contains methods for each HTTP method to be handled.  We’ll just handle GET and HEAD requests, since those are the only kind our blog will respond to anyway.  The pattern matched inside the parentheses will be passed as a parameter to each of these methods (and self is always going to be passed as the first parameter in Python).  The second pattern doesn’t have any parentheses, so that matching string parameter will be missing; our handlers will have to accept that.

Here’s the code for the BlogHandler class:

class BlogHandler(webapp.RequestHandler):
    def get(self, tail = ''):
        self.redirect('https://blog.engelke.com/'+tail, permanent = True)
    def head(self, tail = ''):
        self.redirect('https://blog.engelke.com/'+tail, permanent = True)

This code is pretty simple and should be self-explanatory.  If no second parameter is given, the methods act as if an empty string was passed.  The optional “permanent = True” parameter makes the redirect an HTTP status 301 Moved Permanently so that client programs can know to never bother to look at the old address.

Complete code

The new app.yaml file is:

application: engelkeweb
version: 1
runtime: python
api_version: 1

handlers:
- url: /blog/.*
  script: redirector.py

- url: /blog
  script: redirector.py

- url: (.*)/
  static_files: static\1/index.html
  upload: static/index.html

- url: /
  static_dir: static

The redirector.py script is:

from google.appengine.ext import webapp
from google.appengine.ext.webapp.util import run_wsgi_app

class BlogHandler(webapp.RequestHandler):
    def get(self, tail = ''):
        self.redirect('https://blog.engelke.com/'+tail, permanent = True)
    def head(self, tail = ''):
        self.redirect('https://blog.engelke.com/'+tail, permanent = True)

application = webapp.WSGIApplication(
            [
                (r'^/blog/(.*)', BlogHandler),
                (r'^/blog$',     BlogHandler)
            ])

def main():
   run_wsgi_app(application)

if __name__ == "__main__":
    main()

And that’s my AppEngine hosted web site.  The one problem left is that requests that should have a trailing slash, but don’t, won’t work (other than /blog, handled above).  But there are only two such possible pages on my site, and I’ve never posted links without the trailing slashes.  So I don’t need to deal with them.

Though I did.  I just added mappings for those two raw names (/xhtmlref and /charles/TPC5) to the redirector.py script, and added a class to redirect those requests to the right URL with the trailing slash.  But that doesn’t show any new ideas, so I’m not going to put the additional code here.

Handy GrandCentral Hack – and a Few Complaints

Filed under: Google IO 2008 — Charles Engelke @ 5:48 pm
Tags: ,

[Update on August 1: Chad, the author of the Firefox extension I refer to below, commented and asked for details of the problem I had.  His comment and my answer are available for this post.  In a nutshell, I reinstalled the extension and this time it’s working for me and not causing any problems.  Great responsiveness from a software author, and a great tool for GrandCentral users  Thanks Chad!]

GrandCentral is a service that gives you a virtual phone number that can be pointed at one or more real phone numbers.  If someone phones the GrandCentral number, all the real phones get the call.  There are a lot of extra features, too, like e-mail notification of calls, web voice-mail, switching which phone number you’re using while a call is underway, and more.  But you probably can’t get it because it’s in an invitation-only beta period.

Unfortunately, GrandCentral has been “in beta” for quite a while now.  I got my account in May 2007.  I never seriously used it because I wasn’t sure it would last, and I also wasn’t sure I wanted to be quite that easily reached.  But I saw a tip on Lifehacker the other day about it and gave it a try.  It turns out that you can call anybody from your phone for free using GrandCentral.  Just put the person’s phone number in your address book and you’ll see a Call button.  Click it, and GrandCentral first calls you.  When you answer, it announces that’s it’s placing your call, and it calls the number you selected.  It works perfectly.  I don’t know whether this works for calls outside the US, but I doubt it.

This is handy for me to call the office when I’m working from home.  I prefer to use my home phone instead of my cell phone due to coverage issues.  But our office is a long-distance call, so I use the 800 number.  Which connects me to the receptionist who I have to keep bothering in order to connect to the right extension.  Now with GrandCentral I call the other employee’s direct-dial number from my home phone for free, and don’t have to take up the receptionist’s time.

This trick does require that the person be in your GrandCentral address book, which is a bit of a pain.  The Lifehacker tip was about a Firefox extension that would let you make this kind of call to any number on any web page, but the extension didn’t work for me and caused some other problems in Firefox, so I removed it.  If I could upload a file of phone numbers to the address book this wouldn’t be much of a hassle, but I can’t.  GrandCentral makes you enter each contact individually on the web.

Which brings me to my complaints.  GrandCentral now belongs to Google, and they don’t seem to be doing anything with it.  It’s not integrated with any other Google services (like my GMail contact list).  It’s got an awful flash-heavy user interface that makes every navigation step slow and difficult.  And it doesn’t have any of Google’s normal thoughtful touches (like being able to upload an address book).

I can’t see how anybody can make money with the GrandCentral service, at least as it now exists, which may be the problem.  Perhaps it’s almost an orphan product inside Google.  It wasn’t even mentioned at the Google IO meeting.  Too bad, because it can be really useful.  I guess I’ll use it while I wait to see what Google does with it.

July 30, 2008

Google AppEngine for web hosting

Filed under: How To — Charles Engelke @ 10:45 pm
Tags: ,

Shortly after I first got my own domain I decided to run all the services myself.  It was a good experience but eventually got old, so I’ve been migrating everything to hosted services.  Free hosted services if possible.

DNS went first.  When I first registered my domain the registrar would only register my nameservers, which I then had to manage.  But now they (and apparently all registrars) will manage my nameservers for me through a simple web dashboard.  Step one was done and it was free as part of my domain registration.

Next came e-mail.  Before I ran my own mail servers I had a mail hosting provider, which cost $25 per month back then.  But now I can get mail hosting for free from Google Apps.  It’s easy to use and works great.  If you don’t have your own domain name, you can also register one through them for just $10 per year.

I’d been running my own Movable Type installation for my blog.  I considered moving to hosted TypePad, but I’d heard good things about WordPressWordPress.com now hosts my blog, also for free, but I decided to pay $10 per year to have it at my own domain name.  I love their service.

And now I’d migrated everything but my core web site, which is very minimal.  All I’ve got is a home page and a small number of other pages, because I’d already migrated my photos to Picasa web albums and my presentation files to Google Docs (which is also available as part of Google Apps), all for free.  There is one hitch, though: when I moved to a hosted blog all my old URLs changed, so my main web site has to redirect requests for the old URLs to the correct new ones.  But still, I didn’t want to pay for full web hosting for what is essentially just a starting point.

Google AppEngine to the rescue.  It’s a neat platform for a fancy web application, but it can also be a minimal web server.  It’s hosting my site now, and here’s what’s involved in doing it.

First, register an application name for AppEngine.  You’ll need a regular Google Mail account for this, but that’s easy to get.  It doesn’t matter what you call it; mine was engelkeweb.

Next, create a local folder on your machine to hold your application.  Put a folder inside that containing all your web pages.  It can be called anything, but the AppEngine examples always call it static, so that’s what I’ll do.  If your application folder is at C:\MyAppEngine then this folder will be C:\MyAppEngine\static, and the main home page would probably be C:\MyAppEngine\static\index.html.

Now create a text file called app.yaml in the main folder (in this example, at C:\MyAppEngine\app.yaml) with the following contents (change the engelkeweb entry to the name you registered):

application: engelkeweb
version: 1
runtime: python
api_version: 1

handlers:
- url: (.*)/
  static_files: static\1/index.html
  upload: static/index.html

- url: /
  static_dir: static

Now upload your application (actually, just the static pages and the app.yaml file) using the command:

appcfg.py update C:\MyAppEngine

Voila!  You’ve got a simple static web site.  The second handler specification (for url: /, handled by static_dir: static) means that AppEngine will serve the page static\folder\xyz.ext whenever a user requests http://engelkeweb.appspot.com/folder/xyz.ext.  That’s 95% of a static web site. And the first handler specification (for url: (.*)/) says that any request to a URL ending with a slash should return the index.html page in that folder.  So a request to http://engelkeweb.appspot.com/folder/ would return the file static\folder\index.html.

Note that the order of the handler specifications matter, since the first one matched is what is returned.

You can use your Google Apps settings page to make this site available at your domain’s normal address, too (so in my case, that same page is returned for http://www.engelke.com/folder/xyz.ext and http://engelke.com/folder/xyz.ext).  You have to set up some DNS entries for that, but the Google Apps pages give clear explanations of what to do.

That’s how I’m now serving up engelke.com.  But what about the redirection for my blog pages?  Every URL of the form http://engelke.com/blog/something should now be redirected to https://blog.engelke.com/something.  Doing that requires actually writing a very simple Python handler for the site, just 20 lines long, and adding another handler specification in the app.yaml file.  I’ll cover that in my next post.

July 29, 2008

Cuil Facing Heat

Filed under: Uncategorized — Charles Engelke @ 12:15 pm
Tags: , ,

The new search engine Cuil made a big mistake: it didn’t think about how early adopters and influencers would react to it.

What’s the first thing everyone does with a new search engine?  Search for themselves, of course.  (I suppose not everyone is so ego-driven, but then those people probably don’t write articles or blog.)  So that’s what the influencers did (and so did I).  And the results Cuil returns for that tend to stink.

I have a pretty unusual last name that’s also my domain name, so when I search for engelke on Google I find links referring to me in the first page of results.  I’m at the second search result just now.  But when I search on Cuil I find no references to me on the first page (or second, or third, or… I gave up looking).  When I search for charles engelke (no quotes) on Google, seven of the ten links refer to me.  The same search on Cuil is more ego-reassuring than the first was: nine of the eleven links refer to me.

But Cuil also has a big “Explore by Category” box on that page that just seems bizarre: the categories are Irish Comedians, German Comedians, The Simpsons Characters, American Stand-up Comedians, Second-city Alumni, and American Comedians.  Not a single link on the results page refers to anyone related to any of those things, just to me and to a physicist (who, oddly enough, was at SUNY at Stony Brook the same time as me).

Finally, searching for “charles engelke” (with the quotes) on Google gives about the same result as without the quotes (eight of ten), on Cuil it looks like the quotes made no difference at all, except the weird “Explore by Category” box is gone.

I think that one reason for the big differences I’m seeing is that Cuil is giving much more weight to German language pages than Google seems to.  I don’t know if Google is short-changing German because my search language is English, or if Cuil is over-emphasizing it.  But pretty much all the results on Cuil (and the first one on Google) are for Anke Engelke, a German comic actress and talk show host.  And even though she doesn’t show up on the Cuil results page for charles engelke, I think she’s the reason for the “Explore by Category” box with all the different kinds of comedians listed.

June 2, 2008

Conference Materials are now available

Filed under: Google IO 2008,RailsConf 2008 — Charles Engelke @ 1:57 pm
Tags: , , , , ,

RailsConf 2008 has made most of its presentation materials available for free download.

Google IO is beginning to make videos of its sessions available.

A lot of this stuff is really great, well worth your time.

May 29, 2008

Today at Google IO

Filed under: Google IO 2008,Uncategorized — Charles Engelke @ 5:17 pm
Tags: , , ,

…is going better for me than yesterday.  Sessions are mostly running a bit short, so it’s not nearly as frantic running from room to room.  I even had time to eat a sandwich, and as a result had a nice chat with another attendee.  It turned out she grew up in Gainesville, where I lived and still work.  And she’s working on technology very relevant to what our applications need, and is going to send me some information on new functionality as soon as it’s announced.

That kind of ad hoc meeting is something I like about conferences, and Google IO’s ultra-tight schedule with no break for lunch gets in the way.  I hope they change it next year.

Keynote, Day 2

Filed under: Google IO 2008,Uncategorized — Charles Engelke @ 5:12 pm
Tags: , , ,

Today started with a talk by Marissa Mayer, Google VP of Search and User Experience.  It was again very well done, and she’s an engaging speaker.

We heard some interesting things about how Google designs their pages.  For instance, they added the copyright notice at the bottom of the page not for legal reasons, but because in early user tests people kept waiting after the page was displayed before they’d enter a query.  Why?  They were waiting for the “rest of it”; the page couldn’t be loaded, it was too sparse.  So the copyright notice was added “as punctuation” to signal folks that the page was loaded and ready.

They do a lot of A/B (or A/B/C…) testing, where different users get slightly different pages from Google, and Google gathers and analyzes data about user behavior as a result.  They often find that very tiny changes changes can have a big effect.  The amount of white space between the Google logo and the separator bar on a results page?  The small amount they use results in greater user satisfaction and more Google searches than larger gaps.  Text ads with yellow backgrounds instead of blue?  Measurably better results.

What I took away from this was that you should listen to, or observe, what your users do, not what they say.  Mayer referenced a Henry Ford quote I hadn’t heard before:

“If I had asked people what they wanted, they would have said faster horses.”

After Hours at Google IO

Filed under: Google IO 2008,Uncategorized — Charles Engelke @ 1:33 am
Tags: , , ,

This evening Google had a reception with lots of food and drink, and music by Flight of the Conchords.

State of Ajax

Filed under: Google IO 2008,Uncategorized — Charles Engelke @ 1:27 am
Tags: , , ,

Dion Almaer and Ben Galbraith of Ajaxian told us about Ajax tools and frameworks, and the direction they see Ajax going.  Their talk was really well done, and very interesting.  Google taped all the talks and said they will post them sometime, probably on code.google.com, so you’ll be able to see for yourselves.

Highlights I took down:

There are lots of Ajax frameworks and toolkits, which were created to do different kinds of things. But over time, the leaders all evolved to cover similar broad spectra of functionality. The four families that really matter now are Prototype, Dojo, jQuery, and GWT. (I wonder if including GWT is partly just an acknowledgment of the conference sponsor.)

Future directions are to make the browser as capable as your PC, and will eventually be strong competition for native GUI applications. Tools for that include Fluid (which I hadn’t heard of), Adobe Air, Mozilla Prism, and Google Gears.

Google IOKO

Filed under: Google IO 2008,Uncategorized — Charles Engelke @ 1:07 am
Tags: , , ,

Google’s logo for Google IO is the binary values of ASCII IO, with white circles for 1 and black circles for 0. Their slides do the same thing, but with large and small circles:

The t-shirts they’ve given us use the same coding to spell out Google IO. Except they spell Google KO instead:

Their own t-shirts say the same thing, but in black on white instead of white on black.

A mistake? Or a threat to potential competitors?

May 28, 2008

Keynote talk

Filed under: Google IO 2008,Uncategorized — Charles Engelke @ 7:59 pm
Tags: , , ,

This morning’s keynote for Google IO was the just about the best talk of that kind I’ve ever attended.  It was a broad overview of the topics from Vic Gundotra, a Google Vice President of Engineering, with several short talks on specific topics by relevant staff members.  It was full of useful information and whetted my appetite for the upcoming breakout sessions, and the speakers were all very polished and clearly rehearsed.

Some highlighted topics:

  • Google Javascript APIs.  Google has really opened its services up, and provides easy to use RESTful libraries for getting them to them in Javascript.  I’m not real interested right now in any single one of them, but the breadth of what’s available now is impressive.
  • Android.  Some very nice demos of mobile phones running Android, but I’m not clear on when this technology will actually be available for people like me to use.  I want it, but I bet today’s cellular provides, who always want to lock their users down, probably don’t want it.
  • AppEngine.  This is the technology that triggered my decision to attend the conference.  I’ve done some development in it, like it, and see great uses for it.  The big news for AppEngine is that it is now (as of today) open for anyone to sign up to use.  They also showed some approximate pricing for when it becomes a fully supported product, but committed to making it always free for low volume users.  “Low volume” will be defined in terms of storage, CPU, data transfers, and so on, but the free level will be enough to cover an average of 5 million page views per month.
  • OpenSocial.  Google’s supporting open APIs for social networking in a big way.  Personally, I’m not currently very interested in it.
  • Google Web Toolkit.  In addition to the native Javascript APIs, Google supports client-side development with GWT.  You write the applications in Java, and GWT compiles it to Javascript for deployment to the browser.  I don’t get it.  The speaker made a big deal of how Java was a much better language for this, and had grown-up development tools, but didn’t convince me.  I remember vendors pushing Cobol for developing in OS/2 and Windows for the same kinds of reasons.  Have you seen a lot of Cobol GUI programs?
  • Google Gears.  Now just called Gears.  This is a browser plug-in that lets you do all sorts of great things with Ajax, like store persistent data on the client and access client resources.  They are looking at this as a bleeding edge early preview of HTML 5, and I think it’s going to be important.

That’s not exhaustive, but it’s fairly complete.  I’m glad I came.

Google doesn’t scale

Filed under: Google IO 2008,Uncategorized — Charles Engelke @ 3:37 pm
Tags: , , ,

That is, the Google IO conference starting today in San Francisco isn’t scaling very well.  Registration was very, very slow for some reason, even though all that seemed to be happening was finding preprinted badges and giving them to attendees.  They simply couldn’t get people registered in the 90 minutes from when they opened at 8:00 until the keynote at 9:30, so they decided to let people attend sessions most of today without badges.  People need to go to the desks between sessions and get their badges by the end of the day.

Agenda scheduling doesn’t seem very practical.  Sessions go on non-stop all day with 15 minute gaps between them.  People are supposed to grab food during those breaks, but the food’s on a separate floor, and with such large crowds I don’t think you can even get to the other floor and back in that time.  The other choice would be to skip a session to eat, but the agenda is very strong.  I’m glad I ate a big breakfast.

The conference content has been great so far, with the most polished yet technical talks I’ve ever seen.  I hope that keeps up for the whole time.  Notes on sessions later as I get some breaks.  I’m not going to write during the sessions themselves.

Create a free website or blog at WordPress.com.