Charles Engelke’s Blog

February 28, 2010

One Year with Kindle

Filed under: Uncategorized — Charles Engelke @ 12:51 pm

I got my first Kindle a year ago this week.  It arrived on February 24, 2009.  Here are some statistics about my book-buying (and book-reading) habits in that year.

I “bought” 176 Kindle books at a cost of $873.43 during that year.  I put “bought” in quotes, because a lot of the books, especially at first, were free or nearly so.  62 of those books cost $0.00 each.  Another 9 cost $1.00 or less.  I read only five of the free books, and four of the under $1.00 books.  Another six of of the free books, and the other five under $1.00 books, were books I’d read in the past and figured, hey, maybe I’ll want to read them again someday, and this is a good deal.

From here on out I’ll leave out the free books I was never seriously interested in.  I’ll also leave out the 6 non-free books my wife bought for her Kindle that I didn’t read.  (She’s read books I’ve bought and vice-versa, but for this post I’ll just treat everything I bought or read by itself.)  That leaves 112 books bought for $817.01, for an average price of $7.29.  The Kindle itself cost $359.00; amortized over just the first year, it added $3.21 to the cost of each book.  Of course the Kindle’s still going strong, and new ones are cheaper, so the actual cost of the hardware per book should be considered to be much lower.

The most common price I paid for a Kindle book was the famous $9.99 that’s always talked about.  41 books cost me that much; slightly more than a third of my purchases.  I paid more than that for 9 books: one was $15.83, three were $14.27, and the others ranged from $11.20 to $13.73.  Most of those books are now $9.99 or less (after all, many are now out in paperback), but two have gone up about $0.50 each.  The majority of books I bought were less than $9.99; they averaged $6.13.  Most of the Kindle books (80% or more) were fiction.

I’ve been pretty price-sensitive for the Kindle books.  I’ve notice that a lot of the prices over $9.99 fluctuate from day to day, so I’d watch and buy them only when they dropped enough for me.  It’s not that I’m not willing to pay more for a Kindle book, but I’m not willing to pay more than I think they should cost.  Which, despite what publishers and many authors insist, is significantly less than a paper book.  I’d say about a third less than a hardback.  (And that’s a third less than the price I actually pay for a hardback, not the list price.)  Say about 40% of the list price of a hardback.  I could go up to about 50% of list if they’d get rid of DRM and offer books in formats I could use on any device.  Why not more?  Well, I generally pay 65% of list for a physical book, and not having to print, handle, and ship the book should result in savings passed on to me.

What about other books I bought during that year?  Not counting gifts or books for work, I seem to have bought approximately:

  • 3 novels in Kindle compatible format for a total of $14.99, an average of $5.00 per book.  I wanted to buy all three from Amazon, but the publishers didn’t offer them that way.  One was bought directly from Baen books for $6.00 (I would gladly have paid $9.99), one from Fictionwise for $8.99, and one downloaded from Cory Doctorow’s personal site for free (again, I would have gladly paid).
  • 21 physical books from Amazon for $325.06, average price $15.48.  Some of these were special editions from small presses (Subterranean Press and University of Chicago Press) not otherwise available, one (Donald Westlake’s last novel) was sentimental, a couple were remainders cheaper than the Kindle editions, two just weren’t available in Kindle, and the others (travel and comics) had a lot of images not well suited to the Kindle.
  • 3 physical books bought from overseas because they weren’t yet published in the United States.  It’s a pain to look up their prices, bit I recall them being about $30 each, when shipping was included.
  • 14 technical books from Manning and Pragmatic Programmers, all in electronic format, for $332.86, an average of $23.78.  About half these books were both on paper and electronic, the rest only electronic.
  • A handful of paperbacks bought on impulse while traveling.

So, putting it all together, I bought around 158 books and spent about $1600, or $10 per book.  If you add in the cost of the Kindle itself, I spent about $2000 on books during the year.  And contrary to what some people think, I completed more of the books I bought on the Kindle (at least 95% of them) than the ones on paper, even if you only count fiction.

I have become very reluctant to read a novel in paper form, especially hardback.  They’re too heavy while reading and inconvenient to carry on trips.  If a book isn’t available on the Kindle, I’m very unlikely to buy it now.  Minotaur Books, you should take note.  You’ve lost a few sales to me I would have bought in hardback a year ago because you won’t offer Kindle editions.

January 31, 2010

Macmillan vs. Amazon (vs. Authors and Readers)

Filed under: Uncategorized — Charles Engelke @ 1:55 pm

Amazon.com and Macmillan publishing are playing hardball with each other.  As of right now, Amazon isn’t selling any of Macmillan’s books in any format.  This is very bad for writers and readers, and has drawn a lot of attention.  There have been a several good posts at John Scalzi’s blog, and a great one at Charlie Stross’s blog.  The New York Times Bits blog and Wall Street Journal have also covered this, but not with much clarity or depth.  Macmillan has publicly commented, but so far Amazon has not.

Most of the articles and comments (other than Charlie Stross’s post) say that this about e-b00k prices.  That doesn’t seem to be quite correct.  True, Amazon wants e-books to sell for about $10 and Macmillan wants them to be about $15, but that’s not what has triggered this extreme tactic.  It seems that Macmillan has offered Amazon a choice: sell our e-books as an agent instead of a retailer, or else wait 7 months after hardcover release to sell them at all.  It seems implicit that Macmillan will not allow Amazon to continue paying hardcover wholesale to retail the e-books right away (which is what they do now).

Those are both terrible options for Amazon. If they’re just one of many agents, all selling the exact same product for the exact same price, how do they beat their competition? Smoother payment and delivery systems help, but they’re not enough. And if e-books are delayed for 7 months, they’ll just plain fail as a product.

Amazon’s retaliatory move is extreme and is hurting a lot of bystanders. I don’t support it, but I do support their position that Macmillan’s offer (demand?) is unacceptable. Publisher-fixed pricing or 7 month delays would be bad for Amazon, bad for customers, and (I believe) bad for authors. It might even be bad for the publishers; businesses aren’t always great judges of how future market changes will affect them.

I think Amazon understands the e-book marketplace much better than Macmillan.  I even think they understand the traditional book marketplace better.  Let the market sort this out to see who is right.  Macmillan wants to take the market partly out of the equation: no retailers involved in pricing at all.  There are countries that work that way for physical books, and I think it works poorly there.  It protects incumbent publishers and stifles innovation.  Macmillan’s just wrong here.

Update: Macmillan has now directly posted a statement on Tor.com.  Among many other issues, they state that they are trying to encourage a business model that “encourages healthy competition” and is “is stable and rational”.  Later, it states that the disagreement is about “the long-term viability and stability of the digital book market”.  [Emphasis mine.]

A market with healthy competition is not stable.  A stable market doesn’t have healthy competition.  Macmillan is an incumbent and wants to stay that way, comfortably.  Amazon’s something of an incumbent itself, but it seeks change, not stability, because it has confidence that it can win in a changing world.  I favor Amazon’s point of view.

January 22, 2010

Firefox 3.6 Tabs

Filed under: Uncategorized — Charles Engelke @ 10:39 am

Firefox 3.6 puts newly opened tabs just to to the right of the current tab, instead of all the way to the right of all open tabs.  I can’t really think of why it matters one way or the other.

Except, it really bugs me.  For no good reason.

It’s not at all clear what you can do about this.  I found out, though, and I’m noting it here.  In the address bar, enter about:config, and click to accept any warnings you see.  This gives you a list of very specific options you can adjust to change your browser’s behavior.  Scroll down to browser.tabs.insertRelatedAfterCurrent and change it from true to false.  Just double-clicking on the option’s name will make the change for you.

The warnings for this configuration page are a bit overblown.  Every option that has been changed from the default is shown in bold, so if you make changes and don’t like how things came out, you could always go back and double-click on every bold option until they’ve all gone back to their defaults.

December 16, 2009

Hooray for FedEx! (Boo for me.)

Filed under: Uncategorized — Charles Engelke @ 10:09 pm

I was being so efficient.  I had to ship a defective monitor back to the manufacturer and get a suit altered, and do them both today to meet deadlines.  Last night, I folded the suit on top of the monitor box so I wouldn’t forget either of them.  This morning I left the monitor at our front desk where FedEx picked it up, and after work I went to the tailor.  Where I tried on the suit.

Or tried to try it on.  I had the jacket, but not the pants.

Where were the pants?  I finally figured it out.  I must have packed them with the monitor and shipped them off to Philips electronics.

Yes, I am an idiot.

But FedEx saved the day.  I called their 800 number, and they got on the phone with the local depot that was processing the box at the start of its journey.  They found the box, pulled it into their office, and waited for me to come by, open it, and get my pants.  Then they resealed the box and sent it on its way.

They were incredibly helpful and cheerful, and acted like this was no big deal.  But they went way out of their way for me when they didn’t have to.  I’m really impressed, and I’m not going to forget.  FedEx is getting my shipping business, such as it is, from now on.

October 20, 2009

curl Cheat Sheet

Filed under: Uncategorized — Charles Engelke @ 9:36 am

A lot of what I do lately involves creating or consuming RESTful web services.  (By the way, Leonard Richardson’s and Sam Ruby’s book by that name is fantastic.)  One really nice thing about RESTful services is that you can do many of the operations with a simple web browser.  But not all: web browsers are happy to do GET requests and, with a little effort, POST requests, but none of the other HTTP verbs.  If you’re trying to do anything significant with a RESTful web service, sooner or later you’re going to need to perform PUT, DELETE, and HEAD requests, too.

That’s where curl comes in.  It’s an open source command line tool that can perform just about any kind of HTTP request operation.  It certainly can do everything I ever need to try in testing a RESTful web service.  But every time I use it I have to look at the help page for it, which is incredibly long, because I can’t remember all the command line switches I need.  So I’m writing up a cheat sheet here, with the switches I need to use.

The basic form of a curl command is curl -X verb [options] uri, as in:

   curl -X GET -D headers.txt http://example.com/some/path

The command above will perform an HTTP GET request for the URI http://example.com/some/path.  It will store the response headers in the file headers.txt (that’s what the -D headers.txt option makes happen) and will send the actual response body to standard output.  You can redirect standard output to a file if you want to save the response, or if it’s not simple text (for example, if you want to GET a photograph).

The HTTP verbs that I use with curl are GET, PUT, POST, DELETE, and HEAD.  The command options that I use most often are:

-D filename
Save the response headers in the file filename.
-i
Send the response headers to standard output, along with the response body.
–basic –user username:password
(There are two hyphens each before basic and before user.)  Authenticate the request with HTTP Basic authentication, with the specified username and password.
-H “header: value
Set a request header named header to the specified value. Note the double quotes, which causes your command shell to pass the entire header specification to curl as a single string.  You can specify multiple -H options to set multiple request headers.
-T filename
Send the contents of the specified file as the body of a PUT request. Useful for uploading files.
-k
Ignore SSL certificate problems. Very useful when talking to a development server with a self-signed certificate. Otherwise curl will refuse to connect to it.
-h
For help. Show all the possible command line options.

There are a bunch of other switches, but these are the ones I use all the time. You may need some others. For example, if you use web services that expect form data via HTTP POST, you’ll want to learn about the -F option.

October 9, 2009

Great ThinkPad Service, Lousy UPS Delivery

Filed under: Uncategorized — Charles Engelke @ 3:08 pm

Cory Doctorow put up a glowing post about Lenovo’s ThinkPad warranty service yesterday.  I’m not surprised.  His experience is similar to the ones I’ve had with ThinkPad service over many years.  And, as it happened, I was using ThinkPad warranty service when I saw his post.

Unlike Cory, I didn’t pay for on-site service, just regular “depot” service.  But they still did great.  My ThinkPad just went dead Tuesday evening.  In more than 15 years of using them, I’ve never had a similar problem before.  But I phoned them up to get it fixed.  They actually listened to my description of the diagnostics I’d already performed, believed that I was describing them accurately, then spent about three more minutes having me answer a couple of questions and trying one more thing.  They diagnosed the problem as a bad system board needing replacement, and said they’d send me a box to ship the unit in.

This was Tuesday evening.  Wednesday morning, the empty box arrived.  I put my ThinkPad (without the hard drive) in it and called UPS to get it.  They picked it up that afternoon.  Thursday morning the Lenovo website said my unit was being repaired, and Thursday afternoon, that it had been fixed and shipped back to me.  Friday morning I waited for our receptionist to tell me that UPS had delivered it.

That service turnaround speed is fantastic!  Or, it would have been, if my PC had ever shown up.  I called the ThinkPad support line and they tracked the UPS package and found that an “exception” had occurred.  I looked it up myself.  Here’s what it said:

THE RECEIVER IS ON VACATION. DELIVERY WILL BE ATTEMPTED WHEN THE RECEIVER RETURNS

I’m on vacation?  Shouldn’t I know about that?  Why am I in the office, then?

I checked with our receptionist, who said that UPS had shown up early today but didn’t have any packages to drop off.  UPS and Lenovo agreed that the address label had the right address on it, except that the zip code was off by one (which is still in the same town).  UPS said they’d already fixed that zip code.  So what happened that led to that exception?

What didn’t happen was what they claimed in the tracking log: that they tried to deliver the package to the correct address only to be told that I was on vacation!  My guess is that the driver may have put the package in the wrong part of the truck (due to the zip code) and discovered it after he’d already visited our business.  And then put that stuff about a “vacation” down as an excuse for not having delivered it on time.  This isn’t the first time I’ve had UPS miss a delivery date and put down some completely bogus reason in the tracking log.  It may be the driver doing it, it may be someone in a back office.  But there definitely are UPS staffers who make up false reasons when they don’t make a delivery they should.

UPS refused to make it right by getting the package to me today like they should.  Instead, I can wait until Monday and go without my PC a few more days, or drive over a hundred blocks to their facility in the boondocks to get it myself.  I’ll probably do that, but I’m not happy about the wasted time and money on my part.  UPS is doing nothing at all to make it right.

As I said above, this kind of thing has happened to me before with UPS, maybe about one time out of every twenty or thirty deliveries.  That’s not a big percentage, but it’s a lot higher failure rate than I’ve had with FedEx.  I’ll never use UPS when I have a choice, and I’ve already made a complaint with Lenovo suggesting that they should reconsider using UPS, too.

September 24, 2009

More Code Signing

Filed under: Uncategorized — Charles Engelke @ 2:57 pm

An update to my last post: you can use signtool with a certificate in the Windows certificate store; it doesn’t have to be in a file.  In the command line, instead of specifying a file to use with the /f option, specify part of the certificate’s subject name with the /n option, as in:

signtool sign /n “Part of subject name” /p newpassword hidden.exe

You can leave off the /p (for password) option if you don’t have the certificate protected by a password.

The advantage of this is that some trusted corporate administrator can install the certificate on each developer’s PC, marked as “not exportable”.  Then the developers can use the certificate to sign code on that PC, but can’t take a copy of the certificate elsewhere with them.  It’s not a perfect solution, but it seems a good compromise.

September 3, 2009

Click-once, administrative access, and code signing

Filed under: Uncategorized — Charles Engelke @ 2:05 pm

We need our users to run some legacy software on their own computers in order to work with a service of ours. They do this with one small function of a large software package. But we want to update the behavior of that function pretty frequently, without forcing the users to keep downloading and installing updates for the whole package. Enter Microsoft’s ClickOnce technology.  It’s a very nice way to give users a near-web experience but with near-native software capability.

ClickOnce programs are launched from a link on a web page, self install (and can self update), but run in the .Net environment under Windows.  Unlike web pages they can access Windows APIs to do whatever you really need.  Which is a potentially big security problem.  Just clicking on a link could really open your computer to anything, so Windows restricts what ClickOnce programs can do, and makes sure that users approve anything at all risky.

Some of the user approval requests can be very scary; we want to configure things to make those notices as clear and unalarming as possible.  Signing code helps placate Windows.  The alert window borders change from angry orange to soothing baby blue, the wording does less warning and more asking whether to allow something, and Windows says that it knows who the code comes from, instead of just an unknown or unverified publisher.  The concept of code signing isn’t too hard to follow, but I found the mechanics difficult, in part because the consequences of different scenarios aren’t well described; I had to try them all and see what happened.  This post describes what eventually worked, with a few asides about what didn’t.

Application Structure

The user starts with a web page that does most of the work that’s needed.  But there is a step that must be performed on his or her own PC, for security reasons.  So the web page contains a link to a ClickOnce application to do that work.  The ClickOnce application needs to run a legacy, console mode native Windows application to do the work.  ClickOnce applications are not allowed to do that under any circumstances, so far as I can tell, so the ClickOnce application launches a hidden .Net application that then launches the legacy native application.

It all looks kind of like this:

Application Chain

Issues

When the user clicks the link in the web page, he or she may get a security alert before the program is downloaded, installed and run.  It depends on the exact version of Windows, user privilege level, security policies, and probably the phase of the moon.  In any case, if the alert is shown we want it to be as unthreatening as Windows will allow.

The hidden .Net application has to be built with a manifest specifying that it has to run with Administrative rights in order to launch the native application.  When it starts, Windows might display a really scary alert because of that.  Again, we want to make this as benign as Windows will let it be.

Curiously (to me) starting the native application doesn’t trigger any alerts, even though that’s got the most potential to cause trouble.  I guess it’s because the hidden .Net application already got permission to escalate to the highest possible privilege level. After that, Windows must figure that anything goes.

The ClickOnce Application

When it’s just built and deployed normally, the first time the user runs it he or she sees something like this:

ClickOnceUnsigned

In Visual Studio 2008 (even the Express editions) you can right click on the project name and select Properties to get a tabbed page about the project.  The Signing tab lets you “Sign the ClickOnce manifests“.  You’ll need a certificate, but Visual Studio will offer to create a test certificate for you to use.  However, it has no effect on this warning.  Even though the code is now signed, the certificate wasn’t issued by a valid certificate authority, so Windows doesn’t much care.

Now, if you get a paid code signing certificate, things are a bit different.  We got one from Thawte.  In the Signing tab I just clicked the check box for “Sign the ClickOnce manifests” but then, instead of clicking the button labeled Create Test Certificate… I clicked on Select from Store… and picked our paid certificate.  When I run the published application from a web page after that, there’s no warning at all.  That’s what I want.  I’m not as sure as Microsoft that just having a valid certificate is enough to make me trustworthy, but combining that with the execution restrictions placed on ClickOnce applications makes this a pretty safe operation for the user.

The Hidden Application

This is a regular .Net program, not a ClickOnce application.  It will be included as a delivered resource in the ClickOnce application, and that application will launch it from the ClickOnce private storage area.  Launching any application this way might trigger warnings, but this application definitely does.  That’s because the app.manifest for this program has the entry:

<requestedExecutionLevel level=”requireAdministrator” uiAccess=”false” />

When the ClickOnce application launches the hidden application Windows makes sure that the user knows about it.  I’d love to show you an image of the alert but in Windows 7 all other software, including the Print Screen key, is disabled while the warning is up.  The alert has an angry orange heading that says “Do you want to allow the following program from an unknown publisher make changes to this computer?”  The default answer is No, and there’s a Help me decide link to more information that doesn’t provide reassurance.  Well, it shouldn’t be reassuring.  The program has enough privilege to cause big trouble if it wants to.  But our users have been downloading, installing, and running similar programs from us for years, so they’ve already decided to trust us.  Our problem is to keep this alert from scaring them away.

Code signing helps.  Once the hidden program has been signed, there’s still an alert, but it’s not nearly as frightening.  The alert has a baby blue background and says “Do you want to allow the following program to make changes to this computer?”  It lists the program name, and our company name as the Verified Publisher.  If the user clicks Show Details, he or she can examine the certificate itself.  And the Help me decide link displays text that’s more reassuring:

If the program has a verified publisher, it means that the program has a valid digital signature. A digital signature helps ensure that the program is what it claims to be and comes from a reputable publisher. If the program has an unknown publisher, it doesn’t have a valid digital signature from its publisher. This doesn’t necessarily mean the program is harmful—many older, legitimate programs lack signatures.

It not only strongly implies that we are a “reputable” publisher, it also gives the user the idea that legitimate programs lack signatures only if they are “older”.  Since our program is brand new (to the user), that would be taken as another red flag if it weren’t signed.

The Native Application

This isn’t an issue.  When the hidden program launches it, Windows doesn’t show any alerts.  It just works without any special code signing.

How to Sign

As mentioned above, signing the ClickOnce application is very easy.  Get your certificate from a known Windows certificate authority, and install it in your Windows certificate store.  The procedures for doing those first two steps vary from provider to provider.  But once it’s in the local certificate store, you can sign your code automatically with every build by doing the following steps once:

  1. Right-click on the project name in the Solution Explorer pane, and select Properties from the context menu.  (This is not at all the same properties that show up as a child of the project in solution explorer.)
  2. Click on the Signing tab.
  3. Check the box Sign the ClickOnce manifests.
  4. Click the Select from Store… button, find your purchased certificate in the list, select it, and click OK.
  5. (Optional) Fill in the Timestamp server URL with a value provided by your certificate vendor.

I have to take it on faith that step 5 above will do anything.  The problem it’s designed to solve is what happens when someone runs your ClickOnce application after your certificate expires.  So long as the application was built and signed before the expiration, the signature should still be considered valid and Windows should remain happy to run the program.  But how can it know that it was signed then?  A trusted timestamp server can be used to sign your code with an added timestamp.  That way, Windows can know that the signature was applied while the certificate was still valid.  The only way I can see for myself that using a timestamp server makes a difference is to create one build with it and one without, and then wait a year for my certificate to expire.  Well, I could play with my system clock, but for all I know, Windows checks that value over the network, too.

Signing the hidden .Net application is a little harder.  The Signing pane in Visual Studio says it’s just for ClickOnce.  Depending on what you do here, you might prevent Visual Studio from building the program at all (because it will think it’s a ClickOnce application, and they aren’t allowed to request administrative privilege), or it might just have no visible effect at all.  There’s a check box to Sign the assembly instead, which seems like it’s what I want, but in my tests it either didn’t work (complaining that the key file I chose was already installed, so I couldn’t use it) or had no visible effect.  So I used the command line signtool that’s available as part of the Windows Platform SDK for Windows 7 (or any of the ones for other recent versions of Windows).

To sign a program with signtool you need your certificate and private key in a pfx file, not the certificate store.  If you specified that the key should be exportable when it was installed in Windows then you can export it from the store directly to the needed file.  Otherwise, you’ll have to follow instructions from your certificate provider.  For example, Thawte delivered the certificate I was using as two files: mycert.spc and privatekey.pvk.  I had to create a pfx from those two files with the command pvk2pfx:

pvk2pfx -pvk privatekey.pvk -pi oldpassword -spc mycert.spc -pfx signcert.pfx -po newpassword -f

In this command, oldpassword is whatever password the certificate issuer places on the pvk file, and newpassword is anything I want to use to protect the new pfx file.

Regardless of where the pfx file comes from, once you have it you can use signtool:

signtool sign /f signcert.pfx /p newpassword hidden.exe

If you want a timestamp, signtool can do that, too:

signtool timestamp /t http://somebigca.com/some/path hidden.exe

You can automate this in Visual Studio:

  1. Right click on the hidden application’s project in Solution Explorer and select Properties from the context menu.
  2. Click on the Build Events tab.
  3. Put the signtool commands in the Post-build event command line box.  You can refer to the executable using the macro “($TargetFileName)”.
  4. (Optional) Add a copy command to the Post-build event command line box after the signtool statements to copy the built executable to the ClickOnce project’s folder.  If you use Solution Explorer to Add this file to the deployment package for the ClickOnce application it will be delivered every time a user updates the ClickOnce application.

Loose Ends

Managing the code signing certificates in a multiple developer environment is going to be tricky.  You don’t want a developer who leaves the company to be able to take a usable copy of the certificate with him or her.  The best way I can think of to manage this is to have a highly trusted person install the certificate on each developer’s PC, marking it as not exportable.  Then anyone who logs in to that PC can use the certificate on the PC, but (supposedly) can’t take a copy to a different machine.

That approach won’t work for the signtool examples used for the hidden application above, because they expect the certificate to be in a file, which anybody could copy.  It appears that signtool can directly use a certificate from the Windows certificate store instead of a file, which would reduce or eliminate the chance of a developer copying it.  I may give that a try in a day or two, after I clear my head of this project by working on something else for a while.

August 28, 2009

Poetry

Filed under: Uncategorized — Charles Engelke @ 6:42 am

E-mail from my wife on her drive from Macon to Gainesville:

no service

my phone can’t get a signal it is pouring i am pissed

Hmmm… maybe it would work better this way:

no service

my phone can’t get a signal
it is pouring
i am pissed

Yep, that’s it. She sent it from her Kindle, using Google Mail for domains, which I think was extremely resourceful.

August 25, 2009

QRCode of Code to Generate a QRCode

Filed under: Uncategorized — Charles Engelke @ 1:45 pm

My last post ended with a little bit of Perl to generate a QRCode barcode. Just for fun, I built a barcode with that Perl program as its source. If you’ve got a cell phone with a barcode scanner, you can probably get it to decode the following image back to that sample program:
perlcode

QR Codes and Perl

Filed under: Uncategorized — Charles Engelke @ 1:28 pm

Two-dimensional barcodes like QRCode are pretty neat, and after I got my Google Ion phone with its built-in barcode program I went around scanning every one I could find for a while.  But I never really did anything with them until yesterday, when it occurred to me that they might be useful for a project I’m doing.  The project would augment an existing system written in Perl, so I went to see what Perl has in this vein.

It has GD::Barcode::QRCode (which I got for Windows using ActiveState’s PPM tool and installing Catalyst-View-GD-Barcode-QRcode).  It’s dead easy to use.  Given a string (say, $text), you can just do:

use GD::Barcode::QRcode;
binmode STDOUT;
print GD::Barcode::QRcode->new($text)->plot->png;

Well, it’s that easy if the text is very short. If it’s not, you’ll get a message to STDERR saying something like:

Overflow error. version 1
total bits: 468  max bits: 128

You, see, there are different size QRCodes, and the module doesn’t seem to pick the right one for you (even though the documentation seems to say it will). But there are three parameters you can pass to the new method that give you some control over this: Ecc, Version, and ModuleSize. But what do they mean? The documentation isn’t very clear, but I’ve figured a bit of it out. The purpose of this post is for me to document what I’ve learned so I’ll be able to use it again.

The most important parameter (to me, anyway) is Version. Which is a terrible name, because it implies to me sequential enhancements. Actually, it just determines the size of the square barcode. A version 1 barcode is 21 rows by 21 columns, and each successively larger size is four rows taller and four columns wider than the last. So a version 2 barcode is 25 by 25, version 3 is 29 by 29, and so on. (I don’t know why they go up by four. It might just be convention, or it might be something to do with the algorithms involved.) A version n barcode is therefore 4n+17 rows by the same number of columns. The largest legal value of version is apparently 40.

Ecc stands for Error-Correcting Code, and the possible values are ‘M’, ‘L’, ‘H’, or ‘Q’, with ‘M’ being the default. That’s all the module’s documentation says. Huh? How am I supposed to pick? Well, different values result in different levels of error-correcting codes being used. The more robust the error-correction, the less data you fit into a fixed size barcode. And the values used here? Since ‘L’ reduces available data less than ‘M’, which does so less than ‘H’, my guess is that they mean “low”, “medium”, and “high”. ‘Q’ is between ‘M’ and ‘H’; maybe it means “quality”? If you don’t have to worry about the barcode being badly smudged or losing definition in transmission, ‘L’ for low error correction is fine. If you’re going to have it get dirty, scraped up, and badly scanned, go for ‘H’. Or pick something in the middle.

Finally, ModuleSize is just how big the printed blocks are going to be. If you pick 1 for this value, they’ll use a single pixel each in the resulting image file. I found that 8 was a good choice for a PNG on a web page.

The final piece of the puzzle is just how much data you can put into a barcode with specific values of these parameters. This table tells you. So, if I have 256 bytes of binary data, and I have a fairly clean but not pristine printing, scanning, and transmission environment (so I’ll want to at least use M for medium error correction), I’ll need to set the version to at least 12. Here’s the code:

use GD::Barcode::QRcode;
binmode STDOUT;
my $code = GD::Barcode::QRcode->new($binary_data_256_bytes_long,
   {ECC => 'M', Version => 12, ModuleSize => 8}
);
print $code->plot->png;

May 26, 2009

An “Incident” on the Highway

Filed under: Uncategorized — Charles Engelke @ 9:22 am

Laurie was driving, and I was working on my laptop.  We were on our way down to Sanibel Island in south Florida when suddenly there was a loud bang and a bright flash, and Laurie quickly pulled over to the side of I-75.  We’d hit (actually, we think we’d been hit by) this:

Jack Shaft

Jack Shaft

That’s apparently a jack shaft from a tractor-trailer; both Laurie (former Army transportation officer) and the Highway Patrolman agreed on that.  This piece is less than half of what hit us; we broke it, and the other piece is even larger and heavier.  This part is about two feet long and about 25 pounds.

It shredded a tire, cracked all the way through the wheel rim, and because of how it hit the car, made the two passenger side airbags deploy.  It similarly disabled four other cars.  The truck it fell off of apparently kept on going.

Being hit by airbags is no fun.  Four days later my hearing’s still a bit off; the first day I could hardly hear at all.  I’ve got a bruise on my arm where it hit me, and I was really shaken up the rest of the day.  Worst of all – we spent all most all Saturday of our long weekend in Sanibel at the car dealer instead of the beach, and we’ve got to run around to insurance adjusters, dealers, and an audiologist today.  And I’ve got to catch a flight this afternoon.

May 6, 2009

Rails Metal, Rack and Sinatra

Filed under: Uncategorized — Charles Engelke @ 2:17 pm
Tags: ,

Great talk by Adam Wiggins here at RailsConf.  But this post is more a reminder to me than a synopsis for you.  Find out more at adam.blog.heroku.com and railscasts.com/episodes/150-rails-metal.

Use Named Constants

Filed under: Uncategorized — Charles Engelke @ 12:53 pm
Tags: ,

This morning’s keynote speaker at RailsConf mentioned, in passing, working on some code without named constants.  When something needed to be changed you’d have to find the right instance of it and change it, and hope you changed it everywhere you needed to. It’s elementary that you should use named constants for just that reason.  But this story reminded me of an early programming experience I had with FORTRAN IV on a CDC 6400.

You see, when you write a 1 in your code, you’re kind of using a named constant.  That 1 is is the name of a value (that happens to be one, or 000000000000000000000000000000000000000000000000000000000001 in bits on the 6400).  We always expect that the value of the constant named 1 has to be one.  But in my freshman programming class, I managed to change the value of 1 in my source code to zero.  FORTRAN programmers will probably know how I did that.

This made for a wonderfully oddly behaved program.  The first time through a loop always worked, later ones always failed, and it didn’t matter what values I set before each run.  The story had a happy ending: I learned how to read core dumps and a bit more about machine language, and moved on to other programming languages that make it harder to change the values of literal constants.

May 5, 2009

Testing Your JavaScript

Filed under: Uncategorized — Charles Engelke @ 5:32 pm
Tags: ,

Test your JavaScript in a headless Java browser, as a normal Rake task.  I’m not going to put notes here (I was paying too much attention to write).  Instead, look for the presentation and the necessary plug-in at github.com/relevance/blue-ridge and thinkrelevance.com.  It looks very promising.

RailsConf Keynote

Filed under: Uncategorized — Charles Engelke @ 1:11 pm
Tags: ,

David Heinemeier Hansson is opening the conference, quite appropriately, and saying a lot of interesting things about the past and future of Rails.  The conference is going to post video of the keynote in a day or so, and I recommend watching it.  But what got me sitting up straight and listening more alertly was when he started talking about something else.  Requirements.

David says that programmers treat requirements as commandments from on high.  And that they shouldn’t do that.  The stakeholders aren’t are committed to the requirements as programmers think.  They know what they want, but they’d be just about as happy with a lot of other solutions, too.  And those other solutions might be a whole lot less expensive to provide.

His example concerns chocolate.  He’s got a sweet tooth.  Ask him for requirements to satisfy that sweet tooth, and he’ll specify top quality Belgian chocolates.  Meeting those requirements will take a lot of money and a lot of time.  Maybe that’s necessary.  But try offering a Twix bar right now for just a buck or so instead, and most of the time he’ll actually be happier with that.

The real key to programmer productivity is saying no to requirements as stated, and instead understanding what the stakeholders want to accomplish and opening negotiations with them on the best way to do that.  Requirements are a starting point to that.  They’re not handed down to us on stone tablets.

That means programmers have to be much more involved with stakeholders, and take more responsibility for the ultimate outcome of their efforts.  But they’re going to create a lot more value if they do that.

May 4, 2009

RailsConf 2009 Tutorial – Sinatra

Filed under: Uncategorized — Charles Engelke @ 7:31 pm

My second tutorial of the conference is called “A Hat Full of Tricks with Sinatra”.  Unlike the morning session on jRuby on Rails, I don’t have any specific plans on how I’d use what I learn here.  But the framework looks interesting to me, and I hope to get some good ideas from it even if I never use it for real.

We start by building the minimal “Hello, World” of Rack applications:

run lambda {|env|
[
200,
{'Content-Length' => "2",
'Content-Type'   => "text/html"},
["hi"]
]
}

run lambda {|env|
[
200,
{'Content-Length' => "2",
'Content-Type'   => "text/html"},
["hi"]
]
}

Put that in a file, like “config.ru” and run it with rackup, for example

rackup config.ru -p 3000

The “.ru” extension is for rackup.  You can call it “.rb”, but then you have to set more things up in the application because it’s actually running regular Ruby code, instead of being read and interpreted in a special context by rackup.

It’s starting out as a very hands-on tutorial, very basic.  But that’s what I want.  If it slowly builds from here, I’ll be happy.

Rack runs an application that is given an environment and emits a response.  The structure allows you to easily write “middleware”.  When Rack’s told to run an application, instead it calls your middleware which can do other things and eventually call your app.  This can be chained.  It’s a simple and powerful tool.

Okay, now getting into Sinatra.  Here’s the core of the application:

require ‘rubygems’
require ’sinatra’

Huh?  Where’s the code?  That starts up the framework and gets it all working.  If you put those two lines in application and run it (with Ruby) it will start a web server on port 4567.  When you open that in a web browser, you’ll get a blurb for Sinatra and suggestions to proceed.

Want to handle a request to a particular path?  Here’s code that will respond to requests for the root URL and to /greet:

get ‘/’ do
“Welcome to my Sinatra server”
end
get ‘/greet’ do
“Hello!”
end

Nice and clean.  It can also respond to other HTTP verb requests in a similar way.

Now just create a subdirectory of your application directory called “views”, and put a file named “index.erb” in it containing HTML.  Change the code that responds to “/” to:

get “/” do
erb :index
end

Now when you run your application and get the root URL, the HTML in that index.erb file gets returned.  (ERB is “embedded Ruby” even though we didn’t bother to embed any in that HTML.

I’m really liking how this is set up.  Simple always appeals to me, but there’s a lot of power here without messing up that simplicity.

By the way, the sample Sinatra application I’m playing with works just fine under jRuby, too.

I do have one complaint about this tutorial.  It’s not well-structured or planned.  The speaker seems to think he can just open a terminal window and edit and run source code, and we’ll all have no trouble following.  But as soon as one thing is missed, the student has a hard time getting back in sync.  He needs to go slower, he needs to describe what he’s about to do and why, he needs to do it, then he needs to tell what he did and why.

He also needed to plan and rehearse every single example he was going to show.  Instead, he’s winging a lot of it.  The material is trivial to him, but when you’re teaching a class you’ve got to know ahead of time every single tiny glitch you’ll encounter and have to explain.

Another tutorial falls apart in the second half.

Diet Coke Must Die!

Filed under: Uncategorized — Charles Engelke @ 6:33 pm
Tags: ,

I’m here at RailsConf looking for a bit of cold, free, sweet caffeine at the break.  There are plenty of sodas, and they’re even the right brand – Coca-Cola.

But unless I want a lot of empty calories, my only choice is Diet Coke.  Which is drinkable, but only barely.  (Pepsi fans: don’t gloat.  No other brand of diet cola reaches even that level.)

This is a travesty because now Coke Zero exists.  Somehow they’ve managed to create a diet cola that really tastes like Coke (the only cola that has ever mattered)!  But you can almost never get it at a meeting or restaurant because they’ve got the old standby Diet Coke.  And Coca-Cola is fine with that because it helps in their goal to fill supermarket shelf space with several thousand different products all called Coke.

Coke Zero is a better choice than Diet Coke in every single circumstance.  Coca-Cola, you’ve got a responsibility to us – fix your advertising and branding and GET RID OF DIET COKE!  Let Coke Zero take off and fly!

And while you’re at it, get rid of Cherry Coke, Caffeine Free Coke with Lime and Splenda but without added Vitamins, and all those other stupid variations.  There are only two drinks worthy of the name “Coke” – Coca-Cola and Coke Zero.

RailsConf 2009 Tutorial – jRuby on Rails

Filed under: Uncategorized — Charles Engelke @ 12:06 pm
Tags: , ,

I’m starting this year’s RailsConf with a tutorial on jRuby on Rails.  I’ve fiddled with it a few times, and I have the distinct impression that it’s a more stable production platform than the regular Ruby interpreter.  But there are a few issues I’ve had.  I hope today’s tutorial will help me resolve them.

I’m not much of a note taker, so this (and other posts this week) will probably be pretty thin, with just reminders of core information tidbits I glean here.

The speaker is Nick Sieger of Sun Microsystems, and he’s put his slide desk online at his blog.  The link to the actual slides isn’t there (yet), so I won’t put it here, either.  It looks like he’ll probably soon put up a post with the link.

The talk is going to start from absolute zero.  Since I’ve actually installed and used jRuby (very casually) there will be some familiar stuff for me at first.

Download GlassFish, a Java application server we’ll be using.

Download jRuby, too (this is the latest production version as of today).

You’ll need the real JDK, too.

Install the JDK.  Add its bin subdirectory to your path, and set an environment variable named JAVA_HOME to the root of the JDK install.  Test it by running “java -version”.

Install jRuby by unpacking it anywhere you want.  Add its bin subdirectory to your path (he emphasizes adding it to the end of the path).  Test it by running “jruby -v”.  Run irb or other Ruby tools by prefixing them with “jruby -S”, as in “jruby -S irb”.

Run jRuby with the Java “server” VM with “jruby –server”.  That’s better overall performance, at the cost of slower startup.  You can pass a variety of Java arguments to the JVM with “jruby -J<argument>”.

Oh, you want jRuby at the end of the path so that its utilities don’t run when you want the regular Ruby ones.  So “rake” gives the regular Ruby rake, “jruby -S rake” gives the jRuby one.

jRuby has all the standard 1.8.6 Ruby things – the libraries, RubyGems, Rake, RSpec.  It’s also integrated with the Java libraries.  Go into jRuby’s irb, and type

java.lang.System.out.println “Verbose Hello”

It works!  Not that you’d want to do that, but I’m sure I’m going to want to use the Java cryptographic tools.

Install a bunch of needed gems (I had to be in a command shell with administrative privilege to do this in Windows 7):

jruby -S gem install rails mongrel jruby-openssl

Now, just create a Rails application with “jruby -S rails appname”.

Install ActiveRecord-JDBC or MySql or SQLite3 (or both):

jruby -S gem install activerecord-jdbcmysql-adapter
jruby -S gem install activerecord-jdbcsqlite3-adapter

Now fix your Rails application’s database.yml to point to the JDBC version of the appropriate database adapter.  For example, “adapter: jdbcsqlite3″.

Now here’s a trick I could have used (and will be using)!  Use embedded Ruby to have a single configuration file work for both jRuby and MRI.  Put this in database.yml:

<% jdbc = defined?(JRUBY_VERSION) ? ‘jdbc’ : ” %>
development:
adapter: <%= jdbc %>mysql

[I just noticed that WordPress is "fixing" my single and double quote marks, even in code.  It's easier for me to tell you figure out that they're really all straight quotes than to figure out how to get them in WordPress right now.]

And now create a Rails application in the regular ways.  Just remember to use “jruby” instead of “ruby”, and to prefix other commands (like rake) with “jruby -S”.

Found a bug (in jRuby? in Rails? hard to say, but probably either in Rake, or in a Rails Rake task): if your Rails application’s environment.rb specifies needed gems, you can usually install them with:

rake gems:install

But, if you use:

jruby -S rake gems:install

it says it works, but it doesn’t (“jruby -S gem list” doesn’t show them).  It appears that it installs these gems under MRI instead of under jRuby.  Of course, you can just manually install each needed gem with “jruby gem install”.  It’s just the Rake task that gets derailed.  Note that this happened under Rails 2.2.2, which is what our existing application under development is currently using.  I haven’t checked to see if it still happens in Rails 2.2.3 (the current version as of today).

You can speed up Mongrel under jRuby by removing the mutex it no longer needs.  You have to edit Mongrel’s source to do this.  Look for the block starting “@guard.synchronize” and comment it out.  You have to also set “config.threadsafe!” in your application’s appropriate environment file to have this help (or change it globally for Rails under jRuby by editing lib/initializer.rb there.

The speaker’s JRuby-Rack gem allows you to run any Rack-based application (like Rails) under any Java Servlet API server.  That sounds promising for production use.  I’ve got to learn a lot more about the Java server ecosystem to take advantage of it.

The talk is now really turning to ways an experienced Java shop can run Ruby on Rails well in their infrastructure.  There’s a lot of background I don’t have, so this can point me to the right place to start, but I can’t understand it well enough to use it as a result of this talk.

Lots of talk on Warbler – I don’t have the context to understand it now (or know why I should care).

Google App Engine with jRuby.  Now, this interests me.  There’s going to be hackfest on Wednesday night at CabooseConf (a side meeting to RailsConf) that I’m going to want to visit.  Of course, Java support on Google App Engine is very preliminary, and there are lots of restrictions.  But the speaker says that jRuby on Rails works!  Well, except for a few minor issues like no ActiveRecord.  Not real practical yet.  But very encouraging.  There’s a talk at 11:45 AM this Wednesday that goes into more details.

[Side note: I've been trying to get our current development application running here in different configurations, and found I couldn't get it to run under MRI using MySql (jRuby with MySql, or either with SQLite3 worked fine).  This isn't related to the talk, but I found the solution on Google.  If you're running a 64 bit version of Windows, you either need to be running MySql 5.0.x instead of 5.1.x, or else get the libmysql.dll file from an older version of MySql and put it in your Ruby's bin subdirectory.  I don't want to forget that the next time I wipe and reinstall everything on my laptop, hence this note.  It would be really nice if the precompiled gem for MySql gets fixed to avoid this problem, but I've noticed that the Windows platform is often not given much attention in the Ruby and Rails communities, so I'm not holding my breath.]

The tutorial has hit a real lull for me.  The Java ecosystem is incredibly rich, and layered upon layers.  It’s very hard for an outsider like me to keep track of the 27 billion different kinds of tools out there, and we’ve been talking for about an hour about those.  From the perspective of a Ruby person wanting to leverage some of that Java stuff, well… yawn.

We’re wrapping up now.  There was a lot of value to me in the first half of the talk, but not in the second half.  And I don’t think it’s just me.  It seems like the speaker got really bogged down at this point.  Overall, worthwhile, but it could have been even better.

May 2, 2009

Kindle Demographics

Filed under: Gadgets, Notes — Charles Engelke @ 4:26 pm
Tags:

Engadget pointed me to a post on the Kindle Culture blog that claims that most Kindle owners are over 40.  The research behind the claim is a bit informal, but it seems credible.

So (assuming this is true) why are Kindle owners older than other gadget owners?  The Kindle’s expensive, but I don’t think that’s it.  A lot of extremely expensive gadgets are very popular with teenagers and twenty-somethings.  The ability to change the font size may be a small factor, but I don’t think that’s behind this either.

I think it’s because lifelong readers tend to keep a lot of their books, and eventually there are so many they’ve become overwhelming.  I’m 53, and based on rough estimates from linear shelf space and overload factors, my wife and I have somewhere between 5000 and 10,000 books in our house.  It just seems natural to hold on to them, but we’ll only reread a tiny fraction of them.

The Kindle (well, Kindles, one for each of us) lets us keep reading new books without adding to the shelves, piles, and mounds of books we’ve got.  I was very slow to get one because of the DRM – I don’t really own any of the books I’m buying, and I don’t expect to be able to still read them a few years down the road.  But if I want to reread a book in a decade or two or three, I’ll just buy another copy then.  Thanks to electronic publishing, anything I want is likely to still be “in print”, and it’s less expensive to buy a few duplicate copies than to get a bigger house to save all that paper.  And the Kindle is just about as good a physical reading experience as a typical paper book.

I will still buy some paper books.  The Kindle’s screen isn’t big enough for most technical books (in my opinion).  And there are books I’ll want to have and keep for sentimental reasons.  But I’d be pretty happy if I could get the paper book count in the house down below 1000 some day.

Next Page »

Blog at WordPress.com.