First impressions of Measure Map from Adaptive Path

I got my invitation a few days ago to try out the alpha release of Measure Map, the blog stats service from the folks over at Adapative Path. After a few days of using it, I’m generally impressed. The quickest (but also crudest) way I can think of to describe the service is WebSideStory’s Hitbox or Omniture Site Catalyst for bloggers, since (like Measure Map) both of these services leverage the placement of Javascript code in a site’s pages to deliver reporting, freeing sites from the laborious crunching of log files, filtering out spider/robot traffic, and the many other annoyances of old-school methods of traffic reporting on the web. That being said, even if WebSideStory or Omniture decided to create a blogger offering, you can bet it wouldn’t be as simple, elegant, and useful as Measure Map. Even though it’s alpha, it looks like they’re building the right half of a product, but not a half-assed product. The tag line is “get to know your blog,” and that’s what Measure Maps is already helping me do.

Setup was easy for WordPress. I have no problem editing my templates based on rudimentary written instructions, but I still appreciated the clear visual guidance the Adaptive Path folks give in their instructions. Here’s an example:

Measure Map template editing

Almost immediately, the numbers started rolling in, and I found myself checking my Measure Map stats as often as I had grown accustomed to checking my Feedburner stats (incidentally, I consider these two services complementary at this point, since Measure Map measures non-RSS traffic, and Feedburner measures RSS traffic). Here’s a sample screen, the “Links” screen which tells me inbound links, outbound links (how else are you going to get that info without a bunch of ugly redirects and log crunching?), and search terms used to get to your site.

Measure Map links screen

The outbound links tracking is cool because it includes everything on your blog page, so you can see exactly which photos people are clicking on your Flickr badge, for example. The search terms section lets you know what words people are using to find you on search engines. Like everything else in Measure Map, the information is updated fairly instantly as activity occurs on your site.

Other stats I quickly learned:

  • the browser breakdown for my blog (58% Firefox, 26% IE, 12% Safari, 4% “other”)
  • the geographical distribution of visitors (73% U.S., 5% UK, 5% Canada, 5% Australia, 4% India, the rest spread among 12 other countries)
  • peak usage times (7-9am, 1pm, 5pm)
  • My top 10 posts (#1 is “Super-mashup with Yahoo! APIs: event browser“)

Bottom line: though only in alpha, Measure Map is already quite useful to me. Only one significant glaring hole that I noticed in the materials: no mention of an API on the Measure Map alpha status page under “feature set”: We’ve got a few great features coming soon, including stats for your RSS feed, tracking interesting events in your stats, and deeper tools for understanding search engine traffic. This might very well be on the way, but it would be nice to see it explicitly mentioned. After all, the service itself is being developed on top of some sort of API — why not start surfacing it early? It would be great to be able to do some remixing with the Feedburner API, for example.

Update (for those of you who don’t keep up with the comments): Jeff Veen from Adaptive Path writes: “We’re already working with Feedburner, with the intent of hooking your accounts together and merging the stats. Also, our first peek at a public API will be coming very soon now.”

Openness, advertising, and the Yahoo! Maps API

Amidst all this talk about disruption, maps, and advertising. here’s something I was super-glad to see: a bold clarification about terms of use for maps developers over at the YDN blog from Vince Maniago (yeah, we launched Maps yesterday but no one is sitting still!)

Our position has always been to allow usage of the Yahoo! Maps APIs free of charge for non-commercial use, as well as commercial use granted on a case-by-case basis. This is defined in our FAQ which also has instructions for how to contact us should you want to seek permission for commercial use.

In general, if you are displaying mashups featuring Yahoo! Maps on your site or application and you make your stuff available for free to users, you’re welcome to use the Yahoo! Maps APIs. This is true even if your site is supported by ads — even ads from other vendors.

Without a doubt, we still want you to sign up for for Yahoo! Publisher Network to display our ads on your site (for which you will receive a cut, of course), but we’re not going to ask you to use YPN unless your own stuff isn’t free.

Term Extraction API and TagCloud.com

One of the most inspiring backend pieces of the Event Browser for me was the innovative use of the Content Analysis Term Extraction API, which Ed describes in his post about the Event Browser:

One of the problems we had were that there were no images in our event feed. We knew we wanted to get images from either Flickr or Yahoo Image Search but it wasn’t immediately obvious how we would get an image from a phrase like “Highlights of the Textiles Permanent Collection at the MH De Young Memorial Museum”. It was another Yahoo, Toby Elliott, who suggested that we use the Content Analysis API and then Image Search. Honestly, I didn’t even know Yahoo had term extraction as a public API. To get the images you see in the demo I concatenate the title and venue to get the most important terms extracted and then use that as the image query. The first thumbnail gets used as the photo for each event. It’s really simple code on my end and all the real work is done by the term extraction service. My favorite example for images is the De Young Museum’s list of events.

I’m a total news and politics junkie (when I worked at CNN.com several years ago, it was like giving an alcoholic a job in a liquor store), so last week I started thinking about how using a tag cloud to represent breaking news from a set of RSS feeds that I choose. Last week being a big news week, I envisioned a dynamic tag cloud where words like “Scooter,” “Miers,” and “indictment” would get huge in breaking news situations to tip me off that something big was going on at that moment.

I started digging around and was getting ready to sit down and write the code, but then I found TagCloud.com, which uses (you guessed it) the Yahoo! Term Extraction API to produce a tag cloud built from RSS feeds you specify in an interface that is nicer than anything I could build quickly myself. No need to re-invent the wheel, so I signed up and had my tag cloud in a few minutes, using these feeds: CNN.com Top News, Fox News (to be fair and balanced), MSNBC, NY Times home page, Washington Post top news, and Yahoo! Top News.

Here is the resulting tag cloud from those sources. TagCloud.com offers a nice “stop words” feature so you can remove common (but useless) words from the tag cloud display. I specified “full story” and “story” as stop words, for example, but I also specified “white” and “supreme” because they were generally represented in the phrases “white house” and “supreme court” (terms which were preserved in the tag cloud even though I specified words within those phrases as stop words). (And check out their implementation guide for simple instructions on how to put your TagClouds on your site in badge form).

Displaying this tag cloud more dynamically in a Konfabulator widget would be cool. . . . TagCloud.com has already done the difficult work using the Term Extraction API, perhaps the most underappreciated API in the Yahoo! API arsenal. Sounds like a fun project.

Super-mashup with Yahoo! APIs: event browser

Event Browser In my twelfth week at Yahoo! I’m really happy to be able to finally point to something I have been working on with a small but incredibly talented team of engineers and UI designers (a couple of them even newer than me to Yahoo!) Check out the Event Browser, a super-demo of a bunch of Yahoo! APIs, with the exciting new Maps AJAX API we just announced as the foundation (and if you’re not in to AJAX there are many more Maps APIs to play with).

Frankly, I’ve been getting a little bored with maps mashups in general. Most are just a new set of points on a map from some newly-liberated set of data, which is cool but not as exciting as it used to be. This mashup is different, though. Instead of a standard query interface, the map becomes the center of the experience and your browse movement on the map determines the events you see in a very dynamic way. As you move around on the map, events taking place within your map space appear to the right of the map. All this goodness is happening completely client-side, i.e. Javascript making REST calls. There’s also a dynamic tag cloud with event categories that re-draw themselves as you move around. Very cool.

One more cool thing (hey, did someone use the word “cool”?) The images displayed for particular events take Yahoo! events output and pipe it through our Term Extraction API, then through the Image Search API to produce amazingly appropriate images for the event. Ravi Dronamraju, who put together the team that built this demo, provides his thoughts on this demo. A big thanks to Ed, Jonathan, Mirek, Karon, Sam, Nate, and Toby.

It’s a great team, and working with them on this reminded me of the concept of the “jelled team” from the truly excellent must-read software engineering management book Peopleware, which I wrote about at InfoWorld:

The jelled team is so tightly knit that the whole is greater than the sum of its parts. There is low turnover, a strong sense of identity, a sense of elitism, joint ownership of products, and enjoyment derived from participation.

Can’t wait to do more.

Update: Ed Ho writes in a little more detail about how the demo works, and kindly gives credit to his teammates.

Flock and WebOS

Now that I have Flock, I decided to test it out a little.  My first task, of course, was trying to post to my blog.  That’s what you’re seeing here (please forgive some of the wonky formatting).  I was hoping that the blog tool would allow for offline posting a la ecto, but that doesn’t appear to work.  When I tried to save a draft with no network connection (I turned off my wireless card and pulled my network cable for fun), the app went into a tailspin.  This inability to save locally in what is essentially an extension of the Firefox browser reminded me of Jason Kottke’s post about the WebOS way back in August.

Flock is perhaps the quintessential Web 2.0 company (technology and hype included), but after ten minutes with the application and some reflection on Kottke’s WebOS post, I think Flock could be more than simply “a Web 2.0 on-ramp” (as Kris Krug of Bryght calls it in this Wired story).  I think Flock might be thinking WebOS. From Kottke’s WebOS post:

So this is my best guess as to how an “operating system” based on the Web (which I will refer to as “WebOS”) will work. There are three main parts to the system:

  • The Web browser (along with other browser-ish applications like Konfabulator) becomes the primary application interface through which the user views content, performs services, and manages data on their
    local machine and on the Web, often without even knowing the difference. Something like Firefox, Safari, or IE…ideally browser agnostic.
  • Web applications of the sort we’re all familiar with: Gmail, Flickr, and Bloglines, as well as other applications that are making the Web an ever richer environment for getting stuff done. (And ideally
    all Ajaxed up to provide an experience closer to that of traditional desktop apps.)
  • A local Web server to handle the data delivery and content display from the local machine to the browser. This local server will likely be highly optimized for its task, but would be capable of running locally installed Web applications (e.g. a local copy of Gmail and all its associated data).

That’s it. Aside from the browser and the Web server, applications will be written for the WebOS and won’t be specific to Windows, OS X, or Linux. This is also completely feasible, I think, for organizations like Google, Yahoo, Apple, Microsoft, or the Mozilla Foundation to make happen (more on this below).

“Something like Firefox, Safari, or IE.”  “Web applications of the sort we’re all familiar with.”  “Won’t be specific to Windows, OS X, or Linux.” Pretty close to what Flock encapsulates.  The only area where Flock completely misses is the third point (basically running locally without loss of features), which is understandable given the constraints of current browser technology and the fact that Flock is a “developer preview.” I would still love to be able to save a draft blog post locally using Flock while disconnected on a plane, though.

Going back in the time machine to Kottke’s oh-so-long-ago post in August, he asks, “So who’s going to build these WebOS applications? Hopefully anyone with XHTML/JavaScript/CSS skills, but that depends on how open the platform is. And that depends on whose platform it is. Right now, there are five organizations who are or could be moving in this direction.”

Aside from the usual suspects (including my employer, where we are definitely doing lots of cool things as the post suggests), Kottke lists the Mozilla Foundation:

This is the most unlikely option, but also the most interesting one. If Mozilla could leverage the rapidly increasing user base of Firefox and start bundling a small Web server with it, then you’ve got the beginnings of a WebOS that’s open source and for which anyone, including Microsoft, Google, Yahoo, and anyone with JavaScript chops, could write applications. To market it, they could refer to the whole shebang as a new kind of Web browser, something that sets it apart from IE, a true “next generation” browser capable of running applications no matter where you are or what computer (or portable device) you’re using.

Flock isn’t the Mozilla Foundation and they are “completely independent,” but reading Flock CEO’s Bart Decrem post about how Flock will create “sustainable value” (the answer to all the what is your business model? questions) gives me pause. If you replace Mozilla with Flock in the Kottke quote above you might be tempted to think that Flock is working towards something a lot bigger than a cute Web 2.0 browser. Besides, who needs another browser anyway?

And who needed another search engine?

Want Akismet? Then download Flock. Huh?

(Note on 11/10/05: getting the API key was mildly painful, but Akismet is doing an EXCELLENT job of catching spam. I have zero problems now — it was well worth it!)

I ran across a mention of Akismet, the WordPress spam blocker, on the BusinessWeek blog and since I’ve been seeing a bit more spam these days, I decided to try it out. WordPress has been good to me, and this sounded like just the thing I needed to make my experience almost perfect.

I clicked over to the Akismet site, and see “Akismet is free for personal use.” Looked good. “All you need to start using it today is a WordPress.com API key. Sign up on WordPress.com to get an API key.” No big deal — I sign up for API keys all the time.

I followed the “Sign up on WordPress.com to get an API key” link over to WordPress.com and saw this message: “Want WordPress.com? Then download Flock. . . .”

want wordpress? download flock

Huh? I just want an API key so I can try out Akismet and I followed a link that promised me just that. What does Flock have to do with this? Why should I have to download Flock?

Frustrated, I downloaded Flock, but it was guesswork from there on how to get an API key. I clicked around on the “Five ways to. . .get started” on the Flock startup page (with the Flock app) and chose #2, “. . . get yourself a blog.”

five ways

I was presented with three choices: WordPress, TypePad, and Blogger. I chose the WordPress link and filled in the requisite information (note that “chad” was taken, so I used “chaddickerson,” but didn’t do another screen shot).

wordpress signup

At that point, I got an e-mail with my username and password to login to my WordPress.com blog. Yay. So I logged in to my WordPress.com blog, still not sure where this elusive API key existed. Randomly, I clicked around until I clicked on “Profile” in the main admin menu. And there it was! My API key!

WP API key

This process makes almost zero sense. I expected a lot more from WordPress, and getting siphoned through Flock to get an API key smacks of some backroom arm-twisting marketing deal. Getting an API key has never been so convoluted.

Fortunately, installing the Akismet plugin was as easy as any other WP plugin and I got it installed in a couple of minutes. I entered the API key with no problem. Whew. Now I’ll see how Akismet works.

Oddly enough, it wasn’t until the plugin was installed that I came across a reasonable link about the API key, found on the plugin manager page (yes, it’s a little hard to see below):

plugin manager

For those who don’t want to go through the aggravation I just did, here’s the text:

Akismet checks your comments against the Akismet web serivce to see if they look like spam or not. You need a WordPress.com API key to use this service. You can review the spam it catches under “Manage” and it automatically deletes old spam after 15 days. Hat tip: Michael Hampton and Chris J. Davis for help with the plugin.

What a strange process. I sure would like to know the thought process that went into it, because I certainly can’t make any sense of the actual process.

Update: Michael Hampton responds in the comments, and this post makes things clearer:

Last week I told you all about Automattic Spam Stopper, the new anti-spam solution for WordPress from Matt Mullenweg. There’s been some new news, and you’re going to hear it here first.

First off, the plugin has been renamed to Automattic Kismet, or Akismet for short.

Second, it now requires a WordPress.com API key, which you can find on your WordPress.com Profile page. (Click My Dashboard, then Profile.) If you don’t have a WordPress.com account, you won’t be able to use Akismet at this time, until you somehow finagle yourself an account. The fastest way is probably to use Flock. You don’t actually have to blog at WordPress.com to use Akismet, you just need the account to get the API key. You can use the API key at more than one blog, too.

Scaling: still hard

I think one of the most stubborn myths of our times is that it’s “easy” to build a web company because servers, bandwidth, etc. are so cheap. I’m not so sure. Yes, it is “easy” — until you need to scale. Scaling is still hard.

I was reminded of this when I read Mena Trott’s post about Typepad’s recent problems, in which Mena writes about the pain they are going through and what they are buying to fix it:

Over the next week you should see significant improvement in performance as we get extra equipment on line and finish moving data off of heavily loaded servers. By the end of the move we will have five times the bandwidth we had before, as well as hundreds of thousands of dollars of new equipment, and room and power to add more equipment as needed.

Scaling is hard — and often expensive. Cheaper than it used to be, but still not cheap.

This also reminded me of Dave Winer’s sale of weblogs.com to Verisign, in which he wrote:

The bootstrap of weblogs.com is something a bigco should not attempt, it’s hard to make it go, and most bootstraps don’t, and it requires trust, something an individual is more likely able to inspire than a big company. On the other hand, running a serivce that other bigco’s depend on (like Google, and Microsoft, to name two) is not something a person like myself should attempt [stuff snipped here]. . . I’m good at digging holes, I have to pass off to others to make the trains run on time when the service grows as big as weblogs.com has.

Judging from this post, Dave understood the difficulty of scaling and smartly chose to sell weblogs.com before he hit the wall with it.

I think scaling issues are going to separate the men from the boys in the coming months. I think that many of the Web 2.0 “Flickr-wannabe, flip-it-quick” companies (as Russ describes them quite well and with good humor) have no idea how to scale (PHP and MySQL are fast. . . let’s just write the code!) and when they get popular enough to warrant interest from larger companies, they may be on the verge of being crushed (or already crushed) from a scaling standpoint. The ones who figure out how to scale on the backend while they are blinding the world with AJAX wizardry on the frontend will do very well, I think.

Bonus link: Flickr ops main man John Allspaw recently posted his Hardware Layouts for LAMP Installations presentation from the PHP/Zend conference. Following some of John’s hard-earned advice is a good way to get those scaling issues in order (and it’s still not easy).

Peter Guralnick: from Elvis to Sam Cooke

Last Train to Memphis Peter Guralnick’s two-part biography of Elvis Presley (“Last Train to Memphis: The Rise of Elvis Presley” and “Careless Love: The Unmaking of Elvis Presley“) remains the most compelling work of non-fiction I have ever read (and perhaps the most compelling anything I have ever read). It’s an absolutely sprawling but thoroughly amazing work. Taken together, the two volumes consist of nearly 1400 pages about the life of a man who was obsessively covered by the media of his time and endlessly chronicled after his death. After reading Guralnick’s biography though, while you might recognize an anecdote you’ve heard before (e.g. Elvis shooting his television), the picture of Elvis that emerges is somehow unexpected and surprising (e.g. Elvis taking LSD and sitting by his mother’s grave). I read the first volume in roughly 1996, and it covered from Elvis’ birth until he entered the Army. I had to wait three years for the second, and more explicitly tragic, volume to be released. It picked up where the first volume left off, traveling through the ’68 Comeback Special, the Hawaii and Las Vegas decline, and ultimately ending in Elvis’ lonely and pathetic death.

Careless Love I was reminded of the Elvis biography and Guralnick’s incredible work tonight when I read Charles Taylor’s Salon review of Guralnick’s new biography of Sam Cooke, “Dream Boogie.” In the review, Taylor writes that Guralnick’s magisterial work on Elvis Presley, which can leave you feeling unmoored for days, convinced that you have just read, as Guralnick claims, “the saddest story” he knows, can stand alongside Taylor Branch’s ongoing “America in the King Years” and Robert Dallek’s two-volume life of LBJ as one of the greatest recent accomplishments in American biography. When I read the two books, I remember being so immersed in the overwhelming sadness of Elvis’ life that I could think of nothing else for weeks.

Even if “Dream Boogie” merely hints at the greatness of Guralnick’s earlier Elvis work (and Charles Taylor’s review suggests that might be the case), it’s still a must-read for anyone who cares about music.

The Y2K that wasn't

For reasons I would rather not go into (hint: my idea of Friday night entertainment is tending towards the geriatric and my girlfriend was in Milwaukee hanging out with famous movie directors anyway), I decided that tonight was going to be the night that I sorted through my stack of old VHS tapes with the hopes of tossing 99% of them. Digging through the pile, I found one tape labeled “Y2K worldwide collapse in 2000”:

Y2K collapse VHS tape

The redundant nature of the label is amusing to me by itself, but there’s a story behind this tape that I had forgotten. During the crazy housing boom (which I believe has ended, or at least subsided), I often wondered how I got such a good deal on my house back in 1999, but this tape reminds me. The man who sold the house to me was divesting of his Bay Area real estate at the time. While I was in the process of completing the purchase, he was fortifying a house he had bought in Sacramento in the final days of 1999. My memory is a bit hazy, but I remember talk of generators, tanks of drinking water, and stockpiles of canned food. I think that I was handed this tape during one of those conversations (a cursory viewing tonight showed preachers Stan Johnson and Mark Andrews of the Prophecy Club forecasting the doom that would come upon us all when we flipped over to January 1, 2000). I probably nodded, said, “thanks,” and never viewed the tape.

I do remember that I ran into the guy who sold me the house a couple of days after Y2K proved to be a non-event. He shrugged his shoulders and shook his head: “Oh well.”

My advice: take out a mortgage in a doomsday scenario if you get the chance. If the doomsday doesn’t come to pass, you’ll be happy you took the plunge. If the world ends, there will be no one left to collect on the mortgage. You can’t lose.

The law of modern shipping

Here it is, personally witnessed enough times now that I have promoted it from simple theory to law:

If you order something to be delivered to your house on your work-at-home day, it will be delivered a day late, you will be at work, and you will get stuck in the loop of multiple delivery attempts. The carrier will max out on re-delivery attempts before your next work-at-home day, and you will be hauling yourself down to a distribution center on the edge of town to pick up your package. If you order something to be delivered to your workplace on a day you will be there, it will be delivered to your workplace on your work-at-home day.

(I’m working in Berkeley today and the big monitor I ordered to make myself more productive is sitting at Yahoo! HQ, delivered a day earlier than expected. Sigh.)