Remembering 9/11

On September 11, 2001, I was living in Berkeley, California. One of the things I remember most is the generous outpouring of sympathy for New York and the United States at the time, with Le Monde in Paris writing beautifully Nous sommes tous Américains: we are all Americans (translated version). Having lived in New York for two years now, what I didn’t fully appreciate then was the intense love that New Yorkers feel for the city. The towers themselves could be seen across the East River from the bedroom of my apartment where I live now in Brooklyn, which I know now because I see the Tribute in Light from that spot today. The city has become my own in the past two years of living here and I feel the impact of 9/11 in a way that I didn’t then.

Like most people, I had good friends in NYC at the time, and I caught a close friend working in midtown on IM that morning shortly after the second tower fell. I saved the transcript, and reading it now brings back the chaos and worry of that morning, the feeling of not knowing if the people you cared about were ok, and concern for the people who experienced such horror, mixed in with the human spirit of dark humor from someone who was in a frightening situation. I was lucky that my friends and loved ones were ok, but felt such a sense of sympathy and concern for those who weren’t sure about their love ones.

me: everything ok where you are?
friend: or relatively so?
friend: pendemonium
friend: i'm shaking so hard i can't even see straight
me: i was glad to see you pop up on IM.
friend: thx
friend: i'm looking for [friend of his]
friend: who works downtown
friend: classic [big company] internet moment. i can't post new content to [the web site he was working on]
me: the internet is pretty much hosed right now.
friend: i was on the subway right as it happened, evidently. 8:45
friend: i was just in the WTC on Monday
friend: labor day
me: i'm staying at home. . . not risking driving over the bay bridge today.
friend: i'm gonna be sick
me: is there any way to get back to queens?
friend: don't know, the bridges are closed
friend: nothing stop sspam. i just got 3 mails to enlarge my breast size
me: not surprised.
friend: was outside. so much chaos
friend: not safe to walk around even up here
me: why is it unsafe there? just too much panic?
friend: too much panic and confusion and emergency vehicles

Our conversation ended then, but I at least I knew he was ok at the moment, if not very shaken.

At that point, the news from CNN got to be overwhelming and I hopped on my bike to ride up into the Berkeley hills from the flatlands where I lived. When I got up there (about 1300 feet above sea level), I took some deep breaths and took in the views of the bay that had 101-0102_IMGalways had such a calming effect on me. That day, I remember gazing at the Golden Gate Bridge and taking a photo of it knowing that it had been evacuated out of concerns that terrorists might fly planes into it or blow it up. Would I be one of the last to see it intact? I took some other photos that day, in part to document what that day was like but also because I was legitimately wondering if something might happen to alter the landscape that I was seeing, not to mention the lives of those living in it. I thought of T.S. Eliot’s The Waste Land. These were strange and frightening feelings, but I had seen the towers fall on TV just hours earlier and the rumor mill of what might happen on the west coast was going at full throttle.

I visited NYC about six weeks after 9/11 and the memorials around the city brought me to tears many times. I remember coming across a donation station for food and treats for the dogs who were still looking for victims, and there was still smoke in the air downtown as the remains of the building smoldered. The city was very much in mourning.

My heart goes out to those who lost friends and loved ones that day.

Hacking Matters panel at SXSW

I’m super-excited about the panel proposal, Hacking Matters, that my friend and fellow hacker Tarikh Korula put together for SXSW. Havi Hoffman (@freshelectrons) will be joining us, too. I’ve been to SXSW three times now but have never done a panel, so I hope we get in (vote us up!)

Here’s what I wrote in the comments on the panel picker (I added some links here, which you can’t do in the SXSW comment system). (Read on for some links to some past presentations and blog posts on the subject that I collected that will give you a hint of what we’d be talking about in the panel.)

Hack Days are pure magic — events put together for engineers, by engineers, with the highest signal-to-noise ratio possible. PowerPoint draws hisses, and working demos rule the day. At Yahoo, with the generous help of Havi Hoffman and hundreds of hackers around Yahoo, I put together internal Hack Days on three continents before we did our first Open Hack Day in 2006, where I met Tarikh Korula, who won a prize [along with his partner Josh Rooke-Ley]. In May, Tarikh, Daniel Raffel, and I teamed up with Techcrunch to put the Hack Day together for their Disrupt Conference in NYC. Each event was special in its own way, but common themes and approaches made each one successful.

In this panel, we’ll tell you about our collective experiences with Hack Days and how you can put together your own successful Hack Days, whether you’re a 3-person startup or a Fortune 500 company. We hope you’ll give us a thumbs up! (Chad, @chaddickerson)

I wrote about the magic and unexpected wonders of Hack Days a lot when I was putting them together for Yahoo! — including once when the building hosting us in London was literally struck by lightning — but I’m most excited about giving some updated tips based on helping organize successful Hack Days in entirely new contexts, like the one we put together for Techcrunch Disrupt in NYC and what we’re doing with Hack Days inside a startup like Etsy (which I mentioned in my scaling startups post). Here are a collection of useful things from the past that I would likely be referring to in the panel:

In any case, I’m hoping we’ll see you in Austin next March! (Wait, did I say you could vote for us? Just making sure.)

Scaling startups

This blog post was also syndicated to Fast Company.

The Scaling Startups panel I was on last week at Supernova generated a little coverage, but I wanted to go into a lot more detail than what I saw there, so it seems like a good time to jump back into blogging full-force. The panel abstract read:

How can startups develop a distinctive culture, and sustain it as they grow? Are the characteristics that define today’s high potential technology startups the same as five or ten years ago? And does the current environment of low-cost infrastructure and open information sharing give new companies an advantage, or make it harder for them to maintain a competitive edge and reach critical mass?

Coming from the engineering side, I’m very much a pragmatist about all of this since most of my time is spent on shipping products and the systems to support shipping products. Before Etsy, I had done the “startup within the big company” thing and the Web 1.0 startup dot-com IPO thing, both of which have influenced my thinking on the subject considerably. In the panel, I focused mainly on what it takes to institutionalize a distinctive culture focused on shipping and how to sustain that when you’re growing fast. What you read below is based on the notes I made before the panel along with what I remember from the panel itself. supernova forum 2010: scaling startups I tend to take bad notes during panels because I’m focused on participating, so I don’t have much about the remarks of my co-panelists, who were great (Ethan Mollick of Wharton, our moderator, Steve Cohen of Morgan Lewis, Steve Barsh of PackLate, and Andreas Weigend, advisor to a number of companies and former Chief Scientist at Amazon). A video of the session will be available soon, though, and Jonny Goldstein did an incredible job drawing a visualization of the talk in real time! (see photo on the right)

At Etsy, we’ve quadrupled our engineering team in the past year, so everything I write below is based on that experience. Much of what I write about below was put in place in the last six months, and I think it’s working.

First, to scale a startup, you should always remind yourself why people work at startups in the first place:

  • they like to move fast
  • they want to make impactful decisions
  • they want to DO things and BUILD things (not sit in meetings awash in PowerPoint)
  • they want to be able to take risks

For various reasons, startups and the people who inhabit them as they grow tend to become more risk-averse over time. There’s a great quote from Peter Drucker about risks — laminate this quote, frame it at your desk, make it your screensaver, and put a copy of it in your wallet:

People who don’t take risks generally make about two big mistakes a year. People who do take risks generally make about two big mistakes a year.

In talking about culture earlier, I use the word “institutionalize” very deliberately. You have to actively develop a culture of speed and informed risk-taking and put real mechanisms in place to continually reinforce that culture, especially when you’re growing quickly. Pontificating about wanting to move fast isn’t enough and is often just counter-productive. You have to actually move fast.

You have to have the right amount of process, too, but not too much. In most companies, process can be a little like a trash heap that only grows larger and larger over time. You have to be vigilant about keeping that heap from growing. One of my favorite quotes on process comes from Clay Shirky:

Process is an embedded reaction to prior stupidity.

Process is sometimes put in place because one person did one stupid thing once, sometimes literally YEARS ago. What about the other 999 times that nothing went wrong and progress was made while ignoring the process? Leaders should question process at every turn (though tend to your “good” processes, which are important. The larger your team gets, the more you need some clear rules of engagement or folks just bump into each other.) I recall when Fred Brooks spoke at Etsy, he described some of his career experiences and punctuated one by saying (I paraphrase):

If you follow process religiously, you’ll never get anything done!

(I implicitly trust Fred Brooks.)

To scale as you grow, you have to put processes and structure in place that encourage everyone in your company to take more risks (not fewer), keep the right people informed of the risks you are taking, and protect individuals from silly mistakes that would tempt your company to institute unnecessary embedded processes. Just as many mistakes are committed from people being tentative than being aggressive anyway (the point of Peter Drucker’s quote).

So, how do you do it? Here’s what we do to institutionalize speed and risk-taking at Etsy (your mileage may vary):

Hire well

This goes without saying, and I didn’t mention it in the panel. It’s a big topic probably best left for another post. Hiring great people makes everything else below easier.

Communication

Everyone in the company uses IRC, not just engineers. Everyone, all the time, from the CEO on down. Sure, sometimes you can miss things if you’re not in IRC at the time, but the benefits far outweigh the costs, and you have a lot fewer meetings about day-to-day mundane issues.

Deploy code early and continuously

At Etsy, most new engineers deploy code before they are done with their health insurance paperwork. We practice roughly what has become known as “continuous deployment.” This means one-button deploy, period. We’ve invested in the tooling to make this possible, so it doesn’t come for free. Engineers don’t have to wait to ride on a 2-week release: you can do it now. This is all supported by thousands of real-time monitoring checks, business metrics, and automated tests, so it actually requires MORE discipline and ongoing attention than many shops who have multi-week (or month – yikes!) release cycles. A lot of people say, “that sounds risky!” Well, not really. The team at Etsy once reported a serious XSS bug to a site (which will remain nameless) and they said they were going to deploy a fix — in six weeks, because they only deploy every two months. Talk about risky! We did 204 production deployments in July, about 30% more than June, and we are ready to deploy at all times. Deploy, deploy, deploy. (Doing this requires close dev/ops cooperation. The canonical presentation on that subject is 10+ Deploys Per Day: Dev and Ops Cooperation at Flickr by John Allspaw and Paul Hammond. John has been at Etsy since January.)

Oh, and remember IRC? We have IRC bots that automatically announce code deploys in real-time. Those messages look like this:

PRODUCTION deployed by cdickerson build: 32445-trunk-20100804-033748-UTC took: 17 seconds diff: http://awesomeness.internal.etsy.com/diff/web/32441/32445

Since everyone is in IRC, everyone knows who deployed, when we deployed, and what is being deployed (this also goes out in email to the whole engineering team). This level of transparency is what distinguishes how we operate from “cowboy coding,” which often works in early-stage startups but becomes destructive and distracting as a team grows.

Encourage experimentation

With minimal review (“minimal” meaning making sure key privacy, legal, and other issues are taken into consideration), any engineer at Etsy can push a test application out to our members to get feedback, and it could totally fail. We announced this to the community with all appropriate caveats. Allowing experiments eliminates the “no one will let me do my idea” problem that starts to break down startup culture. This is all about reducing the “idea approval index.” We’re just getting started on this one and I’m really excited about it.

Hack Days

One of our teams did a hack day last week where the engineers on the team went off-site and defined and built a single feature and released it on the same day. Always remember: “Every good work of software starts by scratching a developer’s personal itch.” It almost doesn’t matter what the feature is: it’s the cultural idea that you can JUST DO IT that matters most. The Hack Days we’re doing at Etsy now have one key distinction from the ones I organized at Yahoo (though they share the same inspirations) — you build and ship the same day. Our continual investment in tools and culture makes this possible.

Build a culture that loves engineers

I see folks mess this up all the time, and unfortunately it is a difficult thing to teach since it’s so intuitive. If you’re building a technology-driven company, you better have a culture that loves engineers, and I mean love. All too often, I see entrepreneurs who say they “just need engineers” to “bang out the code” for this great idea of theirs. If you view engineers as interchangeable factory workers instead of partners and creative people, you’re in for a tough time getting huge in a world driven by technology. To scale, you need to tend to engineering culture in your company and build a company where engineers love to work. The headlines about tech companies are always about the comings and goings of CEOs, but the rise and fall of tech companies is usually correlated to where the great engineers are going (of course, great CEOs know this and draw great engineers). I tried to express our view of engineering at Etsy on the about page of our engineering blog, Code as Craft (more on that below). Engineering is creative work, and often joyous work at that. No one describes the inherent joy of building software more eloquently than Fred Brooks when he answers the question “why is programming fun?” Recognizing and encouraging that sense of joy is core to any great engineering culture.

External transparency

We talk about all of this stuff in our engineering blog, Code as Craft, which makes our interview and hiring process more productive since folks come in knowing how we think and operate, and even some of the people on the team. We also get help and advice from the community with particular challenges like batch resizing 135 million photos.

Embracing failure

Moving fast often means making mistakes, though still not as many as you would think (again, the Peter Drucker quote). In the panel, I mentioned the award that Flickr gives annually “to the individual who breaks flickr.com in the most spectacular way.” Of course, this is a little tongue-in-cheek, but the point about accepting mistakes is not. Spectacular “failure” means you were probably working on something big and had a spectacular learning experience. (Side note: my remarks on the panel were construed to mean that we give such an award at Etsy — as reported at CNET — and I don’t think I actually said that; however, I immediately emailed the staff back at Etsy to start brainstorming a name for such an award at Etsy. Stay tuned.) “Rewarding” failure is not some crazy “those dot-com kids!” kind of thing. Companies like pharmaceutical giant Merck have institutionalized practices to deal with failure:

[R&D head Peter S.] Kim is promising stock options to scientists who bail out on losing projects. It’s not the loss per se that’s being rewarded but the decision to accept failure and move on.

Of course, catastrophic mistakes are not to be celebrated, but those sorts of mistakes are exceedingly rare and you should have processes in place to deal with them (including a defined post-mortem process). Most of the tension in companies around mistakes is about fairly mundane issues and is relatively unproductive (again, look at the Drucker quote you have in your wallet).

In the Q&A for the panel, someone asked: “This sounds great for engineering and product — how do you scale everything else at a startup?” While I focused primarily on engineering because it’s what I know, I have a huge appreciation for well-run HR, finance, and other operational functions inside a company because they are critical. They can often be the unsung heroes. You can’t add server capacity quickly if the finance team doesn’t help you move quickly.

To illustrate the point, I asked the crowd how many of them knew who Peter Grant was (only one person did). Peter Grant was Led Zeppelin’s manager and was famously quoted as saying: “Led Zeppelin looks after the music and I do everything else.” As you grow, make sure your startup has Peter Grants to keep the band focused on creating awesome music.

Review: Google Nexus One phone

(one of my New Years resolutions is to write more — here’s a start! Happy New Year!)

I was lucky enough to get a pre-release Google Nexus One phone just a few weeks ago. Engadget has an extended review of the phone, so I’ll leave it to them to give you all of the raw specs and details. This is just my own experience with the phone. Note that I didn’t really spend significant time reading about how to use the phone and judged it largely on how intuitive it would be on its own. If I’m missing any simple tricks, definitely let me know.

First of all, I was excited to get the phone in the mail, but mainly looked at it as a curiosity and a toy to play with before I went back to my beloved iPhone. I decided to take the SIM card out of my iPhone for an afternoon on the day I got the phone, and that afternoon turned into a few weeks. Just now as I’m writing this, I had to turn on my iPhone to remember what apps I was missing (more on that later. . . but I guess I wasn’t missing them that much).

Here’s the breakdown:

Pros

The phone is unlocked. This is huge, and if it hadn’t been unlocked, I would have never been able to try it so easily with my AT&T SIM. Openness FTW. I didn’t really notice that I was only at EDGE speeds.

The apps are solid. I didn’t do exhaustive app research, but I very quickly installed and used these apps: AndChat (IRC), Bank of America, ConnectBot (ssh client), eBuddy (IM), Evernote, Facebook, FlightTrack, Foursquare, GPS Status, gStrings (guitar tuner), Google Maps, Flixster, NYC Bus & Subway, Pandora, Scoreboard, ShopSavvy (barcode comparison shopping), Twidroid (Twitter client), WeatherBug, and Yelp.

Google Voice. If you use Google Voice on this phone, it absolutely KILLS Visual Voicemail on the iPhone. I thought Visual Voicemail was one of the greatest features of the iPhone, so this is significant. Imagine Visual Voicemail with automated (though not 100% accurate) transcripts of the voice mails. The Google Voice experience with my iPhone was frustrating since it isn’t well-integrated into the iPhone. This is the Google phone, so as you would expect, the integration is tight.

Background apps. Anyone with an iPhone knows what it’s like to switch between apps, especially mail using IMAP. I hated waiting for my mail to come down when I switched to the mail app on the iPhone. With the Nexus One and Android, it all happens in the background while other apps are running. In this way, it feels more like a Blackberry experience. This also makes installing and updating other apps much more seamless and behind-the-scenes.

Removable battery. Not since I had my old Treo (my last pre-iPhone phone) have I had the ability to carry an extra battery. That is huge given the poor battery life of today’s power-draining phones. I don’t have the extra battery yet, but I’m sure I will get one.

Cons

Some apps are missing versus the iPhone, as you might expect. I’ve become a Dropbox devotee, and there’s no Dropbox client for Android. I didn’t miss it too much, though, and Dropbox is advertising for a developer to build the client. While I have this as a “con,” the fact that the company behind the app I’m missing most is looking for a developer bodes well for the Android ecosystem. Amazon’s Kindle app isn’t available, and I used to switch between reading books on my Kindle and iPhone readily (especially nice when stuck on the subway with limited space to read). In any case, I think I can live without these.

Something about the way the “home screen” works just doesn’t seem intuitive to me. I don’t have trouble navigating, I just have to think about it every time. Not sure how to describe this in writing!

I miss the hardware sound on/off switch on the side of the iPhone — much easier than the software menus on the Android.

Overall

Overall, I love the phone and am planning to stick with it. If you’re largely using Google applications (Gmail, Voice, Maps), the integration is seamless. Philosophically, the openness of the Android platform is appealing. In the end, the Nexus One turned me into a new Android fan.

Prague's Franz Kafka Airport Named World's Most Alienating Airport

Haven’t blogged in a while, so going to shake the rust out of the engine with a little dark humor (side note: when I joined Salon back in the day, I named our first server “kafka”).

To those of you on the Q train this morning, that was me staring at my iPhone and cracking up. I have newfound respect and admiration for Bobbie Battista.

http://www.theonion.com/content/themes/common/assets/onn_embed/embedded_player.swf
Prague’s Franz Kafka International Named World’s Most Alienating Airport

The Etsy API

In work-related news, I wrote today about the Etsy API we’ve been working on over on Etsy’s blog, the Storque:

http://www.etsy.com/storque/etsy-news/tech-updates-handmade-code-etsys-beta-api-3055/

We’re letting a select group of developers kick the tires in an invite-only beta. If you’re interested in participating, email developer@etsy.com (I’ll see the email there). We’ll have more to say as we get closer to a general release, so stay tuned.

I love this stuff!

We are hiring at Etsy – come join us!

I’m hiring a few key positions on the engineering team at Etsy. See the links below for more details, but before you jump to read them, I wanted to say a little more about the type of person I’m looking for. Of course, I want people who write great code and get things done, but more than anything else, I want to hire people who care about the buyers and sellers that make Etsy what it is.

These are jobs that matter. Each of these positions is a dream job (I know mine is!) You will be building new features for and maintaining a marketplace that creative people around the world depend on to make a living. You will be helping enable and surface beautiful stories like the stories of Robin and Kathy Tucker of woodmosaics. If you read their profile, you’ll learn about how Robin developed his style and approach to woodworking, and what tools he uses, including the foot-powered table saw he made with his brother. The profile ends with this:

I know of no better way to make wood stand out and be truly beautiful.

For a lot of people, this is what dreams are made of, they dream of buying it and I dream of selling it.

Dreams do come true!

How many places can you work where your community says you make dreams come true?

Despite some of the gloom-and-doom in the larger economy, Etsy continues to grow. When I’m talking to people about Etsy as a business, I always ask, “Want to guess what the biggest three days in the history of Etsy were?” Usually, the person thinks for a bit and guesses “the day after Thanksgiving last year” and a few other random days. If I ever ask you that question, here’s a hint. Lately, the answer is nearly always the same: the three biggest days in history at Etsy were yesterday, the day before that, and the day before that. You can read some of the numbers yourself in the monthly updates on our blog. The growth of Etsy is a testament to the continued passion and dedication of the community that you would be working hard to support.

Here are the positions we’re hiring (all are based in Brooklyn):

  1. ArchitectDirector of Engineering
  2. UI Engineering Lead (hired)

If any of these sound like a dream role for you, and you find Etsy as inspiring as I do (just do a search for “love etsy“), please email me with your resume and tell me why you’re the perfect person to join the engineering team at Etsy (chad -at- etsy dot com).