Finding your courage

At Hello Etsy in Berlin a few weeks ago, I gave a talk titled “Finding Your Courage.” It was a different subject than what I’m accustomed to talking about and is probably the most personal talk I’ve given, with little glimpses into my background that I don’t talk about much. I really enjoyed giving it, and I hope those of you who watch like it, too (there’s a baby photo of me in there, as you can see in the photo below).

One of the coolest things about the process of putting the talk together was how the Etsy community helped me along the way by telling their stories in the Etsy forums. The whole process was really satisfying all-around, and I thank the community for their help.

Here’s a link to the video from Livestream (and the slides are here.)

Chad Dickerson, HELLO ETSY Berlin, September 17 & 18, 2011

How we manage development and operations at Etsy

I just found a new reason to love Quora: questions people ask can spur you to write about topics you’ve had kicking around in your head for some time. Such was the case with this one:

How does Etsy manage development and operations?
Etsy seems to have scaled far and fast, whilst continuing to add new features; how is all this managed – is there a strictly-defined process within which engineers operate, or is it a case of hiring clever people and letting them get on with it (Facebook-style)?

I posted my answer, and cross-posted it to the Etsy engineering blog, too.

“A Word on Statistics,” by Wislawa Szymborska

I signed in to Etsy a little while ago to do some holiday shopping and browsing, and I paged through the “items you might like” automated recommendation module we have on our home page now. One of the recommendations was a letterpress book containing the poem “A Word on Statistics,” by Wislawa Szymborska, made by Susan Angebranndt, owner of Green Chair Press.

A Word on Statistics

The title was intriguing, so I took a closer look. I had never heard of this poem or the poet, so I found it online and it is delightful.  I spend a lot of time these days exercising my left brain (the logical, analytical side) but I try to keep my right brain supple by continuing to read poetry and literature, too.  A poem like this brings both of those modes of thinking together quite beautifully.  Of course, I shared this serendipitous discovery (delivered to me using statistical methods!) to the 80+ folks in my Etsy Circle by favoriting it, which puts it in my activity feed on Etsy (we launched Circles and Activity Feeds at Etsy earlier this week).  This series of events was satisfying on so many levels!

Check out the Wikipedia page for Wislawa Szymborska, who won the 1996 Nobel Prize for Literature and also wrote “Pi,” which is featured in another beautiful letterpress book made by Susan at Green Chair Press. Here’s the poem:

A WORD ON STATISTICS

by Wislawa Szymborska
(translated from the Polish by Joanna Trzeciak)

Out of every hundred people,
those who always know better:
fifty-two.

Unsure of every step:
almost all the rest.

Ready to help,
if it doesn’t take long:
forty-nine.

Always good,
because they cannot be otherwise:
four — well, maybe five.

Able to admire without envy:
eighteen.

Led to error
by youth (which passes):
sixty, plus or minus.

Those not to be messed with:
four-and-forty.

Living in constant fear
of someone or something:
seventy-seven.

Capable of happiness:
twenty-some-odd at most.

Harmless alone,
turning savage in crowds:
more than half, for sure.

Cruel
when forced by circumstances:
it’s better not to know,
not even approximately.

Wise in hindsight:
not many more
than wise in foresight.

Getting nothing out of life except things:
thirty
(though I would like to be wrong).

Balled up in pain
and without a flashlight in the dark:
eighty-three, sooner or later.

Those who are just:
quite a few, thirty-five.

But if it takes effort to understand:
three.

Worthy of empathy:
ninety-nine.

Mortal:
one hundred out of one hundred –
a figure that has never varied yet.

Scaling startups

This blog post was also syndicated to Fast Company.

The Scaling Startups panel I was on last week at Supernova generated a little coverage, but I wanted to go into a lot more detail than what I saw there, so it seems like a good time to jump back into blogging full-force. The panel abstract read:

How can startups develop a distinctive culture, and sustain it as they grow? Are the characteristics that define today’s high potential technology startups the same as five or ten years ago? And does the current environment of low-cost infrastructure and open information sharing give new companies an advantage, or make it harder for them to maintain a competitive edge and reach critical mass?

Coming from the engineering side, I’m very much a pragmatist about all of this since most of my time is spent on shipping products and the systems to support shipping products. Before Etsy, I had done the “startup within the big company” thing and the Web 1.0 startup dot-com IPO thing, both of which have influenced my thinking on the subject considerably. In the panel, I focused mainly on what it takes to institutionalize a distinctive culture focused on shipping and how to sustain that when you’re growing fast. What you read below is based on the notes I made before the panel along with what I remember from the panel itself. supernova forum 2010: scaling startups I tend to take bad notes during panels because I’m focused on participating, so I don’t have much about the remarks of my co-panelists, who were great (Ethan Mollick of Wharton, our moderator, Steve Cohen of Morgan Lewis, Steve Barsh of PackLate, and Andreas Weigend, advisor to a number of companies and former Chief Scientist at Amazon). A video of the session will be available soon, though, and Jonny Goldstein did an incredible job drawing a visualization of the talk in real time! (see photo on the right)

At Etsy, we’ve quadrupled our engineering team in the past year, so everything I write below is based on that experience. Much of what I write about below was put in place in the last six months, and I think it’s working.

First, to scale a startup, you should always remind yourself why people work at startups in the first place:

  • they like to move fast
  • they want to make impactful decisions
  • they want to DO things and BUILD things (not sit in meetings awash in PowerPoint)
  • they want to be able to take risks

For various reasons, startups and the people who inhabit them as they grow tend to become more risk-averse over time. There’s a great quote from Peter Drucker about risks — laminate this quote, frame it at your desk, make it your screensaver, and put a copy of it in your wallet:

People who don’t take risks generally make about two big mistakes a year. People who do take risks generally make about two big mistakes a year.

In talking about culture earlier, I use the word “institutionalize” very deliberately. You have to actively develop a culture of speed and informed risk-taking and put real mechanisms in place to continually reinforce that culture, especially when you’re growing quickly. Pontificating about wanting to move fast isn’t enough and is often just counter-productive. You have to actually move fast.

You have to have the right amount of process, too, but not too much. In most companies, process can be a little like a trash heap that only grows larger and larger over time. You have to be vigilant about keeping that heap from growing. One of my favorite quotes on process comes from Clay Shirky:

Process is an embedded reaction to prior stupidity.

Process is sometimes put in place because one person did one stupid thing once, sometimes literally YEARS ago. What about the other 999 times that nothing went wrong and progress was made while ignoring the process? Leaders should question process at every turn (though tend to your “good” processes, which are important. The larger your team gets, the more you need some clear rules of engagement or folks just bump into each other.) I recall when Fred Brooks spoke at Etsy, he described some of his career experiences and punctuated one by saying (I paraphrase):

If you follow process religiously, you’ll never get anything done!

(I implicitly trust Fred Brooks.)

To scale as you grow, you have to put processes and structure in place that encourage everyone in your company to take more risks (not fewer), keep the right people informed of the risks you are taking, and protect individuals from silly mistakes that would tempt your company to institute unnecessary embedded processes. Just as many mistakes are committed from people being tentative than being aggressive anyway (the point of Peter Drucker’s quote).

So, how do you do it? Here’s what we do to institutionalize speed and risk-taking at Etsy (your mileage may vary):

Hire well

This goes without saying, and I didn’t mention it in the panel. It’s a big topic probably best left for another post. Hiring great people makes everything else below easier.

Communication

Everyone in the company uses IRC, not just engineers. Everyone, all the time, from the CEO on down. Sure, sometimes you can miss things if you’re not in IRC at the time, but the benefits far outweigh the costs, and you have a lot fewer meetings about day-to-day mundane issues.

Deploy code early and continuously

At Etsy, most new engineers deploy code before they are done with their health insurance paperwork. We practice roughly what has become known as “continuous deployment.” This means one-button deploy, period. We’ve invested in the tooling to make this possible, so it doesn’t come for free. Engineers don’t have to wait to ride on a 2-week release: you can do it now. This is all supported by thousands of real-time monitoring checks, business metrics, and automated tests, so it actually requires MORE discipline and ongoing attention than many shops who have multi-week (or month – yikes!) release cycles. A lot of people say, “that sounds risky!” Well, not really. The team at Etsy once reported a serious XSS bug to a site (which will remain nameless) and they said they were going to deploy a fix — in six weeks, because they only deploy every two months. Talk about risky! We did 204 production deployments in July, about 30% more than June, and we are ready to deploy at all times. Deploy, deploy, deploy. (Doing this requires close dev/ops cooperation. The canonical presentation on that subject is 10+ Deploys Per Day: Dev and Ops Cooperation at Flickr by John Allspaw and Paul Hammond. John has been at Etsy since January.)

Oh, and remember IRC? We have IRC bots that automatically announce code deploys in real-time. Those messages look like this:

PRODUCTION deployed by cdickerson build: 32445-trunk-20100804-033748-UTC took: 17 seconds diff: http://awesomeness.internal.etsy.com/diff/web/32441/32445

Since everyone is in IRC, everyone knows who deployed, when we deployed, and what is being deployed (this also goes out in email to the whole engineering team). This level of transparency is what distinguishes how we operate from “cowboy coding,” which often works in early-stage startups but becomes destructive and distracting as a team grows.

Encourage experimentation

With minimal review (“minimal” meaning making sure key privacy, legal, and other issues are taken into consideration), any engineer at Etsy can push a test application out to our members to get feedback, and it could totally fail. We announced this to the community with all appropriate caveats. Allowing experiments eliminates the “no one will let me do my idea” problem that starts to break down startup culture. This is all about reducing the “idea approval index.” We’re just getting started on this one and I’m really excited about it.

Hack Days

One of our teams did a hack day last week where the engineers on the team went off-site and defined and built a single feature and released it on the same day. Always remember: “Every good work of software starts by scratching a developer’s personal itch.” It almost doesn’t matter what the feature is: it’s the cultural idea that you can JUST DO IT that matters most. The Hack Days we’re doing at Etsy now have one key distinction from the ones I organized at Yahoo (though they share the same inspirations) — you build and ship the same day. Our continual investment in tools and culture makes this possible.

Build a culture that loves engineers

I see folks mess this up all the time, and unfortunately it is a difficult thing to teach since it’s so intuitive. If you’re building a technology-driven company, you better have a culture that loves engineers, and I mean love. All too often, I see entrepreneurs who say they “just need engineers” to “bang out the code” for this great idea of theirs. If you view engineers as interchangeable factory workers instead of partners and creative people, you’re in for a tough time getting huge in a world driven by technology. To scale, you need to tend to engineering culture in your company and build a company where engineers love to work. The headlines about tech companies are always about the comings and goings of CEOs, but the rise and fall of tech companies is usually correlated to where the great engineers are going (of course, great CEOs know this and draw great engineers). I tried to express our view of engineering at Etsy on the about page of our engineering blog, Code as Craft (more on that below). Engineering is creative work, and often joyous work at that. No one describes the inherent joy of building software more eloquently than Fred Brooks when he answers the question “why is programming fun?” Recognizing and encouraging that sense of joy is core to any great engineering culture.

External transparency

We talk about all of this stuff in our engineering blog, Code as Craft, which makes our interview and hiring process more productive since folks come in knowing how we think and operate, and even some of the people on the team. We also get help and advice from the community with particular challenges like batch resizing 135 million photos.

Embracing failure

Moving fast often means making mistakes, though still not as many as you would think (again, the Peter Drucker quote). In the panel, I mentioned the award that Flickr gives annually “to the individual who breaks flickr.com in the most spectacular way.” Of course, this is a little tongue-in-cheek, but the point about accepting mistakes is not. Spectacular “failure” means you were probably working on something big and had a spectacular learning experience. (Side note: my remarks on the panel were construed to mean that we give such an award at Etsy — as reported at CNET — and I don’t think I actually said that; however, I immediately emailed the staff back at Etsy to start brainstorming a name for such an award at Etsy. Stay tuned.) “Rewarding” failure is not some crazy “those dot-com kids!” kind of thing. Companies like pharmaceutical giant Merck have institutionalized practices to deal with failure:

[R&D head Peter S.] Kim is promising stock options to scientists who bail out on losing projects. It’s not the loss per se that’s being rewarded but the decision to accept failure and move on.

Of course, catastrophic mistakes are not to be celebrated, but those sorts of mistakes are exceedingly rare and you should have processes in place to deal with them (including a defined post-mortem process). Most of the tension in companies around mistakes is about fairly mundane issues and is relatively unproductive (again, look at the Drucker quote you have in your wallet).

In the Q&A for the panel, someone asked: “This sounds great for engineering and product — how do you scale everything else at a startup?” While I focused primarily on engineering because it’s what I know, I have a huge appreciation for well-run HR, finance, and other operational functions inside a company because they are critical. They can often be the unsung heroes. You can’t add server capacity quickly if the finance team doesn’t help you move quickly.

To illustrate the point, I asked the crowd how many of them knew who Peter Grant was (only one person did). Peter Grant was Led Zeppelin’s manager and was famously quoted as saying: “Led Zeppelin looks after the music and I do everything else.” As you grow, make sure your startup has Peter Grants to keep the band focused on creating awesome music.

We are hiring at Etsy – come join us!

I’m hiring a few key positions on the engineering team at Etsy. See the links below for more details, but before you jump to read them, I wanted to say a little more about the type of person I’m looking for. Of course, I want people who write great code and get things done, but more than anything else, I want to hire people who care about the buyers and sellers that make Etsy what it is.

These are jobs that matter. Each of these positions is a dream job (I know mine is!) You will be building new features for and maintaining a marketplace that creative people around the world depend on to make a living. You will be helping enable and surface beautiful stories like the stories of Robin and Kathy Tucker of woodmosaics. If you read their profile, you’ll learn about how Robin developed his style and approach to woodworking, and what tools he uses, including the foot-powered table saw he made with his brother. The profile ends with this:

I know of no better way to make wood stand out and be truly beautiful.

For a lot of people, this is what dreams are made of, they dream of buying it and I dream of selling it.

Dreams do come true!

How many places can you work where your community says you make dreams come true?

Despite some of the gloom-and-doom in the larger economy, Etsy continues to grow. When I’m talking to people about Etsy as a business, I always ask, “Want to guess what the biggest three days in the history of Etsy were?” Usually, the person thinks for a bit and guesses “the day after Thanksgiving last year” and a few other random days. If I ever ask you that question, here’s a hint. Lately, the answer is nearly always the same: the three biggest days in history at Etsy were yesterday, the day before that, and the day before that. You can read some of the numbers yourself in the monthly updates on our blog. The growth of Etsy is a testament to the continued passion and dedication of the community that you would be working hard to support.

Here are the positions we’re hiring (all are based in Brooklyn):

  1. ArchitectDirector of Engineering
  2. UI Engineering Lead (hired)

If any of these sound like a dream role for you, and you find Etsy as inspiring as I do (just do a search for “love etsy“), please email me with your resume and tell me why you’re the perfect person to join the engineering team at Etsy (chad -at- etsy dot com).

A quick report from the trenches at Etsy

“Busy” does not adequately describe my first two weeks living in Brooklyn and working at Etsy.

To give you a glimpse into what I’ve been doing and dealing with, check out my first post on Etsy’s blog, the Storque and the resulting forum thread on Etsy’s forums.

I’ve had to jump in head first and there are plenty of challenges, but I’m loving it. Expect light posting here while I focus on the work at hand.