About 15 years ago, Chuck Rossi became what’s known in Silicon Valley as a release engineer. This is the person tasked with gathering up all the code written by a company’s many engineers and making sure it works together as a whole. Rossi coordinates the process for looking over code for bugs, chatting with engineers about their work and deciding which new features are ready to get baked into a particular version of a product.
It should be noted that Rossi is not just a release engineer. He is THE release engineer. He’s done this job at VMware (VMW), Google (GOOG), and now Facebook (FB). He may well know every single person in the closely knit world of Silicon Valley release engineers. As noted in my cover story this week, the work of Rossi and his team stand as one of the major reasons Facebook has been able to handle a billion users.
So how does one become a release engineer? Well, in Rossi’s case, it was a conscious decision. “It’s like plumbing,” he says. “It’s not the most glamorous thing in the world, but I realized that if you’re good at it, you could go to any software company in the world and they would say: ‘When can you start?’”
Rossi’s main job is to oversee the Push, a daily exercise in which Facebook takes in hundreds of changes to its code from engineers, checks to make sure they’re good, and then adds them to Facebook.com. Over the years, Facebook has built a number of software tools that do the first round of checks on the code, leaving Rossi to manually inspect the additions with potential to cause the most problems.
Ritual surrounds the entire process. Rossi, for example, has a bar to the left of his desk—a serious, full-on bar packed with scotch and tequila and whatever else you may want. The bottles are bribes from engineers trying to persuade Rossi to incorporate their changes into the Push.
There’s also what’s known as Push Kharma. This stems from profile pages tied to each engineer. Rossi can pull up someone’s name and see what code they have submitted. “Every developer is born with four stars to their name,” he says, pointing to a ranking system on the profile page. “If we have an issue when we take someone’s code—and it blows us out of the water—then it takes them down half a star and I write what happened,” Rossi says. The system has a thumbs-down indicator, a feature many Facebook users have long sought. “I am the only guy who has a’ dislike’ button on Facebook,” he says. “A lot of people want [one], but this is the only place you will see it.”
When engineers drop down to two stars, they’re banned from making changes until they’ve completed a review and a retraining process. “People here are pretty freaked-out about losing their stars, but not in a bad way. It’s all done in good fun,” Rossi says. And if you catch an error before it goes up on the site and jump in to save the day, you can earn a half-star back. Failing that, he jokes, “you can bring booze or cupcakes.”
