Amazon.com (AMZN) spent roughly $7.3 billion on capital investments in the last five years—more than 10 times as much last year as in 2008. Much of that money went into its Amazon Web Services division’s server farms, which generate the processing powerAmazon sells to heavy corporate data users on the cheap. The cloud computing division grew so large that the company couldn’t rent out all its spare server capacity. So the e-commerce giant created a supply-and-demand-driven market called Amazon EC2 Spot Instances that allows clients to rent processors for as little as 10 percent of the company’s standard cloud services fees.
To use Spot Instances, companies bid for the rights to a certain number of servers. Winning bidders are billed by the hour, as long as the market price hasn’t risen above an upper bound they specify. The marketplace gives Seattle-based Amazon and its seven-year-old cloud computing division an added advantage over Microsoft (MSFT) and Google (GOOG), which are racing to catch up after a late start. Macquarie Securities analysts estimated in a Jan. 7 report that total Amazon Web Services revenue will almost double this year to $3.8 billion and reach $8.8 billion by 2015. Macquarie estimates Web Services will kick in close to 5 percent of sales this year, up from 3.4 percent last year, and will rise to nearly 8 percent by 2015.
Spot Instances launched in 2009, but the spot market has taken off only in the past year, says Amazon EC2 Vice President Matt Garman. Researchers in genomics and drug design as well as online advertising increasingly use Spot to analyze terabytes of data, while such companies as Cycle Computing, Princeton Consultants, and Numerate have built businesses that track market prices for heavy data users. “We’re finally starting to see the real change in the slope of adoption,” Garman says. While he declined to provide specifics on how profitable the product is or how many customers use it, the company says much of its increased spending on “technology and content” went to add capacity for Amazon Web Services.
Princeton Consultants, a strategy and software firm, is among the companies that developed algorithms to monitor Spot’s pricing and availability for clients in need of server time. Chief Executive Officer Steve Sashihara says his program, OptiSpotter, provides small hedge funds with the edge they need to compete with investment banks and global funds in the cutthroat world of high-frequency trading. “It’s completely game-changing” for clients to cut a project’s processing costs from $100,000 to $10,000 and get supercomputer-like processing power, Sashihara says. “A couple guys with $50 million can be trading at high frequency and be taking on Goldman Sachs (GS) and Morgan Stanley (MS).”
One big catch limits the utility of Spot Instances: If rising demand sends prices skyward, a customer must outbid the higher price to keep its project running. Otherwise, Amazon shuts their servers off in midstream. That’s a deal breaker for clients using it to host websites or stream video. “For people running with hard deadlines, it’s not an appropriate solution,” says Michael Crandell, CEO of RightScale, which helps companies manage cloud services.
Still, Crandell says he expects Spot to keep growing. Up to 15 percent of Amazon servers his customers rent are now from Spot, he says, up from the low single digits in mid-2011.
Julie Black joined Web advertising startup TellApart in 2011 as director of engineering after six years at Google. She uses Spot Instances regularly, because her former employer’s recently launched cloud platform, Google Compute Engine, doesn’t offer a comparably cheap market. Burlingame (Calif.)-based TellApart analyzes online shoppers’ habits for retailers such as Nordstrom (JWN), One Kings Lane, and CafePress (PRSS).
To provide better results than its dozens of competitors, TellApart uses Spot Instances to collate more than 10,000 queries per second from hundreds of millions of people, with the bulk of the work occurring during off-peak hours when prices are lower. Black says the company, backed by almost $18 million from venture investors including Greylock Partners and Bain Capital Ventures, rents hundreds of machines at a time. Its monthly check to Amazon is about 75 percent lower than it would be for cloud servers at the on-demand price. “We process data faster and cheaper because we can use more machines,” she says. “Everything we do relies upon this core piece of technology.”
