That’s a paradox that can take entrepreneurs by surprise. Maybe it’s true about everything? The internet simply makes it happen faster. A web server that was happily serving 100 people today, probably isn’t going to happily serve 100,000 tomorrow. At the very moment the world has finally discovered your solution to life-the-universe-and-everything, that solution dies in a heap of technical errors, and the world goes away again, suitably unimpressed.

This article describes my personal experiences of scaling a website up to serve tens of thousands users each day, most recently by hosting Gnomish literature in a “Cloud” environment.

Growth

The graph below shows total monthly views of pages with content, for El’s Anglin’ and Professions domains. The period is the 12 months from June 2008 to the end of May 2009. The vertical scales for each domain are slightly different: El’s Anglin’ almost reaches 3 million monthly page views, while El’s Professions peaks at 1.3 million. Pageviews are shown here because pageviews tend to cause scaling problems, not specifically visitors.

Monthly Page Views

Anglin’s growth looks fairly modest, but is still quite significant: From 0.9 million page views in September to 2.5 million in December – almost a 3-fold increase in 4 months. El’s Professions saw 4-fold growth from September to October, which then quickly declined again. Both patterns reflect changing demand for content:

• The steady rise in pageviews for El’s Anglin’ towards the end of 2008 is the rise in interest in fishing with the latest World of Warcraft Expansion. As seen in the previous expansion, interest in fishing lags slightly behind other parts of the game.
• The final peak in April 2009 was a new game patch, which added a lot of fishing content.
• The pre-expansion game patch in October 2008 introduced Inscription – the main subject of El’s Professions. In the first week interest in the subject was intense, but very little has since changed.

However, monthly variations only hint at the real hour-by-hour chaos within.

Peaks

The server resource graph below is a good example of the type of short-term peaking in activity that World of Warcraft players can cause. The graph is for Saturday 12 December 2009. The bottom (x-axis) is Pacific Standard Time (hours). The 2 y-axis show average processor load (how much work the server’s CPU is doing) and memory used (how much physical memory is being used by the server), respectively.

Kalu’ak Peaking

The absolute values aren’t important. Instead, look at the variations: Between 02:00 to 05:00, server activity increases by a factor of at least 5. Then after a gap of a few hours a similar pattern occurs again, before finally settling down in the evening.

What’s happening? 12th December was the first time the Kalu’ak Fishing Derby had been held. The event occurred at 14:00 local time. The 05:00 peak is due to European players. The later peaks occur as the contest moves across the 4 main United States timezones.

In this case I was fortunate: Different parts of the world peaked at different times. But you can hopefully visualise what would happen if everyone arrived at about the same time. October 2008 and April 2009 both saw just that. It was messy.

For the website operator this poses some problems: How much slack do you pay for to ensure you can handle the peaks – 5, 10, 20 times as much as normal? Can you even fund that much redundancy? And if there is a peak, you’re probably already going to be busier, with even less spare time to panic about technical problems: For example the peak is caused by new content, which is (normally) still being researched or checked.

Survival of the Fittest

Blaine Cook‘s instruction, to “Cache the Hell out of Everything”, is good starting advice. Here caching means creating a static copy of a page, which can be served to most readers without the webserver having to process any information. When first writing a website it’s normally much easier to hold all the information in one database, and use templates to format that data as users request it. But if (as for El), pages rarely change, this becomes terribly inefficient: During a peak, several pages may be requested each second. The webserver is probably processing precisely the same thing several times each minute, wasting computing power just when it is under greatest pressure.

Another discovery was the use of far fewer supporting files. Scripts, styling, images. For example, El’s Inscription displayed separate small icon images for almost every item on the page. It was intended to help the reader quickly recognize items. These files were tiny in size. But there were a lot of them. So the webserver was actually serving tens of files each second.

Designing with the expectation of high numbers of users helps to keep the peaks manageable. But ultimately a very popular website will exhaust the technical resources of a single webserver, however well-designed the content is.

Clouds

I struggled to define Cloud computing last March, and I’m still struggling. I fear it will become a new “Web 2.0″ – a phrase that’s so cool to use, it ceases to have meaning. The basic concept involves using the pooled resources of many computers. In practice, Cloud hosting is then defined by 2 criteria:

1. Is the cloud all in the same place (data centre) or are the Cloud’s server spread across the world? In the first case, Cloud hosting will only be as reliable as the data centre the computers live in. If the link to that location breaks, or that data centre loses power, it’s game over. Single-location clouds might be called a “very large Virtual Private Server“. Generally hosting in one place is technically easier, meaning fewer limitations, but obviously results in less redundancy.
2. Are you simply buying computing/storage power (which you then administer yourself) or are you buying webspace on a hosting platform (fully maintained by the hosting company)? A lot of commercial Cloud hosting (such as Amazon Web Services) essentially involved buying computing/storage power, and then largely managing it yourself. Great for very large websites, but not so great for me: I need the flexibility to scale from 1 server to erm, maybe 2, and I don’t want the hassle of actively managing anything.

Within “platform clouds”, it seems there is a trade-off between redundancy and software/programming flexibility. For example:

Google’s App Engine scales up from the smallest website, and is hosted in more than one place. Great. But the platform is optimized for code written in specific languages (currently Python and Java), with significant limitations on database queries (notably no joins between tables within queries – no relational databases in the conventional sense). So while it is probably perfect for a new programmer’s project, an existing website would need to be heavily re-written.

Rackspace’s Cloud Sites are optimised for more traditional website programming (PHP/mySQL, ASP/MSSQL), but everything is hosted in one place – slightly more flexibility for slightly less redundancy. Personally, although I use Python on El’s sites (primarily to build static pages), for historic reasons all the dynamic content is written in PHP. In the final analysis, I was looking for a PHP-based platform with just enough Python support to let me run scripts in the background.

Cloud Sites

Moving from a conventional Virtual Private Server to Rackspace Cloud Sites still caused a few problems.

On a conventional web host, you and your code would have access to a command line – a text prompt from which to run programs. Cloud Sites have no command line. This isn’t just a problem for you. Scripts running on the webserver that try and execute a system command tend to fail. For example, PHP’s exec() seems all but worthless. Fortunately Python (and Perl) scripts can be run from the public side of the website. They have to be placed in a specific directory (cgi-bin – just like in the 1990s). I am not sure how efficiently such code is run, so this might not be suitable for every use. But it allowed me to build a small private administration area, loaded with all sorts of useful scripts.

There are many little limitations – things that simply require a different method to be adopted. For example, Symbolic links don’t appear to be supported. These would normally allow a directory to virtually mirror another directory. I host several sets (different sizes and coloring) of every World of Warcraft icon image. About 25,000 files in total. Most will never be used, but it is easier for me to upload them all, than work out what I need. I used to host one copy for El’s Anglin’, and symlink to El’s Professions: The same image became available from either domain. Now all the Professions icons directly reference the Anglin’ site in their URL.

Ironically, the biggest single annoyance is the painfully slow-to-load Rackspace Cloud administration interface. Fortunately once everything is set up, one rarely needs to use it.

Bitter experience has taught me that there is no such thing as a reliable web host. Although generally the more you pay, the more reliable they get. It is too early to judge performance fairly, but the last 2 weeks of December have been less than stellar:

• A complete outage for about 40 minutes. Fortunately, Rackspace’s Cloud hosts many high-profile websites, so outages tend to get noticed (18th).
• Several hours of “degraded performance” – meaning, “your site is so slow most users will give up before anything loads” (27th).
• And the worst user error message I’ve seen for some time:

“Unfortunately there were no suitable nodes available to serve this request.” Say what? This isn’t just user-unfriendly techno-babble. The system seems to ignore my custom files for handling internal server errors, which would give a slightly more friendly message.

The main advantage of the current hosting is that I no longer waste time worrying about whether the allocation of resources is sufficient. That became a significant distraction on the previous Virtual Private Server. Hopefully the technical glitches will get smoothed out as Cloud web-hosting matures.

Costs

Charges are based on a fixed monthly fee, with additional charges for disk space (site storage), bandwidth (size of pages/files transferred), and “computing cycles” (a measure of processor usage) above a basic quota. Based on the first 2 weeks, I estimate that the computing cycles quota will sustain 2-3 million page views per month (including all the related images, styles and scripts), and I won’t run out of bandwidth until at least 5 million page views per month.

Almost every page contains advertising. I currently earn more per advert than it will cost to serve the additional pages. That can’t simply be assumed on a video-games website. We attract the cheapest advertising. And when there is more advertising inventory than advertisers to fill that space, we’re the ones that run empty ad’ spaces. So in a depressed economy, we suffer more than most.

Role

Last year I argued that cloud computing could allow hyper-virality (near-instantaneous adoption of a service or product), and I think that’s still where we’re heading. What’s changing now is that cloud computing is becoming an option for more modest websites, run by people outside the “tech bubble”.

There are trade-offs and limitations. But a lot of individuals and smaller businesses aren’t doing anything more complex than running standard Content Management or ‘blog software. Cloud-based hosting could give them the reassurance that the moment the world suddenly wants to discover them, their website won’t die in a heap of embarrassing technical errors.

• Cloud Computing
• Benefits
• Hyper-Virality
• Funding and Revenue Models
• Exploding Intangibles
• People
• The Caveat

Cloud Computing

Cloud computing is poorly defined. Offering computer resources as a service. Across a network. Probably the internet. I prefer Frank Gens’ description – a “grid-utility model”, that anyone can use. Grid computing is the use of lots of individual computers as one. Utility computing is paying for computing resources as you use them. So instead of investing in lots of private computing capacity, one simply buys computing resources from an internet-based cloud of (effectively infinite) computing resources, as they are needed.

Benefits

What’s the advantage?

Most private computing resources are woefully under-utilised. A typical buisness’s computers lie idle while employees are sleeping, and are probably only partly used when they are working. Most websites/services are hopelessly over-resourced to accomodate occasional peaks in traffic. And then promptly collapse when the peaks transpire to be bigger than anyone expected. Sharing huge clouds of computing resources between many users – each with their own peaks and troughs – logically allows the total current demand for computing resources to be met with far fewer computers. Utility charging provides the mechanism that translates “fewer computers” into “lower cost”.

But there’s far more to cloud computing than “making existing things more efficient”.

Hyper-Virality

I define hyper-virality as the near-instantaneous adoption of a service or product by consumers, distributed through a pyramid of social connections.

It is the theoretical end-point of many current social internet trends: That ideas, optimisations, and methods that enhance the efficiency or well-being of humanity, move around the world ever faster than before. The actions of a handful of dominant, successful, or somehow innovative “thought leaders” are copied by those they lead, who are in turn copied those they lead. Since all the people on the planet are connected to each other by a surprisingly small number of links (you know someone, who knows someone, who knows someone, who… not many more times to link to everyone), distributing something down to absolutely everyone is potentially easy.

In an extreme case, a service that really is, the greatest thing since sliced bread, could almost instantly be benefiting billions of people. Clearly this level of hyper-virality is theoretical. The whole notion of absolute hyper-virality may itself be nonsense: That any one idea can be so clearly the best thing to do, that almost everyone is immediately prepared to adopt it. But in a limited way, it is already happening:

The best current examples of the concept are probably Facebook applications. These are still far from hyper-viral, but do demonstrate basic behaviour: When a user starts using a service, that user automatically informs their “friends”. If the service is desirable, those friends start using it, and in turn, inform all their friends. It is already possible to grow from no users to millions of users in a matter of weeks.

With traditional computing services, it is very difficult to scale computing resources up fast enough to keep pace with that growth in demand. Even if you can find enough machines, you probably cannot organise and configure them. You would need to anticipate millions of users and plan sufficient computing resources beforehand. And probably anticipate the wrong number – either be left with a lot of unused, but costly resources, or be unable to meet demand.

Cloud computing solves this problem, because precisely the right amount of computer resources are available, with costs always in proportion to the amount of resources used.

(Well, actually it is not yet a complete panacea – some caveats are discussed later.)

In concept, this “agile deployment” of internet-based services to the cloud, enables true hyper-virality to occur, since it creates the ability to deliver the ultimate product or service to everyone, almost instantly. This becomes far more important if we assume that our economies are becoming highly intangible entities, which primarily function over communications/internet-type services (see a Socio-Economic Environment based on Nothing) – most economic activity will occur in this communications/internet-type environment. But even if we don’t accept that assumption, the cloud potentially changes how we finance internet-based services.

Funding and Revenue Models

In a hyper-viral environment conventional business plans need to be rethought. A wild guess at future usage won’t do: “Somewhere between 3 and 3 billion users, with growth occurring sometime between today and the next decade”. That’s already a real dilemma for anyone developing a mass-market internet-type application.

Instead one needs to answer the question: How do I rapidly scale to meet demand? At the very least covering marginal costs (operating costs).

Broadly, there are 2 solutions:

1. A revenue model that earns money in proportion to usage, with ultra-efficient cash-flow, that allows increasing revenue to immediately pay for increasing operating costs.
2. An ultra-efficient capital market, that will invest in proportion to growth in usage, pending the later establishment of revenue streams.

Both approaches differ from what we tend to do now: Convince someone to invest some money, and hope we’ve established enough growth/revenue/”something good” by the time we’ve burnt through the cash.

Some fairly basic revenue models fit the first method well, such as display advertising or subscription-only services. In others models revenue may lag behind growth slightly: Premium versions of services that don’t earn money until users have enjoyed a free trial, or methods that rely on first building user engagement or trust, such as micro-transactions.

There is a risk of bankruptcy due to slow cash-flow: If your advertising revenue is still written on a cheque, lost for weeks in the postal or banking systems, what do you pay the cloud computing bill with? Fortunately, both incoming revenue and outgoing costs tend to lag behind by about the same amount. Cloud computing tends to be billed after use, not before, so everything is delayed by the same arcane business payment practices. But there’s still an element of Russian Roulette to whether the credit or debit clears the system first.

Parts of the financial system have the potential to be ultra-efficient. For example, in minutes, stock markets can adjust to changes in relative demands for capital. Except that such markets are dealing in established businesses that investors have some understanding of. Even high-risk investors, such as venture capital or private equity, may be unable to respond fast enough to new hyper-viral investment opportunities.

Note that established organisations will face slightly different problems to new entrants. For example, a historic focus on capital expenditure and fixed budgets may require structural change to occur within the organisation.

Astute readers will note that I have only addressed funding after a product or service’s launch. Cloud computing and virality won’t make it any cheaper to actually develop that product/service to the point where it is ready to be used. I hope to return to the topic of iterative product development on the internet in a later article – I contend that less time should be spent trying to develop a “finished” product before launch, and instead design should gradually iterate improvements on a live, “permanent beta” product or service. A logical extension of agile development into the iterative specification of the product itself.

Exploding Intangibles

Corporate finance still seems to be biased towards physical assets. This is great for factories: You can add up the value of all the machinery, materials and stock – all the physical stuff – and discount it all year-to-year. But if the most expensive physical asset is the coffee machine, classic accountancy tends to struggle. Highly intangible companies are not new – plenty of businesses are built up around intangible assets, like brands or franchises. And methods have evolved to account for the value of these companies, even if the process is inherently less precise.

What’s not clear is whether a rapidly expanding, but totally intangible, business can gain access to any conventional forms of finance. Our hyper-viral, cloud-based internet business didn’t even exist last month, and (because of the hyper-virality of unknown potential competitors) might not exist next month. So what is additional investment secured against? The coffee machine? While it is easy to point towards the rise (and fall) of major technology startups that have grown to be stock market-listed within a few years, the ultimate hyper-viral business would want to make that leap almost instantly.

Can finance move as fast as hyper-viral businesses? Current practices suggest not. But that misunderstands the potential advantage in fast-paced hyper-virality: Everything can move faster, including the return on capital. Invest for a few months, instead of a few years. So perhaps the challenge shifts to creating “stuff” that better informs capital markets, which can then react appropriately to other sectors?

People

Does this work organisationally?

Is it naive to assume that one individual can ever develop a solution that can gain mass-appeal, without significant help? Just as at the start of the first internet bubble, many believed that “2 kids in a garage” were going to destroy established businesses. While small developers do sometimes come to dominate emerging markets, most historic business empires remain in tact. And if that help takes time to organise, won’t that human organisational factor limit the speed of virality, giving existing organisations significant advantages over lone individuals?

Perhaps there are just to many unanswered questions? Technology and capital is easy – people are far more complicated!

Curiously, if people both define the structure through which hyper-virality occurs, and are the reason why hyper-virality isn’t quite as fast as it might be, it will be people that naturally establish a happy equilibrium. It would be limited by people, not technology or capital.

The Caveat

A small caveat: Cloud computing cannot yet deliver all its promised potential.

For example, by consensus the cloud does not handle database-driven applications well, particularly where a lot of data is being written (presumably we need to radically re-think the way databases work). Techniques for scaling resources within the cloud, to meet rapidly changing computing demands, are still evolving. It is not yet as automated as it sounds. There are a plethora of concerns related to issues such as security (by consensus, more perception than actual problem), and standards (some commentators suggest inter-operability will be resolved through clever use of APIs, much as different database software were made to “talk” to each other).

Ownership and privacy is a potentially important, but poorly explored issue: Logically a handful of huge cloud computing providers could gain considerable control and influence over the creativity of individual developers (much as early-modern “publishers” historically came to dominate people that worked as printers).