The graph’s y-axis is the percentage decline in page views at El’s Extreme Anglin’ from the 7 days before each event, to the first 7 days of the event. Pageviews are a good proxy for overall angler interest. El generates hundreds of thousands of page views each week, so even small changes are significant. The x-axis orders events by date, from January 2008. The axis isn’t scaled correctly to show time, but holidays are fairly evenly distributed throughout the year. Events are shown by green dots, with a shortened date (month and year) and the name of the event.

The data is expressed as a percentage of the previous week, because while interest in fishing “waxes and wains” from year-to-year, changes week-to-week are normally minor.

All the events included last at least 7 days. Where one holiday runs concurrently with another event (for example, the “Lunar Festival” and “Love is in the Air” often clash), only the first event in the sequence is included. Interest in fishing also changes dramatically in the month new content is added, so events that clash with major fishing patches have been excluded (Noblegarden 2008 with patch 2.4, Hallow’s End 2008 with patch 3.0.2, and Noblegarden/Children’s Week 2009 with patch 3.1). Winter Veil is also excluded: The period leading to Christmas is particularly unusual – first students stop studying and have a lot of time to play, and then many players stop playing to spend time with family. This causes large changes in activity from week-to-week, which makes it hard to isolate Winter Veil in the data.

Only 12 separate sets of data can be compared. There is one out-lier – Midsummer 2008 – perhaps the early stages of Wrath of the Lich King testing may have caused a small traffic spike in the week before? The pattern shown on the graph is not certain. But I’m growing confident that events are increasingly impacting on fishing activity.

But why?

Explaining the Trend

Fishing is a solo activity that doesn’t change from week to week. So if a player becomes busy because they are involved in special holiday activities, fishing is likely to be one of the easiest thing not to do: It can “wait till next week”. It does not break any group commitments, such as raiding dungeons as a party of players.

Yet that doesn’t explain why holiday events appear to be having an ever-greater impact.

I expected to find a dramatic shift following the introduction of achievements, towards the end of 2008. These added new incentives for players to complete holiday activities. Yet the trend continues for 2 years, achievements or not.

Holiday events are sometimes improved, but the pace of improvements is very slow. They could not explain the trend because the majority of holidays have not been changed in the last 2 years. Similarly, fishing has undergone changes over the last 2 years, especially around the middle of the period shown on the graph. But none of those changes explain the event-decline trend seen.

A possibility is that players are simply becoming more focused on whatever “is new”. That pattern is apparent in “the Warcraft media” (primarily news websites), which increasingly obsess over the slightest change, often months before the change is playable. Part of a collective “addiction to previewing the future“.

Perhaps holidays are becoming more popular because a broader range of players are finding them appealing?

Appeal of Holidays

Holiday activities tend to be fairly easy to complete. There are some exceptions, but generally these events are accessible to a broad range of players, which is not true for all WoW’s activities. It would be easy to conclude that these events have become more popular over time, because of a general “dumbing down” in the average level of player skill.

Perhaps the average amount of time players play World of Warcraft is declining? Players would now be busier people than players in past, so to complete event activities they have to commit a higher proportion of their playtime, so are more likely to not go fishing.

Holidays tend to fairly social events. From sitting around a Romantic Picnic Basket, to sharing food, or placing “bunny-ears” on passing gnomes (to make them look even cuter). So perhaps the trend reflects greater social activity within the game?

The themes of holiday events mirror physical world festivals. From the Chinese New Year (“Lunar Festival”) through to Christmas (“Winter Veil”). So perhaps the trend reflects a greater desire for a connection back to the physical world? Diluting immersion in the virtual.

Partly it isn’t because creators haven’t adjusted their methods to match the new technology – we’re still wedded to a single start-to-finish process, with one outcome at the end. Partly it isn’t because feedback can be hard to gather and digest, and even hard to act upon.

An iterative method has become one of the defining characteristics of how I like to write, organise, and present text on the internet. At least, beyond this domain. But until now, I’ve struggled to apply it to internet-based video.

This article introduces internet-based iterative design, and uses YouTube’s “Hot Spot” analysis to show how we can start to apply an iterative approach to video and movie-making.

Iterative Product Development

The author of a published paper book generally gets one shot. One chance to have their works committed to paper. To be read by milllions. Or tens. A huge amount of effort goes into “getting it right”: Construction of text and story, editing, proof-checking. And in spite of this, book publishing remains a high-risk activity: For every top-selling author, there are others whose work ends up as pulp.

In contrast, the cost of making corrections or changes on the internet can be minor. At the extreme, the author simply types some new words. An update that might have taken a book publisher months or years, can be committed in minutes.

The ability to make rapid changes in response to rapidly gathered feedback makes the internet interesting: The most basic server access logs can be analysed to reveal that chapter 2 is generated much more interest than chapter 1, yet chapter 4 is hardly getting read. With enough readers, those patterns can be seen in days, or even hours. So perhaps the content in chapter 2 should be expanded, and we should re-write chapter 3 to better maintain interest?

Now we’ve expanded chapter 2, and noticed it has become even more popular. Obviously there’s a greater demand for the writing or information in chapter 2 than the author thought there was. Gradually the content evolves and gravitates towards (in the language of entrepreneurs) the nearest unserved market. Iterative product development isn’t just about “making it better”. It’s a way of finding an audience, customers, earnings.

The written word is an easy example to understand, but maybe all good design iterates in response to user feedback?

Probably always has. Stone wheels? Computers sold as recipe books? Especially where a cool technology is looking for a problem: An inventor that doesn’t start by trying to address a problem, but merely discovers a method of doing something. Post-It notes are a popular example, but this pattern is common from the Victorian era onward. For example, it took half a century of different people trying to use pressurized air for land-based transport propulsion, before a market niche was established.

The internet allows this process to happen a lot faster, but only if the presence of the internet becomes integral to the design process.

Personally, this methodology has turned a few highly technical pages on the mechanics of fishing, into a fishing guide read by millions (see Appendix: Iterative Writing in Practice at the bottom).

Limitations

Iterative product development isn’t a panacea. Or a free ride to perfection and untold riches:

• While internet-based products and services are comparatively (to manufactured goods) cheap and easy to change, constantly making changes can become very time-consuming. Designing with the expectation of change, helps. But ultimately you will reach a point where further changes don’t generate enough extra audience interest to (financially) justify the time spent making the changes. This is when to stop.
• There is no guarantee that your product will find the absolute biggest unserved market, merely the biggest such market near to the topic/interest area you started with. If you started developing an idea in an obscure niche, it’s not realistic to expect to grow outside of that niche.
• Iterative development is not an excuse to design garbage. Quality remains important: The first attempt has to be sufficiently “good” for enough people to use/read it to generate feedback.
• The process of analysing feedback, and developing new content, requires 2 distinct skill-sets. Someone that is good at the second, may be unable to do the first. An instinctively good designer might still produce a better product, although (I would argue) their method leaves more to chance.
• It’s not clear that this method could be applied to an entirely physical product – Zara’s version of fast fashion is a good example, yet customer feedback still takes weeks to filter through into new clothing.

Video

Most internet-based text content is easy. Changes can be made and distributed in seconds. Reasonably good feedback is available using tools like Google Analytics.

Video content poses a few problems:

• Minor changes require re-rendering and uploading, which (for anyone without Hollywood-scale production facilities) can take several hours, even for just a few minutes of video footage.
• Major changes mean re-filming, editing, sound design, and similar alterations that could take days. May not even be possible, if showing specific events or people.
• Detailed feedback is hard to get. At best you’ll get a reaction to the whole video – typically a number of people that watched, and the rating or comments of a tiny proportion of them.

The first 2 problems aren’t going away anytime soon. The best defense is to save all the footage you shot, including materials that didn’t make the final edit. Recording at full 1280×720 pixel resolution, 30 frames per second, I find that for each minute of the final movie:

• I shot about 20GB of footage,
• take about 10GB to the editing stage, and
• use about 5GB in the final version.

So I’ve already created a lot of redundancy – until I run out of hard disk space. That redundancy helps make minor changes, such as altering the length of a scene, but it won’t let you re-write the story or change the location.

However, small edit tweaks can make the difference between “good and great”, so some iteration is possible within video. In theory. The problem is that without detailed user feedback, how do we know what to improve?

A friend who worked in “new media” when it was new (in the mid-1990s), used say, “the skill was to know when to stop”. To misquote Damien Hirst, “a painting is finished after a long period of looking at it, during which nothing is added”. Personally, video editing involves a lot the later: Constantly replaying a rough version and making changes, until I start making adjustments that seem to make it worse again.

Unfortunately, the creator is not the audience. It’s easy for them to produce things that simply don’t appeal to anyone apart from themselves, don’t solve whatever problem their audience were having, or don’t appeal to viewers’ emotions.

Hot Spots

Which is why I find YouTube’s Hot Spots fascinating.

As often, it started by accident. I couldn’t upload the video below to the host I normally use for embedded video. So the YouTube version of the video became the primary version. Almost all of the video’s 10,000 daily views were hosted on YouTube. This meant that within a day, YouTube’s “Insight” analytics were displaying some representative data about how users were viewing the movie.

The video is primarily a tutorial, intended to introduce the new World of Warcraft fishing contest. All the footage is captured in-game – some of it “live” during the contest, some recorded afterwards. The whole movie was conceived, scripted, filmed, edited, and rendered over the course of 2 days.

Fishing is a good test, because it isn’t a terribly interesting thing to watch. It’s hard to make a “good” fishing video, especially for an audience that aren’t all hardcore anglers.

The video has been favourably rated, comments are complementary and (critically) not (yet) asking questions that the video was intended to answer. Plus a few other sites have embedded it. Good start, but could it be better?

Frame-Based Feedback

Below is the “Hot Spot” graph generated from the first 20,000 views. YouTube’s explanation of the measures:

The ups-and-downs of viewership at each moment in your video, compared to videos of similar length. The higher the graph, the hotter your video: fewer viewers are leaving your video and they may also be rewinding to watch that point in the video again. Audience attention is an overall measure of your video’s ability to retain its audience.

The base (x) axis shows the time the video has been running – it lasts 2 minutes and 2 seconds.

Let’s try and analyse what the graph shows. Numbers refer to points marked on the graph:

1. I added 12 seconds of introduction and title, primarily to give the viewer time focus, adjust the volume or screen size, or let any navigation/control widgets fade away. There are no credits – this is a 2 minute tutorial, not a feature film. Unfortunately there are several ways to read the initial downward decline:
   • Viewers think they missed something at the very start, so are restarting the video (the first sound is triggered while the screen is still black).
   • Viewers have observed the water, net, and title, and didn’t want to still be observing it 5 seconds later – they’re getting bored and leaving.
   • Some viewers started the video by accident, and never intended to watch, however good or otherwise.

   I compared this video to the graph for an earlier YouTube-hosted video. It has the same structure of introduction, but a longer, less focused (a wide, familiar city-scape) initial image. That second video was much colder over the first 10 seconds. So perhaps introductory shots need to be shorter, and move to “the action” faster?

2. The titles are gone, and the video moves to a slow-paced tutorial, with gentle text and rather repetitive scenes. If you watch, the video, you’ll notice a lot of views of the same Walrus-like character, with a lot of gnomes (the pink-haired creature) casting a line or catching a fish. The graph says that’s “ok”, but remains far from hot. In contrast, the comparison video performs better at this stage. The main difference is that the comparision video moves between topics faster, with less repetition of similar-looking scenes. There are a few reasons for repetition: The gradual building of momentum (ever faster scene changes – see next point) was intended to create the sense of excitement that these “first player wins” competitions create, but I don’t think it works. I also wanted to show that the shark (the aim of the contest) could be caught in lots of different places. Past videos have lead players to conclude that only the one precise place shown in the video was valid. Overall, this stage should drag a lot less than it does, and if possible, be made less repetitive.
3. The video gradually builds momentum, until by point 3, the scenes are changing at the rate of around 2 per second (slightly more by the end of the sequence). Audience engagement warms. Possibly this is “exciting”. Possibly too exciting, forcing some viewers to re-wind because they cannot digest the scenes fast enough?
4. The top of this second rise in temperature is marked by the 3 bangs and flashes, cutting to blurred, greyscale, slow-motion sequences. If those don’t make you look, nothing will! In the video’s narrative this is the first time something happens: The gnome caught the shark, and is now running home, desperate to get back first. It’s one of those heart-stopping moments (and in the original storyboard, was intended to use heartbeats). Cool. That is, hot. But worth noting that special effects alone can provide a negative distraction. For example, the comparison video’s coldest moment is when a sequence of quotes and images of their who said them, are merged together into beautiful blue water. Looks great artistically, but doesn’t engage the audience.
5. The heat is maintained while the first prize is displayed. This may be because the tempo of the video doesn’t slow down enough to let viewers digest everything (I had to win quickly, so making sure I had enough footage wasn’t a priority…). It may also reflect greater interest in one of the prizes (the ring). Either way, this section should have been longer.
6. Contest won, interest is dropping. The 6th point occurs when the runner-up prize is displayed. Sadly, the high-point of the story is “the win”, yet the tutorial aspect of the video has to cover not winning. And chronologically, not winning happens after someone has won!
7. We’re ending on a low, which can’t be good. I suspect this is because there isn’t much interest in “the boots” among many players. Perhaps I shouldn’t have dedicated 10 seconds to showing them being used, when the main aim of the video (how to win the contest) was clearly complete? The final giggle was an attempt to liven this section up a little, but comes too late.

The Hot Spots graph doesn’t show everything. Doesn’t reflect any variation between different people viewing it. It may not even be desirable to keep a movie “hot” throughout. There are almost certainly other ways of analyzing viewer behavior.

But areas for improvement emerged that were not seen when making the video. Even if I don’t re-make this particular video, some of improvements will hopefully filter down into new videos.

Yes, But

It’s easy to dismiss this as an expensive training exercise: Wouldn’t it be better just to ensure movie-makers were experienced before they started? If all of them turned out Casablanca, I’d agree. In reality, expertise does not mean infallibly. While YouTube is almost infamous for showing how apparently (to my eyes) terrible videos can be highly popular.

It would be great to think that Saturday night’s cinema audience might see a slightly better version of a film than Friday night’s audience, based on what the first audience enjoyed most. But not terribly practical. Similarly, television news might be history before the second iteration.

But down at “YouTube level” iterative production methods start to become more viable. Still tricky, but something that only took 2 days to initially create, can probably be remade daily, if required.

This isn’t just about rapid re-production. More important is the ability to start to read the minds of an audience the creator can never see. Try to assess what aspects of the video should be expanded. What the audience want, but are only partly getting. And to do that analytically, without the movie-maker ever meeting their audience.

At the extreme, it’s the introduction of almost scientific methodology into an artistic process, traditionally based around the artist’s opinion of their own work, and their experiences to date.

Most intriguing is that “the next” Steven Spielberg (or similar) probably isn’t learning their art with an amateur 8mm camera. They’ll be uploading camera-phone videos, animating Lego, or “machinimating” goblins. And there’s a chance they will start learning to use the analytical feedback available to them, in a way older generations never could…

Learn More

• The long lost formula for start-up success. No, really – Nigel Eccles (an Edinburgh acquaintance, although I don’t think we’ve ever discussed this) wrote something similar about iterative development. That we’re thinking alike isn’t entirely unexpected, since we’re both rather analytically-minded.
• Incremental versus iterative development – Useful clarification of 2 often-confused terms, by Alistair Cockburn.
• Wikipedia – Introduces various similar software-orientated methods.

Appendix: Iterative Writing in Practice

How did one small gnome attract so many readers, in spite of initially writing about the wrong thing? A healthy chunk of curiosity helps: Search and you may find things that work even better. But since you asked, consider this:

• Split text into separate pages (not like this article): It is far easier to trace and monitor page views, than to work out where on a page a reader is reading.
• Write around the edges of your topic: Both broader introductions and more specific detail than your core starting material. If the introductory material becomes more popular than the core, expand that introduction, and so on.
• Understand who is trying to read: Specifically their education, age, time-pressure – and write to a style and length that this audience can read.
• Watch what users do: Extensive forum discussions or comments are subtle indicators of what you need to offer. See what users do in the absence of anything you’ve written. A 200-post forum discussion about something you thought was trivial, clearly isn’t.
• Personalise it: The internet is a scary place, and you don’t help ease that fear by presenting words as a robot.

That isn’t the whole story. And there are many techniques within that. Remember that all the basic design guidelines on things like the structuring of text still apply.

In my opinion, it’s an on-going exercise in the discovery of the fact that most people aren’t like you, and have different problems and needs. Logical, really: If you write for yourself, you’ve optimized the text for people like you. Yet people like you write… and so have the least need of someone else’s writing!