Iterative Video Development

The internet allows products and services to be rapidly improved based on user feedback. So rapid, that iterative design should become the primary method of designing internet-based services. Not just as an Agile-like method of working, but as a method of specifying the product itself.

Partly it isn’t because creators haven’t adjusted their methods to match the new technology – we’re still wedded to a single start-to-finish process, with one outcome at the end. Partly it isn’t because feedback can be hard to gather and digest, and even hard to act upon.

An iterative method has become one of the defining characteristics of how I like to write, organise, and present text on the internet. At least, beyond this domain. But until now, I’ve struggled to apply it to internet-based video.

This article introduces internet-based iterative design, and uses YouTube’s “Hot Spot” analysis to show how we can start to apply an iterative approach to video and movie-making.

Iterative Product Development

The author of a published paper book generally gets one shot. One chance to have their works committed to paper. To be read by milllions. Or tens. A huge amount of effort goes into “getting it right”: Construction of text and story, editing, proof-checking. And in spite of this, book publishing remains a high-risk activity: For every top-selling author, there are others whose work ends up as pulp.

In contrast, the cost of making corrections or changes on the internet can be minor. At the extreme, the author simply types some new words. An update that might have taken a book publisher months or years, can be committed in minutes.

The ability to make rapid changes in response to rapidly gathered feedback makes the internet interesting: The most basic server access logs can be analysed to reveal that chapter 2 is generated much more interest than chapter 1, yet chapter 4 is hardly getting read. With enough readers, those patterns can be seen in days, or even hours. So perhaps the content in chapter 2 should be expanded, and we should re-write chapter 3 to better maintain interest?

Now we’ve expanded chapter 2, and noticed it has become even more popular. Obviously there’s a greater demand for the writing or information in chapter 2 than the author thought there was. Gradually the content evolves and gravitates towards (in the language of entrepreneurs) the nearest unserved market. Iterative product development isn’t just about “making it better”. It’s a way of finding an audience, customers, earnings.

The written word is an easy example to understand, but maybe all good design iterates in response to user feedback?

Probably always has. Stone wheels? Computers sold as recipe books? Especially where a cool technology is looking for a problem: An inventor that doesn’t start by trying to address a problem, but merely discovers a method of doing something. Post-It notes are a popular example, but this pattern is common from the Victorian era onward. For example, it took half a century of different people trying to use pressurized air for land-based transport propulsion, before a market niche was established.

The internet allows this process to happen a lot faster, but only if the presence of the internet becomes integral to the design process.

Personally, this methodology has turned a few highly technical pages on the mechanics of fishing, into a fishing guide read by millions (see Appendix: Iterative Writing in Practice at the bottom).

Limitations

Iterative product development isn’t a panacea. Or a free ride to perfection and untold riches:

Video

Most internet-based text content is easy. Changes can be made and distributed in seconds. Reasonably good feedback is available using tools like Google Analytics.

Video content poses a few problems:

The first 2 problems aren’t going away anytime soon. The best defense is to save all the footage you shot, including materials that didn’t make the final edit. Recording at full 1280×720 pixel resolution, 30 frames per second, I find that for each minute of the final movie:

So I’ve already created a lot of redundancy – until I run out of hard disk space. That redundancy helps make minor changes, such as altering the length of a scene, but it won’t let you re-write the story or change the location.

However, small edit tweaks can make the difference between “good and great”, so some iteration is possible within video. In theory. The problem is that without detailed user feedback, how do we know what to improve?

A friend who worked in “new media” when it was new (in the mid-1990s), used say, “the skill was to know when to stop”. To misquote Damien Hirst, “a painting is finished after a long period of looking at it, during which nothing is added”. Personally, video editing involves a lot the later: Constantly replaying a rough version and making changes, until I start making adjustments that seem to make it worse again.

Unfortunately, the creator is not the audience. It’s easy for them to produce things that simply don’t appeal to anyone apart from themselves, don’t solve whatever problem their audience were having, or don’t appeal to viewers’ emotions.

Hot Spots

Which is why I find YouTube’s Hot Spots fascinating.

As often, it started by accident. I couldn’t upload the video below to the host I normally use for embedded video. So the YouTube version of the video became the primary version. Almost all of the video’s 10,000 daily views were hosted on YouTube. This meant that within a day, YouTube’s “Insight” analytics were displaying some representative data about how users were viewing the movie.

The video is primarily a tutorial, intended to introduce the new World of Warcraft fishing contest. All the footage is captured in-game – some of it “live” during the contest, some recorded afterwards. The whole movie was conceived, scripted, filmed, edited, and rendered over the course of 2 days.

Fishing is a good test, because it isn’t a terribly interesting thing to watch. It’s hard to make a “good” fishing video, especially for an audience that aren’t all hardcore anglers.

The video has been favourably rated, comments are complementary and (critically) not (yet) asking questions that the video was intended to answer. Plus a few other sites have embedded it. Good start, but could it be better?

Frame-Based Feedback

Below is the “Hot Spot” graph generated from the first 20,000 views. YouTube’s explanation of the measures:

The ups-and-downs of viewership at each moment in your video, compared to videos of similar length. The higher the graph, the hotter your video: fewer viewers are leaving your video and they may also be rewinding to watch that point in the video again. Audience attention is an overall measure of your video’s ability to retain its audience.

The base (x) axis shows the time the video has been running – it lasts 2 minutes and 2 seconds.

Kalu'ak Fishing Derby YouTube Hot Spot Graph.

Let’s try and analyse what the graph shows. Numbers refer to points marked on the graph:

  1. I added 12 seconds of introduction and title, primarily to give the viewer time focus, adjust the volume or screen size, or let any navigation/control widgets fade away. There are no credits – this is a 2 minute tutorial, not a feature film. Unfortunately there are several ways to read the initial downward decline:
    • Viewers think they missed something at the very start, so are restarting the video (the first sound is triggered while the screen is still black).
    • Viewers have observed the water, net, and title, and didn’t want to still be observing it 5 seconds later – they’re getting bored and leaving.
    • Some viewers started the video by accident, and never intended to watch, however good or otherwise.

    I compared this video to the graph for an earlier YouTube-hosted video. It has the same structure of introduction, but a longer, less focused (a wide, familiar city-scape) initial image. That second video was much colder over the first 10 seconds. So perhaps introductory shots need to be shorter, and move to “the action” faster?

  2. The titles are gone, and the video moves to a slow-paced tutorial, with gentle text and rather repetitive scenes. If you watch, the video, you’ll notice a lot of views of the same Walrus-like character, with a lot of gnomes (the pink-haired creature) casting a line or catching a fish. The graph says that’s “ok”, but remains far from hot. In contrast, the comparison video performs better at this stage. The main difference is that the comparision video moves between topics faster, with less repetition of similar-looking scenes. There are a few reasons for repetition: The gradual building of momentum (ever faster scene changes – see next point) was intended to create the sense of excitement that these “first player wins” competitions create, but I don’t think it works. I also wanted to show that the shark (the aim of the contest) could be caught in lots of different places. Past videos have lead players to conclude that only the one precise place shown in the video was valid. Overall, this stage should drag a lot less than it does, and if possible, be made less repetitive.
  3. The video gradually builds momentum, until by point 3, the scenes are changing at the rate of around 2 per second (slightly more by the end of the sequence). Audience engagement warms. Possibly this is “exciting”. Possibly too exciting, forcing some viewers to re-wind because they cannot digest the scenes fast enough?
  4. The top of this second rise in temperature is marked by the 3 bangs and flashes, cutting to blurred, greyscale, slow-motion sequences. If those don’t make you look, nothing will! In the video’s narrative this is the first time something happens: The gnome caught the shark, and is now running home, desperate to get back first. It’s one of those heart-stopping moments (and in the original storyboard, was intended to use heartbeats). Cool. That is, hot. But worth noting that special effects alone can provide a negative distraction. For example, the comparison video’s coldest moment is when a sequence of quotes and images of their who said them, are merged together into beautiful blue water. Looks great artistically, but doesn’t engage the audience.
  5. The heat is maintained while the first prize is displayed. This may be because the tempo of the video doesn’t slow down enough to let viewers digest everything (I had to win quickly, so making sure I had enough footage wasn’t a priority…). It may also reflect greater interest in one of the prizes (the ring). Either way, this section should have been longer.
  6. Contest won, interest is dropping. The 6th point occurs when the runner-up prize is displayed. Sadly, the high-point of the story is “the win”, yet the tutorial aspect of the video has to cover not winning. And chronologically, not winning happens after someone has won!
  7. We’re ending on a low, which can’t be good. I suspect this is because there isn’t much interest in “the boots” among many players. Perhaps I shouldn’t have dedicated 10 seconds to showing them being used, when the main aim of the video (how to win the contest) was clearly complete? The final giggle was an attempt to liven this section up a little, but comes too late.

The Hot Spots graph doesn’t show everything. Doesn’t reflect any variation between different people viewing it. It may not even be desirable to keep a movie “hot” throughout. There are almost certainly other ways of analyzing viewer behavior.

But areas for improvement emerged that were not seen when making the video. Even if I don’t re-make this particular video, some of improvements will hopefully filter down into new videos.

Yes, But

It’s easy to dismiss this as an expensive training exercise: Wouldn’t it be better just to ensure movie-makers were experienced before they started? If all of them turned out Casablanca, I’d agree. In reality, expertise does not mean infallibly. While YouTube is almost infamous for showing how apparently (to my eyes) terrible videos can be highly popular.

It would be great to think that Saturday night’s cinema audience might see a slightly better version of a film than Friday night’s audience, based on what the first audience enjoyed most. But not terribly practical. Similarly, television news might be history before the second iteration.

But down at “YouTube level” iterative production methods start to become more viable. Still tricky, but something that only took 2 days to initially create, can probably be remade daily, if required.

This isn’t just about rapid re-production. More important is the ability to start to read the minds of an audience the creator can never see. Try to assess what aspects of the video should be expanded. What the audience want, but are only partly getting. And to do that analytically, without the movie-maker ever meeting their audience.

At the extreme, it’s the introduction of almost scientific methodology into an artistic process, traditionally based around the artist’s opinion of their own work, and their experiences to date.

Most intriguing is that “the next” Steven Spielberg (or similar) probably isn’t learning their art with an amateur 8mm camera. They’ll be uploading camera-phone videos, animating Lego, or “machinimating” goblins. And there’s a chance they will start learning to use the analytical feedback available to them, in a way older generations never could…

Learn More

Appendix: Iterative Writing in Practice

How did one small gnome attract so many readers, in spite of initially writing about the wrong thing? A healthy chunk of curiosity helps: Search and you may find things that work even better. But since you asked, consider this:

That isn’t the whole story. And there are many techniques within that. Remember that all the basic design guidelines on things like the structuring of text still apply.

In my opinion, it’s an on-going exercise in the discovery of the fact that most people aren’t like you, and have different problems and needs. Logical, really: If you write for yourself, you’ve optimized the text for people like you. Yet people like you write… and so have the least need of someone else’s writing!

Tim Howgego, 12th November 2009. Related topics: Analysis, Design, El, Google, Innovation, Machinima, Video. 2 Comments »

2 comments on "Iterative Video Development"

  1. On November 15th, 2009 at 2:23 am Larísa wrote:

    The question is of cause where your motivation lies. Everyone hasn’t got a goal to reach and please the biggest audience possible. If you’re making a commercial product you probably want to do that, but not necessarily if you’re at an amateur level. While I think your approach is very interesting in my professional role (being into PR myself), I deliberately refrain from it while I’m blogging my heart out at my wow blog just for the fun of it and as a relaxing way of spending my spare time. Yeah, I like the readers to be involved, I love to discuss with them in comments, but no way that I would bother to go back and change my blogposts following the feedback in their comments! I guess blogging isn’t exactly an activity made for an iterative approach? What’s written is written and tomorrow is another day…

  2. On November 15th, 2009 at 12:55 pm Tim Howgego wrote:

    Objectives vary.

    El is certainly a lot more professional now than “she” started out. And when given the opportunity, I tend towards perfection. Where perfection is judged relative to (that is, by) other people. It’s not a surprise I ended up here.

    It is still perplexing more people don’t follow. For example, I see a lot of new “tech” businesses agonize over detailed business plans, only to have to reinvent themselves a few months into the project. Traditional industries with long lead-times and large numbers of employees have to work like that. But a lot of internet-based things do not. So as a broad approach, it has a lot of potential.

    ‘Blog iteration would still be possible, but rather than change individual posts, iterate within new content. Of course ‘blogs tend to be more of a stream (or my case, trickle) of consciousness, where often the author is writing for themselves.

Comments welcome