Partly it isn’t because creators haven’t adjusted their methods to match the new technology – we’re still wedded to a single start-to-finish process, with one outcome at the end. Partly it isn’t because feedback can be hard to gather and digest, and even hard to act upon.

An iterative method has become one of the defining characteristics of how I like to write, organise, and present text on the internet. At least, beyond this domain. But until now, I’ve struggled to apply it to internet-based video.

This article introduces internet-based iterative design, and uses YouTube’s “Hot Spot” analysis to show how we can start to apply an iterative approach to video and movie-making.

Iterative Product Development

The author of a published paper book generally gets one shot. One chance to have their works committed to paper. To be read by milllions. Or tens. A huge amount of effort goes into “getting it right”: Construction of text and story, editing, proof-checking. And in spite of this, book publishing remains a high-risk activity: For every top-selling author, there are others whose work ends up as pulp.

In contrast, the cost of making corrections or changes on the internet can be minor. At the extreme, the author simply types some new words. An update that might have taken a book publisher months or years, can be committed in minutes.

The ability to make rapid changes in response to rapidly gathered feedback makes the internet interesting: The most basic server access logs can be analysed to reveal that chapter 2 is generated much more interest than chapter 1, yet chapter 4 is hardly getting read. With enough readers, those patterns can be seen in days, or even hours. So perhaps the content in chapter 2 should be expanded, and we should re-write chapter 3 to better maintain interest?

Now we’ve expanded chapter 2, and noticed it has become even more popular. Obviously there’s a greater demand for the writing or information in chapter 2 than the author thought there was. Gradually the content evolves and gravitates towards (in the language of entrepreneurs) the nearest unserved market. Iterative product development isn’t just about “making it better”. It’s a way of finding an audience, customers, earnings.

The written word is an easy example to understand, but maybe all good design iterates in response to user feedback?

Probably always has. Stone wheels? Computers sold as recipe books? Especially where a cool technology is looking for a problem: An inventor that doesn’t start by trying to address a problem, but merely discovers a method of doing something. Post-It notes are a popular example, but this pattern is common from the Victorian era onward. For example, it took half a century of different people trying to use pressurized air for land-based transport propulsion, before a market niche was established.

The internet allows this process to happen a lot faster, but only if the presence of the internet becomes integral to the design process.

Personally, this methodology has turned a few highly technical pages on the mechanics of fishing, into a fishing guide read by millions (see Appendix: Iterative Writing in Practice at the bottom).

Limitations

Iterative product development isn’t a panacea. Or a free ride to perfection and untold riches:

• While internet-based products and services are comparatively (to manufactured goods) cheap and easy to change, constantly making changes can become very time-consuming. Designing with the expectation of change, helps. But ultimately you will reach a point where further changes don’t generate enough extra audience interest to (financially) justify the time spent making the changes. This is when to stop.
• There is no guarantee that your product will find the absolute biggest unserved market, merely the biggest such market near to the topic/interest area you started with. If you started developing an idea in an obscure niche, it’s not realistic to expect to grow outside of that niche.
• Iterative development is not an excuse to design garbage. Quality remains important: The first attempt has to be sufficiently “good” for enough people to use/read it to generate feedback.
• The process of analysing feedback, and developing new content, requires 2 distinct skill-sets. Someone that is good at the second, may be unable to do the first. An instinctively good designer might still produce a better product, although (I would argue) their method leaves more to chance.
• It’s not clear that this method could be applied to an entirely physical product – Zara’s version of fast fashion is a good example, yet customer feedback still takes weeks to filter through into new clothing.

Video

Most internet-based text content is easy. Changes can be made and distributed in seconds. Reasonably good feedback is available using tools like Google Analytics.

Video content poses a few problems:

• Minor changes require re-rendering and uploading, which (for anyone without Hollywood-scale production facilities) can take several hours, even for just a few minutes of video footage.
• Major changes mean re-filming, editing, sound design, and similar alterations that could take days. May not even be possible, if showing specific events or people.
• Detailed feedback is hard to get. At best you’ll get a reaction to the whole video – typically a number of people that watched, and the rating or comments of a tiny proportion of them.

The first 2 problems aren’t going away anytime soon. The best defense is to save all the footage you shot, including materials that didn’t make the final edit. Recording at full 1280×720 pixel resolution, 30 frames per second, I find that for each minute of the final movie:

• I shot about 20GB of footage,
• take about 10GB to the editing stage, and
• use about 5GB in the final version.

So I’ve already created a lot of redundancy – until I run out of hard disk space. That redundancy helps make minor changes, such as altering the length of a scene, but it won’t let you re-write the story or change the location.

However, small edit tweaks can make the difference between “good and great”, so some iteration is possible within video. In theory. The problem is that without detailed user feedback, how do we know what to improve?

A friend who worked in “new media” when it was new (in the mid-1990s), used say, “the skill was to know when to stop”. To misquote Damien Hirst, “a painting is finished after a long period of looking at it, during which nothing is added”. Personally, video editing involves a lot the later: Constantly replaying a rough version and making changes, until I start making adjustments that seem to make it worse again.

Unfortunately, the creator is not the audience. It’s easy for them to produce things that simply don’t appeal to anyone apart from themselves, don’t solve whatever problem their audience were having, or don’t appeal to viewers’ emotions.

Hot Spots

Which is why I find YouTube’s Hot Spots fascinating.

As often, it started by accident. I couldn’t upload the video below to the host I normally use for embedded video. So the YouTube version of the video became the primary version. Almost all of the video’s 10,000 daily views were hosted on YouTube. This meant that within a day, YouTube’s “Insight” analytics were displaying some representative data about how users were viewing the movie.

The video is primarily a tutorial, intended to introduce the new World of Warcraft fishing contest. All the footage is captured in-game – some of it “live” during the contest, some recorded afterwards. The whole movie was conceived, scripted, filmed, edited, and rendered over the course of 2 days.

Fishing is a good test, because it isn’t a terribly interesting thing to watch. It’s hard to make a “good” fishing video, especially for an audience that aren’t all hardcore anglers.

The video has been favourably rated, comments are complementary and (critically) not (yet) asking questions that the video was intended to answer. Plus a few other sites have embedded it. Good start, but could it be better?

Frame-Based Feedback

Below is the “Hot Spot” graph generated from the first 20,000 views. YouTube’s explanation of the measures:

The ups-and-downs of viewership at each moment in your video, compared to videos of similar length. The higher the graph, the hotter your video: fewer viewers are leaving your video and they may also be rewinding to watch that point in the video again. Audience attention is an overall measure of your video’s ability to retain its audience.

The base (x) axis shows the time the video has been running – it lasts 2 minutes and 2 seconds.

Let’s try and analyse what the graph shows. Numbers refer to points marked on the graph:

1. I added 12 seconds of introduction and title, primarily to give the viewer time focus, adjust the volume or screen size, or let any navigation/control widgets fade away. There are no credits – this is a 2 minute tutorial, not a feature film. Unfortunately there are several ways to read the initial downward decline:
   • Viewers think they missed something at the very start, so are restarting the video (the first sound is triggered while the screen is still black).
   • Viewers have observed the water, net, and title, and didn’t want to still be observing it 5 seconds later – they’re getting bored and leaving.
   • Some viewers started the video by accident, and never intended to watch, however good or otherwise.

   I compared this video to the graph for an earlier YouTube-hosted video. It has the same structure of introduction, but a longer, less focused (a wide, familiar city-scape) initial image. That second video was much colder over the first 10 seconds. So perhaps introductory shots need to be shorter, and move to “the action” faster?

2. The titles are gone, and the video moves to a slow-paced tutorial, with gentle text and rather repetitive scenes. If you watch, the video, you’ll notice a lot of views of the same Walrus-like character, with a lot of gnomes (the pink-haired creature) casting a line or catching a fish. The graph says that’s “ok”, but remains far from hot. In contrast, the comparison video performs better at this stage. The main difference is that the comparision video moves between topics faster, with less repetition of similar-looking scenes. There are a few reasons for repetition: The gradual building of momentum (ever faster scene changes – see next point) was intended to create the sense of excitement that these “first player wins” competitions create, but I don’t think it works. I also wanted to show that the shark (the aim of the contest) could be caught in lots of different places. Past videos have lead players to conclude that only the one precise place shown in the video was valid. Overall, this stage should drag a lot less than it does, and if possible, be made less repetitive.
3. The video gradually builds momentum, until by point 3, the scenes are changing at the rate of around 2 per second (slightly more by the end of the sequence). Audience engagement warms. Possibly this is “exciting”. Possibly too exciting, forcing some viewers to re-wind because they cannot digest the scenes fast enough?
4. The top of this second rise in temperature is marked by the 3 bangs and flashes, cutting to blurred, greyscale, slow-motion sequences. If those don’t make you look, nothing will! In the video’s narrative this is the first time something happens: The gnome caught the shark, and is now running home, desperate to get back first. It’s one of those heart-stopping moments (and in the original storyboard, was intended to use heartbeats). Cool. That is, hot. But worth noting that special effects alone can provide a negative distraction. For example, the comparison video’s coldest moment is when a sequence of quotes and images of their who said them, are merged together into beautiful blue water. Looks great artistically, but doesn’t engage the audience.
5. The heat is maintained while the first prize is displayed. This may be because the tempo of the video doesn’t slow down enough to let viewers digest everything (I had to win quickly, so making sure I had enough footage wasn’t a priority…). It may also reflect greater interest in one of the prizes (the ring). Either way, this section should have been longer.
6. Contest won, interest is dropping. The 6th point occurs when the runner-up prize is displayed. Sadly, the high-point of the story is “the win”, yet the tutorial aspect of the video has to cover not winning. And chronologically, not winning happens after someone has won!
7. We’re ending on a low, which can’t be good. I suspect this is because there isn’t much interest in “the boots” among many players. Perhaps I shouldn’t have dedicated 10 seconds to showing them being used, when the main aim of the video (how to win the contest) was clearly complete? The final giggle was an attempt to liven this section up a little, but comes too late.

The Hot Spots graph doesn’t show everything. Doesn’t reflect any variation between different people viewing it. It may not even be desirable to keep a movie “hot” throughout. There are almost certainly other ways of analyzing viewer behavior.

But areas for improvement emerged that were not seen when making the video. Even if I don’t re-make this particular video, some of improvements will hopefully filter down into new videos.

Yes, But

It’s easy to dismiss this as an expensive training exercise: Wouldn’t it be better just to ensure movie-makers were experienced before they started? If all of them turned out Casablanca, I’d agree. In reality, expertise does not mean infallibly. While YouTube is almost infamous for showing how apparently (to my eyes) terrible videos can be highly popular.

It would be great to think that Saturday night’s cinema audience might see a slightly better version of a film than Friday night’s audience, based on what the first audience enjoyed most. But not terribly practical. Similarly, television news might be history before the second iteration.

But down at “YouTube level” iterative production methods start to become more viable. Still tricky, but something that only took 2 days to initially create, can probably be remade daily, if required.

This isn’t just about rapid re-production. More important is the ability to start to read the minds of an audience the creator can never see. Try to assess what aspects of the video should be expanded. What the audience want, but are only partly getting. And to do that analytically, without the movie-maker ever meeting their audience.

At the extreme, it’s the introduction of almost scientific methodology into an artistic process, traditionally based around the artist’s opinion of their own work, and their experiences to date.

Most intriguing is that “the next” Steven Spielberg (or similar) probably isn’t learning their art with an amateur 8mm camera. They’ll be uploading camera-phone videos, animating Lego, or “machinimating” goblins. And there’s a chance they will start learning to use the analytical feedback available to them, in a way older generations never could…

Learn More

• The long lost formula for start-up success. No, really – Nigel Eccles (an Edinburgh acquaintance, although I don’t think we’ve ever discussed this) wrote something similar about iterative development. That we’re thinking alike isn’t entirely unexpected, since we’re both rather analytically-minded.
• Incremental versus iterative development – Useful clarification of 2 often-confused terms, by Alistair Cockburn.
• Wikipedia – Introduces various similar software-orientated methods.

Appendix: Iterative Writing in Practice

How did one small gnome attract so many readers, in spite of initially writing about the wrong thing? A healthy chunk of curiosity helps: Search and you may find things that work even better. But since you asked, consider this:

• Split text into separate pages (not like this article): It is far easier to trace and monitor page views, than to work out where on a page a reader is reading.
• Write around the edges of your topic: Both broader introductions and more specific detail than your core starting material. If the introductory material becomes more popular than the core, expand that introduction, and so on.
• Understand who is trying to read: Specifically their education, age, time-pressure – and write to a style and length that this audience can read.
• Watch what users do: Extensive forum discussions or comments are subtle indicators of what you need to offer. See what users do in the absence of anything you’ve written. A 200-post forum discussion about something you thought was trivial, clearly isn’t.
• Personalise it: The internet is a scary place, and you don’t help ease that fear by presenting words as a robot.

That isn’t the whole story. And there are many techniques within that. Remember that all the basic design guidelines on things like the structuring of text still apply.

In my opinion, it’s an on-going exercise in the discovery of the fact that most people aren’t like you, and have different problems and needs. Logical, really: If you write for yourself, you’ve optimized the text for people like you. Yet people like you write… and so have the least need of someone else’s writing!

That hasn’t stopped “screen-scraping” of data or simply typing in paper timetables (from Urban Mapping to many listed here). Unfortunately, the legal basis for scraping is complex, which creates significant risks for anyone building a business. For example, earlier this year, airline Ryanair requested the removal of all their data from Skyscanner, a flight price comparison site that gathers data by scraping airlines’ websites. How many airlines would need to object to their data being scraped before a “price comparison” service becomes unusable?

User-generated mapping content is evolving, often to circumvent restrictive distribution of national mapping. Services include OpenStreetMap and the recently announced Google Map Maker.

Micro-blogging, primarily through Twitter, has started to show the potential of individual travellers to report information about their journeys: Ron Whitman’s Commuter Feed is a good example. Tom Morris has also experimented with London Twitter feeds.

This article outlines why the “social web”/tech-entrepreneur sector may wish to stop trying to use official sources of data, and instead apply the technology it understands best: People.

The Big Picture

I will use the example of UK local bus data to summarise the strategic issues for data providers. I can only presume the issues are similar elsewhere (comments welcome).

Explaining exactly who the data providers are is one of the many problems of trying to extract and use the data. I would provide more detail, but the topic is somewhat sensitive. The most critical point in the chain that constructs and distributes the data are local authorities – sub-regional public bodies, typically those responsible for large cities, conurbations or counties. They process the data, but are not under any statutory requirement to do so (no national government legislation requires it).

There are a number of issues for the existing data providers:

1. Mindset of centralised control: Most operators, public authorities, and other agencies, still have a mindset of centralised control of information, delivered to users via the method the agency believes is appropriate. This is heavily driven by the belief that only the agency can be accountable or impartial, and that incorrect information supplied by an uncontrolled third party is likely to damage the image of local transport service and generally reflect badly on the agency.
2. Mindset of local: Most agencies are locally focused, locally orientated. It seems logical for them to commission a fully-functioning website or piece of information delivery software that is specific to their city, because their target market is local. There’s a lack of global perspective: An agency will typically commission a system that is specific to their city, even when 95% of the features would work for any city, and 90% are already in existing global products.
3. Not appreciating trends in delivery channels: There is still an attitude of “we’ll provide a website”, without a comprehension that the number of channels for delivery of information is exploding far faster than any one agency can hope to construct bespoke user interfaces to cater for. Mobile devices, integration into social software. There would probably a market for a “WiFi-enabled” alarm clock that would ring later if your morning train had been delayed: We simply can’t define the limits for how this information might be used.
4. Not appreciating trends in cost: Even large, well-funded agencies are starting to fall behind the technology. The cost of systems (many millions of dollars invested year on year in some cases) is starting to hurt. Logically the global system should win out, because one city is very much like another: There is considerable scope for sharing systems costs.

What It Means

Long term we are heading for global providers of information, that pool data from local sources. That will be forced by the cost of technology. This can be seen in technology costs driving things like agglomeration in the groceries sector (such as Walmart) over the last 30 years. Also in the move from customised mainframe computing, to shared operating systems and platforms (such as Windows). This will be worse, because the number of systems will be simultaneously exploding alongside the complexity of those systems.

As these issues become progressively better understood, data will become more centralised. Even in agencies where (in my opinion) uniqueness and absolute control are culturally in-breed, such as London Transport/TfL, cost will eventually win the argument.

However, centralised data handling does not automatically make the data open. Quite the opposite.

Contracted Provision

Currently, effective control of data is with local government. Many individuals within local government will naturally attempt to block any change that might leverage power away from them and their organisation. “Job protection” is an over-simplification, but helps explain the underlying position. But by contracting data handling and presentation to a third-party contractor, local government would gain the technological “economies of scale” (assuming the contractor won many contracts from different authorities) and notionally maintain control.

Use of third-party contractors is already common within the local government sector, particularly for Information Technology.

An example can be seen in Edinburgh City Council’s Traffic Map. In spite of how it appears, the information isn’t powered directly by Edinburgh City Council or Google. Instead it is part of Mott MacDonald’s Common Data Management Facility, providing services under contract to many different local authorities.

In the UK public transport arena, Trapeze is a good example of the gradual agglomeration of data handling within a few large businesses, where historically many small software providers could be found.

The example above provides key driver information, and is somewhat useful, but is it the best outcome? I suspect not. Contracts tend to be priced highly, because local government clients are high risk: Their political control means that they can change their strategic direction and requirements unexpectedly. At best, customer feedback loops through local authorities are slow and politicised. At worst the design of the system will reflect the arbitrary views of a self-proclaimed expert (such as myself). Even if you think it is perfect, there is no scope for choice or creativity. Choice is good and need not be expensive.

Social Provision

Instead of using official data, why not let users reconstruct it? User-generated content is cheaper to create than information from professionally staffed sources: Since very many contributors do so little work, no individual expects payment. User-generated content can be just as accurate too, although this is not automatic: For example, a strong community will subject everything to peer review, weeding out poor information and contributors.

This is not an entirely theoretical position. There is a largely untapped human resource, just waiting to help.

The transport enthusiasts (transit fans, “spotters”) already collate and produce some extremely high quality information about certain technical aspects of operations and services. For example, sites such as LondonBusRoutes.net contain detail on the bus route timing and vehicle allocation (type and number of buses), which transpires to be difficult to extract from official sources. While it may be argued that these sites simply repackage official information, their very existence is a testament to the strength of underlying community.

Casual observation of people delayed on trains or in traffic suggests they derive some comfort from picking up their mobile (cell) phone and telling someone about it. Something they can do, in a scenario they otherwise have no control over. Their desire to communicate the same information to drivers or users 10 miles behind them (who might be able to re-plan their route, should they know) is untested. But the potential is intriguing.

Nobody has entirely worked out how to use these people; yet.

Battle Lines

If the social web/tech-entrepreneur sector chooses to fight the “status quo” head on, it does so against large multi-national IT providers who support clients with historically entrenched positions. Not a contest that favours the underdog.

If the tech’ “upstarts” can find a way to use this human resource effectively, they will ultimately provide a more cost-effective solution than the traditional “government IT” sector can offer. Integrate that user-generated information into the wider consumer internet, and the machinery of government simply won’t be able to justify its historic position of pouring millions into systems it controls. The “social web”/tech-entrepreneur sector wins.

The upstarts do not need perfect source data, if the implementation of results is considered to be better by users. The early Xephos vs TransportDirect comparisons provide some evidence. The success or failure of the social web/tech-entrepreneur sector is ultimately dependant on whether they can provide better information than official sources, using the resources and skills they have available to them.

Disclaimer: The contents of this article reflect my own personal analysis of the situation. This does not directly reflect advice to, or views of, government or anyone else involved in the handling and provision of public transportation data.