Publishing to the iPad

Publishing to the iPad

NPG recently launched a new iPad app Nature Journals – an app that allows us to distribute journal content to iPad users. I thought it might be interesting to highlight a few of the design decisions we took and discuss why we took them.

Most publishers when they make an iPad magazine tend to design a skeuomorphic digital facsimile of their printed magazine – they build in lots of interactive features but build it using similar production processes as for print and make it feel like a print magazine. They layout each page (actually they need to layout each page twice one for landscape and one for portrait view) and then produce a big file to be distributed via Apple’s app store.

This approach feels very wrong to me. For starters it doesn’t scale well – every issue needs a bunch of people to layout and produce it; from an end users point of view they get a very big file and I’ve seen nothing to convince me most people want all the extra stuff; and from an engineering point of view the lack of separation of concerns worries me. I just think most iPad Magazines are doing it wrong.

Now to be clear I’m not for a moment suggesting that what we’ve built is perfect – I know its not – but I think, I hope we’re on the right track.

So what did we do?

Our overarching focus was to create a clean, uncluttered user experience. We didn’t want to replicate print nor replicate the Website instead we wanted to take a path that focused on the content at the expense of ‘features’ while giving the reader the essence of the printed journals.

This meant we wanted decent typography, enough branding to connect the user to the journal but no more and the features we did build had to be justified in terms of benefits to a scientist’s understanding of the article. And even then we pushed most of the functionality away from the forefront of the interface so that the reader hopefully isn’t too aware of the app. The best app after all is no app.

In my experience most publishers tend to go the other way (although there are notable exceptions) – most iPad Magazines have a lot of app and a lot of bells and whistles, so many features in fact that many magazines need an instruction manual to help you navigate them! That can’t be right.

As Craig Mod put it – many publishers build a Homer.

TheHomer

When Homer Simpson was asked to design his ideal car, he made The Homer. Given free reign, Homer’s process was additive. He added three horns and a special sound-proof bubble for the children. He layered more atop everything cars had been. More horns, more cup holders.

We didn’t want to build a Homer! We tried to only include features where they really benefit the reader or their community. For example, we built a figure viewer which lets the reader see the figures within the article at any point and tap through to higher resolution images because that’s useful.

You can also bookmark or share an article, download the PDF but these are only there if you need them. The normal reading behaviour assumes you don’t need this stuff and so they are hidden away (until you tap the screen to pull then into focus).

Back to the content…

It’s hard to build good automated pagination unless the content is very simple and homogenous. Beautiful, fast pagination for most content is simply too hard unless you build each page by hand. Nasty, poorly designed and implemented pagination doesn’t help anyone. We therefore decided to go with scrolling within an article and pagination between articles.

Under the hood we wanted to build a system that would scale, could be automated and ensured separation of concerns.

On the server we therefore render EPUB files from the raw XML documents in MarkLogic and bundle those files along with all the images and other assets into a zip file and serve them to the iPad app.

From the readers point of view this means they can download whole issues for offline reading  and the total package is quite small – an issue of Nature is c. 30MB, the Review Journals can be as small as 5MB by way of comparison Wired is c. 250MB.

From our point of view the entire production is automated – we don’t need to have people laying out every page or issue. This also means that as we improve the layout so we can rollout those improvements to all the articles – both new content and the archive (although users would need to re download the content).

Our development manifesto

Our development manifesto

Manifesto’s are quite popular in the tech community — obviously there’s the agile manifesto and I’ve written before about the kaizen manifesto and then there’s the Manifesto for Software Craftsmanship. They all try to put forward a way of working, a way of raising professionalism and a way of improving the quality of what you do and build.

Anyway when we started work on on the BBC’s Nature site we set out our development manifesto. I thought you might be interested in it:

  1. Peristence — only mint a new URIs if one doesn’t already exist: once minted, never delete it
  2. Linked open data — data and documents describe the real world; things in the real world are identified via HTTP URIs; links describe how those things are related to each other.
  3. The website is the API
  4. RESTful — the Web is stateless, work with this architecture, not against it.
  5. One Web – one canonical URI for each resource (thing), dereferenced to the appropriate representation (HTML, JSON, RDF, etc.).
  6. Fix the data don’t hack the code
  7. Books have pages, the web has links
  8. Do it right or don’t do it at all — don’t hack in quick fixes or ‘tactical solutions’ they are bad for users and bad for the code.
  9. Release early, release often — small, incremental changes are easy to test and proof.

It’s worth noting that we didn’t always live up to these standards — but at least when we broke our rules we did so knowingly and had a chance of fixing them at a later date.

What I wish I had made at bbc.co.uk if I stayed

In many ways I’ve been very lucky at the BBC I’ve helped make some cool stuff – well stuff I’m proud of. But since I’ve decided to leave I’ve started to wonder what else I would have like to have made, if I had stayed at the BBC.

There’s a bit of a health warning however, these are just ideas. I’ve no real idea if they are that practical and they almost certainly don’t fit into the current strategy.

Get Excited and make things
Get Excited and make things

My ideas…

Lab UK meets so you want to be a scientist

Lab UK is the part of the BBC’s website where you can participate in scientific experiments. They’ve done some cool stuff – including Brain Test Britain which had 67,000 people sign up and resulted in a paper in Nature [pdf].

The various experiments are tied into TV programmes and this is really important because it helps generate interest and get the number of participants required to make the experiment work. However, it also means that the experiments are designed in advance, by the scientists, and the public’s role is one of test subject.

The experiments do help build knowledge but they probably don’t help people understand science.

So here’s the idea – a bit like Radio 4’s “So You Want To Be A Scientist” the process would start with people suggesting ideas, questions they would like answering, the site would need to provide sufficient support to help people refine their ideas. It might even use material from the BBC archive to help explain some of the basics but at it’s heart it would be a collaborative process.

The ideas would then be voted on and the most popular would then be taken forward. With the help of scientists the experiment would be designed and build and carried out on the Lab UK platform, giving these amateur experiments potential access to a huge audience.

The process would be a rolling series of experiments designed and carried out by the public.

History through the eyes of the BBC

The BBC makes a lot of programmes about history – but much more significantly it has been part of or at least recorded a lot of our more recent history.

So rather than making a history site about the Romans, the Victorians or whatever I would use the BBC archive to tell the history of the world as seen through the eyes of the BBC.

Combining news stories, clips from programmes (broadcast or not), music and photographs the site would tell the story of the world since 18 October 1922.

The site would chart the major political, scientific, sporting, cultural and technological events since 1922 but also the minor events – the ones that remind us of our own past.

The site would provide a page for every day, month, year and decade since the BBC came into existence as well as pages for the people, organisations and events the BBC has featured in that time.

Basically a URI for everything the BBC has recorded in the last 90 odd years.

The site would also allow members of the public to add their thoughts and memories (shared under whatever licensing terms they wish) to enrich it further to create a digital public space for the UK.

One BBC nature

A few weeks ago we merged Wildlife Finder into the nature site and launched a new blog – and today we’ve taken the final step and brought Earth News into the fold to create a consolidated BBC nature site.

From a certain perspective this doesn’t represent a big change – after all we’re still publishing exclusive natural history news stories, video collections and video clips and information about: animals, plants, habitats (and the ancient earth’s habitats, such as Snowball Earth), adaptations & behaviours, places and ecozones, the geological time periods when they lived, the major mass extinction events, including the one that killed the dinosaurs, in fact lots of information on the history of life on earth and the fossil record. We even have a page about fish – and they don’t really exist!

However, from another perspective this is a really big change. It’s a big change because we’ve (hopefully) made everything so much simpler.

Screen grab of the new BBC nature site - features section

We’ve made it simpler by bringing everything together into one site and removed the various sub brands – if you love nature and natural history everything is now in one place: news stories, video clips from the archive, opinion pieces and more.

Bringing everything together has also allowed us to make a few additional changes which should help us more easily publish the content.

In addition to natural history news we have a features section where we can bring together articles and photo galleries (like this one) and a new blog Wonder Monkey written by Matt Walker. Matt has written a few posts so far including this one on the oddball midge that shouldn’t exist.

I really hope you like it. It represents the culmination of two years of work, during which time we launched and evolved both the site itself and the editorial proposition – there now are c.3,000 clips available online (many of which are available worldwide) about almost 900 animals (both prehistoric and living), 50 plants etc.

And of course wildlife data is for machines too.

However, after two years of development this represents the last major release, for a while at least. The site will continue to grow because we are continuing to create great new content as well as digging out the best bits from the archive – like this video collection looking back at David Attenborough’s Madagascar (starting with Zoo Quest 50 years ago). But there won’t be any major new features for a while, not that that’s a major problem – the site should offer a rich experience with amazing content.

As I said yesterday, I’m very proud of what we’ve produced and if I can marshal my thoughts I’ll try and write a post or two about how we went about building the site and the lessons I learnt on the way, until then enjoy the site.

Leaving the BBC

Leaving the BBC

After almost five years this will be my last month at the BBC.

The BBC has been a great place to work – I’ve worked with some amazing people, helped deliver some of the best work of my career and had the opportunity to speak at conferences around the world, including (amazingly) at the Web’s 20th birthday celebrations in CERN.

The BBC can certainly be a challenging place to work but I’m very grateful to Dan Hill and Matt Wood for offering me a job in the first place. I just hope I’ve not let them down because for every challenge, gripe and frustration there have also been opportunities to learn new things, work with brilliant people and help deliver great stuff that has, I think, had an positive impact on what the BBC does online.

So what have I been up to since I’ve been here?

The first project I worked on was /programmes a site that means that every programme the BBC broadcasts now has a web presence – one that both humans and machines can enjoy. The site is sometimes criticized as being a card catalogue of BBC programme metadata but its worth remembering that until the site launched the vast majority of programmes had no URI, had no webpage of any kind; /programmes changed that at a stroke. It was also the first truly dynamic web site on bbc.co.uk and whatever people might say about the aesthetics the site has the prettiest URIs of any site I know (something to thanks Michael Smethurst for).

The music site was my other project while in the FM&T bit of Audio & Music. Building on Musicbrainz the idea was to create a rich graph, linking music programmes with artist pages (available as HTML and RDF etc.) via ‘clickable tracklistings‘.

After a couple of years I left Audio & Music and joined ‘BBC Vision’ – the bit of the the BBC that does the telly – and took on a project known internally as ‘BBC Earth‘. And pretty much tried to replicate the music work but for natural history content.

I say I tried to replicate the music work that’s not really true, or rather its only true to a point. The core underlying concepts where the same, but the manifestation is quite different. For starters we sought to digitise and make available the TV archive but we also created original content – this broke down into exclusive natural history news stories, stories from TV and Radio production teams on location and, curated video collections.

I wanted the nature site to help people discover, explore and understand the natural world through the BBC’s content, I hope we’ve achieve that to some extent. Personally, and I know I’m biased, I think the site is brilliant and one of the best looking and useful semantic web sites around (we publish the data as RDF).

The credit for the site, however, should go to the team that actually made it. I was lucky, the core of the team has remained on the project throughout its development and I’m indebted to those, more talented than me, for making it what it is.

As I’ve said, I think the site is brilliant and I think the editorial, technical and design knowledge and skills of the team shine through, the site is theirs not mine.

There’s much I could write about this work – but I should really do it a bit more justice than the space available here and so I’ll save what I have to say for another post. Also there’s one last thing to push live on the site, to round off its development and it feels wrong to preempt that.

So what now? Well I’m joining Nature Publishing Group as Head of Platform for nature.com. As a failed scientist I’m very excited by the opportunities – Nature is the leading weekly, international scientific journal with a mission to:

Helping achieve that mission on the Web is a really exciting prospect and I hope the next five years prove as productive as the last. Wish me luck.

BBC iPhone apps? I think there’s a better way

Last month the BBC announced that it would launch a limited number of mobile app, starting with News and Sport  and then possibly an iPlayer app. Unsurprisingly the NPA promptly complained that the BBC would “damage the nascent market“, and now the BBC Trust as said that it wants to review the plan and that means a delay.

Well I don’t know about whether such a move by the BBC would have an impact on the market or not (although I agree with Martin, I think it was inevitable that the Trust has would review the plans), but I do think the BBC could tackle the problem in a different, more open way.

BBC News on mobile

There’s a lot of hype and hyperbole around mobile apps – and in some ways you can understand why, lots are downloaded and some folks are making money from them but I’m not so sure it’s going to last. I suspect that mobile apps are successful for a few reasons:

  1. They are hooked into a big marketing push. Apple et al. are all publicising their stores on your handset on the telly, on posters and in papers.
  2. The app stores are targeted and people know where to look, the Web could be The Store (as it’s been for other things) but that’s not how regular folks appear to see software nor do they want to dig about for what to install.
  3. The Web (mostly) only works when you’re online, apps (mostly) work offline too.
  4. Some stuff can only be built as a native app (rather than via the web), probably.

But as phones expose more of their API to the browser, as HTML 5 with its support for offline browsing and other goodies becomes adopted and, as libraries and support become available so the technical and user experience barriers start to become less relevant — it may once again be universally seen as sensible to develop web apps. Of course either the fear of being locked in or being locked out of the relationship with their customer might kick companies along a bit too.

So in the near future we should be able to build web apps every bit as good as mobile apps? Yes, but I would go further: for most of the things the BBC wants to do, the technology is already good enough. And with a web focused mind set you can start to invest in the sorts of things you can only do server side — just look at the sorts of things Google are building: word processors, voice communications, email clients, image recognition, maps etc. I think it’s better to embrace the future than play catch-up with the near past.

But what if I’m wrong and mobile apps are the future of content delivery? Well the BBC could still take a different approach – one where it licensed its content in such a way that others could build apps with its content. Of course, unless things changed, the app would need to be non-commercial and the use of the BBC logo and brand would be protected. Of course the non-commercial aspect might be reviewed under certain circumstances — indeed the BBC already licenses content to third parties both outside and inside the UK via its commercial arm BBC Worldwide, why not online? Although I can’t see any circumstance under which the BBC would allow use of its brand and logo since this is central to protecting its reputation, to avoid this sort of thing.

If the BBC did license its content in such as way as to allow others to build stuff then we might see all sorts of interesting innovation on all sorts of different devices and not just mobiles. Perhaps I’m missing something but I don’t see why the BBC needs to control the entire distribution chain, from encoding to eye balls, when distributing content over IP but not when broadcasting to your TV or radio. The BBC doesn’t make its own televisions nor radios instead it lets the market manage that bit, why not encourage the same sort of thing on the web?

Some thoughts on curation – adding context and telling stories

Just over two years ago I wrote a post about the importance of the resource and the URL — and I still stand by what I said there: the core of a website should be the resource and its URL. And if those resources describe real world things and they are linked together in the way people think about the world then you can navigate the site by hopping from resource to resource in an intuitive fashion. But I think I missed something important in that post — the role of curation, the role of storytelling.

When we started work on Wildlife Finder we designed the site around the core concepts that we knew people cared about and those that we had content about i.e. species, their habitats and adaptations, and we’ve been publishing resources about those concepts since last September. We’ve since published the model (Wildlife Ontology) describing how those concepts relate together. I’ve talked about this work as providing us with the Lego bricks because I also realised that we needed to use those Lego bricks to build stories, to help guide people through the content. Our first foray into online story telling with these Lego bricks are the Collections.

Collections allow us to curate a set of resources – to group and sequence clips and other resources to tell stories like the plight of the tiger or the years work of the BBC’s natural history unit. Silver Oliver has recently written about why he thinks this approach is important, why curation in a metadata driven information architecture — it’s a very good post — you should read it. But I thought I would share a bit about the intellectual framework behind how I think of this stuff. As with most of my ideas it’s not my ideas but one I’ve borrowed from someone brighter than me, in this case Nathan Shedroff who proposed a framework to think about how to build Lego bricks and then things with those bricks. A framework I’ve been using for few years now.

Wildlife Finder provides information by repackaging data from elsewhere – by organising programme clips, news stories etc. around natural history resources and concepts. This is good (I hope) because it provides useful additional context; but it’s not the whole story. In Shedroff’s model this process creates information — by adding context to data by presenting and organising it in a new, useful way. This is really what encyclopedias provide — structured information presented and organised in useful ways.  The next step is to take this information and build stories with it to build knowledge and facilitate conversations.

As I say, with Wildlife Finder, we have started to tell stories by localising the information into Collections, but of course, now we have a unified domain model (which links together programmes and concepts within the natural world) there are other ways in which we can add context and build knowledge on top these resources — in addition to collections. There are lots of ways we can create new experiences, but as you can see from the diagram above, we don’t hold a monopoly in terms of story telling — those that consume the information, our audiences and ‘users’ could also build stories. Although the BBC doesn’t really let people build their own stories other sites and organisations do, notably  Flickr who have a series of interesting approaches to let its users add context to photos through Groups, Galleries, Sets and Collections.
https://derivadow.com/category/organisations/
derivadow.com
crawl=CC-MAIN-2020-24
128187
application/http; msgtype=response
sha1:J66DIF6CVKN7F6RRMWRYZEUADPUC47AE
<urn:uuid:a1e50b8b-c0dc-477f-a1da-926bd72146b6>
2020-06-01T19:31:47Z
192.0.78.25
text/html
sha1:NVF5R2SWW6WPVIRGMWDRJUQ76OBMJQKZ
<urn:uuid:b738f648-eff4-4059-b499-437e732206e9>
https://derivadow.com/category/organisations/
response
<urn:uuid:cf4484c6-04d4-417c-a715-a5d12d8699f3>
isPartOf: CC-MAIN-2020-24
publisher: Common Crawl
description: Wide crawl of the web for May/June 2020
operator: Common Crawl Admin (info@commoncrawl.org)
hostname: ip-10-67-67-84.ec2.internal
software: Apache Nutch 1.16 (modified, https://github.com/commoncrawl/nutch/)
robots: checked via crawler-commons 1.1-SNAPSHOT (https://github.com/crawler-commons/crawler-commons)
format: WARC File Format 1.1
conformsTo: http://iipc.github.io/warc-specifications/specifications/warc-format/warc-1.1/