14Nov

Matt McDonnell and the mother of all gateways

posted by Steve Bowbrick

This is a pretty long post based on a chat with the BBC’s head of search. If you’re interested in search, though, I reckon it’s worth ploughing through. I really learnt a lot from talking to Matt McDonnell: he has a very interesting and very important job working right at the heart of the future BBC.

Search as a gateway to everything

Matt didn’t want me to call him ‘head of search’. It’s not his job title and it sounded like “hagiography” to him. Still, he is in charge of search and I reckon he has a reasonable claim to the title ‘most important person at the BBC’ right now. I’m pretty sure the BBC org chart doesn’t reflect that, though, and I’m also sure that there are plenty of BBC executives who’ve never heard of him.

As the old ways into BBC content fade, search becomes more important. It’s a reasonable assumption that search will be the primary gateway to all BBC content within a few years, including the stuff that goes out on the linear channels (BBC1, BBC2, Radio 1, Radio 2 etc.). The channels themselves are already losing their gateway function. Viewers and listeners are much less likely to use a channel as a way into an evening’s viewing than they were in the pre-digital era. Themes, personalities, strong programme brands: all are becoming more important than channels. This, for instance, is one of the reasons for the BBC’s growing investment in top talent: Jonathan Ross may be an expensive presenter but he’s pretty economical when considered as a gateway to BBC content (at least when he’s not on suspension for being an arse).

On iPlayer, for instance, the channels already play a reduced part in programme selection. Programmes are still organised by channel but that’s an arbitrary echo of the BBC’s org chart: there’s no good reason to classify television content by linear channel once it’s online but nervous channel controllers insist on superimposing the channel name on shows that go out on iPlayer: they fear that their carefully commissioned and scheduled content has been stirred into an undifferentiated soup of shows and that the investment they’ve made in their channel’s brand will be wasted. But users conditioned by exposure to YouTube and MySpace and Google probably don’t even see the channel ident.

Likewise, the BBC’s homepage may be one of the most important in Britain but a growing proportion of users don’t use it to locate content: they find the stuff they want via a search, either using the site’s search field or by searching at Google or Yahoo or ask.com. Sitting next to Matt at his desk in White City it was revealing to watch his own navigation habits: every page he showed me was located via a search, even pages at his own site—no bookmarks, no browsing and no typing in the address field. When search is good enough it replaces all three.

Matt’s just coming to the end of a big programme of work that will sharply reduce the emphasis on web search at bbc.co.uk. The fact is that the BBC’s early ambition to ‘own’ UK web search has probably held the Corporation back from implementing really good site search and useful content structure so this is a big relief. And here’s a truly fascinating aside: when you search the web at bbc.co.uk, the top three results are often sites selected by BBC editors (here’s an example: asthma). Until recently these results were labelled as such (something like ‘best links’) but Matt’s team just removed the label.

The high quality, editor-selected results are still there, right at the top of the list but since the label was removed the click-through rate for these links has actually gone up substantially! Users weren’t clicking on the hand-selected links because they were suspicious that they might be sponsored links. They had learnt from exposure to Google and other search engines that the ‘special’ links at the top of the list are qualitatively different from the others and were avoiding them for that reason. Fascinating and counter-intuitive.

Topics

Another major initiative from the search team involves the creation of ‘topics’ pages: useful pages of information assembled from BBC sources and elsewhere about specific subjects. Topics is still in beta: you can check out the handful of hand-coded topics pages here. Many more are planned and what’s fascinating is that about 95% of them will be automatically generated.

This is all pretty hardcore semantic web stuff. The BBC topics starts by crawling Wikipedia daily and pulling in new pages created since the last visit. Wikipedia provides authority here: confirming that a topic is real (not that it’s relevant or useful: just that it exists) and doing ‘disambiguation’—sorting out the 19 different places called Rome, for instance. If the system finds a new entry at Wikipedia it then searches the BBC for information that’s similar to the Wikipedia entry—using Wikipedia’s text as a ‘training document’. If it finds none then no page is created: the topic is obviously not of sufficient relevance. If it finds content—news stories, programme pages, whatever—it generates a new topic page. John Muth, one of the developers working on the system, says he expects there to be tens of thousands of topic pages pretty soon after launch.

The result will be thousands of new pages, an extraordinarily rich information asset that exposes a lot of authoritative BBC content that would otherwise have been neglected or even lost. This is going to be a real public service win and – let’s face it – a much better idea than trying to make bbc.co.uk a destination for web search. Live syndication of Wikipedia content will also mean that the topic pages improve as Wikipedia does (although pages needn’t use Wikipedia content). Further (the semantic web is a mighty rich and interwoven thing), people will be able to syndicate the BBC topics pages for their own use: they will be published under a Creative Commons licence like the hundreds of thousands of artist pages in the /music hierarchy. Tools will be provided and schools and libraries or even businesses will be able to build useful information resources of their own by tapping into this clever blend of content from the BBC and the commons.

And video too

This all gets even more exciting when you add the potential to search the hundreds of thousands of hours of video produced by the BBC annually. Matt’s team is currently testing a system that analyses video files, creating a transcript that can then be indexed and added to the web of content on the topic pages. The transcript can also be used to ‘chapterise’ the video itself so users can jump to a particular part of the video based on the transcript.

Let’s face it: once the BBC’s audio and video content—the Corporation’s crown jewels obviously—has been opened up to search there’s really no further argument: It’s game over. All other gateways to the BBC’s content will be officially obsolete and search will have won. Maybe I should keep my mouth shut.

33 comments

Comments so far.

  1. Posted by Nick Reynolds (BBC) on Friday 14th November

    Hmmm well search will only have “won” for those who have the internet… and still more people have a TV or radio than an internet connection…

  2. Posted by Nick Reynolds (BBC) on Friday 14th November

    What if broadband penetration sticks where it is rather than rising to the same reach as TV?

    Channels are more than gateways. They are also machines to delivering content to particular audiences.

  3. Posted by Steve Bowbrick on Saturday 15th November

    Hmmm yourself, Nick! What makes you think search as a way of discovering content is confined to a computer or a broadband connection? Video delivered in an IP environment—maybe via Canvas (the BBC’s tentative open IPTV platform) or even via cable/satellite—can be chapterised, indexed and searched. Once search reaches your telly the channel model is truly doomed.

    Imagine a Freeview/Freesat IP digibox or a DAB/WiFi radio getting its video/audio signal off the air and chapter info, transcripts and metadata via an IP connection. Content discovery might still make use of ‘channels’ but I’ll bet you a tenner they’re going to seem a bit primitive and unhelpful alongside search and RSS feeds and a really slick second-generation web-like EPG (presumably iPlayer). Like I said, game over!

  4. Posted by Christopher Sleight on Saturday 15th November

    This sounds pretty exciting, and knowing Matt of old I’m sure he’s doing a good job of it. But the current version of BBC search has definitely been a step back – at least as far as news and sport is concerned.

    It has some baffling characteristics (like changing your search term to what it thinks you want – try searching for GAVIN, for example). And is frankly infuriating when you’re trying to search for anything on the news site that’s more than a few months old.

    There’s no advanced search option and I really can’t work out the Boolean behaviour.

    I’m mainly using it as a journalist on the BBC News website, so maybe I’m not after what your average searcher wants, but I just can’t work the thing.

    Audio/Video search on the other hand seems to work brilliantly, and I’m glad the whole web search thing was dropped. I’m sure it’ll improve but for the moment I’d rather use Google’s advanced search.

  5. Posted by Nick Reynolds (BBC) on Saturday 15th November

    Diagree with you Steve.

    Channels have worked well so far as a way of delivering content to audiences, as brands that mean something and for the BBC as a underlying philosophy.

    BBC ONE and Radio 4 (even Channel 4) mean something to people – what they have to do is reinvent themselves as communities of interest.

    It’s not an either/or choice. They can live alongside search and aggregation.

  6. Posted by Matthew McDonnell on Monday 17th November

    Hi Chris

    We are working on improvements to results for news-type queries and on advanced search functionality. Expect changes soon. Also i agree the “gavin” correction is over-aggressive and will get that changed.

    Best,

    Matt

  7. Posted by Rupert on Monday 17th November

    Great article. I agree that search will be *as* important as channels are now… but I’m not sure that it’s a zero sum thing… Channels still have an important role for media aggregation – filtering programmes by type. I use Google for pretty much everything – even finding old posts on my own site – but I also use channels for media. I subscribe to certain video podcasts or visit certain video blogs, knowing that I’m going to get a certain type of content there. Same with Radio 4. I have the Radio 4 iPlayer Most Recent Programmes bookmarked so that I can browse all the programmes that have played on Radio 4. I don’t want Radio 1 and 2 and 3 and 5 and 6 and 7 programmes cluttering up the same page. Speechification – same thing – I think of it as a channel that aggregates good speech radio, and I trust your judgement as channel controllers more than I trust Google to give me good speech radio picks. So my bookmarks are channels. They help me browse & find something good to listen to or watch quickly when I’m not sure what I want.

  8. Posted by Steve Bowbrick on Wednesday 19th November

    Bookmarks = channels. Like that Rupert. I’ll be making use of that one…

  9. Posted by YoungAmerican on Wednesday 19th November

    Agree with many of the comments here. Channels are really just another type of editorial proposition – like the Times or the Daily Mail or the Court Circular or what have you. Editorial propositions – content made for and tailored to meet the perceived requirements of a specific audience won’t disappear, and so nor will channels, newspapers etc. They’ll continue to change and evolve alongside other forms of content discovey, just as they always have. Search may well develop in different ways than we expect – wasn’t the telephone intended to be a means of enabling the audience at home to listen in to liv musical performances? Rather than the peer to peer communication tool it actually became.

  10. Posted by Steve on Thursday 20th November

    Where has all the user generated content disappeared to??? There’s talk of opening stuff up to the communities but the communities input is hidden.

  11. Posted by Steve Bowbrick on Thursday 20th November

    @Steve I think UGC is implicit to Matt’s work. He’s building a search architecture that’ll index, link and present content with no particular regard to who made it. Having said that, I think I could have made more of it in this piece. How, for instance, will UGC be handled on topic pages? Will links out to third party content (made by you and me, for instance) be permitted? Incidentally, I think might find this post by Max Gadney over the BBC Internet blog interesting from this point of view.

  12. Posted by BBC Common Platform: The BBC should engage with Wikipedia on Friday 9th January

    [...] BBC already makes use of Wikipedia’s content and ontology (see this entry about search at the BBC) but the Corporation should also directly support Wikipedia by encouraging [...]

  13. Posted by The web as a CMS « Derivadow.com on Tuesday 13th January

    [...] want to edit it we edit the web improving those services and those provided by the BBC (including search). Possibly related posts: (automatically generated)BBC Music (beta)Musical and social networking, [...]

  14. Posted by Neo on Thursday 29th April

    cincinnati prime outlets monroe ohio stores
    selena featured on primer impacto
    lauxanh ó
    stuntworldusa cars
    gotti blue pits
    lolitas fuck
    olsat sample
    women braless
    roscoes chicken and waffles recipe
    free unlock codes for polar golf
    cheats on aq com
    the 13th pageant a p us history
    how to convert kilo cups
    long dog dicks
    shemale kris bell
    rebekkah ross naked
    bullet calibers
    ls land pirates
    sears scratch and dent appliances norfolk
    videos de pornografia jovensitas virjenes gratis

  15. Posted by Diesel on Friday 30th April

    asylums in southern california
    phim vietchoi com
    wwe diva kelley
    free printable camo stencils online
    meijers toledo
    ac delco lawn tractor oil filter application
    milliliter to ounce conversion
    glock 21 select fire conversion dvd
    tehachapi state prison
    myanmar love story
    1942 british 303 rifle prices
    cowboy pitbulls
    disney employee portal sign in
    gx xxx
    phequa xem vietnam
    kristy alley swimsuit celebrity
    meaning of hand signals thrown by rappers
    ohio stna registry
    convert dl to cups
    cornrows styles for girls

  16. Posted by Kir on Tuesday 29th June

    armi f lli tanfoglio 22 revolver
    ac delco pf53 cross reference
    adjusting salomon bindings
    american canine association umbilical hernia repair
    350 chevy 4 bbl carb on ebay
    chuck zettl
    ammi ka blouse
    045 fl oz to cc
    actress diane canon nude
    80 oz equals how many milliliters
    3d ultrasound picture ear 3rd trimester
    2000 dodge 3 belt diagram
    al wissam alibaba
    1000000 pillsbury winner
    alextremo aztecaamerica com
    1987 elcamino free wiring diagram downloads
    all hail the heartbreaker spill canvas tabs
    1911 colt markings and id
    5th grade inview test example
    89 chevy 305 tbi to carb

  17. Posted by Kir on Friday 6th August

    viagra cialis levitra buy viagra
    buy cialis
    cialis price online
    buy cialis grand rapids michigan
    subaction showcomments cialis smile online
    information on generic cialis pills
    order cialis online
    cheap cialis sale online
    discount generic cialis pills online
    cialis pills com
    cialis online discount
    buy cialis online australia
    buy cialis without a perscription
    viagra cialis prescription online
    cialis daily pills
    quick forum readtopic cialis answer online
    cialis 20 mg 10 pills
    cialis online prescription

  18. Posted by PDA’s Newsbucket | Richard Hartley on Friday 7th October

    [...] Gazette• The YouTube Presidency >> washingtonpost• Matt McDonnell and BBC search >> BBC Common Platform• Online ad growth grinds to a halt >> TechCrunch• Twitterank wasn't a phishing scam – but [...]

  19. Posted by Hausfinanzierung Rechner on Thursday 15th December

    Hi would you mind letting me know which hosting company you’re utilizing? I’ve loaded your blog in 3 completely different browsers and I must say this blog loads a lot quicker then most. Can you recommend a good hosting provider at a honest price? Thanks, I appreciate it! Hausfinanzierung

  20. Posted by venezuela on Sunday 14th October

    I pay a quick visit daily a few sites and information sites to read
    articles or reviews, except this webpage offers
    quality based posts.

  21. Posted by Fay on Sunday 11th November

    I value the knowledge on your websites. Thnx!

  22. Posted by http://tinyurl.com/anthlambe15828 on Friday 11th January

    Precisely how long did it take u to write “BBC Common Platform: Matt McDonnell and the mother of all gateways”?

    It provides a great deal of really good information.
    Thanks a lot ,Blythe

  23. Posted by Pharmd296 on Wednesday 30th January

    Hello! ekegegb interesting ekegegb site! I’m really like it! Very, very ekegegb good!

  24. Posted by air jordan 13 retro bred on Friday 31st May

    The big-bottomed actress routinely rakes in $250 million in annual sales thanks to the success of her bottled scents. Better yet, ask Michael Jordan, Enrique Iglesias and Tiger Woods. In case a great discount may be the shoes, there are only a couple of possible situations. Lots of people tend to be avid enthusiasts of jordan shoes while using legacy he designed, and, they are simple greatest shoes both former as well as present.! Only people who are ardent lovers with the brand bother looking to get them.When you are aware that you grab a snack for quick energy when you’re tired, you can decide

  25. Posted by Cheese And Crackers on Sunday 30th June

    Hey there! Someone in my Myspace group shared this website with us
    so I came to give it a look. I’m definitely loving the information. I’m bookmarking
    and will be tweeting this to my followers! Wonderful blog and
    brilliant style and design.

  26. Posted by Cash Advance Online on Saturday 27th July

    obliging source web page

  27. Posted by Patty on Tuesday 8th October

    Kerry called Moaz al-Khatib, the leader of the Syrian coalition, the 1285 muscle review debate is purely because the results are excellent.
    You cannot sacrifice your health in multiple 1285 muscle review ways.
    You go out, you’ve got to realize, like, carbs. Yet puffy desperate glands are
    regular harms among the recurrent user of testosterone booster.
    We have him on camera saying that’s easy.

    Also visit my web page – 1285 muscle reviews (Patty)

  28. Posted by easy payday loans Cleveland on Thursday 10th October

    Press the Accept payday loans button and Voila! So, be sure to include any expected funds from your family.

    my web blog :: easy payday loans Cleveland

  29. Posted by Temeka on Thursday 17th October

    Superb post however I was wanting to know if you could
    write a litte more on this subject? I’d be very grateful if
    you could elaborate a little bit more. Kudos!

    Take a look at my webpage … neon signs (Temeka)

  30. Posted by empower network blog beast on Saturday 2nd November

    I believe what you said was very reasonable. But, what about
    this? what if you wrote a catchier title? I am not suggesting your information is not solid, however what if
    you added a title to maybe get a person’s attention?
    I mean BBC Common Platform: Matt McDonnell and the mother of all gateways is a little boring.
    You should glance at Yahoo’s front page and see how they
    create news headlines to get viewers to click. You might
    add a related video or a pic or two to get
    people interested about what you’ve got to say.
    In my opinion, it could make your posts a little livelier.

    Feel free to surf to my web blog; empower network blog beast

  31. Posted by lawyer on Wednesday 18th December

    John Lawrence Allen’s only job is to get your money back on investor fraud cases – http://commonplatform.co.uk/index.php/2008/11/14/matt-mcdonnell-and-search-as-a-gateway-to-the-bbc/

  32. Posted by viagra on Wednesday 9th April

    We’re a group of volunteers and starting a new scheme in our community.

  33. Posted by cialis on Tuesday 15th April

    Thanksamundo for the post.Really thank you! Awesome.

Leave a reply