After a month of running @guardiantagbot as a beta test, today we’ve decided to put our TagBot to rest while we review and analyse our initial results, and see whether we’d like to continue the experiment in its current or different form.
We do have some some learnings we can share with you however. First of all, we now know that even the smallest of Twitter experiments can get a lot of attention. In the first few days after launch, @guardiantagbot got initial coverage from sources like NiemanLab and Mashable, perhaps more than you’d expect for a humble beta robot.
This helped grow interest in our API and has led to some interesting conversations, but it also helped boost Tagbot’s use. We’re still analysing the queries overall, but our experiment does seem to have attracted an interesting mix of social and search usage. The results pages sent out to individuals were viewed on average by 7.5 people each.
One search results page about the Downton Abbey TV show, for example, was viewed an amazing 424 times, with results pages about the Murdochs and phone hacking closely following behind.
In terms of query subjects, perhaps unsurprisingly, the majority of tweets sent to TagBot were related to major news stories happening at that time, such as the Eurozone and Libya. International news was also popular, perhaps reflecting Twitter’s international audience.
We also got a lot of football fans asking for their favourite UK football team, tech fans after the latest tech and mobile news, and foodies after recipes. @BergerofLondon went one step further and asking for a “recipe for disaster”. Tagbot sent back this article on …cider making.
Generally, @guardiantagbot did seem to be returning decent results – out of the 25% of users that gave us feedback on their result, 75% clicked on “Goodbot” suggesting they got a result which at least in part answered their query. Even taking away the goodwill factor, that’s not bad considering that our keywords are perhaps not always as well targeted for search queries as they might be for our own news classification.
For example, users often search for football teams using their slang names eg “Pompey” or place FC at the end, when our keywords generally don’t include the FC part. It’s also revealed some holes in terms of our tagging service – we got a surprising number of users asking what the weather is in their location. Although the Guardian does have a weather page, it isn’t tagged for discovery.
Tagbot was also sadly unable to properly respond to “Why is Jeremy Clarkson?”, “What’s the point of Belgium?” and the plaintive “Does she love me?” – but did get a Goodbot rating for returning pages for “Is David Cameron a lizard” and “Robot apocalypse”.
We hope you’ve enjoyed this beta test – thanks to everyone who used it. We fully expect to feed the lessons into social media services in the future.