While operators continue to sell to their end customers based on network coverage, behind the scenes monitoring of performance has got a lot more complex. Mobile Europe looks at how customer experience monitoring can benefit an operator’s total business case.
Recently, T-Mobile invited journalists to its Oxford Street store to witness a new service it was offering its customers. The operator was lanching a service which would let potential customers get a view of the level of network coverage within any given postal code.
Obviously T-Mobile’s aim was to provide proof to customers of the way in which it now monitors and improves network coverage, and use that as a sales tool at the actual point of sale.
This emphasis on network coverage is no doubt welcome news for those interested to make sure that they don’t buy a phone that turns out to be in a poor area of coverage at their home or work. But as a tool, it seems quite remote from the current rhetoric of the companies supplying the mobile industy with monitoring and measurement tools, which is that network benchmarking has gone far beyond jsut benchmarking the, er, network, and into the realm of measuring how individual services, applications and even devices are performing.
In short, some providers are now saying that just saying a cell has good coverage is not enough — wouldn’t it be useful to provide real statistics on which services work well on which devices, even if it may not be the kind of information operators are yet willing to share with potential customers on the high street?
One of the players which is taking a different look at how to monitor and measure end user customer experience,is Keynote Systems. Keynote’s Mobile Device Perspective 2.0 provides a hosted solution for mobile operators, content and service providers and infrastructure hosts that allow them to analyse their mobile content and services from the perspective of their subscribers.
What Keynote does is use real life devices in a test environment on an actual network. It can remotely command the mobile to carry out certain functions, and then record the results.
Haran Sold, vp and managing director of Keynote Systems, says the benefits affect everyone trying to develop services.
“If you take the example of a small game developer in the UK and India with an Italian operator client. The developer can put a real device on TIM’s network and access that device through a web interface. That opens the door to see how its content performs. Not only that but they can instruct the device to go through the process of buying and downloading the game, and can measure every step of that transaction.”
In the USA, Sold says, one operator wants to work with Keynote to benchmark how competitors are doing in MMS delivery, and to see how it is performing in different markets.
And as the mobile industry opens up to a host of content providers and players from the fixed internet world, Sold says such tools will give those players the ability to see how their content is experienced on any channel they put the content through.
Another company that is making a play to monitor customer experience, rather than mere network performance, is Argogroup.
James Pearce, chief operating officer of Argogroup, says there has been a change of thinking in the practice and purpose of monitoring.
“Traditionally, passive monitoring alone has allowed operators to observe bulk traffic through their networks and deduce what is working well, and what is not, in their network. But this never shows the whole picture, particularly what the users’ experiences actually were.
“So carriers are increasingly turning to active test. This alone is not a panacea, but provides an ‘outside in’ view of the network and how the users see it. It can ensure that synthetic, injected transactions succeed, and then extrapolate overall quality from that.”
This information can then be used to analyse whether the prioritisation of traffic is well-tuned.
“But only some carriers seem to actually make strategic, informed decisions about the complex relationship between traffic types, revenue, and subscriber value – in order to pro-actively get that prioritisation right,” says Pearce.
One of the issues Pearce identifies is that operators can be excellent at moving bytes around, but poor at understanding the demands of wide varieties of content and media. For example, Argogroup used the World Cup to monitor how operators’ content services were performing, looking at video streaming, SMS alerts and other channels.
Pearce says that performance was “very patchy, with problems originating and integrating content (despite being able to move 160 characters out to a phone in a matter of seconds, alerts frequently arrived many minutes after the goals were scored). But it also demonstrated a degree of media un-savviness. At the time of kick off for the first match, one operator’s mobile portal was displaying a Mr Men promotion in place of anything football related.”
Pearce says that the Web, and especially Web 2.0 continues to show how it should be done.
“Alerts from web-based providers, such as from FIFA via Yahoo, arrived in a far more timely manner across all networks – and happened to be free,” he points out.
The idea of such an approach is clearly to help operators reduce lost revenue, by being able to see exactly what a customer mught be experiencing and doing something about it. But as operators face having to open up their networks to other providers, becoming channel partners as much as pure network operators, such information will give competitive advantage in being able to work with the best partners, and make it as easy as possible for applications developers to get a real view into how their product will work on the network.
“The capability for a developer in Seattle to download, test, and measure an application’s performance in real-time over a live network in London, Seoul, or Tokyo is enormously beneficial and cost-effective,” says Chetan Sharma, a communications consultant, identfying the real opportunity for what we might call benchmarking 2.0.