Telenor is the latest of several telcos from around the world to have an emergency calls outage
Norway’s National Communications Authority (Nkom) will now supervise Telenor after it was impossible to call the police’s emergency number 112 for three hours on Thursday evening last week.
The police reported on Thursday 29 August at 21.00 to the media that the police districts could not receive messages from the public on its emergency number 112. Telenor says the error, which also affected the mobile customers of Telia and Ice, is due to a technical error on its part. In addition to being the police’s emergency number, 112 is also the international emergency number that is communicated to everyone who comes to Norway from abroad.
Among other things, Nkom is responsible for ensuring that the operators have adequate security in their mobile networks. Now it has been decided to open supervision based on the incident.
“We take it very seriously that such an error occurs which affects the public’s ability to call emergency numbers. We will therefore go through the course of events and check whether requirements for proper security have been followed. We must do what we can together with the actors to prevent a similar incident from happening again,” said Nkom security director Svein Scheie (above).
Critical condition
Nkom expects that the inspection report will be available at the end of October 2024. Telenor is the latest of a string of telcos around the world, typically incumbents, that have had outages, and each has resulted in investigations that have found process lacking in some way, or a perfect storm of incidents exacerbating an outage such as what happened in Australia this year.
In July Ofcom fined BT £17.5 million for being “ill-prepared to respond to a catastrophic failure of its emergency call handling service” last summer. That network disruption affected 14,000 emergency calls in June 2023 and lasted 10.5 hours. BT connects 999 and 112 calls in the UK and provides relay services for deaf and speech-impaired people.
During the first hour of that outage in 2023, BT’s emergency call handling system was disrupted by what was later found to be a configuration error in a file on its server. This resulted in call handling agents’ systems restarting as soon as a call was received; agents being logged out of the system; calls being disconnected or dropped upon transfer to the emergency authorities; and calls being put back in the queue.
BT was initially unable to determine the issue’s cause and tried to switch to its disaster recovery platform. The first attempt to switch to the disaster recovery platform was unsuccessful due to human error. This was a result of instructions being poorly documented, and the team being unfamiliar with the process. The incident grew from affecting some calls to a total outage of the system.
Ofcom found that BT did not have sufficient warning systems in place for when this kind of incident occurs, nor did it have adequate procedures for promptly assessing the severity, impact and likely cause of any such incident or for identifying mitigating actions. It also found that BT’s disaster recovery platform had insufficient capacity and functionality to deal with a level of demand that might reasonably be expected.
In March, Ofcom launched a separate investigation into cloud communications provider, Vonage. This followed an incident resulting in disruption for its business customers to emergency call services during October and November 2023.
Big consequences
In March this year, Australian incumbent Telstra apologised to family of Victorian who died during triple zero outage. The operator has been transparent with its subsequent investigation which was triggered by a technical fault after there was a high volume of registration requests to the platform that manages CLI for the Triple Zero service. These requests came from medical alert devices, which are IoT devices, that are designed to make emergency calls.
At the time of the incident, these devices were not making calls, but registering on Telstra’s network in preparation should they need to do so in the future. This would not ordinarily cause an issue, however on this occasion it coincided with other system activity that resulted in connections to the database to reach the maximum limit. This triggered an existing but previously undetected software fault, which in turn caused the platform to become unresponsive and not able to recover on its own.
For all 24 state emergency operators, Telstra stores a backup phone number in a secondary database that can be used for manual transfer should it be needed. For eight of the 24 numbers the number stored was not correct, which prevented the operator’s team from making the manual transfer of the call to the respective emergency services operator. In some instances, the operator had to rely on email as a fallback – never ideal in an emergency.
Last week, the Australian Government directed the Australian Communications and Media Authority (ACMA) to make enforceable industry standards to improve how telcos communicate with customers, particularly during major outages. Ironically, the move stemmed from the Post-Incident Review of a major Optus outage of 8 November 2023, which also impacted emergency numbers.
Ther regulator will now make new rules to ensure telcos keep customers informed and updated regarding major outages, for example, through website updates, email updates to customers, social media updates and radio and television news bulletins.
