The idea is to accelerate the group’s safe adoption of GenerativeAI at scale, in parallel with the Group’s data management platform
BT Group’s Digital Unit is to launch a platform for internal use to help the company tap into the power of large language models (LLMs). This includes from the likes of Anthropic, Meta, Claude, Cohere and Amazon. It is live today with the first beta use cases
The GenAI Gateway was built in collaboration with AWS and using Amazon Bedrock, Amazon SageMaker and AWS Professional Services capabilities. It provides secure, private access to natural-language processing and LLMs, which BT says are a “critical tool…[it]… will use as it embeds AI into the way it runs the business”.
For example, a trial in Openreach, BT’s access division, is summarising engineering notes on Ethernet and full fibre jobs, helping to simplify processes and boost productivity for its teams and its service provider customers.
A second use case, supporting contract analysis for the Group’s Business, legal and procurement teams, is also live.
BT Digital says that the Gen AI Gateway is one of “the several key enablers we are deploying to enable BT Group as an AI-enabled enterprise” and that it will also use “our ‘data fabric’ data management platform to help enforce governing policies regarding how data can be used, as well as to manage access control and data sovereignty restrictions”.
Ad hoc use not suitable at scale
The press statement explains that while ad hoc use of LLMs is appropriate for test and development work, is not well suited to large scale use as cost control, security and privacy need more careful management. Also, the performance of LLMs needs to be monitored for unexpected errors, known as hallucinations, and model decay which happens when LLMs no longer behave as expected.
The GenAI Gateway is intended to give BT Group protection against being locked into to any particular LLM if other issues emerge. The operator says the use of GenAI Gateway platform “will encourage BT Group engineers to use the right model for the right use case, at the right price, as it supports per-use case budget tracking”.
Further, it is expected that a single, consolidated platform will reduce duplication of effort and resources, as BT Group scales up its adoption of GenAI. APIs, security configuration and infrastructure management, can all be managed centrally, reducing the risk of error and the cost of maintaining separate LLMs for every use case.
AWS shop with APIs
GenAI Gateway, deployed on AWS, is accessed via secure APIs, like all the components of BT Group’s modular digital architecture. GenAI Gateway uses Amazon Bedrock, a
managed service that offers a choice of foundation models from AI companies like AI21 Labs, Anthropic, Cohere, Meta, Mistral AI, Stability AI, and Amazon through a single API.
Amazon Bedrock also offers Amazon SageMaker, a managed service with a set of tools to enable machine learning “for any use case”.
The platform supports prompt security, chat history, FinOps billing per use case, and enterprise search, as well as use of corporate data sources. Central privacy controls include: separate tenants for each use case; the use of Personal Identifiable Information filters; the location of the data within the UK; and isolates trained models from each other to protect data in line with the Group’s policies and regulation.
Guardrails are built into the GenAI Gateway to limit the risk of jailbreaks or toxic interactions. They filtering out queries that go beyond the remit of specific applications to ensure performance and ethical guardrails are built-in by design.
Showing off GenAI
Deepika Adusumilli, Managing Director, Data & AI, BT Group’s Digital Unit said, “We believe that where our data is a constant, we need flexibility with our LLMs. GenAI Gateway allows us to tap into this powerful new set of technologies at scale, in a way that is safe, responsible, flexible and scalable, delivering the ambition we have for AI to unlock the human potential within BT Group, today and in the future.”
Fabio Cerone, GM EMEA Telco at AWS, added, “The BT Group GenAI Gateway is showcasing how enterprises can effectively deploy generative AI at scale and speed. It’s been a brilliant, pioneering opportunity to collaborate and work backwards from the customer to provide a way to accelerate deployment of generative AI use cases into production with embedded security and compliance.
“The GenAI Gateway will trigger the flywheel effect in the adoption of generative AI, delivering quicker results for BT Group and its customers.”