T-Mobile explains why it had a major outage: ‘We did not meet our own bar for excellence’ - GeekWire
Trending: President Trump plans restrictions on skilled work visas used by tech and other industries
A T-Mobile store in New York City. (GeekWire Photo / Taylor Soper)

T-Mobile apologized to customers and revealed the root cause of its major outage on Monday that affected voice and text service for customers across the U.S.

“Yesterday, we didn’t meet our own bar for excellence,” Neville Ray, president of technology at T-Mobile, wrote in a blog post published Tuesday evening.

Ray said the issue was related to “a leased fiber circuit failure from a third party provider in the Southeast.” From the post:

This is something that happens on every mobile network, so we’ve worked with our vendors to build redundancy and resiliency to make sure that these types of circuit failures don’t affect customers. This redundancy failed us and resulted in an overload situation that was then compounded by other factors. This overload resulted in an IP traffic storm that spread from the Southeast to create significant capacity issues across the IMS (IP multimedia Subsystem) core network that supports VoLTE calls.

Ray said customers on the Sprint network were not affected. T-Mobile just closed its $26 billion merger with Sprint two months ago, bringing together the nation’s third and fourth-largest wireless carriers.

The outage, one of the company’s largest in recent years, drew the ire of FCC Chairman Ajit Pai.

Following reports of Sprint employees being laid off, T-Mobile issued a statement Tuesday, noting that some employees will be asked to “consider a career change inside the company, and others will be supported in their efforts to find a new position outside the company.” T-Mobile said it is adding 5,000 new positions over the next year.

Become a GeekWire Member

Support independent journalism at a time when trusted storytelling and community engagement is more important than ever.

Join today!

Like what you're reading? Subscribe to GeekWire's free newsletters to catch every headline

Comments

Job Listings on GeekWork

Technical Co-Founder // CTO (Stealth Startup)ALLEN INSTITUTE FOR ARTIFICIAL INTELLIGENCE (AI2)
Deep Learning Engineer // Co-FounderALLEN INSTITUTE FOR ARTIFICIAL INTELLIGENCE (AI2)
Find more jobs on GeekWork. Employers, post a job here.