UN Rogers services mostly restored after daylong outage left millions offline - Internet and Phone Service for 1/4 of Canada Goes Down

The article said:
Rogers services are back online for most customers after a daylong outage at the telecom giant that left millions of Canadians without internet and cellular service, while also disrupting government services and payment systems.

Some individual users saw their internet connections and cellphones come back to life Friday evening, and an update sent to CBC's IT department said the problem in Rogers's "core network … looks to have recovered."

In an update Saturday morning, posted to Twitter, Rogers said it has now restored services for the "vast majority of our customers" and that its technical teams are working hard to ensure that the remaining customers are back online as quickly as possible.

The Toronto-based company has offered no timeline for when service may be restored to all customers.

Tony Staffieri, chief executive and president of Rogers, said in an open letter that the company apologizes for the service interruption. He gave no explanation for the outage or how many customers were affected.
The outage began some time early Friday morning; throughout the day the company said little about its cause or when it might end.

"We don't understand how the different levels of redundancy that we build across the network coast to coast have not worked," said Kye Prigg, Rogers' senior vice-president of access networks and operations, on CBC's Power & Politics.

"We are working very, very hard on making sure that we get everything running as soon as possible," he told host Catherine Cullen.

The company has approximately nine million wireless customers and just shy of three million on the cable and internet side of the business.

Responding to questions about compensation, Rogers said earlier that it would be "proactively crediting all customers" — but did not provide further details.

There is "no indication" the outage is due to a cyberattack, according to a statement from Canada's electronic spy agency, the Communications Security Establishment.

The U.S.-based cybersecurity firm Cloudflare agreed with that assessment, saying in a in a blog post that the outage was likely "an internal error."

Whatever the reason, the impact has been dramatic. Internet monitoring watchdog group Netblocks.org reported that total internet traffic in Canada was at 75 per cent of its normal level on Friday morning.

Rogers-owned flanker brands like Fido and Chatr also went offline, as did services not directly controlled by Rogers, such as emergency services, travel and financial networks.

Basically, 1/4th of the Internet and phone service was down yesterday, affecting shit like emergency call centers, Interac/credit/debit transactions, ATMs, etc. Passport offices were also affected, but nobody noticed as they already had lineups around the block.

Some third world shit right here.

Nobody knows why.
 
This was hilarious. So much squealing. I loved it.

When I first started dating Mr. Flan in 2003, I was already not using cash, and using debit cards for coffee- and then there was that 3 day power outage in the eastern part of the continent that knocked out banking services, and I had less than ten bucks on hand.

Mr. Flan helped me out by lending me enough to survive for those few days, and I decided two things: he was a keeper; and damn, I need to have cash somewhere.

He’s got little caches hidden where I don’t even know about them. Last year I texted him from the grocery store, just grumbling because the Interac wasn’t working and I couldn’t get food. He told me he had stashed some cash in my vehicle and I had no idea.

Long story short, I hope this wised up some people who would otherwise sign up for CBDC and not worry about Trudeau eliminating cash, which he would love to do.

Also the Arrivecan app stopped working LMAO.
 
50yambu9v4i41.jpg
>me on telus yesterday watching my coworker on fido going through phone withdrawals
 
Millions returned to the hive after one day of freedom.

Back in 2019 we had a hurricane in NS and virtually all power and telecom were off for three days in my region, there wasn’t even cellular service, everything went dead. All the distractions of modern life gone.

I’d never seen so many people in my life. The parks were full, everyone was outside enjoying themselves, and people were talking and having a good time in spite of the hurricane damage.

The second the power and telecom came back on it reverted to how it had been, but those few days really standout in my mind as something special.

Just seems to be too massive and widespread to be some idiot unplugging a cord, or some cyber attack.

A bad BGP update could easily cause this widespread of an issue.
 
My conspiratorial theory is that this was a government "shut off" switch accidentally triggered. All the fail-safes failed, according to Rogers. Just seems to be too massive and widespread to be some idiot unplugging a cord, or some cyber attack.
Least I’m not the only one who grabbed the tinfoil hat. I doubt Trudy would flip the switch early - no way Schwab would let him, and I doubt he has the capacity for any independent thought - but it’s one that crossed my mind.

Made work entertaining yesterday though - people are fucking illiterate, swear to god…
 
Back in 2019 we had a hurricane in NS and virtually all power and telecom were off for three days in my region, there wasn’t even cellular service, everything went dead. All the distractions of modern life gone.

I’d never seen so many people in my life. The parks were full, everyone was outside enjoying themselves, and people were talking and having a good time in spite of the hurricane damage.

The second the power and telecom came back on it reverted to how it had been, but those few days really standout in my mind as something special.



A bad BGP update could easily cause this widespread of an issue.
BGP update was at least part of it, but a bunch of other services that shouldn’t rely on IP also died, like 911 calls and banking shit. Unless they have literally all their stuff running over IP on the same network there’s more to it than a BGP update.
 
BGP update was at least part of it, but a bunch of other services that shouldn’t rely on IP also died, like 911 calls and banking shit. Unless they have literally all their stuff running over IP on the same network there’s more to it than a BGP update.

Debit in Canada uses Interac, their services are apparently completely dependent on Rogers as their ISP, or their fall back didn’t work right. The credit companies didn’t have this issue, their servers remained accessible.

Phone calls to my knowledge all rely on the internet now to some degree, old fashioned telephone networks don’t really exist anymore in the sense you’re thinking, even for landlines.

This seems to be a case of a lot of eggs in one basket, which we’ve been warned about for years.

A similar situation happened in the matirimes a few years ago. A cable got damaged and phones, internet, and all banking went down as a result for the better part of half a day.
 
Last edited:
Debit in Canada uses Interac, their services are apparently completely dependent on Rogers as their ISP, or their fall back didn’t work right. The credit companies didn’t have this issue, their servers remained accessible.

Phone calls to my knowledge all rely on the internet now to some degree, old fashioned telephone networks don’t really exist anymore in the sense you’re thinking, even for landlines.

This seems to be a case of a lot of eggs in one basket, which we’ve been warned about for years.

A similar situation happened in the matirimes a few years ago. A cable got damaged and phones, internet, and all banking went down as a result for the better part of half a day.
I’m aware of most of that, phonelines run over VoIP nowadays, but 911 calls, at least on mobile, are supposed to be carrier agnostic, but somehow they fucked it up badly enough it just didn’t work. Apparently you could reach 911 without a sim, but not with one from Rogers. Basically I agree the BGP update probably did a lot of the breakage, but apparently DHCP stopped working too (acquiring public IPs from the ISP) which shouldn’t be reliant on the BGP at all.
 
I'm betting the CRTC will react to this by doing absolutely nothing meaningful as per usual.
Don't be so sure on this one. The Shaw deal for one is going to be in very hot water after this fiasco since it gave some serious ammunition to the legal argument against it. Libs always blow with the wind when the going gets tough so won't be surprising seeing them torpedo the deal for the sake of public support (they're desperate for anything to distract from inflation and housing lol).

Debit in Canada uses Interac, their services are apparently completely dependent on Rogers as their ISP, or their fall back didn’t work right. The credit companies didn’t have this issue, their servers remained accessible.
This also is going to fuck Rogers over in the long term. Last thing banks want is lost business, and not having Interac service is a major irk for any fees picked up through the transactions. I fully expect some quiet moves to adopt another network like Bell as contingency which royally fucks over Rogers' price jacking ability.
 
  • Thunk-Provoking
Reactions: PhoBingas
BGP update was at least part of it, but a bunch of other services that shouldn’t rely on IP also died, like 911 calls and banking shit. Unless they have literally all their stuff running over IP on the same network there’s more to it than a BGP update.
Without powerleveling too much a surprising amount of phones in leafland are IP phones. I'd say about 1/3 landline phones to ballpark it.
 
Back