The router that took down America’s antique air traffic system, explained - Vox

Direct Source Verification: This story is aggregated from Vox (vox.com). Full reporting rights and copyright belong to the primary publisher.
America’s air traffic control woes are far from over, aviation reporter Darryl Campbell says.

America’s air traffic control woes are far from over, aviation reporter Darryl Campbell says.

Last week — on Friday the 13th, of all days — everyone became hyperaware of how fragile our air infrastructure is. For four hours and 21 minutes, the Federal Aviation Administration issued a nationwide ground stop, grounding all flights.

Tens of thousands of flights were impacted, even more people were stuck at airports, and theories about the cause spread like wildfire. Then the Trump administration finger-pointing began.

Eventually, however, the truth came out: The grounding wasn’t the result of a cyberattack, some kind of AI incident, or something even stranger. Instead, it was routine — though maybe ill-considered — maintenance gone awry.

Even with the mystery solved, though, the fact that so many flights were grounded over what was revealed to be a problem with a router somewhere in Arkansas is troubling the American public in a real way.

So how did it all go down? Darryl Campbell writes about aviation for The Verge, and he talked with Today, Explained co-host Sean Rameswaram about how one router brought down the entire aviation system, why it’s likely to happen again, and what we could do to stop it.

Below is an excerpt of their conversation, edited for length and clarity. There’s much more in the full podcast, so listen to Today, Explained wherever you get podcasts, including Apple Podcasts, Pandora, and Spotify.

Did this all really happen because of a rogue router situation?

It did. But don’t think about the router that you have in your house for wifi. These are huge industrial-scale routers.

Let me take you to Friday the 13th. There’s a contractor called L3Harris and they’re doing what they hope is routine maintenance on a dinky little part of the aviation infrastructure backbone for data in the middle of nowhere, Arkansas. Because they think it’s this little tiny place, they do it during the day on a Friday, which is usually not when you want to be messing with massive data infrastructure. But they think, “Well, we’re going to do it anyway.”

They plug in this router for the first time. They’re basically pulling out an old telephone switch that is using copper cables and they’re installing one that has these new fiber optics. It’s super fast and all this good stuff, except that they leave a factory setting on. And the factory setting starts pinging the network hundreds of times every second.

Now, if you’re on normal new infrastructure, that’s not a problem. But the thing is the FAA’s data backbone is this kind of patchwork system of old-school technology designed in the ’50s and ’60s, with some replacements in the ’80s and ’90s.

The problem is that this factory setting assumes that you’re plugging it into a brand-new system and it quickly starts overwhelming the data flow with all of these requests and kind of causing data loss in the same sense of what you would get if you’re on a FaceTime call with someone and your wifi signal isn’t very good. So you start losing bits, the audio gets a little bit choppy.

Now the difference here, again, because it’s an industrial-scale thing, is that a FaceTime call doesn’t really matter if you start to lose data. But if you are tracking hundreds of flights and you need up-to-the-second audio that’s perfectly crystal clear with absolutely perfect data, once you start to lose some of that, then you can actually cause a plane crash. And so that’s something that you really, really don’t want to happen.

Why is it so hard to fix these problems?

The weirdest part about it all is that even though everybody flies, the timeline for doing these kinds of upgrades is really out of sync with the political timeline.

You have a major party change, a presidential change every four and eight years. But if you’re going to do a large-scale system upgrade, you literally have to be digging hundreds of miles of cables, ripping out systems, replacing them without causing the whole system to go down, for months on end. It really is a complex logistical ballet and that’s why it takes 10, 15, 20 years to do these kinds of step-change upgrades.

If you’ve got these political parties pulling funding one year and then reinstating it another, or fighting over it because the government shut down, and causing these delays, they can easily turn a short-term project — short-term in the sense of three to five years — into something that takes 10 or 20 or just never happens.

[Transportation Secretary Sean] Duffy says he can’t guarantee this won’t happen again. Does that mean it’s going to happen again? Because it seemed pretty bad on Friday.

It was. If I were Sean Duffy, I would definitely be saying that this is probably going to happen again because honestly, it is.

Ironically, we’ve gotten so good at kind of kicking the can down the road and making the best out of these systems for so long that we’re now to the point where all of these Band-Aid fixes are just falling apart all at once. If we had done a little bit more to cause failures five to 10 years ago, then we probably would be in a better place today, in a weird sense.

It’s pretty clear that we’ve seen the pace of these aviation infrastructure and equipment crises accelerating over the last three to five years. And it’s really because there’s nothing else that we can do to stop it. You just have to take the pain at some point. And we could have done it earlier with better political will, but now the equipment itself is telling us just can’t go on anymore.

Darryl, we famously aren’t the only country with planes, and certainly we can’t be the only country with aging infrastructure.

How are the other countries fixing their aging infrastructure without having to ground all their planes on a Friday night?

Europe probably has a very similar system in terms of the age of its components, but they do a lot of domestic travel through trains. The load on the aviation system is substantially less.

US airspace is the most crowded in the entire world, and it’s because we rely on flights to get from point A to point B. Outside of the Northeast, there’s really no alternative if you want to go more than 200 miles and you don’t want to spend a whole day driving.

If you were redoing the system today, like is happening in China, you would just put in this fancy fiber optic stuff. Now obviously, it’s a lot easier when you’ve got a single political system to rip up entire hundred-mile runs of cable without having to go through all of these permitting processes. But the main problem is still there.

Europe, famously, across the pond. But what about, dare I ask, Canada?

Canada has about 8 to 10 percent of the volume of the United States in terms of air. In fact, Canadian airspace is partially controlled by American air traffic controllers. And Canada actually requires people to pay fees for the infrastructure.

In the US, we’ve decided that aviation is essentially a right for people, so the pressure is to push down costs as much as possible. In Canada, you’re not going to get your $70 airfares to wherever. Airfares are a lot more expensive, partially because they’re putting some of that money back into the system.

So are you saying we could have fixed this with fees?

Partially! I think the big infusion of cash required to do these big infrastructural projects would have taken acts of Congress, but some of the day-to-day maintenance would certainly have been a lot easier if you have a lot of fees just coming in on a regular basis.

Famously, some radar systems in the United States are held together with duct tape and aluminum foil. I really do think that the big stuff like this kind of upgrade is only going to happen with congressional funding — infusions of billions and billions of dollars that you’re really never going to get through simple fee structures.

Does the fact that it went so bad on Friday and that people are so mad and it looks so awful for our country, does that help with the politics of all of this?

I hope so. And I do think that Sean Duffy is smartly deciding to use this as a way to get at least some of his upgrade budget and agenda over the line.

Regardless of political persuasion, we really do need tens of billions of dollars of investment in our air infrastructure in order to make things not collapse spectacularly. But I really think that it’s a systemic problem. If it gets solved this year for a year or two, that would be great. But people just tend to forget about these things, especially if you’re going into a season where you’re going to have winter storms.

It’s just so complex for an individual person to understand all the parts, so you only have a little bit of time to really capitalize on these kinds of things. And it takes years and years of sustained attention to really solve some of these problems.

Original Source
https://www.vox.com/podcasts/511181/faa-ground-stop-router-upgrade-arkansas
Visit Vox ↗
SHARE STORY:
𝕏 f in

Related Coverage in Technology