I've been reading a lot about submarine cables (inspired by the novel Twist) and some say 99% of internet traffic is passed through 'em but, for example, if I'm in the US accessing content from a US server that's all done via domestic fiber, right? Can anyone ELI5 how people arrive at that 99% number? THANK YOU!
The 95-99% figure you are talking about is international internet traffic, not ALL internet traffic.
In terms of bandwidth we're talking terabytes of data a sec going over those cables.
You are only thinking of traffic in one direction, US to US is domestic and never hits the cables.
There's tons of servers and services hosted outside the US that are accessed by US customers everyday.
You also have to consider other countries accessing datacenters and websites within the US.
Those cables also typically carry voice, phone calls
Youre right. Big internet services also use global CDNs with datacenters around the world to have less latency and less intercontinental traffic.
Great point.
This is missing from most comments here, and when you consider how much traffic is handled by “big services” it skews the answer pretty dramatically from what most commenters are indicating.
There’s a metric fuckton of data interchange on undersea cables, but as a proportion of the total internet traffic? Less than might be expected as CDN caching and distributed services handle things nearer to the request source.
Yes. Considering bandwidth we can ignore most websites against youtube etc.
I posted it elsewhere, but Reddit is a good example. Most likely, your comment was uploaded to dozens of servers around the world, ONCE, and now the guy in Belarus is downloading it from a server in Belarus, and I'm getting it from the Toronto server, and you're getting it locally.
There's a lot of optimization in these things to keep intercontinental data minimal.
There was a rather lengthy and detailed article about undersea cables written by Neal Stephenson in the 90s called Mother Earth Motherboard. It’s way out of date now but it would still be a very informative read, I think it was for Wired.
There is a lot more domestic traffic that uses understand cables than a lot of people think. Google undersea fiber optic cable map and look along the coasts. Say for example, you wanted to connect from a computer in Florida to a server in Virginia. The majority of the path that data took is most certainly going to be in an offshore cable. It's simply cheaper and easier to roll a big spool of cable off the back of a boat than it is to deal with private land, swamps, and mountains. Now consider that damn near the entire population lives in a state that touches either the three Coast or a Great Lake.
When I worked for an ISP, any Florida traffic heading north took a hop in Atlanta. Probably just how our network was set up but I do wonder how common it is.Â
Roughly:
70% of international traffic uses submarine cables, 30% is over land borders and a really tiny bit is via satellite.
But, only 15-25% of internet traffic is international, so 75-85% is domestic.
So, if 70% of the 25% uses submarine cables, that's about 18% of all traffic going across submarine cables.
You have got a lot of good answers about the number from people, but if you actually want to know where YOUR internet traffic goes, there is a tool on called traceroute (tracert on windows). You have to use the command prompt to use it, but if you do tracert google.comit will spit back a list of IP/URLs that your connection took to get to google. You can then use https://www.iplocation.net/ip-lookup to get a rough (very rough, just ignore the "GPS coordinates") idea where that link is located and where your connection went.
VERY cool! Had to use terminal and type traceroute for my mac but checking this out!!! Thank you!
Yeah, it’s not true, I don’t know why or how all the TikTok’s and YouTube videos about undersea cables made that number up, but it’s not true. You’re 100% correct that domestic traffic is going to stay domestic and has no reason to pass through undersea cables. Even foreign owned websites/apps like TikTok and Temu are likely using a US-based servers to serve their US customers.
We don't really know.
And for your example in the post details, it is entirely possible that traffic which seems domestic, may also actually be routed to foreign servers as well.
For example, SAP licenses. If I as a customer order something through the SAP system and the order directly lands into the SAP ecosystem of my supplier, it is entirely possible that the traffic was routed through the German servers in Germany of the German company SAP.
You could be accessing a YouTube video from an American YouTuber, but if that American YouTuber has say, 1M subscribers and those represent about 25% of his usual viewers, and remaining 75% of views are from users not subscribed. The YouTuber is not really popular in US, but maybe is popular, let's say in Asia - YouTube will probably be moving his data/videos on their Asia servers - that way they pay less in traffic fees or even though Google has their own cables, they do not clutter the cables.
Technically, you are viewing content made by an American, in America, viewed by an American, but the content isn't stored on YouTube's American servers. Whether to store on Asian servers OR American servers OR on both servers - is a cost question. How many times is the video viewed in America? Does it make sense to store it on American servers and use about $0.10 worth of storage and associated $0.10 of labor to store on American servers or just pay $0.000001 for the data to be sent via the Asian servers through the international undersea cables to US?
Google has data centers - whether AI or data or cloud - I do not know existing in 14 countries as of 2023, and it was planning expansion to Middle East and more countries in Africa, with expectation that they would have data centers in a total of 22 countries by 2030.
Today, you don't really know where the content or websites or apps you are using are stored or hosted from.
This is so helpful! Thank you!
Off Topic ish: about 25 years ago Wired had a great article about FLAG - Fiber Link Around The Globe.
Yes. Archived copy here:
https://euripides.dk/setebos/frx/matrix/ai/books/stephenson_mother_earth_mother_board.pdf
Very long, very interesting, very well written.
Almost 30 years ago. WOW.
I think you misread the claim, its actually “Submarine cables account for over 99% of intercontinental data traffic” not all data traffic.
I doubt they misread it, I’ve seen this exact claim before too even though it’s obviously wrong. Edutainment media over-simplifies stuff or just makes stuff up all the time.Â
As everybody else has said it's intercontinental. However because lines can get congested and the internet works on an "honour system". You can get some very bizarre routing behaviours. For instance during COVID one Chinese router falsely advertised itself as having the world's shortest connection to the Zoom servers. So all US traffic to Zoom, went through China. Incidentally this was just after it was revealed that despite the company's claims, Zoom was virtually unencrypted. So the people running the server could have recorded and gone through all of the Zoom meetings.
More usually a US to US connection can go through Canada, Mexico and occasionally Europe depending on congestion elsewhere.
Rough estimate for submarine traffic is up to 100 terabytes per second
There are a lot of oceanic cables still in use today and being upgraded. This post on reddit has a great visualization of many of the submarine fiberoptic cables.
https://www.reddit.com/r/BeAmazed/comments/1gvitqm/map_of_internet_fiber_optic_cables_at_the_bottom/
Fiber optic cable is one of the cheapest ways to send a lot of data quickly across the planet. (Yes, I know a station wagon full of magnetic tape has amazing bandwidth too)
Don't forget public Internet traffic while being a lot of the traffic we think of. It's only part of what is sent over those connections. There's a lot of phone calls including PTSN, and digital like VoIP and SIP? There's also a lot of private networks and government/military data being sent around on submarine cables too.
Satellite is there for some stuff, but it has its own set of issues and is really terrible for anything interactive. Depending on how high up the satellite being used is, it can be 600-700ms of latency. From memory, Hughes sats did 100-150 Mbps which was great for batch data transfers and was popular for banks and large corporations as an alternate to loading magnetic tapes onto awaiting 747s.
You know how electricity flows along the path of least resistance? For example, if you have a wire of very low resistance and a wire of very high resistance, the electricity will, generally speaking, flow through the lesser resistance wire.
But some can and will flow through the higher resistance wire too. Because for some electrons, this actually is an easier path, because the other wire is too "crowded" (I'm simplifying things).
The same can be true of Internet. Yes, it is designed to flow along the "best" path. But you can network engineer it to also take alternative paths, if your main path is simply too congested.
So it is completely possible that even when you access a US server, maybe it is being routed through a submarine cable and then over a satellite and then to your computer. It can happen during periods of high congestion.
Does it happen like that with 99% of internet traffic? I have no idea. I don't know how they calculated that or if it's even true.
If you think of resistance as ”who does this company have traffic exchange deals with and where” then yea.
IIRC it's that they handle 99 percent of intercontinental traffic, not of all traffic. The only real alternative is satellite, which handles around 1%.
Satellite is not an alternative due to latency. The 1% of intercontinental traffic is over the land bridges between continents.
Satellite is definitely an alternative. Ships use it all the time. Sure, it’s not sufficient for video, but not all Internet traffic is video.
Modern satellites can definitely handle videos.
I wouldn't want to use it for gaming due to the latency but it's plenty enough for most normal usage.
Might not be so bad. Speed of light in a fiber optic cable is about two thirds of what it is in a vacuum. Starlink is about 200 miles up, so using the Earth girdle problem, the distance traveled is about 16% more than sea level. Relay latency sould push it back down to fiber speeds, probably.
Lol reminds me of a quote from How I Met Your Mother when Ted finally gets his skyscraper built and Robin is toasting him and says "To the youngest architect ever to design a skyscraper! ....Probably!" And everyone at the bar cheers "Probably!"
Random story aside, you're probably right, it's probably not too much of a difference. But the weather can play a big role in the consistency as well. If I had to use it, it would definitely be better than nothing lol. But I'll stick with my hard lines 🙂
Especially for the POE2 tutorial boss. I hear that guy is mean
I work with satellite modems and can confirm - a modem with "good" ping is in the 700-800ms range. I've seen 'em get up to 2.5s and still work - you start breaching 3s and you're probably in the "there's a connectivity problem here somewhere" area
Video is literally the best use of satellite internet. Satellite can be super high bandwidth, just
lowhigh latency. That is great for video.High latency
Yes, my bad
Also, Antarctica.Â
Not at the scale we’re talking about here. Every drop of bandwidth capacity available to every satellite in the sky would come a tiny fraction of the amount of data that passes over submarine fiber
The Internet is for porn…
Starlink currently has latency below 30ms. Fiber cables typically allow light to travel at around 65% of the speed of light while Wall Street uses Microwave Towers to transmit data between New York and Chicago at around 95% of the speed of light over the air.
That may be the case but Starlink satellites are still primarily used just as repeaters to a nearby ground station where your traffic then flows through terrestrial links like all other traffic. Your traffic is not traveling intercontinental in the Starlink network currently.
The laser interconnects between satellites is just starting to roll out but they are primarily being used to extend coverage to areas that don’t have a ground station in range. They are still dumping all the traffic off at the nearest ground station possible.
Something like 90% of the active Starlink satellites have laser links. But they are mostly used to connect places that don't have ground stations nearby, as you mentioned.
Microwave links between Chicago and New York City are a specialized tool used when ultra-low latency really matters, like in high-frequency trading.
Fiber-optic cables can carry vastly more data than microwave (by several orders of magnitude), so for most uses, fiber is the better choice. But if shaving off a few milliseconds of delay is worth the cost, like in financial markets, then microwave can make sense despite its lower capacity.
Starlink relies mostly on ground stations to connect to the internet. The radios are just repeaters for ground based fiber optic connections.
It's definitely a step up from traditional satellite internet but it still has a lot of problems.
They are rolling out point to point communication between the satellites but it's only suitable to extend coverage to remote areas, not replace fiber optics.
Starlink is also woefully inadequate for hosting servers or infrastructure. Trying to get consistent service out of them, static IPs, and working with higher end firewall and network gear is a giant pain in the arse.
The point is that it's an important improvement for internet connectivity, but physical limitations make it only suitable for endpoint devices and homes.
Starlink mostly use local connections. The satellite is used to relay data to a local ground station. In some cases it might relay from one satellite to another before reaching the ground station but it is still not suitable for transcontinental traffic.
Microwave links have a range of line of sight. Since you can not see New York from Chicago you can not have a single microwave link between those cities. Microwave links are used in high speed trading to make shortcuts in fiber optic links. Typically across a lake or between two mountain tops. So again no transcontinental traffic.
The problem with satellites is that while the radio communication is at the speed of light, in order to see both ground stations to get a link to both ground stations you need to be high above them. So the distance the radio signal have to travel is much further then any ground microwave link or the local starlink distances. And distance is time.
Also not really an alternative due to bandwidth.
While I'm sure satellite providers on ships and such charge a pretty hefty premium because there's few other alternatives, they wouldn't do it if it meant they were constantly way below capacity on their satellites. That would be leaving money on the table.
The high prices are a reflection of how little data can go through them. If it was reasonably priced, the satellites would likely run out of capacity very fast, as hundreds, or thousands of ship passengers would start using it, rather than just a tiny number and only for emergencies.
Have you been on a modern cruise ship recently? There's definitely more than just a tiny number on the internet at any given time and it's far far from just for emergencies.
Isn't starlink going to the closest base station and through fiber backbone infrastructure from there?
So if your ship is in the gulf of Mexico and you're face timing with your buddy in the Mediterranean the data goes Starlink -> American base station -> transatlantic fiber -> European base station -> Starlink
The satellites have some sort of mesh routing if they can't see a base station directly, but it won't go intercontinental sat to sat
I didn’t think hardly any satellite internet regularly uses sat to sat, I thought it was basically all routed to the closest base station
They might be using starlink or similar systems, which have way more capacity than regular satellites used for maritime communications.
Wouldn't be surprised if those floating cities have something like a netflix video cache server on board either.
But no, I have not been on a modern cruise ship recently
Yeah, it's starlink, which is now broadly in use across basically everything, including cargo ships and small private boats and is quite affordable.
The mental model of satellite data being expensive is outdated.
I mean, relative to ground based fiber it absolutely is extremely expensive. It’s just orders of magnitude cheaper than it used to be. But I can light up fiber that goes across the continent and gives me multiple terabits per second of bandwidth for a fraction of the cost per bps as what it would take to even sniff that throughput in space.
OP of this thread specifically said it was so slow and expensive that cruise ships only use it for emergencies. Meanwhile the reality is you can stream video on a cruise ship all day long and thousands do it for like $50 for a week of unlimited bandwidth.
Of course satellites can't do the whole internet. But they're way faster and cheaper than OP of this thread suggested.
I suppose. My original point still stands though. Even Starlink wouldn't be able to transmit a meaningful portion of the traffic that goes through submarine cables, so there's still a bandwidth issue.
Except you said this:
Which just isn't true. You can watch Netfilx on a cruise ship nowadays for fun. It's like $50 for a week of unlimited bandwidth. And no, it's not cached locally.
Basic internet is included in a lot of cruise ship base prices now.
Of course sats can't do the whole internet, but you specifically said satellite can't handle a cruise ship except for emergencies, which is not true.
My original point was that satellites wouldn't be able to take a significant chunk of transcontinental traffic not just because of latency, as first mentioned in this comment thread, but also because of capacity.
I've already conceded that satellite data on ships is not necessarily expensive anymore because of Starlink and similar services.
As for netflix, it's very common for ISPs to have netflix (and other streaming services) cache servers in their data centers. I wouldn't rule that out for enormous cruise ships that use Starlink, either.
You can stream from any video site, not just Netflix. And there is no evidence there is a cache on the ship. Read up on those and they require huge amounts of data to be changed every night, and only work when you have hundreds of thousands of users downstream, not a few thousand.
And no, your point in this thread was that satellite is so expensive and limited that it's only used on ships for emergencies. Which makes any other point you are trying to make look like it's coming from a pretty uninformed source.
I don't know if it was actually achieved or not; but part of Starlinks original business plan was to sell ultra-low latency bandwidth between stock/etc trading centers to wall street for megabucks.
The speed of light in fiber optics is only about 2/3rds that of a vacuum (or the atmosphere). Even with the somewhat longer paths travelled if they can switch the data fast enough they should be able to move it faster than underwater cables can across the ocean.
There are multiple $10-100m microwave data links between Chicagos Commodity exchanges and NYCs stock exchange that easily paid for themselves by giving the hedge funds who did so a millisecond advantage in knowing what just happened on the other side compared to their competition using land lines. If Starlink can deliver that's a big pile of money to be had.
It is starlink now but it wasn't as recently as 2 or 3 years ago and there was still a lot more device usage than just emergency use.
Satellite is an alternative for applications where latency doesn't matter much: e-mail, most Websites that don't contain audio or video, maybe turn-based gaming. But it's true that you wouldn't want to use it for action games.
Satellite internet links works great for last mile purposes. Just not backbone transcontinental links.
There are probably some small islands that get internet via satellite. Or maybe just one cable that gets cut.
It is very common to see islands getting their one subsea cable cut and losing Internet for a week or two until the cable can be repaired. If an island is within line of sight to the mainland then it is not uncommon to see microwave links as well, lower bandwidth and might cut out due to bad weather but a nice backup system to have. And you are right that satellite systems do provide Internet. But satellites rely on ground stations. So they tend to be used for local links. Transcontinental satellite links have not been practical since the 70s due to requiring geostationary satellites which have very high latency.
I thought it was people swimming hard drives back and forth
https://what-if.xkcd.com/31/
And yes this is a service https://aws.amazon.com/snowball/
Not 10 years ago it wasn't, but Starlink changed that. You can either have a small number of satellites very high up, and deal with high latency, or a large number of satellites close to the Earth, and have almost no latency as a result. Until recently, launch costs simply were not low enough for the latter option to be viable, but that's not the case anymore.
The starlink bandwidth is not there. They are relaying all the traffic to the closest ground station where it can use the transoceanic fiber optic cables. Even then the latency is significantly worse then last mile fiber.
According to whom?
When building a ground station they need to apply for permits. So we have the public records of all the permits they have and can even visit the ground stations themselves. So far we know of about 150 active ground stations with about 30 currently being constructed. The opening of these stations correspond with the activation of starlink in the various areas. In addition we can map out the entire starlink network by sending out probes through it and see how long it takes for them to traverse the network at various locations and what kind of error messages we can induce. And there are clear patterns of local ground stations connected by fiber optic cables.
As for the intersatellite communications this was first introduced in starlink 1.5. And we know it is based on lasers. This is also what fiber optic cables use. But where a subsea fiber optic cable bundle can have a thousand lasers at each end to send data across it this becomes much harder in a satellite. Because there is no fiber optic cable you can not focus the light as well so you can not have parallel lasers at the same frequency as the light will just merge. And while subsea cables have repeaters quite frequently to improve the strength of the signal the satellites do not have repeaters between them and a lot of energy will be lost to dispersion making the signal quality much worse. So physics limits the bandwidth they can carry for their intersattelite communications to a fraction of what is possible with subsea cables. It can extend the coverage of a ground station so you can get starlink coverage in the middle of the ocean for example. But it would not be physically possible to send all starlink traffic between Europe and America on intersattelite communications.
actually LEO satellites have better latency than fiber cables. That’s because speed of light is significantly higher in vacuum.
Problem is bandwidth and cost
That didn't pass the smell test, so I looked it up.
https://vitextech.com/latency-why-and-when-it-matters/
If the website is right about the difference in latency between the speed of light in fiber compared to a vacuum, the difference amounts to fiber adding less than 1ms between London and NYC.
If LEO is faster, it will be due to other factors like differences in switching, routing, amplification, and congestion.
not sure how did you calculate it, but even your source tells you that speed in fibre optic cables is two thirds of speed of light in vacuum.
That means 17ms over 10k km.
According to the link, light travels at 4.9 microseconds/km in fiber, 3.34 microseconds/km in free space.
4.9-3.34 = 1.56 microseconds/km difference. 5500km distance between London and NYC.
1.56 * 5500km = 8,580 microseconds, how much slower fiber is than light between NYC to London.
1,000,000 microseconds/second
.00858 seconds, or 8.5ms. I was off by a decimal.
Even off by a factor of 10, the point still holds.
If it is latency critical application, 8ms is a lot. Sure, for your average webpage it doesn’t matter. But saying satellites are bad because of latency is just not true.
now try Sydney to London
https://www.meter.net/tools/world-ping-test/
I'm not saying satellites are bad, I'm saying real world latency is mostly a function of hardware and network design.
Saving 1.56 microseconds/km would improve those numbers for sure. At distances of 5000km+, the speed of light starts to play a role in latency, but even if the speed of light in fiber was infinite, it wouldn't even cut the latency I get at long distances in half.
Starlink is doing everything right. Full duplex on the ground stations and laser interconnect between satellites aren't enough. It's still slower than terrestrial internet.
Bandwitdh, interference, solar flares, maintenance, cost, space junk and adjusting antennas positions.
It depends which satellite technology. If you mean geostationary, yes. If you mean a technology like Starlink not that much since they use lasers to connect the satellites and the added distance from Earth to the satellites (it's twice the distance from a base station to a single satellite) is anyway negligible compared to the thousands of kilometers between the satellites, which is akin to the length of the undersea cables.
We were talking about transcontinental traffic here. Starlink uses a fiber optic backbone between their ground stations to deliver transcontinental traffic.
Starlink is designed to use laser links between satellites to complete a path. Look at here.
They are still routing all traffic to the closest ground station. Because anything else would be impractical. So there is very little traffic crossing between satellites on starlink. The only place where this might be the case is in North Africa as the ground stations are in Spain and Italy.
cought StarLink cought
And as bonus, it is cheap and fast (vs other satellite options)
Starlink is not used for transcontinental traffic. The satellites are used to relay traffic to a local ground station. The data does not normally go across the world between satellites.
Is that 1% mostly rural customers using services like Dish?
Starlink/Dish/etc aren't using that satellite to send your signal to the server around the world. It's probably beaming it back down to a larger access point surprisingly close to you, from which it's distributed using normal fiber connections. This makes it actually pretty simple - the satellite is just a "wifi extender" in the sky.
Most of your internet traffic is actually local-ish. If you're on Netflix, you're talking to a server in your city - Netflix doesn't want to pay to have you accessing a server in Belarus when you're in San Fran. Same for anything Google (Youtube etc), Facebook, etc. A good example is Reddit - when I upload this comment, it's gonna get sent to a bunch of servers around the world, once. You're downloading it from a local server, and some other guy's downloading it from another local server.
To add to this, minus the satellite stuff. 70% or more of all internet traffic flows through Ashburn Virginia, USA. Want to take the world off line and dump us back into the early to mid 90s? Drop a nuke on Ashburn.