topic Re: Speed reducing to set amount at regular interval when downloading files. in Full Fibre

Speed reducing to set amount at regular interval when downloading files.

chaoticmess — Sat, 24 Jun 2023 13:01:49 GMT

Hi,

I’m trying to understand/fix a problem I’m having with my new full fibre 900/112 install.

On the surface things seem fine at first but now I’ve had a chance to use it I’m finding an odd problem keeps occurring.

In a nut shell the problem is that when I download a large file say from here, http://speedtest.tele2.net/, it will start to download it at a speed I would expect but then after a short while the connection speed suddenly drops off a cliff and goes from say an average of 700mbps to 300mbps. It then stays like that for a short while and then suddenly jumps back up to 700mps, then a short while later it drops off a cliff again and goes back to 300mbps.

Now i appreciate that internet speeds can fluctuate but I find it hard to believe it’s a problem with the internet as a whole because it always drops to the exact same speed and the interval of how long it drops for seems to be the same every time.

It is not limited to just http://speedtest.tele2.net/ either. Downloads of different Linux distribution ISO’s from multiple mirror sites produces the same results and it is always the exact same speed it drops too.

What baffles me even more is that randomly it will download a file at 700mps without any problems and continue to download the whole file at that speed. Sometimes I can even download the same file again a few moments later and it will continue to download at a relatively solid 700mps. But then a can do it again a few moments later and its back to dropping off a cliff at a fixed speed and timing. If I were to download a file 10 times I’d say 2-3 of those times it would download at 700mps without issue but the other 7-8 times it does the regular interval cliff drop.

I’ve included an attachment of the effect I’m trying to describe and if you look at the fluctuation when it’s around the 700mps that is what I would expect from the internet being the internet. The constant intermittent drop however looks too clean and measured like something is artificially reducing the speed to a set amount each time.

This is not limited to just a single computer or windows as it also happens on a Freebsd and Linux based OS I’m using as well.

They are all using a wired connection.

Nothing else is using the line at the same time I’m trying to download. It’s just a single computer connected to the hub.

I’m using the Plusnet hub 2.

Any help appreciated.

Re: Speed reducing to set amount at regular interval when downloading files.

MisterW — Sat, 24 Jun 2023 13:33:38 GMT

Nothing else is using the line at the same time I’m trying to download. It’s just a single computer connected to the hub.

@chaoticmess is there anything trying to upload e.g to a cloud drive at the same time ?

Downloading will still require some upload bandwidth for acknowledging the download data, although with a 115Mb upstream you wouldnt expect that to be a problem.

TBH your 700Mb speedtest looks a bit 'off' , you would expect much closer to 900. I have a 900/115 connection and speedtest usually shows 910 down and 110 up. Try the cli version https://www.speedtest.net/apps/cli as that gives much more consistent results

Re: Speed reducing to set amount at regular interval when downloading files.

chaoticmess — Sat, 24 Jun 2023 14:23:48 GMT

Thanks for the reply.

I have already run several speed tests and like I said above, at first everything seems fine and I get the speed I would expect.

The problem is with real world usage when downloading files this problem occurs. It also seems to require time for the problem to kick in and the speed tests don’t last long enough for that to happen which is why they make it look like everything is fine.

Here is the link to the cli speed test result: https://www.speedtest.net/result/c/fa2abdd4-57d1-4058-bfec-5ea594d67283

Nothing is uploading at the same time I’m downloading. I don’t use cloud services and have windows updates policy controlled so I can do them when I want. It’s a bare bones fresh install with nothing running on it to cause a problem.

Re: Speed reducing to set amount at regular interval when downloading files.

MisterW — Sat, 24 Jun 2023 14:40:21 GMT

Ah! I hadnt appreciated from your first post that it was just file downloading that was the problem, I thought speedtests were not right also.

If its just downloads, can your storage handle the sustained write speeds ? Or it it just the,servers you're downloading from that can't keep up ?

Re: Speed reducing to set amount at regular interval when downloading files.

jab1 — Sat, 24 Jun 2023 14:56:55 GMT

I was going to ask a similar question, @MisterW . Surely, both the originating servers load/speed and the OPs write speed/buffer are the limiting factors here?

Re: Speed reducing to set amount at regular interval when downloading files.

chaoticmess — Sat, 24 Jun 2023 15:51:06 GMT

I don’t think it’s either of those things as it happens to both Crucial and Samsung nvme drives and if it was just one or two servers giving me problems I could accept it was those servers causing the issue but it can happen with any large file I download from anywhere so long as the file is big enough to allow say 15-20 seconds to pass.

I also have access to a dedicated server hosted on the internet that has 1gbps download and upload and it can download the same files at its full line speed so I know the servers are capable of providing the speed.

Re: Speed reducing to set amount at regular interval when downloading files.

jab1 — Sat, 24 Jun 2023 15:58:22 GMT

@chaoticmess wrote:

I also have access to a dedicated server hosted on the internet that has 1gbps download and upload and it can download the same files at its full line speed so I know the servers are capable of providing the speed.

On the same equipment at your end?

Re: Speed reducing to set amount at regular interval when downloading files.

chaoticmess — Sat, 24 Jun 2023 17:55:51 GMT

No the dedicated server is running esxi with some VM’s and a opnsense VM being the router. The dedicated server isn’t anything to do with the problem. I was just mentioning it because it proves the servers I’m downloading from on my home plusnet line are fully capable of delivering 1gbps so its not the servers I’m downloading from that drop the speed.

I can also transfer large files over my local home LAN at a constant @960mbps so the drives are fully capable of accepting near 1gbps speeds.

I was hoping to keep things relatively simple but to further add other things I’ve tied and have been successful but I don’t understand why…

I’ve tried using a spare computer with opnsense on it as a replacement router for the hub 2. On its own it suffers from the same problem as when I use the hub 2.

However I also have another computer with opnsense running on it exclusively running as a vpn client through the plusnet line via the first opnsense computer acting as the main PPOE router.

What I don’t understand is when I download a large file using both the PPOE opnsense router to replace the hub 2 and connect through the vpn connection on the second opnsense computer I can sustain around 650mbps download speeds without the speed dropping at regular intervals.

As soon as I try and replace the PPOE opnsense computer with the hub 2 I get the same reduction in speed even though I’m still connecting through the vpn computer only now its via the hub 2 that’s making the PPOE connection.

It makes me feel like it’s a problem with the hub 2 and some sort of setting or hardware limitation or something like that.

Having tried researching the problem I was wondering more if it was a problem with how PPPOE works, MTU settings and similar settings like that as I can get something approaching working when I remove the hub 2 and connect through the vpn yet if I use the hub 2 on its own or with the vpn it gives the same problem.

I hope that makes sense.

Re: Speed reducing to set amount at regular interval when downloading files.

jab1 — Sat, 24 Jun 2023 17:57:48 GMT

@chaoticmess A little bit over my pay grade, but maybe it makes more sense to @MisterW if he comes back.

Re: Speed reducing to set amount at regular interval when downloading files.

Champnet — Sat, 24 Jun 2023 18:08:34 GMT

@chaoticmess wrote:
I also have access to a dedicated server hosted on the internet that has 1gbps download and upload and it can download the same files at its full line speed so I know the servers are capable of providing the speed.

Are you certain the 1gbps download speed applies to all the connectors and cabling inbetween the devices ?

Re: Speed reducing to set amount at regular interval when downloading files.

MisterW — Sat, 24 Jun 2023 18:25:59 GMT

@chaoticmess

I’ve tried using a spare computer with opnsense on it as a replacement router for the hub 2. On its own it suffers from the same problem as when I use the hub 2.

It makes me feel like it’s a problem with the hub 2 and some sort of setting or hardware limitation or something like that.

wouldn't the 1st statement above contradict the 2nd ?

AFAIK the Hub2 is perfectly capable of sustaining 900Mb download

Re: Speed reducing to set amount at regular interval when downloading files.

chaoticmess — Sat, 24 Jun 2023 18:54:04 GMT

I get what you’re saying but what I mean by what I said was that I believe it’s a setting that needs to be configured properly on both and they don’t contradict each other, they simply agree with each other and give the same problem even though they are different hardware and software.

Difference that makes me think the hub 2 is more of a problem than the opnsense router is that the problem still persists if I connect through the vpn computer via the hub2 where as the problem goes away if I connect through the vpn via the opnsesne router. That makes the hub 2 more problematic in my view.

Just to clarify: When I use the hub 2 no matter what else I do the problem persists. However when I don’t use the hub 2 I can get something to work but only through a vpn connection.

This doesn’t make sense to me as in my mind, if anything, it should be the other way around. The hub 2 should download fine and at full speed and the vpn connection should suffer mtu or some other miss-configured setting yet somehow using the opnsense vpn computer and a opnsense computer to replace the hub 2 stops the speed reduction at regular intervals.

Re: Speed reducing to set amount at regular interval when downloading files.

MisterW — Sat, 24 Jun 2023 19:12:00 GMT

Is your opnsense router configured with the same lan ip,subnet and dns servers as the Hub 2 ?

Re: Speed reducing to set amount at regular interval when downloading files.

chaoticmess — Sat, 24 Jun 2023 19:55:33 GMT

Yes, it’s the same. They both use:

IP: 192.168.0.254

Subnet: 255.255.255.0

DNS:

212.159.13.51

212.159.13.52

They’re not active at the same time though. I disconnect the hub 2 then connect the opnsense router and vice versa when switching between them.

Re: Speed reducing to set amount at regular interval when downloading files.

MisterW — Sun, 25 Jun 2023 05:56:56 GMT

Might be worth a read through this https://forums.thinkbroadband.com/fibre/4727755-download-speeds-on-bt-fttp-slow-when-not-using-a-vpn.html?fpart=all

Re: Speed reducing to set amount at regular interval when downloading files.

chaoticmess — Wed, 05 Jul 2023 00:41:33 GMT

Thanks for the suggestion, I looked at the posts there and while the problem is similar it’s not exactly the same as they don’t have the up and down yoyo effect for a specific time period and speed. I tried what was recommended in the thread anyway and also started my own here: https://forums.thinkbroadband.com/technical/4738529-reducing-speed-when-downloading.html?fpart=all&vc=1

Been a couple of days now and no one seems to have any other suggestions I can try so I’ve come back here for help.

No one has said the up and down speed reduction is normal so I assume there is a fixable fault to be found somewhere. I just feel like I’ve ruled out the problem being on my end at this point.

Does anyone have anything else to suggest that I can try that I haven’t already done in the other thread I made on thinkbroadband?

Or advice on what I should do moving forward if I have nothing left to try and this isn’t normal behavior?

The gears grinding in my mind are thinking about how there was a lot of construction work down our road, main street, general town area and motorway the past year or so that led up to FTTP getting enabled. I’m also the very first person in our street to get it enabled so I’m wondering if something wasn’t installed or configured right when the infrastructure got installed or something of that nature since it’s all new stuff and it feels like something is intermittently working and then not, cutting in and out at a consistent regular interval that’s happening either within the ONT or further up the line.

Re: Speed reducing to set amount at regular interval when downloading files.

chaoticmess — Sat, 15 Jul 2023 06:35:52 GMT

Ok, so little disappointed there have been no further replies.

Been going round and round in circles last couple of weeks continuing to try and figure out what’s going wrong and the only new thing I’ve discovered is that when I said previously how the plusnet router didn’t work with the vpn connection and an opnsense router did it turns out they both work with the vpn but only when the pppoe router makes a “good” connection.

By “good” connection I mean that for whatever reason I can restart the pppoe router (plusnet or opnsense) and the vpn connection will give the same speed reduction fault. However, if I restart the pppoe router again and again, usually 3 or 4 times, I eventually seem to stumble onto “something” that works and I can download a file at the vpn’s full speed (usually around 700mbps with the expected internet fluctuation of 30-40mbps higher or lower) without the speed reduction fault occurring.

Of course the first thing I tried when I realised this was to see if I could then download a file on the raw pppoe connection when the vpn had a good connection but alas this didn’t work and it still gave the same drop off a cliff speed reduction.

This has made me start to believe the problem is a gateway/routing problem within plusnets network and when I can get on a gateway that routes well for the vpn connection it permits the full speed with no reduction over the vpn while still having bad routing for everywhere else on the pppoe connection that results in the speed drop.

Just the fact I can get the line and my equipment to support the more expected behaviour of continuous high speed when downloading a file, but only through the vpn connection, suggests to me it has to be some sort of internal problem within plusnets network and when i can get the stars to align and find a gateway/route that works for my vpn connection it is able to bypass the problem.

It’s definitely the pppoe connection that the fault occurs on as once I’ve got the pppoe connection on a good gateway/route I can restart the vpn router again and again and it keeps its full download speed each time and retains those high speeds indefinitely. Restart the pppoe router and it shifts to a bad gateway/route and the vpn router gives the speed reduction fault once again.

Is it possible to get this kind of fault investigated though?

Re: Speed reducing to set amount at regular interval when downloading files.

James30 — Sat, 15 Jul 2023 09:32:35 GMT

@chaoticmess - Hi Just to check does your pc have a SSD hard drive or a old disk type drive. I'm just making sure that the pc has the capability of saving that amount of data as standard HDD only has a write speed of about 80 MB/s that's about 700Mbps converted not even accounting for bandwidth the OS needs in the first place. The next thing to consider is that the provider you are downloading the file from won't have an endless upload speed, for you to download at 700Mbps the server needs an upload of 700Mbps, and thats just for you to get the file never mind anyone else needing a share of their upload speed. To try and figure out what's going on , I think it's best to at least try it on another bog standard PC with and SSD and try to download a file from elsewhere incase it's being throttled at the servers end. After all it's only in the last 12 months that these speeds were possible for the average population, prior 80Mbps was the fastest on FTTC.

Re: Speed reducing to set amount at regular interval when downloading files.

chaoticmess — Sat, 15 Jul 2023 12:47:11 GMT

Thanks for the reply.

In the thinkbroadband link I gave above I managed to rule out it being a drive issue by using the command: wget -O /dev/null http://90.130.74.153/50GB.zip (The IP of http://ams-speedtest-1.tele2.net)

This doesn’t save the data anywhere ruling out the hard drive being a limiting factor.

Please see the pictures attached below. In them you can see that the hdd stat shows the drive doing nothing both when high or low yet the problem still occurred. This was also done on an i9 10900k using ubuntu 22 and the pppoe connection being setup directly in ubuntu to the ONT so no router was involved at all.

But to answer your question specifically. I have also already tired different computers with different OS’s with old style 7200rpm hdd’s, sdd’s and nvme’s. They all have the same problem. It’s not the computers, OS, Network card, network cable, router or hard drive. I've tried multiples of all of them. The problem starts at the ONT or beyond.

As to the provider not having endless upload speed I solved that being an issue as well because I setup my own dedicated server with a 1gbps connection to let me run an iperf3 test from my server to my plusnet connection so the whole bandwidth was dedicated just to me. The problem still occurred in the exact same specific manner.

If the speed was fluctuating in a manner like this: 710mbps>650>600>500>530>570>620>580>510>480>400>350>300>350>400>390>440>500>490>550>580>610>630>670>710>710>700>720>690>650> etc with each rise and fall happening within a second of each other then I could get on board with that being more typical of the internet being the internet and speeds vary.

However this is not what happens. Instead I get something more like this:

720mbps>770>760>750>760>760>760>760>750>760>350>350>350>350>350>340>350>350>350>350>340>350>350>350>350>340>350>350>350>350>730>740>750>740>760>760>760>730>750>750> etc second to second for a set time period and a max low speed it never goes above, it can still dip lower but never higher during its decreased phase.

Separate to and before I did the iperf3 test I had also already tried downloading from well over at least 50 completely different and independent sites/servers, most offering Linux distribution ISO’s that were mirror sites but not all of them were.

I think it’s highly improbable/impossible that they would all suddenly drop their upload speed to the same precise speed for the exact amount of time as part of an issue to do with them, not to mention my own server which I fully control has the same problem.

Additionally, like I said in my last post, when I can get good gateway/routing my vpn connection has no problem maxing out the speed and sustaining it like it should do. So the servers I’m downloading from have no problem uploading to me at the higher speeds and aren’t the source of the sudden decrease in speed.

The common denominator is the network they are routing into, plusnets. This I feel is further proven by the fact the vpn connection can work at its full speed without the speed reduction occurring because it by passes the plusnet network once it can get a good connection established.