FTP upload speed throttling/management??

[#1]

I am referring to the idea of downloading a file from my FTP server whilst at a remote location (ie uploading from my Plusnet connection).
I have used both active and passive, with all the correct ports forwarded. Adjusted the buffer sizes in Filezilla etc etc, all to no avail. 
I may try SFTP, but it just means breaking a lot of compatibility with remote users' connections.

[#2]

There is no traffic management on uploads with PlusNet (unless you're on a legacy package - I assume you're probably not as you're on fibre).
So that narrows down the problem to something in your set up. Can't think of anything obvious, maybe check whether your router has QoS enabled?

[#3]

Definitely no QoS active on my kit.  Even swapped out my Dreytek for a HomeHub5, but no difference.
Will try with SFTP and report back with the results.
Thanks all.

[#4]

20Mbps on a wired lan is really slow - maybe investigating the cause for that will help.

[#5]

Having read through this thread, and the drip feed of information, to summarise:
You have an FTP server (unpecified software) running on a server at home (OS unspecified)
At home you can only download from this server at 20Mb/s
At a remote site (friends' houses) you can only download from your server at somewhere between 800kb/s to 1.6Mb/s (100kB/s to 200 kB/s)
Plusnet do not apply any upload traffic shaping/throttling
HTTP traffic performs well at your house, though the direction of this traffic is unclear
Questions:
What's the RTT delay between the remote site and your server?
What is the TCP window size on the server PC?
Is their any packet loss on the connection between the FTP client and the server?