Site Errors (Slow pages, slow posting, connection reset, uploads failing)

[#1]

purleigh reported the imap servers might have similarly buggy load balancer issues.
traceroute to community.plus.net (212.159.9.110), 30 hops max, 60 byte packets
1  192.168.0.1 (192.168.0.1)  2.584 ms
2  lo0-central10.pcl-ag06.plus.net (195.166.128.187)  22.743 ms
3  link-a-central10.pcl-gw01.plus.net (212.159.2.180)  21.589 ms
4  xe-10-3-0.pcl-cr01.plus.net (212.159.0.212)  21.429 ms
5  po2.pcl-gw01.plus.net (195.166.129.41)  23.185 ms
6  gi5-8.ptp-cr01.plus.net (84.93.224.48)  28.605 ms
7  vlan2658.ptp-elb02.plus.net (84.93.224.45)  29.284 ms
8  community.plus.net (212.159.9.110)  68.516 ms
traceroute to imap.plus.net (212.159.9.86), 30 hops max, 60 byte packets
1  192.168.0.1 (192.168.0.1)  3.709 ms
2  lo0-central10.pcl-ag06.plus.net (195.166.128.187)  22.553 ms
3  link-a-central10.pcl-gw01.plus.net (212.159.2.180)  21.361 ms
4  xe-10-3-0.pcl-cr01.plus.net (212.159.0.212)  22.564 ms
5  po2.pcl-gw01.plus.net (195.166.129.41)  34.958 ms
6  gi5-8.ptp-cr01.plus.net (84.93.224.48)  29.911 ms
7  vlan2658.ptp-elb02.plus.net (84.93.224.45)  51.852 ms
8  imap.plus.net (212.159.9.86)  36.136 ms
appears to be the same load balancer.

[#2]

And the rest!
mail.plus.net
pop3.plus.net
relay.plus.net
smtp.plus.net
ftp.plus.net
homepages.plus.net
The last few days we've also had issues with the load balancers for ftp - looks like it's the same boxes.

[#3]

purleigh sent me this in a PM (with permission to post), he has his own custom Firewall and DNS

Quote

I frequently get - on every gateway that I've been on in the last nine months.  Most other forum users will get a silent page hang, stalled connection, or unresponsive browsing, and not notice as much when the load-balancer goes off the correct IP address,  but for me I get an instant redirection to the above page and a firewall full of junk where the NAT/SPI is rejecting packets returned from the wrong source. It is worrying that an ISP can have such a major problem with their network, and they seem unable to do anything about it for the best part of a year !

[#4]

Sorry Kelly, what did you say in reply #115? What's new compared to here please?

Quote from: Townman

Kelly, Thanks for the update.  However, this issue was identified weeks ago, exactly how are your discussions going with the vendors?  What is your support SLA with them?  When have they made a commitment to come back with a resolution?  Indeed who are they?  LB can be (but should not be) a black art.

Isn't it about time you were getting a bit stroppy with them? Or has your SLA got so many holes in it you can't hold them to task?

[#5]

The new bit is they told us how to fix it, and although it worked, it introduced a different issue.

[#6]

Poor testing on their part?

[#7]

It's more of a side effect than a bug!

[#8]

Err, some side effect. Sale of goods act - not fit for purpose - to try and give a simple analogy.

[#9]

EJS/Purleigh:
Either of you guys still seeing the rejected packets from IMAP? 

[#10]

@'Kelly' - I've replied to your PM.
YES, I'm still getting spurious IMAP and community server packets,  although currently it seems quieter than usual.

[#11]

... and I've just had another "Connection reset by peer" error 

[#12]

Quote from: Kelly

EJS/Purleigh: Either of you guys still seeing the rejected packets from IMAP?

I don't use the plusnet imap servers, sorry.

[#13]

@'Kelly' - As no progress is being made towards resolving these problems, is it worth telling us the details of the load-balancers, on the off chance that someone reading this thread happens to have experience with whatever device it is ?, or that somebody (by the power of Google) finds a possible solution ?
In years gone by, Plusnet used to publish a diagram of the networking kit, can you provide an up to date diagram ?, to see whether 'many eyes' can find where the weaknesses are,  otherwise this thread will continue to waste everyones time with the speculation of what is and isn't affected by the load-balancers.

[#14]

F F S !  using this forum is absolutely infuriating, hanging or other fault about every third thread !

Is the SMF database connection on the opposite side of the load-balancers from the community servers ?
If not what the hell is causing this frequent fault ?
               

[#15]

+1 for sharing the problem - I might stil have industry contacts who can assist.
Load balancer configurations screwing up operations such as this (multi route, multi server) are legion, but ought not to beyond the wit of PN network managers or the vendor to fix promptly.
Is this a matter of too many issues requiring focus or a limitation of available skills?
How many application incidences are there?  Are they on dedicated servers / IP addresses not used for anything else?  Are there other coresident applications?  Are there separate database incidences for each application incidence?  If separate are they coresident or accessed over the network?  If the latter, do each access their own synchronised copy or is it a virtualised service again behind a load balancer - the same one(s) or different ones?
How difficult would it be to bring up a single instance of the solution to see if it is adequate to cope with the load and function without the issues we see?
Kevin