AS6453.net peering sucks

[#1]

I think there is a problem with one of the London Level3 peering servers at peak times - or at least that is what it seems like.
There are other gaming ISP users reporting the same.

[#2]

Got some links to people complaining of the same external problem?

[#3]

Need to dig the threads out when I get home...but found these in the meantime
http://forums.euw.leagueoflegends.com/board/showthread.php?t=1522304&highlight=level3 - is a PN user
http://us.battle.net/d3/en/forum/topic/11041484504#7 - Sky (issue is att?)
https://community.bt.com/t5/BT-Infinity-Speed-Connection/BTInfinity-customers-problem/td-p/1072242/p...

[#4]

Quote

The geography is irrelevant as long as the latency is good and you aren't seeing packet loss. The packet loss you are seeing on the above trace, is it consistent?

But that's the point the dumb way which  those IP's are being routed is adding extra latency, my base ping to london servers like the BBC is 12-13ms   in my last tracert (example of cack handed peering)  the latency is unnecessarily higher than it should be  As for the packet loss  via Paris ,
I can only say that  on those occassions that  i was routed that way  yes there was always packet loss on some of the hops , and there is  on some of the AS6453.net hops at times  like 3-6am GMT if-3-2.tcore1.W6C-Montreal.as6453.net Suffered bursts of  severe packet loss and huge latency spikes , AS6453.net also sufferes the peak time latency hump at times
Which is why the likes of goscomb aren't peering with them  and are able to offer lower latency  to lb-lsg-prod.ubisoft.com and other ubisoft.com IP's at the same location Montreal Canada , So every ms counts , Latency shouldn't rise from 89-90ms to over 100ms  without a reason, the fact that latency to steampowered.com  has also taken a hit too, which doesn't use AS6453.net   it would seem there are only two  possibilities  left, plusnet's peering  /routing has changed, or it's down to level3
Here's a TCP ping  trace

Target Name: lb-lsg-prod.ubisoft.com
         IP: 216.98.48.56
  Date/Time: 09/01/2014 15:07:40 to 09/01/2014 15:14:47
Hop Sent Err PL% Min Max Avg  Host Name / [IP]
 1   428   0 0.0   0   1   0  home.gateway.home.gateway [192.168.1.254]
 2   428   0 0.0  13 255  23  lo0-central10.pcl-ag03.plus.net [195.166.128.184]
 3   428   0 0.0  12  25  13  link-b-central10.pcl-gw02.plus.net [212.159.2.170]
 4   428   0 0.0  12  48  14  xe-10-2-0.pcl-cr02.plus.net [212.159.0.202]
 5   428   0 0.0  12  89  17  ae2.pcl-cr01.plus.net [195.166.129.6]
 6   225   0 0.0  12  58  13  xe-11-1-0.edge3.London2.Level3.net [212.187.201.209]
 7   428   0 0.0  19  31  19  ae-3-3.ebr1.Paris1.Level3.net [4.69.141.86]
 8   276   0 0.0  19  26  19  ae-81-81.csw3.Paris1.Level3.net [4.69.161.86]
 9   234   0 0.0  19  78  22  [4.69.168.136]
10   428   0 0.0  19  93  20  [141.136.103.181]
11   130   0 0.0 102 170 103  [141.136.106.106]
12   427   0 0.0  96 446 103  ubisoft-gw.ip4.tinet.net [173.241.128.42]
13   427   0 0.0  96 103  97  msr-onl-fw01.ubisoft.com [216.98.51.10]
14   427   0 0.0  97 124  98  lb-lsg-prod.ubisoft.com [216.98.48.56]

Which goes Via Paris no surprise and back to London before actually routing to the USA  adding some 7-8ms  to the base latency  level 3 are a joke when it comes to peering 

[#5]

Not so sure if this belongs here - but someone else blaming Level3 routing for problems with Fifa:
https://community.bt.com/t5/BT-Infinity-Speed-Connection/BT-Infinity-and-Fifa-routing-issues-continu...

[#6]

Target Name: lb-lsg-prod.ubisoft.com
         IP: 216.98.48.56
  Date/Time: 09/01/2014 23:34:20 to 09/01/2014 23:35:06
Hop Sent Err   PL% Min Max Avg  Host Name / [IP]
 1    11   9  81.8   0   0   0  home.gateway.home.gateway [192.168.1.254]
 2    13   0   0.0  12  32  15  lo0-central10.pcl-ag03.plus.net [195.166.128.184]
 3    13   0   0.0  12  14  12  link-b-central10.pcl-gw02.plus.net [212.159.2.170]
 4    13   0   0.0  12  75  17  xe-10-2-0.pcl-cr02.plus.net [212.159.0.202]
 5    13   0   0.0  12  15  12  ae2.pcl-cr01.plus.net [195.166.129.6]
 6    13   0   0.0  12  39  14  ae1.ptw-cr01.plus.net [195.166.129.0]
 7    13   0   0.0  12 139  42  te-3-4.car5.London1.Level3.net [217.163.45.181]
 8    13   0   0.0  12  13  12  ae-52-52.csw2.London1.Level3.net [4.69.139.120]
 9    13   0   0.0  12  13  12  ae-225-3601.edge3.London1.Level3.net [4.69.166.146]
10    13   0   0.0  29  41  35  ix-20-0.tcore1.LDN-London.as6453.net [195.219.83.101]
11    12   0   0.0 121 142 129  if-17-2.tcore1.L78-London.as6453.net [80.231.130.129]
12    12   0   0.0 124 155 133  if-1-2.tcore2.L78-London.as6453.net [80.231.130.122]
13    12   0   0.0 126 148 131  if-20-2.tcore2.NYY-NewYork.as6453.net [216.6.99.13]
14    12   0   0.0 122 142 130  if-12-6.tcore1.CT8-Chicago.as6453.net [216.6.99.46]
15    12   0   0.0 122 152 131  if-22-2.tcore2.CT8-Chicago.as6453.net [64.86.79.1]
16    12   0   0.0 122 178 134  if-3-2.tcore1.W6C-Montreal.as6453.net [66.198.96.45]
Destination not reached in 35 hops

Target Name: steampowered.com
         IP: 208.64.202.68
  Date/Time: 09/01/2014 22:28:29 to 09/01/2014 23:36:45
Hop Sent  Err   PL% Min Max Avg  Host Name / [IP]
 1  1006  455  45.2   0   0   0  home.gateway.home.gateway [192.168.1.254]
 2  1006    0   0.0  12 788  25  lo0-central10.pcl-ag03.plus.net [195.166.128.184]
 3  1006    0   0.0  12 123  16  link-b-central10.pcl-gw02.plus.net [212.159.2.170]
 4  1006    0   0.0  12  97  16  xe-10-2-0.pcl-cr02.plus.net [212.159.0.202]
 5  1006    0   0.0  12 114  18  ae2.pcl-cr01.plus.net [195.166.129.6]
 6  1006    0   0.0  12  93  18  xe-11-1-0.edge3.London2.Level3.net [212.187.201.209]
 7   312  219  70.2  12  34  13  ae-0-11.edge4.London2.Level3.net [4.69.200.126]
 8   316    1   0.3 151 172 152  ae-3-3.ebr1.London1.Level3.net [4.69.141.189]
 9   316    1   0.3 152 232 153  ae-42-42.ebr1.London1.Level3.net [4.69.167.22]
10   316    0   0.0 151 181 152  vlan103.ebr2.London1.Level3.net [4.69.143.94]
11   316    1   0.3 151 178 151  ae-44-44.ebr1.NewYork1.Level3.net [4.69.137.78]
12   316    0   0.0 151 165 152  ae-48-48.ebr2.NewYork2.Level3.net [4.69.201.50]
13   316    1   0.3 152 168 153  ae-1-100.ebr2.NewYork2.Level3.net [4.69.135.254]
14   316    0   0.0 149 193 150  ae-2-2.ebr1.Chicago1.Level3.net [4.69.132.65]
15   316    0   0.0 151 182 151  ae-6-6.ebr1.Chicago2.Level3.net [4.69.140.190]
16  1006    1   0.1 152 191 160  ae-3-3.ebr2.Denver1.Level3.net [4.69.132.61]
17  1006    1   0.1 152 194 160  ae-2-2.ebr2.Seattle1.Level3.net [4.69.132.53]
18  1006    4   0.4 151 265 163  ae-2-52.edge1.Seattle3.Level3.net [4.69.147.170]
19  1006    3   0.3 152 276 163  CENTURYTEL.edge1.Seattle3.Level3.net [4.59.232.74]
20  1006    3   0.3 153 260 163  [205.196.6.33]
21  1004  985  98.1 153 188 161  fw01-core.tuk.valve.net [208.64.202.21]
Destination not reached in 35 hops

 Yet a further increase in LAG tonight
And the TCP

Target Name: lb-lsg-prod.ubisoft.com
         IP: 216.98.48.56
  Date/Time: 09/01/2014 23:34:20 to 09/01/2014 23:44:47
Hop Sent Err  PL% Min Max Avg  Host Name / [IP]
 1   213 104 48.8   0   1   0  home.gateway.home.gateway [192.168.1.254]
 2   214   0  0.0  12 141  17  lo0-central10.pcl-ag03.plus.net [195.166.128.184]
 3   214   0  0.0  12  33  12  link-b-central10.pcl-gw02.plus.net [212.159.2.170]
 4   214   0  0.0  12  75  13  xe-10-2-0.pcl-cr02.plus.net [212.159.0.202]
 5   214   0  0.0  12  69  14  ae2.pcl-cr01.plus.net [195.166.129.6]
 6    31   0  0.0  12  52  15  xe-11-1-0.edge3.London2.Level3.net [212.187.201.209]
 7    45   4  8.9  12  42  14  ae-0-11.edge4.London2.Level3.net [4.69.200.126]
 8   179   0  0.0  12  27  13  ae-52-52.csw2.London1.Level3.net [4.69.139.120]
 9    42   0  0.0  13  14  13  ae-42-42.ebr1.London1.Level3.net [4.69.167.22]
10    46   0  0.0  13  14  13  ae-57-112.csw1.London1.Level3.net [4.69.153.118]
11   193   0  0.0 121 196 141  if-17-2.tcore1.L78-London.as6453.net [80.231.130.129]
12   198   0  0.0  29 105  47  ix-20-0.tcore1.LDN-London.as6453.net [195.219.83.101]
13    36   0  0.0 129 154 138  if-17-2.tcore1.L78-London.as6453.net [80.231.130.129]
14   182   0  0.0 122 202 146  if-1-2.tcore2.L78-London.as6453.net [80.231.130.122]
15   202   0  0.0 121 181 144  if-20-2.tcore2.NYY-NewYork.as6453.net [216.6.99.13]
16   206   0  0.0 122 189 144  if-12-6.tcore1.CT8-Chicago.as6453.net [216.6.99.46]
17     3   0  0.0 145 152 148  if-22-2.tcore2.CT8-Chicago.as6453.net [64.86.79.1]
18   211   0  0.0 122 180 145  if-3-2.tcore1.W6C-Montreal.as6453.net [66.198.96.45]
19   213 165 77.5 130 160 143  lb-lsg-prod.ubisoft.com [216.98.48.56]

Changed from what it was  too, now a 45ms increase ,

[#7]

deathtrap - is this affecting particular games? We may have some data from our traffic management system to help back up what the problem is but I'd need to know the game or games affected.
For example, attached is the average latency from our network to the destination for all steam traffic, Tuesday had a spike and last night a small increase. If we can see the same for specific games then it will certainly help.
For reference this is being tracked internally on problem 79128.

[#8]

Without a doubt the  lb-lsg-prod.ubisoft.com ( Ubisoft.com )  base latency  on my connection is the latency to their master server system,  And  any spikes  in that latency result in a poor/ unplayable gaming experience  for both other players and myself the higher the latency  the more unplayable the game is
Tom Clancy's rainbow six Vegas 2, under 100ms  is just about playable  over  isn't playable
As for the steam, I'm not sure what the steampowered.com  IP actually does, but i know that it's one if not the most prominent IP  that steam uses ,
The inclusion of data from that was to possibly help to identify if it was a level3 or AS6453.net issue  and it does appear to be level3 IMO, The Steam  base ping  has nearly always been around 150ms  i think it did drop a bit to around 142ms for only a brief period , but that has increased to 160ms-180ms,  the only game i have to use steam with is Borderlands 2, which i haven't  played for some time , but i used to see ping's to other players that where in the 50-70ms range  but where they where located geographically i couldn't say  but if in the uk that's quite high

Target Name: lb-lsg-prod.ubisoft.com
         IP: 216.98.48.56
  Date/Time: 10/01/2014 13:02:58 to 10/01/2014 14:00:38
Hop Sent Err  PL% Min Max Avg  Host Name / [IP]
 1  1005   0  0.0  0   1   0  home.gateway.home.gateway [192.168.1.254]
 2  1005   0  0.0  12 384  21  lo0-central10.pcl-ag03.plus.net [195.166.128.184]
 3  1005   0  0.0  12  33  12  link-b-central10.pcl-gw02.plus.net [212.159.2.170]
 4  1005   0  0.0  12 109  13  xe-10-2-0.pcl-cr02.plus.net [212.159.0.202]
 5   995   0  0.0  12  91  15  ae2.pcl-cr01.plus.net [195.166.129.6]
 6     6   0  0.0  13  71  22  xe-11-1-0.edge3.London2.Level3.net [212.187.201.209]
 7    10   0  0.0  19  20  19  ae-3-3.ebr1.Paris1.Level3.net [4.69.141.86]
 8     6   0  0.0  19  21  20  ae-71-71.csw2.Paris1.Level3.net [4.69.161.82]
 9     1   0  0.0  20  20  20  [4.69.168.200]
10    14   0  0.0  19  20  19  [141.136.103.181]
11    15   0  0.0 102 122 104  mtl10.ip4.tinet.net [141.136.106.106]
12    12   0  0.0  97  97  97  ubisoft-gw.ip4.tinet.net [173.241.128.42]
13    14   0  0.0  97  98  97  msr-onl-fw01.ubisoft.com [216.98.51.10]
14  1004 983 97.9  98 112 103  lb-lsg-prod.ubisoft.com [216.98.48.56]

 A tcp Ping  on port 3074  lower than it was last night  and using different routing  although  why it goes via Paris  which will add some latency  is anyone's guess but i would bet they can route traffic to the USA from London
I just found this old tracert to the ubisoft .com IP above  when i was with Bethere http://oi41.tinypic.com/10mst1s.jpg even that shows a lower ping than what i currently am seeing, and notice the lack of level 3

[#9]

Well 17 days on an nothing has changed still got a ping higher than it should be, is there any updates on this >>  problem 79128.

[#10]

Target Name: lb-lsg-prod.ubisoft.com
         IP: 216.98.48.56
  Date/Time: 14/02/2014 19:12:26 to 14/02/2014 19:36:06
Hop Sent Err   PL% Min Max Avg  Host Name / [IP]
 1  1000   0   0.0   0     0   0  home.gateway.home.gateway [192.168.1.254]
 2  1000   0   0.0  13 291  45  lo0-central10.pcl-ag02.plus.net [195.166.128.183]
 3  1000   0   0.0  13 117  33  link-b-central10.pcl-gw02.plus.net [212.159.2.166]
 4  1000   0   0.0  12 108  34  xe-10-1-0.pcl-cr02.plus.net [212.159.0.198]
 5  1000   0   0.0  13  76  33  ae2.pcl-cr01.plus.net [195.166.129.6]
 6  1000   0   0.0  13 165  34  xe-11-2-0.edge3.London2.Level3.net [212.187.201.213]
 7  1000   1   0.1  19 122  39  ae-3-3.ebr1.Paris1.Level3.net [4.69.141.86]
 8  1000   1   0.1  19  95  40  ae-48-48.ebr1.London1.Level3.net [4.69.143.113]
 9  1000   1   0.1  19  62  39  ae-56-111.csw1.London1.Level3.net [4.69.153.114]
10  1000   0   0.0  19 127  43  ae-115-3501.edge3.London1.Level3.net [4.69.166.130]
11  1000   2   0.2  20  95  41  ix-20-0.tcore1.LDN-London.as6453.net [195.219.83.101]
12  1000  34   3.4 113 148 132  if-17-2.tcore1.L78-London.as6453.net [80.231.130.129]
13  1000   1   0.1 111 152 131  if-2-2.tcore2.L78-London.as6453.net [80.231.131.1]
14  1000   1   0.1 113 192 138  if-20-2.tcore2.NYY-NewYork.as6453.net [216.6.99.13]
15  1000   2   0.2 113 151 132  if-12-6.tcore1.CT8-Chicago.as6453.net [216.6.99.46]
16  1000   5   0.5 112 149 132  if-22-2.tcore2.CT8-Chicago.as6453.net [64.86.79.1]
17  1000   0   0.0 113 186 135  if-3-2.tcore1.W6C-Montreal.as6453.net [66.198.96.45]

And today from 10am i'm being bounced from London to Paris and back again before going to NewYork adding even more latency , and still nothing is or has been done, how long does take for plusnet to ask level3 what the hell they are playing at , or switch peering ?

[#11]

We've raised it with them and they don't think there is a problem.  Issue is still open on our side.

[#12]

Maybe they need to think a bit harder then, as i  for one don't agree with what they think , have they seen their ridiculous routing  that is causing an unnecessary increase in latency  it wouldn't supprise me if they haven't bothered to even look in to it
Not only that but the current  ping to ubisoft  is the highest  it been in the past 6yrs!!! nothing wrong what planet did they fall off ?
Also why do their looking glass tracerts not also route the same cack handed way?

raceroute results from London, England
to 216.98.48.56(lb-lsg-prod.ubisoft.com)
  1 ae-52-52.csw2.London1.Level3.net (4.69.139.120) 0 msec
    ae-51-51.csw1.London1.Level3.net (4.69.139.88) 216 msec
    ae-52-52.csw2.London1.Level3.net (4.69.139.120) 4 msec
  2 ae-118-3504.edge3.London1.Level3.net (4.69.166.142) 0 msec
    ae-225-3601.edge3.London1.Level3.net (4.69.166.146) 0 msec
    ae-116-3502.edge3.London1.Level3.net (4.69.166.134) 216 msec
  3 ix-20-0.tcore1.LDN-London.as6453.net (195.219.83.101) [AS6453 {GLOBEINTERNET}] 0 msec 0 msec 4 msec
  4 if-17-2.tcore1.L78-London.as6453.net (80.231.130.129) [AS6453 {GLOBEINTERNET}] 220 msec 216 msec 444 msec
  5 if-2-2.tcore2.L78-London.as6453.net (80.231.131.1) [AS6453 {GLOBEINTERNET}] 156 msec 144 msec 108 msec
  6 if-20-2.tcore2.NYY-NewYork.as6453.net (216.6.99.13) [AS6453 {GLOBEINTERNET}] 104 msec 120 msec 100 msec
  7 if-12-6.tcore1.CT8-Chicago.as6453.net (216.6.99.46) [AS6453 {GLOBEINTERNET}] 160 msec 112 msec 132 msec
  8 if-22-2.tcore2.CT8-Chicago.as6453.net (64.86.79.1) [AS6453 {GLOBEINTERNET}] 96 msec 112 msec 96 msec
  9 if-3-2.tcore1.W6C-Montreal.as6453.net (66.198.96.45) [AS6453 {GLOBEINTERNET}] 96 msec 96 msec 148 msec

Traceroute results from London, England (2)
to 216.98.48.56(lb-lsg-prod.ubisoft.com)
  1 ae-25-25.edge4.London2.Level3.net (4.69.200.89) 4 msec 0 msec 0 msec
  2 ae-3-3.ebr1.London15.Level3.net (4.69.141.189) 0 msec 0 msec 0 msec
  3 ae-43-43.ebr1.London1.Level3.net (4.69.167.26) 0 msec
    ae-44-44.ebr1.London1.Level3.net (4.69.167.30) 0 msec 0 msec
  4 ae-56-111.csw1.London1.Level3.net (4.69.153.114) 4 msec
    ae-59-114.csw1.London1.Level3.net (4.69.153.126) 0 msec
    ae-56-111.csw1.London1.Level3.net (4.69.153.114) 0 msec
  5 ae-115-3501.edge3.London1.Level3.net (4.69.166.130) 0 msec 4 msec
    ae-118-3504.edge3.London1.Level3.net (4.69.166.142) 0 msec
  6 ix-20-0.tcore1.LDN-London.as6453.net (195.219.83.101) [AS6453 {GLOBEINTERNET}] 4 msec 4 msec 4 msec
  7 if-17-2.tcore1.L78-London.as6453.net (80.231.130.129) [AS6453 {GLOBEINTERNET}] 128 msec 108 msec 104 msec
  8 if-2-2.tcore2.L78-London.as6453.net (80.231.131.1) [AS6453 {GLOBEINTERNET}] 108 msec 108 msec 108 msec
  9  *  *  *
 10 if-12-6.tcore1.CT8-Chicago.as6453.net (216.6.99.46) [AS6453 {GLOBEINTERNET}] 152 msec 104 msec 104 msec
 11 if-22-2.tcore2.CT8-Chicago.as6453.net (64.86.79.1) [AS6453 {GLOBEINTERNET}] 104 msec 104 msec 104 msec
 12 if-3-2.tcore1.W6C-Montreal.as6453.net (66.198.96.45) [AS6453 {GLOBEINTERNET}] 112 msec 132 msec 104 msec

Traceroute results from Manchester, UK
to 216.98.48.56(lb-lsg-prod.ubisoft.com)
  1 ae-4-4.ebr1.London1.Level3.net (4.69.133.102) 8 msec 8 msec 4 msec
  2 ae-58-113.csw1.London1.Level3.net (4.69.153.122) 8 msec
    ae-59-114.csw1.London1.Level3.net (4.69.153.126) 8 msec
    ae-56-111.csw1.London1.Level3.net (4.69.153.114) 4 msec
  3 ae-116-3502.edge3.London1.Level3.net (4.69.166.134) 8 msec 4 msec
    ae-115-3501.edge3.London1.Level3.net (4.69.166.130) 8 msec
  4 ix-20-0.tcore1.LDN-London.as6453.net (195.219.83.101) [AS6453 {GLOBEINTERNET}] 8 msec 4 msec 4 msec
  5  *
    if-17-2.tcore1.L78-London.as6453.net (80.231.130.129) [AS6453 {GLOBEINTERNET}] 100 msec 112 msec
  6 if-2-2.tcore2.L78-London.as6453.net (80.231.131.1) [AS6453 {GLOBEINTERNET}] 100 msec 100 msec 120 msec
  7 if-20-2.tcore2.NYY-NewYork.as6453.net (216.6.99.13) [AS6453 {GLOBEINTERNET}] 104 msec 108 msec 108 msec
  8 if-12-6.tcore1.CT8-Chicago.as6453.net (216.6.99.46) [AS6453 {GLOBEINTERNET}] 100 msec 100 msec 116 msec
  9 if-22-2.tcore2.CT8-Chicago.as6453.net (64.86.79.1) [AS6453 {GLOBEINTERNET}] 100 msec 100 msec 120 msec
 10 if-3-2.tcore1.W6C-Montreal.as6453.net (66.198.96.45) [AS6453 {GLOBEINTERNET}] 100 msec 100 msec 116 msec
 11  *  *  *

It seems that plusnet are not perhaps getting the best that level3 can offer

[#13]

Is that route definitely going through Paris?  Or could it be that the hostname is incorrectly labelled? I get very low pings to 4.69.141.86 (less than 10ms) - pinging some of PN's IPs I get higher pings!
I notice that some Level3 and AS6453 router hostnames are "london" despite being located in the US.

[#14]

@Andy H if it was just it's host name  it wouldn't also add an extra 10ms to the base latency, FYI it today has  reverted back to how it was a few days ago  with no Paris showing  ping reduced by 10ms,  so the indirect route by level 3 has been sorted (for now) but my base latency is still higher than it should be due to AS6453.net and their failings, or the issue may lie with how level 3 actually peering with them currently,  because like Paris  Chicago shouldn't be appearing on a tracert to ubisoft it never has done in years  remove that and my base latency would drop back to 89-90ms again,
And if ubisoft release a new open world type shooter game (multiplayer ) peering companies are going to have to get their act together if this new game is going to use the same ubisoft server farm. for authentication and data ,otherwise they will be getting slated by more than only a few
Also  ISP's should have alternative peering agreements in place should there be issues with one peering provider

[#15]

Plusnet can't do anything about the AS6453 routing that far upstream. Really, Ubisoft should have servers in Europe.
Have you complained to them about the routing?