AS6453.net peering sucks

[#1]

Quote from: paulmh5

Quote from: AndyH @ Kelly - What happens if packets are send via one route and come back another? Can this cause further problems? Yes and no.  It depends where it differs in paths and what devices it passes through, some boxes will track 'state' so need to see a complete flow.  Routers on the whole don't really care as long as it gets from A to B however trying to alter global routing in small parts without full end to end control could result in complications. In this example though, the /24 is only available via the DDoS scrubbing centre so it won't make any odds, its always going to have to go via that.

i get what you are saying , but at the moment it isn't taking a direct route London to Amsterdam ? when the  transatlantic  fibre is in the uk  ?  ubi are a joke And as said yesterday  during the breif window where verisign  disappeared  and i was routed level3 onto tinet @ london  i had no connectivity issues to ubi, can this be at least tried

[#2]

Quote from: deathtrap

......can this be at least tried

At the moment theres nothing we can do.  We only have routes to Verisign for that prefix:

phughes@ptw-cr01-re0> show route 216.98.48.56
inet.0: 613195 destinations, 1879555 routes (612498 active, 189 holddown, 515 hidden)
+ = Active Route, - = Last Active, * = Both
216.98.48.0/24     *[BGP/170] 20:28:22, MED 0, localpref 160
                      AS path: 7342 26415 I
                    > to 195.66.225.46 via ae5.0
                    [BGP/170] 20:28:22, MED 0, localpref 160, from 195.166.128.2
                      AS path: 7342 26415 I
                    > to 195.166.129.5 via ae2.0
                    [BGP/170] 20:28:21, localpref 40
                      AS path: 2856 7342 26415 I
                    > to 195.99.126.138 via ae11.0
                    [BGP/170] 20:28:07, MED 0, localpref 40, from 4.68.2.7
                      AS path: 3356 36622 36622 26415 I
                      to 217.163.45.249 via xe-2/3/0.0
                    > to 217.163.45.181 via xe-9/3/0.0

As you can see they all end in the same AS number.
Theres no guarantee to say traffic will take a transatlantic link from the UK.  Like I said previously the internet is a global thing, geography goes out the window when you reach a certain size even if it doesn't seem to make sense to the human at a PC.  In this case the DDoS traffic may have been seen to originate in Europe so thats where they have activated the mitigation, or it could just be where the hardware is located for the mitigation UBI use.  Cleaning the traffic before or after it crosses the pond won't make much difference.

[#3]

Oh ,this is for Andy H  just another nail in the theory that tinet spa where responsible for some BT customers not being able to connect to Uplay
http://forums.ubi.com/showthread.php/773183-Uplay-connection-issues/page41?s=e4be3e89595a13dcc0570bd... seems it's still happening  maybe it's the pathetic laggy DDos  malarky to blame, or as i have said BT's CGN and ubi's servers
https://community.bt.com/t5/BT-Infinity-Speed-Connection/BTInfinity-customers-problem/td-p/1072242/p...
@ paulmh5  this verisign lunacy  appears to have been rolled out globally , and the ubi server lobby is not surprisingly quite  next to no one  playing , compared to less than 2 weeks ago and they had lots of games in the lobby day & night  but ubi don't care they have taken our money  and don't want to properly support their games or customers

[#4]

@ deathtrap
I appreciate what you are saying, but that does not explain why BT customers are/were unable to access http://www.ubi.com
Despite the new routing change, BT Infinity customers are still without access. Ubisoft posted the following update yesterday (and clearly stated it is *not* a BT problem):

Quote

The issue currently affecting UK connections is not directly caused by Uplay or BT, rather it is a technical issue that has been encountered by our internet service provider. We are in close contact with our provider and have made them aware that resolving this issue is a top priority, but we do not have any additional information we can give out at this time. We understand that from the customer's point of view the specific cause of the issue is less important than having the issue resolved and we are working on getting everyone connected.

Maybe it might be worth one of the PN Network guys contacting Ubisoft to try and find a resolution for you?

[#5]

@ Andy H I don't have  a problem connecting to ubisoft.com  my problem is with increased latency , so how would PN contacting UBI  help my situation I can't see how it could/would?,
Not only that but Ubi support are quite useless, plus  that ubi statement  doesn't instil much trust in them even having a clue, because there isn't swathes of  people in the uk that are with other ISP's  seeing the same issue as those  customers of BT, infact i don't think there are any reporting this issue,
Not only this but from the posts by BT customers the games affected  appear to use Uplay  to connect,  So if what the ubi support say is correct  why has it only affected bt customers  and why are they  evading  giving the "specific cause" To me that says we don't have a clue, but maybe we will have one day  ubisoft are a useless company I still think it's down to bt's CGN maybe it's ubi's CDN  that doesn't like it for some reason , maybe ubi have dropped more than one clanger this year, wouldn't be a surprise

[#6]

Quote from: deathtrap

@ Andy H I don't have  a problem connecting to ubisoft.com  my problem is with increased latency , so how would PN contacting UBI  help my situation I can't see how it could/would?,

Because Ubisoft can investigate this and raise it with their peers? Ubisoft pay for peering, so there will be SLG/SLAs in place. If something is not performing as it should, then they need to be made aware. Ubisoft have a lot more control over your latency to them, than Plusnet do.
If Plusnet were able to speak to them, they might be able to get some more information on the routing via vrsn.net as to exactly why that was implemented etc.

Quote from: deathtrap

Not only that but Ubi support are quite useless, plus  that ubi statement  doesn't instil much trust in them even having a clue, because there isn't swathes of  people in the uk that are with other ISP's  seeing the same issue as those  customers of BT, infact i don't think there are any reporting this issue,

From the statement they Ubisoft issued, it would appear that an issue has been identified with their peers. I don't think it's fair to call them clueless when you don't know the full facts as to what has been found/why the problem has occurred.

Quote from: deathtrap

I still think it's down to bt's CGN maybe it's ubi's CDN  that doesn't like it for some reason , maybe ubi have dropped more than one clanger this year, wouldn't be a surprise

The CGN was apparently ruled out because customers with it turned off were still affected.

[#7]

Quote

Because Ubisoft can investigate this and raise it with their peers? Ubisoft pay for peering, so there will be SLG/SLAs in place. If something is not performing as it should, then they need to be made aware. Ubisoft have a lot more control over your latency to them, than Plusnet do. If Plusnet were able to speak to them, they might be able to get some more information on the routing via vrsn.net as to exactly why that was implemented etc.

No doubt that will have SLA's  with their peers or is that now peer , but getting ubisoft to do anything  even getting the speak to someone at that company with any idea  is nigh impossible .,If plusnet want to pursue this  fine by me, i wish them luck as they will need it, But i think that they probably already have the info on their peering

Quote

From the statement they Ubisoft issued, it would appear that an issue has been identified with their peers. I don't think it's fair to call them clueless when you don't know the full facts as to what has been found/why the problem has occurred.

they have previous form for being clueless , they can't even fix /finish their games  or run a stable server system  they don't even have a backup ,  we shall see if they do actually know  what this problem is or not, but we may have to wait a while for them to figure it out or call in someone who can fix it

[#8]

I have found out what i fist suspected to be true, that gaming  traffic to ubisoft  is being diverted to  Amsterdam NL  where verisign has it's scrubbing centre  So there is and extra 7-8ms  routing from london to that, then there is  this scrubbing cleaning  of already clean traffic    (only ubi can fail like this,) Then you have the part of the route back to ubi that is cloaked invisible unless you run a tcp ping  although it isn't that clear  which countries it actually transits  https://www.verisigninc.com/assets/datasheet-ddos-overview.pdf and here http://www.nl-ix.net/docs/verisign/faq-ddos-protection-services.pdf
All of which  has basically killed  their customers online enjoyment , so they may as well of let a DDos attack crash their servers because the games latency sensitive games are more or less unplayable anyway 

[#9]

Just a quick update  it looks like the DDos scrubbing  has stopped so the route has now reverted back to what it was AS6453.net via chicago , Can the direct  route to ubisoft  via tinet spa  be sorted  out,  this resembles  what i briefly saw  the other day  with the exception it went via level3 ,@ London & across the pond http://www.wipmania.com/trace/cache/lb-lsg-prod.ubisoft.com/?c=fc70e7b7196356f
not via Paris  like  some other ubisoft IP's are routed

Target Name: mdc-gs-stun02b.ubisoft.com
         IP: 216.98.48.125
  Date/Time: 15/03/2014 00:23:29 to 15/03/2014 00:29:38
Hop Sent Err   PL% Min Max Avg  Host Name / [IP]
 1    79  10  12.7   0   1   0  home.gateway.home.gateway [192.168.1.254]
 2    79   0   0.0  13 273  25  lo0-central10.ptw-ag01.plus.net [195.166.128.195]
 3    79   0   0.0  13  28  13  link-b-central10.ptw-gw02.plus.net [212.159.2.146]
 4    79   0   0.0  13  35  14  xe-4-2-0.ptw-cr02.plus.net [212.159.0.242]
 5    79   0   0.0  13  58  19  ae2.ptw-cr01.plus.net [195.166.129.4]
 6    79   0   0.0  13  36  13  ae1.pcl-cr01.plus.net [195.166.129.1]
 7    79   0   0.0  13  60  15  xe-11-1-0.edge3.London2.Level3.net [212.187.201.209]
 8    79   0   0.0  20  38  20  ae-3-3.ebr1.Paris1.Level3.net [4.69.141.86]
 9    79   0   0.0  19  22  20  ae-91-91.csw4.Paris1.Level3.net [4.69.161.90]
10    79   0   0.0  19  50  20  ae-4-90.edge5.Paris1.Level3.net [4.69.168.200]
11    79   0   0.0  19  32  20  ae5.par22.ip4.tinet.net [141.136.103.181]
12    79   0   0.0  97 150 102  xe-1-3-0.mtl10.ip4.tinet.net [89.149.184.74]
13    79  79 100.0   0   0   0   [-]

Destination not reached in 35 hops

 like this without the  level3 Paris hops , as this would provide the most direct route and the lowest latency  until  AS6453.net stops routing via Chicago ,cheers

[#10]

I have the actual same routing that i saw briefly the other day  level3 over tinet  So the NOC team can  compare  and hopefully  the route will be the same as the 2 scene grabs ( level3 onwards) Because one or more of the other ubisoft IP's the game uses for  things like ping time in the lobby (matchmaking) the custom cammo's used by some players  that are currently routed via tinet  but routed via paris using maybe congested links as the above tracert   is now showing a really big increase in latency  when routed this way via pairs

Target Name: mdc-gs-stun02b.ubisoft.com
         IP: 216.98.48.125
  Date/Time: 15/03/2014 16:04:25 to 15/03/2014 16:25:33
Hop Sent Err   PL% Min Max Avg  Host Name / [IP]
 1   247   0   0.0    0   0   0  home.gateway.home.gateway [192.168.1.254]
 2   247   0   0.0  13 269  19  lo0-central10.ptw-ag01.plus.net [195.166.128.195]
 3   247   0   0.0  13  95  13  link-b-central10.ptw-gw02.plus.net [212.159.2.146]
 4   247   0   0.0  13  77  17  xe-4-2-0.ptw-cr02.plus.net [212.159.0.242]
 5   247   0   0.0  13  52  13  ae2.ptw-cr01.plus.net [195.166.129.4]
 6   247   0   0.0  13  50  15  ae1.pcl-cr01.plus.net [195.166.129.1]
 7   247   0   0.0  13  64  14  xe-11-1-0.edge3.London2.Level3.net [212.187.201.209]
 8   247   0   0.0  20  38  20  ae-3-3.ebr1.Paris1.Level3.net [4.69.141.86]
 9   247   0   0.0  19  71  20  ae-91-91.csw4.Paris1.Level3.net [4.69.161.90]
10   247   7   2.8  19 100  21  ae-4-90.edge5.Paris1.Level3.net [4.69.168.200]
11   247   0   0.0  19  56  20  ae5.par22.ip4.tinet.net [141.136.103.181]
12   247   0   0.0 115 246 153  xe-1-3-0.mtl10.ip4.tinet.net [89.149.184.74]
Destination not reached in 35 hops

One of the screen shots attached shows routing to the same destination  but using the direct route from level3  onto tinet  
The latency levels in the screen grabs are normal.  where as  latency levels have increased  for plusnet users, So it seems to be congestion over the links that level3  is choosing for plusnet, why are they routing our traffic indirectly and over congested links ,
is this one of the detrimental differences between a bigger mass market ISP that plusnet is becoming  and one of the smaller isp's some who like to maintain  peering quality  ?
because the AS 6453.net  link  that goes via Chicago has increased  latency too, was from last night 108ms  currently is 132ms and glad that i don't want to play  the game at the moment, and if Level3  say nothing wrong then they are fobbing plusnet off  , as they can route traffic onto other less congested AS6453.net links and Tinet links   I believe the GBLX  is AKA global crossing ? which is owned by level 3 ,  deom what i have seen traffic routed via this GBLX  switches seems to escape the congestion, are these prioritised  or something,?  if so then they should be used for gaming traffic
The other question is does plusnet have to always use  level3  can't they peer direct with AS or Tinet , cogent , there are lots of alternative peering providers to level3
Any internet service /provider is only as good as it's peering , if that's poor .broken then so is the service or parts of it ,

[#11]

I get the same pings as in your screenshots...
Network Distance: 14 hops
TRACEROUTE (using port 443/tcp)
HOP RTT      ADDRESS
1  1.94 ms  router.asus.com (192.168.1.1)
2  6.59 ms  lo0-central10.pcl-ag02.plus.net (195.166.128.183)
3  5.93 ms  link-b-central10.pcl-gw02.plus.net (212.159.2.166)
4  9.53 ms  xe-10-1-0.pcl-cr02.plus.net (212.159.0.198)
5  5.94 ms  ae2.pcl-cr01.plus.net (195.166.129.6)
6  8.53 ms  xe-11-1-0.edge3.London2.Level3.net (212.187.201.209)
7  12.58 ms ae-3-3.ebr1.Paris1.Level3.net (4.69.141.86)
8  5.70 ms  ae-51-51.csw1.London1.Level3.net (4.69.139.88)
9  12.59 ms ae-3-80.edge5.Paris1.Level3.net (4.69.168.136)
10  5.79 ms  ix-20-0.tcore1.LDN-London.as6453.net (195.219.83.101)
11  90.40 ms xe-1-3-0.mtl10.ip4.tinet.net (89.149.184.74)
12  89.55 ms ubisoft-gw.ip4.tinet.net (173.241.128.42)
13  ...
14  89.91 ms mdc-gs-stun02b.ubisoft.com (216.98.48.125)

[#12]

Why did you use  port 443 ? the game doesn't use that port or ssl encryption  I get 98ms with a a tcp Ping on port 80 with fluctuations  because the route alternates between providers  AS6453.met and Tinet  as for lb-lsg-prod.ubisoft.com that route doesn't switch between providers and a TCP ping using  port 3074 which is the port the game uses is virtually the same as ICMP
It does seem to be an issue with xe-1-3-0.mtl10.ip4.tinet.net possibly  as that is the last hop that responds to icmp, but i can see it fluctuating  to silly pings  using tcp
a

[#13]

I'm using NMAP and it looks for open ports.
Starting Nmap 6.40-2 ( http://nmap.org ) at 2014-03-15 17:45 GMT
Nmap scan report for mdc-gs-stun02b.ubisoft.com (216.98.48.125)
Host is up (0.098s latency).
Not shown: 998 filtered ports
PORT    STATE  SERVICE
80/tcp  closed http
443/tcp closed https
TRACEROUTE (using port 80/tcp)
HOP RTT       ADDRESS
1   3.05 ms   router.asus.com (192.168.1.1)
2   183.50 ms lo0-central10.pcl-ag02.plus.net (195.166.128.183)
3   6.45 ms   link-b-central10.pcl-gw02.plus.net (212.159.2.166)
4   7.27 ms   xe-10-1-0.pcl-cr02.plus.net (212.159.0.198)
5   5.90 ms   ae2.pcl-cr01.plus.net (195.166.129.6)
6   5.91 ms   ae2.ptw-cr01.plus.net (195.166.129.4)
7   132.69 ms te-4-2.car5.London1.Level3.net (217.163.45.249)
8   6.50 ms   ae-51-51.csw1.London1.Level3.net (4.69.139.88)
9   14.37 ms  ae-1-60.edge5.Paris1.Level3.net (4.69.168.8)
10  18.53 ms  ae5.par22.ip4.tinet.net (141.136.103.181)
11  90.19 ms  xe-1-3-0.mtl10.ip4.tinet.net (89.149.184.74)
12  99.15 ms  if-2-2.tcore2.L78-London.as6453.net (80.231.131.1)
13  102.03 ms if-20-2.tcore2.NYY-New-York.as6453.net (216.6.99.13)
14  109.45 ms if-12-6.tcore1.CT8-Chicago.as6453.net (216.6.99.46)
15  90.61 ms  mdc-gs-stun02b.ubisoft.com (216.98.48.125)
The xe-1-3-0.mtl10.ip4.tinet.net router is in Montreal I think.
Do you have any Ubisoft IPs that respond to ICMP pings?

[#14]

No, none do. They ubi i believe prevented this some years ago after they got DDossed all because of the way they implemented their DRM on some games  according to reports about it on the net  So  we can no longer use  the built in windows tool  to see if their server is on-line or not
Yes Paris ,shouldn't be going that way, maybe level 3 should route via their GBLX switches in london that appear to (well they do for goscombe) then route directly onto tinet backbone to the usa , no faffing about  going here there and everywhere in-between, increasing latency because i would bet some of the interconnects are not fibre  also for some reason my traffic to many ubisoft IP's gets routed via lifeline ae1.pcl-cr01.plus.net before hitting the WWW ?? and Andy H's  goes via PTW ,  strange way of doing things  Why route this way,  if you are on a PTW  gateway then traffic  surely should route from that to the WWW no via PLC or vice versa
It only seems to be ubisoft IP's though, steampowered.com goes directly onto the www  , could this be part of the reason why  we don't get direct routing  once on the www as well ?

[#15]

I think Plusnet has 2 main peering partners: Level 3 and BT - http://bgp.he.net/AS6871#_graph4 and http://bgp.he.net/AS6871#_peers So I don't think peering directly with Cognet/TiNet is an option.
I used NMAP to see which Ubisoft IP addresses out of their ARIN IP range (216.98.48.0 - 216.98.63.255) respond to ICMP ping requests.

Quote

sh-3.2# nmap -sP -PI 216.98.48.0/20 Starting Nmap 6.40-2 ( http://nmap.org ) at 2014-03-16 10:04 GMT Nmap scan report for 216.98.54.42 Host is up (0.090s latency). Nmap scan report for mdc-mon-smok01.ubisoft.com (216.98.57.134) Host is up (0.090s latency). Nmap done: 4096 IP addresses (2 hosts up) scanned in 19.46 seconds

So you should be able to use 216.98.54.42 and 216.98.57.134 for ICMP pings.
I then tried using the Tinet/Level3/Cogent LGs (London servers) to see what the ping response times are like:
Tinet (Lon10)

Quote

--- 216.98.54.42 ping statistics --- 5 packets transmitted, 5 packets received, 0% packet loss round-trip min/avg/max/stddev = 87.728/89.210/90.581/0.955 ms

Quote

--- 216.98.57.134 ping statistics --- 5 packets transmitted, 5 packets received, 0% packet loss round-trip min/avg/max/stddev = 87.597/87.805/88.213/0.216 ms

Tinet (Lon21)

Quote

--- 216.98.57.134 ping statistics --- 5 packets transmitted, 5 packets received, 0% packet loss round-trip min/avg/max/stddev = 77.850/78.671/80.520/0.947 ms

Quote

--- 216.98.54.42 ping statistics --- 5 packets transmitted, 5 packets received, 0% packet loss round-trip min/avg/max/stddev = 78.095/78.952/80.868/0.981 ms

Level3 (Lon)

Quote

----  statistics ---- 5 packets transmitted, 5 packets received, 0% packet loss rtt min/avg/median/max/mdev/stddev = 92/94.4/96/96/1.386/1.96 ms

Quote

----  statistics ---- 5 packets transmitted, 5 packets received, 0% packet loss rtt min/avg/median/max/mdev/stddev = 92/104.8/92/152/4.345/23.651 ms

Quote

----  statistics ---- 100 packets transmitted, 100 packets received, 0% packet loss rtt min/avg/median/max/mdev/stddev = 92/124.32/92/392/6.842/58.125 ms

Level3 (Lon2)

Quote

----  statistics ---- 5 packets transmitted, 5 packets received, 0% packet loss rtt min/avg/median/max/mdev/stddev = 92/95.2/96/96/1.131/1.6 ms

Quote

----  statistics ---- 5 packets transmitted, 5 packets received, 0% packet loss rtt min/avg/median/max/mdev/stddev = 92/95.2/96/96/1.131/1.6 ms

Cogent

Quote

--- 216.98.57.134 ping statistics --- 5 packets transmitted, 5 received, 0% packet loss, time 4086ms rtt min/avg/max/mdev = 79.342/79.548/79.789/0.146 ms

Quote

--- 216.98.54.42 ping statistics --- 5 packets transmitted, 5 received, 0% packet loss, time 4085ms rtt min/avg/max/mdev = 79.382/79.454/79.570/0.365 ms

I am not sure what to read into all of this. Level3 Lon2 appears to be the route via Paris which your traffic with Plusnet takes - the Lon1 gave some high pings, so I assume that is why you do not get routed via the geographically common sense route?
As for Tinet - their two London servers are quite different in the results (10ms). Maybe this is where the problem is?