Routing to Opendns

[#1]

No idea, it's under there control.  In theory you'd send the traffic to the closest server.  That might not be the only reason though!

[#2]

I remind you of what Opendns support said, see my post #14.

Quote

The routing is decided by your ISP and we try to connect to the server closest to you, but if your ISP is routering through Frankfurt our system will see that and route you to the servers we have near that location.

It would be interesting to get level3's comments, then we can have 3 conflicting views

[#3]

Now, I don't know what I'm talking about with routing, but reading their response makes me think the person writing it knows less....  "routering?"

[#4]

Don't disagree with that. [edit Er having re-read this I think I need to clarify that it's opendns support which I believe are talking rubbish.]
IMO level3 may be influencing this.
Do we assume Plusnet are not actively doing anything to get this resolved?

[#5]

I don't think there is anything we can do to influence it (beyond directly peering with them, which we are looking at, but isn't just flipping a switch)
(out of my depth on routing though... so, I may be wrong, but checking)

[#6]

Quote from: Kelly

No idea, it's under there control.   In theory you'd send the traffic to the closest server.  That might not be the only reason though!

At least the trace route isn't showing hops to gchq.co.uk or prism.usa !   

[#7]

I've given up with opendns support, they don't appear to have a clue   all they do is keep blaming the ISP's for the routing.
Having asked in the opendns forum one knowledge user had this to say:

Quote

Yep, you got it! Level-3 is the culprit, they have suboptimal routing configured for those OpenDNS Anycast addresses. It seems OpenDNS is peering in London with network carrier tinet.net. Level-3 seems to attempt to keep the traffic in their own network instead of hand-over to Tinet, therefore routing on to the continent.

This sounds about right to me.
The following tracert confirms opendns peers with tinet.net.

Quote

1    1 ms    1 ms    1 ms  192.168.1.254   2    16 ms    18 ms    17 ms  lo0-central10.ptn-ag03.plus.net [195.166.128.192]   3    15 ms    16 ms    17 ms  link-b-central10.ptn-gw02.plus.net [212.159.2.138]   4    15 ms    15 ms    14 ms  xe-8-2-0.ptw-cr02.plus.net [212.159.1.34]   5    16 ms    15 ms    18 ms  ae2.ptw-cr01.plus.net [195.166.129.4]   6    16 ms    15 ms    17 ms  ae1.pcl-cr01.plus.net [195.166.129.1]   7    16 ms    16 ms    17 ms  xe-11-2-0.edge3.London2.Level3.net [212.187.201.213]   8    17 ms    18 ms    19 ms  ae-0-11.edge4.London2.Level3.net [4.69.200.126]   9    17 ms    15 ms    18 ms  ae-3-3.ebr1.London1.Level3.net [4.69.141.189] 10    22 ms    16 ms    15 ms  ae-58-113.csw1.London1.Level3.net [4.69.153.122] 11    51 ms    15 ms    19 ms  4.69.166.5 12  156 ms    16 ms    16 ms  IntelQ-Tinet-level3-2x10G.London.Level3.net [4.68.110.114] 13    17 ms    19 ms    15 ms  xe-1-0-1.lon10.ip4.tinet.net [89.149.185.226] 14  145 ms    20 ms    19 ms  rtr1.lon.opendns.com [208.69.34.1]

Can someone at plusnet please raise a ticket with level3 regarding routing to 208.67.222.222 and 208.67.220.220 (opendns) going to Amsterdam when it should be routed to London.
I don't see any other way this is going to be resolved.

[#8]

I'll ask Dunc tomorrow.

[#9]

A BGP announced address merely gives a route to an IP address.  A carrier like Level 3 uses that as one piece of information to decide what an 'optimal' route is.
I expect that Level 3 have 'costs' set up for their routes (actually, I don't expect, I know :P).  Their route to Amsterdam will likely have a lower 'cost' associated with it than the route via tinet.  (cost here isn't financial, it's merely a way of weighting routes.)  Keeping the traffic within Level 3 has a lower 'cost' than offloading to another network (where there may actually be peering charges which are financial costs).
They were bringing some of their links from London to Frankfurt offline last weekend, due to having to replace a broken fibre offshore from Folkstone, and they were alternately routing to Amsterdam - this is likely why the destination flipped from Frankfurt to Amsterdam.
B.

[#10]

Kelly - you may be able to favour an alternative route through LINX, rather than through your direct Level3 peer.  Depends on what routing protocols you have, but you could set a higher routing cost for OpenDNS via Level3 to nudge the traffic out on alternate carriers.
Of course, at the end of the day you will probably find the response time from Amsterdam is similar to London anyway, although GeoDNs might be slightly affected, it will likely be trivial.
B.

[#11]

Quote from: Barry

Kelly - you may be able to favour an alternative route through LINX, rather than through your direct Level3 peer.  Depends on what routing protocols you have, but you could set a higher routing cost for OpenDNS via Level3 to nudge the traffic out on alternate carriers.

Hi
Routing via public peering platforms is favored over our transit providers however we do not have a peering with OpenDNS over LINX.  I'm sure the Ops team will look into this however it depends if OpenDNS will want to peer and what processes are in place on their side as to how long that takes.  Trying to alter a global routing table on a per prefix basis becomes complicated at the size of the current table and often will come back to bite you when prefix sizes change as people run out of IPv4 addresses.  In this case, as we take the route via transit we don't currently have anywhere else to move the traffic to (apart from another transit provider).
Thanks

[#12]

Quote from: paulmh5

Trying to alter a global routing table on a per prefix basis becomes complicated at the size of the current table and often will come back to bite

Sound a bit risky, to fix what may be a temporary problem at level3.

[#13]

Did we get any movement on this yet Kelly ?
Tracing route to resolver1-fs.opendns.com [208.67.222.123]
over a maximum of 30 hops:
  1    23 ms    99 ms    99 ms  192.168.0.254
  2    22 ms    19 ms    18 ms  lo0-central10.pcl-ag05.plus.net [195.166.128.186]
  3    17 ms    18 ms    17 ms  link7-central10.pcl-gw01.plus.net [84.93.249.140]
  4    17 ms    17 ms    17 ms  208.core.access.plus.net [212.159.0.208]
  5    17 ms    17 ms    17 ms  xe-11-2-0.edge3.London2.Level3.net [212.187.201.213]
  6    22 ms    18 ms    17 ms  ae-0-11.edge4.London2.Level3.net [4.69.200.126]
  7    25 ms    24 ms    25 ms  ae-3-3.ebr1.London1.Level3.net [4.69.141.189]
  8    24 ms    25 ms    25 ms  vlan103.ebr2.London1.Level3.net [4.69.143.94]
  9    25 ms    25 ms    25 ms  ae-48-48.ebr2.Amsterdam1.Level3.net [4.69.143.81]
10    25 ms    24 ms    25 ms  ae-59-224.csw2.Amsterdam1.Level3.net [4.69.153.214]
11    25 ms    25 ms    25 ms  4.69.139.172
12    25 ms    25 ms    25 ms  OPEN-DNS-IN.edge6.Amsterdam1.Level3.net [212.72.47.62]
13    27 ms    27 ms    27 ms  resolver1-fs.opendns.com [208.67.222.123]
Trace complete.

[#14]

It may be a good idea to switch from opendns to norton dns for those wanting this type of protection.
https://dns.norton.com/dnsweb/huConfigurePc.do
At least that's routing to a London server.

[#15]

Cheers npr, i'll have a proper look later, not that i'm a big fan with anything with Norton in front of it    in all seriousness, it looks good at first glance.

traceroute to 198.153.192.60 (198.153.192.60), 30 hops max, 60 byte packets
1  192.168.0.254                               (192.168.0.254)    95.812 ms  95.775 ms  95.766 ms
2  lo0-central10.pcl-ag05.plus.net             (195.166.128.186)  59.640 ms  61.886 ms  61.884 ms
3  link8-central10.pcl-gw02.plus.net           (84.93.249.142)    21.747 ms  21.748 ms  21.749 ms
4  210.core.access.plus.net                    (212.159.0.210)    19.499 ms  21.731 ms  21.733 ms
5  ae2.pcl-cr01.plus.net                       (195.166.129.6)    32.989 ms  35.225 ms  35.224 ms
6  xe-11-2-0.edge3.London2.Level3.net          (212.187.201.213)  21.719 ms  17.814 ms  17.834 ms
7  ae-0-11.edge4.London2.Level3.net            (4.69.200.126)     20.051 ms  20.040 ms  20.029 ms
8  ae-3-3.ebr1.London1.Level3.net              (4.69.141.189)     20.019 ms  20.009 ms  17.679 ms
9  ae-59-114.csw1.London1.Level3.net           (4.69.153.126)     17.716 ms  19.937 ms  17.856 ms
10 ae-116-3502.edge3.London1.Level3.net        (4.69.166.134)     49.468 ms  47.216 ms  47.205 ms
11 Vlan533.icore1.LDN-London.as6453.net        (195.219.83.101)   28.936 ms * *
12 if-11-0-0-1254.mcore3.LDN-London.as6453.net (195.219.195.89)   17.851 ms  19.982 ms  19.979 ms
13 if-10-1-3.mse1.LRT-London.as6453.net        (195.219.195.22)   19.964 ms  19.980 ms  19.977 ms
14 195.219.100.10                              (195.219.100.10)   19.962 ms  18.235 ms  18.238 ms
15 198.153.192.60                              (198.153.192.60)   18.347 ms  18.224 ms  18.212 ms

traceroute -I -A 198.153.192.60
traceroute to 198.153.192.60                   (198.153.192.60), 30 hops max, 60 byte packets
1  192.168.0.254                              (192.168.0.254)   [AS28513]       100.111 ms  100.072 ms  97.821 ms
2  lo0-central10.pcl-ag05.plus.net            (195.166.128.186) [AS6871]         24.057 ms  26.291 ms  28.432 ms
3  link8-central10.pcl-gw02.plus.net          (84.93.249.142)   [AS6871]         19.684 ms  21.908 ms  21.915 ms
4  210.core.access.plus.net                   (212.159.0.210)   [AS6871]         21.901 ms  21.902 ms  21.895 ms
5  ae2.pcl-cr01.plus.net                      (195.166.129.6)   [AS6871]         21.900 ms  21.891 ms  21.893 ms
6  xe-11-2-0.edge3.London2.Level3.net         (212.187.201.213) [AS3356/AS9057]  21.890 ms  17.221 ms  17.941 ms
7  ae-0-11.edge4.London2.Level3.net           (4.69.200.126)    [AS3356]         20.176 ms  20.165 ms  20.157 ms
8  ae-3-3.ebr1.London1.Level3.net             (4.69.141.189)    [AS3356]         20.141 ms  20.135 ms  20.129 ms
9  ae-59-114.csw1.London1.Level3.net          (4.69.153.126)    [AS3356]         20.116 ms  20.110 ms  18.132 ms
10 ae-116-3502.edge3.London1.Level3.net       (4.69.166.134)    [AS3356]         18.095 ms  18.075 ms  17.545 ms
11 Vlan533.icore1.LDN-London.as6453.net       (195.219.83.101)  [AS6453]         30.734 ms * *
12 if-11-0-0-1254.mcore3.LDN-London.as6453.net(195.219.195.89)  [AS6453]         18.659 ms  20.908 ms  20.897 ms
13 if-10-1-3.mse1.LRT-London.as6453.net       (195.219.195.22)  [AS6453]         20.876 ms  20.866 ms  20.858 ms
14 195.219.100.10                             (195.219.100.10)  [AS25337]        20.839 ms  18.759 ms  18.845 ms
15 198.153.192.60                             (198.153.192.60)  [AS33517]        18.347 ms  17.848 ms  18.322 ms

Many Thanks