DNS/Routing issues for the last 36 hours

[#1]

"this morning"

Ah, if only it were just this morning..!

[#2]

Tracing route to facebook.com [173.252.120.68]
over a maximum of 30 hops:

1 <1 ms <1 ms <1 ms dsldevice.lan [192.168.1.254]
2 18 ms 17 ms 17 ms lo0.10.central11.pcl-bng03.plus.net [195.166.130.165]
3 22 ms 17 ms 24 ms irb.11.PCL-CR02.plus.net [84.93.249.98]
4 18 ms 18 ms 18 ms ae1.ptw-cr02.plus.net [195.166.129.2]
5 * * * Request timed out.
6 19 ms 18 ms 18 ms ae12.bb02.lhr2.tfbnw.net [74.119.78.98]
7 92 ms 92 ms 92 ms be32.bb01.iad3.tfbnw.net [31.13.26.182]
8 108 ms 108 ms 110 ms ae28.bb04.frc3.tfbnw.net [204.15.21.112]
9 108 ms 108 ms 107 ms ae4.dr02.frc3.tfbnw.net [31.13.29.169]
10 * * * Request timed out.
11 * * * Request timed out.
12 * * * Request timed out.
13 * * * Request timed out.
14 107 ms 107 ms 107 ms edge-star-mini-shv-12-frc3.facebook.com [173.252.120.68]

and

Tracing route to ebay.co.uk [66.211.181.235]
over a maximum of 30 hops:

1 1 ms <1 ms <1 ms dsldevice.lan [192.168.1.254]
2 18 ms 17 ms 18 ms lo0.10.central11.pcl-bng03.plus.net [195.166.130.165]
3 18 ms 50 ms 18 ms irb.11.PCL-CR02.plus.net [84.93.249.98]
4 18 ms 20 ms 18 ms 195.99.126.98
5 19 ms 18 ms 18 ms core3-te0-9-0-21.faraday.ukcore.bt.net [109.159.249.165]
6 18 ms 18 ms 18 ms 213.137.183.32
7 18 ms 18 ms 18 ms t2c4-xe-11-3-2-0.uk-lof.eu.bt.net [166.49.237.37]
8 35 ms 35 ms 35 ms t2c4-xe-2-1-1-0.de-fra.eu.bt.net [166.49.208.96]
9 35 ms 35 ms 35 ms t2c3-xe-1-1-1-1.de-fra.eu.bt.net [166.49.237.232]
10 35 ms 36 ms 35 ms 80.157.129.117
11 * * * Request timed out.
12 * * * Request timed out.
13 * * * Request timed out.
14 * * * Request timed out.
15 * * * Request timed out.
16 * * * Request timed out.
17 * * * Request timed out.
18 * * * Request timed out.
19 * * * Request timed out.
20 * * * Request timed out.
21 * * * Request timed out.
22 * * * Request timed out.
23 * * * Request timed out.
24 * * * Request timed out.
25 * * * Request timed out.
26 * * * Request timed out.
27 * * * Request timed out.
28 * * * Request timed out.
29 * * * Request timed out.
30 * * * Request timed out.

[#3]

As Whitesnake so prophetically sang:

"Here we go again ... "

[#4]

Not looking great from the other end either ......

12ms...... WUT

Array
(
    [1] =>  7  *
    [2] =>  8  BT-NET.ear2.London2.Level3.net (212.187.139.178)  12.219 ms
    [3] =>  9  213.137.183.103 (213.137.183.103)  12.426 ms
    [4] => 10  acc2-10GigE-0-2-0-0.l-far.21cn-ipp.bt.net (109.159.249.152)  12.383 ms
    [5] => 11  195.99.126.99 (195.99.126.99)  12.167 ms
    [6] => 12  te3-5.pcl-gw02.plus.net (212.159.0.215)  12.130 ms
    [7] => 13  link-b-central10.pcl-ag06.plus.net (212.159.2.183)  15.554 ms
)

You are currently connected to gateway pcl-ag06.

This is located in City Lifeline House.

[#5]

seeing lots of  request timed out returns in tracert and randomly getting page unavailable messages, so either there's a monster amount of traffic (like the other day when some muppets were hacking pokemon game) and stuff's getting delayed or there are serious problems on the BT backbone.

did some BT tech forget to feed the hamsters again ?

[#6]

UPS failure!

I think Not http://forums.theregister.co.uk/forum/1/2016/07/20/telecity_power_outage_bt_offline/

[#7]

@nanotm

From http://www.voipfonestatus.co.uk/incidents/qh874zmmzx3y

BT have confirmed there is a major outage in Telehouse North, London affecting our interconnect. Staff are on-site to resolve the issue as quickly as possible. We will keep you updated as soon as we have more information.

Posted 21 minutes ago. Jul 21, 2016 - 09:00 BST

So a different data centre today

[#8]

This is a lot worse than yesterday, it has already lasted longer. Yesterdays outage affected my connection for just under an hour and a half, this one is already over two hours.

Also, getting over 95% packet loss on the TBB BQM.

[#9]

This is getting ridiculous, being charged an extortionate amount for fibre broadband, that I cant use!!!

I cant access any sites other this one!!!

[#10]

---

30FTTC06 wrote:

You are currently connected to gateway pcl-ag06.
This is located in City Lifeline House.

---

I'm not that close to this morning's issues, however traffic on the L2TP network that runs through Telehouse is likely to have started routing through City Lifeline instead. The product of this is that there'll be a hell of a lot more traffic being pushed over those links. This will probably lead to congestion/further problems.

Things are probably not very pretty at the moment. Sorry.

[#11]

Not just Plusnet today. EE's painfully slow too so no escape to there for me today.

[#12]

I think all these issues are related to yesterdays and now todays BT problem in Docklands - power failure we are told - so much for single points of failure - now two day running - someone needs some more training   But wait - could it be  Europe pulling the plug in us - LOL

[#13]

Packet loss has dropped down to 2-3% and latency is around 20mS (about double the normal level).

[#14]

Literally while i've been on the phone with plusnet it's just died again for me.

[#15]

Looks like we are now on a BT network but the issues haven't ended:

Lost all my speed, get the full 38mb and 1.8mb normally

My latency has gone up by 10ms

I'm not rubbing anymore salt into the wounds of Plusnet I've had enough, I'm off.
Spoke to a great guy from Zen when I got in from work, it's a bit more money but heard lots of good things about them.