cancel
Showing results for 
Search instead for 
Did you mean: 

DNS weirdness

198kHz
Seasoned Hero
Posts: 5,730
Thanks: 2,773
Fixes: 41
Registered: ‎30-07-2008

Re: DNS weirdness

I normally see very low numbers on my 2Wire 2700HGV's Firewall Monitor, but this is an extract of what it shows now. Is this relevant to the 'DNS weirdness' problem?
Top Attackers
Total Number of Attacks
IP Address       Attacker Domain     High Risk Med Risk Low Risk
212.159.6.10 ptn-cdns02.plus.net   0                     0             60
Attacks Blocked
                                                                    Today This Week
Total Number of Attacks                                 3               92
High Risk                                                         0                 0
Med Risk                                                         0                 0
Low Risk                                                         3               92
TCP Port Scan Detected, Packet Dropped     3                 5
UDP Port Scan Detected, Packet Dropped     0                 87
Murphy was an optimist
Zen FTTC 40/10 + Digital Voice   FRITZ!Box 7530
BT technician (Retired)
bluemonday
Newbie
Posts: 2
Registered: ‎08-07-2009

Re: DNS weirdness

I'm not that technical but am having the same intermittent problem. Rebooting the router sometimes solves it but at times it can happen again within a few minutes.
Have only been a Plus Net customer for a week so it is a pretty annoying start!
jamessealey
Grafter
Posts: 79
Registered: ‎10-08-2008

Re: DNS weirdness

Quote from: bluemonday
Have only been a Plus Net customer for a week so it is a pretty annoying start!

To be fair BlueMonday you have been very unlucky if this is your first week!!! I have been with PN for a year now, and aside from the crappy BT phone line coming into the house when I moved in I have had no issues, in fact I would argue that PN have been more than helpful when setting up my webspace etc.
HOWEVER - This is a very frustrating time for what seems like all of us. I had a drop out about 10 mins ago, and upon attempting to run a tracert to my primary DNS server the tracert just timed out at EVERY step. That may be something to do with my setup?
Being cynical (again), I am baffled as to why there has been no mention of this on the Service Status page, and brings me back to my initial thoughts last night that the only Service Status' we know about are the ones PN want us to know about.
For example, "We've found a problem", "It's gonna take an hour to fix" and "We've fixed it" Look great on a status page, but "We've got a problem, but can't find it or fix it - live with it" Doesn't look so good!
mikeyd
Rising Star
Posts: 120
Thanks: 14
Fixes: 1
Registered: ‎04-02-2008

Re: DNS weirdness

Quote from: jamessealey
Being cynical (again), I am baffled as to why there has been no mention of this on the Service Status page...

It's there on the Service Status page, and it says they are going to update it in, er, an hour ago..  http://www.plus.net/supportpages.html?a=2
Since being terrible this morning everything has been fine since I got home at 4pm.  Seems very intermittent with different people affected at different times.
jamessealey
Grafter
Posts: 79
Registered: ‎10-08-2008

Re: DNS weirdness

Quote from: mikeyd
Quote from: jamessealey
Being cynical (again), I am baffled as to why there has been no mention of this on the Service Status page...

It's there on the Service Status page, and it says they are going to update it in, er, an hour ago..  http://www.plus.net/supportpages.html?a=2

Interesting, I have the RSS feed of the status page in my iGoogle, and have been avidly watching it all day in work. It is only now it is showing up! Strange!
bobpullen
Community Gaffer
Community Gaffer
Posts: 16,869
Thanks: 4,950
Fixes: 315
Registered: ‎04-04-2007

Re: DNS weirdness

Quote from: pd
This hasn't started recently, it's been going on for weeks.  I first raised a ticket on 16th June - not that that was the start of it - it just got worse then, but I gave up with it as the responses were less than helpful.  The problem is at it worst at the busiest times of the day, mid-morning, late afternoon/early evening and again just after midnight.  I think the increased iPlayer traffic has also had a large part to play in it.

I don't think it's directly related to the increase in iPlayer traffic although you may be onto something when you mention that it's worse when the network's busy.
We've been monitoring the caching servers that I spoke about in my last reply and have noticed that:

  • They seem to deal with a noticeably larger number of requests than the caches at our second site (the site you're routed to is mainly dependent on the gateway you connect to).
  • Because of this the CPU usage is much larger at the first site when compared to the second.
  • We're seeing requests sent to the PCL caches being responded to and dumps of the UDP traffic show customer requests being responded to.
  • Since we restarted the DNS service on 3 of the affected servers earlier on, none of them have failed in the load balancer. We've also now done the same with the fourth server.

We could have a memory leak somewhere, this could be a resourcing issue (thus being more frequent at the busiest times of the day), or it could be something else. One thing we certainly don't want to do though is chuck more kit at the platform because if this isn't a resourcing problem then we'll just be delaying the inevitable. Given the intermittent nature of the problem and the length of time some customers have been reporting problems for, we'd much rather have confidence that we've identified the true cause of the issues.
Quote from: jah
These two nmap scans illustrate that there is serious latency whilst performing reverse DNS resolution for the hosts between me and the DNS servers ptn-cdns01 and ptn-cdns02 during the first scan. Note that the latency for ICMP responses is very short.
The second scan is after I dropped and reconnected the ADSL connection and shows a marked improvement in DNS resolution times (2.59s compared with 19.58s):

Your first scan was hitting the site we think's having problems whilst the second one wasn't. The clue's in the load balancer your traces to the DNS caches are hitting.
This is the first site (the one we think has problems):
Quote from: jah
TRACEROUTE (using proto 1/icmp)
HOP RTT  ADDRESS
1  0.00  me
2  31.00 lo0-plusnet.pcl-ag02.plus.net (195.166.128.39)
3  31.00 gi1-2-904.pcl-gw02.plus.net (84.92.0.141)
4  31.00 vl63.pcl-lb01.plus.net (212.159.2.252)
5  31.00 ptn-cdns01.plus.net (212.159.6.9)

And this is the second site (the one that we think should be OK):
Quote from: jah
TRACEROUTE (using proto 1/icmp)
HOP RTT  ADDRESS
1  0.00  me
2  31.00 lo0.plusnet.thn-ag1.plus.net (195.166.128.101)
3  32.00 vl703.thn-gw1.plus.net (84.92.4.241)
4  32.00 ae0-102.ptn-gw01.plus.net (212.159.1.53)
5  32.00 vl55.ptn-lb01.plus.net (212.159.2.124)
6  31.00 ptn-cdns01.plus.net (212.159.6.9)

Quote from: mikeyd
Quote from: jamessealey
Being cynical (again), I am baffled as to why there has been no mention of this on the Service Status page...

It's there on the Service Status page, and it says they are going to update it in, er, an hour ago..  http://www.plus.net/supportpages.html?a=2

Sorry about that, I'll update it now. This post is more comprehensive than what I'll be posting there mind you.
Basically, things looked to be ticking over nicely earlier. I take it people are still having problems then?

Bob Pullen
Plusnet Product Team
If I've been helpful then please give thanks ⤵

rammie1976
Grafter
Posts: 571
Registered: ‎02-09-2007

Re: DNS weirdness

Its okay here - I'm still on Juniper 10 - and has been since almost after I posted that I didn't think rebooting had fixed!  Lips_are_sealed  Grin
mikeyd
Rising Star
Posts: 120
Thanks: 14
Fixes: 1
Registered: ‎04-02-2008

Re: DNS weirdness

Thanks for the update Bob.  Explanations of what is going on is what I like about Plusnet.
djadamuk
Newbie
Posts: 9
Registered: ‎08-07-2009

Re: DNS weirdness

This is why I love being a plus net customer. People that know what they are doing giving you honest valuable explanations, spot on guys thanks!!
I was having the same intermittent issues, my DNS servers were set to automatically obtain and were obtained 212.159.6.9 & 212.159.6.10 as the DNS addresses. I've now switched to manual and am on these addresses: 212.159.13.49, 212.159.13.50. Not sure if this is the correct procedure but it seems to have worked so far, touch wood.
**Update** spoke too soon, still having issues.
Quote
C:\Users\Adam Streather>tracert 212.159.13.49
Tracing route to 212.159.13.49 over a maximum of 30 hops
  1    1 ms    1 ms    1 ms  192.168.0.1
  2    39 ms    39 ms    37 ms  lo0-plusnet.thn-ag3.plus.net [195.166.128.80]
  3    42 ms    32 ms    40 ms  84.92.3.177
  4    32 ms    31 ms    32 ms  ae0-102.ptn-gw01.plus.net [212.159.1.53]
  5    44 ms    33 ms    37 ms  vl55.ptn-lb01.plus.net [212.159.2.124]
  6    35 ms    45 ms    55 ms  pth-cdns01.plus.net [212.159.13.49]
Trace complete.
Tracing route to pth-cdns02.plus.net [212.159.13.50]
over a maximum of 30 hops:
  1    1 ms    2 ms    2 ms  192.168.0.1
  2    50 ms    38 ms    32 ms  lo0-plusnet.thn-ag3.plus.net [195.166.128.80]
  3    41 ms    35 ms    33 ms  vl303.thn-gw1.plus.net [84.92.3.177]
  4    52 ms    47 ms    32 ms  ae0-102.ptn-gw01.plus.net [212.159.1.53]
  5    45 ms    45 ms    45 ms  vl55.ptn-lb01.plus.net [212.159.2.124]
  6    37 ms    47 ms    57 ms  pth-cdns02.plus.net [212.159.13.50]
Trace complete.
Tracing route to ptn-cdns01.plus.net [212.159.6.9]
over a maximum of 30 hops:
  1    1 ms    2 ms    2 ms  192.168.0.1
  2    51 ms    51 ms    39 ms  lo0-plusnet.thn-ag3.plus.net [195.166.128.80]
  3    33 ms    32 ms    32 ms  vl303.thn-gw1.plus.net [84.92.3.177]
  4    32 ms    35 ms    37 ms  ae0-102.ptn-gw01.plus.net [212.159.1.53]
  5    33 ms    39 ms    47 ms  vl55.ptn-lb01.plus.net [212.159.2.124]
  6    48 ms    39 ms    35 ms  ptn-cdns01.plus.net [212.159.6.9]
Trace complete.
Tracing route to ptn-cdns02.plus.net [212.159.6.10]
over a maximum of 30 hops:
  1    1 ms    1 ms    1 ms  192.168.0.1
  2    35 ms    40 ms    34 ms  lo0-plusnet.thn-ag3.plus.net [195.166.128.80]
  3    33 ms    34 ms    32 ms  vl303.thn-gw1.plus.net [84.92.3.177]
  4    35 ms    32 ms    31 ms  ae0-102.ptn-gw01.plus.net [212.159.1.53]
  5    38 ms    32 ms    32 ms  vl55.ptn-lb01.plus.net [212.159.2.124]
  6    46 ms    32 ms    32 ms  ptn-cdns02.plus.net [212.159.6.10]
Trace complete.

bobpullen
Community Gaffer
Community Gaffer
Posts: 16,869
Thanks: 4,950
Fixes: 315
Registered: ‎04-04-2007

Re: DNS weirdness

If our engineers are on the right trail then you could try this...
Open a command prompt (assuming you're using Windows) and run the ipconfig command to find out what DNS server addresses you're being assigned (there'll be loads more info but it's the DNS Servers you're after):
C:\Users\Bobby>ipconfig /all
DNS Servers . . . . . . . . . . . : 212.159.13.49
                                    212.159.6.10

Next run a traceroute to one of the DNS server addresses:
C:\Users\Bobby>tracert 212.159.13.49
Tracing route to pth-cdns01.plus.net [212.159.13.49]
over a maximum of 30 hops:
  1     2 ms     1 ms     2 ms  home.gateway [192.168.1.254]
  2    18 ms    21 ms    16 ms  lo0.plusnet.thn-ag2.plus.net [195.166.128.93]
  3    19 ms    18 ms    16 ms  vl604.thn-gw1.plus.net [84.92.4.169]
  4    16 ms    17 ms    17 ms  ae0-102.ptn-gw01.plus.net [212.159.1.53]
  5    61 ms    17 ms    16 ms  vl55.ptn-lb01.plus.net [212.159.2.124]
  6    17 ms    19 ms    23 ms  pth-cdns01.plus.net [212.159.13.49]
Trace complete.

What you're looking for is the load balancer that your traceroute passes through. This will be labelled either pcl or ptn. It's the pcl site we reckon is causing the problems so if your hitting that, disconnect, reconnect and repeat the above process until you hit the ptn site. I'd be interested to know if there are customers having problems whose DNS requests are actually going to the 'good' ptn site.
e.g.
Good:
 5    61 ms    17 ms    16 ms  vl55.ptn-lb01.plus.net [212.159.2.124]

Bad:
 5    34 ms    23 ms    29 ms  vl63.pcl-lb01.plus.net [212.159.2.252]

Those two are hop 5 for me but the give away is the lb notation that stands for 'load balancer'.
Edit: looks like that might not work if @djadamuk's traces are anything to go by above Sad

Bob Pullen
Plusnet Product Team
If I've been helpful then please give thanks ⤵

jamessealey
Grafter
Posts: 79
Registered: ‎10-08-2008

Re: DNS weirdness

Quote from: Bob
Sorry about that, I'll update it now. This post is more comprehensive than what I'll be posting there mind you.
Basically, things looked to be ticking over nicely earlier. I take it people are still having problems then?

No worries Bob, I can appreciate that you can be alot more in depth and focussed on here than you can on the service status page, I was just concerned as nothing had been mentioned at all!
Re your load balancer theory I have just had another period of being unable to resolve any host names, and upon running a tracert I am infact running through pcl-lb01, so i'll attempt a couple of disconnects to see if I can get on the ptn one you mention. Then I shall monitor things.
Additonally, I have been using my dynamically assigned DNS servers since day one (212.159.6.9 / .6.10) but I have seen alot of people referring to using others, 212.159.13.50 from memory. Are these just addtional DNS servers, or does it matter which ones we use?
Cheers
James
bobpullen
Community Gaffer
Community Gaffer
Posts: 16,869
Thanks: 4,950
Fixes: 315
Registered: ‎04-04-2007

Re: DNS weirdness

Shouldn't really matter which ones you use James as it has little bearing on the site your traffic goes to.
Edit: Yay! My post count's just hit 5,000! Cool

Bob Pullen
Plusnet Product Team
If I've been helpful then please give thanks ⤵

djadamuk
Newbie
Posts: 9
Registered: ‎08-07-2009

Re: DNS weirdness

Quote from: Bob
Edit: looks like that might not work if @djadamuk's traces are anything to go by above Sad

I've run my traces again, all very similar responses. I just had another period of being unable to resolve any host names, similar to @jamessealey, but I am running through ptn-lb01. Not meaning to puzzle yourself or your engineers anymore.
Just for note in case its any help, looking at James's post. I was running on the dynamically assigned addresses (212.159.6.9/.10) and was ever since joining, have now switched to 212.159.13.49/.50 and still experiencing the same issues. However I found that the outage times are usually shorter.
dannykos
Grafter
Posts: 204
Registered: ‎07-08-2007

Re: DNS weirdness

well I can't seem to even see which load balancer i'm on!!
traceroute to 212.159.6.10 (212.159.6.10), 64 hops max, 40 byte packets
1  192.168.0.1 (192.168.0.1)  1.681 ms  6.913 ms  1.168 ms
2  lo0.plusnet.thn-ag2.plus.net (195.166.128.93)  28.285 ms  41.309 ms *
3  vl604.thn-gw1.plus.net (84.92.4.169)  31.785 ms  30.122 ms  40.345 ms
4  ae0-102.ptn-gw01.plus.net (212.159.1.53)  25.284 ms  25.515 ms  26.389 ms
5  * * *
6  * * *
7  * * *
8  * * *
traceroute to 212.159.6.9 (212.159.6.9), 64 hops max, 40 byte packets
1  192.168.0.1 (192.168.0.1)  1.760 ms  1.956 ms  1.123 ms
2  lo0.plusnet.thn-ag2.plus.net (195.166.128.93)  32.056 ms  28.207 ms  42.259 ms
3  vl604.thn-gw1.plus.net (84.92.4.169)  113.884 ms  46.501 ms  31.297 ms
4  ae0-102.ptn-gw01.plus.net (212.159.1.53)  69.853 ms  114.646 ms  34.786 ms
5  * * *
6  * * *
7  * * *
8  * * *
fyi - i'm on the the ADSL2+ trial, so can't even gateway hop Sad
nblackburn
Dabbler
Posts: 24
Registered: ‎08-07-2009

Re: DNS weirdness

I'm still having problems with host resolution, I hope this helps.  A router reboot solves it but only for a few minutes:
My DNS servers were 212.159.6.9 and 212.159.6.10.
C:\Documents and Settings\Desktop User>tracert 212.159.6.9
Tracing route to 212.159.6.9 over a maximum of 30 hops
  1    <1 ms    <1 ms    <1 ms  192.168.0.1
  2    30 ms    33 ms    38 ms  195.166.128.64
  3    30 ms    30 ms    40 ms  84.92.4.2
  4    50 ms    30 ms    45 ms  212.159.1.105
  5    42 ms    31 ms    37 ms  212.159.2.124
  6    31 ms    34 ms    31 ms  212.159.6.9
Trace complete.
C:\Documents and Settings\Desktop User>tracert 212.159.6.10
Tracing route to 212.159.6.10 over a maximum of 30 hops
  1    <1 ms    <1 ms    <1 ms  192.168.0.1
  2    37 ms    30 ms    30 ms  195.166.128.64
  3    32 ms    34 ms    32 ms  84.92.4.1
  4    31 ms    38 ms    35 ms  212.159.0.185
  5    30 ms    32 ms    43 ms  212.159.2.252
  6    32 ms    31 ms    43 ms  212.159.6.10
Trace complete.
C:\Documents and Settings\Desktop User>nslookup bbc.co.uk 212.159.6.9
DNS request timed out.
    timeout was 2 seconds.
*** Can't find server name for address 212.159.6.9: Timed out
Server:  UnKnown
Address:  212.159.6.9
DNS request timed out.
    timeout was 2 seconds.
DNS request timed out.
    timeout was 2 seconds.
*** Request to UnKnown timed-out
C:\Documents and Settings\Desktop User>nslookup bbc.co.uk 212.159.6.10
DNS request timed out.
    timeout was 2 seconds.
*** Can't find server name for address 212.159.6.10: Timed out
Server:  UnKnown
Address:  212.159.6.10
DNS request timed out.
    timeout was 2 seconds.
DNS request timed out.
    timeout was 2 seconds.
*** Request to UnKnown timed-out