Difficulty getting 8dB (Yes 8dB!) variation in Downstream SNR sorted out

[#1]

We have 2 ADSL lines taking an identical route through our building.  One is perfect with negligible SNR variations.  The other has variations of over 8dB in the space of a few minutes on ADSL2+ and about 2dB on basic ADSL.  The DMT tool suggests a large number of bitswaps regularly occurring on the extra 1MHz of spectrum for ADSL2+.
The modem does not drop the line when this happens - instead, the data dissolves into a huge volume of errors and effectively stalls.  So anything vaguely real-time, like web browsing or remote access is unusable (thank goodness this is not the line we use for VOIP).
We've been trying to get this sorted out for weeks and I'm not terribly happy with the support response - which includes a lot of promised phone calls which never came and considerable delay.  What is the business line SLA response time?  If there isn't one, what are we paying extra for?
BT eventually turned up - reset the profile and went away.  All this has achieved was to limit our upstream speed to 448Kbps - presumably they used a residential profile - well done!  Our remote access is now truly unusable.
We have substituted our Cisco modem with a modem with Broadcom chipset (a PlusNet supplied Voyager 2100) which at least means we can play tunes on the target SNR and use basic ADSL to get something slow but reasonably stable - but why should we do this when the other line right next door (literally inches away) is fine.
All of this information is on the support ticket with graphs from monitoring tools but it is being ignored.  We are told this is now a customer premises problem with noise (perhaps true but why is the other line fine?) and effectively Plusnet appear to be trying to let this problem go away because it is "too difficult".  If I ran my business like that I'd go bust!
I'm hoping this might at least get someone senior to look at this problem properly - I would prefer the support ticket system worked properly though.  I've been encouraging our IT guy to post here for weeks - but I think he would rather just move to his preferred supplier - perhaps that is what will happen - let's see.

[#2]

[quote=http://aaisp.net.uk/broadband-trial.html]Broadband We'll fix your line

---

Like most broadband lines we make use of links from BT to connect your broadband line to our network. This is a sensible way of providing a broadband service to the whole country, but does mean we have to deal with BT - particularly if and when things break.
We pride ourselves in having a very capable team of people that deal with faults. We have a close working relationship with BT, and our team are well known within all parts of BT for not tolerating being fobbed off.
We have detailed monitoring of lines that allow us to quickly identify faults and take action to resolve them (whether BT, or changes at your end).
In fact, we are so confident we know what we are doing, we'll even take on problems other ISPs leave behind, and that is why we make this offer.
Give us a month and we'll sort your problem broadband line or your money back.
If you are migrating your service to us, even though you know you have a problem with your broadband line, we'll take on the fault. We'll tackle the problem and get it fixed within one month. If we don't then you can migrate away and owe us nothing for your migration to us and your service charges for that month.
We'll even give you visibility of the dialogue we have with BT on the fault tickets we raise (via our control pages) so you can see what is going on.

[#3]

Can I ask which Cisco you are presently using?
The 8xx series and HWIC-ADSL1 card has a number of interop issues with the 21C network and in some cases, this can be resolved with a change to firmware.
If you can let us know which you are using, including your IOS in play, and the DSL microcode version you are using (version and if or not it is the embedded code from the IOS), we should have a clearer picture on what is happening.

[#4]

Hi Phil,
We're actually using a PlusNet supplied BT Voyager 2100 at the moment.
We swapped out our Cisco 857W.  That had reasonably current firmware since we tried various upgrades (in view of the problems you mention) before recognising that the line might be the problem once I did some detailed SNR monitoring with free tools (DMT  and Routerstats) via the Voyager.  
Because of the nature of the problem, modems seem not to go into a re-sync - presumably because the line recovers over the space of a few seconds.  Thus we have symptoms of data throughput falling to zero for tens of seconds every few minutes, without obvious modem stats telling us what was going on (until we did real-time SNR monitoring with very regular snapshots and looked carefully at the cumulative error counts - which are enormous).
I'll fire up the Cisco and see exactly what versions we were using.
BTW, the other line is using an 857 too.  Not sure if they're on the same code though.

[#5]

I understand my colleague Andy has just spoken with somebody there related to this.
He will be back in contact shortly as agreed to update a few other notes and advise on the further action we will take.

[#6]

Yes, he spoke to me (Andy) and then to our IT guy Adrian.
Now back on Cisco for your testing purposes.
Firmware is 4.0.018 but SNR is bouncing between 6dB and 0dB (don't know if the Cisco reports negative numbers like the Voyager).  And just getting this posted is unbelievably frustrating as the browser session keeps hanging.
We can't stay on the Cisco for long as we can't mitigate the line problems properly with this modem and our Internet is effectively unusable except for email.

[#7]

Thanks for the response.  Engineer coming out (again) tomorrow.
Back on the Voyager with the Broadcom chipset tweaked to give a 19dB target downstream SNR so we can actually use it for now.
And just so you can graphically see the problem - have a look at the attached plot

[#8]

Is there anything other than the router plugged in to any extension on that line? That's the sort of graph I'd expect to see with an unfiltered extension being used. Have you tried ringing the telephone while watching the noise margin?

[#9]

Hi Scottyork..
What you're seeing is intermittent radio-frequency interferance on that line, which could be being introduced anywhere from your premises back to the exchange, but as your other line is OK, that narrows it down if the two lines are routed similarly from your premises.
You best bet is to see if you can reproduce the routerstats log with a single PC connected to the router, in turn connected directly to the master socket test connector with no other wiring connected.  Take a plot with the existing set-up for an hour or more to show the problem - and then directly connected as described above.
If you get an improvement, connect the equipment to the router - it could be via the ethernet wiring - or look at equipment near the router and phone line, try powering down things with plug-top switching power supplies (the light-weight ones) and in your case, given the periodic drop in SNR, I would suggest looking for anything occasionally charging batteries, such as cordless phone base stations - or going on and off in a pattern, like a compressor (e.g. fridge or compressed air supply)
In recent months I have resolved four clients complete loss of ADSL caused by a faulty PC power supply (even in standby mode if only plugged in), a netgear GS105 Switch (the switch itself - not the PSU), a CRT monitor (20 feet from the router!) and a collection of plug-top PSUs crippling download speed.
Good luck - and please report what you find
Robin

[#10]

Yes, definitely time to do some more investigating on the RFI side.  
BT have been out and the line quality checks out well.  A "lift and switch" has taken place at the exchange and we have a new OpenReach filtered faceplate (this is an early ADSL installation from the days before self-install, when BT provided filtered faceplates), and a new cable to the modem from the faceplate.
There are no extensions on this line just a single SIP/PSTN gateway device plugged in to the voice side for VOIP backup - which hardly ever rings.  It has a filtered faceplate, and we have tried swapping this with a discrete filter and now have a new faceplate - all exhibit the same symptoms.
So we've got the same problem with two different modems (and different chipsets) equipment at both end ends of the line has been swapped and the cable checks out with BT meters.  Also, we are so close to the exchange, we have no cabinets between us and the exchange (no FTTC ever for us apparently :'().
After all that, still exactly the same problem.
It is probably not a great environment for electrical noise since the line terminates in a server room with lots of power supplies etc. nearby.  Substitution and turning devices off is going to be difficult in-hours.  However, I'm conscious the other line right next to it does not have the problem so can at least start with things unique to that particular router (directly-connected switches and the SIP/PSTN device etc.).
I'll let you know how it goes.

[#11]

Hi again,
Hmm - with a short line and no other equipment in that environment you are looking for something quite local to the equipment or the telephone line...
A longer routerstats plot(s) over a couple of days would be useful - along with a DMT plot if possible, this may well reveal the particular frequency of the RFI.  We use a reasonably good short wave receiver that goes down to 100KHz to look for specific interferance and narrow things down, which is quite easy once you can see the frequency concerned from the DMT plot.
If the pattern does repeat every few minutes as per your graph, then most constant power sources can be ruled out, but things that float charge batteries (maybe even a UPS) like alarms, cordless phones and such like - or intermittent noise sources like flashing lights in signs, air-conditioning, motors or nearby welding activity are all possible causes.
Even faulty streetlights constantly "starting" when they flash on and off can cause the exact thing you are seeing - our neighhbour was badly effected, but we wern't, due to a marginally different route of the PSTN cables.
If the problem is there 24x7 then ultimately you need to power off just about everything else you can for elimination - actually removing the power rather than turning off - as many devices do not properly power off otherwise.  Once you know the problem is not in your own equipment, then you can start looking else where.
Have you checked for similar problems with adjacent businesses / properties (not easy I know - but you may hit on something)
Good luck, RD

[#12]

Have done a bit of investigating and will do some more over the weekend.
DMT plot attached showing no particular nasties that I can see.  Would be interested to know if this looks typical for an ADSL2+ line.  Most bitswaps are occurring in the upper 1MHz spectrum which I imagine is typical.  This was taken after a re-sync to ADSL2+ - can't use this ADSL mode during office hours.  I've attached a sequence of consecutive plots showing the issue seeming to be broadband noise above 1MHz  (looking at changes in yellow SNR plot over time).
Have discovered that at 8Mb (basic ADSL) speeds our other line exhibits a downstream SNR of 15dB wheres the problem line is at 22dB.  Seems counterintuitive and perhaps suggests they do somehow take a different route back to the exchange.  
I've checked with neighbouring businesses and also have a tenants meeting on Friday so can get some more feedback then.  Nobody coming forward with the same problems but it took us some time to identify what was going on and recognise that our PCs weren't just very slow sometimes!
Current suspect is noise from a lift motor which sometimes correlates quite well with the reduced SNR episodes - but not always - so more investigation required there.

[#13]

Hi again Scottyork,
Your DMT plot looks good - apart from the notches around 900Khz, which are worth checking out with a radio if possible as the rest of the spectrum appears very clean and you're very close to the exchange - about 1.5 miles by the look of it.
Re: the other working line, Is the line loss on the other line similar? - is the other line ADSL2 as well? - and are they on the same make and model of router?
I would suggest taking a few DMT snapshots  to see if the interferance is moving about (frequency wise) - and again, a longer routerstats plot (over a few days) may reveal a lot about the pattern of when the problem is occuring...
Regards,
Robin

[#14]

Sorry Robin,
Missed your post.
Not getting much further here.  Nothing much to receive at 900 kHz on AM radio (at least nothing more than adjacent frequencies).
The other line is not on ADSL2 so I've been running the problem line at basic ADSL both for stability and for comparison purposes.  The comparison is interesting since the other line is rock-solid with variations of tenths of dBs whereas the problem line varies by up to 2dB on ADSL (and much more on the higher speed variants).
They have both been running on Cisco 857 routers with the same firmware, but for now I have the problem line running on a Voyager 2110 because it is easier to use that as a monitoring tool.  Cisco exhibits the same symptoms.  The other line is with Entanet but uses BT also for the local loop.
There is a significant difference between the lines however - the good line has a 15dB SNR whereas the problem line has  22dB SNR (both at full speed, basic ADSL).  This suggest to me that they take a different route somewhere.  I'm trying to get the pairs changed for the problem line because I'm spending an awful lot of time chasing this problem down and little effort seems to be coming from BT or PlusNet.  A pair change is, I think likely to solve the problem, but that is proving surprisingly difficult - selective deafness has set in.
Given that the BT engineer has just missed his third appointment (no messages to say why), I think I will just have to shout a lot louder - extremely unhappy customer here.

[#15]

Quote

Given that the BT engineer has just missed his third appointment (no messages to say why), I think I will just have to shout a lot louder - extremely unhappy customer here.

I'm sorry you're having this difficulty. I've flagged your post to Phil who manages our Business support team. Hopefully we'll get an explanation on why the engineer missed the appointment and then get this moved through to resolution.