cancel
Showing results for 
Search instead for 
Did you mean: 

SNR varying wildly

FIXED
ejs
Aspiring Hero
Posts: 5,442
Thanks: 631
Fixes: 25
Registered: ‎10-06-2010

Re: SNR varying wildly

You seem to be assuming that the stats you get from your modem are completely accurate, meaningful and instantly updated. I'm not sure that they will be, especially around a connection drop. You might not see any change in the counters for the upstream because the values of those counters have to be sent from the DSLAM to your modem, but you can't receive them, because the connection is down.

Wouldn't it be simpler to just get Plusnet to turn off the DLM and set a fixed target SNRM?

summers
Aspiring Pro
Posts: 275
Thanks: 50
Fixes: 1
Registered: ‎01-06-2014

Re: SNR varying wildly

@ejs ASDL routers are a mixture of digital and analogue technology. The analogue side has noise from the connection. The digital side, once digitised should be reliable. Now once digitised the processing *should* be accurate - but yes mine show occasional glitches - e.g. huge numbers that make no sense. These are rare and easily recognised.

Now why would my router suddenly go AWOL just before 4am, introduce its own FEC and CV errors in huge numbers, drag the line down and insist on a higher margin? Its the 4am thing, do you really believe a adsl router could have a digital bug that takes it down at 4am every morning.

Now the router is TP-LINK TD-W8970, you yourself had one of these. Are you saying it was a pile of cr*p?

To me the numbers are indicative of what the router is seeing. The stats are updated in response to what it sees. Yes there may be delays of a second, but why would a router see an error, do nothing about it, then record it at some later time period. That is acuasal. When a router says it has seen an error, it is saying that it sees what it calls an error in that period. Yes its the routers views, but like looking at the world through a set of glasses, whilst it distorts the world, the picture is basically good. Why would routers be otherwise?

ejs
Aspiring Hero
Posts: 5,442
Thanks: 631
Fixes: 25
Registered: ‎10-06-2010

Re: SNR varying wildly


@summers wrote:

Now the router is TP-LINK TD-W8970, you yourself had one of these. Are you saying it was a pile of cr*p?

With the TP-Link firmware, yes that is indeed my opinion of TP-Link. The router hardware itself is probably OK, the supplied microfilter looked poor.

 

Perhaps what I should have said that if the connection is knocked out by some huge burst of noise which is far more than the modem can handle, the noise level may be beyond what the modem can measure accurately. The exact values of the stats during the disconnection are not particularly important, the important aspect is the disconnection itself. The DLM would at some time, take the line down itself to apply a change to the target SNRM. I thought it was fairly common to see a spike in errors whenever a line disconnects for any reason, with no great significance assigned to that spike in errors.

summers
Aspiring Pro
Posts: 275
Thanks: 50
Fixes: 1
Registered: ‎01-06-2014

Re: SNR varying wildly

Well yes - it may well be that the data during a noise spike gets a bit erratic. But data is data, what clues it gives on reality should be looked at. So more data I can get before getting up at 3:30am every day is good. First need the script to relaiably pick up on the events, now I can see the commonality in the events.

With respect to the TP-Link firmware, well in some ways I agree, the DSP commands don't work quite as documented, how much though is down to the router, and how much interactions with the DSLAM I don't know, e.g. can one bit swap on plus net?

Its probably worth saying it isn't specifically an TP-Link issue, its that the SOC is a Lantiq device. The code that runs on the DSP for doing ADSL2+ isn't open source, but is general for the chip set. These days openwrt tends to use the DSP code from AVM fritzbox 7490 - just becuase they release it in an available form. best list is on: https://xdarklight.github.io/lantiq-xdsl-firmware-info/. The latest there is better than the stock one which came with the TP-Link, I'll have to check the version numbers:

Stock: 5.4.8.0.0.6 : 5.4.4.4.0.1
Openwrt: 5.8.1.8.1.6 : 5.8.0.B.1.1

Anyway is there any router DSP with open code for ADSL2+? I'm not aware of it. Is there any router which makes a better job of ADSL2+? Problem is they are small embedded computers built down to a price, but with massive IO including typical 4/5 ethernet ports as well as a DSP. Doing that for the price of a router is a challenge  ...

With microfilters I now use one in the BT socket. Before I used the ADSL Nation microfilers. As far as I can tell both are fine ...

summers
Aspiring Pro
Posts: 275
Thanks: 50
Fixes: 1
Registered: ‎01-06-2014

Re: SNR varying wildly

OK now trying another version of the code. It monitors downstream fec every second. If there is more than 500 fec in a second, then for the next minute it records the the SNR both up and down, this is recorded every second. To lower the load its saved raw, so will need post processing.

Every 10s I save the change in almost all other parameters, and margins, and rates etc.

A fec in a second of over 500 seemed from stats take so far, to be a very rare event, that I think has almost always been due to the line either on the way down, or down. 60s is usually enough for the line to come back, that seems to take about 40s after a drop - so in 60s I'll see what the line comes back as.

I think this is the *quickest* response I can do to the line probably going down. Other option would be just to record 10m from 3:40-3:50 am and see what I get ...

summers
Aspiring Pro
Posts: 275
Thanks: 50
Fixes: 1
Registered: ‎01-06-2014

Re: SNR varying wildly

OK - it has been over a week since the line last went down on the 16/11/18. The margin has dropped off to about 15dB and the line has been super stable. Over that time the training period probably came to and end.

First off a plot of 16-19 Nov

ADSLStats-16-19Nov18.png

You can see the reset of the line on the 16th at 4am. Whats strange is on 19th at ~17h the margin jumped by about 1.5dB - the line was totally stable at that time, stayed up, no errors. There was similar jumps on the 18th. Again with no errors.

Since the 19th the line has been monolithic:

ADSLStats-20-24Nov18.png

So stable, but at 15dB margin.

The error seconds have been very low. Downstream (where most errors have been) I've had less than 20 errors all day, so my meant time between errors is typically up at a couple of hours. A few days have had upsteam errors, the worst 37 errors in a day (24th) so a mean time between errors of 2335s.

So whats clear is that the highly varying margin I had two years ago has now disappeared.

However its also clear that the line wants to settle on 15dB margin. This is strange as it is so incredibly stable. Looking at the noise per tone, and that is also stable - few times there has been noise, we are only talking of a few dB on a range of frequencies - doesn't look like enough to take the line down - and doesn't explain the 15dB margin.

Now @Gandalf suggested raising a fault, and possibly an engineer visit. I'm not sure what good that will do, the line from my side seems very quiet, I'd be surprised if an engineer could find a fault. I don't know what the plusnet logs show, e.g. the radius log show on plusnet side. Also how do the error seconds look on the plusnet side?

So I think this takes us to @ejs suggestion. Is the best thing to just switch off training, and set the line to by had to a target margin. I suspect about 6dB would be the right level - that was what I had on the 11th Nov, which over 1 day gave 137 downstream error seconds, for a mean time between errors of 630s.

Anyone any other ideas?

 

Anoush
Aspiring Hero
Posts: 2,568
Thanks: 572
Fixes: 139
Registered: ‎22-08-2015

Re: SNR varying wildly

I’m afraid that switching off DLM isn’t something we’d do as it seems clear there appears to be a fault somewhere since your line is sacrificing speed for stability by increasing the noise margin.

If I remember correctly (I’m not in the office at the moment), from what the the radius logs can see, the connection was stable, likely because of the very high noise margin.

From what you’ve said if a noise margin of 6dB is generating ~137 errored seconds, this in itself is a problem in my opinion (Anything over 60 is far more than I’d like to see).

While there may not be any audible noise on the line that you can hear, it doesn’t necessarily mean that an engineer won’t find a fault or can’t investigate further. At the very least they’d be able to complete “quality gates” like renew the drop wire, change the D and E side, as well as arranging a lift and shift at the exchange in an effort to try to sort the issue out.
This is my personal Community Forum account to help out around these parts while I'm at home. If I'm posting from the 1st March 2020, this means I'm off-duty with no access to internal systems.
If this post resolved your issue, please click the 'This fixed my problem' button
summers
Aspiring Pro
Posts: 275
Thanks: 50
Fixes: 1
Registered: ‎01-06-2014

Re: SNR varying wildly

@Anoush well by the 14th Nov downstream margin was about 10dB, there were only 20 error seconds all day, so by then I was down to mean time between errors of 4320s. Yet the margin continued to drop down to 15dB.

This is why I'd be interested in what the logs on your side say, as whatever level we seem to say is reasonable, my line continues backing off on the margin, until 15dB is reached. So it seems clear that the DLM on your side still sees a problem, but I can't see it on my side.

So if you could check the logs when back in the office, I'd appreciate it.

Anoush
Aspiring Hero
Posts: 2,568
Thanks: 572
Fixes: 139
Registered: ‎22-08-2015

Re: SNR varying wildly

Happy to supply a radius log tomorrow, but whatever it shows, the fact that the SNR margin keeps on increasing from 6dB to a high level to reduce the errored seconds says that there’s an underlying problem somewhere.
This is my personal Community Forum account to help out around these parts while I'm at home. If I'm posting from the 1st March 2020, this means I'm off-duty with no access to internal systems.
If this post resolved your issue, please click the 'This fixed my problem' button
ejs
Aspiring Hero
Posts: 5,442
Thanks: 631
Fixes: 25
Registered: ‎10-06-2010

Re: SNR varying wildly


@Anoush wrote:
From what you’ve said if a noise margin of 6dB is generating ~137 errored seconds, this in itself is a problem in my opinion (Anything over 60 is far more than I’d like to see).

Per day? In my opinion 137 ES in a day is absolutely fine. It's about 6 ES per hour, one every ten minutes. Our line routinely clocks up a few hundred ES per day every day and this does not cause any problems and the DLM takes no action.

Perhaps it would be useful to look at some of the DLM's data (RRT Historic data graphs), and check the DLM is on the Standard stability profile. The modem stats don't seem to show any significant problems, and the DLM should be acting on the same data.

I'm also quite surprised at the idea that Openreach would swap what amounts to the entire line on a first visit if they don't find any particular fault, but I'm not in a position to know, maybe that's what they do these days.

Anoush
Aspiring Hero
Posts: 2,568
Thanks: 572
Fixes: 139
Registered: ‎22-08-2015

Re: SNR varying wildly

Per day? In my opinion 137 ES in a day is absolutely fine. It's about 6 ES per hour, one every ten minutes. Our line routinely clocks up a few hundred ES per day every day and this does not cause any problems and the DLM takes no action.

Fair. When we see errored seconds, our tests don't show if they're per day etc, but from what they've said that the MTBE at 630 you're probably right that it's not a huge amount to cause problems.

 

Perhaps it would be useful to look at some of the DLM's data (RRT Historic data graphs)

Will pull the RRT data up tomorrow.

 

and check the DLM is on the Standard stability profile.

 It's more than likely as we'd provision all circuits on that by default.

 

The modem stats don't seem to show any significant problems, and the DLM should be acting on the same data.

While that's true I'm a bit concerned that the SNR keeps increasing to a high level, which indicates there seems to be something underlying causing an issue.

 

I'm also quite surprised at the idea that Openreach would swap what amounts to the entire line on a first visit if they don't find any particular fault, but I'm not in a position to know, maybe that's what they do these days.

They may or may not, but we could try pushing for it on 2nd or subsequent visits. 

This is my personal Community Forum account to help out around these parts while I'm at home. If I'm posting from the 1st March 2020, this means I'm off-duty with no access to internal systems.
If this post resolved your issue, please click the 'This fixed my problem' button
Gandalf
Community Gaffer
Community Gaffer
Posts: 26,563
Thanks: 10,265
Fixes: 1,599
Registered: ‎21-04-2017

Re: SNR varying wildly

 

Connection log:

 

RRT Data:

 

Hope this helps.

From 31st October 2022, I no longer have a regular presence here as I’ve moved on to a new role.
Anoush Mortazavi
Plusnet
summers
Aspiring Pro
Posts: 275
Thanks: 50
Fixes: 1
Registered: ‎01-06-2014

Re: SNR varying wildly

Thanks @Gandalf, yes being solidly up for 8 days is what I saw. Last drop yoy show (3pm sat 17/11/18) I vaguely recall - think it resynced at the same speed as disconnected. Anyway I'll check my log when I get home. [Guess the problem with the radius graph is it just tells us that the line is stable at 15dB - this though isn't a surprise. So what it doesn't tell us is why the DLM ups the margin ....]

The RRT data I can't see, is the graphic correct? Looks like the graphic won't load ... ah its a link to https://www.btwholesale.com/RRT/GraphViewer which we can't see out here (via my *french* internet link ....)

Gandalf
Community Gaffer
Community Gaffer
Posts: 26,563
Thanks: 10,265
Fixes: 1,599
Registered: ‎21-04-2017

Re: SNR varying wildly

Not sure what happened to the graphs, I promise they were there when I first posted... Instead of a copy and paste, I've done it the old fashioned way and saved the files then uploaded them now:

Initialisations.jpgLine Attenuatoin.jpgLine Rate.jpgMTBE.jpgNoise Margin.jpgUptime.jpg

 

If you like I could perform a full SNR reset to see if that helps?

From 31st October 2022, I no longer have a regular presence here as I’ve moved on to a new role.
Anoush Mortazavi
Plusnet
summers
Aspiring Pro
Posts: 275
Thanks: 50
Fixes: 1
Registered: ‎01-06-2014

Re: SNR varying wildly

Thanks @Gandalf, suspect what happened is you could see the graphics as you were logged onto bt wholesale with your user account that showed the graphics, but we here aren't on with your account!

The Loss of Link seems the interesting graph to me, that does seem to tally with what I saw, something catastrophic took down the line, in times when otherwise the link looked fine with plenty of margin etc. Anyway IIRC the LoL was one of the parameters I record in my logs, so I'll  check that when I get home - to check it tallys.

Reset the SNR - yes I think that would be helpful. One thing that has changed in the past week is last weekend (~18/11/18) I changed the script monitoring, previously it monitored every 10s, but did a lot of the processing; and this put the router CPU at about 40% load. Last weekend I changed the script to do monitoring at 1s, but with almost no processing - and thius actually lowers the cpu load to about 5%. It also means that now I should be able to pick up a line going down, and record it -  the logs before showed a few seconds of growth of errors before the line when down - but didn't capture it. Now the new script *may* capture it. So yes if you could reset the SNR so it goes through another training signal, whilst i expect it will go back to 15dB - I'll hopefully get some more information on what is causing the loss of link ...

Thanks.