Downstream SNR varying wildly

[#1]

Indeed it is the TP-Link TD-W8970v1, and indeed those numbers only change when the line goes down, and is brought up again ...
And checking the Down Steam Attenuation - no that number does change, its stuck at 45.4dB

[#2]

Those graphs look interesting, but I would not bother with the max rate graphs.  They simply reflect a theoretical rate given the current noise level and available power - they therefore 'resemble' the SNRM plot and add no further value.
What will be interesting is to see if pattern is similar day to day.

[#3]

"Interesting" in the sence they will be a mare to find the cause ...
All I can thing of, is assume its some kind of broadband noise, then when I see the noise increase on the broadband, walk back to the cabinat, with a radio on, and wait for the noise to start.
What I find disapointing with issues like this, is I know there is an issue, and its clear to see - but the chances of getting BT/OR/BTWS to find the fault of the problem seems hard, even managing to contact them. Means that punters like me end up trying to find faults in the line outside my house, when I'm just an amature ....
Anyway, I'll leave the 1s logging on for a week, then I can get a clear picture of the time structure.

[#4]

That sadly is the nature of REIN/SHINE.  Invariably the cause of such issues is only present during certain hours, usually outside of core hours.
In the past I've seen cases in here caused by street lights, take-a-way extraction fans, free view TV boxes, PC screens... And personally I've had CCTV power units for IR cameras cause issues (after dusk they draw more power and the PSU goes pear shaped) and to top the lot, I could time teams passing on the WCML by the SNRM spikes they generated.
Frankly investigating the source of REIN/SHINE needs the level of effort and dedication which can only come from a dedicated amature!  You do appear to have the right approach to this.  Plot the problem, look for time based patterns and then get out and about with a radio and try to hunt down the source.
Do you have no access to a windows platform?  A view of the bits per tone plot with and without the problem might point to the frequency if the interference, which would give you a more precise point to tune your radio to.
Good hunting.

[#5]

I've only seen the bits per tone in the netgear Thomson router that was originally sents with the original plusnet router. With the TP-Link (which is a better device), I've not found any loggin that has detail like that. If I make no progress, may switch back to the netgear Thomson router.
Lets see, ADSL2+ only goes up to 2.2MHz. My SDR doesn't down that low, so have to use a radio. I've a cheap SW radio, where I can do a shed load of bands down to that area.
Anyway, gut feeling, its unlikly to be something just taking out one frequency, I expect broadband noise to be more likely ....
[Edited: Forgot that the plusnet provided router was a Thomson ... My netgear router is an even older beast, used for me to try and hack openwrt onto it via jtag ....]

[#6]

And attached is a HUGE graphic of the Down Stream Noise Margin. x axis is measured in days since 00:00 on 12/11/16.
So you can see the same daily structure, alhough the early morning flucation was for less time on the 14th than the 13th. So interference isn't completly regular.
I've updated my script, so its records each day to a seperate file, but still with 1s resolution. It also processes a fair bit, to make the file smaller ...

[#7]

During the hours of darkness, you can expect to see a drop of up to 2dB  due to AM radio interference.  Hi we're what you are seeing is quite extreme!

[#8]

Just did a speedtest - quite amusing results. I think I'll ignore it ...

[#9]

Indeed, but what do your router stats look like please?

[#10]

My NAS is still monitoring the router stats with 1s fidelity. New set up going well, but with lzma running on the output, it chews its way though a bit of memory. So may change script to just compress output at the end of the day, rather than having it run continously.
I'll hopefully tomorrow get graphs up of the day to day variation. So far all I can think off with the right time structure is steet lighting, e.g. 6pm=midnight, then comming back on at 4am-ish.
Still need to go out with the radio - I may manage it tonight. need to find that smoking gun ...
Chatting to the neigbours - they have the same problem with broadband, so its sounding common to us all. Seems I'm the most technically minded, so I'll be the one going out with the radio - don't think the other neighbous know the telephone route to the cabinet ...

[#11]

Good radio station I guess will be radio 5 on 909kHz, which is right in the middle of the adsl2 frequencies. Just walked back to the cabinet with it - when the line is quiet. Only noise was from the cabinet itself ...

[#12]

If the neighbours are really interested in fixing it and have "Windows" and a suitable router why not get them to install routerstats - the results are instantly available.
see my year long battle with rein. Not sure you have the right idea with using the radio - the Noise can be on any part of the adsl frequency range.
That said if you had the Bits graphs you can sometimes see which areas are most affected.

[#13]

I think when searching for sources of interference, the radio should be tuned away from any radio station. The radio station itself may reduce the bit loading on its frequency, depending on how near you are to the transmitter. Also, a fairly long line will probably be making only limited use of the higher frequencies, and not using the highest frequencies at all.
It is possible to extract more data from the 8970v1, there are two different methods.
1. The downstream error counts could be obtained by repeatedly running curl in a loop, which is how this thread started off. However, I found this method unreliable at best, and I really don't think it would withstand polling at one second intervals.
2. The other method involves "rooting" the router, by uploading a specially modified config file. It's easier to do it with the 8970 than the 9980, because you can just download, decrypt, edit, and upload the config file. This gets you telnet shell level access to the router, where you can get the bits and snr per tone data, amongst other things. The upstream stats still don't get recorded though, that's just how the firmware was built.

[#14]

Re: RouterStats - well neighbours aren't so PC literate. And anywy my simple script pulls all the info from the telnet cli that routerstats grabs, IIRC.
Re: TP-Link router. I *thought* the only way to get a root access to this router was via the UART, so interested to read on the other thread, that LAN3 doesn't have the telnet cli on it. So does it just have a standard telnet on it, so can telnet as root? I didn't go the UART direction btw, as didn't want to crack te case, and have to solder on  pins, on what is my operational router ...
Anyway I'll check th cli, to see if I can dig out error rates. Though that on he TP-Link on the telnet cli, that the command "adsl show info" gave all the useful info. My script just pulled the up/down steam stats from that data ....

[#15]

Which LAN socket you plug in to makes no difference in my experience. The router does contain the standard telnet server, but it is configured to only launch the TP-Link cli when you connect. But you can inject a command into the config file, to trick the router into running a second telnetd process, and obtain shell access.