cancel
Showing results for 
Search instead for 
Did you mean: 

FTTC Issues - excessive resyncs, unusable internet and lacking speed

Ronski
Rising Star
Posts: 262
Thanks: 13
Registered: ‎22-02-2012

FTTC Issues - excessive resyncs, unusable internet and lacking speed

My FTTC was installed on the 20th August and I've had some problems since then. It should be noted I was the very first connection to the cabinet, so more connections may have caused cross talk, which could explain the gradual slow down, but not the initial slow speed for the distance, or the re-syncs and the other strange problem -  number 2 below.

1. Excessive modem re syncs
2. Two instances now where the internet has become unusable.
3. Far lower speed than the BT estimate, and that others get at around the same distance.

I'm roughly 450 meters from the cabinet, the estate was built in the mid seventies, the most logical route for the line which is all underground would be 450 meters, to go any other route by road would mean going around the outside of the estate. I don't believe the line is ally, although I can't be 100% sure on this.
Excessive modem re-sync's
   Here's a graph of the modem re-syncs, if you need a hand understanding the graphs then Alex is very well versed with them - he helped sort out Bald Eagles connection.
   You can see from the graph the longest the modem has remained in sync is about 90 hours, which I'm currently approach again at the moment.
   
   Here's a log of when I have disconnected or rebooted the modem, please note there was a couple of occasions prior to the time I started to keeping the log.
   26-08-2012 22:30    Approximately reboot modem due to ho internet
   04-09-2012 19:00    Disconnected modem cable - replaced filter and checked wiring
   09-09-2012 12:48    Changed modem
   09-09-2012 22:20    Rebooted modem - no internet since about 20:20, email would still check for messages
   12-09-2012 07:08    Change modem cable
    19-09-2012 19:00    Accidentally dislodged plug outlet whilst moving UPS
Internet becomes unusable
   I've had two instances that I'm aware of where the internet has become unusable, but I've still managed to send a test email, I've posted pictures below and also links to TBB ping graphs. On both these occasions FEC, HEC & OHF errors increased massively. I had to reboot the modem to return things to normal - note I've had this happen with two different modems (ones from ebay).
   26 August around 9pm
       
   
   9 September around 20:20 (Note this was on a different HG612 modem)
   We also have DS Error seconds graph as I'd updated the scripts used.
       
       
  The TBB ping graph for 9 September - the red line at about 1pm was where I changed the modem to another one I had. The period around 10pm is when the internet became unusable.


Actual connection speed
   The original BT estimate was 57/20, on checking today BT have lowered it to 47/11* see edit at bottom of post. Now others on the same length line are receiving a lot higher speed, some I've seen at up to 70 meg. Upload has also dropped considerably.So this low speed which seems to be dropping, would also indicate a problem somewhere.
   I have noticed on many occasions that the attainable rate was lower than the sync rate, Bald Eagle stated that he also saw this when he was experiencing connection problems.
   
   
Finally here's a graph showing my sync rate and attainable rate up and down for the last 27 days.

I have changed the filter plate, HG612 modem and also the RJ11 cable, and there appears to be no difference.
I've posted on the Kitz forum  and it would be worth reading that thread, some one has suggested it may be a fault in the DSLAM and I may need a lift & shift.
I have uploaded numerous graphs to Drop Box and I have a live TBB ping graph here
Sorry for the extremely long post but wanted to get as much information in as possible, I will also raise a ticket.
Edit: Should add I've tried a quiet line test and didn't hear any noise.
Edit: On checking this morning (17-09-2012), the BT speed estimate has returned to 57/20.
Quote
Our test also indicates that your line currently supports a fibre technology with an estimated WBC FTTC Broadband where consumers have received downstream line speed of 57Mbps and upstream line speed of 20Mbps.
28 REPLIES 28
Ronski
Rising Star
Posts: 262
Thanks: 13
Registered: ‎22-02-2012

Re: FTTC Issues - excessive resyncs, unusable internet and lacking speed

I forgot to add that I also run some tests from http://www.measurementlab.net/run-ndt, most notable is
Quote
3.242E-5 packets lost during test
185 selective acknowledgement packets received

Here's the full results
Your test results
Summary Details Advanced
Your system: Windows 7 version 6.1
Java version: 1.7.0_07 (x86)
TCP receive window: 261360 current, 261360 maximum
3.242E-5 packets lost during test
Round trip time: 29 msec (minimum), 88 msec (maximum), 44.91 msec (average)
Jitter: 59 msec
0 seconds spend waiting following a timeout
TCP time-out counter: 251
185 selective acknowledgement packets received
No duplex mismatch condition was detected.
The test did not detect a cable fault.
No network congestion was detected.
No network address translation appliance was detected.
0.7908% of the time was not spent in a receiver limited or sender limited state.
20.4% of the time the connection is limited by the client machine's receive buffer.
Optimal receive buffer: 267632640 bytes
0 duplicate ACKs set


WEB100 Kernel Variables: Client: localhost/127.0.0.1 CurMSS: 1452 X_Rcvbuf: 87380 X_Sndbuf: 676216 AckPktsIn: 15426 AckPktsOut: 0 BytesRetrans: 2904 CongAvoid: 11750 CongestionOverCount: 0 CongestionSignals: 1 CountRTT: 15241 CurCwnd: 258456 CurRTO: 251 CurRwinRcvd: 261360 CurRwinSent: 5888 CurSsthresh: 130680 DSACKDups: 0 DataBytesIn: 0 DataBytesOut: 45401600 DataPktsIn: 0 DataPktsOut: 30845 DupAcksIn: 183 ECNEnabled: 0 FastRetran: 1 MaxCwnd: 262812 MaxMSS: 1452 MaxRTO: 288 MaxRTT: 88 MaxRwinRcvd: 261360 MaxRwinSent: 5888 MaxSsthresh: 130680 MinMSS: 1452 MinRTO: 229 MinRTT: 29 MinRwinRcvd: 45264 MinRwinSent: 5840 NagleEnabled: 1 OtherReductions: 0 PktsIn: 15426 PktsOut: 30845 PktsRetrans: 2 RcvWinScale: 7 SACKEnabled: 3 SACKsRcvd: 185 SendStall: 0 SlowStart: 268 SampleRTT: 51 SmoothedRTT: 51 SndWinScale: 2 SndLimTimeRwin: 2050808 SndLimTimeCwnd: 7950685 SndLimTimeSender: 52122 SndLimTransRwin: 4 SndLimTransCwnd: 15 SndLimTransSender: 11 SndLimBytesRwin: 9481152 SndLimBytesCwnd: 35874112 SndLimBytesSender: 46336 SubsequentTimeouts: 0 SumRTT: 684402 Timeouts: 0 TimestampsEnabled: 0 WinScaleRcvd: 2 WinScaleSent: 7 DupAcksOut: 0 StartTimeUsec: 777958 Duration: 10053839 c2sData: 3 c2sAck: 3 s2cData: 4 s2cAck: 5 half_duplex: 0 link: 100 congestion: 0 bad_cable: 0 mismatch: 0 spd: 36.13 bw: 43.33 loss: 0.000032420 avgrtt: 44.91 waitsec: 0.00 timesec: 10.00 order: 0.0119 rwintime: 0.2040 sendtime: 0.0052 cwndtime: 0.7908 rwin: 1.9940 swin: 5.1591 cwin: 2.0051 rttsec: 0.044905 Sndbuf: 676216 aspd: 0.00000 CWND-Limited: 344.03 minCWNDpeak: 2904 maxCWNDpeak: 262812 CWNDpeaks: 1 The theoretical network limit is 43.33 Mbps The NDT server has a 330.0 KByte buffer which limits the throughput to 114.88 Mbps Your PC/Workstation has a 255.0 KByte buffer which limits the throughput to 44.40 Mbps The network based flow control limits the throughput to 44.65 Mbps Client Data reports link is 'Ethernet', Client Acks report link is 'Ethernet' Server Data reports link is 'T3', Server Acks report link is 'FastE'
Ronski
Rising Star
Posts: 262
Thanks: 13
Registered: ‎22-02-2012

Re: FTTC Issues - excessive resyncs, unusable internet and lacking speed

I've had a reply on my ticket (59931118), which basically says nothing is wrong - the above clearly shows something is though!
lexusuk
Grafter
Posts: 567
Registered: ‎20-10-2009

Re: FTTC Issues - excessive resyncs, unusable internet and lacking speed

Hi Ronski,
Thanks for contacting us.  I have updated your fault ticket with further details.
Ronski
Rising Star
Posts: 262
Thanks: 13
Registered: ‎22-02-2012

Re: FTTC Issues - excessive resyncs, unusable internet and lacking speed

Thanks Alex, ticket updated.
WWWombat
Grafter
Posts: 1,412
Thanks: 4
Registered: ‎29-01-2009

Re: FTTC Issues - excessive resyncs, unusable internet and lacking speed

That 1st graph seems to be showing 24 resyncs over 28 days, of which you mention 5 as manually-performed. Given that a "normal" undisturbed line can go for that period without any resyncs at all, I think this indicates that something is indeed wrong. Given that we know BT's DLM will drop your profile readily, and only very slowly recover it, you seem to have a problem that is not being handled well by DLM.
Your other graphs show a sudden burst of errors, both corrected and uncorrected, and a significant %age of packet loss. These suggest a good amount of interference over that time - but unfortunately, it also shows that you're suffering from an intermittant problem, which is the hardest to fix.
What would have been good to see is the behaviour of the SNR figures at the same time, and the results of the quiet line noise test at the same time.
Plusnet Customer
Using FTTC since 2011. Currently on 80/20 Unlimited Fibre Extra.
Ronski
Rising Star
Posts: 262
Thanks: 13
Registered: ‎22-02-2012

Re: FTTC Issues - excessive resyncs, unusable internet and lacking speed

Hi WWWombat, thanks for your reply, a full set of graphs including SNR figures are available on Drop Box You'll need to look in the appropriate folder - they are dated and have the duration of the graphs, match them in with the error pictures above.
Unfortunately I didn't think to do a quiet line test during the two episodes of excessive errors - I'll have to jot the number down and keep it handy if it happens again.
We now have an engineer coming Thursday PM, so we'll see what he finds
Edit:
Log pictures 26 August 2012
Log pictures 9 September 2012
WWWombat
Grafter
Posts: 1,412
Thanks: 4
Registered: ‎29-01-2009

Re: FTTC Issues - excessive resyncs, unusable internet and lacking speed

I've been taking a look at the stats, and have some responses to make. I'll do a few separate posts though, as my time availability is mixed....
First, on the subject of the stats & graphs, you need to be careful on some of the stats that come from the GUI versus those that come from the "--stats" command-line option.... there is a mismatch between the two - especially with FEC.
My belief is this:
- That HEC on GUI is probably correct, and matches the HEC from --stats
- That CRC on GUI is not really correct, as it shows "RS Uncorr" from --stats (which occurs when interleaving is turned on) but more likely should show "OHFErr" from --stats. However, even OHFErr seems to go wrong - when it doesn't show anything when interleaving is turned on.
- That FEC on GUI is incorrect, as it shows the OHFErr.
Important numbers:
- OHF shows the number of frames transmitted/received. These frames have CRC protection.
- OHFErr shows the number of errors occurring - ie a count of OHF frames with CRC errors.
- RS shows the count of Reed-Solomon frames. These are extra frames contained within the OHF frames, and offer an amount of "forward-error correction" by transmitting extra parity bits as an overhead, allowing the receiver to correct any faults. This mode is turned on at the same time as interleaving. Higher levels of interleaving contain more, smaller, RS frames that offer higher "correctability" at the expense of carrying more parity data as an overhead.
- RSCorr shows the count of RS frames that were faulty, but could be reconstructed from the parity data
- RSUncorr shows the count of RS frames that were faulty that could not be reconstructed from the parity data
Less-important numbers:
- FEC ought to be a count of "forward-error corrections", so should match  RSCorr. From above (definition of RS), this should only apply when interleaving is turned on, but the GUI value actually reflects OHFErr, which itself only seems to apply when interleaving is turned off.
- HEC is, I believe, some kind of error count within headers, and ought to be a subset of the CRC error count.
Any faults in the stats appear to be due to bad display by the modem, rather than the program that produces the graphs.
Plusnet Customer
Using FTTC since 2011. Currently on 80/20 Unlimited Fibre Extra.
Ronski
Rising Star
Posts: 262
Thanks: 13
Registered: ‎22-02-2012

Re: FTTC Issues - excessive resyncs, unusable internet and lacking speed

I believe Bald Eagle who produced the scripts is aware of the errors in the GUI and thanks for the info on what the graphs mean, it will be very helpful to get a better understanding.

Something odd just happened to my connection, was looking at my ping graph from work and noticed a resync arround 11:00am with an increase in ping - interleaving must have been increased, but the speeds have also altered, down has increased while up has decreased.
So I downloaded my log file from my server, loaded it into a database and found the following, look at the figures before and after 11:05

Date Time Sync Down Sync Up Attainable Down Attainable Up
18/09/2012 11:00 42684 10088 50908 11539
18/09/2012 11:01 42684 10088 50908 11305
18/09/2012 11:02 42684 10088 50908 11522
18/09/2012 11:03 42684 10088 51032 7413
18/09/2012 11:04 42684 10088 51156 7486
18/09/2012 11:05 49237 6445 57880 7948
18/09/2012 11:06 49237 6445 58492 6554
18/09/2012 11:07 49237 6445 58492 6537
18/09/2012 11:08 49237 6445 58492 6495
18/09/2012 11:09 49237 6445 60780 6629
18/09/2012 11:10 49237 6445 60720 8616
18/09/2012 11:11 49237 6445 60124 7258
18/09/2012 11:12 49237 6445 60104 7221
18/09/2012 11:13 49237 6445 59180 7355
18/09/2012 11:14 49237 6445 59196 7444
18/09/2012 11:15 49237 6445 59080 7379
18/09/2012 11:16 49237 6445 59088 7406
18/09/2012 11:17 49237 6445 59088 7327
18/09/2012 11:18 49237 6445 60884 7441
18/09/2012 11:19 49237 6445 60844 7458
18/09/2012 11:20 49237 6445 60936 7334
18/09/2012 11:21 49237 6445 60904 7337
18/09/2012 11:22 49237 6445 60880 7344
18/09/2012 11:23 49237 6445 60856 7355
18/09/2012 11:24 49237 6445 60720 7310
18/09/2012 11:25 49237 6445 60708 7382
18/09/2012 11:26 49237 6445 60696 7379
18/09/2012 11:27 49237 6445 60684 7389
18/09/2012 11:28 49237 6445 60788 7313
18/09/2012 11:29 49237 6445 60668 7313
18/09/2012 11:30 49237 6445 60776 7310
18/09/2012 11:31 49237 6445 59036 7221
18/09/2012 11:32 49237 6445 58120 7118
18/09/2012 11:33 49237 6445 58128 7076
18/09/2012 11:34 49237 6445 58252 7128
18/09/2012 11:35 49237 6445 58380 7128
18/09/2012 11:36 49237 6445 58380 7052
18/09/2012 11:37 49237 6445 58128 7052
18/09/2012 11:38 49237 6445 58252 7038
18/09/2012 11:39 49237 6445 58128 7063
18/09/2012 11:40 49237 6445 58252 7049
18/09/2012 11:41 49237 6445 58128 7032
18/09/2012 11:42 49237 6445 58128 7011
18/09/2012 11:43 49237 6445 58004 7025
18/09/2012 11:44 49237 6445 57880 7028
18/09/2012 11:45 49237 6445 58128 7011
18/09/2012 11:46 49237 6445 58380 7018
18/09/2012 11:47 49237 6445 58380 7008
18/09/2012 11:48 49237 6445 58256 7001
18/09/2012 11:49 49237 6445 58504 7032

Now looking at the log file in the database I can see what the higest and lowest figures are very easily:
Fastest attainable rate Down: 60936 today at 11:20
Fastest attainable rate Up:17672 on 29th August at 09:06
Fastest sync rate Down: 54533 04:55 to 21:09 on 23 August
Lowest sync rate Down: 42684 6:03 13 September  to 11:05 today
Fastest sync rate Up: 14377 09:06 29 August to 05:53 2 September
Lowest Sync rate Up: 6445 Today at 11:05
Suppose I'd better do some work  Wink
WWWombat
Grafter
Posts: 1,412
Thanks: 4
Registered: ‎29-01-2009

Re: FTTC Issues - excessive resyncs, unusable internet and lacking speed

The first set of graphs that I looked at were the 3 days up to 20:25 on the 29th August - which nicely shows the problem on the evening of the 26th *and* the aftermath.
Here's what I can see:
By the Uptime graph, I can see resyncs at around 22:00 on 26th, 04:00 on 27th, and 09:00 on 29th. The first was said to be manual, the other 2 automatic.
By the RS graph, I can see that you started with interleaving off, but that this was turned on in the resync at 04:00 on 27th. That implies this resync was a deliberate one triggered by DLM, which noticed the faults happening on the 26th. The "interleaving depth" graph shows the same.
By the OHF graph, I can see that with interleaving off, you were getting a frame rate of around 26500 frames/min, and about 27000 frames/min after interleaving was turned on. Looking carefully, you can also see a very subtle change to both frame rates at the 2 resyncs that didn't alter interleaving.
By the OHFErr graph, I can see that you were getting an error rate of around 26500 frames/min on the 26th, from 2100 until the resync. This must have been pretty close to 100% failure - but your TBB graph didn't show packet loss quite that high! It would certainly explain your observation of "no internet".
What caused it? It could have been a real problem on the line that (perhaps) the resync cleared, but could have been a fault in the modems at either end. The graphs would seem to suggest high external interference, but a manual resync wouldn't clear that. It would have to be a line fault that was (spookily) fixed by a resync, or a modem fault.
By the SNR graph, I can see that you get a small degree of drop in SNR in the evening - 1800 to midnight - but it occurs each night, and does not significantly drop (or vary) during the 2100-2200 period of the 26th. That suggests there was no additional noise on the line.
By the sync speed graph, I can see that the resync at 0400 on 27th caused a distinct loss of speed - but at the same time the attainable graph shows a higher maximum speed. The difference (a 2Mbps higher max, yet a 5Mbps lower actual) is probably being used by the extra parity overheads that are turned on when interleaving is turned on. Yes - it can eat this much of your line's capacity.
With interleaving on, what do we now see...
By the RSCorr graph, I can see some small groups of correctable errors occuring, especially in the evening of the 27th and 28th. However, that turns into a huge burst of errors (still correctable) from 2200 on the 28th, continuing through to 0900 on the 29th.
The corresponding RSUnCorr graph shows some faults here-and-there, but the most dense burst was on the evening of the 28th, starting around 1900.
Both behaviours stop at the resync of 0900 - with very few faults  occuring again until the evening.
This is very similar to the problem of the 26th, but with interleaving turned on, the errors are now caught in the RS (error correction) process, rather than the CRC process (never correctable). It is strange to see (again) such a high, pretty constant, error rate that just totally disappears at the resync. Is it external interference? Or a fault in the modem(s)?
It is possible that both sets of interference happen within certain, very limited, tones (frequencies), and that the resyncs (both the 2200 manual one and the 0900 automatic one) just stop using those particular tones for transmission of data. However, such a resync would *surely* result in a very different (lower) sync speed... and that happens in neither case! The only change in sync speed is the (very explainable) one when interleaving is turned on.
However, the SNR graphs doesn't show any kind of noise problem. Which means either there isn't any noise/interference, or the graph isn't managing to show it, or it doesn't appear in the particular stat that is being graphed.
What would I do?
- I'd start by suspecting the modems, but I wouldn't discount a line/noise/interference problem.
- On the modem front, I'd start by trying a replacement modem locally... and later, I'd bear in mind that a shift to another modem in the cabinet might be needed.
- On the line front, I'd try:
-- monitoring both the OHFErr stats (if interleaving is off) and the RSCorr stats (when interleaving is on) to look for similar behaviour.
-- when seen, check for audible noise on the line
-- gather the "current stats" graphs for "before", "during" and "after" for comparison. Obviously to get "before" ones requires you to take regular snapshots just in case - and you should do this after any further resyncs.
I'll check the other date later...
Plusnet Customer
Using FTTC since 2011. Currently on 80/20 Unlimited Fibre Extra.
Ronski
Rising Star
Posts: 262
Thanks: 13
Registered: ‎22-02-2012

Re: FTTC Issues - excessive resyncs, unusable internet and lacking speed

Thank you very much, makes for very interesting reading, wish I could interpret the graphs like that.
Did you see my post immediately before yours (Post 8 ) descibing this morning events?
With regard to the modem, I have tried two HG612 modems, the first was bought off ebay, the second, installed on 9 August at 12:48 was the one BT left when they did the install.
WWWombat
Grafter
Posts: 1,412
Thanks: 4
Registered: ‎29-01-2009

Re: FTTC Issues - excessive resyncs, unusable internet and lacking speed

Quote from: Ronski
Something odd just happened to my connection,

You're not kidding!
I guess the resync happened because the upstream could no longer be sustained against that large drop in the attainable value - which I think is calculated against a standard 6dB margin from the current noise level - and so indicates a lot of noise in the upstream tones for the 2 minutes before the resync.
The downstream change was probably more because of turning off interleaving, or changing the level.
This is the kind of step-change in speed I'd have expected (but didn't see) in my previous post.
Plusnet Customer
Using FTTC since 2011. Currently on 80/20 Unlimited Fibre Extra.
WWWombat
Grafter
Posts: 1,412
Thanks: 4
Registered: ‎29-01-2009

Re: FTTC Issues - excessive resyncs, unusable internet and lacking speed

Quote from: Ronski
With regard to the modem, I have tried two HG612 modems, the first was bought off ebay, the second, installed on 9 August at 12:48 was the one BT left when they did the install.

That probably rules out your modem, but not the cabinet one.
I also just took a look at the Kitz thread, and had a couple of other thoughts from that...
- I see it is an ECI cabinet. I thought such subscribers got a different modem from the Huawei
- Another consideration is your power supply....
- When you tried the new modem, did you change the power supply too?
- Is the modem supplied through your UPS? If so, can you try without for a while?
I also saw you were concerned about the consumer unit nearby, and possible electrical noise, so I wondered what you had done to simplify your setup back to the smallest possible setup, with as much physical separation of the components as you can get.
Plusnet Customer
Using FTTC since 2011. Currently on 80/20 Unlimited Fibre Extra.
Ronski
Rising Star
Posts: 262
Thanks: 13
Registered: ‎22-02-2012

Re: FTTC Issues - excessive resyncs, unusable internet and lacking speed

I have seen the ECI or Huawei debate mentioned many a time on fourms, and I think the genral consensus is that the Huawei will connect slightly faster than the ECI even when connecing to a ECI cabinet.
When I bought the HG612 from ebay it came with a black power supply (rated 0.7amps), even though it was a 3B version modem, I later found out that it should have a white power supply (rated 1amp). At some point I changed the black power supply to the white one. This was at some point during the first week, but can't remember when, posibbly the 23 August as I wanted to wait a couple of days before turning off the modem - didn't what to upset the Dslam.
Yes the modem is supplied through the UPS - this picture was taken about a week prior to the FTTC install
I can try it without the UPS, although I think I'll wait until after the engineers been.
I guess the electrical consumer unit is about 1.5 meters away the other side of the wall, here's a very quick sketch CS should be CU!

It would be a bit tricky to get seperation on everything, I have moved the DECT phone out into the hallway about 3 meters away, this was done Saturday morning about 07:45. I had previously tried using the phone whilst monitoring the connection and it appeared to make no difference, and we generally use mobiles for calls, and there no calls during the two strange episodes of errors.
Ronski
Rising Star
Posts: 262
Thanks: 13
Registered: ‎22-02-2012

Re: FTTC Issues - excessive resyncs, unusable internet and lacking speed

I've uploaded today's graphs to Drop Box, the SNR seems to do a wobbly after 11:00am, and a lot of DS_RScorr errors before 11:00am