Hub Two crashes daily

[#1]

I've had PN service for 18 months without a single glitch or disconnection. However, for the last week the Hub Two router has crashed daily. By crashed I mean it stops passing traffic and does not respond to the web interface

I reported this as a fault and the diagnostics bot "refreshed" the connection to the hub but the hangs keep on coming and only a power cycle restores operation.

There is nothing in the log at the time of the hang - only normal activity on power up.

I would be grateful for advice about where to go next. Possibilities are a hardware problem requiring router replacement (how would I go about asking for this) or a factory reset which I would prefer to avoid having to re-enter port forwarding rules. Or can anyone suggest any other course of action please?  I'd like to resume the very high level of reliability that I've had for so long.

Many thanks

[#2]

@losip I don't know your experience level - yes I know you have been around a while - but posting your event log may give someone a chance to make suggestions, even if there are no indications of the fault that you can see.

[#3]

@jab1 Suggestion accepted.  Below is the log encompassing the last hang at around 08:15 today:

08:25:48, 31 Mar.
PPP LCP Receive Configuration Request

08:25:48, 31 Mar.
PPP CHAP Receive Challenge

08:25:48, 31 Mar.
PPP: LCP up

08:25:48, 31 Mar.
PPP LCP Receive Configuration ACK

08:25:48, 31 Mar.
PPP LCP Send Configuration ACK

08:25:48, 31 Mar.
PPP LCP Receive Configuration Request

08:25:47, 31 Mar.
PPP LCP Send Configuration Request

08:25:44, 31 Mar.
PPP: Received PADS

08:25:44, 31 Mar.
PPP: Sending PADR

08:25:44, 31 Mar.
PPP: Received PADO

08:25:44, 31 Mar.
PPP: Sending PADI

08:25:39, 31 Mar.
PPP: Starting PPP daemon

08:25:38, 31 Mar.
WAN Auto-sensing detected port Ethernet WAN

08:25:37, 31 Mar.
A device connected to Ethernet port 3

08:25:37, 31 Mar.
Wire Lan Port 3 up, Speed 1000 Mbps

08:25:35, 31 Mar.
LAN [ADD] ARP 192.168.2.1 with 60:be:b4:2b:96:c3 from br0(eth2)

08:25:34, 31 Mar.
ARP [add] br0(eth2) 192.168.2.1 60:be:b4:2b:96:c3

08:25:33, 31 Mar.
Booting firmware v0.10.00.04201-PN (Thu Apr 20 16:54:19 2023)

08:25:33, 31 Mar.
System start Button press (PowerButton)

08:25:33, 31 Mar.
System up

08:25:29, 31 Mar.
FWL 'Port Forward configuration added for Server (192.168.2.1) TCP/UDP 80:80 to TCP/UDP 80:80 by user'

08:25:26, 31 Mar.
5G WPS feature disabled

08:25:26, 31 Mar.
2.4G WiFi DOWN

08:25:26, 31 Mar.
5G WPS feature disabled

08:25:26, 31 Mar.
5G WiFi UP

08:25:19, 31 Mar.
WAN Auto-sensing last connection port: Ethernet WAN

08:14:15, 31 Mar.
FWL Port Forward Server(192.168.2.1) TCP 80 accepted a new connection from 64.62.156.192

08:14:15, 31 Mar.
FWL Port Forward Server(192.168.2.1) TCP 80 accepted a new connection from 64.62.156.192

08:14:13, 31 Mar.
FWL Port Forward Server(192.168.2.1) TCP 80 accepted a new connection from 64.62.156.192

08:14:13, 31 Mar.
FWL Port Forward Server(192.168.2.1) TCP 80 accepted a new connection from 64.62.156.192

08:13:39, 31 Mar.
FWL Port Forward Server(192.168.2.1) TCP 80 accepted a new connection from 64.62.156.192

08:12:46, 31 Mar.
FWL Port Forward Server(192.168.2.1) TCP 80 accepted a new connection from 64.62.156.192

08:12:32, 31 Mar.
FWL Port Forward Server(192.168.2.1) TCP 8123 accepted a new connection from 52.87.72.16

08:11:34, 31 Mar.
FWL Port Forward Server(192.168.2.1) TCP 80 accepted a new connection from 64.62.156.198

08:11:08, 31 Mar.
FWL Port Forward Server(192.168.2.1) TCP 80 accepted a new connection from 198.235.24.144

08:09:53, 31 Mar.
FWL Port Forward Server(192.168.2.1) TCP 80 accepted a new connection from 182.40.104.255

08:09:11, 31 Mar.
FWL Port Forward Server(192.168.2.1) TCP 80 accepted a new connection from 64.62.156.192

[#4]

I was hoping you would post the full log to do so, navigate to the page in the image attached, and then click 'export' and attach the resulting .csv file to a reply.

[#5]

Is this what you meant?

[#6]

@losip 

08:14:15, 31 Mar. FWL Port Forward Server(192.168.2.1) TCP 80 accepted a new connection from 64.62.156.192

08:12:32, 31 Mar. FWL Port Forward Server(192.168.2.1) TCP 8123 accepted a new connection from 52.87.72.16

Why do you have a port forward rules for port 80 and 8123, is 192.168.2.1 your Hub two?

If you disable the rules do you still get these hangs?

[#7]

These are incoming services. The hub is 192.168.2.254 and the upstream firewall is 192.168.2.1 accepting connections for a website (port 80) and Home Assistant (port 8123)

I'm loath to disable these rules because that disables the functionality of my system for at least a day. And I will point out that I've had 100% uptime for 18 months

The accepted connections you see are from shadowserver.org (port 80) and UptimeRobot (port 8123)

[#8]

@Dan_the_Van Can I leave this one with you? I  have to go out - which I hadn't planned, and don't know when I'll be back - probably late tonight.

[#9]

@losip I can see a number of manual hub restarts since the 28th March but prior to that none since 17th Feb ?

There's certainly a lot of incoming connections (port forwards) which is unusual but they dont seem to start until the early hours of this morning ?

[#10]

@MisterW Yes, that could be right. It started crashing 28th March at 17:09 according to my monitor logs and daily since (seems longer ago!!). The 17th Feb was an expected restart following loss of power (Hoover - don't ask!).  

I expect a number of incoming connections - that's what it's for - but why a daily time pattern I can only assume is Google bots/Shadowserver etc. UptimeRobot probes every 5 mins.

As you point out, nothing before 28th March so what has failed or changed?  Does it have the hallmarks of a memory leak/exhaustion so that a reboot restores operation for around a day?

[#11]

@losip 

I expect a number of incoming connections - that's what it's for - but why a daily time pattern I can only assume is Google bots/Shadowserver etc. UptimeRobot probes every 5 mins.

They're a lot more frequent than that at times

Have the port forwards always been there ? or is it a recent thing. They dont seem to be logged previously although there's

this in the log

<<<<<<<<<<<<<<<<<<<<

Limit

of

FW

log

>>>>>>>>>>>>>>>>>>>>

@Dan_the_Van any idea what that means ?

[#12]

@MisterW 

From my understanding some log categories have a fixed number of lines. Not sure how many line though

[#13]

Don't know what that means except I guess FW is Firewall. Maybe the firewall logs less?

AFAIK nothing has changed; the port forwards have been there for 18 months. I can afford to stop the port 80 forwards for a day or two. I'll try that now as a diagnostic but I do need to be able to handle these port forwards. Are you suggesting a DDNS attack starting 28th March? Does the Hub Two have any known traffic limitations on port forwarding?  Is it maybe the logging that's getting overloaded?

[#14]

@losip 

Don't know what that means except I guess FW is Firewall. 

Yes FW is firewall.

Does the Hub Two have any known traffic limitations on port forwarding?  Is it maybe the logging that's getting overloaded?

There's nothing that I'm aware of but that doesnt mean it doesnt exist!.

Disabling the port forwards for a while is a useful diagnostic.  Another option would be to try another router if you have access to one ? 

[#15]

@losip 

which full fibre product are you on,  have you considered using your own router?