cancel
Showing results for 
Search instead for 
Did you mean: 

Possible routing issue? Can't connect through to specific site.

ErTnEc
Grafter
Posts: 40
Thanks: 3
Registered: ‎02-07-2015

Possible routing issue? Can't connect through to specific site.

I'm trying to get to the bottom of a possible routing issue where I can't seem to connect through to our local hospitals NHS website.  I've noticed this has been an issue a few times before, and I thought nothing of it until recently when I was trying to get some specific information.

The website in question is https://www.boltonft.nhs.uk/

Any attempt to connect to it (curl, chrome, any other browser) simply results in a timeout.  Here it the output from curl and traceroute when ran within my network:

root@raspi01 ~ # traceroute -I www.boltonft.nhs.uk
traceroute to www.boltonft.nhs.uk (213.104.98.149), 30 hops max, 60 byte packets
 1  254.core.plus.net (195.166.130.254)  13.194 ms  13.203 ms  13.177 ms
 2  84.93.253.115 (84.93.253.115)  13.468 ms  13.700 ms  13.707 ms
 3  core1-BE1.southbank.ukcore.bt.net (195.99.125.130)  13.151 ms  13.191 ms  13.664 ms
 4  peer3-et-0-0-2.redbus.ukcore.bt.net (62.172.103.240)  13.754 ms  13.777 ms  13.781 ms
 5  109.159.253.63 (109.159.253.63)  20.649 ms  20.696 ms  20.733 ms
 6  * * *
 7  bagu-core-2a-ae10-0.network.virginmedia.net (62.254.84.2)  25.504 ms  23.142 ms  23.385 ms
 8  bagu-metnet-3b-lag-56.network.virginmedia.net (82.8.124.178)  23.168 ms  23.247 ms  23.272 ms
 9  bnft-bl4-ia1.network.virginmedia.net (213.104.213.126)  24.374 ms  24.743 ms  24.755 ms
10  * * *
11  * * *
12  * * *
13  * * *
14  * * *
15  * * *
16  * * *
17  * * *
18  * * *
19  * * *
20  * * *
21  * * *
22  * * *
23  * * *
24  * * *
25  * * *
26  * * *
27  * * *
28  * * *
29  * * *
30  * * *


root@raspi01 ~ # curl -v -m30 https://www.boltonft.nhs.uk/ -o out
* Expire in 0 ms for 6 (transfer 0x671950)
* Expire in 30000 ms for 8 (transfer 0x671950)
* Expire in 1 ms for 1 (transfer 0x671950)
  % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
                                 Dload  Upload   Total   Spent    Left  Speed
  0     0    0     0    0     0      0      0 --:--:-- --:--:-- --:--:--     0* Expire in 1 ms for 1 (transfer 0x671950)
* Expire in 2 ms for 1 (transfer 0x671950)
* Expire in 1 ms for 1 (transfer 0x671950)
* Expire in 1 ms for 1 (transfer 0x671950)
* Expire in 1 ms for 1 (transfer 0x671950)
*   Trying 213.104.98.149...
* TCP_NODELAY set
* Expire in 200 ms for 4 (transfer 0x671950)
  0     0    0     0    0     0      0      0 --:--:--  0:00:29 --:--:--     0* Connection timed out after 30001 milliseconds
  0     0    0     0    0     0      0      0 --:--:--  0:00:30 --:--:--     0
* Closing connection 0
curl: (28) Connection timed out after 30001 milliseconds

If I access this via a different connection (in this case route out via VPN, then the site returns just fine with no issues:

root@raspi01 ~ # traceroute -T www.boltonft.nhs.uk
traceroute to www.boltonft.nhs.uk (213.104.98.149), 30 hops max, 60 byte packets
 1  10.35.0.1 (10.35.0.1)  17.397 ms  17.383 ms  17.502 ms
 2  te-3-3-4006.pe3.man4.uk.m247.com (217.64.114.161)  18.457 ms  19.779 ms  19.908 ms
 3  xe-1-2-1-0.core1.man4.uk.m247.com (83.97.21.144)  18.941 ms vlan2902.bb1.fra2.de.m247.com (82.102.29.128)  20.146 ms xe-2-1-0-0.core1.man4.uk.m247.com (77.243.185.12)  20.035 ms
 4  te-12-3-0.core-dc2.man4.uk.m247.com (83.97.21.70)  47.011 ms te-13-4-0.core-dc2.man4.uk.m247.com (77.243.176.47)  47.090 ms te-12-3-0.core-dc2.man4.uk.m247.com (83.97.21.70)  46.992 ms
 5  te-5-8-0.bb1.man2.uk.m247.com (77.243.185.137)  20.013 ms  20.061 ms te-6-5-0.bb1.man2.uk.m247.com (77.243.185.1)  20.102 ms
 6  tcma-ic-2-xe-210-0-0.network.virginmedia.net (212.250.14.189)  20.118 ms  18.890 ms  18.910 ms
 7  * * *
 8  bagu-core-2a-ae10-0.network.virginmedia.net (62.254.84.2)  20.681 ms  20.534 ms  20.497 ms
 9  bagu-metnet-3b-lag-56.network.virginmedia.net (82.8.124.178)  20.556 ms  20.569 ms  20.592 ms
10  bnft-bl4-ia1.network.virginmedia.net (213.104.213.126)  20.860 ms  20.891 ms  20.795 ms
11  91-187-250-212.static.virginm.net (212.250.187.91)  20.805 ms  25.048 ms  25.142 ms
12  149.98-104-213.static.virginmediabusiness.co.uk (213.104.98.149)  24.953 ms  25.062 ms  25.025 ms

root@raspi01 ~ # curl -v -m30 https://www.boltonft.nhs.uk/ -o out
* Expire in 0 ms for 6 (transfer 0xa74950)
* Expire in 30000 ms for 8 (transfer 0xa74950)
* Expire in 1 ms for 1 (transfer 0xa74950)
  % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
                                 Dload  Upload   Total   Spent    Left  Speed
  0     0    0     0    0     0      0      0 --:--:-- --:--:-- --:--:--     0* Expire in 1 ms for 1 (transfer 0xa74950)
* Expire in 2 ms for 1 (transfer 0xa74950)
* Expire in 1 ms for 1 (transfer 0xa74950)
* Expire in 1 ms for 1 (transfer 0xa74950)
* Expire in 1 ms for 1 (transfer 0xa74950)
*   Trying 213.104.98.149...
* TCP_NODELAY set
* Expire in 200 ms for 4 (transfer 0xa74950)
* Connected to www.boltonft.nhs.uk (213.104.98.149) port 443 (#0)
* ALPN, offering h2
* ALPN, offering http/1.1
* successfully set certificate verify locations:
*   CAfile: none
  CApath: /etc/ssl/certs
} [5 bytes data]
* TLSv1.3 (OUT), TLS handshake, Client hello (1):
} [512 bytes data]
* TLSv1.3 (IN), TLS handshake, Server hello (2):
{ [122 bytes data]
* TLSv1.3 (IN), TLS handshake, Encrypted Extensions (8):
{ [21 bytes data]
* TLSv1.3 (IN), TLS handshake, Certificate (11):
{ [3030 bytes data]
* TLSv1.3 (IN), TLS handshake, CERT verify (15):
{ [264 bytes data]
* TLSv1.3 (IN), TLS handshake, Finished (20):
{ [52 bytes data]
* TLSv1.3 (OUT), TLS change cipher, Change cipher spec (1):
} [1 bytes data]
* TLSv1.3 (OUT), TLS handshake, Finished (20):
} [52 bytes data]
* SSL connection using TLSv1.3 / TLS_AES_256_GCM_SHA384
* ALPN, server accepted to use http/1.1
* Server certificate:
*  subject: C=GB; L=Bolton; O=Bolton NHS Foundation Trust; CN=*.boltonft.nhs.uk
*  start date: Jan  7 00:00:00 2022 GMT
*  expire date: Jan  5 23:59:59 2023 GMT
*  subjectAltName: host "www.boltonft.nhs.uk" matched cert's "*.boltonft.nhs.uk"
*  issuer: C=US; O=DigiCert Inc; CN=DigiCert TLS RSA SHA256 2020 CA1
*  SSL certificate verify ok.
} [5 bytes data]
> GET / HTTP/1.1
> Host: www.boltonft.nhs.uk
> User-Agent: curl/7.64.0
> Accept: */*
>
{ [5 bytes data]
* TLSv1.3 (IN), TLS handshake, Newsession Ticket (4):
{ [265 bytes data]
* TLSv1.3 (IN), TLS handshake, Newsession Ticket (4):
{ [265 bytes data]
* old SSL session ID is stale, removing
{ [5 bytes data]
< HTTP/1.1 200 OK
< Date: Thu, 28 Jul 2022 12:39:49 GMT
< Server: Apache/2.4.41 (Ubuntu)
< Link: <https://www.boltonft.nhs.uk/>; rel=shortlink
< Vary: Accept-Encoding
< Transfer-Encoding: chunked
< Content-Type: text/html; charset=UTF-8
<
{ [6 bytes data]
100  278k    0  278k    0     0   721k      0 --:--:-- --:--:-- --:--:--  723k
* Connection #0 to host www.boltonft.nhs.uk left intact

Weirdly, I have to perform ICMP traceroute when running through Plusnet otherwise the TCP one used earlier returns the same hop multiple times:

root@raspi01 ~ # traceroute -T www.boltonft.nhs.uk
traceroute to www.boltonft.nhs.uk (213.104.98.149), 30 hops max, 60 byte packets
 1  149.98-104-213.static.virginmediabusiness.co.uk (213.104.98.149)  10.311 ms  10.278 ms  10.556 ms
 2  149.98-104-213.static.virginmediabusiness.co.uk (213.104.98.149)  11.169 ms  11.113 ms  11.372 ms
 3  149.98-104-213.static.virginmediabusiness.co.uk (213.104.98.149)  11.366 ms  11.282 ms  10.925 ms
 4  149.98-104-213.static.virginmediabusiness.co.uk (213.104.98.149)  11.473 ms  11.168 ms  11.137 ms
 5  149.98-104-213.static.virginmediabusiness.co.uk (213.104.98.149)  20.637 ms  18.463 ms  18.378 ms
 6  * * *
 7  149.98-104-213.static.virginmediabusiness.co.uk (213.104.98.149)  28.309 ms *  28.130 ms
 8  149.98-104-213.static.virginmediabusiness.co.uk (213.104.98.149)  28.190 ms  28.067 ms  28.014 ms
 9  149.98-104-213.static.virginmediabusiness.co.uk (213.104.98.149)  28.021 ms  27.969 ms  27.915 ms
10  149.98-104-213.static.virginmediabusiness.co.uk (213.104.98.149)  27.915 ms  27.904 ms  27.809 ms
11  * * *
12  * * *
13  * * *
14  * * *
15  * * *
16  * * *
17  * * *
18  * * *
19  * * *
20  * * *
21  * * *
22  * * *
23  * * *
24  * * *
25  * * *
26  * * *
27  * * *
28  * * *
29  * * *
30  * * *

So yes, a bit of a loss but also puzzled at the odd behaviour of running traceroute over tcp when going out via Plusnet.  This behaviour appears to happen to every endpoint via tcp.

I've also tethered to my mobile phones data connection and ran the same tests, and they worked just fine including showing the correct hops via traceroute when going via tcp.

 

For reference, I run pfSense which has multiple gateways configured:

WAN -> Plusnet
OVPN -> Via external VPN Provider
HEV -> Hurricane Electric IPv6 Gateway (not used for this test)

To test the connection, I change the outbound firewall rule to send traffic via the OpenVPN gateway instead of the Plusnet gateway for a particular destination (in this case the IP in which resolves against the Bolton NFT Trust website).

4 REPLIES 4
Champnet
Aspiring Hero
Posts: 2,750
Thanks: 1,042
Fixes: 13
Registered: ‎25-07-2007

Re: Possible routing issue? Can't connect through to specific site.

I can get to the website but using tracert boltonnft.nhs.uk is not relying to pings.

bobpullen
Community Gaffer
Community Gaffer
Posts: 16,916
Thanks: 5,021
Fixes: 316
Registered: ‎04-04-2007

Re: Possible routing issue? Can't connect through to specific site.

What public IP address are you assigned during the times you're having problems?

I've just tried from an address in the 80.229.0.0/16 range and everything is fine: -

~$ sudo traceroute -T -p80 www.boltonft.nhs.uk
traceroute to www.boltonft.nhs.uk (213.104.98.149), 30 hops max, 60 byte packets
 1  100.115.92.193 (100.115.92.193)  0.037 ms  0.010 ms  0.007 ms
 2  100.115.92.25 (100.115.92.25)  0.175 ms  0.127 ms  0.118 ms
 3  home.gateway (192.168.1.254)  4.074 ms  3.973 ms  3.871 ms
 4  195.166.130.255 (195.166.130.255)  9.616 ms  9.528 ms  9.421 ms
 5  84.93.253.123 (84.93.253.123)  9.328 ms 84.93.253.127 (84.93.253.127)  9.242 ms  9.102 ms
 6  core1-BE1.southbank.ukcore.bt.net (195.99.125.130)  10.986 ms 195.99.125.142 (195.99.125.142)  11.118 ms 195.99.125.134 (195.99.125.134)  10.972 ms
 7  peer7-et-3-0-5.telehouse.ukcore.bt.net (109.159.252.188)  10.874 ms  10.799 ms peer3-et7-0-6.redbus.ukcore.bt.net (194.72.16.100)  14.565 ms
 8  109.159.253.101 (109.159.253.101)  16.546 ms  16.547 ms  16.423 ms
 9  * * *
10  * bagu-core-2a-ae10-0.network.virginmedia.net (62.254.84.2)  19.036 ms  18.953 ms
11  bagu-metnet-3b-lag-56.network.virginmedia.net (82.8.124.178)  18.845 ms bagu-core-2a-ae10-0.network.virginmedia.net (62.254.84.2)  18.674 ms bagu-metnet-3b-lag-56.network.virginmedia.net (82.8.124.178)  18.757 ms
12  bagu-metnet-3b-lag-56.network.virginmedia.net (82.8.124.178)  17.685 ms bnft-bl4-ia1.network.virginmedia.net (213.104.213.126)  19.344 ms bagu-metnet-3b-lag-56.network.virginmedia.net (82.8.124.178)  18.961 ms
13  bnft-bl4-ia1.network.virginmedia.net (213.104.213.126)  18.871 ms  18.756 ms 91-187-250-212.static.virginm.net (212.250.187.91)  19.810 ms
14  149.98-104-213.static.virginmediabusiness.co.uk (213.104.98.149)  19.388 ms 91-187-250-212.static.virginm.net (212.250.187.91)  18.776 ms  18.613 ms

Bob Pullen
Plusnet Product Team
If I've been helpful then please give thanks ⤵

ErTnEc
Grafter
Posts: 40
Thanks: 3
Registered: ‎02-07-2015

Re: Possible routing issue? Can't connect through to specific site.

My public IP is 81.174.148.33, and it's a statically assigned one.

stephenw10
Rising Star
Posts: 109
Thanks: 11
Registered: ‎14-04-2011

Re: Possible routing issue? Can't connect through to specific site.


@ErTnEc wrote:

Weirdly, I have to perform ICMP traceroute when running through Plusnet otherwise the TCP one used earlier returns the same hop multiple times:

 

Do you have outbound fq-codel shaping enabled?

Do you mean TCP or UDP there?

Sounds like you may be hitting something similar to this: https://redmine.pfsense.org/issues/9263
Though that would not prevent you accessing the site in general.

 

Traceroute fails for the last hop for me too just as you see it but I can still access the site.