Latest revision as of 10:11, 5 September 2023

Overview

This page has lots of information and guidance on dealing with a slow FTTP connection. We suggest it's read by customers and that we then work with you. The page covers slow FTTP with BT Openreach and CityFibre connections.

The information here has a number of objectives:

Explain what could cause slow speeds
Explain the problems with dealing with slow speeds
Try to manage expectations about timescales and what to expect from a 1Gb/s connection
Provide various tests the customer can perform and send the results in to our support team
Provide questions that will help A&A support staff further investigate

Background information

Causes of slow speeds

There are a few things that could cause a FTTP circuit to be slower than expected.

A fault somewhere causing packetloss (this could be local to the customer's home network, or something further afield)
Some misconfiguration or fault in the fibre provider's network (eg broken links causing congestion)
Congestion somewhere between the premises and the internet causing slow-downs
Something else

Useful things to let us know

When reporting a slow connection fault to us it is always useful to have some background information. Answering these questions will help us get a better understanding of the story:

Describe what is slow. (eg a game, streaming, video calls, downloads, etc)
When did the connection become slow (the date)?
Can you think of any change that happened at the time the connection started to be slow?
Was the connection slow with a previous ISP? If so, what was the story there?
Have you already had engineers out to investigate? If so, what did they say?
Is it slow at particular times of day?
Do you know if your neighbours also have slow connections?

Things to be aware of

Committed rates

FTTP from providers like BT or CityFibre use GPON or XGS-PON technology to run fibre from their core network to your premises. This is a shared service, which means you do not have guaranteed sole use of the bandwidth that your physical connection is provided at. eg, a 1Gb/s connection does not mean that BT or CityFibre guarantee that you will have 1Gb/s throughput. Both BT and CityFibre have figures for the guaranteed speed which will me much lower than the connection speed.

eg:

CityFibre 1Gb/s = 70Mb/s
BT 1Gb/s =195Mb/s

Whilst we would expect to see our customers get near the full line rate, this is not guaranteed, and can be complex and long-winded to resolve if the cause of the low speed is due to the back-haul network and the speeds observed are within specification.

In short, we, and our suppliers are within their right to say a 1Gb/s service is working to acceptable standards if the observed throughput is above the figures above. The service is working within the acceptable range. so, contractually, a 1Gb/s service running at 200Mb/s is not a fault. More info about these rates on: https://www.aa.net.uk/broadband/fttp-speeds/

That said, in practice, we would expect to see our customers get the near full line rate. Just bear in mind the service being purchased does have levels of acceptable speeds.

SLA

Whilst BT and CityFibre will have SLA (Service Level Agreements), these usually cover where there is a physical fibre fault and that the times they state would be the anticipated time it would take to fix. Faults can take longer than the SLA to fix.

The SLA would also not be relevant where a speeds are above the 'committed' rate, as according to BT/CityFibre, the service would be working within the specified terms.

Long standing problems

We do get customers moving from a previous ISP to us having had a slow speed problem with the previous ISP.

In these cases, sometimes the slow speed problem disappears the day the service is migrated to us. However, sometimes the slow speed continues. In these cases moving ISPs has helped to narrow down the cause of the problem as a large part of the connection (ie the ISP) has changed. However, the back-haul connection, from your premises to the ISP may not have changed and could be the cause of the fault.

We will try our best but faults where a customer has had previous problems with an ISP, or ISPs, and has moved to us and still has the same problem can take a much longer time to fix, and in some cases it may not be possible - especially if the speed is above the committed rate and all avenues of investigation have been exhausted.

Working with us, not against us

At A&A we do work hard on diagnosing and fixing faults for and with our customers. A slow fibre fault is not always easy to fix - there are many places that could cause slow downs and a fault is not always clear.

We do need cooperation from our customers in running tests, swapping things around and so on. These tests may need to be repeated at different times of day and after a change has been made. This can seem like a waste of time, but this testing and cooperation is needed and is appreciated.

On the other hand, we will work with our suppliers - BT/CityFibre etc, and we do and will escalate and chase for updates. We will also push back if they ask us to do things that are clearly not going to help.

Investigating the cause of slow FTTP (getting practical)

Finding the cause of slowness

When faced with a slow service, the challenge is to find the cause. If a cause can be found then a solution can be found, or at least, explained.

If the problem is easily visible - eg constant packet loss, dropping PPP, high latency then these sort of faults are far easier to resolve than a line that looks perfect in terms of loss/latency but suffers with lower throughput than anticipated.

Things to test and try

Here is some steps to take if you have problems with your FTTP service.

If your FTTP service is slow, eg a speed test is not reporting expected speeds, then try these:

Checking the hardware and connections

wired connection
Connect to your router with cable instead of WiFi - to rule out any WiFi problems.
power cycle the ONT
Unplug and reconnect the power cable of the Openreach/CityFibre ONT - always worth a try!
Ensure the router is capable of the speed
eg, maybe you have a 1G service but an older router? - let us know the model you are using
Different computer/devices
Always worth trying different computers just in case one has a problem.
Check/swap the cable
between the ONT (The Openreach/CityFibre unit that the fibre connects to) and your router - ensure it's a 8-wire CAT5 or CAT6 cable - not one with 4 wires as these will run at 100M and not 1G.
Different router
If possible, try a different router. If not possible, connect PC directly to the ONT as below.
Connect directly in to the ONT
It's possible to use the internet without the use of the router. You can connect your computer directly in to the ONT - set up a new network connection of type PPPoE and use your xxx@a login and password to connect, and run a speed test, pings etc. see: PPPoE_on_a_Computer for more help on this.
Check the CQM Graphs, via the Control Pages, to see if it is slow due to lots of traffic. - staff can help explain the graphs and check them over.

Diagnostics you can run from a computer:

These are tests you can from your computer, some of them (eg the ping) will need you to use a command prompt or terminal to enter the commands. Do get in touch if you need help with how to do that.

View the CQM graph
- We plot loss/latency/traffic graphs and these can be seen on our Control Pages - viewing those is the first port of call, as loss and latency are easily spotted. Feel free to ask our support team to help explain them.
- There are some example CQM graphs at the end of this page
Run a BT diagnostic test:-
- Always worth running a test, even if LOS light is off - there could be a problem further upstream and Openreach may already be aware or we'll need to report a fault:
- Our Control Page will allow you to run an 'End to End' test, this may if Openreach's systems detect a fault, in which case, do get in touch. eg: GTC_FTTP_SERVICE_1005 Possible fault in the Openreach network.
Ping tests:
running ping tests from the 'command line' can help see if there is any strange loss or latency.
Expected results: The ping results should show 0% loss and the latency should be below 20ms, any thing else would need further investigation.
1. ping your router:
  This will test your local connection. eg, if your router (gateway) is 192.168.0.1 then run this to send 100 pings:
```
(on windows)  ping -n 100 192.168.0.1
```
```
(on Mac)  ping -c 100 192.168.0.1
```
  and let us know the last few lines containing the results, which show any loss, the latency and jitter, eg:
```
--- 192.168.0.1 ping statistics ---
100 packets transmitted, 100 received, 0% packet loss, time 100970ms
rtt min/avg/max/mdev = 0.137/0.172/0.736/0.068 ms
```
2. Ping our network:
  This will ping 100 times through your router to us, and therefore tests your connection to our network
```
(on windows)  ping -n 100 81.18781.187
```
```
(on Mac)  ping -c 100 81.18781.187
```
3. Ping with larger packet sizes
  normal pings send small packets, sometimes a problem can affect large packets, so try these two pings, and send in the results:
```
(on windows) ping -n 100 -f -l 1400 81.18781.187
```
```
(on windows) ping -n 100 -f -l 1472 81.18781.187
```
```
(on Mac) ping -c 100 -D -s 1400 81.18781.187
```
```
(on Mac) ping -c 100 -D -s 1472 81.18781.187
```
Speed test web sites:
Use ours and some other 3rd party ones to check up and download speeds run these at different times of day as this can help track 'peak time congestion'. note - many http speed testers are not very good for testing 1G connections - always take results with a pinch of salt. That said, Expected results: The speeds tests should show around 900Mb/s download speed
1. https://speedtest.aa.net.uk
2. https://www.fast.com
iperf
Ask us about testing with 'iperf' - as this is generally better than http speed test websites. This can test single and multi-thread, upload and download. See: https://support.aa.net.uk/Windows_iperf3 Running an iperf for 10 or 20 minutes is a good idea, as this will then be seen on our CQM graphs, and that may show loss/latency etc, and also run upload and download tests along with single and multiple threads.
1. ```
iperf3 -R -P1 -t 600 -c speedtest.aa.net.uk  # Download, singlethread, for 10 minutes
```
2. ```
iperf3 -P1 -t 600 -c speedtest.aa.net.uk  # Upload, single-thread, for 10 minutes
```
3. ```
iperf3 -R -P10 -t 600 -c speedtest.aa.net.uk  # Download, 10- thread, for 10 minutes
```
4. ```
iperf3 -P10 -t 600 -c speedtest.aa.net.uk  # Upload, 10-thread, for 10 minutes
```
Other tests
1. Search Youtube for a '4k test' video, and see if that streams smoothly - there are various nature/landscape videos that are high quality and should stream nicely
...get in touch.

More advanced topics and areas of investigation

Below are some more specific areas of investigation and examples of our own CQM monitoring graphs.

Single thread Vs multi thread performance problems

Sometimes a speed problem occurs for single-thread transfers, whist multi-thread transfers are OK. Broadly speaking, single thread Vs multi-thread speeds should be very similar, and near the top end of the speed of your circuit.

These can be tricky to diagnose and fix, as sometimes the problem is down to congestion in the back-haul network.

As well as testing the usual things (Different computer/router/directly-wired etc) to help diagnose, we suggest using iperf3 rather than http based speed tests as iperf3 is usually more reliable.

Support staff can give access to our own iperf server, and then you can run tests such as:

Single thread transfer from us to you (testing your download)

iperf3 -R -c <speedtest server>

10-thread transfer from us to you (testing your download)

iperf3 -R -P 10 -c <speedtest server>

Single thread transfer from you to us (testing your upload)

iperf3 -c <speedtest server>

10-thread transfer from you to us (testing your upload)

iperf3 -P 10 -c <speedtest server>

Running these at different times of day will give a picture as to whether single-thread is a problem, and if it's a problem due to the time of the day.

example

Here is an example where we are running a single thread iperf3 test for 10 minutes, and then a 10-thread test for 10 minutes - the difference is shown clearly in our CQM graphs:

Alternating between single and multi-thread iperf download tests

The graph should look like this, with equal and full speed for both single and multi thread tests:

Good speeds

I may have packet loss

Packetloss can be seen in a number of ways, check the following to see if the location can be narrowed down to something specific:

Ping your router's LAN IP - this will be the 'default gateway' your computers use - eg, could be something like 192.168.1.1 - if you have loss pinging your local router, check with a wired connection direct in to the router...
Check the CQM Graphs on the A&A Control pages
Ping the A&A endpoint IP: ping 81.187.81.187
ping somewhere on the internet, eg ping 8.8.8.8 and ping bbc.co.uk

Get in touch with us about what you find.

Intermittent low levels of packet loss

Wrongly configured OLT

Packet loss with an odd pattern like this was seen after a CityFibre OLT was upgraded in March 2023 and apparently have some wrong configuration and caused around 5% loss:

Loss caused by a misconfigured CityFibre OLT after being having a firmware upgrade

Slow CityFibre since migrating

We have seen CityFibre connections limited to say 500M when they have been migrated over to us. This could be due to the previous service being a 500M service and CityFibre's systems not updating the profile/bandwidth for the circuit. This can be tested with connecting a PC to the ONT and setting PPPoE_on_a_Computer - if around 500M is the maximum then let us know. The fix is for CityFibre to re-provision the ONT, which we can request.

@@ Line 2: / Line 2: @@
 [[Category:FTTP Faults]]
-==Causes==
+==Overview==
+This page has lots of information and guidance on dealing with a slow FTTP connection. We suggest it's read by customers and that we then work with you. The page covers slow FTTP with BT Openreach and CityFibre connections.
+The information here has a number of objectives:
+*Explain what could cause slow speeds
+*Explain the problems with dealing with slow speeds
+*Try to manage expectations about timescales and what to expect from a 1Gb/s connection
+*Provide various tests the customer can perform and send the results in to our support team
+*Provide questions that will help A&A support staff further investigate
+=Background information=
+==Causes of slow speeds==
 There are a few things that could cause a FTTP circuit to be slower than expected.
 #A fault somewhere causing packetloss (this could be local to the customer's home network, or something further afield)
-#Some miss-configuration or fault in the fibre provider's network (eg broken links causing congestion)
+#Some misconfiguration or fault in the fibre provider's network (eg broken links causing congestion)
 #Congestion somewhere between the premises and the internet  causing slow-downs
 #Something else
@@ Line 12: / Line 24: @@
 ==Useful things to let us know==
 When reporting a slow connection fault to us it is always useful to have some background information. Answering these questions will help us get a better understanding of the story:
-# Describe what is slow.
+# Describe what is slow. (eg a game, streaming, video calls, downloads, etc)
-# When did the connection become slow?
+# When did the connection become slow (the date)?
+# Can you think of any change that happened at the time the connection started to be slow?
 # Was the connection slow with a previous ISP? If so, what was the story there?
 # Have you already had engineers out to investigate? If so, what did they say?
@@ Line 20: / Line 33: @@
 ==Things to be aware of==
+===Committed rates===
 FTTP from providers like BT or CityFibre use GPON or XGS-PON technology to run fibre from their core network to your premises. This is a '''shared service''', which means you do not have guaranteed sole use of the bandwidth that your physical connection is provided at. eg, a 1Gb/s connection does not mean that BT or CityFibre guarantee that you will have 1Gb/s throughput. Both BT and CityFibre have figures for the '''guaranteed speed''' which will me much lower than the connection speed.
 eg:
-*CityFibre 1G = '''70Mb/s'''
+*CityFibre 1Gb/s = '''70Mb/s'''
 *BT 1Gb/s ='''195Mb/s'''
+Whilst we would expect to see our customers get near the full line rate, this is not guaranteed, and can be complex and long-winded to resolve if the cause of the low speed is due to the back-haul network and the speeds observed are within specification.
-More info of the rates on: https://www.aa.net.uk/broadband/fttp-speeds/
+In short, we, and our suppliers are within their right to say a 1Gb/s service is working to acceptable standards if the observed throughput is above the figures above. The service is working within the acceptable range. so, contractually, a 1Gb/s service running at 200Mb/s is not a fault. More info about these rates on: https://www.aa.net.uk/broadband/fttp-speeds/
+That said, in practice, we would expect to see our customers get the near full line rate. Just bear in mind the service being purchased does have levels of acceptable speeds.
+===SLA===
+Whilst BT and CityFibre will have SLA (Service Level Agreements), these usually cover where there is a physical fibre fault and that the times they state would be the anticipated time it would take to fix. Faults can take longer than the SLA to fix.
+The SLA would also not be relevant where a speeds are above the 'committed' rate, as according to BT/CityFibre, the service would be working within the specified terms.
+===Long standing problems===
+We do get customers moving from a previous ISP to us having had a slow speed problem with the previous ISP.
+In these cases, sometimes the slow speed problem disappears the day the service is migrated to us.  However, sometimes the slow speed continues. In these cases moving ISPs has helped to narrow down the cause of the problem as a large part of the connection (ie the ISP) has changed. However, the back-haul connection, from your premises to the ISP may not have changed and could be the cause of the fault.
+We will try our best but faults where a customer has had previous problems with an ISP, or ISPs, and has moved to us and still has the same problem can take a much longer time to fix, and in some cases it may not be possible - especially if the speed is above the committed rate and all avenues of investigation have been exhausted.
+==Working with us, not against us==
+At A&A we do work hard on diagnosing and fixing faults for and with our customers. A slow fibre fault is not always easy to fix - there are many places that could cause slow downs and a fault is not always clear.
+We do need cooperation from our customers in running tests, swapping things around and so on. These tests may need to be repeated at different times of day and after a change has been made. This can seem like a waste of time, but this testing and cooperation is needed and is appreciated.
+On the other hand, we will work with our suppliers - BT/CityFibre etc, and we do and will escalate and chase for updates. We will also push back if they ask us to do things that are clearly not going to help.
+=Investigating the cause of slow FTTP (getting practical)=
 ==Finding the cause of slowness==
 When faced with a slow service, the challenge is to find the cause. If a cause can be found then a solution can be found, or at least, explained.
+If the problem is easily visible - eg constant packet loss, dropping PPP, high latency then these sort of faults are far easier to resolve than a line that looks perfect in terms of loss/latency but suffers with lower throughput than anticipated.
 == Things to test and try ==
@@ Line 44: / Line 85: @@
 # '''Check/swap the cable''' <br> between the ONT (The Openreach/CityFibre unit that the fibre connects to) and your router - ensure it's a '''8-wire''' CAT5 or CAT6 cable - not one with 4 wires as these will run at 100M and not 1G.
 # '''Different router''' <br> If possible, try a different router. If not possible, connect PC directly to the ONT as below.
-# '''Connect directly in to the ONT''' <br> It;s possible to use the internet without the use of the router. You can connect your computer directly in to the ONT - set up a new network connection of type PPPoE and use your xxx@a login and password to connect, and run a speed test. see: [[PPPoE_on_a_Computer]] for more help on this.
+# '''Connect directly in to the ONT''' <br> It's possible to use the internet without the use of the router. You can connect your computer directly in to the ONT - set up a new network connection of type PPPoE and use your xxx@a login and password to connect, and run a speed test, pings etc. see: [[PPPoE_on_a_Computer]] for more help on this.
 # Check the '''CQM Graphs''', via the Control Pages, to see if it is slow due to lots of traffic. - staff can help explain the graphs and check them over.
 ===Diagnostics you can run from a computer:===
-These are tests you can from your computer, some of them (eg the ping) will need you to use a command prompt or terminal to enter the commands. Do gen in touch if you need help with how to do that.
+These are tests you can from your computer, some of them (eg the ping) will need you to use a command prompt or terminal to enter the commands. Do get in touch if you need help with how to do that.
+# '''View the CQM graph'''
+#* We plot loss/latency/traffic graphs and these can be seen on our Control Pages - viewing those is the first port of call, as loss and latency are easily spotted. Feel free to ask our support team to help explain them.
-#'''Ping tests:''' <br> running ping tests from the 'command line' can help see if there is any strange loss or latency.
+#* There are some example CQM graphs at the end of this page
-## '''ping your router:''' <br> This will test your local connection. eg, if your router (gateway) is 192.168.0.1 then run: <syntaxhighlight>ping -c 100 192.168.0.1</syntaxhighlight> and let us know the last few lines containing the results, which show any loss, the latency and jitter, eg:<syntaxhighlight>
+# '''Run a BT diagnostic test:-'''
+#* Always worth running a test, even if LOS light is off - there could be a problem further upstream and Openreach may already be aware or we'll need to report a fault:
+#* Our Control Page will allow you to run an 'End to End' test, this may if Openreach's systems detect a fault, in which case, do get in touch. eg: <tt>GTC_FTTP_SERVICE_1005 Possible fault in the Openreach network.</tt>
+#'''Ping tests:''' <br> running ping tests from the 'command line' can help see if there is any strange loss or latency. <br> ''Expected results'': The ping results should show 0% loss and the latency should be below 20ms, any thing else would need further investigation.
+## '''ping your router:''' <br> This will test your local connection. eg, if your router (gateway) is 192.168.0.1 then run this to send 100 pings: <syntaxhighlight>(on windows)  ping -n 100 192.168.0.1</syntaxhighlight>  <syntaxhighlight>(on Mac)  ping -c 100 192.168.0.1</syntaxhighlight> and let us know the last few lines containing the results, which show any loss, the latency and jitter, eg:<syntaxhighlight>
 --- 192.168.0.1 ping statistics ---
 packets transmitted, 100 received, 0% packet loss, time 100970ms
 rtt min/avg/max/mdev = 0.137/0.172/0.736/0.068 ms</syntaxhighlight>
-## '''Ping our network:''' <br>This will ping through your router to us, and therefore tests your connection to our network <syntaxhighlight>ping -c 100 81.18781.187</syntaxhighlight>
+## '''Ping our network:''' <br>This will ping 100 times through your router to us, and therefore tests your connection to our network <syntaxhighlight>(on windows)  ping -n 100 81.18781.187</syntaxhighlight> <syntaxhighlight>(on Mac)  ping -c 100 81.18781.187</syntaxhighlight>
+##'''Ping with larger packet sizes''' <br> normal pings send small packets, sometimes a problem can affect large packets, so try these two pings, and send in the results: <syntaxhighlight>(on windows) ping -n 100 -f -l 1400 81.18781.187</syntaxhighlight> <syntaxhighlight>(on windows) ping -n 100 -f -l 1472 81.18781.187</syntaxhighlight> <syntaxhighlight>(on Mac) ping -c 100 -D -s 1400 81.18781.187</syntaxhighlight> <syntaxhighlight>(on Mac) ping -c 100 -D -s 1472 81.18781.187</syntaxhighlight>
-#'''Speed test web sites:''' <br>Use ours and some other 3rd party ones to check up and download speeds run these at different times of day as this can help track 'peak time congestion'.  note - many http speed testers are not very good for testing 1G connections - always take results with a pinch of salt
+#'''Speed test web sites:''' <br>Use ours and some other 3rd party ones to check up and download speeds run these at different times of day as this can help track 'peak time congestion'.  note - many http speed testers are not very good for testing 1G connections - always take results with a pinch of salt. That said, ''Expected results:'' The speeds tests should show around 900Mb/s download speed
 ## https://speedtest.aa.net.uk
 ## https://www.fast.com
-# Ask us about testing with ''''iperf'''' - as this is generally better than http speed test websites. See: https://support.aa.net.uk/Windows_iperf3
+# '''iperf''' <br> Ask us about testing with ''''iperf'''' - as this is generally better than http speed test websites. This can test single and multi-thread, upload and download. See: https://support.aa.net.uk/Windows_iperf3 Running an iperf for 10 or 20 minutes is a good idea, as this will then be seen on our CQM graphs, and that may show loss/latency etc, and also run upload and download tests along with single and multiple threads.
+## <syntaxhighlight>iperf3 -R -P1 -t 600 -c speedtest.aa.net.uk  # Download, singlethread, for 10 minutes</syntaxhighlight>
-#Check for '''packet loss''' - see below.
+## <syntaxhighlight>iperf3 -P1 -t 600 -c speedtest.aa.net.uk  # Upload, single-thread, for 10 minutes</syntaxhighlight>
+## <syntaxhighlight>iperf3 -R -P10 -t 600 -c speedtest.aa.net.uk  # Download, 10- thread, for 10 minutes</syntaxhighlight>
+## <syntaxhighlight>iperf3 -P10 -t 600 -c speedtest.aa.net.uk  # Upload, 10-thread, for 10 minutes</syntaxhighlight>
+#'''Other tests'''
+## Search Youtube for a '4k test' video, and see if that streams smoothly - there are various nature/landscape videos that are high quality and should stream nicely
 # ...'''get in touch'''.
+=More advanced topics and areas of investigation=
-Always worth running a test, even if LOS light is off - there could be a problem further upstream and Openreach may already be aware or we'll need to report a fault:
-* '''Run a test:-'''
+Below are some more specific areas of investigation and examples of our own CQM monitoring graphs.
-*# Our Control Page will allow you to run an 'End to End' test, this may if Openreach's systems detect a fault, in which case, do get in touch. eg: <tt>GTC_FTTP_SERVICE_1005 Possible fault in the Openreach network.</tt>
 == Single thread Vs multi thread performance problems ==
@@ Line 113: / Line 165: @@
 Packet loss with an odd pattern like this was seen after a CityFibre OLT was upgraded in March 2023 and apparently have some wrong configuration and caused around 5% loss:
-[[File:FTTP-misconfig-olt.png|frame|none|Loss caused by a missconfigured CityFibre OLT after being having a firmware upgrade]]
+[[File:FTTP-misconfig-olt.png|frame|none|Loss caused by a misconfigured CityFibre OLT after being having a firmware upgrade]]
 ==Slow CityFibre since migrating==