Jump to content

This is the support site for Andrews & Arnold Ltd, a UK Internet provider. Information on these pages is generally for our customers but may be useful to others, enjoy!

MTU: Difference between revisions

3,290 bytes added ,  29 April
Added reference to BT FTTP
(Added reference to BT FTTP)
 
(33 intermediate revisions by 4 users not shown)
<indicator name="Front">[[File:Menu-packet.svg|link=:Category:MTU|30px|Back up to the MTU Page]]</indicator>
This page will be about MTU, it is currently work in progress.
This page will be about MTU,
 
=tl;dr=
=Overview=
To simply test if you have 1500 MTU:
 
===What is my MTU?===
If you can ping a box, you can check what the largest packet size you can get to it is. AAISP customers can ping 81.187.81.187, which is the next hop from your DSL line.
 
For example:
 
[[File:Firebrick-1500MTU-ping.png|200px|thumb|a 1500 ping on FireBrick]]
 
'''On Linux:'''
ping -c1 -M do -s 1472 81.187.81.187
 
'''On OSX (Apple):'''
ping -c1 -D -s 1472 81.187.81.187
 
'''On Windows:'''
ping -n 1 -f -l 1472 81.187.81.187
 
^^ That's the ping command to remember. The rest of this section is a bit of info about that command.
 
Which results in:
 
PING 81.187.81.187 (81.187.81.187) 1472('''1500''') bytes of data.
1480 bytes from 81.187.81.187: icmp_req=1 ttl=59 time=54.1 ms
--- 81.187.81.187 ping statistics ---
'''1 packets transmitted, 1 received''', 0% packet loss, time 0ms
rtt min/avg/max/mdev = 54.108/54.108/54.108/0.000 ms
 
The things to note are highlighted in bold
 
If 1500 bytes cannot pass, then a ping result will look like:
 
PING 81.187.81.187 (81.187.81.187) 1472(1500) bytes of data.
From 81.187.81.187 icmp_seq=1 '''Frag needed and DF set''' (mtu = 1492)
--- 81.187.81.187 ping statistics ---
'''0 packets transmitted, 0 received''', +1 errors
 
 
==Explanation of the ping options==
 
'''-c1 / -n 1''' This is optional, it simply tells ping just to send a single ping
 
'''-M do / -f / -D''' The important option! This means don't fragment the ping packet - we don't want the ping split into multiple packets when testing MTU!
 
''' -s 1472 / -l 1472''' The payload size. When setting size in ping, the size is the payload size - not the full packet size. '''The full packet size is''' <tt> payload (1472) + IP header (20) + ICMP header (8) </tt>, so this is 1500 in total!
 
==Two quick examples==
 
Checking if you really have 1500 MTU:
 
% ping -c1 -M do -s 1472 81.187.81.187
PING 81.187.81.187 (81.187.81.187) 1472''(1500)'' bytes of data.
'''1480 bytes from 81.187.81.187: icmp_req=1 ttl=59 time=12.4 ms'''
--- 81.187.81.187 ping statistics ---
1 packets transmitted, 1 received, 0% [[Packet Loss|packet loss]], time 0ms
rtt min/avg/max/mdev = 12.491/12.491/12.491/0.000 ms
 
So, here we ping with '''1472''' bytes, this is 1472 plus 20 bytes for the IP header and 8 bytes for the ICMP header is a '''1500''' byte MTU!
 
Checking if you have smaller than 1500 MTU:
 
% ping -c1 -M do -s 1472 81.187.81.187
PING 81.187.81.187 (81.187.81.187) 1472''(1500)'' bytes of data.
'''From 90.155.53.53 icmp_seq=1 Frag needed and DF set (mtu = 1492)'''
--- 81.187.81.187 ping statistics ---
1 packets transmitted, 0 received, +1 errors, 100% [[Packet Loss|packet loss]], time 0ms
 
That error says "Frag needed and DF set (mtu = 1492)". So, with an MTU of 1492 we would want a payload of 1464 (1464+28=1492):
 
% ping -c1 -M do -s 1464 81.187.81.187
PING 81.187.81.187 (81.187.81.187) 1464''(1492)'' bytes of data.
'''1472 bytes from 81.187.81.187: icmp_req=1 ttl=59 time=29.5 ms'''
--- 81.187.81.187 ping statistics ---
1 packets transmitted, 1 received, 0% [[Packet Loss|packet loss]], time 0ms
rtt min/avg/max/mdev = 29.595/29.595/29.595/0.000 ms
 
Yay a reply! To double check, lets try one byte larger:
 
% ping -c1 -M do -s 1465 81.187.81.187
PING 81.187.81.187 (81.187.81.187) 1465''(1493)'' bytes of data.
'''From 90.155.42.36 icmp_seq=1 Frag needed and DF set (mtu = 1492)'''
--- 81.187.81.187 ping statistics ---
0 packets transmitted, 0 received, +1 errors
 
Yep, 1465 is too large a payload. It's interesting that the error is from itself (90.155.42.36) that time. The machine we sent the ping from must have received and
remembered the ICMP error!
 
 
If you're unlucky, there may be no error message at all (these are from Debian and Ubuntu through a Netgear router to TTW):
$ ping -c1 -M do -s 1465 81.187.81.187
PING 81.187.81.187 (81.187.81.187) 1465''(1493)'' bytes of data.
--- 81.187.81.187 ping statistics ---
1 packets transmitted, 0 received, 100% [[Packet Loss|packet loss]], time 0ms
 
$ ping -c1 -M do -s 1464 81.187.81.187
PING 81.187.81.187 (81.187.81.187) 1464''(1492)'' bytes of data.
1472 bytes from 81.187.81.187: icmp_seq=1 ttl=63 time=55.4 ms
--- 81.187.81.187 ping statistics ---
1 packets transmitted, 1 received, 0% [[Packet Loss|packet loss]], time 0ms
rtt min/avg/max/mdev = 55.448/55.448/55.448/0.000 ms
 
 
 
 
=MTU Overview=
In simple terms, when you send information from one place on the internet (e.g. a web server) to another (e.g. your computer) the data is broken up in to packets. The sender breaks the overall data in to small chunks, and sends them over the internet. The internet only handles packets. The packets arrive at the other end, and they are put back together to make the whole of the original data. This could be a web page, email, image, or whatever. If a packet gets dropped, the sender works this out and resends it. That way you don't get gaps in what you receive. The process is managed by a protocol called TCP (Transmission Control Protocol). The packets get to the other by a protocol called IP (Internet Protocol). There are other protocols that work over the internet but they all come down to sending packets.
 
 
=How big is a packet?=
The simple answer is a maximum of 1,500 bytes. But it is more complex. The internet is actually a combination of routers and links. Each router has links to other routers. Your packets go from one place to another by being passed along from one router to another over these links.
 
One of the main types of link is [[Ethernet]] which is used for Local Area Networks (LAN). You have probably encountered a LAN as they are used in offices and homes, and connect things together. You probably use [[Ethernet]] to connect your broadband router to your computers in your house. [[Ethernet]] allows 1,500 byte packets to be carried. Internet providers use much faster links such as gigabit and 10 gigabit and these often allow bigger packets up to around 9,000 bytes. Some links on the internet are set up specially for certain traffic and have links that support packet sizes like 1,548 bytes.
In most cases links support 1,500 bytes, so there is not problem. However, where we know there is a specific issue there is another option, an option (C). It is a bit of a bodge, but works. We have added this as a user-settable feature on the line so you can contol if you want us to work-around the issue or not. This is mainly for where the link from us to you is restricted, typically to 1,492 bytes, for PPPoE.
 
It works by exploiting the fact that at the start of a TCP session each end tells the other the MTU it can handle (actually the MSS which is the TCP payload but one is derived from the other). When we have have the fix enabled we check the initial TCP handshake packets and we change them if the MSS specified would be too big. This means each end may say they handle 1,500 bytes, but the other hears that they handle, say, 1,492 bytes.
 
When do you get small links?
Unfortunately the specifications are not that helpful here. RFC1661 defines PPP and states If smaller packets are requested, an implementation MUST still be able to receive the full 1,500 octet information field. RFC2516 defines PPPoE The Maximum-Receive-Unit (MRU) option MUST NOT be negotiated to a larger size than 1,492. But they are not incompatible statements - negotiating 1,492 does not mean you don't have to accept 1,500 byte packets (as per RFC1661), but you can't send on if PPPoE bridging, for example, so logically you would have to fragment the IP packet. Most routers do not do that! This is one of the reasons we get problems with MTU being smaller than 1,500 bytes.
 
There are also ways to do over sizedoversized PPPoE using baby jumbo frames on [[Ethernet]]. The [[Ethernet]] specification still says 1,500 bytes maximum even for gigabit speeds, but it common for gigabit equipment to support jumbo frames - i.e. larger [[Ethernet]] packets typically up to around 9,000 bytes. This is more than you need to do PPPoE with 1,500 bytes - you only need 1,508. However, there are other wrapping and tunnelling cases where just a bit more is useful. Baby jumbo support normally means a bit more than the usual 1,500.
 
As there is no real way to tell if baby jumbo frames are supported on an [[Ethernet]], RFC4638 defines an extra option for PPPoE to negotiate this at the [[Ethernet]] level. Of course two ends could simply agree to handle slightly larger [[Ethernet]] frames by configuration as well. Sadly this is not always the same level of operation or the same equipment that does the MRU negotiation at the PPP level, and if that knows PPPoE is involved it will not negotiate more than 1,492 MRU as per RFC2516. So typically some configuration is needed.
 
The upshot of all this? It is possible to get BT FTTC (Fibre to the Cabinet) and BT FTTP (Fibre to the Premises) circuits (which use PPPoE) working on full 1,500 byte PPP by using modified pppd on the customer end, a suitable network card that will handle 1,508 byte frames at 10/100Mb100Mbit/s. We have done this! (Thanks to TonyHoyle and Simon, customers on irc, for tweaking pppd and testing this for us). The new FireBrick does, of course, support PPPoE with baby jumbo frames to handle 1,500 byte MTU and even bonds multiple lines. Using the right modem (and the DLINK 320B in bridge mode do this) you can negotiate and use 1508 bytes over ADSL as well. In addition, BTs FTTP ONT units support baby jumbo frames and can support a 1,500 byte MTU.
 
 
=Backhaul networks=
 
==TalkTalk Wholesale==
=What is my MTU?=
If you can ping a box, you can check what the largest packet size you can get to it is. AAISP customers can ping 81.187.81.187, which is the next hop from your DSL line.
 
For example:
ping -c1 -M do -s 1472 81.187.81.187
 
^^ That's the ping command to remember. The rest of this section is a bit of info about that command.
 
Explanation:
 
'''-c1''' In these examples, we've only used a count of one ping using the -c option.
 
'''-M do''' Use the path discovery options to ping. The option is -M. See the man page for all the options.
 
TTW's default MTU is 1492 bytes. It is possible to increase this to 1500 bytes by ticking 'TCPFix' with an MTU of '1500' on clueless. At the
''' -s 1472''' When setting size in ping, the size is the payload size - not the full packet size. The full packet size is payload + ICMP header size (28 bytes). The option for payload size is -s.
time of writing (Dec 2014) TTW does not support an MTU of 1508, so users of PPPoE cannot achieve an MTU of more than 1492 bytes.
 
Note: ping is slightly different on different operating systems, the above is a Debian machine.
 
On OSX (Apple), to perform a ping with the same options, you can use:
ping -c1 -D -s 1472 81.187.81.187
 
On a Windows machine to perform a ping with the same options, you can use:
ping -n1 -f -l 1472 81.187.81.187
 
==Two quick examples:==
 
Checking if you really have 1500 MTU:
 
% ping -c1 -M do -s 1472 81.187.81.187
PING 81.187.81.187 (81.187.81.187) 1472''(1500)'' bytes of data.
'''1480 bytes from 81.187.81.187: icmp_req=1 ttl=59 time=12.4 ms'''
--- 81.187.81.187 ping statistics ---
1 packets transmitted, 1 received, 0% [[Packet Loss|packet loss]], time 0ms
rtt min/avg/max/mdev = 12.491/12.491/12.491/0.000 ms
 
 
Checking if you have smaller than 1500 MTU:
 
% ping -c1 -M do -s 1472 81.187.81.187
PING 81.187.81.187 (81.187.81.187) 1472''(1500)'' bytes of data.
'''From 90.155.53.53 icmp_seq=1 Frag needed and DF set (mtu = 1492)'''
--- 81.187.81.187 ping statistics ---
1 packets transmitted, 0 received, +1 errors, 100% [[Packet Loss|packet loss]], time 0ms
 
That error says "Frag needed and DF set (mtu = 1492)". So, with an MTU of 1492 we would want a payload of 1464 (1464+28=1492):
 
% ping -c1 -M do -s 1464 81.187.81.187
PING 81.187.81.187 (81.187.81.187) 1464''(1492)'' bytes of data.
'''1472 bytes from 81.187.81.187: icmp_req=1 ttl=59 time=29.5 ms'''
--- 81.187.81.187 ping statistics ---
1 packets transmitted, 1 received, 0% [[Packet Loss|packet loss]], time 0ms
rtt min/avg/max/mdev = 29.595/29.595/29.595/0.000 ms
 
Yay a reply! To double check, lets try one byte larger:
 
% ping -c1 -M do -s 1465 81.187.81.187
PING 81.187.81.187 (81.187.81.187) 1465''(1493)'' bytes of data.
'''From 90.155.42.36 icmp_seq=1 Frag needed and DF set (mtu = 1492)'''
--- 81.187.81.187 ping statistics ---
0 packets transmitted, 0 received, +1 errors
 
Yep, 1465 is too large a payload. It's interesting that the error is from itself (90.155.42.36) that time. The machine we sent the ping from must have received and
remembered the ICMP error!
 
[Above is the official position. However, there are some experimental results which indicate one might be able to do better. Using a TG582n as the router running PPPoE
and configuring it to support baby jumbo frames:
:eth ifconfig intf=eth_WAN mtu=1508
:ppp ifdetach intf=Internet
:ppp ifconfig intf=Internet mru=1500
:ppp ifattach intf=Internet
then PPP claims to successfully negotiate a 1500 octet MRU with TTW.
 
{Administrator}[ppp]=>:ppp iflist intf=Internet
Internet: dest eth_WAN [03:43:04] retry : 10
admin state = up oper state = up link state = connected
flags = echo magic accomp restart mru addr route savepwd ipv4 ipv6 chap
class = 12 echointerval = 10 echofail = 5 echototaltolerance = 50
'''administrative mru = 1500 negotiation mru = 1500'''
auth type = auto
...
 
Using the ping tests from above, a ping of 1468 octets (i.e. a packet size of 1496) now works
but a ping of 1469 octets (packet size of 1497) doesn't. On further testing, one can receive
1500 octet packets, but can only send 1496 octet packets - i.e. MRU=1500, MTU=1496.
This is despite the bridging modem used for the tests not claiming to support an MTU > 1500 octets
and with PPPoE needing to send 1504 octets and receive 1508.
 
[[Category:MTU]]
[[Category:Internet]] [[Category:Router]] [[Category:Configuring]][[Category:ADSL]]
28

edits