MTU

• Maximum Transmission Unit - of packets, normally for ethernet 1500

• Links: linux/PacketOffloadingMtu

• 2018 Ubuntu 14.04 (old) could not connect to another machine, doing a tcpdump found

  • 22:31:17.504887 IP 10.128.155.77.ssh > 10.128.145.23.46488: Flags [.], seq 45:1493, ack 2013, win 257, options [nop,nop,TS val 2677742 ecr 60427352], length 1448
    22:31:17.505381 IP 172.16.6.2 > 10.128.155.77: ICMP 10.128.145.23 unreachable - need to frag (mtu 1436), length 556

  • Ubuntu was ignoring the ICMP and thus the ssh just hanged.

  • Fix1 - static route with mtu

    # ip route add 10.118.145.0/27 via 10.118.155.65 mtu 1400

  • Fix2 Root Cause - uRPF explained here http://mellowd.co.uk/ccie/?p=5662 a security feature.

    • set sysctl failed

      net.ipv4.conf.default.rp_filter = 0
       net.ipv4.conf.all.rp_filter = 0

  • Fix3 ( from https://blog.cloudflare.com/path-mtu-discovery-in-practice/ )

    • 3) Enable smart MTU black hole detection.
      
      RFC4821 proposes a mechanism to detect ICMP black holes and tries to adjust the path MTU in a smart way. To enable this on Linux type:
      
      echo 1 > /proc/sys/net/ipv4/tcp_mtu_probing
      echo 1024 > /proc/sys/net/ipv4/tcp_base_mss
      The second setting bumps the starting MSS used in discovery from a miserable default of 512 bytes to an RFC4821 suggested 1,024.