The MSS value that was defined via GuiDBedit Tool (mss_value) on external interface on specific Security Gateway is not applied to IPsec VPN traffic, although MSS Adjustment (Clamping) for IPsec VPN traffic is enabled (values of kernel parameters fw_clamp_vpn_mss and sim_clamp_vpn_mss are set to 1 (one) per sk101219 - New VPN features in R77.20).

Setting the value of kernel parameter fw_clamp_tcp_mss_control to true via GuiDBedit Tool applies the MSS value to IPsec VPN traffic. However, it also applies the same MSS value to clear-text TCP traffic, which is undesired.

Output of kernel debug ('fw ctl debug -m fw + vm') during the issue shows for IPsec VPN traffic:
;fw_adjust_mss_ex: entering with mtu 1500 and mss 0

Examples from kernel debug for MSS value of 1200
('fw ctl debug -m fw + vm' and 'fw ctl debug -m VPN + packet'):

1. Parameters and their values:

   mss_value=1200
   fw_clamp_vpn_mss=1
   sim_clamp_vpn_mss=1
   fw_clamp_tcp_mss_control=0
   

   Debug:

   ;fw_filter_chain: Clamping MSS with interface value;
   ;fw_adjust_mss_ex: entering with mtu 1500 and mss 0 ;
   ;vpnk_mtu_4_mss: orig mtu = 1500, dir = 0, conn = 6;
   ... ...
   ;vpnk_mtu_4_mss: new mtu is 1426;
   ;fw_filter_chain: Clamping MSS with vpn clamping value;
   ;fw_adjust_mss_ex: entering with mtu 1426 and mss 0 ;
   ;fw_adjust_mss_ex: Overriding mss with stripped mtu: 1386;
   ;fw_adjust_mss_ex: reducing MSS on packet from 1460 to 1386;
   ;fw_filter_chain: Final switch, action=ACCEPT;
   ;After VM: <dir 0, ... IPP 6> ...;
   ;VM Final action=ACCEPT;
   

2. Parameters and their values:

   mss_value=1200
   fw_clamp_vpn_mss=1
   sim_clamp_vpn_mss=1
   fw_clamp_tcp_mss_control=1
   

   Debug:

   ;fw_filter_chain: Clamping MSS with interface value;
   ;fw_adjust_mss_ex: entering with mtu 1500 and mss 1200 ;
   ;vpnk_mtu_4_mss: orig mtu = 1460, dir = 0, conn = 6;
   ... ...
   ;vpnk_mtu_4_mss: new mtu is 1386;
   ;fw_adjust_mss_ex: VPN mtu adjust: New mtu 1386;
   ;fw_adjust_mss_ex: reducing MSS on packet from 1460 to 1200;
   ;fw_filter_chain: Final switch, action=ACCEPT;
   ;After VM: <dir 0, ... IPP 6> ...;
   ;VM Final action=ACCEPT;