MultiPath TCP - Linux Kernel implementation : Users

If you are looking for MPTCP in Linux >= v5.6:

https://mptcp.dev

For Linux <= 5.4, you can stay on this website.

You can configure MPTCP with several tunables. Have a look here to know their meaning. As the project is still experimental, sometimes changes are introduced to the ABI that depend on the MPTCP version.

First, in order to get the version of MPTCP, do (in a running system):

  dmesg | grep MPTCP

Version >= 0.92.x

To set a sysctl variable, just do:

  sysctl -w net.mptcp.[name of the variable]=[value]

net.mptcp.mptcp_enabled Disable/Enable MPTCP on this machine. Possible values are 0 or 1. (default 1)
Since release 0.89, you can set mptcp_enabled to 2. In this case MPTCP will be enabled if the application has set the socket-option MPTCP_ENABLED (value 42) to 1. MPTCP_ENABLED is part of SOL_TCP:
int enable = 1;
setsockopt(fd, SOL_TCP, MPTCP_ENABLED, &enable, sizeof(enable));
You can use MPTCP_ENABLED also to get information about whether MPTCP has been successfully negotiated by using getsockopt.
net.mptcp.mptcp_checksum Disable/Enable the MPTCP checksum. As described in the draft, both sides (client and server) have to disable DSS-checksums. If one side has it enabled, DSS-checksums will be used. Possible values are 0 or 1. (default 1)
net.mptcp.mptcp_syn_retries Specifies how often we retransmit a SYN with the MP_CAPABLE-option. After this, the SYN will not contain the MP_CAPABLE-option. This is to handle middleboxes that drop SYNs with unknown TCP options. Default is 3.

You can also configure different congestion controls. The available congestion controls are lia (alias Linked Increase Algorithm), olia (alias Opportunistic Linked Increase Algorithm),wVegas (alias Delay-based Congestion Control for MPTCP) and balia (alias Balanced Linked Adaptation Congestion Control Algorithm). You can change them via the sysctl net.ipv4.tcp_congestion_control.

Configure the path-manager:

We have a modular structure for the path-manager. At compile-time you can enable the path-managers through "MPTCP: advanced path-manager control" and select for example the full-mesh path-manager. If you do not select a path-manager, the host will not trigger the creation of new subflows, nor advertise alternative IP-addresses through the ADD_ADDR-option.

At run-time, to select one of the compiled path-managers just set the sysctl net.mptcp.mptcp_path_manager. You have the choice between:

'default': This path-manager actually does not do anything. The host won't announce different IP-addresses nor initiate the creation of new subflows. However, it will accept the passive creation of new subflows.
'fullmesh': It will create a full-mesh of subflows among all available subflows. Since v0.90 it is possible to create multiple subflows for each pair of IP-addresses. Just set /sys/module/mptcp_fullmesh/parameters/num_subflows to a value > 1.
New in v0.92: If you want to re-create subflows after a timeout (e.g., if the NAT-mapping was lost due to idle-time), you can set /sys/module/mptcp_fullmesh/parameters/create_on_err to 1.
'ndiffports': This one will create X subflows across the same pair of IP-addresses, modifying the source-port. To control the number of subflows (X), you can set the sysfs /sys/module/mptcp_ndiffports/parameters/num_subflows to a value > 1.
'binder': The path-manager using Loose Source Routing from the paper Binder: a system to aggregate multiple internet gateways in community networks.
New in v0.95: 'netlink': This path-manager can be controlled from the userspace. See the source code for the API documentation or mptcpd for an Open-Source daemon.

New in v0.92: Alternatively, you can select the path-manager through the socket-option MPTCP_PATH_MANAGER (defined as 44) by doing:

    char pathmanager[] = "ndiffports";
    setsockopt(fd, SOL_TCP, MPTCP_PATH_MANAGER, pathmanager, sizeof(pathmanager));

Configure the scheduler:

We have a modular scheduler infrastructure. At compile-time you can select the schedulers that should be compiled in.

At run-time you can select one of the compiled schedulers through the sysctl net.mptcp.mptcp_scheduler. You have the choice between:

'default': This scheduler is the default one. It will first send data on subflows with the lowest RTT until their congestion-window is full. Then, it will start transmitting on the subflows with the next higher RTT.
'roundrobin': This scheduler will transmit traffic in a round-robin fashion. It is configurable, how many consecutive segments should be sent with the tunable "num_segments" in the sysfs (default 1). Additionally, you can set the boolean tunable "cwnd_limited", to specify whether the scheduler tries to fill the congestion window on all subflows (true) or whether it prefers to leave open space in the congestion window (false) to achieve real round-robin (even if the subflows have very different capacities) (defaults to true). In case you are unsure, never ever enable this scheduler. Its performance is bad and it is only interesting for academic/testing purposes. The default scheduler is the best known up to today.
'redundant': This scheduler will try to transmit the traffic on all available subflows in a redundant way. It is useful when one wants to achieve the lowest possible latency by sacrificing the bandwidth.

New in v0.92: Alternatively, you can select the scheduler through the socket-option MPTCP_SCHEDULER (defined as 43) by doing:

    char scheduler[] = "redundant";
    setsockopt(fd, SOL_TCP, MPTCP_SCHEDULER, scheduler, sizeof(scheduler));

Get detailed information of your subflows

New in v0.92

You can use our new socket-option MPTCP_INFO (defined as 45), to get detailed information about each subflow (e.g., struct tcp_info):

    struct mptcp_info minfo;
    struct mptcp_meta_info meta_info;
    struct tcp_info initial;
    struct tcp_info others[3]; // increase it if needed
    struct mptcp_sub_info others_info[3]; // same
    socklen_t len;

    len = sizeof(minfo);
    minfo.tcp_info_len = sizeof(struct tcp_info);
    minfo.sub_len = sizeof(others);
    minfo.meta_len = sizeof(struct mptcp_meta_info);
    minfo.meta_info = &meta_info;
    minfo.initial = &initial;
    minfo.subflows = &others;
    minfo.sub_info_len = sizeof(struct mptcp_sub_info);
    minfo.total_sub_info_len = sizeof(others_info);
    minfo.subflow_info = &others_info;

    getsockopt(<fd>, IPPROTO_TCP, MPTCP_INFO, &minfo, &len);

The initial subflow might have disappeared at the time when you call the getsockopt. To make sure that the information about the initial subflow is stored, you can let MPTCP know by using setsockopt on MPTCP_INFO with the flag MPTCP_INFO_FLAG_SAVE_MASTER (defined as 0x01):

    #define MPTCP_INFO_FLAG_SAVE_MASTER 0x01
    int val = MPTCP_INFO_FLAG_SAVE_MASTER;
    setsockopt(..., ..., MPTCP_INFO, &val, sizeof(val));

That way, even if the initial subflow has been closed, you will be able to get the information later on when you call the getsockopt.

Version == 0.91.x

To set a sysctl variable, just do:

  sysctl -w net.mptcp.[name of the variable]=[value]

net.mptcp.mptcp_enabled Disable/Enable MPTCP on this machine. Possible values are 0 or 1. (default 1)
Since release 0.89, you can set mptcp_enabled to 2. In this case MPTCP will be enabled if the application has set the socket-option MPTCP_ENABLED (value 26) to 1. MPTCP_ENABLED is part of SOL_TCP:
int enable = 1;
setsockopt(fd, SOL_TCP, MPTCP_ENABLED, &enable, sizeof(enable));
net.mptcp.mptcp_checksum Disable/Enable the MPTCP checksum. As described in the draft, both sides (client and server) have to disable DSS-checksums. If one side has it enabled, DSS-checksums will be used. Possible values are 0 or 1. (default 1)
net.mptcp.mptcp_syn_retries Specifies how often we retransmit a SYN with the MP_CAPABLE-option. After this, the SYN will not contain the MP_CAPABLE-option. This is to handle middleboxes that drop SYNs with unknown TCP options. Default is 3.

Configure the path-manager:

At run-time, to select one of the compiled path-managers just set the sysctl net.mptcp.mptcp_path_manager. You have the choice between:

'default': This path-manager actually does not do anything. The host won't announce different IP-addresses nor initiate the creation of new subflows. However, it will accept the passive creation of new subflows.
'fullmesh': It will create a full-mesh of subflows among all available subflows. Since v0.90 it is possible to create multiple subflows for each pair of IP-addresses. Just set /sys/module/mptcp_fullmesh/parameters/num_subflows to a value > 1.
'ndiffports': This one will create X subflows across the same pair of IP-addresses, modifying the source-port. To control the number of subflows (X), you can set the sysfs /sys/module/mptcp_ndiffports/parameters/num_subflows to a value > 1.
'binder': The path-manager using Loose Source Routing from the paper Binder: a system to aggregate multiple internet gateways in community networks.

Configure the scheduler:

We have a modular scheduler infrastructure. At compile-time you can select the schedulers that should be compiled in.

At run-time you can select one of the compiled schedulers through the sysctl net.mptcp.mptcp_scheduler. You have the choice between:

'default': This scheduler is the default one. It will first send data on subflows with the lowest RTT until their congestion-window is full. Then, it will start transmitting on the subflows with the next higher RTT.
'roundrobin': This scheduler will transmit traffic in a round-robin fashion. It is configurable, how many consecutive segments should be sent with the tunable "num_segments" in the sysfs (default 1). Additionally, you can set the boolean tunable "cwnd_limited", to specify whether the scheduler tries to fill the congestion window on all subflows (true) or whether it prefers to leave open space in the congestion window (false) to achieve real round-robin (even if the subflows have very different capacities) (defaults to true). In case you are unsure, never ever enable this scheduler. Its performance is bad and it is only interesting for academic/testing purposes. The default scheduler is the best known up to today.
'redundant': This scheduler will try to transmit the traffic on all available subflows in a redundant way. It is useful when one wants to achieve the lowest possible latency by sacrificing the bandwidth.

Version == 0.90.x

To set a sysctl variable, just do:

  sysctl -w net.mptcp.[name of the variable]=[value]

net.mptcp.mptcp_enabled Disable/Enable MPTCP on this machine. Possible values are 0 or 1. (default 1)
Since release 0.89, you can set mptcp_enabled to 2. In this case MPTCP will be enabled if the application has set the socket-option MPTCP_ENABLED (value 26) to 1. MPTCP_ENABLED is part of SOL_TCP:
int enable = 1;
setsockopt(fd, SOL_TCP, MPTCP_ENABLED, &enable, sizeof(enable));
net.mptcp.mptcp_checksum Disable/Enable the MPTCP checksum. As described in the draft, both sides (client and server) have to disable DSS-checksums. If one side has it enabled, DSS-checksums will be used. Possible values are 0 or 1. (default 1)
net.mptcp.mptcp_syn_retries Specifies how often we retransmit a SYN with the MP_CAPABLE-option. After this, the SYN will not contain the MP_CAPABLE-option. This is to handle middleboxes that drop SYNs with unknown TCP options. Default is 3.

Configure the path-manager:

At run-time, to select one of the compiled path-managers just set the sysctl net.mptcp.mptcp_path_manager. You have the choice between:

'default': This path-manager actually does not do anything. The host won't announce different IP-addresses nor initiate the creation of new subflows. However, it will accept the passive creation of new subflows.
'fullmesh': It will create a full-mesh of subflows among all available subflows. Since v0.90 it is possible to create multiple subflows for each pair of IP-addresses. Just set /sys/module/mptcp_fullmesh/parameters/num_subflows to a value > 1.
'ndiffports': This one will create X subflows across the same pair of IP-addresses, modifying the source-port. To control the number of subflows (X), you can set the sysfs /sys/module/mptcp_ndiffports/parameters/num_subflows to a value > 1.
'binder': The path-manager using Loose Source Routing from the paper Binder: a system to aggregate multiple internet gateways in community networks.

Configure the scheduler:

We have a modular scheduler infrastructure. At compile-time you can select the schedulers that should be compiled in.

At run-time you can select one of the compiled schedulers through the sysctl net.mptcp.mptcp_scheduler. You have the choice between:

'default': This scheduler is the default one. It will first send data on subflows with the lowest RTT until their congestion-window is full. Then, it will start transmitting on the subflows with the next higher RTT.
'roundrobin': This scheduler will transmit traffic in a round-robin fashion. It is configurable, how many consecutive segments should be sent with the tunable "num_segments" in the sysfs (default 1). Additionally, you can set the boolean tunable "cwnd_limited", to specify whether the scheduler tries to fill the congestion window on all subflows (true) or whether it prefers to leave open space in the congestion window (false) to achieve real round-robin (even if the subflows have very different capacities) (defaults to true). In case you are unsure, never ever enable this scheduler. Its performance is bad and it is only interesting for academic/testing purposes. The default scheduler is the best known up to today.

Version == 0.89.x

To set a sysctl variable, just do:

  sysctl -w net.mptcp.[name of the variable]=[value]

net.mptcp.mptcp_enabled Disable/Enable MPTCP on this machine. Possible values are 0 or 1. (default 1)
Since release 0.89, you can set mptcp_enabled to 2. In this case MPTCP will be enabled if the application has set the socket-option MPTCP_ENABLED (value 26) to 1. MPTCP_ENABLED is part of SOL_TCP:
int enable = 1;
setsockopt(fd, SOL_TCP, MPTCP_ENABLED, &enable, sizeof(enable));
net.mptcp.mptcp_checksum Disable/Enable the MPTCP checksum. As described in the draft, both sides (client and server) have to disable DSS-checksums. If one side has it enabled, DSS-checksums will be used. Possible values are 0 or 1. (default 1)
net.mptcp.mptcp_syn_retries Specifies how often we retransmit a SYN with the MP_CAPABLE-option. After this, the SYN will not contain the MP_CAPABLE-option. This is to handle middleboxes that drop SYNs with unknown TCP options. Default is 3.

You can also configure different congestion controls. The available congestion controls are lia (alias Linked Increase Algorithm), olia (alias Opportunistic Linked Increase Algorithm) and wVegas (alias Delay-based Congestion Control for MPTCP). You can change them via the sysctl net.ipv4.tcp_congestion_control.

Configure the path-manager:

At run-time, to select one of the compiled path-managers just set the sysctl net.mptcp.mptcp_path_manager. You have the choice between:

'default': This path-manager actually does not do anything. The host won't announce different IP-addresses nor initiate the creation of new subflows. However, it will accept the passive creation of new subflows.
'fullmesh': It will create a full-mesh of subflows among all available subflows.
'ndiffports': This one will create X subflows across the same pair of IP-addresses, modifying the source-port. To control the number of subflows (X), you can set the sysfs /sys/module/mptcp_ndiffports/parameters/num_subflows to a value > 1.
'binder': The path-manager using Loose Source Routing from the paper Binder: a system to aggregate multiple internet gateways in community networks.

Configure the scheduler:

We have a modular scheduler infrastructure. At compile-time you can select the schedulers that should be compiled in.

At run-time you can select one of the compiled schedulers through the sysctl net.mptcp.mptcp_scheduler. You have the choice between:

'default': This scheduler is the default one. It will first send data on subflows with the lowest RTT until their congestion-window is full. Then, it will start transmitting on the subflows with the next higher RTT.
'roundrobin': This scheduler will transmit traffic in a round-robin fashion. It is configurable, how many consecutive segments should be sent with the tunable "num_segments" in the sysfs (default 1). Additionally, you can set the boolean tunable "cwnd_limited", to specify whether the scheduler tries to fill the congestion window on all subflows (true) or whether it prefers to leave open space in the congestion window (false) to achieve real round-robin (even if the subflows have very different capacities) (defaults to true). In case you are unsure, never ever enable this scheduler. Its performance is bad and it is only interesting for academic/testing purposes. The default scheduler is the best known up to today.

Version == 0.88.x

To set a sysctl variable, just do:

  sysctl -w net.mptcp.[name of the variable]=[value]

net.mptcp.mptcp_enabled Disable/Enable MPTCP on this machine. Possible values are 0 or 1. (default 1)
net.mptcp.mptcp_checksum Disable/Enable the MPTCP checksum. As described in the draft, both sides (client and server) have to disable DSS-checksums. If one side has it enabled, DSS-checksums will be used. Possible values are 0 or 1. (default 1)
net.mptcp.mptcp_syn_retries Specifies how often we retransmit a SYN with the MP_CAPABLE-option. After this, the SYN will not contain the MP_CAPABLE-option. This is to handle middleboxes that drop SYNs with unknown TCP options. Default is 3.

Configure the path-manager:

Since the release v0.88 we have a modular structure for the path-manager. At compile-time you can enable the path-managers through "MPTCP: advanced path-manager control" and select for example the full-mesh path-manager. If you do not select a path-manager, the host will not trigger the creation of new subflows, nor advertise alternative IP-addresses through the ADD_ADDR-option.

At run-time, to select one of the compiled path-managers just set the sysctl net.mptcp.mptcp_path_manager. You have the choice between:

'default': This path-manager actually does not do anything. The host won't announce different IP-addresses nor initiate the creation of new subflows. However, it will accept the passive creation of new subflows.
'fullmesh': It will create a full-mesh of subflows among all available subflows.
'ndiffports': This one will create X subflows across the same pair of IP-addresses, modifying the source-port. To control the number of subflows (X), you can set the sysctl net.mptcp.mptcp_ndiffports to a value > 1.

Version <= 0.87.x

To set a sysctl variable, just do:

  sysctl -w net.mptcp.[name of the variable]=[value]

net.mptcp.mptcp_enabled Disable/Enable MPTCP on this machine. Possible values are 0 or 1. (default 1)
net.mptcp.mptcp_checksum Disable/Enable the MPTCP checksum. As described in the draft, both sides (client and server) have to disable DSS-checksums. If one side has it enabled, DSS-checksums will be used. Possible values are 0 or 1. (default 1)
net.mptcp.mptcp_syn_retries Specifies how often we retransmit a SYN with the MP_CAPABLE-option. After this, the SYN will not contain the MP_CAPABLE-option. This is to handle middleboxes that drop SYNs with unknown TCP options. Default is 3.
net.mptcp.mptcp_ndiffports Allows to create several subflows among the same IP-addresses (the ones of the initial subflow). If the sysctl's value equals 1, regular MPTCP behavior tries to create a full mesh across all addresses. If it's greater than 1, the specified number of subflows will be created. (default 1)

Congestion-controls:

coupled (alias Linked Increase Algorithm) and olia (alias Opportunistic Linked Increase Algorithm) are available through net.ipv4.tcp_congestion_control.