SetParam QUIC_PARAM_CONN_LOCAL_ADDRESS returns errno 97

[qzhuyan]

Describe the bug

This has been discussed in the discord channel but create the ticket for reference and visibility.

SetParam QUIC_PARAM_CONN_LOCAL_ADDRESS returns 97 when it changes the local addr address family from v6 to v4.

Lttng trace shows that

[0][41aa.48d9][09:08:21.510263][wrkr][0x7f2220079c40] IsActive = 1, Arg = 1
[0][41aa.48d9][09:08:21.510264][conn][0x7f221c131740] Scheduling: 2
[0][41aa.48d9][09:08:21.510264][wrkr][0x7f2220079c40] QueueDelay = 22
[0][41aa.48d9][09:08:21.510265][conn][0x7f221c131740] Execute: 11
[0][41aa.48d9][09:08:21.510266][conn][0x7f221c131740] New Local IP: 127.0.0.1:7190
[0][41aa.48d9][09:08:21.510274][ udp][0x7f21fc323a00] Created, local=127.0.0.1:7190, remote=[::1]:57873
[0][41aa.48d9][09:08:21.510301][ udp][0x7f21fc323a00] ERROR, 97, "connect failed".
[0][41aa.48d9][09:08:21.510306][ udp][0x7f21fc323a00] Destroyed
[0][41aa.48d9][09:08:21.510311][bind][0x7f22202723d0] ERROR, 97, "Create datapath binding".

Affected OS

• Windows Server 2022
• Windows 11
• Windows Insider Preview (specify affected build below)
• Ubuntu
• Debian
• Other (specify below)

Additional OS information

No response

MsQuic version

prerelease/1.6

Steps taken to reproduce bug

1. A Client started a connection to hostname "localhost" port 14567
2. Localhost is resolved to ipv6 address ::1 because ipv6 is preferred on the OS
   At this point, client doesn't know the Addr inuse.
3. The client call SetParam QUIC_PARAM_CONN_LOCAL_ADDRESS to set local addr to "127.0.0.1:0"

Expected behavior

In step3, if addr family is changed, msquic should attempt to create a new path with a new local binding with ipv4 addr family
In step3, SetParam should return the existing QUIC status code whatever the binding success or not.

Actual outcome

SetParam in step 3. returns 97 (EAFNOSUPPORT)

Additional details

This has been discussed in the discord channel but create the ticket for reference and visibility.

relates to the PR:
#2032

This issue is a missing feature as in:
#1946

API doc should be updated to reflect the missing feature.

[nibanks]

Practically, once you connect to a server of a given IP address, you're not going to be able to change address family (i.e. from IPv6 to IPv4) because you can't guarantee you will still be sending packets to the same server. So we're always going to need to return an error in this case. We could continue to let the UDP stack return its error, or we could check at our layer and return a QUIC_STATUS_INVALID_PARAMETER error instead.

[qzhuyan]

@nibanks thanks for reply, I have some considerations wheather msquic should support IPv4/IPv6 migration.
I am not talking about the behavior of SetParam, more like the API discussion .

If the server is running with dual stack, as in this case on localhost I think it could support.

After I change the address family, msquic should start to probing a new PATH with new src addr and dest addr, right? So why the address family matters here? Maybe it was stated in RFC somewhere?

In case the address is routed to different host that could not migrate this connection to, it is ok for msquic to return error.
If the path probe success, it should switch to that PATH.

I am not sure if this call should be a sync call or async call, consider it takes time for probing the new PATH.

I do think there are some scenarios that user switch networks from ipv4 to ipv6, like at home my mobile phone use wifi with ipv4 addr behind NAT, while when I move out of home the 4G/5G mobile network uses ipv6 that changes the address family.

[nibanks]

The problem is the server side, not on the client side. If you are currently communicating over family A to IP address Y, and you want to move to family B, then how do know what address Z to use that will guarantee you get back to the server you were talking to? There can be multiple server farms/units out there, with many different servers in each. Address Y is getting routed to one of those servers, but you have no way of figuring out address Z of family B would also route to that server.

So if even if you have an address from DNS (which might return lots of possibilities) for family B the likelihood is that it will not route correctly and your connection will die. I guess you could try to see if the path works first, and only use it if things go well, but just outright switching without first trying will likely be very bad.

So, for the time being, I'd like to limit things to the same address family for #1946.

[qzhuyan]

If you are currently communicating over family A to IP address Y, and you want to move to family B, then how do know what address Z to use that will guarantee you get back to the server you were talking to?

I think it is the loadbancer's job that supports quic protocol awareness that it can look into the quic packet's connection id and find the backend server?

The client may not need to take that into consideration (if the new path is reachable or not), just probe the path.

I guess you could try to see if the path works first, and only use it if things go well, but just outright switching without first trying will likely be very bad.

True I agree, path should be probed first then switch if I understand RFC9000 correctly.

So, for the time being, I'd like to limit things to the same address family for #1946.
Sure.

Does msquic support on listener listen on more than one address? or could two listeners share the info of connection, so a client could switch connection in between? I feel I should move the discussion to #1946 😃

[nibanks]

For the time being, I'm going close this issue as "By design". If we have a good solution for cross-address family, we can continue discussion on #1946.