Support a handler for checking connection status using Ping frame in HTTP/2

[raccoonback]

Description

In some cases, HTTP/2 connections may remain open even when they are no longer functional. By introducing a periodic Ping frame health check, we can proactively detect and close unhealthy connections.

The Http2ConnectionLivenessHandler schedules and sends Ping frames at a user-defined interval to verify the connection's liveness. If an acknowledgment (ACK) is not received within the specified time, the connection is considered unhealthy and is closed automatically.

However, if other frames are actively being sent or received, the scheduler does not send a Ping frame. This is because the server may delay ACK responses for various reasons. To prevent unnecessary connection termination, Ping frames are only sent when no read or write activity is detected.

Additionally, a configurable retry threshold for Ping frame transmission has been introduced. If a Ping frame fails to receive an ACK response, it will be retried up to the specified threshold before considering the connection unhealthy. This allows fine-tuning of the failure detection mechanism, balancing between aggressive failure detection and avoiding premature disconnections.

Scheduler Flow

Ping ACK Flow

Key Changes

1. Added Http2ConnectionLivenessHandler handler to check connection health with Ping frames at a configurable interval.
2. Introduced a retry threshold setting to limit the number of Ping transmission attempts before marking the connection as unhealthy.
3. Specify the ping frame ack allowable range through Http2SettingsSpec.

HttpClient.create()
	.protocol(HttpProtocol.H2)
	.secure()            
	.http2Settings(
		 builder -> builder.pingAckTimeout(Duration.ofMillis(600))  // Max wait time for ACK
			.pingScheduleInterval(Duration.ofMillis(300))  // Interval for sending PING frames
			.pingAckDropThreshold(2)  // Maximum retries before considering the connection dead
         );

Related Issue

• Fix HTTP/2 PING frame handling #3301

[violetagg]

@raccoonback Can you rebase against branch 1.2.x and switch the target branch for the PR to be 1.2.x. We branched and now in main it is the next development 1.3.x while we want this feature for version 1.2.x

[raccoonback]

@violetagg
Hello!
I have rebased the branch onto 1.2.x and changed the target branch of the PR to 1.2.x.

[violetagg]

@raccoonback I'll review this PR later this week

[violetagg]

@raccoonback Nice idea! I think that we can reuse our current reactor.netty.http.server.IdleTimeoutHandler (we need to check whether the current position in the pipeline will be ok or whether we need to move it) and extend it with the HTTP/2 functionality ACK requirement. At the moment it handles idle timeout for HTTP/2 and H2C on the server. The more ChannelHandler you have in the pipeline the worse is the performance so we always try to keep the ChannelHandlers number to the minimum if possible. Also all methods in reactor.netty.http.server.IdleTimeoutHandler are invoked on the event loop which guarantees that the functionality is thread-safe.

I think this feature is interesting for both server and client, similar to TCP keep-alive that we support for both of them.

[raccoonback]

@violetagg
Hello, and thanks for your suggestion! 😃
I have a few questions for clarification.

1. You suggested reusing reactor.netty.http.server.IdleTimeoutHandler.
   Currently, the implementation works by scheduling a PING frame transmission when there is no read/write activity, then checking whether an ACK is received within the allowed time. (It also supports retries up to a certain threshold.)
   '
   I understand that you’re suggesting leveraging IdleTimeoutHandler to detect when both read and write operations are idle and incorporating it into this logic. Is that correct?
   Specifically, do you mean modifying the handler so that when both read and write operations are idle, a PING frame is sent, and if no ACK is received within the retry threshold, the connection is closed?
   '
   If I’ve misunderstood, could you clarify what exactly you mean by "reuse"? 😮

2. Can I understand that you are proposing to extend this feature not only on the client side, but also on the server side by IdleTimeoutHandler?

   • I thought that this Issue was intended to introduce PING frame support only for the client-side at first.

Looking forward to your feedback! 😊

[violetagg]

@violetagg Hello, and thanks for your suggestion! 😃 I have a few questions for clarification.

1. You suggested reusing reactor.netty.http.server.IdleTimeoutHandler.
   Currently, the implementation works by scheduling a PING frame transmission when there is no read/write activity, then checking whether an ACK is received within the allowed time. (It also supports retries up to a certain threshold.)
   '
   I understand that you’re suggesting leveraging IdleTimeoutHandler to detect when both read and write operations are idle and incorporating it into this logic. Is that correct?

Yes
Currently we use it only for checking that there is no read, because we add this handler between the requests and we are sure there is no write operation. However this handler extends io.netty.handler.timeout.IdleStateHandler and for HTTP/2 we can configure it to check for both read and write.

Specifically, do you mean modifying the handler so that when both read and write operations are idle, a PING frame is sent, and if no ACK is received within the retry threshold, the connection is closed?

Yes
When we receive channelIdle we can add the logic for ACK

'
If I’ve misunderstood, could you clarify what exactly you mean by "reuse"? 😮
2. Can I understand that you are proposing to extend this feature not only on the client side, but also on the server side by IdleTimeoutHandler?

Yes

• I thought that this Issue was intended to introduce PING frame support only for the client-side at first.

Looking forward to your feedback! 😊

We may need to move reactor.netty.http.server.IdleTimeoutHandler to the reactor.netty.http package so that it can be used by both server and client.

I think that this feature is interesting for both client and server. Wdyt?

[raccoonback]

@violetagg

Thank you for the detailed explanation!
I will update reactor.netty.http.server.IdleTimeoutHandler to check the Ping ACK internally when usingHTTP/2 and H2C.

[raccoonback]

@violetagg
Hello!
I will reflect your PR review soon.
Thanks!

[violetagg]

@raccoonback I plan to review this PR next week.

[violetagg]

The maximum time (resolution: ms) that this connection stays opened and waits for HTTP request

Can you please elaborate more on this?

This is HttpClient so it initiates the HTTP request. (May be you mean the connection stays idle in the connection pool?) As part of the connection pool we already have a configuration for dropping the idle connections so this new configuration seems to be not needed? (check Http2Pool)

[raccoonback]

@violetagg
This describes configuring IdleTimeout on the HttpClient side using IdleTimeoutHandler, independently of the connection pool settings.

From my understanding, since the connection pool already manages idle connections, applying an additional IdleTimeoutHandler at the HttpClient level may be redundant.
Could you please confirm if this understanding is accurate?

[violetagg]

yes I think it is redundant

[raccoonback]

@violetagg
Understood!
Then, I will exclude the IdleTimeoutHandler on the client side.