Optimizing MSQuic Large File Transfer Efficiency While Preventing Buffer Overflow

[lion117]

background

1. ubuntu20_x86_64
2. msquic V2.4.8

Issue Description:

I'm developing a large file transfer demo using MSQuic to compare transmission efficiency with TCP. After establishing the connection, I'm using the following loop structure:

while (true) {
    file.read(reinterpret_cast<char*>(buffer.data()), buffer.size());
    size_t bytesRead = file.gcount();
    
    if (bytesRead > 0) {
        QUIC_STATUS status = msQuic_->StreamSend(
            stream_,
            buffer,
            1,
            flags,
            buffer);
        
        if (QUIC_FAILED(status)) {
            LOGE("Failed to send data chunk");
            break;
        }
    }
    
    std::this_thread::sleep_for(milliseconds(10));
}

Problem:

I must add a 10ms sleep after each 4096-byte chunk send to ensure file integrity.
Without this delay, the receiver experiences packet loss/buffer overflow, corrupting the file.

Request:

How can I optimize sending efficiency while guaranteeing buffer stability? Specifically:

What mechanisms in MSQuic can prevent send buffer overflow?
How to implement proper flow control without manual delays?
Are there recommended patterns for bulk data streaming in MSQuic?
Should I adjust congestion control settings for large file transfers?

[nibanks]

You can't reuse your buffer object for a new send until the previous send complete. Use a unique buffer to simplify things.

[lion117]

ok, I will take a try

[lion117]

Every time data is read using a unique buffer for reading file data and passed to SendData(),

but the running result shows the server still has massive packet loss, receiving only about 5% of the data.

quesion

1. How can I send data at high speed without using sleep?
2. Can you show me a case in pseudocode?

bool QuicClient::SendData(const uint8_t* data, uint32_t length, bool fin) {
    std::lock_guard<std::mutex> lock(mutex_);
    if (!isConnected_ || !stream_){
        LOGE("not connect");
        return false;
    } 

    QUIC_BUFFER* buffer = new QUIC_BUFFER{length, const_cast<uint8_t*>(data)};
    QUIC_SEND_FLAGS flags = fin ? QUIC_SEND_FLAG_FIN : QUIC_SEND_FLAG_NONE;

    QUIC_STATUS status = msQuic_->StreamSend(
        stream_,
        buffer,
        1,
        flags,
        buffer);

    if (QUIC_FAILED(status)) {
        LOGE("StreamSend failed: 0x%x", status);
        delete buffer;
        return false;
    }
    return true;
}

send data loop

        while (file) {
                //std::vector<uint8_t> buffer(chunkSize);
                std::unique_ptr<uint8_t[]> buffer(new uint8_t[chunkSize]);
                file.read(reinterpret_cast<char*>(buffer.get()), chunkSize);
                size_t bytesRead = file.gcount();
                
                if (bytesRead > 0) {
                    if (client.SendData(buffer.get(), bytesRead)) {
              
                    } else {
                        LOGE("Failed to send data chunk");
                        break;
                    }
                }
                
                //std::this_thread::sleep_for(milliseconds(10));   // wait for data send complete
            }

[guhetier]

Hi,
There is some documentation to help you here: https://github.com/microsoft/msquic/blob/main/docs/Streams.md#sending
This sample should also help: https://github.com/microsoft/msquic/blob/main/src/tools/post/post.cpp

To get you started, the important bits are:

• create an event SendReady
• add a handler for QUIC_STREAM_EVENT_SEND_COMPLETE
  • in this handler, trigger SendReady
  • it means the previous "send" operation was completed by MsQuic, and you can send more data
• in your send loop:
  • call StreamSend with some data
  • wait on SendReady
  • reset SendReady
  • loop around to send the next chunk

Note that MsQuic notifies QUIC_STREAM_EVENT_SEND_COMPLETE as soon as it is ready for you to give more data: this will be high performance. It doesn't wait until the data has been received by the peer, only until the data is buffered internally.

[nibanks]

If you are using send buffering (which introduces another copy), this will work fine. If not, you need to keep multiple sends pending at any point in time, as well as at least the IDEAL_SEND_BUFFER bytes queued. All this is in the docs Guillaume linked above.