How to use a callback function for receiving chunked data while sending chunked data

[PasinduKollure]

Hi,

I want to use a httplib::SSLClient object to post chunks of data (using Transfer-Encoding: chunked) while an another thread receives chunked data from a server. After looking through the class ClientImpl, I was only able to find the Result Post(const char *path, ContentProviderWithoutLength content_provider, const char *content_type); which returns the response after all the data is sent whereas I would like a Post(...) function which processes incoming data simultaneously. Is this possible to implement?

Thanks

[PixlRainbow]

It appears we're missing a Post function with content receiver. That seems like an oversight.

---

Do note that even if we implement one, you would not be able to stream incoming data "at the same instant" as streaming an upload. The HTTP protocol is synchronous by design; a HTTP standards-compliant server will only start streaming the response after it has received all the request data. The only way to stream an upload and a download at the same time while being standards-compliant is to use two requests on two connections, as you have done, OR to use HTTP to negotiate onto a different protocol that allows asynchronous operation

[yhirose]

@PasinduKollure any response to @PixlRainbow's comment?

[PasinduKollure]

@yhirose @PixlRainbow the response makes sense, thank you!

[PasinduKollure]

Hi again!

I reopened this issue again because I want to ask if its possible to use the SSL_CTX* reference from *ssl_context() to read incoming chunks of data using SSL_peak().

This approach was inspired by this pull-request where it introduced ClientImpl::socket() and I was hoping *ssl_context() can be as useful for reading incoming messages.

Background on my application:

Currently, I have to send all the chunks in a single Post() call and wait for Post() to return the results of all the transmitted chunks. My intention is to use httplib::SSLClient to send chunks of data to a server and have a separate thread use the SSL_ctx to SSL_peak() the incoming chunks from the server as the client sends more chunks. I understand this may not conform with HTTP standards as PixlRainbow mentioned above, but I was hoping this 'approach' is possible.

[yhirose]

@PasinduKollure thanks for feedback again.

I understand this may not conform with HTTP standards as PixlRainbow mentioned above, but I was hoping this 'approach' is possible.

Even if you admit it's not a right thing to do according to the HTTP protocol, why you need to do it? If there is a valid reason and other HTTP libraries allow it, then I may think of accepting the idea.

[PixlRainbow]

That reminds me @yhirose -- what is httplib client's current behavior for a server that sends it's response "early" and then resets the connection?
The HTTP standard specifies that a server may send one response early before it has received the entire request body, and if the client detects this, it should stop sending the request body. This is used in the case where the server detects something wrong with a large payload mid-upload.
In practice, since most web browsers are not capable of detecting early responses, most servers will also reset the TCP/SSL connection after sending the early response to force the web browser to cancel it's upload and check the receive buffer for an error response from the server.

[yhirose]

@PixlRainbow, thank you for the information. Honestly, I didn't know such an 'early response from a server. Where do you know it is described in the HTTP spec? I'll try to implement it.

[PixlRainbow]

In RFC 2616, section 8.2.2

An HTTP/1.1 (or later) client sending a message-body SHOULD monitor the network connection for an error status while it is transmitting the request. If the client sees an error status, it SHOULD immediately cease transmitting the body.

However support for this spec is poor in many client implementations, so knowing this, many server implementations will additionally reset the connection