When we're making a request, we're getting all progress headers in the end of query (when query finished and we got the response)
How can we read stream of headers during execution?
We can't provide a real example, but that's how you can reproduce it with curl if you have some clickhouse node to access:
echo "SOME LONG QUERY WITH PROGRESS" | curl -vvv -XPOST 'clickhouse_node:8123/?user=USER&send_progress_in_http_headers=1&query=' --data-binary @-
> POST /?user=USER&send_progress_in_http_headers=1&query= HTTP/1.1
> Host: clickhouse_node:8123
> User-Agent: curl/7.79.1
> Accept: */*
> Content-Length: 425
> Content-Type: application/x-www-form-urlencoded
>
* Mark bundle as not supporting multiuse
< HTTP/1.1 200 OK
< Date: Thu, 16 Nov 2023 14:10:50 GMT
< Connection: Keep-Alive
< Content-Type: text/tab-separated-values; charset=UTF-8
< Transfer-Encoding: chunked
< X-ClickHouse-Query-Id: QUERY_ID
< X-ClickHouse-Format: TabSeparated
< X-ClickHouse-Timezone: Zulu
< Keep-Alive: timeout=3
< X-ClickHouse-Progress: {"read_rows":"65661","read_bytes":"7794203","written_rows":"0","written_bytes":"0","total_rows_to_read":"47703090","result_rows":"0","result_bytes":"0"}
< X-ClickHouse-Progress: {"read_rows":"2983003","read_bytes":"355204021","written_rows":"0","written_bytes":"0","total_rows_to_read":"5267326958","result_rows":"0","result_bytes":"0"}
< X-ClickHouse-Progress: {"read_rows":"6136877","read_bytes":"729970296","written_rows":"0","written_bytes":"0","total_rows_to_read":"5267326958","result_rows":"0","result_bytes":"0"}
< X-ClickHouse-Progress: {"read_rows":"9207044","read_bytes":"1094719900","written_rows":"0","written_bytes":"0","total_rows_to_read":"5267326958","result_rows":"0","result_bytes":"0"}
< X-ClickHouse-Progress: {"read_rows":"12303523","read_bytes":"1462010911","written_rows":"0","written_bytes":"0","total_rows_to_read":"5267326958","result_rows":"0","result_bytes":"0"}
< X-ClickHouse-Progress: {"read_rows":"15088486","read_bytes":"1794558825","written_rows":"0","written_bytes":"0","total_rows_to_read":"5267326958","result_rows":"0","result_bytes":"0"}
As you can see above, curl allow to stream headers in realtime.
If it really needed, we can setup some dockerized example.
Clickhouse able to send query progress in header during query execution.
Documentation
Example (no anchor, just find send_progress_in_http_headers
When we're making a request, we're getting all progress headers in the end of query (when query finished and we got the response)
How can we read stream of headers during execution?
We can't provide a real example, but that's how you can reproduce it with curl if you have some clickhouse node to access:
As you can see above, curl allow to stream headers in realtime.
If it really needed, we can setup some dockerized example.