Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Significant Lag on Last Packet with HTTPClient over HTTP, but Not HTTPS #10598

Open
1 task done
rvbc1 opened this issue Nov 12, 2024 · 2 comments
Open
1 task done

Significant Lag on Last Packet with HTTPClient over HTTP, but Not HTTPS #10598

rvbc1 opened this issue Nov 12, 2024 · 2 comments
Assignees
@SuGlider
Labels
IDE: PlaformIO Issue relates to PlatformIO IDE Status: Solved Type: Question Only question

Comments

@rvbc1
Copy link
Contributor

rvbc1 commented Nov 12, 2024 •
edited
Loading

Board

ESP32-S3-DevKitC

Device Description

DevKitC

Hardware Configuration

No external hardware.

Version

v3.0.7

IDE Name

Platformio

Operating System

Linux - Ubuntu 24.04

Flash frequency

40MHz

PSRAM enabled

no

Upload speed

115200

Description

My pio:
platform = https://github.com/pioarduino/platform-espressif32/releases/download/51.03.07/platform-espressif32.zip
board = esp32-s3-devkitc-1
framework = arduino

Using HTTPClient over HTTP results in a significant delay on the last packet (around 5 seconds), whereas this delay doesn’t occur over HTTPS. In general, HTTPS is also noticeably faster, which is surprising since encryption is expected to add some overhead.

The file being downloaded is the same in both cases, from the same server, and the only difference in the URL is whether it starts with HTTP or HTTPS.

Could someone help me understand if I’m doing something wrong, or if this might be a bug?”

Sketch

#pragma once

#include "Drivers/String/Driver.h"
#include "Drivers/Timer/Software/Driver.h"
#include "HTTPClient.h"
#include "NetworkClientSecure.h"

namespace Drivers {
    namespace HttpDownloader {
        class Donwloader {
           private:
            auto &getStream() { return *http.getStreamPtr(); }
            void  end() {
                http.end();
                client.stop();
                clientSecure.stop();
            }

           public:
            Donwloader() {}
            ~Donwloader() {}

            std::optional<size_t> start(Drivers::string url) {
                downloadRetry     = 0;
                contentDownloaded = 0;
                contentSize       = 0;
                http.begin(getClient(url), url.c_str());
                int httpResponseCode = http.GET();
                if (httpResponseCode == OK_CODE) {
                    contentSize = http.getSize();
                    return contentSize;
                }
                return std::nullopt;
            }

            std::optional<size_t> process(const uint8_t *buffer, size_t bufferSize) {
                if (softwareTimer.process([] {})) {
                    if (contentDownloaded < contentSize) {
                        auto &stream = getStream();
                        if (stream.available()) {
                            downloadRetry   = 0;
                            auto time       = millis();
                            auto downloaded = stream.readBytes(const_cast<uint8_t *>(buffer), bufferSize);
                            contentDownloaded += downloaded;
                            DEBUG_MESSAGE("Downloaded: " + Drivers::to_string(downloaded) + " | " + Drivers::to_string(contentDownloaded) + "/" + Drivers::to_string(contentSize) + " | " + Drivers::to_string(millis() - time) + "ms");
                            return downloaded;
                        } else {
                            WARNING_MESSAGE("Download retry: " + Drivers::to_string(downloadRetry));
                            if (downloadRetry++ >= DOWNLOAD_RETRY_MAX) {
                                end();
                                return std::nullopt;
                            }
                        }
                    } else {
                        end();
                        return std::nullopt;
                    }
                }
                return 0;
            }

           private:
            NetworkClient &getClient(Drivers::string url) {
                client       = NetworkClient();
                clientSecure = NetworkClientSecure();
                clientSecure.setInsecure();
                if (url.compare(0, 8, "https://") == 0) { return clientSecure; }
                return client;
            }

            using SoftwareTimerType = Drivers::Timer::Software::Driver;

            inline static constexpr const uint8_t  DOWNLOAD_RETRY_MAX = 30;
            inline static constexpr const uint16_t OK_CODE            = 200;

            uint8_t             downloadRetry     = 0;
            size_t              contentDownloaded = 0;
            size_t              contentSize       = 0;
            NetworkClient       client;
            NetworkClientSecure clientSecure;
            HTTPClient          http;

            SoftwareTimerType softwareTimer = SoftwareTimerType(SoftwareTimerType::ConfigType {.timeMs = 100});
        };
    }
}

Debug Message

HTTP:
[.pio/libdeps/hw_v1_debug/ESP32-Drivers/Drivers/HttpDownloader/Downloader.h:45] Downloaded: 4096 | 4096/306743 | 20ms
[.pio/libdeps/hw_v1_debug/ESP32-Drivers/Drivers/HttpDownloader/Downloader.h:45] Downloaded: 4096 | 8192/306743 | 22ms
[.pio/libdeps/hw_v1_debug/ESP32-Drivers/Drivers/HttpDownloader/Downloader.h:45] Downloaded: 4096 | 12288/306743 | 1ms
[.pio/libdeps/hw_v1_debug/ESP32-Drivers/Drivers/HttpDownloader/Downloader.h:45] Downloaded: 4096 | 16384/306743 | 27ms
[.pio/libdeps/hw_v1_debug/ESP32-Drivers/Drivers/HttpDownloader/Downloader.h:45] Downloaded: 4096 | 20480/306743 | 24ms
[.pio/libdeps/hw_v1_debug/ESP32-Drivers/Drivers/HttpDownloader/Downloader.h:45] Downloaded: 4096 | 24576/306743 | 26ms
[.pio/libdeps/hw_v1_debug/ESP32-Drivers/Drivers/HttpDownloader/Downloader.h:45] Downloaded: 4096 | 28672/306743 | 29ms
[.pio/libdeps/hw_v1_debug/ESP32-Drivers/Drivers/HttpDownloader/Downloader.h:45] Downloaded: 4096 | 32768/306743 | 33ms
[.pio/libdeps/hw_v1_debug/ESP32-Drivers/Drivers/HttpDownloader/Downloader.h:45] Downloaded: 4096 | 36864/306743 | 30ms
[.pio/libdeps/hw_v1_debug/ESP32-Drivers/Drivers/HttpDownloader/Downloader.h:45] Downloaded: 4096 | 40960/306743 | 35ms
[.pio/libdeps/hw_v1_debug/ESP32-Drivers/Drivers/HttpDownloader/Downloader.h:45] Downloaded: 4096 | 45056/306743 | 1ms
[.pio/libdeps/hw_v1_debug/ESP32-Drivers/Drivers/HttpDownloader/Downloader.h:45] Downloaded: 4096 | 49152/306743 | 1ms
[.pio/libdeps/hw_v1_debug/ESP32-Drivers/Drivers/HttpDownloader/Downloader.h:45] Downloaded: 4096 | 53248/306743 | 1ms
[.pio/libdeps/hw_v1_debug/ESP32-Drivers/Drivers/HttpDownloader/Downloader.h:45] Downloaded: 4096 | 57344/306743 | 1ms
[.pio/libdeps/hw_v1_debug/ESP32-Drivers/Drivers/HttpDownloader/Downloader.h:45] Downloaded: 4096 | 61440/306743 | 15ms
[.pio/libdeps/hw_v1_debug/ESP32-Drivers/Drivers/HttpDownloader/Downloader.h:45] Downloaded: 4096 | 65536/306743 | 1ms
[.pio/libdeps/hw_v1_debug/ESP32-Drivers/Drivers/HttpDownloader/Downloader.h:45] Downloaded: 4096 | 69632/306743 | 17ms
[.pio/libdeps/hw_v1_debug/ESP32-Drivers/Drivers/HttpDownloader/Downloader.h:45] Downloaded: 4096 | 73728/306743 | 1ms
[.pio/libdeps/hw_v1_debug/ESP32-Drivers/Drivers/HttpDownloader/Downloader.h:45] Downloaded: 4096 | 77824/306743 | 1ms
[.pio/libdeps/hw_v1_debug/ESP32-Drivers/Drivers/HttpDownloader/Downloader.h:45] Downloaded: 4096 | 81920/306743 | 21ms
[.pio/libdeps/hw_v1_debug/ESP32-Drivers/Drivers/HttpDownloader/Downloader.h:45] Downloaded: 4096 | 86016/306743 | 1ms
[.pio/libdeps/hw_v1_debug/ESP32-Drivers/Drivers/HttpDownloader/Downloader.h:45] Downloaded: 4096 | 90112/306743 | 1ms
[.pio/libdeps/hw_v1_debug/ESP32-Drivers/Drivers/HttpDownloader/Downloader.h:45] Downloaded: 4096 | 94208/306743 | 1ms
[.pio/libdeps/hw_v1_debug/ESP32-Drivers/Drivers/HttpDownloader/Downloader.h:45] Downloaded: 4096 | 98304/306743 | 1ms
[.pio/libdeps/hw_v1_debug/ESP32-Drivers/Drivers/HttpDownloader/Downloader.h:45] Downloaded: 4096 | 102400/306743 | 54ms
[.pio/libdeps/hw_v1_debug/ESP32-Drivers/Drivers/HttpDownloader/Downloader.h:45] Downloaded: 4096 | 106496/306743 | 30ms
[.pio/libdeps/hw_v1_debug/ESP32-Drivers/Drivers/HttpDownloader/Downloader.h:45] Downloaded: 4096 | 110592/306743 | 59ms
[.pio/libdeps/hw_v1_debug/ESP32-Drivers/Drivers/HttpDownloader/Downloader.h:45] Downloaded: 4096 | 114688/306743 | 33ms
[.pio/libdeps/hw_v1_debug/ESP32-Drivers/Drivers/HttpDownloader/Downloader.h:45] Downloaded: 4096 | 118784/306743 | 35ms
[.pio/libdeps/hw_v1_debug/ESP32-Drivers/Drivers/HttpDownloader/Downloader.h:45] Downloaded: 4096 | 122880/306743 | 63ms
[.pio/libdeps/hw_v1_debug/ESP32-Drivers/Drivers/HttpDownloader/Downloader.h:45] Downloaded: 4096 | 126976/306743 | 37ms
[.pio/libdeps/hw_v1_debug/ESP32-Drivers/Drivers/HttpDownloader/Downloader.h:45] Downloaded: 4096 | 131072/306743 | 38ms
[.pio/libdeps/hw_v1_debug/ESP32-Drivers/Drivers/HttpDownloader/Downloader.h:45] Downloaded: 4096 | 135168/306743 | 63ms
[.pio/libdeps/hw_v1_debug/ESP32-Drivers/Drivers/HttpDownloader/Downloader.h:45] Downloaded: 4096 | 139264/306743 | 42ms
[.pio/libdeps/hw_v1_debug/ESP32-Drivers/Drivers/HttpDownloader/Downloader.h:45] Downloaded: 4096 | 143360/306743 | 43ms
[.pio/libdeps/hw_v1_debug/ESP32-Drivers/Drivers/HttpDownloader/Downloader.h:45] Downloaded: 4096 | 147456/306743 | 44ms
[.pio/libdeps/hw_v1_debug/ESP32-Drivers/Drivers/HttpDownloader/Downloader.h:45] Downloaded: 4096 | 151552/306743 | 1ms
[.pio/libdeps/hw_v1_debug/ESP32-Drivers/Drivers/HttpDownloader/Downloader.h:45] Downloaded: 4096 | 155648/306743 | 1ms
[.pio/libdeps/hw_v1_debug/ESP32-Drivers/Drivers/HttpDownloader/Downloader.h:45] Downloaded: 4096 | 159744/306743 | 1ms
[.pio/libdeps/hw_v1_debug/ESP32-Drivers/Drivers/HttpDownloader/Downloader.h:45] Downloaded: 4096 | 163840/306743 | 1ms
[.pio/libdeps/hw_v1_debug/ESP32-Drivers/Drivers/HttpDownloader/Downloader.h:45] Downloaded: 4096 | 167936/306743 | 52ms
[.pio/libdeps/hw_v1_debug/ESP32-Drivers/Drivers/HttpDownloader/Downloader.h:45] Downloaded: 4096 | 172032/306743 | 53ms
[.pio/libdeps/hw_v1_debug/ESP32-Drivers/Drivers/HttpDownloader/Downloader.h:45] Downloaded: 4096 | 176128/306743 | 54ms
[.pio/libdeps/hw_v1_debug/ESP32-Drivers/Drivers/HttpDownloader/Downloader.h:45] Downloaded: 4096 | 180224/306743 | 1ms
[.pio/libdeps/hw_v1_debug/ESP32-Drivers/Drivers/HttpDownloader/Downloader.h:45] Downloaded: 4096 | 184320/306743 | 1ms
[.pio/libdeps/hw_v1_debug/ESP32-Drivers/Drivers/HttpDownloader/Downloader.h:45] Downloaded: 4096 | 188416/306743 | 1ms
[.pio/libdeps/hw_v1_debug/ESP32-Drivers/Drivers/HttpDownloader/Downloader.h:45] Downloaded: 4096 | 192512/306743 | 1ms
[.pio/libdeps/hw_v1_debug/ESP32-Drivers/Drivers/HttpDownloader/Downloader.h:45] Downloaded: 4096 | 196608/306743 | 1ms
[.pio/libdeps/hw_v1_debug/ESP32-Drivers/Drivers/HttpDownloader/Downloader.h:45] Downloaded: 4096 | 200704/306743 | 1ms
[.pio/libdeps/hw_v1_debug/ESP32-Drivers/Drivers/HttpDownloader/Downloader.h:45] Downloaded: 4096 | 204800/306743 | 1ms
[.pio/libdeps/hw_v1_debug/ESP32-Drivers/Drivers/HttpDownloader/Downloader.h:45] Downloaded: 4096 | 208896/306743 | 1ms
[.pio/libdeps/hw_v1_debug/ESP32-Drivers/Drivers/HttpDownloader/Downloader.h:45] Downloaded: 4096 | 212992/306743 | 1ms
[.pio/libdeps/hw_v1_debug/ESP32-Drivers/Drivers/HttpDownloader/Downloader.h:45] Downloaded: 4096 | 217088/306743 | 1ms
[.pio/libdeps/hw_v1_debug/ESP32-Drivers/Drivers/HttpDownloader/Downloader.h:45] Downloaded: 4096 | 221184/306743 | 1ms
[.pio/libdeps/hw_v1_debug/ESP32-Drivers/Drivers/HttpDownloader/Downloader.h:45] Downloaded: 4096 | 225280/306743 | 1ms
[.pio/libdeps/hw_v1_debug/ESP32-Drivers/Drivers/HttpDownloader/Downloader.h:45] Downloaded: 4096 | 229376/306743 | 1ms
[.pio/libdeps/hw_v1_debug/ESP32-Drivers/Drivers/HttpDownloader/Downloader.h:45] Downloaded: 4096 | 233472/306743 | 1ms
[.pio/libdeps/hw_v1_debug/ESP32-Drivers/Drivers/HttpDownloader/Downloader.h:45] Downloaded: 4096 | 237568/306743 | 1ms
[.pio/libdeps/hw_v1_debug/ESP32-Drivers/Drivers/HttpDownloader/Downloader.h:45] Downloaded: 4096 | 241664/306743 | 1ms
[.pio/libdeps/hw_v1_debug/ESP32-Drivers/Drivers/HttpDownloader/Downloader.h:45] Downloaded: 4096 | 245760/306743 | 1ms
[.pio/libdeps/hw_v1_debug/ESP32-Drivers/Drivers/HttpDownloader/Downloader.h:45] Downloaded: 4096 | 249856/306743 | 1ms
[.pio/libdeps/hw_v1_debug/ESP32-Drivers/Drivers/HttpDownloader/Downloader.h:45] Downloaded: 4096 | 253952/306743 | 6ms
[.pio/libdeps/hw_v1_debug/ESP32-Drivers/Drivers/HttpDownloader/Downloader.h:45] Downloaded: 4096 | 258048/306743 | 3ms
[.pio/libdeps/hw_v1_debug/ESP32-Drivers/Drivers/HttpDownloader/Downloader.h:45] Downloaded: 4096 | 262144/306743 | 1ms
[.pio/libdeps/hw_v1_debug/ESP32-Drivers/Drivers/HttpDownloader/Downloader.h:45] Downloaded: 4096 | 266240/306743 | 8ms
[.pio/libdeps/hw_v1_debug/ESP32-Drivers/Drivers/HttpDownloader/Downloader.h:45] Downloaded: 4096 | 270336/306743 | 1ms
[.pio/libdeps/hw_v1_debug/ESP32-Drivers/Drivers/HttpDownloader/Downloader.h:45] Downloaded: 4096 | 274432/306743 | 1ms
[.pio/libdeps/hw_v1_debug/ESP32-Drivers/Drivers/HttpDownloader/Downloader.h:45] Downloaded: 4096 | 278528/306743 | 1ms
[.pio/libdeps/hw_v1_debug/ESP32-Drivers/Drivers/HttpDownloader/Downloader.h:45] Downloaded: 4096 | 282624/306743 | 1ms
[.pio/libdeps/hw_v1_debug/ESP32-Drivers/Drivers/HttpDownloader/Downloader.h:45] Downloaded: 4096 | 286720/306743 | 16ms
[.pio/libdeps/hw_v1_debug/ESP32-Drivers/Drivers/HttpDownloader/Downloader.h:45] Downloaded: 4096 | 290816/306743 | 13ms
[.pio/libdeps/hw_v1_debug/ESP32-Drivers/Drivers/HttpDownloader/Downloader.h:45] Downloaded: 4096 | 294912/306743 | 18ms
[.pio/libdeps/hw_v1_debug/ESP32-Drivers/Drivers/HttpDownloader/Downloader.h:45] Downloaded: 4096 | 299008/306743 | 20ms
[.pio/libdeps/hw_v1_debug/ESP32-Drivers/Drivers/HttpDownloader/Downloader.h:45] Downloaded: 4096 | 303104/306743 | 18ms
[.pio/libdeps/hw_v1_debug/ESP32-Drivers/Drivers/HttpDownloader/Downloader.h:45] Downloaded: 3639 | 306743/306743 | 5001ms



HTTPS:
[.pio/libdeps/hw_v1_debug/ESP32-Drivers/Drivers/HttpDownloader/Downloader.h:45] Downloaded: 4096 | 4096/306743 | 5ms
[.pio/libdeps/hw_v1_debug/ESP32-Drivers/Drivers/HttpDownloader/Downloader.h:45] Downloaded: 3204 | 7300/306743 | 4ms
[.pio/libdeps/hw_v1_debug/ESP32-Drivers/Drivers/HttpDownloader/Downloader.h:45] Downloaded: 4096 | 11396/306743 | 3ms
[.pio/libdeps/hw_v1_debug/ESP32-Drivers/Drivers/HttpDownloader/Downloader.h:45] Downloaded: 4096 | 15492/306743 | 5ms
[.pio/libdeps/hw_v1_debug/ESP32-Drivers/Drivers/HttpDownloader/Downloader.h:45] Downloaded: 4096 | 19588/306743 | 4ms
[.pio/libdeps/hw_v1_debug/ESP32-Drivers/Drivers/HttpDownloader/Downloader.h:45] Downloaded: 2771 | 22359/306743 | 4ms
[.pio/libdeps/hw_v1_debug/ESP32-Drivers/Drivers/HttpDownloader/Downloader.h:45] Downloaded: 4096 | 26455/306743 | 3ms
[.pio/libdeps/hw_v1_debug/ESP32-Drivers/Drivers/HttpDownloader/Downloader.h:45] Downloaded: 2749 | 29204/306743 | 3ms
[.pio/libdeps/hw_v1_debug/ESP32-Drivers/Drivers/HttpDownloader/Downloader.h:48] Download retry: 0
[.pio/libdeps/hw_v1_debug/ESP32-Drivers/Drivers/HttpDownloader/Downloader.h:45] Downloaded: 4096 | 33300/306743 | 3ms
[.pio/libdeps/hw_v1_debug/ESP32-Drivers/Drivers/HttpDownloader/Downloader.h:45] Downloaded: 1390 | 34690/306743 | 3ms
[.pio/libdeps/hw_v1_debug/ESP32-Drivers/Drivers/HttpDownloader/Downloader.h:45] Downloaded: 1369 | 36059/306743 | 0ms
[.pio/libdeps/hw_v1_debug/ESP32-Drivers/Drivers/HttpDownloader/Downloader.h:45] Downloaded: 4096 | 40155/306743 | 3ms
[.pio/libdeps/hw_v1_debug/ESP32-Drivers/Drivers/HttpDownloader/Downloader.h:45] Downloaded: 1380 | 41535/306743 | 2ms
[.pio/libdeps/hw_v1_debug/ESP32-Drivers/Drivers/HttpDownloader/Downloader.h:45] Downloaded: 4096 | 45631/306743 | 2ms
[.pio/libdeps/hw_v1_debug/ESP32-Drivers/Drivers/HttpDownloader/Downloader.h:45] Downloaded: 2749 | 48380/306743 | 3ms
[.pio/libdeps/hw_v1_debug/ESP32-Drivers/Drivers/HttpDownloader/Downloader.h:45] Downloaded: 2738 | 51118/306743 | 2ms
[.pio/libdeps/hw_v1_debug/ESP32-Drivers/Drivers/HttpDownloader/Downloader.h:45] Downloaded: 2738 | 53856/306743 | 2ms
[.pio/libdeps/hw_v1_debug/ESP32-Drivers/Drivers/HttpDownloader/Downloader.h:45] Downloaded: 4096 | 57952/306743 | 0ms
[.pio/libdeps/hw_v1_debug/ESP32-Drivers/Drivers/HttpDownloader/Downloader.h:45] Downloaded: 4096 | 62048/306743 | 2ms
[.pio/libdeps/hw_v1_debug/ESP32-Drivers/Drivers/HttpDownloader/Downloader.h:45] Downloaded: 266 | 62314/306743 | 0ms
[.pio/libdeps/hw_v1_debug/ESP32-Drivers/Drivers/HttpDownloader/Downloader.h:45] Downloaded: 2406 | 64720/306743 | 1ms
[.pio/libdeps/hw_v1_debug/ESP32-Drivers/Drivers/HttpDownloader/Downloader.h:45] Downloaded: 4096 | 68816/306743 | 0ms
[.pio/libdeps/hw_v1_debug/ESP32-Drivers/Drivers/HttpDownloader/Downloader.h:45] Downloaded: 4096 | 72912/306743 | 3ms
[.pio/libdeps/hw_v1_debug/ESP32-Drivers/Drivers/HttpDownloader/Downloader.h:45] Downloaded: 266 | 73178/306743 | 0ms
[.pio/libdeps/hw_v1_debug/ESP32-Drivers/Drivers/HttpDownloader/Downloader.h:45] Downloaded: 4096 | 77274/306743 | 1ms
[.pio/libdeps/hw_v1_debug/ESP32-Drivers/Drivers/HttpDownloader/Downloader.h:45] Downloaded: 133 | 77407/306743 | 0ms
[.pio/libdeps/hw_v1_debug/ESP32-Drivers/Drivers/HttpDownloader/Downloader.h:45] Downloaded: 4096 | 81503/306743 | 0ms
[.pio/libdeps/hw_v1_debug/ESP32-Drivers/Drivers/HttpDownloader/Downloader.h:45] Downloaded: 133 | 81636/306743 | 1ms
[.pio/libdeps/hw_v1_debug/ESP32-Drivers/Drivers/HttpDownloader/Downloader.h:45] Downloaded: 4096 | 85732/306743 | 0ms
[.pio/libdeps/hw_v1_debug/ESP32-Drivers/Drivers/HttpDownloader/Downloader.h:45] Downloaded: 4096 | 89828/306743 | 2ms
[.pio/libdeps/hw_v1_debug/ESP32-Drivers/Drivers/HttpDownloader/Downloader.h:45] Downloaded: 266 | 90094/306743 | 1ms
[.pio/libdeps/hw_v1_debug/ESP32-Drivers/Drivers/HttpDownloader/Downloader.h:45] Downloaded: 4096 | 94190/306743 | 0ms
[.pio/libdeps/hw_v1_debug/ESP32-Drivers/Drivers/HttpDownloader/Downloader.h:45] Downloaded: 3298 | 97488/306743 | 3ms
[.pio/libdeps/hw_v1_debug/ESP32-Drivers/Drivers/HttpDownloader/Downloader.h:48] Download retry: 0
[.pio/libdeps/hw_v1_debug/ESP32-Drivers/Drivers/HttpDownloader/Downloader.h:45] Downloaded: 4096 | 101584/306743 | 0ms
[.pio/libdeps/hw_v1_debug/ESP32-Drivers/Drivers/HttpDownloader/Downloader.h:45] Downloaded: 4096 | 105680/306743 | 2ms
[.pio/libdeps/hw_v1_debug/ESP32-Drivers/Drivers/HttpDownloader/Downloader.h:45] Downloaded: 266 | 105946/306743 | 1ms
[.pio/libdeps/hw_v1_debug/ESP32-Drivers/Drivers/HttpDownloader/Downloader.h:45] Downloaded: 4096 | 110042/306743 | 0ms
[.pio/libdeps/hw_v1_debug/ESP32-Drivers/Drivers/HttpDownloader/Downloader.h:45] Downloaded: 4096 | 114138/306743 | 2ms
[.pio/libdeps/hw_v1_debug/ESP32-Drivers/Drivers/HttpDownloader/Downloader.h:45] Downloaded: 266 | 114404/306743 | 1ms
[.pio/libdeps/hw_v1_debug/ESP32-Drivers/Drivers/HttpDownloader/Downloader.h:45] Downloaded: 4096 | 118500/306743 | 0ms
[.pio/libdeps/hw_v1_debug/ESP32-Drivers/Drivers/HttpDownloader/Downloader.h:45] Downloaded: 4096 | 122596/306743 | 2ms
[.pio/libdeps/hw_v1_debug/ESP32-Drivers/Drivers/HttpDownloader/Downloader.h:45] Downloaded: 266 | 122862/306743 | 1ms
[.pio/libdeps/hw_v1_debug/ESP32-Drivers/Drivers/HttpDownloader/Downloader.h:45] Downloaded: 4096 | 126958/306743 | 0ms
[.pio/libdeps/hw_v1_debug/ESP32-Drivers/Drivers/HttpDownloader/Downloader.h:45] Downloaded: 3298 | 130256/306743 | 3ms
[.pio/libdeps/hw_v1_debug/ESP32-Drivers/Drivers/HttpDownloader/Downloader.h:48] Download retry: 0
[.pio/libdeps/hw_v1_debug/ESP32-Drivers/Drivers/HttpDownloader/Downloader.h:45] Downloaded: 4096 | 134352/306743 | 1ms
[.pio/libdeps/hw_v1_debug/ESP32-Drivers/Drivers/HttpDownloader/Downloader.h:45] Downloaded: 4096 | 138448/306743 | 2ms
[.pio/libdeps/hw_v1_debug/ESP32-Drivers/Drivers/HttpDownloader/Downloader.h:45] Downloaded: 266 | 138714/306743 | 1ms
[.pio/libdeps/hw_v1_debug/ESP32-Drivers/Drivers/HttpDownloader/Downloader.h:45] Downloaded: 4096 | 142810/306743 | 0ms
[.pio/libdeps/hw_v1_debug/ESP32-Drivers/Drivers/HttpDownloader/Downloader.h:45] Downloaded: 4096 | 146906/306743 | 2ms
[.pio/libdeps/hw_v1_debug/ESP32-Drivers/Drivers/HttpDownloader/Downloader.h:45] Downloaded: 266 | 147172/306743 | 1ms
[.pio/libdeps/hw_v1_debug/ESP32-Drivers/Drivers/HttpDownloader/Downloader.h:45] Downloaded: 4096 | 151268/306743 | 0ms
[.pio/libdeps/hw_v1_debug/ESP32-Drivers/Drivers/HttpDownloader/Downloader.h:45] Downloaded: 4096 | 155364/306743 | 2ms
[.pio/libdeps/hw_v1_debug/ESP32-Drivers/Drivers/HttpDownloader/Downloader.h:45] Downloaded: 266 | 155630/306743 | 1ms
[.pio/libdeps/hw_v1_debug/ESP32-Drivers/Drivers/HttpDownloader/Downloader.h:45] Downloaded: 4096 | 159726/306743 | 0ms
[.pio/libdeps/hw_v1_debug/ESP32-Drivers/Drivers/HttpDownloader/Downloader.h:45] Downloaded: 3298 | 163024/306743 | 3ms
[.pio/libdeps/hw_v1_debug/ESP32-Drivers/Drivers/HttpDownloader/Downloader.h:48] Download retry: 0
[.pio/libdeps/hw_v1_debug/ESP32-Drivers/Drivers/HttpDownloader/Downloader.h:45] Downloaded: 4096 | 167120/306743 | 1ms
[.pio/libdeps/hw_v1_debug/ESP32-Drivers/Drivers/HttpDownloader/Downloader.h:45] Downloaded: 4096 | 171216/306743 | 3ms
[.pio/libdeps/hw_v1_debug/ESP32-Drivers/Drivers/HttpDownloader/Downloader.h:45] Downloaded: 266 | 171482/306743 | 0ms
[.pio/libdeps/hw_v1_debug/ESP32-Drivers/Drivers/HttpDownloader/Downloader.h:45] Downloaded: 4096 | 175578/306743 | 1ms
[.pio/libdeps/hw_v1_debug/ESP32-Drivers/Drivers/HttpDownloader/Downloader.h:45] Downloaded: 4096 | 179674/306743 | 3ms
[.pio/libdeps/hw_v1_debug/ESP32-Drivers/Drivers/HttpDownloader/Downloader.h:45] Downloaded: 4096 | 183770/306743 | 3ms
[.pio/libdeps/hw_v1_debug/ESP32-Drivers/Drivers/HttpDownloader/Downloader.h:45] Downloaded: 4096 | 187866/306743 | 3ms
[.pio/libdeps/hw_v1_debug/ESP32-Drivers/Drivers/HttpDownloader/Downloader.h:45] Downloaded: 4096 | 191962/306743 | 3ms
[.pio/libdeps/hw_v1_debug/ESP32-Drivers/Drivers/HttpDownloader/Downloader.h:45] Downloaded: 3830 | 195792/306743 | 3ms
[.pio/libdeps/hw_v1_debug/ESP32-Drivers/Drivers/HttpDownloader/Downloader.h:45] Downloaded: 4096 | 199888/306743 | 0ms
[.pio/libdeps/hw_v1_debug/ESP32-Drivers/Drivers/HttpDownloader/Downloader.h:45] Downloaded: 4096 | 203984/306743 | 3ms
[.pio/libdeps/hw_v1_debug/ESP32-Drivers/Drivers/HttpDownloader/Downloader.h:45] Downloaded: 4096 | 208080/306743 | 3ms
[.pio/libdeps/hw_v1_debug/ESP32-Drivers/Drivers/HttpDownloader/Downloader.h:45] Downloaded: 4096 | 212176/306743 | 3ms
[.pio/libdeps/hw_v1_debug/ESP32-Drivers/Drivers/HttpDownloader/Downloader.h:45] Downloaded: 4096 | 216272/306743 | 3ms
[.pio/libdeps/hw_v1_debug/ESP32-Drivers/Drivers/HttpDownloader/Downloader.h:45] Downloaded: 665 | 216937/306743 | 1ms
[.pio/libdeps/hw_v1_debug/ESP32-Drivers/Drivers/HttpDownloader/Downloader.h:48] Download retry: 0
[.pio/libdeps/hw_v1_debug/ESP32-Drivers/Drivers/HttpDownloader/Downloader.h:45] Downloaded: 4096 | 221033/306743 | 1ms
[.pio/libdeps/hw_v1_debug/ESP32-Drivers/Drivers/HttpDownloader/Downloader.h:45] Downloaded: 4096 | 225129/306743 | 0ms
[.pio/libdeps/hw_v1_debug/ESP32-Drivers/Drivers/HttpDownloader/Downloader.h:45] Downloaded: 3431 | 228560/306743 | 1ms
[.pio/libdeps/hw_v1_debug/ESP32-Drivers/Drivers/HttpDownloader/Downloader.h:48] Download retry: 0
[.pio/libdeps/hw_v1_debug/ESP32-Drivers/Drivers/HttpDownloader/Downloader.h:45] Downloaded: 4096 | 232656/306743 | 0ms
[.pio/libdeps/hw_v1_debug/ESP32-Drivers/Drivers/HttpDownloader/Downloader.h:45] Downloaded: 4096 | 236752/306743 | 0ms
[.pio/libdeps/hw_v1_debug/ESP32-Drivers/Drivers/HttpDownloader/Downloader.h:45] Downloaded: 4096 | 240848/306743 | 0ms
[.pio/libdeps/hw_v1_debug/ESP32-Drivers/Drivers/HttpDownloader/Downloader.h:45] Downloaded: 4096 | 244944/306743 | 0ms
[.pio/libdeps/hw_v1_debug/ESP32-Drivers/Drivers/HttpDownloader/Downloader.h:48] Download retry: 0
[.pio/libdeps/hw_v1_debug/ESP32-Drivers/Drivers/HttpDownloader/Downloader.h:48] Download retry: 1
[.pio/libdeps/hw_v1_debug/ESP32-Drivers/Drivers/HttpDownloader/Downloader.h:48] Download retry: 2
[.pio/libdeps/hw_v1_debug/ESP32-Drivers/Drivers/HttpDownloader/Downloader.h:48] Download retry: 3
[.pio/libdeps/hw_v1_debug/ESP32-Drivers/Drivers/HttpDownloader/Downloader.h:45] Downloaded: 4096 | 249040/306743 | 1ms
[.pio/libdeps/hw_v1_debug/ESP32-Drivers/Drivers/HttpDownloader/Downloader.h:45] Downloaded: 4096 | 253136/306743 | 0ms
[.pio/libdeps/hw_v1_debug/ESP32-Drivers/Drivers/HttpDownloader/Downloader.h:45] Downloaded: 4096 | 257232/306743 | 0ms
[.pio/libdeps/hw_v1_debug/ESP32-Drivers/Drivers/HttpDownloader/Downloader.h:45] Downloaded: 4096 | 261328/306743 | 0ms
[.pio/libdeps/hw_v1_debug/ESP32-Drivers/Drivers/HttpDownloader/Downloader.h:48] Download retry: 0
[.pio/libdeps/hw_v1_debug/ESP32-Drivers/Drivers/HttpDownloader/Downloader.h:48] Download retry: 1
[.pio/libdeps/hw_v1_debug/ESP32-Drivers/Drivers/HttpDownloader/Downloader.h:48] Download retry: 2
[.pio/libdeps/hw_v1_debug/ESP32-Drivers/Drivers/HttpDownloader/Downloader.h:45] Downloaded: 4096 | 265424/306743 | 1ms
[.pio/libdeps/hw_v1_debug/ESP32-Drivers/Drivers/HttpDownloader/Downloader.h:45] Downloaded: 4096 | 269520/306743 | 0ms
[.pio/libdeps/hw_v1_debug/ESP32-Drivers/Drivers/HttpDownloader/Downloader.h:45] Downloaded: 4096 | 273616/306743 | 0ms
[.pio/libdeps/hw_v1_debug/ESP32-Drivers/Drivers/HttpDownloader/Downloader.h:45] Downloaded: 4096 | 277712/306743 | 0ms
[.pio/libdeps/hw_v1_debug/ESP32-Drivers/Drivers/HttpDownloader/Downloader.h:48] Download retry: 0
[.pio/libdeps/hw_v1_debug/ESP32-Drivers/Drivers/HttpDownloader/Downloader.h:48] Download retry: 1
[.pio/libdeps/hw_v1_debug/ESP32-Drivers/Drivers/HttpDownloader/Downloader.h:48] Download retry: 2
[.pio/libdeps/hw_v1_debug/ESP32-Drivers/Drivers/HttpDownloader/Downloader.h:48] Download retry: 3
[.pio/libdeps/hw_v1_debug/ESP32-Drivers/Drivers/HttpDownloader/Downloader.h:48] Download retry: 4
[.pio/libdeps/hw_v1_debug/ESP32-Drivers/Drivers/HttpDownloader/Downloader.h:45] Downloaded: 4096 | 281808/306743 | 0ms
[.pio/libdeps/hw_v1_debug/ESP32-Drivers/Drivers/HttpDownloader/Downloader.h:45] Downloaded: 4096 | 285904/306743 | 0ms
[.pio/libdeps/hw_v1_debug/ESP32-Drivers/Drivers/HttpDownloader/Downloader.h:45] Downloaded: 4096 | 290000/306743 | 0ms
[.pio/libdeps/hw_v1_debug/ESP32-Drivers/Drivers/HttpDownloader/Downloader.h:45] Downloaded: 4096 | 294096/306743 | 0ms
[.pio/libdeps/hw_v1_debug/ESP32-Drivers/Drivers/HttpDownloader/Downloader.h:48] Download retry: 0
[.pio/libdeps/hw_v1_debug/ESP32-Drivers/Drivers/HttpDownloader/Downloader.h:48] Download retry: 1
[.pio/libdeps/hw_v1_debug/ESP32-Drivers/Drivers/HttpDownloader/Downloader.h:45] Downloaded: 4096 | 298192/306743 | 1ms
[.pio/libdeps/hw_v1_debug/ESP32-Drivers/Drivers/HttpDownloader/Downloader.h:45] Downloaded: 4096 | 302288/306743 | 0ms
[.pio/libdeps/hw_v1_debug/ESP32-Drivers/Drivers/HttpDownloader/Downloader.h:45] Downloaded: 4096 | 306384/306743 | 0ms
[.pio/libdeps/hw_v1_debug/ESP32-Drivers/Drivers/HttpDownloader/Downloader.h:45] Downloaded: 359 | 306743/306743 | 0ms

Other Steps to Reproduce

No response

I have checked existing issues, online documentation and the Troubleshooting Guide

  • I confirm I have checked existing issues, online documentation and Troubleshooting guide.
@rvbc1 rvbc1 added the Status: Awaiting triage Issue is waiting for triage label Nov 12, 2024
@rvbc1 rvbc1 changed the title HTTPClient big lag on end on HTTP not on HTTPS Significant Lag on Last Packet with HTTPClient over HTTP, but Not HTTPS Nov 12, 2024
@SuGlider
Copy link
Collaborator

Default HTTPClient timeut is 5000 ms (5 seconds).
It can be fixed by adding setTimeout(time_ms);

           NetworkClient &getClient(Drivers::string url) {
                client       = NetworkClient();
                client.setTimeout(100);  // 100ms reading timeout
                clientSecure = NetworkClientSecure();
                clientSecure.setInsecure();
                if (url.compare(0, 8, "https://") == 0) { return clientSecure; }
                return client;
            }

Timeout affects Stream class, here:
auto downloaded = stream.readBytes(const_cast<uint8_t *>(buffer), bufferSize);

@SuGlider SuGlider added Type: Question Only question Status: Solved and removed Status: Awaiting triage Issue is waiting for triage labels Nov 13, 2024
@SuGlider SuGlider self-assigned this Nov 13, 2024
@SuGlider SuGlider added the IDE: PlaformIO Issue relates to PlatformIO IDE label Nov 14, 2024
@rvbc1
Copy link
Contributor Author

rvbc1 commented Nov 15, 2024

Default HTTPClient timeut is 5000 ms (5 seconds). It can be fixed by adding setTimeout(time_ms);

           NetworkClient &getClient(Drivers::string url) {
                client       = NetworkClient();
                client.setTimeout(100);  // 100ms reading timeout
                clientSecure = NetworkClientSecure();
                clientSecure.setInsecure();
                if (url.compare(0, 8, "https://") == 0) { return clientSecure; }
                return client;
            }

Timeout affects Stream class, here: auto downloaded = stream.readBytes(const_cast<uint8_t *>(buffer), bufferSize);

Unfortunately, the proposed solution did not help. That said, isn’t it a bug that when connecting via HTTP, the stream, after downloading the entire resource, waits for a timeout instead of terminating on its own? It seems it should automatically close after the resource is fully downloaded. Especially since this works perfectly fine over HTTPS

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees

@SuGlider SuGlider

Labels
IDE: PlaformIO Issue relates to PlatformIO IDE Status: Solved Type: Question Only question
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
2 participants
@rvbc1 @SuGlider