#1107973 Slow file upload, sends one byte at a time

Package:
w3m
Source:
w3m
Description:
WWW browsable pager with excellent tables/frames support
Submitter:
Robert Nilsson
Date:
2025-06-18 10:15:03 UTC
Severity:
normal
#1107973#5
Date:
2025-06-18 08:16:10 UTC
From:
To:
When uploading a file using a HTML form with
enctype="multipart/form-data", w3m sends the file in tiny pieces,
which makes the upload take a very long time.  The following
steps can show that this is happening:

First, make a C program that will act as a server and print the
received packet sizes:

File: srv.c :

    #include <netinet/ip.h>
    #include <sys/socket.h>
    #include <stdio.h>
    #include <unistd.h>
    int main() {
        int lsock = socket(AF_INET, SOCK_STREAM, 0);
        if (lsock < 0) {
            perror("socket");
            goto err;
        }
        int optval = 1;
        setsockopt(lsock, SOL_SOCKET, SO_REUSEPORT, &optval, sizeof optval);
        struct sockaddr_in addr = {AF_INET, 0x1111};
        if (bind(lsock, (const struct sockaddr*)&addr, sizeof addr) < 0) {
            perror("bind");
            goto err;
        }
        if (listen(lsock, 1) < 0) {
            perror("listen");
            goto err;
        }
        int csock = accept(lsock, 0, 0);
        if (csock < 0) {
            perror("csock");
            goto err;
        }
        ssize_t total = 0;
        for (;;) {
            char buf[64 << 10];
            ssize_t l = read(csock, buf, sizeof buf);
            if (l <= 0) {
                break;
            }
            total += l;
            fprintf(stderr, "Got %ld bytes, total %ld\n", l, total);
        }
    err:
        close(lsock);
    }

Then complie and run it in a terminal:

    cc -o srv srv.c && ./srv

In another terminal, make a HTML form and open it in w3m:

    printf '<form action="http://localhost:4369" method="POST" enctype="multipart/form-data"><input type="file" name="f"><input type="submit"></form>\n' > form.html
    w3m form.html

Now use the form to upload a pretty large file and you will
notice that it takes a long time and the server will print that
it receives the file in tiny packets.  Try the same form with
another browser and compare the results.

#1107973#10
Date:
2025-06-18 10:05:15 UTC
From:
To:
Thanks for the detailed report!

The following patch will speed things up, but is only a quick shot to
proof which part of the code is the problem. It is totally untested
expect for speed. It will take me some time to test it properly. When
done, I will commit it to my fork[0].

Another problem I saw while reproducing is that w3m will copy the
file that will be uploaded to a temporary file first. This should not be
necessary and might lead to problems on systems with little disk space.

[0]: https://w3m-pager.org
--- url.c.orig 2025-06-18 11:43:15.357506834 +0200 +++ url.c 2025-06-18 11:51:03.137277176 +0200 @@ -506,16 +506,14 @@ write_from_file(int sock, char *file) { FILE *fd; - int c; - char buf[1]; - fd = fopen(file, "r"); - if (fd != NULL) { - while ((c = fgetc(fd)) != EOF) { - buf[0] = c; - write(sock, buf, 1); - } - fclose(fd); - } + char buf[1024]; + size_t n; + + if (!(fd = fopen(file, "r"))) return; + + while ((n = fread(buf, 1, 1024, fd))) + write(sock, buf, n); + fclose(fd); } ParsedURL *
#1107973#15
Date:
2025-06-18 10:05:15 UTC
From:
To:
Thanks for the detailed report!

The following patch will speed things up, but is only a quick shot to
proof which part of the code is the problem. It is totally untested
expect for speed. It will take me some time to test it properly. When
done, I will commit it to my fork[0].

Another problem I saw while reproducing is that w3m will copy the
file that will be uploaded to a temporary file first. This should not be
necessary and might lead to problems on systems with little disk space.

[0]: https://w3m-pager.org
--- url.c.orig 2025-06-18 11:43:15.357506834 +0200 +++ url.c 2025-06-18 11:51:03.137277176 +0200 @@ -506,16 +506,14 @@ write_from_file(int sock, char *file) { FILE *fd; - int c; - char buf[1]; - fd = fopen(file, "r"); - if (fd != NULL) { - while ((c = fgetc(fd)) != EOF) { - buf[0] = c; - write(sock, buf, 1); - } - fclose(fd); - } + char buf[1024]; + size_t n; + + if (!(fd = fopen(file, "r"))) return; + + while ((n = fread(buf, 1, 1024, fd))) + write(sock, buf, n); + fclose(fd); } ParsedURL *