Hacker News new | ask | show | jobs
by tom_ 46 days ago
The standard says that fread calls fgetc multiple times for each object:

> For each object, size calls are made to the fgetc function and the results stored, in the order read, in an array of unsigned char exactly overlaying the object

(wording unchanged since C99)

If the file is unbuffered, depending on how the implementation handles buffering, and how it interprets the standard, then perhaps it does end up hitting a path where there's 1 ReadFile call per byte...

I don't know how most implementations get around this. Presumably it's valid to interpret "calls are made" as "behaving as if calls are made", meaning fread can copy data out of the FILE's buffer directly, or make calls directly to whatever routine fgetc defers to, rather than calling fgetc N times literally. Looks like glibc's fread does this.

2 comments

> The standard says that fread calls fgetc multiple times for each object:

>> For each object, size calls are made to the fgetc function and the results stored, in the order read, in an array of unsigned char exactly overlaying the object

Aha! That phrase led me to https://man7.org/linux/man-pages/man3/fread.3p.html. I consulted https://man7.org/linux/man-pages/man3/fread.3.html and https://man.openbsd.org/fread.3. Neither mentions that.

Now, I checked https://cplusplus.com/reference/cstdio/fread/. It doesn’t mention it, either.

⇒ this appears to be POSIX-specific.

Finally, if somebody implements fread as “For each object, size calls are made to the fgetc function”, it doesn’t matter whether you ask for 1 object of size 65,536 or 65,536 objects of size 1; both would call fgetc 65,536 times.

Shouldn't you be using cppreference.com instead of cplusplus.com? Because the former [0] actually has this language:

    Reads up to "count" objects into the array "buffer" from the given input stream "stream"
    as if by calling fgetc "size" times for each object, and storing the results, in the order
    obtained, into the successive positions of buffer, which is reinterpreted as an array of
    "unsigned char".
This whole fread/fwrite's interface is hailing from the time where some OSes used record-based filesystems and were literally unable to read/write less than a record at a time.

[0] https://en.cppreference.com/c/io/fread

Maybe, but that text says “as if” and even “If an error occurs, the resulting value of the file position indicator for the stream is indeterminate. If a partial element is read, its value is indeterminate.” so it doesn’t in any way require implementations to implement it by actually reading one byte at a time.

That description also fits what I saw in the implementations I inspected, both of which simply try to read size × count bytes.

By the way, behold the original (from UNIX Version 7) stdio package [0]:

    fread(ptr, size, count, iop)
    unsigned size, count;
    register char *ptr;
    register FILE *iop;
    {
        register c;
        unsigned ndone, s;
    
        ndone = 0;
        if (size)
        for (; ndone<count; ndone++) {
            s = size;
            do {
                if ((c = getc(iop)) >= 0)
                    *ptr++ = c;
                else
                    return(ndone);
            } while (--s);
        }
        return(ndone);
    }
Thankfully, the definition of getc() [1] is indeed

    #define getc(p)  (--(p)->_cnt>=0? *(p)->_ptr++&0377:_filbuf(p))
Interestingly enough, because there is no explicit multiplication, this loop properly works on systems with e.g. 16-bit unsigned int but 32-bit pointers (and overflows just the same on systems where ints and pointers are the same size).

[0] https://github.com/v7unix/v7unix/blob/master/v7/usr/src/libc...

[1] https://github.com/v7unix/v7unix/blob/master/v7/usr/include/...

I think it’s pretty rare for files to be unbuffered like that. AFAIK it’s mostly stderr that ends up unbuffered, at least on Unix-like systems.
You can call setbuf(fp,NULL) after opening, and now the stream is unbuffered. What this means is apparently implementation-dependent.

As to why you'd do that? - well, who knows the exact circumstances in this case. Perhaps this was faster in some meaningful case that was relevant to some other project (and then maybe the fread doesn't call fgetc after all!). I'm just speculating. Well-reused code often ends up with stuff that needs rethinking, that, even if noticed, nobody has the time or inclination to attempt to fix.