How can I read a File chunk by chunk in Typescript (client-side)

I am using a file upload dialog and need to parse that file and send the parsed data to a server part by part.

The file can be really large (100s of MBs) and as such, reading the file in one go is not desired.

I would like to re-use the same buffer for each subsequent read in order to not having to reallocate and garbage collect the buffer each iteration, especially when the buffer size is small.

In C, such a thing would be relatively easy:

void read_chunks(FILE *file) {
uint8_t buf[CHUNK_SIZE];
size_t read_size;

while (read_size = fread(buffer, 1, sizeof(buffer), file)) > 0) {
handle_data(buf, read_size)
}
}


Now, in Typescript, I am trying to do the same, and the only applicable code that I could find was using a ReadableStreamBYOBReader using an ArrayBuffer as backup. But when I try to do this, I get an error:

async readChunks(file: File) {
let buffer = new ArrayBuffer(CHUNK_SIZE)
let reader = file.stream().getReader({mode: 'byob'})

while (true) {
const {done, value} = await reader.read(new Uint8Array(buffer, 0, buffer.byteLength))
if (done) break
if (value) handleData(value)
}
}


This gives the error TypeError: attempting to access detached ArrayBuffer on the second time I try to call reader.read(). It appears I have to pass a whole new buffer on each invocation, which is very inefficient.

Now, I don't understand what's wrong here since according to the docs: (https://developer.mozilla.org/en-US/docs/Web/API/ReadableStreamBYOBReader/read)

If a chunk of data is supplied, the value property will contain a new view. This will be a view over the same buffer/backing memory (and of the same type) as the original view passed to the read() method, now populated with the new chunk of data. Note that once the promise fulfills, the original view passed to the method will be detached and no longer usable.
​

Now this would imply to me that the view (the Uint8Array) would be detached (whatever that is supposed to mean) but the buffer would be reusable. But the error seems to imply that the buffer itself is detached.

How can I properly read the file chunk by chunk without having to allocate a new buffer or read the whole file in one go?

Alternative suggestions using third-party libraries instead of the streams API are welcome if this is not easily done using that API as it means I do not have to reinvent the wheel.

Continue reading...