Nub need advice on reading binary files

Hi,

I’m a totally new Odin user, actually I never used a low level language before (e.g. C or C++). I’m coming from F# and Rust.

I want to parse the tags of a Flac file, for a tool I’m migrating from F#. I’m more or less trying to convert parts of the Rust metaflac crate to Odin.

I’m wondering what’s the best way to read bytes from a file. Reading the entire file is certainly not an option as these files could be huge, so I’ll probably want to use bufio.Reader or something similar. I do need to read a specific number of bytes, during the parsing of the metadata, and I found there’s io.read_full function that does that, but then I need to create bufio.Reader and convert it to Stream? I’m not sure what’s an idiomatic way to do this…

I’m really trying to avoid AI as I’m trying to learn the language, not vibe code it :slightly_smiling_face: .

Can anyone please point me to a similar example so I’ll see how it’s done?

Thanks

So, after searching and reading answers/code/etc, I think that I need to create a bufio.Reader (to avoid repeated file access when reading small chunks of the file), convert it to io.Stream and do all my actual reading using the stream (read_full). Does this sound correct?

I wouldn’t bother with “what’s idiomatic” or “best” when learning, just get it to work first. Doing it the “best” way also entirely depends on what’s the end goal of your program.

Parsing the metadata blocks correctly should matter more than how you read bytes of the disk, so personally I’d just read_entire_file() until I’m sure the logic is correct then screw around optimizing I/O or memory usage.

2 Likes

hehe, thanks, but reading entire file could take more than 1 or 2 Gigabytes (on high res long tracks). The logic behind parsing the metadata is not complicated, and I’ve already did my homework reading the format docs, RFC, and sample parsers code.

The thing is that actually starting a project (even if you read the book/docs/etc) take some figuring out, but after searching a little bit yesterday, I started today and it seems to be ok.

BTW, I’m certainly not looking for the best solution, just one without obvious errors :slightly_smiling_face: .

Thanks

1 Like

Point is, just start however seems obvious first, or “do the dumb thing first”.

Like I said, how you read bytes of the disk matter the least. You could just mmap the whole file, never commit the memory, and parse the metadata. In the end the program would only use however many pages the metadata spans.

Or just read 4 bytes for the block header, figure out the size and read size + 4 for the data plus next header, etc… More syscalls but can be done with near zero heap allocation.

(Also, 1-2 Gigs is nothing unless you are on a potato. That’s like what, 3 chrome tabs?)

2 Likes

Thanks, for now I’ve already started something (very basic buffered stream reading). In a couple of hours I’ll find out if I did it correctly or I need to rethink :slightly_smiling_face: .

I do agree with the general concept you suggested. I’m certainly currently skipping multiple steps just to get started.

Thanks

There is an article about the buffered approach but I think the example is outdated:

https://odin-lang.org/news/read-a-file-line-by-line/

See also this thread:

Thanks I already knew about this post. I was looking for something involving io.Stream to see how it’s used (e.g. with read_full).