Binding Odin to C -- Structs from Header files

Help
1

Hi, Odin community! I’m working on bindings for WiredTiger (MongoDB’s storage engine) and running into issues with segfaults and invalid arguments. I’d really appreciate some help debugging this.

Current Implementation

Here’s my current binding code:

package wiredtiger_odin

import "core:c"
import "core:fmt"
import "core:os"
import "core:strings"

struct__wt_session :: struct {
    connection: ^WT_CONNECTION,
    close: proc "c" (session: ^WT_SESSION, opts: cstring) -> c.int,
    open_cursor: proc "c" (
        session: ^WT_SESSION,
        uri: Maybe(cstring),
        to_dup: Maybe(^WT_CURSOR),
        config: cstring,
        cursorp: ^^WT_CURSOR,
    ) -> c.int,
    create: proc "c" (
        session: ^WT_SESSION,
        name: cstring,
        config: cstring,
    ) -> c.int,
   ....

@(link_prefix = "wiredtiger_")
foreign wiredtiger {
	open :: proc(home: cstring, event_handler: ^WT_EVENT_HANDLER, config: cstring, connection: ^^WT_CONNECTION) -> c.int ---
	strerror :: proc(error: c.int) -> cstring ---
 }
}

// Example usage in main:
main :: proc() {
    conn: ^WT_CONNECTION = nil
    session: ^WT_SESSION = nil
    cursor: ^WT_CURSOR = nil
    
    // Setup connection
    config_cstr := strings.clone_to_cstring("create,cache_size=100MB,async=(enabled=false)")
    defer delete(config_cstr)
    
    ret := open(home_cstr, nil, config_cstr, &conn)
    
    // Open session
    session_ret := conn.open_session(conn, nil, nil, &session)
    
    // Try to open cursor
    table_uri := strings.clone_to_cstring("table:test")
    defer delete(table_uri)
    
    cursor_ret := session.open_cursor(
        session,
        table_uri,
        nil,
        nil,
        &cursor,
    )
}

The Issue

When conn.open_session() is called, I get the error:

“WT_session → session.open_cursor should be passed either a URI or a cursor to duplicate, but not both”.

I’ve tried:

  1. Using Maybe types for the nullable parameters in the struct definition
  2. Explicitly passing nil for unused parameters
  3. Various string handling approaches

Questions

  1. Am I handling the C function pointers correctly in the struct definitions?
  2. Is there a better way to handle nullable parameters in C bindings?

Relevant Information

  • WiredTiger version: 11.3.0
  • Odin version: Latest
  • OS: Ubuntu Linux running on WSL
  • The C header defines one of these parameters (*to_dup) using the WT_HANDLE_NULLABLE macro which looks like this-
#define	WT_HANDLE_NULLABLE(typename)	typename##_NULLABLE
2

Here’s the link to source - GitHub - iyifr/wiredtiger-odin: Wiredtiger-odin bindings

3 
open_session:      proc "c" (connection: ^WT_CONNECTION, event_handler: union {
	^WT_EVENT_HANDLER,
	rawptr,
}, config: union {
	^WT_EVENT_HANDLER,
	rawptr,
}, sessionp: ^^WT_SESSION) -> c.int,

The unions in the arguments will probably mess with things. Odin’s union is a tagged union and doesn’t directly correspond to a C type. I expect it’ll pass both a pointer and a tag, and I’d guess this will most likely offset the arguments and cause things to not match up with the arguments that they’re supposed to. Have you tried just ^WT_EVENT_HANDLER? Pointers can be nil. You can do Maybe(^WT_EVENT_HANDLER) if you want to explicitly say it can be nil, and in that particular case will be handled as just a plain pointer internally. But if you do a union of two pointer types it needs a tag to know which one it should be.

2 Likes
4

I already tried Maybe(^WT_EVENT_HANDLER) ^WT_EVENT_HANDLER during my initial attempt, but it didn’t work.

The tagged union was just an attempt to pass c.NULL (C’s null type ) to some of the arguments to see what would happen.

I will complete more of the bindings in case I’m missing a function or enum that’s used internally to make the duplicate cursor arg nullable…

5

Dug a little deeper into the actual WT library, and your bindings seem to be excluding a lot of things from the library’s structs. For example, in the Connection struct you start with async_flush and open_session procs, but in the actual header there are a number of other procs there.

It’s important to know that binding is very touchy. Odin doesn’t know anything about what the underlying C library is doing; the struct you define is what Odin uses to know how the memory is laid out, and it takes your word for it. It doesn’t know what layout the C library is using, so it can’t automatically match things up. So, given

struct__wt_connection :: struct {
	async_flush:       proc "c" (connection: ^WT_CONNECTION) -> c.int,
	open_session:      proc "c" (connection: ^WT_CONNECTION, event_handler: Maybe(^WT_EVENT_HANDLER), config: Maybe(^WT_EVENT_HANDLER), sessionp: ^^WT_SESSION) -> c.int,
	close:             proc "c" (connection: ^WT_CONNECTION, config: cstring) -> c.int,
	get_extension_api: proc "c" (connection: ^WT_CONNECTION) -> c.int,
	async_new_op:      proc "c" (
		connection: ^WT_CONNECTION,
		uri: cstring,
		config: cstring,
		async_op_ptr: ^^WT_ASYNC_OP,
	) -> c.int,
}

you’re saying that the WT_CONNECTION struct contains 5 pointers, stored in that order. Meanwhile, the C code has a lot of other things, and in a different order:

// removed a LOT of documentation comments, for brevity
struct __wt_connection {
    int __F(close)(WT_CONNECTION *connection, const char *config);
    int __F(debug_info)(WT_CONNECTION *connection, const char *config);
    int __F(reconfigure)(WT_CONNECTION *connection, const char *config);
    const char *__F(get_home)(WT_CONNECTION *connection);
    int __F(compile_configuration)(WT_CONNECTION *connection, const char *method,
        const char *str, const char **compiled);
    int __F(configure_method)(WT_CONNECTION *connection,
        const char *method, const char *uri,
        const char *config, const char *type, const char *check);
    int __F(is_new)(WT_CONNECTION *connection);
    int __F(open_session)(WT_CONNECTION *connection,
        WT_EVENT_HANDLER *event_handler, const char *config,
        WT_SESSION **sessionp);
    // ... more omitted for brevity
};

This is how C is storing things in memory, and you must inform Odin on how to interpret the memory correctly. You must match this layout; you can’t skip any, and you can’t reorder them. You don’t necessarily have to bind everything completely–a function pointer is just a pointer, so you can use rawptr or equivalent for any values you aren’t interested in. But Odin needs to know what’s there, at least to the extent that it can interpret the C library’s memory correctly. Skipping or re-ordering fields will result in Odin reading something from the wrong location, and calling the wrong function (or worse).

Odin doesn’t have the information to check that you have everything correct.

2 Likes
6

Thanks a lot for this!!

Especially the part about the bindings being in order.

The zig bindings worked in one-shot because it automatically generates the zig code based on the header file and so i’ll just translate all of that zig code to odin code and try again.