Odin and Erl_Interface/NIFs

Showcase
1

Let’s just appreciate how Odin can tap into a battle-tested library in less than 100 lines of code?

I am interested in Elixir for the past few years and I was looking at it again recently. Then I saw two very interesting things and decided to have a blast with it, and why not throw Odin in the mix just because.

Erl_Interface for C nodes

Did you know that Erlang ships with C libraries that let you “pretend” you are a BEAM VM, while running C code instead?

package odin_erl_node

import "core:c"
import "core:fmt"
import "core:os"

@(extra_linker_flags = "-L/path/to/my/erlang/installation/lib/erl_interface-5.6.1/lib")
foreign import ei "system:ei"

ei_x_buff :: struct {
	buff:   [^]c.char,
	buffsz: c.int,
	index:  c.int,
}

// linking to a few erl_interface functions
@(default_calling_convention = "c")
foreign ei {
	ei_init :: proc() -> c.int ---
	ei_x_new :: proc(x: ^ei_x_buff) -> c.int ---
	ei_x_free :: proc(x: ^ei_x_buff) -> c.int ---
	ei_x_encode_tuple_header :: proc(x: ^ei_x_buff, arity: c.int) -> c.int ---
	ei_x_encode_atom :: proc(x: ^ei_x_buff, s: cstring) -> c.int ---
	ei_x_encode_long :: proc(x: ^ei_x_buff, n: c.long) -> c.int ---
	ei_print_term :: proc(fp: rawptr, buf: [^]c.char, index: ^c.int) -> c.int ---
}

foreign _ {
	@(link_name = "__stdoutp")
	stdout: rawptr
}

main :: proc() {
	ei_init()

	buf: ei_x_buff
	ei_x_new(&buf)

	defer ei_x_free(&buf)

	ei_x_encode_tuple_header(&buf, 2)
	ei_x_encode_atom(&buf, "tobbe")
	ei_x_encode_long(&buf, 3886)

	idx: c.int = 0
	ei_print_term(stdout, buf.buff, &idx)
	fmt.println()
}

And you can build the above normally with: odin build -file. It just works. Which is insane, imo.

Granted, this small example isn’t very interesting, but erl_interface ships with functions that let you connect to Erlang (or Elixir) nodes and send Erlang terms to them. All it takes is a few lines of code.

Advantages of this: Odin code participates in the Erlang cluster without having the capability of destroying a VM.

But yeah, sometimes you DO want to destroy a VM. Erlang let’s you do that too, by defining a native function, dubbed “a NIF”. It runs inside the Erlang VM, so any segfaults crash the entire VM.

Odin NIFs

package nif

import "base:runtime"
import "core:c"
import "core:fmt"

ErlNifEnv :: rawptr
ERL_NIF_TERM :: rawptr

ErlNifFunc :: struct {
	name:  cstring,
	arity: c.uint,
	fptr:  proc "c" (env: ErlNifEnv, argc: c.int, argv: [^]ERL_NIF_TERM) -> ERL_NIF_TERM,
	flags: c.uint,
}

ErlNifEntry :: struct {
	major:         c.int,
	minor:         c.int,
	name:          cstring,
	num_of_funcs:  c.int,
	funcs:         [^]ErlNifFunc,
	load:          rawptr,
	reload:        rawptr,
	upgrade:       rawptr,
	unload:        rawptr,
	vm_variant:    cstring,
	options:       c.uint,
	sizeof_struct: c.size_t,
}

// this is actually linked by Erlang/Elixir when compiling the nif.
foreign _ {
	enif_make_int :: proc(env: ErlNifEnv, i: c.int) -> ERL_NIF_TERM ---
	enif_get_int :: proc(env: ErlNifEnv, term: ERL_NIF_TERM, ip: ^c.int) -> c.int ---
	enif_make_badarg :: proc(env: ErlNifEnv) -> ERL_NIF_TERM ---
}

foo_nif :: proc "c" (env: ErlNifEnv, argc: c.int, argv: [^]ERL_NIF_TERM) -> ERL_NIF_TERM {
	context = runtime.default_context()

	fmt.println("FROM FUCKING ODIN, BABY!")
	x: c.int

	if enif_get_int(env, argv[0], &x) == 0 {
		return enif_make_badarg(env)
	}

	ret := x * 2
	return enif_make_int(env, ret)
}

nif_funcs := [1]ErlNifFunc{{"foo", 1, foo_nif, 0}}

entry := ErlNifEntry {
	major         = 2,
	minor         = 17,
	name          = "Elixir.Complex6",
	num_of_funcs  = 1,
	funcs         = &nif_funcs[0],
	vm_variant    = "beam.vanilla",
	sizeof_struct = size_of(ErlNifEntry),
}

@(export)
nif_init :: proc "c" () -> ^ErlNifEntry {
	return &entry
}

We don’t actually link enif_ functions. This is done by Erlang (or Elixir) when compiling this NIF. So, we need to tell Odin that, when building:

odin build nif.odin -file -build-mode:shared -out:complex6.so \
-extra-linker-flags="-undefined dynamic_lookup"

This produces a shared library that can be used from Elixir (or Erlang) like this:

defmodule Complex6 do
  @on_load :load_nifs

  def load_nifs do
    path = ~c"./complex6" |> Path.expand() |> to_charlist()
    :erlang.load_nif(path, 0)
  end

  def foo(_x), do: :erlang.nif_error(:nif_not_loaded)
end

And from IEx:

Interactive Elixir (1.19.4) - press Ctrl+C to exit (type h() ENTER for help)
iex(1)> c("complex6.ex")
[Complex6]
iex(2)> Complex6.foo(20)
FROM FUCKING ODIN, BABY!
                        40

NIFs run in the same process as the VM, so there’s no network boundary. Extremely fast but extremely risky: one segfault and the entire thing burns to the ground.

In fact, erl_nif even exposes enif_alloc to allocate memory using the BEAM’s memory allocators. So you could even build your own mem.Allocator and use the context system to manage Odin memory in the same heap as the BEAM VM, which sometimes might speed up allocations, if the stars align. Memory allocated this way also gets reported by the BEAM VM, when you inspect it. That is extremely sick. I might show an example of how to do this later. (This is NOT GC’d by BEAM though, you still need to free resources!!)

Kinda crazy how you can do this in just a few lines of code. Odin really makes C integration as easy as it gets.

Where to go from here?

IDK. Open to ideas.

Rustler is a Elixir library that provides Rust NIFs. You can just use Odin and maybe some helper functions to do the same, and it compiles a lot faster. You also get zero guarantees, unlike Rust. But maybe you can share read-only memory between processes with (even less) overhead than ETS, by having a NIF managing that read-only memory? Something like that came to my mind recently.

Discord uses Rustler for some data structures, to squeeze more performance where it matters. Immutability is nice but copying memory all over the place is not so nice all the time.

Odin really shines in graphics programming. A game client written in Odin/Sokol (or Raylib) that connects to a game server written in Elixir?

2 Likes