Ported rxi/lite text editor to odin for fun and learning Odin

Showcase
1

So I have been hearing about Odin and Zig for quite some time but never really got to try them out. A few days ago I had some free time so I decided to check out both the languages. When trying out something new, if I don’t have something “realistic” to work on I never really understand that thing so I used the rxi’s github.com/rxi/lite text editor as an exercise to learn some Odin.

Code:

I started with Zig but gave up after a while it got annoying. Odin was much more pleasant to work with in many ways. To name a few:

  • Good out of the box tooling that just works
  • Builtin sanitizers! odin run . -debug -sanitize:address and thats it!
  • Docs on one page which cover the basics and are easy to search in the browser
  • Context system with logging and allocators
  • Very greppable.
  • Vendor library with useful stuff (its mostly graphics only atm though)
  • Somewhat simple generics, no template/compile time shenanigans support. Coming from a C/C++ land I am tired of seeing people abuse these features and making life harder (and slower) for me.
  • GDB worked great! prints and other things worked with 0 extra work

The port went smoothly, I did have some trouble understanding allocators (there are 3 different versions of Arena in the core lib!) and other apis.
In my experience, Odin has pretty much all of C’s stuff if you really wanna reach that far down e.g., pointer arithmetic and unsafe casts etc.

Some of the issues that I had:

  1. Linking is somewhat inflexible, I couldn’t find a way to tell Odin to use a specific version of SDL for e.g. I guess I could modify the “vendor” library or copy it and then import the lib I wanted though.
  2. https://pkg.odin-lang.org/ can be very annoying to use. For e.g, I search for os.read_dir from the main page and I jump to the function. Later I come back to the browser to search for something in core:mem, I can’t. I first need to take my browser back to https://pkg.odin-lang.org/ or the Core library. There should be some way to search globally from anywhere.
  3. context.allocator becomes nil if you call a proc “c” function. Since lite is basically written in lua there is a lot of Odin → lua ← C stuff going on. So if lua calls into a proc "c" function in Odin, it doesn’t have context.allocator for some reason. I worked around it atm by making context.allocator global during initialization. But even then, some allocations don’t get freed no matter how hard I tried, perhaps I am missing something basic
  4. multiple variants of same functions in different parts of the library e.g., allocators

Couple of mildly annoying things:

  • types are optional e.g., x := 1 is valid and okay. But when types are missing in complex declarations its a bit annoying.
  • I accidentally used := to assign a a value to a global variable. Took me a while to realize that it was creating a local variable and my global was left unintialized. A warning would’ve been nice

Also, would be great if Odin is hosted on compiler-explorer .com. Something I plan to look at in the coming days :slight_smile:

Screenshot:

image
image1669×1024 131 KB

8 Likes
2

For #3 you could use context = runtime.default_context(). Making the allocator global is a bit iffy, AFAIK not all of the implementations are thread safe, but default allocator is fine. You might want to look into mem.Tracking_Allocator, it very handy to hunt down hard to spot leaks.

Regarding the global shadowing, compile with -vet-shadowing, afaik should catch it.

Otherwise, congrats on your first project! :slight_smile:

1 Like
3

I clarified in this post what the differences are between the arena types. I’ve had the thought that maybe the .Buffer variant of virtual.Arena is unneeded when we have mem.Arena, and I’ve shared your confusion at one point about this.

This would be a nice addition. I’m unfamiliar with how the pkg website works, but a workaround right now is to keep the search page open and open new tabs from the search links listed.

Only the odin calling convention carries the implicit context pointer with every call, so this is by design. It’s not so much that it becomes nil as much as that it’s not passed to the procedure in the first place. Every Odin odin procedure can be thought of as having an extra pointer to a runtime.Context structure.

The way you handle this for procedures with c and other contextless calling conventions is to provide it either in a global variable or along with a userdata pointer that the procedure can be made aware of.

You can also make a mem.Tracking_Allocator and hook your allocator up to that to keep track of memory leaks, which should pinpoint where they’re coming from (with some exceptions).

You can do x : Type = 1 or x := Type(1). Were you aware of this, or does that not cover your need here? The := contains an implicit declaration of the type which can be made explicit between the colon and equals sign.

As @zen3ger has pointed out, there’s the -vet-shadowing compile-time option, as well as a whole suite of vetting options that you can see with odin build --help. In the same family of options, there is also -strict-style.

The default context allocator, which uses libc malloc on *NIX-likes and HeapAlloc on Windows, should be thread-safe, but refer to your system’s manual if you’re on a *NIX system. I’m unfamiliar with the orca platform, so I can’t speak to that. That’s the one oddball in the base:runtime/heap_allocator_*.odin set of files.

1 Like
4

All the default allocators are supposed to be thread safe and I am pretty sure that’s the case.

5

I checked the C standard. It seems as if they’re implying that malloc and friends are defined to be thread-safe since C11.

C17:

7.22.3 Memory Management Functions

  1. For purposes of determining the existence of a data race, memory allocation functions behave as though they accessed only memory locations accessible through their arguments and not other static duration storage. These functions may, however, visibly modify the storage that they allocate or deallocate. Calls to these functions that allocate or deallocate a particular region of memory shall occur in a single total order, and each such deallocation call shall synchronize with the next allocation (if any) in this order.

C11:

  1. For purposes of determining the existence of a data race, memory allocation functions behave as though they accessed only memory locations accessible through their arguments and not other static duration storage. These functions may, however, visibly modify the storage that they allocate or deallocate. A call to free or realloc that deallocates a region p of memory synchronizes with any allocation call that allocates all or part of the region p. This synchronization occurs after any access of p by the deallocating function, and before any such access by the allocating function.

I’m interpreting the “behave as though they accessed only memory locations accessible through their arguments and not other static duration storage” as the thread-safe implication part. Since malloc only takes a size argument, it is defined to behave as if it does not access some other memory (which it would have to anyway, to get the memory from somewhere), therefore it is not going to race with another thread calling malloc.

It’s not as clear as “yes, they’re thread-safe,” but this seems good enough.

6

Indeed and I mean the allocator is nil after doing this. Without this one can’t even call a procedure with “odin” calling convention. Example a call from proc "c"

which goes to

I am using both mem.Tracking_Allocator and -sanitizer:address to catch all issues, but some are still there.

Regarding the global shadowing, compile with -vet-shadowing, afaik should catch it.

Nice! Didn’t know that, will try it out.

7

Thanks a lot for that, I did read it when I was porting and it helped clarify things :slight_smile:

I mean stuff like: x := myFunctionThatReturnsSth(). Now the type is invisible and makes the code a bit harder to read.

Okay, good to know this, will definitely make it easier to write code next time. But unless I missed it, I dont think this is explicitly documented anywhere. Also, IIRC context.temp_allocator is available i.e., its not nil in a proc "c". So I guess the behavior is different for each context field. However, despite the allocator being nil, the allocations work (in odin land) so there is some fallback in place which complicates things a bit. This is imo very important to be documented in detail as context is key to using Odin correctly. This kind of makes managing memory a bit more trickier than C.

Atm, Some allocations done with new($T, alloc) and make([], alloc) refuse to get freed. I will look a bit deeper, but either this is a bug in my code or odin or there is something else at play which I am not seeing. Was a bug on my side

Also: -vet is great!

8

I second that point about pkg documentation missing a global search. I often run into it too.

9

x : Widget = getSomething() is entirely valid and could be an acceptable way in a particular codebase to declare certain variables that may not have an easily inferred type based on the name of the returning proc. Though, it helps if the type of the return value can be inferred from the procedure’s name.

If you make the type explicit, it can help catch bugs later on if the returning type changes to something different when you didn’t expect it to at some point but the underlying code continues to compile.

Odin Overview - Calling Conventions

That would not make sense with the calling convention as defined. For example, the following does not compile for me.

hellope :: proc "c" () -> ^int {
	return new(int, context.temp_allocator)
}

This is the error, as expected:

Error: 'context' has not been defined within this scope
	return new(int, context.temp_allocator)
	                ^~~~~~^
10

I didn’t mean the calling conventions. (Quoted the wrong part from your earlier reply). Yes, I understand that the context needs to be explicitly “brought in” before use. I meant that after you declare the context

context = runtime.default_context()
assert(context.temp_allocator.data != nil) // ok
assert(context.allocator.data != nil) // fails
// what about other fields such as logging, will they be what was explicitly set
return new(int) // works, which allocator did it use though?

its not documented what runtime.default_context() will be exactly when its “brought into a proc ‘c’”, which fields will be preserved, which fields will be cleared etc. So e.g., if I am using a tracking allocator as my default allocator, will the allocations in a proc “c” be done using that allocator or something else that was default.

The problem is that you can call odin call-conv procedures from a C call-conv procedure and back, and then its totally unclear what state context will be in.

11

To be entirely precise, there is no default allocator that you set. When you set context.allocator, you are setting the allocator field on the current context, and the context can change with different scopes (that is to say, it persists its state on a scope-level granularity). When context is set or changed, even if you’re changing sub-fields, that change persists down through the call stack (assuming all odin calling conventions) and deeper scopes.

package ctx_depth

import "core:fmt"

main :: proc() {
	context.user_index = 1
	fmt.println(context.user_index) // 1
	{
		context.user_index = 2
		fmt.println(context.user_index) // 2
	}
	fmt.println(context.user_index) // 1

	context.allocator = {
		nil,
		nil,
	}
	fmt.println(context.allocator) // nil, nil
	{
		context.allocator.data = rawptr(uintptr(3))
		fmt.println(context.allocator) // nil, 0x3
	}
	fmt.println(context.allocator) // nil, nil

}

runtime.default_context() returns a structure with default values, and it does not point to a global context or some value that is consistent with mutations to any other context. No fields are preserved or cleared; it simply gives you the default values that you would have had at main.

So to answer your question plainly: no, explicitly set loggers or any other values for that matter will not be there. They must be set anew, or you must use a global variable or userdata pointer with a context already set up and ready to go.

This may be the source of your memory leaks, if you had assumed mutating context.allocator in one place would translate to somewhere else from a call to default_context, because that is not the case.

Finally, new() always uses context.allocator unless otherwise specified. So if you have just brought in the context using runtime.default_context(), new is using the default heap allocator that is also present at main.

Hopefully this clears up some of the confusion.

1 Like
12

Indeed it was. Thanks a lot for your words, things are clearer now. Main take away for me is that I need to be extra careful when calling into C land or calling odin from C land

13

Wow! Great work, this is poggers :astonished:

1 Like