Union dependency graph and modularity

[c0ffee]

Hey guys.

I think unions are great and tick almost all boxes of what interfaces do.
I’ve been using them since beginning writing in odin.
Now that my software is growing I’m separating it into packages, and this exposed union inconvenience that makes it not so attractive.

Unlike interfaces union must import all types it contains. This can create cyclic dependencies and dictates package layout. I had to rule out unions from design because of this.

Having compile time interfaces that do same that union does but doesn’t depend on child types would allow moving it into core and remove package dependency inconveniences.

[leecommamichael]

Could you post some real code? We might be able to get the gist with just proc and type names.

[c0ffee]

Here core has to import more concrete Cat and Dog to implement more abstract Animal. This makes a and b packages unable to import core package.

Same problem with proc_groups (I don’t want to focus on it here though).

In other languages abstract interfaces do not import concrete inheritors, this allows packages like a and b to import core.

It’s explicit closed set vs implicit closed set. In theory implicit closed set still might enumerate all types at compile time without the need to import all concrete types, so it would be more flexible to use when separating package into smaller packages.

package core

import "../a"
import "../b"

Animal :: union {
	a.Cat,
	b.Dog,
}

package a

import "core:fmt"

Cat :: struct {
	name:  string,
	lives: int,
}

speak :: proc(c: Cat) -> string {
	return fmt.tprintf("%s says meow (%d lives left)", c.name, c.lives)
}

package b

import "core:fmt"

Dog :: struct {
	name: string,
}

speak :: proc(d: Dog) -> string {
	return fmt.tprintf("%s says woof", d.name)
}

package main

import "core:fmt"

import "a"
import "b"
import "core"

speak_animal :: proc(animal: core.Animal) -> string {
	switch v in animal {
	case a.Cat:
		return a.speak(v)
	case b.Dog:
		return b.speak(v)
	}
	return ""
}

speak :: proc {
	a.speak,
	b.speak,
	speak_animal,
}

main :: proc() {
	cat := a.Cat{name = "Tom", lives = 9}
	dog := b.Dog{name = "Rex"}

	animals := []core.Animal{cat, dog}
	for animal in animals {
		fmt.println(speak(animal))
	}
}

Tom says meow (9 lives left)
Rex says woof

[leecommamichael]

Packages are units of distribution or modularity, not organization. In the program you posted, I’d just have “layers” or “prefixes” to organize the files. That way things can stay in one package.

• dog_api.odin
• dog_impl.odin
• cat_api.odin
• cat_impl.odin

Don’t forget Odin has subtype polymorphism.

The moment Dog or Cat are distributed to other codebases as a dependency, then you’d do what you have posted, and it would be a fine approach.

[c0ffee]

Thanks — that’s fair for a single codebase, and I’ve been using prefixes exactly that way. Two things I’d push back on.

On subtype polymorphism: I tried it and it does give the dependency direction I want — a and b import core. But it’s a different tool: pointer-based, no value semantics, and no exhaustiveness. If I add Fish I get no compile error at the switch sites that should have handled it, which is one of the features union provides. And since ^Animal is a pointer, I lose the flat []Animal layout — variants no longer live inline in one contiguous array, so it becomes []^Animal with a separate allocation and lifetime per element. Please correct if I got memory layout wrong.

On your last point: that once Cat and Dog are distributed, core importing them is fine. That’s the case I’m actually worried about, and I don’t think it holds up. In that layout core depends on every variant, so nothing a or b depends on can live in core without a cycle, it has to go into another package below all three. Also each new variant is an edit to the shared package. That’s the part that doesn’t scale well.

Other languages define the abstraction as a leaf and have concrete types depend on it, so the dependency graph grows the other way. In Java or C#, Animal is an interface that imports nothing, Cat and Dog implement it and can be added without touching it.

What I’d want is: core declares Animal, a and b import core and register Cat and Dog as variants, core imports nothing. Subtypes import the parent, the compiler collects them, and switch is still exhaustively checked. Base types live at the base, concrete types build on them, and dependencies always point downward.

This problem appeared after using union extensively and then trying to split module into individual units, I had to drop union to not overcomplicate dependencies. With inverted dependency graph splitting would be a breeze.

[CyMonk]

your technical vocabulary shows that you’ve deeply internalized object- oriented principles. as soon as you overcome this “indoctrination,” your problems will disappear on their own.

[tienery]

Or, in many ways, as a user of AI myself, that reply looks suspiciously generated by AI, like Claude. Because Claude speaks like that in many places, unless the overuse of AI has led to a change in personality.

To add to the conversation, though, you don’t have to assign variants to the Animal struct and you don’t have to declare unions.

A much simpler approach:

Cat :: struct {
    using animal : core.Animal,

}

Dog :: struct {
    using animal : core.Animal,
}

And in your core package:

Animal :: struct {
    kind : int, // match and cast `data` accordingly
    data : rawptr, 
}

What you will find is that you will work with pointers a lot in Odin. In Java and C#, all their classes are just pointers to an underlying block of memory and everything is heap allocated, so you get no memory management, almost no control over allocations (with some exceptions), etc.

You get trade offs in each domain:

• If you don’t care about memory and you want the path of least resistance: stick with C#/Java.
• If you instead care about memory and wish to learn a new language like Odin, you will need to accept some things that simply are not available in Odin as they are in C# or Java. Just think of it as a new toy. It can be frustrating to work with coming from an OOP background initially, but once you get used to the initial hurdles, I can almost guarantee you will never want to return to OOP.

[c0ffee]

your technical vocabulary shows that you’ve deeply internalized object- oriented principles. as soon as you overcome this “indoctrination,” your problems will disappear on their own.

Yes, Odin is first procedural language I’ve taken seriously. Many things to re-understand.

Conversation drifts towards use of pointers.
Yes, Odin is a fine new toy and has its own quirks.
I really like union because of its compact memory layout and other features, after using it for a while I found one inconvenience with it, which makes much harder to scale application.

Take it as feedback about language feature. I’ll rephrase it in a bug format, only because I find it highly informative and it may uncover mistakes in my thinking.

• union is used in core package

Actual:

• hard to scale because of inverted dependency order(abstract knows concrete)
• core imports all its variants’ packages (base knows its types)
• variants don’t know about its “base” (sometimes an advantage)
• for each variant dependencies now should be below core and this variant package

Expected:

• easy to scale. Dependency order: concrete knows abstract
• variant imports core (concrete knows its base)
• core knows nothing about its variants

Is there a historical reason or real limitation behind this?
Odin compiles whole program and has every package in front of it. Is there something that rules out collecting variants?

[CyMonk]

@c0ffee

Yes, Odin is first procedural language I’ve taken seriously.

no, you don’t- quite the opposite. that’s why i wrote my previous post.

you are trying to mimic OOP, e.g. you misuse packages to represent a class hierarchy, unions to emulate inheritance and procedure groups as a substitute for virtual methods.

the presence of cyclic dependencies is not a sign of gaps or flaws in odin’s design, but rather an indication that you are approaching the problem with the wrong strategy/ mindset.

[SaiMoen]

There are some problems that such a union might have, like what type should have what tag number (i.e. what order are the variants), though this could be made well-defined.

Another problem is the implicit nature that this feature would have, since that leads to the usual OOP issue where the implementations are all over the place and you need to rely on an LSP to find them exhaustively (assuming it’s not public). I suppose the explicit closed set version is not completely immune to this either, but at least the identifiers are listed in one place.

The real problem I think is just the fact that packages have a very strict one-way dependency flow (by design), since they are supposed to be fully self-contained, and therefore a package can’t/shouldn’t make assumptions about what imports it. If a package does know what imports it, then there is (effectively) a cycle in the dependency graph, which indicates packages in that cycle are really the same package. If your core package truly didn’t know anything about its variants, it wouldn’t be possible to make a blob+tag style union like Odin has. Rather, it would require a vtable or equivalent, and at that point you kind of get into more proper ‘Objects’.

[c0ffee]

Thanks for replies, guys.

Another problem is the implicit nature that this feature would have, since that leads to the usual OOP issue where the implementations are all over the place and you need to rely on an LSP to find them exhaustively (assuming it’s not public). I suppose the explicit closed set version is not completely immune to this either, but at least the identifiers are listed in one place.

I also don’t like to dig through all sources to find all concrete types, union with all its types looked really good until I had to split it into packages.

The real problem I think is just the fact that packages have a very strict one-way dependency flow (by design), since they are supposed to be fully self-contained, and therefore a package can’t/shouldn’t make assumptions about what imports it.

Yes I agree, keeping one-way dependency flow is a good strategy for designing a software.

If a package does know what imports it, then there is (effectively) a cycle in the dependency graph, which indicates packages in that cycle are really the same package.

Yes, also agree, package should not know what imports it.
Here I think the dependency flow gets inverted. Usually one-way dependency to me looks like “concrete knows abstract”, this builds a graph which can then be reused at a higher level by more concrete things.

So when union has to import its types, flow reverses to “abstract knows concrete” this fights the design graph. Without this “reverse” nuance — “packages in that cycle are really the same package” goes away, and packages stop being that monolithic and become more flexible.

If your core package truly didn’t know anything about its variants, it wouldn’t be possible to make a blob+tag style union like Odin has. Rather, it would require a vtable or equivalent, and at that point you kind of get into more proper ‘Objects’.

A vtable is needed when the variant set is open at compile time, e.g. across separately-compiled units.

Here, if I understand correctly, the compiler still sees every package in the build, so it can compute max(size_of(variant)) and assign tags exactly like today. Blob+tag is preserved, the only difference is whether the list lives in core.odin or is collected from the packages.

[TheNeighborlyBoy]

This feels like more of a project structure/organization issue (like I believe @leecommamichael suggested) than a language limitation to me. Your example project appears to be structured roughly like the following:

project_directory
\-> main.odin
\-> core
  \-> core.odin
\-> a
  \-> cat.odin
\-> b
  \-> dog.odin

As core, a, and b are packages intertwined in concept and functionality (and presumably authored by you), I would recommend merging all three into a single package. Restructuring the project under a unified animals package will eliminate most, if not all, of your dependency issues.

project_directory
\-> main.odin
\-> animals
  \-> cat.odin
  \-> dog.odin
  \-> core.odin

This new project structure would provide a similar separation of files/types while being a cohesive logical unit for use as a package by others. You would have resulting source changes that could look something like the following:

// file: /main.odin
package main 

import animal "animals"

main :: proc() {
	cat := animal.Cat{name = "Tom", lives = 9}
	dog := animal.Dog{name = "Rex"}

	animals := []animal.Animal{cat, dog}
	for a in animals {
		fmt.println(animal.speak(a))
	}
}

// file: /animals/core.odin
package animals

Animal :: union {
	Cat,
	Dog,
}

speak :: proc(animal: Animal) -> string {
	switch type in animal {
	case Cat: return speak_string_for_cat(type)
	case Dog: return speak_string_for_dog(type)
	}
}

// file: /animals/dog.odin
package animals

import "core:fmt"

Dog :: struct {
	name: string,
}

speak_string_for_dog :: proc(d: Dog) -> string {
	return fmt.tprintf("%s says woof", d.name)
}

// file: /animals/cat.odin
package animals

import "core:fmt"

Cat :: struct {
	name:  string,
	lives: int,
}

speak_string_for_cat :: proc(c: Cat) -> string {
	return fmt.tprintf("%s says meow (%d lives left)", c.name, c.lives)
}

[TheNeighborlyBoy]

Thinking about it a bit more, if core is intended to be a wrapper for combining two external packages that you have not authored, and/or are unable to combine, then that would require a different solution.

The current flow of your dependencies is as follows:

If this is to be a wrapper, then you would want something more like this:

Assuming that you will want your users to interact directly with the core package an not with either a or b , you could expose the necessary types through core.

// file: /core/core.odin
package core

import "../a"
import "../b"

Cat :: a.Cat
Dog :: b.Dog

Animal :: union {
	Cat,
	Dog,
}

speak :: proc(animal: Animal) -> string {
	switch type in animal {
	case Cat: return a.speak(type)
	case Dog: return b.speak(type)
	}
}

// file: main.odin
package main 

import animal "core"

main :: proc() {
	cat := animal.Cat{name = "Tom", lives = 9}
	dog := animal.Dog{name = "Rex"}

	animals := []animal.Animal{cat, dog}
	for a in animals {
		fmt.println(animal.speak(a))
	}
}

[c0ffee]

Yes, I think my mistake was assuming that Animal union was lower level abstraction and putting it into core.
Treating it as higher level abstraction — putting Animal above Cat and Dog should be manageable. Will keep core lowest level for shared code.

Will take time to test on real world examples to check how it goes.