Has sqlite3 ever been considered as an addition to vendors?
5 Likes
They will never be added because as soon as we add one set of database bindings, people will want their favourite one, of which there are dozens, if not hundreds, of different SQL-like databases out there.
It’s not something I believe is a good idea for vendor.
5 Likes
I am not sure there are any viable alternatives to sqlite. Its pretty much the standard inprocess embeddable database and imo it makes a lot of sense in supporting it. Also sqlite is just one C file so its pretty straightforward to support.
There are a lot popular sql databases but none of them are embeddable or in process afaik.
4 Likes
one thing that maybe would fit in vendor might be the sql api specification that a database driver would need to conform to expose sql capabilities. Making sure to handle parametrized queries and such.
Then sqlite would be one of the drivers that lives in the wider odin ecosystem.
I’ve never been sure about those because it seems like it is abstracting something which shouldn’t be. Do you need a generic SQL interface when you know exactly what database your project will be using? It’s not like you can trivially swap between different databases and expect them to work without changing loads of things.
5 Likes
it is useful if you can trivially change between a local test (sqlite) database and the real database in production with no code changes
1 Like
Sure, but maybe this is me being dumb but why isn’t the local database the same kind as the production one?
I think a goal of an interface that makes multiple database vendors compatible is both out-of-scope [for this thread], and unwise. Let’s keep the discussion to SQLite only.
I do agree with @waqar144; SQLite is in a league of its own, quite different from other databases. It (and its format) are used in many places, and for many purposes, not just back-end services like most other database engines. It’s also often [used as] a configuration or serialization format as well as just a database. Not to mention that it is extremely mature and stable, too.
2 Likes
I think a goal of an interface that makes multiple database vendors compatible is both out-of-scope [for this thread] and unwise
+1
SQLite might be quite different to many other databases but that slippery slope is still there. As soon as I allow for just one, I will be inundated with requests for more, and saying “no” won’t be enough.
2 Likes
Ok, but in a way you can make the same argument for a lot of stuff in the vendor directory atm. Its not a big deal though, most things will have to live outside vendor. The reason I thougt it would be a good candidate for vendor was because its a very commonly used lib and would make the vendor a tiny bit more generic which currently is more or less just graphics stuff.
6 Likes
Coming probably a bit late to this discussion.
In golang there is an abstracted sql interface in the standard library and every non-helloworld project eventually ends up not using it as it is quite limiting. Bill is right here - this is not something that should be abstracted. All projects pick a database to specialize on eventually because supporting multiple different databases is a maintenance pain that nobody wants to do.
That being said I think sqlite deserves an exception here. It is extremely useful and it is ubiquitous. Don’t treat it as a one of many possible SQL databases in vendor. Treat it as one of a kind thing. Example: python has sqlite built into the language while no other databases have same level of treatment.
12 Likes
How about adding another tier to the Odin library hierarchy? Have a section on the website with links to libraries/bindings judged to be pretty solid but not “officially” supported. Seems like it would fit pretty well with the Odin package semantics. You’d have to play favorites if there were competing libraries of comparable quality, but that sounds like a good problem to have.
I like the idea of a place to get Odin “shared” libraries that have been vetted similar to how the git examples page does it. It could be another git page managed by a trusted member of the community to except and reject submissions. All submitted libraries would be downloadable to and import directly from Odin’s default collection folder, “shared”.
I personally would prefer to stay happily in the Odin idiomatic paradigm without having to create bindings to foreign code myself. Even though Odin does a fantastic job in supporting that ability, I myself do not enjoy doing it, since I then have to work with foreign code syntax that I don’t particular enjoy using as much. It would be nice if others who are good at that sort of thing, and have a good knowledge of the subject matter to create shared libraries for schleps like me who prefer to stay in Odin/Vendor.
That said … I still wonder though, would this sort of thing be creeping too closely with introducing a kind of dependency Hel? Could there be a good way to structure such a git page, and the rules to go with it, to keep things more Baldr like and less Loki?
I’d say don’t make it a git page, just a page on the odin website with links to solid libraries. No support implied or provided, but gives library developers an extra incentive to polish their libraries enough to qualify for a link.
I have a Noise protocol library that I use for my own purposes that I might bother to polish enough to make useful to others if I had a reasonable expectation that it would get some amount of visibility.
I would be tempted to make a good sqlite package for the same reason. I wouldn’t bother with the polish/API design if only I will ever even look at it.
Turns out this already exists. I would suggest putting a more visible link to thisnon the website. I found this on the last line of the overview (edit: Nevermind this list is very outdated):
Except that you can’t “trivially” change between SQLite and some other database, and it isn’t just syntactical differences with each dialect. There are differing levels of support for different types and type conversions, varying capabilities in trigger functions, and all sorts of significant differences that render it totally nontrivial to move from one database to another. ORMs create the illusion that this is simple, but it absolutely isn’t. And when ORMs make this “simple” they often abstract things in such a way that you probably aren’t getting the full power out of e.g. Postgres and are probably making your database work harder than it really needs to in order to do whatever you’re trying to do. What you actually want is a query builder and a marshalling helper to get row results into real structs, not an ORM.
You really don’t want your test/local database and production database to be different engines in the first place- and it’s even a minor hazard if they’re different versions. It’s way better to run e.g. Postgres in a container or something for your test environment than have to deal with bugs you can’t reproduce locally because you’re using a completely different database engine.
All true. I would provide bindings to both sqlite and postgres (libpq) but I think sqlite is a much more natural fit for Odin. Having database bindings would also help remove some of the “only for game dev” stigma from Odin.
Anyway, this is mostly moot since Karl Zylinskis bindgen makes generating high quality bindings yourself trivial, especially for sqlite.
1 Like
I can see a few significant challenges/quirks in adding SQLite to vendor that haven’t already been mentioned. The SQLite API includes a handful of older deprecated functions alongside newer, and much better, APIs that have better error semantics and configuration. Vendor generally does the most basic/straightforward mapping of stripping prefixes and having an otherwise 1:1 binding with minimal, if any, wrapping. So likely this would mean you end up exposing the bare API, including older functions that don’t necessarily need to be supported. And this is fine, I guess, but it doesn’t really steer the user toward “idiomatic” Odin such as iterators.
To really make a quality Odin SQLite library, more than likely you want a higher level wrapper over the C API which e.g. handles all of the (un)marshaling to/from structs, iterators, simplified allocator setup, migration scripts, query building, etc… and it doesn’t seem like this is really the convention of vendor.
I think the reason that SQLite bindings work for Python is that it doesn’t need to expose the base API and can be reasonably expected to be a higher-level wrapper over it.
1 Like
This. I’d like to see more of this. I was just thinking about it this morning. Something along the lines of place bindings in subfolder of library, then the actual library package imports all that (or could be in same package, but separate file, what-ever) and wraps all the bindings into Odin syntax interfaces. Then the 3rd-party can just focus on Odin syntax when using the library.
I just did something similar with the very simple signal bindings in libc. I wanted to use them without having to declare signal handlers as “cdecl” and constantly casting to i32 to support c’s int.
signal.odin