[Feature Request] Dynamically Linked Shared Objects via Rust Bindings

[FerLuisxd]

By default when adding the Cargo.toml dependency in rust it will call for cmake.
Would love an option to provide the binaries from the releases tab or make it so it auto downloads the binaries from there too.
Just like typescript at the moment it seems?

[cjpais]

As far as I'm aware, Rust wants to statically compile and statically link things, so there's going to be no pre-compiles for the moment

[FerLuisxd]

What about for linking prebuilts for vulkan and cuda? @cjpais since they are not compiled by default is there a way to link them?

[cjpais]

@FerLuisxd can you provide more context for what you are trying to do?

If you want to see an end to end integration of transcribe.cpp distributed take a look at cjpais/Handy

We ship Vulkan as part of the project

[FerLuisxd]

@cjpais
So ok I understand that in rust it needs to compile so cmake is mandatory, with that (on windows) you get a transcribe.dll (and only cpu support)
however you are missing vulkan and cuda for example
If I were to enable support for vulkan/cuda

I guess you have two options:

1. Follow the guide on how to build with rust+cuda/rust+vulkan on the documentation
2. Use the already built binaries for vulkan/cuda from the Release tab (this is what I am wondering how to do)

I hope it is more clear now

[cjpais]

I think the main thing is to read the README for the bindings, and if you want to enable the features for Vulkan or CUDA that is laid out in the README

I think, like, how you framed this whole thing just maybe was a bit odd for me to read because I want to know the context of why you're looking for the dynamic linking, and you didn't explicitly say, like, hey, I have a feature request to use the Rust bindings and dynamically link. And give me the rationale for why you're trying to do that as opposed to the static build. I understand that there's real reasons. I'm just trying to understand the context of the problem because I only have a limited amount of time and I don't want to just give my time to one consumer of something. If there's a reasonable way around it for the time being, and I'm basically looking for feature requests that are going to broadly help the community. And, you know, at the end of the day, I actually think that this is a very reasonable request. But I do need more context before I keep issues open on the repo and basically need to sync my time into them. Like every issue is effectively overhead on me

[FerLuisxd]

With the llama.cpp builds you can download the cudart and the cuda builds
I don't install the cuda bindings system wide in my machine, I use the ones from the releases tab. Why? Because I see that many projects use different cuda versions, transcribe.cpp provides a dll for cuda 12.X but when you try to install the latest version is 13.3, many different versions of cuda are pinned differently with many other projects (for example in pytorch it is still 13.2), so I just started to download the prebuilt binaries and link them (to ease the different version problem).

To answer the question why one would need to dynamically link, it is to avoid installing the requirements and just drop in a dll and start using a faster backend!

My use case, is a tauri app that is an ASR app, I want to allow to switch between backends and models.

I hope this a bit more clear, and sorry for the back and forth.

[cjpais]

Okay, I just want to say I appreciate this discussion. I'm just going to rename the issue and re-open it. It really helps to understand the broader context of what you're trying to achieve and why. So, thank you for taking the time to write it because it really makes it easier for me to try and support. At the end of the day, this is a one-man show, and llama.cpp has many, many, many more contributors than this library right now. I legitimately can't do everything, and also if you want a feature, you can definitely submit a PR for it. And that is generally preferable to issues