Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

add: ggml graph encoding format #66

Merged
merged 1 commit into from
Apr 10, 2024

Conversation

danbev
Copy link
Contributor

@danbev danbev commented Mar 4, 2024

This commit adds the ggml graph encoding format to the graph_encoding enum.

The motivation for this is to allow the wasi-nn interface to support models that are encoded in the ggml format which is the model format used by llama.cpp.

This commit adds the `ggml` graph encoding format to the `graph_encoding`
enum.

The motivation for this is to allow the `wasi-nn` interface to support
models that are encoded in the `ggml` format which is the model format
used by llama.cpp.

Signed-off-by: Daniel Bevenius <[email protected]>
@danbev danbev changed the title add: gguf graph encoding format add: ggml graph encoding format Mar 4, 2024
@abrown
Copy link
Collaborator

abrown commented Mar 14, 2024

Thanks for the PR! I had been wondering when this new encoding would come up. Can you attend the ML working group on the 18th of this month (details)? It might be helpful for others interested in wasi-nn to understand the motivation here.

@danbev
Copy link
Contributor Author

danbev commented Mar 15, 2024

@abrown I'd be happy to attend if I can make the time slot 👍

abrown added a commit to abrown/bytecodealliance-meetings that referenced this pull request Mar 15, 2024
ricochet pushed a commit to bytecodealliance/meetings that referenced this pull request Mar 16, 2024
@devigned
Copy link
Contributor

devigned commented Mar 21, 2024

@danbev, what wasi-nn backend would be used to load and run the ggml encoded models? I might misunderstand, but I don't think any of the backends currently implemented will be able to load these models. Are you suggesting a new backend implementation?

@danbev
Copy link
Contributor Author

danbev commented Mar 22, 2024

what wasi-nn backend would be used to load and run the ggml encoded models.

WasmEdge has support for a llama.cpp backend. I have got a very basic llama.cpp backend working for Wasmtime and would very much like to see support for such a backend in Wasmtime in the future. Having this encoding would hopefully help existing implementations and creating new ones easier.

Are you suggesting a new backend implementation?

Yes, I'd be interested in seeing support for llama.cpp backend in Wasmtime (and other wasm runtimes).

Copy link
Collaborator

@abrown abrown left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Ok, I think we've had this PR open for an adequate amount of time and discussed it in the ML meetings. Let's merge it!

@abrown abrown merged commit c0840c2 into WebAssembly:main Apr 10, 2024
1 check passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants