Skip to content

Scheduler plugins designed for smart scheduling with llmaz.

Notifications You must be signed in to change notification settings

InftyAI/scheduler-plugins

Repository files navigation

vScheduler

A Kubernetes scheduler designed for smart scheduling with llmaz.

Plugins

vScheduler maintains multiple plugins for llm workloads scheduling.

ResourceFungibility Plugin

A llama2-7B model can be run on 1xA100 GPU, can also be run on 1xA10 GPU, this is what we called fungibility.

With resourceFungibility plugin, we can simply achieve this with at most 8 alternative GPU types.

About

Scheduler plugins designed for smart scheduling with llmaz.

Resources

Code of conduct

Stars

Watchers

Forks

Releases

No releases published

Packages

No packages published