feat(ai): add pipelines optimization flags

[rickstaa]

Warning

Do not merge before livepeer/ai-worker#61.

What does this pull request do? Explain your changes. (required)

This commit adds a new OptimizationFlags field to the aiModels config so that users can forward optimization environment variables to the ai-worker for more information see livepeer/ai-worker#61.

Specific updates (required)

• New OptimizationFlags field added to aiModels config.
• Flags forwaded to the ai-worker Warm function.
• A warning is thrown when users try to use the optimizationFlags with non warm containers.

How did you test each of these updates (required)

1. Ensured the package builds.
2. Ensured tests.sh were succesfull.
3. Spung up off-chain broadcaster.
4. Spung up off-chain orchestrator with SFAST optimization flag in the models configuration file.

[
    {
        "pipeline": "image-to-video",
        "model_id": "stabilityai/stable-video-diffusion-img2vid-xt-1-1",
        "price_per_unit": 3390842,
        "warm": true,
        "optimization_flags": {
            "SFAST": "true",
            "SOME": true
        }
    }
]

6. Let the ImageToVideo container warmup and check the container logs to see that SFAST is used and the model is pre-traced.
7. Send a ImageToVideo request to the broadcaster and checked that it stilled worked.

Does this pull request close any open issues?

No.

Checklist:

• Read the contribution guide
• make runs successfully
• All tests in ./test.sh pass - @rickstaa we still need to fix the tests.
• README and other documentation updated
• Pending changelog updated

[rickstaa]

@yondonfu, is there a specific reason why the binary execution proceeds even when an AIModelConfig is not specified? I encountered this issue while trying to activate flags for containers that don't start warm, as I wanted to pass Optimization flags directly to the AIWorker constructor.

go-livepeer/cmd/livepeer/starter/starter.go

Lines 534 to 605 in 1019d42

	if *cfg.AIModels != "" {
	configs, err := core.ParseAIModelConfigs(*cfg.AIModels)
	if err != nil {
	glog.Error("Error parsing -aiModels: %v", err)
	return
	}
	for _, config := range configs {
	modelConstraint := &core.ModelConstraint{Warm: config.Warm}
	// If the config contains a URL we call Warm() anyway because AIWorker will just register
	// the endpoint for an external container
	if config.Warm || config.URL != "" {
	endpoint := worker.RunnerEndpoint{URL: config.URL, Token: config.Token}
	if err := n.AIWorker.Warm(ctx, config.Pipeline, config.ModelID, endpoint); err != nil {
	glog.Errorf("Error AI worker warming %v container: %v", config.Pipeline, err)
	return
	}
	}
	switch config.Pipeline {
	case "text-to-image":
	_, ok := constraints[core.Capability_TextToImage]
	if !ok {
	aiCaps = append(aiCaps, core.Capability_TextToImage)
	constraints[core.Capability_TextToImage] = &core.Constraints{
	Models: make(map[string]*core.ModelConstraint),
	}
	}
	constraints[core.Capability_TextToImage].Models[config.ModelID] = modelConstraint
	n.SetBasePriceForCap("default", core.Capability_TextToImage, config.ModelID, big.NewRat(config.PricePerUnit, config.PixelsPerUnit))
	case "image-to-image":
	_, ok := constraints[core.Capability_ImageToImage]
	if !ok {
	aiCaps = append(aiCaps, core.Capability_ImageToImage)
	constraints[core.Capability_ImageToImage] = &core.Constraints{
	Models: make(map[string]*core.ModelConstraint),
	}
	}
	constraints[core.Capability_ImageToImage].Models[config.ModelID] = modelConstraint
	n.SetBasePriceForCap("default", core.Capability_ImageToImage, config.ModelID, big.NewRat(config.PricePerUnit, config.PixelsPerUnit))
	case "image-to-video":
	_, ok := constraints[core.Capability_ImageToVideo]
	if !ok {
	aiCaps = append(aiCaps, core.Capability_ImageToVideo)
	constraints[core.Capability_ImageToVideo] = &core.Constraints{
	Models: make(map[string]*core.ModelConstraint),
	}
	}
	constraints[core.Capability_ImageToVideo].Models[config.ModelID] = modelConstraint
	n.SetBasePriceForCap("default", core.Capability_ImageToVideo, config.ModelID, big.NewRat(config.PricePerUnit, config.PixelsPerUnit))
	}
	}
	}
	defer func() {
	ctx, cancel := context.WithTimeout(context.Background(), aiWorkerContainerStopTimeout)
	defer cancel()
	if err := n.AIWorker.Stop(ctx); err != nil {
	glog.Errorf("Error stopping AI worker containers: %v", err)
	return
	}
	glog.Infof("Stopped AI worker containers")
	}()
	}

[rickstaa]

Requires livepeer/ai-worker#61.

[yondonfu]

@rickstaa

is there a specific reason why the binary execution proceeds even when an AIModelConfig is not specified?

I think it would sense to require a config file for -aiModels now because the config file is not only used to warm models, but also for specifying prices and determining which models to advertise during capability discovery. Prior to using the config file for configuring prices and capability discovery, the config file was left as optional because the O would just try to execute any request given a model ID. But now, since models have prices and have to be explicitly advertised, the config file is necessary.

[yondonfu]

Noting for the thread that this will need a go mod update to use the latest version of ai-worker once it is merged.

[rickstaa]

@rickstaa

is there a specific reason why the binary execution proceeds even when an AIModelConfig is not specified?

I think it would sense to require a config file for -aiModels now because the config file is not only used to warm models, but also for specifying prices and determining which models to advertise during capability discovery. Prior to using the config file for configuring prices and capability discovery, the config file was left as optional because the O would just try to execute any request given a model ID. But now, since models have prices and have to be explicitly advertised, the config file is necessary.

Ah thanks for the explanation. Makes sense. I will change the behavoir!