Prompt embeds update for T2I, I2I and upscale. #140
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
… with creating go api bindings because of openapi json sceme having a null option.
…i-worker into frame-interpolation
testing workflow
* update to the frame interpolation pipeline, there is some minor issue with creating go api bindings because of openapi json sceme having a null option. * minor changes to requirements * update to requrements to fetch from --index-url * simple patch to solve the go api bindings issue * checking if it works in my system --------- Co-authored-by: Jason Stone <[email protected]>
Bumps the go_modules group with 1 update in the / directory: [github.com/docker/docker](https://github.com/docker/docker). Updates `github.com/docker/docker` from 24.0.7+incompatible to 24.0.9+incompatible - [Release notes](https://github.com/docker/docker/releases) - [Commits](moby/moby@v24.0.7...v24.0.9) --- updated-dependencies: - dependency-name: github.com/docker/docker dependency-type: direct:production dependency-group: go_modules ... Signed-off-by: dependabot[bot] <[email protected]> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
* update to the frame interpolation pipeline, there is some minor issue with creating go api bindings because of openapi json sceme having a null option. * minor changes to requirements * update to requrements to fetch from --index-url * simple patch to solve the go api bindings issue * checking if it works in my system --------- Co-authored-by: Jason Stone <[email protected]>
This commit ensures that the make file uses the right OpenAPI spec to generate the go bindings.
* feat: add Studio Gateway This commit adds the studio Gateway to the list of servers. * chore: update OpenAPI spec This commit updates the OpenAPI spec to add the Studio gateway to the list of servers and thus the documentation.
This commit makes the container map more unique providing users the case of running multiple pipelines behind one external endpoint. Co-authored-by: Rick Staa <[email protected]>
This commit provides developers with a `--version` argument they can use when generating the OpenAPI spec using the `gen_openapi.py` script`.
* feat(pipeline): add SAM2 image segmentation prototype This commit introduces a prototype implementation of the [Segment Anything v2](https://github.com/facebookresearch/segment-anything-2) (SAM2) pipeline within the AI worker. The prototype demonstrates the basic functionality needed to perform segmentation on an image. Note that video segmentation is not yet implemented. Additionally, the dependencies were updated quickly, which may temporarily break other pipelines. * revert Dockerfile, requirements, add sam2 Dockerfile * refactor: enhance SAM2 input handling and error management This commit allows nested arrays to be supplied as JSON strings for SAM2 input. It also implements robust error handling to return a 400 error with a descriptive message when incorrect parameters are provided. * refactor: improve SAM2 return time This commit ensures that we return the masks, iou_predictions and low_res_masks in json format. * Sam2 -> SegmentAnything2 * update go bindings * update multipart.go binding with NewSegmentAnything2Writer * update worker and multipart methods * predictions -> scores, mask -> logits * add sam2 specific multipartwriter fields * add segment-anything-2 to containerHostPorts * fix pipeline name in worker.go * revert Dockerfile, requirements, add sam2 Dockerfile * Sam2 -> SegmentAnything2 * predictions -> scores, mask -> logits * feat: replace JSON.dump with str This commit replaces the JSON.dump method with a simple str method since it is highly unlikely that the string contains invalid data. Co-authored-by: Peter Schroedl <[email protected]> * move pipeline-specific dockerfile * update openapi yaml * add segment anything specific readme * update go bindings * refactor: move SAM2 docker This commit moves the SAM2 docker file inside the docker container. * refactor: add FastAPI descriptions This commit cleansup the codebase and adds FastAPI parameter and pipeline descriptions. * refactor: improve sam2 route function name This commit improves the sam2 route function name so that it is more pythonic and shows up nicer in the OpenAPI spec pipeline summary. * chore(worker): update golang bindings This commit updates the golang bindings so that the runner changes are reflected. * refactor(runner): add media_type This commit adds the media type content MIME type to the segment anything 2 pipeline. * chore(worker): remove debug patch This commit removes the debug patch which was accidentally added to the code. * feat(runnner): add SAM2 model download command This commit adds the SAM2 model download command so that orchestrators can pre-download the model. * refactor(worker): change SAM2 multipart reader param order This commit ensures that the parameters are in the same order as the pipeline parameters. * determine docker image in createContainer * fix: fix examples This commit fixes the example scripts. --------- Co-authored-by: Rick Staa <[email protected]> Co-authored-by: Elite Encoder <[email protected]> Co-authored-by: Peter Schroedl <[email protected]>
…vepeer#167) This commit adds the black-forest-labs/FLUX.1 model download commands. The dev model is placed under the `--restricted` flag since it can not be used for commercial purposes. Co-authored-by: Rick Staa <[email protected]>
This commit updates the version set in the OpenAPI spec.
This commit ensures that the main Docker container is also tagged as the base container so that it can be used as the base for the pipeline specific containers. Co-authored-by: ad-astra-video <[email protected]>
This commit ensures that developers can trigger docker image building.
This commit ensures that the workflow dispatch triggers the docker job.
* chore(docker): add 'base' tag and segment-anything-2 docker image build * update segment-anything-2 to dynamic base image * make more space on runner * refactor(ci): split Docker CI This commit ensures that the pipeline docker build ci is found in a seperate action from the base. * ci(docker): enable pipeline docker workflow dispatch This commit ensures that maintainers can trigger the pipeline specific Docker action using a workflow dispatch. * ci(docker): fix out of space error This commit switches to the oxfort runner to see if it can fis the OS storage error. * ci: cleanup hosted runner This commit cleans up the hosted runner so that we don't run into OS storage issues when trying to build the container. --------- Co-authored-by: Brad P <[email protected]>
This commit ensures that the SAM2 docker has the right tags.
This commit ensures that maintainers can manually run the pipeline docker ci.
* feat(sdks): implement SDK-specific API customizations This commit introduces several SDK-specific OpenAPI configurations to the runner API configuration. These customizations will enhance the SDKs we are planning to release. Co-authored-by: Victor Elias <[email protected]>
This commit enables the [speakeasy retries](https://www.speakeasy.com/docs/customize-sdks/retries#global-retries) feature for the SDKs.
This reverts commit caa4bb7.
This commit releases a new minor version since we had to revert the SDK related changes.
This commit updates the code and documentation to signal we are entering the Beta phase of the AI network journey.
This commit prevents a Key Error from being thrown when the pipelines are called directly.
This commit removes the 'batch_size' argument from the benchmarking script since our current pipeliens don't support batching requests due to us not having a way to estimate VRAM and preventing out of memory errors. For more information see livepeer#66. We can add this option back in when we have solved this.
* update to the frame interpolation pipeline, there is some minor issue with creating go api bindings because of openapi json sceme having a null option. * minor changes to requirements * update to requrements to fetch from --index-url * simple patch to solve the go api bindings issue * checking if it works in my system * test-examples for frame-interpolation * update to sfast optimization to i2i and t2i and upscale pipelines * changes to extra files * added git ignore to the files to remove unnecessary files * files not removed checking again * still in test phase * test-test * Update .gitignore * Delete runner/app/tests-examples directory * update to directory reader as it now reads almost any naming convention --------- Co-authored-by: Jason Stone <[email protected]>
Bumps the go_modules group with 1 update in the / directory: [github.com/docker/docker](https://github.com/docker/docker). Updates `github.com/docker/docker` from 24.0.9+incompatible to 26.1.4+incompatible - [Release notes](https://github.com/docker/docker/releases) - [Commits](moby/moby@v24.0.9...v26.1.4) --- updated-dependencies: - dependency-name: github.com/docker/docker dependency-type: direct:production dependency-group: go_modules ... Signed-off-by: dependabot[bot] <[email protected]> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Bumps the go_modules group with 1 update in the / directory: [github.com/docker/docker](https://github.com/docker/docker). Updates `github.com/docker/docker` from 26.1.4+incompatible to 26.1.5+incompatible - [Release notes](https://github.com/docker/docker/releases) - [Commits](moby/moby@v26.1.4...v26.1.5) --- updated-dependencies: - dependency-name: github.com/docker/docker dependency-type: direct:production dependency-group: go_modules ... Signed-off-by: dependabot[bot] <[email protected]> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
|
@ad-astra-video , i have cleaned up and rebased it using the main branch from ai-worker. In regards to the changing of embeddings from Compel to another is out of the scope of the bounty initially listed. We might have to add another bounty for that as it will require extensive testing with all the models and pipelines. |
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
the all the models worked nicely except timbrooks/pix2pix, when prompting use + or - after the word to increase the weight of the word in generation, you can add multiple ++ or -- to increase or decrease weight.
the inferencing time didnot change at all most of the time or was barely reduced for some cases.
example of prompt "a cat playing with a ball++" , here the ball has higher weight so it will be more focused when generating.