Outpainting Pipeline Implementation on ai-runner Bounty [$850]

[JJassonn69]

Overview

We are excited to announce the expansion of our AI Network with the introduction of Outpainting pipeline. This feature will enable advanced image expansion and modification techniques, improving the versatility of our AI network. 🔥

We are reaching out to the community to contribute to the implementation of this this essential pipeline within the AI-worker side of the AI-Network. Your contribution will be an invaluable addition to our image editing capabilites, pushing Livepeer AI Network at the forefront of AI-media generation. 🚀

Required Skillset

• Proficiency with FastAPI.
• Strong skills in Python.
• Familiarity with Docker and Hugging Face 🤗.

Bounty Requirements

1. Implementation: Develop a functional /out-paint route and pipeline within the AI-worker repository. This new pipeline should be accessible through Docker on port 8700.
2. Functionality: The pipeline must be able to take an initial image, and text input as a prompt, return an modified image as the result. It should also allow for optional parameters like negative_prompt, controlnet_conditioning_scale, num_inference_steps, and guidance_scale to be adjustable according to the outpainting model utilized.

Example workflow

Originally posted by @stronk-dev in #56 (comment)

Scope Exclusions

• This bounty does NOT include the complete end-to-end implementation of this pipeline on the go-livepeer side, such as payment processing or job routing. These elements will be handled by the AI SPE team or in a future bounty.

Implementation Tips

To learn how to create a new pipeline, refer to the Hugging Face pipeline documentation. You can also explore the following pull requests to see how other pipelines were implemented:

• Pull Request #96
• Pull Request #103

In some cases, you may not be able to integrate the new pipeline directly into the regular AI Runner Docker container due to dependency conflicts or missing packages. If this occurs, you can follow the approach outlined in this example to create a custom container for the pipeline. This approach uses the regular AI runner as the base while keeping the base container lean.

Additionally, keep the following best practices in mind:

• Leverage Previous Work: Examine existing implementations of outpainting available in Hugging Face documentation and this space. These can serve as valuable starting points.
• Consult Developer Documentation: Our developer documentation for the worker and runner includes helpful tips for mocking pipelines and direct debugging, speeding up the development process.
• Update OpenAPI Specification: Run the runner/gen_openapi.py script to generate an updated OpenAPI specification.
• Generate Go-Livepeer Bindings: In the main repository directory, execute the make command to generate the necessary bindings, ensuring compatibility with the go-livepeer repository.

How to Apply

1. Express Interest: Comment on this issue to show your interest and explain why you are a great fit for this task.
2. Await Review: Our team will review expressions of interest and select the most suitable candidate.
3. Get Assigned: If chosen, the GitHub issue will be assigned to you.
4. Start Working: Begin the task! If you need help or guidance, comment on the issue or join discussions in the #developer-lounge channel on our Discord server.
5. Submit Your Work: Create a pull request in the relevant repository and request a review.
6. Notify Us: Comment on this GitHub issue when your pull request is ready for review.
7. Receive Your Bounty: Once your pull request is approved, we will arrange the bounty payment.
8. Earn Recognition: Your valuable contributions will be highlighted in our project's changelog.

We appreciate your interest in contributing to our project! 💛

Warning

Please wait for the issue to be assigned to you before starting work. To prevent duplication of effort, submissions for unassigned issues will not be accepted.

[stronk-dev]

Not sure if this is a requirement for good outpainting results, but just wanted to drop this image from Reddit here for the person that will be working on this bounty. Apparently generating a mosaic over the area to be repainted helps?

[rickstaa]

Not sure if this is a requirement for good outpainting results, but just wanted to drop this image from Reddit here for the person that will be working on this bounty. Apparently generating a mosaic over the area to be repainted helps?

@stronk-dev damn that image is great! I added it as an example workflow to the bounty description. Thanks for jumping in 🙏🏻❤️.

[lukiod]

will work on it after finishing the stable-audio pipeline work if someone took on in between no problem will require a time period of a week to finish the previous bounty

[aybanda]

@JJassonn69 can you assign it to me?

[rickstaa]

Hi @aybanda, thanks for your patience! We've been wrapped up with the public launch at Token2049. It’s fantastic to have you contributing to our open-source project 🚀. I’ve assigned you to the bounty now. Don't hesitate to reach out if you have any questions!

[aybanda]

@rickstaa will do my best in delivering this 👍
Thank you