Fix Incorrect Default Parameters in NoiseScheduler for StableDiffusion #2065

yuvraj-wale · 2023-09-08T22:10:00Z

What does this PR do?

Fixes #2049

Before submitting

This PR fixes a typo or improves the docs (you can dismiss the other checks if that's the case).
Did you read the contributor guideline,
Pull Request section?
Was this discussed/approved via a Github issue? Please add a link
to it if that's the case
--> NoiseScheduler parameters for StableDiffusion are wrong #2049
Did you write any new necessary tests?
If this adds a new model, can you run a few training steps on TPU in Colab to ensure that no XLA incompatible OP are used?

Who can review?

Anyone in the community is free to review the PR once the tests have passed. Feel free to tag
members/contributors who may be interested in your PR.

google-cla · 2023-09-08T22:10:03Z

Thanks for your pull request! It looks like this may be your first contribution to a Google open source project. Before we can look at your pull request, you'll need to sign a Contributor License Agreement (CLA).

View this failed invocation of the CLA check for more information.

For the most up to date status, view the checks section at the bottom of the pull request.

jbischof · 2023-09-12T00:06:27Z

Thank you @yuvraj-wale!

Have you tested our API with these new parameters? Do you see any impact on image generation? You can try this notebook gist.

By the way, we will need to update our "golden value" SD test with this change (link). I recommend running the following command in your branch to verify:

pytest keras_cv/models/stable_diffusion --run_extra_large

ianstenbit · 2023-09-12T03:16:32Z

Thank you @yuvraj-wale!

Have you tested our API with these new parameters? Do you see any impact on image generation? You can try this notebook gist.

By the way, we will need to update our "golden value" SD test with this change (link). I recommend running the following command in your branch to verify:
pytest keras_cv/models/stable_diffusion --run_extra_large

This won't change our golden values, because the NoiseScheduler is only relevant for training, not the forward pass.
I think the best thing to verify is one of the fine-tuning flows (e.g. Textual Inversion)

jbischof · 2023-09-12T03:33:18Z

Thanks for the clarification @ianstenbit

divyashreepathihalli · 2023-12-07T23:00:54Z

@yuvraj-wale have you tested with a fine tuning workflow like Textual inversion? - #2065 (comment)

here is the link to it - https://keras.io/examples/generative/fine_tune_via_textual_inversion/

divyashreepathihalli · 2023-12-16T01:37:26Z

@yuvraj-wale can you please rebase the branch with master? and check if the golden values still remain the same?

) * Update PULL_REQUEST_TEMPLATE.md * Update CONTRIBUTING.md * Update CONTRIBUTING.md * Update basic_training.py * More namex fixes

We need to cut a release to get the namex fixes deployed.

Need to cut another minor release for bugfix.

…s-team#2071) * Update deeplab_v3_plus.py Update the comment of the `num_classes`parameter which contains the background class and the classes from the data. * Update deeplab_v3_plus_test.py Update the tests following the updating of 'num_classes' parameter (now including the background class)

Signed-off-by: Pedro Kaj Kjellerup Nacht <[email protected]>

* Start adding components for the segment anything model * SAMLayerNormalization -> keras.layers.LayerNormalization They both behave exactly the same when moving_mean and moving_variance are None and epsilon is 1e-6 * Move the image encoder to detectron2 backbone and fix for tf.keras backend * Address review comments and address saving bug - Use `keras_cv.export_api.keras_cv_export` instead of `keras.saving.register_keras_serializable`. - Add a `SerializableSequential` class to address the saving bug with the `Sequential` model. - Push the helper functions in `keras_cv/layers/detectron2_layers.py` to the bottom of the file. - Add the detectron2 layers to the `keras_cv/layers/__init__.py` file. - Add a test for the `ViTDetPatchingAndEmbedding` layer. * Make the backbone functional; unite MLP and MLPBlock * Address David's review comments * Add SAM Task model; make MaskDecoder and PromptEncoder XLA compatible * Remove a stray file * Add docs for the Task model * Add more references [skip ci] * Remove SerializableSequential layer * detectron2 -> vit_det; add SAM presets; fix ViTDet presets * Increse test tolerence for GCB Run

* Export SD submodels * Remove empty folder

* adding deeplab_v3_plus_presets * update formatting * fixed error * reformatted * fix errors * added num_classses arg * add input_shape arg * update test * update input_shape * updated preset * update test * move update to from_preset to task.py * reformatted code * updated doc string * Update assert * Update deeplab_v3_plus_test.py test * Update task.py * Update deeplab_v3_plus_test.py * Update deeplab_v3_plus_test.py

* fix example in docstring * review comments

* Bump python version requirement to 3.9 Keras-core now requires at least 3.9, and tf 2.14 (out imminently) will require 3.9 as well. We should follow suit for cv/nlp. * Fix build errors with new keras-core version * Update devcontainer to 3.9

* Use "masks" and "boxes" as prompt keys for SAM * Fix tests

* Use "masks" and "boxes" as prompt keys for SAM * Fix tests * Support missing prompts for SAM * Update docstrings * Review fix * Fix partial prompting for top-level model

…eam#2001 (keras-team#2077) * Test to demonstrate failures in import * Explicitly specify full package for more modules, as shown in keras-team#2001

* Update VITDet to conform to KerasCV scaling standards * dtype fix

) * Support importing Keras3 as Keras instead of Keras Core * Fix version parsing * Remove try/catch (not required)

…team#2093) * Export task models as both models.task.Model and models.Model * Same for ImageClassifier

* Add fixes for backwards compatibility for versions <2.15 * Bump version number for presets to version 2 * Port over RetinaNet custom metric fix * Bump DeepLabV3Plus preset version after re-upload

* Modify kaggle handles to full path * Modify preset handling * Fix formatting

* Fix YOLOv8Detector deserialization * Fix nit * Deserialize preditction decoder

Bumps the python group with 4 updates: [tf-nightly-cpu](https://github.com/tensorflow/tensorflow), torch, torchvision and [tf-nightly[and-cuda]](https://github.com/tensorflow/tensorflow). Updates `tf-nightly-cpu` from 2.16.0.dev20231221 to 2.16.0.dev20240104 - [Release notes](https://github.com/tensorflow/tensorflow/releases) - [Changelog](https://github.com/tensorflow/tensorflow/blob/master/RELEASE.md) - [Commits](https://github.com/tensorflow/tensorflow/commits) Updates `torch` from 2.1.2 to 2.1.2+cu121 Updates `torchvision` from 0.16.2 to 0.16.2+cu121 Updates `tf-nightly[and-cuda]` from 2.16.0.dev20231221 to 2.16.0.dev20240104 - [Release notes](https://github.com/tensorflow/tensorflow/releases) - [Changelog](https://github.com/tensorflow/tensorflow/blob/master/RELEASE.md) - [Commits](https://github.com/tensorflow/tensorflow/commits) --- updated-dependencies: - dependency-name: tf-nightly-cpu dependency-type: direct:production update-type: version-update:semver-patch dependency-group: python - dependency-name: torch dependency-type: direct:production update-type: version-update:semver-patch dependency-group: python - dependency-name: torchvision dependency-type: direct:production update-type: version-update:semver-patch dependency-group: python - dependency-name: tf-nightly[and-cuda] dependency-type: direct:production update-type: version-update:semver-patch dependency-group: python ... Signed-off-by: dependabot[bot] <[email protected]> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>

Signed-off-by: Pedro Kaj Kjellerup Nacht <[email protected]>

* fix nms serialization * remove the default function --------- Co-authored-by: Divyashree Sreepathihalli <divyashreepathihalli>

Co-authored-by: Divyashree Sreepathihalli <divyashreepathihalli>

Keras, KerasNLP and KerasTuner all store things liek the CONTRIBUTING.md in the toplevel of the repo. We should do the same here.

* Unify `version` API with keras and keras_nlp * Formatting * Update to keep `version` parity with KerasNLP, support nightly version string * Update version_utils.py * Update version_utils.py

* Update random_crop_and_zoom.py * Update description * rename file

* BASNet model initial code structure * adding test and initial preset details * adding comments * cleaning and formatting code * keras 3 support added * disabling preset test for BASNet

Fixed image.shape issue for tensorflow backend

…eras-team#2313) * Create auto-assignment.yaml * Create auto-assignment.js * Create stale-issue-pr.yaml * Rename auto-assignment.yaml to auto-assignment.yml * Rename stale-issue-pr.yaml to stale-issue-pr.yml

…m#2322)

Bumps the github-actions group with 3 updates: [actions/cache](https://github.com/actions/cache), [actions/upload-artifact](https://github.com/actions/upload-artifact) and [github/codeql-action](https://github.com/github/codeql-action). Updates `actions/cache` from 3 to 4 - [Release notes](https://github.com/actions/cache/releases) - [Changelog](https://github.com/actions/cache/blob/main/RELEASES.md) - [Commits](actions/cache@v3...v4) Updates `actions/upload-artifact` from 4.0.0 to 4.3.0 - [Release notes](https://github.com/actions/upload-artifact/releases) - [Commits](actions/upload-artifact@c7d193f...26f96df) Updates `github/codeql-action` from 3.22.12 to 3.23.2 - [Release notes](https://github.com/github/codeql-action/releases) - [Changelog](https://github.com/github/codeql-action/blob/main/CHANGELOG.md) - [Commits](github/codeql-action@012739e...b7bf0a3) --- updated-dependencies: - dependency-name: actions/cache dependency-type: direct:production update-type: version-update:semver-major dependency-group: github-actions - dependency-name: actions/upload-artifact dependency-type: direct:production update-type: version-update:semver-minor dependency-group: github-actions - dependency-name: github/codeql-action dependency-type: direct:production update-type: version-update:semver-minor dependency-group: github-actions ... Signed-off-by: dependabot[bot] <[email protected]> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>

Added correct default parameters in NoiseScheduler

11a0913

ianstenbit self-requested a review September 12, 2023 03:14

divyashreepathihalli added the kokoro:force-run Runs Tests on GPU label Dec 16, 2023

kokoro-team removed the kokoro:force-run Runs Tests on GPU label Dec 16, 2023

ianstenbit and others added 20 commits February 9, 2024 02:16

Fix some imports in KerasCV that are invalid with Namex (keras-team#2064

5b3ad08

) * Update PULL_REQUEST_TEMPLATE.md * Update CONTRIBUTING.md * Update CONTRIBUTING.md * Update basic_training.py * More namex fixes

Update version to 0.6.3 (keras-team#2066)

c4c0710

We need to cut a release to get the namex fixes deployed.

Fix input path for visualization issue (keras-team#2068)

e3400e8

Update version to 0.6.4 (keras-team#2069)

f81780e

Need to cut another minor release for bugfix.

Set read-only tokens for all workflows (keras-team#2075)

f0c1f88

Signed-off-by: Pedro Kaj Kjellerup Nacht <[email protected]>

Export SD submodels (keras-team#2080)

2c614d3

* Export SD submodels * Remove empty folder

[Yolov8 detector] Fix example in docstring (keras-team#2082)

36368bb

* fix example in docstring * review comments

fixed shape issue in example in docstring (keras-team#2083)

896102d

Use "masks" and "boxes" as prompt keys for SAM (keras-team#2084)

09d9fbb

* Use "masks" and "boxes" as prompt keys for SAM * Fix tests

Support partial prompting in SAM (keras-team#2085)

492bb40

* Use "masks" and "boxes" as prompt keys for SAM * Fix tests * Support missing prompts for SAM * Update docstrings * Review fix * Fix partial prompting for top-level model

Explicitly specify full package for more modules, as shown in keras-t…

2ef46ee

…eam#2001 (keras-team#2077) * Test to demonstrate failures in import * Explicitly specify full package for more modules, as shown in keras-team#2001

Update VITDet to conform to KerasCV scaling standards (keras-team#2086)

bec2706

* Update VITDet to conform to KerasCV scaling standards * dtype fix

Resolve mean/std swap for VITDet backbone (keras-team#2087)

3feef5b

Support importing Keras3 as Keras instead of Keras Core (keras-team#2089

ff4f3c1

) * Support importing Keras3 as Keras instead of Keras Core * Fix version parsing * Remove try/catch (not required)

remove tests from pypi build (keras-team#2094)

5b7cfee

Export task models as both models.task.Model and models.Model (keras-…

03cebe6

…team#2093) * Export task models as both models.task.Model and models.Model * Same for ImageClassifier

nkovela1 and others added 28 commits February 9, 2024 02:16

Modify layer checkpoint dependencies to empty list (keras-team#2247)

9e2a82c

Change layer checkpoint dependencies to empty dict (keras-team#2249)

61dc1a9

Backwards compatibility fixes for Keras <=2.15 (keras-team#2273)

e9ba6ba

* Add fixes for backwards compatibility for versions <2.15 * Bump version number for presets to version 2 * Port over RetinaNet custom metric fix * Bump DeepLabV3Plus preset version after re-upload

Remove weight traversal output

49c7313

Align keras requirments fo torch (keras-team#2272)

b91add7

Modify preset kaggle handles to full path (keras-team#2279)

95545ad

* Modify kaggle handles to full path * Modify preset handling * Fix formatting

Update base image aug layer tensor conversion (keras-team#2281)

0b88c5f

Update version to 0.8.1 (keras-team#2282)

0f3fc04

Fix YOLOv8Detector deserialization (keras-team#2283)

5292bd9

* Fix YOLOv8Detector deserialization * Fix nit * Deserialize preditction decoder

Add Scorecard Action (keras-team#2267)

a495ad0

Signed-off-by: Pedro Kaj Kjellerup Nacht <[email protected]>

add nms serialization test (keras-team#2285)

6698fcf

* fix nms serialization * remove the default function --------- Co-authored-by: Divyashree Sreepathihalli <divyashreepathihalli>

Add dict check for deserialization (keras-team#2286)

2f964c1

updata dataset link (keras-team#2287)

7af945a

Co-authored-by: Divyashree Sreepathihalli <divyashreepathihalli>

Version bump master to 0.8.2 (keras-team#2290)

758a4c2

Move keras-cv markdown files to toplevel (keras-team#2291)

f5f6b9a

Keras, KerasNLP and KerasTuner all store things liek the CONTRIBUTING.md in the toplevel of the repo. We should do the same here.

Add version() API to unify with Keras and KerasNLP (keras-team#2199)

0e51edb

* Unify `version` API with keras and keras_nlp * Formatting * Update to keep `version` parity with KerasNLP, support nightly version string * Update version_utils.py * Update version_utils.py

Update random_crop_and_zoom.py (keras-team#2294)

0444db2

* Update random_crop_and_zoom.py * Update description * rename file

bug fix (keras-team#2303)

18c5297

Add BASNet Segmentation Model (keras-team#2006) (keras-team#2271)

4e5e76a

* BASNet model initial code structure * adding test and initial preset details * adding comments * cleaning and formatting code * keras 3 support added * disabling preset test for BASNet

Fix image.shape type (keras-team#2305)

4b7132a

Fixed image.shape issue for tensorflow backend

Create workflow for auto assignment of issues and for stale issues (k…

b44b32d

…eras-team#2313) * Create auto-assignment.yaml * Create auto-assignment.js * Create stale-issue-pr.yaml * Rename auto-assignment.yaml to auto-assignment.yml * Rename stale-issue-pr.yaml to stale-issue-pr.yml

Fix format and Update Vectorized Base (keras-team#2323)

85a8956

Fix CI Test for Basnet OOM and PyCoCo Test Failure for JAX (keras-tea…

f3809c5

…m#2322)

Reduce memory consumption for BasNet tests (keras-team#2325)

43da18d

Remove Bazel steps from Contributing Guide (keras-team#2324)

4ca55a0

Update version to 0.8.3 (keras-team#2327)

4f2072e

divyashreepathihalli requested review from divyashreepathihalli and removed request for ianstenbit February 12, 2024 18:11

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Fix Incorrect Default Parameters in NoiseScheduler for StableDiffusion #2065

Fix Incorrect Default Parameters in NoiseScheduler for StableDiffusion #2065

yuvraj-wale commented Sep 8, 2023 •

edited

Loading

google-cla bot commented Sep 8, 2023

jbischof commented Sep 12, 2023 •

edited

Loading

ianstenbit commented Sep 12, 2023

jbischof commented Sep 12, 2023

divyashreepathihalli commented Dec 7, 2023

divyashreepathihalli commented Dec 16, 2023

Fix Incorrect Default Parameters in NoiseScheduler for StableDiffusion #2065

Are you sure you want to change the base?

Fix Incorrect Default Parameters in NoiseScheduler for StableDiffusion #2065

Conversation

yuvraj-wale commented Sep 8, 2023 • edited Loading

What does this PR do?

Before submitting

Who can review?

google-cla bot commented Sep 8, 2023

jbischof commented Sep 12, 2023 • edited Loading

ianstenbit commented Sep 12, 2023

jbischof commented Sep 12, 2023

divyashreepathihalli commented Dec 7, 2023

divyashreepathihalli commented Dec 16, 2023

yuvraj-wale commented Sep 8, 2023 •

edited

Loading

jbischof commented Sep 12, 2023 •

edited

Loading