Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Not able to run CLIP Model using from_present #2472

Open
carrycooldude opened this issue Aug 6, 2024 · 5 comments
Open

Not able to run CLIP Model using from_present #2472

carrycooldude opened this issue Aug 6, 2024 · 5 comments

Comments

@carrycooldude
Copy link

Current Behavior:

Screenshot from 2024-08-06 21-21-59

Expected Behavior:

not able to load CLIP model even though I installed keras-nlp too

Steps To Reproduce:

Screenshot from 2024-08-06 21-23-59

Version:

Screenshot from 2024-08-06 21-25-22

Anything else:

This is my colab link

@carrycooldude carrycooldude changed the title Not able to run CLIP Model using from_present Not able to run CLIP Model using from_present Aug 6, 2024
@sachinprasadhs
Copy link
Collaborator

You need to install the dependencies and load the clip model as below.

!pip install -U keras-cv
!pip install -U keras-nlp
!pip install -U keras

import tensorflow as tf
from tensorflow.keras.layers import Input, Dense, Concatenate
from tensorflow.keras.models import Model
from keras_cv.models import CLIP
import numpy as np

# Load the pre-trained CLIP model from KerasCV
clip_model = CLIP.from_preset("clip-vit-base-patch16")

for Image and Text encoder, you can use as below.

# Define inputs
image_input = Input(shape=(224, 224, 3), name='image_input')
text_input = Input(shape=(77,), name='text_input', dtype=tf.int32)

# Process image through CLIP
image_embeds = clip_model.image_encoder(image_input)

# Process text through CLIP
text_embeds = clip_model.text_encoder(text_input)

Copy link

This issue is stale because it has been open for 14 days with no activity. It will be closed if no further activity occurs. Thank you.

@github-actions github-actions bot added the stale label Aug 31, 2024
@carrycooldude
Copy link
Author

thanks @sachinprasadhs , it's working but I guess it's not in docs
Can I contribute to it?

@sachinprasadhs
Copy link
Collaborator

Sure

Copy link

github-actions bot commented Oct 6, 2024

This issue is stale because it has been open for 14 days with no activity. It will be closed if no further activity occurs. Thank you.

@github-actions github-actions bot added the stale label Oct 6, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

No branches or pull requests

2 participants