Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add the new OpenAI Moderation model #811

Open
wants to merge 1 commit into
base: develop
Choose a base branch
from

Conversation

dkotter
Copy link
Collaborator

@dkotter dkotter commented Sep 27, 2024

Description of the Change

In #531 we added a Feature to moderate comments using the OpenAI Moderation tool. They recently released a new Moderation model (omni-moderation-latest, see https://openai.com/index/upgrading-the-moderation-api-with-our-new-multimodal-moderation-model/) so this PR ensures we are using that model. In addition, added in a helper method (similar to what we do in other places) making it easy to filter that model if someone wants to use the older one.

Note: this new moderation model is multi-modal, which means it supports images. I've not added any new functionality in this PR but this new model would open up other possibilities, like moderating image uploads as an example.

How to test the Change

  1. Enable and configure the Moderation Feature
  2. Enable comments on your site
  3. Post a comment with hateful language in it and ensure the comment is saved but is set to a Pending status
  4. With the Feature disabled, post another hateful comment and see that it doesn't get set to Pending
  5. Go to the comments list in the admin (wp-admin/edit-comments.php) and find the new comment. Using the bulk dropdown or the inline links, send the comment to moderation
  6. Ensure it gets set to a Pending status

Changelog Entry

Changed - Use the new OpenAI Moderation model, omni-moderation-latest, in our comment moderation Feature
Developer - New filter, classifai_openai_moderation_model, allowing you to change the moderation model

Credits

Props @dkotter

Checklist:

… requests. Add a helper method to get this model which makes it easy for others to override this
@dkotter dkotter added this to the 3.2.0 milestone Sep 27, 2024
@dkotter dkotter self-assigned this Sep 27, 2024
@dkotter dkotter requested review from jeffpaul and a team as code owners September 27, 2024 18:31
@github-actions github-actions bot added the needs:code-review This requires code review. label Sep 27, 2024
@jeffpaul
Copy link
Member

@dkotter are there any changes to the moderation flags that the newer OAI model categorizes moderated comments into or are they the same as before?

@dkotter
Copy link
Collaborator Author

dkotter commented Sep 27, 2024

@dkotter are there any changes to the moderation flags that the newer OAI model categorizes moderated comments into or are they the same as before?

There are two new categories it will look for: illicit and illicit/violent but the rest are the same

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
needs:code-review This requires code review.
Projects
Status: Code Review
Development

Successfully merging this pull request may close these issues.

2 participants