Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Allow task_processing-launched pods to handle their own cleanup #218

Draft
wants to merge 1 commit into
base: master
Choose a base branch
from

Conversation

nemacysts
Copy link
Member

We have some usecases where we'd like a set of batches to run some cleanup code upon recieving a SIGTERM, so we'll need to remove our force-kill code and instead fallback to the default k8s behavior of giving Pods 30s after sending a SIGTERM before sending a SIGKILL to force-terminate :)

I will also follow this up with a Tron documentation change that documents this behavior.

We have some usecases where we'd like a set of batches to run some
cleanup code upon recieving a SIGTERM, so we'll need to remove our
force-kill code and instead fallback to the default k8s behavior of
giving Pods 30s after sending a SIGTERM before sending a SIGKILL to
force-terminate :)

I will also follow this up with a Tron documentation change that
documents this behavior.
Copy link
Contributor

@jfongatyelp jfongatyelp left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

If we do this is it possible task_proc threads get blocked for 30s while just waiting for this deletion to finish? Or is python's async stuff smart enough to continue doing $other things in the meantime?

@nemacysts
Copy link
Member Author

If we do this is it possible task_proc threads get blocked for 30s while just waiting for this deletion to finish? Or is python's async stuff smart enough to continue doing $other things in the meantime?

err, that's a good question - i think we'd probably want to make this an async request (which will require some testing since we don't currently use the client to make async requests anywhere)

@nemacysts nemacysts marked this pull request as draft July 9, 2024 17:55
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants