You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Currently, on each entry upload, a go routine publishes key-value pairs to Redis for index values (email, subject, etc) mapped to an entry. If the go routine fails, the error is logged, but the request does not fail. This can result in Redis getting out of sync with what's in Trillian.
Once #191 is completed, we can use the pub/sub queue as the source of truth. We can run a subscriber that ingests new entries published to the pub/sub queue and creates Redis indices for each received message. If there is any failure, then the subscriber should not ack the message. The queue will then resend the message until there is a successful ack from the subscriber, meaning that the Redis index has been updated.
Note that publishing to the queue could still fail, but this will be handled as part of #191. Backfilling would then only need to occur with the pub/sub queue rather than the queue and Redis.
Description
Currently, on each entry upload, a go routine publishes key-value pairs to Redis for index values (email, subject, etc) mapped to an entry. If the go routine fails, the error is logged, but the request does not fail. This can result in Redis getting out of sync with what's in Trillian.
Once #191 is completed, we can use the pub/sub queue as the source of truth. We can run a subscriber that ingests new entries published to the pub/sub queue and creates Redis indices for each received message. If there is any failure, then the subscriber should not ack the message. The queue will then resend the message until there is a successful ack from the subscriber, meaning that the Redis index has been updated.
Note that publishing to the queue could still fail, but this will be handled as part of #191. Backfilling would then only need to occur with the pub/sub queue rather than the queue and Redis.
Blocked by #191
The text was updated successfully, but these errors were encountered: