Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Model Refresh #990

Open
jainnitk opened this issue Jun 3, 2024 · 1 comment
Open

Model Refresh #990

jainnitk opened this issue Jun 3, 2024 · 1 comment

Comments

@jainnitk
Copy link

jainnitk commented Jun 3, 2024

Hi,
I am struggling to understand how the Refresh module works. As per my understanding and documentation, the model refresh takes a base model, updates the range for hyperparameters from the base model and then updates the model with following criteria:

  1. Stable baseline compared to the the base model
  2. Media spend changes in the new period are reflected in the refreshed model which is the objective

However, I am unable to wrap my head around on how the above criteria is being handled in the code (pardon my R illiteracy). Further, with only a few data points added to refresh the model, how will the coefficients change to reflect the changes happening in new period added as the weight would be too less for a few data points.

Do we make some changes in the objective function to achieve the desired results. If yes, which part of the code handles this?

Please help here so that I can understand how the refresh module works to reflect the changes in the new period.

@jainnitk
Copy link
Author

jainnitk commented Jun 4, 2024

Found this in the Analyst's guide:
"Robyn then sets the lower and upper bounds of hyperparameters for the new build, which will be in line with the selected hyperparameters of the initial build. This will therefore stabilise the effect of contextual and organic variables across initial and new builds as well as regulating the new share of effect of media variables towards the new added period spend level."

This seems to indicate that setting narrower bounds for refresh model hyperparameters automatically takes care of new share of effect of media variables to skew towards new data. Not sure how does that happen.
Further, the following snippet says:
"Robyn uses modified objective functions to achieve the above. To be precise, the NRMSE only accounts for the new period while refreshing, because the main goal of refreshing is to better describe the new period. The DECOMP.RSSD is modified to drive only effect share closer to the refresh spend share, while it also accounts for the similarity between new and old baseline."

Could you please help me what all measures are taken to achieve the above mentioned objectives and also which part of the code modifies the NRMSE and DECOMP.RSSD for refresh.

Thanks in advance !!!

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

1 participant