Skip to content

Commit

Permalink
add note to view vignettes online
Browse files Browse the repository at this point in the history
[ci skip]
  • Loading branch information
jgabry committed Jul 14, 2020
1 parent 07b2275 commit 28d0c0e
Show file tree
Hide file tree
Showing 10 changed files with 40 additions and 17 deletions.
2 changes: 1 addition & 1 deletion DESCRIPTION
Original file line number Diff line number Diff line change
Expand Up @@ -2,7 +2,7 @@ Package: loo
Type: Package
Title: Efficient Leave-One-Out Cross-Validation and WAIC for Bayesian Models
Version: 2.3.1
Date: 2020-07-13
Date: 2020-07-14
Authors@R: c(person("Aki", "Vehtari", email = "[email protected]", role = c("aut")),
person("Jonah", "Gabry", email = "[email protected]", role = c("cre", "aut")),
person("Mans", "Magnusson", role = c("aut")),
Expand Down
2 changes: 2 additions & 0 deletions vignettes/children/SEE-ONLINE.txt
Original file line number Diff line number Diff line change
@@ -0,0 +1,2 @@
**NOTE: We recommend viewing the fully rendered version of this vignette online at
https://mc-stan.org/loo/articles/**
2 changes: 1 addition & 1 deletion vignettes/children/SETTINGS-knitr.txt
Original file line number Diff line number Diff line change
@@ -1,4 +1,4 @@
```{r, SETTINGS-knitr, include=FALSE}
```{r SETTINGS-knitr, include=FALSE}
stopifnot(require(knitr))
opts_chunk$set(
comment=NA,
Expand Down
29 changes: 16 additions & 13 deletions vignettes/loo2-example.Rmd
Original file line number Diff line number Diff line change
Expand Up @@ -15,6 +15,9 @@ params:
```{r, child="children/SETTINGS-knitr.txt"}
```

```{r, child="children/SEE-ONLINE.txt", eval = if (isTRUE(exists("params"))) !params$EVAL else TRUE}
```

# Introduction

This vignette demonstrates how to use the __loo__ package to carry out
Expand All @@ -35,7 +38,7 @@ encourage readers to refer to the following papers for more details:
In addition to the __loo__ package, we'll also be using __rstanarm__ and
__bayesplot__:

```{r, setup, message=FALSE}
```{r setup, message=FALSE}
library("rstanarm")
library("bayesplot")
library("loo")
Expand Down Expand Up @@ -72,7 +75,7 @@ Because the number of days for which the roach traps were used is not the same
for all apartments in the sample, we use the `offset` argument to specify that
`log(exposure2)` should be added to the linear predictor.

```{r}
```{r data}
# the 'roaches' data frame is included with the rstanarm package
data(roaches)
str(roaches)
Expand All @@ -86,7 +89,7 @@ roaches$roach1 <- roaches$roach1 / 100
We'll fit a simple Poisson regression model using the `stan_glm` function from
the __rstanarm__ package.

```{r, count-roaches-mcmc, results="hide"}
```{r count-roaches-mcmc, results="hide"}
fit1 <-
stan_glm(
formula = y ~ roach1 + treatment + senior,
Expand Down Expand Up @@ -121,7 +124,7 @@ to the `loo` function (see, e.g. `help("loo.array", package = "loo")`). We'll
also use the argument `save_psis = TRUE` to save some intermediate results to be
re-used later.

```{r}
```{r loo1}
loo1 <- loo(fit1, save_psis = TRUE)
```

Expand All @@ -130,7 +133,7 @@ some observations the leave-one-out posteriors are different enough from the
full posterior that importance-sampling is not able to correct the difference.
We can see more details by printing the `loo` object.

```{r}
```{r print-loo1}
print(loo1)
```

Expand Down Expand Up @@ -159,7 +162,7 @@ Using the `plot` method on our `loo1` object produces a plot of the $k$ values
with horizontal lines corresponding to the same categories as in the
printed output above.

```{r, out.width = "70%"}
```{r plot-loo1, out.width = "70%"}
plot(loo1)
```

Expand All @@ -181,7 +184,7 @@ the LOO-PIT values for our model (thick curve) is compared to many
independently generated samples (each the same size as our dataset) from the
standard uniform distribution (thin curves).

```{r}
```{r ppc_loo_pit_overlay}
yrep <- posterior_predict(fit1)
ppc_loo_pit_overlay(
Expand All @@ -202,17 +205,17 @@ regression, which is commonly used for overdispersed count data.
Unlike the Poisson distribution, the negative binomial distribution
allows the conditional mean and variance of $y$ to differ.

```{r, count-roaches-negbin, results="hide"}
```{r count-roaches-negbin, results="hide"}
fit2 <- update(fit1, family = neg_binomial_2)
```

```{r}
```{r loo2}
loo2 <- loo(fit2, save_psis = TRUE, cores = 2)
print(loo2)
```


```{r}
```{r plot-loo2}
plot(loo2, label_points = TRUE)
```

Expand All @@ -232,7 +235,7 @@ calculations are performed exactly for that observation. The results are then
recombined with the approximate LOO calculations already carried out for the
observations without problematic $k$ values:

```{r}
```{r reloo}
if (any(pareto_k_values(loo2) > 0.7)) {
loo2 <- loo(fit2, save_psis = TRUE, k_threshold = 0.7)
}
Expand All @@ -248,7 +251,7 @@ still some degree of model misspecification, but this is much better than the
`p_loo` estimate for the Poisson model.

For further model checking we again examine the LOO-PIT values.
```{r}
```{r ppc_loo_pit_overlay-negbin}
yrep <- posterior_predict(fit2)
ppc_loo_pit_overlay(roaches$y, yrep, lw = weights(loo2$psis_object))
```
Expand All @@ -263,7 +266,7 @@ the data.
We can use the `loo_compare` function to compare our two models on
expected log predictive density (ELPD) for new data:

```{r, count-roaches-loo}
```{r loo_compare}
loo_compare(loo1, loo2)
```

Expand Down
3 changes: 3 additions & 0 deletions vignettes/loo2-large-data.Rmd
Original file line number Diff line number Diff line change
Expand Up @@ -13,6 +13,9 @@ params:
%\VignetteIndexEntry{Using Leave-one-out cross-validation for large data}
-->

```{r, child="children/SEE-ONLINE.txt", eval = if (isTRUE(exists("params"))) !params$EVAL else TRUE}
```

# Introduction

This vignette demonstrates how to do leave-one-out cross-validation for large
Expand Down
3 changes: 3 additions & 0 deletions vignettes/loo2-lfo.Rmd
Original file line number Diff line number Diff line change
Expand Up @@ -26,6 +26,9 @@ knitr::opts_chunk$set(
)
```

```{r, child="children/SEE-ONLINE.txt", eval = if (isTRUE(exists("params"))) !params$EVAL else TRUE}
```

## Introduction

One of the most common goals of a time series analysis is to use the observed
Expand Down
3 changes: 3 additions & 0 deletions vignettes/loo2-moment-matching.Rmd
Original file line number Diff line number Diff line change
Expand Up @@ -15,6 +15,9 @@ params:
```{r, child="children/SETTINGS-knitr.txt"}
```

```{r, child="children/SEE-ONLINE.txt", eval = if (isTRUE(exists("params"))) !params$EVAL else TRUE}
```

# Introduction

This vignette demonstrates how to improve the Monte Carlo sampling accuracy of
Expand Down
3 changes: 3 additions & 0 deletions vignettes/loo2-non-factorized.Rmd
Original file line number Diff line number Diff line change
Expand Up @@ -23,6 +23,9 @@ knitr::opts_chunk$set(
)
```

```{r, child="children/SEE-ONLINE.txt", eval = if (isTRUE(exists("params"))) !params$EVAL else TRUE}
```

# Introduction

When computing ELPD-based LOO-CV for a Bayesian model we need to
Expand Down
7 changes: 5 additions & 2 deletions vignettes/loo2-weights.Rmd
Original file line number Diff line number Diff line change
Expand Up @@ -15,6 +15,9 @@ params:
```{r, child="children/SETTINGS-knitr.txt"}
```

```{r, child="children/SEE-ONLINE.txt", eval = if (isTRUE(exists("params"))) !params$EVAL else TRUE}
```

# Introduction

This vignette demonstrates the new functionality in __loo__ v2.0.0 for
Expand Down Expand Up @@ -273,11 +276,11 @@ a comparison to models without these terms.
fit11 <- update(fit10, formula = total_tools ~ log_pop + contact_high)
fit12 <- update(fit10, formula = total_tools ~ log_pop)
```
```{r}
```{r loo-contact_high}
(loo11 <- loo(fit11))
(loo12 <- loo(fit12))
```
```{r}
```{r relo-contact_high}
loo11 <- loo(fit11, k_threshold=0.7)
loo12 <- loo(fit12, k_threshold=0.7)
lpd_point <- cbind(
Expand Down
3 changes: 3 additions & 0 deletions vignettes/loo2-with-rstan.Rmd
Original file line number Diff line number Diff line change
Expand Up @@ -13,6 +13,9 @@ params:
%\VignetteIndexEntry{Writing Stan programs for use with the loo package}
-->

```{r, child="children/SEE-ONLINE.txt", eval = if (isTRUE(exists("params"))) !params$EVAL else TRUE}
```

# Introduction

This vignette demonstrates how to write a Stan program that computes and stores
Expand Down

0 comments on commit 28d0c0e

Please sign in to comment.