Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add use_memoise helper to memoise chull fct on the fly #80

Merged
merged 29 commits into from
Apr 3, 2024
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
29 commits
Select commit Hold shift + click to select a range
ca52076
Add use_memoise helper to memoise chull fct on the fly
Bisaloo Nov 17, 2023
b6f87e4
Memoise once but dispatch on the fly
Bisaloo Dec 9, 2023
a31c964
Also memoise first computation
Mar 19, 2024
e47fc0c
Add basic tests for use_memoise()
Mar 19, 2024
ec60d07
Slightly better error message
Mar 19, 2024
2b7bc37
Test for non sequential plan
Mar 19, 2024
eb169c1
Add 'withr' as Suggests
Mar 19, 2024
d8bc50f
Fix tests for fd_chull() and fd_chull_intersect()
Mar 19, 2024
a82b8ca
Remove global memoise option in tests
Mar 19, 2024
3e8219b
Memoise fd_chull_intersect() as well
Mar 19, 2024
5d902a9
Update function documentation of memoisation
Mar 19, 2024
2535ec2
Update vignettes to explain memoisation
Mar 20, 2024
d56a970
Made internal function name more explicit than 'f'
Mar 20, 2024
1475a3e
Warn memoise with caution (fixes #71 #77)
Mar 20, 2024
21a1f39
Skip line
Mar 20, 2024
33509b8
Add tests for unmemoised versions
Mar 20, 2024
61c4576
Merged tests
Mar 20, 2024
15ab2bc
Document fundiversity.memoise option via roxygen2
Bisaloo Dec 9, 2023
899ab3e
Add test to ensure use_memoise() really triggers memoise
Bisaloo Mar 22, 2024
7c9ea13
Update R/fd_fric_intersect.R
Mar 22, 2024
4b89657
Update R/use_memoise.R
Mar 22, 2024
e1976ee
Add comment on exists()
Mar 22, 2024
6f4160d
Merge branch 'memoise-on-the-fly' of https://github.com/funecology/fu…
Mar 22, 2024
2e4a8a1
Fix pkgdown
Bisaloo Mar 25, 2024
82b297a
Add warning in README
Mar 25, 2024
b42284e
Update Rmd workflow
Mar 26, 2024
114e95e
Render README
Mar 26, 2024
246c196
Update doc on memoization vs. parallelization
Mar 26, 2024
fffbaab
Correct typo
Mar 26, 2024
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
33 changes: 0 additions & 33 deletions .github/workflows/render-readme.yaml

This file was deleted.

34 changes: 34 additions & 0 deletions .github/workflows/render-rmarkdown.yaml
Original file line number Diff line number Diff line change
@@ -0,0 +1,34 @@
# Workflow derived from https://github.com/r-lib/actions/tree/v2/examples
# Need help debugging build failures? Start at https://github.com/r-lib/actions#where-to-find-help
on:
push:
paths:
- README.Rmd

name: Render README

jobs:
render-rmarkdown:
runs-on: ubuntu-latest
env:
GITHUB_PAT: ${{ secrets.GITHUB_TOKEN }}
steps:
- name: Checkout repo
uses: actions/checkout@v4
with:
fetch-depth: 0

- uses: r-lib/actions/setup-pandoc@v2

- uses: r-lib/actions/setup-r@v2

- uses: r-lib/actions/setup-renv@v2

- name: Render Rmarkdown files and Commit Results
run: |
RMD_PATH=($(git diff --name-only ${{ github.event.before }} ${{ github.sha }} | grep '[.]Rmd$'))
Rscript -e 'for (f in commandArgs(TRUE)) if (file.exists(f)) rmarkdown::render(f)' ${RMD_PATH[*]}
git config --local user.name "$GITHUB_ACTOR"
git config --local user.email "[email protected]"
git commit ${RMD_PATH[*]/.Rmd/.md} -m 'Re-build Rmarkdown files' || echo "No changes to commit"
git push origin || echo "No changes to commit"
5 changes: 3 additions & 2 deletions DESCRIPTION
Original file line number Diff line number Diff line change
Expand Up @@ -23,7 +23,7 @@ Language: en
Encoding: UTF-8
LazyData: true
Roxygen: list(markdown = TRUE)
RoxygenNote: 7.2.3
RoxygenNote: 7.3.1
Depends:
R (>= 2.10)
Imports:
Expand All @@ -36,7 +36,8 @@ Suggests:
knitr,
memoise,
rmarkdown,
testthat (>= 3.0.0)
testthat (>= 3.0.0),
withr
URL: https://funecology.github.io/fundiversity/, https://github.com/funecology/fundiversity
BugReports: https://github.com/funecology/fundiversity/issues
Config/testthat/edition: 3
Expand Down
8 changes: 7 additions & 1 deletion R/fd_fdiv.R
Original file line number Diff line number Diff line change
Expand Up @@ -75,6 +75,12 @@ fd_fdiv <- function(traits, sp_com) {
site_abundances[site_abundances == 0] <- 1 # Account for site with no species
sp_com <- sp_com / site_abundances

convex_hull <- if (use_memoise()) {
fd_chull_memoised
} else {
fd_chull
}

# Compute Functional Divergence
fdiv_site <- future_apply(sp_com, 1, function(sp_site) {

Expand All @@ -88,7 +94,7 @@ fd_fdiv <- function(traits, sp_com) {
# Select traits for species actually in site
sub_traits <- traits[names(sub_site),, drop = FALSE]

ch <- fd_chull(sub_traits)
ch <- convex_hull(sub_traits)

verts <- ch$p[unique(c(ch$hull)),, drop = FALSE]

Expand Down
23 changes: 18 additions & 5 deletions R/fd_fric.R
Original file line number Diff line number Diff line change
Expand Up @@ -25,7 +25,7 @@
#' The computation of this function can be parallelized thanks to
#' [future::plan()]. To get more information on how to parallelize your
#' computation please refer to the parallelization vignette with:
#' `vignette("fundiversity_1-parallel", package = "fundiversity")`
#' `vignette("fundiversity_1-parallel", package = "fundiversity")`.
#'
#' @examples
#' data(traits_birds)
Expand All @@ -34,11 +34,17 @@
#' @details By default, when loading \pkg{fundiversity}, the functions to
#' compute convex hulls are
#' [memoised](https://en.wikipedia.org/wiki/Memoization) through the `memoise`
#' package if it is installed. To deactivate this behavior you can set the
#' package if it is installed (their results are cached to avoid recomputing the
#' same functional volume twice). To deactivate this behavior you can set the
#' option `fundiversity.memoise` to `FALSE` by running the following line:
#' `options(fundiversity.memoise = FALSE)`. If you use it interactively it will
#' only affect your current session. Add it to your script(s) or `.Rprofile`
#' file to avoid toggling it each time.
#' file to avoid toggling it each time. By changing the option, the behavior
#' will automatically change the next time you run the function. **Note**:
#' memoisation is only available when the `memoise` package has been installed
#' **and without parallelization**, otherwise `fundiversity` will use unmemoised
#' versions of the functions. In other words, **memoization and parallelization
#' are mutually exclusive**.
#'
#' @return a data.frame with two columns:
#' * `site` the names of the sites as the row names of the input `sp_com`,
Expand Down Expand Up @@ -105,12 +111,19 @@ fd_fric <- function(traits, sp_com, stand = FALSE) {

max_range <- 1

convex_hull <- if (use_memoise()) {
fd_chull_memoised
} else {
fd_chull
}

if (stand) {
max_range <- fd_chull(traits)$vol
max_range <- convex_hull(traits)$vol
}

fric_site <- future_apply(sp_com, 1, function(site_row) {
fd_chull(traits[site_row > 0,, drop = FALSE])$vol
res <- convex_hull(traits[site_row > 0, , drop = FALSE])
return(res$vol)
}, future.globals = FALSE)

if (any(is.na(fric_site))) {
Expand Down
20 changes: 16 additions & 4 deletions R/fd_fric_intersect.R
Original file line number Diff line number Diff line change
Expand Up @@ -84,8 +84,20 @@ fd_fric_intersect <- function(traits, sp_com, stand = FALSE) {

max_range <- 1

convex_hull <- if (use_memoise()) {
fd_chull_memoised
} else {
fd_chull
}

convex_hull_intersect <- if (use_memoise()) {
fd_chull_intersect_memoised
} else {
fd_chull_intersect
}

if (stand) {
max_range <- fd_chull(traits)$vol
max_range <- convex_hull(traits)$vol
}

# All pairs of sites (not within themselves)
Expand Down Expand Up @@ -113,11 +125,11 @@ fd_fric_intersect <- function(traits, sp_com, stand = FALSE) {
second_row <- sp_com[site_comb[[2]],, drop = TRUE]
second_traits <- traits[second_row > 0,, drop = FALSE]

fd_chull_intersect(first_traits, second_traits)$vol
convex_hull_intersect(first_traits, second_traits)$vol
} else {
# Self-intersection (equivalent to regular convex hulls)
# way more efficient that compute with fd_chull_inters
fd_chull(first_traits)$vol
# way more efficient than computing with convex_hull_intersect()
convex_hull(first_traits)$vol
}
}, future.globals = FALSE)

Expand Down
46 changes: 46 additions & 0 deletions R/use_memoise.R
Original file line number Diff line number Diff line change
@@ -0,0 +1,46 @@
#' Options for \pkg{fundiversity}
#'
#' The memoisation is the convex hull computation in \pkg{fundiversity} is
#' controlled via the `fundiversity.memoise` option:
#' - if unset, the default is to use memoisation if \pkg{memoise} was installed
#' when \pkg{fundiversity} was loaded, and not to use memoisation otherwise.
#' - if `options(fundiversity.memoise = TRUE)`, memoisation is used and an error
#' is thrown if \pkg{memoise} is not installed.
#' - if `options(fundiversity.memoise = FALSE)`, memoisation is not used.
#'
#' @name fundiversity-options
NULL

#' @keywords internal
use_memoise <- function() {

# Cannot use memoise in parallel settings
if (!inherits(future::plan(), "sequential")) {
return(FALSE)
}
Comment on lines +18 to +20
Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Do we need to warn users here?

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yes we should!
A warning or a message?

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It should be a message, but then the risk is because this is triggered each time use_memoise() is used. We wouldn't want for the user to be swamped by such messages when running in parallel.
Do you know a way to deal with that in base R?
I don't want to add more dependencies to fundiversity...

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

My first intuition would be to have a logical variable warned_future_memoise in an internal environment that starts at FALSE and then is set to TRUE once the message has been printed once.

It seems like overengineering a little bit though and I'm not completely sure how it would work in parallel (is the package attached each time in each new worker?) 🤔

Maybe we can just document it very clearly somewhere and leave it as is. It's probably not an actionable piece of info for the users anyways.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'll document as thoroughly as possible everywhere I find it useful.


# explicitly set to TRUE by user
if (isTRUE(getOption("fundiversity.memoise"))) {
if (exists("fd_chull_memoised")) {
return(TRUE)
}
stop(
"memoise is not installed ",
"or was installed after fundiversity was loaded. ",
"Please install memoise and restart R.",
call. = FALSE
)
}
# explicitly set to FALSE by user
if (isFALSE(getOption("fundiversity.memoise"))) {
return(FALSE)
}
# unspecified / default
# TRUE or FALSE depending on whether memoise was installed when fundiversity
# was loaded
return(exists("fd_chull_memoised"))
}

# Added this to make 'testthat::local_mocked_bindings()' work
# in 'test-use_memoise.R'
exists <- NULL
Rekyt marked this conversation as resolved.
Show resolved Hide resolved
10 changes: 3 additions & 7 deletions R/zzz.R
Original file line number Diff line number Diff line change
@@ -1,10 +1,6 @@
# Diverse utility functions

# Memoized version of fd_chull loaded if package is installed
.onLoad <- function(libname, pkgname) {
if (requireNamespace("memoise", quietly = TRUE) &&
isTRUE(getOption("fundiversity.memoise", TRUE))) {
fd_chull <<- memoise::memoise(fd_chull)
fd_chull_intersect <<- memoise::memoise(fd_chull_intersect)
if (requireNamespace("memoise", quietly = TRUE)) {
fd_chull_memoised <<- memoise::memoise(fd_chull)
fd_chull_intersect_memoised <<- memoise::memoise(fd_chull_intersect)
}
}
3 changes: 2 additions & 1 deletion README.Rmd
Original file line number Diff line number Diff line change
Expand Up @@ -101,7 +101,8 @@ future::plan(future::multisession)
fd_fdiv(traits_birds)
```

For more details please refer to the [parallelization vignette](https://funecology.github.io/fundiversity/articles/fundiversity_1-parallel.html) or use `vignette("fundiversity_1-parallel", package = "fundiversity")` within R.
For more details please refer to the [parallelization vignette](https://funecology.github.io/fundiversity/articles/fundiversity_1-parallel.html) or use `vignette("fundiversity_1-parallel", package = "fundiversity")` within R. **Note**: parallelization and memoization are **mutually exclusive**, when doing computation in parallel, fundiversity falls back on **unmemoised** versions of function.


## Available functional diversity indices

Expand Down
8 changes: 5 additions & 3 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -112,7 +112,9 @@ fd_fdiv(traits_birds)
For more details please refer to the [parallelization
vignette](https://funecology.github.io/fundiversity/articles/fundiversity_1-parallel.html)
or use `vignette("fundiversity_1-parallel", package = "fundiversity")`
within R.
within R. **Note**: parallelization and memoization are **mutually
exclusive**, when doing computation in parallel, fundiversity falls back
on **unmemoised** versions of function.

## Available functional diversity indices

Expand Down Expand Up @@ -140,14 +142,14 @@ mention the numerous wrappers around these packages):

| Package Name | Indices included | Has vignettes | Has tests | On GitHub | On CRAN (last updated) |
|--------------------------------------------------------|---------------------------------------------------------------------------------------------------------------------|---------------|-----------|-----------|------------------------------------------------------------|
| [`adiv`](https://github.com/cran/adiv) | Functional Entropy, Functional Redundancy | | ❌ | ❌ | ![](https://www.r-pkg.org/badges/last-release/adiv) |
| [`adiv`](https://github.com/cran/adiv) | Functional Entropy, Functional Redundancy | | ❌ | ❌ | ![](https://www.r-pkg.org/badges/last-release/adiv) |
| [`BAT`](https://github.com/cardosopmb/BAT) | β-diversity indices, Richness, divergence, and evenness with hypervolumes | ❌ | ❌ | ✅ | ![](https://www.r-pkg.org/badges/last-release/BAT) |
| [`betapart`](https://github.com/cran/betapart) | Functional β-diversity | ❌ | ❌ | ❌ | ![](https://www.r-pkg.org/badges/last-release/betapart) |
| [`entropart`](https://github.com/EricMarcon/entropart) | Functional Entropy | ✅ | ✅ | ✅ | ![](https://www.r-pkg.org/badges/last-release/entropart) |
| [`FD`](https://github.com/cran/FD) | FRic, FDiv, FDis, FEve, Rao’s QE, Functional Group Richness | ❌ | ❌ | ❌ | ![](https://www.r-pkg.org/badges/last-release/FD) |
| [`hilldiv`](https://github.com/anttonalberdi/hilldiv) | Dendrogram-based Hill numbers for functional diversity | ❌ | ❌ | ✅ | ![](https://www.r-pkg.org/badges/last-release/hilldiv) |
| [`hillR`](https://github.com/daijiang/hillR) | Functional Diversity Hill Numbers | ❌ | ✅ | ✅ | ![](https://www.r-pkg.org/badges/last-release/hillR) |
| [`hypervolume`](https://github.com/cran/hypervolume) | Hypervolume measure of functional diversity (\~FRic) | ✅ | ❌ | ✅ | ![](https://www.r-pkg.org/badges/last-release/hypervolume) |
| [`hypervolume`](https://github.com/cran/hypervolume) | Hypervolume measure of functional diversity (~FRic) | ✅ | ❌ | ✅ | ![](https://www.r-pkg.org/badges/last-release/hypervolume) |
| [`mFD`](https://github.com/CmlMagneville/mFD) | Functional α- and β-diversity indices, including FRic, FDiv, FDis, FEve, FIde, FMPD, FNND, FOri, FSpe, Hill Numbers | ✅ | ❌ | ✅ | ![](https://www.r-pkg.org/badges/last-release/mFD) |
| [`TPD`](https://github.com/cran/TPD) | FRic, FDiv, FEve but for probability distributions | ✅ | ❌ | ❌ | ![](https://www.r-pkg.org/badges/last-release/TPD) |
| [`vegan`](https://github.com/vegandevs/vegan) | Only dendrogram-based FD (`treedive()`) | ✅ | ✅ | ✅ | ![](https://www.r-pkg.org/badges/last-release/vegan) |
Expand Down
2 changes: 1 addition & 1 deletion man/fd_fdis.Rd

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

12 changes: 9 additions & 3 deletions man/fd_fdiv.Rd

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

2 changes: 1 addition & 1 deletion man/fd_feve.Rd

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

12 changes: 9 additions & 3 deletions man/fd_fric.Rd

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

12 changes: 9 additions & 3 deletions man/fd_fric_intersect.Rd

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

Loading
Loading