docs: add docs for recent feature additions (#957)

docs/multiple_output_cache.md

@@ -0,0 +1,70 @@
+# The cache for multiple outputs
+
+!!!note
+
+    The "multi-output" cache is a little bit different from a compilation cache. If you look for tips and tricks on how to use `sccache` or `ccache` with `rattler-build`, please refer to the [tips and tricks section](tips_and_tricks.md#using-sccache-or-ccache).
+
+Sometimes you build a package and want to split the contents into multiple sub-packages.
+For example, when building a C/C++ package, you might want to create multiple packages for the
+runtime requirements (library), and the development time requirements such as header files.
+
+The "cache" output makes this easy. It allows you to specify a single top-level cache that can produce arbitrary
+files, that can then be used in other packages.
+
+Let's take a look at an example:
+
+```yaml title="recipe.yaml"
+recipe:
+  name: mypackage
+  version: '0.1.0'
+
+cache:
+  requirements:
+    build:
+      - ${{ compiler('c') }}
+  build:
+    script:
+      - mkdir -p $PREFIX/lib
+      - mkdir -p $PREFIX/include
+      - echo "This is the library" > lib/library.txt
+      - echo "This is the header" > include/header.txt
+
+outputs:
+  - package:
+      name: mypackage-library
+    build:
+      files:
+        - lib/*
+
+  - package:
+      name: mypackage-headers
+    build:
+      files:
+        - include/*
+```
+
+!!!note
+
+    Since this is an experimental feature, you need to pass the `--experimental` flag to enable parsing of the `cache` top-level section.
+
+In this example, we have a single package called `mypackage` that creates two outputs: `mypackage-library` and `mypackage-headers`.
+The cache output will run like a regular output, but after the build is finished, the files will be copied to a "cache" directory (in your output folder, under `output/build_cache`).
+
+The files in the cache folder are then copied into the `$PREFIX` of each output package. Since they are "new" files in the prefix, they will be included in the output package.
+The easiest way to select a subset of the files in the prefix is by using the `files` field in the output definition.
+You can use a list of globs to select only the files that you want.
+
+For something more complicated you can also use `include` and `exclude` fields in the `files` selector. Please refer to the [the build options documentation](build_options.md#include-only-certain-files-in-the-package).
+
+## Run exports from the cache
+
+Since the cache output also has build- and host requirements we need to additionally take care of eventual "run-exports" from the cache output.
+Run exports from the cache-dependencies are handled very similar to the run exports from a given output. We append any run exports to the outputs.
+
+If the cache has an "ignore run exports" section, than we apply those filters at the cache level. If the output ignores any run exports, then we also ignore the run-exports if they would come from the cache.
+
+## Caching in the $SRC_DIR
+
+If you used `conda-build` a lot, you might have noticed that a top-level build is also caching the changes in the `$SRC_DIR`. This is not the case for `rattler-build` yet.
+
+You could try to work around by e.g. copying files into the `$PREFIX` and restoring them in each output.

docs/recipe_generation.md

@@ -0,0 +1,129 @@
+# Generating recipes for different ecosystems
+
+Rattler-build has some builtin functionality to generate recipes for different (existing) ecosystems.
+
+Currently we support the following ecosystems:
+
+- `pypi` (Python) - generates a recipe for a Python package
+- `cran` (R) - generates a recipe for an R package
+
+To generate a recipe for a Python package, you can use the following command:
+
+```sh
+rattler-build generate-recipe pypi jinja2
+```
+
+This will generate a recipe for the `jinja2` package from PyPI and print it to the console. To turn it into a recipe, you can either pipe the stdout to a file or use the `-w` flag. The `-w` flag will create a new folder with the recipe in it.
+
+The generated recipe for `jinja2` will look something like:
+
+```yaml title="recipe.yaml"
+package:
+  name: jinja2
+  version: 3.1.4
+
+source:
+- url: https://files.pythonhosted.org/packages/ed/55/39036716d19cab0747a5020fc7e907f362fbf48c984b14e62127f7e68e5d/jinja2-3.1.4.tar.gz
+  sha256: 4a3aee7acbbe7303aede8e9648d13b8bf88a429282aa6122a993f0ac800cb369
+
+build:
+  script: python -m pip install .
+
+requirements:
+  host:
+  - flit_core <4
+  - python >=3.7
+  - pip
+  run:
+  - python >=3.7
+  - markupsafe >=2.0
+  # - babel >=2.7  # extra == 'i18n'
+
+tests: []
+
+about:
+  summary: A very fast and expressive template engine.
+  documentation: https://jinja.palletsprojects.com/
+```
+
+## Generating recipes for R packages
+
+To generate a recipe for an R package, you can use the following command:
+
+```sh
+rattler-build generate-recipe cran dplyr
+```
+
+The `R` recipe generation supports some additional flags:
+
+- `-u/--universe` select an R universe to use (e.g. `bioconductor`)
+- `-t/--tree` generate multiple recipes, for every dependency as well
+
+R packages will be prefixed with `r-` to avoid name conflicts with Python packages. The generated recipe for `dplyr` will look something like:
+
+```yaml title="recipe.yaml"
+package:
+  name: r-dplyr
+  version: 1.1.4
+
+source:
+- url: https://cran.r-project.org/src/contrib/dplyr_1.1.4.tar.gz
+  md5: e3066ea859b26e0d3b992c476ea3af2e
+
+build:
+  script: R CMD INSTALL --build .
+  python: {}
+
+requirements:
+  host:
+  - r-base >=3.5.0
+  run:
+  - r-cli >=3.4.0
+  - r-generics
+  - r-glue >=1.3.2
+  - r-lifecycle >=1.0.3
+  - r-magrittr >=1.5
+  - r-methods
+  - r-pillar >=1.9.0
+  - r-r6
+  - r-rlang >=1.1.0
+  - r-tibble >=3.2.0
+  - r-tidyselect >=1.2.0
+  - r-utils
+  - r-vctrs >=0.6.4
+  # -  r-bench  # suggested
+  # -  r-broom  # suggested
+  # -  r-callr  # suggested
+  # -  r-covr  # suggested
+  # -  r-dbi  # suggested
+  # -  r-dbplyr >=2.2.1  # suggested
+  # -  r-ggplot2  # suggested
+  # -  r-knitr  # suggested
+  # -  r-lahman  # suggested
+  # -  r-lobstr  # suggested
+  # -  r-microbenchmark  # suggested
+  # -  r-nycflights13  # suggested
+  # -  r-purrr  # suggested
+  # -  r-rmarkdown  # suggested
+  # -  r-rmysql  # suggested
+  # -  r-rpostgresql  # suggested
+  # -  r-rsqlite  # suggested
+  # -  r-stringi >=1.7.6  # suggested
+  # -  r-testthat >=3.1.5  # suggested
+  # -  r-tidyr >=1.3.0  # suggested
+  # -  r-withr  # suggested
+
+about:
+  homepage: https://dplyr.tidyverse.org, https://github.com/tidyverse/dplyr
+  summary: A Grammar of Data Manipulation
+  description: |-
+    A fast, consistent tool for working with data frame like
+    objects, both in memory and out of memory.
+  license: MIT
+  license_file: LICENSE
+  repository: https://github.com/cran/dplyr
+```
+
+!!!tip
+
+    You can use the generated recipes to build your own "forge" with `rattler-build`. Read more about it in the [Building your own forge](./tips_and_tricks.md#building-your-own-forge) section.

docs/reference/cli.md

@@ -574,27 +574,77 @@ Generate shell completion script
 
 Generate a recipe from PyPI or CRAN
 
-**Usage:** `rattler-build generate-recipe <SOURCE> <PACKAGE>`
+**Usage:** `rattler-build generate-recipe <COMMAND>`
+
+##### **Subcommands:**
+
+* `pypi` — Generate a recipe for a Python package from PyPI
+* `cran` — Generate a recipe for an R package from CRAN
+
+
+
+#### `pypi`
+
+Generate a recipe for a Python package from PyPI
+
+**Usage:** `rattler-build generate-recipe pypi [OPTIONS] <PACKAGE>`
 
 ##### **Arguments:**
 
-- `<SOURCE>`
+- `<PACKAGE>`
+
+	Name of the package to generate
+
+
+
+##### **Options:**
+
+- `-w`, `--write`
+
+	Whether to write the recipe to a folder
+
+	- Possible values: `true`, `false`
+
 
-	Type of package to generate a recipe for
 
-	- Possible values:
-		- `pypi`:
-			Generate a recipe for a Python package from PyPI
-		- `cran`:
-			Generate a recipe for an R package from CRAN
 
 
+#### `cran`
+
+Generate a recipe for an R package from CRAN
+
+**Usage:** `rattler-build generate-recipe cran [OPTIONS] <PACKAGE>`
+
+##### **Arguments:**
+
 - `<PACKAGE>`
 
 	Name of the package to generate
 
 
 
+##### **Options:**
+
+- `-u`, `--universe <UNIVERSE>`
+
+	The R Universe to fetch the package from (defaults to `cran`)
+
+
+- `-t`, `--tree`
+
+	Wether to create recipes for the whole dependency tree or not
+
+	- Possible values: `true`, `false`
+
+
+- `-w`, `--write`
+
+	Whether to write the recipe to a folder
+
+	- Possible values: `true`, `false`
+
+
+
 
 
 ### `auth`