Skip to content

ArviZ 2021 roadmap

Jump to bottom
Aki Vehtari edited this page Jun 2, 2021 · 17 revisions

Roadmap topics for 2021

The development pace of ArviZ is contingent on external funding opportunities, so this roadmap does not mean to imply any specific dates or deadlines. Listed below are the major areas of work the team of core contributors considers important. These are tasks and projects that we would like to do ourselves, when time and resources permit, or that we would be happy to have done by new contributors.

TFP

  • Clean up TFP interface for current version and TensorFlow 2
  • Provide a "default trace function" that will extract the metrics arviz expects
  • Provide example of producing a fully-featured InferenceData object

Plots

  1. half-eye, dots, see ggdist https://mjskay.github.io/ggdist/
    • Better than violin, smaller, and there are better variations
  2. Dot plots
    • Useful if people need to infer talk probabilities
  3. Calibration plot for classification, see. e.g. https://avehtari.github.io/modelselection/diabetes.html
    • Multiclass would be like a pair plots
    • the example mentioned used equispaced bins, but it might be useful option to allow bins that have some minimum number of observations from both classes (I (Aki) did some experiments on this)
  4. ecdf / ecdf-difference with correct envelopes (more info soon in hopefully Jan)
    • loo-pit has further issues with ecdf
    • Can be used in convergence diagnostics rater than rank plot
  5. Regression plots: hdi, spaghetti plots, multiple outcome plot animations
  6. Add a helper function to easily stack the chain and draws dimensions into a sample dimension, instead of having to do idata.posterior.stack(sample=("chain", "draw")). Often useful when you don't care about which chain a draw is coming from. Issue to track this goal here.

Public API Stability

  • Duplicity of plot_dist and plot_kde
  • plot_hdi draws and samples conversion
  • plot_joint deprecation
  • Plotting API is not good, input and output still a mess
    • Inconsistency in changing things in plots
    • Are input arguments are all the same

Lots of chain plots

  • TFP is creating more chains than samples in some cases
  • Could help with simulation based calibration
  • Rhat computation changes based on short chains versus long chains

Refitting

  • reloo + iwmm? (Importance weighted moment matching in R Loo package and supported by BRMS)
    • Can improve results compared to psis loo with less computation time, doesn't always work
    • Can save time for people, but requires that were using the model again
    • https://arxiv.org/abs/1906.08850
  • LFO-CV

ArviZ dashboard

Explore interactive and online plots for interactive prior elicitation, diagnose runs on the fly... See also https://github.com/evdoxiataka/ipme See proof of concept as a gist.

Julia

  • Getting converter for gen.jl
  • Experimental Plots.jl and/or Makie.jl recipe implementations for plots (see ArviZ.jl#108)
    • Requires reimplementing plot logic in Julia. Including primitives (like rug plot, interval plot, etc), subplots (like KDE plot), and plots (multiple subplots from one or more Datasets)
    • Adopt a more Julian syntax for configuration/visualization options
    • Maintain in parallel with wrappers to call Python ArviZ's plotting functions, so no functionality is lost

Proj pred R Package

  • Determine where this would go in python package world (this seems like a work for Bambi)

Generic Plotting Backend

  • Let people bring their own backend and let ArviZ do all the hard math stuff for them

DEI Outreach

  • Make sure we do a couple DEI
    • Use NumFOCUS money

Exploratory Analysis Repo

  • Held up on paying people to finish that. Otherwise its a big opportunity cost

Inference Data to and from R

  • A way to transfer 1 to 1 mapping from ArviZ to posterior and back
  • Need to discuss with posterior devs to see what the best way would be to do this
  • https://mc-stan.org/posterior/

Social

Funds

  • NASA Roses Grant
  • Can come up with more precise
  • Could pay for developers and DEI

ArviZCon? ArviZ social event?

  • Assume its going to be Finland

Google Summer of Code

  • Definitely

Google Summer of Docs

  • Will for this one

Onboard contributors

  • 2 or 3 new insular
    • Another jl dev if we do generic plotting backend
Clone this wiki locally