Generalised unary vector function framework #1558

andrjohns · 2019-12-29T07:51:09Z

Summary

This pull introduces a framework for generalising unary vector functions (e.g. log_sum_exp or log_softmax) to work with Eigen column and row vectors, std::vectors, and containers of these.

The motivations behind the framework were discussed in #1425, and the design document for the framework is here. There has also been additional discussion over in the forums.

A new vectorisation framework, apply_vector_unary, is proposed. The framework has two functions to address the different types of vector functions:

apply_vector_unary<T>::apply() for: f(vector) -> vector
apply_vector_unary<T>::reduce() for: f(vector) -> scalar

This pull also introduces an example use of each type of this vectorisation:

log_softmax (apply())
log_sum_exp (reduce())

For each of these functions, their prim, rev, and fwd definitions have been re-written in the new framework, and their associated tests expanded.

Tests

Both prim and mix tests have been added to check that the same values and derivatives are returned across the new vector inputs.

Side Effects

I had to update the templating in one header from Eigen::Matrix<T, R, C> to Eigen::MatrixBase<Derived> so that they could work with Eigen::Map inputs:

matrix_vari

Checklist

Math issue Use Eigen::Map to replace arr functions #1425
Copyright holder: Andrew Johnson

The copyright holder is typically you or your assignee, such as a university or company. By submitting this pull request, the copyright holder is agreeing to the license the submitted work under the following licenses:
- Code: BSD 3-clause (https://opensource.org/licenses/BSD-3-Clause)
- Documentation: CC-BY 4.0 (https://creativecommons.org/licenses/by/4.0/)
the basic tests are passing
- unit tests pass (to run, use: ./runTests.py test/unit)
- header checks pass, (make test-headers)
- docs build, (make doxygen)
- code passes the built in C++ standards checks (make cpplint)
the code is written in idiomatic C++ and changes are documented in the doxygen
the new changes are tested

…gs/RELEASE_500/final)

andrjohns · 2019-12-31T08:15:14Z

I've removed the apply_scalar and head implementations, since they don't line up with the expected functionality of head in the integration tests. Under apply_scalar, when passing head a container of vectors (e.g. head(std::vector<VectorXd>, 2)), the framework would apply head to each of those vectors in the container. However, the integration tests expect that head(std::vector<VectorXd>, 2) would return the first two vectors in the container (in full).

This was me misunderstanding the implementation of head and so I've not touched it in this PR.

stan-buildbot · 2019-12-31T20:28:23Z

(stat_comp_benchmarks/benchmarks/gp_pois_regr/gp_pois_regr.stan, 0.99)
(stat_comp_benchmarks/benchmarks/low_dim_corr_gauss/low_dim_corr_gauss.stan, 0.95)
(stat_comp_benchmarks/benchmarks/irt_2pl/irt_2pl.stan, 1.01)
(stat_comp_benchmarks/benchmarks/pkpd/one_comp_mm_elim_abs.stan, 0.93)
(stat_comp_benchmarks/benchmarks/eight_schools/eight_schools.stan, 0.97)
(stat_comp_benchmarks/benchmarks/gp_regr/gp_regr.stan, 1.0)
(stat_comp_benchmarks/benchmarks/arK/arK.stan, 1.0)
(performance.compilation, 1.03)
(stat_comp_benchmarks/benchmarks/low_dim_gauss_mix_collapse/low_dim_gauss_mix_collapse.stan, 1.0)
(stat_comp_benchmarks/benchmarks/low_dim_gauss_mix/low_dim_gauss_mix.stan, 1.0)
(stat_comp_benchmarks/benchmarks/sir/sir.stan, 1.07)
(stat_comp_benchmarks/benchmarks/pkpd/sim_one_comp_mm_elim_abs.stan, 1.01)
(stat_comp_benchmarks/benchmarks/garch/garch.stan, 1.01)
(stat_comp_benchmarks/benchmarks/gp_regr/gen_gp_data.stan, 0.97)
(stat_comp_benchmarks/benchmarks/arma/arma.stan, 1.0)
Result: 0.99535221307
Commit hash: 8ebea40

stan/math/prim/mat/vectorize/apply_vector_unary.hpp

stan/math/rev/core/matrix_vari.hpp

stan/math/fwd/mat/fun/log_sum_exp.hpp

test/unit/math/mix/mat/fun/log_softmax_test.cpp

stan/math/rev/mat/fun/log_sum_exp.hpp

t4c1 · 2020-01-03T12:16:55Z

stan/math/fwd/mat/fun/log_softmax.hpp

+ */
+template <typename T, require_t<is_fvar<scalar_type_t<T>>>...>
+inline auto log_softmax(T&& x) {
+  return apply_vector_unary<T>::apply(std::forward<T>(x), [&](auto& alpha) {


Perfect forwarding brings no benefits here. This would work as well with no additional copys and the code is a bit shorter:

inline auto log_softmax(const T& x) { return apply_vector_unary<T>::apply(x, [&](auto& alpha) {

I think the same holds true for every other place in this PR that uses perfect forwarding.

Perfect forwarding is better than just const refs only if a variable can outlive the function in question. For example if the function can directley return the variable or construct some object that holds the variable.

Perfect forwarding is better than just const refs only if a variable can outlive the function in question. For example if the function can directley return the variable or construct some object that holds the variable.

Can you link me to a doc that talks about that? My understanding of PF is that it tells the compiler "I'm okay with moving ownership of this memory and leaving the original object in an undefined state"

In particular, for the below think about a recursive function that can have a long ref stack that the compiler probably can't sort through. PF gives the compiler the ability to move ownership of that memory. So we don't have to worry about whether the compiler will have alias issues when deciding to inline recurses.

For example if the function can directly return the variable or construct some object that holds the variable.

The CPP guides have a lot on this with a nice little table in F.15 has some good heuristics

https://isocpp.github.io/CppCoreGuidelines/CppCoreGuidelines#fcall-parameter-passing

In particular this seems to recommend Andrew keep it as a forwarding reference

https://isocpp.github.io/CppCoreGuidelines/CppCoreGuidelines#f19-for-forward-parameters-pass-by-tp-and-only-stdforward-the-parameter

It feels like the above func is just something we want to allow the compiler to move through if it's able

Can you link me to a doc that talks about that?

Sorry, that is just my understanding and I a quick search does not find anything on when perfect forwarding is benefitial and when it is not. I can't think of any other example, but please correct me if I am wrong.

My understanding of PF is that it tells the compiler "I'm okay with moving ownership of this memory and leaving the original object in an undefined state"

I would say more accurate would be "I am CAPABLE of either moving a temporary object or using a reference to lvalue object." In this PR no function can actually move the parameter, so const refs are just as good.

PF gives the compiler the ability to move ownership of that memory. So we don't have to worry about whether the compiler will have alias issues when deciding to inline recurses.

I don't know anything about that. Can you link any doc or benchmark that supports this?

Thanks for the links to the guidelines. However I disagree with your conclusion that they suggest we should use perfect forwarding. The first link points to section that says simple means of parameter passing (which exclude perfect forwarding) should be used, unless benefits of something more advanced are demonstrated.

The second link suggests nothing on whether perfect forwarding should be used or not. It only explains what a function accepting a forwarding reference should do with it. It says that a function should be flagged (as violatin those guidelines) if it does anything with this argument except std::forward-ing it exactley once.

I would add to this that std::forward-ing an argument to function is only beneficial if that function either uses perfect forwarding or has two overloads accepting const lvalue ref and rvalue ref.

I agree that we shouldn't clutter the code with perfect forwarding examples where they can't be used. This came up in another PR with autodiff types, which have no data content to move.

@t4c1 at work but ill link some stuff tonight

Maybe we should write up an rfc to talk about what types and sizes should use certain argument types. I tend to be way to pf giddy

t4c1

This looks great! There are a few places it can be improved a bit and a question or two.

rok-cesnovar · 2020-01-05T09:55:58Z

@andrjohns I will make a PR to your fork to fix these merge conflicts. This wont be completely trivial changes so I think a PR is needed. I missed that you are touching fwd files, else I would have hold off on the flattening.

EDIT: It was actually more or less straightforward, so I just pushed the changes. Hopefully that is fine.

…ter_flatten # Conflicts: # test/unit/math/prim/scal/fun/log_sum_exp_test.cpp

…ter_flatten

…ter_flatten # Conflicts: # stan/math/prim/mat.hpp

# Conflicts: # stan/math/fwd/arr.hpp # stan/math/fwd/mat/fun/log_sum_exp.hpp

stan-buildbot · 2020-01-05T17:52:19Z

(stat_comp_benchmarks/benchmarks/gp_pois_regr/gp_pois_regr.stan, 0.98)
(stat_comp_benchmarks/benchmarks/low_dim_corr_gauss/low_dim_corr_gauss.stan, 0.97)
(stat_comp_benchmarks/benchmarks/irt_2pl/irt_2pl.stan, 1.01)
(stat_comp_benchmarks/benchmarks/pkpd/one_comp_mm_elim_abs.stan, 0.99)
(stat_comp_benchmarks/benchmarks/eight_schools/eight_schools.stan, 1.02)
(stat_comp_benchmarks/benchmarks/gp_regr/gp_regr.stan, 1.0)
(stat_comp_benchmarks/benchmarks/arK/arK.stan, 1.0)
(performance.compilation, 1.02)
(stat_comp_benchmarks/benchmarks/low_dim_gauss_mix_collapse/low_dim_gauss_mix_collapse.stan, 1.0)
(stat_comp_benchmarks/benchmarks/low_dim_gauss_mix/low_dim_gauss_mix.stan, 1.0)
(stat_comp_benchmarks/benchmarks/sir/sir.stan, 1.0)
(stat_comp_benchmarks/benchmarks/pkpd/sim_one_comp_mm_elim_abs.stan, 0.95)
(stat_comp_benchmarks/benchmarks/garch/garch.stan, 1.0)
(stat_comp_benchmarks/benchmarks/gp_regr/gen_gp_data.stan, 1.01)
(stat_comp_benchmarks/benchmarks/arma/arma.stan, 0.96)
Result: 0.99359923753
Commit hash: 4e83afb

andrjohns · 2020-01-08T02:04:53Z

Thanks for sorting out those conflicts @rok-cesnovar!

@t4c1 I'll update this PR (and the tests) once #1471 gets merged in. Thanks!

…gs/RELEASE_500/final)

stan-buildbot · 2020-01-11T10:09:04Z

Name	Old Result	New Result	Ratio	Performance change( 1 - new / old )
gp_pois_regr/gp_pois_regr.stan	4.9	4.89	1.0	0.33% faster
low_dim_corr_gauss/low_dim_corr_gauss.stan	0.02	0.02	1.0	-0.25% slower
eight_schools/eight_schools.stan	0.09	0.09	1.0	-0.22% slower
gp_regr/gp_regr.stan	0.22	0.23	0.98	-1.59% slower
irt_2pl/irt_2pl.stan	6.07	6.14	0.99	-1.14% slower
performance.compilation	87.88	86.56	1.02	1.5% faster
low_dim_gauss_mix_collapse/low_dim_gauss_mix_collapse.stan	7.32	7.3	1.0	0.23% faster
pkpd/one_comp_mm_elim_abs.stan	20.86	21.5	0.97	-3.09% slower
sir/sir.stan	97.05	101.95	0.95	-5.04% slower
gp_regr/gen_gp_data.stan	0.04	0.05	0.98	-1.92% slower
low_dim_gauss_mix/low_dim_gauss_mix.stan	2.95	2.95	1.0	0.07% faster
pkpd/sim_one_comp_mm_elim_abs.stan	0.33	0.32	1.03	2.99% faster
arK/arK.stan	2.42	2.43	1.0	-0.35% slower
arma/arma.stan	0.79	0.79	1.0	-0.23% slower
garch/garch.stan	0.53	0.53	1.0	-0.31% slower
Mean result: 0.99433636044

Jenkins Console Log
[Blue Ocean](https://jenkins.mc-stan.org/blue/organizations/jenkins/Math Pipeline/detail/PR-1558/892/pipeline)
Commit hash: 8d8539a

Machine information

ProductName: Mac OS X ProductVersion: 10.11.6 BuildVersion: 15G22010

CPU:
Intel(R) Xeon(R) CPU E5-1680 v2 @ 3.00GHz

G++:
Configured with: --prefix=/Applications/Xcode.app/Contents/Developer/usr --with-gxx-include-dir=/usr/include/c++/4.2.1
Apple LLVM version 7.0.2 (clang-700.1.81)
Target: x86_64-apple-darwin15.6.0
Thread model: posix

Clang:
Apple LLVM version 7.0.2 (clang-700.1.81)
Target: x86_64-apple-darwin15.6.0
Thread model: posix

andrjohns · 2020-01-11T12:40:07Z

stan/math/fwd/fun/log_sum_exp.hpp

-    deriv += v[i].d_ * exp_vi;
-  }
-  return fvar<T>(log_sum_exp(vals), deriv / denominator);
+  return log_sum_exp(x2, x1);


I've removed a redundant definition here (and in the rev) header. Originally, there was a definition for log_sum_exp(const fvar<T>& x1, double x2) and log_sum_exp(double x1, const fvar<T>& x2), but we can just have one definition and change the order of arguments as needed.

andrjohns · 2020-01-11T12:40:43Z

@t4c1 This is ready for another look-over

t4c1

Fix the conflicts and a few details and this is good to go.

t4c1 · 2020-01-13T09:58:27Z

stan/math/prim/mat/fun/log_sum_exp.hpp

-  return max + std::log((x.array() - max).exp().sum());
+template <typename T, require_t<std::is_arithmetic<scalar_type_t<T>>>...>
+inline auto log_sum_exp(const T& x) {
+  return apply_vector_unary<T>::reduce(x, [&](auto& v) {


Suggested change

return apply_vector_unary<T>::reduce(x, [&](auto& v) {

return apply_vector_unary<T>::reduce(x, [&](const auto& v) {

With no perfect forwarding this can be const. Same for all other lambdas you introduced.

stan/math/rev/fun/log_sum_exp.hpp

stan/math/rev/core/matrix_vari.hpp

t4c1

Looks good. I think we have feture freeze now, so merging needs to wait until release.

stan-buildbot · 2020-01-16T16:41:41Z

Name	Old Result	New Result	Ratio	Performance change( 1 - new / old )
gp_pois_regr/gp_pois_regr.stan	4.95	4.91	1.01	0.78% faster
low_dim_corr_gauss/low_dim_corr_gauss.stan	0.02	0.02	0.97	-3.37% slower
eight_schools/eight_schools.stan	0.09	0.09	1.01	1.04% faster
gp_regr/gp_regr.stan	0.23	0.22	1.03	3.33% faster
irt_2pl/irt_2pl.stan	6.08	6.07	1.0	0.21% faster
performance.compilation	88.24	86.4	1.02	2.09% faster
low_dim_gauss_mix_collapse/low_dim_gauss_mix_collapse.stan	7.35	7.32	1.0	0.36% faster
pkpd/one_comp_mm_elim_abs.stan	20.62	20.29	1.02	1.6% faster
sir/sir.stan	104.92	104.51	1.0	0.39% faster
gp_regr/gen_gp_data.stan	0.05	0.04	1.01	0.88% faster
low_dim_gauss_mix/low_dim_gauss_mix.stan	2.99	2.96	1.01	1.28% faster
pkpd/sim_one_comp_mm_elim_abs.stan	0.32	0.32	1.0	0.22% faster
arK/arK.stan	1.76	1.75	1.0	0.33% faster
arma/arma.stan	0.8	0.8	1.0	-0.42% slower
garch/garch.stan	0.59	0.59	0.99	-0.74% slower
Mean result: 1.00554972051

Jenkins Console Log
Blue Ocean
Commit hash: f3b3286

Machine information

ProductName: Mac OS X ProductVersion: 10.11.6 BuildVersion: 15G22010

CPU:
Intel(R) Xeon(R) CPU E5-1680 v2 @ 3.00GHz

G++:
Configured with: --prefix=/Applications/Xcode.app/Contents/Developer/usr --with-gxx-include-dir=/usr/include/c++/4.2.1
Apple LLVM version 7.0.2 (clang-700.1.81)
Target: x86_64-apple-darwin15.6.0
Thread model: posix

Clang:
Apple LLVM version 7.0.2 (clang-700.1.81)
Target: x86_64-apple-darwin15.6.0
Thread model: posix

stan-buildbot · 2020-01-16T21:21:26Z

Name	Old Result	New Result	Ratio	Performance change( 1 - new / old )
gp_pois_regr/gp_pois_regr.stan	4.98	4.92	1.01	1.22% faster
low_dim_corr_gauss/low_dim_corr_gauss.stan	0.02	0.02	1.0	0.02% faster
eight_schools/eight_schools.stan	0.09	0.09	0.97	-3.14% slower
gp_regr/gp_regr.stan	0.23	0.23	1.01	0.62% faster
irt_2pl/irt_2pl.stan	6.19	6.07	1.02	1.92% faster
performance.compilation	87.91	86.69	1.01	1.39% faster
low_dim_gauss_mix_collapse/low_dim_gauss_mix_collapse.stan	7.3	7.32	1.0	-0.27% slower
pkpd/one_comp_mm_elim_abs.stan	20.25	20.49	0.99	-1.17% slower
sir/sir.stan	104.37	105.15	0.99	-0.75% slower
gp_regr/gen_gp_data.stan	0.04	0.04	1.0	0.2% faster
low_dim_gauss_mix/low_dim_gauss_mix.stan	2.96	2.96	1.0	0.16% faster
pkpd/sim_one_comp_mm_elim_abs.stan	0.32	0.32	1.0	0.12% faster
arK/arK.stan	1.75	1.74	1.0	0.43% faster
arma/arma.stan	0.8	0.81	0.99	-0.65% slower
garch/garch.stan	0.59	0.59	0.99	-0.52% slower
Mean result: 0.99983537081

Jenkins Console Log
Blue Ocean
Commit hash: f3b3286

Machine information

ProductName: Mac OS X ProductVersion: 10.11.6 BuildVersion: 15G22010

CPU:
Intel(R) Xeon(R) CPU E5-1680 v2 @ 3.00GHz

G++:
Configured with: --prefix=/Applications/Xcode.app/Contents/Developer/usr --with-gxx-include-dir=/usr/include/c++/4.2.1
Apple LLVM version 7.0.2 (clang-700.1.81)
Target: x86_64-apple-darwin15.6.0
Thread model: posix

Clang:
Apple LLVM version 7.0.2 (clang-700.1.81)
Target: x86_64-apple-darwin15.6.0
Thread model: posix

andrjohns · 2020-01-17T06:33:49Z

@rok-cesnovar I'm guessing this means that the segfaults have been fixed? Is this safe to merge?

SteveBronder · 2020-01-17T06:37:41Z

I think either way we need to wait until next release since we are in code freeze atm

andrjohns · 2020-01-17T06:42:55Z

Good catch, forgot about that part

rok-cesnovar · 2020-01-17T07:05:43Z

Yes, the segfault were resolved (see #1615), but we have to wait until Sunday/Monday to merge after the release.

Andrew Johnson and others added 19 commits December 14, 2019 23:20

Initial implementation

ca89f78

Merge branch 'develop' into feature/vec_gen_design

de964ba

Add forwarding, rev & fwd versions

6b54c52

Merge branch 'develop' into feature/vec_gen_design

eebfd95

Add autodiff tests, remove arr versions

2b085d1

Nested testing

ac3d5e3

Fix tests, update doc

77f3a59

Tidy doc

b3a1132

Merge branch 'develop' into feature/vec_gen_design

27d1b1f

Cpplint

a4b83a7

Tidy missing doc

d97c963

log_softmax doc errors

461f0b0

Merge branch 'develop' into feature/vec_gen_design

cd0a362

[Jenkins] auto-formatting by clang-format version 5.0.0-3~16.04.1 (ta…

9fa5124

…gs/RELEASE_500/final)

Fix failing test

cba3a30

Merge develop

e5a6b28

Revert head replacement

5a934fe

Merge branch 'develop' into feature/vec_gen_design

b7b2171

[Jenkins] auto-formatting by clang-format version 5.0.0-3~16.04.1 (ta…

8ebea40

…gs/RELEASE_500/final)

t4c1 reviewed Jan 3, 2020

View reviewed changes

t4c1 requested changes Jan 3, 2020

View reviewed changes

rok-cesnovar added 6 commits January 5, 2020 11:14

Merge commit '426ad8fe5a2858b9d367aade1b25a631ac5e97e8' into merge_af…

a73be6d

…ter_flatten # Conflicts: # test/unit/math/prim/scal/fun/log_sum_exp_test.cpp

Merge commit 'd7eb73884e5fad18eaf323760e4625317e1c4c91' into merge_af…

8b5cc7f

…ter_flatten

Merge commit '2b2f7ddff32c12e1e253a6179bf81c1845962306' into merge_af…

df34056

…ter_flatten

Merge commit '731b5f8cf6566db4f13a06851d56cc9e54029146' into merge_af…

8a7017a

…ter_flatten # Conflicts: # stan/math/prim/mat.hpp

Merge branch 'develop' into merge_after_flatten

8214c93

# Conflicts: # stan/math/fwd/arr.hpp # stan/math/fwd/mat/fun/log_sum_exp.hpp

merge conflicts fix

d776eac

remove include

4e83afb

rok-cesnovar mentioned this pull request Jan 8, 2020

Move /vectorize files to /meta #1595

Closed

Andrew Johnson and others added 6 commits January 10, 2020 22:01

Merge branch 'develop' into feature/vec_gen_design

2e4f6b1

Address review comments

febffbe

Merge branch 'develop' into feature/vec_gen_design

67e23b8

[Jenkins] auto-formatting by clang-format version 5.0.0-3~16.04.1 (ta…

477cf9f

…gs/RELEASE_500/final)

Fix merge error

02afdb9

cpplint

8d8539a

andrjohns commented Jan 11, 2020

View reviewed changes

rok-cesnovar mentioned this pull request Jan 11, 2020

Flatten /prim and finish flatten #1606

Merged

2 tasks

t4c1 requested changes Jan 13, 2020

View reviewed changes

Andrew Johnson added 2 commits January 13, 2020 23:06

Merge branch 'develop' into feature/vec_gen_design

1fce5ac

Address comments

f3b3286

t4c1 approved these changes Jan 13, 2020

View reviewed changes

This was referenced Jan 15, 2020

Windows compiler segfaults #1615

Closed

[DO NOT MERGE - Debug] Windows segfault debug #1621

Closed

rok-cesnovar mentioned this pull request Jan 24, 2020

Start merging post release now? #1640

Closed

rok-cesnovar merged commit 4c32cfe into stan-dev:develop Jan 24, 2020

t4c1 mentioned this pull request Jan 27, 2020

Stan Math functions should not return Eigen expressions #1643

Closed

SteveBronder mentioned this pull request Jan 29, 2020

fix returned expressions #1644

Merged

5 tasks

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Generalised unary vector function framework #1558

Generalised unary vector function framework #1558

andrjohns commented Dec 29, 2019 •

edited

Loading

andrjohns commented Dec 31, 2019

stan-buildbot commented Dec 31, 2019

t4c1 Jan 3, 2020 •

edited

Loading

SteveBronder Jan 11, 2020

t4c1 Jan 13, 2020 •

edited

Loading

bob-carpenter Jan 13, 2020

SteveBronder Jan 13, 2020

t4c1 left a comment

rok-cesnovar commented Jan 5, 2020 •

edited

Loading

stan-buildbot commented Jan 5, 2020

andrjohns commented Jan 8, 2020

stan-buildbot commented Jan 11, 2020

andrjohns Jan 11, 2020

andrjohns commented Jan 11, 2020

t4c1 left a comment

t4c1 Jan 13, 2020

t4c1 left a comment

stan-buildbot commented Jan 16, 2020

stan-buildbot commented Jan 16, 2020

andrjohns commented Jan 17, 2020

SteveBronder commented Jan 17, 2020

andrjohns commented Jan 17, 2020

rok-cesnovar commented Jan 17, 2020

	return apply_vector_unary<T>::reduce(x, [&](auto& v) {
	return apply_vector_unary<T>::reduce(x, [&](const auto& v) {

Generalised unary vector function framework #1558

Generalised unary vector function framework #1558

Conversation

andrjohns commented Dec 29, 2019 • edited Loading

Summary

Tests

Side Effects

Checklist

andrjohns commented Dec 31, 2019

stan-buildbot commented Dec 31, 2019

t4c1 Jan 3, 2020 • edited Loading

Choose a reason for hiding this comment

SteveBronder Jan 11, 2020

Choose a reason for hiding this comment

t4c1 Jan 13, 2020 • edited Loading

Choose a reason for hiding this comment

bob-carpenter Jan 13, 2020

Choose a reason for hiding this comment

SteveBronder Jan 13, 2020

Choose a reason for hiding this comment

t4c1 left a comment

Choose a reason for hiding this comment

rok-cesnovar commented Jan 5, 2020 • edited Loading

stan-buildbot commented Jan 5, 2020

andrjohns commented Jan 8, 2020

stan-buildbot commented Jan 11, 2020

andrjohns Jan 11, 2020

Choose a reason for hiding this comment

andrjohns commented Jan 11, 2020

t4c1 left a comment

Choose a reason for hiding this comment

t4c1 Jan 13, 2020

Choose a reason for hiding this comment

t4c1 left a comment

Choose a reason for hiding this comment

stan-buildbot commented Jan 16, 2020

stan-buildbot commented Jan 16, 2020

andrjohns commented Jan 17, 2020

SteveBronder commented Jan 17, 2020

andrjohns commented Jan 17, 2020

rok-cesnovar commented Jan 17, 2020

andrjohns commented Dec 29, 2019 •

edited

Loading

t4c1 Jan 3, 2020 •

edited

Loading

t4c1 Jan 13, 2020 •

edited

Loading

rok-cesnovar commented Jan 5, 2020 •

edited

Loading