Energy mismatch between the API / demo and things run locally using the AdsorbML tutorial notebook

[brookwander]

Hello,

Suhas and Jiheon from the University of Toronto brought this to my attention and I have reproduced it on my end as well. For some systems there is a large mismatch between the results you obtain using the AdsorbML notebook tutorial and the API/demo. There are also some examples where they are congruent. I will provide one such disparate example to be explored: *CHO sitting on the 211 surface of Zn (mp-79). In the referenced branch, I have included a pre-run notebook with the disparate results.

link to the run calculation through the API: https://open-catalyst.metademolab.com/results/f00031b8-cc91-4ace-9aa3-89bd3e3c6bbf

Energy from API is 0.055 eV
Energy from the Jupyter notebook: -0.361 ( a 0.4 eV difference!)

I have also confirmed that the low energy structure from the tutorial is present in the results from the API (but it is not the lowest E structure. It is 0.079 eV in the demo). So it is not as if the demo has just missed the lowest E structure by chance / randomness.

I have also run the same calculation multiple times in the demo and realized some pretty large stochasticity in the result (~0.15 eV) where as the Jupyter notebook seems very consistent (<0.05 eV) despite not having a random seed set.

[brookwander]

hmm can't find the branch in the dev section. https://github.com/FAIR-Chem/fairchem/tree/e-matching-issue

[suhasm]

I'm quickly pasting our comparison of webapp vs ocpapi vs brook's spreadsheet (gemnet-oc, I think) vs adsorbml (equiformerv2). The major discrepancies are highlighted in red. The systems were randomly chosen.

[github-actions]

This issue has been marked as stale because it has been open for 30 days with no activity.