You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
The change in #1811 replaced parallel fixturization (= infusing a snippet into a fixture to convert it from human readable but non-compiling code to less human friendly but compiling code) with a serial approach to limit file system spikes.
As it turns out the loading of fixtures is highly inefficient and involves multiple FS calls per snippet. However there is a 1:N relationship between fixtures and snippets, i.e. a single fixture can be used for multiple snippets. That means the number of FS calls to fixturize snippets can be limited to the number of fixtures.
While in the worst case this can still be equal to the number of snippets, in practice it is unlikely.
We can address this inefficiency by loading all fixtures in serial (which will have FS calls), and then execute fixturization of the snippets in parallel (now without FS calls).
Additionally we can look into limiting the parallelism to a constant load.
The text was updated successfully, but these errors were encountered:
The change in #1811 replaced parallel fixturization (= infusing a snippet into a fixture to convert it from human readable but non-compiling code to less human friendly but compiling code) with a serial approach to limit file system spikes.
As it turns out the loading of fixtures is highly inefficient and involves multiple FS calls per snippet. However there is a 1:N relationship between fixtures and snippets, i.e. a single fixture can be used for multiple snippets. That means the number of FS calls to fixturize snippets can be limited to the number of fixtures.
While in the worst case this can still be equal to the number of snippets, in practice it is unlikely.
We can address this inefficiency by loading all fixtures in serial (which will have FS calls), and then execute fixturization of the snippets in parallel (now without FS calls).
Additionally we can look into limiting the parallelism to a constant load.
The text was updated successfully, but these errors were encountered: