tests fail due to large amount of output with highish tests.iters

[msokolov]

Description

I've noticed that some tests fail when tests.iters is set to a highish value (like 20) because stdout grows above the allowed threshold. I assume this is something we'd want to fix, but I'm not clear how to go about it exactly. Some tests I've seen fail are: TestDemo.testIndexSearch (IndexFiles outputs the names of files as they are added to the index); TestIndexWriter.testDocValuesMixedSkippingIndex (prints a stack trace, I think it's expected?); TestTransactions prints out an iteration count. We could work on reducing the amount of printing, but I wonder if the test framework should ideally multiply the acceptable stdout threshold by the iters?

Version and environment details

all versions, probably

[msokolov]

Example failing test output:

org.apache.lucene.demo.TestDemo.classMethod (:lucene:demo)                                                                                                
    Test output: /local/home/sokolovm/workspace/lucene/lucene/demo/build/test-results/test/outputs/OUTPUT-org.apache.lucene.demo.TestDemo.txt                 
    Reproduce with: gradlew :lucene:demo:test --tests "org.apache.lucene.demo.TestDemo.classMethod" -Ptests.jvms=8 "Ptests.jvmargs=-XX:TieredStopAtLevel=1 -XX:+UseParallelGC -XX:ActiveProcessorCount=1" -Ptests.seed=8CEBA6D040DDF9D3 -Ptests.iters=20 -Ptests.gui=false Ptests.file.encoding=UTF-8 -Ptests.vectorsize=default -Ptests.forceintegervectors=false

[msokolov]

There is something about the way TestDocInverterPerFieldErrorInfo.testNoExtraNoise captures the info-stream that causes it to always fail when test.iters > 1. This whole situation makes me wonder if we should occasionally run with tests.iters > 1 in some CI setting just to catch test issues like this where test fixtures leak across runs.