Upgrade main to Lucene 10 #113617
Draft
Upgrade main to Lucene 10 #113617
+8,387
−3,538
This pull request is big! We’re only showing the most recent 250 commits.
Commits on Aug 30, 2024
-
-
Reword javadoc text for SearchServiceTests#testSlicingBehaviourForPar…
…allelCollection Offloading is no longer necessary, but we do want to provide the executor for other operations
-
Address SearchServiceTests compile errors caused by removal of IndexS…
…earcher#getExecutor
-
-
-
-
-
-
-
Replace NOCOMMIT comments with //TODO Lucene 10 upgrade to be able to…
… run tests in CI
Commits on Aug 31, 2024
Commits on Sep 1, 2024
Commits on Sep 2, 2024
-
-
-
-
-
-
-
-
-
Fix QueryStringQueryBuilderTests
Exception message changed slightly.
-
RegEx automata are no longer determinized by default.
-
-
-
-
-
-
Commits on Sep 3, 2024
-
Mute MetadataCreateIndexServiceTests.testValidateDotIndex because of …
…Lucene RegExp tilde operator changes
-
-
-
-
Commits on Sep 4, 2024
Commits on Sep 5, 2024
Commits on Sep 6, 2024
Commits on Sep 7, 2024
Commits on Sep 8, 2024
Commits on Sep 9, 2024
Commits on Sep 10, 2024
-
-
-
-
-
-
Null out missing codec in BWCLucene70Codec
This temporarily gets us past missing codec that we attempt to load at node startup, even if we don't use it.
-
Commits on Sep 11, 2024
Commits on Sep 12, 2024
-
-
Merge branch 'main' into lucene_snapshot_10
Conflicts: build-tools-internal/version.properties modules/analysis-common/src/test/java/org/elasticsearch/analysis/common/EdgeNGramTokenizerTests.java modules/analysis-common/src/test/java/org/elasticsearch/analysis/common/SynonymsAnalysisTests.java modules/reindex/src/internalClusterTest/java/org/elasticsearch/migration/MultiFeatureMigrationIT.java plugins/analysis-phonetic/src/test/java/org/elasticsearch/plugin/analysis/phonetic/AnalysisPhoneticFactoryTests.java qa/rolling-upgrade-legacy/src/test/java/org/elasticsearch/upgrades/RecoveryIT.java server/src/internalClusterTest/java/org/elasticsearch/repositories/IndexSnapshotsServiceIT.java server/src/test/java/org/elasticsearch/action/admin/cluster/stats/VersionStatsTests.java server/src/test/java/org/elasticsearch/cluster/metadata/MetadataCreateIndexServiceTests.java server/src/test/java/org/elasticsearch/cluster/metadata/MetadataTests.java server/src/test/java/org/elasticsearch/index/IndexVersionTests.java server/src/test/java/org/elasticsearch/index/engine/InternalEngineTests.java server/src/test/java/org/elasticsearch/index/mapper/DateFieldMapperTests.java server/src/test/java/org/elasticsearch/index/mapper/DynamicTemplatesTests.java server/src/test/java/org/elasticsearch/index/mapper/FieldNamesFieldMapperTests.java server/src/test/java/org/elasticsearch/index/mapper/ParametrizedMapperTests.java server/src/test/java/org/elasticsearch/index/mapper/vectors/BinaryDenseVectorScriptDocValuesTests.java server/src/test/java/org/elasticsearch/index/similarity/SimilarityServiceTests.java server/src/test/java/org/elasticsearch/indices/analysis/AnalysisModuleTests.java server/src/test/java/org/elasticsearch/script/VectorScoreScriptUtilsTests.java server/src/test/java/org/elasticsearch/script/field/vectors/DenseVectorTests.java test/framework/src/main/java/org/elasticsearch/index/mapper/MetadataMapperTestCase.java test/framework/src/main/java/org/elasticsearch/snapshots/AbstractSnapshotIntegTestCase.java x-pack/plugin/deprecation/src/test/java/org/elasticsearch/xpack/deprecation/IndexDeprecationChecksTests.java x-pack/plugin/spatial/src/test/java/org/elasticsearch/xpack/spatial/index/mapper/ShapeFieldMapperTests.java x-pack/plugin/spatial/src/test/java/org/elasticsearch/xpack/spatial/index/query/LegacyGeoShapeWithDocValuesQueryTests.java
-
-
Override the correct search method
search(List<LeafReaderContext>, Weight, Collector) is deprecated and mostly unused. We need to override the other one that accepts a LeafReaderContextPartition[].
-
Commits on Sep 13, 2024
-
Use RegExp.DEPRECATED_COMPLEMENT where needed
Lucene 10 removed the RegExp.COMPLEMENT flag and the ~ operator but brought it back as a deprecated option in apache/lucene#13739, so for the time being we should still be able to use it.
-
-
Commits on Sep 14, 2024
Commits on Sep 15, 2024
Commits on Sep 16, 2024
-
-
Move AggregatorTestCase to search(Query, CollectorManager)
search(Query, Collector) is deprecated and no longer performs post collection
-
Don't randomize LuceneTestCase concurrency when using "newSearcher"
LuceneTestCase#newSearcher(reader) now randomizes which concurrency type is being used. For some parts we don't support INTRA_SEGMENT concurrency, so we override this widely used utility method so we use INTER_SEGMENT concurrency for now.
-
Fix docs according to changes in Lovins token filter, Pathhierarchy A…
…nalyzer and Noritoken filter
-
Commits on Sep 17, 2024
Commits on Sep 18, 2024
Commits on Sep 19, 2024
-
-
-
Add a capability and transport version for new regex and range interv…
…al query support (#113128) This commit adds a capability and transport version for new regex and range interval query support.
-
Multi term intervals: increase max_expansions (#112826)
Currently multi term interval queries (prefix, wildcard, fuzzy, regexp and range) can expand maximum to 128 terms. To reach parity with span queries that we want to deprecate, set max expansions to indices.query.bool.max_clause_count which is used in span queries. Relates to #110491
Commits on Sep 20, 2024
-
-
-
-
-
-
-
Rephrase comment in OldCodecsAvailableTests
This isn't a blocker for the lucene 10 upgrade, it is for the release of stack 9. It's already annotated @UpdateForV9 so we are good.
-
clarify comment in BWCLucene70Codec
Added update for V9 annotation so we don't forget, not a blocker for lucene 10 upgrade
-
-
-
-
-
-
-
-
Commits on Sep 21, 2024
-
-
-
-
-
Multi term intervals: increase max_expansions (#112826)
Currently multi term interval queries (prefix, wildcard, fuzzy, regexp and range) can expand maximum to 128 terms. To reach parity with span queries that we want to deprecate, set max expansions to indices.query.bool.max_clause_count which is used in span queries. Relates to #110491
-
Commits on Sep 22, 2024
Commits on Sep 23, 2024
-
-
-
Add Lucene70DocValuesFormat to old-lucene-versions plugin (#113377)
Add Lucene70DocValuesFormat to old-lucene-versions plugin. Lucene 10 has remove the old Lucene70DocValuesFormat from its backward_codecs, so we can no longer lookup the doc values format by name (DocValuesFormat.forName("Lucene70")). Instead we make a copy and reference it statically. The code is for the most part a direct copy, with the following changes: 1. package and import updates 2. Elasticsearch's spotless 3. A couple of changes in Lucene70DocValuesConsumer, where we no longer iterate SortedSetDocValues until NO_MORE_DOCS, but rather use a counted loop with docValueCount. As per what is done elsewhere in lucene 10. TheLucene70DocValuesFormatTests has been, similar to how we test other formats in this plugin.
Commits on Sep 24, 2024
Commits on Sep 25, 2024
Commits on Sep 26, 2024
Commits on Sep 27, 2024
-
-
We run in a mixed clusters, replacing the profile description isn't enough, we need to skip the test against previous versions because we get a different output depending on the version of the node that has the shard. I isolated the offending assertion to a separate test, so that we don't need to skip the entire profile test.
-
Commits on Sep 28, 2024
Commits on Sep 29, 2024
-
Revert "[Automated] Update Lucene snapshot to 10.0.0-snapshot-0a8604d…
…908c" This reverts commit f93db49.
-
Commits on Sep 30, 2024
-
Address compile errors after vector api changes upstream (#113766)
Our lucene_snapshot branch requires updating after apache/lucene#13779
-
Update lucene snapshot buildkite config to build from branch_10_0 (#1…
…13797) The Lucene 10 branch has been cut, we want to switch the snapshot creation to the newly created branch until Lucene 10 has been released. Later, we will switch to branch_10x.
-
-
-
Commits on Oct 1, 2024
-
Fix needless use of concurrent collector managers (#113739)
The utility methods for creating the collectors that were removed from Lucene created non-concurrent collector managers. The default constructors create concurrent ones though which leads to a performance regression, especially when sorting. -> fixed by adjusting to non-concurrent managers again
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.