set m_read_opts.ignore_range_deletions properly #1218

rockeet · 2022-08-28T06:51:41Z

ignore_range_deletions should be set to true when range del is not enabled

facebook-github-bot · 2022-08-29T22:35:51Z

@yoshinorim has imported this pull request. If you are a Meta employee, you can view this diff on Phabricator.

Pushapgl · 2022-08-30T23:51:25Z

whats the motivation of this change?

rockeet · 2022-08-31T05:38:54Z

whats the motivation of this change?

if dont ignore_range_deletions, DB reads will try to construct RangeDelAggregator which waste amount of CPU time without any useful work when rocksdb_enable_delete_range_for_drop_index was set to false.

Summary: ignore_range_deletions should be set to true when range del is not enabled Pull Request resolved: #1218 Reviewed By: Pushapgl Differential Revision: D39115879 Pulled By: yoshinorim fbshipit-source-id: 40e7bcb

Upstream commit ID: facebook/mysql-5.6@238afbb PS-8494: Merge percona-202206 (https://jira.percona.com/browse/PS-8494) Summary: ignore_range_deletions should be set to true when range del is not enabled Pull Request resolved: facebook/mysql-5.6#1218 Reviewed By: Pushapgl Differential Revision: D39115879 Pulled By: yoshinorim fbshipit-source-id: 40e7bcb86590c8f9be44dfa3f74277fb3f49a6d5

Summary: ignore_range_deletions should be set to true when range del is not enabled Pull Request resolved: facebook#1218 GitHub Author: leipeng <[email protected]> Test Plan: Imported from GitHub, without a `Test Plan:` line. Reviewers: pgl Reviewed By: pgl Subscribers: pgl, [email protected] Differential Revision: https://phabricator.intern.facebook.com/D39115879 Tags: accept2ship

Upstream commit ID: facebook/mysql-5.6@238afbb PS-8494: Merge percona-202206 (https://jira.percona.com/browse/PS-8494) Summary: ignore_range_deletions should be set to true when range del is not enabled Pull Request resolved: facebook/mysql-5.6#1218 Reviewed By: Pushapgl Differential Revision: D39115879 Pulled By: yoshinorim fbshipit-source-id: 40e7bcb86590c8f9be44dfa3f74277fb3f49a6d5

Summary: ignore_range_deletions should be set to true when range del is not enabled Pull Request resolved: facebook#1218 Reviewed By: Pushapgl Differential Revision: D39115879 Pulled By: yoshinorim fbshipit-source-id: 40e7bcb

Upstream commit ID: facebook/mysql-5.6@238afbb PS-8494: Merge percona-202206 (https://jira.percona.com/browse/PS-8494) Summary: ignore_range_deletions should be set to true when range del is not enabled Pull Request resolved: facebook/mysql-5.6#1218 Reviewed By: Pushapgl Differential Revision: D39115879 Pulled By: yoshinorim fbshipit-source-id: 40e7bcb86590c8f9be44dfa3f74277fb3f49a6d5

Summary: If the intrinsic tmp tables goes beyond some threshold, then we commit the write batch accumulated so far and start new transaction handling the write batch. **Changes** * Added a variable rocksdb_max_intrinsic_tmp_table_write_count which controls the size of write batch allowed for intrinsic tmp table. Once we cross that threshold, we will commit the current transaction and then start a new transaction. * If there are valid open iterators before we commit intrinsic tmp table transaction, then we refresh all iterators back to previous position after committing transaction. Recursive cte opens multiple iterators for same transaction and we need to refresh all iterators after we commit the current write batch for transaction. * I also refreshed the iterators after the first write to the write batch. This is done to handle the special case where we start rocksdb iterator on new transaction(with empty write batch). Then iterator will only see the already committed data, but ignores any new data added in write batch later. So we are pro-actively refresh the iterator after first write in write batch. * Added a status variable rocksdb_intrinsic_tmp_table_commits to count number of intrinsic tmp table commits. * create_ondisk_from_heap method uses table->record[1] for insertion. Myrocks encode_value_slice relies on table->record[0] for value encoding. We have reposition the fields in write_row specifically for intrinsic tmp table to point to record[1]. Better way to fix this is to remove the dependency on record[1] from encode_value_slice method and add dependency on input buf from write_row. Reviewed By: luqun Differential Revision: D37835016 --------------------------------------------------------------------------------------- Temporarily disable some of rocksdb.tmp_table test Summary: RocksDB tmp table needs bug fixes, and meanwhile disabling some of the rocksdb.tmp_table tests so that it won't report failures. Reviewed By: lth Differential Revision: D41391615 --------------------------------------------------------------------------------------- set m_read_opts.ignore_range_deletions properly (facebook#1218) Summary: ignore_range_deletions should be set to true when range del is not enabled Pull Request resolved: facebook#1218 Reviewed By: Pushapgl Differential Revision: D39115879 Pulled By: yoshinorim

Upstream commit ID: facebook/mysql-5.6@238afbb PS-8494: Merge percona-202206 (https://jira.percona.com/browse/PS-8494) Summary: ignore_range_deletions should be set to true when range del is not enabled Pull Request resolved: facebook/mysql-5.6#1218 Reviewed By: Pushapgl Differential Revision: D39115879 Pulled By: yoshinorim fbshipit-source-id: 40e7bcb86590c8f9be44dfa3f74277fb3f49a6d5

Summary: If the intrinsic tmp tables goes beyond some threshold, then we commit the write batch accumulated so far and start new transaction handling the write batch. **Changes** * Added a variable rocksdb_max_intrinsic_tmp_table_write_count which controls the size of write batch allowed for intrinsic tmp table. Once we cross that threshold, we will commit the current transaction and then start a new transaction. * If there are valid open iterators before we commit intrinsic tmp table transaction, then we refresh all iterators back to previous position after committing transaction. Recursive cte opens multiple iterators for same transaction and we need to refresh all iterators after we commit the current write batch for transaction. * I also refreshed the iterators after the first write to the write batch. This is done to handle the special case where we start rocksdb iterator on new transaction(with empty write batch). Then iterator will only see the already committed data, but ignores any new data added in write batch later. So we are pro-actively refresh the iterator after first write in write batch. * Added a status variable rocksdb_intrinsic_tmp_table_commits to count number of intrinsic tmp table commits. * create_ondisk_from_heap method uses table->record[1] for insertion. Myrocks encode_value_slice relies on table->record[0] for value encoding. We have reposition the fields in write_row specifically for intrinsic tmp table to point to record[1]. Better way to fix this is to remove the dependency on record[1] from encode_value_slice method and add dependency on input buf from write_row. Reviewed By: luqun Differential Revision: D37835016 --------------------------------------------------------------------------------------- Temporarily disable some of rocksdb.tmp_table test Summary: RocksDB tmp table needs bug fixes, and meanwhile disabling some of the rocksdb.tmp_table tests so that it won't report failures. Reviewed By: lth Differential Revision: D41391615 --------------------------------------------------------------------------------------- set m_read_opts.ignore_range_deletions properly (facebook#1218) Summary: ignore_range_deletions should be set to true when range del is not enabled Pull Request resolved: facebook#1218 Reviewed By: Pushapgl Differential Revision: D39115879 Pulled By: yoshinorim

Upstream commit ID: facebook/mysql-5.6@238afbb PS-8494: Merge percona-202206 (https://jira.percona.com/browse/PS-8494) Summary: ignore_range_deletions should be set to true when range del is not enabled Pull Request resolved: facebook/mysql-5.6#1218 Reviewed By: Pushapgl Differential Revision: D39115879 Pulled By: yoshinorim fbshipit-source-id: 40e7bcb86590c8f9be44dfa3f74277fb3f49a6d5

Summary: If the intrinsic tmp tables goes beyond some threshold, then we commit the write batch accumulated so far and start new transaction handling the write batch. **Changes** * Added a variable rocksdb_max_intrinsic_tmp_table_write_count which controls the size of write batch allowed for intrinsic tmp table. Once we cross that threshold, we will commit the current transaction and then start a new transaction. * If there are valid open iterators before we commit intrinsic tmp table transaction, then we refresh all iterators back to previous position after committing transaction. Recursive cte opens multiple iterators for same transaction and we need to refresh all iterators after we commit the current write batch for transaction. * I also refreshed the iterators after the first write to the write batch. This is done to handle the special case where we start rocksdb iterator on new transaction(with empty write batch). Then iterator will only see the already committed data, but ignores any new data added in write batch later. So we are pro-actively refresh the iterator after first write in write batch. * Added a status variable rocksdb_intrinsic_tmp_table_commits to count number of intrinsic tmp table commits. * create_ondisk_from_heap method uses table->record[1] for insertion. Myrocks encode_value_slice relies on table->record[0] for value encoding. We have reposition the fields in write_row specifically for intrinsic tmp table to point to record[1]. Better way to fix this is to remove the dependency on record[1] from encode_value_slice method and add dependency on input buf from write_row. Reviewed By: luqun Differential Revision: D37835016 --------------------------------------------------------------------------------------- Temporarily disable some of rocksdb.tmp_table test Summary: RocksDB tmp table needs bug fixes, and meanwhile disabling some of the rocksdb.tmp_table tests so that it won't report failures. Reviewed By: lth Differential Revision: D41391615 --------------------------------------------------------------------------------------- set m_read_opts.ignore_range_deletions properly (facebook#1218) Summary: ignore_range_deletions should be set to true when range del is not enabled Pull Request resolved: facebook#1218 Reviewed By: Pushapgl Differential Revision: D39115879 Pulled By: yoshinorim

Upstream commit ID: facebook/mysql-5.6@238afbb PS-8494: Merge percona-202206 (https://jira.percona.com/browse/PS-8494) Summary: ignore_range_deletions should be set to true when range del is not enabled Pull Request resolved: facebook/mysql-5.6#1218 Reviewed By: Pushapgl Differential Revision: D39115879 Pulled By: yoshinorim fbshipit-source-id: 40e7bcb86590c8f9be44dfa3f74277fb3f49a6d5

set m_read_opts.ignore_range_deletions properly

43cf756

ignore_range_deletions should be set to true when range del is not enabled

facebook-github-bot added the CLA Signed label Aug 28, 2022

set m_read_opts.ignore_range_deletions - fix missing changes

b0030b3

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

set m_read_opts.ignore_range_deletions properly #1218

set m_read_opts.ignore_range_deletions properly #1218

rockeet commented Aug 28, 2022

facebook-github-bot commented Aug 29, 2022

Pushapgl commented Aug 30, 2022

rockeet commented Aug 31, 2022 •

edited

Loading

set m_read_opts.ignore_range_deletions properly #1218

Are you sure you want to change the base?

set m_read_opts.ignore_range_deletions properly #1218

Conversation

rockeet commented Aug 28, 2022

facebook-github-bot commented Aug 29, 2022

Pushapgl commented Aug 30, 2022

rockeet commented Aug 31, 2022 • edited Loading

rockeet commented Aug 31, 2022 •

edited

Loading