-
Notifications
You must be signed in to change notification settings - Fork 229
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
[temporary_table] add docs for temporary table #1135
Open
Yulei-Yang
wants to merge
2
commits into
apache:master
Choose a base branch
from
Yulei-Yang:add_temp_table
base: master
Could not load branches
Branch not found: {{ refName }}
Loading
Could not load tags
Nothing to show
Loading
Are you sure you want to change the base?
Some commits from the old base branch may be removed from the timeline,
and old review comments may become outdated.
Open
Changes from all commits
Commits
Show all changes
2 commits
Select commit
Hold shift + click to select a range
File filter
Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
There are no files selected for viewing
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,61 @@ | ||
--- | ||
title: Temporary Table | ||
language: en | ||
--- | ||
|
||
<!-- | ||
Licensed to the Apache Software Foundation (ASF) under one | ||
or more contributor license agreements. See the NOTICE file | ||
distributed with this work for additional information | ||
regarding copyright ownership. The ASF licenses this file | ||
to you under the Apache License, Version 2.0 (the | ||
"License"); you may not use this file except in compliance | ||
with the License. You may obtain a copy of the License at | ||
|
||
http://www.apache.org/licenses/LICENSE-2.0 | ||
|
||
Unless required by applicable law or agreed to in writing, | ||
software distributed under the License is distributed on an | ||
"AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY | ||
KIND, either express or implied. See the License for the | ||
specific language governing permissions and limitations | ||
under the License. | ||
--> | ||
|
||
When performing complex data processing tasks, breaking down large SQL queries into multiple steps and temporarily saving the results of each step as physical tables is an effective strategy. This method can significantly reduce the complexity of SQL queries and enhance data debuggability. However, it is important to note that these physical tables must be manually cleaned up after they have served their purpose. If non-physical temporary tables are preferred, Doris currently only supports defining them via the `WITH` clause. | ||
|
||
To address the above issues, Doris introduces the temporary table feature. Temporary tables are temporarily existing materialized internal tables with the following key characteristics: | ||
1. **Session Binding**: Temporary tables exist only within the session in which they were created. Their lifecycle is tightly bound to the current session, meaning that when the session ends, the temporary tables created within that session are automatically deleted. | ||
|
||
2. **Session-specific Visibility**: The visibility of temporary tables is strictly confined to the session in which they were created. Even another session started by the same user at the same time cannot access these temporary tables. | ||
|
||
By introducing the temporary table feature, Doris not only simplifies the temporary data storage and management in complex data processing but also further enhances the flexibility and security of data processing. | ||
|
||
|
||
:::note | ||
|
||
Similar to internal tables, temporary tables must be created under a Database within the Internal Catalog. However, since temporary tables are session-based, their naming is not subject to uniqueness constraints. You can create temporary tables with the same name in different sessions or create temporary tables with the same names as other internal tables. | ||
|
||
If a temporary table and a non-temporary table with the same name exist simultaneously in the same Database, the temporary table has the highest access priority. Within that session, all queries and operations on the table with the same name will only affect the temporary table (except for creating materialized views). | ||
::: | ||
|
||
## Usage | ||
|
||
### Creating a Temporay Table | ||
|
||
Tables of various models can be defined as temporary tables, whether they are Unique, Aggregate, or Duplicate models. You can create temporary tables by adding the TEMPORARY keyword in the following SQL statements: | ||
- [CREATE TABLE](../sql-manual/sql-statements/Data-Definition-Statements/Create/CREATE-TABLE) | ||
- [CREATE TABLE AS SELECT](../sql-manual/sql-statements/Data-Definition-Statements/Create/CREATE-TABLE-AS-SELECT) | ||
- [CREATE TABLE LIKE](../sql-manual/sql-statements/Data-Definition-Statements/Create/CREATE-TABLE-LIKE) | ||
|
||
The other uses of temporary tables are basically the same as regular internal tables. Except for the above-mentioned Create statement, other DDL and DML statements do not require adding the TEMPORARY keyword. | ||
|
||
## Notes | ||
|
||
- Temporary tables can only be created in the Internal Catalog. | ||
- The ENGINE must be set to OLAP when creating a table. | ||
- Alter statements are not supported for modifying temporary tables. | ||
- Due to their temporary nature, creating views and materialized views based on temporary tables is not supported. | ||
- Temporary tables cannot be backed up and are not supported for synchronization using CCR/Sync Job. | ||
- Export, Stream Load, Broker Load, S3 Load, MySQL Load, Routine Load, and Spark Load are not supported. | ||
- When a temporary table is deleted, it does not go to the recycle bin but is permanently deleted immediately. |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
60 changes: 60 additions & 0 deletions
60
i18n/zh-CN/docusaurus-plugin-content-docs/current/table-design/temporary-table.md
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,60 @@ | ||
--- | ||
title: 临时表 | ||
language: zh-CN | ||
--- | ||
|
||
<!-- | ||
Licensed to the Apache Software Foundation (ASF) under one | ||
or more contributor license agreements. See the NOTICE file | ||
distributed with this work for additional information | ||
regarding copyright ownership. The ASF licenses this file | ||
to you under the Apache License, Version 2.0 (the | ||
"License"); you may not use this file except in compliance | ||
with the License. You may obtain a copy of the License at | ||
|
||
http://www.apache.org/licenses/LICENSE-2.0 | ||
|
||
Unless required by applicable law or agreed to in writing, | ||
software distributed under the License is distributed on an | ||
"AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY | ||
KIND, either express or implied. See the License for the | ||
specific language governing permissions and limitations | ||
under the License. | ||
--> | ||
|
||
在进行复杂的数据处理任务时,将大型 SQL 查询拆分为多个步骤,并将每个步骤的计算结果临时保存为实体表,是一种有效的策略。这种方法能够显著降低 SQL 查询的复杂度,并提升数据的可调试性。然而,需要注意的是,实体表在完成其使用目的后,需要手动进行清理。若选择使用非实体临时表,当前 Doris 仅支持通过 `WITH` 子句进行定义。 | ||
|
||
为了解决上述问题,Doris 引入了临时表功能。临时表是一种临时存在的物化内表,具备以下关键特性: | ||
1. **会话绑定**:临时表仅存在于创建它的会话(Session)中。其生命周期与当前会话紧密绑定,即当会话结束时,该会话中创建的临时表会自动被删除。 | ||
|
||
2. **会话内可见性**:临时表的可见性严格限制在创建它的会话范围内。即使在同一时间由同一用户启动的另一个会话,也无法访问这些临时表。 | ||
|
||
通过引入临时表功能,Doris 不仅简化了复杂数据处理过程中的临时数据存储与管理,还进一步增强了数据处理的灵活性和安全性。 | ||
|
||
|
||
:::info 备注 | ||
|
||
与内表类似,临时表必须在 Internal Catalog 内的某个 Database 下创建。但由于临时表基于 Session,因此其命名不受唯一性约束。您可以在不同 Session 中创建同名临时表,或创建与其他内表同名的临时表。 | ||
|
||
如果同一 Database 中同时存在同名的临时表和非临时表,临时表具有最高访问优先级。在该 Session 内,所有针对同名表的查询和操作仅对临时表生效(除创建物化视图外)。 | ||
::: | ||
|
||
## 用法 | ||
|
||
### 创建临时表 | ||
各种模型的表都可以被定义为临时表, 不论是 Unique、Aggregate 或是 Duplicate 模型。可以在下列 SQL 中添加 TEMPORARY 关键字创建临时表: | ||
- [CREATE TABLE](../sql-manual/sql-statements/Data-Definition-Statements/Create/CREATE-TABLE) | ||
- [CREATE TABLE AS SELECT](../sql-manual/sql-statements/Data-Definition-Statements/Create/CREATE-TABLE-AS-SELECT) | ||
- [CREATE TABLE LIKE](../sql-manual/sql-statements/Data-Definition-Statements/Create/CREATE-TABLE-LIKE) | ||
|
||
临时表的其它用法基本和普通内表相同。除上述 Create 语句外, 其它 DDL 及 DML 语句无需添加 TEMPORARY 关键字。 | ||
|
||
## 注意事项 | ||
|
||
- 临时表只能在 Internal Catalog 中创建 | ||
- 建表时 `ENGINE` 必须为 `OLAP` | ||
- 不支持使用 Alter 语句修改临时表 | ||
- 由于临时性,不支持基于临时表创建视图和物化视图 | ||
- 不支持备份临时表,不支持使用 CCR / Sync Job 同步临时表 | ||
- 不支持导出、Stream Load、Broker Load、S3 Load、Mysql Load、Routine Load、Spark Load | ||
- 删除临时表时,不进回收站,直接彻底删除 |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Oops, something went wrong.
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Thank you for contribution
1.please use Chinese in docs and use :::info 备注 ::: instead of :::note :::
2.please fix the typo
3.please make sure the capitalization is unified, such as SQL (All caps) , Session(Sentence case) , Rollup (Sentence case)
4.please add blank between Chinese and EN words or Numbers.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
fixed