Commit
This commit does not belong to any branch on this repository, and may belong to a fork outside of the repository.
feat(risingwave): add streaming DDLs (#8239)
Risingwave plays a role as a computing engine and storage system in Streaming ecosystems. Usually, a streaming workload will include two/three systems like this: `upstream data source --> Risingwave` Or, `upstream data source --> Risingwave --> downstream data sink` This PR adds streaming support, mainly new relations related to streaming, for the Risingwave backend. To be specific, Four types of new relations are introduced: - `Source`, data sources that encompass a connector connected to an upstream data system like Kafka. A source works like an extraction or a placeholder for an upstream system, and although it has columns, it does not store any data itself. - `Materialized View`, which is Risingwave's core concept for streaming. One can create an MV with a query on existing tables or sources. The data in MV is automatically updated in a real-time way. Users can access the data with a `select` statement just like accessing a table. - `Table` with connector, works like a combination of a source and a normal table. The difference between `Table` with connector and `Source` is that, once the table is created, it will automatically start to consume data from upstream systems and update it into Risingwave. - `Sink`, unlink a `Materialized View` which stores the result of a query in RW, users can choose to sink the result to a downstream system, e.g. Redis, and then read the data in the downstream system. Unlike an MV, User cannot access the data directly from a sink. Some minor changes: 1. Bump the image version to a new nightly build, as the new image solves some issues found in the [previous PR](#7954). 2. Mark one test, `test_semi_join`, as skipped for risingwave backend. The test sometimes stuck and it seems to be Risingwave's fault. I'll continue to investigate.
- Loading branch information