This new release of RisingWave (v1.3) is packed with new and valuable features for sinks, sources, and SQL functions. Notable enhancements include advanced support for regular expressions, capabilities for delivering data to more downstream systems, and the addition of the WITH ORDINALITY clause.
For the full list of updates, see the release note.
Support for new array functions
With this release, multiple new array functions were added. These new functions include array_min, array_max, array_sort, array_sum. Array functions accept array types as inputs. These differ from aggregate functions, which perform calculations on a set of values. For more details on the new functions, including examples, look for the functions listed above on the Array functions page.
Advanced support for regular expression functions
We have introduced enhanced support for advanced features in our existing regular expression functions. These functions, including regexp_count, regexp_match, regexp_matches, and regexp_replace, now offers additional capabilities such as backreference, positive lookahead, negative lookahead, positive lookbehind, and negative lookbehind.
Support for WITH ORDINALITY clause
The WITH ORDINALITY clause is now also supported. When using set returning functions, this clause can be used at the end of the FROM clause in a SELECT statement. It will add a new column to the query results, which numbers the rows returned by the function, starting from 1. For more details, see WITH ORDINALITY.
SELECT * FROM
unnest(array[0,1,2])
WITH ORDINALITY;
Syntax update for creating Kafka, Kinesis, and Pulsar sinks
Previously, when creating a sink, the type of sink is specified under the WITH options with the type parameter. As an example, the following SQL query creates an append-only Kafka sink.
CREATE SINK s1 FROM mv WITH (
connector = 'kafka',
properties.bootstrap.server = 'localhost:9092',
topic = 'test_topic',
type = 'append-only');
Now the type of sink and data format can be specified with FORMAT...ENCODE... to be consistent with CREATE SOURCE. Note that the append-only type has been updated to PLAIN. For now, only the JSON format is supported. The following SQL query creates an append-only Kafka sink. Note that PLAIN is specified instead of append-only.
CREATE SINK snk FROM mv with (
connector = 'kafka',
properties.bootstrap.server = 'localhost:9092',
topic = 'test_topic')
FORMAT PLAIN ENCODE JSON;
This syntax change only applied to Kafka, Kinesis, and Pulsar sinks. For more details, see the Sink data to Kafka page or the Sink data to AWS Kinesis page.
A group of new sink connectors
This release introduces support for multiple new sink connectors, allowing you to sink data transformed in RisingWave into more systems. RisingWave strives to provide users with the ability to deliver data to popular downstream systems. Check out our Integrations page for a list of supported systems. You can also vote to signal support for a specific integration to prioritize its development.
Cassandra and ScyllaDB sink
Cassandra and ScyllaDB are open-source NoSQL database management systems capable of handling large workloads. Since ScyllaDB is built as a replacement for Cassandra, the RisingWave sink connector for both is the same. See the Cassandra sink guide for more details.
Doris sink
Apache Doris is an open-source distributed analytical database designed for ad-hoc analysis of large volumes of data. See how simple the process is to deliver data from RisingWave to Doris with this guide.
Elasticsearch sink
Elasticsearch is an open-source distributed search and analytics engine designed to process and handle large values of data in real time. Follow our guide to see how to create an Elasticsearch sink in RisingWave.
NATS messaging system
NATS is an open-source messaging system, which follows a pub-sub and request-response messaging pattern, ideal for real-time applications. See the documentation for details on how to sink data to NATS.
Pulsar sink
Apache Pulsar is an open-source distributed messaging system, designed for developing real-time, event-driven applications. With RisingWave, you can now ingest data from and sink data to Pulsar. See our new guide on how to create a sink to deliver data to Pulsar.
CONCLUSION
These are just some of the new features included with the release of RisingWave v1.3. To get the entire list of updates, which includes new regular expression functions, please refer to the detailed release notes.
Look out for next month’s edition to see what new, exciting features will be added. Check out the RisingWave GitHub repository to stay up to date on the newest features and planned releases.
Sign up for our monthly newsletter if you’d like to keep up to date on all the happenings with RisingWave. Follow us on Twitter and Linkedin, and join our Slack community to talk to our engineers and hundreds of streaming enthusiasts worldwide.