eKuiper—a lightweight IoT data analytics and streaming software—is now available in its 1.10 release. eKuiper, an LF Edge project, migrates real-time cloud streaming analytics frameworks such as Apache Spark, Apache Storm and Apache Flink to the edge. eKuiper references these cloud streaming frameworks, incorporates any special requirements of edge analytics and introduces rule engine, which is based on Source, SQL (business logic) and Sink; rule engine is used for developing streaming applications at the edge.
We are thrilled to announce the highly anticipated release of eKuiper version 1.10.0, marking a significant milestone as our first double-digit release. This remarkable release boasts an extensive array of features and a growing community of contributors, for whom we express our deepest gratitude.
In this release, our top priority was enhancing the product’s core dependencies. Notably, we upgraded the Go language version to 1.20, ensuring optimal performance and compatibility. Additionally, eKuiper now extends support for the latest major version of Minnesota (v3) from EdgeX, further expanding its interoperability capabilities. While driving advancements in expressiveness, connectivity, and usability, we diligently focused on maintaining a delicate balance between feature richness and the ability to cater to edge deployments. Our commitment to delivering a streamlined experience for edge computing remains unwavering.
The 1.10 release introduces several significant new features, including:
- Rule scheduling:
- Empower the edge to autonomously manage rule execution, allowing for enhanced flexibility and efficiency in edge computing environments.
- EdgeX Minnesota support:
- Seamlessly integrate with the latest major version of EdgeX.
- Enhanced file sink:
- Expand the capabilities of the file sink by supporting a wider range of file types, introducing rolling and compression features for efficient data storage and management.
- Kafka sink:
- Introduce a new sink to write rule results directly to Kafka, enabling seamless integration with Kafka-based data pipelines and stream processing workflows.
- SQL source/sink enhancement:
- Enhance the SQL source/sink functionality by introducing support for the new ClickHouse driver. Additionally, enable the configuration of maximum connections to optimize performance in scenarios with high connection volumes.
- Batch strategy for sink:
- Introduce common properties for sinks to configure batch strategies, enabling efficient data handling and reducing I/O operations as needed.
- Complex data extraction for sink:
- Simplify data transformation and processing by introducing common properties for sinks to extract specific data fields and optimize column selection, resulting in improved data processing performance.
- Array type payload support:
- Enhance the source to support array type payloads, allowing for seamless handling and processing of complex data structures.
- Unnest function:
- Introduce the unnest function, enabling the conversion of array types into multiple lines of data, facilitating more granular and detailed data processing.
- Array and object functions:
- Expand the array and object functions library with over 10 additional functions, providing extensive support for complex data processing scenarios.
- Dot notation for nested field access:
- Introduce dot notation to access nested fields within data structures, simplifying data retrieval and manipulation within complex nested hierarchies.
- External state access in Redis:
- Enable the ability to read external state information from Redis, facilitating efficient data retrieval and integration with external applications.
- SQL syntax enhancements:
- Enhance the SQL syntax by introducing support for expressions in array indexing, enabling more flexible and dynamic array operations within SQL queries.
- Rule-related functions:
- Introduce additional functions to retrieve the current rule ID and enable pipeline delay, providing greater control and flexibility in rule execution and pipeline management.
- Graph API for lookup table:
- Enabling efficient lookup table and lookup join in Graph API.
With this release, we reaffirm our dedication to empowering organizations with a cutting-edge, reliable, and user-friendly solution for harnessing the full potential of edge analytics and stream processing. Learn more about these and other features of eKuiper’s 1.10 release in the release notes.
What’s Next
In our upcoming version, we are thrilled to prioritize further advancements in the streaming and SQL runtime of eKuiper. Our focus will revolve around enriching the window functionality, particularly empowering the event window to cater to a wider range of scenarios. To ensure seamless integration and compatibility, we are committed to adding and refining the SQL syntax to align with standard SQL specifications. This will provide users with a familiar and intuitive experience when working with eKuiper, while enabling greater flexibility and ease of use.
Stay tuned for more updates and join us on this exciting journey as we continue to shape the future of edge computing with eKuiper.
