The EdgeX Foundry Hanoi Release
By Jim White, EdgeX Foundry TSC Chairman
What a year 2020 has been! If you are like me, you are looking for some stability and normalization right now. For me, in addition to my family, I have come to rely on my work with the incredibly talented and dedicated people of the EdgeX Foundry community as an element of my centeredness and stability.
And just like stable clockwork, the community is delivering its seventh release of EdgeX Foundry. We release EdgeX each spring and fall and this is the seventh consecutive semi-annual release since our founding in April of 2017. That’s showing some pretty good consistency and it is all due to the efforts of some outstanding contributors. This is the “Hanoi” release and it is a minor version release (1.3). It follows and is backward compatible with our Geneva (v1.2) release that came out this spring.
Hanoi Features
Even though this is a minor release, there are all sorts of new features. Too many to list them all, but here is a smattering of some of the more significant highlights:
Restructure of Compose Files: For convenience, the project makes all the EdgeX micro services available in Docker Hub. For further convenience, we have always supplied a set of Docker Compose files which makes deploying and orchestrating all the micro service containers to your target platform easier. However, there have been so many EdgeX service options and configurations that EdgeX adopters typically had to do some customization of the Compose file(s) in order to suit their use case and needs. With the Hanoi release, adopters will find that EdgeX now has a Compose file “make” capability that allows users to more easily customize their Compose file without a lot of manual editing.
Edge Data Tagging (location tagging): EdgeX already has the ability to get the collected sensor data to your choice of cloud, enterprise or other application in the format and structure that you want. Now in Hanoi, you can tag the data coming from an EdgeX instance so that when it arrives in the cloud, enterprise, or in another application you know where it came from. This is important when you have many EdgeX instances sending in edge data. You can configure each EdgeX instance to tag the data as you see fit. You could use the edge node’s address or system identifier, device identifier, a GPS location, node label or any means you desire to pin the incoming data in some meaningful way so that using systems and applications know where the data originated.
CLI tool: EdgeX formally launches its command line interface tool with this release. The CLI allows developers and adopters to issue all sorts of EdgeX API calls to its services using terminal commands. This allows for easier scripting of tasks that take care of duties such as provisioning a device, or setting up a schedule.
UI improvements: EdgeX, as edge middleware, operates in a headless way. The UI was greatly revitalized and improved in this release. It was not constructed for production, but you can use it for development and demonstration purposes. The new UI allows you to see the status of the system, interact with its configuration facilities and even display some of the collected sensor data.
Fledge integration: EdgeX is a member of the LF Edge umbrella project. The purpose of LF Edge is to establish an open, interoperable framework for edge computing independent of hardware, silicon, cloud or operating system. EdgeX has participated with a number of projects within the umbrella, but with this release, EdgeX has provided a sample service to export data from EdgeX to Fledge. This allows EdgeX device connectors and capabilities to be used with Fledge instances. Conversely, with their next release, the Fledge project intends to provide a device service to allow Fledge instances to feed EdgeX instances.
Distributed Device Services: EdgeX, as a micro service platform, supports the idea of distributing the micro services across whatever compute and network you have available. Having said that, actually distributing services of EdgeX to different hosts could be a challenge. In the Hanoi release, we make it easier to distribute device services – that is the “thing” connector services – to other hosts.
Performance and scalability testing: The EdgeX testing/QA team has been hard at work during this release to provide some initial performance and scalability testing apparatus. With this capability, EdgeX has the ability to start to provide some guidance around how EdgeX scales as the amount of data gets pushed through the system or how many devices of particular types you can hang on an instance of EdgeX. Our harness is still in its early stages with this release, but it allows adopters to begin to do some calculations about how a large-scale deployment with EdgeX would look.
Security Guidelines and Improvements: With each release, EdgeX has worked to improve our system security. In this release, we have provided several guidelines for how to improve the EdgeX security posture. For example, we offer a guideline on how to setup and use SSH tunneling (specifically for device service communications) or an overlay network for secure communications between services when needed. In addition to guidelines, we have made several improvements to security services. One such example is a new hook in the security secret store setup service which provides for hardware-assisted protection of the secret store master key when available. In this release, we also completed several important security feature designs that, while not in Hanoi, will show up in Ireland and Jakarta releases in 2021.
Device Service Contributions: during this release cycle, we have had several device service contributions. Many of these device services are not quite ready for formal release, but they are available in our GitHub repositories for exploration. We expect several of these device services to be approved and adopted by the community in the coming months (device services release separately from the rest of EdgeX services). New device service connectors were donated for LLRP (for RFID), CoAP, GPIO, and UART. An LLRP application service has also recently been donated.
Improvements Behind Scenes
Beyond the new Hanoi features and improvements, several other project improvements and ecosystem programs have been added with this release. These changes aren’t directly reflected in our EdgeX micro service platform, but they help improve the software quality, improve our development processes, or make adoption and use of EdgeX easier and better.
New User Experience Program: With Hanoi, the EdgeX community will shortly be announcing the availability of a new user program. In this program, users will attest they can get an EdgeX instance up and running, and have some familiarity with device profiles and getting data through the platform. The goal of the program is to provide awareness of users and their organizations that have EdgeX expertise while also promoting the sharing of EdgeX device connectivity elements (like Device Profiles) and sample data sets which can be used to accelerate adoption of the platform.
Canonical Management of Snaps: Canonical has been a great partner and participant in the EdgeX community. They have added immeasurably to the project in so many areas, and because of their open source experience, they have also provided the project with many lessons learned and guidance. Since the Dehli release of EdgeX, the community has published EdgeX snap packages. Snaps are app packages for desktop, cloud and IoT that are easy to install, secure, cross‐platform and dependency‐free. In providing Snaps, along with Docker images, EdgeX offers two examples of how EdgeX can be packaged and deployed. With this release, Canonical has taken over the maintenance and publishing to the Snap Store of the official EdgeX snaps. Transferring the management and publishing to Canonical is a meaningful change in that it signals Canonical’s continued commitment to the project as well as signaling that EdgeX is important to the edge/IoT communities of the Ubuntu world.
Web site refresh: The EdgeX Web site has undergone an immense refresh during this release cycle under the direction of our marketing group. The web site refresh helps to clarify the purpose and use cases of EdgeX, highlight the efforts of our community members, and will help adopters and users get familiar with EdgeX quicker and easier.
Introduction of the Adopter Series: During this release, a spot light was placed on organizations using and adopting EdgeX in their products and projects. In particular, throughout the summer, Accenture, ThunderSoft, Jiangxing Intelligence, Tibco and Intel all provided webinars on their use of EdgeX – highlighting why they chose EdgeX and what they hoped to see in future EdgeX releases. Additional adopter series presentations are expected from HP and IOTech later this year. This series has been instrumental in helping to drive more adoption, highlight real world EdgeX use cases, and provide critical feedback to the EdgeX community of developers.
DevOps Improvements: The EdgeX release is just the visible tip of a long arduous process of creating open source software. Behind the scenes, teams of people labor to create and test the software. And another team supports the developers that create and test the software. The EdgeX developer operation’s (DevOps) CI/CD processes are some of the most well-constructed and engineer-time-saving systems on the planet. The EdgeX DevOps team is the envy of the open source world and I dare say they would be the envy of most corporations. Intel has substantially led the EdgeX DevOps team for a few years now. In this release, they continued to add, improve and simplify the CI/CD process and tools. These are not elements that end users get to see. Inside the project, we appreciate how much more efficient it makes our developers and allows our project to add more features and fix more bugs.
Improvements in Software Development Processes and Tools: EdgeX Foundry developers take their craft seriously and try to improve the EdgeX product by always looking at instituting the best/latest tools and processes. During this release, a new process was instituted to vet 3rd party packages used by the micro services. The intent is to reduce bloat in the services as well as eliminate the use of poorly maintained or utilized 3rd party packages. In addition, a new tool was put in place to check any micro service’s use of a package (library or module) and notify the project (via automatic pull request) when a new version of the package is available. In this way, EdgeX hopes to keep on top of outside improvements and the evolution of software we use internally. Finally, we adopted the Conventional Commits Specification to help improve our git commit messages, which in turn we hope to use to improve our release notes and release information in the future.
Why Explore the Hanoi Release?
If you are already an adopter of EdgeX and you are using the Edinburgh, Fuji, or Geneva releases, migrating to Hanoi is straightforward as this release is backward compatible with any 1.x release. You want to move up in order to get the bug fixes and improvements without seeing any functionality changes or losses. Furthermore, the EdgeX community works hard to address issues, when possible, with the latest release. If you encounter an issue with earlier releases, the community will ask you to upgrade before putting a lot of effort in trying to address an earlier release issue.
Another important reason to download and use the Hanoi release is to start to explore the version 2 (V2) APIs that have been provided as experimental / beta APIs as part of this release. EdgeX is in the midst of a refactoring and improvement of our micro service APIs. These new V2 APIs, when completed, remove a lot of early EdgeX technical debt and will provide a better informational exchange as well as allow for many new, future release features. For one, the request and response object models in the new APIs are richer and better organized and these models will better support communications via alternate protocols (i.e. message bus versus REST/HTTP communications) in the future.
The construction of the entire V2 API will take us at least two releases. So, they are not complete and not provided for production level use yet (some may change and therefore be non-backward compatible). But with Hanoi, you can start to explore the new APIs and make plans for how the changes and improvements can be used in your solutions. We also hope it will allow our adopters and community to provide feedback on where additional changes or improvements are necessary.
On To Ireland
What’s next for EdgeX? Big things! We are in the midst of planning our Ireland release. It is tentatively scheduled for the spring of 2021. It is likely that Ireland will be EdgeX 2.0 – a major release – and will include a number of significant changes. First and foremost, it will likely contain our new V2 API set and V2 API testing (and allow us to deprecate the V1 APIs and older blackbox testing). It will include a number of security improvements. We are also looking at allowing device services to message application services directly (allowing for better quality of service when needed and bypassing persistence when not needed).
Given this will likely be a 2.0 release (and by definition contain some non-backward compatible features), we will also take the opportunity to sunset and remove several legacy services and items in EdgeX – like use of Mongo for persistence and Drools for the rules engine.
Thanks Keith
This is my first release as the Chair of EdgeX Foundry Technical Steering Committee. I have been with the project since day one, but before me, Keith Steele was our TSC Chair and I served as his Vice. Keith helped the project through 3 years of growth and 6 six successful releases. He taught me a lot and put this project on sound footings. And while he is not gone (he still serves on our TSC and is driving EdgeX outreach), the project is indebted to his service and leadership to the project. He has left very big shoes to fill.
But as I mentioned, this community drives me forward and is my stability. My job is just to try to eliminate road blocks and stay out of their way. It is a fun group of people to work with and collaborate. Adoption of EdgeX is growing (we now enjoy over 7 million container downloads). Our community is strong and friendly. Plenty of room to come join us!
