Exploration and Practices of Edge Computing: Cloud Managing Virtualized Devices

Written by Gavin Lu, LF Edge member, EdgeX Foundry China Project Lead and R&D Director in the VMware Office of the CTO

As an industry leader with vast experience and knowledge, Gavin has been writing a series of articles focused on edge computing. These articles are posted on his personal blog and are posted here with his permission. To read more content from Gavin, visit his website.

Introduction to the Architecture of Pallas

The previous article introduced how to build and install virtualized devices, but did not touch how to manage large-scale virtualized devices from the cloud. This article introduces the architecture of Pallas to achieve the above goal.

Pallas is the second milestone of the project Asteroid after Ceres. In this release, the basic problems of cloud managing virtualized devices are solved:

The device-cloud connection is narrow and unstable
Large scale of devices
Serious security concerns, often forbidding any open ports

The main functions of the Pallas architecture are implemented by the device manager and the agent virtual machine. Its design key points are:

Adopt MQTT protocol suitable for narrow-band, unstable network connections
The device manager in the cloud adopts the design concept of typical Internet architecture, with multiple layers, micro-services, multiple buffers, and read-write separation
The device is automatically registered to the device manager, and the device initiates all connections and response happens in the cloud
Close all ports on the device, run without VPN / SD-WAN, cross public networks
Each device has a randomly created, globally unique and permanent ID

Installation requirements

vSphere Hypervisor is installed on the device.

- Device agent
  - CPU: 1 x86-64 vCore
  - Memory: 512MB
  - Storage: 5GB

Device manager
- CPU: 2 vCores
- Memory: 8GB
- Storage: 100GB
- Network: Open ports 443 and 1883, the device is visible

Note: In order for the agent to work properly, the device requires at least vSphere Essentials Kit license, or apply for the Enterprise Edition 60-day trial using the method described in the previous article.

Download and Installation

Download Pallas

The Pallas installation package can be downloaded from the Flings website of the VMware CTO office. You need to register a VMware community account in advance. The download package contains the device manager OVA, agent VM OVA and user guide.

Note: The download package provided on the Flings website is a technical preview, which does not include commercial support. It is recommended that you carefully read its installation and user guide before initialize the installation.

Install device manager

The way to install device manager is the same as the typical way to install an OVA of virtual machines, which can be completed in sequence by referring to the steps in the Pallas installation guide.

Note: Although device manager is packaged in OVA mode, it does not depend on any specific virtualization infrastructure or cloud platform. In an alternative, it can be converted from OVA to other formats, or install on any cloud platform that supports OVA format.

Install device agent

In order to simplify the process of installing device agent, it is recommended to use the virtual machine OVA instead of the binary package.

You can use OVF Tools to install OVA remotely, or leverage ESXi UI to install directly as below.

ovftool –acceptAllEulas –name=pallas_agent –datastore=DATASTORENAME -dm=thin –X:injectOvfEnv –powerOn pallas_agent_ubuntu.ova ‘vi://USERNAME:PASSWORD@ESXIHOST’

Configuration and Use

Configuration

Before you start using it, it is critical to configure the device agent. In order to ensure communication security, remember to modify /etc/vmware/pallas_agent/pallas_agent.conf file before encrypt the file in the following manner.

python3 /root/agent/install/encrypte_password.py YOUR-PASSWORD

If the device is connected to the cloud via WiFi or a telco carreir mobile network, the corresponding PCIe or USB NIC needs to be passed through to the device agent virtual machine. In this way, the device agent virtual machine can be registered to the device manager.

Use

After the device is registered to the device manager, you can perform CRUD-like operations on users, devices, and virtual machines like other ordinary management tools.

It should be noted that because the communication protocol of metadata between the device manager and the device is based on MQTT, the status updates of the device and the virtual machine on top of it are completed asynchronously. If you don’t find a task completed in “real time”, you may need to wait for a while or refresh the status.

For tasks such as deploying a virtual machine or patching a device, large files are downloaded via HTTPS and auto resuming.

To complete all functions above, there is no need for mutual IP visibility between the manager and the device, nor the installation of any VPN or SD-WAN. The device can also be safely behind a firewall, NAT, or gateway.

With the support of these basic functions, it is easy to expand, manage large-scale virtualized devices from the cloud, and deploy edge applications like EdgeX Foundry framework on them. In the workshop on EdgeX Foundry China Day in December 2019, we have demonstrated the deployment of edge applications based on EdgeX Foundry framework with cloud management of virtualized devices.

The previous two articles described how to build and install virtualized devices, and how to manage virtualized devices from the cloud.

In the preface, in addition to virtualization devices, another solution is containerized devices.

In fact, the use of containerized equipment is very common, and it often implies local orchestration, cloud operation and maintenance.

Local orchestration means that single-point failures of devices cannot be handled well. Even the most common deployment approach of EdgeX Foundry is to deploy core container instances of microservices on one single device.
Cloud operation and maintenance means managing containerized devices from the cloud. Most manufacturers have their own specialized solutions, which brings another problem of technology fragmentation.

The solutions to these two problems will be discussed in subsequent chapters. The next article will introduce a cloud managing containerized device solution to solve the problem of fragmentation.