Written by Sangeeta Ghangam, Platform Architect in the Retail-Banking-Hospitality-Education (RBHE) Division, Internet of Things Group, Intel
On October 7-8, 2019, I participated in the first LF Edge EdgeX Open Hackathon focused on Retail use cases. I had some exposure to EdgeX Foundry before the hackathon but it definitely did not prepare me for the experience that lay ahead. I normally prefer planning my work travel & technical sessions to the point where I know exactly what’s on the agenda, what needs to be done prior to the session and how it can all come together. However, this time around, I had no clue what we were going to do and how we were going to do it. I knew a few of my colleagues were on the team but we don’t work together on most of the projects so it was difficult to know who would do what, plus we had one of our customers/partner (Intuiface) on the team as well. To top it all, pre-work was allowed but, unfortunately, we didn’t get a chance to do any before hand.
Sounds like a disaster waiting to happen, right?! But it was a perfect way to experience the hackathon. The objective of the EdgeX Open was to introduce the EdgeX Foundry infrastructure to the participants – a mix of folks from all backgrounds and expertise, for feedback both technical and from a go-to-market perspective. On the first day, we were told about all the different software and hardware tools available for the participants – from compute systems to sensors to cameras and retail products. We also learned more about the EdgeX Edinburgh release, which was a significant milestone for the software, available as a docker or a snap package. We spent the first 2 hours after the initial presentations just fiddling with all these different options – install EdgeX, rummage through the sensors etc. Finally, we found a camera we thought we could use in addition to the bonus camera that was setup as a mock check-out lane. Now that we got comfortable with the tools we needed a use case or a problem to provide solution for. We considered the topics or themes which were part of the hackathon, the wildcard appealed the most. And a “real” problem surfaced as well – customer friendly checkout in the age of automated & self-checkout?
At some point or another, we have all experienced frustration as we tried to self-checkout quickly resulting in delays, waiting for help, etc. We figured we could use the sensors and camera to gather data, feed it to EdgeX and generate insights that a store associate can use to provide assistance where needed. We decided to split the solution over 3 systems – one running EdgeX, one running analytics over Intel Openvino and another running the dashboard (for the store associate) and use the following list of sensors:
- 2 Cameras – One at our desk and the bonus camera representing the mock check-out lane
- Nexmosphere sensor kit consisting of RFID sensors and LED sensors. We planned to use the RFID sensor to track the product at checkout and the LED sensors to provide a visual status of checkout lane to the store associate.
The multi-system and multi-sensor setup mirrors what is typically found in Retail Stores today which is where EdgeX infrastructure can work as a data highway. We split the work on the 3 systems amongst the 5 team members and started in earnest around mid-afternoon on the first day. I remember we finished 70% of the work on the first day and about 30% on the second.
We had several learnings along the way as we experienced the different software and hardware pieces:
- There is more than one way to install EdgeX on the target system via containers or snaps. Since we were comfortable with containers, we used the same and did not run into any significant issues during setup.
- We ran into some hurdles while using Ubuntu 18.04 for Intel Openvino installation. Turned out Ubuntu 18.04 changes how the mirror sites work for downloading dependencies, reinstating these settings seemed to help.
- It was difficult to make changes to the device profile in EdgeX to add additional variables for the new sensors we wanted to add to the system. It required multiple re-installs of the EdgeX system, I believe this is a known issue and only way to work around this is to purge the database and/or re-install.
- We also wanted to showcase remote management using Intel vPRO but ran into issues trying to use the snap version of the software.
We were able to work through items 1, 2 and 3 with help from the expert team from Intel but had to abandon item 4. The final solution looked like the following:
We had System 1 running camera analytics using Openvino such that any negative emotions detected in the customer were triggering a ‘1’ to the EdgeX MQTT device service. We added two additional device services to receive product information via the bonus camera setup (this was the surprise element in the hackathon and had bonus points if included in the solution) and the RFID sensor. The dashboard created using Intuiface REST API tools enabled a real time view of the sensors. Depending on the product and the customer emotion analysis it would provide real time chart of the check out lane. If the customer exceeded ‘x’ amount of time with neutral, disgust or anger (any negative emotion), the software on system 1 would change the data from ‘0’ to ‘1’ to the MQTT device service. The custom software we wrote to extend the device and cloud services sent this data to the Nexmosphere sensor management software so the LED’s would turn ON. Given the real time chart and the LED status, the store associate can offer to help the customer before being called to help.
On day 2, we explained our motivation behind the work to the judging team ahead of the formal presentation. It was great to hear their questions and understand their perspective, we did not realize we ended up creating a solution that could support multiple use cases. I remember we titled the work and selected the use case ‘Enhanced Customer Experience during Checkout’.
Throughout this flurry of activity, we had opportunities to network with other teams, look at some of the cool work they were doing. We also interacted with the experts from LF Edge member companies like Intel, Dell and Canonical, which gave us first hand opportunity to provide feedback towards the next revision of EdgeX. The hackathon was very well organized, had great food and even included a live music jam session on day 1!
Although it was a new venue and a new team, it all came together in the end. Team Intel/Intuiface won second place!
Stay tuned here for more details about the next EdgeX Open Hackathon! If you missed the EdgeX Open, click here to watch the video from the event.
