ARNEIS project journal - dev-cw11

As I announced in a previous post

Your team “B-AROL-O Bottling Systems” and project “ARNEIS: Automated Recognizer, Network-Enabled, Items Sorter” have been selected as one of just 50 teams to move on to Phase 2 of OpenCV Spatial AI Contest. Your team name and region are listed on the official webpage at https://opencv.org/opencv-spatial-ai-contest/​

Here is the report of our progress in the ARNEIS project after the end of CW11-2022.

What did we accomplish this week?

In Sprint dev-cw11 we closed 15 issues or Pull Requests.

Let's now dig into some of the major results we achieved during last week.

B-AROL-O Team at the OpenCV Weekly Webinar

On Monday, 2022-03-14 we got an email from Phil Nelson informing us that due to technical issues the OpenCV AI Game Show which was supposed to be kicked off on Episode 49 of the OpenCV Weekly Webinar had to be postponed to the week after, and proposing to fill in the empty slot and present ARNEIS instead.

After some internal discussion with Alessandro and Gianluca we accepted the challenge. Unfortunately we had only two days left for the preparation, therefore we had to cut corners and speed up our plan for filling the 1h slot of the webinar.

Moreover, due to work priorities Alessandro would not be able to attend in person, so we decided that I (Gianpaolo) would introduce B-AROL-O team and the ARNEIS project in the first part of the webinar

... then would leave the stage to Gianluca who would talk about the mechanical design and zoom into the CV algorithm and the current issues we are still trying to solve.

We hope that the people who eventually attended the webinar on Thursday, 2022-03-17 were not disappointed. As a matter of fact, by reading the comments in the recorded video (LinkedIn login required) it looks like our project was well welcomed.

According to the Zoom recording we had more than 100 people who watched the webinar:

Among the attendees we recognized a few friends and former colleagues; on the other hand, we were particularly amazed to see so many people from all over the world joining the event and making very interesting questions and suggestions.

We are very thankful to Satya and Phil who invited us to the OpenCV Weekly Webinar, and we are looking forward to more community contributions now that the project has received such exposure.

We also hope we will be able to meet some of you in person during one of next events in the coming weeks where we will be showcasing ARNEIS - more specifically:

• Saturday, 2022-04-09 and Sunday, 2022-04-10: San Mauro Brick Show 2022
• Saturday, 2022-05-28 and Sunday, 2022-05-29: Expo Mattoncino 6 Pinerolo

Please mark both events on your calendar and - if you happen to be around Torino in one of those days - don't forget to come and meet us!

Working on a synthetic dataset for training the ARNEIS NN

As explained by Satya during the 2022-03-17 OpenCV AI Webinar, the most effective way for improving the recognition rate of a Neural Network is to provide a comprehensive dataset for training.

As we could not find any public dataset for "mignon bottles" like the ones we use in the ARNEIS project, so fare we have been using about 500 real pictures taken with the OAK-D-Lite to a subset of the bottles we have available.

Satya also suggested to enhance the dataset by developing 3D models of each bottle, slapping hi-res scans of their labels, then use programs such as Unity or Roboflow to generate plenty of synthetic pictures with different orientation, lighting and obstactles in order to reinforce the training algorithm of the Neural Network.

Stefan Daniel Achirei commented in the chat

toolkit for generating synthetic data:
https://github.com/Unity-Technologies/com.unity.perception

This toolkit was presented in Episode 43 of the OpenCV Weekly Webinar a few weeks ago (skip to 45:28 of the video recording):

Please subscribe to ARNEIS#258 if you are interested in the process or can provide further suggestions or feedbacks.

Built a second ARNEIS MOC

We managed to get our hands on another LEGO® 42100 Set which we have used to verify the building instruction manual of the MOC design which Alessandro released last weeek.

After about 5h of work the MOC is complete

... and there are still A LOT of extra parts that we could use to improve the design!

Next week plans

Our plan for next week is available on GitHub: https://github.com/B-AROL-O/ARNEIS/milestone/10.

Thanks for reading up to the end of such long post!

If you want to help or just say hello, please get in touch with us through GitHub or Twitter.

Gianpaolo on behalf of the B-AROL-O Team.