Programming / AI Machine Learning & Vision Tracking -
At the end of week 2, our programmers worked on learning more about setting up AI machine learning and vision tracking for our robot this season. Our programming mentor reviewed several resources and the process for setting us a system for vision tracking, creating data sets, and using data sets to create an AI vision model for our robot.
We developed the following notes from this discussion and work:
NVIDIA Jetson - NVIDIA Embedded Systems for Next-Gen Autonomous Machines - this is the hardware we use for our vision system - our camera plugs in to this on our robot
Jupyter Notebook - https://jupyter.org/
Roboflow - https://roboflow.com/ for machine vision learning / training; lots of data and images already available there from FRC teams - used these to help train our model - to create a data set for training need to have 100s of images with various lighting and setting and orientation; you can upload your images here, draw a box around each object and label each object to create a data set for training your model (images recommended size 640X640)
Ultralytcs YOLO - You Only Look Once documentation - https://docs.ultralytics.com/
(create your own AI model - they also have pre-trained models for everyday items)
Choose “quickstart”
Jtop - on Jetson give info about how much memory is being used - may not be able to train the model / create the model on the jetson - our jetson did not have enough RAM to create our model - we used google colab to create it.
Google Colab - https://colab.research.google.com/ - to train the model - T4GPU (free) for more memory to complete the training - allows you to write and execute Python in your browser, with:
- Zero configuration required
- Access to GPUs free of charge
- Easy sharing
Tips/steps for Vision Training & Modeling
- Import the photos to RoboFlow and then draw an outline around each object and label each to use train the computer to “learn” what it means to be that object
- Download dataset from robo flow - as zip file to computer - then extract - will create 3 folders - test, train, valid
- Use Ultralytics - YOLO to create an AI model using the images that were labeled
They also use pre-trained models - find the code on yolo documentation
- Use Google Colab to create the model with the code from YOLO and RoboFlow data if you don’t have enough memory on your device
In your robot code you need to be aware of confidence level for identification (we used > >78% on our test code)” boxes.conf > 0.78 “ in our code for the jetson to identify objects on the field
We are just starting to dive into this topic and learn about machine learning and vision tracking so we would welcome any additional tips, suggestions, and resources.
