Hi Everyone,
This year I was one of the lead scouters for team 2468 and I wanted to show off the system we created for the 2023 FRC season and the tools we used to make it.
This year we were able to produce better data and visualizations than we have in our team’s entire history. Over the years, 2468 has attempted to make a useful and consistent scouting app many times, from using paper forms, to using QR codes, to making an entire app with cloud storage (we had a talented software mentor that year). However, none of these solutions have given us the quality and features we wanted. Furthermore, we even had competitions where all of our data got deleted due to programming errors, which made it impossible for us to make a proper picklist. We needed a system that not only provided us with useful visualizations and priceless data, but also tied back to the reason we scout: to gain a competitive advantage. In our 2023 season, we finally accomplished this by partnering with Real Life Data Inc. (RLD), a company started by a 2468 alumnus that specializes in data collection and visualization, to test their application’s alpha release. Our app worked from our first district competition all the way to Champs in Houston, collecting about 120,000 data points along the way.
2468 Scouting System / How we used RLD in 2023
Data Collection
We worked with the Real Life Data software to tackle the difficult questions such as: “How can we collect data effectively?”, “How can we limit the error rate of our data collection?”, and “How can we visualize the data so that our coaches, drive team, and team members can understand with ease?” To answer these questions, we began by thinking “How can we create an input form that gathers as much data as possible while seeming intuitive and easy to use?” We ended up creating a detailed, aesthetic app that is mobile phone capable. We tailor-made it to the experience of the user and went through many different scenarios of how our scouters would go through collecting data for a single game.
(Sample of our collection app as rendered by the RLD form creation tool)
The form was easy to load up, share with the 50+ people on the scouting team, and submit regardless of the cellular connection at the venue with offline data storage. Earlier, we either relied on others having good enough cellular connection in the stands to ensure all the data we wanted got collected. With the new app, advanced scouters could manage the new scouters with ease, and work on collecting qualitative data without having to worry about the quality of the quantitative data. This helped us make our quantitative scouters feel more valued in the team because they knew that their actions made an impact on the performance of our team. In seasons prior, many of the people who scouted for us felt bored scouting and saw their scouting impact as meaningless. After using the new program for just the 2023 season, one of our scouters said that our app “made that larger, more flexible scouting possible. Students didn’t have to have the most intimate knowledge about the game and only a quick explanation of what they needed to do.”
Data Calculations
After all the raw data from the competitions was collected and stored, we needed a way to convert the raw data we had into meaningful numbers and insights that could be visualized and plotted on tables and graphs. To do this, we created calculations within RLD that computed averages, weights, and other stats that allowed us to easily see the performance of teams over time.
(Sample of our weighed total calculation, which helped us calculate an effective rank of a team in order to see how well they would work with us)
These calculations were crucial to our success because they allowed us to turn the raw data that we collected into meaningful insights that our coaches and team members could understand. With the integrated calculations, we were able to devise complex formulas and analyze different aspects of the game to a higher degree than ever before. This data was also exportable in CSV format to other software such as Excel and Python, which helped us combine our data with existing systems such as Elo to deeply analyze the performance of a team.
Data Visualization
When matches came up, we were easily able to transfer what data we had on the competing teams over to the drive team through tables and graphs we created beforehand. All our scouting team had to do was send a quick text over to our drive team with screenshots of the tables and graphs so they could have a summary of who they were going up against. They were even able to log into their RLD account and see it themselves whenever they wanted.
(Sample of the tables and graphs that we used to visualize how teams performed)
All these visualizations worked on phones and laptops
When alliance selection neared the horizon, and our scouts needed to present the data we had to make an informed picklist, we created informative dashboards based on the data to present all the information to our coaches quickly and easily. These dashboards visualized important aspects of each team such as average cycle time, cone or cube preference, and driver skill. We wanted to see their basic metrics as well as their performance over time so we could judge the teams that were going to start underperforming versus those that would start overperforming. We even had multiple tabs in one dashboard so we could evaluate teleop, auto, and endgame performance all in one place.
(Sample of our alliance selection dashboard. The pictures included here are only a few of the many visualizations we made.)
Conclusion / What’s next?
In our 2023 season, we have significantly improved over the past scouting systems that we had in place. With an intuitive app designed for the user experience, we improved our abilities to collect and store data, as well as visualize and present our data to our coaches in short yet meaningful scouting meetings. In earlier years, having our visualizations made little to no impact on our scouting meetings due to issues with data collection. Furthermore, the calculations that we implemented were limited at best, and could not compute the complex formulas that we used for this year. With all of that in mind, one of our quantitative scouters noted that “our 2022-2023 season was much more organized”. For people who have barely seen the app, they were amazed at how it was “very easy to operate on the user end, making scouting a whole a lot easier and stress-free.” And even for our experienced scouts, making critical changes to both the input form and the charts/graphs was easy “even in the middle of a tournament, making it adaptable to suggestions from the scouts”.
As the off-season approaches, we hope to work closely with Real Life Data to keep improving their platform, as well as testing new features such as radar charts and advanced heatmaps. We also want to integrate the FMS system into the Real Life Data system and add extra data from External Data Systems like TBA and Statbotics to complement the data that you may collect. Last but certainly not least, Real Life Data will be integrating AI into the data visualization process, helping to analyze qualitative data and summarize the best findings, while also working with quantitative data to find trends that may uncover teams that fly under the radar of a normal scouting system.
Special thanks to our coaches and mentors who guided us and provided us with valuable knowledge throughout the season. We’d also like to extend our regards to Real Life Data for allowing us to use their software for free, and improve our scouting system to achieve one of the best seasons we have had in the history of our team.
If you’re interested in how Real Life Data works, feel free to check out their website here: https://reallifedata.net/
Real Life Data has information on how to use their platform for scouting, check it out here: https://reallifedata.net/FRC