🚀 Subjective Scouting: The Resurgence
Subjective scouting (making decisions on things that cannot be directly quantified) has seen a recent resurgence over the past few years — predominately through systems such as "Citrus Rank" from 1678. The idea has floated around in various forms and implementations over the DECADES.
However, as teams become more competitive and games become more complex, many teams are needing to shift to collecting QUALITATIVE attributes on teams because simple accurate QUANTITATIVE data no longer provides the strategic edge.
đź’ˇ The Evolution of Ranking Tools
Initially following the concepts of ranking teams on some metric, made popular by 1678, I have worked with some prominent members of the community on design and implementation of improved ranking tools:
Tools that can be used for free, by anyone, any skill level, any team size.
(Re)Introducing Pairwise
I am proud to re-introduce Pairwise, in its new guise as a sleek, qualitative-focused, standalone or complementary solution to handle qualitative data. Previous versions and the development of much of the current site can be found in this Thread
Pairwise was born of the desire to generate a ranked list for a given metric, based off of the concept of tierlists. (S,A,B,C,D, etc). The issue of creating an overall ranking is: not every team plays every other team, so we need to read between the lines with sparse data. Pairwise accomplishes this through some linear algebra - a relatively straight forward and easy to use process that I will address in another post.
The real value added proposition of Pairwise is the data collection interface, which we will explore in this introductory post. This supplies consistent integration with publicly available data (TBA/Statbotics) and a customizable interface for ranking teams based on your chosen criteria with as few as one scout or as many as your heart desires.
When collecting qualitative metrics in this manner with multiple scouts, ranking teams encourages discussion and critical thinking - a skill that vastly improves the efficiency of scouting/strategy meetings both with alliance partners and before alliance selection.
What does Pairwise do?
Pairwise allows for custom tierlists (yes, plural), a rich note-taking interface, data sharing via sharable links, and qualitative data visualization. In addition there are some separate standalone tools (e.g., scout scheduler) that some teams may find useful.-
Tierlists: Users can chose to create rankings on an alliance-by-alliance bais (e.g., compare red robots) or a match basis (e.g., compare all teams in the match).
-
Notes: Notes for a match can be added several ways, through a rapid noting interface or intermittently as you rank teams in tiers. How/if you create notes is entirely up to you.
-
Tags: To make sorting through teams easier, there is a robust and customizable interface to add tags to teams match by match. This ensures consistency in how robot attributes are applied in match so the data are easily read, queryable, etc.
-
Data Sharing: Custom links allow you to share a read-only copy of data that are updated in real time. This can be of particular value to drive/strategy teams.
đź“š So, How does it work?
I am in the process of flushing out functionality and developing documentation, however Pairwise is relatively straightforward to use.
There are a few constraints you should be aware of from the get-go:
- Internet connection required - this may be a deal breaker for some teams, but this was a conscious choice. With ever increasing cell service and number of smart phones, the new robot radios, and limited developer resources, this was a needed tradeoff to meet functionality goals.
- Google account required - Data are stored securely in the cloud, protected by your google account. I choose google authentication because nobody wants to make more accounts.
With that out of the way… drum roll please!
Setup
Pairwise has minimal setup requirements and has been designed to accommodate as much pre-setup before a match schedule is released from an event.
Logging in brings you to a user dashboard, where configuration of tierlists and events are available, (there may be more user specific tools in the dashboard in the future).
Entering configuration you are presented with four key areas.
- The options sidebar - this contains event selection
- Configuration templates - here is where you make/edit tierlist configs
- Event data - Here is where created events will appear and you can access their data
- Archived events - this is provided mostly for event organizational purposes and will likely not see further development for the remainder of the 2025 season
To create configurations, events, and see tips and tricks there is a tutorial here. At its most basic you can create a tierlist (or multiple) to rank teams on a rubric/metric. At the end of the day we want to have something that shows “skill” and is not directly quantifiable. “Field Awareness” is a good one as it shows a driver reacting to the state of the field and aversion/acceptance of risk. Tierlist creation is straight forward and looks like this:
Tierlists configs can be edited as you see fit before you create an event with one, any events created will have the current state of the tier config “baked in” so be sure your tiers are how you like them! If you have multiple configs you will need to “activate” the one you would like to create an event with.
Once and event is created (via having an active config and event selected in the sidebar that has a published match schedule (use 2024 for testing, as of writing there are no 2025 events with published schedules)) you can bring up all the options for it by right clicking (long press on touch screen)
Working down the menu, we can edit tags that are available to be placed on teams, these are categorical and can be color coded, unlike the tierlists tags can be updated mid event:
Some teams have expressed interest in using this tag system for a certain level of autonomous tracking, it may be developed slightly further to better support this.
Now, On to Data Collection
You can get to data collection through the event context (right click) menu, or through the Main Menu -> Tools -> Data Input. You can select/change events using the sidebar.
There is a bit of information here. The search at the top allows you to filter by match or team number (e.g., qm5 or 9271). Below there is a list of matches, these include: match number, teams, tierlist completion, and the status of notes/tags on teams.
Right clicking on a match to bring up its menu reveals two options, rankings and notes mode.
Rankings:
You can drag teams around into tiers as you please.
right clicking on a team will bring up the ability to add a note, and add tags - this is a nice little shortcut.
(Note: there are a few bugs with drag and drop, especially on mobile, these are getting fixed)
Notes:
The notes mode was heavily inspired by team 111's Wildrank system. It consists of a grid of teams. the user experience with a touch screen and keyboard makes this an exceptionally fast method of adding notes for a team.Clicking (tapping) on the tag icon (near the team number above the team’s text box) brings up the tag manager for that team in that match (also accessible by right clicking on a team in ranking mode). The manager display the tags in the categories and order we set up earlier. Tags are toggle on/off. The search bar allows for rapid filtering.
Sharing Data:
Finally, going back to our config page, right clicking (tap and hold) on an event, and creating a sharable link for an event. From this URL we can bring up the summary statistics for the notes organized a few ways, allowing for filtering and examining our match data in a variety of ways. This is somewhat of a work in progress and likely will change a little bit throughout the season.
I hope you find this interesting and worthy of a follow (bookmark).
I will cover the creation of processing the ranking data to get a tierlist in another post soon.
Best and with love to the scouting community,
- Skye
