I was looking at scouting data this morning, and a part of our system pulls data from first-events/TBA for climbing. I found out that some teams that, in my memory, couldn’t traverse at all but magically had traversal climb datapoint. The reason for this is that first-events had the station positions wrong for a few matches. For example, for Qual 9, they flipped the position of 7407 and 6690. How can we fix/compensate for this error?
It also will happen if someone breaks a rule that results in a traverse climb
4 Likes
I noticed another inconsistency at the South Florida Regional where 179 is listed as having a record of 12-0 despite having lost the first match according to the FIRST event API. There may be even more inconsistencies like this that could be affecting the rankings at events, so I hope FIRST looks at/fixes this soon.
2 Likes
Q3 was one of the matches replayed, interesting that the API doesn’t show the updated data yet
We were competing at NE Granite State. One of our scouts went through all the match recordings of the first day (up to qual 59) and found that it is a consistent issue that the climb status of the Red 1 robot and the that of the Red 3 robot are swapped on the TBA data. Here is a list of matches with obvious wrong data. You can verify yourself with the twitch replay. Is this an isolated issue for Granite State or a common issue for the first-events/TBA system?
1 Like
Climb data is populated by what referees enter on the field-side tablets (or occasionally what the scorekeeper enters after the match has ended). The FRC API makes that data available, and TBA pulls from there. It’s possible that a referee hit the wrong button on the field, for instance, and a mistake like this may not have been corrected even if caught because it doesn’t affect the outcome of the match.
1 Like
Waco had some wack API issues— qualification 22 had the correct score displayed after the match, but then the wrong data was somehow given to the API that feeds into TBA. They ended up having the match replayed, but the data feeding into TBA is still wrong showing that Red scored only 4 points and lost, and the Red alliance ended up actually losing an RP from having the replay it. Weird situation all-around and hoping to hear from HQ on this soon.
2 Likes
You may also be interested in the behavior of FRC events. Specifically, the qualification schedule lists the 4-point score you mentioned, and the match breakdown for match 22 404s (which 21 and 23 do not). I would guess (noting that I was not at this event and have not looked at the stream archive!) that the published match results here are for the first play of the match, while rankings would be determined by the last (usually second) play in the case of replays.
1 Like
TBA, as a rule of thumb, tends to reflect the data published by FIRST on FRC-Events. There seem to be multiple inconsistencies reflected on the FIRST side, which would lead me to believe there to be more on the TBA side as a result.
Edit: Speaking as a TBA data mod for these circumstances.
1 Like
Something like this was going on before the season as registration was going on too. I’m pretty sure FIRST has at least 3 different APIs that can return different data at the same time for the same thing.
1 Like
At FiM Detroit, at one time the RP’s being shown on the monitors in the pits were not what was scored and shown on the screen at the field. They later updated to be the correct values, without any intervention by the scorekeeper.
1 Like
I was actually one of the two scorekeepers at Granite State and can tell you that we and the Head Ref were visually verifying - every single time - that what we were seeing recorded on the scorekeepers screen for climbs in both hangars was accurate before submitting the data. I will obviously escalate this discrepency asap.
Note that flipping which robot gets credit for the climb doesn’t change the overall match score, but could affect the stats that are used to break ranking point ties in the rankings. (Honestly, I don’t know offhand the order of use of the stats to break those ties to know where climbing comes in.)
Also note that there were 10ish times the Head Ref directed the scorekeepers to award a transverse climb to a team that didn’t accomplish it because of opposing alliance contact in the hangar during endgame.
1 Like
Concur, the API’s can never be perfect. They are good for general overall stats, because they can be mostly correct. The only way to “mitigate” that is visual tracking, plus you may never know the result of scorekeeper changes as directed by the Head Referee after the match.
There’s also often a lot of confusion among spectators over which stations are 1 and 3, which I want to verify again
(edited for clarification, my apologies)
This alarms me. Every field is the same, and if the field staff doesn’t know, they should probably review the documentation. I’ve only seen this confusion exist amongst teams, and usually a couple matches queued solves the problem.
My apologies, I didn’t mean to imply confusion among field staff. I meant spectators only. My bad.
While it doesn’t affect the outcome of the match, it does affect the part of our scouting system for climbing, and it will be nice if it’s correct. Because this error of flipping Red 1 and 3 seems to happen for all the matches (we only just noticed the obvious ones), I’m wondering if it wasn’t humanly error made by the scorekeeper/refs but one hidden inside of the API.
I can say that the Granite State climb scores for both red and blue were visually verified for team # and climb level by the head ref and scorekeepers for every match before scores were submitted. The only differences were when the head ref awarded a climb for penalty contact in the hangar. I have no authority or ability to comment on what might or might not be going on behind the scenes in the software. But I did escalate the question.
1 Like
4414 awarded traversal hang score adjustment while G108/G208 conflict is is addressed. In Port Hueneme
I also saw some incorrectly scores taxi points when looking at the data from Hueneme Port. Teams that had complicated auto routines that did leave the Tarmac, but returned to it at the end of auto, didn’t get the Taxi points they deserved.