Let's talk about problematic QA questions

Some QA questions are better than others.

Here are some quick techniques to test whether you’ve crafted a QA question worth posing:

  1. Does your question cover the basics, including a good title, rules references, good grammar and no mispellings?

  2. Does your question leave room for the quickest and easiest non-answers? If these common responses can be legitimately applied to your question, then it will probably happen.

  • We cannot rule absolutely on hypothetical ROBOT designs, and the final decision as to legality of a particular ROBOT lies with the Lead ROBOT Inspector (LRI) at each event.
  • We believe Q# answers your question. If it does not, please rephrase your question and resubmit. Please use the Search Q&A button on the navigation bar before posing a new question.
  • We cannot comment on part/design legality, previous calls of Head REFEREES or LRIs, or questions that are overly broad, vague, and/or include no rule references.
  • Official Field Drawings can be found in the Playing Field Information link on the Game & Season Materials webpage.
  • As per Miriam-Webster Dictionary, a [word] is [definition]…
  1. Have you workshopped your question with other mentors?

  2. Does your question carry the risk of making something illegal for all teams? This is especially important if calling a commonly-used part or a standard practice into question. It also depends whether your question is in regards to a prescriptive vs proscriptive part of the ruleset. Please move carefully here. You should first discuss the reason for your question with your local FTAs and LRIs who attend official FIRST-provided training for their roles.

The right way to design a question is to imagine the response you want, and design the question such that no other possible answers can be legitimately given. This requires putting yourself into the GDC’s shoes, adopting their logic. You’re not an FRC team-- you’re an organization responsible for developing guidelines for the safe and “fair” execution of hundreds of robot events across the world.

The ideal QA response is something like:

Correct.

Working backwards from this perfect answer, a good question could have a structure like:

Is it correct that [doing action] is not a violation of G#?

Q148 and Q156 (which were answered at the same time) are examples of great questions of this form, because they adopt & leverage logic that’s already demonstrated in existing rules and prior QA responses.

It may be tempting to ask something like “Is there a rule to prevent [action]?” or “Is there a rule that makes [design] illegal?” I recommend avoiding this format, because it’s so likely that this will (1) solicit an easy, glib response (like “Yeah, there are a lot of rules, pal!”) or (2) unintentionally spur the creation of a new rule (or a new interpretation of an existing rule) that prevents the thing.

The question “Is there a rule that says we must [do action]?” is also problematic in the same way. If there was such a rule, we’d know about it already.

My advice is that your question should not attempt to cause a rule change, although this does happen frequently. If your intent is to cause a rule change, (1) your question should leave room for no acceptable answers under the existing ruleset, and (2) you should have a really good prior understanding of what the new rules will be after the change! This part may require being around FIRST for a long time.

OK, so let’s talk about some problematic questions.

Q27 and its followup Q57 both ask the responder to make a moral judgement about the fairness of a particular action not restricted by the rules, in an attempt to make it illegal. Q27 has the structure “Is there a rule stating we must [do action]?” and Q57 has the structure “Is there a rule preventing us from [doing action]?” Both of these structures can only serve to solicit either new rules or more restrictive interpretations of the existing rules; they’re not designed to bring about any loosening of restrictions.

Q149 has a couple of problems:

  1. It asks the responder to make a legality determination about a particular robot part, but the existing rule has already left this to the Robot Inspector to determine. This gives GDC no choice but to respond with “the final decision as to legality of a particular ROBOT lies with the Lead ROBOT Inspector (LRI) at each event.”
  2. The questioner gives their opinion (the LEDs on this particular robot part are “intense”, therefore its use during match play is subject to the safety-related restriction applied to all “intense” lighting) and then asks whether their interpretation is true. If they wanted to hear the unlikely response “No, the part is not subject to the use restriction,” they’ve set up an additional difficult barrier for themselves: GDC would have to say that they disagree with the team’s opinion.

.

If they can get you asking the wrong questions, they don’t have to worry about the answers.

–Thomas Pynchon

14 Likes

chute door

12 Likes

There are some situations where you can’t craft a Q&A to get a particular response, like the hatch panel stuck between two hatches. It’s obvious that it should only score once, it’s probably only going to count for one of the hatches/bays, but it’s really up to the GDC to figure out exactly how to parse that situation.

Maybe somebody wants to ask a question that leads to a more restrictive interpretation. That does not make the question “problematic.”

4 Likes

I mostly agree with OP, but the key thing to my mind is that you force a “real” answer. At the first level, this means ensuring that your question is not going to draw one of the list of stock un-answers above. At the next level, it means presenting a clear and specific setup which “zooms” right in on the issue and doesn’t get caught up in non-essential details (which again, is likely to draw a stock un-answer).

4 Likes

Yes

The correct answer is Yes, CHUTE DOOR.

2 Likes

I know bro, none of us will ever forget that! I laughed for a week

I still laugh every few months. It’s better if I’m at home or driving. Kind of freaks people out at work.

1 Like

Just going to leave this here.

https://frc-qa.firstinspires.org/qa/178

5 Likes

Someone on 449 was asking about that just last week. I expected them to ask, not 900…

I think you’re missing the obvious, which would cut the number of questions to less than half:

  1. Have you read the entire rule book yet?
4 Likes

RE QA 178: You should also include fair market value for use of the land on which the tree was planted, based on the minimum legal lot/parcel size in effect where the tree was planted, or on a circle with the larger of the root or crown diameter of the tree just prior to harvesting if larger than the minimum legal lot size. :wink:

2 Likes

Looking at the Q&A answers, I found one I disagree with (probably should generate a rule change rather than a change in answer). In particular, Q170.

1 170 Vacuum Storage

Is storage of vacuum on the robot allowed? It would not really be pneumatic storage by the usual definition. Of course, the vacuum pump would need to comply with the requirements (like an allowable motor).
asked a day ago by FRC 3049

Answer

There are no rules that specifically prohibit storage of air under negative pressure.

To my thinking, a “charged” vacuum tank at the beginning of the match constitutes a non-electrical stored energy source which currently violates R46. Someone should ask specifically if a tank containing a partial vacuum/negative gauge pressure at the beginning of the match violates R46.

Lord, forbid that the $400 piece of robot you bought was deemed illegal… by the rules.

An admirable goal right up until a poorly thought out Q&A question and a poorer thought out answer makes batteries count against withholding allowance until the next rule change.

If your goal is to get the more restrictive answer I’d apply the same logic - ask the question in such a way the ONLY answer is the one you want.

Honestly, that’s just good life advice in general, restrict possible outcomes to those that you desire to be acceptable (within the letter of the law if not the spirt… who can tell I’ve been spending time with 900?)

Remember when it turned out that pretty much all programmable electronic components were illegal if they were used before the season? Imagine if that answer came out during competition season.