Ruling on Robonauts Balance

[Tristan Lall]

Originally Posted by wireties

sometimes a bad thing—or at least not uniformly a good thing - really?

One could make that statement about all matters involving one or more humans - it means nothing.

Here's a very straightforward example. Sometimes it is overall less economically efficient to expend resources in a competitive bid process, than it would be to simply award the work to a particular vendor. Tendering it is therefore a "bad thing" under those circumstances. (I said not uniformly good, to qualify the remark, because perhaps some good can still come out of a net loss.)

I gathered from your previous post that you were describing the similarity to a competitive bid as an example of FIRST mirroring good real-world practice. I'm merely saying that competitive bidding isn't necessarily something worth emulating in a robotics competition, because it is not a good thing a priori. (And yes, you can say that about anything—that's why the discussion needs to go beyond what people usually do in the real world, and instead cover why they do it, and what the consequences are.)

Originally Posted by wireties

That is not the net result. Performing for a grossly inadequate rate puts one out of business and/or likely not to make the same mistake again.

Sure; but that's not really a point of contention, is it? One can work to change others' perceptions in any situation.

Originally Posted by wireties

And in FIRST, the troll-bot teams will likely modify their risk assessment methodologies next year.

They probably will. But was teaching them the lesson in this fashion productive?

Do they have reason to feel mistreated or misled? Does it matter to FIRST that they might feel that way, as a result of FIRST's statements? (And should it matter?)

[wireties]

Originally Posted by Tristan Lall

Here's a very straightforward example. Sometimes it is overall less economically efficient to expend resources in a competitive bid process, than it would be to simply award the work to a particular vendor. Tendering it is therefore a "bad thing" under those circumstances. (I said not uniformly good, to qualify the remark, because perhaps some good can still come out of a net loss.)

Agreed - this may be more efficient in isolated instances and with all parties acting benevolently. But as a system, it cannot function. Somehow and somewhere, somebody is making the decision. In a non-competitive system, the criteria are hidden thus possibly (and probably) unfair. In my opinion, if the system is not fair and competitive, the system is unworkable.

Originally Posted by Tristan Lall

Sure; but that's not really a point of contention, is it? One can work to change others' perceptions in any situation.

It is not contentious at all. It was a simple statement of fact. What happens to parties that under-bid and/or under-perform? The system consumes them, as it should. They do not get a chance to repeat their mistakes (often) as you implied.

Originally Posted by Tristan Lall

They probably will. But was teaching them the lesson in this fashion productive?

It was harsh. I wish it had not happened. But it was definitely productive. The troll-bot devotees were not harmed by FIRST. They took a HUGE risk and lost - simple. There are a number of FIRST teams (along with observers) who will now dispassionately assess risk mathematically. The mentors and students involved will not easily repeat the same process and/or error, perhaps for an entire career.

[Tristan Lall]

Originally Posted by pfreivald

No, grasp, grapple, and grab were included in your quotes, and the bumper ones were essentially the same question. Thus, four cases as far as you were able to drum up.

Yes, that is what I said: 3, or 4 if you count the bumpers separately, plus the grasp/grapple/grab thing. So at least four separate issues in total.

Originally Posted by pfreivald

Blue is blue (and this was intended as 'close enough, don't worry about dye lot numbers'), bumpers have to be supported on the ends, vision interference is vision interference, and grasp/grab/grapple mean what they mean. I hardly think that's "so many different situations" that leave "a lot of big ambiguities".

Let's look at those individually. The ambiguities are pretty straightforward, and should have been anticipated. (I also think those ambiguities are consequential enough to be justifiably described as "big". You're entitled to believe otherwise.)

The bumper colour one is an example of good use of the term. It's not very ambiguous at all. As far as I can tell, any reasonably astute observer is probably going to understand this in one way only. (Well, apart from visual impairments, I guess.)

The vision target one is not. Is the standard met when the observer thinks it's similar, or when they think a camera will perceive it similarly? Does the observer's impression of mimicry depend on whether they understand how a shape-tracking algorithm works? What about intent? What is the practical consequence of uncertainty and misinterpretation? These are things that can easily be specified, to guide teams and officials.

The bumper end questions are also open to interpretation. Which parts of which faces constitute the ends? And what about soft parts of the bumper? If FIRST is trying to be permissive, they need to try harder—even after the clarification, they left people guessing as to whether FIRST missed the point of the question (and thus could have meant that a reasonably astute observer would obviously decide that the end face was the "end"), or was trying to cut teams some slack (anything remotely near to the end, on any surface). Again, this would be easy to specify, either numerically/geometrically, or as a functional test for inspectors to perform. (This is also annoying because it's a largely useless requirement.)

And we've already discussed grasp/grab/grapple.

Originally Posted by pfreivald

Yes, yes, a thousand times yes -- they're alike in that they're all extremely obvious if you're not trying to game the system.

Is your understanding of that likeness universal? (And in case your insight about obviousness was a flippant rather than serious response, can you articulate what the meaningful likeness is?) Does it lead to useful, inescapable conclusions that help satisfy the questions I described above?

In other words, by observing the interpretation of one of these things, are you sufficiently informed about how the interpretation of the others will proceed? I think not, because the details are so different, and the people responsible for making the calls cannot help but be inconsistent.

(Also, 118 would be disappointed at your implicit characterization of them, I'm certain.)

Originally Posted by pfreivald

I'll stop you there. We are specifically instructed not to lawyer the rules. Talking about how judges define legal tests is directly contrary to what you're supposed to be doing when you look at these game rules. (And besides, if you're going to drag judicial practice into this, Potter Stewart's "I know it when I see it" is a much better analogy to the "reasonably astute observer" test.)

I'm not the one who first invoked the concept of legal tests. Indeed, there's a fair chance that FIRST had legal tests in mind when they wrote the "reasonably astute observer" standard—and if that's the case, it's undoubtedly an appropriate way to analyze the question. It might well be their own fault, despite their 'no lawyering' mantra.

Even if FIRST did not intend for us to be talking about the law, one can't help but draw the obvious parallels. Those parallels don't just go away, because some members of the GDC are repulsed by the notion of people acting like lawyers.

(Incidentally, "I know it when I see it" isn't a test for obscenity; it's preceded by an apologetic statement that a good definition of such a test escaped Justice Stewart, and that in that case, he was relying on his own instinctual reaction rather than a formal set of criteria. And like "reasonably astute observer", it's inherently subjective and ambiguous. One might ask, fruitlessly: 'Will Stewart think this other, similar movie is obscene?' That problem of ambiguity is precisely why the Miller test exists.)

Besides, if we're supposed to analyze this in the manner of an engineer, without resorting to law, don't you think that precision is a hallmark of that profession as well? Shouldn't we be talking about dimensions and tolerances when it comes to the bumper ends, and functional tests and specifications when it comes to the vision? A thoughtful engineer might even include the Pantone number of the FIRST logo colours, just to avoid questions about whether we were supposed to approximate the version with the light background, or the one with the dark background. As for grasping, an engineer might distill it down to a discussion of force and contact, or explain whether the true objective of the specification is to limit field damage. At the very least, an engineer should be prepared to discuss things in these terms, if questions arise about the initial open-ended specification.

Why then do we get a test that looks like none of the above, and instead looks like a snippet of legalese?

Originally Posted by pfreivald

I'm a bit astounded at this line of reasoning. Are you seriously positing the argument that anyone reading the phrase "a reasonably astute observer" as used in FRC documentation would interpret it as any observer that considers themselves reasonably astute?

I mean, I suppose you could read it that way, but I can't for the life of me imagine why anyone would do so, as doing so would render the rule so subjective as to be meaningless (and thus it cannot possibly be the intent of the GDC).

Surely you've seen teams and wondered, 'what were they thinking'? Why then is it so hard to believe that a team couldn't interpret FIRST's dictum as "if you're reasonable, we'll be lenient"? (In other words, a team member might think that as long as they can describe their reasoning in a way that makes them appear reasonably astute, the inspector will grant them a pass.) That's just a made-up example, but it's not so implausible that FIRST should reject it as a possible outcome.

Another possibility: a team might agree that all of the referees at the field are reasonably astute observers, and that it is the referees that conference at the end of some matches to discuss rulings. So, if it was a violation in any of their opinions (i.e. the action was inappropriate to "a reasonably astute observer"), it should be impermissible. Again, that's a made-up example that doesn't represent my personal take on the situation. But it's not crazy—notice that the manual doesn't say that refereeing decisions aren't conducted like that.

You could go all the way, and wonder: 'With all these reasonably astute observers everywhere, how will FIRST poll them all?' That is blatantly ridiculous, and should probably be dismissed as such—but the fact that the statement could be interpreted that way exposes its weakness.

Originally Posted by pfreivald

This, by the way, falls right into the positivist trap. I can just imagine the question to the Q&A: "When you say 'a reasonably astute observer', what do you mean by 'a'? Does it mean any reasonably astute observer, a particular reasonably astute observer, or an observer that is considered by the team or the person themselves to be reasonably astute? Also, in assessing observers, what is the difference between reasonably astute and unreasonably astute (or reasonably unastute?)"

Reading requires interpretation; finding the correct interpretation means asking yourself, "what does the author most likely mean in this case?" I have a hard time believing that anyone would read "reasonably astute observer" as used in FRC documentation and come to any conclusion other than a generic layman observing the relevant phenomenon.

By that logic, doesn't a referee (likely not a generic layman in their own right) have to adopt their impression of the thought process of a hypothetical individual to make the call? How is that reasonable, or even predictable?

Besides, it's a more complex question than what the author "most likely" meant. You have to think about cases that are less likely, but still plausible (because the referees might interpret the rule differently, within the bounds of what a generic layman might believe). There's nothing wrong with asking FIRST some of the things you alluded to in an effort to narrow the solution space a bit.

Indeed conversely, the author of the rule should be thinking about all the ways the rule could be interpreted by a reasonable person, and hedging against them. Either use precise language, or construct the requirement in a way that precision is immaterial. If FIRST really wants it to come down to the referee's on-the-spot judgment, then they should just say so—no need to invoke a standard that lacks a clear definition, and is potentially applied differently under different circumstances.

[pfreivald]

Originally Posted by Tristan Lall

Let's look at those individually. The ambiguities are pretty straightforward, and should have been anticipated.

I found none of them ambiguous.

Originally Posted by Tristan Lall

These are things that can easily be specified, to guide teams and officials.

No, not 'easily'. When rules are going to be picked apart as much as they are, specificity in this manner creates well more problems than it solves.

Originally Posted by Tristan Lall

(This is also annoying because it's a largely useless requirement.)

This opinion is entirely moot to the conversation. "I find this rule annoying" is not a discussion I'm interested in having.

Originally Posted by Tristan Lall

Is your understanding of that likeness universal?

It doesn't have to be. It just has to be universal enough to be predictable by those who need to incorporate the rule into their design process. (Which again, I would like to emphasize that 118 did very well.)

Originally Posted by Tristan Lall

(And in case your insight about obviousness was a flippant rather than serious response, can you articulate what the meaningful likeness is?)

I was being serious, and I've already articulated the meaningful likeness. That you disagree does not negate the statement.

Originally Posted by Tristan Lall

Does it lead to useful, inescapable conclusions that help satisfy the questions I described above?

"Inescapable" is a bar set far too high. Nothing you or I or anyone else has said on this thread or any other about game rules is "inescapable".

Originally Posted by Tristan Lall

In other words, by observing the interpretation of one of these things, are you sufficiently informed about how the interpretation of the others will proceed?

I have yet to be wrong when interpreting FIRST rules described by the "reasonably astute observer" metric. So the evidence points to 'yes'.

Originally Posted by Tristan Lall

(Also, 118 would be disappointed at your implicit characterization of them, I'm certain.)

I don't believe they would -- they were trying to game the rules to the best of their ability, and good for them. Nothing about that characterization is an insult. They should be rightly proud of what they did, and that they did so in a way that their risk was calculated and inherently mitigated.

Originally Posted by Tristan Lall

Even if FIRST did not intend for us to be talking about the law, one can't help but draw the obvious parallels.

I can help but draw the parallel, because it's entirely unneeded.

Originally Posted by Tristan Lall

(Incidentally, "I know it when I see it" isn't a test for obscenity; it's preceded by an apologetic statement that a good definition of such a test escaped Justice Stewart, and that in that case, he was relying on his own instinctual reaction rather than a formal set of criteria.

Yes, exactly. The entire point is that it's difficult to articulate but easy to ascertain.

Originally Posted by Tristan Lall

(In other words, a team member might think that as long as they can describe their reasoning in a way that makes them appear reasonably astute, the inspector will grant them a pass.)

As I said, I find the fact that anyone might come to that conclusion downright astounding.

Originally Posted by Tristan Lall

Another possibility:

Yes, indeed, you can come up with a gazillion possible interpretations. Your job, though, in reading the rules, is to come up with the most probable one.

Originally Posted by Tristan Lall

By that logic, doesn't a referee (likely not a generic layman in their own right) have to adopt their impression of the thought process of a hypothetical individual to make the call? How is that reasonable, or even predictable?

Yes, they do. It's reasonable and predictable because that's what refs do -- they enforce rules and make judgement calls in doing so.

Originally Posted by Tristan Lall

Besides, it's a more complex question than what the author "most likely" meant.

No, it isn't.

Originally Posted by Tristan Lall

You have to think about cases that are less likely, but still plausible

True. ...and then determine which is most likely.

Originally Posted by Tristan Lall

There's nothing wrong with asking FIRST some of the things you alluded to in an effort to narrow the solution space a bit.

Also true. Which is what was done, and the answer given was the "reasonably astute observer" test, which everyone should have taken for the generality that it is.

Originally Posted by Tristan Lall

Either use precise language, or construct the requirement in a way that precision is immaterial.

Have you ever written a game? I don't believe that you have a good grasp on just how difficult it is to do that -- and how unnecessary in many situations.

[45Auto]

Originally Posted by Tristan Lall

Either use precise language, or construct the requirement in a way that precision is immaterial.

Wow, apparently you have never worked to a set of real world requirements!

[pfreivald]

Originally Posted by 45Auto

Wow, apparently you have never worked to a set of real world requirements!

Thanks for the backup from another perspective.

The funny thing is that I'm not an engineer (though I have some experience with both semiconductor manufacturing equipment and quantum cryptography): my degrees are in theoretical physics, linguistics/ASL, and education, not engineering. I have, however, at one time or another been a professional freelance game designer, so my perspective on this -- and thus my disagreement with Tristan and my defense of the GDC -- comes primarily from what I know about writing game rules as opposed to engineering.

Until you sit down and write a set of rules, and then have people try to play a game (even a mock-up of that game) based on those rules, you have no idea just how hard it is to write good rules -- and that's not even taking into account whether or not the game is fun to play or fun to watch or both.

And as I said before, permissive rules sets are easier to write than (good) prescriptive rules sets, and each lend themselves to a very different type of game. Writing a prescriptive rules set that you know is going to be picked apart by tens of thousands of highly motivated, intelligent people is, to put it mildly, a daunting task. In light of that task, the expectation of language so precise that it can't be gamed is completely, utterly unreasonable.