I’ve been working on putting together a data set that can be used from year to year. Im collecting data on how different teams use their resources and where they use them.
I’m hoping to get as many people from various teams as possible on board. Plus, Id encourage you to fill out the form multiple times for each competition. The more data we have, the better!
Most of the questions aren’t mandatory, so don’t worry about leaving anything blank if you’re unsure.
And, just so you know, the data will be available to anyone who submits their email in the form.
Thanks for joining in! You can find the link here:
P.S:
This has been gauged for Midwest regionals, but any new competitions submitted not already on the form may be added.
Just filled it out! I’ve got a few questions from a methodology standpoint:
What’s the goal with segmenting by event? Do you expect to see correlation of that to anything in particular? Why not group at just a season level?
For the most part, you’re asking 1 person to rank their team on a series of qualitative factors. Do you have any plans for controlling for this? I’d expect someone’s 3 to be someone else’s 6 – especially for the question about perceived luck.
What’s your end goal with this data set? What’s your hypothesis, and what are you expecting to find?
By asking on Chief, you’re going to end up with insane sampling bias. In general, the people who are A) on this godforsaken website and B) are taking the time to contribute to things like this are likely to be in the top bucket of skill/experience/involvement. Do you have plans to correct for this?
Hey! Thanks for reaching out and filling out the form its well appreciated!
segmenting by event instead of just year allows us to examine a smaller period of time as things like “robot ability” “luck” and “strategy” can change from competition to competition. This allows us to examine more closely each event and how the particular data we’re asking for is affecting it.
I am aware that a scale of 1-10 isn’t necessarily the best way of doing things though its a valid point and definitely something worth examining. I have taken precautions to show this to many people before posting it here, inevitably getting feedback. The issue is that the points are important to take and I (nor anyone I’ve talked to) seems to have a solution to it while keeping the same data points. If you have one I would love to hear it out. But text boxes just don’t seem like a viable setup for combing through data. (luck was one I threw in there for fun, good catch though
my goal with this data set is to find places where teams need improvement. So this would mean finding correlations between event performance and where your team is strongest in these categories. I’m expecting to find that 2-3 of these tables will correlate to a higher performance showing us as teams where to put our effort into improving. As for a hypothesis? I’m guessing that “robot performance, strategy, and driver skill” will be the most important end effectors.
and yes, sampling bias is definitely being taken into account. I’m hoping I can snag multiple people from multiple teams eventually, and that with enough data it should eventually balance out. Correcting for it is almost certainly going to happen, and I will write up a thorough and thought-out report on the data, the way bias has affected it, and how a (smallish) sample size has affected it.
Thank you so much for reaching out, and if you have any other concerns, or if you have any ways you think I should change the data set based on the feedback I would love to hear them.
I’d fill out the sheet, but I think you already have the same data I have.
I’m definitely very interested to see how y’all utilize the data you receive, and as an alum of NoMythic (class of 2020) who’s done a significant amount of learning and connecting with others more knowledgeable than I since graduating, would be more than happy to help out with this process where I can. Feel free to send a message my way!
From a qualitative research standpoint you are measuring latent variables here, rather than something that can be measured directly. In this type of research (assuming a good input sample) one would normally approach with a series of questions to gauge an individual’s agreeableness over multiple more tangible statements on a likert scale. I.e. for “luck” (this is just an incomplete example that is haphazardly put together for illustration. There are usually about 4-7 careful chosen questions to measure each latent variable, there is a fair bit of work that goes into putting these together)
Indicate how much you agree with the following statements with 1 being strongly disagree and 7 being strongly agree.
The schedule at events is actively handicaps my team.
Other teams have easier schedules than my team at event.
My team controls for as many variables as possible at an event.
My team suffers more mechanical issues than other teams at events
Other teams tend to try tell my team what to do during matches
From here things can be (weighted) averaged for each latent variable or otherwise combined to a composite score representing the latent variable.
Again, why? Presumably you have control over these options.
I won’t pick at this area anymore myself, but stuff like this has the respondents questioning any integrity of the survey as it feels poorly designed from the end user standpoint.
I don’t really think however much data you collect here, it will answer the questions you are looking for. We sort of know that more budget, more attention to strategy, more practice, and more mentor support (or experienced/trained students) will generally lead to better performance at competitions (neglecting bad luck or poor robustness).
I think rather than a survey it is better on a team basis to consider what deficits you might have, and then ask more detailed questions to the numerous teams represented here on CDChief Delphi or elsewhere, about how they best manage to do these things. For example, asking “How often do you practice and what drills do you do to improve driver skills?”
It is interesting to have surveys to establish some baseline data about different teams, but like some of the other advice here, there are a lot of biases and issues to overcome in generating a data set that is meaningful. When I also think about looking to improve performance, I also find it helpful to find other teams that are somewhat similar to us (budget, mentors & student numbers, other tangibles), so that having a survey that establishes that as a starting point for finding more teams to learn from is good. For example, I know mostly which teams in our region I look to based on similar circumstances and their different strengths to learn from, but when I consider things like OpenAlliance teams in other regions I don’t really know which ones are similar in the baseline to our own. Of course, you can learn from any team but seeing what teams similar to yours are doing can seem more possible to reach that next level in performance.
Just to address the concerns that are coming out of this, id like to remind people that this is a fun hobby project and probably not something that should be taken incredibly seriously. This is one of the few times ive collected data like this and im using it as an opportunity to grow. (as im sure is very obvious) I appreciate all of the feedback I have been given and im taking it into account as much as possible. The real kicker is that I dont want to change the form too much so that the data I already have will go to waste. I hope to redo this project someday again with some more experience someday. I appreciate all of you filling this out and taking the time to write to me any ideas or things you want addressed. its endearing to see how many people care about this project already and are wanting to set me on the right path with it. Im definitely taking all of the suggestions into account and im implementing the few I can without changing the data set too much.
again, thank you for all of the wonderful comments
Late to the party… I did look through the survey. I concur with the earlier comments, the quality of data it will collect is questionable.
This forum is a weird place on the internet - a big crowd of experienced people who just really like giving knowledge away for free, and another crowd of individuals seeking to learn.
Liam, given what I’ve seen so far in this thread, there’s two pieces of info I’d recommend reading through:
Check out some basic principles of survey design for the next round. Think hard about what information you actually want, and how to phrase and structure questions so users provide meaningful information that you can trust
Specifically while doing trials of the data collection … or receiving any feedback … check out the Ten Commandments of Egoless Programming. It’s written from a software development perspective, but you could swap “coding” for just about any other engineering activity and it would be just as applicable.
Specifically what I’d want to highlight from those principle: you are not your work. When people provide feedback on something you’ve done, they’re looking to help you improve your final product. They’re not commenting on you as a person, or even your ability to produce good results.
It’s natural human instinct to internalize feedback as something personal, but it’s important to cultivate the skill of separating the two when working in a collaborative environment.
