Does robotics attract the right students?

[penguinfrk]

I had an interesting conversation about the self-selection of participants in extracurricular activities with an attorney who is running for state legislature come 2012.

Here are some statistics on the five public high schools in Bellevue, WA, all of which have FRC teams, taken from the Washington State Office of Superintendent of Public Instruction (OSPI).

                        International(492)  Newport(948)	Bellevue(949)	Interlake(1899)	Sammamish(2412)
Years with team      	11                  9                  	9       	7       	5
Grad Rate              	100.0%              96.0%       	95.3%   	89.4%        	86.2%
Science test scores    	92.3%               80.1%        	74.9%       	69.1%        	51.4%
Teacher Experience (yr)	11                  12                	9       	10             	10
Student:Teacher Ratio  	16                  17                	18           	16      	13

The correlation between the years that the team has been in existence and the performance of the schools are pretty strong even while other factors such as teacher experience and student:teacher ratio have been relatively constant.

This suggests that whether or not students are on the team has no bearing on the fact that the presence of FIRST programs (or, at least FRC) enhances the educational environment of the schools.

---
Moreover, I personally don't agree with the self-selection thing, if for no other reason than that I personally was not a STEM fanatic when I first joined my FLL team.

I do think that peer pressure, in this case a fantastic thing, causes students who otherwise never would've joined a team to show up at meetings with their friends or boyfriends or what have you. Even if everyone in this group isn't all retained, there are undoubtedly many who become immersed in STEM.

[John]

Originally Posted by penguinfrk

This suggests that whether or not students are on the team has no bearing on the fact that the presence of FIRST programs (or, at least FRC) enhances the educational environment of the schools.

Do you have any data on the scores before FRC? Could the presence of FRC not be the cause of this correlation, instead could the correlation be the result of schools that were already committed to Science (and therefore likely to have better test scores anyway) being more likely to start FRC teams. This would seem especially likely if the scores have not changed much since the teams were started. Though if you can show the scores went up since then your point makes more sense.

[Chris Hibner]

We try to ask teachers in the school to identify students that would benefit from FIRST - typically the students that appear to have potential but may not be living up to it.

I think it's good to have a mix of students on the team. The overachiever types, the underachiever types, "in" crowd students, "out" crowd students, athletes, art students, etc. In that situation everyone learns a lot from the other students that they normally wouldn't hang out with, and hopefully some of the underinspired get a little inspiration from the alread-inspired (or at least they can see how the ambitious people operate).

[penguinfrk]

Originally Posted by John

Do you have any data on the scores before FRC? Could the presence of FRC not be the cause of this correlation, instead could the correlation be the result of schools that were already committed to Science (and therefore likely to have better test scores anyway) being more likely to start FRC teams. This would seem especially likely if the scores have not changed much since the teams were started. Though if you can show the scores went up since then your point makes more sense.

Unfortunately, the statistics available from WA OSPI were not as thorough back then. The Titan Robotics team (492) was formed in the 2000-2001 school year. Newport (948) and Bellevue (949) started competing in 2002.

The first year that a standardized science test was administered was in the 2002-2003 school year; 492 was in their 3rd competition season by the time the test was administered, and 948/949 were in the 2nd competition season, if I'm not mistaking. At the time, 1899 (Interlake) and 2412 (Sammamish) did not exist.

Team #              | 492*  | 948   | 949   | 1899* | 2412
2003 Science Scores | 62.9% | 70.5% | 60.4% | 52.0% | 47.4%
2010 Science Scores | 92.3% | 80.1% | 74.9% | 69.1% | 51.4%
d(scores)           | 29.4% | 9.6%  | 14.5% | 17.1% | 4.0%
d(scores)/year**    | 2.67% | 1.07% | 1.61% | 2.44% | 0.8%

*The Bellevue School District is an awfully inaccurate school district to examine for two reasons.

First, International School (492) has a student enrollment system different from the other schools. It is done by lottery, and very academia-focused (i.e. no school sports, few extracurriculars, etc.) Moreover, the school spans from 6th to 12th grade, meaning that students get 3 years' more exposure to the clubs in the school than they otherwise would at a traditional 4-year senior high school.

Second, Interlake High School (1899) adopted an accelerated program in the 2006-2007 school year (2nd year of 1899 in FRC) as a continuation of the middle school program. Many gifted students that otherwise would've been at 948, 949, and 2412's schools are attending Interlake. The subsequent inflation of test scores makes statistics for 1899 a clear outlier.


**The change in scores per year of FRC team existence is the absolute change in percentage points, not a measure of the rate of growth/improvement on tests.


However, these statistics are enough to draw a few conclusions.

For one, the correlation between 2003 test scores and the presence of FRC teams is very weak; in fact, 948's school had stronger test scores than 492's school. However, by 2010 there clearly has been a drastic change in performance, and every school saw growth. In other words, FRC magnifies academic performance.
Second, the change in test scores is not a linear function. More veteran teams that have a stronger presence in the school will have a greater impact for a given duration of time (i.e. year to year) than will a newer team.


Further extensions: demographic constituency, team size/percentage of school population, mentor number/mentor:student ratio, economic status (as measured by free/reduced lunch qualification), among others. This is the project devised for my internship at FIRSTWA, and I will be mining more thorough data on teams across the entire state of Washington. Once I have better data to back up more sophisticated conclusions later in the year, I'll be sure to post them or publish a white paper or something.

[gblake]

Originally Posted by penguinfrk

...

However, these statistics are enough to draw a few conclusions.

...

Larry,

Be careful. From what you have shown us so far, you can not draw the conclusions you listed in your post. Correlation does not mean causation.

Until you compare datasets in which the ONLY difference is the presence of a STEM robotics program, and until you can see variation in test scores that matches variations in the STEM robotics program; you simply can not tell if the robotics programs are symptoms, causes, or coincidences.

What you see in your data could be the result of many influences.

I hope you can devise a method for showing causation and not just correlation; but I don't think you have that method in place yet.

Blake