Re: R Package for Downloading FIRST API Data
 

Reps to you for all the fine work :)

Reps to the first person who can cut that time in half.

Re: R Package for Downloading FIRST API Data
 

Cut 13 seconds in half? I'm excited that sirwin got it to work at all on that data set! At 13 seconds, the WOPR rankings can be recalculated between matches before the first verse of "Cotton eye Joe" gets played. :rolleyes:

Good job sirwin on the code, and an even better write up of how you did it.

Re: R Package for Downloading FIRST API Data
 

With respect to speeding up the calculation, I had a thought.

It takes my algorithm 9 seconds to produce the 2696 x 2696 A matrix -- but updating the A matrix with new match data should be nearly instantaneous. For every new match, we just have to add 1 to 18 different elements in the A matrix (all other A matrix elements will remain unchanged). Each element of the a matrix is directly accessible by team numbers. So it takes 13 seconds to calculate world OPR from scratch, but as long as we retain the A matrix, it should only take about 5 seconds to update world OPR with new match data. That's 2 seconds for updating the A and B matrices and 3 seconds for solving for OPR.

Of course, this is assuming that R's underlying C code doesn't do anything stupid, like extracting an array element by going through every array element until it gets to the right one. Also, the A matrix would need to be maintained in memory -- disk access would slow things down.

Re: R Package for Downloading FIRST API Data
 

I like to call that symmetric positive definite 2696x2696 normal equations matrix the N matrix... to distinguish it from the 17842x2696 dichotomous design matrix A.

Ax = b .. overdetermined system of linear equations (has no exact solution)

N = A'*A
d = A'*b

Nx = d .. linear system of normal equations (whose solution is the least squares solution for Ax=b)

Re: R Package for Downloading FIRST API Data
 

OK, I was being generous. Cut it by a factor of 10. Seriously.

Re: R Package for Downloading FIRST API Data
 

If there is any interest in learning how to do this within the R community please let me know and we can work it out together.

Re: R Package for Downloading FIRST API Data
 

I took the world OPR data and made a graph.

EDIT: I made a better graph.

Re: R Package for Downloading FIRST API Data
 

https://www.chiefdelphi.com/forums/s...09#post1614009

Re: R Package for Downloading FIRST API Data
 

I created version 2.0.0 of the firstapiR package for downloading and manipulating FIRST API scouting data using the R language.

Here's a link:

http://irwinsnet.github.io

Once I started using the package to do OPR calculations, I realized it needed some improvements, specifically in how the data frames are shaped.

Stacy Irwin

Re: R Package for Downloading FIRST API Data
 

@ Stacy: thanks for posting this. I just send you a PM.

Re: R Package for Downloading FIRST API Data
 

I am aware I am beating a dead horse here :deadhorse: , but using unmodified Sirwin's code I am getting ~12 seconds with an i7-6560U (2.2GHz) and 8gb of RAM, memory usage is low so we can eliminate that as a factor. The bottleneck is (unsurprisingly) CPU clock cycle speed.

I'm using firstapiR v2.0.0

Skye Leake

Re: R Package for Downloading FIRST API Data
 

I am getting ~0.93 seconds on a 10-year-old Pentium D desktop running 32-bit XP with 1G RAM.

Re: R Package for Downloading FIRST API Data
 

:ahh: I obviously am doing something not quite right...

Re: R Package for Downloading FIRST API Data
 

Your results look about right for R code that uses dense matrix technology.

https://www.chiefdelphi.com/forums/s....php?p=1612297

https://www.chiefdelphi.com/forums/s....php?p=1612581

https://www.chiefdelphi.com/forums/s....php?p=1613118

https://www.chiefdelphi.com/forums/s....php?p=1614125

Re: R Package for Downloading FIRST API Data
 

In case anybody wants to learn R, here is a great way to get introduced http://swirlstats.com