View Single Post
  #5   Spotlight this post!  
Unread 28-10-2016, 13:37
Ether's Avatar
Ether Ether is offline
systems engineer (retired)
no team
 
Join Date: Nov 2009
Rookie Year: 1969
Location: US
Posts: 7,987
Ether has a reputation beyond reputeEther has a reputation beyond reputeEther has a reputation beyond reputeEther has a reputation beyond reputeEther has a reputation beyond reputeEther has a reputation beyond reputeEther has a reputation beyond reputeEther has a reputation beyond reputeEther has a reputation beyond reputeEther has a reputation beyond reputeEther has a reputation beyond repute
Re: Optimization, when to stop



Thanks for the encouraging words Foster.

Having done this computation in Octave (Matlab), SciLab, and Python, I knew how long it should take. But my initial R implementation was a factor of 10 (or more) slower. I wasn't willing to accept this.

So while I was awaiting input from the R community here on CD, I researched it myself and got R to do the computation just as fast. In so doing, I learned a lot about R. And that was the whole point of the exercise because, echoing what Aren said, what I learned makes R a far more powerful tool for me to use for other large problems I may encounter in the future.

As I said, I'm not an R guru, so my solution was actually a hybrid. I used the AWK scripting language to pre-process the 8column data into three files: Aij.dat, b.dat, and T.dat. Then my R code reads those three files and computes the OPR.

For the large 8column data set linked above, the AWK script execution time is about 270 miliseconds. The R script execution time is about 660 milliseconds.

If some R guru will take this pseudo-code and write an equivalent R script that runs as fast as my AWK script, I will combine that with my R OPR script and post it here on CD for all R mavens to study and perhaps improve.

Fair deal?


Attached Files
File Type: doc 8col_processing.doc (635 Bytes, 15 views)

Last edited by Ether : 28-10-2016 at 18:36. Reason: added attachment
Reply With Quote