Go to Post I love how the people of Chief Delphi can turn a spam post into a somewhat non-timewasting thread. Either way, congrats CD and thanks for this ;) - logank013 [more]
Home
Go Back   Chief Delphi > Technical > Programming > Python
CD-Media   CD-Spy  
portal register members calendar search Today's Posts Mark Forums Read FAQ rules

 
Reply
Thread Tools Rate Thread Display Modes
  #1   Spotlight this post!  
Unread 03-03-2015, 11:49
Ether's Avatar
Ether Ether is offline
systems engineer (retired)
no team
 
Join Date: Nov 2009
Rookie Year: 1969
Location: US
Posts: 7,995
Ether has a reputation beyond reputeEther has a reputation beyond reputeEther has a reputation beyond reputeEther has a reputation beyond reputeEther has a reputation beyond reputeEther has a reputation beyond reputeEther has a reputation beyond reputeEther has a reputation beyond reputeEther has a reputation beyond reputeEther has a reputation beyond reputeEther has a reputation beyond repute
can Python open a matrix saved by Octave in sparse format?


Attached ZIP file contains a 17842x2696 matrix with 53526 non-zero elements stored in Octave sparse format.

Can Python load a matrix stored in this format?



Attached Files
File Type: zip A_sparse_octave.zip (106.4 KB, 11 views)
Reply With Quote
  #2   Spotlight this post!  
Unread 03-03-2015, 15:01
faust1706's Avatar
faust1706 faust1706 is offline
Registered User
FRC #1706 (Ratchet Rockers)
Team Role: College Student
 
Join Date: Apr 2012
Rookie Year: 2011
Location: St Louis
Posts: 498
faust1706 is infamous around these partsfaust1706 is infamous around these parts
Re: can Python open a matrix saved by Octave in sparse format?

do we know the size of the matrix before hand?
__________________
"You're a gentleman," they used to say to him. "You shouldn't have gone murdering people with a hatchet; that's no occupation for a gentleman."
Reply With Quote
  #3   Spotlight this post!  
Unread 03-03-2015, 15:17
Ether's Avatar
Ether Ether is offline
systems engineer (retired)
no team
 
Join Date: Nov 2009
Rookie Year: 1969
Location: US
Posts: 7,995
Ether has a reputation beyond reputeEther has a reputation beyond reputeEther has a reputation beyond reputeEther has a reputation beyond reputeEther has a reputation beyond reputeEther has a reputation beyond reputeEther has a reputation beyond reputeEther has a reputation beyond reputeEther has a reputation beyond reputeEther has a reputation beyond reputeEther has a reputation beyond repute
Re: can Python open a matrix saved by Octave in sparse format?

Quote:
Originally Posted by faust1706 View Post
do we know the size of the matrix before hand?
Quote:
Originally Posted by Ether View Post
Attached ZIP file contains a 17842x2696 matrix with 53526 non-zero elements
??


Reply With Quote
  #4   Spotlight this post!  
Unread 03-03-2015, 18:00
Ether's Avatar
Ether Ether is offline
systems engineer (retired)
no team
 
Join Date: Nov 2009
Rookie Year: 1969
Location: US
Posts: 7,995
Ether has a reputation beyond reputeEther has a reputation beyond reputeEther has a reputation beyond reputeEther has a reputation beyond reputeEther has a reputation beyond reputeEther has a reputation beyond reputeEther has a reputation beyond reputeEther has a reputation beyond reputeEther has a reputation beyond reputeEther has a reputation beyond reputeEther has a reputation beyond repute
Re: can Python open a matrix saved by Octave in sparse format?

Quote:
Originally Posted by Ether View Post
Attached ZIP file contains a 17842x2696 matrix with 53526 non-zero elements stored in Octave sparse format.

Can Python load a matrix stored in this format?
Octave loads the file in less than half a second, and converts it to full format in about a third of a second:

Code:


GNU Octave, version 3.6.4
Copyright (C) 2013 John W. Eaton and others.
This is free software; see the source code for copying conditions.
Octave was configured for "i686-pc-mingw32".

Additional information about Octave is available at http://www.octave.org.

+ ## load the alliances design matrix
+ tic;
+ load ('$A.dat');
+ toc
Elapsed time is 0.422 seconds.
+ [alliances, teams] = size (A)
alliances =  17842
teams =  2696
+ matches = alliances / 2
matches =  8921
+ ## convert it to full
+ tic;
+ Af = full (A);
+ toc
Elapsed time is 0.343 seconds.
+ endscript;
Reply With Quote
  #5   Spotlight this post!  
Unread 03-03-2015, 18:38
RyanCahoon's Avatar
RyanCahoon RyanCahoon is offline
Disassembling my prior presumptions
FRC #0766 (M-A Bears)
 
Join Date: Dec 2007
Rookie Year: 2007
Location: Mountain View
Posts: 688
RyanCahoon has a reputation beyond reputeRyanCahoon has a reputation beyond reputeRyanCahoon has a reputation beyond reputeRyanCahoon has a reputation beyond reputeRyanCahoon has a reputation beyond reputeRyanCahoon has a reputation beyond reputeRyanCahoon has a reputation beyond reputeRyanCahoon has a reputation beyond reputeRyanCahoon has a reputation beyond reputeRyanCahoon has a reputation beyond reputeRyanCahoon has a reputation beyond repute
Re: can Python open a matrix saved by Octave in sparse format?

Here's a bit of a quick hack:

Code:
import scipy.sparse
import numpy

def read_sparse(filename):
  (r, c, d) = numpy.transpose(numpy.loadtxt(filename, 'int'))
  return scipy.sparse.coo_matrix((d, (r-1, c-1)))
Note that the matrix's dimensions may be truncated if there are trailing rows and columns that are all 0, but this is not the case with your example matrix. It would need a little more work to read the dimensions from the file.

---

A better approach would be to have Octave save the data into MATLAB format (using something like
save -6 A_sparse_octave.mat A), then read it in Python using scipy.io.loadmat
__________________
FRC 2046, 2007-2008, Student member
FRC 1708, 2009-2012, College mentor; 2013-2014, Mentor
FRC 766, 2015-, Mentor
Reply With Quote
  #6   Spotlight this post!  
Unread 03-03-2015, 21:23
Ether's Avatar
Ether Ether is offline
systems engineer (retired)
no team
 
Join Date: Nov 2009
Rookie Year: 1969
Location: US
Posts: 7,995
Ether has a reputation beyond reputeEther has a reputation beyond reputeEther has a reputation beyond reputeEther has a reputation beyond reputeEther has a reputation beyond reputeEther has a reputation beyond reputeEther has a reputation beyond reputeEther has a reputation beyond reputeEther has a reputation beyond reputeEther has a reputation beyond reputeEther has a reputation beyond repute
Re: can Python open a matrix saved by Octave in sparse format?

Quote:
Originally Posted by RyanCahoon View Post
A better approach would be to have Octave save the data into MATLAB format (using something like save -6 A_sparse_octave.mat A, then read it in Python using scipy.io.loadmat
Thanks Ryan. That looks like a good solution.

Do you happen to know where I can find the spec for the Matlab matrix binary storage file format?


Reply With Quote
  #7   Spotlight this post!  
Unread 03-03-2015, 22:07
MrRoboSteve MrRoboSteve is offline
Mentor
AKA: Steve Peterson
FRC #3081 (Kennedy RoboEagles)
Team Role: Mentor
 
Join Date: Mar 2012
Rookie Year: 2011
Location: Bloomington, MN
Posts: 564
MrRoboSteve has a reputation beyond reputeMrRoboSteve has a reputation beyond reputeMrRoboSteve has a reputation beyond reputeMrRoboSteve has a reputation beyond reputeMrRoboSteve has a reputation beyond reputeMrRoboSteve has a reputation beyond reputeMrRoboSteve has a reputation beyond reputeMrRoboSteve has a reputation beyond reputeMrRoboSteve has a reputation beyond reputeMrRoboSteve has a reputation beyond reputeMrRoboSteve has a reputation beyond repute
Re: can Python open a matrix saved by Octave in sparse format?

This?

http://www.mathworks.com/help/pdf_do...ile_format.pdf
__________________
2016-17 events: 10000 Lakes Regional, Northern Lights Regional, FTC Burnsville Qualifying Tournament

2011 - present · FRC 3081 Kennedy RoboEagles mentor
2013 - present · event volunteer at 10000 Lakes Regional, Northern Lights Regional, North Star Regional, Lake Superior Regional, Minnesota State Tournament, PNW District 4 Glacier Peak, MN FTC, CMP
http://twitter.com/MrRoboSteve · www.linkedin.com/in/speterson
Reply With Quote
Reply


Thread Tools
Display Modes Rate This Thread
Rate This Thread:

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

vB code is On
Smilies are On
[IMG] code is On
HTML code is Off
Forum Jump


All times are GMT -5. The time now is 21:37.

The Chief Delphi Forums are sponsored by Innovation First International, Inc.


Powered by vBulletin® Version 3.6.4
Copyright ©2000 - 2017, Jelsoft Enterprises Ltd.
Copyright © Chief Delphi