Chief Delphi - can Python open a matrix saved by Octave in sparse format?

Chief Delphi (http://www.chiefdelphi.com/forums/index.php)

- Python (http://www.chiefdelphi.com/forums/forumdisplay.php?f=187)

- - can Python open a matrix saved by Octave in sparse format? (http://www.chiefdelphi.com/forums/showthread.php?t=135405)

Ether

03-03-2015 11:49

can Python open a matrix saved by Octave in sparse format?

1 Attachment(s)

Attached ZIP file contains a 17842x2696 matrix with 53526 non-zero elements stored in Octave sparse format.

Can Python load a matrix stored in this format?

faust1706

03-03-2015 15:01

Re: can Python open a matrix saved by Octave in sparse format?

do we know the size of the matrix before hand?

Ether

03-03-2015 15:17

Re: can Python open a matrix saved by Octave in sparse format?

Quote:

Originally Posted by faust1706 (Post 1452894)

do we know the size of the matrix before hand?

Quote:

Originally Posted by Ether (Post 1452765)

Attached ZIP file contains a 17842x2696 matrix with 53526 non-zero elements

Ether

03-03-2015 18:00

Re: can Python open a matrix saved by Octave in sparse format?

Quote:

Originally Posted by Ether (Post 1452765)

Attached ZIP file contains a 17842x2696 matrix with 53526 non-zero elements stored in Octave sparse format.

Can Python load a matrix stored in this format?

Octave loads the file in less than half a second, and converts it to full format in about a third of a second:

Code:





GNU Octave, version 3.6.4

Copyright (C) 2013 John W. Eaton and others.

This is free software; see the source code for copying conditions.

Octave was configured for "i686-pc-mingw32".



Additional information about Octave is available at http://www.octave.org.



+ ## load the alliances design matrix

+ tic;

+ load ('$A.dat');

+ toc

Elapsed time is 0.422 seconds.

+ [alliances, teams] = size (A)

alliances =  17842

teams =  2696

+ matches = alliances / 2

matches =  8921

+ ## convert it to full

+ tic;

+ Af = full (A);

+ toc

Elapsed time is 0.343 seconds.

+ endscript;

RyanCahoon

03-03-2015 18:38

Re: can Python open a matrix saved by Octave in sparse format?

Here's a bit of a quick hack:

Code:

import scipy.sparse

import numpy



def read_sparse(filename):

  (r, c, d) = numpy.transpose(numpy.loadtxt(filename, 'int'))

  return scipy.sparse.coo_matrix((d, (r-1, c-1)))

Note that the matrix's dimensions may be truncated if there are trailing rows and columns that are all 0, but this is not the case with your example matrix. It would need a little more work to read the dimensions from the file.

---

A better approach would be to have Octave save the data into MATLAB format (using something like
save -6 A_sparse_octave.mat A), then read it in Python using scipy.io.loadmat

Ether

03-03-2015 21:23

Re: can Python open a matrix saved by Octave in sparse format?

Quote:

Originally Posted by RyanCahoon (Post 1453024)

A better approach would be to have Octave save the data into MATLAB format (using something like save -6 A_sparse_octave.mat A, then read it in Python using scipy.io.loadmat

Thanks Ryan. That looks like a good solution.

Do you happen to know where I can find the spec for the Matlab matrix binary storage file format?

MrRoboSteve

03-03-2015 22:07

Re: can Python open a matrix saved by Octave in sparse format?

This?

http://www.mathworks.com/help/pdf_do...ile_format.pdf

All times are GMT -5. The time now is 19:54.