can Python open a matrix saved by Octave in sparse format?

[Ether]

Attached ZIP file contains a 17842x2696 matrix with 53526 non-zero elements stored in Octave sparse format.

Can Python load a matrix stored in this format?

[faust1706]

do we know the size of the matrix before hand?

[Ether]

do we know the size of the matrix before hand?

Attached ZIP file contains a 17842x2696 matrix with 53526 non-zero elements

??

[Ether]

Attached ZIP file contains a 17842x2696 matrix with 53526 non-zero elements stored in Octave sparse format.

Can Python load a matrix stored in this format?

Octave loads the file in less than half a second, and converts it to full format in about a third of a second:



GNU Octave, version 3.6.4
Copyright (C) 2013 John W. Eaton and others.
This is free software; see the source code for copying conditions.
Octave was configured for "i686-pc-mingw32".

Additional information about Octave is available at http://www.octave.org.

+ ## load the alliances design matrix
+ tic;
+ load ('$A.dat');
+ toc
Elapsed time is 0.422 seconds.
+ [alliances, teams] = size (A)
alliances = 17842
teams = 2696
+ matches = alliances / 2
matches = 8921
+ ## convert it to full
+ tic;
+ Af = full (A);
+ toc
Elapsed time is 0.343 seconds.
+ endscript;

[RyanCahoon]

Here's a bit of a quick hack:

import scipy.sparse
import numpy

def read_sparse(filename):
(r, c, d) = numpy.transpose(numpy.loadtxt(filename, 'int'))
return scipy.sparse.coo_matrix((d, (r-1, c-1)))

Note that the matrix's dimensions may be truncated if there are trailing rows and columns that are all 0, but this is not the case with your example matrix. It would need a little more work to read the dimensions from the file.

---

A better approach would be to have Octave save the data (https://www.gnu.org/software/octave/doc/interpreter/Simple-File-I_002fO.html#XREFsave) into MATLAB format (using something like
save -6 A_sparse_octave.mat A), then read it in Python (http://docs.scipy.org/doc/scipy/reference/tutorial/io.html#matlab-files) using scipy.io.loadmat (http://docs.scipy.org/doc/scipy/reference/generated/scipy.io.loadmat.html)

[Ether]

A better approach would be to have Octave save the data into MATLAB format (using something like save -6 A_sparse_octave.mat A, then read it in Python using scipy.io.loadmat

Thanks Ryan. That looks like a good solution.

Do you happen to know where I can find the spec for the Matlab matrix binary storage file format?

[MrRoboSteve]

This?

http://www.mathworks.com/help/pdf_doc/matlab/matfile_format.pdf