new to octave, confused about load(), dlmread et cetera

[justin.cress]

this seems to be the most asked question, but before I ask it, i'd like to
assure everyone i've been googling around for a couple of hours now and
haven't figured out where i'm going wrong,

What i'd like to be able to do is load a bunch of column vectors from a csv
or text file (created by stata) and have octave grab the headers as variable
names. I've tried all the functions I could find documentation on, but none
of them work... csvread ignores headers, dlmread is the same... and load()
gives

error: load: failed to read matrix from file `ps1data.csv'
error: called from:
error: readtest.m at line 9, column 1

error: load: failed to read matrix from file `ps1data.txt'
error: called from:
error: readtest.m at line 9, column 1

for csv and tab delimit, respectively.

any help would be appreeciated.

[forkandwait]

Post by justin.cress
What i'd like to be able to do is load a bunch of column vectors from a csv
or text file (created by stata) and have octave grab the headers as variable
names.

A matrix doesn't have header names, hence dlmread etc won't work with headers.
There is nothing with header names native to matlab/ octave, really. I would
try to work with the matrix without header names, just remember what each column
means.

What you want (if you want it to look like a statistics language) is to convert
each column into a struct field, and have a "struct array". The problem is that
very few functions work with this data structure, and you will have to convert
it into a matrix in order to do tabulations or whatever. I don't think there is
a function to read in data like this either.

My hunch is that you are trying to make octave/ matlab behave like
stata/R/sas/etc, and octave doesn't really do that well.

That may not be much help, but maybe if share the particular application someone
can help you do it more idiomatically (idiomatic to octave, anyway).

[Justin Cress]

thanks for all the responses everyone, and, I think the general theme is
correct, I was trying to use octave too much like stata...

So, if I don't know exactly which order the columns are in (in the data) is
there an easy way to let

Y = load ( .. args ..) be a Y var

and grab a matrix of X vars like

X = load ( ... args ..)

I guess my question is, if i just wanted to do basic OLS (or something
similar) in octave, what's the general approach?

Post by justin.cress

Post by justin.cress
What i'd like to be able to do is load a bunch of column vectors from a

csv

Post by justin.cress
or text file (created by stata) and have octave grab the headers as

variable

Post by justin.cress
names.

A matrix doesn't have header names, hence dlmread etc won't work with headers.
There is nothing with header names native to matlab/ octave, really. I would
try to work with the matrix without header names, just remember what each column
means.
What you want (if you want it to look like a statistics language) is to convert
each column into a struct field, and have a "struct array". The problem is that
very few functions work with this data structure, and you will have to convert
it into a matrix in order to do tabulations or whatever. I don't think there is
a function to read in data like this either.
My hunch is that you are trying to make octave/ matlab behave like
stata/R/sas/etc, and octave doesn't really do that well.
That may not be much help, but maybe if share the particular application someone
can help you do it more idiomatically (idiomatic to octave, anyway).
_______________________________________________
Help-octave mailing list
https://www-old.cae.wisc.edu/mailman/listinfo/help-octave

[CdeMills]

Post by Justin Cress
thanks for all the responses everyone, and, I think the general theme is
correct, I was trying to use octave too much like stata...
So, if I don't know exactly which order the columns are in (in the data) is
there an easy way to let
Y = load ( .. args ..) be a Y var
and grab a matrix of X vars like
X = load ( ... args ..)
I guess my question is, if i just wanted to do basic OLS (or something
similar) in octave, what's the general approach?

Octave is mostly used by engineers, so is more oriented towards numerical
computations, linear algebra, signal processing and the like.

Regarding the problem of identifying columns from their name, this is the
purpose of the 'dataframe' package I mentionned, which is similar to R
data.frame.

About OLS, let say that A is your design matrix, X your unknown and B your
observations,
you have to solve Ax = B
which can be computed as x = A\B
The '\' is a specific operator, and can detect cases of undetermined / badly
scaled systems.

Regards

Pascal

[Jaroslav Hajek]

Post by justin.cress
this seems to be the most asked question, but before I ask it, i'd like to
assure everyone i've been googling around for a couple of hours now and
haven't figured out where i'm going wrong,
What i'd like to be able to do is load a bunch of column vectors from a csv
or text file (created by stata) and have octave grab the headers as variable
names.  I've tried all the functions I could find documentation on, but none
of them work... csvread ignores headers, dlmread is the same... and load()
gives
error: load: failed to read matrix from file `ps1data.csv'
error:   readtest.m at line 9, column 1
error: load: failed to read matrix from file `ps1data.txt'
error:   readtest.m at line 9, column 1
for csv and tab delimit, respectively.
any help would be appreeciated.

The reason why there is no "just load it" function in Octave for this
kind of data is that there is no built-in datatype for matrices with
named (and possibly heterogeneous) columns. Octave's arrays are, in
general, multidimensional and homogeneous (except cell and struct
arrays), and carry no row or column labels.

[CdeMills]

Post by Jaroslav Hajek
The reason why there is no "just load it" function in Octave for this
kind of data is that there is no built-in datatype for matrices with
named (and possibly heterogeneous) columns. Octave's arrays are, in
general, multidimensional and homogeneous (except cell and struct
arrays), and carry no row or column labels.

If you are ready to deal with occasional bugs, I released a few days ago a
package on octave-forge whose name is 'dataframe', with preciselly the goal
of having matrix-like structures with named rows and columns, and the
abilitity to select by name. It can also read CSV files, infer data types,
rows and columns names. To try it:
go to http://octave.sourceforge.net/packages.php
download dataframe-0.4.tar.gz
from octave:
pkg install '~/the/path/with/dataframe-0.4.tar.gz'
pkg load dataframe
test and enjoy

Regards

Pascal

[forkandwait]

Post by justin.cress
What i'd like to be able to do is load a bunch of column vectors from a csv
or text file (created by stata) and have octave grab the headers as variable
names.

You might also try using "textread" (help textread). With textread you can
snarf columns into particular variables.

[Andy Buckle]

Post by justin.cress
this seems to be the most asked question, but before I ask it, i'd like to
assure everyone i've been googling around for a couple of hours now and
haven't figured out where i'm going wrong,
What i'd like to be able to do is load a bunch of column vectors from a csv
or text file (created by stata) and have octave grab the headers as variable
names.

Another idea!

If you have a lot of control (I have no idea what stata can do) you
may be able to get stata to automatically write an octave script to
create your variables and headers.