oct (and mex) files for different architectures

[Richard Crozier]

Hello,

On Matlab, I can compile several different mex files for different machine
architectures and Matlab will call the appropriate one for the local setup.
The compiled mex files are given appropriate extensions e.g. mexa64 for
64bit linux, mexglx for 32 bit linux, mexw32 for 32 bit windows etc.

Is the equivalent possible for oct files (or mex files created in Octave)?
Otherwise, how are different platforms dealt with? I only see a single
extension mentioned in the docs, and I often run batch jobs on several
machines with access to a common directory containing my mfile sources and
compiled mex files. I can't think how I would do this without this
functionality.

I should also admit I haven't actually tried using oct files yet, so sorry
if I've misunderstood something.

Thanks,

Richard

--
View this message in context: http://octave.1599824.n4.nabble.com/oct-and-mex-files-for-different-architectures-tp3807347p3807347.html
Sent from the Octave - General mailing list archive at Nabble.com.

[Jordi Gutiérrez Hermoso]

Post by Richard Crozier
On Matlab, I can compile several different mex files for different machine
architectures and Matlab will call the appropriate one for the local setup.
The compiled mex files are given appropriate extensions e.g. mexa64 for
64bit linux, mexglx for 32 bit linux, mexw32 for 32 bit windows etc.
Is the equivalent possible for oct files (or mex files created in Octave)?

Not currently, no. I find that putting the architecture in the
filename extension is a little weird to begin with. Why don't you see
.exe64 and .exe extensions for Windows executables? Can't you just
recompile and distribute your oct files for the architecture you need
them? I get the feeling that Matlab lets you do that because people
don't like recompiling and/or don't like source code.

- Jordi G. H.

[Richard Crozier]

Post by Jordi Gutiérrez Hermoso

Post by Richard Crozier
On Matlab, I can compile several different mex files for different machine
architectures and Matlab will call the appropriate one for the local setup.
The compiled mex files are given appropriate extensions e.g. mexa64 for
64bit linux, mexglx for 32 bit linux, mexw32 for 32 bit windows etc.
Is the equivalent possible for oct files (or mex files created in Octave)?

Not currently, no. I find that putting the architecture in the
filename extension is a little weird to begin with. Why don't you see
.exe64 and .exe extensions for Windows executables? Can't you just
recompile and distribute your oct files for the architecture you need
them? I get the feeling that Matlab lets you do that because people
don't like recompiling and/or don't like source code.
- Jordi G. H.

I have matlab instances running on several different platforms all
accessing the same code directory simultaneously (hosted on a SAMBA file
share). If I recompiled the source, I would need a new directory for the
mex file for that machine and would have to tell the matlab instance on
that machine to access that mex file rather than another mex file in the
path.

The matlab way lets me have one directory for the C/C++/Fortran code and
compile my mex file in that directory without fiddling with the path
etc. If there's a better way which does not involve my copying my entire
matlab path to a local directory on every machine I'd consider it.

[Michael Goffioul]

Post by Richard Crozier
I have matlab instances running on several different platforms all
accessing the same code directory simultaneously (hosted on a SAMBA file
share). If I recompiled the source, I would need a new directory for the
mex file for that machine and would have to tell the matlab instance on
that machine to access that mex file rather than another mex file in the
path.
The matlab way lets me have one directory for the C/C++/Fortran code and
compile my mex file in that directory without fiddling with the path
etc. If there's a better way which does not involve my copying my entire
matlab path to a local directory on every machine I'd consider it.

You could put the compiled versions into subdirectories using
octave_config_info.canonical_host_type, then in your octaverc you
can use addpath by appending the same canonical_host_type to
a common root directory.

Michael.

[Richard Crozier]

Post by Michael Goffioul

Post by Richard Crozier
I have matlab instances running on several different platforms all
accessing the same code directory simultaneously (hosted on a SAMBA file
share). If I recompiled the source, I would need a new directory for the
mex file for that machine and would have to tell the matlab instance on
that machine to access that mex file rather than another mex file in the
path.
The matlab way lets me have one directory for the C/C++/Fortran code and
compile my mex file in that directory without fiddling with the path
etc. If there's a better way which does not involve my copying my entire
matlab path to a local directory on every machine I'd consider it.

You could put the compiled versions into subdirectories using
octave_config_info.canonical_host_type, then in your octaverc you
can use addpath by appending the same canonical_host_type to
a common root directory.
Michael.

That sounds like a reasonable solution for Octave, thanks.
--
Richard Crozier, Research Associate
Institute for Energy Systems
School of Engineering and Electronics
The University of Edinburgh
Faraday Building, Room 4.124
The King’s Buildings
Edinburgh EH9 3JL

Phone: [phone]
Fax: [phone]

If possible, please avoid sending me files in the proprietory Microsoft Word file format. Plain text, pdf, odf (OpenOffice/Google Docs), or html are all preferable alternatives.


The University of Edinburgh is a charitable body, registered in
Scotland, with registration number SC005336.

[Jordi Gutiérrez Hermoso]

Post by Richard Crozier

Post by Jordi Gutiérrez Hermoso

Post by Richard Crozier
On Matlab, I can compile several different mex files for different machine
architectures and Matlab will call the appropriate one for the local setup.
The compiled mex files are given appropriate extensions e.g. mexa64 for
64bit linux, mexglx for 32 bit linux, mexw32 for 32 bit windows etc.
Is the equivalent possible for oct files (or mex files created in Octave)?

Not currently, no. I find that putting the architecture in the
filename extension is a little weird to begin with. Why don't you see
.exe64 and .exe extensions for Windows executables? Can't you just
recompile and distribute your oct files for the architecture you need
them? I get the feeling that Matlab lets you do that because people
don't like recompiling and/or don't like source code.

I have matlab instances running on several different platforms all
accessing the same code directory simultaneously (hosted on a SAMBA file
share).

And what do you do about the Octave instances? Do you have
octave.exe64 and octave.exe32 running as well?

Also, the architecture is not the biggest problem. Even for the same
architecture, you would have to recompile your octfiles if your
machines have slightly different libraries on the OS you're compiling
for. So you would need .oct64deb-6.0, oct64ubu-11.08, .oct64fedora-13
and .oct64suse-5.

I just don't see how filename extensions can solve this problem. It
might solve it for Windows, but I don't see how it would work for
other things. Octave runs on a lot more architectures than Matlab too;
and we compile it for all 11 architectures of Debian; coming up with a
filename extension per architecture seems unwieldy.

I think the solution for now, even if it's inconvenient, is to put
your oct files in directories that have a filename extension specific
to the architecture you care about and in your m-scripts you
conditionally pick to load the directory based on system information
(perhaps after running system("uname -a") or similar).

- Jordi G. H.

[Richard Crozier]

Post by Jordi Gutiérrez Hermoso

Post by Richard Crozier

Post by Jordi Gutiérrez Hermoso

Post by Richard Crozier
On Matlab, I can compile several different mex files for different machine
architectures and Matlab will call the appropriate one for the local setup.
The compiled mex files are given appropriate extensions e.g. mexa64 for
64bit linux, mexglx for 32 bit linux, mexw32 for 32 bit windows etc.
Is the equivalent possible for oct files (or mex files created in Octave)?

Not currently, no. I find that putting the architecture in the
filename extension is a little weird to begin with. Why don't you see
.exe64 and .exe extensions for Windows executables? Can't you just
recompile and distribute your oct files for the architecture you need
them? I get the feeling that Matlab lets you do that because people
don't like recompiling and/or don't like source code.

I have matlab instances running on several different platforms all
accessing the same code directory simultaneously (hosted on a SAMBA file
share).

And what do you do about the Octave instances? Do you have
octave.exe64 and octave.exe32 running as well?
Also, the architecture is not the biggest problem. Even for the same
architecture, you would have to recompile your octfiles if your
machines have slightly different libraries on the OS you're compiling
for. So you would need .oct64deb-6.0, oct64ubu-11.08, .oct64fedora-13
and .oct64suse-5.
I just don't see how filename extensions can solve this problem. It
might solve it for Windows, but I don't see how it would work for
other things. Octave runs on a lot more architectures than Matlab too;
and we compile it for all 11 architectures of Debian; coming up with a
filename extension per architecture seems unwieldy.
I think the solution for now, even if it's inconvenient, is to put
your oct files in directories that have a filename extension specific
to the architecture you care about and in your m-scripts you
conditionally pick to load the directory based on system information
(perhaps after running system("uname -a") or similar).
- Jordi G. H.

OK, but won't the main difference in performance be the difference
between code compiler optimised for 64bit and 32 bit? I would actually
like to know this.

In my university the library files in linux will be the same across all
the machines in the pool, as dictated by the IT department. There are a
couple of pools of machines, and all Linux machines for instance run
Scientific Linux.

Richard

[Jordi Gutiérrez Hermoso]

Post by Richard Crozier
OK, but won't the main difference in performance be the difference
between code compiler optimised for 64bit and 32 bit? I would actually
like to know this.

I find it not a good idea to encode too much information in the
filename extension (that's how those mypic.jpg.exe trojans spread back
then, perhaps still do). That information as to what executable format
the file is in can be best obtained from reading the file itself, e.g.
by running file(1) on the file.

- Jordi G. H.

[Richard Crozier]

Post by Jordi Gutiérrez Hermoso

Post by Richard Crozier
OK, but won't the main difference in performance be the difference
between code compiler optimised for 64bit and 32 bit? I would actually
like to know this.

I find it not a good idea to encode too much information in the
filename extension (that's how those mypic.jpg.exe trojans spread back
then, perhaps still do). That information as to what executable format
the file is in can be best obtained from reading the file itself, e.g.
by running file(1) on the file.
- Jordi G. H.

Ok, I'll maybe just consider creating one directory for binaries for
each platform and move all that platform's binaries in there after
compilation.

It'll be a bit of extra work to get everything working automatically,
but I suppose that's just the way it is.

Richard

[Richard Crozier]

Post by Jordi Gutiérrez Hermoso
And what do you do about the Octave instances? Do you have
octave.exe64 and octave.exe32 running as well?

I forgot to say, I obviously don't have this as there is only one
instance of octave required per machine, which can be compiled/installed
locally.

I just want to share my path across the machines via the Samba file
share so I can update it in one place.

[John Swensen]

Post by Jordi Gutiérrez Hermoso

Post by Richard Crozier

Post by Jordi Gutiérrez Hermoso

Post by Richard Crozier
On Matlab, I can compile several different mex files for different machine
architectures and Matlab will call the appropriate one for the local setup.
The compiled mex files are given appropriate extensions e.g. mexa64 for
64bit linux, mexglx for 32 bit linux, mexw32 for 32 bit windows etc.
Is the equivalent possible for oct files (or mex files created in Octave)?

Not currently, no. I find that putting the architecture in the
filename extension is a little weird to begin with. Why don't you see
.exe64 and .exe extensions for Windows executables? Can't you just
recompile and distribute your oct files for the architecture you need
them? I get the feeling that Matlab lets you do that because people
don't like recompiling and/or don't like source code.

I have matlab instances running on several different platforms all
accessing the same code directory simultaneously (hosted on a SAMBA file
share).

And what do you do about the Octave instances? Do you have
octave.exe64 and octave.exe32 running as well?
Also, the architecture is not the biggest problem. Even for the same
architecture, you would have to recompile your octfiles if your
machines have slightly different libraries on the OS you're compiling
for. So you would need .oct64deb-6.0, oct64ubu-11.08, .oct64fedora-13
and .oct64suse-5.
I just don't see how filename extensions can solve this problem. It
might solve it for Windows, but I don't see how it would work for
other things. Octave runs on a lot more architectures than Matlab too;
and we compile it for all 11 architectures of Debian; coming up with a
filename extension per architecture seems unwieldy.
I think the solution for now, even if it's inconvenient, is to put
your oct files in directories that have a filename extension specific
to the architecture you care about and in your m-scripts you
conditionally pick to load the directory based on system information
(perhaps after running system("uname -a") or similar).
- Jordi G. H.

Octave is already loading the OCT/MEX files dynamically. This is simply using the architecture and binary that is currently being run to determine which OCT/MEX file to load. This makes a lot of sense for binary distributions (the developer can release all platforms to end-users rather than telling them to "learn how to compile it yourself"). I'm not saying I know how to determine a binary's architecture at runtime (or have time to do it), but clearly Matlab is doing it and I'm sure it isn't as hard as one might think (you may even be able to just use various #defines from the autoconf process).

John Swensen

[c.]

Post by Richard Crozier
Hello,
On Matlab, I can compile several different mex files for different machine
architectures and Matlab will call the appropriate one for the local setup.
The compiled mex files are given appropriate extensions e.g. mexa64 for
64bit linux, mexglx for 32 bit linux, mexw32 for 32 bit windows etc.
Is the equivalent possible for oct files (or mex files created in Octave)?
Otherwise, how are different platforms dealt with? I only see a single
extension mentioned in the docs, and I often run batch jobs on several
machines with access to a common directory containing my mfile sources and
compiled mex files. I can't think how I would do this without this
functionality.
I should also admit I haven't actually tried using oct files yet, so sorry
if I've misunderstood something.
Thanks,
Richard

In installing packages octave separates architecture independent interpreted files from binary architecture
dependent files and places the latter in a directory whos name refers to the machine architecture and octave API version, e.g.

~/octave/nurbs-1.3.5
~/octave/nurbs-1.3.5/x86_64-apple-darwin10.8.0-api-v44+/

c.