Proposal: BeanShell for Interactive Numerical Computing

classic Classic list List threaded Threaded
4 messages Options
Reply | Threaded
Open this post in threaded view
|

Proposal: BeanShell for Interactive Numerical Computing

Bob Fischer
Hello,

This is a proposal on BeanShell extensions that would make it suitable
as the scripting language for a Java-based numerical computing
environment.


The Problem of Exploratory Numerical Computing
==============================================

I am looking to build a system that could feasibly replace
MATLAB/SciLab/Octave (all somewhat compatible with each other) for
scientific computing.  These existing systems are great if you just sit
down and start "doodling"; you can build a matrix and do complex
operations on it in no time.  However, for building large applications,
they fall woefully short:

 1. They are scripting languages, hence they offer no type consistency.
It becomes nearly impossible to build a large system without getting
random runtime errors.
 2. OO support is atrocious.
 3. Support for areas other than applied math (eg: GUI, network I/O,
databases, etc) is poor.  Why not use existing high-quality libraries
for those purposes?
 4. Integration with anything NOT written in the system (i.e. that handy
Java FTP class you have lying around) is even worse.

The one saving grace here is that recent versions of MATLAB (but not
SciLab or Octave) have been "outfitted" to effectively become a weird
hybrid Java scripting language.  You can load up Java classes, call
them, etc.  But there is a big mismatch between the Java and MATLAB
type/object systems.  And you CAN'T just call MATLAB code from Java
without hacking servers and processes and all sorts of stuff you don't
want to do.  Integration is still a big nightmare.


A Java Approach
===============

So... a new approach is needed.
 1. I would like to build it on a Java or C# base.  There are minor pros
and cons to either choice, but I think I'm going to do a Java base for
now.  Java/.NET are both solid bases with lots of quality components
that can be integrated together.
 2. Add a standardized set of basic OO matrix types and matrix
manipulation subroutines.  After looking at a lot of stuff, I think MTJ
(LGPL) will fit the bill.  Even better, it can use JNI interface to
LAPACK and BLAS, which ensure top-notch speed on large matrices.
 3. Add a good plotting program.  JFreeChart (LGPL) should work well, at
least for now.  PtPlot is the other contender.  Both need some
additional work for this purpose, but they're "good enough" for a lot of
stuff.
 4. Add a generally useful file format for storing and retrieving data.
Java serialization turns out to be a very bad way to store numerical
data.  On the other hand, NetCDF (LGPL) is flexible and efficient for
this purpose, is used widely and will help with interoperability with
other NetCDF-based software.  It has a few shortcomings, but in general
offers a lot for the person running simulations.  For those apps that
it's not appropriate, one is free to use another storage facility (JDBC
or Db4o come to mind).

I have done so far integrated all these parts: it provides enough to
start building serious numerical applications in Java.  In one fell
swoop, this solves all the above problems of MATLAB, and provides a
solid foundation to build additional specialized numerical routines.

Unfortunately, there are some things MATLAB does very well that this
approach does very badly.  Namely: (a) the Java syntax is very clumsy
for numerical computation, and (b) interactive "getting at" your data
through an interactive scripting language session is just not possible
with Java.

MATLAB's scripting language is well suited for exploratory work.


Java Scripting Languages
========================

The obvious solution to this problem is to:
 5. Add a good Java-based scripting language to the system
and
 6. As desired, add GUI sugar allowing point-and-click navigation of the
state of your objects in the JVM / scripting interpreter.

I have looked at a LOT of Java scripting languages, and I'm sad to say
that NONE of them are appropriate out-of-the-box.  General problems tend
to be:
 a) Lack of syntactic extensibility.  A few key points of syntactic
sugar are REALLY important here; the whole point is to make this more
convenient than plain Java.
 b) Too far from Java.  Many scripting languages get carried away making
their own type systems, object models, etc --- so much so that there
becomes a mis-match between using the "scripting" language and the
"underlying" (Java) language.  Then you have to write increasing amounts
of binding code just to (conveniently) use your Java classes in your
scripting framework.  I want to write my numerical routines once (in
Java), and be able to call them easily from Java, and at least as easily
from the scripting language.  Certainly, the scripting language should
respect standard Java types.


What is Needed
==============

After careful consideration, I believe that a modified BeanShell would
best suit my needs for this project.  I need the following changes, in
order of decreasing importance:

1. New literal syntax.  Matlab allows you to define a matrix as follows:
   a = [1 2 3; 4 5 6]
There is no good substitute for this syntax in Java.  It needs to be
added.


2. Multi-dimensional indexing syntax.  Java does not have 2-D arrays,
relying on an array-of-arrays for that functionality.  This approach
does not work very well for numerical computing; consequently, matrix
libraries typically define a Matrix class that keeps a 1-D array and
does its own 2-D indexing into that array.  In Java, this introduces
syntactic clumsiness.  Now, I must index my matrix as:
   m.setValue(17,23, 55.0)
when I really want to index it like:
   m[17, 23] = 55.0;
So this needs to be added.



3. Generalized indexing syntax.  One VERY powerful feature of MATLAB is
its ability to use one array to index into a matrix.  For example, you
can use:
    m[1 2 3 , 3:end]
to select rows 1,2 and 3, and columns 3 through the last column of a
matrix. This would be implemented in Java by:
 (a) Create an array of integers specifying the indices desired.
 (b) pass them to a method
        Matrix Matrix.subsection(int[], int[])
 (c) Add syntactic sugar to BeanShell to pull it all together.


4. Operator Overloading.  I need a dispatch table or set of ovverrides
in the interpreter that allows me to map all the standard operators to
arbitrary method calls (static or non-static).  I need to be able to
write:
    A + B
where A and B are of the class Matrix, and I need to be able to call
either:
    A.add(B)                   (a non-static method)
or
    MatrixUtilities.add(A,B)   (a static method)
This system should NOT assume things that might not be true with
matrices --- for example, with matrices, A*B != B*A


5. New operators.  I want a bunch of new operators that are not in
standard Java.  They include: .* ./, some kind of unitary transpose
operator, and probably others.

5. Easy import of static methods.  Supose I have a Java class
my.package.MyClass with a bunch of static methods in it.  I should be
able to say something like
     import my.package.MyClass.*;
and, voila!, all the static methods in MyClass suddenly appear as
functions in my BeanShell space.  This allows one to import Java methods
into BeanShell with the least amount of effort.  Numerical routines are
typically NOT implemented in the class on which they operate --- the
number of matrix operations is FAR too great (and varied) to implement
all in the Matrix class.  Instead, numerical routines are often
implemented as a long series of static methods in a "utility" class, and
there's no point in either having to say "MyClass.function()" all over
the place (which is clumsy) or having to write a zillion BeanShell
functions that do nothing but pass along to the appropriate static
methods in MyClass.


6. "Extensible" classes.  Suppose I have a Matrix class from a library
I'm integrating with BeanShell.  Suppose that library neglected to
implement a method I would find useful for my app (eg, toString()).  I
would like to add a toString() method to it without having to go in and
modify that library.  Subclassing is not an option because the library
already returns Matrix all over the place, and there's no way to coerce
those to "MyMatrix".  Instead, the interpreter could maintain a table of
function aliases within the scripting language.  It would know, for
example:

      Matrix.toString(void) [non-static method]
             maps to
      MatrixUtilities.toString(Matrix) [static method]

once this association is set up, Matrix has, for all intents and
purposes, been "extended".  This will prevent having to go in and modify
the Matrix library in a way useful only to this one application, which I
would then have to keep up-to-date with every new release.


7. Functions dispatched according to the argument types (at runtime),
not just the number of arguments.  (I think BeanShell alreayd does this,
but some scripting languages don't).  For example, I should be able to
declare:
   function add(Matrix A, Matrix B)
and
   function add(Matrix A, Vector B)
and have it choose one or the other function when I call add(A,B),
depending on the types of A and B.


8. Class aliasing.  Some libraries make poor choices of names.  For
example, the Matrix library defines a class called Vector, which will
conflict all over the place with java.util.Vector.  As with extensible
classes, the interpreter could dynamically alias this class, rather than
make me change the library in an incompatible way.


9. Easy printing.  I shouldn't have to say print(...) on the command
line every time I want to see something.  MATLAB automaticaly prints
whatever result you type (which is annoying in its own way).
personally, I like the old BASIC syntax of "? 3 + 4"


10. State inspection.  I don't know if this is possible already.  But
there should be a way to build a GUI "inspector" that can go in and look
at the variables in the BeanShell space.  This is really nice for
exploratory numerical activity; you compute a Matrix, then you want to
look at it to see if it looks right.


11. Complex numbers.  This is a ways off, and not immediately needed by
me.  The problem with the "obvious" approach of adding a Complex Java
class is that it's incredibly inefficient.  Complex arrays can be
represented as double arrays that are twice as long --- presuming you
have numerical routines that can work on them properly.


Evaluation
==========

This is not a request to change BeanShell along the lines above, since
that would cause BeanShell to deviate from its own goals of simplicity
and generality.  Rather, I am looking for guidance on how to best make
some of these changes.  This is something I will do, whether or not it
fits into the BeanShell project agenda, because BeanShell seems to be
the closest to what I need of the Java Scripting Languages that are out
there.  Maybe a fork of some kind is needed, but I'd obviously like to
fork in the least disruptive way (or not at all if possible).

I also think we should continue to consider building a generalized
BeanShell extension mechanism.  The system I propose would be a first
"real world" example of what such a mechanism would need to do.

All feedback is welcome here.  I can't emphasize enough how big a
difference this system would make in the world of science if it could be
put together; PLENTY of people would be eager to use it.  Most of the
pieces are there, they just need to be assembled and tweaked.  The
reason it has not been done yet is because most physical scientists and
engineers don't have the symbolic software skills, and most computer
scientists don't have the numerical skills.  Hence, we're still using
outdated systems from the early 1990's that are barely serving our needs
anymore.



-------------------------------------------------------------------------
Using Tomcat but need to do more? Need to support web services, security?
Get stuff done quickly with pre-integrated technology to make your job easier
Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo
http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642
_______________________________________________
Beanshell-developers mailing list
[hidden email]
https://lists.sourceforge.net/lists/listinfo/beanshell-developers
Reply | Threaded
Open this post in threaded view
|

Re: Proposal: BeanShell for Interactive NumericalComputing

Yemi I. D. Bedu
Hello,
 I just needed to comment on the concluding evaluation statement. I am a
computer science and have both symbolic and numerical skills. Please do
not confuse a simple computer programmer or software developer with a
computer scientist. True I am also an applied math minor, but my
school's curriculum has this pair well intergrated to almost a single
major with different perspectives of minors of computation and
mathematics. Also longevity is not the same as out-dated. I will go
through your points with a cs perspective and comment on your points.
Some are ok, others are relatively redundant, and a few remaining are
redundant with no improvement. Good day.


Yemi Bedu

P&R Fasteners
325 Pierce St
Somerset, NJ 08873
732-302-3600

-----Original Message-----
From: [hidden email]
[mailto:[hidden email]] On Behalf Of
Bob Fischer
Sent: Friday, October 27, 2006 7:01 PM
To: [hidden email]
Subject: [Beanshell-dev] Proposal: BeanShell for Interactive
NumericalComputing

Hello,

This is a proposal on BeanShell extensions that would make it suitable
as the scripting language for a Java-based numerical computing
environment.


The Problem of Exploratory Numerical Computing
==============================================
...
All feedback is welcome here.  I can't emphasize enough how big a
difference this system would make in the world of science if it could be
put together; PLENTY of people would be eager to use it.  Most of the
pieces are there, they just need to be assembled and tweaked.  The
reason it has not been done yet is because most physical scientists and
engineers don't have the symbolic software skills, and most computer
scientists don't have the numerical skills.  Hence, we're still using
outdated systems from the early 1990's that are barely serving our needs
anymore.



------------------------------------------------------------------------
-
Using Tomcat but need to do more? Need to support web services,
security?
Get stuff done quickly with pre-integrated technology to make your job
easier Download IBM WebSphere Application Server v.1.0.1 based on Apache
Geronimo
http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642
_______________________________________________
Beanshell-developers mailing list
[hidden email]
https://lists.sourceforge.net/lists/listinfo/beanshell-developers

-------------------------------------------------------------------------
Take Surveys. Earn Cash. Influence the Future of IT
Join SourceForge.net's Techsay panel and you'll get the chance to share your
opinions on IT & business topics through brief surveys - and earn cash
http://www.techsay.com/default.php?page=join.php&p=sourceforge&CID=DEVDEV
_______________________________________________
Beanshell-developers mailing list
[hidden email]
https://lists.sourceforge.net/lists/listinfo/beanshell-developers
Reply | Threaded
Open this post in threaded view
|

Re: Proposal: BeanShell for Interactive NumericalComputing

Raphaël Valyi
Hi, did you look at the FrAid java scripting language? It's missing a few common data structures like matrix but functionnal solutions are often possible and a pure java back end function is always an option. For instance there is already a Runge Kutta solver whithout the need of matrix.

What I like in FrAid is that:
* it's more a Matlab like syntax (unlike BeanShell) (I used it in a simulator dedicated to Matlab users).
* FrAid isn't a monster (unlike some other scripting languages). It rather follows a simple clear BNF grammar specification compiled into an interpreter via the JavaCC and JJTree standard tools and only focuses on filling the gap between the math functionnal syntax and the java language.
* It comes with a bunch of built'in math functions

May be you can have a look:
<a href="http://fraid.sourceforge.net/" target="_blank" onclick="return top.js.OpenExtLink(window,event,this)">http://fraid.sourceforge.net/
there is also a set of nice applet demos such as: <a href="http://fraid.sourceforge.net/begin/math3d_demo.html" target="_blank" onclick="return top.js.OpenExtLink(window,event,this)"> http://fraid.sourceforge.net/begin/math3d_demo.html
and the integration with java is detailed here: <a href="http://fraid.sourceforge.net/java/embedding.html" target="_blank" onclick="return top.js.OpenExtLink(window,event,this)"> http://fraid.sourceforge.net/java/embedding.html

Best regards and let us know please...

Raphael Valyi, France.




On 10/30/06, Yemi I. D. Bedu <[hidden email]> wrote:
Hello,
I just needed to comment on the concluding evaluation statement. I am a
computer science and have both symbolic and numerical skills. Please do
not confuse a simple computer programmer or software developer with a
computer scientist. True I am also an applied math minor, but my
school's curriculum has this pair well intergrated to almost a single
major with different perspectives of minors of computation and
mathematics. Also longevity is not the same as out-dated. I will go
through your points with a cs perspective and comment on your points.
Some are ok, others are relatively redundant, and a few remaining are
redundant with no improvement. Good day.


Yemi Bedu

P&R Fasteners
325 Pierce St
Somerset, NJ 08873
732-302-3600

-----Original Message-----
From: [hidden email]
[mailto:[hidden email]] On Behalf Of
Bob Fischer
Sent: Friday, October 27, 2006 7:01 PM
To: [hidden email]
Subject: [Beanshell-dev] Proposal: BeanShell for Interactive
NumericalComputing

Hello,

This is a proposal on BeanShell extensions that would make it suitable
as the scripting language for a Java-based numerical computing
environment.


The Problem of Exploratory Numerical Computing
==============================================
...
All feedback is welcome here.  I can't emphasize enough how big a
difference this system would make in the world of science if it could be
put together; PLENTY of people would be eager to use it.  Most of the
pieces are there, they just need to be assembled and tweaked.  The
reason it has not been done yet is because most physical scientists and
engineers don't have the symbolic software skills, and most computer
scientists don't have the numerical skills.  Hence, we're still using
outdated systems from the early 1990's that are barely serving our needs
anymore.



------------------------------------------------------------------------
-
Using Tomcat but need to do more? Need to support web services,
security?
Get stuff done quickly with pre-integrated technology to make your job
easier Download IBM WebSphere Application Server v.1.0.1 based on Apache
Geronimo
<a href="http://sel.as-us.falkag.net/sel?cmd=lnk&amp;kid=120709&amp;bid=263057&amp;dat=121642" target="_blank" onclick="return top.js.OpenExtLink(window,event,this)">http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642
_______________________________________________
Beanshell-developers mailing list
[hidden email]
<a href="https://lists.sourceforge.net/lists/listinfo/beanshell-developers" target="_blank" onclick="return top.js.OpenExtLink(window,event,this)"> https://lists.sourceforge.net/lists/listinfo/beanshell-developers

-------------------------------------------------------------------------
Take Surveys. Earn Cash. Influence the Future of IT
Join SourceForge.net 's Techsay panel and you'll get the chance to share your
opinions on IT & business topics through brief surveys - and earn cash
<a href="http://www.techsay.com/default.php?page=join.php&amp;p=sourceforge&amp;CID=DEVDEV" target="_blank" onclick="return top.js.OpenExtLink(window,event,this)"> http://www.techsay.com/default.php?page=join.php&p=sourceforge&CID=DEVDEV
_______________________________________________
Beanshell-developers mailing list
[hidden email]
<a href="https://lists.sourceforge.net/lists/listinfo/beanshell-developers" target="_blank" onclick="return top.js.OpenExtLink(window,event,this)">https://lists.sourceforge.net/lists/listinfo/beanshell-developers


-------------------------------------------------------------------------
Take Surveys. Earn Cash. Influence the Future of IT
Join SourceForge.net's Techsay panel and you'll get the chance to share your
opinions on IT & business topics through brief surveys - and earn cash
http://www.techsay.com/default.php?page=join.php&p=sourceforge&CID=DEVDEV
_______________________________________________
Beanshell-developers mailing list
[hidden email]
https://lists.sourceforge.net/lists/listinfo/beanshell-developers
Reply | Threaded
Open this post in threaded view
|

Re: Proposal: BeanShell for Interactive NumericalComputing

Raphaël Valyi
In reply to this post by Yemi I. D. Bedu
Also one final comment:

IMHO, the dreamed Java scripting language would be JRuby (the Ruby language in the JVM) associated with the functions defined in FrAid plus a few others like from the Colt package for instance. The advantage of using JRuby is that you would get the ruby language built'in features to deal with complex data structures (like tables, matrix, multidimensional arrays...) without the need to build yet an other interpreter monster that will turn into unmaintainable and thus unmaintained code sooner than you might expect.

Whether you can translate the FrAid java functions to JRuby depends on how much time you can invest. But still it might be less than adding a whole new syntax + backend functions to Beanshell which is nice playing with Java syntax only. If you do so I would be very interrested in using such a  language (or JRuby addon if you prefer), unfortunately I've no time to contribute those days I'm sorry. If you lack of time, sticking with FrAid might be the faster.

Raphaël Valyi



On 11/28/06, Raphaël Valyi <[hidden email]> wrote:
Hi, did you look at the FrAid java scripting language? It's missing a few common data structures like matrix but functionnal solutions are often possible and a pure java back end function is always an option. For instance there is already a Runge Kutta solver whithout the need of matrix.

What I like in FrAid is that:
* it's more a Matlab like syntax (unlike BeanShell) (I used it in a simulator dedicated to Matlab users).
* FrAid isn't a monster (unlike some other scripting languages). It rather follows a simple clear BNF grammar specification compiled into an interpreter via the JavaCC and JJTree standard tools and only focuses on filling the gap between the math functionnal syntax and the java language.
* It comes with a bunch of built'in math functions

May be you can have a look:
<a href="http://fraid.sourceforge.net/" target="_blank" onclick="return top.js.OpenExtLink(window,event,this)">http://fraid.sourceforge.net/
there is also a set of nice applet demos such as: <a href="http://fraid.sourceforge.net/begin/math3d_demo.html" target="_blank" onclick="return top.js.OpenExtLink(window,event,this)"> http://fraid.sourceforge.net/begin/math3d_demo.html
and the integration with java is detailed here: <a href="http://fraid.sourceforge.net/java/embedding.html" target="_blank" onclick="return top.js.OpenExtLink(window,event,this)"> http://fraid.sourceforge.net/java/embedding.html

Best regards and let us know please...

Raphael Valyi, France.




On 10/30/06, Yemi I. D. Bedu <[hidden email]> wrote:
Hello,
I just needed to comment on the concluding evaluation statement. I am a
computer science and have both symbolic and numerical skills. Please do
not confuse a simple computer programmer or software developer with a
computer scientist. True I am also an applied math minor, but my
school's curriculum has this pair well intergrated to almost a single
major with different perspectives of minors of computation and
mathematics. Also longevity is not the same as out-dated. I will go
through your points with a cs perspective and comment on your points.
Some are ok, others are relatively redundant, and a few remaining are
redundant with no improvement. Good day.


Yemi Bedu

P&R Fasteners
325 Pierce St
Somerset, NJ 08873
732-302-3600

-----Original Message-----
From: [hidden email]
[mailto:[hidden email]] On Behalf Of
Bob Fischer
Sent: Friday, October 27, 2006 7:01 PM
To: [hidden email]
Subject: [Beanshell-dev] Proposal: BeanShell for Interactive
NumericalComputing

Hello,

This is a proposal on BeanShell extensions that would make it suitable
as the scripting language for a Java-based numerical computing
environment.


The Problem of Exploratory Numerical Computing
==============================================
...
All feedback is welcome here.  I can't emphasize enough how big a
difference this system would make in the world of science if it could be
put together; PLENTY of people would be eager to use it.  Most of the
pieces are there, they just need to be assembled and tweaked.  The
reason it has not been done yet is because most physical scientists and
engineers don't have the symbolic software skills, and most computer
scientists don't have the numerical skills.  Hence, we're still using
outdated systems from the early 1990's that are barely serving our needs
anymore.



------------------------------------------------------------------------
-
Using Tomcat but need to do more? Need to support web services,
security?
Get stuff done quickly with pre-integrated technology to make your job
easier Download IBM WebSphere Application Server v.1.0.1 based on Apache
Geronimo
<a href="http://sel.as-us.falkag.net/sel?cmd=lnk&amp;kid=120709&amp;bid=263057&amp;dat=121642" target="_blank" onclick="return top.js.OpenExtLink(window,event,this)">http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642
_______________________________________________
Beanshell-developers mailing list
[hidden email]
<a href="https://lists.sourceforge.net/lists/listinfo/beanshell-developers" target="_blank" onclick="return top.js.OpenExtLink(window,event,this)"> https://lists.sourceforge.net/lists/listinfo/beanshell-developers

-------------------------------------------------------------------------
Take Surveys. Earn Cash. Influence the Future of IT
Join SourceForge.net 's Techsay panel and you'll get the chance to share your
opinions on IT & business topics through brief surveys - and earn cash
<a href="http://www.techsay.com/default.php?page=join.php&amp;p=sourceforge&amp;CID=DEVDEV" target="_blank" onclick="return top.js.OpenExtLink(window,event,this)"> http://www.techsay.com/default.php?page=join.php&p=sourceforge&CID=DEVDEV
_______________________________________________
Beanshell-developers mailing list
[hidden email]
<a href="https://lists.sourceforge.net/lists/listinfo/beanshell-developers" target="_blank" onclick="return top.js.OpenExtLink(window,event,this)">https://lists.sourceforge.net/lists/listinfo/beanshell-developers



-------------------------------------------------------------------------
Take Surveys. Earn Cash. Influence the Future of IT
Join SourceForge.net's Techsay panel and you'll get the chance to share your
opinions on IT & business topics through brief surveys - and earn cash
http://www.techsay.com/default.php?page=join.php&p=sourceforge&CID=DEVDEV
_______________________________________________
Beanshell-developers mailing list
[hidden email]
https://lists.sourceforge.net/lists/listinfo/beanshell-developers