Comparison of mathematical programs for data analysis 

Author :  Stefan Steinhaus (University of Frankfurt, Germany) 
EMail / Homepage :  stst@informatik.unifrankfurt.de / http://www.unifrankfurt.de/~stst/ 
Edition :  2.12 
Date :  September 8, 1997 
Last modified :  September 27, 1997 
Location :  http://www.unifrankfurt.de/~stst/ncrunch.html 
In the first part of this test report you can find an overview over the mathematical functionality divided into different subsections, in the second part you can find an overview about the graphical functionality, in the third part you can find an overview about the programming environment, in the fourth part you can find a list of available export/import options, in the fifth part there is a list of operating systems for which the programs are available and in the sixth part you can find a speed comparison on these systems. In the seventh part there will be a final summarize to relate all the different parts of the test report. The relation between these six parts will be 38 : 10 : 8 : 5 : 2 : 37 but for those who want to have their own weighting I will describe detailed the calculation for the summarize so that you can easily calculate the result with your own weighting.
The actual tested programs are :
+  Function is implemented
in the program
m  Function is supported by
an additional module
  Function is not implemented
Functions
(Version) 
GAUSS  Mac
syma 
Maple  Mathe
matica 
Mat
lab 
Mu
PAD 
O
Matrix 
Ox  S
Plus 
(3.2.29)  (2.2.1)  (V4)  (3.0.1)  (5.1)  (1.3.0)  (3.2)  (1.11)  (V4)  
Trig. functions  +  +  +  +  +  +  +  +  + 
Gamma function  +  +  +  +  +  +      + 
Poly gamma    +  +  +        +   
LogGammafunc.  +  +  +  +  +  +  +  +  + 
Beta function    +  +  +  +  +  +  +   
Implemented functions  60.000%
(3/5) 
100.000%
(5/5) 
100.000%
(5/5) 
100.000%
(5/5) 
80.000%
(4/5) 
80.000%
(4/5) 
60.000%
(3/5) 
80.000%
(4/5) 
60.000%
(3/5) 
Functions
(Version) 
GAUSS  Mac
syma 
Maple  Mathe
matica 
Mat
lab 
Mu
PAD 
O
Matrix 
Ox  S
Plus 
(3.2.29)  (2.2.1)  (V4)  (3.0.1)  (5.1)  (1.3.0)  (3.2)  (1.11)  (V4)  
Eigenvalues  +  +  +  +  +  +  +  +  + 
Eigenvectors  +  +  +  +  +  +  +  +  + 
Cholesky decomposition  +  +  +  +  +  +  +  +  + 
Crout decomposition  +                 
LU decomposition  +  +  +  +  +      +  + 
Singular value decomposition  +  +  +  +  +    +  +  + 
Upper Hessenberg form  +  +      +         
Toeplitz matrix  +  +  +  m  +         
Schur form of quadratic matrix  +  +    +  +        + 
Optimization
(Unconstr. / Constr.) 
m / m  + /   + /   + / +  m / m   /   + / +  + /   + / + 
Linear equation solver  +  +  +  +  +  +  +  +  + 
Nonlinear equation solver  m  +  +  +  m  +  +    + 
Ordinary Differential Equation solver  m  +  +  +  +  +  +    + 
Partial Differential Equation solver    m  +  +  m         
Sparse matrices handling  +  +    +  +         
MoorePenrose pseudoinverse  +  +    +  +      +  + 
Implemented functions  94.118%
(16/17) 
88.235%
(15/17) 
64.706%
(11/17) 
88.235%
(15/17) 
94.118%
(16/17) 
35.294%
(6/17) 
52.941%
(9/17) 
47.059%
(8/17) 
70.588%
(12/17) 
Functions
(Version) 
GAUSS  Mac
syma 
Maple  Mathe
matica 
Mat
lab 
Mu
PAD 
O
Matrix 
Ox  S
Plus 
(3.2.29)  (2.2.1)  (V4)  (3.0.1)  (5.1)  (1.3.0)  (3.2)  (1.11)  (V4)  
Numerical integration  +  +  +  +  +  +  +  m  + 
Double integration  +  +  +  +  +  +  +     
Triple integration  +  +  +  +    +       
Numerical differentiation  +  +  +  +  +  +    +  + 
Fourier transf.
(1D / 2D /multidim.) 
+ / + / +  + /  /   + /  /   + / + / +  + / + / +  + /  /   + / + /   + /  /   + / + / + 
Inverse Fourier transformation
(1D / 2D / multidim.) 
+ / + / +  + /  /   + /  /   + / + / +  + / + / +  + /  /   + / + /   + /  /   + / + / + 
Implemented functions  100.000%
(10/10) 
60.000%
(6/10) 
60.000%
(6/10) 
100.000%
(10/10) 
90.000%
(9/10) 
60.000%
(6/10) 
60.000%
(6/10) 
40.000%
(4/10) 
60.000%
(6/10) 
Functions
(Version) 
GAUSS  Mac
syma 
Maple  Mathe
matica 
Mat
lab 
Mu
PAD 
O
Matrix 
Ox  S
Plus 
(3.2.29)  (2.2.1)  (V4)  (3.0.1)  (5.1)  (1.3.0)  (3.2)  (1.11)  (V4)  
Classical Interpolation    +  +  +  +  +  +    + 
Pade Interpolation  m  +  +  +    +  +     
kSpline Interpolation  +  +  +  +  m    +  +  + 
BSpline Interpolation      +  +  m      +  + 
Newton method for finding roots  m  +  +  +  m  +  +     
Bisection  m  +    m  m         
Runge Kutta method for solving ODE  m  +  +  +  m    +     
Implemented functions  71.429%
(5/7) 
85.714%
(6/7) 
85.714%
(6/7) 
100.000%
(7/7) 
85.714%
(6/7) 
42.857%
(3/7) 
71.429%
(5/7) 
28.571%
(2/7) 
42.857%
(3/7) 
Functions
(Version) 
GAUSS  Mac
syma 
Maple  Mathe
matica 
Mat
lab 
Mu
PAD 
O
Matrix 
Ox  S
Plus 
(3.2.29)  (2.2.1)  (V4)  (3.0.1)  (5.1)  (1.3.0)  (3.2)  (1.11)  (V4)  
Markov models  m  +               
Mean  +  +  +  +  +  +  +  +  + 
Median  +  +  +  +  +    +  +  + 
Mode  m    +  +    +       
Variance  +  +  +  +  +  +  +  +  + 
Beta distribution
(Density / CDF / random num.) 
m / + / +  + / + /   + / + / +  + / + / +  m / m / m   /  /    /  /   + / + / +  + / + / + 
Chisquared distr.
(Density / CDF / random num.) 
m / + / m  + / + /   + / + / +  + / + / +  m / m / m   /  /    /  /   + / + / +  + / + / + 
Gamma distr.
(Density / CDF / random num.) 
m / + / +  + / + /   + / + / +  + / + / +  m / m / m   /  /    /  /   + / + / +  + / + / + 
Lognormal distr.
(Density / CDF / random num.) 
+ / + / m  + / + /   + / + / +  + / + / +  m / m / m   /  /    /  /    /  /   + / + / + 
Normal distr.
(Density / CDF / random num.) 
+ / + / +  + / + /   + / + / +  + / + / +  + / + / +   /  /    / + / +  + / + / +  + / + / + 
Poisson distr.
(Density / CDF / random num.) 
m / m / +  + / + /   + / + / +  + / + / +  m / m / m   /  /    /  /    /  / +  + / + / + 
Uniform distr.
(Density / CDF / random num.) 
m / m / +  + / + / +  + / + / +  + / + / +  + / + / +   /  / +   /  / +   /  / +  + / + / + 
More distr.
(Density / CDF / random num.) 
+ / + / +  + / + /   + / + / +  + / + / +  m / m / m   /  /    / + /   + / + / +  + / + / + 
Implemented functions  100.000%
(29/29) 
72.414%
(21/29) 
96.552%
(28/29) 
96.552%
(28/29) 
93.103%
(27/29) 
13.793%
(4/29) 
24.138%
(7/29) 
68.966%
(20/29) 
93.103%
(27/29) 
Functions
(Version) 
GAUSS  Mac
syma 
Maple  Mathe
matica 
Mat
lab 
Mu
PAD 
O
Matrix 
Ox  S
Plus 
(3.2.29)  (2.2.1)  (V4)  (3.0.1)  (5.1)  (1.3.0)  (3.2)  (1.11)  (V4)  
Linear regression  +  +  +  +  +    +  +  + 
Polynomial regression  m  +  +  +  +    +    + 
Nonlinear Regression  m  +    +  m    +    + 
Loess regression  +      m  +        + 
LOGIT regression  m                + 
PROBIT regression  m              m  + 
PSN regression  m                 
Event count models  m                + 
Duration models  m                 
Goodness of fit test  m                + 
TTest        +  m    +    + 
FTest        +      +    + 
QTest  m                 
ZTest          m        + 
Maximum Likelihood
(Unconstr. / Constr.) 
m / m   /    /    /   m /    /    /    /   + / + 
ARIMA  m      m  m      m  + 
Time series analysis
(Stationary / Nonstat.) 
m / m   /    /   m / m  m / m   /    /   + / m  + / + 
GARCH models
(Univariate / Multivar.) 
m / m   /    /   m / m   /    /    /    /   m / m 
Wavelets  m      m  m    +    m 
Cluster analysis                  + 
Survival analysis                  + 
Implemented functions  79.167%
(19/24) 
12.500%
(3/24) 
8.333%
(2/24) 
50.000%
(12/24) 
45.833%
(11/24) 
0.000%
(0/24) 
25.000%
(6/24) 
20.833%
(5/24) 
87.500%
(21/24) 
Functions
(Version) 
GAUSS  Mac
syma 
Maple  Mathe
matica 
Mat
lab 
Mu
PAD 
O
Matrix 
Ox  S
Plus 
(3.2.29)  (2.2.1)  (V4)  (3.0.1)  (5.1)  (1.3.0)  (3.2)  (1.11)  (V4)  
Cointegration models  m              +   
Black Scholes model    +  +  m  m         
Dynamic rational expectation models  m                 
Linear rational expectation models  m        m         
Nonlinear rational expectation models  m                 
Social network models  m                 
Kalman filter  m      m  m    +  m   
Neuronal networks  m      m  m         
Regressiveautore
gressive models 
m      m  m        + 
Portfolio analysis  m      m  m         
Statespace models  m      m  m      m   
Implemented
functions 
90.909%
(10/11) 
9.091%
(1/11) 
9.091%
(1/11) 
54.545%
(6/11) 
63.636%
(7/11) 
0.000%
(0/11) 
9.091%
(1/11) 
27.273%
(3/11) 
9.091%
(1/11) 
Functions
(Version) 
GAUSS  Mac
syma 
Maple  Mathe
matica 
Mat
lab 
Mu
PAD 
O
Matrix 
Ox  S
Plus 
(3.2.29)  (2.2.1)  (V4)  (3.0.1)  (5.1)  (1.3.0)  (3.2)  (1.11)  (V4)  
Standard mathematics (5%)  60.000%  100.000%  100.000%  100.000%  80.000%  80.000%  60.000%  80.000%  60.000% 
Linear Algebra (15%)  94.118%  88.235%  64.706%  88.235%  94.118%  35.294%  52.941%  47.059%  70.588% 
Analysis (10%)  100.000%  60.000%  60.000%  100.000%  90.000%  60.000%  60.000%  40.000%  60.000% 
Numerical mathematics (10%)  71.429%  85.714%  85.714%  100.000%  85.714%  42.857%  71.429%  28.571%  42.857% 
Stochastic (20%)  100.000%  72.414%  96.552%  96.552%  93.103%  13.793%  24.138%  68.966%  93.103% 
Statistics (20%)  79.167%  12.500%  8.333%  50.000%  45.833%  0.000%  25.000%  20.833%  87.500% 
Other mathematics (20%)  90.909%  9.091%  9.091%  54.545%  63.636%  0.000%  9.091%  27.273%  9.091% 
Overall result
(100% = Best) 
88.276%
(92/103) 
51.608%
(57/103) 
52.073%
(59/103) 
78.455%
(83/103) 
76.204%
(80/103) 
22.338%
(23/103) 
35.730%
(37/103) 
41.330%
(46/103) 
61.813%
(75/103) 
100% would mean that all listed functions are implemented (103 of 103).
Functions
(Version) 
GAUSS  Mac
syma 
Maple  Mathe
matica 
Mat
lab 
Mu
PAD 
O
Matrix 
Ox  S
Plus 
(3.2.29)  (2.2.1)  (V4)  (3.0.1)  (5.1)  (1.3.0)  (3.2)  (1.11)  (V4)  
2DGraphics  
Bar charts  +  +  +  +  +    +    + 
Other charts  +      +  +    +    + 
Error bars        +  +    +    + 
HighLowAverage Plot        m  m        + 
Histograms  +  +  +  +  +    +  +  + 
Log Plot  +  +  +  +  +    +    + 
Loglog Plot  +  +  +  +  +    +    + 
Polar Plot  +  +  +  +  +  +  +    + 
XY Plot  +  +  +  +  +  +  +  +  + 
3DGraphics  
Charts        +  +        + 
Contour Plot  +  +  +  +  +  +  +    + 
Error bars                   
Height colors  +  +  +  +  +  +  +    + 
Surface Plot  +  +  +  +  +  +  +    + 
XYZ Plot  +  +  +  +  +  +  +    + 
Special graphic types and functions  
Animations    +  +  +  +  +       
Bollinger bands        m  m         
Box & Whisker Plots  +    +  m  m        + 
Candlestick charts        m  m         
Cluster graphs                  m 
Dendograms                  m 
Periodograms                +  + 
QQ Plot      +    m      +  + 
Overall result
(100% = Best) 
52.174%
(12/23) 
47.826%
(11/23) 
56.522%
(13/23) 
78.261%
(18/23) 
82.609%
(19/23) 
30.435%
(7/23) 
52.174%
(12/23) 
17.391%
(4/23) 
82.609%
(19/23) 
Programming facilities  GAUSS  Mac
syma 
Maple  Mathe
matica 
Mat
lab 
Mu
PAD 
O
Matrix 
Ox  S
Plus 
(3.2.29)  (2.2.1)  (V4)  (3.0.1)  (5.1)  (1.3.0)  (3.2)  (1.11)  (V4)  
Editing features  
Builtin editor  +  +  +  +  +  +  +  +  + 
External editor configurable  +    +  +  +        + 
Source code formatting  m  +      +        + 
Syntax highlighting          +      m   
Debugging  
Breakpoints  +  +  +    +  +  +    + 
Function Tracer    +  +  +    +       
Line Tracing  +        +  +  +    + 
Profiler    +  +  m  +  +  +     
Stack inspection  +  +    +  +  +  +    + 
Variable inspection  +  +    +  +  +  +    + 
Language features  
APIinterface  +      +  +    +  +  m 
DDE support          +  +      + 
GUI programming        +  +    +    + 
Ndimensional arrays (> 3)    +  +  +  +  +    +  + 
Object oriented programming        +  +  +  +  +  + 
OLE support      +  +  +  +      + 
P code compiling  +  +    +  +      +   
Language interfaces  
Assembler  +      +        +   
C/C++  +      +  +      +  + 
FORTRAN  +    +  +  +      +  + 
GAUSS  +              +   
Macsyma    +               
Maple  m    +    m         
Mathematica        +  m         
Matlab    +    m  +         
MuPAD            +       
OMatrix              +     
Ox                +   
SPlus                  + 
DLLCalls  +      +  +    +  +  + 
Overall result
(100% = Best) 
50.000%
(15/30) 
36.667%
(11/30) 
30.000%
(9/30) 
60.000%
(18/30) 
73.333%
(22/30) 
40.000%
(12/30) 
36.667%
(11/30) 
40.000%
(12/30) 
56.667%
(17/30) 
Data import/export
possibilities 
GAUSS  Mac
syma 
Maple  Mathe
matica 
Mat
lab 
Mu
PAD 
O
Matrix 
Ox  S
Plus 
(3.2.29)  (2.2.1)  (V4)  (3.0.1)  (5.1)  (1.3.0)  (3.2)  (1.11)  (V4)  
ACCESS        m          + 
Applixware          m         
ASCII  +  +  +  +  +  +  +  +  + 
AutoCAD    +    +           
Binary  +    +  +  +  +  +  +   
dBase  m                + 
Excel  m      m  m      +  + 
FoxPro  m                + 
GAUSS  +              +  + 
Hypercard        m           
Informix        m           
Labview        m           
Lotus 123  m        +      +  + 
Lotus Symphony  m                + 
Matlab        m  +    +    + 
ODBCconnections                  + 
Ox                +   
Paradox  m                + 
Quattro Pro  m                + 
SAS                  + 
SigmaPlot                  + 
SPlus                  + 
SPSS                  + 
STATA                  + 
Systat                  + 
Transform        m           
Overall result
(100% = Best) 
38.462%
(10/26) 
7.692%
(2/26) 
7.692%
(2/26) 
38.462%
(10/26) 
23.077%
(6/26) 
7.692%
(2/26) 
11.538%
(3/26) 
23.077%
(6/26) 
69.231%
(18/26) 
Platform
(Version) 
GAUSS  Mac
syma 
Maple  Mathe
matica 
Mat
lab 
Mu
PAD 
O
Matrix 
Ox  S
Plus 
(3.2.29)  (2.2.1)  (V4)  (3.0.1)  (5.1)  (1.3.0)  (3.2)  (1.11)  (V4)  
Convex      +  +    +       
DEC
(Linux / UNIX / VMS) 
 / + / +   /  / +   / + /    / + / +   / + / +  + / + / +   /  /    / + /    / + /  
HP 9000
(HPUX / NextStep) 
+ /   + /   + /   + / +  + /   + /    /   + /   + /  
IBM RISC
(IBM AIX) 
+  +  +  +  +  +    +  + 
Intel (DOS)  +    +      +    +   
Intel (OS/2)  +    +  +    +       
Intel
(Win. 3.1x / 95,NT) 
 / +  + / +  + / +   / +   / +   / +  + / +  + / +  + / + 
Intel (Linux)  +    +  +  +  +    +   
Intel
(Solaris x86/NextStep) 
+ /    /    /    / +   /   + / +   /    /    /  
Motorola
(MAC OS / NextStep) 
 /    /   + / +  + / +  + /   + /    /    /    /  
SGI (SGI IRIX)  +  +  +  +  +  +    +  + 
SUN (Solaris)  +  +  +  +  +  +    +  + 
Total amount  61.111%
(11/18) 
33.333%
(6/18) 
72.222%
(13/18) 
77.778%
(14/18) 
50.000%
(9/18) 
83.333%
(15/18) 
11.111%
(2/18) 
50.000%
(9/18) 
38.889%
(7/18) 
The assessment for this part of the test report is also calculated by the key amount of available platforms divided by the total amount of listed platforms and will be displayed in percentage.
Functions  GAUSS  Mac
syma 
Maple  Mathe
matica 
Mat
lab 
Mu
PAD 
O
Matrix 
Ox  S
Plus 
(3.2.29)  (2.2.1)  (V4)  (3.0.1)  (5.1)  (1.3.0)  (3.2)  (1.11)  (V4)  
400 x 400 random matrix^1000  16.476  136.747  > 1h  30.023  5.468  > 1h  0.400  1.011  1.260 
Eigenvalues of a 300 x 300 random matrix  44.670  9.984  > 1h  45.626  48.760  > 1h  11.937  41.202  49.830 
Inverse of a 500 x 500 random matrix  27.476  149.014  > 1h  142.054  27.504  *  26.678  26.969  99.280 
Sorting of 500000 random values  8.202  434.013  ***  108.550  33.782  54.58  11.877  8.258  5.150 
Creation of a 800 x 800 Toeplitzmatrix  1.718  1660.788  207.748  187.289  2.009  > 1h  37.735  0.307  13.160 
Cholesky decomposition of a 500 x 500 random matrix  4.100  **  > 1h  927.624  3.939  **  134.453  4.089  9.750 
Creation of a 500 x 500 crossproduct matrix  28.265  > 1h  > 1h  59.366  24.423  > 1h  7.621  13.205  35.320 
FFT over 100000 random values  6.940  2103.384  ****  23.054  2.686  46.76  1.502  5.651  3.180 
Gaussian error function over a 500 x 500 random matrix  4.048  966.599  > 1h  55.74  15.790  > 1h  1.572  1.228   
Gamma function on a 600 x 600 random matrix  2.715  > 1h  > 1h  94.946  27.886  > 1h  6.138  6.723  6.600 
Linear regression over a 500 x 500 random matrix  22.184  2017.682  77.702  > 1h  10.886    7.661  16.941  29.770 
Overall performance  49.937%  10.971%  0.013%  7.674%  39.975%  1.658%  70.799%  66.207%  37.176% 
The overall performance will be calculated in the following way :
The best timing result of a benchmark function makes 100%; for calculating the results for each function I'll take the fastest timing and divide it by the timing of the tested program (the formula will look MIN(A1;A2;...)/A2 for example) and that makes the ranking in percentage. To calculate the final "Overall performance" I'll than sum the percentage values for each tested program and divide by the amount of tested functions (in the moment 12) which will make the result in percentage again.
Test  GAUSS  Mac
syma 
Maple  Mathe
matica 
Mat
lab 
Mu
PAD 
O
Matrix 
Ox  S
Plus 
(3.2.29)  (2.2.1)  (V4)  (3.0.1)  (5.1)  (1.3.0)  (3.2)  (1.11)  (V4)  
Comparison of the mathematical functionality (38%)  88.276%  51.608%  52.073%  78.455%  76.204%  22.338%  35.730%  41.330%  61.813% 
Comparison of the graphical functionality (10%)  52.174%  47.826%  56.522%  78.261%  82.609%  30.435%  52.174%  17.391%  82.609% 
Functionality of the programming environment (8%)  50.000%  36.667%  30.000%  60.000%  73.333%  40.000%  36.667%  40.000%  56.667% 
Data import/export (5%)  38.462%  7.692%  7.692%  38.462%  23.077%  7.692%  11.538%  23.077%  69.231% 
Available platforms (2%)  61.111%  33.333%  72.222%  77.778%  50.000%  83.333%  11.111%  50.000%  38.889% 
Speed comparison (37%)  49.937%  10.971%  0.013%  7.674%  39.975%  1.658%  70.799%  66.207%  37.176% 
Overall result  64.384%  32.437%  29.674%  48.757%  60.029%  17.397%  48.723%  47.295%  54.277% 
Note : The overall results of some tested programs are pretty bad due to the specific weighting of this testreport. I would like to mention that this does of course not mean that the software is bad but that the programs are maybe not perfect for the specific usage mentioned in the testreport, for other weightings/usages they might be much better or even leading.
Pricing : Another important point to mention is also that I have not reported the prices for each software product nor the additional modules which might make some products quiete expensive. Summarizing I would like to notice that the programs MuPAD, the lightversion of OMatrix and Ox are available free of charge for educational and academic usage. The other software products are varying very strongly in their prices and in their license politics.