Revision 0.4.1

Unstable changes

• Improved serialize/deserialize functions, reimplemented all the serialization
  procedure.
• Added exceptions support to LuaPkg and APRIL-ANN, allowing to capture C++
  errors into Lua code.
• Added set class.
• Added series class.
• Added data_frame class, similar to Python Pandas DataFrame.
• Serialization and deserilization have been updated with more robust and
  reusable API, implemented in util.serialize() and util.deserialize()
  functions.
• Added matrix.ext.broadcast utility (similar to broadcast in numpy).
• Added ProbablisitcMatrixANNComponent, which allow to implement probabilistic
  mixtures of posteriors and/or likelihoods.

API Changes

• Added batch normalization ANN component.
• Allowing matrix.join to add new axis.
• Added methods prod(), cumsum() and cumprod() at matrix classes.
• Added methods count_eq() and count_neq() at matrix classes.
• Serializable objects API have been augmented with methods ctor_name() and
  ctor_params() in Lua, refered to luaCtorName() and luaCtorParams() in
  C++.
• Added cast.to to dynamic cast C++ objects pushed into Lua, allowing to
  convert base class objects into any of its derived classes.
• Added matrix.sparse as valid values for targets in ann.loss.mse and
  ann.loss.cross_entropy.
• Changed matrix metamethods __index and __newindex, allowing to use
  matrix objects with standard Lua operator[].
• Added matrix.masked_fill and matrix.masked_copy matrix.
• Added matrix.indexed_fill and matrix.indexed_copy matrix.
• Added ann.components.probabilistic_matrix, and its corresponding
  specializations ann.components.left_probabilistic_matrix and
  ann.components.right_probabilistic_matrix.
• Added operator[] in the right side of matrix operations.
• Added ann.components.transpose.
• Added max_gradients_norm in traianble.supervised_trainer, to avoid
  gradients exploding.
• Added ann.components.actf.sparse_logistic a logistic activation function
  with sparsity penalty.
• Simplified math.add, math.sub, ... and other math extensions for
  reductions, their original behavior can be emulated by using bind function.
• Added bind function to freeze any positional argument of any Lua function.
• Function stats.boot uses multiple_unpack to allow a table of sizes and the
  generation of multiple index matrices.
• Added multiple_unpack Lua function.
• Added __tostring metamethod to numeric memory blocks in Lua.
• Added dataset.token.sparse_matrix, a dataset which allow to traverse by rows
  a sparse matrix instance.
• Added matrix.sparse.builders.dok, a builder which uses the
  Dictionary-of-Keys format to construct a sparse matrix from scratch.
• Added method data to numeric matrix classes.
• Added methods values, indices, first_index to sparse matrix class.

Bugs fixed

• Fixed bugs when reading bad formed CSV files.
• Fixed bugs at statistical distributions.
• FloatRGB bug solved on equal (+=, -=, ...) operators. This bug affected
  ImageRGB operations such as resize.
• Solved problems when chaining methods in Lua, some objects end to be garbage
  collected.
• Improved support of strings in auto-completion (rlcompleter package).
• Solved bug at SparseMatrix<T> when reading it from a file.
• Solved bug in Image<T>::rotate90_cw methods.
• Solved bug in SparseMatrix::toDense() method.

C/C++

• Better LuaTable accessors, using [] operator.
• Implementation of matrix __index, __newindex and __call metamethods in
  C++.
• Implementation of matProd(), matCumSum() and matCumProd() functions.
• Implementation of matCountEq() and matCountNeq() functions for
  Matrix<T>.
• Updated matrix_ext_operations.h to change API of matrix operations. All
  functions have been overloaded to accept an in-place operation and another
  version which receives a destination matrix.
• Adding iterators to language models.
• Added MatrixScalarMap2 which receives as input2 a SparaseMatrix
  instance. This functions needs to be generalized to work with CPU and CUDA.
• The method SparseMatrix<T>::fromDenseMatrix() uses a DOKBuilder object
  to build the sparse matrix.
• The conversion of a Matrix<T> into a SparseMatrix<T> has been changed from
  a constructor overload to the static method
  SparseMatrix<T>::fromDenseMatrix().

Other

• Added support for IPyLua.
• Optimized matrix access for confusion matrix.
• Minor changes in class.lua.
• Improved binding to avoid multiple object copies when pushing C++ objects.
• Added Git commit hash and compilation time.

Release v0.4.0

Take into account that changes in the order of computations has introduced minor differences in optimizers of ANNs. In CNNs this difference is more important.

The most important changes are:

• Matrix classes doesn't have a major order flag. They are created always following row-major order, and it is possible to change this order by transposing dimensions of the original matrix. Math operations has been re-implemented asking for the order of the data, and working properly depending in that.
• Tokens are automatically unwrapped when they are pushed from C++ to Lua, and automatically wrapped when they are taken in C++ bindings. For this behavior, use the name AuxToken when using macros LUABIND_GET_* or LUABIND_RETURN.
• A LuaTable class has been added to allow a direct communication between C++ and Lua without replicating in C++ the data stored in Lua. It is useful for options tables, or to get dictionaries as weight matrix dictionaries of ANNs in trainable package.

Unstable changes

• Added matrix.op.repmat function.
• Added matrix.ext.iterate iterator.
• Added statistical distributions in stats.dist.
• Added matrix.ext.convolution and matrix.ext.real_fftwh.
• Added Matrix<T>::convolution method. It is in experimental stage, please,
  be advice to use them by your own risk.
• Because of the changes in Image, several classes in package imaging has
  been reviewed, consistent tests are necessary to merge with master branch.

API Changes

• Removed major order differentiation in matrix.
• tokens.matrix automatically wraps matrix instances, from Lua to C++.
• matrix automatically unwraps tokens.matrix instances, from C++ to Lua.
• Added new methods to AprilMath::Limits class.
• Added metrics.roc for ROC computation.
• Added new class behavior taken from
  Lua OOP-iter, in order to introduce
  more modularity in APRIL-ANN.
• New april_doc and april_set_doc API, works with values instead of strings.
• Added table.values, table.ivalues, table.keys and table.ikeys
  iterators.
• matrix.dict could store sparse and dense float matrices.
• Added matrix.cholesky(...) method.

Bugs removed

• Solved bug at matrix:max() and matrix:min() methods.
• Removed memory leak at SelectANNComponent::doBackprop() method.
• Solved bug at CopyANNComponent::doBackprop() method, incorrect behavior for
  multi-dimensional matrices.
• Solved bug at ZCAWhiteningANNComponent::doBackprop() method was wrong.
• Solved bug in stats.boot() function, it wasn't correctly updated to new
  class functions.
• Solve bug in trainable.supervised_trainer, problem with smooth_gradients
  flag.
• Solved bug in trainable.lua train_holdout:execute() method, the pocket
  algorithm wasn't work with negative loss functions.
• Solved bug at class.lua, inifite loop when calling class.is_a().
• Solved bug at autoencoder training.
• Solved bug in ROC computation.
• CUDA is working, time performance of convolutions needs a review.
• Solved bug in constString extract numeric methods which returns false when
  the extraction procedure ends up to the character after the last valid number.
• Solved bug at Matrix<T>::div and SparseMatrix<T>::div methods.

C/C++

• Added LuaTable class to allow access of Lua tables from C++ code.
• Added TAR support in C/C++, allowing to use streams as the standard I/O
  objects in APRIL-ANN.
• Added basics namespace which stores almost all C/C++ code in basics
  packages.
• Simplified read/write of matrices using the new StreamInterface objects.
• Added new StreamInterface for input/output operations, with specializations
  for C files, C strings, Lua strings, GZIP files, ZIP files. Other
  specializations are easier to implement in the future.
• Added statistical distribution classes.
• Removed C stack trace in ERROR_PRINT and ERROR_EXIT when compiling without
  debug.
• Image class code has been reviewed to be more coherent with current Matrix
  class implementation.

Added Travis CI support and other new features

Unstable changes

• Added clustering and knn algorithms.
• Added ann.components.zca_whitening and ann.components.pca_whitening
  classes.
• Added stats.zca_whitening and stats.pca_whitening functions.
• Added packages autodiff and autodiff.ann, with automatic differentiation
  algorithms.
• Added ann.optimizer.quickprop algorithm, but stills untested.
• Gradients smoothing, based on weights sharing and bunch_size, is applied by
  trainable.supervised_trainer objects, not by the optimizer.
• Added ann.components.dropout class.
• Added ann.autoencoders.ae class, in order to factorize the SDAE code.
• Solved problems with CUDA compilation, but it stills not working because
  an error during cuMemAlloc.
• Class interest_points and layout algorithms (still in development).
• Added the geometry c++ class for working in to lines and point geometry.

API Changes

• Modified bootstrapping API in order to be similar with R: stats.boot() and
  stats.boot.ci().
• Added ann.optimizer.asgd for Averaged SGD optimization.
• Added trainable.qlearning_trainer for reinforcement learning tasks.
• Enhanced util.serialize and util.deserialize to accept functions as
  argument, allowing to serialize/deserialize over general streams.
• Added iscallable(...) function.
• Added utest package, for unit testing.
• Added methods to matrix which work with sparse matrices.
• Added class matrix.sparse.
• Added method m:toTabStream(file or gzfile) to matrix.
• Added operations math.log1p and math.logadd.
• Added operator % to Lua string metatable, allowing to do Python-like
  string formatting, and map-like substitutions.
• Added index matrix for min/max operations in matrix objects.
• Added serialize and deserialize Lua functions, in util package.

Bugs removed

• Solved bug which makes to load unitialized weight matrices when loading old
  trainable.supervised_trainer (without matrix.dict object).
• Solved bug at Matrix<T>::sliding_window which makes an early end of the
  iterator when window has an offset different of 0.

C/C++

• Added -lcblas option to build_release.lua and build_debug.lua.
• Added sparse CBLAS wrappers, for CUDA (not tested) and standard release.
• Added CBLAS wrappers for float, double, and ComplexF types.
• Added geometric parametrizer for off-line text preprocessing.
• Added dependency with liblapacke for PCA when not available MKL or MacOS X.

Other

• Added argument check at mathcore.block.* constructors.
• Travis is monitoring only master and devel branches, for push and
  pull-requests.
• Added travis compilation and testing platform for pull-requests.

April-ANN v0.3.1-alpha (pre-release of v0.3.1-beta)

API Changes

• Added matrix:linspace(a,b) and matrix:linspace(a,b,base).
• matrix:transpose() returns a matrix which references the original. Any
  modification to the tranposed matrix, will be reflected at the original.
• matrix:cmul method is now in-place, so, if you don't want to modify the
  original matrix, you need to do m:clone():cmul(...) instead of
  m:cmul(...).
• update property is added to ann.optimizers.sgd object in order to compute
  the momentum.
• Optimizers execute method receives a functions which computes the loss, the
  gradient (mandatory). Any other data is optional.
• Removed dropout code from activation functions.
• Deleted options from ANN components.
• Added methods unroll and get to ann.components.stack class.
• Added inf and sup limits to Hard-Tanh activation function.
• Added random:to_lua_string() method.
• Moved ann.loss.__base__ to ann.loss.
• Moved ann.components.actf.__base__ to ann.components.actf.

New features

• Added matrix.dict, a hash set dictionary in C++ binded to Lua, which allows
  to execute basic math operations and reductions over the whole set of
  contained matrices. It major purpose is to represent a set of connection
  weights or gradients in ANNs.
• Added dataset.token.filter, which allows ANN components as filters.
• Added trainable.train_holdout_validation class, which replace
  trainable.supervised_trainer:train_holdout_validation method.
• Added trainable.train_wo_validation class, which replace
  trainable.supervised_trainer:train_wo_validation method.
• Added trainable.dataset_pair_iterator and
  trainable.dataset_multiple_iterator, useful to iterate over datasets
  following different traversal schemes: sequential, shuffled, shuffled with
  replacement, shuffled with distribution.
• Added method precompute_output_size in ANN components.
• Added ann.optimizer.cg, Conjugate Gradient algorithm.
• Added ann.optimizer.rprop, Resilient Prop algorithm.
• Added batch_fmeasure_micro_avg and batch_fmeasure_macro_avg for
  multi-class FMeasure computation.
• Renamed loss function local_fmeasure as batch_fmeasure, and improved to
  work with multi-class models.
• Added ann.loss.zero_one loss function.
• Added DEPENDENCIES-INSTALLER.sh.
• Added syntactic sugar for matrix:slice(...) method: m("1:2","3:4") or
  m({1,2},{3,4}), like in Matlab or Octave.
• Added matrix.svd and matrix.diagonalize.
• Added stats.pca, stats.mean_centered, stats.pca_whitening.

Bugs removed

• Memory leak due to the GPUMirroredMemoryBlock pool was solved.
• Solved bug at stats.correlation.pearson.
• Solved bug at trainable when using use_dataset, train_dataset,
  validate_dataset, grad_check_dataset methods without a bunch_size
  parameter, and with a trainer which has not a bunch_size defined at
  construction.
• Stabilization of log-logistic activation function.
• Stabilization of training with cross-entropy and multi-class-entropy.
• Solved bug when reading using matrix.fromTabFilename. The loader failed
  when the file had empty lines.
• Solved bug at Matrix<T>::select(...) C++ method. The matrix offset wasn't be
  added to the resulting matrix offset.
• Solved bug at SlidingWindow::setAtWindow(...) C++ method. The matrix offset
  wasn't be added to the computed window position.
• Solved bug at buffered_memory.h. Such bug introduces an early stop when
  reading matrices, ignoring the last lines of files.
• Solved problem with rlcompleter, which didn't work properly when loaded as a
  Lua module.
• Modified configure.sh to inform about any error during Lua building.
• Loadable modules are working on MacOs X.

C/C++ code changes

• Added Matrix<T>::random_acces_iterator, which reduces the access overhead
  for random access of a Matrix object. It retains the memory pointer forcing
  an update between host and device (GPU) memory.
• Generalized GPUMirroredBlockBase to allow the reinterpretation of the
  underlying memory pointer using different types (reinterpret_cast).
• Added MatrixSet class template, which stores a dictionary of STRING->MATRIX,
  useful for ANNs and gradient descent purposes.
• Added StochasticANNComponent which is base class for stochastic components.
• Simplified coupling between ANN components, ANN loss functions, introducing
  automatic binding between MatrixFloat and TokenMatrixFloat.
• ANN Components has a pointer to a MatrixFloat instead of ANN::Connections.
• ANN::Connections is a static class with helper functions, and it is binded
  as ann.connections.
• Old-weights property is removed from ANN connections.
• Added automatic conversion between DataSetFloat and DataSetToken in
  dataset.token.filter and dataset.token.union.
• Added FunctionInterface class, in Lua as functions class, superclass of
  ANN components.

April-ANN v0.3.0-beta: deploying April-ANN as a Lua 5.2 module

The more important changes are:

• April-ANN could be loaded as a Lua 5.2 module.
• The ANNs learning algorithm has been splitted in two parts:
  • The gradient computation, done in ANN components.
  • The optimization procedure implemented in specific Lua objects.
• Some bugs and memory leaks has been solved.

API Changes

• Added loss parameter to trainable.supervised_trainer methods.
• Added optimizer parameter to trainable.supervised_trainer methods.
• Added ann.optimizer package, which has the implementation of weight update
  based on weight gradient. So, the ANN components only compute gradients.
  This allows to implement different optimization methods (as "Conjugate
  Gradient", or "Linear Search Brack-Propagation") with the same gradient
  computation.
• Loss functions ann.loss API has been changed, now the loss computation is
  done in two separated steps:
  - matrix = loss:compute_loss(output,target): which returns a matrix with
  the loss of every pair of patterns, allowing to perform several loss
  computations without taken them into account.
  - matrix = loss:accum_loss(matrix): which accumulates the loss in the
  internal state of the class.
• Added a new version of loss function, which computes mean and
  sample variance of the loss. Besides, the loss computation is done
  with doubles, being more accurated than before.
• replacement parameter in SDAE doesn't force on_the_fly parameter, they are
  independent.
• SDAE training has been changed in order to allow the use of LUA datasets,
• Replaced cpp_class_binding_extension by class_extension function,
  adding Lua classes support besides to CPP binded classes.
• Modified class and class_instance functions to be more homogeneous
  with C++ binding.
• Added support for GZipped matrices load and save from C++, so functions
  matrix.savefile and matrix.loadfile (and its correspondence for complex
  numbers, double, int32, and char) were removed. Methods matrix.fromFilename
  and matrix.toFilename accept '.gz' extension.

Packages rename

• Changed package sdae to ann.autoencoders.
• Changed package loss_functions to ann.loss.
• Splitted mathcore package into mathcore and complex packages.
• Renamed math package to mathcore to avoid the collision with Lua standard
  math library.

New features

• April-ANN is deployed as a standalone executable and as a shared library for
  Lua 5.2.
• Modified lua.h to incorporate the GIT commit number in the disclaimer.
• Added Lua autocompletion when readline is available.
• Implemented SignalHandler class in C++.
• Added signal.register and signal.receive functions to Lua.
• Added to matrix the methods map, contiguous, join, abs, tan,
  atan, atanh, sinh, asin, asinh, cosh, acos, acosh, fromMMap,
  toMMap, div, max, min.
• Added iterator class, which is a wrapper around Lua iterators, but
  provides a more natural interface with functional programming procedures
  as map, filter, apply, or reduce.
• Added methods iterate, field, select to iterator Lua class.
• table.insert returns the table, which is useful for reduction operations.
• Added table method to iterator class.
• Added naive L1_norm regularization.
• Added dataset.clamp.
• Added mathcore.set_mmap_allocation function, which allows to forces the
  use of mmap for matrix memory allocation.
• Added ann.components.slice.
• Added GS-PCA algorithm for efficient computation of PCA (iterative algorithm),
  stats.iterative_pca Lua function.
• Added basic MapReduce implementation in Lua.
• Added stats.correlation.pearson Lua class.
• Added stats.bootstrap_resampling function.
• Added math.add, math.sub, math.mul, math.div functions.
• trainable and ann.mlp.all_all are using matrix:to_lua_string()
  method.
• Added method to_lua_string() in all matrix types, so the method produce
  a Lua chunk which is loadable and produce a matrix.
• Added serialization to parallel_foreach, allowing to produce outputs which
  could be loaded by the caller process.
• Declaration of luatype function as global, it wasn't.
• Added iterable_map and multiple_ipairs functions to the Lua utilities.
• Added SubAndDivNormalizationDataSet, applies a substraction and a division of
  the feature vectors.
• Added stepDataset.

Bugs removed

• Solved bug at luabind_template.cc, which introduced spurious segmentation
  faults due to Lua execution of garbage collection in the middle of a
  lua_pushClassName.
• Solved bug at glob function.
• Solved bug at matrix iterators operator=.
• Solved bug at method matrix::best_span_iterator::setAtWindow. Becaose of It
  the method didn't works when the matrix was a sub-matrix (slice) of other
  matrix.
• Solved bugs at Matrix template constructor which affects to rewrap lua
  method, and to select method, which affects to select lua method.
• Added binarizer::init() to a binded static_constructor, it is needed to
  execute init() before decode/encode double numbers, because of endianism.
• Solved bug at constString when extracting double numbers in binary format.
• MacOS compilation problems solved.
• Solved problems with CUDA, it is working again.
• Dynamic loading of C modules is working now.

C/C++ code changes

• Added BIND_STRING_CONSTANT to luabind, so it is possible to export C string
  constants to Lua.
• Removed warning of clang about unused variables, adding a new macro
  UNUSED_VARIABLE(x) defined in the header utils/c_src/unused_variable.h.
• Matrix fromString and toString Lua methods have been improved to write/read
  directly from Lua string buffer, so the memory print has been reduced.
• The C++ routines to write and read files is generalized to work with streams,
  under the BufferedStream template, and it is instantiated to FILE and gzFile
  formats.
• Added sanity check to cross-entropy and multi-class cross-entropy loss
  functions, to detect the use of non logarithmic outputs.

Updated with lua-5.2.2

This release compiles with Lua 5.2.2, instead of 5.1.4. It the following new features:

• matrices with float (matrix), complex numbers (matrixComplex), double (matrixDouble), and int32 (matrixInt32).
• GZIO is fresh code, a wrapper over ZLIB, binded using LUABIND.
• Added LAPACK support, for matrix inversion computation.
• Updated Lua class mean_var to follow the method of Knuth.
• CUDA compiles, but stills not working :SSSSSS
• Solved problems of table.tostring.
• Generalization of matrix math operators.
• Specialization of math wrappers for float and complex numbers.
• Support for MAT-file format (Matlab matrices).
• Added mutual_information package.
• Added APRIL_EXEC and APRIL_TOOLS_DIR environment variables to configure.sh
• Added scripts for NBEST, but not working :SSSSSS
• Some memory leaks, and bugs are solved.