update jsonlab towards v1.9 release

Author: Qianqian Fang

ChangeLog.txt

@@ -6,6 +6,39 @@
 
 JSONlab ChangeLog (key features marked by *):
 
+== JSONlab 1.9 (codename: Magnus - alpha), FangQ <q.fang <at> neu.edu> ==
+
+ 2019-05-04 [2e317c9] explain extra compression fields
+ 2019-05-02 [1b1be65] avoid side effect of removing singletarray
+ 2019-05-02*[8360fd1] support zmat based base64 encoding and decoding
+ 2019-05-01*[c797bb2] integrating zmat, for zlib/gzip data compression
+ 2019-04-29 [70551fe] remove warnings from matlab
+ 2019-04-28 [0d61c4b] complete data compression support, close #52
+ 2019-04-27 [804115b] avoid typecast error
+ 2019-04-27 [c166aa7] change default compressarraysize to 100
+ 2019-04-27*[3322f6f] major new feature: support array compression and decompression
+ 2019-03-13*[9c01046] support saving function handles, close #51
+ 2019-03-13 [a8fde38] add option to parse string array or convert to char, close #50
+ 2019-03-12 [ed2645e] treat string array as cell array in newer matlab
+ 2018-11-18 [c3eb021] allow saving uint64 integers in saveubjson, fix #49
+
+== JSONlab 1.8 (codename: Nominus), FangQ <q.fang <at> neu.edu> ==
+
+ 2018-07-12 [03a6c25] update documentation, bump version to 1.8, tag Nominus
+ 2018-07-12*[1597106] add patch provided by pjkoprowski to support MATLAB table, add RowNames support, fix #29
+ 2018-07-12 [f16cc57] fix #31, throw an error when : array construct is used
+ 2018-07-12 [956e000] drop octave 3.x support, fix ubjson error in octave
+ 2018-07-12 [e090f0a] fix octave warning for saveubjson
+ 2018-07-12*[34284c7] fix issues #34 #39 #44 and #45, support double-quoted strings
+ 2017-09-06 [474d8c8] Merge pull request #41 from dasantonym/master
+ 2017-08-07*[38b24fb] added package.json to be able to intall via npm package manager, converted readme to utf-8, added basic .gitignore file
+ 2017-07-19 [ae7a5d9] Merge pull request #40 from astorfi/master
+ 2017-07-17 [154ef61] Rename README.txt to README.rst
+ 2017-03-27 [31b5bdc] simplify condition flow in matching_bracket
+ 2017-03-27 [86ef12a] avoid error in matlab 2017a, close #34
+ 2017-02-18 [4a09ac3] Merge pull request #32 from vrichter/master
+ 2017-02-14 [e67d3a3] respect integer types
+
 == JSONlab 1.5 (codename: Nominus - alpha), FangQ <q.fang <at> neu.edu> ==
 
  2017/01/02 *use Big-endian format to store floating points (d/D) in saveubjson (Issue #25)
@@ -18,7 +51,7 @@ JSONlab ChangeLog (key features marked by *):
 
 == JSONlab 1.2 (codename: Optimus - Update 2), FangQ <q.fang <at> neu.edu> ==
 
- 2015/12/16  replacing string concatenation by str cells to gain 2x speed in savejson (Issue#17)
+ 2015/12/16 *replacing string concatenation by str cells to gain 2x speed in savejson (Issue#17)
  2015/12/11  fix FileName option case bug (SVN rev#495)
  2015/12/11  add SingletCell option, add SingletArray to replace NoRowBracket (Issue#15,#8)
  2015/11/10  fix bug for inerpreting file names as JSON string - by Mykhailo Bratukha (Pull#14)

LICENSE_BSD.txt

@@ -1,4 +1,4 @@
-Copyright 2011-2018 Qianqian Fang <q.fang <at> neu.edu>. All rights reserved.
+Copyright 2011-2019 Qianqian Fang <q.fang <at> neu.edu>. All rights reserved.
 
 Redistribution and use in source and binary forms, with or without modification, are
 permitted provided that the following conditions are met:

LICENSE_GPLv3.txt

@@ -3,7 +3,7 @@
 =           An open-source MATLAB/Octave JSON encoder and decoder             =
 ===============================================================================
 
-Copyright (C) 2011-2018  Qianqian Fang <q.fang <at> neu.edu>
+Copyright (C) 2011-2019  Qianqian Fang <q.fang <at> neu.edu>
 
 -------------------------------------------------------------------------------
 

README.rst

@@ -3,9 +3,9 @@
 =           An open-source MATLAB/Octave JSON encoder and decoder             =
 ===============================================================================
 
-*Copyright (C) 2011-2018  Qianqian Fang <q.fang at neu.edu>
+*Copyright (C) 2011-2019  Qianqian Fang <q.fang at neu.edu>
 *License: BSD or GNU General Public License version 3 (GPL v3), see License*.txt
-*Version: 1.8 (Nominus)
+*Version: 1.9 (Magnus - alpha)
 
 -------------------------------------------------------------------------------
 
@@ -21,6 +21,13 @@ V.  Contribution and feedback
 
 I.  Introduction
 
+JSONLab is a free and open-source implementation of a JSON/UBJSON encoder 
+and a decoder in the native MATLAB language. It can be used to convert a MATLAB 
+data structure (array, struct, cell, struct array and cell array) into 
+JSON/UBJSON formatted strings, or to decode a JSON/UBJSON file into MATLAB 
+data structure. JSONLab supports both MATLAB and  
+[http://www.gnu.org/software/octave/ GNU Octave] (a free MATLAB clone).
+
 JSON ([http://www.json.org/ JavaScript Object Notation]) is a highly portable, 
 human-readable and "[http://en.wikipedia.org/wiki/JSON fat-free]" text format 
 to represent complex and hierarchical data. It is as powerful as 
@@ -29,7 +36,7 @@ used for data-exchange in applications, and is essential for the wild success
 of [http://en.wikipedia.org/wiki/Ajax_(programming) Ajax] and 
 [http://en.wikipedia.org/wiki/Web_2.0 Web2.0]. 
 
-UBJSON (Universal Binary JSON) is a binary JSON format, specifically 
+UBJSON ([<http://ubjson.org/ Universal Binary JSON]) is a binary JSON format, specifically 
 optimized for compact file size and better performance while keeping
 the semantics as simple as the text-based JSON format. Using the UBJSON
 format allows to wrap complex binary data in a flexible and extensible
@@ -43,13 +50,6 @@ general-purpose file specifications, such as
 [http://www.hdfgroup.org/HDF5/whatishdf5.html HDF5], with significantly 
 reduced complexity and enhanced performance.
 
-JSONLab is a free and open-source implementation of a JSON/UBJSON encoder 
-and a decoder in the native MATLAB language. It can be used to convert a MATLAB 
-data structure (array, struct, cell, struct array and cell array) into 
-JSON/UBJSON formatted strings, or to decode a JSON/UBJSON file into MATLAB 
-data structure. JSONLab supports both MATLAB and  
-[http://www.gnu.org/software/octave/ GNU Octave] (a free MATLAB clone).
-
 -------------------------------------------------------------------------------
 
 II. Installation
@@ -62,10 +62,20 @@ by using the following command:
     addpath('/path/to/jsonlab');
 
 If you want to add this path permanently, you need to type "pathtool", 
-browse to the jsonlab root folder and add to the list, then click "Save".
-Then, run "rehash" in MATLAB, and type "which loadjson", if you see an 
+browse to the zmat root folder and add to the list, then click "Save".
+Then, run "rehash" in MATLAB, and type "which savejson", if you see an 
 output, that means JSONLab is installed for MATLAB/Octave.
 
+If you use MATLAB in a shared environment such as a Linux server, the
+best way to add path is to type 
+
+   mkdir ~/matlab/
+   nano ~/matlab/startup.m
+
+and type addpath('/path/to/jsonlab') in this file, save and quit the editor.
+MATLAB will execute this file every time it starts. For Octave, the file
+you need to edit is ~/.octaverc , where "~" is your home directory.
+
 -------------------------------------------------------------------------------
 
 III.Using JSONLab
@@ -122,6 +132,7 @@ JSON. The detailed help info for the four functions can be found below:
                           array of 1D vectors; setting to 4 will return a
                           3D cell array.
             opt.ShowProgress [0|1]: if set to 1, loadjson displays a progress bar.
+            opt.ParseStringArray [0|1]: if set to 1, loadjson displays a progress bar.
 
   output:
        dat: a cell array, where {...} blocks are converted into cell arrays,
@@ -134,7 +145,7 @@ JSON. The detailed help info for the four functions can be found below:
 
   license:
       BSD or GPL version 3, see LICENSE_{BSD,GPLv3}.txt files for details 
- </pre>
+  </pre>
 
 === savejson.m ===
 
@@ -206,7 +217,22 @@ JSON. The detailed help info for the four functions can be found below:
                           back to the string form
          opt.SaveBinary [0|1]: 1 - save the JSON file in binary mode; 0 - text mode.
          opt.Compact [0|1]: 1- out compact JSON format (remove all newlines and tabs)
- 
+         opt.Compression  'zlib' or 'gzip': specify array compression
+                          method; currently only supports 'gzip' or 'zlib'. The
+                          data compression only applicable to numerical arrays 
+                          in 3D or higher dimensions, or when ArrayToStruct
+                          is 1 for 1D or 2D arrays. If one wants to
+                          compress a long string, one must convert
+                          it to uint8 or int8 array first. The compressed
+                          array uses three extra fields
+                          "_ArrayCompressionMethod_": the opt.Compression value. 
+                          "_ArrayCompressionSize_": a 1D interger array to
+                             store the pre-compressed (but post-processed)
+                             array dimensions, and 
+                          "_ArrayCompressedData_": the "base64" encoded
+                              compressed binary array data. 
+         opt.CompressArraySize [100|int]: only to compress an array if the total 
+                          element count is larger than this number.
          opt can be replaced by a list of ('param',value) pairs. The param 
          string is equivallent to a field in opt and is case sensitive.
   output:
@@ -334,6 +360,23 @@ JSON. The detailed help info for the four functions can be found below:
                           wrapped inside a function call as 'foo(...);'
          opt.UnpackHex [1|0]: conver the 0x[hex code] output by loadjson 
                           back to the string form
+         opt.Compression  'zlib' or 'gzip': specify array compression
+                          method; currently only supports 'gzip' or 'zlib'. The
+                          data compression only applicable to numerical arrays 
+                          in 3D or higher dimensions, or when ArrayToStruct
+                          is 1 for 1D or 2D arrays. If one wants to
+                          compress a long string, one must convert
+                          it to uint8 or int8 array first. The compressed
+                          array uses three extra fields
+                          "_ArrayCompressionMethod_": the opt.Compression value. 
+                          "_ArrayCompressionSize_": a 1D interger array to
+                             store the pre-compressed (but post-processed)
+                             array dimensions, and 
+                          "_ArrayCompressedData_": the binary stream of
+                             the compressed binary array data WITHOUT
+                             'base64' encoding
+         opt.CompressArraySize [100|int]: only to compress an array if the total 
+                          element count is larger than this number.
 
          opt can be replaced by a list of ('param',value) pairs. The param 
          string is equivallent to a field in opt and is case sensitive.
@@ -399,33 +442,31 @@ that everyone else can enjoy the improvement. For anyone who want to contribute,
 please download JSONLab source code from its source code repositories by using the
 following command:
 
- git clone https://github.com/fangq/jsonlab.git jsonlab
+      git clone https://github.com/fangq/jsonlab.git jsonlab
 
 or browsing the github site at
 
- https://github.com/fangq/jsonlab
-
-alternatively, if you prefer svn, you can checkout the latest code by using
-
- svn checkout svn://svn.code.sf.net/p/iso2mesh/code/trunk/jsonlab jsonlab
-
-You can make changes to the files as needed. Once you are satisfied with your
-changes, and ready to share it with others, please cd the root directory of 
-JSONLab, and type
-
- git diff --no-prefix > yourname_featurename.patch
-
-or
+      https://github.com/fangq/jsonlab
 
- svn diff > yourname_featurename.patch
+Sometimes, you may find it is necessary to modify JSONLab to achieve your 
+goals, or attempt to modify JSONLab functions to fix a bug that you have 
+encountered. If you are happy with your changes and willing to share those
+changes to the upstream author, you are recommended to create a pull-request
+on github. 
 
-You then email the .patch file to JSONLab's maintainer, Qianqian Fang, at
-the email address shown in the beginning of this file. Qianqian will review 
-the changes and commit it to the subversion if they are satisfactory.
+To create a pull-request, you first need to "fork" jsonlab on Github by 
+clicking on the "fork" button on top-right of jsonlab's github page. Once you forked
+jsonlab to your own directory, you should then implement the changes in your
+own fork. After thoroughly testing it and you are confident the modification 
+is complete and effective, you can then click on the "New pull request" 
+button, and on the left, select fangq/jsonlab as the "base". Then type
+in the description of the changes. You are responsible to format the code
+updates using the same convention (tab-width: 8, indentation: 4 spaces) as
+the upstream code.
 
 We appreciate any suggestions and feedbacks from you. Please use the following
 mailing list to report any questions you may have regarding JSONLab:
 
-https://groups.google.com/forum/?hl=en#!forum/jsonlab-users
+https://groups.google.com/forum/?hl=en#!forum/iso2mesh-users
 
 (Subscription to the mailing list is needed in order to post messages).

README.txt

@@ -345,6 +345,7 @@ function parse_char(inStr, c)
                 pos = pos + 1;
         end
     end
+    str=unescapejsonstring(str);
     error_pos('End of file while expecting end of inStr',inStr);
 
 %%-------------------------------------------------------------------------
@@ -506,3 +507,11 @@ function error_pos(msg, inStr)
 if(endpos==0) 
     error('unmatched "]"');
 end
+
+function newstr=unescapejsonstring(str)
+newstr=str;
+escapechars={'\\','\"','\/','\a','\b','\f','\n','\r','\t','\v'};
+for i=1:length(escapechars);
+    newstr=regexprep(newstr,regexprep(escapechars{i},'\\','\\\\'), escapechars{i});
+end
+newstr=regexprep(newstr,'\\\\(u[0-9a-fA-F]{4}[^0-9a-fA-F]*)','\\$1');

loadjson.m

@@ -1,6 +1,6 @@
 {
   "name": "jsonlab",
-  "version": "1.5.0",
+  "version": "1.9",
   "description": "An open-source MATLAB/Octave JSON encoder and decoder",
   "directories": {
     "example": "examples"
@@ -17,5 +17,5 @@
   "bugs": {
     "url": "https://github.com/fangq/jsonlab/issues"
   },
-  "homepage": "https://github.com/fangq/jsonlab#readme"
+  "homepage": "https://iso2mesh.sf.net/jsonlab/"
 }

package.json

@@ -68,15 +68,19 @@
 %        opt.SaveBinary [0|1]: 1 - save the JSON file in binary mode; 0 - text mode.
 %        opt.Compact [0|1]: 1- out compact JSON format (remove all newlines and tabs)
 %        opt.Compression  'zlib' or 'gzip': specify array compression
-%                         method; currently only support 'gzip' or 'zlib'. The
-%                         data compression only applies to numerical arrays when 
-%                         ArrayToStruct is 1. The compressed array use two extra
-%                         fields "_ArrayCompressionSize_" and "_ArrayCompressedData_"
-%                         where the former is a 1D interger array to store the 
-%                         pre-compressed (but post-processed) array dimensions, and
-%                         the latter stores the "base64" encoded compressed binary 
-%                         array data. In addition, an extra field 
-%                         "_ArrayCompressionMethod_" stores the opt.Compression value.
+%                         method; currently only supports 'gzip' or 'zlib'. The
+%                         data compression only applicable to numerical arrays 
+%                         in 3D or higher dimensions, or when ArrayToStruct
+%                         is 1 for 1D or 2D arrays. If one wants to
+%                         compress a long string, one must convert
+%                         it to uint8 or int8 array first. The compressed
+%                         array uses three extra fields
+%                         "_ArrayCompressionMethod_": the opt.Compression value. 
+%                         "_ArrayCompressionSize_": a 1D interger array to
+%                            store the pre-compressed (but post-processed)
+%                            array dimensions, and 
+%                         "_ArrayCompressedData_": the "base64" encoded
+%                             compressed binary array data. 
 %        opt.CompressArraySize [100|int]: only to compress an array if the total 
 %                         element count is larger than this number.
 %        opt can be replaced by a list of ('param',value) pairs. The param 
@@ -118,20 +122,22 @@
 opt.IsOctave=exist('OCTAVE_VERSION','builtin');
 
 dozip=jsonopt('Compression','',opt);
-if(~opt.IsOctave && ~isempty(dozip))
+if(~isempty(dozip))
     if(~(strcmpi(dozip,'gzip') || strcmpi(dozip,'zlib')))
         error('compression method "%s" is not supported',dozip);
     end
-    try
-        error(javachk('jvm'));
+    if(exist('zmat')~=3)
         try
-           base64decode('test');
+            error(javachk('jvm'));
+            try
+               base64decode('test');
+            catch
+               matlab.net.base64decode('test');
+            end
         catch
-           matlab.net.base64decode('test');
+            error('java-based compression is not supported');
         end
-    catch
-        error('java-based compression is not supported');
-    end
+    end    
     opt.Compression=dozip;
 end
 
@@ -202,7 +208,7 @@
 elseif(ischar(item))
     txt=str2json(name,item,level,varargin{:});
 elseif(isobject(item))
-    if(~exist('OCTAVE_VERSION','builtin') && istable(item))
+    if(~exist('OCTAVE_VERSION','builtin') && exist('istable','builtin') && istable(item))
         txt=matlabtable2json(name,item,level,varargin{:});
     else
         txt=matlabobject2json(name,item,level,varargin{:});
@@ -403,10 +409,6 @@
 dozip=jsonopt('Compression','',varargin{:});
 zipsize=jsonopt('CompressArraySize',100,varargin{:});
 
-if(islogical(item))
-    item=uint8(item);
-end
-
 if(length(size(item))>2 || issparse(item) || ~isreal(item) || ...
    (isempty(item) && any(size(item))) ||jsonopt('ArrayToStruct',0,varargin{:}) || (~isempty(dozip) && numel(item)>zipsize))
     if(isempty(name))
@@ -487,6 +489,9 @@
     if(~isempty(dozip) && numel(item)>zipsize)
         if(isreal(item))
             fulldata=item(:)';
+            if(islogical(fulldata))
+                fulldata=uint8(fulldata);
+            end
         else
             txt=sprintf(dataformat,txt,padding0,'"_ArrayIsComplex_": ','1', sep);
             fulldata=[real(item(:)) imag(item(:))];