Multiple values per event in uproot4

[XDongiang]

I meet same problem fixed in uproot3, but i meet it again in uproot4!

[array([[ 0.7882904 ,  0.61012289,  0.04211533,  0.06050724],
        [ 0.55939287,  0.13029095,  0.16960895,  0.15316812],
        [ 1.37675754, -0.7024612 , -1.02975954, -0.31287042],
        [ 0.96134215, -0.03795264,  0.81803526,  0.09919506]])
 array([[ 0.60124435, -0.30947612, -0.14544959, -0.02909869],
        [ 1.1608169 , -0.63750589,  0.75457903, -0.35773679],
        [ 0.83054014, -0.02685332, -0.56037676,  0.3624041 ],
        [ 1.09318157,  0.97383533, -0.04875268,  0.02443138]])
 array([[ 0.5964254 ,  0.29379539, -0.04998751,  0.15228852],
        [ 1.24460004,  0.74962303,  0.45007822,  0.73539593],
        [ 0.90096949, -0.48115726, -0.48470123, -0.31871845],
        [ 0.94378803, -0.56226116,  0.08461052, -0.568966  ]]) ...
 array([[ 0.65629214,  0.10312235, -0.39342784,  0.14691054],
        [ 1.18158272, -0.29126532, -1.0321247 ,  0.04799862],
        [ 1.23043986, -0.01224176,  1.11501659, -0.16387006],
        [ 0.61746824,  0.20038474,  0.31053595, -0.0310391 ]])
 array([[ 0.55299992,  0.16850911,  0.10276983, -0.15210185],
        [ 0.68238664,  0.39755071, -0.03958946, -0.24964106],
        [ 1.24395833, -0.67337457,  0.91814977, -0.08534078],
        [ 1.20643806,  0.10731475, -0.98133014,  0.48708369]])
 array([[ 0.79923378,  0.01862661, -0.45713698, -0.43097156],
        [ 1.02001038,  0.25929245, -0.58030893, -0.6266683 ],
        [ 0.65326153, -0.07903001,  0.37375   ,  0.19261061],
        [ 1.21327726, -0.19888905,  0.66369591,  0.86502926]])]

https://github.com/scikit-hep/uproot3#multiple-values-per-event-fixed-size-arrays

tree = uproot3.open("http://scikit-hep.org/uproot3/examples/nesteddirs.root")["one/two/tree"]
array = tree.array("ArrayInt64", entrystop=20)
array
# array([[ 0,  0,  0,  0,  0,  0,  0,  0,  0,  0],
#        [ 1,  1,  1,  1,  1,  1,  1,  1,  1,  1],
#        [ 2,  2,  2,  2,  2,  2,  2,  2,  2,  2],
#        [ 3,  3,  3,  3,  3,  3,  3,  3,  3,  3],
#        [ 4,  4,  4,  4,  4,  4,  4,  4,  4,  4],
#        [ 5,  5,  5,  5,  5,  5,  5,  5,  5,  5],
#        [ 6,  6,  6,  6,  6,  6,  6,  6,  6,  6],
#        [ 7,  7,  7,  7,  7,  7,  7,  7,  7,  7],
#        [ 8,  8,  8,  8,  8,  8,  8,  8,  8,  8],
#        [ 9,  9,  9,  9,  9,  9,  9,  9,  9,  9],
#        [10, 10, 10, 10, 10, 10, 10, 10, 10, 10],
#        [11, 11, 11, 11, 11, 11, 11, 11, 11, 11],
#        [12, 12, 12, 12, 12, 12, 12, 12, 12, 12],
#        [13, 13, 13, 13, 13, 13, 13, 13, 13, 13],
#        [14, 14, 14, 14, 14, 14, 14, 14, 14, 14],
#        [15, 15, 15, 15, 15, 15, 15, 15, 15, 15],
#        [16, 16, 16, 16, 16, 16, 16, 16, 16, 16],
#        [17, 17, 17, 17, 17, 17, 17, 17, 17, 17],
#        [18, 18, 18, 18, 18, 18, 18, 18, 18, 18],
#        [19, 19, 19, 19, 19, 19, 19, 19, 19, 19]])

[XDongiang]

I did not get the correct shape using uproot3, so I very much doubt that your program is correct.Doing rigorous scientific research requires stable tools, not error-prone toys.

[jpivarski]

Both Uproot 3 and Uproot 4 return an array of the correct shape for this file/tree/branch. In Uproot 4,

>>> import uproot
>>> tree = uproot.open("http://scikit-hep.org/uproot3/examples/nesteddirs.root")["one/two/tree"]
>>> tree["ArrayInt64"].array()
<Array [[0, 0, 0, 0, 0, ... 99, 99, 99, 99]] type='100 * 10 * int64'>

or as a NumPy array,

>>> tree["ArrayInt64"].array(library="np")
array([[ 0,  0,  0,  0,  0,  0,  0,  0,  0,  0],
       [ 1,  1,  1,  1,  1,  1,  1,  1,  1,  1],
       [ 2,  2,  2,  2,  2,  2,  2,  2,  2,  2],
       ...
       [97, 97, 97, 97, 97, 97, 97, 97, 97, 97],
       [98, 98, 98, 98, 98, 98, 98, 98, 98, 98],
       [99, 99, 99, 99, 99, 99, 99, 99, 99, 99]])

which has shape (100, 10).

Doing the same thing in ROOT,

>>> import ROOT
>>> file = ROOT.TFile("nesteddirs.root")
TClass::Init:0: RuntimeWarning: no dictionary for class Event is available
TClass::Init:0: RuntimeWarning: no dictionary for class P3 is available
>>> tree = file.Get("one/two/tree")
>>> for x in tree:
...     print(list(x.ArrayInt64))
... 
[0, 0, 0, 0, 0, 0, 0, 0, 0, 0]
[1, 1, 1, 1, 1, 1, 1, 1, 1, 1]
[2, 2, 2, 2, 2, 2, 2, 2, 2, 2]
...
[97, 97, 97, 97, 97, 97, 97, 97, 97, 97]
[98, 98, 98, 98, 98, 98, 98, 98, 98, 98]
[99, 99, 99, 99, 99, 99, 99, 99, 99, 99]

Doing rigorous scientific research requires stable tools, not error-prone toys.

Really? If this had been a bug, I would have fixed it right away, added a test, and released a new version. The response is usually same-day. That way, the distribution of testing and maintenance conforms to the use-cases: the bugs people report get fixed and the tests ensure that they never regress—progress is always forward, though it is iterative. If we tried to guarantee perfection before each release, we would first have to define a domain of use-cases to test, but the users have a much better idea about what their own problems are than we do. This is, loosely speaking, the "agile" method, to be contrasted with "big design up-front," which has been disfavored for the past two decades because it never was very effective. (It's a guiding principle of the ROOT project as well: see the intro to the ROOT User's Guide. That intro was added in August 2000.)

But, there was no bug. The above is the correct behavior.

[XDongiang]

Have you really tested your code strictly? LOL

[XDongiang]

uproot3 also has wrong shape

[jpivarski]

Without access to PhiKK_DATA.root, I have no context to know that what you're showing me is wrong. In the case of nesteddirs.root, the shape (100, 10) is correct.

Although Uproot 3 and 4 are rather different in their presentation of the data, they do look consistent with each other here. This seems to be some TBranch with a variable-length type (in principle, the array in an event can have any length) with constant length for all the entries shown (in practice, the first and last arrays happen to have the same lengths). That kind of data can be most efficiently turned into a 2-dimensional NumPy array by reading it into an Awkward Array (library="ak", the default) and then passing that through ak.to_numpy (faster than NumPy dtype="O", because that dtype is just a collection of Python references; Awkward preserves the contiguousness of the numerical data).

On the other hand, I don't even know what aspect of what you're showing me is not what you want. I'm just guessing.

[XDongiang]

Thank you for your answer, What you answered is exactly what i need.
Because of the previous question, I don’t have much confidence in this program