Skip to content

Efficiency monitoring fails (9c324a1) #1

@vkhodygo

Description

@vkhodygo

I tried to follow the guideline and run the most recent version of the tool at the moment. It fails and throws an error:

$ python ./check_efficiency.py s.user.name 7
Command:
sacct -u s.user.name --start 2021-08-03 --end 2021-08-10 --parsable2 --format Account,AdminComment,AllocCPUS,AllocGRES,AllocNodes,AllocTRES,AssocID,AveCPU,AveCPUFreq,AveDiskRead,AveDiskWrite,AvePages,AveRSS,AveVMSize,BlockID,Cluster,Comment,Constraints,ConsumedEnergy,ConsumedEnergyRaw,CPUTime,CPUTimeRAW,DerivedExitCode,Elapsed,ElapsedRaw,Eligible,End,ExitCode,Flags,GID,Group,JobID,JobIDRaw,JobName,Layout,MaxDiskRead,MaxDiskReadNode,MaxDiskReadTask,MaxDiskWrite,MaxDiskWriteNode,MaxDiskWriteTask,MaxPages,MaxPagesNode,MaxPagesTask,MaxRSS,MaxRSSNode,MaxRSSTask,MaxVMSize,MaxVMSizeNode,MaxVMSizeTask,McsLabel,MinCPU,MinCPUNode,MinCPUTask,NCPUS,NNodes,NodeList,NTasks,Priority,Partition,QOS,QOSRAW,Reason,ReqCPUFreq,ReqCPUFreqMin,ReqCPUFreqMax,ReqCPUFreqGov,ReqCPUS,ReqGRES,ReqMem,ReqNodes,ReqTRES,Reservation,ReservationId,Reserved,ResvCPU,ResvCPURAW,Start,State,Submit,Suspended,SystemCPU,SystemComment,Timelimit,TimelimitRaw,TotalCPU,TRESUsageInAve,TRESUsageInMax,TRESUsageInMaxNode,TRESUsageInMaxTask,TRESUsageInMin,TRESUsageInMinNode,TRESUsageInMinTask,TRESUsageInTot,TRESUsageOutAve,TRESUsageOutMax,TRESUsageOutMaxNode,TRESUsageOutMaxTask,TRESUsageOutMin,TRESUsageOutMinNode,TRESUsageOutMinTask,TRESUsageOutTot,UID,User,UserCPU,WCKey,WCKeyID,WorkDir
Error stream:

Traceback (most recent call last):
  File "/apps/local/languages/anaconda/2020.07/lib/python3.8/site-packages/pandas/core/indexes/base.py", line 2646, in get_loc
    return self._engine.get_loc(key)
  File "pandas/_libs/index.pyx", line 111, in pandas._libs.index.IndexEngine.get_loc
  File "pandas/_libs/index.pyx", line 138, in pandas._libs.index.IndexEngine.get_loc
  File "pandas/_libs/hashtable_class_helper.pxi", line 1619, in pandas._libs.hashtable.PyObjectHashTable.get_item
  File "pandas/_libs/hashtable_class_helper.pxi", line 1627, in pandas._libs.hashtable.PyObjectHashTable.get_item
KeyError: ''

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "./check_efficiency.py", line 167, in <module>
    print("Efficiency:", compute_global_efficiency(df))
  File "./check_efficiency.py", line 119, in compute_global_efficiency
    cpu_time_actively_used = df.TotalCPU.loc[(slice(None), '')].sum()
  File "/apps/local/languages/anaconda/2020.07/lib/python3.8/site-packages/pandas/core/indexing.py", line 1762, in __getitem__
    return self._getitem_tuple(key)
  File "/apps/local/languages/anaconda/2020.07/lib/python3.8/site-packages/pandas/core/indexing.py", line 1272, in _getitem_tuple
    return self._getitem_lowerdim(tup)
  File "/apps/local/languages/anaconda/2020.07/lib/python3.8/site-packages/pandas/core/indexing.py", line 1373, in _getitem_lowerdim
    return self._getitem_nested_tuple(tup)
  File "/apps/local/languages/anaconda/2020.07/lib/python3.8/site-packages/pandas/core/indexing.py", line 1433, in _getitem_nested_tuple
    result = self._handle_lowerdim_multi_index_axis0(tup)
  File "/apps/local/languages/anaconda/2020.07/lib/python3.8/site-packages/pandas/core/indexing.py", line 1360, in _handle_lowerdim_multi_index_axis0
    raise ek
  File "/apps/local/languages/anaconda/2020.07/lib/python3.8/site-packages/pandas/core/indexing.py", line 1352, in _handle_lowerdim_multi_index_axis0
    return self._get_label(tup, axis=axis)
  File "/apps/local/languages/anaconda/2020.07/lib/python3.8/site-packages/pandas/core/indexing.py", line 621, in _get_label
    return self.obj._xs(label, axis=axis)
  File "/apps/local/languages/anaconda/2020.07/lib/python3.8/site-packages/pandas/core/generic.py", line 3535, in xs
    loc, new_index = self.index.get_loc_level(key, drop_level=drop_level)
  File "/apps/local/languages/anaconda/2020.07/lib/python3.8/site-packages/pandas/core/indexes/multi.py", line 2823, in get_loc_level
    k = self._get_level_indexer(k, level=i)
  File "/apps/local/languages/anaconda/2020.07/lib/python3.8/site-packages/pandas/core/indexes/multi.py", line 2929, in _get_level_indexer
    code = self._get_loc_single_level_index(level_index, key)
  File "/apps/local/languages/anaconda/2020.07/lib/python3.8/site-packages/pandas/core/indexes/multi.py", line 2598, in _get_loc_single_level_index
    return level_index.get_loc(key)
  File "/apps/local/languages/anaconda/2020.07/lib/python3.8/site-packages/pandas/core/indexes/base.py", line 2648, in get_loc
    return self._engine.get_loc(self._maybe_cast_indexer(key))
  File "pandas/_libs/index.pyx", line 111, in pandas._libs.index.IndexEngine.get_loc
  File "pandas/_libs/index.pyx", line 138, in pandas._libs.index.IndexEngine.get_loc
  File "pandas/_libs/hashtable_class_helper.pxi", line 1619, in pandas._libs.hashtable.PyObjectHashTable.get_item
  File "pandas/_libs/hashtable_class_helper.pxi", line 1627, in pandas._libs.hashtable.PyObjectHashTable.get_item
KeyError: ''

My environment:

$ python --version
Python 3.8.3

I also have tabulate==0.8.9 and pandas==1.0.5 installed, the former is pip-based, the latter comes with the anaconda build.

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions