Slow/stalled run of multiscale phate on large datasets using python 3.7.6

[PaulFremont3]

Hello,

I am having trouble running multiscale phate using a dataset with size (>200000, 20).

My code:

"
data = load_matrix('data_filename.txt')
stats = load_vector('stations_filename.txt ')
unigenes = load_vector('genes_filename.txt')
df = pd.DataFrame(data, columns = stats, index=unigenes)
n_cores=40

print(type(df))
print(df.shape)
mp_op = multiscale_phate.Multiscale_PHATE(n_jobs=n_cores, random_state=1)
levels = mp_op.fit(df)
"

output::
"
<class 'pandas.core.frame.DataFrame'>
(274774, 20)
Calculating Multiscale PHATE tree...
Calculating PCA...
Calculated PCA in 0.33 seconds.
Calculating partitions...
"
After running PCA quite fast, the algorithm seems to be stalled when calculating partitions: I let it run for several days of calculation using multiple cores (which I verified are running) but the job is still at the same point ('calculating partitions...').

From your examples, multiscale PHATE should run much faster than this on this kind of dataset.

My dataset is:
in rows: genes expression (scaled between 0 and 1 with many 0)
in columns: different samples

Let me know if you have some insights on this.

Thank you

[bai101315]

Hello. I've also run into the same mistake as you. The program seems to be stuck in the 'Calculating partitions...' operation all the time. But when I was reproducing the results in the paper, this operation was relatively fast.I would greatly appreciate it if you could provide me with some assistance. Thank you.