Error when running modules = list(modules_from_adjacencies(adjacencies, ex_matrix))

[morganee261]

Hello,

I am getting this error the running modules = list(modules_from_adjacencies(adjacencies, ex_matrix)) :

2020-03-23 13:46:47,461 - pyscenic.utils - INFO - Calculating Pearson correlations.

2020-03-23 13:46:47,461 - pyscenic.utils - WARNING - Note on correlation calculation: the default behaviour for calculating the correlations has changed after pySCENIC verion 0.9.16. Previously, the default was to calculate the correlation between a TF and target gene using only cells with non-zero expression values (mask_dropouts=True). The current default is now to use all cells to match the behavior of the R verision of SCENIC. The original settings can be retained by setting 'rho_mask_dropouts=True' in the modules_from_adjacencies function, or '--mask_dropouts' from the CLI.
Dropout masking is currently set to [False].
Traceback (most recent call last):
File "/home/Morgane/anaconda3/lib/python3.7/site-packages/pandas/core/indexes/base.py", line 2897, in get_loc
return self._engine.get_loc(key)
File "pandas/_libs/index.pyx", line 107, in pandas._libs.index.IndexEngine.get_loc
File "pandas/_libs/index.pyx", line 128, in pandas._libs.index.IndexEngine.get_loc
File "pandas/_libs/index_class_helper.pxi", line 91, in pandas._libs.index.Int64Engine._check_type
KeyError: 'RPS19'

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
File "", line 1, in
File "/home/Morgane/anaconda3/lib/python3.7/site-packages/pyscenic/utils.py", line 268, in modules_from_adjacencies
rho_threshold=rho_threshold, mask_dropouts=rho_mask_dropouts)
File "/home/Morgane/anaconda3/lib/python3.7/site-packages/pyscenic/utils.py", line 136, in add_correlation
rhos = np.array([corr_mtx[s2][s1] for s1, s2 in zip(adjacencies.TF, adjacencies.target)])
File "/home/Morgane/anaconda3/lib/python3.7/site-packages/pyscenic/utils.py", line 136, in
rhos = np.array([corr_mtx[s2][s1] for s1, s2 in zip(adjacencies.TF, adjacencies.target)])
File "/home/Morgane/anaconda3/lib/python3.7/site-packages/pandas/core/frame.py", line 2980, in getitem
indexer = self.columns.get_loc(key)
File "/home/Morgane/anaconda3/lib/python3.7/site-packages/pandas/core/indexes/base.py", line 2899, in get_loc
return self._engine.get_loc(self._maybe_cast_indexer(key))
File "pandas/_libs/index.pyx", line 107, in pandas._libs.index.IndexEngine.get_loc
File "pandas/_libs/index.pyx", line 128, in pandas._libs.index.IndexEngine.get_loc
File "pandas/_libs/index_class_helper.pxi", line 91, in pandas._libs.index.Int64Engine._check_type
KeyError: 'RPS19'

I have read other users errors and tried trouble shooting without any luck.

thanks for your help
Morgane

[cflerin]

Hi,

This issue looks similar to #103 , but I'm not sure if it was every really solved. Perhaps you could try running this step via the CLI?

[morganee261]

Hello,

Do you have a tutorial for the CLI? Can I start with the expression matrix filtered fro the R package or do I need to run correlation first and then export it?

thanks for your answer
Morgane

[cflerin]

From this tutorial (search for "cisTarget"), you can see the usage of the command line:

pyscenic ctx adj.tsv \
    {f_db_names} \
    --annotations_fname {f_motif_path} \
    --expression_mtx_fname {f_loom_path_scenic} \
    --output reg.csv \
    --num_workers 20

The correlation step is baked into this already, but you may need to write out your grn output (adj.tsv) to a text file if you're running from Jupyter.

[morganee261]

Thanks for your help,
I had to re run the first part to get the adjacencies file in the correct form. I started the second part to get regulons 2 days ago and it is still running. do you have an idea of how long it might take? my matrix is about 40,000 cells and more than 22,000 genes.
Morgane

[cflerin]

@morganee261 , sorry for the delayed reply. It sounds like you fixed this particular problem so I'll close this