ValueError: Transition matrix is not row stochastic, 8216 rows do not sum to 1.

[simonekats]

...

Computing transition matrix based on pseudotime
100%|████████████████████████████████| 48370/48370 [00:03<00:00, 12904.07cell/s]
WARNING: Biased k-NN graph is disconnected

ValueError Traceback (most recent call last)
Cell In[133], line 1
----> 1 pk.compute_transition_matrix()

File ~/anaconda3/lib/python3.10/site-packages/cellrank/kernels/_pseudotime_kernel.py:173, in PseudotimeKernel.compute_transition_matrix(self, threshold_scheme, frac_to_keep, b, nu, check_irreducibility, n_jobs, backend, show_progress_bar, **kwargs)
170 if check_irreducibility and not _irreducible(biased_conn):
171 logg.warning("Biased k-NN graph is not irreducible")
--> 173 self.transition_matrix = biased_conn
174 logg.info(" Finish", time=start)
176 return self

File ~/anaconda3/lib/python3.10/site-packages/cellrank/kernels/_base_kernel.py:720, in Kernel.transition_matrix(self, matrix)
718 @transition_matrix.setter
719 def transition_matrix(self, matrix: Any) -> None:
--> 720 KernelExpression.transition_matrix.fset(self, matrix)

File ~/anaconda3/lib/python3.10/site-packages/cellrank/kernels/_base_kernel.py:433, in KernelExpression.transition_matrix(self, matrix)
431 if should_norm(matrix): # some rows are all 0s/contain invalid values
432 n_inv = np.sum(~np.isclose(np.asarray(matrix.sum(1)).squeeze(), 1.0, rtol=1e-12))
--> 433 raise ValueError(f"Transition matrix is not row stochastic, {n_inv} rows do not sum to 1.")
434 # fmt: on
436 self._transition_matrix = matrix

ValueError: Transition matrix is not row stochastic, 8216 rows do not sum to 1.

[Marius1311]

Looks like your transition matrix might be a bit ill conditioned and you get rows that don't sum to one.

[WeilerP]

@simonekats, can you check if the error persists when you combine your kernel with a ConnectivityKernel ck, i.e. 0.8 * pk + 0.2 * ck, for example? We observed in the past that it can improve the condition of the transition matrix.

[simonekats]

I get this error:
RuntimeError: PseudotimeKernel[n=48370] is uninitialized. Compute its transition matrix first as .compute_transition_matrix().

[WeilerP]

Well, you obviously need to initialize it; since you did not provide a code snippet, I presumed you were using the PseudotimeKernel since you called the kernel pk. Please have a look at our tutorials to see how you can combine two or more kernels.

[simonekats]

[simonekats]

It is initialized, just does not have transition matrix computed which is my problem

[WeilerP]

Sorry, my bad! I was answering another issue at the same time and forgot that the problem already occurs when computing the transition matrix, not afterward.

[simonekats]

No worries, do you possibly have a solution?

[Marius1311]

Can you compute the transition matrix for the PseudotimeKernel before combining it with the ConnectivityKernel?

[WeilerP]

Can you compute the transition matrix for the PseudotimeKernel before combining it with the ConnectivityKernel?

@Marius1311, that's the same idea I had but computing the transition matrix for the PseudotimeKernel is the step that's actually failing.

[simonekats]

I also tried another one of my objects and ran into this error with the transition matrix:
NetworkXError: Adjacency matrix not square: nx,ny=(6915, 6914)

[simonekats]

I also want to mention that I am working with subsetted objects, could this have some sort of effect on this?

[WeilerP]

I also want to mention that I am working with subsetted objects, could this have some sort of effect on this?

Definitely, if you do not recompute the neighbor graph, I would say.