From e4bbaa76a40a6da0e2e820eecae4030ca176e3ad Mon Sep 17 00:00:00 2001 From: Robrecht Cannoodt Date: Tue, 28 Jul 2026 14:15:56 +0200 Subject: [PATCH 1/2] say what the ks metrics actually compute All 31 of these describe themselves as "The Kolmogorov-Smirnov statistic", but both components call `ks::kde.test()`, which is a kernel density based global two-sample comparison test -- a different thing that happens to live in a package called ks. The referenced doi (Duong's kernel smoothing book) was right all along. Left the metric ids alone so the results stay comparable across runs. --- .../ks_statistic_gene_cell/config.vsh.yaml | 112 +++++++++--------- .../ks_statistic_sc_features/config.vsh.yaml | 12 +- 2 files changed, 62 insertions(+), 62 deletions(-) diff --git a/src/metrics/ks_statistic_gene_cell/config.vsh.yaml b/src/metrics/ks_statistic_gene_cell/config.vsh.yaml index 6a3e475a..b7d0b053 100644 --- a/src/metrics/ks_statistic_gene_cell/config.vsh.yaml +++ b/src/metrics/ks_statistic_gene_cell/config.vsh.yaml @@ -10,9 +10,9 @@ info: metrics: - name: ks_statistic_frac_zero_genes_zstat label: Fraction of zeros per gene - summary: KS statistic of the fraction of zeros per gene. + summary: Kernel density two-sample statistic of the fraction of zeros per gene. description: | - The Kolmogorov-Smirnov statistic comparing the fraction of zeros per gene in the real datasets versus the fraction of zeros per gene in the simulated datasets. + The kernel density based global two-sample statistic (`ks::kde.test`) comparing the fraction of zeros per gene in the real datasets versus the fraction of zeros per gene in the simulated datasets. min: -Inf max: +Inf maximize: false @@ -20,9 +20,9 @@ info: doi: 10.1201/9780429485572 - name: ks_statistic_frac_zero_cells_zstat label: Fraction of zeros per cell - summary: KS statistic of the fraction of zeros per spot (cell). + summary: Kernel density two-sample statistic of the fraction of zeros per spot (cell). description: | - The Kolmogorov-Smirnov statistic comparing the fraction of zeros per spot (cell) in the real datasets versus the fraction of zeros per spot (cell) in the simulated datasets. + The kernel density based global two-sample statistic (`ks::kde.test`) comparing the fraction of zeros per spot (cell) in the real datasets versus the fraction of zeros per spot (cell) in the simulated datasets. min: -Inf max: +Inf maximize: false @@ -30,9 +30,9 @@ info: doi: 10.1201/9780429485572 - name: ks_statistic_lib_size_cells_zstat label: Library size - summary: KS statistic of the library size. + summary: Kernel density two-sample statistic of the library size. description: | - The Kolmogorov-Smirnov statistic comparing the total sum of UMI counts across all genes in the real datasets versus the total sum of UMI counts across all genes in the simmulated datasets. + The kernel density based global two-sample statistic (`ks::kde.test`) comparing the total sum of UMI counts across all genes in the real datasets versus the total sum of UMI counts across all genes in the simmulated datasets. min: -Inf max: +Inf maximize: false @@ -40,9 +40,9 @@ info: doi: 10.1201/9780429485572 - name: ks_statistic_efflib_size_cells_zstat label: Effective library size - summary: KS statistic of the effective library size. + summary: Kernel density two-sample statistic of the effective library size. description: | - The Kolmogorov-Smirnov statistic comparing the effective library size of the real datasets versus the effective library size of the simulated datasets. + The kernel density based global two-sample statistic (`ks::kde.test`) comparing the effective library size of the real datasets versus the effective library size of the simulated datasets. min: -Inf max: +Inf maximize: false @@ -50,9 +50,9 @@ info: doi: 10.1201/9780429485572 - name: ks_statistic_tmm_cells_zstat label: TMM - summary: KS statistic of the weight trimmed mean of M-values normalization factor (TMM). + summary: Kernel density two-sample statistic of the weight trimmed mean of M-values normalization factor (TMM). description: | - The Kolmogorov-Smirnov statistic comparing the weight trimmed mean of M-values normalization factor for the real datasets versus the weight trimmed mean of M-values normalization factor for the simulated datasets. + The kernel density based global two-sample statistic (`ks::kde.test`) comparing the weight trimmed mean of M-values normalization factor for the real datasets versus the weight trimmed mean of M-values normalization factor for the simulated datasets. min: -Inf max: +Inf maximize: false @@ -60,9 +60,9 @@ info: doi: 10.1201/9780429485572 - name: ks_statistic_scaled_var_cells_zstat label: Scaled variance cell - summary: KS statistic of the spot- (or cell-) level scaled variance of the expression matrix. + summary: Kernel density two-sample statistic of the spot- (or cell-) level scaled variance of the expression matrix. description: | - The Kolmogorov-Smirnov statistic comparing the spot-level z-score standardization of the variance of expression matrix in terms of log2(CPM) in the real datasets versus the simulated datasets. + The kernel density based global two-sample statistic (`ks::kde.test`) comparing the spot-level z-score standardization of the variance of expression matrix in terms of log2(CPM) in the real datasets versus the simulated datasets. min: -Inf max: +Inf maximize: false @@ -70,9 +70,9 @@ info: doi: 10.1201/9780429485572 - name: ks_statistic_scaled_mean_cells_zstat label: Scaled mean cells - summary: KS statistic of the spot- (or cell-) level scaled mean of the expression matrix. + summary: Kernel density two-sample statistic of the spot- (or cell-) level scaled mean of the expression matrix. description: | - The Kolmogorov-Smirnov statistic comparing the z-score standardization of the mean of expression matrix in terms of log2(CPM) in the real datasets versus the simulated datasets. + The kernel density based global two-sample statistic (`ks::kde.test`) comparing the z-score standardization of the mean of expression matrix in terms of log2(CPM) in the real datasets versus the simulated datasets. min: -Inf max: +Inf maximize: false @@ -80,9 +80,9 @@ info: doi: 10.1201/9780429485572 - name: ks_statistic_lib_fraczero_cells_zstat label: Library size vs fraction zero - summary: KS statistic of the relationship between library size and the proportion of zeros per spot (cell). + summary: Kernel density two-sample statistic of the relationship between library size and the proportion of zeros per spot (cell). description: | - The Kolmogorov-Smirnov statistic comparing the relationship between library size and the proportion of zeros per spot (cell) in the real datasets versus the simulated datasets. + The kernel density based global two-sample statistic (`ks::kde.test`) comparing the relationship between library size and the proportion of zeros per spot (cell) in the real datasets versus the simulated datasets. min: -Inf max: +Inf maximize: false @@ -90,9 +90,9 @@ info: doi: 10.1201/9780429485572 - name: ks_statistic_pearson_cells_zstat label: Sample Pearson correlation - summary: KS statistic of the sample Pearson correlation. + summary: Kernel density two-sample statistic of the sample Pearson correlation. description: | - The Kolmogorov-Smirnov statistic comparing the sample Pearson correlation of the real datasets versus the sample Pearson correlation of the simulated datasets. + The kernel density based global two-sample statistic (`ks::kde.test`) comparing the sample Pearson correlation of the real datasets versus the sample Pearson correlation of the simulated datasets. min: -Inf max: +Inf maximize: false @@ -100,9 +100,9 @@ info: doi: 10.1201/9780429485572 - name: ks_statistic_scaled_var_genes_zstat label: Scaled variance genes - summary: KS statistic of the gene-level scaled variance of the expression matrix. + summary: Kernel density two-sample statistic of the gene-level scaled variance of the expression matrix. description: | - The Kolmogorov-Smirnov statistic comparing the gene-level z-score standardization of the variance of expression matrix in terms of log2(CPM) in the real datasets versus the simulated datasets. + The kernel density based global two-sample statistic (`ks::kde.test`) comparing the gene-level z-score standardization of the variance of expression matrix in terms of log2(CPM) in the real datasets versus the simulated datasets. min: -Inf max: +Inf maximize: false @@ -110,9 +110,9 @@ info: doi: 10.1201/9780429485572 - name: ks_statistic_scaled_mean_genes_zstat label: Scaled mean genes - summary: KS statistic of the gene-level scaled mean of the expression matrix. + summary: Kernel density two-sample statistic of the gene-level scaled mean of the expression matrix. description: | - The Kolmogorov-Smirnov statistic comparing the gene-level z-score standardization of the mean of expression matrix in terms of log2(CPM) in the real datasets versus the simulated datasets. + The kernel density based global two-sample statistic (`ks::kde.test`) comparing the gene-level z-score standardization of the mean of expression matrix in terms of log2(CPM) in the real datasets versus the simulated datasets. min: -Inf max: +Inf maximize: false @@ -120,9 +120,9 @@ info: doi: 10.1201/9780429485572 - name: ks_statistic_pearson_genes_zstat label: Gene Pearson correlation - summary: KS statistic of the gene Pearson correlation. + summary: Kernel density two-sample statistic of the gene Pearson correlation. description: | - The Kolmogorov-Smirnov statistic comparing the gene Pearson correlation of the real datasets versus the gene Pearson correlation of the simulated datasets. + The kernel density based global two-sample statistic (`ks::kde.test`) comparing the gene Pearson correlation of the real datasets versus the gene Pearson correlation of the simulated datasets. min: -Inf max: +Inf maximize: false @@ -130,9 +130,9 @@ info: doi: 10.1201/9780429485572 - name: ks_statistic_mean_var_genes_zstat label: Mean vs variance - summary: KS statistic of the relationship between mean expression and variance expression. + summary: Kernel density two-sample statistic of the relationship between mean expression and variance expression. description: | - The Kolmogorov-Smirnov statistic comparing the relationship between mean expression and variance expression in the real datasets versus the simulated datasets. + The kernel density based global two-sample statistic (`ks::kde.test`) comparing the relationship between mean expression and variance expression in the real datasets versus the simulated datasets. min: -Inf max: +Inf maximize: false @@ -140,9 +140,9 @@ info: doi: 10.1201/9780429485572 - name: ks_statistic_mean_fraczero_genes_zstat label: Mean vs fraction zero - summary: KS statistic of the relationship between mean expression and the proportion of zero per gene. + summary: Kernel density two-sample statistic of the relationship between mean expression and the proportion of zero per gene. description: | - The Kolmogorov-Smirnov statistic comparing the relationship between mean expression and the proportion of zero per gene in the real datasets versus the simulated datasets. + The kernel density based global two-sample statistic (`ks::kde.test`) comparing the relationship between mean expression and the proportion of zero per gene in the real datasets versus the simulated datasets. min: -Inf max: +Inf maximize: false @@ -150,9 +150,9 @@ info: doi: 10.1201/9780429485572 - name: ks_statistic_frac_zero_genes_tstat label: Fraction of zeros per gene - summary: KS statistic of the fraction of zeros per gene. + summary: Kernel density two-sample statistic of the fraction of zeros per gene. description: | - The Kolmogorov-Smirnov statistic comparing the fraction of zeros per gene in the real datasets versus the fraction of zeros per gene in the simulated datasets. + The kernel density based global two-sample statistic (`ks::kde.test`) comparing the fraction of zeros per gene in the real datasets versus the fraction of zeros per gene in the simulated datasets. min: -Inf max: +Inf maximize: false @@ -160,9 +160,9 @@ info: doi: 10.1201/9780429485572 - name: ks_statistic_frac_zero_cells_tstat label: Fraction of zeros per cell - summary: KS statistic of the fraction of zeros per spot (cell). + summary: Kernel density two-sample statistic of the fraction of zeros per spot (cell). description: | - The Kolmogorov-Smirnov statistic comparing the fraction of zeros per spot (cell) in the real datasets versus the fraction of zeros per spot (cell) in the simulated datasets. + The kernel density based global two-sample statistic (`ks::kde.test`) comparing the fraction of zeros per spot (cell) in the real datasets versus the fraction of zeros per spot (cell) in the simulated datasets. min: -Inf max: +Inf maximize: false @@ -170,9 +170,9 @@ info: doi: 10.1201/9780429485572 - name: ks_statistic_lib_size_cells_tstat label: Library size - summary: KS statistic of the library size. + summary: Kernel density two-sample statistic of the library size. description: | - The Kolmogorov-Smirnov statistic comparing the total sum of UMI counts across all genes in the real datasets versus the total sum of UMI counts across all genes in the simmulated datasets. + The kernel density based global two-sample statistic (`ks::kde.test`) comparing the total sum of UMI counts across all genes in the real datasets versus the total sum of UMI counts across all genes in the simmulated datasets. min: -Inf max: +Inf maximize: false @@ -180,9 +180,9 @@ info: doi: 10.1201/9780429485572 - name: ks_statistic_efflib_size_cells_tstat label: Effective library size - summary: KS statistic of the effective library size. + summary: Kernel density two-sample statistic of the effective library size. description: | - The Kolmogorov-Smirnov statistic comparing the effective library size of the real datasets versus the effective library size of the simulated datasets. + The kernel density based global two-sample statistic (`ks::kde.test`) comparing the effective library size of the real datasets versus the effective library size of the simulated datasets. min: -Inf max: +Inf maximize: false @@ -190,9 +190,9 @@ info: doi: 10.1201/9780429485572 - name: ks_statistic_tmm_cells_tstat label: TMM - summary: KS statistic of the weight trimmed mean of M-values normalization factor (TMM). + summary: Kernel density two-sample statistic of the weight trimmed mean of M-values normalization factor (TMM). description: | - The Kolmogorov-Smirnov statistic comparing the weight trimmed mean of M-values normalization factor for the real datasets versus the weight trimmed mean of M-values normalization factor for the simulated datasets. + The kernel density based global two-sample statistic (`ks::kde.test`) comparing the weight trimmed mean of M-values normalization factor for the real datasets versus the weight trimmed mean of M-values normalization factor for the simulated datasets. min: -Inf max: +Inf maximize: false @@ -200,9 +200,9 @@ info: doi: 10.1201/9780429485572 - name: ks_statistic_scaled_var_cells_tstat label: Scaled variance cell - summary: KS statistic of the spot- (or cell-) level scaled variance of the expression matrix. + summary: Kernel density two-sample statistic of the spot- (or cell-) level scaled variance of the expression matrix. description: | - The Kolmogorov-Smirnov statistic comparing the spot-level z-score standardization of the variance of expression matrix in terms of log2(CPM) in the real datasets versus the simulated datasets. + The kernel density based global two-sample statistic (`ks::kde.test`) comparing the spot-level z-score standardization of the variance of expression matrix in terms of log2(CPM) in the real datasets versus the simulated datasets. min: -Inf max: +Inf maximize: false @@ -210,9 +210,9 @@ info: doi: 10.1201/9780429485572 - name: ks_statistic_scaled_mean_cells_tstat label: Scaled mean cells - summary: KS statistic of the spot- (or cell-) level scaled mean of the expression matrix. + summary: Kernel density two-sample statistic of the spot- (or cell-) level scaled mean of the expression matrix. description: | - The Kolmogorov-Smirnov statistic comparing the z-score standardization of the mean of expression matrix in terms of log2(CPM) in the real datasets versus the simulated datasets. + The kernel density based global two-sample statistic (`ks::kde.test`) comparing the z-score standardization of the mean of expression matrix in terms of log2(CPM) in the real datasets versus the simulated datasets. min: -Inf max: +Inf maximize: false @@ -220,9 +220,9 @@ info: doi: 10.1201/9780429485572 - name: ks_statistic_lib_fraczero_cells_tstat label: Library size vs fraction zero - summary: KS statistic of the relationship between library size and the proportion of zeros per spot (cell). + summary: Kernel density two-sample statistic of the relationship between library size and the proportion of zeros per spot (cell). description: | - The Kolmogorov-Smirnov statistic comparing the relationship between library size and the proportion of zeros per spot (cell) in the real datasets versus the simulated datasets. + The kernel density based global two-sample statistic (`ks::kde.test`) comparing the relationship between library size and the proportion of zeros per spot (cell) in the real datasets versus the simulated datasets. min: -Inf max: +Inf maximize: false @@ -230,9 +230,9 @@ info: doi: 10.1201/9780429485572 - name: ks_statistic_pearson_cells_tstat label: Sample Pearson correlation - summary: KS statistic of the sample Pearson correlation. + summary: Kernel density two-sample statistic of the sample Pearson correlation. description: | - The Kolmogorov-Smirnov statistic comparing the sample Pearson correlation of the real datasets versus the sample Pearson correlation of the simulated datasets. + The kernel density based global two-sample statistic (`ks::kde.test`) comparing the sample Pearson correlation of the real datasets versus the sample Pearson correlation of the simulated datasets. min: -Inf max: +Inf maximize: false @@ -240,9 +240,9 @@ info: doi: 10.1201/9780429485572 - name: ks_statistic_scaled_var_genes_tstat label: Scaled variance genes - summary: KS statistic of the gene-level scaled variance of the expression matrix. + summary: Kernel density two-sample statistic of the gene-level scaled variance of the expression matrix. description: | - The Kolmogorov-Smirnov statistic comparing the gene-level z-score standardization of the variance of expression matrix in terms of log2(CPM) in the real datasets versus the simulated datasets. + The kernel density based global two-sample statistic (`ks::kde.test`) comparing the gene-level z-score standardization of the variance of expression matrix in terms of log2(CPM) in the real datasets versus the simulated datasets. min: -Inf max: +Inf maximize: false @@ -250,9 +250,9 @@ info: doi: 10.1201/9780429485572 - name: ks_statistic_scaled_mean_genes_tstat label: Scaled mean genes - summary: KS statistic of the gene-level scaled mean of the expression matrix. + summary: Kernel density two-sample statistic of the gene-level scaled mean of the expression matrix. description: | - The Kolmogorov-Smirnov statistic comparing the gene-level z-score standardization of the mean of expression matrix in terms of log2(CPM) in the real datasets versus the simulated datasets. + The kernel density based global two-sample statistic (`ks::kde.test`) comparing the gene-level z-score standardization of the mean of expression matrix in terms of log2(CPM) in the real datasets versus the simulated datasets. min: -Inf max: +Inf maximize: false @@ -260,9 +260,9 @@ info: doi: 10.1201/9780429485572 - name: ks_statistic_pearson_genes_tstat label: Gene Pearson correlation - summary: KS statistic of the gene Pearson correlation. + summary: Kernel density two-sample statistic of the gene Pearson correlation. description: | - The Kolmogorov-Smirnov statistic comparing the gene Pearson correlation of the real datasets versus the gene Pearson correlation of the simulated datasets. + The kernel density based global two-sample statistic (`ks::kde.test`) comparing the gene Pearson correlation of the real datasets versus the gene Pearson correlation of the simulated datasets. min: -Inf max: +Inf maximize: false @@ -270,9 +270,9 @@ info: doi: 10.1201/9780429485572 - name: ks_statistic_mean_var_genes_tstat label: Mean vs variance - summary: KS statistic of the relationship between mean expression and variance expression. + summary: Kernel density two-sample statistic of the relationship between mean expression and variance expression. description: | - The Kolmogorov-Smirnov statistic comparing the relationship between mean expression and variance expression in the real datasets versus the simulated datasets. + The kernel density based global two-sample statistic (`ks::kde.test`) comparing the relationship between mean expression and variance expression in the real datasets versus the simulated datasets. min: -Inf max: +Inf maximize: false @@ -280,9 +280,9 @@ info: doi: 10.1201/9780429485572 - name: ks_statistic_mean_fraczero_genes_tstat label: Mean vs fraction zero - summary: KS statistic of the relationship between mean expression and the proportion of zero per gene. + summary: Kernel density two-sample statistic of the relationship between mean expression and the proportion of zero per gene. description: | - The Kolmogorov-Smirnov statistic comparing the relationship between mean expression and the proportion of zero per gene in the real datasets versus the simulated datasets. + The kernel density based global two-sample statistic (`ks::kde.test`) comparing the relationship between mean expression and the proportion of zero per gene in the real datasets versus the simulated datasets. min: -Inf max: +Inf maximize: false diff --git a/src/metrics/ks_statistic_sc_features/config.vsh.yaml b/src/metrics/ks_statistic_sc_features/config.vsh.yaml index 8fc5bd65..2e594c50 100644 --- a/src/metrics/ks_statistic_sc_features/config.vsh.yaml +++ b/src/metrics/ks_statistic_sc_features/config.vsh.yaml @@ -10,9 +10,9 @@ info: metrics: - name: ks_statistic_L_stats label: L statistics - summary: KS statistic of the L statistics + summary: Kernel density two-sample statistic of the L statistics description: | - The Kolmogorov-Smirnov statistic comparing the L statistics in the real datasets versus the L statistics in the simulated datasets. + The kernel density based global two-sample statistic (`ks::kde.test`) comparing the L statistics in the real datasets versus the L statistics in the simulated datasets. min: -Inf max: +Inf maximize: false @@ -20,9 +20,9 @@ info: doi: 10.1201/9780429485572 - name: ks_statistic_nn_correlation label: Nearest-neighbour correlation - summary: KS statistic of the nearest-neighbour correlation. + summary: Kernel density two-sample statistic of the nearest-neighbour correlation. description: | - The Kolmogorov-Smirnov statistic comparing the nn correlation in the real datasets versus the nn correlation in the simulated datasets. + The kernel density based global two-sample statistic (`ks::kde.test`) comparing the nn correlation in the real datasets versus the nn correlation in the simulated datasets. min: -Inf max: +Inf maximize: false @@ -30,9 +30,9 @@ info: doi: 10.1201/9780429485572 - name: ks_statistic_morans_I label: Moran's I - summary: KS statistic of Moran's I. + summary: Kernel density two-sample statistic of Moran's I. description: | - The Kolmogorov-Smirnov statistic comparing the Moran's I of the real datasets versus the Moran's I of the simulated datasets. + The kernel density based global two-sample statistic (`ks::kde.test`) comparing the Moran's I of the real datasets versus the Moran's I of the simulated datasets. min: -Inf max: +Inf maximize: false From 64663187b7681919216a08ad47db3a048e714289 Mon Sep 17 00:00:00 2001 From: Robrecht Cannoodt Date: Tue, 28 Jul 2026 14:19:45 +0200 Subject: [PATCH 2/2] update changelog --- CHANGELOG.md | 3 +++ 1 file changed, 3 insertions(+) diff --git a/CHANGELOG.md b/CHANGELOG.md index e3468069..a053c6fd 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -1,6 +1,9 @@ # task_spatial_simulators dev Bug fixes: + - `ks_statistic_gene_cell` and `ks_statistic_sc_features`: the metric + descriptions said Kolmogorov-Smirnov, but both call `ks::kde.test()`, which + is a kernel density based two-sample test. - `run_benchmark`: raise `uns_length_cutoff` from 15 to 50, so that `extract_uns_metadata` no longer drops the `metric_ids` of components that emit more than 15 metrics. All 28 `ks_statistic_gene_cell` metrics were