single-cell-annotation-skills-with-omicverse

Original source / Raw SKILL.md

---
name: single-cell-annotation-skills-with-omicverse
title: Single-cell annotation skills with omicverse
description: Guide Claude through SCSA, MetaTiME, CellVote, CellMatch, GPTAnno, and weighted KNN transfer workflows for annotating single-cell modalities.
---

# Single-cell annotation skills with omicverse

## Overview
Use this skill to reproduce and adapt the single-cell annotation playbook captured in omicverse tutorials: SCSA [`t_cellanno.ipynb`](../../../omicverse_guide/docs/Tutorials-single/t_cellanno.ipynb), MetaTiME [`t_metatime.ipynb`](../../../omicverse_guide/docs/Tutorials-single/t_metatime.ipynb), CellVote [`t_cellvote.md`](../../../omicverse_guide/docs/Tutorials-single/t_cellvote.md) & [`t_cellvote_pbmc3k.ipynb`](../../../omicverse_guide/docs/Tutorials-single/t_cellvote_pbmc3k.ipynb), CellMatch [`t_cellmatch.ipynb`](../../../omicverse_guide/docs/Tutorials-single/t_cellmatch.ipynb), GPTAnno [`t_gptanno.ipynb`](../../../omicverse_guide/docs/Tutorials-single/t_gptanno.ipynb), and label transfer [`t_anno_trans.ipynb`](../../../omicverse_guide/docs/Tutorials-single/t_anno_trans.ipynb). Each section below highlights required inputs, training/inference steps, and how to read the outputs.

## Instructions
1. **SCSA automated cluster annotation**
   - *Data requirements*: PBMC3k raw counts from 10x Genomics (`pbmc3k_filtered_gene_bc_matrices.tar.gz`) or the processed `sample/rna.h5ad`. Download instructions are embedded in the notebook; unpack to `data/filtered_gene_bc_matrices/hg19/`. Ensure an SCSA SQLite database is available (e.g. `pySCSA_2024_v1_plus.db` from the Figshare/Drive links listed in the tutorial) and point `model_path` to its location.
   - *Preprocessing & model fit*: Load with `sc.read_10x_mtx`, run QC (`ov.pp.qc`), normalization and HVG selection (`ov.pp.preprocess`), scaling (`ov.pp.scale`), PCA (`ov.pp.pca`), neighbors, Leiden clustering, and compute rank markers (`sc.tl.rank_genes_groups`). Instantiate `scsa = ov.single.pySCSA(...)` choosing `target='cellmarker'` or `'panglaodb'`, tissue scope, and thresholds (`foldchange`, `pvalue`).
   - *Inference & interpretation*: Call `scsa.cell_anno(clustertype='leiden', result_key='scsa_celltype_cellmarker')` or `scsa.cell_auto_anno` to append predictions to `adata.obs`. Compare to manual marker-based labels via `ov.utils.embedding` or `sc.pl.dotplot`, inspect marker dictionaries (`ov.single.get_celltype_marker`), and query supported tissues with `scsa.get_model_tissue()`. Use the ROI/ROE helpers (`ov.utils.roe`, `ov.utils.plot_cellproportion`) to validate abundance trends.

2. **MetaTiME tumour microenvironment states**
   - *Data requirements*: Batched TME AnnData with an scVI latent embedding. The tutorial uses `TiME_adata_scvi.h5ad` from Figshare (`https://figshare.com/ndownloader/files/41440050`). If starting from counts, run scVI (`scvi.model.SCVI`) first to populate `adata.obsm['X_scVI']`.
   - *Preprocessing & model fit*: Optionally subset to non-malignant cells via `adata.obs['isTME']`. Rebuild neighbors on the latent representation (`sc.pp.neighbors(adata, use_rep="X_scVI")`) and embed with pymde (`adata.obsm['X_mde'] = ov.utils.mde(...)`). Initialise `TiME_object = ov.single.MetaTiME(adata, mode='table')` and, if finer granularity is desired, over-cluster with `TiME_object.overcluster(resolution=8, clustercol='overcluster')`.
   - *Inference & interpretation*: Run `TiME_object.predictTiME(save_obs_name='MetaTiME')` to assign minor states and `Major_MetaTiME`. Visualise using `TiME_object.plot` or `sc.pl.embedding`. Interpret the outputs by comparing cluster-level distributions and confirming that MetaTiME and Major_MetaTiME columns align with expected niches.

3. **CellVote consensus labelling**
   - *Data requirements*: A clustered AnnData (e.g. PBMC3k stored as `CELLVOTE_PBMC3K` env var or `data/pbmc3k.h5ad`) plus at least two precomputed annotation columns (simulated in the tutorial as `scsa_annotation`, `gpt_celltype`, `gbi_celltype`). Prepare per-cluster marker genes via `sc.tl.rank_genes_groups`.
   - *Preprocessing & model fit*: After standard preprocessing (normalize, log1p, HVGs, PCA, neighbors, Leiden) build a marker dictionary `marker_dict = top_markers_from_rgg(adata, 'leiden', topn=10)` or via `ov.single.get_celltype_marker`. Instantiate `cv = ov.single.CellVote(adata)`.
   - *Inference & interpretation*: Call `cv.vote(clusters_key='leiden', cluster_markers=marker_dict, celltype_keys=[...], species='human', organization='PBMC', provider='openai', model='gpt-4o-mini')`. Offline examples monkey-patch arbitration to avoid API calls; online voting requires valid credentials. Final consensus labels live in `adata.obs['CellVote_celltype']`. Compare each cluster’s majority vote with the input sources (`adata.obs[['leiden', 'scsa_annotation', ...]]`) to justify decisions.

4. **CellMatch ontology mapping**
   - *Data requirements*: Annotated AnnData such as `pertpy.dt.haber_2017_regions()` with `adata.obs['cell_label']`. Download Cell Ontology JSON (`cl.json`) via `ov.single.download_cl(...)` or manual links, and optionally Cell Taxonomy resources (`Cell_Taxonomy_resource.txt`). Ensure access to a SentenceTransformer model (`sentence-transformers/all-MiniLM-L6-v2`, `BAAI/bge-base-en-v1.5`, etc.), downloading to `local_model_dir` if offline.
   - *Preprocessing & model fit*: Create the mapper with `ov.single.CellOntologyMapper(cl_obo_file='new_ontology/cl.json', model_name='sentence-transformers/all-MiniLM-L6-v2', local_model_dir='./my_models')`. Run `mapper.map_adata(...)` to assign ontology-derived labels/IDs, optionally enabling taxonomy matching (`use_taxonomy=True` after calling `load_cell_taxonomy_resource`).
   - *Inference & interpretation*: Explore mapping summaries (`mapper.print_mapping_summary_taxonomy`) and inspect embeddings coloured by `cell_ontology`, `cell_ontology_cl_id`, or `enhanced_cell_ontology`. Use helper queries such as `mapper.find_similar_cells('T helper cell')`, `mapper.get_cell_info(...)`, and category browsing to validate ontology coverage.

5. **GPTAnno LLM-powered annotation**
   - *Data requirements*: The same PBMC3k dataset (raw matrix or `.h5ad`) and cluster assignments. Access to an LLM endpoint—configure `AGI_API_KEY` for OpenAI-compatible providers (`provider='openai'`, `'qwen'`, `'kimi'`, etc.), or supply a local model path for `ov.single.gptcelltype_local`.
   - *Preprocessing & model fit*: Follow the QC, normalization, HVG, scaling, PCA, neighbor, Leiden, and marker discovery steps described above (reusing outputs from the SCSA workflow). Build the marker dictionary automatically with `ov.single.get_celltype_marker(adata, clustertype='leiden', rank=True, key='rank_genes_groups', foldchange=2, topgenenumber=5)`.
   - *Inference & interpretation*: Invoke `ov.single.gptcelltype(...)` specifying tissue/species context and desired provider/model. Post-process responses to keep clean labels (`result[key].split(': ')[-1]...`) and write them to `adata.obs['gpt_celltype']`. Compare embeddings (`ov.pl.embedding(..., color=['leiden','gpt_celltype'])`) to verify cluster identities. If operating offline, call `ov.single.gptcelltype_local` with a downloaded instruction-tuned checkpoint.

6. **Weighted KNN annotation transfer**
   - *Data requirements*: Cross-modal GLUE outputs with aligned embeddings, e.g. `data/analysis_lymph/rna-emb.h5ad` (annotated RNA) and `data/analysis_lymph/atac-emb.h5ad` (query ATAC) where both contain `obsm['X_glue']`.
   - *Preprocessing & model fit*: Load both modalities, optionally concatenate for QC plots, and compute a shared low-dimensional embedding with `ov.utils.mde`. Train a neighbour model using `ov.utils.weighted_knn_trainer(train_adata=rna, train_adata_emb='X_glue', n_neighbors=15)`.
   - *Inference & interpretation*: Transfer labels via `labels, uncert = ov.utils.weighted_knn_transfer(query_adata=atac, query_adata_emb='X_glue', label_keys='major_celltype', knn_model=knn_transformer, ref_adata_obs=rna.obs)`. Store predictions in `atac.obs['transf_celltype']` and uncertainties in `atac.obs['transf_celltype_unc']`; copy to `major_celltype` if you want consistent naming. Visualise (`ov.utils.embedding`) and inspect uncertainty to flag ambiguous cells.

## Critical API Reference - EXACT Function Signatures

### pySCSA - IMPORTANT: Parameter is `clustertype`, NOT `cluster`

**CORRECT usage:**
```python
# Step 1: Initialize pySCSA
scsa = ov.single.pySCSA(
    adata,
    foldchange=1.5,
    pvalue=0.01,
    species='Human',
    tissue='All',
    target='cellmarker'  # or 'panglaodb'
)

# Step 2: Run annotation - NOTE: use clustertype='leiden', NOT cluster='leiden'!
anno_result = scsa.cell_anno(clustertype='leiden', cluster='all')

# Step 3: Add cell type labels to adata.obs
scsa.cell_auto_anno(adata, clustertype='leiden', key='scsa_celltype')
# Results are stored in adata.obs['scsa_celltype']
```

**WRONG - DO NOT USE:**
```python
# WRONG! 'cluster' is NOT a valid parameter for cell_auto_anno!
# scsa.cell_auto_anno(adata, cluster='leiden')  # ERROR!
```

### COSG Marker Genes - Results stored in adata.uns, NOT adata.obs

**CORRECT usage:**
```python
# Step 1: Run COSG marker gene identification
ov.single.cosg(adata, groupby='leiden', n_genes_user=50)

# Step 2: Access results from adata.uns (NOT adata.obs!)
marker_names = adata.uns['rank_genes_groups']['names']  # DataFrame with cluster columns
marker_scores = adata.uns['rank_genes_groups']['scores']

# Step 3: Get top markers for specific cluster
cluster_0_markers = adata.uns['rank_genes_groups']['names']['0'][:10].tolist()

# Step 4: To create celltype column, manually map clusters to cell types
cluster_to_celltype = {
    '0': 'T cells',
    '1': 'B cells',
    '2': 'Monocytes',
}
adata.obs['cosg_celltype'] = adata.obs['leiden'].map(cluster_to_celltype)
```

**WRONG - DO NOT USE:**
```python
# WRONG! COSG does NOT create adata.obs columns directly!
# adata.obs['cosg_celltype']  # This key does NOT exist after running COSG!
# adata.uns['cosg_celltype']  # This key also does NOT exist!
```

### Common Pitfalls to Avoid

1. **pySCSA parameter confusion**:
   - `clustertype` = which obs column contains cluster labels (e.g., 'leiden')
   - `cluster` = which specific clusters to annotate ('all' or specific cluster IDs)
   - These are DIFFERENT parameters!

2. **COSG result access**:
   - COSG is a marker gene finder, NOT a cell type annotator
   - Results are per-cluster gene rankings stored in `adata.uns['rank_genes_groups']`
   - To assign cell types, you must manually map clusters to cell types based on markers

3. **Result storage patterns in OmicVerse**:
   - Cell type annotations → `adata.obs['<key>']`
   - Marker gene results → `adata.uns['<key>']` (includes 'names', 'scores', 'logfoldchanges')
   - Differential expression → `adata.uns['rank_genes_groups']`

## Examples
- "Run SCSA with both CellMarker and PanglaoDB references on PBMC3k, then benchmark against manual marker assignments before feeding the results into CellVote."
- "Annotate tumour microenvironment states in the MetaTiME Figshare dataset, highlight Major_MetaTiME classes, and export the label distribution per patient."
- "Download Cell Ontology resources, map `haber_2017_regions` clusters to ontology terms, and enrich ambiguous clusters using Cell Taxonomy hints."
- "Propagate RNA-derived `major_celltype` labels onto GLUE-integrated ATAC cells and report clusters with high transfer uncertainty."

## References
- Tutorials and notebooks: [`t_cellanno.ipynb`](../../../omicverse_guide/docs/Tutorials-single/t_cellanno.ipynb), [`t_metatime.ipynb`](../../../omicverse_guide/docs/Tutorials-single/t_metatime.ipynb), [`t_cellvote.md`](../../../omicverse_guide/docs/Tutorials-single/t_cellvote.md), [`t_cellvote_pbmc3k.ipynb`](../../../omicverse_guide/docs/Tutorials-single/t_cellvote_pbmc3k.ipynb), [`t_cellmatch.ipynb`](../../../omicverse_guide/docs/Tutorials-single/t_cellmatch.ipynb), [`t_gptanno.ipynb`](../../../omicverse_guide/docs/Tutorials-single/t_gptanno.ipynb), [`t_anno_trans.ipynb`](../../../omicverse_guide/docs/Tutorials-single/t_anno_trans.ipynb).
- Sample data & assets: PBMC3k matrix from 10x Genomics, MetaTiME `TiME_adata_scvi.h5ad` (Figshare), SCSA database downloads, GLUE embeddings under `data/analysis_lymph/`, Cell Ontology `cl.json`, and Cell Taxonomy resource.
- Quick copy commands: [`reference.md`](reference.md).


---

## Referenced Files

> The following files are referenced in this skill and included for context.

### ../../../omicverse_guide/docs/Tutorials-single/t_cellvote.md

```markdown
# Consensus annotation with CellVote

`CellVote` combines results from multiple cell type annotation approaches using a simple majority vote. By aggregating predictions, inconsistent labels can be resolved and the most plausible identity can be assigned to each cluster.

## Prerequisites

1. Clustering results should be available in `adata.obs` (e.g. the `leiden` field).
2. At least two independent cell type annotation results need to be stored in `adata.obs`. Typical methods include `scsa_anno`, `scMulan_anno` or GPT-based annotations such as `gpt_celltype`.
3. A dictionary of marker genes for each cluster is required. You can generate this with `ov.single.get_celltype_marker`.

## Basic usage

```python
import ov

# adata contains clustering results in "leiden"
cv = ov.single.CellVote(adata)
markers = ov.single.get_celltype_marker(adata)

cv.vote(
    clusters_key="leiden",
    cluster_markers=markers,
    celltype_keys=["scsa_annotation", "scMulan_anno"],
)

print(adata.obs["CellVote_celltype"].value_counts())
```

The final consensus label is stored in `adata.obs['CellVote_celltype']`.

## Example Notebook (PBMC3k)

For a complete, step-by-step walkthrough on PBMC3k, see the Jupyter notebook:

- Tutorials-single/t_cellvote_pbmc3k.ipynb

It covers preprocessing, clustering, marker selection, simulated multi-method annotations, and both offline and optional online CellVote arbitration.

## Advanced options

The `vote` method exposes a few additional arguments:

- `model`, `base_url` and `provider` allow you to specify a large language model when using GPT-based annotation as one of the voting sources.
- `result_key` changes the output column name.

```python
cv.vote(
    clusters_key="leiden",
    cluster_markers=markers,
    celltype_keys=["scsa_annotation", "gpt_celltype"],
    model="gpt-3.5-turbo",  # choose any model supported by your provider
    provider="openai",
    result_key="vote_label",
)
```

## Tips

- Ensure that the marker dictionary contains biologically meaningful genes to help resolve disagreements between annotation methods.
- You can inspect `adata.obs[['scsa_annotation','scMulan_anno','CellVote_celltype']]` to compare individual predictions with the final vote.
- Any number of annotation columns can be provided in `celltype_keys`.

```

### reference.md

```markdown
# Quick reference: single-cell annotation workflows

## SCSA (pySCSA)
```python
import scanpy as sc
import omicverse as ov

adata = sc.read_10x_mtx('data/filtered_gene_bc_matrices/hg19/', var_names='gene_symbols', cache=True)
adata = ov.pp.qc(adata, tresh={'mito_perc': 0.05, 'nUMIs': 500, 'detected_genes': 250})
adata = ov.pp.preprocess(adata, mode='shiftlog|pearson', n_HVGs=2000)
ov.pp.scale(adata)
ov.pp.pca(adata, layer='scaled', n_pcs=50)
sc.pp.neighbors(adata, n_neighbors=15, n_pcs=50, use_rep='scaled|original|X_pca')
sc.tl.leiden(adata)
sc.tl.rank_genes_groups(adata, 'leiden', method='wilcoxon', use_raw=False)

scsa = ov.single.pySCSA(
    adata,
    foldchange=1.5,
    pvalue=0.01,
    celltype='normal',
    target='cellmarker',
    tissue='All',
    model_path='temp/pySCSA_2024_v1_plus.db',
)
scsa.cell_auto_anno(adata, key='scsa_celltype_cellmarker')
```

## MetaTiME
```python
import scanpy as sc
import omicverse as ov

data_path = 'TiME_adata_scvi.h5ad'  # download from https://figshare.com/ndownloader/files/41440050
adata = sc.read(data_path)
sc.pp.neighbors(adata, use_rep='X_scVI')
adata.obsm['X_mde'] = ov.utils.mde(adata.obsm['X_scVI'])

TiME = ov.single.MetaTiME(adata, mode='table')
TiME.overcluster(resolution=8, clustercol='overcluster')
TiME.predictTiME(save_obs_name='MetaTiME')
```

## CellVote
```python
import anndata as ad
import omicverse as ov

adata = ad.read_h5ad('data/pbmc3k.h5ad')
ov.single.CellVote(adata).vote(
    clusters_key='leiden',
    cluster_markers=ov.single.get_celltype_marker(adata, clustertype='leiden', rank=True),
    celltype_keys=['scsa_annotation', 'gpt_celltype', 'gbi_celltype'],
    species='human',
    organization='PBMC',
    provider='openai',
    model='gpt-4o-mini',
)
```

## CellMatch / CellOntologyMapper
```bash
mkdir -p new_ontology
wget http://purl.obolibrary.org/obo/cl/cl.json -O new_ontology/cl.json
wget https://download.cncb.ac.cn/celltaxonomy/Cell_Taxonomy_resource.txt -O new_ontology/Cell_Taxonomy_resource.txt
```
```python
import pertpy as pt
import omicverse as ov

data = pt.dt.haber_2017_regions()
mapper = ov.single.CellOntologyMapper(
    cl_obo_file='new_ontology/cl.json',
    model_name='sentence-transformers/all-MiniLM-L6-v2',
    local_model_dir='./my_models',
)
mapper.load_cell_taxonomy_resource(
    'new_ontology/Cell_Taxonomy_resource.txt',
    species_filter=['Homo sapiens', 'Mus musculus'],
)
mapper.map_adata_with_taxonomy(
    data,
    cell_name_col='cell_label',
    new_col_name='enhanced_cell_ontology',
    expand_abbreviations=True,
    use_taxonomy=True,
    species='Mus musculus',
    tissue_context='Gut',
    threshold=0.3,
)
```

## GPTAnno (remote or local LLM)
```python
import os
import omicverse as ov

os.environ['AGI_API_KEY'] = 'sk-your-key'
markers = ov.single.get_celltype_marker(
    adata,
    clustertype='leiden',
    rank=True,
    key='rank_genes_groups',
    foldchange=2,
    topgenenumber=5,
)
result = ov.single.gptcelltype(
    markers,
    tissuename='PBMC',
    speciename='human',
    model='qwen-plus',
    provider='qwen',
    topgenenumber=5,
)
adata.obs['gpt_celltype'] = adata.obs['leiden'].map({
    k: v.split(': ')[-1].split(' (')[0].split('. ')[1]
    for k, v in result.items()
})
```
```python
# Offline alternative
ov.single.gptcelltype_local(
    markers,
    tissuename='PBMC',
    speciename='human',
    model_name='~/models/Qwen2-7B-Instruct',
    topgenenumber=5,
)
```

## Weighted KNN label transfer
```python
import scanpy as sc
import omicverse as ov

rna = sc.read('data/analysis_lymph/rna-emb.h5ad')
atac = sc.read('data/analysis_lymph/atac-emb.h5ad')
knn_model = ov.utils.weighted_knn_trainer(rna, train_adata_emb='X_glue', n_neighbors=15)
labels, uncert = ov.utils.weighted_knn_transfer(
    query_adata=atac,
    query_adata_emb='X_glue',
    label_keys='major_celltype',
    knn_model=knn_model,
    ref_adata_obs=rna.obs,
)
atac.obs['transf_celltype'] = labels.loc[atac.obs.index, 'major_celltype']
atac.obs['transf_celltype_unc'] = uncert.loc[atac.obs.index, 'major_celltype']
```

```