Import SCOT by scottgigante-immunai · Pull Request #333 · openproblems-bio/openproblems

scottgigante-immunai · 2022-03-16T15:09:45Z

Per codecov, SCOT was never tested or benchmarked

Submission type

This submission adds a new method

Testing

This submission was written on a forked copy of SingleCellOpenProblems
GitHub Actions "Run Benchmark" tests are passing on this base branch of this pull request (include link to passed test: https://github.com/scottgigante-immunai/openproblems/runs/6025251317?check_suite_focus=true)
If this pull request is not ready for review (including passing the "Run Benchmark" tests), I will open this PR as a draft (click on the down arrow next to the "Create Pull Request" button)

Submission guidelines

This submission follows the guidelines in our Contributing document
I have checked to ensure there aren't other open Pull Requests for the same update/change

PR review checklist

This PR will be evaluated on the basis of the following checks:

The task addresses a valid open problem in single-cell analysis
The latest version of master is merged and tested
The methods/metrics are imported to __init__.py and were tested in the pipeline
Method and metric decorators are annotated with paper title, year, author, code version, and date
The README gives an outline of the methods, metrics and datasets in the folder

…ante-immunai/openproblems into scottgigante/bugfix/scot_import

scottgigante-immunai · 2022-03-29T21:55:07Z

@atong01 I might need your help here. Turns out when you merged SCOT into the repo, it was never included in init.py, so it was never tested. When I added it, I've hit some errors that I don't know how to fix -- SCOT is producing outputs that are not finite. See, e.g., https://github.com/scottgigante-immunai/openproblems/runs/5744360234?check_suite_focus=true

…ante-immunai/openproblems into scottgigante/bugfix/scot_import

…t_import

scottgigante-immunai · 2022-04-14T15:53:13Z

Balanced SCOT is working, so I'll merge this. Unbalanced SCOT is broken (there's an overflow happening somewhere) and I'm not sure if we should fix it or remove it.

codecov-commenter · 2022-04-14T16:46:40Z

Codecov Report

Merging #333 (a8be9cd) into main (0e17517) will increase coverage by 2.02%.
The diff coverage is 100.00%.

@@            Coverage Diff             @@
##             main     #333      +/-   ##
==========================================
+ Coverage   88.42%   90.45%   +2.02%     
==========================================
  Files          77       77              
  Lines        1685     1687       +2     
  Branches       94       94              
==========================================
+ Hits         1490     1526      +36     
+ Misses        167      133      -34     
  Partials       28       28

Flag	Coverage Δ
unittests	`90.45% <100.00%> (+2.02%)`	⬆️

Flags with carried forward coverage won't be shown. Click here to find out more.

Impacted Files	Coverage Δ
.../tasks/multimodal_data_integration/methods/scot.py	`80.95% <ø> (+80.95%)`	⬆️
...nproblems/tasks/multimodal_data_integration/api.py	`100.00% <100.00%> (ø)`

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 0e17517...a8be9cd. Read the comment docs.

* upstream/main: Fix benchmark commit (openproblems-bio#362) Remove scot unbalanced (openproblems-bio#360) store results in /tmp (openproblems-bio#361) fix gh actions badge link # ci skip (openproblems-bio#359) fix coverage badge # ci skip (openproblems-bio#358) Import SCOT (openproblems-bio#333) fix parsing and committing of results on tag (openproblems-bio#356)

* upstream/main: Fix benchmark commit (openproblems-bio#362) Remove scot unbalanced (openproblems-bio#360) store results in /tmp (openproblems-bio#361) fix gh actions badge link # ci skip (openproblems-bio#359) fix coverage badge # ci skip (openproblems-bio#358) Import SCOT (openproblems-bio#333) fix parsing and committing of results on tag (openproblems-bio#356) If using GH actions image, test for git diff on dockerfile (openproblems-bio#350) Run benchmark only after unittests (openproblems-bio#349) Label docker images based on build location (openproblems-bio#351)

* Label docker images based on build location (#351) * label docker images * fix syntax * Run benchmark only after unittests (#349) * run benchmark after unittests * always run cleanup * cleanup * If using GH actions image, test for git diff on dockerfile (#350) * if using gh actions image, test for git diff on dockerfile * allow empty tag for now * decode * if image doesn't exist, automatically github actions * fix quotes * fix parsing and committing of results on tag (#356) * Import SCOT (#333) * import SCOT * pre-commit * scran requires R * check that aligned spaces are finite * exclude unbalanced SCOT for now Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com> Co-authored-by: Scott Gigante <scottgigante@users.noreply.github.com> * fix coverage badge # ci skip (#358) * fix gh actions badge link # ci skip (#359) * store results in /tmp (#361) * Remove scot unbalanced (#360) * Fix benchmark commit (#362) * store results in /tmp * add skip_on_empty * class doesn't have skip on empty * remove scot altogether (#363) * Allow codecov to fail on forks * docker images separate PR (#354) * docker images separate PR * all R requirements in r_requirements.txt * move github r packages to requirements file * pre-commit Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com> * Ignore AWS warning and clean up s3 properly (#366) * ci cleanup * ignore aws batch warning * remove citeseq cbmc from DR (#367) Co-authored-by: Scott Gigante <scott.gigante@immunai.com> * Update benchmark results # ci skip (#368) Co-authored-by: SingleCellOpenProblems <singlecellopenproblems@protonmail.com> * Jamboree dimensionality reduction methods (#318) * add densMAP package to python-extras * pre-commit * Add Ivis method * Explicitly mention it's CPU implementation * Add forgotten import in __init__ * Remove redundant filtering * Move ivis inside the function * Make var names unique, add ivis[cpu] to README * Pin tensorflow version * Add NeuralEE skeleton * Implement method * added densmap and densne * Fix typo pytoch -> torch * pre-commit * remove densne * Add forgotten detach/cpu/numpy * formatting * pre-commit * formatting * formatting * pre-commit * formatting * formatting * formatting * pre-commit * formatting * umap-learn implementation * pre-commit * Add docker image * Add skeleton method * formatting * Implement method * Fix some small bugs * Add preprocessing * Change batch size to 1k cells for aff. matrix * Add new preprocessing * Add new preprocessing * Fix preprocessing * Fix preprocessing * pre-commit * updated template for PR with PR evaluation checks (#314) * Update alra.py (#304) * Update alra.py Fix pre-processing and transformation back into the original space * pre-commit * Update alra.py * make sure necessary methods are imported Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com> Co-authored-by: Daniel Burkhardt <burkhardt.d.b@gmail.com> * Add scanpy preprocessing to densmap dimred method * Rename preprocess_scanpy() to preprocess_logCPM_1kHVG() * Add preprocessing suffix to dimred methods * Subset object in preprocess_logCPM_1kHVG() * Use standard names for input * Add neuralee_logCPM_1kHVG method * Add densmap_pca method * Fix preprocess_logCPM_1kHVG() Now returns an AnnData rather than acting in place - Subsetting wasn't working in place Also set HVG flavor to "cell_ranger" * Add test argument to dimred methods * Move preprocess_logCPM_1kHVG() to tools.normalize * Change name in python-method-scvis Docker README * Rename openproblems-python-method-scvis container Now called open-problems-python36 * Fix AnnData ref in merge * Copy object when subsetting in preprocess_logCPM_1kHVG() * Move PCA to dimred methods * Use preprocess_logCPM_1kHVG() in nn_ranking metrics * Fix path in python36 dockerfile * Add test kwarg to neuralee_default method * Add check for n_var to preprocess_logCPM_1kHVG() Should fix tests that were failing due to scverse/scanpy#2230 * Store raw counts in NeuralEE default method * Update dimred README * Replace X_input with PCA in ivis dimred method * Refactor preprocess_logCPM_1kHVG() to log_cpm_hvg() * Remove ivis * pre-commit Co-authored-by: Ben DeMeo <bdemeo@mit.edu> Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com> Co-authored-by: Michal Klein <michal.klein@protonmail.com> Co-authored-by: michalk8 <46717574+michalk8@users.noreply.github.com> Co-authored-by: bendemeo <45370700+bendemeo@users.noreply.github.com> Co-authored-by: MalteDLuecken <m.d.luecken@gmail.com> Co-authored-by: Wesley Lewis <59123674+wes-lewis@users.noreply.github.com> Co-authored-by: Daniel Burkhardt <burkhardt.d.b@gmail.com> Co-authored-by: Scott Gigante <84813314+scottgigante-immunai@users.noreply.github.com> * Only cleanup AWS on success (#371) * only cleanup on success * pre-commit Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com> * Jamboree label_projection task (#313) * Add scvi-tools docker image * add scanvi * hvg command use 2000 * update scvi-tools version; use image * train size * scanvi mask test labels * move import * hvg on train only, fix hvg command * add scarches scanvi * use string labels in testing * enforce batch metadata in dataset * add batch metadata in pancreas random * use train adata for scarches * Add majority vote simple baseline * test_mode * use test instead of test mode, update contributing * update contributing guide * Added helper function to introduce label noise * Actually return data with label noise * Only introduce label noise on training data * Made a pancreas dataset with label nosie * Reformat docstring * Added reference to example label noise dataset in datasets __init__.py * Add cengen C elegans data loader (#2) * add CeNGEN C elegans neuron dataset * add CeNGEN C elegans dataset for global tasks and for label_projection task * fix lines being too long * Reformat cengen data loader * Create tabula_muris_senis.py Need dataframe containing sample information in './tabula_muris_senis_data_objects/tabula_muris_senis_data_objects.csv' load_tabula_muris_senis(method_list, organ_list) takes in methods and organs to extract data from and combines into one anndata object. If method_list or organ_list = None, do not filter based on that input. EX: load_tabula_muris_senis(method_list=['facs'], organ_list = None) returns all facs experiments for all organs in one anndata object. * pre-commit * Modify anndata in place in add_label_noise rather than copy * Added CSV file with tabula muris senis data links * Update tabula_muris_senis.py * Add random_labels baseline to label_projection task * Update tabula_muris_senis.py * Update tabula_muris_senis.py * pre-commit * Update tabula_muris_senis.py * pre-commit * fix missing labels at prediction time * Handle test flag through tests and docker, pass to methods * If test method run, use 1 max_epoch for scvi-tools * Use only 2 batches for sample dataset for label_projection * Remove zebrafish random dataset * Fix decorator dependency to <5.0.0 * Remove functools.wraps from docker decorator for test parameterization * Fix cengen missing batch info * Use functools.update_wrapper for docker test * Add batch to pancreas_random_label_noise * Make cengen test dataset have more cells per batch * Set span=0.8 for hvg call for scanvi_hvg methods * Set span=0.8 for HVG selection only in test mode for scvi * Revert "Handle test flag through tests and docker, pass to methods" This reverts commit 3b940c0. * Add test parameter to label proj baselines * Fix flake remove unused import * Revert "Remove zebrafish random dataset" This reverts commit 3915798. * Update scVI setup_anndata to new version * pre-commit * Reformat and rerun tests * Add code_url and code_version for baseline label proj methods * Fallback HVG flavor for label projection task * pre-commit * Fix unused import * Fix using highly_variable_genes * Pin scvi-tools to 0.15.5 * Unpin scvi-tools, pin jax==0.3.6, see optuna/optuna-examples#99 * Add scikit-misc as requirement for scvi docker * Pin jaxlib as well * pin jaxlib along with jax * Set paper_year to year of implementation * Set random zebrafish split to 0.8+0.2 * Add tabula_muris_senis_lung_random dataset to label_projection * pre-commit * Add tabula muris senis datasets csv * Fix loading tabula muris csv * pre-commit * Test loader for tabula muris senis Co-authored-by: adamgayoso <adamgayoso@users.noreply.github.com> Co-authored-by: Valentine Svensson <v@nxn.se> Co-authored-by: Eduardo Beltrame <munfred@brandeis.edu> Co-authored-by: atchen <81202350+atchen@users.noreply.github.com> Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com> Co-authored-by: Scott Gigante <84813314+scottgigante-immunai@users.noreply.github.com> * Run `test_benchmark` on a self-hosted runner (#373) * set up cirun * use ubuntu standard AMI * run nextflow on the self-hosted machine * add to CONTRIBUTING * update ami * install unzip * set up docker * install docker from curl * use t2.micro not nano * use custom AMI * pythonLocation * add scripts to path * larger disk size * new image again * chown for now * chmod 755 * fixed permissions * use tower workspace * test nextflow * try again * nextflow -q * redirect stderr * increase memory * cleanup * sudo install * name * try setting pythonpath * fix branch env * another fix * fix run name * typo * fix pythonpath: * don't use pushd * pass pythonpath * set nousersite * empty * sudo install * run attempt * revert temporary changes * cleanup * fix contributing * add instructions for tower * fix repo name * move ami setup into script * Import Olsson 2016 dataset for dimred task (#352) * Import Olsson 2016 dataset for dimred task * Fix path to Olsson dataset loader * Filter genes cells before subsetting Olsson data in test * Use highly expressed genes for test Olsson dataset Test dataset is now 700 genes by 300 cells (was 500 x 500) * Add ivis dimred method (#369) * add densMAP package to python-extras * pre-commit * Add Ivis method * Explicitly mention it's CPU implementation * Add forgotten import in __init__ * Remove redundant filtering * Move ivis inside the function * Make var names unique, add ivis[cpu] to README * Pin tensorflow version * Add NeuralEE skeleton * Implement method * added densmap and densne * Fix typo pytoch -> torch * pre-commit * remove densne * Add forgotten detach/cpu/numpy * formatting * pre-commit * formatting * formatting * pre-commit * formatting * formatting * formatting * pre-commit * formatting * umap-learn implementation * pre-commit * Add docker image * Add skeleton method * formatting * Implement method * Fix some small bugs * Add preprocessing * Change batch size to 1k cells for aff. matrix * Add new preprocessing * Add new preprocessing * Fix preprocessing * Fix preprocessing * pre-commit * updated template for PR with PR evaluation checks (#314) * Update alra.py (#304) * Update alra.py Fix pre-processing and transformation back into the original space * pre-commit * Update alra.py * make sure necessary methods are imported Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com> Co-authored-by: Daniel Burkhardt <burkhardt.d.b@gmail.com> * Add scanpy preprocessing to densmap dimred method * Rename preprocess_scanpy() to preprocess_logCPM_1kHVG() * Add preprocessing suffix to dimred methods * Subset object in preprocess_logCPM_1kHVG() * Use standard names for input * Add neuralee_logCPM_1kHVG method * Add densmap_pca method * Fix preprocess_logCPM_1kHVG() Now returns an AnnData rather than acting in place - Subsetting wasn't working in place Also set HVG flavor to "cell_ranger" * Add test argument to dimred methods * Move preprocess_logCPM_1kHVG() to tools.normalize * Change name in python-method-scvis Docker README * Rename openproblems-python-method-scvis container Now called open-problems-python36 * Fix AnnData ref in merge * Copy object when subsetting in preprocess_logCPM_1kHVG() * Move PCA to dimred methods * Use preprocess_logCPM_1kHVG() in nn_ranking metrics * Fix path in python36 dockerfile * Add test kwarg to neuralee_default method * Add check for n_var to preprocess_logCPM_1kHVG() Should fix tests that were failing due to scverse/scanpy#2230 * Store raw counts in NeuralEE default method * Update dimred README * Replace X_input with PCA in ivis dimred method * Refactor preprocess_logCPM_1kHVG() to log_cpm_hvg() * Re-add ivis Co-authored-by: Ben DeMeo <bdemeo@mit.edu> Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com> Co-authored-by: Michal Klein <michal.klein@protonmail.com> Co-authored-by: michalk8 <46717574+michalk8@users.noreply.github.com> Co-authored-by: bendemeo <45370700+bendemeo@users.noreply.github.com> Co-authored-by: MalteDLuecken <m.d.luecken@gmail.com> Co-authored-by: Wesley Lewis <59123674+wes-lewis@users.noreply.github.com> Co-authored-by: Daniel Burkhardt <burkhardt.d.b@gmail.com> Co-authored-by: Scott Gigante <84813314+scottgigante-immunai@users.noreply.github.com> * hotfix timeout-minutes (#374) * use branch of scprep to provide R traceback (#376) Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com> Co-authored-by: Scott Gigante <scottgigante@users.noreply.github.com> Co-authored-by: Daniel Strobl <50872326+danielStrobl@users.noreply.github.com> Co-authored-by: SingleCellOpenProblems <singlecellopenproblems@protonmail.com> Co-authored-by: Luke Zappia <lazappi@users.noreply.github.com> Co-authored-by: Ben DeMeo <bdemeo@mit.edu> Co-authored-by: Michal Klein <michal.klein@protonmail.com> Co-authored-by: michalk8 <46717574+michalk8@users.noreply.github.com> Co-authored-by: bendemeo <45370700+bendemeo@users.noreply.github.com> Co-authored-by: MalteDLuecken <m.d.luecken@gmail.com> Co-authored-by: Wesley Lewis <59123674+wes-lewis@users.noreply.github.com> Co-authored-by: Daniel Burkhardt <burkhardt.d.b@gmail.com> Co-authored-by: Nikolay Markov <nikolai.markov@icloud.com> Co-authored-by: adamgayoso <adamgayoso@users.noreply.github.com> Co-authored-by: Valentine Svensson <v@nxn.se> Co-authored-by: Eduardo Beltrame <munfred@brandeis.edu> Co-authored-by: atchen <81202350+atchen@users.noreply.github.com>

* Label docker images based on build location (#351) * label docker images * fix syntax * Run benchmark only after unittests (#349) * run benchmark after unittests * always run cleanup * cleanup * If using GH actions image, test for git diff on dockerfile (#350) * if using gh actions image, test for git diff on dockerfile * allow empty tag for now * decode * if image doesn't exist, automatically github actions * fix quotes * fix parsing and committing of results on tag (#356) * Import SCOT (#333) * import SCOT * pre-commit * scran requires R * check that aligned spaces are finite * exclude unbalanced SCOT for now Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com> Co-authored-by: Scott Gigante <scottgigante@users.noreply.github.com> * fix coverage badge # ci skip (#358) * fix gh actions badge link # ci skip (#359) * store results in /tmp (#361) * Remove scot unbalanced (#360) * Fix benchmark commit (#362) * store results in /tmp * add skip_on_empty * class doesn't have skip on empty * remove scot altogether (#363) * Allow codecov to fail on forks * docker images separate PR (#354) * docker images separate PR * all R requirements in r_requirements.txt * move github r packages to requirements file * pre-commit Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com> * Ignore AWS warning and clean up s3 properly (#366) * ci cleanup * ignore aws batch warning * remove citeseq cbmc from DR (#367) Co-authored-by: Scott Gigante <scott.gigante@immunai.com> * Update benchmark results # ci skip (#368) Co-authored-by: SingleCellOpenProblems <singlecellopenproblems@protonmail.com> * Jamboree dimensionality reduction methods (#318) * add densMAP package to python-extras * pre-commit * Add Ivis method * Explicitly mention it's CPU implementation * Add forgotten import in __init__ * Remove redundant filtering * Move ivis inside the function * Make var names unique, add ivis[cpu] to README * Pin tensorflow version * Add NeuralEE skeleton * Implement method * added densmap and densne * Fix typo pytoch -> torch * pre-commit * remove densne * Add forgotten detach/cpu/numpy * formatting * pre-commit * formatting * formatting * pre-commit * formatting * formatting * formatting * pre-commit * formatting * umap-learn implementation * pre-commit * Add docker image * Add skeleton method * formatting * Implement method * Fix some small bugs * Add preprocessing * Change batch size to 1k cells for aff. matrix * Add new preprocessing * Add new preprocessing * Fix preprocessing * Fix preprocessing * pre-commit * updated template for PR with PR evaluation checks (#314) * Update alra.py (#304) * Update alra.py Fix pre-processing and transformation back into the original space * pre-commit * Update alra.py * make sure necessary methods are imported Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com> Co-authored-by: Daniel Burkhardt <burkhardt.d.b@gmail.com> * Add scanpy preprocessing to densmap dimred method * Rename preprocess_scanpy() to preprocess_logCPM_1kHVG() * Add preprocessing suffix to dimred methods * Subset object in preprocess_logCPM_1kHVG() * Use standard names for input * Add neuralee_logCPM_1kHVG method * Add densmap_pca method * Fix preprocess_logCPM_1kHVG() Now returns an AnnData rather than acting in place - Subsetting wasn't working in place Also set HVG flavor to "cell_ranger" * Add test argument to dimred methods * Move preprocess_logCPM_1kHVG() to tools.normalize * Change name in python-method-scvis Docker README * Rename openproblems-python-method-scvis container Now called open-problems-python36 * Fix AnnData ref in merge * Copy object when subsetting in preprocess_logCPM_1kHVG() * Move PCA to dimred methods * Use preprocess_logCPM_1kHVG() in nn_ranking metrics * Fix path in python36 dockerfile * Add test kwarg to neuralee_default method * Add check for n_var to preprocess_logCPM_1kHVG() Should fix tests that were failing due to scverse/scanpy#2230 * Store raw counts in NeuralEE default method * Update dimred README * Replace X_input with PCA in ivis dimred method * Refactor preprocess_logCPM_1kHVG() to log_cpm_hvg() * Remove ivis * pre-commit Co-authored-by: Ben DeMeo <bdemeo@mit.edu> Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com> Co-authored-by: Michal Klein <michal.klein@protonmail.com> Co-authored-by: michalk8 <46717574+michalk8@users.noreply.github.com> Co-authored-by: bendemeo <45370700+bendemeo@users.noreply.github.com> Co-authored-by: MalteDLuecken <m.d.luecken@gmail.com> Co-authored-by: Wesley Lewis <59123674+wes-lewis@users.noreply.github.com> Co-authored-by: Daniel Burkhardt <burkhardt.d.b@gmail.com> Co-authored-by: Scott Gigante <84813314+scottgigante-immunai@users.noreply.github.com> * Only cleanup AWS on success (#371) * only cleanup on success * pre-commit Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com> * Jamboree label_projection task (#313) * Add scvi-tools docker image * add scanvi * hvg command use 2000 * update scvi-tools version; use image * train size * scanvi mask test labels * move import * hvg on train only, fix hvg command * add scarches scanvi * use string labels in testing * enforce batch metadata in dataset * add batch metadata in pancreas random * use train adata for scarches * Add majority vote simple baseline * test_mode * use test instead of test mode, update contributing * update contributing guide * Added helper function to introduce label noise * Actually return data with label noise * Only introduce label noise on training data * Made a pancreas dataset with label nosie * Reformat docstring * Added reference to example label noise dataset in datasets __init__.py * Add cengen C elegans data loader (#2) * add CeNGEN C elegans neuron dataset * add CeNGEN C elegans dataset for global tasks and for label_projection task * fix lines being too long * Reformat cengen data loader * Create tabula_muris_senis.py Need dataframe containing sample information in './tabula_muris_senis_data_objects/tabula_muris_senis_data_objects.csv' load_tabula_muris_senis(method_list, organ_list) takes in methods and organs to extract data from and combines into one anndata object. If method_list or organ_list = None, do not filter based on that input. EX: load_tabula_muris_senis(method_list=['facs'], organ_list = None) returns all facs experiments for all organs in one anndata object. * pre-commit * Modify anndata in place in add_label_noise rather than copy * Added CSV file with tabula muris senis data links * Update tabula_muris_senis.py * Add random_labels baseline to label_projection task * Update tabula_muris_senis.py * Update tabula_muris_senis.py * pre-commit * Update tabula_muris_senis.py * pre-commit * fix missing labels at prediction time * Handle test flag through tests and docker, pass to methods * If test method run, use 1 max_epoch for scvi-tools * Use only 2 batches for sample dataset for label_projection * Remove zebrafish random dataset * Fix decorator dependency to <5.0.0 * Remove functools.wraps from docker decorator for test parameterization * Fix cengen missing batch info * Use functools.update_wrapper for docker test * Add batch to pancreas_random_label_noise * Make cengen test dataset have more cells per batch * Set span=0.8 for hvg call for scanvi_hvg methods * Set span=0.8 for HVG selection only in test mode for scvi * Revert "Handle test flag through tests and docker, pass to methods" This reverts commit 3b940c0. * Add test parameter to label proj baselines * Fix flake remove unused import * Revert "Remove zebrafish random dataset" This reverts commit 3915798. * Update scVI setup_anndata to new version * pre-commit * Reformat and rerun tests * Add code_url and code_version for baseline label proj methods * Fallback HVG flavor for label projection task * pre-commit * Fix unused import * Fix using highly_variable_genes * Pin scvi-tools to 0.15.5 * Unpin scvi-tools, pin jax==0.3.6, see optuna/optuna-examples#99 * Add scikit-misc as requirement for scvi docker * Pin jaxlib as well * pin jaxlib along with jax * Set paper_year to year of implementation * Set random zebrafish split to 0.8+0.2 * Add tabula_muris_senis_lung_random dataset to label_projection * pre-commit * Add tabula muris senis datasets csv * Fix loading tabula muris csv * pre-commit * Test loader for tabula muris senis Co-authored-by: adamgayoso <adamgayoso@users.noreply.github.com> Co-authored-by: Valentine Svensson <v@nxn.se> Co-authored-by: Eduardo Beltrame <munfred@brandeis.edu> Co-authored-by: atchen <81202350+atchen@users.noreply.github.com> Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com> Co-authored-by: Scott Gigante <84813314+scottgigante-immunai@users.noreply.github.com> * Run `test_benchmark` on a self-hosted runner (#373) * set up cirun * use ubuntu standard AMI * run nextflow on the self-hosted machine * add to CONTRIBUTING * update ami * install unzip * set up docker * install docker from curl * use t2.micro not nano * use custom AMI * pythonLocation * add scripts to path * larger disk size * new image again * chown for now * chmod 755 * fixed permissions * use tower workspace * test nextflow * try again * nextflow -q * redirect stderr * increase memory * cleanup * sudo install * name * try setting pythonpath * fix branch env * another fix * fix run name * typo * fix pythonpath: * don't use pushd * pass pythonpath * set nousersite * empty * sudo install * run attempt * revert temporary changes * cleanup * fix contributing * add instructions for tower * fix repo name * move ami setup into script * Import Olsson 2016 dataset for dimred task (#352) * Import Olsson 2016 dataset for dimred task * Fix path to Olsson dataset loader * Filter genes cells before subsetting Olsson data in test * Use highly expressed genes for test Olsson dataset Test dataset is now 700 genes by 300 cells (was 500 x 500) * Add ivis dimred method (#369) * add densMAP package to python-extras * pre-commit * Add Ivis method * Explicitly mention it's CPU implementation * Add forgotten import in __init__ * Remove redundant filtering * Move ivis inside the function * Make var names unique, add ivis[cpu] to README * Pin tensorflow version * Add NeuralEE skeleton * Implement method * added densmap and densne * Fix typo pytoch -> torch * pre-commit * remove densne * Add forgotten detach/cpu/numpy * formatting * pre-commit * formatting * formatting * pre-commit * formatting * formatting * formatting * pre-commit * formatting * umap-learn implementation * pre-commit * Add docker image * Add skeleton method * formatting * Implement method * Fix some small bugs * Add preprocessing * Change batch size to 1k cells for aff. matrix * Add new preprocessing * Add new preprocessing * Fix preprocessing * Fix preprocessing * pre-commit * updated template for PR with PR evaluation checks (#314) * Update alra.py (#304) * Update alra.py Fix pre-processing and transformation back into the original space * pre-commit * Update alra.py * make sure necessary methods are imported Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com> Co-authored-by: Daniel Burkhardt <burkhardt.d.b@gmail.com> * Add scanpy preprocessing to densmap dimred method * Rename preprocess_scanpy() to preprocess_logCPM_1kHVG() * Add preprocessing suffix to dimred methods * Subset object in preprocess_logCPM_1kHVG() * Use standard names for input * Add neuralee_logCPM_1kHVG method * Add densmap_pca method * Fix preprocess_logCPM_1kHVG() Now returns an AnnData rather than acting in place - Subsetting wasn't working in place Also set HVG flavor to "cell_ranger" * Add test argument to dimred methods * Move preprocess_logCPM_1kHVG() to tools.normalize * Change name in python-method-scvis Docker README * Rename openproblems-python-method-scvis container Now called open-problems-python36 * Fix AnnData ref in merge * Copy object when subsetting in preprocess_logCPM_1kHVG() * Move PCA to dimred methods * Use preprocess_logCPM_1kHVG() in nn_ranking metrics * Fix path in python36 dockerfile * Add test kwarg to neuralee_default method * Add check for n_var to preprocess_logCPM_1kHVG() Should fix tests that were failing due to scverse/scanpy#2230 * Store raw counts in NeuralEE default method * Update dimred README * Replace X_input with PCA in ivis dimred method * Refactor preprocess_logCPM_1kHVG() to log_cpm_hvg() * Re-add ivis Co-authored-by: Ben DeMeo <bdemeo@mit.edu> Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com> Co-authored-by: Michal Klein <michal.klein@protonmail.com> Co-authored-by: michalk8 <46717574+michalk8@users.noreply.github.com> Co-authored-by: bendemeo <45370700+bendemeo@users.noreply.github.com> Co-authored-by: MalteDLuecken <m.d.luecken@gmail.com> Co-authored-by: Wesley Lewis <59123674+wes-lewis@users.noreply.github.com> Co-authored-by: Daniel Burkhardt <burkhardt.d.b@gmail.com> Co-authored-by: Scott Gigante <84813314+scottgigante-immunai@users.noreply.github.com> * hotfix timeout-minutes (#374) * use branch of scprep to provide R traceback (#376) * Install libgeos-dev Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com> Co-authored-by: Scott Gigante <scottgigante@users.noreply.github.com> Co-authored-by: Daniel Strobl <50872326+danielStrobl@users.noreply.github.com> Co-authored-by: SingleCellOpenProblems <singlecellopenproblems@protonmail.com> Co-authored-by: Luke Zappia <lazappi@users.noreply.github.com> Co-authored-by: Ben DeMeo <bdemeo@mit.edu> Co-authored-by: Michal Klein <michal.klein@protonmail.com> Co-authored-by: michalk8 <46717574+michalk8@users.noreply.github.com> Co-authored-by: bendemeo <45370700+bendemeo@users.noreply.github.com> Co-authored-by: MalteDLuecken <m.d.luecken@gmail.com> Co-authored-by: Wesley Lewis <59123674+wes-lewis@users.noreply.github.com> Co-authored-by: Daniel Burkhardt <burkhardt.d.b@gmail.com> Co-authored-by: Nikolay Markov <nikolai.markov@icloud.com> Co-authored-by: adamgayoso <adamgayoso@users.noreply.github.com> Co-authored-by: Valentine Svensson <v@nxn.se> Co-authored-by: Eduardo Beltrame <munfred@brandeis.edu> Co-authored-by: atchen <81202350+atchen@users.noreply.github.com>

* Label docker images based on build location (#351) * label docker images * fix syntax * Run benchmark only after unittests (#349) * run benchmark after unittests * always run cleanup * cleanup * If using GH actions image, test for git diff on dockerfile (#350) * if using gh actions image, test for git diff on dockerfile * allow empty tag for now * decode * if image doesn't exist, automatically github actions * fix quotes * fix parsing and committing of results on tag (#356) * Import SCOT (#333) * import SCOT * pre-commit * scran requires R * check that aligned spaces are finite * exclude unbalanced SCOT for now Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com> Co-authored-by: Scott Gigante <scottgigante@users.noreply.github.com> * fix coverage badge # ci skip (#358) * fix gh actions badge link # ci skip (#359) * store results in /tmp (#361) * Remove scot unbalanced (#360) * Fix benchmark commit (#362) * store results in /tmp * add skip_on_empty * class doesn't have skip on empty * remove scot altogether (#363) * Allow codecov to fail on forks * docker images separate PR (#354) * docker images separate PR * all R requirements in r_requirements.txt * move github r packages to requirements file * pre-commit Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com> * Ignore AWS warning and clean up s3 properly (#366) * ci cleanup * ignore aws batch warning * remove citeseq cbmc from DR (#367) Co-authored-by: Scott Gigante <scott.gigante@immunai.com> * Update benchmark results # ci skip (#368) Co-authored-by: SingleCellOpenProblems <singlecellopenproblems@protonmail.com> * Jamboree dimensionality reduction methods (#318) * add densMAP package to python-extras * pre-commit * Add Ivis method * Explicitly mention it's CPU implementation * Add forgotten import in __init__ * Remove redundant filtering * Move ivis inside the function * Make var names unique, add ivis[cpu] to README * Pin tensorflow version * Add NeuralEE skeleton * Implement method * added densmap and densne * Fix typo pytoch -> torch * pre-commit * remove densne * Add forgotten detach/cpu/numpy * formatting * pre-commit * formatting * formatting * pre-commit * formatting * formatting * formatting * pre-commit * formatting * umap-learn implementation * pre-commit * Add docker image * Add skeleton method * formatting * Implement method * Fix some small bugs * Add preprocessing * Change batch size to 1k cells for aff. matrix * Add new preprocessing * Add new preprocessing * Fix preprocessing * Fix preprocessing * pre-commit * updated template for PR with PR evaluation checks (#314) * Update alra.py (#304) * Update alra.py Fix pre-processing and transformation back into the original space * pre-commit * Update alra.py * make sure necessary methods are imported Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com> Co-authored-by: Daniel Burkhardt <burkhardt.d.b@gmail.com> * Add scanpy preprocessing to densmap dimred method * Rename preprocess_scanpy() to preprocess_logCPM_1kHVG() * Add preprocessing suffix to dimred methods * Subset object in preprocess_logCPM_1kHVG() * Use standard names for input * Add neuralee_logCPM_1kHVG method * Add densmap_pca method * Fix preprocess_logCPM_1kHVG() Now returns an AnnData rather than acting in place - Subsetting wasn't working in place Also set HVG flavor to "cell_ranger" * Add test argument to dimred methods * Move preprocess_logCPM_1kHVG() to tools.normalize * Change name in python-method-scvis Docker README * Rename openproblems-python-method-scvis container Now called open-problems-python36 * Fix AnnData ref in merge * Copy object when subsetting in preprocess_logCPM_1kHVG() * Move PCA to dimred methods * Use preprocess_logCPM_1kHVG() in nn_ranking metrics * Fix path in python36 dockerfile * Add test kwarg to neuralee_default method * Add check for n_var to preprocess_logCPM_1kHVG() Should fix tests that were failing due to scverse/scanpy#2230 * Store raw counts in NeuralEE default method * Update dimred README * Replace X_input with PCA in ivis dimred method * Refactor preprocess_logCPM_1kHVG() to log_cpm_hvg() * Remove ivis * pre-commit Co-authored-by: Ben DeMeo <bdemeo@mit.edu> Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com> Co-authored-by: Michal Klein <michal.klein@protonmail.com> Co-authored-by: michalk8 <46717574+michalk8@users.noreply.github.com> Co-authored-by: bendemeo <45370700+bendemeo@users.noreply.github.com> Co-authored-by: MalteDLuecken <m.d.luecken@gmail.com> Co-authored-by: Wesley Lewis <59123674+wes-lewis@users.noreply.github.com> Co-authored-by: Daniel Burkhardt <burkhardt.d.b@gmail.com> Co-authored-by: Scott Gigante <84813314+scottgigante-immunai@users.noreply.github.com> * Only cleanup AWS on success (#371) * only cleanup on success * pre-commit Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com> * Jamboree label_projection task (#313) * Add scvi-tools docker image * add scanvi * hvg command use 2000 * update scvi-tools version; use image * train size * scanvi mask test labels * move import * hvg on train only, fix hvg command * add scarches scanvi * use string labels in testing * enforce batch metadata in dataset * add batch metadata in pancreas random * use train adata for scarches * Add majority vote simple baseline * test_mode * use test instead of test mode, update contributing * update contributing guide * Added helper function to introduce label noise * Actually return data with label noise * Only introduce label noise on training data * Made a pancreas dataset with label nosie * Reformat docstring * Added reference to example label noise dataset in datasets __init__.py * Add cengen C elegans data loader (#2) * add CeNGEN C elegans neuron dataset * add CeNGEN C elegans dataset for global tasks and for label_projection task * fix lines being too long * Reformat cengen data loader * Create tabula_muris_senis.py Need dataframe containing sample information in './tabula_muris_senis_data_objects/tabula_muris_senis_data_objects.csv' load_tabula_muris_senis(method_list, organ_list) takes in methods and organs to extract data from and combines into one anndata object. If method_list or organ_list = None, do not filter based on that input. EX: load_tabula_muris_senis(method_list=['facs'], organ_list = None) returns all facs experiments for all organs in one anndata object. * pre-commit * Modify anndata in place in add_label_noise rather than copy * Added CSV file with tabula muris senis data links * Update tabula_muris_senis.py * Add random_labels baseline to label_projection task * Update tabula_muris_senis.py * Update tabula_muris_senis.py * pre-commit * Update tabula_muris_senis.py * pre-commit * fix missing labels at prediction time * Handle test flag through tests and docker, pass to methods * If test method run, use 1 max_epoch for scvi-tools * Use only 2 batches for sample dataset for label_projection * Remove zebrafish random dataset * Fix decorator dependency to <5.0.0 * Remove functools.wraps from docker decorator for test parameterization * Fix cengen missing batch info * Use functools.update_wrapper for docker test * Add batch to pancreas_random_label_noise * Make cengen test dataset have more cells per batch * Set span=0.8 for hvg call for scanvi_hvg methods * Set span=0.8 for HVG selection only in test mode for scvi * Revert "Handle test flag through tests and docker, pass to methods" This reverts commit 3b940c0. * Add test parameter to label proj baselines * Fix flake remove unused import * Revert "Remove zebrafish random dataset" This reverts commit 3915798. * Update scVI setup_anndata to new version * pre-commit * Reformat and rerun tests * Add code_url and code_version for baseline label proj methods * Fallback HVG flavor for label projection task * pre-commit * Fix unused import * Fix using highly_variable_genes * Pin scvi-tools to 0.15.5 * Unpin scvi-tools, pin jax==0.3.6, see optuna/optuna-examples#99 * Add scikit-misc as requirement for scvi docker * Pin jaxlib as well * pin jaxlib along with jax * Set paper_year to year of implementation * Set random zebrafish split to 0.8+0.2 * Add tabula_muris_senis_lung_random dataset to label_projection * pre-commit * Add tabula muris senis datasets csv * Fix loading tabula muris csv * pre-commit * Test loader for tabula muris senis Co-authored-by: adamgayoso <adamgayoso@users.noreply.github.com> Co-authored-by: Valentine Svensson <v@nxn.se> Co-authored-by: Eduardo Beltrame <munfred@brandeis.edu> Co-authored-by: atchen <81202350+atchen@users.noreply.github.com> Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com> Co-authored-by: Scott Gigante <84813314+scottgigante-immunai@users.noreply.github.com> * Run `test_benchmark` on a self-hosted runner (#373) * set up cirun * use ubuntu standard AMI * run nextflow on the self-hosted machine * add to CONTRIBUTING * update ami * install unzip * set up docker * install docker from curl * use t2.micro not nano * use custom AMI * pythonLocation * add scripts to path * larger disk size * new image again * chown for now * chmod 755 * fixed permissions * use tower workspace * test nextflow * try again * nextflow -q * redirect stderr * increase memory * cleanup * sudo install * name * try setting pythonpath * fix branch env * another fix * fix run name * typo * fix pythonpath: * don't use pushd * pass pythonpath * set nousersite * empty * sudo install * run attempt * revert temporary changes * cleanup * fix contributing * add instructions for tower * fix repo name * move ami setup into script * Import Olsson 2016 dataset for dimred task (#352) * Import Olsson 2016 dataset for dimred task * Fix path to Olsson dataset loader * Filter genes cells before subsetting Olsson data in test * Use highly expressed genes for test Olsson dataset Test dataset is now 700 genes by 300 cells (was 500 x 500) * Add ivis dimred method (#369) * add densMAP package to python-extras * pre-commit * Add Ivis method * Explicitly mention it's CPU implementation * Add forgotten import in __init__ * Remove redundant filtering * Move ivis inside the function * Make var names unique, add ivis[cpu] to README * Pin tensorflow version * Add NeuralEE skeleton * Implement method * added densmap and densne * Fix typo pytoch -> torch * pre-commit * remove densne * Add forgotten detach/cpu/numpy * formatting * pre-commit * formatting * formatting * pre-commit * formatting * formatting * formatting * pre-commit * formatting * umap-learn implementation * pre-commit * Add docker image * Add skeleton method * formatting * Implement method * Fix some small bugs * Add preprocessing * Change batch size to 1k cells for aff. matrix * Add new preprocessing * Add new preprocessing * Fix preprocessing * Fix preprocessing * pre-commit * updated template for PR with PR evaluation checks (#314) * Update alra.py (#304) * Update alra.py Fix pre-processing and transformation back into the original space * pre-commit * Update alra.py * make sure necessary methods are imported Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com> Co-authored-by: Daniel Burkhardt <burkhardt.d.b@gmail.com> * Add scanpy preprocessing to densmap dimred method * Rename preprocess_scanpy() to preprocess_logCPM_1kHVG() * Add preprocessing suffix to dimred methods * Subset object in preprocess_logCPM_1kHVG() * Use standard names for input * Add neuralee_logCPM_1kHVG method * Add densmap_pca method * Fix preprocess_logCPM_1kHVG() Now returns an AnnData rather than acting in place - Subsetting wasn't working in place Also set HVG flavor to "cell_ranger" * Add test argument to dimred methods * Move preprocess_logCPM_1kHVG() to tools.normalize * Change name in python-method-scvis Docker README * Rename openproblems-python-method-scvis container Now called open-problems-python36 * Fix AnnData ref in merge * Copy object when subsetting in preprocess_logCPM_1kHVG() * Move PCA to dimred methods * Use preprocess_logCPM_1kHVG() in nn_ranking metrics * Fix path in python36 dockerfile * Add test kwarg to neuralee_default method * Add check for n_var to preprocess_logCPM_1kHVG() Should fix tests that were failing due to scverse/scanpy#2230 * Store raw counts in NeuralEE default method * Update dimred README * Replace X_input with PCA in ivis dimred method * Refactor preprocess_logCPM_1kHVG() to log_cpm_hvg() * Re-add ivis Co-authored-by: Ben DeMeo <bdemeo@mit.edu> Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com> Co-authored-by: Michal Klein <michal.klein@protonmail.com> Co-authored-by: michalk8 <46717574+michalk8@users.noreply.github.com> Co-authored-by: bendemeo <45370700+bendemeo@users.noreply.github.com> Co-authored-by: MalteDLuecken <m.d.luecken@gmail.com> Co-authored-by: Wesley Lewis <59123674+wes-lewis@users.noreply.github.com> Co-authored-by: Daniel Burkhardt <burkhardt.d.b@gmail.com> Co-authored-by: Scott Gigante <84813314+scottgigante-immunai@users.noreply.github.com> * hotfix timeout-minutes (#374) * use branch of scprep to provide R traceback (#376) Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com> Co-authored-by: Scott Gigante <scottgigante@users.noreply.github.com> Co-authored-by: Daniel Strobl <50872326+danielStrobl@users.noreply.github.com> Co-authored-by: SingleCellOpenProblems <singlecellopenproblems@protonmail.com> Co-authored-by: Luke Zappia <lazappi@users.noreply.github.com> Co-authored-by: Ben DeMeo <bdemeo@mit.edu> Co-authored-by: Michal Klein <michal.klein@protonmail.com> Co-authored-by: michalk8 <46717574+michalk8@users.noreply.github.com> Co-authored-by: bendemeo <45370700+bendemeo@users.noreply.github.com> Co-authored-by: MalteDLuecken <m.d.luecken@gmail.com> Co-authored-by: Wesley Lewis <59123674+wes-lewis@users.noreply.github.com> Co-authored-by: Daniel Burkhardt <burkhardt.d.b@gmail.com> Co-authored-by: Nikolay Markov <nikolai.markov@icloud.com> Co-authored-by: adamgayoso <adamgayoso@users.noreply.github.com> Co-authored-by: Valentine Svensson <v@nxn.se> Co-authored-by: Eduardo Beltrame <munfred@brandeis.edu> Co-authored-by: atchen <81202350+atchen@users.noreply.github.com>

* label docker images * fix syntax * Delete run_benchmark.yml * Update from main (#378) * Label docker images based on build location (#351) * label docker images * fix syntax * Run benchmark only after unittests (#349) * run benchmark after unittests * always run cleanup * cleanup * If using GH actions image, test for git diff on dockerfile (#350) * if using gh actions image, test for git diff on dockerfile * allow empty tag for now * decode * if image doesn't exist, automatically github actions * fix quotes * fix parsing and committing of results on tag (#356) * Import SCOT (#333) * import SCOT * pre-commit * scran requires R * check that aligned spaces are finite * exclude unbalanced SCOT for now Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com> Co-authored-by: Scott Gigante <scottgigante@users.noreply.github.com> * fix coverage badge # ci skip (#358) * fix gh actions badge link # ci skip (#359) * store results in /tmp (#361) * Remove scot unbalanced (#360) * Fix benchmark commit (#362) * store results in /tmp * add skip_on_empty * class doesn't have skip on empty * remove scot altogether (#363) * Allow codecov to fail on forks * docker images separate PR (#354) * docker images separate PR * all R requirements in r_requirements.txt * move github r packages to requirements file * pre-commit Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com> * Ignore AWS warning and clean up s3 properly (#366) * ci cleanup * ignore aws batch warning * remove citeseq cbmc from DR (#367) Co-authored-by: Scott Gigante <scott.gigante@immunai.com> * Update benchmark results # ci skip (#368) Co-authored-by: SingleCellOpenProblems <singlecellopenproblems@protonmail.com> * Jamboree dimensionality reduction methods (#318) * add densMAP package to python-extras * pre-commit * Add Ivis method * Explicitly mention it's CPU implementation * Add forgotten import in __init__ * Remove redundant filtering * Move ivis inside the function * Make var names unique, add ivis[cpu] to README * Pin tensorflow version * Add NeuralEE skeleton * Implement method * added densmap and densne * Fix typo pytoch -> torch * pre-commit * remove densne * Add forgotten detach/cpu/numpy * formatting * pre-commit * formatting * formatting * pre-commit * formatting * formatting * formatting * pre-commit * formatting * umap-learn implementation * pre-commit * Add docker image * Add skeleton method * formatting * Implement method * Fix some small bugs * Add preprocessing * Change batch size to 1k cells for aff. matrix * Add new preprocessing * Add new preprocessing * Fix preprocessing * Fix preprocessing * pre-commit * updated template for PR with PR evaluation checks (#314) * Update alra.py (#304) * Update alra.py Fix pre-processing and transformation back into the original space * pre-commit * Update alra.py * make sure necessary methods are imported Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com> Co-authored-by: Daniel Burkhardt <burkhardt.d.b@gmail.com> * Add scanpy preprocessing to densmap dimred method * Rename preprocess_scanpy() to preprocess_logCPM_1kHVG() * Add preprocessing suffix to dimred methods * Subset object in preprocess_logCPM_1kHVG() * Use standard names for input * Add neuralee_logCPM_1kHVG method * Add densmap_pca method * Fix preprocess_logCPM_1kHVG() Now returns an AnnData rather than acting in place - Subsetting wasn't working in place Also set HVG flavor to "cell_ranger" * Add test argument to dimred methods * Move preprocess_logCPM_1kHVG() to tools.normalize * Change name in python-method-scvis Docker README * Rename openproblems-python-method-scvis container Now called open-problems-python36 * Fix AnnData ref in merge * Copy object when subsetting in preprocess_logCPM_1kHVG() * Move PCA to dimred methods * Use preprocess_logCPM_1kHVG() in nn_ranking metrics * Fix path in python36 dockerfile * Add test kwarg to neuralee_default method * Add check for n_var to preprocess_logCPM_1kHVG() Should fix tests that were failing due to scverse/scanpy#2230 * Store raw counts in NeuralEE default method * Update dimred README * Replace X_input with PCA in ivis dimred method * Refactor preprocess_logCPM_1kHVG() to log_cpm_hvg() * Remove ivis * pre-commit Co-authored-by: Ben DeMeo <bdemeo@mit.edu> Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com> Co-authored-by: Michal Klein <michal.klein@protonmail.com> Co-authored-by: michalk8 <46717574+michalk8@users.noreply.github.com> Co-authored-by: bendemeo <45370700+bendemeo@users.noreply.github.com> Co-authored-by: MalteDLuecken <m.d.luecken@gmail.com> Co-authored-by: Wesley Lewis <59123674+wes-lewis@users.noreply.github.com> Co-authored-by: Daniel Burkhardt <burkhardt.d.b@gmail.com> Co-authored-by: Scott Gigante <84813314+scottgigante-immunai@users.noreply.github.com> * Only cleanup AWS on success (#371) * only cleanup on success * pre-commit Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com> * Jamboree label_projection task (#313) * Add scvi-tools docker image * add scanvi * hvg command use 2000 * update scvi-tools version; use image * train size * scanvi mask test labels * move import * hvg on train only, fix hvg command * add scarches scanvi * use string labels in testing * enforce batch metadata in dataset * add batch metadata in pancreas random * use train adata for scarches * Add majority vote simple baseline * test_mode * use test instead of test mode, update contributing * update contributing guide * Added helper function to introduce label noise * Actually return data with label noise * Only introduce label noise on training data * Made a pancreas dataset with label nosie * Reformat docstring * Added reference to example label noise dataset in datasets __init__.py * Add cengen C elegans data loader (#2) * add CeNGEN C elegans neuron dataset * add CeNGEN C elegans dataset for global tasks and for label_projection task * fix lines being too long * Reformat cengen data loader * Create tabula_muris_senis.py Need dataframe containing sample information in './tabula_muris_senis_data_objects/tabula_muris_senis_data_objects.csv' load_tabula_muris_senis(method_list, organ_list) takes in methods and organs to extract data from and combines into one anndata object. If method_list or organ_list = None, do not filter based on that input. EX: load_tabula_muris_senis(method_list=['facs'], organ_list = None) returns all facs experiments for all organs in one anndata object. * pre-commit * Modify anndata in place in add_label_noise rather than copy * Added CSV file with tabula muris senis data links * Update tabula_muris_senis.py * Add random_labels baseline to label_projection task * Update tabula_muris_senis.py * Update tabula_muris_senis.py * pre-commit * Update tabula_muris_senis.py * pre-commit * fix missing labels at prediction time * Handle test flag through tests and docker, pass to methods * If test method run, use 1 max_epoch for scvi-tools * Use only 2 batches for sample dataset for label_projection * Remove zebrafish random dataset * Fix decorator dependency to <5.0.0 * Remove functools.wraps from docker decorator for test parameterization * Fix cengen missing batch info * Use functools.update_wrapper for docker test * Add batch to pancreas_random_label_noise * Make cengen test dataset have more cells per batch * Set span=0.8 for hvg call for scanvi_hvg methods * Set span=0.8 for HVG selection only in test mode for scvi * Revert "Handle test flag through tests and docker, pass to methods" This reverts commit 3b940c0. * Add test parameter to label proj baselines * Fix flake remove unused import * Revert "Remove zebrafish random dataset" This reverts commit 3915798. * Update scVI setup_anndata to new version * pre-commit * Reformat and rerun tests * Add code_url and code_version for baseline label proj methods * Fallback HVG flavor for label projection task * pre-commit * Fix unused import * Fix using highly_variable_genes * Pin scvi-tools to 0.15.5 * Unpin scvi-tools, pin jax==0.3.6, see optuna/optuna-examples#99 * Add scikit-misc as requirement for scvi docker * Pin jaxlib as well * pin jaxlib along with jax * Set paper_year to year of implementation * Set random zebrafish split to 0.8+0.2 * Add tabula_muris_senis_lung_random dataset to label_projection * pre-commit * Add tabula muris senis datasets csv * Fix loading tabula muris csv * pre-commit * Test loader for tabula muris senis Co-authored-by: adamgayoso <adamgayoso@users.noreply.github.com> Co-authored-by: Valentine Svensson <v@nxn.se> Co-authored-by: Eduardo Beltrame <munfred@brandeis.edu> Co-authored-by: atchen <81202350+atchen@users.noreply.github.com> Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com> Co-authored-by: Scott Gigante <84813314+scottgigante-immunai@users.noreply.github.com> * Run `test_benchmark` on a self-hosted runner (#373) * set up cirun * use ubuntu standard AMI * run nextflow on the self-hosted machine * add to CONTRIBUTING * update ami * install unzip * set up docker * install docker from curl * use t2.micro not nano * use custom AMI * pythonLocation * add scripts to path * larger disk size * new image again * chown for now * chmod 755 * fixed permissions * use tower workspace * test nextflow * try again * nextflow -q * redirect stderr * increase memory * cleanup * sudo install * name * try setting pythonpath * fix branch env * another fix * fix run name * typo * fix pythonpath: * don't use pushd * pass pythonpath * set nousersite * empty * sudo install * run attempt * revert temporary changes * cleanup * fix contributing * add instructions for tower * fix repo name * move ami setup into script * Import Olsson 2016 dataset for dimred task (#352) * Import Olsson 2016 dataset for dimred task * Fix path to Olsson dataset loader * Filter genes cells before subsetting Olsson data in test * Use highly expressed genes for test Olsson dataset Test dataset is now 700 genes by 300 cells (was 500 x 500) * Add ivis dimred method (#369) * add densMAP package to python-extras * pre-commit * Add Ivis method * Explicitly mention it's CPU implementation * Add forgotten import in __init__ * Remove redundant filtering * Move ivis inside the function * Make var names unique, add ivis[cpu] to README * Pin tensorflow version * Add NeuralEE skeleton * Implement method * added densmap and densne * Fix typo pytoch -> torch * pre-commit * remove densne * Add forgotten detach/cpu/numpy * formatting * pre-commit * formatting * formatting * pre-commit * formatting * formatting * formatting * pre-commit * formatting * umap-learn implementation * pre-commit * Add docker image * Add skeleton method * formatting * Implement method * Fix some small bugs * Add preprocessing * Change batch size to 1k cells for aff. matrix * Add new preprocessing * Add new preprocessing * Fix preprocessing * Fix preprocessing * pre-commit * updated template for PR with PR evaluation checks (#314) * Update alra.py (#304) * Update alra.py Fix pre-processing and transformation back into the original space * pre-commit * Update alra.py * make sure necessary methods are imported Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com> Co-authored-by: Daniel Burkhardt <burkhardt.d.b@gmail.com> * Add scanpy preprocessing to densmap dimred method * Rename preprocess_scanpy() to preprocess_logCPM_1kHVG() * Add preprocessing suffix to dimred methods * Subset object in preprocess_logCPM_1kHVG() * Use standard names for input * Add neuralee_logCPM_1kHVG method * Add densmap_pca method * Fix preprocess_logCPM_1kHVG() Now returns an AnnData rather than acting in place - Subsetting wasn't working in place Also set HVG flavor to "cell_ranger" * Add test argument to dimred methods * Move preprocess_logCPM_1kHVG() to tools.normalize * Change name in python-method-scvis Docker README * Rename openproblems-python-method-scvis container Now called open-problems-python36 * Fix AnnData ref in merge * Copy object when subsetting in preprocess_logCPM_1kHVG() * Move PCA to dimred methods * Use preprocess_logCPM_1kHVG() in nn_ranking metrics * Fix path in python36 dockerfile * Add test kwarg to neuralee_default method * Add check for n_var to preprocess_logCPM_1kHVG() Should fix tests that were failing due to scverse/scanpy#2230 * Store raw counts in NeuralEE default method * Update dimred README * Replace X_input with PCA in ivis dimred method * Refactor preprocess_logCPM_1kHVG() to log_cpm_hvg() * Re-add ivis Co-authored-by: Ben DeMeo <bdemeo@mit.edu> Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com> Co-authored-by: Michal Klein <michal.klein@protonmail.com> Co-authored-by: michalk8 <46717574+michalk8@users.noreply.github.com> Co-authored-by: bendemeo <45370700+bendemeo@users.noreply.github.com> Co-authored-by: MalteDLuecken <m.d.luecken@gmail.com> Co-authored-by: Wesley Lewis <59123674+wes-lewis@users.noreply.github.com> Co-authored-by: Daniel Burkhardt <burkhardt.d.b@gmail.com> Co-authored-by: Scott Gigante <84813314+scottgigante-immunai@users.noreply.github.com> * hotfix timeout-minutes (#374) * use branch of scprep to provide R traceback (#376) Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com> Co-authored-by: Scott Gigante <scottgigante@users.noreply.github.com> Co-authored-by: Daniel Strobl <50872326+danielStrobl@users.noreply.github.com> Co-authored-by: SingleCellOpenProblems <singlecellopenproblems@protonmail.com> Co-authored-by: Luke Zappia <lazappi@users.noreply.github.com> Co-authored-by: Ben DeMeo <bdemeo@mit.edu> Co-authored-by: Michal Klein <michal.klein@protonmail.com> Co-authored-by: michalk8 <46717574+michalk8@users.noreply.github.com> Co-authored-by: bendemeo <45370700+bendemeo@users.noreply.github.com> Co-authored-by: MalteDLuecken <m.d.luecken@gmail.com> Co-authored-by: Wesley Lewis <59123674+wes-lewis@users.noreply.github.com> Co-authored-by: Daniel Burkhardt <burkhardt.d.b@gmail.com> Co-authored-by: Nikolay Markov <nikolai.markov@icloud.com> Co-authored-by: adamgayoso <adamgayoso@users.noreply.github.com> Co-authored-by: Valentine Svensson <v@nxn.se> Co-authored-by: Eduardo Beltrame <munfred@brandeis.edu> Co-authored-by: atchen <81202350+atchen@users.noreply.github.com> * Install libgeos-dev (#377) * Label docker images based on build location (#351) * label docker images * fix syntax * Run benchmark only after unittests (#349) * run benchmark after unittests * always run cleanup * cleanup * If using GH actions image, test for git diff on dockerfile (#350) * if using gh actions image, test for git diff on dockerfile * allow empty tag for now * decode * if image doesn't exist, automatically github actions * fix quotes * fix parsing and committing of results on tag (#356) * Import SCOT (#333) * import SCOT * pre-commit * scran requires R * check that aligned spaces are finite * exclude unbalanced SCOT for now Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com> Co-authored-by: Scott Gigante <scottgigante@users.noreply.github.com> * fix coverage badge # ci skip (#358) * fix gh actions badge link # ci skip (#359) * store results in /tmp (#361) * Remove scot unbalanced (#360) * Fix benchmark commit (#362) * store results in /tmp * add skip_on_empty * class doesn't have skip on empty * remove scot altogether (#363) * Allow codecov to fail on forks * docker images separate PR (#354) * docker images separate PR * all R requirements in r_requirements.txt * move github r packages to requirements file * pre-commit Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com> * Ignore AWS warning and clean up s3 properly (#366) * ci cleanup * ignore aws batch warning * remove citeseq cbmc from DR (#367) Co-authored-by: Scott Gigante <scott.gigante@immunai.com> * Update benchmark results # ci skip (#368) Co-authored-by: SingleCellOpenProblems <singlecellopenproblems@protonmail.com> * Jamboree dimensionality reduction methods (#318) * add densMAP package to python-extras * pre-commit * Add Ivis method * Explicitly mention it's CPU implementation * Add forgotten import in __init__ * Remove redundant filtering * Move ivis inside the function * Make var names unique, add ivis[cpu] to README * Pin tensorflow version * Add NeuralEE skeleton * Implement method * added densmap and densne * Fix typo pytoch -> torch * pre-commit * remove densne * Add forgotten detach/cpu/numpy * formatting * pre-commit * formatting * formatting * pre-commit * formatting * formatting * formatting * pre-commit * formatting * umap-learn implementation * pre-commit * Add docker image * Add skeleton method * formatting * Implement method * Fix some small bugs * Add preprocessing * Change batch size to 1k cells for aff. matrix * Add new preprocessing * Add new preprocessing * Fix preprocessing * Fix preprocessing * pre-commit * updated template for PR with PR evaluation checks (#314) * Update alra.py (#304) * Update alra.py Fix pre-processing and transformation back into the original space * pre-commit * Update alra.py * make sure necessary methods are imported Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com> Co-authored-by: Daniel Burkhardt <burkhardt.d.b@gmail.com> * Add scanpy preprocessing to densmap dimred method * Rename preprocess_scanpy() to preprocess_logCPM_1kHVG() * Add preprocessing suffix to dimred methods * Subset object in preprocess_logCPM_1kHVG() * Use standard names for input * Add neuralee_logCPM_1kHVG method * Add densmap_pca method * Fix preprocess_logCPM_1kHVG() Now returns an AnnData rather than acting in place - Subsetting wasn't working in place Also set HVG flavor to "cell_ranger" * Add test argument to dimred methods * Move preprocess_logCPM_1kHVG() to tools.normalize * Change name in python-method-scvis Docker README * Rename openproblems-python-method-scvis container Now called open-problems-python36 * Fix AnnData ref in merge * Copy object when subsetting in preprocess_logCPM_1kHVG() * Move PCA to dimred methods * Use preprocess_logCPM_1kHVG() in nn_ranking metrics * Fix path in python36 dockerfile * Add test kwarg to neuralee_default method * Add check for n_var to preprocess_logCPM_1kHVG() Should fix tests that were failing due to scverse/scanpy#2230 * Store raw counts in NeuralEE default method * Update dimred README * Replace X_input with PCA in ivis dimred method * Refactor preprocess_logCPM_1kHVG() to log_cpm_hvg() * Remove ivis * pre-commit Co-authored-by: Ben DeMeo <bdemeo@mit.edu> Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com> Co-authored-by: Michal Klein <michal.klein@protonmail.com> Co-authored-by: michalk8 <46717574+michalk8@users.noreply.github.com> Co-authored-by: bendemeo <45370700+bendemeo@users.noreply.github.com> Co-authored-by: MalteDLuecken <m.d.luecken@gmail.com> Co-authored-by: Wesley Lewis <59123674+wes-lewis@users.noreply.github.com> Co-authored-by: Daniel Burkhardt <burkhardt.d.b@gmail.com> Co-authored-by: Scott Gigante <84813314+scottgigante-immunai@users.noreply.github.com> * Only cleanup AWS on success (#371) * only cleanup on success * pre-commit Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com> * Jamboree label_projection task (#313) * Add scvi-tools docker image * add scanvi * hvg command use 2000 * update scvi-tools version; use image * train size * scanvi mask test labels * move import * hvg on train only, fix hvg command * add scarches scanvi * use string labels in testing * enforce batch metadata in dataset * add batch metadata in pancreas random * use train adata for scarches * Add majority vote simple baseline * test_mode * use test instead of test mode, update contributing * update contributing guide * Added helper function to introduce label noise * Actually return data with label noise * Only introduce label noise on training data * Made a pancreas dataset with label nosie * Reformat docstring * Added reference to example label noise dataset in datasets __init__.py * Add cengen C elegans data loader (#2) * add CeNGEN C elegans neuron dataset * add CeNGEN C elegans dataset for global tasks and for label_projection task * fix lines being too long * Reformat cengen data loader * Create tabula_muris_senis.py Need dataframe containing sample information in './tabula_muris_senis_data_objects/tabula_muris_senis_data_objects.csv' load_tabula_muris_senis(method_list, organ_list) takes in methods and organs to extract data from and combines into one anndata object. If method_list or organ_list = None, do not filter based on that input. EX: load_tabula_muris_senis(method_list=['facs'], organ_list = None) returns all facs experiments for all organs in one anndata object. * pre-commit * Modify anndata in place in add_label_noise rather than copy * Added CSV file with tabula muris senis data links * Update tabula_muris_senis.py * Add random_labels baseline to label_projection task * Update tabula_muris_senis.py * Update tabula_muris_senis.py * pre-commit * Update tabula_muris_senis.py * pre-commit * fix missing labels at prediction time * Handle test flag through tests and docker, pass to methods * If test method run, use 1 max_epoch for scvi-tools * Use only 2 batches for sample dataset for label_projection * Remove zebrafish random dataset * Fix decorator dependency to <5.0.0 * Remove functools.wraps from docker decorator for test parameterization * Fix cengen missing batch info * Use functools.update_wrapper for docker test * Add batch to pancreas_random_label_noise * Make cengen test dataset have more cells per batch * Set span=0.8 for hvg call for scanvi_hvg methods * Set span=0.8 for HVG selection only in test mode for scvi * Revert "Handle test flag through tests and docker, pass to methods" This reverts commit 3b940c0. * Add test parameter to label proj baselines * Fix flake remove unused import * Revert "Remove zebrafish random dataset" This reverts commit 3915798. * Update scVI setup_anndata to new version * pre-commit * Reformat and rerun tests * Add code_url and code_version for baseline label proj methods * Fallback HVG flavor for label projection task * pre-commit * Fix unused import * Fix using highly_variable_genes * Pin scvi-tools to 0.15.5 * Unpin scvi-tools, pin jax==0.3.6, see optuna/optuna-examples#99 * Add scikit-misc as requirement for scvi docker * Pin jaxlib as well * pin jaxlib along with jax * Set paper_year to year of implementation * Set random zebrafish split to 0.8+0.2 * Add tabula_muris_senis_lung_random dataset to label_projection * pre-commit * Add tabula muris senis datasets csv * Fix loading tabula muris csv * pre-commit * Test loader for tabula muris senis Co-authored-by: adamgayoso <adamgayoso@users.noreply.github.com> Co-authored-by: Valentine Svensson <v@nxn.se> Co-authored-by: Eduardo Beltrame <munfred@brandeis.edu> Co-authored-by: atchen <81202350+atchen@users.noreply.github.com> Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com> Co-authored-by: Scott Gigante <84813314+scottgigante-immunai@users.noreply.github.com> * Run `test_benchmark` on a self-hosted runner (#373) * set up cirun * use ubuntu standard AMI * run nextflow on the self-hosted machine * add to CONTRIBUTING * update ami * install unzip * set up docker * install docker from curl * use t2.micro not nano * use custom AMI * pythonLocation * add scripts to path * larger disk size * new image again * chown for now * chmod 755 * fixed permissions * use tower workspace * test nextflow * try again * nextflow -q * redirect stderr * increase memory * cleanup * sudo install * name * try setting pythonpath * fix branch env * another fix * fix run name * typo * fix pythonpath: * don't use pushd * pass pythonpath * set nousersite * empty * sudo install * run attempt * revert temporary changes * cleanup * fix contributing * add instructions for tower * fix repo name * move ami setup into script * Import Olsson 2016 dataset for dimred task (#352) * Import Olsson 2016 dataset for dimred task * Fix path to Olsson dataset loader * Filter genes cells before subsetting Olsson data in test * Use highly expressed genes for test Olsson dataset Test dataset is now 700 genes by 300 cells (was 500 x 500) * Add ivis dimred method (#369) * add densMAP package to python-extras * pre-commit * Add Ivis method * Explicitly mention it's CPU implementation * Add forgotten import in __init__ * Remove redundant filtering * Move ivis inside the function * Make var names unique, add ivis[cpu] to README * Pin tensorflow version * Add NeuralEE skeleton * Implement method * added densmap and densne * Fix typo pytoch -> torch * pre-commit * remove densne * Add forgotten detach/cpu/numpy * formatting * pre-commit * formatting * formatting * pre-commit * formatting * formatting * formatting * pre-commit * formatting * umap-learn implementation * pre-commit * Add docker image * Add skeleton method * formatting * Implement method * Fix some small bugs * Add preprocessing * Change batch size to 1k cells for aff. matrix * Add new preprocessing * Add new preprocessing * Fix preprocessing * Fix preprocessing * pre-commit * updated template for PR with PR evaluation checks (#314) * Update alra.py (#304) * Update alra.py Fix pre-processing and transformation back into the original space * pre-commit * Update alra.py * make sure necessary methods are imported Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com> Co-authored-by: Daniel Burkhardt <burkhardt.d.b@gmail.com> * Add scanpy preprocessing to densmap dimred method * Rename preprocess_scanpy() to preprocess_logCPM_1kHVG() * Add preprocessing suffix to dimred methods * Subset object in preprocess_logCPM_1kHVG() * Use standard names for input * Add neuralee_logCPM_1kHVG method * Add densmap_pca method * Fix preprocess_logCPM_1kHVG() Now returns an AnnData rather than acting in place - Subsetting wasn't working in place Also set HVG flavor to "cell_ranger" * Add test argument to dimred methods * Move preprocess_logCPM_1kHVG() to tools.normalize * Change name in python-method-scvis Docker README * Rename openproblems-python-method-scvis container Now called open-problems-python36 * Fix AnnData ref in merge * Copy object when subsetting in preprocess_logCPM_1kHVG() * Move PCA to dimred methods * Use preprocess_logCPM_1kHVG() in nn_ranking metrics * Fix path in python36 dockerfile * Add test kwarg to neuralee_default method * Add check for n_var to preprocess_logCPM_1kHVG() Should fix tests that were failing due to scverse/scanpy#2230 * Store raw counts in NeuralEE default method * Update dimred README * Replace X_input with PCA in ivis dimred method * Refactor preprocess_logCPM_1kHVG() to log_cpm_hvg() * Re-add ivis Co-authored-by: Ben DeMeo <bdemeo@mit.edu> Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com> Co-authored-by: Michal Klein <michal.klein@protonmail.com> Co-authored-by: michalk8 <46717574+michalk8@users.noreply.github.com> Co-authored-by: bendemeo <45370700+bendemeo@users.noreply.github.com> Co-authored-by: MalteDLuecken <m.d.luecken@gmail.com> Co-authored-by: Wesley Lewis <59123674+wes-lewis@users.noreply.github.com> Co-authored-by: Daniel Burkhardt <burkhardt.d.b@gmail.com> Co-authored-by: Scott Gigante <84813314+scottgigante-immunai@users.noreply.github.com> * hotfix timeout-minutes (#374) * use branch of scprep to provide R traceback (#376) * Install libgeos-dev Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com> Co-authored-by: Scott Gigante <scottgigante@users.noreply.github.com> Co-authored-by: Daniel Strobl <50872326+danielStrobl@users.noreply.github.com> Co-authored-by: SingleCellOpenProblems <singlecellopenproblems@protonmail.com> Co-authored-by: Luke Zappia <lazappi@users.noreply.github.com> Co-authored-by: Ben DeMeo <bdemeo@mit.edu> Co-authored-by: Michal Klein <michal.klein@protonmail.com> Co-authored-by: michalk8 <46717574+michalk8@users.noreply.github.com> Co-authored-by: bendemeo <45370700+bendemeo@users.noreply.github.com> Co-authored-by: MalteDLuecken <m.d.luecken@gmail.com> Co-authored-by: Wesley Lewis <59123674+wes-lewis@users.noreply.github.com> Co-authored-by: Daniel Burkhardt <burkhardt.d.b@gmail.com> Co-authored-by: Nikolay Markov <nikolai.markov@icloud.com> Co-authored-by: adamgayoso <adamgayoso@users.noreply.github.com> Co-authored-by: Valentine Svensson <v@nxn.se> Co-authored-by: Eduardo Beltrame <munfred@brandeis.edu> Co-authored-by: atchen <81202350+atchen@users.noreply.github.com> * Install libgeos-dev * Update test_docker (#379) * Label docker images based on build location (#351) * label docker images * fix syntax * Run benchmark only after unittests (#349) * run benchmark after unittests * always run cleanup * cleanup * If using GH actions image, test for git diff on dockerfile (#350) * if using gh actions image, test for git diff on dockerfile * allow empty tag for now * decode * if image doesn't exist, automatically github actions * fix quotes * fix parsing and committing of results on tag (#356) * Import SCOT (#333) * import SCOT * pre-commit * scran requires R * check that aligned spaces are finite * exclude unbalanced SCOT for now Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com> Co-authored-by: Scott Gigante <scottgigante@users.noreply.github.com> * fix coverage badge # ci skip (#358) * fix gh actions badge link # ci skip (#359) * store results in /tmp (#361) * Remove scot unbalanced (#360) * Fix benchmark commit (#362) * store results in /tmp * add skip_on_empty * class doesn't have skip on empty * remove scot altogether (#363) * Allow codecov to fail on forks * docker images separate PR (#354) * docker images separate PR * all R requirements in r_requirements.txt * move github r packages to requirements file * pre-commit Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com> * Ignore AWS warning and clean up s3 properly (#366) * ci cleanup * ignore aws batch warning * remove citeseq cbmc from DR (#367) Co-authored-by: Scott Gigante <scott.gigante@immunai.com> * Update benchmark results # ci skip (#368) Co-authored-by: SingleCellOpenProblems <singlecellopenproblems@protonmail.com> * Jamboree dimensionality reduction methods (#318) * add densMAP package to python-extras * pre-commit * Add Ivis method * Explicitly mention it's CPU implementation * Add forgotten import in __init__ * Remove redundant filtering * Move ivis inside the function * Make var names unique, add ivis[cpu] to README * Pin tensorflow version * Add NeuralEE skeleton * Implement method * added densmap and densne * Fix typo pytoch -> torch * pre-commit * remove densne * Add forgotten detach/cpu/numpy * formatting * pre-commit * formatting * formatting * pre-commit * formatting * formatting * formatting * pre-commit * formatting * umap-learn implementation * pre-commit * Add docker image * Add skeleton method * formatting * Implement method * Fix some small bugs * Add preprocessing * Change batch size to 1k cells for aff. matrix * Add new preprocessing * Add new preprocessing * Fix preprocessing * Fix preprocessing * pre-commit * updated template for PR with PR evaluation checks (#314) * Update alra.py (#304) * Update alra.py Fix pre-processing and transformation back into the original space * pre-commit * Update alra.py * make sure necessary methods are imported Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com> Co-authored-by: Daniel Burkhardt <burkhardt.d.b@gmail.com> * Add scanpy preprocessing to densmap dimred method * Rename preprocess_scanpy() to preprocess_logCPM_1kHVG() * Add preprocessing suffix to dimred methods * Subset object in preprocess_logCPM_1kHVG() * Use standard names for input * Add neuralee_logCPM_1kHVG method * Add densmap_pca method * Fix preprocess_logCPM_1kHVG() Now returns an AnnData rather than acting in place - Subsetting wasn't working in place Also set HVG flavor to "cell_ranger" * Add test argument to dimred methods * Move preprocess_logCPM_1kHVG() to tools.normalize * Change name in python-method-scvis Docker README * Rename openproblems-python-method-scvis container Now called open-problems-python36 * Fix AnnData ref in merge * Copy object when subsetting in preprocess_logCPM_1kHVG() * Move PCA to dimred methods * Use preprocess_logCPM_1kHVG() in nn_ranking metrics * Fix path in python36 dockerfile * Add test kwarg to neuralee_default method * Add check for n_var to preprocess_logCPM_1kHVG() Should fix tests that were failing due to scverse/scanpy#2230 * Store raw counts in NeuralEE default method * Update dimred README * Replace X_input with PCA in ivis dimred method * Refactor preprocess_logCPM_1kHVG() to log_cpm_hvg() * Remove ivis * pre-commit Co-authored-by: Ben DeMeo <bdemeo@mit.edu> Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com> Co-authored-by: Michal Klein <michal.klein@protonmail.com> Co-authored-by: michalk8 <46717574+michalk8@users.noreply.github.com> Co-authored-by: bendemeo <45370700+bendemeo@users.noreply.github.com> Co-authored-by: MalteDLuecken <m.d.luecken@gmail.com> Co-authored-by: Wesley Lewis <59123674+wes-lewis@users.noreply.github.com> Co-authored-by: Daniel Burkhardt <burkhardt.d.b@gmail.com> Co-authored-by: Scott Gigante <84813314+scottgigante-immunai@users.noreply.github.com> * Only cleanup AWS on success (#371) * only cleanup on success * pre-commit Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com> * Jamboree label_projection task (#313) * Add scvi-tools docker image * add scanvi * hvg command use 2000 * update scvi-tools version; use image * train size * scanvi mask test labels * move import * hvg on train only, fix hvg command * add scarches scanvi * use string labels in testing * enforce batch metadata in dataset * add batch metadata in pancreas random * use train adata for scarches * Add majority vote simple baseline * test_mode * use test instead of test mode, update contributing * update contributing guide * Added helper function to introduce label noise * Actually return data with label noise * Only introduce label noise on training data * Made a pancreas dataset with label nosie * Reformat docstring * Added reference to example label noise dataset in datasets __init__.py * Add cengen C elegans data loader (#2) * add CeNGEN C elegans neuron dataset * add CeNGEN C elegans dataset for global tasks and for label_projection task * fix lines being too long * Reformat cengen data loader * Create tabula_muris_senis.py Need dataframe containing sample information in './tabula_muris_senis_data_objects/tabula_muris_senis_data_objects.csv' load_tabula_muris_senis(method_list, organ_list) takes in methods and organs to extract data from and combines into one anndata object. If method_list or organ_list = None, do not filter based on that input. EX: load_tabula_muris_senis(method_list=['facs'], organ_list = None) returns all facs experiments for all organs in one anndata object. * pre-commit * Modify anndata in place in add_label_noise rather than copy * Added CSV file with tabula muris senis data links * Update tabula_muris_senis.py * Add random_labels baseline to label_projection task * Update tabula_muris_senis.py * Update tabula_muris_senis.py * pre-commit * Update tabula_muris_senis.py * pre-commit * fix missing labels at prediction time * Handle test flag through tests and docker, pass to methods * If test method run, use 1 max_epoch for scvi-tools * Use only 2 batches for sample dataset for label_projection * Remove zebrafish random dataset * Fix decorator dependency to <5.0.0 * Remove functools.wraps from docker decorator for test parameterization * Fix cengen missing batch info * Use functools.update_wrapper for docker test * Add batch to pancreas_random_label_noise * Make cengen test dataset have more cells per batch * Set span=0.8 for hvg call for scanvi_hvg methods * Set span=0.8 for HVG selection only in test mode for scvi * Revert "Handle test flag through tests and docker, pass to methods" This reverts commit 3b940c0. * Add test parameter to label proj baselines * Fix flake remove unused import * Revert "Remove zebrafish random dataset" This reverts commit 3915798. * Update scVI setup_anndata to new version * pre-commit * Reformat and rerun tests * Add code_url and code_version for baseline label proj methods * Fallback HVG flavor for label projection task * pre-commit * Fix unused import * Fix using highly_variable_genes * Pin scvi-tools to 0.15.5 * Unpin scvi-tools, pin jax==0.3.6, see optuna/optuna-examples#99 * Add scikit-misc as requirement for scvi docker * Pin jaxlib as well * pin jaxlib along with jax * Set paper_year to year of implementation * Set random zebrafish split to 0.8+0.2 * Add tabula_muris_senis_lung_random dataset to label_projection * pre-commit * Add tabula muris senis datasets csv * Fix loading tabula muris csv * pre-commit * Test loader for tabula muris senis Co-authored-by: adamgayoso <adamgayoso@users.noreply.github.com> Co-authored-by: Valentine Svensson <v@nxn.se> Co-authored-by: Eduardo Beltrame <munfred@brandeis.edu> Co-authored-by: atchen <81202350+atchen@users.noreply.github.com> Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com> Co-authored-by: Scott Gigante <84813314+scottgigante-immunai@users.noreply.github.com> * Run `test_benchmark` on a self-hosted runner (#373) * set up cirun * use ubuntu standard AMI * run nextflow on the self-hosted machine * add to CONTRIBUTING * update ami * install unzip * set up docker * install docker from curl * use t2.micro not nano * use custom AMI * pythonLocation * add scripts to path * larger disk size * new image again * chown for now * chmod 755 * fixed permissions * use tower workspace * test nextflow * try again * nextflow -q * redirect stderr * increase memory * cleanup * sudo install * name * try setting pythonpath * fix branch env * another fix * fix run name * typo * fix pythonpath: * don't use pushd * pass pythonpath * set nousersite * empty * sudo install * run attempt * revert temporary changes * cleanup * fix contributing * add instructions for tower * fix repo name * move ami setup into script * Import Olsson 2016 dataset for dimred task (#352) * Import Olsson 2016 dataset for dimred task * Fix path to Olsson dataset loader * Filter genes cells before subsetting Olsson data in test * Use highly expressed genes for test Olsson dataset Test dataset is now 700 genes by 300 cells (was 500 x 500) * Add ivis dimred method (#369) * add densMAP package to python-extras * pre-commit * Add Ivis method * Explicitly mention it's CPU implementation * Add forgotten import in __init__ * Remove redundant filtering * Move ivis inside the function * Make var names unique, add ivis[cpu] to README * Pin tensorflow version * Add NeuralEE skeleton * Implement method * added densmap and densne * Fix typo pytoch -> torch * pre-commit * remove densne * Add forgotten detach/cpu/numpy * formatting * pre-commit * formatting * formatting * pre-commit * formatting * formatting * formatting * pre-commit * formatting * umap-learn implementation * pre-commit * Add docker image * Add skeleton method * formatting * Implement method * Fix some small bugs * Add preprocessing * Change batch size to 1k cells for aff. matrix * Add new preprocessing * Add new preprocessing * Fix preprocessing * Fix preprocessing * pre-commit * updated template for PR with PR evaluation checks (#314) * Update alra.py (#304) * Update alra.py Fix pre-processing and transformation back into the original space * pre-commit * Update alra.py * make sure necessary methods are imported Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com> Co-authored-by: Daniel Burkhardt <burkhardt.d.b@gmail.com> * Add scanpy preprocessing to densmap dimred method * Rename preprocess_scanpy() to preprocess_logCPM_1kHVG() * Add preprocessing suffix to dimred methods * Subset object in preprocess_logCPM_1kHVG() * Use standard names for input * Add neuralee_logCPM_1kHVG method * Add densmap_pca method * Fix preprocess_logCPM_1kHVG() Now returns an AnnData rather than acting in place - Subsetting wasn't working in place Also set HVG flavor to "cell_ranger" * Add test argument to dimred methods * Move preprocess_logCPM_1kHVG() to tools.normalize * Change name in python-method-scvis Docker README * Rename openproblems-python-method-scvis container Now called open-problems-python36 * Fix AnnData ref in merge * Copy object when subsetting in preprocess_logCPM_1kHVG() * Move PCA to dimred methods * Use preprocess_logCPM_1kHVG() in nn_ranking metrics * Fix path in python36 dockerfile * Add test kwarg to neuralee_default method * Add check for n_var to preprocess_logCPM_1kHVG() Should fix tests that were failing due to scverse/scanpy#2230 * Store raw counts in NeuralEE default method * Update dimred README * Replace X_input with PCA in ivis dimred method * Refactor preprocess_logCPM_1kHVG() to log_cpm_hvg() * Re-add ivis Co-authored-by: Ben DeMeo <bdemeo@mit.edu> Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com> Co-authored-by: Michal Klein <michal.klein@protonmail.com> Co-authored-by: michalk8 <46717574+michalk8@users.noreply.github.com> Co-authored-by: bendemeo <45370700+bendemeo@users.noreply.github.com> Co-authored-by: MalteDLuecken <m.d.luecken@gmail.com> Co-authored-by: Wesley Lewis <59123674+wes-lewis@users.noreply.github.com> Co-authored-by: Daniel Burkhardt <burkhardt.d.b@gmail.com> Co-authored-by: Scott Gigante <84813314+scottgigante-immunai@users.noreply.github.com> * hotfix timeout-minutes (#374) * use branch of scprep to provide R traceback (#376) Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com> Co-authored-by: Scott Gigante <scottgigante@users.noreply.github.com> Co-authored-by: Daniel Strobl <50872326+danielStrobl@users.noreply.github.com> Co-authored-by: SingleCellOpenProblems <singlecellopenproblems@protonmail.com> Co-authored-by: Luke Zappia <lazappi@users.noreply.github.com> Co-authored-by: Ben DeMeo <bdemeo@mit.edu> Co-authored-by: Michal Klein <michal.klein@protonmail.com> Co-authored-by: michalk8 <46717574+michalk8@users.noreply.github.com> Co-authored-by: bendemeo <45370700+bendemeo@users.noreply.github.com> Co-authored-by: MalteDLuecken <m.d.luecken@gmail.com> Co-authored-by: Wesley Lewis <59123674+wes-lewis@users.noreply.github.com> Co-authored-by: Daniel Burkhardt <burkhardt.d.b@gmail.com> Co-authored-by: Nikolay Markov <nikolai.markov@icloud.com> Co-authored-by: adamgayoso <adamgayoso@users.noreply.github.com> Co-authored-by: Valentine Svensson <v@nxn.se> Co-authored-by: Eduardo Beltrame <munfred@brandeis.edu> Co-authored-by: atchen <81202350+atchen@users.noreply.github.com> * clean up dockerfile Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com> Co-authored-by: Scott Gigante <scottgigante@users.noreply.github.com> Co-authored-by: Daniel Strobl <50872326+danielStrobl@users.noreply.github.com> Co-authored-by: SingleCellOpenProblems <singlecellopenproblems@protonmail.com> Co-authored-by: Luke Zappia <lazappi@users.noreply.github.com> Co-authored-by: Ben DeMeo <bdemeo@mit.edu> Co-authored-by: Michal Klein <michal.klein@protonmail.com> Co-authored-by: michalk8 <46717574+michalk8@users.noreply.github.com> Co-authored-by: bendemeo <45370700+bendemeo@users.noreply.github.com> Co-authored-by: MalteDLuecken <m.d.luecken@gmail.com> Co-authored-by: Wesley Lewis <59123674+wes-lewis@users.noreply.github.com> Co-authored-by: Daniel Burkhardt <burkhardt.d.b@gmail.com> Co-authored-by: Nikolay Markov <nikolai.markov@icloud.com> Co-authored-by: adamgayoso <adamgayoso@users.noreply.github.com> Co-authored-by: Valentine Svensson <v@nxn.se> Co-authored-by: Eduardo Beltrame <munfred@brandeis.edu> Co-authored-by: atchen <81202350+atchen@users.noreply.github.com>

* Fix rgeos install (#380) * label docker images * fix syntax * Delete run_benchmark.yml * Update from main (#378) * Label docker images based on build location (#351) * label docker images * fix syntax * Run benchmark only after unittests (#349) * run benchmark after unittests * always run cleanup * cleanup * If using GH actions image, test for git diff on dockerfile (#350) * if using gh actions image, test for git diff on dockerfile * allow empty tag for now * decode * if image doesn't exist, automatically github actions * fix quotes * fix parsing and committing of results on tag (#356) * Import SCOT (#333) * import SCOT * pre-commit * scran requires R * check that aligned spaces are finite * exclude unbalanced SCOT for now Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com> Co-authored-by: Scott Gigante <scottgigante@users.noreply.github.com> * fix coverage badge # ci skip (#358) * fix gh actions badge link # ci skip (#359) * store results in /tmp (#361) * Remove scot unbalanced (#360) * Fix benchmark commit (#362) * store results in /tmp * add skip_on_empty * class doesn't have skip on empty * remove scot altogether (#363) * Allow codecov to fail on forks * docker images separate PR (#354) * docker images separate PR * all R requirements in r_requirements.txt * move github r packages to requirements file * pre-commit Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com> * Ignore AWS warning and clean up s3 properly (#366) * ci cleanup * ignore aws batch warning * remove citeseq cbmc from DR (#367) Co-authored-by: Scott Gigante <scott.gigante@immunai.com> * Update benchmark results # ci skip (#368) Co-authored-by: SingleCellOpenProblems <singlecellopenproblems@protonmail.com> * Jamboree dimensionality reduction methods (#318) * add densMAP package to python-extras * pre-commit * Add Ivis method * Explicitly mention it's CPU implementation * Add forgotten import in __init__ * Remove redundant filtering * Move ivis inside the function * Make var names unique, add ivis[cpu] to README * Pin tensorflow version * Add NeuralEE skeleton * Implement method * added densmap and densne * Fix typo pytoch -> torch * pre-commit * remove densne * Add forgotten detach/cpu/numpy * formatting * pre-commit * formatting * formatting * pre-commit * formatting * formatting * formatting * pre-commit * formatting * umap-learn implementation * pre-commit * Add docker image * Add skeleton method * formatting * Implement method * Fix some small bugs * Add preprocessing * Change batch size to 1k cells for aff. matrix * Add new preprocessing * Add new preprocessing * Fix preprocessing * Fix preprocessing * pre-commit * updated template for PR with PR evaluation checks (#314) * Update alra.py (#304) * Update alra.py Fix pre-processing and transformation back into the original space * pre-commit * Update alra.py * make sure necessary methods are imported Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com> Co-authored-by: Daniel Burkhardt <burkhardt.d.b@gmail.com> * Add scanpy preprocessing to densmap dimred method * Rename preprocess_scanpy() to preprocess_logCPM_1kHVG() * Add preprocessing suffix to dimred methods * Subset object in preprocess_logCPM_1kHVG() * Use standard names for input * Add neuralee_logCPM_1kHVG method * Add densmap_pca method * Fix preprocess_logCPM_1kHVG() Now returns an AnnData rather than acting in place - Subsetting wasn't working in place Also set HVG flavor to "cell_ranger" * Add test argument to dimred methods * Move preprocess_logCPM_1kHVG() to tools.normalize * Change name in python-method-scvis Docker README * Rename openproblems-python-method-scvis container Now called open-problems-python36 * Fix AnnData ref in merge * Copy object when subsetting in preprocess_logCPM_1kHVG() * Move PCA to dimred methods * Use preprocess_logCPM_1kHVG() in nn_ranking metrics * Fix path in python36 dockerfile * Add test kwarg to neuralee_default method * Add check for n_var to preprocess_logCPM_1kHVG() Should fix tests that were failing due to scverse/scanpy#2230 * Store raw counts in NeuralEE default method * Update dimred README * Replace X_input with PCA in ivis dimred method * Refactor preprocess_logCPM_1kHVG() to log_cpm_hvg() * Remove ivis * pre-commit Co-authored-by: Ben DeMeo <bdemeo@mit.edu> Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com> Co-authored-by: Michal Klein <michal.klein@protonmail.com> Co-authored-by: michalk8 <46717574+michalk8@users.noreply.github.com> Co-authored-by: bendemeo <45370700+bendemeo@users.noreply.github.com> Co-authored-by: MalteDLuecken <m.d.luecken@gmail.com> Co-authored-by: Wesley Lewis <59123674+wes-lewis@users.noreply.github.com> Co-authored-by: Daniel Burkhardt <burkhardt.d.b@gmail.com> Co-authored-by: Scott Gigante <84813314+scottgigante-immunai@users.noreply.github.com> * Only cleanup AWS on success (#371) * only cleanup on success * pre-commit Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com> * Jamboree label_projection task (#313) * Add scvi-tools docker image * add scanvi * hvg command use 2000 * update scvi-tools version; use image * train size * scanvi mask test labels * move import * hvg on train only, fix hvg command * add scarches scanvi * use string labels in testing * enforce batch metadata in dataset * add batch metadata in pancreas random * use train adata for scarches * Add majority vote simple baseline * test_mode * use test instead of test mode, update contributing * update contributing guide * Added helper function to introduce label noise * Actually return data with label noise * Only introduce label noise on training data * Made a pancreas dataset with label nosie * Reformat docstring * Added reference to example label noise dataset in datasets __init__.py * Add cengen C elegans data loader (#2) * add CeNGEN C elegans neuron dataset * add CeNGEN C elegans dataset for global tasks and for label_projection task * fix lines being too long * Reformat cengen data loader * Create tabula_muris_senis.py Need dataframe containing sample information in './tabula_muris_senis_data_objects/tabula_muris_senis_data_objects.csv' load_tabula_muris_senis(method_list, organ_list) takes in methods and organs to extract data from and combines into one anndata object. If method_list or organ_list = None, do not filter based on that input. EX: load_tabula_muris_senis(method_list=['facs'], organ_list = None) returns all facs experiments for all organs in one anndata object. * pre-commit * Modify anndata in place in add_label_noise rather than copy * Added CSV file with tabula muris senis data links * Update tabula_muris_senis.py * Add random_labels baseline to label_projection task * Update tabula_muris_senis.py * Update tabula_muris_senis.py * pre-commit * Update tabula_muris_senis.py * pre-commit * fix missing labels at prediction time * Handle test flag through tests and docker, pass to methods * If test method run, use 1 max_epoch for scvi-tools * Use only 2 batches for sample dataset for label_projection * Remove zebrafish random dataset * Fix decorator dependency to <5.0.0 * Remove functools.wraps from docker decorator for test parameterization * Fix cengen missing batch info * Use functools.update_wrapper for docker test * Add batch to pancreas_random_label_noise * Make cengen test dataset have more cells per batch * Set span=0.8 for hvg call for scanvi_hvg methods * Set span=0.8 for HVG selection only in test mode for scvi * Revert "Handle test flag through tests and docker, pass to methods" This reverts commit 3b940c0. * Add test parameter to label proj baselines * Fix flake remove unused import * Revert "Remove zebrafish random dataset" This reverts commit 3915798. * Update scVI setup_anndata to new version * pre-commit * Reformat and rerun tests * Add code_url and code_version for baseline label proj methods * Fallback HVG flavor for label projection task * pre-commit * Fix unused import * Fix using highly_variable_genes * Pin scvi-tools to 0.15.5 * Unpin scvi-tools, pin jax==0.3.6, see optuna/optuna-examples#99 * Add scikit-misc as requirement for scvi docker * Pin jaxlib as well * pin jaxlib along with jax * Set paper_year to year of implementation * Set random zebrafish split to 0.8+0.2 * Add tabula_muris_senis_lung_random dataset to label_projection * pre-commit * Add tabula muris senis datasets csv * Fix loading tabula muris csv * pre-commit * Test loader for tabula muris senis Co-authored-by: adamgayoso <adamgayoso@users.noreply.github.com> Co-authored-by: Valentine Svensson <v@nxn.se> Co-authored-by: Eduardo Beltrame <munfred@brandeis.edu> Co-authored-by: atchen <81202350+atchen@users.noreply.github.com> Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com> Co-authored-by: Scott Gigante <84813314+scottgigante-immunai@users.noreply.github.com> * Run `test_benchmark` on a self-hosted runner (#373) * set up cirun * use ubuntu standard AMI * run nextflow on the self-hosted machine * add to CONTRIBUTING * update ami * install unzip * set up docker * install docker from curl * use t2.micro not nano * use custom AMI * pythonLocation * add scripts to path * larger disk size * new image again * chown for now * chmod 755 * fixed permissions * use tower workspace * test nextflow * try again * nextflow -q * redirect stderr * increase memory * cleanup * sudo install * name * try setting pythonpath * fix branch env * another fix * fix run name * typo * fix pythonpath: * don't use pushd * pass pythonpath * set nousersite * empty * sudo install * run attempt * revert temporary changes * cleanup * fix contributing * add instructions for tower * fix repo name * move ami setup into script * Import Olsson 2016 dataset for dimred task (#352) * Import Olsson 2016 dataset for dimred task * Fix path to Olsson dataset loader * Filter genes cells before subsetting Olsson data in test * Use highly expressed genes for test Olsson dataset Test dataset is now 700 genes by 300 cells (was 500 x 500) * Add ivis dimred method (#369) * add densMAP package to python-extras * pre-commit * Add Ivis method * Explicitly mention it's CPU implementation * Add forgotten import in __init__ * Remove redundant filtering * Move ivis inside the function * Make var names unique, add ivis[cpu] to README * Pin tensorflow version * Add NeuralEE skeleton * Implement method * added densmap and densne * Fix typo pytoch -> torch * pre-commit * remove densne * Add forgotten detach/cpu/numpy * formatting * pre-commit * formatting * formatting * pre-commit * formatting * formatting * formatting * pre-commit * formatting * umap-learn implementation * pre-commit * Add docker image * Add skeleton method * formatting * Implement method * Fix some small bugs * Add preprocessing * Change batch size to 1k cells for aff. matrix * Add new preprocessing * Add new preprocessing * Fix preprocessing * Fix preprocessing * pre-commit * updated template for PR with PR evaluation checks (#314) * Update alra.py (#304) * Update alra.py Fix pre-processing and transformation back into the original space * pre-commit * Update alra.py * make sure necessary methods are imported Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com> Co-authored-by: Daniel Burkhardt <burkhardt.d.b@gmail.com> * Add scanpy preprocessing to densmap dimred method * Rename preprocess_scanpy() to preprocess_logCPM_1kHVG() * Add preprocessing suffix to dimred methods * Subset object in preprocess_logCPM_1kHVG() * Use standard names for input * Add neuralee_logCPM_1kHVG method * Add densmap_pca method * Fix preprocess_logCPM_1kHVG() Now returns an AnnData rather than acting in place - Subsetting wasn't working in place Also set HVG flavor to "cell_ranger" * Add test argument to dimred methods * Move preprocess_logCPM_1kHVG() to tools.normalize * Change name in python-method-scvis Docker README * Rename openproblems-python-method-scvis container Now called open-problems-python36 * Fix AnnData ref in merge * Copy object when subsetting in preprocess_logCPM_1kHVG() * Move PCA to dimred methods * Use preprocess_logCPM_1kHVG() in nn_ranking metrics * Fix path in python36 dockerfile * Add test kwarg to neuralee_default method * Add check for n_var to preprocess_logCPM_1kHVG() Should fix tests that were failing due to scverse/scanpy#2230 * Store raw counts in NeuralEE default method * Update dimred README * Replace X_input with PCA in ivis dimred method * Refactor preprocess_logCPM_1kHVG() to log_cpm_hvg() * Re-add ivis Co-authored-by: Ben DeMeo <bdemeo@mit.edu> Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com> Co-authored-by: Michal Klein <michal.klein@protonmail.com> Co-authored-by: michalk8 <46717574+michalk8@users.noreply.github.com> Co-authored-by: bendemeo <45370700+bendemeo@users.noreply.github.com> Co-authored-by: MalteDLuecken <m.d.luecken@gmail.com> Co-authored-by: Wesley Lewis <59123674+wes-lewis@users.noreply.github.com> Co-authored-by: Daniel Burkhardt <burkhardt.d.b@gmail.com> Co-authored-by: Scott Gigante <84813314+scottgigante-immunai@users.noreply.github.com> * hotfix timeout-minutes (#374) * use branch of scprep to provide R traceback (#376) Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com> Co-authored-by: Scott Gigante <scottgigante@users.noreply.github.com> Co-authored-by: Daniel Strobl <50872326+danielStrobl@users.noreply.github.com> Co-authored-by: SingleCellOpenProblems <singlecellopenproblems@protonmail.com> Co-authored-by: Luke Zappia <lazappi@users.noreply.github.com> Co-authored-by: Ben DeMeo <bdemeo@mit.edu> Co-authored-by: Michal Klein <michal.klein@protonmail.com> Co-authored-by: michalk8 <46717574+michalk8@users.noreply.github.com> Co-authored-by: bendemeo <45370700+bendemeo@users.noreply.github.com> Co-authored-by: MalteDLuecken <m.d.luecken@gmail.com> Co-authored-by: Wesley Lewis <59123674+wes-lewis@users.noreply.github.com> Co-authored-by: Daniel Burkhardt <burkhardt.d.b@gmail.com> Co-authored-by: Nikolay Markov <nikolai.markov@icloud.com> Co-authored-by: adamgayoso <adamgayoso@users.noreply.github.com> Co-authored-by: Valentine Svensson <v@nxn.se> Co-authored-by: Eduardo Beltrame <munfred@brandeis.edu> Co-authored-by: atchen <81202350+atchen@users.noreply.github.com> * Install libgeos-dev (#377) * Label docker images based on build location (#351) * label docker images * fix syntax * Run benchmark only after unittests (#349) * run benchmark after unittests * always run cleanup * cleanup * If using GH actions image, test for git diff on dockerfile (#350) * if using gh actions image, test for git diff on dockerfile * allow empty tag for now * decode * if image doesn't exist, automatically github actions * fix quotes * fix parsing and committing of results on tag (#356) * Import SCOT (#333) * import SCOT * pre-commit * scran requires R * check that aligned spaces are finite * exclude unbalanced SCOT for now Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com> Co-authored-by: Scott Gigante <scottgigante@users.noreply.github.com> * fix coverage badge # ci skip (#358) * fix gh actions badge link # ci skip (#359) * store results in /tmp (#361) * Remove scot unbalanced (#360) * Fix benchmark commit (#362) * store results in /tmp * add skip_on_empty * class doesn't have skip on empty * remove scot altogether (#363) * Allow codecov to fail on forks * docker images separate PR (#354) * docker images separate PR * all R requirements in r_requirements.txt * move github r packages to requirements file * pre-commit Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com> * Ignore AWS warning and clean up s3 properly (#366) * ci cleanup * ignore aws batch warning * remove citeseq cbmc from DR (#367) Co-authored-by: Scott Gigante <scott.gigante@immunai.com> * Update benchmark results # ci skip (#368) Co-authored-by: SingleCellOpenProblems <singlecellopenproblems@protonmail.com> * Jamboree dimensionality reduction methods (#318) * add densMAP package to python-extras * pre-commit * Add Ivis method * Explicitly mention it's CPU implementation * Add forgotten import in __init__ * Remove redundant filtering * Move ivis inside the function * Make var names unique, add ivis[cpu] to README * Pin tensorflow version * Add NeuralEE skeleton * Implement method * added densmap and densne * Fix typo pytoch -> torch * pre-commit * remove densne * Add forgotten detach/cpu/numpy * formatting * pre-commit * formatting * formatting * pre-commit * formatting * formatting * formatting * pre-commit * formatting * umap-learn implementation * pre-commit * Add docker image * Add skeleton method * formatting * Implement method * Fix some small bugs * Add preprocessing * Change batch size to 1k cells for aff. matrix * Add new preprocessing * Add new preprocessing * Fix preprocessing * Fix preprocessing * pre-commit * updated template for PR with PR evaluation checks (#314) * Update alra.py (#304) * Update alra.py Fix pre-processing and transformation back into the original space * pre-commit * Update alra.py * make sure necessary methods are imported Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com> Co-authored-by: Daniel Burkhardt <burkhardt.d.b@gmail.com> * Add scanpy preprocessing to densmap dimred method * Rename preprocess_scanpy() to preprocess_logCPM_1kHVG() * Add preprocessing suffix to dimred methods * Subset object in preprocess_logCPM_1kHVG() * Use standard names for input * Add neuralee_logCPM_1kHVG method * Add densmap_pca method * Fix preprocess_logCPM_1kHVG() Now returns an AnnData rather than acting in place - Subsetting wasn't working in place Also set HVG flavor to "cell_ranger" * Add test argument to dimred methods * Move preprocess_logCPM_1kHVG() to tools.normalize * Change name in python-method-scvis Docker README * Rename openproblems-python-method-scvis container Now called open-problems-python36 * Fix AnnData ref in merge * Copy object when subsetting in preprocess_logCPM_1kHVG() * Move PCA to dimred methods * Use preprocess_logCPM_1kHVG() in nn_ranking metrics * Fix path in python36 dockerfile * Add test kwarg to neuralee_default method * Add check for n_var to preprocess_logCPM_1kHVG() Should fix tests that were failing due to scverse/scanpy#2230 * Store raw counts in NeuralEE default method * Update dimred README * Replace X_input with PCA in ivis dimred method * Refactor preprocess_logCPM_1kHVG() to log_cpm_hvg() * Remove ivis * pre-commit Co-authored-by: Ben DeMeo <bdemeo@mit.edu> Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com> Co-authored-by: Michal Klein <michal.klein@protonmail.com> Co-authored-by: michalk8 <46717574+michalk8@users.noreply.github.com> Co-authored-by: bendemeo <45370700+bendemeo@users.noreply.github.com> Co-authored-by: MalteDLuecken <m.d.luecken@gmail.com> Co-authored-by: Wesley Lewis <59123674+wes-lewis@users.noreply.github.com> Co-authored-by: Daniel Burkhardt <burkhardt.d.b@gmail.com> Co-authored-by: Scott Gigante <84813314+scottgigante-immunai@users.noreply.github.com> * Only cleanup AWS on success (#371) * only cleanup on success * pre-commit Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com> * Jamboree label_projection task (#313) * Add scvi-tools docker image * add scanvi * hvg command use 2000 * update scvi-tools version; use image * train size * scanvi mask test labels * move import * hvg on train only, fix hvg command * add scarches scanvi * use string labels in testing * enforce batch metadata in dataset * add batch metadata in pancreas random * use train adata for scarches * Add majority vote simple baseline * test_mode * use test instead of test mode, update contributing * update contributing guide * Added helper function to introduce label noise * Actually return data with label noise * Only introduce label noise on training data * Made a pancreas dataset with label nosie * Reformat docstring * Added reference to example label noise dataset in datasets __init__.py * Add cengen C elegans data loader (#2) * add CeNGEN C elegans neuron dataset * add CeNGEN C elegans dataset for global tasks and for label_projection task * fix lines being too long * Reformat cengen data loader * Create tabula_muris_senis.py Need dataframe containing sample information in './tabula_muris_senis_data_objects/tabula_muris_senis_data_objects.csv' load_tabula_muris_senis(method_list, organ_list) takes in methods and organs to extract data from and combines into one anndata object. If method_list or organ_list = None, do not filter based on that input. EX: load_tabula_muris_senis(method_list=['facs'], organ_list = None) returns all facs experiments for all organs in one anndata object. * pre-commit * Modify anndata in place in add_label_noise rather than copy * Added CSV file with tabula muris senis data links * Update tabula_muris_senis.py * Add random_labels baseline to label_projection task * Update tabula_muris_senis.py * Update tabula_muris_senis.py * pre-commit * Update tabula_muris_senis.py * pre-commit * fix missing labels at prediction time * Handle test flag through tests and docker, pass to methods * If test method run, use 1 max_epoch for scvi-tools * Use only 2 batches for sample dataset for label_projection * Remove zebrafish random dataset * Fix decorator dependency to <5.0.0 * Remove functools.wraps from docker decorator for test parameterization * Fix cengen missing batch info * Use functools.update_wrapper for docker test * Add batch to pancreas_random_label_noise * Make cengen test dataset have more cells per batch * Set span=0.8 for hvg call for scanvi_hvg methods * Set span=0.8 for HVG selection only in test mode for scvi * Revert "Handle test flag through tests and docker, pass to methods" This reverts commit 3b940c0. * Add test parameter to label proj baselines * Fix flake remove unused import * Revert "Remove zebrafish random dataset" This reverts commit 3915798. * Update scVI setup_anndata to new version * pre-commit * Reformat and rerun tests * Add code_url and code_version for baseline label proj methods * Fallback HVG flavor for label projection task * pre-commit * Fix unused import * Fix using highly_variable_genes * Pin scvi-tools to 0.15.5 * Unpin scvi-tools, pin jax==0.3.6, see optuna/optuna-examples#99 * Add scikit-misc as requirement for scvi docker * Pin jaxlib as well * pin jaxlib along with jax * Set paper_year to year of implementation * Set random zebrafish split to 0.8+0.2 * Add tabula_muris_senis_lung_random dataset to label_projection * pre-commit * Add tabula muris senis datasets csv * Fix loading tabula muris csv * pre-commit * Test loader for tabula muris senis Co-authored-by: adamgayoso <adamgayoso@users.noreply.github.com> Co-authored-by: Valentine Svensson <v@nxn.se> Co-authored-by: Eduardo Beltrame <munfred@brandeis.edu> Co-authored-by: atchen <81202350+atchen@users.noreply.github.com> Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com> Co-authored-by: Scott Gigante <84813314+scottgigante-immunai@users.noreply.github.com> * Run `test_benchmark` on a self-hosted runner (#373) * set up cirun * use ubuntu standard AMI * run nextflow on the self-hosted machine * add to CONTRIBUTING * update ami * install unzip * set up docker * install docker from curl * use t2.micro not nano * use custom AMI * pythonLocation * add scripts to path * larger disk size * new image again * chown for now * chmod 755 * fixed permissions * use tower workspace * test nextflow * try again * nextflow -q * redirect stderr * increase memory * cleanup * sudo install * name * try setting pythonpath * fix branch env * another fix * fix run name * typo * fix pythonpath: * don't use pushd * pass pythonpath * set nousersite * empty * sudo install * run attempt * revert temporary changes * cleanup * fix contributing * add instructions for tower * fix repo name * move ami setup into script * Import Olsson 2016 dataset for dimred task (#352) * Import Olsson 2016 dataset for dimred task * Fix path to Olsson dataset loader * Filter genes cells before subsetting Olsson data in test * Use highly expressed genes for test Olsson dataset Test dataset is now 700 genes by 300 cells (was 500 x 500) * Add ivis dimred method (#369) * add densMAP package to python-extras * pre-commit * Add Ivis method * Explicitly mention it's CPU implementation * Add forgotten import in __init__ * Remove redundant filtering * Move ivis inside the function * Make var names unique, add ivis[cpu] to README * Pin tensorflow version * Add NeuralEE skeleton * Implement method * added densmap and densne * Fix typo pytoch -> torch * pre-commit * remove densne * Add forgotten detach/cpu/numpy * formatting * pre-commit * formatting * formatting * pre-commit * formatting * formatting * formatting * pre-commit * formatting * umap-learn implementation * pre-commit * Add docker image * Add skeleton method * formatting * Implement method * Fix some small bugs * Add preprocessing * Change batch size to 1k cells for aff. matrix * Add new preprocessing * Add new preprocessing * Fix preprocessing * Fix preprocessing * pre-commit * updated template for PR with PR evaluation checks (#314) * Update alra.py (#304) * Update alra.py Fix pre-processing and transformation back into the original space * pre-commit * Update alra.py * make sure necessary methods are imported Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com> Co-authored-by: Daniel Burkhardt <burkhardt.d.b@gmail.com> * Add scanpy preprocessing to densmap dimred method * Rename preprocess_scanpy() to preprocess_logCPM_1kHVG() * Add preprocessing suffix to dimred methods * Subset object in preprocess_logCPM_1kHVG() * Use standard names for input * Add neuralee_logCPM_1kHVG method * Add densmap_pca method * Fix preprocess_logCPM_1kHVG() Now returns an AnnData rather than acting in place - Subsetting wasn't working in place Also set HVG flavor to "cell_ranger" * Add test argument to dimred methods * Move preprocess_logCPM_1kHVG() to tools.normalize * Change name in python-method-scvis Docker README * Rename openproblems-python-method-scvis container Now called open-problems-python36 * Fix AnnData ref in merge * Copy object when subsetting in preprocess_logCPM_1kHVG() * Move PCA to dimred methods * Use preprocess_logCPM_1kHVG() in nn_ranking metrics * Fix path in python36 dockerfile * Add test kwarg to neuralee_default method * Add check for n_var to preprocess_logCPM_1kHVG() Should fix tests that were failing due to scverse/scanpy#2230 * Store raw counts in NeuralEE default method * Update dimred README * Replace X_input with PCA in ivis dimred method * Refactor preprocess_logCPM_1kHVG() to log_cpm_hvg() * Re-add ivis Co-authored-by: Ben DeMeo <bdemeo@mit.edu> Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com> Co-authored-by: Michal Klein <michal.klein@protonmail.com> Co-authored-by: michalk8 <46717574+michalk8@users.noreply.github.com> Co-authored-by: bendemeo <45370700+bendemeo@users.noreply.github.com> Co-authored-by: MalteDLuecken <m.d.luecken@gmail.com> Co-authored-by: Wesley Lewis <59123674+wes-lewis@users.noreply.github.com> Co-authored-by: Daniel Burkhardt <burkhardt.d.b@gmail.com> Co-authored-by: Scott Gigante <84813314+scottgigante-immunai@users.noreply.github.com> * hotfix timeout-minutes (#374) * use branch of scprep to provide R traceback (#376) * Install libgeos-dev Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com> Co-authored-by: Scott Gigante <scottgigante@users.noreply.github.com> Co-authored-by: Daniel Strobl <50872326+danielStrobl@users.noreply.github.com> Co-authored-by: SingleCellOpenProblems <singlecellopenproblems@protonmail.com> Co-authored-by: Luke Zappia <lazappi@users.noreply.github.com> Co-authored-by: Ben DeMeo <bdemeo@mit.edu> Co-authored-by: Michal Klein <michal.klein@protonmail.com> Co-authored-by: michalk8 <46717574+michalk8@users.noreply.github.com> Co-authored-by: bendemeo <45370700+bendemeo@users.noreply.github.com> Co-authored-by: MalteDLuecken <m.d.luecken@gmail.com> Co-authored-by: Wesley Lewis <59123674+wes-lewis@users.noreply.github.com> Co-authored-by: Daniel Burkhardt <burkhardt.d.b@gmail.com> Co-authored-by: Nikolay Markov <nikolai.markov@icloud.com> Co-authored-by: adamgayoso <adamgayoso@users.noreply.github.com> Co-authored-by: Valentine Svensson <v@nxn.se> Co-authored-by: Eduardo Beltrame <munfred@brandeis.edu> Co-authored-by: atchen <81202350+atchen@users.noreply.github.com> * Install libgeos-dev * Update test_docker (#379) * Label docker images based on build location (#351) * label docker images * fix syntax * Run benchmark only after unittests (#349) * run benchmark after unittests * always run cleanup * cleanup * If using GH actions image, test for git diff on dockerfile (#350) * if using gh actions image, test for git diff on dockerfile * allow empty tag for now * decode * if image doesn't exist, automatically github actions * fix quotes * fix parsing and committing of results on tag (#356) * Import SCOT (#333) * import SCOT * pre-commit * scran requires R * check that aligned spaces are finite * exclude unbalanced SCOT for now Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com> Co-authored-by: Scott Gigante <scottgigante@users.noreply.github.com> * fix coverage badge # ci skip (#358) * fix gh actions badge link # ci skip (#359) * store results in /tmp (#361) * Remove scot unbalanced (#360) * Fix benchmark commit (#362) * store results in /tmp * add skip_on_empty * class doesn't have skip on empty * remove scot altogether (#363) * Allow codecov to fail on forks * docker images separate PR (#354) * docker images separate PR * all R requirements in r_requirements.txt * move github r packages to requirements file * pre-commit Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com> * Ignore AWS warning and clean up s3 properly (#366) * ci cleanup * ignore aws batch warning * remove citeseq cbmc from DR (#367) Co-authored-by: Scott Gigante <scott.gigante@immunai.com> * Update benchmark results # ci skip (#368) Co-authored-by: SingleCellOpenProblems <singlecellopenproblems@protonmail.com> * Jamboree dimensionality reduction methods (#318) * add densMAP package to python-extras * pre-commit * Add Ivis method * Explicitly mention it's CPU implementation * Add forgotten import in __init__ * Remove redundant filtering * Move ivis inside the function * Make var names unique, add ivis[cpu] to README * Pin tensorflow version * Add NeuralEE skeleton * Implement method * added densmap and densne * Fix typo pytoch -> torch * pre-commit * remove densne * Add forgotten detach/cpu/numpy * formatting * pre-commit * formatting * formatting * pre-commit * formatting * formatting * formatting * pre-commit * formatting * umap-learn implementation * pre-commit * Add docker image * Add skeleton method * formatting * Implement method * Fix some small bugs * Add preprocessing * Change batch size to 1k cells for aff. matrix * Add new preprocessing * Add new preprocessing * Fix preprocessing * Fix preprocessing * pre-commit * updated template for PR with PR evaluation checks (#314) * Update alra.py (#304) * Update alra.py Fix pre-processing and transformation back into the original space * pre-commit * Update alra.py * make sure necessary methods are imported Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com> Co-authored-by: Daniel Burkhardt <burkhardt.d.b@gmail.com> * Add scanpy preprocessing to densmap dimred method * Rename preprocess_scanpy() to preprocess_logCPM_1kHVG() * Add preprocessing suffix to dimred methods * Subset object in preprocess_logCPM_1kHVG() * Use standard names for input * Add neuralee_logCPM_1kHVG method * Add densmap_pca method * Fix preprocess_logCPM_1kHVG() Now returns an AnnData rather than acting in place - Subsetting wasn't working in place Also set HVG flavor to "cell_ranger" * Add test argument to dimred methods * Move preprocess_logCPM_1kHVG() to tools.normalize * Change name in python-method-scvis Docker README * Rename openproblems-python-method-scvis container Now called open-problems-python36 * Fix AnnData ref in merge * Copy object when subsetting in preprocess_logCPM_1kHVG() * Move PCA to dimred methods * Use preprocess_logCPM_1kHVG() in nn_ranking metrics * Fix path in python36 dockerfile * Add test kwarg to neuralee_default method * Add check for n_var to preprocess_logCPM_1kHVG() Should fix tests that were failing due to scverse/scanpy#2230 * Store raw counts in NeuralEE default method * Update dimred README * Replace X_input with PCA in ivis dimred method * Refactor preprocess_logCPM_1kHVG() to log_cpm_hvg() * Remove ivis * pre-commit Co-authored-by: Ben DeMeo <bdemeo@mit.edu> Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com> Co-authored-by: Michal Klein <michal.klein@protonmail.com> Co-authored-by: michalk8 <46717574+michalk8@users.noreply.github.com> Co-authored-by: bendemeo <45370700+bendemeo@users.noreply.github.com> Co-authored-by: MalteDLuecken <m.d.luecken@gmail.com> Co-authored-by: Wesley Lewis <59123674+wes-lewis@users.noreply.github.com> Co-authored-by: Daniel Burkhardt <burkhardt.d.b@gmail.com> Co-authored-by: Scott Gigante <84813314+scottgigante-immunai@users.noreply.github.com> * Only cleanup AWS on success (#371) * only cleanup on success * pre-commit Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com> * Jamboree label_projection task (#313) * Add scvi-tools docker image * add scanvi * hvg command use 2000 * update scvi-tools version; use image * train size * scanvi mask test labels * move import * hvg on train only, fix hvg command * add scarches scanvi * use string labels in testing * enforce batch metadata in dataset * add batch metadata in pancreas random * use train adata for scarches * Add majority vote simple baseline * test_mode * use test instead of test mode, update contributing * update contributing guide * Added helper function to introduce label noise * Actually return data with label noise * Only introduce label noise on training data * Made a pancreas dataset with label nosie * Reformat docstring * Added reference to example label noise dataset in datasets __init__.py * Add cengen C elegans data loader (#2) * add CeNGEN C elegans neuron dataset * add CeNGEN C elegans dataset for global tasks and for label_projection task * fix lines being too long * Reformat cengen data loader * Create tabula_muris_senis.py Need dataframe containing sample information in './tabula_muris_senis_data_objects/tabula_muris_senis_data_objects.csv' load_tabula_muris_senis(method_list, organ_list) takes in methods and organs to extract data from and combines into one anndata object. If method_list or organ_list = None, do not filter based on that input. EX: load_tabula_muris_senis(method_list=['facs'], organ_list = None) returns all facs experiments for all organs in one anndata object. * pre-commit * Modify anndata in place in add_label_noise rather than copy * Added CSV file with tabula muris senis data links * Update tabula_muris_senis.py * Add random_labels baseline to label_projection task * Update tabula_muris_senis.py * Update tabula_muris_senis.py * pre-commit * Update tabula_muris_senis.py * pre-commit * fix missing labels at prediction time * Handle test flag through tests and docker, pass to methods * If test method run, use 1 max_epoch for scvi-tools * Use only 2 batches for sample dataset for label_projection * Remove zebrafish random dataset * Fix decorator dependency to <5.0.0 * Remove functools.wraps from docker decorator for test parameterization * Fix cengen missing batch info * Use functools.update_wrapper for docker test * Add batch to pancreas_random_label_noise * Make cengen test dataset have more cells per batch * Set span=0.8 for hvg call for scanvi_hvg methods * Set span=0.8 for HVG selection only in test mode for scvi * Revert "Handle test flag through tests and docker, pass to methods" This reverts commit 3b940c0. * Add test parameter to label proj baselines * Fix flake remove unused import * Revert "Remove zebrafish random dataset" This reverts commit 3915798. * Update scVI setup_anndata to new version * pre-commit * Reformat and rerun tests * Add code_url and code_version for baseline label proj methods * Fallback HVG flavor for label projection task * pre-commit * Fix unused import * Fix using highly_variable_genes * Pin scvi-tools to 0.15.5 * Unpin scvi-tools, pin jax==0.3.6, see optuna/optuna-examples#99 * Add scikit-misc as requirement for scvi docker * Pin jaxlib as well * pin jaxlib along with jax * Set paper_year to year of implementation * Set random zebrafish split to 0.8+0.2 * Add tabula_muris_senis_lung_random dataset to label_projection * pre-commit * Add tabula muris senis datasets csv * Fix loading tabula muris csv * pre-commit * Test loader for tabula muris senis Co-authored-by: adamgayoso <adamgayoso@users.noreply.github.com> Co-authored-by: Valentine Svensson <v@nxn.se> Co-authored-by: Eduardo Beltrame <munfred@brandeis.edu> Co-authored-by: atchen <81202350+atchen@users.noreply.github.com> Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com> Co-authored-by: Scott Gigante <84813314+scottgigante-immunai@users.noreply.github.com> * Run `test_benchmark` on a self-hosted runner (#373) * set up cirun * use ubuntu standard AMI * run nextflow on the self-hosted machine * add to CONTRIBUTING * update ami * install unzip * set up docker * install docker from curl * use t2.micro not nano * use custom AMI * pythonLocation * add scripts to path * larger disk size * new image again * chown for now * chmod 755 * fixed permissions * use tower workspace * test nextflow * try again * nextflow -q * redirect stderr * increase memory * cleanup * sudo install * name * try setting pythonpath * fix branch env * another fix * fix run name * typo * fix pythonpath: * don't use pushd * pass pythonpath * set nousersite * empty * sudo install * run attempt * revert temporary changes * cleanup * fix contributing * add instructions for tower * fix repo name * move ami setup into script * Import Olsson 2016 dataset for dimred task (#352) * Import Olsson 2016 dataset for dimred task * Fix path to Olsson dataset loader * Filter genes cells before subsetting Olsson data in test * Use highly expressed genes for test Olsson dataset Test dataset is now 700 genes by 300 cells (was 500 x 500) * Add ivis dimred method (#369) * add densMAP package to python-extras * pre-commit * Add Ivis method * Explicitly mention it's CPU implementation * Add forgotten import in __init__ * Remove redundant filtering * Move ivis inside the function * Make var names unique, add ivis[cpu] to README * Pin tensorflow version * Add NeuralEE skeleton * Implement method * added densmap and densne * Fix typo pytoch -> torch * pre-commit * remove densne * Add forgotten detach/cpu/numpy * formatting * pre-commit * formatting * formatting * pre-commit * formatting * formatting * formatting * pre-commit * formatting * umap-learn implementation * pre-commit * Add docker image * Add skeleton method * formatting * Implement method * Fix some small bugs * Add preprocessing * Change batch size to 1k cells for aff. matrix * Add new preprocessing * Add new preprocessing * Fix preprocessing * Fix preprocessing * pre-commit * updated template for PR with PR evaluation checks (#314) * Update alra.py (#304) * Update alra.py Fix pre-processing and transformation back into the original space * pre-commit * Update alra.py * make sure necessary methods are imported Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com> Co-authored-by: Daniel Burkhardt <burkhardt.d.b@gmail.com> * Add scanpy preprocessing to densmap dimred method * Rename preprocess_scanpy() to preprocess_logCPM_1kHVG() * Add preprocessing suffix to dimred methods * Subset object in preprocess_logCPM_1kHVG() * Use standard names for input * Add neuralee_logCPM_1kHVG method * Add densmap_pca method * Fix preprocess_logCPM_1kHVG() Now returns an AnnData rather than acting in place - Subsetting wasn't working in place Also set HVG flavor to "cell_ranger" * Add test argument to dimred methods * Move preprocess_logCPM_1kHVG() to tools.normalize * Change name in python-method-scvis Docker README * Rename openproblems-python-method-scvis container Now called open-problems-python36 * Fix AnnData ref in merge * Copy object when subsetting in preprocess_logCPM_1kHVG() * Move PCA to dimred methods * Use preprocess_logCPM_1kHVG() in nn_ranking metrics * Fix path in python36 dockerfile * Add test kwarg to neuralee_default method * Add check for n_var to preprocess_logCPM_1kHVG() Should fix tests that were failing due to scverse/scanpy#2230 * Store raw counts in NeuralEE default method * Update dimred README * Replace X_input with PCA in ivis dimred method * Refactor preprocess_logCPM_1kHVG() to log_cpm_hvg() * Re-add ivis Co-authored-by: Ben DeMeo <bdemeo@mit.edu> Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com> Co-authored-by: Michal Klein <michal.klein@protonmail.com> Co-authored-by: michalk8 <46717574+michalk8@users.noreply.github.com> Co-authored-by: bendemeo <45370700+bendemeo@users.noreply.github.com> Co-authored-by: MalteDLuecken <m.d.luecken@gmail.com> Co-authored-by: Wesley Lewis <59123674+wes-lewis@users.noreply.github.com> Co-authored-by: Daniel Burkhardt <burkhardt.d.b@gmail.com> Co-authored-by: Scott Gigante <84813314+scottgigante-immunai@users.noreply.github.com> * hotfix timeout-minutes (#374) * use branch of scprep to provide R traceback (#376) Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com> Co-authored-by: Scott Gigante <scottgigante@users.noreply.github.com> Co-authored-by: Daniel Strobl <50872326+danielStrobl@users.noreply.github.com> Co-authored-by: SingleCellOpenProblems <singlecellopenproblems@protonmail.com> Co-authored-by: Luke Zappia <lazappi@users.noreply.github.com> Co-authored-by: Ben DeMeo <bdemeo@mit.edu> Co-authored-by: Michal Klein <michal.klein@protonmail.com> Co-authored-by: michalk8 <46717574+michalk8@users.noreply.github.com> Co-authored-by: bendemeo <45370700+bendemeo@users.noreply.github.com> Co-authored-by: MalteDLuecken <m.d.luecken@gmail.com> Co-authored-by: Wesley Lewis <59123674+wes-lewis@users.noreply.github.com> Co-authored-by: Daniel Burkhardt <burkhardt.d.b@gmail.com> Co-authored-by: Nikolay Markov <nikolai.markov@icloud.com> Co-authored-by: adamgayoso <adamgayoso@users.noreply.github.com> Co-authored-by: Valentine Svensson <v@nxn.se> Co-authored-by: Eduardo Beltrame <munfred@brandeis.edu> Co-authored-by: atchen <81202350+atchen@users.noreply.github.com> * clean up dockerfile Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com> Co-authored-by: Scott Gigante <scottgigante@users.noreply.github.com> Co-authored-by: Daniel Strobl <50872326+danielStrobl@users.noreply.github.com> Co-authored-by: SingleCellOpenProblems <singlecellopenproblems@protonmail.com> Co-authored-by: Luke Zappia <lazappi@users.noreply.github.com> Co-authored-by: Ben DeMeo <bdemeo@mit.edu> Co-authored-by: Michal Klein <michal.klein@protonmail.com> Co-authored-by: michalk8 <46717574+michalk8@users.noreply.github.com> Co-authored-by: bendemeo <45370700+bendemeo@users.noreply.github.com> Co-authored-by: MalteDLuecken <m.d.luecken@gmail.com> Co-authored-by: Wesley Lewis <59123674+wes-lewis@users.noreply.github.com> Co-authored-by: Daniel Burkhardt <burkhardt.d.b@gmail.com> Co-authored-by: Nikolay Markov <nikolai.markov@icloud.com> Co-authored-by: adamgayoso <adamgayoso@users.noreply.github.com> Co-authored-by: Valentine Svensson <v@nxn.se> Co-authored-by: Eduardo Beltrame <munfred@brandeis.edu> Co-authored-by: atchen <81202350+atchen@users.noreply.github.com> * only skip CI if command is in commit headline (#381) * only skip if ci skip is in commit headline * try using endsWith instead # ci skip * Fix CI skip (#382) * only skip if ci skip is in commit headline * try using endsWith instead # ci skip * make actions run * upgrade AMI (#384) * upgrade AMI * uncomment docker * uncomment tests * Revert "Run test_benchmark on a self-hosted runner (#373)" (#386) * revert 2d57868 * bash -x * /bin/bash * Bugfix CI (#387) * upgrade AMI * uncomment docker * uncomment tests * clean up testing * tighter diff for testing * more memory * Revert "Bugfix CI (#387)" (#388) This reverts commit b50a909. * pass test arg to methods through CLI (#390) * make scvi run faster on test mode (#385) * make scvi run faster on test mode * pass test argument through cli * dirty hack to fix docker_build (#391) * remove ivis temporarily (#392) * neuralee fix (#383) * build images before testing * try something different * needs * fewer linebreaks * try as string * move the if * remove one condition * fix * cancel more quickly * run benchmark * don't build on main in run_benchmark Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com> Co-authored-by: Scott Gigante <scottgigante@users.noreply.github.com> Co-authored-by: Daniel Strobl <50872326+danielStrobl@users.noreply.github.com> Co-authored-by: SingleCellOpenProblems <singlecellopenproblems@protonmail.com> Co-authored-by: Luke Zappia <lazappi@users.noreply.github.com> Co-authored-by: Ben DeMeo <bdemeo@mit.edu> Co-authored-by: Michal Klein <michal.klein@protonmail.com> Co-authored-by: michalk8 <46717574+michalk8@users.noreply.github.com> Co-authored-by: bendemeo <45370700+bendemeo@users.noreply.github.com> Co-authored-by: MalteDLuecken <m.d.luecken@gmail.com> Co-authored-by: Wesley Lewis <59123674+wes-lewis@users.noreply.github.com> Co-authored-by: Daniel Burkhardt <burkhardt.d.b@gmail.com> Co-authored-by: Nikolay Markov <nikolai.markov@icloud.com> Co-authored-by: adamgayoso <adamgayoso@users.noreply.github.com> Co-authored-by: Valentine Svensson <v@nxn.se> Co-authored-by: Eduardo Beltrame <munfred@brandeis.edu> Co-authored-by: atchen <81202350+atchen@users.noreply.github.com>

* fix neurips2021 wf and script * refactor script

* fix neurips2021 wf and script * refactor script Former-commit-id: 3a21a29

scottgigante-immunai and others added 2 commits March 16, 2022 17:02

import SCOT

0f4a13d

pre-commit

24a99ac

scottgigante-immunai requested a review from LuckyMD March 16, 2022 15:10

LuckyMD approved these changes Mar 16, 2022

View reviewed changes

scottgigante-immunai and others added 5 commits March 16, 2022 17:43

Merge branch 'main' into scottgigante/bugfix/scot_import

0bea721

Merge branch 'scottgigante/bugfix/scot_import' of github.com:scottgig…

c4976b5

…ante-immunai/openproblems into scottgigante/bugfix/scot_import

scran requires R

d93cb11

Merge branch 'main' into scottgigante/bugfix/scot_import

f401d7f

check that aligned spaces are finite

dd6e31c

scottgigante-immunai and others added 5 commits April 1, 2022 11:18

Merge branch 'main' into scottgigante/bugfix/scot_import

e7bbbcf

Merge branch 'main' into scottgigante/bugfix/scot_import

1ad6aa5

exclude unbalanced SCOT for now

d72be16

Merge branch 'scottgigante/bugfix/scot_import' of github.com:scottgig…

c6c673e

…ante-immunai/openproblems into scottgigante/bugfix/scot_import

Merge remote-tracking branch 'base/main' into scottgigante/bugfix/sco…

a8be9cd

…t_import

scottgigante-immunai marked this pull request as ready for review April 14, 2022 15:52

scottgigante-immunai merged commit a68ce4d into openproblems-bio:main Apr 14, 2022

scottgigante-immunai deleted the scottgigante/bugfix/scot_import branch April 14, 2022 17:21

rcannood added a commit that referenced this pull request Sep 4, 2024

fix neurips2021 wf and script (#333)

3a21a29

* fix neurips2021 wf and script * refactor script

rcannood added a commit that referenced this pull request Sep 4, 2024

fix neurips2021 wf and script (#333)

35bb82a

* fix neurips2021 wf and script * refactor script Former-commit-id: 3a21a29

Navigation Menu

Uh oh!

Import SCOT - #333

Import SCOT#333
scottgigante-immunai merged 12 commits into
openproblems-bio:mainfrom
scottgigante-immunai:scottgigante/bugfix/scot_import

scottgigante-immunai commented Mar 16, 2022 •

edited

Loading

Uh oh!

scottgigante-immunai commented Mar 29, 2022

Uh oh!

scottgigante-immunai commented Apr 14, 2022

Uh oh!

codecov-commenter commented Apr 14, 2022

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

4 participants

Uh oh!

Conversation

scottgigante-immunai commented Mar 16, 2022 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Submission type

Testing

Submission guidelines

PR review checklist

Uh oh!

scottgigante-immunai commented Mar 29, 2022

Uh oh!

scottgigante-immunai commented Apr 14, 2022

Uh oh!

codecov-commenter commented Apr 14, 2022

Codecov Report

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

4 participants

scottgigante-immunai commented Mar 16, 2022 •

edited

Loading