Skip to content
Open
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 1 addition & 1 deletion book/data_management.md
Original file line number Diff line number Diff line change
Expand Up @@ -953,7 +953,7 @@ However,*git* does not work well for version control on larger datasets using bi

### Using *DataLad* for version control on larger datasets

A solution to this problem is to use a version control tool that is specifically designed for large data. There are several tools that address this problem; we will focus on [Dat*aLad](https://www.datalad.org/), which is a data management system that functions very similarly to git. It is based on a tool called [*git-annex*](https://git-annex.branchable.com/), but provides much greater ease of use for researchers. (Full disclosure: Our group collaborates with the *DataLad* group and our grants have supported some of their development work.)
A solution to this problem is to use a version control tool that is specifically designed for large data. There are several tools that address this problem; we will focus on [*DataLad*](https://www.datalad.org/) [@Halchenko:2021aa], which is a data management system that functions very similarly to git. It is based on a tool called [*git-annex*](https://git-annex.branchable.com/), but provides much greater ease of use for researchers. (Full disclosure: Our group collaborates with the *DataLad* group and our grants have supported some of their development work.)

An important note: *DataLad* is quite powerful but has a significant learning curve, and takes a bit of time to get accustomed to. In particular, its use of symbolic links can sometimes confuse new users. Having said that, let's look at some simple examples.

Expand Down
13 changes: 13 additions & 0 deletions book/references.bib
Original file line number Diff line number Diff line change
Expand Up @@ -1337,3 +1337,16 @@ @article{Open-Science-Collaboration:2015aa
volume = {349},
year = {2015},
bdsk-url-1 = {https://doi.org/10.1126/science.aac4716}}

@article{Halchenko:2021aa,
author = {Halchenko, Yaroslav O. and Meyer, Kyle and Poldrack, Benjamin and Solanky, Debanjum Singh and Wagner, Adina S. and Gors, Jason and MacFarlane, Dave and Pustina, Dorian and Sochat, Vanessa and Ghosh, Satrajit S. and M{\"o}nch, Christian and Markiewicz, Christopher J. and Waite, Laura and Shlyakhter, Ilya and de la Vega, Alejandro and Hayashi, Soichi and H{\"a}usler, Christian Olaf and Poline, Jean-Baptiste and Kadelka, Tobias and Skyt{\'e}n, Kusti and Jarecka, Dorota and Kennedy, David and Strauss, Ted and Cieslak, Matt and Vavra, Peter and Ioanas, Horea-Ioan and Schneider, Robin and Pfl{\"u}ger, Mika and Haxby, James V. and Eickhoff, Simon B. and Hanke, Michael},
doi = {10.21105/joss.03262},
journal = {Journal of Open Source Software},
number = {63},
pages = {3262},
publisher = {The Open Journal},
title = {DataLad: distributed system for joint management of code, data, and their relationship},
url = {https://doi.org/10.21105/joss.03262},
volume = {6},
year = {2021},
bdsk-url-1 = {https://doi.org/10.21105/joss.03262}}
Loading