summary one of repositories downloads fails with timeout error
environment
Distributor ID: Ubuntu
Description: Ubuntu 18.04.3 LTS
Release: 18.04
Codename: bionic
GOMAXPROCS is 8
8 CPU 16G RAM
steps to reproduce
just run this command
./gitcollector download --library=/home/lwsanty/gitcollector/ --orgs=src-d,bblfsh
expected all corresponding repositories downloaded
observed one of downloads fails with timeout
[2019-09-10T16:54:41.685787348+03:00] ERROR failed error=timeout exceeded: unable to retrieve repository from location cc761bed408a1d6a20457d48d8bf807d3531db4f in transactional mode.
github.com/src-d/go-borges/siva.(*transactioner).Start
/home/lwsanty/goproj/gopath/pkg/mod/github.com/src-d/go-borges@v0.0.0-20190704083038-44867e8f2a2a/siva/transactioner.go:45
github.com/src-d/go-borges/siva.(*Location).repository
/home/lwsanty/goproj/gopath/pkg/mod/github.com/src-d/go-borges@v0.0.0-20190704083038-44867e8f2a2a/siva/location.go:528
github.com/src-d/go-borges/siva.(*Location).Init
/home/lwsanty/goproj/gopath/pkg/mod/github.com/src-d/go-borges@v0.0.0-20190704083038-44867e8f2a2a/siva/location.go:242
github.com/src-d/gitcollector/downloader.PrepareRepository
/home/lwsanty/goproj/lwsanty/gitcollector/downloader/git.go:199
github.com/src-d/gitcollector/downloader.downloadRepository
/home/lwsanty/goproj/lwsanty/gitcollector/downloader/download.go:171
github.com/src-d/gitcollector/downloader.Download
/home/lwsanty/goproj/lwsanty/gitcollector/downloader/download.go:78
github.com/src-d/gitcollector/library.(*Job).Process
/home/lwsanty/goproj/lwsanty/gitcollector/library/job.go:57
github.com/src-d/gitcollector.(*worker).consumeJob.func1
/home/lwsanty/goproj/lwsanty/gitcollector/worker.go:61
runtime.goexit
/usr/local/go/src/runtime/asm_amd64.s:1337 id=72f28f0b-50f1-40e1-8457-e27d138525e4 job=download url=https://github.com/src-d/go-mysql-server
note issue was reproduced only once and cannot be reproduced after 3 attempts
question it looks like the problem is in transactions timeout of borges, we could avoid this problem if we propagate transaction timeout from gitcollector to borges by adding new flag to gitcollector
summary one of repositories downloads fails with timeout error
environment
steps to reproduce
just run this command
expected all corresponding repositories downloaded
observed one of downloads fails with timeout
note issue was reproduced only once and cannot be reproduced after 3 attempts
question it looks like the problem is in transactions timeout of
borges, we could avoid this problem if we propagate transaction timeout fromgitcollectortoborgesby adding new flag togitcollector