Climbnet is a CCN that detects holds on climbing gym walls and returns the appropriate boundary mask for use in instance segmentation.
HOLD - Includes holds as well as the large shaped volumes.
VOLUME - This refers to any and all box volumes.
They are usually made of wood, triangular in shape, and often have bolt holes so that holds can be mounted on them. Sometimes they are themselves a hold or specific to a particular route as opposed to being just another part of the wall.
| Category | Total |
|---|---|
| Hold | 7307 |
| Volume | 520 |
This project uses Facebook's detectron2 implmentation of Mask R-CNN and was trained using 210 images.
The weights are available for download using the following link.
There are two ways to run the model.
Use the demo.py file provided with this project.
Install the required dependencies
# python 3.6+
pip install -r requirements.txt
Run the demo
cd test
python demo.py demo_image1.jpg <path_to_model_weights>
Follow the steps outlined in the detectron2 getting started guide. Replace the model weights with those from this project.
Classes returned during inference will be incorrect. Instead of
HoldsandVolumesyou will instead see classes from theCOCOdataset.
python demo.py --config-file ../configs/COCO-InstanceSegmentation/mask_rcnn_R_50_FPN_3x.yaml \
--input input1.jpg input2.jpg \
[--other-options]
--opts MODEL.WEIGHTS <climbnet weights>
Most of the imagery used was sourced from the internet. As a result of that, many of the images have already had compression or filters applied to them and are rather small in size, usually 1280x1280 or smaller.
A lot of the imagery was taken from the same vantage point (directly facing the wall), which may lead to issues when running inference on imagery with different vantage points(sport climbs) or extremely sloped walls.
One of the downsides of using detectron2 is that the size of the available networks+weights are very large. The inference time is also not very fast, especially when using a cpu.
There are some segmented polygons in the data which affect the inference accuracy. They exist because the program that I used for tagging Hyperlabel does not support grouping as per the COCO specification. Both Holds and Volumes are affected.
Ex. Both pieces of the polygon below should be sent through the network as a part of a single hold but are currently passed as two distinctly separate holds
Contributions and suggestions are welcome and encouraged, especially additional high quality image for training.
Open an issue or send me send me an e-mail
- Add open source imagery
- Collect and tag more images
- Upload pre-processing script
- Upload mask export script
- Create video explaining tagging process for contributors
- Explore reducing model size using Centermask, Mobilenet, etc.
- Correct segmented polygons in the data
- Include training stats. BOX AP, MASK AP etc.