Add YOLO PreProcessor#25
Conversation
1bf7f79 to
8ad1335
Compare
zoq
left a comment
There was a problem hiding this comment.
Looks great to me, from the first comment it sounds like the next step is to implement/extend the test case.
|
Hey @kartikdutt18 , I just wanted to know whether you were stuck anywhere with this because we were planning to finish this first. |
|
Hey @saksham189, I've probably written some expression wrong somewhere and I wasn't able to track it down before Saturday so as it was decided to start with the translator from Saturday, I implemented the weight translator here so we can now transfer weights of Darknet model or any other model from PyTorch to mlpack. However other than weights we also need to transfer running mean and variance for BatchNorm Layer and with that I'm stuck with the error here. I'll take a look at the preprocessor again today. In the meanwhile, I tried training Darknet 19 model on Cifar10, It didn't converge, so I also added Darknet 1 (same as YOLO model) on Cifar10 to see if we can train that and it takes 3 hours for a single epoch so we should also be able to train YOLO model (in less time than Darknet), if needed. Here is the log for the first epoch (20% accuracy). |
|
Looks good and the training time is also considerably lower. So, maybe we should just train that first and look at the accuracy. |
Great, I'm letting the model train. In the meanwhile, I will try to get running mean and variance also loaded into mlpack. Also, with transferred weights we can also train the model on imagenette (that way running mean and variance will be set in training). |
|
Hey @saksham189, I fixed the errors in the implementation as well added tests fro single image, multiple images and image with multiple bounding box. |
0c54099 to
dc43533
Compare
| arma::vec desiredSum(3); | ||
| desiredSum << 8.3342 << 18.4093 << 7.13195 << arma::endr; | ||
| for (size_t i = 0; i < output.n_cols; i++) | ||
| BOOST_REQUIRE_CLOSE(arma::accu(output.col(i)), desiredSum(i), 1e-3); |
There was a problem hiding this comment.
Maybe we should also compare the normalized boxes with pre calculated ones, what do you think?
There was a problem hiding this comment.
Sure, I can add that as well.
Hey @saksham189, @KimSangYeon-DGU, I continued the training. Initially it was for 5 epochs (the learning rate was 0.01) and after 5 epochs it remained constant and the below graph was obtained. Then, I set learning rate to 0.001 and it's still training and has completed nearly 25 epochs out of 40. The training and validation accuracy are continuously increasing from 20% and 7% respectively to 31% and roughly 20%. The graph for that is also attached below. Also, about using larger image sizes such as imagenette, The training time doesn't change significantly with larger images however time for loading the dataset is very large. Imagenette with 320 x 320 x 3 image size and 5k images nearly took an hour. First attempt to train (5 epochs / lr = 1e-2) (training accuracy = 26%, Graphs are shown below) |
5e00f10 to
8026eaa
Compare
saksham189
left a comment
There was a problem hiding this comment.
Looks mostly good to me! Just one minor comment.
KimSangYeon-DGU
left a comment
There was a problem hiding this comment.
Left minor comments about styles
|
I think this is ready to go as well. |
KimSangYeon-DGU
left a comment
There was a problem hiding this comment.
Sorry for the late response. I left a question for me to get understood this feature.
| * @param annotations Field object created using model's dataloader containing | ||
| * annotation for images. | ||
| * @param output Output matrix where output will be stored. |
There was a problem hiding this comment.
Hey @kartikdutt18, can you explain the input, output, and terms of cell and grid which are used in this implementation?
There was a problem hiding this comment.
Hey @KimSangYeon-DGU, no worries, The input here would be a field type containing bounding boxes, the output would be an arma::mat which be would be of size batchSize * (featureMapSize) where featureSize would correspond to S * S * (numBoundingBoxes * (numClasses + 5)) for YOLOv3 and S * S * (numBoundingBoxes * 5 + numClasses). Here cell is a single cell / data point in Feature Size. And the grid is featureSize reshaped as a cube (S, S, (numBoundingBoxes * 5 + numClasses)).
|
Thanks for the great contribution @kartikdutt18 ! |
|
Thanks lot @KimSangYeon-DGU, @saksham189, @zoq for the reviews and all the help. Also, thanks a lot for mentoring me during GSoC. |
Hey everyone,
This PR aims to add preprocessor function for YOLO class. This basically converts annotations into feature maps that can be used for training YOLO model i.e. have the same format as output of YOLO model.
To Do :
CC : @KimSangYeon-DGU, @saksham189, @zoq