about network: FastSAM is only to train a YOLOV8-seg?

[suliangxu]

When I trained FastSAM on coco128-seg dataset, I found that the part that needed training was the YOLOV8-seg model. So FastSam is only to train a YOLOV8-seg and then adding prompting oprations to it？

[diya-he]

Yeah, I have the same doubt as you do in this regard. The text_prompt only load 'ViT-B/32' weights and don't take any finetune. So, I guass the FastSAM.pt is just that the yolov8-seg.pt. But I try to train yolov8-seg with my datasets. When I use it in FastSAM,I have encountered a problem. There is no complete tutorial for developer about how to train a FastSAM model.

[diya-he]

[suliangxu]

you can get the training code at this link https://github.com/CASIA-IVA-Lab/FastSAM/releases/tag/v0.0.2

[diya-he]

I use it to train my model, but when I use it in FastSAM code, it always occurs error.

[diya-he]

I solved my problem. It needs to put cfg folder to ultralytics install path

[suliangxu]

I can successfully train the model with the above-mentioned training code. And the contents of train_sa.py are:

from ultralytics import YOLO
model = YOLO(model="./FastSAM.pt")   # the path of the pretrained model

model.train(data="coco.yaml",
            epochs=100,
            batch=2,
            imgsz=1024,
            overlap_mask=False,
            save=True,
            save_period=5,
            device='4',
            project='fastsam',
            name='test', 
            val=False,)

Then, running this file we can successfully train the model.

( If you have any more detailed questions, you can contact me at suliangxu@nuaa.edu.cn.)

[joshua-atolagbe]

Hello @suliangxu,

Thank you for opening this issue and your useful comments. Using the training and validation codes released, I have been trying to train the FastSAM on my custom dataset for instance segmentation (with 6 classes). I structured my dataset following the coco8-seg. But I am getting this error:

Seems the problem is with the augmentation, so I explicitly disabled augmentation by setting augment=False. But I still get the same error.

Here's my train_sa.py:

from ultralytics import YOLO

model = YOLO(model="FastSAM-s.pt")
model.train(
    data="sa.yaml",
    task='segment',
    epochs=3,
    augment=False,
    batch=8,
    imgsz=255,
    overlap_mask=False,
    save=True,
    save_period=5,
    project="fastsam",
    name="test",
    val=False,
)

I will appreciate your help in fixing this. Thank you!

[suliangxu]

Hello @suliangxu,

Thank you for opening this issue and your useful comments. Using the training and validation codes released, I have been trying to train the FastSAM on my custom dataset for instance segmentation (with 6 classes). I structured my dataset following the coco8-seg. But I am getting this error:

Seems the problem is with the augmentation, so I explicitly disabled augmentation by setting augment=False. But I still get the same error.

Here's my train_sa.py:

from ultralytics import YOLO

model = YOLO(model="FastSAM-s.pt")
model.train(
    data="sa.yaml",
    task='segment',
    epochs=3,
    augment=False,
    batch=8,
    imgsz=255,
    overlap_mask=False,
    save=True,
    save_period=5,
    project="fastsam",
    name="test",
    val=False,
)

I will appreciate your help in fixing this. Thank you!

It seems this error occurs because of the img.shape[2]. Here is the debug info for training the coco-seg128 dataset. I hope it is helpful for you.

[joshua-atolagbe]

Hello @suliangxu,
Thank you for opening this issue and your useful comments. Using the training and validation codes released, I have been trying to train the FastSAM on my custom dataset for instance segmentation (with 6 classes). I structured my dataset following the coco8-seg. But I am getting this error:

Seems the problem is with the augmentation, so I explicitly disabled augmentation by setting augment=False. But I still get the same error.
Here's my train_sa.py:

from ultralytics import YOLO

model = YOLO(model="FastSAM-s.pt")
model.train(
    data="sa.yaml",
    task='segment',
    epochs=3,
    augment=False,
    batch=8,
    imgsz=255,
    overlap_mask=False,
    save=True,
    save_period=5,
    project="fastsam",
    name="test",
    val=False,
)

I will appreciate your help in fixing this. Thank you!

It seems this error occurs because of the img.shape[2]. Here is the debug info for training the coco-seg128 dataset. I hope it is helpful for you.

Thank you for your response @suliangxu , I am working with grayscale images (single channel of with shape [h, w], and not [h,w,c]). Initially, I had saved the images, which is made up of floating point array into .tif. Is there anyway around this? Because it seems the error message is with the shape in the Mosaic class. I tried to set the p=0.0 to disable the mosaic augmentation but it is not working. I will appreciate your help. Thanks!

[suliangxu]

Hello @suliangxu,
Thank you for opening this issue and your useful comments. Using the training and validation codes released, I have been trying to train the FastSAM on my custom dataset for instance segmentation (with 6 classes). I structured my dataset following the coco8-seg. But I am getting this error:

Seems the problem is with the augmentation, so I explicitly disabled augmentation by setting augment=False. But I still get the same error.
Here's my train_sa.py:

from ultralytics import YOLO

model = YOLO(model="FastSAM-s.pt")
model.train(
    data="sa.yaml",
    task='segment',
    epochs=3,
    augment=False,
    batch=8,
    imgsz=255,
    overlap_mask=False,
    save=True,
    save_period=5,
    project="fastsam",
    name="test",
    val=False,
)

I will appreciate your help in fixing this. Thank you!

It seems this error occurs because of the img.shape[2]. Here is the debug info for training the coco-seg128 dataset. I hope it is helpful for you.

Thank you for your response @suliangxu , I am working with grayscale images (single channel of with shape [h, w], and not [h,w,c]). Initially, I had saved the images, which is made up of floating point array into .tif. Is there anyway around this? Because it seems the error message is with the shape in the Mosaic class. I tried to set the p=0.0 to disable the mosaic augmentation but it is not working. I will appreciate your help. Thanks!

I changed the hyper-parameter mosaic: 0.0 # (float) image mosaic (probability) in yolo/cfg/default.yaml, and successfully disabled the mosaic augmentation. Maybe you can try it!

[joshua-atolagbe]

Hello @suliangxu,
Thank you for opening this issue and your useful comments. Using the training and validation codes released, I have been trying to train the FastSAM on my custom dataset for instance segmentation (with 6 classes). I structured my dataset following the coco8-seg. But I am getting this error:

Seems the problem is with the augmentation, so I explicitly disabled augmentation by setting augment=False. But I still get the same error.
Here's my train_sa.py:

from ultralytics import YOLO

model = YOLO(model="FastSAM-s.pt")
model.train(
    data="sa.yaml",
    task='segment',
    epochs=3,
    augment=False,
    batch=8,
    imgsz=255,
    overlap_mask=False,
    save=True,
    save_period=5,
    project="fastsam",
    name="test",
    val=False,
)

I will appreciate your help in fixing this. Thank you!

It seems this error occurs because of the img.shape[2]. Here is the debug info for training the coco-seg128 dataset. I hope it is helpful for you.

Thank you for your response @suliangxu , I am working with grayscale images (single channel of with shape [h, w], and not [h,w,c]). Initially, I had saved the images, which is made up of floating point array into .tif. Is there anyway around this? Because it seems the error message is with the shape in the Mosaic class. I tried to set the p=0.0 to disable the mosaic augmentation but it is not working. I will appreciate your help. Thanks!

I changed the hyper-parameter mosaic: 0.0 # (float) image mosaic (probability) in yolo/cfg/default.yaml, and successfully disabled the mosaic augmentation. Maybe you can try it!

Thank you @suliangxu . That solved the augmentation issue. But another error popped up. I think the issue still revolves around the number of color channels. Is it that the FastSAM can't be trained on single-channel images? Any help?

[joshua-atolagbe]

@suliangxu . Thank you for your help. I fixed the issue already and have been able to train the model on my custom dataset. I had to convert my grayscale images to three-color channel images. Seems the training and validation codes were designed to work explicitly with BGR images.

[joshua-atolagbe]

When I trained FastSAM on coco128-seg dataset, I found that the part that needed training was the YOLOV8-seg model. So FastSAM is only to train a YOLOV8-seg and then adding prompting operations to it？

@glenn-jocher can you please clarify this?

[BirATMAN]

Hey, I'm training the yolov8seg model as provided in the latest released train codes. Has anyone trained FASTsam or added the prompt selection part? Please share the codes if possible.