In this PR,

The main idea is to log the semantic segmentation (IOU, Accuracy) metrics during training so that they can be compared with the validation metrics in addition to the loss.

Changes

• Added a SemSegEvaluatorTrain hook that writes the training metrics to the tensorboard and wandb.
• Added the hook in the default configuration for scannetpp methods.
• Semantic segmenation models now also return the seg_logits during training for evaluation.

Demo

The full logs can be found here.

Note this PR depends on another open PR, which can be found here.

Looks awesome

@Streakfull
From now on, one day on pr. I will process this tmr.

@Gofinge Alright, sounds good thanks! We ran into an issue today, when trying to reproduce some evaluation results with stored checkpoints for configs/scannetpp/semseg-spunet-v1m1-0-base.py , not sure yet, what else isn't working. This is the issue:

• Running the evaluation on commit e4de3c25f57d0625cdcb66589cf180e838a05b19 gives 0.478 mIou (this is correct and matches our previous results in the fork)
• Running the evaluation in the latest main branch gives 0.4 mIou
• After some debugging, I was able to narrow it down to differences in the datasets/transform file. I.E I replaced the datasets/transform file in the working branch with the latest datasets/transform file and adjusted the configuration accordingly, then got the 0.4 mIou.

Do you have any suggestions on how to proceed with this? Should I open a separate issue?

@Gofinge Alright, sounds good thanks! We ran into an issue today, when trying to reproduce some evaluation results with stored checkpoints for configs/scannetpp/semseg-spunet-v1m1-0-base.py , not sure yet, what else isn't working. This is the issue:

• Running the evaluation on commit e4de3c25f57d0625cdcb66589cf180e838a05b19 gives 0.478 mIou (this is correct and matches our previous results in the fork)
• Running the evaluation in the latest main branch gives 0.4 mIou
• After some debugging, I was able to narrow it down to differences in the datasets/transform file. I.E I replaced the datasets/transform file in the working branch with the latest datasets/transform file and adjusted the configuration accordingly, then got the 0.4 mIou.

Do you have any suggestions on how to proceed with this? Should I open a separate issue?

The change I think maybe NormalizeColor. Previous range is [-1, 1] but the latest version is [0,1].

This snapshot is from previous version.

Thanks @RayYoh