Synthetic Data Expt. Log | Notion

Human Segmentation

Dataset

Dataset	Recommend	Detail	Resolution	Total	Trainset	Testset	Download
Automatic Portrait Matting	✔	One person/pic	600×800	2,000	1,700	300	http://www.cse.cuhk.edu.hk/~leojia/projects/automatting/

Backbone: MobilenetV3-small, large

Loss: SoftIoULoss

Optimizer: Adam

Different MACs

Model	MAC (G)	0*	2000	4000	6000	8000	10000	Δ data simulation
160x90_small_0.5	0.02	87.84%	90.44%	90.72%	90.87%	90.63%	90.73%	3.03%
160x90_small_1.0	0.04	88.56%	91.09%	91.42%	91.44%	91.42%	91.19%	2.88%
256x144_small_1.0	0.09	90.59%	91.74%	92.46%	92.77%	92.74%	92.69%	2.18%
256x144_large_1.0	0.38	92.19%	93.42%	93.79%	93.78%	93.52%	93.40%	1.60%
256x144_large_1.5	0.57	93.18%	94.52%	94.27%	93.90%	93.64%	93.70%	1.34%

Same resolution, different width channel

Model	0*	2000	4000	6000	8000	10000	Δ data simulation
160x90_small_0.5	87.84%	90.44%	90.72%	90.87%	90.63%	90.73%	3.03%
160x90_small_0.75	88.55%	90.31%	91.42%	91.46%	91.41%	90.97%	2.91%
160x90_small_1.0	88.56%	91.09%	91.42%	91.44%	91.42%	91.19%	2.88%

Compared with data augmentation

Model	Data augmentation	0*	2000	4000	6000	8000	10000	Δ data simulation
160x90_small_1.0		88.56%	91.09%	91.42%	91.44%	91.42%	91.19%	2.88%
160x90_small_1.0	✔	90.68%	91.66%	91.87%	91.78%	91.62%	91.65%	1.19%
256x144_small_1.0		90.59%	91.74%	92.46%	92.77%	92.74%	92.63%	2.18%
256x144_small_1.0	✔	92.41%	92.56%	92.85%	92.74%	93.00%	92.69%	0.59%
256x144_large_1.0		92.19%	93.42%	93.79%	93.78%	93.52%	93.40%	1.60%
256x144_large_1.0	✔	93.70%	93.78%	93.87%	94.17%	94.04%	93.60%	0.47%

Ablation study

👇 baseline: 160x90 mobilenetv3-small 1.0

Data Augmentation	Data Simulation	mIoU
		88.56%
✔		90.68% (+2.12%)
	✔	91.44% (+2.88%)
✔	✔	91.78% (+3.22%)

Unity Simulated Data and its GT

training_simulation_gt

training_simulation_gt

training_raw_gt

training_raw_gt

testing_raw_gt

testing_raw_gt

Conclusion

For the baseline(160x90 resolution, mobilenetv3-small as the backbone, mIoU 88.56%, without data augmentation), mIoU that model trained on simulated data can increase by 2.88%. It can still improve by 1.19% with both data augmentation.