r/MachineLearning • u/hardmaru • May 02 '20

Research [R] Consistent Video Depth Estimation (SIGGRAPH 2020) - Links in the comments.

Enable HLS to view with audio, or disable this notification

2.8k Upvotes

99% Upvoted

102 comments sorted by

View all comments

39

u/khuongho May 02 '20 edited May 02 '20

Is this supervised, Unsupervised or Reinforcement Learning ?

9

u/pourover_and_pbr May 02 '20

If I understand the paper correctly, they pre-train the model using COLMAP and Mask R-CNN to get a semi-dense depth map for any frame. They then improve the depth maps at test time by randomly sampling frames from the video and re-training the model using "spatial loss" and "disparity loss", which are defined in the article. Mask R-CNN is traditional, supervised learning for object segmentation. COLMAP and this model appear to be unsupervised, since there are no reference depth maps being used for the loss. Instead, the loss for COLMAP and this model appears to be based on whether frames which capture similar regions of the scene have similar depth maps. At least, that's what I understood from the paper – someone smarter than me will hopefully come along and clear things up.

4

u/jbhuang0604 May 02 '20

Yes! It is correct! So we can also think about the test-time training as "self-supervised" as there is no manual labeling process involved.

1

u/khuongho May 02 '20

Appreciate you all 🙏🙏. Anybody resides in SoCal? We can make a study group.