Avatar

Ukcheol Shin

Postdoctoral Fellow

Robotics Institute, CMU


CV | Google Scholar | Github


I am a postdoctoral researcher at Robotics Institute, CMU. I obtained my Ph.D. and M.S. at KAIST, advised by Professor In So Kweon. I am a recipient of the Best Student Paper Award from WACV 2023.

My research focuses on developing a robust physical AI that can perceive and navigate the dynamic world in challenging conditions, with a specific interest in self-supervised learning of 3D geometry, reinforcement learning, multi-sensor fusion, and vision-action learning. I am interested in the following areas, but also open to other explorable/challenging domains.

  • Unsupervised and Self-supervised 3D Geometry (depth, optical flow, odometry, SLAM)
  • 3D Geometry in Challenging Conditions (rainy, snowy, over-exposed, low-lighted, etc)
  • Multi-modal Sensor Fusion
  • Learning Representation from Multi-modal Sensor and Self-supervision
  • Deep Reinforcement Learning for sensor and robot

Contact

  • shinwc159 [at] gmail.com

    ushin [at] andrew.cmu.edu

  • Robotics Institute, 1723 Murray Ave, Pittsburgh, PA 15217

Education

  • Ph.D. in Electrical Engineering, KAIST, 2023

    Dissertation: "Self-supervised 3D Geometric Perception in Adverse Real-world Environment"

    Advisor: Prof. In So Kweon

  • M.S. in Electrical Engineering, KAIST, 2019

    Thesis: "Noise-Aware Camera Exposure Control for Robust Robot Vision"

    Advisor: Prof. In So Kweon

  • BS in Electrical and Information Engineering, SNUST, 2017

    Project: "Real-Time Ethernet Protocol based Omni Directional Mobile Robot"

    Advisor: Prof. Byoung Wook Choi

Research Experiences

  • Carnegie Mellon University, Pittsburgh, United States
    Aug. 2023 - Present

    Postdoctoral Fellow (Postdoc), Bot Intelligence Group (Advisor: Jean Oh).
  • Korea Advanced Institute of Science and Technology, Daejeon, Korea
    Sep. 2019 - Aug. 2023

    Research Assistant (Ph.D.), Robotics and Computer Vision Lab (Advisor: In So Kweon).
    Research topics : Self-supervised learning, 3D Geometry, Thermal camera, Reinforcement Learning.
  • Korea Advanced Institute of Science and Technology, Daejeon, Korea
    Sep. 2017 - Aug. 2019

    Research Assistant (M.S.), Robotics and Computer Vision Lab (Advisor: In So Kweon).
    Research topics : Camera Exposure Control, Low-level Vision, 3D Geometry.
  • Seoul National University of Science and Technology, Seoul, Korea
    Mar. 2015 - Jun. 2017

    Research Intern, Embedded System Lab (Advisor: Byoung Wook Choi).
    Research topics : Embedded Linux, Real-time Operating System, Real-time Ethernet Protocol.

Publications

  • Flow4D: Leveraging 4D Voxel Network for LiDAR Scene Flow Estimatoin

    Jaeyeul Kim*, Jungwan Woo*, Ukcheol Shin, Jean Oh, and Sunghoon Im (*Equal Contribution)

    Under-review, 2024

    1st Place Award, CVPR 2024 Autonomous Driving Workshop "Argoverse Scene Flow Challenge"

    [ paper / code ]

  • Bridging Spectral-wise and Multi-spectral Depth Estimation via Geometry-guided Contrastive Learning

    Ukcheol Shin, Kyunghyun Lee, and Jean Oh

    Under-review, 2024

    [ paper / ]

  • FIReStereo: Forest InfraRed Stereo Dataset for UAS Depth Perception in Visually Degraded Environments

    Devansh Dhrafan*, Yifei Liu*, Andrew Jong, Ukcheol Shin, Yao He, Tyler Harp, Yaoyu Hu, Jean Oh, Sebastian Scherer (*Equal Contribution)

    Under-review, 2024

    [ paper / project / code ]

  • VPOcc: Exploiting Vanishing Point for Monocular 3D Semantic Occupancy Prediction

    Junhee Lee*, Junsu Kim*, Ukcheol Shin, Jean Oh, Kyungdon Joo (*Equal Contribution)

    Under-review, 2024

    [ paper ]

  • Exploiting Cross-modal Cost Volume for Multi-sensor Depth Estimation

    Janghyun Kim, Ukcheol Shin, Seokyong Heo, Jinsun Park

    Asian Conference on Computer Vision (ACCV), 2024

    [ paper ]

  • Density-aware Domain Generalization for LiDAR Semantic Segmentation

    Jaeyeul Kim*, Jungwan Woo*, Ukcheol Shin, Jean Oh, and Sunghoon Im (*Equal Contribution)

    IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS), 2024

    [ paper ]

  • Learning to Control Camera Exposure via Reinforcement Learning

    Kyunghyun Lee*, Ukcheol Shin*, Byeong-Uk Lee (* equal contribution)

    Proceedings of the IEEE conference on computer vision and pattern recognition (CVPR), 2024

    [ paper / project ]

  • Learning Quadrupedal Locomotion with Impaired Joints Using Random Joint Masking

    Mincheol Kim, Ukcheol Shin, Jung-Yup Kim

    IEEE International Conference on Robotics and Automation (ICRA), 2024 Oral

    Media coverage: covered by IEEE Spectrum

    [ paper / project ]

  • Complementary Random Masking for RGB-T Semantic Segmentation

    Ukcheol Shin, Kyunghyun Lee, In So Kweon, Jean Oh

    IEEE International Conference on Robotics and Automation (ICRA), 2024 Oral

    [ paper / code / project ]

  • Stable Surface Regularization for Fast Few-Shot NeRF

    ByeongIn Joung, Byeong-Uk Lee, Jaesung Choe, Ukcheol Shin, Minjun Kang, Taeyeop Lee, In-So Kweon, Kuk-Jin Yoon

    International Conference on 3D Vision (3DV), 2024

    [ paper ]

  • Joint Self-supervised Learning and Adversarial Adaptation for Monocular Depth Estimation from Thermal Image

    Ukcheol Shin, Kwanyong Park, Kyunghyun Lee, Byeong-Uk Lee, In So Kweon

    Machine Vision and Applications (MVA), 2023

    [ paper / code ]

  • Deep Depth Estimation from Thermal Image

    Ukcheol Shin, Jinsun Park, In So Kweon

    Proceedings of the IEEE conference on computer vision and pattern recognition (CVPR), 2023

    [ paper / benchmark / MS2 dataset / Github ]

  • Self-supervised Monocular Depth Estimation from Thermal Images via Adversarial Multi-spectral Adaptation

    Ukcheol Shin, Kwanyong Park, Byeong-Uk Lee, Kyunghyun Lee, In So Kweon

    IEEE Winter Conference on Applications of Computer Vision (WACV), 2023 Oral

    Best Student Paper Award, Out of 641 accepted papers (out of 1577 submissions)

    [ paper / code / poster / award ]

  • Maximizing Self-supervision from Thermal Image for Effective Self-supervised Learning of Depth and Ego-motion

    Ukcheol Shin, Kyunghyun Lee, Byeong-Uk Lee, In So Kweon

    IEEE Robotics and Automation Letters, 2022 (RA-L)

    IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS), 2022 Oral

    [ paper / code / project ]

  • DRL-ISP: Multi-Objective Camera ISP with Deep Reinforcement Learning

    Ukcheol Shin*, Kyunghyun Lee*, In So Kweon (* equal contribution)

    IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS), 2022 Oral

    [ paper / code / project ]

  • UDA-COPE: Unsupervised Domain Adaptation for Category-level Object Pose Estimation

    Taeyeop Lee, Byeong-Uk Lee, Inkyu Shin, Jaesung Choe, Ukcheol Shin, In So Kweon, Kuk-Jin Yoon

    Proceedings of the IEEE conference on computer vision and pattern recognition (CVPR), 2022

    [ paper / project ]

  • Self-supervised Depth and Ego-motion Estimation for Monocular Thermal Video using Multi-spectral Consistency Loss

    Ukcheol Shin, Kyunghyun Lee, Seokju Lee, In So Kweon

    IEEE Robotics and Automation Letters, 2021 (RA-L)

    IEEE International Conference on Robotics and Automation (ICRA), 2022 Oral

    [ paper / code / project ]

  • MS-UDA: Multi-Spectral Unsupervised Domain Adaptation for Thermal Image Semantic Segmentation

    YeongHyeon Kim, Ukcheol Shin, Jinsun Park, In So Kweon

    IEEE Robotics and Automation Letters, 2021 (RA-L)

    IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS), 2021 Oral

    [ paper / code / project ]

  • An Efficient Asynchronous Method for Integrating Evolutionary and Gradient-based Policy Search

    Kyunghyun Lee, Byeong-Uk Lee, Ukcheol Shin, In So Kweon

    Conference on Neural Information Processing Systems (NeurIPS), 2020 Oral (Acceptance: 105/9454 ≈ 1.1%)

    [ paper / code ]

  • Vehicular multi-camera sensor system for automated visual inspection of electric power distribution equipment

    Jinsun Park, Ukcheol Shin, Gyumin Shim, Kyungdon Joo, Francois Rameau, Junhyeok Kim, Dong-Geol Choi, In So Kweon

    IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS), 2019 Oral

    Media coverage: covered by MBN, EnergyDaily, and many local media

    [ paper ]

  • Camera exposure control for robust robot vision with noise-aware image quality assessment

    Ukcheol Shin, Jinsun Park, Gyumin Shim, Francois Rameau, In So Kweon

    IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS), 2019 Oral

    [ paper / dataset / code / project ]

  • Performance evaluation of real-time mechanisms on open embedded hardware platforms

    Ukcheol Shin, Byoung Wook Choi

    Journal of Institute of Control, Robotics and Systems, 2017

    [ paper ]

  • Development and control of an omnidirectional mobile robot on an ethercat network

    Raimarius Delgado, UkCheol Shin, Chang Hwi Hong, Byoung Wook Choi

    International Journal of Applied Engineering Research, 2016

    [ paper ]

Awards & Honors

  • 1st Place Award, CVPR 2024 Autonomous Driving Workshop "Argoverse Scene Flow Challenge", June 2024
  • Honorable Mention, 29th HumanTech Paper Award, Samsung Electronics Co., Ltd. Feb. 2023 ($2,000)
  • Best Student Paper Award, IEEE/CVF Winter Conference on Applications of Computer Vision (WACV), Jan. 2023
  • Graduation with Honors (Top 3%), SNUST in Electrical and Information Engineering, Feb. 2017
  • Honorable Mention, All-semester Design Based Learning (ADBL) Capstone Contest, June 2016
  • Second Prize, Robot Open Academy, Feb. 2016
  • Grand Prize, All-semester Design Based Learning (ADBL) Capstone Contest, Dec. 2015
  • Military Service, Army Sergeant, Honorable discharge, Feb. 2013 - Nov. 2014
  • Scholarship for Academic Excellence, Scholarship for the entire B.S. program (2011-2012, 2015-2016)

Academic Activities

Journal Reviewer

  • IEEE Transactions on Cybernetics (T-CYB)(2022)
  • IEEE Transactions on Intelligent Vehicles (T-IV)(2023,2024)
  • IEEE Robotics and Automation Letters (RA-L) (2022,2023,2024)
  • ACM Transactions on Multimedia Computing, Communications, and Applications (T-OMM)(2024)
  • Pattern Recognition (PR) (2023)
  • Neural Processing Letters (2023)

Conference Reviewer

  • IEEE International Conference on Computer Vision and Pattern Recognition (CVPR) (2022~)
  • IEEE International Conference on Computer Vision (ICCV) (2021,2023)
  • European Conference on Computer Vision (ECCV) (2024)
  • IEEE/CVF Winter Conference on Applications of Computer Vision (WACV) (2023)
  • IEEE International Conference on Robotics and Automation (ICRA) (2024~)
  • Robotics: Science and Systems (RSS) (2024~)
  • Conference on Neural Information Processing Systems (NeurIPS) (2021~)
  • International Conference on Machine Learning (ICML) (2022~)
  • International Conference on Learning Representations(ICLR) (2022~)
  • AAAI Conference on Artificial Intelligence (AAAI) (2021~)

Invited Talks

  • KAIST (Korea Advanced Institute of Science and Technology), Sep. 2024
  • Hanyang University, May 2024
  • SNU (Seoul National University), Feb. 2024
  • GIST (Gwangju Institute of Science and Technology), Sep. 2023
  • UNIST (Ulsan National Institute of Science and Technology), July 2023
  • Pusan National University, July 2023