Hi, I recently joined Amazon Ring as an Applied Scientist on July 22, 2024. Prior to this, I completed my PhD final exam in Computer Science at the School of Electrical Engineering and Computer Science, Oregon State University, where I was advised by Prof. Sinisa Todorovic. My research primarily focused on Computer Vision, exploring areas such as Object Detection, Image and Video Semantic/Instance Segmentation, Object Tracking, Cross-domain Learning, Video Understanding, and Vision Language Models.

News


Publications


    Bidirectional Alignment for Domain Adaptive Detection with Transformers (ICCV 23)

    Liqiang He, Wei Wang, Albert Chen, Min Sun, Cheng-hao Kuo, Sinisa Todorovic
    [PDF] [Code] [Poster]

    We propose a Bidirectional Alignment for domain adaptive Detection with Transformers (BiADT) to improve cross domain object detection performance. Existing adversarial learning based methods use gradient reverse layer (GRL) to reduce the domain gap between the source and target domains in feature representations. Since different image parts and objects may exhibit various degrees of domain-specific characteristics, directly applying GRL on a global image or object representation may not be suitable. Our proposed BiADT explicitly estimates token-wise domain-invariant and domain-specific features in the image and ob- ject token sequences. BiADT has a novel deformable attention and self-attention, aimed at bi-directional domain alignment and mutual information minimization. These two objectives reduce the domain gap in domain-invariant representations, and simultaneously increase the distinctive- ness of domain-specific features. Our experiments show that BiADT achieves very competitive performance to SOTA consistently on Cityscapes-to-FoggyCityscapes, Sim10K-to-Citiscapes and Cityscapes-to-BDD100K, outperforming the strong baseline, AQT, by 1.9, 2.1, and 2.4 in mAP50, respectively.

    Automatic estimation of trunk cross sectional area using deep learning (ECPA 23)

    T Wang, P Sankari, J Brown, A Paudel, L He, M Karkee, A Thompson, C Grimm, JR Davidson, S Todorovic
    [PDF] [Code] [Poster]

    This paper presents an automated method for estimating the trunk cross sectional area of fruit trees. An Intel RealSense 435i was used to capture RGB images and point clouds of individual trunks. To segment the trunk in the image from the background, a Maskedattention Mask Transformer model was adopted. The segmentation results were integrated with the 3D point cloud to estimate trunk widths in 3D. The width estimation was evaluated on three diverse datasets collected from a commercial apple orchard using human measurements as ground truth. With a mean absolute error less than 5%, the method is sufficiently accurate to assist orchard operations.

    DESTR: Object Detection with Split Transformer (CVPR 22)

    Liqiang He, Sinisa Todorovic
    [PDF] [Code] [Poster]

    Self- and cross-attention in Transformers provide for high model capacity, making them viable models for object detection. However, Transformers still lag in performance behind CNN-based detectors. This is, we believe, because: (a) Cross-attention is used for both classification and bounding-box regression tasks; (b) Transformer's decoder poorly initializes content queries; and (c) Self-attention poorly accounts for certain prior knowledge which could help improve inductive bias. These limitations are addressed with the corresponding three contributions. First, we propose a new Detection Split Transformer (DESTR) that separates estimation of cross-attention into two independent branches -- one tailored for classification and the other for box regression. Second, we use a mini-detector to initialize the content queries in the decoder with classification and regression embeddings of the respective heads in the mini-detector. Third, we augment self-attention in the decoder to additionally account for pairs of adjacent object queries. Our experiments on the MS-COCO dataset show that DESTR outperforms DETR and its successors.

    A polar-edge context-aware (PECA) network for mirror segmentation (Image and Vision Computing, 22)

    Liqiang He, Lu Xia, Jiajia Luo, Ke Zhang, Yuyin Sun, Nan Qiao, Cheng-Hao Kuo, Sinisa Todorovic
    [PDF]

    This paper presents Polar-Edge Contrast-Aware Network (PECA) for mirror instance segmentation in images of indoor scenes. General instance segmentation methods typi- cally rely on the surface appearance of the object to identify the foreground from background. However, these approaches do not directly apply to mirrors as the mirror surfaces are less reliable due to reflections of the surroundings. On the other hand, the existing saliency-based mirror segmentation methods are prone to predict false positives in images with no mirrors, especially for the indoor scenes that have mirror-shape objects, such as doors and windows. In this work, we propose a novel boundary-based mirror localization method PECA that achieves both high segmentation accuracy on mirrors and low false positive rate on negative samples. PECA uses a context-aware module to extract features along the instance contour and in this way incorporates boundary information for improving mirror detection. The predicted mirror candidates are further refined with a local contrast module for the final mirror instance segmentation. PECA achieves IoU 80.29% on the benchmark Mirror Segmentation dataset (MSD), outperforming the state-of-the-art method MirrorNet (IoU=78.95%) by 1.34%. It also produces a significantly smaller false positive rate (43.37%) than existing methods (91.39%) on our challenging Negative Mir- ror Dataset (NMD) without retraining. After training on both MSD and NMD training sets, our model further reduces the false positive rate to 0.08% on NMD testing set, while keeping IoU of 73.07% on MSD, enabling realistic real-world applications.
  • Rebecca Hutchinson, Liqiang He , Sarah Emerson. "Species distribution modeling of citizen science data as a classification problem with class-conditional noise.", Proceedings of the AAAI Conference on Artificial Intelligence 31. [PDF]