Rameswar Panda

I am a research scientist at MIT-IBM Watson AI Lab, where I work on computer vision and machine learning.

I received my Ph.D. from UC Riverside in 2018, under the supervision of Prof. Amit K. Roy-Chowdhury. During Ph.D., I was very fortunate to have interned at NEC Labs, Adobe Research and Siemens Research.

Email  /  CV  /  Google Scholar  /  LinkedIn

My research interests mainly lie in the areas of computer vision, machine learning and more recently natural language processing, with a special focus on efficient AI, including data efficiency and model efficiency. I am also interested in image/video understanding, unsupervised/self-supervised representation learning and multimodal learning.

We develop a new framework for learning discriminative and invariant feature representation while preventing intrinsic negative transfer in partial domain adaptation.

We propose an auto-encoder-based approach combined with a simultaneous learning scheme to align source and target features for semi-supervised domain adaptation.

We propose a new benchmark, SynAPT, for studying the transferability of synthetic video representations for action recognition.

We introduce FETA benchmark to understand technical documentations, via learning to match their graphical illustrations to corresponding language descriptions.

We demonstrate the performance disparities across different subgroups for selective regression and develop novel methods to mitigate such disparities.

We introduce a visual hallucination framework which requires only source sentences at inference time and instead uses hallucinated visual representations for multimodal machine translation.

We propose a unified model mapping downstream task representations to optimal simulation parameters to generate synthetic pre-training data for them.

We propose a new architecture that adopts the pyramid structure and employ a regional-to-local attention rather than global self-attention in vision transformers.

We introduce the idea of rearranging input video frames into super images to re-purpose an image classifer for video action recognition.

We introduce an input-dependent and interpretable dynamic inference framework for vision transformer, which adaptively decides the patch tokens to compute per input instance.

We propose a new contrastive learning framework that aims to learn discriminative invariant feature representations for unsupervised video domain adaptation.

We address the problem of cross-domain few-shot learning by proposing a simple distillation-based method to facilitate unlabeled images from the novel dataset.

We propose an adaptive multi-modal learning framework that selects on-the-fly the optimal modalities for each segment conditioned on the input for efficient video recognition.

We introduce video instance-aware quantization that decides what precision should be used on a per frame basis for efficient video inference.

We develop a dual-branch vision transformer by combining image patches of different sizes to extract multi-scale feature representations for image classification.

We conduct extensive analysis on the transferability of contrastive learning on the downstream image classification, few-shot recognition, and object detection tasks.

We extend the concept of instance-level contrastive learning with a multimodal clustering step in the training pipeline to capture semantic similarities across modalities.

We propose a detector-free approach for weakly supervised grounding by learning to separate randomly blended images conditioned on the corresponding texts.

We prove that sufficiency can be used to train fairer selective classifiers which ensure that precision always increases as coverage is decreased for all groups.

We develop a unified framework for action recognition models and systematically compare them to better understand the differences and spatio-temporal behavior on large-scale datasets.

We propose a temporal contrastive learning framework for semi-supervised action recognition by using contrastive losses between different videos and groups of videos with similar actions.

We introduce an adaptive temporal fusion network that dynamically fuses channels from current and past feature maps for strong temporal modelling in action recognition.

We propose an input-dependent adaptive framework for efficient video recognition that automatically decides what feature maps to compute per input instance.

We analyze the architecture transferability of different NAS methods by performing a series of experiments on several large scale image benchmarks.

We propose a NAS method based on polyharmonic splines that can perform architecture search directly on large scale image datasets like ImageNet22K.

We propose a novel approach for adaptively determining the feature sharing pattern across multiple tasks (what layers to share across which tasks) in deep multi-task learning.

We propose a consistent cross-view matching framework, in which global camera network constraints are exploited to address the problem of unsupervised video-based re-identification.

We present a novel adversarial framework for transferring knowledge from internet-scale unlabeled data to improve the performance of a classifier on a given visual recognition task.

We propose an adaptive approach to select optimal resolution for each frame conditioned on the input for efficient action recognition in long untrimmed video.

We introduce a joint dataset repairment strategy by combining classifier with a GAN that makes up for the deficit of training examples from the minority class by producing additional examples.

We present an approach to estimate the consistency in performance of classifiers across populations with varying skin tones in skin disease benchmarks.

We introduce a novel non-adversarial framework for generating a wide range of diverse videos from latent noise vectors without any any conditional input reference frame.

We propose an approach to swiftly on-board new camera(s) in an existing re-id network using only source models and limited labeled data, but without having access to source camera data.

We introduce a novel approach that focuses on object relations to effectively transfer knowledge for minimizing the effect of catastrophic forgetting in incremental learning of object detectors.

This paper extends our CVPR 2017 paper providing a new source-target selective adaptation strategy and rigorous experiments on more person re-id datasets.

This paper extends our MM 2016 paper where we employ a joint visual-semantic space to simultaneously utilize both images and text from the web for dataset construction.

We present an approach to estimate skin tone in benchmark skin disease datasets, and investigate whether model performance is dependent on this measure

We investigate different dataset biases and propose a curriculum guided webly supervised approach for learning a generalizable emotion recognition model.

This work exploits large scale web data for learning an effective multi-modal embedding without requiring large amount of human-crafted training data.

We introduce an online framework for fast-forwarding a video while presenting its important and interesting content on the fly without processing or even obtaining the entire video.

We introduce a weakly supervised approach that requires only video-level annotations for summarizing long unconstrained web videos.

We propose an unsupervised adaptation scheme for re-identification models where a new camera may be temporarily inserted into an existing system to get additional information.

This paper presents a collaborative video summarization approach that exploits visual context from a set of topic-related videos to extract an informative summary of a given video.

This paper extends our ICPR 2016 paper providing new theoretical insights with a joint optimization and experimenting on spatio-temporal features and datasets.

This paper introduces a new generalized sparse optimization framework for summarizing multiple videos generated from a video search or from a multi-view camera network.

We present a fast solution for movie scene detection using Nystrom approximated multi-similarity spectral clustering with a temporal integrity constraint.

This paper presents a diversity-aware sparse optimization framework for summarizing topi-related videos generated from a video search.

We addressed the problem of online learning of identification systems where unlabeled data comes in small minibatches, with human in the loop.

This paper presents a framework for summarizing multi-view videos by exploiting both intra- and inter-view content correlations in a joint embedding space.

This paper presents a stochastic multi-view frame embedding based on KL divergence to preserve correlations in multi-view learning.

This paper presents a semi-supervised sparse coding framework which can be used to both create a dataset from scratch or enrich an existing dataset with diverse examples.

We present a continuous learning re-id system with a human in the loop which not only provides image labels but also improves the learned model by providing attribute based explanations..

This paper presents a scalable video summarization framework for both the analysis of the input video as well as the generation of summaries according to user-specified length constraints.

This paper extends our ICPR 2012 paper providing new theoretical insights and experiments on more datasets including different key frame visualization techniques.

This paper uses dynamic Delunay grpah clustering for summarizing videos.

• Workshop Organizer: Dynamic Neural Networks (DNetCV) at CVPR 2022, Dynamic Neural Networks (DNetCV) at CVPR 2021, Neural Architecture Search (NAS) at CVPR 2021, Multi-Modal Video Analysis at ECCV 2020, Neural Architecture Search (NAS) at CVPR 2020, Multi-modal Video Analysis and Moments in Time Challenge at ICCV 2019.
  
• Tutorial Organizer: Efficient Video Understanding at ICCV 2021, Visual Data Summarization at CVPR 2019.
  
• Reviewer: CVPR, ICCV, ECCV, ICLR, NeurIPS, AAAI, WACV, ACCV, BMVC, TPAMI, TIP, TMM, TCSVT, IJCV, PR, PRL.
  

* Equal Contribution  /  Website template from this great guy!