Publications

2021

1. Incremental Learning for Animal Pose Estimation using RBF k-DPP Nayak, Gaurav Kumar; Shah, Het; and Chakraborty, Anirban British Machine Vision Conference – BMVC 2021 2021 [Abs] [arXiv] [PDF]

   Pose estimation is the task of locating keypoints for an object of interest in an image. Animal Pose estimation is more challenging than estimating human pose due to high inter and intra class variability in animals. Existing works solve this problem for a fixed set of predefined animal categories. Models trained on such sets usually do not work well with new animal categories. Retraining the model on new categories makes the model overfit and leads to catastrophic forgetting. Thus, in this work, we propose a novel problem of "Incremental Learning for Animal Pose Estimation". Our method uses an exemplar memory, sampled using Determinantal Point Processes (DPP) to continually adapt to new animal categories without forgetting the old ones. We further propose a new variant of k-DPP that uses RBF kernel (termed as "RBF k-DPP") which gives more gain in performance over traditional k-DPP. Due to memory constraints, the limited number of exemplars along with new class data can lead to class imbalance. We mitigate it by performing image warping as an augmentation technique. This helps in crafting diverse poses, which reduces overfitting and yields further improvement in performance. The efficacy of our proposed approach is demonstrated via extensive experiments and ablations where we obtain significant improvements over state-of-the-art baseline methods.

2. Empirical Study of Data-Free Iterative Knowledge Distillation Shah, Het; Vaswani, Ashwin; Dash, Tirtharaj; Hebbalaguppe, Ramya; and Srinivasan, Ashwin Artificial Neural Networks and Machine Learning – ICANN 2021 2021 [Abs]

   Iterative Knowledge Distillation (IKD) [20] is an iterative variant of Hinton’s knowledge distillation framework for deep neural network compression. IKD has shown promising model compression results for image classification tasks where a large amount of training data is available for training the teacher and student models. In this paper, we consider problems where training data is not available, making it impractical to use the usual IKD approach. We propose a variant of the IKD framework, called Data-Free IKD (or DF-IKD), that adopts recent results from data-free learning of deep models [2]. This exploits generative adversarial networks (GANs), in which a readily available pre-trained teacher model is regarded as a fixed discriminator, and a generator (a deep network) is used to generate training samples. The goal of the generator is to generate samples that can obtain a maximum predictive response from the discriminator. In DF-IKD, the student model at every IKD iteration is a compressed version of the original discriminator (‘teacher’). Our experiments suggest: (a) DF-IKD results in a student model that is significantly smaller in size than the original parent model; (b) the predictive performance of the compressed student model is comparable to that of the parent model.

2020

1. KD-Lib: A PyTorch library for Knowledge Distillation, Pruning and Quantization Shah, Het; Khare, Avishree; Shah, Neelay; and Siddiqui, Khizir arXiv preprint 2020 [Abs] [arXiv] [PDF] [Code]

   In recent years, the growing size of neural networks has led to a vast amount of research concerning compression techniques to mitigate the drawbacks of such large sizes. Most of these research works can be categorized into three broad families : Knowledge Distillation, Pruning, and Quantization. While there has been steady research in this domain, adoption and commercial usage of the proposed techniques has not quite progressed at the rate. We present KD-Lib, an open-source PyTorch based library, which contains state-of-the-art modular implementations of algorithms from the three families on top of multiple abstraction layers. KD-Lib is model and algorithm-agnostic, with extended support for hyperparameter tuning using Optuna and Tensorboard for logging and monitoring. The library can be found at - https://github.com/SforAiDl/KD_Lib

2. An Autoencoder Based Approach to Simulate Sports Games Vaswani, Ashwin; Ganguly, Rijul; Shah, Het; S, Sharan Ranjit; Pandit, Shrey; and Bothara, Samruddhi 7th Workshop on Machine Learning and Data Mining for Sports Analytics at ECML-PKDD 2020 2020 [Abs] [arXiv] [PDF] [Code]

   Sports data has become widely available in the recent past. With the improvement of machine learning techniques, there have been attempts to use sports data to analyze not only the outcome of individual games but also to improve insights and strategies. The outbreak of COVID-19 has interrupted sports leagues globally, giving rise to increasing questions and speculations about the outcome of this season’s leagues. What if the season was not interrupted and concluded normally? Which teams would end up winning trophies? Which players would perform the best? Which team would end their season on a high and which teams would fail to keep up with the pressure? We aim to tackle this problem and develop a solution. In this paper, we proposeUCLData, which is a dataset containing detailed information of UEFA Champions League games played over the past six years. We also propose a novel autoencoder based machine learning pipeline that can come up with a story on how the rest of the season will pan out.