Understanding Transformers! (Part 1: Model Architecture)

In the past, we’ve had different architectures for different modalities of data like CNNs for Images, RNNs for Text and GNNs for Graphs. Recently we have seen the adoption of Transformers for processing all types of data modalities. Transformers can be thought of like general purpose trainable architecture. Since the adoption of transformers has been growing so rapidly, It might be a good idea to revisit the original paper. Introduction In sequence modelling and transduction problems, recurrent neural networks, and in particular extended short-term memory and gated recurrent neural networks, have become standard....

October 31, 2022 · 10 min · 2033 words · Me

Understanding the mathematical foundations and applications of Graph Convolutional Networks

Recent advances in neural networks have driven study of data mining and pattern recognition. End-to-end deep learning models including convolutional neural networks (CNNs), recurrent neural networks (RNNs), and autoencoders have breathed new life into traditional machine learning tasks such as object detection, machine translation, and speech recognition. Deep Learning is effective at uncovering hidden patterns among Euclidean data (images, text, videos). But what about applications that rely on data originating from non-Euclidean domains, which is typically represented as a graph with intricate interdependencies and relationships among objects?...

June 19, 2021 · 8 min · 1496 words · Me

RUL(Remaining Useful Life) and SOH(State of Health) estimation of Lithium-ion satellite power systems using Support-Vector-Regression

Batteries are a critical part of any satellite’s power system. They are used to provide power: During launch and after the launch of the satellite till the solar panels are deployed To the spacecraft, its equipment, and payload during the shadow phase For communication and data transfer To maintain the electronics at a specific temperature Batteries with higher gravimetric(higher mass) and volumetric(higher volume) energy densities lead to lesser mass and volume for the power systems and thereby increase payload and mission capabilities....

September 16, 2019 · 9 min · 1810 words · Me