Bhadani

Road to Reinforcement LearningCover photo generated by the author using an AI tool Midjourney (Licenses as Creative Commons Noncommercial 4.0 asset license)Through the previous two articles: (1) Markov States, Markov Chain, and Markov Decision Process, and (2) Solving Markov Decision Process, I set up a foundation for developing a detailed concept of reinforcement learning (RL). The RL problem is formulated as Markov Decision Process (MDP) which can be solved for optimal policies (i.e. what action needs to be taken by an…

Road to Reinforcement LearningCover photo generated by the author using an AI tool Dreamstudio (Licenses as https://creativecommons.org/publicdomain/zero/1.0/)In the first part, I discussed some basic concepts to establish a foundation for reinforcement learning (RL) such as Markov states, the Markov chain, and the Markov decision process (MDP). Reinforcement learning problems are built on top of MDP.An MDP is a 4-tuple model (𝓢, 𝓐, 𝓟, 𝓡) where s ∈ 𝓢 is a state, a ∈ 𝓐 is an action taken while an agent is a state s, 𝓟(s’ |…

Road to Reinforcement LearningCover photo generated by the author using an AI tool Dreamstudio (Licenses as https://creativecommons.org/publicdomain/zero/1.0/)Reinforcement learning (RL) is a type of machine learning in which an agent learns to interact with its environment by trial and error in order to maximize a reward. It is different from supervised learning, in which an agent is trained on labeled examples, and unsupervised learning, in which an agent learns to identify patterns in unlabeled data. In reinforcement…

A commonly overlooked topic by machine learning practitionersCover photo generated by the author using an AI tool Dreamstudio.Data sampling is at the core of data science. From a given population f(x), we sample data points. All these data points are collectively called random samples denoted by random variable X. But as we know, data science is a game of probability, often, we repeat the experiment many times. In such a scenario, we end up with n random samples X₁, X₂, … Xₙ (not to be confused with the number of data…

Tales of geometry for information theory and machine learningThe image is drawn using an AI tool by the AuthorIn high school, some of us had love-hate relationships with geometry, especially coordinate and 3D geometry. Even more, calculus with geometry was frowned upon. Then came a boom in information technology followed by a craze for machine learning, artificial intelligence, and data science. The cause has inspired many to dig deeper into some mystics of mathematics and information geometry is one of them. Information…

Generative Models for Tractable DistributionsLake Arrowhead in California, Picture by the AuthorIn my previous episodes of the Stat Stories series, I talked about methods of variable transformation to generate new distribution. The discussion on the variable transformation, both for univariate and multivariate distributions leads to Normalizing Flows.I recommend reading the discussion on variable transformation for generating new distributions as a prerequisite to understanding Normalizing Flows.A big challenge in…

Orthogonal Polynomials for Data ScienceThe picture was generated by the author using an AI tool.Legendre polynomials of degrees 1 through 6: Picture generated by AuthorOrthogonal polynomials are a useful tool for solving and interpreting many times of differential equations. Further, they are handy mathematical tools for least square approximations of a function, difference equations, and Fourier series. Another big application of the orthogonal polynomial is error-correcting code and sphere packing. Some other obscure…

A Precursor to Normalizing FlowsPicture taken by Author in San Bernardino, CaliforniaIn a previous episode of Stat Stories, I discussed variable transformation for a univariate continuous distribution. Such variable transformation is essential for generating new and complex distributions from a simpler one. However, the discussion was limited to a single variable. In this article, we will discuss the transformation of bivariate distribution. Understanding the mechanism of multivariate transformation is the first step…

Tools to create models for your dataThe University of Arizona Main Library. Picture taken by the authorIn part 1 of “Common Families of Statistical Distributions”, we saw families of discrete distributions that help in modeling events such as the arrival of photons, population size estimation, acceptance sampling, etc. In part 2 of the families of distributions, we will look at continuous statistical distribution where random variable X can take any real number ℝ.Continuous distributions can be used to model physical…

Tools to create models for your dataENR2 Building (that houses the Mathematics and Statistics & Data Science program), The University of Arizona. Photo taken by the authorAs a data scientist, statistician, computer engineer, or data analyst, people are dealing with a deluge of data that is being obtained from a variety of sources, through a number of physical processes and encompasses a wide variety of domains including transportation, photonics, bioinformatics, and astronomy. Statisticians and Data Scientists spend a…