Stochastic

Deep Dive on Stochastic Gradient Descent. Algorithm, assumptions, benefits, formula, and practical implementationImage by DALL-E-2IntroductionThe image above is not just an appealing visual that drew you to this article (despite its length), but it also represents a potential journey of the SGD algorithm in search of a global minimum. In this journey, it navigates rocky paths where the height symbolizes the loss. If this doesn’t sound clear now, don’t worry, it will be by the end of this article.Index:· 1: Understanding…

Deterministic trends vs stochastic trends, and how to deal with themPhoto by Ali Abdul Rahman on UnsplashDetecting and dealing with the trend is a key step in the modeling of time series.In this article, we’ll:Describe what is the trend of a time series, and its different characteristics;Explore how to detect it;Discuss ways of dealing with trend;Trend as a building block of time seriesAt any given time, a time series can be decomposed into three parts: trend, seasonality, and the remainder.Additive decomposition of a…

Why we let randomness dictate our action selection in Reinforcement LearningRock-paper-scissors would be a boring affair with deterministic policies If you are used to deterministic decision-making policies (e.g., as in Deep Q-learning), the need for and use of stochastic policies might elude you. After all, deterministic policies offer a convenient state-action mapping π:s ↦ a, ideally even the optimal mapping (that is, if all the Bellman equations are learned to perfection).In contrast, stochastic policies — represented…

Part 1: On the Markov Decision Model which forms the theoretical foundation of reinforcement learning problemsExample of a simple MDP with three states (green circles) and two actions (orange circles), with two rewards (Image source: Wikipedia)Reinforcement learning (RL) is a type of machine learning that enables an agent to learn to achieve a goal in an uncertain environment by taking actions. An important aspect of reinforcement learning is that it evaluates the actions taken rather than instructs by giving correct…

Photo by Taylor Vick on UnsplashFormulating and solving a two-stage stochastic server farm problemStochastic programming (SP) is a framework for modeling optimization problems that involve uncertainty . In many cases, SP models take the form of a two-stage problem. The first stage involves finding the optimal deterministic decisions. These decisions are based on information we know to be certain (AKA the here-and-now decisions). The second stage involves making decisions that rely on randomness (also called the recourse…

A week and a half before the midterm elections, a man broke into Speaker Nancy Pelosi’s house, screaming “Where’s Nancy?” and attacked her husband with a hammer. David DePape, charged in the attack, had posted a slew of rants that included references to a sprawling conspiracy theory known as QAnon, which claims that Democratic, Satan-worshipping pedophiles are trying to control the world’s politics and media. Several hours before, Fox News’s Tucker Carlson interviewed right-wing activist Christopher Rufo, who claimed…

One tip to find regular patterns like sine waves from randomness.Anomaly detection in time series data is a common task in data science. We treat anomalies as data patterns that exist not as expected. Today, let’s focus on detecting anomalies in a special univariate time series generated by a stochastic process.The data should look noisy, chaotic, and random in those stochastic time series. Unexpected changes should be happening all the time. If the value is not changing or changing with a deterministic pattern, something…

Continuity of probability measure, Radon-Nikodym derivative, and Girsanov theoremImage from UnsplashThe Girsanov theorem and Radon-Nikodym theorem are frequently used in financial mathematics for the pricing of financial derivatives. And they are deeply related. The usage of those theorems can also be found in machine learning (very theoretical though). An example is given in , where Girsanov’s theory is applied in a new policy gradient algorithm for reinforcement learning.However, those theorems are far from intuitive…