Jessie Hobb 7582 posts 0 comments

Understanding why LLMs like GPT-3 work so well…(Photo by Waldemar Brandt on Unsplash)Language models (LMs) are incredibly generic–they take text as input and produce text as output. Recent research has revealed that this generic text-to-text structure can be exploited to solve a variety of tasks without task-specific adaptation (i.e., no fine-tuning or architectural modifications) by using prompting techniques to perform accurate zero and few-shot inference. Put simply, we can pre-train the LM over a large, unlabeled text…

Improving your big-data analysis workflows with an open-source libraryIf you’re a data scientist working with large datasets, you must have run out of memory (OOM) when performing analytics or training machine learning models.Supercharge your workflow. Photo by Cara Fuller on UnsplashThat’s not surprising. The memory available on a desktop or laptop computer can easily exceed large datasets, making loading the entire dataset nearly impossible. We are forced to work with only a small subset of data at a time, which can…

Finding the coefficients that maximize the log-partial likelihood in PythonLog-partial likelihood of the Cox model with local optimum. Image by author.IntroductionThe Cox proportional hazard modelOptimization problemImplementationConclusionsReferencesSurvival analysis encompasses a collection of statistical methods for describing time to event data.In this post, we introduce a popular survival analysis algorithm, the Cox proportional hazards model¹. Then, we define its log-partial likelihood and the gradient, and optimize…

and Data warehouse Gitflow pipelines to run it automaticallyimage by authorDo you unit test your data warehouse scripts?I am going to talk about unit tests for complex SQL queries which might consist of multiple operations (actions).I tried it before using BigQuery scripting:However, here is another (and probably better) way to do it.Dataform is a great free tool for SQL data transformation. It helps to keep your data warehouse clean and well-organized. It has nice dependency graphs explaining data lineage and it serves…

PLAXIS AUTOMATION SERIESStep-by-step guide toward automationPhoto by Hitesh Choudhary on UnsplashPLAXIS is a Finite Element modelling software that geotechnical engineers will definitely come across at a certain point in their careers. Modelling complex geotechnical problems in a PLAXIS can be challenging and represents time-consuming modelling work often involving repetitive tasks.As an engineer, we always want to automate the boring and engineer the awesome.In this context, programming, more specifically Python, is the…

PLAXIS AUTOMATION SERIESStep-by-step guide toward automationPhoto by Mika Baumeister on UnsplashIf you are a frequent PLAXIS user like me, you must have asked this question: why on earth I need to keep repeating this copy and paste action to extract results?PLAXIS, with no doubt, is a powerful FE software. As Geotechnical engineers, we want to focus on the analysis and spend less time on post-processing. Python provides the most extensive possibilities toward automating output extraction and visualisation, thanks to the…

PLAXIS AUTOMATION SERIESStep-by-step guide toward automationPhoto by Luke Chesser on UnsplashThe PLAXIS Output app offers the possibility of using the built-in plotting tool to compare outputs at different phases. Although it provides some basic plot functions which allow users to visualise results, people often go with Excel as it allows flexibility in plot formatting, data manipulation and results sharing.With no doubt, Excel is a powerful and well-known software for data visualisation. Though, PLAXIS does not provide…

Road to Reinforcement LearningCover photo generated by the author using an AI tool Dreamstudio (Licenses as https://creativecommons.org/publicdomain/zero/1.0/)Reinforcement learning (RL) is a type of machine learning in which an agent learns to interact with its environment by trial and error in order to maximize a reward. It is different from supervised learning, in which an agent is trained on labeled examples, and unsupervised learning, in which an agent learns to identify patterns in unlabeled data. In reinforcement…

An introduction to exponential smoothing and implementing it in Python to carry out forecastsPhoto by Shot by Cerqueira on UnsplashThe area of forecasting contains sundry models. In my previous post we discussed some basic forecasting techniques, whom often provide a good base to iterate from and build more complex models. One such natural iteration from these basic models is exponential smoothing.The idea of exponential smoothing originated in the 1950s and is basically a fancy way of stating that we will put more weight…

Learn the basics of Python, including data types, flow control statements, functions, and modulesPhoto by Jefferson Santos on UnsplashPython is a popular, high-level programming language known for its simplicity, readability, and flexibility. It is a versatile language that can be used for a wide range of applications, from web development and scientific computing to data analysis and artificial intelligence.To start using Python, you need to have it installed on your computer. The latest version of Python can be…