A pipeline for fast experimentation on Kubernetes | by Pascal Janetzky | Jan, 2023
Using native Python packages onlyManually creating a novel configuration file for every new experiment is a tedious process. Especially if you want to rapidly deploy a vast number of jobs on a Kubernetes cluster, an automated setup is a must. With python, it’s straightforward to build a simple scheduling script that reads an experiment’s configuration, such as the batch size, writes it into the YAML file, and creates a new job. In this post, we’ll discuss the how. The best is that we require no additional packages!Photo…