Fugue

Speed Up Iteration and Cut Computation CostMany data teams begin by establishing an analytics practice on data warehouses such as BigQuery. However, solely relying on BigQuery for data science workloads may not be the best approach due to various reasons:Advanced needs beyond SQL: Use cases such as data validation, visualization, and machine learning forecasting may require more advanced functionalities beyond the limitations of SQL grammar.Costly for exploration: BigQuery may not be the most cost-effective solution for…

Profiling large-scale data for use cases such as anomaly detection, drift detection, and data validationHusky with a magnifying glass — Image by AuthorData pipelines have the potential to produce unexpected results in a variety of ways. Anomalous data can cause data to be scaled incorrectly. Machine learning model drift can lead to reduced prediction accuracy. Failures in the upstream collection could cause null values as the data pipeline executes. How do we safeguard against these failure cases?One way to monitor the…

Scalable Time Series Modeling with open-source projects StatsForecast, Fugue, and SparkBy Kevin Kho, Han Wang, Max Mergenthaler and Federico Garza Ramírez.TL:DR We will show how you can leverage the distributed power of Spark and the highly efficient code from StatsForecast to fit millions of models in a couple of minutes.Time-series modeling, analysis, and prediction of trends and seasonalities for data collected over time is a rapidly growing category of software applications.Businesses, from electricity and economics…