Integrating equation solvers with probabilistic programming through differentiable programming
November 24 2022 in Julia, Programming, Science, Scientific ML | Tags: differential equations, julia, probabilistic programming, scientific machine learning, sciml | Author: Christopher Rackauckas
Part of the COMPUTATIONAL ABSTRACTIONS FOR PROBABILISTIC AND DIFFERENTIABLE PROGRAMMING WORKSHOP
Abstract: Many probabilistic programming languages (PPLs) attempt to integrate with equation solvers (differential equations, nonlinear equations, partial differential equations, etc.) from the inside, i.e. the developers of the PPLs like Stan provide differential equation solver choices as part of the suite. However, as equation solvers are an entire discipline to themselves with many active development communities and subfields, this places an immense burden on PPL developers to keep up with the changing landscape of tens of thousands of independent researchers. In this talk we will explore how Julia PPLs such as Turing.jl support of equation solvers from the outside, i.e. how the tools of differentiable programming allows equation solver libraries to be compatible with PPLs … READ MORE
Learning Epidemic Models That Extrapolate, AI4Pandemics
July 25 2021 in Differential Equations, Julia, Mathematics, Science, Scientific ML | Tags: epidemics, scientific machine learning, sciml | Author: Christopher Rackauckas
I think this talk was pretty good so I wanted to link it here!
Title: Learning Epidemic Models That Extrapolate
Speaker Chris Rackauckas, https://chrisrackauckas.com/
Abstract:
Modern techniques of machine learning are uncanny in their ability to automatically learn predictive models directly from data. However, they do not tend to work beyond their original training dataset. Mechanistic models utilize characteristics of the problem to ensure accurate qualitative extrapolation but can lack in predictive power. How can we build techniques which integrate the best of both approaches? In this talk we will discuss the body of work around universal differential equations, a technique which mixes traditional differential equation modeling with machine learning for accurate extrapolation from small data. We will showcase how incorporating different variations of the technique, such as … READ MORE
COVID-19 Epidemic Mitigation via Scientific Machine Learning (SciML)
July 7 2020 in Differential Equations, Julia, Mathematics, Programming, Science, Scientific ML | Tags: covid-19, epidemic modeling, scientific machine learning, sciml | Author: Christopher Rackauckas
Chris Rackauckas
Applied Mathematics Instructor, MIT
Senior Research Analyst, University of Maryland, Baltimore School of Pharmacy
This was a seminar talk given to the COVID modeling journal club on scientific machine learning for epidemic modeling.
Resources:
https://sciml.ai/
https://diffeqflux.sciml.ai/dev/
https://datadriven.sciml.ai/dev/
https://docs.sciml.ai/latest/
https://safeblues.org/
Cheap But Effective: Instituting Effective Pandemic Policies Without Knowing Who’s Infected
July 2 2020 in Biology, Differential Equations, Julia, Mathematics, Science, Scientific ML | Tags: covid-19, scientific machine learning, sciml | Author: Christopher Rackauckas
Cheap But Effective: Instituting Effective Pandemic Policies Without Knowing Who’s Infected
Chris Rackauckas
MIT Applied Mathematics Instructor
One way to find out how many people are infected is to figure out who’s infected, but that’s working too hard! In this talk we will look into cheaper alternatives for effective real-time policy making. To this end we introduce SafeBlues, a project that simulates fake virus strands over Bluetooth and utilizes deep neural networks mixed within differential equations to accurately approximate infection statistics weeks before updated statistics are available. We then introduce COEXIST, a quarantine policy which utilizes inexpensive “useless” tests to perform accurate regional case isolation. This work is all being done as part of the Microsoft Pandemic Modeling Project, where the Julia SciML tooling has accelerated the COEXIST simulations by … READ MORE
Generalized Physics-Informed Learning through Language-Wide Differentiable Programming (Video)
March 31 2020 in Differential Equations, Mathematics, Science, Scientific ML | Tags: physics-informed machine learning, pinn, scientific machine learning, scientific ml, sciml | Author: Christopher Rackauckas
Chris Rackauckas (MIT), “Generalized Physics-Informed Learning through Language-Wide Differentiable Programming”
Scientific computing is increasingly incorporating the advancements in machine learning to allow for data-driven physics-informed modeling approaches. However, re-targeting existing scientific computing workloads to machine learning frameworks is both costly and limiting, as scientific simulations tend to use the full feature set of a general purpose programming language. In this manuscript we develop an infrastructure for incorporating deep learning into existing scientific computing code through Differentiable Programming (∂P). We describe a ∂P system that is able to take gradients of full Julia programs, making Automatic Differentiation a first class language feature and compatibility with deep learning pervasive. Our system utilizes the one-language nature of Julia package development to augment the existing package ecosystem with deep learning, supporting almost all … READ MORE
Universal Differential Equations for Scientific Machine Learning (Video)
March 6 2020 in Uncategorized | Tags: neural ODEs, neural ordinary differential equations, scientific machine learning, scientific ml, sciml | Author: Christopher Rackauckas
Colloquium with Chris Rackauckas
Department of Mathematics
Massachusetts Institute of Technology
“Universal Differential Equations for Scientific Machine Learning”
Feb 19, 2020, 3:30 p.m., 499 DSL
https://arxiv.org/abs/2001.04385
Abstract:
In the context of science, the well-known adage “a picture is worth a thousand words” might well be “a model is worth a thousand datasets.” Scientific models, such as Newtonian physics or biological gene regulatory networks, are human-driven simplifications of complex phenomena that serve as surrogates for the countless experiments that validated the models. Recently, machine learning has been able to overcome the inaccuracies of approximate modeling by directly learning the entire set of nonlinear interactions from data. However, without any predetermined structure from the scientific basis behind the problem, machine learning approaches are flexible but data-expensive, requiring large databases of homogeneous labeled training data. … READ MORE
The Essential Tools of Scientific Machine Learning (Scientific ML)
August 20 2019 in Differential Equations, Julia, Mathematics, Programming, Scientific ML | Tags: ai, differential equations, natural language processing, scientific machine learning, scientific ml, sciml | Author: Christopher Rackauckas
Scientific machine learning is a burgeoning discipline which blends scientific computing and machine learning. Traditionally, scientific computing focuses on large-scale mechanistic models, usually differential equations, that are derived from scientific laws that simplified and explained phenomena. On the other hand, machine learning focuses on developing non-mechanistic data-driven models which require minimal knowledge and prior assumptions. The two sides have their pros and cons: differential equation models are great at extrapolating, the terms are explainable, and they can be fit with small data and few parameters. Machine learning models on the other hand require “big data” and lots of parameters but are not biased by the scientists ability to correctly identify valid laws and assumptions.
However, the recent trend has been to merge the two disciplines, allowing explainable models that are data-driven, require less data than traditional machine learning, and utilize the … READ MORE