Neural Jump SDEs (Jump Diffusions) and Neural PDEs
June 5 2019 in Differential Equations, Julia, Mathematics, Stochastics | Tags: CUDA, differentiable programming, DifferentialEquations.jl, gpu, julia, stochastic differential equations | Author: Christopher Rackauckas
This is just an exploration of some new neural models I decided to jot down for safe keeping. DiffEqFlux.jl gives you the differentiable programming tools to allow you to use any DifferentialEquations.jl problem type (DEProblem) mixed with neural networks. We demonstrated this before, not just with neural ordinary differential equations, but also with things like neural stochastic differential equations and neural delay differential equations.
At the time we made DiffEqFlux, we were the “first to the gate” for many of these differential equations types and left it as an open question for people to find a use for these tools. And judging by the Arxiv papers that went out days after NeurIPS submissions were due, it looks like people now have justified some machine learning use cases for them. There were two separate papers on neural … READ MORE
Some State of the Art Packages in Julia v1.0
August 27 2018 in Julia, Programming | Tags: | Author: Christopher Rackauckas
In this post I would like to explain some of the state of the art packages in Julia and how they are pushing forward their disciplines. I think the part I would really like to make clear is the ways which Julia gives these packages a competitive advantage and how these features are being utilized to reach their end goals. Maybe this will generate some more ideas!
What are the advantages for Julia in terms of package development?
Using Python/R to use other people’s packages is perfectly fine since then your code will just be as fast as the package code if all your time is in package code calls (I purposely leave MATLAB off this list because its interoperability is much more difficult to use). There is one big exception where using an efficient package can still be slow even when … READ MORE
PuMaS.jl: Pharmaceutical Modeling and Simulation Engine
August 13 2018 in Biology, Julia, Programming | Tags: | Author: Christopher Rackauckas
Here is an introduction to a pharmaceutical modeling project which I will be releasing in the near future. More details to come.
Solving Partial Differential Equations with Julia
August 8 2018 in Differential Equations, HPC, Julia, Mathematics, Programming, Uncategorized | Tags: | Author: Christopher Rackauckas
Here is a talk from JuliaCon 2018 where I describe how to use the tooling across the Julia ecosystem to solve partial differential equations (PDEs), and how the different areas of the ecosystem are evolving to give top-notch PDE solver support.
Why Numba and Cython are not substitutes for Julia
August 6 2018 in Julia, Programming | Tags: | Author: Christopher Rackauckas
Sometimes people ask: why does Julia need to be a new language? What about Julia is truly different from tools like Cython and Numba? The purpose of this blog post is to describe how Julia’s design gives a very different package development experience than something like Cython, and how that can lead to many more optimizations. What I really want to show is:
- Julia’s compilation setup is built for specialization of labor which is required for scientific progress
- Composition of Julia codes can utilize the compilation process to build new programs which are greater than the sum of the parts
I will also explain some of the engineering tradeoffs which were made to make this happen. There are many state-of-the-art scientific computing and data science packages in Julia and what I want to describe is how these are using the more “hardcore … READ MORE
DifferentialEquations.jl’s Confederated Modular API
July 18 2018 in Differential Equations, Julia, Mathematics, Programming | Tags: | Author: Christopher Rackauckas
I wrote a manuscript describing DifferentialEquations.jl’s confederated modular API and its effect on the local scientific computing ecosystem. It’s now on Arxiv until we can find the right venue for it.
Using Juno for Interactive Test-Driven Julia Package Development
February 19 2018 in Julia, Programming | Tags: | Author: Christopher Rackauckas
I found myself explaining my development workflow all of the time, so I thought I’d show it in a video. This is some of the stuff I do. You can do it other ways. I like this way.
Algorithm efficiency comes from problem information
January 4 2018 in Differential Equations, Julia, Mathematics, Programming | Tags: | Author: Christopher Rackauckas
This is a high level post about algorithms (especially mathematical, scientific, and data analysis algorithms) which I hope can help people who are not researchers or numerical software developers better understand how to choose and evaluate algorithms. When scientists and programmers think about efficiency of algorithms, they tend to think about high level ideas like temporary arrays, choice of language, and parallelism. Or they tend to think about low level ideas like pointer indirection, cache efficiency, and SIMD vectorization. However, while these features matter, most decent implementations of the same algorithm tend to get quite close in efficiency to each other (probably <3x if everyone is using a performant enough programming language or constructs). If you're sitting down thinking "what is the next algorithm that I am going to invent that will be truly a step beyond what existed before?", ... READ MORE
Solving Systems of Stochastic PDEs and using GPUs in Julia
December 11 2017 in CUDA, Differential Equations, Julia, Mathematics, Stochastics | Tags: | Author: Christopher Rackauckas
What I want to describe in this post is how to solve stochastic PDEs in Julia using GPU parallelism. I will go from start to finish, describing how to use the type-genericness of the DifferentialEquations.jl library in order to write a code that uses within-method GPU-parallelism on the system of PDEs. This is mostly a proof of concept: the most efficient integrators for this problem are not compatible with GPU parallelism yet, and the GPU parallelism isn’t fully efficient yet. However, I thought it would be nice to show an early progress report showing that it works and what needs to be fixed in Base Julia and various libraries for us to get the full efficiency.
Edit May 2019
As of DifferentialEquations.jl v6.4.0, this is no longer a proof of concept. The whole library, including implicit solvers with GMRES, etc., and … READ MORE
DifferentialEquations.jl 3.0 and a Roadmap for 4.0
October 30 2017 in Differential Equations, Julia, Mathematics, Programming | Tags: | Author: Christopher Rackauckas
I am pleased to announce the release of DifferentialEquations.jl 3.0. In the last DiffEq blog post I described the current state of JuliaDiffEq and DifferentialEquations.jl along with the trajectory that we hoped to take. We identified (at that time) current shortcomings of the software and our plans to remedy them. I also recently did a survey of differential equation suites in order to understand where we stand and see where we need to improve. These research efforts were used to put together a list of goals that were systematically achieved during 3.0. What I would like to do this time around is give a broad overview of what we have released in the 3.0 timeframe, the goals that we have achieved, and the goals that we are putting off (for next Google Summer of Code?). And then, … READ MORE