Why Julia’s GPU Accelerated ODE Solvers are 20x-100x Faster than Jax and PyTorch
January 11 2026 in Differential Equations, Julia, Mathematics, Programming | Tags: gpu, julia, python | Author: Christopher Rackauckas
You may have seen the benchmark results and thought, “how the heck are the Julia ODE solvers on GPUs orders of magnitude faster than the GPU-accelerated Python libraries, that can’t be true?” In this talk I will go into detail about the architectural differences between the Julia approaches to generating GPU-accelerated solvers vs the standard ML library approach to GPU usage. By the end of the talk you’ll have a good enough understanding of models of GPU acceleration to understand why this performance difference exists, and the many applications that can take advantage of this performance improvement.
Claude Code sucks but is still useful: experiences maintaining Julia’s SciML scientific computing infrastructure
October 6 2025 in Differential Equations, Julia, Mathematics, Programming, Science, Scientific ML | Tags: agentic ai, Claude, differential equations, julia, llm, numerical analysis, physics, scientific computing, vibecoding | Author: Christopher Rackauckas
Claude Code sucks but is still useful: experiences maintaining Julia’s SciML scientific computing infrastructure
So it’s pretty public that for about a month now I’ve had 32 processes setup on one of the 64 core 128gb RAM servers to just ssh in, tmux to a window, and tell it to slam on some things non-stop. And it has been really successful!… with the right definition of success. Let me explain.
This is a repost of the long post in the Julia Discourse.
* How is Claude being used, and how useful has it been?
j-bowhay, post:1, topic:131009
I think the first will answer the others. Basically, Claude is really not smart at all. There is no extensive algorithm implementation that has come from AI. I know some GSoCers and SciML Small Grants applicants have used AI (many without disclosure) but no wholesale usage has … READ MORE
Machine learning with hard constraints: Neural Differential-Algebraic Equations (DAEs) as a general formalism
June 3 2025 in Differential Equations, Julia, Mathematics, Programming, Science, Scientific ML, Uncategorized | Tags: adjoint methods, differential-algebraic equations, julia, modelingtoolkit, neural dae, numerical solvers | Author: Christopher Rackauckas
We recently released a new manuscript Semi-Explicit Neural DAEs: Learning Long-Horizon Dynamical Systems with Algebraic Constraints where we showed a way to develop neural networks where any arbitrary constraint function can be directly imposed throughout the evolution equation to near floating point accuracy. However, in true academic form it focuses directly on getting to the point about the architecture, but here I want to elaborate about the mathematical structures that surround the object, particularly the differential-algebraic equation (DAE), how its various formulations lead to the various architectures (such as stabilized neural ODEs), and elaborate on the other related architectures which haven’t had a paper yet but how you’d do it (and in what circumstances they would make sense).
READ MORESymbolic-Numerics: how compiler smarts can help improve the performance of numerical methods (nonlinear solvers in Julia)
February 13 2024 in Julia, Mathematics, Programming | Tags: julia, nonlinear solve, pydata, symbolic-numerics | Author: Christopher Rackauckas
Many problems can be reduced down to solving f(x)=0, maybe even more than you think! Solving a stiff differential equation? Finding out where the ball hits the ground? Solving an inverse problem to find the parameters to fit a model? In this talk we’ll showcase how SciML’s NonlinearSolve.jl is a general system for solving nonlinear equations and demonstrate its ability to efficiently handle these kinds of problems with high stability and performance. We will focus on how compilers are being integrated into the numerical stack so that many of the things that were manual before, such as defining sparsity patterns, Jacobians, and adjoints, are all automated out-of-the-box making it greatly outperform purely numerical codes like SciPy or NLsolve.jl.
PyData Global 2023
ChatGPT performs better on Julia than Python (and R) for Large Language Model (LLM) Code Generation. Why?
November 19 2023 in Julia, Programming | Tags: ai, artificial intelligence, chatgpt, julia, large language models, llm, machine learning, MATLAB, python, r | Author: Christopher Rackauckas
Machine learning is all about examples. The more data you have, the better it should perform, right? With the rise of ChatGPT and Large Language Models (LLMs) as a code helping tool, it was thus just an assumption that the most popular languages like Python would likely be the best for LLMs. But because of the increased productivity, I tend to use a lot of Julia, a language with an estimated user-base of around a million programmers. For this reason, people have often asked me how it fairs with ChatGPT, Github Copilot, etc., and so I checked out those pieces and… was stunned. It’s really good. It seemed better than Python actually?
The data is in: Julia does well with ChatGPT
This question was recently put to the test by a researcher named Alessio Buscemi in A Comparative Study … READ MORE
Summary of Julia Plotting Packages
June 17 2023 in Julia | Tags: data science, ggplot2, julia, plots, programming language, startup time, ttfx, visualization | Author: Christopher Rackauckas
This is a repost of my response on the Julia Discourse on this topic. I was asked to make a blog post so here you go!
The “Main” Plotting Packages
Here’s a quick summary of the most widely used plotting packages. I may have missed one, but I haven’t missed one that is very widely used.
- Plots.jl is the most used. It’s probably the most documented, used in the most tutorials, and is used in many videos.
- Pros: Its main draw is that it has a lot of plugins to other packages through its recipes system, which means that a lot of odd things like `plot(sol::ODESolution)` or showing the sparsity of a `BandedMatrix` just works. With all of these integrations, it’s normally what I would recommend first to newcomers since they will generally get the most done with the least work. It … READ MORE
- Pros: Its main draw is that it has a lot of plugins to other packages through its recipes system, which means that a lot of odd things like `plot(sol::ODESolution)` or showing the sparsity of a `BandedMatrix` just works. With all of these integrations, it’s normally what I would recommend first to newcomers since they will generally get the most done with the least work. It … READ MORE
Integrating equation solvers with probabilistic programming through differentiable programming
November 24 2022 in Julia, Programming, Science, Scientific ML | Tags: differential equations, julia, probabilistic programming, scientific machine learning, sciml | Author: Christopher Rackauckas
Part of the COMPUTATIONAL ABSTRACTIONS FOR PROBABILISTIC AND DIFFERENTIABLE PROGRAMMING WORKSHOP
Abstract: Many probabilistic programming languages (PPLs) attempt to integrate with equation solvers (differential equations, nonlinear equations, partial differential equations, etc.) from the inside, i.e. the developers of the PPLs like Stan provide differential equation solver choices as part of the suite. However, as equation solvers are an entire discipline to themselves with many active development communities and subfields, this places an immense burden on PPL developers to keep up with the changing landscape of tens of thousands of independent researchers. In this talk we will explore how Julia PPLs such as Turing.jl support of equation solvers from the outside, i.e. how the tools of differentiable programming allows equation solver libraries to be compatible with … READ MORE
Engineering Trade-Offs in Automatic Differentiation: from TensorFlow and PyTorch to Jax and Julia
December 25 2021 in Julia, Programming, Science, Scientific ML | Tags: automatic differentiation, compilers, differentiable programming, jax, julia, machine learning, pytorch, tensorflow, XLA | Author: Christopher Rackauckas
To understand the differences between automatic differentiation libraries, let’s talk about the engineering trade-offs that were made. I would personally say that none of these libraries are “better” than another, they simply all make engineering trade-offs based on the domains and use cases they were aiming to satisfy. The easiest way to describe these trade-offs is to follow the evolution and see how each new library tweaked the trade-offs made of the previous.
Early TensorFlow used a graph building system, i.e. it required users to essentially define variables in a specific graph language separate from the host language. You had to define “TensorFlow variables” and “TensorFlow ops”, and the AD would then be performed on this static graph. Control flow constructs were limited to the constructs that could be represented statically. For example, an `ifelse` function statement is very different from … READ MORE
Useful Algorithms That Are Not Optimized By Jax, PyTorch, or Tensorflow
July 21 2021 in Julia, Programming, Science, Scientific ML | Tags: automatic differentiation, differentiable programming, julia, machine learning | Author: Christopher Rackauckas
In some previous blog posts we described in details how one can generalize automatic differentiation to give automatically stability enhancements and all sorts of other niceties by incorporating graph transformations into code generation. However, one of the things which we didn’t go into too much is the limitation of these types of algorithms. This limitation is what we have termed “quasi-static” which is the property that an algorithm can be reinterpreted as some static algorithm. It turns out that for very fundamental reasons, this is the same limitation that some major machine learning frameworks impose on the code that they can fully optimize, such as Jax or Tensorflow. This led us to the question: are there algorithms which are not optimizable within this mindset, and why? The answer is now published at ICML 2021, so lets dig into … READ MORE
JuliaCall Update: Automated Julia Installation for R Packages
January 18 2021 in Differential Equations, Julia, Mathematics, Programming, R | Tags: devops, differentialequations, installation, julia, juliacall, modelingtoolkit, r | Author: Christopher Rackauckas
Some sneakily cool features made it into the JuliaCall v0.17.2 CRAN release. With the latest version there is now an install_julia function for automatically installing Julia. This makes Julia a great high performance back end for R packages. For example, the following is an example from the diffeqr package that will work, even without Julia installed:
install.packages("diffeqr")
library(diffeqr)
de <- diffeqr::diffeq_setup()
lorenz <- function (u,p,t){
du1 = p[1]*(u[2]-u[1])
du2 = u[1]*(p[2]-u[3]) - u[2]
du3 = u[1]*u[2] - p[3]*u[3]
c(du1,du2,du3)
}
u0 <- c(1.0,1.0,1.0)
tspan <- c(0.0,100.0)
p <- c(10.0,28.0,8/3)
prob <- de$ODEProblem(lorenz,u0,tspan,p)
fastprob <- diffeqr::jitoptimize_ode(de,prob)
sol <- de$solve(fastprob,de$Tsit5(),saveat=0.01)
Under the hood it’s using the DifferentialEquations.jl package and the SciML stack, but it’s abstracted from users so much that Julia is essentially an alternative to Rcpp with easier interactive development. The following example really brings the seamless … READ MORE