Get notified of new posts
PuLP: rust core¶
I recently got a paid Cursor license and decided to make good use of it. So I decided to vibe code my way into a new Rust core for the PuLP library.
It was an interesting way of being introduced to vibe coding, and to a lesser extent to Rust, which I knew only the basics of.
I do not want to focus on the vibe coding process because I think there is just too much written about it already. I'll add my thoughts on it at the end so it is easy to ignore. I wish I could ignore half the content I get on it, to be honest.
I will write mostly about the design decisions I had to do while making the changes, which took over a week. This experience forced me to find out how other libraries (e.g., gurobi, ortools) handle things: gen AI also helped me to query design decisions of those libraries.
Why risk migrating something that works¶
Warning
I'm not sure if this task was a good idea. I don't even know if the path I took was the right one. But what's done is done.
Until now (and it's still the case in pypi and the master branch), PuLP is a pure-python package without any mandatory dependency. This made it very easy for people to read, install, contribute and understand the insides. And this is very valuable.
A few months ago, when I wrote a post on the current state of PuLP, a few people came up and showed interest in improving the performance of the library. I've always argued that the performance of the modelling library is usually irrelevant when considering the time it takes to solve an LP/MIP problem by any solver (see benchmarks).
Many python libraries and tools are now being migrated/ powered by Rust via PyO3. Some I use: uv, ruff, polars, pydantic.
Regardless, it was a learning experience and it probably brings some benefit to a few projects. So why not. On top of that, I was actually looking for an excuse to learn a bit more of Rust.
What was migrated¶
I swapped to Rust as much as I could get away with without changing the API too much: the memory storage of the model, the building of constraints, linear expressions, the reading and writing of LP and MPS files. The idea was to take as much advantage of Rust's speed and safety as possible, while keeping the Python syntax flexibility for the modelling.
Below I go over some changes.
Variables now belong inside the model¶
This is actually the largest (the only?) API breaking change on the python modelling syntax.
In the past, PuLP variables never belonged anywhere: they were just objects floating inside the functions that created them. Their references lived inside constraints (and, more broadly, inside expressions). A model's variables where references to those that belong to at least one constraint. So if you created a variable and never used it, it was never included in the model. Which makes sense actually.
Anyways, Rust doesn't like orphan objects running around. In fact, everything needs to belong in one place only (or mainly). So, as other python modelling libraries do, PuLP variables are now created and stored (deep) inside the LpProblem object when created.
The model stores all info¶
In the Rust core, the model is the only source of truth: all variable and constraint information is stored inside the model (named ModelCore in the Rust side). Rust Variable and Constraint objects (and their python counterparts LpVariable and LpConstraint) become just "views" of this information -> pretty much an id that is used to query a list inside that model.
On top of this "id", Variables and Constraints need to keep a "weak reference" to the model, so they can get their own information (name, bounds, category, elements, etc.). It's "weak" because Variables and Constraints do not "own" the model. This is a consequence of Rust's strict ownership rules. I do not understand this 100% but I think it means that they can access the model's information as long as the model exists but that they risk raising an runtime error when the model disappears without notice (fair as they do not make sense without it).
Implications¶
There are several implications that I'm aware of. And I'm sure there are a ton that I'm not yet. The ones I know:
- API changes: variable-creation is different now. Old code will break as I decided not to have backwards compatible code around.
- building from source: building PuLP requires having a rust environment now and a C++ compiler, more on the readme.
Benchmarks¶
I do not have an easy-to-test large model at hand. I tested a small problem and obviously I saw no difference. Actually, I believe that it is hard to find models that are big enough to take long modelling times while being small enough to be solved efficiently by a MIP solver. Especially well-crafted models in well-written python code.
I hope someone will proof me wrong by testing this library it in a large optimization model. I hope this person will write back to me, justifying all of this effort in a single email.
Further work, questions, feedback¶
Beyond the "why did you even do it?", which I answered above, there are many remaining questions.
Was there any other way I could have done the changes? Maybe maintaining the current python API? Does it make sense to deploy this version as PuLP 4.0.0? Or should I create a new pypi package?
What other parts of the codebase should be migrated? I haven't touched the solver API part, for example. Some of that is linked to other python libraries and that would be troublesome to migrate. But part of the processing that takes place inside these classes could potentially be done faster in Rust. Also, now that Rust is available inside the project, connecting to C or C++ APIs becomes feasible, a few days of vibe-coding away. Who know what libraries could be added.
Finally, one reason other libraries migrate to Rust is because it is very easy to parallelize. I haven't taken that path yet as I'm not 100% sure which part deserves running in parallel, but maybe someone has ideas on that.
Which leads me to...
Beta testers¶
Are you bored and want to give this new library a try on your huge model to see how much faster it goes? I haven't published any version to pypi, but you can download the branch and follow the steps to build it from source.
I put the link of the rust-core branch here: https://github.com/coin-or/pulp/tree/rust_core The instructions on installing it are here.
My experience using Cursor¶
Since I'm not a Rust expert, I had extensive preliminary dialogs with mostly chatgpt to understand the implications, the coding conventions and what to expect / ask in a migration. With this, a first draft was produced (a markdown file written by chatgpt), which then got even more modifications through Cursor "Planning" option.
The first draft migration was not at all what I wanted: it consisted of a duplication of logic in python and Rust that made no sense. Also, it was full of backwards compatible code that was a mess to read.
It took several iterations, all using the "Planning" option to refine very well each step. At some point, the design choices started to make sense (and to work), even if the migration was very limited: only the storage and querying of the Rust data model.
I experimented (a bit) with alternative LLMs, always planning. I tried to be as ambitious as I could with the changes, grouping them into large tasks. Still, it took many days and many many changes, reviews and tests.
It was tedious: but a (very small) fraction of what it would have taken by myself.