Result and Learnings
The completed Rust port is now available on Github or available for installation in Rust programs from crates.io.
In short, AI tools were very useful and saved some time and most of all attention, but it was far from automatic, and confidence in the result depends a lot on the complexity of the logic and how exhaustively all its states can be tested.
- Porting is unlikely to be a one-shot task even with lots of preparation.
- An analysis-based approach to the port, or even to evaluate the result, is unlikely to be reliable.
- How well one can test functionality is a better indicator for confidence, and that depends a lot on the complexity of the system.
- It is harder to estimate the time needed for a test-based port, as you do not get the feel for the quality of different parts of the codebase and any test can reveal gaping omissions.
- Even with good tests, the AI can often need additional prompting to implement solutions to root causes instead of superficial fixes to make particular tests pass.
Motivation
In 2017 when we were building application security startup tCell.io (later acquired by Rapid7) I wrote about our use of Rust and how easy it made it to integrate C libraries. However, something always felt wrong about integrating memory-unsafe code into security-sensitive applications (for reasons Apple recently experienced when they had to tell everyone to urgently upgrade MacOS and iOS because attackers could send users malformed images that would cause memory corruption exploits in the operating system image processing code).
Both Microsoft and Google's Chrome Team have described ([1][2]) memory safety as the top source of security vulnerabilities. The demand for improving security is also getting more urgent as hackers can now use AI to find exploits in code.
I figured I would see what it would be like to use AI tools to port a real-world complicated C library to memory-safe Rust. Even an imperfect port to a memory-safe language can improve security as the effects of memory corruption bugs are so severe.
I also wanted to explore the capabilities and automation possibilities of the latest AI coding tools for a real-world software engineering task where correctness and details matter more than user interface looks.
The Task
The library I chose to port was libinjection. Libinjection is a library that is used to detect SQL injection and cross-site scripting (XSS) attacks in string inputs.
In order to explore the automation possibilities of the task, I decided to rely on AI as much as possible without trying to look into the code itself more than necessary, hence "vibe porting" from the term "vibe coding" coined by Andrej Karpathy:
I used GPT-5 through the free ChatGPT app to create the initial porting plan and then completed the rest of the port using Claude Code on the $100/month Max plan running in VS Code using its IDE extension.
The Process
Plan-based approach
I opened ChatGPT and prompted it to create the porting plan as follows: "Create a detailed plan for an LLM agent to port the libinjection library from C to Rust, including how to verify the functionality with its tests and also compare performance."
In different steps, I then pasted in the first three phases of the plan into Claude Code to have it create the structure of the libinjectionrs repository, the documents with the porting strategy, and finally implement the port according to the strategy.
Claude Code ran into context length issues with some of the longer auto-generated source files and its port was very far from the "Port line-by-line with minimal changes" instruction described in the porting document with missing functionality or very simplified functionality (sometimes leaving only a TODO). It was clear Claude Code tries to avoid reading more context than necessary so a different strategy was needed rather than simply increasing its context size.
I re-did the port a few times after different preparation steps to see if Claude could generate a working port of the core logic in one shot with the right circumstances. This included instructing Claude to translate the scripts that auto-generated the longer code files into Rust build scripts and also having Claude create a detailed analysis plan for the different functions in the C library. Even after assuring me that the port followed the analysis documents of the C library, it was clear that there were lots of omissions and deviations in anything it produced.
Test-based approach
I eventually stopped trying to get the port correct in one shot and instead focused on a test-based approach to complete the port. This started with having Claude write a runner for the test files from the original C repository followed by differential fuzz testing that randomly mutates input for the library and checks whether the Rust port produces the same output as the C library.
Targeting root causes instead of specific tests
Claude produced test and debug programs to go through each failing test, but would rarely follow the instruction to research and implement a fix in a way that the Rust code followed the C library's logic exactly.
Instead, Claude would usually deprioritize, simplify, make "practically the same", or come up with some other creative excuse to just fix the test without understanding the root cause.
One follow-up prompt that I found useful to make Claude identify the root cause was to ask it to provide a comment referencing the lines in the C code that were responsible for the logic it implemented. This would often result in a correction:
It was also helpful to ask Claude to perform a comprehensive search for the same problem when it identified a new class of problems.
Time and uncertainty
Even though I eventually allowed Claude Code to bypass all permission checks when writing and running its test-replication programs, going through each test case took considerable time for the agent. It is possible this could be sped up with a higher-cost plan exploring more test cases in parallel, but that would also require more supervision as one needed to stay on top of the AI's shortcuts and prompt it to explore the root causes of problems.
Another problem was the uncertainty. One had a sense of progress while working with the existing tests from the C library, but as fuzz testing started and each new difference revealed missing code pieces or other large omissions or simplifications, it was hard to get a sense of how long the port would take. Eventually the differences that the fuzzer detected became fewer and fewer and I decided to release the port after running the fuzzer for hours without a difference being detected.
What AI is good at and attention savings
AI models like Claude embed a lot of knowledge and even the simpler Sonnet model performed great when being directed to sort out details, like handling signed/unsigned character conversions between Rust and C, while the advanced Opus model had a slight edge during root cause analysis.
This meant that the role of the developer became more about supervising the work than keeping focused attention on state machines and inspecting details. Most of the time the model was asked to show code, it was to populate the model's context with details so it could better solve the problems.
AI coding works best when the problem can be isolated to the environment that the model operates in and can be tested comprehensively, such as this library that consumes strings and outputs analytics. It becomes more challenging when there are external factors, such as environment differences or more complex data, and when testing is harder because of scale or external system interactions.
Future Outlook
I expect agentic coding tools will soon attempt to perform this process fully autonomously, but we may need to go through hype corrections and develop better methods for evaluating AI-written code before enterprises can become confident in the results.