A few weeks ago, I wrote about a project our team has been working on called Distill. A simple application that summarizes and extracts important details from our daily meetings. At the end of that post, I promised you a CLI version written in Rust. After some code reviews from Rustaceans on Amazon and a bit of polishing, today I’m ready to share the Distill CLI.
After building from source, simply pass the Distill CLI a media file and select the S3 bucket where you want to store the file. Today, Distill supports generating summaries as Word documents, text files, and printing directly to the terminal (the default). You’ll find it’s easily extensible: my team (OCTO) is already using it to export summaries of our team meetings directly to Slack (and working on support for Markdown).
Playing is a good way to learn and be curious.
The way we build has changed quite a bit since I started working with distributed systems. Today, if you want it, computing, storage, databases and networking are available on demand. As builders, our focus has shifted toward increasingly rapid innovation, and along the way, modifying the system level has become a lost art. But touch-ups are as important now as ever. I vividly remember the hours I spent tinkering with BSD 2.8 to get it to work on the PDP-11s, and it cemented my never-ending love for operating system software. Tinkering gives us the opportunity to really get to know our systems. Experiment with new languages, frameworks and tools. Look for efficiencies big and small. To find inspiration. And this is exactly what happened with Distill.
We rewrote one of our Lambda functions in Rust and saw that cold starts were 12x faster and memory usage decreased by 73%. Before I knew it, I started thinking about other ways to make the whole process more efficient for my use case.
The original proof of concept stored media files, transcripts, and summaries in S3, but since I’m running the CLI locally, I realized I could store the transcripts and summaries in memory and save myself some writes to S3. I also wanted an easy way to load media and monitor the rollup process without leaving the command line, so I put together a simple UI that provides status updates and alerts me when something fails. The original showed what was possible, left room for tweaks, and was the model I used to write the Distill CLI in Rust.
I encourage you to proveand let me know when you find any bugs, edge cases or have ideas to improve it.
Builders choose Rust
As technologists, we have a responsibility to build sustainably. And this is where I really see the potential of Rust. With its emphasis on performance, memory safety, and concurrency, there is a real opportunity to lower computational and maintenance costs. Its memory safety guarantees eliminate obscure bugs that plague C and C++ projects, reducing crashes without compromising performance. Its concurrency model imposes strict controls at compile time, avoiding data races and maximizing multicore processors. And while build errors can be very irritating right now, fewer developers chasing bugs and more time focused on innovation are always good things. That’s why it has become a go-to for builders who thrive by solving problems on an unprecedented scale.
Since 2018, we have increasingly leveraged Rust for critical workloads across various services such as S3, EC2, DynamoDB, Lambda, Fargate, and Nitro, especially in scenarios where hardware costs are expected to dominate over time. In his guest post last year, Andy Warfield wrote a bit about ShardStore, the lowest layer of the S3 storage stack that manages the data on each individual disk. Rust was chosen to get type safety and structured language support to help identify bugs earlier, and how they wrote libraries to extend that type safety to applications on disk structures. If you haven’t done so yet, I recommend that you read the postand the SOSP paper.
This trend is reflected throughout the industry. Discord moved its Read States service from Go to Rust to address large latency spikes caused by garbage collection. It is 10 times faster and its worst tail latencies have been reduced almost 100x. Similarly, figure They rewrote performance-sensitive parts of their multiplayer service in Rust, and have seen significant improvements in server-side performance, such as reducing average maximum CPU usage per machine by 6x.
The thing is, if you’re serious about cost and sustainability, there’s no reason not to consider Rust.
Rust is hard…
Rust has a reputation for being a difficult language to learn and I don’t dispute that there is a learning curve. It will take you time to get familiar with the loan checker and you will have to fight with the compiler. It’s a lot like writing a FAQ for a new idea on Amazon. There’s a lot of friction from the start, which is sometimes difficult when all you really want to do is jump into the IDE and start building. But once you’re on the other side, there’s tremendous potential to gain speed. Remember, the cost of building a system, service or application is nothing compared to the cost of operating it, so how you build it should be under continuous scrutiny.
But you don’t have to take my word for it. At the beginning of this year, The Registry published findings from Google showing that their Rust teams were twice as productive as teams using C++, and that the same-sized team using Rust instead of Go was just as productive with more fixes in their code. There are no bonus points for increasing headcount to address avoidable problems.
Final thoughts
I want to be very clear: this is not a call to rewrite everything in Rust. Just as Monoliths are not dinosaurs.There is no one-size-fits-all programming language and not all applications will have the same business or technical requirements. It’s all about using the right tool for the right job. This means questioning the status quo and continually looking for ways to incrementally optimize your systems, to change things and measure what happens. Something as simple as changing the library you use to serialize and deserialize json from the Python standard library to orjson It could be all you need to speed up your application, reduce memory usage, and reduce costs in the process.
If you take nothing else away from this post, I encourage you to actively seek efficiencies in all aspects of your work. Gypsy. Extent. Because everything has a cost, and cost is a good indicator of a sustainable system.
Now go build!
Special thanks to AWS Rustaceans Niko Matsakis and Grant Gurvis for their code reviews and feedback while developing the Distill CLI.
