Rust's 2017 roadmap, six months in

July 5, 2017 · Nicholas Matsakis

In January of this year, we adopted the 2017 Rust Roadmap, which laid out our plans for 2017. As part of the roadmap process, we plan to regularly release updates on the progress of each roadmap item. This post marks the halfway point through the year.

Tracking progress via roadmap issues

First, a meta note. If you'd like to follow along with the progress on a particular roadmap initiative, or to find out more about how you can get involved, the best place to go is the list of issues on the rust-roadmap repo. There you will find an issue dedicated to each of the major initiatives that we are pushing on. These issues contain links to ongoing work. You'll find a number of links to issues like this in the descriptions that follow.

Rust should have a lower learning curve

The most direct way to make Rust easier to learn is to improve the way that we teach it. To that end, we've been hard at work on a brand new edition of the official "Rust" book (roadmap issue), and we now have a complete draft available online. This new edition puts ownership front and center and it also has expanded coverage of a number of other areas in Rust, such as error handling, testing, matching, modules, and more. Even better, you can pre-order a printed version through No Starch Press. If you'd like to help out, there is still lots of editing work to be done!

We've also been working on a number of language changes aimed at improving language ergonomics. These range from long-standing proposals, like non-lexical lifetimes or impl Trait, to newer ideas, like the recently approved RFCs on trait aliases and match ergonomics. On the roadmap issue, you will find a large list of initiatives, organized by the part of the language that they target (e.g., ownership/borrowing, the trait system, etc). We are actively looking for people to help with writing RFCs but also implementing the accepted RFCs -- if you're interested in helping out with something, look for the "mentoring" contacts listed in the roadmap issue or on the tracking issue for a specific RFC. And, of course, if you think you've got a good idea that's not listed, open up a thread on internals and let's talk about it!

Rust should have a pleasant edit-compile-debug cycle

We've been targeting the problem of improving compiler performance in a number of different ways. One of the simplest is the cargo check command that we released in Rust 1.16 -- this command does a limited form of compilation which skips code-generation and simply looks for errors. Since code generation typically takes 50% or more of compilation time, this can be really useful in the early stages when you are writing new code and trying to get it to compile, particularly if you have a multi-crate project. This command is also used by the RLS.

Of course, eventually you want to run your code, and for that you need a full compilation. In order to make that faster, we've been hard at work retooling the compiler to work in an incremental fashion. Earlier this year, we advertised the "beta" release of incremental on nightly builds. While the beta version sometimes achieved quite large speedups, we also found that the dependency tracking was not as robust or effective as we would like. Therefore, we are now adopting a new and improved approach to incremental compilation that we expect be ready in the next month or so. If you're interested in helping with the transition to the new system, check out the incremental compilation roadmap issue or the tracking issue for the new algorithm itself; you can also follow this internals thread, where we regularly post links to bugs that include mentoring instructions.

Looking beyond incremental compilation, we've also been taking steps to optimize compilation time in other ways. Probably the most important step in that respect has been getting a new version of the perf.rust-lang.org website up and running. The "perf" website tracks the effect of each and every PR on compilation performance, so that we can detect and correct regressions. In fact, the new website immediately led to some major performance improvements. There is still lots of work that could be done to improve it, however, ranging from evaluating and improving our benchmark suite to improving the web interface; see the tracking issue on this topic for more.

Rust should provide a solid, but basic IDE experience

Since it first debuted at RustConf last year, the Rust Language Service (RLS) has been growing rapidly (roadmap issue). It now offers support for most basic IDE operations, such as "jump to definition" or "find all uses", as well as offering code completion (via the racer project). At this point, the focus is primarily on polish: making the RLS easier to install and fixing bugs. For example, we recently made it possible to install the RLS directly through rustup.

If you'd like to give the RLS a spin, the easiest way is to use the VSCode plugin; however, the RLS is a generic server (it speaks Microsoft's Language Server Protocol), and there are also clients available for a number of other editors. A word of warning: at this point, the RLS is still in an "alpha" period. While it is eminently usable, you may encounter bugs or other limitations.

If you'd like to get involved with the RLS, check out the roadmap issue or the RLS issue listing; in particular, those things tagged with "good first issue".

Rust should provide easy access to high quality crates

As the size of the crates.io ecosystem has grown, the simple search and sorting criteria used by the crates.io website are no longer that helpful for figuring out which crates you should use. To address this, we've added categories and a number of badges that crate authors can add to their crates. These help people find crates for a particular purpose and judge a crate’s quality at a glance. In addition, RFC 1824 laid out a plan for improving the default sort in crates.io and exposing additional information to help in selecting a crate. There is a tracking issue that lists the pieces of this RFC, and we’d love contributions towards those pieces! Once the RFC is completely implemented and people have had a chance to use the features for a while, we plan to ask the community for feedback and make adjustments.

In addition, the effort on the "cookbook" described below will also be a boon for discovering crates in a task-centric way.

Rust should be well-equipped for writing robust servers

We've made some excellent strides the first half of this year towards making Rust well equipped for writing robust servers. The futures crate and Tokio project continue to explore the asynchronous I/O ecosystem and have seen some heavy usage through crates like Hyper and production users like linkerd-tcp. Additionally we've seen projects like Rocket continue to tirelessly improve the ergonomics of Rust-on-the-server. A recent discussion of what the biggest blockers are today has highlighted that async/await notation, better Tokio/futures documentation, and a solid HTTP foundation for the ecosystem are some of the next highest priorities. We plan to enable async/await notation on the nightly Rust channel by the end of the year and position it for stabilization in early 2018. This in turn should help fuel a rewrite of Tokio's/future's documentation and continue to grow community support for crates such as HTTP.

Rust should have 1.0-level crates for essential tasks

The Libz Blitz proceeds apace! The Libz Blitz is a systematic effort to identify the most broadly used crates in the Rust ecosystem and to ensure that they all meet a consistent level of completeness and quality. This effort entails collaborating on the internals forum to review crates according to the API guidelines, filing and fixing the issues that arise, and writing examples for a new "cookbook" of Rust examples.

The effort is structured to be highly amenable to contribution, particularly from new Rust developers, and so far has resolved 99 crate issues across 10 crates, and created more than 30 examples for the cookbook, thanks to the efforts of 53 contributors.

If you're interested in participating, take a look at the introductory post on the internals thread.

Rust should integrate easily into large build systems

Most work on build system integration has focused on more clearly identifying what the challenges are and developing concrete proposals that target them. Some of the current avenues for exploration are:

We are hoping to pick up the pace on this work in the second half of the year, but could use help doing so. If either of the above Cargo improvements is of interest to you, or if you have insights on build system integration in general, please reach out on the tracking issue!

Rust's community should provide mentoring at all levels

When it comes to mentoring, we've been pursuing a few different efforts. The first, RustBridge, is specifically aimed at bringing underrepresented folks into tech; it also serves as a great curriculum for people completely new to Rust. The materials have already been through several iterations and continue to evolve and improve, and we are going to be running a RustBridge workshop the day before RustConf. We would like to see more RustBridge events.

We also just launched Increasing Rust's Reach, an initiative for hearing more from groups currently underrepresented in Rust (or technology more generally), and working together to make Rust accessible to a wider audience. While this isn't a mentorship program per se (in many cases, it's the participants who are doing the teaching!), it is still geared toward lowering the on-ramp to Rust's community.

In addition, the various Rust teams have been pursuing a number of different initiatives trying to encourage people to get involved in the Rust project itself:

  • We've added three new teams -- infrastructure, cargo, and dev-tools -- and hence created new areas where people can get involved.
  • The lang team has adopted the new shepherd role. Shepherds are experienced members of the community who have demonstrated both excellent technical acumen and the ability to build consensus and understand alternative perspectives. Shepherds attend language meetings and help to steer discussion on RFCs and guide them towards a useful conclusion.
  • The lang team has also been working on "mentoring" RFCs. The roadmap issue for the ergonomics initiative, for example, lists a number of RFCs where we are actively recruiting.
  • A big part of the "Libz Blitz" is helping to direct community effort to pushing libraries over the finish line.
  • The compiler team has been actively pursuing "mentoring bugs" (bugs tagged as E-mentor), which include written instructions, as well as drawing up plans to improve the documentation of the code and workflows.

Areas of Exploration

In addition to the main roadmap items, the roadmap RFC called out two "areas of exploration". These are areas with strong potential for Rust that are still in a more exploratory phase.

Embedded Rust initiative

The embedded Rust ecosystem continues to grow. A bare metal concurrency framework for Cortex-M microcontrollers geared towards robotics and control systems has been recently developed. And Tock, an embedded OS that also targets Cortex-M microcontrollers, has been making progress towards pure Rust userland applications and continues growing its driver support.

On the compiler side support for the MSP430 architecture has been improving thanks to the community's effort, and support for the AVR architecture is also being worked on, out of tree, by the community.

On the community side efforts in standardizing the development workflow are on going with the creation and development of guides, project templates, core crates and tooling. Recently a survey to identify the current needs of embedded developers was performed and the corresponding roadmap issue was updated to reflect the results. In response to the survey results a community effort to create a Hardware Abstraction Layer, that will serve as a base for building the embedded crate ecosystem, has been started and the design discussion is currently on going. An "Are we embedded yet?" web site that will reflect the up to date state of the embedded ecosystem and track its progress is also being worked on.

(Thanks to Jorge Aparicio for this writeup.)

Integrating with Other Languages: bindgen Update

C and C++

bindgen is the frontier for automating integration of C and C++ libraries into Rust code bases. bindgen takes header files as input, and outputs the appropriate foreign function and type declarations so that the C/C++ library can be used with minimal effort from Rust.

bindgen has become a crucial infrastructure for the Stylo project, which brings Servo's style system to Firefox. As the Stylo project nears shipping, we've been hammering bindgen into shape for production. This has manifested itself as tons of reliability and correctness work. Our test suite is ever expanding, we've been focusing on correctness of struct layout, size, and alignment for ABI corner cases such as bitfields, as well as test coverage and support across different versions of libclang.

At the same time, we've been working to improve the contribution experience. We've been documenting workflows, adding visualizations of our internal representation for debugging, and mentoring new contributors. Since the creation of the Rust DevTools team, we've also been talking with other toolsmiths about fostering contribution to common tools. Expect to hear more on this soon.

Finally, we also introduced a bindgen Users Guide to help folks get up and running with bindgen in their project.

(Thanks to Nick Fitzgerald for this writeup.)

Other languages/environments

Higher level languages come with their own integration challenges, often involving cooperation with an external runtime system (which possibly includes a garbage collector). Here's a quick rundown of some of the major projects on this front:

  • Ruby: the Helix project is geared toward writing Ruby extensions in Rust, and publicly launched a couple of months ago.
  • Node.js: the Neon project is similarly geared toward writing Node.js modules in Rust, and continues to see active development.
  • The GNOME Object system: after a sprint pairing up Rust and GNOME core developers, we have the basics of an integration story for Rust and the GObject system.
  • The Rust FFI Omnibus gives guidance for the basics of calling into Rust from a variety of languages.

There are many less high-profile projects in this space as well; if you'd like yours to be tracked as part of the roadmap, please leave a comment on the tracking issue!

Conclusion

In conclusion, you can see that it's been a very busy six months in Rust land. I'd like to thank all the many people who have been involved in pushing these projects over the finish line!