LCA: Chrome OS and open firmware

At linux.conf.au, Google's Duncan Laurie presented a talk (slides) about the company's recent work writing open source firmware for its Chrome OS-based laptops. Although there are still some instances of closed firmware on the current crop of Chrome OS devices, the project is clearly making progress on that front—with potential benefits for other free software systems as well.

Laurie, who outlined his previous work on open source firmware projects at Cobalt Networks and Sun, started off by emphasizing that Chrome OS is not a general-purpose operating system. Instead, it is targeted at specific devices, over which Google has control—although, he added, there are people who have built Chromium OS (the open source version of Chrome OS, analogous to the Chromium browser) distributions for other hardware. Google's control over the devices means imposing requirements on the hardware supplier that other Linux distributions generally do not mandate, including support for security features like Chrome OS's verified boot process and inclusion of a Trusted Platform Module (TPM).

But as long as the company is dictating to the hardware supplier, he said, it thought it would be interesting to see if it could be disruptive and open up parts of the firmware payload that are usually closed. The first two generations of Chrome OS devices shipped with an EFI firmware stack that Google licensed from an existing firmware vendor. But Google has no freedom to release any of that code under an open source license, including the code it wrote to integrate its security features.

Opening up the firmware code offers several advantages to the company, including the ability to devote more resources to the firmware project if the project is in danger of missing its ship date, as well as simply having more eyes scrutinize the code. Developing firmware includes tackling a number of hard problems on modern Intel and ARM systems, he said, and working on it regularly uncovers new bugs, from the hardware level on up. Controlling the firmware also gives Google the opportunity to deliver consistency across multiple architectures, and to work on reducing power consumption and increasing performance.

BIOS and booting

Laurie then provided a detailed tour through the Chrome OS firmware stack, highlighting which pieces are open, which are not but could become open, and which have no real chance of ever being opened. His talk focused on the x86 architecture, because it is the platform he and the project as a whole have spent the most time on.

At the core of the firmware stack is coreboot, the open source BIOS replacement. Both coreboot's founder and its current maintainer both work on the Chrome OS firmware team, a fact that Laurie said made coreboot "a pretty easy choice for us." Coreboot's overall structure is quite similar to EFI, he said, since these days there are a fairly well-established set of stages involved in getting a system up and running. One key difference is that coreboot does not include a bootloader; it has a more general "payload" instead, which allows you more flexibility in what to do when bringing up the board. Other interesting differences include the fact that coreboot's first stage ("bootblock") is written partly in assembly, but partly in C that is compiled with romcc, a special compiler that uses only processor registers for variable storage.

Chrome OS uses the U-Boot bootloader as its coreboot payload. This is a bit unusual for x86, he said, since U-Boot is mostly commonly known as an ARM bootloader, but the project chose it because Google had already done the work to integrate its verified boot process into U-Boot. Nevertheless, U-Boot is a bit more complicated than Chrome OS really requires, so the project is keeping the door open to other bootloaders in future releases.

Closed bits and open bits

Coreboot also executes some closed-source binaries, the first of which is the Intel reference code binary required to get memory up and running. This code is provided by Intel to licensed BIOS vendors. It is normally supplied as an EFI module, Laurie said, but Chrome OS wrote wrappers around it to enable its use by coreboot. The project has put binary blobs of those packages on coreboot.org for Sandy Bridge and Ivy Bridge, but the distribution terms prevent them for doing so for other generations of hardware. Google has been working with Intel to find a way for the chip vendor to distribute these newer coreboot binaries, however. The effort is called the Intel Firmware Support Package, and is documented online.

Another blob of closed code provided by Intel is the firmware for the Management Engine, a microcontroller handling various features like clock signal generation. Laurie said the Management Engine makes life more difficult for Chrome OS, since it is quite large (from 1.5 to 5 MB) and difficult to configure and debug. Configuration of the blob is mandatory, and can only be done with an application Intel provides for Windows machines only.

Chrome OS does not normally need to initialize video hardware, since the Intel i915 graphics driver in the Linux kernel can bring up and initialize graphics on its own. However, Chrome OS does need video capabilities for recovery mode and developer mode, so the project currently includes the Intel-provided binary blob for this purpose. But it is exploring another option: extracting the initialization code from the kernel driver to use separately.

The Embedded Controller (EC) is another microcontroller found in virtually all notebooks, and which is responsible for a variety of platform tasks like fan control, battery charging (which, after all, must work even when the computer is powered down), and lid and power-button control. On x86, the EC usually has both a firmware interface and an ACPI interface. Laurie described the EC as the component "closest to his heart;" the project wrote its own EC code after failing to design hardware that would work without an embedded controller at all. The code is open, and is now shipping in the Samsung ARM Chromebook. EC firmware is a tempting (if largely unknown) security target, he said, so Chrome OS has incorporated it into its verified boot process.

Firmware features

Speaking of verified boot, Laurie explained that the feature has a similar design to UEFI Secure Boot, but that its "root of trust" is read-only firmware. To replace this read-only firmware requires physical access to the machine, but Google provided documentation on how to do this for all Chrome OS products. In operation, the read-only firmware verifies the signature of the updatable "read-write" firmware, which in turns verifies the signature of the kernel. The read-only portion of the firmware contains the initial coreboot stages and U-Boot; the read-write firmware would contain any updates and fixes, but is primarily used to verify and boot the kernel.

Chrome OS also includes a "recovery mode" that can be used to restore a system that has been compromised: recovery mode is stored in read-only firmware and will only boot a signed image from a USB stick, not from the local disk. Recovery mode can be triggered if the verified boot process detects a compromise, or can be initiated by the user (on current Chromebooks, via a hardware switch which ensures the user is physically present). Similar methods allow the user to access "developer mode," which is a jailbreak mode built into every Chromebook device. "Chrome OS is very locked down by default, but we really don't want to lock people out from doing interesting things with the hardware they own".

Chrome OS firmware also features a persistent log of system events—because, as Laurie explained, firmware is often the first place people point fingers when they encounter a bug. The log is based on the SMBIOS System Event log, but it has a kernel sysfs interface as well—which makes it possible to debug some kernel problems through the firmware log. The firmware also saves all console output generated during the boot process to a memory buffer, which is then exported at /sys/firmware/log.

Laurie provided a brief overview of the ARM boot process as it differs from the x86 equivalent. Although the vendor-provided binaries appear in different places, they are present on the ARM devices as well. Nevertheless, the boot process is quite similar, ending up with U-Boot booting the signature-verified kernel.

Ultimately, he concluded, it is important to have as much open firmware as possible, since the different layers interact so much even in seemingly simple scenarios. For example, opening a laptop lid cycles through a number of policy decisions (such as whether the user is logged in or not) and hardware events (such as laptop lid switch signals, routed through the Embedded Controller and relayed to the kernel through ACPI events). Most free software fans would likely agree with Laurie's sentiment, although the full inventory of firmware modules involved on a modern PC might come as a surprise to some.