Embedded Firmware Solutions: Development Best Practices for the Internet of Things (2015)
By Ron Minnich
Creator of LinuxBIOS (later renamed coreboot)
Software Engineer at Google, Inc.
I started the coreboot project at Los Alamos National Lab in 1999. At the time, it was seen by hardware vendors as an impractical idea that would soon vanish. Now, 15 years later, it is mainstream: millions of x86- and ARM-based Chromebooks and Chromeboxes run coreboot, as its speed and reliability are an essential part of the Chrome OS user experience. coreboot is now a key component of the fastest-growing consumer laptop segment.
It might come as a surprise to embedded programmers that the initial goal of coreboot was to make very large supercomputing clusters manageable. We had a 128-node VA Linux cluster at Los Alamos in 1999 that had no keyboards or displays. BIOS upgrades required that we wheel around a "crash cart" with a keyboard and monitor; boot DOS on a floppy, which in turn started an autoexec.bat script; and wait 5 to 10 minutes for the process to complete for each node. If anything went wrong, it got more fun: we had to crack open the case, move a jumper, and do the recovery with no working graphics. As if this were not bad enough, the vendor BIOS had a habit of coming up displaying "No keyboard present—hit F1 to continue" on a nonexistent monitor, asking us to hit F1 on a keyboard that it had already discovered was not there!
Could this possibly get any worse? It could, and did, on the Thunderbird cluster at Sandia National Labs: 4400 nodes, none having a keyboard or a monitor, came up one day with that same vendor BIOS message. The fix? Dispatch 20 people with 20 keyboards to 220 machines each; they had to plug in the keyboard, hit F1, and hope it all went well.
By 2002, we had a 1024-node Linux cluster using coreboot. The reflash process for all 1024 nodes took 30 seconds total, not five days. If something went wrong, coreboot would figure it out on the next boot, switch to a backup BIOS image, and boot up: the nodes could not be put into an unrecoverable state. There was no longer a need to open the nodes and move a jumper. coreboot represented a huge jump in the manageability of cluster nodes.
coreboot has had many uses since its inception: everything from the smallest systems (Apache Military Modem II) to some of the largest supercomputers. While there is wide adoption in Chrome OS systems, coreboot’s earliest and continuing use is in embedded systems such as televisions, network switches, and robotic systems. In fact, about the same time we deployed a supercomputer using coreboot, iRobot had ported coreboot to its Packbot robot.
Embedded systems used to be very simple: a low-power CPU connected to low-performance memory and peripherals, used in low-performance and limited applications such as digital clocks and automobile computers. But in the last ten years, we can see low-power embedded CPUs used in unexpected places. The highest-end systems—such as IBM Blue Gene supercomputers, which were the fastest in the world for many years—used 65,536 embedded PowerPC CPUs with 18 cores each. We now see higher-power CPUs used in small embedded systems such as Chromebox videoconferencing systems—an inexpensive system with a very powerful Intel CPU.
These embedded Intel CPUs have memory bandwidth much higher than classic Cray vector supercomputers and hence are fiendishly complex to design. Once designed, this high-performance hardware is quite difficult to initialize, and even should we wish to write the code to manage the initialization, the programming information is not public.
This has led to a dilemma: How can we enable coreboot on complex systems that are not fully open?
This book shows one path. Intel has released in binary form a basic set of functions to initialize the messiest—and hardest to program—bits of the Intel chipset. The calling conventions and behavior of the binary are completely and clearly documented. The developer is freed from having to deal with very difficult chipset setup. Upgrades of this software are simple: just replace the old binary with a new one. This code is called the Firmware Support Package, a.k.a. FSP.
The result is that high-performance Intel chipsets can be used with coreboot in all kinds of systems, including embedded ones, with a binary supplied by Intel that removes much of the porting difficulty.
Wearable embedded systems are a growing area right now, and many use the CPU described in this book. This book is an ideal companion for those wishing to be current with current and future embedded technology.
coreboot has succeeded because of the efforts of the many talented people involved in the project for the last 15 years. The reader is fortunate that this book is written by four of the best minds in the business. There is a lot to learn here and it will stand you in good stead if you continue to work in embedded systems.
I’d like to thank Jiming Sun and the team for conceiving FSP and bringing it to fruition. Without their tireless efforts and diplomacy, we would not have FSP or this book.