15 Years of i.MX in Mainline Linux

Today it has been 15 years since we mainlined support for Freescale/NXP's i.MX architecture in the Linux kernel! That was one small step for [a] man, one giant leap for (industrial Linux users') mankind :-) Here is some background about why it happened and what you might want to learn from history for your next embedded Linux project.

How to react to risk?

Back in 2004, Pengutronix was contacted by a customer who developed a medical device: they had selected a nice CPU for their new hardware, all relevant peripherals matched the requirements and the chip manufacturer promised 14 years of device availability for that shiny MX1. In fact, MX1 was the first ARM based processor from Freescale, still labeled as "DragonBall" at that time (which was the marketing name of the previous series of M68k based devices), contained a 200 MHz ARM920T core and looked like a perfect choice for the job. Marketing material had a check mark at "Linux Support", so what could possibly go wrong...

At the time we got contacted, someone had asked Freescale if they also provide reliable operating system software support for those 14 years, and it quickly turned out that the Linux port was an "Enablement BSP", made to show that the hardware was operable. However, as a medical device manufacturer, there are strong regulatory processes in place which in fact makes it absolutely necessary that you are able to react if some kind of risk turns up. So what do you do if, for instance, a security flaw in the kernel is found during that 14 years? You could port the i.MX1 patches forward to a newer kernel that contains fixes, but this plan quickly turned out to be impossible. Hundreds of patches had been stacked to an already old kernel at that time, with less than optimal topic separation and documentation, so the effort for even understanding what had to be done was unacceptable. Even worse, like almost all vendor code at that time, the kernel port had been made without talking to the community or reviewing the patches due to confidentiality reasons.

So what should our customer do? Everyone was absolutely convinced that it would be impossible for a small project team to make a mainline port, taken that obviously not even a big chip manufacturer was in a position to do that.

So we sat in the office and thought: Challenge accepted!

Taking back control with the power of Open Source

When we were asked for assistance, we quickly found out that, while there was a lot of code for all the many peripherals in i.MX1, our customer fortunately only used a small subset for his actual project. We analyzed the patches carefully, ripped them apart, sorted them into interesting and non-interesting pieces, modified the better ones and rewrote other parts of the port and moved all that on top of the latest mainline kernel. It took us about 6 weeks to have a first mainline based kernel. Linux code is licensed under the terms of the GPLv2, so modifying other people's kernel code was always legally possible, so even we as just-a-few-random-hackers were allowed to just do what was necessary without lengthy discussions with the legal teams of big companies. After the six weeks, we had a port that was good enough to actually work for the customer, and even good enough to be posted on the Linux ARM Kernel Mailing List (LAKML). Patches were discussed, reviewed by other kernel hackers, improvements had been added, and finally we proposed the patch stack to be included in the mainline kernel.

After some time, we were asked if Sascha Hauer would be interested in being the official i.MX maintainer, we agreed and during the next years all the i.MX patches went through our hands. MX27, MX31, MX25, MX51 and finally MX53 brought us many new industrial projects: the word spread quickly in the industrial community that this mainline port was of high software quality. In fact, it was not us being some magic genius, but simply because of the development process of the Linux kernel that makes best use of the community's technology-focussed quality process and involves many, many smart people out there to focus all that expertise in one mainline kernel.

Happy birthday, i.MX Mainline Port!

Many industrial device manufacturers based their product development on mainline since then - with and without our help. After a while, Freescale joined the community and let many of their developers contribute to the mainline kernel and even to the maintenance effort.

The community grew, i.MX6 got a big success story in the industry. With Etnaviv and the VPU drivers, open source support for the 3D GPU and the video encoders/decoders were added, and today we are working on mainline support for i.MX8M.

Finally, even today, 15 years after the first port, a top-of-tree 5.3-rc kernel is still able to boot on i.MX1 with only minor modifications (see a screencast of a boot from today below)

Happy birthday, i.MX!


Weiterführende Links

umpf - Git on a New Level

Moderne Softwareentwicklung ohne begleitende Versionsverwaltung wie Git ist heutzutage unvorstellbar - Änderungen am Quellcode sollen schließlich nachvollziehbar dokumentiert und beliebige Verssionsstände jederzeit einfach reproduziert werden können. Für Arbeiten an komplexeren Projekten wie etwa dem BSP ("Board Support Package") eines eingebetteten Systems mit mehreren Entwicklungssträngen skaliert ein bloßes Aufeinanderstapeln der einzelnen Änderungen jedoch nicht.

Die Pengutronix Kernel-Beiträge in 2021

2022 hat begonnen, und obwohl Corona unsere Workflows stark verändert hat, hat das Pengutronix Team auch in diesem Jahr wieder etliche Beiträge zum Linux-Kernel geleistet. Das letzte Kernel-Release in 2020 war 5.10, das letzte in 2021 war 5.15 - schauen wir also, was sich dazwischen bei uns getan hat.

Showcase: Embedded off-the-shelf

Ein Firmware-Upgrade ist fällig. Eine neu implementierte Funktion muss ausgerollt, eine Sicherheitslücke gepatcht oder neue Hardware-Unterstützung hinzugefügt werden. Die Software ist zwar leistungsfähig, aber komplex. Pengutronix' Strategie, mit dieser Komplexität umzugehen, ist die Arbeit an einem versionskontrollierten Board Support Package (BSP) mit kontinuierlichen Updates und Tests auf dem neuesten Mainline-Linux-Kernel.

Pengutronix at FOSDEM 2021

"FOSDEM is a free event for software developers to meet, share ideas and collaborate. Every year, thousands of developers of free and open source software from all over the world gather at the event in Brussels. In 2021, they will gather online." -- FOSDEM

Pengutronix auf dem ESE-Kongress

Auch der Embedded Software Engineering Kongress findet dieses Jahr online statt, und wir möchten die Chance nutzen, diesen als Event-Partner zu unterstützen, mit Leuten ins Gespräch zu kommen und gleichzeitig ein paar interessanten Vorträgen zu lauschen.

What Kernel Should I Use (Embedded Edition)

Some days ago, Greg Kroah-Hartmann wrote a great blogpost about Which Stable Kernel One Should Use?. I fully agree with his position; however, I'd like to make some additions for the industry device manufacturer use case and some common pitfalls and misunderstandings we see in that area.