diff options
Diffstat (limited to 'poky/documentation/kernel-dev/kernel-dev-concepts-appx.xml')
| -rw-r--r-- | poky/documentation/kernel-dev/kernel-dev-concepts-appx.xml | 622 |
1 files changed, 622 insertions, 0 deletions
diff --git a/poky/documentation/kernel-dev/kernel-dev-concepts-appx.xml b/poky/documentation/kernel-dev/kernel-dev-concepts-appx.xml new file mode 100644 index 000000000..6d675a6d5 --- /dev/null +++ b/poky/documentation/kernel-dev/kernel-dev-concepts-appx.xml @@ -0,0 +1,622 @@ +<!DOCTYPE chapter PUBLIC "-//OASIS//DTD DocBook XML V4.2//EN" +"http://www.oasis-open.org/docbook/xml/4.2/docbookx.dtd" +[<!ENTITY % poky SYSTEM "../poky.ent"> %poky; ] > + +<appendix id='kernel-dev-concepts-appx'> +<title>Advanced Kernel Concepts</title> + + <section id='kernel-big-picture'> + <title>Yocto Project Kernel Development and Maintenance</title> + + <para> + Kernels available through the Yocto Project (Yocto Linux kernels), + like other kernels, are based off the Linux kernel releases from + <ulink url='http://www.kernel.org'></ulink>. + At the beginning of a major Linux kernel development cycle, the + Yocto Project team chooses a Linux kernel based on factors such as + release timing, the anticipated release timing of final upstream + <filename>kernel.org</filename> versions, and Yocto Project + feature requirements. + Typically, the Linux kernel chosen is in the final stages of + development by the Linux community. + In other words, the Linux kernel is in the release candidate + or "rc" phase and has yet to reach final release. + But, by being in the final stages of external development, the + team knows that the <filename>kernel.org</filename> final release + will clearly be within the early stages of the Yocto Project + development window. + </para> + + <para> + This balance allows the Yocto Project team to deliver the most + up-to-date Yocto Linux kernel possible, while still ensuring that + the team has a stable official release for the baseline Linux + kernel version. + </para> + + <para> + As implied earlier, the ultimate source for Yocto Linux kernels + are released kernels from <filename>kernel.org</filename>. + In addition to a foundational kernel from + <filename>kernel.org</filename>, the available Yocto Linux kernels + contain a mix of important new mainline developments, non-mainline + developments (when no alternative exists), Board Support Package + (BSP) developments, and custom features. + These additions result in a commercially released Yocto + Project Linux kernel that caters to specific embedded designer + needs for targeted hardware. + </para> + + <para> + You can find a web interface to the Yocto Linux kernels in the + <ulink url='&YOCTO_DOCS_OM_URL;#source-repositories'>Source Repositories</ulink> + at + <ulink url='&YOCTO_GIT_URL;'></ulink>. + If you look at the interface, you will see to the left a + grouping of Git repositories titled "Yocto Linux Kernel". + Within this group, you will find several Linux Yocto kernels + developed and included with Yocto Project releases: + <itemizedlist> + <listitem><para> + <emphasis><filename>linux-yocto-4.1</filename>:</emphasis> + The stable Yocto Project kernel to use with the Yocto + Project Release 2.0. + This kernel is based on the Linux 4.1 released kernel. + </para></listitem> + <listitem><para> + <emphasis><filename>linux-yocto-4.4</filename>:</emphasis> + The stable Yocto Project kernel to use with the Yocto + Project Release 2.1. + This kernel is based on the Linux 4.4 released kernel. + </para></listitem> + <listitem><para> + <emphasis><filename>linux-yocto-4.6</filename>:</emphasis> + A temporary kernel that is not tied to any Yocto Project + release. + </para></listitem> + <listitem><para> + <emphasis><filename>linux-yocto-4.8</filename>:</emphasis> + The stable yocto Project kernel to use with the Yocto + Project Release 2.2. + </para></listitem> + <listitem><para> + <emphasis><filename>linux-yocto-4.9</filename>:</emphasis> + The stable Yocto Project kernel to use with the Yocto + Project Release 2.3. + This kernel is based on the Linux 4.9 released kernel. + </para></listitem> + <listitem><para> + <emphasis><filename>linux-yocto-4.10</filename>:</emphasis> + The default stable Yocto Project kernel to use with the + Yocto Project Release 2.3. + This kernel is based on the Linux 4.10 released kernel. + </para></listitem> + <listitem><para> + <emphasis><filename>linux-yocto-4.12</filename>:</emphasis> + The default stable Yocto Project kernel to use with the + Yocto Project Release 2.4. + This kernel is based on the Linux 4.12 released kernel. + </para></listitem> + <listitem><para> + <emphasis><filename>yocto-kernel-cache</filename>:</emphasis> + The <filename>linux-yocto-cache</filename> contains + patches and configurations for the linux-yocto kernel + tree. + This repository is useful when working on the linux-yocto + kernel. + For more information on this "Advanced Kernel Metadata", + see the + "<link linkend='kernel-dev-advanced'>Working With Advanced Metadata (<filename>yocto-kernel-cache</filename>)</link>" + Chapter. + </para></listitem> + <listitem><para> + <emphasis><filename>linux-yocto-dev</filename>:</emphasis> + A development kernel based on the latest upstream release + candidate available. + </para></listitem> + </itemizedlist> + <note><title>Notes</title> + Long Term Support Initiative (LTSI) for Yocto Linux + kernels is as follows: + <itemizedlist> + <listitem><para> + For Yocto Project releases 1.7, 1.8, and 2.0, + the LTSI kernel is + <filename>linux-yocto-3.14</filename>. + </para></listitem> + <listitem><para> + For Yocto Project releases 2.1, 2.2, and 2.3, + the LTSI kernel is <filename>linux-yocto-4.1</filename>. + </para></listitem> + <listitem><para> + For Yocto Project release 2.4, the LTSI kernel is + <filename>linux-yocto-4.9</filename> + </para></listitem> + <listitem><para> + <filename>linux-yocto-4.4</filename> is an LTS + kernel. + </para></listitem> + </itemizedlist> + </note> + </para> + + <para> + Once a Yocto Linux kernel is officially released, the Yocto + Project team goes into their next development cycle, or upward + revision (uprev) cycle, while still continuing maintenance on the + released kernel. + It is important to note that the most sustainable and stable way + to include feature development upstream is through a kernel uprev + process. + Back-porting hundreds of individual fixes and minor features from + various kernel versions is not sustainable and can easily + compromise quality. + </para> + + <para> + During the uprev cycle, the Yocto Project team uses an ongoing + analysis of Linux kernel development, BSP support, and release + timing to select the best possible <filename>kernel.org</filename> + Linux kernel version on which to base subsequent Yocto Linux + kernel development. + The team continually monitors Linux community kernel development + to look for significant features of interest. + The team does consider back-porting large features if they have a + significant advantage. + User or community demand can also trigger a back-port or creation + of new functionality in the Yocto Project baseline kernel during + the uprev cycle. + </para> + + <para> + Generally speaking, every new Linux kernel both adds features and + introduces new bugs. + These consequences are the basic properties of upstream + Linux kernel development and are managed by the Yocto Project + team's Yocto Linux kernel development strategy. + It is the Yocto Project team's policy to not back-port minor + features to the released Yocto Linux kernel. + They only consider back-porting significant technological + jumps ‐ and, that is done after a complete gap analysis. + The reason for this policy is that back-porting any small to + medium sized change from an evolving Linux kernel can easily + create mismatches, incompatibilities and very subtle errors. + </para> + + <para> + The policies described in this section result in both a stable + and a cutting edge Yocto Linux kernel that mixes forward ports of + existing Linux kernel features and significant and critical new + functionality. + Forward porting Linux kernel functionality into the Yocto Linux + kernels available through the Yocto Project can be thought of as + a "micro uprev." + The many “micro uprevs” produce a Yocto Linux kernel version with + a mix of important new mainline, non-mainline, BSP developments + and feature integrations. + This Yocto Linux kernel gives insight into new features and + allows focused amounts of testing to be done on the kernel, + which prevents surprises when selecting the next major uprev. + The quality of these cutting edge Yocto Linux kernels is evolving + and the kernels are used in leading edge feature and BSP + development. + </para> + </section> + + <section id='yocto-linux-kernel-architecture-and-branching-strategies'> + <title>Yocto Linux Kernel Architecture and Branching Strategies</title> + + <para> + As mentioned earlier, a key goal of the Yocto Project is + to present the developer with a kernel that has a clear and + continuous history that is visible to the user. + The architecture and mechanisms, in particular the branching + strategies, used achieve that goal in a manner similar to + upstream Linux kernel development in + <filename>kernel.org</filename>. + </para> + + <para> + You can think of a Yocto Linux kernel as consisting of a + baseline Linux kernel with added features logically structured + on top of the baseline. + The features are tagged and organized by way of a branching + strategy implemented by the Yocto Project team using the + Source Code Manager (SCM) Git. + <note><title>Notes</title> + <itemizedlist> + <listitem><para> + Git is the obvious SCM for meeting the Yocto Linux + kernel organizational and structural goals described + in this section. + Not only is Git the SCM for Linux kernel development in + <filename>kernel.org</filename> but, Git continues to + grow in popularity and supports many different work + flows, front-ends and management techniques. + </para></listitem> + <listitem><para> + You can find documentation on Git at + <ulink url='http://git-scm.com/documentation'></ulink>. + You can also get an introduction to Git as it + applies to the Yocto Project in the + "<ulink url='&YOCTO_DOCS_OM_URL;#git'>Git</ulink>" + section in the Yocto Project Overview and Concepts + Manual. + The latter reference provides an overview of + Git and presents a minimal set of Git commands + that allows you to be functional using Git. + You can use as much, or as little, of what Git + has to offer to accomplish what you need for your + project. + You do not have to be a "Git Expert" in order to + use it with the Yocto Project. + </para></listitem> + </itemizedlist> + </note> + </para> + + <para> + Using Git's tagging and branching features, the Yocto Project + team creates kernel branches at points where functionality is + no longer shared and thus, needs to be isolated. + For example, board-specific incompatibilities would require + different functionality and would require a branch to + separate the features. + Likewise, for specific kernel features, the same branching + strategy is used. + </para> + + <para> + This "tree-like" architecture results in a structure that has + features organized to be specific for particular functionality, + single kernel types, or a subset of kernel types. + Thus, the user has the ability to see the added features and the + commits that make up those features. + In addition to being able to see added features, the user + can also view the history of what made up the baseline + Linux kernel. + </para> + + <para> + Another consequence of this strategy results in not having to + store the same feature twice internally in the tree. + Rather, the kernel team stores the unique differences required + to apply the feature onto the kernel type in question. + <note> + The Yocto Project team strives to place features in the tree + such that features can be shared by all boards and kernel + types where possible. + However, during development cycles or when large features + are merged, the team cannot always follow this practice. + In those cases, the team uses isolated branches to merge + features. + </note> + </para> + + <para> + BSP-specific code additions are handled in a similar manner to + kernel-specific additions. + Some BSPs only make sense given certain kernel types. + So, for these types, the team creates branches off the end + of that kernel type for all of the BSPs that are supported on + that kernel type. + From the perspective of the tools that create the BSP branch, + the BSP is really no different than a feature. + Consequently, the same branching strategy applies to BSPs as + it does to kernel features. + So again, rather than store the BSP twice, the team only + stores the unique differences for the BSP across the supported + multiple kernels. + </para> + + <para> + While this strategy can result in a tree with a significant number + of branches, it is important to realize that from the developer's + point of view, there is a linear path that travels from the + baseline <filename>kernel.org</filename>, through a select + group of features and ends with their BSP-specific commits. + In other words, the divisions of the kernel are transparent and + are not relevant to the developer on a day-to-day basis. + From the developer's perspective, this path is the "master" branch + in Git terms. + The developer does not need to be aware of the existence of any + other branches at all. + Of course, value exists in the having these branches in the tree, + should a person decide to explore them. + For example, a comparison between two BSPs at either the commit + level or at the line-by-line code <filename>diff</filename> level + is now a trivial operation. + </para> + + <para> + The following illustration shows the conceptual Yocto + Linux kernel. + <imagedata fileref="figures/kernel-architecture-overview.png" width="6in" depth="7in" align="center" scale="100" /> + </para> + + <para> + In the illustration, the "Kernel.org Branch Point" marks the + specific spot (or Linux kernel release) from which the + Yocto Linux kernel is created. + From this point forward in the tree, features and differences + are organized and tagged. + </para> + + <para> + The "Yocto Project Baseline Kernel" contains functionality that + is common to every kernel type and BSP that is organized + further along in the tree. + Placing these common features in the tree this way means + features do not have to be duplicated along individual + branches of the tree structure. + </para> + + <para> + From the "Yocto Project Baseline Kernel", branch points represent + specific functionality for individual Board Support Packages + (BSPs) as well as real-time kernels. + The illustration represents this through three BSP-specific + branches and a real-time kernel branch. + Each branch represents some unique functionality for the BSP + or for a real-time Yocto Linux kernel. + </para> + + <para> + In this example structure, the "Real-time (rt) Kernel" branch has + common features for all real-time Yocto Linux kernels and + contains more branches for individual BSP-specific real-time + kernels. + The illustration shows three branches as an example. + Each branch points the way to specific, unique features for a + respective real-time kernel as they apply to a given BSP. + </para> + + <para> + The resulting tree structure presents a clear path of markers + (or branches) to the developer that, for all practical + purposes, is the Yocto Linux kernel needed for any given set of + requirements. + <note> + Keep in mind the figure does not take into account all the + supported Yocto Linux kernels, but rather shows a single + generic kernel just for conceptual purposes. + Also keep in mind that this structure represents the Yocto + Project + <ulink url='&YOCTO_DOCS_OM_URL;#source-repositories'>Source Repositories</ulink> + that are either pulled from during the build or established + on the host development system prior to the build by either + cloning a particular kernel's Git repository or by + downloading and unpacking a tarball. + </note> + </para> + + <para> + Working with the kernel as a structured tree follows recognized + community best practices. + In particular, the kernel as shipped with the product, should be + considered an "upstream source" and viewed as a series of + historical and documented modifications (commits). + These modifications represent the development and stabilization + done by the Yocto Project kernel development team. + </para> + + <para> + Because commits only change at significant release points in the + product life cycle, developers can work on a branch created + from the last relevant commit in the shipped Yocto Project Linux + kernel. + As mentioned previously, the structure is transparent to the + developer because the kernel tree is left in this state after + cloning and building the kernel. + </para> + </section> + + <section id='kernel-build-file-hierarchy'> + <title>Kernel Build File Hierarchy</title> + + <para> + Upstream storage of all the available kernel source code is + one thing, while representing and using the code on your host + development system is another. + Conceptually, you can think of the kernel source repositories + as all the source files necessary for all the supported + Yocto Linux kernels. + As a developer, you are just interested in the source files + for the kernel on which you are working. + And, furthermore, you need them available on your host system. + </para> + + <para> + Kernel source code is available on your host system several + different ways: + <itemizedlist> + <listitem><para> + <emphasis>Files Accessed While using <filename>devtool</filename>:</emphasis> + <filename>devtool</filename>, which is available with the + Yocto Project, is the preferred method by which to + modify the kernel. + See the + "<link linkend='kernel-modification-workflow'>Kernel Modification Workflow</link>" + section. + </para></listitem> + <listitem><para> + <emphasis>Cloned Repository:</emphasis> + If you are working in the kernel all the time, you probably + would want to set up your own local Git repository of the + Yocto Linux kernel tree. + For information on how to clone a Yocto Linux kernel + Git repository, see the + "<link linkend='preparing-the-build-host-to-work-on-the-kernel'>Preparing the Build Host to Work on the Kernel</link>" + section. + </para></listitem> + <listitem><para> + <emphasis>Temporary Source Files from a Build:</emphasis> + If you just need to make some patches to the kernel using + a traditional BitBake workflow (i.e. not using the + <filename>devtool</filename>), you can access temporary + kernel source files that were extracted and used during + a kernel build. + </para></listitem> + </itemizedlist> + </para> + + <para> + The temporary kernel source files resulting from a build using + BitBake have a particular hierarchy. + When you build the kernel on your development system, all files + needed for the build are taken from the source repositories + pointed to by the + <ulink url='&YOCTO_DOCS_REF_URL;#var-SRC_URI'><filename>SRC_URI</filename></ulink> + variable and gathered in a temporary work area where they are + subsequently used to create the unique kernel. + Thus, in a sense, the process constructs a local source tree + specific to your kernel from which to generate the new kernel + image. + </para> + + <para> + The following figure shows the temporary file structure + created on your host system when you build the kernel using + Bitbake. + This + <ulink url='&YOCTO_DOCS_REF_URL;#build-directory'>Build Directory</ulink> + contains all the source files used during the build. + <imagedata fileref="figures/kernel-overview-2-generic.png" + width="6in" depth="5in" align="center" scale="100" /> + </para> + + <para> + Again, for additional information on the Yocto Project kernel's + architecture and its branching strategy, see the + "<link linkend='yocto-linux-kernel-architecture-and-branching-strategies'>Yocto Linux Kernel Architecture and Branching Strategies</link>" + section. + You can also reference the + "<link linkend='using-devtool-to-patch-the-kernel'>Using <filename>devtool</filename> to Patch the Kernel</link>" + and + "<link linkend='using-traditional-kernel-development-to-patch-the-kernel'>Using Traditional Kernel Development to Patch the Kernel</link>" + sections for detailed example that modifies the kernel. + </para> + </section> + + <section id='determining-hardware-and-non-hardware-features-for-the-kernel-configuration-audit-phase'> + <title>Determining Hardware and Non-Hardware Features for the Kernel Configuration Audit Phase</title> + + <para> + This section describes part of the kernel configuration audit + phase that most developers can ignore. + For general information on kernel configuration including + <filename>menuconfig</filename>, <filename>defconfig</filename> + files, and configuration fragments, see the + "<link linkend='configuring-the-kernel'>Configuring the Kernel</link>" + section. + </para> + + <para> + During this part of the audit phase, the contents of the final + <filename>.config</filename> file are compared against the + fragments specified by the system. + These fragments can be system fragments, distro fragments, + or user-specified configuration elements. + Regardless of their origin, the OpenEmbedded build system + warns the user if a specific option is not included in the + final kernel configuration. + </para> + + <para> + By default, in order to not overwhelm the user with + configuration warnings, the system only reports missing + "hardware" options as they could result in a boot + failure or indicate that important hardware is not available. + </para> + + <para> + To determine whether or not a given option is "hardware" or + "non-hardware", the kernel Metadata in + <filename>yocto-kernel-cache</filename> contains files that + classify individual or groups of options as either hardware + or non-hardware. + To better show this, consider a situation where the + <filename>yocto-kernel-cache</filename> contains the following + files: + <literallayout class='monospaced'> + yocto-kernel-cache/features/drm-psb/hardware.cfg + yocto-kernel-cache/features/kgdb/hardware.cfg + yocto-kernel-cache/ktypes/base/hardware.cfg + yocto-kernel-cache/bsp/mti-malta32/hardware.cfg + yocto-kernel-cache/bsp/fsl-mpc8315e-rdb/hardware.cfg + yocto-kernel-cache/bsp/qemu-ppc32/hardware.cfg + yocto-kernel-cache/bsp/qemuarma9/hardware.cfg + yocto-kernel-cache/bsp/mti-malta64/hardware.cfg + yocto-kernel-cache/bsp/arm-versatile-926ejs/hardware.cfg + yocto-kernel-cache/bsp/common-pc/hardware.cfg + yocto-kernel-cache/bsp/common-pc-64/hardware.cfg + yocto-kernel-cache/features/rfkill/non-hardware.cfg + yocto-kernel-cache/ktypes/base/non-hardware.cfg + yocto-kernel-cache/features/aufs/non-hardware.kcf + yocto-kernel-cache/features/ocf/non-hardware.kcf + yocto-kernel-cache/ktypes/base/non-hardware.kcf + yocto-kernel-cache/ktypes/base/hardware.kcf + yocto-kernel-cache/bsp/qemu-ppc32/hardware.kcf + </literallayout> + The following list provides explanations for the various + files: + <itemizedlist> + <listitem><para> + <filename>hardware.kcf</filename>: + Specifies a list of kernel Kconfig files that contain + hardware options only. + </para></listitem> + <listitem><para> + <filename>non-hardware.kcf</filename>: + Specifies a list of kernel Kconfig files that contain + non-hardware options only. + </para></listitem> + <listitem><para> + <filename>hardware.cfg</filename>: + Specifies a list of kernel <filename>CONFIG_</filename> + options that are hardware, regardless of whether or not + they are within a Kconfig file specified by a hardware + or non-hardware Kconfig file (i.e. + <filename>hardware.kcf</filename> or + <filename>non-hardware.kcf</filename>). + </para></listitem> + <listitem><para> + <filename>non-hardware.cfg</filename>: + Specifies a list of kernel <filename>CONFIG_</filename> + options that are not hardware, regardless of whether or + not they are within a Kconfig file specified by a + hardware or non-hardware Kconfig file (i.e. + <filename>hardware.kcf</filename> or + <filename>non-hardware.kcf</filename>). + </para></listitem> + </itemizedlist> + Here is a specific example using the + <filename>kernel-cache/bsp/mti-malta32/hardware.cfg</filename>: + <literallayout class='monospaced'> + CONFIG_SERIAL_8250 + CONFIG_SERIAL_8250_CONSOLE + CONFIG_SERIAL_8250_NR_UARTS + CONFIG_SERIAL_8250_PCI + CONFIG_SERIAL_CORE + CONFIG_SERIAL_CORE_CONSOLE + CONFIG_VGA_ARB + </literallayout> + The kernel configuration audit automatically detects these + files (hence the names must be exactly the ones discussed here), + and uses them as inputs when generating warnings about the + final <filename>.config</filename> file. + </para> + + <para> + A user-specified kernel Metadata repository, or recipe space + feature, can use these same files to classify options that are + found within its <filename>.cfg</filename> files as hardware + or non-hardware, to prevent the OpenEmbedded build system from + producing an error or warning when an option is not in the + final <filename>.config</filename> file. + </para> + </section> +</appendix> +<!-- +vim: expandtab tw=80 ts=4 +--> |