Monday, June 22, 2015

Boot Args - Basics, Usage and How to add a custom boot arg?



In the past, I have added quite a few custom boot args that improve debugging ability of linux kernel code. In this post, I will go over basics of linux boot args, usage in Ubuntu context and how to add a custom boot arg.


What are Boot Args?

Boot Args are also referred to as boot params or boot options.

Just the way a linux program's options can be supplied in command line, linux kernel can be supplied with options that can define the kernel behavior.

For example, Boot args are useful for the following:
  • enable some debug features (say slub_debug)
  • Override default configuration values (say panic)
  • supply information about the hardware kernel might not be able to determine on it's own
 
Most of the boot arguments have the form:
           name[=value_1][,value_2]...[,value_10]

where 'name' is a unique keyword that is used to identify what part of the kernel the associated values (if any) are to be given to.


Some well-known boot args:
  • root: 
This argument tells the kernel what device is to be used as the root file system while booting. The default of this setting is determined at compile time, and usually is the value of the root device of the system that the kernel was built on. To override this value, and select the second floppy drive as the root device, one would use  
                     'root=/dev/fd1' 
The root device can be specified symbolically or numerically. A symbolic specification has the form /dev/XXYN, where XX designates the device type (e.g., 'hd' for ST-506 compatible hard disk, with Y in 'a'-'d'; 'sd' for SCSI compatible disk, with Y in 'a'-'e'), Y the driver letter or number, and N the number (in decimal) of the partition on this device.
Note that this has nothing to do with the designation of these devices on your filesystem. The '/dev/' part is purely conventional.
The more awkward and less portable numeric specification of the above possible root devices in major/minor format is also accepted. (For example, /dev/sda3 is major 8, minor 3, so you could use 'root=0x803' as an alternative.)
Example: root=/dev/sda1
  • rootfstype:
The 'rootfstype' option tells the kernel to mount the root file system as if it where of the type specified. This can be useful (for example) to mount an ext3 filesystem as ext2 and then remove the journal in the root filesystem, in fact reverting its format from ext3 to ext2 without the need to boot the box from alternate media.
Example: rootfstype=ext4
  • initrdSpecify the location of the initial ramdisk.
Example: initrd=\initramfs-linux.img
  • init:
This sets the initial command to be executed by the kernel.If this is not set, or cannot be found, the kernel will try /sbin/init, then /etc/init, then /bin/init, then /bin/sh and panic if all of this fails.
Example: init=/bin/sh 
  • debug
This boot argument sets console log level to KERNEL_DEBUG & hence causing a lot of debug messages on console. The console loglevel can also be set on a booted system via the /proc/sys/kernel/printk file.
 Example: debug 
  • quiet:
This is pretty much the opposite of the `debug' argument. Set the default kernel log level to KERN_WARNING, which suppresses all console messages during boot except extremely serious ones. Normal messages about hardware detection at boot are suppressed.
Example: quiet  
  • slub_debug: Usage - slub_debug[=options[,slabs]]
Enabling slub_debug allows one to determine the culprit if slab objects become corrupted. Enabling slub_debug can create guard zones around objects and may poison objects when not in use. Also tracks the last alloc / free. For more information see Documentation/vm/slub.txt. 
Example: slub_debug  
  • kmemeleak:
Boot-time kmemleak enable/disable
Valid arguments: on, off
Default: on
Example: kmemleak=off 
  • kgdoc
kgdb over consoles. Requires a tty driver that supports console polling, or a supported polling keyboard driver (non-usb). 
Serial only format: <serial_device>[,baud]
keyboard only format: kbd
keyboard and serial format: kbd,<serial_device>[,baud]
Optional Kernel mode setting:
 kms, kbd format: kms,kbd
 kms, kbd and serial format: kms,kbd,<ser_dev>[,baud]
Example: kgdboc=ttyS0,115200  
  • panic: Usage - panic=N
By default, the kernel will not reboot after a panic, but this option will cause a kernel reboot after N seconds from the panic event (if N is greater than zero). This panic timeout can also be set by
                  echo N > /proc/sys/kernel/panic
 Example: panic=15 
  • ....
  • ....


The descriptions of the above boot args are copy pasted from one of the references listed at the end of this post.

Sample bootarg command lines:
  • bootargs=root=/dev/sda1 rootfstype=ext4 quiet
  • bootargs=root=/dev/sda2 rootfstype=ext4 quiet slub_debug panic=10 init=/bin/sh


How to edit boot options in Ubuntu?

Editing boot args is different with different boot loaders. I have GRUB boot loader in the Ubuntu VM I created. The GRUB how to for editing boot args can be found in my other post here : How to edit boot options during linux booting?

What is maximum allowed size of bootargs and maximum number of bootargs?
The number of kernel parameters is not limited, but the length of the complete command line (parameters including spaces etc.) is limited to a fixed number of characters. This limit depends on the architecture and is between 256 and 4096 characters. It is defined in the file ./include/asm/setup.h as COMMAND_LINE_SIZE. For x86 architecture, it is set to 2K: https://lxr.missinglinkelectronics.com/linux/arch/x86/include/asm/setup.h#L6 




After the system boots, how can we find the boot args it was loaded?

The bootloader passes the boot parameters to the Linux kernel in a memory buffer called the kernel command line. A copy of the kernel command line is in the file /proc/cmdline.
root@babu-VirtualBox:/# cat /proc/cmdline
root=/dev/sda1 rootfstype=ext4 quiet
root@babu-VirtualBox:/# 
How to create a new/custom boot arg?

Anywhere in linux kernel code, add a function setup function as done for slub_debug: https://lxr.missinglinkelectronics.com/linux/mm/slub.c#L1167

Also, define the setup function too as done for slub_debug: https://lxr.missinglinkelectronics.com/linux/mm/slub.c#L1099


Where are boot args initialized in the kernel initialization sequence?
I just went through the kernel init code to understand the stage at which boot args are being initiatlied - https://lxr.missinglinkelectronics.com/linux/init/main.c#L535.Looks like it's pretty early - even before initialization of memory management code. There are only around 10 init functions before boot args are being initalized. 

Can a boot arg setup function be part of a kernel module?

No. 
Boot args are loaded first. Modules are loaded much later. So, if we define these __setup function for a boot param inside a kernel module, kernel would assume that corresponding boot arg set up function is not defined (as the module is not loaded yet, symbol won't be found). And hence skips processing that boot param. So, the correct method is to include all the __setup functions as part of core kernel code itself.



References:



Monday, June 15, 2015

Web based Code browsing for Linux and my own hosting experiments


In this post, I will provide some useful resources to browse indexed linux code on the web.

The order below reflects my preference (it can be different for others, just try it out):
  • Missing Link Electronics: lxr.linux.no was my earlier preference, but that seems to be down for a while now. After it's down, I found this website that seems to be the next best alternative. These are based on LXRng engine, which is a light weight variation of LXR engine.
  • lxr.free-electrons.com: This is based on LXR engine which is more robust I believe, but I haven't tried using the engine myself yet. I also don't like the interface as much as I like the LXRng engine based websites.

The other non-web based indexing alternative I used before switching to web based indexer was cscope/ctags combo. The primary advantage of web based indexer is being able to browse code via links.

In fact, I liked web based indexing over cscope/ctags combo so much that I created an internal website at Brocade that helps developers and managers to browse the latest code for Brocade's internal development and release branches. LXrng engine can index any code not just linux. It became popular enough at Brocade to draw more than 1000 hits/day on average & has been up for more than 3 years now. Creating this service wasn't as simple of just using LXRng engine, but involved the following challenges:
  • Bring up LXRng engine on a linux machine: LXRng engine just indexes the code repositories it is fed with. 
  • Scripting engine for auto updation of code repositories: As internal developmental branches are very dynamic and change on daily basis, I thought a daily indexer based on latest time stamps is a better idea. So, I created an additional scripting engine that can be run in a crontab on a daily basis. This engine updates the LXRng engine's code repositories and configuration files daily.
  • DNS entry for internal Website: Pick an internal website name & create a DNS entry to the IP address of linux server on which LXRng engine is being run.
  • Statistics: Develop a page that helps find the usage of the web service.

The main advantage of the service I created was in the scripting engine that automatically updates the latest code. That helps developers access latest code without having to spend time to index their code daily - which can take more than a hour for cscope/ctags users.


Some of the issues I observed while indxing code using LXRng engine are:
  • Issue1 - Incorrect Results for Text Search: The results displayed in "Freetext Search" are found to be incorrect in some cases. 
  • Issue2 - Some Defunct Inlcude file links within C files: Reasons for this:
    • The ability to click on an include file link is an advanced feature provided by the open source engine (LXRng) used for indexing. To provide this feature, LXRng requires information about include file directory names in it's configuration file. Linux's include files are all located in one directory: include/linux & hence configuring LXRng config file for linux code repository is easy. However, this need not be same with all code repositories. Include files may be distributed than centralized. So, providing information about all include paths of a code repository can be challenging for those who may not be aware of code organization thoroughly. If you miss any of the include paths, it results in this issue.
    • If the directory name, under which your files exist, is not unique directory name in the system, local include file links of the c files in those directories will not open. For example, lets take linux code repository. And lets say a.h is used in a.c file under linux_2_6_34/mm. "a.h" in "a.c" doesn't open, as the directory (linux_2_6_34/mm) in which these files reside is not a unique directory name in linux codebase (there exists linux_2_6_34/arch/powerpc/mm is the duplicate directory). I haven't yet found a solution for this issue.
  • Issue3 - Incorrect references for symbols in defunct h files: Another issue observed due to last two issues mentioned in Issue2 above is is that search results of structures or functions defined in defunct .h files would show references only to the h file in which it is defined, but ignores references in .c files. You can work around this issue by searching for the symbol in "Search" box instead.
  • Issue4 - References not displayed in Compile macro "Code Search" results: "Code Search" section of Compile macro search results doesn't display the references. However, the references can be found in "Free text search" section.
  • Issue5 - Incorrect interpretation of EXPORT_SYMBOL argument: The argument of EXPORT_SYMBOL calls is incorrectly reported as variable definitions.
  • Issue6 - Links result on "Code search" results but not "Free text search" results: When displayed links are clicked, only "Code Search" results are displayed, but not "Free text Search" results. To get even "Free text results",  search box can be used though.


For those interested in downloading the LXRng engine, here is the reference to my githib repository: https://github.com/babuneelam/lxrng.

Hope this helps.


References:

UA-48797665-1