• The Linux Kernel Archives

  Welcome to the Linux Kernel Archives. This is the primary site for the Linux kernel source, but it has much more than just kernels.

• Patchdot.org: Tracking Linux Kernel Patches
• Kernel Traffic

  This site publishes newsletters that track the technical developments of various projects of the Free and Open Source world. "Kernel Traffic" is the name of the overall collection of newsletters, and also the name of the original newsletter by Zack Brown that got the project started.

• Rik van Riel's Linux kernel patches
• The kernel patch homepage of Con Kolivas

• Fabulous Fonts in Linux
  • Microsoft's TrueType core fonts on linux
  • The FREETYPE Project

• Gentoo Linux
• Mandrake Linux

• User-mode linux kernel
• Linux From Scratch
• ConTest Home Page
• The Linux Game Tome

Disk schedulers in current operating systems are generally work-conserving, i.e., they schedule a request as soon as the previous request has finished. Such schedulers often require multiple outstanding requests from each process to meet system-level goals of performance and quality of service. Unfortunately, many common applications issue disk read requests in a synchronous manner, interspersing successive requests with short periods of computation. The scheduler chooses the next request too early; this induces deceptive idleness, a condition where the scheduler incorrectly assumes that the last request issuing process has no further requests, and becomes forced to switch to a request from another process.

We propose the anticipatory disk scheduling framework to solve this problem in a simple, general and transparent way, based on the non-work-conserving scheduling discipline. Our FreeBSD implementation is observed to yield large benefits on a range of microbenchmarks and real workloads. The Apache webserver delivers between 29% and 71% more throughput on a disk-intensive workload. The Andrew filesystem benchmark runs faster by 8%, due to a speedup of 54% in its read-intensive phase. Variants of the TPC-B database benchmark exhibit improvements between 2% and 60%. Proportional-share schedulers are seen to achieve their contracts accurately and efficiently.

• Richard Gooch - Linux Contributions
• Linux Devfs (Device File System) FAQ

Devfs is an alternative to "real" character and block special devices on your root filesystem. Kernel device drivers can register devices by name rather than major and minor numbers. These devices will appear in devfs automatically, with whatever default ownership and protection the driver specified. A daemon (devfsd) can be used to override these defaults. Devfs has been in the kernel since 2.3.46.

Here is some available projects I have at the moment. The list was larger but I cut it drastically as some of the projects were outdated and some merely embarassing at this stage.

• Mel: Linux VM Project

  Documentation. The first two documents are the major documentation effort. The first is a guide to the VM which is pretty comprehensive but not completed yet. Watch out for updates. The second is a code commentary to give a guided tour. The rest of the docs are quiet small, generally badly written documents which describe how different parts of the VM work but may be a lot more inaccurate than the two first documents.

• Understanding The Linux Virtual Memory Manager

  The development of Linux is unusual in that it was built more with an emphasis on the practical rather than a theoretical one. While many of the algorithms used in the Virtual Memory (VM) system were designed by theorists, the implementations have diverged from the theory considerably. Instead of following the traditional development cycle of design to implementation, changes are made in reaction to how the system behaved in the ``real world'' and intuitive decisions by developers.

• Code Commentary On The Linux Virtual Memory Manager

Ever wish you had a place to let your Linux applications play -- where they wouldn't hurt anything else? Do your killer apps spend too much time killing each other? Originally conceived as a kernel developer's tool, UML lets you set up multiple virtual machines that are isolated from each other and from the hardware. Now, you can test applications all the way to failure without breaking the host system -- or even requiring a reboot. Veteran administrator Carla Schroder shows you how.

The story of how BitKeeper has come to be Linus Torvalds' (and many other kernel hackers) tool of choice in maintaining the Linux development tree is worthy of a book. Here's the Cliff Note's version of McVoy's contribution to Linux kernel development, BitKeeper, and countless hours of flaming on the role of open source and proprietary software. (3,200 words)

Welcome to The Memory Management Reference! This is a resource for programmers and computer scientists interested in memory management and garbage collection. For an introduction, see the Beginner's Guide.

• The Memory Management Reference: Beginner's Guide: Allocation

  Memory allocation is the process of assigning blocks of memory on request. Typically the allocator receives memory from the operating system in a small number of large blocks that it must divide up to satisfy the requests for smaller blocks. It must also make any returned blocks available for reuse. There are many common ways to perform this, with different strengths and weaknesses.

has documents on the linux kernel's vm

This project aims to document the linux kernel at the source code level. More like kernel commentry along with an overview and detailed explanations. This project has been actually started as a means to understand how the kernel works.

Cscope is a developer's tool for browsing source code. It has an impeccable Unix pedigree, having been originally developed at Bell Labs back in the days of the PDP-11. Cscope was part of the official AT&T Unix distribution for many years, and has been used to manage projects involving 20 million lines of code!

Cbrowser is a graphical C/C++ source code searching and browsing tool, and a hierarchical function call viewer

OProfile is a system-wide profiler for Linux x86 systems, capable of profiling all running code at low overhead. OProfile is released under the GNU GPL.

syscalltrack allows you - the 'root' user - to track invocations of system calls across your Linux system. You specify rules that specify which system call invocations will be tracked, and what to do when a rule matches a system call invocation. You can log the system call invocation, fail it (i.e. force it to return some error code), or suspend the process executing it (e.g. so you could attach a debugger to the process at that point, or so you could find who is the parent process of that process, etc.). You could even kill the process, if you were feeling particularly sadistic.

The Linux operating system is a multiprogramming, multiuser system. This means that it is able to handle multiple running programs at once. On a uniprocessor system (a computer with only one microprocessor), this is achieved by sharing this precious resource among multiple tasks, letting each execute for a certain period of time and then switching to another. The selection and switching of processes is handled by the Linux kernel, which also is a program and therefore also needs some time from the processor. It is also responsible for fulfilling certain requests by the programs it manages, dealing with error conditions, etc.

To drive Single System Image scalability issues in Linux both from a large systems perspective (large processor counts, IO configs, etc) as well a from an intensive enterprise workload perspective (large number of processes, memory requirements, bandwidth requirements).

Treeps is designed to make viewing and interacting with the running programs on your system easy and intuative. When started treeps displays a process tree displaying your foreground and background processes. These processes are arranged in a tree diagram showing the invocation hierarchy, i.e. who started who. The init process, and on some systems sched, are always displayed to show the common ancestry and provide a root for the tree.

This is a detailed guide to kernel configuration, compilation, upgrades, and troubleshooting for ix86-based systems.

During my past few articles, I have reviewed volume management and file system concepts and issues. If you have not read those articles, it might be best to review them because a number of background concepts were covered that provide understanding of how everything works internally.

The RPM Package Manager (RPM) is a powerful command line driven package management system capable of installing, uninstalling, verifying, querying, and updating computer software packages. Each software package consists of an archive of files along with information about the package like its version, a description, and the like. There is also a related API ("Application Program Interface"), permitting advanced developers to bypass 'shelling out' to a command line, and to manage such transactions from within a native coding language.

Following the recent release of an anticipatory IO scheduler [story], Andrea Arcangeli started a lengthy thread in which he proposed an SFQ (Stochastic Fair Queuing) disk scheduler. The idea was picked up by Jens Axboe who had evidently worked on a similar idea earlier. Jens quickly posted two different disk schedulers utilizing "fair queuing" algorithms, more commonly used in handling network traffic. When someone suggested he was reinventing the wheel, Jens replied, "There's no wheel reinventing here, just applying the goodies from network scheduling to disk scheduling."

Jens' first disk scheduler utilizes SFQ, or Stochastic Fair Queuing. Fair queuing would allow many processes demanding large levels of disk IO to each get fair access to the device, preventing any one process from denying the others. SFQ is one of the simpler and less accurate fair queuing algorithms that works well on average, its primary benefit being that it requires very minimal overhead. Essentially, SFQ works by dividing IO requests among a large number of queues using a frequently changing hash algorithm, then serving the requests round robin. The term 'stochastic' is used as each process does not get its own queue, leaving some of the derived benefit to random chance. In other words, the queues are stored in a hash table, and it is possible for processes requesting IO to hash to the same bucket, or collide, thus sharing a queue.

The second disk scheduler Jens posted utilizes CFQ, or as he defines it Complete Fair Queuing. Jens' prefers his CFQ implemenation, which builds upon his earlier SFQ patch. This scheduler creates additional queues as needed, avoiding the random nature of the stochastic scheduler by entirely preventing collisions. Recent benchmarking suggests that this new scheduler still needs a fair amount of tuning.