uncompiled.com

You can now use the Linux kernel of your choice when you boot up an Amazon EC2 instance.

We have created a set of AKIs (Amazon Kernel Images) which contain the PV-Grub loader. This loader simply chain-boots the kernel provided in the associated AMI (Amazon Machine Image). Net-net, your instance ends up running the kernel in the AMI instead of the kernel specified in the boot process.

You need to install an “EC2 compatible” kernel and create an initrd (initial RAM disk) as part of your AMI. You also need to create a menu (/boot/grub/menu.lst) for the Grub boot loader. Once you’ve done this you can create the AMI and then launch instances by using one of the PV-Grub “kernels” as described above. You may find this document to be helpful if you want to learn more about the Linux boot process.

To be compatible with EC2, a Linux kernel must support Xen’s pv_ops (paravirtual ops) infrastructure with XSAVE disabled or the Xen 3.0.2 interface. The following kernels have been tested and/or have vendor support:

Fedora 8-12 Xen kernels
SLES/openSUSE 10x, 11.0, and 11.1 Xen kernels
SLES/openSUSE 11.x EC2 Variant
Ubuntu EC2 Variant
RHEL 5.x
CentOS 5.x
Other kernels may not start reliably within EC2. We’re working with the providers of popular AMIs to make sure that they will start to use PV-Grub in the near future.

You can read more about this in our “Enabling User Provided Kernels in Amazon EC2″ document.

– Jeff;

Source

With Wednesday’s beta release of its flagship operating system, Red Hat Enterprise Linux (RHEL), Red Hat has added a number of new capabilities that should help data centers better support virtualization and cloud computing.

RHEL 6.0 will also have at least one less feature as well. This will be the first version of the OS not to include the Xen hypervisor. Instead the company plans to focus its virtualization efforts around the kernel-based Virtual Machine (KVM), said Tim Burke, Red Hat vice president of platform engineering.

For RHEL 6, “Virtualization has been a key focus, as has providing infrastructure that will be part of our cloud services,” Burke said.

To help in cloud deployments, the RHEL 6 OS has the ability to dynamically allocate kernel data structures. “This will allow cloud service providers to give better service-level agreements,” Burke said. As virtual machines are loaded on to the OS, the administrator can specify how much memory, how many processing cycles and how much network bandwidth can be allotted to each machine.

Another new addition is the Completely Fair Scheduler (CFS), which “more dynamically balances the workloads among the tasks,” distributing the CPU resources more evenly across all the applications. Borrowing techniques from Red Hat’s software for running latency-intolerant services, it also does a more sophisticated job of scheduling high-priority processes over low-priority ones, Burke said.

Power savings features have been added. The timing infrastructure has been reorganized as well, and uses something called the tickless kernel enhancement. Previously, the kernel would interrupt the CPU 1,000 times per second to take a time measurement, which prevented the CPU going into power-saving sleep mode. The tickless kernel feature relies instead on hardware-based timers, allowing the CPU to go to sleep in those periods when there are no other chores to complete.

The file systems space has been revamped for larger data sets. This is the first version of RHEL to use ext4 as the default file system. (Formerly it used ext3.) RHEL can now run file systems of up to 16 terabytes. The new file system also runs file system checks much more speedily, which means faster recovery times after unclean shutdowns. For really big data sets, RHEL also includes an option to upgrade to SGI’s XFS file system, which can scale to 128 terabytes.

With Red Hat’s emphasis on supporting cloud computing, the company’s decision to drop Xen may seem surprising. But over the past few years, Red Hat has increasingly thrown its support behind KVM. In 2008, the company purchased virtualization software provider Qumranet, whose developers pioneered much of the early KVM work.

Burke said that one of the reasons Xen was dropped is that the company was duplicating a lot of effort in maintaining two hypervisors, a job that requires an increasing amount of energy. For instance, Intel added some virtualization support capabilities in its just-released Nehalem server processor, but these capabilities required some modifications in both sets of software.

Source

Linux® as a hypervisor includes a number of innovations, and one of the more interesting changes in the 2.6.32 kernel is Kernel Shared Memory (KSM). KSM allows the hypervisor to increase the number of concurrent virtual machines by consolidating identical memory pages. Explore the ideas behind KSM (such as storage de-duplication), its implementation, and how you manage it.

Source

Booting an operating system has always been considered a challenging task. In this document we will take a look at the different aspects of the boot process. Such as the BIOS which is the first code which runs, the boot loaders that can load different operating systems, pass arguments to the kernel, load it from different sources like a hard drive, a flash, and network & finally the kernel itself. Though loading the kernel & setting it up to execute is not all that is to be done, we need to bring the system up with different user specific configurations. We will look at the scripts, which deal with this.

Linux has grown from a system that used to boot from a floppy providing no luxurious features to the user, to the current jazzy Linux systems. It is important to have an insight of the Linux boot procedure. Say for Linux to serve the purpose on embedded systems, the generic boot procedure must almost always be modified to meet the needs of the target application.

Source

Con Kolivas had stopped working on the Linux kernel for two years after he became fed up with the kernel development community, but last year he made a return by introducing the BFS scheduler. The BFS scheduler for the Linux kernel is quite simple in design compared to other schedulers, but it performed fairly well on desktop systems. Due to Con’s past frustrations, he has no intentions of mainlining the Brain Fuck Scheduler, but he has now offered up another batch of patches.

Kolivas has released a new set of patches this morning that are “designed to improve system responsiveness and interactivity with specific emphasis on the desktop.” There are 13 patches he has made available that can be applied against the freshly released Linux 2.6.33 kernel. One of the patches is BFS, another changes the default timer frequency to 1000Hz, another adds new values that allows the timer frequency to be upped to 10,000Hz, and various other changes.

While Con Kolivas is not likely trying to get these patches pushed into the Linux 2.6.34 kernel, he has published this to the Linux kernel mailing list. His patches can be found in the 2.6.33-ck1 directory.

Source

According to a study by researchers at the University of Oviedo (Universidad de Oviedo) in Spain, the estimated total value of the 2.6.30 Linux kernel, released in June of 2009, is more than €1 billion. Using the kernel development history from version 2.6.11 to 2.6.30, the researchers calculated the costs by using the Constructive Cost Model 81 (COCOMO 81) and taking the average annual salary for a developer in 2006 in the European Union as a parameter. According to EUROSTAT, that was approximately €31,000. The Linux Foundation published a similar study in October of 2008.

The COCOMO algorithm calculates the value of software using a number of specific metrics, specifically the number of lines of code written. The study looks at the estimated annual research and development (R&D) costs of the kernel releases and shows that the annual Linux kernel R&D cost increased significantly in 2008. Between 2005 and 2006, annual R&D was estimated at between €72 to €94 million, however, in 2008, that number rose to more than €228 million.

The two researchers responsible for the study, Jesús García-García and Mª Isabel Alonso de Magdaleno, will be be presenting their findings at the Concord 2010 conference on corporate R&D taking place on the 3rd and 4th of March, 2010 in Seville, Spain.

Source

Hi Dave & list:

I’m writing to report a bug in recent vanilla kernels regarding the
ability to execute in non-executable pages on SPARC. I’m no SPARC
expert, but I’ll try to explain as best I can the problem and make
myself available for debugging/testing any fixes.

I have 4 sparc systems currently, a Netra T1, an Ultra 10, a Sunfire
V210, and a Blade 2500. The first two systems run the latest 2.4
kernels and experience no problems. The second two systems run recent
2.6 kernels (2.6.31 and 2.6.32) and both utilize the Cheetah+ MMU. Both
of these systems running the recent 2.6 kernels exhibit the problem.

I’ve provided two simple testcases that illustrate the problem. Either
run them in a loop or just multiple times — eventually instead of
receiving a segfault (as it should every time for attempting to execute
on the stack, which is non-executable by default per ABI) the shellcode
I’ve set up on the stack will execute without problems. In the first
testcase I haven’t set up enough code to perform a return from the
function pointer, so you see the varied signals when an instruction
fetch is attempted on the 2nd instruction (made up of whatever
happened to be located on the stack). In the second case I set up the
proper ret/restore and the program is able to exit cleanly.

I’m willing to do anything to help debug the problem, but I thought it
would
be wiser to report it first in case anyone had any immediate ideas on
what the problem could be. I figured it would also help in being able
to debug the issue more effectively, given its seemingly somewhat-random
nature.

Please keep the PaX team and myself CC’d as we’re not subscribed to the
list.

Thanks for your help,
-Brad

cat /proc/self/maps output (showing the stack non-executable):
00010000-00014000 r-xp 00000000 08:02 1769474 /bin/cat
00024000-00026000 rwxp 00004000 08:02 1769474 /bin/cat
00026000-00048000 rwxp 00000000 00:00 0 [heap]
f7cf0000-f7e2a000 r–p 00000000 08:02 295830 /usr/lib/locale/locale-archive
f7e2c000-f7fa4000 r-xp 00000000 08:02 1277958 /lib/ultra3/libc-2.7.so
f7fa4000-f7fb4000 —p 00178000 08:02 1277958 /lib/ultra3/libc-2.7.so
f7fb4000-f7fba000 rwxp 00178000 08:02 1277958 /lib/ultra3/libc-2.7.so
f7fba000-f7fbc000 rwxp 00000000 00:00 0
f7fbc000-f7fde000 r-xp 00000000 08:02 1270085 /lib/ld-2.7.so
f7fec000-f7ff0000 rwxp 00020000 08:02 1270085 /lib/ld-2.7.so
f7ff0000-f7ff2000 rw-p 00000000 00:00 0
ffdb6000-ffde0000 rw-p 00000000 00:00 0 [stack]

First test case:
#include

typedef int (* _wee)(void);

int main(void)
{
char buf[4] = { ‘\x81′, ‘\xc7′, ‘\xe0′, ‘\x08′};
_wee wee;
printf(“%p\n”, &buf);
wee = (_wee)&buf;
wee();

return 0;
}

gdb output in 90-95% of the cases:
Program received signal SIGSEGV, Segmentation fault.
0xff9d9cb0 in ?? ()

gdb output in the other 5-10% of cases:
(gdb) run
The program being debugged has been started already.
Start it from the beginning? (y or n) y
Starting program: /root/test
0xff811cb0

Program received signal SIGBUS, Bus error.
0xff811cb4 in ?? ()
(gdb) x/x 0xff811cb0
0xff811cb0: 0x81c7e008
(gdb) x/i 0xff811cb0
0xff811cb0: ret
0xff811cb4: lda [ %g4 + %l0 ] (229), %f31

(gdb) run
The program being debugged has been started already.
Start it from the beginning? (y or n) y
Starting program: /root/test
0xff97dcb0

Program received signal SIGILL, Illegal instruction.
0xff97dcb4 in ?? ()
(gdb) x/8x $pc-4
0xff97dcb0: 0x81c7e008 0xff97dcb0 0xf7f3af00
0×00000000
0xff97dcc0: 0×00000000 0×00000000 0xfffffffc
0×00000000
(gdb) x/i $pc-4
0xff97dcb0: ret
0xff97dcb4: ldqa [ %i7 + %l0 ] (229), %f62

modified code so it executed a ret / restore:
#include

typedef int (* _wee)(void);

int main(void)
{
char buf[8] = { ‘\x81′, ‘\xc7′, ‘\xe0′, ‘\x08′, ‘\x81′, ‘\xe8′,
‘\x00′, ‘\x00′ };
_wee wee;
printf(“%p\n”, &buf);
wee = (_wee)&buf;
wee();

return 0;
}

gdb output in the 5-10% case:
Starting program: /root/test
0xffb4fca8

Program exited with code 01.

Source

A report has been received from Juniper at 4:25pm under bulletin PSN-2010-01-623 that a crafted malformed TCP field option in the TCP header of a packet will cause the JUNOS kernel to core (crash). In other words the kernel on the network device (gateway router) will crash and reboot if a packet containing this crafted option is received on a listening TCP port. The JUNOS firewall filter is unable to filter a TCP packet with this issue. Juniper claims this issue as exploit was identified during investigation of a vendor interoperability issue.

Source

If you were sat at your Linux computer one dark evening in late 1999, things would have been considerably different. Your machine would probably be running either Red Hat 6.1 or Mandrake 6. Outside your window, the world was going crazy for all things dotcom. Microsoft was prepping both Windows 2000 and its ill-fated Millennium edition, while Apple had just released OS 9 and its Power Mac G4.

As a Linux user, you’d have been an uber-geek, someone with an obsessive interest in computing and far too much time on your hands. But things have changed. Linux is now an operating system anyone can install and use, and it’s growing stronger every year. Here’s how it happened.

Source

Brad Spengler noted on Twitter today:

Cool, the Chrome OS docs recommend applying grsec: http://bit.ly/26p2ac (has other hardening tips that apply to any Linux system too)

From the document:

Efforts to secure Linux environments tend to revolve around the principle of least privilege and applying exploit mitigation tactics wherever possible. While the exploit mitigation techniques are effective, they are never a perfect defense and often the specific techniques deployed vary from distribution to distribution. In addition, the principle of least privilege is excellent in a server environment and for locking down system services on desktops. However, desktop systems are meant to be general purpose. This makes it incredibly difficult to determine the least privilege needed if a program has not ever been seen on the system before (or was written since the system was installed!). The end result is that the risks from interactively executed applications are addressed only using exploit mitigations and not as comprehensively as desired.

Chromium OS has an advantage. All native programs run by the end user are known in advance since all general purpose applications are web applications. We use this knowledge to apply comprehensive access control enforcement in addition to the well-known exploit mitigation techniques. This combination allows Chromium OS to benefit from the great work securing Linux in both end-user and server enviroments!

Source