Researchers Create Hypervisor-Based Tool For Blocking Rootkits

[Howell, Paul]

At
http://www.darkreading.com/vulnerability_management/security/attacks/sho
wArticle.jhtml?articleID=221600127

Researchers Create Hypervisor-Based Tool For Blocking Rootkits
New technology 'patches' the operating system kernel, protects it from
rootkits

Nov 03, 2009 | 03:19 PM

By Kelly Jackson Higgins
DarkReading
Researchers at North Carolina State University and Microsoft Research
have come up with a way to combat rootkits by using the machine's own
hardware-based memory protection: the so-called HookSafe tool basically
protects the operating system kernel from rootkits.

Rootkits are the most difficult of malware to detect and remove: they
often evade detection by anti-malware software, and even if they are
discovered, they can still be difficult to completely eradicate. A
rootkit typically hijacks "hooks" in the operating system -- basically
the control data in the kernel used to augment or extend the features of
an OS -- in order to hide out in the OS. This in turn lets the rootkit
intercept and manipulate the system's data, remain invisible to the user
and anti-malware tools, and to install other malware aimed at stealing
data from the system.

"Then the rootkit can hijack and manipulate the results seen by the user
applications ... only allowing a user to see what it wants them to see,"
says Xuxian Jiang, assistant professor of computer science at NC State
and a member of the research team.

"The best way to [defend against rootkits] is to prevent them in the
first place," he says. "It's a mess trying to clean them up."

The researchers have devised a way to move the potentially tens of
thousands of hooks in the kernel to a centralized location so they're
easier to monitor and more difficult to abuse. Their HookSafe prototype
is a hypervisor-based system that is able to protect nearly 6,000
different kernel hooks and has successfully stopped nine different
rootkits.

HookSafe runs in Ubuntu Linux 8.04 and leverages hardware-based memory
protection in the system to stop rootkits from hijacking kernel hooks.
"[It] includes a patch to the OS kernel to relocate the kernel hooks,"
Jiang says. "It also includes an extension to commodity hypervisors
[such as Xen] to enforce the hook protection with the hardware-based
memory protection."

The main tradeoff of the tool thus far is a slight performance hit,
about a 6 percent slowdown in system performance.

Jiang says the researchers designed the hypervisor-based hook to enforce
hook usage because the OS kernel is vulnerable and could already be
corrupted by a rootkit and thus not reliable for monitoring the hooks
itself.

Greg Hoglund, CEO and founder of HBGary and a rootkit expert, says the
new research addresses one of the main areas of rootkit infection, but
is no silver bullet.

"This is a subset of the problem. They are protecting the kernel, but
not preventing the rootkits from operating," Hoglund says. "Right now we
have rootkits that will bypass this technology: there are simply too
many places where execution control can be gained" by rootkits, he says.

But NC State's Jiang says HookSafe is for both preventing rootkits
altogether as well as preventing them from using hooks: "The reason is
that if a hook cannot be hijacked by rootkits, the rootkit will not be
able to hide its presence in the system," he says. "And the very hiding
capability is the defining characteristic of a rootkit."

With the help of Microsoft Research, the research team also has a
version of HookSafe under development for the Windows research kernel,
which can be found here.

Jiang and his colleagues will present their paper, titled "Countering
Kernel Rootkits with Lightweight Hook Protection" (PDF) on November 12
at the 16th ACM Conference on Computer and Communications Security in
Chicago.

"The exciting part of this research is that it effectively blocks one of
most commonly used attack vectors by rootkits -- through kernel hooks.
And the blocking can be done efficiently, thanks to the hardware-based
memory protection," Jiang says.

They have proposed several techniques for protecting the OS kernel
overall, including previous research on rootkit profiling and kernel
code integrity. Jiang says the team is also looking how an OS kernel can
be redesigned to make kernel rootkits more difficult to deploy in the
first place.


------------------------------------------------------------------------
To unsubscribe from netsec, send mail to majordomo@merit.edu
with a body consisting of the words "unsubscribe netsec" --
without the quotes. For more help, send a message to majordomo@merit.edu
with the word "help" as the body.
------------------------------------------------------------------------