1 D-Visor OS

Transcription

1 DEOS-FY2013-VS-02J 2013 D-Visor86 + D-System Monitor DEOS

2 1 D-Visor OS initrd initrd BusyBox Ubuntu initrd initrd initrd OS OS OS (/dev/sda) OS (/dev/sdb) OS OS (/dev/sda) OS (/dev/sdb) Grub rootkit D-System Monitor D-System Monitor VIRTIO OS dsysmon

3 VIRTIO FoxyKBD RootkitLibra LMS VIRTIO virtio_console virtio_net host-to-host bridge virtio_blk FoxyKBD RootkitLibra LMS GUI dsm-gui.py dsm-gui.py dsm-gui.py FoxyKBD RootkitLibra LMS rootkit file_rootkit.ko process_rootkit.ko ttyrpld rpld_receiver A B FoxyKBD RootkitLibra LMS

4 1 D-Visor86 D-Visor86 Veidt VMM (VMM) Hyper Threading Technology i386 Linux D-Visor86 1 1: D-Visor86 (Grub) VMM VMM initrd VMM VMM sysfs VMM D-Visor86 veidt veidt - 4 -

5 2 2.1 D-Visor86 BX-100n-DC5000-C01 1 1: CONTEC BX-100n-DC5000-C01 CPU Intel Atom Processor N GHz Intel 945GSE + ICH7M-DH BIOS Award PC DDR2 SDRAM 1GB Intel 945GSE AC SATA HDD 2 CF Type 1 2 IDE IF RS-232C 5ch 9 D-sub 1000BASE-T 2ch (Intel 82573L) 2.2 D-Visor86 Ubuntu OS Linux initrd BusyBox Linux BusyBox

6 3 D-Visor86 Ubuntu i386 D-Visor OS Ubuntu bit Ubuntu URL Ubuntu patch zlib1g-dev 3.2 D-Visor86 dvisor86-<date>.tar.gz dvisor tar.gz 2 $HOME Linux linux tar.gz Linux 1) $HOME Linux linux $ tar xvzf linux tar.gz 2) Linux linux linux dv86 $ mv linux linux dv86 3) Linux D-Visor86 $ make linux-apply-patch - 6 -

7 2: D-Visor86 Makefile mk/ files/ files/initrd/ tools/ host-tools/ rootkits/ vmm/ configs/ linux native-config linux watcher-config linux user-config busybox config patches/ linux dv86-patches ubuntu init-getty.patch ubuntu nfs-kernel-server.patch Makefile Makefile initrd D-System Monitor (dsysmon) D-System Monitor rootkit D-Visor86 VMM Linux BusyBox D-Visor86 initrd busybox Linux OS OS ttys1, hvc0 getty OS NFS 3.3 D-Visor86 $HOME make all $ cd $HOME $ make all VMM 3 Linux make 1) $HOME/vmm/lib VMM veidtdefs.h 2) $HOME/vmm/kernel VMM 3) $HOME/vmm VMM VMM VMM 4) $HOME/build/linux-watcher 5) $HOME/build/linux-user 6) $HOME/build/linux-native native - 7 -

8 7) $HOME/tools D-System Monitor (dsysmon) 8) $HOME/rootkits D-System Monitor (dsysmon) rootkit 9) $HOME/build/initrd initrd initrd make all 3 3: D-Visor86 vmm/dv86-vmm build/linux-user/vmlinux.bin build/linux-user/arch/x86/boot/bzimage build/linux-watcher/vmlinux.bin build/linux-watcher/arch/x86/boot/bzimage build/linux-native/vmlinux.bin build/linux-native/arch/x86/boot/bzimage tools/dsysmon/dsysmon tools/dsm-gui/dsm-gui.py tools/rtkl-collect/rtkl-collect tools/dxfeed/dxfeed tools/gp_fault/gp_fault tools/profile/profile rootkits/file-rootkit/file_rootkit.ko rootkits/process-rootkit/process_rootkit.ko rootkits/rpld-receiver/rpld_receiver rootkits/ttyrpld-2.6.0/k_linux2.6/rpldev.ko rootkits/ttyrpld-2.6.0/user/rpld build/initrd/user-kernel.map build/initrd/dv86-scripts/ * build/initrd-dv86.img VMM bzimage bzimage native native bzimage D-System Monitor D-System Monitor GUI RootkitLibra OS DXFEED GP VMM rootkit rootkit TTY TCP rpld TTY rootkit D-Visor86 BusyBox D-System Monitor initrd D-Visor86 GNU make parallel-make make -j n n - 8 -

9 parallel-make 3.4 D-Visor86 Linux initrd initrd OS initrd D-Visor86 initrd $HOME/build/initrd-dv86.img initrd D-Visor86 $HOME/build/initrd-dv86.img 3 1) BusyBox 2) $HOME/files/initrd 3) $HOME/build/initrd cpio gzip initrd 1 4 initrd-dv86.img busybox initrd-dv86.img $HOME/files/initrd $HOME/build/initrd-dv86.cpio make initrd $ cd $HOME $ cp < > files/initrd/ $ rm build/initrd-dv86.cpio $ make initrd BusyBox D-Visor86 initrd BusyBox BusyBox $HOME/files/busybox-installed.img initrd cpio + gzip BusyBox $HOME/configs/busybox config BusyBox BusyBox BusyBox $HOME/files/busybox-installed.img 1) Ubuntu # apt-get build-dep busybox # apt-get install libncurses5-dev 1 D-Visor86 initrd cpio + gzip initramfs initrd - 9 -

10 4: initrd-dv86.img /init /user-kernel.map /etc/inittab /etc/fstab /etc/{passwd,group} /etc/init.d/rcs /etc/rcs.d/ /bin/busybox /dv86-scripts/start_virtio /dv86-scripts/setup_bridge /dsm-tools/dsysmon /dsm-tools/dxfeed /dsm-tools/rtkl-collect /dsm-tools/gp_fault /dsm-tools/profile /dsm-tools/demo-a.sh /dsm-tools/demo-b-dsm.sh /dsm-tools/demo-b-gui.sh /dsm-tools/dsm-gui.py /rootkits/dsm-rtkt.sh /rootkits/file_rootkit.ko /rootkits/process_rootkit.ko /rootkits/rpld /rootkits/rpld.conf /rootkits/rpld_receiver /rootkits/rpldev.ko /bin/init busybox VIRTIO VIRTIO D-System Monitor DXFEED RootkitLibra GP VMM A dsysmon B dsysmon B dsm-gui.py D-System Monitor GUI rootkit rootkit rootkit TTY rpld TTY TCP rpld 2) $HOME BusyBox $ cd $HOME $ wget $ tar xvzf busybox tar.bz2 $ cd busybox ) $ cp $HOME/configs/busybos config.config $ make oldconfig $ make 4) BusyBox _install $ make install

11 5) $HOME/files/busybox-installed.img $ cd _install $ find. cpio --quiet -H newc -o gzip -c > $HOME/files/busybox-installed.img Ubuntu initrd OS OS initrd initrd OS initrd initrd Ubuntu initrd scsi_wait_scan.ko 1) $HOME 2) 2 $ make -C linux dv86 O=$HOME/build/linux-user modules 3) root # make -C linux dv86 O=$HOME/build/linux-user modules_install 4) $ make -C linux dv86 O=$HOME/build/linux-watcher modules 5) root # make -C linux dv86 O=$HOME/build/linux-watcher modules_install /lib/modules/ dv86-user /lib/modules/ dv86-watcher initrd Ubuntu mkinitramfs Ubuntu initrd $HOME $ /usr/sbin/mkinitramfs -v -o./ubuntu-initrd-dv86-user.img dv86-user $ /usr/sbin/mkinitramfs -v -o./ubuntu-initrd-dv86-watcher.img dv86-watcher ubuntu-initrd-dv86-user.img ubuntu-initrd-dv86-watcher.img Ubuntu initrd 2 O=

12 3.5 D-Visor86 VMM initrd initrd /boot 5 5: /boot dv86-vmm vmm/dv86-vmm D-Visor86 VMM initrd-dv86.img build/initrd-dv86.img initrd vmlinux.bin-dv86-watcher build/linux-watcher/vmlinux.bin vmlinux.bin-dv86-user build/linux-user/vmlinux.bin vmlinux.bin-dv86-native build/linux-native/vmlinux.bin native vmlinuz-dv86-watcher build/linux-watcher/arch/x86/boot/bzimage (bzimage) vmlinuz-dv86-user build/linux-user/arch/x86/boot/bzimage (bzimage) vmlinuz-dv86-native build/linux-native/arch/x86/boot/bzimage native (bzimage) ubuntu-initrd-dv86-user.img OS Ubuntu initrd ubuntu-initrd-dv86-watcher.img OS Ubuntu initrd menu.lst.bx100n-sample files/menu.lst.bx100n-sample Grub 5 OS OS 1) $HOME 2) make runtime initrd $HOME/runtime $ make runtime 3) OS OS initrd $HOME/runtime/boot $ cp $HOME/ubuntu-initrd-dv86-user.img runtime/boot $ cp $HOME/ubuntu-initrd-dv86-watcher.img runtime/boot 4) OS OS $HOME/runtime $ cp $HOME/patches/ubuntu init-getty.patch runtime/ $ cp $HOME/patches/ubuntu nfs-kernel-server.patch runtime/ 5) $HOME/runtime $ tar cvzf dv86-kernels.tar.gz -C runtime

13 ) $HOME 2) /lib/modules/ dv86-user /lib/modules/ dv86-watcher $ tar cvzf dv86-user.tar.gz -C /lib/modules dv86-user $ tar cvzf dv86-watcher.tar.gz -C /lib/modules dv86-watcher

14 4 D-Visor BX-100n-DC5000-C01 OS OS 1) BIOS Optimized Default HDD Master USB 3 2) HDD Ubuntu Master (sda) Slave (sdb) 3) COM1 VMM VMM COM1 4) COM2 OS OS bps 4 5) OS 6) OS A/B rootfs /etc/ fstab rootfs swap 5 7) OS OS VGA/GUI getty 8) B OS GUI OS swap 9) B OS nfsroot (eth0) OS 4.1 OS Ubuntu bit HDD OS OS OS OS OS Master HDD (/dev/sda) OS Slave HDD (/dev/sdb) OS Slave HDD (/dev/sdb) OS /dev/sda /dev/sdb OS Master HDD (/dev/sda) 3 IDE AHCI AHCI 4 OS COM2 5 rootfs

15 BX-100n-DC5000-C01 USB CD-ROM 4.2 OS OS OS # apt-get update OS (/dev/sda) Network Manager # apt-get purge network-manager ureadahead # apt-get purge ureadahead grub-pc (GRUB2) /boot/grub grub-pc # mv /boot/grub /boot/grub2 # apt-get purge grub-pc grub (GRUB Legacy) update-grub Y grub-install OS /dev/sda # mkdir /boot/grub # apt-get install grub # update-grub # grub-install /dev/sda patch # apt-get install patch libhx22 TTY rpld # apt-get install libhx22 libhx-dev OS (/dev/sdb) Network Manager # apt-get purge network-manager ureadahead # apt-get purge ureadahead grub-pc (GRUB2) /boot/grub grub-pc # mv /boot/grub /boot/grub2 # apt-get purge grub-pc

16 grub (GRUB Legacy) update-grub Y grub-install OS /dev/sdb # mkdir /boot/grub # apt-get install grub # update-grub # grub-install /dev/sdb patch # apt-get install patch nfs-kernel-server OS NFS # apt-get install nfs-kernel-server python-wxgtk2.8 python-matplotlib D-System Monitor GUI dsm-gui.py # apt-get install python-wxgtk2.8 python-matplotlib bridge-utils virtio_net bridge brctl bridge # apt-get install brige-utils uml-utilities virtio_net host-to-host tun/tap tunctl tun/tap # apt-get install uml-utilities Grub /dev/sdb Ubuntu /dev/sdb Ubuntu Grub /dev/sda Ubuntu /dev/sdb 1) Grub OS 2) e OS e 3) uuid... d 4) kernel... e 5) root=uuid=... root=/dev/sdb1 6) 7) b

17 4.3 OS OS OS 3.5 dv86-kernels.tar.gz OS OS 1) dv86-kernels.tar.gz # tar xvzf dv86-kernels.tar.gz ~/ 2) 6 6: dv86-kernels.tar.gz ubuntu init-getty.patch ubuntu nfs-kernel-server.patch boot/ dsm-tools/ rootkits/ ttys1, hvc0 getty NFS VMM/Kernel D-System Monitor rootkit OS (/dev/sda) OS Network Manager /etc/ network/interfaces /etc/network/interfaces lo auto OS /etc/fstab rootfs swap OS rootfs HDD NFS (virtio_net) getty ttys1 hvc0 (virtio_console) getty /etc/init ttys1.conf hvc0.conf 4.3 ubuntu init-getty.patch # cd /etc/init # patch -p2 < ubuntu init-getty.patch

18 4.3.2 OS (/dev/sdb) OS Network Manager /etc/ network/interfaces /etc/network/interfaces lo auto getty ttys1 hvc0 (virtio_console) getty /etc/init ttys1.conf hvc0.conf 4.3 ubuntu init-getty.patch # cd /etc/init # patch -p2 < ubuntu init-getty.patch NFS Ubuntu nfs-kernel-server /etc/init.d/nfskernel-server NFS 4.3 ubuntu nfs-kernel-server.patch # cd /etc/init.d # patch -p2 < ubuntu nfs-kernel-server.patch NFS export B OS rootfs NFS OS rootfs /dev/sda1 /export/user-root /etc/exports Grub menu.lst menu.lst # mkdir -p /export/user-root /etc/exports /export/user-root /24(rw,sync,no_root_squash,no_subtree_check) boot OS OS /boot 1) OS OS ~/boot /boot # cp ~/boot/* /boot dv86-user.tar.gz dv86-watcher.tar.gz OS OS /lib/modules 1) OS dv86-user.tar.gz

19 # tar xvzf dv86-user.tar.gz -C /lib/modules 2) OS dv86-watcher.tar.gz # tar xvzf dv86-watcher.tar.gz -C /lib/modules dv86-kernels.tar.gz, dv86-user.tar.gz dv86-watcher.tar.gz Grub D-Visor86 VMM Linux initrd Grub Grub kernel VMM module initrd /boot/grub/menu.lst D-Visor86 VMM 01: title D-Visor86 (Without VMM, User PV-Kernel sda1) 02: root (hd0,0) 03: kernel /boot/vmlinuz.dv86-user root=/dev/sda1 ro 04: initrd /boot/ubuntu-dv86-initrd.img 05: 06: title D-Visor86 (COM1->VMM COM2->Watcher VGA->User sda1) 07: root (hd0,0) 08: kernel /boot/dv86-vmm loglevel=info console=serial,0, \ 09: cpu0_lower_mem=0x1000,16k cpu0_upper_mem=0x ,384m \ 10: cpu0_use=com2 \ 11: cpu1_lower_mem=0x11000,256k cpu1_upper_mem=0x1b000000,512m \ 12: cpu1_use=vga,kbd,pci,ata cpu1_no_auto_start 13: module /boot/vmlinux.bin-dv86-watcher console=ttys1, \ 14: earlyprintk=ttys1, : module /boot/initrd-dv86.img 16: module /boot/vmlinux.bin-dv86-user root=/dev/sda1 ro 17: module /boot/ubuntu-initrd-dv86-user.img hiddenmenu (timeout) /etc/fstab 1 4 VMM initrd initrd VMM initrd 3 16 root= 8 12 kernel VMM console= loglevel= serial,< >,[< >] < > 0 < > info error detail1 detail2 detail3 6 D-Visor

20 cpu#_uses= cpu#_os= # CPU 0 1 CPU OS com1 com2 com3 com4 vga kdb pci ata, # CPU 0 1 CPU OS linux cpu#_load_addr= # CPU 0 1 CPU OS cpu#_lower_mem= # CPU 0 1 CPU OS < >,< > cpu#_upper_mem= # CPU 0 1 CPU OS < >,< > < > CONFIG_PHYSICAL_START 0x x1b cpu#_no_auto_start # CPU 0 1 CPU OS cpu1_no_auto_start 3.5 dv86-kernels.tar.gz menu.lst.bx100n-sample menu.lst menu.lst # cp /boot/menu.lst.bx100n-sample /boot/grub/menu.lst menu.lst.bx100n-sample root= UUID UUID blkid # blkid /dev/sda1: UUID="cd9ec974-7b e-9bfc8e53f9e5" TYPE="ext4" /dev/sda5: UUID="1add48bb-eab5-4f8b a76b96742b" TYPE="swap" /dev/sdb1: UUID="27222c67-b401-49c9-a6d5-63ec0b30deb2" TYPE="ext4" /dev/sdb5: UUID="e0857da8-155d-41fc-a517-a36cd " TYPE="swap" BIOS HDD Primary OS menu.lst OS /boot VMM, kernel, initrd BIOS HDD Secondary OS

21 4.6 OS OS rootfs 4.3 dsm-tools rootkits OS OS ~/dsm-tools / # cp -r ~/dsm-tools / ~/rootkits / # cp -r ~/rootkits / 4.7 rootkit OS rootkit /rootkits/dsm-rtkt.sh /etc/rc3.d/s95dsm-rtkt # cd /etc/rc3.d # ln -s../../rootkits/dsm-rtkt.sh./s95-dsm-rtkt rootkit S95-dsm-rtkt /etc/rc3.d Linux runlevel 3 Grub menu.lst run-level 2 rootkit D-System Monitor run-level 3 rootkit dsm-rtkt.sh eth1 IP ttyrpld TTY /var/log/rpl/rpld.log rootkit /rootkits 4 rpldev.ko file_rootkit.ko process_rootkit.ko rpld rootkit ttyrpld rpld rpld PID /var/run/rpld.pid 4.8 1) Grub 4.5 D-Visor86 (COM1->VMM COM2->Watcher VGA->User sda1) D-Visor86 VMM

22 2) 0 (COM1) D-Visor86 VMM [init] starting vmm. [init] vmm using 0xfc xfd7fffff [cpu0] [Startup memory map] [cpu0] guest 0x x00004fff -> 0x A,D,RW [cpu0] guest 0x x02ffffff -> 0x3f A,D,RW [cpu0] guest 0x x24ffffff -> 0x A,D,RW [cpu0] guest 0x xfbffffff -> 0x3f A,D,RW [cpu0] vmmkernel 0xfc xfc142fff -> 0x G,A,D,RW [cpu0] percpu 0xfc xfc55bfff -> 0x0170d000- G,A,D,RW [cpu0] iomem(apic) 0xfd55c xfd55cfff -> 0xfee00000 G,A,D,PCD,RW [cpu0] Linux kernel loaded in 0x x033c9cde [cpu0] Linux initrd loaded in 0x03bbe x03fff3ff [cpu0] Command line is console=ttys1, earlyprintk=ttys1, \ nosep idle=halt reservetop= [cpu0] starting guest. 3) 1 (COM2) BusyBox [ ] Initializing cgroup subsys cpu [ ] Linux version dv86-watcher(tom@endgame.soum.co.jp) \ (gcc version (Ubuntu ubuntu5) ) #3 Mon Aug 22 05:52:25 JST 2011 [ ] KERNEL supported cpus: [ ] Intel GenuineIntel Mounting filesystems. Hello, this is Watcher OS. Please press Enter to activate this console. ~ # /dsm-tools/dsysmon -s 4) 1 (COM2) dsysmon -s CPU1 0 (COM1) VMM VGA [cpu1] [Startup memory map] [cpu1] guest 0x x00000fff -> 0x A,D,RW [cpu1] guest 0x x00010fff -> 0x3f A,D,RW [cpu1] guest 0x x00050fff -> 0x A,D,RW [cpu1] guest 0x x0009efff -> 0x3f A,D,RW [cpu1] guest 0x0009f x000c7fff -> 0x0009f000- A,D,RW [cpu1] guest 0x000c x000effff -> 0x3f A,D,RW [cpu1] guest 0x000f x000fffff -> 0x000f0000- A,D,RW [cpu1] guest 0x x03ffffff -> 0x3f A,D,RW [cpu1] guest 0x x24ffffff -> 0x A,D,RW [cpu1] guest 0x x3f6dffff -> 0x3f A,D,RW [cpu1] guest 0x3f6e xfbffffff -> 0x3f6e0000- A,D,RW [cpu1] vmmkernel 0xfc xfc142fff -> 0x G,A,D,RW [cpu1] percpu 0xfc xfc55bfff -> 0x01b G,A,D,RW [cpu1] iomem(apic) 0xfd55c xfd55cfff -> 0xfee00000 G,A,D,PCD,RW [cpu1] Linux kernel loaded in 0x x a [cpu1] Linux initrd loaded in 0x23b x23fff1ff [cpu1] Command line is root=/dev/sda1 ro nosep idle=halt reservetop= [cpu1] starting guest

23 5 D-System Monitor D-System Monitor (DSM) OS D-Visor86 VMM D-Visor86 D-System Monitor D-System Monitor OS dsysmon D-System Monitor dsysmon 5.1 D-System Monitor OS OS OS ( ) D-System Monitor D-Visor86 VMM OS (dde) OS dsysmon VIRTIO D-System Monitor OS Linux VIRTIO VIRTIO OS OS OS VIRTIO OS dsysmon dsysmon D-System Monitor VIRTIO VIRTIO virtio_console virtio_net virtio_bkl (HDD) VIRTIO dsysmon OS D-System Monitor OS OS OS OS

24 OS D-Visor86 VMM VMM VMM OS dsysmon OS VMM D-System Monitor OS DXFEED VMM OS VMM DXFEED OS dsysmon D-System Monitor FoxyKBD, RootkitLibra, Lightweight Monitoring Service (LMS) 3 FoxyKBD FoxyKBD OS D- System Monitor OS Linux TTY RootkitLibra RootkitLibra OS RootkitLibra OS NFS NFS OS Lightweight Monitoring Service (LMS) Lightweight Monitoring Service LMS OS LMS OS dsysmon 5.2 dsysmon D-System Monitor dsysmon VIRTIO dsysmon VIRTIO dsysmon

25 5.2.1 verbose ( v) verbose dsysmon verbose v debug ( d) debug dsysmon dsysmon debug d coop ( c) (cooperation) dsysmon /tmp/dsysmon<pid> FIFO FIFO dsysmon GUI coop c start ( s) VIRTIO OS start s help dsysmon VIRTIO console virtio_console dsysmon OS /dev/hvc0 tunnet=<ipaddr>[:<macaddr>] virtio_net host-to-host OS /dev/tun OS TAP tapn OS ethn OS TAP <ipaddr> IP <macaddr> MAC tunnet=bridge:<bridgename>[:<macaddr>] virtio_net bridge OS /dev/tun OS <bridgename> OS ethn OS <macaddr> MAC

26 block=<devname> virtio_blk <devname> OS /dev/vda0 <devname> /dev/sda FoxyKBD fkbd FoxyKBD FoxyKBD fkbd injection=<interval>[:<start-delay>:<runtime>] virtio_console <interval>, <start-delay>, <runtime> <start-delay>, <runtime> injection console VIRTIO netlog<index>[=<filename>] virtio_net <index> OS <filename> <filename> FIFO <index> 0 netlog tunnet VIRTIO hddlog<index>[=<filename>] virtio_blk <index> OS <filename> <filename> FIFO <index> 0 hddlog block VIRTIO RootkitLibra rtkl RootkitLibra RootkitLibra rtkl RootkitLibra rtkl interval OS RootkitLibra tunnet VIRTIO OS NFS

27 rtkl-interval=<interval> OS LMS lms LMS LMS lms LMS lms interval OS lms-interval=<interval> OS lms-loose=<count> OS 5 5 init-task=<addr> OS init_task init task dsysmon OS dsysmon init task per-cpu-rq=<addr> OS per_cpu runqueues per cpu rq dsysmon OS dsysmon per cpu rq 5.3 VIRTIO D-System Monitor VIRTIO virtio_console virtio_console dsysmon console dsysmon OS OS /dev/hvc0 OS virtio_console OS dsysmon # dsysmon -s --console

28 OS virtio_console OS # /sbin/getty hvc0 OS dsysmon OS TTY Ubuntu LTS dv86-1.soum.co.jp hvc0 dv86-1.soum.co.jp login: tom Password: Last login: Fri Aug 19 19:31:13 JST 2011 on tty1 Linux dv86-1.soum.co.jp dv86-user #3 Mon Aug 22 05:52:41 JST 2011 \ i686 GNU/Linux Ubuntu LTS Welcome to Ubuntu! * Documentation: Warning: no access to tty (Inappropriate ioctl for device). Thus no job control in this shell. % virtio_net virtio_net dsysmon tunnet OS ethn host-to-host OS ethn OS tapn bridge OS brn OS ethn OS host-to-host host-to-host OS OS ethn OS tapn OS tapn dsysmon tunnet IP OS ethn IP OS virtio_net host-to-host OS dsysmon # dsysmon -s --tunnet= OS /dev/tun dsysmon tun0 tun0 IP OS ethn 2 eth2 7 7 OS 2 D-Visor86 VMM

29 OS eth2 IP OS tun0 # ifconfig eth2 inet up # ping PING ( ): 56(84) bytes of data. 64 bytes from : icmp_seq=0 ttl=64 time=5.196ms 64 bytes from : icmp_seq=1 ttl=64 time=0.556ms 64 bytes from : icmp_seq=2 ttl=64 time=0.528ms bridge bridge dsysmon OS brctl OS D-Visor86 VMM OS # brctl addbr br0 # brctl addif br0 eth0 # ifconfig br0 up # ifconfig eth up [ ] device eth0 entered promiscuous mode [ ] e1000e: eth0 NIC Link is Up 1000 Mbps Full Duplex, Flow Control: RX/TX [ ] br0: port 1(eth0) entering learning state [ ] br0: port 1(eth0) entering forwarding state OS br0 eth0 IP initrd /dv86-scrips/setup_bridge OS virtio_net bridge OS dsysmon # dsysmon -s --tunnet=bridge:br0 [ ] device tap0 entered promiscuous mode [ ] br0: port 2(tap0) entering learning state [ ] br0: port 2(tap0) entering forwarding state OS ethn eth0 OS eth0 IP OS br0 eth0 # ifconfig eth0 inet up # ping PING ( ): 56(84) bytes of data. 64 bytes from : icmp_seq=0 ttl=64 time=2.87ms 64 bytes from : icmp_seq=1 ttl=64 time=0.710ms 64 bytes from : icmp_seq=2 ttl=64 time=0.682ms virtio_blk virtio_blk dsysmon block block

30 OS /dev/vdan N OS virtio_block OS dsysmon # dsysmon -s --block=/dev/sda OS /dev/sda /dev/sda /dev/sda1, /dev/sda2, /dev/sda5 /dev/vda1, /dev/vda2, /dev/vda5 OS Ubuntu (/dev/sda) OS virtio_blk OS root=/dev/vda1 OS [ ] Kernel command line: root=/dev/vda1 ro pci=veidt_busset=1 \ nosep idle=halt reservetop= [ ] virtio: device 0 virtqueue 0 at 0x IRQ 7 [ ] vda: vda1 vda2 < vda5 > [ ] EXT4-fs (vda1): mounted filesystem with ordered data mode 5.4 FoxyKBD FoxyKBD OS FoxyKBD FoxyKBD FoxyKBD injection : ms :5000:10000 netlog hddlog <timestamp> <length> <timestamp> dsysmon <length> FoxyKBD dsysmon OS virtio_console virtio_net bridge

31 virtio_blk /dev/sda ms 10 net.log hdd.log FoxyKBD # brctl addbr br0 # brctl addif br0 eth0 # ifconfig br0 up # ifconfig eth up # dsysmon -s --console --tunnet=bridge:br0 --block=/dev/sda \ --injection=5000:100: netlog=net.log --hddlog=hdd.log 5.5 RootkitLibra RootkitLibra OS NFS NFS RootkitLibra virtio_net DXFEED rtkl-collect OS rtkl-collect D-Visor86 $HOME/tools/rtkl-collect D-Visor86 initrd /dsm-tools/rtkl-collect rtkl-collect OS OS RootkitLibra rtkl interval [<timestamp>] RTKL: file size mismatch, i-node I (nfs:n!= sys:s ) <timestamp> dsysmon I i-node N NFS trusted view S untrusted view [<timestamp] RTKL: number of directory entry mismatch i-node I (nfs:n!= sys:s ) <tmiestamp> dsysmon I i-node N NFS trusted view S untrusted view OS virtio_net bridge RootkitLibra 5 (5000 ms) OS dsysmon # brctl addbr br0 # brctl addif br0 eth0 # ifconfig br0 up # ifconfig eth up # dsysmon -s --tunnet=bridge:br0 --rtkl --rtkl-interval=

32 OS NFS rtklcollect OS # ifconfig eth0 inet up # mount :/export /mnt # rtkl-collect -r /mnt 5.6 LMS LMS D-System Monitor OS LMS OS dsysmon VIRTIO LMS lms interval OS [<timestamp>] LMS: found hidden process, PID P [<proc-name>] <timestamp> dsysmon P ID <proc-name> PIDP OS LMS 5 (5000 ms) OS dsysmon # dsysmon -s --lms --lms-interval=

33 6 GUI dsm-gui.py D-System Monitor dsysmon 3 GUI dsm-gui.py dsm-gui.py Python Python dsm-gui.py dsysmon FIFO FoxyKBD FoxyKBD FoxyKBD RootkitLibra rootkit RootkitLibra rootkit RootkitLibra LMS rootkit LMS LMS dsm-gui.py GUI dsm-gui.py dsm-gui.py d <coop-dir> dsysmon FIFO g <graph-style> FoxyKBD bar plot i <interval> l <device> r <device> FoxyKBD 2 FoxyKBD netlog<index> virtio_net hddlog<index> virtio_blk <index> <index> 0 l netlog FoxyKBD netlog<index> virtio_net hddlog<index> virtio_blk <index> <index>

34 2: dsm-gui.py 0 r hddlog s <grid> FoxyKBD 90 v h dsm-gui.py 6.2 dsm-gui.py dsm-gui.py 4 1) 2) FoxyKBD 3) RootkitLibra 4) LMS

35 6.2.1 dsm-gui.py File GUI Alt+F Quit GUI Ctrl+Q View GUI Alt+V Fullscreen GUI F11 Dsysmon D-System Monitor Alt+D Start User OS OS OS OS Ctrl+S Clear RTKL warning RootkitLibra Rootkit Warnings Clear LMS warning LMS Rootkit Warnings Clear ALL warning RootkitLibra LMS Rootkit Warnings Clear ALL graphs FoxyKBD FoxyKBD FoxyKBD dsm-gui.py dsm-gui.py l netlog dsm-gui.py r hddlog 3 1 FoxyKBD

36 Start Stop Change FoxyKBD dsm-gui.py FoxyKBD RootkitLibra RootkitLibra 1 3 Rootkit Warnings RootkitLibra 3 RootkitLibra Start Stop Clear RootkitLibra dsm-gui.py RootkitLibra RootkitLibra LMS Lightweight Monitoring Service Rootkit Warnings LMS 3 1 LMS Start Stop Change LMS dsm-gui.py LMS

37 7 rootkit D-System Monitor OS 3 OS rootkit 7.1 file_rootkit.ko file_rootkit.ko file_rootkit.ko i-node file_rootkit.ko i-node open(), close(), oldfstat(), fstat(), fstat64(), oldlstat(), lstat(), lstat64(), getdents(), getdents64() file_rootkit.ko # insmod file_rootkit.ko /proc/fhide i-node $ echo <i-node number> > /proc/fhide /proc/fhide i-node $ echo <i-node number> > /proc/fhide /proc/fmodify i-node inode 500 $ echo <i-node number> +500 > /proc/fmodify inode 500 $ echo <i-node number> -500 > /proc/fmodify inode 500 $ echo <i-node number> 500 > /proc/fmodify i-node /proc/fmodify $ echo <i-node number> > /proc/fmodify /proc/fhide /proc/fmodify cat i-node

38 7.2 process_rootkit.ko process_rootkit.ko process_rootkit.ko PID process_rootkit.ko /proc PID /proc PID task_struct PID process_rootkit.ko # insmod process_rootkit.ko /proc/phide PID $ echo <PID> > /proc/phide process_rootkit.ko task_struct 7.3 ttyrpld ttyrpld URL tty logging daemon TTY ttyrpld rpld TTY # rpld rpld rpld.conf rpld.conf OFMT=<PATH> TTY rpld_receiver NET_ADDR=<IP_ADDR> TTY NET_PORT=<PORT> TTY NET_RETRY=<1 0> TTY TTY

39 7.3.1 rpld_receiver rpld_receiver rpld TTY TCP $ rpld_receiver rpld_receiver -o < > TTY -p < > rpld_receiver TTY -v -h

40 8 1) VMM OS 2) OS dsysmon 3) OS 4) OS OS 8.1 A: OS OS VGA/GUI OS initrd (LMS) B: OS OS OS VGA/GUI FoxyKBD RootkitLibra LMS OS VIRTIO 7 8 7: A OS VIRTIO A: CPU#0 CPU#1 DV86-User DV86-Watcher HDD (sda1) initrd VGA/GUI ttys1 (COM2) VIRTIO 8: B OS VIRTIO B: CPU#0 CPU#1 DV86-User DV86-Watcher NFS (virtio_net) HDD (sdb1) hvc0 (virtio_console) VGA/GUI VIRTIO NET0/NET1/CONSOLE (backend)

41 8.2 A 1) Grub D-System Monitor DEMO scenario A [W:initrd / U:sda1+GUI] OS ttys1 (COM2) OS initrd COM2 BusyBox 2) LMS dsysmon OS # dsysmon --start --lms 3) VGA OS GUI rootkit 4) OS LMS OS rpld (COM2) [ ] LMS: found hidden process, PID 1515 [rpld] dsysmon DEMO-A.sh A # /dsm-tools/demo-a.sh LMS rpld rpld $ echo 1000 > /proc/rpl_loop /proc/rpl_loop rpld = 1, 000, B 1) Grub D-System Monitor DEMO scenario B [W:sdb1+GUI / U:NFS+hvc0] OS VGA GUI 2) OS rootfs /export/user-root # mount /dev/sda1 /export/user-root 3) OS dsysmon # /dsm-tools/dsysmon --coop --console \ --tunnet= tunnet= \ --fkbd --netlog0 --netlog1 --rtkl --lms

42 virtio-net IP 9 9: B IP OS OS virtio-net-0 tap eth virtio-net-1 tap eth nfsroot virtio-net-0 4) OS ttyrpld OS rpld_receiver # /rootkits/rpld_receiver [-o <log-file>] -o 5) GUI dsysmon FIFO # ls /tmp/dsysmon* dsysmon1855 -d GUI # /dsm-tools/dsm-gui.py -l netlog0 -r netlog1 \ -d /tmp/dsysmon1855 virtio-net-0 virtio-net-1 6) GUI Dsysmon Start USER OS OS dsysmon OS 7) GUI GUI DEMO-B-dsm.sh DEMO-B-gui.sh DEMO-B-dsm.sh 2) 3) DEMO-B-gui.sh 4) 5) B DEMO-B-dsm.sh OS # /dsm-tools/demo-b-dsm.sh DEMO-B-gui.sh GUI # /dsm-tools/demo-b-gui.sh

43 8.3.1 FoxyKBD FoxyKBD virtio-net-0 nfsroot virtio-net-1 rpld Start Stop 300ms (ms) Change 8 virtio-net-1 rpld nfsroot virtionet-0 OS RootkitLibra RootkitLibra NFS NFS B OS rtkl-collect rtkl-collect -r <dir> <dir> B OS /rootkit/rpldev.ko /rootkit/file_rootkit.ko /rootkit/process_rootkit.ko /rootkit/rpld rtkl-collect OS # /dsm-tools/rtkl-collect -r /rootkits Start Stop 5 [ ] RTKL: file size mismatch, i-node XXX (nfs:xxx!= sys:xxx) [ ] RTKL: number of directory entry mismatch, i-node XXX (nfs:xxx!= sys:xxx) 8 BX-100n-DC5000-C01 300ms dsm-gui.py Stop # echo stop > /tmp/dsysmon<pid>/fkbd-ctl dsysmon FIFO 500ms dsm-gui.py

44 [ ] dsysmon Clear RootkitLibra RootkitLibra RootkitLibra NFS READDIR READDIRPLUS rtkl-collect READDIR READDIRPLUS OS RootkitLibra NFS OS READDIR READDIRPLUS NFS OS READDIR READDIRPLUS # echo 3 > /proc/sys/vm/drop_caches LMS Start Stop LMS OS 1 (ms) Change [ ] LMS: found hidden process, PID 1515 [rpld] [ ] dsysmon LMS B rpld OS LMS OS LMS 1) /proc/rpl_loop rpld OS rpld OS OS /proc/rpl_loop 9 9 BX-100n-DC5000-C01 OS /proc/rpl_loop

45 2) LMS OS OS OS LMS lms-loose=<count> dsysmon <count>