HTTP-FUSE-CLOOP Linux http://unit.aist.go.jp/itri/knoppix 1), 1), 1), ), 1) 1), 2) HTTP-FUSE-CLOOP HTTP HTTP-FUSE-CLOOP Internet mount Linux HTTP-FUSE KNOPPIX Network compressed loopback device HTTP-FUSE-CLOOP and Linux which boot form it. Kuniyasu Suzaki 1), Toshiki Yagi 1), Kengo Iijima 1), Kenji Kitagawa 2), Shuichi Tashiro 1) National Institute of Advanced Industrial Science and Technology 1), Alpha Systems Inc. 2) Abstract: We developed network compressed loopback device HTTP-FUSE-COOP which can re-construct block device from the split-compressed block files. The split-compressed block files are transparently treated between local storage and remote network. We developed a Linux HTTP-FUSE KNOPPIX which mounts root file system using HTTP-FUSE-CLOOP via Internet. In this paper we describe the detail of implementation and its performance. 1. Internet OS Internet HTTP Proxy OS Internet iscsi[1] NFS4[2], Open-AFS[3], (MD5) SFS[4], SHFS[5] HTTP-FUSE-CLOOP HTTP Plan9 Venti[6], CFS[7], Bittorrent[8], BTSlave[9]
2 HTTP-FUSE-CLOOP 1CD/1DVD Linux KNOPPIX [10][11] Venti Internet OS HTTP-FUSE KNOPPIX [12] HTTP proxy KNOPPIX4.0 Internet CD DVD 3.8GB HTTP-FUSE KNOPPIX 2.2. CFS: Cooperative File System DVD KNOPPIX DVD CFS[7]MIT chord P2P CFS HTTP-FUSE P2P KNOPPIX iso chord 5MB CD CFS DVD KNOPPIX CFS DVD 2 3 2.3. Bittorrent BTSlave HTTP-FUSE-CLOOP 4 Bittorrent[8] 5 HTTP-FUSE KNOPPIX P2P Bittorrent 6 7 P2P 2. Bittorent 2.1. Plan9 Venti Plan9 archival block storage 1. server Venti[6] archival file server fossil Venti 2. bencode HTTP-FUSE KNOPPIX MD5 Bittorrent
Bittorent BTSlave[9] 3. KNOPPIX cloop KNOPPIX cloop cloop cloop cloop driver KNOPPIX CD-ROM cloop cloop(compressed loopback device) cloop 1) 3.1 cloop cloop ( 64KB) zlib 2 700MB (CD 700MB ) (iso, ext2, etc) CD KNOPPIX cloop CD-ROM cloop /cdrom/knoppx/koppix /dev/cloop ( 256KB) # insmod f /modules/cloop.o file=/cdrom/knoppix/knoppix mount # mount /dev/cloop /KNOPPIX 3.2 cloop cloop 1KNOPPIX (HTTP-FUSE-CLOOP) cloop
3.3. Internet cloop USB (MD5) Internet FUSE wrapper libcurl (3) RAM-Disk FUSE 2 (Filesystem in Userspace)[13] (2) FUSE wrapper FUSE cloop HTTP-FUSE-CLOOP cloop zlib cloop FUSE cloop cloop cloop 3.4. index.idx cloop index.idx FUSE 4 cloop ext2 (2) 2 FUSE-cloop 3 HTTP FUSE-cloop iso9660 CD KNOPPIX ext2 cloop HTTP-FUSE-CLOOP index.idx
3.5. (netselect DLAHEAD) HTTP-FUSE-CLOOP ext2 -> cloop -> FUSE -> (HTTP Internet) -> file cloop Internet NFS 4 FUSE-cloop MD5 HTTP-FUSE-CLOOP index.idx (netselect) (DLAHEAD) 3.5.1 netselect HTTP-FUSE-CLOOP Internet 5 HTTP IP netselect HTTP-FUSE KNOPPIX netselect traceroute CD-ROM KNOPPIX HTTP-FUSE KNOPPIX 5 HTTP-FUSE KNOPPIX 3.5.2 DLAHEAD HTTP-FUSE-CLOOP DLAHEAD(download ahead) DLAHEAD HTTP-FUSE-CLOOP
( DLAHEAD ) cloop 3 3 HTTP-FUSE-CLOOP 4. 4.1. HTTP-FUSE-CLOOP delay HTTP dd 2.51 MBps 200msec delay 0.194 MBps HTTP-FUSE-CLOOP HTTP tar 6.41 MBps 0.516 MBps KNOPPIX4.0 DVD HD dd 15.4 MBps [12] HD tar 20.4 MBps 256KB 1 () cloop 1 256KB 9.56 MBps wget 0.320 MBps cloop 2.71MBps 0.197 MBps wget 3.15GB 35,504 3.09GB Max: 2,622,230 HTTP-FUSE-CLOOP ( Min: 277 9.7GB) Ave: 93,581 cloop 64KB 256KB Window 64KB cloop NIC Window TCP keep-alive tar dd 2.5 HTTP-FUSE-CLOOP CPU 2 2 HTTP-Server: HTTP-FUSE Client: IBM ThinkPAD T24 CPU Pentium III 1GHz, IBM ThinkPAD T42 CPU Pentium M 1.8GHz, 4.2. HTTP-FUSE KNOPPIX 1GB, 100M NIC 2GB, 1000M NIC apache 1.3 24 CD HTTP-FUSE KNOPPIX FreeBSD dummynet ( 200msec 100msec ) dd tar 3,107
234MB DLAHEAD (DL-FULL) T-block(23sec) T-KDE DLAHEAD (97sec) DL-2500(98sec) 512MB 512MB 2500 (DL-2500) T-KDE (113sec) 186MB DLAHEAD latency DVD DLAHEAD latency 200msec DLAHEAD DVD DLAHEAD 4.2.1 (1,413sec) boot: (437sec 644sec) (T-root) 4.2.2 (T-blcok) KDE (T-KDE) tcpdump 5 ( ) latency KDE (T-KDE) (T-root) (T-block) DL FULL 23 97 DL 2500 61 98 DL 37 76 113 latency 6,7latency 200msec 8,9 4.2.2.1 6 DLAHEAD latency DL FULL 402 437 10-20 KNOPPIX Autoconfig 200msec DL 2500 607 644 DL 1376 1413 DLAHEAD DVD ( ) 22 273 295 T-root CD DVD DL-FULL HTTP-FUSE KNOPPIX DVD T-root DL-2500 HTTP-FUFE T-KDE T-root T-block DLAHEAD latency DL-FULL HTTP-FUSE-CLOOP
6 laency 8 laency 200msec 7 latency 9 latency 200msec 7 DLAHEAD (95Mbps) 3.2.2.1 200msec 10 4 8 6 DLAHEAD DLAHEAD HTTP-FUSE-CLOOP 234MB DLAHEAD 250MB 1 DLAHREAD HTTP-FUSE-CLOOP 3 (2.71MBps21.7Mbps) HTTP-FUSE-CLOOP wget libcurl
DLAHEAD 200msec delay (Window 64KB 2.5Mbps) ICP(Internet Cache Protocol) [19] DLAHEAD 2.5Mbps DLAHEAD Akamai FreeFlow 10Mbps DLAHEAD WAFS(Wide Area File System) WAFS Internet LAN LAN latency SMB/CIFS Internet WAFS 5. HTTP WAFS HTTP-FUSE KNOPPIX KNOPPIX4.0 DVD HTTP HTTP-FUSE-CLOOP ring coral P2P 6. ring[14] 6.1 20 HTTP-FUSE KNOPPIX WebDAV netselect coral [15,16].nyud.net:8090 URL P2P www.aist.go.jp.nyud.net:8090/index.html www.aist.go.jp P2P.nyud.net:8090 coral PlanetLab[17,18] P2P coral P2P 6.2. (MD5)
http://lc.linux.or.jp/paper/lc2005/cp-02.pdf [13] FUSE, http://fuse.sourceforge.net/ [14] Ring, http://www.ring.or.jp [15] coral, http://www.coralcdn.org/ index.idx [16] M.J.Freedman, E.Freudenthal, and D.Mazières, https Democratizing Content Publication with Coral, 1st USENIX/ACM Symposium on Networked Systems Design and Implementation (NSDI '04) http://www.coralcdn.org/docs/coral-nsdi04.pdf [17] PlanetLab,http://www.planet-lab.org/ [18]L.Peterson and T.Roscoe The Design Principles of PlanetLab, Draft Paper, June 2004 7. http://www.planet-lab.org/pdn/pdn-04-021/pdn-04-021.pdf [19],,,, Peer-to-Peer Network Web Cache,, DPS (2002 HTTP-FUSE-CLOOP HTTP-FUSE- CLOOP HTTP KNOPPIX HTTP-FUSE KNOPPIX Proxy OS HTTP-FUSE KONPPIX &URL [1] iscsi, http://www.ietf.org/rfc/rfc3720.txt [2] NFS4 http://www.nfsv4.org/ [3] Open-AFS, http://www.openafs.org/ [4] SFS, http://www.fs.net/sfswww [5] SHFS, http://shfs.sourceforge.net/ [6] Quinlan, S. and Dorward, D.: Venti: a new approach to archival storage, the USENIX Conference on File and Storage Technologies, Monterey, CA, pp. 89 102 (2002). [7] Dabek, F. Kaashoek, M. Karger, D. Morris R. Stoica,I.: Wide-area cooperative storage with CFS, 18th ACM Symposium on Operating Systems Principles (SOSP '01), (2001). [8] Cohen, B.: Incentives Build Robustness in BitTorrent, First Workshop on Economics of Pexr-to-Peer Systems,(2003). [9] Cox. B: BTSlave: http://btslave.sf.net/ [10] KNOPPIX, http://www.knopper.net/knoppix [11] KNOPPIX, http://unit.aist.go.jp/itri/knoppix/ [12],,,, HTTP-FUSE KNOPPIX,,Linux Conference 2005.