ThumbStick: 1,a) 2,b) ThumbStick ThumbStick ThumbStick: Two-dimensional Thumb Motion Recognition System for One-handed Smartwatch Input Aoyama Shuhei

Size: px

Start display at page:

Download "ThumbStick: 1,a) 2,b) ThumbStick ThumbStick ThumbStick: Two-dimensional Thumb Motion Recognition System for One-handed Smartwatch Input Aoyama Shuhei"

ただきよほうねん
4 years ago
Views:

1 ThumbStick: 1,a) 2,b) ThumbStick ThumbStick ThumbStick: Two-dimensional Thumb Motion Recognition System for One-handed Smartwatch Input Aoyama Shuhei 1,a) Shizuki Buntarou 2,b) Abstract: In this paper, we show the system ThumbStick, which realizes a two-dimensional input on a smartwatch, with only the thumb of the hand wearing the smartwatch. The system estimates the thumb position using the change of the wrist s contour. Thus, the user can operate the smartwatch with a joystick-like two-dimensional input using the thumb Department of Computer Science, Graduate School of Systems and Information Engineering, University of Tsukuba 2 Faculty of Engineering, Information and Systems, University of Tsukuba a) aoyama@iplab.cs.tsukuba.ac.jp b) shizuki@cs.tsukuba.ac.jp 623

2 1 ThumbStick. Fig. 1 Operation image of a smartwatch with ThumbStick. ThumbStick Thumb- Stick Gong WristWhirl [1] Sun Float [2] Kerber [3] [4] Huang DigitSpace [5] Huang 2.2 / / Zhang Tomo [6] Rekimoto [7]Dementyev [8] Fukui [9] Ortega-Avila [10] Gong [1] McIntosh SensIR [11] McIntosh A 2B 624

2 A B C Fig. 2 Wrist-worn sensor device.

4 Experimental application P29 P0 P1 A B S13 S14 S15 S12 S11 P30 S10 S9 3 Fig.

75 mm natural, Shenzhen Esun Industrial 3D FLASHFORGE Dreamer, Apple Tree 1 mm 2C

3 SoC, Nordic Semiconductor mbed mbed TY51822r3, Switch Science) 2 mm 4.

3 2 A B C Fig. 2 Wrist-worn sensor device. A) Overview, B) the side of sensors array, C) the inside of the device. 4 Fig. 4 Experimental application P29 P0 P1 A B S13 S14 S15 S12 S11 P30 S10 S9 3 Fig. 3 The thumb position to be identified. S8 S7 S6 S5 S4 S3 S2 S1 PVA Filament 1.75 mm natural, Shenzhen Esun Industrial 3D FLASHFORGE Dreamer, Apple Tree 1 mm 2C (CD74HC4067, Texas Instruments) 16 TPR-105F, GENIXTEK 7.68 mm BLE nrf51822 Rev.3 SoC, Nordic Semiconductor mbed mbed TY51822r3, Switch Science) 2 mm P0 P29 P30 P0 P29 P30 5 AB Fig. 5 Experiment environment: A) Posture, B) The position of each sensor in the sensor device. 4.1 Processing ver A 5B BLE USB PC 625

4 Table 1 1 Confusion matrix of classification result of thumb positions. P0 P1 P2 P3 P4 P5 P6 P7 P8 P9 P10 P11 P12 P13 P14 P15 P16 P17 P18 P19 P20 P21 P22 P23 P24 P25 P26 P27 P28 P29 P30 P P P P P P P P P P P P P P P P P P P P P P P P P P P P P P P A 12 P29 P P30 1 P Weka Machine Learning Toolkit[12] 10 fold LIBVSVM[13] % % % 90% 5. ThumbStick ThumbStick ThumbStick 2 626

ThumbStick A B C 5.1 SONY Xperia Z5 E6653; Android 6.0.

2 ThumbStick LIBSVM[13] 6A 6B 0 1 X Y P 29 X = {P rob i cos(rad i )} r ( r < X

5 ThumbStick A B C 5.1 SONY Xperia Z5 E6653; Android SONY SmartWatch 3 SWR50; Android Wear BLE 5.2 ThumbStick LIBSVM[13] 6A 6B 0 1 X Y P 29 X = {P rob i cos(rad i )} r ( r < X < r) i=p 0 P 29 Y = {P rob i sin(rad i )} r ( r < Y < r) i=p 0 P rob i i Rad i i r 6B 5.3 6C 6 ThumbStick A B C) Fig. 6 Application screenshot: A) Smartphone application - before training, B) smartphone application - after training, C) smartwatch application 6. ThumbStick Float[2]

6 [1] Gong, J., Yang, X.-D. and Irani, P.: WristWhirl: Onehanded Continuous Smartwatch Input Using Wrist Gestures, in Proceedings of the 29th Annual Symposium on User Interface Software and Technology, UIST 16, pp , New York, NY, USA (2016), ACM. [2] Sun, K., Wang, Y., Yu, C., Yan, Y., Wen, H. and Shi, Y.: Float: One-Handed and Touch-Free Target Selection on Smartwatches, in Proceedings of the 2017 CHI Conference on Human Factors in Computing Systems, CHI 17, pp , New York, NY, USA (2017), ACM. [3] Kerber, F., Lessel, P. and Krüger, A.: Same-side Hand Interactions with Arm-placed Devices Using EMG, in Proceedings of the 33rd Annual ACM Conference Extended Abstracts on Human Factors in Computing Systems, CHI EA 15, pp , New York, NY, USA (2015), ACM. [4],,, Vol. 40, No. 2, pp (1999). [5] Huang, D.-Y., Chan, L., Yang, S., Wang, F., Liang, R.- H., Yang, D.-N., Hung, Y.-P. and Chen, B.-Y.: DigitSpace: Designing Thumb-to-Fingers Touch Interfaces for One-Handed and Eyes-Free Interactions, in Proceedings of the 2016 CHI Conference on Human Factors in Computing Systems, CHI 16, pp , New York, NY, USA (2016), ACM. [6] Zhang, Y. and Harrison, C.: Tomo: Wearable, Low- Cost Electrical Impedance Tomography for Hand Gesture Recognition, in Proceedings of the 28th Annual ACM Symposium on User Interface SoftwareTechnology, UIST 15, pp , New York, NY, USA (2015), ACM. [7] Rekimoto, J.: GestureWrist and GesturePad: Unobtrusive Wearable Interaction Devices, in Proceedings of the 5th IEEE International Symposium on Wearable Computers, ISWC 01, pp , Washington, DC, USA (2001), IEEE Computer Society. [8] Dementyev, A. and Paradiso, J. A.: WristFlex: Lowpower Gesture Input with Wrist-worn Pressure Sensors, in Proceedings of the 27th Annual ACM Symposium on User Interface Software and Technology, UIST 14, pp , New York, NY, USA (2014), ACM. [9] Fukui, R., Watanabe, M., Shimosaka, M. and Sato, T.: Hand Shape Classification in Various Pronation Angles using a Wearable Wrist Contour Sensor, Advanced Robotics, Vol. 29, No. 1, pp (2015). [10] Ortega-Avila, S., Rakova, B., Sadi, S. and Mistry, P.: Non-invasive Optical Detection of Hand Gestures, in Proceedings of the 6th Augmented Human International Conference, AH 15, pp , New York, NY, USA (2015), ACM. [11] McIntosh, J., Marzo, A. and Fraser, M.: SensIR: Detecting Hand Gestures with a Wearable Bracelet Using Infrared Transmission and Reflection, in Proceedings of the 30th Annual ACM Symposium on User Interface Software and Technology, UIST 17, pp , ACM (2017). [12] Hall, M., Frank, E., Holmes, G., Pfahringer, B., Reutemann, P. and Witten, I. H.: The WEKA Data Mining Software: An Update, Vol. 11, pp , New York, NY, USA (2009), ACM. [13] Chang, C.-C. and Lin, C.-J.: LIBSVM: A Library for Support Vector Machines, Vol. 2, pp. 27:1 27:27, New York, NY, USA (2011), ACM. 628

ipod touch 1 2 Apple ipod touch ipod touch 3 ( ) ipod touch ( 1 ) Apple ( 2 ) Web 1),2) 3. ipod touch 1 2 ipod touch x y z i

ipod touch 1 2 Apple ipod touch ipod touch 3 ( ) ipod touch ( 1 ) Apple ( 2 ) Web 1),2) 3. ipod touch 1 2 ipod touch x y z i ipod touch 1 1 ipod touch. 1) 6 2) 3) A library for detecting movements of an ipod touch by 3D acceleration Akira Kotaki 1 and Mariko Sasakura 1 The aim of this study is to develop a library for detecting