1 NUI TA OpenGL Wii Wii OpenGL Kinect URL USB

Size: px

Start display at page:

Download "1 NUI TA OpenGL Wii Wii OpenGL Kinect URL USB"

れいがおえづか
4 years ago
Views:

1 1 NUI TA OpenGL Wii Wii OpenGL Kinect URL USB

2 TA mireport@cyber.t.u-tokyo.ac.jp MI-abcdefgh abcdefgh (11/17 23:59) 1.3 Wii Bluetooth Kinect TA USB Wii Wii Wii Linux C Wii Wiimote) CWiid CWiid 00:00:00:00:00:00 Wii Bluetooth $ wmgui 00:00:00:00:00:00 wmgui File Connect ( 1) 2 Wii 1 2 LED OK Connect Settings Acc Data Wii 2.3 CWiid Wii (Wiimote) ( 4

3 2.3 CWiid 3 1 Connect 2 Connect 1. Ubuntu 2. libcwiid 3. libcwiid-dev libbluetooth-dev $ sudo apt-get install libbluetooth-dev $ sudo apt-get install libcwiid-dev

4 Ubuntu CWiid 3 Wii Wii Bluetooth

3.1 Wii 5 Wii 3.1 Wii Wii Wii Wii Wii Fit Wii Wii 3 IC Bluetooth IC CMOS Bluetooth 3 5 Wii http://www.nintendo.co.jp/wii/features/wii remote.html 3.

5 3.1 Wii 5 Wii 3.1 Wii Wii Wii Wii Wii Fit Wii Wii 3 IC Bluetooth IC CMOS Bluetooth 3 5 Wii remote.html 3.2 CWiid CWiid Linux C Wii Wiimote) API Wii API Wiimote GTK GUI CWiid CMOS Wii Wii cwiid open() bdaddr bluetooth BDADDR ANY (1 )Wii Wii cwiid wiimote handle

6 6 3 cwiid_wiimote_t *cwiid_open(bdaddr_t *bdaddr, int flags); Wii cwiid set mesg callback() typedef void cwiid_mesg_callback_t(cwiid_wiimote_t *, int, union cwiid_mesg []); int cwiid_set_mesg_callback(cwiid_wiimote_t *wiimote, cwiid_mesg_callback_t *callback); Wii if ((wiimote = cwiid_open(&bdaddr, CWIID_FLAG_MESG_IFC)) == NULL) { printf("unable to connect\n"); else { if (cwiid_set_mesg_callback(wiimote, &wiimote_callback)) { printf("unable to set message callback\n");... cwiid close() cwiid wiimote handle int cwiid_close(cwiid_wiimote_t *wiimote); if(cwiid_close(wiimote)){ fprintf(stderr,"error on wiimote disconnect\n"); exit(-1); Wii Wii union cwiid mesg mesg array[] mesg array.type acc zero, acc one zero point 1G point

7 3.2 CWiid 7 void wiimote_callback(cwiid_wiimote_t *wiimote, int mesg_count, union cwiid_mesg mesg_array[], struct timespec *timestamp) { int i; for (i=0; i < mesg_count; i++) { switch (mesg_array[i].type) { case CWIID_MESG_ACC: wiimote_acc(&mesg_array[i].acc_mesg); break; case CWIID_MESG_BTN: wiimote_btn(&mesg_array[i].btn_mesg); break; default: break; Wii cwiid command() state 1 ON 0 OFF command CWIID CMD RUMBLE CWIID CMD LED LED cwiid_command(wiimote, command, int state); LED ON/OFF //vibration ON cwiid_command(wiimote, CWIID_CMD_RUMBLE, 1); //vibration OFF cwiid_command(wiimote, CWIID_CMD_RUMBLE, 0); //LED 1 ON cwiid_command(wiimote, CWIID_CMD_LED, 1); //LED 1 OFF cwiid_command(wiimote, CWIID_CMD_LED, 0); Wii

8 8 3 void set_report_mode(void) { uint8_t rpt_mode; rpt_mode = CWIID_RPT_STATUS CWIID_RPT_BTN; rpt_mode = CWIID_RPT_ACC; cwiid_command(wiimote, CWIID_CMD_RPT_MODE, rpt_mode); Wii mesg- acc[cwiid X], mesg- acc[cwiid X], mesg- acc[cwiid X] void wiimote_acc(struct cwiid_acc_mesg *mesg) { printf("x: %d, y: %d, z: %d\t", mesg->x, mesg->y, mesg->z); double a_x, a_y, a_z; a_x = ((double)mesg->acc[cwiid_x] - acc_zero.x) / (acc_one.x - acc_zero.x); a_y = ((double)mesg->acc[cwiid_y] - acc_zero.y) / (acc_one.y - acc_zero.y); a_z = ((double)mesg->acc[cwiid_z] - acc_zero.z) / (acc_one.z - acc_zero.z); Wii void wiimote_btn(struct cwiid_btn_mesg *mesg){ if(mesg->buttons & CWIID_BTN_UP){...

3.2 CWiid 9 CWIID BTN UP CWIID BTN DOWN CWIID BTN RIGHT

CWIID BTN B CWIID BTN HOME CWIID BTN 1 CWIID BTN 2 A B HOME

9 3.2 CWiid 9 CWIID BTN UP CWIID BTN DOWN CWIID BTN RIGHT CWIID BTN LEFT CWIID BTN PLUS CWIID BTN MINUS CWIID BTN A CWIID BTN B CWIID BTN HOME CWIID BTN 1 CWIID BTN 2 A B HOME 1 2 Wii Wii 6 Wii axis2.png Yaw, Pitch, Roll Yaw, Pitch, Roll 7 7 Yaw,Pitch,Roll

10 10 3 CWiid CWiid Makefile CC = g++ COMMON =. RM = rm -f TARGET = sample OBJS = sample.o CFLAGS = -Wall -fpermissive LDFLAGS = LDLIBS = -lglut -lgl -lglu -lcwiid -lrt -lbluetooth -lpthread.c.o: ${CC -c ${CFLAGS $< ${TARGET: ${OBJS ${CC -o ${TARGET ${OBJS ${LDLIBS ${LDFLAGS clean: ${RM ${TARGET *.o *~

11 3.2 CWiid kadai1-1 sample.c Bluetooth 00:00:00:00:00:00 Bluetooth Wii./sample 00:00:00:00:00:00 a Wii A B b 1! 2 Wii LED 1 2 r R] Wii c WireCube 2. kadai1-2 sample.c Wii Teapot a b B c 3. kadai1-3 sample.c Wii. disp flag a b Wii c d

12 Wii 9 Teapot Wii 10 Wii

13 4 Wii OpenGL Wii Wii 11 12 Teapot 2 1.

13 13 4 Wii OpenGL Wii Wii Teapot 2 1. kadai2 cvglview wii.c Bluetooth 00:00:00:00:00:00 Bluetooth Wii./cvglview_wii 00:00:00:00:00:00 2. Wii Wii glulookat 11 CG 12 Teapot 3. Wii Teapot

14 14 4 Wii Wii 2 Wii AR 3 1. kadai3 cvglarview.c Bluetooth 00:00:00:00:00:00 Bluetooth Wii./cvglarview 00:00:00:00:00:00 2. Wii glulookat teapot 4. Wii Teapot

15 15 5 Natural User Interface NUI(Natural User Interface) NUI CUI(Character User Interface) PC GUI(Graphical User Interface) ( ) NUI 5.1 Kinect Wii Kinect Kinect 2010 Microsoft Xbox 360 Kinect Wii Kinect RGB Kinect PC OpenNI Kinect Windows SDK Kinect Kinect Xtion PRO Xtion PRO LIVE 5.2 OpenNI OpenNI (Open Natural Interaction) Kinect PrimeSence OpenNI Kinect OpenNI GitHub OpenNI RGB Kinect RGB Kinect.oni GNU Lesser General Public License (LGPL) Kinect wiki ( ) Kinect wiki Windows Mac OS X

での利用方法については Ubuntu で OpenNI と Kinect ドライバと NITE

16 5 三次元コンピュータグラフィックスと Natural User Interface 16 図 15 手をつかったジェスチャの認識図 16 スケルトンの認識ル方法が紹介されている Linux (Ubuntu) での利用方法については Ubuntu で OpenNI と Kinect ドライバと NITE のダウンロードとビルドとインストール ( ) 等に解説がある

17 5.3 OpenNI OpenNI ROS OpenNI Kinect ROS OpenNI ROS localhost turtlebot roslaunch openni launch export ROS_HOSTNAME=localhost export ROS_MASTER_URI= roslaunch openni_launch openni.launch roslaunch kinect 01-depth-test main.cpp Kinect Kinect 500mm 500mm DepthGenerator //DepthGenerator DepthGenerator depth; nretval = context.findexistingnode(xn_node_type_depth, depth); // DepthMetaData depthmd; depth.getmetadata(depthmd); // printf("frame %d Middle point is: %u. FPS: %f\n", depthmd.frameid(), depthmd(depthmd.xres() / 2, depthmd.yres() / 2), xnfpscalc(&xnfps)); DepthGenerator depth.getmetadata(depthmd) depthmd depthmd(x, y) x, y depthmd.xres() depthmd.yres() 02-simple-image simple-image.cpp Kinect OpenNI ROS OpenNI simple-image.cpp ROS MODE 1 ROS ROS MODE 0 OpenNI

18 18 5 Natural User Interface //ImageGenerator rc = g_context.findexistingnode(xn_node_type_image, g_image); // rc = g_context.waitanyupdateall(); // g_image.getmetadata(g_imagemd); // const XnUInt8* pimagerow = g_imagemd.data(); ImageGenerator image.getmetadata(imagemd) imagemd imagemd.data() 03-simple-depth simple-depth.cpp Kinect Kinect 17 simple-depth.cpp

19 5.3 OpenNI 19 // unsigned int nnumberofpoints = 0; for (XnUInt y = 0; y < g_depthmd.yres(); ++y) { for (XnUInt x = 0; x < g_depthmd.xres(); ++x, ++pdepth) { if (*pdepth!= 0) { g_pdepthhist[*pdepth]++; nnumberofpoints++; for (int nindex=1; nindex<g_nzres; ++nindex) { g_pdepthhist[nindex] += g_pdepthhist[nindex - 1]; if (nnumberofpoints) { for (int nindex=1; nindex<g_nzres; ++nindex) { g_pdepthhist[nindex] = (unsigned int)(256 * (1.0f - (g_pdepthhist[nindex] / nnumberofpoints))); depth-test main.cpp simple-image simple-image.cpp simple-depth simple-depth.cpp 4. main.cpp simple-image.cpp

20 20 5 Natural User Interface 5.4 OpenNI Kinect NUI Kinect Kinect 04-user user.cpp // nretval = g_context.findexistingnode(xn_node_type_user, g_usergenerator); // void XN_CALLBACK_TYPE User_NewUser(xn::UserGenerator& generator, XnUserID nid, void* pcookie) { printf("new user: %d\n", nid); // void XN_CALLBACK_TYPE User_LostUser(xn::UserGenerator& generator, XnUserID nid, void* pcookie) { printf("lost user: %d\n", nid); // XnCallbackHandle husercallbacks; nretval = g_usergenerator.registerusercallbacks(user_newuser, User_LostUser, NULL, husercallbacks); UserGenerator.GetUserPixels(0, scenemd) UserGenerator.GetUsers(aUsers, nusers) i

21 5.4 OpenNI 21 UserGenerator.GetCoM(aUsers[i], com) // g_usergenerator.getuserpixels(0, scenemd); //plabels 0 if (g_bdrawbackground *plabels!= 0) { nvalue = *pdepth; XnLabel label = *plabels; XnUInt32 ncolorid = label % ncolors; if (label == 0) { ncolorid = ncolors; if (nvalue!= 0) { nhistvalue = pdepthhist[nvalue]; pdestimage[0] = nhistvalue * Colors[nColorID][0]; pdestimage[1] = nhistvalue * Colors[nColorID][1]; pdestimage[2] = nhistvalue * Colors[nColorID][2]; // g_usergenerator.getusers(ausers, nusers); // XnPoint3D com; g_usergenerator.getcom(ausers[i], com); // glrasterpos2i(com.x, com.y); glprintstring(glut_bitmap_helvetica_18, strlabel); 05-skelton main.cpp SceneDrawer.cpp

22 22 5 Natural User Interface 18 user.cpp // void XN_CALLBACK_TYPE UserPose_PoseDetected (xn::posedetectioncapability& /*capability*/, const XnChar* strpose, XnUserID nid, void* /*pcookie*/) // void XN_CALLBACK_TYPE UserCalibration_CalibrationStart (xn::skeletoncapability& /*capability*/, XnUserID nid, void* /*pcookie*/) // void XN_CALLBACK_TYPE UserCalibration_CalibrationComplete (xn::skeletoncapability& /*capability*/, XnUserID nid, XnCalibrationStatus estatus, void* /*pcookie*/) UserGenerator.GetSkeletonCap().IsTracking(aUsers[i]) UserGenerator.GetSkeletonCap().GetSkeletonJointPosition(player, ejoint, joint) ejoint XN SKEL HEAD = 1

$5.4 OpenNI 23 // if (g_bdrawskeleton && g_usergenerator.getskeletoncap().istracking(ausers[i])) { //player ejoint joint XnSkeletonJointPosition joint; g_usergenerator.getskeletoncap().getskeletonjointposition (player, ejoint, joint); //joint if (joint.$

23 5.4 OpenNI 23 // if (g_bdrawskeleton && g_usergenerator.getskeletoncap().istracking(ausers[i])) { //player ejoint joint XnSkeletonJointPosition joint; g_usergenerator.getskeletoncap().getskeletonjointposition (player, ejoint, joint); //joint if (joint.fconfidence < 0.5) { return; //joint XnPoint3D pt; pt = joint.position; g_depthgenerator.convertrealworldtoprojective(1, &pt, &pt); drawcircle(pt.x, pt.y, 2); skelton

24 5 Natural User Interface 5 1. 04-user user.cpp 2. 05-skelton glutkeyboard 3.

24 24 5 Natural User Interface user user.cpp skelton glutkeyboard head-teapot teapot head-teapot teapot 6 1. Wii Kinect Kinect CG CG head-teapot teapot

25 kadai2.png kadai2.jpg kadai5.png kadai5.jpg Wii , Vol.110, No.1069, pp

26 26 6

TA TA TA abcdefgh abcdefgh C PC Wii bluetooth 2.2 Bluetooth USB Princeton PTM-UBT3S 1 1

TA TA TA abcdefgh abcdefgh C PC Wii bluetooth 2.2 Bluetooth USB Princeton PTM-UBT3S 1 1 1 22 (2) TA: 2010 12 13 1 OpenGL Wii Wii OpenGL USB (3DCG) 1.1 http://www.cyber.t.u-tokyo.ac.jp/~kuni/enshu2010/ URL USB 2 2 1.2 TA 16 15 TA TA mireport@cyber.t.u-tokyo.ac.jp 20101213 abcdefgh abcdefgh