34 (2017 ) Advances in machine learning technologies make inductive programming a reality. As opposed to the conventional (deductive) programming, the development process for inductive programming is such a way that the requirements are translated into a training data set and the implementation is (semi-) automatically done by a machine learning algorithm. However, currently machine learning-based systems are developed on mostly trial-and-error basis and no common methodology is established. This paper discusses how systems with machine learning capability should be developed and operated and proposes a new discipline, machine learning engineering, to organize a body of knowledge. 1 ( F = 1.8 C + 32 ) ( ) ( ) 2 ( 1) Towards Machine Learning Engineering This work is a translated and extended version of the paper presented at The First International Workshop on Sharing and Reuse of AI Work Products [16]. Copyrights belong to the Author. Hiroshi Maruyama, Preferred Networks, Preferred Networks, Inc. 1
[3] ( ) 2016 [16] 2 2. 1 2 ( ) 2 2 ETL(Extract-Transfer-Load) 4 1. 2. ( ) 3. 4. training set validation set Training set validation set GPGPU(General-Purpose Graphic Processing Unit) 2 training set
( ) validation set ( 3) 3 [7] 2 2. 2 1. 2. 3. 2. 3 IT IT 4 ( ) KPI(Key Performance Indicator) IT PoC (Proof of Concept) PoC ( ) KPI
4 ( 100%) KPI (concept drift ) IT IBM Cognitive Value Assessment 1 3 ([12] ) SWEBOK [1] 2 3 2 IT 3. 1 2 ( 2 http://www.mckinsey.com/business-functions/digital- 1 https://www.ibm.com/blogs/watson/2016/12/cognitivevalue-assessmentmckinsey/our-insights/big-data-the-next-frontier-forinnovation
MNIST 3 Imagenet [4] ) Cityscape [2] 5,000 47 7 [14] 3. 2 4 5 1 2 fine tuning 2 Caffe 3 http://yann.lecun.com/exdb/mnist/ 4 Chainer [13] 5 Patterns of Pretrained Model Reuse Model Zoo 5 3 (Ensemble) [5] API 4 (Distillation)[8] API 5 http://caffe.berkeleyvision.org/model zoo.html
[14] 4 2 4. 1 CACE(Change Anything Changes Everything)[11] [10] CACE 2 CACE ( [9]) 4. 2 ( ) 2 training set validation set validation set Validation set validation set validation set [15] 4. 3 [6]
( 6) 5 6 [6] 4. 4 ( ) 6 IT IoT 2020 47,000 1960 [ 1 ] Bourque, P., Fairley, R. E., et al.: Guide to the software engineering body of knowledge (SWEBOK (R)): Version 3.0, IEEE Computer Society Press, 2014. [ 2 ] Cordts, M., Omran, M., Ramos, S., Scharwächter, T., Enzweiler, M., Benenson, R., Franke, U., Roth, S., and Schiele, B.: The cityscapes dataset, CVPR Workshop on the Future of Datasets in Vision, Vol. 1, No. 2, 2015, pp. 3. [ 3 ] Cybenko, G.: Approximations by superpositions of sigmoidal functions, Mathematics of Control, Signals, and Systems, Vol. 2, No. 4(1989), pp. 303 314. [ 4 ] Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., and Fei-Fei, L.: Imagenet: A large-scale hierarchical image database, Computer Vision and Pattern Recognition, 2009. CVPR 2009. IEEE Conference on, IEEE, 2009, pp. 248 255. [ 5 ] Dietterich, T. G. et al.: Ensemble methods in machine learning, Multiple classifier systems, Vol. 1857(2000), pp. 1 15. [ 6 ] Evtimov, I., Eykholt, K., Fernandes, E., Kohno, T., Li, B., Prakash, A., Rahmati, A., and Song, D.: 6 http://www.nedo.go.jp/content/100862412.pdf
Robust Physical-World Attacks on Machine Learning Models, ArXiv e-prints, (2017). [ 7 ] Goodfellow, I., Bengio, Y., and Courville, A.: Deep Learning, MIT Press, 2016, chapter 11. [ 8 ] Hinton, G., Vinyals, O., and Dean, J.: Distilling the knowledge in a neural network, arxiv preprint arxiv:1503.02531, (2015). [ 9 ] Koh, P. W. and Liang, P.: Understanding black-box predictions via influence functions, arxiv preprint arxiv:1703.04730, (2017). [10] Parnas, D. L.: On the criteria to be used in decomposing systems into modules, Communications of the ACM, Vol. 15, No. 12(1972), pp. 1053 1058. [11] Sculley, D., Phillips, T., Ebner, D., Chaudhary, V., and Young, M.: Machine learning: The highinterest credit card of technical debt, (2014). [12] Smith, L. N.: Best Practices for Applying Deep Learning to Novel Applications, arxiv preprint arxiv:1704.01568, (2017). [13] Tokui, S., Oono, K., Hido, S., and Clayton, J.: Chainer: a next-generation open source framework for deep learning, Proceedings of workshop on machine learning systems (LearningSys) in the twentyninth annual conference on neural information processing systems (NIPS), Vol. 5, 2015. [14] Ueno, T.: Copyright Issues on Artificial Intelligence and Machine Learning, The First International Workshop on Sharing and Reuse of AI Work Products, 2017. [15] Wujek, B., Hall, P., and Güneș, F.: Best Practices for Machine Learning Applications, SAS Institute Inc, (2016). [16] :,, Vol. 4, No. 1(2016).