WHITE PAPER RNN

WHITE PAPER RNN

ii 1... 1 2 RNN?... 1 2.1 ARIMA... 1 2.2... 2 2.3 RNN Recurrent Neural Network... 3 3 RNN... 5 3.1 RNN... 6 3.2 RNN... 6 3.3 RNN... 7 4 SAS Viya RNN... 8 4.1... 9 4.2... 11 4.3... 15 5... 17 Appendix... 18 A. CAS Server... 18 B. CAS Action Python API SWAT DLPy... 19 C.... 20

1 1 SAS Viya RNN Recurrent Neural Network ARIMA RNN RNN SAS Viya RNN 2 RNN? 2.1 ARIMA RNN ARIMA AR Autoregressive model x t = a 1 x t 1 + a 2 x t 2 1 2 t x t 1 x t 2 x t x t 1 x t 2 x t AR ARIMA ARIMA AR ARIMA

2 S t S t+1 S t+2 x t1 x t+1 x t+2 AR (a 1, a 2 ) 1 ARIMA 2 2.2 RNN 3 y = 1 f 1 (x) + 2 f 2 (x) +... x y y = f 1 ( f 2 ( f 3 (x))) 1 2 3

3 入力層隠れ層出力層 x 1 x 2 y x 3 ノード x D 1 層 2 層 3 層 4 層 4 5 2.3 RNN Recurrent Neural Network RNN RNN 隠れ層 x t 1 x t 4 + 3 + 1 4 5 Montufar, Guido F., et al. "On the number of linear regions of deep neural networks." Advances in neural information processing systems. 2014

4 x t 1 x t h t h t+1 h t+2 h t+10 x t+10 x t x t+1 x t+9 100 100 RNN Recurrent Neural Network RNN 隠れ層 入力 出力 x t h t h 展開 t h t+1 h t+2 x t x t+1 x t+2

5 RNN = RNN RNN LSTM Long Short-Term Memory LSTM RNN LSTM RNN LSTM LSTM ブロック 入力ゲート 出力ゲート 活性化活性化 x t CEC 関数 関数 h t 忘却ゲート LSTM LSTM 3 RNN RNN RNN RNN

6 3.1 RNN RNN RNN ARIMA ( ARIMA RNN) RNN RNN 100 6 RNN RNN RNN 3.2 RNN RNN RNN RNN 3 36 RNN RNN RNN RNN ARIMA RNN 6 1 =1 1 =12 1 =1 1 =365

7 RNN RNN RNN RNN 3.3 RNN RNN RNN RNN RNN RNN RNN RNN RNN RNN 時系列モデリングにおける RNN の適用可能性判定 学習オブザベーション数が多い Yes No ビジネス的にブラックボックス化されたモデルが許容される RNN は適さない Yes No RNN の長所が現れ 一般的に検証できた場合例えば 言語処理等 RNN は適さない Yes No RNN が適している RNN と他のモデルを比較し 精度の高いモデルを採用すべき

8 RNN RNN RNN 4 SAS Viya RNN SAS Viya CAS Cloud Analytic Services CAS REST API Python R Jupyter Notebook Python SAS Viya Python SWAT DLPy SWAT DLPy SAS Viya Python SAS REST API Python SAS Viya SAS Viya SWAT DLPy Appendix SAS 9 380 kwh SAS Viya RNN 4 Solar_Power 1 Global_Horizontal_Irradiance Hour 24 Solar_Elevation_Angle Day_Time_Indicator 1 0 1 100 RNN

9 4.1 RNN CAS csv Pandas 'Solar_ Power' 26 RNN SAS Viya CAS Table from_ pandas() Pandas SAS Viya CAS Table Python Pandas Python SAS Viya

10 SAS Viya RNN 1 1 7 5 5 DLPy prepare_ subsequences() DLPy timeseries_partion() 8 7 {x1, x 2, x 3,...} {x 11, x 12, x 13, x 14 } {x 11, x 12, x 13, x 14 } 1 8

11 SAS data step SAS data step 4.2 RNN 2 RNN LSTM SAS Viya RNN 2 15 RNN Sequential() solar_forecast add() "std=" STD STD 0 1 LSTM 15 2 1 "output_type=" samelength "init=" XAVIER 9 2 encoding "reversed=" 1 2 "reversed=false" "act=" IDENTITY 10 "error=" NORMAL 9 Glorot, X., and Bengio, Y. Understanding the difficulty of training deep feedforward neural networks. In Proceedings of the thirteenth international conference on artificial intelligence and statistics, pp. 249-256, 2010. 10

12 RNN RNN RNN RNN ( ) 4

13 RNN mini_batch_size RNN 4 max_epochs 150 150 ADAM AdamSolver() ADAM learning_rate 0.01 learning_rate_policy step gamma 0.2 step_size 30 0.01 gamma 30 crip_grad 11 10000-10000

14 12 input_length 5 target_length token_size token_size 1 1 4 token_size 5 sequence_opt 3 prepare_subsequences() token_size 1 1 5 train() "seed=" 1234 SAS Viya GPU GPU CPU GPU CPU train() "gpu=1" GPU 11 12 4 5 1 RNN SAS Viya

15 4.3 1 score() "init_weights=" "copy_ vars=" 'Solar_Power' 'datetime' solar_ forecast_score_out score() "gpu=1" GPU RNN 0 0 0 0 _DL_Pred_2

16 WAPE 13 RMSE 14 WAPE=6.17% WAPE 13 WAPE: Weighted Absolute Percentage Error % 0 14 RMSE: Root Mean Squared Error 2 1 1

17 5 SAS Viya RNN Recurrent Neural Network ARIMA RNN RNN SAS Viya RNN SAS Viya RNN RNN DLPy DLPy

18 Appendix A. CAS Server SAS Viya SAS Cloud Analytic Services CAS CAS SMP/MPP SAS Viya GPU SAS Viya CAS CAS # 1 import swat import dlpy sess = swat.cas( host, port, user, password) image_table = dlpy.imagetable.load_files( sess, path='./imgs/', casout={'name':'my_images'}) swat dlpy CAS CAS # 2 from dlpy import applications vgg16model = applications.vgg16( sess, pre_trained_weights=true, pre_trained_weights_file="./vgg_ilsvrc_16_layers.caffemodel.h5", include_top=true) CAS result # 3 result = vgg16model.predict(image_table)

19 Jupyter CAS ❶ ❷ ❸ Memory / Storate Memory Viya Client (Jupyter) Viya Server ❶ ローカルにあるファイルやデータを読み込んで CAS サーバーに格納する (# コード 1) ❷ CAS サーバー上にモデルを構築する (# コード 2) ❸ 分析を実行し 結果を取得する (# コード 3) B. CAS Action Python API SWAT DLPy CAS CAS WebAPI WebAPI REST Python WebAPI SWAT Scripting Wrapper for Analytics Transfer SWAT WebAPI Python API API REST API CAS Action Action ActionSet ImageActionSet DeeplearnActionSet Action SWAT ActionSet Actionset Action SWAT SWAT DLPy DLPy SWAT DeeplearnActionSet Keras PyTorch Pythonic DNN ONNX SWAT CAS Action ActionSet SWAT DLPy

20 Client App DLPy Viya Client (Jupyter) deeplearn SWAT Image addlayer augumentimages buildmodel compareimages : : ActionSet Action Viya Server CAS C. SAS Viya Getting Started with SAS Viya 3.4 for Python https://go.documentation.sas.com/?cdcid=pgmsascdc&cdcversion=9.4_3.4 &docsetid=caspg3&docsettarget=titlepage.htm&locale=en SWAT API 1.5.0 https://sassoftware.github.io/python-swat/api.html SWAT https://github.com/sassoftware/python-swat DLPy API 1.0.1 https://sassoftware.github.io/python-dlpy/index.html DLPy https://github.com/sassoftware/python-dlpy DLPy https://github.com/sassoftware/python-dlpy/tree/master/examples An Introduction to SAS Viya 3.4 Programming https://go.documentation.sas.com/?cdcid=pgmsascdc&cdcversion=9.4_3.4 &docsetid=pgmdiff&docsettarget=titlepage.htm&locale=en

21 SAS Cloud Analytic Services 3.4: Fundamentals http://documentation.sas.com/?cdcid=pgmsascdc&cdcversion=9.4_3.4&doc setid=casfun&docsettarget=titlepage.htm&locale=ja Deep Learning Action Set API http://documentation.sas.com/?cdcid=pgmsascdc&cdcversion=9.4_3.4&doc setid=casdlpg&docsettarget=cas-deeplearn-tblofactions.htm&locale=ja

SAS Institute Japan www.sas.com/jp jpnsasinfo@sas.com 106-6111 6-10-1 11F Tel: 03 6434 3000 Fax: 03 6434 3001 530-0004 1-4-16 12F Tel: 06 6345 5700 Fax: 06 6345 5655 SAS SAS SAS Institute Inc. SAS Institute Inc. Copyright 2019, SAS Institute Inc. All rights reserved. JP2019WP_Viya-RNN_FK