Automatic structure and parameter tuning method for deep neural network soft sensor in chemical industries
(2.Tsinghua National Laboratory for Information Science and Technology, Beijing, China 100084)
【Abstract】Deep learning has been applied to the field of soft sensing in process industries. However, the structure and parameters of deep neural network (DNN) have to be tuned manually, which requires solid fundamental knowledge about machine learning and rich experiences on parameter tuning. Complicated tuning procedure restricts generalization application of deep learning in chemical industries. A structure and parameter tuning method for DNN soft sensor with little manual intervention is proposed by systematic analysis on selection process of each essential DNN parameter from massive experiments. The presented method can greatly simplify the tuning procedure and offer a reference for engineers to study and use deep learning. Studies on crude-oil distillation and coal gasification process verify the effectiveness and generality of the proposed method.
【Keywords】 deep learning; prediction; parameter tuning; algorithm; neural network;
(Translated by CAI TX)
 HUANG D X, WANG J C, JIN Y H. Process Control Systems [M]. Beijing: Tsinghua University Press, 2011 (in Chinese).
 HUANG D X, YE X Y, ZHU J M, et al. Advanced Process Control in Chemical Industrial Processes [M]. Beijing: Chemical Industry Press, 2006 (in Chinese).
 GALICIA H J, HE Q P, WANG J. A reduced order soft sensor approach and its application to a continuous digester [J]. Journal of Process Control, 2011, 21 (4): 489–500.
 QIN S J. Neural networks for intelligent sensors and control—practical issues and some solutions [J]. Neural Systems for Control, 1997, 8 (1): 213–234.
 LLDIKO E F, JEROME H F. A statistical view of some chemometrics regression tools [J]. Technometrics, 1993, 35 (2): 109–135.
 KADLEC P, GABRYS B, STRANDT S. Data–driven soft sensors in the process industry [J]. Computers & Chemical Engineering, 2009, 33 (4): 795–814.
 LIN B, RECKE B, KNUDSEN J K H, et al. A systematic approach for soft sensor development [J]. Computers & Chemical Engineering, 2007, 31 (5): 419–425.
 PARK S, HAN C. A nonlinear soft sensor based on multivariate smoothing procedure for quality estimation in distillation columns [J]. Computers & Chemical Engineering, 2000, 24 (2): 871–877.
 YANG Y, CHAI T. Soft sensing based on artificial neural network [C] //American Control Conference, 1997. Proceedings of the IEEE. 2002: 674–678.
 QIN S J, MCAVOY T J. Nonlinear PLS modeling using neural networks [J]. Computers & Chemical Engineering, 1992, 16 (4): 379–391.
 YAN W, SHAO H, WANG X. Soft sensing modeling based on support vector machine and Bayesian model selection [J]. Computers & Chemical Engineering, 2004, 28 (8): 1489–1498.
 HINTON G E, SALAKHUTDINOV R R. Reducing the dimensionality of data with neural networks [J]. Science, 2006, 313 (5786): 504–507.
 BENGIO Y. Learning deep architectures for AI [J]. Foundations & Trends®in Machine Learning, 2009, 2 (1): 1–127.
 PAN Z, LIU Y, LIU G, et al. Topic Network: Topic Model with Deep Learning for Image Classification [C] //International Conference on Knowledge Science, Engineering and Management, 2015: 65–73.
 DENG L, HINTON G, KINGSBURY B. New types of deep neural network learning for speech recognition and related applications: an overview [C] //IEEE International Conference on Acoustics, Speech and Signal Processing. IEEE. 2013: 8599–8603.
 MNIH A, HINTON G. A scalable hierarchical distributed language model [C] //International Conference on Neural Information Processing Systems. Curran Associates Inc. 2008: 1081–1088.
 SHANG C, YANG F, HUANG D, et al. Data-driven soft sensor development based on deep learning technique [J]. Journal of Process Control, 2014, 24 (3): 223–233.
 MAO K Z, TAN K C, SER W. Probabilistic neural-network structure determination for pattern classification [J]. IEEE Transactions on Neural Networks, 2000, 11 (4): 1009–1016.
 LEUNG F H F, LAM H K, LING S H, et al. Tuning of the structure and parameters of a neural network using an improved genetic algorithm [J]. IEEE Trans. Neural Netw., 2003, 14 (1): 79–88.
 TSAI J T, CHOU J H, LIU T K. Tuning the structure and parameters of a neural network by using hybrid Taguchi-genetic algorithm [J]. IEEE Transactions on Neural Networks, 2006, 17 (1): 69–80.
 BENGIO Y, GUYON G, DROR V, et al. Deep learning of representations for unsupervised and transfer learning [C] //Workshop on Unsupervised & Transfer Learning. 2012.
 BERGSTRA J, BENGIO Y. Random search for hyper-parameter optimization [J]. Journal of Machine Learning Research, 2012, 13 (1): 281–305.
 BENGIO Y. Practical Recommendations for Gradient–Based Training of Deep Architectures [M]//Neural Networks: Tricks of the Trade. Berlin: Springer Berlin Heidelberg, 2012: 133–144.
 BENGIO Y, LAMBLIN P, POPOVICI D, et al. Greedy layer-wise training of deep networks [J]. Advances in Neural Information Processing Systems, 2007, 19 (1): 153–160.
 ZHOU Z H. Machine Learning [M]. Beijing: Tsinghua University Press, 2016 (in Chinese).
 LV F, WEN C, BAO Z, et al. Fault diagnosis based on deep learning [C] //American Control Conference. 2016: 6851–6856.
 CARREIRA-PERPINAN M A, HINTON G E. On contrastive divergence learning [C] //Artificial Intelligence & Statistics. 2005.
 HINTON G. A practical guide to training restricted Boltzmann machines [J]. Momentum, 2010, 9 (1): 926–946.
 MONAGHAN R F D. Dynamic reduced order modeling of entrained flow gasifiers [D]. Cambridge: Massachusetts Institute of Technology, 2010.
 JI P, GAO X, HUANG D, et al. Prediction of syngas compositions in shell coal gasification process via dynamic soft-sensing method [C] //IEEE International Conference on Control and Automation. IEEE. 2013: 244–249.