Abstract: Deep learning rejuvenated artificial intelligence; in particular, it led to tremendous progress in classification tasks. This talk discusses two enhanced deep learning methodologies for supervised classification. A deep belief network (DBN), an unsupervised learning model, is used to initialize classifiers before tuning on a labeled training dataset. We develop supervised models incorporating DBN to improve such two-phase learning. Our approach allows single-phase learning through bilevel programming and new loss functions. The improvements over two-phase are consistent. In addition, we develop a combined model that simultaneously extracts hierarchical representations of images by a convolutional neural network and learns a tree of classes to predict a final label by a recurrent neural network. The model leads to image classification that captures the hierarchical characteristics of the classes. The improvements over standard convolutional networks are 2% on average.