Search

We propose a novel introspective model for variational neural machine translation (IntroVNMT) in this paper, inspired by the recent successful application of introspective variational autoencoder (IntroVAE) in high quality image synthesis. Different from the vanilla variational NMT model, IntroVNMT is capable of improving itself introspectively by evaluating the quality of the generated target sentences according to the high-level latent variables of the real and generated target sentences. As a consequence of introspective training, the proposed model is able to discriminate between the generated and real sentences of the target language via the latent variables generated by the encoder of the model. In this way, IntroVNMT is able to generate more realistic target sentences in practice. In the meantime, IntroVNMT inherits the advantages of the variational autoencoders (VAEs), and the model training process is more stable than the generative adversarial network (GAN) based models. Experimental results on different translation tasks demonstrate that the proposed model can achieve significant improvements over the vanilla variational NMT model.

We consider the problem of minimizing the regularized empirical risk function which is represented as the average of a large number of convex loss functions plus a possibly non-smooth convex regularization term. In this paper, we propose a fast variance reducing (VR) stochastic method called Prox2-SAGA. Different from traditional VR stochastic methods, Prox2-SAGA replaces the stochastic gradient of the loss function with the corresponding gradient mapping. In addition, Prox2-SAGA also computes the gradient mapping of the regularization term. These two gradient mappings constitute a Douglas-Rachford splitting step. For strongly convex and smooth loss functions, we prove that Prox2-SAGA can achieve a linear convergence rate comparable to other accelerated VR stochastic methods. In addition, Prox2-SAGA is more practical as it involves only the stepsize to tune. When each loss function is smooth but non-strongly convex, we prove a convergence rate of $${\mathcal {O}}(1/k)$$ for the proposed Prox2-SAGA method, where k is the number of iterations. Moreover, experiments show that Prox2-SAGA is valid for non-smooth loss functions, and for strongly convex and smooth loss functions, Prox2-SAGA is prominently faster when loss functions are ill-conditioned.

With the increase in the amount of data and the expansion of model scale, distributed parallel training becomes an important and successful technique to address the optimization challenges. Nevertheless, although distributed stochastic gradient descent (SGD) algorithms can achieve a linear iteration speedup, they are limited significantly in practice by the communication cost, making it difficult to achieve a linear time speedup. In this paper, we propose a computation and communication decoupled stochastic gradient descent (CoCoD-SGD) algorithm to run computation and communication in parallel to reduce the communication cost. We prove that CoCoD-SGD has a linear iteration speedup with respect to the total computation capability of the hardware resources. In addition, it has a lower communication complexity and better time speedup comparing with traditional distributed SGD algorithms. Experiments on deep neural network training demonstrate the significant improvements of CoCoD-SGD: when training ResNet18 and VGG16 with 16 Geforce GTX 1080Ti GPUs, CoCoD-SGD is up to 2-3$\times$ faster than traditional synchronous SGD., IJCAI2019, 20 pages, 21 figures

Recent advances in the field of network embedding have shown that low-dimensional network representation is playing a critical role in network analysis. Most existing network embedding methods encode the local proximity of a node, such as the first- and second-order proximities. While being efficient, these methods are short of leveraging the global structural information between nodes distant from each other. In addition, most existing methods learn embeddings on one single fixed network, and thus cannot be generalized to unseen nodes or networks without retraining. In this paper we present SPINE, a method that can jointly capture the local proximity and proximities at any distance, while being inductive to efficiently deal with unseen nodes or networks. Extensive experimental results on benchmark datasets demonstrate the superiority of the proposed framework over the state of the art., IJCAI 2019

To study the effects and the therapeutic mechanism of hyperbaric oxygen (HBO) on prostaglandin E(2) (PGE(2)) in alveolar bone and gingiva of experimental periodontitis in animal.Experimental periodontitis was produced by silk thread sutures combined with high content sugar diet. For HBO therapy, they were exposed to a pressure of 0.25 MPa (2.5ATA), breathing pure oxygen one session a day for 60 min. The treatment course was 2 weeks. The value of PGE(2) in gingiva and alveolar bone was analyzed by enzyme immunoassay (EIA).The value of PGE(2) in gingiva of control group was 3.21 ng/g, and that of PGE(2) in alveolar bone was 3.22 ng/g. The contents of PGE(2) in gingiva (13.96 ng/g) and alveolar bone (13.32 ng/g) of periodontitis group increased markedly than control group (P0.01). The contents of PGE(2) in gingiva (5.21 ng/g) of HBO group were 62.7% which was lower than that of periodontitis group, and the value of PGE(2) in alveolar bone (4.05 ng/g) were 69.6% lower than that of periodontitis group. The difference of PGE(2) in gingiva or alveolar bone was significant for the HBO group and periodontitis group (P0.01).The contents of PGE(2) in alveolar bone and gingiva increased markedly when experimental periodontitis has formed. The value of PGE(2) in alveolar bone and gingiva reduce markedly after HBO exposure, and the decreased rate of PGE(2) in alveolar bone is more evident than that of PGE(2) in gingiva after HBO therapy.

IN this paper, a zonal vortex method and its application to simulating the impulsively started flow of a circular cylinder are presented. This method treats the attached viscous flow region and the separated flow region separately. The attached flow region is computed through solving boundarylayer equations by the finite-difference method. Only the separated flow is computed through solving Navier-Stokes equations by the vortex method. Since the separated flow region has a length scale of 0(1), in the vortex method used for this region, the number of new vortices introduced at the surface of the body per time step is relatively insensitive to the Reynolds number of the flow. For simulation of highReynolds-number flows with massive separation, the total number of vortices in the flowfield, hence the computer storage and computer time, is greatly reduced. Contents The impulsively started flow around a circular cylinder is complex and all of the phenomena of fluid mechanics are presented. Because of its fundamental importance, this flow has become a typical problem in the study of separated flows. Bouard and Coutanceau1 have done careful experimental studies on this problem and provided valuable data for comparison with numerical solutions. Smith and Stansby2 have recently calculated this flow using a vortex method. In this method, the vorticity equation is solved using random walk for diffusion and the vortex-in-cell method for convection in a time-stepping procedure. Vortices are introduced at the surface at each time step to satisfy the no-slip condition. Their calculations revealed detailed flow features of the wake and were in good agreement with experimental observations in Ref. 1. Their numerical experiment showed that accurate simulation requires the introduction of a sufficiently large number of vortices at the surface per time step. This number increases as the Reynolds number increases. Therefore, for simulation of a high-Reynolds-number flow by the vortex method, the total number of vortices in the flow may become very large as time increases, requiring large computer storage and computer time. In this paper, the vortex method is improved by a zonal approach. For a high-Reynolds- number flow with massive separation, the vorticity in the upstream of the separation point is confined in a layer near the surface of the body. The length scale of the thickness of this layer is 0(Re~l/2). The length scale of the separated flow region is 0(1). It is difficult to accommodate these two scales when solving the NavierStokes equations in the whole flow region. The approach of