Search

Optical metasurface has brought a revolution in label-free molecular sensing, attracting extensive attention. Currently, such sensing approaches are being designed to respond to peak wavelengths with a higher Q factor in the visible and near-infrared regions.Nevertheless, a higher Q factor that enhances light confinement will inevitably deteriorate the wavelength sensitivity and complicate the sensing system. We propose a Q-switched sensing mechanism, which enables the real part of the refractive index to effectively perturbate the damping loss of the oscillator, resulting in a boost of peak intensity.Consequently, a higher Q factor in Q-switched sensor can further enhance the peak sensitivity while remaining compatible with broadband light sources, simultaneously meeting the requirements of high performance and a compact system.This is achieved in a unique 3D bound-state-in-continuum (BIC) metasurface which can be mass-produced by wafer-scale aluminum-nanoimprinting technology and provides a peak intensity sensitivity up to 928 %/RIU.Therefore, a miniaturized BIC biosensing system is realized, with a limit of detection to 10E-5 refractive index units and 129 aM extracellular vesicles in clinical lung cancer diagnosis, both of which are magnitudes lower than those of current state-of-the-art biosensors. It further demonstrates significant potential for home cancer self-testing equipment for post-operative follow-up. This Q-switched sensing mechanism offers a new perspective for the commercialization of advanced and practical BIC optical biosensing systems in real-setting scenarios.

Multi-object tracking in sports scenarios has become one of the focal points in computer vision, experiencing significant advancements through the integration of deep learning techniques. Despite these breakthroughs, challenges remain, such as accurately re-identifying players upon re-entry into the scene and minimizing ID switches. In this paper, we propose an appearance-based global tracklet association algorithm designed to enhance tracking performance by splitting tracklets containing multiple identities and connecting tracklets seemingly from the same identity. This method can serve as a plug-and-play refinement tool for any multi-object tracker to further boost their performance. The proposed method achieved a new state-of-the-art performance on the SportsMOT dataset with HOTA score of 81.04%. Similarly, on the SoccerNet dataset, our method enhanced multiple trackers' performance, consistently increasing the HOTA score from 79.41% to 83.11%. These significant and consistent improvements across different trackers and datasets underscore our proposed method's potential impact on the application of sports player tracking. We open-source our project codebase at https://github.com/sjc042/gta-link.git., Comment: Accepted by ACCV 2024 MLCSA Workshop

In this work, we proposed a new 3D integration technology: the Flip 3D integration (F3D), consisting of the 3D transistor stacking, the 3D dual-sided interconnects, the 3D die-to-die stacking and the dual-sided Monolithic 3D (M3D). Based on a 32-bit FFET RISCV core, besides the scaling benefits of the Flip FET (FFET), the dual-sided signal routing shows even more routing flexibility with 6.8% area reduction and 5.9% EDP improvement. Novel concepts of Multi-Flipping processes (Double Flips and Triple Flips) were proposed to relax the thermal budget constraints in the F3D and thus support the dual-sided M3D in the F3D. The core's EDP and frequency are improved by up to 3.2% and 2.3% respectively, after BEOL optimizations based on the Triple Flips compared with unoptimized ones., Comment: Accepted by EDTM 2025

Despite the remarkable success of Transformer-based Large Language Models (LLMs) across various domains, understanding and enhancing their mathematical capabilities remains a significant challenge. In this paper, we conduct a rigorous theoretical analysis of LLMs' mathematical abilities, with a specific focus on their arithmetic performances. We identify numerical precision as a key factor that influences their effectiveness in mathematical tasks. Our results show that Transformers operating with low numerical precision fail to address arithmetic tasks, such as iterated addition and integer multiplication, unless the model size grows super-polynomially with respect to the input length. In contrast, Transformers with standard numerical precision can efficiently handle these tasks with significantly smaller model sizes. We further support our theoretical findings through empirical experiments that explore the impact of varying numerical precision on arithmetic tasks, providing valuable insights for improving the mathematical reasoning capabilities of LLMs.

The non-Abelian braiding describes the exchange behavior of anyons, which can be leveraged to encode qubits for quantum computing. Recently, this concept has been realized in classical photonic and acoustic systems. However, these implementations are constrained by adiabatic conditions, necessitating long operation distances and impeding practical applications. Here, we conceive and demonstrate a shortcut to adiabatic (STA) braiding of telecommunication light in three-dimensional silicon photonic chips. Our device comprises tri-layer silicon waveguides stacked and embedded in the SU-8 polymer, employing an STA strategy to expedite the braiding operations and give rise to compact devices that function as photonic quantum X, Y, and Z gates. We further experimentally observed non-Abelian braiding behaviors based on this STA-braiding scheme. Remarkably, this achievement represents the most compact braiding apparatus ever reported, with a size reduction of nearly three orders of magnitude compared to previous works. This work presents a feasible approach to accelerating adiabatic braiding evolutions, paving the way for compact, CMOS-compatible non-Abelian photonic devices.

Assessing gross motor development in toddlers is crucial for understanding their physical development and identifying potential developmental delays or disorders. However, existing datasets for action recognition primarily focus on adults, lacking the diversity and specificity required for accurate assessment in toddlers. In this paper, we present ToddlerAct, a toddler gross motor action recognition dataset, aiming to facilitate research in early childhood development. The dataset consists of video recordings capturing a variety of gross motor activities commonly observed in toddlers aged under three years old. We describe the data collection process, annotation methodology, and dataset characteristics. Furthermore, we benchmarked multiple state-of-the-art methods including image-based and skeleton-based action recognition methods on our datasets. Our findings highlight the importance of domain-specific datasets for accurate assessment of gross motor development in toddlers and lay the foundation for future research in this critical area. Our dataset will be available at https://github.com/ipl-uw/ToddlerAct., Comment: Accepted by 2024 ECCV ABAW Workshop

Discrete diffusion models with absorbing processes have shown promise in language modeling. The key quantities to be estimated are the ratios between the marginal probabilities of two transitive states at all timesteps, called the concrete score. In this paper, we reveal that the concrete score in absorbing diffusion can be expressed as conditional probabilities of clean data, multiplied by a time-dependent scalar in an analytic form. Motivated by this finding, we propose reparameterized absorbing discrete diffusion (RADD), a dedicated diffusion model without time-condition that characterizes the time-independent conditional probabilities. Besides its simplicity, RADD can reduce the number of function evaluations (NFEs) by caching the output of the time-independent network when the noisy sample remains unchanged in a sampling interval. Empirically, RADD is up to 3.5 times faster while achieving similar performance with the strongest baseline. Built upon the new perspective of conditional distributions, we further unify absorbing discrete diffusion and any-order autoregressive models (AO-ARMs), showing that the upper bound on the negative log-likelihood for the diffusion model can be interpreted as an expected negative log-likelihood for AO-ARMs. Further, our RADD models achieve SOTA performance among diffusion models on 5 zero-shot language modeling benchmarks (measured by perplexity) at the GPT-2 scale. Our code is available at https://github.com/ML-GSAI/RADD.

Large language models have consistently struggled with complex reasoning tasks, such as mathematical problem-solving. Investigating the internal reasoning mechanisms of these models can help us design better model architectures and training strategies, ultimately enhancing their reasoning capability. In this study, we constructed a symbolic dataset to investigate the mechanisms by which Transformer models employ vertical thinking strategy based on their inherent structure and horizontal thinking strategy based on Chain of Thought to achieve multi-step reasoning. We introduced the concept of buffer mechanism: the model stores various information in distinct buffers and selectively extracts them through the query-key matrix. We proposed a random matrix-based algorithm to enhance the model's reasoning ability, resulting in a 75% reduction in the training time required for the GPT-2 model to achieve generalization capability on the PrOntoQA dataset. These findings provide new insights into understanding the mechanisms of large language models.

The construction of Sweedler duality is an important tool in the theory of Hopf algebras over a field, which is a right adjoint to the dual algebra functor. In this paper, we study the Sweedler duality of Hom-algebras and their Hom-modules. We delve into the structure of Hom-coalgebras and derive the linear morphisms associated with them. Additionally, as an application, we present the (right) Hom-(co)module morphisms under the Sweedler duality., Comment: 11 pages

Photonic structures with Weyl points (WPs), including type-I and type-II, promise nontrivial surface modes and intriguing light manipulations for their three-dimensional topological bands. While previous studies mainly focus on exploring WPs in a uniform Weyl structure, here we establish Weyl heterostructures (i.e., a nonuniform Weyl lattice) with different rotational orientations in the synthetic dimension by nanostructured photonic waveguides. In this work, we unveil a transition between bound and extended modes on the interface of type-II Weyl heterostructures by tuning their rotational phases, despite the reversed topological order across the interface. This mode transition is also manifested from the total transmission to total reflection at the interface. All of these unconventional effects are attributed to the tilted dispersion of type-II Weyl band structure that can lead to mismatched bands and gaps across the interface. As a comparison, the type-I Weyl heterostructures lack the phase transition due to the untilted band structure. This work establishes a flexible scheme of artificial Weyl heterostructures that opens a new avenue towards high-dimensional topological effects and significantly enhances our capabilities in on-chip light manipulations., Comment: to be published in Phys. Rev. Lett

Non-Hermitian physics has greatly enriched our understanding of nonequilibrium phenomena and uncovered novel effects such as the non-Hermitian skin effect (NHSE) that has profoundly revolutionized the field. NHSE is typically predicted in systems with nonreciprocal couplings which, however, are difficult to realize in experiments. Without nonreciprocal couplings, the NHSE can also emerge in systems with coexisting gauge fields and loss or gain (e.g., in Floquet non-Hermitian systems). However, such Floquet NHSE remains largely unexplored in experiments. Here, we realize the Floquet NHSEs in periodically modulated optical waveguides integrated on a silicon photonics platform. By engineering the artificial gauge fields induced by the periodical modulation, we observe various Floquet NHSEs and unveil their rich topological transitions. Remarkably, we discover the transitions between the normal unipolar NHSEs and an unconventional bipolar NHSE which is accompanied by the directional reversal of the NHSEs. The underlying physics is revealed by the band winding in complex quasienergy space which undergoes a topology change from isolated loops with the same winding to linked loops with opposite windings. Our work unfolds a new route toward Floquet NHSEs originating from the interplay between gauge fields and dissipation effects and offers fundamentally new ways for steering light and other waves., Comment: 12 pages, 3 figures

We study an optimization problem originated from the Grothendieck constant. A generalized normal equation is proposed and analyzed. We establish a correspondence between solutions of the general normal equation and its dual equation. Explicit solutions are described for the two-dimensional case., Comment: 6pp

Achieving spatiotemporal control of light at high-speeds presents immense possibilities for various applications in communication, computation, metrology, and sensing. The integration of subwavelength metasurfaces and optical waveguides offers a promising approach to manipulate light across multiple degrees of freedom at high-speed in compact photonic integrated circuit (PICs) devices. Here, we demonstrate a gigahertz-rate-switchable wavefront shaping by integrating metasurface, lithium niobite on insulator (LNOI) photonic waveguide and electrodes within a PIC device. As proofs of concept, we showcase the generation of a focus beam with reconfigurable arbitrary polarizations, switchable focusing with lateral focal positions and focal length, orbital angular momentum light beams (OAMs) as well as Bessel beams. Our measurements indicate modulation speeds of up to gigahertz rate. This integrated platform offers a versatile and efficient means of controlling light field at high-speed within a compact system, paving the way for potential applications in optical communication, computation, sensing, and imaging.

Guidance in conditional diffusion generation is of great importance for sample quality and controllability. However, existing guidance schemes are to be desired. On one hand, mainstream methods such as classifier guidance and classifier-free guidance both require extra training with labeled data, which is time-consuming and unable to adapt to new conditions. On the other hand, training-free methods such as universal guidance, though more flexible, have yet to demonstrate comparable performance. In this work, through a comprehensive investigation into the design space, we show that it is possible to achieve significant performance improvements over existing guidance schemes by leveraging off-the-shelf classifiers in a training-free fashion, enjoying the best of both worlds. Employing calibration as a general guideline, we propose several pre-conditioning techniques to better exploit pretrained off-the-shelf classifiers for guiding diffusion generation. Extensive experiments on ImageNet validate our proposed method, showing that state-of-the-art diffusion models (DDPM, EDM, DiT) can be further improved (up to 20%) using off-the-shelf classifiers with barely any extra computational cost. With the proliferation of publicly available pretrained classifiers, our proposed approach has great potential and can be readily scaled up to text-to-image generation tasks. The code is available at https://github.com/AlexMaOLS/EluCD/tree/main.

Diffusion Probabilistic Models (DPMs) have achieved considerable success in generation tasks. As sampling from DPMs is equivalent to solving diffusion SDE or ODE which is time-consuming, numerous fast sampling methods built upon improved differential equation solvers are proposed. The majority of such techniques consider solving the diffusion ODE due to its superior efficiency. However, stochastic sampling could offer additional advantages in generating diverse and high-quality data. In this work, we engage in a comprehensive analysis of stochastic sampling from two aspects: variance-controlled diffusion SDE and linear multi-step SDE solver. Based on our analysis, we propose SA-Solver, which is an improved efficient stochastic Adams method for solving diffusion SDE to generate data with high quality. Our experiments show that SA-Solver achieves: 1) improved or comparable performance compared with the existing state-of-the-art sampling methods for few-step sampling; 2) SOTA FID scores on substantial benchmark datasets under a suitable number of function evaluations (NFEs)., Comment: Accepted in Neurips 2023

In generative modeling, numerous successful approaches leverage a low-dimensional latent space, e.g., Stable Diffusion models the latent space induced by an encoder and generates images through a paired decoder. Although the selection of the latent space is empirically pivotal, determining the optimal choice and the process of identifying it remain unclear. In this study, we aim to shed light on this under-explored topic by rethinking the latent space from the perspective of model complexity. Our investigation starts with the classic generative adversarial networks (GANs). Inspired by the GAN training objective, we propose a novel "distance" between the latent and data distributions, whose minimization coincides with that of the generator complexity. The minimizer of this distance is characterized as the optimal data-dependent latent that most effectively capitalizes on the generator's capacity. Then, we consider parameterizing such a latent distribution by an encoder network and propose a two-stage training strategy called Decoupled Autoencoder (DAE), where the encoder is only updated in the first stage with an auxiliary decoder and then frozen in the second stage while the actual decoder is being trained. DAE can improve the latent distribution and as a result, improve the generative performance. Our theoretical analyses are corroborated by comprehensive experiments on various models such as VQGAN and Diffusion Transformer, where our modifications yield significant improvements in sample quality with decreased model complexity., Comment: Accepted to NeurIPS 2023 (Spotlight)

Energy-Based Models (EBMs) have been widely used for generative modeling. Contrastive Divergence (CD), a prevailing training objective for EBMs, requires sampling from the EBM with Markov Chain Monte Carlo methods (MCMCs), which leads to an irreconcilable trade-off between the computational burden and the validity of the CD. Running MCMCs till convergence is computationally intensive. On the other hand, short-run MCMC brings in an extra non-negligible parameter gradient term that is difficult to handle. In this paper, we provide a general interpretation of CD, viewing it as a special instance of our proposed Diffusion Contrastive Divergence (DCD) family. By replacing the Langevin dynamic used in CD with other EBM-parameter-free diffusion processes, we propose a more efficient divergence. We show that the proposed DCDs are both more computationally efficient than the CD and are not limited to a non-negligible gradient term. We conduct intensive experiments, including both synthesis data modeling and high-dimensional image denoising and generation, to show the advantages of the proposed DCDs. On the synthetic data learning and image denoising experiments, our proposed DCD outperforms CD by a large margin. In image generation experiments, the proposed DCD is capable of training an energy-based model for generating the Celab-A $32\times 32$ dataset, which is comparable to existing EBMs.

Deep learning-based object detectors have driven notable progress in multi-object tracking algorithms. Yet, current tracking methods mainly focus on simple, regular motion patterns in pedestrians or vehicles. This leaves a gap in tracking algorithms for targets with nonlinear, irregular motion, like athletes. Additionally, relying on the Kalman filter in recent tracking algorithms falls short when object motion defies its linear assumption. To overcome these issues, we propose a novel online and robust multi-object tracking approach named deep ExpansionIoU (Deep-EIoU), which focuses on multi-object tracking for sports scenarios. Unlike conventional methods, we abandon the use of the Kalman filter and leverage the iterative scale-up ExpansionIoU and deep features for robust tracking in sports scenarios. This approach achieves superior tracking performance without adopting a more robust detector, all while keeping the tracking process in an online fashion. Our proposed method demonstrates remarkable effectiveness in tracking irregular motion objects, achieving a score of 77.2% HOTA on the SportsMOT dataset and 85.4% HOTA on the SoccerNet-Tracking dataset. It outperforms all previous state-of-the-art trackers on various large-scale multi-object tracking benchmarks, covering various kinds of sports scenarios. The code and models are available at https://github.com/hsiangwei0903/Deep-EIoU.

The Bloch electron energy spectrum of a crystalline solid is determined by the underlying lattice structure at the atomic level. In a 2-dimensional (2d) crystal it is possible to impose a superlattice with nanometer-scale periodicity, allowing to tune the fundamental Bloch electron spectrum, and enabling novel physical properties which are not accessible in the original crystal. In recent years, a top-down approach for creating 2d superlattices on monolayer graphene by means of nanopatterned electric gates has been studied, which allows the formation of isolated energy bands and Hofstadter Butterfly physics in quantizing magnetic fields. Within this approach, however, evidence of electron correlations which drive many problems at the forefront of physics research remains to be uncovered. In this work we demonstrate signatures of a correlated insulator phase in Bernal-stacked bilayer graphene (BLG) modulated by a gate-defined superlattice potential, manifested as a set of resistance peaks centered at carrier densities of integer multiples of a single electron per unit cell of the superlattice potential. We associate the correlated insulator phase to the formation of flat energy bands due to the superlattice potential combined with inversion symmetry breaking. Inducing correlated electron phases with nanopatterning defined electric gates paves the way to custom-designed superlattices with arbitrary geometries and symmetries for studying band structure engineering and strongly correlated electrons in 2d materials.

Due to the ease of training, ability to scale, and high sample quality, diffusion models (DMs) have become the preferred option for generative modeling, with numerous pre-trained models available for a wide variety of datasets. Containing intricate information about data distributions, pre-trained DMs are valuable assets for downstream applications. In this work, we consider learning from pre-trained DMs and transferring their knowledge to other generative models in a data-free fashion. Specifically, we propose a general framework called Diff-Instruct to instruct the training of arbitrary generative models as long as the generated samples are differentiable with respect to the model parameters. Our proposed Diff-Instruct is built on a rigorous mathematical foundation where the instruction process directly corresponds to minimizing a novel divergence we call Integral Kullback-Leibler (IKL) divergence. IKL is tailored for DMs by calculating the integral of the KL divergence along a diffusion process, which we show to be more robust in comparing distributions with misaligned supports. We also reveal non-trivial connections of our method to existing works such as DreamFusion, and generative adversarial training. To demonstrate the effectiveness and universality of Diff-Instruct, we consider two scenarios: distilling pre-trained diffusion models and refining existing GAN models. The experiments on distilling pre-trained diffusion models show that Diff-Instruct results in state-of-the-art single-step diffusion-based models. The experiments on refining GAN models show that the Diff-Instruct can consistently improve the pre-trained generators of GAN models across various settings.

The diffusion probabilistic generative models are widely used to generate high-quality data. Though they can synthetic data that does not exist in the training set, the rationale behind such generalization is still unexplored. In this paper, we formally define the generalization of the generative model, which is measured by the mutual information between the generated data and the training set. The definition originates from the intuition that the model which generates data with less correlation to the training set exhibits better generalization ability. Meanwhile, we show that for the empirical optimal diffusion model, the data generated by a deterministic sampler are all highly related to the training set, thus poor generalization. This result contradicts the observation of the trained diffusion model's (approximating empirical optima) extrapolation ability (generating unseen data). To understand this contradiction, we empirically verify the difference between the sufficiently trained diffusion model and the empirical optima. We found, though obtained through sufficient training, there still exists a slight difference between them, which is critical to making the diffusion model generalizable. Moreover, we propose another training objective whose empirical optimal solution has no potential generalization problem. We empirically show that the proposed training objective returns a similar model to the original one, which further verifies the generalization ability of the trained diffusion model.

Fast scanning-speed and low-power-consumption are becoming progressively more and more important in realizing high-performance chiplet optical phased arrays (OPAs). Here, we establish an integrated OPA based on thin-film lithium niobate-on-insulator (LNOI) platform to access these outstanding performances. Significantly, a lithium niobate (LN) OPA chip is implemented by 32/48 channels LN waveguides enabled by electro-optic modulations, which showcases the low power consumption (1.11nJ/{\pi}}) and fast operation speed (14.4 ns) promising the advantage of the LNOI platform for integrated OPAs. As results, we experimentally achieved a beam steering with a 62.2{\deg}*8.8{\deg} field of view (FOV) and a beam divergence of 2.4{\deg}*1.2{\deg}. Moreover, by employing sparse aperiodic arrays in waveguides design we obtained a significant reduction of lateral divergence to 0.33{\deg} for the radiation beam. This work demonstrate that remarkable advantage of LNOI platform for power-saving and scalable OPA chips for various applications., Comment: 22 pages,5 figures,conferences added

In an effort to further advance semi-supervised generative and classification tasks, we propose a simple yet effective training strategy called dual pseudo training (DPT), built upon strong semi-supervised learners and diffusion models. DPT operates in three stages: training a classifier on partially labeled data to predict pseudo-labels; training a conditional generative model using these pseudo-labels to generate pseudo images; and retraining the classifier with a mix of real and pseudo images. Empirically, DPT consistently achieves SOTA performance of semi-supervised generation and classification across various settings. In particular, with one or two labels per class, DPT achieves a Fr\'echet Inception Distance (FID) score of 3.08 or 2.52 on ImageNet 256x256. Besides, DPT outperforms competitive semi-supervised baselines substantially on ImageNet classification tasks, achieving top-1 accuracies of 59.0 (+2.8), 69.5 (+3.0), and 74.4 (+2.0) with one, two, or five labels per class, respectively. Notably, our results demonstrate that diffusion can generate realistic images with only a few labels (e.g., <0.1%) and generative augmentation remains viable for semi-supervised classification. Our code is available at https://github.com/ML-GSAI/DPT., Comment: Accepted to NeurIPS 2023

Extensive empirical evidence demonstrates that conditional generative models are easier to train and perform better than unconditional ones by exploiting the labels of data. So do score-based diffusion models. In this paper, we analyze the phenomenon formally and identify that the key of conditional learning is to partition the data properly. Inspired by the analyses, we propose self-conditioned diffusion models (SCDM), which is trained conditioned on indices clustered by the k-means algorithm on the features extracted by a model pre-trained in a self-supervised manner. SCDM significantly improves the unconditional model across various datasets and achieves a record-breaking FID of 3.94 on ImageNet 64x64 without labels. Besides, SCDM achieves a slightly better FID than the corresponding conditional model on CIFAR10.

We show that bilayer graphene in the presence of a 2D superlattice potential provides a highly tunable setup that can realize a variety of flat band phenomena. We focus on two regimes: (i) topological flat bands with non-zero Chern numbers, C, including bands with higher Chern numbers |C| > 1; and (ii) an unprecedented phase consisting of a stack of nearly perfect flat bands with C = 0. For realistic values of the potential and superlattice periodicity, this stack can span nearly 100 meV, encompassing nearly all of the low-energy spectrum. We further show that in the topological regime, the topological flat band has a favorable band geometry for realizing a fractional Chern insulator (FCI) and use exact diagonalization to show that the FCI is in fact the ground state at 1/3 filling. Our results provide a realistic guide for future experiments to realize a new platform for flat band phenomena., Comment: 14 pages including supplementary materials. Published version

Diffusion probabilistic models (DPMs) are a class of powerful deep generative models (DGMs). Despite their success, the iterative generation process over the full timesteps is much less efficient than other DGMs such as GANs. Thus, the generation performance on a subset of timesteps is crucial, which is greatly influenced by the covariance design in DPMs. In this work, we consider diagonal and full covariances to improve the expressive power of DPMs. We derive the optimal result for such covariances, and then correct it when the mean of DPMs is imperfect. Both the optimal and the corrected ones can be decomposed into terms of conditional expectations over functions of noise. Building upon it, we propose to estimate the optimal covariance and its correction given imperfect mean by learning these conditional expectations. Our method can be applied to DPMs with both discrete and continuous timesteps. We consider the diagonal covariance in our implementation for computational efficiency. For an efficient practical implementation, we adopt a parameter sharing scheme and a two-stage training process. Empirically, our method outperforms a wide variety of covariance design on likelihood results, and improves the sample quality especially on a small number of timesteps., Comment: Accepted in ICML 2022

Gradient-based methods for the distributed training of residual networks (ResNets) typically require a forward pass of the input data, followed by back-propagating the error gradient to update model parameters, which becomes time-consuming as the network goes deeper. To break the algorithmic locking and exploit synchronous module parallelism in both the forward and backward modes, auxiliary-variable methods have attracted much interest lately but suffer from significant communication overhead and lack of data augmentation. In this work, a novel joint learning framework for training realistic ResNets across multiple compute devices is established by trading off the storage and recomputation of external auxiliary variables. More specifically, the input data of each independent processor is generated from its low-capacity auxiliary network (AuxNet), which permits the use of data augmentation and realizes forward unlocking. The backward passes are then executed in parallel, each with a local loss function that originates from the penalty or augmented Lagrangian (AL) methods. Finally, the proposed AuxNet is employed to reproduce the updated auxiliary variables through an end-to-end training process. We demonstrate the effectiveness of our methods on ResNets and WideResNets across CIFAR-10, CIFAR-100, and ImageNet datasets, achieving speedup over the traditional layer-serial training method while maintaining comparable testing accuracy.