Search

This paper proposes a simple yet effective way of regularising the encoder-decoder-based automatic speech recognition (ASR) models that enhance the robustness of the model and improve the generalisation to out-of-domain scenarios. The proposed approach is dubbed as $\textbf{De}$coder-$\textbf{C}$entric $\textbf{R}$egularisation in $\textbf{E}$ncoder-$\textbf{D}$ecoder (DeCRED) architecture for ASR, where auxiliary classifier(s) is introduced in layers of the decoder module. Leveraging these classifiers, we propose two decoding strategies that re-estimate the next token probabilities. Using the recent E-branchformer architecture, we build strong ASR systems that obtained competitive WERs as compared to Whisper-medium and outperformed OWSM v3; while relying only on a fraction of training data and model size. On top of such a strong baseline, we show that DeCRED can further improve the results and, moreover, generalise much better to out-of-domain scenarios, where we show an absolute reduction of 2.7 and 2.9 WERs on AMI and Gigaspeech datasets, respectively. We provide extensive analysis and accompanying experiments that support the benefits of the proposed regularisation scheme.

This work introduces the framed curvature flow, a generalization of both the curve shortening flow and the vortex filament equation. Here, the magnitude of the velocity vector is still determined by the curvature, but its direction is given by an associated time-dependent moving frame. After establishing local existence and global estimates, we analyze the trajectory surfaces generated by different variations of this flow, specifically those leading to surfaces of constant mean or Gaussian curvature., Comment: 25 pages

We introduce RGB2Point, an unposed single-view RGB image to a 3D point cloud generation based on Transformer. RGB2Point takes an input image of an object and generates a dense 3D point cloud. Contrary to prior works based on CNN layers and diffusion denoising approaches, we use pre-trained Transformer layers that are fast and generate high-quality point clouds with consistent quality over available categories. Our generated point clouds demonstrate high quality on a real-world dataset, as evidenced by improved Chamfer distance (51.15%) and Earth Mover's distance (45.96%) metrics compared to the current state-of-the-art. Additionally, our approach shows a better quality on a synthetic dataset, achieving better Chamfer distance (39.26%), Earth Mover's distance (26.95%), and F-score (47.16%). Moreover, our method produces 63.1% more consistent high-quality results across various object categories compared to prior works. Furthermore, RGB2Point is computationally efficient, requiring only 2.3GB of VRAM to reconstruct a 3D point cloud from a single RGB image, and our implementation generates the results 15,133x faster than a SOTA diffusion-based model., Comment: Accepted to IEEE/CVF Winter Conference on Applications of Computer Vision (WACV) 2025

We introduce Tree D-fusion, featuring the first collection of 600,000 environmentally aware, 3D simulation-ready tree models generated through Diffusion priors. Each reconstructed 3D tree model corresponds to an image from Google's Auto Arborist Dataset, comprising street view images and associated genus labels of trees across North America. Our method distills the scores of two tree-adapted diffusion models by utilizing text prompts to specify a tree genus, thus facilitating shape reconstruction. This process involves reconstructing a 3D tree envelope filled with point markers, which are subsequently utilized to estimate the tree's branching structure using the space colonization algorithm conditioned on a specified genus., Comment: Accepted to ECCV24

We investigate the motion of closed, smooth non-self-intersecting curves that evolve in space $\mathbb{R}^3$. The geometric evolutionary equation for the evolution of the curve is accompanied by a parabolic equation for the scalar quantity evaluated over the evolving curve. We apply the direct Lagrangian approach to describe the geometric flow of 3D curves resulting in a system of degenerate parabolic equations. We prove the local existence and uniqueness of classical H\"older smooth solutions to the governing system of nonlinear parabolic equations. A numerical discretization scheme has been constructed using the method of flowing finite volumes. We present several numerical examples of the evolution of curves in 3D with a scalar quantity. In this paper, we analyze the flow of curves with no torsion evolving in rotating and parallel planes. Next, we present examples of the evolution of curves with initially knotted and unknotted curves.

We introduce LAESI, a Synthetic Leaf Dataset of 100,000 synthetic leaf images on millimeter paper, each with semantic masks and surface area labels. This dataset provides a resource for leaf morphology analysis primarily aimed at beech and oak leaves. We evaluate the applicability of the dataset by training machine learning models for leaf surface area prediction and semantic segmentation, using real images for validation. Our validation shows that these models can be trained to predict leaf surface area with a relative error not greater than an average human annotator. LAESI also provides an efficient framework based on 3D procedural models and generative AI for the large-scale, controllable generation of data with potential further applications in agriculture and biology. We evaluate the inclusion of generative AI in our procedural data generation pipeline and show how data filtering based on annotation consistency results in datasets which allow training the highest performing vision models., Comment: 10 pages, 12 figures, 1 table

One of the urgent issues in high energy physics is the experimental confirmation of the electroweak monopole predicted by the standard model, and currently MoEDAL at LHC is actively searching for the monopole. However, the present LHC cannot produce the monopole if the mass is bigger than 7 TeV, while the monopole mass is expected to be around $M_W/\alpha \simeq 11~\text{TeV}$. In this paper we discuss how LHC could circumbent this energy constraint and produce the monopole even when the mass is bigger than 7 TeV, based on the following ideas. First, in the topological production of the monopole the baby monopole mass at creation could be considerably smaller than the adolescent mass. Second, the binding energy of the monopole-antimonopole pair could effectively reduce the mass of the bound state. We discuss how these ideas can actually be realized at LHC to produce the monopole pairs. In particular, we argue that LHC could produce the baby electroweak monopoles whose mass could be around 5.3 TeV, smaller than the adolescent monopole mass around 11.0 TeV. Moreover, we show that LHC could produce the monopolium bound state with mass around 2.5 TeV, even when the total mass of the monopole-antimonopole pair is around 10.6 TeV. Our analysis could play an important role for MoEDAL experiment.

We report on a search for magnetic monopoles (MMs) produced in ultraperipheral Pb--Pb collisions during Run-1 of the LHC. The beam pipe surrounding the interaction region of the CMS experiment was exposed to 184.07 \textmu b$^{-1}$ of Pb--Pb collisions at 2.76 TeV center-of-mass energy per collision in December 2011, before being removed in 2013. It was scanned by the MoEDAL experiment using a SQUID magnetometer to search for trapped MMs. No MM signal was observed. The two distinctive features of this search are the use of a trapping volume very close to the collision point and ultra-high magnetic fields generated during the heavy-ion run that could produce MMs via the Schwinger effect. These two advantages allowed setting the first reliable, world-leading mass limits on MMs with high magnetic charge. In particular, the established limits are the strongest available in the range between 2 and 45 Dirac units, excluding MMs with masses of up to 80 GeV at 95\% confidence level., Comment: As accepted by PRL. 7 pages, 4 figures

The paper introduces Hands-Free VR, a voice-based natural-language interface for VR. The user gives a command using their voice, the speech audio data is converted to text using a speech-to-text deep learning model that is fine-tuned for robustness to word phonetic similarity and to spoken English accents, and the text is mapped to an executable VR command using a large language model that is robust to natural language diversity. Hands-Free VR was evaluated in a controlled within-subjects study (N = 22) that asked participants to find specific objects and to place them in various configurations. In the control condition participants used a conventional VR user interface to grab, carry, and position the objects using the handheld controllers. In the experimental condition participants used Hands-Free VR. The results confirm that: (1) Hands-Free VR is robust to spoken English accents, as for 20 of our participants English was not their first language, and to word phonetic similarity, correctly transcribing the voice command 96.71% of the time; (2) Hands-Free VR is robust to natural language diversity, correctly mapping the transcribed command to an executable command in 97.83% of the time; (3) Hands-Free VR had a significant efficiency advantage over the conventional VR interface in terms of task completion time, total viewpoint translation, total view direction rotation, and total left and right hand translations; (4) Hands-Free VR received high user preference ratings in terms of ease of use, intuitiveness, ergonomics, reliability, and desirability.

We have witnessed significant progress in deep learning-based 3D vision, ranging from neural radiance field (NeRF) based 3D representation learning to applications in novel view synthesis (NVS). However, existing scene-level datasets for deep learning-based 3D vision, limited to either synthetic environments or a narrow selection of real-world scenes, are quite insufficient. This insufficiency not only hinders a comprehensive benchmark of existing methods but also caps what could be explored in deep learning-based 3D analysis. To address this critical gap, we present DL3DV-10K, a large-scale scene dataset, featuring 51.2 million frames from 10,510 videos captured from 65 types of point-of-interest (POI) locations, covering both bounded and unbounded scenes, with different levels of reflection, transparency, and lighting. We conducted a comprehensive benchmark of recent NVS methods on DL3DV-10K, which revealed valuable insights for future research in NVS. In addition, we have obtained encouraging results in a pilot study to learn generalizable NeRF from DL3DV-10K, which manifests the necessity of a large-scale scene-level dataset to forge a path toward a foundation model for learning 3D representation. Our DL3DV-10K dataset, benchmark results, and models will be publicly accessible at https://dl3dv-10k.github.io/DL3DV-10K/.

We discuss a class of effective extensions of the $SU(2)$ Georgi-Glashow model and discuss its Bogomol'nyi-Prasad-Sommerfield (BPS) limit. We identify a specific subclass of these models that admit analytical solutions of the monopole type. We present some concrete examples and find that the resulting monopoles tend to have their energy concentrated not in their center, but rather in a spherical shell around it., Comment: 7 pages, 3 eps figures; from proceedings of the XII. International Symposium on Quantum Theory and Symmetries (QTS12), Prague, Czech Republic, July 24-28, 2023

This paper describes the joint effort of Brno University of Technology (BUT), AGH University of Krakow and University of Buenos Aires on the development of Automatic Speech Recognition systems for the CHiME-7 Challenge. We train and evaluate various end-to-end models with several toolkits. We heavily relied on Guided Source Separation (GSS) to convert multi-channel audio to single channel. The ASR is leveraging speech representations from models pre-trained by self-supervised learning, and we do a fusion of several ASR systems. In addition, we modified external data from the LibriSpeech corpus to become a close domain and added it to the training. Our efforts were focused on the far-field acoustic robustness sub-track of Task 1 - Distant Automatic Speech Recognition (DASR), our systems use oracle segmentation., Comment: 6 pages, Chime-7 challenge 2023

Bokeh is widely used in photography to draw attention to the subject while effectively isolating distractions in the background. Computational methods simulate bokeh effects without relying on a physical camera lens. However, in the realm of digital bokeh synthesis, the two main challenges for bokeh synthesis are color bleeding and partial occlusion at object boundaries. Our primary goal is to overcome these two major challenges using physics principles that define bokeh formation. To achieve this, we propose a novel and accurate filtering-based bokeh rendering equation and a physically-based occlusion-aware bokeh renderer, dubbed Dr.Bokeh, which addresses the aforementioned challenges during the rendering stage without the need of post-processing or data-driven approaches. Our rendering algorithm first preprocesses the input RGBD to obtain a layered scene representation. Dr.Bokeh then takes the layered representation and user-defined lens parameters to render photo-realistic lens blur. By softening non-differentiable operations, we make Dr.Bokeh differentiable such that it can be plugged into a machine-learning framework. We perform quantitative and qualitative evaluations on synthetic and real-world images to validate the effectiveness of the rendering quality and the differentiability of our method. We show Dr.Bokeh not only outperforms state-of-the-art bokeh rendering algorithms in terms of photo-realism but also improves the depth quality from depth-from-defocus.

End-to-end (e2e) systems have recently gained wide popularity in automatic speech recognition. However, these systems do generally not provide well-calibrated word-level confidences. In this paper, we propose Hystoc, a simple method for obtaining word-level confidences from hypothesis-level scores. Hystoc is an iterative alignment procedure which turns hypotheses from an n-best output of the ASR system into a confusion network. Eventually, word-level confidences are obtained as posterior probabilities in the individual bins of the confusion network. We show that Hystoc provides confidences that correlate well with the accuracy of the ASR hypothesis. Furthermore, we show that utilizing Hystoc in fusion of multiple e2e ASR systems increases the gains from the fusion by up to 1\,\% WER absolute on Spanish RTVE2020 dataset. Finally, we experiment with using Hystoc for direct fusion of n-best outputs from multiple systems, but we only achieve minor gains when fusing very similar systems.

In this paper, we propose DeepTree, a novel method for modeling trees based on learning developmental rules for branching structures instead of manually defining them. We call our deep neural model situated latent because its behavior is determined by the intrinsic state -- encoded as a latent space of a deep neural model -- and by the extrinsic (environmental) data that is situated as the location in the 3D space and on the tree structure. We use a neural network pipeline to train a situated latent space that allows us to locally predict branch growth only based on a single node in the branch graph of a tree model. We use this representation to progressively develop new branch nodes, thereby mimicking the growth process of trees. Starting from a root node, a tree is generated by iteratively querying the neural network on the newly added nodes resulting in the branching structure of the whole tree. Our method enables generating a wide variety of tree shapes without the need to define intricate parameters that control their growth and behavior. Furthermore, we show that the situated latents can also be used to encode the environmental response of tree models, e.g., when trees grow next to obstacles. We validate the effectiveness of our method by measuring the similarity of our tree models and by procedurally generated ones based on a number of established metrics for tree form.

Many terrain modelling methods have been proposed for the past decades, providing efficient and often interactive authoring tools. However, they generally do not include any notion of style, which is a critical aspect for designers in the entertainment industry. We introduce StyleDEM, a new generative adversarial network method for terrain synthesis and authoring, with a versatile toolbox of authoring methods with style. This method starts from an input sketch or an existing terrain. It outputs a terrain with features that can be authored using interactive brushes and enhanced with additional tools such as style manipulation or super-resolution. The strength of our approach resides in the versatility and interoperability of the toolbox.

Deep learning-based 3D object reconstruction has achieved unprecedented results. Among those, the transformer deep neural model showed outstanding performance in many applications of computer vision. We introduce SnakeVoxFormer, a novel, 3D object reconstruction in voxel space from a single image using the transformer. The input to SnakeVoxFormer is a 2D image, and the result is a 3D voxel model. The key novelty of our approach is in using the run-length encoding that traverses (like a snake) the voxel space and encodes wide spatial differences into a 1D structure that is suitable for transformer encoding. We then use dictionary encoding to convert the discovered RLE blocks into tokens that are used for the transformer. The 1D representation is a lossless 3D shape data compression method that converts to 1D data that use only about 1% of the original data size. We show how different voxel traversing strategies affect the effect of encoding and reconstruction. We compare our method with the state-of-the-art for 3D voxel reconstruction from images and our method improves the state-of-the-art methods by at least 2.8% and up to 19.8%.

We present a systematic exploration of a general family of effective $SU(2)$ models with an adjoint scalar. First, we discuss a redundancy in this class of models and use it to identify seemingly different, yet physically equivalent models. Next, we construct the Bogomol'nyi-Prasad-Sommerfield (BPS) limit and derive analytic monopole solutions. In contrast to the 't Hooft-Polyakov monopole, included here as a special case, these solutions tend to exhibit more complex energy density profiles. Typically, we obtain monopoles with a hollow cavity at their core where virtually no energy is concentrated; accordingly, most of the monopole's energy is stored in a spherical shell around its core. Moreover, the shell itself can be structured, with several "sub-shells". A recipe for the construction of these analytic solutions is presented., Comment: 13 pages, 4 figures; revised version published in PRD

We consider formal matched asymptotics to show the convergence of a degenerate area preserving surface Allen-Cahn equation to its sharp interface limit of area preserving geodesic curvature flow. The degeneracy results from a surface de Gennes-Cahn-Hilliard energy and turns out to be essential to numerically resolve the dependency of the solution on geometric properties of the surface. We experimentally demonstrate convergence of the numerical algorithm, which considers a graph formulation, adaptive finite elements and a semi-implicit discretization in time, and uses numerical solutions of the sharp interface limit, also considered in a graph formulation, as benchmark solutions., Comment: 8 pages, 2 figures

Lighting effects such as shadows or reflections are key in making synthetic images realistic and visually appealing. To generate such effects, traditional computer graphics uses a physically-based renderer along with 3D geometry. To compensate for the lack of geometry in 2D Image compositing, recent deep learning-based approaches introduced a pixel height representation to generate soft shadows and reflections. However, the lack of geometry limits the quality of the generated soft shadows and constrain reflections to pure specular ones. We introduce PixHt-Lab, a system leveraging an explicit mapping from pixel height representation to 3D space. Using this mapping, PixHt-Lab reconstructs both the cutout and background geometry and renders realistic, diverse, lighting effects for image compositing. Given a surface with physically-based materials, we can render reflections with varying glossiness. To generate more realistic soft shadows, we further propose to use 3D-aware buffer channels to guide a neural renderer. Both quantitative and qualitative evaluations demonstrate that PixHt-Lab significantly improves soft shadow generation., Comment: 11 pages, 10 figures

Space curve motion describes dynamics of material defects or interfaces, can be found in image processing or vortex dynamics. This article analyses some properties of space curves evolved by the curve shortening flow. In contrast to the classical case of shrinking planar curves, space curves do not obey the Avoidance principle in general. They can lose their convexity or develop non-circular singularities even if they are simple. In the first part of the text, we show that even though the convexity of space curves is not preserved during the motion, their orthogonal projections remain convex. In the second part, the Avoidance principle for spherical curves under the curve shortening flow in $\mathbb{R}^3$ is shown by generalizing the arguments developed by Hamilton and Gage., Comment: Published in SIAM Journal on Mathematical Analysis, Vol. 52, Iss. 2 (2020)

We study quark and lepton masses and mixings in a double covering of modular $A_4$ flavor symmetry in which we search for common solution of a single modulus $\tau$, applying chi-square numerical analysis under the as minimum framework as possible. We have found the common region of $\tau$ within $5\sigma$ interval that is narrow in case of normal hierarchy, but not found in case of inverted hierarchy., Comment: 9 pages, 2 figures, 3 tables. arXiv admin note: substantial text overlap with arXiv:2206.12629, arXiv:2202.10089

This paper explores semi-supervised training for sequence tasks, such as Optical Character Recognition or Automatic Speech Recognition. We propose a novel loss function $\unicode{x2013}$ SoftCTC $\unicode{x2013}$ which is an extension of CTC allowing to consider multiple transcription variants at the same time. This allows to omit the confidence based filtering step which is otherwise a crucial component of pseudo-labeling approaches to semi-supervised learning. We demonstrate the effectiveness of our method on a challenging handwriting recognition task and conclude that SoftCTC matches the performance of a finely-tuned filtering based pipeline. We also evaluated SoftCTC in terms of computational efficiency, concluding that it is significantly more efficient than a na\"ive CTC-based approach for training on multiple transcription variants, and we make our GPU implementation public., Comment: 21 pages, 8 figures, 6 tables, accepted to International Journal on Document Analysis and Recognition (IJDAR)

The description of distributions related to grain microstructure helps physicists to understand the processes in materials and their properties. This paper presents a general statistical methodology for the analysis of crystallographic orientations of grains in a 3D Laguerre tessellation dataset which represents the microstructure of a polycrystalline material. We introduce complex stochastic models which may substitute expensive laboratory experiments: conditional on the Laguerre tessellation, we suggest interaction models for the distribution of cubic crystal lattice orientations, where the interaction is between pairs of orientations for neighbouring grains in the tessellation. We discuss parameter estimation and model comparison methods based on maximum pseudolikelihood as well as graphical procedures for model checking using simulations. Our methodology is applied for analysing a dataset representing a nickel-titanium shape memory alloy.

We present a "primitive" way of realizing finite-mass Dirac monopoles in $U(1)$ gauge theories involving a single non-minimally interacting scalar field. Typically, the energy density of this type of monopole is not concentrated at its core, but it is distributed in a spherical shell, as we illustrate on several exact solutions in the Bogomol'nyi-Prasad-Sommerfield (BPS) limit. We show that our construction can be interpreted as a limit of infinitely massive $W$ bosons coupled to electromagnetic field-strength via a dipole moment. Combining our approach with ideas of Weinberg and Lee, we present a general landscape of $U(1)$ gauge models that support a finite-mass Dirac monopole. In fact, all classical monopoles, i.e., Wu-Yang, 't Hooft-Polyakov, Cho-Maison, etc., are special points on this landscape., Comment: 5 pages, 3 figures, revised version published in PRD

In the Amazon KDD Cup 2022, we aim to apply natural language processing methods to improve the quality of search results that can significantly enhance user experience and engagement with search engines for e-commerce. We discuss our practical solution for this competition, ranking 6th in task one, 2nd in task two, and 2nd in task 3. The code is available at https://github.com/wufanyou/KDD-Cup-2022-Amazon., Comment: Submitted to Amazon KDD Cup 2022 workshop

Shadows are essential for realistic image compositing. Physics-based shadow rendering methods require 3D geometries, which are not always available. Deep learning-based shadow synthesis methods learn a mapping from the light information to an object's shadow without explicitly modeling the shadow geometry. Still, they lack control and are prone to visual artifacts. We introduce pixel heigh, a novel geometry representation that encodes the correlations between objects, ground, and camera pose. The pixel height can be calculated from 3D geometries, manually annotated on 2D images, and can also be predicted from a single-view RGB image by a supervised approach. It can be used to calculate hard shadows in a 2D image based on the projective geometry, providing precise control of the shadows' direction and shape. Furthermore, we propose a data-driven soft shadow generator to apply softness to a hard shadow based on a softness input parameter. Qualitative and quantitative evaluations demonstrate that the proposed pixel height significantly improves the quality of the shadow generation while allowing for controllability., Comment: 15 pages, 11 figures