Search

Cross-modality understanding is essential for AI to tackle complex tasks that require both deterministic and generative capabilities, such as correlating music and visual art. The existing state-of-the-art methods of audio-visual correlation often rely on single-dimension information, focusing either on semantic or emotional attributes, thus failing to capture the full depth of these inherently complex modalities. Addressing this limitation, we introduce a novel approach that perceives music–image correlation as multilayered rather than as a direct one-to-one correspondence. To this end, we present a pioneering dataset with two segments: an artistic segment that pairs music with art based on both emotional and semantic attributes, and a realistic segment that links music with images through affective–semantic layers. In modeling emotional layers for the artistic segment, we found traditional 2D affective models inadequate, prompting us to propose a more interpretable hybrid-emotional rating system that serves both experts and non-experts. For the realistic segment, we utilize a web-based dataset with tags, dividing tag information into semantic and affective components to ensure a balanced and nuanced representation of music–image correlation. We conducted an in-depth statistical analysis and user study to evaluate our dataset’s effectiveness and applicability for AI-driven understanding. This work provides a foundation for advanced explorations into the complex relationships between auditory and visual art modalities, advancing the development of more sophisticated cross-modal AI systems.

For ultra-high-speed digital transmission, required by 5G/6G communications, ultra-wideband common-mode rejection (CMR) structures with autonomous phase-balancing capability are proposed. Common-mode noise, caused by phase and amplitude unbalances, is one of the most undesired disturbances affecting modern digital circuits. According to the circuit design guides with a typically used differential line (DL) for high-speed digital transmission, common-mode rejection is achieved using CMR filters, and the unbalanced phase, caused by a length difference between the two signal lines of a DL, is compensated by inserting an additional delay line. However, due to nonlinear phase interactions between the two DLs and unbalanced electromagnetic (EM) interferences, the conventional compensation method is frequency-limited at around 10 GHz. To significantly enhance the common-mode rejection level and extend the phase recovery bandwidth, the proposed CMR structure utilizes a planar balanced line (BL), such as a coplanar stripline (CPS) or a parallel stripline (PSL), along with additional conductor strips arranged laterally near the BL. To demonstrate the performance of the proposed BL-based CMR structures, various types of CMR structures are fabricated, and the measurement results are compared with the 3D EM simulation results. As a result, it is proven that the proposed BL-based CMR structures have the capability to reject the common-mode noise with suppression levels of more than 10 dB and to simultaneously recover the phase balance from near DC to over 40 GHz.

Compositional scripts like Hangeul (Korean characters) and Chinese characters involve numerous characters, making manual font design labor-intensive and cost-ineffective work. Although many few-shot font generation methods have been introduced, they have at least one of the limitations, i.e., lacking local styles of font, additional component labeling, and high complexity in network structure and training. To solve these limitations, given our observation that font style can be perceived at a patch-level rather than a component-level, we propose Region-Aware Contrastive loss (RAC-loss) so that the generator can capture the local style by self-supervision. The proposed loss maximizes the style information between patches of the generated image and the style reference image. And we introduce an attention mechanism to the patch-level contrastive loss to handle multiple patch correspondences. This attention learns style similarity between two glyph images, which serves as a patch-correspondence map. RAC-loss gives more fine-grained feedback to the generator than component-level loss, allowing it to incorporate local styles, even in a straightforward structure like a visual geometry group network (VGGNet). This results in a fast inference latency (3.02 ms), and the proposed method achieved 43.18 mean Fréchet Inception Distance (mFID) on the test dataset, a notable decrease of 5.42 compared to the previous method.

A design method for an ultra-wideband coplanar-stripline-based vertical transition that can be used for ultra-high-speed digital interfaces is proposed. A conventional via structure, based on a differential line (DL), inherently possesses performance limitations (

An efficient design method for a compact and ultra-wideband multi-stage Wilkinson power divider in a parallel stripline (PSL) is proposed. To enhance the frequency bandwidth of the proposed power divider while reducing its size, the isolation branch is modified; that is, two capacitors are connected to both sides of a resistor at each isolation branch. For an efficient design process, the PSL power divider is equivalently represented by two microstrip power dividers, and the design equations are derived. Based on the design equations, an in-house algorithm is utilized to optimally determine the design parameters, including the line impedance, resistance, and capacitance of each stage. For example, a three-stage PSL power divider is designed with three λ/4 transmission lines at a base frequency of 5 GHz. To verify the accuracy of the design procedure, 3D EM simulations and measurements are performed, and the results show good agreement. Compared with the conventional three-stage Wilkinson power divider, the proposed PSL power divider achieves a wider frequency bandwidth of 1.16 to 6.51 GHz (139.5%) and a 23% shorter transmission line length of 207°, while exhibiting an insertion loss of 0.7 to 1.4 dB.

Early feed-forward neural methods of arbitrary image style transfer utilized the encoded feature map up to its second-order statistics, i.e., mean and variance (or covariance) of the encoded feature map. Recent methods have begun to utilize feature statistics beyond the 2nd-order statistics, e.g., a higher-order moment, slicing-based method, and histogram matching for better style representation. However, even in the distribution matching of the latest method, considering the channel-correlation and the order of duplicated values in the feature map is missing. This resulted in insufficient style transfer or artifacts on style-transferred images e.g., unexpected color or pattern co-occurrence, skewed color or style pattern into one side, noise between background and foreground, etc. In this work, we show how to make a feature transform layer to consider both channel-correlation and duplicated values in the feature map for exact feature distribution matching in style transfer. Our experimental results prove that the stylized images obtained with our method are more similar to the target style images qualitatively without losing content clearness and superior to the existing methods quantitatively in all possible style measures, e.g., 0.0311 lower sliced Wasserstein distance than the previous method and the most favored choice with 25.54% preference.

Herein, the impact of cross‐temperature on 3D NAND flash memory is modeled by considering adjacent cells using machine learning. The cells comprising NAND flash memory exhibit diverse states and connectivity patterns. To effectively capture this complexity, the 3D NAND flash is converted to graph structure and the graph neural network (GNN) is leveraged, known for its exceptional performance in handling graph data. To the best of the authors' knowledge, this is the first attempt to model 3D NAND flash memory using GNN. This method has good generalization performance across various retention times and temperatures, achieving a remarkable overlap of 95.28% between ground truth and predicted distributions. Moreover, two applications of this method are introduced that contribute to the NAND flash memory improvement. One is a GNN‐assist program, which leverages well‐trained GNN to suppress the Vth degradation affected by cross‐temperature, resulting in reduced Vth shift and narrower Vth distribution width. The other is the sensitivity decomposition to identify parameters influencing the cell at cross‐temperature. It is found that cross‐temperature impact extends beyond physically connected cells to adjacent cells at close distances. Overall, this work provides valuable insights into modeling 3D NAND flash memory using GNNs and offers practical methods for enhancing NAND flash memory reliability.

A machine learning (ML) method was used to optimize the trap distribution of the charge trap nitride (CTN) to simultaneously improve its performance/reliability (P/R) characteristics, which are tradeoffs in 3-D NAND flash memories. Using an artificial neural network (ANN), we modeled the relationship between trap distributions and P/R characteristics. The ANN was trained using a large experimentally-calibrated technology computer-aided design (TCAD) simulation dataset. The gradient descent method was adapted to optimize the trap distribution, achieving the best P/R characteristics based on the well-trained ANN. Eventually, we found the best trap profile distributed in both space and energy. In particular, the energetic trap distribution had a larger impact on the P/R characteristics than that of the spatial trap distribution. Furthermore, in terms of the P/R characteristics, it was generally preferable to increase all inputs of the energetic trap distribution. However, the acceptor-like trap energy level ( $E_{TA}$ ) and its standard deviation ( $\sigma _{EA}$ ) caused a tradeoff between P/R characteristics; therefore, ML was used to determine their optimal points. The proposed ML method allows the optimization of trap distribution to obtain the best P/R characteristics rapidly and quantitatively. Our findings could be used as a guideline for determining the physical properties of CTN in 3-D NAND flash cells.

Herein, novel neural network (NN) methods that improve prediction accuracy and reduce output variance of the optimized input in the gradient method for cross‐sectional data are proposed, and the variability evaluation approach of optimized inputs in the semiconductor process is suggested. Specifically, electrical parameter measurements (EPMs) and power‐delay product of industrial high‐k metal gate DRAM peripheral 29‐stage ring oscillator circuits, including NMOS, PMOS, and interconnects, are focused on. The proposed methods find an optimized input to achieve a lower NN output variance in the gradient descent than one multilayer perceptron (MLP) and mean ensemble of MLPs even when considering the variabilities of the devices and interconnects. The local optima problem of one MLP is resolved by utilizing multiple MLPs trained with different train/validation data, their trimmed mean, and an additional learnable layer. Moreover, adding the learnable layer secures versatility for various parametric datasets. The methods improve the prediction accuracy (R2) by 5.6–15.6% in sparse data space compared to one MLP and the mean ensemble, decrease the NN output variance of the optimized input by 73.0–81.6% compared to one MLP and the mean ensemble, and are successfully verified by implementing it on EPMs of 3977 test patterns of 314 wafers and 16 lots.

We proposed a neural network (NN) approach that uses two multi-layer perceptron (MLP) NNs an encoder and a decoder to estimate the structural parameter (Spara) of a 14-nm node fully depleted silicon on insulator (FDSOI) field-effect transistor (FET). When outputs defined by the same input exist, the proposed NN algorithm achieves loss function convergence during NN training. The decoder takes inputs of on/off current ratio, delay, and power to represent DC/AC performance for high performance (HP), low operating power (LOP), and low standby power (LSTP) applications. With the pre-trained encoder learned with R coefficients of the regression plot over 0.99 and an average percent error of approximately 1%, the decoder was modeled to estimate the Spara. Our decoder successfully estimated all Spara within the range that satisfies the technology node. The tendency of Spara satisfying the desired figure-of-merits (FOMs) in device design can be confirmed by comparing the estimated Spara of the upper 5 % and 10 % cases. Furthermore, it can provide device design guidance from various perspectives by presenting numerous alternatives of distinct Spara sets, even when the FOM value is the same (duplicate input values). If undesirable FOMs are extracted, it is possible to determine the causal Spara and provide immediate process feedback on the related unit process using the Spara estimated from the lower 5 % of FOMs. We performed a detailed physical analysis as an example of a delay in LOP application. NN estimation results were analyzed using gate length ( $L_{g}$ ), SOI thickness ( $T_{soi}$ ), and drain-side spacer length ( $L_{spd}$ ), which mainly affect gate capacitance ( $C_{g}$ ) and effective current ( $I_{eff}$ ). In addition, source-side spacer length ( $L_{sps}$ ) and source/drain junction gradient ( $L_{sdj}$ ) showed behaviors different from those generally selected by human experts and cases where maximal values were not estimated within the set range. The estimation of Spara using the NN was effective and powerful, reducing process cost and feedback time.

Non-maximum suppression (NMS) is an unavoidable post-processing step in the object detection pipeline. NMS selects the bounding boxes with a locally maximum confidence score and eliminates its neighboring candidates which have a large overlap with the selected boxes. Because this procedure is a sequential and iterative algorithm of $O(N^{2})$ complexity, NMS running time is too slow to be applied to real-time object detection on the image which has many objects. To consider this issue, we propose a parallel computation method using GPU multi-cores to compute faster than the previous NMS. Our parallel NMS replicates the candidate boxes and performs both IoU calculation and comparison in parallel. We drastically reduced the complexity from $O(N^{2})$ to $O(N)$ and the time consumption of NMS to be applied to real-time detection with negligible degradation of detection performance and very slight additional memory consumption. Furthermore, when there is a small number of overlapped objects, our parallel NMS achieved an improvement in precision.

Recent image-style transfer methods use the structure of a VGG feature network to encode and decode the feature map of the image. Since the network is designed for the general image-classification task, it has a number of channels and, accordingly, requires a huge amount of memory and high computational power, which is not mandatory for such a relatively simple task as image-style transfer. In this paper, we propose a new technique to size down the previously used style transfer network for eliminating the redundancy of the VGG feature network in memory consumption and computational cost. Our method automatically finds a number of consistently inactive convolution channels during the network training phase by using two new losses, i.e., channel loss and xor loss. The former maximizes the number of inactive channels and the latter fixes the positions of these inactive channels to be the same for the image. Our method improves the image generation speed to be up to 49% faster and reduces the number of parameters by 20% while maintaining style transferring performance. Additionally, our losses are also effective in pruning the VGG16 classifier network, i.e., parameter reduction by 26% and top-1 accuracy improvement by 0.16% on CIFAR-10.

With the prevalence of degenerative diseases due to the increase in the aging population, we have encountered many spine-related disorders. Since the spine is a crucial part of the body, fast and accurate diagnosis is critically important. Generally, clinicians use X-ray images to diagnose the spine, but X-ray images are commonly occluded by the shadows of some bones, making it hard to identify the whole spine. Therefore, recently, various deep-learning-based spinal X-ray image analysis approaches have been proposed to help diagnose the spine. However, these approaches did not consider the characteristics of frequent occlusion in the X-ray image and the properties of the vertebra shape. Therefore, based on the X-ray image properties and vertebra shape, we present a novel landmark detection network specialized in lumbar X-ray images. The proposed network consists of two stages: The first step detects the centers of the lumbar vertebrae and the upper end plate of the first sacral vertebra (S1), and the second step detects the four corner points of each lumbar vertebra and two corner points of S1 from the image obtained in the first step. We used random spine cutout augmentation in the first step to robustify the network against the commonly obscured X-ray images. Furthermore, in the second step, we used CoordConv to make the network recognize the location distribution of landmarks and part affinity fields to understand the morphological features of the vertebrae, resulting in more accurate landmark detection. The proposed network was evaluated using 304 X-ray images, and it achieved 98.02% accuracy in center detection and 8.34% relative distance error in corner detection. This indicates that our network can detect spinal landmarks reliably enough to support radiologists in analyzing the lumbar X-ray images.

Style transfer is an application that applies colors and patterns of a style image to a content image. In the previous style transfer method, a decoder was trained by using only positive valued feature maps from the last rectified linear unit (ReLU) layer of an encoder network. Then, the trained decoder was used to generate a stylized image from a transformed feature map, which may have unseen negative values due to the transformer operation, and this resulted in odd colors and patterns on the output stylized image. In this paper, we propose a simple but effective technique, called the feature map clamping method, which eliminates negative values from a transformed feature map to resolve style degradation in output stylized image. Our experiments that compare the output style qualities of the previous and our methods verified that our method removed odd colors and patterns appearing in the previous method and that our method had 12.07 % lower averaged feature loss and 7.22 % higher averaged user preference than the previous method without losing computational efficiency.

In this article, by using neural network, we proposed a method to optimize Fully-Depleted (FD) Silicon-on-Insulator (SOI) Field-Effect-Transistor (FET) structures to maximize the on/off current ratio for 14-nm node (70-nm Gate Pitch) System-on-Chip (SoC) and sequential 3-dimensional integrated circuit (3DIC). Using machine learning method, the neural network accurately predicted the electrical behaviors of 14-nm node FDSOI FETs. Also by using backpropagation and gradient descent method, the device structures were modified to improve on/off current ratios for high performance (HP), low operating power (LOP), and low stand-by power (LSTP) applications. These optimized structures were secured within the process range of conventional FDSOI FETs. Among the optimized parameters, drain-side spacer length (Lspd), source/drain junction gradient (Lsdj), and thickness of source/drain epi (Tsd) showed different behaviors for each application and thickness of buried oxide (Tbox) was maximal in optimization results. The detailed physical analysis was conducted to evaluate these parameters for each application. The neural network based optimization was powerful and efficient while saving time and cost in device design.

An optimal design of semiconductor device and its process uniformity are critical factors affecting desired figure-of-merits as well as reducing fabrication cost of fixing possible malfunctioning in semiconductor manufacturing. Two main tasks in optimal device design for semiconductor manufacturing, i.e., parameter optimization and modeling, have been typically used either to characterize the devices by understanding how each parameter affects the device performances or to calibrate the parameters for SPICE circuit simulation. However, there still remains limitations in describing the relationship between all manufacturing parameters and figure-of-merits using several simple equations human experts can utilize. Even with the best model currently available, the optimal design of semiconductor device heavily relies on experiences of human experts and deals with time-consuming ad-hoc trials and non-holistic approaches. In this paper, we propose a new approach for data-based accurate electrical modeling of transistor, which is the most fundamental unit device of semiconductor, and fast optimization of its manufacturing parameters. Instead of the previous analytic approaches, finding finite equations derived from semiconductor physics, we utilize machine learning technique and neural networks to find appropriate modeling functions from data pairs of parameters and figure-of-merits. And for given desired figure-of-merits, we find optimal manufacturing parameters in holistic manner by using the learned functions of neural networks and fast gradient-based optimization method. Experimental results show that our neural-network-based-model directly estimate figure-of-merits with competitive accuracy and that our holistic optimization technique accurately and rapidly adapts the manufacturing parameters to meet desired figure-of-merits.

Image style transferring process generates an output image in the target style with a specific strength for a given pair of content and target image. Recently, feed-forward neural networks have been employed in this process to fastly decode a linearly interpolated feature in encoded feature space. However, to date, no studies have been conducted to analyze the effectiveness of this style interpolation method. In this article, we tackle the missing work of the in-depth analysis of style interpolation and propose a new method that is more effective in controlling the strength of the desired style. The existing methods are biased because the training of a network is performed with one-sided data of full style strength. Therefore, such methods do not guarantee the generation of a satisfactory output image in an intermediate style strength. To resolve this problem of a biased network, we propose an unbiased learning technique, which uses unbiased training data and loss to allow a feed-forward network to learn the desired regression of style consistent with a specific interpolation function in encoded feature space. The experimental results verified that our unbiased method achieved a better regression learning between style control parameter and output image style, and more stable style transfer that is insensitive to the weight of style loss without adding complexity in image generating process.

For decades, co-relating different data domains to attain the maximum potential of machines has driven research, especially in neural networks. Similarly, text and visual data (images and videos) are two distinct data domains with extensive research in the past. Recently, using natural language to process 2D or 3D images and videos with the immense power of neural nets has witnessed a promising future. Despite the diverse range of remarkable work in this field, notably in the past few years, rapid improvements have also solved future challenges for researchers. Moreover, the connection between these two domains is mainly subjected to GAN, thus limiting the horizons of this field. This review analyzes Text-to-Image (T2I) synthesis as a broader picture, Text-guided Visual-output (T2Vo), with the primary goal being to highlight the gaps by proposing a more comprehensive taxonomy. We broadly categorize text-guided visual output into three main divisions and meaningful subdivisions by critically examining an extensive body of literature from top-tier computer vision venues and closely related fields, such as machine learning and human–computer interaction, aiming at state-of-the-art models with a comparative analysis. This study successively follows previous surveys on T2I, adding value by analogously evaluating the diverse range of existing methods, including different generative models, several types of visual output, critical examination of various approaches, and highlighting the shortcomings, suggesting the future direction of research.

The development of recent image style transfer methods allows the quick transformation of an input content image into an arbitrary style. However, these methods have a limitation that the scale-across style pattern of a style image cannot be fully transferred into a content image. In this paper, we propose a new style transfer method, named total style transfer, that resolves this limitation by utilizing intra/inter-scale statistics of multi-scaled feature maps without losing the merits of the existing methods. First, we use a more general feature transform layer that employs intra/inter-scale statistics of multi-scaled feature maps and transforms the multi-scaled style of a content image into that of a style image. Secondly, we generate a multi-scaled stylized image by using only a single decoder network with skip-connections, in which multi-scaled features are merged. Finally, we optimize the style loss for the decoder network in the intra/inter-scale statistics of image style. Our improved total style transfer can generate a stylized image with a scale-across style pattern from a pair of content and style images in one forwarding pass. Our method achieved less memory consumption and faster feed-forwarding speed compared with the recent cascade scheme and the lowest style loss among the recent style transfer methods.

A machine-learning (ML) technique was used to optimize the energetic-trap distributions of nano-scaled charge trap nitride (CTN) in 3D NAND Flash to widen the threshold voltage (Vth) window, which is crucial for NAND operation. The energetic-trap distribution is a critical material property of the CTN that affects the Vth window between the erase and program Vth. An artificial neural network (ANN) was used to model the relationship between the energetic-trap distributions as an input parameter and the Vth window as an output parameter. A well-trained ANN was used with the gradient-descent method to determine the specific inputs that maximize the outputs. The trap densities (NTD and NTA) and their standard deviations (σTD and σTA) were found to most strongly impact the Vth window. As they increased, the Vth window increased because of the availability of a larger number of trap sites. Finally, when the ML-optimized energetic-trap distributions were simulated, the Vth window increased by 49% compared with the experimental value under the same bias condition. Therefore, the developed ML technique can be applied to optimize cell transistor processes by determining the material properties of the CTN in 3D NAND Flash.

Making a new font requires graphical designs for all base characters, and this designing process consumes lots of time and human resources. Especially for languages including a large number of combinations of consonants and vowels, it is a heavy burden to design all such combinations independently. Automatic font generation methods have been proposed to reduce this labor-intensive design problem. Most of the methods are GAN-based approaches, and they are limited to generate the trained fonts. In some previous methods, they used two encoders, one for content, the other for style, but their disentanglement of content and style is not sufficiently effective in generating arbitrary fonts. Arbitrary font generation is a challenging task because learning text and font design separately from given font images is very difficult, where the font images have both text content and font style in each image. In this paper, we propose a new automatic font generation method to solve this disentanglement problem. First, we use two stacked inputs, i.e., images with the same text but different font style as content input and images with the same font style but different text as style input. Second, we propose new consistency losses that force any combination of encoded features of the stacked inputs to have the same values. In our experiments, we proved that our method can extract consistent features of text contents and font styles by separating content and style encoders and this works well for generating unseen font design from a small number of reference font images that are human-designed. Comparing to the previous methods, the font designs generated with our method showed better quality both qualitatively and quantitatively than those with the previous methods for Korean, Chinese, and English characters. e.g., 17.84 lower FID in unseen font compared to other methods.

Generating images of artistic style from input images, also known as image style transfer, has been improved in the quality of output style and the speed of image generation since deep neural networks have been applied in the field of computer vision research. However, the previous approaches used feature alignment techniques that were too simple in their transform layer to cover the characteristics of style features of images. In addition, they used an inconsistent combination of transform layers and loss functions in the training phase to embed arbitrary styles in a decoder network. To overcome these shortcomings, the second-order statistics of the encoded features are exploited to build an optimal arbitrary image style transfer technique. First, a new correlation-aware loss and a correlation-aware feature alignment technique are proposed. Using this consistent combination of loss and feature alignment methods strongly matches the second-order statistics of content features to those of the target-style features and, accordingly, the style capacity of the decoder network is increased. Secondly, a new component-wise style controlling method is proposed. This method can generate various styles from one or several style images by using style-specific components from second-order feature statistics. We experimentally prove that the proposed method achieves improvements in both the style capacity of the decoder network and the style variety without losing the ability of real-time processing (less than 200 ms) on Graphics Processing Unit (GPU) devices.

A broadband and high gain tapered slot antenna (TSA) by utilizing a broadband microstrip- (MS-) to-coplanar stripline (CPS) balun has been developed for millimeter-wave imaging systems and sensors. This antenna exhibits ultrawideband performance for frequency ranges from 70 to over 110 GHz with the high antenna gain, low sidelobe levels, and narrow beamwidth. The validity of this antenna as imaging arrays is also demonstrated by analyzing mutual couplings and 4-element linear array. This antenna can be applied to mm-wave phased array, imaging array for plasma diagnostics applications.