Search

Recently, segmentation-based methods are quite popular in scene text detection, as the segmentation results can more accurately describe scene text of various shapes such as curve text. However, the post-processing of binarization is essential for segmentation-based detection, which converts probability maps produced by a segmentation method into bounding boxes/regions of text. In this paper, we propose a module named Differentiable Binarization (DB), which can perform the binarization process in a segmentation network. Optimized along with a DB module, a segmentation network can adaptively set the thresholds for binarization, which not only simplifies the post-processing but also enhances the performance of text detection. Based on a simple segmentation network, we validate the performance improvements of DB on five benchmark datasets, which consistently achieves state-of-the-art results, in terms of both detection accuracy and speed. In particular, with a light-weight backbone, the performance improvements by DB are significant so that we can look for an ideal tradeoff between detection accuracy and efficiency. Specifically, with a backbone of ResNet-18, our detector achieves an F-measure of 82.8, running at 62 FPS, on the MSRA-TD500 dataset. Code is available at: https://github.com/MhLiao/DB, Accepted to AAAI 2020

Existing text recognition methods usually need large-scale training data. Most of them rely on synthetic training data due to the lack of annotated real images. However, there is a domain gap between the synthetic data and real data, which limits the performance of the text recognition models. Recent self-supervised text recognition methods attempted to utilize unlabeled real images by introducing contrastive learning, which mainly learns the discrimination of the text images. Inspired by the observation that humans learn to recognize the texts through both reading and writing, we propose to learn discrimination and generation by integrating contrastive learning and masked image modeling in our self-supervised method. The contrastive learning branch is adopted to learn the discrimination of text images, which imitates the reading behavior of humans. Meanwhile, masked image modeling is firstly introduced for text recognition to learn the context generation of the text images, which is similar to the writing behavior. The experimental results show that our method outperforms previous self-supervised text recognition methods by 10.2%-20.2% on irregular scene text recognition datasets. Moreover, our proposed text recognizer exceeds previous state-of-the-art text recognition methods by averagely 5.3% on 11 benchmarks, with similar model size. We also demonstrate that our pre-trained model can be easily applied to other text-related tasks with obvious performance gain. The code is available at https://github.com/ayumiymk/DiG., Comment: Accepted by ACM MM 2022. The code is available at https://github.com/ayumiymk/DiG

The raster scanning imaging mode is widely used in scanning electron microscopes (SEMs), transmission electron microscopes (TEM), and atomic force microscopes (AFM), and can achieve subatomic resolution. However, only a point on the shallow surface of an object can be imaged at one time using the raster scanning imaging mode, whereas the entire surface of the object can be imaged in the image plane once and instantaneously using the optical imaging mode, which is a parallel imaging mode. Therefore, the image distortion and blur for the scanning imaging mode are different from the optical imaging. In this paper, we propose a theory to describe the mechanism of the scanning imaging process and restore the degraded image (distorted and blurred image) obtained using an SEM. The theory consists of a scanning equation, motion or deformation equations, and an assumption called the intensity-invariant hypothesis. Numerical simulations of the scanning imaging process and restoration of the degraded images are performed using the scanning imaging formulas, spatial non-uniform point spread function, and inverse restoration algorithms, including algebraic, interpolation, and their hybrid methods to verify the feasibility of our theory. In situ experiments on uniform linear motion, uniaxial tensile, and fatigue were also conducted to demonstrate the validity and efficiency of the proposed scanning imaging theory and restoration methods. We anticipate that this imaging and restoration theory will enable the scanning imaging mode to be used in in situ dynamic imaging and for mechanical property measurement of materials.

Over the past few years, the field of scene text detection has progressed rapidly that modern text detectors are able to hunt text in various challenging scenarios. However, they might still fall short when handling text instances of extreme aspect ratios and varying scales. To tackle such difficulties, we propose in this paper a new algorithm for scene text detection, which puts forward a set of strategies to significantly improve the quality of text localization. Specifically, a Text Feature Alignment Module (TFAM) is proposed to dynamically adjust the receptive fields of features based on initial raw detections; a Position-Aware Non-Maximum Suppression (PA-NMS) module is devised to selectively concentrate on reliable raw detections and exclude unreliable ones; besides, we propose an Instance-wise IoU loss for balanced training to deal with text instances of different scales. An extensive ablation study demonstrates the effectiveness and superiority of the proposed strategies. The resulting text detection system, which integrates the proposed strategies with a leading scene text detector EAST, achieves state-of-the-art or competitive performance on various standard benchmarks for text detection while keeping a fast running speed., Comment: Accepted by CVPR21

BackgroundThis study aimed to evaluate the quality and associated factors that potentially influence complementary feeding practices among Chinese caregivers living in Hunan, a rapidly developing province in China. MethodsThree indicators( minimum dietary diversity(MDD), minimum meal frequency(MMF) and minimum acceptable diet(MAD)) were used to evaluate the rationality of complementary feeding. A multi-factor unconditional logistic regression model was employed to analyzeanalyse the relevant factors of complementary feeding. ResultsThe eligibility rates of MDD, MMF and MAD for children aged 6-23 months were 62.3%, 75.1% and 44.0%, respectively. The ineligible OR values of MDD for children aged 12-17 and 18-23 months were 0.238 and 0.103, respectively, and those of MAD were 0.425 and 0.376, respectively, compared with those for children aged 6-11 months. Compared children whose mothers moved out for work, the ineligible MMF for children whose mothers were not migrant workers was 2.069. Compared with parents who scored less than 60 in knowledge and attitude towards nutrition and health, the ineligible OR values of MDD, MMF and MAD for children whose parents obtained 60-79 were 0.653, 0.689 and 0.668, respectively, and those for children whose parents obtained 80-100 were 0.543, 0.428 and 0.496, respectively (all P values

Scene text detection and recognition is a universal text recognition technology, which has become a hot research topic in the field of computer vision and document analysis in recent years. It is widely applied in geographical positioning, license plate recognition, and driverless applications. Compared to traditional document text detection and recognition, scene text varies more dramatically in font, color, scale, layout, and background. Owing to its excellent performance, deep learning has been widely adopted in this field. In this paper, we mainly review our representative studies based on deep learning in this field and describe the future research trends in this field.

Background In recent ten years, with the rapid socioeconomic development and the extensive implementation of children nutrition improvement projects, the previous epidemiological data cannot reflect the actual level of anemia among children in China, especially in rural areas. Therefore, this study analyzed the prevalence, severity and associated determinant factors of anemia among children aged 6-71 months in rural Hunan Province. Methods A community-based cross-sectional study was conducted. Through multistage stratified cluster sampling, 5229 children aged 6 to 71 months and their caregivers were randomly selected from 72 villages across 24 towns in 12 counties from rural Hunan. The demographic characteristics of children and their caregivers, feeding practice, caregivers’ anemia-related feeding knowledge, and gestational conditions of mothers were acquired by using a unified questionnaire. Peripheral blood from the left-hand middle fingertip was sampled from each child, and hemoglobin concentration was measured using a HemoCue301 portable hemoglobin analyzer (Sweden). Associated determinant factors analyses involving overall anemia and anemic severity were conducted on multivariate logistic regression models. Results The overall anemia prevalence was 9.7% (506/5229), and the prevalence of mild, moderate and severe anemia was 6.7% (349/5229), 2.9% (151/5229) and 0.1% (6/5229), respectively. Children age groups of 6-11 months, 12-23 months and 36-47 months, exclusive breast-feeding within 6 months after birth, and maternal moderate/severe anemia were significantly associated with an increased risk of overall anemia in children. Children age groups of 6-11 months and 12-23 months, maternal moderate/severe anemia were significantly associated with an increased risk of mild anemia in children. Children age groups of 6-11 months, 12-23 months and 36-47 months, lower caregivers’ anemia-related feeding knowledge level, and maternal moderate/severe anemia were significantly associated with an increased risk of moderate/severe anemia in children. Children who underwent regular physical examination were less likely to have moderate/severe anemia. The common protective factor for overall, mild and moderate/severe anemia in children was high family income. Conclusions The anemia status of preschool children in rural Hunan Province was a mild public health problem and associated with children age group, feeding practice, regular physical examination, family income, caregivers’ anemia-related feeding knowledge level, and maternal moderate/severe anemia.

Additional file 1. The corrected values of hemoglobin at different altitudes.

Chinese scene text reading is one of the most challenging problems in computer vision and has attracted great interest. Different from English text, Chinese has more than 6000 commonly used characters and Chinese characters can be arranged in various layouts with numerous fonts. The Chinese signboards in street view are a good choice for Chinese scene text images since they have different backgrounds, fonts and layouts. We organized a competition called ICDAR2019-ReCTS, which mainly focuses on reading Chinese text on signboard. This report presents the final results of the competition. A large-scale dataset of 25,000 annotated signboard images, in which all the text lines and characters are annotated with locations and transcriptions, were released. Four tasks, namely character recognition, text line recognition, text line detection and end-to-end recognition were set up. Besides, considering the Chinese text ambiguity issue, we proposed a multi ground truth (multi-GT) evaluation method to make evaluation fairer. The competition started on March 1, 2019 and ended on April 30, 2019. 262 submissions from 46 teams are received. Most of the participants come from universities, research institutes, and tech companies in China. There are also some participants from the United States, Australia, Singapore, and Korea. 21 teams submit results for Task 1, 23 teams submit results for Task 2, 24 teams submit results for Task 3, and 13 teams submit results for Task 4. The official website for the competition is http://rrc.cvc.uab.es/?ch=12.

Unifying text detection and text recognition in an end-to-end training fashion has become a new trend for reading text in the wild, as these two tasks are highly relevant and complementary. In this paper, we investigate the problem of scene text spotting, which aims at simultaneous text detection and recognition in natural images. An end-to-end trainable neural network named as Mask TextSpotter is presented. Different from the previous text spotters that follow the pipeline consisting of a proposal generation network and a sequence-to-sequence recognition network, Mask TextSpotter enjoys a simple and smooth end-to-end learning procedure, in which both detection and recognition can be achieved directly from two-dimensional space via semantic segmentation. Further, a spatial attention module is proposed to enhance the performance and universality. Benefiting from the proposed two-dimensional representation on both detection and recognition, it easily handles text instances of irregular shapes, for instance, curved text. We evaluate it on four English datasets and one multi-language dataset, achieving consistently superior performance over state-of-the-art methods in both detection and end-to-end text recognition tasks. Moreover, we further investigate the recognition module of our method separately, which significantly outperforms state-of-the-art methods on both regular and irregular text datasets for scene text recognition., Accepted by TPAMI. An extension of the conference version. arXiv admin note: text overlap with arXiv:1807.02242

Reading text in the wild is a very challenging task due to the diversity of text instances and the complexity of natural scenes. Recently, the community has paid increasing attention to the problem of recognizing text instances with irregular shapes. One intuitive and effective way to handle this problem is to rectify irregular text to a canonical form before recognition. However, these methods might struggle when dealing with highly curved or distorted text instances. To tackle this issue, we propose in this paper a Symmetry-constrained Rectification Network (ScRN) based on local attributes of text instances, such as center line, scale and orientation. Such constraints with an accurate description of text shape enable ScRN to generate better rectification results than existing methods and thus lead to higher recognition accuracy. Our method achieves state-of-the-art performance on text with both regular and irregular shapes. Specifically, the system outperforms existing algorithms by a large margin on datasets that contain quite a proportion of irregular text instances, e.g., ICDAR 2015, SVT-Perspective and CUTE80., Comment: The paper was accepted to ICCV2019

With the development of deep neural networks, the demand for a significant amount of annotated training data becomes the performance bottlenecks in many fields of research and applications. Image synthesis can generate annotated images automatically and freely, which gains increasing attention recently. In this paper, we propose to synthesize scene text images from the 3D virtual worlds, where the precise descriptions of scenes, editable illumination/visibility, and realistic physics are provided. Different from the previous methods which paste the rendered text on static 2D images, our method can render the 3D virtual scene and text instances as an entirety. In this way, real-world variations, including complex perspective transformations, various illuminations, and occlusions, can be realized in our synthesized scene text images. Moreover, the same text instances with various viewpoints can be produced by randomly moving and rotating the virtual camera, which acts as human eyes. The experiments on the standard scene text detection benchmarks using the generated synthetic data demonstrate the effectiveness and superiority of the proposed method. The code and synthetic data is available at: https://github.com/MhLiao/SynthText3D, Comment: Accepted by SCIS

Inspired by speech recognition, recent state-of-the-art algorithms mostly consider scene text recognition as a sequence prediction problem. Though achieving excellent performance, these methods usually neglect an important fact that text in images are actually distributed in two-dimensional space. It is a nature quite different from that of speech, which is essentially a one-dimensional signal. In principle, directly compressing features of text into a one-dimensional form may lose useful information and introduce extra noise. In this paper, we approach scene text recognition from a two-dimensional perspective. A simple yet effective model, called Character Attention Fully Convolutional Network (CA-FCN), is devised for recognizing the text of arbitrary shapes. Scene text recognition is realized with a semantic segmentation network, where an attention mechanism for characters is adopted. Combined with a word formation module, CA-FCN can simultaneously recognize the script and predict the position of each character. Experiments demonstrate that the proposed algorithm outperforms previous methods on both regular and irregular text datasets. Moreover, it is proven to be more robust to imprecise localizations in the text detection phase, which are very common in practice., To appear in AAAI 2019

Scene text detection is an important step of scene text recognition system and also a challenging problem. Different from general object detection, the main challenges of scene text detection lie on arbitrary orientations, small sizes, and significantly variant aspect ratios of text in natural images. In this paper, we present an end-to-end trainable fast scene text detector, named TextBoxes++, which detects arbitrary-oriented scene text with both high accuracy and efficiency in a single network forward pass. No post-processing other than an efficient non-maximum suppression is involved. We have evaluated the proposed TextBoxes++ on four public datasets. In all experiments, TextBoxes++ outperforms competing methods in terms of text localization accuracy and runtime. More specifically, TextBoxes++ achieves an f-measure of 0.817 at 11.6fps for 1024*1024 ICDAR 2015 Incidental text images, and an f-measure of 0.5591 at 19.8fps for 768*768 COCO-Text images. Furthermore, combined with a text recognizer, TextBoxes++ significantly outperforms the state-of-the-art approaches for word spotting and end-to-end text recognition tasks on popular benchmarks. Code is available at: https://github.com/MhLiao/TextBoxes_plusplus, Comment: 15 pages

Text in natural images is of arbitrary orientations, requiring detection in terms of oriented bounding boxes. Normally, a multi-oriented text detector often involves two key tasks: 1) text presence detection, which is a classification problem disregarding text orientation; 2) oriented bounding box regression, which concerns about text orientation. Previous methods rely on shared features for both tasks, resulting in degraded performance due to the incompatibility of the two tasks. To address this issue, we propose to perform classification and regression on features of different characteristics, extracted by two network branches of different designs. Concretely, the regression branch extracts rotation-sensitive features by actively rotating the convolutional filters, while the classification branch extracts rotation-invariant features by pooling the rotation-sensitive features. The proposed method named Rotation-sensitive Regression Detector (RRD) achieves state-of-the-art performance on three oriented scene text benchmark datasets, including ICDAR 2015, MSRA-TD500, RCTW-17 and COCO-Text. Furthermore, RRD achieves a significant improvement on a ship collection dataset, demonstrating its generality on oriented object detection., accepted by CVPR 2018

Recently, models based on deep neural networks have dominated the fields of scene text detection and recognition. In this paper, we investigate the problem of scene text spotting, which aims at simultaneous text detection and recognition in natural images. An end-to-end trainable neural network model for scene text spotting is proposed. The proposed model, named as Mask TextSpotter, is inspired by the newly published work Mask R-CNN. Different from previous methods that also accomplish text spotting with end-to-end trainable deep neural networks, Mask TextSpotter takes advantage of simple and smooth end-to-end learning procedure, in which precise text detection and recognition are acquired via semantic segmentation. Moreover, it is superior to previous methods in handling text instances of irregular shapes, for example, curved text. Experiments on ICDAR2013, ICDAR2015 and Total-Text demonstrate that the proposed method achieves state-of-the-art results in both scene text detection and end-to-end text recognition tasks.

Chinese is the most widely used language in the world. Algorithms that read Chinese text in natural images facilitate applications of various kinds. Despite the large potential value, datasets and competitions in the past primarily focus on English, which bares very different characteristics than Chinese. This report introduces RCTW, a new competition that focuses on Chinese text reading. The competition features a large-scale dataset with 12,263 annotated images. Two tasks, namely text localization and end-to-end recognition, are set up. The competition took place from January 20 to May 31, 2017. 23 valid submissions were received from 19 teams. This report includes dataset description, task definitions, evaluation protocols, and results summaries and analysis. Through this competition, we call for more future research on the Chinese text reading problem. The official website for the competition is http://rctw.vlrlab.net

Zinc finger AN1-type domain 3 (ZFAND3) is essential for spermatogenesis in mice. However, its function in teleosts remains unclear. In this study, we characterized the ZFAND3 gene (termed as OsZFAND3) in an important food fish, tilapia. The OsZFAND3 cDNA sequence is 1,050 bp in length, containing an ORF of 615 bp, which encodes a putative peptide of 204 amino acid residues. Quantitative real-time PCR revealed that the OsZFAND3 transcripts were exclusively expressed in the testis and ovary. In situ hybridization showed that the high expression of OsZFAND3 transcripts was predominantly localized in the spermatocyte and spermatid. These results suggest that OsZFAND3 is involved in male germ cell maturation. Three single nucleotide polymorphisms (SNPs) were detected in the introns of OsZFAND3. The OsZFAND3 gene was mapped in the sex-determining locus on linkage group 1 (LG1). The three SNPs in the OsZFAND3 gene were strictly associated with sex phenotype, suggesting that the OsZFAND3 gene is tightly linked to the sex-determining locus. Our study provides new insights into the functions of the OsZFAND3 gene in tilapia and a foundation for further detailed analysis of the OsZFAND3 gene in sex determination and differentiation.