Search

Abstract Point cloud completion technology is used to address incomplete three-dimensional point cloud data, predicting and reconstructing the original shape and details to achieve virtual restoration. While existing learning-based methods have made significant progress in point cloud completion, they still face challenges when dealing with noise and invisible data. To address these issues, this paper proposes a multi-layer upsampling network based on a graph attention mechanism, called GANet. GANet consists of three main components: (1) feature extraction; (2) seed point generation; (3) State Space Model-based Point Cloud Upsampling Layer. GANet demonstrates exceptional robustness in handling noise and invisible data. To validate the effectiveness of GANet, we applied it to Terracotta Warrior data. The Terracotta Warriors, as important cultural heritage, present a challenging test case due to damage and missing parts caused by prolonged burial and environmental factors. We trained and tested GANet on both the PCN dataset and Terracotta Warrior data, comparing it with several recent learning-based methods. Experimental results show that GANet can effectively reconstruct missing or damaged parts of 3D point clouds, providing more detailed and structurally accurate completion results. These completion models not only validate GANet’s effectiveness but also offer valuable references for cultural heritage restoration work.

Abstract With the development of laser scanners and machine learning, point cloud semantic segmentation plays a significant role in autonomous driving, scene reconstruction, human-computer interaction, and other fields. In recent years, point cloud semantic segmentation based on deep learning has become one of the key research directions in point cloud processing. Due to the limited ability to exploit geometric details and contextual information in point clouds, most methods that adopt encoder-decoder architecture lose local structural information easily, especially detailed features, and extract features insufficiently. To address this issue, the edge-preserving inception DenseGCN U-Net (entitled as EIDU-Net) is proposed. EIDU-Net makes full use of the complementation between geometric details in the original point cloud and high-level features. The edge-preserved graph pooling (EGP) layer, the key module of the EIDU-Net, is designed to retain additional edge feature information from the original point cloud during pooling operations. Accordingly, the edge-preserved graph unpooling (EGU) layer can restore the feature graph more efficiently based on the additionally retained edge features. Extensive experiments demonstrate that our proposed EIDU-Net has remarkable improvements on semantic segmentation tasks under whatever S3DIS or Terracotta Warrior fragments. Our code is publicly available at https://github.com/caoxin918/EIDU-Net .

Abstract Point cloud registration technology, by precisely aligning repair components with the original artifacts, can accurately record the geometric shape of cultural heritage objects and generate three-dimensional models, thereby providing reliable data support for the digital preservation, virtual exhibition, and restoration of cultural relics. However, traditional point cloud registration methods face challenges when dealing with cultural heritage data, including complex morphological and structural variations, sparsity and irregularity, and cross-dataset generalization. To address these challenges, this paper introduces an innovative method called Enhancing Point Cloud Registration with Transformer (EPCRT). Firstly, we utilize local geometric perception for positional encoding and combine it with a dynamic adjustment mechanism based on local density information and geometric angle encoding, enhancing the flexibility and adaptability of positional encoding to better characterize the complex local morphology and structural variations of artifacts. Additionally, we introduce a convolutional-Transformer hybrid module to facilitate interactive learning of artifact point cloud features, effectively achieving local–global feature fusion and enhancing detail capture capabilities, thus effectively handling the sparsity and irregularity of artifact point cloud data. We conduct extensive evaluations on the 3DMatch, ModelNet, KITTI, and MVP-RG datasets, and validate our method on the Terracotta Warriors cultural heritage dataset. The results demonstrate that our method has significant performance advantages in handling the complexity of morphological and structural variations, sparsity and irregularity of relic data, and cross-dataset generalization.

Abstract Utilizing Generative Adversarial Networks (GANs) to generate 3D representations of the Terracotta Warriors offers a novel approach for the preservation and restoration of cultural heritage. Through GAN technology, we can produce complete 3D models of the Terracotta Warriors’ faces, aiding in the repair of damaged or partially destroyed figures. This paper proposes a distillation model, DIGAN, for generating 3D Terracotta Warrior faces. By extracting knowledge from StyleGAN2, we train an innovative 3D generative network. G2D, the primary component of the generative network, produces detailed and realistic 2D images. The 3D generator modularly decomposes the generation process, covering texture, shape, lighting, and pose, ultimately rendering 2D images of the Terracotta Warriors’ faces. The model enhances the learning of 3D shapes through symmetry constraints and multi-view data, resulting in high-quality 2D images that closely resemble real faces. Experimental results demonstrate that our method outperforms existing GAN-based generation methods.

Abstract Digital modeling is an essential means for preserving and passing down historical culture within cultural heritage. Point cloud registration technology, by aligning point cloud data captured from multiple perspectives, enhances the accuracy of reconstructing the complex structures of artifacts and buildings and provides a reliable digital foundation for their protection, exhibition, and research. Due to the challenges posed by complex morphology, noise, and missing data when processing cultural heritage data, this paper proposes a point cloud registration method based on the Diffusion Transformer (PointDT). Compared to traditional methods, the Diffusion Transformer can better capture both the global features and local structures of point cloud data, more accurately capturing the geometric and semantic information of the target point cloud, thereby achieving precise digital reconstruction. In this study, we trained our method using indoor datasets such as 3DMatch and large-scale outdoor datasets like KITTI, and validated it on various cultural heritage datasets, including those of the Terracotta Warriors and heritage buildings. The results demonstrate that this method not only significantly improves accuracy but also shows advantages in computational efficiency.

Abstract The image classification task of porcelain fragments is of great significance for the digital preservation of cultural heritage. However, common issues are encountered in the image processing of porcelain fragments, including the low computation speed, decreased accuracy due to the uneven distribution of sample categories, and model instability. This study proposes a novel Capsule Network model, referred to as LBCapsNet, which is suitable for the extraction of features from images of porcelain artifacts fragments. A bottleneck-like channel transformation module denoted by ChannelTrans, which resides between the convolutional layer and the PrimaryCaps layer, was first designed. This module is used to reduce the computational complexity and enhance the processing speed when dealing with intricate porcelain images. The MF-R loss function was then proposed by incorporating focal loss into the original loss function. This allows to address the issue of imbalanced distribution of ceramic shard samples and reduce the classification errors, which leads to faster convergence with smoother trend. Finally, an adaptive dynamic routing mechanism is designed with a dynamic learning rate to enhance the overall stability of the classification process. The experimental results obtained on public datasets, such as MNIST, Fashion- MNIST, CIFAR10, FMD and DTD as well as porcelain fragments dataset, demonstrate that LBCapsNet achieves high classification accuracy with faster and more stable computation compared with existing methods. Furthermore, the ability of LBCapsNet to process special textures can provide technical support for the digital preservation and restoration of cultural heritage.

Abstract Due to the antiquity and difficulty of excavation, the Terracotta Warriors have suffered varying degrees of damage. To restore the cultural relics to their original appearance, utilizing point clouds to repair damaged Terracotta Warriors has always been a hot topic in cultural relic protection. The output results of existing methods in point cloud completion often lack diversity. Probability-based models represented by Denoising Diffusion Probabilistic Models have recently achieved great success in the field of images and point clouds and can output a variety of results. However, one drawback of diffusion models is that too many samples result in slow generation speed. Toward this issue, we propose a new neural network for Terracotta Warriors fragments completion. During the reverse diffusion stage, we initially decrease the number of sampling steps to generate a coarse result. This preliminary outcome undergoes further refinement through a multi-scale refine network. Additionally, we introduce a novel approach called Partition Attention Sampling to enhance the representation capabilities of features. The effectiveness of the proposed model is validated in the experiments on the real Terracotta Warriors dataset and public dataset. The experimental results conclusively demonstrate that our model exhibits competitive performance in comparison to other existing models.

The semantic segmentation of satellite and UAV remote sensing imagery is pivotal for address exploration, change detection, quantitative analysis and urban planning. Recent advancements have seen an influx of segmentation networks utilizing convolutional neural networks and transformers. However, the intricate geographical features and varied land cover boundary interferences in remote sensing imagery still challenge conventional segmentation networks’ spatial representation and long-range dependency capabilities. This paper introduces a novel U-Net-like network for UAV image segmentation. We developed a link aggregation Mamba at the critical skip connection stage of UNetFormer. This approach maps and aggregates multi-scale features from different stages into a unified linear dimension through four Mamba branches containing state-space models (SSMs), ultimately decoupling and fusing these features to restore the contextual relationships in the mask. Moreover, the Mix-Mamba module is incorporated, leveraging a parallel self-attention mechanism with SSMs to merge the advantages of a global receptive field and reduce modeling complexity. This module facilitates nonlinear modeling across different channels and spaces through multipath activation, catering to international and local long-range dependencies. Evaluations on public remote sensing datasets like LovaDA, UAVid and Vaihingen underscore the state-of-the-art performance of our approach.

As an important representative of ancient Chinese cultural heritage, the classification of Terracotta Warriors point cloud data aids in cultural heritage preservation and digital reconstruction. However, these data face challenges such as complex morphological and structural variations, sparsity, and irregularity. This paper proposes a method named Multi-scale Local Geometric Transformer-Mamba (MLGTM) to improve the accuracy and robustness of Terracotta Warriors point cloud classification tasks. To effectively capture the geometric information of point clouds, we introduce local geometric encoding, including local coordinates and feature information, effectively capturing the complex local morphology and structural variations of the Terracotta Warriors and extracting representative local features. Additionally, we propose a multi-scale Transformer-Mamba information aggregation module, which employs a dual-branch Transformer with a Mamba structure and finally aggregates them on multiple scales to effectively handle the sparsity and irregularity of the Terracotta Warriors point cloud data. We conducted experiments on several datasets, including the ModelNet40, ScanObjectNN, ShapeNetPart, ETH, and 3D Terracotta Warriors fragment datasets. The results show that our method significantly improves the classification task of Terracotta Warriors point clouds, demonstrating strong accuracy.

Accurate and robust 3D human modeling from a single image presents significant challenges. Existing methods have shown potential, but they often fail to generate reconstructions that match the level of detail in the input image. These methods particularly struggle with loose clothing. They typically employ parameterized human models to constrain the reconstruction process, ensuring the results do not deviate too far from the model and produce anomalies. However, this also limits the recovery of loose clothing. To address this issue, we propose an end-to-end method called IHRPN for reconstructing clothed humans from a single 2D human image. This method includes a feature extraction module for semantic extraction of image features. We propose an image semantic feature extraction aimed at achieving pixel model space consistency and enhancing the robustness of loose clothing. We extract features from the input image to infer and recover the SMPL-X mesh, and then combine it with a normal map to guide the implicit function to reconstruct the complete clothed human. Unlike traditional methods, we use local features for implicit surface regression. Our experimental results show that our IHRPN method performs excellently on the CAPE and AGORA datasets, achieving good performance, and the reconstruction of loose clothing is noticeably more accurate and robust.

Abstract To obtain a higher simplification rate while retaining geometric features, a simplification framework for the point cloud is proposed. Firstly, multi-angle images of the original point cloud are obtained with a virtual camera. Then, feature lines of each image are extracted by deep neural network. Furthermore, according to the proposed mapping relationship between the acquired 2D feature lines and original point cloud, feature points of the point cloud are extracted automatically. Finally, the simplified point cloud is obtained by fusing feature points and simplified non-feature points. The proposed simplification method is applied to four data sets and compared with the other six algorithms. The experimental results demonstrate that our proposed simplification method has the superiority in terms of both retaining geometric features and high simplification rate.

At present, our team focuses on cultural relics restoration and fragment splicing research. In the research process of terracotta warrior splicing, we find that the existing calibrated fragment data is relatively small, which is not enough for related research. Therefore, we need to calibrate and segment different parts of the intact terracotta warrior data and extract some data we need to use in the future. However, at present, we are short of human resources. If we want to carry out manual calibration, it will take much time, bringing trouble to our future work. Therefore, we hope to design a method to automatically calibrate the terracotta warrior dataset with a small amount of calibrated data. The existing 3D neural network research mainly focuses on supervised classification, segmentation, and unsupervised reconstruction. We cannot find enough schemes to refer to, and the existing methods do not perform well on our terracotta warrior dataset. Therefore, in this article, we propose EGG-Net to solve this problem. EGG-Net is an end-to-end self-supervised model, and it consists of two modules. The first module is an encoder based on dynamic graph and edge convolution. We can extract point cloud features with this module. The second module, called segmenter, is based on multi-layer perceptron, adding labels to points and segmenting the point cloud. After the neural network, we add point refinement operation to the pipeline. Point refinement can adjust the cluster label estimated by the neural network with superpoint, which can optimize the loss function and help us train the neural network. Our EGG-Net can back-propagate with the refinement operation. We evaluated EGG-Net on the terracotta warrior data and ShapeNet Part by measuring the accuracy and the latency. The experiment result shows that our EGG-Net outperforms the state-of-the-art methods.

Unsupervised image-to-image translation (UI2I) tasks aim to find a mapping between the source and the target domains from unpaired training data. Previous methods can not effectively capture the differences between the source and the target domain on different scales and often leads to poor quality of the generated images, noise, distortion, and other conditions that do not match human vision perception, and has high time complexity. To address this problem, we propose a multi-scale training structure and a progressive growth generator method to solve UI2I task. Our method refines the generated images from global structures to local details by adding new convolution blocks continuously and shares the network parameters in different scales and also in the same scale of network. Finally, we propose a new Cross-CBAM mechanism (CRCBAM), which uses a multi-layer spatial attention and channel attention cross structure to generate more refined style images. Experiments on our collected Opera Face, and other open datasets Summer↔Winter, Horse↔Zebra, Photo↔Van Gogh, show that the proposed algorithm is superior to other state-of-art algorithms.

Abstract Geometry images parameterise a mesh with a square domain and store the information in a single chart. A one-to-one correspondence between the 2D plane and the 3D model is convenient for processing 3D models. However, the parameterised vertices are not all located at the intersection of the gridlines the existing geometry images. Thus, errors are unavoidable when a 3D mesh is reconstructed from the chart. In this paper, we propose parameterise surface onto a novel geometry image that preserves the constraint of topological neighbourhood information at integer coordinate points on a 2D grid and ensures that the shape of the reconstructed 3D mesh does not change from supplemented image data. We find a collection of edges that opens the mesh into simply connected surface with a single boundary. The point distribution with approximate blue noise spectral characteristics is computed by capacity-constrained delaunay triangulation without retriangulation. We move the vertices to the constrained mesh intersection, adjust the degenerate triangles on a regular grid, and fill the blank part by performing a local affine transformation between each triangle in the mesh and image. Unlike other geometry images, the proposed method results in no error in the reconstructed surface model when floating-point data are stored in the image. High reconstruction accuracy is achieved when the xyz positions are in a 16-bit data format in each image channel because only rounding errors exist in the topology-preserving geometry images, there are no sampling errors. This method performs one-to-one mapping between the 3D surface mesh and the points in the 2D image, while foldovers do not appear in the 2D triangular mesh, maintaining the topological structure. This also shows the potential of using a 2D image processing algorithm to process 3D models.

Brown adipose tissue (BAT) is a kind of adipose tissue engaging in thermoregulatory thermogenesis, metaboloregulatory thermogenesis, and secretory. Current studies have revealed that BAT activity is negatively correlated with adult body weight and is considered a target tissue for the treatment of obesity and other metabolic-related diseases. Additionally, the activity of BAT presents certain differences between different ages and genders. Clinically, BAT segmentation based on PET/CT data is a reliable method for brown fat research. However, most of the current BAT segmentation methods rely on the experience of doctors. In this paper, an improved U-net network, ICA-Unet, is proposed to achieve automatic and precise segmentation of BAT. First, the traditional 2D convolution layer in the encoder is replaced with a depth-wise over-parameterized convolutional (Do-Conv) layer. Second, the channel attention block is introduced between the double-layer convolution. Finally, the image information entropy (IIE) block is added in the skip connections to strengthen the edge features. Furthermore, the performance of this method is evaluated on the dataset of PET/CT images from 368 patients. The results demonstrate a strong agreement between the automatic segmentation of BAT and manual annotation by experts. The average DICE coefficient (DSC) is 0.9057, and the average Hausdorff distance is 7.2810. Experimental results suggest that the method proposed in this paper can achieve efficient and accurate automatic BAT segmentation and satisfy the clinical requirements of BAT.

The Terracotta Warriors are terracotta sculptures created for China's first emperor more than 2,000 years ago. They are among the most precious unearthed cultural relics of China. However, these relics have been predominantly found in fragments. Fragment classification is currently performed manually on enormous quantities of fragments, which is a time-consuming, inaccurate, and subjective task for archaeologists and conservators. In this study, an automatic method based on a deep learning network combined with template guidance is proposed to classify 3D fragments of the Terracotta Warriors. The fragments are initially classified using PointNet. Then, misclassified fragments are secondly categorized based on their best match to a complete Terracotta Warrior model. Extensive experiments were performed to verify the effectiveness of the proposed method. The promising results demonstrate that the method is the most accurate technique for classifying 3D Terracotta Warrior fragments to date. Moreover, the proposed method can significantly increase the efficiency of future fragment reassembly for the Terracotta Warriors.

We present a simple pairwise matching method for 3D fragment reassembly that uses boundary curves and concave-convex patches to accelerate and optimize the matching. Given the boundary curves of two fracture surfaces that completely or partially coincide, we can quickly exclude the fracture surface pairs with low boundary curve similarities, which can substantially reduce the computational cost of the subsequent patch matching, where we extract and delineate concave-convex patches of the selected fracture surfaces. A modified iterative closest point algorithm is applied on these concave-convex patches to refine the alignment. Finally, we determine the matched 3D fragments according to the overlap ratio of their fracture surfaces. The results of experiments on real-world examples demonstrate that our proposed algorithm is both accurate and efficient.

Online museums and online cultural relic information provide abundant data for relation extraction research. However, in the relation extraction task of modelling space information, spatially insensitive methods of convolutional neural networks and long short term memory network in most current works still remain challenging in rich text structures, which makes models difficult to encode effectively and lacks the ability of text expression. To address this issue, we propose a framework named WAtt-Capsnet (the capsule network with word-attention dynamic routing), which is based on capsule networks with word-attention dynamic routing for the relation extraction task of online cultural relic data for capturing richer instantiation features. We further present combination embedding for capturing the characteristic information of Chinese sentences by considering the contribution of word embedding, parts of speech, character embedding and the position of words to capture rich internal structure information of sentences. More importantly, to reduce the decay of useful information in long sentences, we propose a routing algorithm based on a word-attention mechanism to focus on informative words. The experimental results demonstrate that the proposed method achieves significant performance for the relation extraction task of online cultural relic data.

Music classification is an essential application of Music Information Retrieval (MIR) in organizing extensive collections of music. The tasks to classify different music with reliable accuracy observed to be challenging. Most of these tasks employ handcrafted feature engineering to build a classifier, yet unable to identify the original characteristics of music. Several combinations of neural networks using convolutional neural networks (CNNs) and recurrent neural networks (RNNs) have been in consideration of many researchers. However, it has been noticed that the joint architecture of CNN and RNN suffers some problems due to batch normalization, which causes low accuracy and more training time. To handle these issues, the Global Layer Regularization (GLR) technique is proposed on the hybrid model of CNN and RNN using Mel-spectrograms for the evaluation of training and accuracy. Our experiments, with few hyper-parameters, improve performance on GTZAN and Free Music Achieve (FMA) datasets by achieving modest accuracy of 87.79% and 68.87% respectively. Empirically, our proposed model takes the advantages of spatiotemporal domain features and the global layer regularization technique to accomplish reliable accuracy as compared to the other state of art works.

With the objective of addressing the problem of the fixed convolutional kernel of a standard convolution neural network and the isotropy of features making 3D point cloud data ineffective in feature learning, this paper proposes a point cloud processing method based on graph convolution multilayer perceptron, named GC-MLP. Unlike traditional local aggregation operations, the algorithm generates an adaptive kernel through the dynamic learning features of points, so that it can dynamically adapt to the structure of the object, i.e., the algorithm first adaptively assigns different weights to adjacent points according to the different relationships between the different points captured. Furthermore, local information interaction is then performed with the convolutional layers through a weight-sharing multilayer perceptron. Experimental results show that, under different task benchmark datasets (including ModelNet40 dataset, ShapeNet Part dataset, S3DIS dataset), our proposed algorithm achieves state-of-the-art for both point cloud classification and segmentation tasks.

Cerenkov luminescence tomography (CLT) is a promising imaging modality in the field of optical molecular imaging (OMI), which successfully bridges the OMI and tradition nuclear medical imaging and provides the location and quantitative analysis of the distribution of radionuclide probes inside the biological objects. As the CLT is an inherent highly ill-posed inverse problem, it is still a challenge to obtain an accurate reconstruction result. Here, we proposed a novel reconstruction framework based on stacking denoising autoencoders (SDAE), which serve as one famous structure of the artificial neural network (ANN). In our framework, the initial permission region is the whole domain and then a traditional reconstruction algorithm is used to reconstruct each node's energy. Then these nodes are clustered into two regions: permission region and non-permission region, and the permission region is used to start a new reconstruction loop where a new result can be obtained. The procedures above are repeated before the result meets the stop conditions. The numerical simulation experiments, physical phantom experiments and in vivo experiments are all carried out to validate the feasibility and potential of our framework. Results demonstrate that the proposed framework can indeed achieve a good performance in CLT reconstruction.

We present a method to compute the descriptor of components of point clouds, therefore, a novel component-oriented partial matching of point clouds is achieved based on the component descriptor. We observe that 3D components can be constructed by stacking 2D shapes using certain criteria so that the centers of the 2D shapes form a curve called a skeletal curve that is the trajectory of the 2D shapes. In addition, the scaling factors of the 2D shapes also impact the shape of the 3D components. Motivated by these observations, the computation of the component descriptor that is termed 2to3SSC (from 2D to 3D: 2D Shape and Skeletal Curve) is formulated as a 2D shape and skeletal curve extraction problem, and the component-oriented partial matching of the point clouds is based on the dissimilarity measure of 2to3SSCs of the components. Furthermore, for the 2D shape matching, which is crucial to the matching of the components, we present a novel 2D shape descriptor called VDTL (Vertical Distances to the Tangent Line). The proposed method outperforms previously proposed methods because it simultaneously encodes the local and global features of the components as opposed to only encoding the local or partial features as in previous studies. Finally, the effectiveness and performance of 2to3SSCs are compared with those of state-of-the-art feature description and matching methods for different point cloud datasets. Further, the benefits and the applicability of the proposed method are demonstrated; favorable results are obtained for real-world point clouds of Terracotta fragments.

With the rapid development of 3D scanners, the cultural heritage artifacts can be stored as a point cloud and displayed through the Internet. However, due to natural and human factors, many cultural relics had some surface damage when excavated. As a result, the holes caused by these damages still exist in the generated point cloud model. This work proposes a multi-scale upsampling GAN (MU-GAN) based framework for completing these holes. Firstly, a 3D mesh model based on the original point cloud is reconstructed, and the method of detecting holes is presented. Secondly, the point cloud patch contains hole regions and is extracted from the point cloud. Then the patch is input into the MU-GAN to generate a high-quality dense point cloud. Finally, the empty areas on the original point cloud are filled with the generated dense point cloud patches. A series of real-world experiments are conducted on real scan data to demonstrate that the proposed framework can fill the holes of 3D heritage models with grained details. We hope that our work can provide a useful tool for cultural heritage protection.

This study proposes a blocks matching method based on contour curves and feature regions that improve the matching precision and speed with which rigid blocks with a specified thickness in point clouds are matched. The method comprises two steps: coarse matching and fine matching. In the coarse matching step, the rigid blocks are first segmented into a series of surfaces and the fracture surfaces are distinguished. Then, the contour curves of the fracture surfaces are extracted using an improved boundary growth method and the rigid blocks are coarsely matched with them. In the fine matching step, feature regions are first extracted from the fracture surfaces. Then, the centroid of each feature region is calculated and the fine matching of rigid blocks with the centroid sets is completed using an improved iterative closest point (ICP) algorithm. The improved ICP algorithm integrates the rotation angle constraint and dynamic iteration coefficient into a probability ICP algorithm, which significantly improves matching precision and speed. Experiments conducted using public blocks and Terracotta Warriors blocks indicate that the proposed method carries out rigid blocks matching more accurately and rapidly than various conventional methods.

Simultaneous algebraic reconstruction technique (SART) is a well-known iterative method in X-ray computed tomography, which provides better image quality than analytical methods when dealing with incomplete or noisy data. The disadvantage of SART is the slow speed compared with the analytical methods. Since forward projection and backprojection are two major time-consuming operations in iterative reconstruction, we propose an algorithm for fast forward projection and improved backprojection for the line integral model-based SART. Using the proposed algorithm, the SART method was implemented on a GPU platform with NVIDIA's parallel computing architecture. Both computer simulations and physical phantom experiments were carried out, and their results show that our approach is highly efficient and accurate. The computation time for the system matrix using our proposed projector is 10 times faster than that using the Siddon's projector, and our improved backprojection algorithm is 1.5 times faster than Li's method in determining the minimum bounding interval. The GPU-based SART using our proposed projection algorithm can obtain about 7.4 times reconstruction speed-up compared with that using the traditional projection approach, while preserving the accuracy of the results.

Automatically selecting a set of representative views of a 3D virtual cultural relic is crucial for constructing wisdom museums. There is no consensus regarding the definition of a good view in computer graphics; the same is true of multiple views. View-based methods play an important role in the field of 3D shape retrieval and classification. However, it is still difficult to select views that not only conform to subjective human preferences but also have a good feature description. In this study, we define two novel measures based on information entropy, named depth variation entropy and depth distribution entropy. These measures were used to determine the amount of information about the depth swings and different depth quantities of each view. Firstly, a canonical pose 3D cultural relic was generated using principal component analysis. A set of depth maps obtained by orthographic cameras was then captured on the dense vertices of a geodesic unit-sphere by subdividing the regular unit-octahedron. Afterwards, the two measures were calculated separately on the depth maps gained from the vertices and the results on each one-eighth sphere form a group. The views with maximum entropy of depth variation and depth distribution were selected, and further scattered viewpoints were selected. Finally, the threshold word histogram derived from the vector quantization of salient local descriptors on the selected depth maps represented the 3D cultural relic. The viewpoints obtained by the proposed method coincided with an arbitrary pose of the 3D model. The latter eliminated the steps of manually adjusting the model’s pose and provided acceptable display views for people. In addition, it was verified on several datasets that the proposed method, which uses the Bag-of-Words mechanism and a deep convolution neural network, also has good performance regarding retrieval and classification when dealing with only four views.

As an essential step in the restoration of Terracotta Warriors, the results of fragments classification will directly affect the performance of fragments matching and splicing. However, most of the existing methods are based on traditional technology and have low accuracy in classification. A practical and effective classification method for fragments is an urgent need. In this case, an attention-based multi-scale neural network named AMS-Net is proposed to extract significant geometric and semantic features. AMS-Net is a hierarchical structure consisting of a multi-scale set abstraction block (MS-BLOCK) and a fully connected (FC) layer. MS-BLOCK consists of a local-global layer (LGLayer) and an improved multi-layer perceptron (IMLP). With a multi-scale strategy, LGLayer can parallel extract the local and global features from different scales. IMLP can concatenate the high-level and low-level features for classification tasks. Extensive experiments on the public data set (ModelNet40/10) and the real-world Terracotta Warrior fragments data set are conducted. The accuracy results with normal can achieve 93.52% and 96.22%, respectively. For real-world data sets, the accuracy is best among the existing methods. The robustness and effectiveness of the performance on the task of 3D point cloud classification are also investigated. It proves that the proposed end-to-end learning network is more effective and suitable for the classification of the Terracotta Warrior fragments.

As one of China′s most precious cultural relics, the excavation and protection of the Terracotta Warriors pose significant challenges to archaeologists. A fairly common situation in the excavation is that the Terracotta Warriors are mostly found in the form of fragments, and manual reassembly among numerous fragments is laborious and time-consuming. This work presents a fracture-surface-based reassembling method, which is composed of SiamesePointNet, principal component analysis (PCA), and deep closest point (DCP), and is named SPPD. Firstly, SiamesePointNet is proposed to determine whether a pair of point clouds of 3D Terracotta Warrior fragments can be reassembled. Then, a coarse-to-fine registration method based on PCA and DCP is proposed to register the two fragments into a reassembled one. The above two steps iterate until the termination condition is met. A series of experiments on real-world examples are conducted, and the results demonstrate that the proposed method performs better than the conventional reassembling methods. We hope this work can provide a valuable tool for the virtual restoration of three-dimension cultural heritage artifacts.

Computer-aided classification serves as the basis of virtual cultural relic management and display. The majority of the existing cultural relic classification methods require labelling of the samples of the dataset; however, in practical applications, there is often a lack of category labels of samples or an uneven distribution of samples of different categories. To solve this problem, we propose a 3D cultural relic classification method based on a low dimensional descriptor and unsupervised learning. First, the scale-invariant heat kernel signature (Si-HKS) was computed. The heat kernel signature denotes the heat flow of any two vertices across a 3D shape and the heat diffusion propagation is governed by the heat equation. Secondly, the Bag-of-Words (BoW) mechanism was utilized to transform the Si-HKS descriptor into a low-dimensional feature tensor, named a SiHKS-BoW descriptor that is related to entropy. Finally, we applied an unsupervised learning algorithm, called MKDSIF-FCM, to conduct the classification task. A dataset consisting of 3D models from 41 Tang tri-color Hu terracotta Eures was utilized to validate the effectiveness of the proposed method. A series of experiments demonstrated that the SiHKS-BoW descriptor along with the MKDSIF-FCM algorithm showed the best classification accuracy, up to 99.41%, which is a solution for an actual case with the absence of category labels and an uneven distribution of different categories of data. The present work promotes the application of virtual reality in digital projects and enriches the content of digital archaeology.

Knowledge graph completion can make knowledge graphs more complete, which is a meaningful research topic. However, the existing methods do not make full use of entity semantic information. Another challenge is that a deep model requires large-scale manually labelled data, which greatly increases manual labour. In order to alleviate the scarcity of labelled data in the field of cultural relics and capture the rich semantic information of entities, this paper proposes a model based on the Bidirectional Encoder Representations from Transformers (BERT) with entity-type information for the knowledge graph completion of the Chinese texts of cultural relics. In this work, the knowledge graph completion task is treated as a classification task, while the entities, relations and entity-type information are integrated as a textual sequence, and the Chinese characters are used as a token unit in which input representation is constructed by summing token, segment and position embeddings. A small number of labelled data are used to pre-train the model, and then, a large number of unlabelled data are used to fine-tune the pre-training model. The experiment results show that the BERT-KGC model with entity-type information can enrich the semantics information of the entities to reduce the degree of ambiguity of the entities and relations to some degree and achieve more effective performance than the baselines in triple classification, link prediction and relation prediction tasks using 35% of the labelled data of cultural relics.

Increasingly, popular online museums have significantly changed the way people acquire cultural knowledge. These online museums have been generating abundant amounts of cultural relics data. In recent years, researchers have used deep learning models that can automatically extract complex features and have rich representation capabilities to implement named-entity recognition (NER). However, the lack of labeled data in the field of cultural relics makes it difficult for deep learning models that rely on labeled data to achieve excellent performance. To address this problem, this paper proposes a semi-supervised deep learning model named SCRNER (Semi-supervised model for Cultural Relics’ Named Entity Recognition) that utilizes the bidirectional long short-term memory (BiLSTM) and conditional random fields (CRF) model trained by seldom labeled data and abundant unlabeled data to attain an effective performance. To satisfy the semi-supervised sample selection, we propose a repeat-labeled (relabeled) strategy to select samples of high confidence to enlarge the training set iteratively. In addition, we use embeddings from language model (ELMo) representations to dynamically acquire word representations as the input of the model to solve the problem of the blurred boundaries of cultural objects and Chinese characteristics of texts in the field of cultural relics. Experimental results demonstrate that our proposed model, trained on limited labeled data, achieves an effective performance in the task of named entity recognition of cultural relics.

As an emerging molecular imaging modality, cone-beam X-ray luminescence computed tomography (CB-XLCT) uses X-ray-excitable probes to produce near-infrared (NIR) luminescence and then reconstructs three-dimensional (3D) distribution of the probes from surface measurements. A proper photon-transportation model is critical to accuracy of XLCT. Here, we presented a systematic comparison between the common-used Monte Carlo model and simplified spherical harmonics (SPN). The performance of the two methods was evaluated over several main spectrums using a known XLCT material. We designed both a global measurement based on the cosine similarity and a locally-averaged relative error, to quantitatively assess these methods. The results show that the SP3 could reach a good balance between the modeling accuracy and computational efficiency for all of the tested emission spectrums. Besides, the SP1 (which is equivalent to the diffusion equation (DE)) can be a reasonable alternative model for emission wavelength over 692nm. In vivo experiment further demonstrates the reconstruction performance of the SP3 and DE. This study would provide a valuable guidance for modeling the photon-transportation in CB-XLCT.

Social media and health-related forums, including the expression of customer reviews, have recently provided data sources for adverse drug reaction (ADR) identification research. However, in the existing methods, the neglect of noise data and the need for manually labeled data reduce the accuracy of the prediction results and greatly increase manual labor. We propose a novel architecture named the weakly supervised mechanism (WSM) convolutional neural network (CNN) long-short-term memory (WSM-CNN-LSTM), which combines the strength of CNN and bi-directional long short-term memory (Bi-LSTM). The WSM applies the weakly labeled data to pre-train the parameters of the model and then uses the labeled data to fine-tune the initialized network parameters. The CNN employs a convolutional layer to study the characteristics of the drug reviews and active features at different scales, and then the feed-forward and feed-back neural networks of the Bi-LSTM utilize these salient features to output the regression results. The experimental results effectively demonstrate that our model marginally outperforms the comparison models in ADR identification and that a small quantity of labeled samples results in an optimal performance, which decreases the influence of noise and reduces the manual data-labeling requirements.

As the numbers of 3D models available grow in many application fields, there is an increasing need for a search method to help people find them. Unfortunately, traditional search techniques are not always effective for 3D data. In this paper, we describe a novel method of interactive 3D model retrieval with building blocks. First, by using a cube block as the baseblock in a 3D virtual space, we may construct the query model with human-computer interaction method. Then through retrieving the polygon model of the database generated by the voxel model, we may get retrieval results in real time. Experiments are conducted to evaluate the performance of the proposed method.