Search

Traditional recommender systems heavily rely on ID features, which often encounter challenges related to cold-start and generalization. Modeling pre-extracted content features can mitigate these issues, but is still a suboptimal solution due to the discrepancies between training tasks and model parameters. End-to-end training presents a promising solution for these problems, yet most of the existing works mainly focus on retrieval models, leaving the multimodal techniques under-utilized. In this paper, we propose an industrial multimodal recommendation framework named EM3: End-to-end training of Multimodal Model and ranking Model, which sufficiently utilizes multimodal information and allows personalized ranking tasks to directly train the core modules in the multimodal model to obtain more task-oriented content features, without overburdening resource consumption. First, we propose Fusion-Q-Former, which consists of transformers and a set of trainable queries, to fuse different modalities and generate fixed-length and robust multimodal embeddings. Second, in our sequential modeling for user content interest, we utilize Low-Rank Adaptation technique to alleviate the conflict between huge resource consumption and long sequence length. Third, we propose a novel Content-ID-Contrastive learning task to complement the advantages of content and ID by aligning them with each other, obtaining more task-oriented content embeddings and more generalized ID embeddings. In experiments, we implement EM3 on different ranking models in two scenario, achieving significant improvements in both offline evaluation and online A/B test, verifying the generalizability of our method. Ablation studies and visualization are also performed. Furthermore, we also conduct experiments on two public datasets to show that our proposed method outperforms the state-of-the-art methods., Comment: 9 pages, 8 figures

Fashion image retrieval task aims to search relevant clothing items of a query image from the gallery. The previous recipes focus on designing different distance-based loss functions, pulling relevant pairs to be close and pushing irrelevant images apart. However, these methods ignore fine-grained features (e.g. neckband, cuff) of clothing images. In this paper, we propose a novel fashion image retrieval method leveraging both global and fine-grained features, dubbed Multi-Granular Alignment (MGA). Specifically, we design a Fine-Granular Aggregator(FGA) to capture and aggregate detailed patterns. Then we propose Attention-based Token Alignment (ATA) to align image features at the multi-granular level in a coarse-to-fine manner. To prove the effectiveness of our proposed method, we conduct experiments on two sub-tasks (In-Shop & Consumer2Shop) of the public fashion datasets DeepFashion. The experimental results show that our MGA outperforms the state-of-the-art methods by 1.8% and 0.6% in the two sub-tasks on the R@1 metric, respectively.

In this work we develop a novel approach using deep neural networks to reconstruct the conductivity distribution in elliptic problems from one measurement of the solution over the whole domain. The approach is based on a mixed reformulation of the governing equation and utilizes the standard least-squares objective, with deep neural networks as ansatz functions to approximate the conductivity and flux simultaneously. We provide a thorough analysis of the deep neural network approximations of the conductivity for both continuous and empirical losses, including rigorous error estimates that are explicit in terms of the noise level, various penalty parameters and neural network architectural parameters (depth, width and parameter bound). We also provide multiple numerical experiments in two- and multi-dimensions to illustrate distinct features of the approach, e.g., excellent stability with respect to data noise and capability of solving high-dimensional problems., Comment: corrected a few typos

Current Large Language Model-based agents reason within an exploration-evaluation framework, navigating problem-solving processes in a tree-like manner. However, these methods often neglect successful reasoning trajectories once a problem is resolved, leading to inefficient use of these trajectories for future analogous problems. To address this inefficiency, we adopt a state machine to record experience derived from previous reasoning trajectories. Within the state machine, states represent decomposed sub-problems, while state transitions reflect the dependencies among sub-problems. The state machine records both successful and failed trajectories. Utilizing the experience from the state machine, our proposed State Machine of Thoughts (SMoT) selects the most optimal sub-solutions and avoids incorrect ones. Our experiments show that SMoT can significantly improve problem-solving abilities in two exploration-intensive problems: the 24-point game and a taxi navigation reinforcement learning game., Comment: 9 pages, 4 figures

The goal of Universal Cross-Domain Retrieval (UCDR) is to achieve robust performance in generalized test scenarios, wherein data may belong to strictly unknown domains and categories during training. Recently, pre-trained models with prompt tuning have shown strong generalization capabilities and attained noteworthy achievements in various downstream tasks, such as few-shot learning and video-text retrieval. However, applying them directly to UCDR may not sufficiently to handle both domain shift (i.e., adapting to unfamiliar domains) and semantic shift (i.e., transferring to unknown categories). To this end, we propose \textbf{Pro}mpting-to-\textbf{S}imulate (ProS), the first method to apply prompt tuning for UCDR. ProS employs a two-step process to simulate Content-aware Dynamic Prompts (CaDP) which can impact models to produce generalized features for UCDR. Concretely, in Prompt Units Learning stage, we introduce two Prompt Units to individually capture domain and semantic knowledge in a mask-and-align way. Then, in Context-aware Simulator Learning stage, we train a Content-aware Prompt Simulator under a simulated test scenarios to produce the corresponding CaDP. Extensive experiments conducted on three benchmark datasets show that our method achieves new state-of-the-art performance without bringing excessive parameters. Our method is publicly available at https://github.com/fangkaipeng/ProS.

Knowledge Graph Completion (KGC) aims to conduct reasoning on the facts within knowledge graphs and automatically infer missing links. Existing methods can mainly be categorized into structure-based or description-based. On the one hand, structure-based methods effectively represent relational facts in knowledge graphs using entity embeddings. However, they struggle with semantically rich real-world entities due to limited structural information and fail to generalize to unseen entities. On the other hand, description-based methods leverage pre-trained language models (PLMs) to understand textual information. They exhibit strong robustness towards unseen entities. However, they have difficulty with larger negative sampling and often lag behind structure-based methods. To address these issues, in this paper, we propose Momentum Contrast for knowledge graph completion with Structure-Augmented pre-trained language models (MoCoSA), which allows the PLM to perceive the structural information by the adaptable structure encoder. To improve learning efficiency, we proposed momentum hard negative and intra-relation negative sampling. Experimental results demonstrate that our approach achieves state-of-the-art performance in terms of mean reciprocal rank (MRR), with improvements of 2.5% on WN18RR and 21% on OpenBG500.

Conductivity imaging represents one of the most important tasks in medical imaging. In this work we develop a neural network based reconstruction technique for imaging the conductivity from the magnitude of the internal current density. It is achieved by formulating the problem as a relaxed weighted least-gradient problem, and then approximating its minimizer by standard fully connected feedforward neural networks. We derive bounds on two components of the generalization error, i.e., approximation error and statistical error, explicitly in terms of properties of the neural networks (e.g., depth, total number of parameters, and the bound of the network parameters). We illustrate the performance and distinct features of the approach on several numerical experiments. Numerically, it is observed that the approach enjoys remarkable robustness with respect to the presence of data noise., Comment: 29 pp, 9 figures (several typos are corrected in the new version), to appear at Inverse Problems

In Domain Generalization (DG) tasks, models are trained by using only training data from the source domains to achieve generalization on an unseen target domain, this will suffer from the distribution shift problem. So it's important to learn a classifier to focus on the common representation which can be used to classify on multi-domains, so that this classifier can achieve a high performance on an unseen target domain as well. With the success of cross attention in various cross-modal tasks, we find that cross attention is a powerful mechanism to align the features come from different distributions. So we design a model named CADG (cross attention for domain generalization), wherein cross attention plays a important role, to address distribution shift problem. Such design makes the classifier can be adopted on multi-domains, so the classifier will generalize well on an unseen domain. Experiments show that our proposed method achieves state-of-the-art performance on a variety of domain generalization benchmarks compared with other single model and can even achieve a better performance than some ensemble-based methods.

Aging of craniofacial skeleton significantly impairs the repair and regeneration of trauma-induced bony defects, and complicates dental treatment outcomes. Age-related alveolar bone loss could be attributed to decreased progenitor pool through senescence, imbalance in bone metabolism and bone-fat ratio. Mesenchymal stem cells isolated from oral bones (OMSCs) have distinct lineage propensities and characteristics compared to MSCs from long bones, and are more suited for craniofacial regeneration. However, the effect of epigenetic modifications regulating OMSC differentiation and senescence in aging has not yet been investigated. In this study, we found that the histone demethylase KDM4B plays an essential role in regulating the osteogenesis of OMSCs and oral bone aging. Loss of KDM4B in OMSCs leads to inhibition of osteogenesis. Moreover, KDM4B loss promoted adipogenesis and OMSC senescence which further impairs bone-fat balance in the mandible. Together, our data suggest that KDM4B may underpin the molecular mechanisms of OMSC fate determination and alveolar bone homeostasis in skeletal aging, and present as a promising therapeutic target for addressing craniofacial skeletal defects associated with age-related deteriorations.

Highlights • Lithospheric mantle stabilization under Jiao-Liao-Ji Belt at 2.8Ga (SmNd isochron) • Liaohe mafic meta-igneous rocks formed in active continental margin subduction zone • Emplacement of Liaohe mafic igneous rocks at ~2.2 Ga (LuHf isochron) • Amphibolite retrograde metamorphism from exhumation at 1824 ± 19 Ma (PbPb isochron) • Cooling of terrane to ~500 °C at 1671 ± 58 Ma (RbSr isochron) Abstract The assembly and long-term evolution of the Eastern Block of the North China Craton are poorly constrained. Here we use bulk rock geochronological and geochemical data from mafic meta-igneous rocks (hornblendites, amphibolites and a metagabbro) of the Liaohe Group to reconstruct the Neoarchean to Paleoproterozoic history of the Jiao-Liao-Ji Belt, located between the Longgang and Nangrim blocks that together form the Eastern Block of the North China Craton. The mafic/ultramafic meta-igneous rocks have intrusive or tectonic contacts with the Liaoji granitic rocks (~2.2–2.0 Ga), which form the basement of the Jiao-Liao-Ji Belt. The major and trace element data indicate that the protoliths had calc-alkaline composition and formed along an active continental margin subduction zone. The mafic rocks form a whole-rock 176Lu/177Hf isochron with an age of 2.25 ± 0.31 Ga, overlapping with UPb zircon ages for mafic and granitic rocks from the Jiao-Liao-Ji Belt and consistent with being the emplacement age of the mafic protoliths along the active continental margin. In contrast, the whole-rock 147Sm/144Nd isochron age of 2.83 ± 0.18 Ga is likely to reflect the average age of the lithospheric mantle source from which the mafic/ultramafic protoliths were extracted. Together with geological evidence, we propose that the southwestern portion of the Longgang Block was an active continental margin since at least the early Paleoproteorozic. Literature age data from metamorphic zircons show that peak granulite metamorphism took place at ~1.96–1.88 Ga, resulting from the collisional

Highlights • Lithospheric mantle stabilization under Jiao-Liao-Ji Belt at 2.8Ga (SmNd isochron) • Liaohe mafic meta-igneous rocks formed in active continental margin subduction zone • Emplacement of Liaohe mafic igneous rocks at ~2.2 Ga (LuHf isochron) • Amphibolite retrograde metamorphism from exhumation at 1824 ± 19 Ma (PbPb isochron) • Cooling of terrane to ~500 °C at 1671 ± 58 Ma (RbSr isochron) Abstract The assembly and long-term evolution of the Eastern Block of the North China Craton are poorly constrained. Here we use bulk rock geochronological and geochemical data from mafic meta-igneous rocks (hornblendites, amphibolites and a metagabbro) of the Liaohe Group to reconstruct the Neoarchean to Paleoproterozoic history of the Jiao-Liao-Ji Belt, located between the Longgang and Nangrim blocks that together form the Eastern Block of the North China Craton. The mafic/ultramafic meta-igneous rocks have intrusive or tectonic contacts with the Liaoji granitic rocks (~2.2–2.0 Ga), which form the basement of the Jiao-Liao-Ji Belt. The major and trace element data indicate that the protoliths had calc-alkaline composition and formed along an active continental margin subduction zone. The mafic rocks form a whole-rock 176Lu/177Hf isochron with an age of 2.25 ± 0.31 Ga, overlapping with UPb zircon ages for mafic and granitic rocks from the Jiao-Liao-Ji Belt and consistent with being the emplacement age of the mafic protoliths along the active continental margin. In contrast, the whole-rock 147Sm/144Nd isochron age of 2.83 ± 0.18 Ga is likely to reflect the average age of the lithospheric mantle source from which the mafic/ultramafic protoliths were extracted. Together with geological evidence, we propose that the southwestern portion of the Longgang Block was an active continental margin since at least the early Paleoproteorozic. Literature age data from metamorphic zircons show that peak granulite metamorphism took place at ~1.96–1.88 Ga, resulting from the collisional

Highlights • Lithospheric mantle stabilization under Jiao-Liao-Ji Belt at 2.8Ga (SmNd isochron) • Liaohe mafic meta-igneous rocks formed in active continental margin subduction zone • Emplacement of Liaohe mafic igneous rocks at ~2.2 Ga (LuHf isochron) • Amphibolite retrograde metamorphism from exhumation at 1824 ± 19 Ma (PbPb isochron) • Cooling of terrane to ~500 °C at 1671 ± 58 Ma (RbSr isochron) Abstract The assembly and long-term evolution of the Eastern Block of the North China Craton are poorly constrained. Here we use bulk rock geochronological and geochemical data from mafic meta-igneous rocks (hornblendites, amphibolites and a metagabbro) of the Liaohe Group to reconstruct the Neoarchean to Paleoproterozoic history of the Jiao-Liao-Ji Belt, located between the Longgang and Nangrim blocks that together form the Eastern Block of the North China Craton. The mafic/ultramafic meta-igneous rocks have intrusive or tectonic contacts with the Liaoji granitic rocks (~2.2–2.0 Ga), which form the basement of the Jiao-Liao-Ji Belt. The major and trace element data indicate that the protoliths had calc-alkaline composition and formed along an active continental margin subduction zone. The mafic rocks form a whole-rock 176Lu/177Hf isochron with an age of 2.25 ± 0.31 Ga, overlapping with UPb zircon ages for mafic and granitic rocks from the Jiao-Liao-Ji Belt and consistent with being the emplacement age of the mafic protoliths along the active continental margin. In contrast, the whole-rock 147Sm/144Nd isochron age of 2.83 ± 0.18 Ga is likely to reflect the average age of the lithospheric mantle source from which the mafic/ultramafic protoliths were extracted. Together with geological evidence, we propose that the southwestern portion of the Longgang Block was an active continental margin since at least the early Paleoproteorozic. Literature age data from metamorphic zircons show that peak granulite metamorphism took place at ~1.96–1.88 Ga, resulting from the collisional

Any plate has a growth process from small to large. The micro-blocks or micro-plates are sometimes the precursors of large plates. The origin, growth, aborting, extinction and residual process of micro-blocks are of great significance for the study of plate tectonics and pre-plate tectonics. The micro-block can be divided into continental, oceanic and mantle micro-blocks according to their compositions. In this paper, the micro-blocks in the global oceans have been summarized according to the following five environments: mid-ocean ridge system, subduction system, transform fault system, deep-sea intraplate system and extension-rift system. We first propose a genetic classification of micro-blocks comprising: detachment-derived, rifting-derived, transform-derived, propagation-derived, ridge jumping-derived, subduction-derived, accretion-derived, collision-derived and delamination-derived micro-blocks. The different types of micro-block boundaries such as active or fossil detachment fault, subduction zone, mid-ocean ridge, transform fault, fracture zone, transfer fault, accommodation zone, lithosphere-scale strike-slip fault, pseudofault, intra-oceanic convergent zone, overlapping spreading centre, non-transform offset, rheological crustal or mantle discontinuity, are systematically discussed for different micro-blocks. Thus, the number of triple junctions will be more than the 16 in the traditional Plate Tectonic Theory. A stability analysis of these triple junctions is the key to understanding the causes of micro-blocks. However, the micro-block has no ultimate cause, so it is unnecessary to pursue one ultimate cause or initiation of plate tectonics. These micro-blocks within oceanic basins, along oceanic margins or within the deep mantle, can not only be used to develop deep ocean fine structural analysis and plate tectonic reconstruction, but also to explain the causes of micro-blocks in some orogens. This will enrich the research of more detailed pre-orogenic or

Any plate has a growth process from small to large. The micro-blocks or micro-plates are sometimes the precursors of large plates. The origin, growth, aborting, extinction and residual process of micro-blocks are of great significance for the study of plate tectonics and pre-plate tectonics. The micro-block can be divided into continental, oceanic and mantle micro-blocks according to their compositions. In this paper, the micro-blocks in the global oceans have been summarized according to the following five environments: mid-ocean ridge system, subduction system, transform fault system, deep-sea intraplate system and extension-rift system. We first propose a genetic classification of micro-blocks comprising: detachment-derived, rifting-derived, transform-derived, propagation-derived, ridge jumping-derived, subduction-derived, accretion-derived, collision-derived and delamination-derived micro-blocks. The different types of micro-block boundaries such as active or fossil detachment fault, subduction zone, mid-ocean ridge, transform fault, fracture zone, transfer fault, accommodation zone, lithosphere-scale strike-slip fault, pseudofault, intra-oceanic convergent zone, overlapping spreading centre, non-transform offset, rheological crustal or mantle discontinuity, are systematically discussed for different micro-blocks. Thus, the number of triple junctions will be more than the 16 in the traditional Plate Tectonic Theory. A stability analysis of these triple junctions is the key to understanding the causes of micro-blocks. However, the micro-block has no ultimate cause, so it is unnecessary to pursue one ultimate cause or initiation of plate tectonics. These micro-blocks within oceanic basins, along oceanic margins or within the deep mantle, can not only be used to develop deep ocean fine structural analysis and plate tectonic reconstruction, but also to explain the causes of micro-blocks in some orogens. This will enrich the research of more detailed pre-orogenic or

The ocean floor is leaky because it has numerous faults, cracks and joints upon formation and during the process of seafloor spreading. In time, these fractures are often closed after ocean floor cooling, hydrothermal circulation, and vein filling. The crack-seal mechanism of fractures of the oceanic crust is thus important for understanding its kinematics, kinetics and evolution. Coring and log data from IODP Expedition 324 reveal that the Shatsky Rise, an oceanic plateau in the NW Pacific Ocean, developed abundant joints and veins, and some veins formed along previous joints. We use log data from the Formation Micro-scanner Scanner (FMS) to reconstruct the original dip and dip direction of these structural elements. Using FMS microstructural analyses, the dip directions of joints and arrangement of solid inclusions in fibrous veins were examined for Holes U1347A, U1348A and U1349A. We found two types of veins, non-fibrous and fibrous, based on their physical appearance and mineralogical composition. Common to all samples is a straight fibrous inclusion fabric, associated with bands oriented parallel to the vein wall and trails typically at high angles to the vein wall. Cross-cutting relationships between the bands and the straight fibrous inclusions imply that inclusion bands reflect simple crack-seal increments. In the veins, inclusion bands are a sufficient criterion to infer the crack-seal mechanism. Further evidence for solid inclusions formed by the classic crack-seal mechanism is given by inclusion bands in carbon crystals grown in basalts. During incorporation, solid inclusions can remain undeformed, depending on their orientation with respect to the opening and spreading direction of mid-oceanic ridges. Simple displacement fields within the veins are recorded by straight crystal fibres, which track the opening direction. Based on the arrangement of solid inclusions within the veins, we suggest that the veins grew continuously during post-tectonic vein form

The ocean floor is leaky because it has numerous faults, cracks and joints upon formation and during the process of seafloor spreading. In time, these fractures are often closed after ocean floor cooling, hydrothermal circulation, and vein filling. The crack-seal mechanism of fractures of the oceanic crust is thus important for understanding its kinematics, kinetics and evolution. Coring and log data from IODP Expedition 324 reveal that the Shatsky Rise, an oceanic plateau in the NW Pacific Ocean, developed abundant joints and veins, and some veins formed along previous joints. We use log data from the Formation Micro-scanner Scanner (FMS) to reconstruct the original dip and dip direction of these structural elements. Using FMS microstructural analyses, the dip directions of joints and arrangement of solid inclusions in fibrous veins were examined for Holes U1347A, U1348A and U1349A. We found two types of veins, non-fibrous and fibrous, based on their physical appearance and mineralogical composition. Common to all samples is a straight fibrous inclusion fabric, associated with bands oriented parallel to the vein wall and trails typically at high angles to the vein wall. Cross-cutting relationships between the bands and the straight fibrous inclusions imply that inclusion bands reflect simple crack-seal increments. In the veins, inclusion bands are a sufficient criterion to infer the crack-seal mechanism. Further evidence for solid inclusions formed by the classic crack-seal mechanism is given by inclusion bands in carbon crystals grown in basalts. During incorporation, solid inclusions can remain undeformed, depending on their orientation with respect to the opening and spreading direction of mid-oceanic ridges. Simple displacement fields within the veins are recorded by straight crystal fibres, which track the opening direction. Based on the arrangement of solid inclusions within the veins, we suggest that the veins grew continuously during post-tectonic vein form