Search

Recently, Segment Anything Model (SAM) has become a research hotspot in the fields of multimedia and computer vision, which exhibits powerful yet versatile capabilities on various (un) conditional image segmentation tasks. Although SAM can support different types of segmentation prompts, we note that, compared to point- and box-guided segmentations, it performs much worse on text-instructed tasks, e.g., referring image segmentation (RIS). In this paper, we argue that deep text instruction tuning is key to mitigate such shortcoming caused by the shallow fusion scheme in its default light-weight mask decoder. To address this issue, we propose two simple yet effective deep instruction tuning (DIT) methods for SAM, one is end-to-end and the other is layer-wise. With minimal modifications, DITs can directly transform the image encoder of SAM as a stand-alone vision-language learner in contrast to building another deep fusion branch, maximizing the benefit of its superior segmentation capability. Extensive experiments on three highly competitive benchmark datasets of RIS show that a simple end-to-end DIT can improve SAM by a large margin, while the layer-wise DIT can further boost the performance to state-of-the-art with much less data and training expenditures. Our code is released at: https://github.com/wysnzzzz/DIT.

We present a framework, DISORF, to enable online 3D reconstruction and visualization of scenes captured by resource-constrained mobile robots and edge devices. To address the limited computing capabilities of edge devices and potentially limited network availability, we design a framework that efficiently distributes computation between the edge device and the remote server. We leverage on-device SLAM systems to generate posed keyframes and transmit them to remote servers that can perform high-quality 3D reconstruction and visualization at runtime by leveraging recent advances in neural 3D methods. We identify a key challenge with online training where naive image sampling strategies can lead to significant degradation in rendering quality. We propose a novel shifted exponential frame sampling method that addresses this challenge for online training. We demonstrate the effectiveness of our framework in enabling high-quality real-time reconstruction and visualization of unknown scenes as they are captured and streamed from cameras in mobile robots and edge devices.

Hepatoid carcinoma is an extrahepatic primary tumor displaying characteristics reminiscent of hepatocellular carcinoma differentiation, which is found in various organs, such as the stomach, ovaries, gallbladder, and pancreas. Reports of pancreatic hepatoid carcinoma remain scarce. Consequently, understanding of this disease remains a priority, with no established consensus on its diagnosis and management. Here, we reported the case of a 45-year-old woman diagnosed with hepatoid carcinoma located in the pancreatic head, accompanied by multiple lymph node metastases. Following pancreaticoduodenectomy, the patient developed liver metastases within 3 months. Subsequently, she underwent adjuvant therapy consisting of Teysuno and Durvalumab following microwave ablation for the liver metastases. Remarkably, the patient has survived for one year without significant disease progression. This case underscores the potential efficacy of immunotherapy as a promising treatment option for pancreatic hepatoid carcinoma. Further research and clinical trials are warranted to explore the optimal management strategies for this rare and challenging malignancy., Competing Interests: The authors declare that the research was conducted in the absence of any commercial or financial relationships that could be construed as a potential conflict of interest., (Copyright © 2024 Huang, Guo and Yu.)