Search

Purpose: Federated training is often hindered by heterogeneous datasets due to divergent data storage options, inconsistent naming schemes, varied annotation procedures, and disparities in label quality. This is particularly evident in the emerging multi-modal learning paradigms, where dataset harmonization including a uniform data representation and filtering options are of paramount importance. Methods: DICOM structured reports enable the standardized linkage of arbitrary information beyond the imaging domain and can be used within Python deep learning pipelines with highdicom. Building on this, we developed an open platform for data integration and interactive filtering capabilities that simplifies the process of assembling multi-modal datasets. Results: In this study, we extend our prior work by showing its applicability to more and divergent data types, as well as streamlining datasets for federated training within an established consortium of eight university hospitals in Germany. We prove its concurrent filtering ability by creating harmonized multi-modal datasets across all locations for predicting the outcome after minimally invasive heart valve replacement. The data includes DICOM data (i.e. computed tomography images, electrocardiography scans) as well as annotations (i.e. calcification segmentations, pointsets and pacemaker dependency), and metadata (i.e. prosthesis and diagnoses). Conclusion: Structured reports bridge the traditional gap between imaging systems and information systems. Utilizing the inherent DICOM reference system arbitrary data types can be queried concurrently to create meaningful cohorts for clinical studies. The graphical interface as well as example structured report templates will be made publicly available.

Federated learning is a renowned technique for utilizing decentralized data while preserving privacy. However, real-world applications often face challenges like partially labeled datasets, where only a few locations have certain expert annotations, leaving large portions of unlabeled data unused. Leveraging these could enhance transformer architectures ability in regimes with small and diversely annotated sets. We conduct the largest federated cardiac CT analysis to date (n=8,104) in a real-world setting across eight hospitals. Our two-step semi-supervised strategy distills knowledge from task-specific CNNs into a transformer. First, CNNs predict on unlabeled data per label type and then the transformer learns from these predictions with label-specific heads. This improves predictive accuracy and enables simultaneous learning of all partial labels across the federation, and outperforms UNet-based models in generalizability on downstream tasks. Code and model weights are made openly available for leveraging future cardiac CT analysis.

We introduce the Riskman ontology & shapes for representing and analysing information about risk management for medical devices. Risk management is concerned with taking necessary precautions so a medical device does not cause harms for users or the environment. To date, risk management documentation is submitted to notified bodies (for certification) in the form of semi-structured natural language text. We propose to use classes from the Riskman ontology to logically model risk management documentation and to use the included SHACL constraints to check for syntactic completeness and conformity to relevant standards. In particular, the ontology is modelled after ISO 14971 and the recently published VDE Spec 90025. Our proposed methodology has the potential to save many person-hours for both manufacturers (when creating risk management documentation) as well as notified bodies (when assessing submitted applications for certification), and thus offers considerable benefits for healthcare and, by extension, society as a whole.

The term credential encompasses educational certificates, degrees, certifications, and government-issued licenses. An occupational credential is a verification of an individuals qualification or competence issued by a third party with relevant authority. Job seekers often leverage such credentials as evidence that desired qualifications are satisfied by their holders. Many U.S. education and workforce development organizations have recognized the importance of credentials for employment and the challenges of understanding the value of credentials. In this study, we identified and ontologically defined credential and credential-related terms at the textual and semantic levels based on the Occupation Ontology (OccO), a BFO-based ontology. Different credential types and their authorization logic are modeled. We additionally defined a high-level hierarchy of credential related terms and relations among many terms, which were initiated in concert with the Alabama Talent Triad (ATT) program, which aims to connect learners, earners, employers and education/training providers through credentials and skills. To our knowledge, our research provides for the first time systematic ontological modeling of the important domain of credentials and related contents, supporting enhanced credential data and knowledge integration in the future., Comment: 11

This study presents a multi-faceted approach combining stereotactic biopsy with standard clinical open-craniotomy for sample collection, voxel-wise analysis of MR images, regression-based Generalized Additive Models (GAM), & whole-exome sequencing. This work aims to demonstrate the potential of machine learning algorithms to predict variations in cellular & molecular tumor characteristics. This retrospective study enrolled ten treatment-naive patients with radiologically confirmed glioma (5 WHO grade II, 5 WHO grade IV). Each patient underwent a multiparametric MR scan (T1W, T1W-CE, T2W, T2W-FLAIR, DWI) prior to surgery (27.9+/-34.0 days). During standard craniotomy procedure, at least 1 stereotactic biopsy was collected from each patient, with screenshots of the sample locations saved for spatial registration to pre-surgical MR data. Whole-exome sequencing was performed on flash-frozen tumor samples, prioritizing the signatures of five glioma-related genes: IDH1, TP53, EGFR, PIK3CA, & NF1. Regression was implemented with a GAM using a univariate shape function for each predictor. Standard receiver operating characteristic analyses were used to evaluate detection, with AUC (area under curve) calculated for each gene target & MR contrast combination. The mean AUC for the five gene targets & 31 MR contrast combinations was 0.75+/-0.11; individual AUCs were as high as 0.96 for both IDH1 & TP53 with T2W-FLAIR & ADC & 0.99 for EGFR with T2W & ADC. An average AUC of 0.85 across the five mutations was achieved using the combination of T1W, T2W-FLAIR, & ADC. These results suggest the possibility of predicting exome-wide mutation events from non-invasive, in vivo imaging by combining stereotactic localization of glioma samples & a semi-parametric deep learning method. This approach holds potential for refining targeted therapy by better addressing the genomic heterogeneity of glioma tumors., Comment: Keywords: multiparametric MRI, intra-tumoral heterogeneity, machine learning, brain tumor, stereotactic biopsy. Note: A portion of this study was presented as a talk in RSNA (Radiological Society of North America) 2021 and was awarded the RSNA Chynn Award for Neuroradiology Research

We propose CG-HOI, the first method to address the task of generating dynamic 3D human-object interactions (HOIs) from text. We model the motion of both human and object in an interdependent fashion, as semantically rich human motion rarely happens in isolation without any interactions. Our key insight is that explicitly modeling contact between the human body surface and object geometry can be used as strong proxy guidance, both during training and inference. Using this guidance to bridge human and object motion enables generating more realistic and physically plausible interaction sequences, where the human body and corresponding object move in a coherent manner. Our method first learns to model human motion, object motion, and contact in a joint diffusion process, inter-correlated through cross-attention. We then leverage this learned contact for guidance during inference to synthesize realistic and coherent HOIs. Extensive evaluation shows that our joint contact-based human-object interaction approach generates realistic and physically plausible sequences, and we show two applications highlighting the capabilities of our method. Conditioned on a given object trajectory, we can generate the corresponding human motion without re-training, demonstrating strong human-object interdependency learning. Our approach is also flexible, and can be applied to static real-world 3D scene scans., Comment: Project page: https://cg-hoi.christian-diller.de Video: https://www.youtube.com/watch?v=GNyQwTwZ15s

Objective: To create a commons for infectious disease (ID) epidemiology in which epidemiologists, public health officers, data producers, and software developers can not only share data and software, but receive assistance in improving their interoperability. Materials and Methods: We represented 586 datasets, 54 software, and 24 data formats in OWL 2 and then used logical queries to infer potentially interoperable combinations of software and datasets, as well as statistics about the FAIRness of the collection. We represented the objects in DATS 2.2 and a software metadata schema of our own design. We used these representations as the basis for the Content, Search, FAIR-o-meter, and Workflow pages that constitute the MIDAS Digital Commons. Results: Interoperability was limited by lack of standardization of input and output formats of software. When formats existed, they were human-readable specifications (22/24; 92%); only 3 formats (13%) had machine-readable specifications. Nevertheless, logical search of a triple store based on named data formats was able to identify scores of potentially interoperable combinations of software and datasets. Discussion: We improved the findability and availability of a sample of software and datasets and developed metrics for assessing interoperability. The barriers to interoperability included poor documentation of software input/output formats and little attention to standardization of most types of data in this field. Conclusion: Centralizing and formalizing the representation of digital objects within a commons promotes FAIRness, enables its measurement over time and the identification of potentially interoperable combinations of data and software., Comment: 12 pages, 6 figures

Miniature magnetic tools have the potential to enable minimally invasive surgical techniques to be applied to space-restricted surgical procedures in areas such as neurosurgery. However, typical magnetic navigation systems, which create the magnetic fields to drive such tools, either cannot generate large enough fields, or surround the patient in a way that obstructs surgeon access to the patient. This paper introduces the design of a magnetic navigation system with eight electromagnets arranged completely under the operating table, to endow the system with maximal workspace accessibility, which allows the patient to lie down on the top surface of the system without any constraints. The found optimal geometric layout of the electromagnets maximizes the field strength and uniformity over a reasonable neurosurgical operating volume. The system can generate non-uniform magnetic fields up to 38 mT along the x and y axes and 47 mT along the z axis at a working distance of 120 mm away from the actuation system workbench, deep enough to deploy magnetic microsurgical tools in the brain. The forces which can be exerted on millimeter-scale magnets used in prototype neurosurgical tools are validated experimentally. Due to its large workspace, this system could be used to control milli-robots in a variety of surgical applications.

We prove an equidistribution result for iterated preimages of curves by a large class of rational maps $f:\mathbb{CP}^2\dashrightarrow\mathbb{CP}^2$ that cannot be birationally conjugated to algebraically stable maps. The maps, which include recent examples with transcendental first dynamical degree, are distinguished by the fact that they have constant Jacobian determinant relative to the natural holomorphic two form on the algebraic torus. Under the additional hypothesis that $f$ has "small topological degree'' we also prove an equidistribution result for iterated forward images of curves. To prove our results we systematically develop the idea of a positive closed $(1,1)$ current and its cohomology class on the inverse limit of all toric surfaces. This, in turn, relies upon a careful study of positive closed $(1,1)$ currents on individual toric surfaces. This framework may be useful in other contexts., Comment: To appear in Commentarii Mathematici Helvetici. 57 pages. Changes made to address referee report and to add some references. Comments welcome

We present a generative approach to forecast long-term future human behavior in 3D, requiring only weak supervision from readily available 2D human action data. This is a fundamental task enabling many downstream applications. The required ground-truth data is hard to capture in 3D (mocap suits, expensive setups) but easy to acquire in 2D (simple RGB cameras). Thus, we design our method to only require 2D RGB data at inference time while being able to generate 3D human motion sequences. We use a differentiable 2D projection scheme in an autoregressive manner for weak supervision, and an adversarial loss for 3D regularization. Our method predicts long and complex human behavior sequences (e.g., cooking, assembly) consisting of multiple sub-actions. We tackle this in a semantically hierarchical manner, jointly predicting high-level coarse action labels together with their low-level fine-grained realizations as characteristic 3D human poses. We observe that these two action representations are coupled in nature, and joint prediction benefits both action and pose forecasting. Our experiments demonstrate the complementary nature of joint action and 3D pose prediction: our joint approach outperforms each task treated individually, enables robust longer-term sequence prediction, and improves over alternative approaches to forecast actions and characteristic 3D poses., Comment: Project Page: https://future-human-3d.christian-diller.de/ Video: https://www.youtube.com/watch?v=18du85YFXL0

Multi-agent systems (MAS) may encounter uncertainties in the form of unexpected environmental conditions, sub-optimal system configurations, and unplanned interactions between autonomous agents. The number of combinations of such uncertainties may be innumerable, however run-time testing may reduce the issues impacting such a system. We posit that search heuristics can augment a run-time testing process, in-situ, for a MAS. To support our position we discuss our in-progress experimental testbed to realize this goal and highlight challenges we anticipate for this domain.

We give examples of birational selfmaps of $\mathbb{P}^d, d \geq 3$, whose dynamical degree is a transcendental number. This contradicts a conjecture by Bellon and Viallet. The proof uses a combination of techniques from algebraic dynamics and diophantine approximation., Comment: To appear in Proc. Lond. Math. Soc

We propose the task of forecasting characteristic 3d poses: from a short sequence observation of a person, predict a future 3d pose of that person in a likely action-defining, characteristic pose -- for instance, from observing a person picking up an apple, predict the pose of the person eating the apple. Prior work on human motion prediction estimates future poses at fixed time intervals. Although easy to define, this frame-by-frame formulation confounds temporal and intentional aspects of human action. Instead, we define a semantically meaningful pose prediction task that decouples the predicted pose from time, taking inspiration from goal-directed behavior. To predict characteristic poses, we propose a probabilistic approach that models the possible multi-modality in the distribution of likely characteristic poses. We then sample future pose hypotheses from the predicted distribution in an autoregressive fashion to model dependencies between joints. To evaluate our method, we construct a dataset of manually annotated characteristic 3d poses. Our experiments with this dataset suggest that our proposed probabilistic approach outperforms state-of-the-art methods by 26% on average., Comment: CVPR 2022; Project Page: https://charposes.christian-diller.de/; Paper Video: https://youtu.be/kVhn8OWMgME

From disinfection and remote triage, to logistics and delivery, countries around the world are making use of robots to address the unique challenges presented by the COVID-19 pandemic. Robots are being used to manage the pandemic in Canada too, but relative to other regions, we have been more cautious in our adoption -- this despite the important role that robots of Canadian origin are now playing on the global stage. This white paper discusses why this is the case, and argues that strategic investment and support for the Canadian robotics industry are urgently needed to bring the benefits of robotics home, where we have more control in shaping the future of this game-changing technology. Such investments will not only serve to support Canada's current pandemic response and post pandemic recovery, but will also prepare this country to weather future crises. Without such support, Canada risks falling behind other developed nations that are investing heavily in hardware automation at this time., Comment: 104 pages, 18 figures, 13 tables. Corresponding Author: H Siegel

Glioblastoma multiform carries a dismal prognosis with poor response to gold standard treatment. Innovative data analysis methods have been developed to characterize tumor genomic expression with histologic features. In a clinical setting, biopsy selection methods may be constrained by time and financial burden to the patient. Thus, we investigate the impact biopsy selection has on correlation between prognostic and histologic features in 35 patients with GBM. We compared methods using limited volumes, moderate volumes, and enblock tumor volumes. Additionally, we investigated the impact of random versus strategic methods for limited and moderate volume biopsies. Finally, we compared correlation results by selecting one to five small biopsy. We observed a wide range in correlation significance across selection methods. These findings may aid clinical management of GBM and direct better biopsy selection necessary for the development and deployment of targeted therapies., Comment: 9 Pages, 2 figures, 1 table

Powerful formalisms for abstract argumentation have been proposed, among them abstract dialectical frameworks (ADFs) that allow for a succinct and flexible specification of the relationship between arguments, and the GRAPPA framework which allows argumentation scenarios to be represented as arbitrary edge-labelled graphs. The complexity of ADFs and GRAPPA is located beyond NP and ranges up to the third level of the polynomial hierarchy. The combined complexity of Answer Set Programming (ASP) exactly matches this complexity when programs are restricted to predicates of bounded arity. In this paper, we exploit this coincidence and present novel efficient translations from ADFs and GRAPPA to ASP. More specifically, we provide reductions for the five main ADF semantics of admissible, complete, preferred, grounded, and stable interpretations, and exemplify how these reductions need to be adapted for GRAPPA for the admissible, complete and preferred semantics. Under consideration in Theory and Practice of Logic Programming (TPLP)., Comment: Under consideration in Theory and Practice of Logic Programming (TPLP)

We present a novel approach that converts partial and noisy RGB-D scans into high-quality 3D scene reconstructions by inferring unobserved scene geometry. Our approach is fully self-supervised and can hence be trained solely on real-world, incomplete scans. To achieve self-supervision, we remove frames from a given (incomplete) 3D scan in order to make it even more incomplete; self-supervision is then formulated by correlating the two levels of partialness of the same scan while masking out regions that have never been observed. Through generalization across a large training set, we can then predict 3D scene completion without ever seeing any 3D scan of entirely complete geometry. Combined with a new 3D sparse generative neural network architecture, our method is able to predict highly-detailed surfaces in a coarse-to-fine hierarchical fashion, generating 3D scenes at 2cm resolution, more than twice the resolution of existing state-of-the-art methods as well as outperforming them by a significant margin in reconstruction quality., Comment: CVPR 2020; Video: https://youtu.be/rN6D3QmMNuU

Background and Purpose: Biopsy is the main determinants of glioma clinical management, but require invasive sampling that fail to detect relevant features because of tumor heterogeneity. The purpose of this study was to evaluate the accuracy of a voxel-wise, multiparametric MRI radiomic method to predict features and develop a minimally invasive method to objectively assess neoplasms. Methods: Multiparametric MRI were registered to T1-weighted gadolinium contrast-enhanced data using a 12 degree-of-freedom affine model. The retrospectively collected MRI data included T1-weighted, T1-weighted gadolinium contrast-enhanced, T2-weighted, fluid attenuated inversion recovery, and multi-b-value diffusion-weighted acquired at 1.5T or 3.0T. Clinical experts provided voxel-wise annotations for five disease states on a subset of patients to establish a training feature vector of 611,930 observations. Then, a k-nearest-neighbor (k-NN) classifier was trained using a 25% hold-out design. The trained k-NN model was applied to 13,018,171 observations from seventeen histologically confirmed glioma patients. Linear regression tested overall survival (OS) relationship to predicted disease compositions (PDC) and diagnostic age (alpha = 0.05). Canonical discriminant analysis tested if PDC and diagnostic age could differentiate clinical, genetic, and microscopic factors (alpha = 0.05). Results: The model predicted voxel annotation class with a Dice similarity coefficient of 94.34% +/- 2.98. Linear combinations of PDCs and diagnostic age predicted OS (p = 0.008), grade (p = 0.014), and endothelia proliferation (p = 0.003); but fell short predicting gene mutations for TP53BP1 and IDH1. Conclusions: This voxel-wise, multi-parametric MRI radiomic strategy holds potential as a non-invasive decision-making aid for clinicians managing patients with glioma., Comment: 13 pages, 3 figures, 5 tables

We propose a statistical multiscale mapping approach to identify microscopic and molecular heterogeneity across a tumor microenvironment using multiparametric MR (mp-MR). Twenty-nine patients underwent pre-surgical mp-MR followed by MR-guided stereotactic core biopsy. The locations of the biopsy cores were identified in the pre-surgical images using stereotactic bitmaps acquired during surgery. Feature matrices mapped the multiparametric voxel values in the vicinity of the biopsy cores to the pathologic outcome variables for each patient and logistic regression tested the individual and collective predictive power of the MR contrasts. A non-parametric weighted k-nearest neighbor classifier evaluated the feature matrices in a leave-one-out cross validation design across patients. Resulting class membership probabilities were converted to chi-square statistics to develop full-brain parametric maps, implementing Gaussian random field theory to estimate inter-voxel dependencies. Corrections for family-wise error rates were performed using Benjamini-Hochberg and random field theory, and the resulting accuracies were compared. The combination of all five image contrasts correlated with outcome (P<.001) for all four microscopic variables. The probabilistic mapping method using Benjamini-Hochberg generated statistically significant results (P<.05) for three of the four dependent variables: 1) IDH1, 2) MGMT, and 3) microvascular proliferation, with an average classification accuracy of 0.984 +/- 0.02 and an average classification sensitivity of 1.567% +/- 0.967. The images corrected by random field theory demonstrated improved classification accuracy (0.989 +/- 0.008) and classification sensitivity (5.967% +/- 2.857) compared with Benjamini-Hochberg. Microscopic and molecular tumor properties can be assessed with statistical confidence across the brain from minimally-invasive, mp-MR.

We give an example of a dominant rational selfmap of the projective plane whose dynamical degree is a transcendental number., Comment: 26 pages. Exposition has been changed after receiving a careful referee report. To appear in Acta Math