Search

Objectives Currently available computer-aided diagnosis (CAD) systems for the detection of anti-nuclear antibodies (ANA) by indirect immunofluorescence (IIF) assay enable a standardized measurement of system-specific fluorescent intensity (FI) measures. We aimed to evaluate an internal quality control (iQC) program that controls the total ANA IIF process in routine practice. Methods In addition to the kit iQC materials, supplemental quality indicators were integrated in a total quality assurance (QA) program: patient-derived iQC’s samples (negative, 1/160 fine speckled and 1/160 homogeneous), median sample FI per run and percentage of ANA IIF positive samples per run. Analytical rejection criteria were based on the imprecision of the positivity index (PI) measure of the Zenit PRO system (Menarini). Clinical rejection criteria were based on changes in FI that correspond to a change in ANA IIF titer of ≥2. To evaluate the QA program, different artificial errors were introduced during the ANA IIF process. After every run, quality indicators were evaluated and compared to the pre-set target values. Results Rescanning the ANA IIF slides five times, using an old conjugate and a needle obstruction resulted in analytically and even clinically relevant errors in ANA IIF results. All errors were correctly detected by the different defined quality indicators. Traditional Westgard rules, including analytically (and clinically) defined rejection limits were useful in monitoring quality indicators. Conclusions The integration of a total process iQC program in CAD systems, based on the specific FI measurands and performance criteria of the system, adds value to QA.

The Compact Muon Solenoid (CMS) experiment at the European Organization for Nuclear Research (CERN) deploys its data collections, simulation and analysis activities on a distributed computing infrastructure involving more than 70 sites worldwide. The historical usage data recorded by this large infrastructure is a rich source of information for system tuning and capacity planning. In this paper we investigate how to leverage machine learning on this huge amount of data in order to discover patterns and correlations useful to enhance the overall efficiency of the distributed infrastructure in terms of CPU utilization and task completion time. In particular we propose a scalable pipeline of components built on top of the Spark engine for large-scale data processing, whose goal is collecting from different sites the dataset access logs, organizing them into weekly snapshots, and training, on these snapshots, predictive models able to forecast which datasets will become popular over time. The high accuracy achieved indicates the ability of the learned model to correctly separate popular datasets from unpopular ones. Dataset popularity predictions are then exploited within a novel data caching policy, called PPC (Popularity Prediction Caching). We evaluate the performance of PPC against popular caching policy baselines like LRU (Least Recently Used). The experiments conducted on large traces of real dataset accesses show that PPC outperforms LRU reducing the number of cache misses up to 20% in some sites.

The CERN IT provides a set of Hadoop clusters featuring more than 5 PBytes of raw storage with different open-source, user-level tools available for analytical purposes. The CMS experiment started collecting a large set of computing meta-data, e.g. dataset, file access logs, since 2015. These records represent a valuable, yet scarcely investigated, set of information that needs to be cleaned, categorized and analyzed. CMS can use this information to discover useful patterns and enhance the overall efficiency of the distributed data, improve CPU and site utilization as well as tasks completion time. Here we present evaluation of Apache Spark platform for CMS needs. We discuss two main use-cases CMS analytics and ML studies where efficient process billions of records stored on HDFS plays an important role. We demonstrate that both Scala and Python (PySpark) APIs can be successfully used to execute extremely I/O intensive queries and provide valuable data insight from collected meta-data., Submitted to ACAT 2017 conference proceedings

Performance data and metadata of the computing operations at the CMS experiment are collected through a distributed monitoring infrastructure, currently relying on a traditional Oracle database system. This paper shows how to harness Big Data architectures in order to improve the throughput and the efficiency of such monitoring. A large set of operational data - user activities, job submissions, resources, file transfers, site efficiencies, software releases, network traffic, machine logs - is being injected into a readily available Hadoop cluster, via several data streamers. The collected metadata is further organized running fast arbitrary queries; this offers the ability to test several Map&Reduce-based; frameworks and measure the system speed-up when compared to the original database infrastructure. By leveraging a quality Hadoop data store and enabling an analytics framework on top, it is possible to design a mining platform to predict dataset popularity and discover patterns and correlations.

Grid computing has the intrinsic disadvantages of a batch system: the undetermined delay between the time a job is submitted and the time it is completed. Our approach to this problem is iGrid, a framework that proposes to change the Grid from a batch system to a more interactive distributed platform. Agents are deployed on nodes and wait to be contacted by an interactive job to start the computation, thus bypassing the regular Grid scheduler. A CPU fairshare mechanism allows users to get their fair iGrid machine share over a long period. Our prototype implementation scales to all 400 Grid nodes we were granted executing on top a software for data intensive parallel computing.

Fluorescence in situ hybridization (FISH) is a cytogenetic technique used to detect the presence of specific DNA sequences on chromosomes. An application of FISH targets the measurement of the amplification of the HER-2 gene within the chromosomes. This requires computation of the ratio of HER-2 over centromere 17 (CEP-17) markers detected within a representative number of nuclei. Selection of the nuclei that are used to compute this ratio is a crucial step to yield an accurate estimation of HER-2 amplification. In fact, errors deriving from overlapping, clustered and oversegmented nuclei can bias the estimation of the ratio and yield inaccurate evaluation of HER-2 amplification. In this paper, a model is presented to associate a reliability score with each nucleus. The reliability score enables refinement of segmentation results—yielding more accurate identification of the boundary of each nucleus—as well as ranking of detected nuclei so as to use just the most reliable nuclei to compute the ratio. Experimental results demonstrate that the adoption of the proposed model improves the estimation of HER-2 amplification.

In this technical demonstration we show an integrated web system for video search and annotation based on ontologies. The system is composed by three components: the Orione ontology-based search engine, the Sirio\footnote{Sirio was the hound of Orione. It was a dog so swift that no prey could escape it.} search interface, and the Pan web-based video annotation tool. The system is currently being developed within the EU IM3I project. The goal of the system is to provide an integrated environment for video annotation and retrieval of videos, for both technical and non-technical users. In fact, the search engine has different interfaces that permit different query modalities: free-text, natural language, graphical composition of concepts using Boolean and temporal relations and query by visual example. In addition, the ontology structure is exploited to encode semantic relations between concepts permitting, for example, to expand queries to synonyms and concept specializations. The annotation tool can be used to create ground-truth annotations to train automatic annotations systems, or to complement the results of automatic annotation, e.g. adding geolocalized information.

The ALICE experiment at CERN LHC is using a PROOF-enabled cluster for fast physics analysis, detector calibration and reconstruction of small data samples. The current system (CAF - CERN Analysis Facility) consists of some 120 CPU cores and about 45 TB of disk space distributed across the CAF hosts. One of the most important aspects of the data analysis on the CAF is the speed with which it can be carried out. The system is particularly aimed at the prototyping phase of analyses that need a high number of iterations and thus require a short response time. Quasi-online quality assurance of data can be obtained. The paper describes the design principles of the PROOF frame work and presents the current setup, performance tests and usage statistics. Subsets of selected data can be automatically staged in CAF from the Grid storage systems, therefore data distribution and staging techniques are described in depth. A fairshare algorithm to adjust the priorities of concurrently running sessions is also examined. Further more, the adaptation of PROOF to the AliEn/gLite Grid middleware is described. This approach enables a dynamic startup of PROOF nodes worldwide with the purpose to process much larger physics datasets.

The HLT (High-Level Trigger) group of the ALICE experiment at the LHC has prepared a virtual Parallel ROOT Facility (PROOF) enabled cluster (HAF - HLT Analysis Facility) for fast physics analysis, detector calibration and reconstruction of data samples. The HLT-Cluster currently consists of 2860 CPU cores and 175TB of storage. Its purpose is the online filtering of the relevant part of data produced by the particle detector. However, data taking is not running continuously and exploiting unused cluster resources for other applications is highly desirable and improves the usage-cost ratio of the HLT cluster. As such, unused computing resources are dedicated to a PROOF-enabled virtual cluster available to the entire collaboration. This setup is especially aimed at the prototyping phase of analyses that need a high number of development iterations and a short response time, e. g. tuning of analysis cuts, calibration and alignment. HAF machines are enabled and disabled upon user request to start or complete analysis tasks. This is achieved by a virtual machine scheduling framework which dynamically assigns and migrates virtual machines running PROOF workers to unused physical resources. Using this approach we extend the HLT usage scheme to running both online and offline computing, thereby optimizing the resource usage.

The ALICE experiment at CERN LHC is intensively using a PROOF cluster for fast analysis and reconstruction. The current system (CAF - CERN Analysis Facility) consists of 120 CPU cores and about 45 TB of local space. PROOF enables interactive parallel processing of data distributed on clusters Of computers or multi-core machines. Subsets of selected data are automatically staged onto CAF from the Grid storage systems. However, a cluster of the size of CAF can only hold a fraction of the yearly 3 PB data accumulated by ALICE. The impracticability to store and process such data volume in one single computing centre leads to the need to extend the concept of PROOF to the Grid paradigm.