Search

Meeting the requirements of future services with time sensitivity and handling sudden load spikes of the services in Fog computing environments are challenging tasks due to the lack of publicly available Fog nodes and their characteristics. Researchers have assumed that the traditional autoscaling techniques, with lightweight virtualisation technology (containers), can be used to provide autoscaling features in Fog computing environments, few researchers have built the platform by exploiting the default autoscaling techniques of the containerisation orchestration tools or systems. However, the adoption of these techniques alone, in a publicly available Fog infrastructure, does not guarantee Quality of Service (QoS) due to the heterogeneity of Fog devices and their characteristics, such as frequent resource changes and high mobility. To tackle this challenge, in this work we developed a Fog as a Service (FaaS) framework that can create, configure and manage the containers which are running on the Fog devices to deploy services. This work presents the key techniques and algorithms which are responsible for handling sudden load spikes of the services to meet the QoS of the application. This work provides an evaluation by comparing it with existing techniques under real scenarios. The experiment results show that our proposed approach maximises the satisfied service requests by an average of 1.9 times in different scenarios., Comment: 10 Pages, 13 Figures

Objective The coordination of human movement directly reflects function of the central nervous system. Small deficits in movement are often the first sign of an underlying neurological problem. The objective of this research is to develop a new end-to-end, deep learning-based system, Rapid-Motion-Track (RMT) that can track the fastest human movement accurately when webcams or laptop cameras are used. Materials and Methods We applied RMT to finger tapping, a well-validated test of motor control that is one of the most challenging human motions to track with computer vision due to the small keypoints of digits and the high velocities that are generated. We recorded 160 finger tapping assessments simultaneously with a standard 2D laptop camera (30 frames/sec) and a high-speed wearable sensor-based 3D motion tracking system (250 frames/sec). RMT and a range of DLC models were applied to the video data with tapping frequencies up to 8Hz to extract movement features. Results The movement features (e.g. speed, rhythm, variance) identified with the new RMT system exhibited very high concurrent validity with the gold-standard measurements (97.3\% of RMT measures were within +/-0.5Hz of the Optotrak measures), and outperformed DLC and other advanced computer vision tools (around 88.2\% of DLC measures were within +/-0.5Hz of the Optotrak measures). RMT also accurately tracked a range of other rapid human movements such as foot tapping, head turning and sit-to -stand movements. Conclusion: With the ubiquity of video technology in smart devices, the RMT method holds potential to transform access and accuracy of human movement assessment.

Open vocabulary models (e.g. CLIP) have shown strong performance on zero-shot classification through their ability generate embeddings for each class based on their (natural language) names. Prior work has focused on improving the accuracy of these models through prompt engineering or by incorporating a small amount of labeled downstream data (via finetuning). However, there has been little focus on improving the richness of the class names themselves, which can pose issues when class labels are coarsely-defined and are uninformative. We propose Classification with Hierarchical Label Sets (or CHiLS), an alternative strategy for zero-shot classification specifically designed for datasets with implicit semantic hierarchies. CHiLS proceeds in three steps: (i) for each class, produce a set of subclasses, using either existing label hierarchies or by querying GPT-3; (ii) perform the standard zero-shot CLIP procedure as though these subclasses were the labels of interest; (iii) map the predicted subclass back to its parent to produce the final prediction. Across numerous datasets with underlying hierarchical structure, CHiLS leads to improved accuracy in situations both with and without ground-truth hierarchical information. CHiLS is simple to implement within existing zero-shot pipelines and requires no additional training cost. Code is available at: https://github.com/acmi-lab/CHILS., Comment: Accepted at ICML 2023

An authorisation has been recognised as an important security measure for preventing unauthorised access to critical resources, such as devices and data, within the Internet of Things (IoT) networks. Existing authorisation methods for the IoT network are based on traditional access control models, which have several drawbacks, including architecture centralisation, policy tampering, access rights validation, malicious third-party policy assignment and control, and network-related overheads. The increasing trend of integrating Blockchain technology with IoT networks demonstrates its importance and potential to address the shortcomings of traditional IoT network authorisation mechanisms. This paper proposes a decentralised, secure, dynamic, and flexible authorisation scheme for IoT networks based on attribute-based access control (ABAC) fine-grained policies stored on a distributed immutable ledger. We design a Blockchain-based ABAC policy management framework divided into Attribute Management Authority (AMA) and Policy Management Authority (PMA) frameworks that use smart contract features to initialise, store, and manage attributes and policies on the Blockchain. To achieve flexibility and dynamicity in the authorisation process, we capture and utilise the environmental-related attributes in conjunction with the subject and object attributes of the ABAC model to define the policies. Furthermore, we designed the Blockchain-based Access Management Framework (AMF) to manage user requests to access IoT devices while maintaining the privacy and auditability of user requests and assigned policies. We implemented a prototype of our proposed scheme and executed it on the local Ethereum Blockchain. Finally, we demonstrated the applicability and flexibility of our proposed scheme for an IoT-based smart home scenario, taking into account deployment, execution and financial costs.

In machine learning, we traditionally evaluate the performance of a single model, averaged over a collection of test inputs. In this work, we propose a new approach: we measure the performance of a collection of models when evaluated on a $\textit{single input point}$. Specifically, we study a point's $\textit{profile}$: the relationship between models' average performance on the test distribution and their pointwise performance on this individual point. We find that profiles can yield new insights into the structure of both models and data -- in and out-of-distribution. For example, we empirically show that real data distributions consist of points with qualitatively different profiles. On one hand, there are "compatible" points with strong correlation between the pointwise and average performance. On the other hand, there are points with weak and even $\textit{negative}$ correlation: cases where improving overall model accuracy actually $\textit{hurts}$ performance on these inputs. We prove that these experimental observations are inconsistent with the predictions of several simplified models of learning proposed in prior work. As an application, we use profiles to construct a dataset we call CIFAR-10-NEG: a subset of CINIC-10 such that for standard models, accuracy on CIFAR-10-NEG is $\textit{negatively correlated}$ with accuracy on CIFAR-10 test. This illustrates, for the first time, an OOD dataset that completely inverts "accuracy-on-the-line" (Miller, Taori, Raghunathan, Sagawa, Koh, Shankar, Liang, Carmon, and Schmidt 2021), Comment: GK and NG contributed equally. v2: Added Figures 4, 5

Researchers investigating example hardness have increasingly focused on the dynamics by which neural networks learn and forget examples throughout training. Popular metrics derived from these dynamics include (i) the epoch at which examples are first correctly classified; (ii) the number of times their predictions flip during training; and (iii) whether their prediction flips if they are held out. However, these metrics do not distinguish among examples that are hard for distinct reasons, such as membership in a rare subpopulation, being mislabeled, or belonging to a complex subpopulation. In this paper, we propose $second$-$split$ $forgetting$ $time$ (SSFT), a complementary metric that tracks the epoch (if any) after which an original training example is forgotten as the network is fine-tuned on a randomly held out partition of the data. Across multiple benchmark datasets and modalities, we demonstrate that $mislabeled$ examples are forgotten quickly, and seemingly $rare$ examples are forgotten comparatively slowly. By contrast, metrics only considering the first split learning dynamics struggle to differentiate the two. At large learning rates, SSFT tends to be robust across architectures, optimizers, and random seeds. From a practical standpoint, the SSFT can (i) help to identify mislabeled samples, the removal of which improves generalization; and (ii) provide insights about failure modes. Through theoretical analysis addressing overparameterized linear models, we provide insights into how the observed phenomena may arise. Code for reproducing our experiments can be found here: https://github.com/pratyushmaini/ssft, Comment: Accepted at NeurIPS 2022

Additional file 1. Questionnaire Series of questions that participants are invited to complete at the end of TAS Test.

For most natural language processing tasks, the dominant practice is to finetune large pretrained transformer models (e.g., BERT) using smaller downstream datasets. Despite the success of this approach, it remains unclear to what extent these gains are attributable to the massive background corpora employed for pretraining versus to the pretraining objectives themselves. This paper introduces a large-scale study of self-pretraining, where the same (downstream) training data is used for both pretraining and finetuning. In experiments addressing both ELECTRA and RoBERTa models and 10 distinct downstream classification datasets, we observe that self-pretraining rivals standard pretraining on the BookWiki corpus (despite using around $10\times$--$500\times$ less data), outperforming the latter on $7$ and $5$ datasets, respectively. Surprisingly, these task-specific pretrained models often perform well on other tasks, including the GLUE benchmark. Besides classification tasks, self-pretraining also provides benefits on structured output prediction tasks such as span based question answering and commonsense inference, often providing more than $50\%$ of the performance boosts provided by pretraining on the BookWiki corpus. Our results hint that in many scenarios, performance gains attributable to pretraining are driven primarily by the pretraining objective itself and are not always attributable to the use of external pretraining data in massive amounts. These findings are especially relevant in light of concerns about intellectual property and offensive content in web-scale pretraining data., Comment: ACL2023 Camera Ready

Real-world machine learning deployments are characterized by mismatches between the source (training) and target (test) distributions that may cause performance drops. In this work, we investigate methods for predicting the target domain accuracy using only labeled source data and unlabeled target data. We propose Average Thresholded Confidence (ATC), a practical method that learns a threshold on the model's confidence, predicting accuracy as the fraction of unlabeled examples for which model confidence exceeds that threshold. ATC outperforms previous methods across several model architectures, types of distribution shifts (e.g., due to synthetic corruptions, dataset reproduction, or novel subpopulations), and datasets (Wilds, ImageNet, Breeds, CIFAR, and MNIST). In our experiments, ATC estimates target performance $2$-$4\times$ more accurately than prior methods. We also explore the theoretical foundations of the problem, proving that, in general, identifying the accuracy is just as hard as identifying the optimal predictor and thus, the efficacy of any method rests upon (perhaps unstated) assumptions on the nature of the shift. Finally, analyzing our method on some toy distributions, we provide insights concerning when it works. Code is available at https://github.com/saurabhgarg1996/ATC_code/., Comment: Accepted at ICLR 2022

To assess generalization, machine learning scientists typically either (i) bound the generalization gap and then (after training) plug in the empirical risk to obtain a bound on the true risk; or (ii) validate empirically on holdout data. However, (i) typically yields vacuous guarantees for overparameterized models. Furthermore, (ii) shrinks the training set and its guarantee erodes with each re-use of the holdout set. In this paper, we introduce a method that leverages unlabeled data to produce generalization bounds. After augmenting our (labeled) training set with randomly labeled fresh examples, we train in the standard fashion. Whenever classifiers achieve low error on clean data and high error on noisy data, our bound provides a tight upper bound on the true risk. We prove that our bound is valid for 0-1 empirical risk minimization and with linear classifiers trained by gradient descent. Our approach is especially useful in conjunction with deep learning due to the early learning phenomenon whereby networks fit true labels before noisy labels but requires one intuitive assumption. Empirically, on canonical computer vision and NLP tasks, our bound provides non-vacuous generalization guarantees that track actual performance closely. This work provides practitioners with an option for certifying the generalization of deep nets even when unseen labeled data is unavailable and provides theoretical insights into the relationship between random label noise and generalization., Comment: ICML 2021 (Long Talk)

Fog computing is an emerging computing paradigm which is mainly suitable for time-sensitive and real-time Internet of Things (IoT) applications. Academia and industries are focusing on the exploration of various aspects of Fog computing for market adoption. The key idea of the Fog computing paradigm is to use idle computation resources of various handheld, mobile, stationery and network devices around us, to serve the application requests in the Fog-IoT environment. The devices in the Fog environment are autonomous and not exclusively dedicated to Fog application processing. Due to that, the probability of device failure in the Fog environment is high compared with other distributed computing paradigms. Solving failure issues in Fog is crucial because successful application execution can only be ensured if failure can be handled carefully. To handle failure, there are several techniques available in the literature, such as checkpointing and task migration, each of which works well in cloud based enterprise applications that mostly deals with static or transactional data. These failure handling methods are not applicable to highly dynamic Fog environment. In contrast, this work focuses on solving the problem of managing application failure in the Fog environment by proposing a composite solution (combining fuzzy logic-based task checkpointing and task migration techniques with task replication) for failure handling and generating a robust schedule. We evaluated the proposed methods using real failure traces in terms of application execution time, delay and cost. Average delay and total processing time improved by 56% and 48% respectively, on an average for the proposed solution, compared with the existing failure handling approaches., Comment: 12 Pages,12 Figures

Under label shift, the label distribution p(y) might change but the class-conditional distributions p(x|y) do not. There are two dominant approaches for estimating the label marginal. BBSE, a moment-matching approach based on confusion matrices, is provably consistent and provides interpretable error bounds. However, a maximum likelihood estimation approach, which we call MLLS, dominates empirically. In this paper, we present a unified view of the two methods and the first theoretical characterization of MLLS. Our contributions include (i) consistency conditions for MLLS, which include calibration of the classifier and a confusion matrix invertibility condition that BBSE also requires; (ii) a unified framework, casting BBSE as roughly equivalent to MLLS for a particular choice of calibration method; and (iii) a decomposition of MLLS's finite-sample error into terms reflecting miscalibration and estimation error. Our analysis attributes BBSE's statistical inefficiency to a loss of information due to coarse calibration. Experiments on synthetic data, MNIST, and CIFAR10 support our findings., Comment: Accepted at Neurips 2020

Emerging paradigms of big data and Software-Defined Networking (SDN) in cloud data centers have gained significant attention from industry and academia. The integration and coordination of big data and SDN are required to improve the application and network performance of big data applications. While empirical evaluation and analysis of big data and SDN can be one way of observing proposed solutions, it is often impractical or difficult to apply for several reasons, such as expensive undertakings, time consuming, and complexity; in addition, it is beyond the reach of most individuals. Thus, simulation tools are preferable options for performing costeffective, scalable experimentation in a controllable, repeatable, and configurable manner. To fill this gap, we present a new, self-contained simulation tool named BigDataSDNSim that enables the modeling and simulating of big data management systems (YARN), big data programming models (MapReduce), and SDN-enabled networks within cloud computing environments. To demonstrate the efficacy, effectiveness, and features of BigDataSDNSim, a use-case that compares SDN-enabled networks with legacy networks in terms of the performance and energy consumption is presented.

Stream workflow application such as online anomaly detection or online traffic monitoring, integrates multiple streaming big data applications into data analysis pipeline. This application can be highly dynamic in nature, where the data velocity may change at runtime and therefore the resources should be managed overtime. To manage these changes, the orchestration of this application requires a dynamic execution environment and dynamic scheduling technique. For the former requirement, Multicloud environment is a visible solution to cope with the dynamic aspects of this workflow application. While for the latter requirement, dynamic scheduling technique not only need to adhere to end user's requirements in terms of data processing and deadline for decision making, and data stream sources location constraints, but also adjust provisioning and scheduling plan at runtime to cope with dynamic variations of stream data rates. Therefore, we propose a two-phase adaptive scheduling technique to efficiently schedule dynamic workflow application in Multicloud environment that can respond to changes in the velocity of data at runtime. The experimental results showed that the proposed technique is close to the lower bound and effective for different experiment scenarios., Comment: 17 pages, 11 figures

This Perspective, based partly on the author's personal experiences with the Indian Model Bilateral Investment Treaty reform process, discusses what reforms are needed globally for the international investment regime, with a focus on the investor-state dispute resolution mechanism.

A disruptive technology that is influencing not only computing paradigm but every other business is the rise of big data. Internet of Things (IoT) applications are considered to be a major source of big data. Such IoT applications are in general supported through clouds where data is stored and processed by big data processing systems. In order to improve the efficiency of cloud infrastructure so that they can efficiently support IoT big data applications, it is important to understand how these applications and the corresponding big data processing systems will perform in cloud computing environments. However, given the scalability and complex requirements of big data processing systems, an empirical evaluation on actual cloud infrastructure can hinder the development of timely and cost effective IoT solutions. Therefore, a simulator supporting IoT applications in cloud environment is highly demanded, but such work is still in its infancy. To fill this gap, we have designed and implemented IOTSim which supports and enables simulation of IoT big data processing using MapReduce model in cloud computing environment. A real case study validates the efficacy of the simulator.

Many IoT applications and services such as smart parking and smart traffic control contain a network of different analytical components, which are composed in the form of a workflow to make better decisions. These workflows are also known as stream workflows. The focus of existing research works is on the streaming operator graph, which differs from stream workflow application as it involves heterogeneity, multiple data sources and multiple outputs. Considering the complexity and dynamism of stream workflow, meeting real-time data analysis requirements at deployment time is not the whole story as the velocity of data changes over time. This change is the most dynamic form of stream workflow that occurs frequently during the execution of this application. In this article, we propose a new dynamic scheduling technique that manages cloud resources over time to handle data velocity changes in stream workflow while maintaining user-defined real-time data analysis requirements and minimising execution cost. The efficiency of the proposed technique is evaluated, and experimental results showed that this technique outperformed its competitors and is close to the lower bound. Refereed/Peer-reviewed

Stream workflow is a network of big data streaming applications that acts as key enabler for real-time analysis from Internet of Things data. Smart traffic management and smart grid are examples of stream workflow. The focus of existing work is on streaming operator graphs which differs from stream workflow and handling data fluctuations without significant consideration of different dynamic forms that could happen in the structure of data pipelines. This paper investigates the scheduling problem of stream workflow to support runtime alterations of stream workflow deployment, so that scheduling plans will be revised to handle stream workflow applications with continuously changing characteristics. It proposes a pluggable dynamic scheduling technique that accepts user-defined algorithms to handle stream workflow runtime changes. It also presents three different plug-in algorithms and methods to enable auto-scaling of this workflow in a Multicloud environment. The experimental results of the quality of the solution showed that the proposed plug-in optimisation technique is more efficient than baseline and dynamic fair-share techniques to handle runtime changes. Refereed/Peer-reviewed

Software-defined networking (SDN) has evolved as an approach that allows network administrators to program and initialize, control, change and manage networking components (mostly at L2-L3 layers) of the OSI model. SDN is designed to address the programmability shortcomings of traditional networking architectures commonly used in cloud datacenters (CDC). Deployment of SDN solutions have demonstrated significant improvements in areas such as flow optimization and bandwidth allocation in a CDC. However, the benefits are significantly less explored when considering Software-Defined Wide Area Networks (SD-WAN) architectures in the context of delivering solutions by networking multiple CDCs. To support the testing and bench-marking of data-driven applications that rely on data ingestion and processing (e.g., Smart Energy Cloud, Content Delivery Networks) across multiple cloud datacenters, this paper presents the simulator, IoTSim-SDWAN. To the best of our knowledge, IoTSim-SDWAN is the first simulator that facilitates the modeling, simulating, and evaluating of new algorithms, policies, and designs in the context of SD-WAN ecosystems and SDN-enabled multiple cloud datacenters. Finally, IoTSim-SDWAN simulator is evaluated for network performance and energy to illustrate the difference between classical WAN and SD-WAN environments. The obtained results show that SD-WAN surpasses the classical WAN in terms of accelerating traffic flows and reducing power consumption. Refereed/Peer-reviewed

Recent years have witnessed the booming of big data analytical applications (BDAAs). This trend provides unrivaled opportunities to reveal the latent patterns and correlations embedded in the data, and thus productive decisions may be made. This was previously a grand challenge due to the notoriously high dimensionality and scale of big data, whereas the quality of service offered by providers is the first priority. As BDAAs are routinely deployed on Clouds with great complexities and uncertainties, it is a critical task to manage the service level agreements (SLAs) so that a high quality of service can then be guaranteed. This study performs a systematic literature review of the state of the art of SLA-specific management for Cloud-hosted BDAAs. The review surveys the challenges and contemporary approaches along this direction centering on SLA. A research taxonomy is proposed to formulate the results of the systematic literature review. A new conceptual SLA model is defined and a multi-dimensional categorization scheme is proposed on its basis to apply the SLA metrics for an in-depth understanding of managing SLAs and the motivation of trends for future research. Refereed/Peer-reviewed

In the era of big data, the high velocity of data imposes the demand for processing such data in real-time to gain real-time insights. Various real-time big data platforms/services (i.e. Apache Storm, Amazon Kinesis) allow to develop real-time big data applications to process continuous data to get incremental results. Composing those applications to form a workflow that is designed to accomplish certain goal is the becoming more important nowadays. However, given the current need of composing those applications into data pipelines forming stream workflow applications (aka stream graph applications) to support decision making, a simulation toolkit is required to simulate the behaviour of this graph application in Cloud computing environment. Therefore, in this paper, we propose an IoT Simulator for Stream processing on the big data (named IoTSim-Stream) that offers an environment to model complex stream graph applications in Multicloud environment, where the large-scale simulation-based studies can be conducted to evaluate and analyse these applications. The experimental results show that IoTSim-Stream is effective in modelling and simulating different structures of complex stream graph applications with excellent performance and scalability. Refereed/Peer-reviewed

Interest in processing big data has increased rapidly to gain insights that can transform businesses, government policies, and research outcomes. This has led to advancement in communication, programming, and processing technologies, including cloud computing services and technologies such as Hadoop, Spark, and Storm. This trend also affects the needs of analytical applications, which are no longer monolithic but composed of several individual analytical steps running in the form of a workflow. These big data workflows are vastly different in nature from traditional workflows. Researchers are currently facing the challenge of how to orchestrate and manage the execution of such workflows. In this article, we discuss in detail orchestration requirements of these workflows as well as the challenges in achieving these requirements. We also survey current trends and research that supports orchestration of big data workflows and identify open research challenges to guide future developments in this area. Refereed/Peer-reviewed

Ensuring privacy in recommender systems for smart cities remains a research challenge, and in this paper we study collaborative filtering recommender systems for privacy-aware smart cities. Specifically, we use the rating matrix to establish connections between a privacy-aware smart city and k-coRating, a novel privacy-preserving rating data publishing model. First, we model privacy concerns in a smart city as the problem of privacy-preserving collaborative filtering recommendation. Then, we introduce k-coRating to address privacy concerns in published rating matrices, by filling the null ratings with predicted scores. This allows us to mask the original ratings to preserve k-anonymity-like data privacy, and enhance data utility (quantified using prediction accuracy in this paper). We show that the optimal k-coRated mapping is an NP-hard problem and design an efficient greedy algorithm to achieve k-coRating. We then demonstrate the utility of our approach empirically. Refereed/Peer-reviewed