Search

For many data-intensive real-world applications, such as recognizing objects from images, detecting spam emails, and recommending items on retail websites, the most successful current approaches involve learning rich prediction rules from large datasets. There are many challenges in these machine learning tasks. For example, as the size of the datasets and the complexity of these prediction rules increase, there is a significant challenge in designing scalable methods that can effectively exploit the availability of distributed computing units. As another example, in many machine learning applications, there can be data corruptions, communication errors, and even adversarial attacks during training and test. Therefore, to build reliable machine learning models, we also have to tackle the challenge of robustness in machine learning.In this dissertation, we study several topics on the scalability and robustness in large-scale learning, with a focus of establishing solid theoretical foundations for these problems, and demonstrate recent progress towards the ambitious goal of building more scalable and robust machine learning models. We start with the speedup saturation problem in distributed stochastic gradient descent (SGD) algorithms with large mini-batches. We introduce the notion of gradient diversity, a metric of the dissimilarity between concurrent gradient updates, and show its key role in the convergence and generalization performance of mini-batch SGD. We then move forward to Byzantine distributed learning, a topic that involves both scalability and robustness in distributed learning. In the Byzantine setting that we consider, a fraction of distributed worker machines can have arbitrary or even adversarial behavior. We design statistically and computationally efficient algorithms to defend against Byzantine failures in distributed optimization with convex and non-convex objectives. Lastly, we discuss the adversarial example phenomenon. We provide theoretical a

For many data-intensive real-world applications, such as recognizing objects from images, detecting spam emails, and recommending items on retail websites, the most successful current approaches involve learning rich prediction rules from large datasets. There are many challenges in these machine learning tasks. For example, as the size of the datasets and the complexity of these prediction rules increase, there is a significant challenge in designing scalable methods that can effectively exploit the availability of distributed computing units. As another example, in many machine learning applications, there can be data corruptions, communication errors, and even adversarial attacks during training and test. Therefore, to build reliable machine learning models, we also have to tackle the challenge of robustness in machine learning.In this dissertation, we study several topics on the scalability and robustness in large-scale learning, with a focus of establishing solid theoretical foundations for these problems, and demonstrate recent progress towards the ambitious goal of building more scalable and robust machine learning models. We start with the speedup saturation problem in distributed stochastic gradient descent (SGD) algorithms with large mini-batches. We introduce the notion of gradient diversity, a metric of the dissimilarity between concurrent gradient updates, and show its key role in the convergence and generalization performance of mini-batch SGD. We then move forward to Byzantine distributed learning, a topic that involves both scalability and robustness in distributed learning. In the Byzantine setting that we consider, a fraction of distributed worker machines can have arbitrary or even adversarial behavior. We design statistically and computationally efficient algorithms to defend against Byzantine failures in distributed optimization with convex and non-convex objectives. Lastly, we discuss the adversarial example phenomenon. We provide theoretical a

Nanosecond pulsed electric fields (nsPEF) induce apoptotic pathways in human cancer cells. The potential therapeutic effective of nsPEF has been reported in cell lines and in xenograft animal tumor model. The present study investigated the ability of nsPEF to cause cancer cell death in vivo using carcinogen-induced animal tumor model, and the pulse duration of nsPEF was only 7 and 14 nano second (ns). An nsPEF generator as a prototype medical device was used in our studies, which is capable of delivering 7-30 nanosecond pulses at various programmable amplitudes and frequencies. Seven cutaneous squamous cell carcinoma cell lines and five other types of cancer cell lines were used to detect the effect of nsPEF in vitro. Rate of cell death in these 12 different cancer cell lines was dependent on nsPEF voltage and pulse number. To examine the effect of nsPEF in vivo, carcinogen-induced cutaneous papillomas and squamous cell carcinomas in mice were exposed to nsPEF with three pulse numbers (50, 200, and 400 pulses), two nominal electric fields (40 KV/cm and 31 KV/cm), and two pulse durations (7 ns and 14 ns). Carcinogen-induced cutaneous papillomas and squamous carcinomas were eliminated efficiently using one treatment of nsPEF with 14 ns duration pulses (33/39 = 85%), and all remaining lesions were eliminated after a 2nd treatment (6/39 = 15%). 13.5% of carcinogen-induced tumors (5 of 37) were eliminated using 7 ns duration pulses after one treatment of nsPEF. Associated with tumor lysis, expression of the anti-apoptotic proteins Bcl-xl and Bcl-2 were markedly reduced and apoptosis increased (TUNEL assay) after nsPEF treatment. nsPEF efficiently causes cell death in vitro and removes papillomas and squamous cell carcinoma in vivo from skin of mice. nsPEF has the therapeutic potential to remove human squamous carcinoma.

Nanosecond pulsed electric fields (nsPEF) induce apoptotic pathways in human cancer cells. The potential therapeutic effective of nsPEF has been reported in cell lines and in xenograft animal tumor model. The present study investigated the ability of nsPEF to cause cancer cell death in vivo using carcinogen-induced animal tumor model, and the pulse duration of nsPEF was only 7 and 14 nano second (ns). An nsPEF generator as a prototype medical device was used in our studies, which is capable of delivering 7-30 nanosecond pulses at various programmable amplitudes and frequencies. Seven cutaneous squamous cell carcinoma cell lines and five other types of cancer cell lines were used to detect the effect of nsPEF in vitro. Rate of cell death in these 12 different cancer cell lines was dependent on nsPEF voltage and pulse number. To examine the effect of nsPEF in vivo, carcinogen-induced cutaneous papillomas and squamous cell carcinomas in mice were exposed to nsPEF with three pulse numbers (50, 200, and 400 pulses), two nominal electric fields (40 KV/cm and 31 KV/cm), and two pulse durations (7 ns and 14 ns). Carcinogen-induced cutaneous papillomas and squamous carcinomas were eliminated efficiently using one treatment of nsPEF with 14 ns duration pulses (33/39 = 85%), and all remaining lesions were eliminated after a 2nd treatment (6/39 = 15%). 13.5% of carcinogen-induced tumors (5 of 37) were eliminated using 7 ns duration pulses after one treatment of nsPEF. Associated with tumor lysis, expression of the anti-apoptotic proteins Bcl-xl and Bcl-2 were markedly reduced and apoptosis increased (TUNEL assay) after nsPEF treatment. nsPEF efficiently causes cell death in vitro and removes papillomas and squamous cell carcinoma in vivo from skin of mice. nsPEF has the therapeutic potential to remove human squamous carcinoma.