Search

Many robot manipulation skills can be represented with deterministic characteristics and there exist efficient techniques for learning parameterized motor plans for those skills. However, one of the active research challenge still remains to sustain manipulation capabilities in situation of a mechanical failure. Ideally, like biological creatures, a robotic agent should be able to reconfigure its control policy by adapting to dynamic adversaries. In this paper, we propose a method that allows an agent to survive in a situation of mechanical loss, and adaptively learn manipulation with compromised degrees of freedom -- we call our method Survivable Robotic Learning (SRL). Our key idea is to leverage Bayesian policy gradient by encoding knowledge bias in posterior estimation, which in turn alleviates future policy search explorations, in terms of sample efficiency and when compared to random exploration based policy search methods. SRL represents policy priors as Gaussian process, which allows tractable computation of approximate posterior (when true gradient is intractable), by incorporating guided bias as proxy from prior replays. We evaluate our proposed method against off-the-shelf model free learning algorithm (DDPG), testing on a hexapod robot platform which encounters incremental failure emulation, and our experiments show that our method improves largely in terms of sample requirement and quantitative success ratio in all failure modes. A demonstration video of our experiments can be viewed at: https://sites.google.com/view/survivalrl, Comment: 6 pages, published in CASE 2021 as regular paper

In this paper, a reinforcement learning method is presented to generate earthworm-like gaits for a hyperredundant earthworm-like manipulator robot. Partially inspired by human brain’s learning mechanism, the proposed learning framework builds its preliminary belief by first starting with adapting rudimentary gaits governed by a generic kinematic knowledge of undulatory, sidewinding and circular patterns. The preliminary belief is then represented as a prior ensemble to learn new gaits by leveraging apriori knowledge and learning a policy by inferring posterior over prior distribution. While the fundamental idea of incorporating Bayesian learning with reinforcement learning is not new, this paper extends Bayesian actor-critic approach by introducing augmented prior-based directed bias in policy search, aiding in faster parameter learning and reduced sampling requirements. We show results on an in-house built 10-DoF earthworm-like robot that exhibits adaptive development, qualitatively learning different locomotion modes, while given with only rudimentary generic gait behaviors. The results are compared against deterministic policy gradient method (DDPG) for continuous control as the baseline. We show that our proposed method can characterize effective performance over DDPG, and it also achieves faster kinematic indexes in various gaits.

Policy based reinforcement learning methods are widely used for multi-agent systems to learn optimal actions given any state; with partial or even no model representation. However multi-agent systems with complex structures (curse of dimensionality) or with high constraints (like bio-inspired (a) snake or serpentine robots) show limited performance in such environments due to sparse-reward nature of environment and no fully observable model representation. In this paper we present a constructive learning and planning scheme that reduces the complexity of high-diemensional agent model by decomposing it into identical, connected and scaled down multiagent structure and then apply learning framework in layers of local and global ranking. Our layered hierarchy method also decomposes the final goal into multiple sub-tasks and a global task (final goal) that is bias-induced function of local sub-tasks. Local layer deals with learning ‘reusable’ local policy for a local agent to achieve a sub-task optimally; that local policy can also be reused by other identical local agents. Furthermore, global layer learns a policy to apply right combination of local policies that are parameterized over entire connected structure of local agents to achieve the global task by collaborative construction of local agents. After learning local policies and while learning global policy, the framework generates sub-tasks for each local agent, and accepts local agents’ intrinsic rewards as positive bias towards maximum global reward based of optimal sub-tasks assignments. The advantage of proposed approach includes better exploration due to decomposition of dimensions, and reusability of learning paradigm over extended dimension spaces. We apply the constructive policy method to serpentine robot with hyper-redundant degrees of freedom (DOF), for achieving optimal control and we also outline connection to hierarchical apprenticeship learning methods which can be seen as layered learning framework for complex control tasks.

We present reusable policy method for modular reinforcement learning problem in continuous state space. Our method relies on two-layered learning architecture. The first layer partitions the agent’s problem space into n-folds sub-agents that are inter-connected with each other with dexterity identical to original problem. It further learns a local control policy for standalone 1-fold sub-agent. The second layer learns a global policy to reuse ‘already learnt’ standalone local policy over each n sub-agents by sampling local policy with global parameters for each sub-agent—parameterizing local policy independently to approximate non-linear interconnections between sub-agents. We demonstrate our method on simulation example of 12-DOF modular robot that learns maneuver pattern of snake-like gait. We also compare our proposed method against standard single-policy learning methods to benchmark optimality.

The frequent use of transesophageal echocardiogram (TEE) has led to the increased recognition of aortic atheromas. Retrospective and prospective follow-up studies have reported an association between aortic atheromas and stroke in the high-risk patient population, with complex plaques being more likely to embolize than simple plaques. However, TEE-based studies in the low-risk cohorts have failed to show a similar association. There is growing body of evidence suggesting that aortic atheroma is a marker of generalized atherosclerosis. Although magnetic resonance (MR) imaging and computed tomography (CT) scan are emerging as a powerful noninvasive tool for characterization of aortic atheromas, TEE is the imaging modality of choice. Currently, treatment of aortic atheromas is not well defined, and mixed outcomes have been reported for anticoagulation therapy with warfarin. Statins appear promising based on their plaque stabilization properties. However, there are no randomized control trials to establish the role of both anticoagulation and statins in patients with aortic atheromas, and are warranted in the future.

Cell transplantation is an innovative technology that involves the implantation of a variety of myogenic and angiogenic cell types. The transplanted cells proliferate and augment left ventricular performance and therein ameliorate the heart failure symptoms. The concept of cell transplantation has followed the footsteps of angiogenesis starting as bench side research. The latter half of the decade saw the transformation of this potential mechanism to a promising therapy for ischemic heart failure. More than 150 patients have been treated with cellular transplantation worldwide. This novel application has the potential to revolultionalize alternative therapeutic approaches to management of heart failure.

o d in th S g p in h n Adenosine is a naturally occurring nucleoside found roughout the human body. It has both antiarrhythmic nd vasodilatory properties, making it a useful diagnostic nd therapeutic agent in clinical cardiology. Adenosine as been used in the United States since 1989 for the anagement of reentrant supraventricular tachycardia VT) and is considered to be the initial drug of choice r the acute treatment of this frequently encountered rrhythmia. It is also commonly used off label to ifferentiate wide-complex tachycardias and is widely sed as a pharmacologic stress agent in conjunction with yocardial perfusion imaging (MPI). Adenosine has been reported to have an excellent fety profile, and serious adverse effects have been frequently reported. The most commonly encountered dverse effects are flushing, chest pain, and shortness of reath. Far less common, more serious adverse effects clude atrioventricular block, bronchospasm, bradycaria, and hemodynamically significant hypotension. ecause of adenosine’s short half-life, these effects are sually rapidly reversed with the termination of its dministration. Though not generally recognized, atrial fibrillation a potentially serious adverse effect of adenosine that as been reported, particularly when used to treat VT. Silverman et al reported reproducible adenosineduced atrial fibrillation in 5 patients undergoing elecophysiology testing for SVT. Garratt et al reported a ngle case of nonsustained atrial fibrillation in a patient sinus rhythm who received adenosine. Cases of susined atrial fibrillation after adenosine administration ave also been reported by McIntosh-Yellin et al and riffith et al. Atrial fibrillation occurring during adenosine pharacologic stress testing has been far less frequently ported. In fact, atrial fibrillation is not listed as a side

Cardiovascular disease (CVD) in the developed countries continues to grow at an epidemic proportion. There are a significant number of young adults with no clinical evidence of CVD, but who have two or more risk factors that predispose them to CV events and death. Many of these risk factors are modifiable, and by controlling these factors, the CVD burden can be decreased significantly. Recent statistics have shown that, if all major forms of CVD were eliminated, the life expectancy would rise by almost 7 years. Hence it is imperative that primary prevention efforts should be initiated at a young age to avert decades of unattended risk factors. Hyperlipidemia has been linked to CVD almost a century ago. Since then various clinical trials have not only supported this link, but have also shown the CV benefits in aggressively treating patients with hyperlipidemia. In this generation, we have various therapeutic agents that are capable of reducing the elevated lipid levels. With drugs like statins, we are able to reduce the risk of CVD by about 30% and avoid major adverse events. Newer drugs are being researched and introduced in the treatment of hyperlipidemia in humans. These can be used in combination therapy resulting in optimal levels of lipids. The new National Cholesterol Education Program (NCEP)/Adult Treatment Panel III (ATP III) guidelines have come as a wake-up call to clinicians about primary prevention of CVD through strict lipid management and multifaceted risk management approach in the prevention of CVD.

The present study prospectively evaluated adjunctive antiplatelet therapy in patients without insulin-requiring diabetes during elective coronary stenting. Three hundred patients were randomized to one of three treatment groups: clopidogrel pretreatment, adjunctive abciximab, or control. Stenting was successful in 98% and no deaths occurred. Thirty-day and 1-year major adverse coronary events (MACEs) was similar in all groups. A subgroup of 109 patients undergoing single-vessel stenting of type A/B1 lesions with short guidewire times had no postprocedure myocardial infarction or 30-day MACE. We conclude that patients with these characteristics may safely undergo elective coronary stenting without adjunctive antiplatelet therapy.

Cardiovascular disease is the leading cause of death in patients aged 65 and above. Although elderly persons represent only 12.4% of the US population, they account for about a third of drug expenditures. However the appropriate use of cardiovascular medications in these patients has been shown to reduce the rate of cardiovascular morbidity and mortality. The normal aging and the disease process in the elderly result in significant changes at the structural and molecular level in the elderly. The changes that take place in the autonomic nervous system, the kidneys, and the liver in the elderly modify the metabolism and clinical effects of most medications. Elderly patients are also susceptible to side effects and adverse drug reactions. Physicians should have a clear understanding of the normal aging processes, the abnormal changes due to disease process and the changes in the pharmacology of drugs in the elderly to deliver proper care to the elderly patient.

Acute left main coronary artery occlusion is a catastrophic and mostly fatal event. Patients may present with sudden death or cardiogenic shock. Intra-aortic balloon pump support and emergency revascularization is indicated to preserve the left ventricular function. We describe a case of left main thrombus in a health 24-year-old young male with no risk factors for coronary atherosclerosis.

Case history: a 76-year-old African-American female presented to a community hospital with complaints of chest pain. Her medical history was significant for hypertension, gastroesophageal reflux disease (GERD), hyperlipidemia, and bipolar disorder. On arrival at the emergency department, electrocardiogram showed acute inferior wall myocardial infarction (MI). The patient received thrombolytic therapy with tenecteplase (TNKase™) with successful reperfusion. On arrival at our hospital, the patient denied any chest pain and her electrocardiogram showed sinus bradycardia with small Q-waves in the inferior leads II, III, and AVF (see Figure 1). She was treated with aspirin, heparin, lisinopril, simvastatin, and clopidogrel. Beta-blocker was not started as the patient was having bradycardia. The patient was scheduled to undergo myocardial perfusion scan 48 hours following her uncomplicated inferior wall MI. Figure 1 Initial Electrocardiogram Showing Q-waves in the Inferior Leads At rest, the patient received 12.0mCi of technetium 99msestamibi. Seventy-five minutes later, rest myocardial perfusion images were obtained by non-gated single photon emission computed tomography (SPECT). Then she received adenosine infusion at 140µg/kg/minute for six minutes. Blood pressure and 12-lead electrocardiogram were obtained every minute and the patient’s symptoms were recorded. Resting heart rate was 67. Heart rate three minutes into the adenosine infusion was 73, and heart rate six minutes into the adenosine infusion was 85. Resting blood pressure was 152/69. Blood pressure three minutes into the adenosine infusion was 128/54, and blood pressure six minutes into the adenosine infusion was 70/47. Resting electrocardiogram showed normal sinus rhythm, small inferior Q-waves, and ST–T changes. There was an additional 1–2mm ST segment depression in the electrocardiographic leads I, AVL, and V2, plus 1–3mm additional ST segment elevation was noted in II, III, and AVF during and up to three minutes-post adenosine infusion (see Figure 2). There were occasional premature ventricular complexes and premature atrial complexes before, during, and post-adenosine infusion. The patient did not report chest pain during or after adenosine infusion. Figure 2 Electrocardiogram Obtained During Adenosine Infusion Showing ST Segment Elevation in the Inferior Leads Three minutes into the adenosine infusion, 44mCi of technetium 99m sestamibi was injected intravenously. One hour and 50 minutes later, the stress (adenosine) myocardial perfusion images were obtained by gated SPECT. SPECT images were acquired with a dual-head gamma camera using the step-and-shoot detector rotation, obtaining 32 projections over 180° arcs (45° right anterior oblique to 45° left posterior oblique). The camera was equipped with a low-energy, high-resolution collimator. Rest and stress images were acquired with a 20% window centered over the 140KeV photo peak. Gated acquisitions were obtained using eight frames per cardiac cycle with a 40% acceptance window. Acquisition times were 40 seconds per projection. The summed projection data sets were filtered with a butterworth filter (order 5 cut-off 0.33). Short-, vertical-, long-, and horizontal-axis images were evaluated for transient, partially reversible, and fixed perfusion defects. The rest and stress (adenosine) myocardial perfusion images revealed normal-sized left ventricle and mild to moderate transient myocardial perfusion defect (mild to moderate ischemia) superimposed on mild fixed myocardial perfusion defect (superimposed on mild scar) involving the inferior, inferoapex, and inferolateral wall of the left ventricle (see Figure 3). There were hypokinetic inferior, inferoapex, and inferolateral walls of the left ventricle, with resting global left ventricular ejection fraction (LVEF) of 40%. The right ventricle was normal in size and function. Figure 3 Stress Myocardial Perfusion Polar Map Depicting Flow Defect in the Region of the Right Coronary Artery Following the adenosine myocardial perfusion scanning, the patient underwent cardiac catheterization. Cardiac catheterization showed normal left main, left anterior descending, and circumflex arteries. The right coronary artery showed a 75% stenosis in the mid segment (see Figure 4). The patient had a 3.5×16mm stent deployed without any complications. Her post-procedure course in the hospital was uneventful and she was discharged the following day in a stable condition. Figure 4 Left Anterior Oblique View of the Right Coronary Artery

In 1953, Dr. Helmut Hertz of Sweden together with Dr. Inge Edler began to use a commercial ultrasonoscope to examine the heart, thus starting the era of clinical echocardiography [1].

Clinical cardiac electrophysiology is a subspecialty of cardiology dealing with the evaluation and management of patients with complex rhythm or conduction abnormalities. In the last four decades, clinical cardiac electrophysiology has evolved into an established discipline credited with improving and saving hundreds of thousands of lives. We briefly review the basic electrophysiologic principles, anatomy of the electric system of the heart, mechanism of arrhythmias, genetical predisposition to arrhythmias, types of arrhythmias, diagnostic electrophysiologic studies, and pharmacologic and device therapies available for the treatment of different types of arrhythmias.

Myocardial perfusion imaging (MPI) is widely utilized in the non-invasive diagnosis and management of coronary artery disease (CAD). Radionuclide MPI has been well studied in the evaluation, risk stratification and identification of patients with CAD who will benefit from revascularization. Myocardial revascularization using percutaneous coronary interventions (PCI) and coronary artery bypass graft (CABG) has become a mainstay in the treatment of patients with CAD. The idea of performing a functional study after revascularization is to identify the areas that are not revascularized or to look for restenosis and progression of native vessel disease or disease in the grafts.

Cardiovascular diseases (CVD) have reached epidemic proportions in developed countries, and are the leading cause of morbidity and mortality. An estimated 16.7 million – or 29.2% of total global deaths – result from the various forms of CVD, many of which are preventable by healthy diet, regular physical activity, and smoking cessation [1]. CVD is no longer the disease of the developed world: some 80% of all CVD deaths worldwide took place in developing, low and middle-income countries and these countries also account for 86% of the global CVD disease burden. By 2010, CVD will be the leading cause of death in developing countries. The rise in CVD reflects a significant change in diet habits, physical activity levels, and tobacco consumption worldwide as a result of industrialization, urbanization, economic development and food market globalization. To deal with this changing situation, an extraordinary array of new cardiovascular therapies continues to become available. These drugs and therapies are efficacious and are better tolerated than their predecessors, not only in the management but also in prevention of CVD. This results in ever more difficulty for the practitioners of medicine in understanding the mechanism of action of these medications and also in deciding the proper medication for their patients. This chapter is a concise presentation of the pharmacology of cardiovascular medications.

The incidence of diabetes has reached epidemic proportions across the world. In patients with diabetes, there is a two to four times increased risk of developing coronary artery disease (CAD). Diabetes seems to eliminate the protective benefits of hormones in women against CAD. Patients with type II diabetes also have hypertension, dyslipidemia, obesity, endothelial dysfunction and prothrombotic factors, called 'the metabolic syndrome'. Not only the incidence of CAD is higher in diabetes, the mortality of the diabetic patients after a cardiac event is significantly increased as compared to non-diabetics, including sudden death. Although in the past 35 years there has been a decline in the rate of death due to CAD in the general population, this has not been seen among patients with diabetes. Primary prevention can play an important role in decreasing the incidence of CAD in diabetic patients. Aggressive treatment of hyperlipidemia and hypertension is essential. Recent knowledge about the protective effects of aspirin, statins, angiotension converting enzyme inhibitors, and glitazones in the diabetic patients, if used appropriately will go a long way in primary and secondary prevention of CAD in patients with diabetes.

The prevalence of ischemic heart disease (IHD) has been increasing among the women in developed countries. The well recognized IHD excess in men has often obscured the fact that IHD is the leading cause of death in women. Women have atypical symptoms of IHD that lead to a delay in the diagnosis and an overall poor prognosis. Women have a delay in the onset of IHD due to the beneficial effects of their sex hormones. Postmenopausal women lose this beneficial effect of estrogen and undergo significant changes in their lipid profile, arterial pressure, glucose tolerance, and vascular reactivity that increase their risk for development of IHD. Recently there has been considerable interest in the sex hormones and their role in IHD in women. The general belief that hormone replacement therapy (HRT) has an overall beneficial effect on cardiovascular disease (CVD) in women and hence decreases CVD mortality and morbidity has not been shown in the recent multicenter prospective studies. With the availability of various types of estrogen and progestins, physicians prescribing these agents should take into consideration their varying effects on the cardiovascular system. Risk factor modifications should include diet, weight loss, regular exercise, smoking cessation and adequate control of hypertension (HTN), diabetes (DM) and hyperlipidemia. In the appropriate setting, treatment with proven beneficial agents like aspirin, beta-blockers, angiotensin converting enzyme (ACE) inhibitors and statins will help decrease the burden of IHD in women.

Ischemic heart disease is the leading cause of death in the developed countries for those older than 65 years of age. In patients suspected to have coronary artery disease a stress test should be performed to identify the vulnerability of the myocardium to ischemia. As a rule of thumb, the evaluation of coronary artery disease is best done by exercise stress test. In patients who are not able to exercise adequately, pharmacological stress agents are used. The commonly used agents are the coronary vasodilators, adenosine and dipyridamole and the catecholamines, dobutamine and arbutamine. These agents are combined with imaging techniques to increase the sensitivity and specificity of the test. These agents have been widely used and have an excellent safety profile. Another advantage in using pharmacological stress agents is that they do not affect the image quality, especially with echocardiography and magnetic resonance imaging. Ongoing developments hold promise for safer and more reliable pharmacological stress agents in the future.