Search

The authors argue that where appropriate, research papers in computer programming should be accompanied by software artifacts that let others test the repeatability of researchers' claims. The authors summarize results of an artifact evaluation process begun on an experimental basis at conferences of the Association of Computing Machinery, and provide commentary the mechanics of evaluating artifacts, who the evaluators should be, and the benefits of the evaluation process. Consideration is also given to whether artifacts should be published.

A call for papers on computer programming and information science is presented.

The article focuses on the research behind hiding secrets within computer programs, also known as program obfuscation. The author discusses how program obfuscation could revolutionize cryptography, examines how it could impact reverse engineering with a proposed virtual black box (VBB), and explores the alternate security of so-called indistinguishability obfuscation (iO)

Presents comments and suggestions for the publication the journal "Communications of the ACM." Proposition for the publication of papers; Criticisms on the article of Peter Naur in the December 1975 issue entitled "Programming Languages, Natural Languages, and Mathematics"; Comments on the contribution of Wallace E. Lin in the January 1976 issue.

This article focuses on conferences related to computer programs. Architecture. The program for the Fourth Annual Symposium on Computer Architecture, sponsored by ACM SIGARCH and IEEEes and scheduled for March 23-25 at the Sheraton Silver Spring Motor Inn, Silver Spring, Maryland, will include two special tutorials: Microprocessor Architecture at the Chip Level and Microprocessor System Architecture: Application to Industrial Control; The 1977 IEEE Workshop on Picture Data Description and Management will be held April 21-22,The 1977 IEEE International Symposium on Information Theory will be held on the campus of Cornell University, Ithaca, New York, on October 10-14, 1977. 1977 at the Sheraton-Chicago Hotel. Its program, consisting of invited and contributed papers and panel discussions, will include sessions on: vector graphics; picture description and parsing techniques; pictorial databases; picture machines and image processors.

The article presents a list of Association for Computing Machinery proceedings and special publications. Conference Record of the 1982 ACM Symposium on Lisp and Functional Programming, which was held August 15-18 in Pittsburgh, Pennsylvania. The Proceedings of the Eight Symposium on Operating Systems Principles, which was held December 14-16, 1981 in California. The "1982/83 ACM Graduate Assistantship Directory in the Computer Sciences," 15th ed, available for free to ACM student members upon request. The Proceedings of the Seventh Data Communications Symposium, which was held in Mexico City, Mexico, during October 27-29, 1981.

Problems involved in the production of reliable software are now attracting substantial attention. In many applications, reliability is at least as important as more traditional goals such as efficiency and portability. The programming community has started to examine the effects that its tools and techniques have on reliability. Reliability has not been an explicit goal in the design of most programming languages, and it is not surprising that each of today's dominant programming languages contains major sources of unreliability. Data about the effects of languages on the reliability of programs is difficult to obtain and easy to challenge. A new crop of languages promising improvements has appeared, but many issues are still vigorously debated by their designers. Users need guidelines for selecting "safe" subsets that avoid the worst problems of existing languages, for establishing suitable programming conventions, and for choosing between languages where that is an option.

The article discusses the forthcoming events related to computer science. Focusing on "Association for Computing Machinery (ACM) Computer Science Conference," it states that software engineering, fifth generation computers, and artificial intelligence are among the topics to be covered at the 1986 ACM Computer Science Conference. The conference is to be held during February 4-6, 1986, in Cincinnati, Ohio. It is further added that the "Second Annual ACM Northeast Regional Conference" is planned for October 28-30, 1985, in Framingham, Massachusetts. The conference theme is "Integrating the Information Workplace: The Key to Productivity." Focusing on yet another conference the article states that "Eurit 86; European Conference on Information Technology in Education," is to be held during May 20-23, 1986, in Enschede, Netherlands is to deal with the development of educational software and courseware.

Investigates the evolution of computer program for predictive writing. Usability of the program for infinite sequencing of symbols; Advantage of computer usage in written works; Use of a system array of processing elements in the machine.

This paper studies undefined behavior arising in systems programming languages such as C/C++. Undefined behavior bugs lead to unpredictable and subtle systems behavior, and their effects can be further amplified by compiler optimizations. Undefined behavior bugs are present in many systems, including the Linux kernel and the Postgres database. The consequences range from incorrect functionality to missing security checks. This paper proposes a formal and practical approach, which finds undefined behavior bugs by finding “unstable code” in terms of optimizations that leverage undefined behavior. Using this approach, we introduce a new static checker called Stack that precisely identifies undefined behavior bugs. Applying Stack to widely used systems has uncovered 161 new bugs that have been confirmed and fixed by developers. [ABSTRACT FROM AUTHOR]

The article looks at issues in the history of computing and computer science. The author assesses computer scientist Donald Knuth's criticisms of historiographic trends in the history of computer science, particularly as described by computer scientist Martin Campbell-Kelly in a 2007 paper entitled "The History of the History of Software." Topics include computer scientists versus historians as researchers on the history of computing.

This article focuses on conferences related to computer systems. The 1981 Winter Simulation Conference to be held on December 9-11 in Atlanta, Georgia, is to feature papers, panel discussions, and tutorials on discrete and combined (Discrete/Continuous) simulation modeling; The 15th Annual Simulation Symposium is planned for March 17-19, 1982 in Tampa, Florida; The Fifteenth Hawaii International Conference on System Sciences, Honolulu, January 6-8. 1982, is to cover developments in theory or practice in the areas of computer software and hardware and advanced systems applications; The National Educational Computing Conference 81 is set for June 17-19 at North Texas State University in Denton, Texas; An International Conference on Systems Methodology, organized by the Society for General Systems Research and sponsored by the International Federation for Systems Research, is planned for January 5-9, 1982, in Washington D.C.

This article presents news and events related to the Association for Computing Machinery (ACM). The advance program for the International Conference on Software Engineering, September 13-16, 1982, in Tokyo, includes sessions on language processing; software configuration management; quantitative aspects of software; requirement techniques; programming environments; maintenance evaluation and tools; specification techniques; testing and tools; software notations; and program analysis. Up to two $1000 prizes for best submitted papers are to be awarded by the Technical Committee on Software Engineering. The Symposium on Programming Language Issues in Software Systems sponsored by ACM SIGPLAN, to be held during June 27-29, 1983 in San Francisco, California, is to focus on innovation in two major areas: new programming language constructs and abstraction mechanisms; and the application of programming language principles to the design of software systems. Relevant topics in the first area encompass languages or language constructs; abstraction techniques; specification methods; and techniques for supporting concurrency or exception handling. In the second area topics include the application of programming language principles and techniques to the design and organization of software systems.

This article focuses on a list of conferences related to computer systems. The ACM SIGACT-SIGMOD Conference on Principles of Database Systems is planned for March 29-31, 1982 in Los Angeles, California; The Symposium on Reliability in Distributed Software and Database Systems, to be held July 21-22, 1981, in Pittsburgh, Pennsylvania, will have sessions on robust concurrency control in distributed database management systems; language issues for reliability in distributed systems; The Annual Conference of the Association for the Development of Computer- Based Instructional Systems is designed to provide a setting for both formal and informal sharing of CBI experience, research findings. This meeting is to be held May 10-14, 1982, in Vancouver, British Columbia. An international conference. The Impact of Computerization on Social Science Research, is scheduled for September 14-18. 1981, in Grenoble, France.

On January 1, 1992 an era in the computer field ended with the death of admiral/Dr. Grace Murray Hopper. She saw the beginning of the computer field with her work on MARK I, a physically large electro-mechanical machine, and lived to see and influence people who could carry around small electronic computers with a capability many orders of magnitude greater. Grace Hopper's lifework epitomized the development of the software field from primitive programs to sophisticated compilers and their verification. She was the greatest woman in the computer field, and one of the dozen top people in the field to date. Hopper's first real computing publication was based on her initial work on the Harvard MARK I and was "A Manual of Operation for the Automatic Sequence Controlled Calculator," Annals of the Harvard Computation Laboratory. Her first significant paper dealing with software "The Education of a Computer" was presented in 1952 at one of the very early ACM conferences. It was reprinted in the Annals of the History of Computing.

This paper discusses a model for finding an efficient implementation of a business application system whose logical specifications have been determined in advance. The model views file and program design as a problem of systematically coordinating the configurations of datasets and computations. It uses a straightforward search technique to determine aggregations of computations, aggregations of datasets, device, organization, and key order for each dataset, key order for each computation, and access method for each dataset-computation pair. Although computational results are presented for a sample problem involving 54 computations and 49 datasets, the main point of the paper is that the underlying model works computationally and is simple enough to be adapted to many file design situations. [ABSTRACT FROM AUTHOR]

Proofs of program correctness tend to be long and tedious, whereas testing, though useful in detecting errors, usually does not guarantee correctness. This paper introduces a technique whereby test data can be used in proving program correctness. In addition to simplifying the process of proving correctness, this method simplifies the process of providing accurate specification for a program. The applicability of this technique to procedures and recursive programs is demonstrated. [ABSTRACT FROM AUTHOR]

This paper describes a recovery scheme for syntax errors which provides automatically-generated high quality recovery with good diagnostic information at relatively low cost. Previous recovery techniques are summarized and empirical comparisons are made. Suggestions for further research on this topic conclude the paper. [ABSTRACT FROM AUTHOR]

The article lists several publications from the Association for Computing Machinery (ACM). Some of them are Proceedings of the Symposium on Architectural Support for Programming Languages and Operating Systems; Abstracts of the 1982 ACM Computer Science Conference; The Papers of the Thirteenth SIGCSE Technical Symposium on Computer Science Education; 1981 Winter Simulation Conference Proceedings; ACM 81 Conference Proceedings; Proceedings of the 1981 Conference on Functional Programming Languages and Computer Architecture.

This article details Charles Babbage’s theoretical computer program, the first attempt to specific how to mechanize complex algorithms with a computer, written in 1837. The article also discusses his theoretical computer, the Analytical Engine, that his program would have run on. Topics include Babbage’s first and second code tables and the design of the Analytical Engine.

This paper reports on the results of a multidisciplinary project involving lawyers and computer scientists with the aim to put forward a set of methods and tools to (1) define software liability in a precise and unambiguous way and (2) establish such liability in case of incident. The overall approach taken in the project is presented through an electronic signature case study. The case study illustrates a situation where, in order to reduce legal uncertainties, the parties wish to include in the contract specific clauses to define as precisely as possible the share of liabilities between them for the main types of failures of the system. [ABSTRACT FROM AUTHOR]

The author discusses the development of computer hardware and its isolation from computer software and algorithms. She mentions the early history of computers when computers were single use, how the time and cost of hardware development present barriers to software development, and the need for collaborative research.

The article presents abstracts from other publications of the Association fo Computing Machinery (ACM), including an article about the problem of providing security to statistical databases against disclosure of confidential information, a paper presenting an overview of deterministic functional RAM chip testing, and an article on software design for one-dimensional partial differential equations.

Presents the authors' views on a paper entitled "Program Verification: The Very Idea," by James Fetzer which was published in the September 1988 issue of the journal "Communications of the ACM." Issue of accurate reporting of science and accurate representation of research; Rage caused by the paper in the program verification community; Failure of Fetzer to give a clear picture of program verification.

Different battery chemistries perform better on different axes, such as energy density, cost, peak power, recharge time, longevity, and efficiency. Mobile system designers are constrained by existing technology, and are forced to select a single chemistry that best meets their diverse needs, thereby compromising other desirable features. In this paper, we present a new hardware-software system, called Software Defined Battery (SDB), which allows system designers to integrate batteries of different chemistries. SDB exposes application programming interfaces (APIs) to the operating system, which controls the amount of charge flowing in and out of each battery, enabling it to dynamically trade one battery property for another depending on application and/or user needs. Using microbenchmarks from our prototype SDB implementation, and through detailed simulations, we demonstrate that it is possible to combine batteries which individually excel along different axes to deliver an enhanced collective performance when compared to traditional battery packs. [ABSTRACT FROM AUTHOR]

The article highlights the "International Research Conference on History of Computing," held from June 10 to 15, 1976 in Los Alamos, New Mexico. The conference was organized by Nick Metropolis and Jack Worlton of the Los Alamos Scientific Laboratory. Lecturer Donald Knuth presented his paper on the history of programming languages. Other papers presented at the conference focused on historic names, historic machines, pre-history and software.

The article focuses on new design possibilities that visual and task focused systems offer. In the 1970s when the infrastructure, methodologies, specifications, architecture and goals for large scale corporate systems were set, the focus of applications software was data processing. The hardware was, of course primitive by today's standards. But hardware aside, the first 10 to 15 years of application development were focused on structuring environments to capture, transform, and report on data-largely transaction data. The first systems author remember were insurance, banking people used to keypunch from when batch systems were the rage. Of course, since the paper forms couldn't be represented well and since display screen real estate was such a priority, these "data entry"screens contained huge numbers of fields with minimalist cryptic labels. Since people weren't doing anything but entering data that existed on source paper forms, the focus was on rapid data entry. And since data transmission was costly, the goal was to have as much on a display as possible before pressing transmit or enter.

Presents several letters concerning on the use of computer softwares in the United States. Discussion on the reduction of the magnitude of complexity of a song; Evaluation on computer software systems safety; Examination on the status of prepublication review of papers in cryptology by the National Security Agency.

Time and state descriptions form the core of a simulation model representation. The historical influence of initial application areas and the exigencies of language implementations have created a muddled view of the time and state relationships. As a consequence, users of simulation programming languages work in relative isolation; model development, simulation application, model portability, and the communication of results are inhibited and simulation practice fails to contribute to the recognition of an underlying foundation or integrating structure. A model representation structure has been forged from a small set of basic definitions which carefully distinguish the state and time relationships. This paper focuses on the coordination of the time and state concepts using "object" as the link. In addition to clarifying the relationships, the structure relates the concept of ""state-sequenced simulation" to the variations in time flow mechanisms. In conclusion, some speculations are offered regarding alternative algorithms for time flow mechanisms. [ABSTRACT FROM AUTHOR]

Several principles are identified from work on the verification of parallel programs. Concrete examples of the ways these principles can be applied, even when formal verification is not the goal, are then described. The purpose of this paper is to demonstrate ways in which the concepts of program verification yield insight into the programming process, programming languages, and program properties. [ABSTRACT FROM AUTHOR]

Consistency control has to be enforced in database management systems (DBMS) where several transactions may concurrently access the database. This control is usually achieved by dividing the database into locking units or granules, and by specifying a locking policy which ensures integrity of the information. However, a drawback of integrity enforcement through locking policies is the degradation of the global system performance. This is mainly due to the restriction imposed by the locking policies to the access of transactions to the database, and to the overheads involved with the management of locks. A framework for the quantitative analysis of the impact of these factors on the performance of DBMS is presented in this paper. In a first step, the main factors which determine the behavior of these systems are pointed out and analyzed independently. The results hereby obtained are aggregated in a second step to yield a global performance evaluation. Throughout this hierarchical modeling approach various analytical techniques are used and the results are illustrated by numerical examples. The paper concludes by pointing out the final results' sensitivity to some basic assumptions concerning transaction behavior and the need for more experimental studies in this area. [ABSTRACT FROM AUTHOR]

As computer technology matures, our growing ability to create large systems is leading to basic changes in the nature of programming. Current programming language concepts will not be adequate for building and maintaining systems of the complexity called for by the tasks we attempt. Just as high level languages enabled the programmer to escape from the intricacies of a machine's order code, higher level programming systems can provide the means to understand and manipulate complex systems and components. In order to develop such systems, we need to shift our attention away from the detailed specification of algorithms, towards the description of the properties of the packages and objects with which we build. This paper analyzes some of the shortcomings of programming languages as they now exist, and lays out some possible directions for future research. [ABSTRACT FROM AUTHOR]

Optimal decision table conversion has been tackled in the literature using two approaches, dynamic programming and branch-and-bound. The former technique is quite effective, but its time and space requirements are independent of how "easy" the given table is. Furthermore, it cannot be used to produce good, quasioptimal solutions. The branch-and-bound technique uses a good heuristic to direct the search, but is cluttered up by an enormous search space, since the number of solutions increases with the number of test variables according to a double exponential. In this paper we suggest a heuristically guided top-down search algorithm which, like dynamic programming, recognizes identical subproblems but which can be used to find both optimal and quasioptimal solutions. The heuristic search method introduced in this paper combines the positive aspects of the above two techniques. Compressed tables with a large number of variables can be handled without deriving expanded tables first. [ABSTRACT FROM AUTHOR]

n this paper an algorithm to detect logical errors in a limited-entry decision table and in loop-free programs with embedded decision tables is developed. All the conditions in the decision tables are assumed to be inequalities or equalities relating linear expressions. It is also assumed that actions in a decision table are linear in variables which occur in the condition stub of the decision table (or tables) to which control is transferred from the table. The algorithm is based on determining whether a set of linear inequalities has or does not have a solution. The algorithm described in the paper is implemented in Fortran IV. [ABSTRACT FROM AUTHOR]

Recent developments in optimizing techniques have allowed a new design for compilers to emerge. Such a compiler translates the parsed source code into lower level code by a sequence of steps. Each step expands higher level statements into blocks of lower level code and then performs optimizations on the result. Each statement has only one possible expansion - the task of tailoring this code to take advantage of any special cases is done by the optimizations. This paper provides evidence that this strategy can indeed result in good object code. The traditionally difficult PL/I concatenate statement was investigated as a detailed example. A set of fairly simple optimizations was identified which allow the compiler to produce good code. More elaborate optimizations can further improve the object code. For most contexts of the concatenate statement, the code produced by a compiler using the expansion-optimization strategy described above compares favorably with the code produced by a conventional PL/I optimizing compiler. [ABSTRACT FROM AUTHOR]

A sequential programming language forces the programmer to prescribe explicitly the order in which the operations in his program have to be executed, even if the order is not relevant to the solution of his problem. The requirement to Indicate Irrelevant sequencing can be removed if the language provides facilities for specifying a task in a nonprocedural manner. In general, a program specified in this way will allow concurrent evaluation. This paper describes a conceptual framework for a high level programming language providing both nonprocedural and sequential facilities. Within a program, nonprocedural and sequential program modules may be nested freely. [ABSTRACT FROM AUTHOR]

This paper describes the design philosophy used in the construction of a family of operating systems. It is shown that the concepts of module and level do not coincide in a hierarchy of functions. Family members can share much software as a result of the implementation of run-time modules at the lowest system level. [ABSTRACT FROM AUTHOR]

An axiomatic method for proving a number of properties of parallel programs is presented. Hoare has given a set of axioms for partial correctness, but they are not strong enough in most cases. This paper defines a more powerful deductive system which is in some sense complete for partial correctness. A crucial axiom provides for the use of auxiliary variables, which are added to a parallel program as an aid to proving it correct. The information in a partial correctness proof can be used to prove such properties as mutual exclusion, freedom from deadlock, and program termination. Techniques for verifying these properties are presented and illustrated by application to the dining philosophers problem. [ABSTRACT FROM AUTHOR]

This article examines how the development of batteryless devices are the future of Internet of Things devices. The article first looks into the research that allows for intermittent computing. Necessary components for these batteryless systems, including new programming languages and wireless networks, are discussed. Then the article explores six required directions to make batteryless devices successful, including energy-minimal computer architecture and foundation experimental infrastructure.

This article discusses the application of indistinguishability obfuscation for circuits in the context of the pseudo-canonicalization. The article first discusses arithmetic circuits and the use of pseudo-canonical form for polynomials when hiding implementation choices, then applies this to computer programs with discussion of public-key cryptography and program obfuscation. Then the indistinguishability obfuscator for circuits is defined, its assumptions are detailed, and its applications are discussed.

Mesa is a highly scalable analytic data warehousing system that stores critical measurement data related to Google's Internet advertising business. Mesa is designed to satisfy a complex and challenging set of user and systems requirements, including near real-time data ingestion and retrieval, as well as high availability, reliability, fault tolerance, and scalability for large data and query volumes. Specifically, Mesa handles petabytes of data, processes millions of row updates per second, and serves billions of queries that fetch trillions of rows per day. Mesa is geo-replicated across multiple datacenters and provides consistent and repeatable query answers at low latency, even when an entire datacenter fails. This paper presents the Mesa system and reports the performance and scale that it achieves. [ABSTRACT FROM AUTHOR]

Natural languages like English are rich, complex, and powerful. The highly creative and graceful use of languages like English and Tamil, by masters like Shakespeare and Avvaiyar, can certainly delight and inspire. But in practice, given cognitive constraints and the exigencies of daily life, most human utterances are far simpler and much more repetitive and predictable. In fact, these utterances can be very usefully modeled using modern statistical methods. This fact has led to the phenomenal success of statistical approaches to speech recognition, natural language translation, questionanswering, and text mining and comprehension. We begin with the conjecture that most software is also natural, in the sense that it is created by humans at work, with all the attendant constraints and limitations-and thus, like natural language, it is also likely to be repetitive and predictable. We then proceed to ask whether (a) code can be usefully modeled by statistical language models and (b) such models can be leveraged to support software engineers. Using the widely adopted n-gram model, we provide empirical evidence supportive of a positive answer to both these questions. We show that code is also very regular, and, in fact, even more so than natural languages. As an example use of the model, we have developed a simple code completion engine for Java that, despite its simplicity, already improves Eclipse's completion capability. We conclude the paper by laying out a vision for future research in this area. [ABSTRACT FROM AUTHOR]

The article presents information on the guidelines issued by the Association for Computing Machinery (ACM) which are required for an acceptable degree program in computer science. The concern over the educational product due to an increase in the number of computer science departments in college or university is the main concern of these guidelines. The guidelines include curriculum, faculty, course offerings and facilities. According to the ACM, curriculum must include topics like computer programming, computer software and hardware organization, and system programming.

Contrary to recent claims that specification should be completed before implementation begins, this paper presents two arguments that the two processes must be intertwined. First, limitations of available implementation technology may force a specification change. For example, deciding to implement a stack as an array (rather than as a linked list) may impose a fixed limit on the depth of the stack. Second, implementation choices may suggest augmentations to the original specification. For example deciding to use an existing pattern-match routine to implement the search command in an editor may lead to incorporating some of the routine's features into the specification, such as the ability to include wild cards in the search key. This paper elaborates these points and illustrates how they arise in the specification of a con- troller for a package router. [ABSTRACT FROM AUTHOR]

The article looks at trends in computer science research in 2013. The expansion of computer science as a research field is attributed to emerging application domains and improving hardware and software. The availability of grants is considered as a reflection of the importance of funding for success of computer science research, particularly grants from the U.S. National Science Foundation (NSF). The importance of funding for maintaining research momentum and researcher interest is noted. The field is expected to experience continuous and fundamental transformation.

Computer scientists have long believed that software is different from physical systems in one fundamental way: while the latter have continuous dynamics, the former do not. In this paper, we argue that notions of continuity from mathematical analysis are relevant and interesting even for software. First, we demonstrate that many everyday programs are continuous (i.e., arbitrarily small changes to their inputs only cause arbitrarily small changes to their outputs) or Lipschitz continuous (i.e., when their inputs change, their outputs change at most proportionally). Second, we give an mostly-automatic framework for verifying that a program is continuous or Lipschitz, showing that traditional, discrete approaches to proving programs correct can be extended to reason about these properties. An immediate application of our analysis is in reasoning about the robustness of programs that execute on uncertain inputs. In the longer run, it raises hopes for a toolkit for reasoning about programs that freely combines logical and analytical mathematics. [ABSTRACT FROM AUTHOR]

There are many methods for detecting and mitigating software errors but few generic methods for automatically repairing errors once they are discovered. This paper highlights recent work combining program analysis methods with evolutionary computation to automatically repair bugs in off-the-shelf legacy C programs. The method takes as input the buggy C source code, a failed test case that demonstrates the bug, and a small number of other test cases that encode the required functionality of the program. The repair procedure does not rely on formal specifications, making it applicable to a wide range of extant software for which formal specifications rarely exist. [ABSTRACT FROM AUTHOR]

This paper reports on the development and formal verification (proof of semantic preservation) of Comp Cert, a compiler from Clight (a large subset of the C programming language) to PowerPC assembly code, using the Coq proof assistant both for programming the compiler and for proving its correctness. Such a verified compiler is useful in the context of critical software and its formal verification: the verification of the compiler guarantees that the safety properties proved on the source code hold for the executable compiled code as well. [ABSTRACT FROM AUTHOR]

Evaluation methodology underpins all innovation in experimental computer science. It requires relevant workloads, appropriate experimental design, and rigorous analysis. Unfortunately, methodology is not keeping pace with the changes in our field. The rise of managed languages such as Java, C#, and Ruby in the past decade and the imminent rise of commodity multicore architectures for the next decade pose new methodological challenges that are not yet widely understood. This paper explores the consequences of our collective inattention to methodology on innovation, makes recommendations for addressing this problem in one domain, and provides guidelines for other domains. We describe benchmark suite design, experimental design, and analysis for evaluating Java applications. For example, we introduce new criteria for measuring and selecting diverse applications for a benchmark suite. We show that the complexity and nondeterminism of the Java runtime system make experimental design a first-order consideration, and we recommend mechanisms for addressing complexity and nondeterminism. Drawing on these results, we suggest how to adapt methodology more broadly. To continue to deliver innovations, our field needs to significantly increase participation in and funding for developing sound methodological foundations. [ABSTRACT FROM AUTHOR]

The article offers information on issues relating to the Association for Computing Machinery (ACM). Particular focus is given to the ACM Task Force on Data, Software and Reproducibility. Additional topics discussed include experimental computer science, the "Journal of Experimental Algorithmics," and public confidence in the scientific enterprise.