Search

Large Language Models (LLMs) have had considerable difficulty when prompted with mathematical questions, especially those within theory of computing (ToC) courses. In this paper, we detail two experiments regarding our own ToC course and the ChatGPT LLM. For the first, we evaluated ChatGPT's ability to pass our own ToC course's exams. For the second, we created a database of sample ToC questions and responses to accommodate other ToC offerings' choices for topics and structure. We scored each of ChatGPT's outputs on these questions. Overall, we determined that ChatGPT can pass our ToC course, and is adequate at understanding common formal definitions and answering "simple"-style questions, e.g., true/false and multiple choice. However, ChatGPT often makes nonsensical claims in open-ended responses, such as proofs., Comment: Accepted to SIGCSE Virtual 2024

Best practices in programming need to be emphasized in a CS1 course as bad student habits persist if not reinforced well. The C++ programming language, although a relatively old language, has been regularly updated with new versions since 2011, on the pace of once every three years. Each new version contains important features that make the C++ language more complex for backwards compatibility, but often introduce new features to make common use cases simpler to implement. This poster contains experiences in designing a CS1 course that uses the C++ programming language that incorporates ``modern'' versions of the language from the start, as well as recent conferences about the language. Our goals were to prevent many common bad habits among C++ programmers., Comment: Accepted to SIGCSE TS 2024 (poster)

The design of any technical Computer Science course must involve its context within the institution's CS program, but also incorporate any new material that is relevant and appropriately accessible to students. In many institutions, theory of computing (ToC) courses within undergraduate CS programs are often placed near the end of the program, and have a very common structure of building off previous sections of the course. The central question behind any such course is ``What are the limits of computers?'' for various types of computational models. However, what is often intuitive for students about what a ``computer'' is--a Turing machine--is taught at the end of the course, which necessitates motivation for earlier models. This poster contains our experiences in designing a ToC course that teaches the material effectively ``backwards,'' with pedagogic motivation of instead asking the question ``What suitable restrictions can we place on computers to make their problems tractable?'' We also give recommendations for future course design., Comment: Accepted to SIGCSE TS 2024 (poster)

Factors within a large-scale software system that simultaneously interact and strongly impact the system's response under a configuration are often difficult to identify. Although screening such a system for the existence of such interactions is important, determining their location is more useful for system engineers. Combinatorial interaction testing (CIT) concerns creation of test suites that nonadaptively either detect or locate the desired interactions, each of at most a specified size or show that no such set exists. Under the assumption that there are at most a given number of such interactions causing such a response, locating arrays (LAs) guarantee unique location for every such set of interactions and an algorithm to deal with outliers and nondeterministic behavior from real systems, we additionally require the LAs to have a "separation" between these collections. State-of-the-art approaches generate LAs that can locate at most one interaction of size at most three, due to the massive number of interaction combinations for larger parameters if no constraints are given. This paper presents LocAG, a two-stage algorithm that generates (unconstrained) LAs using a simple, but powerful partitioning strategy of these combinations. In particular, we are able to generate LAs with more factors, with any desired separation, and greater interaction size than existing approaches.

Theory of computing (ToC) courses are a staple in many undergraduate CS curricula as they lay the foundation of why CS is important to students. Although not a stated goal, an inevitable outcome of the course is enhancing the students' technical reading and writing abilities as it often contains formal reasoning and proof writing. Separately, many undergraduate students are interested in performing research, but often lack these abilities. Based on this observation, we emulated a common research environment within our ToC course by creating a mock conference assignment, where students (in groups) both wrote a technical paper solving an assigned problem and (individually) anonymously refereed other groups' papers. In this paper we discuss the details of this assignment and our experiences, and conclude with reflections and future work about similar courses., Comment: Accepted to SIGCSE TS 2024

A \emph{covering array} is an $N \times k$ array of elements from a $v$-ary alphabet such that every $N \times t$ subarray contains all $v^t$ tuples from the alphabet of size $t$ at least $\lambda$ times; this is denoted as $\CA_\lambda(N; t, k, v)$. Covering arrays have applications in the testing of large-scale complex systems; in systems that are nondeterministic, increasing $\lambda$ gives greater confidence in the system's correctness. The \emph{covering array number}, $\CAN_\lambda(t,k,v)$ is the smallest number of rows for which a covering array on the other parameters exists. For general $\lambda$, only several nontrivial bounds are known, the smallest of which was asymptotically $\log k + \lambda \log \log k + o(\lambda)$ when $v, t$ are fixed. Additionally it has been conjectured that the $\log \log k$ term can be removed. First, we affirm the conjecture by deriving an asymptotically optimal bound for $\CAN_\lambda(t,k,v)$ for general $\lambda$ and when $v, t$ are constant using the Stein--Lov\'asz--Johnson paradigm. Second, we improve upon the constants of this method using the Lov\'asz local lemma. Third, when $\lambda=2$, we extend a two-stage paradigm of Sarkar and Colbourn that improves on the general bound and often produces better bounds than even when $\lambda=1$ of other results. Fourth, we extend this two-stage paradigm further for general $\lambda$ to obtain an even stronger upper bound, including using graph coloring. And finally, we determine a bound on how large $\lambda$ can be for when the number of rows is fixed., Comment: 19 pages

Column replacement techniques for creating covering arrays rely on the construction of perfect and distributing hash families with few rows, having as many columns as possible for a specified number of symbols. To construct distributing hash families in which the number of rows is less than the strength, we examine a method due to Blackburn and extend it in three ways. First, the method is generalized from homogeneous hash families (in which every row has the same number of symbols) to heterogeneous ones. Second, the extension treats distributing hash families, in which only separation into a prescribed number of parts is required, rather than perfect hash families, in which columns must be completely separated. Third, the requirements on one of the main ingredients are relaxed to permit the use of a large class of distributing hash families, which we call fractal. Constructions for fractal perfect and distributing hash families are given, and applications to the construction of perfect hash families of large strength are developed., Comment: 21 pages, 0 figures

It is imperative for testing to determine if the components within large-scale software systems operate functionally. Interaction testing involves designing a suite of tests, which guarantees to detect a fault if one exists among a small number of components interacting together. The cost of this testing is typically modeled by the number of tests, and thus much effort has been taken in reducing this number. Here, we incorporate redundancy into the model, which allows for testing in non-deterministic environments. Existing algorithms for constructing these test suites usually involve one "fast" algorithm for generating most of the tests, and another "slower" algorithm to "complete" the test suite. We employ a genetic algorithm that generalizes these approaches that also incorporates redundancy by increasing the number of algorithms chosen, which we call "stages." By increasing the number of stages, we show that not only can the number of tests be reduced compared to existing techniques, but the computational time in generating them is also greatly reduced., Comment: Submitted to Genetic and Evolutionary Computation Conference 2020 (GECCO '20)

Let $x$ and $y$ be words. We consider the languages whose words $z$ are those for which the numbers of occurrences of $x$ and $y$, as subwords of $z$, are the same (resp., the number of $x$'s is less than the number of $y$'s, resp., is less than or equal). We give a necessary and sufficient condition on $x$ and $y$ for these languages to be regular, and we show how to check this condition efficiently.