Search

Recent work shows Large Language Models (LLMs) struggle to understand natural language constraints for various text generation tasks in zero- and few-shot settings. While, in the code domain, there is wide usage of constraints in code format to maintain the integrity of code written in Domain-Specific Languages (DSLs) like JSON and YAML which are widely used for system-level programming tasks in enterprises. Given that LLMs are increasingly used for system-level code tasks, evaluating if they can comprehend these code constraints is crucial. However, no work has been done to evaluate their controllability over code constraints. Hence, we introduce ConCodeEval, a first-of-its-kind benchmark having two novel tasks for code constraints across five representations. Our findings suggest that language models struggle with code constraints. Code languages that perform excellently for normal code tasks do not perform well when the same languages represent fine-grained constraints.

Recent developments show that Large Language Models (LLMs) produce state-of-the-art performance on natural language (NL) to code generation for resource-rich general-purpose languages like C++, Java, and Python. However, their practical usage for structured domain-specific languages (DSLs) such as YAML, JSON is limited due to domain-specific schema, grammar, and customizations generally unseen by LLMs during pre-training. Efforts have been made to mitigate this challenge via in-context learning through relevant examples or by fine-tuning. However, it suffers from problems, such as limited DSL samples and prompt sensitivity but enterprises maintain good documentation of the DSLs. Therefore, we propose DocCGen, a framework that can leverage such rich knowledge by breaking the NL-to-Code generation task for structured code languages into a two-step process. First, it detects the correct libraries using the library documentation that best matches the NL query. Then, it utilizes schema rules extracted from the documentation of these libraries to constrain the decoding. We evaluate our framework for two complex structured languages, Ansible YAML and Bash command, consisting of two settings: Out-of-domain (OOD) and In-domain (ID). Our extensive experiments show that DocCGen consistently improves different-sized language models across all six evaluation metrics, reducing syntactic and semantic errors in structured code. We plan to open-source the datasets and code to motivate research in constrained code generation.

A numerical simulation of Kelvin-Helmholtz Instability (KHI) in parallel shear flows subjected to external rotation is carried out using a pseudo-spectral technique. The Coriolis force, arising in a rotation frame under the beta plane approximation, tends to suppress the growth of KHI modes. The numerical results show a close qualitative agreement with the analytical results obtained for a step-wise shear flow profile. Experimental evidence demonstrates that particles in a rotating frame experience the Coriolis force, mathematically equivalent to the Lorentz force. Therefore, the Coriolis force affects fluid dynamics in a manner similar to the Lorentz force in magnetized shear flows. This paper exploits the analogy between the magnetic field and rotation to study effects equivalent to a magnetic field on KHI in a rotating frame. Similar to the magnetic field case, the Coriolis force suppresses KHI and tends to form compressed and elongated KH vortex structures. However, the magnetic field and Coriolis force act on different scales, with the latter suppressing long-wavelength mode perturbations. A higher number of vortices are observed in the presence of rotation compared to non-rotating cases

While text summarization is a well-known NLP task, in this paper, we introduce a novel and useful variant of it called functionality extraction from Git README files. Though this task is a text2text generation at an abstract level, it involves its own peculiarities and challenges making existing text2text generation systems not very useful. The motivation behind this task stems from a recent surge in research and development activities around the use of large language models for code-related tasks, such as code refactoring, code summarization, etc. We also release a human-annotated dataset called FuncRead, and develop a battery of models for the task. Our exhaustive experimentation shows that small size fine-tuned models beat any baseline models that can be designed using popular black-box or white-box large language models (LLMs) such as ChatGPT and Bard. Our best fine-tuned 7 Billion CodeLlama model exhibit 70% and 20% gain on the F1 score against ChatGPT and Bard respectively.

Prompting with natural language instructions has recently emerged as a popular method of harnessing the capabilities of large language models. Given the inherent ambiguity present in natural language, it is intuitive to consider the possible advantages of prompting with less ambiguous prompt styles, such as the use of pseudo-code. In this paper we explore if prompting via pseudo-code instructions helps improve the performance of pre-trained language models. We manually create a dataset of pseudo-code prompts for 132 different tasks spanning classification, QA and generative language tasks, sourced from the Super-NaturalInstructions dataset. Using these prompts along with their counterparts in natural language, we study their performance on two LLM families - BLOOM and CodeGen. Our experiments show that using pseudo-code instructions leads to better results, with an average increase (absolute) of 7-16 points in F1 scores for classification tasks and an improvement (relative) of 12-38% in aggregate ROUGE-L scores across all tasks. We include detailed ablation studies which indicate that code comments, docstrings, and the structural clues encoded in pseudo-code all contribute towards the improvement in performance. To the best of our knowledge our work is the first to demonstrate how pseudo-code prompts can be helpful in improving the performance of pre-trained LMs., Comment: Published in EMNLP 2023 main track

Strongly coupled systems occupying the transitional range between the Wigner crystal and fluid phases are most dynamic constituents of the nature. Highly localized but strongly interacting elements in this phase posses enough thermal energy to trigger the transition between a variety of short to long range order phases. Nonlinear excitations are often the carriers of proliferating structural modifications in the strongly coupled Yukawa systems. Well represented by a laboratory dusty plasma, these systems show explicit propagation of nonlinear shocks and solitary structures both in experiments and in first principle simulations. The shorter scale length contributions remain absent at strong screening in present approximate models which nevertheless prescribe nonlinear solitary solutions that consequently lose their coherence in a numerical evolution of the system under a special implementation of the quasi-localized charge approximation formulation. The stable coherent structures self-consistently emerge following an initial transient in the numerical evolution which adapts QLCA approach to spatiotemporal domain for accessing the nonlinear excitations in the strong screening limit. The present kappa ~ 1 limit of the existing Yukawa fluid models to show agreement with the experiment and MD simulations has therefore been overcome and the coherent nonlinear excitaitons have become characterizable up to kappa ~ 2.7, before they becoming computationally challenging in present implementation., Comment: 17 figures

Collective response dynamics of a strongly coupled system departs from the continuum phase upon transition to the quasicrystalline phase, or formation of a Wigner lattice. The wave nonlinearity leading to the modulational instability in recent studies, for example, of a quasicrystalline dusty plasma lattice, predicts inevitable emergence of macroscopic structures from mesoscopic carrier fluctuations. The modulational instability in the quasi crystalline or amorphous phase of a strongly coupled system, uniquely accessed under the quasi-localized charge approximation (QLCA), generates a narrower instability regime for entire spectral range. In comparison to the linear one dimensional chains of strongly coupled dust grains, the longitudinal modes for quasicrystalline phase show finite distinction in terms of the instability regime. The present QLCA based analysis shows system to be stable for arbitrarily long wavelength of perturbation for full range of screening parameter $\kappa=a/\lambda_{\rm D}$ beyond the value $\kappa=0.182$, where $a$ is the inter dust separation and $\lambda_{\rm D}$ is the plasma Debye length. However, this unstable region continuously grows with increase in the dust temperature which invoke the weak coupling effects. The present results show that as compared to the one dimensional chains, the more practical 2D and 3D strongly coupled systems are potentially stable with respect to the macroscopic amplitude modulations. The development of macroscopic structures from the mesoscopic fluctuations is therefore predicted to be rather restricted for strongly coupled systems with implications for systems where strongly coupled species are in a quasi-localized (semi-solid) phase.

Collective excitations of rotating dusty plasma are analyzed under the quasi localized charge approximation (QLCA) framework for strongly coupled systems by explicitly accounting for the dust rotation in the analysis. Considering the firm analogy of magnetoplasmons with "rotoplasmons" established by the recent rotating dusty plasma experiments, the relaxation introduced by the rotation in their strong coupling and 2-dimensional (often introduced by gravitational sedimentation) characteristics is emphasized in their dispersion. Finite rotation version of both strong and weak coupling dispersions is derived and analyzed, showing correspondence between a `faster rotating but weakly coupled' branch and its strongly coupled counterpart, relevant to both magnetized and unmagnetized dust experiments, in gravity or microgravity conditions. The first correspondence between their measurements in rotating plasmas and the QLCA produced dispersions in a rotating frame, with an independent numerical validation, is presented in detail.