Quality evaluation of floss projects: Application to ERP systems

Context The selection and adoption of open source software can significantly influence the competitiveness of organisations. Open source software solutions offer great opportunities for cost reduction and quality improvement, especially for small and medium enterprises that typically have to address major difficulties due to the limited resources available for selecting and adopting a new software system. Objective This paper aims to provide support for selecting the open source software that is most suitable to the specific needs of an enterprise from among the options offering equivalent or overlapping functionality. Method This paper proposes a framework for evaluating the quality and functionality of open source software systems. The name of the framework is EFFORT (Evaluation Framework for Free/Open souRce projecTs). It supports the evaluation of product quality, community trustworthiness and product attractiveness. The framework needs to be customised to the analysis of software systems for a specific context. Results The paper presents the customisation of EFFORT for evaluating Enterprise Resource Planning (ERP) open source software systems. The customised framework was applied to the evaluation and comparison of five ERP open source software systems. The results obtained permitted both the refinement of the measurement framework and the identification of the ERP open source software system that achieved the highest score for each chosen characteristic. Conclusion EFFORT is a useful tool for evaluating and selecting an open source software system. It may significantly reduce the amount of negotiation conducted among an enterprise’s members and reduce the time and cost required for gathering and interpreting data. The EFFORT framework also considers the users’ opinions by introducing relevance markers associated with the metrics and questions in the data aggregation process.