\documentclass[sigconf,review]{acmart}
\acmConference[SPLC'25]{29th International Systems and Software Product Line Conference}{September 01--September 05, 2025}{A Coruña, Spain}

By submitting your paper to an ACM Publication, you are acknowledging that you and your co-authors are subject to all ACM Publications Policies, including ACM’s new Publications Policy on Research Involving Human Participants and Subjects. Alleged violations of this policy or any ACM Publications Policy will be investigated by ACM and may result in a full retraction of your paper, in addition to other potential penalties, as per ACM Publications Policy.

Please ensure that you and your co-authors obtain an ORCID ID, so you can complete the publishing process for your accepted paper. ACM has been involved in ORCID from the start and has made a commitment to collect ORCID IDs from all authors.

Important Dates

• Challenge case submission: March 13, 2025
• Challenge case notification: April 03, 2025
• Camera-ready version of challenge case: April 10, 2025
• New challenge cases released: April 10, 2025
• Challenge solution submission: June 12, 2025
• Challenge solution notification: July 03, 2025
• Camera-ready version of challenge solution: July 10, 2024

Submission Link and Instructions

TBA

Submission Guidelines

Phase 1 – Call for Challenges: Ideally, a challenge is both feasible and well-defined. Preferably, a challenge case represents a benchmark with well-established data, tooling, and evaluation metrics. However, more open and exploratory challenges are welcome too; as long as they clearly define success criteria for a solution. A challenge should particularly define a set of concrete tasks (some may be optional) to solve and provide links to (additional) resources. If you are unsure how to best present your challenge, please refer to the Section Topics of Interest and the previous challenges linked in it.

In principle, challenges should be kept simple, as complex ones can discourage people from trying to solve them. A challenge can be new, but it is also possible to update and improve existing challenges. For example, there could be new datasets or automated tools for benchmarking that deserve to expand or refine an existing challenge.

Any challenge should at least provide the following information:

• One or more concrete questions/problems you want others to solve, providing a clear motivation why it is an interesting/important challenge (for research, for a company, for an open-source community…).
• For each question/problem, defined criteria for evaluating an answer/solution. Please note again that benchmarks with well-defined criteria are particularly welcome, but we understand that some
  (particularly novel or open problem) challenges may not have a feasible ground truth to define precise metrics. So, evaluation criteria may be (examples):
  
  • Metrics (e.g., precision, recall, accuracy etc. depending on the problem)
  • Test cases participants can evaluate their solution against (e.g., provide inputs and expected outputs, and participants are expected to provide a solution that gets from one point to the other).
  • Existing solutions to evaluate a solution against.
  • A reference implementation to compare against, according to particular metrics.
• A description of any artifacts/tools provided to solve the challenge. These depend on the type of challenge you propose, for example:
  
  • A challenge may be a dataset on which you have questions, so describe
    
    • What is the dataset and how was it obtained?
    • What is the size of the dataset?
    • How can the data be accessed?
    • Do you provide tools to process the data?
    • What are limitations of the dataset?
  • A challenge may be calling to solve a concrete problem, so describe
    
    • What is the problem you want participants to solve?
    • What are resources they can work with/on?
    • What are the constraints?
• A URL to a public repository or artifact page containing all artifacts and instructions (e.g., readme, how to get started) needed to initiate the challenge. We expect that, upon acceptance
  when announcing the new challenge, the artifacts are moved to a persistent open-access repository (e.g., Zenodo) to allow replications and persistency with the final call.
• Any requirements for a solution or a requirement specification document should be published in the repository. The requirements’ description should help researchers who want to address
  the challenge.

Phase 2 – Call for Solutions: Interested in addressing one of the (newly) proposed challenges? Select one, develop a solution, and submit a paper on the results. You are welcome to submit multiple papers, where each paper is related to a different case. Papers must be accompanied by solution artifacts if necessary, depending on the challenge and the concrete tasks that you solved. Thus, solution artifacts may range from hand-crafted sketches or models to automatically generated development artifacts or analysis results. Ideally, solution artifacts are made available in a persistent public repository (e.g., Zenodo) to ensure reproducibility and extend the ground truth for future iterations. A solution paper should include a description of your solution and include an evaluation according to the criteria stated in the respective challenge description. Note that early ideas/results, partial solutions, and experiences of employing existing tools are welcome, too. If you are unsure how to best present your solution, please refer to the Section Topics of Interest and the previous challenges with their solutions linked to it.

Topics of Interest

The topics of interest for challenges are very broad, including, but not limited to:

• Software product lines
• Configuring
• Software features and interactions
• Variability modeling
• Software evolution
• Adaptive systems
• Problem/Solution space variability
• Fork-based development
• Clone and own development

and may relate to various types of techniques, including machine learning, automated reasoning, constraint programming, reverse engineering, performance prediction, testing, fault localization, model checking, or model-driven engineering among others.

As inspiration for a new challenge case or to start working on a solution to a previous challenge, you can visit (new challenges/solutions will be added):

https://variability-challenges.github.io/

Concretely, the previous challenges you can contribute to are:

• Resource Interaction Failures in Mobile Applications: A Challenge for the Software Product Line Testing Community
• Generating Pairwise Covering Arrays for Highly Configurable Software Systems
• A Benchmark for Active Learning of Variability-Intensive Systems
• Variability Fault Localization: A Benchmark
• Managing Systems Evolving in Space and Time: Four Challenges for Maintenance, Evolution and Composition of Variants
• Testing Configurable Software Systems: The Failure Observation Challenge
• Variability Management meets Microservices: Six Challenges of Re-Engineering Microservice-Based Webshops
• A BDD for Linux? The Knowledge Compilation Challenge for Variability
• Applying Product Line Engineering Concepts to Deep Neural Networks
• Product Sampling for Product Lines: The Scalability Challenge
• Apo-Games – A Case Study for Reverse Engineering Variability from Cloned Java Variants
• Feature Location Benchmark with ArgoUML SPL
• Interoperability of Software Product Line Variants
• Localizing Configurations in Highly-Configurable Systems

Track Chairs

Maider Azanza

University of the Basque Country, Spain

Jacob Krüger

Eindhoven University of Technology, The Netherlands

Program Committee