ByteDance Software Engineering Lab

Repo2Run is the first LLM-based agent that automates environment configuration via atomic synthesis and dual-environment architecture, generating error-free Dockerfiles for Python repositories with 86% success rate (63.9% higher than baselines).

We introduce CoReQA, a new benchmark for repository-level question answering, to evaluate large language models' performance in understanding code repositories, highlighting the limitations of current models in effectively addressing repository-level questions.

This paper introduces ContextModule, a framework that enhances LLM-based code completion by retrieving and integrating user behavior-based code, similar code snippets, and critical symbol definitions from the repository, thereby improving accuracy and user acceptance rates.

We propose an automatic classification method using a transformer-based neural network to address the test oracle problem for Go programs.

Map issues and pull requests from GitHub Repositories and produce a benchmark to assess question answering, fault localisation and code editing capabilities.

Proposed a bug reproduction framework via LLM-based agents, which can transform repository issues to test scripts to reproduce bugs.

An empirical study comparing LLM-based agents for automated bug fixing and analysing their performance and capabilities.

By building a multi-agent collaboration framework and providing interactive interfaces and tools for code retrieval, debugging, and editing, MarsCode Agent has fixed 39.33% real software bugs from the SWE-bench Lite benchmark.

A real and industry-level benchmark to assess LLM's code completion capabilities.

This paper proposes Subtask-oriented Reinforced Fine-Tuning (SoRFT), a novel two-stage training framework combining rejection-sampled supervised fine-tuning and rule-based reinforcement learning (PPO with ground-truth rewards), which addresses the limitations of commercial model dependency and poor generalization in issue-resolving tasks. SoRFT achieves SOTA performance among open-source models (e.g., 21.4% resolution on SWE-Bench Verified) through structured subtask decomposition and ground-truth enhanced training.

Studied how LLMs perform on end-to-end software development and provided detailed analysis of product requirement document generation, environment setup, coding, unit test generation and acceptance test generation.

This paper introduces DialogAgent, an automated tool that generates high-fidelity synthetic training data to enhance the performance of large language models in code generation, comprehension, and repair tasks.

Proposed a prompt framework based on user behaviour simulation and evaluation metrics based on user acceptance prediction, to study how LLMs perform on code completion tasks.

Proposed a whole-project code editing framework based on LLMs, for locating, generating and interacting with code edits. Code editing is predicted via related edits, interaction properties and chained effect estimation.

Comparing the similarity between mobile app pages based on graph convolutional networks and detecting visual bugs using app designs and rendered pages on different platforms.

We analyse, improve, and effectively utilise virtual devices for large-scale mobile app testing, demonstrating that high-fidelity testing can be achieved through sensible design and implementation of virtual device farms, and presents solutions to significantly improve testing fidelity, while sharing experiences on enhancing testing efficiency and accessibility with virtual device farms.

Trained a deep reinforcement learning network based on the exploration history of mobile app testing to quickly trigger code-related changes, improving the test effectiveness and fault finding capabilities.

Detecting text glitches for mobile games based on computer vision, uncovering and locating text overlap, text overstep and other text display issues introduced by the internationalisation of mobile apps.

A novel testing approach for detecting Go concurrency bugs through primitive-constrained interleaving exploration, utilizing execution histories to identify new interleavings instead of relying on exhaustive exploration or random scheduling.

An automated unit test generation tool for Go based on random testing and suitable for microservices.

Conducted the first large-scale empirical study to understand and characterize system setting-related defects in mobile apps, triggered by changes in system settings and proposed two synergistic bug-finding techniques: setting-wise metamorphic fuzzing for GUI-level dynamic testing and a static analysis tool for code-level detection.

Modeling history exploration data via reinforcement learning and probabilistic models to improve the effectiveness and fault finding capabilities of mobile app GUI testing.

Presented the design and deployment of the fully automated server interface reliability testing platform at [our company that provides capabilities including (1) traffic data generation based on combinatorial testing and fuzzing, (2) scenario testing for complicated business logics, and (3) automated test execution with fault localization in a controlled environment that does not affect online services.

Generating logical and human-imitating GUI testing event sequences based on a neural network trained on open-source corpus via imitation learning.

Trained a neural network to pair similar Go code snippets for clone detection.

Modeling code graph and code-widget via static analysis and for code changes, selecting actions in GUI testing that are likely to trigger changes.

Existing GUI model-based testing tools may fail when applied to apps with industrial complexity and scalability. We present a multi-agent system which performs model construction on the server end and applies a multi-agent collaboration mechanism to speed up the model construction procedure.