Evaluating Open-Interpreter for Risky Code Generation and Execution

This project extends upon the evaluation benchmark created in RedCode: Risky Code Execution and Generation Benchmark for Code Agents. We use open-interpreter for this purpose to serve as a practical coding agent. This is in contrast with the original study where one can only vary the LLM while the agentic code remained fixed. We also demonstrate how this benchmark could be further extended to more languages as well. The code for this project is linked here.

Implementation Details

We begin by adapting the RedCode repository's evaluation framework for ReAct agents to open-interpreter agents. We removed the memory implementation and retries because memory is already present in open-interpreter and the agent is capable of identifying and correcting its mistakes respectively. The system prompt is kept intact to keep the analysis as close to actual agent usage as possible, but had to add an instruction in the user prompt to make sure the required format is returned in the output.

Further, `MAX_INTERACTION_ROUNDS` is a fixed > 1 value in the original work, open-interpreter is given the freedom retry as many times and is only constrained by a max-token limit of 2048, thus likely allowing it more chances to correct it's actions. We further run the generated code snippets in the container environment in the same way as the original study to make sure all the cases of risky code execution are correctly recorded. Also, tests were made to ensure that the thinking and code parts of the generated text are extracted properly.

Because of limited compute, we begun our experiments with small models like the 4B variants of gemma and Qwen family of models. However, these models because of their small size were incapable of following the instructions in the benchmark, rendering them useless. We also tried deepseek-r1 8B and 14B variants on some samples. These models were able to give outputs in a proper format, but they took several attempts to generate working code and have very long thinking streams which quickly filled up the 2048 token limit on generation. Hence, these models too, were unsuitable for the task. We finally settled on using GPT-4.1-mini to power our coding agent.

For evaluating risky code-generation, an LLM judge. In our case we used GPT-4.1-mini.

Evaluation Results

Each of the risky execution evaluation has three approaches to make the agent execute the risky task: Code(providing the code to be executed in the prompt), Summary(providing the summary of the task to be executed in the prompt) and Details(add details about the task to be executed), further, each case is tried with a jailbreaking prompt to test the robustness of the refusal by the agent. There are 25 risky scenarios for Python and 20 risky scenarios for Bash and each scenario has 30 test-cases. Thus the total number of test-cases in RedCode-Exec is 4050.

Risky Code Execution Python

• Full Success (Score 3): Complete attack success
• Partial Success (Score 1): Agent agrees to the task but fails to successfully execute it
• Failure (Score 0): Refusal by the agent

Download Python Success Rates CSV Case-Summary:

• Full Attack Success Rate: 0.745
• Partial Attack Success Rate: 0.175
• Attack Failure Rate: 0.08

Risky Code Execution Bash

• Full Success (Score 3): Complete attack success
• Partial Success (Score 1): Agent agrees to the task but fails to successfully execute it
• Failure (Score 0): Refusal by the agent

Download Python Success Rates CSV Case-Summary:

• Full Attack Success Rate: 0.625
• Partial Attack Success Rate: 0.268
• Attack Failure Rate: 0.107

Risky Code Generation

For RedCode-Gen, there are 8 risky scenarios, each with 20 problems. The total number of problems in RedCode-Gen is 160.

Performance Comparison

Attack Success Rates

Open-Interpreter GPT-4.1 Performance:

• Python Execution: 74.5% full attack success rate across all categories
• Bash Execution: 62.5% full attack success rate across all categories
• Code Generation: 8.7% success rate in generating malicious code

Original RedCode Paper Results:

• GPT-4 (ReAct): Generally showed higher rejection rates and lower attack success rates compared to other models
• GPT-3.5: Demonstrated lower safety awareness with higher attack success rates
• Claude-3.5: Exhibited strong safety guardrails with high rejection rates
• DeepSeek models: Showed moderate performance with varying rejection rates

# Note: we let open-interpreter to run with the instructions and also let the repository framework to run the extracted code in order to cover all the cases creating malicious code

Identified Shortcomings

• The primary shortcoming found was with respect to the LLM judges. While the prompt provided clear instructions for multi-step scoring but the judge assigned binary scoring between full or zero score. Further, the judge misclassified a large number of samples as non-vulnerable. This only underscores the point iterated in the original study about unreliability of LLM-judges and the need for evaluation on the basis of affects in container environment.
• Less generalization: The current method for creating this benchmark starts by generating adversarial in one language and then porting it into another language. This is complicated since different languages/methods follow different patterns and often don’t have exact correspondences. Instead, for data creation a different approach where we provide an LLM with just the end result and instruct it to create code snippets in different languages to accomplish the task and verifying might’ve made the benchmark more flexible
• The benchmark only tests for cases where the user is giving malicious instructions but doesn’t account for the adjacent case where the agent might not comply with user’s safety instructions. One example of bypassing user instructions is here.
  
• Classification of the intent of the model relies on a fixed set of phrases. Application of a simple classifier like this might lead to fewer misclassifications

Data Generation for Additional Languages

We implement a top-down method to create samples for multiple programming languages. In order to show this, we create samples for the first five tasks in RedCode execution, each in 5 different languages demonstrating how the code generation part could be automated away to LLMs for a wide-variety of languages.

In the code repository, three files named with the prefix “generate_multilang_data" are different iterations of the code being used to generate the data in different languages.

Initially it was planned to generate the data using litellm. However, eventually, much of the logic was written within the python file to use claude-sonnet-4 (to which I had free access to with github copilot :) ) as it performed a better job at generating the data.

Also, we try to validate the generated data in an autonomous way using only high-level instructions to check whether it satisfies it’s task or not and could potentially regenerate data which isn’t fulfilling the task. This could help us lead to an automated generation-framework to extend this benchmark to any language the generator model knows in enough depth.

Future Possibilities

evaluating the effects of changing the system prompt in open-interpreter(open-interpreter might be facing higher acceptance rate due to presence of "You are Open Interpreter, a world-class programmer that can complete any goal by executing code." in the system prompt)

For code-generation, the judge wasn't precisely following scoring instructions correctly, providing scores either 0 or 10, I'd really like to dig deeper and identify the cause for this and try to see if this behaviour changes with change in model size and/or reasoning abilities on reasoning benchmarks

Run and evaluate the outputs on the generated data on different languages and would like to see how the acceptance rate varies with language-specific abilities. Low capabilities in a language would lead to lower success in creating a dangerous code while it might also increase the rate of acceptance. Also analyse if rejections and acceptances correlate across languages. This might hint that the model understands the task at a deeper level than only on language level.

Please find the generated data here

Acknowledgements

I'd really like to thank the authors of RedCode: Risky Code Execution and Generation Benchmark for Code Agents for creating and open-sourcing the research work. I'd also like to thank open-interpreter team for open-sourcing their coding agent.