Skip to content

Commit 8e4f3d9

Browse files
WangGLJosephWangGLJosephMarti2203autocoderover[bot]
authored
Update Pytest to 80% coverage
* add PyTest framework * update pytest yml * update CI * activate workflow * fix branch name * add missing checkout * add correct docker command * add sonar source files * output converage to both term and xml * use acr-pytest for container name * Update pytest.yml * Update sonar-project.properties * sonar fix and update sonar exclusions * add coverage file path * reduce code duplication * add temp dir in workflow for coverage report * update sonar config * use tox and pytest outside container * add missing tox.ini file * separate pytest and build workflow * fix conda env name in workflow * update shell to use login mode for pytest workflow * update conda config * set PYTHONPATH inside tox.ini * update tox.ini * include Coverage.py in tox * add passenv to tox * change build yaml to only run on PR merge * remove redundant deps in tox.ini * abort Sonarqube Scan if coverage.xml is missing * specify source and test files for sonar * add test_search_utils * change build to docker_build * update tox usage in workflow and documentation * add test for search_utils.py * fix get_code_region_around_line * update testcase (need to clarify class signature) * fix sonar issues and update README * add empty test files for search subdir * add testcase for internal methods of SearchBackend * update tox.ini for more accurate coverage percentage * fix test_search_utils * add tests and add `converage` to whitelist for tox testenv * update test cases * update test_saerch_backend * add testcases for search subdir * update TESTING.md with setup instructions for adding `conda-forge` * update tox.ini * add empty test files for api subdir * implement fix from PR #81 * Fake error * Fix failing test * Fix fake error and add new one * Update app/data_structures.py Co-authored-by: autocoderover[bot] <182712390+autocoderover[bot]@users.noreply.github.com> * update readme * improve coverage of search_manage.py * improve coverage for search_utils * show only analysis cov * add empty test_sbfl * remove redundant import stmts * add test cases for sbfl.py * run actions on pytest-unit branch * add testing for api/validation.py * refactor common test utils and make it a package, add test cases * ensure coverage on new code * add test cases for api/eval_helper.py * enable printing branch coverage on CI * cover some missing branches in search_backend * complete coverage for search/search_backend.py * remove redundant collection * add test for gpt.py * update test for catching BadRequestError * disable pytest output capturing to show stdout * improve coverage of test_gpt.py * refactor common test code to pytest_utils * add unit test for model/claude.py * remove redundant declaration (refactored into pytest_utils) * add tests for models * add tests for azure.py * test bedrock * add problematic definition for Sonar scan * try changing sonar.sources to include test/ * modify sonar scan config * revert to previous sonar config for showing coverage data * finish testing models submodule * add empty test agent files * update ci * improve coverage of test_agent_search * test_patch_utils * test_agent_reproducer * test_agent_write_patch * test_agent_select * test_agent_reviewer * test_agent_proxy and refactor DummyModel to pytest_utils * test main app dir classes * test_manage * add unit test for GitHubTask and LocalTask * test_inference * move app test files * add test files * update build files for osx-arm64 envO * add app level testing * add tests for running raw tasks * improve coverage * setup multi-env for tox pytest * update ci yaml * update ci yaml to refer to github secrets for LLM API keys * add anthropic integration test (negative case with dummy api key) * update integration test ci * integration test with anthropic * remove logging * add simple openai integration test * remove print * update integration * update ci branch names * add missing eof `fi` * update logging * update env setup files * setup python black * setup pyright * remove logging * update ci * update actions * update ci yml * update regex for test branches --------- Co-authored-by: WangGLJoseph <[email protected]> Co-authored-by: Martin Mirchev <[email protected]> Co-authored-by: autocoderover[bot] <182712390+autocoderover[bot]@users.noreply.github.com>
1 parent 7ac5c95 commit 8e4f3d9

Some content is hidden

Large Commits have some content hidden by default. Use the searchbox below for content that may be hidden.

48 files changed

+8285
-544
lines changed

‎.github/workflows/pytest.yml

+23-2
Original file line numberDiff line numberDiff line change
@@ -4,14 +4,19 @@ on:
44
push:
55
branches:
66
- main
7-
- pytest-ci
7+
- pytest-latest
8+
- pytest-integration
89
pull_request:
910
branches:
1011
- main
1112

1213
jobs:
1314
pytest:
1415
runs-on: ubuntu-latest
16+
env:
17+
ANTHROPIC_API_KEY: ${{ secrets.ANTHROPIC_API_KEY }}
18+
OPENAI_KEY: ${{ secrets.OPENAI_KEY }}
19+
GROQ_API_KEY: ${{ secrets.GROQ_API_KEY }}
1520
defaults:
1621
run:
1722
shell: bash -l {0}
@@ -41,7 +46,16 @@ jobs:
4146
run: conda install -y tox
4247

4348
- name: Run tox tests
44-
run: tox
49+
run: |
50+
branch="${{ github.ref_name }}"
51+
echo "Current branch: $branch"
52+
if [[ "$branch" =~ ^(main(-.*)?|feat/.*/integration-.*)$ ]]; then
53+
echo "Running unit + integration tests."
54+
tox -e unit,integration
55+
else
56+
echo "Running unit tests only."
57+
tox -e unit
58+
fi
4559
4660
- name: Check Coverage Report Exists
4761
run: |
@@ -50,6 +64,13 @@ jobs:
5064
exit 1
5165
fi
5266
67+
- name: Print Branch Coverage
68+
run: |
69+
echo "Branch Coverage Details:"
70+
echo "Total Branches: $(grep -o 'branches-valid="[0-9]*"' coverage.xml | sed 's/branches-valid="//;s/"//')"
71+
echo "Branches Covered: $(grep -o 'branches-covered="[0-9]*"' coverage.xml | sed 's/branches-covered="//;s/"//')"
72+
echo "Branch Rate: $(grep -o 'branch-rate="[0-9\.]*"' coverage.xml | sed 's/branch-rate="//;s/"//')"
73+
5374
- name: SonarQube Scan
5475
uses: SonarSource/sonarqube-scan-action@v4
5576
env:

‎.gitignore

+2-1
Original file line numberDiff line numberDiff line change
@@ -159,4 +159,5 @@ cython_debug/
159159
# option (not recommended) you can uncomment the following to ignore the entire idea folder.
160160
#.idea/
161161

162-
output/
162+
output/
163+
setup/

‎ACR.py

+1
Original file line numberDiff line numberDiff line change
@@ -1,3 +1,4 @@
11
from app import main
2+
23
if __name__ == "__main__":
34
main.main()

‎TESTING.md

+9
Original file line numberDiff line numberDiff line change
@@ -11,10 +11,19 @@ The testing suite uses the following libraries and tools:
1111
- Pytest, to execute the tests
1212
- Coverage, (the Coverage.py tool) to measure the code coverage
1313

14+
15+
Creating the `auto-code-rover` environment using the `environment.yaml` file:
16+
```bash
17+
conda env create -f environment.yml
18+
conda activate auto-code-rover
19+
```
20+
21+
1422
In the `auto-code-rover` environment, add `conda-forge` as a channel, then install the required libraries by running the following command:
1523

1624
```bash
1725
conda config --add channels conda-forge
26+
conda config --set channel_priority flexible
1827
conda install -y tox
1928
```
2029

‎app/data_structures.py

+1-1
Original file line numberDiff line numberDiff line change
@@ -51,7 +51,7 @@ def __init__(
5151
)
5252

5353
def __str__(self):
54-
return f"FunctionCallIntent(func_name={str(self.func_name)}, arguments={str(self.arg_values)})"
54+
return f"FunctionCallIntent(func_name={self.func_name}, arguments={self.arg_values})"
5555

5656
def to_dict(self):
5757
return {"func_name": self.func_name, "arguments": self.arg_values}

‎environment.osx-arm64.yml

+22
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,22 @@
1+
name: auto-code-rover
2+
channels:
3+
- conda-forge
4+
- defaults
5+
dependencies:
6+
- bzip2=1.0.8
7+
- ca-certificates=2024.7.2
8+
- libffi=3.4.4
9+
- ncurses=6.5
10+
- openssl=3.3.1
11+
- pip=24.2
12+
- python=3.10.11
13+
- readline=8.2
14+
- setuptools=68.2.2
15+
- sqlite=3.46.0
16+
- tk=8.6.14
17+
- tzdata=2023d
18+
- wheel=0.41.2
19+
- xz=5.4.6
20+
- rich=13.8.1
21+
- pip:
22+
- -r requirements.osx-arm64.txt

‎pyrightconfig.json

+6
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,6 @@
1+
{
2+
"include": ["app"],
3+
"exclude": ["venv"],
4+
"pythonVersion": "3.10",
5+
"typeCheckingMode": "basic"
6+
}

‎pytest.ini

+4
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,4 @@
1+
[pytest]
2+
norecursedirs = setup output
3+
markers =
4+
integration: mark a test as an integration test.

‎requirements.osx-arm64.txt

+115
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,115 @@
1+
aiohttp==3.10.2
2+
aiosignal==1.3.1
3+
annotated-types==0.6.0
4+
antlr4-tools==0.2.1
5+
anyio==4.2.0
6+
astroid==3.2.3
7+
attrs==23.2.0
8+
beautifulsoup4==4.12.3
9+
black==25.1.0
10+
Brotli
11+
certifi==2024.7.4
12+
cffi
13+
cfgv==3.4.0
14+
charset-normalizer
15+
click==8.1.7
16+
coverage==7.5.3
17+
Cython==3.0.8
18+
dill==0.3.8
19+
discord==2.3.2
20+
discord.py==2.3.2
21+
distlib==0.3.8
22+
distro==1.9.0
23+
docker==7.0.0
24+
docstring-parser==0.15
25+
emojis==0.7.0
26+
filelock==3.13.1
27+
frozenlist==1.4.1
28+
fsspec==2024.6.1
29+
h11==0.14.0
30+
h2
31+
hpack==4.0.0
32+
httpcore==1.0.5
33+
httpx==0.27.0
34+
huggingface-hub==0.23.5
35+
hyperframe
36+
icecream==2.1.3
37+
identify==2.5.33
38+
idna==3.7
39+
importlib-metadata==7.0.1
40+
install-jdk==1.1.0
41+
isort==5.13.2
42+
javalang==0.13.0
43+
Jinja2==3.1.4
44+
jsonschema==4.22.0
45+
jsonschema-specifications==2023.12.1
46+
libclang==18.1.1
47+
linkify-it-py==2.0.2
48+
litellm==1.44.8
49+
loguru==0.7.2
50+
lxml==5.1.0
51+
markdown-it-py==3.0.0
52+
MarkupSafe==2.1.5
53+
mccabe==0.7.0
54+
mdit-py-plugins==0.4.0
55+
mdurl==0.1.2
56+
more-itertools==10.2.0
57+
mpmath==1.3.0
58+
multidict==6.0.4
59+
networkx==3.2.1
60+
nodeenv==1.8.0
61+
numpy==1.26.4
62+
natsort==8.4.0
63+
ollama==0.3.3
64+
openai==1.50.2
65+
opt-einsum==3.3.0
66+
packaging==23.2
67+
platformdirs==4.1.0
68+
polars==0.20.31
69+
pre-commit==3.6.0
70+
pycparser
71+
pydantic==2.5.3
72+
pydantic_core==2.14.6
73+
Pygments==2.17.2
74+
pylint==3.2.3
75+
pyro-api==0.1.2
76+
pyro-ppl==1.9.0
77+
PySocks
78+
pyright==1.1.397
79+
pytest==8.3.4
80+
pytest-cov==6.0.0
81+
python-dotenv==1.0.0
82+
PyYAML==6.0.1
83+
referencing==0.32.1
84+
regex==2024.5.15
85+
rich==13.7.1
86+
rpds-py==0.16.2
87+
semver==3.0.2
88+
slack_sdk==3.26.2
89+
sniffio==1.3.0
90+
soupsieve==2.5
91+
sympy==1.13.0
92+
tenacity==8.2.3
93+
termcolor==2.4.0
94+
textual==0.52.1
95+
tiktoken==0.7.0
96+
timeout-decorator==0.5.0
97+
tokenizers==0.19.1
98+
tomlkit==0.13.0
99+
torch==2.2.1
100+
tqdm==4.66.4
101+
tree-sitter==0.21.3
102+
tree-sitter-c==0.21.4
103+
tree-sitter-cpp==0.22.2
104+
tree-sitter-java==0.21.0
105+
tree-sitter-languages==1.10.2
106+
types-jsonschema==4.21.0.20240311
107+
typing_extensions==4.12.2
108+
uc-micro-py==1.0.2
109+
unidiff==0.7.5
110+
unittest-xml-reporting==3.2.0
111+
urllib3
112+
virtualenv==20.25.0
113+
yarl==1.9.4
114+
zipp==3.19.2
115+
zstandard==0.22.0

‎test/__init__.py

Whitespace-only changes.

‎test/app/agents/test_agent_common.py

+12-4
Original file line numberDiff line numberDiff line change
@@ -1,23 +1,29 @@
11
from app.data_structures import MessageThread
22
from app.agents.agent_common import replace_system_prompt, InvalidLLMResponse
33

4+
45
def test_replace_system_prompt():
56
# Setup: create a MessageThread with a system message and another message
67
original_prompt = "Original System Prompt"
78
new_prompt = "New System Prompt"
89
messages = [
910
{"role": "system", "content": original_prompt},
10-
{"role": "user", "content": "Hello"}
11+
{"role": "user", "content": "Hello"},
1112
]
1213
msg_thread = MessageThread(messages=messages)
1314

1415
# Execute: replace the system prompt
1516
updated_thread = replace_system_prompt(msg_thread, new_prompt)
1617

1718
# Verify: first message should now have the new prompt
18-
assert updated_thread.messages[0]["content"] == new_prompt, "System prompt was not replaced correctly."
19+
assert (
20+
updated_thread.messages[0]["content"] == new_prompt
21+
), "System prompt was not replaced correctly."
1922
# Verify: the rest of the messages remain unchanged
20-
assert updated_thread.messages[1]["content"] == "Hello", "User message was unexpectedly modified."
23+
assert (
24+
updated_thread.messages[1]["content"] == "Hello"
25+
), "User message was unexpectedly modified."
26+
2127

2228
def test_replace_system_prompt_returns_same_object():
2329
# Setup: create a MessageThread with a single system message
@@ -29,4 +35,6 @@ def test_replace_system_prompt_returns_same_object():
2935
result = replace_system_prompt(msg_thread, new_prompt)
3036

3137
# Verify: the same MessageThread instance is returned (in-place modification)
32-
assert result is msg_thread, "replace_system_prompt should return the same MessageThread object."
38+
assert (
39+
result is msg_thread
40+
), "replace_system_prompt should return the same MessageThread object."

0 commit comments

Comments
 (0)