From 13442f859f452578397766dedc7904928794610a Mon Sep 17 00:00:00 2001 From: Hassan Abouelela Date: Mon, 7 Jun 2021 05:28:07 +0300 Subject: Switches To Pytest As Test Runner Switches the test runner from unittest to pytest, to allow the usage of plugins such as xdist. This commit also adds pytest-cov purely as a generator for .coverage files. Signed-off-by: Hassan Abouelela --- tests/README.md | 6 +++++- 1 file changed, 5 insertions(+), 1 deletion(-) (limited to 'tests/README.md') diff --git a/tests/README.md b/tests/README.md index 1a17c09bd..a757f96c6 100644 --- a/tests/README.md +++ b/tests/README.md @@ -11,10 +11,14 @@ We are using the following modules and packages for our unit tests: - [unittest](https://docs.python.org/3/library/unittest.html) (standard library) - [unittest.mock](https://docs.python.org/3/library/unittest.mock.html) (standard library) - [coverage.py](https://coverage.readthedocs.io/en/stable/) +- [pytest-cov](https://pytest-cov.readthedocs.io/en/latest/index.html) + +We also use the following package as a test runner: +- [pytest](https://docs.pytest.org/en/6.2.x/) To ensure the results you obtain on your personal machine are comparable to those generated in the CI, please make sure to run your tests with the virtual environment defined by our [Poetry Project](/pyproject.toml). To run your tests with `poetry`, we've provided two "scripts" shortcuts: -- `poetry run task test` will run `unittest` with `coverage.py` +- `poetry run task test` will run `pytest` with `pytest-cov`. - `poetry run task test path/to/test.py` will run a specific test. - `poetry run task report` will generate a coverage report of the tests you've run with `poetry run task test`. If you append the `-m` flag to this command, the report will include the lines and branches not covered by tests in addition to the test coverage report. -- cgit v1.2.3 From 37584a8b8774c04b4111c29d96f8d06b31c89d84 Mon Sep 17 00:00:00 2001 From: Hassan Abouelela Date: Mon, 7 Jun 2021 05:58:57 +0300 Subject: Adds Fast-Test Task Signed-off-by: Hassan Abouelela --- pyproject.toml | 3 ++- tests/README.md | 1 + 2 files changed, 3 insertions(+), 1 deletion(-) (limited to 'tests/README.md') diff --git a/pyproject.toml b/pyproject.toml index 2c9181889..774fe075c 100644 --- a/pyproject.toml +++ b/pyproject.toml @@ -61,6 +61,7 @@ lint = "pre-commit run --all-files" precommit = "pre-commit install" build = "docker build -t ghcr.io/python-discord/bot:latest -f Dockerfile ." push = "docker push ghcr.io/python-discord/bot:latest" -test = "pytest -n auto --cov-report= --cov bot " +fast-test = "pytest -n auto" +test = "pytest -n auto --cov-report= --cov bot" html = "coverage html" report = "coverage report" diff --git a/tests/README.md b/tests/README.md index a757f96c6..b5fba9611 100644 --- a/tests/README.md +++ b/tests/README.md @@ -18,6 +18,7 @@ We also use the following package as a test runner: To ensure the results you obtain on your personal machine are comparable to those generated in the CI, please make sure to run your tests with the virtual environment defined by our [Poetry Project](/pyproject.toml). To run your tests with `poetry`, we've provided two "scripts" shortcuts: +- `poetry run task fast-test` will run `pytest`. - `poetry run task test` will run `pytest` with `pytest-cov`. - `poetry run task test path/to/test.py` will run a specific test. - `poetry run task report` will generate a coverage report of the tests you've run with `poetry run task test`. If you append the `-m` flag to this command, the report will include the lines and branches not covered by tests in addition to the test coverage report. -- cgit v1.2.3 From e45843ebe0a199d30b09ec7a0dcffc1ed9d4d9d7 Mon Sep 17 00:00:00 2001 From: Hassan Abouelela Date: Mon, 7 Jun 2021 22:33:15 +0300 Subject: Fix Script Count In Documentation Signed-off-by: Hassan Abouelela --- tests/README.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) (limited to 'tests/README.md') diff --git a/tests/README.md b/tests/README.md index b5fba9611..339108951 100644 --- a/tests/README.md +++ b/tests/README.md @@ -16,7 +16,7 @@ We are using the following modules and packages for our unit tests: We also use the following package as a test runner: - [pytest](https://docs.pytest.org/en/6.2.x/) -To ensure the results you obtain on your personal machine are comparable to those generated in the CI, please make sure to run your tests with the virtual environment defined by our [Poetry Project](/pyproject.toml). To run your tests with `poetry`, we've provided two "scripts" shortcuts: +To ensure the results you obtain on your personal machine are comparable to those generated in the CI, please make sure to run your tests with the virtual environment defined by our [Poetry Project](/pyproject.toml). To run your tests with `poetry`, we've provided the following "script" shortcuts: - `poetry run task fast-test` will run `pytest`. - `poetry run task test` will run `pytest` with `pytest-cov`. -- cgit v1.2.3 From 43659c7bac2e8127df402c97bbc3a26edf8256b6 Mon Sep 17 00:00:00 2001 From: Hassan Abouelela Date: Sat, 12 Jun 2021 21:12:31 +0300 Subject: Renamed Test Task In Documentation Signed-off-by: Hassan Abouelela --- tests/README.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) (limited to 'tests/README.md') diff --git a/tests/README.md b/tests/README.md index 339108951..0192f916e 100644 --- a/tests/README.md +++ b/tests/README.md @@ -18,7 +18,7 @@ We also use the following package as a test runner: To ensure the results you obtain on your personal machine are comparable to those generated in the CI, please make sure to run your tests with the virtual environment defined by our [Poetry Project](/pyproject.toml). To run your tests with `poetry`, we've provided the following "script" shortcuts: -- `poetry run task fast-test` will run `pytest`. +- `poetry run task test-nocov` will run `pytest`. - `poetry run task test` will run `pytest` with `pytest-cov`. - `poetry run task test path/to/test.py` will run a specific test. - `poetry run task report` will generate a coverage report of the tests you've run with `poetry run task test`. If you append the `-m` flag to this command, the report will include the lines and branches not covered by tests in addition to the test coverage report. -- cgit v1.2.3 From 089ef6219d98cbfcf9eed8bced6cb3ca43f0c55b Mon Sep 17 00:00:00 2001 From: Hassan Abouelela Date: Tue, 6 Jul 2021 02:33:57 +0300 Subject: Adds Documentation For Running A Single Test (#1669) Adds a portion to the testing README explaining how and when to run an individual test file when working with tests. Additionally adds a table of contents as the document has become quite long. Signed-off-by: Hassan Abouelela --- tests/README.md | 31 +++++++++++++++++++++++++++++++ 1 file changed, 31 insertions(+) (limited to 'tests/README.md') diff --git a/tests/README.md b/tests/README.md index 0192f916e..b7fddfaa2 100644 --- a/tests/README.md +++ b/tests/README.md @@ -4,6 +4,14 @@ Our bot is one of the most important tools we have for running our community. As _**Note:** This is a practical guide to getting started with writing tests for our bot, not a general introduction to writing unit tests in Python. If you're looking for a more general introduction, you can take a look at the [Additional resources](#additional-resources) section at the bottom of this page._ +### Table of contents: +- [Tools](#tools) +- [Running tests](#running-tests) +- [Writing tests](#writing-tests) +- [Mocking](#mocking) +- [Some considerations](#some-considerations) +- [Additional resources](#additional-resources) + ## Tools We are using the following modules and packages for our unit tests: @@ -25,6 +33,29 @@ To ensure the results you obtain on your personal machine are comparable to thos If you want a coverage report, make sure to run the tests with `poetry run task test` *first*. +## Running tests +There are multiple ways to run the tests, which one you use will be determined by your goal, and stage in development. + +When actively developing, you'll most likely be working on one portion of the codebase, and as a result, won't need to run the entire test suite. +To run just one file, and save time, you can use the following command: +```shell +poetry run task test-nocov +``` + +For example: +```shell +poetry run task test-nocov tests/bot/exts/test_cogs.py +``` +will run the test suite in the `test_cogs` file. + +If you'd like to collect coverage as well, you can append `--cov` to the command above. + + +If you're done and are preparing to commit and push your code, it's a good idea to run the entire test suite as a sanity check: +```shell +poetry run task test +``` + ## Writing tests Since consistency is an important consideration for collaborative projects, we have written some guidelines on writing tests for the bot. In addition to these guidelines, it's a good idea to look at the existing code base for examples (e.g., [`test_converters.py`](/tests/bot/test_converters.py)). -- cgit v1.2.3