mirror of
https://github.com/Telecominfraproject/wlan-testing.git
synced 2025-12-26 15:37:13 +00:00
8f3d94ff5e
* Updated Interop testing markers in update_badges * Changed Regression to Performance in Interop * Changed Regression to Performance in update_badges * corrected the performance marker
166 lines
7.5 KiB
Markdown
166 lines
7.5 KiB
Markdown
## TIP Open WiFi Test Automation
|
|
|
|
### Overview
|
|
|
|
This repository contains the test automation framework and scripts for TIP Open WiFi.
|
|
Test Framework is implemented in pytest
|
|
|
|
|
|
|
|
The vision is to create an open test framework to ensure production grade Open Wifi Solution.
|
|
|
|
This test automation tied into our overall tooling at TIP as shown below:
|
|
|
|
|
|
|
|
### General guidelines
|
|
|
|
This testing code adheres to generic [pep8](https://www.python.org/dev/peps/pep-0008/#introduction) style guidelines, most notably:
|
|
|
|
1. [Documentation strings](https://www.python.org/dev/peps/pep-0008/#documentation-strings)
|
|
2. [Naming conventions](https://www.python.org/dev/peps/pep-0008/#prescriptive-naming-conventions)
|
|
3. [Sphynx docstring format](https://sphinx-rtd-tutorial.readthedocs.io/en/latest/docstrings.html)
|
|
|
|
Using [pytest](https://docs.pytest.org/en/6.2.x/) as the test execution framework.
|
|
|
|
Using [pylint](http://pylint.pycqa.org) for code quality monitoring.
|
|
|
|
We are using the `pylint` package to do the linting. Documentation for it can be found [here](http://pylint.pycqa.org/en/latest/).
|
|
In general, the customizations are possible via the `.pylintrc` file:
|
|
|
|
1. Line length below 120 characters is fine (search for max-line-length)
|
|
2. No new line at the end of file is fine (search for missing-final-newline)
|
|
3. Multiple new lines at the end of file are fine (search for trailing-newlines)
|
|
4. Indent using 4 spaces (search for indent-string)
|
|
|
|
In future we should enforce a policy, where we cannot merge a code where the pylint scoe goes below 7:
|
|
|
|
```shell
|
|
pylint --fail-under=7 *py
|
|
```
|
|
|
|
the command above would produce a non-zero exit code if the score drops below 7.
|
|
|
|
### TIP Controller Integration
|
|
|
|
Using Swagger Autogenerated CloudSDK Library pypi package (implemented with [swagger codegen](https://github.com/swagger-api/swagger-codegen)).
|
|
|
|
### Reporting
|
|
|
|
Currently we use the [pytest allure integration](https://docs.qameta.io/allure/#_pytest) to create visual reports for the test outcomes.
|
|
|
|
Additionally we generate overview pages, e.g. like [this one](http://openwifi-allure-reports.s3-website-us-east-1.amazonaws.com/sanity/overview/) to consolidate multiple test reports of the same test type.
|
|
|
|
These overview pages are being deployed by [this workflow](.github/workflows/update-overviews.yml) and their configuration is set in [this file](./.allure-overview/overviews.json).
|
|
|
|
### Miscellaneous
|
|
|
|
1. Do not use old style string formatting: `"Hello %s" % var`; use `f"Hello {var}` instead
|
|
2. use `"""` in Docstrings
|
|
|
|
### Useful links
|
|
|
|
https://docs.pytest.org/en/latest/example/markers.html
|
|
https://docs.pytest.org/en/latest/usage.html
|
|
http://pythontesting.net/framework/pytest/pytest-introduction/
|
|
|
|
### Test status
|
|
|
|
|
|
|
|
[](https://github.com/Telecominfraproject/wlan-testing/actions/workflows/quali.yml)
|
|
|
|
|
|
[](https://github.com/Telecominfraproject/wlan-testing/actions/workflows/interop.yml)
|
|
|
|
|
|
[](https://github.com/Telecominfraproject/wlan-testing/actions/workflows/performance.yml)
|
|
|
|
|
|
[](https://github.com/Telecominfraproject/wlan-testing/actions/workflows/quali-advanced.yml)
|
|
|
|
|
|
[](https://github.com/Telecominfraproject/wlan-testing/actions/workflows/regression.yml)
|
|
|
|
|
|
[](https://github.com/Telecominfraproject/wlan-testing/actions/workflows/interop_performance.yml)
|
|
|
|
|
|
### Best Practice
|
|
|
|
1. Ensure to have a python version > 3
|
|
2. We strongly recommend to use virtualenv to ensure that your dev environment sandbox is created.
|
|
|
|
### Code style
|
|
|
|
All code must be written in python 3 and conform to PEP 8 style guide. The test framework is built using pytest.
|
|
|
|
[](https://www.python.org/)
|
|
[](https://www.python.org/dev/peps/pep-0008/)
|
|
|
|
### Directory Structure
|
|
|
|
```bash
|
|
├── lanforge - /* to be migrated */
|
|
├── libs
|
|
│ ├── controller_tests -/* Library Support for controller_tests part */
|
|
├── apnos -/* Library Support for Access Points (uses AP SSH) */
|
|
│ ├── lanforge -/* Library Support for LanForge Traffic Generator */
|
|
│ ├── perfecto -/* Library Support for Perfecto Systems */
|
|
│ ├── testrails -/* Result Visualization (will be depreciated ) */
|
|
├── tests -/* Pytest cases Directory */
|
|
│ ├── _basic_test_setup
|
|
│ ├── access_point_tests
|
|
│ ├── controller_tests
|
|
│ ├── e2e
|
|
├── advanced
|
|
├── basic
|
|
├── interOp
|
|
├── mdu
|
|
|── mesh
|
|
|── scale
|
|
|── README.md -/* Pytest framework and testcases information */
|
|
```
|
|
|
|
# Test Automation Setup Instructions
|
|
```
|
|
# Let's assume '~/automation' directory for setting wlan-testing
|
|
cd ~/
|
|
mkdir automation
|
|
cd automation
|
|
git clone https://github.com/Telecominfraproject/wlan-testing
|
|
git clone https://github.com/Telecominfraproject/wlan-lanforge-scripts
|
|
cd wlan-testing
|
|
./sync_repos.bash
|
|
mkdir ~/.pip
|
|
echo "[global]" > ~/.pip/pip.conf
|
|
echo "index-url = https://pypi.org/simple" >> ~/.pip/pip.conf
|
|
echo "extra-index-url = https://tip-read:tip-read@tip.jfrog.io/artifactory/api/pypi/tip-wlan-python-pypi-local/simple" >> ~/.pip/pip.conf
|
|
pip3 install -r requirements.txt
|
|
|
|
cd tests
|
|
vim configuration.py # Setup your testbed data info here
|
|
|
|
# Check if everything is good
|
|
pytest --collect-only
|
|
|
|
|
|
# for visualizing allure reports, Install nodejs
|
|
# Set Java home
|
|
|
|
# install allure command
|
|
npm install -g allure-commandline --save-dev
|
|
|
|
```
|
|
|
|
|
|
### Executing and Contributing new test cases
|
|
|
|
Follow instructions provided [here](./tests/README.md) to understand the Testing Framework, and How to run and
|
|
write new tests
|
|
|
|
### Issue Reporting
|
|
Please submit issues using our JIRA project [here](https://telecominfraproject.atlassian.net/browse/WIFI)
|
|
|
|
Use the Test Automation [backlog](https://telecominfraproject.atlassian.net/secure/RapidBoard.jspa?projectKey=WIFI&rapidView=48&view=planning.nodetail)
|