diff --git a/README.md b/README.md index 7eb77022e..d9100cc00 100644 --- a/README.md +++ b/README.md @@ -1,17 +1,93 @@ -## TIP Open WiFi Testing +## TIP Open WiFi Test Automation + +### Overview + This repository contains the test automation framework and scripts for TIP Open WiFi. Test Framework is implemented in pytest -## Test Cases are Classified as follows +![Automation Overview](./automation_overview.png) + +The vision is to create an open test framework to ensure production grade Open Wifi Solution. + +This test automation tied into our overall tooling at TIP as shown below: + +![Tools and Test Automation](./Tools_and_Testing.png) + +### General guidelines + +This testing code adheres to generic [pep8](https://www.python.org/dev/peps/pep-0008/#introduction) style guidelines, most notably: + +1. [Documentation strings](https://www.python.org/dev/peps/pep-0008/#documentation-strings) +2. [Naming conventions](https://www.python.org/dev/peps/pep-0008/#prescriptive-naming-conventions) +3. [Sphynx docstring format](https://sphinx-rtd-tutorial.readthedocs.io/en/latest/docstrings.html) + +Using [pytest](https://docs.pytest.org/en/6.2.x/) as the test execution framework. + +Using [pylint](http://pylint.pycqa.org) for code quality monitoring. + +We are using the `pylint` package to do the linting. Documentation for it can be found [here](http://pylint.pycqa.org/en/latest/). +In general, the customizations are possible via the `.pylintrc` file: + +1. Line length below 120 characters is fine (search for max-line-length) +2. No new line at the end of file is fine (search for missing-final-newline) +3. Multiple new lines at the end of file are fine (search for trailing-newlines) +4. Indent using 4 spaces (search for indent-string) + +In future we should enforce a policy, where we cannot merge a code where the pylint scoe goes below 7: + +```shell +pylint --fail-under=7 *py +``` + +the command above would produce a non-zero exit code if the score drops below 7. + + +### TIP Controller Integration + +Using Swagger Autogenerated CloudSDK Library pypi package (implemented with [swagger codegen](https://github.com/swagger-api/swagger-codegen)). + +### Reporting + +Currently the plan is to use pytest integrated with [allure](https://docs.qameta.io/allure/#_pytest) to create visual reports for the test outcomes + +### Miscelanneous + +1. Do not use old style string formatting: `"Hello %s" % var`; use `f"Hello {var}` instead +2. use `"""` in Docstrings + +### Useful links + +https://docs.pytest.org/en/latest/example/markers.html +https://docs.pytest.org/en/latest/usage.html +http://pythontesting.net/framework/pytest/pytest-introduction/ + +### Build status + +[![Build Status](https://github.com/Telecominfraproject/wlan-testing/workflows/nightly%20build/badge.svg)] + +### Best Practice + +1. Ensure to have a python version > 3 +2. We strongly recommend to use virtualenv to ensure that your dev environment sandbox is created. + +### Code style + +All code must be written in python 3 and conform to PEP 8 style guide. The test framework is built using pytest. + +[![made-with-python](https://img.shields.io/badge/Made%20with-Python-1f425f.svg)](https://www.python.org/) +[![PEP8](https://img.shields.io/badge/code%20style-pep8-orange.svg)](https://www.python.org/dev/peps/pep-0008/) + + +### Directory Structure ```bash -├── lanforge - /* to be migrated */ +├── lanforge - /* to be migrated */ ├── libs │ ├── controller_tests -/* Library Support for controller_tests part */ - ├── apnos -/* Library Support for Access Points (uses AP SSH) */ -│ ├── lanforge -│ ├── perfecto -/* Library Support for Perfecto Traffic Generator*/ -│ ├── testrails -/* Result Visualization (will be depreciated )*/ -├── tests - /* Pytest cases Directory */ + ├── apnos -/* Library Support for Access Points (uses AP SSH) */ +│ ├── lanforge -/* Library Support for LanForge Traffic Generator */ +│ ├── perfecto -/* Library Support for Perfecto Systems */ +│ ├── testrails -/* Result Visualization (will be depreciated ) */ +├── tests -/* Pytest cases Directory */ │ ├── _basic_test_setup │ ├── access_point_tests │ ├── controller_tests @@ -22,13 +98,17 @@ Test Framework is implemented in pytest ├── mdu |── mesh |── scale - |── README.md - /* Pytest framework and testcases information */ + |── README.md -/* Pytest framework and testcases information */ ``` -## Follow the below instructions to setup the Testing and Development Environment +### Setup Instructions -```bash -STEP 1 +#### Step 1 +Set up the repository in your base directory of choice. Currently, it is +important to clone the wlan-lanforge-scripts repository and link to wlan-testing +directory (The sync_repos.bash provides that linkage). + +```shell git clone https://github.com/Telecominfraproject/wlan-testing git clone https://github.com/Telecominfraproject/wlan-lanforge-scripts @@ -36,14 +116,16 @@ Make sure this directory structure exists ├── wlan-lanforge-scripts ├── wlan-testing - cd wlan-testing ./sync_repos.bash +``` -STEP 2 +#### Step 2 -Make sure you have python version >= 3.8 +Please ensure you have python version >= 3 is required. Double check this by running "python --version" command. + +```shell User this to check "python --version" Lets Install the Python Packages needed @@ -52,10 +134,12 @@ pip3 install pytest==6.2.2 pip3 install bs4 pip3 install paramiko pip3 install xlsxwriter +``` +#### Step 3 +Please ensure you follow the steps outlined in [here](./libs/README.md) -STEP 3 - +```shell Lets Install Controller Libraries, follow below steps: mkdir ~/.pip @@ -64,14 +148,15 @@ echo "index-url = https://pypi.org/simple" >> ~/.pip/pip.conf echo "extra-index-url = https://tip-read:tip-read@tip.jfrog.io/artifactory/api/pypi/tip-wlan-python-pypi-local/simple" >> ~/.pip/pip.conf echo "tip-wlan-cloud" > ~/.pip/requirements.txt pip3 install -r ~/.pip/requirements.txt +``` -STEP 4 +#### Step 4 Setup configuration.py file This file is autogenerated by Quali-Lab-Orchestration You can custom create the configuration file, by following the sample configuration data structure (tests/configuration.py) -STEP 5 +#### Step 5 You are all set, @@ -80,27 +165,12 @@ To verify that everything is working, cd wlan-testing/tests pytest -m test_connection -s -vvv +### Executing and Contributing new test cases -Now, follow tests/README.md to understand the Testing Framework, and How to run and +Follow instructions provided [here](./tests/README.md) to understand the Testing Framework, and How to run and write new tests -Incase of any Issue regarding setup, -email: shivam.thakur@candelatech.com - -``` - - -## Build status - -[![Build Status](https://github.com/Telecominfraproject/wlan-testing/workflows/nightly%20build/badge.svg)] - -## Best Practice -1. Ensure to have a python version > 3.8 -2. We strongly recommend to use virtualenv to ensure that your dev environment sandbox is created. - -## Code style -All code must be written in python 3 and conform to PEP 8 style guide. The test framework is built using pytest. - -[![made-with-python](https://img.shields.io/badge/Made%20with-Python-1f425f.svg)](https://www.python.org/) -[![PEP8](https://img.shields.io/badge/code%20style-pep8-orange.svg)](https://www.python.org/dev/peps/pep-0008/) +### Issue Reporting +Please submit issues using our JIRA project [here](https://telecominfraproject.atlassian.net/browse/WIFI) +Use the Test Automation [backlog](https://telecominfraproject.atlassian.net/secure/RapidBoard.jspa?projectKey=WIFI&rapidView=48&view=planning.nodetail) diff --git a/Tools_and_Testing.png b/Tools_and_Testing.png new file mode 100644 index 000000000..29b2ac701 Binary files /dev/null and b/Tools_and_Testing.png differ diff --git a/automation_overview.png b/automation_overview.png new file mode 100644 index 000000000..bc773d5f2 Binary files /dev/null and b/automation_overview.png differ diff --git a/libs/controller/README.md b/libs/controller/README.md index 848b4161c..abcade46a 100644 --- a/libs/controller/README.md +++ b/libs/controller/README.md @@ -56,45 +56,4 @@ and you can use something like this to develop your code: ```shell docker run -it -v %path_to_this_dir%/old_pytest wlan-tip-test -``` - -### General guidelines - -This testing code adheres to generic [pep8](https://www.python.org/dev/peps/pep-0008/#introduction) style guidelines, most notably: - -1. [Documentation strings](https://www.python.org/dev/peps/pep-0008/#documentation-strings) -2. [Naming conventions](https://www.python.org/dev/peps/pep-0008/#prescriptive-naming-conventions) -3. [Sphynx docstring format](https://sphinx-rtd-tutorial.readthedocs.io/en/latest/docstrings.html) - -We are using the `pylint` package to do the linting. Documentation for it can be found [here](http://pylint.pycqa.org/en/latest/). -In general, the customizations are possible via the `.pylintrc` file: - -1. Line length below 120 characters is fine (search for max-line-length) -2. No new line at the end of file is fine (search for missing-final-newline) -3. Multiple new lines at the end of file are fine (search for trailing-newlines) -4. Indent using 4 spaces (search for indent-string) -5. todo - -In future we should enforce a policy, where we cannot merge a code where the pylint scoe goes below 7: - -```shell -pylint --fail-under=7 *py -``` - -the command above would produce a non-zero exit code if the score drops below 7. - -### Reporting - -Currently the plan is to use pytest integrated with [allure](https://docs.qameta.io/allure/#_pytest) to create visual reports for the test outcomes - -### Miscelanneous - -1. Do not use old style string formatting: `"Hello %s" % var`; use `f"Hello {var}` instead -2. use `"""` in Docstrings -3. todo - -### Useful links - -https://docs.pytest.org/en/latest/example/markers.html -https://docs.pytest.org/en/latest/usage.html -http://pythontesting.net/framework/pytest/pytest-introduction/ \ No newline at end of file +``` \ No newline at end of file