13 KiB
This work is licensed under the OpenAFC Project License, a copy of which is included with this software program.
Table of Contents
- Introduction
- Description
- Basic functionality
- Commands and procedures
- Tests execution options
- Add new test vectors
- Dry-run test vectors
- Dump test database
- Export AFC Server admin configuration
- Add AFC Server admin configuration
- Remove AFC Server admin configuration
- List AFC Server admin configuration
- Export tests from WFA excel file to test DB
- Compare AFC config records
- Reacquisition response records for exist requests
- How to run HTTPs access test
- Testing setup
- Change testing database manually
Introduction
This document describes the setup for testing environment to the openAFC github project. In addition, describes the execution procedure for the tests. Please contact support@telecominfraproject.com in case you need access to the openAFC project.
Description
The system consists of a testing utility that communicates with the AFC server. The utility sends WFA standard AFC requests and gets AFC responses. It also provides an analysis of the results. The utility also can be used to keep and export the AFC Server administration configuration used during validation. It consists of users, APs, and server configurations.
Basic functionality
Testing sequence
The sequence executes all required tests and returns fail if even one test fails. The testing procedure begins by preparing a test database with test vectors and responses. Follows run of test utility with server and test options (for details see #tests-execution-options).
afc_tests.py --addr <server address> --cmd run [--testcase_indexes <test index>] [--testcase_ids <test ids>]
Current implementation provides an already prepared database file (afc_input.sqlite3).
Set AFC Server configuration
There is a need to configure AFC Server prior testing. The configuration parameters include a list of APs, their users and AFC Server configurations per predefined user. Those parameters are stored in the test database and may be exported to an external JSON file to be used on the AFC server. The procedure of configuring the AFC server has 2 steps. To export parameters from the testing database into JSON file (#export-afc-server-admin-configuration) and to provide it to the local configuration utility (#add-afc-server-admin-configuration).
Test database
The testing database keeps all required AFC Server configuration data (admin configuration data) in SQL tables: user_config, ap_config, afc_config. All tables keep information as string without splitting JSON into fields which can be redesigned by request.
Commands and procedures
Tests execution options
Start sequential run of tests according to the data in the database.
afc_tests.py --addr 1.2.3.4 --cmd run
Run a test or number of tests according to test case row index.
afc_tests.py --addr 1.2.3.4 --cmd run --testcase_indexes 3,5,6,7
Run a test or number of tests according to test case ids.
afc_tests.py --addr 1.2.3.4 --cmd run --testcase_ids AFCS.IBP.5, AFCS.FSP.18
Run a test and save test result to csv format file.
afc_tests.py --addr 1.2.3.4 --outfile=filename.csv --cmd run --testcase_indexes 22
Start run all tests and save results as ‘golden reference’ in the database.
afc_tests.py --addr 1.2.3.4 --cmd add_reqs
Run the 1st test from test DB explicitly use HTTP.
afc_tests.py --addr 1.2.3.4 --prot http --port 80 --testcase_indexes 1 --cmd run
Run the 100 tests from test DB. If less exist reuse from the beggining of the DB. For example, if there are 11 testcases in DB and required to run 33, so the test app runs exist testcases 3 times.
afc_tests.py --addr 1.2.3.4 --cmd run --tests2run 1234
Add new test vectors
Add new test vectors to the database provide a file to the following command line.
afc_tests.py --addr=1.2.3.4 --cmd add_reqs --infile add_test_vector.txt
Provided file consists of any number of test vectors in the following format of AFC request.
{"availableSpectrumInquiryRequests": [{"inquiredChannels": [{"globalOperatingClass": 131}, {"globalOperatingClass": 132}, {"globalOperatingClass": 133}, {"globalOperatingClass": 134}], "deviceDescriptor": {"rulesetIds": ["US_47_CFR_PART_15_SUBPART_E"], "serialNumber": "Alpha001", "certificationId": [{"nra": "AlphaNRA01", "id": "AlphaID01"}]}, "vendorExtensions": [], "inquiredFrequencyRange": [{"lowFrequency": 5925, "highFrequency": 6425}, {"lowFrequency": 6525, "highFrequency": 6875}], "minDesiredPower": 18, "location": {"indoorDeployment": 2, "elevation": {"verticalUncertainty": 5, "heightType": "AGL", "height": 129}, "ellipse": {"orientation": 45, "minorAxis": 50, "center": {"latitude": 40.75924, "longitude": -73.97434}, "majorAxis": 100}}, "requestId": "1"}], "version": "1.1"}, {"testCase": "AFC.FSP.1”, "optonal":”optional”}
By default, a new test vector is sent to the AFC server in order to acquire it’s response and keep it in the database.
Dry-run test vectors
This is an option to run a test vector from a file without further saving it or it's response to the database file.
afc_tests.py --addr 1.2.3.4 --cmd dry_run --infile add_test_vector.txt
Dump test database
Release testing database binary together with SQLITE dump file. Steps to produce such dump file (dump.sql).
sqlite3 afc_input.sqlite3
>
> .output dump.sql
> .dump
> .quit
Show all entries from the database (requests and responses)
afc_tests.py --cmd dump_db
Export AFC Server admin configuration
Export from the testing database to text file in JSON format.
afc_tests.py --cmd exp_adm_cfg --outfile export_admin_cfg.json
Add AFC Server admin configuration
Read configurations from provided file and set AFC server
rat-manage-api cfg add src=./add_admin_cfg.json
Remove AFC Server admin configuration
Read configurations from provided file and remove them from AFC server
rat-manage-api cfg del src=./del_admin_cfg.json
List AFC Server admin configuration
List internal data from provided admin configuration file
rat-manage-api cfg list src=./add_admin_cfg.json
Export tests from WFA excel file to test DB
First to export test vectors from WFA excel file into JSON format file.
afc_tests.py --cmd parse_tests --infile <input file> --outfile <output file>
For example, export all WFA test vectors
afc_tests.py --cmd parse_tests --infile "AFC System (SUT) Test Vectors r6.xlsx" --outfile abc.txt --test_id all
Next step, to import those test vectors into test DB.
afc_tests.py --cmd ins_reqs --infile abc.txt
Add extended (enhanced tests) that are not wfa. For example:
afc_tests.py --cmd ext_reqs --infile brcm_ext.txt
Next step, to get new AFC responses for test vectors. It requires to make reacquisition. The commands sends every test vector, gets relevant response and inserts into test DB.
afc_tests.py --cmd reacq --addr <ip address> --port <number>
Following example exports test vectors and corresponded responses to WFA format files. All files created in local directory "wfa_test".
afc_tests.py --cmd dump_db --table wfa --outfile <output file>
An example how to use it with docker container, named "test".
Following sequence creates files at path "pwd/data/wfa_test".
docker run --rm -v `pwd`/data:/data test --cmd dump_db --table wfa --outpath /data
dump all test requests and responses, including extended ones :
docker run --rm -v `pwd`/data:/data test --cmd dump_db --table all --outpath /data
There is a way to export device descriptors from WFA excel file into JSON format file.
afc_tests.py --cmd parse_tests --dev_desc --infile <input file> --outfile <output file>
Next step, to import those device descriptors into test DB.
afc_tests.py --cmd ins_devs --infile abc.txt
Compare AFC config records
Compare provided AFC configuration with all/any records in the database.
afc_tests.py --cmd cmp_cfg --infile ../../afc_config.json
afc_tests.py --cmd cmp_cfg --infile ../../afc_config.json --idx 1
Reacquisition response records for exist requests
Re-run tests from the database and update corresponded responses.
afc_tests.py --cmd reacq --addr 1.2.3.4
How to run HTTPs access test
Follows an example on how to run an HTTPS access test. The test utility only does protocol handshake using mutual TLS.
cd <open-afc path>/tests
./afc_tests.py --cmd run --addr <ip addr> --port <port> --ca_cert tmp.bundle.pem
--cli_key tmp_cli.key.pem --cli_cert tmp_cli.bundle.pem
How to run specific tests with existing AFC servers using a docker image. It uses afc_tests.py utility internally in addition to script 'certs.sh' which creates client certificates at runtime.
docker run --rm <test image> --addr <ip addr> --port <port> --cmd run --testcase_ids AFCS.FSP.1
--prefix_cmd /usr/app/certs.sh cert_client --cli_cert /usr/app/test_cli/test_cli_crt.pem
--cli_key /usr/app/test_cli/test_cli_key.pem
How to build a docker image for testing from sources. As a result there is a docker image 'afc_test'.
cd <open-afc path>
docker build -f tests/Dockerfile -t afc_test .
How to send an email with test results
Current implementation provides ability to send an mail with tests results in attached file. By default it uses Google mail server "smtp.gmail.com", port 465. It is required to provide output file (option --outfile ) to collect tests results and to attach to the email.
cd <open-afc path>/tests
./afc_tests.py --cmd run --addr <ip addr> --port <port> --ca_cert tmp.bundle.pem
--cli_key tmp_cli.key.pem --cli_cert tmp_cli.bundle.pem ----email_from <sender email> --email_to <recipient email> --email_pwd <password> --outfile res.csv
Use of Google mail server required to make certain configurations to email account. One alternative to permit less secure application access by the following link. https://www.google.com/settings/security/lesssecureapps Another alternative to enable 2-step verification on a Google account and configure it App passwords.
Testing setup
The testing setup consists of utility and database. The utility provides options to send AFC request, to receive AFC response, analyze the response, acquire response data, export AFC server admin data to server admin configuration file. The admin server configuration file has data required for creation of server test users and server test APs. Each server user is provided with corresponding AFC server configuration. The AFC server admin configuration file is in JSON format by following structure:
{“afcAdminConfig”:
{“afcConfig: {...}},
“userConfig: { “username”: ”alpha”, “password”: ”123456”, “rolename”: [“AP”, “Admin”]},
“apConfig”: [
{ “serialNumber”: ”Alpha01”, “certificationId”: [{ “”nra”:”AlphaNRA02”, ”id”:”AlphaID01”}]},
{ “serialNumber”: ”Alpha02”, “certificationId”: [{ “”nra”:”AlphaNRA02”, ”id”:”AlphaID02”}]}
]
}
The AFC server admin configuration file provided to the AFC server utility (rat-manage-api) in order to configure the server.
There is also the ability to delete AFC server administration configuration by providing a JSON file of the following format.
{“afcAdminConfig”:
“userConfig: { “username”: ”alpha”},
“apConfig”: [
{ “serialNumber”: ”Alpha01”},
{ “serialNumber”: ”Alpha02”}
]
}
Change testing database manually
The procedure requires to drop the DB into a SQL file, edit if needed and create a new DB based on changed SQL file.
Open the DB and dump to a SQL file.
sqlite3 ./afc_input.sqlite3 .dump > dump.sql
Open and edit dump.sql with any editor as it is a text file with not complicated SQL code.
Make a backup from original DB.
mv ./afc_input.sqlite3 ./afc_input.sqlite3_backup
Open a new file and create tables in it.
sqlite3 ./afc_input.sqlite3
.read dump.sql
.quit
At this stage there is a fixed DB that can be used.
Happy usage!