Commit Graph

245 Commits

Author SHA1 Message Date
Matthew Stidham
f4ecff6bb4 regression-test: Always produce an HTML report
Signed-off-by: Matthew Stidham <stidmatt@gmail.com>
2022-07-12 15:45:45 +05:30
Matthew Stidham
56f8c0a820 regression_test: don't quote the command when not in an eval command
add test_l3 to regression_test

Signed-off-by: Matthew Stidham <stidmatt@gmail.com>
2022-07-12 15:45:45 +05:30
Matthew Stidham
ae78aee979 regression_test: Fix scenario.py method call
Signed-off-by: Matthew Stidham <stidmatt@gmail.com>
2022-07-12 15:45:45 +05:30
Ben Greear
615dd1c9c3 regression-test: check test script return code to detect failure.
Also:  merge stderr/stdout, add more specific help example,
allow configuring report dir, add safety check for missing 'jq'.

Signed-off-by: Ben Greear <greearb@candelatech.com>
2022-07-12 15:45:45 +05:30
Matthew Stidham
ad845bafa3 regression_test: If the test is a success, reset LOGGING to a blank string
Signed-off-by: Matthew Stidham <stidmatt@gmail.com>
2022-07-12 15:45:33 +05:30
Matthew Stidham
ac9d722a18 regression_test: add back logging recording logic
Signed-off-by: Matthew Stidham <stidmatt@gmail.com>
2022-07-12 15:45:33 +05:30
Matthew Stidham
38724f7daa regression_test: Check if user is using anaconda, if so, install lanforge-scripts appropriately
if the user is not using any virtual environment, install from pip using --user flag

Signed-off-by: Matthew Stidham <stidmatt@gmail.com>
2022-07-12 15:45:33 +05:30
Matthew Stidham
ea9bad3ce6 regression_test: append to filename.txt, don't overwrite data from scenario.py
Signed-off-by: Matthew Stidham <stidmatt@gmail.com>
2022-07-12 15:45:33 +05:30
Matthew Stidham
63211997a9 regression_test: Use correct arguments for lf_dataplane_test
Signed-off-by: Matthew Stidham <stidmatt@gmail.com>
2022-07-12 15:45:33 +05:30
Matthew Stidham
4c1ab7edbc regression_test: Fix mamin logic of regression-test which was broken. Restore it back to where it was and add in the new features
Signed-off-by: Matthew Stidham <stidmatt@gmail.com>
2022-07-12 15:45:33 +05:30
Matthew Stidham
a6241dc08c regression_test: Store report_data in report_data
Signed-off-by: Matthew Stidham <stidmatt@gmail.com>
2022-07-12 15:45:33 +05:30
Matthew Stidham
c89b3930e0 regression_test: fix logic, fix arguments
Signed-off-by: Matthew Stidham <stidmatt@gmail.com>
2022-07-12 15:45:33 +05:30
Matthew Stidham
91259d1474 regression_test: Save report-data to report-data.
Signed-off-by: Matthew Stidham <stidmatt@gmail.com>
2022-07-12 15:45:33 +05:30
Matthew Stidham
ffacc6dde3 regression_test: ADd debug to three commands
Signed-off-by: Matthew Stidham <stidmatt@gmail.com>
2022-07-12 15:45:33 +05:30
Matthew Stidham
9c5223637f Revert "regression_test: Add debug to three more commands": Broke debugging
This reverts commit da8109a14b295b83c6e28af5de9eeef9acb9b8ea.
2022-07-12 15:45:33 +05:30
Matthew Stidham
27f24bf2bd regression_test: Add debug to three more commands
Signed-off-by: Matthew Stidham <stidmatt@gmail.com>
2022-07-12 15:45:33 +05:30
Matthew Stidham
24f7a64092 regression-test: if exit on error is 0, run the test
Signed-off-by: Matthew Stidham <stidmatt@gmail.com>
2022-07-12 15:45:33 +05:30
Matthew Stidham
0563d0bb46 regression_test: bash varibles need to be in curly braces
Signed-off-by: Matthew Stidham <stidmatt@gmail.com>
2022-07-12 15:45:33 +05:30
Matthew Stidham
739b12d8e1 regression_test: Add exit on error flag
Signed-off-by: Matthew Stidham <stidmatt@gmail.com>
2022-07-12 15:45:33 +05:30
Matthew Stidham
c7ae54338e regression_test: Name log from gui log as gui_log.txt
Signed-off-by: Matthew Stidham <stidmatt@gmail.com>
2022-07-12 15:45:33 +05:30
Matthew Stidham
530551dd48 regression_test: Save LANforge logs to logging directory
Signed-off-by: Matthew Stidham <stidmatt@gmail.com>
2022-07-12 15:45:33 +05:30
Matthew Stidham
6c8e2dd9cb regression_test: Append scenario.py output to STDOUT file
Signed-off-by: Matthew Stidham <stidmatt@gmail.com>
2022-07-12 15:45:33 +05:30
Matthew Stidham
aa5b4b14ef regression_test: log_filter isn't ready for production
Signed-off-by: Matthew Stidham <stidmatt@gmail.com>
2022-07-12 15:45:33 +05:30
Matthew Stidham
0ac894d599 regression_test: Grab run_client and run_mgr log files
Signed-off-by: Matthew Stidham <stidmatt@gmail.com>
2022-07-12 15:45:33 +05:30
Matthew Stidham
3467b5c13d regression_test: Cleanup log files so we only have the text which is relevant to the test which just ran
Signed-off-by: Matthew Stidham <stidmatt@gmail.com>
2022-07-12 15:45:33 +05:30
Matthew Stidham
d8fa486d13 regression_test: run in virtual environment if virtual environment exists
Signed-off-by: Matthew Stidham <stidmatt@gmail.com>
2022-07-12 15:45:33 +05:30
Matthew Stidham
381ae85172 regression_test: Make a simpler call for the beginning of function test() when we call scenario.py
Signed-off-by: Matthew Stidham <stidmatt@gmail.com>
2022-07-12 15:45:33 +05:30
Matthew Stidham
f5f29ea9ee regression_test: fix using = in python argparse argument error
Signed-off-by: Matthew Stidham <stidmatt@gmail.com>
2022-07-12 15:45:33 +05:30
Matthew Stidham
58a6dd07cb regression_test: simplify start_tests and add logic for checking ports in scenario.py
Signed-off-by: Matthew Stidham <stidmatt@gmail.com>
2022-07-12 15:45:33 +05:30
Matthew Stidham
d667912bca regression_test: Fix links to logging directories
Signed-off-by: Matthew Stidham <stidmatt@gmail.com>
2022-07-12 15:45:33 +05:30
Matthew Stidham
164a58a723 regression_test: we only need to mkdir log_dir/name once, so I removed it on line 548
Signed-off-by: Matthew Stidham <stidmatt@gmail.com>
2022-07-12 15:45:33 +05:30
Matthew Stidham
bc608b913c regression-test: logging links need to point to sub directory
Signed-off-by: Matthew Stidham <stidmatt@gmail.com>
2022-07-12 15:45:33 +05:30
Matthew Stidham
f26c994880 regression: tail needs a lowercase n, not an uppercase n to grab the last n lines
Signed-off-by: Matthew Stidham <stidmatt@gmail.com>
2022-07-12 15:45:33 +05:30
Matthew Stidham
f12a85fc9b regression_test: Simplify logic to store logging information by writing it only once
Signed-off-by: Matthew Stidham <stidmatt@gmail.com>
2022-07-12 15:45:33 +05:30
Matthew Stidham
a2931eea9e regression_test: Record log information about failed tests to LANforge device, search for Starting Scenario string which is automatically logged by scenario.py
Signed-off-by: Matthew Stidham <stidmatt@gmail.com>
2022-07-12 15:45:33 +05:30
Matthew Stidham
9dc92b3763 regression_test: Fix regression test logs address logic
Signed-off-by: Matthew Stidham <stidmatt@gmail.com>
2022-07-12 15:45:33 +05:30
Matthew Stidham
3a9357142b regression_test: else statements don't require the word then, fix my bash mistake
Signed-off-by: Matthew Stidham <stidmatt@gmail.com>
2022-07-12 15:45:33 +05:30
Matthew Stidham
7076c08a3e regression_test: save error logs to folder if a test fails and link to that folder in the regression report
Signed-off-by: Matthew Stidham <stidmatt@gmail.com>
2022-07-12 15:45:33 +05:30
Matthew Stidham
19dc262e51 regression_test: Add VAP_SSID argument for tests which create a VAP
Signed-off-by: Matthew Stidham <stidmatt@gmail.com>
2022-07-12 15:45:33 +05:30
Jed Reynolds
8c8b5279b4 regression_test.sh: adds meta tag to discourage browser caching
Signed-off-by: Jed Reynolds <jed@bitratchet.com>
2022-07-12 15:45:32 +05:30
Matthew Stidham
b5fcfa133e regression_test: test_l3_unicast_traffic_gen: update arguments
Signed-off-by: Matthew Stidham <stidmatt@gmail.com>
2022-07-12 15:45:32 +05:30
Matthew Stidham
c900dc06b6 regression_test: add test_l3_unicast_traffic_gen to regression_test
Signed-off-by: Matthew Stidham <stidmatt@gmail.com>
2022-07-12 15:45:32 +05:30
Matthew Stidham
47c49b57e5 regression_test: Run git pull --rebase at the beginning of the script, as a safety check
Signed-off-by: Matthew Stidham <stidmatt@gmail.com>
2022-07-12 15:45:24 +05:30
Matthew Stidham
a77b12e964 regression_test: Use BSSID variable wherever applicable in script
Signed-off-by: Matthew Stidham <stidmatt@gmail.com>
2022-07-12 15:45:17 +05:30
Matthew Stidham
f5e87fb594 regression_test: Always use the mgr flag when runnin scenario.py
Signed-off-by: Matthew Stidham <stidmatt@gmail.com>
2022-07-12 15:45:17 +05:30
Matthew Stidham
9851cee303 regression_text: add in end text function regression commands file
Signed-off-by: Matthew Stidham <stidmatt@gmail.com>
2022-07-12 15:44:27 +05:30
Matthew Stidham
5edff0b54c regression_test: Don't test ipv6 and ttls for now
Signed-off-by: Matthew Stidham <stidmatt@gmail.com>
2022-07-12 15:44:27 +05:30
Matthew Stidham
798e4d81c8 regression_test: Ensure pip is up to date before updating scripts
Signed-off-by: Matthew Stidham <stidmatt@gmail.com>
2022-07-12 15:44:27 +05:30
Matthew Stidham
9354c02bd7 regression_test: Use mgr flag in sta_scan_test and measure_station_time_up
Signed-off-by: Matthew Stidham <stidmatt@gmail.com>
2022-07-12 15:44:27 +05:30
Matthew Stidham
19678db8ce regression_test: sta_connect needs a manual BSSID
Signed-off-by: Matthew Stidham <stidmatt@gmail.com>
2022-07-12 15:44:26 +05:30