Commit Graph

251 Commits

Author SHA1 Message Date
Ben Greear
ec2dd1b4b3 regression-test: Improve lf_dataplane_test
Use DUT that earlier test logic created, allow specifying testbed
name on the cmd line.

Add exit code for dpt, but not sure it is correct since currently
it fails trying to copy the results locally since my GUI is on localhost
and LANforge system is elsewhere.  Added TODO to allow this usage.

Signed-off-by: Ben Greear <greearb@candelatech.com>
2022-07-12 15:45:46 +05:30
Ben Greear
169203b57b regression: Fix ./test_l3.py line
It was missing a closing quote, so it didn't execute at all.  I've no
idea why it shows success in the html output.
2022-07-12 15:45:46 +05:30
Ben Greear
9287aff711 create-macvlan: mac-vlans have nothing to do with radios.
So remove reference to radios from it.

Signed-off-by: Ben Greear <greearb@candelatech.com>
2022-07-12 15:45:46 +05:30
Ben Greear
7db4c9a329 create-cv: Improve --line and --ssid parser.
Use shlex to parse the tokens for each --line and --ssid content.
This fixes using ssid with spaces, for instance.
Fix parser to error out if user inputs invalid data.

Add TODO items for future work.

Fix regression script:  DUT must be created before Scenario tries to use it,
and fix typos and other mistakes in how the regression script tried to
create dut and chamberview.  This part has not been tested yet, but it is
at least closer than before.

Signed-off-by: Ben Greear <greearb@candelatech.com>
2022-07-12 15:45:45 +05:30
Ben Greear
b774389d54 remove create_bond_and_station logic.
It makes no sense at all to pair a station and a bridge.
Remove the combined test in regression script.

Add option to not clean up bridge and bond interfaces.
Improve bridge and bond to handle multiple resources
(hopefully, that part is not tested)

Signed-off-by: Ben Greear <greearb@candelatech.com>
2022-07-12 15:45:45 +05:30
Matthew Stidham
e824080c24 regression_test: Use eval so that commands with quotes are properly executed
Signed-off-by: Matthew Stidham <stidmatt@gmail.com>
2022-07-12 15:45:45 +05:30
Matthew Stidham
f4ecff6bb4 regression-test: Always produce an HTML report
Signed-off-by: Matthew Stidham <stidmatt@gmail.com>
2022-07-12 15:45:45 +05:30
Matthew Stidham
56f8c0a820 regression_test: don't quote the command when not in an eval command
add test_l3 to regression_test

Signed-off-by: Matthew Stidham <stidmatt@gmail.com>
2022-07-12 15:45:45 +05:30
Matthew Stidham
ae78aee979 regression_test: Fix scenario.py method call
Signed-off-by: Matthew Stidham <stidmatt@gmail.com>
2022-07-12 15:45:45 +05:30
Ben Greear
615dd1c9c3 regression-test: check test script return code to detect failure.
Also:  merge stderr/stdout, add more specific help example,
allow configuring report dir, add safety check for missing 'jq'.

Signed-off-by: Ben Greear <greearb@candelatech.com>
2022-07-12 15:45:45 +05:30
Matthew Stidham
ad845bafa3 regression_test: If the test is a success, reset LOGGING to a blank string
Signed-off-by: Matthew Stidham <stidmatt@gmail.com>
2022-07-12 15:45:33 +05:30
Matthew Stidham
ac9d722a18 regression_test: add back logging recording logic
Signed-off-by: Matthew Stidham <stidmatt@gmail.com>
2022-07-12 15:45:33 +05:30
Matthew Stidham
38724f7daa regression_test: Check if user is using anaconda, if so, install lanforge-scripts appropriately
if the user is not using any virtual environment, install from pip using --user flag

Signed-off-by: Matthew Stidham <stidmatt@gmail.com>
2022-07-12 15:45:33 +05:30
Matthew Stidham
ea9bad3ce6 regression_test: append to filename.txt, don't overwrite data from scenario.py
Signed-off-by: Matthew Stidham <stidmatt@gmail.com>
2022-07-12 15:45:33 +05:30
Matthew Stidham
63211997a9 regression_test: Use correct arguments for lf_dataplane_test
Signed-off-by: Matthew Stidham <stidmatt@gmail.com>
2022-07-12 15:45:33 +05:30
Matthew Stidham
4c1ab7edbc regression_test: Fix mamin logic of regression-test which was broken. Restore it back to where it was and add in the new features
Signed-off-by: Matthew Stidham <stidmatt@gmail.com>
2022-07-12 15:45:33 +05:30
Matthew Stidham
a6241dc08c regression_test: Store report_data in report_data
Signed-off-by: Matthew Stidham <stidmatt@gmail.com>
2022-07-12 15:45:33 +05:30
Matthew Stidham
c89b3930e0 regression_test: fix logic, fix arguments
Signed-off-by: Matthew Stidham <stidmatt@gmail.com>
2022-07-12 15:45:33 +05:30
Matthew Stidham
91259d1474 regression_test: Save report-data to report-data.
Signed-off-by: Matthew Stidham <stidmatt@gmail.com>
2022-07-12 15:45:33 +05:30
Matthew Stidham
ffacc6dde3 regression_test: ADd debug to three commands
Signed-off-by: Matthew Stidham <stidmatt@gmail.com>
2022-07-12 15:45:33 +05:30
Matthew Stidham
9c5223637f Revert "regression_test: Add debug to three more commands": Broke debugging
This reverts commit da8109a14b295b83c6e28af5de9eeef9acb9b8ea.
2022-07-12 15:45:33 +05:30
Matthew Stidham
27f24bf2bd regression_test: Add debug to three more commands
Signed-off-by: Matthew Stidham <stidmatt@gmail.com>
2022-07-12 15:45:33 +05:30
Matthew Stidham
24f7a64092 regression-test: if exit on error is 0, run the test
Signed-off-by: Matthew Stidham <stidmatt@gmail.com>
2022-07-12 15:45:33 +05:30
Matthew Stidham
0563d0bb46 regression_test: bash varibles need to be in curly braces
Signed-off-by: Matthew Stidham <stidmatt@gmail.com>
2022-07-12 15:45:33 +05:30
Matthew Stidham
739b12d8e1 regression_test: Add exit on error flag
Signed-off-by: Matthew Stidham <stidmatt@gmail.com>
2022-07-12 15:45:33 +05:30
Matthew Stidham
c7ae54338e regression_test: Name log from gui log as gui_log.txt
Signed-off-by: Matthew Stidham <stidmatt@gmail.com>
2022-07-12 15:45:33 +05:30
Matthew Stidham
530551dd48 regression_test: Save LANforge logs to logging directory
Signed-off-by: Matthew Stidham <stidmatt@gmail.com>
2022-07-12 15:45:33 +05:30
Matthew Stidham
6c8e2dd9cb regression_test: Append scenario.py output to STDOUT file
Signed-off-by: Matthew Stidham <stidmatt@gmail.com>
2022-07-12 15:45:33 +05:30
Matthew Stidham
aa5b4b14ef regression_test: log_filter isn't ready for production
Signed-off-by: Matthew Stidham <stidmatt@gmail.com>
2022-07-12 15:45:33 +05:30
Matthew Stidham
0ac894d599 regression_test: Grab run_client and run_mgr log files
Signed-off-by: Matthew Stidham <stidmatt@gmail.com>
2022-07-12 15:45:33 +05:30
Matthew Stidham
3467b5c13d regression_test: Cleanup log files so we only have the text which is relevant to the test which just ran
Signed-off-by: Matthew Stidham <stidmatt@gmail.com>
2022-07-12 15:45:33 +05:30
Matthew Stidham
d8fa486d13 regression_test: run in virtual environment if virtual environment exists
Signed-off-by: Matthew Stidham <stidmatt@gmail.com>
2022-07-12 15:45:33 +05:30
Matthew Stidham
381ae85172 regression_test: Make a simpler call for the beginning of function test() when we call scenario.py
Signed-off-by: Matthew Stidham <stidmatt@gmail.com>
2022-07-12 15:45:33 +05:30
Matthew Stidham
f5f29ea9ee regression_test: fix using = in python argparse argument error
Signed-off-by: Matthew Stidham <stidmatt@gmail.com>
2022-07-12 15:45:33 +05:30
Matthew Stidham
58a6dd07cb regression_test: simplify start_tests and add logic for checking ports in scenario.py
Signed-off-by: Matthew Stidham <stidmatt@gmail.com>
2022-07-12 15:45:33 +05:30
Matthew Stidham
d667912bca regression_test: Fix links to logging directories
Signed-off-by: Matthew Stidham <stidmatt@gmail.com>
2022-07-12 15:45:33 +05:30
Matthew Stidham
164a58a723 regression_test: we only need to mkdir log_dir/name once, so I removed it on line 548
Signed-off-by: Matthew Stidham <stidmatt@gmail.com>
2022-07-12 15:45:33 +05:30
Matthew Stidham
bc608b913c regression-test: logging links need to point to sub directory
Signed-off-by: Matthew Stidham <stidmatt@gmail.com>
2022-07-12 15:45:33 +05:30
Matthew Stidham
f26c994880 regression: tail needs a lowercase n, not an uppercase n to grab the last n lines
Signed-off-by: Matthew Stidham <stidmatt@gmail.com>
2022-07-12 15:45:33 +05:30
Matthew Stidham
f12a85fc9b regression_test: Simplify logic to store logging information by writing it only once
Signed-off-by: Matthew Stidham <stidmatt@gmail.com>
2022-07-12 15:45:33 +05:30
Matthew Stidham
a2931eea9e regression_test: Record log information about failed tests to LANforge device, search for Starting Scenario string which is automatically logged by scenario.py
Signed-off-by: Matthew Stidham <stidmatt@gmail.com>
2022-07-12 15:45:33 +05:30
Matthew Stidham
9dc92b3763 regression_test: Fix regression test logs address logic
Signed-off-by: Matthew Stidham <stidmatt@gmail.com>
2022-07-12 15:45:33 +05:30
Matthew Stidham
3a9357142b regression_test: else statements don't require the word then, fix my bash mistake
Signed-off-by: Matthew Stidham <stidmatt@gmail.com>
2022-07-12 15:45:33 +05:30
Matthew Stidham
7076c08a3e regression_test: save error logs to folder if a test fails and link to that folder in the regression report
Signed-off-by: Matthew Stidham <stidmatt@gmail.com>
2022-07-12 15:45:33 +05:30
Matthew Stidham
19dc262e51 regression_test: Add VAP_SSID argument for tests which create a VAP
Signed-off-by: Matthew Stidham <stidmatt@gmail.com>
2022-07-12 15:45:33 +05:30
Jed Reynolds
8c8b5279b4 regression_test.sh: adds meta tag to discourage browser caching
Signed-off-by: Jed Reynolds <jed@bitratchet.com>
2022-07-12 15:45:32 +05:30
Matthew Stidham
b5fcfa133e regression_test: test_l3_unicast_traffic_gen: update arguments
Signed-off-by: Matthew Stidham <stidmatt@gmail.com>
2022-07-12 15:45:32 +05:30
Matthew Stidham
c900dc06b6 regression_test: add test_l3_unicast_traffic_gen to regression_test
Signed-off-by: Matthew Stidham <stidmatt@gmail.com>
2022-07-12 15:45:32 +05:30
Matthew Stidham
47c49b57e5 regression_test: Run git pull --rebase at the beginning of the script, as a safety check
Signed-off-by: Matthew Stidham <stidmatt@gmail.com>
2022-07-12 15:45:24 +05:30
Matthew Stidham
a77b12e964 regression_test: Use BSSID variable wherever applicable in script
Signed-off-by: Matthew Stidham <stidmatt@gmail.com>
2022-07-12 15:45:17 +05:30