lazy_re had an issue when importing sonic-cfggen in another application that
uses re.search(). There is no much improvement of lazy_re today after many
other good optimization work done for sonic-cfggen. It served as a quick
temporary solution.
Some quick test for fast-reboot and warm-reboot done on top of 201911 branch:
Fast-reboot: from ASIC reset to ports in up state:
with lazy_re: 18 sec
without lazy_re: 18 sec
Warm-reboot: LAG restoration time:
with lazy_re: 73 sec
without lazy_re: 72 sec
So, there is no real optimization since the number of sonic-cfggen calls is greatly
reduced in latest SONiC. This means it is time to revert this change.
Signed-off-by: Stepan Blyschak <stepanb@nvidia.com>
Why I did it
This PR adds changes in sonic-config-engine to consume configuration data in SONiC Yang schema and generate config_db entries
How I did it
Add a new file sonic_yang_cfg_generator .
This file has the functions to
parse yang data json and convert them in config_db json format.
Validate the converted config_db entries to make sure all the dependencies and constraints are met.
Add a new option -Y to the sonic-cfggen command for this purpose
Add unit tests
This capability is support only in sonic-config-engine Python3 package only
importlib-resources v4.0.0 was released today (2020-12-23) and drops support for Python 2. This caused the sonic-config-engine Python 2 wheel build to fail.
Reference: https://pypi.org/project/importlib-resources/
Pin 'importlib-resources' package to v3.3.1 for Python 2
Unrelated: remove pinned version of zipp for sonic-bgpcfgd because we no longer build a Python 2 version of that package
As part of consolidating all common Python-based functionality into the new sonic-py-common package, this pull request:
1. Redirects all Python applications/scripts in sonic-buildimage repo which previously imported sonic_device_util or sonic_daemon_base to instead import sonic-py-common, which was added in https://github.com/Azure/sonic-buildimage/pull/5003
2. Replaces all calls to `sonic_device_util.get_platform_info()` to instead call `sonic_py_common.get_platform()` and removes any calls to `sonic_device_util.get_machine_info()` which are no longer necessary (i.e., those which were only used to pass the results to `sonic_device_util.get_platform_info()`.
3. Removes unused imports to the now-deprecated sonic-daemon-base package and sonic_device_util.py module
This is the next step toward resolving https://github.com/Azure/sonic-buildimage/issues/4999
Also reverted my previous change in which device_info.get_platform() would first try obtaining the platform ID string from Config DB and fall back to gathering it from machine.conf upon failure because this function is called by sonic-cfggen before the data is in the DB, in which case, the db_connect() call will hang indefinitely, which was not the behavior I expected. As of now, the function will always reference machine.conf.
* [sonic-cfggen] optimize execution time
a lot of template rendering causes switch to start longer because jinja2
needs to parse them. Introducing RedisBytecodeCache to store parsed buckets of
internal template bytecode to speedup same template rendering during start
* [sonic-cfggen] do lazy regexp compilation to speedup sonic-cfggen
* [sonic-cfggen] address pep8 related comments
Signed-off-by: Stepan Blyschak <stepanb@mellanox.com>
* fix name conflict between sonic_platfrom package and sonic_platform.py
* update sonic-utility submodule to pickup lastest fix
* Revert "update sonic-utility submodule to pickup lastest fix"
This reverts commit f66aa99738e06373bc2e141f420eab71839ff4e8.
* update sonic-utility sub module
* [sonic-cfggen] Add support to generate sample t1 config
* Fix typo
* Fix space issue
* Add list of preset messages in help message
* Utilize choice support of argparse
* Fix missing module in setup.py
* [acl-loader]: acl-loader integration.
acl-loader was inplemented based on translate_acl source code. acl-loader
can't be tested with unittest. Same testcases will be covered with testbed tests.
- Remove translate_acl utility and tests.
- Remove mirror template.
- Do not run ACL rules and mirror configuration generators.
- Adopt minigraph parser to work with acl-loader.
* Update sonic-swss-common and sonic-swss submodules.
Enable ACL dynamic config feature.
* Update sonic-utilities submodule.
Include fix for acl-loader incremental update command.
* [sonic-cfggen]: Add test for everflow cfggen configuration.
Move port_config.ini related parsing code out of minigraph.py and create an independent portconfig.py.
Add support to multiple optional fields according to Porting Guide
- Add -p --port-config option to feed sonic-cfggen with port_config.ini
file when necessary.
- Update minigraph.py file to accept the -p option
- Add test_j2files.py test to test config.sh and all .j2 templates
* Currently test_teamd is added to test both the config.sh and teamd.j2
file works well with the t0 sample minigraph and sample port config
file
* The sample output is added to the folder sample_output for comparison
Signed-off-by: Shuotian Cheng <shuche@microsoft.com>
Add a sonic-config-engine to help generate config file based on minigraph and other data on runtime. Modify fpm, teamd, lldp, snmp, and platform-monitor docker to use sonic-config-engine to generate config in docker upon load.
