The work of enable dpdk in starlingx needs to achieve the overrides
of parameters such as images, tags, labels, and pods. This function
is being implemented through the support of ovs per-host overrides.
In order to transfer the parameters such as images after overrides
to the daemonset file to achieve the corresponding functions, the
functions of overrides need to be upgraded. Move the $daemonset_yaml
parameter in the daemonset file to the overrides file, so that
daemonset file uses parameters after overrides.This patch and
https://review.opendev.org/#/c/707775/ depend on each other.
Change-Id: I210e54b28e32ba1b8e281659fee8e6eda38d79d0
Story: 2007291
Task: 38754
Depends-on: https://review.opendev.org/#/c/708894/
Signed-off-by: songgongjun <gongjun.song@intel.com>
This updates the overrides provided for deploying fluentd as a
daemonset to get kernel messages from the journal instead of
/var/log/kern.log directly, and also uses the journal to get
messages associated with logging to auth.log (syslog facility
10). This provides additional metadata and
a cleaner interface for gathering these logs via fluentd
Change-Id: I8e832db276095771d6a869e998d7a69795dfee37
Signed-off-by: Steve Wilkerson <sw5822@att.com>
This moves from using the docker profile to the default
runtime profile - which allows container engines other than
docker to work out of the box.
Change-Id: Ica5a48f8c43b90f07969b41e10dc472a772b5b43
Signed-off-by: Pete Birley <pete@port.direct>
This change adds a means of introducing new storage classes
and local persistent volumes.
Change-Id: I340c75f3d0a1678f3149f3cf62e4ab104823cc49
Co-Authored-By: Steven Fitzpatrick <steven.fitzpatrick@att.com>
This updates the Elastic Beats charts to 7.1.0 to keep them
aligned with the Kibana and Elasticsearch chart versions, which
is required for compatibility
This also updates the experimental job to use the single node
minikube deployment as opposed to the standard 5 node multinode
deployment
Change-Id: I4baba6ca2ea2f3785f11905138b67979a4501caa
Signed-off-by: Steve Wilkerson <sw5822@att.com>
This change adds in missing network policy overrides for
fluent-daemonset and prometheus-exporter, as well as removes
existing mariadb network policies overrides that were causing
the network policy check job to fail.
Change-Id: Ib7a33f3d14617f9a9fda264f32cde7729a923193
This updates the overrides used in the apparmor nonvoting job, as
recent changes to the Elasticsearch chart values structure have
resulted in this jobs repeated failure
Change-Id: Id5427cd19a382e72435ab361003bbd5f99d678ce
Signed-off-by: Steve Wilkerson <sw5822@att.com>
This updates the deployment scripts for Prometheus to leverage the
feature gate functionality rather than bash generation of the list
of override files to use for alerting rules
Change-Id: Ie497ae930f7cc4db690a4ddc812a92e4491cde93
Signed-off-by: Steve Wilkerson <sw5822@att.com>
This change adds a non-voting bandit check to openstack-helm-infra
similar to what is ran in the openstack-helm repo.
This check will be made voting in a future change once the current
failures are addressed.
Similarly this check will be modified in a future change to
only be ran when affected python files are changed.
Change-Id: I177940f7b050fbe8882d298628c458bbd935ee89
This change updates the Ceph charts to use Ceph Nautilus images
built on Ubuntu Bionic instead of Xenial. The mirror that hosts
Ceph packages only provides Nautilus packages for Bionic at
present, so this is necessary for Nautilus deployment.
There are also several configuration and scripting changes
included to provide compatibility with Ceph Nautilus. Most of
these simply allow existing logic to execute for Nautilus
deployments, but some logical changes are required to support
Nautilus as well.
NOTE: The cephfs test has been disabled because it was failing
the gate. This test has passed in multiple dev environments, and
since cephfs isn't used by any openstack-helm-infra components we
don't want this to block getting this change merged. The gate
issue will be investigated and addressed in a subsequent patch
set.
Change-Id: Id2d9d7b35d4dc66e93a0aacc9ea514e85ae13467
This updates the osh-infra-logging single node job to omit the
fluentbit deployment step, as having multiple logging daemonsets
deployed to the single node jobs is causing IO issues. Also, it
was noted that the fluentd-deployment step was missing the
overrides to move the fluentd-deployment release from utilizing a
daemonset to a deployment. This resulted in 3 logging daemons
being deployed to a single host
Change-Id: I4a0c5550e6ea6a331aab0082a975f161e65704bf
Signed-off-by: Steve Wilkerson <sw5822@att.com>
This adds a helm-toolkit util for consuming arbitrary secret env
variables via pod env variables. It also updates the Fluentd chart
to add a release secret that is used to house the secret env
variables defined in the chart's values.yaml. This can be used as
an example to expand to other charts where this functionality is
desired
Change-Id: I9ef606840af92e54b2204e637c58442085e2c748
Signed-off-by: Steve Wilkerson <sw5822@att.com>
This updates the Armada manifests to remove the explicit enabling
of helm tests (as the default armada behavior is now to test by
default) and updates the ceph-osd chart document to set the native
helm wait behavior to false (required for the update-uuid job to
complete successfully)
Change-Id: Ia84f20ce0f38be5f07dedce70b3fbe424a037ba2
Signed-off-by: Steve Wilkerson <sw5822@att.com>
This removes the default dashboards from the Grafana chart and
instead places them in the values_overrides directory, similar to
what was done for the Prometheus rules. As Grafana dashboards
will likely be heavily dependent upon end-user needs, the old
default dashboard configs should only be used as a reference
instead of opinionated defaults that are difficult to override.
The previous defaults made using specialized labels for dashboard
variables difficult, as they were making dangerous assumptions
about deployed namespaces and host fqdns. By removing the defaults
entirely, end users can define their own dashboards to meet their
specialized needs
Change-Id: I7def8df68371deda0b75a685363c8a73b818dd45
Signed-off-by: Steve Wilkerson <sw5822@att.com>
Currently using envsubst to perform substitution of value overrides in
the feature gate caused conflicts as gotpl gets templated into those
overrides. This adds in '%%%REPLACE_${var}%%%' and uses sed to perform
the substitution instead to address the issue.
Change-Id: I9d3d630b53a2f3d828866229a5072bb04440ae15
Signed-off-by: Tin Lam <tin@irrational.io>
This patch set adds the feature gate capability to OpenStack-Helm-Infra
repository without depending on the main OpenStack-Helm repository.
Change-Id: I70b8fac4fd2365f8eedcf50519f125eb34534f2f
Signed-off-by: Tin Lam <tlam@omegaprime.dev>
Signed-off-by: Tin Lam <tin@irrational.io>
This proposes adding a kafka chart to osh-infra that aligns
with the design patterns laid out by the other charts in osh-infra
and osh.
danielqsj's kafka-exporter image is leveraged to deploy a prometheus
exporter for kafka alongside the main application if enabled in
values.yaml
Change-Id: I5997b0994fc3aef9bd1b222c373cc3a013112566
Co-Authored-By: Meghan Heisler <mh783g@att.com>
This updates the Elasticsearch and Kibana charts to deploy
version 7.1.0. This move required significant changes to both
charts, including: changing elasticsearch masters to a statefulset
to utilize reliable dns names for the discovery process, config
updates to reflect deprecated/updated/removed values, use the
kibana saved objects api for managing index patterns and setting
the default index, and updating the elasticsearch entrypoint
scripts to reflect the use of elastic-keystore for storing s3
credentials instead of defining them in the configuration file
Change-Id: I270d905f266fc15492e47d8376714ba80603e66d
Signed-off-by: Steve Wilkerson <sw5822@att.com>
Using `--network-plugin=cni` for `minikube start` will have minikube
wait for Kubernetes components to spin up and not require the Node to be
in ready status.
Change-Id: I08bf40ac4790955c107e8fee6a004b930c333d16
This adds support for arbitrary object definitions via the conf
key in the Nagios chart. This allows for customizing the
definitions required by different deployment targets instead of
assuming all nagios deployments are monitoring and targeting the
same hosts and executing the same service checks and commands.
This also adds reference overrides to the chart for elasticsearch,
postgresql, and openstack nagios objects that are deployed in the
single and multinode jobs here
Change-Id: I6475ca980447591b5b691220eb841a2ab958e854
Signed-off-by: Steve Wilkerson <sw5822@att.com>
The way that the minikube K8s script orders a host's resolv.conf file
leaves service endpoints inaccessible from the host itself even though
they are accessible within the cluster, leaving the OpenStack client
unusable from the minikube node. This change resolves the service access
issues by reordering the DNS entries in the host's resolv.conf file.
Change-Id: I58bf6d541e59f3049a0e350291e07241f6a6b544
Signed-off-by: Drew Walters <andrew.walters@att.com>
This updates the Grafana chart to support the definition of
multiple datasources. This moves to defining a template in the
chart's values.yaml file that allows for inline gotpl for
defining an arbitrary number of datasources. This also updates the
grafana dashboards to include a selector for the Prometheus
datasource to use via a drop down selector. This is vetted out in
the federated monitoring job
Change-Id: I55171fed5c2b343130d135d0b42bc96ff11c4712
Signed-off-by: Steve Wilkerson <sw5822@att.com>
This updates the Prometheus chart to support federation. This
moves to defining the Prometheus configuration file via a template
in the values.yaml file instead of through raw yaml. This allows
for overriding the chart's default configuration wholesale, as
this would be required for a hierarchical federated setup. This
also strips out all of the default rules defined in the chart for
the same reason. There are example rules defined for the various
aspects of OSH's infrastructure in the prometheus/values_overrides
directory that are executed as part of the normal CI jobs. This
also adds a nonvoting federated-monitoring job that vets out the
ability to federate prometheus in a hierarchical fashion with
extremely basic overrides
Change-Id: I0f121ad5e4f80be4c790dc869955c6b299ca9f26
Signed-off-by: Steve Wilkerson <sw5822@att.com>
This removes the elasticsearch-ldap.sh script from the single node
osh-infra-logging job, as this step does not provide any real
value and is tightly coupled to the elasticsearch version used.
This sort of validation should be reserved for smoke tests in
future helm tests for charts
Change-Id: I7ca4805a8809568cb09c8bab6c239c008528fd6a
Signed-off-by: Steve Wilkerson <sw5822@att.com>
This updates the Grafana version deployed by default from 5.0.0 to
6.2.0
Change-Id: I39b5405cc3f3fe7754ed6544a8388ff912a4ef58
Signed-off-by: Steve Wilkerson <sw5822@att.com>
This proposes adding a zookeeper chart to osh-infra that aligns
with the design patterns laid out by the other charts in osh-infra
and osh.
Change-Id: I25edc58fc951e7f81f7275ade6cf9c97e0afae02
Signed-off-by: Steve Wilkerson <sw5822@att.com>
Co-Authored-By: Steven Fitzpatrick <steven.fitzpatrick@att.com>
This updates the ceph health check command in Nagios to use the
updated plugin that determines the active ceph-mgr instance
endpoint to use before querying for ceph's health. This results in
more robust and reliable reporting of ceph's overall health
Depends-On: https://review.opendev.org/#/c/693900/
Change-Id: I5eeb076e5af3c820dbdcc3cc321cefcb5f85ef8d
Signed-off-by: Steve Wilkerson <sw5822@att.com>
This change updates the deploy-selenium playbook
to install Selenium using pip 3.
The shell scripts which call our selenium test python
files have been updated to use python 3.
Change-Id: I7fc82ecd830d460639c718614ce1cca9fd4d1953
This change would update the LMA selenium tests to share a more similar
structure. Additional logging is introduced, and the scripts will fail
gracefully if the targeted service cannot be reached.
Change-Id: Ief7f69b952b2f0e33b7b738bb76083247766f662
This updates the kubeadm and minikube Kubernetes deployments to
deploy version 1.16.2
Change-Id: I324f9665a24c9383c59376fb77cdb853facd0f18
Signed-off-by: Steve Wilkerson <sw5822@att.com>
This patch set is one of many to migrate existing code/script to be
python-3 compatible as python-2 is sunsetting in January 2020.
Change-Id: I4a8fa4c07fd36583716b5ccfdcb0bcdc008db3e7
Signed-off-by: Tin Lam <tin@irrational.io>
This patch set adds the ability to add a label to the following namespaces:
- kube-public
- kube-system
- default
This was previously done in kubeadm in patch [0] but appears to be
removed as the script moved around. Also, this PS adds this to the
minikube deployment.
[0] https://review.opendev.org/#/c/540131/
Change-Id: I0f06e8ae0cd7742313b447dc2d563c7d92318fb0
Signed-off-by: Tin Lam <tin@irrational.io>
