Commit Graph

94 Commits

Author SHA1 Message Date
Steve Wilkerson
4f1f180e8e Add prometheus annotations to tiller-deploy service
This adds the prometheus annotations to the tiller chart, allowing
for the scraping of metrics from tiller.  currently, tiller exports
metrics specific to the grpc calls being made against it

Change-Id: Ibe10098689cc37a9de6306af86f4d63d8aab318b
2018-02-20 10:05:05 -06:00
portdirect
3713266f4c K8s: udpate KubeADM-AIO to use K8s v1.9.3
This PS updates the version of K8s used in the gate to v1.9.3

Change-Id: I284083e0cdf828701c439c44edfd3a975dcd69c3
2018-02-17 15:21:06 -05:00
Steve Wilkerson
5d95b0e2cb Add fluentd prometheus exporter for logging metrics
This adds templates for a prometheus exporter for fluentd to
adequately capture log metrics and fluentd service metrics for
consumption by prometheus

Change-Id: I6d6a8c2be07af819dc6d99b8ce5f1d4b635a69f0
2018-02-13 19:09:35 +00:00
portdirect
7a322f0a08 KubeADM: Label namespaces
This PS removes the duplicate labeling of K8s namespaces everytime
the kubeadm image is run - as this action is performed by the
container itself while deploying k8s. It also updates the playbook
to label the kube-public ns.

Change-Id: Icfaabe54d263e0bbc52f2ac11835cb5d92ad32a4
2018-02-11 13:59:15 -05:00
Zuul
3b130197e7 Merge "Fix namespace label" 2018-02-09 16:30:26 +00:00
Zuul
4297f786b0 Merge "Add elasticsearch exporter" 2018-02-07 23:30:04 +00:00
Steve Wilkerson
094e0103a9 Add elasticsearch exporter
This adds a chart for an elasticsearch exporter to expose metrics
for prometheus. This also moves the exporter to be included as
part of the chart it's meant to target as opposed to its own
separate chart

Change-Id: I491f4d1efba633827d8a6255218daeb9d427f922
2018-02-07 10:59:27 -06:00
Steve Wilkerson
6af03cd6d9 Update prometheus metrics gathering in post-job
This updates the prometheus metrics gathering role to dynamically
detect deployed namespaces instead of using hardcoded tasks for
each namespace. This allows the use of this job in openstack-helm
without needing to manually add additional namespaces

Change-Id: I2304aceab8dcda5471af708f57924193f03ee8f3
2018-02-05 18:21:49 -06:00
Tin Lam
ae54ad7cd0 Fix namespace label
This patch set fixes the namespace labels and adds in a new log file to
describe all namespaces to troubleshoot gate issues where labels are not
applied properly.

Change-Id: I34c43c345f9a49df8d7fcf2e7824220a22698d46
Signed-off-by: Tin Lam <tin@irrational.io>
2018-02-05 15:43:40 -06:00
portdirect
2d725fd00b Rollback helm version to 2.7.2
The update to helm 2.8.0 resulted in issues with releases not
registering as ready before timing out. This rolls the version back
until those issues are addressed

Change-Id: Id58f85f150054d82c047bd360258ebe9e571360f
2018-01-31 21:28:58 +00:00
portdirect
27a2b2e16d K8s/Helm: Bump versions to current release
This PS bumps the versions of helm and K8s to their current GA
release.

Change-Id: Ia205f47d4f6b89d6d54bb2a71fde28a5730d2a47
2018-01-28 23:34:52 -05:00
Steve Wilkerson
3ec7f5f0ff Gate fix: httpd image, elasticsearch, openstack-exporter
There was a change in the upstream reference httpd image for
apache that changed how modules were built for apache.
This change adds the required fix to accomodate the change.
See isssue here https://github.com/docker-library/httpd/pull/87

The Elasticsearch image tag was updated to accomodate the kernel
versions used in the gate as part of the kernel update playbook
See https://github.com/elastic/elasticsearch/issues/28349#issuecomment-360233779

The openstack-exporter binary was changed to reflect changes made
to the openstack-exporter image

Change-Id: I1deb9e7cde794421dd33fade566c2a9fdb5007e6
2018-01-28 15:07:24 -06:00
Zuul
b380b68bba Merge "Gate: collect host level info" 2018-01-20 21:29:05 +00:00
Zuul
08a6266654 Merge "Add label to namespaces" 2018-01-19 14:50:08 +00:00
portdirect
6a150bd0f9 KubeADM: Increase pod limit for hosts
This PS increases the pod limit per hosts - useful in dev envs.

Change-Id: I3bdb933d7f5001cb91db6f7f227101e587532a54
2018-01-17 18:57:12 -05:00
Tin Lam
f6347e7d28 Add label to namespaces
This patch set adds label to "kube-system" and "default"
namespaces used for podSelector.

Change-Id: Ia67a0e4d9adf6f5575d74aebf77673aae3660c62
Signed-off-by: Tin Lam <tin@irrational.io>
2018-01-17 16:08:24 -06:00
Zuul
c1a8572584 Merge "KubeADM-AIO: Move to upstream debian base image" 2018-01-17 08:03:29 +00:00
portdirect
0025eb9444 Gate: collect host level info
This PS adds basic host level log collection to the gate.

Change-Id: I5ee3905e134b2d9abdad08121ecb7e257c7165c8
2018-01-14 17:41:25 -05:00
portdirect
9b40b8656d Prometheus Openstack Exporter: tidy chart and add ks user
This PS adds keystone user management to the prometheus-openstack-exporter
chart, and also performs some spring cleaning.

Change-Id: I69e40c523867f751ecd8c63169aefdfdf4eb5cd2
2018-01-14 09:51:06 -05:00
portdirect
76d0f57e64 KubeADM-AIO: Move to upstream debian base image
This PS moves KubeADM-AIO to use the upstream debian base image
that is used to build the Kubernetes release images:
 * https://github.com/kubernetes/kubernetes/tree/master/build/debian-base

Change-Id: I153a3aee43613b429793c179b2707c916821d39a
2018-01-13 23:46:33 -05:00
portdirect
217385a0dd Gate: Simplify and speed up pod and k8s object log collection
This PS simplifys and accelerates the collection of pod container
logs and k8s object info in the zuul gates, which can currently take
40 minutes for the 5 node OSH gate to complete.

Change-Id: Ie9b23174fade3df4a87f2b771ea654e2081b4f4e
2018-01-13 21:39:42 -05:00
portdirect
3dfcde8849 KubeADM-AIO: allow cluster domain to be customised
This PS allows the cluster domain to be customised if desired.

Change-Id: I86b9271b6248a36ce39a367b814cffa2bea6d4b3
2018-01-11 16:22:27 +00:00
Zuul
3e68d02fc2 Merge "Grafana: support multiple replicas and data persistance" 2018-01-11 16:02:28 +00:00
portdirect
21dc4f8af8 Grafana: support multiple replicas and data persistance
This PS adds support for multiple replicas of Grafana to be
deployed, and adds MySQL based data persistance to the chart.

Change-Id: Ife44985a6d5024cc2074346340fba1d8efdecbfa
2018-01-11 02:39:34 -05:00
portdirect
3d633fca7a Gate: Increase timeouts for awating node to come back from reboot
This PS increases the timeout in waiting for the node to come back
online following a reboot.

Change-Id: I92d1e5b665006bf9693f56ad8272330c19e6ccfa
2018-01-10 10:48:02 -05:00
portdirect
13852ceee4 Gate: Deploy HWE kernel on ubuntu hosts
This PS deploys the HWE kernel on Ubuntu Hosts, which is required
for CephFS:
 * https://github.com/kubernetes-incubator/external-storage/issues/345

Change-Id: I2ebd46eadf5a4c7a857d42302f388511691ab0db
2018-01-09 04:04:18 +00:00
portdirect
2e0b57ad93 KubeADM-AIO: Drive basic CNI configuration via values
This PS drives basic CNI options via ansible playbook in the
KubeADM-AIO container and modifies the calico chart to support
configuration via values.

Change-Id: Iaf2f9807438c3a34e797c62c2c6913edb677997c
2018-01-07 12:32:40 -05:00
Zuul
227df8bf87 Merge "Gate: Harden helm serve playbook" 2018-01-07 06:19:32 +00:00
Zuul
604fb28fd3 Merge "Gate: Move pull images to its own playbook" 2018-01-07 03:57:20 +00:00
portdirect
f5ccebb791 Gate: Harden helm serve playbook
This PS make the helm server more robust by moving the server to be
managed via a systemd unit.

Change-Id: I651bf3b2670812ea255c3692bd933bb6092397f9
2018-01-06 20:14:19 -05:00
Zuul
8d14de64a4 Merge "Perserve env variables" 2018-01-06 23:30:18 +00:00
portdirect
229c5dd45c Gate: Move pull images to its own playbook
This PS moves build images to its own playbook, reduducing the time
taken to run the gates when not required.

Change-Id: I17203a6fa2e3c7898175389f13d680fe9347b920
2018-01-06 17:32:42 -05:00
Zuul
91548a6648 Merge "Kubernetes: Move to version 1.9.1" 2018-01-06 17:01:07 +00:00
lamt
9c31548c9b Perserve env variables
This patch set adds in -E flag for ``sudo pip install`` so
any environment variable is preserved for the install.  This
allows for some proxy-related variables such as ``http_proxy``
to pass through for pip.

This mirrors the change in [0].

[0] https://review.openstack.org/#/c/531270/1

Change-Id: Ib513fce862f31d784634645ac6b038c6a35f4cd6
2018-01-05 08:57:25 -06:00
rakesh-patnaik
fc52cda24a Openstack metrics exporter for prometheus
This exporter provides a means for Prometheus to gather openstack
service metrics related to overlying openstack-helm deployments
Change-Id: I5f1789c62b4547add0c67edb51540f712bf43da8
2018-01-05 07:51:31 -06:00
portdirect
7b6b2274ab Kubernetes: Move to version 1.9.1
This PS moves the version of kubernetes used in the gates to v1.9.1

Change-Id: Ida17cb20b0a5ce11e03ccfafddb0f58fadda76f1
2018-01-05 00:56:07 -05:00
Zuul
17b13c42d7 Merge "Add Grafana chart to OSH infra" 2018-01-05 04:59:51 +00:00
Steve Wilkerson
a1f608ed74 Add Grafana chart to OSH infra
Moves the grafana chart to OSH infra along with basic rbac rules
that may be tightened with future work.

Change-Id: Ie14627530a73d4b7b01eb93ca5f7174d99d9caec
2018-01-04 09:10:55 -06:00
Zuul
2e4b15f16e Merge "Docker: Run docker without iptables" 2018-01-02 21:42:03 +00:00
Zuul
0d5a2a64a3 Merge "Add support for out of branch variables" 2017-12-31 03:14:59 +00:00
portdirect
bbf32935dc Docker: Run docker without iptables
This PS updates the docker daemon settings to run without
apply iptables rules. This simplifies host network management by
removing one of the actors interacting with iptables.

Change-Id: I335247afddf736b60212d199a3b860c3c792977f
2017-12-30 21:59:19 -05:00
Matt McEuen
134d7cab0c Add support for out of branch variables
This change allows users of the gate script to optionally
override the location of the ansible inventory and variables files,
so that they can live outside of the source tree.

Change-Id: Ibe7666ce366c7ad34e8ee6ff5ef3f23589aa10ce
2017-12-30 13:58:56 -06:00
Zuul
41c2a6e8f3 Merge "Gate: simplify helm release log gathering" 2017-12-29 18:28:08 +00:00
portdirect
6c3786aef3 Gate: simplify helm release log gathering
This PS simplifys the helm release log gathering, to collect logs
from all charts released into the cluster, and also adds a `logs`
sub-target to the dev-deploy target in the Makefile.

Change-Id: I016a5e08163eaccf13331db6faa45fab1e9cf4f2
2017-12-28 15:40:32 -05:00
Steve Wilkerson
45ba95a2de Elasticsearch: Add curator snapshot action and PVC for fs repo
This provides an example action in the curator config for taking
snapshots of the elasticsearch indexes. As the snapshot action
requires a repository registered with Elasticsearch, this also
adds a PVC for a filesystem repository backed with NFS and a job
for registering the repository with Elasticsearch.

Change-Id: I26b788c58f52844e997bde5002459bddc1bb685e
2017-12-28 13:42:47 -06:00
Steve Wilkerson
b45e8ddcbd Move kibana to OSH infra
This moves the Kibana chart to OSH infra, which finalizes moving
the logging components to OSH infra

Change-Id: Iacbfde8d5d7099fcb4dde8a437e030c2d4936de6
2017-12-26 18:26:58 -06:00
portdirect
8b6d6c43cb Gate: collect infor about more kubernetes objects
This PS fleshes out the list of objects that info is collected for
in the gate.

Change-Id: I8f9560a05f5a5c7f5b27dcc7108ea3edc991206f
2017-12-19 09:56:34 -05:00
Steve Wilkerson
938bce7370 Include prometheus- prefix for select monitoring charts
This adds the prometheus- prefix to the alertmanager,
kube-state-metrics and node exporter charts to reflect their
intended usage as part of a prometheus centric monitoring solution

This will imply a logical grouping of these components, similar to
their deployment in the osh-infra gates

Change-Id: I4f391a10b64389022f01a94ea3704c110f8f9bb5
2017-12-17 23:22:50 -05:00
portdirect
c189522fdb Kubernetes: Update to v1.9.0
This PS updates the OpenStack-Infra Gate to use Kubernetes v1.9.0

Change-Id: I2c2d8180a6e05ac6babc72f6347f00a19bf7e0fd
2017-12-17 17:06:40 +00:00
portdirect
f472531ace Docker: user json-file on Fedora and CentOS
The current fluent-bit implementation only supports the json-file
log driver for docker, this PS moves CentOS and Fedora to use that
until we can support Journald.

Change-Id: I8aa876aa96119d9a1a0e06c28873e3c4c1e3ace5
2017-12-17 17:06:16 +00:00