Commit Graph

92 Commits

Author SHA1 Message Date
portdirect
dc023525e0 Flannel: Fix RBAC definitions
This PS fixes the RBAC declarations for the Flannel Chart.

Change-Id: I9fab67b6089efcd11cac8a04ec5da0f8451b8f2c
2017-12-26 11:58:08 -05:00
Zuul
8673d55243 Merge "Helm-Toolkit: Split prom pod annotations templates into files" 2017-12-21 17:42:45 +00:00
Zuul
6321ff8400 Merge "Fix Makefile" 2017-12-20 22:58:36 +00:00
portdirect
3b6596c56e Prometheus: Update values to be yaml rather than freeform text
This PS udpates the Prometheus values to use yaml rather than text.

It also consolates all configuration into a single `etc` configmap,
inline with other OSH charts.

Change-Id: I162d4817a2b1b842499ef27d754707f8fce23bf3
2017-12-20 01:49:40 -05:00
Zuul
558ed8cd27 Merge "RBAC: Consolidate serviceaccounts and restrict rbac" 2017-12-20 04:38:15 +00:00
portdirect
c11c45dda7 Helm-Toolkit: Split prom pod annotations templates into files
This PS splits the `prometheus_pod_annotations.tpl` into seperate
files for each definition contained within it to be consistent
with other funstions in Helm-Toolkit, which can be located by path
from their name.

Change-Id: Ief9e31ead7eb1028cedd8e608d6b11e53e63e515
2017-12-19 20:24:39 -05:00
Tin Lam
628fd3007d RBAC: Consolidate serviceaccounts and restrict rbac
Currently, services have two serviceaccounts: one specified in the
chart that cannot read anything, and one injected via helm-toolkit
that can read everything. This patch set refactors the logic to:

- cleanup the roles and their binding automatically when the helm
  chart is deleted;
- remove the need to separately mount a serviceaccount  with secret;
- better handling of namespaces resource restriction.

Co-Authored-By: portdirect <pete@port.direct>

Change-Id: I47d41e0cad9b5b002f59fc9652bad2cc025538dc
2017-12-19 20:22:57 -05:00
Zuul
770a53521d Merge "Fix elasticsearch curator file entry in configmap" 2017-12-19 20:54:02 +00:00
portdirect
8b6d6c43cb Gate: collect infor about more kubernetes objects
This PS fleshes out the list of objects that info is collected for
in the gate.

Change-Id: I8f9560a05f5a5c7f5b27dcc7108ea3edc991206f
2017-12-19 09:56:34 -05:00
Zuul
1545482b8b Merge "Add alert templates via alertmanager's values.yaml file" 2017-12-19 06:16:28 +00:00
Zuul
e963c8fc9d Merge "Add peer meshing to Alertmanager" 2017-12-19 06:16:26 +00:00
Steve Wilkerson
917865ed97 Add alert templates via alertmanager's values.yaml file
This adds the ability to define custom alert template via the
values.yaml file for Alertmanager. This will provide the ability
for an operator to define actions to be taken upon an alert firing
such as sending Slack alerts, email alerts, or any other
organization-specific action

Change-Id: I78a40e43cfeb7391699908a1f73b57846fedbcbb
2017-12-18 11:24:45 -06:00
Zuul
93e7dc14b7 Merge "NFS-Provisioner: Add support to back NFS with volume claims" 2017-12-18 17:22:48 +00:00
Steve Wilkerson
9fdbd235be Add peer meshing to Alertmanager
Adds additional flags to Alertmanager for the peer meshing. This
also adds a headless discovery service so each instance can
calculate the DNS names of its mesh peers on startup.

Change-Id: I2ba7f4aec88f73e6bc3ff31117973ebb4e85ceba
2017-12-18 10:15:14 -06:00
Zuul
4c9383b216 Merge "Include prometheus- prefix for select monitoring charts" 2017-12-18 15:51:38 +00:00
Zuul
fd9cde08f5 Merge "Gate: move all checks to voting" 2017-12-18 15:51:38 +00:00
portdirect
dae9b82918 NFS-Provisioner: Add support to back NFS with volume claims
This ps adds the ability for the NFS-provisioner to use a volume
claim for providing storage for other services. This provides the
ability to provide read-write-many access backed by a
read-write-once storage class, in situations where such a
requirement exists.

Change-Id: I7dcf79b871fd4fa699ee4e3a50151a654f27761f
2017-12-18 10:40:25 -05:00
Steve Wilkerson
1753d19a68 Fix elasticsearch curator file entry in configmap
Fixes an issue preventing the elastic curator configuration being
populated via elasticsearch's values.yaml

Change-Id: I74901c1aa99abc56a06ea95ca9ea9d818761d79b
2017-12-18 08:13:12 -06:00
Steve Wilkerson
938bce7370 Include prometheus- prefix for select monitoring charts
This adds the prometheus- prefix to the alertmanager,
kube-state-metrics and node exporter charts to reflect their
intended usage as part of a prometheus centric monitoring solution

This will imply a logical grouping of these components, similar to
their deployment in the osh-infra gates

Change-Id: I4f391a10b64389022f01a94ea3704c110f8f9bb5
2017-12-17 23:22:50 -05:00
Zuul
8d309f5cff Merge "Update Prometheus to version 2.0" 2017-12-18 04:19:21 +00:00
Tin Lam
e3e9dcabb2 Fix Makefile
This patch sets fixes the make clean target as it currently attempts
to delete the generated */charts up the parents, where the directory
is guaranteed to be non-empty.

Change-Id: Id1327998cc1cdc73bdf0113d5ec68330d9fc70f0
2017-12-17 23:47:46 +00:00
portdirect
6e5fe71d9c Elasticsearch: Move default storage access-mode to ReadWriteOnce
This PS moves the default storage access-mode to ReadWriteOnce, as
the PVC is created inline with the statefulset. So ReadWriteMany will
have no effect, as a volume is created per pod.

Change-Id: I2a6a28832c0b1beedeb3e280572b3717628f7b88
2017-12-17 21:34:10 +00:00
Steve Wilkerson
ca6322da87 Update Prometheus to version 2.0
Updates the Prometheus chart to use version 2.0 by default. This
introduces a change in the rules format (to yaml), and changes the
flags required for the storage layer.

Change-Id: Icb06a6570683b7accebc142f75901530c6359180
2017-12-17 20:47:09 +00:00
portdirect
8c00d623ef Gate: move all checks to voting
This PS moves all the current checks to voting gates.

Change-Id: Ie3a8d67a64d6f9a9a58f8c6d935bd5cf204f98ca
2017-12-17 20:33:22 +00:00
portdirect
611a78fb34 Fluent-Logging: Update fluent-bit to use common OSH entrypoint pattern
This PS updates the fluent-logging chart to use the same entrypoint
pattern as other OSH components.

Change-Id: I3bf9baf9824e1b7f7e46c4fcae292240566d9153
2017-12-17 17:33:24 +00:00
portdirect
c189522fdb Kubernetes: Update to v1.9.0
This PS updates the OpenStack-Infra Gate to use Kubernetes v1.9.0

Change-Id: I2c2d8180a6e05ac6babc72f6347f00a19bf7e0fd
2017-12-17 17:06:40 +00:00
portdirect
f472531ace Docker: user json-file on Fedora and CentOS
The current fluent-bit implementation only supports the json-file
log driver for docker, this PS moves CentOS and Fedora to use that
until we can support Journald.

Change-Id: I8aa876aa96119d9a1a0e06c28873e3c4c1e3ace5
2017-12-17 17:06:16 +00:00
tin.l.lam
e4de36d97b CentOS: Fix jq and pip installation
This PS fixes pip and jq installation on CentOS. It also removes
some duplicate code in the gate playbooks.

Co-Authored-By: portdirect <pete@port.direct>

This patch set should fix an issue where centos cannot find
python-pip in the EPEL.

Change-Id: If3a437e0756a363b8cefaa9a8bdd1c3498fedbfd
2017-12-17 17:05:47 +00:00
Zuul
2fb6a79b49 Merge "Fluent-logging helm chart" 2017-12-15 20:29:50 +00:00
sungil
2862f038e2 Fluent-logging helm chart
This introduces an initial helm chart for fluent logging.
It provides a functional fluent-bit and fluentd deployment to
use in conjunction with elasticsearch and kibana to consume
and aggregate logs from all resource types in a cluster.
It can deliver logs to kafka for external tools to consume.

This PS moves fluent-logging chart from osh-addons, osh to
osh-infra repo.
previous ps(addon): https://review.openstack.org/#/c/507023/
previous ps(osh): https://review.openstack.org/#/c/514622/

Specification: https://review.openstack.org/#/c/505491/
Partially implements: blueprint osh-logging-framework

Change-Id: I72e580aa3a197550060fc07af8396a7c8368d40b
2017-12-15 10:52:16 -06:00
Zuul
740dbf8b37 Merge "Makefile: Allow pulling of all images for a targeted chart" 2017-12-14 22:40:32 +00:00
Zuul
093e64631d Merge "Add missing prometheus and alertmanager resources" 2017-12-14 00:20:52 +00:00
Zuul
6bda2f5acf Merge "Enable ceph-mgr label on nodes to support luminous" 2017-12-13 20:25:19 +00:00
Ganesh Maharaj Mahalingam
b3e63a9b47 Enable ceph-mgr label on nodes to support luminous
Change-Id: I17359df62a720cbd0b3ff79b1d642f99b3e81b3f
Signed-off-by: Ganesh Maharaj Mahalingam <ganesh.mahalingam@intel.com>
2017-12-13 10:06:15 -08:00
Alan Meadows
1af212c0ab Support IP addresses as hosts within keystone_endpoint_uri_lookup
This allows the keystone endpoint uri lookup function to
avoid adding a FQDN suffix to an IP address based host
entry.

Change-Id: I016e6512fb21182a8be9e3de1e4a2da59a20fb36
2017-12-13 16:45:48 +00:00
portdirect
64f6e66cb3 Makefile: Allow pulling of all images for a targeted chart
This PS brings the OSH-Infra Make inline with OSH and allows images
to be pulled on a per chart basis to the local machine.

Change-Id: Ieda89adf97140a2ad3824ff36e969bd016ccdf00
2017-12-12 19:05:59 -05:00
Zuul
1c66ca1c12 Merge "Add Elasticsearch liveness/readiness probes" 2017-12-12 20:20:31 +00:00
Steve Wilkerson
bea44e53bf Add Elasticsearch liveness/readiness probes
The elasticsearch tests fail because the pods don't have
readiness or liveliness probes in the templates. This adds those
definitions

Change-Id: I4fd25aec5ae02d89ae1b933d8b083a3e9cafc55a
2017-12-11 18:47:46 -06:00
Zuul
39968768e9 Merge "Helm: update to 2.7.2" 2017-12-12 00:27:31 +00:00
Steve Wilkerson
605faded2e Gather deployment information after checks run
Adds tasks to gather descriptions of kubernetes objects, logs from
deployed pods, logs from helm test pods, the status of each
helm release, and metrics from endpoints exposed to prometheus

Change-Id: I606797c6a5d75ba446ed2c16a9710f7b0227f910
2017-12-11 09:57:28 -06:00
Pete Birley
6faa1b4c0b Helm: update to 2.7.2
This PS updates the version of helm to v2.7.2 which includes some
security fixes.

Change-Id: Ic6de39985a88c4e0fb062c35f9ee6c256b3d20d9
2017-12-05 15:07:58 -06:00
Steve Wilkerson
5fae0f2880 Add missing prometheus and alertmanager resources
Alertmanager and prometheus were missing entries for job
resources in values.yaml. Also added resources to the prometheus
helm test template

Change-Id: I11dbad19d1f881c398a4b4dcd0c0eab23fccf278
2017-12-01 18:30:47 -06:00
Steve Wilkerson
4b94e47c94 Add Elasticsearch to OSH Infra
This moves the Elasticsearch chart to OSH infra, along with rbac
rules for running Elasticsearch. It includes a cronjob for running
ES Curator for cleaning up old indices

Change-Id: I69fcbe8b77de8b594eba5340a6e4340f389ba5bf
2017-12-01 17:42:23 -06:00
Zuul
9051ec4ac2 Merge "Prometheus monitoring for OSH infra" 2017-11-30 23:06:45 +00:00
Steve Wilkerson
429a4edd86 Prometheus monitoring for OSH infra
This will move prometheus to OSH-infra to be included as part of
the basic infrastructure deploy for openstack-helm. It includes
charts for Prometheus, Node Exporter, Kube-State-Metrics, and
Alertmanager. It provides a base for monitoring and alerting
for the underlying infrastructure

Partially Implements: blueprint osh-monitoring

Change-Id: Ie453373b54c5f1825339ce0566e4b5d0f74abc20
2017-11-30 09:43:17 -06:00
Steve Wilkerson
2a915c4a15 Add support for running helm tests for charts
Adds a common-helm-test task executed as part of the
deploy-helm-packages playbook. It allows the ability to run helm
tests against a chart by including a helm_test key in the chart
definition

Change-Id: I52bd5ca2fafa4eb704369590f7903c127133b090
2017-11-30 09:09:46 -06:00
Steve Wilkerson
13c4199742 Adds extra args for kube_scheduler deploy
This adds extra args for the kube-scheduler to expose metrics on
port 10251 on all interfaces. This allows for Prometheus to gather
metrics for the scheduler

Change-Id: I26d473f511f6541a14a9387be1ce56841572bfff
2017-11-28 08:15:31 -06:00
portdirect
70e26302f7 Gate: Fix vars for local development use
This PS resolves an issue in the way ansible variables are merged
re-enableing use of the gate scripts for local development use.

Change-Id: Ia18a9f92816e4cc49f1c5423c04bed8a80abc6dc
2017-11-21 15:37:25 -05:00
Zuul
a2449cebcb Merge "KubeADM: Move shell commands to use bash and set env vars" 2017-11-20 23:56:20 +00:00
Zuul
775b1f9f88 Merge "Gate: Modularise gate for development use" 2017-11-20 23:44:59 +00:00