Commit Graph

2809 Commits

Author SHA1 Message Date
Chinasubbareddy Mallavarapu
6a0feecaef [ceph-osd] Fix the sync issue between osds when using shared disk for metadata
This is to fix the sync between ceph osds when they are using shared
disk for metadata as they are having conflict while preparing the metadata disk.

we are adding a lock when first osd preparing the sahred metadata disk so that
other osd will wait for the lock, also adding udev settle in few places to get
latest tags on lvm devices.

Change-Id: I018bd12a3f02cf8cd3486b9c97e14b138b5dac76
2020-10-11 04:04:53 +00:00
Andrii Ostapenko
3e52027702 Adds vexxhost based nodeset
Adds ubuntu-bionic-expanded nodeset based on ubuntu-bionic-expanded-vexxhost
that is 16c/~32GB
Switches long running support job to new nodeset to reduce build time.

Change-Id: Iec27c00bf54efa7d686b3176998fc8ad6c9f287e
Signed-off-by: Andrii Ostapenko <andrii.ostapenko@att.com>
2020-10-09 10:32:06 -05:00
Zuul
a20afe253f Merge "adding archiving to postgres" 2020-10-08 22:34:50 +00:00
Zuul
4e5fe5ec10 Merge "[Libvirt] Values overrides for external Ceph Cinder backend" 2020-10-08 20:09:55 +00:00
Apurva Gokani
85cbd6f04b adding archiving to postgres
To safeguard postgres from clogging up wal files
in pg_xlog directory, This change does the following:
1) adding postgres archiving to move the WAL file to different directory
2) Makes sure that archive is in different Persistent volume.

Change-Id: I59bc76f27384d4f3836ef609855afcc33a7b99d0
2020-10-08 13:14:03 -05:00
Zuul
27d6f4aa70 Merge "Add conntrack required for k8s 1.18.x" 2020-10-08 17:12:42 +00:00
Huang, Sophie (sh879n)
df3918b4fe [Libvirt] Values overrides for external Ceph Cinder backend
Example values_overrides file is added to indicate how to
override the Libvirt manifest for configure an additional
externally managed Ceph Cinder backend.

Change-Id: I8e7a294059a2d98fb7854a281a29dcff80530d2b
2020-10-08 15:01:58 +00:00
Andrii Ostapenko
9d7d4e4ba6 Add conntrack required for k8s 1.18.x
Change-Id: I8c85e11ad984fa283139b3d71d52bc7c3fb8e900
Signed-off-by: Andrii Ostapenko <andrii.ostapenko@att.com>
2020-10-07 18:34:34 +00:00
Zuul
72658e2712 Merge "Revert "Prometheus: Render Rules as Templates"" 2020-10-07 15:11:14 +00:00
Zuul
f5caa0c7da Merge "Add extensible command line flags to Alertmanager" 2020-10-06 19:32:03 +00:00
Zuul
d75d0d8f00 Merge "[ceph-osd] Don't try to prepare OSD disks that are already deployed" 2020-10-06 18:17:51 +00:00
Zuul
c658ac37df Merge "Prometheus: Add configurable readiness/liveness Probes" 2020-10-06 16:11:57 +00:00
Steven Fitzpatrick
cdd0f33d0c Revert "Prometheus: Render Rules as Templates"
This reverts commit fb7fc87d23.

I first submitted that as a way to add dynamic capability to the
prometheus rules (they infamously don't support ENV variable
substitution there). However this be done easily with another solution,
and would clean up the prometheus chart values significantly.

Change-Id: Ibec512d92490798ae5522468b915b49e7746806a
2020-10-06 15:21:18 +00:00
Stephen Taylor
38d9f35c05 [ceph-osd] Don't try to prepare OSD disks that are already deployed
This addresses an issue that can prevent some OSDs from being able
to restart properly after they have been deployed. Some OSDs try to
prepare their disks again on restart and end up crash looping. This
change fixes that.

Change-Id: I9edc1326c3544d9f3e8b6e3ff83529930a28dfc6
2020-10-05 18:40:48 -05:00
Zuul
1af5482e2d Merge "Alerta update to slience alert in Alertmanager" 2020-10-05 20:34:42 +00:00
Steven Fitzpatrick
f4bdb713c1 Prometheus: Add configurable readiness/liveness Probes
This change adds probes to the prometheus statefulset using the HTK
probe generation functions

Change-Id: I249d662dd0d23dd964f7118af94c733bbdc5db92
2020-10-05 19:28:00 +00:00
Steven Fitzpatrick
2bdf4f8239 Add extensible command line flags to Alertmanager
Alertmanager is configured similarly to Prometheus. This change
brings the utils.command_line_flags template from the osh-infra
prometheus chart to Alertmanager, allowing these flags to be
configured in Values.yaml

Change-Id: Ieca94c09881bc52b62500efa4c6f8730b9208d3b
2020-10-05 17:05:26 +00:00
Zuul
006835b59e Merge "[kube-state-metrics] Update to make current" 2020-10-05 15:43:16 +00:00
Xiaoguang(William) Zhang
afe0a7830c Alerta update to slience alert in Alertmanager
Provide Alerta function to be able to slience alert in Alertmanger from Alerta

Change-Id: Iae1599f15ddcccd9f8ec05d8acee24a3dcc573d1
2020-10-05 10:13:44 -04:00
Phil Sphicas
1f5e3ad8c7 Fix disappearing metacontroller CRDs on upgrade
The existing metacontroller chart conditionally only renders the CRDs if
the metacontroller does not exist.

This creates an oscillatory effect every time the chart is upgraded - if
CRDs are present, then they will be removed, and if they are absent,
they will be installed.

This change removes the metacontroller.k8s.io/v1alpha1 capabilities
check, and relies on the values.yaml option 'manifests.crds' only to
decide whether or not to render the CRDs. In an upgrade, tiller should
do the right thing based on whether the CRDs need updating.

Change-Id: I683c9e5695b7fcdddc8b6ef8622cddb96797111c
2020-10-05 03:44:17 +00:00
Chris Wedgwood
25d2b06c16 [kube-state-metrics] Update to make current
Update image to a version appropriate for current Kubernetes versions,
adjust RBAC appropriately.

Change-Id: I6c7835cb18737f98e37a433bde8fd232d6f5479e
2020-10-04 21:43:25 +00:00
Andrii Ostapenko
9d5b9a9e42 Unpin images built with osh-images
Enabling ability to automate testing and auto promotion.
Unpinning ovs, mariadb and node-problem-detector images.

Change-Id: I6256452d575d23f84f4fd5c728437b0e4e9423f3
Signed-off-by: Andrii Ostapenko <andrii.ostapenko@att.com>
2020-10-01 18:43:13 +00:00
Huang, Sophie (sh879n)
d86502a7b7 Fix MariaDB backup script
When multiple users are granted access to a database, the
MariaDB backup script failed to retrieve the grants for that
database, which caused the backup job to fail. This patchset
updates the script.

Change-Id: I9076b2e7363ae0ec216d4e822f385fa949df8f54
2020-09-30 22:01:22 +00:00
Zuul
4665fdad03 Merge "Add default value for property in x-kubernetes-list-map-keys" 2020-09-30 16:34:31 +00:00
Zuul
724fa773ff Merge "mariadb security best practice fixes" 2020-09-30 16:26:00 +00:00
Zuul
5576b3644f Merge "[ceph-osd] Search for complete logical volume name for OSD data volumes" 2020-09-30 15:20:49 +00:00
Taylor, Stephen (st053q)
173bf928df [ceph-osd] Search for complete logical volume name for OSD data volumes
The existing search for logical volumes to determine if an OSD data
is already being used is incomplete and can yield false positives in
some cases. This change makes the search more correct and specific in
order to avoid those.

Change-Id: Ic2d06f7539567f0948efef563c1942b71e0293ff
2020-09-30 04:25:30 +00:00
Andrii Ostapenko
0deef8370a Unpin prometheus-openstack-exporter image
This enables ability to continuously update and test an image with
osh-infra gate and periodic pipeline.

Change-Id: I34ad5f8033038216129955b049d3ed09dfc0c140
Signed-off-by: Andrii Ostapenko <andrii.ostapenko@att.com>
2020-09-30 04:20:37 +00:00
Zuul
aad20a35c4 Merge "Make database backups work with openstack Train" 2020-09-29 19:49:55 +00:00
Luna Das
739ad9efe2 Add default value for property in x-kubernetes-list-map-keys
This PS fixes the CRD spec validation errors seen in k8s 1.18.6,
the errors were not seen in the previous k8s version.

Change-Id: Iec1381eca2a21268d40827dbce105899b8d129b3
2020-09-29 03:46:46 -05:00
rajesh.kudaka
9a3844aac6 mariadb security best practice fixes
This commit ensures the below mariadb settings with reference to [0]:
 - 'local_infile' Is Disabled
 - 'have_symlink' Is Disabled
 - 'secure_file_priv' Is Not Empty
 - 'sql_mode' Contains 'STRICT_ALL_TABLES'

[0] https://dev.mysql.com/doc/mysql-security-excerpt/8.0/en/general-security-issues.html

Change-Id: I701b9bc2bdfb91d67aef91e88f953a09ac72d8be
2020-09-29 03:14:17 -05:00
Parsons, Cliff (cp769u)
fc8d855a43 Make database backups work with openstack Train
This PS fixes a problem with the main backup script in the helm-toolkit,
which tries to create a swift container using the SWIFT_URL. The problem
is that the SWIFT_URL is malformed because the call to openstack get
catalog list has a different format in Train than it did in Stein. So a
solution that works for both Train and Stein is needed. This patch will
use openstack catalog show instead and will extract the public URL from
that output.

Change-Id: Ic326b0b4717951525e6b17ab015577f28e1d321a
2020-09-28 20:04:05 +00:00
Zuul
b3cea56e55 Merge "Override images provided as artifacts from buildset registry" 2020-09-28 19:25:49 +00:00
Zuul
094bce5e97 Merge "Realize libvirt SSL" 2020-09-28 19:09:55 +00:00
Andrii Ostapenko
7a0558bd78 Override images provided as artifacts from buildset registry
Switch from using images from defined in docker_images to provided as
zuul artifacts. Currently to be used in conjunction with [0] in
openstack-helm-images pipelines.

[0] https://review.opendev.org/741551

Change-Id: I43dbd38906e8854c87a361f2e5e479f57850252f
Signed-off-by: Andrii Ostapenko <andrii.ostapenko@att.com>
2020-09-28 17:38:29 +00:00
Zuul
aa26614df4 Merge "[ceph-osd] wait for only osd pods from post apply job" 2020-09-28 17:10:48 +00:00
Chinasubbareddy Mallavarapu
67c905cae8 [ceph-osd] wait for only osd pods from post apply job
This is to wait only for osd pods during ceph-osd chart install/upgrade
process.

Change-Id: I99bc7c1548f7b13c93059ac832b9f0589b049fc7
2020-09-25 08:45:51 -05:00
okozachenko
3bcb347a5b Realize libvirt SSL
Motivation: libvirt 127.0.0.1 listen is terrible for live migration.
To resolve that, we can use 0.0.0.0 but it is not secure so tried
to realize SSL.
Once create secrets for cacert, client&server cert and keys then it will
 mounted on libvirt daemonset.
It means all instances use the same key and cert. This is not ideal
 but can be considered as the first stage.

Change-Id: Ic3407e484039afaf98495e0f6028254c4c2a0a78
2020-09-25 16:36:42 +03:00
Phil Sphicas
f7ed96c701 Add extra DNS names to Ingress (helm-toolkit 0.1.1)
The existing helm-toolkit function "helm-toolkit.manifests.ingress"
will create namespace-fqdn and cluster-fqdn Ingress objects when the
host_fqdn_override parameter is used, but only for a single hostname.

This change allows additional FQDNs to be associated with the same
Ingress, including the names defined in the list:

    endpoints.$service.host_fqdn_override.$endpoint.tls.dnsNames

For example:

    endpoints:
      grafana:
        host_fqdn_override:
          public:
            host: grafana.openstackhelm.example
            tls:
              dnsNames:
                - grafana-alt.openstackhelm.example

Will produce the following:

      spec:
        tls:
          - secretName: grafana-tls-public
            hosts:
              - grafana.openstackhelm.example
              - grafana-alt.openstackhelm.example
        rules:
          - host: grafana.openstackhelm.example
            http:
              # ...
          - host: grafana-alt.openstackhelm.example
            http:
              # ...

Change-Id: I9b068f10d25923bf61220112da98d6fbfdf7ef8a
2020-09-24 23:16:32 +00:00
Andrii Ostapenko
1532958c80 Change helm-toolkit dependency version to ">= 0.1.0"
Since we introduced chart version check in gates, requirements are not
satisfied with strict check of 0.1.0

Change-Id: I15950b735b4f8566bc0018fe4f4ea9ba729235fc
Signed-off-by: Andrii Ostapenko <andrii.ostapenko@att.com>
2020-09-24 12:19:28 -05:00
Tin Lam
6d5b84a458 chore(ver): updates the k8s-keystone-auth version
The default value of the kubernetes keystone authorization webhook is
grossly outdated (v0.2). This patch set brings the default up to the
latest of this patch set (v1.19).

Change-Id: Idbf8d027ad6d5f4fb8bdedaf3047c06c66eef27d
Signed-off-by: Tin Lam <tin@irrational.io>
2020-09-24 05:41:44 +00:00
Steven Fitzpatrick
22ac30d879 Alerta: Fix values reference in ingress manifests
This change corrects the path in these Values references.

Change-Id: Ibbbd528fd0ecfb98b98ac0e0b95d108c4f320817
2020-09-22 15:51:33 -05:00
Mohammed Naser
52093576aa publish: fix missing task for creating new index
The task was missed which was causing the post pipeline to fail, this
patch should fix it by adding the missing task.

Change-Id: I13955b1c9ac3899325f7397da6bf5379b3991241
2020-09-18 16:16:50 -04:00
Zuul
87535287f3 Merge "lint: enable running on changed charts only" 2020-09-18 19:21:17 +00:00
Zuul
fa7bb3cdba Merge "[ceph-client] Fix issue with checking if autoscaler should be enabled" 2020-09-18 18:02:14 +00:00
Zuul
487c50a40a Merge "Publish Helm charts into tarballs.openstack.org" 2020-09-18 17:08:07 +00:00
Brian Wickersham
11ab577099 [ceph-client] Fix issue with checking if autoscaler should be enabled
This corrects an issue in the create_pool function with checking
if the pg autoscaler should be enabled.

Change-Id: Id9be162fd59cc452477f5cc5c5698de7ae5bb141
2020-09-18 13:19:55 +00:00
Huang, Sophie (sh879n)
b0fcd5a411 [Libvirt] Add support for Cinder external ceph backend
This patchset adds a libvirt secret for the Cinder uuid of external
ceph backend when Cinder externally managed ceph backend is
enabled.

Change-Id: I3667c13c31e49f00d2be02efa6d791ce0a580a8d
2020-09-17 22:40:19 +00:00
Mohammed Naser
d4e2228f03 lint: enable running on changed charts only
At the moment, we are using --all which means run the linting on all of
the charts.  However, the problem with using --all is that it disables
version checking which means we can't enforce version changes on Helm
charts.

This patch drops it which means the chart-testing logic will go over the
changed files and make sure that it lints those charts which have
undergone changes.

Because we use a mdoel of 1 commit per merge within Gerrit, this should
still give us the exact coverage that we need without potentially
missing any linting changes.

Change-Id: I64c7896b25c1f3daaa4f61723de8a6c722aaf3a6
2020-09-17 13:50:26 -04:00
Zuul
2bfce96304 Merge "Run chart-testing on all charts" 2020-09-17 14:38:19 +00:00