Commit Graph

160 Commits

Author SHA1 Message Date
Huang, Sophie (sh879n)
8a7b7ba086 Enable Cinder backends to use iSCSI for data traffic
In this patchset, the iSCSI protocol support is added
to enable Cinder to use iSCSI based storage backends.

Bootable volumes are not supported, only VM attached
volumes are supported for this initial patchset.

Change-Id: I1b35290b62d2cebae4bd8be62126a53f230ac6c0
2020-03-16 14:23:23 +00:00
zhen
df4d104841 [cinder] Fix misspelling
Security_context'readOnlyRootFilesystem' under container cinder-backup
is misspelled as' redOnlyRootFilesystem', this commit fixed this

Change-Id: I5a1a9c1a9ae66d027199057a13e3119a326ef015
2020-02-24 20:47:50 +08:00
Gage Hugo
f9dbba7043 Revert "Revert "Keystone Authtoken Cache: allow universal secret key to be set""
This reverts commit 90d070390d.

Change-Id: I017c6e9676b872e1aab21f9dc8aa2f93db58d49f
2020-02-21 11:16:55 -06:00
Gayathri Devi Kathiri
8401db0e77 Modify incorrect format of Cinder policy file
Cinder default format of policy file is changed from
"json" to "yaml" in stein. This patch set modifies
Cinder chart templates to load policies in yaml format.

Change-Id: I28f3d5be6609cd28bbc1ce8e5fc1d1cf4730b760
2020-02-18 18:09:24 +00:00
Vasyl Saienko
90d070390d Revert "Keystone Authtoken Cache: allow universal secret key to be set"
This reverts commit 1c85fdc390.

Do not use randomly generated strings in configmaps as this leads to
whole helm release redeployment even no values are changed. The random
items have to be generated outside of helm chart and provided via
values.
Also previous behaviour didn't allow to use cache during rolling upgrade
as new pods were spawned with new key.

Change-Id: I423611b18fca0d65e2e721a9c6a0c3d8df0813d2
2020-02-12 11:18:06 +00:00
Tin Lam
b17b378390 Add rally clean up script
This patch set adds in a script that cleans up orphaned or
lingering rally helm test pods.

Depends-On: https://review.opendev.org/#/c/683759/

Change-Id: I94fc8d067b421248cf74fe40b2e8520f63d4417c
Signed-off-by: Tin Lam <tin@irrational.io>
2020-01-31 19:18:01 +00:00
Tin Lam
12bee1bb97 Migrate default release to Stein
This patch set updates the default job to use OpenStack Stein release.
The previously default Ocata release will be place in separate job.

Change-Id: I489324f762a179a2cab5499a6d8e57e97c81297f
Signed-off-by: Tin Lam <tin@irrational.io>
2020-01-09 10:00:31 -06:00
Chinasubbareddy Mallavarapu
d03a8a1a22 Support the Nautilus release for Ceph
- Change all tests to support Nautilus,Mimic and Luminous releases
- Update ceph-config-helper image

Change-Id: I557b1efa12529d0ee51d4c5b9d4beb4abf1b0574
2019-12-17 16:23:15 +00:00
Tin Lam
a25eccb7cb Implements egress network policy
This patch set adds in the egress policy for core OpenStack Services.

Depends-On: https://review.opendev.org/#/c/679853/

Change-Id: I585ddabcbd640db784520c913af8eddecaee3843
Signed-off-by: Tin Lam <tlam@omegaprime.dev>
2019-11-22 01:16:49 +00:00
Steve Wilkerson
9736f5f544 Update kubernetes-entrypoint image reference
This updates the kubernetes-entrypoint image reference to consume
the publicly available kubernetes-entrypoint image that is built
and maintained under the airshipit namespace, as the stackanetes
image is no longer actively maintainedy

Depends-On: https://review.opendev.org/688435

Change-Id: I8e76cdcc9d4db8975b330e97169754a2a407341f
Signed-off-by: Steve Wilkerson <sw5822@att.com>
2019-10-21 13:58:22 +00:00
Steve Wilkerson
6e4ab4aa0c Update ceph-config-helper image
This updates the ceph-config-helper image for the ubuntu distro
based jobs to use an image that includes kubernetes 1.16.2

Change-Id: If063db5e6f0abfab10cd0195b3633c41d8ed560f
Signed-off-by: Steve Wilkerson <sw5822@att.com>
2019-10-18 08:36:26 -05:00
Tin Lam
50c982eab0 Allow allowPrivilegeEscalation
Cinder requires rootwrap functionality that we cannot disable
allowPrivilegeEscalation. This patch set removes the line
that disables it by default.

Change-Id: I23b35aee298e2e414e93ff34cd0a7012a9099e3d
Signed-off-by: Tin Lam <tin@irrational.io>
2019-07-31 19:22:18 -05:00
Pete Birley
59a017d834 RabbitMQ: Dont mirror reply queues
This PS updates the default RMQ policy to not mirror reply queues
as they cause signifigant blocking when resorting a rabbit node to
a cluster, with no advantage.

Change-Id: I6f8d4eaa482fcdf3e877bd38caa9b24358ea5be0
Signed-off-by: Pete Birley <pete@port.direct>
2019-07-30 09:54:16 -05:00
Zuul
777e5994e8 Merge "Cinder support multiple Ceph volume backends" 2019-07-29 15:16:14 +00:00
Daniel Badea
11d7e1d49a Cinder support multiple Ceph volume backends
Wrap code making the assumption there is only one Ceph backend
(using is_ceph_volume_configured) in a "range" and use
ceph_backend_list helper to iterate all available Ceph backends.

Move Ceph pool application name in values.yaml from
conf.software.rbd.rbd_pool_app_name* to conf.ceph.pools.*.app_name

Change-Id: If1126e51fe9ebb85185e375dc282e83db63d934c
Depends-On: Iaa67061b05a9d355228ad7d3f5ee0f4f04dbdc66
Signed-off-by: Daniel Badea <daniel.badea@windriver.com>
2019-07-23 16:57:08 +03:00
Oleh Hryhorov
e0b6f6b9c9 Adding dependency in cinder for keystone-api
The patch adds dependency in cinder for create_internal_tenant job which
uses openstack client to create internal_project_name and internal_user_name
but if identity server is not ready yet the job crashes therefore cinder-volume
doesn't work as well.

Change-Id: I4386f127b834a9777093ac1d3c269937947c7bcf
2019-07-15 16:48:28 +03:00
pd2839
77dfdc99c9 Using HTK for cinder security policies
Overriding the values in values.yaml

Change-Id: Ib7cd343b4b95cae088a4fe564524448af79de785
2019-07-08 18:00:59 +00:00
Pete Birley
a750802331 Cinder: Require cinder volume to be running before attempting bootstrap
This PS udpates the deps on the cinder boostrap job to ensure that the
cinder volume service has started prior to attempting bootstrap.

This crtainly could be enhanced further, but is the 1st step in preventing
the liklyhood of a race.

Change-Id: Id0f958077b296750242450179b41c0a1b703b4a5
Signed-off-by: Pete Birley <pete@port.direct>
2019-07-05 11:52:53 -05:00
Zuul
8ec7e3208a Merge "Cinder: Support backup driver specification by module or class name" 2019-06-19 05:28:04 +00:00
Pete Birley
9bcf0df94c Messaging: use htk function to directly hit RabbitMQ servers
This PS updates the charts to use the htk function recently introduced
to allow oslo.messaging clients ans servers to directly hit their
backends rather than using either DNS or K8S svc based routing.

Depends-On: I5150a64bd29fa062e30496c1f2127de138322863

Change-Id: I458b4313c57fc50c8181cedeca9919670487926a
Signed-off-by: Pete Birley <pete@port.direct>
2019-06-18 21:47:45 +00:00
Zuul
cd460f12c2 Merge "Rafactoring volume mount variables in db sync job" 2019-06-18 18:24:18 +00:00
Robert Church
aa4ca5ad56 Cinder: Support backup driver specification by module or class name
During the Queens cycle, Cinder introduced the ability to specify the
backup driver via class name and deprecated backup driver initialization
using the module name. (Id6bee9e7d0da8ead224a04f86fe79ddfb5b286cf)

Legacy support for initialization by module name was dropped in Stein.
(I3ada2dee1857074746b1893b82dd5f6641c6e579)

This change will support both methods of initialization and leave the
driver defaults enabled for module based initialization (valid through
Rocky images).

This change has been tested using the OSH default Cinder (Ocata) images
and StarlingX images based on master (Train).

Change-Id: Iec7bc6f4dd089aaa08ca652bebd9a10ef49da556
Signed-off-by: Robert Church <robert.church@windriver.com>
2019-06-18 13:21:35 -04:00
Pete Birley
d0b135cd77 AMPQ: update ha policy regex
Change-Id: I2f023c2e41a52b5753cdb77e93c9e876bc60a87d
Signed-off-by: Pete Birley <pete@port.direct>
2019-06-11 12:17:22 -05:00
Gage Hugo
976cab856c Create separate users for helm test
Currently each service uses the same name for their helm test user,
"test". While this works when services are ran sequentially, when
multiple services are deployed and tested at the same time, it can
lead to a race condition where one service deletes the user before
the other is done testing, causing a failure.

This change makes it so that each service defines its own test user
in the form of [service]-test.

Change-Id: Idd7ad3bef78a039f23fb0dd79391e3588e94b73c
2019-06-03 11:26:18 -05:00
Itxaka
c38443de4c cinder: allow configuring the rbd app name
Instead of hardcoding it, let us override it with
custom values for normal volumes and backups

Change-Id: I3abb343877abd0436c592a3371372f82ef581790
2019-05-28 14:34:20 +02:00
John Haan
0ea9be7ade Rafactoring volume mount variables in db sync job
This patch make the db sync job template follows the same pattern
that other templates utilize the variables to make in a predictable
pattern.

Change-Id: Idbedd046c6b4fd001cf63004ffac792173a5778b
Story: 2005754
Task: 33457
2019-05-22 17:47:03 +09:00
Zuul
f8adab245b Merge "Point to OSH-images images" 2019-05-18 19:12:58 +00:00
Jean-Philippe Evrard
1d335146fa Point to OSH-images images
We now have a process for OSH-images image building,
using Zuul, so we should point the images by default to those
images, instead of pointing to stale images.

Without this, the osh-images build process is completely not
in use, and updating the osh-images process or patching its
code has no impact on OSH.

This should fix it.

Change-Id: I672b8755bf9e182b15eff067479b662529a13477
2019-05-13 10:58:02 +02:00
Roy Tang (rt7380)
5df6fa3789 Expose Anti-Affinity Weight Setting.
Add weight default setting to anti-affinity.

Depends-on: Id8eb303674764ef8b0664f62040723aaf77e0a54
Change-Id: I09f96522cddf3a77dae73daca4557877eda5df50
2019-05-10 22:05:24 -05:00
jh629g
0afa508f06 Fix results of openstack volume type list --default
Currently openstack volume type list --default
returns 404 if the openstack environment is
deployed using openstack-helm. The value
default_volume_type was previously unimplemented
in values.yaml for cinder.

Change-Id: Ifb6d54d875d5cbaffc8330d15d999ef830010177
2019-04-20 03:36:09 +00:00
Zuul
db87cf6390 Merge "Add internal tenant id in conf" 2019-04-17 05:59:23 +00:00
Liang Fang
d1c8e778a7 Add internal tenant id in conf
Cinder raw cache feature requires internal tenant id be set in
/etc/cinder/cinder.conf, something like:

cinder_internal_tenant_project_id = b7455b8974bb4064ad247c8f375eae6c
cinder_internal_tenant_user_id = f46924c112a14c80ab0a24a613d95eef

This patch get or create if not exist intenal user id and project id, and then
set in cinder.conf

reference: Cinder cache feature:
https://docs.openstack.org/cinder/latest/admin/blockstorage-image-volume-cache.html

Story: 2004869
Task: 29121
Change-Id: I07954d2efa905a56ca8482d0ec147534c97d01ea
Signed-off-by: Liang Fang <liang.a.fang@intel.com>
2019-04-16 16:09:31 +00:00
Gage Hugo
2f46c057a4 Enable audit pipeline for cinder
This change adds the keystonemiddleware audit paste filter[0]
and enables it for the cinder-api and cinder-scheduler services.
This provides the ability to audit API requests for cinder.

[0] https://docs.openstack.org/keystonemiddleware/latest/audit.html

Change-Id: If81b88a4003bc4394ef4a378626cf5d6edb9c4ae
2019-04-11 18:57:18 +00:00
Pete Birley
d83afc8f91 Cinder: make rabbitmq queues ha
This PS tells cinder to make rabbitmq queues ha when available.

Change-Id: I680e14b2f48ab7d50b8688d29d690c92fc33d696
Signed-off-by: Pete Birley <pete@port.direct>
2019-03-29 20:12:05 +00:00
Zuul
ee477d4562 Merge "Implement Security Context for Cinder" 2019-03-22 21:18:29 +00:00
pd2839
ba593e1a6b Implement Security Context for Cinder
Implement container security context for the following Cinder resources:
 - Cinder server deployment

Change-Id: Ic319fc8ccfea4c8d640ceecd0bbc93912173d172
2019-03-15 21:46:54 +00:00
Jiří Suchomel
a2a5dda47c Added volume related keys to pod mounts to ease the overriding
If user wants to add an extra volumeMounts/volume to a pod,
amd uses override values e.g. like this

pod:
  mounts:
    nova_placement:
      init_container: null
      nova_placement:
        volumeMounts:
          - name: nova-etc
          ...

helm template parser complains with

Warning: The destination item 'nova_placement' is a table and ignoring the source 'nova_placement' as it has a non-table value of: <nil>

So when we create empty values for such keys in values.yaml, the source
will be present and warning does not need to be shown.

Change-Id: Ib8dc53c3a54e12014025de8fafe16fbe9721c0da
2019-03-15 16:29:19 +00:00
Robert Church
d1fbf4ac14 Add resource_filters.json to Cinder's configmap
During the Stein development cycle, Cinder removed the deprecated
query_volume_filters configuration option with
Icd311db7f88c3c274d9a362eb96519e46c7e4d17.

This chart update will add resource_filters.json to the configmap and
provides the default values for the filter keys to enable filtering in
the list APIs.

Change-Id: I31263e9ce06d31773e961ae5d1252e062a38a4e5
Signed-off-by: Robert Church <robert.church@windriver.com>
2019-03-06 04:34:53 +00:00
Zuul
1ad5467252 Merge "Increase default logging" 2019-03-06 04:09:58 +00:00
Jean-Philippe Evrard
5890ebf4f8 Increase default logging
The current helm chart defaults drops logs of any warnings
(and above) for any logger outside of the namespace
of the deployed chart.

This is a problem, as logging could reveal important information for
operators. While this could be done with a value override, there
is no reason to hide warning, errors, or critical information that
are happening in the cycle of the operation of the software
deployed with the helm charts. For example, nothing would get
logged in oslo_service, which is a very important part of running
OpenStack.

This fixes it by logging to stdout all the warnings (and above)
for OpenStack apps.

Change-Id: I16f77f4cc64caf21b21c8519e6da34eaf5d31498
2019-02-28 09:53:01 +00:00
Zuul
66ba0be81c Merge "Explicitly set datefmt for logging" 2019-02-27 01:31:09 +00:00
Zuul
c869b4ef4a Merge "Add startingDeadlineSeconds field to cronJobs" 2019-02-23 21:18:26 +00:00
sg774j
93658c75ec enable_proxy_headers_parsing to various services.
Adding this parameter to Cinder, Heat, Glance,and Neutron
config. Adding this parameter allows proper handling to resource
links in response using API services behind https proxy.

Change-Id: Ib99a16b6252b15d9f138417485731ec401cb8f81
2019-02-21 15:01:45 +00:00
Pavlo Shchelokovskyy
55645c7e73 Explicitly set datefmt for logging
the defaults in Python [0] and oslo.log [1] are such that when using
separate config file for logging configuration (log-config-append)
the log fomat of dates containes miliseconds twice (as in sec,ms.ms)
which is exactly what is currently seen in logs of OpenStack services
deployed by openstack-helm.

When not provided with datefmt log formatter option, Python effectively
uses '%Y-%m-%d %H:%M:%S,%f' [0] as a default time formatting string to
render `%(asctime)s`, but the defaults in oslo.log add another `.%f`
to it [1].

Since `log-date-format` oslo.log option has no effect when using
log-config-append, we need to explicitly set date format to avoid double
miliseconds rendering in date of log entries.

[0] 6ee41793d2/Lib/logging/__init__.py (L427-L428)
[1] http://git.openstack.org/cgit/openstack/oslo.log/tree/oslo_log/_options.py?id=7c5f8362b26313217b6c248e77be3dc8e2ef74a5#n148

Change-Id: I47aa7ce96770d94b905b56d6fe4abad428f01047
2019-02-21 08:28:35 +00:00
Dmitrii Kabanov
34a092a7f4 Add startingDeadlineSeconds field to cronJobs
This patch set adds "startingDeadlineSeconds" field to cronJobs.
When the field is not set, the controller counts how many missed
jobs occured from the last scheduled time till now. And if it happends
more than 100 time the job will not be scheduled. To avoid this
the "startingDeadlineSeconds" field should be set to sufficient period
of time. In this case the controller counts how many missed jobs occured
during this period of time. The value of the field should be less than
time (in seconds) needed for running >100 jobs (according to schedule).

Change-Id: I3bf7c7077b55ca5a3421052bd0b59b70c9bbcf24
2019-02-20 04:11:05 +00:00
Pete Birley
23f5b18f1c Cinder: Update auth_strategy to explictly specify keystone
In accordance with the OpenStack Security Guide this PS updates
the cinder.conf to explicity set the auth_strategy param.

Change-Id: Ie0a2b9ffebb597166851226eabac4924c34e1404
Signed-off-by: Pete Birley <pete@port.direct>
2019-01-31 15:09:29 -06:00
Jaesang Lee
2a03fd65bf Upgrade rally to 1.3.0
Since rally 1.0, rally has been a platform for testing, and rally for
openstack has been separated by rally-openstack. The current version
of rally in openstack-helm is version 0.8 which corresponds to ocata.
This patch tests with the latest version of rally-openstack, version
1.3.0, and removes scenarios that are no longer in use.

Change-Id: I380a976c0f48c4af0796c9d866fc8787025ce548
2018-12-20 00:22:49 +00:00
Mike Pham
95a6a2e875 Enable Egress policy enforcement
This PS is enable the Egress policies
and enforces them in Openstack-helm.

Depends-On: Icbe2a18c98dba795d15398dcdcac64228f6a7b4c
Change-Id: I6ef3cd157749fd562acb2f89ad44e63be4f7e975
2018-12-17 14:38:29 +00:00
Tin Lam
29f32a07ac Enable network policy enforcement
This patch set updates the gate to by default uses network policy
for all components and enforces them in Openstack-helm.

Change-Id: I70c90b5808075797f02670f21481a4f968205325
Depends-On: I78e87ef3276e948ae4dd2eb462b4b8012251c8c8
Co-Authored-By: Mike Pham <tp6510@att.com>
Signed-off-by: Tin Lam <tin@irrational.io>
2018-10-23 14:58:13 +00:00
Matthew Heler
80e0dfc1b9 Set report_discard_supported to true for the RBD Cinder backend.
This enables Ceph discard/trim support with cinder volumes.
Note: that you still need to use a paravirtualized storage backend that 
supports discard/trim (virtio-scsi for example).

Change-Id: I7d564ac22ee8faedda0336ee5c67f8312c9f7d06
2018-10-19 15:28:10 +00:00