scottk/kubernetes
1
0
Fork 0
mirror of https://github.com/optim-enterprises-bv/kubernetes.git synced 2025-11-04 04:08:16 +00:00
Code Issues Packages Projects Releases Wiki Activity

Merge pull request #13025 from pires/example_elasticsearch

Browse Source 
Revamped Elasticsearch example
This commit is contained in:
Quinton Hoole
2015-09-04 12:28:17 -07:00
parent 80cda53b9f 0a64995b7b
commit e4fbfa9777
21 changed files with 618 additions and 873 deletions
Download Patch File Download Diff File Expand all files Collapse all files

18
examples/elasticsearch/Dockerfile
View File

@@ -1,18 +0,0 @@
FROM java:7-jre
RUN apt-get update && \
apt-get install -y curl && \
apt-get clean
RUN cd / && \
curl -O https://download.elastic.co/elasticsearch/elasticsearch/elasticsearch-1.5.2.tar.gz && \
tar xf elasticsearch-1.5.2.tar.gz && \
rm elasticsearch-1.5.2.tar.gz
COPY elasticsearch.yml /elasticsearch-1.5.2/config/elasticsearch.yml
COPY run.sh /
COPY elasticsearch_discovery /
EXPOSE 9200 9300
CMD ["/run.sh"]

14
examples/elasticsearch/Makefile
View File

@@ -1,14 +0,0 @@
.PHONY: elasticsearch_discovery build push all
TAG = 1.2
build: elasticsearch_discovery
docker build -t kubernetes/elasticsearch:$(TAG) .
push:
docker push kubernetes/elasticsearch:$(TAG)
elasticsearch_discovery:
go build elasticsearch_discovery.go
all: elasticsearch_discovery build push

415
examples/elasticsearch/README.md
View File

@@ -33,206 +33,130 @@ Documentation for other releases can be found at
# Elasticsearch for Kubernetes # Elasticsearch for Kubernetes
This directory contains the source for a Docker image that creates an instance Kubernetes makes it trivial for anyone to easily build and scale [Elasticsearch](http://www.elasticsearch.org/) clusters. Here, you'll find how to do so.
of [Elasticsearch](https://www.elastic.co/products/elasticsearch) 1.5.2 which can Current Elasticsearch version is `1.7.1`.
be used to automatically form clusters when used
with [replication controllers](../../docs/user-guide/replication-controller.md). This will not work with the library Elasticsearch image
because multicast discovery will not find the other pod IPs needed to form a cluster. This
image detects other Elasticsearch [pods](../../docs/user-guide/pods.md) running in a specified [namespace](../../docs/user-guide/namespaces.md) with a given
label selector. The detected instances are used to form a list of peer hosts which
are used as part of the unicast discovery mechanism for Elasticsearch. The detection
of the peer nodes is done by a program which communicates with the Kubernetes API
server to get a list of matching Elasticsearch pods.
Here is an example replication controller specification that creates 4 instances of Elasticsearch. [A more robust example that follows Elasticsearch best-practices of separating nodes concern is also available](production_cluster/README.md).
<!-- BEGIN MUNGE: EXAMPLE music-rc.yaml --> <img src="http://kubernetes.io/img/warning.png" alt="WARNING" width="25" height="25"> Current pod descriptors use an `emptyDir` for storing data in each data node container. This is meant to be for the sake of simplicity and [should be adapted according to your storage needs](../../docs/design/persistent-storage.md).
```yaml ## Docker image
apiVersion: v1
kind: ReplicationController This example uses [this pre-built image](https://github.com/pires/docker-elasticsearch-kubernetes) will not be supported. Feel free to fork to fit your own needs, but mind yourself that you will need to change Kubernetes descriptors accordingly.
metadata:
labels: ## Deploy
name: music-db
namespace: mytunes Let's kickstart our cluster with 1 instance of Elasticsearch.
name: music-db
spec: ```
replicas: 4 kubectl create -f examples/elasticsearch/service-account.yaml
selector: kubectl create -f examples/elasticsearch/es-svc.yaml
name: music-db kubectl create -f examples/elasticsearch/es-rc.yaml
template:
metadata:
labels:
name: music-db
spec:
containers:
- name: es
image: kubernetes/elasticsearch:1.2
env:
- name: "CLUSTER_NAME"
value: "mytunes-db"
- name: "SELECTOR"
value: "name=music-db"
- name: "NAMESPACE"
value: "mytunes"
ports:
- name: es
containerPort: 9200
- name: es-transport
containerPort: 9300
``` ```
[Download example](music-rc.yaml) Let's see if it worked:
<!-- END MUNGE: EXAMPLE music-rc.yaml -->
The `CLUSTER_NAME` variable gives a name to the cluster and allows multiple separate clusters to
exist in the same namespace.
The `SELECTOR` variable should be set to a label query that identifies the Elasticsearch
nodes that should participate in this cluster. For our example we specify `name=music-db` to
match all pods that have the label `name` set to the value `music-db`.
The `NAMESPACE` variable identifies the namespace
to be used to search for Elasticsearch pods and this should be the same as the namespace specified
for the replication controller (in this case `mytunes`).
Replace `NAMESPACE` with the actual namespace to be used. In this example we shall use
the namespace `mytunes`.
```yaml
kind: Namespace
apiVersion: v1
metadata:
name: mytunes
labels:
name: mytunes
``` ```
$ kubectl get pods
First, let's create the namespace:
```console
$ kubectl create -f examples/elasticsearch/mytunes-namespace.yaml
namespaces/mytunes
```
Now you are ready to create the replication controller which will then create the pods:
```console
$ kubectl create -f examples/elasticsearch/music-rc.yaml --namespace=mytunes
replicationcontrollers/music-db
```
Let's check to see if the replication controller and pods are running:
```console
$ kubectl get rc,pods --namespace=mytunes
CONTROLLER CONTAINER(S) IMAGE(S) SELECTOR REPLICAS
music-db es kubernetes/elasticsearch:1.2 name=music-db 4
NAME READY STATUS RESTARTS AGE NAME READY STATUS RESTARTS AGE
music-db-5p46b 1/1 Running 0 34s es-kfymw 1/1 Running 0 7m
music-db-8re0f 1/1 Running 0 34s kube-dns-p3v1u 3/3 Running 0 19m
music-db-eq8j0 1/1 Running 0 34s
music-db-uq5px 1/1 Running 0 34s
``` ```
It's also useful to have a [service](../../docs/user-guide/services.md) with an load balancer for accessing the Elasticsearch ```
cluster. $ kubectl logs es-kfymw
log4j:WARN No such property [maxBackupIndex] in org.apache.log4j.DailyRollingFileAppender.
<!-- BEGIN MUNGE: EXAMPLE music-service.yaml --> log4j:WARN No such property [maxBackupIndex] in org.apache.log4j.DailyRollingFileAppender.
log4j:WARN No such property [maxBackupIndex] in org.apache.log4j.DailyRollingFileAppender.
```yaml [2015-08-30 10:01:31,946][INFO ][node ] [Hammerhead] version[1.7.1], pid[7], build[b88f43f/2015-07-29T09:54:16Z]
apiVersion: v1 [2015-08-30 10:01:31,946][INFO ][node ] [Hammerhead] initializing ...
kind: Service [2015-08-30 10:01:32,110][INFO ][plugins ] [Hammerhead] loaded [cloud-kubernetes], sites []
metadata: [2015-08-30 10:01:32,153][INFO ][env ] [Hammerhead] using [1] data paths, mounts [[/data (/dev/sda9)]], net usable_space [14.4gb], net total_space [15.5gb], types [ext4]
name: music-server [2015-08-30 10:01:37,188][INFO ][node ] [Hammerhead] initialized
namespace: mytunes [2015-08-30 10:01:37,189][INFO ][node ] [Hammerhead] starting ...
labels: [2015-08-30 10:01:37,499][INFO ][transport ] [Hammerhead] bound_address {inet[/0:0:0:0:0:0:0:0:9300]}, publish_address {inet[/10.244.48.2:9300]}
name: music-db [2015-08-30 10:01:37,550][INFO ][discovery ] [Hammerhead] myesdb/n2-6uu_UT3W5XNrjyqBPiA
spec: [2015-08-30 10:01:43,966][INFO ][cluster.service ] [Hammerhead] new_master [Hammerhead][n2-6uu_UT3W5XNrjyqBPiA][es-kfymw][inet[/10.244.48.2:9300]]{master=true}, reason: zen-disco-join (elected_as_master)
selector: [2015-08-30 10:01:44,010][INFO ][http ] [Hammerhead] bound_address {inet[/0:0:0:0:0:0:0:0:9200]}, publish_address {inet[/10.244.48.2:9200]}
name: music-db [2015-08-30 10:01:44,011][INFO ][node ] [Hammerhead] started
ports: [2015-08-30 10:01:44,042][INFO ][gateway ] [Hammerhead] recovered [0] indices into cluster_state
- name: db
port: 9200
targetPort: es
type: LoadBalancer
``` ```
[Download example](music-service.yaml) So we have a 1-node Elasticsearch cluster ready to handle some work.
<!-- END MUNGE: EXAMPLE music-service.yaml -->
Let's create the service with an external load balancer: ## Scale
```console Scaling is as easy as:
$ kubectl create -f examples/elasticsearch/music-service.yaml --namespace=mytunes
services/music-server
```
Let's check the status of the service:
```console
$ kubectl get service --namespace=mytunes
NAME LABELS SELECTOR IP(S) PORT(S)
music-server name=music-db name=music-db 10.0.185.179 9200/TCP
``` ```
kubectl scale --replicas=3 rc es
Although this service has an IP address `10.0.185.179` internal to the cluster we don't yet have
an external IP address provisioned. Let's wait a bit and try again...
```console
$ kubectl get service --namespace=mytunes
NAME LABELS SELECTOR IP(S) PORT(S)
music-server name=music-db name=music-db 10.0.185.179 9200/TCP
104.197.114.130
``` ```
Now we have an external IP address `104.197.114.130` available for accessing the service Did it work?
from outside the cluster.
Let's see what we've got: ```
$ kubectl get pods
```console
$ kubectl get pods,rc,services --namespace=mytunes
NAME READY STATUS RESTARTS AGE NAME READY STATUS RESTARTS AGE
music-db-5p46b 1/1 Running 0 7m es-78e0s 1/1 Running 0 8m
music-db-8re0f 1/1 Running 0 7m es-kfymw 1/1 Running 0 17m
music-db-eq8j0 1/1 Running 0 7m es-rjmer 1/1 Running 0 8m
music-db-uq5px 1/1 Running 0 7m kube-dns-p3v1u 3/3 Running 0 30m
CONTROLLER CONTAINER(S) IMAGE(S) SELECTOR REPLICAS
music-db es kubernetes/elasticsearch:1.2 name=music-db 4
NAME LABELS SELECTOR IP(S) PORT(S)
music-server name=music-db name=music-db 10.0.185.179 9200/TCP
104.197.114.130
NAME TYPE DATA
default-token-gcilu kubernetes.io/service-account-token 2
``` ```
This shows 4 instances of Elasticsearch running. After making sure that port 9200 is accessible for this cluster (e.g. using a firewall rule for Google Compute Engine) we can make queries via the service which will be fielded by the matching Elasticsearch pods. Let's take a look at logs:
```console ```
$ curl 104.197.114.130:9200 $ kubectl logs es-kfymw
log4j:WARN No such property [maxBackupIndex] in org.apache.log4j.DailyRollingFileAppender.
log4j:WARN No such property [maxBackupIndex] in org.apache.log4j.DailyRollingFileAppender.
log4j:WARN No such property [maxBackupIndex] in org.apache.log4j.DailyRollingFileAppender.
[2015-08-30 10:01:31,946][INFO ][node ] [Hammerhead] version[1.7.1], pid[7], build[b88f43f/2015-07-29T09:54:16Z]
[2015-08-30 10:01:31,946][INFO ][node ] [Hammerhead] initializing ...
[2015-08-30 10:01:32,110][INFO ][plugins ] [Hammerhead] loaded [cloud-kubernetes], sites []
[2015-08-30 10:01:32,153][INFO ][env ] [Hammerhead] using [1] data paths, mounts [[/data (/dev/sda9)]], net usable_space [14.4gb], net total_space [15.5gb], types [ext4]
[2015-08-30 10:01:37,188][INFO ][node ] [Hammerhead] initialized
[2015-08-30 10:01:37,189][INFO ][node ] [Hammerhead] starting ...
[2015-08-30 10:01:37,499][INFO ][transport ] [Hammerhead] bound_address {inet[/0:0:0:0:0:0:0:0:9300]}, publish_address {inet[/10.244.48.2:9300]}
[2015-08-30 10:01:37,550][INFO ][discovery ] [Hammerhead] myesdb/n2-6uu_UT3W5XNrjyqBPiA
[2015-08-30 10:01:43,966][INFO ][cluster.service ] [Hammerhead] new_master [Hammerhead][n2-6uu_UT3W5XNrjyqBPiA][es-kfymw][inet[/10.244.48.2:9300]]{master=true}, reason: zen-disco-join (elected_as_master)
[2015-08-30 10:01:44,010][INFO ][http ] [Hammerhead] bound_address {inet[/0:0:0:0:0:0:0:0:9200]}, publish_address {inet[/10.244.48.2:9200]}
[2015-08-30 10:01:44,011][INFO ][node ] [Hammerhead] started
[2015-08-30 10:01:44,042][INFO ][gateway ] [Hammerhead] recovered [0] indices into cluster_state
[2015-08-30 10:08:02,517][INFO ][cluster.service ] [Hammerhead] added {[Tenpin][2gv5MiwhRiOSsrTOF3DhuA][es-78e0s][inet[/10.244.54.4:9300]]{master=true},}, reason: zen-disco-receive(join from node[[Tenpin][2gv5MiwhRiOSsrTOF3DhuA][es-78e0s][inet[/10.244.54.4:9300]]{master=true}])
[2015-08-30 10:10:10,645][INFO ][cluster.service ] [Hammerhead] added {[Evilhawk][ziTq2PzYRJys43rNL2tbyg][es-rjmer][inet[/10.244.33.3:9300]]{master=true},}, reason: zen-disco-receive(join from node[[Evilhawk][ziTq2PzYRJys43rNL2tbyg][es-rjmer][inet[/10.244.33.3:9300]]{master=true}])
```
So we have a 3-node Elasticsearch cluster ready to handle more work.
## Access the service
*Don't forget* that services in Kubernetes are only acessible from containers in the cluster. For different behavior you should [configure the creation of an external load-balancer](http://kubernetes.io/v1.0/docs/user-guide/services.html#type-loadbalancer). While it's supported within this example service descriptor, its usage is out of scope of this document, for now.
```
$ kubectl get service elasticsearch
NAME LABELS SELECTOR IP(S) PORT(S)
elasticsearch component=elasticsearch component=elasticsearch 10.100.108.94 9200/TCP
9300/TCP
```
From any host on your cluster (that's running `kube-proxy`), run:
```
$ curl 10.100.108.94:9200
```
You should see something similar to the following:
```json
{ {
"status" : 200, "status" : 200,
"name" : "Warpath", "name" : "Hammerhead",
"cluster_name" : "mytunes-db", "cluster_name" : "myesdb",
"version" : { "version" : {
"number" : "1.5.2", "number" : "1.7.1",
"build_hash" : "62ff9868b4c8a0c45860bebb259e21980778ab1c", "build_hash" : "b88f43fc40b0bcd7f173a1f9ee2e97816de80b19",
"build_timestamp" : "2015-04-27T09:21:06Z", "build_timestamp" : "2015-07-29T09:54:16Z",
"build_snapshot" : false,
"lucene_version" : "4.10.4"
},
"tagline" : "You Know, for Search"
}
$ curl 104.197.114.130:9200
{
"status" : 200,
"name" : "Callisto",
"cluster_name" : "mytunes-db",
"version" : {
"number" : "1.5.2",
"build_hash" : "62ff9868b4c8a0c45860bebb259e21980778ab1c",
"build_timestamp" : "2015-04-27T09:21:06Z",
"build_snapshot" : false, "build_snapshot" : false,
"lucene_version" : "4.10.4" "lucene_version" : "4.10.4"
}, },
@@ -240,128 +164,33 @@ $ curl 104.197.114.130:9200
} }
``` ```
We can query the nodes to confirm that an Elasticsearch cluster has been formed. Or if you want to check cluster information:
```console
$ curl 104.197.114.130:9200/_nodes?pretty=true ```
curl 10.100.108.94:9200/_cluster/health?pretty
```
You should see something similar to the following:
```json
{ {
"cluster_name" : "mytunes-db", "cluster_name" : "myesdb",
"nodes" : { "status" : "green",
"u-KrvywFQmyaH5BulSclsA" : { "timed_out" : false,
"name" : "Jonas Harrow", "number_of_nodes" : 3,
... "number_of_data_nodes" : 3,
"discovery" : { "active_primary_shards" : 0,
"zen" : { "active_shards" : 0,
"ping" : { "relocating_shards" : 0,
"unicast" : { "initializing_shards" : 0,
"hosts" : [ "10.244.2.48", "10.244.0.24", "10.244.3.31", "10.244.1.37" ] "unassigned_shards" : 0,
}, "delayed_unassigned_shards" : 0,
... "number_of_pending_tasks" : 0,
"name" : "Warpath", "number_of_in_flight_fetch" : 0
... }
"discovery" : {
"zen" : {
"ping" : {
"unicast" : {
"hosts" : [ "10.244.2.48", "10.244.0.24", "10.244.3.31", "10.244.1.37" ]
},
...
"name" : "Callisto",
...
"discovery" : {
"zen" : {
"ping" : {
"unicast" : {
"hosts" : [ "10.244.2.48", "10.244.0.24", "10.244.3.31", "10.244.1.37" ]
},
...
"name" : "Vapor",
...
"discovery" : {
"zen" : {
"ping" : {
"unicast" : {
"hosts" : [ "10.244.2.48", "10.244.0.24", "10.244.3.31", "10.244.1.37" ]
...
``` ```
Let's ramp up the number of Elasticsearch nodes from 4 to 10:
```console
$ kubectl scale --replicas=10 replicationcontrollers music-db --namespace=mytunes
scaled
$ kubectl get pods --namespace=mytunes
NAME READY STATUS RESTARTS AGE
music-db-0n8rm 0/1 Running 0 9s
music-db-4izba 1/1 Running 0 9s
music-db-5dqes 0/1 Running 0 9s
music-db-5p46b 1/1 Running 0 10m
music-db-8re0f 1/1 Running 0 10m
music-db-eq8j0 1/1 Running 0 10m
music-db-p9ajw 0/1 Running 0 9s
music-db-p9u1k 1/1 Running 0 9s
music-db-rav1q 0/1 Running 0 9s
music-db-uq5px 1/1 Running 0 10m
```
Let's check to make sure that these 10 nodes are part of the same Elasticsearch cluster:
```console
$ curl 104.197.114.130:9200/_nodes?pretty=true | grep name
"cluster_name" : "mytunes-db",
"name" : "Killraven",
"name" : "Killraven",
"name" : "mytunes-db"
"vm_name" : "OpenJDK 64-Bit Server VM",
"name" : "eth0",
"name" : "Tefral the Surveyor",
"name" : "Tefral the Surveyor",
"name" : "mytunes-db"
"vm_name" : "OpenJDK 64-Bit Server VM",
"name" : "eth0",
"name" : "Jonas Harrow",
"name" : "Jonas Harrow",
"name" : "mytunes-db"
"vm_name" : "OpenJDK 64-Bit Server VM",
"name" : "eth0",
"name" : "Warpath",
"name" : "Warpath",
"name" : "mytunes-db"
"vm_name" : "OpenJDK 64-Bit Server VM",
"name" : "eth0",
"name" : "Brute I",
"name" : "Brute I",
"name" : "mytunes-db"
"vm_name" : "OpenJDK 64-Bit Server VM",
"name" : "eth0",
"name" : "Callisto",
"name" : "Callisto",
"name" : "mytunes-db"
"vm_name" : "OpenJDK 64-Bit Server VM",
"name" : "eth0",
"name" : "Vapor",
"name" : "Vapor",
"name" : "mytunes-db"
"vm_name" : "OpenJDK 64-Bit Server VM",
"name" : "eth0",
"name" : "Timeslip",
"name" : "Timeslip",
"name" : "mytunes-db"
"vm_name" : "OpenJDK 64-Bit Server VM",
"name" : "eth0",
"name" : "Magik",
"name" : "Magik",
"name" : "mytunes-db"
"vm_name" : "OpenJDK 64-Bit Server VM",
"name" : "eth0",
"name" : "Brother Voodoo",
"name" : "Brother Voodoo",
"name" : "mytunes-db"
"vm_name" : "OpenJDK 64-Bit Server VM",
"name" : "eth0",
```
<!-- BEGIN MUNGE: GENERATED_ANALYTICS --> <!-- BEGIN MUNGE: GENERATED_ANALYTICS -->
[![Analytics](https://kubernetes-site.appspot.com/UA-36037335-10/GitHub/examples/elasticsearch/README.md?pixel)]() [![Analytics](https://kubernetes-site.appspot.com/UA-36037335-10/GitHub/examples/elasticsearch/README.md?pixel)]()
<!-- END MUNGE: GENERATED_ANALYTICS --> <!-- END MUNGE: GENERATED_ANALYTICS -->

385
examples/elasticsearch/elasticsearch.yml
View File

@@ -1,385 +0,0 @@
##################### Elasticsearch Configuration Example #####################
# This file contains an overview of various configuration settings,
# targeted at operations staff. Application developers should
# consult the guide at <http://elasticsearch.org/guide>.
#
# The installation procedure is covered at
# <http://elasticsearch.org/guide/en/elasticsearch/reference/current/setup.html>.
#
# Elasticsearch comes with reasonable defaults for most settings,
# so you can try it out without bothering with configuration.
#
# Most of the time, these defaults are just fine for running a production
# cluster. If you're fine-tuning your cluster, or wondering about the
# effect of certain configuration option, please _do ask_ on the
# mailing list or IRC channel [http://elasticsearch.org/community].
# Any element in the configuration can be replaced with environment variables
# by placing them in ${...} notation. For example:
#
#node.rack: ${RACK_ENV_VAR}
# For information on supported formats and syntax for the config file, see
# <http://elasticsearch.org/guide/en/elasticsearch/reference/current/setup-configuration.html>
################################### Cluster ###################################
# Cluster name identifies your cluster for auto-discovery. If you're running
# multiple clusters on the same network, make sure you're using unique names.
#
cluster.name: ${CLUSTER_NAME}
#################################### Node #####################################
# Node names are generated dynamically on startup, so you're relieved
# from configuring them manually. You can tie this node to a specific name:
#
#node.name: "Franz Kafka"
# Every node can be configured to allow or deny being eligible as the master,
# and to allow or deny to store the data.
#
# Allow this node to be eligible as a master node (enabled by default):
#
node.master: ${NODE_MASTER}
#
# Allow this node to store data (enabled by default):
#
node.data: ${NODE_DATA}
# You can exploit these settings to design advanced cluster topologies.
#
# 1. You want this node to never become a master node, only to hold data.
# This will be the "workhorse" of your cluster.
#
#node.master: false
#node.data: true
#
# 2. You want this node to only serve as a master: to not store any data and
# to have free resources. This will be the "coordinator" of your cluster.
#
#node.master: true
#node.data: false
#
# 3. You want this node to be neither master nor data node, but
# to act as a "search load balancer" (fetching data from nodes,
# aggregating results, etc.)
#
#node.master: false
#node.data: false
# Use the Cluster Health API [http://localhost:9200/_cluster/health], the
# Node Info API [http://localhost:9200/_nodes] or GUI tools
# such as <http://www.elasticsearch.org/overview/marvel/>,
# <http://github.com/karmi/elasticsearch-paramedic>,
# <http://github.com/lukas-vlcek/bigdesk> and
# <http://mobz.github.com/elasticsearch-head> to inspect the cluster state.
# A node can have generic attributes associated with it, which can later be used
# for customized shard allocation filtering, or allocation awareness. An attribute
# is a simple key value pair, similar to node.key: value, here is an example:
#
#node.rack: rack314
# By default, multiple nodes are allowed to start from the same installation location
# to disable it, set the following:
#node.max_local_storage_nodes: 1
#################################### Index ####################################
# You can set a number of options (such as shard/replica options, mapping
# or analyzer definitions, translog settings, ...) for indices globally,
# in this file.
#
# Note, that it makes more sense to configure index settings specifically for
# a certain index, either when creating it or by using the index templates API.
#
# See <http://elasticsearch.org/guide/en/elasticsearch/reference/current/index-modules.html> and
# <http://elasticsearch.org/guide/en/elasticsearch/reference/current/indices-create-index.html>
# for more information.
# Set the number of shards (splits) of an index (5 by default):
#
#index.number_of_shards: 5
# Set the number of replicas (additional copies) of an index (1 by default):
#
#index.number_of_replicas: 1
# Note, that for development on a local machine, with small indices, it usually
# makes sense to "disable" the distributed features:
#
#index.number_of_shards: 1
#index.number_of_replicas: 0
# These settings directly affect the performance of index and search operations
# in your cluster. Assuming you have enough machines to hold shards and
# replicas, the rule of thumb is:
#
# 1. Having more *shards* enhances the _indexing_ performance and allows to
# _distribute_ a big index across machines.
# 2. Having more *replicas* enhances the _search_ performance and improves the
# cluster _availability_.
#
# The "number_of_shards" is a one-time setting for an index.
#
# The "number_of_replicas" can be increased or decreased anytime,
# by using the Index Update Settings API.
#
# Elasticsearch takes care about load balancing, relocating, gathering the
# results from nodes, etc. Experiment with different settings to fine-tune
# your setup.
# Use the Index Status API (<http://localhost:9200/A/_status>) to inspect
# the index status.
#################################### Paths ####################################
# Path to directory containing configuration (this file and logging.yml):
#
#path.conf: /path/to/conf
# Path to directory where to store index data allocated for this node.
#
#path.data: /path/to/data
#
# Can optionally include more than one location, causing data to be striped across
# the locations (a la RAID 0) on a file level, favouring locations with most free
# space on creation. For example:
#
#path.data: /path/to/data1,/path/to/data2
# Path to temporary files:
#
#path.work: /path/to/work
# Path to log files:
#
#path.logs: /path/to/logs
# Path to where plugins are installed:
#
#path.plugins: /path/to/plugins
#################################### Plugin ###################################
# If a plugin listed here is not installed for current node, the node will not start.
#
#plugin.mandatory: mapper-attachments,lang-groovy
################################### Memory ####################################
# Elasticsearch performs poorly when JVM starts swapping: you should ensure that
# it _never_ swaps.
#
# Set this property to true to lock the memory:
#
#bootstrap.mlockall: true
# Make sure that the ES_MIN_MEM and ES_MAX_MEM environment variables are set
# to the same value, and that the machine has enough memory to allocate
# for Elasticsearch, leaving enough memory for the operating system itself.
#
# You should also make sure that the Elasticsearch process is allowed to lock
# the memory, eg. by using `ulimit -l unlimited`.
############################## Network And HTTP ###############################
# Elasticsearch, by default, binds itself to the 0.0.0.0 address, and listens
# on port [9200-9300] for HTTP traffic and on port [9300-9400] for node-to-node
# communication. (the range means that if the port is busy, it will automatically
# try the next port).
# Set the bind address specifically (IPv4 or IPv6):
#
#network.bind_host: 192.168.0.1
# Set the address other nodes will use to communicate with this node. If not
# set, it is automatically derived. It must point to an actual IP address.
#
#network.publish_host: 192.168.0.1
# Set both 'bind_host' and 'publish_host':
#
#network.host: 192.168.0.1
# Set a custom port for the node to node communication (9300 by default):
#
transport.tcp.port: ${TRANSPORT_PORT}
# Enable compression for all communication between nodes (disabled by default):
#
#transport.tcp.compress: true
# Set a custom port to listen for HTTP traffic:
#
http.port: ${HTTP_PORT}
# Set a custom allowed content length:
#
#http.max_content_length: 100mb
# Disable HTTP completely:
#
#http.enabled: false
################################### Gateway ###################################
# The gateway allows for persisting the cluster state between full cluster
# restarts. Every change to the state (such as adding an index) will be stored
# in the gateway, and when the cluster starts up for the first time,
# it will read its state from the gateway.
# There are several types of gateway implementations. For more information, see
# <http://elasticsearch.org/guide/en/elasticsearch/reference/current/modules-gateway.html>.
# The default gateway type is the "local" gateway (recommended):
#
#gateway.type: local
# Settings below control how and when to start the initial recovery process on
# a full cluster restart (to reuse as much local data as possible when using shared
# gateway).
# Allow recovery process after N nodes in a cluster are up:
#
#gateway.recover_after_nodes: 1
# Set the timeout to initiate the recovery process, once the N nodes
# from previous setting are up (accepts time value):
#
#gateway.recover_after_time: 5m
# Set how many nodes are expected in this cluster. Once these N nodes
# are up (and recover_after_nodes is met), begin recovery process immediately
# (without waiting for recover_after_time to expire):
#
#gateway.expected_nodes: 2
############################# Recovery Throttling #############################
# These settings allow to control the process of shards allocation between
# nodes during initial recovery, replica allocation, rebalancing,
# or when adding and removing nodes.
# Set the number of concurrent recoveries happening on a node:
#
# 1. During the initial recovery
#
#cluster.routing.allocation.node_initial_primaries_recoveries: 4
#
# 2. During adding/removing nodes, rebalancing, etc
#
#cluster.routing.allocation.node_concurrent_recoveries: 2
# Set to throttle throughput when recovering (eg. 100mb, by default 20mb):
#
#indices.recovery.max_bytes_per_sec: 20mb
# Set to limit the number of open concurrent streams when
# recovering a shard from a peer:
#
#indices.recovery.concurrent_streams: 5
################################## Discovery ##################################
# Discovery infrastructure ensures nodes can be found within a cluster
# and master node is elected. Multicast discovery is the default.
# Set to ensure a node sees N other master eligible nodes to be considered
# operational within the cluster. This should be set to a quorum/majority of
# the master-eligible nodes in the cluster.
#
#discovery.zen.minimum_master_nodes: 1
# Set the time to wait for ping responses from other nodes when discovering.
# Set this option to a higher value on a slow or congested network
# to minimize discovery failures:
#
#discovery.zen.ping.timeout: 3s
# For more information, see
# <http://elasticsearch.org/guide/en/elasticsearch/reference/current/modules-discovery-zen.html>
# Unicast discovery allows to explicitly control which nodes will be used
# to discover the cluster. It can be used when multicast is not present,
# or to restrict the cluster communication-wise.
#
# 1. Disable multicast discovery (enabled by default):
#
discovery.zen.ping.multicast.enabled: ${MULTICAST}
#
# 2. Configure an initial list of master nodes in the cluster
# to perform discovery when new nodes (master or data) are started:
#
#discovery.zen.ping.unicast.hosts: ${UNICAST_HOSTS}
# EC2 discovery allows to use AWS EC2 API in order to perform discovery.
#
# You have to install the cloud-aws plugin for enabling the EC2 discovery.
#
# For more information, see
# <http://elasticsearch.org/guide/en/elasticsearch/reference/current/modules-discovery-ec2.html>
#
# See <http://elasticsearch.org/tutorials/elasticsearch-on-ec2/>
# for a step-by-step tutorial.
# GCE discovery allows to use Google Compute Engine API in order to perform discovery.
#
# You have to install the cloud-gce plugin for enabling the GCE discovery.
#
# For more information, see <https://github.com/elasticsearch/elasticsearch-cloud-gce>.
# Azure discovery allows to use Azure API in order to perform discovery.
#
# You have to install the cloud-azure plugin for enabling the Azure discovery.
#
# For more information, see <https://github.com/elasticsearch/elasticsearch-cloud-azure>.
################################## Slow Log ##################################
# Shard level query and fetch threshold logging.
#index.search.slowlog.threshold.query.warn: 10s
#index.search.slowlog.threshold.query.info: 5s
#index.search.slowlog.threshold.query.debug: 2s
#index.search.slowlog.threshold.query.trace: 500ms
#index.search.slowlog.threshold.fetch.warn: 1s
#index.search.slowlog.threshold.fetch.info: 800ms
#index.search.slowlog.threshold.fetch.debug: 500ms
#index.search.slowlog.threshold.fetch.trace: 200ms
#index.indexing.slowlog.threshold.index.warn: 10s
#index.indexing.slowlog.threshold.index.info: 5s
#index.indexing.slowlog.threshold.index.debug: 2s
#index.indexing.slowlog.threshold.index.trace: 500ms
################################## GC Logging ################################
#monitor.jvm.gc.young.warn: 1000ms
#monitor.jvm.gc.young.info: 700ms
#monitor.jvm.gc.young.debug: 400ms
#monitor.jvm.gc.old.warn: 10s
#monitor.jvm.gc.old.info: 5s
#monitor.jvm.gc.old.debug: 2s
################################## Security ################################
# Uncomment if you want to enable JSONP as a valid return transport on the
# http server. With this enabled, it may pose a security risk, so disabling
# it unless you need it is recommended (it is disabled by default).
#
#http.jsonp.enable: true

79
examples/elasticsearch/elasticsearch_discovery.go
View File

@@ -1,79 +0,0 @@
/*
Copyright 2015 The Kubernetes Authors All rights reserved.
Licensed under the Apache License, Version 2.0 (the "License");
you may not use this file except in compliance with the License.
You may obtain a copy of the License at
http://www.apache.org/licenses/LICENSE-2.0
Unless required by applicable law or agreed to in writing, software
distributed under the License is distributed on an "AS IS" BASIS,
WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
See the License for the specific language governing permissions and
limitations under the License.
*/
package main
import (
"flag"
"fmt"
"strings"
"time"
"github.com/golang/glog"
"k8s.io/kubernetes/pkg/api"
client "k8s.io/kubernetes/pkg/client/unversioned"
"k8s.io/kubernetes/pkg/fields"
"k8s.io/kubernetes/pkg/labels"
)
var (
namespace = flag.String("namespace", api.NamespaceDefault, "The namespace containing Elasticsearch pods")
selector = flag.String("selector", "", "Selector (label query) for selecting Elasticsearch pods")
)
func main() {
flag.Parse()
glog.Info("Elasticsearch discovery")
glog.Infof("Namespace: %q", *namespace)
glog.Infof("selector: %q", *selector)
c, err := client.NewInCluster()
if err != nil {
glog.Fatalf("Failed to make client: %v", err)
}
l, err := labels.Parse(*selector)
if err != nil {
glog.Fatalf("Failed to parse selector %q: %v", *selector, err)
}
pods, err := c.Pods(*namespace).List(l, fields.Everything())
if err != nil {
glog.Fatalf("Failed to list pods: %v", err)
}
glog.Infof("Elasticsearch pods in namespace %s with selector %q", *namespace, *selector)
podIPs := []string{}
for i := range pods.Items {
p := &pods.Items[i]
for attempt := 0; attempt < 10; attempt++ {
glog.Infof("%d: %s PodIP: %s", i, p.Name, p.Status.PodIP)
if p.Status.PodIP != "" {
podIPs = append(podIPs, fmt.Sprintf(`"%s"`, p.Status.PodIP))
break
}
time.Sleep(1 * time.Second)
p, err = c.Pods(*namespace).Get(p.Name)
if err != nil {
glog.Warningf("Failed to get pod %s: %v", p.Name, err)
}
}
if p.Status.PodIP == "" {
glog.Warningf("Failed to obtain PodIP for %s", p.Name)
}
}
fmt.Printf("discovery.zen.ping.unicast.hosts: [%s]\n", strings.Join(podIPs, ", "))
}

54
examples/elasticsearch/es-rc.yaml Normal file
View File

@@ -0,0 +1,54 @@
apiVersion: v1
kind: ReplicationController
metadata:
name: es
labels:
component: elasticsearch
spec:
replicas: 1
selector:
component: elasticsearch
template:
metadata:
labels:
component: elasticsearch
spec:
serviceAccount: elasticsearch
containers:
- name: es
securityContext:
capabilities:
add:
- IPC_LOCK
image: quay.io/pires/docker-elasticsearch-kubernetes:1.7.1-4
env:
- name: KUBERNETES_CA_CERTIFICATE_FILE
value: /var/run/secrets/kubernetes.io/serviceaccount/ca.crt
- name: NAMESPACE
valueFrom:
fieldRef:
fieldPath: metadata.namespace
- name: "CLUSTER_NAME"
value: "myesdb"
- name: "DISCOVERY_SERVICE"
value: "elasticsearch"
- name: NODE_MASTER
value: "true"
- name: NODE_DATA
value: "true"
- name: HTTP_ENABLE
value: "true"
ports:
- containerPort: 9200
name: http
protocol: TCP
- containerPort: 9300
name: transport
protocol: TCP
volumeMounts:
- mountPath: /data
name: storage
volumes:
- name: storage
source:
emptyDir: {}

17
examples/elasticsearch/es-svc.yaml Normal file
View File

@@ -0,0 +1,17 @@
apiVersion: v1
kind: Service
metadata:
name: elasticsearch
labels:
component: elasticsearch
spec:
type: LoadBalancer
selector:
component: elasticsearch
ports:
- name: http
port: 9200
protocol: TCP
- name: transport
port: 9300
protocol: TCP

32
examples/elasticsearch/music-rc.yaml
View File

@@ -1,32 +0,0 @@
apiVersion: v1
kind: ReplicationController
metadata:
labels:
name: music-db
namespace: mytunes
name: music-db
spec:
replicas: 4
selector:
name: music-db
template:
metadata:
labels:
name: music-db
spec:
containers:
- name: es
image: kubernetes/elasticsearch:1.2
env:
- name: "CLUSTER_NAME"
value: "mytunes-db"
- name: "SELECTOR"
value: "name=music-db"
- name: "NAMESPACE"
value: "mytunes"
ports:
- name: es
containerPort: 9200
- name: es-transport
containerPort: 9300

15
examples/elasticsearch/music-service.yaml
View File

@@ -1,15 +0,0 @@
apiVersion: v1
kind: Service
metadata:
name: music-server
namespace: mytunes
labels:
name: music-db
spec:
selector:
name: music-db
ports:
- name: db
port: 9200
targetPort: es
type: LoadBalancer

6
examples/elasticsearch/mytunes-namespace.yaml
View File

@@ -1,6 +0,0 @@
kind: Namespace
apiVersion: v1
metadata:
name: mytunes
labels:
name: mytunes

222
examples/elasticsearch/production_cluster/README.md Normal file
View File

@@ -0,0 +1,222 @@
<!-- BEGIN MUNGE: UNVERSIONED_WARNING -->
<!-- BEGIN STRIP_FOR_RELEASE -->
<img src="http://kubernetes.io/img/warning.png" alt="WARNING"
width="25" height="25">
<img src="http://kubernetes.io/img/warning.png" alt="WARNING"
width="25" height="25">
<img src="http://kubernetes.io/img/warning.png" alt="WARNING"
width="25" height="25">
<img src="http://kubernetes.io/img/warning.png" alt="WARNING"
width="25" height="25">
<img src="http://kubernetes.io/img/warning.png" alt="WARNING"
width="25" height="25">
<h2>PLEASE NOTE: This document applies to the HEAD of the source tree</h2>
If you are using a released version of Kubernetes, you should
refer to the docs that go with that version.
<strong>
The latest 1.0.x release of this document can be found
[here](http://releases.k8s.io/release-1.0/examples/elasticsearch/production_cluster/README.md).
Documentation for other releases can be found at
[releases.k8s.io](http://releases.k8s.io).
</strong>
--
<!-- END STRIP_FOR_RELEASE -->
<!-- END MUNGE: UNVERSIONED_WARNING -->
# Elasticsearch for Kubernetes
Kubernetes makes it trivial for anyone to easily build and scale [Elasticsearch](http://www.elasticsearch.org/) clusters. Here, you'll find how to do so.
Current Elasticsearch version is `1.7.1`.
Before we start, one needs to know that Elasticsearch best-practices recommend to separate nodes in three roles:
* `Master` nodes - intended for clustering management only, no data, no HTTP API
* `Client` nodes - intended for client usage, no data, with HTTP API
* `Data` nodes - intended for storing and indexing your data, no HTTP API
This is enforced throughout this document.
<img src="http://kubernetes.io/img/warning.png" alt="WARNING" width="25" height="25"> Current pod descriptors use an `emptyDir` for storing data in each data node container. This is meant to be for the sake of simplicity and [should be adapted according to your storage needs](../../../docs/design/persistent-storage.md).
## Docker image
This example uses [this pre-built image](https://github.com/pires/docker-elasticsearch-kubernetes) will not be supported. Feel free to fork to fit your own needs, but mind yourself that you will need to change Kubernetes descriptors accordingly.
## Deploy
```
kubectl create -f examples/elasticsearch/production_cluster/service-account.yaml
kubectl create -f examples/elasticsearch/production_cluster/es-discovery-svc.yaml
kubectl create -f examples/elasticsearch/production_cluster/es-svc.yaml
kubectl create -f examples/elasticsearch/production_cluster/es-master-rc.yaml
```
Wait until `es-master` is provisioned, and
```
kubectl create -f examples/elasticsearch/production_cluster/es-client-rc.yaml
```
Wait until `es-client` is provisioned, and
```
kubectl create -f examples/elasticsearch/production_cluster/es-data-rc.yaml
```
Wait until `es-data` is provisioned.
Now, I leave up to you how to validate the cluster, but a first step is to wait for containers to be in ```RUNNING``` state and check the Elasticsearch master logs:
```
$ kubectl get pods
NAME READY STATUS RESTARTS AGE
es-client-2ep9o 1/1 Running 0 2m
es-data-r9tgv 1/1 Running 0 1m
es-master-vxl6c 1/1 Running 0 6m
```
```
$ kubectl logs es-master-vxl6c
log4j:WARN No such property [maxBackupIndex] in org.apache.log4j.DailyRollingFileAppender.
log4j:WARN No such property [maxBackupIndex] in org.apache.log4j.DailyRollingFileAppender.
log4j:WARN No such property [maxBackupIndex] in org.apache.log4j.DailyRollingFileAppender.
[2015-08-21 10:58:51,324][INFO ][node ] [Arc] version[1.7.1], pid[8], build[b88f43f/2015-07-29T09:54:16Z]
[2015-08-21 10:58:51,328][INFO ][node ] [Arc] initializing ...
[2015-08-21 10:58:51,542][INFO ][plugins ] [Arc] loaded [cloud-kubernetes], sites []
[2015-08-21 10:58:51,624][INFO ][env ] [Arc] using [1] data paths, mounts [[/data (/dev/sda9)]], net usable_space [14.4gb], net total_space [15.5gb], types [ext4]
[2015-08-21 10:58:57,439][INFO ][node ] [Arc] initialized
[2015-08-21 10:58:57,439][INFO ][node ] [Arc] starting ...
[2015-08-21 10:58:57,782][INFO ][transport ] [Arc] bound_address {inet[/0:0:0:0:0:0:0:0:9300]}, publish_address {inet[/10.244.15.2:9300]}
[2015-08-21 10:58:57,847][INFO ][discovery ] [Arc] myesdb/-x16XFUzTCC8xYqWoeEOYQ
[2015-08-21 10:59:05,167][INFO ][cluster.service ] [Arc] new_master [Arc][-x16XFUzTCC8xYqWoeEOYQ][es-master-vxl6c][inet[/10.244.15.2:9300]]{data=false, master=true}, reason: zen-disco-join (elected_as_master)
[2015-08-21 10:59:05,202][INFO ][node ] [Arc] started
[2015-08-21 10:59:05,238][INFO ][gateway ] [Arc] recovered [0] indices into cluster_state
[2015-08-21 11:02:28,797][INFO ][cluster.service ] [Arc] added {[Gideon][4EfhWSqaTqikbK4tI7bODA][es-data-r9tgv][inet[/10.244.59.4:9300]]{master=false},}, reason: zen-disco-receive(join from node[[Gideon][4EfhWSqaTqikbK4tI7bODA][es-data-r9tgv][inet[/10.244.59.4:9300]]{master=false}])
[2015-08-21 11:03:16,822][INFO ][cluster.service ] [Arc] added {[Venomm][tFYxwgqGSpOejHLG4umRqg][es-client-2ep9o][inet[/10.244.53.2:9300]]{data=false, master=false},}, reason: zen-disco-receive(join from node[[Venomm][tFYxwgqGSpOejHLG4umRqg][es-client-2ep9o][inet[/10.244.53.2:9300]]{data=false, master=false}])
```
As you can assert, the cluster is up and running. Easy, wasn't it?
## Scale
Scaling each type of node to handle your cluster is as easy as:
```
kubectl scale --replicas=3 rc es-master
kubectl scale --replicas=2 rc es-client
kubectl scale --replicas=2 rc es-data
```
Did it work?
```
$ kubectl get pods
NAME READY STATUS RESTARTS AGE
es-client-2ep9o 1/1 Running 0 4m
es-client-ye5s1 1/1 Running 0 50s
es-data-8az22 1/1 Running 0 47s
es-data-r9tgv 1/1 Running 0 3m
es-master-57h7k 1/1 Running 0 52s
es-master-kuwse 1/1 Running 0 52s
es-master-vxl6c 1/1 Running 0 8m
```
Let's take another look of the Elasticsearch master logs:
```
$ kubectl logs es-master-vxl6c
log4j:WARN No such property [maxBackupIndex] in org.apache.log4j.DailyRollingFileAppender.
log4j:WARN No such property [maxBackupIndex] in org.apache.log4j.DailyRollingFileAppender.
log4j:WARN No such property [maxBackupIndex] in org.apache.log4j.DailyRollingFileAppender.
[2015-08-21 10:58:51,324][INFO ][node ] [Arc] version[1.7.1], pid[8], build[b88f43f/2015-07-29T09:54:16Z]
[2015-08-21 10:58:51,328][INFO ][node ] [Arc] initializing ...
[2015-08-21 10:58:51,542][INFO ][plugins ] [Arc] loaded [cloud-kubernetes], sites []
[2015-08-21 10:58:51,624][INFO ][env ] [Arc] using [1] data paths, mounts [[/data (/dev/sda9)]], net usable_space [14.4gb], net total_space [15.5gb], types [ext4]
[2015-08-21 10:58:57,439][INFO ][node ] [Arc] initialized
[2015-08-21 10:58:57,439][INFO ][node ] [Arc] starting ...
[2015-08-21 10:58:57,782][INFO ][transport ] [Arc] bound_address {inet[/0:0:0:0:0:0:0:0:9300]}, publish_address {inet[/10.244.15.2:9300]}
[2015-08-21 10:58:57,847][INFO ][discovery ] [Arc] myesdb/-x16XFUzTCC8xYqWoeEOYQ
[2015-08-21 10:59:05,167][INFO ][cluster.service ] [Arc] new_master [Arc][-x16XFUzTCC8xYqWoeEOYQ][es-master-vxl6c][inet[/10.244.15.2:9300]]{data=false, master=true}, reason: zen-disco-join (elected_as_master)
[2015-08-21 10:59:05,202][INFO ][node ] [Arc] started
[2015-08-21 10:59:05,238][INFO ][gateway ] [Arc] recovered [0] indices into cluster_state
[2015-08-21 11:02:28,797][INFO ][cluster.service ] [Arc] added {[Gideon][4EfhWSqaTqikbK4tI7bODA][es-data-r9tgv][inet[/10.244.59.4:9300]]{master=false},}, reason: zen-disco-receive(join from node[[Gideon][4EfhWSqaTqikbK4tI7bODA][es-data-r9tgv][inet[/10.244.59.4:9300]]{master=false}])
[2015-08-21 11:03:16,822][INFO ][cluster.service ] [Arc] added {[Venomm][tFYxwgqGSpOejHLG4umRqg][es-client-2ep9o][inet[/10.244.53.2:9300]]{data=false, master=false},}, reason: zen-disco-receive(join from node[[Venomm][tFYxwgqGSpOejHLG4umRqg][es-client-2ep9o][inet[/10.244.53.2:9300]]{data=false, master=false}])
[2015-08-21 11:04:40,781][INFO ][cluster.service ] [Arc] added {[Erik Josten][QUJlahfLTi-MsxzM6_Da0g][es-master-kuwse][inet[/10.244.59.5:9300]]{data=false, master=true},}, reason: zen-disco-receive(join from node[[Erik Josten][QUJlahfLTi-MsxzM6_Da0g][es-master-kuwse][inet[/10.244.59.5:9300]]{data=false, master=true}])
[2015-08-21 11:04:41,076][INFO ][cluster.service ] [Arc] added {[Power Princess][V4qnR-6jQOS5ovXQsPgo7g][es-master-57h7k][inet[/10.244.53.3:9300]]{data=false, master=true},}, reason: zen-disco-receive(join from node[[Power Princess][V4qnR-6jQOS5ovXQsPgo7g][es-master-57h7k][inet[/10.244.53.3:9300]]{data=false, master=true}])
[2015-08-21 11:04:53,966][INFO ][cluster.service ] [Arc] added {[Cagliostro][Wpfx5fkBRiG2qCEWd8laaQ][es-client-ye5s1][inet[/10.244.15.3:9300]]{data=false, master=false},}, reason: zen-disco-receive(join from node[[Cagliostro][Wpfx5fkBRiG2qCEWd8laaQ][es-client-ye5s1][inet[/10.244.15.3:9300]]{data=false, master=false}])
[2015-08-21 11:04:56,803][INFO ][cluster.service ] [Arc] added {[Thog][vkdEtX3ESfWmhXXf-Wi0_Q][es-data-8az22][inet[/10.244.15.4:9300]]{master=false},}, reason: zen-disco-receive(join from node[[Thog][vkdEtX3ESfWmhXXf-Wi0_Q][es-data-8az22][inet[/10.244.15.4:9300]]{master=false}])
```
## Access the service
*Don't forget* that services in Kubernetes are only acessible from containers in the cluster. For different behavior you should [configure the creation of an external load-balancer](http://kubernetes.io/v1.0/docs/user-guide/services.html#type-loadbalancer). While it's supported within this example service descriptor, its usage is out of scope of this document, for now.
```
$ kubectl get service elasticsearch
NAME LABELS SELECTOR IP(S) PORT(S)
elasticsearch component=elasticsearch,role=client component=elasticsearch,role=client 10.100.134.2 9200/TCP
```
From any host on your cluster (that's running `kube-proxy`), run:
```
curl http://10.100.134.2:9200
```
You should see something similar to the following:
```json
{
"status" : 200,
"name" : "Cagliostro",
"cluster_name" : "myesdb",
"version" : {
"number" : "1.7.1",
"build_hash" : "b88f43fc40b0bcd7f173a1f9ee2e97816de80b19",
"build_timestamp" : "2015-07-29T09:54:16Z",
"build_snapshot" : false,
"lucene_version" : "4.10.4"
},
"tagline" : "You Know, for Search"
}
```
Or if you want to check cluster information:
```
curl http://10.100.134.2:9200/_cluster/health?pretty
```
You should see something similar to the following:
```json
{
"cluster_name" : "myesdb",
"status" : "green",
"timed_out" : false,
"number_of_nodes" : 7,
"number_of_data_nodes" : 2,
"active_primary_shards" : 0,
"active_shards" : 0,
"relocating_shards" : 0,
"initializing_shards" : 0,
"unassigned_shards" : 0,
"delayed_unassigned_shards" : 0,
"number_of_pending_tasks" : 0,
"number_of_in_flight_fetch" : 0
}
```
<!-- BEGIN MUNGE: GENERATED_ANALYTICS -->
[![Analytics](https://kubernetes-site.appspot.com/UA-36037335-10/GitHub/examples/elasticsearch/production_cluster/README.md?pixel)]()
<!-- END MUNGE: GENERATED_ANALYTICS -->

55
examples/elasticsearch/production_cluster/es-client-rc.yaml Normal file
View File

@@ -0,0 +1,55 @@
apiVersion: v1
kind: ReplicationController
metadata:
name: es-client
labels:
component: elasticsearch
role: client
spec:
replicas: 1
selector:
component: elasticsearch
role: client
template:
metadata:
labels:
component: elasticsearch
role: client
spec:
serviceAccount: elasticsearch
containers:
- name: es-client
securityContext:
capabilities:
add:
- IPC_LOCK
image: quay.io/pires/docker-elasticsearch-kubernetes:1.7.1-4
env:
- name: KUBERNETES_CA_CERTIFICATE_FILE
value: /var/run/secrets/kubernetes.io/serviceaccount/ca.crt
- name: NAMESPACE
valueFrom:
fieldRef:
fieldPath: metadata.namespace
- name: "CLUSTER_NAME"
value: "myesdb"
- name: NODE_MASTER
value: "false"
- name: NODE_DATA
value: "false"
- name: HTTP_ENABLE
value: "true"
ports:
- containerPort: 9200
name: http
protocol: TCP
- containerPort: 9300
name: transport
protocol: TCP
volumeMounts:
- mountPath: /data
name: storage
volumes:
- name: storage
source:
emptyDir: {}

50
examples/elasticsearch/production_cluster/es-data-rc.yaml Normal file
View File

@@ -0,0 +1,50 @@
apiVersion: v1
kind: ReplicationController
metadata:
name: es-data
labels:
component: elasticsearch
role: data
spec:
replicas: 1
selector:
component: elasticsearch
role: data
template:
metadata:
labels:
component: elasticsearch
role: data
spec:
serviceAccount: elasticsearch
containers:
- name: es-data
securityContext:
capabilities:
add:
- IPC_LOCK
image: quay.io/pires/docker-elasticsearch-kubernetes:1.7.1-4
env:
- name: KUBERNETES_CA_CERTIFICATE_FILE
value: /var/run/secrets/kubernetes.io/serviceaccount/ca.crt
- name: NAMESPACE
valueFrom:
fieldRef:
fieldPath: metadata.namespace
- name: "CLUSTER_NAME"
value: "myesdb"
- name: NODE_MASTER
value: "false"
- name: HTTP_ENABLE
value: "false"
ports:
- containerPort: 9300
name: transport
protocol: TCP
volumeMounts:
- mountPath: /data
name: storage
volumes:
- name: storage
source:
emptyDir: {}

15
examples/elasticsearch/production_cluster/es-discovery-svc.yaml Normal file
View File

@@ -0,0 +1,15 @@
apiVersion: v1
kind: Service
metadata:
name: elasticsearch-discovery
labels:
component: elasticsearch
role: master
spec:
selector:
component: elasticsearch
role: master
ports:
- name: transport
port: 9300
protocol: TCP

52
examples/elasticsearch/production_cluster/es-master-rc.yaml Normal file
View File

@@ -0,0 +1,52 @@
apiVersion: v1
kind: ReplicationController
metadata:
name: es-master
labels:
component: elasticsearch
role: master
spec:
replicas: 1
selector:
component: elasticsearch
role: master
template:
metadata:
labels:
component: elasticsearch
role: master
spec:
serviceAccount: elasticsearch
containers:
- name: es-master
securityContext:
capabilities:
add:
- IPC_LOCK
image: quay.io/pires/docker-elasticsearch-kubernetes:1.7.1-4
env:
- name: KUBERNETES_CA_CERTIFICATE_FILE
value: /var/run/secrets/kubernetes.io/serviceaccount/ca.crt
- name: NAMESPACE
valueFrom:
fieldRef:
fieldPath: metadata.namespace
- name: "CLUSTER_NAME"
value: "myesdb"
- name: NODE_MASTER
value: "true"
- name: NODE_DATA
value: "false"
- name: HTTP_ENABLE
value: "false"
ports:
- containerPort: 9300
name: transport
protocol: TCP
volumeMounts:
- mountPath: /data
name: storage
volumes:
- name: storage
source:
emptyDir: {}

16
examples/elasticsearch/production_cluster/es-svc.yaml Normal file
View File

@@ -0,0 +1,16 @@
apiVersion: v1
kind: Service
metadata:
name: elasticsearch
labels:
component: elasticsearch
role: client
spec:
type: LoadBalancer
selector:
component: elasticsearch
role: client
ports:
- name: http
port: 9200
protocol: TCP

4
examples/elasticsearch/production_cluster/service-account.yaml Normal file
View File

@@ -0,0 +1,4 @@
apiVersion: v1
kind: ServiceAccount
metadata:
name: elasticsearch

24
examples/elasticsearch/run.sh
View File

@@ -1,24 +0,0 @@
#!/bin/bash
# Copyright 2015 The Kubernetes Authors All rights reserved.
#
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.
export CLUSTER_NAME=${CLUSTER_NAME:-elasticsearch-default}
export NODE_MASTER=${NODE_MASTER:-true}
export NODE_DATA=${NODE_DATA:-true}
export MULTICAST=${MULTICAST:-false}
/elasticsearch_discovery --namespace="${NAMESPACE}" --selector="${SELECTOR}" >> /elasticsearch-1.5.2/config/elasticsearch.yml
export HTTP_PORT=${HTTP_PORT:-9200}
export TRANSPORT_PORT=${TRANSPORT_PORT:-9300}
/elasticsearch-1.5.2/bin/elasticsearch

4
examples/elasticsearch/service-account.yaml Normal file
View File

@@ -0,0 +1,4 @@
apiVersion: v1
kind: ServiceAccount
metadata:
name: elasticsearch

6
examples/examples_test.go
View File

@@ -240,9 +240,9 @@ func TestExampleObjectSchemas(t *testing.T) {
"dapi-volume": &api.Pod{}, "dapi-volume": &api.Pod{},
}, },
"../examples/elasticsearch": { "../examples/elasticsearch": {
"mytunes-namespace": &api.Namespace{}, "es-rc": &api.ReplicationController{},
"music-rc": &api.ReplicationController{}, "es-svc": &api.Service{},
"music-service": &api.Service{}, "service-account": nil,
}, },
"../examples/explorer": { "../examples/explorer": {
"pod": &api.Pod{}, "pod": &api.Pod{},

8
hack/verify-flags/exceptions.txt
View File

@@ -63,10 +63,10 @@ docs/getting-started-guides/coreos/azure/lib/deployment_logic/kubernetes.js:var
docs/getting-started-guides/logging-elasticsearch.md: "cluster_name" : "kubernetes-logging", docs/getting-started-guides/logging-elasticsearch.md: "cluster_name" : "kubernetes-logging",
docs/user-guide/accessing-the-cluster.md: "cluster_name" : "kubernetes_logging", docs/user-guide/accessing-the-cluster.md: "cluster_name" : "kubernetes_logging",
examples/cluster-dns/images/frontend/client.py: service_address = socket.gethostbyname(hostname) examples/cluster-dns/images/frontend/client.py: service_address = socket.gethostbyname(hostname)
examples/elasticsearch/README.md: "cluster_name" : "mytunes-db", examples/elasticsearch/README.md: "cluster_name" : "myesdb",
examples/elasticsearch/README.md: "cluster_name" : "mytunes-db", examples/elasticsearch/README.md: "cluster_name" : "myesdb",
examples/elasticsearch/README.md: "cluster_name" : "mytunes-db", examples/elasticsearch/production_cluster/README.md: "cluster_name" : "myesdb",
examples/elasticsearch/README.md:"cluster_name" : "mytunes-db", examples/elasticsearch/production_cluster/README.md: "cluster_name" : "myesdb",
hack/lib/logging.sh: local source_file=${BASH_SOURCE[$frame_no]} hack/lib/logging.sh: local source_file=${BASH_SOURCE[$frame_no]}
hack/lib/logging.sh: local source_file=${BASH_SOURCE[$stack_skip]} hack/lib/logging.sh: local source_file=${BASH_SOURCE[$stack_skip]}
hack/local-up-cluster.sh: runtime_config="--runtime-config=\"${RUNTIME_CONFIG}\"" hack/local-up-cluster.sh: runtime_config="--runtime-config=\"${RUNTIME_CONFIG}\""