From cf74edaac51f0eb850658ec50edc08b3b5920dc9 Mon Sep 17 00:00:00 2001 From: stephb9959 Date: Thu, 8 Sep 2022 21:55:45 -0700 Subject: [PATCH] Update to v2.7.0 Signed-off-by: stephb9959 --- microservice_sample_cpp/README.md | 178 +++++++----------- .../ow_helloworld.properties | 67 ++----- microservice_sample_cpp/ow_helloworld.service | 2 +- microservice_sample_cpp/set_env.sh | 4 +- microservice_sample_cpp/test_scripts/curl/cli | 6 +- 5 files changed, 87 insertions(+), 170 deletions(-) diff --git a/microservice_sample_cpp/README.md b/microservice_sample_cpp/README.md index c0777d9..4c9f664 100644 --- a/microservice_sample_cpp/README.md +++ b/microservice_sample_cpp/README.md @@ -4,10 +4,10 @@ This is a skeleton Micro Service that shows off all the basic facilities of the OW architecture. ## Building -In order to build the uCentralGW, you will need to install its dependencies, which includes the following: +In order to build the `hello_world` micro service, you will need to install its dependencies, which includes the following: - cmake -- POCO 1.12 or later (special build) -- a C++17 compiler +- POCO 1.12 from Arilia +- C++17 compiler - openssl - libpq-dev (PortgreSQL development libraries) - mysql-client (MySQL client) @@ -16,7 +16,7 @@ In order to build the uCentralGW, you will need to install its dependencies, whi The build is done in 2 parts. The first part is to build a local copy of the framework tailored to your environment. This framework is called [Poco](https://github.com/pocoproject/poco). The version used in this project has a couple of fixes -from the master copy needed for cmake. Please use the version of this [Poco fix](https://github.com/stephb9959/poco). Building +from the master copy needed for cmake. Please use the version of this [Poco fix](https://github.com/AriliaWireless/poco). Building Poco may take several minutes depending on the platform you are building on. ### Ubuntu/Debian @@ -24,10 +24,11 @@ These instructions have proven to work on Ubuntu 20.4. ``` sudo apt install git cmake g++ libssl-dev libmariadb-dev sudo apt install libpq-dev libaprutil1-dev apache2-dev libboost-all-dev -sudo apt install librdkafka-dev libmysqlclient-dev default-libmysqlclient-dev +sudo apt install librdkafka-dev // default-libmysqlclient-dev +sudo apt install nlohmann-json-dev cd ~ -git clone https://github.com/stephb9959/poco +git clone https://github.com/AriliaWireless/poco --branch poco-tip-v1 cd poco mkdir cmake-build cd cmake-build @@ -36,7 +37,7 @@ cmake --build . --config Release sudo cmake --build . --target install cd ~ -git clone https://github.com/stephb9959/cppkafka +git clone https://github.com/AriliaWireless/cppkafka --branch tip-v1 cd cppkafka mkdir cmake-build cd cmake-build @@ -45,16 +46,7 @@ cmake --build . --config Release sudo cmake --build . --target install cd ~ -git clone https://github.com/nlohmann/json.git -cd json -mkdir cmake-build -cd cmake-build -cmake .. -make -j -sudo make install - -cd ~ -git clone https://github.com/pboettch/json-schema-validator.git +git clone https://github.com/pboettch/json-schema-validator.git --branch 2.1.0 cd json-schema-validator mkdir cmake-build cd cmake-build @@ -62,6 +54,14 @@ cmake .. make -j sudo make install +git clone https://github.com/fmtlib/fmt --branch 9.0.0 /fmtlib +cd fmtlib +mkdir cmake-build +cd cmake-build +cmake .. +make +make install + cd ~ git clone https://github.com/Telecominfraproject/wlan-cloud-tools cd wlan-cloud-tools @@ -72,11 +72,8 @@ cmake .. make -j ``` -### After completing the build -After completing the build, you can remove the Poco source as it is no longer needed. - #### Expected directory layout -From the directory where your cloned source is, you will need to create the `certs`, `logs`, and `uploads` directories. +From the directory where your cloned source is, you will need to create the `certs`, `data`, and `logs` directories. ```shell mkdir certs mkdir logs @@ -87,7 +84,7 @@ mkdir data Love'em of hate'em, we gotta use'em. So we tried to make this as easy as possible for you. #### The `certs` directory -For all deployments, you will need the following certs directory, populated with the proper files. +For all deployments, you will need the following `certs` directory, populated with the proper files. ```asm certs ---+ @@ -97,122 +94,70 @@ certs ---+ ``` #### Configuration -The configuration for this service is kept in a properties file. This file is called `owgw.properties` and you can -see the latest version [here](https://github.com/Telecominfraproject/wlan-cloud-ucentralgw/blob/main/owgw.properties). The file will be loaded from -the directory set by the environment variable `UCENTRALGW_CONFIG`. To use environment variables in the configuration, +The configuration for this service is kept in a properties file. This file is called `ow_helloworld.properties`. The file will be loaded +from root the directory set by the environment variable `OWHELLOWORLD_CONFIG`. To use environment variables in the configuration, you must use `$`. Only `path names` support the use of environment variables. The sample configuration requires very little changes if you keep the suggested directory structure. For the sample configuration to work, you need to define 2 environment variables. ``` -export OWGW_ROOT=`pwd` -export UCENTRALGW_CONFIG=`pwd` +export $OWHELLOWORLD_ROOT=`pwd` +export $OWHELLOWORLD_CONFIG=`pwd` ``` -If you current working directory is the root of the project, this will set the variables properly. Otherwise, you can set the variables +If your current working directory is the root of the project, this will set the variables properly. Otherwise, you can set the variables to point to wherever is necessary. ##### Important config entries -###### This is the logging directory -``` -logging.channels.c2.path = $OWGW_ROOT/logs/sample.log -``` ###### This is the type of storage in use ```asm storage.type = sqlite ``` -###### Autoprovisioning settings -```asm -openwifi.autoprovisioning = true -openwifi.devicetypes.0 = AP:linksys_ea8300,edgecore_eap101,linksys_e8450-ubi -openwifi.devicetypes.1 = SWITCH:edgecore_ecs4100-12ph -openwifi.devicetypes.2 = IOT:esp32 -``` - ###### This is the RESTAPI endpoint - ```asm openwifi.restapi.host.0.backlog = 100 openwifi.restapi.host.0.security = relaxed -openwifi.restapi.host.0.rootca = $OWHELLOW_ROOT/certs/restapi-ca.pem +openwifi.restapi.host.0.rootca = $OWHELLOWORLD_ROOT/certs/restapi-ca.pem openwifi.restapi.host.0.address = * -openwifi.restapi.host.0.port = 16002 -openwifi.restapi.host.0.cert = $OWHELLOW_ROOT/certs/restapi-cert.pem -openwifi.restapi.host.0.key = $OWHELLOW_ROOT/certs/restapi-key.pem +openwifi.restapi.host.0.port = 16051 +openwifi.restapi.host.0.cert = $OWHELLOWORLD_ROOT/certs/restapi-cert.pem +openwifi.restapi.host.0.key = $OWHELLOWORLD_ROOT/certs/restapi-key.pem openwifi.restapi.host.0.key.password = mypassword openwifi.internal.restapi.host.0.backlog = 100 openwifi.internal.restapi.host.0.security = relaxed -openwifi.internal.restapi.host.0.rootca = $OWHELLOW_ROOT/certs/restapi-ca.pem +openwifi.internal.restapi.host.0.rootca = $OWHELLOWORLD_ROOT/certs/restapi-ca.pem openwifi.internal.restapi.host.0.address = * -openwifi.internal.restapi.host.0.port = 17002 -openwifi.internal.restapi.host.0.cert = $OWHELLOW_ROOT/certs/restapi-cert.pem -openwifi.internal.restapi.host.0.key = $OWHELLOW_ROOT/certs/restapi-key.pem +openwifi.internal.restapi.host.0.port = 17051 +openwifi.internal.restapi.host.0.cert = $OWHELLOWORLD_ROOT/certs/restapi-cert.pem +openwifi.internal.restapi.host.0.key = $OWHELLOWORLD_ROOT/certs/restapi-key.pem openwifi.internal.restapi.host.0.key.password = mypassword ``` ##### This is the end point for the devices to connect with -This is the crucial section. I bet that 97.4% of all your problems will come from here, and it's boring. So put some good music on, -give the kids the iPad, get a cup of coffee, and pay attention. Every field will be explained. +2 types of endpoints: `restapi` and `internal`. -###### openwifi.websocket.host.0.backlog -This is the number of concurrent devices you are expecting to call all at once. Not the current number of devices. This is how many will connect in the same exact second. -Take the total number of devices you have and divide by 100. That's a good rule of thumb. Never go above 500. +###### `restapi` +This is the public side of the API. This should always be secured and exposed through your container or a firewall. +This should be protected. The IP address use by this interface should have an FQDN matching its certificate. That +certificate and key are in `openwifi.restapi.host.0.cert` and `openwifi.restapi.host.0.key`. The port you specify +under `openwifi.restapi.host.0.port` is the one you will use in your URIs to get into the micro service. This +should be open in your firewall or docker container. -###### openwifi.websocket.host.0.rootca -This is the root file as supplied by Digicert. You can find it [here](https://github.com/Telecominfraproject/wlan-cloud-ucentralgw/blob/main/certificates/root.pem) - -###### openwifi.websocket.host.0.issuer -This is the issuer file as supplied by Digicert. You can find it [here](https://github.com/Telecominfraproject/wlan-cloud-ucentralgw/blob/main/certificates/issuer.pem) - -###### openwifi.websocket.host.0.cert -This is a `pem` file that you will receive from Digicert for the gateway itself. This is the certificate for the gateway. - -###### openwifi.websocket.host.0.key -This is a `pem` file that you will receive from Digicert for the gateway itself. The is the private key for the gateway. - -###### openwifi.websocket.host.0.clientcas -This is a `pem` file that contains both the issuer and the root CA certificates. You can find it You can find it [here](https://github.com/Telecominfraproject/wlan-cloud-ucentralgw/blob/main/certificates/clientcas.pem) - -###### openwifi.websocket.host.0.cas -This is a directory where you will copy your own `cert.pem`, the `root.pem`, and the `issuer.pem` files. - -###### openwifi.websocket.host.0.address -Leve this a `*` in teh case you want to bind to all interfaces on your gateway host or select the address of a single interface. - -###### openwifi.websocket.host.0.port -Leave to 15002 for now. - -###### openwifi.websocket.host.0.security -Leave this as strict for now for devices. - -###### openwifi.websocket.host.0.key.password -If you key file uses a password, please enter it here. - -###### openwifi.websocket.maxreactors -A single reactor can handle between 1000-2000 devices. Never leave this smaller than 5 or larger than 50. - -#### Conclusion -You will need to get the `cert.pem` and `key.pem` from Digicert. The rest is here. - -###### host.0.address entries -If you want to limit traffic to a specific interface, you should specify the IP address of that interface instead of -the `*`. Using the `*` means all interfaces will be able to accept connections. You can add multiple interfaces -by changing the `0` to another index. You need to repeat the whole configuration block for each index. Indexes must be sequential -start at `0`. - -```asm -openwifi.service.key = $OWGW_ROOT/certs/websocket-key.pem -``` +###### `internal` +This is the private side of the API. This should never be exposed through your container or a firewall. +This should be protected. That certificate and key are in `openwifi.restapi.host.0.cert` and `openwifi.restapi.host.0.key`. +The port you specify under `openwifi.restapi.host.0.port` is the one you will use in your URIs to get into the micro service +behind the firewall or using docker networking. This port is assumed secured. Never expose it. #### Command line options The current implementation supports the following. If you use the built-in configuration file, you do not need to use any command-line options. However, you may decide to use the `--daemon` or `umask` options. ```bash -./ucentralgw --help -usage: ucentralgw OPTIONS -A uCentral gateway implementation for TIP. +./ow_helloworld --help +usage: ow_helloworld OPTIONS +A ow_helloworld implementation for TIP. --daemon Run application as a daemon. --umask=mask Set the daemon's umask (octal, e.g. 027). @@ -224,35 +169,41 @@ A uCentral gateway implementation for TIP. ``` ##### file -This allows you to point to another file without specifying the UCENTRALGW_CONFIG variable. The file name must end in `.properties`. +This allows you to point to another file without specifying the OWHELLOWORLD_CONFIG variable. The file name must end in `.properties`. ##### daemon Run this as a UNIX service ##### pidfile -When running as a daemon, the pid of the running service will be set in the speficied file +When running as a daemon, the pid of the running service will be set in the specified file ##### debug Run the service in debug mode. ##### logs -Speficy where logs should be kept. You must include an existing directory and a file name. For example `/var/ucentral/logs/log.0`. +Specify where logs should be kept. You must include an existing directory and a file name. For example `/var/ucentral/logs/log.0`. ##### umask -Seet the umask for the running service. +Set the umask for the running service. ### ALB Support -Support for AWS ALB is provided through the following configuration elements +Support for AWS ALB is provided through the following configuration elements. This is the built-in AWS Application +Load Balancer ```asm alb.enable = true alb.port = 16102 ``` ## Kafka integration -So what about Kafka? Well, the gateway has basic integration with Kafka. It is turned off by default, to turn it on, in the configuration: - +So what about Kafka? Well, the micro service framework has basic integration with Kafka. It is turned off by default, +to turn it on, in the configuration: ```asm openwifi.kafka.group.id = hello_world openwifi.kafka.client.id = hello_world1 openwifi.kafka.enable = false -openwifi.kafka.brokerlist = 127.0.0.1:9092 +openwifi.kafka.brokerlist = kafka:9092 openwifi.kafka.commit = false openwifi.kafka.queue.buffering.max.ms = 50 +openwifi.kafka.ssl.ca.location = +openwifi.kafka.ssl.certificate.location = +openwifi.kafka.ssl.key.location = +openwifi.kafka.ssl.key.password = + ``` #### `openwifi.kafka.group.id` This must be set to your service's name and should be unique @@ -264,13 +215,16 @@ This must be set to your service's name and should be unique Kind of obvious but hey, set `true` or `false`. Default is `false` #### `openwifi.kafka.brokerlist` -This is a comma separator list of the brokers in your `kafka` deployment. +This is a comma separator list of the brokers in your `kafka` deployment. If you are using Docker, this will usually be something like +`kafka:9092`. If you are using an external Kafka cluster, `kafka-cluster.domain.com:9092`. #### Kafka topics -Toe read more about Kafka, follow the [document](https://github.com/Telecominfraproject/wlan-cloud-ucentralgw/blob/main/KAFKA.md) +To read more about Kafka, follow the [document](https://github.com/Telecominfraproject/wlan-cloud-ucentralgw/blob/main/KAFKA.md) #### Securing `kafka` This is beyond the scope of this document. As it stands today, the communication between the gateway and `kafka` is expected to be behind a firewall. +However, the framework also allows secure Kafka access. To use secured access, you must fill the fields that +start with `openwifi.kafka.ssl`. ## Contributors We love ya! We need more of ya! If you want to contribute, make sure you review diff --git a/microservice_sample_cpp/ow_helloworld.properties b/microservice_sample_cpp/ow_helloworld.properties index 98a3499..03505ea 100644 --- a/microservice_sample_cpp/ow_helloworld.properties +++ b/microservice_sample_cpp/ow_helloworld.properties @@ -8,31 +8,31 @@ # openwifi.restapi.host.0.backlog = 100 openwifi.restapi.host.0.security = relaxed -openwifi.restapi.host.0.rootca = $OWHELLOW_ROOT/certs/restapi-ca.pem +openwifi.restapi.host.0.rootca = $OWHELLOWORLD_ROOT/certs/restapi-ca.pem openwifi.restapi.host.0.address = * -openwifi.restapi.host.0.port = 16501 -openwifi.restapi.host.0.cert = $OWHELLOW_ROOT/certs/restapi-cert.pem -openwifi.restapi.host.0.key = $OWHELLOW_ROOT/certs/restapi-key.pem +openwifi.restapi.host.0.port = 16051 +openwifi.restapi.host.0.cert = $OWHELLOWORLD_ROOT/certs/restapi-cert.pem +openwifi.restapi.host.0.key = $OWHELLOWORLD_ROOT/certs/restapi-key.pem openwifi.restapi.host.0.key.password = mypassword openwifi.internal.restapi.host.0.backlog = 100 openwifi.internal.restapi.host.0.security = relaxed -openwifi.internal.restapi.host.0.rootca = $OWHELLOW_ROOT/certs/restapi-ca.pem +openwifi.internal.restapi.host.0.rootca = $OWHELLOWORLD_ROOT/certs/restapi-ca.pem openwifi.internal.restapi.host.0.address = * -openwifi.internal.restapi.host.0.port = 16601 -openwifi.internal.restapi.host.0.cert = $OWHELLOW_ROOT/certs/restapi-cert.pem -openwifi.internal.restapi.host.0.key = $OWHELLOW_ROOT/certs/restapi-key.pem +openwifi.internal.restapi.host.0.port = 17051 +openwifi.internal.restapi.host.0.cert = $OWHELLOWORLD_ROOT/certs/restapi-cert.pem +openwifi.internal.restapi.host.0.key = $OWHELLOWORLD_ROOT/certs/restapi-key.pem openwifi.internal.restapi.host.0.key.password = mypassword # # Generic section that all microservices must have # -openwifi.service.key = $OWHELLOW_ROOT/certs/restapi-key.pem +openwifi.service.key = $OWHELLOWORLD_ROOT/certs/restapi-key.pem openwifi.service.key.password = mypassword -openwifi.system.data = $OWHELLOW_ROOT/data +openwifi.system.data = $OWHELLOWORLD_ROOT/data openwifi.system.debug = true -openwifi.system.uri.private = https://localhost:17002 -openwifi.system.uri.public = https://external.servername.com:16002 +openwifi.system.uri.private = https://localhost:17032 +openwifi.system.uri.public = https://external.servername.com:16032 openwifi.system.uri.ui = https://main-ui.servername.com openwifi.system.commandchannel = /tmp/app.owhelloworld @@ -87,44 +87,7 @@ storage.type.mysql.database = dbname storage.type.mysql.port = 3306 storage.type.mysql.connectiontimeout = 60 -######################################################################## -######################################################################## -# -# Logging: please leave as is for now. -# -######################################################################## - -logging.formatters.f1.class = PatternFormatter -logging.formatters.f1.pattern = %s: [%p] %t -logging.formatters.f1.times = UTC -logging.channels.c1.class = ConsoleChannel -logging.channels.c1.formatter = f1 - -# This is where the logs will be written. This path MUST exist -logging.channels.c2.class = FileChannel -logging.channels.c2.path = $OWHELLOW_ROOT/logs/log -logging.channels.c2.formatter.class = PatternFormatter -logging.channels.c2.formatter.pattern = %Y-%m-%d %H:%M:%S %s: [%p] %t -logging.channels.c2.rotation = 20 M -logging.channels.c2.archive = timestamp -logging.channels.c2.purgeCount = 20 -logging.channels.c3.class = ConsoleChannel -logging.channels.c3.pattern = %s: [%p] %t - -# External Channel -logging.loggers.root.channel = c2 -logging.loggers.root.level = debug - -# Inline Channel with PatternFormatter -# logging.loggers.l1.name = logger1 -# logging.loggers.l1.channel.class = ConsoleChannel -# logging.loggers.l1.channel.pattern = %s: [%p] %t -# logging.loggers.l1.level = information -# SplitterChannel -# logging.channels.splitter.class = SplitterChannel -# logging.channels.splitter.channels = l1,l2 -# logging.loggers.l2.name = logger2 -# logging.loggers.l2.channel = splitter - - +logging.type = file +logging.path = $OWHELLOWORLD_ROOT/logs +logging.level = debug diff --git a/microservice_sample_cpp/ow_helloworld.service b/microservice_sample_cpp/ow_helloworld.service index 2f69ef1..d0a3083 100644 --- a/microservice_sample_cpp/ow_helloworld.service +++ b/microservice_sample_cpp/ow_helloworld.service @@ -5,7 +5,7 @@ Wants=network-online.target [Service] Type=simple -Environment="OWHELLOW_ROOT=/home/admin/dev/wlan-cloud-ow_helloworld" +Environment="$OWHELLOWORLD_ROOT=/home/admin/dev/wlan-cloud-ow_helloworld" ExecStart=/home/admin/dev/wlan-cloud-ow_helloworld/cmake-build/ow_helloworld WorkingDirectory=/home/admin/dev/wlan-cloud-ow_helloworld # ExecReload=/bin/kill -s HUP $MAINPID diff --git a/microservice_sample_cpp/set_env.sh b/microservice_sample_cpp/set_env.sh index 87497be..02aad63 100755 --- a/microservice_sample_cpp/set_env.sh +++ b/microservice_sample_cpp/set_env.sh @@ -1,4 +1,4 @@ #!/bin/bash -export OWHELLOW_CONFIG=`pwd` -export OWHELLOW_ROOT=`pwd` +export $OWHELLOWORLD_CONFIG=`pwd` +export $OWHELLOWORLD_ROOT=`pwd` diff --git a/microservice_sample_cpp/test_scripts/curl/cli b/microservice_sample_cpp/test_scripts/curl/cli index d9bbdc6..ed8d8c9 100755 --- a/microservice_sample_cpp/test_scripts/curl/cli +++ b/microservice_sample_cpp/test_scripts/curl/cli @@ -2,9 +2,9 @@ # # License type: BSD 3-Clause License -# License copy: https://github.com/Telecominfraproject/wlan-cloud-ucentralgw/blob/master/LICENSE +# License copy: https://github.com/Telecominfraproject/wlan-cloud-tools/blob/master/LICENSE # -# Created by Stephane Bourque on 2021-03-04. +# Created by Stephane Bourque on 2022-08-04. # Arilia Wireless Inc. # @@ -141,7 +141,7 @@ reloadsubsystem() { help() { echo - echo "hello Get the device JSON document." + echo "hello Get the device JSON document." echo "setloglevel Set the logging system level for individual subsystems." echo " sys:ufileuploader/websocket/storage/restapi/commandmanager/auth/deviceregistry/all" echo " level:none/fatal/critical/error/warning/notice/information/debug/trace"