Telecominfraproject/wlan-cloud-analytics
1
0
Fork 0
mirror of https://github.com/Telecominfraproject/wlan-cloud-analytics.git synced 2026-01-27 02:22:21 +00:00
Code Issues Packages Projects Releases Wiki Activity

https://telecominfraproject.atlassian.net/browse/WIFI-12068

Browse Source 
Signed-off-by: stephb9959 <stephane.bourque@gmail.com>
This commit is contained in:
stephb9959
2023-02-23 12:45:16 -08:00
parent 9511e778aa
commit 8ec5c3f258
2 changed files with 433 additions and 3 deletions
Download Patch File Download Diff File Expand all files Collapse all files

203
CONFIGURATION.md Normal file
View File

@@ -0,0 +1,203 @@
# OWANALYTICS Configuration
Here is the list of parameters you can configure in the `owanalytics.properties` file.
## OWANALYTICS Specific Parameters
```properties
firmware.updater.upgrade = false
firmware.updater.releaseonly = false
```
## Generic OpenWiFi SDK parameters
### REST API External parameters
These are the parameters required for the configuration of the external facing REST API server
```properties
openwifi.restapi.host.0.backlog = 100
openwifi.restapi.host.0.security = relaxed
openwifi.restapi.host.0.rootca = $OWANALYTICS_ROOT/certs/restapi-ca.pem
openwifi.restapi.host.0.address = *
openwifi.restapi.host.0.port = 16005
openwifi.restapi.host.0.cert = $OWANALYTICS_ROOT/certs/restapi-cert.pem
openwifi.restapi.host.0.key = $OWANALYTICS_ROOT/certs/restapi-key.pem
openwifi.restapi.host.0.key.password = mypassword
```
#### openwifi.restapi.host.0.backlog
This is the number of concurrent REST API calls that maybe be kept in the backlog for processing. That's a good rule of thumb. Never go above 500.
#### openwifi.restapi.host.0.rootca
This is the root file of your own certificate CA in `pem` format.
#### openwifi.restapi.host.0.cert
This is your own server certificate in `pem` format..
#### openwifi.restapi.host.0.key
This is the private key associated with your own certificate in `pem` format.
#### openwifi.restapi.host.0.address
Leve this a `*` in the case you want to bind to all interfaces on your gateway host or select the address of a single interface.
#### openwifi.restapi.host.0.port
The port on which the REST API server is listening. By default, this is 16002.
#### openwifi.restapi.host.0.security
Leave this as `relaxed` for now for devices.
#### openwifi.restapi.host.0.key.password
If you key file uses a password, please enter it here.
### REST API Intra microservice parameters
The following parameters describe the configuration for the inter-microservice HTTP server. You may use the same certificate/key
you are using for your extenral server or another certificate.
```properties
openwifi.internal.restapi.host.0.backlog = 100
openwifi.internal.restapi.host.0.security = relaxed
openwifi.internal.restapi.host.0.rootca = $OWANALYTICS_ROOT/certs/restapi-ca.pem
openwifi.internal.restapi.host.0.address = *
openwifi.internal.restapi.host.0.port = 17005
openwifi.internal.restapi.host.0.cert = $OWANALYTICS_ROOT/certs/restapi-cert.pem
openwifi.internal.restapi.host.0.key = $OWANALYTICS_ROOT/certs/restapi-key.pem
openwifi.internal.restapi.host.0.key.password = mypassword
```
#### openwifi.internal.host.0.backlog
This is the number of concurrent REST API calls that maybe be kept in the backlog for processing. That's a good rule of thumb. Never go above 500.
#### openwifi.internal.host.0.rootca
This is the root file of your own certificate CA in `pem` format.
#### openwifi.internal.host.0.cert
This is your own server certificate in `pem` format..
#### openwifi.internal.host.0.key
This is the private key associated with your own certificate in `pem` format.
#### openwifi.internal.host.0.address
Leve this a `*` in the case you want to bind to all interfaces on your gateway host or select the address of a single interface.
#### openwifi.internal.host.0.port
The port on which the REST API server is listening. By default, this is 17002.
#### openwifi.internal.host.0.security
Leave this as `relaxed` for now for devices.
#### openwifi.internal.host.0.key.password
If you key file uses a password, please enter it here.
### Microservice information
These are different Microservie parameters. Following is a brief explanation.
```properties
openwifi.service.key = $OWANALYTICS_ROOT/certs/restapi-key.pem
openwifi.service.key.password = mypassword
openwifi.system.data = $OWANALYTICS_ROOT/data
openwifi.system.uri.private = https://localhost:17005
openwifi.system.uri.public = https://openwifi.example.com:16005
openwifi.system.uri.ui = https://provisioning-ui.arilia.com
openwifi.security.restapi.disable = false
openwifi.system.commandchannel = /tmp/app.ucentralanalytics
openwifi.autoprovisioning = true
```
#### openwifi.service.key
From time to time, the microservice must encrypt information. This is the key it should use. You may use the
same keey as you RESTAPI or your server.
#### openwifi.service.key.password
The password for the `openwifi.service.key`
#### openwifi.system.data
The location of system data. This path must exist.
#### openwifi.system.uri.private
The URI to reach the controller on the internal port.
#### openwifi.system.uri.public
The URI to reach the controller from the outside world.
#### openwifi.system.uri.ui
The URI of the UI to manage this service
#### openwifi.security.restapi.disable
This allows to disable security for internal and external API calls. This should only be used if the controller
sits behind an application load balancer that will actually do TLS. Setting this to `true` disables security.
#### openwifi.system.commandchannel
The UNIX socket command channel used by this service.
#### openwifi.autoprovisioning
Allow unknown devices to be provisioned by the system.
### ALB Support
In order to support an application load balancer health check verification, your need to provide the following parameters.
```properties
alb.enable = true
alb.port = 16105
```
### Kafka
The controller use Kafka, like all the other microservices. You must configure the kafka section in order for the
system to work.
```properties
openwifi.kafka.group.id = analytics
openwifi.kafka.client.id = analytics1
openwifi.kafka.enable = true
openwifi.kafka.brokerlist = my_Kafka.example.com:9092
openwifi.kafka.auto.commit = false
openwifi.kafka.queue.buffering.max.ms = 50
```
### openwifi.kafka.group.id
The group ID is a single word that should identify the type of service tuning. In the case `analytics`
### openwifi.kafka.client.id
The client ID is a single service within that group ID. Each participant must have a unique client ID.
### openwifi.kafka.enable
Kafka should always be enabled.
### openwifi.kafka.brokerlist
The list of servers where your Kafka server is running. Comma separated.
### openwifi.kafka.auto.commit
Auto commit flag in Kafka. Leave as `false`.
### openwifi.kafka.queue.buffering.max.ms
Kafka buffering. Leave as `50`.
### Kafka security
If you intend to use SSL, you should look into Kafka Connect and specify the certificates below.
```properties
penwifi.kafka.ssl.ca.location =
openwifi.kafka.ssl.certificate.location =
openwifi.kafka.ssl.key.location =
openwifi.kafka.ssl.key.password =
```
### DB Type
The controller supports 3 types of Database. SQLite should only be used for sites with less than 100 APs or for testing in the lab.
In order to select which database to use, you must set the `storage.type` value to sqlite, postgresql, or mysql.
```properties
storage.type = sqlite
#storage.type = postgresql
#storage.type = mysql
```
### Storage SQLite parameters
Additional parameters to set for SQLite. The only important one is `storage.type.sqlite.db` which is the database name on disk.
```properties
storage.type.sqlite.db = analytics.db
storage.type.sqlite.idletime = 120
storage.type.sqlite.maxsessions = 128
```
### Storage Postgres
Additional parameters to set if you select Postgres for your database. You must specify `host`, `username`, `password`,
`database`, and `port`.
```properties
storage.type.postgresql.maxsessions = 64
storage.type.postgresql.idletime = 60
storage.type.postgresql.host = localhost
storage.type.postgresql.username = analytics
storage.type.postgresql.password = analytics
storage.type.postgresql.database = analytics
storage.type.postgresql.port = 5432
storage.type.postgresql.connectiontimeout = 60
```
### Storage MySQL/MariaDB
Additional parameters to set if you select mysql for your database. You must specify `host`, `username`, `password`,
`database`, and `port`.
```properties
storage.type.mysql.maxsessions = 64
storage.type.mysql.idletime = 60
storage.type.mysql.host = localhost
storage.type.postgresql.username = analytics
storage.type.postgresql.password = analytics
storage.type.postgresql.database = analytics
storage.type.mysql.port = 3306
storage.type.mysql.connectiontimeout = 60
```
### Logging Parameters
The microservice provides extensive logging. If you would like to keep logging on disk, set the `logging.type = file`. If you only want
console logging, `set logging.type = console`. When selecting file, `logging.path` must exist. `logging.level` sets the
basic logging level for the entire controller. `logging.websocket` disables WebSocket logging.
```properties
logging.type = file
logging.path = $OWANALYTICS_ROOT/logs
logging.level = information
logging.asynch = true
logging.websocket = false
```

233
README.md
View File

@@ -1,10 +1,237 @@
<p align=center><img src=images/project/logo.svg?sanitize=true/ width="200px" height="200px"></p>
# wlan-cloud-analytics
OpenWiFi Analytics V0.01
# OpenWiFI Analytics Service (OWANALYTICS)
## What is it?
The OpenWiFi Analytics Service is a service for the TIP OpenWiFi CloudSDK (OWSDK).
OWANALYTICS gathers statistics about device used in OpenWiFI and groups them according to their
provisioning (OWPROV) entities or venues. OWANALYTICS, like all other OWSDK microservices, is
defined using an OpenAPI definition and uses the ucentral communication protocol to interact with Access Points. To use
the OWANALYTICS, you either need to [build it](#building) or use the [Docker version](#docker).
## Building
In order to build the OWANALYTICS, you will need to install its dependencies, which includes the following:
- cmake
- boost
- POCO 1.10.1 or later
- a C++17 compiler
- openssl
- libpq-dev (PortgreSQL development libraries)
- mysql-client (MySQL client)
- librdkafka
- cppkafka
The build is done in 2 parts. The first part is to build a local copy of the framework tailored to your environment. This
framework is called [Poco](https://github.com/pocoproject/poco). The version used in this project has a couple of fixes
from the master copy needed for cmake. Please use the version of this [Poco fix](https://github.com/AriliaWireless/poco). Building
Poco may take several minutes depending on the platform you are building on.
### Ubuntu
These instructions have proven to work on Ubuntu 20.4.
```bash
sudo apt install git cmake g++ libssl-dev libmariadb-dev
sudo apt install libpq-dev libaprutil1-dev apache2-dev libboost-all-dev
sudo apt install librdkafka-dev default-libmysqlclient-dev
sudo apt install nlohmann-json-dev
cd ~
git clone https://github.com/AriliaWireless/poco --branch poco-tip-v1
cd poco
mkdir cmake-build
cd cmake-build
cmake ..
cmake --build . --config Release
sudo cmake --build . --target install
cd ~
git clone https://github.com/AriliaWireless/cppkafka --branch tip-v1
cd cppkafka
mkdir cmake-build
cd cmake-build
cmake ..
cmake --build . --config Release
sudo cmake --build . --target install
cd ~
git clone https://github.com/AriliaWireless/valijson --branch tip-v1
cd valijson
mkdir cmake-build
cd cmake-build
cmake ..
cmake --build . --config Release
sudo cmake --build . --target install
git clone https://github.com/fmtlib/fmt --branch 9.0.0 /fmtlib
cd fmtlib
mkdir cmake-build
cd cmake-build
cmake ..
make
make install
cd ~
git clone https://github.com/Telecominfraproject/wlan-cloud-analytics
cd wlan-cloud-analytics
mkdir cmake-build
cd cmake-build
cmake ..
make -j 8
```
### Fedora
The following instructions have proven to work on Fedora 33
```bash
sudo yum install cmake g++ openssl-devel mysql-devel mysql apr-util-devel boost boost-devel
sudo yum install yaml-cpp-devel lua-devel
sudo dnf install postgresql.x86_64 librdkafka-devel
sudo dnf install postgresql-devel json-devel
git clone https://github.com/AriliaWireless/poco --branch poco-tip-v1
cd poco
mkdir cmake-build
cd cmake-build
cmake ..
cmake --build . --config Release
sudo cmake --build . --target install
git clone https://github.com/AriliaWireless/cppkafka --branch tip-v1
cd cppkafka
mkdir cmake-build
cd cmake-build
cmake ..
cmake --build . --config Release
sudo cmake --build . --target install
cd ~
git clone https://github.com/AriliaWireless/valijson --branch tip-v1
cd valijson
mkdir cmake-build
cd cmake-build
cmake ..
cmake --build . --config Release
sudo cmake --build . --target install
cd ~
git clone https://github.com/Telecominfraproject/wlan-cloud-analytics
cd wlan-cloud-analytics
mkdir cmake-build
cd cmake-build
cmake ..
make
```
### macOS Build
The following instructions have proven to work on macOS Big Sur. You need to install [Homebrew](https://brew.sh/). You must also have installed [XCode for OS X](https://www.freecodecamp.org/news/how-to-download-and-install-xcode/).
```bash
brew install openssl \
cmake \
libpq \
mysql-client \
apr \
apr-util \
boost \
yaml-cpp \
postgresql \
librdkafka \
nlohmann-json \
fmt
git clone https://github.com/AriliaWireless/poco --branch poco-tip-v1
pushd poco
mkdir cmake-build
push cmake-build
cmake -DOPENSSL_ROOT_DIR=</path/to/openssl> -DENABLE_NETSSL=1 -DENABLE_JWT=1 -DENABLE_CRYPTO=1 ..
cmake --build . --config Release
sudo cmake --build . --target install
popd
popd
git clone https://github.com/AriliaWireless/cppkafka --branch tip-v1
pushd cppkafka
mkdir cmake-build
pushd cmake-build
cmake ..
cmake --build . --config Release
sudo cmake --build . --target install
popd
popd
git clone https://github.com/AriliaWireless/valijson --branch tip-v1
cd valijson
mkdir cmake-build
cd cmake-build
cmake ..
cmake --build . --config Release
sudo cmake --build . --target install
popd
popd
git clone https://github.com/Telecominfraproject/wlan-cloud-analytics
pushd wlan-cloud-analytics
mkdir cmake-build
pushd cmake-build
cmake ..
make -j
popd
popd
```
### Raspberry
The build on a rPI takes a while. You can shorten that build time and requirements by disabling all the larger database
support. You can build with only SQLite support by not installing the packages for PostgreSQL, and MySQL by
adding -DSMALL_BUILD=1 on the cmake build line.
```bash
sudo apt install git cmake g++ libssl-dev libaprutil1-dev apache2-dev libboost-all-dev libyaml-cpp-dev
git clone https://github.com/stephb9959/poco
cd poco
mkdir cmake-build
cd cmake-build
cmake ..
cmake --build . --config Release
sudo cmake --build . --target install
cd ~
git clone https://github.com/Telecominfraproject/wlan-cloud-analytics
cd wlan-cloud-ucentralgw
mkdir cmake-build
cd cmake-build
cmake -DSMALL_BUILD=1 ..
make
```
### OWANALYTICS Service Configuration
The configuration is kept in a file called `owanalytics.properties`. To understand the content of this file,
please look [here](https://github.com/Telecominfraproject/wlan-cloud-analytics/blob/main/CONFIGURATION.md)
## OpenAPI
You may get static page with OpenAPI docs generated from the definition on [GitHub Page](https://telecominfraproject.github.io/wlan-cloud-analytics/).
Also you may use [Swagger UI](https://petstore.swagger.io/#/) with OpenAPI definition file raw link (i.e. [latest version file](https://raw.githubusercontent.com/Telecominfraproject/wlan-cloud-analytics/main/openapi/owanalytics.yaml)) to get interactive docs page.
Also, you may use [Swagger UI](https://petstore.swagger.io/#/) with OpenAPI definition file raw link (i.e. [latest version file](https://raw.githubusercontent.com/Telecominfraproject/wlan-cloud-analytics/main/openapi/owanalytics.yaml)) to get interactive docs page.
## Firewall Considerations
| Port | Description | Configurable |
|:------|:-----------------------------------------------|:------------:|
| 16005 | Default port for REST API Access to the OWPROV | yes |
## Kafka topics
Toe read more about Kafka, follow the [document](https://github.com/Telecominfraproject/wlan-cloud-ucentralgw/blob/main/KAFKA.md)
## Contributions
We need more contributors. Should you wish to contribute,
please follow the [contributions](https://github.com/Telecominfraproject/wlan-cloud-ucentralgw/blob/master/CONTRIBUTING.md) document.
## Pull Requests
Please create a branch with the Jira addressing the issue you are fixing or the feature you are implementing.
Create a pull-request from the branch into master.
## Additional OWSDK Microservices
Here is a list of additional OWSDK microservices
| Name | Description | Link | OpenAPI |
| :--- | :--- | :---: | :---: |
| OWSEC | Security Service | [here](https://github.com/Telecominfraproject/wlan-cloud-ucentralsec) | [here](https://github.com/Telecominfraproject/wlan-cloud-ucentralsec/blob/main/openpapi/owsec.yaml) |
| OWGW | Controller Service | [here](https://github.com/Telecominfraproject/wlan-cloud-ucentralgw) | [here](https://github.com/Telecominfraproject/wlan-cloud-ucentralgw/blob/master/openapi/owgw.yaml) |
| OWFMS | Firmware Management Service | [here](https://github.com/Telecominfraproject/wlan-cloud-ucentralfms) | [here](https://github.com/Telecominfraproject/wlan-cloud-ucentralfms/blob/main/openapi/owfms.yaml) |
| OWPROV | Provisioning Service | [here](https://github.com/Telecominfraproject/wlan-cloud-owprov) | [here](https://github.com/Telecominfraproject/wlan-cloud-owprov/blob/main/openapi/owprov.yaml) |
| OWANALYTICS | Analytics Service | [here](https://github.com/Telecominfraproject/wlan-cloud-analytics) | [here](https://github.com/Telecominfraproject/wlan-cloud-analytics/blob/main/openapi/owanalytics.yaml) |