add missing custom config directory references

added option to define the config location, can be different than the relative location of the agent binary

Dockerfile

@@ -10,7 +10,7 @@ ENV GOSUMDB=off
 ##########################################
 # Installing some additional dependencies.
 
-RUN apt-get update && apt-get install -y --no-install-recommends \
+RUN apt-get upgrade -y && apt-get update && apt-get install -y --no-install-recommends \
 	git build-essential cmake pkg-config unzip libgtk2.0-dev \
 	curl ca-certificates libcurl4-openssl-dev libssl-dev libjpeg62-turbo-dev && \
 	rm -rf /var/lib/apt/lists/*
@@ -32,7 +32,7 @@ RUN cat /go/src/github.com/kerberos-io/agent/machinery/version
 
 RUN cd /go/src/github.com/kerberos-io/agent/machinery && \
 	go mod download && \
-	go build -tags timetzdata,netgo --ldflags '-s -w -extldflags "-static -latomic"' main.go && \
+	go build -tags timetzdata,netgo,osusergo --ldflags '-s -w -extldflags "-static -latomic"' main.go && \
 	mkdir -p /agent && \
 	mv main /agent && \
 	mv version /agent && \
@@ -147,4 +147,4 @@ HEALTHCHECK CMD curl --fail http://localhost:80 || exit 1
 # Leeeeettttt'ssss goooooo!!!
 # Run the shizzle from the right working directory.
 WORKDIR /home/agent
-CMD ["./main", "run", "opensource", "80"]
+CMD ["./main", "-action", "run", "-port", "80"]

README.md

@@ -78,12 +78,9 @@ If you want to connect to an USB or Raspberry Pi camera, [you'll need to run our
 
 ## Quickstart - Balena
 
-Run Kerberos Agent with Balena super powers. Monitor your agent with seamless remote access, and an encrypted https endpoint.
-Checkout our fleet on [Balena Hub](https://hub.balena.io/fleets?0%5B0%5D%5Bn%5D=any&0%5B0%5D%5Bo%5D=full_text_search&0%5B0%5D%5Bv%5D=agent), and add your agent.
+Run Kerberos Agent with [Balena Cloud](https://www.balena.io/) super powers. Monitor your Kerberos Agent with seamless remote access, over the air updates, an encrypted public `https` endpoint and many more. Checkout our application `video-surveillance` on [Balena Hub](https://hub.balena.io/apps/2064752/video-surveillance), and create your first or fleet of Kerberos Agent(s).
 
-[![balena deploy button](https://www.balena.io/deploy.svg)](https://dashboard.balena-cloud.com/deploy?repoUrl=https://github.com/kerberos-io/agent)
-
-**_Work In Progress_** - Currently we only support IP and USB Cameras, we have [an approach for leveraging the Raspberry Pi camera](https://github.com/kerberos-io/camera-to-rtsp), but this isn't working as expected with Balena. If you require this, you'll need to use the traditional Docker deployment with sidecar as mentioned above.
+[![deploy with balena](https://balena.io/deploy.svg)](https://dashboard.balena-cloud.com/deploy?repoUrl=https://github.com/kerberos-io/balena-agent)
 
 ## A world of Kerberos Agents
 
@@ -122,6 +119,7 @@ We have documented the different deployment models [in the `deployments` directo
 - [Red Hat OpenShift with Ansible](https://github.com/kerberos-io/agent/tree/master/deployments#4-red-hat-ansible-and-openshift)
 - [Terraform](https://github.com/kerberos-io/agent/tree/master/deployments#5-terraform)
 - [Salt](https://github.com/kerberos-io/agent/tree/master/deployments#6-salt)
+- [Balena](https://github.com/kerberos-io/agent/tree/master/deployments#8-balena)
 
 By default your Kerberos Agents will store all its configuration and recordings inside the container. To help you automate and have a more consistent data governance, you can attach volumes to configure and persist data of your Kerberos Agents, and/or configure each Kerberos Agent through environment variables.
 
@@ -166,6 +164,7 @@ Next to attaching the configuration file, it is also possible to override the co
 | Name                                    | Description                                                                                     | Default Value                  |
 | --------------------------------------- | ----------------------------------------------------------------------------------------------- | ------------------------------ |
 | `AGENT_MODE`                            | You can choose to run this in 'release' for production, and or 'demo' for showcasing.           | "release"                      |
+| `AGENT_TLS_INSECURE`                    | Specify if you want to use `InsecureSkipVerify` for the internal HTTP client.                   | "false"                        |
 | `AGENT_USERNAME`                        | The username used to authenticate against the Kerberos Agent login page.                        | "root"                         |
 | `AGENT_PASSWORD`                        | The password used to authenticate against the Kerberos Agent login page.                        | "root"                         |
 | `AGENT_KEY`                             | A unique identifier for your Kerberos Agent, this is auto-generated but can be overriden.       | ""                             |
@@ -184,8 +183,11 @@ Next to attaching the configuration file, it is also possible to override the co
 | `AGENT_CAPTURE_IPCAMERA_ONVIF_XADDR`    | ONVIF endpoint/address running on the camera.                                                   | ""                             |
 | `AGENT_CAPTURE_IPCAMERA_ONVIF_USERNAME` | ONVIF username to authenticate against.                                                         | ""                             |
 | `AGENT_CAPTURE_IPCAMERA_ONVIF_PASSWORD` | ONVIF password to authenticate against.                                                         | ""                             |
+| `AGENT_CAPTURE_MOTION`                  | Toggle for enabling or disabling motion.                                                        | "true"                         |
+| `AGENT_CAPTURE_LIVEVIEW`                | Toggle for enabling or disabling liveview.                                                      | "true"                         |
+| `AGENT_CAPTURE_SNAPSHOTS`               | Toggle for enabling or disabling snapshot generation.                                           | "true"                         |
 | `AGENT_CAPTURE_RECORDING`               | Toggle for enabling making recordings.                                                          | "true"                         |
-| `AGENT_CAPTURE_CONTINUOUS`              | Toggle for enabling continuous or motion based recording.                                       | "false"                        |
+| `AGENT_CAPTURE_CONTINUOUS`              | Toggle for enabling continuous "true" or motion "false".                                        | "false"                        |
 | `AGENT_CAPTURE_PRERECORDING`            | If `CONTINUOUS` set to `false`, specify the recording time (seconds) before after motion event. | "10"                           |
 | `AGENT_CAPTURE_POSTRECORDING`           | If `CONTINUOUS` set to `false`, specify the recording time (seconds) after motion event.        | "20"                           |
 | `AGENT_CAPTURE_MAXLENGTH`               | The maximum length of a single recording (seconds).                                             | "30"                           |
@@ -203,7 +205,7 @@ Next to attaching the configuration file, it is also possible to override the co
 | `AGENT_HUB_URI`                         | The Kerberos Hub API, defaults to our Kerberos Hub SAAS.                                        | "https://api.hub.domain.com"   |
 | `AGENT_HUB_KEY`                         | The access key linked to your account in Kerberos Hub.                                          | ""                             |
 | `AGENT_HUB_PRIVATE_KEY`                 | The secret access key linked to your account in Kerberos Hub.                                   | ""                             |
-| `AGENT_HUB_USERNAME`                    | Your Kerberos Hub username, which owns the above access and secret keys.                        | ""                             |
+| `AGENT_HUB_REGION`                      | The Kerberos Hub region, to which you want to upload.                                           | ""                             |
 | `AGENT_HUB_SITE`                        | The site ID of a site you've created in your Kerberos Hub account.                              | ""                             |
 | `AGENT_KERBEROSVAULT_URI`               | The Kerberos Vault API url.                                                                     | "https://vault.domain.com/api" |
 | `AGENT_KERBEROSVAULT_ACCESS_KEY`        | The access key of a Kerberos Vault account.                                                     | ""                             |
@@ -234,9 +236,9 @@ On opening of the GitHub Codespace, some dependencies will be installed. Once th
     const dev = {
       ENV: 'dev',
       HOSTNAME: externalHost,
-      //API_URL: `${protocol}//${hostname}:8080/api`,
-      //URL: `${protocol}//${hostname}:8080`,
-      //WS_URL: `${websocketprotocol}//${hostname}:8080/ws`,
+      //API_URL: `${protocol}//${hostname}:80/api`,
+      //URL: `${protocol}//${hostname}:80`,
+      //WS_URL: `${websocketprotocol}//${hostname}:80/ws`,
 
       // Uncomment, and comment the above lines, when using codespaces or other special DNS names (which you can't control)
       API_URL: `${protocol}//${externalHost}/api`,
@@ -249,7 +251,7 @@ Go and open two terminals one for the `ui` project and one for the `machinery` p
 1.  Terminal A:
 
         cd machinery/
-        go run main.go run camera 80
+        go run main.go -action run -port 80
 
 2.  Terminal B:
 
@@ -290,7 +292,7 @@ You can simply run the `machinery` using following commands.
 
     git clone https://github.com/kerberos-io/agent
     cd machinery
-    go run main.go run mycameraname 80
+    go run main.go -action run -port 80
 
 This will launch the Kerberos Agent and run a webserver on port `80`. You can change the port by your own preference. We strongly support the usage of [Goland](https://www.jetbrains.com/go/) or [Visual Studio Code](https://code.visualstudio.com/), as it comes with all the debugging and linting features builtin.
 

deployments/README.md

@@ -14,6 +14,7 @@ We will discuss following deployment models.
 - [5. Kerberos Factory](#5-kerberos-factory)
 - [6. Terraform](#6-terraform)
 - [7. Salt](#7-salt)
+- [8. Balena](#8-balena)
 
 ## 0. Static binary
 
@@ -58,3 +59,11 @@ To be written
 ## 7. Salt
 
 To be written
+
+## 8. Balena
+
+Balena Cloud provide a seamless way of building and deploying applications at scale through the conceps of `blocks`, `apps` and `fleets`. Once you have your `app` deployed, for example our Kerberos Agent, you can benefit from features such as: remote access, over the air updates, an encrypted public `https` endpoint and many more.
+
+Together with the Balena.io team we've build a Balena App, called [`video-surveillance`](https://hub.balena.io/apps/2064752/video-surveillance), which any can use to deploy a video surveillance system in a matter of minutes with all the expected management features you can think of.
+
+> Learn more [about Kerberos Agent with Balena](https://github.com/kerberos-io/agent/tree/master/deployments/balena).

deployments/ansible-openshift/playbook.yml

@@ -82,7 +82,7 @@
 
                 initContainers:
                   - name: download-config
-                    image: kerberos/agent:1b96d01
+                    image: kerberos/agent:latest
                     volumeMounts:
                       - name: kerberos-data
                         mountPath: /home/agent/data/config
@@ -96,7 +96,7 @@
 
                 containers:
                   - name: agent
-                    image: kerberos/agent:1b96d01
+                    image: kerberos/agent:latest
                     volumeMounts:
                       - name: kerberos-data
                         mountPath: /home/agent/data/config

deployments/balena/README.md

@@ -0,0 +1,31 @@
+# Deployment with Balena
+
+Balena Cloud provide a seamless way of building and deploying applications at scale through the conceps of `blocks`, `apps` and `fleets`. Once you have your `app` deployed, for example our Kerberos Agent, you can benefit from features such as: remote access, over the air updates, an encrypted public `https` endpoint and many more.
+
+We provide two mechanisms to deploy Kerberos Agent to a Balena Cloud fleet:
+
+1. Use Kerberos Agent as [a block part of your application](https://github.com/kerberos-io/balena-agent-block).
+2. Use Kerberos Agent as [a stand-alone application](https://github.com/kerberos-io/balena-agent).
+
+## Block
+
+Within Balena you can build the concept of a block, which is the equivalent of container image or a function in a typical programming language. The idea of blocks, you can find a more thorough explanation [here](https://docs.balena.io/learn/develop/blocks/), is that you can compose and combine multiple `blocks` to level up to the concept an `app`.
+
+You as a developer can choose which `blocks` you would like to use, to build the desired `application` state you prefer. For example you can use the [Kerberos Agent block](https://hub.balena.io/blocks/2064662/agent) to compose a video surveillance system as part of your existing set of blocks.
+
+You can the `Kerberos Agent` block by defining following elements in your `compose` file.
+
+    agent:
+        image: bh.cr/kerberos_io/agent
+
+## App
+
+Next to building individual `blocks` you as a developer can also decide to build up an application, composed of one or more `blocks` or third-party containers, and publish it as an `app` to the Balena Hub. This is exactly [what we've done..](https://hub.balena.io/apps/2064752/video-surveillance)
+
+On Balena Hub we have created the []`video-surveillance` application](https://hub.balena.io/apps/2064752/video-surveillance) that utilises the [Kerberos Agent `block`](https://hub.balena.io/blocks/2064662/agent). The idea of this application is that utilises the foundation of our Kerberos Agent, but that it might include more `blocks` over time to increase and improve functionalities from other community projects.
+
+To deploy the application you can simply press below `Deploy button` or you can navigate to the [Balena Hub apps page](https://hub.balena.io/apps/2064752/video-surveillance).
+
+[![deploy with balena](https://balena.io/deploy.svg)](https://dashboard.balena-cloud.com/deploy?repoUrl=https://github.com/kerberos-io/agent)
+
+You can find the source code, `balena.yaml` and `docker-compose.yaml` files in the [`balena-agent` repository](https://github.com/kerberos-io/balena-agent).

deployments/kubernetes/deployment-agent-volume.yml

@@ -21,7 +21,7 @@ spec:
 
       initContainers:
         - name: download-config
-          image: kerberos/agent:1b96d01
+          image: kerberos/agent:latest
           volumeMounts:
             - name: kerberos-data
               mountPath: /home/agent/data/config

machinery/.vscode/launch.json

@@ -10,7 +10,7 @@
             "request": "launch",
             "mode": "auto",
             "program": "main.go",
-            "args": ["run", "cameraname", "8080"],
+            "args": ["-action run"],
             "envFile": "${workspaceFolder}/.env",
             "buildFlags": "--tags dynamic",
         },

machinery/data/config/config.json

@@ -95,7 +95,7 @@
 	"s3": {
 		"proxyuri": "http://proxy.kerberos.io",
 		"bucket": "kerberosaccept",
-		"region": "eu-west1"
+		"region": "eu-west-1"
 	},
 	"kstorage": {},
 	"dropbox": {},
@@ -112,4 +112,4 @@
 	"hub_private_key": "",
 	"hub_site": "",
 	"condition_uri": ""
-}
+}

machinery/main.go

@@ -2,6 +2,7 @@ package main
 
 import (
 	"context"
+	"flag"
 	"os"
 	"time"
 
@@ -49,10 +50,23 @@ func main() {
 	}
 
 	// Start the show ;)
-	action := os.Args[1]
+	// We'll parse the flags (named variables), and start the agent.
+
+	var action string
+	var configDirectory string
+	var name string
+	var port string
+	var timeout string
+
+	flag.StringVar(&action, "action", "version", "Tell us what you want do 'run' or 'version'")
+	flag.StringVar(&configDirectory, "config", ".", "Where is the configuration stored")
+	flag.StringVar(&name, "name", "agent", "Provide a name for the agent")
+	flag.StringVar(&port, "port", "80", "On which port should the agent run")
+	flag.StringVar(&timeout, "timeout", "2000", "Number of milliseconds to wait for the ONVIF discovery to complete")
+	flag.Parse()
 
 	timezone, _ := time.LoadLocation("CET")
-	log.Log.Init(timezone)
+	log.Log.Init(configDirectory, timezone)
 
 	switch action {
 
@@ -60,14 +74,10 @@ func main() {
 		log.Log.Info("You are currrently running Kerberos Agent " + VERSION)
 
 	case "discover":
-		timeout := os.Args[2]
 		log.Log.Info(timeout)
 
 	case "run":
 		{
-			name := os.Args[2]
-			port := os.Args[3]
-
 			// Print Kerberos.io ASCII art
 			utils.PrintASCIIArt()
 
@@ -82,7 +92,7 @@ func main() {
 			configuration.Port = port
 
 			// Open this configuration either from Kerberos Agent or Kerberos Factory.
-			components.OpenConfig(&configuration)
+			components.OpenConfig(configDirectory, &configuration)
 
 			// We will override the configuration with the environment variables
 			components.OverrideWithEnvironmentVariables(&configuration)
@@ -92,18 +102,18 @@ func main() {
 
 			// Check the folder permissions, it might be that we do not have permissions to write
 			// recordings, update the configuration or save snapshots.
-			utils.CheckDataDirectoryPermissions()
+			utils.CheckDataDirectoryPermissions(configDirectory)
 
 			// Set timezone
 			timezone, _ := time.LoadLocation(configuration.Config.Timezone)
-			log.Log.Init(timezone)
+			log.Log.Init(configDirectory, timezone)
 
 			// Check if we have a device Key or not, if not
 			// we will generate one.
 			if configuration.Config.Key == "" {
 				key := utils.RandStringBytesMaskImpr(30)
 				configuration.Config.Key = key
-				err := components.StoreConfig(configuration.Config)
+				err := components.StoreConfig(configDirectory, configuration.Config)
 				if err == nil {
 					log.Log.Info("Main: updated unique key for agent to: " + key)
 				} else {
@@ -121,10 +131,10 @@ func main() {
 				CancelContext:   &cancel,
 				HandleBootstrap: make(chan string, 1),
 			}
-			go components.Bootstrap(&configuration, &communication)
+			go components.Bootstrap(configDirectory, &configuration, &communication)
 
 			// Start the REST API.
-			routers.StartWebserver(&configuration, &communication)
+			routers.StartWebserver(configDirectory, &configuration, &communication)
 		}
 	default:
 		log.Log.Error("Main: Sorry I don't understand :(")

machinery/src/capture/main.go

@@ -17,7 +17,7 @@ import (
 	"github.com/kerberos-io/joy4/av"
 )
 
-func CleanupRecordingDirectory(configuration *models.Configuration) {
+func CleanupRecordingDirectory(configDirectory string, configuration *models.Configuration) {
 	autoClean := configuration.Config.AutoClean
 	if autoClean == "true" {
 		maxSize := configuration.Config.MaxDirectorySize
@@ -25,7 +25,7 @@ func CleanupRecordingDirectory(configuration *models.Configuration) {
 			maxSize = 300
 		}
 		// Total size of the recording directory.
-		recordingsDirectory := "./data/recordings"
+		recordingsDirectory := configDirectory + "/data/recordings"
 		size, err := utils.DirSize(recordingsDirectory)
 		if err == nil {
 			sizeInMB := size / 1000 / 1000
@@ -51,7 +51,7 @@ func CleanupRecordingDirectory(configuration *models.Configuration) {
 	}
 }
 
-func HandleRecordStream(queue *pubsub.Queue, configuration *models.Configuration, communication *models.Communication, streams []av.CodecData) {
+func HandleRecordStream(queue *pubsub.Queue, configDirectory string, configuration *models.Configuration, communication *models.Communication, streams []av.CodecData) {
 
 	config := configuration.Config
 
@@ -134,13 +134,13 @@ func HandleRecordStream(queue *pubsub.Queue, configuration *models.Configuration
 					}
 
 					// Create a symbol link.
-					fc, _ := os.Create("./data/cloud/" + name)
+					fc, _ := os.Create(configDirectory + "/data/cloud/" + name)
 					fc.Close()
 
 					recordingStatus = "idle"
 
 					// Clean up the recording directory if necessary.
-					CleanupRecordingDirectory(configuration)
+					CleanupRecordingDirectory(configDirectory, configuration)
 				}
 
 				// If not yet started and a keyframe, let's make a recording
@@ -192,7 +192,7 @@ func HandleRecordStream(queue *pubsub.Queue, configuration *models.Configuration
 						"769"
 
 					name = s + ".mp4"
-					fullName = "./data/recordings/" + name
+					fullName = configDirectory + "/data/recordings/" + name
 
 					// Running...
 					log.Log.Info("Recording started")
@@ -259,7 +259,7 @@ func HandleRecordStream(queue *pubsub.Queue, configuration *models.Configuration
 					}
 
 					// Create a symbol link.
-					fc, _ := os.Create("./data/cloud/" + name)
+					fc, _ := os.Create(configDirectory + "/data/cloud/" + name)
 					fc.Close()
 
 					recordingStatus = "idle"
@@ -315,7 +315,7 @@ func HandleRecordStream(queue *pubsub.Queue, configuration *models.Configuration
 					"769"
 
 				name := s + ".mp4"
-				fullName := "./data/recordings/" + name
+				fullName := configDirectory + "/data/recordings/" + name
 
 				// Running...
 				log.Log.Info("HandleRecordStream: Recording started")
@@ -406,11 +406,11 @@ func HandleRecordStream(queue *pubsub.Queue, configuration *models.Configuration
 				}
 
 				// Create a symbol linc.
-				fc, _ := os.Create("./data/cloud/" + name)
+				fc, _ := os.Create(configDirectory + "/data/cloud/" + name)
 				fc.Close()
 
 				// Clean up the recording directory if necessary.
-				CleanupRecordingDirectory(configuration)
+				CleanupRecordingDirectory(configDirectory, configuration)
 			}
 		}
 

machinery/src/cloud/Cloud.go

@@ -15,14 +15,12 @@ import (
 	"github.com/gin-gonic/gin"
 	"github.com/golang-module/carbon/v2"
 	"github.com/kerberos-io/joy4/av/pubsub"
-	"github.com/minio/minio-go/v6"
 
 	mqtt "github.com/eclipse/paho.mqtt.golang"
 	av "github.com/kerberos-io/joy4/av"
 	"github.com/kerberos-io/joy4/cgo/ffmpeg"
 
 	"net/http"
-	"net/url"
 	"strconv"
 	"time"
 
@@ -34,8 +32,8 @@ import (
 	"github.com/kerberos-io/agent/machinery/src/webrtc"
 )
 
-func PendingUpload() {
-	ff, err := utils.ReadDirectory("./data/cloud/")
+func PendingUpload(configDirectory string) {
+	ff, err := utils.ReadDirectory(configDirectory + "/data/cloud/")
 	if err == nil {
 		for _, f := range ff {
 			log.Log.Info(f.Name())
@@ -43,12 +41,12 @@ func PendingUpload() {
 	}
 }
 
-func HandleUpload(configuration *models.Configuration, communication *models.Communication) {
+func HandleUpload(configDirectory string, configuration *models.Configuration, communication *models.Communication) {
 
 	log.Log.Debug("HandleUpload: started")
 
 	config := configuration.Config
-	watchDirectory := "./data/cloud/"
+	watchDirectory := configDirectory + "/data/cloud/"
 
 	if config.Offline == "true" {
 		log.Log.Debug("HandleUpload: stopping as Offline is enabled.")
@@ -85,9 +83,9 @@ func HandleUpload(configuration *models.Configuration, communication *models.Com
 					uploaded := false
 					configured := false
 					err = nil
-					if config.Cloud == "s3" {
-						uploaded, configured, err = UploadS3(configuration, fileName)
-					} else if config.Cloud == "kstorage" {
+					if config.Cloud == "s3" || config.Cloud == "kerberoshub" {
+						uploaded, configured, err = UploadKerberosHub(configuration, fileName)
+					} else if config.Cloud == "kstorage" || config.Cloud == "kerberosvault" {
 						uploaded, configured, err = UploadKerberosVault(configuration, fileName)
 					} else if config.Cloud == "dropbox" {
 						uploaded, configured, err = UploadDropbox(configuration, fileName)
@@ -103,6 +101,13 @@ func HandleUpload(configuration *models.Configuration, communication *models.Com
 						// Todo: implement ftp upload
 					} else if config.Cloud == "sftp" {
 						// Todo: implement sftp upload
+					} else if config.Cloud == "aws" {
+						// Todo: need to be updated, was previously used for hub.
+						uploaded, configured, err = UploadS3(configuration, fileName)
+					} else if config.Cloud == "azure" {
+						// Todo: implement azure upload
+					} else if config.Cloud == "google" {
+						// Todo: implement google upload
 					}
 					// And so on... (have a look here -> https://github.com/kerberos-io/agent/issues/95)
 
@@ -117,7 +122,7 @@ func HandleUpload(configuration *models.Configuration, communication *models.Com
 						// Check if we need to remove the original recording
 						// removeAfterUpload is set to false by default
 						if config.RemoveAfterUpload == "true" {
-							err := os.Remove("./data/recordings/" + fileName)
+							err := os.Remove(configDirectory + "/data/recordings/" + fileName)
 							if err != nil {
 								log.Log.Error("HandleUpload: " + err.Error())
 							}
@@ -348,7 +353,16 @@ loop:
 				req, _ := http.NewRequest("POST", url, buffy)
 				req.Header.Set("Content-Type", "application/json")
 
-				client := &http.Client{}
+				var client *http.Client
+				if os.Getenv("AGENT_TLS_INSECURE") == "true" {
+					tr := &http.Transport{
+						TLSClientConfig: &tls.Config{InsecureSkipVerify: true},
+					}
+					client = &http.Client{Transport: tr}
+				} else {
+					client = &http.Client{}
+				}
+
 				resp, err := client.Do(req)
 				if resp != nil {
 					resp.Body.Close()
@@ -357,7 +371,9 @@ loop:
 					communication.CloudTimestamp.Store(time.Now().Unix())
 					log.Log.Info("HandleHeartBeat: (200) Heartbeat received by Kerberos Hub.")
 				} else {
-					communication.CloudTimestamp.Store(0)
+					if communication.CloudTimestamp != nil && communication.CloudTimestamp.Load() != nil {
+						communication.CloudTimestamp.Store(int64(0))
+					}
 					log.Log.Error("HandleHeartBeat: (400) Something went wrong while sending to Kerberos Hub.")
 				}
 
@@ -368,8 +384,6 @@ loop:
 					buffy = bytes.NewBuffer(jsonStr)
 					req, _ = http.NewRequest("POST", vaultURI+"/devices/heartbeat", buffy)
 					req.Header.Set("Content-Type", "application/json")
-
-					client = &http.Client{}
 					resp, err = client.Do(req)
 					if resp != nil {
 						resp.Body.Close()
@@ -536,15 +550,23 @@ func VerifyHub(c *gin.Context) {
 	err := c.BindJSON(&config)
 
 	if err == nil {
-		hubKey := config.HubKey
 		hubURI := config.HubURI
+		publicKey := config.HubKey
+		privateKey := config.HubPrivateKey
 
-		content := []byte(`{"message": "fake-message"}`)
-		body := bytes.NewReader(content)
-		req, err := http.NewRequest("POST", hubURI+"/queue/test", body)
+		req, err := http.NewRequest("POST", hubURI+"/subscription/verify", nil)
 		if err == nil {
-			req.Header.Set("X-Kerberos-Cloud-Key", hubKey)
-			client := &http.Client{}
+			req.Header.Set("X-Kerberos-Hub-PublicKey", publicKey)
+			req.Header.Set("X-Kerberos-Hub-PrivateKey", privateKey)
+			var client *http.Client
+			if os.Getenv("AGENT_TLS_INSECURE") == "true" {
+				tr := &http.Transport{
+					TLSClientConfig: &tls.Config{InsecureSkipVerify: true},
+				}
+				client = &http.Client{Transport: tr}
+			} else {
+				client = &http.Client{}
+			}
 
 			resp, err := client.Do(req)
 			if err == nil {
@@ -592,7 +614,7 @@ func VerifyHub(c *gin.Context) {
 // @Summary Will verify the persistence.
 // @Description Will verify the persistence.
 // @Success 200 {object} models.APIResponse
-func VerifyPersistence(c *gin.Context) {
+func VerifyPersistence(c *gin.Context, configDirectory string) {
 
 	var config models.Config
 	err := c.BindJSON(&config)
@@ -600,88 +622,88 @@ func VerifyPersistence(c *gin.Context) {
 
 		if config.Cloud == "dropbox" {
 			VerifyDropbox(config, c)
-		} else if config.Cloud == "s3" {
+		} else if config.Cloud == "s3" || config.Cloud == "kerberoshub" {
 
-			// timestamp_microseconds_instanceName_regionCoordinates_numberOfChanges_token
-			// 1564859471_6-474162_oprit_577-283-727-375_1153_27.mp4
-			// - Timestamp
-			// - Size + - + microseconds
-			// - device
-			// - Region
-			// - Number of changes
-			// - Token
-
-			aws_access_key_id := config.S3.Publickey
-			aws_secret_access_key := config.S3.Secretkey
-			aws_region := config.S3.Region
-
-			// This is the new way ;)
-			if config.HubKey != "" {
-				aws_access_key_id = config.HubKey
-			}
-			if config.HubPrivateKey != "" {
-				aws_secret_access_key = config.HubPrivateKey
-			}
-
-			s3Client, err := minio.NewWithRegion("s3.amazonaws.com", aws_access_key_id, aws_secret_access_key, true, aws_region)
-			if err != nil {
+			if config.HubURI == "" ||
+				config.HubKey == "" ||
+				config.HubPrivateKey == "" ||
+				config.S3.Region == "" {
+				msg := "VerifyPersistence: Kerberos Hub not properly configured."
+				log.Log.Info(msg)
 				c.JSON(400, models.APIResponse{
-					Data: "Creation of Kerberos Hub connection failed: " + err.Error(),
+					Data: msg,
 				})
 			} else {
 
-				// Check if we need to use the proxy.
-				if config.S3.ProxyURI != "" {
-					var transport http.RoundTripper = &http.Transport{
-						Proxy: func(*http.Request) (*url.URL, error) {
-							return url.Parse(config.S3.ProxyURI)
-						},
-						TLSClientConfig: &tls.Config{InsecureSkipVerify: true},
-					}
-					s3Client.SetCustomTransport(transport)
+				// Open test-480p.mp4
+				file, err := os.Open(configDirectory + "/data/test-480p.mp4")
+				if err != nil {
+					msg := "VerifyPersistence: error reading test-480p.mp4: " + err.Error()
+					log.Log.Error(msg)
+					c.JSON(400, models.APIResponse{
+						Data: msg,
+					})
+				}
+				defer file.Close()
+
+				req, err := http.NewRequest("POST", config.HubURI+"/storage/upload", file)
+				if err != nil {
+					msg := "VerifyPersistence: error reading Kerberos Hub HEAD request, " + config.HubURI + "/storage: " + err.Error()
+					log.Log.Error(msg)
+					c.JSON(400, models.APIResponse{
+						Data: msg,
+					})
 				}
 
-				deviceKey := "fake-key"
-				devicename := "justatest"
-				coordinates := "200-200-400-400"
-				eventToken := "769"
-
 				timestamp := time.Now().Unix()
-				fileName := strconv.FormatInt(timestamp, 10) + "_6-967003_justatest_200-200-400-400_24_769.mp4"
-				content := []byte("test-file")
-				body := bytes.NewReader(content)
+				fileName := strconv.FormatInt(timestamp, 10) +
+					"_6-967003_" + config.Name + "_200-200-400-400_24_769.mp4"
+				req.Header.Set("X-Kerberos-Storage-FileName", fileName)
+				req.Header.Set("X-Kerberos-Storage-Capture", "IPCamera")
+				req.Header.Set("X-Kerberos-Storage-Device", config.Key)
+				req.Header.Set("X-Kerberos-Hub-PublicKey", config.HubKey)
+				req.Header.Set("X-Kerberos-Hub-PrivateKey", config.HubPrivateKey)
+				req.Header.Set("X-Kerberos-Hub-Region", config.S3.Region)
 
-				n, err := s3Client.PutObject(config.S3.Bucket,
-					config.S3.Username+"/"+fileName,
-					body,
-					body.Size(),
-					minio.PutObjectOptions{
-						ContentType:  "video/mp4",
-						StorageClass: "ONEZONE_IA",
-						UserMetadata: map[string]string{
-							"event-timestamp":         strconv.FormatInt(timestamp, 10),
-							"event-microseconds":      deviceKey,
-							"event-instancename":      devicename,
-							"event-regioncoordinates": coordinates,
-							"event-numberofchanges":   deviceKey,
-							"event-token":             eventToken,
-							"productid":               deviceKey,
-							"publickey":               aws_access_key_id,
-							"uploadtime":              "now",
-						},
-					})
-
-				if err != nil {
-					c.JSON(400, models.APIResponse{
-						Data: "Upload of fake recording failed: " + err.Error(),
-					})
+				var client *http.Client
+				if os.Getenv("AGENT_TLS_INSECURE") == "true" {
+					tr := &http.Transport{
+						TLSClientConfig: &tls.Config{InsecureSkipVerify: true},
+					}
+					client = &http.Client{Transport: tr}
 				} else {
-					c.JSON(200, models.APIResponse{
-						Data: "Upload Finished: file has been uploaded to bucket: " + strconv.FormatInt(n, 10),
+					client = &http.Client{}
+				}
+
+				resp, err := client.Do(req)
+				if resp != nil {
+					defer resp.Body.Close()
+				}
+
+				if err == nil && resp != nil {
+					if resp.StatusCode == 200 {
+						msg := "VerifyPersistence: Upload allowed using the credentials provided (" + config.HubKey + ", " + config.HubPrivateKey + ")"
+						log.Log.Info(msg)
+						c.JSON(200, models.APIResponse{
+							Data: msg,
+						})
+					} else {
+						msg := "VerifyPersistence: Upload NOT allowed using the credentials provided (" + config.HubKey + ", " + config.HubPrivateKey + ")"
+						log.Log.Info(msg)
+						c.JSON(400, models.APIResponse{
+							Data: msg,
+						})
+					}
+				} else {
+					msg := "VerifyPersistence: Error creating Kerberos Hub request"
+					log.Log.Info(msg)
+					c.JSON(400, models.APIResponse{
+						Data: msg,
 					})
 				}
 			}
-		} else if config.Cloud == "kstorage" {
+
+		} else if config.Cloud == "kstorage" || config.Cloud == "kerberosvault" {
 
 			uri := config.KStorage.URI
 			accessKey := config.KStorage.AccessKey
@@ -690,10 +712,18 @@ func VerifyPersistence(c *gin.Context) {
 			provider := config.KStorage.Provider
 
 			if err == nil && uri != "" && accessKey != "" && secretAccessKey != "" {
-				var postData = []byte(`{"title":"Buy cheese and bread for breakfast."}`)
-				client := &http.Client{}
-				req, err := http.NewRequest("POST", uri+"/ping", bytes.NewReader(postData))
 
+				var client *http.Client
+				if os.Getenv("AGENT_TLS_INSECURE") == "true" {
+					tr := &http.Transport{
+						TLSClientConfig: &tls.Config{InsecureSkipVerify: true},
+					}
+					client = &http.Client{Transport: tr}
+				} else {
+					client = &http.Client{}
+				}
+
+				req, err := http.NewRequest("POST", uri+"/ping", nil)
 				req.Header.Add("X-Kerberos-Storage-AccessKey", accessKey)
 				req.Header.Add("X-Kerberos-Storage-SecretAccessKey", secretAccessKey)
 				resp, err := client.Do(req)
@@ -705,33 +735,44 @@ func VerifyPersistence(c *gin.Context) {
 
 						if provider != "" || directory != "" {
 
-							hubKey := config.KStorage.CloudKey
-							// This is the new way ;)
-							if config.HubKey != "" {
-								hubKey = config.HubKey
-							}
-
 							// Generate a random name.
 							timestamp := time.Now().Unix()
 							fileName := strconv.FormatInt(timestamp, 10) +
-								"_6-967003_justatest_200-200-400-400_24_769.mp4"
-							content := []byte("test-file")
-							body := bytes.NewReader(content)
-							//fileSize := int64(len(content))
+								"_6-967003_" + config.Name + "_200-200-400-400_24_769.mp4"
 
-							req, err := http.NewRequest("POST", uri+"/storage", body)
+							// Open test-480p.mp4
+							file, err := os.Open(configDirectory + "/test-480p.mp4")
+							if err != nil {
+								msg := "VerifyPersistence: error reading test-480p.mp4: " + err.Error()
+								log.Log.Error(msg)
+								c.JSON(400, models.APIResponse{
+									Data: msg,
+								})
+							}
+							defer file.Close()
+
+							req, err := http.NewRequest("POST", uri+"/storage", file)
 							if err == nil {
 
 								req.Header.Set("Content-Type", "video/mp4")
-								req.Header.Set("X-Kerberos-Storage-CloudKey", hubKey)
+								req.Header.Set("X-Kerberos-Storage-CloudKey", config.HubKey)
 								req.Header.Set("X-Kerberos-Storage-AccessKey", accessKey)
 								req.Header.Set("X-Kerberos-Storage-SecretAccessKey", secretAccessKey)
 								req.Header.Set("X-Kerberos-Storage-Provider", provider)
 								req.Header.Set("X-Kerberos-Storage-FileName", fileName)
-								req.Header.Set("X-Kerberos-Storage-Device", "test")
+								req.Header.Set("X-Kerberos-Storage-Device", config.Key)
 								req.Header.Set("X-Kerberos-Storage-Capture", "IPCamera")
 								req.Header.Set("X-Kerberos-Storage-Directory", directory)
-								client := &http.Client{}
+
+								var client *http.Client
+								if os.Getenv("AGENT_TLS_INSECURE") == "true" {
+									tr := &http.Transport{
+										TLSClientConfig: &tls.Config{InsecureSkipVerify: true},
+									}
+									client = &http.Client{Transport: tr}
+								} else {
+									client = &http.Client{}
+								}
 
 								resp, err := client.Do(req)
 
@@ -744,41 +785,45 @@ func VerifyPersistence(c *gin.Context) {
 												c.JSON(200, body)
 											} else {
 												c.JSON(400, models.APIResponse{
-													Data: "Something went wrong while verifying your persistence settings. Make sure your provider is the same as the storage provider in your Kerberos Vault, and the relevant storage provider is configured properly.",
+													Data: "VerifyPersistence: Something went wrong while verifying your persistence settings. Make sure your provider is the same as the storage provider in your Kerberos Vault, and the relevant storage provider is configured properly.",
 												})
 											}
 										}
 									}
 								} else {
 									c.JSON(400, models.APIResponse{
-										Data: "Upload of fake recording failed: " + err.Error(),
+										Data: "VerifyPersistence: Upload of fake recording failed: " + err.Error(),
 									})
 								}
 							} else {
 								c.JSON(400, models.APIResponse{
-									Data: "Something went wrong while creating /storage POST request." + err.Error(),
+									Data: "VerifyPersistence: Something went wrong while creating /storage POST request." + err.Error(),
 								})
 							}
 						} else {
 							c.JSON(400, models.APIResponse{
-								Data: "Provider and/or directory is missing from the request.",
+								Data: "VerifyPersistence: Provider and/or directory is missing from the request.",
 							})
 						}
 					} else {
 						c.JSON(400, models.APIResponse{
-							Data: "Something went wrong while verifying storage credentials: " + string(body),
+							Data: "VerifyPersistence: Something went wrong while verifying storage credentials: " + string(body),
 						})
 					}
 				} else {
 					c.JSON(400, models.APIResponse{
-						Data: "Something went wrong while verifying storage credentials:" + err.Error(),
+						Data: "VerifyPersistence: Something went wrong while verifying storage credentials:" + err.Error(),
 					})
 				}
+			} else {
+				c.JSON(400, models.APIResponse{
+					Data: "VerifyPersistence: please fill-in the required Kerberos Vault credentials.",
+				})
 			}
 		}
 	} else {
 		c.JSON(400, models.APIResponse{
-			Data: "No persistence was specified, so do not know what to verify:" + err.Error(),
+			Data: "VerifyPersistence: No persistence was specified, so do not know what to verify:" + err.Error(),
 		})
 	}
 }

machinery/src/cloud/KerberosHub.go

@@ -0,0 +1,131 @@
+package cloud
+
+import (
+	"crypto/tls"
+	"errors"
+	"io/ioutil"
+	"net/http"
+	"os"
+
+	"github.com/kerberos-io/agent/machinery/src/log"
+	"github.com/kerberos-io/agent/machinery/src/models"
+)
+
+func UploadKerberosHub(configuration *models.Configuration, fileName string) (bool, bool, error) {
+	config := configuration.Config
+
+	if config.HubURI == "" ||
+		config.HubKey == "" ||
+		config.HubPrivateKey == "" ||
+		config.S3.Region == "" {
+		err := "UploadKerberosHub: Kerberos Hub not properly configured."
+		log.Log.Info(err)
+		return false, false, errors.New(err)
+	}
+
+	// timestamp_microseconds_instanceName_regionCoordinates_numberOfChanges_token
+	// 1564859471_6-474162_oprit_577-283-727-375_1153_27.mp4
+	// - Timestamp
+	// - Size + - + microseconds
+	// - device
+	// - Region
+	// - Number of changes
+	// - Token
+
+	log.Log.Info("UploadKerberosHub: Uploading to Kerberos Hub (" + config.HubURI + ")")
+	log.Log.Info("UploadKerberosHub: Upload started for " + fileName)
+	fullname := "data/recordings/" + fileName
+
+	// Check if we still have the file otherwise we abort the request.
+	file, err := os.OpenFile(fullname, os.O_RDWR, 0755)
+	if file != nil {
+		defer file.Close()
+	}
+	if err != nil {
+		err := "UploadKerberosHub: Upload Failed, file doesn't exists anymore."
+		log.Log.Info(err)
+		return false, true, errors.New(err)
+	}
+
+	// Check if we are allowed to upload to the hub with these credentials.
+	// There might be different reasons like (muted, read-only..)
+	req, err := http.NewRequest("HEAD", config.HubURI+"/storage/upload", nil)
+	if err != nil {
+		errorMessage := "UploadKerberosHub: error reading HEAD request, " + config.HubURI + "/storage: " + err.Error()
+		log.Log.Error(errorMessage)
+		return false, true, errors.New(errorMessage)
+	}
+
+	req.Header.Set("X-Kerberos-Storage-FileName", fileName)
+	req.Header.Set("X-Kerberos-Storage-Capture", "IPCamera")
+	req.Header.Set("X-Kerberos-Storage-Device", config.Key)
+	req.Header.Set("X-Kerberos-Hub-PublicKey", config.HubKey)
+	req.Header.Set("X-Kerberos-Hub-PrivateKey", config.HubPrivateKey)
+	req.Header.Set("X-Kerberos-Hub-Region", config.S3.Region)
+
+	var client *http.Client
+	if os.Getenv("AGENT_TLS_INSECURE") == "true" {
+		tr := &http.Transport{
+			TLSClientConfig: &tls.Config{InsecureSkipVerify: true},
+		}
+		client = &http.Client{Transport: tr}
+	} else {
+		client = &http.Client{}
+	}
+
+	resp, err := client.Do(req)
+	if resp != nil {
+		defer resp.Body.Close()
+	}
+
+	if err == nil {
+		if resp != nil {
+			if err == nil {
+				if resp.StatusCode == 200 {
+					log.Log.Info("UploadKerberosHub: Upload allowed using the credentials provided (" + config.HubKey + ", " + config.HubPrivateKey + ")")
+				} else {
+					log.Log.Info("UploadKerberosHub: Upload NOT allowed using the credentials provided (" + config.HubKey + ", " + config.HubPrivateKey + ")")
+					return false, true, nil
+				}
+			}
+		}
+	}
+
+	// Now we know we are allowed to upload to the hub, we can start uploading.
+	req, err = http.NewRequest("POST", config.HubURI+"/storage/upload", file)
+	if err != nil {
+		errorMessage := "UploadKerberosHub: error reading POST request, " + config.KStorage.URI + "/storage/upload: " + err.Error()
+		log.Log.Error(errorMessage)
+		return false, true, errors.New(errorMessage)
+	}
+	req.Header.Set("Content-Type", "video/mp4")
+	req.Header.Set("X-Kerberos-Storage-FileName", fileName)
+	req.Header.Set("X-Kerberos-Storage-Capture", "IPCamera")
+	req.Header.Set("X-Kerberos-Storage-Device", config.Key)
+	req.Header.Set("X-Kerberos-Hub-PublicKey", config.HubKey)
+	req.Header.Set("X-Kerberos-Hub-PrivateKey", config.HubPrivateKey)
+	req.Header.Set("X-Kerberos-Hub-Region", config.S3.Region)
+	resp, err = client.Do(req)
+	if resp != nil {
+		defer resp.Body.Close()
+	}
+
+	if err == nil {
+		if resp != nil {
+			body, err := ioutil.ReadAll(resp.Body)
+			if err == nil {
+				if resp.StatusCode == 200 {
+					log.Log.Info("UploadKerberosHub: Upload Finished, " + resp.Status + ".")
+					return true, true, nil
+				} else {
+					log.Log.Info("UploadKerberosHub: Upload Failed, " + resp.Status + ", " + string(body))
+					return false, true, nil
+				}
+			}
+		}
+	}
+
+	errorMessage := "UploadKerberosHub: Upload Failed, " + err.Error()
+	log.Log.Info(errorMessage)
+	return false, true, errors.New(errorMessage)
+}

machinery/src/cloud/KerberosVault.go

@@ -1,6 +1,7 @@
 package cloud
 
 import (
+	"crypto/tls"
 	"errors"
 	"io/ioutil"
 	"net/http"
@@ -67,7 +68,16 @@ func UploadKerberosVault(configuration *models.Configuration, fileName string) (
 	req.Header.Set("X-Kerberos-Storage-Device", config.Key)
 	req.Header.Set("X-Kerberos-Storage-Capture", "IPCamera")
 	req.Header.Set("X-Kerberos-Storage-Directory", config.KStorage.Directory)
-	client := &http.Client{}
+
+	var client *http.Client
+	if os.Getenv("AGENT_TLS_INSECURE") == "true" {
+		tr := &http.Transport{
+			TLSClientConfig: &tls.Config{InsecureSkipVerify: true},
+		}
+		client = &http.Client{Transport: tr}
+	} else {
+		client = &http.Client{}
+	}
 
 	resp, err := client.Do(req)
 	if resp != nil {

machinery/src/components/Config.go

@@ -20,14 +20,14 @@ import (
 	"go.mongodb.org/mongo-driver/bson"
 )
 
-func GetImageFromFilePath() (image.Image, error) {
-	snapshotDirectory := "./data/snapshots"
+func GetImageFromFilePath(configDirectory string) (image.Image, error) {
+	snapshotDirectory := configDirectory + "/data/snapshots"
 	files, err := ioutil.ReadDir(snapshotDirectory)
 	if err == nil && len(files) > 1 {
 		sort.Slice(files, func(i, j int) bool {
 			return files[i].ModTime().Before(files[j].ModTime())
 		})
-		filePath := "./data/snapshots/" + files[1].Name()
+		filePath := configDirectory + "/data/snapshots/" + files[1].Name()
 		f, err := os.Open(filePath)
 		if err != nil {
 			return nil, err
@@ -42,11 +42,11 @@ func GetImageFromFilePath() (image.Image, error) {
 // ReadUserConfig Reads the user configuration of the Kerberos Open Source instance.
 // This will return a models.User struct including the username, password,
 // selected language, and if the installation was completed or not.
-func ReadUserConfig() (userConfig models.User) {
+func ReadUserConfig(configDirectory string) (userConfig models.User) {
 	for {
-		jsonFile, err := os.Open("./data/config/user.json")
+		jsonFile, err := os.Open(configDirectory + "/data/config/user.json")
 		if err != nil {
-			log.Log.Error("Config file is not found " + "./data/config/user.json, trying again in 5s: " + err.Error())
+			log.Log.Error("Config file is not found " + configDirectory + "/data/config/user.json, trying again in 5s: " + err.Error())
 			time.Sleep(5 * time.Second)
 		} else {
 			log.Log.Info("Successfully Opened user.json")
@@ -66,7 +66,7 @@ func ReadUserConfig() (userConfig models.User) {
 	return
 }
 
-func OpenConfig(configuration *models.Configuration) {
+func OpenConfig(configDirectory string, configuration *models.Configuration) {
 
 	// We are checking which deployment this is running, so we can load
 	// into the configuration as expected.
@@ -146,9 +146,9 @@ func OpenConfig(configuration *models.Configuration) {
 
 		// Open device config
 		for {
-			jsonFile, err := os.Open("./data/config/config.json")
+			jsonFile, err := os.Open(configDirectory + "/data/config/config.json")
 			if err != nil {
-				log.Log.Error("Config file is not found " + "./data/config/config.json" + ", trying again in 5s.")
+				log.Log.Error("Config file is not found " + configDirectory + "/data/config/config.json" + ", trying again in 5s.")
 				time.Sleep(5 * time.Second)
 			} else {
 				log.Log.Info("Successfully Opened config.json from " + configuration.Name)
@@ -401,12 +401,12 @@ func OverrideWithEnvironmentVariables(configuration *models.Configuration) {
 			case "AGENT_HUB_PRIVATE_KEY":
 				configuration.Config.HubPrivateKey = value
 				break
-			case "AGENT_HUB_USERNAME":
-				configuration.Config.S3.Username = value
-				break
 			case "AGENT_HUB_SITE":
 				configuration.Config.HubSite = value
 				break
+			case "AGENT_HUB_REGION":
+				configuration.Config.S3.Region = value
+				break
 
 			/* When storing in a Kerberos Vault */
 			case "AGENT_KERBEROSVAULT_URI":
@@ -437,11 +437,11 @@ func OverrideWithEnvironmentVariables(configuration *models.Configuration) {
 	}
 }
 
-func SaveConfig(config models.Config, configuration *models.Configuration, communication *models.Communication) error {
+func SaveConfig(configDirectory string, config models.Config, configuration *models.Configuration, communication *models.Communication) error {
 	if !communication.IsConfiguring.IsSet() {
 		communication.IsConfiguring.Set()
 
-		err := StoreConfig(config)
+		err := StoreConfig(configDirectory, config)
 		if err != nil {
 			communication.IsConfiguring.UnSet()
 			return err
@@ -462,7 +462,7 @@ func SaveConfig(config models.Config, configuration *models.Configuration, commu
 	}
 }
 
-func StoreConfig(config models.Config) error {
+func StoreConfig(configDirectory string, config models.Config) error {
 	// Save into database
 	if os.Getenv("DEPLOYMENT") == "factory" || os.Getenv("MACHINERY_ENVIRONMENT") == "kubernetes" {
 		// Write to mongodb
@@ -484,7 +484,7 @@ func StoreConfig(config models.Config) error {
 		// Save into file
 	} else if os.Getenv("DEPLOYMENT") == "" || os.Getenv("DEPLOYMENT") == "agent" {
 		res, _ := json.MarshalIndent(config, "", "\t")
-		err := ioutil.WriteFile("./data/config/config.json", res, 0644)
+		err := ioutil.WriteFile(configDirectory+"/data/config/config.json", res, 0644)
 		return err
 	}
 

machinery/src/components/Kerberos.go

@@ -23,7 +23,7 @@ import (
 	"github.com/tevino/abool"
 )
 
-func Bootstrap(configuration *models.Configuration, communication *models.Communication) {
+func Bootstrap(configDirectory string, configuration *models.Configuration, communication *models.Communication) {
 	log.Log.Debug("Bootstrap: started")
 
 	// We will keep track of the Kerberos Agent up time
@@ -79,7 +79,7 @@ func Bootstrap(configuration *models.Configuration, communication *models.Commun
 	for {
 
 		// This will blocking until receiving a signal to be restarted, reconfigured, stopped, etc.
-		status := RunAgent(configuration, communication, mqttClient, uptimeStart, cameraSettings, decoder, subDecoder)
+		status := RunAgent(configDirectory, configuration, communication, mqttClient, uptimeStart, cameraSettings, decoder, subDecoder)
 
 		if status == "stop" {
 			break
@@ -87,7 +87,7 @@ func Bootstrap(configuration *models.Configuration, communication *models.Commun
 
 		if status == "not started" {
 			// We will re open the configuration, might have changed :O!
-			OpenConfig(configuration)
+			OpenConfig(configDirectory, configuration)
 			// We will override the configuration with the environment variables
 			OverrideWithEnvironmentVariables(configuration)
 		}
@@ -107,7 +107,7 @@ func Bootstrap(configuration *models.Configuration, communication *models.Commun
 	log.Log.Debug("Bootstrap: finished")
 }
 
-func RunAgent(configuration *models.Configuration, communication *models.Communication, mqttClient mqtt.Client, uptimeStart time.Time, cameraSettings *models.Camera, decoder *ffmpeg.VideoDecoder, subDecoder *ffmpeg.VideoDecoder) string {
+func RunAgent(configDirectory string, configuration *models.Configuration, communication *models.Communication, mqttClient mqtt.Client, uptimeStart time.Time, cameraSettings *models.Camera, decoder *ffmpeg.VideoDecoder, subDecoder *ffmpeg.VideoDecoder) string {
 
 	log.Log.Debug("RunAgent: bootstrapping agent")
 	config := configuration.Config
@@ -260,10 +260,10 @@ func RunAgent(configuration *models.Configuration, communication *models.Communi
 		}
 
 		// Handle recording, will write an mp4 to disk.
-		go capture.HandleRecordStream(queue, configuration, communication, streams)
+		go capture.HandleRecordStream(queue, configDirectory, configuration, communication, streams)
 
 		// Handle Upload to cloud provider (Kerberos Hub, Kerberos Vault and others)
-		go cloud.HandleUpload(configuration, communication)
+		go cloud.HandleUpload(configDirectory, configuration, communication)
 
 		// Handle ONVIF actions
 		go onvif.HandleONVIFActions(configuration, communication)
@@ -284,7 +284,7 @@ func RunAgent(configuration *models.Configuration, communication *models.Communi
 		(*communication.CancelContext)()
 
 		// We will re open the configuration, might have changed :O!
-		OpenConfig(configuration)
+		OpenConfig(configDirectory, configuration)
 
 		// We will override the configuration with the environment variables
 		OverrideWithEnvironmentVariables(configuration)

machinery/src/computervision/main.go

@@ -165,7 +165,7 @@ func ProcessMotion(motionCursor *pubsub.QueueCursor, configuration *models.Confi
 					if detectMotion && isPixelChangeThresholdReached {
 
 						// If offline mode is disabled, send a message to the hub
-						if config.Offline == "false" {
+						if config.Offline != "true" {
 							if mqttClient != nil {
 								if key != "" {
 									mqttClient.Publish("kerberos/"+key+"/device/"+config.Key+"/motion", 2, false, "motion")

machinery/src/database/main.go

@@ -28,10 +28,10 @@ func New() *mongo.Client {
 	password := os.Getenv("MONGODB_PASSWORD")
 	authentication := "SCRAM-SHA-256"
 
-	ctx, cancel := context.WithTimeout(context.Background(), 10*time.Second)
-	defer cancel()
-
 	_init_ctx.Do(func() {
+		ctx, cancel := context.WithTimeout(context.Background(), 10*time.Second)
+		defer cancel()
+
 		_instance = new(DB)
 		mongodbURI := fmt.Sprintf("mongodb://%s:%s@%s", username, password, host)
 		if replicaset != "" {

machinery/src/log/main.go

@@ -21,7 +21,7 @@ var Log = Logging{
 
 var gologging = logging.MustGetLogger("gologger")
 
-func ConfigureGoLogging(timezone *time.Location) {
+func ConfigureGoLogging(configDirectory string, timezone *time.Location) {
 	// Logging
 	var format = logging.MustStringFormatter(
 		`%{color}%{time:15:04:05.000} %{shortfunc} ▶ %{level:.4s} %{id:03x}%{color:reset} %{message}`,
@@ -32,7 +32,7 @@ func ConfigureGoLogging(timezone *time.Location) {
 	stdBackend := logging.NewLogBackend(os.Stderr, "", 0)
 	stdBackendLeveled := logging.NewBackendFormatter(stdBackend, format)
 	fileBackend := logging.NewLogBackend(&lumberjack.Logger{
-		Filename: "./data/log/machinery.txt",
+		Filename: configDirectory + "/data/log/machinery.txt",
 		MaxSize:  2,    // megabytes
 		Compress: true, // disabled by default
 	}, "", 0)
@@ -75,10 +75,10 @@ type Logging struct {
 	Level  string
 }
 
-func (self *Logging) Init(timezone *time.Location) {
+func (self *Logging) Init(configDirectory string, timezone *time.Location) {
 	switch self.Logger {
 	case "go-logging":
-		ConfigureGoLogging(timezone)
+		ConfigureGoLogging(configDirectory, timezone)
 	case "logrus":
 		ConfigureLogrus(timezone)
 	default:

machinery/src/routers/http/Routes.go

@@ -17,7 +17,7 @@ import (
 	"github.com/kerberos-io/agent/machinery/src/utils"
 )
 
-func AddRoutes(r *gin.Engine, authMiddleware *jwt.GinJWTMiddleware, configuration *models.Configuration, communication *models.Communication) *gin.RouterGroup {
+func AddRoutes(r *gin.Engine, authMiddleware *jwt.GinJWTMiddleware, configDirectory string, configuration *models.Configuration, communication *models.Communication) *gin.RouterGroup {
 
 	r.GET("/ws", func(c *gin.Context) {
 		websocket.WebsocketHandler(c, communication)
@@ -40,7 +40,7 @@ func AddRoutes(r *gin.Engine, authMiddleware *jwt.GinJWTMiddleware, configuratio
 		var config models.Config
 		err := c.BindJSON(&config)
 		if err == nil {
-			err := components.SaveConfig(config, configuration, communication)
+			err := components.SaveConfig(configDirectory, config, configuration, communication)
 			if err == nil {
 				c.JSON(200, gin.H{
 					"data": "☄ Reconfiguring",
@@ -78,7 +78,7 @@ func AddRoutes(r *gin.Engine, authMiddleware *jwt.GinJWTMiddleware, configuratio
 			}
 
 			// The total number of recordings stored in the directory.
-			recordingDirectory := "./data/recordings"
+			recordingDirectory := configDirectory + "/data/recordings"
 			numberOfRecordings := utils.NumberOfMP4sInDirectory(recordingDirectory)
 
 			// All days stored in this agent.
@@ -115,7 +115,7 @@ func AddRoutes(r *gin.Engine, authMiddleware *jwt.GinJWTMiddleware, configuratio
 				if eventFilter.NumberOfElements == 0 {
 					eventFilter.NumberOfElements = 10
 				}
-				recordingDirectory := "./data/recordings"
+				recordingDirectory := configDirectory + "/data/recordings"
 				files, err := utils.ReadDirectory(recordingDirectory)
 				if err == nil {
 					events := utils.GetSortedDirectory(files)
@@ -137,7 +137,7 @@ func AddRoutes(r *gin.Engine, authMiddleware *jwt.GinJWTMiddleware, configuratio
 		})
 
 		api.GET("/days", func(c *gin.Context) {
-			recordingDirectory := "./data/recordings"
+			recordingDirectory := configDirectory + "/data/recordings"
 			files, err := utils.ReadDirectory(recordingDirectory)
 			if err == nil {
 				events := utils.GetSortedDirectory(files)
@@ -165,7 +165,7 @@ func AddRoutes(r *gin.Engine, authMiddleware *jwt.GinJWTMiddleware, configuratio
 			var config models.Config
 			err := c.BindJSON(&config)
 			if err == nil {
-				err := components.SaveConfig(config, configuration, communication)
+				err := components.SaveConfig(configDirectory, config, configuration, communication)
 				if err == nil {
 					c.JSON(200, gin.H{
 						"data": "☄ Reconfiguring",
@@ -205,7 +205,7 @@ func AddRoutes(r *gin.Engine, authMiddleware *jwt.GinJWTMiddleware, configuratio
 		})
 
 		api.POST("/persistence/verify", func(c *gin.Context) {
-			cloud.VerifyPersistence(c)
+			cloud.VerifyPersistence(c, configDirectory)
 		})
 
 		// Streaming handler
@@ -215,7 +215,7 @@ func AddRoutes(r *gin.Engine, authMiddleware *jwt.GinJWTMiddleware, configuratio
 				// We will only send an image once per second.
 				time.Sleep(time.Second * 1)
 				log.Log.Info("AddRoutes (/stream): reading from MJPEG stream")
-				img, err := components.GetImageFromFilePath()
+				img, err := components.GetImageFromFilePath(configDirectory)
 				return img, err
 			}
 			h := components.StartMotionJPEG(imageFunction, 80)

machinery/src/routers/http/Server.go

@@ -35,7 +35,7 @@ import (
 // @in header
 // @name Authorization
 
-func StartServer(configuration *models.Configuration, communication *models.Communication) {
+func StartServer(configDirectory string, configuration *models.Configuration, communication *models.Communication) {
 
 	// Initialize REST API
 	r := gin.Default()
@@ -57,12 +57,12 @@ func StartServer(configuration *models.Configuration, communication *models.Comm
 	}
 
 	// Add all routes
-	AddRoutes(r, authMiddleware, configuration, communication)
+	AddRoutes(r, authMiddleware, configDirectory, configuration, communication)
 
 	// Update environment variables
-	environmentVariables := "./www/env.js"
+	environmentVariables := configDirectory + "/www/env.js"
 	if os.Getenv("AGENT_MODE") == "demo" {
-		demoEnvironmentVariables := "./www/env.demo.js"
+		demoEnvironmentVariables := configDirectory + "/www/env.demo.js"
 		// Move demo environment variables to environment variables
 		err := os.Rename(demoEnvironmentVariables, environmentVariables)
 		if err != nil {
@@ -71,12 +71,14 @@ func StartServer(configuration *models.Configuration, communication *models.Comm
 	}
 
 	// Add static routes to UI
-	r.Use(static.Serve("/", static.LocalFile("./www", true)))
-	r.Use(static.Serve("/dashboard", static.LocalFile("./www", true)))
-	r.Use(static.Serve("/media", static.LocalFile("./www", true)))
-	r.Use(static.Serve("/settings", static.LocalFile("./www", true)))
-	r.Use(static.Serve("/login", static.LocalFile("./www", true)))
-	r.Handle("GET", "/file/*filepath", Files)
+	r.Use(static.Serve("/", static.LocalFile(configDirectory+"/www", true)))
+	r.Use(static.Serve("/dashboard", static.LocalFile(configDirectory+"/www", true)))
+	r.Use(static.Serve("/media", static.LocalFile(configDirectory+"/www", true)))
+	r.Use(static.Serve("/settings", static.LocalFile(configDirectory+"/www", true)))
+	r.Use(static.Serve("/login", static.LocalFile(configDirectory+"/www", true)))
+	r.Handle("GET", "/file/*filepath", func(c *gin.Context) {
+		Files(c, configDirectory)
+	})
 
 	// Run the api on port
 	err = r.Run(":" + configuration.Port)
@@ -85,8 +87,8 @@ func StartServer(configuration *models.Configuration, communication *models.Comm
 	}
 }
 
-func Files(c *gin.Context) {
+func Files(c *gin.Context, configDirectory string) {
 	c.Header("Access-Control-Allow-Origin", "*")
 	c.Header("Content-Type", "video/mp4")
-	c.File("./data/recordings" + c.Param("filepath"))
+	c.File(configDirectory + "/data/recordings" + c.Param("filepath"))
 }

machinery/src/routers/main.go

@@ -5,6 +5,6 @@ import (
 	"github.com/kerberos-io/agent/machinery/src/routers/http"
 )
 
-func StartWebserver(configuration *models.Configuration, communication *models.Communication) {
-	http.StartServer(configuration, communication)
+func StartWebserver(configDirectory string, configuration *models.Configuration, communication *models.Communication) {
+	http.StartServer(configDirectory, configuration, communication)
 }

machinery/src/utils/main.go

@@ -110,15 +110,15 @@ func CountDigits(i int64) (count int) {
 	return count
 }
 
-func CheckDataDirectoryPermissions() error {
-	recordingsDirectory := "./data/recordings"
-	configDirectory := "./data/config"
-	snapshotsDirectory := "./data/snapshots"
-	cloudDirectory := "./data/cloud"
+func CheckDataDirectoryPermissions(configDirectory string) error {
+	recordingsDirectory := configDirectory + "/data/recordings"
+	configurationDirectory := configDirectory + "/data/config"
+	snapshotsDirectory := configDirectory + "/data/snapshots"
+	cloudDirectory := configDirectory + "/data/cloud"
 
 	err := CheckDirectoryPermissions(recordingsDirectory)
 	if err == nil {
-		err = CheckDirectoryPermissions(configDirectory)
+		err = CheckDirectoryPermissions(configurationDirectory)
 		if err == nil {
 			err = CheckDirectoryPermissions(snapshotsDirectory)
 			if err == nil {

ui/src/config.js

@@ -9,9 +9,9 @@ const dev = {
   ENV: 'dev',
   // Comment the below lines, when using codespaces or other special DNS names (which you can't control)
   HOSTNAME: hostname,
-  API_URL: `${protocol}//${hostname}:8080/api`,
-  URL: `${protocol}//${hostname}:8080`,
-  WS_URL: `${websocketprotocol}//${hostname}:8080/ws`,
+  API_URL: `${protocol}//${hostname}:80/api`,
+  URL: `${protocol}//${hostname}:80`,
+  WS_URL: `${websocketprotocol}//${hostname}:80/ws`,
   MODE: window['env']['mode'],
   // Uncomment, and comment the above lines, when using codespaces or other special DNS names (which you can't control)
   // HOSTNAME: externalHost,

ui/src/pages/Settings/Settings.jsx

@@ -2114,17 +2114,6 @@ class Settings extends React.Component {
                   />
                   {config.cloud === this.KERBEROS_HUB && (
                     <>
-                      <Input
-                        noPadding
-                        label={t('settings.persistence.kerberoshub_proxyurl')}
-                        placeholder={t(
-                          'settings.persistence.kerberoshub_description_proxyurl'
-                        )}
-                        value={config.s3 ? config.s3.proxyuri : ''}
-                        onChange={(value) =>
-                          this.onUpdateField('s3', 'proxyuri', value, config.s3)
-                        }
-                      />
                       <Input
                         noPadding
                         label={t('settings.persistence.kerberoshub_region')}
@@ -2136,28 +2125,6 @@ class Settings extends React.Component {
                           this.onUpdateField('s3', 'region', value, config.s3)
                         }
                       />
-                      <Input
-                        noPadding
-                        label={t('settings.persistence.kerberoshub_bucket')}
-                        placeholder={t(
-                          'settings.persistence.kerberoshub_description_bucket'
-                        )}
-                        value={config.s3 ? config.s3.bucket : ''}
-                        onChange={(value) =>
-                          this.onUpdateField('s3', 'bucket', value, config.s3)
-                        }
-                      />
-                      <Input
-                        noPadding
-                        label={t('settings.persistence.kerberoshub_username')}
-                        placeholder={t(
-                          'settings.persistence.kerberoshub_description_username'
-                        )}
-                        value={config.s3 ? config.s3.username : ''}
-                        onChange={(value) =>
-                          this.onUpdateField('s3', 'username', value, config.s3)
-                        }
-                      />
                     </>
                   )}
                   {config.cloud === this.KERBEROS_VAULT && (