add osusergo

Dockerfile

@@ -32,7 +32,7 @@ RUN cat /go/src/github.com/kerberos-io/agent/machinery/version
 
 RUN cd /go/src/github.com/kerberos-io/agent/machinery && \
 	go mod download && \
-	go build -tags timetzdata,netgo --ldflags '-s -w -extldflags "-static -latomic"' main.go && \
+	go build -tags timetzdata,netgo,osusergo --ldflags '-s -w -extldflags "-static -latomic"' main.go && \
 	mkdir -p /agent && \
 	mv main /agent && \
 	mv version /agent && \

README.md

@@ -78,12 +78,9 @@ If you want to connect to an USB or Raspberry Pi camera, [you'll need to run our
 
 ## Quickstart - Balena
 
-Run Kerberos Agent with Balena super powers. Monitor your agent with seamless remote access, and an encrypted https endpoint.
-Checkout our fleet on [Balena Hub](https://hub.balena.io/fleets?0%5B0%5D%5Bn%5D=any&0%5B0%5D%5Bo%5D=full_text_search&0%5B0%5D%5Bv%5D=agent), and add your agent.
+Run Kerberos Agent with [Balena Cloud](https://www.balena.io/) super powers. Monitor your Kerberos Agent with seamless remote access, over the air updates, an encrypted public `https` endpoint and many more. Checkout our application `video-surveillance` on [Balena Hub](https://hub.balena.io/apps/2064752/video-surveillance), and create your first or fleet of Kerberos Agent(s).
 
-[![balena deploy button](https://www.balena.io/deploy.svg)](https://dashboard.balena-cloud.com/deploy?repoUrl=https://github.com/kerberos-io/agent)
-
-**_Work In Progress_** - Currently we only support IP and USB Cameras, we have [an approach for leveraging the Raspberry Pi camera](https://github.com/kerberos-io/camera-to-rtsp), but this isn't working as expected with Balena. If you require this, you'll need to use the traditional Docker deployment with sidecar as mentioned above.
+[![deploy with balena](https://balena.io/deploy.svg)](https://dashboard.balena-cloud.com/deploy?repoUrl=https://github.com/kerberos-io/balena-agent)
 
 ## A world of Kerberos Agents
 
@@ -122,6 +119,7 @@ We have documented the different deployment models [in the `deployments` directo
 - [Red Hat OpenShift with Ansible](https://github.com/kerberos-io/agent/tree/master/deployments#4-red-hat-ansible-and-openshift)
 - [Terraform](https://github.com/kerberos-io/agent/tree/master/deployments#5-terraform)
 - [Salt](https://github.com/kerberos-io/agent/tree/master/deployments#6-salt)
+- [Balena](https://github.com/kerberos-io/agent/tree/master/deployments#8-balena)
 
 By default your Kerberos Agents will store all its configuration and recordings inside the container. To help you automate and have a more consistent data governance, you can attach volumes to configure and persist data of your Kerberos Agents, and/or configure each Kerberos Agent through environment variables.
 

deployments/README.md

@@ -14,6 +14,7 @@ We will discuss following deployment models.
 - [5. Kerberos Factory](#5-kerberos-factory)
 - [6. Terraform](#6-terraform)
 - [7. Salt](#7-salt)
+- [8. Balena](#8-balena)
 
 ## 0. Static binary
 
@@ -58,3 +59,11 @@ To be written
 ## 7. Salt
 
 To be written
+
+## 8. Balena
+
+Balena Cloud provide a seamless way of building and deploying applications at scale through the conceps of `blocks`, `apps` and `fleets`. Once you have your `app` deployed, for example our Kerberos Agent, you can benefit from features such as: remote access, over the air updates, an encrypted public `https` endpoint and many more.
+
+Together with the Balena.io team we've build a Balena App, called [`video-surveillance`](https://hub.balena.io/apps/2064752/video-surveillance), which any can use to deploy a video surveillance system in a matter of minutes with all the expected management features you can think of.
+
+> Learn more [about Kerberos Agent with Balena](https://github.com/kerberos-io/agent/tree/master/deployments/balena).

deployments/ansible-openshift/playbook.yml

@@ -82,7 +82,7 @@
 
                 initContainers:
                   - name: download-config
-                    image: kerberos/agent:1b96d01
+                    image: kerberos/agent:latest
                     volumeMounts:
                       - name: kerberos-data
                         mountPath: /home/agent/data/config
@@ -96,7 +96,7 @@
 
                 containers:
                   - name: agent
-                    image: kerberos/agent:1b96d01
+                    image: kerberos/agent:latest
                     volumeMounts:
                       - name: kerberos-data
                         mountPath: /home/agent/data/config

deployments/balena/README.md

@@ -0,0 +1,31 @@
+# Deployment with Balena
+
+Balena Cloud provide a seamless way of building and deploying applications at scale through the conceps of `blocks`, `apps` and `fleets`. Once you have your `app` deployed, for example our Kerberos Agent, you can benefit from features such as: remote access, over the air updates, an encrypted public `https` endpoint and many more.
+
+We provide two mechanisms to deploy Kerberos Agent to a Balena Cloud fleet:
+
+1. Use Kerberos Agent as [a block part of your application](https://github.com/kerberos-io/balena-agent-block).
+2. Use Kerberos Agent as [a stand-alone application](https://github.com/kerberos-io/balena-agent).
+
+## Block
+
+Within Balena you can build the concept of a block, which is the equivalent of container image or a function in a typical programming language. The idea of blocks, you can find a more thorough explanation [here](https://docs.balena.io/learn/develop/blocks/), is that you can compose and combine multiple `blocks` to level up to the concept an `app`.
+
+You as a developer can choose which `blocks` you would like to use, to build the desired `application` state you prefer. For example you can use the [Kerberos Agent block](https://hub.balena.io/blocks/2064662/agent) to compose a video surveillance system as part of your existing set of blocks.
+
+You can the `Kerberos Agent` block by defining following elements in your `compose` file.
+
+    agent:
+        image: bh.cr/kerberos_io/agent
+
+## App
+
+Next to building individual `blocks` you as a developer can also decide to build up an application, composed of one or more `blocks` or third-party containers, and publish it as an `app` to the Balena Hub. This is exactly [what we've done..](https://hub.balena.io/apps/2064752/video-surveillance)
+
+On Balena Hub we have created the []`video-surveillance` application](https://hub.balena.io/apps/2064752/video-surveillance) that utilises the [Kerberos Agent `block`](https://hub.balena.io/blocks/2064662/agent). The idea of this application is that utilises the foundation of our Kerberos Agent, but that it might include more `blocks` over time to increase and improve functionalities from other community projects.
+
+To deploy the application you can simply press below `Deploy button` or you can navigate to the [Balena Hub apps page](https://hub.balena.io/apps/2064752/video-surveillance).
+
+[![deploy with balena](https://balena.io/deploy.svg)](https://dashboard.balena-cloud.com/deploy?repoUrl=https://github.com/kerberos-io/agent)
+
+You can find the source code, `balena.yaml` and `docker-compose.yaml` files in the [`balena-agent` repository](https://github.com/kerberos-io/balena-agent).

deployments/kubernetes/deployment-agent-volume.yml

@@ -21,7 +21,7 @@ spec:
 
       initContainers:
         - name: download-config
-          image: kerberos/agent:1b96d01
+          image: kerberos/agent:latest
           volumeMounts:
             - name: kerberos-data
               mountPath: /home/agent/data/config

machinery/src/cloud/Cloud.go

@@ -2,7 +2,6 @@ package cloud
 
 import (
 	"bytes"
-	"crypto/tls"
 	"encoding/base64"
 	"encoding/json"
 	"fmt"
@@ -15,14 +14,12 @@ import (
 	"github.com/gin-gonic/gin"
 	"github.com/golang-module/carbon/v2"
 	"github.com/kerberos-io/joy4/av/pubsub"
-	"github.com/minio/minio-go/v6"
 
 	mqtt "github.com/eclipse/paho.mqtt.golang"
 	av "github.com/kerberos-io/joy4/av"
 	"github.com/kerberos-io/joy4/cgo/ffmpeg"
 
 	"net/http"
-	"net/url"
 	"strconv"
 	"time"
 
@@ -545,14 +542,14 @@ func VerifyHub(c *gin.Context) {
 	err := c.BindJSON(&config)
 
 	if err == nil {
-		hubKey := config.HubKey
 		hubURI := config.HubURI
+		publicKey := config.HubKey
+		privateKey := config.HubPrivateKey
 
-		content := []byte(`{"message": "fake-message"}`)
-		body := bytes.NewReader(content)
-		req, err := http.NewRequest("POST", hubURI+"/queue/test", body)
+		req, err := http.NewRequest("POST", hubURI+"/subscription/verify", nil)
 		if err == nil {
-			req.Header.Set("X-Kerberos-Cloud-Key", hubKey)
+			req.Header.Set("X-Kerberos-Hub-PublicKey", publicKey)
+			req.Header.Set("X-Kerberos-Hub-PrivateKey", privateKey)
 			client := &http.Client{}
 
 			resp, err := client.Do(req)
@@ -609,88 +606,80 @@ func VerifyPersistence(c *gin.Context) {
 
 		if config.Cloud == "dropbox" {
 			VerifyDropbox(config, c)
-		} else if config.Cloud == "s3" {
+		} else if config.Cloud == "s3" || config.Cloud == "kerberoshub" {
 
-			// timestamp_microseconds_instanceName_regionCoordinates_numberOfChanges_token
-			// 1564859471_6-474162_oprit_577-283-727-375_1153_27.mp4
-			// - Timestamp
-			// - Size + - + microseconds
-			// - device
-			// - Region
-			// - Number of changes
-			// - Token
-
-			aws_access_key_id := config.S3.Publickey
-			aws_secret_access_key := config.S3.Secretkey
-			aws_region := config.S3.Region
-
-			// This is the new way ;)
-			if config.HubKey != "" {
-				aws_access_key_id = config.HubKey
-			}
-			if config.HubPrivateKey != "" {
-				aws_secret_access_key = config.HubPrivateKey
-			}
-
-			s3Client, err := minio.NewWithRegion("s3.amazonaws.com", aws_access_key_id, aws_secret_access_key, true, aws_region)
-			if err != nil {
+			if config.HubURI == "" ||
+				config.HubKey == "" ||
+				config.HubPrivateKey == "" ||
+				config.S3.Region == "" {
+				msg := "VerifyPersistence: Kerberos Hub not properly configured."
+				log.Log.Info(msg)
 				c.JSON(400, models.APIResponse{
-					Data: "Creation of Kerberos Hub connection failed: " + err.Error(),
+					Data: msg,
 				})
 			} else {
 
-				// Check if we need to use the proxy.
-				if config.S3.ProxyURI != "" {
-					var transport http.RoundTripper = &http.Transport{
-						Proxy: func(*http.Request) (*url.URL, error) {
-							return url.Parse(config.S3.ProxyURI)
-						},
-						TLSClientConfig: &tls.Config{InsecureSkipVerify: true},
-					}
-					s3Client.SetCustomTransport(transport)
+				// Open test-480p.mp4
+				file, err := os.Open("./data/test-480p.mp4")
+				if err != nil {
+					msg := "VerifyPersistence: error reading test-480p.mp4: " + err.Error()
+					log.Log.Error(msg)
+					c.JSON(400, models.APIResponse{
+						Data: msg,
+					})
+				}
+				defer file.Close()
+
+				req, err := http.NewRequest("POST", config.HubURI+"/storage/upload", file)
+				if err != nil {
+					msg := "VerifyPersistence: error reading Kerberos Hub HEAD request, " + config.HubURI + "/storage: " + err.Error()
+					log.Log.Error(msg)
+					c.JSON(400, models.APIResponse{
+						Data: msg,
+					})
 				}
 
-				deviceKey := "fake-key"
-				devicename := "justatest"
-				coordinates := "200-200-400-400"
-				eventToken := "769"
-
 				timestamp := time.Now().Unix()
-				fileName := strconv.FormatInt(timestamp, 10) + "_6-967003_justatest_200-200-400-400_24_769.mp4"
-				content := []byte("test-file")
-				body := bytes.NewReader(content)
+				fileName := strconv.FormatInt(timestamp, 10) +
+					"_6-967003_" + config.Name + "_200-200-400-400_24_769.mp4"
+				req.Header.Set("X-Kerberos-Storage-FileName", fileName)
+				req.Header.Set("X-Kerberos-Storage-Capture", "IPCamera")
+				req.Header.Set("X-Kerberos-Storage-Device", config.Key)
+				req.Header.Set("X-Kerberos-Hub-PublicKey", config.HubKey)
+				req.Header.Set("X-Kerberos-Hub-PrivateKey", config.HubPrivateKey)
+				req.Header.Set("X-Kerberos-Hub-Region", config.S3.Region)
 
-				n, err := s3Client.PutObject(config.S3.Bucket,
-					config.S3.Username+"/"+fileName,
-					body,
-					body.Size(),
-					minio.PutObjectOptions{
-						ContentType:  "video/mp4",
-						StorageClass: "ONEZONE_IA",
-						UserMetadata: map[string]string{
-							"event-timestamp":         strconv.FormatInt(timestamp, 10),
-							"event-microseconds":      deviceKey,
-							"event-instancename":      devicename,
-							"event-regioncoordinates": coordinates,
-							"event-numberofchanges":   deviceKey,
-							"event-token":             eventToken,
-							"productid":               deviceKey,
-							"publickey":               aws_access_key_id,
-							"uploadtime":              "now",
-						},
-					})
+				client := &http.Client{}
 
-				if err != nil {
-					c.JSON(400, models.APIResponse{
-						Data: "Upload of fake recording failed: " + err.Error(),
-					})
+				resp, err := client.Do(req)
+				if resp != nil {
+					defer resp.Body.Close()
+				}
+
+				if err == nil && resp != nil {
+					if resp.StatusCode == 200 {
+						msg := "VerifyPersistence: Upload allowed using the credentials provided (" + config.HubKey + ", " + config.HubPrivateKey + ")"
+						log.Log.Info(msg)
+						c.JSON(200, models.APIResponse{
+							Data: msg,
+						})
+					} else {
+						msg := "VerifyPersistence: Upload NOT allowed using the credentials provided (" + config.HubKey + ", " + config.HubPrivateKey + ")"
+						log.Log.Info(msg)
+						c.JSON(400, models.APIResponse{
+							Data: msg,
+						})
+					}
 				} else {
-					c.JSON(200, models.APIResponse{
-						Data: "Upload Finished: file has been uploaded to bucket: " + strconv.FormatInt(n, 10),
+					msg := "VerifyPersistence: Error creating Kerberos Hub request"
+					log.Log.Info(msg)
+					c.JSON(400, models.APIResponse{
+						Data: msg,
 					})
 				}
 			}
-		} else if config.Cloud == "kstorage" {
+
+		} else if config.Cloud == "kstorage" || config.Cloud == "kerberosvault" {
 
 			uri := config.KStorage.URI
 			accessKey := config.KStorage.AccessKey
@@ -699,10 +688,9 @@ func VerifyPersistence(c *gin.Context) {
 			provider := config.KStorage.Provider
 
 			if err == nil && uri != "" && accessKey != "" && secretAccessKey != "" {
-				var postData = []byte(`{"title":"Buy cheese and bread for breakfast."}`)
-				client := &http.Client{}
-				req, err := http.NewRequest("POST", uri+"/ping", bytes.NewReader(postData))
 
+				client := &http.Client{}
+				req, err := http.NewRequest("POST", uri+"/ping", nil)
 				req.Header.Add("X-Kerberos-Storage-AccessKey", accessKey)
 				req.Header.Add("X-Kerberos-Storage-SecretAccessKey", secretAccessKey)
 				resp, err := client.Do(req)
@@ -714,32 +702,35 @@ func VerifyPersistence(c *gin.Context) {
 
 						if provider != "" || directory != "" {
 
-							hubKey := config.KStorage.CloudKey
-							// This is the new way ;)
-							if config.HubKey != "" {
-								hubKey = config.HubKey
-							}
-
 							// Generate a random name.
 							timestamp := time.Now().Unix()
 							fileName := strconv.FormatInt(timestamp, 10) +
-								"_6-967003_justatest_200-200-400-400_24_769.mp4"
-							content := []byte("test-file")
-							body := bytes.NewReader(content)
-							//fileSize := int64(len(content))
+								"_6-967003_" + config.Name + "_200-200-400-400_24_769.mp4"
 
-							req, err := http.NewRequest("POST", uri+"/storage", body)
+							// Open test-480p.mp4
+							file, err := os.Open("./data/test-480p.mp4")
+							if err != nil {
+								msg := "VerifyPersistence: error reading test-480p.mp4: " + err.Error()
+								log.Log.Error(msg)
+								c.JSON(400, models.APIResponse{
+									Data: msg,
+								})
+							}
+							defer file.Close()
+
+							req, err := http.NewRequest("POST", uri+"/storage", file)
 							if err == nil {
 
 								req.Header.Set("Content-Type", "video/mp4")
-								req.Header.Set("X-Kerberos-Storage-CloudKey", hubKey)
+								req.Header.Set("X-Kerberos-Storage-CloudKey", config.HubKey)
 								req.Header.Set("X-Kerberos-Storage-AccessKey", accessKey)
 								req.Header.Set("X-Kerberos-Storage-SecretAccessKey", secretAccessKey)
 								req.Header.Set("X-Kerberos-Storage-Provider", provider)
 								req.Header.Set("X-Kerberos-Storage-FileName", fileName)
-								req.Header.Set("X-Kerberos-Storage-Device", "test")
+								req.Header.Set("X-Kerberos-Storage-Device", config.Key)
 								req.Header.Set("X-Kerberos-Storage-Capture", "IPCamera")
 								req.Header.Set("X-Kerberos-Storage-Directory", directory)
+
 								client := &http.Client{}
 
 								resp, err := client.Do(req)
@@ -753,41 +744,45 @@ func VerifyPersistence(c *gin.Context) {
 												c.JSON(200, body)
 											} else {
 												c.JSON(400, models.APIResponse{
-													Data: "Something went wrong while verifying your persistence settings. Make sure your provider is the same as the storage provider in your Kerberos Vault, and the relevant storage provider is configured properly.",
+													Data: "VerifyPersistence: Something went wrong while verifying your persistence settings. Make sure your provider is the same as the storage provider in your Kerberos Vault, and the relevant storage provider is configured properly.",
 												})
 											}
 										}
 									}
 								} else {
 									c.JSON(400, models.APIResponse{
-										Data: "Upload of fake recording failed: " + err.Error(),
+										Data: "VerifyPersistence: Upload of fake recording failed: " + err.Error(),
 									})
 								}
 							} else {
 								c.JSON(400, models.APIResponse{
-									Data: "Something went wrong while creating /storage POST request." + err.Error(),
+									Data: "VerifyPersistence: Something went wrong while creating /storage POST request." + err.Error(),
 								})
 							}
 						} else {
 							c.JSON(400, models.APIResponse{
-								Data: "Provider and/or directory is missing from the request.",
+								Data: "VerifyPersistence: Provider and/or directory is missing from the request.",
 							})
 						}
 					} else {
 						c.JSON(400, models.APIResponse{
-							Data: "Something went wrong while verifying storage credentials: " + string(body),
+							Data: "VerifyPersistence: Something went wrong while verifying storage credentials: " + string(body),
 						})
 					}
 				} else {
 					c.JSON(400, models.APIResponse{
-						Data: "Something went wrong while verifying storage credentials:" + err.Error(),
+						Data: "VerifyPersistence: Something went wrong while verifying storage credentials:" + err.Error(),
 					})
 				}
+			} else {
+				c.JSON(400, models.APIResponse{
+					Data: "VerifyPersistence: please fill-in the required Kerberos Vault credentials.",
+				})
 			}
 		}
 	} else {
 		c.JSON(400, models.APIResponse{
-			Data: "No persistence was specified, so do not know what to verify:" + err.Error(),
+			Data: "VerifyPersistence: No persistence was specified, so do not know what to verify:" + err.Error(),
 		})
 	}
 }

machinery/src/computervision/main.go

@@ -165,7 +165,7 @@ func ProcessMotion(motionCursor *pubsub.QueueCursor, configuration *models.Confi
 					if detectMotion && isPixelChangeThresholdReached {
 
 						// If offline mode is disabled, send a message to the hub
-						if config.Offline == "false" {
+						if config.Offline != "true" {
 							if mqttClient != nil {
 								if key != "" {
 									mqttClient.Publish("kerberos/"+key+"/device/"+config.Key+"/motion", 2, false, "motion")

machinery/src/database/main.go

@@ -28,10 +28,10 @@ func New() *mongo.Client {
 	password := os.Getenv("MONGODB_PASSWORD")
 	authentication := "SCRAM-SHA-256"
 
-	ctx, cancel := context.WithTimeout(context.Background(), 10*time.Second)
-	defer cancel()
-
 	_init_ctx.Do(func() {
+		ctx, cancel := context.WithTimeout(context.Background(), 10*time.Second)
+		defer cancel()
+
 		_instance = new(DB)
 		mongodbURI := fmt.Sprintf("mongodb://%s:%s@%s", username, password, host)
 		if replicaset != "" {

ui/src/pages/Settings/Settings.jsx

@@ -2114,17 +2114,6 @@ class Settings extends React.Component {
                   />
                   {config.cloud === this.KERBEROS_HUB && (
                     <>
-                      <Input
-                        noPadding
-                        label={t('settings.persistence.kerberoshub_proxyurl')}
-                        placeholder={t(
-                          'settings.persistence.kerberoshub_description_proxyurl'
-                        )}
-                        value={config.s3 ? config.s3.proxyuri : ''}
-                        onChange={(value) =>
-                          this.onUpdateField('s3', 'proxyuri', value, config.s3)
-                        }
-                      />
                       <Input
                         noPadding
                         label={t('settings.persistence.kerberoshub_region')}
@@ -2136,28 +2125,6 @@ class Settings extends React.Component {
                           this.onUpdateField('s3', 'region', value, config.s3)
                         }
                       />
-                      <Input
-                        noPadding
-                        label={t('settings.persistence.kerberoshub_bucket')}
-                        placeholder={t(
-                          'settings.persistence.kerberoshub_description_bucket'
-                        )}
-                        value={config.s3 ? config.s3.bucket : ''}
-                        onChange={(value) =>
-                          this.onUpdateField('s3', 'bucket', value, config.s3)
-                        }
-                      />
-                      <Input
-                        noPadding
-                        label={t('settings.persistence.kerberoshub_username')}
-                        placeholder={t(
-                          'settings.persistence.kerberoshub_description_username'
-                        )}
-                        value={config.s3 ? config.s3.username : ''}
-                        onChange={(value) =>
-                          this.onUpdateField('s3', 'username', value, config.s3)
-                        }
-                      />
                     </>
                   )}
                   {config.cloud === this.KERBEROS_VAULT && (