Updated README.md

* added examples/README.md and schema/README.md

Signed-off-by: Toni Uhlig <matzeton@googlemail.com>
This commit is contained in:
Toni Uhlig
2021-03-24 14:35:54 +01:00
parent 1ab6b9e042
commit ef4a22ffac
4 changed files with 92 additions and 10 deletions

1
.gitignore vendored
View File

@@ -1,6 +1,7 @@
# c executables
/nDPId
/nDPIsrvd
/nDPId-test
/examples/c-collectd/c-collectd
/examples/c-captured/c-captured
/examples/c-json-stdout/c-json-stdout

View File

@@ -1,7 +1,7 @@
# abstract
nDPId is a set of daemons and tools to capture, process and classify network flows.
It's only dependencies (besides a half-way modern c library and POSIX threads) are libnDPI (>= 3.4.0, current github dev branch) and libpcap.
It's only dependencies (besides a half-way modern c library and POSIX threads) are libnDPI (>= 3.6.0 or current github dev branch) and libpcap.
The core daemon nDPId uses pthread but does use mutexes for performance reasons.
Instead synchronization is achieved by a packet distribution mechanism.
@@ -12,7 +12,6 @@ nDPId uses libnDPI's JSON serialization to produce meaningful JSON output which
High level applications can connect to nDPIsrvd to get the latest flow/packet events from nDPId.
Unfortunately nDPIsrvd does currently not support any encryption/authentication for TCP connections.
TODO: Provide some sort of AEAD for connecting distributor clients via TCP (somehow very critical).
# architecture
@@ -48,36 +47,71 @@ All JSON strings sent need to be in the following format:
## Example:
```text
0015{"key":"value"}
00015{"key":"value"}
```
where `0015` describes the length of a **complete** JSON string.
where `00015` describes the length of a **complete** JSON string.
TODO: Describe data format via JSON schema.
# build
# build (CMake)
```shell
mkdir build
cd build
cmake ..
```
or
```shell
mkdir build
cd build
ccmake ..
```
# build (old style GNU Make)
To get an overview over all build options, run:
```shell
make help
make -f Makefile.old help
```
To build nDPId and nDPIsrvd, run:
```shell
make all
make -f Makefile.old all
```
To build nDPId and nDPIsrvd with sanitizer, debug mode enabled and a custom/not-your-distro libnDPI, run:
```shell
make ENABLE_DEBUG=yes ENABLE_SANITIZER=yes CUSTOM_LIBNDPI=[path-to-libndpi].[a|so] all
make -f Makefile.old ENABLE_DEBUG=yes ENABLE_SANITIZER=yes CUSTOM_LIBNDPI=[path-to-libndpi].[a|so] all
```
If you get any linker errors, try one of the
```shell
make -f Makefile.old | grep '^NDPI_WITH_'
```
e.g.
```shell
make -f Makefile.old NDPI_WITH_GCRYPT=yes ENABLE_DEBUG=yes ENABLE_SANITIZER=yes CUSTOM_LIBNDPI=[path-to-libndpi].[a|so] all
```
or let pkg-config do the job for you:
```shell
PKG_CONFIG_PATH="[path-to-optional-nDPI-pkg-config-dir]" make -f Makefile.old PKG_CONFIG_BIN=pkg-config ENABLE_DEBUG=yes ENABLE_SANITIZER=yes all
```
To build nDPId and nDPIsrvd and examples, run:
```shell
make all examples
make -f Makefile.old all examples
```
# run
Generate a nDPId compatible JSON dump:
```shell
./nDPId-test [path-to-a-PCAP-file]
```
Daemons:
```shell
./nDPIsrvd -d
@@ -91,7 +125,6 @@ or for a usage printout:
```
And why not a flow-info example?
Run
```shell
./examples/py-flow-info/flow-info.py
```

43
examples/README.md Normal file
View File

@@ -0,0 +1,43 @@
# examples
Some ready-2-use/ready-2-extend examples/utils.
All examples are prefixed with their used LANG.
## c-captured
A capture daemon suitable for low-resource devices.
It saves flows that were guessed/undetected/risky/midstream to a PCAP file for manual analysis.
Basicially a combination of `py-flow-undetected-to-pcap` and `py-risky-flow-to-pcap`.
## c-collectd
A collecd-exec compatible middleware that gathers statistic values from nDPId.
## c-json-stdout
Tiny nDPId json dumper. Does not provide any useful funcationality besides dumping parsed JSON objects.
## go-dashboard
A discontinued tty/ncurses nDPId dashboard. I've figured out that Go + NCurses is a bad idea.
## py-flow-info
Prints prettyfied information about flow events.
## py-flow-undetected-to-pcap
Captures and saves undetected flows to a PCAP file.
## py-json-stdout
Dump received and parsed JSON strings.
## py-risky-flow-to-pcap
Captures and saves risky flows to a PCAP file.
## py-schema-validation
Validate nDPId JSON strings against pre-defined JSON schema's.
See `schema/`.

5
schema/README.md Normal file
View File

@@ -0,0 +1,5 @@
# schema
All schema's placed in here are nDPId exclusive, meaning that they are not necessarily representing a "real-world" JSON string received by e.g. `./example/py-json-stdout`.
This is due to the fact that libnDPI itself add's some JSON information to the serializer of which we have no control over.
IMHO it makes no sense to include stuff here that is part of libnDPI.