Read and parse configuration files. Fixes#41.
* supports nDPId / nDPIsrvd via command line parameter `-f`
* nDPId: read general/tuning and libnDPI settings
* support for settings risk domains libnDPI option via config file or via `-R` (Fixes#45, thanks to @UnveilTech)
* added some documentation in the config file
* adjusted Systemd and Debian packaging to make use of config files
Signed-off-by: Toni Uhlig <matzeton@googlemail.com>
* Add MacOS to Github CI builds.
* Fixed libnDPI-4.8 CI build.
* Fixed missing include for `struct sockaddr*`.
* Reworked IPv4 address and netmask retrieval.
Signed-off-by: Toni Uhlig <matzeton@googlemail.com>
* Finalize Event I/O abstraction layer.
* Fix possible fd leakage, Gitlab-CI build and error logging.
* Fixed possible uninitialized signalfd variable.
* Fixed possible memory leak.
* Fixed some SonarCloud complaints.
* Fixed nDPId-test nDPIsrvd-arpa-mockup stuck indefinitely.
* Add nDPId / nDPIsrvd command line option to use poll() on Linux instead of the default epoll().
Signed-off-by: Toni Uhlig <matzeton@googlemail.com>
* split logic into seperate jobs; nDPIsrvd and Keras
* nDPIsrvd: break event processing and re-run `epoll_wait()` after client disconnected
Signed-off-by: Toni Uhlig <matzeton@googlemail.com>
* nDPId/nDPIsrvd/c-examples: Parameter parsing needs to be improved
if `strdup()` in combination with static strings is used.
* Other non-critical fixes.
Signed-off-by: Toni Uhlig <matzeton@googlemail.com>
* Still not perfect, but the code before was not even able to deal with JSON arrays.
Use common "speaking" function names for all functions in nDPIsrvd.h
* Provide a more or less generic and easy extendable JSON walk function.
* Modified C examples to align with the changed C API.
* c-collectd: Reduced lot's of code duplication by providing mapping tables.
* nDPId: IAT array requires one slot less (first packet has always an IAT of 0).
Signed-off-by: Toni Uhlig <matzeton@googlemail.com>
* The goal was to provide a separate event for extracted feature that are not required
and only useful for a few (e.g. someone who wants do ML).
* Increased network buffer size to 32kB (8192 * 4).
* Switched timestamp precision from ms to us for *ALL* timestamps.
Signed-off-by: Toni Uhlig <matzeton@googlemail.com>
* nDPIsrvd: Collector/Distributor logging improved
* nDPIsrvd: Command line option for max remote descriptors
* nDPId: Stop spamming nDPIsrvd Collector with the same events over and over again
* nDPId: Refactored some variable names and events
Signed-off-by: Toni Uhlig <matzeton@googlemail.com>
* fixed GitLab pipeline
* nDPId: added static assert (just for a test)
* nDPId: memory profiling for total bytes compressed
* nDPId-test: enable zLib compression if configured with ENABLE_ZLIB
Signed-off-by: Toni Uhlig <matzeton@googlemail.com>
- nDPId: fixed invalid IP4/IP6 tuple compare
- nDPIsrvd: fixed caching issue (finally)
- added tiny c example (can be used to check flow manager sanity)
- c-captured: use flow_last_seen timestamp from `struct nDPIsrvd_flow`
- README.md update: added example JSON sequence
- nDPId: added new flow event `update` necessary for correct
timeout handling (and other future use-cases)
- nDPIsrvd.h and nDPIsrvd.py: switched to an instance
(consists of an alias/source tuple) based flow manager
- every flow related event **must** now serialize `alias`, `source`,
`flow_id`, `flow_last_seen` and `flow_idle_time` to make the timeout
handling and verification process work correctly
- nDPIsrvd.h: ability to profile any dynamic memory (de-)allocation
- nDPIsrvd.py: removed PcapPacket class (unused)
- py-flow-dashboard and py-flow-multiprocess: fixed race condition
- py-flow-info: print statusbar with probably useful information
- nDPId/nDPIsrvd.h: switched from packet-flow only timestamps (`pkt_*sec`)
to a generic flow event timestamp `ts_msec`
- nDPId-test: added additional checks
- nDPId: increased ICMP flow timeout
- nDPId: using event based i/o if capturing packets from a device
- nDPIsrvd: fixed memory leak on shutdown if remote descriptors
were still connected
Signed-off-by: Toni Uhlig <matzeton@googlemail.com>
* nDPId-test: disable #include <syslog.h> if NO_MAIN macro defined
* nDPId-test: mock syslog flags and functions
* gitlab-ci: force -Werror
Signed-off-by: Toni Uhlig <matzeton@googlemail.com>
* c-collectd gives the user control over collectd-exec instance name
* added missing collectd type `flow_l4_icmp_count`
Signed-off-by: Toni Uhlig <matzeton@googlemail.com>
* improved Makefile.old install targets
* splitted nDPIsrvd_parse into nDPIsrvd_parse_line and nDPIsrvd_parse_all for the sake of readability
* minor Python script improvments (check for nDPIsrvd.py on multiple locations, may be superseeded by setuptools in the future)
* some paths needs to be absolute (chdir() during daemonize) and therefor additional checks introduced
* test run script checks and fails if certain files are are missing (PCAP file <=> result output file)
* removed not very useful "internal format error" JSON serialization if a BUG for same exists
* fixed invalid l4 type statistics counters for nDPIsrvd-collectd
Signed-off-by: Toni Uhlig <matzeton@googlemail.com>
* fixed invalid flow event schema type
* added run_tests.sh to generate/diff JSON dumps
* renamed lot's of vars/fns in nDPId.c/nDPIsrvd.c, so nDPId-test.c can include "*.c"
* improved CMake dependency checks
Signed-off-by: Toni Uhlig <matzeton@googlemail.com>
