scottk/nDPId
1
0
Fork 0
mirror of https://github.com/optim-enterprises-bv/nDPId.git synced 2025-10-29 09:22:23 +00:00
Code Issues Packages Projects Releases Wiki Activity
761 Commits 1 Branch 0 Tags
0e792ba3011faba982e8d6bd06fb6d5e7c5a8378
Commit Graph

179 Commits

Author SHA1 Message Date
Toni Uhlig
9ef17b7bd8 Added some static assertion based sanity checks. 
Signed-off-by: Toni Uhlig <matzeton@googlemail.com>
 2024-09-11 13:28:20 +02:00
Toni Uhlig
1c9aa85485 Save hostname after detection finished for later use within analyse/end/idle flow events. Fixes #39. 
Signed-off-by: Toni Uhlig <matzeton@googlemail.com>
 2024-09-11 13:01:23 +02:00
Toni
5e4005162b Add PF_RING support. (#38) 2024-08-19 18:33:18 +02:00
Toni Uhlig
930aaf9276 Added global (heap) memory stats for daemon status events. 
* added new CMake option `ENABLE_MEMORY_STATUS` to restore the old behavior
   (and increase performance)
 * splitted `ENABLE_MEMORY_PROFILING` into `ENABLE_MEMORY_STATUS` and `ENABLE_MEMORY_PROFILING`

Signed-off-by: Toni Uhlig <matzeton@googlemail.com>
 2024-06-19 14:25:42 +02:00
Toni Uhlig
c7eace426c bump libnDPI to 9185c2ccc402d3368fc28ac90ab281b4f951719e 
* incorporated API changes from 41eef9246c6a3055e3876e3dd7aeaadecb4b76c0

Signed-off-by: Toni Uhlig <matzeton@googlemail.com>
 2024-04-04 11:49:48 +02:00
Toni Uhlig
ea968180a2 Read Ipv6 address and netmask using getifaddrs() instead of reading /proc/net/if_inet6. 
* fixes a compatibility issue with Mac OSX

Signed-off-by: Toni Uhlig <matzeton@googlemail.com>
 2024-02-07 14:25:14 +01:00
Toni Uhlig
556025b34d Removed API version macro check as it's inconsistent on different platforms. 
* set min required nDPI version to 4.9.0

Signed-off-by: Toni Uhlig <matzeton@googlemail.com>
 2024-02-06 10:49:47 +01:00
Toni Uhlig
feb2583ef6 bump libnDPI to 4543385d107fcc5a7e8632e35d9a60bcc40cb4f4 
* incorporated API changes from nDPI

Signed-off-by: Toni Uhlig <matzeton@googlemail.com>
 2024-02-06 10:34:52 +01:00
Toni Uhlig
71d933b0cd Fixed an event issue. 
* a "detection-update" event was thrown even if nothing changed
 * in some cases "not-detected" events were spammed if detection not completed
 * tell `libnDPI` how many packets per flow we want to dissect
 * `nDPId-test` validates total active flows in the right way

Signed-off-by: Toni Uhlig <matzeton@googlemail.com>
 2023-12-14 15:45:08 +01:00
Toni Uhlig
f5c5bc88a7 Replaced ambiguous naming of "JSON string" to more accurate "JSON message". #2 
Signed-off-by: Toni Uhlig <matzeton@googlemail.com>
 2023-12-04 13:13:05 +01:00
Toni Uhlig
53d8a28582 Replaced ambiguous naming of "JSON string" to more accurate "JSON message". 
Signed-off-by: Toni Uhlig <matzeton@googlemail.com>
 2023-12-04 13:01:27 +01:00
Toni Uhlig
37f3770e3e Improved zlib compression ratio. 
Signed-off-by: Toni Uhlig <matzeton@googlemail.com>
 2023-12-01 06:43:39 +01:00
Toni Uhlig
ff77bab398 Warn about unused return values that are quite important. 
* CI: ArchLinux build should now instrument `-Werror`
 * CI: Increased OpenWrt build verbosity

Signed-off-by: Toni Uhlig <matzeton@googlemail.com>
 2023-12-01 06:43:39 +01:00
Toni Uhlig
fa7e76cc75 Fixed SonarCloud complaints. 
Signed-off-by: Toni Uhlig <matzeton@googlemail.com>
 2023-11-20 13:40:48 +01:00
Toni Uhlig
82934b7271 Fixed clang-tidy warnings. 
* fixed/improved c-captured logging

Signed-off-by: Toni Uhlig <matzeton@googlemail.com>
 2023-11-13 13:52:42 +01:00
Toni Uhlig
dcb595e161 bump libnDPI to b08c787fe267053afdea82701071f3878c09244b 
* fix ndpi data anylsis struct min/max issue
 * py-flow-info cosmetics in printing some information

Signed-off-by: Toni Uhlig <matzeton@googlemail.com>
 2023-11-09 19:52:36 +01:00
Toni Uhlig
b667f9e1da Forcefully reset NDPI_UNIDIRECTIONAL_TRAFFIC if classification was done after the first packet. Nonsense. 
Signed-off-by: Toni Uhlig <matzeton@googlemail.com>
 2023-11-08 17:07:20 +01:00
Toni Uhlig
55c8a848d3 Fixed missing deflate during flow event json serializing. 
* caused by recently added serializing some nDPI data even packet processing is still ongoing

Signed-off-by: Toni Uhlig <matzeton@googlemail.com>
 2023-11-08 13:16:57 +01:00
Toni Uhlig
d80ea84d2e Reset Unidirectional Traffc risk if packets from both directions processed. 
* Fixed risk hash value calculation, which was only done lower 32 bits.
 * Reduced default reader threads count to two if cross compiling.

Signed-off-by: Toni Uhlig <matzeton@googlemail.com>
 2023-11-08 01:27:42 +01:00
Toni Uhlig
5d56288a11 Fixed more SonarCloud complaints. 
Signed-off-by: Toni Uhlig <matzeton@googlemail.com>
 2023-11-07 00:02:36 +01:00
Toni Uhlig
84b12cd02c Fixed some SonarCloud complaints. 
Signed-off-by: Toni Uhlig <matzeton@googlemail.com>
 2023-11-06 13:31:59 +01:00
Toni
93498fff02 Apple/BSD port (#30) 
* Add MacOS to Github CI builds.
* Fixed libnDPI-4.8 CI build.
* Fixed missing include for `struct sockaddr*`.
* Reworked IPv4 address and netmask retrieval.

Signed-off-by: Toni Uhlig <matzeton@googlemail.com>
 2023-11-06 13:25:49 +01:00
Toni
1b67927169 Event I/O abstraction layer. (#28) 
* Finalize Event I/O abstraction layer.
* Fix possible fd leakage, Gitlab-CI build and error logging.
* Fixed possible uninitialized signalfd variable.
* Fixed possible memory leak.
* Fixed some SonarCloud complaints.
* Fixed nDPId-test nDPIsrvd-arpa-mockup stuck indefinitely.
* Add nDPId / nDPIsrvd command line option to use poll() on Linux instead of the default epoll().

Signed-off-by: Toni Uhlig <matzeton@googlemail.com>
 2023-11-06 12:38:15 +01:00
Toni Uhlig
dfd0449306 Fix issues detected by SonarCloud. 
Signed-off-by: Toni Uhlig <matzeton@googlemail.com>
 2023-10-05 17:26:06 +02:00
Toni Uhlig
e42e3fe406 Serialize nDPId / libnDPI versions within daemon events. 
* changed nDPI version hints / requirements

Signed-off-by: Toni Uhlig <matzeton@googlemail.com>
 2023-09-10 19:32:08 +02:00
Toni Uhlig
dfb8d3379f bump libnDPI to 1f693c3f5a5dcd9d69dffb610b9a81bd33f95382 
Signed-off-by: Toni Uhlig <matzeton@googlemail.com>
 2023-08-27 21:45:14 +02:00
Toni Uhlig
cc60e819e8 Fixed invalid base64 encoding in some rare cases. 
* nDPId-test may also verify the correct encoding/decoding

Signed-off-by: Toni Uhlig <matzeton@googlemail.com>
 2023-08-24 22:51:57 +02:00
Toni Uhlig
ebb439d959 Tiny improvments. 
Signed-off-by: Toni Uhlig <matzeton@googlemail.com>
 2023-08-02 21:00:39 +02:00
Toni Uhlig
bc0a5782cc bump libnDPI to 2b230e28e0612e8654ad617534deb9aaaabd51b7 
* fixes loading of gambling lists which increased nDPId's memory usage *a lot*
 * nDPId: handle EINTR correctly

Signed-off-by: Toni Uhlig <matzeton@googlemail.com>
 2023-07-23 18:43:58 +02:00
Toni Uhlig
7217b90cd1 nDPId: `-v' give information about libnDPI linkage 
Signed-off-by: Toni Uhlig <matzeton@googlemail.com>
 2023-07-17 19:32:41 +02:00
Toni Uhlig
74a9f7d86b nDPId: `-v' prints also information about dependecies 
Signed-off-by: Toni Uhlig <matzeton@googlemail.com>
 2023-07-17 19:19:52 +02:00
Toni Uhlig
967381a599 get-and-build-libndpi.sh uses GMake MAKEFLAGS for sub-make (required for e.g. jobserver) 
* fixed invalid CMake `test -r ...`

Signed-off-by: Toni Uhlig <matzeton@googlemail.com>
 2023-07-16 22:06:31 +02:00
lns
2b1db0a556 Required libnDPI version increases (>=4.8.0) due to an API change. 
* fix CI issues

Signed-off-by: lns <matzeton@googlemail.com>
 2023-05-31 12:53:49 +02:00
lns
5a9b40779d bump libnDPI to 04f5c5196e790db8b8cc39e42c8645fb7f3dd141 
* added custom nDPI logging callback

Signed-off-by: lns <matzeton@googlemail.com>
 2023-05-30 09:30:24 +02:00
lns
8a936a5072 Fixed integer overflow for tcp timeout (>INT_MAX). 
Signed-off-by: lns <matzeton@googlemail.com>
 2023-05-26 11:17:38 +02:00
Toni Uhlig
99accd03a2 Moved datalink json key/value from error to packet events and renamed it to pkt_datalink. 
Signed-off-by: Toni Uhlig <matzeton@googlemail.com>
 2023-04-21 16:48:40 +02:00
Toni Uhlig
655393e953 nDPid: Fixed base64encode bug which lead to invalid base64 strings. 
* py-semantic-validation: Decode base64 raw packet data as well
 * nDPIsrvd.py: Added PACKETS_PLEN_MAX
 * nDPIsrvd.py: Improved JSON parse error/exception handling

Signed-off-by: Toni Uhlig <matzeton@googlemail.com>
 2023-01-09 01:43:24 +01:00
Toni Uhlig
d21a38cf02 Limit the size of base64 serialized raw packet data (8192 bytes per packet). 
Signed-off-by: Toni Uhlig <matzeton@googlemail.com>
 2022-12-06 12:52:52 +01:00
Toni Uhlig
2c95b31210 nDPId-test: Reworked I/O handling to prevent some endless loop scenarios. Fixed a race condition in the memory wrapper as well. 
* nDPId: Instead of sending too long JSON strings, log an error and some parts.

Signed-off-by: Toni Uhlig <matzeton@googlemail.com>
 2022-12-02 22:11:57 +01:00
Toni Uhlig
64f6abfdbe Unified nDPId/nDPIsrvd command line argument storage. 
Signed-off-by: Toni Uhlig <matzeton@googlemail.com>
 2022-11-21 11:26:05 +01:00
Toni Uhlig
ce567ae5b7 Improved the point of time when to append the raw packet base64 data to the serializer. 
* nDPId-test: Increased the max-packets-per-flow-to-send from 3 to 5.
   This is quite useful for TCP as the first 3 packets are usually part of the three-way-handshake.

Signed-off-by: Toni Uhlig <matzeton@googlemail.com>
 2022-11-15 06:25:16 +01:00
Toni Uhlig
ea1698504c nDPIsrvd: Provide workaround for change user/group. 
* nDPId/nDPIsrvd/c-examples: Parameter parsing needs to be improved
                              if `strdup()` in combination with static strings is used.
 * Other non-critical fixes.

Signed-off-by: Toni Uhlig <matzeton@googlemail.com>
 2022-11-13 11:21:55 +01:00
Toni Uhlig
bc346a28f4 nDPId: Fixed base64 encoding issue. 
* The issue can result in an error message like:
   `Base64 encoding failed with: Buffer too small.`
   and also in too big JSON strings generated by nDPId
   which nDPIsrvd does not like as it's length is
   greater than `NETWORK_BUFFER_MAX_SIZE`.
 * nDPId will now obey `NETWORK_BUFFER_MAX_SIZE` while
   trying to base64 encode raw packet data.

Signed-off-by: Toni Uhlig <matzeton@googlemail.com>
 2022-11-13 09:26:04 +01:00
Toni Uhlig
7515c8aeec Experimental systemd support. 
Signed-off-by: Toni Uhlig <matzeton@googlemail.com>
 2022-11-06 12:58:55 +01:00
Toni Uhlig
49352698a0 nDPId: Added error event threshold to prevent event spamming which may be abused. 
Signed-off-by: Toni Uhlig <matzeton@googlemail.com>
 2022-10-17 06:36:30 +02:00
Toni Uhlig
4654faf381 Improved py-machine-learning example. 
* c-analysed: fixed quoting bug
 * nDPId: fixed invalid iat storing/serialisation
 * nDPId: free data analysis after event was sent

Signed-off-by: Toni Uhlig <matzeton@googlemail.com>
Signed-off-by: lns <matzeton@googlemail.com>
 2022-10-09 18:31:45 +02:00
Toni Uhlig
14f6b87551 Added nDPIsrvd-analysed to generate CSV files from analyse events. 
* nDPIsrvd.h: iterate over JSON arrays
 * nDPId: calculate l3 payload packet entropies for analysis

Signed-off-by: Toni Uhlig <matzeton@googlemail.com>
 2022-09-30 19:28:49 +02:00
Toni Uhlig
2103ee0811 Refactored client distributor C API. 
* Still not perfect, but the code before was not even able to deal with JSON arrays.
   Use common "speaking" function names for all functions in nDPIsrvd.h
 * Provide a more or less generic and easy extendable JSON walk function.
 * Modified C examples to align with the changed C API.
 * c-collectd: Reduced lot's of code duplication by providing mapping tables.
 * nDPId: IAT array requires one slot less (first packet has always an IAT of 0).

Signed-off-by: Toni Uhlig <matzeton@googlemail.com>
 2022-09-25 00:54:39 +02:00
Toni Uhlig
9a28475bba Improved flown analyse event: 
* store packet directions
 * merged direction based IATs
 * merged direction based PKTLENs

Signed-off-by: Toni Uhlig <matzeton@googlemail.com>
 2022-09-22 19:07:08 +02:00
Toni Uhlig
08f263e409 nDPId: Reduced flow-updates for TCP flows to 1/4 of the timeout value. 
* nDPId: Fixed broken validation tests.
 * nDPId: Removed TICK_RESOLUTION, not required anymore.
 * c-collectd: Improved total layer4 payload calculation/update handling.
 * c-collectd: Updated RRD Graph script according to total layer4 payload changes.
 * py-flow-info.py: Fixed several bugs and syntax errors.
 * Python scripts: Added dirname(argv[0]) as search path for nDPIsrvd.py.
 * nDPIsrvd&nDPId-test: Fixed missing EPOLLERR check.

Signed-off-by: Toni Uhlig <matzeton@googlemail.com>
 2022-09-19 19:39:49 +02:00