18 Commits

Author SHA1 Message Date
Toni Uhlig
a9934e9c9e Removed nDPI/nDPId version/api serialization for nDPId-test to reduce result diff's
* fixed some SonarCloud complains

Signed-off-by: Toni Uhlig <matzeton@googlemail.com>
2025-09-09 12:52:58 +02:00
Toni Uhlig
19036951c7 bump libnDPI to 1216ec6a2719408a487f696f5b601bdb9eec727d
* incorporated upstream API changes related to detection protocol bitmasks
 * added missing flow detection categories

Signed-off-by: Toni Uhlig <matzeton@googlemail.com>
2025-09-09 12:05:26 +02:00
Toni Uhlig
4e7e361d84 bump libnDPI to f8869cd670adc439cc41bde0bd04960e1befafc5
* fix API issue due to changed name of a public struct

Signed-off-by: Toni Uhlig <matzeton@googlemail.com>
2025-05-20 11:05:53 +02:00
Toni Uhlig
1aa7d9bdb6 nDPId daemon status event: serialize nDPI API version + Size/Flow
Signed-off-by: Toni Uhlig <matzeton@googlemail.com>
2024-11-17 13:12:33 +01:00
Toni Uhlig
7b2cd268bf Updated JSON schema files and a test to make use of the UUID feature.
Signed-off-by: Toni Uhlig <matzeton@googlemail.com>
2024-10-29 15:25:19 +01:00
Toni
5e4005162b Add PF_RING support. (#38) 2024-08-19 18:33:18 +02:00
Toni Uhlig
930aaf9276 Added global (heap) memory stats for daemon status events.
* added new CMake option `ENABLE_MEMORY_STATUS` to restore the old behavior
   (and increase performance)
 * splitted `ENABLE_MEMORY_PROFILING` into `ENABLE_MEMORY_STATUS` and `ENABLE_MEMORY_PROFILING`

Signed-off-by: Toni Uhlig <matzeton@googlemail.com>
2024-06-19 14:25:42 +02:00
Toni Uhlig
e42e3fe406 Serialize nDPId / libnDPI versions within daemon events.
* changed nDPI version hints / requirements

Signed-off-by: Toni Uhlig <matzeton@googlemail.com>
2023-09-10 19:32:08 +02:00
Toni Uhlig
08f263e409 nDPId: Reduced flow-updates for TCP flows to 1/4 of the timeout value.
* nDPId: Fixed broken validation tests.
 * nDPId: Removed TICK_RESOLUTION, not required anymore.
 * c-collectd: Improved total layer4 payload calculation/update handling.
 * c-collectd: Updated RRD Graph script according to total layer4 payload changes.
 * py-flow-info.py: Fixed several bugs and syntax errors.
 * Python scripts: Added dirname(argv[0]) as search path for nDPIsrvd.py.
 * nDPIsrvd&nDPId-test: Fixed missing EPOLLERR check.

Signed-off-by: Toni Uhlig <matzeton@googlemail.com>
2022-09-19 19:39:49 +02:00
Toni Uhlig
d4633c1192 New flow event: 'analysis'.
* The goal was to provide a separate event for extracted feature that are not required
   and only useful for a few (e.g. someone who wants do ML).
 * Increased network buffer size to 32kB (8192 * 4).
 * Switched timestamp precision from ms to us for *ALL* timestamps.

Signed-off-by: Toni Uhlig <matzeton@googlemail.com>
2022-09-13 22:05:08 +02:00
Toni Uhlig
b5d4da8793 bump libnDPI to 8f6a006e36eef0ae386f7e663d3ebecfad6a2dc9
* try to use same wording wherever possible e.g.
   renamed workflow->total_l4_data_len to workflow->total_l4_payload_len

Signed-off-by: Toni Uhlig <matzeton@googlemail.com>
2022-07-01 13:50:53 +02:00
Toni Uhlig
c0b7bdacbc Reworked nDPIsrvd.h C-API.
* nDPIsrvd.h: Provide nDPId thread storage.
 * nDPIsrvd.py: Fixed instance cleanup bug.
 * nDPIsrvd.h: Support for instance/thread user data and cleanup callback.
 * nDPIsrvd.h: Most recent flow time stored in thread ht instead of instance ht.
 * nDPId: Moved flow logger out the memory profilier into SIGUSR1 signal handling.
 * nDPId: Added signal fd to be usable within epoll's event handling (live-capture only!)
 * nDPId: Added information about ZLib compressions to daemon status/shutdown events.

Signed-off-by: Toni Uhlig <matzeton@googlemail.com>
2022-03-21 15:56:01 +01:00
Toni Uhlig
46f68501d5 Added daemon event: DAEMON_EVENT_STATUS (periodically send's daemon statistics.)
* Improved distributor timeout handling (per-thread).
 * flow-info.py / flow-dash.py: Distinguish between flow risk severities.
 * nDPId: Skip tag switch datalink packet dissection / processing.
 * nDPId: Fixed incorrect value for current active flows.
 * Improved JSON schema's.

Signed-off-by: Toni Uhlig <matzeton@googlemail.com>
2022-03-06 17:38:05 +01:00
Toni Uhlig
9e07a57566 Major nDPId extension. Sorry for the huge commit.
- nDPId: fixed invalid IP4/IP6 tuple compare
 - nDPIsrvd: fixed caching issue (finally)
 - added tiny c example (can be used to check flow manager sanity)
 - c-captured: use flow_last_seen timestamp from `struct nDPIsrvd_flow`
 - README.md update: added example JSON sequence
 - nDPId: added new flow event `update` necessary for correct
   timeout handling (and other future use-cases)
 - nDPIsrvd.h and nDPIsrvd.py: switched to an instance
   (consists of an alias/source tuple) based flow manager
 - every flow related event **must** now serialize `alias`, `source`,
   `flow_id`, `flow_last_seen` and `flow_idle_time` to make the timeout
   handling and verification process work correctly
 - nDPIsrvd.h: ability to profile any dynamic memory (de-)allocation
 - nDPIsrvd.py: removed PcapPacket class (unused)
 - py-flow-dashboard and py-flow-multiprocess: fixed race condition
 - py-flow-info: print statusbar with probably useful information
 - nDPId/nDPIsrvd.h: switched from packet-flow only timestamps (`pkt_*sec`)
   to a generic flow event timestamp `ts_msec`
 - nDPId-test: added additional checks
 - nDPId: increased ICMP flow timeout
 - nDPId: using event based i/o if capturing packets from a device
 - nDPIsrvd: fixed memory leak on shutdown if remote descriptors
   were still connected

Signed-off-by: Toni Uhlig <matzeton@googlemail.com>
2022-01-20 00:50:38 +01:00
Toni Uhlig
da4942b41c Use layer4 specific flow timeouts.
* default values "stolen" from nf_conntrack

Signed-off-by: Toni Uhlig <matzeton@googlemail.com>
2021-05-13 15:41:24 +02:00
Toni Uhlig
4e583cd4de Added JSON schema validation to run_tests.sh
* Python3 scripts are now compatible with versions <3.6
 * improved and prettified run_tests.sh

Signed-off-by: Toni Uhlig <matzeton@googlemail.com>
2021-04-09 14:12:48 +02:00
Toni Uhlig
0b5b177c14 Extended nDPIsrvd.h with address parsing.
* nDPId supports looading of custom nDPI protocol/category files
 * extended JSON schemas according to nDPI / nDPId JSON serializing
 * removed memory holes in nDPId
 * extended examples/c-captured

Signed-off-by: Toni Uhlig <matzeton@googlemail.com>
2021-02-23 14:46:47 +01:00
Toni Uhlig
a1805eb891 Added JSON schema files and a Python schema validator.
Signed-off-by: Toni Uhlig <matzeton@googlemail.com>
2021-02-17 14:00:10 +01:00