github-personal/nDPId
1
0
Fork 0
mirror of https://github.com/outbackdingo/nDPId.git synced 2026-01-27 18:19:39 +00:00
Code Issues Packages Projects Releases Wiki Activity
548 Commits 7 Branches 11 Tags
b7d8564b6542f57812e243f2d86e7e45feb7ad58
Commit Graph

548 Commits

This Branch
This Branch
All Branches
Author SHA1 Message Date
lns
b7d8564b65 Generate code coverage w/o external shell script, use CMake. 
* upload codecov/dist artifacts

Signed-off-by: lns <matzeton@googlemail.com>
 2022-10-03 15:45:17 +02:00
lns
49ea4f8474 Small fixes. 
Signed-off-by: lns <matzeton@googlemail.com>
 2022-10-01 22:37:25 +02:00
Toni Uhlig
b6060b897e c-analysed: improved feature extraction from "analyse" events 
* c-captured: update detected risks on "detection-update" events
 * c-collectd: added missing flow breed
 * c-collectd: PUTVAL macros are more flexible now

Signed-off-by: Toni Uhlig <matzeton@googlemail.com>
 2022-10-01 18:01:56 +02:00
Toni Uhlig
14f6b87551 Added nDPIsrvd-analysed to generate CSV files from analyse events. 
* nDPIsrvd.h: iterate over JSON arrays
 * nDPId: calculate l3 payload packet entropies for analysis

Signed-off-by: Toni Uhlig <matzeton@googlemail.com>
 2022-09-30 19:28:49 +02:00
Toni Uhlig
74f71643da nDPId-test: Force collector blocking mode. 
Signed-off-by: Toni Uhlig <matzeton@googlemail.com>
 2022-09-25 16:24:05 +02:00
Toni Uhlig
2103ee0811 Refactored client distributor C API. 
* Still not perfect, but the code before was not even able to deal with JSON arrays.
   Use common "speaking" function names for all functions in nDPIsrvd.h
 * Provide a more or less generic and easy extendable JSON walk function.
 * Modified C examples to align with the changed C API.
 * c-collectd: Reduced lot's of code duplication by providing mapping tables.
 * nDPId: IAT array requires one slot less (first packet has always an IAT of 0).

Signed-off-by: Toni Uhlig <matzeton@googlemail.com>
 2022-09-25 00:54:39 +02:00
Toni Uhlig
36f1786bde nDPIsrvd.h: Fixed bug during token parsing/hashing. Do not hash array contents. 
Signed-off-by: Toni Uhlig <matzeton@googlemail.com>
 2022-09-23 00:13:19 +02:00
Toni Uhlig
9a28475bba Improved flown analyse event: 
* store packet directions
 * merged direction based IATs
 * merged direction based PKTLENs

Signed-off-by: Toni Uhlig <matzeton@googlemail.com>
 2022-09-22 19:07:08 +02:00
Toni Uhlig
28971cd764 flow-info.py: Command line arguments --no-color, --no-statusbar (both useful for tests/CI) and --print-analyse-results. 
* run_tests.sh: Use flow-info.py for additional DIFF tests.

Signed-off-by: Toni Uhlig <matzeton@googlemail.com>
 2022-09-22 08:00:21 +02:00
Toni Uhlig
3c7bd6a4ba Merge branch 'main' of github.com:utoni/nDPId 2022-09-19 19:39:54 +02:00
Toni Uhlig
08f263e409 nDPId: Reduced flow-updates for TCP flows to 1/4 of the timeout value. 
* nDPId: Fixed broken validation tests.
 * nDPId: Removed TICK_RESOLUTION, not required anymore.
 * c-collectd: Improved total layer4 payload calculation/update handling.
 * c-collectd: Updated RRD Graph script according to total layer4 payload changes.
 * py-flow-info.py: Fixed several bugs and syntax errors.
 * Python scripts: Added dirname(argv[0]) as search path for nDPIsrvd.py.
 * nDPIsrvd&nDPId-test: Fixed missing EPOLLERR check.

Signed-off-by: Toni Uhlig <matzeton@googlemail.com>
 2022-09-19 19:39:49 +02:00
Damiano Verzulli
ab7f7d05f3 Improve README 
- link to already-existing JSON-schemas have been added
- a graphical schema detailing flow-events timeline have
  been added in both PNG and source-Drawio formats.
  Link to PNG have been included in the README
 2022-09-19 17:23:11 +02:00
Toni Uhlig
015a739efd Added layer4 payload length bins. 
Signed-off-by: Toni Uhlig <matzeton@googlemail.com>
 2022-09-19 10:14:37 +02:00
Toni Uhlig
31715295d9 bump libnDPI to 174cd739dbb1358ab012c4779e42e0221bef835c 
* ReadMe stuff
 * OpenWrt Makefile: set NEED_LINKING_AGAINST_LIBM=ON

Signed-off-by: Toni Uhlig <matzeton@googlemail.com>
 2022-09-18 13:34:43 +02:00
Toni Uhlig
06bce24c0e Add -Werror to OpenWrt package TARGET_CFLAGS. 
Signed-off-by: Toni Uhlig <matzeton@googlemail.com>
 2022-09-17 18:53:17 +02:00
Toni Uhlig
efaa76e978 Provide thread sync via locking on architectures that do not support Compare&Swap. 
Signed-off-by: Toni Uhlig <matzeton@googlemail.com>
 2022-09-17 18:27:17 +02:00
Toni Uhlig
b3e9af495c Add OpenWrt CI via Github Actions. 
Signed-off-by: Toni Uhlig <matzeton@googlemail.com>
 2022-09-17 10:31:26 +02:00
lns
b8cfe1d6d3 Fixed last pkt time. 
Signed-off-by: lns <matzeton@googlemail.com>
 2022-09-14 11:22:41 +02:00
Toni Uhlig
d4633c1192 New flow event: 'analysis'. 
* The goal was to provide a separate event for extracted feature that are not required
   and only useful for a few (e.g. someone who wants do ML).
 * Increased network buffer size to 32kB (8192 * 4).
 * Switched timestamp precision from ms to us for *ALL* timestamps.

Signed-off-by: Toni Uhlig <matzeton@googlemail.com>
 2022-09-13 22:05:08 +02:00
Toni Uhlig
aca1615dc1 OpenWrt packaging support. 
Signed-off-by: Toni Uhlig <matzeton@googlemail.com>
 2022-09-13 13:53:48 +02:00
Toni Uhlig
94aa02b298 nDPIsrvd-collectd: Stdout should be unbuffered. 
Signed-off-by: Toni Uhlig <matzeton@googlemail.com>
 2022-09-12 13:32:50 +02:00
Toni Uhlig
20ced3e636 nDPIsrvd-collectd: RRD Graph generation script and a basic static HTML5 website for viewing the generated image files. 
Signed-off-by: Toni Uhlig <matzeton@googlemail.com>
 2022-09-12 13:23:50 +02:00
Toni Uhlig
83409e5b79 Use CMake XCompile and collect host-triplet from ${CC}. 
Signed-off-by: Toni Uhlig <matzeton@googlemail.com>
 2022-09-11 16:50:56 +02:00
Toni Uhlig
3bc6627dcc nDPId: Removed thread_id nonsense as it does not provide any useful information and is not portable at all, not even on Linux systems .. 
* nDPId: Removed blocking I/O warning, which causes logspams..

Signed-off-by: Toni Uhlig <matzeton@googlemail.com>
 2022-09-10 23:11:03 +02:00
Toni Uhlig
7594180301 include fix 
Signed-off-by: Toni Uhlig <matzeton@googlemail.com>
 2022-09-10 22:55:06 +02:00
Toni Uhlig
a992c79ab6 Fixed compilation warnings on linux32 platforms. 
Signed-off-by: Toni Uhlig <matzeton@googlemail.com>
 2022-09-10 22:45:12 +02:00
Toni Uhlig
6fe5d1da69 Do not use pthread_t as numeric value. Some systems define pthread_t as struct * 
Signed-off-by: Toni Uhlig <matzeton@googlemail.com>
 2022-09-10 22:34:00 +02:00
Toni Uhlig
38c71af2f4 nDPIsrvd: Fixed NUL pointer deref during logging attempt. 
Signed-off-by: Toni Uhlig <matzeton@googlemail.com>
 2022-09-10 16:46:25 +02:00
Toni Uhlig
ac2e5ed796 CI: fix minimum supported libnDPI version 
Signed-off-by: Toni Uhlig <matzeton@googlemail.com>
 2022-09-06 16:56:35 +02:00
Toni Uhlig
f9bd7d29ce Bump libnDPI to 37f918322c0a489b5143a987c8f1a44a6f78a6f3 and updated flow json schema file. 
* export env vars AR / CMAKE_C_COMPILER_AR and RANLIB / CMAKE_C_COMPILER_RANLIB while building libnDPI
 * nDPId check API version during startup (macro vs. function call) and print a warning if they are different

Signed-off-by: Toni Uhlig <matzeton@googlemail.com>
 2022-09-06 14:50:46 +02:00
Toni Uhlig
c5c7d83c97 Added https://gitlab.com/verzulli/ndpid-rt-analyzer.git to examples. 
Signed-off-by: Toni Uhlig <matzeton@googlemail.com>
 2022-09-06 11:08:01 +02:00
Toni Uhlig
70f517b040 Merge branch 'main' of github.com:utoni/nDPId 2022-09-04 17:26:21 +02:00
Toni Uhlig
dcf78ad3ed Disable timestamp generation in nDPIsrvd-collectd as default. 
* collectd's rrdtool write plugin does silently fail with those ones (dunno why)

Signed-off-by: Toni Uhlig <matzeton@googlemail.com>
 2022-09-04 17:24:10 +02:00
lns
d646ec5ab4 nDPId: Fixed fcntl() issue; invalid fcntl() set after a blocking-write. 
* nDPId: imrpvoed collector socket error messages on connect/write/etc failures
 * reverted `netcat` parts of the README

Signed-off-by: lns <matzeton@googlemail.com>
 2022-08-29 15:29:07 +02:00
lns
dea30501a4 Add documentation about events and flow states. 
Signed-off-by: lns <matzeton@googlemail.com>
 2022-08-27 14:18:59 +02:00
lns
d9fadae718 nDPId: improved error messages if UNIX/UDP endpoint refuses connections/datagrams 
Signed-off-by: lns <matzeton@googlemail.com>
 2022-08-27 14:18:59 +02:00
Toni Uhlig
5e09a00062 nDPId: support for custom UDP endpoints 
Signed-off-by: Toni Uhlig <matzeton@googlemail.com>
 2022-08-27 14:18:59 +02:00
lns
d0b0a50609 nDPId: improved error messages if UNIX/UDP endpoint refuses connections/datagrams 
Signed-off-by: lns <matzeton@googlemail.com>
 2022-08-27 13:04:17 +02:00
Toni Uhlig
e2e7c82d7f nDPId: support for custom UDP endpoints 
Signed-off-by: Toni Uhlig <matzeton@googlemail.com>
 2022-08-27 13:04:17 +02:00
Toni Uhlig
0fd59f060e Split *_l4_payload_len' into *_src_l4_payload_len' and `*_dst_l4_payload_len'. 
Signed-off-by: Toni Uhlig <matzeton@googlemail.com>
 2022-08-15 22:55:19 +02:00
lns
905545487d Split flow_packets_processed' into flow_src_packets_processed' and `flow_dst_packets_processed'. 
* no use for `flow_avg_l4_payload_len' -> removed
 * test/run_tests.sh does not fail if git-worktree's are used

Signed-off-by: lns <matzeton@googlemail.com>
 2022-08-15 18:36:49 +02:00
Toni Uhlig
2cb2c86cb5 c-collectd: fixed incorrect PUTVAL 
* get rid of types.db

Signed-off-by: Toni Uhlig <matzeton@googlemail.com>
 2022-08-15 16:42:59 +02:00
Toni
8c092dacfe Merge pull request #2 from verzulli/main 
Improved README.
 2022-08-12 18:10:34 +02:00
Damiano Verzulli
96b9129918 Improve README 
- slightly improve the README to better enhance the streaming
  capability of `nDPId`, regardless of `nDPIsrvd`
- add a screencast showing the install step and the
  `nDPId` usage, alone (with ncat as unix-socket listener)
- add "build" to .gitignore
 2022-08-12 11:10:45 +02:00
lns
ae37631e23 Do not SIGSEGV if a subopt has no value. 
Signed-off-by: lns <matzeton@googlemail.com>
 2022-08-08 09:33:26 +02:00
Toni Uhlig
ef94b83a62 Replaced outdated nDPI version info with the correct one. 
* add CI job to verify the lowest known-to-work-libnDPI-version

Signed-off-by: Toni Uhlig <matzeton@googlemail.com>
 2022-07-14 04:09:15 +02:00
Toni Uhlig
fc442180da c-collectd: fixed possible undefined behavior 
Signed-off-by: Toni Uhlig <matzeton@googlemail.com>
 2022-07-14 03:48:57 +02:00
Toni Uhlig
a606586a32 bump libnDPI to 7c19de49047a5731f3107ff17854e9afe839cc61 
Signed-off-by: Toni Uhlig <matzeton@googlemail.com>
 2022-07-14 03:48:06 +02:00
Toni Uhlig
4a397ac646 Github Actions: Renamed branch 'master' to 'main'. 
Signed-off-by: Toni Uhlig <matzeton@googlemail.com>
 2022-07-12 16:06:05 +02:00
Toni Uhlig
28602ca095 README update 
Signed-off-by: Toni Uhlig <matzeton@googlemail.com>
 2022-07-12 16:03:51 +02:00