Commit Graph

54 Commits

Author SHA1 Message Date
Toni Uhlig
adb8fe96f5 CMake: add coverage-clean target and fix coverage dependency issue.
* improve/fix README

Signed-off-by: Toni Uhlig <matzeton@googlemail.com>
2024-08-20 11:49:38 +02:00
Toni Uhlig
4efe7e43a2 Improved installation instructions. Fixes #40.
Signed-off-by: Toni Uhlig <matzeton@googlemail.com>
2024-08-19 18:39:35 +02:00
Toni Uhlig
5290f76b5f flow-info.py: Set min risk severity required to print a risk.
* ReadMe update

Signed-off-by: Toni Uhlig <matzeton@googlemail.com>
2024-05-08 00:25:31 +02:00
Toni Uhlig
8949ba39e6 Added test mode for influx push daemon.
* required for regression testing
 * added new confidence value (match by custom rule)
 * updated / tweaked grafana exported dashboard

Signed-off-by: Toni Uhlig <matzeton@googlemail.com>
2024-02-08 01:01:35 +01:00
Toni Uhlig
556025b34d Removed API version macro check as it's inconsistent on different platforms.
* set min required nDPI version to 4.9.0

Signed-off-by: Toni Uhlig <matzeton@googlemail.com>
2024-02-06 10:49:47 +01:00
Toni Uhlig
feb2583ef6 bump libnDPI to 4543385d107fcc5a7e8632e35d9a60bcc40cb4f4
* incorporated API changes from nDPI

Signed-off-by: Toni Uhlig <matzeton@googlemail.com>
2024-02-06 10:34:52 +01:00
Toni Uhlig
53d8a28582 Replaced ambiguous naming of "JSON string" to more accurate "JSON message".
Signed-off-by: Toni Uhlig <matzeton@googlemail.com>
2023-12-04 13:01:27 +01:00
Paul Donald
a5dcc17396 Update README.md (#32)
Sp/gr. 

Co-authored-by: Toni <matzeton@googlemail.com>
2023-11-27 09:08:25 +01:00
Toni Uhlig
3416db11dc Updated ReadMe's, ToDo's and ChangeLog.
* 1.6-release

Signed-off-by: Toni Uhlig <matzeton@googlemail.com>
2023-11-20 23:39:47 +01:00
Toni Uhlig
17c21e1d27 Updated ToDo and added ChangeLog.
Signed-off-by: Toni Uhlig <matzeton@googlemail.com>
2023-11-05 16:00:47 +01:00
Toni Uhlig
44adfc0b7d Sonarcloud integration
Signed-off-by: Toni Uhlig <matzeton@googlemail.com>
2023-10-05 17:37:42 +02:00
Toni Uhlig
e42e3fe406 Serialize nDPId / libnDPI versions within daemon events.
* changed nDPI version hints / requirements

Signed-off-by: Toni Uhlig <matzeton@googlemail.com>
2023-09-10 19:32:08 +02:00
Toni Uhlig
7b15838696 Added docker build&push to the CI.
* update some git submodules

Signed-off-by: Toni Uhlig <matzeton@googlemail.com>
2023-08-03 10:00:50 +02:00
Toni Uhlig
d915530feb Circle CI integration
Signed-off-by: Toni Uhlig <matzeton@googlemail.com>
2023-07-27 18:39:11 +02:00
lns
2b1db0a556 Required libnDPI version increases (>=4.8.0) due to an API change.
* fix CI issues

Signed-off-by: lns <matzeton@googlemail.com>
2023-05-31 12:53:49 +02:00
Toni Uhlig
e9443d7618 Fix libnDPI build script.
* added ntop Webinar 2022 reference

Signed-off-by: Toni Uhlig <matzeton@googlemail.com>
2022-12-26 19:35:12 +01:00
Toni Uhlig
60741d5649 Strace support for diff tests.
* tiny README update

Signed-off-by: Toni Uhlig <matzeton@googlemail.com>
2022-12-03 01:21:20 +01:00
Toni Uhlig
532961af33 Fixed MD format issues.
Signed-off-by: Toni Uhlig <matzeton@googlemail.com>
2022-11-21 11:34:10 +01:00
Toni Uhlig
9b78939096 Updated README's.
Signed-off-by: Toni Uhlig <matzeton@googlemail.com>
2022-11-20 22:25:18 +01:00
Toni Uhlig
e5f4af4890 Special Thanks to Damiano Verzulli (@verzulli).
Signed-off-by: Toni Uhlig <matzeton@googlemail.com>
2022-10-20 16:12:40 +02:00
Toni Uhlig
3c7bd6a4ba Merge branch 'main' of github.com:utoni/nDPId 2022-09-19 19:39:54 +02:00
Toni Uhlig
08f263e409 nDPId: Reduced flow-updates for TCP flows to 1/4 of the timeout value.
* nDPId: Fixed broken validation tests.
 * nDPId: Removed TICK_RESOLUTION, not required anymore.
 * c-collectd: Improved total layer4 payload calculation/update handling.
 * c-collectd: Updated RRD Graph script according to total layer4 payload changes.
 * py-flow-info.py: Fixed several bugs and syntax errors.
 * Python scripts: Added dirname(argv[0]) as search path for nDPIsrvd.py.
 * nDPIsrvd&nDPId-test: Fixed missing EPOLLERR check.

Signed-off-by: Toni Uhlig <matzeton@googlemail.com>
2022-09-19 19:39:49 +02:00
Damiano Verzulli
ab7f7d05f3 Improve README
- link to already-existing JSON-schemas have been added
- a graphical schema detailing flow-events timeline have
  been added in both PNG and source-Drawio formats.
  Link to PNG have been included in the README
2022-09-19 17:23:11 +02:00
Toni Uhlig
31715295d9 bump libnDPI to 174cd739dbb1358ab012c4779e42e0221bef835c
* ReadMe stuff
 * OpenWrt Makefile: set NEED_LINKING_AGAINST_LIBM=ON

Signed-off-by: Toni Uhlig <matzeton@googlemail.com>
2022-09-18 13:34:43 +02:00
Toni Uhlig
d4633c1192 New flow event: 'analysis'.
* The goal was to provide a separate event for extracted feature that are not required
   and only useful for a few (e.g. someone who wants do ML).
 * Increased network buffer size to 32kB (8192 * 4).
 * Switched timestamp precision from ms to us for *ALL* timestamps.

Signed-off-by: Toni Uhlig <matzeton@googlemail.com>
2022-09-13 22:05:08 +02:00
Toni Uhlig
ac2e5ed796 CI: fix minimum supported libnDPI version
Signed-off-by: Toni Uhlig <matzeton@googlemail.com>
2022-09-06 16:56:35 +02:00
lns
d646ec5ab4 nDPId: Fixed fcntl() issue; invalid fcntl() set after a blocking-write.
* nDPId: imrpvoed collector socket error messages on connect/write/etc failures
 * reverted `netcat` parts of the README

Signed-off-by: lns <matzeton@googlemail.com>
2022-08-29 15:29:07 +02:00
lns
dea30501a4 Add documentation about events and flow states.
Signed-off-by: lns <matzeton@googlemail.com>
2022-08-27 14:18:59 +02:00
Damiano Verzulli
96b9129918 Improve README
- slightly improve the README to better enhance the streaming
  capability of `nDPId`, regardless of `nDPIsrvd`
- add a screencast showing the install step and the
  `nDPId` usage, alone (with ncat as unix-socket listener)
- add "build" to .gitignore
2022-08-12 11:10:45 +02:00
Toni Uhlig
ef94b83a62 Replaced outdated nDPI version info with the correct one.
* add CI job to verify the lowest known-to-work-libnDPI-version

Signed-off-by: Toni Uhlig <matzeton@googlemail.com>
2022-07-14 04:09:15 +02:00
Toni Uhlig
28602ca095 README update
Signed-off-by: Toni Uhlig <matzeton@googlemail.com>
2022-07-12 16:03:51 +02:00
lns
3caf7727fd bump libnDPI to 0b3f8ed849cdf9971224c49a3958f0904a2bbbb5
* README/nDPId: fixed typ0

Signed-off-by: lns <matzeton@googlemail.com>
2022-06-06 00:34:13 +02:00
lns
f5b0021413 README update
Signed-off-by: lns <matzeton@googlemail.com>
2022-05-31 23:57:22 +02:00
Toni
73ca7fff3c Updated CI badges. 2022-05-08 21:41:01 +02:00
Toni Uhlig
cb80c415d8 Improved py-flow-info to provide more optional information about received timestamps.
* py-flow-dashboard: Added color mapping for PieCharts/Graph that make more sense
 * nDPId: Renamed `flow_type' to a more precisely `flow_state'
 * nDPId: Changed the default setting to process only as much packets as libnDPI does

Signed-off-by: Toni Uhlig <matzeton@googlemail.com>
2022-02-05 15:27:13 +01:00
Toni Uhlig
7022d0b1c5 nDPIsrvd: Fixed memory leak caused be not clearing buffer cache after a client disconnected.
* README.md: Fixed a typ0 and added a meh image from examples/py-flow-dashboard/flow-dash.py

Signed-off-by: Toni Uhlig <matzeton@googlemail.com>
2022-01-26 15:38:43 +01:00
Toni Uhlig
9e07a57566 Major nDPId extension. Sorry for the huge commit.
- nDPId: fixed invalid IP4/IP6 tuple compare
 - nDPIsrvd: fixed caching issue (finally)
 - added tiny c example (can be used to check flow manager sanity)
 - c-captured: use flow_last_seen timestamp from `struct nDPIsrvd_flow`
 - README.md update: added example JSON sequence
 - nDPId: added new flow event `update` necessary for correct
   timeout handling (and other future use-cases)
 - nDPIsrvd.h and nDPIsrvd.py: switched to an instance
   (consists of an alias/source tuple) based flow manager
 - every flow related event **must** now serialize `alias`, `source`,
   `flow_id`, `flow_last_seen` and `flow_idle_time` to make the timeout
   handling and verification process work correctly
 - nDPIsrvd.h: ability to profile any dynamic memory (de-)allocation
 - nDPIsrvd.py: removed PcapPacket class (unused)
 - py-flow-dashboard and py-flow-multiprocess: fixed race condition
 - py-flow-info: print statusbar with probably useful information
 - nDPId/nDPIsrvd.h: switched from packet-flow only timestamps (`pkt_*sec`)
   to a generic flow event timestamp `ts_msec`
 - nDPId-test: added additional checks
 - nDPId: increased ICMP flow timeout
 - nDPId: using event based i/o if capturing packets from a device
 - nDPIsrvd: fixed memory leak on shutdown if remote descriptors
   were still connected

Signed-off-by: Toni Uhlig <matzeton@googlemail.com>
2022-01-20 00:50:38 +01:00
Toni Uhlig
7829bfe4e6 CI extended and fixups
Signed-off-by: Toni Uhlig <matzeton@googlemail.com>
2021-09-19 11:30:55 +02:00
Toni Uhlig
c152e41cfb README.md ascii update
Signed-off-by: Toni Uhlig <matzeton@googlemail.com>
2021-09-14 18:54:33 +02:00
Toni Uhlig
f4c8d96dd9 Gitlab-CI
Signed-off-by: Toni Uhlig <matzeton@googlemail.com>
2021-08-03 13:06:12 +02:00
Toni Uhlig
9ffaeef24d README.md update
Signed-off-by: Toni Uhlig <matzeton@googlemail.com>
2021-05-14 00:16:47 +02:00
Toni Uhlig
182867a071 Reduced superfluous Travis-CI yaml content.
Signed-off-by: Toni Uhlig <matzeton@googlemail.com>
2021-05-12 15:17:07 +02:00
Toni Uhlig
fa079d2346 Git submodule libnDPI update.
* enable ctest to run integration tests (**only** if BUILD_NDPI=ON)

Signed-off-by: Toni Uhlig <matzeton@googlemail.com>
2021-05-12 12:46:49 +02:00
Toni Uhlig
785603c276 Clone nDPI as submodule via CMake (if BUILD_NDPI=ON).
Signed-off-by: Toni Uhlig <matzeton@googlemail.com>
2021-05-11 14:18:31 +02:00
Toni Uhlig
5b0a751159 README.md updated
* removed all those outdated Makefile.old references
 * added additional CMake build instructions

Signed-off-by: Toni Uhlig <matzeton@googlemail.com>
2021-05-10 15:38:01 +02:00
Toni Uhlig
4e3dda70a3 Unified and improved dependency build/mgmt via CMake and travis-ci.
Signed-off-by: Toni Uhlig <matzeton@googlemail.com>
2021-04-22 17:37:55 +02:00
Toni Uhlig
ef4a22ffac Updated README.md
* added examples/README.md and schema/README.md

Signed-off-by: Toni Uhlig <matzeton@googlemail.com>
2021-03-24 14:44:27 +01:00
Toni Uhlig
1c3ef69faa nDPIsrvd collectd-exec overhaul.
* Install targets updated.
 * Removed nDPIsrvd.h token validation function (done automatically by token_get).

Signed-off-by: Toni Uhlig <matzeton@googlemail.com>
2021-03-15 14:39:43 +01:00
Toni Uhlig
257cce1dcc Fixed braindead failure - Increase of NETWORK_BUFFER_MAX_SIZE > 9999 means to also increase NETWORK_BUFFER_LENGTH_DIGITS to 5.
* Fixed ARM32 xcompile warnings; Other GCC versions, other uint64_t's..
 * Replaced ridiculous nDPIsrvd_JSON_BYTES with NETWORK_BUFFER_LENGTH_DIGITS.

Signed-off-by: Toni Uhlig <matzeton@googlemail.com>
2021-01-27 18:05:13 +01:00
Toni Uhlig
9e4e41e5f6 Updated README.md ASCII architecture.
Signed-off-by: Toni Uhlig <matzeton@googlemail.com>
2020-09-29 17:43:36 +02:00