Commit Graph

  • 0a7ad7a76a nDPId-test: added JSON distribution + JSON parsing (Multithreaded design re-using most of nDPId/nDPIsrvd core) Toni Uhlig 2021-04-08 20:33:25 +02:00
  • e576162a43 Reverted internal ndpi structs to dynamic memory allocation hoping that it will help to reduce the average memory consumption. How? After the detection finished, internal ndpi structs can be free'd as they are not needed anymore. Toni Uhlig 2021-03-26 14:46:35 +01:00
  • 38c6904bff run_tests.sh checks/print-stderr-on-fail Toni Uhlig 2021-03-24 15:04:04 +01:00
  • ef4a22ffac Updated README.md Toni Uhlig 2021-03-24 14:35:54 +01:00
  • 1ab6b9e042 Updated test outputs and pinned travis-ci's nDPI git repo sync to a specific commit hash. Toni Uhlig 2021-03-24 12:46:25 +01:00
  • 77b4b88b14 Added diff'able nDPId-test JSON dumps and travis-ci integration. Toni Uhlig 2021-03-24 11:29:26 +01:00
  • e835d36f63 Added nDPId-test as all-in-one JSON dumper. Toni Uhlig 2021-03-24 11:17:49 +01:00
  • bdc8c5df2a Reduced code duplication. Preps for nDPId-test. Toni Uhlig 2021-03-23 14:16:17 +01:00
  • c68c1750ba Switched to CMake build system. Toni Uhlig 2021-03-17 17:41:19 +01:00
  • 1c3ef69faa nDPIsrvd collectd-exec overhaul. Toni Uhlig 2021-03-15 14:09:20 +01:00
  • 9a06b97473 Fixed collectd-exec issues. Toni Uhlig 2021-03-12 17:28:10 +01:00
  • 772b67b767 Added basic collectd-exec example. Toni Uhlig 2021-03-10 18:18:30 +01:00
  • 412d8feba0 Added missing enum strings. Toni Uhlig 2021-03-09 17:46:16 +01:00
  • 1073c9626b nDPIsrvd refactoring Toni Uhlig 2021-03-05 12:24:24 +01:00
  • 4699263d65 Fixed possible NULL pointer deref in dependencies/nDPIsrvd.h and examples/c-captured/c-captured.c Toni Uhlig 2021-02-28 11:27:22 +01:00
  • 06ff353099 Added JA3 / SSL SHA1 fingerprint blacklists. Toni Uhlig 2021-02-27 17:38:31 +01:00
  • e0310d7e1d Finalized examples/c-captured to dump packet bytes to PCAP for further analysis. Toni Uhlig 2021-02-24 18:43:26 +01:00
  • 0b5b177c14 Extended nDPIsrvd.h with address parsing. Toni Uhlig 2021-02-23 14:46:47 +01:00
  • 0a6d44dc60 C-Api uses similiar flow key hash calculation as Python-Api. Toni Uhlig 2021-02-19 18:39:14 +01:00
  • 9ccd52134b Reverted f8692c6be1 Toni Uhlig 2021-02-17 14:53:04 +01:00
  • a1805eb891 Added JSON schema files and a Python schema validator. Toni Uhlig 2021-02-17 14:00:10 +01:00
  • 893f437051 Aligned nDPIsrvd.py to nDPIsrvd.h for consistency. Toni Uhlig 2021-02-16 20:37:29 +01:00
  • 7218990e58 Removed superfluous nDPI (header) dependency for examples/c-captured. Use header definitions from libc instead. Toni Uhlig 2021-02-06 16:01:54 +01:00
  • 69b4e662c7 nDPIsrvd C API overhaul and massive simplification. Toni Uhlig 2021-02-01 15:36:00 +01:00
  • 257cce1dcc Fixed braindead failure - Increase of NETWORK_BUFFER_MAX_SIZE > 9999 means to also increase NETWORK_BUFFER_LENGTH_DIGITS to 5. Toni Uhlig 2021-01-27 17:53:18 +01:00
  • 9564b0ce2c Increased JSON buffer size to 12288 (libnDPI serializes more and more information). Toni Uhlig 2021-01-27 17:10:06 +01:00
  • 102b61175c All colors are beautiful for py-flow-info. Toni Uhlig 2021-01-26 19:07:16 +01:00
  • 25c23e3d23 Finished (mostly) C guessed/undetected flow dumper. Toni Uhlig 2021-01-26 18:31:24 +01:00
  • a94ae79cc4 Switched to a slightly modified base64 encoder basicially from wikibooks. Toni Uhlig 2021-01-20 16:15:42 +01:00
  • 3a1afb9434 Added simple python json out example used for comparing JSON output. Toni Uhlig 2021-01-20 16:10:16 +01:00
  • f5d5c076a3 Added internal/external packet processing modes. #2 Toni Uhlig 2021-01-10 19:08:03 +01:00
  • b82a535a45 Increased network buffer size for python scripts (and raise exception if capacity reached). Toni Uhlig 2021-01-08 10:54:44 +01:00
  • f637802c0a Added internal/external packet processing modes. Toni Uhlig 2021-01-06 19:01:27 +01:00
  • 613e60ca2a Declared nDPI id and flow structs in nDPId flow info struct. Two reasons: * reduce heap memory allocations * nDPId flow info struct may be inflated in the future (more bytes to compress) Toni Uhlig 2020-12-30 13:22:15 +01:00
  • e0be911633 Set an optional bpf filter string for pcap based packet capture. Toni Uhlig 2020-12-29 17:52:37 +01:00
  • 9f3d7b479c nDPIsrvd: Distributor clients which are too slow can cause buffer bloat. Switching back to blocking mode works as a quick fix but is not sufficient. See comments. Toni Uhlig 2020-12-19 20:51:21 +01:00
  • b37cafe910 Improved testing capabilities via Makefile. Toni Uhlig 2020-12-15 10:32:16 +01:00
  • cda7124fa5 nDPId option validation is now less restrictive Toni Uhlig 2020-12-14 23:25:57 +01:00
  • e864787848 Generic nDPIsrvd.h event handling integration and flow tracking with uthash. Toni Uhlig 2020-12-06 17:01:06 +01:00
  • 00e5132a80 Merge commit 'c8bf38e5fb717d40635a2a89b22ed71b0de4266b' as 'dependencies/uthash' Toni Uhlig 2020-12-01 13:33:34 +01:00
  • c8bf38e5fb Squashed 'dependencies/uthash/' content from commit 8e67ced Toni Uhlig 2020-12-01 13:33:34 +01:00
  • 32b192df3b Removed all LUA leftover as preparation for UTHASH. Toni Uhlig 2020-12-01 13:31:07 +01:00
  • 6f734f9eff Set rpath when compiling with use of pkg-config. Toni Uhlig 2020-12-01 10:28:44 +01:00
  • f8692c6be1 Let travis use pcap files from nDPI to produce some JSON output. (disabled, needs further testing..) Toni Uhlig 2020-11-19 19:34:11 +01:00
  • a9b87e5739 Moved datalink layer processing into a separate function. Toni Uhlig 2020-11-18 18:16:00 +01:00
  • ba179cc724 Increased serialization buffer size as required for recent nDPI builds. Toni Uhlig 2020-11-18 18:06:55 +01:00
  • dd5ff3b8ed Refactored Makefile pkg-config part to be able to work with multiple pkg-config based projects. Toni Uhlig 2020-11-18 13:26:12 +01:00
  • 54dd72676d Provide functionality do deal with tokens from JSMN. Toni Uhlig 2020-11-17 19:29:10 +01:00
  • a03e0c8ba8 Fixed inconsistent callback function call if a token parsed. Toni Uhlig 2020-11-13 19:49:15 +01:00
  • aac0e77a2c Add 'dependencies/jsmn/' from commit '053d3cd29200edb1bfd181d917d140c16c1f8834' Toni Uhlig 2020-11-13 14:45:57 +01:00
  • f79991d70c Re-Cloned JSMN as subtree. Toni Uhlig 2020-11-13 14:44:39 +01:00
  • 7853c79cd6 Renamed misleading "contrib" subfolder to "dependencies". Toni Uhlig 2020-11-13 14:15:23 +01:00
  • 7362da8c5f Finished unfinished nDPIsrvd.h helper functionality. Toni Uhlig 2020-11-13 14:02:39 +01:00
  • 8c81c7c8db Fixed `make help' printout issues. Toni Uhlig 2020-11-11 17:32:02 +01:00
  • 41729212e5 More detailed `nanosleep() before close()' description. Toni Uhlig 2020-11-11 16:17:14 +01:00
  • fa25caf92d Improved default capture device handling if none set via `-i'. Toni Uhlig 2020-11-08 17:41:12 +01:00
  • ca28fd9a03 Make sure that DAEMON_EVENT_SHUTDOWN gets transmitted before close() (hacky, but works). Toni Uhlig 2020-11-08 15:13:43 +01:00
  • 54f6ab2192 forced --prefix for travis-ci Toni Uhlig 2020-11-04 11:46:55 +01:00
  • 97563faaa3 Added json-c to package dependencies required for libndpi unit tests. Toni Uhlig 2020-10-28 13:42:50 +01:00
  • 279868f4c8 build go example only if GOCC set Toni Uhlig 2020-10-17 12:50:24 +02:00
  • 8cbac2ac88 cosmetics, exception verbosity Toni Uhlig 2020-10-17 12:42:26 +02:00
  • ec9dc251d2 py-flow-info: Print BasicEvents. Toni Uhlig 2020-10-12 20:02:20 +02:00
  • 3cfb4d9c73 nDPId/nDPIsrvd: change_user_group does now chown/chmod collector/distributor socket paths Toni Uhlig 2020-10-07 14:38:10 +02:00
  • 4dd4537524 Travi CI Toni Uhlig 2020-10-06 17:39:40 +02:00
  • f3fd51a74f nDPId: Fixed BUG if flow was guessed but "Unknown" detected protocol serialized. Toni Uhlig 2020-10-01 19:16:21 +02:00
  • 9ea078b05e Improved python event validation with focus on readability. Toni Uhlig 2020-09-30 20:35:28 +02:00
  • 2f7a052f43 Centralized some python code parts to make future modifications easier. Toni Uhlig 2020-09-29 18:37:24 +02:00
  • 1a7342e798 nDPIsrvd: Added very important warning and use UDS now by default. Toni Uhlig 2020-09-29 17:48:24 +02:00
  • 9e4e41e5f6 Updated README.md ASCII architecture. Toni Uhlig 2020-09-29 17:43:36 +02:00
  • f60e7a097d nDPIsrvd: Improved UDS related code parts. Toni Uhlig 2020-09-29 17:22:29 +02:00
  • a2bf721722 Added additional datalink header check for DLT_NULL/IPv6. Toni Uhlig 2020-09-28 21:13:23 +02:00
  • ab10561dcd Added alias support for an nDPId instance. Toni Uhlig 2020-09-27 00:44:52 +02:00
  • 458b77c3eb Added daemon events. Toni Uhlig 2020-09-27 00:15:37 +02:00
  • 04bbbcd055 Improved event parsing for Python scripts. Toni Uhlig 2020-09-26 15:15:15 +02:00
  • 8c61f88362 Added event validation functions and fixed Python scripts. Toni Uhlig 2020-09-26 11:19:21 +02:00
  • b6b62ac883 Use --squash for git subtree pull. Toni Uhlig 2020-09-26 00:51:59 +02:00
  • 84712686a7 Centralized EventName validation and moved code parts. Toni Uhlig 2020-09-26 00:48:24 +02:00
  • adce2272dc Removed DISABLE_JSMN define for c-json-stdout (built-in per default). Toni Uhlig 2020-09-26 00:44:29 +02:00
  • d3bb1ccf33 Added shell script to update/pull JSMN. Toni Uhlig 2020-09-26 00:43:15 +02:00
  • 9abd1d8cf9 Merge commit 'e8af059ab3deb2a49e75c20ddcaf14ee19bcc223' as 'contrib/jsmn' Toni Uhlig 2020-09-26 00:49:15 +02:00
  • e8af059ab3 Squashed 'contrib/jsmn/' content from commit 053d3cd Toni Uhlig 2020-09-26 00:49:15 +02:00
  • a3ed1af871 Removed examples/c-json-stdout/jsmn as it's new location will be contrib/jsmn, soon. Toni Uhlig 2020-09-26 00:39:51 +02:00
  • 5b9965ce63 Added host_server_name length to hash to send a detection update if length changed (hacky). Toni Uhlig 2020-09-26 00:35:37 +02:00
  • 6c0ac8b045 Added new flow event: FLOW_EVENT_DETECTION_UPDATE Toni Uhlig 2020-09-25 20:03:14 +02:00
  • 370ca7c00d Process extra packets with nDPI, still not perfect but results in a more accurate detection. Toni Uhlig 2020-09-24 23:38:53 +02:00
  • ec1ffb6723 nDPId: Change user/group after init. Toni Uhlig 2020-09-21 19:34:17 +02:00
  • 5e5c1a4d20 nDPIsrvd: Use of an anonymous enum for collector/distributor socket types fits best. Toni Uhlig 2020-09-07 21:05:47 +02:00
  • ab374d193f nDPIsrvd: Change user/group, allow listening on UNIX socket for incoming distributor connections Toni Uhlig 2020-09-05 17:52:34 +02:00
  • ec101fbc0f go-dashboard: Print unmarshalled JSON string in a textbox. Toni Uhlig 2020-09-03 20:34:57 +02:00
  • 001f84af3b go-dashboard: Added event structs and JSON unmarshal semantic. Toni Uhlig 2020-09-02 18:53:39 +02:00
  • 0a03293d16 go-dashboard: go mod/vendor support + termdash text user interface Toni Uhlig 2020-09-01 23:44:09 +02:00
  • 70febd225b nDPId: Print compiled-in libgcrypt version (if libndpi was compiled with libgcrypt support). Toni Uhlig 2020-08-31 16:12:13 +02:00
  • 9c3b5170fc Added golang JSON deserializer example. Toni Uhlig 2020-08-31 08:45:36 +02:00
  • 905d84506e Increased network buffer size from 8448 to 9216 with the hope that it might be finally enough. Toni Uhlig 2020-08-27 15:40:43 +02:00
  • 93fa7fcabf nDPId: Improved command line option parsing, app usage and subopts for (carefully) tuning some daemon options. Toni Uhlig 2020-08-26 17:24:12 +02:00
  • 1653137edb Makefile: Allow pkg-config based builds and also allow setting libndpi build options manually. Toni Uhlig 2020-08-24 18:16:43 +02:00
  • efcdb81350 risky-flow-to-pcap.py: dump the first few packets of a "risky" flow to a PCAP file Toni Uhlig 2020-08-23 13:39:04 +02:00
  • bae579e926 nDPId: subopt parsing skeleton Toni Uhlig 2020-08-19 20:16:11 +02:00
  • 5830beb70f nDPIsrvd.py: improved PCAP writing for guessed/undetected flows (ignore empty UDP/TCP packets) Toni Uhlig 2020-08-19 19:54:49 +02:00
  • 4c9e099d0f added pkt_type and pkt_ipoffset to json serialization Toni Uhlig 2020-08-19 19:52:38 +02:00