14 Commits

Author SHA1 Message Date
Toni Uhlig
2b48eb0514 Added vlan_id dissection of the most outer (first) 802.1Q header. Fixes #50
Signed-off-by: Toni Uhlig <matzeton@googlemail.com>
2024-11-02 15:48:45 +01:00
Toni Uhlig
7b2cd268bf Updated JSON schema files and a test to make use of the UUID feature.
Signed-off-by: Toni Uhlig <matzeton@googlemail.com>
2024-10-29 15:25:19 +01:00
Toni Uhlig
99accd03a2 Moved datalink json key/value from error to packet events and renamed it to pkt_datalink.
Signed-off-by: Toni Uhlig <matzeton@googlemail.com>
2023-04-21 16:48:40 +02:00
Toni Uhlig
ce567ae5b7 Improved the point of time when to append the raw packet base64 data to the serializer.
* nDPId-test: Increased the max-packets-per-flow-to-send from 3 to 5.
   This is quite useful for TCP as the first 3 packets are usually part of the three-way-handshake.

Signed-off-by: Toni Uhlig <matzeton@googlemail.com>
2022-11-15 06:25:16 +01:00
Toni Uhlig
36e428fc89 Sync unit tests.
Signed-off-by: Toni Uhlig <matzeton@googlemail.com>
2022-11-13 16:19:07 +01:00
Toni Uhlig
d4633c1192 New flow event: 'analysis'.
* The goal was to provide a separate event for extracted feature that are not required
   and only useful for a few (e.g. someone who wants do ML).
 * Increased network buffer size to 32kB (8192 * 4).
 * Switched timestamp precision from ms to us for *ALL* timestamps.

Signed-off-by: Toni Uhlig <matzeton@googlemail.com>
2022-09-13 22:05:08 +02:00
Toni Uhlig
9f9e881b3f bump libnDPI to bb12837ca75efc2691ecb18fd5f56e2d097ef26b
Signed-off-by: Toni Uhlig <matzeton@googlemail.com>
2022-03-24 02:16:33 +01:00
Toni Uhlig
ed1647b944 Disconnect nDPIsrvd clients immediately instead waiting for a failed write().
* nDPIsrvd: Collector/Distributor logging improved
 * nDPIsrvd: Command line option for max remote descriptors
 * nDPId: Stop spamming nDPIsrvd Collector with the same events over and over again
 * nDPId: Refactored some variable names and events

Signed-off-by: Toni Uhlig <matzeton@googlemail.com>
2022-03-13 02:28:10 +01:00
Toni Uhlig
46f68501d5 Added daemon event: DAEMON_EVENT_STATUS (periodically send's daemon statistics.)
* Improved distributor timeout handling (per-thread).
 * flow-info.py / flow-dash.py: Distinguish between flow risk severities.
 * nDPId: Skip tag switch datalink packet dissection / processing.
 * nDPId: Fixed incorrect value for current active flows.
 * Improved JSON schema's.

Signed-off-by: Toni Uhlig <matzeton@googlemail.com>
2022-03-06 17:38:05 +01:00
Toni Uhlig
1a0d7ddbfa Process additional layer 3 protocols.
* bump libnDPI to c53c82d4823b5a8f856d1375155ac5112b68e8af
 * run_tests.sh: improved execution from non-git directories e.g. via `make dist`
 * updated JSON schema to be more restrictive
 * nDPId: splitted generic get_ip_from_sockaddr into IPv4/IPv6 to prevent compiler warnings on some platforms

Signed-off-by: Toni Uhlig <matzeton@googlemail.com>
2022-01-31 20:54:02 +01:00
Toni Uhlig
9e07a57566 Major nDPId extension. Sorry for the huge commit.
- nDPId: fixed invalid IP4/IP6 tuple compare
 - nDPIsrvd: fixed caching issue (finally)
 - added tiny c example (can be used to check flow manager sanity)
 - c-captured: use flow_last_seen timestamp from `struct nDPIsrvd_flow`
 - README.md update: added example JSON sequence
 - nDPId: added new flow event `update` necessary for correct
   timeout handling (and other future use-cases)
 - nDPIsrvd.h and nDPIsrvd.py: switched to an instance
   (consists of an alias/source tuple) based flow manager
 - every flow related event **must** now serialize `alias`, `source`,
   `flow_id`, `flow_last_seen` and `flow_idle_time` to make the timeout
   handling and verification process work correctly
 - nDPIsrvd.h: ability to profile any dynamic memory (de-)allocation
 - nDPIsrvd.py: removed PcapPacket class (unused)
 - py-flow-dashboard and py-flow-multiprocess: fixed race condition
 - py-flow-info: print statusbar with probably useful information
 - nDPId/nDPIsrvd.h: switched from packet-flow only timestamps (`pkt_*sec`)
   to a generic flow event timestamp `ts_msec`
 - nDPId-test: added additional checks
 - nDPId: increased ICMP flow timeout
 - nDPId: using event based i/o if capturing packets from a device
 - nDPIsrvd: fixed memory leak on shutdown if remote descriptors
   were still connected

Signed-off-by: Toni Uhlig <matzeton@googlemail.com>
2022-01-20 00:50:38 +01:00
Toni Uhlig
315f90f982 Fixed invalid "flow_last_seen" timestamp for the first packet.
* After the first packet was processed, "flow_last_seen" was still 0.
   This behaviour is invalid as the first packet may contain l4 payload data e.g. for UDP
   and it also breaks nDPId json consistency "flow_first_seen" > 0, but "flow_last_seen" == 0.
 * JSON schema: set minimum timestamp value for Epoch timestamps to 24710 for flow_*_seen and
   1 for pcap packet ts. Those values are dependant on some manipulated pcap's in libnDPI/tests/pcap.

Signed-off-by: Toni Uhlig <matzeton@googlemail.com>
2021-10-08 11:31:58 +02:00
Toni Uhlig
0b5b177c14 Extended nDPIsrvd.h with address parsing.
* nDPId supports looading of custom nDPI protocol/category files
 * extended JSON schemas according to nDPI / nDPId JSON serializing
 * removed memory holes in nDPId
 * extended examples/c-captured

Signed-off-by: Toni Uhlig <matzeton@googlemail.com>
2021-02-23 14:46:47 +01:00
Toni Uhlig
a1805eb891 Added JSON schema files and a Python schema validator.
Signed-off-by: Toni Uhlig <matzeton@googlemail.com>
2021-02-17 14:00:10 +01:00