usnavalresearchlaboratory / mgen Goto Github PK

Hi!

I am using mgen to generate traffic within an EMANE+ETCE environment. I want all of the traffic generated to be broadcast. I'd like to clarify my understanding on how to achieve this:

Currently I am using mgen scripts for all the nodes having "JOIN INTERFACE bmf0" and then sending and receiving to/from that address. This seems to be working great.

I found that EMANE's Virtual transport has broadcast mode though, so I was wondering if setting this parameter to true would make a difference to the above setup, or if broadcast from the Virtual Transport's point of view is different.

mgen also has a broadcast option (which the documentation says is on by default). My understanding is that this definition of broadcast just means that we are in a wireless setting where packets traveling across the medium will be sent out to all nodes but whether those packets will be received and processed by nodes depends on "if 1) MGEN is LISTENing on that port, and 2) either they are destined for that node via unicast, or the node has JOINed the multicast" #19

Please let me know if some of my understanding is erroneous or if you have any suggestions or insights.

Thank you so much for your help!

Hi,
Would you please help me in sorting out the following error. I was trying to install these code in Kali Linux. Thanks.

$ make -f Makefile.linux
g++ -c -g -DPROTO_DEBUG -DHAVE_GPS -DUNIX -Wall -Wcast-align -fPIC -DLINUX -DHAVE_SCHED -DHAVE_GETLOGIN -DHAVE_LOCKF -DHAVE_DIRFD -DHAVE_ASSERT -D_FILE_OFFSET_BITS=64 -DUNIX -DUSE_SELECT -DUSE_TIMERFD -DHAVE_PSELECT -DHAVE_IPV6 -DHAVE_PCAP -I/usr/X11R6/include -I../makefiles -I../include -I../protolib/include -o ../src/common/mgenApp.o ../src/common/mgenApp.cpp
In file included from ../include/mgenEvent.h:5,
from ../include/mgenFlow.h:4,
from ../include/mgen.h:7,
from ../src/common/mgenApp.cpp:2:
../include/mgenPattern.h:7:10: fatal error: pcap.h: No such file or directory
7 | #include <pcap.h> // for CLONE tcpdump files
| ^~~~~~~~
compilation terminated.
make: *** [Makefile.common:30: ../src/common/mgenApp.o] Error 1

Hello,

I am facing a compilation issue with various releases of mgen on at least recent debian & ubuntu platforms:

git clone https://github.com/USNavalResearchLaboratory/mgen.git
cd mgen
git clone https://github.com/USNavalResearchLaboratory/protolib.git
cd makefiles
make -f Makefile.linux

which raises the following error:

In file included from /usr/include/pcap/pcap.h:87,
                 from /usr/include/pcap.h:43,
                 from ../include/mgenPattern.h:7,
                 from ../include/mgenEvent.h:5,
                 from ../src/common/mgenEvent.cpp:2:
../protolib/include/protoDispatcher.h:468:37: error: expected identifier before ‘int’
  468 |                 enum Type {GENERIC, SOCKET, CHANNEL, TIMER, EVENT};
      |                                     ^~~~~~
../protolib/include/protoDispatcher.h:468:37: error: expected ‘}’ before ‘int’
In file included from ../protolib/include/protokit.h:24,

This is due to the following line in pcap/socket.h:

#define SOCKET int

Simply renaming the enum variable solves the issue. Some earlier versions of mgen (such as the one shipped in debian 5.02b) do not present the problem, likely because of a different include pattern in protolib.

Thanks

Hi,

It seems flow termination event is not properly raised at the client with v5.1.1 (from master), while this works with v5.02.c (from tag).

With v5.02.c

Server

$ mgen event "listen tcp 5200"
[...]
mgen: version 5.02c
[...]
09:16:59.967811 LISTEN proto>TCP port>5200
09:17:04.045735 ACCEPT src>127.0.0.1/5100 dstPort>5200
09:17:04.046713 RECV proto>TCP flow>1 seq>0 src>127.0.0.1/5100 dst>127.0.0.1/5200 sent>09:17:04.045711 size>65535 gps>INVALID,999.000000,999.000000,4294966297 flags>0x01
[...]
09:17:05.046231 OFF src>127.0.0.1/5100 dstPort>5200 

Client

$ mgen event "ON 1 TCP DST 127.0.0.1/5200 SRC 5100 PERIODIC [1 1000000.0] COUNT 1"
[...]
mgen: version 5.02c
[...]
08:58:09.981302 START Mgen Version 5.02c
enter ProtoSocket::Connect() ...
08:58:09.981461 ON flow>1 srcPort>5100 dst>127.0.0.1/5200 
08:58:09.981478 CONNECT flow>1 srcPort>5100 dst>127.0.0.1/5200 
08:58:10.981676 SHUTDOWN flow>1 srcPort>5100 dst>127.0.0.1/5200 
08:58:10.981922 OFF flow>1 srcPort>5100 dst>127.0.0.1/5200 
08:58:10.982008 STOP

With v5.1.1 (master)

Server

$ mgen event "listen tcp 5200"
[...]
mgen: version 5.1.1
[...]
09:14:47.577689 START Mgen Version 5.1.1
09:14:47.577975 LISTEN proto>TCP port>5200
09:14:51.282399 ACCEPT src>127.0.0.1/5100 dstPort>5200
09:14:51.283418 RECV proto>TCP flow>1 seq>0 src>127.0.0.1/5100 dst>127.0.0.1/5200 sent>09:14:51.282379 size>65535 gps>INVALID,999.000000,999.000000,4294966297 flags>0x01 
[...]

(last message corresponds to the final chunk)

Client

$ mgen event "ON 1 TCP DST 127.0.0.1/5200 SRC 5100 PERIODIC [1 1000000.0] COUNT 1"
[...]
mgen: version 5.1.1
[...]
09:42:57.449319 START Mgen Version 5.1.1
enter ProtoSocket::Connect() ...
09:42:57.449463 ON flow>1 srcPort>5100 dst>127.0.0.1/5200 
09:42:57.449483 CONNECT flow>1 srcPort>5100 dst>127.0.0.1/5200

(hangs here)

Does ubuntu16.04 support MGEN？

I am running MGEN 5.02b on Ubuntu,
I see that 5.02C is the newest version. I have tried updating using apt, but it does not update?

Hi,

I am use mgen to create traffic flow for three EMANE nodes, when I use the default configuration, I find all of three nodes receive data from mgen, can you please tell me how I can just let one node receive data from mgen and the other two doesn't?

Best,
Shaoyi

I can't compile mgen on gentoo 17.1 kernel 4.19.72 gcc 9.2.0
Too much errors.

Hello,

I'm using MGEN to measure flow completion time of TCP flows of various flow sizes in a congested network. I've noticed that for any flow size, the delay between a SEND event and SHUTDOWN will always be greater or around 1s. Even if flow size is really small (< 1MTU for instance).

Client-side:

olivier@oroques-dev:~$ mgen event "ON 1 TCP DST localhost/5000 PERIODIC [1 1000] COUNT 1,OFF" txlog 2> /dev/null
14:45:22.556771 START Mgen Version 5.1.1
14:45:22.557019 ON flow>1 srcPort>34853 dst>127.0.0.1/5000
14:45:22.557066 CONNECT flow>1 srcPort>34853 dst>127.0.0.1/5000
14:45:22.557090 SEND proto>TCP flow>1 seq>0 srcPort>34853 dst>127.0.0.1/5000 size>1000
14:45:23.557127 SHUTDOWN flow>1 srcPort>34853 dst>127.0.0.1/5000  # ~1s later, even though data has been received long before that
14:45:23.557280 OFF flow>1 srcPort>34853 dst>127.0.0.1/5000
14:45:23.557338 STOP

Server-side:

olivier@oroques-dev:~$ mgen event "LISTEN TCP 5000" 2> /dev/null
14:45:20.579566 START Mgen Version 5.1.1
14:45:20.579641 LISTEN proto>TCP port>5000
14:45:22.557066 ACCEPT src>127.0.0.1/34853 dstPort>5000
14:45:22.557144 RECV proto>TCP flow>1 seq>0 src>127.0.0.1/34853 dst>127.0.0.1/5000 sent>14:45:22.557090 size>1000 gps>INVALID,999.000000,999.000000,4294966297
14:45:23.557247 OFF src>127.0.0.1/34853 dstPort>5000

Where does that delay come from? Is there a way to disable it so that we can have a reliable FCT for small flows ? Thanks.

Hi,

I'm revising the MGEN source code to let MGEN produce a traffic flow whose TOS value in IP header is different from the default code. I plan to change the 8th bit in type-of-service (TOS), which is a reserved bit, from 0 to 1. I try to use the command TOS 0x01 in the command line to change it, but it doesn't work. So I revise the c++ source code to change the default value of TOS. I change the the initialized value of the TOS in MGEN class constructor (line 56 in mgen.cpp) from 0 to 1 and set the default_tos_lock variable to be true.

default_broadcast(true), default_tos(1),
default_tos_lock(true), default_multicast_ttl_lock(false),

However, the TOS value in traffic flow is still 0 after the package building. Can you please give me some suggestions about how to change the default TOS value?

One more thing I am working on is to assign each packet a different data value in user payload but not fixed data for all the packets in a flow. Can you please tell me how I can change this part? Thanks!

Best,

Hi,

I use MGEN to generate traffic flow for EMANE, my MGEN version is 5.02. I set the DATA part in the packet payload part using DATA command, and I try to extract it in EMANE in MAC layer according to 9. MGEN Message Payload in the user manual, but they doesn't match. The way I extract it is following:

1. Substract the ethernet header, IPv4 header and UDP header in order and get the MGEN message payload.
2. Shift the pointer from the beginning of the message payload buffer address to the data payload part.

However, the data is not I set in MGEN, I am confused about the UDP IPv4 and Ethernet header format set in MGEN and the message payload format, is there any material I can check about this?

Thanks,
Shaoyi

Hello,

I was wondering if there is a quicker or more efficient way to write traffic pattern mgen files according to different traffic patterns. Currently, I have been hand-writing the commands in files for each node from scratch, but I find it to be error prone and time intensive for more complicated traffic patterns or networks with a lot of different nodes. Do you have any suggestions?

Thank you!

Hello,

When building older releases through tags, there is no easy way to know which version of protolib was used at the time of the release. Using submodules would simplify this task.

Cheers,
Jordan

Was working with DSCP / TOS marking found that both MGEN events are marked with the same TOS value.

MGEN COMMAND:
mgen event "0.0 ON 1 UDP DST 127.0.0.1/5111 PERIODIC [ 1 128 ] TOS 16" event "0.0 ON 2 UDP DST 127.0.0.1/5222 PERIODIC [ 1 128 ] TOS 8"

TCPDUMP on lo:
sudo tcpdump -lnnei lo -v port 5111 or port 5222

13:38:56.527050 00:00:00:00:00:00 > 00:00:00:00:00:00, ethertype IPv4 (0x0800), length 170: (tos 0x8, ttl 255, id 63041, offset 0, flags [DF], proto UDP (17), length 156)
127.0.0.1.43691 > 127.0.0.1.5111: UDP, length 128
13:38:56.527101 00:00:00:00:00:00 > 00:00:00:00:00:00, ethertype IPv4 (0x0800), length 170: (tos 0x8, ttl 255, id 63042, offset 0, flags [DF], proto UDP (17), length 156)
127.0.0.1.43691 > 127.0.0.1.5222: UDP, length 128

Hello,

Question on using TOS/DSCP values in MGEN in a EMANE+ETCE environment. I am also using OLSRD environment, where my config file has IpVersion set to 4. I am using IEEE 802.11abg radio model from EMANE and want to simulate traffic for video/voice.

My assumption was that if I changed the TOS value in the flow definitions in MGEN scripts, this would change the Class of the flow's traffic (e.g. TOS 64==0x40=>CS2, TOS 32==0x20=>CS1), which should map to all the different queues corresponding to the "[four different traffic classes](four different traffic classes)" in the IEEE 802.11abg radio model.

Below is my mgen.script for node-1 to test this:

TTL 64

0.0 LISTEN UDP 5001

0.0 JOIN 225.1.20.1 INTERFACE bmf0

1.0 ON 1 UDP SRC 5001 DST 225.1.20.1/5001 BURST [RANDOM 10.0 PERIODIC [1 1024] EXPONENTIAL 5.0] TOS 32 INTERFACE bmf0

10.0 ON 100 UDP SRC 5001 DST 225.1.20.1/5001 BURST [RANDOM 10.0 PERIODIC [1 1024] EXPONENTIAL 5.0] TOS 64 INTERFACE bmf0

20.0 ON 200 UDP SRC 5001 DST 225.1.20.1/5001 BURST [RANDOM 10.0 PERIODIC [1 1024] EXPONENTIAL 5.0] TOS 96 INTERFACE bmf0

900.0 OFF 1

900.0 OFF 100

900.0 OFF 200

900.0 LEAVE 225.1.20.1 INTERFACE bmf0

900.0 IGNORE UDP 5001

However, when I looked at the EMANE tables for node-1, I only saw packets on queues 0 and 3:

[emanesh (node-1:47000)] ## get table 1 mac
nem 1   mac BroadcastPacketAcceptTable0
| NEM | Num Pkts Tx | Num Bytes Tx | Num Pkts Rx | Num Bytes Rx |
| 1   | 1118        | 1241047      | 0           | 0            |
| 2   | 0           | 0            | 1527        | 435138       |
| 3   | 0           | 0            | 1523        | 434606       |
| 4   | 0           | 0            | 1525        | 434930       |
| 5   | 0           | 0            | 1522        | 434420       |
| 6   | 0           | 0            | 1523        | 434826       |
| 7   | 0           | 0            | 1519        | 433450       |
| 8   | 0           | 0            | 1521        | 433710       |
| 9   | 0           | 0            | 3           | 210          |
| 10  | 0           | 0            | 1524        | 434432       |
| 11  | 0           | 0            | 1522        | 434208       |
| 12  | 0           | 0            | 1525        | 435014       |
| 13  | 0           | 0            | 1524        | 434732       |
| 14  | 0           | 0            | 1520        | 434164       |
| 15  | 0           | 0            | 1525        | 435258       |
| 16  | 0           | 0            | 1525        | 434506       |
| 17  | 0           | 0            | 1528        | 435568       |
| 18  | 0           | 0            | 1524        | 434692       |
| 19  | 0           | 0            | 1526        | 435440       |
| 20  | 0           | 0            | 1528        | 435336       |

nem 1   mac BroadcastPacketAcceptTable1
| NEM | Num Pkts Tx | Num Bytes Tx | Num Pkts Rx | Num Bytes Rx |

nem 1   mac BroadcastPacketAcceptTable2
| NEM | Num Pkts Tx | Num Bytes Tx | Num Pkts Rx | Num Bytes Rx |

nem 1   mac BroadcastPacketAcceptTable3
| NEM | Num Pkts Tx | Num Bytes Tx | Num Pkts Rx | Num Bytes Rx |
| 1   | 871         | 215975       | 0           | 0            |

Is there something in my understanding I'm missing? I have seen issue #28 and my understanding is that I want to change the DSCP value, not the ECN value; thus I calculated some good TOS values to be 32, 64, and 96.

Please let me know if this question is more suited for the EMANE repo.

Thank you so much!

Hello,

We observe an issue with the master branch (which interests us because of analytics functionalities) when generating large TCP messages through the following :

Client:

$ cat client.conf 
0.0 ON 1 TCP DST 192.168.2.2/5000 PERIODIC [1 10000000] COUNT 1
$ mgen input client.conf txlog debug 99

Server:

$ cat server.conf 
0.0 LISTEN TCP 5000
$ mgen input server.conf output server.drc 

The output is:

$ mgen input client.conf txlog debug 99
[...]
enter ProtoSocket::Connect() ...
06:46:40.792108 ON flow>1 srcPort>38151 dst>192.168.2.2/5000 
06:46:40.804052 CONNECT flow>1 srcPort>38151 dst>192.168.2.2/5000 
Proto Debug: MgenFlow::SendMessage() error sending message flow>1 seq>0.
06:46:41.792286 SEND proto>TCP flow>1 seq>0 srcPort>38151 dst>192.168.2.2/5000 size>10000000 
Proto Debug: MgenFlow::SendMessage() error sending message flow>1 seq>0.
06:46:42.140386 SEND proto>TCP flow>1 seq>0 srcPort>38151 dst>192.168.2.2/5000 size>10000000 
[...]

and the pattern repeats indefinitely.

It seems the issue is related to MgenTcpTransport::SendMessage returning MSG_SEND_BLOCKED. It does not trigger with shorter messages (eg. with [1 1000000]), as the socket buffer likely absorbs the burst of traffic and the MgenTcpTransport::SendMessage() functions returns MSG_SEND_OK, nor when testing version 5.02.c.

Thanks

I am following the instructions in the read me to build mgen and coming up with the following error:

[root makefiles]# make -f Makefile.linux 
g++ -c -g -DPROTO_DEBUG -DHAVE_GPS -DUNIX -Wall -Wcast-align -fPIC -DLINUX -DHAVE_SCHED -DHAVE_GETLOGIN -DHAVE_LOCKF -DHAVE_DIRFD -DHAVE_ASSERT  -D_FILE_OFFSET_BITS=64 -DUNIX -DUSE_SELECT -DUSE_TIMERFD -DHAVE_PSELECT -DHAVE_IPV6 -DHAVE_PCAP -I/usr/X11R6/include  -I../makefiles -I../include -I../protolib/include -o ../src/common/mgenApp.o ../src/common/mgenApp.cpp
In file included from ../include/mgenEvent.h:5:0,
                 from ../include/mgenFlow.h:4,
                 from ../include/mgen.h:7,
                 from ../src/common/mgenApp.cpp:2:
../include/mgenPattern.h:7:50: fatal error: pcap.h: No such file or directory
 #include <pcap.h>      // for CLONE tcpdump files
                                                  ^
compilation terminated.
make: *** [../src/common/mgenApp.o] Error 1

It seems like an expect pcap.h file is not there. I tried manually searching for it and came up dry, except for ezx-pcap.h in unrelated directories:

[root mgen-master]# find / -name "*pcap.h*"
/usr/src/kernels/3.10.0-1062.18.1.el7.x86_64/include/linux/mfd/ezx-pcap.h
/usr/src/kernels/3.10.0-1062.18.1.el7.x86_64.debug/include/linux/mfd/ezx-pcap.h

Now, I suspect since git is just taking the master branch that the instructions may want us pointing to a release version or some previous version. I tried a variety of them such as 5.02, 5.02c and protolib 3.0b1, etc. No combination worked (though the errors were a bit different each time).

My set up is Centos 7 and I'm experimenting doing this with Docker. That said, I tried the executable in the release .tgz file however that had no effect.

I attached my docker files and the command I use to build:
docker build --rm -t local/c7-systemd . -f Dockerfile-centos-systemd
docker build -f Dockerfile-centos . -t rmbn-test-centos
(Remove the .txt extension from the files, I needed to add to upload)

Dockerfile-centos-systemd.txt
Dockerfile-centos.txt

It'd be great if we can review the make process? And maybe, if possible, instead of manually making this, it could be added to the mgen release assets?

Suggest logfmt because of wide support in log aggregators and parsers.

Hi,
On my Ubuntu machine I have two network interfaces opened (let's call them X and Y). Is it possbile to specify outbound interface for UDP unicast traffic? In iperf3 we can force it using -B flag e.g.

iperf3 -c <server_addr> -p <server_port> -B <outbound_interface_addr>

Now, in mgen server logs we can read that X interface address is always source address of received packets (I want it to be Y address).

Hi,

I am run a demo with 3 nodes multi-hop network topology in EMANE, I use multicast address 225.1.20.1/5001 as destination in mgen. I have changed the path loss between node1 node2 and node 3 to achieve a topology to let node 1 transmit packet through relay node2 to node3 rather than transmit packet directly to node3. Because mgen sends packets to all of three nodes, but I want to just let mgen send packet to node 1 and node 1 deliver packets to node 3 through node 2. I don't know how to use a unicast address to let mgen send packet to node 1 firstly, and packets are sent to node 3 through node 2 next my topology. Can you give me some ideas?

The following figure may describe my topology clearly.

Thank you very much.

Best,
Shaoyi

Hi,

I have started two bulk flows:

0.0 ON 1 UDP SRC 5001 DST 10.1.1.5/5001 PERIODIC [300000 1024]
0.0 ON 2 UDP SRC 5002 DST 10.1.1.5/5001 POISSON [300000 1024]

The network has 25 Gbps capacity, so that is not a problem.
No issue when I used 1/3 number of packets.

I am getting the warning:
Proto Error: ProtoTimerMgr: Warning! real time failure interval:0.000003 (delta:-1.583321)

There is precision time protocol running in the background.

Can anyone please explain why it is happening, how to avoid it and potential impacts of the warning?

Thanks