FTP DATA not detected about ndpi

[lucaderi]

@alexhebra Alex please provide a pcap (full packets) containing both the FTP connection and data.

[alexhebra]

Hi Lucas,

Here is the link to download my pcap output: https://mega.nz/#!GUcEnBLA!2Wo3_1_YCoLQdv1RZJkcuy21CSqZahK1Tn9BRj6zxcg

As you can see i was trying to download ftp://ftp.debian.org/debian/dists/jessie/main/Contents-source.gz, about 42MB size file.

It wasn't detected as FTP data, but you can find it as Unknown protocol. Thanks.

[Skryabind]

@alexhebra , I have the same problem. As I see in sources, ftp-data protocol only detects if you download one of certain filetypes, defined in source code.
Problem of ftp-data is in passive mode: data and commands flows in different ports, thus nDPI have a problem to detect a data connection without information about command connection.
In my program I make a hack: if protocol of flow is not defined, I find established connection between this two hosts with port 21, and if its exists, then this undefined flow is probably ftp-data.

[alexhebra]

Hi @Skryabind @lucaderi,

Yes, i saw only some filetypes are detected. But even this kind of files have problems. I've tried it also and same issue persists.

But anyway i think you should provide your patch, sounds a smart approach to this issue. I don't know what @lucaderi thinks. Can you?

Thanks.

[Skryabind]

Sorry, in my case this is a part of my program, not a patch of libndpi. I don't know, how to change this issue in libndpi source :(
My code is like this:

if ( protocol of flow is unknown after 20 packets ) {
   if (flow between this hosts with 21 port exists ) {
      Set protocol of current flow to ftp-data  
   }
}

Also I use some optimization and save all flows with 21 port to separate search tree. This tree is small and thus this code works fast enough.

[lucaderi]

We have enhanced the FTP_DATA dissector as the @Skryabind suggestion (keep track of open port 20/21) is not feasible because nDPI does not keep intra-flow stats/dependencies. We have introduced a fix in 5af906a that we think is a good compromise. Please give it a try.

[alexhebra]

Hi @lucaderi ,

I've just tried with development ndpi version and didn't work. I don't know if @Skryabind has tested already. What seems strange is: it only worked with pcap file that i sent you. If i try to access any another FTP site and download anything it wasn't recognized. I used ndpiReader to try.

Let me know if i can help you.

Thanks.

[lucaderi]

@alexhebra What you want to do can be done at the application level, not inside nDPI that has a per-flow knowledge only. nDPI receives the packets only until a flow has been detected, then it has no possibility of implementing log like "flow between this hosts with 21 port exists". In essence this is on the app that uses nDPI that has knowledge of flows and that has to make that decision.