webDAV and WireShark

[QFang]

Hi all,

I noticed "NI_WebDAV.lvlib:Direcotry Listing.vi" takes ~45 seconds to return with the remote server listing, while Internet Explorer completes the same task in 1.3 seconds.

I ran a wireshark with capture filter "host <development laptop ip> and host <webdav server IP>.

When capturing the Internet Explorer traffic, I get 1821 packets and the wireshark capture file is ~2.7MB large.

When I capture the LabVIEW "Directory Listing" traffic (using a breakpoint immidiately before the VI so I can start the capture after the WebDAV session is created), I get 9316 packets and the capture file is 14MB large.

Curios about two things:

1) Why is the labview function trashing the network requesting so much data to obtain more or less the same information (maybe a "Basic Directory Listing.VI" should be created that returns the same information as Internet Explorer, that is: list of all files and folders, date and time of files, size of files(, name of files) and nothing else?

2) In BOTH cases, I cannot for the life of me find ANY packets going FROM laptop TO the Server?!  -I even expanded by capturing ALL traffic from ALL interfaces in promiscuous mode, starting the VI after the capture was running, waiting until the VI completed (open webdav session, request directory listing <...43 seconds...> close webdav connection, then using display filters to show me all traffic with eth.dest == <MAC address of webdav server> and I still see no packets TO the server?!?!?   So HOW are sessions/connections to a webDAV server handled and how are requests/commands sent to the server, and how can I capture that traffic!?

[QFang]

I should mention that the webDAV server, being a prototype internal network only server, is setup as an http server with no encryption of any kind.

[Miro_T]

Hello

It looks like you are already working with one of our engineers on this issue. I spoken with her and she will be your main point of contact. 

Thank you 

[QFang]

Hi Miro, yes and no 🙂

This thread is trying to address a side-question to the main issue that I am working with NI Support.. (The main issue is how Internet Explorer manages to do essentially a "folder list" on a remote webdav server in less than 2 seconds while the "Directory Listing.vi" takes over 40 seconds to do the same.  While trying to figure out the differences, I did Wireshark traces, and that brings me to the topic of this thread:

No matter how I configure WireShark, I seem unable to capture any of the "commands" being sent TO the webDAV server.  This seems mighty strange to me, as the log-on and various request "commands" must also travel over Ethernet in some way shape or form.

[QFang · Accepted solution]

To answer my own question:

Several types of "over the network" communication exists at a very low level deep in the OS kernel. Such (outgoing) packets are not visible to software running on the same computer.  This is why wireshark cannot see these packets.

The solution is to get something like the SharkTap network sniffer for ~$70, OR use an old school HUB (with absolutley no switching abilities, so in other words pretty much impossible to find these days as weven products labeled as Hubs are actually primitive switches), OR an expensive managed switch that you can program to do various types of forwards to a "service port" on the switch.

I ended up using the SharkTap.  So now, I can do a "man in the middle" WireShark capture using my laptop off of the "center port" on the SharkTap, with another device (PC, cRIO, etc) and the network/server on the other two ports and voila, I can now see WebDAV log on, WebDAV requests and other low-level (windows remote desktop, windows network fileshares etc.) packets going between my "client" and my "server'.

As far as capture filters and tricks I have found useful:

You can use boolean operators such as AND and OR along with (parentheses) to construct elaborate filters, however, I found I mostly just need a couple of and's for what I do anyway.  I typically just filter by "destination" MAC's using the sender and receiver MAC addresses.  This cuts out all the random ARP's and other broadcasts that I'm (typically) not concerned with, while ensuring I get EVERYTHING else.. If that is too much, I add in protocol or port filters to further cut down on messages.

This capture filter configures to capture only packets going to this MAC OR that MAC:  

ether dst XX:XX:XX:XX:XX:XX or ether dst XX:XX:XX:XX:XX:XX   (you could use ether src for source, or ether host for variations of this filter).

keywords

webdav wireshark