FTP transfer slows down when application is running

[richferrara]

I am developing a test application for a radar project using LabWindows 2015, and I'm having an FTP problem.

Our radar system has a gigabit Ethernet interface. My test set software runs a test that connects to the radar and commands it via Telnet to execute a test function. This function creates a file back on the test set PC's hard drive and writes 10 MB of data to it via FTP. When the test is done, the test set measures the FTP transfer rate to verify that it is 30 MB/sec or better. My application is also running a few other programs in the background that are unrelated to this particular test.

We have two test set PCs with identical operating systems and network hardware (Windows 7 Enterprise, Intel I210 gigabit Ethernet adapters, driver version 12.7.28.0 dated 5/13/2013). For some mysterious reason, the test is only passing on one of the PCs. On the other one, the test passes only if I shut down my software and all its background programs, then run the test manually by typing the commands into a window in TeraTerm. If I run it the normal way, the FTP transfer quits in the middle of the file.

I ran data captures on both PCs using Wireshark. On the faulty PC, the FTP transfers happen much more slowly when the test set software is running in the background, and there are fewer transfers completed for each acknowledge sent by the PC. The other PC appears to be unaffected whether the test set software is running or not.

Both PCs are running the FileZilla FTP server, but I've repeated the test using WFTPD32 with the same results.

Nobody here can explain what we're seeing. We're running in a classified lab, so we have a suspicion that there might be a security setting to blame, but we have no idea what.

Can anyone shed some light?

Faulty PC, program not running - test passes

Faulty PC, program running - test fails

[samazing]

How are you doing your tests to verify that the test is 30 MB/sec or better? What tests are you running when you say you do this manually vs through your program?

When you say you run it the normal way & the FTP quits mid file - does this happen every time with the non-working computer? Such that you are never transfering all of the data from your radar to your host?

Are both computers on the same network? Would it be possible to disable firewalls/antivirus for one (quick) run to see if that is a cause of the issue?

Could you post MAX Tech reports of the working and non working computer?

[richferrara]

How are you doing your tests to verify that the test is 30 MB/sec or better?

The test sequence works like this:

1. The LabWindows test set application ("PLATE") mounts the radar processor's IP address to a drive letter (W:) and copies a VxWorks executable containing the test function to this drive.  The PLATE also creates a folder on the host PC's hard drive to store the output of the test.

2. The PLATE launches a TeraTerm session, connects to the radar's IP address via TELNET, and executes a macro.  This macro commands the radar to load the VxWorks executable and run the test function.  The macro pauses for 10 seconds to allow the test function to complete, then exits.

3. The radar processor's test function opens an FTP connection back to the host PC, creates a file in the output folder, starts a clock, writes 10 MB of data to the file, and stops the clock.  Dividing the size of the data transferred by the elapsed time gives the average throughput.  This needs to be at least 30 MB/sec for the test to pass.

4. The PLATE reads the throughput figure and determines whether the test passed or failed.

What tests are you running when you say you do this manually vs through your program?

The way we're supposed to run the test is through the PLATE as described above.  This fails.

To run the test manually, I start the PLATE but only use it to turn the radar power on.  Then I exit the PLATE with the radar still powered, start the TeraTerm session from Windows, and type the commands into TeraTerm that the PLATE would normally send by running the macro.  So the radar runs the test function and prints out the throughput figure in the terminal window.  I just read the result.

When you say you run it the normal way & the FTP quits mid file - does this happen every time with the non-working computer? Such that you are never transfering all of the data from your radar to your host?

Yes.  The transfer aborts at a different point every time but it never completes.

Are both computers on the same network? Would it be possible to disable firewalls/antivirus for one (quick) run to see if that is a cause of the issue?

There is no network.  Each computer is a standalone PC connected only to its radar unit.  Windows Firewall is disabled on both PCs.

Could you post MAX Tech reports of the working and non working computer?

What's MAX Tech?

[samazing]

NI MAX is a piece of NI software that is used to manage other NI software and hardware. A MAX tech report gives us a comprehensive list of software and hardware on your computer as well as other computer specs. If you do not have NI MAX on your computer you can install it by installing a light package called NI System Configuration. Once in MAX, go to File»Create Report and then selecting 'Technical support report' . You will want to generate reports from both computers.

[richferrara]

Okay, here are the reports.  I ran the test on two more of our PLATE systems.  The test passes on #1 and 5 and fails on #3 and 4.

[richferrara]

one more report

[TommyDunkz]

Hi richferrara,

You've mentioned that each station is a standalone computer connected to its radar unit. I am wondering whether it's possible to isolate the radar units from the equation. Are you able to swap the radar units between computers to troubleshoot?

Also, am I correct in my interpretation that the radar units are running VxWorks? Do you know that they are all running the same version of VxWorks?

[richferrara]

We've tried using different radar units.  It's always the same host PCs that fail.  The radar units are all running VxWorks 6.3.

[TommyDunkz]

Hi richferrara,

There must be some part of the system that is different for PLATEs 3 and 4.

I took a look at the MAX tech reports, and saw that the report for PLATE 4 indicates that there is no hardware connected, and that that computer has a processor which is different from the other machines. However, I don't think either of those details is indicative of the problem we are dealing with--especially since the PLATE 3 system shows hardware and software identical to that of PLATE 1 and 5.

Overall, the hardware and software between your systems is remarkably similar. Did you use a common OS image on all of these computers? Is it feasible to copy an image from one of the 'working' computers to one of the 'non-working' computers? We might not want to jump to this as a troubleshooting step yet, but it could be something worth doing further down the line if we aren't able to find anything else.

Also, I've been wondering about the cables you are using. Each computer is connected directly to its radar unit via ethernet, correct? Are these ethernet cables crossover or straight-through? Have you tried swapping the cables between systems?