GigE Camera removed problem IMAQdx

[Fabriae]

Hello, I´m having some problems with my GigE cameras, particularly when I have 2 or 3 of them connected to the same switch. Sometimes when my software is idle, I get the error -1074360308 camera removed.

At first, I´ve tested my power supply because I thought it was that. Then I´ve changed cables, GigE switches and I get everytime the same result.

I also installed Wireshark to audit my connection.

After the error, the disconnected camera instantly get available in Nimax.

Cameras model is Basler acA1440

Switch: Cisco SG110D-05 / Weidmuller 2682210000 / Phoenix FLswitch 1105N

Ethernet driver: Intel i211/ Intel i219

Camera power supply: 12v dedicated for each

Trigger: external 

Has anyone ever had a problem like this?

I attached 2 wireshark fragments and my code

[Bob_Schor]

We've been running 24 Basler cameras (I don't remember the model) over TCP/IP from a room four floors away using I think a 100BaseT switch to a 5-6 year old Dell Xeon-based workstation.  We are able to run the cameras and acquire images (640 x 400, 10 fps, as I remember) for several hours without difficulty (of course, some "interesting" LabVIEW code).  We recently rewrote the routine to use a higher frame rate, but one of us (I'm blushing) forgot to compute the number of bits per second flowing through the EtherWire -- turns out at this frame rate, we max out the Ethernet at about 9-10 cameras (we need a Gigabit drop ...).

Don't know the particulars of your hardware.  Do the calculation I forgot to do -- estimate the bit rate you need for the images you are acquiring, and make sure your PC, your Cameras, and your Ethernet hardware have the bandwidth to handle it.

Bob Schor

[Fabriae]

Thanks for your reply Bob, i will add more info below

Bandwidth of each camera is limited to 300mb on Nimax (3*300mb = 900mb to fit the GigE comunication).

In my application I only acquire 1 image at a time (no video streaming).

Hardware: Intel Core i3 6100 with an industrial motherboard. Ram 8gb DDR4.

My main problem is when the cameras are doing nothing.

[Bob_Schor]

This is a very complex VI, whose logic I cannot follow.  You call Open Camera and Close Camera at least twice, you Configure Acquisition twice (with different Buffer parameters each time), you use a "naked" Event Structure with Get Image to get the Last Buffer (instead of a simple Get Image with the default "Next Buffer", which needs no Event Structure, and other mysteries.

My advice would be to do your best to simplify the loops.  You should be able to make a VI with about 3-5 IMAQdx calls to open the Camera, set up its parameters (don't try to fiddle with all of them), and acquire (and display) Images until you push the Stop button.  Use the Error Line with all of your IMAQdx functions.  This VI should have no Case Structure, only a single While Loop.  If this works (and it should), think carefully what you need to add to it, but Keep It Simple.  If it doesn't, come back here and we'll take another look.

Bob Schor

[hyeong1905]

Hello,

I am suffering with exact same problem.

Did you solve it?

[MrJackHamilton]

We've seen this problem with GigE. There are a couple of factors.

1. Cameras now have larger image formats - so they are pushing more data thru the network.

2. The GigE implementation IMO was not exactly fully implemented. As certain features don't work.. such as 'Listener' mode - it simply does not work. [We have even confirmed this with Basler and Basler Pylon Code - it is not a 'LabVIEW' issue - that part of the standard is not really functional.

3. Higher resolution cameras - readily available [cause more pixels are what we all want!] now are consuming and taxing the bandwidth of your 1Gig network hubs, computer ports and switches.

Solution [That we had to implement]

A. We replaced all hubs - by throwing them away! - they do not manage or buffer traffic, and when taxed they don't stream data reliably.

B. Goto Enterprise 10Gig Managed switches - *only* Seriously - we have a pile of '10gig' switches that don't work.

C. We found that the Fiber linked 10gig enterprise switches worked robustly. Get the higher tier as you can afford.

D. Get CAT6+ cables - the real ones - with the metal shield! - don't, do not, get the cheap ones. The entire system will slow down to your worse cable.

E. Stay off your corporate network - it is busy doing other things.

F. Stay off your corporate network - the corporate network guys really don't know much about how networks actually work.

G. Get a proper 10Gig add-on interface in your computers - then turn ON Jumbo packets once you install them.

H. If you are using a USB-Ethernet adapter...please leave the room- now...If that USB adapter is plugged into a USB hub to your Laptop - run from the room now..

I. The cool thin [cheap] laptops your work gave you - don't have the Ethernet chipset to support proper 1Gig much less 10Gig jumbo packets. Lenovo's are below the bottom of the barrel. You need a hideously good memory buffer on the ethernet receiving end to stream image data.

J. The cool thin [cheap] laptops your work gave you - Have anti-virus and Malware protection and scanning the network traffic. It is like your mother-in-law she won't go away...even if you ask politely...Go buy a clean blank lap computer.

We have 23 gigE cameras on a private 10Gig fiber network with 6+ Enterprise switches and it works - OK...just OK...it does not cure cancer, or allow impossible things like streaming all 23 at the same time.

We had a C & Python programmer take a crack at writing multi camera streaming code and they ran into the same issues as LV. They tried Basler Pylon - it does not do any better.

Slow the frame rate of the cameras - do you really need 120FPS. Lower the camera resolution.

Oddly, outside the science community not a lot of people connect 'N + 4' cameras on the same network - so we learned on our own. Product support is very limited.

Regards

Jack Hamilton

[nguyenquangsonnam@gmail.c]

Hi, i have the same problem like this. Do you have a solution? 

[rolfk]

---

nguyenquangsonnam@gmail.c wrote:

Hi, i have the same problem like this. Do you have a solution? 

---

Have you read the message before your inquiry? It sums up the things pretty much.

=> Go bold! Invest money! Or reduce the resolution and frame rate of your cameras CONSIDERABLY!

[MrJackHamilton]

Rolf,

Thanks...Even the easy things are hard....!

Warm Regards

Jack Hamilton

[nguyenquangsonnam@gmail.c]

Hi, 

firstly, thank you for the reply. Although it was written not in a nice way. 

Normally the topic owner must choose the reply that solves their Problem as "Solution", so the reader will be able to know if the problem is solved or not. I have not seen any confirmation here, which says your reply is the solution for this Problem, Jack. It can work in your case, does not mean it works for his/her case. 

However, I read and considered your reply as a suggestion.

About my system, we have 2 gigE Cameras. https://www.alliedvision.com/en/camera-selector/detail/mako/g-1242/

These two cameras are connected directly with Ethernet cable from Allied Vision for this camera, to the GigE Vision Adapter, which is attached on a PC. We dont use any hub or switch here. Directly from camera to PC Card

Here its information:  

https://cdn.alliedvision.com/fileadmin/content/documents/products/accessories/host-adapter/GigE_data...

We also set up the Jumbo Frames for this.

The PC has:

Processor: Core i5-10400 CPU @2.90GHz

16 GB RAM

The PC is strong enough for a Test System in Shopfloor and we have used it for years. So it's not a thin and cheap laptop.

My Problem is: 

I let the two camera as Free Run and collecting Live Images. I used the Example Reconnecting Function of LabVIEW, so it will re-establish the connection if the camera is disconnected from PC. 

I tried to plug it out and again in, and it works. That means the Re connecting Function is working. 

We are facing the Problem that after 2 or 3 days of running, the camera suddenly does not show any live image anymore but only black image. I saved this black image and the size is only 12 KB.

When I restart the LabVIEW Program, the correct live image is there again. 

I found this thread and read the suggestion from Jack but I dont find any unnormal things in our system till now. 

That's why I asked if the thread owner already found the solution.

So please, do not misunderstand why I asked this question and thank you for any suggestion.