Previous in Forum: What is the Name of the Opposite of a Rubber Grommet to Protect an OD Edge?   Next in Forum: How Much is Quiescent Current Iq of OP27 OP AMP?
Close
Close
Close
24 comments
Associate

Join Date: Feb 2011
Posts: 25
Good Answers: 2

Measuring Interference in Control System Cabinet

09/07/2024 12:11 PM

Hi,

We have a ESD system in one of our several plant locations that has reported Noise/ Interference several times in past two months in Safety Control System Digital Output modules and IO Bus (internal bus to connect and communicate Control System IO Racks). It also reported module faults which we replaced with new ones. When we checked these faulty modules in other less critical system spare slots, we found them working without problem after keeping them under observation of more than a week. In parallel, the Tech. support team of Safety System manufacturer was contacted and they have been very supportive. So far we have replaced several DO, AI modules, one AC/DC Power supply and one rack bus module.

Since, this ESD system is very critical for plant operation, it can't be shutdown for detailed investigation. We are suspecting interference is playing a major role here and causing modules capacitor to glitch and making them go to fault state. We are planning to execute some tests in Online system to find root cause and need your review and advise on the same.

i) Performing ground test on AC incomer and DC power supplies by using multimeter.

ii) Performing ground test to make sure all field instrument cables shield are connected to Instrument Earth plate on one side.

iii) Performing integrity/continuity test on field instruments shield cables by using jumper on Junction box side to short two shield cables and measuring on system side.

iv) Radio interference measurement using RF meter (need to check if Telecom team has the tool)

v) Static charge test?? (advise needed)

vi) Interference measurement using oscilloscope?? (advise needed)

vii) Any other test suggestion?

__________________
For 'tis the sport to have the engineer Hoist with his own petar; and't shall go hard But I will delve one yard below their mines And blow them at the moon. - William Shakespeare -
Register to Reply
Pathfinder Tags: EMI interference noise RF
Interested in this topic? By joining CR4 you can "subscribe" to
this discussion and receive notification when new comments are added.

Comments rated to be Good Answers:

These comments received enough positive ratings to make them "good answers".

Comments rated to be "almost" Good Answers:

Check out these comments that don't yet have enough votes to be "official" good answers and, if you agree with them, rate them!
Guru
Hobbies - CNC - New Member Hobbies - DIY Welding - New Member Engineering Fields - Electromechanical Engineering - New Member

Join Date: Aug 2007
Posts: 23517
Good Answers: 419
#1

Re: Measuring Interference in Control System Cabinet

09/07/2024 12:25 PM

Are your high voltage lines separated/shielded from your low voltage control lines?

__________________
“ When people get what they want, they are often surprised when they get what they deserve " - James Wood
Register to Reply
Associate

Join Date: Feb 2011
Posts: 25
Good Answers: 2
#2
In reply to #1

Re: Measuring Interference in Control System Cabinet

09/07/2024 12:47 PM

In ESD cabinet drawing, AC (240VAC) and DC (24VDC) lines are separate and perpendicular. AC GND line is connected to Protective Earth plate while DC GND line is connected to Instrument Earth plate. AC Supply is fed from UPS.

I will test with multimeter to confirm the resistance between IE and PE plates.

__________________
For 'tis the sport to have the engineer Hoist with his own petar; and't shall go hard But I will delve one yard below their mines And blow them at the moon. - William Shakespeare -
Register to Reply
Guru
United Kingdom - Member - Indeterminate Engineering Fields - Control Engineering - New Member

Join Date: Jan 2007
Location: In the bothy, 7 chains down the line from Dodman's Lane level crossing, in the nation formerly known as Great Britain. Kettle's on.
Posts: 32049
Good Answers: 838
#8
In reply to #2

Re: Measuring Interference in Control System Cabinet

09/10/2024 4:10 AM

To what is the <...Instrument Earth Plate...> connected, please?

__________________
"Did you get my e-mail?" - "The biggest problem in communication is the illusion that it has taken place" - George Bernard Shaw, 1856
Register to Reply
Associate

Join Date: Feb 2011
Posts: 25
Good Answers: 2
#12
In reply to #8

Re: Measuring Interference in Control System Cabinet

09/12/2024 3:23 PM

All DC Supplies -ve reference and field wiring shield cables are connected to Instrument Earth plate.

__________________
For 'tis the sport to have the engineer Hoist with his own petar; and't shall go hard But I will delve one yard below their mines And blow them at the moon. - William Shakespeare -
Register to Reply
Guru
United Kingdom - Member - Indeterminate Engineering Fields - Control Engineering - New Member

Join Date: Jan 2007
Location: In the bothy, 7 chains down the line from Dodman's Lane level crossing, in the nation formerly known as Great Britain. Kettle's on.
Posts: 32049
Good Answers: 838
#24
In reply to #2

Re: Measuring Interference in Control System Cabinet

09/23/2024 6:52 AM

<...test with multimeter to confirm the resistance between IE and PE plates....>

All this will do is confirm the soil resistance between them.

__________________
"Did you get my e-mail?" - "The biggest problem in communication is the illusion that it has taken place" - George Bernard Shaw, 1856
Register to Reply
Guru

Join Date: Mar 2007
Location: by the beach in Florida
Posts: 33233
Good Answers: 1807
#3

Re: Measuring Interference in Control System Cabinet

09/07/2024 10:38 PM

I would start by checking if any new equipment has been added in the last 2 months...then check for ground degradation on all grounding connections...maybe add some dielectric grease at these connections...Then I would check all electric connections on all equipment and if that equipment is operating at spec...being a critical application you should have in stock replacement modules to switch out known good modules with existing modules for possible replacement with new modules...

__________________
All living things seek to control their own destiny....this is the purpose of life
Register to Reply
Associate

Join Date: Feb 2011
Posts: 25
Good Answers: 2
#13
In reply to #3

Re: Measuring Interference in Control System Cabinet

09/12/2024 3:29 PM

Thanks for reply SolarEagle.

I confirmed with Field Instrumentation team that no modification was performed in the system. You are right about ground bonding degradation and that was one of the cause of module faults.

The team has cleaned the connections and applied the dielectric grease. In addition, they are also digging new earthing pit for both IE and PE.

The detailed the root cause finding I will wrote in new comment.

__________________
For 'tis the sport to have the engineer Hoist with his own petar; and't shall go hard But I will delve one yard below their mines And blow them at the moon. - William Shakespeare -
Register to Reply
Guru

Join Date: Jul 2016
Posts: 1246
Good Answers: 90
#4

Re: Measuring Interference in Control System Cabinet

09/08/2024 1:06 AM

A high speed power system analyzer such as Dranetz should show you if the problem is externally generated. On your ac supply line. You should be able to rent one, as they are quite expensive. Also, an experienced electrical engineer should supervise the installation and operation of the instrument. A survey of the loads, the environment, including the many details you have had to leave undescribed will be helpful in eliminating the source of the problem. Often, in cases like this, it turns out that the problem is the one item thought to be bulletproof, such as the grounding method of the I/O card chassis frames, or a corroded sub-grade earth connection that is not easy to access. Power factor capacitor switching 4 distribution layers away, that sort of thing.

Register to Reply Score 1 for Good Answer
2
Guru
Engineering Fields - Electrical Engineering - Been there, done that. Engineering Fields - Control Engineering - New Member

Join Date: Dec 2008
Location: Long Island NY
Posts: 15501
Good Answers: 957
#5

Re: Measuring Interference in Control System Cabinet

09/08/2024 4:36 PM

One thing many people forget is bad or loose cabling and connections. These bad signal paths rarely occur inside a chassis but... Wiggle some wires!

__________________
"Don't disturb my circles." translation of Archimedes last words
Register to Reply Good Answer (Score 2)
Guru
Hobbies - CNC - New Member Hobbies - DIY Welding - New Member Engineering Fields - Electromechanical Engineering - New Member

Join Date: Aug 2007
Posts: 23517
Good Answers: 419
#6
In reply to #5

Re: Measuring Interference in Control System Cabinet

09/08/2024 4:48 PM

I had something similar, had an electrical engineer wannabe, he was getting crapping inconsistent signals.

he wanted AB to send someone out from Texas to fix it. I looked at how he was powering it, he had an electrical cord with 3-4 splices on the electrical cord. It was about 5:00 pm and I had an appointment, I told him and one of the suppliers that was there.. ‘I’d look at getting a stable power supply first by getting a new electrical cord.’ And pointed to it.

I then left, next morning I got back in and this wannabe electrical engineer was bragging how late he stayed to get it running.

out of curiosity I asked what the problems was… well it turned out I was right..

__________________
“ When people get what they want, they are often surprised when they get what they deserve " - James Wood
Register to Reply
Guru

Join Date: Jul 2008
Posts: 1681
Good Answers: 144
#7

Re: Measuring Interference in Control System Cabinet

09/08/2024 10:04 PM

Some information about the "ESD System" might be helpful. Is it a single purchased system, a DYI system, a bunch of purchased items band-aided together? This is a bit like people telling me "facts" by saying "they said". My standard response is "who is/are they"?

__________________
Few things limit our potential as much as knowing answers and setting aside questions.
Register to Reply
Associate

Join Date: Feb 2011
Posts: 25
Good Answers: 2
#14
In reply to #7

Re: Measuring Interference in Control System Cabinet

09/12/2024 3:32 PM

It is a single purchased system by HIMA. It is in operation for almost 20 years.
We found the root cause of the issue that I will explain in new comment.

__________________
For 'tis the sport to have the engineer Hoist with his own petar; and't shall go hard But I will delve one yard below their mines And blow them at the moon. - William Shakespeare -
Register to Reply
Guru
Popular Science - Cosmology - Let's keep knowledge expanding Engineering Fields - Retired Engineers / Mentors - Hobbies - HAM Radio - New Member

Join Date: Dec 2006
Location: North America, Earth
Posts: 4484
Good Answers: 106
#9

Re: Measuring Interference in Control System Cabinet

09/10/2024 9:50 AM

I have 2 suggestions:

(1) Check all AC power connections in breaker box (ground, neutral, & hot).

(2) Get a spectrum analyzer with a loop of wire as an antenna, and move the loop close to all equipment one at a time to see what is radiating.

__________________
“I would rather have questions that can't be answered than answers that can't be questioned.” - Richard Feynman
Register to Reply Score 1 for Good Answer
Guru

Join Date: May 2018
Location: Under the spreading Bunya Trees, South Burnett, Queensland, Australia
Posts: 759
Good Answers: 64
#10

Re: Measuring Interference in Control System Cabinet

09/10/2024 3:44 PM

I would suggest definitely using the oscilloscope, preferably multichannel to compare signals.

__________________
Hare today, goon tomorrow!
Register to Reply
Associate

Join Date: Feb 2011
Posts: 25
Good Answers: 2
#15
In reply to #10

Re: Measuring Interference in Control System Cabinet

09/12/2024 3:34 PM

It was little risky to use oscilloscope but it was my second line of action if initial tests I wrote in parent message would come negative.

Luckily, we found the root cause of issue and I will write about it in new comment.

__________________
For 'tis the sport to have the engineer Hoist with his own petar; and't shall go hard But I will delve one yard below their mines And blow them at the moon. - William Shakespeare -
Register to Reply
Guru

Join Date: Oct 2006
Location: 44.56024"N 15.307971E
Posts: 7806
Good Answers: 264
#11

Re: Measuring Interference in Control System Cabinet

09/12/2024 4:14 AM

Be sure to use an isolated bonding conductor(green wire) from the main to the destination.Conduits and boxes are unreliable bonding grounds.Usually ,the receptacles with isolated bonding will be orange,such as hospitals etc.If the bonding becomes loose anywhere it can put noise on the line.A loose locknut,a bump on the conduit by machinery,etc.It is worth checking.

It sounds like the problem may be confined to one slot,so check the back plane for loose or broken connections,and any board traces for cracks.They are sometimes hard to see.I found a crack in a motherboard that looked like a hair running across a trace.It got a little wider under heat and revealed itself.

Have any Variable Frequency Drives been installed recently? They can cause problems all around if not properly grounded or using the proper wiring methods.

Also if the motor is not designed for VFD it can cause a lot of interference(no grounding ring,or ceramic bearings).

Try looking at the voltage supply with a scope.

Please follow up when you find the cause.

__________________
"A man never stands so tall as when he stoops to help a child." "Never argue with a stupid person.They will drag you down to their level and beat you with experience" "Homo homini lupus"
Register to Reply
Associate

Join Date: Feb 2011
Posts: 25
Good Answers: 2
#16
In reply to #11

Re: Measuring Interference in Control System Cabinet

09/12/2024 3:36 PM

Thanks for suggestion.

I was suspecting Bonding and Earthing issues from the beginning and that came out true.

We have found the root cause of issue and I will explain it in new comment.

__________________
For 'tis the sport to have the engineer Hoist with his own petar; and't shall go hard But I will delve one yard below their mines And blow them at the moon. - William Shakespeare -
Register to Reply
2
Associate

Join Date: Feb 2011
Posts: 25
Good Answers: 2
#17

Re: Measuring Interference in Control System Cabinet

09/12/2024 4:24 PM

Thanks for all the suggestions and replies.

We were able to find the root cause(s) of issue and it was cumulative in nature.

The ESD system under discussion, with its Logic solver modules and power supplies, has been running for 20 years. As per manual, couple of module types have Electrolytic capacitors in them having documented operating life of 10 years (as per OEM) and required replacement afterwards, but they were not replaced.

During investigation, we found System cabinet floor (from where cables entered the system) was not properly sealed. Very hot and humid dirty air was found coming from the crevices and it created a sludge on IO racks and modules.

As per drawing checks, the IE and PE bars were supposed to be segregated and each having their own earthing pit but when cabinet floor was opened, we found cable connection between IE and PE bar, and there was only 1 pit i.e. IE pit and no PE pit.

So it was concluded that the dirty and hot humid condition of cabinet aggravated the modules electrolytic capacitors ageing and caused them to go into fault state. The noise/ interference might be generated by these electrolytic capacitors due to change in temperature, or from AC supply source (bad earthing) which further damaged already compromised modules.

__________________
For 'tis the sport to have the engineer Hoist with his own petar; and't shall go hard But I will delve one yard below their mines And blow them at the moon. - William Shakespeare -
Register to Reply Good Answer (Score 2)
Guru

Join Date: Dec 2010
Posts: 1686
Good Answers: 116
#18
In reply to #17

Re: Measuring Interference in Control System Cabinet

09/14/2024 12:19 PM

Robotics Engineer could have saved a lot of speculation by writing that the safety system had worked for tens of years before the problems! I might have commented about a trial triplicated, voting, governor which threw up repeated discrepancy alarms over a few months which could never be tied down to causes despite expensive investigations - proven unduplicated governors going for years without trouble.

Electrolytics go open or short after tens of years, even tantalum reservoirs in psu which blew out after tens of years, stopping the governor & generator. They are most likely to fail of electronic components. I would mention back-up batteries which get forgotten although the maintenance manuals give them a life.

I am dubious about the value of separate protective and instrument "earth pits" - your system has worked without them for 20 years?. The value is in keeping the mind concentrated on keeping the "earths" separate so that power faults or noise do not circulate through instrument commons [opening the link IE to PE and measuring DC resistance may be revealing, AC test even more, should be done periodically]. I have seen many specifications for power plant earth systems requiring 0.5 ohms for good reasons - which is 60+ rods in good agricultural soil, a doddle in a salt marsh [most big power stations were by rivers or sea] but fantasy on dry sand. If two earths on the same site are each 0.5 ohms to a remote earth potential, the resistance between them is low, so what separation?.

In reality, what matters is that everything power electrical in the building [it could be all wood] is bonded together & to the incoming power source. Aircraft do not have even one earth, airships filled with hydrogen worried about safe static discharge each landing. Aircraft are regularly struck by lightning, the proof test is about 200kA peak and microseconds.

If you want to measure a resistance, bonding or between earths, I note that local building wiring regulations require 0.5 amp DC current test, multimeter currents are too low to reveal unsecure joints. Digital multimeters have too much susceptibility to EMC. The rows of electromechanical AVOmeters at military EMC test facilities I have seen are there because they are far more resistant than electronics and check the basics.

To check a substantial bond or between earths use a battery, as many amps as possible and analog meters as a check on the clever electronic meters [and swap the polarity].

Register to Reply
Guru

Join Date: Oct 2006
Location: 44.56024"N 15.307971E
Posts: 7806
Good Answers: 264
#19
In reply to #18

Re: Measuring Interference in Control System Cabinet

09/14/2024 1:23 PM

I agree with you on the analog meters vs digital for some applications.The digital meters will not show a wet ground because they do not have enough sensing voltage to ionize the water,whereas an analog meter will read a wet ground.

It is important to know the difference between bonding and grounding.

This confuses many people,even those in the trade.

Plenty of info on the net about this.

Some sites require an unbroken conductor between all ground rods and CAD welding of all grounding rod clamps to prevent possible loosening.

__________________
"A man never stands so tall as when he stoops to help a child." "Never argue with a stupid person.They will drag you down to their level and beat you with experience" "Homo homini lupus"
Register to Reply
Associate

Join Date: Feb 2011
Posts: 25
Good Answers: 2
#20
In reply to #18

Re: Measuring Interference in Control System Cabinet

09/14/2024 2:44 PM

Hi 67model,

Thanks for sharing your experience.

We (my team and OEM Tech support) were assuming the same before that ageing is the primary cause of multiple modules faults but when these faulty modules were tested in less critical ESD system in other location, they became operational and working since without any faults (still under observation for more than three weeks). So, this presumption of ageing alone as primary factor was wrong and I kept it out of my initial post.

The IE and PE segregation might not be important for small factories or localized systems but for Oil & Gas and Petrochemical plants, standards and regulations are seriously followed and tied to System warranty and Plant insurance so this is somewhat important. It also makes sense to have earthing pits near Cabinet locations in decentralized architecture.

Technically though, when we measured the IE bar resistance with earth pit, it showed very high resistance (KOhms) and visually bonding was rusty. For sure, it did not happen in a week or months and might took years, so if any intermittent shorts or current inducement happened after this period, it was degrading further the life of system and its components.

DMM was used to give initial direction and hints in troubleshooting. As plant and ESD system are both operational, we could not use Earth meters, Meggers etc. as they might induce interference and cause plant trips.

__________________
For 'tis the sport to have the engineer Hoist with his own petar; and't shall go hard But I will delve one yard below their mines And blow them at the moon. - William Shakespeare -
Register to Reply
Guru

Join Date: Dec 2010
Posts: 1686
Good Answers: 116
#21
In reply to #20

Re: Measuring Interference in Control System Cabinet

09/14/2024 5:35 PM

Since post #17 by Robotics Engineer indicates PE & IE bars were linked [ a serious case of "not to drawing"] and only IE pit existed it would appear a philosophy of "You can never test that because the system is operating" has led to neither PE or IE being adequate.

If a small cable were run with the main cable to the pit, it would be possible test the loop to the earth pit routinely with a multimeter. Also regular visual inspection for corrosion, a pessimist would consider all the earth rods rusted away below ground!

Register to Reply
Guru
United Kingdom - Member - Indeterminate Engineering Fields - Control Engineering - New Member

Join Date: Jan 2007
Location: In the bothy, 7 chains down the line from Dodman's Lane level crossing, in the nation formerly known as Great Britain. Kettle's on.
Posts: 32049
Good Answers: 838
#23
In reply to #17

Re: Measuring Interference in Control System Cabinet

09/19/2024 5:53 AM

<...created a sludge on IO racks and modules...>

No wonder it fell over!

It sounds as though periodical planned preventative maintenance activity on the equipment is inadequate; this is a procedural issue, best dealt with by the maintenance operation locally.

__________________
"Did you get my e-mail?" - "The biggest problem in communication is the illusion that it has taken place" - George Bernard Shaw, 1856
Register to Reply
Guru
United Kingdom - Member - Indeterminate Engineering Fields - Control Engineering - New Member

Join Date: Jan 2007
Location: In the bothy, 7 chains down the line from Dodman's Lane level crossing, in the nation formerly known as Great Britain. Kettle's on.
Posts: 32049
Good Answers: 838
#22

Re: Measuring Interference in Control System Cabinet

09/19/2024 5:51 AM

<deleted at edit>

__________________
"Did you get my e-mail?" - "The biggest problem in communication is the illusion that it has taken place" - George Bernard Shaw, 1856
Register to Reply Off Topic (Score 5)
Register to Reply 24 comments
Interested in this topic? By joining CR4 you can "subscribe" to
this discussion and receive notification when new comments are added.

Comments rated to be Good Answers:

These comments received enough positive ratings to make them "good answers".

Comments rated to be "almost" Good Answers:

Check out these comments that don't yet have enough votes to be "official" good answers and, if you agree with them, rate them!
Copy to Clipboard

Users who posted comments:

67model (2); BruceFlorida (1); HiTekRedNek (2); phoenix911 (2); PWSlack (4); redfred (1); Robotics Engineer (8); rwilliams (1); SolarEagle (1); StandardsGuy (1); Stef (1)

Previous in Forum: What is the Name of the Opposite of a Rubber Grommet to Protect an OD Edge?   Next in Forum: How Much is Quiescent Current Iq of OP27 OP AMP?

Advertisement