Oil in the intercooler is normal. I'll explain why, and what you can do to reduce it.
Diesels are high compression engines and the pressure in the combustion chamber is quite considerable. Some of the combustion gas seeps past the piston rings (we call this "blow-by" ) and enters the crankcase. This gas is understandably very hot. It does cause the oil in the crankcase to heat up enough for a small amount to form a vapour.
So that the crankcase doesn't pressurise to the point where it explodes, blow-by is vented through an opening in the top of the camshaft housing (usually) called the Positive Crankcase Ventilation (PCV) valve. It usually passes through a cooler before reaching the PCV, in an attempt to extract most of the oil.
This blow-by is then piped to the air intake just in front of the turbocharger, where it enters the engine's intake and is combusted to remove the oil from the environment. Of course, the air path from this point has to pass through the intercooler, which is designed to cool down the air heated up by compressing it through the turbocharger (air comes out of the turbocharger at higher pressure, and somewhere between 200 and 250 degress celcius). As the air cools, some of the remaining oil mist also cools, and stays inside the intercooler.
So that's HOW it happens.
You can reduce the amount of oil by putting a "catch can" in the hose that goes from the PCV to the air inlet. This won't get rid of the oil altogether, but it will catch a great deal of it, particularly if the catch can is mounted forward of the engine where the ambient air is a little cooler. You won't get rid of all of the oil, and you'll have to empty the catch can manually (or add a fitting that allows you to open a tap and drain the oil back into the engine's sump).
You can't get rid of the oil altogether without venting the PCV directly to atmosphere and I don't recommend that you do that now, because of the risk it presents to other vehicles. We used to do that, and anyone who used to drive back in the 80s or earlier would remember the dark patches on the roads at the traffic lights. Motorcyclists learned very quickly to stay off those!
So you'll have to clean the intercooler. Petrol is a good solvent for this job, it mostly dries right up and won't harm your car. I've done it myself. One hand over one opening, pour about 100ml of petrol in the other. Put your other hand over the top opening then invert the intercooler, releasing the upper hand. Swish it around a few times. Repeat until the petrol comes out clear.