Urban, I know you said you might want to enlarge 35 or 120 film, hence the need for an enlarger, but for contact printing alone, you don't need the large sized Ilford contrast filters, the smaller ones will work, depending on your light source.
I cobbled together a light from a metal soup can, an electrical socket, wiring and a few other bits I had on hand. I suspend it above the darkroom table. Thus far I've used it for preflashing paper negatives, and it has a ~1/4" aperture, along with a 7.5 watt bulb, to limit the intensity. But for contact printing I only need to enlarge the aperture and/or use a higher wattage bulb. The enclosure and aperture are small enough to use the smallest contrast filters.
My point is that just for contact printing you don't need to spend a lot of money, there are ways to make do.
~Joe