Is it bad to use several grounding points for a device? Especially when it comes to input and output?

It's not bad, it's necessary. If the devices in your signal path don't share a ground reference, you're not gonna get a good audio signal. The 'better performance' you describe is probably a result of signals being greatly attenuated.

I am doing a quasi c.bend on an old car audio system eq. So this just a general question, not really chip.

...

Long winding description:

It has 4 channels out (speakers) with a ground connection each, I put 4 rca female here. The 2 input channels share 1 ground, I put a stereo jack female here.

Sometimes there is better performance, lower noise, no loud pops, when I detach some grounding cables, but then other, random it seems, things does not work, like panning. I just can not figure out how it works.