I've gone through this process for two chiptune cover albums, although I covered more prominent bands (Neutral Milk Hotel and The Flaming Lips) and wanted to do everything completely legal so went through the proper channels, reached out to the bands, their manager and lawyer, notified them of my intent, and used Easy Song Licensing to set up royalty payments (but you could use any licensing site). I also got permission from band members (via their manager) to use their likeness in artwork, as well as reached out to the album artwork designers for approval on my designs, which were derivative of their own work. Basically if you're doing any sort of derivative work (a cover, a pixel art version of a graphic, etc), you need permission and/or to compensate the artist.
Now given, the artists you're covering might not be listed through a licensing site like ESL unless they've been around awhile and have released multiple albums through a label or collective, so you may have to go to them directly if that's the case. If they (or their representative) haven't registered the copyrights, they likely won't be in any database - but their work is inherently copyrighted even if it's not registered. I haven't gone 'directly to the source' before but in all my interaction with the members and associates of previous bands whose work I covered, they were always nice, answered my questions and told me what they expected from me (a la paying royalties, proper credits, etc).
It's also important to note that even if you don't stand to make any money from a project (eg putting it on Bandcamp for free or in the context of a YouTube video) it is still copyright infringement all the same as if you were seeking monetary gain. Any work that is a cover or derivative of copyrighted source material is inherently an infringement if certain steps aren't taken (and if they don't fall under fair use, which a cover album does not) and if they registered the work(s), they have extra legal power to protect it. As an example, I released my first cover album through Kickstarter and included downloads with the physical vinyl records for backers but still had to pay royalties on the 'free' downloads, in addition to royalties on the physical copies. just because you give it out for free doesn't mean it's exempt.
I think that most people don't realize how many people on YT, BC, etc are infringing on copyright because too many people don't look at it legally but logistically - for example, most people think that if you're not making money, you can't be sued, but you have to consider that any work that is protected has that protection not just to prevent others from making money using it (or some version of it) but to protect the integrity. Like if I covered a song from a kid's show and changed the lyrics to be raunchy, that would be a problem for the image/brand. But that's just an extreme example where they would be far more likely to pursue litigation.
I'm not a lawyer and have no extensive knowledge of copyright law but as a general rule of thumb, better safe then sorry. If you don't know with 100% certainty that it's safe and legal, don't do it unless you're willing to risk litigation. Given, there will always be a subjective %risk with anything, and I've certainly taken my fair share of (what I considered to be low) risks with projects in the past, but that's a personal choice you need to make. A good example of that is me covering the soundtrack to Dr. Horrible's Sing-along Blog. I received a cease and desist from Joss Whedon's lawyers but was able to reach out through one of the writers of the series to get them to back off. On the one hand, you have a company like Capcom endorsing artists who cover music from their games while Nintendo has issued DMCA take downs on multiple fan projects. And then there's the photographer credited with Miles Davis' "Kind of Blue" album cover who sued an artist for his derivative pixel art version of that album artwork for a cover album of the original. You just never know how the original artist, or their lawyer(s), will react.