Sure, I think that would work! You'd want to make sure you left space around the housing just to be safe, I use Vortex 542 cfm 6" inlines, and they don't really get hot (I would think most inline fans would be self cooling). I think I would build a plywood box the correct size with the necessary cutouts, maybe a hinged door for access...and then line it with some type of insulation. I don't like rock wool all that much for growing, but it makes one of the very best sound insulators out there. If you glued a 2 inch layer to the inside of your box and then tucked it around the duct holes That would probably do the trick. They also make mufflers if you need to quiet the sound coming out of the exhaust vent itself.
Make sure your airflow is good around the ballast, upper shelves in closets can get really hot... I know we like to put them up out of the way, but if you have a way to mount them lower, convection will be able to do a better job at moving the hot air away from it. The cooler you can keep your ballasts while they are running the longer they will last!
Hey bro, I just noticed my dyslexia kicked in, and I reversed the numbers in the cfm above... its 452 cfm! It's too late to edit the post so I just want it to be accurate!