The shift drum shaft should move freely, otherwise the trans wouldn't shift. The fixing bolt is there to limit endplay. Sounds like it's either a transmission or primary issue. The tranny mainshaft spins even in neutral. So, it's either not completing the gearchange into neutral or it's in the primary drive. A worn clutch pilot bushing, something worn/bent in the clutch or even a chipped tooth on the primary drive could be the culprit. There are other possibilities, as well. As luck sometimes goes, you don't have to split the cases to check the primary drive and shift linkage, just pull the clutch cover. If something is amiss, it should be easy enough to see. If not, you may have worn shift forks, pins or the drum; there could also be excessive endplay in the shift drum assembly.
One quick check would be to remove the chain, fire up the engine and see if the noise is present in gear. If it is, especially in all gears, then the primary drive looks even more promising as the source.