Take a close look at the brake cam. There may be a non-splined portion of the outside circumference and, if so, it lines-up with the un-splined ID of the brake arm. CT70 rears are like this, can't recall if the early Z50s are, or not. It's a foolproof system.
As for brake shoe wear, that makes a big difference in brake cam rotation, and the tension adjustment. The later CT70s have wear indicators, placed over the brake cam splines...along with an index pointer in the brake plate casting. If you have new brake shoes, use `em. Vintage originals usually suck out loud...no stopping power. They also contain asbestos.
ainkiller: