Some guy on the int. Discord figured it out.
You can use the char modifier, but instead of just adding and subtracting (like -1 Hard Hit, +1 COTA), you first have to add to _every_ CA and after that then modify to your hearts content.
Question answered. Can be closed.