sac.py: to build single sac agent
masac.py: to build multi-sac-agents
utils.py: to put the utility function
replaybuffer_ma.py: replay buffer for multiagent
main_masac.py: the main function for the code, with train and test in it
###################
The new CRPO modification is in add_crpo branch, a diff between main and add_crpo could see the changes.