I'm feeling a little worried about my job, should I go back?

You done the right thing but before you did you shouldn't have let it go so long and should have reminded her that she is only a Temp. I would not leave the job because of her as she is only a Temp, is there a supervisor you could talk to in private for advice. It could be very hard to get another job and if you do they will want a previous work reference? Good luck.