Yes, if you are no longer experiencing soreness, your muscles are getting used to it. You are not strengthening them as much as you were, you are just keeping them fit and toned.
Supposed you do 25 sit-ups one day and you feel sore. Next time you do it, you still feel sore. If you keep doing it, eventually you don’t feel sore. 25 sit-ups is no longer a big deal to your muscles. So then one day you do 50 sit-ups, and you are sore again…
You want to occasionally work your muscles so that you feel that same tired soreness. Not everyday, but occasionally. Since you swim M-F, I would suggest working hard on Friday, and then you have a few days to recover.
Try faster and longer on Friday. Make it a goal to work yourself really hard one day a week.