I don’t know how you hold your phone but most people don’t hold them in a way where the camera is pointed towards people. In most circumstances it’s pretty easy to tell whether someone is just on their phone or is actively recording you.