Noah asked himself why they didn't just right out attack him while his guard was down but instead monitored what he was doing.
He didn't think that, to them, it might've seemed strange that he let his guard down. It definitely seemed strange, so the other parties were extremely cautious.
For all they knew, it might've been a trap to lure them out and deal them damage.
In other not to fall into that trap, they decided to simply mess with him and his surroundings.
It was during this time that Noah decided it was a good idea to clear the surroundings of the rogues so it made them even more cautious, especially when he took on the stubborn rogues himself and came out unscathed.
They watched his movements a lot more from there and then Noah began to notice it. Even with his guard down, with the increasing forces of the people watching him, it was almost impossible for him not to notice.
