First, let's make clear a harsh truth of the world: Companies are economically incentivized to pay you as little as possible, even if you think they're generous.
You don’t get what you deserve, you get what you ask for. Many candidates are unaware how high compensation can go, and therefore never ask about it. This leads to them being underpaid, and the information asymmetry is used to benefit the company.
Recruiters and companies talk to candidates every week or even every day. On the other hand, you probably only talk to a few companies every few years. So recruiters have way more information about what you’re likely making and what you may expect.