But it's important to realise that there are some things that I am never going to experience just because of the way I was born and it's better to let those people talk on that subject instead of me
If we are talking about personal experiences you are somewhat correct, no one can experience something for you and you are the best advocate for said experiences. Empathy suggests that there is leeway but it is a decent rule of thumb for most situations.
If we are talking about a particular subject, one that personally affected you, then the answer is emphatically no. This is where the "Check your Privilege" because caustic for not only shutting down criticism but it perpetuates a sense of victimhood.
If I am talking about how I was shot on the subway, Only "I" could talk about how it felt to be shot, even other people who were shot didn't have "My" experience. I am not, however, a expert on gun violence, victimization, or a psychologist NOR is my viewpoint a collective viewpoint. When you extend my experience outside of a personal experience you are creating problems.
Finally just because I said I was shot and it felt a certain way it doesn't mean that you must take my personal account. I might be lying, I might be mistaken, I might have come to the wrong conclusions, my experience could be extremely niche. As well how much people value my Point of View is up to them as well.
Sure, you could say that being homeless is easy and contradict an actual homeless person. You would be wrong and people would be bound to point that out, but the way you don't handle it is to say "Hey, that homeless person had it rough, your privilege disavows you from speaking". (Still remember that guy who as an experiment was homeless for a day and said it was easy)
Second Finally. The point of speaking of your experiences is to relate them to other people. They hear what you are saying, they ask questions, and they come to their own conclusions, conclusions that they might tell other people. What "Check your Privilege" wants is for you to take their conclusions and not to ask these questions.