Can somebody explain me how it is decided which DNS server is used for resolving domain? I understand that I register name server set with my registrar and requests for my name servers are responded by TLD NS. How exactly related are NS in my zone?

I noticed that round-robin is done by all NS servers but for me it is undesirable behaviour as ns1.linode.com is about 200ms from my average visitor and ns5.linode.com is 30ms away.

Is it possible to tell recursive DNS to ask ns5.linode.com first and if it is down jump to 4 3 2 1?

I can set my zone NS to use only ns5 and ns4 but I like to have redundancy without this "balancing" feature that is kind of pain in the a...

Pick a number, any number. Use that server. That's what it comes down to.

MS DNS servers tend to go with the first nameserver that they get in a reply, so most DNS servers randomize the order of the records that are returned.

Or in short, it is almost entirely random.

As long as you don't have your TTL set unnecessarily low, it shouldn't matter too much. Users should only need to do the DNS lookup once.