What happens if one of my name servers is down?

Queries to the "down" server would timeout (5 seconds). The other name servers authoritative for the zone would respond.