Britain says it won't hand over Gibraltar as long as most residents want to remain British.