More than five million people in the UK are annoyed by their neighbours, but a quarter fail to take any action, according to research.
Consumer group Which? said noise was the most complained-about irritant, with around three in five people getting angry about voices, arguing, blaring music and TVs.
The report found 10 million people have had a neighbour problem in the last year.
Richard Lloyd, the Which? executive director, said: "That so many people are losing sleep, getting stressed and struggling at work because of noise from next door shows the damage this does."
A quarter of those affected are annoyed by slamming doors, a similar number have been disturbed by noisy pets, and one in five by regular parties.
Researchers found 5% have heard their neighbours having sex.
The results show a quarter of people who are frustrated with their neighbours have made no attempts to rectify the problem.
Mr Lloyd added: "People needn't suffer in silence. If trying to solve the problem with your neighbour doesn't work or simply isn't possible, then get in touch with your local authority who can take action for you."
The report showed 10% chose to take revenge by becoming nuisance neighbours themselves, a third spoke calmly to their neighbours about the issue, while one in five contacted their local authority.
It found 17% of people were forced to call the police.