HA believes that the physical associations between high redshift objects (typically quasars) and low redshift galaxies provide a major challenge to the Big Bang. Here is a brief list of some of the reasons that most astronomers are sceptical of Arp's claims (in no particular order):
1) The observed superpositions of quasars and nearby galaxies could be accounted for by chance. Arp's claims to the contrary ignore two key effects. First, nearby galaxies are studied intensively (by Arp among others) so it's more likely that quasars would have been identified there than in random regions of the sky. Second, gravitational lensing makes quasars behind galaxies look brighter, and thus further increases the likelihood of identifying a quasar.
2) Arp was initially led to his belief in non-cosmological distances for quasars because there was no good theory of how the huge energies implied by large distances of quasars could be supplied. We now understand this --- accretion onto supermassive black holes is strongly supported by, among other things, the observations of accretion disks we discussed a few weeks ago.
3) The Big Bang cosmology that provides the framework for cosmological quasar distances is strongly supported by other evidence.
4) The quasar distance scale must be correct, or the gravitationally lensed quasars like this would not be observed.
5) Several of Arp's specific examples the higher redshift object has been shown to be further away than the low redshift object. In the case of Stephan's Quintet for example, stars in the foreground galaxy have been shown to be brighter than those in the higher redshift galaxies, confirming the relative distances suggested by the redshifts.