PSP2600 is one of the oldest Atari 2600 emulators out there. It replicates the Atari 2600 hardware and provides access to a bunch of classic games – E.T. the Extraterrestrial, Missile Command, Space Invaders, Demon Attack or Yars' Revenge, among many others. The emulator was designed with accuracy in mind. Unlike other modern emulators, it does not improve the speed or smoothness, so the old fashioned experience is unchanged. The emulator is specifically created for PSP. It is relatively small and does not require too much space. It is also based on a different Atari 2600 emulator – Stella.
PSP2600 is available for free over the Internet. The project has been abandoned though, so new updates are less likely to come out. There are also a few reports of showing corrupted files on modern PSP devices, so it may not work for everyone – it is totally worth a try though.