Here are tips from HungryFlix.

Video content must be MPEG4. We highly recommend encoding with the H.264 codec. While encoding times will increase, H.264 will yield higher quality video with smaller file sizes.

This works very well for movie downloads.

MPEG4 with H.264 Codec (file formats: .m4v, .mp4)
Lower-quality iPod/iTunes Video: Up to 768 Kbps, 320 x 240 pixels, 30 frames per second, Baseline Profile up to Level 1.3. Audio: AAC-LC up to 160 Kbps, 48 KHz, stereo audio
High-quality iPod: Up to 1.5 Mbps, Baseline Low-Complexity with AAC-LC audio up to 160 Kbps (maximum resolution: 640x480 pixels at 30 fps) in .m4v format
High-quality Apple TV: Up to 3 Mbps, Progressive Main Profile (CAVLC) with AAC-LC audio up to 160 Kbps (maximum resolution: 1280 by 720 pixels at 24 fps, 960 by 540 pixels at 30 fps) in .m4v, .mp4 and .mov file formats

MPEG-4 video
PSP Video: Up to 2.5 Mbps, 480 x 480 pixels, 30 frames per second, Simple Profile. Audio: AAC-LC up to 160 Kbps, 48 KHz, stereo audio
High-quality iPod/Apple TV MPEG-4: Up to 3 Mbps, Simple Profile with AAC-LC audio up to 160 Kbps (maximum resolution: 720 by 432 pixels at 30 fps) in .m4v, .mp4 and .mov file formats

MPEG-4 High Definition Video
File formats: .m4v, .mp4 and .mov
Video: 1280 x 720 pixels, 30 frames per second. Audio: AAC-LC up to 160 Kbps, 48 KHz, stereo

MP3 Audio
Up to 160 Kbps, 48 KHz, stereo audio

AAC Audio
AAC-LC up to 160 Kbps, 48 KHz, stereo audio

MAXIMUM FILE SIZE FOR ALL CONTENT – 700 MB