Harvester, I did a number of tests yesterday with AM panoramas, trying to combine them with AVIs or GIF sequences that were the size of the panoramic image. Not much luck, I'm afraid --- none actually. I feel your pain....
With a panorama and a GIF sequence running on top of it, you get the panoramic rotation, but the gif sequence only proceeds (very roughly) each time you click. With a looping video playing either in a hotspot within the panorama or as a simple looping video in a hotspot, you get a frame-sized piece of the video showing, no panorama, no cursor movement --- but the video is playing. Sorry.
I don't really know of an engine that will do a 3d panorama with the video in hotspot that you're wanting. In SCream you can have a panorama with a moving sky, but no other video or gif support. With Wintermute, you can do parallax scrolling plus video. And in AM (if you forego panoramas, which have poor quality anyway,) you can transition from side to side between single frames and use both video and gif animations. You can also add ShadowHunter's KeyGuard to AM and cause the transiton to slide left or right on hover over hotspot.
If you do find an engine that will do panoramas with video in hotspot, by all means, please let us know.