Hi, using movie stimuli in Psychopy doesn't really work smoothly. Whenever a movie with sound is played I experience dropped frames. I am attaching a little benchmark that can be used to show this (done in the builder, just select a movie file and run it). This issue is only present when sound is actually played. When it is not played, even when a sound stream is present in the file and avbin just 'ignores' it, the stimulus is smooth. All files and codec combinations I tested play smooth in avplay, vlc, mplayer and others. I see this issue on amd64 and i386 -- independent of whether Psychopys own system check reports 'dropped frames' on a particular system or not. I tested Debian Jessie and Ubuntu 13.04, both show the problem. Psychopy upstream recommends avbin 5 for movie stimuli, but Debian never had it (according to snapshots.d.o). Psychopy upstream also recommend uncompressed PCM audio -- that makes no difference in comparison to, e.g. FLAC. Interestingly, I see a similar jerky playback in openshot. However, when using the MELT player (from the framework that openshot is based on) everything is smooth. This is also not an obvious IO issue. Even if I cache the entire movie file in memory, the frame drops are still present. System load at playback is negligible. Any advise would be very much appreciated. Thanks, Michael
I guess this is hardware independent? (i.e. you have tried on few boxes/laptops with the same success) have you asked PsychoPy guys themselves? O mighty Mario, would you happen to have ideas where to look at?
I tried 4 machines from three vendors so far. They had intel or nvidia hardware, debian wheezy, jessie or ubuntu 13.04 -- all with the same performance pattern. Not yet. Michael
This doesn't seem to be the case. I have redone my test using the movie clip that is shipped with psychopy in order to minimize the chance that my movie encoding is the source of the problem /usr/share/pyshared/psychopy/demos/coder/stimuli/jwpIntro.mov Even with a frame size of 40x30 it reports dropped frames (although at this size I cannot confirm their presence visually anymore). I tried h264, mp4, mpeg2 and mpeg1, which should be increasingly easier to decompress, and I tried different bitrates -- nothing makes the clip play smooth. At the same time I see no significant CPU usage on an i7-3667U CPU @ 2.00GHz. Most of the tests happened on machines with INTEL hardware, but on NVIDIA hardware I see the same issue. Michael
This is pure guesswork, but as you asked ... ;-) I don't know how PsychoPy does this or how avbin works, but my best guess from the info below would be that PsychoPy could be inefficient in converting video frames to OpenGL textures for rendering, so it can't keep up with avbin? Assuming of course that PsychoPy or some backend library converts video frames to OpenGL textures for presentation? In any modern movie player i know of, if there are any audio tracks played back, the principle is that audio timing drives video timing and the player drops or holds back video frames to keep audio and video from drifting apart. If avbin does the same, then it would drop video frames if the client can't keep up with fetching/processing them, to keep audio-video sync. If there wouldn't be any audio it might just wait for the client to catch up - video would not drop frames but play too slowly or jittery. You could check if the problem goes away/is reduced if you use movies with very small frame sizes, so inefficient conversion into OpenGL textures doesn't matter. And/Or try on graphics cards from different vendors/different drivers etc. But this is all just a guess based on how Psychtoolbox does it. One can easily have over an order of magnitude of performance difference in the video->OpenGL conversion stage, depending on implementation and/or graphics driver. I spent multiple months optimizing that part of ptb, so this can be non-trivial. best, -mario