-
-
Notifications
You must be signed in to change notification settings - Fork 32
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Video stuttering when rendering to SFML texture #203
Comments
you can print the draw interval |
I would need a little more help please. |
if (new_frame == true) {
const auto t = steady_clock::now();
printf("elapsed: %lld\n", duration_cast<milliseconds>(t-t0).count());
t0 = t; |
Playback of 30fps video at 60Hz
|
Playback of 60fps video at 60Hz
|
Elapsed print after
|
Video1
Video2
|
what about the interval in render callback? what about changing new_frame type to atomic and check |
I've updated the code in my first post elapsed: 30 new_frame: 1
elapsed: 156 new_frame: 2
elapsed: 2 new_frame: 3
elapsed: 1 new_frame: 4
elapsed: 41 new_frame: 1
elapsed: 2 new_frame: 2
elapsed: 43 new_frame: 1
elapsed: 31 new_frame: 1
elapsed: 30 new_frame: 1
elapsed: 30 new_frame: 1
elapsed: 62 new_frame: 1
elapsed: 30 new_frame: 1
elapsed: 15 new_frame: 1
elapsed: 30 new_frame: 1
elapsed: 47 new_frame: 1
elapsed: 30 new_frame: 1
elapsed: 30 new_frame: 1
elapsed: 30 new_frame: 1
elapsed: 30 new_frame: 1
elapsed: 31 new_frame: 1
elapsed: 46 new_frame: 1
elapsed: 29 new_frame: 1
elapsed: 46 new_frame: 1
elapsed: 29 new_frame: 1
elapsed: 31 new_frame: 1
elapsed: 31 new_frame: 1
elapsed: 29 new_frame: 1
elapsed: 46 new_frame: 1
elapsed: 31 new_frame: 1
elapsed: 30 new_frame: 1
elapsed: 31 new_frame: 1
elapsed: 31 new_frame: 1
elapsed: 31 new_frame: 1
elapsed: 30 new_frame: 1
elapsed: 31 new_frame: 1
elapsed: 30 new_frame: 1
elapsed: 46 new_frame: 1
elapsed: 30 new_frame: 1
elapsed: 31 new_frame: 1
elapsed: 31 new_frame: 1
elapsed: 31 new_frame: 1
elapsed: 31 new_frame: 1
elapsed: 48 new_frame: 1
elapsed: 30 new_frame: 1
elapsed: 29 new_frame: 1
elapsed: 31 new_frame: 1
elapsed: 31 new_frame: 1
elapsed: 31 new_frame: 1
elapsed: 46 new_frame: 1
elapsed: 30 new_frame: 1
elapsed: 15 new_frame: 1
elapsed: 47 new_frame: 1
elapsed: 31 new_frame: 1
elapsed: 31 new_frame: 1
elapsed: 30 new_frame: 1
elapsed: 30 new_frame: 1
elapsed: 47 new_frame: 1
elapsed: 15 new_frame: 1
elapsed: 46 new_frame: 1
elapsed: 31 new_frame: 1
elapsed: 31 new_frame: 1
elapsed: 30 new_frame: 1
elapsed: 30 new_frame: 1
elapsed: 47 new_frame: 1
elapsed: 31 new_frame: 1
elapsed: 30 new_frame: 1
elapsed: 31 new_frame: 1
elapsed: 31 new_frame: 1
elapsed: 45 new_frame: 1
elapsed: 31 new_frame: 1
elapsed: 32 new_frame: 1
elapsed: 32 new_frame: 1
elapsed: 15 new_frame: 1
elapsed: 46 new_frame: 1
elapsed: 31 new_frame: 1
elapsed: 30 new_frame: 1
elapsed: 45 new_frame: 1
elapsed: 15 new_frame: 1
elapsed: 46 new_frame: 1
elapsed: 30 new_frame: 1
elapsed: 30 new_frame: 1
elapsed: 30 new_frame: 1
elapsed: 30 new_frame: 1
elapsed: 45 new_frame: 1
elapsed: 31 new_frame: 1
elapsed: 30 new_frame: 1
elapsed: 30 new_frame: 1
elapsed: 31 new_frame: 1
elapsed: 30 new_frame: 1
elapsed: 30 new_frame: 1 |
I've noticed that removing |
But with Threaded Optimization set to ON the log looks different:
|
With the following hardcoded framerate example I was able to render 30fps@60Hz smoothly. #include <SFML/Window.hpp>
#include <SFML/Graphics.hpp>
#include "mdk/Player.h"
using namespace MDK_NS;
#include <iostream>
int main(int argc, char** argv)
{
int flipflop = 1;
Player player;
player.setMedia(argv[argc-1]);
player.prepare();
for(;;)
{
if (player.mediaStatus() > MediaStatus::Buffering) break;
if (player.mediaStatus() == MediaStatus::Invalid) return 0;
}
auto video_info = player.mediaInfo().video[0];
player.setVideoSurfaceSize(video_info.codec.width * video_info.codec.par, video_info.codec.height);
sf::RenderWindow window(sf::VideoMode(800, 600), "Test");
window.setVerticalSyncEnabled(true);
sf::RenderTexture texture;
if (!texture.create(video_info.codec.width * video_info.codec.par, video_info.codec.height))
return -1;
player.set(State::Playing);
while (window.isOpen())
{
sf::Event event;
while (window.pollEvent(event))
{
if (event.type == sf::Event::Closed)
window.close();
if (event.type == sf::Event::KeyPressed)
{
if (event.key.code == sf::Keyboard::Escape)
{
player.setVideoSurfaceSize(-1, -1);
return 0;
}
}
}
if (flipflop == 1)
{
texture.setActive(true);
sf::Clock clk;
player.renderVideo();
std::cout << clk.getElapsedTime().asMicroseconds() << std::endl;
texture.display();
texture.setActive(false);
}
flipflop *= -1;
window.setActive(true);
window.clear();
sf::Sprite sprite(texture.getTexture());
window.draw(sprite);
window.display();
}
return 0;
} |
The following code is stuttering very little with TO-OFF, Can this be achieved with callbacks? #include <SFML/Window.hpp>
#include <SFML/Graphics.hpp>
#include "mdk/Player.h"
using namespace MDK_NS;
#include <iostream>
int main(int argc, char** argv)
{
sf::Context ctx;
Player player;
player.setMedia(argv[argc-1]);
player.prepare();
for(;;)
{
if (player.mediaStatus() > MediaStatus::Buffering) break;
if (player.mediaStatus() == MediaStatus::Invalid) return 0;
}
auto video_info = player.mediaInfo().video[0];
player.setVideoSurfaceSize(video_info.codec.width * video_info.codec.par, video_info.codec.height);
float fps = video_info.codec.frame_rate;
std::cout << video_info.codec.frame_rate << std::endl;
ctx.setActive(false);
sf::RenderWindow window(sf::VideoMode(640, 480), "Test");
window.setVerticalSyncEnabled(true);
sf::RenderTexture texture;
if (!texture.create(video_info.codec.width * video_info.codec.par, video_info.codec.height))
return -1;
player.set(State::Playing);
sf::Clock clock;
sf::Time timeSinceLastUpdate = sf::Time::Zero;
sf::Time timePerFrame = sf::seconds(1.0f / fps);
while (window.isOpen())
{
sf::Event event;
while (window.pollEvent(event))
{
if (event.type == sf::Event::Closed)
window.close();
if (event.type == sf::Event::KeyPressed)
{
if (event.key.code == sf::Keyboard::Escape)
{
player.setVideoSurfaceSize(-1, -1);
return 0;
}
}
}
timeSinceLastUpdate += clock.restart();
if (timeSinceLastUpdate > timePerFrame)
{
ctx.setActive(true);
texture.setActive(true);
// sf::Clock c;
player.renderVideo();
// std::cout << c.getElapsedTime().asMicroseconds() << std::endl;
texture.display();
texture.setActive(false);
ctx.setActive(false);
timeSinceLastUpdate -= timePerFrame;
}
window.setActive(true);
window.clear();
sf::Sprite sprite(texture.getTexture());
window.draw(sprite);
window.display();
}
return 0;
} 30.mp460.mp4I've attached test videos I'm using |
player.setFrameRate(fps) |
Thank you for sharing this undocumented function. This definitely helps in the situation when for example the framerate is 29.97 and the refresh rate of the monitor is 59.97. Unfortunately the hassle with calling renderVideo() at a certain moment persists. I'm still unable to call
on each frame, because that is causing a massive judder. Would it be possible to make |
It appears that this juddering is not caused by my implementation. Below is a screen capture of a smooth 60fps video posted above played using glfwplay.exe |
So, is this solvable? |
render callback is invoked when a frame is ready, then call renderVideo() by user
depending on video frame rate and display frame rate |
This judder is weird. It skips frames and repeats at the same time, check the video I posted frame by frame. For a framerate mismatch it should only do one or the other thing, not both. Seems like a timing issue to me. |
This works almost perfect, a little frame skip once for a while, but no forward and back judder. I think the videoRender callback should be fixed. #include <SFML/Window.hpp>
#include <SFML/Graphics.hpp>
#include "mdk/Player.h"
using namespace MDK_NS;
using namespace std;
#include <iostream>
#include <cmath>
int main(int argc, char** argv)
{
Player player;
player.setMedia(argv[argc-1]);
player.prepare();
for(;;)
{
if (player.mediaStatus() > MediaStatus::Buffering) break;
if (player.mediaStatus() == MediaStatus::Invalid) return 0;
}
auto video_info = player.mediaInfo().video[0];
player.setVideoSurfaceSize(video_info.codec.width * video_info.codec.par, video_info.codec.height);
float fps = video_info.codec.frame_rate;
sf::RenderWindow window(sf::VideoMode(640, 480), "Test");
window.setVerticalSyncEnabled(true);
sf::RenderTexture texture;
if (!texture.create(video_info.codec.width * video_info.codec.par, video_info.codec.height))
return -1;
player.set(State::Playing);
// player.setFrameRate(29);
sf::Clock clock;
sf::Time timeSinceLastFrame = sf::Time::Zero;
sf::Time timePerFrame = sf::seconds(1.0 / fps);
float framerate = 60.0f;
sf::Font font;
if (!font.loadFromFile("RobotoMono-Regular.ttf")){cout << "Font not found." << endl;}
sf::Text fps_text;
fps_text.setFont(font);
fps_text.setString(to_string(fps));
fps_text.setCharacterSize(24);
sf::Text framerate_text;
framerate_text.setFont(font);
framerate_text.setString(to_string(framerate));
framerate_text.setCharacterSize(24);
framerate_text.setPosition(0, floor(fps_text.getLocalBounds().height * 1.25));
sf::Clock framerate_clock;
sf::Time next_timestamp = sf::seconds(0);
sf::Clock video_timer;
while (window.isOpen())
{
sf::Event event;
while (window.pollEvent(event))
{
if (event.type == sf::Event::Closed)
window.close();
if (event.type == sf::Event::KeyPressed)
{
if (event.key.code == sf::Keyboard::Escape)
{
player.setVideoSurfaceSize(-1, -1);
return 0;
}
}
}
if ( next_timestamp < video_timer.getElapsedTime() - timePerFrame )
{
texture.setActive(true);
sf::Time last_timestamp = sf::seconds(player.renderVideo());
next_timestamp += timePerFrame;
texture.display();
texture.setActive(false);
}
window.setActive(true);
window.clear();
sf::Sprite sprite(texture.getTexture());
window.draw(sprite);
window.draw(fps_text);
window.draw(framerate_text);
window.display();
sf::Time elapsedTime = framerate_clock.restart();
if(elapsedTime.asMilliseconds() > 1.0f)
framerate = framerate * 0.99f + 0.01f / elapsedTime.asSeconds();
framerate_text.setString(to_string(framerate));
}
return 0;
} |
I was trying to add |
I'm getting a very uneven framerate when rendering the video to SFML texure.
When I play a 30fps video at 60Hz or 120Hz for example the callback function should fire every 2nd or 4th frame, shouldn't it?
I've also tried putting all the calls from
if (new_frame == true)
inside the callback, but the result is the same.Am I using
setRenderCallback()
correctly?Platform: Windows 11
The text was updated successfully, but these errors were encountered: