Is Microsoft’s hyperlapse algorithm the solution to shaky GoPro videos?
Video lovers have released first-person videos before but it’s complexity was a big challenge back then. However, thanks to the arrival of GoPro and Google Glass, this has now become widespread.
Do you want to know what it’s like biking down a mountain? There’s probably a video of that up on YouTube. Would you like to see what it’s like to take a plunge from a very tall mountain? You’d most likely find videos of people screaming for dear life up on any video sharing site.
Head-mounted cameras like those released by GoPro and Google Glass have been really fascinating in allowing people to capture action and to share these with their friends, family and the world.
As great as it is in doing so, the resulting video is often so jittery that it leaves watchers with nausea.
However, software engineers at Microsoft Research are trying to change that. They have developed an algorithm that is capable of converting footage shot in first-person into a smooth hyperlapse video. And take note: this is regardless of how jittery the original recording was.
Now, hyperlapsing is a method that is used in multimedia where a collection of photographs or video frames are strung together in order to create a cinematic experience. This technique is much well known as stop-motion time-lapse.
When the algorithm is applied, it produces a result that is void of shakes, head bobs and other movements. Instead, you get a video with a smoother quality that makes the viewing all the more pleasant.
Microsoft was able to demonstrate the capability of their algorithm by converting a bike ride video into a flying hyperlapse. The same process also turns a mountain climb into a smooth pan across a cliff side.
Smooth as butter
Researchers Johannes Kopf, Michael Cohen and Richard Szeliski noted that first-person video cameras are great at capturing interesting footage with activities such as rock climbing or biking, but would be “dead boring” when watched at normal speeds.
They found that increasing the video speed by 10x – known as subsampling – did not give them the results they wanted. Instead, it created a very shaky video that amplified every moment. When they tried to stabilize the video, they found that they did not have that much information. So, they created an algorithm that involved a three-step process in producing really stable videos.
What the hyperlapse method does is it looks at the footage in order to create a 3D path. This path is based on the movement made by the person wearing the camera in the scene.
The algorithm then uses the information in order to trace a smooth path. At the same time, it also stitches together multiple images into new frames into this stabilized path.
This is not saying that every converted video results perfectly. No. The footage that comes out after the algorithm is applied is not perfect. For example, you get instances where a person appears in one frame then disappears in the next one. The way the images are stitched together causes details to appear in a way that is unnatural.
However, head-mounted recording gear is set to be the future of the wearable gadgets world. And with the algorithm created by software engineers at Microsoft, users can expect to see much smoother first-person videos.
Now, the hyperlapse tool that Microsoft engineers have created exists is still a research project. However, the team is looking to release a Windows app soon.