Serving videos through nginx

asked 2013-03-25 19:00:56 -0500

ykim gravatar image

Hello,

We are interested in using Swift as a solution to our media storage. However, as our current setup relies on nginx handling video serving, we were wondering if there is an easy way for Swift to use nginx to handle file serving. From documentations I've seen, it would see that I could use install a wsgi module into nginx then configure it's mp4 module to directly handle the mp4 serving. I'm not sure if this is the correct method, so I'd like to ask if anyone here has already tried something similar to this.

Thanks, Young

edit retag flag offensive close merge delete

8 answers

Sort by ยป oldest newest most voted
0

answered 2013-03-27 02:55:40 -0500

redbo gravatar image

hmmm... I wonder if you could run nginx on the object servers and figure out how to make sure the proxy forwards any information needed for pseudostreaming down to them.

One problem with nginx is that it usually buffers entire requests, even if they're a 5GB upload. I don't know if that's true for when uwsgi is behind it.

edit flag offensive delete link more
0

answered 2013-03-26 08:52:48 -0500

clay-gerrard gravatar image

So I'm definitely a novice at streaming media, but interested in how swift could perform as a media server. Maybe you could start with educating me a bit on your requirements and and I'll try to offer what I can WRT how swift supports pseudo streaming.

I started by installing 'nginx-extras' on my precise box and setup a location for /video/with a mp4 directive and a couple of mp4's. To my surprise Chrome and Safari's built in players both used native range requests (instead of the start=0 query params) and I was able to watch the videos (and jump around) even when serving them directly out of /usr/share/nginx/www.

After that i was less surprised when pseudo-streaming "just worked out of the box" with swift, after I uploaded the files into swift, added .r:* to my video container and hit swift directly from the web-browsers.

However, I'm guessing that some older browser's won't have built in players, or some sites may require a specific media player that doesn't support native range requests.

I did some tests with jwplayer using the following html snippet uploaded into the same container next to the video:

<html>

<script type="text/javascript" src="http://www.longtailvideo.com/jwplayer/jwplayer.js"></script> <script type="text/javascript" src="http://code.jquery.com/jquery-1.9.1.min.js"></script> <script type="text/javascript"> $(document).ready(function() { jwplayer("myElement").setup({ file: "movie.mp4", }); }); </script>

<body>

</body> </html>

Not matter what I set (or didn't set) for startparam (see http://www.longtailvideo.com/support/jw-player/29392/flash-pseudo-streaming/ (http://www.longtailvideo.com/support/...) ), I couldn't verify it was doing anything other than normal byterange requests.

Any suggestion how I can get a browser old enough that it doesn't support native byterange psudeo streaming onto my mac? Do you think your intended audience is going to have HTML5 capable browsers?

That said, if we can find a test senario that requires support for startparam style requests, we could try to make something work with nginx just proxying requests to a local swift-proxy bound to loopback. You'd probably still want general swift traffic (i.e. uploads) to go directly into a swift proxy server (either running on the same box/different interface or another machine entirely).

I'm not sure how successful we'd be serving swift's proxy application inside the wsgi module for nginx. You could try it though!

Another option would be to implement some middleware that can request the the first few bytes of the video and read/cache the keyframe offsets - although a quick search seems to indicate that there may not be a widely adopted python library which can parse seekpoint metadata - but I'd be glad to know if one exists?

edit flag offensive delete link more
0

answered 2013-03-26 18:42:35 -0500

ykim gravatar image

Thanks for the quick reply Clay!

In regards to our requirements, we're looking into serving MP4 to multitude of clients, notably web (i.e. Flowplayer or the equivalent), Android, and other modern platforms. I believe that most of these clients implement pseudo-streaming by seeking the MP4 through seconds or byte ranges. Nginx is capable of parsing the MP4's atom (i.e. the metadata with this information) to quickly seek, buffer, and return the requested segments. I'm not entirely too sure if Swift is capable of this optimization natively, which is why I ask this question.

I'm not entirely too sure on how well using the wsgi module for nginx would be since it seems like nginx's mp4 serving module needs a direct contact with the file itself. I was planning on testing this out to see how well it'd work, but I'm not entirely too sure on the internals of Swift, so I'd figure I ask first to see if someone had tried this approach.

As for the middleware approach, this seems to be the best approach, but as you pointed out, there's a lack of python libraries to parse the metadata. :/

edit flag offensive delete link more
0

answered 2013-04-02 22:19:03 -0500

ykim gravatar image

Quick update here for Clay: We found a test scenario. In particular, the flash version of Flowplayer that we're currently using requires the enabling of start parameters for proper seeking, as it is not capable of byte-range requests.

As I'm not too entirely familiar with Swift, am I correct in thinking that it might be troublesome to get nginx's MP4 module working with Swift's Object serving? It seems to me that implementing a middleware would be the more correct option, but I can't seem to find any resources detailing how middlewares work and are created.

edit flag offensive delete link more
0

answered 2013-04-02 22:43:17 -0500

torgomatic gravatar image

Swift middleware is just WSGI middleware.

edit flag offensive delete link more
0

answered 2013-04-03 00:17:34 -0500

clay-gerrard gravatar image

Wow, so I just could not get Flowplayer to make byte range requests.

It says it can if you jump through enough hoops: http://flash.flowplayer.org/plugins/streaming/pseudostreaming.html (http://flash.flowplayer.org/plugins/s...)

... but you have to use flv instead of mp4?

Here's as close as I could get:

<html>

<script type="text/javascript" src="http://releases.flowplayer.org/js/flowplayer-3.2.12.min.js"></script> <script type="text/javascript" src="http://code.jquery.com/jquery-1.9.1.min.js"></script> <script type="text/javascript"> $(document).ready(function() { $f("myElement", "http://releases.flowplayer.org/swf/flowplayer-3.2.16.swf", { debug: true, clip: { url: 'movie.mp4', provider: 'swift' }, plugins: { swift: { url: "http://releases.flowplayer.org/swf/flowplayer.pseudostreaming-3.2.12.swf", rangeRequests: true } } }); }); </script>

<body> </body> </html>

Having dealt with exactly two video players for almost a hour total I feel totally qualified to suggest you'd be happiest just switching off of Flowplayer :P

But seriously, I'm sure it's a nice product, or you might be tried to it for other reasons; either way you might be better off contacting their support first and seeing if that have a solution or workaround for native byte range requests for mp4's. Lots of people want a for fix this:

http://stackoverflow.com/questions/14028190/amazon-cloudfront-and-flowplayer-pseudo-stream-by-https-byterange (http://stackoverflow.com/questions/14...)

https://code.google.com/p/flowplayer-core/issues/detail?id=145 (https://code.google.com/p/flowplayer-...)

I'd have to assume it'd be easier to patch the client that already halfway supports a psueudo-streaming than writing a custom middleware that can get "enough" bytes from an object request for the mp4 data to identify the byte offset of key-frame nearest the timeoffset given as the start param. Might be interesting, from a purely technical endeavor, but it seems that video streaming is moving to more native http when possible.

I don't even... I don't really know what redbo's talking about with the having nginx serving /objects from the storage node's directly and then something somewhere to ring something hand wave hand wave forward request? Maybe he'd want to elaborate...

edit flag offensive delete link more
0

answered 2013-04-03 04:52:17 -0500

ykim gravatar image

I just tried using Flowplayer like you did, and I reached the same conclusion also. Ideally, I'd like to also ditch using Flowplayer, but I believe some variants of Android also requires the use of the start parameters. :(

Everything seems to be pointing to making a middleware that is capable of doing exactly what nginx's mp4 module is doing. I searched around, and it seems like there is a library in python that is capable of parsing a mp4 for its atom ( https://pypi.python.org/pypi/mp4file ). I'm fine with actually coding this, but it would be nice to have a starting point of how I would go about writing middleware specifically for Swift.

edit flag offensive delete link more
0

answered 2013-04-03 15:34:35 -0500

clay-gerrard gravatar image

You can look in the swift source for some good examples of existing middleware:

https://github.com/openstack/swift/tree/master/swift/common/middleware (https://github.com/openstack/swift/tr...)

It's funny, cause I'm way more worried about the atom parsing than the WSGI middleware :P

I looked at mp4file ( https://github.com/billnapier/mp4file/blob/master/src/mp4file/atom.py (https://github.com/billnapier/mp4file...) ); it will be interesting to see how that library breaks down when applied to streams; I see a lot of .seek and .tell - but also a lot of read(4) and read(1) - so maybe you can just buffer a little bit in a StringIO or something... You might try and ping the author.

Feel free to catch me in #openstack-swift on Freenode, or email me directly if you need anymore help... I'll be curious see how things turn out. Good luck!

edit flag offensive delete link more

Get to know Ask OpenStack

Resources for moderators

Question Tools

1 follower

Stats

Asked: 2013-03-25 19:00:56 -0500

Seen: 563 times

Last updated: Apr 03 '13