To cache Youtube videos using Squid proxy server so that subsequent requests for the same video can be served from the cache saving the loading time and some bandwidth.
Use Cases
To reduce the loading time for the subsequent requests on a shared network.
To save some bandwidth which is wasted in serving the same video again and again.
When the clients on your proxy are in thousands and they love browsing youtube all the time (universities are typical examples ), the 2nd factor becomes an issue.
When you don’t want to hack the refresh_pattern in squid and don’t want to disturb the dynamic content.
How to proceed
I tried a lot of squid hacks listed all around the web. But none of them seems to work for me. And even if they work, the weird load balancing system of youtube servers result in weird behaviors. I was working on my GSOC project which is also related to caching but of RPMs, when I got this idea of caching youtube videos using custom squid redirectors. Sometime back, I posted a tutorial on how to write custom squid redirectors in Python. I extended that to write a squid plugin to cache youtube videos which I call youtube_cache.
Method of operation for caching
Squid gets a request for a url from the client.
youtube_cache taps the url
CHECK: if url is .youtube.com and has a get_video request
Extract the 11 character long unique video id ( this is different for all the videos on youtube)
CHECK: if video is already in cache
CHECK: if cached video is newer or same as remote video (last-modified timestamp)
Serve from cache and exit
ELSE: Download and cache the new video in background ( using daemon forking )
ELSE: Download and cache the new video in background ( using daemon forking )
ELSE: url is not from any of the youtube servers. Let squid handle it
Tags:balancing system, proxy server, squid, video request, Youtube caching, youtube videos
November 26th, 2008 at 1:26 pm
I have written a squid url rewriter plugin in python to handle caching from 10 websites. Please check http://cachevideos.com/.
January 15th, 2009 at 8:13 am
How to auto clear cache on Squid
June 30th, 2010 at 3:53 pm
Did you talk about how to configure it. But you don’t show your youtube_cache. Can you send it to me?
Or explain better how to do a plugin to Squid? I’m searching but I didn’t found anything about cache plugin to it.
October 26th, 2010 at 5:11 pm
[...] Note with Squid Proxy you’ll need a third party module to cache YouTube [...]
December 24th, 2010 at 4:00 pm
best result of video cache in production level is thundercache 3.0 witch is very simple and use full. wit squid u can use it as put some lines to just filter and redirect the youtube requestes to an other service as thundercache to handle not by squid.