This is a description of the new features that can be found in libevent 2.x. One of the main reasons for using it would be the inclusion of sendfile() operations (where the OS does all the work when sending large files from disk), and big performance improvements, specially in the HTTP code:
Preliminary results around April 2009 indicated that the rewritten codebase had improved our large-payload HTTP benchmark performance by between 35 and 174 percent. We hope that as we profile and benchmark this code more aggressively, we can find more opportunities for improvement here.
This has been confirmed by some other people but, looking at this recent survey on WSGI servers performance, it also seems there could be some memory leaks in libevent 2.x…
Filed under Networking, Programming and Technology. |
XtreemFS. It seems to have all the neccessary things for developing you own CDN at the OS level with minimum effort. Similar to Coda, but it gives you on-demand file replication (read-only, of course) on WAN environments, POSIX semantics (so you don’t need to modify you application) , stripping and parallel I/O (equals to high performance), etc.
So, what else do you need? Maybe a web server like cherokee or nginx for streaming (with throttling) flv and h264, and you are ready to have your low-cost CDN…
Filed under Media, Networking and Technology. |
An interesting article on video codecs licensing and how we are all being out of the law. The conclusion is very simple: Theora is a free standard we can all use for commercial purposes, we know, but just because it is not widely used!. Otherwise, legions of unknown patent holders would sue you right when you release the version 1.0 of your software. And why is it not so popular? Well, just because it is not up to the task… or at least not as good as h264. So we are stuck with h264 for a while…
Filed under Media and Technology. |
I´m using gevent for implementing a basic HTTP stresser and I must say I´m impressed with the performance I can get with this framework. My program launches thousands of HTTP requests (with urllib2) to my server, asking for very long files (>200MBytes), and I´m getting up to 2 Gbit/sec with almost no CPU usage in the client. Really good for a slow language like Python…
Filed under Networking and Technology. |
More on Python for networking: a performance comparison of different asynchronous frameworks, including Twisted and Tornado. It focuses on raw performance (it is just an HTTP echo server implemented in each platform, and tested with httpperf), so it ignores the features provided…
However, it has brought to my attention two frameworks based on libevent: Gevent and Concurrence. Gevent provides some bindings for libevent for Python, adding the use of greenlets to the equation. Concurrence goes one step further by using “Lightweight-tasks-with-message-passing approach to concurrency“, providing “an easier programming model for writing high performance network applications than existing solutions (Multi-threading, Twisted, asyncore etc).” Concurrence can be used with the greenlets library or with ‘Stackless Python’, a branch of the standard CPython that is supposed to give better performance on multi-threading programs.
All these frameworks looks really promising if you are looking for raw performance, but I stick with Twisted for my experiments. The huge number of protocols implemented by the standard distribution, its more-than-good documentation, and all the mailing lists and blogs, make Twisted a good platform for developing networking applications, even if there are other, faster alternatives on the market.
Filed under Networking and Technology. |
