Random ...
 
December 2017
S M T W T F S
          1 2
3 4 5 6 7 8 9
10 11 12 13 14 15 16
17 18 19 20 21 22 23
24 25 26 27 28 29 30
31            
Tags ...
Links



python+zope++:: A video site with zope, mencoder, ffmpeg, flowplayer and lighttpd
Posted at 17.Sep,2008 22:01  Comments 2 / Trackbacks 0 / Like this post!
Technorati tag(s):

In a jiffy. We're going to celebrate Eid al Fitri end of this month.

It's a tradition here in Malaysia that those abroad or away return home; where ever home may be. To many, it's where their parents are.

Some may not be able to return and thus sends greeting cards, videos, etc.

That's where this story came about. We were asked to build a greeting site that will accept videos and display the videos for all to see.

The catch is we have to do it in a jiffy. So, please excuse the plain site.

What we did:

  • create a zope product that stores the video properties, and add the video to the file system
  • have a python script poll a directory and process the video files using mencoder (transcode to a flv), ffmpeg (take a snapshot of the splash page) and copy the flv so that lighttpd will serve the flv.

Since the site owner wants the site yesterday, we could not do a proper stress test.

But we did manage to do apache bench (ab).

Here's the result:

  # ab -n 500 -c 150 http://salamperantau.my/sp/
 This is ApacheBench, Version 2.0.41-dev <$Revision:  1.121.2.12 $> apache-2.0
 Copyright (c) 1996 Adam Twiss, Zeus Technology Ltd,  http://www.zeustech.net/
Copyright (c) 1998-2002 The Apache Software  Foundation, http://www.apache.org/

 Benchmarking salamperantau.my (be patient)
 Completed 100 requests
 Completed 200 requests
 Completed 300 requests
 Completed 400 requests
 Finished 500 requests

 Server Software:        Zope/(Zope
 Server Hostname:        salamperantau.my
 Server Port:            80

 Document Path:          /sp/
 Document Length:        28515 bytes

 Concurrency Level:      150
 Time taken for tests:   29.434537 seconds
 Complete requests:      500
 Failed requests:        0
 Write errors:           0
 Total transferred:      15466896 bytes
 HTML transferred:       15328321 bytes
 Requests per second:    16.99 [#/sec] (mean)
 Time per request:       8830.361 [ms] (mean)
 Time per request:       58.869 [ms] (mean, across all concurrent requests)
 Transfer rate:          513.14 [Kbytes/sec] received

 Connection Times (ms)
              min  mean[+/-sd] median   max
 Connect:       34 1097 1602.6    118    9125
 Processing:   162 4971 4686.2   3366   27697
 Waiting:       62 1368 3011.3    208   15803
 Total:        385 6069 5148.0   4484   27752

 Percentage of the requests served within a certain time (ms)
  50%   4484
  66%   6832
  75%   8817
  80%   9926
  90%  13480
  95%  16129
  98%  20260
  99%  23103
 100%  27752 (longest request)

 # ab -n 1500 -c 550 http://salamperantau.my/sp/
 This is ApacheBench, Version 2.0.41-dev <$Revision:  1.121.2.12 $> apache-2.0
 Copyright (c) 1996 Adam Twiss, Zeus Technology Ltd,  http://www.zeustech.net/
Copyright (c) 1998-2002 The Apache Software  Foundation, http://www.apache.org/

 Benchmarking salamperantau.my (be patient)
 Completed 150 requests
 Completed 300 requests
 Completed 450 requests
 Completed 600 requests
 Completed 750 requests
 Completed 900 requests
 Completed 1050 requests
 Completed 1200 requests
 Completed 1350 requests
 Finished 1500 requests

 Server Software:        Zope/(Zope
 Server Hostname:        salamperantau.my
 Server Port:            80

 Document Path:          /sp/
 Document Length:        28515 bytes

 Concurrency Level:      550
 Time taken for tests:   93.437926 seconds
 Complete requests:      1500
 Failed requests:        18
   (Connect: 0, Length: 18, Exceptions: 0)
 Write errors:           0
 Total transferred:      48610908 bytes
 HTML transferred:       48155900 bytes 
 Requests per second:    16.05 [#/sec] (mean)
 Time per request:       34260.573 [ms] (mean)
 Time per request:       62.292 [ms] (mean, across all concurrent requests)
 Transfer rate:          508.05 [Kbytes/sec] received

 Connection Times (ms)
              min  mean[+/-sd] median   max
 Connect:        6 1846 2714.0    131   24625
 Processing:   273 21232 19422.3  14800   91218
 Waiting:    -1221655310235 -2147483648  83287756765.2    566   84931
 Total:        353 23079 19735.5  17220   93121

 Percentage of the requests served within a certain time (ms)
  50%  17220
  66%  27006
  75%  34304
  80%  38438
  90%  52617
  95%  64155
  98%  75835
  99%  82032
 100%  93121 (longest request)

 # ab -n 1000 -c 900 http://salamperantau.my/sp/vsection?category=Profesional
 This is ApacheBench, Version 2.0.41-dev <$Revision:  1.121.2.12 $> apache-2.0
Copyright (c) 1996 Adam Twiss, Zeus Technology Ltd,  http://www.zeustech.net/
 Copyright (c) 1998-2002 The Apache Software   Foundation, http://www.apache.org/

 Benchmarking salamperantau.my (be patient)
 Completed 100 requests
 Completed 200 requests
 Completed 300 requests
 Completed 400 requests
 Completed 500 requests
 Completed 600 requests
 Completed 700 requests
 Completed 800 requests
 Completed 900 requests
 Finished 1000 requests

 Server Software:        Zope/(Zope
 Server Hostname:        salamperantau.my
 Server Port:            80

 Document Path:          /sp/vsection?category=Profesional
 Document Length:        22624 bytes

 Concurrency Level:      900
 Time taken for tests:   60.998529 seconds
 Complete requests:      1000
 Failed requests:        0
 Write errors:           0
 Total transferred:      31148705 bytes
 HTML transferred:       30732257 bytes
 Requests per second:    16.39 [#/sec] (mean)
 Time per request:       54898.675 [ms] (mean)
 Time per request:       60.999 [ms] (mean, across all  concurrent requests)
 Transfer rate:          498.67 [Kbytes/sec] received

 Connection Times (ms)
              min  mean[+/-sd] median   max
 Connect:        6 3711 3617.8   3347   24709
 Processing:    75 21388 14940.2  18707   57504
 Waiting:       27 6282 10214.6   2526   44927
 Total:         85 25100 16103.4  23186   60869

 Percentage of the requests served within a certain time (ms)
  50%  23186
  66%  30877
  75%  36829
  80%  39896
  90%  49676
  95%  54790
  98%  58847
  99%  59815
 100%  60869 (longest request)

From the result, it doesn't matter how hard we hit it (increase # of requests and concurrency), we will always get about 16 requests/s.

There were a few failed requests though, especially when the concurrency was high.

I just hope the site can stand traffic, if ever it spikes :P

If ever we need to scale, we might:

  • chuck in zope clients and start using zeo
  • use rsync to sync the flv files across
  • have keepalived loadbalance requests

Bookmark and Share

Is this entry helpful? Comments/Donate/Click some google ads.  
Trackback is http://myzope.kedai.com.my/blogs/kedai/206/tbping 

Comments
Lulz

no 3 at Google search engine, you deserve some clicks.

bazet  at 24.Sep,2008 08:38
hehe

wuz at #1.

kedai  at 25.Sep,2008 23:21
Post a comment