6. geventtools

6.1. geventtools

geventtools.gevent_pool(func, spawn_list, pool_size=5)[source]
Parameters:
  • func (object) – main run function
  • spawn_list (list) – a list of midfunc()
  • pool_size (int) – gevent pool concurrency
geventtools.midfunc(*args, **kwargs)[source]
Return type:tuple
Returns:(args, kwargs)

6.2. How to use

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
import urllib2
from datetime import datetime

def fetch_data(url):
    print datetime.now(), 'Start', url
    result = urllib2.urlopen(url)
    print datetime.now(), 'Fetch OK', url
    print url, result
    return result.info()

url = [
        midfunc('http://www.google.com.tw/'),
        midfunc('http://toomore.net/'),
        midfunc('http://pinkoi.toomore.net/'),
        midfunc(url='http://www.pinkoi.com/'),
      ]
t1 = datetime.now()
result = gevent_pool(fetch_data, url*20, 20)

for i in result:
    print i.get()
print result
print datetime.now() - t1

Table Of Contents

Previous topic

5. awscloudfronttools

Next topic

7. Some scripts.

This Page