Mugen - HTTP for Asynchronous Requests¶

Mugen is library for http asynchronous requests.

Only running on python 3.4.0+

ok, code demo:

import asyncio
import mugen

async def task():
    url = 'http://www.google.com'
    resp = await mugen.get(url)
    print(resp.text)

loop = asyncio.get_event_loop()
asyncio.ensure_future(task())
loop.run_forever()

Mugen is a name from Samurai Champloo (サムライチャンプル, 混沌武士)

Feature Support¶

Keep-Alive & Connection Pooling
DNS cache
Sessions with Cookie Persistence
Automatic Decompression
Automatic Content Decoding
HTTP(S) Proxy Support
Connection Timeouts

The User Guide¶

Installation¶

Pip Install Requests¶

Mugen is running on Python ^3.7, installing with pip3:

$ pip3 install mugen

Install from Source Code¶

Mugen is developed on GitHub. You can find here.

Clone the repository:

$ git clone git://github.com/PeterDing/mugen.git

or, download the tarball:

$ curl -OL https://github.com/PeterDing/mugen/tarball/master

Install it:

$ python3 setup.py install

Quickstart¶

Following is simple demo to show you how mugen works.

Make a Request¶

To make a asynchronous request with mugen, firstly, you must import the standard library asyncio and mugen:

import asyncio
import mugen

OK, let us define a coroutine function as an asynchronous task, so asyncio can be use it. The task will make an asynchronous request. Finally, We give the task to one asynchronous loop which is generated by asyncio:

async def task():
    resp = await mugen.get('http://httpbin.org/headers')
    print(resp.text)

loop = asyncio.get_event_loop()
asyncio.ensure_future(task())
loop.run_forever()

Here, we get a Response object called resp after the request is over. resp contains all the information of the request.

Warning

We will ignore loop define, after above code, as you has known it.

We can also make a POST request, simply as following:

resp = await mugen.post('http://httpbin.org/post', data='fooboo')

Now, mugen supports HTTP request types: GET, POST, HEAD. Other methods is on the load.

Parameters in URLs¶

Like requests, Passing Parameters to urls is easy.

params = {'foo': 'boo'}
resp = await mugen.get('http://httpbin.org/post', params=params)

Response Content¶

We can read the content of the server’s response.

resp = await mugen.get('https://api.github.com/events')
print(resp.content)

resp.content is the origin bytes of the response (decoded if transfer-encodings is gzip or deflate), we can get its unicode string by resp.text which decoded by resp.encoding. If resp.encoding is wrong, setting it to the right encoding.

JSON Response Content¶

Handling JSON response content, mugen use standard library json.

resp = await mugen.get('https://api.github.com/events')
print(resp.json())

POST a Multipart-Encoded File¶

Now, mugen does not support post a multipart-encoded file.

Response Status Codes, Headers, Cookies, redirection¶

The status code of one response can be get by Request.status_code:

resp = await mugen.get('https://api.github.com/events')
print(resp.status_code)

The headers of the response is at resp.headers, cookies is resp.cookies and redirecting history locates resp.history.

Timeouts¶

Setting second timeout to a request:

resp = await mugen.get('https://api.github.com/events', timeout=10)

Connection Pool¶

Mugen uses only one global connection pool to manage all connections.

We assume that ALL connections will be used more then once. So, when we use loop.run_until_complete(task()) to start the loop, at the end of loop, ALL connections will not be closed automatically. This could raise a exception.

If we just want to use one connection only once, please, setting recycle=False to the request:

import asyncio
import mugen

async def task():
    url = 'http://www.google.com'
    resp = await mugen.get(url, recycle=False)
    print(resp.text)

loop = asyncio.get_event_loop()
loop.run_until_complete(task())

Sessions¶

Mugen supports session like other http library.

session = mugen.session()
resp = await session.get('http://www.google.com')

mugen.session() makes a Session object session for us, using to handle parts of requests.

Proxies¶

Mugen supports HTTP and HTTPS proxy. But it doesn’t like requests.

# for http proxy
proxy = 'http://127.0.0.1:8080'
resp = await mugen.get('http://www.google.com', proxy=proxy)

# for https proxy
proxy = 'http://127.0.0.1:8080'
resp = await mugen.get('https://www.google.com', proxy=proxy)

SOCKS¶

New in version 0.4.0.

Mugen also supports SOCKS5 protocol proxy from version 0.4.0.

# for SOCKS5 proxy
proxy = 'socks5://127.0.0.1:1080'
resp = await mugen.get('https://www.google.com', proxy=proxy)

# or
proxy = 'socks5://username:password@host:port'
resp = await mugen.get('https://www.google.com', proxy=proxy)