tomputils.downloader

tomputils.downloader

A simple segmenting downloader.

license:CC0 1.0 Universal http://creativecommons.org/publicdomain/zero/1.0/
tomputils.downloader.fetch(req_url, output=None)[source]

Fetch a single URL using default settings.

Parameters:req_url (unicode or str) – URL to request. File will be written to teh current working directory.
class tomputils.downloader.Downloader(max_retry=5, min_seg_size=16384, max_con=4)[source]

Bases: future.types.newobject.newobject

Download a file, possibly in segments.

Parameters:
  • max_retry (int, optional) – Maximum attempts that will be made to retrieve a segment.
  • min_seg_size (int, optional) – Largest file size, in bytes, that will not trigger segmenting.
  • max_con (int, optional) – Maximum number of concurrent connections to the remote server.
fetch(req_url, output=None)[source]

Fetch a file.

Parameters:
  • req_url (str) – URL of the file to retrieve
  • output (str, optional) – filename, possibly with path, of the downloaded file.
  • TODO (test can_segment == false) –