Download all of humble bundle books in parallel

Humble Bundle is a great site which offers technical book bundles. The problem is that they present the user with a huge list of links for all the different formats and it is a tedious task to right click each link and save it to your hard drive.

In order to solve this you can open the Developer Tools while on the download page and paste the following:

var pattern = /(MOBI|EPUB|PDF( ?\(H.\))?|CBZ|Download)$/i;
var nodes = document.getElementsByTagName('a');
var downloadCmd = '';
for (i in nodes) {
    var a = nodes[i];
    if (a && a.text && pattern.test(a.text.trim()) && a.attributes['data-web']) {
        downloadCmd += 'wget --content-disposition "' + a.attributes['data-web'].value + "\"\n";
    }
}
var output = document.createElement("pre");
output.textContent = downloadCmd;
document.getElementById("papers-content").prepend(output);

credit: https://gist.github.com/graymouser/a33fbb75f94f08af7e36

This will add a pre tag to the page with a bunch of wget commands. Go ahead and copy those to your clipboard.

Using GNU Parallel (brew install parallel). First save the contents of your clipboard into a file, for example download_jobs then run the following command:

parallel -j 4 < download_jobs

Replace 4 with the number of cores you have on your machine.

Then sit back and watch your directory get populated with files.

Tweet