Jun 01 2013

How to download all files of certain type from a web page in Linux

Published by at 9:08 am under Internet,linux,Programación

I’ve been trying to download all the mp3 files of a page (only one url, not the entire website) with wget but I’ve not achieved it because the mp3 files are in different domain than the main url so I’ve made a little script to download all of them in one command.

I’ve used lynx to extract all the links of the page and xargs to create a thread pool to have 4 simoultaneous downloads at the same time. The command is the following:

lynx -dump -listonly -nonumbers http://www.example.com/index.html | grep "\.mp3" | xargs -P 4 -I URL wget URL

2 responses so far

2 Responses to “How to download all files of certain type from a web page in Linux”

  1. CarlosTwon 03 Jun 2013 at 12:28 am

    Yo lo que uso para este tipo de cosas es DownThemAll, un plugin para Firefox. Permite elegir el número de descargas simultáneas y el número de conexiones por fichero.

    La versión fácil para usuarios no avanzados 🙂

  2. jvron 04 Jul 2013 at 6:20 pm

    Gracias Carlos! lo probaré para la próxima vez, aunque por comandos es mucho más divertido 😉

Trackback URI | Comments RSS

Leave a Reply