On my blog I wrote a few articles that explain step-by-step how the code of this program is set up.
- Defining items in scrapy
- Using pipelines in scrapy
- Creating a spider with scrapy
- Using extensions in scrapy
- Avoding IP banning
For further questions you can comment on the blog posts or contact me via Twitter (@bertcarremans)