This Crawler is for crawl an online shop Rakitan. It will save the item name, link, categories and price in MySQL.
-
Clone this repository
git clone https://github.com/clasense4/scrapy-rakitan.git
-
Edit
scrapy_rakitan/settings.py
change yourscrapy
andMySQL
setting -
Insert this SQL query :
CREATE TABLE `rakitan` ( `IID` int(11) NOT NULL AUTO_INCREMENT, `Hash` text NOT NULL, `Item_name` text NOT NULL, `Item_price` text NOT NULL, `Item_category` text NOT NULL, `Item_link` text NOT NULL, `timestamp` timestamp NOT NULL DEFAULT CURRENT_TIMESTAMP, PRIMARY KEY (`IID`) ) ENGINE=InnoDB AUTO_INCREMENT=6642 DEFAULT CHARSET=latin1
-
Start your crawler with this command
$> scrapy crawl scrapy_rakitan
-
At
04 October 2013 : 4.00 AM
, this script give me6641 Items
.
The script is still sucks, just for fun, not follow scrapy standards, use at your own risks.
mail me at clasense4[at]gmail[dot]com