CrawlSpider v2¶
Introduction¶
TODO: introduction
Rules Matching¶
TODO: describe purpose of rules
Request Extractors & Processors¶
TODO: describe purpose of extractors & processors
Examples¶
TODO: plenty of examples
Reference¶
CrawlSpider¶
TODO: describe crawlspider
-
class
scrapy.contrib_exp.crawlspider.spider.CrawlSpider¶ TODO: describe class
Rules¶
TODO: describe spider rules
-
class
scrapy.contrib_exp.crawlspider.rules.Rule¶ TODO: describe Rules class
Request Extractors¶
TODO: describe extractors purpose
-
class
scrapy.contrib_exp.crawlspider.reqext.BaseSgmlRequestExtractor¶ TODO: describe base extractor
-
class
scrapy.contrib_exp.crawlspider.reqext.SgmlRequestExtractor¶ TODO: describe sgml extractor
-
class
scrapy.contrib_exp.crawlspider.reqext.XPathRequestExtractor¶ TODO: describe xpath request extractor
Request Processors¶
TODO: describe request processors
-
class
scrapy.contrib_exp.crawlspider.reqproc.Canonicalize¶ TODO: describe proc
-
class
scrapy.contrib_exp.crawlspider.reqproc.Unique¶ TODO: describe unique
-
class
scrapy.contrib_exp.crawlspider.reqproc.FilterDomain¶ TODO: describe filter domain
-
class
scrapy.contrib_exp.crawlspider.reqproc.FilterUrl¶ TODO: describe filter url
Request/Response Matchers¶
TODO: describe matchers
-
class
scrapy.contrib_exp.crawlspider.matchers.BaseMatcher¶ TODO: describe base matcher
-
class
scrapy.contrib_exp.crawlspider.matchers.UrlMatcher¶ TODO: describe url matcher
-
class
scrapy.contrib_exp.crawlspider.matchers.UrlRegexMatcher¶ TODO: describe UrlListMatcher
-
class
scrapy.contrib_exp.crawlspider.matchers.UrlListMatcher¶ TODO: describe url list matcher