Here is a list of tips and advice on using Firefox for scraping, along with a list of useful Firefox add-ons to ease the scraping process.
<tbody> elements to tables. Scrapy, on the other hand, does not modify the original page HTML, so you won’t be able to extract any data if you use
<tbody in your XPath expressions.
Therefore, you should keep in mind the following things when working with Firefox and XPath:
- Never include
<tbody>elements in your XPath expressions unless you really know what you’re doing
See Using Firebug for scraping for a detailed guide on how to use Firebug with Scrapy.