Comments (3)
Can you tell me what is the page you are trying to scrape so it helps me to digg down the problem?
from wombat.
Ok, no problem
require 'wombat'
class TelelistaScraper
include Wombat::Crawler
base_url "http://www.telelistas.net/br/restaurantes"
path "/?pagina=9"
some_text xpath: '//*[@id="Content_Regs"]/table[1]/tbody/tr/td[2]/table/tbody/tr[4]/td[2]/text()[2]'
end
puts TelelistaScraper.new.crawl
from wombat.
I suppose they are using some kind of javascript hack to avoid being scraped
1.9.3-p194 :018 > Nokogiri::HTML("http://www.telelistas.net/br/restaurantes/?pagina=9")
=> #<Nokogiri::HTML::Document:0x3ffeed8eb83c name="document" children=[#<Nokogiri::XML::DTD:0x3ffeed8f04b8 name="html">, #<Nokogiri::XML::Element:0x3ffeed8ef16c name="html" children=[#<Nokogiri::XML::Element:0x3ffeed8f2b3c name="body" children=[#<Nokogiri::XML::Element:0x3ffeed8f26dc name="p" children=[#<Nokogiri::XML::Text:0x3ffeed8f1f5c "http://www.telelistas.net/br/restaurantes/?pagina=9">]>]>]>]>
1.9.3-p194 :019 > html.inner_html
=> "<html><body><p>http://www.telelistas.net/br/restaurantes/?pagina=9</p></body></html>"
from wombat.
Related Issues (20)
- Is it possible to programmatically create the properties to search for? HOT 2
- Cannot run the example code. HOT 4
- How to get image src link HOT 2
- Support many proxy server
- How to go over multiple subpaths
- local file support HOT 3
- Get url of followed link HOT 3
- Clearing cookies HOT 1
- `set_proxy` not working HOT 1
- Use more strict dependencies
- Dynamically set base_url and path. HOT 2
- How do i access the siblings using css selector in wombat
- customize http header
- page function is not work
- Allow hash selector as last argument to property
- Image provision
- How do I remove a node?
- Documentation links are incorrect
- LoadError in sample code → mechanize.rb: cannot load net/http/digest_auth
- pagination infinite scroll HOT 1
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from wombat.