Ruby Mechanize is a brilliant gem. Love it. But the documentation is sparse and you’ll find yourself having to poke and prod objects to coerce what you need out of them when things gone wrong. To make your life easier (and as crib notes for myself) here are 3 tips that made my life easier.
1. User Agent List
There are 14 default user agents available, but I wasn’t able to readily find them in the documentation. Open up irb and run the following command:
require 'mechanize' pp Mechanize::AGENT_ALIASES
You’ll get a nice hash printout with the following list:
{
"Mechanize" => "Mechanize/2.3 Ruby/1.9.2p290 (http://github.com/tenderlove/mechanize/)",
"Linux Firefox" => "Mozilla/5.0 (X11; U; Linux i686; en-US; rv:1.9.2.1) Gecko/20100122 firefox/3.6.1",
"Linux Konqueror" => "Mozilla/5.0 (compatible; Konqueror/3; Linux)",
"Linux Mozilla" => "Mozilla/5.0 (X11; U; Linux i686; en-US; rv:1.4) Gecko/20030624",
"Mac FireFox" => "Mozilla/5.0 (Macintosh; U; Intel Mac OS X 10.6; en-US; rv:1.9.2) Gecko/20100115 Firefox/3.6",
"Mac Mozilla" => "Mozilla/5.0 (Macintosh; U; PPC Mac OS X Mach-O; en-US; rv:1.4a) Gecko/20030401",
"Mac Safari 4" => "Mozilla/5.0 (Macintosh; U; Intel Mac OS X 10_6_2; de-at) AppleWebKit/531.21.8 (KHTML, like Gecko) Version/4.0.4 Safari/531.21.10",
"Mac Safari" => "Mozilla/5.0 (Macintosh; Intel Mac OS X 10_7_2) AppleWebKit/534.51.22 (KHTML, like Gecko) Version/5.1.1 Safari/534.51.22",
"Windows IE 6" => "Mozilla/4.0 (compatible; MSIE 6.0; Windows NT 5.1)",
"Windows IE 7" => "Mozilla/4.0 (compatible; MSIE 7.0; Windows NT 5.1; .NET CLR 1.1.4322; .NET CLR 2.0.50727)",
"Windows IE 8" => "Mozilla/5.0 (compatible; MSIE 8.0; Windows NT 5.1; Trident/4.0; .NET CLR 1.1.4322; .NET CLR 2.0.50727)",
"Windows IE 9" => "Mozilla/5.0 (compatible; MSIE 9.0; Windows NT 6.1; Trident/5.0)",
"Windows Mozilla" => "Mozilla/5.0 (Windows; U; Windows NT 5.0; en-US; rv:1.4b) Gecko/20030516 Mozilla Firebird/0.6",
"iPhone" => "Mozilla/5.0 (iPhone; U; CPU like Mac OS X; en) AppleWebKit/420+ (KHTML, like Gecko) Version/3.0 Mobile/1C28 Safari/419.3"
}
2. Logs
Watching the logs is incredibly helpful when things go wrong. You can attach a logger to mechanize and redirect the output to STDOUT
require 'logger' @agent.log = Logger.new(STDOUT)
With the logs in place you’re able to see the full extend of what gets POSTed to a server, or which url is being used for a GET request.
3. Submit isn’t Submitted
The submit button doesn’t actually get submitted, kind of contrary to what the docs say:
Submit this form with the button passed in
My first time through I read that as “the submit button gets submitted with the form”. However it doesn’t. And it took a long time to figure that out. It’s an easy enough problem to solve, I simply added the submit button as a field on my form:
@agent.get(url) do |page|
page.form_with(id: 'the-form') do |form|
# Other form fields
# Required otherwise form submission fails silently
form["submit"] = 'submit'
end.submit
end
The other alternative to the above code is to use the click_button method which will automatically include the button with the form submission.
Any other tricks you have for working with Mechanize? Post them in the comments.
