John Yerhot

Yes, Jen and I got some hot rods.. two wheeled hot rods with pedals…

We are now the proud owners of two Specialized bikes.   Jen has a Cross Roads and I got a Hard Rock.  It’s by far the nicest bike I’ve ever ridden, though both are used and a couple years old and not top of the line by hardcore biker standards.

Anyways, we’ve been riding all over our new neighborhood and I have to say it is really fun.  We’ve set a goal of riding (most of) the Munger Trail by the end of the summer.  Sweet.  Maybe we’ll even buy some of those spandex jump suits to bike around in.

Jen’s got the purple-ish one and I have the white one.

It has been a very long, but very fun day. We’re both pretty jet lagged and turning in for the day but some tthings we’ve noticed about the fine citizens of Seattle:

• They do not jaywalk.
• Pretty good and nice drivers
• They park very close to each other. Like an inch.
   

Heres some pictures from the day.  Tomorrow I think we are going to go and see Mt. Rainier… more updates to follow.

Two things…

First - Jen and I are taking a trip to Seattle for 7 days starting this Saturday.  We’re super pumped.  

Second - Over Christmas last year I started on this lil Rails app I was calling duluthrentalreview.com.  Pretty obvious what it was.  I get so pissed at landlords in this town, and even more pissed at my fellow renters for putting up with the crap quality of said rentals.  The problem is this:  Landlords in this town do not keep up their rentals because renters do not demand that they are nice and decent and that the landlords keep them up.  You get the idea. 

Ok, so yeah I started writing this rental review site over christmas, got like halfway though it and just didn’t have time to finish it.. till now.  I re-wrote almost all of it, convertging it to REST.   I’m gonna try to continue updating it over the next couple days till our trip.  Oh, ad the url changed from duluthrentalreview.com to rentalrundown.com… liked the sound of rentalrundown better.

Yesterday we moved into our new apartment.  The day was ‘interesting’.

• Got up at 6:30am to get the UHaul at 7:30.
• There was a line of people at UHaul and only 1 guy working.
• Got the UHaul at around 8:15 and had to return it at 2pm.
• Loaded box after box into the UHal and by 10:30 had most of the big stuff in and the UHaul was full.
• Dumped it all off at the new place, met the neighbors (and I was super sweaty had crazy hair from moving)
• Jen’s friend Anita came over to help unload right when the last box got unloaded, but helped go get the new couch.
• by this time it was almost 1pm and we had to get the UHaul back. 
• Dropped it off, ate at good old McDonalds and talked about how sore we all were from lifting all that stuff.
• I went back to the old apt to clean and get any last stuff out. Realized there were 2 closets we never boxed up.  Crap.
• Took 2 car loads of stuff to the new apartment.
• Took a break from 5 - 7 to eat dinner and not die of exhaustion.
• Got back to the old apartment at 7:15 to find some girl sitting on the stairs along with tons of furniture.. my bullshit alarms started to go off.  I asked if she needed help, she said ‘no’.  As I walked up the stairs at to our apartment, she asked if it was our stuff in there.  I said ‘yep’ we are finishing up cleaning.  ”Well, (our landlords) said you would be out and I could move in today…”, to which I replied, “I haven’t told them that and I paid through the end of the month”.
• Whipped out the phone called the land lords.
• They said they told her she could move in early if we called them and said we were out.  
• Ok the reasons this whole thins is BS are:
  • I’ve never seen this girl, Jen’s never seen this girl. They must have showed out apartment when we were not home and without or authorization. Under law they have to give 24 hours notice to enter the apartment.
  • The management never did a walk through.  They said they would do one after the new tenents moved in.  Umm.. no.  They would have no grounds to determine what, if any, damage was our fault.
  • There were  multiple maintenance requests that were never addressed.  I’m not understanding how they can let a new tenant move in with a drawer that has the front broken off cause the wood is starting to rot.
• The girl said that she would do the cleaning if we’d quick move what was left out and let her dump her stuff off.  What the hell I said.
• We did. We went to our new home.  We went to sleep.

What a day.  This morning I can barley move I’m so sore. 

Pics of the new apartment full of boxes….

Though using subdomains is all the rage right now, there are certianly instances where you may want to deploy your rails application to a subdirectory such as:

http://www.johnyerhot.com/myrailsapp

Of course Nginx makes it super easy to do so. If you need to get your webserver ready with Nginx, PHP running as a FCGI instance, and Rails check out my other how to.  

Now, onward!

First create a new virtual host.  In my case, for yerhot.org it would look like this:
server {
listen 80;
server_name yerhot.org;
access_log /var/www/yerhot.org/logs/access.log;
error_log /var/www/yerhot.org/logs/error.log;
location / {
root /var/www/yerhot.org/;
index index.html;
}
}
Pretty simple setup, telling Nginx to listen on port 80 for requests for yerhot.org, where to store logs, and finally setting up the site root at /var/www/yerhot.org.

All we would have to do to have Nginx redirect to our Rails app when looking for yerhot.org/myrailsapp is make another location block (in other words, place this right before the last curly brace).

location /myrailsapp {
proxy_pass http://localhost:8000;
}
Now, all requests for /myrailsapp will get proxied to port 8000. Now fire up your Rails app on port 8000.
mongrel_rails start -e production -p 8000 -d
Restart Nginx:
/etc/init.d/nginx stop
/etc/init.d/nginx start
And….

Crap. Rails is looking for a ‘myrailsapp’ route, which there is none.  No, no, don’t create one - we’ll need to use a little known feature of Mongrel to fix the problem.. the prefix.
Stop Mongrel…
mongrel_rails stop
And try this:
mongrel_rails start -e production -p 8000 -d --prefix=/myrailsapp

And… your app should fire right up. Pretty neat, if you wanted to you could use it forward to a Mongrel Cluster, a FCGI instance of PHP (from my other post), or lots of stuff.

Holy crap, apparently I’m the 1,257,367th most visited site in the world… or  the  the 740,255th most popular in the US.. apparently. I have  a hard time believing either of these  figures.  I know that the PHP/Nginx/Rails Post is very popular, as well as  the Google Maps and the one about how to use Rails Helpers in controllers, and finally the one about BackgrounDRB, but damn.

Anyways, pretty neat.

Quite Possibly the most awesome music video ever. Thank you Weezer for restoring my faith in the world…

Just booked a little week long excursion to Washington State for next month.  We’re flying in to Seattle on June 14th and will be returning on 21st.  Should be super sweet, the Pacific Northwest is my favorite region of The States and its been about 6-7 years since I’ve been out there.

Jen’s never been there before, so I’m very sure her mind will be blown.  The tentative itinerary is 2-3 days in Seattle, 1 day for The Olympic Nat’l Park/Washington Coastline, and a day for a quick Portland/Mt. St. Helen’s trip.  Maybe only 2 days in Seattle and another day split between Mt. St. Helen’s and Mt. Rainier and a full day for Portland… hell, we don’t know maybe we’ll just see what we feel like doing, thats what we usually end up doing on vacations.  For our trip to Florida, we literally had no plan except to go to Florida and went with the flow the whole time and it was great.

I’d also like to go out to Spokane and visit some relatives, we’ll have to see.. so much to do so little time.

I’ll for sure be posting some pics when we get back…

Alrighty folks. Quick walk though for scraping remote web data with Ruby. This is how I did it for my little web scraper I wrote on Saturday..

DISCLAIMER: Web Scraping is kind of a gray area.. don’t steal things that are copywritten and don’t be a jerk. Give credit where credit is due..

First thing is first. You’ll need to install the Mechanize Ruby Gem.

sudo gem install mechanize

Mechanize is pretty slick. It will iterate through a given url and let you access various html elements. Further, you can use Hpricot methods to further grab data.Lets get going..

require ‘rubygems’require ‘mechanize’require ‘uri’

url = “http://www.johnyerhot.com”

The way this is set up, you MUST have a complete url.

@mech = WWW::Mechanize.new

@page = @mech.get(url)

Now, lets say we want to get all the urls for embedded images from the webpage (http://www.johnyerhot.com)..

@imgs = @page.search(”img[@src]“).map {|src| src['src']}

You’ve now got an array (@imgs) with all the urls for embeded images! What we actually did was use Hpricot’s search method to look for and image tags and sucked out the src attribute of that tag. Mechanize does have its own methods for grabbing tags also, for example, you can grab all the link targets from every link in the web page.

#remember @page is just our mechanize instance
# w/ http://www.johnyerhot.com

@links = Array.new
@page.links.each do |link|
@links << link.href
end

Now lets weed out any links to non-images:

@links.each do |link| #yeah we’re only collecting jpgs

if (link.to_s.include? “.jpg”) || (link.to_s.include? “.JPG”) || (link.to_s.include? “.jpeg”) || (link.to_s.include? “.JPEG”)

@imgs << link
end
end

Finally, lets actually grab all those pictures and save them locally using Open URI…

@counter =0
@imgs.each do |image|
url = URI.parse(image)#parse the url and separate need info
Net::HTTP.start(url.host, url.port) { |http|
#appeand the image path with the web root.
image = http.get(image)#actually make the file to save
open(”#{url.host}_#{counter}.jpg”, “wb”) { |file|
file.write(image.body)
counter = counter + 1
}
end

And there you have it! Put it all together and you should have a functioning Ruby web scraper…Sort of. You still have to account for relative vs. absolute urls, are you gonna let in more than jpgs?, what if you need basic authentication for the url? There are still some missing pieces that need to be implemented to have this be ready for general use, but the core is there.
Further Reading
Mechanize Docs
Hpricot
Open URI
Ruby net/http Docs

Spent the afternoon down at the Lake today with Leeroy and the Mrs.  I have to admit, Duluth in the summer time is not that bad.  If the tourists were not around it would better, but today was pretty beautiful.   More pictures here.