The other day I visited the Old World Wisconsin web site at http://www.wisconsinhistory.org/oww/. On one of the pages, there was a place to send a query to them, so I did.
I received an immediate response from Marty Perkins of Old World Wisconsin that said:
The Four Mile House, built in Rolling Prairie, Wisconsin, sat midway between Horicon and Beaver Dam. It was constructed in the early 1850s for New Yorker Ranslow Smith and represents a common mid-nineteenth century hotel form - adorned with Greek Revival elements. I acquired the building for Old World Wisconsin back in the early 1980s. It's 3 stories with a ballroom on the third floor, bedrooms on the second, and public rooms on the first. We have gathered a fairly large amount of research material on the families associated with the building and the building itself. It now sits at the heart of the Crossroads Village exhibit area at OWW. I hope you will make plans to visit this wonderful family heirloom. Please keep me abreast of your plans. I look forward to staying in touch.
Now I know where it was located originally, and where it is now.
This is the second building of Ranslow Smith's still standing - I've been by the house in Henderson in Jefferson County NY, but not in it. It's featured in a local newspaper article about Old Houses in the Northern NY area, and was built in the 1830's.
Needless to say, southeastern Wisconsin just jumped up near the top of our list of places to visit! The really neat thing is that they have a Norwegian area with two farm houses and a schoolhouse. Linda's paternal line came from Norway to Dane County WI in the 1850's.
It's nice to get answers to your questions, eh? With a good answer, too.