By the 1860s all three houses began to attract travelers. To accommodate additional boarders, a wing was added to Second House in 1880 by Arthur Benson, who had purchased all of Montauk at an auction in 1879. The arrival of the Long Island Railroad in 1895 undoubtedly added to the number of vacationers to Montauk. An advertisement in 1900 noted that the house held 25 guests, each paying $10 a week to stay there. At that time Ulysses Tillinghast Payne was the proprietor, and made extensive alterations and improvements. He would be the last keeper of Second House.

Over the years the house had its share of wealthy boarders, including members of the Morgan, Pierrepont and van Rensselaer families, as well as the Gardiners from nearby Gardiner’s Island. The Sag Harbor Express of September 15, 1892 described the house as “a favorite for the pleasure seekers over and amid the ever memorable and romantic hills, causing it to be well filled throughout the season, to say nothing of the many, going to and from the Point, who stop for a passing meal.”

On October 5, 1896 Montauk’s first school opened at Second House with Miss Martha Osborne, of Wainscott, as teacher. Classes were held in a wing off the kitchen, with about a dozen Montauk students in attendance. Classes were held here until a schoolhouse at Hither Plains, Montauk, was opened in 1900.

The first marriage of people other than Montaukett Indians at Montauk took place at Second House on June 25, 1901 when Betsy Dayton Payne, daughter of owners Ulysses and Nellie Payne, wed Charles Cooper Taylor. Five years later the Paynes’ son Edward married Elizabeth Amanda Bennett at Second House.

Two frequent summer visitors to Second House, Mr. and Mrs. David Kennedy, purchased the house in June 1909 and turned it into their private summer residence, ending its days as a boarding house, though they did operate it as an inn for a couple of years in 1934-1935. Mr. Kennedy founded Kentile Floors in Brooklyn in 1898, manufacturing tiles that eventually became one of the most popular styles of flooring used in the post World War II building boom.