In 1835 the British government agreed that a new Government House in Sydney had become a necessity, and the royal architect, Edward Blore, was instructed to draw up plans. Construction commenced in 1837 and was supervised by colonial architect Mortimer Lewis and Colonel Barney of the Royal Engineers. Stone, cedar, and marble for the construction were obtained from various areas of New South Wales. A ball in honour of the birthday of Queen Victoria was held in the new building in 1843, although construction was not complete. The first resident, Governor George Gipps, did not move in until 1845. 

Government House, with its setting on Sydney Harbour, has a garden area of 5 hectares and is located south of the Sydney Opera House, overlooking Farm Cove. It was designed in a romantic Gothic revival style—castellated, crenellated, turreted and is decorated with oil portraits and the coats of arms of its successive occupants. Additions have included a front portico in 1873, an eastern verandah in 1879 and extensions to the ballroom and governor's study in 1900–01.  

From 1845 until 1901, the building served as the Governor's residence, office and official reception space. http://en.wikipedia.org/wiki/Government_House,_Sydney