Lavender Bay was named after the Boatswain (bosun), George Lavender, from the prison hulk "Phoenix", which was moored there for many years. The bay is dual-named Gooweebahree,[2] (also sometimes written as Quiberee[3]) in the Dharug language of the local inhabitants, the Cammeraygal people of the Eora nation. The colonists also called it Hulk Bay and sometimes Phoenix Bay. George Lavender lived on 14 acres (57,000 m2) adjacent to the property of Billy Blue.
On 30 May 1915 Lavender Bay railway station was opened to take the place of Milsons Point railway station. This only lasted for seven weeks, as passengers refused to alight here and demanded that trains stop at Milsons Point. During the harbour bridge construction, Lavender Bay Station was the terminus for the North Shore Line. The area is now railway storage sidings.[4]
In the 2016 Census, there were 941 people in Lavender Bay. 57.2% of people were born in Australia and 76.1% of people spoke only English at home. The most common responses for religion were No Religion 33.4% and Catholic 26.4%.[1]
A wharf is located in the bay which provides access to private vessels.
The Lavender Bay Baths (1910), also known as Cavill's Baths, a tidal swimming pool, were once popular with swimmers, located in the area beside the ferry wharf.
Wendy Whiteley created a garden adjacent to her home, in the area between Clark Park and the old railway line. It is sometimes known as Wendy's Secret Garden, and is open to the public.
Sir Donald Bradman lived in the harbourfront Bay View Street, and was one of the first few Australians to get a private telephone number while living in Bay View Street.[6]
Artist Norman Lindsay lived at 'Heidelberg' at 9 Bay View Street; another artist, John Firth-Smith, occupied this same house many decades later.[citation needed]
Artist Brett Whiteley lived in a house overlooking the Bay with his wife Wendy Whiteley; Wendy continued to live there as of 2019[update].[7] In 2018 the NSW Government granted heritage status to the Whiteleys' house, its views of Sydney Harbour, and Wendy's remarkable Secret Garden in recognition of the significance of the place to the history of Australian art.[8]