Ko Samet is popular with Thais, foreign travellers and expats from Bangkok and Rayong, so there's always a good mix of people. However, this also means that it can get rather hectic unless you arrive mid-week.
Being so close to Bangkok, Ko Samet gets crowded on weekends and holidays, and prices rise accordingly -- sometimes by as much as 60%. If bungalows are full when you arrive, it is possible to camp on the beach, despite what you may be told.