Hong Kong was once thought of as a cheap destination, but that is not really the case anymore. Compared to most other large cities in Asia it’s fairly expensive, though there are definitely a fair amount of cheap options for those on backpacker budgets.
Hotels in particular can be pricey in Hong Kong, especially if you want to stay on the island itself. Staying in the southern part of Kowloon will save you some money while keeping you in the center of the action. Hostels in Hong Kong offer good bargains as well, and private rooms are often quite a bit cheaper than 2-star hotels, but with similar services. Strangely enough, the hostels in Hong Kong almost all tend to get horrible reviews, so don’t expect much.
Hong Kong is loaded with Michelin-star restaurants and some very pricey options, but it’s also home to thousands of neighborhood noodle joints that are good and cheap. Alcohol in proper bars here is very expensive so it’s poor value for budget travelers. Instead you can do like the locals and buy cheap drinks at 7-Eleven and consume them (legally) on the streets in the nightlife districts.
As long as you don’t plan on going to Disneyland or the other super-commercial options then the attractions in Hong Kong are reasonably priced. More good news is that food can be quite cheap if you stick more to street stalls and other places where locals eat, but if you go to places that cater mostly to Western tourists you can spend a bundle. The Hong Kong Free Walking Tour is a great way to see a lot of the city in a few social hours.