Athens is certainly one of the world’s great cities, at least for people who care about culture and history. It’s also a great stop for a few days for people on their way to or from the Greek Islands, and fortunately you can see all the main highlights in two days or so. The city is cheap compared to places like Rome or Paris, but not as cheap as many might expect it to be.
Hotels in particular can be pricey, partly because the city center is quite compact and places with good locations can get away with charging quite a bit. There are several new hostels, a few near the foot of the Acropolis, so if you are on a tight budget you might consider those, whether in a dorm or private room.
The main sites aren’t too expensive, at least compared to how amazing and important they are, so you won’t need to spend a fortune during the day. Food and drink can be cheap in places meant mostly for locals and quite pricey in the places with English-language menus out front. It’s easy enough to stick to a budget if you are careful, and even many of the tourist restaurants have good specials.