Actually it kind of is, if you read Fleming's books Bond talks of his first expensive tailor, suggesting usually he has used a cheaper tailor. He dresses in rather average clothes for the time just with the touch of tailored class and particular individual preference. He doesn't wear really expensive gear all the time and he wears a tux sometimes because back then you HAD to wear one to get into a few places Bond went.
Often in the books he wears trousers, shirt and loafers (he hates shoe laces). He is often very casual, and in the 50's 60's that was the equivalent today of chinos, loafers and a casual shirt or something. Not at all fancy or really expensive. Back then suits were what everyone wore as everyday attire.
Okay correction That's not the cinematic Bond who clearly loves fancy thing and is an expert on much of it.