Who gets to decide what is "work"? Only that which earns money? Then why not tax the money? Oh yeah, that's what we already do. If someone is not working for money, how do you tax them? Take away some of their free time by making them slaves?
I generally work only a couple hours a day for money. This earns me a reasonably comfortable life and I have plenty of free time to pursue other things like growing food, land restoration, art, talking politics on messageboards. Should I be taxed for the time I spend growing food? Or only for the time I spend talking politics on messageboards? If I happen to sell some of the art I made in my free time, does that get taxed at the money-earning rate or at the leisure rate? What about folks who spend part of their workday talking politics on messageboards? Are they working, or are they at leisure? Who decides and how is each moment taxed?
I'm not really into the whole "I'm rich because I work so hard, stop picking on me!" thingy. Lots of folks work hard who are not rich. Lots of rich folks barely work at all. If folks don't want to work so hard maybe they should figure out how to reduce their need to earn so they can slack off like I do.