From 1980 until 2007, U.S. average hours worked increased by thirteen percent, due to a large increase in female hours. At the same time, the U.S. labor wedge, measured as the discrepancy between a representative household's marginal rate of substitution between consumption and leisure and the marginal product of labor, declined substantially. We examine these trends in a model with heterogeneous households: married couples, single males and single females. Our quantitative analysis shows that the shrinking gender wage gaps and increasing labor income taxes observed in U.S. data are key determinants of hours and the labor wedge. Changes in our model's labor wedge are driven by distortionary taxes and non-distortionary factors, such as cross-sectional differences in households' labor supply and productivity. We conclude that the labor wedge measured from a representative household model partly reflects imperfect household aggregation.