It's not bad and doesn't affect the flavor. Bottle-conditioned beers can be quite good and the yeast is a part of the experience. In this case, the floaters tend to look like snowflakes rather than the yeast sediment from bottle conditioning.
To summarize the other answers here, the beer may taste "worse" at the bottom because: 1) Change in temperature, allowing bad tastes to become more apparent when warmer. 2) Loss of carbonation. 3) Possible yeast or other particulate matter in unfiltered beer resting at the bottom of the glass.