One big reason we can never fully know the past, let alone fully know the present or the future, is that these three theoretical categories don't really live in separate boxes, not out here in the real world.
So : energy from cosmic rays, originating from places trillions upon trillions upon trillions of miles away and hundreds of millions of years ago, smashes into our bodies every single minute of our lives, constantly mutating the DNA destined to become that of our children's children.