The big reason why we don't have space colonies and regular trips to the moon is that flying into outer space is just plain 'hard.' The business of safely transporting people off the Earth is a costly affair that requires a lot of technology.