When I worked at the NCI, I tried to figure out why standard immunology methods failed to develop vaccines against cancers. I used to ask real immunologists the same questions as yours.
One of them answered that developing the highly successful vaccines against polio was a rare home run in vaccine development. It worked well right out of the gate, without requiring extensive development. Essentially, we were quite lucky with polio. Measles is a similar success story, but it took many years longer. It's like a young baseball player wondering if he should give up because he can't hit just like Ted Williams. Keep trying, and don't beat yourself up if you don't hit a home run on the first pitch.
Developing vaccines against other viral diseases have been highly variable. We have fairly successful vaccines against polio, small pox, measles, mumps, rubella, chicken pox, human papiloma virus (HPV), to name a few – but where are the vaccines against HIV, Herpes, any of the common cold viruses, respiratory syncytial virus (RSV), and others?
In contrast to viral diseases, the vaccines developed against bacterial diseases (diptheria, whooping cough [pertussis], tetanus, etc.) have been much more successful. And then there are the numerous failed attempts at developing vaccines against a higher microorganism, malaria.
If we understood more about the immune system, we'd be hitting more home runs.
Ask me again 10 years from now.
It might be useful to read about the history of the development of some of the more successful vaccines. These efforts took years, decades, or longer. Only polio was a home run at the first swing.