I would think that you would still get a fair amount of reflection off the surface and not enough image dispersion. The main reason you want to start with that surface is that it is 100% flat. No waves in it the way drywall is, plus it is not a 'fixed' surface, so you can move it around once it is mounted if you need to tweak it some.
Proper light dispersion is what the paint coats add so that when the projected light hits it, light reflects and scatters evenly across the viewing area. To much reflection and you see severe hot spots. To little reflection and the image may appear dull.
You don't have to go with 'Goo', but you can use any number of different paints to get very good results. I personally am making a screen with blackout cloth and 1x3 lumber. Light weight, portable, flat, and overall pretty inexpensive.