It sounds to me like WinDVD encounters an error reading the DVD and doesn't handle it well. It could be a bug in WinDVD or it could have been an unrecoverable error from the operating system that the program cannot deal with such as a page fault or general protection error. The underlying cause is hard to guess without further investigation.
If you are using Windows 2000 or Windows XP, the system restarts because your startup and recovery settings are set to restart. Go to Control Panel->System->Advanced tab and click on Settings in the Startup and Recovery section at the bottom. Under System Failure, uncheck everything except 'Write an event to the system log'. It will then no longer automatcally restart on faults.
Right click on the My Computer icon and choose Manage. Under System Tools is Event Viewer which shows the error logs for Application, Security, and System. There should be an error message, probably under Application or System. You could clear all of the event logs and then try playing the DVD so that there will be fewer messages to sort through.