Don't worry about the light's, that's ok. Only NIC cards will keep the light's on, but if you take the power cord from your CPU off, the light will go off as well.
Okay, Linksys. Now, go on your web browser and type
http://192.168.1.1
If you never did this, the login/password should be admin/admin (please, do change this if you haven't).
On your very first page, you'll find out what type of IP you have. If it says 'DHCP' well, you probably will need to upgrade your firmware or contact Linksys. If this field looks like this:
http://www.datatruck.com/images/WRT54Gscreenshot.GIF
You will need to find a way to input this IP/DNS manually into your DVD player.
Another link if you need:
http://www.tech-faq.com/setup-linksys-router.shtml