Lossless video compression

[haraldo]

G'day

I do have a whole bunch of old Super-8 movies that I'm gonna have scanned and digitized so that I can save them for future, it's something like 50 to 60 hours. The shop doing this work digitizes them in uncompressed .avi, which according to them will result in files of about 30GB per hour. Even with today's disk prices this is somewhat problematic, not to mention the issue of managing close to 2TB of movies, making backup. handing copies to others +++

The argument from the shop to make them uncompressed is that any editing of compressed files with increasingly degrade qualtiy.... I can see this argument if it's lossy compression, however, with lossless compression this should not be an issue.

I'm not very much into the different physical formats, I can however say that WMA is not an option as I rely heavily on use of Linux (and also Windows) and WMA is troublesome on Linux.

Now my question is if there's any good lossless compression that I can use (as an anology... the use of FLAC for audio files) is there anything good open format with lossless compression for video files that will easily play on all OS platforms.

Any good tips and ideas on this will be gracefully appreciated

 

[krzywica]

Lossless compression is an oxymoron....there is no such thing and FLAC is not an example of this as FLAC is lossless.

I would grab a program call Handbrake and play around with it.

If I were you I would buy a 2TB drive and throw everything on there. I would then encode all the videos via a batch job after playing with some settings and finding a happy compression medium and store them somewhere else so you can burn them to DVD easeir. After you have finished encoding the files park the 2TB drive in a safe place not connected to a PC for a safe backup. These can be had for under $100 these days.

 

[BMXTRIX]

You want to get multiple 2TB hard drives and get these videos in their native high quality scan format as AVI files.

When you edit, however you edit, you always start with the highest quality original, then create the edit you want, and then you get the final video file. This edited video should ALSO be uncompressed AVI format.

The very, absolute last thing you do is you create the DVD (or other) file you want for distribution.

The key is starting with good material, editing it as high quality material, and creating a master cut which is top shelf.

Then work down in quality. Disc space is relatively cheap and for the type of work you are talking about you really need to value the quality of the work you are paying for. Put it on a hard drive, back up that hard drive, and then work at thei highest quality level, then go to compression. H.264 compression (MPEG4) is the gold standard right now, but is most certainly isn't lossless, even though at high enough bit rates it could appear to be.

 

[BoredSysAdmin]

Lossless compression is an oxymoron....there is no such thing and FLAC is not an example of this as FLAC is lossless.

Not correct.
Lossless Video Compression is not an oxymoron.
These are rare creatures, but they do exist:
http://www.100fps.com/file_sizes_of_lossless_compression.htm
http://en.wikipedia.org/wiki/MSU_Lossless_Video_Codec

In fact I used HuffYuv codec before. But you'd be better off using hi-bitrare lossy codec as MPeg4-AVC aka h264 and handbrake is good tool for that

 

[krzywica]

Not correct.
Lossless Video Compression is not an oxymoron.
These are rare creatures, but they do exist:
http://www.100fps.com/file_sizes_of_lossless_compression.htm
http://en.wikipedia.org/wiki/MSU_Lossless_Video_Codec

In fact I used HuffYuv codec before. But you'd be better off using hi-bitrare lossy codec as MPeg4-AVC aka h264 and handbrake is good tool for that

In this sense there is no feasible way for him to obtain lossless compression, hence the oxymoron statement.

If I were doing it I would do everything x.264 in an mkv container. Avi is a thing of the past and cant hold a candle to mkv.

Either way I hope you have a really fast CPU in your PC. Encoding 2TB of content is going to take a LONG time!

 

[bandphan]

Not for nothing divx formats are very good especially lower quality PQ.

 

[BoredSysAdmin]

In this sense there is no feasible way for him to obtain lossless compression, hence the oxymoron statement.

Maybe my English is not my mother tongue, but I believe "feasible" means doable ... so is lossless feasible - the answer is yes - HuffYuv codec is FREE and can be relatively easily used
http://neuron2.net/www.math.berkeley.edu/benrg/huffyuv.html
but yea, I'm being details Nazi.

So feasible - yes, but good choice - not so much...

 

[BMXTRIX]

At the end of the day, the result is the same. For portability, something other than lossless is required. For archival purposes, you maintain the original lossless AVI files. For people who edit, I'm not aware of .AVI being "a thing of the past". In fact, AVI is still regularly used because of the easy nature of the format to maintain so much information.

It's not at all suited for web environments and for portability of a single video, the MPEG4 versions trump it by far. But, for raw video editing, .AVI is still a great way to get things done with all major video editing programs working with it.

The final portable product should match the environment (DIVX/MPG/AVC/etc.) but the raw video should remain pristine no matter what.

If there is a requirement to have a secondary copy for distribution, then I would skip lossless and use a good AVC encoder and take them all to a file size which can be backed up to data discs.

If you want something that plays in DVD players, you need DVD creation software and the best codec will be mpeg2. The DVD creation software should handle the encode automatically.

There is a reason why there is a list of professionals whose job it is, is to make videos. There are hundreds of tools and codecs available and many of them have their own unique purpose.

 

[pzaur]

DVDFlick is a handy, open source DVD maker. If you take some time to figure out and understand how the Title & Menu pages are created, you can make your own custom ones.

I started using this program within the last year. It handles most every format known. I've only had issues of incompatibility when the file has DRM on it.

-pat

 

[haraldo]

How about AVCHD, that Canon and others uses in their camcorders?
Or is this just mpeg-4?

 

[krzywica]

How about AVCHD, that Canon and others uses in their camcorders?
Or is this just mpeg-4?

AVCHD is for HD content.

 

[haraldo]

AVCHD is for HD content.

Yes, but you can still record and encode using AVCHD for lower resolution content

 

[BoredSysAdmin]

How about AVCHD, that Canon and others uses in their camcorders?
Or is this just mpeg-4?

AVCHD is just a container just like AVI ,MP4 or DVD

"..AVCHD utilizes MPEG-4 AVC/H.264 (AVC) video compression codec and either Dolby AC-3 (Dolby Digital) or linear PCM audio compression codec. "

"Aside from recorded audio and video, AVCHD includes features to improve media presentation: menu navigation, slide shows and subtitles. The menu navigation system is similar to DVD-video, allowing access to individual videos from a common intro screen. Slide shows are prepared from a sequence of AVC still frames, and can be accompanied by a background audio track. Subtitles are used in some camcorders to timestamp the recordings."

Source: http://en.wikipedia.org/wiki/AVCHD

Haraldo, in other words using AVCHD for storing your old videos might not be best idea since you would be severally constraint yourself to specific proprietary container which you might have hard time to convert/edit in the future (while getting no tangible benefits) vs industry standard like mp4

 

[krzywica]

This is an excellent deal on a 2TB drive for $100.

 

[haraldo]

This is an excellent deal on a 2TB drive for $100.

Thx, there's a different market for hard drives and NAS boxes around here, generally prices are somewhat higher, best price I seen so far is 1TB for 100$ or $185 for 2TB bundled in external cabinet with USB2 support

2GB s-ata disks bundles around $150+, so it's not expensive

So it's not as cheap as US, but quite close

 

[haraldo]

Thx for all the good responses.... I'll probably keep the master's in .avi and if I need to distribute to others I may make compressed versions, but probably a good idea to keep original/masters as is in uncompressed avi

 

[BoredSysAdmin]

Haraldo, I decided to go back to your first post actually read it whole
If I understand correctly you'll be attempting to do DIY Super8 mm film to digital transfer??

I found few links - you might be interested in:
http://www.truetex.com/telecine.htm
http://jiminger.com/s8/index.html
http://www.super-8mm.net/5.html

http://lightpress.tv/super-8mm/ -

I think this is pretty good example of what can be achieved on pro equipment:

<object width="640" height="480"><param name="allowfullscreen" value="true" /><param name="allowscriptaccess" value="always" /><param name="movie" value="http://vimeo.com/moogaloop.swf?clip_id=7620729&amp;server=vimeo.com&amp;show_title=1&amp;show_byline=1&amp;show_portrait=0&amp;color=&amp;fullscreen=1" /><embed src="http://vimeo.com/moogaloop.swf?clip_id=7620729&amp;server=vimeo.com&amp;show_title=1&amp;show_byline=1&amp;show_portrait=0&amp;color=&amp;fullscreen=1" type="application/x-shockwave-flash" allowfullscreen="true" allowscriptaccess="always" width="640" height="480"></embed></object>

 

[haraldo]

THX for all these great links!!!!!!!!!!!!!!!!!!!

Well I thought about DIY, but there's too much equipment going into this and quite difficult, so I'm using a professional workshop for this.....

This is mainly old Super 8 b/w movvies without sound... some newer ones with color, but mostly B/W. This is very old movies 30 - 40+ years.... and so it needs careful attention....
They will clean the movies, scan them and digitize them into .avi files, so I just get the files readymade on a hard disk

This is quite a lot of work and I can't also find anywhere to lease the equipment to do this myself

So I decided to use pro's to do the conversion... not cheap, but this is my life as a very small kid..... in video, so it's a small prize to pay to get it done by pros.

-Cheers H

 

[BMXTRIX]

Haraldo, I can't emphasize enough what a good decision that is. The beauty of it all is that you can do whatever you want with the originals as far as editing and you've picked up a lot of very useful information on what you do with the videos for distribution and editing down the line.

Remember that it is very important to have two matching 2TB drives with all the files on them for archival purposes.

But, when you are ready to edit, you may find that a full conversion to MPG4 makes a lot of sense and then master the video as a far more portable HD MPG4 file. With a Blu-ray burner, you can fit several hours onto a single disc with phenomenal quality. Not as good as the masters, but still, very good.

 

[haraldo]

Got my first load of movies yesterday...... transferred from Super 8, digitized to uncompressed .avi format, and all of them were surprisingly good, quite good colors, and nice quality, thinking that it's very old movies.... first day at school and many other things

Not so bad when it comes to size, 30GB for 2.5 hrs of movies.

Thinking about what they do it's quite a tough job, for each frame in the movie they scan it, every single frame.... I been told that it's at least 6 hrs work to digitize 1hr worth of movie.

we were almost laughing till tears..... this is a treasure that must be kept for future and I'm very happy that I started this process

So far I got 2.5 hours converted at a price of $800,- certainly not cheap, but Worth it.
There's probably 25 hours more .....
And it actually takes them weeks to process each batch of movies that I'm gonna ship to them