Re: image formats - compression and loading

[Peregrine McGehee <mcgehee@cfht.hawaii.edu>]

Mike Hoffberg writes:
 > I work at the APS in the Experimental Facilities Division.  We are
 > working on cameras with resolution from 512x512 to 2K x 2K (at least
 > 12 bits/pixel).  Our frame rates could be as high as 100 frames per
 > second.  So I guess we would qualify as having large amounts of data.
 > We plan to be able to store 250 MB - 500 MB of images in RAM on the
 > IOC.
 > 
 > Originally we are storing our data as just arrays of numbers.  I would
 > anticipate that if we were to compress our data on the IOC it would
 > take less time to transfer it off of the crate, and also less amount
 > of disk space to store it (not as important to me).
 > 
 > Our motivation would be to compress the data on the IOC so that we
 > could 'quickly' transfer it off of the crate.  Our IOC will be a VXI
 > crate, the transfer rate from the memory card to the processor (on the
 > backplane) will probably be well under 5 MB/second.  Transfer over
 > the ethernet will be be under 1 MB/second (using thin-net), so any
 > compression we could do on the CPU would save time transferring the
 > data off of the IOC to permanent storage.
 > 
 > At the moment we do not have a real working system, but we plan to
 > have the above system running within a year.
 > 
 > I am not too familiar with compression algorithms, what sort of CPU
 > usage will they require?  Can this compression be done on a 25 MHz
 > 68040 with relative ease?  Does it require floating point?  Would it
 > be better to have a dedicated DSP processor for this?
 > 
Anyone have a feel for the compute load of common compression
algorithms?

We'd like to perform image compression both in the IOC environment and
on the workstations - so we need something that is fairly
portable. Here at CFHT we have the usual mix of HP9000 and Sun Sparc 
workstations but are running EPICS on Force Sparc 5 CPUs.

 > As an experimenter it is important that the compression be lossless.
 > I would not advocate limiting the images to 16 bit pixels.  With the
 > cost of electronics going down that the quality going up, I would
 > suspect that there could be detectors that have 20 bit resolution.

Agreed - 16 bits (or even 20 bits) could be an arbitrary limit that
could soon be a real drawback.

 > Please keep me informed on any progress made on this topic.
 > 
 > MIKE
 > 
 > Michael Hoffberg                 /.\ Argonne       god of a bitch
 > hoffberg@phebos.aps.anl.gov     // \\ Nationa      spelled backwards
 > mike@anl.gov                   //_O_\\ Lab         is dog - anonymous
 > Standard Disclaimer Applies   /__| |__\      (yes, it is still there)
 > 

-------------------------------------------------------------------------------
Peregrine M. McGehee			http://www.cfht.hawaii.edu/~mcgehee
Telescope Control Systems Group 	Canada-France-Hawaii Telescope
(808) 885-3178				P.O. Box 1597, Kamuela, Hawaii 96743