UAD CODEC WHITE PAPER

From OCTOPUSCINEMA
Revision as of 19:22, 12 October 2021 by Ocwikiadmin (talk | contribs)
Jump to: navigation, search

Introduction

UAD (Uncompressed Adaptive Downsampling) is a new open source and radically different approach to image acquisition. UAD does not compress data, but rather it dynamically alters the data frequency in the spatial domain - in the context of image data, this means dynamically altering resolution within a single image.

As CMOS image sensors increase in resolution, the bandwidth and storage requirements race upwards. When an image sensor increases in physical size, however, the optical behaviour creates a larger disparity between what is in and out of focus. At the extreme end, the large imaging area of a medium format camera creates images with typically only a small percentage of in-focus pixels. Despite this, medium format CMOS sensors are usually much higher resolution than their smaller counterparts.

UAD exploits the notion that only some areas of an image or frame will benefit from an image sensor's full resolving power. Instead of compressing images by looking for entropy to predict image data, UAD simply reduces the resolution of lower precedence areas.


Benefits

Benefits of UAD's encoding method include:

  • Uncompressed quality Colour accuracy and fidelity is not affected by downsampling, higher reduction ratios only ever trade off resolution.
  • GPU efficiency Resizing image data can be done on GPUs extremely fast. 8K UAD can be encoded at faster than real-time (30fps+) even on mobile class hardware with integrated GPUs.
  • Freedom from compression patent infringement UAD's lack of compression or entropy approach means it does not use any compression algorithms which could be considered prior art.
  • Multi-resolution decoding Decoding can be performed directly to lower resolutions allowing for manageable real-time performance on low spec hardware.


Supported Formats

Codec development is on-going with image format support as follows:

  • 12-bit log encoded CFA Bayer RAW (Fully implemented)
  • 12-bit log encoded monochrome (Planned)
  • 12-bit YUV video (Planned)


Data Rate

The codec is designed for quality and throughput at the expense of size and recommends the following size reduction ratios for high fidelity capture:

  • 4:1
  • 3:1
  • 2:1


RAW Acquisition

One of UADs core features is the ability to encode and decode RAW image sensor data. By recording raw sensor data, the image processing pipeline is no longer confined to the hardware of the camera. This allows not only for more complex higher quality raw processing but far more creative control over how the image is developed.

Efficiency

Raw sensor data from a colour image sensor contains only a single colour channel per pixel. It is only after debayering this raw data that the image contains the three expected red, green and blue channels per pixel. This transformation of one channel into three channels increases the image size by a multiple of three. For this reason, it is much more efficient to encode image data prior to the debayering transformation.

Performance

The debayering and image processing phase evaded by raw acquisition must still be done in order to work with sensor data. The workload of the decoder is much greater than a typical non-raw video format. UAD implements the entire debayering and image processing pipeline in a single OpenCL program allowing for efficient real-time 8K raw playback on mid-range computer hardware. Additionally, UAD can decode natively at half and quarter resolutions, allowing for smooth real-time playback on lower-end or mobile class computer hardware.