UAD CODEC WHITE PAPER

From OCTOPUSCINEMA
Revision as of 21:28, 20 October 2021 by Ocwikiadmin (talk | contribs)
Jump to: navigation, search

Introduction

UAD (Uncompressed Adaptive Downsampling) is a new open source and radically different approach to image acquisition. UAD does not compress data, but rather it dynamically alters the data frequency in the spatial domain - in the context of image data, this means dynamically altering resolution within a single image.

As CMOS image sensors increase in resolution, the bandwidth and storage requirements race upwards. When an image sensor increases in physical size, however, the optical behaviour creates a larger disparity between what is in and out of focus. At the extreme end, the large imaging area of a medium format camera creates images with typically only a small percentage of in-focus pixels. Despite this, medium format CMOS sensors are usually much higher resolution than their smaller counterparts.

UAD exploits the notion that only some areas of an image or frame will benefit from an image sensor's full resolving power. Instead of compressing images by looking for entropy to predict image data, UAD simply reduces the resolution of lower precedence areas.


Benefits

Benefits of UAD's encoding method include:

  • Uncompressed quality Colour accuracy and fidelity is not affected by downsampling, higher reduction ratios only ever trade off resolution.
  • GPU efficiency Resizing image data can be done on GPUs extremely fast. 8K UAD can be encoded at faster than real-time (30fps+) even on mobile class hardware with integrated GPUs.
  • Freedom from compression patent infringement UAD's lack of compression or entropy approach means it does not use any compression algorithms which could be considered prior art.
  • Multi-resolution decoding Decoding can be performed directly to lower resolutions allowing for manageable real-time performance on low spec hardware.


Supported Formats

Codec development is on-going with image format support as follows:

  • 12-bit log encoded CFA Bayer RAW (Fully implemented)
  • 12-bit log encoded monochrome (Planned)
  • 12-bit YUV video (Planned)


Data Rate

The codec is designed for quality and throughput at the expense of size and recommends the following size reduction ratios for high fidelity capture:

  • 4:1
  • 3:1
  • 2:1


True Open Source

UAD is a true open source codec and the only truly open source RAW video codec. The UAD code base is publicly accessible on GitHub. The UAD repository is licenced with the permissive Apache 2.0 License.

  • Future proof Open source software can be ported to any platform by anyone and can be updated to work with new hardware and environments as they emerge.
  • Work smarter Avoiding the proprietary black-box approach promotes learning and the understanding of how tools work. The entire raw developing process for example lives in RawProcess.cl. Even with no programming knowledge, one can see how parameters like exposure, contrast and saturation are are actually affecting the image.
  • Stability Opening up the codec to the public allows for democratic and rapid improvements and bug fixes.


RAW Acquisition

A core feature of UAD is the ability to encode and decode raw image sensor data. By recording raw sensor data, the image processing pipeline is no longer confined to running on the hardware of the camera within the time duration of one frame. This not only allows for more complex higher quality raw processing but also far more creative control over how the image is developed.


Efficiency

Raw sensor data from a colour image sensor contains only a single colour channel per pixel. It is only after debayering this raw data that the image contains the three expected red, green and blue channels per pixel. This transformation of one channel into three channels increases the image size by a factor of three. For this reason, it is much more efficient to encode image data prior to the debayering transformation.


Performance

The debayering and image processing phase evaded by raw acquisition must still be done in order to display the final image. The workload of a raw decoder is therefore much greater than a typical non-raw image format. UAD implements the entire debayering and image processing pipeline in a single OpenCL program allowing for efficient real-time 8K raw playback on mid-range computer hardware. Additionally, UAD can decode natively at half and quarter resolutions, allowing for smooth real-time playback on lower-end or mobile class computer hardware.