Difference between revisions of "UAD CODEC WHITE PAPER"

From OCTOPUSCINEMA
Jump to: navigation, search
m
m
Line 34: Line 34:
  
 
<br>
 
<br>
 +
==True Open Source==
 +
UAD is a true open source codec and the only truly open source RAW video codec. The UAD code base is publicly accessible on [https://github.com/octopuscinema/uad GitHub].
 +
The UAD repository is licenced with the permissive [https://www.apache.org/licenses/LICENSE-2.0 Apache 2.0 License].
 +
====Why====
 +
Avoiding proprietary black-box technology promotes the learning and understanding of how tools work. Opening up the codec to the public allows for a more democratic approach to improvements and bug fixes.
 +
 +
The raw developing pipeline for example
 +
Working with raw images for example, requires an understanding of colour and gamma spaces. guessing colour and gamma spaces can lead to incorrect
 +
Colour science and image processing pipelines which are hidden away from the user
 +
should not be hidden away from the user. Without an open source RAW developing process, determining
 +
 
==RAW Acquisition==
 
==RAW Acquisition==
 
A core feature of UAD is the ability to encode and decode ''raw'' image sensor data. By recording raw sensor data, the image processing pipeline is no longer confined to running on the hardware of the camera within the time duration of one ''frame''. This not only allows for more complex higher quality raw processing but also far more creative control over how the image is ''developed''.
 
A core feature of UAD is the ability to encode and decode ''raw'' image sensor data. By recording raw sensor data, the image processing pipeline is no longer confined to running on the hardware of the camera within the time duration of one ''frame''. This not only allows for more complex higher quality raw processing but also far more creative control over how the image is ''developed''.

Revision as of 21:05, 20 October 2021

Introduction

UAD (Uncompressed Adaptive Downsampling) is a new open source and radically different approach to image acquisition. UAD does not compress data, but rather it dynamically alters the data frequency in the spatial domain - in the context of image data, this means dynamically altering resolution within a single image.

As CMOS image sensors increase in resolution, the bandwidth and storage requirements race upwards. When an image sensor increases in physical size, however, the optical behaviour creates a larger disparity between what is in and out of focus. At the extreme end, the large imaging area of a medium format camera creates images with typically only a small percentage of in-focus pixels. Despite this, medium format CMOS sensors are usually much higher resolution than their smaller counterparts.

UAD exploits the notion that only some areas of an image or frame will benefit from an image sensor's full resolving power. Instead of compressing images by looking for entropy to predict image data, UAD simply reduces the resolution of lower precedence areas.


Benefits

Benefits of UAD's encoding method include:

  • Uncompressed quality Colour accuracy and fidelity is not affected by downsampling, higher reduction ratios only ever trade off resolution.
  • GPU efficiency Resizing image data can be done on GPUs extremely fast. 8K UAD can be encoded at faster than real-time (30fps+) even on mobile class hardware with integrated GPUs.
  • Freedom from compression patent infringement UAD's lack of compression or entropy approach means it does not use any compression algorithms which could be considered prior art.
  • Multi-resolution decoding Decoding can be performed directly to lower resolutions allowing for manageable real-time performance on low spec hardware.


Supported Formats

Codec development is on-going with image format support as follows:

  • 12-bit log encoded CFA Bayer RAW (Fully implemented)
  • 12-bit log encoded monochrome (Planned)
  • 12-bit YUV video (Planned)


Data Rate

The codec is designed for quality and throughput at the expense of size and recommends the following size reduction ratios for high fidelity capture:

  • 4:1
  • 3:1
  • 2:1


True Open Source

UAD is a true open source codec and the only truly open source RAW video codec. The UAD code base is publicly accessible on GitHub. The UAD repository is licenced with the permissive Apache 2.0 License.

Why

Avoiding proprietary black-box technology promotes the learning and understanding of how tools work. Opening up the codec to the public allows for a more democratic approach to improvements and bug fixes.

The raw developing pipeline for example Working with raw images for example, requires an understanding of colour and gamma spaces. guessing colour and gamma spaces can lead to incorrect Colour science and image processing pipelines which are hidden away from the user should not be hidden away from the user. Without an open source RAW developing process, determining

RAW Acquisition

A core feature of UAD is the ability to encode and decode raw image sensor data. By recording raw sensor data, the image processing pipeline is no longer confined to running on the hardware of the camera within the time duration of one frame. This not only allows for more complex higher quality raw processing but also far more creative control over how the image is developed.


Efficiency

Raw sensor data from a colour image sensor contains only a single colour channel per pixel. It is only after debayering this raw data that the image contains the three expected red, green and blue channels per pixel. This transformation of one channel into three channels increases the image size by a factor of three. For this reason, it is much more efficient to encode image data prior to the debayering transformation.


Performance

The debayering and image processing phase evaded by raw acquisition must still be done in order to display the final image. The workload of a raw decoder is therefore much greater than a typical non-raw image format. UAD implements the entire debayering and image processing pipeline in a single OpenCL program allowing for efficient real-time 8K raw playback on mid-range computer hardware. Additionally, UAD can decode natively at half and quarter resolutions, allowing for smooth real-time playback on lower-end or mobile class computer hardware.