In June 2020, Canon announced that it had successfully developed the world’s first 1-megapixel single-photon avalanche diode (SPAD) image sensor. Now Canon has published an article about it, says that this sensor will be utilized when ultra-high frames-per-second shooting speeds are needed. How much FPS? Up to 24,000 frames-per-second! This ability significantly enriches the possibilities in imaging.
High-pixel-count SPAD sensor
As state in the document published by Canon: “Until recently, it was considered difficult to create a high-pixel-count SPAD sensor. On each pixel, the sensing site (surface area available for detecting incoming light as signals) was already small. Making the pixels smaller so that more pixels could be incorporated in the image sensor would cause the sensing sites to become even smaller, in turn resulting in very little light entering the sensor, which would also be a big problem”. As opposed to traditional CMOS image sensors, the SPAD (single-photon avalanche diode) features an electronic element at each pixel that receives a single photon and turns it into an “avalanche” of electrons, resulting in a relatively large electrical pulse, and thus elevation of sensitivity when capturing photos. However, the SPADs are much harder to be scaled to allow a high pixel count. Nevertheless, Canon has resolved it with this world’s first 1-megapixel SPAD sensor. Explore the SPAD vs CMOS slide below (by Canon).
SPAD sensor vs CMOS sensor: Gathering light
Both SPAD and CMOS sensors make use of the fact that light is made up of particles. However, with CMOS sensors, each pixel measures the amount of light that reaches the pixel within a given time, whereas SPAD sensors measure each individual light particle (i.e., photon) that reaches the pixel. Each photon that enters the pixel immediately gets converted into an electric charge, and the electrons that result are eventually multiplied like an avalanche until they form a large signal charge that can be extracted.
SPAD sensor vs CMOS sensor: Noise level
CMOS sensors read light as electric signals by measuring the volume of light that accumulates in a pixel within a certain time frame, which makes it possible for noise to enter the pixel along with the light particles (photons), hence contaminating the information received. Meanwhile, SPAD sensors digitally count individual photon particles, making it hard for electronic noise to enter. This makes it possible to obtain a clear image.
Producing high pixel count
Canon incorporated a proprietary structural design that used technologies cultivated through the production of commercial-use CMOS sensors. This design successfully kept the aperture rate at 100% regardless of the pixel size, making it possible to capture all light that entered without any leakage, even if the number of pixels was increased. The result was the achievement of an unprecedented 1,000,000-pixel SPAD sensor. Have a look at the slide below which demonstrates Canon’s challenges and solutions regarding the development of high pixel count SPAD.
Global shutter and 24,000FPS
The SPAD sensor that Canon has developed is also equipped with a global shutter that can capture videos of fast-moving subjects while keeping their shapes accurate and distortion-free. Unlike the rolling shutter method that exposes by activating a sensor’s consecutive rows of pixels one after another, the SPAD sensor controls exposure on all the pixels at the same time, reducing exposure time to as short as 3.8 nanoseconds and achieving an ultra-high frame rate of up to 24,000 frames-per-second (fps) in 1-bit output. This enables the sensor to capture slow-motion videos of phenomena that occur in extremely short time frames and were previously impossible to capture.
Final thoughts
Utilizing SPAD technology with the elevation of pixel-count unleashes the possibilities regarding ultra-high FPS imaging. This SPAD sensor is premature to be used on high-speed video applications, however, the technology and knowledge can facilitate the development of a higher pixel count that may be good enough for high-quality video. Thus, I wouldn’t be surprised if Canon will announce in a couple of years its first ultra-high-speed EOS.
This is bordering on too good to be true, so fingers crossed that it is!
So, 24000fps @ 1-bit… that’s about 1-fps at 14.5-bit… monochrome. 1-bit is true/false, photon/no-photon – to get a greyscale you’d have to stop down the imager so that it receives infinitesimal numbers of photons over a given exposure time. I’d see this as useful if it could be combined with the PIXIM ADC CMOS cells that Sony bought up (I think now marketed under Starvis/Pregius) – where the rate at which charge saturation is achieved is measured in addition to the final cell charge, this allows for per-pixel AGC and HDR natively, and keeps the cell from being “just white” when it fully saturates. In this case you’d measure the avalanche amplification coefficient, and count the cell charge – or take the time required to fully charge the cell relative to the amplification coefficient, to get cells where the SPAD amplification is too high for the incoming photon count. Balancing these two axis (time and total charge) so that mid-tone grey is a full cell charge at the end of the given exposure time – you can do per-pixel AGC, plus tune the whole imager peak highlight amplitude so that you maximize the sensor’s dynamic range, sensitivity, and gradient.
This kind of reminds me though of how Geiger Counters work with a sensor tube – a large voltage charge is given to a gas, and when a radioactive photon (Alpha, Beta, Gamma, or x-ray) hits the tube, it causes and avalanche charge in the tube which is a detectable current at the output. One side effect of that however is the recovery time of the tube – it’s effectively blind to subsequent photons until the avalanche is quenched. I’d be interested to stew on if this technology suffers the same issue. 24,000fps is roughly one photon detected in 41.6microseconds – I’ll bet since this is single bit, that you’d miss >1 photon before the next evaluation… just a quick Google search shows that 1Lumen is about 1.21^1021 photons per second, divided by 24K it’s still 1.39^80 photons (effectively at 1Lumen you will miss >99.999% of the light available). Kinda puts into perspective what this sensor is really good for – remember 1candelpower is about 12.5lumens… so a candle lit room would blind the sensor.