Example: Camera

This page describes getting a video stream input into an FPGA development board via a Camera PMOD.

This example is from a series of examples designed for dev boards.

The sections below describe the code found in the demo examples/camera/ov2640_dvp_test.c file.

Camera Module

The camera used in this demo is the Omnivision OV2640.

Using the camera is documented in hardware application notes and consists of:

• Clocks + Reset:
  • Hold the camera in reset
  • Provide clock to camera for it's PLL
  • Take the camera out of reset
• Configure the camera over I2C like Serial Camera Control Bus (SCCB)
• Receive video stream over Digital Video Port (DVP)

Pinout

On the Arty dev board PMOD connectors C and D are used. Arty PMOD C = Camera PMOD J2, Arty PMOD D = Camera PMOD J3.

The camera PMOD D port only connects the top eight bits of the ten bit Y port on the camera.

Building on top of the arty.h header for PMOD pins, the first section of demo file includes the top level IO pin mapping.

Two top level pins are of particular importance:

• jc[1] : the pixel clock input to the design, requiring special tool settings.
• jc[7] : the SCCB/I2C bus bidirectional data pin that requires tri-state and pull up logic.

Clocks

The XVCLK System clock input pin is the primary clock input for the camera. The camera datasheet and online examples use a typical rate of 24MHz. On the camera PMOD J2 connector, the system clock input pin labeled XK.

The PCLK pixel clock output from the camera has typical rate of 25MHz. On the camera PMOD J2 connector, the system clock input pin labeled PK.

Reset

The hardware application notes show holding in reset (active low) for at least 3ms. Then SCCB initialization should occur at least 3ms after that.

The PMOD used in this design does not expose the camera reset signal. The camera comes out of reset at power on and will be initialized after the above specified initialization delay.

Part of the initialization sequence sent over SCCB is a software reset and delay that replaces the need for a power on reset.

Serial Camera Control Bus (SCCB)

Per the SCCB spec: Reads from the camera, and writes to the camera are completed by transferring bytes. Each 8b value is a 'phase'. Phases are padded with a 9th control/status bit (called the 'Don't-Care' bit in SCCB and 'ACK/NACK' for I2C).

Clock and Data

Both SCCB and I2C have two primary pins: a 'clock' typically driven by the FPGA into the I2C device and a 'data' pin controlled by both sides of the bus at different phases of operation.

The typical clock cycle time from SCCB spec is 10us, or 100KHz.

The SCCB spec does not specify the need for pull resistors. However, I2C does specify the need for pull-up resistors on the data bus. This demo with the OV2640 camera does not work without them. Enabling FPGA internal pull resistors is device specific.

In VHDL with Xilinx Vivado enabling a pull up resistor on the required PMOD pin looks like so:

jc_7_pullup : PULLUP port map( 
    O => jc(7)
);

In Xilinx Design Constraint .xdc files it looks like so:

set_property PULLUP true [get_ports {jc[7]}]

Like I2C, serial data is to be sampled during (at the middle of) clock=1 time, data changes during (at the middle of) clock=0 time.

Idle, Start, Stop

• Idle:
  • Clock: 1,high
  • Data: tri-state, floating, high-z (pulled high)
• Start:
  • Data: 1 to 0 during clock high time
  • Clock: high to low
• Stop:
  • Data: 0 to 1 during clock high time
  • Clock: low to high

Read

A read transaction occurs over two two-phase transmission cycles:

Transfer 1: two phase write to specify device and register

• ID Address: 7b device id, to select a specific device, and 1b write=0 flag.
• Sub-address: 8b register address, to select specific register

Transfer 2: two phase read, to specify read from device and receive data

• ID Address: 7b device id, to select a specific device, and 1b read=1 flag.
• Read Data: 8b data read from the selected device's register
  • Camera drives the bus during this phase

The SCCB ID to select the OV2640 camera is 0x30. Often the 1b write=0 flag is included and documents will show 0x60 instead.

For example: reading the camera's manufacturer id register looks like:

• ID=0x30
• Address=0x0A to select the PIDH product ID number MSBs, expected read data value of PIDH=0x26

Write

A write transaction requires three phases of values:

• ID Address: 7b device id, to select a specific device, and 1b write=0 flag.
• Sub-address, 8b register address, to select specific register
• Write Data, 8b data to be written to the selected device's register

Video Stream

Video stream consists of these signals all sync to the pixel clock:

• Pixel Clock
• VSYNC
• HREF
• 8bit data

Clock Input

Xilinx Vivado needs to be told about the expected pixel clock rate. In this test 25MHz (40ns period) SVGA timing is used.

Constraining the appropriate PMOD pin looks like so:

create_clock -period 40.000 -name cam_pixel_clk -waveform {0.000 20.000} [get_ports {jc[1]}]

Additionally, because the PMOD connector does not use a dedicated clock input pin for the input pixel clock signal, Vivado requires this constraint to allow that to happen:

set_property CLOCK_DEDICATED_ROUTE FALSE [get_nets jc_IBUF[1]]

Output Format

Details of configuring the camera for various output formats are found in the software application notes (and various example projects).

For example SVGA frame timing looks like so:

This example uses 16 bit RGB565 output. The 16b pixel is split over two eight bit cycles like so:

The demo code copied the software notes Section 13.2 RGB 565 Reference Setting and additionally turned on the color bars test pattern.

typedef struct dvp_rgb565_decoder_t{
  uint5_t r;
  uint6_t g;
  uint5_t b;
  uint16_t x;
  uint16_t y;
  uint1_t pixel_valid;
  uint16_t width;
  uint16_t height;
  uint1_t frame_size_valid;
} dvp_rgb565_decoder_t;
dvp_rgb565_decoder_t dvp_rgb565_decoder(
  uint1_t data[8],
  uint1_t href,
  uint1_t vsync,
  uint1_t vsync_pol,
  uint1_t cam_init_done
)

The dvp_rgb565_decoder function is used to decode the byte stream from the camera and output a stream of RGB565 pixels. Note: the VSYNC polarity shown in the above image is opposite of what the camera outputs, and the vsync_pol is set appropriately.

The top level of the design is marked for debug, meaning its registers can be viewed with FPGA integrated logic analyzers (ILAs). It was using that functionality that the stream of pixels was evaluated and confirmed the camera's color bars test pattern was being correctly received.