pynq.lib.video Module

The pynq.lib.video module is a driver capturing streaming HDMI input, producing streaming HDMI output and hardware-accelerated colorspace conversion.

pynq.lib.video.clocks Module

class pynq.lib.video.clocks.DP159(master, address)

Bases: object

Class to configure the TI SNDP159 HDMI redriver/retimer

set_clock(refclk, line_rate)

Configure the device based on the line rate

class pynq.lib.video.clocks.IDT_8T49N24(master, address)

Bases: object

Driver for the IDT 8T49N24x series of clock generators

check_device_id()

enable(active)

set_clock(freq, line_rate)

Configure the device based on the line rate

The parameter line_rate is left to keep consistent API with other clock drivers.

class pynq.lib.video.clocks.SI_5324C(master, address)

Bases: object

Driver for the SI 5324C series of clock generators

check_device_id()

enable(active)

set_clock(freq, line_rate)

pynq.lib.video.common Module

class pynq.lib.video.common.PixelFormat(bits_per_pixel, in_color, out_color, fourcc=None)

Bases: object

Wrapper for all of the information about a video format

bits_per_pixel

Number of bits for each pixel

Type:

int

in_color

Coefficients from BGR stream to pixel format

Type:

list of float

out_color

Coefficient from pixel format to BGR stream

Type:

list of float

class pynq.lib.video.common.VideoMode(width, height, bits_per_pixel, fps=60, stride=None)

Bases: object

Class for holding the information about a video mode

height

Height of the video frame in lines

Type:

int

width

Width of the video frame in pixels

Type:

int

stride

Width of a line in the video frame in bytes

Type:

int

bits_per_pixel

Bits per pixel

Type:

int

bytes_per_Pixel

Bytes required to represent each pixel

Type:

int

shape

Numpy-style tuple describing the video frame

Type:

tuple of int

pynq.lib.video.dma Module

class pynq.lib.video.dma.AxiVDMA(description, framecount=None)

Bases: DefaultIP

Driver class for the Xilinx VideoDMA IP core

The driver is split into input and output channels are exposed using the readchannel and writechannel attributes. Each channel has start and stop methods to control the data transfer. All channels MUST be stopped before reprogramming the bitstream or inconsistent behaviour may result.

The DMA uses a single ownership model of frames in that frames are either owned by the DMA or the user code but not both. S2MMChannel.readframe and MM2SChannel.newframe both return a frame to the user. It is the user’s responsibility to either free the frame using the freebuffer() method or to hand ownership back to the DMA using MM2SChannel.writeframe. Once ownership has been returned the user should not access the contents of the frame as the underlying memory may be deleted without warning.

readchannel

Video input DMA channel

Type:

AxiVDMA.S2MMChannel

writechannel

Video output DMA channel

Type:

AxiVDMA.MM2SChannel

class MM2SChannel(parent, interrupt, memory)

Bases: object

DMA channel from memory to a video output.

Will continually repeat the most recent frame written.

mode

Video mode of the DMA channel

Type:

VideoMode

cacheable_frames

Whether frames should be stored in cacheable or non-cacheable memory

Type:

bool

property activeframe

property desiredframe

property framedelay

property mode

The video mode of the DMA, must be called prior to starting. If changed while the DMA channel is running the channel will be stopped

newframe()

Returns a frame of the appropriate size for the video mode.

The contents of the frame are undefined and should not be assumed to be black

Return type:

numpy.ndarray video frame

property parked

Is the channel parked or running in circular buffer mode

reload()

Reload the configuration of the DMA. Should only be called by the _FrameList class or if you really know what you are doing

reset()

Soft reset the DMA channel

property running

setframe(frame)

Sets a frame without blocking or taking ownership. In most circumstances writeframe() is more appropriate

start()

Start the DMA channel with a blank screen. The mode must be set prior to calling or a RuntimeError will result.

stop()

Stop the DMA channel and empty the frame cache

writeframe(frame)

Schedule the specified frame to be the next one displayed. Assumes ownership of frame which should no longer be modified by the user. May block if there is already a frame scheduled.

async writeframe_async(frame)

Same as writeframe() but yields instead of blocking if a frame is already scheduled

class S2MMChannel(parent, interrupt, memory)

Bases: object

Read channel of the Video DMA

Brings frames from the video input into memory. Hands ownership of the read frames to the user code.

mode

The video mode of the DMA channel

Type:

VideoMode

cacheable_frames

Whether frames should be stored in cacheable or non-cacheable memory

Type:

bool

property activeframe

The frame index currently being processed by the DMA

This process requires clearing any error bits in the DMA channel

property desiredframe

The next frame index to the processed by the DMA

property irqframecount

property mode

The video mode of the DMA. Must be set prior to starting. Changing this while the DMA is running will result in the DMA being stopped.

property parked

Is the channel parked or running in circular buffer mode

readframe()

Read a frame from the channel and return to the user

This function may block until a complete frame has been read. A single frame buffer is kept so the first frame read after a long pause in reading may return a stale frame. To ensure an up-to-date frame when starting processing video read an additional time before starting the processing loop.

Return type:

numpy.ndarray of the video frame

async readframe_async()

Read a frame from the channel, yielding instead of blocking if no data is available. See readframe for more details

reload()

Reload the configuration of the DMA. Should only be called by the _FrameList class or if you really know what you are doing

reset()

Soft reset the DMA. Finishes all transfers before starting the reset process

property running

Is the DMA channel running

start()

Start the DMA. The mode must be set prior to this being called

stop()

Stops the DMA, clears the frame cache and unhooks any tied outputs

tie(channel)

Ties an output channel to this input channel. This is used to pass video from input to output without invoking the CPU for each frame. Main use case is when some slow processing is being done on a subset of frames while the video is passed through directly to the output. Only one output may be tied to an output. The tie is broken either by calling tie(None) or writing a frame to the tied output channel.

bindto = ['xilinx.com:ip:axi_vdma:6.2', 'xilinx.com:ip:axi_vdma:6.3']

pynq.lib.video.drm Module

class pynq.lib.video.drm.DrmDriver(device_path, event_loop=None)

Bases: object

Driver for DRM-based output through the Linux kernel

This driver provides a zero-copy interface to the DRM subsystem exposing a similar API to the HDMI interface. The API should be configured with a PixelFormat containing a FourCC which will be passed to the Linux video subsystem.

Once configured frames can be retrieved using newframe which returns a numpy array mapped to a frame buffer. The frame can be switched using writeframe. Once a frame has been written it should not be modified as ownership has been transferred to the kernel.

close()

Close the display device

configure(mode, pixelformat)

Configure the display output

Raises an exception if the initialisation fails.

Parameters:

• mode (VideoMode) – The resolution to set the output display to

• pixelformat (PixelFormat) – The pixel format to use - must contain a fourcc

newframe()

Return a new frame which can later be written

Frames are not zeroed before being returned so the calling application should make sure the frame is fully written.

Returns:

pynq.PynqBuffer

Return type:

numpy.ndarray mapped to a hardware frame

return_pointer(pointer)

start()

Dummy function to match the HDMI interface

stop()

Dummy function to match the HDMI interface

writeframe(frame)

Write a frame to the display.

Raises an exception if the operation fails and blocks until a page-flip if there is already a frame scheduled to be displayed.

Parameters:

frame (pynq.ContiguousArray) – Frame to write - must have been created by newframe

async writeframe_async(frame)

Write a frame to the display.

Raises an exception if the operation fails and yields until a page-flip if there is already a frame scheduled to be displayed.

Parameters:

frame (pynq.ContiguousArray) – Frame to write - must have been created by newframe

pynq.lib.video.dvi Module

class pynq.lib.video.dvi.HDMIInFrontend(description)

Bases: VideoInFrontend, DefaultHierarchy

Class for interacting the with HDMI input frontend

This class is used for enabling the HDMI input and retrieving the mode of the incoming video

mode

The detected mode of the incoming video stream

Type:

VideoMode

static checkhierarchy(description)

Function to check if the driver matches a particular hierarchy

This function should be redefined in derived classes to return True if the description matches what is expected by the driver. The default implementation always returns False so that drivers that forget don’t get loaded for hierarchies they don’t expect.

property mode

start(init_timeout=60)

Method that blocks until the video mode is successfully detected

stop()

Currently empty function included for symmetry with the HDMIOutFrontend class

class pynq.lib.video.dvi.HDMIOutFrontend(description)

Bases: VideoOutFrontend, DefaultHierarchy

Class for interacting the HDMI output frontend

This class is used for enabling the HDMI output and setting the desired mode of the video stream

mode

Desired mode for the output video. Must be set prior to calling start

Type:

VideoMode

static checkhierarchy(description)

Function to check if the driver matches a particular hierarchy

This function should be redefined in derived classes to return True if the description matches what is expected by the driver. The default implementation always returns False so that drivers that forget don’t get loaded for hierarchies they don’t expect.

property mode

Get or set the video mode for the HDMI output, must be set to one of the following resolutions:

640x480 800x600 1280x720 1280x1024 1920x1080

Any other resolution will result in a ValueError being raised. The bits per pixel will always be 24 when retrieved and ignored when set.

start

Start the HDMI output - requires the that mode is already set

stop

Stop the HDMI output

pynq.lib.video.frontend Module

class pynq.lib.video.frontend.VideoInFrontend(*args, **kwargs)

Bases: object

class pynq.lib.video.frontend.VideoOutFrontend(*args, **kwargs)

Bases: object

pynq.lib.video.hierarchies Module

class pynq.lib.video.hierarchies.HDMIWrapper(description)

Bases: DefaultHierarchy

Hierarchy driver for the entire video subsystem.

Exposes the input, output and video DMA as attributes. For most use cases the wrappers for the input and output pipelines are sufficient and the VDMA will not need to be used directly.

hdmi_in

The HDMI input pipeline

Type:

pynq.lib.video.HDMIIn

hdmi_out

The HDMI output pipeline

Type:

pynq.lib.video.HDMIOut

axi_vdma

The video DMA.

Type:

pynq.lib.video.AxiVDMA

static checkhierarchy(description)

Function to check if the driver matches a particular hierarchy

This function should be redefined in derived classes to return True if the description matches what is expected by the driver. The default implementation always returns False so that drivers that forget don’t get loaded for hierarchies they don’t expect.

class pynq.lib.video.hierarchies.VideoIn(description, vdma=None)

Bases: DefaultHierarchy

Wrapper for the input video pipeline.

This wrapper assumes the following pipeline structure and naming

color_convert_in -> pixel_pack ->axi_vdma with vtc_in and axi_gpio_hdmiiin helper IP

frontend

The HDMI frontend for signal detection

Type:

pynq.lib.video.HDMIInFrontend

color_convert

The input color format converter

Type:

pynq.lib.video.ColorConverter

pixel_pack

Converts the input pixel size to that required by the VDMA

Type:

pynq.lib.video.PixelPacker

property cacheable_frames

Whether frames should be cacheable or non-cacheable

Only valid if a VDMA has been specified

static checkhierarchy(description)

Function to check if the driver matches a particular hierarchy

This function should be redefined in derived classes to return True if the description matches what is expected by the driver. The default implementation always returns False so that drivers that forget don’t get loaded for hierarchies they don’t expect.

close()

Uninitialise the drivers, stopping the pipeline beforehand

property colorspace

The colorspace of the pipeline, can be changed without stopping the pipeline

configure(pixelformat=<pynq.lib.video.common.PixelFormat object>)

Configure the pipeline to use the specified pixel format.

If the pipeline is running it is stopped prior to the configuration being changed

Parameters:

pixelformat (PixelFormat) – The pixel format to configure the pipeline for

property mode

Video mode of the input

readframe()

Read a video frame

See AxiVDMA.S2MMChannel.readframe for details

async readframe_async()

Read a video frame

See AxiVDMA.S2MMChannel.readframe for details

start()

Start the pipeline

stop()

Stop the pipeline

tie(output)

Mirror the video input on to an output channel

Parameters:

output (HDMIOut) – The output to mirror on to

class pynq.lib.video.hierarchies.VideoOut(description, vdma=None)

Bases: DefaultHierarchy

Wrapper for the output video pipeline.

This wrapper assumes the following pipeline structure and naming

axi_vdma -> pixel_unpack -> color_convert -> frontend with vtc_out and axi_dynclk helper IP

frontend

The HDMI frontend for mode setting

Type:

pynq.lib.video.HDMIOutFrontend

color_convert

The output color format converter

Type:

pynq.lib.video.ColorConverter

pixel_unpack

Converts the input pixel size to 24 bits-per-pixel

Type:

pynq.lib.video.PixelPacker

property cacheable_frames

Whether frames should be cacheable or non-cacheable

Only valid if a VDMA has been specified

static checkhierarchy(description)

Function to check if the driver matches a particular hierarchy

This function should be redefined in derived classes to return True if the description matches what is expected by the driver. The default implementation always returns False so that drivers that forget don’t get loaded for hierarchies they don’t expect.

close()

Close the pipeline an unintialise the drivers

property colorspace

Set the colorspace for the pipeline - can be done without stopping the pipeline

configure(mode, pixelformat=None)

Configure the pipeline to use the specified pixel format and size.

If the pipeline is running it is stopped prior to the configuration being changed

Parameters:

• mode (VideoMode) – The video mode to output

• pixelformat (PixelFormat) – The pixel format to configure the pipeline for

property mode

The currently configured video mode

newframe()

Return an unintialised video frame of the correct type for the pipeline

start()

Start the pipeline

stop()

Stop the pipeline

writeframe(frame)

Write the frame to the video output

See AxiVDMA.MM2SChannel.writeframe for more details

async writeframe_async(frame)

Write the frame to the video output

See AxiVDMA.MM2SChannel.writeframe for more details

pynq.lib.video.pipeline Module

class pynq.lib.video.pipeline.ColorConverter(description)

Bases: DefaultIP

Driver for the color space converter

The colorspace convert implements a 3x4 matrix for performing arbitrary linear color conversions. Each coefficient is represented as a 10 bit signed fixed point number with 2 integer bits. The result of the computation can visualised as a table

# in1 in2 in3 1 # out1 c1 c2 c3 c10 # out2 c4 c5 c6 c11 # out3 c7 c8 c9 c12

The color can be changed mid-stream.

colorspace

The coefficients of the colorspace conversion

Type:

list of float

bindto = ['xilinx.com:hls:color_convert:1.0', 'xilinx.com:hls:color_convert_2:1.0']

property colorspace

The colorspace to convert. See the class description for details of the coefficients. The coefficients are a list of floats of length 12

class pynq.lib.video.pipeline.PixelPacker(description)

Bases: DefaultIP

Driver for the pixel format convert

Changes the number of bits per pixel in the video stream. The stream should be paused prior to the width being changed. This can be targeted at either a pixel_pack or a pixel_unpack IP core.For a packer the input is always 24 bits per pixel while for an unpacker the output 24 bits per pixel.

bindto = ['xilinx.com:hls:pixel_pack:1.0', 'xilinx.com:hls:pixel_unpack:1.0', 'xilinx.com:hls:pixel_pack_2:1.0', 'xilinx.com:hls:pixel_unpack_2:1.0']

property bits_per_pixel

Number of bits per pixel in the stream

Valid values are 8, 24 and 32. The following table describes the operation for packing and unpacking for each width

Mode Pack Unpack 8 bpp Keep only the first channel Pad other channels with 0 16 bpp Dependent on resample Dependent on resample 24 bpp No change No change 32 bpp Pad channel 4 with 0 Discard channel 4

property resample

Perform chroma resampling in 16 bpp mode

Boolean property that only affects 16 bpp mode. If True then the two chroma channels are multiplexed on to the second output pixel, otherwise only the first and second channels are transferred and the third is discarded

pynq.lib.video.xilinx_hdmi Module

class pynq.lib.video.xilinx_hdmi.HdmiRxSs(description)

Bases: DefaultIP, VideoInFrontend

Driver for the HDMI receiver subsystem

bindto = ['xilinx.com:ip:v_hdmi_rx_ss:3.1', 'xilinx.com:ip:v_hdmi_rx_ss:3.2']

load_edid(data)

Configure the EDID data exposed by the receiver

The EDID should be between 128 and 256 bytes depending on the resolutions desired. In order to trigger the EDID to be read by the source the HPD line should be toggled after the EDID has been loaded.

Parameters:

data (bytes-like) – EDID data to load

property mode

Return the mode of the attached device

report()

Write the status of the receiver to stdout

set_hpd(value)

Set the Host presence detect line

1 or True advertises the presence of a monitor to the source 0 or False shows a disconnected cable

Parameters:

value (int or Boolean) – The desired value of the HPD line

set_phy(phy)

Set the attached PHY

The subsystem must be attached to a Xilinx HDMI PHY to operate

Parameters:

phy (HdmiVPhy) – The driver for the PHY

start()

Start the receiver

Blocks until the signal is stabilised

stop()

Stop the receiver

class pynq.lib.video.xilinx_hdmi.HdmiTxSs(description)

Bases: DefaultIP, VideoOutFrontend

Driver for the HDMI transmit subsystem

DviMode()

Output using DVI framing

HdmiMode()

Output using HDMI framing

bindto = ['xilinx.com:ip:v_hdmi_tx_ss:3.1', 'xilinx.com:ip:v_hdmi_tx_ss:3.2']

handle_events()

Ensure that interrupt handlers are called

read_edid()

Return the EDID of the attached monitor

Returns:

bytes

Return type:

256 bytes of EDID data

report()

Write the status of the transmitter to stdout

set_phy(phy)

Set the attached PHY

The subsystem must be attached to a Xilinx HDMI PHY to operate

Parameters:

phy (HdmiVPhy) – The driver for the PHY

start()

Start the HDMI output

The mode attribute and the PHY of the driver must be set before the transmitter can be started.

stop()

Stop the HDMI transmitter

wait_for_connect()

Wait for a cable to connected to the transmitter port

class pynq.lib.video.xilinx_hdmi.Vphy(description)

Bases: DefaultIP

Driver for Xilinx HDMI PHY

bindto = ['xilinx.com:ip:vid_phy_controller:2.2']

initialize()

report()

Write the status of the PHY to stdout