Rendering SVG on the GPU

I'm currently working on an extension to Inkscape to do shape recognition and as part of that extension I'm more or less rendering an SVG file, supporting everything that you can have in a path, so basically every SVG shape.  Because of that my plan was to create a very light weight SVG renderer that can be used for prototyping various possible methods of performance rendering of vector graphics, not excluding rendering on the GPU.  I think the current stumbling block that people have come up against when trying to render vector graphics on the GPU, is they see the work, graphics and try to throw graphics at it as opposed to seeing it as a massively parallel vector CPU and throwing vectors at it to process.

    One of the key limitations of GPU processing, I believe, is the inability to allocate memory on the graphics card with the GPU. And instead all buffers have to be pre-allocated, though not necessarily pre-populated. This causes issues with data transport as data has to be transferred between the CPU and GPU over the bus whenever new buffers are needed for processing. The best approach is there for to either perform the calculations needed to calculate the size of the buffers needed for rendering the vectors we have and or to render the vectors' ion bite size chunks of predetermined buffer footprints. To determine which of the methods is possible and how to go about calculating the buffer sizes we need to address the approaches that can be used for rendering vectors on the GPU.
OpenCL provides a good around method of balancing intensive work between the GPU and CPU.
 The primary components of Vector line drawing in SVG are as follows:

<filter>
<circle>, <ellipse>, <line>, <path>, <polygon>, <polyline>, <rect>  all of which are kinds of <path>
<text>, <textPath>
<image>
<linearGradient>, <radialGradient>, <hatch>, <hatchpath>, <symbol>, <solidcolor>, <meshgradient>, <meshpatch>, <meshrow>
<mask>

I'll go over each one of them in order and look at how they can be rendered using the GPU, or a combination of the GPU and CPU.


<filter>
There is a OpenCL implementation of SVG filters to go on.
 https://github.com/rsnemmen/OpenCL-examples/tree/master/sum_array
 https://bugs.webkit.org/show_bug.cgi?id=70099
 https://bugs.webkit.org/show_bug.cgi?id=107444
 https://www.chromium.org/developers/design-documents/image-filters

, so we know we can do filters on the GPU, so lets look at the other components of SVG.


 <path> etc..
 Attributes of a path relating to drawing are:
 listed here: https://developer.mozilla.org/en-US/docs/Web/SVG/Element/path under presentation attributes.
 A full list of presentation attributes can be found here:
 https://developer.mozilla.org/en-US/docs/Web/SVG/Attribute/Presentation


    Can be rendered using Bézier splines for everything except arcs (though a quadratic could be able to do that). Bézier's splines can be processed as an array of vector operations. The size of the buffers can be pre-calculated by knowing the length of the path line segments and allowing one buffer element per ~pixel sizes of the vectors to hold the path segment lengths can also be pre-calculated, one per path segment. Calculating path length is relatively CPU intensive but can be performed in parallel, one path segment per thread. It is however low memory footprint so not really viable for executing on the GPU due to BUS speeds. Bézier's splines can be broken down into two iterations of a * p[0] + b * p[2] + c * p[4] + d * p[6] per step  with the fractional component used to calculate a,b,c and d having one step per pixel for perfect fidelity, though this could be reduced where studio quality isn't required or the user's system isn't of high enough specification, and they haven't desired studio quality.  The 8 values of p are fixed for the segment run buffers of the correct size to hold one fractional value per-step, or per-pixel would have to be pre-allocated and may require pre-population of fractions to perform the Bézier calculations. It may however be possible to do all of this on the GPU, leaving the CPU only needing to calculate the segment lengths, allocate the buffers, and populate vectors with the segment lengths and the relevant p-values.  The GPU could then populate the buffers with the calculated values for the position of each fraction of each Bézier of each segment of each path. This list would then contain all the positional details needed to render the <path> components of an SVG file as line graphics. The following presentation attributes of the <path> would then need to be handled:

<clip-path>, <clip-rule>

<color>, <color-interpolation>, <color-rendering>, <opacity>, <visibility>, <stroke-opacity>, <stroke>, <stroke-opacity>

<stroke-width>, <stroke-dasharray>, <stroke-dashoffset>, <vector-effect>

<fill>, <fill-opacity>, <fill-rule>

<filter>, <mask>

<color>,  etc...
assuming changing, can be calculated along with the spline coordinates and stored in per-fraction buffers.

<stroke-width> etc...
    Requires that the line not only have a pixel position but also a with, the best way of drawing a width on a GPU is to use a triangle stripe that extends half the normal of the Bézier in either direction for half the stroke width.. This also allows for sub, pixel widths. Calculating the normal for the Bézier can be done relatively easily for each fractional component, meaning instead of having a buffer of 32bits x 2 per pixel you would need buffers to contain triangular stripes.
 The triangular stripes could then be rendered by the GPU with the correct <color> etc... assignments, or indeed anything a GPU can render a triangle as which is not limited to lighting, textures and shaders.

<fill> etc...
    Fill may prove to be difficult, the path segments will need to be triangulated to be able to do this feasibly on the GPU, unless some way of sorting the Bézier buffers and iterating over them to produce <fill> stripes can be found.

<fill>, <text>
    I don't think the task of triangulation lends itself well to the GPU, though there are two projects that support Bézier text rendering SLUG and pathfinder http://jcgt.org/published/0006/02/02/
 https://github.com/servo/pathfinder

    Filling algorithms seem to have been developed for Bézier-based glyphs at least, though their performance may cause issues.  <filter> as we have already mentions is supported via OpenCL and Masking can be supported when rendering the triangle strips onto a surface. We can say from SLUG and pathfinder that <text> can be supported

 <image> etc...

    Just a plane image, so is supported by rendering a texture onto a surface using a rectangle described by two triangles.

That just leaves:

<gradients>  etc...
    Which should be supportable be standard shaders, meaning they will be supportable in an OpenCL implementation.



On top of this, is implementing a caching mechanism for the drawable SVG that can be clipped and provides good performance when the SVG contents is changed. Either edited or animated.

So, in summary, the only two operations of SVG rendering that are not known to be able to be performed on the GPU are clipping and calculation of Bézier lengths to determine buffer sizes.