BVR's Home DelTA Lab IIT Kanpur

Can I teach a robot to replicate a line art
Raghav BV, Subham Kumar, Vinay P Namboodiri

(PDF) (arXiv) (code coming soon…)


Figure 1: Left to Right. An artist drawing; digital copy of the drawing; segmented version; drawing by a CNC plotter.

How to cite
        title = {Can {I} teach a robot to replicate a
                  line art},
        booktitle = {Proceedings 26th {IEEE} {Winter}
                  {Conference} on {Applications} of
                  {Computer} {Vision} ({WACV})},
        author = {Raghav, B.V. and Kumar, Subham and
                  Namboodiri, Vinay P},
        year = {2020},

Line art is arguably one of the fundamental and versatile modes of expression. We propose a pipeline for a robot to look at a grayscale line art and redraw it. The key novel elements of our pipeline are: a) we propose a novel task of mimicking line drawings, b) to solve the pipeline we modify the Quick-draw dataset to obtain supervised training for converting a line drawing into a series of strokes c) we propose a multi-stage segmentation and graph interpretation pipeline for solving the problem. The resultant method has also been deployed on a CNC plotter as well as a robotic arm. We have trained several variations of the proposed methods and evaluate these on a dataset obtained from Quick-draw. Through the best methods we observe an accuracy of around 98% for this task, which is a significant improvement over the baseline architecture we adapted from. This therefore allows for deployment of the method on robots for replicating line art in a reliable manner. We also show that while the rule-based vectorization methods do suffice for simple drawings, it fails for more complicated sketches, unlike our method which generalizes well to more complicated distributions.

The Network Architecture

We reinterpret the U-Net architecture with modular blocks for downsampling, upsampling and bottom, as a 4-tuple \((k_1\, k_2\, r\, d)\) for easy parameterization and architecture search.


Figure 2: Reinterpretation of the U-Net architecture.


We take the Quick-draw dataset and synthesise the input as sketches, and output as their segmentation mask.


Figure 3: Top. Samples from the Quick-draw dataset; Bottom. Rendered true labels.

Qualitative Results

Here is a list of qualitative results. First we compare the quality of our segmentation model against a popular vectorization method.


Figure 4: Comparison of our segmentation (left column) against Favreau’s vectorization (right column). Top to Bottom. A simple line sketch from Favreau’s examples; A simple line sketch from Quick-draw dataset; A complicated hatch pattern with zoomed in details.

Then we show the results of two of our graph interpretation models. The former model uses a set of fixed global parameters, and the latter updates parameters for every edge using feedback from comparison with the input image.


Figure 5: The top row shows the input. The subsequent rows show reconstruction using a graph interpreted from segmentation mask, with values of mask width and threshold \((\beta,\, \tau)\) set as \((2,\, 0.3)\), \((3,\, 0.22)\), \((5,\, 0.2)\), \((7,\, 0.15)\) respectively downwards.


Figure 6: The results of feedback loop in graph interpretation. Rows top to bottom. the input image; naive graph interpretation; interpretation after \(N = 5\) updates; and after \(N = 10\) updates.

Updated 2019-11-13 Wed 18:08