BVR's Home DelTA Lab IIT Kanpur

Can I teach a robot to replicate a line art
Raghav BV, Subham Kumar, Vinay P Namboodiri

(PDF) (arXiv) (code coming soon…)

teaser.png

Figure 1: Left to Right. An artist drawing; digital copy of the drawing; segmented version; drawing by a CNC plotter.

How to cite
@inproceedings{RKN19,
        title = {Can {I} teach a robot to replicate a
                  line art},
        booktitle = {Proceedings 26th {IEEE} {Winter}
                  {Conference} on {Applications} of
                  {Computer} {Vision} ({WACV})},
        author = {Raghav, B.V. and Kumar, Subham and
                  Namboodiri, Vinay P},
        year = {2020},
}
Abstract

Line art is arguably one of the fundamental and versatile modes of expression. We propose a pipeline for a robot to look at a grayscale line art and redraw it. The key novel elements of our pipeline are: a) we propose a novel task of mimicking line drawings, b) to solve the pipeline we modify the Quick-draw dataset to obtain supervised training for converting a line drawing into a series of strokes c) we propose a multi-stage segmentation and graph interpretation pipeline for solving the problem. The resultant method has also been deployed on a CNC plotter as well as a robotic arm. We have trained several variations of the proposed methods and evaluate these on a dataset obtained from Quick-draw. Through the best methods we observe an accuracy of around 98% for this task, which is a significant improvement over the baseline architecture we adapted from. This therefore allows for deployment of the method on robots for replicating line art in a reliable manner. We also show that while the rule-based vectorization methods do suffice for simple drawings, it fails for more complicated sketches, unlike our method which generalizes well to more complicated distributions.


The Network Architecture

We reinterpret the U-Net architecture with modular blocks for downsampling, upsampling and bottom, as a 4-tuple \((k_1\, k_2\, r\, d)\) for easy parameterization and architecture search.

skeunet-block.png

Figure 2: Reinterpretation of the U-Net architecture.

Dataset

We take the Quick-draw dataset and synthesise the input as sketches, and output as their segmentation mask.

dataset.png

Figure 3: Top. Samples from the Quick-draw dataset; Bottom. Rendered true labels.

Qualitative Results

Here is a list of qualitative results. First we compare the quality of our segmentation model against a popular vectorization method.

favreau-comparison.png

Figure 4: Comparison of our segmentation (left column) against Favreau’s vectorization (right column). Top to Bottom. A simple line sketch from Favreau’s examples; A simple line sketch from Quick-draw dataset; A complicated hatch pattern with zoomed in details.

Then we show the results of two of our graph interpretation models. The former model uses a set of fixed global parameters, and the latter updates parameters for every edge using feedback from comparison with the input image.

graph-interp-fixed-tau_white_4c.png

Figure 5: The top row shows the input. The subsequent rows show reconstruction using a graph interpreted from segmentation mask, with values of mask width and threshold \((\beta,\, \tau)\) set as \((2,\, 0.3)\), \((3,\, 0.22)\), \((5,\, 0.2)\), \((7,\, 0.15)\) respectively downwards.

graph-reiter_white_4c.png

Figure 6: The results of feedback loop in graph interpretation. Rows top to bottom. the input image; naive graph interpretation; interpretation after \(N = 5\) updates; and after \(N = 10\) updates.

Updated 2019-11-13 Wed 18:08

Validate