natural-looking strokes for drawing applications

Visual Comput (2006) 22: 415–423DOI 10.1007/s00371-006-0021-7 O R I G I N A L A R T I C L E

Kyoko MurakamiReiji TsurunoEtsuo Genda

Natural-looking strokes for drawingapplications

Published online: 9 May 2006© Springer-Verlag 2006

K. Murakami (�)Tsuruno lab., Faculty of Design, KyushuUniversity, Fukuoka, [email protected]

R. Tsuruno · E. GendaFaculty of Design, Kyushu University,Fukuoka, Japan{genda,tsuruno}@design.kyushu-u.ac.jp

Abstract This paper presents analgorithm for generating realisticdrawing strokes in real-time that cantake on the appearance of pastels,charcoals, or crayons. The similaritybetween the pigment deposit patternson paper surfaces produced by pastelstrokes and the shadows/shades onilluminated paper surfaces have beeninvestigated. Multiple paper textureshave been prepared and the papersurfaces have been ascertained by

illumination from various directionsto represent strokes in arbitrarydirections. These textures havebeen processed as if they could beused as a height field, and pigmentsdeposited on the paper have beencalculated using the height field andtablet inputs.

Keywords NPR · Stroke · Drawing ·Multiple paper texture

1 Introduction

The modeling of traditional drawing/painting materialsand techniques is an ongoing and open-ended topic inthe field of non-photorealistic rendering (NPR). A varietyof characteristics can be expressed in the final renderedimages through elaborate simulations. Because most realpictures are constructed through a series of strokes, manyNPR studies have focused on modeling the strokes them-selves. For each individual work, the stroke that is inherentin each painting/drawing tool is generated specifically foreach system and is affected by brush types, support media,and the interaction of these materials.

When using drawing tools that have soft tips, like pen-cils, pigments are scraped from the tip and then physicallydeposited onto the surface of the support media. In thiscase, paper is often used as the support medium. Whenthe tip of the drawing tool is softer than that of pen-cils, including pastels, charcoals, chalks, or crayons, thepaper grains are not flattened by the stroke and the pa-per texture becomes visible. Realistic art tools cannot berepresented accurately without simulating the interactionbetween such materials. However, elaborate simulation re-

quires considerable computation time for a drawing toolthat is to work in real-time. In this work, realistic draw-ing strokes are generated in real-time using twelve illu-minated paper surface samples. Instead of calculating theinteraction between each pigment and paper, the grada-tion of the illuminated paper surfaces is used as the heightfield, which requires less time. The method of Northrupand Markosian [12] is basically employed to generate thedrawing tools (a stroke and an eraser).

2 Related works

In several NPR studies on drawings, the results are highlyaffected by the support medium because of soft-firmedpigments, including pencils [16], colored pencils [18],charcoals [2], crayons [15] and pastels [11]. These stud-ies can be divided into two categories: texturing methodsand media-simulation methods. Several authors have pro-posed a texturing method for soft tips and pigments [2, 8]and [5]. Bleser et al. [2] generated a stroke from sampleimages of real charcoal strokes, and regenerated virtualcharcoal strokes by combining these sample images using

416 K. Murakami et al.

information from inputs generated by a five degree-of-freedom stylus. Scanning and regeneration methods havealso been used in some studies. Kalnins et al. [8] ren-dered various crayon and pencil strokes with a texturecreated from scanned images of several real strokes. Du-rand et al.[5] generated a threshold matrix from a scannedimage and represented various pen pressures and overlap.These texture-based methods are beneficial for both real-time rendering and generating realistic strokes. However,in real drawings, the textures that appear on strokes areaffected by various elements, such as pen pressure varia-tion, pen pressure distribution on the cross-section, cross-section form variation due to rubbing down or collapsingtips, and stroke direction. Several stroke samples cannotadequately represent the entire span of variation. Media-simulation methods of drawing tools can sufficiently rep-resent complicated pigment behavior [15, 16, 18] and [11].In some studies, all of the simulated media suitable for thesystem are generated. Takagi et al. [18] generated paperusing a volumetric approach that considered its construc-tion. They approximated colored pencil tips as spheresand used “offset distance accessibility” (ODA) [10] tomodel pigment deposition. Sousa and Buchanan’s papermodel [16] uses the height field from digitized real paperor photographed paper samples converted from 0.0 to 1.0.They also generated the 3D height field procedurally, asdetailed by Curtis et al. [4]. The height field is used in bothfaithful and simple simulations in research on various arttools. Tips of all pencil drawing tools are defined as poly-gons that have the pencil pressure at the center and the ver-tices of the polygon. Leads are scraped when they comeinto contact with the higher parts of the paper surface.The NPR by Rudolf et al. [15] defined the wax depositedby crayons on paper as the overlapping cross-sections ofthe crayon and the paper. Murakami and Tsuruno [11] de-posited particles of pigment on a paper surface to generatepastel strokes. All of these methods simulate drawing toolsthat allow for arbitrary stroke styles. Unfortunately, thesemethods also include some drawbacks. Suitable interac-tion is needed between the paper surface and the drawingtools in order to effectively regenerate the strokes, and nu-merous parameters are carefully set based on actual data toproduce “realistic” results. Furthermore, elaborate simula-tion requires considerable processing time in some cases,making it difficult to draw in real-time without a high-endwork station or GPU. For example, the crayon renderingof Rudolf et al. requires up to four seconds per strokewhen calculated on a 2.4-GHz Pentium 4. Computer gen-erated strokes are commonly used in existing paintingapplications at a real-time rate. Painter [3] determined theappearance of strokes according to the paper texture, andtheir shapes are a train of the stylus cross-section. Pho-toshop also uses the cross-section stylus for stroke shapedecisions, and adds slight rotation and translation to itsplacement. The strokes in Illustrator [7] are assigned ac-cording to a vector pattern placed along the base path.

However, these strokes do not adequately resemble thoseproduced by real art tools. The proposed method simu-lates pigment deposition of drawing strokes by texturingthat combines these two approaches. Based on the strokegeneration for the real-time 3D NPR system, we have de-veloped a method for a drawing system that requires bothreality and interactivity.

3 Paper model

This paper focuses on the similarity between the appear-ance of drawn pastel strokes and paper texture that is illu-minated by a light. Multiple illuminated paper textures areused instead of simulating the interactions between toolsand paper surfaces. This method obtains realistic strokesthat can be used in real-time systems such as drawing ap-plications.

3.1 Illuminated paper texture

This section takes a case where a pastel stroke is drawnon paper as an example. The pastel is a typical drawing

Fig. 1. Pastel strokes drawn in different directions

Natural-looking strokes for drawing applications 417

Fig. 2. Microstructure of paper and deposited pigments

Fig. 3. Height field from the paper height/illumination

tool and is comprised primarily of pigments. Pastel strokesdrawn in different directions on a paper surface generatesdifferent pattern appearances, as shown in Fig. 1.

Fig. 2 shows the zoom up of the paper and the pig-ment deposition by a pastel stroke as observed by animage scanner. When a stroke is drawn on paper, pigmentsscraped by irregularities in the paper are deposited notonly on top of the convex area, but also on the area nearthe top to a greater extent. Therefore, to generate a realis-tic stroke representation using the height field as a paperheight, a careful simulation that sometimes incorporates

Fig. 4. The interface of thedrawing application

the interaction between the drawing tool and the papersurface is necessary. In contrast, when an irregular papersurface is illuminated from one direction, causing shadeand a shadow to appear, the brightest part is not the highestpart, but an area slightly lower than the highest part in thedirection of the light. Fig. 3 represents this phenomenonusing a pattern diagram that depicts a cross-section of thepaper surface and its height field. Based on this similarity,the illuminated paper texture is used as the height field forsimulating pigment deposition by a stroke drawn in one di-rection. When a stroke is drawn, in fact, the stroke usesa height field consisting of the converted paper texture il-luminated by a light coming from the same direction as thestroke. In the presented application, twelve paper texturesare designated as key textures and interpolations betweenthem are performed for an arbitrary drawing direction.

3.2 Preparing paper texture

Paper textures are captured by the equipment shown inFig. 4. A camera is positioned above a piece of paper thatis attached to the flat ground. In order to illuminate thepaper in such a way as to distinguish paper irregularities,a light source is mounted at a suitable position with an el-evation angle of 45 degrees. The light intensity is arbitraryunless it obscures paper features. Next, the twelve surfaceimages are prepared. For each image, the paper surface isilluminated by a single light source with the azimuth set inincrements of 30 degrees from 0 to 360 degrees.

Five types of paper textures were captured: drawingpaper, canson mi-tientes paper, mermaid paper, OK muse-cotton paper, and Watson paper. The sizes of the papersamples depend on the resolution and accuracy of the cap-turing process. The selected areas contain no warp causedby camera lens on the photographed paper image. As a re-sult, they range from 100 ×100 to 300 ×300 pixels. Thedrawing paper samples are shown in Fig. 4. In this fig-ure, the images are converted to 0-1 gray values, so thatthe minimum value in the 12 original images becomeszero and the maximum value becomes 1. This value isthen used as the deposit efficiency of the pigments. Theseconverted paper surfaces are referred to as paper tex-ture Ti, and each texture Ti has a direction vector to light


source Vt . The paper texture is tiled on the pigment bufferwith a wrap-over at the edge. The pigment buffer is thesame width and height as the resulting image, and the pig-ments are stored within it. The final image depends on theamount of pigment deposited in a pigment buffer.

4 Implementation

A stroke is defined as a stroke mask. Inside the strokemask, the number of pigments deposited in the pigmentbuffer is calculated by considering the interaction betweenthe pen pressure and the paper texture. All of the tools areproduced using basically the same method.

4.1 Stroke mask

There are various methods for generating strokes [6, 14,17]. In this study, a two-dimensional stroke mask based onthat of Northrup and Markosian [12] was used, which isideally defined as a train of cross-sectional surfaces. The“base path” consists of control points that are denotedby a mouse or tablet. WinTab API (LCS/TelegraphicsConfigdential) [9] is used for obtaining information froma tablet as packets. In the computer environment, the num-ber of packets is large enough to attain a smooth shapeusing only the linear interpolation between them. How-ever, a Catmull-Rom spline also can be used, such as thatof Kainins et al. [8], in the case when sparse control pointsneed to be defined. A stroke width is then added to bothsides of the base path. It is placed perpendicularly to thestroke vector that is calculated by two contiguous con-trol points. The locations for the stroke width are alsocalculated using a pen azimuth, as they are perpendicu-lar to the direction of pen azimuth. The pen azimuth is 0when the tip of a stylus indicates the top of the tablet, andreaches a maximum when the tip indicates the bottom. Thestroke contour is then defined as the stroke mask consist-ing of a triangle strip from the base path and the strokewidth. It is rare that a constant pressure is applied whena stroke is drawn. The convex part at the cross-sectionalsurface of the drawing tool receives more pressure thanthe other points, causing more pigment to be depositedon the paper surface. Furthermore, most drawing tools arefragile: convex parts are immediately flattened, causingnew convex points to appear. This results in variation ofgradation inside a stroke. In order to accurately representthis effect, a maximum pressure line segment consistingof maximum pressure points at each cross-section alongthe base path is implemented, as shown in Fig. 5. At eachtransverse of the stroke mask, a local 2D coordinate sys-tem is assigned in which x indicates the pressure positionc (0 for the rightmost side and 1 for the leftmost side ofthe stroke mask in the stroke direction) and y indicatesthe pen pressure F set vertically to the stroke mask. Then,

Fig. 5. Stroke definition

the maximum pressure points are set on the cross-sectionsthat contain the control points. The value c that decides thepositions of maximum pressure points are calculated bythe Perlin noise function [13], which is frequently used inrepresenting hand drawn-like appearance. The maximumpressure line segment is calculated by linear interpolationbetween the maximum pressure points. At the transversesof a stroke, a linear or a quadratic curve is used to inter-polate the c values at the maximum pressure line segmentand both sides where a minimum value exists. The min-imum value is set to 2/3 of the maximum value in ourexperiment. Controlling the coefficient and the intercept ofthe functions that set c values containing a maximum anda minimum value allows for variation in the stroke.

4.2 Selecting paper texture

After the stroke mask is set, the paper texture is applied.As described above, the paper texture that appears ona stroke varies with the stroke direction. Therefore, the ap-propriate drawing direction vector must be selected to ob-tain the appropriate paper texture. Furthermore, the pres-sure distribution displaces pigments inside the stroke. Thisis approximated by considering the movement of the max-imum pressure points on the stroke segments along thestroke when selecting the paper texture. When a stroke isdrawn in one direction, its direction vector Vs is used to se-lect two paper textures T1 and T2, which have Vt1and Vt2similar to Vs:

Gs = Gt1(1− ratio)+ Gt2 ratio , (1)

with Gs: the gray value for the stroke, and

ratio = (Vs − Vt1)/(Vt2 − Vt1) . (2)


Fig. 6. Selecting paper textures by stroke vector

This process is shown in Fig. 6. The same processusing pressure movement direction vector Vp is used forselecting textures for the maximum pressure line segmentand the gray value Gp. The resulting gray value G is theaverage of Gs and Gp, which is used as the height field.The pigments that are deposited in the pigment buffer de-pend on pen pressure F and the height field.

THRESHOLD = 1− Fif (G < THRESHOLD) deposit pigments

The displayed color is equal to the original color, butindependent of the quantity of pigments. To prevent alias-ing when the difference between the gray value G andTHRESHOLD is small (defined as less than 0.02), trans-parency is given to the pigments according to the differ-ence:

TRANS = 1− (THRESHOLD+0.02− G) DEC , (3)

where DEC is the attenuation value. The transparencyvalue TRANS varies between 0 (transparent) and 1(opaque).

4.3 Additional effects

4.3.1 Overlap

If pigments are already deposited on a paper, it is difficultto deposit new pigments, because the paper irregularitiesare filled with the pigments on the paper surface in a realdrawing. To account for this effect, the new pigments aredeposited in an inverse proportion as follows:

deposit pigments = (Pm − Ps)/Pm Pn , (4)

with Ps being the amount of pigments stored in the pig-ment buffer, Pm the maximum pigments able to be storedin the pigment buffer, and Pn the amount of pigments thatare scraped by drawing. The proportion of the amount ofdeposit pigments and Pn is used as proportion of colorblending. Some works use the Kubelka–Munk modelthat approximates translucent pigments for color blend-ing [15]. Instead, this study uses simple subtractive color

mixing for short-time calculation such as that of Baxter etal. [1].

4.3.2 Width variation

The widths of real strokes vary with pen pressure and thetilt the tool from the paper surface. In most cases, thestroke width becomes broader when the pen pressure ishigh and the tilt is small, and becomes narrow when thepen pressure is low and the tilt is large. In this paper, thestroke width is alternated from wmin to wmax by multi-plying the stroke width by pen pressure F and tilt altitude,

Fig. 7. Strokes with multiple paper textures

Fig. 8. Strokes with different pressure interpolation


Fig. 9. Comparison betweenreal pastel strokes (each left)and our strokes (each right)

as described below:

//width variation achieved by varying the pen pressurewidth = wmin+ (int)(wmax−wmin) F scale;//width variation achieved by varying the pen tiltwidth ← width/altitude;where scale is a scale factor of width variation from 0 (nowidth variation by pen pressure) to 1 (pen pressure affectsthe width variation strongly).

4.3.3 Eraser

Pigments on the pigment buffer can be removed by aneraser mask that is defined using the same method as the

stroke mask. The amount of removed pigment is deter-mined by the pen pressure. In real drawings, an eraserflattens the paper surface. In this study, Pm was reducedwhere an eraser mask is applied, making it difficult to de-posit new pigments on these planes. However, this processis optional, because it is often preferable to redraw be-cause the pigment is difficult to deposit after erasures inmany drawing and painting applications.

5 Results and discussion

All of the experiments were performed using a Pentium4 2.0-GHz, 512-MB RAM in real-time. The experimentswere also performed using lower-performance machines,


Fig. 10. Comparison of overlapping

such as a Celeron 1.0-GHz computer having 256 MBRAM, without a GPU. Figure 7 shows the effects of mul-tiple paper textures on the resulting image. The strokesare drawn using the height field with different paper tex-ture numbers and using the same path. From top to bot-tom, 1, 2, 4, and 12 textures were used for generating theheight field. As the number of textures increase, the pat-terns on the texture decrease. This is because the presentmethod deposits the pigments on different areas of thepaper surface along the drawing directions. Stroke varia-tion using a pen pressure interpolation is shown in Fig. 8.Strokes (a) and (b) use basic linear interpolation, with (b)maintaining the maximum pressure points on one edge.Strokes (c) and (d) use quadratic curves with differentcoefficients for interpolation. Fig. 9 shows a comparisonbetween real strokes and virtual strokes produced by the

Fig. 12. Drawing examples usingour application

Fig. 11. The interface of the drawing application

proposed method. For this comparison, pastel strokes wereused for the real drawing stroke samples because they arehighly affected by paper surfaces without collapsing thepaper structure. Realistic strokes, similar to those drawnon (a) drawing paper, (b) OK muse-cotton paper and (e)Watson paper were generated. Multiple paper textures ina stroke reveal a “crater” that is produced by the inter-action between paper and pigments. On (c) mermaid pa-per and (d) canson mi-tientes paper, however, the strokescould not accurately render the paper textures. The dif-ference between paper textures lies in the scale of paperirregularity: the paper surfaces of papers (a), (b) and (e)are shallower than paper types (c) and (d). Two solutionsto this problem are considered. The first solution involvessetting the converting parameters more carefully, while thesecond involves adjusting the incident angle of illumina-tion. Because the incident angle of illumination affectsthe paper shadow and shade, the optimum angle is dif-ferent for different types of paper. The similarity betweenthe regenerated strokes and the real pastel strokes also de-pends on the resolution of the paper textures. Overlappingstrokes are shown in Fig. 10. In both the real (a, b) andvirtual (c, d) images, the pink strokes are drawn over theblue strokes. There is some similarity in the figure panels


in that the pink pigments cannot be deposited on the paperin areas where the blue pigments have already filled the ir-regularities in the paper. However, in real pastel drawing,the second pink stroke deposits the blue pigment in otherareas. The lack of this effect reduces the reality of the finaldrawing. The application shown in Fig. 11 was developedfor easier control of all variables in stroke generation.Using the interface, a user can control the minimum andmaximum stroke widths, the stroke and paper color, thepaper type, and the pressure interpolation type (linear orquadric with a coefficient). Fig. 12 shows some exampleimages using the application.

6 Conclusion and future works

This paper has presented a method for regenerating therealistic strokes of an artistic tool that is highly affected

by the irregularities of the support medium. The heightfield value, which is calculated from the gray scale of il-luminated paper surfaces, is similar to a real pastel strokedrawing in any direction. This texture-based simulationgenerates “realistic” pastel strokes in real-time. The samemethod was used to generate an eraser and a blender, al-lowing for the creation of a pastel-like drawing tool. Sev-eral issues remain with regard to the proposed pastel draw-ing tool. As shown by Fig. 9, sometimes the strokes couldnot accurately render the paper textures. Further investi-gations are required to achieve realistic and simple strokegeneration. Moreover, the proposed method does not pro-vide specialized drawing tools. For example, blendingis important for pastel drawing and smoother smudgingshould generate better resulting images. Smudging effectsvary according to the blender, and artists often use pastelsin a powdery state. To implement these effects, the strokesoftness inherent in pastel drawings must be considered.

References1. Baxter, B., Scheib, V., Lin, M.C.,

Manocha, D.: Dab: interactive hapticpainting with 3D virtual brushes. In:SIGGRAPH ’01: Proceedings of the 28thAnnual Conference on Computer Graphicsand Interactive Techniques, pp. 461–468.ACM Press, New York (2001). DOIhttp://doi.acm.org/10.1145/383259.383313

2. Bleser, T.W., Sibert, J.L., McGee, J.P.:Charcoal sketching: returning control to theartist. ACM Trans. Graph. 7(1), 76–81(1988). DOIhttp://doi.acm.org/10.1145/42188.42230

3. Corp., C.: Corel painter.http://www.corel.com/

4. Curtis, C.J., Anderson, S.E., Seims, J.E.,Fleischer, K.W., Salesin, D.H.:Computer-generated watercolor. In:SIGGRAPH ’97: Proceedings of the 24thAnnual Conference On Computer GraphicsAnd Interactive Techniques, pp. 421–430.ACM Press/Addison-Wesley PublishingCo., New York (1997). DOIhttp://doi.acm.org/10.1145/258734.258896

5. Durand, F., Ostromoukhov, V., Miller, M.,Duranleau, F., Dorsey, J.: Decouplingstrokes and high-level attributes forinteractive traditional drawing. In:Proceedings of the 12th EurographicsWorkshop on Rendering Techniques,pp. 71–82. Springer, London (2001)

6. Hsu, S.C., Lee, I.H.H.: Drawing andanimation using skeletal strokes. In:SIGGRAPH ’94: Proceedings of the 21st

Annual Conference On Computer GraphicsAnd Interactive Techniques, pp. 109–118.ACM Press, New York (1994). DOIhttp://doi.acm.org/10.1145/192161.192186

7. inc., A.S.: Adobe illustrator.http://www.adobe.com/

8. Kalnins, R.D., Markosian, L., Meier, B.J.,Kowalski, M.A., Lee, J.C., Davidson, P.L.,Webb, M., Hughes, J.F., Finkelstein, A.:Wysiwyg npr: drawing strokes directly on3D models. In: SIGGRAPH ’02:Proceedings of the 29th Annual ConferenceOn Computer Graphics And InteractiveTechniques, pp. 755–762. ACM Press, NewYork (2002). DOIhttp://doi.acm.org/10.1145/566570.566648

9. LCS/Telegraphics: Wintab10. Miller, G.: Efficient algorithms for local

and global accessibility shading. In:SIGGRAPH ’94: Proceedings of the 21stAnnual Conference On Computer GraphicsAnd Interactive Techniques, pp. 319–326.ACM Press, New York (1994). DOIhttp://doi.acm.org/10.1145/192161.192244

11. Murakami, K., Tsuruno, R.: Pastel-likerendering considering the properties ofpigments and support medium. In: GRAPH’02: Proceedings of the SIGGRAPH 2002Conference on Sketches, p. 227. ACMPress, New York (2002)

12. Northrup, J.D., Markosian, L.: Artisticsilhouettes: a hybrid approach. In: NPAR’00: Proceedings of the 1st InternationalSymposium on Non-photorealistic

Animation and Rendering, pp. 31–37. ACMPress, New York (2000). DOIhttp://doi.acm.org/10.1145/340916.340920

13. Perlin, K.: An image synthesizer. In:SIGGRAPH ’85: Proceedings of the 12thAnnual Conference On Computer GraphicsAnd Interactive Techniques, pp. 287–296.ACM Press, New York (1985). DOIhttp://doi.acm.org/10.1145/325334.325247

14. Pham, B.: Expressive brush strokes.CVGIP: Graph. Models Image Process.53(1), 1–6 (1991).DOI http://dx.doi.org/10.1016/1049-9652(91)90013-A

15. Rudolf, D., Mould, D., Neufeld, E.:A bidirectional deposition model of waxcrayons. Comput. Graph. Forum 24(1),27–39 (2005)

16. Sousa, M.C., Buchanan, J.W.: Observationalmodels of graphite pencil materials.Comput. Graph. Forum 19(1), 27–49(2000)

17. Strassmann, S.: Hairy brushes. In:SIGGRAPH ’86: Proceedings of the 13thAnnual Conference On Computer GraphicsAnd Interactive Techniques, pp. 225–232.ACM Press, New York (1986). DOIhttp://doi.acm.org/10.1145/15922.15911

18. Takagi, S., Nakajima, M., Fujishiro, I.:Volumetric modeling of colored pencildrawing. In: PG ’99: Proceedings of the 7thPacific Conference on Computer Graphicsand Applications, p. 250. IEEE ComputerSociety, Washington, DC (1999)


KYOKO MURAKAMI received her BS, MSand PhD degrees in Design from the KyushuInstitute of Design in 2000, 2002 and 2005.Her reserach interest include non-photorealisticrendering and computer graphics.

REIJI TSURUNO has been an Associate Pro-fessor in the Faculty of Design at KyushuUniversity since 2003. His research interestsinclude techniques of computer graphics andnon-photorealistic rendering. He received hisMS and PhD degrees in Computer Science fromOsaka Prefecture University in 1987 and 1993.

ETSUO GENDA has been a Professor in theFaculty of Design at Kyushu University since2003. His research interests include educationof design and media art. He received his Mas-ter’s degree in Fine Arts from Tokyo NationalUniversity of Fine Arts and Music in 1976.He is a chairman of the Asia Digital Art andAssociation (ADAA) and a board member ofthe Japanese Society of Science and Design(JSSD).

natural-looking strokes for drawing applications

Documents