dealing with multi-language garbage? data - sap q&a€¦ · ¤sap teched ’05 / im101 / 4...

Session ID: IM101Dealing with Multi-Language Garbage?Data – Lessons Learned

Hideki YamamotoSAP Shared Services & Center of ExpertiseInformation TechnologyHewlett-Packard Company

IntroductionFundamentals of Multi-Language data / garbagedata in SAP systemsExamplesSummary

SAP TechEd ’05 / IM101 / 4

Introduction

Hideki YamamotoSAP Shared Services & Center of Expertise, Information Technology atHPFocusing on SAP Internationalization, Multi-Language solution since R/3Version 2.x

First R/3 with MNLS for an worldwide project in 1994/1995Working in the “Multi-byte” world - Came from DEC Japan to the U.S.

Apply Multi-language solutions for our global systems at HPMNLS, MDMP, Blended codepage, Unicode etc.with SAP Globalization team.

ASUG Global Implementation SIG


Introduction – SAP Implementations at HP

History of the Multi-Language enabled SAP system at HP.. So far..

We had/have MNLS, Blended codepage, MDMP, Unicode…

2002 – MDMP, FirstUnicode R/3HP

DEC 1995 – MNLS implementation

DEC, SAP AG/America/JAPAN

Compaq1996 –MNLS EN + JA

English, Japanese, Western European

1998 – MDMP

EN + JA

Merger - ALE

2000 – MDMP

China implementation

EN + JA + ZH

Add Simplified Chinese

1995 – MNLS implementations

Various language requirements and implementations hadbeen made from earlier stage of SAP implementation forEurope, Asian countries

MNLS/MDMP, Blendedcodepage

MNLS/MDMP,Blended

codepage,Unicode

evaluation

Merger

However, SAP programdecided “Englishonly“ implementation in 1996NLS data support in Regional /Legacy systems


Introduction – HP SAP ImplementationsMDMP Currently Everywhere @ hp

MDMP 4.6CMaster dataCustomer, Vendor,Material, PricingDE GermanEN EnglishHU HungarianCS CzechTR TurkishEL GreekHE HebrewZF Chinese trad.ZH ChineseKO KoreanJA Japanese

UnicodeR/3 Enterprise

Master dataGermanEnglishSpanishPortugueseJapaneseSimplified ChineseKoreanCentral Europe

MDMP 4.7

Services30 languages

FinancialMDMP 4.7

EN EnglishPT PortugueseES SpanishZH Chinese

FinancialMDMP 4.5BEN EnglishZF Chinese trad.ZH ChineseKO KoreanJA Japanese

MDMP

MDMP

MDMP

MDMP R/3 4.6CMaster dataGermanEnglishCentral EuropeJapaneseSimplified Chinese

MDMP

MDMP 4.6COrder Mgmt.

APJEN EnglishZF Chinese trad.ZH ChineseKO KoreanJA Japanese

MDMP 4.6COrder Mgmt.

EMEADE GermanEN EnglishEN EnglishHU HungarianCS CzechTR TurkishEL GreekHE Hebrew

MDMPMDMP

3.1ICompaqGermanEnglishCentral EuropeJapaneseSimplified Chinese

MDMP

MDMP 4.7

ServicesP01, PJ1, PN130 languages

MDMP 4.7

Service30 languages

MDMP Unicode

MDMP 4.6cSoftwareBusinessEnglishCentral EuropeJapaneseSimplified Chinese

MDMP

BlendedcodepageEducationTran Siberian

Blendedcodepage


Introduction – SAP Implementations at HP

Native Language support and current usage in HP’s SAP ecosystem

Summary:16 MDMP production instances1 Unicode R/3 production instance1 Unicode BW instance (Non-production)1 Blended codepage production instance

Why Unicode…?- Multi-language data support is mandatory requirement to run business.- Full Language code ISO-639-2 & Territory+Language (e.g. British English)

SAP release reasonsNo further support of R/3 MDMP from ERP2005Several SAP systems require Unicode and need corresponding interfaceadaptation

GTS, CRM, BW, APO – Single codepage or Unicode...XI, EP, MDME - NetWeaver based solutions are Unicode


Have you seen these.. ?

Localization is necessary.Invoice, Billing, Shipping…,Legal requirements,Operation

Typical items to be localizedMaterial descriptionCustomer nameAddressVendor nameGL descriptionUnit of Measurementetc…


Have you seen these..?

European accent sign does notdisplay in correct letters.

R/3 displays the “#” sign which issupposed to be Japanese text.

Enter data in invalid logonlanguage.Error will be caused by trying toenter the text from other file by“copy and paste” method, ifcodepage is not installed.We have installed/migrated toUnicode. How native languagedata is used?

or the texts becomes “garbagecharacters”.


Fundamentals of Multi-Language data in SAP systems.

Focus on SAP systems - Multi-language Data concernsData with SAPgui, other applications, Web and Flat files.These may possibly contain multi-language text data.

Transaction via Web

Interface with otherSAP systems

Interface with externalsystems/Legacy systems

Data move by Flat files

UserUpload/Downloaddata with SAPsystems

Transactions via SAPgui



ASCIIQuestion from the last ASUG Forum:

Q: Does ASCII includes accent signs ?A: No. Accent signs are only included in ISO-Latin-x (1,2..)

ASCII only has 0-9, !”#$%&’()=~.. a-z, A-Z

The decimal character set is:

0 nul 16 dle 32 sp 48 0 64 @ 80 P 96 ` 112 p1 soh 17 dc1 33 ! 49 1 65 A 81 Q 97 a 113 q2 stx 18 dc2 34 " 50 2 66 B 82 R 98 b 114 r3 etx 19 dc3 35 # 51 3 67 C 83 S 99 c 115 s4 eot 20 dc4 36 $ 52 4 68 D 84 T 100 d 116 t5 enq 21 nak 37 % 53 5 69 E 85 U 101 e 117 u6 ack 22 syn 38 & 54 6 70 F 86 V 102 f 118 v7 bel 23 etb 39 ' 55 7 71 G 87 W 103 g 119 w8 bs 24 can 40 ( 56 8 72 H 88 X 104 h 120 x9 ht 25 em 41 ) 57 9 73 I 89 Y 105 i 121 y

10 nl 26 sub 42 * 58 : 74 J 90 Z 106 j 122 z11 vt 27 esc 43 + 59 ; 75 K 91 [ 107 k 123 {12 np 28 fs 44 , 60 < 76 L 92 \ 108 l 124 |13 cr 29 gs 45 - 61 = 77 M 93 ] 109 m 125 }14 so 30 rs 46 . 62 > 78 N 94 ^ 110 n 126 ~15 si 31 us 47 / 63 ? 79 O 95 _ 111 o 127 del



Understanding the codepage (locale) in OS and in SAP.Image of Character mapping in Windows Character map.ASCII + Latin-1, Latin-2.. SJIS, GB2312... etc.

There are more characters can be found in Character map.Above images are abstract image, Therefore mapping of the character may not beaccurate with codepage due to the usage of Windows character map application.

S-JIS Japanese (MS Kanji)

GB2312 Simplified Chinese

ISO Latin-2 Central Europe

ISO-8859-9 Latin-5 Turkish

ISO Latin-1 – Western Europe

KSC Korean (Hangle)

ISO-8859-5 Cyrillic Russian

ISO-8859-7 Greek

ASCII Text










KSC Korean (Hangle)


ISO-8859-7 Greek










KSC Korean (Hangul)


ISO-8859-7 Greek










KSC Korean (Hangle)


ISO-8859-7 Greek



Codepage on OS level

UnicodeBasic Latin Geometric Shapes Latin-1 Supplement Miscellaneous Symbols Latin

Extended-A Dingbats Latin Extended-B Miscellaneous Mathematical Symbols-AIPA Extensions Supplemental Arrows-A Spacing Modifier Letters Braille PatternsCombining Diacritical Marks Supplemental Arrows-B Greek MiscellaneousMathematical Symbols-B Cyrillic Supplemental Mathematical Operators CyrillicSupplement Miscellaneous Symbols and Arrows Armenian CJK RadicalsSupplement Hebrew Kangxi Radicals Arabic Ideographic Description CharactersSyriac CJK Symbols and Punctuation Thaana Hiragana Devanagari KatakanaBengali Bopomofo Gurmukhi Hangul Compatibility Jamo Gujarati Kanbun OriyaBopomofo Extended Tamil Katakana Phonetic Extensions Telugu Enclosed CJKLetters and Months Kannada CJK Compatibility Malayalam CJK UnifiedIdeographs Extension A Sinhala Yijing Hexagram Symbols Thai CJK UnifiedIdeographs Lao Yi Syllables Tibetan Yi Radicals Myanmar Hangul Syllables (7MB)Georgian High Surrogates Hangul Jamo Low Surrogates Ethiopic Private UseArea Cherokee CJK Compatibility Ideographs Unified Canadian Aboriginal SyllabicAlphabetic Presentation Forms Ogham Arabic Presentation Forms-A RunicVariation Selectors Tagalog Combining Half Marks Hanunoo CJK CompatibilityForms Buhid Small Form Variants Tagbanwa Arabic Presentation Forms-B KhmerHalfwidth and Fullwidth Forms Mongolian Specials Limbu Linear B Syllabary TaiLe Linear B Ideograms Khmer Symbols Aegean Numbers Phonetic Extensions OldItalic Latin Extended Additional Gothic Greek Extended Ugaritic GeneralPunctuation Desert Superscripts and Subscripts Shavian Currency SymbolsOmani Combining Marks for Symbols Cypriot Syllabary Letter like SymbolsByzantine Musical Symbols Number Forms Musical Symbols Arrows Tai XianJingo Symbols Mathematical Operators Mathematical Alphanumeric SymbolsMiscellaneous Technical CJK Unified Ideographs Extension B (13MB) ControlPictures CJK Compatibility Ideographs Supplement Optical Character RecognitionTags Enclosed Alphanumeric Variation Selectors Supplement Box DrawingSupplementary Private Use Area-A Block Elements Supplementary Private UseArea-B

Non-Unicode

Thai (TIS620)Korean (KSC5601)

Japanese (S-JIS)

Traditional chinese (big5)Simplified chinese (GB2312-80)

Hebrew (ISO 8859-8)Greek (ISO 8859-7)

Arabic (ISO 8859-6)Bulgarian, russian (ISO 8859-5)

Turkish (ISO 8859-3)

Croatian,czech,hungarian,polish,rumanian,slovak, slovene (ISO 8859-2)

Danish, dutch, english, finnish, french, german,italian, norwegian, portuguese, spanish,swedish (ISO 8859-1)



History of the Multi-Language Solution1995 2.2F MNLS (Multi-National Language System) solution.

1997-98, 3.0F- 3.1G SAP Blended codepage.

S-JISJapaneseAppServer

DBServerLatin-1 English, Western

Europe AppServer

EuroJapanAppServer

EuroJapanAppServer

DBServerDBServer


Fundamentals of Multi-Language data in SAP systems

Non-Unicode Systems = MDMPMDMP has been in use since 1997

MDMP Configuration.

CentralEurope1401

Russia1404

U.S,

Western Europe

1100

Japan8000

Korea8500

D,E,F,G,R,C,J,1,2,3…

China8400

11001100

14041404

84008400

80008000

85008500

14011401Binary data

as US-ASCII

Binary data

as US-ASCII

11001100

14041404

14011401

80008000

84008400

85008500

All requiredcodepages need tobe installed.

All requiredcodepages need tobe installed.

Taiwan8300

83008300

83008300



Unicode R/3 Enterprise – Ramp-up started in 2002Base codepage is a Unicode (UTF-16, UTF-8)

UTF-16

SAP Application Server(SAP Kernel 620)Users

UTF-8(CESU8)

DatabaseOracle 9i

SAPgui640or later



Length of bytes

1 byte – Alphabets Numeric “1-9, ABC.. +-*!@#$”

3 byte – Asian letters “ ”

Non-Unicode

Asian Double byte codepagesShift JIS JapaneseGB2312-80 Simplified ChineseBig 5 Traditional ChineseKSC5601 Korean

ISO8859-1 Western EuropeanISO8859-2 Eastern EuropeanISO8859-3 Other Latin LanguagesISO8859-4 Northern EuropeanISO8859-5 Latin/CyrillicISO8859-6 Latin/ArabicISO8859-7 Latin/GreekISO8859-8 Latin/HebrewISO8859-9 Latin/TurkishTIS620 Thai

ISO8859 +ASCII +Accent Sign

ASCII – 7 bits AlphabetsNumeric “1-9, ABC.. +-*!@#$”

Unicode

UTF-8 – Variable 8 – 24 bit (1-3 bytes)

UTF-16 – Fixed 16 bit

Any characters are in 16 bit (2 bytes:0000-ffff)

= 8 bit (1 byte)

2 byte – Accent Sign “ÄÃÇñÏÜ..”



We want to use double-byte data in….SAP R/3 & mySAP ERPBW – loading native language dataALE/EDI/IDoc – native language dataGTS – loading native language dataNetWeaver - Data communication between XI and MDMP R/3EP and MDMPCan APO handle Native language data ? MDMP/Unicode ?

…

españolfrançais


Example 1 – SAPgui setting – Setup the i18n mode –double-byte

I have to maintain payment terms in English and Japanesetext.

Installed Japanese language on my local machine throughregional and language settings on the control panel andchosen Japanese as my default window language whilelogging on to SAP.

In my SAP user profile, I have maintained JA- language codefor Japanese as my default SAP logon language.


Example 1 – SAPgui settingSetup the i18n mode


Example 1 - How to find the codepage?

There is the garbage data. Is this really a garbage? – Wear the“codepage“ glasses.

On SAP screenData extract - Flat file

S-JIS

GB2312

Latin-2

Latin-5

Unicode


Example 1 – How to find the codepage?

There is the garbage data. Is this really a garbage?On SAP screenMDMP has auto detect – “Character set”Unicode



There is the garbage data. Is thisreally a garbage?

Data extract - Flat file – IDoc, fileextract (SE16), Batch input file

Quickest way – save as “***.html”then view by Internet Browser.

GB2312

Latin-2 Latin-5

“Latin-1”



There is the garbage data. Is thisreally a garbage?

Data extract - Flat file – IDoc, fileextract (SE16), Batch input file

Quickest way – save as “***.html”then view by Internet Browser.

GB2312

Latin-2 Latin-5

S-JIS

“Encoding”

“S-JIS”



Use the character map tool in Windows.Start -> Run

-> charmap.exe

Find the codepageUnicode character mapping


Example 2 – Data Input Method

How to enter the native language letters ?Use “IME” – Input Method Editor in Microsoft Windows

Type “Desktop”


Example 2 – Data Input Method

Example - “IME” – Input Method in Microsoft WindowsJapanese IME

“desukutoppu”

Switch to IME - “Alt” + “~”

“Enter” to display Candidates tochoose the correct word.


HP DESKTOP PC 6000EN

HP 6000ZH

HP 6000KO

HP ordinateur de bureau 6000FR

HP 6000EL

HP sobremesa 6000ES

HP Masaüstü bilgisayarlar 6000TR

HP 6000RU

HP 6000ZF

HP 6000JA

Example 3 – Data Input

“Cut and Paste/Copy and Paste” works !?Copy the text from spread sheet, text tile etc, paste onto SAP.

(Maintain Forms, Master data, GL etc.) Ctrl-c, Ctrl-v, Ctrl-c, Ctrl-v…

It works, but Be careful!Know the codepage of data.Know your system…

MDMPUnicodeIf its just Latin-1 – Don’t cheat the system

Copy and Paste


Example 4 – Enabling multi-language capability

Regional Setting in OS (Microsoft Windows)


Example 5 – Address Version with Native language

Use of CAM – Central addressManagement = Internationaladdress version

KNA1/LFA1 TableCustomer / Vender Master

key - Address number

ADRC TableAddress Management

Unicode R/3


Example 6 – Flat file interface batch input

The Flat file is generated from Microsoft Excel spread sheet whichextracted as GB2312 encoding at Non-Unicode R/3 system.

MDMP

Save as .txt




Save as Unicode (.txt)

Unicode


Example 6 – Flat file interface - batch input


Unicode

After R/3 system becameUnicode, How we canhandle this?

Save as Unicode




At Unicode system, the Flat file must be Unicode encoding when it’sextracted from Microsoft Excel Spread sheet.

In order to use the program in Unicode R/3 Enterprise system, theABAP/4 source code has to be reviewed and qualified as “Unicodeenabled” ABAP code.

The ws_upload and ws_download function modules are obsoleteand result in errors on Unicode systems or on systems that maypotentially be Unicode enabled (R/3 Basis Release 6.20) -gui_upload / gui_download.


Example 7 – MDMP Character detect

Error Example: If Japanese encoded data were entered into Latin-1system, System will detect the data with error.

The input field containsprohibited characters

(Already deleted)


Example 8 – XML

R/3 Enterprise:

<?xml version="1.0" ?>

ISO Latin character is notsupported ?

Only supports ASCII Character ?

Or Unicode.


Example 8 – XML

Vendor master. XML File format.

<?xml version="1.0" encoding="big5" ?>- <ZEXTCRE1>- <IDOC BEGIN="1">- <EDI_DC40 SEGMENT="1">- <TABNAM> <![CDATA[ EDI_DC40 ]]>

</TABNAM><MANDT>445</MANDT><DOCNUM>000000012345670</DOCNUM><DOCREL>46C</DOCREL><STATUS>30</STATUS><DIRECT>1</DIRECT>

:<KTOKK>TWAC</KTOKK><LAND1>TW</LAND1><NAME1>HP CORP.</NAME1><NAME3> </NAME3><ORT01>TAIPEI</ORT01><SORTL>HP </SORTL><SPRAS>M</SPRAS>

- <STRAS> <![CDATA[ 106 9]]>

</STRAS>

The XML page cannot be displayed Cannot view XMLinput using XSL style sheet. Please correct the error andthen click the Refresh button, or try again later. Aninvalid character was found in text content. Errorprocessing resource 'file:///C:/Documents andSettings/yamamoto/Local Settings/TemporaryInternet Files/OLK79/old.xml'. Line 1, Position 810<?xml version ="1.0"?><ZEXTCRE1><IDOC BEGIN="1"><EDI_DC40SEGMENT="1"><TABNAM><![CDATA[EDI_DC40]]></TABNAM><MANDT>445</MANDT><DOCNUM>000000012345670</DOCNUM><DOCREL>46C</DOCREL><STATUS>30</STATUS><DIRECT>1</DIRECT>

encoding tag !

http://firepn01.boise.itc.hp.com:8080/sap/bc/bsp/sap/zhplic_rel_mgmt/details.htm?mode=D&COMPANYID=HP&CUSTOMERID=30184&INSTALLATION=0810001501&SYSID=US1

http://firepn01.boise.itc.hp.com:8080/sap/bc/bsp/sap/zhplic_rel_mgmt/details.htm?mode=D&COMPANYID=HP&CUSTOMERID=0000027334&INSTALLATION=1710000022&SYSID=HPS

http://firepn01.boise.itc.hp.com:8080/sap/bc/bsp/sap/zhplic_rel_mgmt/details.htm?mode=D&COMPANYID=HP&CUSTOMERID=30184&INSTALLATION=0020109862&SYSID=LH1

http://firepn01.boise.itc.hp.com:8080/sap/bc/bsp/sap/zhplic_rel_mgmt/details.htm?mode=D&COMPANYID=HP&CUSTOMERID=30184&INSTALLATION=0020074828&SYSID=BW1

http://firepn01.boise.itc.hp.com:8080/sap/bc/bsp/sap/zhplic_rel_mgmt/details.htm?mode=D&COMPANYID=HP&CUSTOMERID=32207&INSTALLATION=0810000487&SYSID=EXE


Example 8 – XML

in SAP - ISO encoding setting

<?xml version="1.0" encoding=“utf-8"?>

<?xml version="1.0" ?><?xml version="1.0" encoding=“utf-8" ?>

- <ZEXTCRE1>- <IDOC BEGIN="1">- <EDI_DC40 SEGMENT="1">- <TABNAM> <![CDATA[ EDI_DC40 ]]>

</TABNAM><MANDT>445</MANDT><DOCNUM>000000012345670</DOCNUM><DOCREL>46C</DOCREL><STATUS>30</STATUS><DIRECT>1</DIRECT>

:<KTOKK>TWAC</KTOKK><LAND1>TW</LAND1><NAME1>HP CORP.</NAME1><NAME3> </NAME3><ORT01>TAIPEI</ORT01><SORTL>HP </SORTL><SPRAS>M</SPRAS>

- <STRAS> <![CDATA[ 106 9 ]]></STRAS>

http://firepn01.boise.itc.hp.com:8080/sap/bc/bsp/sap/zhplic_rel_mgmt/details.htm?mode=D&COMPANYID=HP&CUSTOMERID=30184&INSTALLATION=0810001487&SYSID=R00

http://firepn01.boise.itc.hp.com:8080/sap/bc/bsp/sap/zhplic_rel_mgmt/details.htm?mode=D&COMPANYID=HP&CUSTOMERID=34278&INSTALLATION=0020082001&SYSID=PRD

http://firepn01.boise.itc.hp.com:8080/sap/bc/bsp/sap/zhplic_rel_mgmt/details.htm?mode=D&COMPANYID=HP&CUSTOMERID=30184&INSTALLATION=0020106944&SYSID=A50

http://firepn01.boise.itc.hp.com:8080/sap/bc/bsp/sap/zhplic_rel_mgmt/details.htm?mode=D&COMPANYID=HP&CUSTOMERID=28166&INSTALLATION=0110003261&SYSID=R01

http://firepn01.boise.itc.hp.com:8080/sap/bc/bsp/sap/zhplic_rel_mgmt/details.htm?mode=D&COMPANYID=HP&CUSTOMERID=32207&INSTALLATION=0820021498&SYSID=SPP


Example 8 – XML

Another option/request for XML portin < 4.6C.

Make the “Unicode” encoding optionon < 4.6C as well

<?xml version="1.0"encoding=“UTF-8"?>

XML with UTF-8 encoding is widelyused and current standard in theworld.

Enable ISO encoding tag in XMLMake “Selection” Option from WE21transaction.Use default logon language.

By codepage from RFC logonlanguage.

Unicode encoding settingCurrently, UTF-8 has already in 4.7systemAdd the UTF-8 choice of “XMLFormat” in 4.6C systems for morecompatibility. (Optional)


Example 9 – Unicode display

Unicode System – User can operate any language key in the Englishlogon.


Example 10 - Unicode & Non-Unicode RFC

Be aware of the interface Unicode and Non-Unicode (MDMP)

ALEData conversion for Unicode -> Non-Unicode

Other interfaces like file upload - batch input, XI via adaptors

RFCLogon

ENLatin-1



ALE/IDoc issue with MDMP systemsfrom MDMP (Non-Unicode) to UnicodeTest result in detail - data

Unicode - Logon as EN (English)Material ABC0123-ZZEN HP Desktop 2004ZH HP ×ÀÃæ¼ÆËã»ú 2004JA HP #f#X#N#g#b#v#@2004

MDMP - Logon as JA (Japanese)Material ABC0123-ZZJA HP 2004ZH HP 2004EN HP Desktop 2004

MDMP - Logon as ZH (Simplified Chinese)Material ABC0123-ZZZH HP 2004EN HP Desktop 2004JA HP 2004

MDMP - Logon as EN (English)Material ABC0123-ZZEN HP Desktop 2004ZH HP ×ÀÃæ¼ÆËã»ú 2004JA HP ƒfƒXƒNƒgƒbƒv @2004

In the MDMP system, native languagedata can only be displayed when its thelogon language matched.

In the Unicode system, any nativelanguage data can be displayed in thesame menu. However the data fromMDMP was corrupted.



ALE/IDoc issue with MDMP systemsfrom MDMP (Non-Unicode) to UnicodeTest result in detail - data

Unicode - Logon as EN (English)Material ABC0123-ZZEN HP Desktop 2004ZH HP ×ÀÃæ¼ÆËã»ú 2004JA HP #f#X#N#g#b#v#@2004

MDMP - Logon as JA (Japanese)Material ABC0123-ZZJA HP 2004ZH HP 2004EN HP Desktop 2004

MDMP - Logon as ZH (Simplified Chinese)Material ABC0123-ZZZH HP 2004EN HP Desktop 2004JA HP 2004

MDMP - Logon as EN (English)Material ABC0123-ZZEN HP Desktop 2004ZH HP ×ÀÃæ¼ÆËã»ú 2004JA HP ƒfƒXƒNƒgƒbƒv @2004

In the MDMP system, native languagedata can only be displayed when its thelogon language matched.

Unicode - Logon as EN (English)Material ABC0123-ZZEN HP Desktop 2004ZH HP 2004JA HP 2004

There are solutions for RFC - ALE/IDoc.OSS: 745030, 656350

To avoid the data corruption, the IDochas to be converted between theUnicode system and the MDMP.


Example 11 – Loading invalid codepage data

GTS: Loading "garbage?! - double byte” S-JIS data into Non-Unocode"Latin-1" GTS, Is this ok?

No MDMP way for Non-R/3 systems…GTS is Single codepage – Latin-1Source: R/3 MDMP Latin-1+S-JIS+.. - This is definitely not ok.Check the language requirements – How many languages does thesystem need.Unicode conversion will be the solution – how to deal with existingdata?


Example 12 – Data Conversion tips

Data conversion makes data compatible with codepages usedinside/outside of SAP system.

e.g. One of data conversion examples in Unix.

iconv(1)

NAME

iconv - Converts encoded characters to another code set

SYNOPSIS

iconv -f from_code -t to_code [file...]

DESCRIPTION

The iconv command converts the encoding of characters in file from one

coded character set to another and writes the results to standard output.

1. To convert the contents of the eucJP_data file from the eucJP to the

SJIS code set and save the results in the sjis_data file, enter:

iconv -f eucJP -t SJIS eucJP_data > sjis_data

EUC -> S-JIS

Roman-8 -> Latin-1

CCDC -> Big5

S-JIS > UTF-8

GB2312 –> UTF8

Latin2 – UTF8

:

How will new “SAPiconv” work?OSS 752859: sapiconv - a tool for converting the encoding of files.


Unicode printing challengespool device, device type

Customer data in

Unicode Output

Device type:

HPUTF8 PCL5 UTF-8

Unicode enabledprinter.

Unicode Font

Hewlett-Packard Company3000 Hanover StreetPalo Alto, CA 94304-1185 USA

140-8641 2-2-24

HP23-6 HP

99 38

112

, 115054, ,

Hewlett-Packard GmbHHerrenberger Straße 14071034 Böblingen

Hewlett-Packard Company3000 Hanover StreetPalo Alto, CA 94304-1185 USA

140-8641 2-2-24

HP23-6 HP

99 38

112

, 115054, ,

Hewlett-Packard GmbHHerrenberger Straße 14071034 BöblingenDevice types:

HPLJ4JPHPLJ4CNHPLJ4SAPWINJPSAPWINCNSAPWIN

HP Output Management

Using HP International printing solution and HP Output Management

Example 13 – Printing

SAP System

Generate Unicodebased spooloutput


Summary

Know, and Support localized data more than Alphabets.ABC…XYZ, 0123…789, !”#$%&

We have installed/migrated to Unicode. Now, how we can use nativelanguage data ? How we can verify?

Improve the consistency of Globalization strategies across yourSAP ecosystem.

Develop effective and reusable model of framework for analysis, design &implementation of projects to maximize multi-lingual capabilitiesEvaluate and recommend architectural standards for internationalized SAPimplementations (e.g. Unicode)Disseminator guidelines for localizable usability relating to the multi-language solutions (e.g. display, printing etc.)Work with the local teams, and across the organizations to gatheropinion/requirement from usability to manageability of internationalizationfacts and to exchange the case studies and facts experiences for the topicsof Multi-language, Localization, Internationalization.


Summary

Globalization program - Dealing with Multi-language data

Goal: Avoid any solution inconsistencies for localization amongSAP project(s).

The NLS solution across the SAP projects should bestandardized to minimize the data conversion and tominimize the interface to develop. The local requirementshould be respected and apply the realistic solution.

Each projects may have different NLSsolutions by own way.

Workaround B

We have beendoing this way…

Workaround A

Here is the solution…

Workaround C

The consultantsaid..

Workaround D

Just enter it…


Summary

Current Tasks on Unicode based solutionsNLS support and SAP Technical issue.

Practice of Unicode R/3, BW/SEM, installation/upgradeUnicode R/3

R/3 Enterprise for master data.Tacking Interface issues

Unicode BW/SEMConnection with MDMP R/3

XI (NetWeaver) – MDMPWAS620/640 & Unicode/Non-Unicode

NLS support for Interface XML/ALE/EDIUnicode, Interface with MDMP, External system, Flat file

Database technology.Sizing, Data format (Codepage)

PrintingDevelopment with NLS data – Smartform/SAPscript/Webbased reporting.Unicode & Non-UnicodeSystem - Printer/Device type

Localization for user InterfaceNLS usage for ABAP/ICM/Java based processing/Web Dympro.


Summary – Unicode and Non-Unicode

Conversions in transition termUnicode R/3MDMP R/3Single codepage R/3Other Applications

UnicodeR/3

R/3 +WAS

R/3

R/3 MDMPR/3

R/3

ALE/IDoc withConversion

MATMASGLMASTADRMASDEBMAS

:

ALE/IDoc withConversion

MATMASGLMASTADRMASDEBMAS

:

MDMP

Non-SAPApplications

Latin-1Latin-2S-JISBig5GB2312

Non-SAPApplications

mySAP…NetWeaver

File transfer

MiddlewareFile share

3rd vendor toolEDI

:

Flat file:

Flat file:

Conversioniconv etc..

Conversionvia RFCOSS: 745030

OSS: 656350

WAS xml-over-h

ttp

MDMP

MDMP

Unicode


Summary

Learning from projects

Unicode will solve the NLS problems. But… Always consider thelanguage requirements and prepare for any NLS issues around thesystem. Know how your Native language data is used today.

Be prepared - for native language requirement.Check the language requirements in each project. If the project is called“Global” or “Worldwide” SAP implementation. Check the concrete planfor localization.

“We only need English for the initial implementation.”Some data may contain other special character?Can we add later ? when the language requirementcame up with?

Always consider the language requirements and the futureexpansion.


Summary

Research activitiesInternal IT

Internal SAP Forum.Globalization Focus group – virtual community across the groups

ASUG ConferencesWeb cast – “Multi-language/Unicode out reach call” - System Management +Global Implementation SIGAnnual Conference, ForumGlobal Implementations SIG

SAPSAP Marketplace globalization, unicode@sap, OSS notesSDN

Garbage data is not really the garbage,know how to deal with :-)


Further Information

Public Web:www.sap.comwww.sdn.sap.comSAP NetWeaver Developer‘s Guide:www.sdn.sap.com/sdn/developersguide.sdn

Americas’ SAP Users’ Group (ASUG)www.asug.com

Global Implementations SIG page – discussion forum

http://www.sap.com/

http://www.sdn.sap.com/

http://www.sdn.sap.com/sdn/developersguide.sdn

http://www.asug.com/


Q&A

Questions?


Please complete your session evaluation.

Be courteous — deposit your trash,and do not take the handouts for the following session.

Feedback

Thank You !


No part of this publication may be reproduced or transmitted in any form or for any purpose without the express permission of SAP AG. The informationcontained herein may be changed without prior notice.Some software products marketed by SAP AG and its distributors contain proprietary software components of other software vendors.Microsoft, Windows, Outlook, and PowerPoint are registered trademarks of Microsoft Corporation.IBM, DB2, DB2 Universal Database, OS/2, Parallel Sysplex, MVS/ESA, AIX, S/390, AS/400, OS/390, OS/400, iSeries, pSeries, xSeries, zSeries, z/OS, AFP,Intelligent Miner, WebSphere, Netfinity, Tivoli, and Informix are trademarks or registered trademarks of IBM Corporation in the United States and/or othercountries.Oracle is a registered trademark of Oracle Corporation.UNIX, X/Open, OSF/1, and Motif are registered trademarks of the Open Group.Citrix, ICA, Program Neighborhood, MetaFrame, WinFrame, VideoFrame, and MultiWin are trademarks or registered trademarks of Citrix Systems, Inc.HTML, XML, XHTML and W3C are trademarks or registered trademarks of W3C®, World Wide Web Consortium, Massachusetts Institute of Technology.Java is a registered trademark of Sun Microsystems, Inc.JavaScript is a registered trademark of Sun Microsystems, Inc., used under license for technology invented and implemented by Netscape.MaxDB is a trademark of MySQL AB, Sweden.SAP, R/3, mySAP, mySAP.com, xApps, xApp, SAP NetWeaver and other SAP products and services mentioned herein as well as their respective logos aretrademarks or registered trademarks of SAP AG in Germany and in several other countries all over the world. All other product and service names mentionedare the trademarks of their respective companies. Data contained in this document serves informational purposes only. National product specificationsmay vary.

The information in this document is proprietary to SAP. No part of this document may be reproduced, copied, or transmitted in any form or for any purposewithout the express prior written permission of SAP AG.This document is a preliminary version and not subject to your license agreement or any other agreement with SAP. This document contains only intendedstrategies, developments, and functionalities of the SAP® product and is not intended to be binding upon SAP to any particular course of business, productstrategy, and/or development. Please note that this document is subject to change and may be changed by SAP at any time without notice.SAP assumes no responsibility for errors or omissions in this document. SAP does not warrant the accuracy or completeness of the information, text, graphics,links, or other items contained within this material. This document is provided without a warranty of any kind, either express or implied, including but not limitedto the implied warranties of merchantability, fitness for a particular purpose, or non-infringement.SAP shall have no liability for damages of any kind including without limitation direct, special, indirect, or consequential damages that may result from the useof these materials. This limitation shall not apply in cases of intent or gross negligence.The statutory liability for personal injury and defective products is not affected. SAP has no control over the information that you may access through the useof hot links contained in these materials and does not endorse your use of third-party Web pages nor provide any warranty whatsoever relating to third-partyWeb pages.

Copyright 2005 SAP AG. All Rights Reserved